2026-02-24T21:50:23.785 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-24T21:50:23.801 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-24T21:50:23.882 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/suriarte-2026-02-24_21:43:07-rgw-wip-d4n-remove-bucket-distro-default-trial/68775 branch: wip-d4n-remove-bucket description: rgw/verify/{0-install accounts$/{none} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off-no-gc} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{admin-pagination bucket-check cls mp_reupload rados-pool-quota ragweed reshard s3tests versioning zzz-s3tests-java} validater/lockdep} email: null first_in_suite: false flavor: default job_id: '68775' kernel: branch: distro kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: trial name: suriarte-2026-02-24_21:43:07-rgw-wip-d4n-remove-bucket-distro-default-trial no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-d4n-remove-bucket ceph: conf: client: debug rgw: 20 debug rgw notification: 20 lockdep: true rgw bucket counters cache: true rgw crypt require ssl: false rgw crypt s3 kms backend: testing rgw crypt s3 kms encryption keys: testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= rgw d3n l1 datacache persistent path: /tmp/rgw_datacache/ rgw d3n l1 datacache size: 10737418240 rgw d3n l1 local datacache enabled: true rgw enable ops log: true rgw max chunk size: 4194304 rgw obj stripe size: 6291456 rgw reshard progress judge interval: 10 rgw s3 auth use sts: true rgw sts key: abcdefghijklmnop rgw torrent flag: true rgw user counters cache: true setgroup: ceph setuser: ceph global: mon client directed command retry: 5 ms inject socket failures: 5000 osd_max_pg_log_entries: 10 osd_min_pg_log_entries: 10 rgw override bucket index max shards: 1 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 lockdep: true osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug objclass: 20 debug osd: 20 debug rocksdb: 4/10 lockdep: true mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true rgw gc max queue size: 0 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) - \(POOL_APP_NOT_ENABLED\) - not have an application enabled - \(OSD_SLOW_PING_TIME - reached quota - POOL_FULL - pool\(s\) full sha1: f7e05f4fdea5af173284f247d6b0935b337d4ccf ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: f7e05f4fdea5af173284f247d6b0935b337d4ccf openssl_keys: rgw.client.0: ca: root client: client.0 embed-key: true root: client: client.0 cn: teuthology install: - client.0 key-type: rsa:4096 rgw: client.0: ssl certificate: rgw.client.0 compression type: random datacache: true datacache_path: /tmp/rgw_datacache ec-data-pool: false frontend: beast inline data: false storage classes: FROZEN: null LUKEWARM: null s3tests: accounts: iam alt root: RGW99999999999999999 iam root: RGW88888888888888888 force-branch: ceph-master storage classes: LUKEWARM, FROZEN thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-d4n-remove-bucket sha1: f7e05f4fdea5af173284f247d6b0935b337d4ccf owner: scheduled_suriarte@soko04.front.sepia.ceph.com priority: 150 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 6673 sha1: f7e05f4fdea5af173284f247d6b0935b337d4ccf sleep_before_teardown: 0 suite: rgw suite_branch: wip-d4n-remove-bucket suite_path: /home/teuthworker/src/github.com_samarahu_ceph_f7e05f4fdea5af173284f247d6b0935b337d4ccf/qa suite_relpath: qa suite_repo: https://github.com/samarahu/ceph.git suite_sha1: f7e05f4fdea5af173284f247d6b0935b337d4ccf targets: trial046.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJqPa3MRsJyiJog6c04vkzfx6nDOxInfdr4kGc1hiUs5cYBqZmzxwozxHjenOYhm3tYqHP0qtPrf7+XW4QFQKmM= trial110.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBa0UOTuK+rY9wKdUikGx94zfBwP2Fv89fu8JG00sL0xR45EN+Jqu0bbptxVqaHVwtxeypYiQFhOiEzBL7Cc6sU= tasks: - install: extra_system_packages: deb: - s3cmd - maven - golang rpm: - s3cmd - maven - golang - ceph: null - openssl_keys: null - rgw: client.0: null - tox: - client.0 - workunit: clients: client.0: - rgw/run-datacache.sh env: RGW_DATACACHE_PATH: /tmp/rgw_datacache - workunit: clients: client.0: - rgw/run-admin-pagination.sh - workunit: clients: client.0: - rgw/run-bucket-check.sh - workunit: clients: client.0: - cls/test_cls_lock.sh - cls/test_cls_log.sh - cls/test_cls_refcount.sh - cls/test_cls_rgw.sh - cls/test_cls_rgw_gc.sh - cls/test_cls_rgw_stats.sh - cls/test_cls_cmpomap.sh - cls/test_cls_2pc_queue.sh - cls/test_cls_user.sh - cls/test_cls_sem_set.sh - rgw/test_rgw_gc_log.sh - rgw/test_rgw_obj.sh - rgw/test_rgw_datalog.sh - rgw/test_librgw_file.sh - rgw/test_awssdkv4_sig.sh - rgw/test_gosdk2.sh - workunit: clients: client.0: - rgw/test_rgw_s3_mp_reupload.sh - workunit: clients: client.0: - rgw/run-rados-pool-quota.sh - ragweed: client.0: default-branch: ceph-master rgw_server: client.0 stages: prepare,check - workunit: clients: client.0: - rgw/run-reshard.sh - s3tests: client.0: rgw_server: client.0 - workunit: clients: client.0: - rgw/run-versioning.sh - s3tests-java: client.0: force-branch: ceph-master force-repo: https://github.com/ceph/java_s3tests.git teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-24_21:43:07 tube: trial user: suriarte verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.904699 2026-02-24T21:50:23.882 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_samarahu_ceph_f7e05f4fdea5af173284f247d6b0935b337d4ccf/qa; will attempt to use it 2026-02-24T21:50:23.882 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_samarahu_ceph_f7e05f4fdea5af173284f247d6b0935b337d4ccf/qa/tasks 2026-02-24T21:50:23.882 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-24T21:50:23.884 INFO:teuthology.task.internal:Checking packages... 2026-02-24T21:50:23.884 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'f7e05f4fdea5af173284f247d6b0935b337d4ccf' 2026-02-24T21:50:23.884 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-24T21:50:23.884 INFO:teuthology.packaging:ref: None 2026-02-24T21:50:23.884 INFO:teuthology.packaging:tag: None 2026-02-24T21:50:23.884 INFO:teuthology.packaging:branch: wip-d4n-remove-bucket 2026-02-24T21:50:23.884 INFO:teuthology.packaging:sha1: f7e05f4fdea5af173284f247d6b0935b337d4ccf 2026-02-24T21:50:23.884 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-d4n-remove-bucket 2026-02-24T21:50:24.002 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-4957-gf7e05f4f-1jammy 2026-02-24T21:50:24.003 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-24T21:50:24.015 INFO:teuthology.task.internal:no buildpackages task found 2026-02-24T21:50:24.015 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-24T21:50:24.033 INFO:teuthology.task.internal:Saving configuration 2026-02-24T21:50:24.051 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-24T21:50:24.063 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-24T21:50:24.140 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial046.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/suriarte-2026-02-24_21:43:07-rgw-wip-d4n-remove-bucket-distro-default-trial/68775', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-24 21:45:21.611866', 'locked_by': 'scheduled_suriarte@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDj6XO2dridVgMCFkDgVzQ4LqQ16iPzLsYxpMQSc512/'} 2026-02-24T21:50:24.196 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial110.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/suriarte-2026-02-24_21:43:07-rgw-wip-d4n-remove-bucket-distro-default-trial/68775', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-24 21:45:21.612463', 'locked_by': 'scheduled_suriarte@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKSsMjk17gaepXfhlIm7UqRZbCUkLECTv0LFVwstu/f7'} 2026-02-24T21:50:24.196 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-24T21:50:24.206 INFO:teuthology.task.internal:roles: ubuntu@trial046.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2026-02-24T21:50:24.206 INFO:teuthology.task.internal:roles: ubuntu@trial110.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2026-02-24T21:50:24.206 INFO:teuthology.run_tasks:Running task console_log... 2026-02-24T21:50:24.334 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x784308547d80>, signals=[15]) 2026-02-24T21:50:24.334 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-24T21:50:24.350 INFO:teuthology.task.internal:Opening connections... 2026-02-24T21:50:24.350 DEBUG:teuthology.task.internal:connecting to ubuntu@trial046.front.sepia.ceph.com 2026-02-24T21:50:24.351 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T21:50:24.414 DEBUG:teuthology.task.internal:connecting to ubuntu@trial110.front.sepia.ceph.com 2026-02-24T21:50:24.415 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T21:50:24.478 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-24T21:50:24.493 DEBUG:teuthology.orchestra.run.trial046:> uname -m 2026-02-24T21:50:24.496 INFO:teuthology.orchestra.run.trial046.stdout:x86_64 2026-02-24T21:50:24.497 DEBUG:teuthology.orchestra.run.trial046:> cat /etc/os-release 2026-02-24T21:50:24.543 INFO:teuthology.orchestra.run.trial046.stdout:PRETTY_NAME="Ubuntu 22.04.5 LTS" 2026-02-24T21:50:24.543 INFO:teuthology.orchestra.run.trial046.stdout:NAME="Ubuntu" 2026-02-24T21:50:24.543 INFO:teuthology.orchestra.run.trial046.stdout:VERSION_ID="22.04" 2026-02-24T21:50:24.543 INFO:teuthology.orchestra.run.trial046.stdout:VERSION="22.04.5 LTS (Jammy Jellyfish)" 2026-02-24T21:50:24.543 INFO:teuthology.orchestra.run.trial046.stdout:VERSION_CODENAME=jammy 2026-02-24T21:50:24.543 INFO:teuthology.orchestra.run.trial046.stdout:ID=ubuntu 2026-02-24T21:50:24.543 INFO:teuthology.orchestra.run.trial046.stdout:ID_LIKE=debian 2026-02-24T21:50:24.543 INFO:teuthology.orchestra.run.trial046.stdout:HOME_URL="https://www.ubuntu.com/" 2026-02-24T21:50:24.543 INFO:teuthology.orchestra.run.trial046.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2026-02-24T21:50:24.543 INFO:teuthology.orchestra.run.trial046.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2026-02-24T21:50:24.543 INFO:teuthology.orchestra.run.trial046.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2026-02-24T21:50:24.543 INFO:teuthology.orchestra.run.trial046.stdout:UBUNTU_CODENAME=jammy 2026-02-24T21:50:24.544 INFO:teuthology.lock.ops:Updating trial046.front.sepia.ceph.com on lock server 2026-02-24T21:50:24.614 DEBUG:teuthology.orchestra.run.trial110:> uname -m 2026-02-24T21:50:24.618 INFO:teuthology.orchestra.run.trial110.stdout:x86_64 2026-02-24T21:50:24.618 DEBUG:teuthology.orchestra.run.trial110:> cat /etc/os-release 2026-02-24T21:50:24.662 INFO:teuthology.orchestra.run.trial110.stdout:PRETTY_NAME="Ubuntu 22.04.5 LTS" 2026-02-24T21:50:24.662 INFO:teuthology.orchestra.run.trial110.stdout:NAME="Ubuntu" 2026-02-24T21:50:24.662 INFO:teuthology.orchestra.run.trial110.stdout:VERSION_ID="22.04" 2026-02-24T21:50:24.663 INFO:teuthology.orchestra.run.trial110.stdout:VERSION="22.04.5 LTS (Jammy Jellyfish)" 2026-02-24T21:50:24.663 INFO:teuthology.orchestra.run.trial110.stdout:VERSION_CODENAME=jammy 2026-02-24T21:50:24.663 INFO:teuthology.orchestra.run.trial110.stdout:ID=ubuntu 2026-02-24T21:50:24.663 INFO:teuthology.orchestra.run.trial110.stdout:ID_LIKE=debian 2026-02-24T21:50:24.663 INFO:teuthology.orchestra.run.trial110.stdout:HOME_URL="https://www.ubuntu.com/" 2026-02-24T21:50:24.663 INFO:teuthology.orchestra.run.trial110.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2026-02-24T21:50:24.663 INFO:teuthology.orchestra.run.trial110.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2026-02-24T21:50:24.663 INFO:teuthology.orchestra.run.trial110.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2026-02-24T21:50:24.663 INFO:teuthology.orchestra.run.trial110.stdout:UBUNTU_CODENAME=jammy 2026-02-24T21:50:24.664 INFO:teuthology.lock.ops:Updating trial110.front.sepia.ceph.com on lock server 2026-02-24T21:50:24.739 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-24T21:50:24.751 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-24T21:50:24.763 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-24T21:50:24.763 DEBUG:teuthology.orchestra.run.trial046:> test '!' -e /home/ubuntu/cephtest 2026-02-24T21:50:24.764 DEBUG:teuthology.orchestra.run.trial110:> test '!' -e /home/ubuntu/cephtest 2026-02-24T21:50:24.768 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-24T21:50:24.777 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-24T21:50:24.777 DEBUG:teuthology.orchestra.run.trial046:> test -z $(ls -A /var/lib/ceph) 2026-02-24T21:50:24.809 DEBUG:teuthology.orchestra.run.trial110:> test -z $(ls -A /var/lib/ceph) 2026-02-24T21:50:24.812 INFO:teuthology.orchestra.run.trial046.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2026-02-24T21:50:24.818 INFO:teuthology.orchestra.run.trial110.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2026-02-24T21:50:24.819 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-24T21:50:24.977 INFO:teuthology.run_tasks:Running task kernel... 2026-02-24T21:50:25.004 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-24T21:50:25.005 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'mon.b': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-24T21:50:25.005 DEBUG:teuthology.orchestra.run.trial046:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-24T21:50:25.005 DEBUG:teuthology.orchestra.run.trial110:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-24T21:50:25.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T21:50:25.008 DEBUG:teuthology.orchestra.run.trial046:> uname -r 2026-02-24T21:50:25.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T21:50:25.008 DEBUG:teuthology.orchestra.run.trial110:> uname -r 2026-02-24T21:50:25.050 INFO:teuthology.orchestra.run.trial046.stdout:5.15.0-170-generic 2026-02-24T21:50:25.051 INFO:teuthology.task.kernel:Running kernel on trial046: 5.15.0-170-generic 2026-02-24T21:50:25.051 DEBUG:teuthology.orchestra.run.trial046:> sudo apt-get clean 2026-02-24T21:50:25.053 INFO:teuthology.orchestra.run.trial110.stdout:5.15.0-170-generic 2026-02-24T21:50:25.053 INFO:teuthology.task.kernel:Running kernel on trial110: 5.15.0-170-generic 2026-02-24T21:50:25.053 DEBUG:teuthology.orchestra.run.trial110:> sudo apt-get clean 2026-02-24T21:50:25.108 DEBUG:teuthology.orchestra.run.trial046:> sudo apt-get update 2026-02-24T21:50:25.115 DEBUG:teuthology.orchestra.run.trial110:> sudo apt-get update 2026-02-24T21:50:25.551 INFO:teuthology.orchestra.run.trial110.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-24T21:50:25.603 INFO:teuthology.orchestra.run.trial046.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-24T21:50:25.624 INFO:teuthology.orchestra.run.trial110.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2026-02-24T21:50:25.693 INFO:teuthology.orchestra.run.trial046.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2026-02-24T21:50:25.902 INFO:teuthology.orchestra.run.trial110.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2026-02-24T21:50:26.023 INFO:teuthology.orchestra.run.trial110.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2026-02-24T21:50:26.029 INFO:teuthology.orchestra.run.trial046.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2026-02-24T21:50:26.138 INFO:teuthology.orchestra.run.trial110.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [3269 kB] 2026-02-24T21:50:26.177 INFO:teuthology.orchestra.run.trial046.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2026-02-24T21:50:26.316 INFO:teuthology.orchestra.run.trial046.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [3269 kB] 2026-02-24T21:50:26.467 INFO:teuthology.orchestra.run.trial110.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [497 kB] 2026-02-24T21:50:26.477 INFO:teuthology.orchestra.run.trial110.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [19.2 kB] 2026-02-24T21:50:26.477 INFO:teuthology.orchestra.run.trial110.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1256 kB] 2026-02-24T21:50:26.505 INFO:teuthology.orchestra.run.trial110.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [314 kB] 2026-02-24T21:50:26.511 INFO:teuthology.orchestra.run.trial110.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [30.4 kB] 2026-02-24T21:50:26.512 INFO:teuthology.orchestra.run.trial110.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [5379 kB] 2026-02-24T21:50:26.642 INFO:teuthology.orchestra.run.trial110.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [1022 kB] 2026-02-24T21:50:26.661 INFO:teuthology.orchestra.run.trial110.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 Packages [3007 kB] 2026-02-24T21:50:26.716 INFO:teuthology.orchestra.run.trial046.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [497 kB] 2026-02-24T21:50:26.728 INFO:teuthology.orchestra.run.trial046.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [19.2 kB] 2026-02-24T21:50:26.729 INFO:teuthology.orchestra.run.trial046.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1256 kB] 2026-02-24T21:50:26.732 INFO:teuthology.orchestra.run.trial110.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy-security/main Translation-en [430 kB] 2026-02-24T21:50:26.739 INFO:teuthology.orchestra.run.trial110.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [14.1 kB] 2026-02-24T21:50:26.739 INFO:teuthology.orchestra.run.trial110.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1018 kB] 2026-02-24T21:50:26.763 INFO:teuthology.orchestra.run.trial046.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [314 kB] 2026-02-24T21:50:26.769 INFO:teuthology.orchestra.run.trial110.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy-security/universe Translation-en [225 kB] 2026-02-24T21:50:26.770 INFO:teuthology.orchestra.run.trial046.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [30.4 kB] 2026-02-24T21:50:26.771 INFO:teuthology.orchestra.run.trial046.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [5379 kB] 2026-02-24T21:50:26.776 INFO:teuthology.orchestra.run.trial110.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [22.7 kB] 2026-02-24T21:50:26.777 INFO:teuthology.orchestra.run.trial110.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [5217 kB] 2026-02-24T21:50:26.893 INFO:teuthology.orchestra.run.trial110.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy-security/restricted Translation-en [993 kB] 2026-02-24T21:50:26.929 INFO:teuthology.orchestra.run.trial046.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [1022 kB] 2026-02-24T21:50:26.951 INFO:teuthology.orchestra.run.trial046.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 Packages [3007 kB] 2026-02-24T21:50:27.039 INFO:teuthology.orchestra.run.trial046.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy-security/main Translation-en [430 kB] 2026-02-24T21:50:27.047 INFO:teuthology.orchestra.run.trial046.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [14.1 kB] 2026-02-24T21:50:27.048 INFO:teuthology.orchestra.run.trial046.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1018 kB] 2026-02-24T21:50:27.083 INFO:teuthology.orchestra.run.trial046.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy-security/universe Translation-en [225 kB] 2026-02-24T21:50:27.091 INFO:teuthology.orchestra.run.trial046.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [22.7 kB] 2026-02-24T21:50:27.091 INFO:teuthology.orchestra.run.trial046.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [5217 kB] 2026-02-24T21:50:27.238 INFO:teuthology.orchestra.run.trial046.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy-security/restricted Translation-en [993 kB] 2026-02-24T21:50:30.530 INFO:teuthology.orchestra.run.trial110.stdout:Fetched 23.1 MB in 3s (8626 kB/s) 2026-02-24T21:50:30.696 INFO:teuthology.orchestra.run.trial046.stdout:Fetched 23.1 MB in 3s (7943 kB/s) 2026-02-24T21:50:31.071 INFO:teuthology.orchestra.run.trial110.stdout:Reading package lists... 2026-02-24T21:50:31.084 DEBUG:teuthology.orchestra.run.trial110:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-24T21:50:31.116 INFO:teuthology.orchestra.run.trial110.stdout:Reading package lists... 2026-02-24T21:50:31.225 INFO:teuthology.orchestra.run.trial110.stdout:Building dependency tree... 2026-02-24T21:50:31.226 INFO:teuthology.orchestra.run.trial110.stdout:Reading state information... 2026-02-24T21:50:31.228 INFO:teuthology.orchestra.run.trial046.stdout:Reading package lists... 2026-02-24T21:50:31.242 DEBUG:teuthology.orchestra.run.trial046:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-24T21:50:31.273 INFO:teuthology.orchestra.run.trial046.stdout:Reading package lists... 2026-02-24T21:50:31.315 INFO:teuthology.orchestra.run.trial110.stdout:The following packages were automatically installed and are no longer required: 2026-02-24T21:50:31.315 INFO:teuthology.orchestra.run.trial110.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-24T21:50:31.316 INFO:teuthology.orchestra.run.trial110.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-24T21:50:31.316 INFO:teuthology.orchestra.run.trial110.stdout: sg3-utils-udev 2026-02-24T21:50:31.316 INFO:teuthology.orchestra.run.trial110.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-24T21:50:31.316 INFO:teuthology.orchestra.run.trial110.stdout:The following additional packages will be installed: 2026-02-24T21:50:31.316 INFO:teuthology.orchestra.run.trial110.stdout: linux-generic linux-headers-5.15.0-171 linux-headers-5.15.0-171-generic 2026-02-24T21:50:31.316 INFO:teuthology.orchestra.run.trial110.stdout: linux-headers-generic linux-image-5.15.0-171-generic 2026-02-24T21:50:31.316 INFO:teuthology.orchestra.run.trial110.stdout: linux-modules-5.15.0-171-generic linux-modules-extra-5.15.0-171-generic 2026-02-24T21:50:31.316 INFO:teuthology.orchestra.run.trial110.stdout:Suggested packages: 2026-02-24T21:50:31.316 INFO:teuthology.orchestra.run.trial110.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2026-02-24T21:50:31.329 INFO:teuthology.orchestra.run.trial110.stdout:The following NEW packages will be installed: 2026-02-24T21:50:31.329 INFO:teuthology.orchestra.run.trial110.stdout: linux-headers-5.15.0-171 linux-headers-5.15.0-171-generic 2026-02-24T21:50:31.329 INFO:teuthology.orchestra.run.trial110.stdout: linux-image-5.15.0-171-generic linux-modules-5.15.0-171-generic 2026-02-24T21:50:31.329 INFO:teuthology.orchestra.run.trial110.stdout: linux-modules-extra-5.15.0-171-generic 2026-02-24T21:50:31.329 INFO:teuthology.orchestra.run.trial110.stdout:The following packages will be upgraded: 2026-02-24T21:50:31.329 INFO:teuthology.orchestra.run.trial110.stdout: linux-generic linux-headers-generic linux-image-generic 2026-02-24T21:50:31.383 INFO:teuthology.orchestra.run.trial046.stdout:Building dependency tree... 2026-02-24T21:50:31.383 INFO:teuthology.orchestra.run.trial046.stdout:Reading state information... 2026-02-24T21:50:31.435 INFO:teuthology.orchestra.run.trial110.stdout:3 upgraded, 5 newly installed, 0 to remove and 14 not upgraded. 2026-02-24T21:50:31.435 INFO:teuthology.orchestra.run.trial110.stdout:Need to get 113 MB of archives. 2026-02-24T21:50:31.435 INFO:teuthology.orchestra.run.trial110.stdout:After this operation, 585 MB of additional disk space will be used. 2026-02-24T21:50:31.435 INFO:teuthology.orchestra.run.trial110.stdout:Get:1 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-171-generic amd64 5.15.0-171.181 [22.7 MB] 2026-02-24T21:50:31.470 INFO:teuthology.orchestra.run.trial046.stdout:The following packages were automatically installed and are no longer required: 2026-02-24T21:50:31.471 INFO:teuthology.orchestra.run.trial046.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-24T21:50:31.471 INFO:teuthology.orchestra.run.trial046.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-24T21:50:31.471 INFO:teuthology.orchestra.run.trial046.stdout: sg3-utils-udev 2026-02-24T21:50:31.471 INFO:teuthology.orchestra.run.trial046.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-24T21:50:31.471 INFO:teuthology.orchestra.run.trial046.stdout:The following additional packages will be installed: 2026-02-24T21:50:31.471 INFO:teuthology.orchestra.run.trial046.stdout: linux-generic linux-headers-5.15.0-171 linux-headers-5.15.0-171-generic 2026-02-24T21:50:31.471 INFO:teuthology.orchestra.run.trial046.stdout: linux-headers-generic linux-image-5.15.0-171-generic 2026-02-24T21:50:31.471 INFO:teuthology.orchestra.run.trial046.stdout: linux-modules-5.15.0-171-generic linux-modules-extra-5.15.0-171-generic 2026-02-24T21:50:31.471 INFO:teuthology.orchestra.run.trial046.stdout:Suggested packages: 2026-02-24T21:50:31.471 INFO:teuthology.orchestra.run.trial046.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2026-02-24T21:50:31.483 INFO:teuthology.orchestra.run.trial046.stdout:The following NEW packages will be installed: 2026-02-24T21:50:31.483 INFO:teuthology.orchestra.run.trial046.stdout: linux-headers-5.15.0-171 linux-headers-5.15.0-171-generic 2026-02-24T21:50:31.484 INFO:teuthology.orchestra.run.trial046.stdout: linux-image-5.15.0-171-generic linux-modules-5.15.0-171-generic 2026-02-24T21:50:31.484 INFO:teuthology.orchestra.run.trial046.stdout: linux-modules-extra-5.15.0-171-generic 2026-02-24T21:50:31.484 INFO:teuthology.orchestra.run.trial046.stdout:The following packages will be upgraded: 2026-02-24T21:50:31.484 INFO:teuthology.orchestra.run.trial046.stdout: linux-generic linux-headers-generic linux-image-generic 2026-02-24T21:50:31.627 INFO:teuthology.orchestra.run.trial110.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-171-generic amd64 5.15.0-171.181 [11.6 MB] 2026-02-24T21:50:31.673 INFO:teuthology.orchestra.run.trial110.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-171-generic amd64 5.15.0-171.181 [63.9 MB] 2026-02-24T21:50:31.834 INFO:teuthology.orchestra.run.trial046.stdout:3 upgraded, 5 newly installed, 0 to remove and 14 not upgraded. 2026-02-24T21:50:31.834 INFO:teuthology.orchestra.run.trial046.stdout:Need to get 113 MB of archives. 2026-02-24T21:50:31.834 INFO:teuthology.orchestra.run.trial046.stdout:After this operation, 585 MB of additional disk space will be used. 2026-02-24T21:50:31.834 INFO:teuthology.orchestra.run.trial046.stdout:Get:1 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-171-generic amd64 5.15.0-171.181 [22.7 MB] 2026-02-24T21:50:32.059 INFO:teuthology.orchestra.run.trial110.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.171.160 [1658 B] 2026-02-24T21:50:32.060 INFO:teuthology.orchestra.run.trial110.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.171.160 [2486 B] 2026-02-24T21:50:32.060 INFO:teuthology.orchestra.run.trial110.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-171 all 5.15.0-171.181 [12.4 MB] 2026-02-24T21:50:32.111 INFO:teuthology.orchestra.run.trial110.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-171-generic amd64 5.15.0-171.181 [2849 kB] 2026-02-24T21:50:32.121 INFO:teuthology.orchestra.run.trial110.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.171.160 [2332 B] 2026-02-24T21:50:32.319 INFO:teuthology.orchestra.run.trial110.stdout:Fetched 113 MB in 1s (143 MB/s) 2026-02-24T21:50:32.340 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package linux-modules-5.15.0-171-generic. 2026-02-24T21:50:32.516 INFO:teuthology.orchestra.run.trial110.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 167442 files and directories currently installed.) 2026-02-24T21:50:32.518 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../0-linux-modules-5.15.0-171-generic_5.15.0-171.181_amd64.deb ... 2026-02-24T21:50:32.519 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking linux-modules-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T21:50:32.858 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package linux-image-5.15.0-171-generic. 2026-02-24T21:50:32.867 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../1-linux-image-5.15.0-171-generic_5.15.0-171.181_amd64.deb ... 2026-02-24T21:50:32.888 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking linux-image-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T21:50:32.929 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-171-generic. 2026-02-24T21:50:32.937 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-171-generic_5.15.0-171.181_amd64.deb ... 2026-02-24T21:50:32.938 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking linux-modules-extra-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T21:50:32.943 INFO:teuthology.orchestra.run.trial046.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-171-generic amd64 5.15.0-171.181 [11.6 MB] 2026-02-24T21:50:33.222 INFO:teuthology.orchestra.run.trial046.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-171-generic amd64 5.15.0-171.181 [63.9 MB] 2026-02-24T21:50:34.049 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../3-linux-generic_5.15.0.171.160_amd64.deb ... 2026-02-24T21:50:34.050 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking linux-generic (5.15.0.171.160) over (5.15.0.170.159) ... 2026-02-24T21:50:34.075 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.171.160_amd64.deb ... 2026-02-24T21:50:34.075 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking linux-image-generic (5.15.0.171.160) over (5.15.0.170.159) ... 2026-02-24T21:50:34.092 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package linux-headers-5.15.0-171. 2026-02-24T21:50:34.100 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../5-linux-headers-5.15.0-171_5.15.0-171.181_all.deb ... 2026-02-24T21:50:34.101 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking linux-headers-5.15.0-171 (5.15.0-171.181) ... 2026-02-24T21:50:34.727 INFO:teuthology.orchestra.run.trial046.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.171.160 [1658 B] 2026-02-24T21:50:34.728 INFO:teuthology.orchestra.run.trial046.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.171.160 [2486 B] 2026-02-24T21:50:34.728 INFO:teuthology.orchestra.run.trial046.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-171 all 5.15.0-171.181 [12.4 MB] 2026-02-24T21:50:35.005 INFO:teuthology.orchestra.run.trial046.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-171-generic amd64 5.15.0-171.181 [2849 kB] 2026-02-24T21:50:35.071 INFO:teuthology.orchestra.run.trial046.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.171.160 [2332 B] 2026-02-24T21:50:35.268 INFO:teuthology.orchestra.run.trial046.stdout:Fetched 113 MB in 4s (31.6 MB/s) 2026-02-24T21:50:35.286 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package linux-modules-5.15.0-171-generic. 2026-02-24T21:50:35.429 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package linux-headers-5.15.0-171-generic. 2026-02-24T21:50:35.443 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../6-linux-headers-5.15.0-171-generic_5.15.0-171.181_amd64.deb ... 2026-02-24T21:50:35.443 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking linux-headers-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T21:50:35.468 INFO:teuthology.orchestra.run.trial046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 167442 files and directories currently installed.) 2026-02-24T21:50:35.470 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../0-linux-modules-5.15.0-171-generic_5.15.0-171.181_amd64.deb ... 2026-02-24T21:50:35.471 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking linux-modules-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T21:50:35.810 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package linux-image-5.15.0-171-generic. 2026-02-24T21:50:35.819 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../1-linux-image-5.15.0-171-generic_5.15.0-171.181_amd64.deb ... 2026-02-24T21:50:35.837 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking linux-image-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T21:50:35.878 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-171-generic. 2026-02-24T21:50:35.887 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-171-generic_5.15.0-171.181_amd64.deb ... 2026-02-24T21:50:35.888 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking linux-modules-extra-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T21:50:36.034 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.171.160_amd64.deb ... 2026-02-24T21:50:36.035 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking linux-headers-generic (5.15.0.171.160) over (5.15.0.170.159) ... 2026-02-24T21:50:36.061 INFO:teuthology.orchestra.run.trial110.stdout:Setting up linux-headers-5.15.0-171 (5.15.0-171.181) ... 2026-02-24T21:50:36.062 INFO:teuthology.orchestra.run.trial110.stdout:Setting up linux-headers-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T21:50:36.063 INFO:teuthology.orchestra.run.trial110.stdout:Setting up linux-headers-generic (5.15.0.171.160) ... 2026-02-24T21:50:36.065 INFO:teuthology.orchestra.run.trial110.stdout:Setting up linux-image-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T21:50:36.851 INFO:teuthology.orchestra.run.trial110.stdout:I: /boot/vmlinuz.old is now a symlink to vmlinuz-5.15.0-170-generic 2026-02-24T21:50:36.852 INFO:teuthology.orchestra.run.trial110.stdout:I: /boot/initrd.img.old is now a symlink to initrd.img-5.15.0-170-generic 2026-02-24T21:50:36.852 INFO:teuthology.orchestra.run.trial110.stdout:I: /boot/vmlinuz is now a symlink to vmlinuz-5.15.0-171-generic 2026-02-24T21:50:36.852 INFO:teuthology.orchestra.run.trial110.stdout:I: /boot/initrd.img is now a symlink to initrd.img-5.15.0-171-generic 2026-02-24T21:50:36.859 INFO:teuthology.orchestra.run.trial110.stdout:Setting up linux-modules-extra-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T21:50:36.985 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../3-linux-generic_5.15.0.171.160_amd64.deb ... 2026-02-24T21:50:36.985 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking linux-generic (5.15.0.171.160) over (5.15.0.170.159) ... 2026-02-24T21:50:37.011 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.171.160_amd64.deb ... 2026-02-24T21:50:37.011 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking linux-image-generic (5.15.0.171.160) over (5.15.0.170.159) ... 2026-02-24T21:50:37.028 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package linux-headers-5.15.0-171. 2026-02-24T21:50:37.037 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../5-linux-headers-5.15.0-171_5.15.0-171.181_all.deb ... 2026-02-24T21:50:37.037 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking linux-headers-5.15.0-171 (5.15.0-171.181) ... 2026-02-24T21:50:37.675 INFO:teuthology.orchestra.run.trial110.stdout:Setting up linux-modules-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T21:50:38.389 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package linux-headers-5.15.0-171-generic. 2026-02-24T21:50:38.403 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../6-linux-headers-5.15.0-171-generic_5.15.0-171.181_amd64.deb ... 2026-02-24T21:50:38.404 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking linux-headers-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T21:50:38.483 INFO:teuthology.orchestra.run.trial110.stdout:Setting up linux-image-generic (5.15.0.171.160) ... 2026-02-24T21:50:38.485 INFO:teuthology.orchestra.run.trial110.stdout:Setting up linux-generic (5.15.0.171.160) ... 2026-02-24T21:50:38.486 INFO:teuthology.orchestra.run.trial110.stdout:Processing triggers for linux-image-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T21:50:38.491 INFO:teuthology.orchestra.run.trial110.stdout:/etc/kernel/postinst.d/initramfs-tools: 2026-02-24T21:50:38.491 INFO:teuthology.orchestra.run.trial110.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-171-generic 2026-02-24T21:50:38.994 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.171.160_amd64.deb ... 2026-02-24T21:50:38.995 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking linux-headers-generic (5.15.0.171.160) over (5.15.0.170.159) ... 2026-02-24T21:50:39.021 INFO:teuthology.orchestra.run.trial046.stdout:Setting up linux-headers-5.15.0-171 (5.15.0-171.181) ... 2026-02-24T21:50:39.022 INFO:teuthology.orchestra.run.trial046.stdout:Setting up linux-headers-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T21:50:39.023 INFO:teuthology.orchestra.run.trial046.stdout:Setting up linux-headers-generic (5.15.0.171.160) ... 2026-02-24T21:50:39.024 INFO:teuthology.orchestra.run.trial046.stdout:Setting up linux-image-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T21:50:39.804 INFO:teuthology.orchestra.run.trial046.stdout:I: /boot/vmlinuz.old is now a symlink to vmlinuz-5.15.0-170-generic 2026-02-24T21:50:39.804 INFO:teuthology.orchestra.run.trial046.stdout:I: /boot/initrd.img.old is now a symlink to initrd.img-5.15.0-170-generic 2026-02-24T21:50:39.804 INFO:teuthology.orchestra.run.trial046.stdout:I: /boot/vmlinuz is now a symlink to vmlinuz-5.15.0-171-generic 2026-02-24T21:50:39.804 INFO:teuthology.orchestra.run.trial046.stdout:I: /boot/initrd.img is now a symlink to initrd.img-5.15.0-171-generic 2026-02-24T21:50:39.813 INFO:teuthology.orchestra.run.trial046.stdout:Setting up linux-modules-extra-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T21:50:40.619 INFO:teuthology.orchestra.run.trial046.stdout:Setting up linux-modules-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T21:50:41.411 INFO:teuthology.orchestra.run.trial046.stdout:Setting up linux-image-generic (5.15.0.171.160) ... 2026-02-24T21:50:41.412 INFO:teuthology.orchestra.run.trial046.stdout:Setting up linux-generic (5.15.0.171.160) ... 2026-02-24T21:50:41.414 INFO:teuthology.orchestra.run.trial046.stdout:Processing triggers for linux-image-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T21:50:41.420 INFO:teuthology.orchestra.run.trial046.stdout:/etc/kernel/postinst.d/initramfs-tools: 2026-02-24T21:50:41.420 INFO:teuthology.orchestra.run.trial046.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-171-generic 2026-02-24T21:50:45.586 INFO:teuthology.orchestra.run.trial110.stdout:W: Possible missing firmware /lib/firmware/ast_dp501_fw.bin for module ast 2026-02-24T21:50:48.483 INFO:teuthology.orchestra.run.trial046.stdout:W: Possible missing firmware /lib/firmware/ast_dp501_fw.bin for module ast 2026-02-24T21:50:52.371 INFO:teuthology.orchestra.run.trial110.stdout:/etc/kernel/postinst.d/zz-update-grub: 2026-02-24T21:50:52.371 INFO:teuthology.orchestra.run.trial110.stdout:Sourcing file `/etc/default/grub' 2026-02-24T21:50:52.392 INFO:teuthology.orchestra.run.trial110.stdout:Sourcing file `/etc/default/grub.d/50-curtin-settings.cfg' 2026-02-24T21:50:52.393 INFO:teuthology.orchestra.run.trial110.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2026-02-24T21:50:52.395 INFO:teuthology.orchestra.run.trial110.stdout:Generating grub configuration file ... 2026-02-24T21:50:52.480 INFO:teuthology.orchestra.run.trial110.stdout:Found linux image: /boot/vmlinuz-5.15.0-171-generic 2026-02-24T21:50:52.486 INFO:teuthology.orchestra.run.trial110.stdout:Found initrd image: /boot/initrd.img-5.15.0-171-generic 2026-02-24T21:50:52.663 INFO:teuthology.orchestra.run.trial110.stdout:Found linux image: /boot/vmlinuz-5.15.0-170-generic 2026-02-24T21:50:52.669 INFO:teuthology.orchestra.run.trial110.stdout:Found initrd image: /boot/initrd.img-5.15.0-170-generic 2026-02-24T21:50:52.712 INFO:teuthology.orchestra.run.trial110.stdout:Found linux image: /boot/vmlinuz-5.15.0-164-generic 2026-02-24T21:50:52.717 INFO:teuthology.orchestra.run.trial110.stdout:Found initrd image: /boot/initrd.img-5.15.0-164-generic 2026-02-24T21:50:52.791 INFO:teuthology.orchestra.run.trial110.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2026-02-24T21:50:52.791 INFO:teuthology.orchestra.run.trial110.stdout:Systems on them will not be added to the GRUB boot configuration. 2026-02-24T21:50:52.791 INFO:teuthology.orchestra.run.trial110.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2026-02-24T21:50:52.793 INFO:teuthology.orchestra.run.trial110.stdout:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-24T21:50:52.801 INFO:teuthology.orchestra.run.trial110.stdout:done 2026-02-24T21:50:53.207 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T21:50:53.207 INFO:teuthology.orchestra.run.trial110.stdout:Pending kernel upgrade! 2026-02-24T21:50:53.207 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T21:50:53.208 INFO:teuthology.orchestra.run.trial110.stdout:Running kernel version: 2026-02-24T21:50:53.208 INFO:teuthology.orchestra.run.trial110.stdout: 5.15.0-170-generic 2026-02-24T21:50:53.208 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T21:50:53.208 INFO:teuthology.orchestra.run.trial110.stdout:Diagnostics: 2026-02-24T21:50:53.208 INFO:teuthology.orchestra.run.trial110.stdout: The currently running kernel version is not the expected kernel version 5.15.0-171-generic. 2026-02-24T21:50:53.208 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T21:50:53.208 INFO:teuthology.orchestra.run.trial110.stdout:Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. [Return] 2026-02-24T21:50:53.208 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T21:50:53.208 INFO:teuthology.orchestra.run.trial110.stdout:The processor microcode seems to be up-to-date. 2026-02-24T21:50:53.208 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T21:50:53.209 INFO:teuthology.orchestra.run.trial110.stdout:No services need to be restarted. 2026-02-24T21:50:53.209 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T21:50:53.209 INFO:teuthology.orchestra.run.trial110.stdout:No containers need to be restarted. 2026-02-24T21:50:53.209 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T21:50:53.209 INFO:teuthology.orchestra.run.trial110.stdout:No user sessions are running outdated binaries. 2026-02-24T21:50:53.209 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T21:50:53.209 INFO:teuthology.orchestra.run.trial110.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-24T21:50:54.041 DEBUG:teuthology.orchestra.run.trial110:> dpkg -s linux-image-generic 2026-02-24T21:50:54.051 INFO:teuthology.orchestra.run.trial110.stdout:Package: linux-image-generic 2026-02-24T21:50:54.051 INFO:teuthology.orchestra.run.trial110.stdout:Status: install ok installed 2026-02-24T21:50:54.051 INFO:teuthology.orchestra.run.trial110.stdout:Priority: optional 2026-02-24T21:50:54.051 INFO:teuthology.orchestra.run.trial110.stdout:Section: kernel 2026-02-24T21:50:54.051 INFO:teuthology.orchestra.run.trial110.stdout:Installed-Size: 22 2026-02-24T21:50:54.051 INFO:teuthology.orchestra.run.trial110.stdout:Maintainer: Ubuntu Kernel Team 2026-02-24T21:50:54.051 INFO:teuthology.orchestra.run.trial110.stdout:Architecture: amd64 2026-02-24T21:50:54.051 INFO:teuthology.orchestra.run.trial110.stdout:Source: linux-meta 2026-02-24T21:50:54.051 INFO:teuthology.orchestra.run.trial110.stdout:Version: 5.15.0.171.160 2026-02-24T21:50:54.051 INFO:teuthology.orchestra.run.trial110.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-171), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-24T21:50:54.051 INFO:teuthology.orchestra.run.trial110.stdout:Depends: linux-image-5.15.0-171-generic, linux-modules-extra-5.15.0-171-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-24T21:50:54.051 INFO:teuthology.orchestra.run.trial110.stdout:Recommends: thermald 2026-02-24T21:50:54.051 INFO:teuthology.orchestra.run.trial110.stdout:Description: Generic Linux kernel image 2026-02-24T21:50:54.051 INFO:teuthology.orchestra.run.trial110.stdout: This package will always depend on the latest generic kernel image 2026-02-24T21:50:54.051 INFO:teuthology.orchestra.run.trial110.stdout: available. 2026-02-24T21:50:54.052 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-170-generic Expected: 5.15.0-171-generic 2026-02-24T21:50:54.052 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-24T21:50:54.052 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-24T21:50:54.052 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2026-02-24T21:50:54.052 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial110.front.sepia.ceph.com, path=None, version=distro) 2026-02-24T21:50:54.052 DEBUG:teuthology.orchestra.run.trial110:> sudo apt-get clean 2026-02-24T21:50:54.133 DEBUG:teuthology.orchestra.run.trial110:> sudo apt-get update 2026-02-24T21:50:54.566 INFO:teuthology.orchestra.run.trial110.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-24T21:50:54.633 INFO:teuthology.orchestra.run.trial110.stdout:Hit:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-24T21:50:54.705 INFO:teuthology.orchestra.run.trial110.stdout:Hit:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-24T21:50:54.777 INFO:teuthology.orchestra.run.trial110.stdout:Hit:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-24T21:50:55.083 INFO:teuthology.orchestra.run.trial046.stdout:/etc/kernel/postinst.d/zz-update-grub: 2026-02-24T21:50:55.083 INFO:teuthology.orchestra.run.trial046.stdout:Sourcing file `/etc/default/grub' 2026-02-24T21:50:55.102 INFO:teuthology.orchestra.run.trial046.stdout:Sourcing file `/etc/default/grub.d/50-curtin-settings.cfg' 2026-02-24T21:50:55.104 INFO:teuthology.orchestra.run.trial046.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2026-02-24T21:50:55.106 INFO:teuthology.orchestra.run.trial046.stdout:Generating grub configuration file ... 2026-02-24T21:50:55.190 INFO:teuthology.orchestra.run.trial046.stdout:Found linux image: /boot/vmlinuz-5.15.0-171-generic 2026-02-24T21:50:55.197 INFO:teuthology.orchestra.run.trial046.stdout:Found initrd image: /boot/initrd.img-5.15.0-171-generic 2026-02-24T21:50:55.374 INFO:teuthology.orchestra.run.trial046.stdout:Found linux image: /boot/vmlinuz-5.15.0-170-generic 2026-02-24T21:50:55.379 INFO:teuthology.orchestra.run.trial046.stdout:Found initrd image: /boot/initrd.img-5.15.0-170-generic 2026-02-24T21:50:55.423 INFO:teuthology.orchestra.run.trial046.stdout:Found linux image: /boot/vmlinuz-5.15.0-164-generic 2026-02-24T21:50:55.428 INFO:teuthology.orchestra.run.trial046.stdout:Found initrd image: /boot/initrd.img-5.15.0-164-generic 2026-02-24T21:50:55.445 INFO:teuthology.orchestra.run.trial110.stdout:Reading package lists... 2026-02-24T21:50:55.459 DEBUG:teuthology.orchestra.run.trial110:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-24T21:50:55.492 INFO:teuthology.orchestra.run.trial110.stdout:Reading package lists... 2026-02-24T21:50:55.502 INFO:teuthology.orchestra.run.trial046.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2026-02-24T21:50:55.502 INFO:teuthology.orchestra.run.trial046.stdout:Systems on them will not be added to the GRUB boot configuration. 2026-02-24T21:50:55.502 INFO:teuthology.orchestra.run.trial046.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2026-02-24T21:50:55.505 INFO:teuthology.orchestra.run.trial046.stdout:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-24T21:50:55.513 INFO:teuthology.orchestra.run.trial046.stdout:done 2026-02-24T21:50:55.602 INFO:teuthology.orchestra.run.trial110.stdout:Building dependency tree... 2026-02-24T21:50:55.602 INFO:teuthology.orchestra.run.trial110.stdout:Reading state information... 2026-02-24T21:50:55.688 INFO:teuthology.orchestra.run.trial110.stdout:linux-image-generic is already the newest version (5.15.0.171.160). 2026-02-24T21:50:55.689 INFO:teuthology.orchestra.run.trial110.stdout:linux-image-generic set to manually installed. 2026-02-24T21:50:55.689 INFO:teuthology.orchestra.run.trial110.stdout:The following packages were automatically installed and are no longer required: 2026-02-24T21:50:55.689 INFO:teuthology.orchestra.run.trial110.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-24T21:50:55.689 INFO:teuthology.orchestra.run.trial110.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 2026-02-24T21:50:55.689 INFO:teuthology.orchestra.run.trial110.stdout: linux-headers-5.15.0-164 linux-headers-5.15.0-164-generic 2026-02-24T21:50:55.689 INFO:teuthology.orchestra.run.trial110.stdout: linux-image-5.15.0-164-generic linux-modules-5.15.0-164-generic 2026-02-24T21:50:55.689 INFO:teuthology.orchestra.run.trial110.stdout: linux-modules-extra-5.15.0-164-generic sg3-utils sg3-utils-udev 2026-02-24T21:50:55.689 INFO:teuthology.orchestra.run.trial110.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-24T21:50:55.708 INFO:teuthology.orchestra.run.trial110.stdout:0 upgraded, 0 newly installed, 0 to remove and 14 not upgraded. 2026-02-24T21:50:55.710 DEBUG:teuthology.orchestra.run.trial110:> dpkg -s linux-image-generic 2026-02-24T21:50:55.763 INFO:teuthology.orchestra.run.trial110.stdout:Package: linux-image-generic 2026-02-24T21:50:55.764 INFO:teuthology.orchestra.run.trial110.stdout:Status: install ok installed 2026-02-24T21:50:55.764 INFO:teuthology.orchestra.run.trial110.stdout:Priority: optional 2026-02-24T21:50:55.764 INFO:teuthology.orchestra.run.trial110.stdout:Section: kernel 2026-02-24T21:50:55.764 INFO:teuthology.orchestra.run.trial110.stdout:Installed-Size: 22 2026-02-24T21:50:55.764 INFO:teuthology.orchestra.run.trial110.stdout:Maintainer: Ubuntu Kernel Team 2026-02-24T21:50:55.764 INFO:teuthology.orchestra.run.trial110.stdout:Architecture: amd64 2026-02-24T21:50:55.764 INFO:teuthology.orchestra.run.trial110.stdout:Source: linux-meta 2026-02-24T21:50:55.764 INFO:teuthology.orchestra.run.trial110.stdout:Version: 5.15.0.171.160 2026-02-24T21:50:55.765 INFO:teuthology.orchestra.run.trial110.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-171), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-24T21:50:55.765 INFO:teuthology.orchestra.run.trial110.stdout:Depends: linux-image-5.15.0-171-generic, linux-modules-extra-5.15.0-171-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-24T21:50:55.765 INFO:teuthology.orchestra.run.trial110.stdout:Recommends: thermald 2026-02-24T21:50:55.765 INFO:teuthology.orchestra.run.trial110.stdout:Description: Generic Linux kernel image 2026-02-24T21:50:55.765 INFO:teuthology.orchestra.run.trial110.stdout: This package will always depend on the latest generic kernel image 2026-02-24T21:50:55.765 INFO:teuthology.orchestra.run.trial110.stdout: available. 2026-02-24T21:50:55.766 DEBUG:teuthology.orchestra.run.trial110:> mktemp 2026-02-24T21:50:55.810 INFO:teuthology.orchestra.run.trial110.stdout:/tmp/tmp.EK9rnNs19F 2026-02-24T21:50:55.811 DEBUG:teuthology.orchestra.run.trial110:> sudo cp /boot/grub/grub.cfg /tmp/tmp.EK9rnNs19F 2026-02-24T21:50:55.863 DEBUG:teuthology.orchestra.run.trial110:> sudo chmod 0666 /tmp/tmp.EK9rnNs19F 2026-02-24T21:50:55.912 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T21:50:55.912 INFO:teuthology.orchestra.run.trial046.stdout:Pending kernel upgrade! 2026-02-24T21:50:55.912 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T21:50:55.913 INFO:teuthology.orchestra.run.trial046.stdout:Running kernel version: 2026-02-24T21:50:55.913 INFO:teuthology.orchestra.run.trial046.stdout: 5.15.0-170-generic 2026-02-24T21:50:55.913 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T21:50:55.913 INFO:teuthology.orchestra.run.trial046.stdout:Diagnostics: 2026-02-24T21:50:55.913 INFO:teuthology.orchestra.run.trial046.stdout: The currently running kernel version is not the expected kernel version 5.15.0-171-generic. 2026-02-24T21:50:55.913 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T21:50:55.913 INFO:teuthology.orchestra.run.trial046.stdout:Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. [Return] 2026-02-24T21:50:55.913 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T21:50:55.913 INFO:teuthology.orchestra.run.trial046.stdout:The processor microcode seems to be up-to-date. 2026-02-24T21:50:55.914 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T21:50:55.914 INFO:teuthology.orchestra.run.trial046.stdout:No services need to be restarted. 2026-02-24T21:50:55.914 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T21:50:55.914 INFO:teuthology.orchestra.run.trial046.stdout:No containers need to be restarted. 2026-02-24T21:50:55.914 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T21:50:55.914 INFO:teuthology.orchestra.run.trial046.stdout:No user sessions are running outdated binaries. 2026-02-24T21:50:55.914 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T21:50:55.914 INFO:teuthology.orchestra.run.trial046.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-24T21:50:55.965 DEBUG:teuthology.orchestra.remote:trial110:/tmp/tmp.EK9rnNs19F is 9KB 2026-02-24T21:50:55.973 DEBUG:teuthology.orchestra.run.trial110:> rm -fr /tmp/tmp.EK9rnNs19F 2026-02-24T21:50:55.977 DEBUG:teuthology.orchestra.run.trial110:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2026-02-24T21:50:56.031 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T21:50:56.031 DEBUG:teuthology.orchestra.run.trial110:> sudo dd of=/etc/grub.d/01_ceph_kernel 2026-02-24T21:50:56.031 DEBUG:teuthology.orchestra.run.trial110:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2026-02-24T21:50:56.088 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-171-generic 2026-02-24T21:50:56.088 DEBUG:teuthology.orchestra.run.trial110:> sudo update-grub 2026-02-24T21:50:56.321 INFO:teuthology.orchestra.run.trial110.stderr:Sourcing file `/etc/default/grub' 2026-02-24T21:50:56.340 INFO:teuthology.orchestra.run.trial110.stderr:Sourcing file `/etc/default/grub.d/50-curtin-settings.cfg' 2026-02-24T21:50:56.341 INFO:teuthology.orchestra.run.trial110.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2026-02-24T21:50:56.344 INFO:teuthology.orchestra.run.trial110.stderr:Generating grub configuration file ... 2026-02-24T21:50:56.426 INFO:teuthology.orchestra.run.trial110.stderr:Found linux image: /boot/vmlinuz-5.15.0-171-generic 2026-02-24T21:50:56.432 INFO:teuthology.orchestra.run.trial110.stderr:Found initrd image: /boot/initrd.img-5.15.0-171-generic 2026-02-24T21:50:56.604 INFO:teuthology.orchestra.run.trial110.stderr:Found linux image: /boot/vmlinuz-5.15.0-170-generic 2026-02-24T21:50:56.609 INFO:teuthology.orchestra.run.trial110.stderr:Found initrd image: /boot/initrd.img-5.15.0-170-generic 2026-02-24T21:50:56.651 INFO:teuthology.orchestra.run.trial110.stderr:Found linux image: /boot/vmlinuz-5.15.0-164-generic 2026-02-24T21:50:56.655 INFO:teuthology.orchestra.run.trial110.stderr:Found initrd image: /boot/initrd.img-5.15.0-164-generic 2026-02-24T21:50:56.728 INFO:teuthology.orchestra.run.trial110.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2026-02-24T21:50:56.728 INFO:teuthology.orchestra.run.trial110.stderr:Systems on them will not be added to the GRUB boot configuration. 2026-02-24T21:50:56.728 INFO:teuthology.orchestra.run.trial110.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2026-02-24T21:50:56.731 INFO:teuthology.orchestra.run.trial110.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-24T21:50:56.732 DEBUG:teuthology.orchestra.run.trial046:> dpkg -s linux-image-generic 2026-02-24T21:50:56.738 INFO:teuthology.orchestra.run.trial110.stderr:done 2026-02-24T21:50:56.739 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial110 2026-02-24T21:50:56.739 DEBUG:teuthology.orchestra.run.trial110:> sudo bash -c ' 2026-02-24T21:50:56.739 DEBUG:teuthology.orchestra.run.trial110:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-24T21:50:56.739 DEBUG:teuthology.orchestra.run.trial110:> ' 2026-02-24T21:50:56.740 INFO:teuthology.orchestra.run.trial046.stdout:Package: linux-image-generic 2026-02-24T21:50:56.740 INFO:teuthology.orchestra.run.trial046.stdout:Status: install ok installed 2026-02-24T21:50:56.741 INFO:teuthology.orchestra.run.trial046.stdout:Priority: optional 2026-02-24T21:50:56.741 INFO:teuthology.orchestra.run.trial046.stdout:Section: kernel 2026-02-24T21:50:56.741 INFO:teuthology.orchestra.run.trial046.stdout:Installed-Size: 22 2026-02-24T21:50:56.741 INFO:teuthology.orchestra.run.trial046.stdout:Maintainer: Ubuntu Kernel Team 2026-02-24T21:50:56.741 INFO:teuthology.orchestra.run.trial046.stdout:Architecture: amd64 2026-02-24T21:50:56.741 INFO:teuthology.orchestra.run.trial046.stdout:Source: linux-meta 2026-02-24T21:50:56.741 INFO:teuthology.orchestra.run.trial046.stdout:Version: 5.15.0.171.160 2026-02-24T21:50:56.741 INFO:teuthology.orchestra.run.trial046.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-171), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-24T21:50:56.741 INFO:teuthology.orchestra.run.trial046.stdout:Depends: linux-image-5.15.0-171-generic, linux-modules-extra-5.15.0-171-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-24T21:50:56.741 INFO:teuthology.orchestra.run.trial046.stdout:Recommends: thermald 2026-02-24T21:50:56.741 INFO:teuthology.orchestra.run.trial046.stdout:Description: Generic Linux kernel image 2026-02-24T21:50:56.741 INFO:teuthology.orchestra.run.trial046.stdout: This package will always depend on the latest generic kernel image 2026-02-24T21:50:56.741 INFO:teuthology.orchestra.run.trial046.stdout: available. 2026-02-24T21:50:56.741 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-170-generic Expected: 5.15.0-171-generic 2026-02-24T21:50:56.741 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-24T21:50:56.741 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-24T21:50:56.741 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-24T21:50:56.741 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial046.front.sepia.ceph.com, path=None, version=distro) 2026-02-24T21:50:56.741 DEBUG:teuthology.orchestra.run.trial046:> sudo apt-get clean 2026-02-24T21:50:56.827 DEBUG:teuthology.orchestra.run.trial046:> sudo apt-get update 2026-02-24T21:50:57.332 INFO:teuthology.orchestra.run.trial046.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-24T21:50:57.418 INFO:teuthology.orchestra.run.trial046.stdout:Hit:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-24T21:50:57.510 INFO:teuthology.orchestra.run.trial046.stdout:Hit:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-24T21:50:57.602 INFO:teuthology.orchestra.run.trial046.stdout:Hit:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-24T21:50:58.253 INFO:teuthology.orchestra.run.trial046.stdout:Reading package lists... 2026-02-24T21:50:58.267 DEBUG:teuthology.orchestra.run.trial046:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-24T21:50:58.301 INFO:teuthology.orchestra.run.trial046.stdout:Reading package lists... 2026-02-24T21:50:58.417 INFO:teuthology.orchestra.run.trial046.stdout:Building dependency tree... 2026-02-24T21:50:58.418 INFO:teuthology.orchestra.run.trial046.stdout:Reading state information... 2026-02-24T21:50:58.508 INFO:teuthology.orchestra.run.trial046.stdout:linux-image-generic is already the newest version (5.15.0.171.160). 2026-02-24T21:50:58.508 INFO:teuthology.orchestra.run.trial046.stdout:linux-image-generic set to manually installed. 2026-02-24T21:50:58.508 INFO:teuthology.orchestra.run.trial046.stdout:The following packages were automatically installed and are no longer required: 2026-02-24T21:50:58.508 INFO:teuthology.orchestra.run.trial046.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-24T21:50:58.508 INFO:teuthology.orchestra.run.trial046.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 2026-02-24T21:50:58.508 INFO:teuthology.orchestra.run.trial046.stdout: linux-headers-5.15.0-164 linux-headers-5.15.0-164-generic 2026-02-24T21:50:58.508 INFO:teuthology.orchestra.run.trial046.stdout: linux-image-5.15.0-164-generic linux-modules-5.15.0-164-generic 2026-02-24T21:50:58.509 INFO:teuthology.orchestra.run.trial046.stdout: linux-modules-extra-5.15.0-164-generic sg3-utils sg3-utils-udev 2026-02-24T21:50:58.509 INFO:teuthology.orchestra.run.trial046.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-24T21:50:58.528 INFO:teuthology.orchestra.run.trial046.stdout:0 upgraded, 0 newly installed, 0 to remove and 14 not upgraded. 2026-02-24T21:50:58.530 DEBUG:teuthology.orchestra.run.trial046:> dpkg -s linux-image-generic 2026-02-24T21:50:58.580 INFO:teuthology.orchestra.run.trial046.stdout:Package: linux-image-generic 2026-02-24T21:50:58.580 INFO:teuthology.orchestra.run.trial046.stdout:Status: install ok installed 2026-02-24T21:50:58.580 INFO:teuthology.orchestra.run.trial046.stdout:Priority: optional 2026-02-24T21:50:58.581 INFO:teuthology.orchestra.run.trial046.stdout:Section: kernel 2026-02-24T21:50:58.581 INFO:teuthology.orchestra.run.trial046.stdout:Installed-Size: 22 2026-02-24T21:50:58.581 INFO:teuthology.orchestra.run.trial046.stdout:Maintainer: Ubuntu Kernel Team 2026-02-24T21:50:58.581 INFO:teuthology.orchestra.run.trial046.stdout:Architecture: amd64 2026-02-24T21:50:58.581 INFO:teuthology.orchestra.run.trial046.stdout:Source: linux-meta 2026-02-24T21:50:58.581 INFO:teuthology.orchestra.run.trial046.stdout:Version: 5.15.0.171.160 2026-02-24T21:50:58.581 INFO:teuthology.orchestra.run.trial046.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-171), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-24T21:50:58.581 INFO:teuthology.orchestra.run.trial046.stdout:Depends: linux-image-5.15.0-171-generic, linux-modules-extra-5.15.0-171-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-24T21:50:58.581 INFO:teuthology.orchestra.run.trial046.stdout:Recommends: thermald 2026-02-24T21:50:58.581 INFO:teuthology.orchestra.run.trial046.stdout:Description: Generic Linux kernel image 2026-02-24T21:50:58.581 INFO:teuthology.orchestra.run.trial046.stdout: This package will always depend on the latest generic kernel image 2026-02-24T21:50:58.582 INFO:teuthology.orchestra.run.trial046.stdout: available. 2026-02-24T21:50:58.583 DEBUG:teuthology.orchestra.run.trial046:> mktemp 2026-02-24T21:50:58.627 INFO:teuthology.orchestra.run.trial046.stdout:/tmp/tmp.DnCszAIODc 2026-02-24T21:50:58.628 DEBUG:teuthology.orchestra.run.trial046:> sudo cp /boot/grub/grub.cfg /tmp/tmp.DnCszAIODc 2026-02-24T21:50:58.675 DEBUG:teuthology.orchestra.run.trial046:> sudo chmod 0666 /tmp/tmp.DnCszAIODc 2026-02-24T21:50:58.770 DEBUG:teuthology.orchestra.remote:trial046:/tmp/tmp.DnCszAIODc is 9KB 2026-02-24T21:50:58.778 DEBUG:teuthology.orchestra.run.trial046:> rm -fr /tmp/tmp.DnCszAIODc 2026-02-24T21:50:58.782 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2026-02-24T21:50:58.832 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T21:50:58.832 DEBUG:teuthology.orchestra.run.trial046:> sudo dd of=/etc/grub.d/01_ceph_kernel 2026-02-24T21:50:58.832 DEBUG:teuthology.orchestra.run.trial046:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2026-02-24T21:50:58.889 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-171-generic 2026-02-24T21:50:58.890 DEBUG:teuthology.orchestra.run.trial046:> sudo update-grub 2026-02-24T21:50:59.117 INFO:teuthology.orchestra.run.trial046.stderr:Sourcing file `/etc/default/grub' 2026-02-24T21:50:59.135 INFO:teuthology.orchestra.run.trial046.stderr:Sourcing file `/etc/default/grub.d/50-curtin-settings.cfg' 2026-02-24T21:50:59.136 INFO:teuthology.orchestra.run.trial046.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2026-02-24T21:50:59.138 INFO:teuthology.orchestra.run.trial046.stderr:Generating grub configuration file ... 2026-02-24T21:50:59.222 INFO:teuthology.orchestra.run.trial046.stderr:Found linux image: /boot/vmlinuz-5.15.0-171-generic 2026-02-24T21:50:59.228 INFO:teuthology.orchestra.run.trial046.stderr:Found initrd image: /boot/initrd.img-5.15.0-171-generic 2026-02-24T21:50:59.397 INFO:teuthology.orchestra.run.trial046.stderr:Found linux image: /boot/vmlinuz-5.15.0-170-generic 2026-02-24T21:50:59.403 INFO:teuthology.orchestra.run.trial046.stderr:Found initrd image: /boot/initrd.img-5.15.0-170-generic 2026-02-24T21:50:59.445 INFO:teuthology.orchestra.run.trial046.stderr:Found linux image: /boot/vmlinuz-5.15.0-164-generic 2026-02-24T21:50:59.451 INFO:teuthology.orchestra.run.trial046.stderr:Found initrd image: /boot/initrd.img-5.15.0-164-generic 2026-02-24T21:50:59.524 INFO:teuthology.orchestra.run.trial046.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2026-02-24T21:50:59.524 INFO:teuthology.orchestra.run.trial046.stderr:Systems on them will not be added to the GRUB boot configuration. 2026-02-24T21:50:59.525 INFO:teuthology.orchestra.run.trial046.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2026-02-24T21:50:59.526 INFO:teuthology.orchestra.run.trial046.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-24T21:50:59.533 INFO:teuthology.orchestra.run.trial046.stderr:done 2026-02-24T21:50:59.535 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial046 2026-02-24T21:50:59.535 DEBUG:teuthology.orchestra.run.trial046:> sudo bash -c ' 2026-02-24T21:50:59.536 DEBUG:teuthology.orchestra.run.trial046:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-24T21:50:59.536 DEBUG:teuthology.orchestra.run.trial046:> ' 2026-02-24T21:51:26.741 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2026-02-24T21:51:26.741 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial110.front.sepia.ceph.com' 2026-02-24T21:51:26.742 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T21:51:29.538 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-24T21:51:29.539 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial046.front.sepia.ceph.com' 2026-02-24T21:51:29.539 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T21:52:26.807 DEBUG:teuthology.orchestra.remote:timed out 2026-02-24T21:52:29.544 DEBUG:teuthology.orchestra.remote:timed out 2026-02-24T21:52:35.813 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial110.front.sepia.ceph.com' 2026-02-24T21:52:35.814 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T21:52:35.943 DEBUG:teuthology.orchestra.run.trial110:> true 2026-02-24T21:52:36.338 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial110.front.sepia.ceph.com' 2026-02-24T21:52:36.338 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-171-generic"... 2026-02-24T21:52:36.338 DEBUG:teuthology.orchestra.run.trial110:> uname -r 2026-02-24T21:52:36.388 INFO:teuthology.orchestra.run.trial110.stdout:5.15.0-171-generic 2026-02-24T21:52:36.389 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-171-generic vs 5.15.0-171-generic 2026-02-24T21:52:36.389 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-24T21:52:36.389 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2026-02-24T21:52:37.390 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2026-02-24T21:52:37.390 DEBUG:teuthology.orchestra.run.trial110:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-24T21:52:37.403 INFO:teuthology.orchestra.run.trial110.stdout:ttyS1 2026-02-24T21:52:37.423 DEBUG:teuthology.parallel:result is None 2026-02-24T21:52:38.545 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial046.front.sepia.ceph.com' 2026-02-24T21:52:38.545 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T21:52:38.628 DEBUG:teuthology.orchestra.run.trial046:> true 2026-02-24T21:52:39.024 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial046.front.sepia.ceph.com' 2026-02-24T21:52:39.024 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-171-generic"... 2026-02-24T21:52:39.024 DEBUG:teuthology.orchestra.run.trial046:> uname -r 2026-02-24T21:52:39.073 INFO:teuthology.orchestra.run.trial046.stdout:5.15.0-171-generic 2026-02-24T21:52:39.073 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-171-generic vs 5.15.0-171-generic 2026-02-24T21:52:39.073 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-24T21:52:39.073 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2026-02-24T21:52:40.073 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-24T21:52:40.074 DEBUG:teuthology.orchestra.run.trial046:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-24T21:52:40.087 INFO:teuthology.orchestra.run.trial046.stdout:ttyS1 2026-02-24T21:52:40.119 DEBUG:teuthology.parallel:result is None 2026-02-24T21:52:40.120 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-24T21:52:40.133 INFO:teuthology.task.internal:Creating test directory... 2026-02-24T21:52:40.133 DEBUG:teuthology.orchestra.run.trial046:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-24T21:52:40.163 DEBUG:teuthology.orchestra.run.trial110:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-24T21:52:40.167 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-24T21:52:40.177 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-24T21:52:40.190 INFO:teuthology.task.internal:Creating archive directory... 2026-02-24T21:52:40.190 DEBUG:teuthology.orchestra.run.trial046:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-24T21:52:40.207 DEBUG:teuthology.orchestra.run.trial110:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-24T21:52:40.217 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-24T21:52:40.227 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-24T21:52:40.228 DEBUG:teuthology.orchestra.run.trial046:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-24T21:52:40.251 DEBUG:teuthology.orchestra.run.trial110:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-24T21:52:40.258 INFO:teuthology.orchestra.run.trial046.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T21:52:40.262 INFO:teuthology.orchestra.run.trial110.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T21:52:40.265 INFO:teuthology.orchestra.run.trial046.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T21:52:40.269 INFO:teuthology.orchestra.run.trial110.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T21:52:40.270 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-24T21:52:40.282 INFO:teuthology.task.internal:Configuring sudo... 2026-02-24T21:52:40.283 DEBUG:teuthology.orchestra.run.trial046:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-24T21:52:40.311 DEBUG:teuthology.orchestra.run.trial110:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-24T21:52:40.321 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-24T21:52:40.333 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-24T21:52:40.333 DEBUG:teuthology.orchestra.run.trial046:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-24T21:52:40.363 DEBUG:teuthology.orchestra.run.trial110:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-24T21:52:40.368 DEBUG:teuthology.orchestra.run.trial046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T21:52:40.410 DEBUG:teuthology.orchestra.run.trial046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T21:52:40.457 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T21:52:40.457 DEBUG:teuthology.orchestra.run.trial046:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-24T21:52:40.505 DEBUG:teuthology.orchestra.run.trial110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T21:52:40.509 DEBUG:teuthology.orchestra.run.trial110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T21:52:40.556 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T21:52:40.556 DEBUG:teuthology.orchestra.run.trial110:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-24T21:52:40.608 DEBUG:teuthology.orchestra.run.trial046:> sudo service rsyslog restart 2026-02-24T21:52:40.610 DEBUG:teuthology.orchestra.run.trial110:> sudo service rsyslog restart 2026-02-24T21:52:40.670 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-24T21:52:40.680 INFO:teuthology.task.internal:Starting timer... 2026-02-24T21:52:40.680 INFO:teuthology.run_tasks:Running task pcp... 2026-02-24T21:52:40.696 INFO:teuthology.run_tasks:Running task selinux... 2026-02-24T21:52:40.708 DEBUG:teuthology.task.selinux:Excluding trial046: OS 'ubuntu' does not support SELinux 2026-02-24T21:52:40.708 DEBUG:teuthology.task.selinux:Excluding trial110: OS 'ubuntu' does not support SELinux 2026-02-24T21:52:40.708 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-24T21:52:40.708 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2026-02-24T21:52:40.708 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-24T21:52:40.708 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-24T21:52:40.720 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-24T21:52:40.728 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-24T21:52:40.728 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial046.front.sepia.ceph.com,trial110.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-24T21:54:49.653 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial046.front.sepia.ceph.com'), Remote(name='ubuntu@trial110.front.sepia.ceph.com')] 2026-02-24T21:54:49.654 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial046.front.sepia.ceph.com' 2026-02-24T21:54:49.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T21:54:49.720 DEBUG:teuthology.orchestra.run.trial046:> true 2026-02-24T21:54:49.968 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial046.front.sepia.ceph.com' 2026-02-24T21:54:49.968 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial110.front.sepia.ceph.com' 2026-02-24T21:54:49.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T21:54:50.035 DEBUG:teuthology.orchestra.run.trial110:> true 2026-02-24T21:54:50.275 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial110.front.sepia.ceph.com' 2026-02-24T21:54:50.275 INFO:teuthology.run_tasks:Running task clock... 2026-02-24T21:54:50.288 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-24T21:54:50.289 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-24T21:54:50.289 DEBUG:teuthology.orchestra.run.trial046:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-24T21:54:50.290 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-24T21:54:50.290 DEBUG:teuthology.orchestra.run.trial110:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-24T21:54:50.309 INFO:teuthology.orchestra.run.trial046.stdout:24 Feb 21:54:50 ntpd[8820]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2026-02-24T21:54:50.309 INFO:teuthology.orchestra.run.trial046.stdout:24 Feb 21:54:50 ntpd[8820]: Command line: ntpd -gq 2026-02-24T21:54:50.309 INFO:teuthology.orchestra.run.trial046.stdout:24 Feb 21:54:50 ntpd[8820]: ---------------------------------------------------- 2026-02-24T21:54:50.309 INFO:teuthology.orchestra.run.trial046.stdout:24 Feb 21:54:50 ntpd[8820]: ntp-4 is maintained by Network Time Foundation, 2026-02-24T21:54:50.309 INFO:teuthology.orchestra.run.trial046.stdout:24 Feb 21:54:50 ntpd[8820]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2026-02-24T21:54:50.309 INFO:teuthology.orchestra.run.trial046.stdout:24 Feb 21:54:50 ntpd[8820]: corporation. Support and training for ntp-4 are 2026-02-24T21:54:50.309 INFO:teuthology.orchestra.run.trial046.stdout:24 Feb 21:54:50 ntpd[8820]: available at https://www.nwtime.org/support 2026-02-24T21:54:50.309 INFO:teuthology.orchestra.run.trial046.stdout:24 Feb 21:54:50 ntpd[8820]: ---------------------------------------------------- 2026-02-24T21:54:50.309 INFO:teuthology.orchestra.run.trial046.stdout:24 Feb 21:54:50 ntpd[8820]: proto: precision = 0.030 usec (-25) 2026-02-24T21:54:50.309 INFO:teuthology.orchestra.run.trial046.stdout:24 Feb 21:54:50 ntpd[8820]: basedate set to 2022-02-04 2026-02-24T21:54:50.309 INFO:teuthology.orchestra.run.trial046.stdout:24 Feb 21:54:50 ntpd[8820]: gps base set to 2022-02-06 (week 2196) 2026-02-24T21:54:50.309 INFO:teuthology.orchestra.run.trial046.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-24T21:54:50.309 INFO:teuthology.orchestra.run.trial046.stdout:24 Feb 21:54:50 ntpd[8820]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-24T21:54:50.309 INFO:teuthology.orchestra.run.trial046.stdout:24 Feb 21:54:50 ntpd[8820]: restrict ::: KOD does nothing without LIMITED. 2026-02-24T21:54:50.310 INFO:teuthology.orchestra.run.trial046.stderr:restrict ::: KOD does nothing without LIMITED. 2026-02-24T21:54:50.310 INFO:teuthology.orchestra.run.trial046.stdout:24 Feb 21:54:50 ntpd[8820]: Listen and drop on 0 v6wildcard [::]:123 2026-02-24T21:54:50.310 INFO:teuthology.orchestra.run.trial046.stdout:24 Feb 21:54:50 ntpd[8820]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2026-02-24T21:54:50.310 INFO:teuthology.orchestra.run.trial046.stdout:24 Feb 21:54:50 ntpd[8820]: Listen normally on 2 lo 127.0.0.1:123 2026-02-24T21:54:50.310 INFO:teuthology.orchestra.run.trial046.stdout:24 Feb 21:54:50 ntpd[8820]: Listen normally on 3 enp1s0f0 10.20.193.46:123 2026-02-24T21:54:50.310 INFO:teuthology.orchestra.run.trial046.stdout:24 Feb 21:54:50 ntpd[8820]: Listen normally on 4 lo [::1]:123 2026-02-24T21:54:50.310 INFO:teuthology.orchestra.run.trial046.stdout:24 Feb 21:54:50 ntpd[8820]: Listen normally on 5 enp1s0f0 [fe80::925a:8ff:fe77:606e%2]:123 2026-02-24T21:54:50.310 INFO:teuthology.orchestra.run.trial046.stdout:24 Feb 21:54:50 ntpd[8820]: Listening on routing socket on fd #22 for interface updates 2026-02-24T21:54:50.336 INFO:teuthology.orchestra.run.trial110.stdout:24 Feb 21:54:50 ntpd[8818]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2026-02-24T21:54:50.336 INFO:teuthology.orchestra.run.trial110.stdout:24 Feb 21:54:50 ntpd[8818]: Command line: ntpd -gq 2026-02-24T21:54:50.336 INFO:teuthology.orchestra.run.trial110.stdout:24 Feb 21:54:50 ntpd[8818]: ---------------------------------------------------- 2026-02-24T21:54:50.336 INFO:teuthology.orchestra.run.trial110.stdout:24 Feb 21:54:50 ntpd[8818]: ntp-4 is maintained by Network Time Foundation, 2026-02-24T21:54:50.336 INFO:teuthology.orchestra.run.trial110.stdout:24 Feb 21:54:50 ntpd[8818]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2026-02-24T21:54:50.336 INFO:teuthology.orchestra.run.trial110.stdout:24 Feb 21:54:50 ntpd[8818]: corporation. Support and training for ntp-4 are 2026-02-24T21:54:50.336 INFO:teuthology.orchestra.run.trial110.stdout:24 Feb 21:54:50 ntpd[8818]: available at https://www.nwtime.org/support 2026-02-24T21:54:50.336 INFO:teuthology.orchestra.run.trial110.stdout:24 Feb 21:54:50 ntpd[8818]: ---------------------------------------------------- 2026-02-24T21:54:50.336 INFO:teuthology.orchestra.run.trial110.stdout:24 Feb 21:54:50 ntpd[8818]: proto: precision = 0.030 usec (-25) 2026-02-24T21:54:50.336 INFO:teuthology.orchestra.run.trial110.stdout:24 Feb 21:54:50 ntpd[8818]: basedate set to 2022-02-04 2026-02-24T21:54:50.336 INFO:teuthology.orchestra.run.trial110.stdout:24 Feb 21:54:50 ntpd[8818]: gps base set to 2022-02-06 (week 2196) 2026-02-24T21:54:50.336 INFO:teuthology.orchestra.run.trial110.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-24T21:54:50.336 INFO:teuthology.orchestra.run.trial110.stdout:24 Feb 21:54:50 ntpd[8818]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-24T21:54:50.336 INFO:teuthology.orchestra.run.trial110.stdout:24 Feb 21:54:50 ntpd[8818]: restrict ::: KOD does nothing without LIMITED. 2026-02-24T21:54:50.337 INFO:teuthology.orchestra.run.trial110.stderr:restrict ::: KOD does nothing without LIMITED. 2026-02-24T21:54:50.337 INFO:teuthology.orchestra.run.trial110.stdout:24 Feb 21:54:50 ntpd[8818]: Listen and drop on 0 v6wildcard [::]:123 2026-02-24T21:54:50.337 INFO:teuthology.orchestra.run.trial110.stdout:24 Feb 21:54:50 ntpd[8818]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2026-02-24T21:54:50.337 INFO:teuthology.orchestra.run.trial110.stdout:24 Feb 21:54:50 ntpd[8818]: Listen normally on 2 lo 127.0.0.1:123 2026-02-24T21:54:50.337 INFO:teuthology.orchestra.run.trial110.stdout:24 Feb 21:54:50 ntpd[8818]: Listen normally on 3 enp1s0f0 10.20.193.110:123 2026-02-24T21:54:50.337 INFO:teuthology.orchestra.run.trial110.stdout:24 Feb 21:54:50 ntpd[8818]: Listen normally on 4 lo [::1]:123 2026-02-24T21:54:50.337 INFO:teuthology.orchestra.run.trial110.stdout:24 Feb 21:54:50 ntpd[8818]: Listen normally on 5 enp1s0f0 [fe80::925a:8ff:fe77:631c%2]:123 2026-02-24T21:54:50.337 INFO:teuthology.orchestra.run.trial110.stdout:24 Feb 21:54:50 ntpd[8818]: Listening on routing socket on fd #22 for interface updates 2026-02-24T21:54:51.309 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:51 ntpd[8820]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:51.309 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:51 ntpd[8820]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:51.322 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:51 ntpd[8820]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:51.322 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:51 ntpd[8820]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:51.322 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:51 ntpd[8820]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T21:54:51.322 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:51 ntpd[8820]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T21:54:51.336 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:51 ntpd[8818]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:51.336 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:51 ntpd[8818]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:51.344 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:51 ntpd[8818]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:51.345 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:51 ntpd[8818]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:51.345 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:51 ntpd[8818]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T21:54:51.345 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:51 ntpd[8818]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T21:54:52.336 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:52 ntpd[8818]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:52.336 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:52 ntpd[8818]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:52.336 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:52 ntpd[8818]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:52.336 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:52 ntpd[8818]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:52.339 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:52 ntpd[8818]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:52.339 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:52 ntpd[8818]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:52.340 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:52 ntpd[8818]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T21:54:52.340 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:52 ntpd[8818]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T21:54:52.425 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:52 ntpd[8818]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:52.425 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:52 ntpd[8818]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:52.425 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:52 ntpd[8818]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T21:54:52.425 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:52 ntpd[8818]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T21:54:53.309 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:53 ntpd[8820]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:53.309 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:53 ntpd[8820]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:53.309 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:53 ntpd[8820]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:53.309 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:53 ntpd[8820]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:53.322 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:53 ntpd[8820]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:53.322 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:53 ntpd[8820]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:53.322 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:53 ntpd[8820]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T21:54:53.322 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:53 ntpd[8820]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T21:54:53.336 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:53 ntpd[8818]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:53.336 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:53 ntpd[8818]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:53.344 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:53 ntpd[8818]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:53.344 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:53 ntpd[8818]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:53.344 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:53 ntpd[8818]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T21:54:53.344 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:53 ntpd[8818]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T21:54:53.381 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:53 ntpd[8820]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:53.381 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:53 ntpd[8820]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:53.381 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:53 ntpd[8820]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T21:54:53.381 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:53 ntpd[8820]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T21:54:54.309 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:54 ntpd[8820]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:54.309 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:54 ntpd[8820]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:54.336 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:54 ntpd[8818]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:54.336 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:54 ntpd[8818]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:54.336 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:54 ntpd[8818]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:54.336 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:54 ntpd[8818]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:54.339 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:54 ntpd[8818]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:54.339 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:54 ntpd[8818]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:54.339 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:54 ntpd[8818]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T21:54:54.339 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:54 ntpd[8818]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T21:54:54.425 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:54 ntpd[8818]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:54.425 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:54 ntpd[8818]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:54.425 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:54 ntpd[8818]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T21:54:54.425 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:54 ntpd[8818]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T21:54:55.309 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:55 ntpd[8820]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:55.309 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:55 ntpd[8820]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:55.309 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:55 ntpd[8820]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:55.310 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:55 ntpd[8820]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:55.310 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:55 ntpd[8820]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:55.310 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:55 ntpd[8820]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:55.322 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:55 ntpd[8820]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:55.322 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:55 ntpd[8820]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:55.322 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:55 ntpd[8820]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T21:54:55.322 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:55 ntpd[8820]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T21:54:55.336 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:55 ntpd[8818]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:55.336 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:55 ntpd[8818]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:55.336 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:55 ntpd[8818]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:55.336 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:55 ntpd[8818]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:55.344 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:55 ntpd[8818]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:55.344 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:55 ntpd[8818]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:55.344 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:55 ntpd[8818]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T21:54:55.344 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:55 ntpd[8818]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T21:54:55.347 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:55 ntpd[8820]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:55.347 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:55 ntpd[8820]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:55.347 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:55 ntpd[8820]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T21:54:55.347 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:55 ntpd[8820]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T21:54:55.378 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:55 ntpd[8818]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:55.378 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:55 ntpd[8818]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:55.378 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:55 ntpd[8818]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T21:54:55.378 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:55 ntpd[8818]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T21:54:55.381 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:55 ntpd[8820]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:55.381 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:55 ntpd[8820]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:55.381 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:55 ntpd[8820]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T21:54:55.381 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:55 ntpd[8820]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T21:54:56.336 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:56 ntpd[8818]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:56.336 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:56 ntpd[8818]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:56.336 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:56 ntpd[8818]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:56.336 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:56 ntpd[8818]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:56.339 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:56 ntpd[8818]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:56.339 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:56 ntpd[8818]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:56.339 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:56 ntpd[8818]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T21:54:56.339 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:56 ntpd[8818]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T21:54:56.425 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:56 ntpd[8818]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:56.425 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:56 ntpd[8818]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:56.425 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:56 ntpd[8818]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T21:54:56.425 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:56 ntpd[8818]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T21:54:57.309 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:57 ntpd[8820]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:57.309 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:57 ntpd[8820]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:57.309 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:57 ntpd[8820]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:57.309 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:57 ntpd[8820]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:57.309 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:57 ntpd[8820]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:57.309 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:57 ntpd[8820]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:57.322 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:57 ntpd[8820]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:57.322 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:57 ntpd[8820]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:57.322 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:57 ntpd[8820]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T21:54:57.322 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:57 ntpd[8820]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T21:54:57.322 INFO:teuthology.orchestra.run.trial046.stdout:24 Feb 21:54:57 ntpd[8820]: ntpd: time set +0.252215 s 2026-02-24T21:54:57.322 INFO:teuthology.orchestra.run.trial046.stdout:ntpd: time set +0.252215s 2026-02-24T21:54:57.322 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:57 ntpd[8820]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2026-02-24T21:54:57.323 INFO:teuthology.orchestra.run.trial046.stderr:24 Feb 21:54:57 ntpd[8820]: can't open /var/log/ntpstats/loopstats.20260224: Permission denied 2026-02-24T21:54:57.336 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:57 ntpd[8818]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:57.336 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:57 ntpd[8818]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:57.336 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:57 ntpd[8818]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:57.336 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:57 ntpd[8818]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:57.344 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:57 ntpd[8818]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T21:54:57.344 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:57 ntpd[8818]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T21:54:57.344 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:57 ntpd[8818]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T21:54:57.344 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:57 ntpd[8818]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T21:54:57.344 INFO:teuthology.orchestra.run.trial110.stdout:24 Feb 21:54:57 ntpd[8818]: ntpd: time set -0.330968 s 2026-02-24T21:54:57.344 INFO:teuthology.orchestra.run.trial110.stdout:ntpd: time set -0.330968s 2026-02-24T21:54:57.345 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:57 ntpd[8818]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2026-02-24T21:54:57.345 INFO:teuthology.orchestra.run.trial110.stderr:24 Feb 21:54:57 ntpd[8818]: can't open /var/log/ntpstats/loopstats.20260224: Permission denied 2026-02-24T21:54:57.357 INFO:teuthology.orchestra.run.trial046.stdout: remote refid st t when poll reach delay offset jitter 2026-02-24T21:54:57.358 INFO:teuthology.orchestra.run.trial046.stdout:============================================================================== 2026-02-24T21:54:57.358 INFO:teuthology.orchestra.run.trial046.stdout: time-h.den.code .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-24T21:54:57.407 INFO:teuthology.orchestra.run.trial110.stdout: remote refid st t when poll reach delay offset jitter 2026-02-24T21:54:57.407 INFO:teuthology.orchestra.run.trial110.stdout:============================================================================== 2026-02-24T21:54:57.407 INFO:teuthology.orchestra.run.trial110.stdout: time-h.den.code .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-24T21:54:57.407 INFO:teuthology.orchestra.run.trial110.stdout: ntp.maxhost.io .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-24T21:54:57.408 INFO:teuthology.orchestra.run.trial110.stdout: 198.137.202.32 .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-24T21:54:57.408 INFO:teuthology.orchestra.run.trial110.stdout: white.web-ster. .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-24T21:54:57.408 INFO:teuthology.run_tasks:Running task install... 2026-02-24T21:54:57.418 DEBUG:teuthology.task.install:project ceph 2026-02-24T21:54:57.418 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f7e05f4fdea5af173284f247d6b0935b337d4ccf'}} 2026-02-24T21:54:57.418 DEBUG:teuthology.task.install:config {'extra_system_packages': {'deb': ['s3cmd', 'maven', 'golang'], 'rpm': ['s3cmd', 'maven', 'golang']}, 'flavor': 'default', 'sha1': 'f7e05f4fdea5af173284f247d6b0935b337d4ccf'} 2026-02-24T21:54:57.419 INFO:teuthology.task.install:Using flavor: default 2026-02-24T21:54:57.423 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-24T21:54:57.423 INFO:teuthology.task.install:extra packages: [] 2026-02-24T21:54:57.423 DEBUG:teuthology.orchestra.run.trial046:> sudo apt-key list | grep Ceph 2026-02-24T21:54:57.424 DEBUG:teuthology.orchestra.run.trial110:> sudo apt-key list | grep Ceph 2026-02-24T21:54:57.474 INFO:teuthology.orchestra.run.trial046.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2026-02-24T21:54:57.495 INFO:teuthology.orchestra.run.trial110.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2026-02-24T21:54:57.504 INFO:teuthology.orchestra.run.trial046.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2026-02-24T21:54:57.505 INFO:teuthology.orchestra.run.trial046.stdout:uid [ unknown] Ceph.com (release key) 2026-02-24T21:54:57.505 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-osd, ceph-osd-classic, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2026-02-24T21:54:57.505 INFO:teuthology.task.install.deb:Installing system (non-project) packages: s3cmd, maven, golang on remote deb x86_64 2026-02-24T21:54:57.505 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=f7e05f4fdea5af173284f247d6b0935b337d4ccf 2026-02-24T21:54:57.524 INFO:teuthology.orchestra.run.trial110.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2026-02-24T21:54:57.524 INFO:teuthology.orchestra.run.trial110.stdout:uid [ unknown] Ceph.com (release key) 2026-02-24T21:54:57.524 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-osd, ceph-osd-classic, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2026-02-24T21:54:57.524 INFO:teuthology.task.install.deb:Installing system (non-project) packages: s3cmd, maven, golang on remote deb x86_64 2026-02-24T21:54:57.524 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=f7e05f4fdea5af173284f247d6b0935b337d4ccf 2026-02-24T21:54:57.680 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default/ 2026-02-24T21:54:57.680 INFO:teuthology.task.install.deb:Package version is 20.3.0-4957-gf7e05f4f-1jammy 2026-02-24T21:54:57.691 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default/ 2026-02-24T21:54:57.691 INFO:teuthology.task.install.deb:Package version is 20.3.0-4957-gf7e05f4f-1jammy 2026-02-24T21:54:58.019 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T21:54:58.019 DEBUG:teuthology.orchestra.run.trial046:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2026-02-24T21:54:58.029 DEBUG:teuthology.orchestra.run.trial046:> sudo apt-get update 2026-02-24T21:54:58.064 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T21:54:58.064 DEBUG:teuthology.orchestra.run.trial110:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2026-02-24T21:54:58.075 DEBUG:teuthology.orchestra.run.trial110:> sudo apt-get update 2026-02-24T21:54:58.256 INFO:teuthology.orchestra.run.trial110.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-24T21:54:58.261 INFO:teuthology.orchestra.run.trial110.stdout:Hit:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-24T21:54:58.271 INFO:teuthology.orchestra.run.trial110.stdout:Hit:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-24T21:54:58.280 INFO:teuthology.orchestra.run.trial110.stdout:Hit:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-24T21:54:58.391 INFO:teuthology.orchestra.run.trial046.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy InRelease 2026-02-24T21:54:58.432 INFO:teuthology.orchestra.run.trial110.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy InRelease 2026-02-24T21:54:58.460 INFO:teuthology.orchestra.run.trial046.stdout:Hit:2 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-24T21:54:58.460 INFO:teuthology.orchestra.run.trial046.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy Release [7689 B] 2026-02-24T21:54:58.494 INFO:teuthology.orchestra.run.trial110.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy Release [7689 B] 2026-02-24T21:54:58.522 INFO:teuthology.orchestra.run.trial046.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy Release.gpg 2026-02-24T21:54:58.527 INFO:teuthology.orchestra.run.trial046.stdout:Hit:5 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-24T21:54:58.562 INFO:teuthology.orchestra.run.trial110.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy Release.gpg 2026-02-24T21:54:58.591 INFO:teuthology.orchestra.run.trial046.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 Packages [19.2 kB] 2026-02-24T21:54:58.599 INFO:teuthology.orchestra.run.trial046.stdout:Hit:7 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-24T21:54:58.630 INFO:teuthology.orchestra.run.trial110.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 Packages [19.2 kB] 2026-02-24T21:54:58.671 INFO:teuthology.orchestra.run.trial046.stdout:Hit:8 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-24T21:54:58.796 INFO:teuthology.orchestra.run.trial110.stdout:Fetched 26.9 kB in 1s (48.7 kB/s) 2026-02-24T21:54:58.815 INFO:teuthology.orchestra.run.trial046.stdout:Fetched 26.9 kB in 1s (43.4 kB/s) 2026-02-24T21:54:59.324 INFO:teuthology.orchestra.run.trial110.stdout:Reading package lists... 2026-02-24T21:54:59.337 DEBUG:teuthology.orchestra.run.trial110:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=20.3.0-4957-gf7e05f4f-1jammy cephadm=20.3.0-4957-gf7e05f4f-1jammy ceph-mds=20.3.0-4957-gf7e05f4f-1jammy ceph-mgr=20.3.0-4957-gf7e05f4f-1jammy ceph-osd=20.3.0-4957-gf7e05f4f-1jammy ceph-osd-classic=20.3.0-4957-gf7e05f4f-1jammy ceph-common=20.3.0-4957-gf7e05f4f-1jammy ceph-fuse=20.3.0-4957-gf7e05f4f-1jammy ceph-test=20.3.0-4957-gf7e05f4f-1jammy ceph-volume=20.3.0-4957-gf7e05f4f-1jammy radosgw=20.3.0-4957-gf7e05f4f-1jammy python3-rados=20.3.0-4957-gf7e05f4f-1jammy python3-rgw=20.3.0-4957-gf7e05f4f-1jammy python3-cephfs=20.3.0-4957-gf7e05f4f-1jammy python3-rbd=20.3.0-4957-gf7e05f4f-1jammy libcephfs2=20.3.0-4957-gf7e05f4f-1jammy libcephfs-dev=20.3.0-4957-gf7e05f4f-1jammy librados2=20.3.0-4957-gf7e05f4f-1jammy librbd1=20.3.0-4957-gf7e05f4f-1jammy rbd-fuse=20.3.0-4957-gf7e05f4f-1jammy 2026-02-24T21:54:59.339 INFO:teuthology.orchestra.run.trial046.stdout:Reading package lists... 2026-02-24T21:54:59.353 DEBUG:teuthology.orchestra.run.trial046:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=20.3.0-4957-gf7e05f4f-1jammy cephadm=20.3.0-4957-gf7e05f4f-1jammy ceph-mds=20.3.0-4957-gf7e05f4f-1jammy ceph-mgr=20.3.0-4957-gf7e05f4f-1jammy ceph-osd=20.3.0-4957-gf7e05f4f-1jammy ceph-osd-classic=20.3.0-4957-gf7e05f4f-1jammy ceph-common=20.3.0-4957-gf7e05f4f-1jammy ceph-fuse=20.3.0-4957-gf7e05f4f-1jammy ceph-test=20.3.0-4957-gf7e05f4f-1jammy ceph-volume=20.3.0-4957-gf7e05f4f-1jammy radosgw=20.3.0-4957-gf7e05f4f-1jammy python3-rados=20.3.0-4957-gf7e05f4f-1jammy python3-rgw=20.3.0-4957-gf7e05f4f-1jammy python3-cephfs=20.3.0-4957-gf7e05f4f-1jammy python3-rbd=20.3.0-4957-gf7e05f4f-1jammy libcephfs2=20.3.0-4957-gf7e05f4f-1jammy libcephfs-dev=20.3.0-4957-gf7e05f4f-1jammy librados2=20.3.0-4957-gf7e05f4f-1jammy librbd1=20.3.0-4957-gf7e05f4f-1jammy rbd-fuse=20.3.0-4957-gf7e05f4f-1jammy 2026-02-24T21:54:59.370 INFO:teuthology.orchestra.run.trial110.stdout:Reading package lists... 2026-02-24T21:54:59.385 INFO:teuthology.orchestra.run.trial046.stdout:Reading package lists... 2026-02-24T21:54:59.480 INFO:teuthology.orchestra.run.trial110.stdout:Building dependency tree... 2026-02-24T21:54:59.481 INFO:teuthology.orchestra.run.trial110.stdout:Reading state information... 2026-02-24T21:54:59.492 INFO:teuthology.orchestra.run.trial046.stdout:Building dependency tree... 2026-02-24T21:54:59.492 INFO:teuthology.orchestra.run.trial046.stdout:Reading state information... 2026-02-24T21:54:59.570 INFO:teuthology.orchestra.run.trial110.stdout:The following packages were automatically installed and are no longer required: 2026-02-24T21:54:59.570 INFO:teuthology.orchestra.run.trial110.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-24T21:54:59.570 INFO:teuthology.orchestra.run.trial110.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 2026-02-24T21:54:59.570 INFO:teuthology.orchestra.run.trial110.stdout: linux-headers-5.15.0-164 linux-headers-5.15.0-164-generic 2026-02-24T21:54:59.570 INFO:teuthology.orchestra.run.trial110.stdout: linux-image-5.15.0-164-generic linux-modules-5.15.0-164-generic 2026-02-24T21:54:59.570 INFO:teuthology.orchestra.run.trial110.stdout: linux-modules-extra-5.15.0-164-generic sg3-utils sg3-utils-udev 2026-02-24T21:54:59.570 INFO:teuthology.orchestra.run.trial110.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-24T21:54:59.570 INFO:teuthology.orchestra.run.trial110.stdout:The following additional packages will be installed: 2026-02-24T21:54:59.570 INFO:teuthology.orchestra.run.trial110.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2026-02-24T21:54:59.570 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon jq libcephfs-daemon 2026-02-24T21:54:59.570 INFO:teuthology.orchestra.run.trial110.stdout: libcephfs-proxy2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 libnbd0 2026-02-24T21:54:59.570 INFO:teuthology.orchestra.run.trial110.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-24T21:54:59.570 INFO:teuthology.orchestra.run.trial110.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2026-02-24T21:54:59.570 INFO:teuthology.orchestra.run.trial110.stdout: nvme-cli python-asyncssh-doc python3-asyncssh python3-cachetools 2026-02-24T21:54:59.570 INFO:teuthology.orchestra.run.trial110.stdout: python3-ceph-argparse python3-ceph-common python3-cheroot python3-cherrypy3 2026-02-24T21:54:59.570 INFO:teuthology.orchestra.run.trial110.stdout: python3-google-auth python3-iniconfig python3-jaraco.classes 2026-02-24T21:54:59.570 INFO:teuthology.orchestra.run.trial110.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2026-02-24T21:54:59.570 INFO:teuthology.orchestra.run.trial110.stdout: python3-joblib python3-kubernetes python3-natsort python3-pluggy 2026-02-24T21:54:59.570 INFO:teuthology.orchestra.run.trial110.stdout: python3-portend python3-prettytable python3-psutil python3-py 2026-02-24T21:54:59.570 INFO:teuthology.orchestra.run.trial110.stdout: python3-pygments python3-pytest python3-repoze.lru python3-requests-oauthlib 2026-02-24T21:54:59.570 INFO:teuthology.orchestra.run.trial110.stdout: python3-routes python3-rsa python3-simplejson python3-sklearn 2026-02-24T21:54:59.570 INFO:teuthology.orchestra.run.trial110.stdout: python3-sklearn-lib python3-tempora python3-threadpoolctl python3-toml 2026-02-24T21:54:59.571 INFO:teuthology.orchestra.run.trial110.stdout: python3-wcwidth python3-webob python3-websocket python3-zc.lockfile 2026-02-24T21:54:59.571 INFO:teuthology.orchestra.run.trial110.stdout: qttranslations5-l10n socat xmlstarlet 2026-02-24T21:54:59.571 INFO:teuthology.orchestra.run.trial110.stdout:Suggested packages: 2026-02-24T21:54:59.571 INFO:teuthology.orchestra.run.trial110.stdout: python3-influxdb liblua5.3-dev luarocks python-natsort-doc python-psutil-doc 2026-02-24T21:54:59.571 INFO:teuthology.orchestra.run.trial110.stdout: subversion python-pygments-doc ttf-bitstream-vera python3-paste python3-dap 2026-02-24T21:54:59.571 INFO:teuthology.orchestra.run.trial110.stdout: python-sklearn-doc ipython3 python-webob-doc 2026-02-24T21:54:59.571 INFO:teuthology.orchestra.run.trial110.stdout:Recommended packages: 2026-02-24T21:54:59.571 INFO:teuthology.orchestra.run.trial110.stdout: btrfs-tools 2026-02-24T21:54:59.581 INFO:teuthology.orchestra.run.trial046.stdout:The following packages were automatically installed and are no longer required: 2026-02-24T21:54:59.581 INFO:teuthology.orchestra.run.trial046.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-24T21:54:59.581 INFO:teuthology.orchestra.run.trial046.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 2026-02-24T21:54:59.581 INFO:teuthology.orchestra.run.trial046.stdout: linux-headers-5.15.0-164 linux-headers-5.15.0-164-generic 2026-02-24T21:54:59.581 INFO:teuthology.orchestra.run.trial046.stdout: linux-image-5.15.0-164-generic linux-modules-5.15.0-164-generic 2026-02-24T21:54:59.581 INFO:teuthology.orchestra.run.trial046.stdout: linux-modules-extra-5.15.0-164-generic sg3-utils sg3-utils-udev 2026-02-24T21:54:59.581 INFO:teuthology.orchestra.run.trial046.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-24T21:54:59.582 INFO:teuthology.orchestra.run.trial046.stdout:The following additional packages will be installed: 2026-02-24T21:54:59.582 INFO:teuthology.orchestra.run.trial046.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2026-02-24T21:54:59.582 INFO:teuthology.orchestra.run.trial046.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon jq libcephfs-daemon 2026-02-24T21:54:59.582 INFO:teuthology.orchestra.run.trial046.stdout: libcephfs-proxy2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 libnbd0 2026-02-24T21:54:59.582 INFO:teuthology.orchestra.run.trial046.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-24T21:54:59.582 INFO:teuthology.orchestra.run.trial046.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2026-02-24T21:54:59.582 INFO:teuthology.orchestra.run.trial046.stdout: nvme-cli python-asyncssh-doc python3-asyncssh python3-cachetools 2026-02-24T21:54:59.582 INFO:teuthology.orchestra.run.trial046.stdout: python3-ceph-argparse python3-ceph-common python3-cheroot python3-cherrypy3 2026-02-24T21:54:59.582 INFO:teuthology.orchestra.run.trial046.stdout: python3-google-auth python3-iniconfig python3-jaraco.classes 2026-02-24T21:54:59.582 INFO:teuthology.orchestra.run.trial046.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2026-02-24T21:54:59.582 INFO:teuthology.orchestra.run.trial046.stdout: python3-joblib python3-kubernetes python3-natsort python3-pluggy 2026-02-24T21:54:59.582 INFO:teuthology.orchestra.run.trial046.stdout: python3-portend python3-prettytable python3-psutil python3-py 2026-02-24T21:54:59.582 INFO:teuthology.orchestra.run.trial046.stdout: python3-pygments python3-pytest python3-repoze.lru python3-requests-oauthlib 2026-02-24T21:54:59.582 INFO:teuthology.orchestra.run.trial046.stdout: python3-routes python3-rsa python3-simplejson python3-sklearn 2026-02-24T21:54:59.582 INFO:teuthology.orchestra.run.trial046.stdout: python3-sklearn-lib python3-tempora python3-threadpoolctl python3-toml 2026-02-24T21:54:59.582 INFO:teuthology.orchestra.run.trial046.stdout: python3-wcwidth python3-webob python3-websocket python3-zc.lockfile 2026-02-24T21:54:59.582 INFO:teuthology.orchestra.run.trial046.stdout: qttranslations5-l10n socat xmlstarlet 2026-02-24T21:54:59.582 INFO:teuthology.orchestra.run.trial046.stdout:Suggested packages: 2026-02-24T21:54:59.582 INFO:teuthology.orchestra.run.trial046.stdout: python3-influxdb liblua5.3-dev luarocks python-natsort-doc python-psutil-doc 2026-02-24T21:54:59.582 INFO:teuthology.orchestra.run.trial046.stdout: subversion python-pygments-doc ttf-bitstream-vera python3-paste python3-dap 2026-02-24T21:54:59.582 INFO:teuthology.orchestra.run.trial046.stdout: python-sklearn-doc ipython3 python-webob-doc 2026-02-24T21:54:59.582 INFO:teuthology.orchestra.run.trial046.stdout:Recommended packages: 2026-02-24T21:54:59.582 INFO:teuthology.orchestra.run.trial046.stdout: btrfs-tools 2026-02-24T21:54:59.603 INFO:teuthology.orchestra.run.trial110.stdout:The following NEW packages will be installed: 2026-02-24T21:54:59.603 INFO:teuthology.orchestra.run.trial110.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2026-02-24T21:54:59.604 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2026-02-24T21:54:59.604 INFO:teuthology.orchestra.run.trial110.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-osd-classic ceph-test 2026-02-24T21:54:59.604 INFO:teuthology.orchestra.run.trial110.stdout: ceph-volume cephadm jq libcephfs-daemon libcephfs-dev libcephfs-proxy2 2026-02-24T21:54:59.604 INFO:teuthology.orchestra.run.trial110.stdout: libcephfs2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 libnbd0 2026-02-24T21:54:59.604 INFO:teuthology.orchestra.run.trial110.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-24T21:54:59.604 INFO:teuthology.orchestra.run.trial110.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2026-02-24T21:54:59.604 INFO:teuthology.orchestra.run.trial110.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python3-asyncssh 2026-02-24T21:54:59.604 INFO:teuthology.orchestra.run.trial110.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2026-02-24T21:54:59.604 INFO:teuthology.orchestra.run.trial110.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-24T21:54:59.604 INFO:teuthology.orchestra.run.trial110.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-24T21:54:59.604 INFO:teuthology.orchestra.run.trial110.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-natsort 2026-02-24T21:54:59.604 INFO:teuthology.orchestra.run.trial110.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2026-02-24T21:54:59.604 INFO:teuthology.orchestra.run.trial110.stdout: python3-pygments python3-pytest python3-rados python3-rbd python3-repoze.lru 2026-02-24T21:54:59.604 INFO:teuthology.orchestra.run.trial110.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2026-02-24T21:54:59.604 INFO:teuthology.orchestra.run.trial110.stdout: python3-simplejson python3-sklearn python3-sklearn-lib python3-tempora 2026-02-24T21:54:59.604 INFO:teuthology.orchestra.run.trial110.stdout: python3-threadpoolctl python3-toml python3-wcwidth python3-webob 2026-02-24T21:54:59.604 INFO:teuthology.orchestra.run.trial110.stdout: python3-websocket python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2026-02-24T21:54:59.604 INFO:teuthology.orchestra.run.trial110.stdout: socat xmlstarlet 2026-02-24T21:54:59.617 INFO:teuthology.orchestra.run.trial046.stdout:The following NEW packages will be installed: 2026-02-24T21:54:59.617 INFO:teuthology.orchestra.run.trial046.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2026-02-24T21:54:59.617 INFO:teuthology.orchestra.run.trial046.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2026-02-24T21:54:59.617 INFO:teuthology.orchestra.run.trial046.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-osd-classic ceph-test 2026-02-24T21:54:59.617 INFO:teuthology.orchestra.run.trial046.stdout: ceph-volume cephadm jq libcephfs-daemon libcephfs-dev libcephfs-proxy2 2026-02-24T21:54:59.617 INFO:teuthology.orchestra.run.trial046.stdout: libcephfs2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 libnbd0 2026-02-24T21:54:59.617 INFO:teuthology.orchestra.run.trial046.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-24T21:54:59.617 INFO:teuthology.orchestra.run.trial046.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2026-02-24T21:54:59.617 INFO:teuthology.orchestra.run.trial046.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python3-asyncssh 2026-02-24T21:54:59.617 INFO:teuthology.orchestra.run.trial046.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2026-02-24T21:54:59.617 INFO:teuthology.orchestra.run.trial046.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-24T21:54:59.617 INFO:teuthology.orchestra.run.trial046.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-24T21:54:59.617 INFO:teuthology.orchestra.run.trial046.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-natsort 2026-02-24T21:54:59.617 INFO:teuthology.orchestra.run.trial046.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2026-02-24T21:54:59.617 INFO:teuthology.orchestra.run.trial046.stdout: python3-pygments python3-pytest python3-rados python3-rbd python3-repoze.lru 2026-02-24T21:54:59.617 INFO:teuthology.orchestra.run.trial046.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2026-02-24T21:54:59.617 INFO:teuthology.orchestra.run.trial046.stdout: python3-simplejson python3-sklearn python3-sklearn-lib python3-tempora 2026-02-24T21:54:59.617 INFO:teuthology.orchestra.run.trial046.stdout: python3-threadpoolctl python3-toml python3-wcwidth python3-webob 2026-02-24T21:54:59.617 INFO:teuthology.orchestra.run.trial046.stdout: python3-websocket python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2026-02-24T21:54:59.617 INFO:teuthology.orchestra.run.trial046.stdout: socat xmlstarlet 2026-02-24T21:54:59.974 INFO:teuthology.orchestra.run.trial110.stdout:0 upgraded, 87 newly installed, 0 to remove and 14 not upgraded. 2026-02-24T21:54:59.974 INFO:teuthology.orchestra.run.trial110.stdout:Need to get 695 MB of archives. 2026-02-24T21:54:59.974 INFO:teuthology.orchestra.run.trial110.stdout:After this operation, 2587 MB of additional disk space will be used. 2026-02-24T21:54:59.974 INFO:teuthology.orchestra.run.trial110.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 20.3.0-4957-gf7e05f4f-1jammy [3793 kB] 2026-02-24T21:54:59.976 INFO:teuthology.orchestra.run.trial046.stdout:0 upgraded, 87 newly installed, 0 to remove and 14 not upgraded. 2026-02-24T21:54:59.976 INFO:teuthology.orchestra.run.trial046.stdout:Need to get 695 MB of archives. 2026-02-24T21:54:59.976 INFO:teuthology.orchestra.run.trial046.stdout:After this operation, 2587 MB of additional disk space will be used. 2026-02-24T21:54:59.976 INFO:teuthology.orchestra.run.trial046.stdout:Get:1 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2026-02-24T21:54:59.989 INFO:teuthology.orchestra.run.trial046.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 20.3.0-4957-gf7e05f4f-1jammy [3793 kB] 2026-02-24T21:55:00.017 INFO:teuthology.orchestra.run.trial110.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2026-02-24T21:55:00.312 INFO:teuthology.orchestra.run.trial046.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2026-02-24T21:55:00.322 INFO:teuthology.orchestra.run.trial046.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2026-02-24T21:55:00.393 INFO:teuthology.orchestra.run.trial046.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2006 kB] 2026-02-24T21:55:00.432 INFO:teuthology.orchestra.run.trial110.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2026-02-24T21:55:00.445 INFO:teuthology.orchestra.run.trial110.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2026-02-24T21:55:00.531 INFO:teuthology.orchestra.run.trial110.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2006 kB] 2026-02-24T21:55:00.594 INFO:teuthology.orchestra.run.trial046.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2026-02-24T21:55:00.605 INFO:teuthology.orchestra.run.trial046.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2026-02-24T21:55:00.631 INFO:teuthology.orchestra.run.trial046.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2026-02-24T21:55:00.639 INFO:teuthology.orchestra.run.trial046.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2026-02-24T21:55:00.641 INFO:teuthology.orchestra.run.trial046.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2026-02-24T21:55:00.642 INFO:teuthology.orchestra.run.trial046.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2026-02-24T21:55:00.643 INFO:teuthology.orchestra.run.trial046.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2026-02-24T21:55:00.662 INFO:teuthology.orchestra.run.trial046.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 liboath0 amd64 2.6.7-3ubuntu0.1 [41.3 kB] 2026-02-24T21:55:00.662 INFO:teuthology.orchestra.run.trial046.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9030 B] 2026-02-24T21:55:00.667 INFO:teuthology.orchestra.run.trial046.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2026-02-24T21:55:00.695 INFO:teuthology.orchestra.run.trial110.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 20.3.0-4957-gf7e05f4f-1jammy [2910 kB] 2026-02-24T21:55:00.735 INFO:teuthology.orchestra.run.trial046.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6452 B] 2026-02-24T21:55:00.736 INFO:teuthology.orchestra.run.trial046.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8716 B] 2026-02-24T21:55:00.737 INFO:teuthology.orchestra.run.trial046.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2026-02-24T21:55:00.737 INFO:teuthology.orchestra.run.trial046.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2026-02-24T21:55:00.738 INFO:teuthology.orchestra.run.trial046.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7240 B] 2026-02-24T21:55:00.738 INFO:teuthology.orchestra.run.trial046.stdout:Get:21 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8980 B] 2026-02-24T21:55:00.739 INFO:teuthology.orchestra.run.trial046.stdout:Get:22 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2026-02-24T21:55:00.742 INFO:teuthology.orchestra.run.trial046.stdout:Get:23 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2026-02-24T21:55:00.742 INFO:teuthology.orchestra.run.trial046.stdout:Get:24 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libfuse2 amd64 2.9.9-5ubuntu3 [90.3 kB] 2026-02-24T21:55:00.776 INFO:teuthology.orchestra.run.trial110.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2026-02-24T21:55:00.789 INFO:teuthology.orchestra.run.trial110.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2026-02-24T21:55:00.813 INFO:teuthology.orchestra.run.trial046.stdout:Get:25 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python3-asyncssh all 2.5.0-1ubuntu0.1 [189 kB] 2026-02-24T21:55:00.816 INFO:teuthology.orchestra.run.trial046.stdout:Get:26 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2026-02-24T21:55:00.816 INFO:teuthology.orchestra.run.trial046.stdout:Get:27 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2026-02-24T21:55:00.819 INFO:teuthology.orchestra.run.trial046.stdout:Get:28 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2058 kB] 2026-02-24T21:55:00.821 INFO:teuthology.orchestra.run.trial110.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2026-02-24T21:55:00.831 INFO:teuthology.orchestra.run.trial110.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2026-02-24T21:55:00.834 INFO:teuthology.orchestra.run.trial110.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2026-02-24T21:55:00.835 INFO:teuthology.orchestra.run.trial110.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2026-02-24T21:55:00.836 INFO:teuthology.orchestra.run.trial110.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2026-02-24T21:55:00.844 INFO:teuthology.orchestra.run.trial046.stdout:Get:29 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2026-02-24T21:55:00.846 INFO:teuthology.orchestra.run.trial046.stdout:Get:30 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2026-02-24T21:55:00.856 INFO:teuthology.orchestra.run.trial110.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 liboath0 amd64 2.6.7-3ubuntu0.1 [41.3 kB] 2026-02-24T21:55:00.857 INFO:teuthology.orchestra.run.trial110.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9030 B] 2026-02-24T21:55:00.865 INFO:teuthology.orchestra.run.trial110.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2026-02-24T21:55:00.885 INFO:teuthology.orchestra.run.trial046.stdout:Get:31 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1829 kB] 2026-02-24T21:55:00.908 INFO:teuthology.orchestra.run.trial046.stdout:Get:32 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9722 B] 2026-02-24T21:55:00.908 INFO:teuthology.orchestra.run.trial046.stdout:Get:33 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2026-02-24T21:55:00.908 INFO:teuthology.orchestra.run.trial046.stdout:Get:34 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2026-02-24T21:55:00.946 INFO:teuthology.orchestra.run.trial110.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6452 B] 2026-02-24T21:55:00.946 INFO:teuthology.orchestra.run.trial110.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8716 B] 2026-02-24T21:55:00.946 INFO:teuthology.orchestra.run.trial110.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2026-02-24T21:55:00.947 INFO:teuthology.orchestra.run.trial110.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2026-02-24T21:55:00.947 INFO:teuthology.orchestra.run.trial110.stdout:Get:21 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7240 B] 2026-02-24T21:55:00.947 INFO:teuthology.orchestra.run.trial110.stdout:Get:22 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8980 B] 2026-02-24T21:55:00.947 INFO:teuthology.orchestra.run.trial110.stdout:Get:23 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2026-02-24T21:55:00.953 INFO:teuthology.orchestra.run.trial110.stdout:Get:24 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2026-02-24T21:55:00.954 INFO:teuthology.orchestra.run.trial110.stdout:Get:25 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libfuse2 amd64 2.9.9-5ubuntu3 [90.3 kB] 2026-02-24T21:55:00.958 INFO:teuthology.orchestra.run.trial046.stdout:Get:35 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2026-02-24T21:55:00.958 INFO:teuthology.orchestra.run.trial046.stdout:Get:36 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2026-02-24T21:55:00.961 INFO:teuthology.orchestra.run.trial046.stdout:Get:37 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2026-02-24T21:55:00.968 INFO:teuthology.orchestra.run.trial046.stdout:Get:38 https://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2026-02-24T21:55:00.970 INFO:teuthology.orchestra.run.trial046.stdout:Get:39 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libjq1 amd64 1.6-2.1ubuntu3.1 [133 kB] 2026-02-24T21:55:00.971 INFO:teuthology.orchestra.run.trial046.stdout:Get:40 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 jq amd64 1.6-2.1ubuntu3.1 [52.5 kB] 2026-02-24T21:55:01.032 INFO:teuthology.orchestra.run.trial046.stdout:Get:41 https://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2026-02-24T21:55:01.037 INFO:teuthology.orchestra.run.trial046.stdout:Get:42 https://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2026-02-24T21:55:01.038 INFO:teuthology.orchestra.run.trial110.stdout:Get:26 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python3-asyncssh all 2.5.0-1ubuntu0.1 [189 kB] 2026-02-24T21:55:01.039 INFO:teuthology.orchestra.run.trial046.stdout:Get:43 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2026-02-24T21:55:01.041 INFO:teuthology.orchestra.run.trial110.stdout:Get:27 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2026-02-24T21:55:01.041 INFO:teuthology.orchestra.run.trial110.stdout:Get:28 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2026-02-24T21:55:01.042 INFO:teuthology.orchestra.run.trial110.stdout:Get:29 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2058 kB] 2026-02-24T21:55:01.044 INFO:teuthology.orchestra.run.trial046.stdout:Get:44 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python-asyncssh-doc all 2.5.0-1ubuntu0.1 [309 kB] 2026-02-24T21:55:01.072 INFO:teuthology.orchestra.run.trial110.stdout:Get:30 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 20.3.0-4957-gf7e05f4f-1jammy [882 kB] 2026-02-24T21:55:01.072 INFO:teuthology.orchestra.run.trial110.stdout:Get:31 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2026-02-24T21:55:01.074 INFO:teuthology.orchestra.run.trial110.stdout:Get:32 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2026-02-24T21:55:01.075 INFO:teuthology.orchestra.run.trial110.stdout:Get:33 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1829 kB] 2026-02-24T21:55:01.104 INFO:teuthology.orchestra.run.trial046.stdout:Get:45 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6024 B] 2026-02-24T21:55:01.104 INFO:teuthology.orchestra.run.trial046.stdout:Get:46 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2026-02-24T21:55:01.105 INFO:teuthology.orchestra.run.trial046.stdout:Get:47 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2026-02-24T21:55:01.107 INFO:teuthology.orchestra.run.trial046.stdout:Get:48 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2026-02-24T21:55:01.108 INFO:teuthology.orchestra.run.trial046.stdout:Get:49 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-pygments all 2.11.2+dfsg-2ubuntu0.1 [750 kB] 2026-02-24T21:55:01.115 INFO:teuthology.orchestra.run.trial046.stdout:Get:50 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2026-02-24T21:55:01.146 INFO:teuthology.orchestra.run.trial110.stdout:Get:34 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9722 B] 2026-02-24T21:55:01.146 INFO:teuthology.orchestra.run.trial110.stdout:Get:35 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2026-02-24T21:55:01.146 INFO:teuthology.orchestra.run.trial110.stdout:Get:36 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2026-02-24T21:55:01.147 INFO:teuthology.orchestra.run.trial110.stdout:Get:37 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2026-02-24T21:55:01.147 INFO:teuthology.orchestra.run.trial110.stdout:Get:38 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2026-02-24T21:55:01.147 INFO:teuthology.orchestra.run.trial110.stdout:Get:39 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2026-02-24T21:55:01.176 INFO:teuthology.orchestra.run.trial046.stdout:Get:51 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2026-02-24T21:55:01.178 INFO:teuthology.orchestra.run.trial046.stdout:Get:52 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2026-02-24T21:55:01.181 INFO:teuthology.orchestra.run.trial046.stdout:Get:53 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2026-02-24T21:55:01.182 INFO:teuthology.orchestra.run.trial046.stdout:Get:54 https://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1983 kB] 2026-02-24T21:55:01.196 INFO:teuthology.orchestra.run.trial110.stdout:Get:40 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 20.3.0-4957-gf7e05f4f-1jammy [354 kB] 2026-02-24T21:55:01.206 INFO:teuthology.orchestra.run.trial110.stdout:Get:41 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 20.3.0-4957-gf7e05f4f-1jammy [32.8 kB] 2026-02-24T21:55:01.220 INFO:teuthology.orchestra.run.trial110.stdout:Get:42 https://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2026-02-24T21:55:01.222 INFO:teuthology.orchestra.run.trial110.stdout:Get:43 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libjq1 amd64 1.6-2.1ubuntu3.1 [133 kB] 2026-02-24T21:55:01.223 INFO:teuthology.orchestra.run.trial110.stdout:Get:44 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 jq amd64 1.6-2.1ubuntu3.1 [52.5 kB] 2026-02-24T21:55:01.262 INFO:teuthology.orchestra.run.trial110.stdout:Get:45 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 20.3.0-4957-gf7e05f4f-1jammy [224 kB] 2026-02-24T21:55:01.264 INFO:teuthology.orchestra.run.trial110.stdout:Get:46 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 20.3.0-4957-gf7e05f4f-1jammy [87.6 kB] 2026-02-24T21:55:01.267 INFO:teuthology.orchestra.run.trial110.stdout:Get:47 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 20.3.0-4957-gf7e05f4f-1jammy [329 kB] 2026-02-24T21:55:01.308 INFO:teuthology.orchestra.run.trial110.stdout:Get:48 https://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2026-02-24T21:55:01.312 INFO:teuthology.orchestra.run.trial110.stdout:Get:49 https://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2026-02-24T21:55:01.314 INFO:teuthology.orchestra.run.trial110.stdout:Get:50 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2026-02-24T21:55:01.319 INFO:teuthology.orchestra.run.trial110.stdout:Get:51 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python-asyncssh-doc all 2.5.0-1ubuntu0.1 [309 kB] 2026-02-24T21:55:01.322 INFO:teuthology.orchestra.run.trial110.stdout:Get:52 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6024 B] 2026-02-24T21:55:01.322 INFO:teuthology.orchestra.run.trial110.stdout:Get:53 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2026-02-24T21:55:01.322 INFO:teuthology.orchestra.run.trial110.stdout:Get:54 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2026-02-24T21:55:01.330 INFO:teuthology.orchestra.run.trial110.stdout:Get:55 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 20.3.0-4957-gf7e05f4f-1jammy [9090 kB] 2026-02-24T21:55:01.397 INFO:teuthology.orchestra.run.trial110.stdout:Get:56 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2026-02-24T21:55:01.397 INFO:teuthology.orchestra.run.trial110.stdout:Get:57 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-pygments all 2.11.2+dfsg-2ubuntu0.1 [750 kB] 2026-02-24T21:55:01.405 INFO:teuthology.orchestra.run.trial110.stdout:Get:58 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2026-02-24T21:55:01.485 INFO:teuthology.orchestra.run.trial110.stdout:Get:59 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2026-02-24T21:55:01.487 INFO:teuthology.orchestra.run.trial110.stdout:Get:60 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2026-02-24T21:55:01.487 INFO:teuthology.orchestra.run.trial110.stdout:Get:61 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2026-02-24T21:55:01.488 INFO:teuthology.orchestra.run.trial110.stdout:Get:62 https://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1983 kB] 2026-02-24T21:55:02.499 INFO:teuthology.orchestra.run.trial110.stdout:Get:63 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 20.3.0-4957-gf7e05f4f-1jammy [111 kB] 2026-02-24T21:55:02.501 INFO:teuthology.orchestra.run.trial110.stdout:Get:64 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 20.3.0-4957-gf7e05f4f-1jammy [261 kB] 2026-02-24T21:55:02.557 INFO:teuthology.orchestra.run.trial110.stdout:Get:65 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 20.3.0-4957-gf7e05f4f-1jammy [31.5 MB] 2026-02-24T21:55:02.916 INFO:teuthology.orchestra.run.trial046.stdout:Get:55 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 20.3.0-4957-gf7e05f4f-1jammy [2910 kB] 2026-02-24T21:55:05.104 INFO:teuthology.orchestra.run.trial046.stdout:Get:56 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 20.3.0-4957-gf7e05f4f-1jammy [882 kB] 2026-02-24T21:55:05.698 INFO:teuthology.orchestra.run.trial046.stdout:Get:57 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 20.3.0-4957-gf7e05f4f-1jammy [354 kB] 2026-02-24T21:55:05.925 INFO:teuthology.orchestra.run.trial046.stdout:Get:58 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 20.3.0-4957-gf7e05f4f-1jammy [32.8 kB] 2026-02-24T21:55:05.933 INFO:teuthology.orchestra.run.trial046.stdout:Get:59 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 20.3.0-4957-gf7e05f4f-1jammy [224 kB] 2026-02-24T21:55:06.108 INFO:teuthology.orchestra.run.trial046.stdout:Get:60 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 20.3.0-4957-gf7e05f4f-1jammy [87.6 kB] 2026-02-24T21:55:06.138 INFO:teuthology.orchestra.run.trial046.stdout:Get:61 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 20.3.0-4957-gf7e05f4f-1jammy [329 kB] 2026-02-24T21:55:06.347 INFO:teuthology.orchestra.run.trial046.stdout:Get:62 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 20.3.0-4957-gf7e05f4f-1jammy [9090 kB] 2026-02-24T21:55:07.316 INFO:teuthology.orchestra.run.trial110.stdout:Get:66 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 20.3.0-4957-gf7e05f4f-1jammy [4978 kB] 2026-02-24T21:55:08.013 INFO:teuthology.orchestra.run.trial110.stdout:Get:67 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 20.3.0-4957-gf7e05f4f-1jammy [249 kB] 2026-02-24T21:55:08.057 INFO:teuthology.orchestra.run.trial110.stdout:Get:68 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 20.3.0-4957-gf7e05f4f-1jammy [127 kB] 2026-02-24T21:55:08.072 INFO:teuthology.orchestra.run.trial110.stdout:Get:69 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 20.3.0-4957-gf7e05f4f-1jammy [922 kB] 2026-02-24T21:55:08.210 INFO:teuthology.orchestra.run.trial110.stdout:Get:70 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 20.3.0-4957-gf7e05f4f-1jammy [7532 kB] 2026-02-24T21:55:09.279 INFO:teuthology.orchestra.run.trial110.stdout:Get:71 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd-classic amd64 20.3.0-4957-gf7e05f4f-1jammy [467 MB] 2026-02-24T21:55:10.777 INFO:teuthology.orchestra.run.trial046.stdout:Get:63 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 20.3.0-4957-gf7e05f4f-1jammy [111 kB] 2026-02-24T21:55:10.824 INFO:teuthology.orchestra.run.trial046.stdout:Get:64 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 20.3.0-4957-gf7e05f4f-1jammy [261 kB] 2026-02-24T21:55:10.925 INFO:teuthology.orchestra.run.trial046.stdout:Get:65 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 20.3.0-4957-gf7e05f4f-1jammy [31.5 MB] 2026-02-24T21:55:19.356 INFO:teuthology.orchestra.run.trial046.stdout:Get:66 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 20.3.0-4957-gf7e05f4f-1jammy [4978 kB] 2026-02-24T21:55:20.977 INFO:teuthology.orchestra.run.trial046.stdout:Get:67 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 20.3.0-4957-gf7e05f4f-1jammy [249 kB] 2026-02-24T21:55:21.094 INFO:teuthology.orchestra.run.trial046.stdout:Get:68 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 20.3.0-4957-gf7e05f4f-1jammy [127 kB] 2026-02-24T21:55:21.142 INFO:teuthology.orchestra.run.trial046.stdout:Get:69 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 20.3.0-4957-gf7e05f4f-1jammy [922 kB] 2026-02-24T21:55:21.551 INFO:teuthology.orchestra.run.trial046.stdout:Get:70 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 20.3.0-4957-gf7e05f4f-1jammy [7532 kB] 2026-02-24T21:55:26.646 INFO:teuthology.orchestra.run.trial046.stdout:Get:71 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd-classic amd64 20.3.0-4957-gf7e05f4f-1jammy [467 MB] 2026-02-24T22:01:39.456 INFO:teuthology.orchestra.run.trial110.stdout:Get:72 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 20.3.0-4957-gf7e05f4f-1jammy [180 kB] 2026-02-24T22:01:39.566 INFO:teuthology.orchestra.run.trial110.stdout:Get:73 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 20.3.0-4957-gf7e05f4f-1jammy [14.1 kB] 2026-02-24T22:01:39.596 INFO:teuthology.orchestra.run.trial110.stdout:Get:74 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 20.3.0-4957-gf7e05f4f-1jammy [1030 kB] 2026-02-24T22:01:40.309 INFO:teuthology.orchestra.run.trial110.stdout:Get:75 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 20.3.0-4957-gf7e05f4f-1jammy [2438 kB] 2026-02-24T22:01:41.792 INFO:teuthology.orchestra.run.trial110.stdout:Get:76 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 20.3.0-4957-gf7e05f4f-1jammy [1066 kB] 2026-02-24T22:01:42.383 INFO:teuthology.orchestra.run.trial110.stdout:Get:77 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 20.3.0-4957-gf7e05f4f-1jammy [189 kB] 2026-02-24T22:01:42.489 INFO:teuthology.orchestra.run.trial110.stdout:Get:78 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 20.3.0-4957-gf7e05f4f-1jammy [48.9 MB] 2026-02-24T22:02:08.851 INFO:teuthology.orchestra.run.trial046.stdout:Get:72 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 20.3.0-4957-gf7e05f4f-1jammy [180 kB] 2026-02-24T22:02:08.991 INFO:teuthology.orchestra.run.trial046.stdout:Get:73 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 20.3.0-4957-gf7e05f4f-1jammy [14.1 kB] 2026-02-24T22:02:09.016 INFO:teuthology.orchestra.run.trial046.stdout:Get:74 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 20.3.0-4957-gf7e05f4f-1jammy [1030 kB] 2026-02-24T22:02:09.881 INFO:teuthology.orchestra.run.trial046.stdout:Get:75 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 20.3.0-4957-gf7e05f4f-1jammy [2438 kB] 2026-02-24T22:02:11.975 INFO:teuthology.orchestra.run.trial046.stdout:Get:76 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 20.3.0-4957-gf7e05f4f-1jammy [1066 kB] 2026-02-24T22:02:13.002 INFO:teuthology.orchestra.run.trial046.stdout:Get:77 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 20.3.0-4957-gf7e05f4f-1jammy [189 kB] 2026-02-24T22:02:13.163 INFO:teuthology.orchestra.run.trial046.stdout:Get:78 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 20.3.0-4957-gf7e05f4f-1jammy [48.9 MB] 2026-02-24T22:02:26.893 INFO:teuthology.orchestra.run.trial110.stdout:Get:79 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 20.3.0-4957-gf7e05f4f-1jammy [8625 kB] 2026-02-24T22:02:34.806 INFO:teuthology.orchestra.run.trial110.stdout:Get:80 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 20.3.0-4957-gf7e05f4f-1jammy [14.2 kB] 2026-02-24T22:02:34.824 INFO:teuthology.orchestra.run.trial110.stdout:Get:81 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 20.3.0-4957-gf7e05f4f-1jammy [68.6 MB] 2026-02-24T22:02:57.610 INFO:teuthology.orchestra.run.trial046.stdout:Get:79 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 20.3.0-4957-gf7e05f4f-1jammy [8625 kB] 2026-02-24T22:03:03.519 INFO:teuthology.orchestra.run.trial046.stdout:Get:80 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 20.3.0-4957-gf7e05f4f-1jammy [14.2 kB] 2026-02-24T22:03:03.520 INFO:teuthology.orchestra.run.trial046.stdout:Get:81 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 20.3.0-4957-gf7e05f4f-1jammy [68.6 MB] 2026-02-24T22:03:23.106 INFO:teuthology.orchestra.run.trial110.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 20.3.0-4957-gf7e05f4f-1jammy [135 kB] 2026-02-24T22:03:23.235 INFO:teuthology.orchestra.run.trial110.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-daemon amd64 20.3.0-4957-gf7e05f4f-1jammy [43.7 kB] 2026-02-24T22:03:23.268 INFO:teuthology.orchestra.run.trial110.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-proxy2 amd64 20.3.0-4957-gf7e05f4f-1jammy [30.8 kB] 2026-02-24T22:03:23.270 INFO:teuthology.orchestra.run.trial110.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 20.3.0-4957-gf7e05f4f-1jammy [44.2 kB] 2026-02-24T22:03:23.308 INFO:teuthology.orchestra.run.trial110.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 20.3.0-4957-gf7e05f4f-1jammy [18.3 MB] 2026-02-24T22:03:38.096 INFO:teuthology.orchestra.run.trial110.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 20.3.0-4957-gf7e05f4f-1jammy [93.7 kB] 2026-02-24T22:03:38.439 INFO:teuthology.orchestra.run.trial110.stdout:Fetched 695 MB in 8min 39s (1341 kB/s) 2026-02-24T22:03:38.459 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2026-02-24T22:03:38.624 INFO:teuthology.orchestra.run.trial110.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 202926 files and directories currently installed.) 2026-02-24T22:03:38.628 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../00-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2026-02-24T22:03:38.629 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-24T22:03:38.650 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2026-02-24T22:03:38.665 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../01-libdouble-conversion3_3.1.7-4_amd64.deb ... 2026-02-24T22:03:38.666 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-24T22:03:38.682 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2026-02-24T22:03:38.695 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../02-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2026-02-24T22:03:38.695 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-24T22:03:38.713 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libqt5core5a:amd64. 2026-02-24T22:03:38.723 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../03-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-24T22:03:38.728 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-24T22:03:38.760 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2026-02-24T22:03:38.770 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../04-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-24T22:03:38.770 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-24T22:03:38.787 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libqt5network5:amd64. 2026-02-24T22:03:38.797 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../05-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-24T22:03:38.798 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-24T22:03:38.818 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2026-02-24T22:03:38.828 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../06-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2026-02-24T22:03:38.828 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-24T22:03:38.845 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package librados2. 2026-02-24T22:03:38.855 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../07-librados2_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:03:38.855 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking librados2 (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:38.897 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libnbd0. 2026-02-24T22:03:38.907 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../08-libnbd0_1.10.5-1_amd64.deb ... 2026-02-24T22:03:38.908 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libnbd0 (1.10.5-1) ... 2026-02-24T22:03:38.923 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package librbd1. 2026-02-24T22:03:38.933 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../09-librbd1_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:03:38.933 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking librbd1 (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:38.971 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libcephfs2. 2026-02-24T22:03:38.981 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../10-libcephfs2_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:03:38.982 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libcephfs2 (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:39.002 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-rados. 2026-02-24T22:03:39.012 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../11-python3-rados_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:03:39.012 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-rados (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:39.030 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-ceph-argparse. 2026-02-24T22:03:39.039 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../12-python3-ceph-argparse_20.3.0-4957-gf7e05f4f-1jammy_all.deb ... 2026-02-24T22:03:39.040 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-ceph-argparse (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:39.055 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-cephfs. 2026-02-24T22:03:39.065 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../13-python3-cephfs_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:03:39.065 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-cephfs (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:39.082 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-ceph-common. 2026-02-24T22:03:39.092 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../14-python3-ceph-common_20.3.0-4957-gf7e05f4f-1jammy_all.deb ... 2026-02-24T22:03:39.093 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-ceph-common (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:39.111 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-wcwidth. 2026-02-24T22:03:39.120 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../15-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2026-02-24T22:03:39.121 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-24T22:03:39.137 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-prettytable. 2026-02-24T22:03:39.147 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../16-python3-prettytable_2.5.0-2_all.deb ... 2026-02-24T22:03:39.148 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2026-02-24T22:03:39.162 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-rbd. 2026-02-24T22:03:39.172 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../17-python3-rbd_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:03:39.172 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-rbd (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:39.190 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package librdkafka1:amd64. 2026-02-24T22:03:39.200 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../18-librdkafka1_1.8.0-1build1_amd64.deb ... 2026-02-24T22:03:39.200 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-24T22:03:39.219 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package librgw2. 2026-02-24T22:03:39.229 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../19-librgw2_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:03:39.229 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking librgw2 (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:39.328 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-rgw. 2026-02-24T22:03:39.338 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../20-python3-rgw_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:03:39.339 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-rgw (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:39.355 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package liboath0:amd64. 2026-02-24T22:03:39.365 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../21-liboath0_2.6.7-3ubuntu0.1_amd64.deb ... 2026-02-24T22:03:39.365 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-24T22:03:39.380 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libradosstriper1. 2026-02-24T22:03:39.391 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../22-libradosstriper1_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:03:39.391 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libradosstriper1 (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:39.407 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package ceph-common. 2026-02-24T22:03:39.417 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../23-ceph-common_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:03:39.417 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking ceph-common (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:39.679 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package ceph-base. 2026-02-24T22:03:39.689 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../24-ceph-base_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:03:39.693 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking ceph-base (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:39.752 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-jaraco.functools. 2026-02-24T22:03:39.763 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../25-python3-jaraco.functools_3.4.0-2_all.deb ... 2026-02-24T22:03:39.763 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2026-02-24T22:03:39.778 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-cheroot. 2026-02-24T22:03:39.787 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../26-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2026-02-24T22:03:39.788 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-24T22:03:39.806 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-jaraco.classes. 2026-02-24T22:03:39.817 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../27-python3-jaraco.classes_3.2.1-3_all.deb ... 2026-02-24T22:03:39.817 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2026-02-24T22:03:39.832 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-jaraco.text. 2026-02-24T22:03:39.841 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../28-python3-jaraco.text_3.6.0-2_all.deb ... 2026-02-24T22:03:39.842 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2026-02-24T22:03:39.856 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-jaraco.collections. 2026-02-24T22:03:39.866 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../29-python3-jaraco.collections_3.4.0-2_all.deb ... 2026-02-24T22:03:39.867 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2026-02-24T22:03:39.882 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-tempora. 2026-02-24T22:03:39.891 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../30-python3-tempora_4.1.2-1_all.deb ... 2026-02-24T22:03:39.892 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-tempora (4.1.2-1) ... 2026-02-24T22:03:39.907 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-portend. 2026-02-24T22:03:39.918 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../31-python3-portend_3.0.0-1_all.deb ... 2026-02-24T22:03:39.918 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-portend (3.0.0-1) ... 2026-02-24T22:03:39.933 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-zc.lockfile. 2026-02-24T22:03:39.943 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../32-python3-zc.lockfile_2.0-1_all.deb ... 2026-02-24T22:03:39.943 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2026-02-24T22:03:39.958 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-cherrypy3. 2026-02-24T22:03:39.968 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../33-python3-cherrypy3_18.6.1-4_all.deb ... 2026-02-24T22:03:39.969 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2026-02-24T22:03:39.993 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-natsort. 2026-02-24T22:03:40.003 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../34-python3-natsort_8.0.2-1_all.deb ... 2026-02-24T22:03:40.003 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-natsort (8.0.2-1) ... 2026-02-24T22:03:40.020 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2026-02-24T22:03:40.030 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../35-ceph-mgr-modules-core_20.3.0-4957-gf7e05f4f-1jammy_all.deb ... 2026-02-24T22:03:40.031 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking ceph-mgr-modules-core (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:40.058 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2026-02-24T22:03:40.068 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../36-libsqlite3-mod-ceph_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:03:40.068 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libsqlite3-mod-ceph (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:40.084 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package ceph-mgr. 2026-02-24T22:03:40.094 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../37-ceph-mgr_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:03:40.095 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking ceph-mgr (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:40.115 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package ceph-mon. 2026-02-24T22:03:40.125 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../38-ceph-mon_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:03:40.125 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking ceph-mon (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:40.189 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libfuse2:amd64. 2026-02-24T22:03:40.200 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../39-libfuse2_2.9.9-5ubuntu3_amd64.deb ... 2026-02-24T22:03:40.200 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-24T22:03:40.216 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package ceph-osd-classic. 2026-02-24T22:03:40.226 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../40-ceph-osd-classic_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:03:40.226 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking ceph-osd-classic (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:44.251 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package ceph-osd. 2026-02-24T22:03:44.267 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../41-ceph-osd_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:03:44.267 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking ceph-osd (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:44.287 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package ceph. 2026-02-24T22:03:44.302 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../42-ceph_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:03:44.303 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking ceph (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:44.320 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package ceph-fuse. 2026-02-24T22:03:44.335 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../43-ceph-fuse_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:03:44.335 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking ceph-fuse (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:44.364 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package ceph-mds. 2026-02-24T22:03:44.377 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../44-ceph-mds_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:03:44.378 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking ceph-mds (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:44.410 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package cephadm. 2026-02-24T22:03:44.420 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../45-cephadm_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:03:44.421 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking cephadm (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:44.438 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-asyncssh. 2026-02-24T22:03:44.448 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../46-python3-asyncssh_2.5.0-1ubuntu0.1_all.deb ... 2026-02-24T22:03:44.448 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-24T22:03:44.470 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2026-02-24T22:03:44.480 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../47-ceph-mgr-cephadm_20.3.0-4957-gf7e05f4f-1jammy_all.deb ... 2026-02-24T22:03:44.480 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking ceph-mgr-cephadm (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:44.502 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-repoze.lru. 2026-02-24T22:03:44.512 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../48-python3-repoze.lru_0.7-2_all.deb ... 2026-02-24T22:03:44.512 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2026-02-24T22:03:44.528 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-routes. 2026-02-24T22:03:44.538 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../49-python3-routes_2.5.1-1ubuntu1_all.deb ... 2026-02-24T22:03:44.538 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2026-02-24T22:03:44.559 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2026-02-24T22:03:44.569 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../50-ceph-mgr-dashboard_20.3.0-4957-gf7e05f4f-1jammy_all.deb ... 2026-02-24T22:03:44.569 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking ceph-mgr-dashboard (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:45.111 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2026-02-24T22:03:45.123 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../51-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2026-02-24T22:03:45.123 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-24T22:03:45.162 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-joblib. 2026-02-24T22:03:45.172 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../52-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2026-02-24T22:03:45.173 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2026-02-24T22:03:45.201 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-threadpoolctl. 2026-02-24T22:03:45.211 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../53-python3-threadpoolctl_3.1.0-1_all.deb ... 2026-02-24T22:03:45.212 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2026-02-24T22:03:45.227 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-sklearn. 2026-02-24T22:03:45.237 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../54-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2026-02-24T22:03:45.237 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-24T22:03:45.321 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2026-02-24T22:03:45.332 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../55-ceph-mgr-diskprediction-local_20.3.0-4957-gf7e05f4f-1jammy_all.deb ... 2026-02-24T22:03:45.332 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking ceph-mgr-diskprediction-local (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:45.497 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-cachetools. 2026-02-24T22:03:45.508 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../56-python3-cachetools_5.0.0-1_all.deb ... 2026-02-24T22:03:45.508 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2026-02-24T22:03:45.523 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-rsa. 2026-02-24T22:03:45.534 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../57-python3-rsa_4.8-1_all.deb ... 2026-02-24T22:03:45.535 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-rsa (4.8-1) ... 2026-02-24T22:03:45.553 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-google-auth. 2026-02-24T22:03:45.564 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../58-python3-google-auth_1.5.1-3_all.deb ... 2026-02-24T22:03:45.564 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2026-02-24T22:03:45.581 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-requests-oauthlib. 2026-02-24T22:03:45.592 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../59-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2026-02-24T22:03:45.593 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-24T22:03:45.609 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-websocket. 2026-02-24T22:03:45.620 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../60-python3-websocket_1.2.3-1_all.deb ... 2026-02-24T22:03:45.620 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-websocket (1.2.3-1) ... 2026-02-24T22:03:45.638 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-kubernetes. 2026-02-24T22:03:45.649 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../61-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2026-02-24T22:03:45.649 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-24T22:03:45.743 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2026-02-24T22:03:45.754 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../62-ceph-mgr-k8sevents_20.3.0-4957-gf7e05f4f-1jammy_all.deb ... 2026-02-24T22:03:45.754 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking ceph-mgr-k8sevents (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:45.769 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libonig5:amd64. 2026-02-24T22:03:45.780 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../63-libonig5_6.9.7.1-2build1_amd64.deb ... 2026-02-24T22:03:45.780 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-24T22:03:45.796 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libjq1:amd64. 2026-02-24T22:03:45.807 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../64-libjq1_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-24T22:03:45.807 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-24T22:03:45.821 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package jq. 2026-02-24T22:03:45.832 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../65-jq_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-24T22:03:45.832 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking jq (1.6-2.1ubuntu3.1) ... 2026-02-24T22:03:45.847 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package socat. 2026-02-24T22:03:45.858 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../66-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2026-02-24T22:03:45.858 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2026-02-24T22:03:45.878 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package xmlstarlet. 2026-02-24T22:03:45.889 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../67-xmlstarlet_1.6.1-2.1_amd64.deb ... 2026-02-24T22:03:45.889 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2026-02-24T22:03:45.924 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package ceph-test. 2026-02-24T22:03:45.934 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../68-ceph-test_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:03:45.935 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking ceph-test (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:46.569 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package ceph-volume. 2026-02-24T22:03:46.580 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../69-ceph-volume_20.3.0-4957-gf7e05f4f-1jammy_all.deb ... 2026-02-24T22:03:46.580 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking ceph-volume (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:46.603 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libcephfs-daemon. 2026-02-24T22:03:46.614 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../70-libcephfs-daemon_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:03:46.614 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libcephfs-daemon (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:46.629 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libcephfs-proxy2. 2026-02-24T22:03:46.640 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../71-libcephfs-proxy2_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:03:46.640 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libcephfs-proxy2 (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:46.655 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libcephfs-dev. 2026-02-24T22:03:46.665 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../72-libcephfs-dev_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:03:46.666 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libcephfs-dev (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:46.682 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package nvme-cli. 2026-02-24T22:03:46.692 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../73-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2026-02-24T22:03:46.693 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2026-02-24T22:03:46.725 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python-asyncssh-doc. 2026-02-24T22:03:46.735 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../74-python-asyncssh-doc_2.5.0-1ubuntu0.1_all.deb ... 2026-02-24T22:03:46.736 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-24T22:03:46.766 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-iniconfig. 2026-02-24T22:03:46.776 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../75-python3-iniconfig_1.1.1-2_all.deb ... 2026-02-24T22:03:46.776 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2026-02-24T22:03:46.792 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-pluggy. 2026-02-24T22:03:46.802 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../76-python3-pluggy_0.13.0-7.1_all.deb ... 2026-02-24T22:03:46.803 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2026-02-24T22:03:46.819 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-psutil. 2026-02-24T22:03:46.830 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../77-python3-psutil_5.9.0-1build1_amd64.deb ... 2026-02-24T22:03:46.830 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2026-02-24T22:03:46.849 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-py. 2026-02-24T22:03:46.859 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../78-python3-py_1.10.0-1_all.deb ... 2026-02-24T22:03:46.859 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-py (1.10.0-1) ... 2026-02-24T22:03:46.880 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-pygments. 2026-02-24T22:03:46.890 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../79-python3-pygments_2.11.2+dfsg-2ubuntu0.1_all.deb ... 2026-02-24T22:03:46.891 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-24T22:03:46.932 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-toml. 2026-02-24T22:03:46.942 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../80-python3-toml_0.10.2-1_all.deb ... 2026-02-24T22:03:46.943 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-toml (0.10.2-1) ... 2026-02-24T22:03:46.958 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-pytest. 2026-02-24T22:03:46.968 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../81-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2026-02-24T22:03:46.968 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2026-02-24T22:03:46.990 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-simplejson. 2026-02-24T22:03:47.000 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../82-python3-simplejson_3.17.6-1build1_amd64.deb ... 2026-02-24T22:03:47.001 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2026-02-24T22:03:47.026 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package python3-webob. 2026-02-24T22:03:47.036 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../83-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2026-02-24T22:03:47.037 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-24T22:03:47.053 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package qttranslations5-l10n. 2026-02-24T22:03:47.064 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../84-qttranslations5-l10n_5.15.3-1_all.deb ... 2026-02-24T22:03:47.064 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2026-02-24T22:03:47.134 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package radosgw. 2026-02-24T22:03:47.145 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../85-radosgw_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:03:47.145 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking radosgw (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:47.302 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package rbd-fuse. 2026-02-24T22:03:47.313 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../86-rbd-fuse_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:03:47.314 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking rbd-fuse (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:47.343 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2026-02-24T22:03:47.409 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-24T22:03:47.410 INFO:teuthology.orchestra.run.trial110.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2026-02-24T22:03:47.482 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /lib/systemd/system/nvmefc-boot-connections.service. 2026-02-24T22:03:47.725 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service → /lib/systemd/system/nvmf-autoconnect.service. 2026-02-24T22:03:48.108 INFO:teuthology.orchestra.run.trial110.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2026-02-24T22:03:48.118 INFO:teuthology.orchestra.run.trial110.stdout:Setting up cephadm (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:48.162 INFO:teuthology.orchestra.run.trial110.stdout:Adding system user cephadm....done 2026-02-24T22:03:48.169 INFO:teuthology.orchestra.run.trial110.stdout:usermod: no changes 2026-02-24T22:03:48.175 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2026-02-24T22:03:48.243 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-24T22:03:48.245 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2026-02-24T22:03:48.317 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-repoze.lru (0.7-2) ... 2026-02-24T22:03:48.391 INFO:teuthology.orchestra.run.trial110.stdout:Setting up liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-24T22:03:48.392 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-py (1.10.0-1) ... 2026-02-24T22:03:48.487 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2026-02-24T22:03:48.611 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-cachetools (5.0.0-1) ... 2026-02-24T22:03:48.687 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2026-02-24T22:03:48.763 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-ceph-argparse (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:48.831 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-24T22:03:48.832 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libnbd0 (1.10.5-1) ... 2026-02-24T22:03:48.833 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-24T22:03:48.834 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-24T22:03:48.835 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2026-02-24T22:03:48.954 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-natsort (8.0.2-1) ... 2026-02-24T22:03:49.030 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libcephfs-proxy2 (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:49.031 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2026-02-24T22:03:49.111 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2026-02-24T22:03:49.197 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-24T22:03:49.445 INFO:teuthology.orchestra.run.trial110.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2026-02-24T22:03:49.446 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-24T22:03:49.540 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-24T22:03:49.666 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-24T22:03:49.746 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2026-02-24T22:03:49.817 INFO:teuthology.orchestra.run.trial110.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2026-02-24T22:03:49.818 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-ceph-common (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:49.907 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-24T22:03:50.400 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-24T22:03:50.405 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-toml (0.10.2-1) ... 2026-02-24T22:03:50.481 INFO:teuthology.orchestra.run.trial110.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-24T22:03:50.482 INFO:teuthology.orchestra.run.trial110.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2026-02-24T22:03:50.483 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2026-02-24T22:03:50.556 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2026-02-24T22:03:50.628 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-24T22:03:50.629 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-rsa (4.8-1) ... 2026-02-24T22:03:50.706 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-tempora (4.1.2-1) ... 2026-02-24T22:03:50.779 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-prettytable (2.5.0-2) ... 2026-02-24T22:03:50.855 INFO:teuthology.orchestra.run.trial110.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-24T22:03:50.856 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-websocket (1.2.3-1) ... 2026-02-24T22:03:50.939 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-24T22:03:50.941 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-24T22:03:51.016 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-24T22:03:51.099 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2026-02-24T22:03:51.174 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-24T22:03:51.175 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2026-02-24T22:03:51.307 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-portend (3.0.0-1) ... 2026-02-24T22:03:51.380 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-24T22:03:51.381 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-google-auth (1.5.1-3) ... 2026-02-24T22:03:51.469 INFO:teuthology.orchestra.run.trial110.stdout:Setting up jq (1.6-2.1ubuntu3.1) ... 2026-02-24T22:03:51.470 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2026-02-24T22:03:51.599 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-24T22:03:51.600 INFO:teuthology.orchestra.run.trial110.stdout:Setting up librados2 (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:51.601 INFO:teuthology.orchestra.run.trial110.stdout:Setting up librgw2 (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:51.602 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libsqlite3-mod-ceph (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:51.602 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-24T22:03:52.117 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libcephfs2 (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:52.118 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libradosstriper1 (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:52.119 INFO:teuthology.orchestra.run.trial110.stdout:Setting up librbd1 (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:52.120 INFO:teuthology.orchestra.run.trial110.stdout:Setting up ceph-mgr-modules-core (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:52.121 INFO:teuthology.orchestra.run.trial110.stdout:Setting up ceph-fuse (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:52.193 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-24T22:03:52.194 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-24T22:03:52.571 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libcephfs-dev (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:52.572 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-rados (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:52.573 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libcephfs-daemon (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:52.574 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-rbd (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:52.575 INFO:teuthology.orchestra.run.trial110.stdout:Setting up rbd-fuse (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:52.575 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-rgw (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:52.576 INFO:teuthology.orchestra.run.trial110.stdout:Setting up python3-cephfs (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:52.577 INFO:teuthology.orchestra.run.trial110.stdout:Setting up ceph-common (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:52.605 INFO:teuthology.orchestra.run.trial110.stdout:Adding group ceph....done 2026-02-24T22:03:52.640 INFO:teuthology.orchestra.run.trial110.stdout:Adding system user ceph....done 2026-02-24T22:03:52.650 INFO:teuthology.orchestra.run.trial110.stdout:Setting system user ceph properties....done 2026-02-24T22:03:52.656 INFO:teuthology.orchestra.run.trial110.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2026-02-24T22:03:52.741 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /lib/systemd/system/ceph.target. 2026-02-24T22:03:52.989 INFO:teuthology.orchestra.run.trial046.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 20.3.0-4957-gf7e05f4f-1jammy [135 kB] 2026-02-24T22:03:53.007 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service → /lib/systemd/system/rbdmap.service. 2026-02-24T22:03:53.192 INFO:teuthology.orchestra.run.trial046.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-daemon amd64 20.3.0-4957-gf7e05f4f-1jammy [43.7 kB] 2026-02-24T22:03:53.260 INFO:teuthology.orchestra.run.trial046.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-proxy2 amd64 20.3.0-4957-gf7e05f4f-1jammy [30.8 kB] 2026-02-24T22:03:53.322 INFO:teuthology.orchestra.run.trial046.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 20.3.0-4957-gf7e05f4f-1jammy [44.2 kB] 2026-02-24T22:03:53.379 INFO:teuthology.orchestra.run.trial046.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 20.3.0-4957-gf7e05f4f-1jammy [18.3 MB] 2026-02-24T22:03:53.388 INFO:teuthology.orchestra.run.trial110.stdout:Setting up ceph-test (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:53.389 INFO:teuthology.orchestra.run.trial110.stdout:Setting up radosgw (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:53.637 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-24T22:03:53.637 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-24T22:03:53.990 INFO:teuthology.orchestra.run.trial110.stdout:Setting up ceph-base (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:54.105 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /lib/systemd/system/ceph-crash.service. 2026-02-24T22:03:54.480 INFO:teuthology.orchestra.run.trial110.stdout:Setting up ceph-mds (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:54.539 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-24T22:03:54.539 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-24T22:03:54.915 INFO:teuthology.orchestra.run.trial110.stdout:Setting up ceph-mgr (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:54.995 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-24T22:03:54.995 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-24T22:03:55.370 INFO:teuthology.orchestra.run.trial110.stdout:Setting up ceph-mgr-k8sevents (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:55.371 INFO:teuthology.orchestra.run.trial110.stdout:Setting up ceph-osd-classic (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:55.388 INFO:teuthology.orchestra.run.trial110.stdout:update-alternatives: using /usr/bin/ceph-osd-classic to provide /usr/bin/ceph-osd (ceph-osd) in auto mode 2026-02-24T22:03:55.398 INFO:teuthology.orchestra.run.trial110.stdout:Setting up ceph-mgr-diskprediction-local (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:55.409 INFO:teuthology.orchestra.run.trial110.stdout:Setting up ceph-mon (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:55.478 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-24T22:03:55.478 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-24T22:03:55.846 INFO:teuthology.orchestra.run.trial110.stdout:Setting up ceph-mgr-cephadm (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:55.856 INFO:teuthology.orchestra.run.trial110.stdout:Setting up ceph-mgr-dashboard (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:55.867 INFO:teuthology.orchestra.run.trial110.stdout:Setting up ceph-osd (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:55.953 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-24T22:03:55.954 INFO:teuthology.orchestra.run.trial110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-24T22:03:56.335 INFO:teuthology.orchestra.run.trial110.stdout:Setting up ceph (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:56.336 INFO:teuthology.orchestra.run.trial110.stdout:Setting up ceph-volume (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:03:56.444 INFO:teuthology.orchestra.run.trial110.stdout:Processing triggers for man-db (2.10.2-1) ... 2026-02-24T22:03:56.530 INFO:teuthology.orchestra.run.trial110.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.13) ... 2026-02-24T22:03:57.181 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:03:57.181 INFO:teuthology.orchestra.run.trial110.stdout:Running kernel seems to be up-to-date. 2026-02-24T22:03:57.181 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:03:57.181 INFO:teuthology.orchestra.run.trial110.stdout:The processor microcode seems to be up-to-date. 2026-02-24T22:03:57.181 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:03:57.181 INFO:teuthology.orchestra.run.trial110.stdout:No services need to be restarted. 2026-02-24T22:03:57.181 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:03:57.181 INFO:teuthology.orchestra.run.trial110.stdout:No containers need to be restarted. 2026-02-24T22:03:57.181 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:03:57.182 INFO:teuthology.orchestra.run.trial110.stdout:No user sessions are running outdated binaries. 2026-02-24T22:03:57.182 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:03:57.182 INFO:teuthology.orchestra.run.trial110.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-24T22:03:57.988 INFO:teuthology.orchestra.run.trial110.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2026-02-24T22:03:57.991 DEBUG:teuthology.orchestra.run.trial110:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install s3cmd maven golang 2026-02-24T22:03:58.076 INFO:teuthology.orchestra.run.trial110.stdout:Reading package lists... 2026-02-24T22:03:58.186 INFO:teuthology.orchestra.run.trial110.stdout:Building dependency tree... 2026-02-24T22:03:58.186 INFO:teuthology.orchestra.run.trial110.stdout:Reading state information... 2026-02-24T22:03:58.274 INFO:teuthology.orchestra.run.trial110.stdout:The following packages were automatically installed and are no longer required: 2026-02-24T22:03:58.275 INFO:teuthology.orchestra.run.trial110.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-24T22:03:58.275 INFO:teuthology.orchestra.run.trial110.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 2026-02-24T22:03:58.275 INFO:teuthology.orchestra.run.trial110.stdout: linux-headers-5.15.0-164 linux-headers-5.15.0-164-generic 2026-02-24T22:03:58.275 INFO:teuthology.orchestra.run.trial110.stdout: linux-image-5.15.0-164-generic linux-modules-5.15.0-164-generic 2026-02-24T22:03:58.275 INFO:teuthology.orchestra.run.trial110.stdout: linux-modules-extra-5.15.0-164-generic sg3-utils sg3-utils-udev 2026-02-24T22:03:58.275 INFO:teuthology.orchestra.run.trial110.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-24T22:03:58.275 INFO:teuthology.orchestra.run.trial110.stdout:The following additional packages will be installed: 2026-02-24T22:03:58.275 INFO:teuthology.orchestra.run.trial110.stdout: golang-1.18 golang-1.18-doc golang-1.18-go golang-1.18-src golang-doc 2026-02-24T22:03:58.275 INFO:teuthology.orchestra.run.trial110.stdout: golang-go golang-src libaopalliance-java libapache-pom-java 2026-02-24T22:03:58.275 INFO:teuthology.orchestra.run.trial110.stdout: libatinject-jsr330-api-java libcdi-api-java libcommons-cli-java 2026-02-24T22:03:58.276 INFO:teuthology.orchestra.run.trial110.stdout: libcommons-io-java libcommons-lang3-java libcommons-parent-java 2026-02-24T22:03:58.276 INFO:teuthology.orchestra.run.trial110.stdout: libgeronimo-annotation-1.3-spec-java libgeronimo-interceptor-3.0-spec-java 2026-02-24T22:03:58.276 INFO:teuthology.orchestra.run.trial110.stdout: libguava-java libguice-java libhawtjni-runtime-java libjansi-java 2026-02-24T22:03:58.276 INFO:teuthology.orchestra.run.trial110.stdout: libjansi-native-java libjsr305-java libmaven-parent-java 2026-02-24T22:03:58.276 INFO:teuthology.orchestra.run.trial110.stdout: libmaven-resolver-java libmaven-shared-utils-java libmaven3-core-java 2026-02-24T22:03:58.276 INFO:teuthology.orchestra.run.trial110.stdout: libplexus-cipher-java libplexus-classworlds-java 2026-02-24T22:03:58.276 INFO:teuthology.orchestra.run.trial110.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2026-02-24T22:03:58.276 INFO:teuthology.orchestra.run.trial110.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2026-02-24T22:03:58.276 INFO:teuthology.orchestra.run.trial110.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2026-02-24T22:03:58.276 INFO:teuthology.orchestra.run.trial110.stdout: libwagon-http-shaded-java libwagon-provider-api-java pkg-config 2026-02-24T22:03:58.276 INFO:teuthology.orchestra.run.trial110.stdout:Suggested packages: 2026-02-24T22:03:58.276 INFO:teuthology.orchestra.run.trial110.stdout: bzr | brz mercurial subversion libaopalliance-java-doc 2026-02-24T22:03:58.277 INFO:teuthology.orchestra.run.trial110.stdout: libatinject-jsr330-api-java-doc libel-api-java libcommons-io-java-doc 2026-02-24T22:03:58.277 INFO:teuthology.orchestra.run.trial110.stdout: libcommons-lang3-java-doc libasm-java libcglib-java libjsr305-java-doc 2026-02-24T22:03:58.277 INFO:teuthology.orchestra.run.trial110.stdout: libmaven-shared-utils-java-doc liblogback-java 2026-02-24T22:03:58.277 INFO:teuthology.orchestra.run.trial110.stdout: libplexus-classworlds-java-doc libplexus-sec-dispatcher-java-doc 2026-02-24T22:03:58.277 INFO:teuthology.orchestra.run.trial110.stdout: libplexus-utils2-java-doc testng libcommons-logging-java liblog4j1.2-java 2026-02-24T22:03:58.289 INFO:teuthology.orchestra.run.trial110.stdout:The following NEW packages will be installed: 2026-02-24T22:03:58.289 INFO:teuthology.orchestra.run.trial110.stdout: golang golang-1.18 golang-1.18-doc golang-1.18-go golang-1.18-src golang-doc 2026-02-24T22:03:58.289 INFO:teuthology.orchestra.run.trial110.stdout: golang-go golang-src libaopalliance-java libapache-pom-java 2026-02-24T22:03:58.290 INFO:teuthology.orchestra.run.trial110.stdout: libatinject-jsr330-api-java libcdi-api-java libcommons-cli-java 2026-02-24T22:03:58.290 INFO:teuthology.orchestra.run.trial110.stdout: libcommons-io-java libcommons-lang3-java libcommons-parent-java 2026-02-24T22:03:58.290 INFO:teuthology.orchestra.run.trial110.stdout: libgeronimo-annotation-1.3-spec-java libgeronimo-interceptor-3.0-spec-java 2026-02-24T22:03:58.290 INFO:teuthology.orchestra.run.trial110.stdout: libguava-java libguice-java libhawtjni-runtime-java libjansi-java 2026-02-24T22:03:58.290 INFO:teuthology.orchestra.run.trial110.stdout: libjansi-native-java libjsr305-java libmaven-parent-java 2026-02-24T22:03:58.290 INFO:teuthology.orchestra.run.trial110.stdout: libmaven-resolver-java libmaven-shared-utils-java libmaven3-core-java 2026-02-24T22:03:58.290 INFO:teuthology.orchestra.run.trial110.stdout: libplexus-cipher-java libplexus-classworlds-java 2026-02-24T22:03:58.290 INFO:teuthology.orchestra.run.trial110.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2026-02-24T22:03:58.290 INFO:teuthology.orchestra.run.trial110.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2026-02-24T22:03:58.290 INFO:teuthology.orchestra.run.trial110.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2026-02-24T22:03:58.290 INFO:teuthology.orchestra.run.trial110.stdout: libwagon-http-shaded-java libwagon-provider-api-java maven pkg-config s3cmd 2026-02-24T22:03:58.394 INFO:teuthology.orchestra.run.trial110.stdout:0 upgraded, 43 newly installed, 0 to remove and 14 not upgraded. 2026-02-24T22:03:58.394 INFO:teuthology.orchestra.run.trial110.stdout:Need to get 92.9 MB of archives. 2026-02-24T22:03:58.394 INFO:teuthology.orchestra.run.trial110.stdout:After this operation, 452 MB of additional disk space will be used. 2026-02-24T22:03:58.394 INFO:teuthology.orchestra.run.trial110.stdout:Get:1 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 golang-1.18-doc all 1.18.1-1ubuntu1.2 [104 kB] 2026-02-24T22:03:58.431 INFO:teuthology.orchestra.run.trial110.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 golang-1.18-src all 1.18.1-1ubuntu1.2 [16.3 MB] 2026-02-24T22:03:58.545 INFO:teuthology.orchestra.run.trial110.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 golang-1.18-go amd64 1.18.1-1ubuntu1.2 [66.1 MB] 2026-02-24T22:03:58.791 INFO:teuthology.orchestra.run.trial110.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 golang-1.18 all 1.18.1-1ubuntu1.2 [11.2 kB] 2026-02-24T22:03:58.792 INFO:teuthology.orchestra.run.trial110.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy/main amd64 golang-src all 2:1.18~0ubuntu2 [4438 B] 2026-02-24T22:03:58.792 INFO:teuthology.orchestra.run.trial110.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy/main amd64 golang-go amd64 2:1.18~0ubuntu2 [41.8 kB] 2026-02-24T22:03:58.792 INFO:teuthology.orchestra.run.trial110.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy/main amd64 golang-doc all 2:1.18~0ubuntu2 [2784 B] 2026-02-24T22:03:58.792 INFO:teuthology.orchestra.run.trial110.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy/main amd64 golang amd64 2:1.18~0ubuntu2 [2732 B] 2026-02-24T22:03:58.792 INFO:teuthology.orchestra.run.trial110.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libapache-pom-java all 18-1 [4720 B] 2026-02-24T22:03:58.792 INFO:teuthology.orchestra.run.trial110.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libatinject-jsr330-api-java all 1.0+ds1-5 [5348 B] 2026-02-24T22:03:58.792 INFO:teuthology.orchestra.run.trial110.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-interceptor-3.0-spec-java all 1.0.1-4fakesync [8616 B] 2026-02-24T22:03:58.792 INFO:teuthology.orchestra.run.trial110.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libcdi-api-java all 1.2-3 [54.3 kB] 2026-02-24T22:03:58.800 INFO:teuthology.orchestra.run.trial110.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-cli-java all 1.4-2 [55.8 kB] 2026-02-24T22:03:58.802 INFO:teuthology.orchestra.run.trial110.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-parent-java all 43-1 [10.8 kB] 2026-02-24T22:03:58.802 INFO:teuthology.orchestra.run.trial110.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-io-java all 2.11.0-2 [297 kB] 2026-02-24T22:03:58.803 INFO:teuthology.orchestra.run.trial110.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-lang3-java all 3.11-1 [526 kB] 2026-02-24T22:03:58.805 INFO:teuthology.orchestra.run.trial110.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-annotation-1.3-spec-java all 1.3-1 [11.2 kB] 2026-02-24T22:03:58.805 INFO:teuthology.orchestra.run.trial110.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libjsr305-java all 0.1~+svn49-11 [27.0 kB] 2026-02-24T22:03:58.806 INFO:teuthology.orchestra.run.trial110.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libguava-java all 29.0-6 [2418 kB] 2026-02-24T22:03:58.815 INFO:teuthology.orchestra.run.trial110.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libaopalliance-java all 20070526-6 [9084 B] 2026-02-24T22:03:58.815 INFO:teuthology.orchestra.run.trial110.stdout:Get:21 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libguice-java all 4.2.3-2 [1434 kB] 2026-02-24T22:03:58.820 INFO:teuthology.orchestra.run.trial110.stdout:Get:22 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libhawtjni-runtime-java all 1.17-1 [28.8 kB] 2026-02-24T22:03:58.821 INFO:teuthology.orchestra.run.trial110.stdout:Get:23 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-native-java all 1.8-1 [23.8 kB] 2026-02-24T22:03:58.821 INFO:teuthology.orchestra.run.trial110.stdout:Get:24 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-java all 1.18-1 [56.8 kB] 2026-02-24T22:03:58.821 INFO:teuthology.orchestra.run.trial110.stdout:Get:25 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-parent-java all 31-2 [5140 B] 2026-02-24T22:03:58.821 INFO:teuthology.orchestra.run.trial110.stdout:Get:26 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-utils2-java all 3.3.0-1 [250 kB] 2026-02-24T22:03:58.822 INFO:teuthology.orchestra.run.trial110.stdout:Get:27 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-provider-api-java all 3.3.4-1 [48.5 kB] 2026-02-24T22:03:58.822 INFO:teuthology.orchestra.run.trial110.stdout:Get:28 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-resolver-java all 1.4.2-3build1 [555 kB] 2026-02-24T22:03:58.829 INFO:teuthology.orchestra.run.trial110.stdout:Get:29 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libmaven-shared-utils-java all 3.3.0-1ubuntu0.22.04.1 [148 kB] 2026-02-24T22:03:58.830 INFO:teuthology.orchestra.run.trial110.stdout:Get:30 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-cipher-java all 1.8-2 [15.1 kB] 2026-02-24T22:03:58.834 INFO:teuthology.orchestra.run.trial110.stdout:Get:31 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-classworlds-java all 2.6.0-1 [49.4 kB] 2026-02-24T22:03:58.835 INFO:teuthology.orchestra.run.trial110.stdout:Get:32 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-component-annotations-java all 2.1.0-1 [6564 B] 2026-02-24T22:03:58.835 INFO:teuthology.orchestra.run.trial110.stdout:Get:33 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-interpolation-java all 1.26-1 [76.8 kB] 2026-02-24T22:03:58.835 INFO:teuthology.orchestra.run.trial110.stdout:Get:34 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-sec-dispatcher-java all 1.4-4 [28.1 kB] 2026-02-24T22:03:58.836 INFO:teuthology.orchestra.run.trial110.stdout:Get:35 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libslf4j-java all 1.7.32-1 [141 kB] 2026-02-24T22:03:58.836 INFO:teuthology.orchestra.run.trial110.stdout:Get:36 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-inject-java all 0.3.4-2 [347 kB] 2026-02-24T22:03:58.837 INFO:teuthology.orchestra.run.trial110.stdout:Get:37 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-plexus-java all 0.3.4-3 [181 kB] 2026-02-24T22:03:58.839 INFO:teuthology.orchestra.run.trial110.stdout:Get:38 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven3-core-java all 3.6.3-5 [1535 kB] 2026-02-24T22:03:58.848 INFO:teuthology.orchestra.run.trial110.stdout:Get:39 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-file-java all 3.3.4-1 [7928 B] 2026-02-24T22:03:58.849 INFO:teuthology.orchestra.run.trial110.stdout:Get:40 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-http-shaded-java all 3.3.4-1 [1855 kB] 2026-02-24T22:03:58.856 INFO:teuthology.orchestra.run.trial110.stdout:Get:41 https://archive.ubuntu.com/ubuntu jammy/universe amd64 maven all 3.6.3-5 [17.8 kB] 2026-02-24T22:03:58.856 INFO:teuthology.orchestra.run.trial110.stdout:Get:42 https://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2026-02-24T22:03:58.857 INFO:teuthology.orchestra.run.trial110.stdout:Get:43 https://archive.ubuntu.com/ubuntu jammy/universe amd64 s3cmd all 2.2.0-1 [120 kB] 2026-02-24T22:03:59.084 INFO:teuthology.orchestra.run.trial110.stdout:Fetched 92.9 MB in 1s (164 MB/s) 2026-02-24T22:03:59.099 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package golang-1.18-doc. 2026-02-24T22:03:59.146 INFO:teuthology.orchestra.run.trial110.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 210764 files and directories currently installed.) 2026-02-24T22:03:59.149 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../00-golang-1.18-doc_1.18.1-1ubuntu1.2_all.deb ... 2026-02-24T22:03:59.149 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking golang-1.18-doc (1.18.1-1ubuntu1.2) ... 2026-02-24T22:03:59.168 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package golang-1.18-src. 2026-02-24T22:03:59.178 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../01-golang-1.18-src_1.18.1-1ubuntu1.2_all.deb ... 2026-02-24T22:03:59.179 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking golang-1.18-src (1.18.1-1ubuntu1.2) ... 2026-02-24T22:04:00.244 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package golang-1.18-go. 2026-02-24T22:04:00.258 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../02-golang-1.18-go_1.18.1-1ubuntu1.2_amd64.deb ... 2026-02-24T22:04:00.259 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking golang-1.18-go (1.18.1-1ubuntu1.2) ... 2026-02-24T22:04:00.995 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package golang-1.18. 2026-02-24T22:04:01.007 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../03-golang-1.18_1.18.1-1ubuntu1.2_all.deb ... 2026-02-24T22:04:01.007 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking golang-1.18 (1.18.1-1ubuntu1.2) ... 2026-02-24T22:04:01.021 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package golang-src. 2026-02-24T22:04:01.033 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../04-golang-src_2%3a1.18~0ubuntu2_all.deb ... 2026-02-24T22:04:01.033 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking golang-src (2:1.18~0ubuntu2) ... 2026-02-24T22:04:01.049 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package golang-go:amd64. 2026-02-24T22:04:01.060 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../05-golang-go_2%3a1.18~0ubuntu2_amd64.deb ... 2026-02-24T22:04:01.062 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking golang-go:amd64 (2:1.18~0ubuntu2) ... 2026-02-24T22:04:01.080 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package golang-doc. 2026-02-24T22:04:01.091 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../06-golang-doc_2%3a1.18~0ubuntu2_all.deb ... 2026-02-24T22:04:01.091 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking golang-doc (2:1.18~0ubuntu2) ... 2026-02-24T22:04:01.106 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package golang:amd64. 2026-02-24T22:04:01.118 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../07-golang_2%3a1.18~0ubuntu2_amd64.deb ... 2026-02-24T22:04:01.118 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking golang:amd64 (2:1.18~0ubuntu2) ... 2026-02-24T22:04:01.134 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libapache-pom-java. 2026-02-24T22:04:01.145 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../08-libapache-pom-java_18-1_all.deb ... 2026-02-24T22:04:01.145 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libapache-pom-java (18-1) ... 2026-02-24T22:04:01.162 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libatinject-jsr330-api-java. 2026-02-24T22:04:01.173 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../09-libatinject-jsr330-api-java_1.0+ds1-5_all.deb ... 2026-02-24T22:04:01.173 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libatinject-jsr330-api-java (1.0+ds1-5) ... 2026-02-24T22:04:01.189 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libgeronimo-interceptor-3.0-spec-java. 2026-02-24T22:04:01.200 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../10-libgeronimo-interceptor-3.0-spec-java_1.0.1-4fakesync_all.deb ... 2026-02-24T22:04:01.200 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2026-02-24T22:04:01.216 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libcdi-api-java. 2026-02-24T22:04:01.227 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../11-libcdi-api-java_1.2-3_all.deb ... 2026-02-24T22:04:01.227 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libcdi-api-java (1.2-3) ... 2026-02-24T22:04:01.246 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libcommons-cli-java. 2026-02-24T22:04:01.257 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../12-libcommons-cli-java_1.4-2_all.deb ... 2026-02-24T22:04:01.257 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libcommons-cli-java (1.4-2) ... 2026-02-24T22:04:01.275 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libcommons-parent-java. 2026-02-24T22:04:01.286 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../13-libcommons-parent-java_43-1_all.deb ... 2026-02-24T22:04:01.287 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libcommons-parent-java (43-1) ... 2026-02-24T22:04:01.302 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libcommons-io-java. 2026-02-24T22:04:01.313 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../14-libcommons-io-java_2.11.0-2_all.deb ... 2026-02-24T22:04:01.313 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libcommons-io-java (2.11.0-2) ... 2026-02-24T22:04:01.330 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libcommons-lang3-java. 2026-02-24T22:04:01.341 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../15-libcommons-lang3-java_3.11-1_all.deb ... 2026-02-24T22:04:01.341 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libcommons-lang3-java (3.11-1) ... 2026-02-24T22:04:01.376 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libgeronimo-annotation-1.3-spec-java. 2026-02-24T22:04:01.387 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../16-libgeronimo-annotation-1.3-spec-java_1.3-1_all.deb ... 2026-02-24T22:04:01.387 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2026-02-24T22:04:01.402 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libjsr305-java. 2026-02-24T22:04:01.413 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../17-libjsr305-java_0.1~+svn49-11_all.deb ... 2026-02-24T22:04:01.413 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libjsr305-java (0.1~+svn49-11) ... 2026-02-24T22:04:01.431 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libguava-java. 2026-02-24T22:04:01.442 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../18-libguava-java_29.0-6_all.deb ... 2026-02-24T22:04:01.442 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libguava-java (29.0-6) ... 2026-02-24T22:04:01.547 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libaopalliance-java. 2026-02-24T22:04:01.564 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../19-libaopalliance-java_20070526-6_all.deb ... 2026-02-24T22:04:01.564 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libaopalliance-java (20070526-6) ... 2026-02-24T22:04:01.582 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libguice-java. 2026-02-24T22:04:01.598 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../20-libguice-java_4.2.3-2_all.deb ... 2026-02-24T22:04:01.599 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libguice-java (4.2.3-2) ... 2026-02-24T22:04:01.676 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libhawtjni-runtime-java. 2026-02-24T22:04:01.692 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../21-libhawtjni-runtime-java_1.17-1_all.deb ... 2026-02-24T22:04:01.693 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libhawtjni-runtime-java (1.17-1) ... 2026-02-24T22:04:01.712 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libjansi-native-java. 2026-02-24T22:04:01.728 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../22-libjansi-native-java_1.8-1_all.deb ... 2026-02-24T22:04:01.728 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libjansi-native-java (1.8-1) ... 2026-02-24T22:04:01.747 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libjansi-java. 2026-02-24T22:04:01.761 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../23-libjansi-java_1.18-1_all.deb ... 2026-02-24T22:04:01.761 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libjansi-java (1.18-1) ... 2026-02-24T22:04:01.780 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libmaven-parent-java. 2026-02-24T22:04:01.794 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../24-libmaven-parent-java_31-2_all.deb ... 2026-02-24T22:04:01.794 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libmaven-parent-java (31-2) ... 2026-02-24T22:04:01.810 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libplexus-utils2-java. 2026-02-24T22:04:01.821 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../25-libplexus-utils2-java_3.3.0-1_all.deb ... 2026-02-24T22:04:01.821 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libplexus-utils2-java (3.3.0-1) ... 2026-02-24T22:04:01.846 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libwagon-provider-api-java. 2026-02-24T22:04:01.856 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../26-libwagon-provider-api-java_3.3.4-1_all.deb ... 2026-02-24T22:04:01.857 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libwagon-provider-api-java (3.3.4-1) ... 2026-02-24T22:04:01.875 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libmaven-resolver-java. 2026-02-24T22:04:01.886 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../27-libmaven-resolver-java_1.4.2-3build1_all.deb ... 2026-02-24T22:04:01.886 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libmaven-resolver-java (1.4.2-3build1) ... 2026-02-24T22:04:01.930 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libmaven-shared-utils-java. 2026-02-24T22:04:01.943 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../28-libmaven-shared-utils-java_3.3.0-1ubuntu0.22.04.1_all.deb ... 2026-02-24T22:04:01.943 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libmaven-shared-utils-java (3.3.0-1ubuntu0.22.04.1) ... 2026-02-24T22:04:01.960 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libplexus-cipher-java. 2026-02-24T22:04:01.970 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../29-libplexus-cipher-java_1.8-2_all.deb ... 2026-02-24T22:04:01.971 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libplexus-cipher-java (1.8-2) ... 2026-02-24T22:04:01.987 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libplexus-classworlds-java. 2026-02-24T22:04:01.998 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../30-libplexus-classworlds-java_2.6.0-1_all.deb ... 2026-02-24T22:04:01.999 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libplexus-classworlds-java (2.6.0-1) ... 2026-02-24T22:04:02.017 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libplexus-component-annotations-java. 2026-02-24T22:04:02.027 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../31-libplexus-component-annotations-java_2.1.0-1_all.deb ... 2026-02-24T22:04:02.028 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libplexus-component-annotations-java (2.1.0-1) ... 2026-02-24T22:04:02.044 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libplexus-interpolation-java. 2026-02-24T22:04:02.055 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../32-libplexus-interpolation-java_1.26-1_all.deb ... 2026-02-24T22:04:02.055 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libplexus-interpolation-java (1.26-1) ... 2026-02-24T22:04:02.074 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libplexus-sec-dispatcher-java. 2026-02-24T22:04:02.084 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../33-libplexus-sec-dispatcher-java_1.4-4_all.deb ... 2026-02-24T22:04:02.085 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libplexus-sec-dispatcher-java (1.4-4) ... 2026-02-24T22:04:02.101 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libslf4j-java. 2026-02-24T22:04:02.112 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../34-libslf4j-java_1.7.32-1_all.deb ... 2026-02-24T22:04:02.112 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libslf4j-java (1.7.32-1) ... 2026-02-24T22:04:02.133 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libsisu-inject-java. 2026-02-24T22:04:02.144 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../35-libsisu-inject-java_0.3.4-2_all.deb ... 2026-02-24T22:04:02.144 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libsisu-inject-java (0.3.4-2) ... 2026-02-24T22:04:02.172 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libsisu-plexus-java. 2026-02-24T22:04:02.183 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../36-libsisu-plexus-java_0.3.4-3_all.deb ... 2026-02-24T22:04:02.184 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libsisu-plexus-java (0.3.4-3) ... 2026-02-24T22:04:02.206 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libmaven3-core-java. 2026-02-24T22:04:02.217 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../37-libmaven3-core-java_3.6.3-5_all.deb ... 2026-02-24T22:04:02.218 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libmaven3-core-java (3.6.3-5) ... 2026-02-24T22:04:02.296 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libwagon-file-java. 2026-02-24T22:04:02.313 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../38-libwagon-file-java_3.3.4-1_all.deb ... 2026-02-24T22:04:02.313 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libwagon-file-java (3.3.4-1) ... 2026-02-24T22:04:02.332 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package libwagon-http-shaded-java. 2026-02-24T22:04:02.347 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../39-libwagon-http-shaded-java_3.3.4-1_all.deb ... 2026-02-24T22:04:02.347 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking libwagon-http-shaded-java (3.3.4-1) ... 2026-02-24T22:04:02.428 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package maven. 2026-02-24T22:04:02.444 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../40-maven_3.6.3-5_all.deb ... 2026-02-24T22:04:02.445 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking maven (3.6.3-5) ... 2026-02-24T22:04:02.468 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package pkg-config. 2026-02-24T22:04:02.485 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../41-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2026-02-24T22:04:02.485 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2026-02-24T22:04:02.505 INFO:teuthology.orchestra.run.trial110.stdout:Selecting previously unselected package s3cmd. 2026-02-24T22:04:02.522 INFO:teuthology.orchestra.run.trial110.stdout:Preparing to unpack .../42-s3cmd_2.2.0-1_all.deb ... 2026-02-24T22:04:02.522 INFO:teuthology.orchestra.run.trial110.stdout:Unpacking s3cmd (2.2.0-1) ... 2026-02-24T22:04:02.556 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libslf4j-java (1.7.32-1) ... 2026-02-24T22:04:02.556 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libplexus-utils2-java (3.3.0-1) ... 2026-02-24T22:04:02.557 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libplexus-classworlds-java (2.6.0-1) ... 2026-02-24T22:04:02.558 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libjsr305-java (0.1~+svn49-11) ... 2026-02-24T22:04:02.559 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libaopalliance-java (20070526-6) ... 2026-02-24T22:04:02.560 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libcommons-cli-java (1.4-2) ... 2026-02-24T22:04:02.561 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libplexus-component-annotations-java (2.1.0-1) ... 2026-02-24T22:04:02.562 INFO:teuthology.orchestra.run.trial110.stdout:Setting up s3cmd (2.2.0-1) ... 2026-02-24T22:04:02.645 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libplexus-cipher-java (1.8-2) ... 2026-02-24T22:04:02.646 INFO:teuthology.orchestra.run.trial110.stdout:Setting up golang-1.18-doc (1.18.1-1ubuntu1.2) ... 2026-02-24T22:04:02.647 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2026-02-24T22:04:02.648 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2026-02-24T22:04:02.648 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libapache-pom-java (18-1) ... 2026-02-24T22:04:02.649 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libatinject-jsr330-api-java (1.0+ds1-5) ... 2026-02-24T22:04:02.650 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libplexus-interpolation-java (1.26-1) ... 2026-02-24T22:04:02.651 INFO:teuthology.orchestra.run.trial110.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2026-02-24T22:04:02.671 INFO:teuthology.orchestra.run.trial110.stdout:Setting up golang-1.18-src (1.18.1-1ubuntu1.2) ... 2026-02-24T22:04:02.672 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libplexus-sec-dispatcher-java (1.4-4) ... 2026-02-24T22:04:02.673 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libwagon-http-shaded-java (3.3.4-1) ... 2026-02-24T22:04:02.674 INFO:teuthology.orchestra.run.trial110.stdout:Setting up golang-src (2:1.18~0ubuntu2) ... 2026-02-24T22:04:02.675 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libcdi-api-java (1.2-3) ... 2026-02-24T22:04:02.676 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libhawtjni-runtime-java (1.17-1) ... 2026-02-24T22:04:02.676 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libwagon-provider-api-java (3.3.4-1) ... 2026-02-24T22:04:02.677 INFO:teuthology.orchestra.run.trial110.stdout:Setting up golang-1.18-go (1.18.1-1ubuntu1.2) ... 2026-02-24T22:04:02.678 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libmaven-parent-java (31-2) ... 2026-02-24T22:04:02.679 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libcommons-parent-java (43-1) ... 2026-02-24T22:04:02.680 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libsisu-inject-java (0.3.4-2) ... 2026-02-24T22:04:02.681 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libsisu-plexus-java (0.3.4-3) ... 2026-02-24T22:04:02.682 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libmaven-resolver-java (1.4.2-3build1) ... 2026-02-24T22:04:02.682 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libguava-java (29.0-6) ... 2026-02-24T22:04:02.683 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libcommons-lang3-java (3.11-1) ... 2026-02-24T22:04:02.684 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libjansi-native-java (1.8-1) ... 2026-02-24T22:04:02.685 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libwagon-file-java (3.3.4-1) ... 2026-02-24T22:04:02.686 INFO:teuthology.orchestra.run.trial110.stdout:Setting up golang-1.18 (1.18.1-1ubuntu1.2) ... 2026-02-24T22:04:02.686 INFO:teuthology.orchestra.run.trial110.stdout:Setting up golang-go:amd64 (2:1.18~0ubuntu2) ... 2026-02-24T22:04:02.687 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libcommons-io-java (2.11.0-2) ... 2026-02-24T22:04:02.688 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libguice-java (4.2.3-2) ... 2026-02-24T22:04:02.689 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libjansi-java (1.18-1) ... 2026-02-24T22:04:02.690 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libmaven-shared-utils-java (3.3.0-1ubuntu0.22.04.1) ... 2026-02-24T22:04:02.691 INFO:teuthology.orchestra.run.trial110.stdout:Setting up golang-doc (2:1.18~0ubuntu2) ... 2026-02-24T22:04:02.691 INFO:teuthology.orchestra.run.trial110.stdout:Setting up golang:amd64 (2:1.18~0ubuntu2) ... 2026-02-24T22:04:02.692 INFO:teuthology.orchestra.run.trial110.stdout:Setting up libmaven3-core-java (3.6.3-5) ... 2026-02-24T22:04:02.693 INFO:teuthology.orchestra.run.trial110.stdout:Setting up maven (3.6.3-5) ... 2026-02-24T22:04:02.697 INFO:teuthology.orchestra.run.trial110.stdout:update-alternatives: using /usr/share/maven/bin/mvn to provide /usr/bin/mvn (mvn) in auto mode 2026-02-24T22:04:02.699 INFO:teuthology.orchestra.run.trial110.stdout:Processing triggers for man-db (2.10.2-1) ... 2026-02-24T22:04:03.112 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:03.113 INFO:teuthology.orchestra.run.trial110.stdout:Running kernel seems to be up-to-date. 2026-02-24T22:04:03.113 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:03.113 INFO:teuthology.orchestra.run.trial110.stdout:The processor microcode seems to be up-to-date. 2026-02-24T22:04:03.113 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:03.113 INFO:teuthology.orchestra.run.trial110.stdout:No services need to be restarted. 2026-02-24T22:04:03.113 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:03.113 INFO:teuthology.orchestra.run.trial110.stdout:No containers need to be restarted. 2026-02-24T22:04:03.113 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:03.113 INFO:teuthology.orchestra.run.trial110.stdout:No user sessions are running outdated binaries. 2026-02-24T22:04:03.113 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:03.113 INFO:teuthology.orchestra.run.trial110.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-24T22:04:03.913 INFO:teuthology.orchestra.run.trial110.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2026-02-24T22:04:03.920 DEBUG:teuthology.parallel:result is None 2026-02-24T22:04:07.213 INFO:teuthology.orchestra.run.trial046.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/wip-d4n-remove-bucket/f7e05f4fdea5af173284f247d6b0935b337d4ccf/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 20.3.0-4957-gf7e05f4f-1jammy [93.7 kB] 2026-02-24T22:04:07.547 INFO:teuthology.orchestra.run.trial046.stdout:Fetched 695 MB in 9min 8s (1270 kB/s) 2026-02-24T22:04:07.563 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2026-02-24T22:04:07.731 INFO:teuthology.orchestra.run.trial046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 202926 files and directories currently installed.) 2026-02-24T22:04:07.734 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../00-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2026-02-24T22:04:07.735 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-24T22:04:07.757 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2026-02-24T22:04:07.771 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../01-libdouble-conversion3_3.1.7-4_amd64.deb ... 2026-02-24T22:04:07.772 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-24T22:04:07.788 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2026-02-24T22:04:07.799 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../02-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2026-02-24T22:04:07.800 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-24T22:04:07.818 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libqt5core5a:amd64. 2026-02-24T22:04:07.828 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../03-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-24T22:04:07.833 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-24T22:04:07.865 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2026-02-24T22:04:07.875 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../04-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-24T22:04:07.875 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-24T22:04:07.893 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libqt5network5:amd64. 2026-02-24T22:04:07.902 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../05-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-24T22:04:07.903 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-24T22:04:07.922 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2026-02-24T22:04:07.932 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../06-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2026-02-24T22:04:07.932 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-24T22:04:07.948 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package librados2. 2026-02-24T22:04:07.958 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../07-librados2_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:04:07.959 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking librados2 (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:08.004 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libnbd0. 2026-02-24T22:04:08.014 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../08-libnbd0_1.10.5-1_amd64.deb ... 2026-02-24T22:04:08.014 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libnbd0 (1.10.5-1) ... 2026-02-24T22:04:08.029 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package librbd1. 2026-02-24T22:04:08.039 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../09-librbd1_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:04:08.040 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking librbd1 (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:08.079 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libcephfs2. 2026-02-24T22:04:08.089 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../10-libcephfs2_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:04:08.090 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libcephfs2 (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:08.109 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-rados. 2026-02-24T22:04:08.119 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../11-python3-rados_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:04:08.119 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-rados (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:08.137 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-ceph-argparse. 2026-02-24T22:04:08.147 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../12-python3-ceph-argparse_20.3.0-4957-gf7e05f4f-1jammy_all.deb ... 2026-02-24T22:04:08.147 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-ceph-argparse (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:08.162 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-cephfs. 2026-02-24T22:04:08.172 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../13-python3-cephfs_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:04:08.173 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-cephfs (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:08.189 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-ceph-common. 2026-02-24T22:04:08.199 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../14-python3-ceph-common_20.3.0-4957-gf7e05f4f-1jammy_all.deb ... 2026-02-24T22:04:08.199 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-ceph-common (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:08.218 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-wcwidth. 2026-02-24T22:04:08.228 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../15-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2026-02-24T22:04:08.228 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-24T22:04:08.244 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-prettytable. 2026-02-24T22:04:08.254 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../16-python3-prettytable_2.5.0-2_all.deb ... 2026-02-24T22:04:08.255 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2026-02-24T22:04:08.270 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-rbd. 2026-02-24T22:04:08.280 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../17-python3-rbd_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:04:08.280 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-rbd (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:08.297 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package librdkafka1:amd64. 2026-02-24T22:04:08.307 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../18-librdkafka1_1.8.0-1build1_amd64.deb ... 2026-02-24T22:04:08.308 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-24T22:04:08.326 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package librgw2. 2026-02-24T22:04:08.336 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../19-librgw2_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:04:08.337 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking librgw2 (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:08.434 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-rgw. 2026-02-24T22:04:08.444 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../20-python3-rgw_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:04:08.444 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-rgw (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:08.458 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package liboath0:amd64. 2026-02-24T22:04:08.468 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../21-liboath0_2.6.7-3ubuntu0.1_amd64.deb ... 2026-02-24T22:04:08.468 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-24T22:04:08.483 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libradosstriper1. 2026-02-24T22:04:08.493 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../22-libradosstriper1_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:04:08.494 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libradosstriper1 (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:08.511 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package ceph-common. 2026-02-24T22:04:08.520 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../23-ceph-common_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:04:08.521 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking ceph-common (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:08.778 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package ceph-base. 2026-02-24T22:04:08.788 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../24-ceph-base_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:04:08.793 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking ceph-base (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:08.853 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-jaraco.functools. 2026-02-24T22:04:08.863 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../25-python3-jaraco.functools_3.4.0-2_all.deb ... 2026-02-24T22:04:08.863 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2026-02-24T22:04:08.879 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-cheroot. 2026-02-24T22:04:08.889 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../26-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2026-02-24T22:04:08.889 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-24T22:04:08.906 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-jaraco.classes. 2026-02-24T22:04:08.916 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../27-python3-jaraco.classes_3.2.1-3_all.deb ... 2026-02-24T22:04:08.916 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2026-02-24T22:04:08.932 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-jaraco.text. 2026-02-24T22:04:08.942 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../28-python3-jaraco.text_3.6.0-2_all.deb ... 2026-02-24T22:04:08.942 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2026-02-24T22:04:08.957 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-jaraco.collections. 2026-02-24T22:04:08.967 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../29-python3-jaraco.collections_3.4.0-2_all.deb ... 2026-02-24T22:04:08.967 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2026-02-24T22:04:08.982 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-tempora. 2026-02-24T22:04:08.992 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../30-python3-tempora_4.1.2-1_all.deb ... 2026-02-24T22:04:08.993 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-tempora (4.1.2-1) ... 2026-02-24T22:04:09.008 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-portend. 2026-02-24T22:04:09.018 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../31-python3-portend_3.0.0-1_all.deb ... 2026-02-24T22:04:09.018 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-portend (3.0.0-1) ... 2026-02-24T22:04:09.033 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-zc.lockfile. 2026-02-24T22:04:09.043 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../32-python3-zc.lockfile_2.0-1_all.deb ... 2026-02-24T22:04:09.043 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2026-02-24T22:04:09.059 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-cherrypy3. 2026-02-24T22:04:09.068 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../33-python3-cherrypy3_18.6.1-4_all.deb ... 2026-02-24T22:04:09.069 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2026-02-24T22:04:09.093 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-natsort. 2026-02-24T22:04:09.103 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../34-python3-natsort_8.0.2-1_all.deb ... 2026-02-24T22:04:09.103 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-natsort (8.0.2-1) ... 2026-02-24T22:04:09.121 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2026-02-24T22:04:09.131 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../35-ceph-mgr-modules-core_20.3.0-4957-gf7e05f4f-1jammy_all.deb ... 2026-02-24T22:04:09.131 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking ceph-mgr-modules-core (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:09.158 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2026-02-24T22:04:09.168 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../36-libsqlite3-mod-ceph_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:04:09.169 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libsqlite3-mod-ceph (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:09.184 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package ceph-mgr. 2026-02-24T22:04:09.194 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../37-ceph-mgr_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:04:09.194 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking ceph-mgr (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:09.216 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package ceph-mon. 2026-02-24T22:04:09.226 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../38-ceph-mon_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:04:09.226 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking ceph-mon (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:09.293 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libfuse2:amd64. 2026-02-24T22:04:09.303 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../39-libfuse2_2.9.9-5ubuntu3_amd64.deb ... 2026-02-24T22:04:09.304 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-24T22:04:09.321 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package ceph-osd-classic. 2026-02-24T22:04:09.330 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../40-ceph-osd-classic_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:04:09.331 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking ceph-osd-classic (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:13.247 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package ceph-osd. 2026-02-24T22:04:13.262 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../41-ceph-osd_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:04:13.262 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking ceph-osd (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:13.282 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package ceph. 2026-02-24T22:04:13.297 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../42-ceph_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:04:13.297 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking ceph (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:13.314 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package ceph-fuse. 2026-02-24T22:04:13.329 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../43-ceph-fuse_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:04:13.329 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking ceph-fuse (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:13.353 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package ceph-mds. 2026-02-24T22:04:13.364 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../44-ceph-mds_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:04:13.365 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking ceph-mds (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:13.397 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package cephadm. 2026-02-24T22:04:13.408 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../45-cephadm_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:04:13.408 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking cephadm (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:13.425 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-asyncssh. 2026-02-24T22:04:13.435 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../46-python3-asyncssh_2.5.0-1ubuntu0.1_all.deb ... 2026-02-24T22:04:13.435 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-24T22:04:13.457 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2026-02-24T22:04:13.467 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../47-ceph-mgr-cephadm_20.3.0-4957-gf7e05f4f-1jammy_all.deb ... 2026-02-24T22:04:13.468 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking ceph-mgr-cephadm (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:13.489 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-repoze.lru. 2026-02-24T22:04:13.499 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../48-python3-repoze.lru_0.7-2_all.deb ... 2026-02-24T22:04:13.499 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2026-02-24T22:04:13.515 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-routes. 2026-02-24T22:04:13.524 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../49-python3-routes_2.5.1-1ubuntu1_all.deb ... 2026-02-24T22:04:13.525 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2026-02-24T22:04:13.546 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2026-02-24T22:04:13.555 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../50-ceph-mgr-dashboard_20.3.0-4957-gf7e05f4f-1jammy_all.deb ... 2026-02-24T22:04:13.556 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking ceph-mgr-dashboard (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:14.101 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2026-02-24T22:04:14.112 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../51-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2026-02-24T22:04:14.113 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-24T22:04:14.151 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-joblib. 2026-02-24T22:04:14.161 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../52-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2026-02-24T22:04:14.161 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2026-02-24T22:04:14.188 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-threadpoolctl. 2026-02-24T22:04:14.198 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../53-python3-threadpoolctl_3.1.0-1_all.deb ... 2026-02-24T22:04:14.199 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2026-02-24T22:04:14.214 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-sklearn. 2026-02-24T22:04:14.224 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../54-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2026-02-24T22:04:14.225 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-24T22:04:14.308 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2026-02-24T22:04:14.318 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../55-ceph-mgr-diskprediction-local_20.3.0-4957-gf7e05f4f-1jammy_all.deb ... 2026-02-24T22:04:14.319 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking ceph-mgr-diskprediction-local (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:14.486 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-cachetools. 2026-02-24T22:04:14.497 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../56-python3-cachetools_5.0.0-1_all.deb ... 2026-02-24T22:04:14.497 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2026-02-24T22:04:14.513 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-rsa. 2026-02-24T22:04:14.523 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../57-python3-rsa_4.8-1_all.deb ... 2026-02-24T22:04:14.523 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-rsa (4.8-1) ... 2026-02-24T22:04:14.541 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-google-auth. 2026-02-24T22:04:14.552 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../58-python3-google-auth_1.5.1-3_all.deb ... 2026-02-24T22:04:14.552 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2026-02-24T22:04:14.570 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-requests-oauthlib. 2026-02-24T22:04:14.581 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../59-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2026-02-24T22:04:14.581 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-24T22:04:14.597 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-websocket. 2026-02-24T22:04:14.607 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../60-python3-websocket_1.2.3-1_all.deb ... 2026-02-24T22:04:14.607 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-websocket (1.2.3-1) ... 2026-02-24T22:04:14.625 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-kubernetes. 2026-02-24T22:04:14.635 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../61-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2026-02-24T22:04:14.636 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-24T22:04:14.732 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2026-02-24T22:04:14.743 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../62-ceph-mgr-k8sevents_20.3.0-4957-gf7e05f4f-1jammy_all.deb ... 2026-02-24T22:04:14.743 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking ceph-mgr-k8sevents (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:14.759 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libonig5:amd64. 2026-02-24T22:04:14.769 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../63-libonig5_6.9.7.1-2build1_amd64.deb ... 2026-02-24T22:04:14.769 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-24T22:04:14.785 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libjq1:amd64. 2026-02-24T22:04:14.795 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../64-libjq1_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-24T22:04:14.795 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-24T22:04:14.811 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package jq. 2026-02-24T22:04:14.821 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../65-jq_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-24T22:04:14.821 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking jq (1.6-2.1ubuntu3.1) ... 2026-02-24T22:04:14.836 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package socat. 2026-02-24T22:04:14.846 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../66-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2026-02-24T22:04:14.846 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2026-02-24T22:04:14.866 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package xmlstarlet. 2026-02-24T22:04:14.876 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../67-xmlstarlet_1.6.1-2.1_amd64.deb ... 2026-02-24T22:04:14.876 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2026-02-24T22:04:14.912 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package ceph-test. 2026-02-24T22:04:14.922 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../68-ceph-test_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:04:14.923 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking ceph-test (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:15.549 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package ceph-volume. 2026-02-24T22:04:15.559 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../69-ceph-volume_20.3.0-4957-gf7e05f4f-1jammy_all.deb ... 2026-02-24T22:04:15.559 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking ceph-volume (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:15.582 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libcephfs-daemon. 2026-02-24T22:04:15.593 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../70-libcephfs-daemon_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:04:15.593 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libcephfs-daemon (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:15.608 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libcephfs-proxy2. 2026-02-24T22:04:15.619 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../71-libcephfs-proxy2_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:04:15.619 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libcephfs-proxy2 (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:15.634 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libcephfs-dev. 2026-02-24T22:04:15.644 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../72-libcephfs-dev_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:04:15.644 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libcephfs-dev (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:15.661 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package nvme-cli. 2026-02-24T22:04:15.671 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../73-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2026-02-24T22:04:15.672 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2026-02-24T22:04:15.704 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python-asyncssh-doc. 2026-02-24T22:04:15.714 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../74-python-asyncssh-doc_2.5.0-1ubuntu0.1_all.deb ... 2026-02-24T22:04:15.715 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-24T22:04:15.744 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-iniconfig. 2026-02-24T22:04:15.754 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../75-python3-iniconfig_1.1.1-2_all.deb ... 2026-02-24T22:04:15.755 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2026-02-24T22:04:15.769 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-pluggy. 2026-02-24T22:04:15.779 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../76-python3-pluggy_0.13.0-7.1_all.deb ... 2026-02-24T22:04:15.779 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2026-02-24T22:04:15.796 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-psutil. 2026-02-24T22:04:15.806 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../77-python3-psutil_5.9.0-1build1_amd64.deb ... 2026-02-24T22:04:15.807 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2026-02-24T22:04:15.826 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-py. 2026-02-24T22:04:15.836 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../78-python3-py_1.10.0-1_all.deb ... 2026-02-24T22:04:15.836 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-py (1.10.0-1) ... 2026-02-24T22:04:15.856 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-pygments. 2026-02-24T22:04:15.866 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../79-python3-pygments_2.11.2+dfsg-2ubuntu0.1_all.deb ... 2026-02-24T22:04:15.867 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-24T22:04:15.909 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-toml. 2026-02-24T22:04:15.919 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../80-python3-toml_0.10.2-1_all.deb ... 2026-02-24T22:04:15.920 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-toml (0.10.2-1) ... 2026-02-24T22:04:15.935 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-pytest. 2026-02-24T22:04:15.945 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../81-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2026-02-24T22:04:15.946 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2026-02-24T22:04:15.968 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-simplejson. 2026-02-24T22:04:15.979 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../82-python3-simplejson_3.17.6-1build1_amd64.deb ... 2026-02-24T22:04:15.979 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2026-02-24T22:04:16.004 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package python3-webob. 2026-02-24T22:04:16.015 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../83-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2026-02-24T22:04:16.015 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-24T22:04:16.033 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package qttranslations5-l10n. 2026-02-24T22:04:16.043 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../84-qttranslations5-l10n_5.15.3-1_all.deb ... 2026-02-24T22:04:16.043 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2026-02-24T22:04:16.111 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package radosgw. 2026-02-24T22:04:16.121 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../85-radosgw_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:04:16.122 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking radosgw (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:16.281 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package rbd-fuse. 2026-02-24T22:04:16.292 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../86-rbd-fuse_20.3.0-4957-gf7e05f4f-1jammy_amd64.deb ... 2026-02-24T22:04:16.292 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking rbd-fuse (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:16.320 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2026-02-24T22:04:16.385 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-24T22:04:16.386 INFO:teuthology.orchestra.run.trial046.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2026-02-24T22:04:16.456 INFO:teuthology.orchestra.run.trial046.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /lib/systemd/system/nvmefc-boot-connections.service. 2026-02-24T22:04:16.727 INFO:teuthology.orchestra.run.trial046.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service → /lib/systemd/system/nvmf-autoconnect.service. 2026-02-24T22:04:17.103 INFO:teuthology.orchestra.run.trial046.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2026-02-24T22:04:17.113 INFO:teuthology.orchestra.run.trial046.stdout:Setting up cephadm (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:17.155 INFO:teuthology.orchestra.run.trial046.stdout:Adding system user cephadm....done 2026-02-24T22:04:17.162 INFO:teuthology.orchestra.run.trial046.stdout:usermod: no changes 2026-02-24T22:04:17.167 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2026-02-24T22:04:17.240 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-24T22:04:17.241 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2026-02-24T22:04:17.315 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-repoze.lru (0.7-2) ... 2026-02-24T22:04:17.391 INFO:teuthology.orchestra.run.trial046.stdout:Setting up liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-24T22:04:17.392 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-py (1.10.0-1) ... 2026-02-24T22:04:17.486 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2026-02-24T22:04:17.610 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-cachetools (5.0.0-1) ... 2026-02-24T22:04:17.686 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2026-02-24T22:04:17.762 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-ceph-argparse (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:17.829 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-24T22:04:17.830 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libnbd0 (1.10.5-1) ... 2026-02-24T22:04:17.831 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-24T22:04:17.832 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-24T22:04:17.833 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2026-02-24T22:04:17.952 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-natsort (8.0.2-1) ... 2026-02-24T22:04:18.029 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libcephfs-proxy2 (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:18.030 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2026-02-24T22:04:18.106 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2026-02-24T22:04:18.193 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-24T22:04:18.438 INFO:teuthology.orchestra.run.trial046.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2026-02-24T22:04:18.439 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-24T22:04:18.534 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-24T22:04:18.658 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-24T22:04:18.740 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2026-02-24T22:04:18.815 INFO:teuthology.orchestra.run.trial046.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2026-02-24T22:04:18.816 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-ceph-common (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:18.906 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-24T22:04:19.402 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-24T22:04:19.406 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-toml (0.10.2-1) ... 2026-02-24T22:04:19.483 INFO:teuthology.orchestra.run.trial046.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-24T22:04:19.484 INFO:teuthology.orchestra.run.trial046.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2026-02-24T22:04:19.485 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2026-02-24T22:04:19.558 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2026-02-24T22:04:19.631 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-24T22:04:19.632 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-rsa (4.8-1) ... 2026-02-24T22:04:19.709 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-tempora (4.1.2-1) ... 2026-02-24T22:04:19.782 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-prettytable (2.5.0-2) ... 2026-02-24T22:04:19.858 INFO:teuthology.orchestra.run.trial046.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-24T22:04:19.859 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-websocket (1.2.3-1) ... 2026-02-24T22:04:19.942 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-24T22:04:19.943 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-24T22:04:20.019 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-24T22:04:20.102 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2026-02-24T22:04:20.177 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-24T22:04:20.178 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2026-02-24T22:04:20.313 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-portend (3.0.0-1) ... 2026-02-24T22:04:20.384 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-24T22:04:20.385 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-google-auth (1.5.1-3) ... 2026-02-24T22:04:20.465 INFO:teuthology.orchestra.run.trial046.stdout:Setting up jq (1.6-2.1ubuntu3.1) ... 2026-02-24T22:04:20.466 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2026-02-24T22:04:20.597 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-24T22:04:20.598 INFO:teuthology.orchestra.run.trial046.stdout:Setting up librados2 (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:20.599 INFO:teuthology.orchestra.run.trial046.stdout:Setting up librgw2 (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:20.600 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libsqlite3-mod-ceph (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:20.601 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-24T22:04:21.111 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libcephfs2 (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:21.112 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libradosstriper1 (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:21.113 INFO:teuthology.orchestra.run.trial046.stdout:Setting up librbd1 (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:21.114 INFO:teuthology.orchestra.run.trial046.stdout:Setting up ceph-mgr-modules-core (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:21.115 INFO:teuthology.orchestra.run.trial046.stdout:Setting up ceph-fuse (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:21.184 INFO:teuthology.orchestra.run.trial046.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-24T22:04:21.184 INFO:teuthology.orchestra.run.trial046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-24T22:04:21.562 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libcephfs-dev (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:21.563 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-rados (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:21.564 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libcephfs-daemon (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:21.565 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-rbd (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:21.566 INFO:teuthology.orchestra.run.trial046.stdout:Setting up rbd-fuse (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:21.567 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-rgw (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:21.567 INFO:teuthology.orchestra.run.trial046.stdout:Setting up python3-cephfs (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:21.568 INFO:teuthology.orchestra.run.trial046.stdout:Setting up ceph-common (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:21.596 INFO:teuthology.orchestra.run.trial046.stdout:Adding group ceph....done 2026-02-24T22:04:21.630 INFO:teuthology.orchestra.run.trial046.stdout:Adding system user ceph....done 2026-02-24T22:04:21.640 INFO:teuthology.orchestra.run.trial046.stdout:Setting system user ceph properties....done 2026-02-24T22:04:21.646 INFO:teuthology.orchestra.run.trial046.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2026-02-24T22:04:21.738 INFO:teuthology.orchestra.run.trial046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /lib/systemd/system/ceph.target. 2026-02-24T22:04:22.013 INFO:teuthology.orchestra.run.trial046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service → /lib/systemd/system/rbdmap.service. 2026-02-24T22:04:22.370 INFO:teuthology.orchestra.run.trial046.stdout:Setting up ceph-test (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:22.371 INFO:teuthology.orchestra.run.trial046.stdout:Setting up radosgw (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:22.618 INFO:teuthology.orchestra.run.trial046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-24T22:04:22.618 INFO:teuthology.orchestra.run.trial046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-24T22:04:22.973 INFO:teuthology.orchestra.run.trial046.stdout:Setting up ceph-base (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:23.085 INFO:teuthology.orchestra.run.trial046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /lib/systemd/system/ceph-crash.service. 2026-02-24T22:04:23.442 INFO:teuthology.orchestra.run.trial046.stdout:Setting up ceph-mds (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:23.502 INFO:teuthology.orchestra.run.trial046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-24T22:04:23.502 INFO:teuthology.orchestra.run.trial046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-24T22:04:23.847 INFO:teuthology.orchestra.run.trial046.stdout:Setting up ceph-mgr (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:23.923 INFO:teuthology.orchestra.run.trial046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-24T22:04:23.923 INFO:teuthology.orchestra.run.trial046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-24T22:04:24.296 INFO:teuthology.orchestra.run.trial046.stdout:Setting up ceph-mgr-k8sevents (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:24.297 INFO:teuthology.orchestra.run.trial046.stdout:Setting up ceph-osd-classic (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:24.314 INFO:teuthology.orchestra.run.trial046.stdout:update-alternatives: using /usr/bin/ceph-osd-classic to provide /usr/bin/ceph-osd (ceph-osd) in auto mode 2026-02-24T22:04:24.324 INFO:teuthology.orchestra.run.trial046.stdout:Setting up ceph-mgr-diskprediction-local (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:24.335 INFO:teuthology.orchestra.run.trial046.stdout:Setting up ceph-mon (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:24.408 INFO:teuthology.orchestra.run.trial046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-24T22:04:24.408 INFO:teuthology.orchestra.run.trial046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-24T22:04:24.781 INFO:teuthology.orchestra.run.trial046.stdout:Setting up ceph-mgr-cephadm (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:24.791 INFO:teuthology.orchestra.run.trial046.stdout:Setting up ceph-mgr-dashboard (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:24.802 INFO:teuthology.orchestra.run.trial046.stdout:Setting up ceph-osd (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:24.878 INFO:teuthology.orchestra.run.trial046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-24T22:04:24.878 INFO:teuthology.orchestra.run.trial046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-24T22:04:25.258 INFO:teuthology.orchestra.run.trial046.stdout:Setting up ceph (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:25.259 INFO:teuthology.orchestra.run.trial046.stdout:Setting up ceph-volume (20.3.0-4957-gf7e05f4f-1jammy) ... 2026-02-24T22:04:25.367 INFO:teuthology.orchestra.run.trial046.stdout:Processing triggers for man-db (2.10.2-1) ... 2026-02-24T22:04:25.431 INFO:teuthology.orchestra.run.trial046.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.13) ... 2026-02-24T22:04:26.099 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:26.099 INFO:teuthology.orchestra.run.trial046.stdout:Running kernel seems to be up-to-date. 2026-02-24T22:04:26.100 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:26.100 INFO:teuthology.orchestra.run.trial046.stdout:The processor microcode seems to be up-to-date. 2026-02-24T22:04:26.100 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:26.100 INFO:teuthology.orchestra.run.trial046.stdout:No services need to be restarted. 2026-02-24T22:04:26.100 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:26.100 INFO:teuthology.orchestra.run.trial046.stdout:No containers need to be restarted. 2026-02-24T22:04:26.100 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:26.100 INFO:teuthology.orchestra.run.trial046.stdout:No user sessions are running outdated binaries. 2026-02-24T22:04:26.100 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:26.100 INFO:teuthology.orchestra.run.trial046.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-24T22:04:26.910 INFO:teuthology.orchestra.run.trial046.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2026-02-24T22:04:26.913 DEBUG:teuthology.orchestra.run.trial046:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install s3cmd maven golang 2026-02-24T22:04:26.987 INFO:teuthology.orchestra.run.trial046.stdout:Reading package lists... 2026-02-24T22:04:27.094 INFO:teuthology.orchestra.run.trial046.stdout:Building dependency tree... 2026-02-24T22:04:27.095 INFO:teuthology.orchestra.run.trial046.stdout:Reading state information... 2026-02-24T22:04:27.183 INFO:teuthology.orchestra.run.trial046.stdout:The following packages were automatically installed and are no longer required: 2026-02-24T22:04:27.183 INFO:teuthology.orchestra.run.trial046.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-24T22:04:27.183 INFO:teuthology.orchestra.run.trial046.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 2026-02-24T22:04:27.183 INFO:teuthology.orchestra.run.trial046.stdout: linux-headers-5.15.0-164 linux-headers-5.15.0-164-generic 2026-02-24T22:04:27.183 INFO:teuthology.orchestra.run.trial046.stdout: linux-image-5.15.0-164-generic linux-modules-5.15.0-164-generic 2026-02-24T22:04:27.183 INFO:teuthology.orchestra.run.trial046.stdout: linux-modules-extra-5.15.0-164-generic sg3-utils sg3-utils-udev 2026-02-24T22:04:27.183 INFO:teuthology.orchestra.run.trial046.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-24T22:04:27.184 INFO:teuthology.orchestra.run.trial046.stdout:The following additional packages will be installed: 2026-02-24T22:04:27.184 INFO:teuthology.orchestra.run.trial046.stdout: golang-1.18 golang-1.18-doc golang-1.18-go golang-1.18-src golang-doc 2026-02-24T22:04:27.184 INFO:teuthology.orchestra.run.trial046.stdout: golang-go golang-src libaopalliance-java libapache-pom-java 2026-02-24T22:04:27.184 INFO:teuthology.orchestra.run.trial046.stdout: libatinject-jsr330-api-java libcdi-api-java libcommons-cli-java 2026-02-24T22:04:27.184 INFO:teuthology.orchestra.run.trial046.stdout: libcommons-io-java libcommons-lang3-java libcommons-parent-java 2026-02-24T22:04:27.184 INFO:teuthology.orchestra.run.trial046.stdout: libgeronimo-annotation-1.3-spec-java libgeronimo-interceptor-3.0-spec-java 2026-02-24T22:04:27.184 INFO:teuthology.orchestra.run.trial046.stdout: libguava-java libguice-java libhawtjni-runtime-java libjansi-java 2026-02-24T22:04:27.184 INFO:teuthology.orchestra.run.trial046.stdout: libjansi-native-java libjsr305-java libmaven-parent-java 2026-02-24T22:04:27.184 INFO:teuthology.orchestra.run.trial046.stdout: libmaven-resolver-java libmaven-shared-utils-java libmaven3-core-java 2026-02-24T22:04:27.184 INFO:teuthology.orchestra.run.trial046.stdout: libplexus-cipher-java libplexus-classworlds-java 2026-02-24T22:04:27.184 INFO:teuthology.orchestra.run.trial046.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2026-02-24T22:04:27.184 INFO:teuthology.orchestra.run.trial046.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2026-02-24T22:04:27.184 INFO:teuthology.orchestra.run.trial046.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2026-02-24T22:04:27.185 INFO:teuthology.orchestra.run.trial046.stdout: libwagon-http-shaded-java libwagon-provider-api-java pkg-config 2026-02-24T22:04:27.185 INFO:teuthology.orchestra.run.trial046.stdout:Suggested packages: 2026-02-24T22:04:27.185 INFO:teuthology.orchestra.run.trial046.stdout: bzr | brz mercurial subversion libaopalliance-java-doc 2026-02-24T22:04:27.185 INFO:teuthology.orchestra.run.trial046.stdout: libatinject-jsr330-api-java-doc libel-api-java libcommons-io-java-doc 2026-02-24T22:04:27.185 INFO:teuthology.orchestra.run.trial046.stdout: libcommons-lang3-java-doc libasm-java libcglib-java libjsr305-java-doc 2026-02-24T22:04:27.185 INFO:teuthology.orchestra.run.trial046.stdout: libmaven-shared-utils-java-doc liblogback-java 2026-02-24T22:04:27.185 INFO:teuthology.orchestra.run.trial046.stdout: libplexus-classworlds-java-doc libplexus-sec-dispatcher-java-doc 2026-02-24T22:04:27.185 INFO:teuthology.orchestra.run.trial046.stdout: libplexus-utils2-java-doc testng libcommons-logging-java liblog4j1.2-java 2026-02-24T22:04:27.197 INFO:teuthology.orchestra.run.trial046.stdout:The following NEW packages will be installed: 2026-02-24T22:04:27.198 INFO:teuthology.orchestra.run.trial046.stdout: golang golang-1.18 golang-1.18-doc golang-1.18-go golang-1.18-src golang-doc 2026-02-24T22:04:27.198 INFO:teuthology.orchestra.run.trial046.stdout: golang-go golang-src libaopalliance-java libapache-pom-java 2026-02-24T22:04:27.198 INFO:teuthology.orchestra.run.trial046.stdout: libatinject-jsr330-api-java libcdi-api-java libcommons-cli-java 2026-02-24T22:04:27.198 INFO:teuthology.orchestra.run.trial046.stdout: libcommons-io-java libcommons-lang3-java libcommons-parent-java 2026-02-24T22:04:27.198 INFO:teuthology.orchestra.run.trial046.stdout: libgeronimo-annotation-1.3-spec-java libgeronimo-interceptor-3.0-spec-java 2026-02-24T22:04:27.198 INFO:teuthology.orchestra.run.trial046.stdout: libguava-java libguice-java libhawtjni-runtime-java libjansi-java 2026-02-24T22:04:27.198 INFO:teuthology.orchestra.run.trial046.stdout: libjansi-native-java libjsr305-java libmaven-parent-java 2026-02-24T22:04:27.198 INFO:teuthology.orchestra.run.trial046.stdout: libmaven-resolver-java libmaven-shared-utils-java libmaven3-core-java 2026-02-24T22:04:27.198 INFO:teuthology.orchestra.run.trial046.stdout: libplexus-cipher-java libplexus-classworlds-java 2026-02-24T22:04:27.198 INFO:teuthology.orchestra.run.trial046.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2026-02-24T22:04:27.198 INFO:teuthology.orchestra.run.trial046.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2026-02-24T22:04:27.199 INFO:teuthology.orchestra.run.trial046.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2026-02-24T22:04:27.199 INFO:teuthology.orchestra.run.trial046.stdout: libwagon-http-shaded-java libwagon-provider-api-java maven pkg-config s3cmd 2026-02-24T22:04:27.327 INFO:teuthology.orchestra.run.trial046.stdout:0 upgraded, 43 newly installed, 0 to remove and 14 not upgraded. 2026-02-24T22:04:27.327 INFO:teuthology.orchestra.run.trial046.stdout:Need to get 92.9 MB of archives. 2026-02-24T22:04:27.328 INFO:teuthology.orchestra.run.trial046.stdout:After this operation, 452 MB of additional disk space will be used. 2026-02-24T22:04:27.328 INFO:teuthology.orchestra.run.trial046.stdout:Get:1 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 golang-1.18-doc all 1.18.1-1ubuntu1.2 [104 kB] 2026-02-24T22:04:27.365 INFO:teuthology.orchestra.run.trial046.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 golang-1.18-src all 1.18.1-1ubuntu1.2 [16.3 MB] 2026-02-24T22:04:27.517 INFO:teuthology.orchestra.run.trial046.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 golang-1.18-go amd64 1.18.1-1ubuntu1.2 [66.1 MB] 2026-02-24T22:04:27.904 INFO:teuthology.orchestra.run.trial046.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 golang-1.18 all 1.18.1-1ubuntu1.2 [11.2 kB] 2026-02-24T22:04:27.904 INFO:teuthology.orchestra.run.trial046.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy/main amd64 golang-src all 2:1.18~0ubuntu2 [4438 B] 2026-02-24T22:04:27.904 INFO:teuthology.orchestra.run.trial046.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy/main amd64 golang-go amd64 2:1.18~0ubuntu2 [41.8 kB] 2026-02-24T22:04:27.904 INFO:teuthology.orchestra.run.trial046.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy/main amd64 golang-doc all 2:1.18~0ubuntu2 [2784 B] 2026-02-24T22:04:27.905 INFO:teuthology.orchestra.run.trial046.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy/main amd64 golang amd64 2:1.18~0ubuntu2 [2732 B] 2026-02-24T22:04:27.905 INFO:teuthology.orchestra.run.trial046.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libapache-pom-java all 18-1 [4720 B] 2026-02-24T22:04:27.905 INFO:teuthology.orchestra.run.trial046.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libatinject-jsr330-api-java all 1.0+ds1-5 [5348 B] 2026-02-24T22:04:27.905 INFO:teuthology.orchestra.run.trial046.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-interceptor-3.0-spec-java all 1.0.1-4fakesync [8616 B] 2026-02-24T22:04:27.905 INFO:teuthology.orchestra.run.trial046.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libcdi-api-java all 1.2-3 [54.3 kB] 2026-02-24T22:04:27.913 INFO:teuthology.orchestra.run.trial046.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-cli-java all 1.4-2 [55.8 kB] 2026-02-24T22:04:27.915 INFO:teuthology.orchestra.run.trial046.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-parent-java all 43-1 [10.8 kB] 2026-02-24T22:04:27.915 INFO:teuthology.orchestra.run.trial046.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-io-java all 2.11.0-2 [297 kB] 2026-02-24T22:04:27.916 INFO:teuthology.orchestra.run.trial046.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-lang3-java all 3.11-1 [526 kB] 2026-02-24T22:04:27.920 INFO:teuthology.orchestra.run.trial046.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-annotation-1.3-spec-java all 1.3-1 [11.2 kB] 2026-02-24T22:04:27.920 INFO:teuthology.orchestra.run.trial046.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libjsr305-java all 0.1~+svn49-11 [27.0 kB] 2026-02-24T22:04:27.920 INFO:teuthology.orchestra.run.trial046.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libguava-java all 29.0-6 [2418 kB] 2026-02-24T22:04:27.936 INFO:teuthology.orchestra.run.trial046.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libaopalliance-java all 20070526-6 [9084 B] 2026-02-24T22:04:27.936 INFO:teuthology.orchestra.run.trial046.stdout:Get:21 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libguice-java all 4.2.3-2 [1434 kB] 2026-02-24T22:04:27.943 INFO:teuthology.orchestra.run.trial046.stdout:Get:22 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libhawtjni-runtime-java all 1.17-1 [28.8 kB] 2026-02-24T22:04:27.943 INFO:teuthology.orchestra.run.trial046.stdout:Get:23 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-native-java all 1.8-1 [23.8 kB] 2026-02-24T22:04:27.944 INFO:teuthology.orchestra.run.trial046.stdout:Get:24 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-java all 1.18-1 [56.8 kB] 2026-02-24T22:04:27.944 INFO:teuthology.orchestra.run.trial046.stdout:Get:25 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-parent-java all 31-2 [5140 B] 2026-02-24T22:04:27.944 INFO:teuthology.orchestra.run.trial046.stdout:Get:26 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-utils2-java all 3.3.0-1 [250 kB] 2026-02-24T22:04:27.946 INFO:teuthology.orchestra.run.trial046.stdout:Get:27 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-provider-api-java all 3.3.4-1 [48.5 kB] 2026-02-24T22:04:27.947 INFO:teuthology.orchestra.run.trial046.stdout:Get:28 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-resolver-java all 1.4.2-3build1 [555 kB] 2026-02-24T22:04:27.951 INFO:teuthology.orchestra.run.trial046.stdout:Get:29 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libmaven-shared-utils-java all 3.3.0-1ubuntu0.22.04.1 [148 kB] 2026-02-24T22:04:27.955 INFO:teuthology.orchestra.run.trial046.stdout:Get:30 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-cipher-java all 1.8-2 [15.1 kB] 2026-02-24T22:04:27.956 INFO:teuthology.orchestra.run.trial046.stdout:Get:31 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-classworlds-java all 2.6.0-1 [49.4 kB] 2026-02-24T22:04:27.956 INFO:teuthology.orchestra.run.trial046.stdout:Get:32 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-component-annotations-java all 2.1.0-1 [6564 B] 2026-02-24T22:04:27.957 INFO:teuthology.orchestra.run.trial046.stdout:Get:33 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-interpolation-java all 1.26-1 [76.8 kB] 2026-02-24T22:04:27.958 INFO:teuthology.orchestra.run.trial046.stdout:Get:34 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-sec-dispatcher-java all 1.4-4 [28.1 kB] 2026-02-24T22:04:27.958 INFO:teuthology.orchestra.run.trial046.stdout:Get:35 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libslf4j-java all 1.7.32-1 [141 kB] 2026-02-24T22:04:27.965 INFO:teuthology.orchestra.run.trial046.stdout:Get:36 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-inject-java all 0.3.4-2 [347 kB] 2026-02-24T22:04:27.966 INFO:teuthology.orchestra.run.trial046.stdout:Get:37 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-plexus-java all 0.3.4-3 [181 kB] 2026-02-24T22:04:27.967 INFO:teuthology.orchestra.run.trial046.stdout:Get:38 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven3-core-java all 3.6.3-5 [1535 kB] 2026-02-24T22:04:27.977 INFO:teuthology.orchestra.run.trial046.stdout:Get:39 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-file-java all 3.3.4-1 [7928 B] 2026-02-24T22:04:27.978 INFO:teuthology.orchestra.run.trial046.stdout:Get:40 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-http-shaded-java all 3.3.4-1 [1855 kB] 2026-02-24T22:04:27.989 INFO:teuthology.orchestra.run.trial046.stdout:Get:41 https://archive.ubuntu.com/ubuntu jammy/universe amd64 maven all 3.6.3-5 [17.8 kB] 2026-02-24T22:04:27.990 INFO:teuthology.orchestra.run.trial046.stdout:Get:42 https://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2026-02-24T22:04:27.990 INFO:teuthology.orchestra.run.trial046.stdout:Get:43 https://archive.ubuntu.com/ubuntu jammy/universe amd64 s3cmd all 2.2.0-1 [120 kB] 2026-02-24T22:04:28.218 INFO:teuthology.orchestra.run.trial046.stdout:Fetched 92.9 MB in 1s (118 MB/s) 2026-02-24T22:04:28.235 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package golang-1.18-doc. 2026-02-24T22:04:28.278 INFO:teuthology.orchestra.run.trial046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 210764 files and directories currently installed.) 2026-02-24T22:04:28.280 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../00-golang-1.18-doc_1.18.1-1ubuntu1.2_all.deb ... 2026-02-24T22:04:28.281 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking golang-1.18-doc (1.18.1-1ubuntu1.2) ... 2026-02-24T22:04:28.301 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package golang-1.18-src. 2026-02-24T22:04:28.311 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../01-golang-1.18-src_1.18.1-1ubuntu1.2_all.deb ... 2026-02-24T22:04:28.312 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking golang-1.18-src (1.18.1-1ubuntu1.2) ... 2026-02-24T22:04:29.342 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package golang-1.18-go. 2026-02-24T22:04:29.356 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../02-golang-1.18-go_1.18.1-1ubuntu1.2_amd64.deb ... 2026-02-24T22:04:29.357 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking golang-1.18-go (1.18.1-1ubuntu1.2) ... 2026-02-24T22:04:30.089 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package golang-1.18. 2026-02-24T22:04:30.100 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../03-golang-1.18_1.18.1-1ubuntu1.2_all.deb ... 2026-02-24T22:04:30.101 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking golang-1.18 (1.18.1-1ubuntu1.2) ... 2026-02-24T22:04:30.117 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package golang-src. 2026-02-24T22:04:30.128 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../04-golang-src_2%3a1.18~0ubuntu2_all.deb ... 2026-02-24T22:04:30.128 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking golang-src (2:1.18~0ubuntu2) ... 2026-02-24T22:04:30.144 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package golang-go:amd64. 2026-02-24T22:04:30.155 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../05-golang-go_2%3a1.18~0ubuntu2_amd64.deb ... 2026-02-24T22:04:30.157 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking golang-go:amd64 (2:1.18~0ubuntu2) ... 2026-02-24T22:04:30.175 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package golang-doc. 2026-02-24T22:04:30.186 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../06-golang-doc_2%3a1.18~0ubuntu2_all.deb ... 2026-02-24T22:04:30.187 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking golang-doc (2:1.18~0ubuntu2) ... 2026-02-24T22:04:30.202 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package golang:amd64. 2026-02-24T22:04:30.214 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../07-golang_2%3a1.18~0ubuntu2_amd64.deb ... 2026-02-24T22:04:30.214 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking golang:amd64 (2:1.18~0ubuntu2) ... 2026-02-24T22:04:30.229 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libapache-pom-java. 2026-02-24T22:04:30.240 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../08-libapache-pom-java_18-1_all.deb ... 2026-02-24T22:04:30.241 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libapache-pom-java (18-1) ... 2026-02-24T22:04:30.257 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libatinject-jsr330-api-java. 2026-02-24T22:04:30.269 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../09-libatinject-jsr330-api-java_1.0+ds1-5_all.deb ... 2026-02-24T22:04:30.269 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libatinject-jsr330-api-java (1.0+ds1-5) ... 2026-02-24T22:04:30.285 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libgeronimo-interceptor-3.0-spec-java. 2026-02-24T22:04:30.296 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../10-libgeronimo-interceptor-3.0-spec-java_1.0.1-4fakesync_all.deb ... 2026-02-24T22:04:30.297 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2026-02-24T22:04:30.312 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libcdi-api-java. 2026-02-24T22:04:30.323 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../11-libcdi-api-java_1.2-3_all.deb ... 2026-02-24T22:04:30.323 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libcdi-api-java (1.2-3) ... 2026-02-24T22:04:30.341 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libcommons-cli-java. 2026-02-24T22:04:30.352 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../12-libcommons-cli-java_1.4-2_all.deb ... 2026-02-24T22:04:30.353 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libcommons-cli-java (1.4-2) ... 2026-02-24T22:04:30.371 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libcommons-parent-java. 2026-02-24T22:04:30.382 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../13-libcommons-parent-java_43-1_all.deb ... 2026-02-24T22:04:30.382 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libcommons-parent-java (43-1) ... 2026-02-24T22:04:30.398 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libcommons-io-java. 2026-02-24T22:04:30.410 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../14-libcommons-io-java_2.11.0-2_all.deb ... 2026-02-24T22:04:30.410 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libcommons-io-java (2.11.0-2) ... 2026-02-24T22:04:30.426 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libcommons-lang3-java. 2026-02-24T22:04:30.438 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../15-libcommons-lang3-java_3.11-1_all.deb ... 2026-02-24T22:04:30.438 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libcommons-lang3-java (3.11-1) ... 2026-02-24T22:04:30.473 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libgeronimo-annotation-1.3-spec-java. 2026-02-24T22:04:30.486 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../16-libgeronimo-annotation-1.3-spec-java_1.3-1_all.deb ... 2026-02-24T22:04:30.487 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2026-02-24T22:04:30.502 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libjsr305-java. 2026-02-24T22:04:30.513 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../17-libjsr305-java_0.1~+svn49-11_all.deb ... 2026-02-24T22:04:30.513 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libjsr305-java (0.1~+svn49-11) ... 2026-02-24T22:04:30.532 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libguava-java. 2026-02-24T22:04:30.543 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../18-libguava-java_29.0-6_all.deb ... 2026-02-24T22:04:30.543 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libguava-java (29.0-6) ... 2026-02-24T22:04:30.646 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libaopalliance-java. 2026-02-24T22:04:30.663 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../19-libaopalliance-java_20070526-6_all.deb ... 2026-02-24T22:04:30.663 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libaopalliance-java (20070526-6) ... 2026-02-24T22:04:30.682 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libguice-java. 2026-02-24T22:04:30.698 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../20-libguice-java_4.2.3-2_all.deb ... 2026-02-24T22:04:30.698 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libguice-java (4.2.3-2) ... 2026-02-24T22:04:30.775 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libhawtjni-runtime-java. 2026-02-24T22:04:30.789 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../21-libhawtjni-runtime-java_1.17-1_all.deb ... 2026-02-24T22:04:30.789 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libhawtjni-runtime-java (1.17-1) ... 2026-02-24T22:04:30.809 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libjansi-native-java. 2026-02-24T22:04:30.825 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../22-libjansi-native-java_1.8-1_all.deb ... 2026-02-24T22:04:30.826 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libjansi-native-java (1.8-1) ... 2026-02-24T22:04:30.844 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libjansi-java. 2026-02-24T22:04:30.859 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../23-libjansi-java_1.18-1_all.deb ... 2026-02-24T22:04:30.859 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libjansi-java (1.18-1) ... 2026-02-24T22:04:30.878 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libmaven-parent-java. 2026-02-24T22:04:30.889 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../24-libmaven-parent-java_31-2_all.deb ... 2026-02-24T22:04:30.889 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libmaven-parent-java (31-2) ... 2026-02-24T22:04:30.905 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libplexus-utils2-java. 2026-02-24T22:04:30.916 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../25-libplexus-utils2-java_3.3.0-1_all.deb ... 2026-02-24T22:04:30.917 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libplexus-utils2-java (3.3.0-1) ... 2026-02-24T22:04:30.943 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libwagon-provider-api-java. 2026-02-24T22:04:30.954 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../26-libwagon-provider-api-java_3.3.4-1_all.deb ... 2026-02-24T22:04:30.954 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libwagon-provider-api-java (3.3.4-1) ... 2026-02-24T22:04:30.972 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libmaven-resolver-java. 2026-02-24T22:04:30.984 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../27-libmaven-resolver-java_1.4.2-3build1_all.deb ... 2026-02-24T22:04:30.984 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libmaven-resolver-java (1.4.2-3build1) ... 2026-02-24T22:04:31.025 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libmaven-shared-utils-java. 2026-02-24T22:04:31.038 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../28-libmaven-shared-utils-java_3.3.0-1ubuntu0.22.04.1_all.deb ... 2026-02-24T22:04:31.039 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libmaven-shared-utils-java (3.3.0-1ubuntu0.22.04.1) ... 2026-02-24T22:04:31.054 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libplexus-cipher-java. 2026-02-24T22:04:31.065 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../29-libplexus-cipher-java_1.8-2_all.deb ... 2026-02-24T22:04:31.065 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libplexus-cipher-java (1.8-2) ... 2026-02-24T22:04:31.082 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libplexus-classworlds-java. 2026-02-24T22:04:31.093 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../30-libplexus-classworlds-java_2.6.0-1_all.deb ... 2026-02-24T22:04:31.093 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libplexus-classworlds-java (2.6.0-1) ... 2026-02-24T22:04:31.111 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libplexus-component-annotations-java. 2026-02-24T22:04:31.122 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../31-libplexus-component-annotations-java_2.1.0-1_all.deb ... 2026-02-24T22:04:31.123 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libplexus-component-annotations-java (2.1.0-1) ... 2026-02-24T22:04:31.139 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libplexus-interpolation-java. 2026-02-24T22:04:31.150 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../32-libplexus-interpolation-java_1.26-1_all.deb ... 2026-02-24T22:04:31.150 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libplexus-interpolation-java (1.26-1) ... 2026-02-24T22:04:31.169 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libplexus-sec-dispatcher-java. 2026-02-24T22:04:31.180 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../33-libplexus-sec-dispatcher-java_1.4-4_all.deb ... 2026-02-24T22:04:31.180 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libplexus-sec-dispatcher-java (1.4-4) ... 2026-02-24T22:04:31.198 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libslf4j-java. 2026-02-24T22:04:31.208 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../34-libslf4j-java_1.7.32-1_all.deb ... 2026-02-24T22:04:31.209 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libslf4j-java (1.7.32-1) ... 2026-02-24T22:04:31.229 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libsisu-inject-java. 2026-02-24T22:04:31.240 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../35-libsisu-inject-java_0.3.4-2_all.deb ... 2026-02-24T22:04:31.240 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libsisu-inject-java (0.3.4-2) ... 2026-02-24T22:04:31.271 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libsisu-plexus-java. 2026-02-24T22:04:31.282 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../36-libsisu-plexus-java_0.3.4-3_all.deb ... 2026-02-24T22:04:31.282 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libsisu-plexus-java (0.3.4-3) ... 2026-02-24T22:04:31.305 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libmaven3-core-java. 2026-02-24T22:04:31.316 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../37-libmaven3-core-java_3.6.3-5_all.deb ... 2026-02-24T22:04:31.317 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libmaven3-core-java (3.6.3-5) ... 2026-02-24T22:04:31.398 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libwagon-file-java. 2026-02-24T22:04:31.415 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../38-libwagon-file-java_3.3.4-1_all.deb ... 2026-02-24T22:04:31.415 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libwagon-file-java (3.3.4-1) ... 2026-02-24T22:04:31.434 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package libwagon-http-shaded-java. 2026-02-24T22:04:31.449 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../39-libwagon-http-shaded-java_3.3.4-1_all.deb ... 2026-02-24T22:04:31.450 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking libwagon-http-shaded-java (3.3.4-1) ... 2026-02-24T22:04:31.531 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package maven. 2026-02-24T22:04:31.547 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../40-maven_3.6.3-5_all.deb ... 2026-02-24T22:04:31.548 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking maven (3.6.3-5) ... 2026-02-24T22:04:31.568 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package pkg-config. 2026-02-24T22:04:31.579 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../41-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2026-02-24T22:04:31.580 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2026-02-24T22:04:31.598 INFO:teuthology.orchestra.run.trial046.stdout:Selecting previously unselected package s3cmd. 2026-02-24T22:04:31.608 INFO:teuthology.orchestra.run.trial046.stdout:Preparing to unpack .../42-s3cmd_2.2.0-1_all.deb ... 2026-02-24T22:04:31.609 INFO:teuthology.orchestra.run.trial046.stdout:Unpacking s3cmd (2.2.0-1) ... 2026-02-24T22:04:31.642 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libslf4j-java (1.7.32-1) ... 2026-02-24T22:04:31.643 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libplexus-utils2-java (3.3.0-1) ... 2026-02-24T22:04:31.643 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libplexus-classworlds-java (2.6.0-1) ... 2026-02-24T22:04:31.644 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libjsr305-java (0.1~+svn49-11) ... 2026-02-24T22:04:31.645 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libaopalliance-java (20070526-6) ... 2026-02-24T22:04:31.646 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libcommons-cli-java (1.4-2) ... 2026-02-24T22:04:31.647 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libplexus-component-annotations-java (2.1.0-1) ... 2026-02-24T22:04:31.647 INFO:teuthology.orchestra.run.trial046.stdout:Setting up s3cmd (2.2.0-1) ... 2026-02-24T22:04:31.732 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libplexus-cipher-java (1.8-2) ... 2026-02-24T22:04:31.733 INFO:teuthology.orchestra.run.trial046.stdout:Setting up golang-1.18-doc (1.18.1-1ubuntu1.2) ... 2026-02-24T22:04:31.733 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2026-02-24T22:04:31.734 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2026-02-24T22:04:31.735 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libapache-pom-java (18-1) ... 2026-02-24T22:04:31.736 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libatinject-jsr330-api-java (1.0+ds1-5) ... 2026-02-24T22:04:31.737 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libplexus-interpolation-java (1.26-1) ... 2026-02-24T22:04:31.737 INFO:teuthology.orchestra.run.trial046.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2026-02-24T22:04:31.758 INFO:teuthology.orchestra.run.trial046.stdout:Setting up golang-1.18-src (1.18.1-1ubuntu1.2) ... 2026-02-24T22:04:31.759 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libplexus-sec-dispatcher-java (1.4-4) ... 2026-02-24T22:04:31.759 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libwagon-http-shaded-java (3.3.4-1) ... 2026-02-24T22:04:31.760 INFO:teuthology.orchestra.run.trial046.stdout:Setting up golang-src (2:1.18~0ubuntu2) ... 2026-02-24T22:04:31.761 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libcdi-api-java (1.2-3) ... 2026-02-24T22:04:31.762 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libhawtjni-runtime-java (1.17-1) ... 2026-02-24T22:04:31.763 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libwagon-provider-api-java (3.3.4-1) ... 2026-02-24T22:04:31.763 INFO:teuthology.orchestra.run.trial046.stdout:Setting up golang-1.18-go (1.18.1-1ubuntu1.2) ... 2026-02-24T22:04:31.764 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libmaven-parent-java (31-2) ... 2026-02-24T22:04:31.765 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libcommons-parent-java (43-1) ... 2026-02-24T22:04:31.766 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libsisu-inject-java (0.3.4-2) ... 2026-02-24T22:04:31.767 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libsisu-plexus-java (0.3.4-3) ... 2026-02-24T22:04:31.767 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libmaven-resolver-java (1.4.2-3build1) ... 2026-02-24T22:04:31.768 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libguava-java (29.0-6) ... 2026-02-24T22:04:31.769 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libcommons-lang3-java (3.11-1) ... 2026-02-24T22:04:31.770 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libjansi-native-java (1.8-1) ... 2026-02-24T22:04:31.771 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libwagon-file-java (3.3.4-1) ... 2026-02-24T22:04:31.771 INFO:teuthology.orchestra.run.trial046.stdout:Setting up golang-1.18 (1.18.1-1ubuntu1.2) ... 2026-02-24T22:04:31.772 INFO:teuthology.orchestra.run.trial046.stdout:Setting up golang-go:amd64 (2:1.18~0ubuntu2) ... 2026-02-24T22:04:31.773 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libcommons-io-java (2.11.0-2) ... 2026-02-24T22:04:31.774 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libguice-java (4.2.3-2) ... 2026-02-24T22:04:31.775 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libjansi-java (1.18-1) ... 2026-02-24T22:04:31.775 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libmaven-shared-utils-java (3.3.0-1ubuntu0.22.04.1) ... 2026-02-24T22:04:31.776 INFO:teuthology.orchestra.run.trial046.stdout:Setting up golang-doc (2:1.18~0ubuntu2) ... 2026-02-24T22:04:31.777 INFO:teuthology.orchestra.run.trial046.stdout:Setting up golang:amd64 (2:1.18~0ubuntu2) ... 2026-02-24T22:04:31.778 INFO:teuthology.orchestra.run.trial046.stdout:Setting up libmaven3-core-java (3.6.3-5) ... 2026-02-24T22:04:31.779 INFO:teuthology.orchestra.run.trial046.stdout:Setting up maven (3.6.3-5) ... 2026-02-24T22:04:31.783 INFO:teuthology.orchestra.run.trial046.stdout:update-alternatives: using /usr/share/maven/bin/mvn to provide /usr/bin/mvn (mvn) in auto mode 2026-02-24T22:04:31.784 INFO:teuthology.orchestra.run.trial046.stdout:Processing triggers for man-db (2.10.2-1) ... 2026-02-24T22:04:32.187 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:32.188 INFO:teuthology.orchestra.run.trial046.stdout:Running kernel seems to be up-to-date. 2026-02-24T22:04:32.188 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:32.188 INFO:teuthology.orchestra.run.trial046.stdout:The processor microcode seems to be up-to-date. 2026-02-24T22:04:32.188 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:32.188 INFO:teuthology.orchestra.run.trial046.stdout:No services need to be restarted. 2026-02-24T22:04:32.188 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:32.188 INFO:teuthology.orchestra.run.trial046.stdout:No containers need to be restarted. 2026-02-24T22:04:32.188 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:32.188 INFO:teuthology.orchestra.run.trial046.stdout:No user sessions are running outdated binaries. 2026-02-24T22:04:32.188 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:32.189 INFO:teuthology.orchestra.run.trial046.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-24T22:04:32.992 INFO:teuthology.orchestra.run.trial046.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2026-02-24T22:04:32.999 DEBUG:teuthology.parallel:result is None 2026-02-24T22:04:32.999 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=f7e05f4fdea5af173284f247d6b0935b337d4ccf 2026-02-24T22:04:33.178 DEBUG:teuthology.orchestra.run.trial046:> dpkg-query -W -f '${Version}' ceph 2026-02-24T22:04:33.190 INFO:teuthology.orchestra.run.trial046.stdout:20.3.0-4957-gf7e05f4f-1jammy 2026-02-24T22:04:33.190 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-4957-gf7e05f4f-1jammy 2026-02-24T22:04:33.191 INFO:teuthology.task.install:The correct ceph version 20.3.0-4957-gf7e05f4f-1jammy is installed. 2026-02-24T22:04:33.193 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=f7e05f4fdea5af173284f247d6b0935b337d4ccf 2026-02-24T22:04:33.340 DEBUG:teuthology.orchestra.run.trial110:> dpkg-query -W -f '${Version}' ceph 2026-02-24T22:04:33.352 INFO:teuthology.orchestra.run.trial110.stdout:20.3.0-4957-gf7e05f4f-1jammy 2026-02-24T22:04:33.352 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-4957-gf7e05f4f-1jammy 2026-02-24T22:04:33.352 INFO:teuthology.task.install:The correct ceph version 20.3.0-4957-gf7e05f4f-1jammy is installed. 2026-02-24T22:04:33.354 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-24T22:04:33.354 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:04:33.355 DEBUG:teuthology.orchestra.run.trial046:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-24T22:04:33.365 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T22:04:33.365 DEBUG:teuthology.orchestra.run.trial110:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-24T22:04:33.405 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-24T22:04:33.405 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:04:33.405 DEBUG:teuthology.orchestra.run.trial046:> sudo dd of=/usr/bin/daemon-helper 2026-02-24T22:04:33.416 DEBUG:teuthology.orchestra.run.trial046:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-24T22:04:33.466 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T22:04:33.466 DEBUG:teuthology.orchestra.run.trial110:> sudo dd of=/usr/bin/daemon-helper 2026-02-24T22:04:33.475 DEBUG:teuthology.orchestra.run.trial110:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-24T22:04:33.524 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-24T22:04:33.525 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:04:33.525 DEBUG:teuthology.orchestra.run.trial046:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-24T22:04:33.535 DEBUG:teuthology.orchestra.run.trial046:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-24T22:04:33.582 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T22:04:33.582 DEBUG:teuthology.orchestra.run.trial110:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-24T22:04:33.593 DEBUG:teuthology.orchestra.run.trial110:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-24T22:04:33.644 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-24T22:04:33.644 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:04:33.644 DEBUG:teuthology.orchestra.run.trial046:> sudo dd of=/usr/bin/stdin-killer 2026-02-24T22:04:33.653 DEBUG:teuthology.orchestra.run.trial046:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-24T22:04:33.702 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T22:04:33.702 DEBUG:teuthology.orchestra.run.trial110:> sudo dd of=/usr/bin/stdin-killer 2026-02-24T22:04:33.712 DEBUG:teuthology.orchestra.run.trial110:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-24T22:04:33.764 INFO:teuthology.run_tasks:Running task ceph... 2026-02-24T22:04:33.880 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-24T22:04:33.880 DEBUG:teuthology.orchestra.run.trial046:> sudo chmod 777 /var/log/ceph 2026-02-24T22:04:33.906 DEBUG:teuthology.orchestra.run.trial110:> sudo chmod 777 /var/log/ceph 2026-02-24T22:04:33.915 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-24T22:04:33.915 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-24T22:04:33.955 DEBUG:teuthology.orchestra.run.trial110:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-24T22:04:33.963 INFO:tasks.ceph:Creating extra log directories... 2026-02-24T22:04:33.963 DEBUG:teuthology.orchestra.run.trial046:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-24T22:04:34.004 DEBUG:teuthology.orchestra.run.trial110:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-24T22:04:34.030 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-24T22:04:34.030 INFO:tasks.ceph:config {'conf': {'client': {'debug rgw': 20, 'debug rgw notification': 20, 'lockdep': True, 'rgw bucket counters cache': True, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw d3n l1 datacache persistent path': '/tmp/rgw_datacache/', 'rgw d3n l1 datacache size': 10737418240, 'rgw d3n l1 local datacache enabled': True, 'rgw enable ops log': True, 'rgw max chunk size': 4194304, 'rgw obj stripe size': 6291456, 'rgw reshard progress judge interval': 10, 'rgw s3 auth use sts': True, 'rgw sts key': 'abcdefghijklmnop', 'rgw torrent flag': True, 'rgw user counters cache': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000, 'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10, 'rgw override bucket index max shards': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'lockdep': True}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug objclass': 20, 'debug osd': 20, 'debug rocksdb': '4/10', 'lockdep': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'rgw gc max queue size': 0}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled', '\\(OSD_SLOW_PING_TIME', 'reached quota', 'POOL_FULL', 'pool\\(s\\) full'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-24T22:04:34.030 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/suriarte-2026-02-24_21:43:07-rgw-wip-d4n-remove-bucket-distro-default-trial/68775', 'branch': 'wip-d4n-remove-bucket', 'description': 'rgw/verify/{0-install accounts$/{none} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off-no-gc} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{admin-pagination bucket-check cls mp_reupload rados-pool-quota ragweed reshard s3tests versioning zzz-s3tests-java} validater/lockdep}', 'email': None, 'first_in_suite': False, 'flavor': 'default', 'job_id': '68775', 'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'trial', 'name': 'suriarte-2026-02-24_21:43:07-rgw-wip-d4n-remove-bucket-distro-default-trial', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-d4n-remove-bucket'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'debug rgw notification': 20, 'lockdep': True, 'rgw bucket counters cache': True, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw d3n l1 datacache persistent path': '/tmp/rgw_datacache/', 'rgw d3n l1 datacache size': 10737418240, 'rgw d3n l1 local datacache enabled': True, 'rgw enable ops log': True, 'rgw max chunk size': 4194304, 'rgw obj stripe size': 6291456, 'rgw reshard progress judge interval': 10, 'rgw s3 auth use sts': True, 'rgw sts key': 'abcdefghijklmnop', 'rgw torrent flag': True, 'rgw user counters cache': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000, 'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10, 'rgw override bucket index max shards': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'lockdep': True}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug objclass': 20, 'debug osd': 20, 'debug rocksdb': '4/10', 'lockdep': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'rgw gc max queue size': 0}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled', '\\(OSD_SLOW_PING_TIME', 'reached quota', 'POOL_FULL', 'pool\\(s\\) full'], 'sha1': 'f7e05f4fdea5af173284f247d6b0935b337d4ccf'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'f7e05f4fdea5af173284f247d6b0935b337d4ccf'}}, 'openssl_keys': {'rgw.client.0': {'ca': 'root', 'client': 'client.0', 'embed-key': True}, 'root': {'client': 'client.0', 'cn': 'teuthology', 'install': ['client.0'], 'key-type': 'rsa:4096'}}, 'rgw': {'client.0': {'ssl certificate': 'rgw.client.0'}, 'compression type': 'random', 'datacache': True, 'datacache_path': '/tmp/rgw_datacache', 'ec-data-pool': False, 'frontend': 'beast', 'inline data': False, 'storage classes': {'FROZEN': None, 'LUKEWARM': None}}, 's3tests': {'accounts': {'iam alt root': 'RGW99999999999999999', 'iam root': 'RGW88888888888888888'}, 'force-branch': 'ceph-master', 'storage classes': 'LUKEWARM, FROZEN'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-d4n-remove-bucket', 'sha1': 'f7e05f4fdea5af173284f247d6b0935b337d4ccf'}}, 'owner': 'scheduled_suriarte@soko04.front.sepia.ceph.com', 'priority': 150, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 6673, 'sha1': 'f7e05f4fdea5af173284f247d6b0935b337d4ccf', 'sleep_before_teardown': 0, 'suite': 'rgw', 'suite_branch': 'wip-d4n-remove-bucket', 'suite_path': '/home/teuthworker/src/github.com_samarahu_ceph_f7e05f4fdea5af173284f247d6b0935b337d4ccf/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/samarahu/ceph.git', 'suite_sha1': 'f7e05f4fdea5af173284f247d6b0935b337d4ccf', 'targets': {'trial046.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJqPa3MRsJyiJog6c04vkzfx6nDOxInfdr4kGc1hiUs5cYBqZmzxwozxHjenOYhm3tYqHP0qtPrf7+XW4QFQKmM=', 'trial110.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBa0UOTuK+rY9wKdUikGx94zfBwP2Fv89fu8JG00sL0xR45EN+Jqu0bbptxVqaHVwtxeypYiQFhOiEzBL7Cc6sU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_system_packages': {'deb': ['s3cmd', 'maven', 'golang'], 'rpm': ['s3cmd', 'maven', 'golang']}, 'flavor': 'default', 'sha1': 'f7e05f4fdea5af173284f247d6b0935b337d4ccf'}}, {'ceph': None}, {'openssl_keys': None}, {'rgw': {'client.0': None}}, {'tox': ['client.0']}, {'workunit': {'clients': {'client.0': ['rgw/run-datacache.sh']}, 'env': {'RGW_DATACACHE_PATH': '/tmp/rgw_datacache'}}}, {'workunit': {'clients': {'client.0': ['rgw/run-admin-pagination.sh']}}}, {'workunit': {'clients': {'client.0': ['rgw/run-bucket-check.sh']}}}, {'workunit': {'clients': {'client.0': ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'cls/test_cls_rgw_gc.sh', 'cls/test_cls_rgw_stats.sh', 'cls/test_cls_cmpomap.sh', 'cls/test_cls_2pc_queue.sh', 'cls/test_cls_user.sh', 'cls/test_cls_sem_set.sh', 'rgw/test_rgw_gc_log.sh', 'rgw/test_rgw_obj.sh', 'rgw/test_rgw_datalog.sh', 'rgw/test_librgw_file.sh', 'rgw/test_awssdkv4_sig.sh', 'rgw/test_gosdk2.sh']}}}, {'workunit': {'clients': {'client.0': ['rgw/test_rgw_s3_mp_reupload.sh']}}}, {'workunit': {'clients': {'client.0': ['rgw/run-rados-pool-quota.sh']}}}, {'ragweed': {'client.0': {'default-branch': 'ceph-master', 'rgw_server': 'client.0', 'stages': 'prepare,check'}}}, {'workunit': {'clients': {'client.0': ['rgw/run-reshard.sh']}}}, {'s3tests': {'client.0': {'rgw_server': 'client.0'}}}, {'workunit': {'clients': {'client.0': ['rgw/run-versioning.sh']}}}, {'s3tests-java': {'client.0': {'force-branch': 'ceph-master', 'force-repo': 'https://github.com/ceph/java_s3tests.git'}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'fa17720d0088c3ac28e473468bfc79eeaff5cd38', 'timestamp': '2026-02-24_21:43:07', 'tube': 'trial', 'user': 'suriarte', 'verbose': True, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.904699'} 2026-02-24T22:04:34.031 DEBUG:teuthology.orchestra.run.trial046:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-24T22:04:34.052 DEBUG:teuthology.orchestra.run.trial110:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-24T22:04:34.066 DEBUG:teuthology.orchestra.run.trial046:> sudo install -d -m0777 -- /var/run/ceph 2026-02-24T22:04:34.104 DEBUG:teuthology.orchestra.run.trial110:> sudo install -d -m0777 -- /var/run/ceph 2026-02-24T22:04:34.116 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:04:34.116 DEBUG:teuthology.orchestra.run.trial046:> dd if=/scratch_devs of=/dev/stdout 2026-02-24T22:04:34.155 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-24T22:04:34.155 DEBUG:teuthology.orchestra.run.trial046:> stat /dev/vg_nvme/lv_1 2026-02-24T22:04:34.202 INFO:teuthology.orchestra.run.trial046.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-24T22:04:34.203 INFO:teuthology.orchestra.run.trial046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T22:04:34.203 INFO:teuthology.orchestra.run.trial046.stdout:Device: 5h/5d Inode: 899 Links: 1 2026-02-24T22:04:34.203 INFO:teuthology.orchestra.run.trial046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T22:04:34.203 INFO:teuthology.orchestra.run.trial046.stdout:Access: 2026-02-24 21:54:37.515270846 +0000 2026-02-24T22:04:34.203 INFO:teuthology.orchestra.run.trial046.stdout:Modify: 2026-02-24 21:54:37.351273633 +0000 2026-02-24T22:04:34.203 INFO:teuthology.orchestra.run.trial046.stdout:Change: 2026-02-24 21:54:37.351273633 +0000 2026-02-24T22:04:34.203 INFO:teuthology.orchestra.run.trial046.stdout: Birth: - 2026-02-24T22:04:34.204 DEBUG:teuthology.orchestra.run.trial046:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-24T22:04:34.250 INFO:teuthology.orchestra.run.trial046.stderr:1+0 records in 2026-02-24T22:04:34.250 INFO:teuthology.orchestra.run.trial046.stderr:1+0 records out 2026-02-24T22:04:34.250 INFO:teuthology.orchestra.run.trial046.stderr:512 bytes copied, 0.000203901 s, 2.5 MB/s 2026-02-24T22:04:34.251 DEBUG:teuthology.orchestra.run.trial046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-24T22:04:34.300 DEBUG:teuthology.orchestra.run.trial046:> stat /dev/vg_nvme/lv_2 2026-02-24T22:04:34.346 INFO:teuthology.orchestra.run.trial046.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-24T22:04:34.347 INFO:teuthology.orchestra.run.trial046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T22:04:34.347 INFO:teuthology.orchestra.run.trial046.stdout:Device: 5h/5d Inode: 925 Links: 1 2026-02-24T22:04:34.347 INFO:teuthology.orchestra.run.trial046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T22:04:34.347 INFO:teuthology.orchestra.run.trial046.stdout:Access: 2026-02-24 21:54:37.835265406 +0000 2026-02-24T22:04:34.347 INFO:teuthology.orchestra.run.trial046.stdout:Modify: 2026-02-24 21:54:37.683267990 +0000 2026-02-24T22:04:34.347 INFO:teuthology.orchestra.run.trial046.stdout:Change: 2026-02-24 21:54:37.683267990 +0000 2026-02-24T22:04:34.347 INFO:teuthology.orchestra.run.trial046.stdout: Birth: - 2026-02-24T22:04:34.348 DEBUG:teuthology.orchestra.run.trial046:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-24T22:04:34.393 INFO:teuthology.orchestra.run.trial046.stderr:1+0 records in 2026-02-24T22:04:34.394 INFO:teuthology.orchestra.run.trial046.stderr:1+0 records out 2026-02-24T22:04:34.394 INFO:teuthology.orchestra.run.trial046.stderr:512 bytes copied, 0.000170629 s, 3.0 MB/s 2026-02-24T22:04:34.395 DEBUG:teuthology.orchestra.run.trial046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-24T22:04:34.444 DEBUG:teuthology.orchestra.run.trial046:> stat /dev/vg_nvme/lv_3 2026-02-24T22:04:34.490 INFO:teuthology.orchestra.run.trial046.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-24T22:04:34.491 INFO:teuthology.orchestra.run.trial046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T22:04:34.491 INFO:teuthology.orchestra.run.trial046.stdout:Device: 5h/5d Inode: 946 Links: 1 2026-02-24T22:04:34.491 INFO:teuthology.orchestra.run.trial046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T22:04:34.491 INFO:teuthology.orchestra.run.trial046.stdout:Access: 2026-02-24 21:54:38.151260037 +0000 2026-02-24T22:04:34.491 INFO:teuthology.orchestra.run.trial046.stdout:Modify: 2026-02-24 21:54:37.991262756 +0000 2026-02-24T22:04:34.491 INFO:teuthology.orchestra.run.trial046.stdout:Change: 2026-02-24 21:54:37.991262756 +0000 2026-02-24T22:04:34.491 INFO:teuthology.orchestra.run.trial046.stdout: Birth: - 2026-02-24T22:04:34.491 DEBUG:teuthology.orchestra.run.trial046:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-24T22:04:34.537 INFO:teuthology.orchestra.run.trial046.stderr:1+0 records in 2026-02-24T22:04:34.538 INFO:teuthology.orchestra.run.trial046.stderr:1+0 records out 2026-02-24T22:04:34.538 INFO:teuthology.orchestra.run.trial046.stderr:512 bytes copied, 0.000180247 s, 2.8 MB/s 2026-02-24T22:04:34.539 DEBUG:teuthology.orchestra.run.trial046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-24T22:04:34.588 DEBUG:teuthology.orchestra.run.trial046:> stat /dev/vg_nvme/lv_4 2026-02-24T22:04:34.634 INFO:teuthology.orchestra.run.trial046.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-24T22:04:34.635 INFO:teuthology.orchestra.run.trial046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T22:04:34.635 INFO:teuthology.orchestra.run.trial046.stdout:Device: 5h/5d Inode: 972 Links: 1 2026-02-24T22:04:34.635 INFO:teuthology.orchestra.run.trial046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T22:04:34.635 INFO:teuthology.orchestra.run.trial046.stdout:Access: 2026-02-24 21:54:38.483254393 +0000 2026-02-24T22:04:34.635 INFO:teuthology.orchestra.run.trial046.stdout:Modify: 2026-02-24 21:54:38.319257181 +0000 2026-02-24T22:04:34.635 INFO:teuthology.orchestra.run.trial046.stdout:Change: 2026-02-24 21:54:38.319257181 +0000 2026-02-24T22:04:34.635 INFO:teuthology.orchestra.run.trial046.stdout: Birth: - 2026-02-24T22:04:34.636 DEBUG:teuthology.orchestra.run.trial046:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-24T22:04:34.682 INFO:teuthology.orchestra.run.trial046.stderr:1+0 records in 2026-02-24T22:04:34.682 INFO:teuthology.orchestra.run.trial046.stderr:1+0 records out 2026-02-24T22:04:34.682 INFO:teuthology.orchestra.run.trial046.stderr:512 bytes copied, 0.000174747 s, 2.9 MB/s 2026-02-24T22:04:34.683 DEBUG:teuthology.orchestra.run.trial046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-24T22:04:34.732 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-24T22:04:34.732 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T22:04:34.732 DEBUG:teuthology.orchestra.run.trial110:> dd if=/scratch_devs of=/dev/stdout 2026-02-24T22:04:34.738 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-24T22:04:34.738 DEBUG:teuthology.orchestra.run.trial110:> stat /dev/vg_nvme/lv_1 2026-02-24T22:04:34.784 INFO:teuthology.orchestra.run.trial110.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-24T22:04:34.784 INFO:teuthology.orchestra.run.trial110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T22:04:34.784 INFO:teuthology.orchestra.run.trial110.stdout:Device: 5h/5d Inode: 905 Links: 1 2026-02-24T22:04:34.784 INFO:teuthology.orchestra.run.trial110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T22:04:34.784 INFO:teuthology.orchestra.run.trial110.stdout:Access: 2026-02-24 21:54:37.927853137 +0000 2026-02-24T22:04:34.784 INFO:teuthology.orchestra.run.trial110.stdout:Modify: 2026-02-24 21:54:37.923853206 +0000 2026-02-24T22:04:34.784 INFO:teuthology.orchestra.run.trial110.stdout:Change: 2026-02-24 21:54:37.923853206 +0000 2026-02-24T22:04:34.784 INFO:teuthology.orchestra.run.trial110.stdout: Birth: - 2026-02-24T22:04:34.785 DEBUG:teuthology.orchestra.run.trial110:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-24T22:04:34.831 INFO:teuthology.orchestra.run.trial110.stderr:1+0 records in 2026-02-24T22:04:34.831 INFO:teuthology.orchestra.run.trial110.stderr:1+0 records out 2026-02-24T22:04:34.832 INFO:teuthology.orchestra.run.trial110.stderr:512 bytes copied, 0.000176137 s, 2.9 MB/s 2026-02-24T22:04:34.833 DEBUG:teuthology.orchestra.run.trial110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-24T22:04:34.882 DEBUG:teuthology.orchestra.run.trial110:> stat /dev/vg_nvme/lv_2 2026-02-24T22:04:34.928 INFO:teuthology.orchestra.run.trial110.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-24T22:04:34.928 INFO:teuthology.orchestra.run.trial110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T22:04:34.928 INFO:teuthology.orchestra.run.trial110.stdout:Device: 5h/5d Inode: 941 Links: 1 2026-02-24T22:04:34.928 INFO:teuthology.orchestra.run.trial110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T22:04:34.928 INFO:teuthology.orchestra.run.trial110.stdout:Access: 2026-02-24 21:54:38.243847752 +0000 2026-02-24T22:04:34.928 INFO:teuthology.orchestra.run.trial110.stdout:Modify: 2026-02-24 21:54:38.239847822 +0000 2026-02-24T22:04:34.928 INFO:teuthology.orchestra.run.trial110.stdout:Change: 2026-02-24 21:54:38.239847822 +0000 2026-02-24T22:04:34.928 INFO:teuthology.orchestra.run.trial110.stdout: Birth: - 2026-02-24T22:04:34.929 DEBUG:teuthology.orchestra.run.trial110:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-24T22:04:34.976 INFO:teuthology.orchestra.run.trial110.stderr:1+0 records in 2026-02-24T22:04:34.976 INFO:teuthology.orchestra.run.trial110.stderr:1+0 records out 2026-02-24T22:04:34.976 INFO:teuthology.orchestra.run.trial110.stderr:512 bytes copied, 0.000195954 s, 2.6 MB/s 2026-02-24T22:04:34.978 DEBUG:teuthology.orchestra.run.trial110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-24T22:04:35.025 DEBUG:teuthology.orchestra.run.trial110:> stat /dev/vg_nvme/lv_3 2026-02-24T22:04:35.072 INFO:teuthology.orchestra.run.trial110.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-24T22:04:35.072 INFO:teuthology.orchestra.run.trial110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T22:04:35.072 INFO:teuthology.orchestra.run.trial110.stdout:Device: 5h/5d Inode: 965 Links: 1 2026-02-24T22:04:35.072 INFO:teuthology.orchestra.run.trial110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T22:04:35.072 INFO:teuthology.orchestra.run.trial110.stdout:Access: 2026-02-24 21:54:38.707839848 +0000 2026-02-24T22:04:35.072 INFO:teuthology.orchestra.run.trial110.stdout:Modify: 2026-02-24 21:54:38.559842369 +0000 2026-02-24T22:04:35.072 INFO:teuthology.orchestra.run.trial110.stdout:Change: 2026-02-24 21:54:38.559842369 +0000 2026-02-24T22:04:35.072 INFO:teuthology.orchestra.run.trial110.stdout: Birth: - 2026-02-24T22:04:35.073 DEBUG:teuthology.orchestra.run.trial110:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-24T22:04:35.119 INFO:teuthology.orchestra.run.trial110.stderr:1+0 records in 2026-02-24T22:04:35.119 INFO:teuthology.orchestra.run.trial110.stderr:1+0 records out 2026-02-24T22:04:35.120 INFO:teuthology.orchestra.run.trial110.stderr:512 bytes copied, 0.000185414 s, 2.8 MB/s 2026-02-24T22:04:35.121 DEBUG:teuthology.orchestra.run.trial110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-24T22:04:35.173 DEBUG:teuthology.orchestra.run.trial110:> stat /dev/vg_nvme/lv_4 2026-02-24T22:04:35.220 INFO:teuthology.orchestra.run.trial110.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-24T22:04:35.220 INFO:teuthology.orchestra.run.trial110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T22:04:35.220 INFO:teuthology.orchestra.run.trial110.stdout:Device: 5h/5d Inode: 976 Links: 1 2026-02-24T22:04:35.220 INFO:teuthology.orchestra.run.trial110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T22:04:35.220 INFO:teuthology.orchestra.run.trial110.stdout:Access: 2026-02-24 21:54:39.027834399 +0000 2026-02-24T22:04:35.220 INFO:teuthology.orchestra.run.trial110.stdout:Modify: 2026-02-24 21:54:38.867837124 +0000 2026-02-24T22:04:35.220 INFO:teuthology.orchestra.run.trial110.stdout:Change: 2026-02-24 21:54:38.867837124 +0000 2026-02-24T22:04:35.220 INFO:teuthology.orchestra.run.trial110.stdout: Birth: - 2026-02-24T22:04:35.221 DEBUG:teuthology.orchestra.run.trial110:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-24T22:04:35.271 INFO:teuthology.orchestra.run.trial110.stderr:1+0 records in 2026-02-24T22:04:35.271 INFO:teuthology.orchestra.run.trial110.stderr:1+0 records out 2026-02-24T22:04:35.271 INFO:teuthology.orchestra.run.trial110.stderr:512 bytes copied, 0.000175085 s, 2.9 MB/s 2026-02-24T22:04:35.272 DEBUG:teuthology.orchestra.run.trial110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-24T22:04:35.321 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2026-02-24T22:04:35.321 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial046.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@trial110.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2026-02-24T22:04:35.321 INFO:tasks.ceph:Generating config... 2026-02-24T22:04:35.324 INFO:tasks.ceph:[client] debug rgw = 20 2026-02-24T22:04:35.324 INFO:tasks.ceph:[client] debug rgw notification = 20 2026-02-24T22:04:35.324 INFO:tasks.ceph:[client] lockdep = True 2026-02-24T22:04:35.324 INFO:tasks.ceph:[client] rgw bucket counters cache = True 2026-02-24T22:04:35.324 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2026-02-24T22:04:35.324 INFO:tasks.ceph:[client] rgw crypt s3 kms backend = testing 2026-02-24T22:04:35.324 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2026-02-24T22:04:35.324 INFO:tasks.ceph:[client] rgw d3n l1 datacache persistent path = /tmp/rgw_datacache/ 2026-02-24T22:04:35.325 INFO:tasks.ceph:[client] rgw d3n l1 datacache size = 10737418240 2026-02-24T22:04:35.325 INFO:tasks.ceph:[client] rgw d3n l1 local datacache enabled = True 2026-02-24T22:04:35.325 INFO:tasks.ceph:[client] rgw enable ops log = True 2026-02-24T22:04:35.325 INFO:tasks.ceph:[client] rgw max chunk size = 4194304 2026-02-24T22:04:35.325 INFO:tasks.ceph:[client] rgw obj stripe size = 6291456 2026-02-24T22:04:35.325 INFO:tasks.ceph:[client] rgw reshard progress judge interval = 10 2026-02-24T22:04:35.325 INFO:tasks.ceph:[client] rgw s3 auth use sts = True 2026-02-24T22:04:35.325 INFO:tasks.ceph:[client] rgw sts key = abcdefghijklmnop 2026-02-24T22:04:35.325 INFO:tasks.ceph:[client] rgw torrent flag = True 2026-02-24T22:04:35.325 INFO:tasks.ceph:[client] rgw user counters cache = True 2026-02-24T22:04:35.325 INFO:tasks.ceph:[client] setgroup = ceph 2026-02-24T22:04:35.325 INFO:tasks.ceph:[client] setuser = ceph 2026-02-24T22:04:35.326 INFO:tasks.ceph:[global] mon client directed command retry = 5 2026-02-24T22:04:35.326 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2026-02-24T22:04:35.326 INFO:tasks.ceph:[global] osd_max_pg_log_entries = 10 2026-02-24T22:04:35.326 INFO:tasks.ceph:[global] osd_min_pg_log_entries = 10 2026-02-24T22:04:35.326 INFO:tasks.ceph:[global] rgw override bucket index max shards = 1 2026-02-24T22:04:35.326 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-24T22:04:35.326 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-24T22:04:35.326 INFO:tasks.ceph:[mon] debug mon = 20 2026-02-24T22:04:35.326 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-24T22:04:35.326 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-24T22:04:35.326 INFO:tasks.ceph:[mon] lockdep = True 2026-02-24T22:04:35.326 INFO:tasks.ceph:[osd] bdev async discard = True 2026-02-24T22:04:35.326 INFO:tasks.ceph:[osd] bdev enable discard = True 2026-02-24T22:04:35.327 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2026-02-24T22:04:35.327 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2026-02-24T22:04:35.327 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2026-02-24T22:04:35.327 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2026-02-24T22:04:35.327 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2026-02-24T22:04:35.327 INFO:tasks.ceph:[osd] debug ms = 1 2026-02-24T22:04:35.327 INFO:tasks.ceph:[osd] debug objclass = 20 2026-02-24T22:04:35.327 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-24T22:04:35.327 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2026-02-24T22:04:35.327 INFO:tasks.ceph:[osd] lockdep = True 2026-02-24T22:04:35.327 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2026-02-24T22:04:35.327 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2026-02-24T22:04:35.328 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2026-02-24T22:04:35.328 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2026-02-24T22:04:35.328 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2026-02-24T22:04:35.328 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-24T22:04:35.328 INFO:tasks.ceph:[osd] rgw gc max queue size = 0 2026-02-24T22:04:35.328 INFO:tasks.ceph:Setting up mon.a... 2026-02-24T22:04:35.328 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-24T22:04:35.346 INFO:teuthology.orchestra.run.trial046.stdout:creating /etc/ceph/ceph.keyring 2026-02-24T22:04:35.349 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-24T22:04:35.410 DEBUG:teuthology.orchestra.run.trial046:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-24T22:04:35.458 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.46'), ('mon.c', '[v2:10.20.193.46:3301,v1:10.20.193.46:6790]'), ('mon.b', '10.20.193.110')] 2026-02-24T22:04:35.459 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.46,[v2:10.20.193.46:3301,v1:10.20.193.46:6790],10.20.193.110', 'mon client directed command retry': 5, 'ms inject socket failures': 5000, 'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10, 'rgw override bucket index max shards': 1}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug objclass': 20, 'debug osd': 20, 'debug rocksdb': '4/10', 'lockdep': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'rgw gc max queue size': 0}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'lockdep': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': True, 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rgw': 20, 'debug rgw notification': 20, 'lockdep': True, 'rgw bucket counters cache': True, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw d3n l1 datacache persistent path': '/tmp/rgw_datacache/', 'rgw d3n l1 datacache size': 10737418240, 'rgw d3n l1 local datacache enabled': True, 'rgw max chunk size': 4194304, 'rgw obj stripe size': 6291456, 'rgw reshard progress judge interval': 10, 'rgw s3 auth use sts': True, 'rgw sts key': 'abcdefghijklmnop', 'rgw torrent flag': True, 'rgw user counters cache': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2026-02-24T22:04:35.460 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:04:35.461 DEBUG:teuthology.orchestra.run.trial046:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-24T22:04:35.503 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.46 --addv c '[v2:10.20.193.46:3301,v1:10.20.193.46:6790]' --add b 10.20.193.110 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-24T22:04:35.565 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:04:35.565 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:04:35.565 INFO:teuthology.orchestra.run.trial046.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-24T22:04:35.566 INFO:teuthology.orchestra.run.trial046.stdout:monmaptool: generated fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:04:35.566 INFO:teuthology.orchestra.run.trial046.stdout:setting min_mon_release = tentacle 2026-02-24T22:04:35.566 INFO:teuthology.orchestra.run.trial046.stdout:epoch 0 2026-02-24T22:04:35.566 INFO:teuthology.orchestra.run.trial046.stdout:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:04:35.566 INFO:teuthology.orchestra.run.trial046.stdout:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:04:35.566 INFO:teuthology.orchestra.run.trial046.stdout:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:04:35.566 INFO:teuthology.orchestra.run.trial046.stdout:min_mon_release 20 (tentacle) 2026-02-24T22:04:35.566 INFO:teuthology.orchestra.run.trial046.stdout:election_strategy: 1 2026-02-24T22:04:35.566 INFO:teuthology.orchestra.run.trial046.stdout:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:04:35.566 INFO:teuthology.orchestra.run.trial046.stdout:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:04:35.567 INFO:teuthology.orchestra.run.trial046.stdout:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:04:35.567 INFO:teuthology.orchestra.run.trial046.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2026-02-24T22:04:35.568 DEBUG:teuthology.orchestra.run.trial046:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-24T22:04:35.614 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 262e7508-9310-4d2a-bc05-2c8861262eff... 2026-02-24T22:04:35.616 DEBUG:teuthology.orchestra.run.trial046:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-24T22:04:35.656 DEBUG:teuthology.orchestra.run.trial110:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-24T22:04:35.675 INFO:teuthology.orchestra.run.trial046.stdout:[global] 2026-02-24T22:04:35.675 INFO:teuthology.orchestra.run.trial046.stdout: chdir = "" 2026-02-24T22:04:35.676 INFO:teuthology.orchestra.run.trial046.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-24T22:04:35.676 INFO:teuthology.orchestra.run.trial046.stdout: auth supported = cephx 2026-02-24T22:04:35.676 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.676 INFO:teuthology.orchestra.run.trial046.stdout: filestore xattr use omap = true 2026-02-24T22:04:35.676 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.676 INFO:teuthology.orchestra.run.trial046.stdout: mon clock drift allowed = 1.000 2026-02-24T22:04:35.676 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.676 INFO:teuthology.orchestra.run.trial046.stdout: osd crush chooseleaf type = 0 2026-02-24T22:04:35.676 INFO:teuthology.orchestra.run.trial046.stdout: auth debug = true 2026-02-24T22:04:35.676 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.677 INFO:teuthology.orchestra.run.trial046.stdout: ms die on old message = true 2026-02-24T22:04:35.677 INFO:teuthology.orchestra.run.trial046.stdout: ms die on bug = true 2026-02-24T22:04:35.677 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.677 INFO:teuthology.orchestra.run.trial046.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-24T22:04:35.677 INFO:teuthology.orchestra.run.trial046.stdout: mon pg warn max object skew = 0 2026-02-24T22:04:35.677 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.677 INFO:teuthology.orchestra.run.trial046.stdout: # disable pg_autoscaler by default for new pools 2026-02-24T22:04:35.677 INFO:teuthology.orchestra.run.trial046.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-24T22:04:35.677 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.677 INFO:teuthology.orchestra.run.trial046.stdout: osd pool default size = 2 2026-02-24T22:04:35.677 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.677 INFO:teuthology.orchestra.run.trial046.stdout: mon osd allow primary affinity = true 2026-02-24T22:04:35.678 INFO:teuthology.orchestra.run.trial046.stdout: mon osd allow pg remap = true 2026-02-24T22:04:35.678 INFO:teuthology.orchestra.run.trial046.stdout: mon warn on legacy crush tunables = false 2026-02-24T22:04:35.678 INFO:teuthology.orchestra.run.trial046.stdout: mon warn on crush straw calc version zero = false 2026-02-24T22:04:35.678 INFO:teuthology.orchestra.run.trial046.stdout: mon warn on no sortbitwise = false 2026-02-24T22:04:35.678 INFO:teuthology.orchestra.run.trial046.stdout: mon warn on osd down out interval zero = false 2026-02-24T22:04:35.678 INFO:teuthology.orchestra.run.trial046.stdout: mon warn on too few osds = false 2026-02-24T22:04:35.678 INFO:teuthology.orchestra.run.trial046.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-24T22:04:35.678 INFO:teuthology.orchestra.run.trial046.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-24T22:04:35.678 INFO:teuthology.orchestra.run.trial046.stdout: mon_allow_pool_size_one = true 2026-02-24T22:04:35.678 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.678 INFO:teuthology.orchestra.run.trial046.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-24T22:04:35.678 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.679 INFO:teuthology.orchestra.run.trial046.stdout: osd default data pool replay window = 5 2026-02-24T22:04:35.679 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.679 INFO:teuthology.orchestra.run.trial046.stdout: mon allow pool delete = true 2026-02-24T22:04:35.679 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.679 INFO:teuthology.orchestra.run.trial046.stdout: mon cluster log file level = debug 2026-02-24T22:04:35.679 INFO:teuthology.orchestra.run.trial046.stdout: debug asserts on shutdown = true 2026-02-24T22:04:35.679 INFO:teuthology.orchestra.run.trial046.stdout: mon health detail to clog = false 2026-02-24T22:04:35.679 INFO:teuthology.orchestra.run.trial046.stdout: mon host = "10.20.193.46,[v2:10.20.193.46:3301,v1:10.20.193.46:6790],10.20.193.110" 2026-02-24T22:04:35.679 INFO:teuthology.orchestra.run.trial046.stdout: mon client directed command retry = 5 2026-02-24T22:04:35.679 INFO:teuthology.orchestra.run.trial046.stdout: ms inject socket failures = 5000 2026-02-24T22:04:35.679 INFO:teuthology.orchestra.run.trial046.stdout: osd_max_pg_log_entries = 10 2026-02-24T22:04:35.679 INFO:teuthology.orchestra.run.trial046.stdout: osd_min_pg_log_entries = 10 2026-02-24T22:04:35.680 INFO:teuthology.orchestra.run.trial046.stdout: rgw override bucket index max shards = 1 2026-02-24T22:04:35.680 INFO:teuthology.orchestra.run.trial046.stdout: fsid = 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:04:35.680 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.680 INFO:teuthology.orchestra.run.trial046.stdout:[osd] 2026-02-24T22:04:35.680 INFO:teuthology.orchestra.run.trial046.stdout: osd journal size = 100 2026-02-24T22:04:35.680 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.680 INFO:teuthology.orchestra.run.trial046.stdout: osd scrub load threshold = 5.0 2026-02-24T22:04:35.680 INFO:teuthology.orchestra.run.trial046.stdout: osd scrub max interval = 600 2026-02-24T22:04:35.680 INFO:teuthology.orchestra.run.trial046.stdout: osd mclock profile = high_recovery_ops 2026-02-24T22:04:35.680 INFO:teuthology.orchestra.run.trial046.stdout: osd mclock skip benchmark = true 2026-02-24T22:04:35.680 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.680 INFO:teuthology.orchestra.run.trial046.stdout: osd recover clone overlap = true 2026-02-24T22:04:35.681 INFO:teuthology.orchestra.run.trial046.stdout: osd recovery max chunk = 1048576 2026-02-24T22:04:35.681 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.681 INFO:teuthology.orchestra.run.trial046.stdout: osd debug shutdown = true 2026-02-24T22:04:35.681 INFO:teuthology.orchestra.run.trial046.stdout: osd debug op order = true 2026-02-24T22:04:35.681 INFO:teuthology.orchestra.run.trial046.stdout: osd debug verify stray on activate = true 2026-02-24T22:04:35.681 INFO:teuthology.orchestra.run.trial046.stdout: osd debug trim objects = true 2026-02-24T22:04:35.681 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.681 INFO:teuthology.orchestra.run.trial046.stdout: osd open classes on start = true 2026-02-24T22:04:35.681 INFO:teuthology.orchestra.run.trial046.stdout: osd debug pg log writeout = true 2026-02-24T22:04:35.681 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.681 INFO:teuthology.orchestra.run.trial046.stdout: osd deep scrub update digest min age = 30 2026-02-24T22:04:35.681 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.681 INFO:teuthology.orchestra.run.trial046.stdout: osd map max advance = 10 2026-02-24T22:04:35.682 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.682 INFO:teuthology.orchestra.run.trial046.stdout: journal zero on create = true 2026-02-24T22:04:35.682 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.682 INFO:teuthology.orchestra.run.trial046.stdout: filestore ondisk finisher threads = 3 2026-02-24T22:04:35.682 INFO:teuthology.orchestra.run.trial046.stdout: filestore apply finisher threads = 3 2026-02-24T22:04:35.682 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.682 INFO:teuthology.orchestra.run.trial046.stdout: bdev debug aio = true 2026-02-24T22:04:35.682 INFO:teuthology.orchestra.run.trial046.stdout: osd debug misdirected ops = true 2026-02-24T22:04:35.682 INFO:teuthology.orchestra.run.trial046.stdout: bdev async discard = True 2026-02-24T22:04:35.682 INFO:teuthology.orchestra.run.trial046.stdout: bdev enable discard = True 2026-02-24T22:04:35.682 INFO:teuthology.orchestra.run.trial046.stdout: bluestore allocator = bitmap 2026-02-24T22:04:35.682 INFO:teuthology.orchestra.run.trial046.stdout: bluestore block size = 96636764160 2026-02-24T22:04:35.682 INFO:teuthology.orchestra.run.trial046.stdout: bluestore fsck on mount = True 2026-02-24T22:04:35.683 INFO:teuthology.orchestra.run.trial046.stdout: debug bluefs = 1/20 2026-02-24T22:04:35.683 INFO:teuthology.orchestra.run.trial046.stdout: debug bluestore = 1/20 2026-02-24T22:04:35.683 INFO:teuthology.orchestra.run.trial046.stdout: debug ms = 1 2026-02-24T22:04:35.683 INFO:teuthology.orchestra.run.trial046.stdout: debug objclass = 20 2026-02-24T22:04:35.683 INFO:teuthology.orchestra.run.trial046.stdout: debug osd = 20 2026-02-24T22:04:35.683 INFO:teuthology.orchestra.run.trial046.stdout: debug rocksdb = 4/10 2026-02-24T22:04:35.683 INFO:teuthology.orchestra.run.trial046.stdout: lockdep = True 2026-02-24T22:04:35.683 INFO:teuthology.orchestra.run.trial046.stdout: mon osd backfillfull_ratio = 0.85 2026-02-24T22:04:35.683 INFO:teuthology.orchestra.run.trial046.stdout: mon osd full ratio = 0.9 2026-02-24T22:04:35.683 INFO:teuthology.orchestra.run.trial046.stdout: mon osd nearfull ratio = 0.8 2026-02-24T22:04:35.683 INFO:teuthology.orchestra.run.trial046.stdout: osd failsafe full ratio = 0.95 2026-02-24T22:04:35.683 INFO:teuthology.orchestra.run.trial046.stdout: osd objectstore = bluestore 2026-02-24T22:04:35.683 INFO:teuthology.orchestra.run.trial046.stdout: osd shutdown pgref assert = True 2026-02-24T22:04:35.684 INFO:teuthology.orchestra.run.trial046.stdout: rgw gc max queue size = 0 2026-02-24T22:04:35.684 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.684 INFO:teuthology.orchestra.run.trial046.stdout:[mgr] 2026-02-24T22:04:35.684 INFO:teuthology.orchestra.run.trial046.stdout: debug ms = 1 2026-02-24T22:04:35.684 INFO:teuthology.orchestra.run.trial046.stdout: debug mgr = 20 2026-02-24T22:04:35.684 INFO:teuthology.orchestra.run.trial046.stdout: debug mon = 20 2026-02-24T22:04:35.684 INFO:teuthology.orchestra.run.trial046.stdout: debug auth = 20 2026-02-24T22:04:35.684 INFO:teuthology.orchestra.run.trial046.stdout: mon reweight min pgs per osd = 4 2026-02-24T22:04:35.684 INFO:teuthology.orchestra.run.trial046.stdout: mon reweight min bytes per osd = 10 2026-02-24T22:04:35.684 INFO:teuthology.orchestra.run.trial046.stdout: mgr/telemetry/nag = false 2026-02-24T22:04:35.684 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.684 INFO:teuthology.orchestra.run.trial046.stdout:[mon] 2026-02-24T22:04:35.685 INFO:teuthology.orchestra.run.trial046.stdout: debug ms = 1 2026-02-24T22:04:35.685 INFO:teuthology.orchestra.run.trial046.stdout: debug mon = 20 2026-02-24T22:04:35.685 INFO:teuthology.orchestra.run.trial046.stdout: debug paxos = 20 2026-02-24T22:04:35.685 INFO:teuthology.orchestra.run.trial046.stdout: debug auth = 20 2026-02-24T22:04:35.685 INFO:teuthology.orchestra.run.trial046.stdout: mon data avail warn = 5 2026-02-24T22:04:35.685 INFO:teuthology.orchestra.run.trial046.stdout: mon mgr mkfs grace = 240 2026-02-24T22:04:35.685 INFO:teuthology.orchestra.run.trial046.stdout: mon reweight min pgs per osd = 4 2026-02-24T22:04:35.685 INFO:teuthology.orchestra.run.trial046.stdout: mon osd reporter subtree level = osd 2026-02-24T22:04:35.685 INFO:teuthology.orchestra.run.trial046.stdout: mon osd prime pg temp = true 2026-02-24T22:04:35.685 INFO:teuthology.orchestra.run.trial046.stdout: mon reweight min bytes per osd = 10 2026-02-24T22:04:35.685 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.685 INFO:teuthology.orchestra.run.trial046.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-24T22:04:35.685 INFO:teuthology.orchestra.run.trial046.stdout: auth mon ticket ttl = 660 # 11m 2026-02-24T22:04:35.686 INFO:teuthology.orchestra.run.trial046.stdout: auth service ticket ttl = 240 # 4m 2026-02-24T22:04:35.686 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.686 INFO:teuthology.orchestra.run.trial046.stdout: # don't complain about insecure global_id in the test suite 2026-02-24T22:04:35.686 INFO:teuthology.orchestra.run.trial046.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-24T22:04:35.686 INFO:teuthology.orchestra.run.trial046.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-24T22:04:35.686 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.686 INFO:teuthology.orchestra.run.trial046.stdout: # 1m isn't quite enough 2026-02-24T22:04:35.686 INFO:teuthology.orchestra.run.trial046.stdout: mon_down_mkfs_grace = 2m 2026-02-24T22:04:35.686 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.686 INFO:teuthology.orchestra.run.trial046.stdout: mon_warn_on_filestore_osds = false 2026-02-24T22:04:35.686 INFO:teuthology.orchestra.run.trial046.stdout: lockdep = True 2026-02-24T22:04:35.686 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:35.686 INFO:teuthology.orchestra.run.trial046.stdout:[client] 2026-02-24T22:04:35.687 INFO:teuthology.orchestra.run.trial046.stdout: rgw cache enabled = true 2026-02-24T22:04:35.687 INFO:teuthology.orchestra.run.trial046.stdout: rgw enable ops log = True 2026-02-24T22:04:35.687 INFO:teuthology.orchestra.run.trial046.stdout: rgw enable usage log = true 2026-02-24T22:04:35.687 INFO:teuthology.orchestra.run.trial046.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-24T22:04:35.687 INFO:teuthology.orchestra.run.trial046.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-24T22:04:35.687 INFO:teuthology.orchestra.run.trial046.stdout: debug rgw = 20 2026-02-24T22:04:35.687 INFO:teuthology.orchestra.run.trial046.stdout: debug rgw notification = 20 2026-02-24T22:04:35.687 INFO:teuthology.orchestra.run.trial046.stdout: lockdep = True 2026-02-24T22:04:35.687 INFO:teuthology.orchestra.run.trial046.stdout: rgw bucket counters cache = True 2026-02-24T22:04:35.687 INFO:teuthology.orchestra.run.trial046.stdout: rgw crypt require ssl = False 2026-02-24T22:04:35.687 INFO:teuthology.orchestra.run.trial046.stdout: rgw crypt s3 kms backend = testing 2026-02-24T22:04:35.687 INFO:teuthology.orchestra.run.trial046.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2026-02-24T22:04:35.688 INFO:teuthology.orchestra.run.trial046.stdout: rgw d3n l1 datacache persistent path = /tmp/rgw_datacache/ 2026-02-24T22:04:35.688 INFO:teuthology.orchestra.run.trial046.stdout: rgw d3n l1 datacache size = 10737418240 2026-02-24T22:04:35.688 INFO:teuthology.orchestra.run.trial046.stdout: rgw d3n l1 local datacache enabled = True 2026-02-24T22:04:35.688 INFO:teuthology.orchestra.run.trial046.stdout: rgw max chunk size = 4194304 2026-02-24T22:04:35.688 INFO:teuthology.orchestra.run.trial046.stdout: rgw obj stripe size = 6291456 2026-02-24T22:04:35.688 INFO:teuthology.orchestra.run.trial046.stdout: rgw reshard progress judge interval = 10 2026-02-24T22:04:35.688 INFO:teuthology.orchestra.run.trial046.stdout: rgw s3 auth use sts = True 2026-02-24T22:04:35.688 INFO:teuthology.orchestra.run.trial046.stdout: rgw sts key = abcdefghijklmnop 2026-02-24T22:04:35.688 INFO:teuthology.orchestra.run.trial046.stdout: rgw torrent flag = True 2026-02-24T22:04:35.688 INFO:teuthology.orchestra.run.trial046.stdout: rgw user counters cache = True 2026-02-24T22:04:35.688 INFO:teuthology.orchestra.run.trial046.stdout: setgroup = ceph 2026-02-24T22:04:35.688 INFO:teuthology.orchestra.run.trial046.stdout: setuser = ceph 2026-02-24T22:04:35.688 INFO:teuthology.orchestra.run.trial046.stdout:[mon.a] 2026-02-24T22:04:35.688 INFO:teuthology.orchestra.run.trial046.stdout:[mon.c] 2026-02-24T22:04:35.689 INFO:teuthology.orchestra.run.trial046.stdout:[mon.b] 2026-02-24T22:04:35.690 INFO:teuthology.orchestra.run.trial110.stdout:[global] 2026-02-24T22:04:35.691 INFO:teuthology.orchestra.run.trial110.stdout: chdir = "" 2026-02-24T22:04:35.691 INFO:teuthology.orchestra.run.trial110.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-24T22:04:35.691 INFO:teuthology.orchestra.run.trial110.stdout: auth supported = cephx 2026-02-24T22:04:35.691 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.691 INFO:teuthology.orchestra.run.trial110.stdout: filestore xattr use omap = true 2026-02-24T22:04:35.691 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.691 INFO:teuthology.orchestra.run.trial110.stdout: mon clock drift allowed = 1.000 2026-02-24T22:04:35.691 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.691 INFO:teuthology.orchestra.run.trial110.stdout: osd crush chooseleaf type = 0 2026-02-24T22:04:35.691 INFO:teuthology.orchestra.run.trial110.stdout: auth debug = true 2026-02-24T22:04:35.692 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.692 INFO:teuthology.orchestra.run.trial110.stdout: ms die on old message = true 2026-02-24T22:04:35.692 INFO:teuthology.orchestra.run.trial110.stdout: ms die on bug = true 2026-02-24T22:04:35.692 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.692 INFO:teuthology.orchestra.run.trial110.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-24T22:04:35.692 INFO:teuthology.orchestra.run.trial110.stdout: mon pg warn max object skew = 0 2026-02-24T22:04:35.692 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.692 INFO:teuthology.orchestra.run.trial110.stdout: # disable pg_autoscaler by default for new pools 2026-02-24T22:04:35.692 INFO:teuthology.orchestra.run.trial110.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-24T22:04:35.692 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.692 INFO:teuthology.orchestra.run.trial110.stdout: osd pool default size = 2 2026-02-24T22:04:35.692 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.693 INFO:teuthology.orchestra.run.trial110.stdout: mon osd allow primary affinity = true 2026-02-24T22:04:35.693 INFO:teuthology.orchestra.run.trial110.stdout: mon osd allow pg remap = true 2026-02-24T22:04:35.693 INFO:teuthology.orchestra.run.trial110.stdout: mon warn on legacy crush tunables = false 2026-02-24T22:04:35.693 INFO:teuthology.orchestra.run.trial110.stdout: mon warn on crush straw calc version zero = false 2026-02-24T22:04:35.693 INFO:teuthology.orchestra.run.trial110.stdout: mon warn on no sortbitwise = false 2026-02-24T22:04:35.693 INFO:teuthology.orchestra.run.trial110.stdout: mon warn on osd down out interval zero = false 2026-02-24T22:04:35.693 INFO:teuthology.orchestra.run.trial110.stdout: mon warn on too few osds = false 2026-02-24T22:04:35.693 INFO:teuthology.orchestra.run.trial110.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-24T22:04:35.693 INFO:teuthology.orchestra.run.trial110.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-24T22:04:35.693 INFO:teuthology.orchestra.run.trial110.stdout: mon_allow_pool_size_one = true 2026-02-24T22:04:35.693 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.693 INFO:teuthology.orchestra.run.trial110.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-24T22:04:35.694 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.694 INFO:teuthology.orchestra.run.trial110.stdout: osd default data pool replay window = 5 2026-02-24T22:04:35.694 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.694 INFO:teuthology.orchestra.run.trial110.stdout: mon allow pool delete = true 2026-02-24T22:04:35.694 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.694 INFO:teuthology.orchestra.run.trial110.stdout: mon cluster log file level = debug 2026-02-24T22:04:35.694 INFO:teuthology.orchestra.run.trial110.stdout: debug asserts on shutdown = true 2026-02-24T22:04:35.694 INFO:teuthology.orchestra.run.trial110.stdout: mon health detail to clog = false 2026-02-24T22:04:35.694 INFO:teuthology.orchestra.run.trial110.stdout: mon host = "10.20.193.46,[v2:10.20.193.46:3301,v1:10.20.193.46:6790],10.20.193.110" 2026-02-24T22:04:35.694 INFO:teuthology.orchestra.run.trial110.stdout: mon client directed command retry = 5 2026-02-24T22:04:35.694 INFO:teuthology.orchestra.run.trial110.stdout: ms inject socket failures = 5000 2026-02-24T22:04:35.694 INFO:teuthology.orchestra.run.trial110.stdout: osd_max_pg_log_entries = 10 2026-02-24T22:04:35.695 INFO:teuthology.orchestra.run.trial110.stdout: osd_min_pg_log_entries = 10 2026-02-24T22:04:35.695 INFO:teuthology.orchestra.run.trial110.stdout: rgw override bucket index max shards = 1 2026-02-24T22:04:35.695 INFO:teuthology.orchestra.run.trial110.stdout: fsid = 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:04:35.695 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.695 INFO:teuthology.orchestra.run.trial110.stdout:[osd] 2026-02-24T22:04:35.695 INFO:teuthology.orchestra.run.trial110.stdout: osd journal size = 100 2026-02-24T22:04:35.695 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.695 INFO:teuthology.orchestra.run.trial110.stdout: osd scrub load threshold = 5.0 2026-02-24T22:04:35.695 INFO:teuthology.orchestra.run.trial110.stdout: osd scrub max interval = 600 2026-02-24T22:04:35.695 INFO:teuthology.orchestra.run.trial110.stdout: osd mclock profile = high_recovery_ops 2026-02-24T22:04:35.695 INFO:teuthology.orchestra.run.trial110.stdout: osd mclock skip benchmark = true 2026-02-24T22:04:35.695 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.696 INFO:teuthology.orchestra.run.trial110.stdout: osd recover clone overlap = true 2026-02-24T22:04:35.696 INFO:teuthology.orchestra.run.trial110.stdout: osd recovery max chunk = 1048576 2026-02-24T22:04:35.696 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.696 INFO:teuthology.orchestra.run.trial110.stdout: osd debug shutdown = true 2026-02-24T22:04:35.696 INFO:teuthology.orchestra.run.trial110.stdout: osd debug op order = true 2026-02-24T22:04:35.696 INFO:teuthology.orchestra.run.trial110.stdout: osd debug verify stray on activate = true 2026-02-24T22:04:35.696 INFO:teuthology.orchestra.run.trial110.stdout: osd debug trim objects = true 2026-02-24T22:04:35.696 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.696 INFO:teuthology.orchestra.run.trial110.stdout: osd open classes on start = true 2026-02-24T22:04:35.696 INFO:teuthology.orchestra.run.trial110.stdout: osd debug pg log writeout = true 2026-02-24T22:04:35.696 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.696 INFO:teuthology.orchestra.run.trial110.stdout: osd deep scrub update digest min age = 30 2026-02-24T22:04:35.697 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.697 INFO:teuthology.orchestra.run.trial110.stdout: osd map max advance = 10 2026-02-24T22:04:35.697 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.697 INFO:teuthology.orchestra.run.trial110.stdout: journal zero on create = true 2026-02-24T22:04:35.697 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.697 INFO:teuthology.orchestra.run.trial110.stdout: filestore ondisk finisher threads = 3 2026-02-24T22:04:35.697 INFO:teuthology.orchestra.run.trial110.stdout: filestore apply finisher threads = 3 2026-02-24T22:04:35.697 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.697 INFO:teuthology.orchestra.run.trial110.stdout: bdev debug aio = true 2026-02-24T22:04:35.697 INFO:teuthology.orchestra.run.trial110.stdout: osd debug misdirected ops = true 2026-02-24T22:04:35.697 INFO:teuthology.orchestra.run.trial110.stdout: bdev async discard = True 2026-02-24T22:04:35.697 INFO:teuthology.orchestra.run.trial110.stdout: bdev enable discard = True 2026-02-24T22:04:35.698 INFO:teuthology.orchestra.run.trial110.stdout: bluestore allocator = bitmap 2026-02-24T22:04:35.698 INFO:teuthology.orchestra.run.trial110.stdout: bluestore block size = 96636764160 2026-02-24T22:04:35.698 INFO:teuthology.orchestra.run.trial110.stdout: bluestore fsck on mount = True 2026-02-24T22:04:35.698 INFO:teuthology.orchestra.run.trial110.stdout: debug bluefs = 1/20 2026-02-24T22:04:35.698 INFO:teuthology.orchestra.run.trial110.stdout: debug bluestore = 1/20 2026-02-24T22:04:35.698 INFO:teuthology.orchestra.run.trial110.stdout: debug ms = 1 2026-02-24T22:04:35.698 INFO:teuthology.orchestra.run.trial110.stdout: debug objclass = 20 2026-02-24T22:04:35.698 INFO:teuthology.orchestra.run.trial110.stdout: debug osd = 20 2026-02-24T22:04:35.698 INFO:teuthology.orchestra.run.trial110.stdout: debug rocksdb = 4/10 2026-02-24T22:04:35.698 INFO:teuthology.orchestra.run.trial110.stdout: lockdep = True 2026-02-24T22:04:35.698 INFO:teuthology.orchestra.run.trial110.stdout: mon osd backfillfull_ratio = 0.85 2026-02-24T22:04:35.699 INFO:teuthology.orchestra.run.trial110.stdout: mon osd full ratio = 0.9 2026-02-24T22:04:35.699 INFO:teuthology.orchestra.run.trial110.stdout: mon osd nearfull ratio = 0.8 2026-02-24T22:04:35.699 INFO:teuthology.orchestra.run.trial110.stdout: osd failsafe full ratio = 0.95 2026-02-24T22:04:35.699 INFO:teuthology.orchestra.run.trial110.stdout: osd objectstore = bluestore 2026-02-24T22:04:35.699 INFO:teuthology.orchestra.run.trial110.stdout: osd shutdown pgref assert = True 2026-02-24T22:04:35.699 INFO:teuthology.orchestra.run.trial110.stdout: rgw gc max queue size = 0 2026-02-24T22:04:35.699 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.699 INFO:teuthology.orchestra.run.trial110.stdout:[mgr] 2026-02-24T22:04:35.699 INFO:teuthology.orchestra.run.trial110.stdout: debug ms = 1 2026-02-24T22:04:35.699 INFO:teuthology.orchestra.run.trial110.stdout: debug mgr = 20 2026-02-24T22:04:35.699 INFO:teuthology.orchestra.run.trial110.stdout: debug mon = 20 2026-02-24T22:04:35.699 INFO:teuthology.orchestra.run.trial110.stdout: debug auth = 20 2026-02-24T22:04:35.699 INFO:teuthology.orchestra.run.trial110.stdout: mon reweight min pgs per osd = 4 2026-02-24T22:04:35.700 INFO:teuthology.orchestra.run.trial110.stdout: mon reweight min bytes per osd = 10 2026-02-24T22:04:35.700 INFO:teuthology.orchestra.run.trial110.stdout: mgr/telemetry/nag = false 2026-02-24T22:04:35.700 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.700 INFO:teuthology.orchestra.run.trial110.stdout:[mon] 2026-02-24T22:04:35.700 INFO:teuthology.orchestra.run.trial110.stdout: debug ms = 1 2026-02-24T22:04:35.700 INFO:teuthology.orchestra.run.trial110.stdout: debug mon = 20 2026-02-24T22:04:35.700 INFO:teuthology.orchestra.run.trial110.stdout: debug paxos = 20 2026-02-24T22:04:35.700 INFO:teuthology.orchestra.run.trial110.stdout: debug auth = 20 2026-02-24T22:04:35.700 INFO:teuthology.orchestra.run.trial110.stdout: mon data avail warn = 5 2026-02-24T22:04:35.700 INFO:teuthology.orchestra.run.trial110.stdout: mon mgr mkfs grace = 240 2026-02-24T22:04:35.700 INFO:teuthology.orchestra.run.trial110.stdout: mon reweight min pgs per osd = 4 2026-02-24T22:04:35.700 INFO:teuthology.orchestra.run.trial110.stdout: mon osd reporter subtree level = osd 2026-02-24T22:04:35.700 INFO:teuthology.orchestra.run.trial110.stdout: mon osd prime pg temp = true 2026-02-24T22:04:35.701 INFO:teuthology.orchestra.run.trial110.stdout: mon reweight min bytes per osd = 10 2026-02-24T22:04:35.701 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.701 INFO:teuthology.orchestra.run.trial110.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-24T22:04:35.701 INFO:teuthology.orchestra.run.trial110.stdout: auth mon ticket ttl = 660 # 11m 2026-02-24T22:04:35.701 INFO:teuthology.orchestra.run.trial110.stdout: auth service ticket ttl = 240 # 4m 2026-02-24T22:04:35.701 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.701 INFO:teuthology.orchestra.run.trial110.stdout: # don't complain about insecure global_id in the test suite 2026-02-24T22:04:35.701 INFO:teuthology.orchestra.run.trial110.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-24T22:04:35.701 INFO:teuthology.orchestra.run.trial110.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-24T22:04:35.701 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.701 INFO:teuthology.orchestra.run.trial110.stdout: # 1m isn't quite enough 2026-02-24T22:04:35.701 INFO:teuthology.orchestra.run.trial110.stdout: mon_down_mkfs_grace = 2m 2026-02-24T22:04:35.702 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.702 INFO:teuthology.orchestra.run.trial110.stdout: mon_warn_on_filestore_osds = false 2026-02-24T22:04:35.702 INFO:teuthology.orchestra.run.trial110.stdout: lockdep = True 2026-02-24T22:04:35.702 INFO:teuthology.orchestra.run.trial110.stdout: 2026-02-24T22:04:35.702 INFO:teuthology.orchestra.run.trial110.stdout:[client] 2026-02-24T22:04:35.702 INFO:teuthology.orchestra.run.trial110.stdout: rgw cache enabled = true 2026-02-24T22:04:35.702 INFO:teuthology.orchestra.run.trial110.stdout: rgw enable ops log = True 2026-02-24T22:04:35.702 INFO:teuthology.orchestra.run.trial110.stdout: rgw enable usage log = true 2026-02-24T22:04:35.702 INFO:teuthology.orchestra.run.trial110.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-24T22:04:35.702 INFO:teuthology.orchestra.run.trial110.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-24T22:04:35.702 INFO:teuthology.orchestra.run.trial110.stdout: debug rgw = 20 2026-02-24T22:04:35.702 INFO:teuthology.orchestra.run.trial110.stdout: debug rgw notification = 20 2026-02-24T22:04:35.702 INFO:teuthology.orchestra.run.trial110.stdout: lockdep = True 2026-02-24T22:04:35.703 INFO:teuthology.orchestra.run.trial110.stdout: rgw bucket counters cache = True 2026-02-24T22:04:35.703 INFO:teuthology.orchestra.run.trial110.stdout: rgw crypt require ssl = False 2026-02-24T22:04:35.703 INFO:teuthology.orchestra.run.trial110.stdout: rgw crypt s3 kms backend = testing 2026-02-24T22:04:35.703 INFO:teuthology.orchestra.run.trial110.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2026-02-24T22:04:35.703 INFO:teuthology.orchestra.run.trial110.stdout: rgw d3n l1 datacache persistent path = /tmp/rgw_datacache/ 2026-02-24T22:04:35.703 INFO:teuthology.orchestra.run.trial110.stdout: rgw d3n l1 datacache size = 10737418240 2026-02-24T22:04:35.703 INFO:teuthology.orchestra.run.trial110.stdout: rgw d3n l1 local datacache enabled = True 2026-02-24T22:04:35.703 INFO:teuthology.orchestra.run.trial110.stdout: rgw max chunk size = 4194304 2026-02-24T22:04:35.703 INFO:teuthology.orchestra.run.trial110.stdout: rgw obj stripe size = 6291456 2026-02-24T22:04:35.703 INFO:teuthology.orchestra.run.trial110.stdout: rgw reshard progress judge interval = 10 2026-02-24T22:04:35.703 INFO:teuthology.orchestra.run.trial110.stdout: rgw s3 auth use sts = True 2026-02-24T22:04:35.703 INFO:teuthology.orchestra.run.trial110.stdout: rgw sts key = abcdefghijklmnop 2026-02-24T22:04:35.703 INFO:teuthology.orchestra.run.trial110.stdout: rgw torrent flag = True 2026-02-24T22:04:35.704 INFO:teuthology.orchestra.run.trial110.stdout: rgw user counters cache = True 2026-02-24T22:04:35.704 INFO:teuthology.orchestra.run.trial110.stdout: setgroup = ceph 2026-02-24T22:04:35.704 INFO:teuthology.orchestra.run.trial110.stdout: setuser = ceph 2026-02-24T22:04:35.704 INFO:teuthology.orchestra.run.trial110.stdout:[mon.a] 2026-02-24T22:04:35.704 INFO:teuthology.orchestra.run.trial110.stdout:[mon.c] 2026-02-24T22:04:35.704 INFO:teuthology.orchestra.run.trial110.stdout:[mon.b] 2026-02-24T22:04:35.704 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-24T22:04:35.705 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-24T22:04:35.753 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-24T22:04:35.753 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:04:35.753 DEBUG:teuthology.orchestra.run.trial046:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-24T22:04:35.799 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:04:35.799 DEBUG:teuthology.orchestra.run.trial046:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-24T22:04:35.847 INFO:tasks.ceph:Sending monmap to node ubuntu@trial046.front.sepia.ceph.com 2026-02-24T22:04:35.847 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:04:35.847 DEBUG:teuthology.orchestra.run.trial046:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-24T22:04:35.847 DEBUG:teuthology.orchestra.run.trial046:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-24T22:04:35.904 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:04:35.904 DEBUG:teuthology.orchestra.run.trial046:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-24T22:04:35.950 INFO:tasks.ceph:Sending monmap to node ubuntu@trial110.front.sepia.ceph.com 2026-02-24T22:04:35.951 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T22:04:35.951 DEBUG:teuthology.orchestra.run.trial110:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-24T22:04:35.951 DEBUG:teuthology.orchestra.run.trial110:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-24T22:04:35.967 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T22:04:35.967 DEBUG:teuthology.orchestra.run.trial110:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-24T22:04:36.012 INFO:tasks.ceph:Setting up mon nodes... 2026-02-24T22:04:36.012 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-24T22:04:36.013 DEBUG:teuthology.orchestra.run.trial046:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2026-02-24T22:04:36.036 INFO:teuthology.orchestra.run.trial046.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2026-02-24T22:04:36.038 DEBUG:teuthology.orchestra.run.trial110:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-24T22:04:36.085 INFO:teuthology.orchestra.run.trial110.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-24T22:04:36.089 INFO:tasks.ceph:Setting up mds nodes... 2026-02-24T22:04:36.089 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-24T22:04:36.089 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-24T22:04:36.107 INFO:teuthology.orchestra.run.trial046.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-24T22:04:36.114 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2026-02-24T22:04:36.149 INFO:teuthology.orchestra.run.trial110.stdout:creating /etc/ceph/ceph.client.1.keyring 2026-02-24T22:04:36.157 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-24T22:04:36.157 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial046.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@trial110.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2026-02-24T22:04:36.158 DEBUG:teuthology.orchestra.run.trial046:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-24T22:04:36.166 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-24T22:04:36.166 INFO:tasks.ceph:role: osd.0 2026-02-24T22:04:36.166 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial046.front.sepia.ceph.com 2026-02-24T22:04:36.166 DEBUG:teuthology.orchestra.run.trial046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-24T22:04:36.216 INFO:teuthology.orchestra.run.trial046.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T22:04:36.216 INFO:teuthology.orchestra.run.trial046.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T22:04:36.216 INFO:teuthology.orchestra.run.trial046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T22:04:36.216 INFO:teuthology.orchestra.run.trial046.stdout: = reflink=1 bigtime=0 inobtcount=0 2026-02-24T22:04:36.217 INFO:teuthology.orchestra.run.trial046.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T22:04:36.217 INFO:teuthology.orchestra.run.trial046.stdout: = sunit=0 swidth=0 blks 2026-02-24T22:04:36.217 INFO:teuthology.orchestra.run.trial046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T22:04:36.217 INFO:teuthology.orchestra.run.trial046.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T22:04:36.217 INFO:teuthology.orchestra.run.trial046.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T22:04:36.217 INFO:teuthology.orchestra.run.trial046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T22:04:36.796 INFO:teuthology.orchestra.run.trial046.stdout:Discarding blocks...Done. 2026-02-24T22:04:36.798 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial046.front.sepia.ceph.com -o noatime 2026-02-24T22:04:36.798 DEBUG:teuthology.orchestra.run.trial046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-24T22:04:36.855 DEBUG:teuthology.orchestra.run.trial046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-24T22:04:36.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T22:04:36.905 INFO:teuthology.orchestra.run.trial046.stderr:sudo: /sbin/restorecon: command not found 2026-02-24T22:04:36.906 DEBUG:teuthology.orchestra.run.trial046:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-24T22:04:36.955 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-24T22:04:36.955 INFO:tasks.ceph:role: osd.1 2026-02-24T22:04:36.955 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial046.front.sepia.ceph.com 2026-02-24T22:04:36.956 DEBUG:teuthology.orchestra.run.trial046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-24T22:04:37.008 INFO:teuthology.orchestra.run.trial046.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T22:04:37.008 INFO:teuthology.orchestra.run.trial046.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T22:04:37.009 INFO:teuthology.orchestra.run.trial046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T22:04:37.009 INFO:teuthology.orchestra.run.trial046.stdout: = reflink=1 bigtime=0 inobtcount=0 2026-02-24T22:04:37.009 INFO:teuthology.orchestra.run.trial046.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T22:04:37.009 INFO:teuthology.orchestra.run.trial046.stdout: = sunit=0 swidth=0 blks 2026-02-24T22:04:37.009 INFO:teuthology.orchestra.run.trial046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T22:04:37.009 INFO:teuthology.orchestra.run.trial046.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T22:04:37.009 INFO:teuthology.orchestra.run.trial046.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T22:04:37.009 INFO:teuthology.orchestra.run.trial046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T22:04:37.574 INFO:teuthology.orchestra.run.trial046.stdout:Discarding blocks...Done. 2026-02-24T22:04:37.575 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial046.front.sepia.ceph.com -o noatime 2026-02-24T22:04:37.576 DEBUG:teuthology.orchestra.run.trial046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-24T22:04:37.631 DEBUG:teuthology.orchestra.run.trial046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-24T22:04:37.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T22:04:37.681 INFO:teuthology.orchestra.run.trial046.stderr:sudo: /sbin/restorecon: command not found 2026-02-24T22:04:37.681 DEBUG:teuthology.orchestra.run.trial046:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-24T22:04:37.731 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-24T22:04:37.731 INFO:tasks.ceph:role: osd.2 2026-02-24T22:04:37.731 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial046.front.sepia.ceph.com 2026-02-24T22:04:37.731 DEBUG:teuthology.orchestra.run.trial046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-24T22:04:37.780 INFO:teuthology.orchestra.run.trial046.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T22:04:37.780 INFO:teuthology.orchestra.run.trial046.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T22:04:37.780 INFO:teuthology.orchestra.run.trial046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T22:04:37.781 INFO:teuthology.orchestra.run.trial046.stdout: = reflink=1 bigtime=0 inobtcount=0 2026-02-24T22:04:37.781 INFO:teuthology.orchestra.run.trial046.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T22:04:37.781 INFO:teuthology.orchestra.run.trial046.stdout: = sunit=0 swidth=0 blks 2026-02-24T22:04:37.781 INFO:teuthology.orchestra.run.trial046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T22:04:37.781 INFO:teuthology.orchestra.run.trial046.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T22:04:37.781 INFO:teuthology.orchestra.run.trial046.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T22:04:37.781 INFO:teuthology.orchestra.run.trial046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T22:04:38.344 INFO:teuthology.orchestra.run.trial046.stdout:Discarding blocks...Done. 2026-02-24T22:04:38.345 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial046.front.sepia.ceph.com -o noatime 2026-02-24T22:04:38.346 DEBUG:teuthology.orchestra.run.trial046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-24T22:04:38.404 DEBUG:teuthology.orchestra.run.trial046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-24T22:04:38.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T22:04:38.453 INFO:teuthology.orchestra.run.trial046.stderr:sudo: /sbin/restorecon: command not found 2026-02-24T22:04:38.454 DEBUG:teuthology.orchestra.run.trial046:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2026-02-24T22:04:38.503 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-24T22:04:38.503 INFO:tasks.ceph:role: osd.3 2026-02-24T22:04:38.503 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@trial046.front.sepia.ceph.com 2026-02-24T22:04:38.503 DEBUG:teuthology.orchestra.run.trial046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2026-02-24T22:04:38.557 INFO:teuthology.orchestra.run.trial046.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T22:04:38.557 INFO:teuthology.orchestra.run.trial046.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T22:04:38.557 INFO:teuthology.orchestra.run.trial046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T22:04:38.557 INFO:teuthology.orchestra.run.trial046.stdout: = reflink=1 bigtime=0 inobtcount=0 2026-02-24T22:04:38.557 INFO:teuthology.orchestra.run.trial046.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T22:04:38.557 INFO:teuthology.orchestra.run.trial046.stdout: = sunit=0 swidth=0 blks 2026-02-24T22:04:38.557 INFO:teuthology.orchestra.run.trial046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T22:04:38.558 INFO:teuthology.orchestra.run.trial046.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T22:04:38.558 INFO:teuthology.orchestra.run.trial046.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T22:04:38.558 INFO:teuthology.orchestra.run.trial046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T22:04:39.118 INFO:teuthology.orchestra.run.trial046.stdout:Discarding blocks...Done. 2026-02-24T22:04:39.120 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@trial046.front.sepia.ceph.com -o noatime 2026-02-24T22:04:39.120 DEBUG:teuthology.orchestra.run.trial046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2026-02-24T22:04:39.176 DEBUG:teuthology.orchestra.run.trial046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2026-02-24T22:04:39.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T22:04:39.225 INFO:teuthology.orchestra.run.trial046.stderr:sudo: /sbin/restorecon: command not found 2026-02-24T22:04:39.226 DEBUG:teuthology.orchestra.run.trial046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T22:04:39.289 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:04:39.280+0000 7ff937ddca40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2026-02-24T22:04:39.290 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:04:39.280+0000 7ff937ddca40 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2026-02-24T22:04:39.290 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:04:39.280+0000 7ff937ddca40 -1 bdev(0x560d0085d800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2026-02-24T22:04:39.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:04:39.280+0000 7ff937ddca40 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2026-02-24T22:04:40.155 DEBUG:teuthology.orchestra.run.trial046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-24T22:04:40.206 DEBUG:teuthology.orchestra.run.trial046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T22:04:40.271 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:04:40.260+0000 7f075547ca40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2026-02-24T22:04:40.271 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:04:40.264+0000 7f075547ca40 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2026-02-24T22:04:40.272 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:04:40.264+0000 7f075547ca40 -1 bdev(0x558c6568d800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2026-02-24T22:04:40.272 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:04:40.264+0000 7f075547ca40 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2026-02-24T22:04:41.175 DEBUG:teuthology.orchestra.run.trial046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-24T22:04:41.226 DEBUG:teuthology.orchestra.run.trial046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T22:04:41.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:04:41.284+0000 7f4fe1501a40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2026-02-24T22:04:41.292 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:04:41.284+0000 7f4fe1501a40 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2026-02-24T22:04:41.292 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:04:41.284+0000 7f4fe1501a40 -1 bdev(0x5597b2e5f800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2026-02-24T22:04:41.292 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:04:41.284+0000 7f4fe1501a40 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2026-02-24T22:04:42.180 DEBUG:teuthology.orchestra.run.trial046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-24T22:04:42.231 DEBUG:teuthology.orchestra.run.trial046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T22:04:42.295 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:04:42.288+0000 7efe594c9a40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2026-02-24T22:04:42.296 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:04:42.288+0000 7efe594c9a40 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2026-02-24T22:04:42.296 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:04:42.288+0000 7efe594c9a40 -1 bdev(0x562981943800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2026-02-24T22:04:42.296 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:04:42.288+0000 7efe594c9a40 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2026-02-24T22:04:43.184 DEBUG:teuthology.orchestra.run.trial046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2026-02-24T22:04:43.235 DEBUG:teuthology.orchestra.run.trial110:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2026-02-24T22:04:43.245 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2026-02-24T22:04:43.245 INFO:tasks.ceph:role: osd.4 2026-02-24T22:04:43.246 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial110.front.sepia.ceph.com 2026-02-24T22:04:43.246 DEBUG:teuthology.orchestra.run.trial110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-24T22:04:43.298 INFO:teuthology.orchestra.run.trial110.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T22:04:43.298 INFO:teuthology.orchestra.run.trial110.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T22:04:43.298 INFO:teuthology.orchestra.run.trial110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T22:04:43.298 INFO:teuthology.orchestra.run.trial110.stdout: = reflink=1 bigtime=0 inobtcount=0 2026-02-24T22:04:43.298 INFO:teuthology.orchestra.run.trial110.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T22:04:43.298 INFO:teuthology.orchestra.run.trial110.stdout: = sunit=0 swidth=0 blks 2026-02-24T22:04:43.298 INFO:teuthology.orchestra.run.trial110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T22:04:43.299 INFO:teuthology.orchestra.run.trial110.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T22:04:43.299 INFO:teuthology.orchestra.run.trial110.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T22:04:43.299 INFO:teuthology.orchestra.run.trial110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T22:04:43.869 INFO:teuthology.orchestra.run.trial110.stdout:Discarding blocks...Done. 2026-02-24T22:04:43.871 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial110.front.sepia.ceph.com -o noatime 2026-02-24T22:04:43.871 DEBUG:teuthology.orchestra.run.trial110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2026-02-24T22:04:43.925 DEBUG:teuthology.orchestra.run.trial110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2026-02-24T22:04:43.974 INFO:teuthology.orchestra.run.trial110.stderr:sudo: /sbin/restorecon: command not found 2026-02-24T22:04:43.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T22:04:43.975 DEBUG:teuthology.orchestra.run.trial110:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2026-02-24T22:04:44.024 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2026-02-24T22:04:44.024 INFO:tasks.ceph:role: osd.5 2026-02-24T22:04:44.025 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial110.front.sepia.ceph.com 2026-02-24T22:04:44.025 DEBUG:teuthology.orchestra.run.trial110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-24T22:04:44.078 INFO:teuthology.orchestra.run.trial110.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T22:04:44.078 INFO:teuthology.orchestra.run.trial110.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T22:04:44.078 INFO:teuthology.orchestra.run.trial110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T22:04:44.079 INFO:teuthology.orchestra.run.trial110.stdout: = reflink=1 bigtime=0 inobtcount=0 2026-02-24T22:04:44.079 INFO:teuthology.orchestra.run.trial110.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T22:04:44.079 INFO:teuthology.orchestra.run.trial110.stdout: = sunit=0 swidth=0 blks 2026-02-24T22:04:44.079 INFO:teuthology.orchestra.run.trial110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T22:04:44.079 INFO:teuthology.orchestra.run.trial110.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T22:04:44.079 INFO:teuthology.orchestra.run.trial110.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T22:04:44.079 INFO:teuthology.orchestra.run.trial110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T22:04:44.645 INFO:teuthology.orchestra.run.trial110.stdout:Discarding blocks...Done. 2026-02-24T22:04:44.646 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial110.front.sepia.ceph.com -o noatime 2026-02-24T22:04:44.647 DEBUG:teuthology.orchestra.run.trial110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2026-02-24T22:04:44.705 DEBUG:teuthology.orchestra.run.trial110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2026-02-24T22:04:44.753 INFO:teuthology.orchestra.run.trial110.stderr:sudo: /sbin/restorecon: command not found 2026-02-24T22:04:44.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T22:04:44.755 DEBUG:teuthology.orchestra.run.trial110:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2026-02-24T22:04:44.804 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2026-02-24T22:04:44.804 INFO:tasks.ceph:role: osd.6 2026-02-24T22:04:44.804 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial110.front.sepia.ceph.com 2026-02-24T22:04:44.804 DEBUG:teuthology.orchestra.run.trial110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-24T22:04:44.858 INFO:teuthology.orchestra.run.trial110.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T22:04:44.858 INFO:teuthology.orchestra.run.trial110.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T22:04:44.858 INFO:teuthology.orchestra.run.trial110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T22:04:44.858 INFO:teuthology.orchestra.run.trial110.stdout: = reflink=1 bigtime=0 inobtcount=0 2026-02-24T22:04:44.858 INFO:teuthology.orchestra.run.trial110.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T22:04:44.858 INFO:teuthology.orchestra.run.trial110.stdout: = sunit=0 swidth=0 blks 2026-02-24T22:04:44.858 INFO:teuthology.orchestra.run.trial110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T22:04:44.858 INFO:teuthology.orchestra.run.trial110.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T22:04:44.858 INFO:teuthology.orchestra.run.trial110.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T22:04:44.858 INFO:teuthology.orchestra.run.trial110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T22:04:45.421 INFO:teuthology.orchestra.run.trial110.stdout:Discarding blocks...Done. 2026-02-24T22:04:45.422 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial110.front.sepia.ceph.com -o noatime 2026-02-24T22:04:45.423 DEBUG:teuthology.orchestra.run.trial110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2026-02-24T22:04:45.477 DEBUG:teuthology.orchestra.run.trial110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2026-02-24T22:04:45.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T22:04:45.527 INFO:teuthology.orchestra.run.trial110.stderr:sudo: /sbin/restorecon: command not found 2026-02-24T22:04:45.527 DEBUG:teuthology.orchestra.run.trial110:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2026-02-24T22:04:45.576 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2026-02-24T22:04:45.576 INFO:tasks.ceph:role: osd.7 2026-02-24T22:04:45.577 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@trial110.front.sepia.ceph.com 2026-02-24T22:04:45.577 DEBUG:teuthology.orchestra.run.trial110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2026-02-24T22:04:45.630 INFO:teuthology.orchestra.run.trial110.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T22:04:45.630 INFO:teuthology.orchestra.run.trial110.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T22:04:45.630 INFO:teuthology.orchestra.run.trial110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T22:04:45.631 INFO:teuthology.orchestra.run.trial110.stdout: = reflink=1 bigtime=0 inobtcount=0 2026-02-24T22:04:45.631 INFO:teuthology.orchestra.run.trial110.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T22:04:45.631 INFO:teuthology.orchestra.run.trial110.stdout: = sunit=0 swidth=0 blks 2026-02-24T22:04:45.631 INFO:teuthology.orchestra.run.trial110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T22:04:45.631 INFO:teuthology.orchestra.run.trial110.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T22:04:45.631 INFO:teuthology.orchestra.run.trial110.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T22:04:45.631 INFO:teuthology.orchestra.run.trial110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T22:04:46.192 INFO:teuthology.orchestra.run.trial110.stdout:Discarding blocks...Done. 2026-02-24T22:04:46.194 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@trial110.front.sepia.ceph.com -o noatime 2026-02-24T22:04:46.194 DEBUG:teuthology.orchestra.run.trial110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2026-02-24T22:04:46.253 DEBUG:teuthology.orchestra.run.trial110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2026-02-24T22:04:46.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T22:04:46.306 INFO:teuthology.orchestra.run.trial110.stderr:sudo: /sbin/restorecon: command not found 2026-02-24T22:04:46.307 DEBUG:teuthology.orchestra.run.trial110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T22:04:46.369 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T22:04:46.370+0000 7f0468e27a40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2026-02-24T22:04:46.370 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T22:04:46.370+0000 7f0468e27a40 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2026-02-24T22:04:46.370 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T22:04:46.370+0000 7f0468e27a40 -1 bdev(0x55cf5fd09800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2026-02-24T22:04:46.370 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T22:04:46.370+0000 7f0468e27a40 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2026-02-24T22:04:47.289 DEBUG:teuthology.orchestra.run.trial110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2026-02-24T22:04:47.342 DEBUG:teuthology.orchestra.run.trial110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T22:04:47.405 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T22:04:47.406+0000 7fa98bf1da40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2026-02-24T22:04:47.406 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T22:04:47.406+0000 7fa98bf1da40 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2026-02-24T22:04:47.406 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T22:04:47.406+0000 7fa98bf1da40 -1 bdev(0x55ce12649800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2026-02-24T22:04:47.406 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T22:04:47.406+0000 7fa98bf1da40 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2026-02-24T22:04:48.270 DEBUG:teuthology.orchestra.run.trial110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2026-02-24T22:04:48.321 DEBUG:teuthology.orchestra.run.trial110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T22:04:48.385 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T22:04:48.386+0000 7fb2ba733a40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2026-02-24T22:04:48.386 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T22:04:48.386+0000 7fb2ba733a40 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2026-02-24T22:04:48.386 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T22:04:48.386+0000 7fb2ba733a40 -1 bdev(0x55d676d1b800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2026-02-24T22:04:48.386 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T22:04:48.386+0000 7fb2ba733a40 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2026-02-24T22:04:49.389 DEBUG:teuthology.orchestra.run.trial110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2026-02-24T22:04:49.441 DEBUG:teuthology.orchestra.run.trial110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T22:04:49.510 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T22:04:49.510+0000 7faa133f1a40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2026-02-24T22:04:49.510 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T22:04:49.510+0000 7faa133f1a40 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2026-02-24T22:04:49.511 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T22:04:49.510+0000 7faa133f1a40 -1 bdev(0x55a9e0cb5800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2026-02-24T22:04:49.511 INFO:teuthology.orchestra.run.trial110.stderr:2026-02-24T22:04:49.510+0000 7faa133f1a40 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2026-02-24T22:04:50.429 DEBUG:teuthology.orchestra.run.trial110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2026-02-24T22:04:50.481 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-24T22:04:50.481 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:04:50.481 DEBUG:teuthology.orchestra.run.trial046:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2026-02-24T22:04:50.492 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:04:50.493 DEBUG:teuthology.orchestra.run.trial046:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-24T22:04:50.543 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:04:50.544 DEBUG:teuthology.orchestra.run.trial046:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-24T22:04:50.594 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:04:50.594 DEBUG:teuthology.orchestra.run.trial046:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-24T22:04:50.642 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:04:50.642 DEBUG:teuthology.orchestra.run.trial046:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2026-02-24T22:04:50.690 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T22:04:50.690 DEBUG:teuthology.orchestra.run.trial110:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-24T22:04:50.701 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T22:04:50.702 DEBUG:teuthology.orchestra.run.trial110:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2026-02-24T22:04:50.753 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T22:04:50.753 DEBUG:teuthology.orchestra.run.trial110:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2026-02-24T22:04:50.805 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T22:04:50.805 DEBUG:teuthology.orchestra.run.trial110:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2026-02-24T22:04:50.857 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T22:04:50.857 DEBUG:teuthology.orchestra.run.trial110:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2026-02-24T22:04:50.909 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:04:50.909 DEBUG:teuthology.orchestra.run.trial046:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-24T22:04:50.914 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T22:04:50.914 DEBUG:teuthology.orchestra.run.trial110:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2026-02-24T22:04:50.960 INFO:tasks.ceph:Adding keys to all mons... 2026-02-24T22:04:50.960 DEBUG:teuthology.orchestra.run.trial046:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-24T22:04:50.962 DEBUG:teuthology.orchestra.run.trial110:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-24T22:04:51.006 INFO:teuthology.orchestra.run.trial046.stdout:[mgr.y] 2026-02-24T22:04:51.006 INFO:teuthology.orchestra.run.trial046.stdout: key = AQB0IJ5pOzfuARAAATcuRRxWkviTXmG3hWeGpQ== 2026-02-24T22:04:51.006 INFO:teuthology.orchestra.run.trial046.stdout:[osd.0] 2026-02-24T22:04:51.006 INFO:teuthology.orchestra.run.trial046.stdout: key = AQB3IJ5pGWMQERAAPTYkjEF5Dc5X7tb5nA3WqQ== 2026-02-24T22:04:51.006 INFO:teuthology.orchestra.run.trial046.stdout:[osd.1] 2026-02-24T22:04:51.007 INFO:teuthology.orchestra.run.trial046.stdout: key = AQB4IJ5pjS/xDxAAvHgyjppzTvRtAMDmnrfZfA== 2026-02-24T22:04:51.007 INFO:teuthology.orchestra.run.trial046.stdout:[osd.2] 2026-02-24T22:04:51.007 INFO:teuthology.orchestra.run.trial046.stdout: key = AQB5IJ5p6mEvERAAUndchtSbzR7w07QSbHE51g== 2026-02-24T22:04:51.007 INFO:teuthology.orchestra.run.trial046.stdout:[osd.3] 2026-02-24T22:04:51.007 INFO:teuthology.orchestra.run.trial046.stdout: key = AQB6IJ5p90hrERAA4bVlMR8KPPivQlCFdS5S8w== 2026-02-24T22:04:51.007 INFO:teuthology.orchestra.run.trial046.stdout:[mgr.x] 2026-02-24T22:04:51.007 INFO:teuthology.orchestra.run.trial046.stdout: key = AQB0IJ5pjyBTBRAAHxewVRkBxmMZf0nGkuEmRw== 2026-02-24T22:04:51.007 INFO:teuthology.orchestra.run.trial046.stdout:[osd.4] 2026-02-24T22:04:51.007 INFO:teuthology.orchestra.run.trial046.stdout: key = AQB+IJ5pW2xGFhAA/0bBC/LBWtShIEeS0JKjhw== 2026-02-24T22:04:51.007 INFO:teuthology.orchestra.run.trial046.stdout:[osd.5] 2026-02-24T22:04:51.007 INFO:teuthology.orchestra.run.trial046.stdout: key = AQB/IJ5ppZJmGBAAa3oXFpOXq7mzqncgBHw1uw== 2026-02-24T22:04:51.007 INFO:teuthology.orchestra.run.trial046.stdout:[osd.6] 2026-02-24T22:04:51.008 INFO:teuthology.orchestra.run.trial046.stdout: key = AQCAIJ5pSPc3FxAAJnSxI+rY9vSZjK8Lc3ZlxA== 2026-02-24T22:04:51.008 INFO:teuthology.orchestra.run.trial046.stdout:[osd.7] 2026-02-24T22:04:51.008 INFO:teuthology.orchestra.run.trial046.stdout: key = AQCBIJ5p/julHhAAD6tmEmJS6LZZw3WSv6vKaw== 2026-02-24T22:04:51.008 INFO:teuthology.orchestra.run.trial046.stdout:[client.0] 2026-02-24T22:04:51.008 INFO:teuthology.orchestra.run.trial046.stdout: key = AQB0IJ5pNk4tBhAAsXRcPypH1CbdAr4pJNzaTQ== 2026-02-24T22:04:51.008 INFO:teuthology.orchestra.run.trial046.stdout:[client.1] 2026-02-24T22:04:51.008 INFO:teuthology.orchestra.run.trial046.stdout: key = AQB0IJ5pm/koCRAAR34uonW/v3zsFUym/Zh2Vg== 2026-02-24T22:04:51.011 INFO:teuthology.orchestra.run.trial110.stdout:[mgr.y] 2026-02-24T22:04:51.011 INFO:teuthology.orchestra.run.trial110.stdout: key = AQB0IJ5pOzfuARAAATcuRRxWkviTXmG3hWeGpQ== 2026-02-24T22:04:51.011 INFO:teuthology.orchestra.run.trial110.stdout:[osd.0] 2026-02-24T22:04:51.011 INFO:teuthology.orchestra.run.trial110.stdout: key = AQB3IJ5pGWMQERAAPTYkjEF5Dc5X7tb5nA3WqQ== 2026-02-24T22:04:51.011 INFO:teuthology.orchestra.run.trial110.stdout:[osd.1] 2026-02-24T22:04:51.011 INFO:teuthology.orchestra.run.trial110.stdout: key = AQB4IJ5pjS/xDxAAvHgyjppzTvRtAMDmnrfZfA== 2026-02-24T22:04:51.011 INFO:teuthology.orchestra.run.trial110.stdout:[osd.2] 2026-02-24T22:04:51.011 INFO:teuthology.orchestra.run.trial110.stdout: key = AQB5IJ5p6mEvERAAUndchtSbzR7w07QSbHE51g== 2026-02-24T22:04:51.012 INFO:teuthology.orchestra.run.trial110.stdout:[osd.3] 2026-02-24T22:04:51.012 INFO:teuthology.orchestra.run.trial110.stdout: key = AQB6IJ5p90hrERAA4bVlMR8KPPivQlCFdS5S8w== 2026-02-24T22:04:51.012 INFO:teuthology.orchestra.run.trial110.stdout:[mgr.x] 2026-02-24T22:04:51.012 INFO:teuthology.orchestra.run.trial110.stdout: key = AQB0IJ5pjyBTBRAAHxewVRkBxmMZf0nGkuEmRw== 2026-02-24T22:04:51.012 INFO:teuthology.orchestra.run.trial110.stdout:[osd.4] 2026-02-24T22:04:51.012 INFO:teuthology.orchestra.run.trial110.stdout: key = AQB+IJ5pW2xGFhAA/0bBC/LBWtShIEeS0JKjhw== 2026-02-24T22:04:51.012 INFO:teuthology.orchestra.run.trial110.stdout:[osd.5] 2026-02-24T22:04:51.012 INFO:teuthology.orchestra.run.trial110.stdout: key = AQB/IJ5ppZJmGBAAa3oXFpOXq7mzqncgBHw1uw== 2026-02-24T22:04:51.012 INFO:teuthology.orchestra.run.trial110.stdout:[osd.6] 2026-02-24T22:04:51.012 INFO:teuthology.orchestra.run.trial110.stdout: key = AQCAIJ5pSPc3FxAAJnSxI+rY9vSZjK8Lc3ZlxA== 2026-02-24T22:04:51.012 INFO:teuthology.orchestra.run.trial110.stdout:[osd.7] 2026-02-24T22:04:51.012 INFO:teuthology.orchestra.run.trial110.stdout: key = AQCBIJ5p/julHhAAD6tmEmJS6LZZw3WSv6vKaw== 2026-02-24T22:04:51.013 INFO:teuthology.orchestra.run.trial110.stdout:[client.0] 2026-02-24T22:04:51.013 INFO:teuthology.orchestra.run.trial110.stdout: key = AQB0IJ5pNk4tBhAAsXRcPypH1CbdAr4pJNzaTQ== 2026-02-24T22:04:51.013 INFO:teuthology.orchestra.run.trial110.stdout:[client.1] 2026-02-24T22:04:51.013 INFO:teuthology.orchestra.run.trial110.stdout: key = AQB0IJ5pm/koCRAAR34uonW/v3zsFUym/Zh2Vg== 2026-02-24T22:04:51.014 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-24T22:04:51.052 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-24T22:04:51.077 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T22:04:51.116 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T22:04:51.140 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T22:04:51.180 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T22:04:51.204 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T22:04:51.244 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T22:04:51.268 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T22:04:51.308 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T22:04:51.333 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-24T22:04:51.372 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-24T22:04:51.396 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T22:04:51.436 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T22:04:51.460 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T22:04:51.500 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T22:04:51.524 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T22:04:51.564 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T22:04:51.588 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T22:04:51.628 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T22:04:51.649 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-24T22:04:51.692 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-24T22:04:51.714 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-24T22:04:51.756 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-24T22:04:51.778 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-24T22:04:51.778 DEBUG:teuthology.orchestra.run.trial046:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-24T22:04:51.826 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-24T22:04:51.892 DEBUG:teuthology.orchestra.run.trial046:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-24T22:04:51.942 DEBUG:teuthology.orchestra.run.trial046:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2026-02-24T22:04:51.990 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-24T22:04:52.056 DEBUG:teuthology.orchestra.run.trial046:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2026-02-24T22:04:52.106 DEBUG:teuthology.orchestra.run.trial110:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2026-02-24T22:04:52.116 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-24T22:04:52.185 DEBUG:teuthology.orchestra.run.trial110:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2026-02-24T22:04:52.236 DEBUG:teuthology.orchestra.run.trial046:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-24T22:04:52.238 DEBUG:teuthology.orchestra.run.trial110:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-24T22:04:52.283 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-24T22:04:52.284 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-24T22:04:52.284 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-24T22:04:52.286 INFO:tasks.ceph.mon.a:Started 2026-02-24T22:04:52.286 INFO:tasks.ceph.mon.c:Restarting daemon 2026-02-24T22:04:52.286 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2026-02-24T22:04:52.288 INFO:tasks.ceph.mon.c:Started 2026-02-24T22:04:52.289 INFO:tasks.ceph.mon.b:Restarting daemon 2026-02-24T22:04:52.289 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2026-02-24T22:04:52.325 INFO:tasks.ceph.mon.b:Started 2026-02-24T22:04:52.325 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-24T22:04:52.325 INFO:tasks.ceph.mgr.y:Restarting daemon 2026-02-24T22:04:52.326 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2026-02-24T22:04:52.327 INFO:tasks.ceph.mgr.y:Started 2026-02-24T22:04:52.327 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-24T22:04:52.328 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-24T22:04:52.329 INFO:tasks.ceph.mgr.x:Started 2026-02-24T22:04:52.330 DEBUG:tasks.ceph:set 0 configs 2026-02-24T22:04:52.330 DEBUG:teuthology.orchestra.run.trial046:> sudo ceph --cluster ceph config dump 2026-02-24T22:04:57.685 INFO:teuthology.orchestra.run.trial046.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-24T22:04:57.696 INFO:tasks.ceph:Setting crush tunables to default 2026-02-24T22:04:57.696 DEBUG:teuthology.orchestra.run.trial046:> sudo ceph --cluster ceph osd crush tunables default 2026-02-24T22:04:57.793 INFO:teuthology.orchestra.run.trial046.stderr:adjusted tunables profile to default 2026-02-24T22:04:57.802 INFO:tasks.ceph:check_enable_crimson: False 2026-02-24T22:04:57.802 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-24T22:04:57.803 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:04:57.803 DEBUG:teuthology.orchestra.run.trial046:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-24T22:04:57.811 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:04:57.812 DEBUG:teuthology.orchestra.run.trial046:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-24T22:04:57.864 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:04:57.864 DEBUG:teuthology.orchestra.run.trial046:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-24T22:04:57.914 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:04:57.914 DEBUG:teuthology.orchestra.run.trial046:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2026-02-24T22:04:57.961 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T22:04:57.961 DEBUG:teuthology.orchestra.run.trial110:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2026-02-24T22:04:57.971 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T22:04:57.971 DEBUG:teuthology.orchestra.run.trial110:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2026-02-24T22:04:58.020 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T22:04:58.020 DEBUG:teuthology.orchestra.run.trial110:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2026-02-24T22:04:58.067 DEBUG:teuthology.orchestra.run.trial110:> set -ex 2026-02-24T22:04:58.068 DEBUG:teuthology.orchestra.run.trial110:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2026-02-24T22:04:58.115 DEBUG:teuthology.orchestra.run.trial110:> sudo ceph --cluster ceph osd new eecfa846-f234-4fa7-a191-99fa94beee15 0 2026-02-24T22:04:58.253 INFO:teuthology.orchestra.run.trial110.stdout:0 2026-02-24T22:04:58.262 DEBUG:teuthology.orchestra.run.trial110:> sudo ceph --cluster ceph osd new 611f1433-2d52-40f4-9aa5-e3440fbcec42 1 2026-02-24T22:04:58.365 INFO:teuthology.orchestra.run.trial110.stdout:1 2026-02-24T22:04:58.377 DEBUG:teuthology.orchestra.run.trial110:> sudo ceph --cluster ceph osd new a509993f-f201-4228-a138-5f362e8e65c4 2 2026-02-24T22:04:58.477 INFO:teuthology.orchestra.run.trial110.stdout:2 2026-02-24T22:04:58.486 DEBUG:teuthology.orchestra.run.trial110:> sudo ceph --cluster ceph osd new 7cfa4f22-9b2f-48e6-842c-c2cecb64a410 3 2026-02-24T22:04:58.585 INFO:teuthology.orchestra.run.trial110.stdout:3 2026-02-24T22:04:58.595 DEBUG:teuthology.orchestra.run.trial110:> sudo ceph --cluster ceph osd new 2338abec-ea2d-4351-9610-321f26463100 4 2026-02-24T22:04:58.694 INFO:teuthology.orchestra.run.trial110.stdout:4 2026-02-24T22:04:58.703 DEBUG:teuthology.orchestra.run.trial110:> sudo ceph --cluster ceph osd new a4b1d335-43cd-49d5-b998-971f50dc15ab 5 2026-02-24T22:04:58.802 INFO:teuthology.orchestra.run.trial110.stdout:5 2026-02-24T22:04:58.811 DEBUG:teuthology.orchestra.run.trial110:> sudo ceph --cluster ceph osd new f19fb057-9193-4afb-bfe8-866c2109d2fb 6 2026-02-24T22:04:58.910 INFO:teuthology.orchestra.run.trial110.stdout:6 2026-02-24T22:04:58.919 DEBUG:teuthology.orchestra.run.trial110:> sudo ceph --cluster ceph osd new 0dd2edb1-de90-4632-b940-655f96b44dd3 7 2026-02-24T22:04:58.976 INFO:tasks.ceph.mgr.y.trial046.stderr:ERROR:root:Module 'jmespath' is not installed. 2026-02-24T22:04:59.017 INFO:teuthology.orchestra.run.trial110.stdout:7 2026-02-24T22:04:59.019 INFO:tasks.ceph.mgr.x.trial110.stderr:ERROR:root:Module 'jmespath' is not installed. 2026-02-24T22:04:59.026 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-24T22:04:59.026 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2026-02-24T22:04:59.028 INFO:tasks.ceph.osd.0:Started 2026-02-24T22:04:59.029 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-24T22:04:59.029 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2026-02-24T22:04:59.030 INFO:tasks.ceph.mgr.y.trial046.stderr:ERROR:dashboard.controllers.nvmeof:Failed to import NVMeoFClient and related components: cannot import name 'NVMeoFClient' from 'dashboard.services.nvmeof_client' (/usr/share/ceph/mgr/dashboard/services/nvmeof_client.py) 2026-02-24T22:04:59.031 INFO:tasks.ceph.osd.1:Started 2026-02-24T22:04:59.031 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-24T22:04:59.031 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2026-02-24T22:04:59.033 INFO:tasks.ceph.osd.2:Started 2026-02-24T22:04:59.033 INFO:tasks.ceph.osd.3:Restarting daemon 2026-02-24T22:04:59.033 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2026-02-24T22:04:59.035 INFO:tasks.ceph.osd.3:Started 2026-02-24T22:04:59.035 INFO:tasks.ceph.osd.4:Restarting daemon 2026-02-24T22:04:59.036 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2026-02-24T22:04:59.036 INFO:tasks.ceph.mgr.y.trial046.stderr:ERROR:sso:Module 'jmespath' is not installed. 2026-02-24T22:04:59.038 INFO:tasks.ceph.osd.4:Started 2026-02-24T22:04:59.038 INFO:tasks.ceph.osd.5:Restarting daemon 2026-02-24T22:04:59.038 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2026-02-24T22:04:59.040 INFO:tasks.ceph.osd.5:Started 2026-02-24T22:04:59.040 INFO:tasks.ceph.osd.6:Restarting daemon 2026-02-24T22:04:59.040 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2026-02-24T22:04:59.042 INFO:tasks.ceph.osd.6:Started 2026-02-24T22:04:59.042 INFO:tasks.ceph.osd.7:Restarting daemon 2026-02-24T22:04:59.043 DEBUG:teuthology.orchestra.run.trial110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2026-02-24T22:04:59.044 INFO:tasks.ceph.osd.7:Started 2026-02-24T22:04:59.045 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-24T22:04:59.075 INFO:tasks.ceph.mgr.x.trial110.stderr:ERROR:dashboard.controllers.nvmeof:Failed to import NVMeoFClient and related components: cannot import name 'NVMeoFClient' from 'dashboard.services.nvmeof_client' (/usr/share/ceph/mgr/dashboard/services/nvmeof_client.py) 2026-02-24T22:04:59.080 INFO:tasks.ceph.mgr.x.trial110.stderr:ERROR:sso:Module 'jmespath' is not installed. 2026-02-24T22:04:59.086 INFO:tasks.ceph.mgr.y.trial046.stderr:ERROR:root:Module 'xmltodict' is not installed. 2026-02-24T22:04:59.130 INFO:tasks.ceph.mgr.x.trial110.stderr:ERROR:root:Module 'xmltodict' is not installed. 2026-02-24T22:04:59.141 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:04:59.141 INFO:teuthology.orchestra.run.trial046.stdout:{"epoch":10,"fsid":"262e7508-9310-4d2a-bc05-2c8861262eff","created":"2026-02-24T22:04:57.636187+0000","modified":"2026-02-24T22:04:59.012805+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T22:04:59.012805+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"eecfa846-f234-4fa7-a191-99fa94beee15","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"611f1433-2d52-40f4-9aa5-e3440fbcec42","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a509993f-f201-4228-a138-5f362e8e65c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7cfa4f22-9b2f-48e6-842c-c2cecb64a410","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2338abec-ea2d-4351-9610-321f26463100","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a4b1d335-43cd-49d5-b998-971f50dc15ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f19fb057-9193-4afb-bfe8-866c2109d2fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0dd2edb1-de90-4632-b940-655f96b44dd3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T22:04:59.152 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-24T22:04:59.152 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-24T22:04:59.516 INFO:tasks.ceph.mgr.y.trial046.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-24T22:04:59.516 INFO:tasks.ceph.mgr.y.trial046.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-24T22:04:59.516 INFO:tasks.ceph.mgr.y.trial046.stderr: from numpy import show_config as show_numpy_config 2026-02-24T22:04:59.565 INFO:tasks.ceph.mgr.x.trial110.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-24T22:04:59.565 INFO:tasks.ceph.mgr.x.trial110.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-24T22:04:59.565 INFO:tasks.ceph.mgr.x.trial110.stderr: from numpy import show_config as show_numpy_config 2026-02-24T22:04:59.603 INFO:tasks.ceph.osd.3.trial046.stderr:2026-02-24T22:04:59.595+0000 7faee3339a40 -1 Falling back to public interface 2026-02-24T22:04:59.607 INFO:tasks.ceph.osd.0.trial046.stderr:2026-02-24T22:04:59.599+0000 7f91037f8a40 -1 Falling back to public interface 2026-02-24T22:04:59.615 INFO:tasks.ceph.osd.1.trial046.stderr:2026-02-24T22:04:59.607+0000 7f1eec79ea40 -1 Falling back to public interface 2026-02-24T22:04:59.621 INFO:tasks.ceph.osd.4.trial110.stderr:2026-02-24T22:04:59.622+0000 7fca39ffba40 -1 Falling back to public interface 2026-02-24T22:04:59.629 INFO:tasks.ceph.osd.6.trial110.stderr:2026-02-24T22:04:59.630+0000 7f1cbf370a40 -1 Falling back to public interface 2026-02-24T22:04:59.631 INFO:tasks.ceph.osd.2.trial046.stderr:2026-02-24T22:04:59.623+0000 7fd1ab856a40 -1 Falling back to public interface 2026-02-24T22:04:59.645 INFO:tasks.ceph.osd.5.trial110.stderr:2026-02-24T22:04:59.646+0000 7fb898cc5a40 -1 Falling back to public interface 2026-02-24T22:04:59.645 INFO:tasks.ceph.osd.7.trial110.stderr:2026-02-24T22:04:59.646+0000 7fe3e3fa3a40 -1 Falling back to public interface 2026-02-24T22:04:59.957 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T22:05:00.046 INFO:teuthology.misc.health.trial046.stdout: 2026-02-24T22:05:00.046 INFO:teuthology.misc.health.trial046.stdout:{"epoch":10,"fsid":"262e7508-9310-4d2a-bc05-2c8861262eff","created":"2026-02-24T22:04:57.636187+0000","modified":"2026-02-24T22:04:59.012805+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T22:04:59.012805+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"eecfa846-f234-4fa7-a191-99fa94beee15","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"611f1433-2d52-40f4-9aa5-e3440fbcec42","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a509993f-f201-4228-a138-5f362e8e65c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7cfa4f22-9b2f-48e6-842c-c2cecb64a410","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2338abec-ea2d-4351-9610-321f26463100","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a4b1d335-43cd-49d5-b998-971f50dc15ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f19fb057-9193-4afb-bfe8-866c2109d2fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0dd2edb1-de90-4632-b940-655f96b44dd3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T22:05:00.054 DEBUG:teuthology.misc:0 of 8 OSDs are up 2026-02-24T22:05:00.594 INFO:tasks.ceph.mgr.y.trial046.stderr:2026-02-24T22:05:00.587+0000 7f3b73f69640 -1 mgr.server handle_report got status from non-daemon mon.c 2026-02-24T22:05:00.595 INFO:tasks.ceph.mgr.y.trial046.stderr:2026-02-24T22:05:00.587+0000 7f3b73f69640 -1 mgr.server handle_report got status from non-daemon mon.b 2026-02-24T22:05:00.713 INFO:tasks.ceph.osd.3.trial046.stderr:2026-02-24T22:05:00.703+0000 7faee3339a40 -1 osd.3 0 log_to_monitors true 2026-02-24T22:05:00.767 INFO:tasks.ceph.osd.1.trial046.stderr:2026-02-24T22:05:00.759+0000 7f1eec79ea40 -1 osd.1 0 log_to_monitors true 2026-02-24T22:05:00.770 INFO:tasks.ceph.osd.4.trial110.stderr:2026-02-24T22:05:00.770+0000 7fca39ffba40 -1 osd.4 0 log_to_monitors true 2026-02-24T22:05:00.797 INFO:tasks.ceph.osd.7.trial110.stderr:2026-02-24T22:05:00.798+0000 7fe3e3fa3a40 -1 osd.7 0 log_to_monitors true 2026-02-24T22:05:00.801 INFO:tasks.ceph.osd.2.trial046.stderr:2026-02-24T22:05:00.791+0000 7fd1ab856a40 -1 osd.2 0 log_to_monitors true 2026-02-24T22:05:00.815 INFO:tasks.ceph.osd.5.trial110.stderr:2026-02-24T22:05:00.814+0000 7fb898cc5a40 -1 osd.5 0 log_to_monitors true 2026-02-24T22:05:00.816 INFO:tasks.ceph.osd.6.trial110.stderr:2026-02-24T22:05:00.814+0000 7f1cbf370a40 -1 osd.6 0 log_to_monitors true 2026-02-24T22:05:00.828 INFO:tasks.ceph.osd.0.trial046.stderr:2026-02-24T22:05:00.819+0000 7f91037f8a40 -1 osd.0 0 log_to_monitors true 2026-02-24T22:05:02.658 INFO:tasks.ceph.osd.2.trial046.stderr:2026-02-24T22:05:02.651+0000 7fd1a77de640 -1 osd.2 0 waiting for initial osdmap 2026-02-24T22:05:02.659 INFO:tasks.ceph.osd.1.trial046.stderr:2026-02-24T22:05:02.651+0000 7f1ee8726640 -1 osd.1 0 waiting for initial osdmap 2026-02-24T22:05:02.660 INFO:tasks.ceph.osd.0.trial046.stderr:2026-02-24T22:05:02.651+0000 7f90ff780640 -1 osd.0 0 waiting for initial osdmap 2026-02-24T22:05:02.660 INFO:tasks.ceph.osd.3.trial046.stderr:2026-02-24T22:05:02.651+0000 7faedf2c1640 -1 osd.3 0 waiting for initial osdmap 2026-02-24T22:05:02.660 INFO:tasks.ceph.osd.6.trial110.stderr:2026-02-24T22:05:02.658+0000 7f1cbb2f8640 -1 osd.6 0 waiting for initial osdmap 2026-02-24T22:05:02.661 INFO:tasks.ceph.osd.5.trial110.stderr:2026-02-24T22:05:02.658+0000 7fb894c4d640 -1 osd.5 0 waiting for initial osdmap 2026-02-24T22:05:02.661 INFO:tasks.ceph.osd.4.trial110.stderr:2026-02-24T22:05:02.658+0000 7fca35f83640 -1 osd.4 0 waiting for initial osdmap 2026-02-24T22:05:02.661 INFO:tasks.ceph.osd.7.trial110.stderr:2026-02-24T22:05:02.662+0000 7fe3dff2b640 -1 osd.7 0 waiting for initial osdmap 2026-02-24T22:05:02.663 INFO:tasks.ceph.osd.5.trial110.stderr:2026-02-24T22:05:02.662+0000 7fb88c32d640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T22:05:02.664 INFO:tasks.ceph.osd.6.trial110.stderr:2026-02-24T22:05:02.662+0000 7f1cb29d8640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T22:05:02.664 INFO:tasks.ceph.osd.4.trial110.stderr:2026-02-24T22:05:02.662+0000 7fca2d663640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T22:05:02.664 INFO:tasks.ceph.osd.7.trial110.stderr:2026-02-24T22:05:02.662+0000 7fe3d760b640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T22:05:02.665 INFO:tasks.ceph.osd.2.trial046.stderr:2026-02-24T22:05:02.655+0000 7fd19eebe640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T22:05:02.665 INFO:tasks.ceph.osd.0.trial046.stderr:2026-02-24T22:05:02.655+0000 7f90f6e60640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T22:05:02.665 INFO:tasks.ceph.osd.1.trial046.stderr:2026-02-24T22:05:02.655+0000 7f1edfe06640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T22:05:02.665 INFO:tasks.ceph.osd.3.trial046.stderr:2026-02-24T22:05:02.655+0000 7faed69a1640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T22:05:06.859 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T22:05:07.006 INFO:teuthology.misc.health.trial046.stdout: 2026-02-24T22:05:07.006 INFO:teuthology.misc.health.trial046.stdout:{"epoch":16,"fsid":"262e7508-9310-4d2a-bc05-2c8861262eff","created":"2026-02-24T22:04:57.636187+0000","modified":"2026-02-24T22:05:06.666508+0000","last_up_change":"2026-02-24T22:05:03.655652+0000","last_in_change":"2026-02-24T22:04:59.012805+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-24T22:05:05.621245+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"eecfa846-f234-4fa7-a191-99fa94beee15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6810","nonce":3488974090},{"type":"v1","addr":"10.20.193.46:6811","nonce":3488974090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6812","nonce":3488974090},{"type":"v1","addr":"10.20.193.46:6813","nonce":3488974090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6816","nonce":3488974090},{"type":"v1","addr":"10.20.193.46:6817","nonce":3488974090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6814","nonce":3488974090},{"type":"v1","addr":"10.20.193.46:6815","nonce":3488974090}]},"public_addr":"10.20.193.46:6811/3488974090","cluster_addr":"10.20.193.46:6813/3488974090","heartbeat_back_addr":"10.20.193.46:6817/3488974090","heartbeat_front_addr":"10.20.193.46:6815/3488974090","state":["exists","up"]},{"osd":1,"uuid":"611f1433-2d52-40f4-9aa5-e3440fbcec42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6818","nonce":2329512561},{"type":"v1","addr":"10.20.193.46:6819","nonce":2329512561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6820","nonce":2329512561},{"type":"v1","addr":"10.20.193.46:6821","nonce":2329512561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6824","nonce":2329512561},{"type":"v1","addr":"10.20.193.46:6825","nonce":2329512561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6822","nonce":2329512561},{"type":"v1","addr":"10.20.193.46:6823","nonce":2329512561}]},"public_addr":"10.20.193.46:6819/2329512561","cluster_addr":"10.20.193.46:6821/2329512561","heartbeat_back_addr":"10.20.193.46:6825/2329512561","heartbeat_front_addr":"10.20.193.46:6823/2329512561","state":["exists","up"]},{"osd":2,"uuid":"a509993f-f201-4228-a138-5f362e8e65c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6826","nonce":3065971976},{"type":"v1","addr":"10.20.193.46:6827","nonce":3065971976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6828","nonce":3065971976},{"type":"v1","addr":"10.20.193.46:6829","nonce":3065971976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6832","nonce":3065971976},{"type":"v1","addr":"10.20.193.46:6833","nonce":3065971976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6830","nonce":3065971976},{"type":"v1","addr":"10.20.193.46:6831","nonce":3065971976}]},"public_addr":"10.20.193.46:6827/3065971976","cluster_addr":"10.20.193.46:6829/3065971976","heartbeat_back_addr":"10.20.193.46:6833/3065971976","heartbeat_front_addr":"10.20.193.46:6831/3065971976","state":["exists","up"]},{"osd":3,"uuid":"7cfa4f22-9b2f-48e6-842c-c2cecb64a410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6802","nonce":2359184024},{"type":"v1","addr":"10.20.193.46:6803","nonce":2359184024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6804","nonce":2359184024},{"type":"v1","addr":"10.20.193.46:6805","nonce":2359184024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6808","nonce":2359184024},{"type":"v1","addr":"10.20.193.46:6809","nonce":2359184024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6806","nonce":2359184024},{"type":"v1","addr":"10.20.193.46:6807","nonce":2359184024}]},"public_addr":"10.20.193.46:6803/2359184024","cluster_addr":"10.20.193.46:6805/2359184024","heartbeat_back_addr":"10.20.193.46:6809/2359184024","heartbeat_front_addr":"10.20.193.46:6807/2359184024","state":["exists","up"]},{"osd":4,"uuid":"2338abec-ea2d-4351-9610-321f26463100","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6800","nonce":341553926},{"type":"v1","addr":"10.20.193.110:6801","nonce":341553926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6802","nonce":341553926},{"type":"v1","addr":"10.20.193.110:6803","nonce":341553926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6806","nonce":341553926},{"type":"v1","addr":"10.20.193.110:6807","nonce":341553926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6804","nonce":341553926},{"type":"v1","addr":"10.20.193.110:6805","nonce":341553926}]},"public_addr":"10.20.193.110:6801/341553926","cluster_addr":"10.20.193.110:6803/341553926","heartbeat_back_addr":"10.20.193.110:6807/341553926","heartbeat_front_addr":"10.20.193.110:6805/341553926","state":["exists","up"]},{"osd":5,"uuid":"a4b1d335-43cd-49d5-b998-971f50dc15ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6816","nonce":2680757198},{"type":"v1","addr":"10.20.193.110:6818","nonce":2680757198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6820","nonce":2680757198},{"type":"v1","addr":"10.20.193.110:6822","nonce":2680757198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6828","nonce":2680757198},{"type":"v1","addr":"10.20.193.110:6830","nonce":2680757198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6824","nonce":2680757198},{"type":"v1","addr":"10.20.193.110:6826","nonce":2680757198}]},"public_addr":"10.20.193.110:6818/2680757198","cluster_addr":"10.20.193.110:6822/2680757198","heartbeat_back_addr":"10.20.193.110:6830/2680757198","heartbeat_front_addr":"10.20.193.110:6826/2680757198","state":["exists","up"]},{"osd":6,"uuid":"f19fb057-9193-4afb-bfe8-866c2109d2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6808","nonce":884480935},{"type":"v1","addr":"10.20.193.110:6809","nonce":884480935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6810","nonce":884480935},{"type":"v1","addr":"10.20.193.110:6811","nonce":884480935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6814","nonce":884480935},{"type":"v1","addr":"10.20.193.110:6815","nonce":884480935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6812","nonce":884480935},{"type":"v1","addr":"10.20.193.110:6813","nonce":884480935}]},"public_addr":"10.20.193.110:6809/884480935","cluster_addr":"10.20.193.110:6811/884480935","heartbeat_back_addr":"10.20.193.110:6815/884480935","heartbeat_front_addr":"10.20.193.110:6813/884480935","state":["exists","up"]},{"osd":7,"uuid":"0dd2edb1-de90-4632-b940-655f96b44dd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6817","nonce":3330554869},{"type":"v1","addr":"10.20.193.110:6819","nonce":3330554869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6821","nonce":3330554869},{"type":"v1","addr":"10.20.193.110:6823","nonce":3330554869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6829","nonce":3330554869},{"type":"v1","addr":"10.20.193.110:6831","nonce":3330554869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6825","nonce":3330554869},{"type":"v1","addr":"10.20.193.110:6827","nonce":3330554869}]},"public_addr":"10.20.193.110:6819/3330554869","cluster_addr":"10.20.193.110:6823/3330554869","heartbeat_back_addr":"10.20.193.110:6831/3330554869","heartbeat_front_addr":"10.20.193.110:6827/3330554869","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.826515+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.811034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.765092+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.662393+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.739442+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.809572+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.843167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.792855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T22:05:07.017 DEBUG:teuthology.misc:8 of 8 OSDs are up 2026-02-24T22:05:07.018 INFO:tasks.ceph:Creating RBD pool 2026-02-24T22:05:07.018 DEBUG:teuthology.orchestra.run.trial046:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-24T22:05:07.679 INFO:teuthology.orchestra.run.trial046.stderr:pool 'rbd' created 2026-02-24T22:05:07.692 DEBUG:teuthology.orchestra.run.trial046:> rbd --cluster ceph pool init rbd 2026-02-24T22:05:07.708 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:05:07.708 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:05:10.697 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-24T22:05:10.698 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-24T22:05:10.698 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-24T22:05:10.891 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:05:10.903 INFO:teuthology.orchestra.run.trial046.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-24T22:05:10.904 INFO:tasks.ceph_manager:config epoch is 1 2026-02-24T22:05:10.904 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-24T22:05:10.904 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-24T22:05:10.904 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-24T22:05:11.081 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:05:11.090 INFO:teuthology.orchestra.run.trial046.stdout:{"epoch":5,"flags":0,"active_gid":4106,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6800","nonce":530373280},{"type":"v1","addr":"10.20.193.46:6801","nonce":530373280}]},"active_addr":"10.20.193.46:6801/530373280","active_change":"2026-02-24T22:04:59.580930+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4544132024016699391,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.46:0","nonce":3213208447}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.46:0","nonce":3577506920}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.46:0","nonce":1414767331}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.46:0","nonce":507091437}]}]} 2026-02-24T22:05:11.096 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-24T22:05:11.096 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-24T22:05:11.097 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-24T22:05:11.263 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:05:11.264 INFO:teuthology.orchestra.run.trial046.stdout:{"epoch":20,"fsid":"262e7508-9310-4d2a-bc05-2c8861262eff","created":"2026-02-24T22:04:57.636187+0000","modified":"2026-02-24T22:05:10.681676+0000","last_up_change":"2026-02-24T22:05:03.655652+0000","last_in_change":"2026-02-24T22:04:59.012805+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-24T22:05:05.621245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-24T22:05:07.173093+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"eecfa846-f234-4fa7-a191-99fa94beee15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6810","nonce":3488974090},{"type":"v1","addr":"10.20.193.46:6811","nonce":3488974090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6812","nonce":3488974090},{"type":"v1","addr":"10.20.193.46:6813","nonce":3488974090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6816","nonce":3488974090},{"type":"v1","addr":"10.20.193.46:6817","nonce":3488974090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6814","nonce":3488974090},{"type":"v1","addr":"10.20.193.46:6815","nonce":3488974090}]},"public_addr":"10.20.193.46:6811/3488974090","cluster_addr":"10.20.193.46:6813/3488974090","heartbeat_back_addr":"10.20.193.46:6817/3488974090","heartbeat_front_addr":"10.20.193.46:6815/3488974090","state":["exists","up"]},{"osd":1,"uuid":"611f1433-2d52-40f4-9aa5-e3440fbcec42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6818","nonce":2329512561},{"type":"v1","addr":"10.20.193.46:6819","nonce":2329512561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6820","nonce":2329512561},{"type":"v1","addr":"10.20.193.46:6821","nonce":2329512561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6824","nonce":2329512561},{"type":"v1","addr":"10.20.193.46:6825","nonce":2329512561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6822","nonce":2329512561},{"type":"v1","addr":"10.20.193.46:6823","nonce":2329512561}]},"public_addr":"10.20.193.46:6819/2329512561","cluster_addr":"10.20.193.46:6821/2329512561","heartbeat_back_addr":"10.20.193.46:6825/2329512561","heartbeat_front_addr":"10.20.193.46:6823/2329512561","state":["exists","up"]},{"osd":2,"uuid":"a509993f-f201-4228-a138-5f362e8e65c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6826","nonce":3065971976},{"type":"v1","addr":"10.20.193.46:6827","nonce":3065971976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6828","nonce":3065971976},{"type":"v1","addr":"10.20.193.46:6829","nonce":3065971976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6832","nonce":3065971976},{"type":"v1","addr":"10.20.193.46:6833","nonce":3065971976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6830","nonce":3065971976},{"type":"v1","addr":"10.20.193.46:6831","nonce":3065971976}]},"public_addr":"10.20.193.46:6827/3065971976","cluster_addr":"10.20.193.46:6829/3065971976","heartbeat_back_addr":"10.20.193.46:6833/3065971976","heartbeat_front_addr":"10.20.193.46:6831/3065971976","state":["exists","up"]},{"osd":3,"uuid":"7cfa4f22-9b2f-48e6-842c-c2cecb64a410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6802","nonce":2359184024},{"type":"v1","addr":"10.20.193.46:6803","nonce":2359184024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6804","nonce":2359184024},{"type":"v1","addr":"10.20.193.46:6805","nonce":2359184024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6808","nonce":2359184024},{"type":"v1","addr":"10.20.193.46:6809","nonce":2359184024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6806","nonce":2359184024},{"type":"v1","addr":"10.20.193.46:6807","nonce":2359184024}]},"public_addr":"10.20.193.46:6803/2359184024","cluster_addr":"10.20.193.46:6805/2359184024","heartbeat_back_addr":"10.20.193.46:6809/2359184024","heartbeat_front_addr":"10.20.193.46:6807/2359184024","state":["exists","up"]},{"osd":4,"uuid":"2338abec-ea2d-4351-9610-321f26463100","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6800","nonce":341553926},{"type":"v1","addr":"10.20.193.110:6801","nonce":341553926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6802","nonce":341553926},{"type":"v1","addr":"10.20.193.110:6803","nonce":341553926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6806","nonce":341553926},{"type":"v1","addr":"10.20.193.110:6807","nonce":341553926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6804","nonce":341553926},{"type":"v1","addr":"10.20.193.110:6805","nonce":341553926}]},"public_addr":"10.20.193.110:6801/341553926","cluster_addr":"10.20.193.110:6803/341553926","heartbeat_back_addr":"10.20.193.110:6807/341553926","heartbeat_front_addr":"10.20.193.110:6805/341553926","state":["exists","up"]},{"osd":5,"uuid":"a4b1d335-43cd-49d5-b998-971f50dc15ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6816","nonce":2680757198},{"type":"v1","addr":"10.20.193.110:6818","nonce":2680757198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6820","nonce":2680757198},{"type":"v1","addr":"10.20.193.110:6822","nonce":2680757198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6828","nonce":2680757198},{"type":"v1","addr":"10.20.193.110:6830","nonce":2680757198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6824","nonce":2680757198},{"type":"v1","addr":"10.20.193.110:6826","nonce":2680757198}]},"public_addr":"10.20.193.110:6818/2680757198","cluster_addr":"10.20.193.110:6822/2680757198","heartbeat_back_addr":"10.20.193.110:6830/2680757198","heartbeat_front_addr":"10.20.193.110:6826/2680757198","state":["exists","up"]},{"osd":6,"uuid":"f19fb057-9193-4afb-bfe8-866c2109d2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6808","nonce":884480935},{"type":"v1","addr":"10.20.193.110:6809","nonce":884480935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6810","nonce":884480935},{"type":"v1","addr":"10.20.193.110:6811","nonce":884480935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6814","nonce":884480935},{"type":"v1","addr":"10.20.193.110:6815","nonce":884480935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6812","nonce":884480935},{"type":"v1","addr":"10.20.193.110:6813","nonce":884480935}]},"public_addr":"10.20.193.110:6809/884480935","cluster_addr":"10.20.193.110:6811/884480935","heartbeat_back_addr":"10.20.193.110:6815/884480935","heartbeat_front_addr":"10.20.193.110:6813/884480935","state":["exists","up"]},{"osd":7,"uuid":"0dd2edb1-de90-4632-b940-655f96b44dd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6817","nonce":3330554869},{"type":"v1","addr":"10.20.193.110:6819","nonce":3330554869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6821","nonce":3330554869},{"type":"v1","addr":"10.20.193.110:6823","nonce":3330554869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6829","nonce":3330554869},{"type":"v1","addr":"10.20.193.110:6831","nonce":3330554869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6825","nonce":3330554869},{"type":"v1","addr":"10.20.193.110:6827","nonce":3330554869}]},"public_addr":"10.20.193.110:6819/3330554869","cluster_addr":"10.20.193.110:6823/3330554869","heartbeat_back_addr":"10.20.193.110:6831/3330554869","heartbeat_front_addr":"10.20.193.110:6827/3330554869","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.826515+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.811034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.765092+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.662393+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.739442+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.809572+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.843167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.792855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T22:05:11.275 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-24T22:05:11.275 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-24T22:05:11.435 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:05:11.435 INFO:teuthology.orchestra.run.trial046.stdout:{"epoch":20,"fsid":"262e7508-9310-4d2a-bc05-2c8861262eff","created":"2026-02-24T22:04:57.636187+0000","modified":"2026-02-24T22:05:10.681676+0000","last_up_change":"2026-02-24T22:05:03.655652+0000","last_in_change":"2026-02-24T22:04:59.012805+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-24T22:05:05.621245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-24T22:05:07.173093+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"eecfa846-f234-4fa7-a191-99fa94beee15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6810","nonce":3488974090},{"type":"v1","addr":"10.20.193.46:6811","nonce":3488974090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6812","nonce":3488974090},{"type":"v1","addr":"10.20.193.46:6813","nonce":3488974090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6816","nonce":3488974090},{"type":"v1","addr":"10.20.193.46:6817","nonce":3488974090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6814","nonce":3488974090},{"type":"v1","addr":"10.20.193.46:6815","nonce":3488974090}]},"public_addr":"10.20.193.46:6811/3488974090","cluster_addr":"10.20.193.46:6813/3488974090","heartbeat_back_addr":"10.20.193.46:6817/3488974090","heartbeat_front_addr":"10.20.193.46:6815/3488974090","state":["exists","up"]},{"osd":1,"uuid":"611f1433-2d52-40f4-9aa5-e3440fbcec42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6818","nonce":2329512561},{"type":"v1","addr":"10.20.193.46:6819","nonce":2329512561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6820","nonce":2329512561},{"type":"v1","addr":"10.20.193.46:6821","nonce":2329512561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6824","nonce":2329512561},{"type":"v1","addr":"10.20.193.46:6825","nonce":2329512561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6822","nonce":2329512561},{"type":"v1","addr":"10.20.193.46:6823","nonce":2329512561}]},"public_addr":"10.20.193.46:6819/2329512561","cluster_addr":"10.20.193.46:6821/2329512561","heartbeat_back_addr":"10.20.193.46:6825/2329512561","heartbeat_front_addr":"10.20.193.46:6823/2329512561","state":["exists","up"]},{"osd":2,"uuid":"a509993f-f201-4228-a138-5f362e8e65c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6826","nonce":3065971976},{"type":"v1","addr":"10.20.193.46:6827","nonce":3065971976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6828","nonce":3065971976},{"type":"v1","addr":"10.20.193.46:6829","nonce":3065971976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6832","nonce":3065971976},{"type":"v1","addr":"10.20.193.46:6833","nonce":3065971976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6830","nonce":3065971976},{"type":"v1","addr":"10.20.193.46:6831","nonce":3065971976}]},"public_addr":"10.20.193.46:6827/3065971976","cluster_addr":"10.20.193.46:6829/3065971976","heartbeat_back_addr":"10.20.193.46:6833/3065971976","heartbeat_front_addr":"10.20.193.46:6831/3065971976","state":["exists","up"]},{"osd":3,"uuid":"7cfa4f22-9b2f-48e6-842c-c2cecb64a410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6802","nonce":2359184024},{"type":"v1","addr":"10.20.193.46:6803","nonce":2359184024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6804","nonce":2359184024},{"type":"v1","addr":"10.20.193.46:6805","nonce":2359184024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6808","nonce":2359184024},{"type":"v1","addr":"10.20.193.46:6809","nonce":2359184024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6806","nonce":2359184024},{"type":"v1","addr":"10.20.193.46:6807","nonce":2359184024}]},"public_addr":"10.20.193.46:6803/2359184024","cluster_addr":"10.20.193.46:6805/2359184024","heartbeat_back_addr":"10.20.193.46:6809/2359184024","heartbeat_front_addr":"10.20.193.46:6807/2359184024","state":["exists","up"]},{"osd":4,"uuid":"2338abec-ea2d-4351-9610-321f26463100","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6800","nonce":341553926},{"type":"v1","addr":"10.20.193.110:6801","nonce":341553926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6802","nonce":341553926},{"type":"v1","addr":"10.20.193.110:6803","nonce":341553926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6806","nonce":341553926},{"type":"v1","addr":"10.20.193.110:6807","nonce":341553926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6804","nonce":341553926},{"type":"v1","addr":"10.20.193.110:6805","nonce":341553926}]},"public_addr":"10.20.193.110:6801/341553926","cluster_addr":"10.20.193.110:6803/341553926","heartbeat_back_addr":"10.20.193.110:6807/341553926","heartbeat_front_addr":"10.20.193.110:6805/341553926","state":["exists","up"]},{"osd":5,"uuid":"a4b1d335-43cd-49d5-b998-971f50dc15ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6816","nonce":2680757198},{"type":"v1","addr":"10.20.193.110:6818","nonce":2680757198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6820","nonce":2680757198},{"type":"v1","addr":"10.20.193.110:6822","nonce":2680757198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6828","nonce":2680757198},{"type":"v1","addr":"10.20.193.110:6830","nonce":2680757198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6824","nonce":2680757198},{"type":"v1","addr":"10.20.193.110:6826","nonce":2680757198}]},"public_addr":"10.20.193.110:6818/2680757198","cluster_addr":"10.20.193.110:6822/2680757198","heartbeat_back_addr":"10.20.193.110:6830/2680757198","heartbeat_front_addr":"10.20.193.110:6826/2680757198","state":["exists","up"]},{"osd":6,"uuid":"f19fb057-9193-4afb-bfe8-866c2109d2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6808","nonce":884480935},{"type":"v1","addr":"10.20.193.110:6809","nonce":884480935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6810","nonce":884480935},{"type":"v1","addr":"10.20.193.110:6811","nonce":884480935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6814","nonce":884480935},{"type":"v1","addr":"10.20.193.110:6815","nonce":884480935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6812","nonce":884480935},{"type":"v1","addr":"10.20.193.110:6813","nonce":884480935}]},"public_addr":"10.20.193.110:6809/884480935","cluster_addr":"10.20.193.110:6811/884480935","heartbeat_back_addr":"10.20.193.110:6815/884480935","heartbeat_front_addr":"10.20.193.110:6813/884480935","state":["exists","up"]},{"osd":7,"uuid":"0dd2edb1-de90-4632-b940-655f96b44dd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6817","nonce":3330554869},{"type":"v1","addr":"10.20.193.110:6819","nonce":3330554869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6821","nonce":3330554869},{"type":"v1","addr":"10.20.193.110:6823","nonce":3330554869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6829","nonce":3330554869},{"type":"v1","addr":"10.20.193.110:6831","nonce":3330554869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6825","nonce":3330554869},{"type":"v1","addr":"10.20.193.110:6827","nonce":3330554869}]},"public_addr":"10.20.193.110:6819/3330554869","cluster_addr":"10.20.193.110:6823/3330554869","heartbeat_back_addr":"10.20.193.110:6831/3330554869","heartbeat_front_addr":"10.20.193.110:6827/3330554869","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.826515+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.811034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.765092+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.662393+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.739442+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.809572+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.843167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.792855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T22:05:11.447 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-24T22:05:11.447 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-24T22:05:11.448 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-24T22:05:11.448 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2026-02-24T22:05:11.449 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2026-02-24T22:05:11.449 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2026-02-24T22:05:11.450 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2026-02-24T22:05:11.450 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2026-02-24T22:05:11.541 INFO:teuthology.orchestra.run.trial046.stdout:55834574851 2026-02-24T22:05:11.542 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2026-02-24T22:05:11.542 INFO:teuthology.orchestra.run.trial046.stdout:55834574851 2026-02-24T22:05:11.543 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2026-02-24T22:05:11.544 INFO:teuthology.orchestra.run.trial046.stdout:55834574852 2026-02-24T22:05:11.545 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-24T22:05:11.546 INFO:teuthology.orchestra.run.trial046.stdout:55834574851 2026-02-24T22:05:11.547 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2026-02-24T22:05:11.548 INFO:teuthology.orchestra.run.trial046.stdout:55834574851 2026-02-24T22:05:11.548 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-24T22:05:11.550 INFO:teuthology.orchestra.run.trial046.stdout:55834574851 2026-02-24T22:05:11.550 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2026-02-24T22:05:11.552 INFO:teuthology.orchestra.run.trial046.stdout:55834574852 2026-02-24T22:05:11.552 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-24T22:05:11.554 INFO:teuthology.orchestra.run.trial046.stdout:55834574852 2026-02-24T22:05:11.554 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-24T22:05:11.703 INFO:teuthology.orchestra.run.trial046.stdout:55834574851 2026-02-24T22:05:11.708 INFO:teuthology.orchestra.run.trial046.stdout:55834574851 2026-02-24T22:05:11.716 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.5 2026-02-24T22:05:11.716 INFO:teuthology.orchestra.run.trial046.stdout:55834574851 2026-02-24T22:05:11.717 DEBUG:teuthology.parallel:result is None 2026-02-24T22:05:11.721 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.7 2026-02-24T22:05:11.721 DEBUG:teuthology.parallel:result is None 2026-02-24T22:05:11.726 INFO:teuthology.orchestra.run.trial046.stdout:55834574852 2026-02-24T22:05:11.728 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.6 2026-02-24T22:05:11.729 DEBUG:teuthology.parallel:result is None 2026-02-24T22:05:11.739 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.3 2026-02-24T22:05:11.739 DEBUG:teuthology.parallel:result is None 2026-02-24T22:05:11.756 INFO:teuthology.orchestra.run.trial046.stdout:55834574851 2026-02-24T22:05:11.760 INFO:teuthology.orchestra.run.trial046.stdout:55834574852 2026-02-24T22:05:11.763 INFO:teuthology.orchestra.run.trial046.stdout:55834574852 2026-02-24T22:05:11.768 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.2 2026-02-24T22:05:11.768 DEBUG:teuthology.parallel:result is None 2026-02-24T22:05:11.774 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.0 2026-02-24T22:05:11.774 DEBUG:teuthology.parallel:result is None 2026-02-24T22:05:11.776 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.1 2026-02-24T22:05:11.777 DEBUG:teuthology.parallel:result is None 2026-02-24T22:05:11.787 INFO:teuthology.orchestra.run.trial046.stdout:55834574851 2026-02-24T22:05:11.800 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.4 2026-02-24T22:05:11.800 DEBUG:teuthology.parallel:result is None 2026-02-24T22:05:11.800 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-24T22:05:11.800 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-24T22:05:11.995 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:05:11.996 INFO:teuthology.orchestra.run.trial046.stderr:dumped all 2026-02-24T22:05:12.006 INFO:teuthology.orchestra.run.trial046.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2026-02-24T22:05:11.586591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15,"ondisk_log_size":15,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218680,"kb_used_data":2744,"kb_used_omap":64,"kb_used_meta":215487,"kb_avail":754756040,"statfs":{"total":773094113280,"available":772870184960,"internally_reserved":0,"allocated":2809856,"data_stored":1716438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":66289,"internal_metadata":220658959},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.902778"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-24T22:05:10.808178+0000","last_change":"2026-02-24T22:05:10.808402+0000","last_active":"2026-02-24T22:05:10.808178+0000","last_peered":"2026-02-24T22:05:10.808178+0000","last_clean":"2026-02-24T22:05:10.808178+0000","last_became_active":"2026-02-24T22:05:08.689542+0000","last_became_peered":"2026-02-24T22:05:08.689542+0000","last_unstale":"2026-02-24T22:05:10.808178+0000","last_undegraded":"2026-02-24T22:05:10.808178+0000","last_fullsized":"2026-02-24T22:05:10.808178+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:23:22.404541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038816099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-24T22:05:10.689267+0000","last_change":"2026-02-24T22:05:10.689444+0000","last_active":"2026-02-24T22:05:10.689267+0000","last_peered":"2026-02-24T22:05:10.689267+0000","last_clean":"2026-02-24T22:05:10.689267+0000","last_became_active":"2026-02-24T22:05:08.682913+0000","last_became_peered":"2026-02-24T22:05:08.682913+0000","last_unstale":"2026-02-24T22:05:10.689267+0000","last_undegraded":"2026-02-24T22:05:10.689267+0000","last_fullsized":"2026-02-24T22:05:10.689267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:20:01.744772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000359878,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-24T22:05:10.808359+0000","last_change":"2026-02-24T22:05:10.808508+0000","last_active":"2026-02-24T22:05:10.808359+0000","last_peered":"2026-02-24T22:05:10.808359+0000","last_clean":"2026-02-24T22:05:10.808359+0000","last_became_active":"2026-02-24T22:05:08.765129+0000","last_became_peered":"2026-02-24T22:05:08.765129+0000","last_unstale":"2026-02-24T22:05:10.808359+0000","last_undegraded":"2026-02-24T22:05:10.808359+0000","last_fullsized":"2026-02-24T22:05:10.808359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:05:15.499818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439276,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-24T22:05:10.689235+0000","last_change":"2026-02-24T22:05:10.689379+0000","last_active":"2026-02-24T22:05:10.689235+0000","last_peered":"2026-02-24T22:05:10.689235+0000","last_clean":"2026-02-24T22:05:10.689235+0000","last_became_active":"2026-02-24T22:05:08.758282+0000","last_became_peered":"2026-02-24T22:05:08.758282+0000","last_unstale":"2026-02-24T22:05:10.689235+0000","last_undegraded":"2026-02-24T22:05:10.689235+0000","last_fullsized":"2026-02-24T22:05:10.689235+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:28:00.429651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033355899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"20'2","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-24T22:05:10.695226+0000","last_change":"2026-02-24T22:05:10.695226+0000","last_active":"2026-02-24T22:05:10.695226+0000","last_peered":"2026-02-24T22:05:10.695226+0000","last_clean":"2026-02-24T22:05:10.695226+0000","last_became_active":"2026-02-24T22:05:08.690042+0000","last_became_peered":"2026-02-24T22:05:08.690042+0000","last_unstale":"2026-02-24T22:05:10.695226+0000","last_undegraded":"2026-02-24T22:05:10.695226+0000","last_fullsized":"2026-02-24T22:05:10.695226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:27:03.349026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010128400000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-24T22:05:10.684276+0000","last_change":"2026-02-24T22:05:10.684538+0000","last_active":"2026-02-24T22:05:10.684276+0000","last_peered":"2026-02-24T22:05:10.684276+0000","last_clean":"2026-02-24T22:05:10.684276+0000","last_became_active":"2026-02-24T22:05:08.682035+0000","last_became_peered":"2026-02-24T22:05:08.682035+0000","last_unstale":"2026-02-24T22:05:10.684276+0000","last_undegraded":"2026-02-24T22:05:10.684276+0000","last_fullsized":"2026-02-24T22:05:10.684276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:59:46.243250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047531599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-24T22:05:10.808411+0000","last_change":"2026-02-24T22:05:10.808592+0000","last_active":"2026-02-24T22:05:10.808411+0000","last_peered":"2026-02-24T22:05:10.808411+0000","last_clean":"2026-02-24T22:05:10.808411+0000","last_became_active":"2026-02-24T22:05:08.689972+0000","last_became_peered":"2026-02-24T22:05:08.689972+0000","last_unstale":"2026-02-24T22:05:10.808411+0000","last_undegraded":"2026-02-24T22:05:10.808411+0000","last_fullsized":"2026-02-24T22:05:10.808411+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:59:56.330904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048345700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"18'1","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-24T22:05:10.694469+0000","last_change":"2026-02-24T22:05:10.694707+0000","last_active":"2026-02-24T22:05:10.694469+0000","last_peered":"2026-02-24T22:05:10.694469+0000","last_clean":"2026-02-24T22:05:10.694469+0000","last_became_active":"2026-02-24T22:05:08.689983+0000","last_became_peered":"2026-02-24T22:05:08.689983+0000","last_unstale":"2026-02-24T22:05:10.694469+0000","last_undegraded":"2026-02-24T22:05:10.694469+0000","last_fullsized":"2026-02-24T22:05:10.694469+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:42:42.134312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046251899999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'112","reported_seq":155,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-24T22:05:10.808042+0000","last_change":"2026-02-24T22:05:06.709275+0000","last_active":"2026-02-24T22:05:10.808042+0000","last_peered":"2026-02-24T22:05:10.808042+0000","last_clean":"2026-02-24T22:05:10.808042+0000","last_became_active":"2026-02-24T22:05:06.708577+0000","last_became_peered":"2026-02-24T22:05:06.708577+0000","last_unstale":"2026-02-24T22:05:10.808042+0000","last_undegraded":"2026-02-24T22:05:10.808042+0000","last_fullsized":"2026-02-24T22:05:10.808042+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:05.664397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:05.664397+0000","last_clean_scrub_stamp":"2026-02-24T22:05:05.664397+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:46:55.314440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":780,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":798720,"data_stored":658784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8770,"internal_metadata":27581886},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":200,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":204800,"data_stored":68416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8774,"internal_metadata":27581882},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":204,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":208896,"data_stored":68435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8775,"internal_metadata":27581881},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27160,"kb_used_data":200,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":204800,"data_stored":68416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8124,"internal_metadata":27582532},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":188,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":192512,"data_stored":62584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8122,"internal_metadata":27582534},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":200,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":204800,"data_stored":68416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6824,"internal_metadata":27583832},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":204,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":208896,"data_stored":68435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8775,"internal_metadata":27581881},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":768,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":786432,"data_stored":652952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8125,"internal_metadata":27582531},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T22:05:12.007 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-24T22:05:12.167 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:05:12.167 INFO:teuthology.orchestra.run.trial046.stderr:dumped all 2026-02-24T22:05:12.177 INFO:teuthology.orchestra.run.trial046.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2026-02-24T22:05:11.586591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15,"ondisk_log_size":15,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218680,"kb_used_data":2744,"kb_used_omap":64,"kb_used_meta":215487,"kb_avail":754756040,"statfs":{"total":773094113280,"available":772870184960,"internally_reserved":0,"allocated":2809856,"data_stored":1716438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":66289,"internal_metadata":220658959},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.902778"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-24T22:05:10.808178+0000","last_change":"2026-02-24T22:05:10.808402+0000","last_active":"2026-02-24T22:05:10.808178+0000","last_peered":"2026-02-24T22:05:10.808178+0000","last_clean":"2026-02-24T22:05:10.808178+0000","last_became_active":"2026-02-24T22:05:08.689542+0000","last_became_peered":"2026-02-24T22:05:08.689542+0000","last_unstale":"2026-02-24T22:05:10.808178+0000","last_undegraded":"2026-02-24T22:05:10.808178+0000","last_fullsized":"2026-02-24T22:05:10.808178+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:23:22.404541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038816099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-24T22:05:10.689267+0000","last_change":"2026-02-24T22:05:10.689444+0000","last_active":"2026-02-24T22:05:10.689267+0000","last_peered":"2026-02-24T22:05:10.689267+0000","last_clean":"2026-02-24T22:05:10.689267+0000","last_became_active":"2026-02-24T22:05:08.682913+0000","last_became_peered":"2026-02-24T22:05:08.682913+0000","last_unstale":"2026-02-24T22:05:10.689267+0000","last_undegraded":"2026-02-24T22:05:10.689267+0000","last_fullsized":"2026-02-24T22:05:10.689267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:20:01.744772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000359878,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-24T22:05:10.808359+0000","last_change":"2026-02-24T22:05:10.808508+0000","last_active":"2026-02-24T22:05:10.808359+0000","last_peered":"2026-02-24T22:05:10.808359+0000","last_clean":"2026-02-24T22:05:10.808359+0000","last_became_active":"2026-02-24T22:05:08.765129+0000","last_became_peered":"2026-02-24T22:05:08.765129+0000","last_unstale":"2026-02-24T22:05:10.808359+0000","last_undegraded":"2026-02-24T22:05:10.808359+0000","last_fullsized":"2026-02-24T22:05:10.808359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:05:15.499818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439276,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-24T22:05:10.689235+0000","last_change":"2026-02-24T22:05:10.689379+0000","last_active":"2026-02-24T22:05:10.689235+0000","last_peered":"2026-02-24T22:05:10.689235+0000","last_clean":"2026-02-24T22:05:10.689235+0000","last_became_active":"2026-02-24T22:05:08.758282+0000","last_became_peered":"2026-02-24T22:05:08.758282+0000","last_unstale":"2026-02-24T22:05:10.689235+0000","last_undegraded":"2026-02-24T22:05:10.689235+0000","last_fullsized":"2026-02-24T22:05:10.689235+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:28:00.429651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033355899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"20'2","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-24T22:05:10.695226+0000","last_change":"2026-02-24T22:05:10.695226+0000","last_active":"2026-02-24T22:05:10.695226+0000","last_peered":"2026-02-24T22:05:10.695226+0000","last_clean":"2026-02-24T22:05:10.695226+0000","last_became_active":"2026-02-24T22:05:08.690042+0000","last_became_peered":"2026-02-24T22:05:08.690042+0000","last_unstale":"2026-02-24T22:05:10.695226+0000","last_undegraded":"2026-02-24T22:05:10.695226+0000","last_fullsized":"2026-02-24T22:05:10.695226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:27:03.349026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010128400000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-24T22:05:10.684276+0000","last_change":"2026-02-24T22:05:10.684538+0000","last_active":"2026-02-24T22:05:10.684276+0000","last_peered":"2026-02-24T22:05:10.684276+0000","last_clean":"2026-02-24T22:05:10.684276+0000","last_became_active":"2026-02-24T22:05:08.682035+0000","last_became_peered":"2026-02-24T22:05:08.682035+0000","last_unstale":"2026-02-24T22:05:10.684276+0000","last_undegraded":"2026-02-24T22:05:10.684276+0000","last_fullsized":"2026-02-24T22:05:10.684276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:59:46.243250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047531599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-24T22:05:10.808411+0000","last_change":"2026-02-24T22:05:10.808592+0000","last_active":"2026-02-24T22:05:10.808411+0000","last_peered":"2026-02-24T22:05:10.808411+0000","last_clean":"2026-02-24T22:05:10.808411+0000","last_became_active":"2026-02-24T22:05:08.689972+0000","last_became_peered":"2026-02-24T22:05:08.689972+0000","last_unstale":"2026-02-24T22:05:10.808411+0000","last_undegraded":"2026-02-24T22:05:10.808411+0000","last_fullsized":"2026-02-24T22:05:10.808411+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:59:56.330904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048345700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"18'1","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-24T22:05:10.694469+0000","last_change":"2026-02-24T22:05:10.694707+0000","last_active":"2026-02-24T22:05:10.694469+0000","last_peered":"2026-02-24T22:05:10.694469+0000","last_clean":"2026-02-24T22:05:10.694469+0000","last_became_active":"2026-02-24T22:05:08.689983+0000","last_became_peered":"2026-02-24T22:05:08.689983+0000","last_unstale":"2026-02-24T22:05:10.694469+0000","last_undegraded":"2026-02-24T22:05:10.694469+0000","last_fullsized":"2026-02-24T22:05:10.694469+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:42:42.134312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046251899999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'112","reported_seq":155,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-24T22:05:10.808042+0000","last_change":"2026-02-24T22:05:06.709275+0000","last_active":"2026-02-24T22:05:10.808042+0000","last_peered":"2026-02-24T22:05:10.808042+0000","last_clean":"2026-02-24T22:05:10.808042+0000","last_became_active":"2026-02-24T22:05:06.708577+0000","last_became_peered":"2026-02-24T22:05:06.708577+0000","last_unstale":"2026-02-24T22:05:10.808042+0000","last_undegraded":"2026-02-24T22:05:10.808042+0000","last_fullsized":"2026-02-24T22:05:10.808042+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:05.664397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:05.664397+0000","last_clean_scrub_stamp":"2026-02-24T22:05:05.664397+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:46:55.314440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":780,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":798720,"data_stored":658784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8770,"internal_metadata":27581886},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":200,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":204800,"data_stored":68416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8774,"internal_metadata":27581882},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":204,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":208896,"data_stored":68435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8775,"internal_metadata":27581881},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27160,"kb_used_data":200,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":204800,"data_stored":68416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8124,"internal_metadata":27582532},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":188,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":192512,"data_stored":62584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8122,"internal_metadata":27582534},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":200,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":204800,"data_stored":68416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6824,"internal_metadata":27583832},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":204,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":208896,"data_stored":68435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8775,"internal_metadata":27581881},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":768,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":786432,"data_stored":652952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8125,"internal_metadata":27582531},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T22:05:12.179 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-24T22:05:12.179 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-24T22:05:12.179 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-24T22:05:12.179 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T22:05:12.354 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:05:12.355 INFO:teuthology.orchestra.run.trial046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-24T22:05:12.366 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-24T22:05:12.366 INFO:teuthology.run_tasks:Running task openssl_keys... 2026-02-24T22:05:12.386 DEBUG:teuthology.task:Applying overrides for task openssl_keys: {'rgw.client.0': {'ca': 'root', 'client': 'client.0', 'embed-key': True}, 'root': {'client': 'client.0', 'cn': 'teuthology', 'install': ['client.0'], 'key-type': 'rsa:4096'}} 2026-02-24T22:05:12.387 DEBUG:teuthology.orchestra.run.trial046:> mkdir -p /home/ubuntu/cephtest/ca 2026-02-24T22:05:12.392 DEBUG:teuthology.orchestra.run.trial046:> openssl req -x509 -nodes -newkey rsa:4096 -keyout /home/ubuntu/cephtest/ca/root.key -days 365 -out /home/ubuntu/cephtest/ca/root.crt -subj /CN=teuthology 2026-02-24T22:05:12.654 INFO:teuthology.orchestra.run.trial046.stderr:.....+....+..+.......+......+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.....+...+......+.....+.+......+........+.+......+..+...............+......+.+...........+.........+.+..+............+..................+...+....+......+...+......+.....+...+.+.....+.+........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.....+...............+....+.....+............+.........+.......+..+...+..................+....+......+..+....+...........+...................+...+..+.+..............+.+..+..................+.......+..+.........+....+...+.....+............+...............+.+..+..................+......+....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2026-02-24T22:05:13.454 INFO:teuthology.orchestra.run.trial046.stderr:....+...+............+..+.+..+.......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.+.....+.............+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+.+...+...+...............+..+.....................+.......+..+...+....+.....+...+...............+.+......+...+..+.........+......+......+...+................+.........+.....+.......+...+..+....+..............+......+.+..+.+...........+..........+.......................+..........+......+......+...........+...+....+..+...................+......+..........................+...+....+.....+...............+.....................+......+...+.+......+......+.........+.....+......+...+.+.....+.+........+....+....................+.+..+...+.+......+.....+....+.....+.+...........................+...........+....+..+...............+.+..+.+......+...+...............+......+..+.........+...+.+.....+.+.....+.+..+...+...+.+........+..........+..................+...+...........+....+.........+.....+.......+.....+...............+..........+...+..+...+...+..................+.......+..+...+..................+.......+........+.........+.............+........+.........+............+.........+....+........+............................+.....+.+.........+.....+......+...+.............+.....+...+.+......+...+.........+......+..+........................................+...............+...+..+...+.+......+..+...+....+......+...+...............+......+........+.+.........+........+....+..............................+.....+......+...+....+..+......+...+....+..+.............+.....+.............+.....+..................+............+.........+...+...+.......+...............+...+..+...+....+...+..+..........+.........+.....+.......+..+......+...................+...+.........+......+...+...........+.+..+...+.......+..+......+.......+........+....+.....+.+...............+...+......+.....+.+...............+........+.....................+...+.......+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2026-02-24T22:05:13.465 INFO:teuthology.orchestra.run.trial046.stderr:----- 2026-02-24T22:05:13.470 DEBUG:teuthology.orchestra.run.trial046:> sudo cp /home/ubuntu/cephtest/ca/root.crt /usr/local/share/ca-certificates/root.crt 2026-02-24T22:05:13.523 DEBUG:teuthology.orchestra.run.trial046:> sudo update-ca-certificates 2026-02-24T22:05:13.573 INFO:teuthology.orchestra.run.trial046.stdout:Updating certificates in /etc/ssl/certs... 2026-02-24T22:05:14.284 INFO:teuthology.orchestra.run.trial046.stderr:rehash: warning: skipping ca-certificates.crt,it does not contain exactly one certificate or CRL 2026-02-24T22:05:14.297 INFO:teuthology.orchestra.run.trial046.stdout:1 added, 0 removed; done. 2026-02-24T22:05:14.298 INFO:teuthology.orchestra.run.trial046.stdout:Running hooks in /etc/ca-certificates/update.d... 2026-02-24T22:05:14.301 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:05:14.493 INFO:teuthology.orchestra.run.trial046.stdout:Adding debian:root.pem 2026-02-24T22:05:14.503 INFO:teuthology.orchestra.run.trial046.stdout:done. 2026-02-24T22:05:14.503 INFO:teuthology.orchestra.run.trial046.stdout:done. 2026-02-24T22:05:14.507 DEBUG:teuthology.orchestra.run.trial046:> mkdir -p /home/ubuntu/cephtest/ca 2026-02-24T22:05:14.553 DEBUG:teuthology.orchestra.run.trial046:> openssl req -nodes -newkey rsa:2048 -keyout /home/ubuntu/cephtest/ca/rgw.client.0.key -out /home/ubuntu/cephtest/ca/rgw.client.0.csr -subj /CN=trial046.front.sepia.ceph.com 2026-02-24T22:05:14.636 INFO:teuthology.orchestra.run.trial046.stderr:.+............+....+..+...+......+.............+...+........+.......+..+.+..+.......+.....+...+.............+.....+.+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.......+..............+.............+........+....+...+..+.+.....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...+..+.......+......+......+.....................+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2026-02-24T22:05:14.679 INFO:teuthology.orchestra.run.trial046.stderr:...+..........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.+...+..+.........+.+..+...............+................+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.+...+......+.+..+...+...+............+.......+........+.........+....+......+..............+.......+........+.........+...+............+.......+...........+.........+....+..+....+.....+...+..........+............+.........+.....+.+.....+.+.....+...+.+...+...+............+...+...+.........+...+..............+...+................+..+....+...+......+...........+...+.+..+....+...........+.........+.+............+...+..+................+..+....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2026-02-24T22:05:14.681 INFO:teuthology.orchestra.run.trial046.stderr:----- 2026-02-24T22:05:14.682 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:05:14.682 DEBUG:teuthology.orchestra.run.trial046:> dd of=/home/ubuntu/cephtest/ca/rgw.client.0.ext 2026-02-24T22:05:14.730 DEBUG:teuthology.orchestra.run.trial046:> openssl x509 -req -in /home/ubuntu/cephtest/ca/rgw.client.0.csr -CA /home/ubuntu/cephtest/ca/root.crt -CAkey /home/ubuntu/cephtest/ca/root.key -CAcreateserial -out /home/ubuntu/cephtest/ca/rgw.client.0.crt -days 365 -sha256 -extfile /home/ubuntu/cephtest/ca/rgw.client.0.ext 2026-02-24T22:05:14.778 INFO:teuthology.orchestra.run.trial046.stderr:Certificate request self-signature ok 2026-02-24T22:05:14.778 INFO:teuthology.orchestra.run.trial046.stderr:subject=CN = trial046.front.sepia.ceph.com 2026-02-24T22:05:14.783 DEBUG:teuthology.orchestra.run.trial046:> rm -f /home/ubuntu/cephtest/ca/rgw.client.0.csr /home/ubuntu/cephtest/ca/root.srl /home/ubuntu/cephtest/ca/rgw.client.0.ext 2026-02-24T22:05:14.829 DEBUG:teuthology.orchestra.run.trial046:> openssl verify -CAfile /home/ubuntu/cephtest/ca/root.crt /home/ubuntu/cephtest/ca/rgw.client.0.crt 2026-02-24T22:05:14.875 INFO:teuthology.orchestra.run.trial046.stdout:/home/ubuntu/cephtest/ca/rgw.client.0.crt: OK 2026-02-24T22:05:14.875 DEBUG:teuthology.orchestra.run.trial046:> cat /home/ubuntu/cephtest/ca/rgw.client.0.key >> /home/ubuntu/cephtest/ca/rgw.client.0.crt 2026-02-24T22:05:14.922 INFO:teuthology.run_tasks:Running task rgw... 2026-02-24T22:05:14.952 DEBUG:tasks.rgw:config is {'client.0': {'ssl certificate': 'rgw.client.0'}} 2026-02-24T22:05:14.952 DEBUG:tasks.rgw:client list is dict_keys(['client.0']) 2026-02-24T22:05:14.953 INFO:tasks.rgw:Creating data pools 2026-02-24T22:05:14.953 DEBUG:tasks.rgw:Obtaining remote for client client.0 2026-02-24T22:05:14.953 DEBUG:teuthology.orchestra.run.trial046:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster ceph 2026-02-24T22:05:15.177 INFO:teuthology.orchestra.run.trial046.stderr:pool 'default.rgw.buckets.data' created 2026-02-24T22:05:15.191 DEBUG:teuthology.orchestra.run.trial046:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster ceph 2026-02-24T22:05:16.177 INFO:teuthology.orchestra.run.trial046.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2026-02-24T22:05:16.194 DEBUG:teuthology.orchestra.run.trial046:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster ceph 2026-02-24T22:05:17.178 INFO:teuthology.orchestra.run.trial046.stderr:pool 'default.rgw.buckets.index' created 2026-02-24T22:05:17.195 DEBUG:teuthology.orchestra.run.trial046:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster ceph 2026-02-24T22:05:18.180 INFO:teuthology.orchestra.run.trial046.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2026-02-24T22:05:18.200 DEBUG:tasks.rgw:Pools created 2026-02-24T22:05:18.200 INFO:tasks.rgw:Configuring compression type = random 2026-02-24T22:05:18.200 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2026-02-24T22:05:18.200 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2026-02-24T22:05:18.201 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2026-02-24T22:05:18.218 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:05:18.218 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:05:20.210 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:20.199+0000 7f601d4e0640 0 -- 10.20.193.46:0/2896659542 >> [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] conn(0x55bc2b96fb00 msgr2=0x55bc2b96ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:05:20.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:20.403+0000 7f601d4e0640 0 --2- 10.20.193.46:0/2896659542 >> [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] conn(0x55bc2b96fb00 0x55bc2b96ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2026-02-24T22:05:20.413 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:20.403+0000 7f601d4e0640 0 -- 10.20.193.46:0/2074843213 >> [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] conn(0x55bc2b998c70 msgr2=0x55bc2b9990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:05:20.418 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:20.411+0000 7f601fa0ca80 4 Realm: () 2026-02-24T22:05:20.418 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:20.411+0000 7f601fa0ca80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:05:20.418 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:20.411+0000 7f601fa0ca80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:05:20.418 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:20.411+0000 7f601fa0ca80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:05:20.418 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:20.411+0000 7f601fa0ca80 20 zonegroup default 2026-02-24T22:05:20.418 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:20.411+0000 7f601fa0ca80 20 rados->read ofs=0 len=0 2026-02-24T22:05:22.195 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.187+0000 7f601fa0ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:05:22.195 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.187+0000 7f601fa0ca80 20 rados->read ofs=0 len=0 2026-02-24T22:05:22.196 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.187+0000 7f601fa0ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:05:22.197 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.187+0000 7f601fa0ca80 20 started sync module instance, tier type = 2026-02-24T22:05:22.197 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.187+0000 7f601fa0ca80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:05:22.198 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.187+0000 7f600e7fc640 5 boost::asio::awaitable, obj_version> > logback_generations::read(const DoutPrefixProvider*):446: oid=data_loggenerations_metadata not found 2026-02-24T22:05:22.198 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.187+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.0 2026-02-24T22:05:22.198 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.187+0000 7f600e7fc640 20 do_open: entering 2026-02-24T22:05:22.908 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.899+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.0 does not exist 2026-02-24T22:05:22.909 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.899+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.1 2026-02-24T22:05:22.909 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.899+0000 7f600dffb640 20 do_open: entering 2026-02-24T22:05:22.909 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.1 does not exist 2026-02-24T22:05:22.910 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.2 2026-02-24T22:05:22.910 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f600d7fa640 20 do_open: entering 2026-02-24T22:05:22.910 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.2 does not exist 2026-02-24T22:05:22.911 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.3 2026-02-24T22:05:22.911 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f601e4e2640 20 do_open: entering 2026-02-24T22:05:22.911 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.3 does not exist 2026-02-24T22:05:22.911 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.4 2026-02-24T22:05:22.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f600ffff640 20 do_open: entering 2026-02-24T22:05:22.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.4 does not exist 2026-02-24T22:05:22.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.5 2026-02-24T22:05:22.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f600f7fe640 20 do_open: entering 2026-02-24T22:05:22.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.5 does not exist 2026-02-24T22:05:22.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.6 2026-02-24T22:05:22.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f600effd640 20 do_open: entering 2026-02-24T22:05:22.913 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.6 does not exist 2026-02-24T22:05:22.913 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.7 2026-02-24T22:05:22.913 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f601f76b640 20 do_open: entering 2026-02-24T22:05:22.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.7 does not exist 2026-02-24T22:05:22.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.8 2026-02-24T22:05:22.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f600e7fc640 20 do_open: entering 2026-02-24T22:05:22.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.8 does not exist 2026-02-24T22:05:22.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.9 2026-02-24T22:05:22.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f600dffb640 20 do_open: entering 2026-02-24T22:05:22.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.9 does not exist 2026-02-24T22:05:22.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.10 2026-02-24T22:05:22.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.903+0000 7f600d7fa640 20 do_open: entering 2026-02-24T22:05:22.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.10 does not exist 2026-02-24T22:05:22.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.11 2026-02-24T22:05:22.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f601e4e2640 20 do_open: entering 2026-02-24T22:05:22.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.11 does not exist 2026-02-24T22:05:22.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.12 2026-02-24T22:05:22.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600ffff640 20 do_open: entering 2026-02-24T22:05:22.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.12 does not exist 2026-02-24T22:05:22.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.13 2026-02-24T22:05:22.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600f7fe640 20 do_open: entering 2026-02-24T22:05:22.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.13 does not exist 2026-02-24T22:05:22.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.14 2026-02-24T22:05:22.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600effd640 20 do_open: entering 2026-02-24T22:05:22.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.14 does not exist 2026-02-24T22:05:22.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.15 2026-02-24T22:05:22.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f601f76b640 20 do_open: entering 2026-02-24T22:05:22.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.15 does not exist 2026-02-24T22:05:22.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.16 2026-02-24T22:05:22.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600e7fc640 20 do_open: entering 2026-02-24T22:05:22.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.16 does not exist 2026-02-24T22:05:22.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.17 2026-02-24T22:05:22.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600dffb640 20 do_open: entering 2026-02-24T22:05:22.919 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.17 does not exist 2026-02-24T22:05:22.919 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.18 2026-02-24T22:05:22.919 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600d7fa640 20 do_open: entering 2026-02-24T22:05:22.920 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.18 does not exist 2026-02-24T22:05:22.920 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.19 2026-02-24T22:05:22.920 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f601e4e2640 20 do_open: entering 2026-02-24T22:05:22.920 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.19 does not exist 2026-02-24T22:05:22.920 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.20 2026-02-24T22:05:22.920 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600ffff640 20 do_open: entering 2026-02-24T22:05:22.920 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.20 does not exist 2026-02-24T22:05:22.920 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.21 2026-02-24T22:05:22.920 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600f7fe640 20 do_open: entering 2026-02-24T22:05:22.920 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.21 does not exist 2026-02-24T22:05:22.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.22 2026-02-24T22:05:22.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600effd640 20 do_open: entering 2026-02-24T22:05:22.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.22 does not exist 2026-02-24T22:05:22.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.23 2026-02-24T22:05:22.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f601f76b640 20 do_open: entering 2026-02-24T22:05:22.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.23 does not exist 2026-02-24T22:05:22.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.24 2026-02-24T22:05:22.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600e7fc640 20 do_open: entering 2026-02-24T22:05:22.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.24 does not exist 2026-02-24T22:05:22.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.25 2026-02-24T22:05:22.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600dffb640 20 do_open: entering 2026-02-24T22:05:22.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.25 does not exist 2026-02-24T22:05:22.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.26 2026-02-24T22:05:22.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f600d7fa640 20 do_open: entering 2026-02-24T22:05:22.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.26 does not exist 2026-02-24T22:05:22.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.27 2026-02-24T22:05:22.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.907+0000 7f601e4e2640 20 do_open: entering 2026-02-24T22:05:22.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.27 does not exist 2026-02-24T22:05:22.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.28 2026-02-24T22:05:22.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600ffff640 20 do_open: entering 2026-02-24T22:05:22.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.28 does not exist 2026-02-24T22:05:22.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.29 2026-02-24T22:05:22.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600f7fe640 20 do_open: entering 2026-02-24T22:05:22.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.29 does not exist 2026-02-24T22:05:22.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.30 2026-02-24T22:05:22.923 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600effd640 20 do_open: entering 2026-02-24T22:05:22.923 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.30 does not exist 2026-02-24T22:05:22.923 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.31 2026-02-24T22:05:22.923 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f601f76b640 20 do_open: entering 2026-02-24T22:05:22.923 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.31 does not exist 2026-02-24T22:05:22.923 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.32 2026-02-24T22:05:22.923 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600e7fc640 20 do_open: entering 2026-02-24T22:05:22.923 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.32 does not exist 2026-02-24T22:05:22.923 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.33 2026-02-24T22:05:22.923 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600dffb640 20 do_open: entering 2026-02-24T22:05:22.923 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.33 does not exist 2026-02-24T22:05:22.923 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.34 2026-02-24T22:05:22.923 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600d7fa640 20 do_open: entering 2026-02-24T22:05:22.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.34 does not exist 2026-02-24T22:05:22.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.35 2026-02-24T22:05:22.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f601e4e2640 20 do_open: entering 2026-02-24T22:05:22.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.35 does not exist 2026-02-24T22:05:22.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.36 2026-02-24T22:05:22.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600ffff640 20 do_open: entering 2026-02-24T22:05:22.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.36 does not exist 2026-02-24T22:05:22.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.37 2026-02-24T22:05:22.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600f7fe640 20 do_open: entering 2026-02-24T22:05:22.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.37 does not exist 2026-02-24T22:05:22.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.38 2026-02-24T22:05:22.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600effd640 20 do_open: entering 2026-02-24T22:05:22.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.38 does not exist 2026-02-24T22:05:22.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.39 2026-02-24T22:05:22.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f601f76b640 20 do_open: entering 2026-02-24T22:05:22.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.39 does not exist 2026-02-24T22:05:22.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.40 2026-02-24T22:05:22.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600e7fc640 20 do_open: entering 2026-02-24T22:05:22.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.40 does not exist 2026-02-24T22:05:22.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.41 2026-02-24T22:05:22.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600dffb640 20 do_open: entering 2026-02-24T22:05:22.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.41 does not exist 2026-02-24T22:05:22.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.42 2026-02-24T22:05:22.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600d7fa640 20 do_open: entering 2026-02-24T22:05:22.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.42 does not exist 2026-02-24T22:05:22.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.43 2026-02-24T22:05:22.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f601e4e2640 20 do_open: entering 2026-02-24T22:05:22.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.43 does not exist 2026-02-24T22:05:22.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.44 2026-02-24T22:05:22.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600ffff640 20 do_open: entering 2026-02-24T22:05:22.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.44 does not exist 2026-02-24T22:05:22.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.45 2026-02-24T22:05:22.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600f7fe640 20 do_open: entering 2026-02-24T22:05:22.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.45 does not exist 2026-02-24T22:05:22.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.46 2026-02-24T22:05:22.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f600effd640 20 do_open: entering 2026-02-24T22:05:22.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.46 does not exist 2026-02-24T22:05:22.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.47 2026-02-24T22:05:22.933 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.911+0000 7f601f76b640 20 do_open: entering 2026-02-24T22:05:22.933 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.47 does not exist 2026-02-24T22:05:22.933 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.48 2026-02-24T22:05:22.933 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600e7fc640 20 do_open: entering 2026-02-24T22:05:22.933 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.48 does not exist 2026-02-24T22:05:22.933 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.49 2026-02-24T22:05:22.933 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600dffb640 20 do_open: entering 2026-02-24T22:05:22.933 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.49 does not exist 2026-02-24T22:05:22.933 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.50 2026-02-24T22:05:22.933 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600d7fa640 20 do_open: entering 2026-02-24T22:05:22.933 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.50 does not exist 2026-02-24T22:05:22.933 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.51 2026-02-24T22:05:22.933 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f601e4e2640 20 do_open: entering 2026-02-24T22:05:22.934 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.51 does not exist 2026-02-24T22:05:22.934 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.52 2026-02-24T22:05:22.934 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600ffff640 20 do_open: entering 2026-02-24T22:05:22.934 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.52 does not exist 2026-02-24T22:05:22.934 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.53 2026-02-24T22:05:22.934 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600f7fe640 20 do_open: entering 2026-02-24T22:05:22.934 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.53 does not exist 2026-02-24T22:05:22.934 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.54 2026-02-24T22:05:22.934 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600effd640 20 do_open: entering 2026-02-24T22:05:22.934 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.54 does not exist 2026-02-24T22:05:22.934 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.55 2026-02-24T22:05:22.934 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f601f76b640 20 do_open: entering 2026-02-24T22:05:22.935 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.55 does not exist 2026-02-24T22:05:22.935 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.56 2026-02-24T22:05:22.935 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600e7fc640 20 do_open: entering 2026-02-24T22:05:22.935 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.56 does not exist 2026-02-24T22:05:22.935 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.57 2026-02-24T22:05:22.935 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600dffb640 20 do_open: entering 2026-02-24T22:05:22.935 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.57 does not exist 2026-02-24T22:05:22.935 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.58 2026-02-24T22:05:22.935 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600d7fa640 20 do_open: entering 2026-02-24T22:05:22.935 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.58 does not exist 2026-02-24T22:05:22.935 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.59 2026-02-24T22:05:22.935 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f601e4e2640 20 do_open: entering 2026-02-24T22:05:22.935 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.59 does not exist 2026-02-24T22:05:22.936 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.60 2026-02-24T22:05:22.936 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600ffff640 20 do_open: entering 2026-02-24T22:05:22.936 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.60 does not exist 2026-02-24T22:05:22.936 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.61 2026-02-24T22:05:22.936 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600f7fe640 20 do_open: entering 2026-02-24T22:05:22.936 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.61 does not exist 2026-02-24T22:05:22.936 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.62 2026-02-24T22:05:22.936 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600effd640 20 do_open: entering 2026-02-24T22:05:22.936 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.62 does not exist 2026-02-24T22:05:22.936 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.63 2026-02-24T22:05:22.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f601f76b640 20 do_open: entering 2026-02-24T22:05:22.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.63 does not exist 2026-02-24T22:05:22.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.64 2026-02-24T22:05:22.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600e7fc640 20 do_open: entering 2026-02-24T22:05:22.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.64 does not exist 2026-02-24T22:05:22.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.65 2026-02-24T22:05:22.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600dffb640 20 do_open: entering 2026-02-24T22:05:22.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.65 does not exist 2026-02-24T22:05:22.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.66 2026-02-24T22:05:22.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600d7fa640 20 do_open: entering 2026-02-24T22:05:22.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.66 does not exist 2026-02-24T22:05:22.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.67 2026-02-24T22:05:22.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f601e4e2640 20 do_open: entering 2026-02-24T22:05:22.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.67 does not exist 2026-02-24T22:05:22.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.68 2026-02-24T22:05:22.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600ffff640 20 do_open: entering 2026-02-24T22:05:22.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.68 does not exist 2026-02-24T22:05:22.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.69 2026-02-24T22:05:22.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600f7fe640 20 do_open: entering 2026-02-24T22:05:22.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.69 does not exist 2026-02-24T22:05:22.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.70 2026-02-24T22:05:22.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.915+0000 7f600effd640 20 do_open: entering 2026-02-24T22:05:22.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.70 does not exist 2026-02-24T22:05:22.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.71 2026-02-24T22:05:22.939 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f601f76b640 20 do_open: entering 2026-02-24T22:05:22.939 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.71 does not exist 2026-02-24T22:05:22.939 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.72 2026-02-24T22:05:22.939 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600e7fc640 20 do_open: entering 2026-02-24T22:05:22.939 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.72 does not exist 2026-02-24T22:05:22.939 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.73 2026-02-24T22:05:22.939 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600dffb640 20 do_open: entering 2026-02-24T22:05:22.939 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.73 does not exist 2026-02-24T22:05:22.939 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.74 2026-02-24T22:05:22.939 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600d7fa640 20 do_open: entering 2026-02-24T22:05:22.939 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.74 does not exist 2026-02-24T22:05:22.939 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.75 2026-02-24T22:05:22.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f601e4e2640 20 do_open: entering 2026-02-24T22:05:22.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.75 does not exist 2026-02-24T22:05:22.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.76 2026-02-24T22:05:22.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600ffff640 20 do_open: entering 2026-02-24T22:05:22.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.76 does not exist 2026-02-24T22:05:22.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.77 2026-02-24T22:05:22.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600f7fe640 20 do_open: entering 2026-02-24T22:05:22.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.77 does not exist 2026-02-24T22:05:22.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.78 2026-02-24T22:05:22.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600effd640 20 do_open: entering 2026-02-24T22:05:22.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.78 does not exist 2026-02-24T22:05:22.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.79 2026-02-24T22:05:22.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f601f76b640 20 do_open: entering 2026-02-24T22:05:22.941 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.79 does not exist 2026-02-24T22:05:22.941 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.80 2026-02-24T22:05:22.941 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600e7fc640 20 do_open: entering 2026-02-24T22:05:22.941 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.80 does not exist 2026-02-24T22:05:22.941 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.81 2026-02-24T22:05:22.941 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600dffb640 20 do_open: entering 2026-02-24T22:05:22.941 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.81 does not exist 2026-02-24T22:05:22.941 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.82 2026-02-24T22:05:22.941 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600d7fa640 20 do_open: entering 2026-02-24T22:05:22.941 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.82 does not exist 2026-02-24T22:05:22.941 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.83 2026-02-24T22:05:22.941 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f601e4e2640 20 do_open: entering 2026-02-24T22:05:22.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.83 does not exist 2026-02-24T22:05:22.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.84 2026-02-24T22:05:22.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600ffff640 20 do_open: entering 2026-02-24T22:05:22.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.84 does not exist 2026-02-24T22:05:22.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.85 2026-02-24T22:05:22.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600f7fe640 20 do_open: entering 2026-02-24T22:05:22.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.85 does not exist 2026-02-24T22:05:22.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.86 2026-02-24T22:05:22.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600effd640 20 do_open: entering 2026-02-24T22:05:22.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.86 does not exist 2026-02-24T22:05:22.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.87 2026-02-24T22:05:22.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f601f76b640 20 do_open: entering 2026-02-24T22:05:22.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.87 does not exist 2026-02-24T22:05:22.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.88 2026-02-24T22:05:22.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600e7fc640 20 do_open: entering 2026-02-24T22:05:22.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.88 does not exist 2026-02-24T22:05:22.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.89 2026-02-24T22:05:22.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600dffb640 20 do_open: entering 2026-02-24T22:05:22.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.89 does not exist 2026-02-24T22:05:22.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.90 2026-02-24T22:05:22.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600d7fa640 20 do_open: entering 2026-02-24T22:05:22.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.90 does not exist 2026-02-24T22:05:22.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.91 2026-02-24T22:05:22.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f601e4e2640 20 do_open: entering 2026-02-24T22:05:22.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.91 does not exist 2026-02-24T22:05:22.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.92 2026-02-24T22:05:22.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.919+0000 7f600ffff640 20 do_open: entering 2026-02-24T22:05:22.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.92 does not exist 2026-02-24T22:05:22.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.93 2026-02-24T22:05:22.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600f7fe640 20 do_open: entering 2026-02-24T22:05:22.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.93 does not exist 2026-02-24T22:05:22.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.94 2026-02-24T22:05:22.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600effd640 20 do_open: entering 2026-02-24T22:05:22.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.94 does not exist 2026-02-24T22:05:22.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.95 2026-02-24T22:05:22.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f601f76b640 20 do_open: entering 2026-02-24T22:05:22.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.95 does not exist 2026-02-24T22:05:22.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.96 2026-02-24T22:05:22.945 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600e7fc640 20 do_open: entering 2026-02-24T22:05:22.948 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.96 does not exist 2026-02-24T22:05:22.948 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.97 2026-02-24T22:05:22.948 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600dffb640 20 do_open: entering 2026-02-24T22:05:22.948 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.97 does not exist 2026-02-24T22:05:22.948 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.98 2026-02-24T22:05:22.949 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600d7fa640 20 do_open: entering 2026-02-24T22:05:22.949 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.98 does not exist 2026-02-24T22:05:22.949 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.99 2026-02-24T22:05:22.949 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f601e4e2640 20 do_open: entering 2026-02-24T22:05:22.949 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.99 does not exist 2026-02-24T22:05:22.949 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.100 2026-02-24T22:05:22.949 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600ffff640 20 do_open: entering 2026-02-24T22:05:22.949 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.100 does not exist 2026-02-24T22:05:22.949 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.101 2026-02-24T22:05:22.949 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600f7fe640 20 do_open: entering 2026-02-24T22:05:22.949 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.101 does not exist 2026-02-24T22:05:22.949 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.102 2026-02-24T22:05:22.949 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600effd640 20 do_open: entering 2026-02-24T22:05:22.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.102 does not exist 2026-02-24T22:05:22.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.103 2026-02-24T22:05:22.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f601f76b640 20 do_open: entering 2026-02-24T22:05:22.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.103 does not exist 2026-02-24T22:05:22.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.104 2026-02-24T22:05:22.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600e7fc640 20 do_open: entering 2026-02-24T22:05:22.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.104 does not exist 2026-02-24T22:05:22.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.105 2026-02-24T22:05:22.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600dffb640 20 do_open: entering 2026-02-24T22:05:22.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.105 does not exist 2026-02-24T22:05:22.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.106 2026-02-24T22:05:22.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600d7fa640 20 do_open: entering 2026-02-24T22:05:22.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.106 does not exist 2026-02-24T22:05:22.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.107 2026-02-24T22:05:22.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f601e4e2640 20 do_open: entering 2026-02-24T22:05:22.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.107 does not exist 2026-02-24T22:05:22.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.108 2026-02-24T22:05:22.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600ffff640 20 do_open: entering 2026-02-24T22:05:22.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.108 does not exist 2026-02-24T22:05:22.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.109 2026-02-24T22:05:22.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.923+0000 7f600f7fe640 20 do_open: entering 2026-02-24T22:05:22.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.109 does not exist 2026-02-24T22:05:22.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.110 2026-02-24T22:05:22.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600effd640 20 do_open: entering 2026-02-24T22:05:22.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.110 does not exist 2026-02-24T22:05:22.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.111 2026-02-24T22:05:22.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f601f76b640 20 do_open: entering 2026-02-24T22:05:22.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.111 does not exist 2026-02-24T22:05:22.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.112 2026-02-24T22:05:22.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600e7fc640 20 do_open: entering 2026-02-24T22:05:22.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.112 does not exist 2026-02-24T22:05:22.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.113 2026-02-24T22:05:22.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600dffb640 20 do_open: entering 2026-02-24T22:05:22.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.113 does not exist 2026-02-24T22:05:22.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.114 2026-02-24T22:05:22.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600d7fa640 20 do_open: entering 2026-02-24T22:05:22.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.114 does not exist 2026-02-24T22:05:22.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.115 2026-02-24T22:05:22.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f601e4e2640 20 do_open: entering 2026-02-24T22:05:22.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.115 does not exist 2026-02-24T22:05:22.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.116 2026-02-24T22:05:22.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600ffff640 20 do_open: entering 2026-02-24T22:05:22.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.116 does not exist 2026-02-24T22:05:22.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.117 2026-02-24T22:05:22.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600f7fe640 20 do_open: entering 2026-02-24T22:05:22.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.117 does not exist 2026-02-24T22:05:22.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.118 2026-02-24T22:05:22.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600effd640 20 do_open: entering 2026-02-24T22:05:22.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.118 does not exist 2026-02-24T22:05:22.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.119 2026-02-24T22:05:22.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f601f76b640 20 do_open: entering 2026-02-24T22:05:22.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.119 does not exist 2026-02-24T22:05:22.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.120 2026-02-24T22:05:22.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600e7fc640 20 do_open: entering 2026-02-24T22:05:22.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.120 does not exist 2026-02-24T22:05:22.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600dffb640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.121 2026-02-24T22:05:22.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600dffb640 20 do_open: entering 2026-02-24T22:05:22.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.121 does not exist 2026-02-24T22:05:22.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600d7fa640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.122 2026-02-24T22:05:22.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600d7fa640 20 do_open: entering 2026-02-24T22:05:22.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.122 does not exist 2026-02-24T22:05:22.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f601e4e2640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.123 2026-02-24T22:05:22.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f601e4e2640 20 do_open: entering 2026-02-24T22:05:22.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.123 does not exist 2026-02-24T22:05:22.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600ffff640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.124 2026-02-24T22:05:22.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600ffff640 20 do_open: entering 2026-02-24T22:05:22.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.124 does not exist 2026-02-24T22:05:22.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600f7fe640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.125 2026-02-24T22:05:22.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600f7fe640 20 do_open: entering 2026-02-24T22:05:22.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.125 does not exist 2026-02-24T22:05:22.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600effd640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.126 2026-02-24T22:05:22.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600effd640 20 do_open: entering 2026-02-24T22:05:22.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.126 does not exist 2026-02-24T22:05:22.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f601f76b640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):59 probing obj=data_log.127 2026-02-24T22:05:22.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f601f76b640 20 do_open: entering 2026-02-24T22:05:22.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600e7fc640 20 boost::asio::awaitable {anonymous}::probe_shard(const DoutPrefixProvider*, neorados::RADOS, const neorados::Object&, const neorados::IOContext&, bool&):78: obj=data_log.127 does not exist 2026-02-24T22:05:22.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.927+0000 7f600e7fc640 20 do_create: entering 2026-02-24T22:05:22.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.931+0000 7f600dffb640 20 do_open: entering 2026-02-24T22:05:22.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.931+0000 7f601fa0ca80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:05:22.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:22.931+0000 7f601fa0ca80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:05:25.201 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.191+0000 7f601fa0ca80 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.meta 2026-02-24T22:05:25.201 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.191+0000 7f601fa0ca80 5 note: GC not initialized 2026-02-24T22:05:25.201 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.195+0000 7f5fc6fe5640 20 reqs_thread_entry: start 2026-02-24T22:05:25.249 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.239+0000 7f601fa0ca80 20 init_complete bucket index max shards: 1 2026-02-24T22:05:25.249 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.239+0000 7f601fa0ca80 20 Filter name: none 2026-02-24T22:05:25.249 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.239+0000 7f5fc57e2640 20 reqs_thread_entry: start 2026-02-24T22:05:25.469 INFO:teuthology.orchestra.run.trial046.stdout:[] 2026-02-24T22:05:25.470 DEBUG:tasks.util.rgw: json result: [] 2026-02-24T22:05:25.470 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2026-02-24T22:05:25.470 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2026-02-24T22:05:25.471 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement modify --rgw-zone default --placement-id default-placement --compression random 2026-02-24T22:05:25.533 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:05:25.533 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:05:25.546 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.539+0000 7f421d889a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:05:25.546 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.539+0000 7f421d889a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:05:25.546 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.539+0000 7f41c9fe3640 20 reqs_thread_entry: start 2026-02-24T22:05:25.555 INFO:teuthology.orchestra.run.trial046.stdout:{"id":"301937db-3a1f-489c-b387-e7c21a76f9b9","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","dedup_pool":"default.rgw.dedup","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","bucket_logging_pool":"default.rgw.log:logging","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":"","restore_pool":"default.rgw.log:restore"} 2026-02-24T22:05:25.556 DEBUG:tasks.util.rgw: json result: {'id': '301937db-3a1f-489c-b387-e7c21a76f9b9', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'dedup_pool': 'default.rgw.dedup', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'bucket_logging_pool': 'default.rgw.log:logging', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': '', 'restore_pool': 'default.rgw.log:restore'} 2026-02-24T22:05:25.556 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2026-02-24T22:05:25.556 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2026-02-24T22:05:25.557 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2026-02-24T22:05:25.614 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:05:25.614 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:05:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.623+0000 7fd9c3737a80 4 Realm: () 2026-02-24T22:05:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.623+0000 7fd9c3737a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:05:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.623+0000 7fd9c3737a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:05:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.623+0000 7fd9c3737a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:05:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.623+0000 7fd9c3737a80 20 zonegroup default 2026-02-24T22:05:25.631 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.623+0000 7fd9c3737a80 20 rados->read ofs=0 len=0 2026-02-24T22:05:25.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.623+0000 7fd9c3737a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:05:25.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.623+0000 7fd9c3737a80 20 rados->read ofs=0 len=0 2026-02-24T22:05:25.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.623+0000 7fd9c3737a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:05:25.633 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.623+0000 7fd9c3737a80 20 started sync module instance, tier type = 2026-02-24T22:05:25.633 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.623+0000 7fd9c3737a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:05:25.634 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.627+0000 7fd9c3737a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:05:25.634 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.627+0000 7fd9c3737a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:05:25.634 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.627+0000 7fd9c3737a80 5 note: GC not initialized 2026-02-24T22:05:25.634 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.627+0000 7fd96efe5640 20 reqs_thread_entry: start 2026-02-24T22:05:25.637 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.627+0000 7fd9c1a0c640 0 -- 10.20.193.46:0/1187150020 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x557a44185080 msgr2=0x557a441a5460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:05:25.845 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.835+0000 7fd9c120b640 0 -- 10.20.193.46:0/1187150020 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x557a440b05c0 msgr2=0x557a440b0a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:05:25.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.863+0000 7fd9c3737a80 20 init_complete bucket index max shards: 1 2026-02-24T22:05:25.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.863+0000 7fd9c3737a80 20 Filter name: none 2026-02-24T22:05:25.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.863+0000 7fd96d7e2640 20 reqs_thread_entry: start 2026-02-24T22:05:25.891 INFO:teuthology.orchestra.run.trial046.stdout:[] 2026-02-24T22:05:25.891 DEBUG:tasks.util.rgw: json result: [] 2026-02-24T22:05:25.891 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--placement-inline-data', 'false'] 2026-02-24T22:05:25.891 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--placement-inline-data', 'false'] 2026-02-24T22:05:25.892 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement modify --rgw-zone default --placement-id default-placement --placement-inline-data false 2026-02-24T22:05:25.957 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:05:25.957 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:05:25.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.959+0000 7f2fa06cea80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:05:25.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.959+0000 7f2fa06cea80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:05:25.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:25.959+0000 7f2f4d7e2640 20 reqs_thread_entry: start 2026-02-24T22:05:25.976 INFO:teuthology.orchestra.run.trial046.stdout:{"id":"301937db-3a1f-489c-b387-e7c21a76f9b9","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","dedup_pool":"default.rgw.dedup","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","bucket_logging_pool":"default.rgw.log:logging","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":false}}],"realm_id":"","restore_pool":"default.rgw.log:restore"} 2026-02-24T22:05:25.977 DEBUG:tasks.util.rgw: json result: {'id': '301937db-3a1f-489c-b387-e7c21a76f9b9', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'dedup_pool': 'default.rgw.dedup', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'bucket_logging_pool': 'default.rgw.log:logging', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': '', 'restore_pool': 'default.rgw.log:restore'} 2026-02-24T22:05:25.977 INFO:tasks.rgw:Preparing directory for rgw datacache at /tmp/rgw_datacache 2026-02-24T22:05:25.977 DEBUG:teuthology.orchestra.run.trial046:> mkdir -p /tmp/rgw_datacache 2026-02-24T22:05:26.021 DEBUG:teuthology.orchestra.run.trial046:> sudo chmod a+rwx /tmp/rgw_datacache 2026-02-24T22:05:26.073 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2026-02-24T22:05:26.074 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2026-02-24T22:05:26.074 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2026-02-24T22:05:26.138 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:05:26.138 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:05:26.151 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.143+0000 7f9ff79c3a80 4 Realm: () 2026-02-24T22:05:26.151 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.143+0000 7f9ff79c3a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:05:26.151 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.143+0000 7f9ff79c3a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:05:26.152 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.143+0000 7f9ff79c3a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:05:26.152 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.143+0000 7f9ff79c3a80 20 zonegroup default 2026-02-24T22:05:26.152 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.143+0000 7f9ff79c3a80 20 rados->read ofs=0 len=0 2026-02-24T22:05:26.153 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.143+0000 7f9ff79c3a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:05:26.153 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.143+0000 7f9ff79c3a80 20 rados->read ofs=0 len=0 2026-02-24T22:05:26.154 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.147+0000 7f9ff79c3a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:05:26.154 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.147+0000 7f9ff79c3a80 20 started sync module instance, tier type = 2026-02-24T22:05:26.154 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.147+0000 7f9ff79c3a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:05:26.155 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.147+0000 7f9ff79c3a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:05:26.155 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.147+0000 7f9ff79c3a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:05:26.155 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.147+0000 7f9ff79c3a80 5 note: GC not initialized 2026-02-24T22:05:26.155 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.147+0000 7f9fa2fe5640 20 reqs_thread_entry: start 2026-02-24T22:05:26.159 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.151+0000 7f9ff5497640 0 -- 10.20.193.46:0/3106241608 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55ed6b32b2d0 msgr2=0x55ed6b34b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:05:26.369 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.359+0000 7f9ff5c98640 0 -- 10.20.193.46:0/3106241608 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x55ed6b370cc0 msgr2=0x55ed6b3910a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:05:26.593 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.583+0000 7f9ff79c3a80 20 init_complete bucket index max shards: 1 2026-02-24T22:05:26.593 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.583+0000 7f9ff79c3a80 20 Filter name: none 2026-02-24T22:05:26.594 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.583+0000 7f9fa17e2640 20 reqs_thread_entry: start 2026-02-24T22:05:26.611 INFO:teuthology.orchestra.run.trial046.stdout:[] 2026-02-24T22:05:26.612 DEBUG:tasks.util.rgw: json result: [] 2026-02-24T22:05:26.612 INFO:tasks.rgw:Configuring storage class = FROZEN 2026-02-24T22:05:26.612 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2026-02-24T22:05:26.612 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2026-02-24T22:05:26.613 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN 2026-02-24T22:05:26.673 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:05:26.673 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:05:26.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.675+0000 7f1c3452ea80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:05:26.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.675+0000 7f1c3452ea80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:05:26.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.675+0000 7f1bc3fff640 20 reqs_thread_entry: start 2026-02-24T22:05:26.692 INFO:teuthology.orchestra.run.trial046.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["FROZEN","STANDARD"]}}] 2026-02-24T22:05:26.692 DEBUG:tasks.util.rgw: json result: [{'key': 'default-placement', 'val': {'name': 'default-placement', 'tags': [], 'storage_classes': ['FROZEN', 'STANDARD']}}] 2026-02-24T22:05:26.693 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2026-02-24T22:05:26.693 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2026-02-24T22:05:26.693 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN --data-pool default.rgw.buckets.data.frozen 2026-02-24T22:05:26.753 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:05:26.753 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:05:26.763 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.755+0000 7f4c86687a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:05:26.763 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.755+0000 7f4c86687a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:05:26.763 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.755+0000 7f4c337de640 20 reqs_thread_entry: start 2026-02-24T22:05:26.772 INFO:teuthology.orchestra.run.trial046.stdout:{"id":"301937db-3a1f-489c-b387-e7c21a76f9b9","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","dedup_pool":"default.rgw.dedup","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","bucket_logging_pool":"default.rgw.log:logging","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"FROZEN":{"data_pool":"default.rgw.buckets.data.frozen"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":false}}],"realm_id":"","restore_pool":"default.rgw.log:restore"} 2026-02-24T22:05:26.772 DEBUG:tasks.util.rgw: json result: {'id': '301937db-3a1f-489c-b387-e7c21a76f9b9', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'dedup_pool': 'default.rgw.dedup', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'bucket_logging_pool': 'default.rgw.log:logging', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'FROZEN': {'data_pool': 'default.rgw.buckets.data.frozen'}, 'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': '', 'restore_pool': 'default.rgw.log:restore'} 2026-02-24T22:05:26.772 INFO:tasks.rgw:Configuring storage class = LUKEWARM 2026-02-24T22:05:26.772 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2026-02-24T22:05:26.773 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2026-02-24T22:05:26.773 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM 2026-02-24T22:05:26.832 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:05:26.832 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:05:26.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.835+0000 7f212b5f7a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:05:26.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.835+0000 7f212b5f7a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:05:26.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:26.835+0000 7f20d7fe7640 20 reqs_thread_entry: start 2026-02-24T22:05:27.870 INFO:teuthology.orchestra.run.trial046.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["FROZEN","LUKEWARM","STANDARD"]}}] 2026-02-24T22:05:27.870 DEBUG:tasks.util.rgw: json result: [{'key': 'default-placement', 'val': {'name': 'default-placement', 'tags': [], 'storage_classes': ['FROZEN', 'LUKEWARM', 'STANDARD']}}] 2026-02-24T22:05:27.870 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2026-02-24T22:05:27.871 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2026-02-24T22:05:27.871 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM --data-pool default.rgw.buckets.data.lukewarm 2026-02-24T22:05:27.933 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:05:27.934 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:05:27.945 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:27.935+0000 7f703deb4a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:05:27.945 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:27.935+0000 7f703deb4a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:05:27.945 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:05:27.935+0000 7f6fea7e4640 20 reqs_thread_entry: start 2026-02-24T22:05:27.954 INFO:teuthology.orchestra.run.trial046.stdout:{"id":"301937db-3a1f-489c-b387-e7c21a76f9b9","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","dedup_pool":"default.rgw.dedup","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","bucket_logging_pool":"default.rgw.log:logging","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"FROZEN":{"data_pool":"default.rgw.buckets.data.frozen"},"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":false}}],"realm_id":"","restore_pool":"default.rgw.log:restore"} 2026-02-24T22:05:27.954 DEBUG:tasks.util.rgw: json result: {'id': '301937db-3a1f-489c-b387-e7c21a76f9b9', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'dedup_pool': 'default.rgw.dedup', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'bucket_logging_pool': 'default.rgw.log:logging', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'FROZEN': {'data_pool': 'default.rgw.buckets.data.frozen'}, 'LUKEWARM': {'data_pool': 'default.rgw.buckets.data.lukewarm'}, 'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': '', 'restore_pool': 'default.rgw.log:restore'} 2026-02-24T22:05:27.955 INFO:tasks.rgw:Starting rgw... 2026-02-24T22:05:27.955 INFO:tasks.rgw:rgw client.0 config is {'ssl certificate': 'rgw.client.0'} 2026-02-24T22:05:27.955 INFO:tasks.rgw:Using beast as radosgw frontend 2026-02-24T22:05:27.955 DEBUG:teuthology.orchestra.run.trial046:> sudo echo -n https://trial046.front.sepia.ceph.com:443 | sudo tee /home/ubuntu/cephtest/url_file 2026-02-24T22:05:28.011 INFO:teuthology.orchestra.run.trial046.stdout:https://trial046.front.sepia.ceph.com:443 2026-02-24T22:05:28.011 DEBUG:teuthology.orchestra.run.trial046:> sudo chown ceph /home/ubuntu/cephtest/url_file 2026-02-24T22:05:28.061 DEBUG:teuthology.orchestra.run.trial046:> sudo keytool -import -alias trial046.front.sepia.ceph.com -keystore $(readlink -e $(dirname $(readlink -e $(which keytool)))/../lib/security/cacerts) -file /home/ubuntu/cephtest/ca/rgw.client.0.crt -storepass changeit 2026-02-24T22:05:28.176 INFO:teuthology.orchestra.run.trial046.stderr:Warning: use -cacerts option to access cacerts keystore 2026-02-24T22:05:28.258 INFO:teuthology.orchestra.run.trial046.stderr:Certificate was added to keystore 2026-02-24T22:05:28.287 INFO:tasks.rgw.client.0:Restarting daemon 2026-02-24T22:05:28.287 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast ssl_certificate=/home/ubuntu/cephtest/ca/rgw.client.0.crt ssl_port=443' -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.ceph.client.0.stdout 2>&1 2026-02-24T22:05:28.289 INFO:tasks.rgw.client.0:Started 2026-02-24T22:05:28.289 INFO:tasks.rgw:Polling client.0 until it starts accepting connections on https://trial046.front.sepia.ceph.com:443/ 2026-02-24T22:05:28.289 DEBUG:teuthology.orchestra.run.trial046:> curl https://trial046.front.sepia.ceph.com:443/ 2026-02-24T22:05:28.298 DEBUG:teuthology.orchestra.run:got remote process result: 7 2026-02-24T22:05:28.298 INFO:teuthology.orchestra.run.trial046.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2026-02-24T22:05:28.298 INFO:teuthology.orchestra.run.trial046.stderr: Dload Upload Total Spent Left Speed 2026-02-24T22:05:28.299 INFO:teuthology.orchestra.run.trial046.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2026-02-24T22:05:28.299 INFO:teuthology.orchestra.run.trial046.stderr:curl: (7) Failed to connect to trial046.front.sepia.ceph.com port 443 after 0 ms: Connection refused 2026-02-24T22:05:29.299 DEBUG:teuthology.orchestra.run.trial046:> curl https://trial046.front.sepia.ceph.com:443/ 2026-02-24T22:05:29.319 INFO:teuthology.orchestra.run.trial046.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2026-02-24T22:05:29.319 INFO:teuthology.orchestra.run.trial046.stderr: Dload Upload Total Spent Left Speed 2026-02-24T22:05:29.331 DEBUG:teuthology.orchestra.run:got remote process result: 7 2026-02-24T22:05:29.332 INFO:teuthology.orchestra.run.trial046.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2026-02-24T22:05:29.332 INFO:teuthology.orchestra.run.trial046.stderr:curl: (7) Failed to connect to trial046.front.sepia.ceph.com port 443 after 11 ms: Connection refused 2026-02-24T22:05:29.976 INFO:tasks.rgw.client.0.trial046.stdout:2026-02-24T22:05:29.967+0000 7f25b499bb40 -1 ssl_private_key was not found: rgw/cert//default.key 2026-02-24T22:05:31.333 DEBUG:teuthology.orchestra.run.trial046:> curl https://trial046.front.sepia.ceph.com:443/ 2026-02-24T22:05:31.342 INFO:teuthology.orchestra.run.trial046.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2026-02-24T22:05:31.342 INFO:teuthology.orchestra.run.trial046.stderr: Dload Upload Total Spent Left Speed 2026-02-24T22:05:31.402 INFO:teuthology.orchestra.run.trial046.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 187 0 187 0 0 3112 0 --:--:-- --:--:-- --:--:-- 3116 2026-02-24T22:05:31.404 INFO:teuthology.orchestra.run.trial046.stdout:anonymous 2026-02-24T22:05:31.404 INFO:teuthology.run_tasks:Running task tox... 2026-02-24T22:05:31.420 INFO:tasks.tox:Deploying tox from pip... 2026-02-24T22:05:31.420 DEBUG:teuthology.orchestra.run.trial046:> python3 -m venv /home/ubuntu/cephtest/tox-venv 2026-02-24T22:05:32.757 DEBUG:teuthology.orchestra.run.trial046:> source /home/ubuntu/cephtest/tox-venv/bin/activate && pip install tox 2026-02-24T22:05:33.039 INFO:teuthology.orchestra.run.trial046.stdout:Collecting tox 2026-02-24T22:05:33.069 INFO:teuthology.orchestra.run.trial046.stdout: Downloading tox-4.46.0-py3-none-any.whl (200 kB) 2026-02-24T22:05:33.085 INFO:teuthology.orchestra.run.trial046.stdout: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 200.3/200.3 KB 12.6 MB/s eta 0:00:00 2026-02-24T22:05:33.123 INFO:teuthology.orchestra.run.trial046.stdout:Collecting pluggy>=1.6 2026-02-24T22:05:33.128 INFO:teuthology.orchestra.run.trial046.stdout: Downloading pluggy-1.6.0-py3-none-any.whl (20 kB) 2026-02-24T22:05:33.183 INFO:teuthology.orchestra.run.trial046.stdout:Collecting virtualenv>=20.38 2026-02-24T22:05:33.187 INFO:teuthology.orchestra.run.trial046.stdout: Downloading virtualenv-20.39.0-py3-none-any.whl (5.8 MB) 2026-02-24T22:05:33.306 INFO:teuthology.orchestra.run.trial046.stdout: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.8/5.8 MB 49.4 MB/s eta 0:00:00 2026-02-24T22:05:33.336 INFO:teuthology.orchestra.run.trial046.stdout:Collecting platformdirs>=4.9.2 2026-02-24T22:05:33.340 INFO:teuthology.orchestra.run.trial046.stdout: Downloading platformdirs-4.9.2-py3-none-any.whl (21 kB) 2026-02-24T22:05:33.361 INFO:teuthology.orchestra.run.trial046.stdout:Collecting tomli>=2.4 2026-02-24T22:05:33.365 INFO:teuthology.orchestra.run.trial046.stdout: Downloading tomli-2.4.0-py3-none-any.whl (14 kB) 2026-02-24T22:05:33.383 INFO:teuthology.orchestra.run.trial046.stdout:Collecting cachetools>=7.0.1 2026-02-24T22:05:33.387 INFO:teuthology.orchestra.run.trial046.stdout: Downloading cachetools-7.0.1-py3-none-any.whl (13 kB) 2026-02-24T22:05:33.406 INFO:teuthology.orchestra.run.trial046.stdout:Collecting filelock>=3.24.3 2026-02-24T22:05:33.410 INFO:teuthology.orchestra.run.trial046.stdout: Downloading filelock-3.24.3-py3-none-any.whl (24 kB) 2026-02-24T22:05:33.423 INFO:teuthology.orchestra.run.trial046.stdout:Collecting pyproject-api>=1.10 2026-02-24T22:05:33.427 INFO:teuthology.orchestra.run.trial046.stdout: Downloading pyproject_api-1.10.0-py3-none-any.whl (13 kB) 2026-02-24T22:05:33.445 INFO:teuthology.orchestra.run.trial046.stdout:Collecting packaging>=26 2026-02-24T22:05:33.450 INFO:teuthology.orchestra.run.trial046.stdout: Downloading packaging-26.0-py3-none-any.whl (74 kB) 2026-02-24T22:05:33.451 INFO:teuthology.orchestra.run.trial046.stdout: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 74.4/74.4 KB 77.2 MB/s eta 0:00:00 2026-02-24T22:05:33.467 INFO:teuthology.orchestra.run.trial046.stdout:Collecting typing-extensions>=4.15 2026-02-24T22:05:33.473 INFO:teuthology.orchestra.run.trial046.stdout: Downloading typing_extensions-4.15.0-py3-none-any.whl (44 kB) 2026-02-24T22:05:33.475 INFO:teuthology.orchestra.run.trial046.stdout: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 44.6/44.6 KB 45.2 MB/s eta 0:00:00 2026-02-24T22:05:33.496 INFO:teuthology.orchestra.run.trial046.stdout:Collecting colorama>=0.4.6 2026-02-24T22:05:33.500 INFO:teuthology.orchestra.run.trial046.stdout: Downloading colorama-0.4.6-py2.py3-none-any.whl (25 kB) 2026-02-24T22:05:33.531 INFO:teuthology.orchestra.run.trial046.stdout:Collecting distlib<1,>=0.3.7 2026-02-24T22:05:33.534 INFO:teuthology.orchestra.run.trial046.stdout: Downloading distlib-0.4.0-py2.py3-none-any.whl (469 kB) 2026-02-24T22:05:33.539 INFO:teuthology.orchestra.run.trial046.stdout: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 469.0/469.0 KB 108.8 MB/s eta 0:00:00 2026-02-24T22:05:33.561 INFO:teuthology.orchestra.run.trial046.stdout:Installing collected packages: distlib, typing-extensions, tomli, pluggy, platformdirs, packaging, filelock, colorama, cachetools, virtualenv, pyproject-api, tox 2026-02-24T22:05:33.778 INFO:teuthology.orchestra.run.trial046.stdout:Successfully installed cachetools-7.0.1 colorama-0.4.6 distlib-0.4.0 filelock-3.24.3 packaging-26.0 platformdirs-4.9.2 pluggy-1.6.0 pyproject-api-1.10.0 tomli-2.4.0 tox-4.46.0 typing-extensions-4.15.0 virtualenv-20.39.0 2026-02-24T22:05:33.813 INFO:teuthology.run_tasks:Running task workunit... 2026-02-24T22:05:33.839 INFO:tasks.workunit:Pulling workunits from ref f7e05f4fdea5af173284f247d6b0935b337d4ccf 2026-02-24T22:05:33.839 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-24T22:05:33.839 DEBUG:teuthology.orchestra.run.trial046:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-24T22:05:33.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T22:05:33.844 INFO:teuthology.orchestra.run.trial046.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-24T22:05:33.845 DEBUG:teuthology.orchestra.run.trial046:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-24T22:05:33.889 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-24T22:05:33.889 DEBUG:teuthology.orchestra.run.trial046:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-24T22:05:33.932 INFO:tasks.workunit:timeout=3h 2026-02-24T22:05:33.933 INFO:tasks.workunit:cleanup=True 2026-02-24T22:05:33.933 DEBUG:teuthology.orchestra.run.trial046:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/samarahu/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout f7e05f4fdea5af173284f247d6b0935b337d4ccf 2026-02-24T22:05:33.983 INFO:tasks.workunit.client.0.trial046.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-24T22:05:56.120 INFO:tasks.workunit.client.0.trial046.stderr:Note: switching to 'f7e05f4fdea5af173284f247d6b0935b337d4ccf'. 2026-02-24T22:05:56.121 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:05:56.121 INFO:tasks.workunit.client.0.trial046.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-24T22:05:56.121 INFO:tasks.workunit.client.0.trial046.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-24T22:05:56.121 INFO:tasks.workunit.client.0.trial046.stderr:state without impacting any branches by switching back to a branch. 2026-02-24T22:05:56.121 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:05:56.121 INFO:tasks.workunit.client.0.trial046.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-24T22:05:56.121 INFO:tasks.workunit.client.0.trial046.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-24T22:05:56.121 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:05:56.121 INFO:tasks.workunit.client.0.trial046.stderr: git switch -c 2026-02-24T22:05:56.121 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:05:56.122 INFO:tasks.workunit.client.0.trial046.stderr:Or undo this operation with: 2026-02-24T22:05:56.122 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:05:56.122 INFO:tasks.workunit.client.0.trial046.stderr: git switch - 2026-02-24T22:05:56.122 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:05:56.122 INFO:tasks.workunit.client.0.trial046.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-24T22:05:56.122 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:05:56.122 INFO:tasks.workunit.client.0.trial046.stderr:HEAD is now at f7e05f4fdea rgw/d4n: Implement bucket check_empty and remove methods 2026-02-24T22:05:56.127 DEBUG:teuthology.orchestra.run.trial046:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-24T22:05:56.173 INFO:tasks.workunit.client.0.trial046.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2026-02-24T22:05:56.174 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-24T22:05:56.175 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2026-02-24T22:05:56.269 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2026-02-24T22:05:56.300 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2026-02-24T22:05:56.326 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-24T22:05:56.328 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-24T22:05:56.328 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2026-02-24T22:05:56.352 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-24T22:05:56.356 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:05:56.356 DEBUG:teuthology.orchestra.run.trial046:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-24T22:05:56.401 INFO:tasks.workunit:Running workunits matching rgw/run-datacache.sh on client.0... 2026-02-24T22:05:56.404 INFO:tasks.workunit:Running workunit rgw/run-datacache.sh... 2026-02-24T22:05:56.404 DEBUG:teuthology.orchestra.run.trial046:workunit test rgw/run-datacache.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7e05f4fdea5af173284f247d6b0935b337d4ccf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RGW_DATACACHE_PATH=/tmp/rgw_datacache adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-datacache.sh 2026-02-24T22:05:56.452 INFO:tasks.workunit.client.0.trial046.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-datacache.sh 2026-02-24T22:05:56.452 INFO:tasks.workunit.client.0.trial046.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2026-02-24T22:05:56.453 INFO:tasks.workunit.client.0.trial046.stderr:+ python3 -m venv /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2026-02-24T22:05:57.782 INFO:tasks.workunit.client.0.trial046.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/activate 2026-02-24T22:05:57.782 INFO:tasks.workunit.client.0.trial046.stderr:++ deactivate nondestructive 2026-02-24T22:05:57.782 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n '' ']' 2026-02-24T22:05:57.782 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n '' ']' 2026-02-24T22:05:57.782 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2026-02-24T22:05:57.782 INFO:tasks.workunit.client.0.trial046.stderr:++ hash -r 2026-02-24T22:05:57.782 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n '' ']' 2026-02-24T22:05:57.782 INFO:tasks.workunit.client.0.trial046.stderr:++ unset VIRTUAL_ENV 2026-02-24T22:05:57.783 INFO:tasks.workunit.client.0.trial046.stderr:++ unset VIRTUAL_ENV_PROMPT 2026-02-24T22:05:57.783 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' '!' nondestructive = nondestructive ']' 2026-02-24T22:05:57.783 INFO:tasks.workunit.client.0.trial046.stderr:++ VIRTUAL_ENV=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2026-02-24T22:05:57.783 INFO:tasks.workunit.client.0.trial046.stderr:++ export VIRTUAL_ENV 2026-02-24T22:05:57.783 INFO:tasks.workunit.client.0.trial046.stderr:++ _OLD_VIRTUAL_PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T22:05:57.783 INFO:tasks.workunit.client.0.trial046.stderr:++ PATH=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T22:05:57.783 INFO:tasks.workunit.client.0.trial046.stderr:++ export PATH 2026-02-24T22:05:57.783 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n '' ']' 2026-02-24T22:05:57.783 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -z '' ']' 2026-02-24T22:05:57.783 INFO:tasks.workunit.client.0.trial046.stderr:++ _OLD_VIRTUAL_PS1= 2026-02-24T22:05:57.783 INFO:tasks.workunit.client.0.trial046.stderr:++ PS1='(rgw) ' 2026-02-24T22:05:57.783 INFO:tasks.workunit.client.0.trial046.stderr:++ export PS1 2026-02-24T22:05:57.783 INFO:tasks.workunit.client.0.trial046.stderr:++ VIRTUAL_ENV_PROMPT='(rgw) ' 2026-02-24T22:05:57.784 INFO:tasks.workunit.client.0.trial046.stderr:++ export VIRTUAL_ENV_PROMPT 2026-02-24T22:05:57.784 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2026-02-24T22:05:57.784 INFO:tasks.workunit.client.0.trial046.stderr:++ hash -r 2026-02-24T22:05:57.784 INFO:tasks.workunit.client.0.trial046.stderr:+ pip install pip --upgrade 2026-02-24T22:05:57.963 INFO:tasks.workunit.client.0.trial046.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2026-02-24T22:05:58.026 INFO:tasks.workunit.client.0.trial046.stdout:Collecting pip 2026-02-24T22:05:58.053 INFO:tasks.workunit.client.0.trial046.stdout: Downloading pip-26.0.1-py3-none-any.whl (1.8 MB) 2026-02-24T22:05:58.116 INFO:tasks.workunit.client.0.trial046.stdout: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 29.4 MB/s eta 0:00:00 2026-02-24T22:05:58.130 INFO:tasks.workunit.client.0.trial046.stdout:Installing collected packages: pip 2026-02-24T22:05:58.130 INFO:tasks.workunit.client.0.trial046.stdout: Attempting uninstall: pip 2026-02-24T22:05:58.130 INFO:tasks.workunit.client.0.trial046.stdout: Found existing installation: pip 22.0.2 2026-02-24T22:05:58.306 INFO:tasks.workunit.client.0.trial046.stdout: Uninstalling pip-22.0.2: 2026-02-24T22:05:58.309 INFO:tasks.workunit.client.0.trial046.stdout: Successfully uninstalled pip-22.0.2 2026-02-24T22:05:58.651 INFO:tasks.workunit.client.0.trial046.stdout:Successfully installed pip-26.0.1 2026-02-24T22:05:58.679 INFO:tasks.workunit.client.0.trial046.stderr:+ pip install configobj 2026-02-24T22:05:58.916 INFO:tasks.workunit.client.0.trial046.stdout:Collecting configobj 2026-02-24T22:05:58.953 INFO:tasks.workunit.client.0.trial046.stdout: Downloading configobj-5.0.9-py2.py3-none-any.whl.metadata (3.2 kB) 2026-02-24T22:05:58.960 INFO:tasks.workunit.client.0.trial046.stdout:Downloading configobj-5.0.9-py2.py3-none-any.whl (35 kB) 2026-02-24T22:05:58.974 INFO:tasks.workunit.client.0.trial046.stdout:Installing collected packages: configobj 2026-02-24T22:05:58.988 INFO:tasks.workunit.client.0.trial046.stdout:Successfully installed configobj-5.0.9 2026-02-24T22:05:59.054 INFO:tasks.workunit.client.0.trial046.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/python3 /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_datacache.py 2026-02-24T22:05:59.070 INFO:tasks.workunit.client.0.trial046.stderr:DEBUG:root:datacache dir from config is: /tmp/rgw_datacache 2026-02-24T22:05:59.070 INFO:tasks.workunit.client.0.trial046.stderr:DEBUG:root:exec_cmd(pwd) 2026-02-24T22:05:59.071 INFO:tasks.workunit.client.0.trial046.stderr:INFO:root:command succeeded 2026-02-24T22:05:59.071 INFO:tasks.workunit.client.0.trial046.stderr:INFO:root:b'/home/ubuntu/cephtest/mnt.0/client.0/tmp\n' 2026-02-24T22:05:59.071 INFO:tasks.workunit.client.0.trial046.stderr:DEBUG:root:pwd is: /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T22:05:59.071 INFO:tasks.workunit.client.0.trial046.stderr:DEBUG:root:exec_cmd(sudo ss -nltp | egrep "rados|valgr|memcheck-") 2026-02-24T22:05:59.095 INFO:tasks.workunit.client.0.trial046.stderr:INFO:root:command succeeded 2026-02-24T22:05:59.095 INFO:tasks.workunit.client.0.trial046.stderr:INFO:root:b'LISTEN 0 4096 0.0.0.0:443 0.0.0.0:* users:(("radosgw",pid=23896,fd=79)) \nLISTEN 0 4096 [::]:443 [::]:* users:(("radosgw",pid=23896,fd=81)) \n' 2026-02-24T22:05:59.095 INFO:tasks.workunit.client.0.trial046.stderr:INFO:root:radosgw port: 443 2026-02-24T22:05:59.095 INFO:tasks.workunit.client.0.trial046.stderr:INFO:root:radosgw endpoint is: 127.0.0.1 2026-02-24T22:05:59.095 INFO:tasks.workunit.client.0.trial046.stderr:DEBUG:root:exec_cmd(dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/client.0/tmp/7M.dat bs=1M count=7) 2026-02-24T22:05:59.114 INFO:tasks.workunit.client.0.trial046.stderr:INFO:root:command succeeded 2026-02-24T22:05:59.114 INFO:tasks.workunit.client.0.trial046.stderr:INFO:root:b'' 2026-02-24T22:05:59.114 INFO:tasks.workunit.client.0.trial046.stderr:DEBUG:root:exec_cmd(radosgw-admin user create --uid rgw_datacache_user --display-name DatacacheUser --access-key NX5QOQKC6BH2IDN8HC7A --secret LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn) 2026-02-24T22:05:59.408 INFO:tasks.workunit.client.0.trial046.stderr:INFO:root:command succeeded 2026-02-24T22:05:59.408 INFO:tasks.workunit.client.0.trial046.stderr:INFO:root:b'{\n "user_id": "rgw_datacache_user",\n "display_name": "DatacacheUser",\n "email": "",\n "suspended": 0,\n "max_buckets": 1000,\n "subusers": [],\n "keys": [\n {\n "user": "rgw_datacache_user",\n "access_key": "NX5QOQKC6BH2IDN8HC7A",\n "secret_key": "LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn",\n "active": true,\n "create_date": "2026-02-24T22:05:59.393405Z"\n }\n ],\n "swift_keys": [],\n "caps": [],\n "op_mask": "read, write, delete",\n "default_placement": "",\n "default_storage_class": "",\n "placement_tags": [],\n "bucket_quota": {\n "enabled": false,\n "check_on_raw": false,\n "max_size": -1,\n "max_size_kb": 0,\n "max_objects": -1\n },\n "user_quota": {\n "enabled": false,\n "check_on_raw": false,\n "max_size": -1,\n "max_size_kb": 0,\n "max_objects": -1\n },\n "temp_url_keys": [],\n "type": "rgw",\n "mfa_ids": [],\n "account_id": "",\n "path": "/",\n "create_date": "2026-02-24T22:05:59.393390Z",\n "tags": [],\n "group_ids": []\n}\n\n' 2026-02-24T22:05:59.408 INFO:tasks.workunit.client.0.trial046.stderr:INFO:root:Creating s3cmd config... 2026-02-24T22:05:59.408 INFO:tasks.workunit.client.0.trial046.stderr:INFO:root:proto for s3cmd config is https 2026-02-24T22:05:59.408 INFO:tasks.workunit.client.0.trial046.stderr:INFO:root:s3cmd config written 2026-02-24T22:05:59.409 INFO:tasks.workunit.client.0.trial046.stderr:DEBUG:root:exec_cmd(s3cmd --access_key=NX5QOQKC6BH2IDN8HC7A --secret_key=LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn --config=/home/ubuntu/cephtest/mnt.0/client.0/tmp/s3cfg --no-check-hostname --host=127.0.0.1 mb s3://datacachebucket) 2026-02-24T22:05:59.524 INFO:tasks.workunit.client.0.trial046.stderr:INFO:root:command succeeded 2026-02-24T22:05:59.524 INFO:tasks.workunit.client.0.trial046.stderr:INFO:root:b"Bucket 's3://datacachebucket/' created\n" 2026-02-24T22:05:59.524 INFO:tasks.workunit.client.0.trial046.stderr:DEBUG:root:exec_cmd(s3cmd --access_key=NX5QOQKC6BH2IDN8HC7A --secret_key=LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn --config=/home/ubuntu/cephtest/mnt.0/client.0/tmp/s3cfg --no-check-hostname --host=127.0.0.1 put /home/ubuntu/cephtest/mnt.0/client.0/tmp/7M.dat s3://datacachebucket) 2026-02-24T22:05:59.693 INFO:tasks.workunit.client.0.trial046.stderr:INFO:root:command succeeded 2026-02-24T22:05:59.694 INFO:tasks.workunit.client.0.trial046.stderr:INFO:root:b"upload: '/home/ubuntu/cephtest/mnt.0/client.0/tmp/7M.dat' -> 's3://datacachebucket/7M.dat' (7340032 bytes in 0.1 seconds, 138.08 MB/s) [1 of 1]\n" 2026-02-24T22:05:59.694 INFO:tasks.workunit.client.0.trial046.stderr:DEBUG:root:exec_cmd(s3cmd --access_key=NX5QOQKC6BH2IDN8HC7A --secret_key=LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn --config=/home/ubuntu/cephtest/mnt.0/client.0/tmp/s3cfg --no-check-hostname --host=127.0.0.1 get s3://datacachebucket/7M.dat /home/ubuntu/cephtest/mnt.0/client.0/tmp/7M-get.dat --force) 2026-02-24T22:05:59.825 INFO:tasks.workunit.client.0.trial046.stderr:INFO:root:command succeeded 2026-02-24T22:05:59.825 INFO:tasks.workunit.client.0.trial046.stderr:INFO:root:b"download: 's3://datacachebucket/7M.dat' -> '/home/ubuntu/cephtest/mnt.0/client.0/tmp/7M-get.dat' (7340032 bytes in 0.0 seconds, 319.73 MB/s)\n" 2026-02-24T22:05:59.825 INFO:tasks.workunit.client.0.trial046.stderr:DEBUG:root:exec_cmd(radosgw-admin object stat --bucket=datacachebucket --object=7M.dat) 2026-02-24T22:05:59.906 INFO:tasks.workunit.client.0.trial046.stderr:INFO:root:command succeeded 2026-02-24T22:05:59.906 INFO:tasks.workunit.client.0.trial046.stderr:INFO:root:b'{\n "name": "7M.dat",\n "size": 7340032,\n "policy": {\n "acl": {\n "acl_user_map": [\n {\n "user": "rgw_datacache_user",\n "acl": 15\n }\n ],\n "acl_group_map": [],\n "grant_map": [\n {\n "id": "rgw_datacache_user",\n "grant": {\n "type": {\n "type": 0\n },\n "id": "rgw_datacache_user",\n "name": "DatacacheUser",\n "permission": {\n "flags": 15\n }\n }\n }\n ]\n },\n "owner": {\n "id": "rgw_datacache_user",\n "display_name": "DatacacheUser"\n }\n },\n "etag": "acb666625c8ac7a731f8ad29023fa1bc",\n "tag": "301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8467522195107442757",\n "manifest": {\n "objs": [],\n "obj_size": 7340032,\n "explicit_objs": false,\n "head_size": 0,\n "max_head_size": 0,\n "prefix": ".A72zbtDP3mGtbTNWtWyWJBB2qX4nEZj_",\n "rules": [\n {\n "key": 0,\n "val": {\n "start_part_num": 0,\n "start_ofs": 0,\n "part_size": 0,\n "stripe_max_size": 6291456,\n "override_prefix": ""\n }\n }\n ],\n "tail_instance": "",\n "tail_placement": {\n "bucket": {\n "name": "datacachebucket",\n "marker": "301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1",\n "bucket_id": "301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1",\n "tenant": "",\n "explicit_placement": {\n "data_pool": "",\n "data_extra_pool": "",\n "index_pool": ""\n }\n },\n "placement_rule": "default-placement"\n },\n "tier_type": "",\n "begin_iter": {\n "part_ofs": 0,\n "stripe_ofs": 0,\n "ofs": 0,\n "stripe_size": 6291456,\n "cur_part_id": 0,\n "cur_stripe": 0,\n "cur_override_prefix": "",\n "location": {\n "placement_rule": "default-placement",\n "obj": {\n "bucket": {\n "name": "datacachebucket",\n "marker": "301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1",\n "bucket_id": "301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1",\n "tenant": "",\n "explicit_placement": {\n "data_pool": "",\n "data_extra_pool": "",\n "index_pool": ""\n }\n },\n "key": {\n "name": ".A72zbtDP3mGtbTNWtWyWJBB2qX4nEZj_0",\n "instance": "",\n "ns": "shadow"\n }\n },\n "raw_obj": {\n "pool": "",\n "oid": "",\n "loc": ""\n },\n "is_raw": false\n }\n },\n "end_iter": {\n "part_ofs": 0,\n "stripe_ofs": 6291456,\n "ofs": 7340032,\n "stripe_size": 1048576,\n "cur_part_id": 0,\n "cur_stripe": 1,\n "cur_override_prefix": "",\n "location": {\n "placement_rule": "default-placement",\n "obj": {\n "bucket": {\n "name": "datacachebucket",\n "marker": "301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1",\n "bucket_id": "301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1",\n "tenant": "",\n "explicit_placement": {\n "data_pool": "",\n "data_extra_pool": "",\n "index_pool": ""\n }\n },\n "key": {\n "name": ".A72zbtDP3mGtbTNWtWyWJBB2qX4nEZj_1",\n "instance": "",\n "ns": "shadow"\n }\n },\n "raw_obj": {\n "pool": "",\n "oid": "",\n "loc": ""\n },\n "is_raw": false\n }\n }\n },\n "pg_ver": 0,\n "source_zone": 0,\n "torrent": "",\n "mtime": "2026-02-24T22:05:59.680067Z",\n "attrs": {\n "user.rgw.content_type": "application/octet-stream",\n "user.rgw.storage_class": "STANDARD",\n "user.rgw.tail_tag": "301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8467522195107442757",\n "user.rgw.x-amz-meta-s3cmd-attrs": "atime:1771970759/ctime:1771970759/gid:1002/gname:ubuntu/md5:acb666625c8ac7a731f8ad29023fa1bc/mode:33204/mtime:1771970759/uid:1000/uname:ubuntu"\n }\n}\n' 2026-02-24T22:05:59.906 INFO:tasks.workunit.client.0.trial046.stderr:DEBUG:root:Cached object name is: .A72zbtDP3mGtbTNWtWyWJBB2qX4nEZj_ 2026-02-24T22:05:59.906 INFO:tasks.workunit.client.0.trial046.stderr:DEBUG:root:*** stripe_size = 6291456 2026-02-24T22:05:59.907 INFO:tasks.workunit.client.0.trial046.stderr:DEBUG:root:*** stripe_max_size = 6291456 2026-02-24T22:05:59.907 INFO:tasks.workunit.client.0.trial046.stderr:DEBUG:root:*** compression_type = none 2026-02-24T22:05:59.907 INFO:tasks.workunit.client.0.trial046.stderr:DEBUG:root:exec_cmd(find /tmp/rgw_datacache -type f | wc -l) 2026-02-24T22:05:59.908 INFO:tasks.workunit.client.0.trial046.stderr:INFO:root:command succeeded 2026-02-24T22:05:59.908 INFO:tasks.workunit.client.0.trial046.stderr:INFO:root:b'0\n' 2026-02-24T22:05:59.908 INFO:tasks.workunit.client.0.trial046.stderr:DEBUG:root:*** cache_dir_file_count = 0 2026-02-24T22:05:59.909 INFO:tasks.workunit.client.0.trial046.stderr:DEBUG:root:*** NOTICE: Running under 'stripe-greater-than-chunk' permutation - skipping the RGW Datacache test 2026-02-24T22:05:59.909 INFO:tasks.workunit.client.0.trial046.stderr:INFO:root:Completed Datacache tests 2026-02-24T22:05:59.913 INFO:tasks.workunit.client.0.trial046.stderr:+ deactivate 2026-02-24T22:05:59.913 INFO:tasks.workunit.client.0.trial046.stderr:+ '[' -n /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin ']' 2026-02-24T22:05:59.913 INFO:tasks.workunit.client.0.trial046.stderr:+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T22:05:59.913 INFO:tasks.workunit.client.0.trial046.stderr:+ export PATH 2026-02-24T22:05:59.913 INFO:tasks.workunit.client.0.trial046.stderr:+ unset _OLD_VIRTUAL_PATH 2026-02-24T22:05:59.913 INFO:tasks.workunit.client.0.trial046.stderr:+ '[' -n '' ']' 2026-02-24T22:05:59.913 INFO:tasks.workunit.client.0.trial046.stderr:+ '[' -n /usr/bin/bash -o -n '' ']' 2026-02-24T22:05:59.914 INFO:tasks.workunit.client.0.trial046.stderr:+ hash -r 2026-02-24T22:05:59.914 INFO:tasks.workunit.client.0.trial046.stderr:+ '[' -n '' ']' 2026-02-24T22:05:59.914 INFO:tasks.workunit.client.0.trial046.stderr:+ unset VIRTUAL_ENV 2026-02-24T22:05:59.914 INFO:tasks.workunit.client.0.trial046.stderr:+ unset VIRTUAL_ENV_PROMPT 2026-02-24T22:05:59.914 INFO:tasks.workunit.client.0.trial046.stderr:+ '[' '!' '' = nondestructive ']' 2026-02-24T22:05:59.914 INFO:tasks.workunit.client.0.trial046.stderr:+ unset -f deactivate 2026-02-24T22:05:59.914 INFO:tasks.workunit.client.0.trial046.stderr:+ echo OK. 2026-02-24T22:05:59.915 INFO:tasks.workunit.client.0.trial046.stdout:OK. 2026-02-24T22:05:59.915 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T22:05:59.916 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T22:05:59.967 INFO:tasks.workunit:Stopping ['rgw/run-datacache.sh'] on client.0... 2026-02-24T22:05:59.967 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-24T22:06:00.328 DEBUG:teuthology.parallel:result is None 2026-02-24T22:06:00.328 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-24T22:06:00.337 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-24T22:06:00.337 DEBUG:teuthology.orchestra.run.trial046:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-24T22:06:00.384 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-24T22:06:00.384 INFO:teuthology.run_tasks:Running task workunit... 2026-02-24T22:06:00.399 INFO:tasks.workunit:Pulling workunits from ref f7e05f4fdea5af173284f247d6b0935b337d4ccf 2026-02-24T22:06:00.399 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-24T22:06:00.399 DEBUG:teuthology.orchestra.run.trial046:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-24T22:06:00.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T22:06:00.428 INFO:teuthology.orchestra.run.trial046.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-24T22:06:00.428 DEBUG:teuthology.orchestra.run.trial046:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-24T22:06:00.472 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-24T22:06:00.472 DEBUG:teuthology.orchestra.run.trial046:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-24T22:06:00.516 INFO:tasks.workunit:timeout=3h 2026-02-24T22:06:00.516 INFO:tasks.workunit:cleanup=True 2026-02-24T22:06:00.516 DEBUG:teuthology.orchestra.run.trial046:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/samarahu/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout f7e05f4fdea5af173284f247d6b0935b337d4ccf 2026-02-24T22:06:00.561 INFO:tasks.workunit.client.0.trial046.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-24T22:06:22.767 INFO:tasks.workunit.client.0.trial046.stderr:Note: switching to 'f7e05f4fdea5af173284f247d6b0935b337d4ccf'. 2026-02-24T22:06:22.768 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:06:22.768 INFO:tasks.workunit.client.0.trial046.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-24T22:06:22.768 INFO:tasks.workunit.client.0.trial046.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-24T22:06:22.768 INFO:tasks.workunit.client.0.trial046.stderr:state without impacting any branches by switching back to a branch. 2026-02-24T22:06:22.768 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:06:22.768 INFO:tasks.workunit.client.0.trial046.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-24T22:06:22.768 INFO:tasks.workunit.client.0.trial046.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-24T22:06:22.768 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:06:22.768 INFO:tasks.workunit.client.0.trial046.stderr: git switch -c 2026-02-24T22:06:22.768 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:06:22.769 INFO:tasks.workunit.client.0.trial046.stderr:Or undo this operation with: 2026-02-24T22:06:22.769 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:06:22.769 INFO:tasks.workunit.client.0.trial046.stderr: git switch - 2026-02-24T22:06:22.769 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:06:22.769 INFO:tasks.workunit.client.0.trial046.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-24T22:06:22.769 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:06:22.769 INFO:tasks.workunit.client.0.trial046.stderr:HEAD is now at f7e05f4fdea rgw/d4n: Implement bucket check_empty and remove methods 2026-02-24T22:06:22.774 DEBUG:teuthology.orchestra.run.trial046:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-24T22:06:22.820 INFO:tasks.workunit.client.0.trial046.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2026-02-24T22:06:22.822 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-24T22:06:22.822 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2026-02-24T22:06:22.851 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2026-02-24T22:06:22.884 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2026-02-24T22:06:22.910 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-24T22:06:22.912 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-24T22:06:22.912 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2026-02-24T22:06:22.935 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-24T22:06:22.939 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:06:22.939 DEBUG:teuthology.orchestra.run.trial046:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-24T22:06:22.984 INFO:tasks.workunit:Running workunits matching rgw/run-admin-pagination.sh on client.0... 2026-02-24T22:06:22.987 INFO:tasks.workunit:Running workunit rgw/run-admin-pagination.sh... 2026-02-24T22:06:22.987 DEBUG:teuthology.orchestra.run.trial046:workunit test rgw/run-admin-pagination.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7e05f4fdea5af173284f247d6b0935b337d4ccf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-admin-pagination.sh 2026-02-24T22:06:23.036 INFO:tasks.workunit.client.0.trial046.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-admin-pagination.sh 2026-02-24T22:06:23.037 INFO:tasks.workunit.client.0.trial046.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2026-02-24T22:06:23.037 INFO:tasks.workunit.client.0.trial046.stderr:+ python3 -m venv /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2026-02-24T22:06:24.362 INFO:tasks.workunit.client.0.trial046.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/activate 2026-02-24T22:06:24.362 INFO:tasks.workunit.client.0.trial046.stderr:++ deactivate nondestructive 2026-02-24T22:06:24.362 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n '' ']' 2026-02-24T22:06:24.362 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n '' ']' 2026-02-24T22:06:24.362 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2026-02-24T22:06:24.362 INFO:tasks.workunit.client.0.trial046.stderr:++ hash -r 2026-02-24T22:06:24.363 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n '' ']' 2026-02-24T22:06:24.363 INFO:tasks.workunit.client.0.trial046.stderr:++ unset VIRTUAL_ENV 2026-02-24T22:06:24.363 INFO:tasks.workunit.client.0.trial046.stderr:++ unset VIRTUAL_ENV_PROMPT 2026-02-24T22:06:24.363 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' '!' nondestructive = nondestructive ']' 2026-02-24T22:06:24.363 INFO:tasks.workunit.client.0.trial046.stderr:++ VIRTUAL_ENV=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2026-02-24T22:06:24.363 INFO:tasks.workunit.client.0.trial046.stderr:++ export VIRTUAL_ENV 2026-02-24T22:06:24.363 INFO:tasks.workunit.client.0.trial046.stderr:++ _OLD_VIRTUAL_PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T22:06:24.363 INFO:tasks.workunit.client.0.trial046.stderr:++ PATH=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T22:06:24.363 INFO:tasks.workunit.client.0.trial046.stderr:++ export PATH 2026-02-24T22:06:24.363 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n '' ']' 2026-02-24T22:06:24.363 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -z '' ']' 2026-02-24T22:06:24.363 INFO:tasks.workunit.client.0.trial046.stderr:++ _OLD_VIRTUAL_PS1= 2026-02-24T22:06:24.364 INFO:tasks.workunit.client.0.trial046.stderr:++ PS1='(rgw) ' 2026-02-24T22:06:24.364 INFO:tasks.workunit.client.0.trial046.stderr:++ export PS1 2026-02-24T22:06:24.364 INFO:tasks.workunit.client.0.trial046.stderr:++ VIRTUAL_ENV_PROMPT='(rgw) ' 2026-02-24T22:06:24.364 INFO:tasks.workunit.client.0.trial046.stderr:++ export VIRTUAL_ENV_PROMPT 2026-02-24T22:06:24.364 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2026-02-24T22:06:24.364 INFO:tasks.workunit.client.0.trial046.stderr:++ hash -r 2026-02-24T22:06:24.364 INFO:tasks.workunit.client.0.trial046.stderr:+ pip install pip --upgrade 2026-02-24T22:06:24.541 INFO:tasks.workunit.client.0.trial046.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2026-02-24T22:06:24.602 INFO:tasks.workunit.client.0.trial046.stdout:Collecting pip 2026-02-24T22:06:24.606 INFO:tasks.workunit.client.0.trial046.stdout: Using cached pip-26.0.1-py3-none-any.whl (1.8 MB) 2026-02-24T22:06:24.622 INFO:tasks.workunit.client.0.trial046.stdout:Installing collected packages: pip 2026-02-24T22:06:24.622 INFO:tasks.workunit.client.0.trial046.stdout: Attempting uninstall: pip 2026-02-24T22:06:24.622 INFO:tasks.workunit.client.0.trial046.stdout: Found existing installation: pip 22.0.2 2026-02-24T22:06:24.795 INFO:tasks.workunit.client.0.trial046.stdout: Uninstalling pip-22.0.2: 2026-02-24T22:06:24.798 INFO:tasks.workunit.client.0.trial046.stdout: Successfully uninstalled pip-22.0.2 2026-02-24T22:06:25.135 INFO:tasks.workunit.client.0.trial046.stdout:Successfully installed pip-26.0.1 2026-02-24T22:06:25.163 INFO:tasks.workunit.client.0.trial046.stderr:+ pip install boto3 requests 2026-02-24T22:06:25.503 INFO:tasks.workunit.client.0.trial046.stdout:Collecting boto3 2026-02-24T22:06:25.535 INFO:tasks.workunit.client.0.trial046.stdout: Downloading boto3-1.42.56-py3-none-any.whl.metadata (6.7 kB) 2026-02-24T22:06:25.548 INFO:tasks.workunit.client.0.trial046.stdout:Collecting requests 2026-02-24T22:06:25.552 INFO:tasks.workunit.client.0.trial046.stdout: Downloading requests-2.32.5-py3-none-any.whl.metadata (4.9 kB) 2026-02-24T22:06:25.680 INFO:tasks.workunit.client.0.trial046.stdout:Collecting botocore<1.43.0,>=1.42.56 (from boto3) 2026-02-24T22:06:25.683 INFO:tasks.workunit.client.0.trial046.stdout: Downloading botocore-1.42.56-py3-none-any.whl.metadata (5.9 kB) 2026-02-24T22:06:25.692 INFO:tasks.workunit.client.0.trial046.stdout:Collecting jmespath<2.0.0,>=0.7.1 (from boto3) 2026-02-24T22:06:25.696 INFO:tasks.workunit.client.0.trial046.stdout: Downloading jmespath-1.1.0-py3-none-any.whl.metadata (7.6 kB) 2026-02-24T22:06:25.708 INFO:tasks.workunit.client.0.trial046.stdout:Collecting s3transfer<0.17.0,>=0.16.0 (from boto3) 2026-02-24T22:06:25.712 INFO:tasks.workunit.client.0.trial046.stdout: Downloading s3transfer-0.16.0-py3-none-any.whl.metadata (1.7 kB) 2026-02-24T22:06:25.732 INFO:tasks.workunit.client.0.trial046.stdout:Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.43.0,>=1.42.56->boto3) 2026-02-24T22:06:25.736 INFO:tasks.workunit.client.0.trial046.stdout: Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) 2026-02-24T22:06:25.750 INFO:tasks.workunit.client.0.trial046.stdout:Collecting urllib3!=2.2.0,<3,>=1.25.4 (from botocore<1.43.0,>=1.42.56->boto3) 2026-02-24T22:06:25.754 INFO:tasks.workunit.client.0.trial046.stdout: Downloading urllib3-2.6.3-py3-none-any.whl.metadata (6.9 kB) 2026-02-24T22:06:25.763 INFO:tasks.workunit.client.0.trial046.stdout:Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.43.0,>=1.42.56->boto3) 2026-02-24T22:06:25.772 INFO:tasks.workunit.client.0.trial046.stdout: Downloading six-1.17.0-py2.py3-none-any.whl.metadata (1.7 kB) 2026-02-24T22:06:25.829 INFO:tasks.workunit.client.0.trial046.stdout:Collecting charset_normalizer<4,>=2 (from requests) 2026-02-24T22:06:25.848 INFO:tasks.workunit.client.0.trial046.stdout: Downloading charset_normalizer-3.4.4-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (37 kB) 2026-02-24T22:06:25.857 INFO:tasks.workunit.client.0.trial046.stdout:Collecting idna<4,>=2.5 (from requests) 2026-02-24T22:06:25.861 INFO:tasks.workunit.client.0.trial046.stdout: Downloading idna-3.11-py3-none-any.whl.metadata (8.4 kB) 2026-02-24T22:06:25.875 INFO:tasks.workunit.client.0.trial046.stdout:Collecting certifi>=2017.4.17 (from requests) 2026-02-24T22:06:25.879 INFO:tasks.workunit.client.0.trial046.stdout: Downloading certifi-2026.1.4-py3-none-any.whl.metadata (2.5 kB) 2026-02-24T22:06:25.884 INFO:tasks.workunit.client.0.trial046.stdout:Downloading boto3-1.42.56-py3-none-any.whl (140 kB) 2026-02-24T22:06:25.892 INFO:tasks.workunit.client.0.trial046.stdout:Downloading botocore-1.42.56-py3-none-any.whl (14.6 MB) 2026-02-24T22:06:26.004 INFO:tasks.workunit.client.0.trial046.stdout: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.6/14.6 MB 133.6 MB/s 0:00:00 2026-02-24T22:06:26.008 INFO:tasks.workunit.client.0.trial046.stdout:Downloading jmespath-1.1.0-py3-none-any.whl (20 kB) 2026-02-24T22:06:26.013 INFO:tasks.workunit.client.0.trial046.stdout:Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2026-02-24T22:06:26.019 INFO:tasks.workunit.client.0.trial046.stdout:Downloading s3transfer-0.16.0-py3-none-any.whl (86 kB) 2026-02-24T22:06:26.024 INFO:tasks.workunit.client.0.trial046.stdout:Downloading urllib3-2.6.3-py3-none-any.whl (131 kB) 2026-02-24T22:06:26.029 INFO:tasks.workunit.client.0.trial046.stdout:Downloading requests-2.32.5-py3-none-any.whl (64 kB) 2026-02-24T22:06:26.035 INFO:tasks.workunit.client.0.trial046.stdout:Downloading charset_normalizer-3.4.4-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (153 kB) 2026-02-24T22:06:26.041 INFO:tasks.workunit.client.0.trial046.stdout:Downloading idna-3.11-py3-none-any.whl (71 kB) 2026-02-24T22:06:26.047 INFO:tasks.workunit.client.0.trial046.stdout:Downloading certifi-2026.1.4-py3-none-any.whl (152 kB) 2026-02-24T22:06:26.057 INFO:tasks.workunit.client.0.trial046.stdout:Downloading six-1.17.0-py2.py3-none-any.whl (11 kB) 2026-02-24T22:06:26.087 INFO:tasks.workunit.client.0.trial046.stdout:Installing collected packages: urllib3, six, jmespath, idna, charset_normalizer, certifi, requests, python-dateutil, botocore, s3transfer, boto3 2026-02-24T22:06:26.493 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:06:26.493 INFO:tasks.workunit.client.0.trial046.stdout:Successfully installed boto3-1.42.56 botocore-1.42.56 certifi-2026.1.4 charset_normalizer-3.4.4 idna-3.11 jmespath-1.1.0 python-dateutil-2.9.0.post0 requests-2.32.5 s3transfer-0.16.0 six-1.17.0 urllib3-2.6.3 2026-02-24T22:06:26.539 INFO:tasks.workunit.client.0.trial046.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/python3 /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_admin_pagination.py 2026-02-24T22:06:40.557 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:06:40.557 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:06:54.920 INFO:tasks.workunit.client.0.trial046.stderr:........ 2026-02-24T22:06:54.920 INFO:tasks.workunit.client.0.trial046.stderr:---------------------------------------------------------------------- 2026-02-24T22:06:54.921 INFO:tasks.workunit.client.0.trial046.stderr:Ran 8 tests in 28.295s 2026-02-24T22:06:54.921 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:06:54.921 INFO:tasks.workunit.client.0.trial046.stderr:OK 2026-02-24T22:06:54.921 INFO:tasks.workunit.client.0.trial046.stdout:Number of buckets: 0 2026-02-24T22:06:54.921 INFO:tasks.workunit.client.0.trial046.stdout:Populating 1500 buckets for test user... 2026-02-24T22:06:54.967 INFO:tasks.workunit.client.0.trial046.stderr:+ deactivate 2026-02-24T22:06:54.967 INFO:tasks.workunit.client.0.trial046.stderr:+ '[' -n /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin ']' 2026-02-24T22:06:54.967 INFO:tasks.workunit.client.0.trial046.stderr:+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T22:06:54.967 INFO:tasks.workunit.client.0.trial046.stderr:+ export PATH 2026-02-24T22:06:54.967 INFO:tasks.workunit.client.0.trial046.stderr:+ unset _OLD_VIRTUAL_PATH 2026-02-24T22:06:54.967 INFO:tasks.workunit.client.0.trial046.stderr:+ '[' -n '' ']' 2026-02-24T22:06:54.968 INFO:tasks.workunit.client.0.trial046.stderr:+ '[' -n /usr/bin/bash -o -n '' ']' 2026-02-24T22:06:54.968 INFO:tasks.workunit.client.0.trial046.stderr:+ hash -r 2026-02-24T22:06:54.968 INFO:tasks.workunit.client.0.trial046.stderr:+ '[' -n '' ']' 2026-02-24T22:06:54.968 INFO:tasks.workunit.client.0.trial046.stderr:+ unset VIRTUAL_ENV 2026-02-24T22:06:54.968 INFO:tasks.workunit.client.0.trial046.stderr:+ unset VIRTUAL_ENV_PROMPT 2026-02-24T22:06:54.968 INFO:tasks.workunit.client.0.trial046.stderr:+ '[' '!' '' = nondestructive ']' 2026-02-24T22:06:54.968 INFO:tasks.workunit.client.0.trial046.stderr:+ unset -f deactivate 2026-02-24T22:06:54.968 INFO:tasks.workunit.client.0.trial046.stderr:+ echo OK. 2026-02-24T22:06:54.969 INFO:tasks.workunit.client.0.trial046.stdout:OK. 2026-02-24T22:06:54.969 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T22:06:54.970 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T22:06:55.020 INFO:tasks.workunit:Stopping ['rgw/run-admin-pagination.sh'] on client.0... 2026-02-24T22:06:55.020 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-24T22:06:55.439 DEBUG:teuthology.parallel:result is None 2026-02-24T22:06:55.440 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-24T22:06:55.448 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-24T22:06:55.448 DEBUG:teuthology.orchestra.run.trial046:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-24T22:06:55.495 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-24T22:06:55.495 INFO:teuthology.run_tasks:Running task workunit... 2026-02-24T22:06:55.511 INFO:tasks.workunit:Pulling workunits from ref f7e05f4fdea5af173284f247d6b0935b337d4ccf 2026-02-24T22:06:55.511 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-24T22:06:55.511 DEBUG:teuthology.orchestra.run.trial046:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-24T22:06:55.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T22:06:55.539 INFO:teuthology.orchestra.run.trial046.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-24T22:06:55.540 DEBUG:teuthology.orchestra.run.trial046:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-24T22:06:55.587 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-24T22:06:55.587 DEBUG:teuthology.orchestra.run.trial046:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-24T22:06:55.631 INFO:tasks.workunit:timeout=3h 2026-02-24T22:06:55.631 INFO:tasks.workunit:cleanup=True 2026-02-24T22:06:55.632 DEBUG:teuthology.orchestra.run.trial046:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/samarahu/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout f7e05f4fdea5af173284f247d6b0935b337d4ccf 2026-02-24T22:06:55.676 INFO:tasks.workunit.client.0.trial046.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-24T22:07:17.778 INFO:tasks.workunit.client.0.trial046.stderr:Note: switching to 'f7e05f4fdea5af173284f247d6b0935b337d4ccf'. 2026-02-24T22:07:17.779 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:17.779 INFO:tasks.workunit.client.0.trial046.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-24T22:07:17.779 INFO:tasks.workunit.client.0.trial046.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-24T22:07:17.779 INFO:tasks.workunit.client.0.trial046.stderr:state without impacting any branches by switching back to a branch. 2026-02-24T22:07:17.779 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:17.779 INFO:tasks.workunit.client.0.trial046.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-24T22:07:17.779 INFO:tasks.workunit.client.0.trial046.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-24T22:07:17.779 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:17.779 INFO:tasks.workunit.client.0.trial046.stderr: git switch -c 2026-02-24T22:07:17.780 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:17.780 INFO:tasks.workunit.client.0.trial046.stderr:Or undo this operation with: 2026-02-24T22:07:17.780 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:17.780 INFO:tasks.workunit.client.0.trial046.stderr: git switch - 2026-02-24T22:07:17.780 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:17.780 INFO:tasks.workunit.client.0.trial046.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-24T22:07:17.780 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:17.780 INFO:tasks.workunit.client.0.trial046.stderr:HEAD is now at f7e05f4fdea rgw/d4n: Implement bucket check_empty and remove methods 2026-02-24T22:07:17.785 DEBUG:teuthology.orchestra.run.trial046:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-24T22:07:17.831 INFO:tasks.workunit.client.0.trial046.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2026-02-24T22:07:17.833 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-24T22:07:17.833 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2026-02-24T22:07:17.862 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2026-02-24T22:07:17.890 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2026-02-24T22:07:17.915 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-24T22:07:17.916 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-24T22:07:17.916 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2026-02-24T22:07:17.938 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-24T22:07:17.942 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:07:17.942 DEBUG:teuthology.orchestra.run.trial046:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-24T22:07:17.988 INFO:tasks.workunit:Running workunits matching rgw/run-bucket-check.sh on client.0... 2026-02-24T22:07:17.991 INFO:tasks.workunit:Running workunit rgw/run-bucket-check.sh... 2026-02-24T22:07:17.991 DEBUG:teuthology.orchestra.run.trial046:workunit test rgw/run-bucket-check.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7e05f4fdea5af173284f247d6b0935b337d4ccf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-bucket-check.sh 2026-02-24T22:07:18.039 INFO:tasks.workunit.client.0.trial046.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-bucket-check.sh 2026-02-24T22:07:18.040 INFO:tasks.workunit.client.0.trial046.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2026-02-24T22:07:18.040 INFO:tasks.workunit.client.0.trial046.stderr:+ python3 -m venv /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2026-02-24T22:07:19.383 INFO:tasks.workunit.client.0.trial046.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/activate 2026-02-24T22:07:19.384 INFO:tasks.workunit.client.0.trial046.stderr:++ deactivate nondestructive 2026-02-24T22:07:19.384 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n '' ']' 2026-02-24T22:07:19.384 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n '' ']' 2026-02-24T22:07:19.384 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2026-02-24T22:07:19.384 INFO:tasks.workunit.client.0.trial046.stderr:++ hash -r 2026-02-24T22:07:19.384 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n '' ']' 2026-02-24T22:07:19.384 INFO:tasks.workunit.client.0.trial046.stderr:++ unset VIRTUAL_ENV 2026-02-24T22:07:19.384 INFO:tasks.workunit.client.0.trial046.stderr:++ unset VIRTUAL_ENV_PROMPT 2026-02-24T22:07:19.385 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' '!' nondestructive = nondestructive ']' 2026-02-24T22:07:19.385 INFO:tasks.workunit.client.0.trial046.stderr:++ VIRTUAL_ENV=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2026-02-24T22:07:19.385 INFO:tasks.workunit.client.0.trial046.stderr:++ export VIRTUAL_ENV 2026-02-24T22:07:19.385 INFO:tasks.workunit.client.0.trial046.stderr:++ _OLD_VIRTUAL_PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T22:07:19.385 INFO:tasks.workunit.client.0.trial046.stderr:++ PATH=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T22:07:19.385 INFO:tasks.workunit.client.0.trial046.stderr:++ export PATH 2026-02-24T22:07:19.385 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n '' ']' 2026-02-24T22:07:19.385 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -z '' ']' 2026-02-24T22:07:19.385 INFO:tasks.workunit.client.0.trial046.stderr:++ _OLD_VIRTUAL_PS1= 2026-02-24T22:07:19.385 INFO:tasks.workunit.client.0.trial046.stderr:++ PS1='(rgw) ' 2026-02-24T22:07:19.385 INFO:tasks.workunit.client.0.trial046.stderr:++ export PS1 2026-02-24T22:07:19.385 INFO:tasks.workunit.client.0.trial046.stderr:++ VIRTUAL_ENV_PROMPT='(rgw) ' 2026-02-24T22:07:19.386 INFO:tasks.workunit.client.0.trial046.stderr:++ export VIRTUAL_ENV_PROMPT 2026-02-24T22:07:19.386 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2026-02-24T22:07:19.386 INFO:tasks.workunit.client.0.trial046.stderr:++ hash -r 2026-02-24T22:07:19.386 INFO:tasks.workunit.client.0.trial046.stderr:+ pip install pip --upgrade 2026-02-24T22:07:19.563 INFO:tasks.workunit.client.0.trial046.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2026-02-24T22:07:19.622 INFO:tasks.workunit.client.0.trial046.stdout:Collecting pip 2026-02-24T22:07:19.626 INFO:tasks.workunit.client.0.trial046.stdout: Using cached pip-26.0.1-py3-none-any.whl (1.8 MB) 2026-02-24T22:07:19.642 INFO:tasks.workunit.client.0.trial046.stdout:Installing collected packages: pip 2026-02-24T22:07:19.643 INFO:tasks.workunit.client.0.trial046.stdout: Attempting uninstall: pip 2026-02-24T22:07:19.643 INFO:tasks.workunit.client.0.trial046.stdout: Found existing installation: pip 22.0.2 2026-02-24T22:07:19.818 INFO:tasks.workunit.client.0.trial046.stdout: Uninstalling pip-22.0.2: 2026-02-24T22:07:19.821 INFO:tasks.workunit.client.0.trial046.stdout: Successfully uninstalled pip-22.0.2 2026-02-24T22:07:20.162 INFO:tasks.workunit.client.0.trial046.stdout:Successfully installed pip-26.0.1 2026-02-24T22:07:20.190 INFO:tasks.workunit.client.0.trial046.stderr:+ pip install boto3 2026-02-24T22:07:20.512 INFO:tasks.workunit.client.0.trial046.stdout:Collecting boto3 2026-02-24T22:07:20.512 INFO:tasks.workunit.client.0.trial046.stdout: Using cached boto3-1.42.56-py3-none-any.whl.metadata (6.7 kB) 2026-02-24T22:07:20.631 INFO:tasks.workunit.client.0.trial046.stdout:Collecting botocore<1.43.0,>=1.42.56 (from boto3) 2026-02-24T22:07:20.631 INFO:tasks.workunit.client.0.trial046.stdout: Using cached botocore-1.42.56-py3-none-any.whl.metadata (5.9 kB) 2026-02-24T22:07:20.641 INFO:tasks.workunit.client.0.trial046.stdout:Collecting jmespath<2.0.0,>=0.7.1 (from boto3) 2026-02-24T22:07:20.642 INFO:tasks.workunit.client.0.trial046.stdout: Using cached jmespath-1.1.0-py3-none-any.whl.metadata (7.6 kB) 2026-02-24T22:07:20.651 INFO:tasks.workunit.client.0.trial046.stdout:Collecting s3transfer<0.17.0,>=0.16.0 (from boto3) 2026-02-24T22:07:20.651 INFO:tasks.workunit.client.0.trial046.stdout: Using cached s3transfer-0.16.0-py3-none-any.whl.metadata (1.7 kB) 2026-02-24T22:07:20.659 INFO:tasks.workunit.client.0.trial046.stdout:Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.43.0,>=1.42.56->boto3) 2026-02-24T22:07:20.660 INFO:tasks.workunit.client.0.trial046.stdout: Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) 2026-02-24T22:07:20.672 INFO:tasks.workunit.client.0.trial046.stdout:Collecting urllib3!=2.2.0,<3,>=1.25.4 (from botocore<1.43.0,>=1.42.56->boto3) 2026-02-24T22:07:20.673 INFO:tasks.workunit.client.0.trial046.stdout: Using cached urllib3-2.6.3-py3-none-any.whl.metadata (6.9 kB) 2026-02-24T22:07:20.681 INFO:tasks.workunit.client.0.trial046.stdout:Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.43.0,>=1.42.56->boto3) 2026-02-24T22:07:20.681 INFO:tasks.workunit.client.0.trial046.stdout: Using cached six-1.17.0-py2.py3-none-any.whl.metadata (1.7 kB) 2026-02-24T22:07:20.700 INFO:tasks.workunit.client.0.trial046.stdout:Using cached boto3-1.42.56-py3-none-any.whl (140 kB) 2026-02-24T22:07:20.701 INFO:tasks.workunit.client.0.trial046.stdout:Using cached botocore-1.42.56-py3-none-any.whl (14.6 MB) 2026-02-24T22:07:20.710 INFO:tasks.workunit.client.0.trial046.stdout:Using cached jmespath-1.1.0-py3-none-any.whl (20 kB) 2026-02-24T22:07:20.710 INFO:tasks.workunit.client.0.trial046.stdout:Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2026-02-24T22:07:20.711 INFO:tasks.workunit.client.0.trial046.stdout:Using cached s3transfer-0.16.0-py3-none-any.whl (86 kB) 2026-02-24T22:07:20.711 INFO:tasks.workunit.client.0.trial046.stdout:Using cached urllib3-2.6.3-py3-none-any.whl (131 kB) 2026-02-24T22:07:20.712 INFO:tasks.workunit.client.0.trial046.stdout:Using cached six-1.17.0-py2.py3-none-any.whl (11 kB) 2026-02-24T22:07:20.738 INFO:tasks.workunit.client.0.trial046.stdout:Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3 2026-02-24T22:07:21.072 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:07:21.073 INFO:tasks.workunit.client.0.trial046.stdout:Successfully installed boto3-1.42.56 botocore-1.42.56 jmespath-1.1.0 python-dateutil-2.9.0.post0 s3transfer-0.16.0 six-1.17.0 urllib3-2.6.3 2026-02-24T22:07:21.113 INFO:tasks.workunit.client.0.trial046.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/python3 /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_bucket_check.py 2026-02-24T22:07:21.200 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin user create --uid check-tester --display-name "Check Testing" --access-key OJODXSLNX4LUNHQG99PA --secret 3l6ffld34qaymfomuh832j94738aie2x4p2o8h6n 2026-02-24T22:07:21.218 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:07:21.218 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:07:21.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.230+0000 7fe4dd261a80 4 Realm: () 2026-02-24T22:07:21.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.230+0000 7fe4dd261a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:07:21.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.230+0000 7fe4dd261a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:07:21.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.230+0000 7fe4dd261a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:07:21.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.230+0000 7fe4dd261a80 20 zonegroup default 2026-02-24T22:07:21.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.230+0000 7fe4dd261a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:21.236 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.234+0000 7fe4dd261a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:21.237 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.234+0000 7fe4dd261a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:21.237 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.234+0000 7fe4dd261a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:21.237 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.234+0000 7fe4dd261a80 20 started sync module instance, tier type = 2026-02-24T22:07:21.237 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.234+0000 7fe4dd261a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:07:21.238 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.234+0000 7fe4dad35640 0 -- 10.20.193.46:0/2832942586 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x562168a1ae50 msgr2=0x562168a3b300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:21.239 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.234+0000 7fe4dd261a80 20 add_watcher() i=1 2026-02-24T22:07:21.240 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.234+0000 7fe4dd261a80 20 add_watcher() i=5 2026-02-24T22:07:21.240 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.234+0000 7fe4dd261a80 20 add_watcher() i=3 2026-02-24T22:07:21.240 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.238+0000 7fe4dd261a80 20 add_watcher() i=0 2026-02-24T22:07:21.240 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.238+0000 7fe4dd261a80 20 add_watcher() i=7 2026-02-24T22:07:21.240 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.238+0000 7fe4dd261a80 20 add_watcher() i=4 2026-02-24T22:07:21.240 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.238+0000 7fe4dd261a80 20 add_watcher() i=2 2026-02-24T22:07:21.241 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.238+0000 7fe4dd261a80 20 add_watcher() i=6 2026-02-24T22:07:21.241 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.238+0000 7fe4dd261a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:07:21.242 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.238+0000 7fe4dd261a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:07:21.242 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.238+0000 7fe4dd261a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:07:21.242 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.238+0000 7fe4dd261a80 5 note: GC not initialized 2026-02-24T22:07:21.242 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.238+0000 7fe488fe1640 20 reqs_thread_entry: start 2026-02-24T22:07:21.243 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.238+0000 7fe4db536640 0 -- 10.20.193.46:0/3479749587 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x562168b30860 msgr2=0x562168b30c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:07:21.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.478+0000 7fe4dd261a80 20 init_complete bucket index max shards: 1 2026-02-24T22:07:21.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.478+0000 7fe4dd261a80 20 Filter name: none 2026-02-24T22:07:21.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.478+0000 7fe3f2ffd640 20 reqs_thread_entry: start 2026-02-24T22:07:21.483 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.478+0000 7fe4dd261a80 10 cache get: name=default.rgw.meta+users.uid+check-tester : miss 2026-02-24T22:07:21.483 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.478+0000 7fe4dd261a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:21.483 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.478+0000 7fe4dd261a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:21.483 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.478+0000 7fe4dd261a80 10 cache put: name=default.rgw.meta+users.uid+check-tester info.flags=0x0 2026-02-24T22:07:21.483 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.478+0000 7fe4dd261a80 10 adding default.rgw.meta+users.uid+check-tester to cache LRU end 2026-02-24T22:07:21.483 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.478+0000 7fe4dd261a80 10 cache get: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA : miss 2026-02-24T22:07:21.483 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.478+0000 7fe4dd261a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:21.483 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.478+0000 7fe4dd261a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:21.483 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.478+0000 7fe4dd261a80 10 cache put: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA info.flags=0x0 2026-02-24T22:07:21.483 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.478+0000 7fe4dd261a80 10 adding default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA to cache LRU end 2026-02-24T22:07:21.483 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.478+0000 7fe4dd261a80 10 cache get: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA : hit (negative entry) 2026-02-24T22:07:21.483 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.478+0000 7fe4dd261a80 10 cache get: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA : hit (negative entry) 2026-02-24T22:07:21.484 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.478+0000 7fe4dd261a80 10 cache put: name=default.rgw.meta+users.uid+check-tester info.flags=0x17 2026-02-24T22:07:21.484 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.478+0000 7fe4dd261a80 10 moving default.rgw.meta+users.uid+check-tester to cache LRU end 2026-02-24T22:07:21.484 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.478+0000 7fe4dd261a80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:check-tester, ofs0, ns] 2026-02-24T22:07:21.484 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.482+0000 7fe4b67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 158913789955 cookie 94701490271952 notifier 4652 bl.length()=593 2026-02-24T22:07:21.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.482+0000 7fe4b67fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+check-tester info.flags=0x17 2026-02-24T22:07:21.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.482+0000 7fe4b67fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+check-tester to cache LRU end 2026-02-24T22:07:21.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.482+0000 7fe4dd261a80 10 cache put: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA info.flags=0x7 2026-02-24T22:07:21.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.482+0000 7fe4dd261a80 10 moving default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA to cache LRU end 2026-02-24T22:07:21.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.482+0000 7fe4dd261a80 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.keys:OJODXSLNX4LUNHQG99PA, ofs0, ns] 2026-02-24T22:07:21.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.482+0000 7fe4b6ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 158913791369 cookie 94701489331696 notifier 4652 bl.length()=186 2026-02-24T22:07:21.486 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.482+0000 7fe4b6ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA info.flags=0x7 2026-02-24T22:07:21.486 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.482+0000 7fe4b6ffd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA to cache LRU end 2026-02-24T22:07:21.488 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.486+0000 7fe4dd261a80 20 remove_watcher() i=1 2026-02-24T22:07:21.489 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.486+0000 7fe4dd261a80 2 removed watcher, disabling cache 2026-02-24T22:07:21.489 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.486+0000 7fe4dd261a80 20 remove_watcher() i=0 2026-02-24T22:07:21.489 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.486+0000 7fe4dd261a80 20 remove_watcher() i=3 2026-02-24T22:07:21.489 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.486+0000 7fe4dd261a80 20 remove_watcher() i=2 2026-02-24T22:07:21.489 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.486+0000 7fe4dd261a80 20 remove_watcher() i=6 2026-02-24T22:07:21.489 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.486+0000 7fe4dd261a80 20 remove_watcher() i=5 2026-02-24T22:07:21.489 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.486+0000 7fe4dd261a80 20 remove_watcher() i=4 2026-02-24T22:07:21.489 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.486+0000 7fe4dd261a80 20 remove_watcher() i=7 2026-02-24T22:07:21.567 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:21.567 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:21.613 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:21.614 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:21.657 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:21.658 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:21.666 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:21.666 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:21.673 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:21.673 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:21.676 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:21.676 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:21.681 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:21.681 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:21.684 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:21.684 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:21.687 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:21.687 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:21.697 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:21.697 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:21.700 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:21.700 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:21.709 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:21.710 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:21.712 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:21.712 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:21.722 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:21.723 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:21.725 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:21.725 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:21.735 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:21.735 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:21.737 INFO:tasks.workunit.client.0.trial046.stderr:TEST: recalculated bucket check stats are correct 2026-02-24T22:07:21.737 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:21.737 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket check --fix --bucket check-bucket 2026-02-24T22:07:21.752 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:07:21.752 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:07:21.768 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.766+0000 7fbfe0369a80 4 Realm: () 2026-02-24T22:07:21.769 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.766+0000 7fbfe0369a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:07:21.769 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.766+0000 7fbfe0369a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:07:21.769 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.766+0000 7fbfe0369a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:07:21.769 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.766+0000 7fbfe0369a80 20 zonegroup default 2026-02-24T22:07:21.769 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.766+0000 7fbfe0369a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:21.770 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.766+0000 7fbfe0369a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:21.770 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.766+0000 7fbfe0369a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:21.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.766+0000 7fbfe0369a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:21.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.766+0000 7fbfe0369a80 20 started sync module instance, tier type = 2026-02-24T22:07:21.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.766+0000 7fbfe0369a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:07:21.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.770+0000 7fbfdde3d640 0 -- 10.20.193.46:0/966286003 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x558f8a701810 msgr2=0x558f8a721c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:21.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.770+0000 7fbfe0369a80 20 add_watcher() i=1 2026-02-24T22:07:21.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.770+0000 7fbfe0369a80 20 add_watcher() i=3 2026-02-24T22:07:21.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.770+0000 7fbfe0369a80 20 add_watcher() i=5 2026-02-24T22:07:21.773 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.770+0000 7fbfe0369a80 20 add_watcher() i=0 2026-02-24T22:07:21.773 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.770+0000 7fbfe0369a80 20 add_watcher() i=2 2026-02-24T22:07:21.774 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.770+0000 7fbfe0369a80 20 add_watcher() i=7 2026-02-24T22:07:21.774 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.770+0000 7fbfe0369a80 20 add_watcher() i=6 2026-02-24T22:07:21.774 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.770+0000 7fbfe0369a80 20 add_watcher() i=4 2026-02-24T22:07:21.774 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.770+0000 7fbfe0369a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:07:21.775 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.770+0000 7fbfe0369a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:07:21.775 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.770+0000 7fbfe0369a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:07:21.775 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.770+0000 7fbfe0369a80 5 note: GC not initialized 2026-02-24T22:07:21.775 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.774+0000 7fbf0e7fc640 20 reqs_thread_entry: start 2026-02-24T22:07:21.776 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:21.774+0000 7fbfde63e640 0 -- 10.20.193.46:0/92823415 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x558f8a87b090 msgr2=0x558f8a875e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:07:22.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.014+0000 7fbfe0369a80 20 init_complete bucket index max shards: 1 2026-02-24T22:07:22.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.014+0000 7fbfe0369a80 20 Filter name: none 2026-02-24T22:07:22.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.014+0000 7fbf0cff9640 20 reqs_thread_entry: start 2026-02-24T22:07:22.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.014+0000 7fbfe0369a80 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2026-02-24T22:07:22.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.014+0000 7fbfe0369a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:22.019 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.014+0000 7fbfe0369a80 20 rados_obj.operate() r=0 bl.length=196 2026-02-24T22:07:22.019 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.014+0000 7fbfe0369a80 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2026-02-24T22:07:22.019 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.014+0000 7fbfe0369a80 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2026-02-24T22:07:22.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.014+0000 7fbfe0369a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 : miss 2026-02-24T22:07:22.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.014+0000 7fbfe0369a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:22.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.014+0000 7fbfe0369a80 20 rados_obj.operate() r=0 bl.length=426 2026-02-24T22:07:22.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.014+0000 7fbfe0369a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 info.flags=0x17 2026-02-24T22:07:22.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.014+0000 7fbfe0369a80 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 to cache LRU end 2026-02-24T22:07:22.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.014+0000 7fbfe0369a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:07:22.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.014+0000 7fbfe0369a80 10 updating xattr: name=user.rgw.acl bl.length()=179 2026-02-24T22:07:22.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.014+0000 7fbfe0369a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 2026-02-24T22:07:22.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.014+0000 7fbfe0369a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:22.026 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.022+0000 7fbfe0369a80 20 remove_watcher() i=3 2026-02-24T22:07:22.026 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.022+0000 7fbfe0369a80 2 removed watcher, disabling cache 2026-02-24T22:07:22.026 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.022+0000 7fbfe0369a80 20 remove_watcher() i=5 2026-02-24T22:07:22.026 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.022+0000 7fbfe0369a80 20 remove_watcher() i=2 2026-02-24T22:07:22.026 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.022+0000 7fbfe0369a80 20 remove_watcher() i=4 2026-02-24T22:07:22.026 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.022+0000 7fbfe0369a80 20 remove_watcher() i=1 2026-02-24T22:07:22.027 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.022+0000 7fbfe0369a80 20 remove_watcher() i=6 2026-02-24T22:07:22.027 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.022+0000 7fbfe0369a80 20 remove_watcher() i=0 2026-02-24T22:07:22.027 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.022+0000 7fbfe0369a80 20 remove_watcher() i=7 2026-02-24T22:07:22.033 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket check-bucket 2026-02-24T22:07:22.048 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:07:22.048 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:07:22.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.062+0000 7f85aca1da80 4 Realm: () 2026-02-24T22:07:22.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.062+0000 7f85aca1da80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:07:22.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.062+0000 7f85aca1da80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:07:22.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.062+0000 7f85aca1da80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:07:22.066 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.062+0000 7f85aca1da80 20 zonegroup default 2026-02-24T22:07:22.066 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.062+0000 7f85aca1da80 20 rados->read ofs=0 len=0 2026-02-24T22:07:22.066 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.062+0000 7f85aca1da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:22.066 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.062+0000 7f85aca1da80 20 rados->read ofs=0 len=0 2026-02-24T22:07:22.067 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.062+0000 7f85aca1da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:22.067 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.062+0000 7f85aca1da80 20 started sync module instance, tier type = 2026-02-24T22:07:22.067 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.062+0000 7f85aca1da80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:07:22.068 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.066+0000 7f85aca1da80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:07:22.068 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.066+0000 7f85aca1da80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:07:22.068 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.066+0000 7f85aca1da80 5 note: GC not initialized 2026-02-24T22:07:22.068 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.066+0000 7f84cbfff640 20 reqs_thread_entry: start 2026-02-24T22:07:22.071 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.070+0000 7f85aacf2640 0 -- 10.20.193.46:0/1261342655 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55b42d879d90 msgr2=0x55b42d8736c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:07:22.283 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.278+0000 7f85aa4f1640 0 -- 10.20.193.46:0/1261342655 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55b42d84e320 msgr2=0x55b42d87bed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:22.309 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.306+0000 7f85aca1da80 20 init_complete bucket index max shards: 1 2026-02-24T22:07:22.310 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.306+0000 7f85aca1da80 20 Filter name: none 2026-02-24T22:07:22.310 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.306+0000 7f84ca7fc640 20 reqs_thread_entry: start 2026-02-24T22:07:22.310 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.306+0000 7f85aca1da80 20 rados->read ofs=0 len=0 2026-02-24T22:07:22.311 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.306+0000 7f85aca1da80 20 rados_obj.operate() r=0 bl.length=196 2026-02-24T22:07:22.311 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.306+0000 7f85aca1da80 20 rados->read ofs=0 len=0 2026-02-24T22:07:22.311 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.306+0000 7f85aca1da80 20 rados_obj.operate() r=0 bl.length=426 2026-02-24T22:07:22.311 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.306+0000 7f85aca1da80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:07:22.311 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.306+0000 7f85aca1da80 20 rados->read ofs=0 len=0 2026-02-24T22:07:22.311 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.306+0000 7f85aca1da80 20 rados_obj.operate() r=0 bl.length=196 2026-02-24T22:07:22.311 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.306+0000 7f85aca1da80 20 rados->read ofs=0 len=0 2026-02-24T22:07:22.311 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.306+0000 7f85aca1da80 20 rados_obj.operate() r=0 bl.length=426 2026-02-24T22:07:22.311 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.306+0000 7f85aca1da80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:07:22.322 INFO:tasks.workunit.client.0.trial046.stderr:{'rgw.main': {'size': 54, 'size_actual': 24576, 'size_utilized': 109, 'size_kb': 1, 'size_kb_actual': 24, 'size_kb_utilized': 1, 'num_objects': 6}} 2026-02-24T22:07:22.322 INFO:tasks.workunit.client.0.trial046.stderr:TEST: bucket check unlinked does not report normal entries 2026-02-24T22:07:22.322 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:22.322 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --min-age-hours 0 --dump-keys 2026-02-24T22:07:22.338 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:07:22.338 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:07:22.356 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.354+0000 7f5d2b2eca80 4 Realm: () 2026-02-24T22:07:22.356 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.354+0000 7f5d2b2eca80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:07:22.356 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.354+0000 7f5d2b2eca80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:07:22.356 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.354+0000 7f5d2b2eca80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:07:22.356 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.354+0000 7f5d2b2eca80 20 zonegroup default 2026-02-24T22:07:22.356 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.354+0000 7f5d2b2eca80 20 rados->read ofs=0 len=0 2026-02-24T22:07:22.357 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.354+0000 7f5d2b2eca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:22.357 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.354+0000 7f5d2b2eca80 20 rados->read ofs=0 len=0 2026-02-24T22:07:22.358 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.354+0000 7f5d2b2eca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:22.358 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.354+0000 7f5d2b2eca80 20 started sync module instance, tier type = 2026-02-24T22:07:22.358 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.354+0000 7f5d2b2eca80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:07:22.358 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.354+0000 7f5d23fff640 0 -- 10.20.193.46:0/2975280516 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x5598a2e6d200 msgr2=0x5598a2e8d6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:22.359 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.358+0000 7f5d2b2eca80 20 add_watcher() i=5 2026-02-24T22:07:22.359 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.358+0000 7f5d2b2eca80 20 add_watcher() i=3 2026-02-24T22:07:22.360 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.358+0000 7f5d2b2eca80 20 add_watcher() i=6 2026-02-24T22:07:22.360 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.358+0000 7f5d2b2eca80 20 add_watcher() i=1 2026-02-24T22:07:22.361 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.358+0000 7f5d2b2eca80 20 add_watcher() i=0 2026-02-24T22:07:22.361 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.358+0000 7f5d2b2eca80 20 add_watcher() i=4 2026-02-24T22:07:22.361 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.358+0000 7f5d2b2eca80 20 add_watcher() i=7 2026-02-24T22:07:22.361 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.358+0000 7f5d2b2eca80 20 add_watcher() i=2 2026-02-24T22:07:22.361 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.358+0000 7f5d2b2eca80 2 all 8 watchers are set, enabling cache 2026-02-24T22:07:22.361 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.358+0000 7f5d23fff640 0 -- 10.20.193.46:0/370658670 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f5c48008af0 msgr2=0x7f5c48028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:22.362 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.358+0000 7f5d2b2eca80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:07:22.362 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.358+0000 7f5d2b2eca80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:07:22.362 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.358+0000 7f5d2b2eca80 5 note: GC not initialized 2026-02-24T22:07:22.362 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.358+0000 7f5cd5fe3640 20 reqs_thread_entry: start 2026-02-24T22:07:22.395 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.390+0000 7f5d2b2eca80 20 init_complete bucket index max shards: 1 2026-02-24T22:07:22.395 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.390+0000 7f5d2b2eca80 20 Filter name: none 2026-02-24T22:07:22.395 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.390+0000 7f5c2ffff640 20 reqs_thread_entry: start 2026-02-24T22:07:22.395 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.390+0000 7f5d2b2eca80 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2026-02-24T22:07:22.395 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.390+0000 7f5d2b2eca80 20 rados->read ofs=0 len=0 2026-02-24T22:07:22.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.390+0000 7f5d2b2eca80 20 rados_obj.operate() r=0 bl.length=196 2026-02-24T22:07:22.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.390+0000 7f5d2b2eca80 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2026-02-24T22:07:22.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.390+0000 7f5d2b2eca80 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2026-02-24T22:07:22.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.390+0000 7f5d2b2eca80 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 : miss 2026-02-24T22:07:22.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.390+0000 7f5d2b2eca80 20 rados->read ofs=0 len=0 2026-02-24T22:07:22.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.390+0000 7f5d2b2eca80 20 rados_obj.operate() r=0 bl.length=426 2026-02-24T22:07:22.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.390+0000 7f5d2b2eca80 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 info.flags=0x17 2026-02-24T22:07:22.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.390+0000 7f5d2b2eca80 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 to cache LRU end 2026-02-24T22:07:22.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.390+0000 7f5d2b2eca80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:07:22.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.390+0000 7f5d2b2eca80 10 updating xattr: name=user.rgw.acl bl.length()=179 2026-02-24T22:07:22.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.390+0000 7f5d2b2eca80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 2026-02-24T22:07:22.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.390+0000 7f5d2b2eca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:22.397 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.394+0000 7f5d2b2eca80 1 NOTICE: finished shard 0 (0 entries found) 2026-02-24T22:07:22.397 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.394+0000 7f5d2b2eca80 1 NOTICE: finished all shards (0 entries found) 2026-02-24T22:07:22.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.394+0000 7f5d2b2eca80 20 remove_watcher() i=3 2026-02-24T22:07:22.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.394+0000 7f5d2b2eca80 2 removed watcher, disabling cache 2026-02-24T22:07:22.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.394+0000 7f5d2b2eca80 20 remove_watcher() i=1 2026-02-24T22:07:22.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.394+0000 7f5d2b2eca80 20 remove_watcher() i=4 2026-02-24T22:07:22.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.394+0000 7f5d2b2eca80 20 remove_watcher() i=2 2026-02-24T22:07:22.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.394+0000 7f5d2b2eca80 20 remove_watcher() i=6 2026-02-24T22:07:22.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.394+0000 7f5d2b2eca80 20 remove_watcher() i=5 2026-02-24T22:07:22.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.394+0000 7f5d2b2eca80 20 remove_watcher() i=0 2026-02-24T22:07:22.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:22.394+0000 7f5d2b2eca80 20 remove_watcher() i=7 2026-02-24T22:07:22.405 INFO:tasks.workunit.client.0.trial046.stderr:ceph config set client rgw_debug_inject_set_olh_err 2 2026-02-24T22:07:22.611 INFO:tasks.workunit.client.0.trial046.stderr:ceph config set client rgw_debug_inject_olh_cancel_modification_err true 2026-02-24T22:07:23.820 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:23.821 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:23.871 INFO:tasks.workunit.client.0.trial046.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2026-02-24T22:07:23.872 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bi list --bucket check-bucket --object c 2026-02-24T22:07:23.888 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:07:23.888 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:07:23.905 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.902+0000 7f6c3cd38a80 4 Realm: () 2026-02-24T22:07:23.905 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.902+0000 7f6c3cd38a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:07:23.905 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.902+0000 7f6c3cd38a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:07:23.905 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.902+0000 7f6c3cd38a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:07:23.906 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.902+0000 7f6c3cd38a80 20 zonegroup default 2026-02-24T22:07:23.906 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.902+0000 7f6c3cd38a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:23.907 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.902+0000 7f6c3cd38a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:23.907 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.902+0000 7f6c3cd38a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:23.907 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.906+0000 7f6c3cd38a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:23.907 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.906+0000 7f6c3cd38a80 20 started sync module instance, tier type = 2026-02-24T22:07:23.908 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.906+0000 7f6c3cd38a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:07:23.909 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.906+0000 7f6c3cd38a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:07:23.909 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.906+0000 7f6c3cd38a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:07:23.909 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.906+0000 7f6c3cd38a80 5 note: GC not initialized 2026-02-24T22:07:23.909 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.906+0000 7f6be97e2640 20 reqs_thread_entry: start 2026-02-24T22:07:23.912 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.910+0000 7f6c3b00d640 0 -- 10.20.193.46:0/14180632 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55e5ee133cc0 msgr2=0x55e5ee12d680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:23.918 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.914+0000 7f6c3a80c640 0 -- 10.20.193.46:0/14180632 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55e5ee1082c0 msgr2=0x55e5ee135e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:23.944 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.942+0000 7f6c3cd38a80 20 init_complete bucket index max shards: 1 2026-02-24T22:07:23.944 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.942+0000 7f6c3cd38a80 20 Filter name: none 2026-02-24T22:07:23.944 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.942+0000 7f6b537fe640 20 reqs_thread_entry: start 2026-02-24T22:07:23.944 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.942+0000 7f6c3cd38a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:23.944 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.942+0000 7f6c3cd38a80 20 rados_obj.operate() r=0 bl.length=196 2026-02-24T22:07:23.944 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.942+0000 7f6c3cd38a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:23.945 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.942+0000 7f6c3cd38a80 20 rados_obj.operate() r=0 bl.length=426 2026-02-24T22:07:23.945 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.942+0000 7f6c3cd38a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:07:23.945 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.942+0000 7f6c3cd38a80 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2026-02-24T22:07:23.945 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.942+0000 7f6c3cd38a80 20 INFO: main: starting shard=0 2026-02-24T22:07:23.945 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.942+0000 7f6c3cd38a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:23.946 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.942+0000 7f6c3cd38a80 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, next_marker=�1001_c 2026-02-24T22:07:23.946 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:23.942+0000 7f6c3cd38a80 20 INFO: main: done 2026-02-24T22:07:23.955 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:23.955 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:23.958 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:23.958 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:23.960 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:23.960 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:24.007 INFO:tasks.workunit.client.0.trial046.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2026-02-24T22:07:24.008 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bi list --bucket check-bucket --object d 2026-02-24T22:07:24.024 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:07:24.024 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:07:24.041 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.038+0000 7ff1bfb6ea80 4 Realm: () 2026-02-24T22:07:24.042 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.038+0000 7ff1bfb6ea80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:07:24.042 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.038+0000 7ff1bfb6ea80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:07:24.042 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.038+0000 7ff1bfb6ea80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:07:24.042 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.038+0000 7ff1bfb6ea80 20 zonegroup default 2026-02-24T22:07:24.042 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.038+0000 7ff1bfb6ea80 20 rados->read ofs=0 len=0 2026-02-24T22:07:24.043 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.042+0000 7ff1bfb6ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:24.043 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.042+0000 7ff1bfb6ea80 20 rados->read ofs=0 len=0 2026-02-24T22:07:24.044 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.042+0000 7ff1bfb6ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:24.044 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.042+0000 7ff1bfb6ea80 20 started sync module instance, tier type = 2026-02-24T22:07:24.044 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.042+0000 7ff1bfb6ea80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:07:24.045 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.042+0000 7ff1bfb6ea80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:07:24.046 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.042+0000 7ff1bfb6ea80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:07:24.046 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.042+0000 7ff1bfb6ea80 5 note: GC not initialized 2026-02-24T22:07:24.046 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.042+0000 7ff16afe5640 20 reqs_thread_entry: start 2026-02-24T22:07:24.049 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.046+0000 7ff1bde43640 0 -- 10.20.193.46:0/3566000377 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55dcb6314d00 msgr2=0x55dcb6307f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:24.259 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.254+0000 7ff1bd642640 0 -- 10.20.193.46:0/3566000377 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55dcb6310780 msgr2=0x55dcb630b4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:24.285 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.282+0000 7ff1bfb6ea80 20 init_complete bucket index max shards: 1 2026-02-24T22:07:24.285 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.282+0000 7ff1bfb6ea80 20 Filter name: none 2026-02-24T22:07:24.285 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.282+0000 7ff1697e2640 20 reqs_thread_entry: start 2026-02-24T22:07:24.286 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.282+0000 7ff1bfb6ea80 20 rados->read ofs=0 len=0 2026-02-24T22:07:24.286 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.282+0000 7ff1bfb6ea80 20 rados_obj.operate() r=0 bl.length=196 2026-02-24T22:07:24.286 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.282+0000 7ff1bfb6ea80 20 rados->read ofs=0 len=0 2026-02-24T22:07:24.286 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.282+0000 7ff1bfb6ea80 20 rados_obj.operate() r=0 bl.length=426 2026-02-24T22:07:24.287 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.282+0000 7ff1bfb6ea80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:07:24.287 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.282+0000 7ff1bfb6ea80 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2026-02-24T22:07:24.287 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.282+0000 7ff1bfb6ea80 20 INFO: main: starting shard=0 2026-02-24T22:07:24.287 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.282+0000 7ff1bfb6ea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:24.287 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.282+0000 7ff1bfb6ea80 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, next_marker=�1001_d 2026-02-24T22:07:24.287 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.282+0000 7ff1bfb6ea80 20 INFO: main: done 2026-02-24T22:07:24.296 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:24.296 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:24.299 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:24.299 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:24.302 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:24.302 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:24.355 INFO:tasks.workunit.client.0.trial046.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2026-02-24T22:07:24.356 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bi list --bucket check-bucket --object e 2026-02-24T22:07:24.372 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:07:24.372 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:07:24.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.386+0000 7fe380cb6a80 4 Realm: () 2026-02-24T22:07:24.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.386+0000 7fe380cb6a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:07:24.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.386+0000 7fe380cb6a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:07:24.389 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.386+0000 7fe380cb6a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:07:24.389 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.386+0000 7fe380cb6a80 20 zonegroup default 2026-02-24T22:07:24.389 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.386+0000 7fe380cb6a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:24.390 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.386+0000 7fe380cb6a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:24.390 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.386+0000 7fe380cb6a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:24.391 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.386+0000 7fe380cb6a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:24.391 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.386+0000 7fe380cb6a80 20 started sync module instance, tier type = 2026-02-24T22:07:24.391 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.386+0000 7fe380cb6a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:07:24.392 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.390+0000 7fe380cb6a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:07:24.393 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.390+0000 7fe380cb6a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:07:24.393 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.390+0000 7fe380cb6a80 5 note: GC not initialized 2026-02-24T22:07:24.393 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.390+0000 7fe29ffff640 20 reqs_thread_entry: start 2026-02-24T22:07:24.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.394+0000 7fe37ef8b640 0 -- 10.20.193.46:0/2785743853 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x5612c87cf500 msgr2=0x5612c87c8e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:24.401 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.398+0000 7fe37e78a640 0 -- 10.20.193.46:0/2785743853 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5612c87a3ad0 msgr2=0x5612c87d16a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:24.423 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.422+0000 7fe380cb6a80 20 init_complete bucket index max shards: 1 2026-02-24T22:07:24.423 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.422+0000 7fe380cb6a80 20 Filter name: none 2026-02-24T22:07:24.424 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.422+0000 7fe29e7fc640 20 reqs_thread_entry: start 2026-02-24T22:07:24.424 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.422+0000 7fe380cb6a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:24.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.422+0000 7fe380cb6a80 20 rados_obj.operate() r=0 bl.length=196 2026-02-24T22:07:24.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.422+0000 7fe380cb6a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:24.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.422+0000 7fe380cb6a80 20 rados_obj.operate() r=0 bl.length=426 2026-02-24T22:07:24.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.422+0000 7fe380cb6a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:07:24.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.422+0000 7fe380cb6a80 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2026-02-24T22:07:24.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.422+0000 7fe380cb6a80 20 INFO: main: starting shard=0 2026-02-24T22:07:24.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.422+0000 7fe380cb6a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:24.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.422+0000 7fe380cb6a80 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, next_marker=�1000_eiJve8zxBRkXAskYz-8ZxM7mTQYzvk.Ur 2026-02-24T22:07:24.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.422+0000 7fe380cb6a80 20 INFO: main: done 2026-02-24T22:07:24.434 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:24.434 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:24.437 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:24.437 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:24.487 INFO:tasks.workunit.client.0.trial046.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2026-02-24T22:07:24.488 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bi list --bucket check-bucket --object f 2026-02-24T22:07:24.504 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:07:24.504 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:07:24.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.518+0000 7f67948dba80 4 Realm: () 2026-02-24T22:07:24.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.518+0000 7f67948dba80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:07:24.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.518+0000 7f67948dba80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:07:24.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.518+0000 7f67948dba80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:07:24.521 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.518+0000 7f67948dba80 20 zonegroup default 2026-02-24T22:07:24.521 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.518+0000 7f67948dba80 20 rados->read ofs=0 len=0 2026-02-24T22:07:24.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.518+0000 7f67948dba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:24.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.518+0000 7f67948dba80 20 rados->read ofs=0 len=0 2026-02-24T22:07:24.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.518+0000 7f67948dba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:24.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.518+0000 7f67948dba80 20 started sync module instance, tier type = 2026-02-24T22:07:24.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.518+0000 7f67948dba80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:07:24.524 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.522+0000 7f67948dba80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:07:24.524 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.522+0000 7f67948dba80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:07:24.524 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.522+0000 7f67948dba80 5 note: GC not initialized 2026-02-24T22:07:24.524 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.522+0000 7f66b3fff640 20 reqs_thread_entry: start 2026-02-24T22:07:24.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.526+0000 7f6792bb0640 0 -- 10.20.193.46:0/462704017 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55c91f5fc5a0 msgr2=0x55c91f5ef810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:24.534 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.530+0000 7f67923af640 0 -- 10.20.193.46:0/462704017 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55c91f5f8020 msgr2=0x55c91f5f2d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:24.556 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.554+0000 7f67948dba80 20 init_complete bucket index max shards: 1 2026-02-24T22:07:24.556 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.554+0000 7f67948dba80 20 Filter name: none 2026-02-24T22:07:24.556 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.554+0000 7f66b27fc640 20 reqs_thread_entry: start 2026-02-24T22:07:24.556 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.554+0000 7f67948dba80 20 rados->read ofs=0 len=0 2026-02-24T22:07:24.556 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.554+0000 7f67948dba80 20 rados_obj.operate() r=0 bl.length=196 2026-02-24T22:07:24.556 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.554+0000 7f67948dba80 20 rados->read ofs=0 len=0 2026-02-24T22:07:24.556 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.554+0000 7f67948dba80 20 rados_obj.operate() r=0 bl.length=426 2026-02-24T22:07:24.556 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.554+0000 7f67948dba80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:07:24.557 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.554+0000 7f67948dba80 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2026-02-24T22:07:24.557 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.554+0000 7f67948dba80 20 INFO: main: starting shard=0 2026-02-24T22:07:24.557 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.554+0000 7f67948dba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:24.557 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.554+0000 7f67948dba80 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, next_marker=�1000_fiA2RSKHR3yRMnZzckyTxMcSq.x8z0hcI 2026-02-24T22:07:24.557 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:24.554+0000 7f67948dba80 20 INFO: main: done 2026-02-24T22:07:24.566 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:24.566 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:24.568 INFO:tasks.workunit.client.0.trial046.stderr:ceph config rm client rgw_debug_inject_set_olh_err 2026-02-24T22:07:24.776 INFO:tasks.workunit.client.0.trial046.stderr:ceph config rm client rgw_debug_inject_olh_cancel_modification_err 2026-02-24T22:07:24.983 INFO:tasks.workunit.client.0.trial046.stderr:TEST: bucket check unlinked finds unlistable entries 2026-02-24T22:07:24.983 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:24.984 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --min-age-hours 0 --dump-keys 2026-02-24T22:07:25.000 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:07:25.000 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:07:25.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.014+0000 7fc0fe5d6a80 4 Realm: () 2026-02-24T22:07:25.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.014+0000 7fc0fe5d6a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:07:25.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.014+0000 7fc0fe5d6a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:07:25.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.014+0000 7fc0fe5d6a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:07:25.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.014+0000 7fc0fe5d6a80 20 zonegroup default 2026-02-24T22:07:25.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.014+0000 7fc0fe5d6a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:25.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.014+0000 7fc0fe5d6a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:25.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.014+0000 7fc0fe5d6a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:25.019 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.014+0000 7fc0fe5d6a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:25.019 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.014+0000 7fc0fe5d6a80 20 started sync module instance, tier type = 2026-02-24T22:07:25.019 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.014+0000 7fc0fe5d6a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:07:25.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.018+0000 7fc0fc8ab640 0 -- 10.20.193.46:0/3203187215 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x562d71b620d0 msgr2=0x562d71b824b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:25.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.018+0000 7fc0fe5d6a80 20 add_watcher() i=1 2026-02-24T22:07:25.021 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.018+0000 7fc0fe5d6a80 20 add_watcher() i=6 2026-02-24T22:07:25.022 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.018+0000 7fc0fe5d6a80 20 add_watcher() i=5 2026-02-24T22:07:25.022 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.018+0000 7fc0fe5d6a80 20 add_watcher() i=3 2026-02-24T22:07:25.022 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.018+0000 7fc0fe5d6a80 20 add_watcher() i=0 2026-02-24T22:07:25.022 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.018+0000 7fc0fe5d6a80 20 add_watcher() i=7 2026-02-24T22:07:25.022 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.018+0000 7fc0fe5d6a80 20 add_watcher() i=2 2026-02-24T22:07:25.022 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.018+0000 7fc0fe5d6a80 20 add_watcher() i=4 2026-02-24T22:07:25.022 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.018+0000 7fc0fe5d6a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:07:25.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.018+0000 7fc0fe5d6a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:07:25.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.018+0000 7fc0fe5d6a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:07:25.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.018+0000 7fc0fe5d6a80 5 note: GC not initialized 2026-02-24T22:07:25.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.022+0000 7fc0a9fdb640 20 reqs_thread_entry: start 2026-02-24T22:07:25.024 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.022+0000 7fc0fc8ab640 0 -- 10.20.193.46:0/2671923358 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x562d71c9d810 msgr2=0x562d71c9dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:25.470 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.466+0000 7fc0fe5d6a80 20 init_complete bucket index max shards: 1 2026-02-24T22:07:25.470 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.466+0000 7fc0fe5d6a80 20 Filter name: none 2026-02-24T22:07:25.470 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.466+0000 7fc003fff640 20 reqs_thread_entry: start 2026-02-24T22:07:25.470 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.466+0000 7fc0fe5d6a80 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2026-02-24T22:07:25.471 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.466+0000 7fc0fe5d6a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:25.471 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.466+0000 7fc0fe5d6a80 20 rados_obj.operate() r=0 bl.length=196 2026-02-24T22:07:25.471 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.466+0000 7fc0fe5d6a80 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2026-02-24T22:07:25.471 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.466+0000 7fc0fe5d6a80 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2026-02-24T22:07:25.471 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.466+0000 7fc0fe5d6a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 : miss 2026-02-24T22:07:25.471 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.466+0000 7fc0fe5d6a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:25.471 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.466+0000 7fc0fe5d6a80 20 rados_obj.operate() r=0 bl.length=426 2026-02-24T22:07:25.471 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.466+0000 7fc0fe5d6a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 info.flags=0x17 2026-02-24T22:07:25.471 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.466+0000 7fc0fe5d6a80 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 to cache LRU end 2026-02-24T22:07:25.471 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.466+0000 7fc0fe5d6a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:07:25.471 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.466+0000 7fc0fe5d6a80 10 updating xattr: name=user.rgw.acl bl.length()=179 2026-02-24T22:07:25.472 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.466+0000 7fc0fe5d6a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 2026-02-24T22:07:25.472 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.466+0000 7fc0fe5d6a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.472 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.470+0000 7fc0fe5d6a80 1 NOTICE: finished shard 0 (4 entries found) 2026-02-24T22:07:25.472 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.470+0000 7fc0fe5d6a80 1 NOTICE: finished all shards (4 entries found) 2026-02-24T22:07:25.475 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.470+0000 7fc0fe5d6a80 20 remove_watcher() i=3 2026-02-24T22:07:25.475 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.470+0000 7fc0fe5d6a80 2 removed watcher, disabling cache 2026-02-24T22:07:25.475 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.470+0000 7fc0fe5d6a80 20 remove_watcher() i=2 2026-02-24T22:07:25.475 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.470+0000 7fc0fe5d6a80 20 remove_watcher() i=4 2026-02-24T22:07:25.475 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.470+0000 7fc0fe5d6a80 20 remove_watcher() i=6 2026-02-24T22:07:25.475 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.470+0000 7fc0fe5d6a80 20 remove_watcher() i=5 2026-02-24T22:07:25.475 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.470+0000 7fc0fe5d6a80 20 remove_watcher() i=7 2026-02-24T22:07:25.475 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.470+0000 7fc0fe5d6a80 20 remove_watcher() i=1 2026-02-24T22:07:25.475 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.470+0000 7fc0fe5d6a80 20 remove_watcher() i=0 2026-02-24T22:07:25.480 INFO:tasks.workunit.client.0.trial046.stderr:TEST: unlinked entries are not listable 2026-02-24T22:07:25.480 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:25.481 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:25.481 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:25.484 INFO:tasks.workunit.client.0.trial046.stderr:TEST: GET returns 404 for unlinked entry keys that have no other versions 2026-02-24T22:07:25.485 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:25.486 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:25.486 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:25.537 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:25.537 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:25.579 INFO:tasks.workunit.client.0.trial046.stderr:TEST: bucket check unlinked fixes unlistable entries 2026-02-24T22:07:25.579 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:25.580 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --fix --min-age-hours 0 --rgw-olh-pending-timeout-sec 0 --dump-keys 2026-02-24T22:07:25.597 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:07:25.597 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:07:25.616 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.614+0000 7f4af5752a80 4 Realm: () 2026-02-24T22:07:25.616 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.614+0000 7f4af5752a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:07:25.616 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.614+0000 7f4af5752a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:07:25.616 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.614+0000 7f4af5752a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:07:25.616 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.614+0000 7f4af5752a80 20 zonegroup default 2026-02-24T22:07:25.617 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.614+0000 7f4af5752a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:25.617 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.614+0000 7f4af5752a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:25.618 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.614+0000 7f4af5752a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:25.618 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.614+0000 7f4af5752a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:25.618 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.614+0000 7f4af5752a80 20 started sync module instance, tier type = 2026-02-24T22:07:25.618 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.614+0000 7f4af5752a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:07:25.619 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.614+0000 7f4aeeffd640 0 -- 10.20.193.46:0/1813406671 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x55a4326b8480 msgr2=0x55a4326d8900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:25.619 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.618+0000 7f4af5752a80 20 add_watcher() i=1 2026-02-24T22:07:25.620 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.618+0000 7f4af5752a80 20 add_watcher() i=6 2026-02-24T22:07:25.620 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.618+0000 7f4af5752a80 20 add_watcher() i=5 2026-02-24T22:07:25.620 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.618+0000 7f4af5752a80 20 add_watcher() i=3 2026-02-24T22:07:25.621 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.618+0000 7f4af5752a80 20 add_watcher() i=0 2026-02-24T22:07:25.621 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.618+0000 7f4af5752a80 20 add_watcher() i=4 2026-02-24T22:07:25.621 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.618+0000 7f4af5752a80 20 add_watcher() i=7 2026-02-24T22:07:25.621 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.618+0000 7f4aeeffd640 0 -- 10.20.193.46:0/1813406671 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x7f4ab409a940 msgr2=0x7f4ab40bad40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:25.621 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.618+0000 7f4af5752a80 20 add_watcher() i=2 2026-02-24T22:07:25.621 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.618+0000 7f4af5752a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:07:25.623 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.618+0000 7f4af5752a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:07:25.623 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.618+0000 7f4af5752a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:07:25.623 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.618+0000 7f4af5752a80 5 note: GC not initialized 2026-02-24T22:07:25.623 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.622+0000 7f4a0bfff640 20 reqs_thread_entry: start 2026-02-24T22:07:25.656 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.654+0000 7f4af5752a80 20 init_complete bucket index max shards: 1 2026-02-24T22:07:25.656 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.654+0000 7f4af5752a80 20 Filter name: none 2026-02-24T22:07:25.656 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.654+0000 7f4a0a7fc640 20 reqs_thread_entry: start 2026-02-24T22:07:25.656 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.654+0000 7f4af5752a80 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2026-02-24T22:07:25.656 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.654+0000 7f4af5752a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:25.657 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.654+0000 7f4af5752a80 20 rados_obj.operate() r=0 bl.length=196 2026-02-24T22:07:25.658 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.654+0000 7f4af5752a80 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2026-02-24T22:07:25.658 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.654+0000 7f4af5752a80 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2026-02-24T22:07:25.658 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.654+0000 7f4af5752a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 : miss 2026-02-24T22:07:25.658 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.654+0000 7f4af5752a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:25.658 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.654+0000 7f4af5752a80 20 rados_obj.operate() r=0 bl.length=426 2026-02-24T22:07:25.658 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.654+0000 7f4af5752a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 info.flags=0x17 2026-02-24T22:07:25.658 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.654+0000 7f4af5752a80 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 to cache LRU end 2026-02-24T22:07:25.658 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.654+0000 7f4af5752a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:07:25.658 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.654+0000 7f4af5752a80 10 updating xattr: name=user.rgw.acl bl.length()=179 2026-02-24T22:07:25.658 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.654+0000 7f4af5752a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 2026-02-24T22:07:25.658 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.654+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.658 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.654+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.659 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.654+0000 7f4af5752a80 20 get_obj_state: octx=0x55a4326d0610 obj=check-bucket:c state=0x55a43279bb88 s->prefetch_data=0 2026-02-24T22:07:25.659 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.654+0000 7f4af5752a80 20 get_obj_state: setting s->obj_tag to lnow21rzfypnt5fhs28sidtyd1yemgx6 2026-02-24T22:07:25.659 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.654+0000 7f4af5752a80 20 get_obj_state_impl: setting s->olh_tag to oqiug55j6ehjojax0loay7tse2jidreh 2026-02-24T22:07:25.659 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.654+0000 7f4af5752a80 20 bucket_index_guard_olh_op(): olh_state.olh_tag=oqiug55j6ehjojax0loay7tse2jidreh 2026-02-24T22:07:25.660 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.654+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.660 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.658+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.660 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.658+0000 7f4af5752a80 20 olh_log_entry: epoch=1771970845663230816 op=3 key=c[zKbHaaHlLQTB4uXBAIYkTS.wy0pMdGE] 2026-02-24T22:07:25.660 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.658+0000 7f4af5752a80 20 get_obj_state: octx=0x55a4326d0610 obj=check-bucket:_:zKbHaaHlLQTB4uXBAIYkTS.wy0pMdGE_c state=0x55a4327861a8 s->prefetch_data=0 2026-02-24T22:07:25.661 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.658+0000 7f4af5752a80 10 manifest: total_size = 9 2026-02-24T22:07:25.661 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.658+0000 7f4af5752a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5352147836155486887 2026-02-24T22:07:25.661 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.658+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.662 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.658+0000 7f4af5752a80 20 get_obj_state: octx=0x55a4326d0610 obj=check-bucket:_:zKbHaaHlLQTB4uXBAIYkTS.wy0pMdGE_c state=0x55a4327861a8 s->prefetch_data=0 2026-02-24T22:07:25.662 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.658+0000 7f4af5752a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:07:25.662 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.658+0000 7f4af5752a80 20 RGWObjManifest::operator++(): result: ofs=9 stripe_ofs=9 part_ofs=0 rule->part_size=0 2026-02-24T22:07:25.662 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.658+0000 7f4af5752a80 0 deleting objects inline since gc isn't initialized 2026-02-24T22:07:25.663 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.658+0000 7f4acdffb640 20 handle_completion(): completion ok for obj=c[zKbHaaHlLQTB4uXBAIYkTS.wy0pMdGE] 2026-02-24T22:07:25.663 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.662+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.664 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.662+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.664 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.662+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.665 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.662+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.665 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.662+0000 7f4af5752a80 20 get_obj_state: octx=0x55a4326d0610 obj=check-bucket:d state=0x55a43279bb88 s->prefetch_data=0 2026-02-24T22:07:25.665 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.662+0000 7f4af5752a80 20 get_obj_state: setting s->obj_tag to n34an9zcro68ptzsezsgatkndyai910d 2026-02-24T22:07:25.666 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.662+0000 7f4af5752a80 20 get_obj_state_impl: setting s->olh_tag to 2s7rr58r99aho24gkjda8rluwnp03haj 2026-02-24T22:07:25.666 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.662+0000 7f4af5752a80 20 bucket_index_guard_olh_op(): olh_state.olh_tag=2s7rr58r99aho24gkjda8rluwnp03haj 2026-02-24T22:07:25.666 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.662+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.667 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.666+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.667 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.666+0000 7f4af5752a80 20 olh_log_entry: epoch=1771970845670672837 op=3 key=d[zLuTUOmkV7eQROdYlGVTpmMdOyAkFMi] 2026-02-24T22:07:25.668 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.666+0000 7f4af5752a80 20 get_obj_state: octx=0x55a4326d0610 obj=check-bucket:_:zLuTUOmkV7eQROdYlGVTpmMdOyAkFMi_d state=0x55a4327810e8 s->prefetch_data=0 2026-02-24T22:07:25.668 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.666+0000 7f4af5752a80 10 manifest: total_size = 12 2026-02-24T22:07:25.668 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.666+0000 7f4af5752a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14529479808365748411 2026-02-24T22:07:25.668 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.666+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.669 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.666+0000 7f4af5752a80 20 get_obj_state: octx=0x55a4326d0610 obj=check-bucket:_:zLuTUOmkV7eQROdYlGVTpmMdOyAkFMi_d state=0x55a4327810e8 s->prefetch_data=0 2026-02-24T22:07:25.670 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.666+0000 7f4af5752a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:07:25.670 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.666+0000 7f4af5752a80 20 RGWObjManifest::operator++(): result: ofs=12 stripe_ofs=12 part_ofs=0 rule->part_size=0 2026-02-24T22:07:25.670 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.666+0000 7f4af5752a80 0 deleting objects inline since gc isn't initialized 2026-02-24T22:07:25.670 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.666+0000 7f4ace7fc640 20 handle_completion(): completion ok for obj=d[zLuTUOmkV7eQROdYlGVTpmMdOyAkFMi] 2026-02-24T22:07:25.671 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.670+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.671 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.670+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.672 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.670+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.672 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.670+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.673 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.670+0000 7f4af5752a80 20 get_obj_state: octx=0x55a43279dfc0 obj=check-bucket:e state=0x55a43279bb88 s->prefetch_data=0 2026-02-24T22:07:25.673 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.670+0000 7f4af5752a80 20 get_obj_state: setting s->obj_tag to 6smsdwol7cr24ihjmkeqdx3jr34es1sm 2026-02-24T22:07:25.673 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.670+0000 7f4af5752a80 20 get_obj_state_impl: setting s->olh_tag to 3jjb80j8tiranwlpv377gczkv98ua8bk 2026-02-24T22:07:25.673 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.670+0000 7f4af5752a80 20 bucket_index_guard_olh_op(): olh_state.olh_tag=3jjb80j8tiranwlpv377gczkv98ua8bk 2026-02-24T22:07:25.674 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.670+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.670+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.674+0000 7f4af5752a80 20 olh_log_entry: epoch=1771970845678144553 op=2 key=e[] 2026-02-24T22:07:25.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.674+0000 7f4af5752a80 20 olh_log_entry: epoch=1771970845678144553 op=3 key=e[Jve8zxBRkXAskYz-8ZxM7mTQYzvk.Ur] 2026-02-24T22:07:25.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.674+0000 7f4af5752a80 20 get_obj_state: octx=0x55a43279dfc0 obj=check-bucket:_:Jve8zxBRkXAskYz-8ZxM7mTQYzvk.Ur_e state=0x55a4326177d8 s->prefetch_data=0 2026-02-24T22:07:25.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.674+0000 7f4af5752a80 10 manifest: total_size = 11 2026-02-24T22:07:25.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.674+0000 7f4af5752a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15259498760890293142 2026-02-24T22:07:25.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.674+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.677 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.674+0000 7f4af5752a80 20 get_obj_state: octx=0x55a43279dfc0 obj=check-bucket:_:Jve8zxBRkXAskYz-8ZxM7mTQYzvk.Ur_e state=0x55a4326177d8 s->prefetch_data=0 2026-02-24T22:07:25.677 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.674+0000 7f4af5752a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:07:25.677 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.674+0000 7f4af5752a80 20 RGWObjManifest::operator++(): result: ofs=11 stripe_ofs=11 part_ofs=0 rule->part_size=0 2026-02-24T22:07:25.677 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.674+0000 7f4af5752a80 0 deleting objects inline since gc isn't initialized 2026-02-24T22:07:25.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.674+0000 7f4acdffb640 20 handle_completion(): completion ok for obj=e[Jve8zxBRkXAskYz-8ZxM7mTQYzvk.Ur] 2026-02-24T22:07:25.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.674+0000 7f4af5752a80 20 get_obj_state: octx=0x55a43279dfc0 obj=check-bucket:e state=0x55a43279bb88 s->prefetch_data=0 2026-02-24T22:07:25.679 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.674+0000 7f4af5752a80 20 bucket_index_guard_olh_op(): olh_state.olh_tag=3jjb80j8tiranwlpv377gczkv98ua8bk 2026-02-24T22:07:25.680 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.678+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.678+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.678+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.682 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.678+0000 7f4af5752a80 20 get_obj_state: octx=0x7f4ad8069650 obj=check-bucket:f state=0x55a43279bb88 s->prefetch_data=0 2026-02-24T22:07:25.682 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.678+0000 7f4af5752a80 20 get_obj_state: setting s->obj_tag to u5wl00x49x2gdjntv7ycxz96ggt731k4 2026-02-24T22:07:25.682 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.678+0000 7f4af5752a80 20 get_obj_state_impl: setting s->olh_tag to u8ubxt7i96ckjd5qxsc3aq3bqs00tjjb 2026-02-24T22:07:25.682 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.678+0000 7f4af5752a80 20 bucket_index_guard_olh_op(): olh_state.olh_tag=u8ubxt7i96ckjd5qxsc3aq3bqs00tjjb 2026-02-24T22:07:25.683 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.678+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.684 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.682+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.684 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.682+0000 7f4af5752a80 20 olh_log_entry: epoch=1771970845687169615 op=2 key=f[] 2026-02-24T22:07:25.684 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.682+0000 7f4af5752a80 20 olh_log_entry: epoch=1771970845687169615 op=3 key=f[A2RSKHR3yRMnZzckyTxMcSq.x8z0hcI] 2026-02-24T22:07:25.684 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.682+0000 7f4af5752a80 20 get_obj_state: octx=0x7f4ad8069650 obj=check-bucket:_:A2RSKHR3yRMnZzckyTxMcSq.x8z0hcI_f state=0x55a4326167c8 s->prefetch_data=0 2026-02-24T22:07:25.685 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.682+0000 7f4af5752a80 10 manifest: total_size = 12 2026-02-24T22:07:25.685 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.682+0000 7f4af5752a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16553310458524941050 2026-02-24T22:07:25.685 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.682+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.682+0000 7f4af5752a80 20 get_obj_state: octx=0x7f4ad8069650 obj=check-bucket:_:A2RSKHR3yRMnZzckyTxMcSq.x8z0hcI_f state=0x55a4326167c8 s->prefetch_data=0 2026-02-24T22:07:25.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.682+0000 7f4af5752a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:07:25.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.682+0000 7f4af5752a80 20 RGWObjManifest::operator++(): result: ofs=12 stripe_ofs=12 part_ofs=0 rule->part_size=0 2026-02-24T22:07:25.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.682+0000 7f4af5752a80 0 deleting objects inline since gc isn't initialized 2026-02-24T22:07:25.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.682+0000 7f4acdffb640 20 handle_completion(): completion ok for obj=f[A2RSKHR3yRMnZzckyTxMcSq.x8z0hcI] 2026-02-24T22:07:25.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.686+0000 7f4af5752a80 20 get_obj_state: octx=0x7f4ad8069650 obj=check-bucket:f state=0x55a43279bb88 s->prefetch_data=0 2026-02-24T22:07:25.688 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.686+0000 7f4af5752a80 20 bucket_index_guard_olh_op(): olh_state.olh_tag=u8ubxt7i96ckjd5qxsc3aq3bqs00tjjb 2026-02-24T22:07:25.689 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.686+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.690 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.686+0000 7f4af5752a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.690 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.686+0000 7f4af5752a80 1 NOTICE: finished shard 0 (4 entries removed) 2026-02-24T22:07:25.690 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.686+0000 7f4af5752a80 1 NOTICE: finished all shards (4 entries removed) 2026-02-24T22:07:25.693 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.690+0000 7f4af5752a80 20 remove_watcher() i=3 2026-02-24T22:07:25.693 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.690+0000 7f4af5752a80 2 removed watcher, disabling cache 2026-02-24T22:07:25.693 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.690+0000 7f4af5752a80 20 remove_watcher() i=1 2026-02-24T22:07:25.693 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.690+0000 7f4af5752a80 20 remove_watcher() i=6 2026-02-24T22:07:25.694 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.690+0000 7f4af5752a80 20 remove_watcher() i=4 2026-02-24T22:07:25.694 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.690+0000 7f4af5752a80 20 remove_watcher() i=2 2026-02-24T22:07:25.694 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.690+0000 7f4af5752a80 20 remove_watcher() i=0 2026-02-24T22:07:25.694 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.690+0000 7f4af5752a80 20 remove_watcher() i=7 2026-02-24T22:07:25.694 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.690+0000 7f4af5752a80 20 remove_watcher() i=5 2026-02-24T22:07:25.701 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:25.702 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:25.704 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:25.704 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:25.706 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:25.706 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:25.708 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:25.708 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:25.709 INFO:tasks.workunit.client.0.trial046.stderr:TEST: bucket check unlinked does not affect normal entries 2026-02-24T22:07:25.709 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:25.710 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:25.710 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:25.757 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:25.757 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:25.760 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:25.760 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:25.762 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:25.762 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:25.765 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:25.765 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:25.767 INFO:tasks.workunit.client.0.trial046.stderr:TEST: bucket check unlinked does not find new unlistable entries after fix 2026-02-24T22:07:25.767 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:25.767 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --min-age-hours 0 --dump-keys 2026-02-24T22:07:25.785 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:07:25.785 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:07:25.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.798+0000 7f1448b4ca80 4 Realm: () 2026-02-24T22:07:25.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.798+0000 7f1448b4ca80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:07:25.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.798+0000 7f1448b4ca80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:07:25.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.798+0000 7f1448b4ca80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:07:25.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.798+0000 7f1448b4ca80 20 zonegroup default 2026-02-24T22:07:25.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.798+0000 7f1448b4ca80 20 rados->read ofs=0 len=0 2026-02-24T22:07:25.803 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.802+0000 7f1448b4ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:25.803 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.802+0000 7f1448b4ca80 20 rados->read ofs=0 len=0 2026-02-24T22:07:25.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.802+0000 7f1448b4ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:25.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.802+0000 7f1448b4ca80 20 started sync module instance, tier type = 2026-02-24T22:07:25.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.802+0000 7f1448b4ca80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:07:25.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.802+0000 7f1446620640 0 -- 10.20.193.46:0/1878983097 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x55d6ab174480 msgr2=0x55d6ab194900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:25.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.802+0000 7f1448b4ca80 20 add_watcher() i=1 2026-02-24T22:07:25.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.802+0000 7f1448b4ca80 20 add_watcher() i=6 2026-02-24T22:07:25.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.802+0000 7f1448b4ca80 20 add_watcher() i=3 2026-02-24T22:07:25.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.802+0000 7f1448b4ca80 20 add_watcher() i=0 2026-02-24T22:07:25.807 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.802+0000 7f1448b4ca80 20 add_watcher() i=5 2026-02-24T22:07:25.807 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.802+0000 7f1448b4ca80 20 add_watcher() i=2 2026-02-24T22:07:25.807 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.802+0000 7f1448b4ca80 20 add_watcher() i=4 2026-02-24T22:07:25.807 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.802+0000 7f1448b4ca80 20 add_watcher() i=7 2026-02-24T22:07:25.807 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.802+0000 7f1448b4ca80 2 all 8 watchers are set, enabling cache 2026-02-24T22:07:25.808 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.806+0000 7f1445e1f640 0 -- 10.20.193.46:0/2625991918 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f1368008af0 msgr2=0x7f1368028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:07:25.809 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.806+0000 7f1448b4ca80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:07:25.809 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.806+0000 7f1448b4ca80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:07:25.809 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.806+0000 7f1448b4ca80 5 note: GC not initialized 2026-02-24T22:07:25.809 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.806+0000 7f135f7fe640 20 reqs_thread_entry: start 2026-02-24T22:07:25.844 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.842+0000 7f1448b4ca80 20 init_complete bucket index max shards: 1 2026-02-24T22:07:25.844 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.842+0000 7f1448b4ca80 20 Filter name: none 2026-02-24T22:07:25.844 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.842+0000 7f135dffb640 20 reqs_thread_entry: start 2026-02-24T22:07:25.844 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.842+0000 7f1448b4ca80 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2026-02-24T22:07:25.844 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.842+0000 7f1448b4ca80 20 rados->read ofs=0 len=0 2026-02-24T22:07:25.845 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.842+0000 7f1448b4ca80 20 rados_obj.operate() r=0 bl.length=196 2026-02-24T22:07:25.845 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.842+0000 7f1448b4ca80 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2026-02-24T22:07:25.845 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.842+0000 7f1448b4ca80 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2026-02-24T22:07:25.845 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.842+0000 7f1448b4ca80 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 : miss 2026-02-24T22:07:25.845 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.842+0000 7f1448b4ca80 20 rados->read ofs=0 len=0 2026-02-24T22:07:25.845 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.842+0000 7f1448b4ca80 20 rados_obj.operate() r=0 bl.length=426 2026-02-24T22:07:25.845 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.842+0000 7f1448b4ca80 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 info.flags=0x17 2026-02-24T22:07:25.845 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.842+0000 7f1448b4ca80 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 to cache LRU end 2026-02-24T22:07:25.845 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.842+0000 7f1448b4ca80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:07:25.845 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.842+0000 7f1448b4ca80 10 updating xattr: name=user.rgw.acl bl.length()=179 2026-02-24T22:07:25.845 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.842+0000 7f1448b4ca80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 2026-02-24T22:07:25.845 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.842+0000 7f1448b4ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:25.846 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.842+0000 7f1448b4ca80 1 NOTICE: finished shard 0 (0 entries found) 2026-02-24T22:07:25.846 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.842+0000 7f1448b4ca80 1 NOTICE: finished all shards (0 entries found) 2026-02-24T22:07:25.847 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.842+0000 7f1448b4ca80 20 remove_watcher() i=3 2026-02-24T22:07:25.848 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.842+0000 7f1448b4ca80 2 removed watcher, disabling cache 2026-02-24T22:07:25.848 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.842+0000 7f1448b4ca80 20 remove_watcher() i=6 2026-02-24T22:07:25.848 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.842+0000 7f1448b4ca80 20 remove_watcher() i=2 2026-02-24T22:07:25.848 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.846+0000 7f1448b4ca80 20 remove_watcher() i=1 2026-02-24T22:07:25.848 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.846+0000 7f1448b4ca80 20 remove_watcher() i=7 2026-02-24T22:07:25.848 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.846+0000 7f1448b4ca80 20 remove_watcher() i=4 2026-02-24T22:07:25.848 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.846+0000 7f1448b4ca80 20 remove_watcher() i=0 2026-02-24T22:07:25.848 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.846+0000 7f1448b4ca80 20 remove_watcher() i=5 2026-02-24T22:07:25.856 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:25.856 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:25.866 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:25.866 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:25.874 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bi list --bucket check-bucket --object c 2026-02-24T22:07:25.889 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:07:25.889 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:07:25.906 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.902+0000 7f344268da80 4 Realm: () 2026-02-24T22:07:25.906 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.902+0000 7f344268da80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:07:25.906 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.902+0000 7f344268da80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:07:25.906 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.902+0000 7f344268da80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:07:25.906 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.902+0000 7f344268da80 20 zonegroup default 2026-02-24T22:07:25.907 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.902+0000 7f344268da80 20 rados->read ofs=0 len=0 2026-02-24T22:07:25.908 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.906+0000 7f344268da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:25.908 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.906+0000 7f344268da80 20 rados->read ofs=0 len=0 2026-02-24T22:07:25.908 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.906+0000 7f344268da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:25.909 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.906+0000 7f344268da80 20 started sync module instance, tier type = 2026-02-24T22:07:25.909 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.906+0000 7f344268da80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:07:25.910 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.906+0000 7f344268da80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:07:25.910 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.906+0000 7f344268da80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:07:25.910 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.906+0000 7f344268da80 5 note: GC not initialized 2026-02-24T22:07:25.911 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.906+0000 7f33ee7dc640 20 reqs_thread_entry: start 2026-02-24T22:07:25.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:25.910+0000 7f343bfff640 0 -- 10.20.193.46:0/1972128928 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55bd686b7310 msgr2=0x55bd686b7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:26.126 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.122+0000 7f3440962640 0 -- 10.20.193.46:0/1972128928 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x55bd686a89d0 msgr2=0x55bd686a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:07:26.353 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.350+0000 7f344268da80 20 init_complete bucket index max shards: 1 2026-02-24T22:07:26.353 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.350+0000 7f344268da80 20 Filter name: none 2026-02-24T22:07:26.354 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.350+0000 7f33ecfd9640 20 reqs_thread_entry: start 2026-02-24T22:07:26.354 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.350+0000 7f344268da80 20 rados->read ofs=0 len=0 2026-02-24T22:07:26.354 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.350+0000 7f344268da80 20 rados_obj.operate() r=0 bl.length=196 2026-02-24T22:07:26.355 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.350+0000 7f344268da80 20 rados->read ofs=0 len=0 2026-02-24T22:07:26.355 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.350+0000 7f344268da80 20 rados_obj.operate() r=0 bl.length=426 2026-02-24T22:07:26.355 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.350+0000 7f344268da80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:07:26.355 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.350+0000 7f344268da80 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2026-02-24T22:07:26.355 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.350+0000 7f344268da80 20 INFO: main: starting shard=0 2026-02-24T22:07:26.355 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.350+0000 7f344268da80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:26.355 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.354+0000 7f344268da80 20 INFO: main: bi_list() returned without error; entries.size()=2, is_truncated=0, next_marker=�1001_c 2026-02-24T22:07:26.356 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.354+0000 7f344268da80 20 INFO: main: done 2026-02-24T22:07:26.364 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bi list --bucket check-bucket --object d 2026-02-24T22:07:26.380 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:07:26.380 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:07:26.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.394+0000 7f8b66014a80 4 Realm: () 2026-02-24T22:07:26.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.394+0000 7f8b66014a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:07:26.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.394+0000 7f8b66014a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:07:26.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.394+0000 7f8b66014a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:07:26.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.394+0000 7f8b66014a80 20 zonegroup default 2026-02-24T22:07:26.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.394+0000 7f8b66014a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:26.400 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.398+0000 7f8b66014a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:26.400 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.398+0000 7f8b66014a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:26.401 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.398+0000 7f8b66014a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:26.401 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.398+0000 7f8b66014a80 20 started sync module instance, tier type = 2026-02-24T22:07:26.401 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.398+0000 7f8b66014a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:07:26.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.398+0000 7f8b66014a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:07:26.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.398+0000 7f8b66014a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:07:26.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.398+0000 7f8b66014a80 5 note: GC not initialized 2026-02-24T22:07:26.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.398+0000 7f8b117e2640 20 reqs_thread_entry: start 2026-02-24T22:07:26.406 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.402+0000 7f8b5ffff640 0 -- 10.20.193.46:0/2355418523 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x564e0169de20 msgr2=0x564e01694610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:26.413 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.410+0000 7f8b5f7fe640 0 -- 10.20.193.46:0/2355418523 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x564e0166f2c0 msgr2=0x564e0169ce20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:26.436 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.434+0000 7f8b66014a80 20 init_complete bucket index max shards: 1 2026-02-24T22:07:26.436 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.434+0000 7f8b66014a80 20 Filter name: none 2026-02-24T22:07:26.436 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.434+0000 7f8a7b7fe640 20 reqs_thread_entry: start 2026-02-24T22:07:26.436 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.434+0000 7f8b66014a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:26.436 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.434+0000 7f8b66014a80 20 rados_obj.operate() r=0 bl.length=196 2026-02-24T22:07:26.436 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.434+0000 7f8b66014a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:26.437 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.434+0000 7f8b66014a80 20 rados_obj.operate() r=0 bl.length=426 2026-02-24T22:07:26.437 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.434+0000 7f8b66014a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:07:26.437 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.434+0000 7f8b66014a80 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2026-02-24T22:07:26.437 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.434+0000 7f8b66014a80 20 INFO: main: starting shard=0 2026-02-24T22:07:26.437 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.434+0000 7f8b66014a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:26.437 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.434+0000 7f8b66014a80 20 INFO: main: bi_list() returned without error; entries.size()=2, is_truncated=0, next_marker=�1001_d 2026-02-24T22:07:26.438 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.434+0000 7f8b66014a80 20 INFO: main: done 2026-02-24T22:07:26.445 INFO:tasks.workunit.client.0.trial046.stderr:TEST: bucket check olh finds leftover OLH entries 2026-02-24T22:07:26.445 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:26.445 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket check olh --bucket check-bucket --dump-keys 2026-02-24T22:07:26.460 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:07:26.461 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:07:26.479 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.478+0000 7f1078a9ba80 4 Realm: () 2026-02-24T22:07:26.479 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.478+0000 7f1078a9ba80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:07:26.480 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.478+0000 7f1078a9ba80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:07:26.480 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.478+0000 7f1078a9ba80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:07:26.480 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.478+0000 7f1078a9ba80 20 zonegroup default 2026-02-24T22:07:26.480 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.478+0000 7f1078a9ba80 20 rados->read ofs=0 len=0 2026-02-24T22:07:26.481 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.478+0000 7f1078a9ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:26.481 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.478+0000 7f1078a9ba80 20 rados->read ofs=0 len=0 2026-02-24T22:07:26.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.478+0000 7f1078a9ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:26.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.478+0000 7f1078a9ba80 20 started sync module instance, tier type = 2026-02-24T22:07:26.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.478+0000 7f1078a9ba80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:07:26.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.478+0000 7f107656f640 0 -- 10.20.193.46:0/553099377 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x556acd4077c0 msgr2=0x556acd427c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:26.483 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.482+0000 7f1078a9ba80 20 add_watcher() i=1 2026-02-24T22:07:26.484 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.482+0000 7f1078a9ba80 20 add_watcher() i=3 2026-02-24T22:07:26.484 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.482+0000 7f1078a9ba80 20 add_watcher() i=0 2026-02-24T22:07:26.484 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.482+0000 7f1078a9ba80 20 add_watcher() i=5 2026-02-24T22:07:26.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.482+0000 7f1078a9ba80 20 add_watcher() i=2 2026-02-24T22:07:26.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.482+0000 7f1078a9ba80 20 add_watcher() i=7 2026-02-24T22:07:26.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.482+0000 7f1078a9ba80 20 add_watcher() i=4 2026-02-24T22:07:26.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.482+0000 7f1078a9ba80 20 add_watcher() i=6 2026-02-24T22:07:26.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.482+0000 7f1078a9ba80 2 all 8 watchers are set, enabling cache 2026-02-24T22:07:26.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.482+0000 7f1075d6e640 0 -- 10.20.193.46:0/3121211353 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f0f98008af0 msgr2=0x7f0f98028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:26.486 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.482+0000 7f1078a9ba80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:07:26.486 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.482+0000 7f1078a9ba80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:07:26.486 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.482+0000 7f1078a9ba80 5 note: GC not initialized 2026-02-24T22:07:26.486 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.482+0000 7f0f8f7fe640 20 reqs_thread_entry: start 2026-02-24T22:07:26.518 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.514+0000 7f1078a9ba80 20 init_complete bucket index max shards: 1 2026-02-24T22:07:26.518 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.514+0000 7f1078a9ba80 20 Filter name: none 2026-02-24T22:07:26.518 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.514+0000 7f0f8dffb640 20 reqs_thread_entry: start 2026-02-24T22:07:26.518 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.514+0000 7f1078a9ba80 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2026-02-24T22:07:26.518 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.514+0000 7f1078a9ba80 20 rados->read ofs=0 len=0 2026-02-24T22:07:26.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.514+0000 7f1078a9ba80 20 rados_obj.operate() r=0 bl.length=196 2026-02-24T22:07:26.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.514+0000 7f1078a9ba80 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2026-02-24T22:07:26.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.514+0000 7f1078a9ba80 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2026-02-24T22:07:26.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.514+0000 7f1078a9ba80 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 : miss 2026-02-24T22:07:26.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.514+0000 7f1078a9ba80 20 rados->read ofs=0 len=0 2026-02-24T22:07:26.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.514+0000 7f1078a9ba80 20 rados_obj.operate() r=0 bl.length=426 2026-02-24T22:07:26.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.514+0000 7f1078a9ba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 info.flags=0x17 2026-02-24T22:07:26.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.514+0000 7f1078a9ba80 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 to cache LRU end 2026-02-24T22:07:26.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.514+0000 7f1078a9ba80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:07:26.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.514+0000 7f1078a9ba80 10 updating xattr: name=user.rgw.acl bl.length()=179 2026-02-24T22:07:26.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.514+0000 7f1078a9ba80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 2026-02-24T22:07:26.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.514+0000 7f1078a9ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:26.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.514+0000 7f1078a9ba80 1 NOTICE: finished shard 0 (2 entries found) 2026-02-24T22:07:26.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.514+0000 7f1078a9ba80 1 NOTICE: finished all shards (2 entries found) 2026-02-24T22:07:26.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.518+0000 7f1078a9ba80 20 remove_watcher() i=6 2026-02-24T22:07:26.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.518+0000 7f1078a9ba80 2 removed watcher, disabling cache 2026-02-24T22:07:26.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.518+0000 7f1078a9ba80 20 remove_watcher() i=3 2026-02-24T22:07:26.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.518+0000 7f1078a9ba80 20 remove_watcher() i=7 2026-02-24T22:07:26.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.518+0000 7f1078a9ba80 20 remove_watcher() i=0 2026-02-24T22:07:26.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.518+0000 7f1078a9ba80 20 remove_watcher() i=5 2026-02-24T22:07:26.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.518+0000 7f1078a9ba80 20 remove_watcher() i=2 2026-02-24T22:07:26.523 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.518+0000 7f1078a9ba80 20 remove_watcher() i=4 2026-02-24T22:07:26.523 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.518+0000 7f1078a9ba80 20 remove_watcher() i=1 2026-02-24T22:07:26.528 INFO:tasks.workunit.client.0.trial046.stderr:TEST: bucket check olh fixes leftover OLH entries 2026-02-24T22:07:26.528 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:26.529 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket check olh --bucket check-bucket --fix --rgw-olh-pending-timeout-sec 0 --dump-keys 2026-02-24T22:07:26.545 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:07:26.545 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:07:26.564 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.562+0000 7fc5745ada80 4 Realm: () 2026-02-24T22:07:26.564 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.562+0000 7fc5745ada80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:07:26.564 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.562+0000 7fc5745ada80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:07:26.564 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.562+0000 7fc5745ada80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:07:26.564 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.562+0000 7fc5745ada80 20 zonegroup default 2026-02-24T22:07:26.565 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.562+0000 7fc5745ada80 20 rados->read ofs=0 len=0 2026-02-24T22:07:26.565 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.562+0000 7fc5745ada80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:26.565 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.562+0000 7fc5745ada80 20 rados->read ofs=0 len=0 2026-02-24T22:07:26.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.562+0000 7fc5745ada80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:26.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.562+0000 7fc5745ada80 20 started sync module instance, tier type = 2026-02-24T22:07:26.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.562+0000 7fc5745ada80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:07:26.567 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.562+0000 7fc572882640 0 -- 10.20.193.46:0/811213124 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55b2c7974a40 msgr2=0x55b2c7994e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:26.567 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.566+0000 7fc5745ada80 20 add_watcher() i=1 2026-02-24T22:07:26.568 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.566+0000 7fc5745ada80 20 add_watcher() i=6 2026-02-24T22:07:26.568 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.566+0000 7fc5745ada80 20 add_watcher() i=5 2026-02-24T22:07:26.568 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.566+0000 7fc5745ada80 20 add_watcher() i=3 2026-02-24T22:07:26.568 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.566+0000 7fc5745ada80 20 add_watcher() i=0 2026-02-24T22:07:26.568 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.566+0000 7fc5745ada80 20 add_watcher() i=7 2026-02-24T22:07:26.568 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.566+0000 7fc5745ada80 20 add_watcher() i=4 2026-02-24T22:07:26.569 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.566+0000 7fc5745ada80 20 add_watcher() i=2 2026-02-24T22:07:26.569 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.566+0000 7fc5745ada80 2 all 8 watchers are set, enabling cache 2026-02-24T22:07:26.570 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.566+0000 7fc5745ada80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:07:26.570 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.566+0000 7fc5745ada80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:07:26.570 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.566+0000 7fc5745ada80 5 note: GC not initialized 2026-02-24T22:07:26.570 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.566+0000 7fc48b7fe640 20 reqs_thread_entry: start 2026-02-24T22:07:26.571 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.566+0000 7fc572882640 0 -- 10.20.193.46:0/2446652210 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55b2c7aafcb0 msgr2=0x55b2c7a9d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:26.810 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.806+0000 7fc5745ada80 20 init_complete bucket index max shards: 1 2026-02-24T22:07:26.810 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.806+0000 7fc5745ada80 20 Filter name: none 2026-02-24T22:07:26.810 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.806+0000 7fc489ffb640 20 reqs_thread_entry: start 2026-02-24T22:07:26.811 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.806+0000 7fc5745ada80 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2026-02-24T22:07:26.811 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.806+0000 7fc5745ada80 20 rados->read ofs=0 len=0 2026-02-24T22:07:26.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.806+0000 7fc5745ada80 20 rados_obj.operate() r=0 bl.length=196 2026-02-24T22:07:26.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.806+0000 7fc5745ada80 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2026-02-24T22:07:26.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.806+0000 7fc5745ada80 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2026-02-24T22:07:26.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.806+0000 7fc5745ada80 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 : miss 2026-02-24T22:07:26.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.806+0000 7fc5745ada80 20 rados->read ofs=0 len=0 2026-02-24T22:07:26.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.806+0000 7fc5745ada80 20 rados_obj.operate() r=0 bl.length=426 2026-02-24T22:07:26.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.806+0000 7fc5745ada80 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 info.flags=0x17 2026-02-24T22:07:26.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.806+0000 7fc5745ada80 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 to cache LRU end 2026-02-24T22:07:26.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.806+0000 7fc5745ada80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:07:26.813 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.806+0000 7fc5745ada80 10 updating xattr: name=user.rgw.acl bl.length()=179 2026-02-24T22:07:26.813 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.806+0000 7fc5745ada80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 2026-02-24T22:07:26.813 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.806+0000 7fc5745ada80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:26.813 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.810+0000 7fc5745ada80 20 get_obj_state: octx=0x55b2c7aa5d10 obj=check-bucket:c state=0x55b2c79937a8 s->prefetch_data=0 2026-02-24T22:07:26.813 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.810+0000 7fc5745ada80 20 get_obj_state: setting s->obj_tag to lnow21rzfypnt5fhs28sidtyd1yemgx6 2026-02-24T22:07:26.813 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.810+0000 7fc5745ada80 20 get_obj_state_impl: setting s->olh_tag to oqiug55j6ehjojax0loay7tse2jidreh 2026-02-24T22:07:26.813 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.810+0000 7fc5745ada80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:26.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.810+0000 7fc5745ada80 20 olh_log_entry: epoch=1771970845862587901 op=2 key=c[] 2026-02-24T22:07:26.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.810+0000 7fc5745ada80 1 apply_olh_log epoch collision detected for c; incoming op: 2(00000000699e211doozdpevdit9x95ck) 2026-02-24T22:07:26.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.810+0000 7fc5745ada80 20 olh_log_entry: epoch=1771970845862587901 op=3 key=c[Ur7SSqBJLOsTTDypXJcQto9wJnVyGan] 2026-02-24T22:07:26.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.810+0000 7fc5745ada80 1 apply_olh_log epoch collision detected for c[Ur7SSqBJLOsTTDypXJcQto9wJnVyGan]; incoming op: 3(00000000699e211doozdpevdit9x95ck) 2026-02-24T22:07:26.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.810+0000 7fc5745ada80 20 get_obj_state: octx=0x55b2c7953c80 obj=check-bucket:_:Ur7SSqBJLOsTTDypXJcQto9wJnVyGan_c state=0x55b2c7ab9358 s->prefetch_data=0 2026-02-24T22:07:26.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.810+0000 7fc5745ada80 20 get_obj_state: octx=0x55b2c7953c80 obj=check-bucket:c state=0x55b2c7abb9d8 s->prefetch_data=0 2026-02-24T22:07:26.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.810+0000 7fc5745ada80 20 get_obj_state: setting s->obj_tag to lnow21rzfypnt5fhs28sidtyd1yemgx6 2026-02-24T22:07:26.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.810+0000 7fc5745ada80 20 get_obj_state_impl: setting s->olh_tag to oqiug55j6ehjojax0loay7tse2jidreh 2026-02-24T22:07:26.815 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.810+0000 7fc5745ada80 20 bucket_index_guard_olh_op(): olh_state.olh_tag=oqiug55j6ehjojax0loay7tse2jidreh 2026-02-24T22:07:26.815 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.810+0000 7fc5745ada80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:26.816 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.814+0000 7fc5745ada80 20 get_obj_state: octx=0x55b2c7992f20 obj=check-bucket:d state=0x55b2c79932a8 s->prefetch_data=0 2026-02-24T22:07:26.817 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.814+0000 7fc5745ada80 20 get_obj_state: setting s->obj_tag to n34an9zcro68ptzsezsgatkndyai910d 2026-02-24T22:07:26.817 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.814+0000 7fc5745ada80 20 get_obj_state_impl: setting s->olh_tag to 2s7rr58r99aho24gkjda8rluwnp03haj 2026-02-24T22:07:26.817 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.814+0000 7fc5745ada80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:26.817 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.814+0000 7fc5745ada80 20 olh_log_entry: epoch=1771970845872425667 op=2 key=d[] 2026-02-24T22:07:26.818 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.814+0000 7fc5745ada80 1 apply_olh_log epoch collision detected for d; incoming op: 2(00000000699e211di3uw115rbw6ze5o3) 2026-02-24T22:07:26.818 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.814+0000 7fc5745ada80 20 olh_log_entry: epoch=1771970845872425667 op=3 key=d[LZNnIS5kAUzk.U58shOf41HUi-fJOoq] 2026-02-24T22:07:26.818 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.814+0000 7fc5745ada80 1 apply_olh_log epoch collision detected for d[LZNnIS5kAUzk.U58shOf41HUi-fJOoq]; incoming op: 3(00000000699e211di3uw115rbw6ze5o3) 2026-02-24T22:07:26.818 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.814+0000 7fc5745ada80 20 get_obj_state: octx=0x55b2c7953c80 obj=check-bucket:_:LZNnIS5kAUzk.U58shOf41HUi-fJOoq_d state=0x55b2c7ab40b8 s->prefetch_data=0 2026-02-24T22:07:26.818 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.814+0000 7fc5745ada80 20 get_obj_state: octx=0x55b2c7953c80 obj=check-bucket:d state=0x55b2c7ab40b8 s->prefetch_data=0 2026-02-24T22:07:26.818 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.814+0000 7fc5745ada80 20 get_obj_state: setting s->obj_tag to n34an9zcro68ptzsezsgatkndyai910d 2026-02-24T22:07:26.819 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.814+0000 7fc5745ada80 20 get_obj_state_impl: setting s->olh_tag to 2s7rr58r99aho24gkjda8rluwnp03haj 2026-02-24T22:07:26.819 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.814+0000 7fc5745ada80 20 bucket_index_guard_olh_op(): olh_state.olh_tag=2s7rr58r99aho24gkjda8rluwnp03haj 2026-02-24T22:07:26.820 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.818+0000 7fc5745ada80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:26.821 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.818+0000 7fc5745ada80 1 NOTICE: finished shard 0 (2 entries removed) 2026-02-24T22:07:26.821 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.818+0000 7fc5745ada80 1 NOTICE: finished all shards (2 entries removed) 2026-02-24T22:07:26.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.822+0000 7fc5745ada80 20 remove_watcher() i=1 2026-02-24T22:07:26.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.822+0000 7fc5745ada80 2 removed watcher, disabling cache 2026-02-24T22:07:26.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.822+0000 7fc5745ada80 20 remove_watcher() i=3 2026-02-24T22:07:26.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.822+0000 7fc5745ada80 20 remove_watcher() i=0 2026-02-24T22:07:26.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.822+0000 7fc5745ada80 20 remove_watcher() i=4 2026-02-24T22:07:26.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.822+0000 7fc5745ada80 20 remove_watcher() i=6 2026-02-24T22:07:26.825 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.822+0000 7fc5745ada80 20 remove_watcher() i=2 2026-02-24T22:07:26.825 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.822+0000 7fc5745ada80 20 remove_watcher() i=7 2026-02-24T22:07:26.825 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.822+0000 7fc5745ada80 20 remove_watcher() i=5 2026-02-24T22:07:26.829 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bi list --bucket check-bucket --object c 2026-02-24T22:07:26.845 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:07:26.845 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:07:26.862 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.858+0000 7f6e19253a80 4 Realm: () 2026-02-24T22:07:26.862 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.858+0000 7f6e19253a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:07:26.862 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.858+0000 7f6e19253a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:07:26.862 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.858+0000 7f6e19253a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:07:26.862 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.858+0000 7f6e19253a80 20 zonegroup default 2026-02-24T22:07:26.863 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.858+0000 7f6e19253a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:26.863 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.862+0000 7f6e19253a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:26.864 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.862+0000 7f6e19253a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:26.864 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.862+0000 7f6e19253a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:26.864 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.862+0000 7f6e19253a80 20 started sync module instance, tier type = 2026-02-24T22:07:26.864 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.862+0000 7f6e19253a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:07:26.866 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.862+0000 7f6e19253a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:07:26.866 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.862+0000 7f6e19253a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:07:26.866 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.862+0000 7f6e19253a80 5 note: GC not initialized 2026-02-24T22:07:26.866 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.862+0000 7f6dc57e2640 20 reqs_thread_entry: start 2026-02-24T22:07:26.870 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.866+0000 7f6e17528640 0 -- 10.20.193.46:0/2198218279 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x558a56ea6600 msgr2=0x558a56e9ffc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:26.876 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.874+0000 7f6e16d27640 0 -- 10.20.193.46:0/2198218279 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x558a56e7ac00 msgr2=0x558a56ea87d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:26.899 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.894+0000 7f6e19253a80 20 init_complete bucket index max shards: 1 2026-02-24T22:07:26.899 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.894+0000 7f6e19253a80 20 Filter name: none 2026-02-24T22:07:26.899 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.898+0000 7f6e19253a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:26.899 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.898+0000 7f6d2f7fe640 20 reqs_thread_entry: start 2026-02-24T22:07:26.900 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.898+0000 7f6e19253a80 20 rados_obj.operate() r=0 bl.length=196 2026-02-24T22:07:26.900 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.898+0000 7f6e19253a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:26.900 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.898+0000 7f6e19253a80 20 rados_obj.operate() r=0 bl.length=426 2026-02-24T22:07:26.900 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.898+0000 7f6e19253a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:07:26.900 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.898+0000 7f6e19253a80 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2026-02-24T22:07:26.900 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.898+0000 7f6e19253a80 20 INFO: main: starting shard=0 2026-02-24T22:07:26.900 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.898+0000 7f6e19253a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:26.901 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.898+0000 7f6e19253a80 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, next_marker= 2026-02-24T22:07:26.901 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.898+0000 7f6e19253a80 20 INFO: main: done 2026-02-24T22:07:26.908 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bi list --bucket check-bucket --object d 2026-02-24T22:07:26.924 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:07:26.924 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:07:26.942 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.938+0000 7f1336af8a80 4 Realm: () 2026-02-24T22:07:26.942 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.938+0000 7f1336af8a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:07:26.942 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.938+0000 7f1336af8a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:07:26.942 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.938+0000 7f1336af8a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:07:26.942 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.938+0000 7f1336af8a80 20 zonegroup default 2026-02-24T22:07:26.942 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.938+0000 7f1336af8a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:26.943 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.942+0000 7f1336af8a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:26.943 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.942+0000 7f1336af8a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:26.944 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.942+0000 7f1336af8a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:26.944 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.942+0000 7f1336af8a80 20 started sync module instance, tier type = 2026-02-24T22:07:26.944 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.942+0000 7f1336af8a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:07:26.946 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.942+0000 7f1336af8a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:07:26.946 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.942+0000 7f1336af8a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:07:26.946 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.942+0000 7f1336af8a80 5 note: GC not initialized 2026-02-24T22:07:26.946 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.942+0000 7f12e1fe3640 20 reqs_thread_entry: start 2026-02-24T22:07:26.949 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.946+0000 7f1334dcd640 0 -- 10.20.193.46:0/554217102 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x557c8ae56850 msgr2=0x557c8ae56c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:26.956 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.954+0000 7f132ffff640 0 -- 10.20.193.46:0/554217102 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x557c8ae59100 msgr2=0x557c8ae53e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:26.981 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.978+0000 7f1336af8a80 20 init_complete bucket index max shards: 1 2026-02-24T22:07:26.981 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.978+0000 7f1336af8a80 20 Filter name: none 2026-02-24T22:07:26.981 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.978+0000 7f1243fff640 20 reqs_thread_entry: start 2026-02-24T22:07:26.981 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.978+0000 7f1336af8a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:26.982 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.978+0000 7f1336af8a80 20 rados_obj.operate() r=0 bl.length=196 2026-02-24T22:07:26.982 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.978+0000 7f1336af8a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:26.982 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.978+0000 7f1336af8a80 20 rados_obj.operate() r=0 bl.length=426 2026-02-24T22:07:26.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.978+0000 7f1336af8a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:07:26.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.978+0000 7f1336af8a80 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2026-02-24T22:07:26.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.978+0000 7f1336af8a80 20 INFO: main: starting shard=0 2026-02-24T22:07:26.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.978+0000 7f1336af8a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:26.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.978+0000 7f1336af8a80 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, next_marker= 2026-02-24T22:07:26.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:26.978+0000 7f1336af8a80 20 INFO: main: done 2026-02-24T22:07:26.991 INFO:tasks.workunit.client.0.trial046.stderr:TEST: bucket check olh does not find new leftover OLH entries after fix 2026-02-24T22:07:26.991 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:26.991 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket check olh --bucket check-bucket --dump-keys 2026-02-24T22:07:27.008 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:07:27.008 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:07:27.026 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.022+0000 7f6ba9a7aa80 4 Realm: () 2026-02-24T22:07:27.026 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.022+0000 7f6ba9a7aa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:07:27.026 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.022+0000 7f6ba9a7aa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:07:27.026 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.022+0000 7f6ba9a7aa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:07:27.026 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.022+0000 7f6ba9a7aa80 20 zonegroup default 2026-02-24T22:07:27.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.226+0000 7f6ba9a7aa80 20 rados->read ofs=0 len=0 2026-02-24T22:07:27.229 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.226+0000 7f6ba9a7aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:27.229 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.226+0000 7f6ba9a7aa80 20 rados->read ofs=0 len=0 2026-02-24T22:07:27.229 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.226+0000 7f6ba9a7aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:27.230 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.226+0000 7f6ba9a7aa80 20 started sync module instance, tier type = 2026-02-24T22:07:27.230 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.226+0000 7f6ba9a7aa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:07:27.231 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.226+0000 7f6ba2ffd640 0 -- 10.20.193.46:0/2494585004 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x55ebc5a41e00 msgr2=0x55ebc5a62280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:27.231 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.230+0000 7f6ba9a7aa80 20 add_watcher() i=1 2026-02-24T22:07:27.232 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.230+0000 7f6ba9a7aa80 20 add_watcher() i=0 2026-02-24T22:07:27.232 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.230+0000 7f6ba9a7aa80 20 add_watcher() i=6 2026-02-24T22:07:27.232 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.230+0000 7f6ba9a7aa80 20 add_watcher() i=3 2026-02-24T22:07:27.232 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.230+0000 7f6ba9a7aa80 20 add_watcher() i=5 2026-02-24T22:07:27.233 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.230+0000 7f6ba9a7aa80 20 add_watcher() i=7 2026-02-24T22:07:27.233 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.230+0000 7f6ba9a7aa80 20 add_watcher() i=4 2026-02-24T22:07:27.233 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.230+0000 7f6ba9a7aa80 20 add_watcher() i=2 2026-02-24T22:07:27.233 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.230+0000 7f6ba9a7aa80 2 all 8 watchers are set, enabling cache 2026-02-24T22:07:27.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.230+0000 7f6ba27fc640 0 -- 10.20.193.46:0/124526427 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f6ac8008af0 msgr2=0x7f6ac8028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:27.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.234+0000 7f6ba9a7aa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:07:27.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.234+0000 7f6ba9a7aa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:07:27.236 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.234+0000 7f6ba9a7aa80 5 note: GC not initialized 2026-02-24T22:07:27.236 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.234+0000 7f6abffff640 20 reqs_thread_entry: start 2026-02-24T22:07:27.271 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.270+0000 7f6ba9a7aa80 20 init_complete bucket index max shards: 1 2026-02-24T22:07:27.271 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.270+0000 7f6ba9a7aa80 20 Filter name: none 2026-02-24T22:07:27.272 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.270+0000 7f6abe7fc640 20 reqs_thread_entry: start 2026-02-24T22:07:27.272 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.270+0000 7f6ba9a7aa80 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2026-02-24T22:07:27.272 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.270+0000 7f6ba9a7aa80 20 rados->read ofs=0 len=0 2026-02-24T22:07:27.273 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.270+0000 7f6ba9a7aa80 20 rados_obj.operate() r=0 bl.length=196 2026-02-24T22:07:27.273 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.270+0000 7f6ba9a7aa80 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2026-02-24T22:07:27.273 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.270+0000 7f6ba9a7aa80 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2026-02-24T22:07:27.273 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.270+0000 7f6ba9a7aa80 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 : miss 2026-02-24T22:07:27.273 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.270+0000 7f6ba9a7aa80 20 rados->read ofs=0 len=0 2026-02-24T22:07:27.273 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.270+0000 7f6ba9a7aa80 20 rados_obj.operate() r=0 bl.length=426 2026-02-24T22:07:27.273 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.270+0000 7f6ba9a7aa80 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 info.flags=0x17 2026-02-24T22:07:27.273 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.270+0000 7f6ba9a7aa80 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 to cache LRU end 2026-02-24T22:07:27.273 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.270+0000 7f6ba9a7aa80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:07:27.274 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.270+0000 7f6ba9a7aa80 10 updating xattr: name=user.rgw.acl bl.length()=179 2026-02-24T22:07:27.274 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.270+0000 7f6ba9a7aa80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 2026-02-24T22:07:27.274 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.270+0000 7f6ba9a7aa80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:27.274 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.270+0000 7f6ba9a7aa80 1 NOTICE: finished shard 0 (0 entries found) 2026-02-24T22:07:27.274 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.270+0000 7f6ba9a7aa80 1 NOTICE: finished all shards (0 entries found) 2026-02-24T22:07:27.276 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.270+0000 7f6ba9a7aa80 20 remove_watcher() i=3 2026-02-24T22:07:27.276 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.270+0000 7f6ba9a7aa80 2 removed watcher, disabling cache 2026-02-24T22:07:27.276 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.274+0000 7f6ba9a7aa80 20 remove_watcher() i=1 2026-02-24T22:07:27.276 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.274+0000 7f6ba9a7aa80 20 remove_watcher() i=6 2026-02-24T22:07:27.276 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.274+0000 7f6ba9a7aa80 20 remove_watcher() i=4 2026-02-24T22:07:27.276 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.274+0000 7f6ba9a7aa80 20 remove_watcher() i=2 2026-02-24T22:07:27.276 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.274+0000 7f6ba9a7aa80 20 remove_watcher() i=0 2026-02-24T22:07:27.276 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.274+0000 7f6ba9a7aa80 20 remove_watcher() i=5 2026-02-24T22:07:27.276 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:27.274+0000 7f6ba9a7aa80 20 remove_watcher() i=7 2026-02-24T22:07:27.281 INFO:tasks.workunit.client.0.trial046.stderr:TEST: verify that bucket check fixes do not affect null version objects 2026-02-24T22:07:27.281 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:27.283 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:27.283 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:27.286 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:27.286 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:27.288 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:27.288 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:27.332 INFO:tasks.workunit.client.0.trial046.stderr:TEST: bucket check stats are correct in the presence of unlinked entries 2026-02-24T22:07:27.332 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:27.333 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:27.333 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:27.377 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:27.377 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:27.424 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:27.424 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:27.433 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:27.433 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:27.436 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:27.436 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:27.445 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:27.445 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:27.448 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:27.448 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:27.457 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:27.457 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:27.459 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:27.459 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:27.469 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:27.469 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:27.472 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:27.472 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:27.481 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:27.481 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:27.483 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:27.483 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:27.493 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:27.494 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:27.496 INFO:tasks.workunit.client.0.trial046.stderr:ceph config set client rgw_debug_inject_set_olh_err 2 2026-02-24T22:07:27.705 INFO:tasks.workunit.client.0.trial046.stderr:ceph config set client rgw_debug_inject_olh_cancel_modification_err true 2026-02-24T22:07:28.914 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:28.914 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:28.963 INFO:tasks.workunit.client.0.trial046.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2026-02-24T22:07:28.964 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bi list --bucket check-bucket --object c 2026-02-24T22:07:28.980 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:07:28.980 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:07:28.996 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:28.994+0000 7f186a462a80 4 Realm: () 2026-02-24T22:07:28.996 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:28.994+0000 7f186a462a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:07:28.996 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:28.994+0000 7f186a462a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:07:28.996 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:28.994+0000 7f186a462a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:07:28.996 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:28.994+0000 7f186a462a80 20 zonegroup default 2026-02-24T22:07:28.997 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:28.994+0000 7f186a462a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:28.998 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:28.994+0000 7f186a462a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:28.998 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:28.994+0000 7f186a462a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:28.998 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:28.994+0000 7f186a462a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:28.999 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:28.994+0000 7f186a462a80 20 started sync module instance, tier type = 2026-02-24T22:07:28.999 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:28.994+0000 7f186a462a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:07:29.000 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:28.998+0000 7f186a462a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:07:29.000 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:28.998+0000 7f186a462a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:07:29.000 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:28.998+0000 7f186a462a80 5 note: GC not initialized 2026-02-24T22:07:29.001 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:28.998+0000 7f18157e2640 20 reqs_thread_entry: start 2026-02-24T22:07:29.005 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.002+0000 7f18637fe640 0 -- 10.20.193.46:0/1766692041 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x5597319c75d0 msgr2=0x5597319bee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:07:29.216 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.214+0000 7f1863fff640 0 -- 10.20.193.46:0/1766692041 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x5597319c0400 msgr2=0x5597319aff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:29.444 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.442+0000 7f186a462a80 20 init_complete bucket index max shards: 1 2026-02-24T22:07:29.444 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.442+0000 7f186a462a80 20 Filter name: none 2026-02-24T22:07:29.444 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.442+0000 7f177f7fe640 20 reqs_thread_entry: start 2026-02-24T22:07:29.444 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.442+0000 7f186a462a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:29.444 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.442+0000 7f186a462a80 20 rados_obj.operate() r=0 bl.length=196 2026-02-24T22:07:29.444 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.442+0000 7f186a462a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:29.445 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.442+0000 7f186a462a80 20 rados_obj.operate() r=0 bl.length=426 2026-02-24T22:07:29.445 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.442+0000 7f186a462a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:07:29.445 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.442+0000 7f186a462a80 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2026-02-24T22:07:29.445 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.442+0000 7f186a462a80 20 INFO: main: starting shard=0 2026-02-24T22:07:29.445 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.442+0000 7f186a462a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:29.446 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.442+0000 7f186a462a80 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, next_marker=�1001_c 2026-02-24T22:07:29.446 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.442+0000 7f186a462a80 20 INFO: main: done 2026-02-24T22:07:29.455 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:29.455 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:29.457 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:29.457 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:29.459 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:29.459 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:29.507 INFO:tasks.workunit.client.0.trial046.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2026-02-24T22:07:29.508 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bi list --bucket check-bucket --object d 2026-02-24T22:07:29.524 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:07:29.524 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:07:29.542 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.538+0000 7fb2f4e99a80 4 Realm: () 2026-02-24T22:07:29.542 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.538+0000 7fb2f4e99a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:07:29.542 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.538+0000 7fb2f4e99a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:07:29.542 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.538+0000 7fb2f4e99a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:07:29.542 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.538+0000 7fb2f4e99a80 20 zonegroup default 2026-02-24T22:07:29.542 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.538+0000 7fb2f4e99a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:29.544 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.542+0000 7fb2f4e99a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:29.544 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.542+0000 7fb2f4e99a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:29.544 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.542+0000 7fb2f4e99a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:29.544 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.542+0000 7fb2f4e99a80 20 started sync module instance, tier type = 2026-02-24T22:07:29.544 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.542+0000 7fb2f4e99a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:07:29.546 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.542+0000 7fb2f4e99a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:07:29.546 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.542+0000 7fb2f4e99a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:07:29.546 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.542+0000 7fb2f4e99a80 5 note: GC not initialized 2026-02-24T22:07:29.546 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.542+0000 7fb2a17e2640 20 reqs_thread_entry: start 2026-02-24T22:07:29.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.546+0000 7fb2f216c640 0 -- 10.20.193.46:0/969734166 >> [v2:10.20.193.46:6826/3065971976,v1:10.20.193.46:6827/3065971976] conn(0x55d800cc4a00 msgr2=0x55d800cc4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:29.759 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.754+0000 7fb2f216c640 0 -- 10.20.193.46:0/969734166 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x55d800cc30c0 msgr2=0x55d800cb02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:29.989 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.986+0000 7fb2f4e99a80 20 init_complete bucket index max shards: 1 2026-02-24T22:07:29.989 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.986+0000 7fb2f4e99a80 20 Filter name: none 2026-02-24T22:07:29.989 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.986+0000 7fb20b7fe640 20 reqs_thread_entry: start 2026-02-24T22:07:29.990 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.986+0000 7fb2f4e99a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:29.990 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.986+0000 7fb2f4e99a80 20 rados_obj.operate() r=0 bl.length=196 2026-02-24T22:07:29.990 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.986+0000 7fb2f4e99a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:29.990 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.986+0000 7fb2f4e99a80 20 rados_obj.operate() r=0 bl.length=426 2026-02-24T22:07:29.990 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.986+0000 7fb2f4e99a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:07:29.990 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.986+0000 7fb2f4e99a80 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2026-02-24T22:07:29.990 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.986+0000 7fb2f4e99a80 20 INFO: main: starting shard=0 2026-02-24T22:07:29.990 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.986+0000 7fb2f4e99a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:29.991 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.986+0000 7fb2f4e99a80 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, next_marker=�1001_d 2026-02-24T22:07:29.991 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:29.986+0000 7fb2f4e99a80 20 INFO: main: done 2026-02-24T22:07:30.000 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:30.000 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:30.003 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:30.003 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:30.051 INFO:tasks.workunit.client.0.trial046.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2026-02-24T22:07:30.052 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bi list --bucket check-bucket --object e 2026-02-24T22:07:30.067 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:07:30.068 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:07:30.086 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.082+0000 7f026a011a80 4 Realm: () 2026-02-24T22:07:30.086 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.082+0000 7f026a011a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:07:30.086 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.082+0000 7f026a011a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:07:30.086 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.082+0000 7f026a011a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:07:30.086 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.082+0000 7f026a011a80 20 zonegroup default 2026-02-24T22:07:30.087 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.082+0000 7f026a011a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:30.087 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.086+0000 7f026a011a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:30.087 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.086+0000 7f026a011a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:30.088 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.086+0000 7f026a011a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:30.088 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.086+0000 7f026a011a80 20 started sync module instance, tier type = 2026-02-24T22:07:30.088 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.086+0000 7f026a011a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:07:30.089 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.086+0000 7f0263fff640 0 -- 10.20.193.46:0/3592380295 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f0190008af0 msgr2=0x7f0190028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:30.090 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.086+0000 7f026a011a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:07:30.090 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.086+0000 7f026a011a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:07:30.090 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.086+0000 7f026a011a80 5 note: GC not initialized 2026-02-24T22:07:30.090 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.086+0000 7f02117e2640 20 reqs_thread_entry: start 2026-02-24T22:07:30.094 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.090+0000 7f0263fff640 0 -- 10.20.193.46:0/3592380295 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55caf1fa6d00 msgr2=0x55caf1f9d4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:07:30.123 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.118+0000 7f026a011a80 20 init_complete bucket index max shards: 1 2026-02-24T22:07:30.123 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.118+0000 7f026a011a80 20 Filter name: none 2026-02-24T22:07:30.123 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.118+0000 7f017b7fe640 20 reqs_thread_entry: start 2026-02-24T22:07:30.123 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.118+0000 7f026a011a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:30.123 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.118+0000 7f026a011a80 20 rados_obj.operate() r=0 bl.length=196 2026-02-24T22:07:30.123 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.118+0000 7f026a011a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:30.124 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.122+0000 7f026a011a80 20 rados_obj.operate() r=0 bl.length=426 2026-02-24T22:07:30.124 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.122+0000 7f026a011a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:07:30.124 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.122+0000 7f026a011a80 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2026-02-24T22:07:30.124 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.122+0000 7f026a011a80 20 INFO: main: starting shard=0 2026-02-24T22:07:30.124 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.122+0000 7f026a011a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:30.125 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.122+0000 7f026a011a80 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, next_marker=�1000_ei72ykNb8tOOdTZNc6XKei6oXxVhB4RKI 2026-02-24T22:07:30.125 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.122+0000 7f026a011a80 20 INFO: main: done 2026-02-24T22:07:30.133 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:30.133 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:30.136 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:30.136 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:30.183 INFO:tasks.workunit.client.0.trial046.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2026-02-24T22:07:30.184 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bi list --bucket check-bucket --object f 2026-02-24T22:07:30.200 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:07:30.200 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:07:30.218 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.214+0000 7fe7b857da80 4 Realm: () 2026-02-24T22:07:30.218 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.214+0000 7fe7b857da80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:07:30.218 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.214+0000 7fe7b857da80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:07:30.218 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.214+0000 7fe7b857da80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:07:30.218 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.214+0000 7fe7b857da80 20 zonegroup default 2026-02-24T22:07:30.420 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.418+0000 7fe7b857da80 20 rados->read ofs=0 len=0 2026-02-24T22:07:30.421 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.418+0000 7fe7b857da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:30.421 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.418+0000 7fe7b857da80 20 rados->read ofs=0 len=0 2026-02-24T22:07:30.422 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.418+0000 7fe7b857da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:30.422 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.418+0000 7fe7b857da80 20 started sync module instance, tier type = 2026-02-24T22:07:30.422 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.418+0000 7fe7b857da80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:07:30.424 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.422+0000 7fe7b857da80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:07:30.424 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.422+0000 7fe7b857da80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:07:30.424 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.422+0000 7fe7b857da80 5 note: GC not initialized 2026-02-24T22:07:30.424 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.422+0000 7fe6d7fff640 20 reqs_thread_entry: start 2026-02-24T22:07:30.424 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:30.422+0000 7fe7b6051640 0 -- 10.20.193.46:0/1812972127 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55a4f6cd41a0 msgr2=0x55a4f6d01d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:31.038 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.034+0000 7fe7b6051640 0 -- 10.20.193.46:0/1812972127 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55a4f6cfae80 msgr2=0x55a4f6cfb2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:31.273 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.270+0000 7fe7b857da80 20 init_complete bucket index max shards: 1 2026-02-24T22:07:31.274 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.270+0000 7fe7b857da80 20 Filter name: none 2026-02-24T22:07:31.274 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.270+0000 7fe6d67fc640 20 reqs_thread_entry: start 2026-02-24T22:07:31.274 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.270+0000 7fe7b857da80 20 rados->read ofs=0 len=0 2026-02-24T22:07:31.274 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.270+0000 7fe7b857da80 20 rados_obj.operate() r=0 bl.length=196 2026-02-24T22:07:31.274 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.270+0000 7fe7b857da80 20 rados->read ofs=0 len=0 2026-02-24T22:07:31.274 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.270+0000 7fe7b857da80 20 rados_obj.operate() r=0 bl.length=426 2026-02-24T22:07:31.274 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.270+0000 7fe7b857da80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:07:31.274 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.270+0000 7fe7b857da80 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2026-02-24T22:07:31.274 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.270+0000 7fe7b857da80 20 INFO: main: starting shard=0 2026-02-24T22:07:31.275 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.270+0000 7fe7b857da80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:31.275 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.270+0000 7fe7b857da80 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, next_marker=�1000_ficzRZvwUD1yFt7c.DGfo0Bf3VycY1aQn 2026-02-24T22:07:31.275 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.270+0000 7fe7b857da80 20 INFO: main: done 2026-02-24T22:07:31.284 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:31.284 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:31.286 INFO:tasks.workunit.client.0.trial046.stderr:ceph config rm client rgw_debug_inject_set_olh_err 2026-02-24T22:07:31.495 INFO:tasks.workunit.client.0.trial046.stderr:ceph config rm client rgw_debug_inject_olh_cancel_modification_err 2026-02-24T22:07:31.701 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket check --fix --bucket check-bucket 2026-02-24T22:07:31.716 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:07:31.716 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:07:31.733 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.730+0000 7f2d86a99a80 4 Realm: () 2026-02-24T22:07:31.733 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.730+0000 7f2d86a99a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:07:31.734 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.730+0000 7f2d86a99a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:07:31.734 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.730+0000 7f2d86a99a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:07:31.734 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.730+0000 7f2d86a99a80 20 zonegroup default 2026-02-24T22:07:31.734 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.730+0000 7f2d86a99a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:31.735 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.734+0000 7f2d86a99a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:31.735 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.734+0000 7f2d86a99a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:31.736 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.734+0000 7f2d86a99a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:31.736 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.734+0000 7f2d86a99a80 20 started sync module instance, tier type = 2026-02-24T22:07:31.736 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.734+0000 7f2d86a99a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:07:31.737 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.734+0000 7f2d84d6e640 0 -- 10.20.193.46:0/1857692041 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x558fc9edd7e0 msgr2=0x558fc9efdbc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:31.737 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.734+0000 7f2d86a99a80 20 add_watcher() i=1 2026-02-24T22:07:31.738 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.734+0000 7f2d86a99a80 20 add_watcher() i=6 2026-02-24T22:07:31.738 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.734+0000 7f2d86a99a80 20 add_watcher() i=5 2026-02-24T22:07:31.738 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.734+0000 7f2d86a99a80 20 add_watcher() i=3 2026-02-24T22:07:31.738 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.734+0000 7f2d86a99a80 20 add_watcher() i=2 2026-02-24T22:07:31.738 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.734+0000 7f2d86a99a80 20 add_watcher() i=7 2026-02-24T22:07:31.739 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.734+0000 7f2d86a99a80 20 add_watcher() i=4 2026-02-24T22:07:31.739 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.738+0000 7f2d86a99a80 20 add_watcher() i=0 2026-02-24T22:07:31.739 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.738+0000 7f2d86a99a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:07:31.741 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.738+0000 7f2d86a99a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:07:31.741 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.738+0000 7f2d86a99a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:07:31.741 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.738+0000 7f2d86a99a80 5 note: GC not initialized 2026-02-24T22:07:31.741 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.738+0000 7f2d31fdb640 20 reqs_thread_entry: start 2026-02-24T22:07:31.742 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.738+0000 7f2d84d6e640 0 -- 10.20.193.46:0/4240735395 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x558fca0153c0 msgr2=0x558fca015790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:31.981 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.978+0000 7f2d86a99a80 20 init_complete bucket index max shards: 1 2026-02-24T22:07:31.982 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.978+0000 7f2d86a99a80 20 Filter name: none 2026-02-24T22:07:31.982 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.978+0000 7f2c8bfff640 20 reqs_thread_entry: start 2026-02-24T22:07:31.982 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.978+0000 7f2d86a99a80 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2026-02-24T22:07:31.982 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.978+0000 7f2d86a99a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:31.982 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.978+0000 7f2d86a99a80 20 rados_obj.operate() r=0 bl.length=196 2026-02-24T22:07:31.982 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.978+0000 7f2d86a99a80 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2026-02-24T22:07:31.982 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.978+0000 7f2d86a99a80 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2026-02-24T22:07:31.982 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.978+0000 7f2d86a99a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 : miss 2026-02-24T22:07:31.982 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.978+0000 7f2d86a99a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:31.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.978+0000 7f2d86a99a80 20 rados_obj.operate() r=0 bl.length=426 2026-02-24T22:07:31.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.978+0000 7f2d86a99a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 info.flags=0x17 2026-02-24T22:07:31.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.978+0000 7f2d86a99a80 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 to cache LRU end 2026-02-24T22:07:31.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.978+0000 7f2d86a99a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:07:31.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.978+0000 7f2d86a99a80 10 updating xattr: name=user.rgw.acl bl.length()=179 2026-02-24T22:07:31.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.978+0000 7f2d86a99a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 2026-02-24T22:07:31.984 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.978+0000 7f2d86a99a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:31.989 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.986+0000 7f2d86a99a80 20 remove_watcher() i=3 2026-02-24T22:07:31.989 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.986+0000 7f2d86a99a80 2 removed watcher, disabling cache 2026-02-24T22:07:31.989 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.986+0000 7f2d86a99a80 20 remove_watcher() i=2 2026-02-24T22:07:31.989 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.986+0000 7f2d86a99a80 20 remove_watcher() i=4 2026-02-24T22:07:31.989 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.986+0000 7f2d86a99a80 20 remove_watcher() i=6 2026-02-24T22:07:31.989 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.986+0000 7f2d86a99a80 20 remove_watcher() i=5 2026-02-24T22:07:31.989 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.986+0000 7f2d86a99a80 20 remove_watcher() i=7 2026-02-24T22:07:31.989 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.986+0000 7f2d86a99a80 20 remove_watcher() i=1 2026-02-24T22:07:31.989 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:31.986+0000 7f2d86a99a80 20 remove_watcher() i=0 2026-02-24T22:07:31.996 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --fix --min-age-hours 0 --rgw-olh-pending-timeout-sec 0 --dump-keys 2026-02-24T22:07:32.014 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:07:32.014 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:07:32.031 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.030+0000 7fe31accea80 4 Realm: () 2026-02-24T22:07:32.031 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.030+0000 7fe31accea80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:07:32.032 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.030+0000 7fe31accea80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:07:32.032 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.030+0000 7fe31accea80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:07:32.032 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.030+0000 7fe31accea80 20 zonegroup default 2026-02-24T22:07:32.032 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.030+0000 7fe31accea80 20 rados->read ofs=0 len=0 2026-02-24T22:07:32.033 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.030+0000 7fe31accea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:32.033 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.030+0000 7fe31accea80 20 rados->read ofs=0 len=0 2026-02-24T22:07:32.236 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.234+0000 7fe31accea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:32.236 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.234+0000 7fe31accea80 20 started sync module instance, tier type = 2026-02-24T22:07:32.236 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.234+0000 7fe31accea80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:07:32.237 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.234+0000 7fe3137fe640 0 -- 10.20.193.46:0/1079458430 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55e35c42c6f0 msgr2=0x55e35c44cb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:32.238 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.234+0000 7fe31accea80 20 add_watcher() i=6 2026-02-24T22:07:32.238 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.234+0000 7fe31accea80 20 add_watcher() i=3 2026-02-24T22:07:32.239 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.234+0000 7fe31accea80 20 add_watcher() i=1 2026-02-24T22:07:32.239 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.234+0000 7fe31accea80 20 add_watcher() i=0 2026-02-24T22:07:32.239 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.234+0000 7fe31accea80 20 add_watcher() i=5 2026-02-24T22:07:32.239 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.234+0000 7fe31accea80 20 add_watcher() i=2 2026-02-24T22:07:32.240 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.234+0000 7fe31accea80 20 add_watcher() i=4 2026-02-24T22:07:32.240 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.234+0000 7fe31accea80 20 add_watcher() i=7 2026-02-24T22:07:32.240 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.234+0000 7fe31accea80 2 all 8 watchers are set, enabling cache 2026-02-24T22:07:32.240 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.238+0000 7fe3137fe640 0 -- 10.20.193.46:0/2043125303 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7fe238008af0 msgr2=0x7fe238028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:32.241 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.238+0000 7fe31accea80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:07:32.241 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.238+0000 7fe31accea80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:07:32.241 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.238+0000 7fe31accea80 5 note: GC not initialized 2026-02-24T22:07:32.242 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.238+0000 7fe2c5fdb640 20 reqs_thread_entry: start 2026-02-24T22:07:32.277 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.274+0000 7fe31accea80 20 init_complete bucket index max shards: 1 2026-02-24T22:07:32.277 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.274+0000 7fe31accea80 20 Filter name: none 2026-02-24T22:07:32.277 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.274+0000 7fe21ffff640 20 reqs_thread_entry: start 2026-02-24T22:07:32.277 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.274+0000 7fe31accea80 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2026-02-24T22:07:32.277 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.274+0000 7fe31accea80 20 rados->read ofs=0 len=0 2026-02-24T22:07:32.278 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.274+0000 7fe31accea80 20 rados_obj.operate() r=0 bl.length=196 2026-02-24T22:07:32.278 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.274+0000 7fe31accea80 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2026-02-24T22:07:32.278 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.274+0000 7fe31accea80 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2026-02-24T22:07:32.278 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.274+0000 7fe31accea80 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 : miss 2026-02-24T22:07:32.278 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.274+0000 7fe31accea80 20 rados->read ofs=0 len=0 2026-02-24T22:07:32.278 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.274+0000 7fe31accea80 20 rados_obj.operate() r=0 bl.length=426 2026-02-24T22:07:32.278 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.274+0000 7fe31accea80 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 info.flags=0x17 2026-02-24T22:07:32.279 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.274+0000 7fe31accea80 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 to cache LRU end 2026-02-24T22:07:32.279 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.274+0000 7fe31accea80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:07:32.279 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.274+0000 7fe31accea80 10 updating xattr: name=user.rgw.acl bl.length()=179 2026-02-24T22:07:32.279 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.274+0000 7fe31accea80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502 2026-02-24T22:07:32.279 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.274+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.280 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.274+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.280 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.274+0000 7fe31accea80 20 get_obj_state: octx=0x55e35c446d10 obj=check-bucket:c state=0x55e35c5118f8 s->prefetch_data=0 2026-02-24T22:07:32.280 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.274+0000 7fe31accea80 20 get_obj_state: setting s->obj_tag to lxwmqvuscet5lk0cun2s3e0i0d9ot9sh 2026-02-24T22:07:32.280 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.274+0000 7fe31accea80 20 get_obj_state_impl: setting s->olh_tag to 1mx9wnv7yx35y5mbc3nr995stgzn6eot 2026-02-24T22:07:32.280 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.274+0000 7fe31accea80 20 bucket_index_guard_olh_op(): olh_state.olh_tag=1mx9wnv7yx35y5mbc3nr995stgzn6eot 2026-02-24T22:07:32.280 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.278+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.278+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.278+0000 7fe31accea80 20 olh_log_entry: epoch=1771970852283740383 op=3 key=c[PVK7OJ1ObLKqs5lW5j4HM6xqpW2H50.] 2026-02-24T22:07:32.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.278+0000 7fe31accea80 20 get_obj_state: octx=0x55e35c446d10 obj=check-bucket:_:PVK7OJ1ObLKqs5lW5j4HM6xqpW2H50._c state=0x55e35c51b1d8 s->prefetch_data=0 2026-02-24T22:07:32.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.278+0000 7fe31accea80 10 manifest: total_size = 11 2026-02-24T22:07:32.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.278+0000 7fe31accea80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14130347534532188205 2026-02-24T22:07:32.282 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.278+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.283 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.278+0000 7fe31accea80 20 get_obj_state: octx=0x55e35c446d10 obj=check-bucket:_:PVK7OJ1ObLKqs5lW5j4HM6xqpW2H50._c state=0x55e35c51b1d8 s->prefetch_data=0 2026-02-24T22:07:32.283 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.278+0000 7fe31accea80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:07:32.283 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.278+0000 7fe31accea80 20 RGWObjManifest::operator++(): result: ofs=11 stripe_ofs=11 part_ofs=0 rule->part_size=0 2026-02-24T22:07:32.283 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.278+0000 7fe31accea80 0 deleting objects inline since gc isn't initialized 2026-02-24T22:07:32.283 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.282+0000 7fe2e6ffd640 20 handle_completion(): completion ok for obj=c[PVK7OJ1ObLKqs5lW5j4HM6xqpW2H50.] 2026-02-24T22:07:32.284 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.282+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.284 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.282+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.285 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.282+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.285 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.282+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.286 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.282+0000 7fe31accea80 20 get_obj_state: octx=0x7fe30803a8a0 obj=check-bucket:d state=0x55e35c5118f8 s->prefetch_data=0 2026-02-24T22:07:32.286 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.282+0000 7fe31accea80 20 get_obj_state: setting s->obj_tag to n5y9ira11wdfnllz6xrrj1epsjs2bn7r 2026-02-24T22:07:32.286 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.282+0000 7fe31accea80 20 get_obj_state_impl: setting s->olh_tag to xj52nc2nyt7dvzx30qfa9cevc5s4llcr 2026-02-24T22:07:32.286 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.282+0000 7fe31accea80 20 bucket_index_guard_olh_op(): olh_state.olh_tag=xj52nc2nyt7dvzx30qfa9cevc5s4llcr 2026-02-24T22:07:32.287 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.286+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.288 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.286+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.288 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.286+0000 7fe31accea80 20 olh_log_entry: epoch=1771970852291361597 op=3 key=d[S9vx7Ok8aQopLS0hcsk7grS0YqSngMQ] 2026-02-24T22:07:32.288 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.286+0000 7fe31accea80 20 get_obj_state: octx=0x7fe30803a8a0 obj=check-bucket:_:S9vx7Ok8aQopLS0hcsk7grS0YqSngMQ_d state=0x55e35c51a1c8 s->prefetch_data=0 2026-02-24T22:07:32.289 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.286+0000 7fe31accea80 10 manifest: total_size = 22 2026-02-24T22:07:32.289 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.286+0000 7fe31accea80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6362510996226314163 2026-02-24T22:07:32.289 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.286+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.290 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.286+0000 7fe31accea80 20 get_obj_state: octx=0x7fe30803a8a0 obj=check-bucket:_:S9vx7Ok8aQopLS0hcsk7grS0YqSngMQ_d state=0x55e35c51a1c8 s->prefetch_data=0 2026-02-24T22:07:32.290 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.286+0000 7fe31accea80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:07:32.290 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.286+0000 7fe31accea80 20 RGWObjManifest::operator++(): result: ofs=22 stripe_ofs=22 part_ofs=0 rule->part_size=0 2026-02-24T22:07:32.290 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.286+0000 7fe31accea80 0 deleting objects inline since gc isn't initialized 2026-02-24T22:07:32.291 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.286+0000 7fe2e6ffd640 20 handle_completion(): completion ok for obj=d[S9vx7Ok8aQopLS0hcsk7grS0YqSngMQ] 2026-02-24T22:07:32.292 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.290+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.292 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.290+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.292 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.290+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.293 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.290+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.293 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.290+0000 7fe31accea80 20 get_obj_state: octx=0x7fe30c100430 obj=check-bucket:e state=0x55e35c4711c8 s->prefetch_data=0 2026-02-24T22:07:32.293 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.290+0000 7fe31accea80 20 get_obj_state: setting s->obj_tag to xr809mrrvjz8foafr68cdtaalj3kd8tx 2026-02-24T22:07:32.294 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.290+0000 7fe31accea80 20 get_obj_state_impl: setting s->olh_tag to s0essyu0vd2wod9fc0z65y4ikodis805 2026-02-24T22:07:32.294 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.290+0000 7fe31accea80 20 bucket_index_guard_olh_op(): olh_state.olh_tag=s0essyu0vd2wod9fc0z65y4ikodis805 2026-02-24T22:07:32.294 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.290+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.295 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.294+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.295 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.294+0000 7fe31accea80 20 olh_log_entry: epoch=1771970852298469587 op=2 key=e[] 2026-02-24T22:07:32.296 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.294+0000 7fe31accea80 20 olh_log_entry: epoch=1771970852298469587 op=3 key=e[72ykNb8tOOdTZNc6XKei6oXxVhB4RKI] 2026-02-24T22:07:32.296 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.294+0000 7fe31accea80 20 get_obj_state: octx=0x7fe30c100430 obj=check-bucket:_:72ykNb8tOOdTZNc6XKei6oXxVhB4RKI_e state=0x55e35c512908 s->prefetch_data=0 2026-02-24T22:07:32.296 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.294+0000 7fe31accea80 10 manifest: total_size = 22 2026-02-24T22:07:32.296 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.294+0000 7fe31accea80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2884505299792630298 2026-02-24T22:07:32.296 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.294+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.294+0000 7fe31accea80 20 get_obj_state: octx=0x7fe30c100430 obj=check-bucket:_:72ykNb8tOOdTZNc6XKei6oXxVhB4RKI_e state=0x55e35c512908 s->prefetch_data=0 2026-02-24T22:07:32.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.294+0000 7fe31accea80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:07:32.298 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.294+0000 7fe31accea80 20 RGWObjManifest::operator++(): result: ofs=22 stripe_ofs=22 part_ofs=0 rule->part_size=0 2026-02-24T22:07:32.298 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.294+0000 7fe31accea80 0 deleting objects inline since gc isn't initialized 2026-02-24T22:07:32.298 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.294+0000 7fe2e6ffd640 20 handle_completion(): completion ok for obj=e[72ykNb8tOOdTZNc6XKei6oXxVhB4RKI] 2026-02-24T22:07:32.299 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.298+0000 7fe31accea80 20 get_obj_state: octx=0x7fe30c100430 obj=check-bucket:e state=0x55e35c4711c8 s->prefetch_data=0 2026-02-24T22:07:32.299 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.298+0000 7fe31accea80 20 bucket_index_guard_olh_op(): olh_state.olh_tag=s0essyu0vd2wod9fc0z65y4ikodis805 2026-02-24T22:07:32.300 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.298+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.298+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.298+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.298+0000 7fe31accea80 20 get_obj_state: octx=0x7fe308066510 obj=check-bucket:f state=0x55e35c4711c8 s->prefetch_data=0 2026-02-24T22:07:32.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.298+0000 7fe31accea80 20 get_obj_state: setting s->obj_tag to zz2m0gm9zkdxw0aq3z8a7fu334f0wsb1 2026-02-24T22:07:32.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.298+0000 7fe31accea80 20 get_obj_state_impl: setting s->olh_tag to gf2nxp6xghyaxctzscypnymsqtc8z63c 2026-02-24T22:07:32.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.298+0000 7fe31accea80 20 bucket_index_guard_olh_op(): olh_state.olh_tag=gf2nxp6xghyaxctzscypnymsqtc8z63c 2026-02-24T22:07:32.303 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.298+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.304 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.302+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.304 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.302+0000 7fe31accea80 20 olh_log_entry: epoch=1771970852307134290 op=2 key=f[] 2026-02-24T22:07:32.304 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.302+0000 7fe31accea80 20 olh_log_entry: epoch=1771970852307134290 op=3 key=f[czRZvwUD1yFt7c.DGfo0Bf3VycY1aQn] 2026-02-24T22:07:32.304 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.302+0000 7fe31accea80 20 get_obj_state: octx=0x7fe308066510 obj=check-bucket:_:czRZvwUD1yFt7c.DGfo0Bf3VycY1aQn_f state=0x55e35c512908 s->prefetch_data=0 2026-02-24T22:07:32.305 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.302+0000 7fe31accea80 10 manifest: total_size = 11 2026-02-24T22:07:32.305 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.302+0000 7fe31accea80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13315098622678760961 2026-02-24T22:07:32.305 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.302+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.302+0000 7fe31accea80 20 get_obj_state: octx=0x7fe308066510 obj=check-bucket:_:czRZvwUD1yFt7c.DGfo0Bf3VycY1aQn_f state=0x55e35c512908 s->prefetch_data=0 2026-02-24T22:07:32.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.302+0000 7fe31accea80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:07:32.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.302+0000 7fe31accea80 20 RGWObjManifest::operator++(): result: ofs=11 stripe_ofs=11 part_ofs=0 rule->part_size=0 2026-02-24T22:07:32.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.302+0000 7fe31accea80 0 deleting objects inline since gc isn't initialized 2026-02-24T22:07:32.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.302+0000 7fe2e77fe640 20 handle_completion(): completion ok for obj=f[czRZvwUD1yFt7c.DGfo0Bf3VycY1aQn] 2026-02-24T22:07:32.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.306+0000 7fe31accea80 20 get_obj_state: octx=0x7fe308066510 obj=check-bucket:f state=0x55e35c4711c8 s->prefetch_data=0 2026-02-24T22:07:32.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.306+0000 7fe31accea80 20 bucket_index_guard_olh_op(): olh_state.olh_tag=gf2nxp6xghyaxctzscypnymsqtc8z63c 2026-02-24T22:07:32.309 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.306+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.310 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.306+0000 7fe31accea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1502.0 2026-02-24T22:07:32.310 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.306+0000 7fe31accea80 1 NOTICE: finished shard 0 (4 entries removed) 2026-02-24T22:07:32.310 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.306+0000 7fe31accea80 1 NOTICE: finished all shards (4 entries removed) 2026-02-24T22:07:32.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.310+0000 7fe31accea80 20 remove_watcher() i=3 2026-02-24T22:07:32.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.310+0000 7fe31accea80 2 removed watcher, disabling cache 2026-02-24T22:07:32.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.310+0000 7fe31accea80 20 remove_watcher() i=2 2026-02-24T22:07:32.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.310+0000 7fe31accea80 20 remove_watcher() i=5 2026-02-24T22:07:32.314 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.310+0000 7fe31accea80 20 remove_watcher() i=1 2026-02-24T22:07:32.314 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.310+0000 7fe31accea80 20 remove_watcher() i=4 2026-02-24T22:07:32.314 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.310+0000 7fe31accea80 20 remove_watcher() i=6 2026-02-24T22:07:32.314 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.310+0000 7fe31accea80 20 remove_watcher() i=0 2026-02-24T22:07:32.314 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.310+0000 7fe31accea80 20 remove_watcher() i=7 2026-02-24T22:07:32.320 INFO:tasks.workunit.client.0.trial046.stderr:"bucket check unlinked" returned [{'name': 'c', 'instance': 'PVK7OJ1ObLKqs5lW5j4HM6xqpW2H50.'}, {'name': 'd', 'instance': 'S9vx7Ok8aQopLS0hcsk7grS0YqSngMQ'}, {'name': 'e', 'instance': '72ykNb8tOOdTZNc6XKei6oXxVhB4RKI'}, {'name': 'f', 'instance': 'czRZvwUD1yFt7c.DGfo0Bf3VycY1aQn'}], expecting ['c', 'd', 'e', 'f'] 2026-02-24T22:07:32.321 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:32.321 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:32.325 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:32.325 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:32.368 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket check-bucket 2026-02-24T22:07:32.383 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:07:32.383 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:07:32.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.398+0000 7ff6253d2a80 4 Realm: () 2026-02-24T22:07:32.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.398+0000 7ff6253d2a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:07:32.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.398+0000 7ff6253d2a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:07:32.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.398+0000 7ff6253d2a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:07:32.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.398+0000 7ff6253d2a80 20 zonegroup default 2026-02-24T22:07:32.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.398+0000 7ff6253d2a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:32.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.402+0000 7ff6253d2a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:32.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.402+0000 7ff6253d2a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:32.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.402+0000 7ff6253d2a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:07:32.405 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.402+0000 7ff6253d2a80 20 started sync module instance, tier type = 2026-02-24T22:07:32.405 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.402+0000 7ff6253d2a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:07:32.405 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.402+0000 7ff6253d2a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:07:32.406 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.402+0000 7ff6253d2a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:07:32.406 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.402+0000 7ff6253d2a80 5 note: GC not initialized 2026-02-24T22:07:32.406 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.402+0000 7ff5d17e2640 20 reqs_thread_entry: start 2026-02-24T22:07:32.409 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.406+0000 7ff6236a7640 0 -- 10.20.193.46:0/2091823158 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55cea42598a0 msgr2=0x55cea4260010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:32.415 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.410+0000 7ff622ea6640 0 -- 10.20.193.46:0/2091823158 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55cea425ba90 msgr2=0x55cea4256800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:07:32.441 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.438+0000 7ff6253d2a80 20 init_complete bucket index max shards: 1 2026-02-24T22:07:32.441 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.438+0000 7ff6253d2a80 20 Filter name: none 2026-02-24T22:07:32.441 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.438+0000 7ff53b7fe640 20 reqs_thread_entry: start 2026-02-24T22:07:32.441 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.438+0000 7ff6253d2a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:32.441 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.438+0000 7ff6253d2a80 20 rados_obj.operate() r=0 bl.length=196 2026-02-24T22:07:32.442 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.438+0000 7ff6253d2a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:32.442 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.438+0000 7ff6253d2a80 20 rados_obj.operate() r=0 bl.length=426 2026-02-24T22:07:32.442 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.438+0000 7ff6253d2a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:07:32.443 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.438+0000 7ff6253d2a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:32.443 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.438+0000 7ff6253d2a80 20 rados_obj.operate() r=0 bl.length=196 2026-02-24T22:07:32.443 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.438+0000 7ff6253d2a80 20 rados->read ofs=0 len=0 2026-02-24T22:07:32.443 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.438+0000 7ff6253d2a80 20 rados_obj.operate() r=0 bl.length=426 2026-02-24T22:07:32.443 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:07:32.438+0000 7ff6253d2a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:07:32.657 INFO:tasks.workunit.client.0.trial046.stderr:{'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}} 2026-02-24T22:07:32.657 INFO:tasks.workunit.client.0.trial046.stderr:Deleting bucket check-bucket 2026-02-24T22:07:32.658 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:32.658 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:32.661 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:07:32.661 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:07:32.672 INFO:tasks.workunit.client.0.trial046.stderr:Completed bucket check tests 2026-02-24T22:07:32.673 INFO:tasks.workunit.client.0.trial046.stdout:Could not connect to the endpoint URL: "http://localhost:80/" 2026-02-24T22:07:32.673 INFO:tasks.workunit.client.0.trial046.stdout:Could not connect to the endpoint URL: "http://localhost:8000/" 2026-02-24T22:07:32.673 INFO:tasks.workunit.client.0.trial046.stdout:connected to https://localhost:443 2026-02-24T22:07:32.715 INFO:tasks.workunit.client.0.trial046.stderr:+ deactivate 2026-02-24T22:07:32.715 INFO:tasks.workunit.client.0.trial046.stderr:+ '[' -n /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin ']' 2026-02-24T22:07:32.715 INFO:tasks.workunit.client.0.trial046.stderr:+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T22:07:32.715 INFO:tasks.workunit.client.0.trial046.stderr:+ export PATH 2026-02-24T22:07:32.716 INFO:tasks.workunit.client.0.trial046.stderr:+ unset _OLD_VIRTUAL_PATH 2026-02-24T22:07:32.716 INFO:tasks.workunit.client.0.trial046.stderr:+ '[' -n '' ']' 2026-02-24T22:07:32.716 INFO:tasks.workunit.client.0.trial046.stderr:+ '[' -n /usr/bin/bash -o -n '' ']' 2026-02-24T22:07:32.716 INFO:tasks.workunit.client.0.trial046.stderr:+ hash -r 2026-02-24T22:07:32.716 INFO:tasks.workunit.client.0.trial046.stderr:+ '[' -n '' ']' 2026-02-24T22:07:32.716 INFO:tasks.workunit.client.0.trial046.stderr:+ unset VIRTUAL_ENV 2026-02-24T22:07:32.716 INFO:tasks.workunit.client.0.trial046.stderr:+ unset VIRTUAL_ENV_PROMPT 2026-02-24T22:07:32.716 INFO:tasks.workunit.client.0.trial046.stderr:+ '[' '!' '' = nondestructive ']' 2026-02-24T22:07:32.716 INFO:tasks.workunit.client.0.trial046.stderr:+ unset -f deactivate 2026-02-24T22:07:32.716 INFO:tasks.workunit.client.0.trial046.stderr:+ echo OK. 2026-02-24T22:07:32.717 INFO:tasks.workunit.client.0.trial046.stdout:OK. 2026-02-24T22:07:32.718 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T22:07:32.718 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T22:07:32.771 INFO:tasks.workunit:Stopping ['rgw/run-bucket-check.sh'] on client.0... 2026-02-24T22:07:32.771 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-24T22:07:33.181 DEBUG:teuthology.parallel:result is None 2026-02-24T22:07:33.181 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-24T22:07:33.190 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-24T22:07:33.190 DEBUG:teuthology.orchestra.run.trial046:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-24T22:07:33.238 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-24T22:07:33.238 INFO:teuthology.run_tasks:Running task workunit... 2026-02-24T22:07:33.262 INFO:tasks.workunit:Pulling workunits from ref f7e05f4fdea5af173284f247d6b0935b337d4ccf 2026-02-24T22:07:33.265 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-24T22:07:33.266 DEBUG:teuthology.orchestra.run.trial046:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-24T22:07:33.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T22:07:33.284 INFO:teuthology.orchestra.run.trial046.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-24T22:07:33.285 DEBUG:teuthology.orchestra.run.trial046:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-24T22:07:33.331 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-24T22:07:33.331 DEBUG:teuthology.orchestra.run.trial046:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-24T22:07:33.375 INFO:tasks.workunit:timeout=3h 2026-02-24T22:07:33.375 INFO:tasks.workunit:cleanup=True 2026-02-24T22:07:33.376 DEBUG:teuthology.orchestra.run.trial046:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/samarahu/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout f7e05f4fdea5af173284f247d6b0935b337d4ccf 2026-02-24T22:07:33.419 INFO:tasks.workunit.client.0.trial046.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-24T22:07:55.599 INFO:tasks.workunit.client.0.trial046.stderr:Note: switching to 'f7e05f4fdea5af173284f247d6b0935b337d4ccf'. 2026-02-24T22:07:55.599 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:55.599 INFO:tasks.workunit.client.0.trial046.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-24T22:07:55.600 INFO:tasks.workunit.client.0.trial046.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-24T22:07:55.600 INFO:tasks.workunit.client.0.trial046.stderr:state without impacting any branches by switching back to a branch. 2026-02-24T22:07:55.600 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:55.600 INFO:tasks.workunit.client.0.trial046.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-24T22:07:55.600 INFO:tasks.workunit.client.0.trial046.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-24T22:07:55.600 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:55.600 INFO:tasks.workunit.client.0.trial046.stderr: git switch -c 2026-02-24T22:07:55.600 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:55.600 INFO:tasks.workunit.client.0.trial046.stderr:Or undo this operation with: 2026-02-24T22:07:55.600 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:55.600 INFO:tasks.workunit.client.0.trial046.stderr: git switch - 2026-02-24T22:07:55.601 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:55.601 INFO:tasks.workunit.client.0.trial046.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-24T22:07:55.601 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:07:55.601 INFO:tasks.workunit.client.0.trial046.stderr:HEAD is now at f7e05f4fdea rgw/d4n: Implement bucket check_empty and remove methods 2026-02-24T22:07:55.606 DEBUG:teuthology.orchestra.run.trial046:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-24T22:07:55.651 INFO:tasks.workunit.client.0.trial046.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2026-02-24T22:07:55.652 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-24T22:07:55.652 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2026-02-24T22:07:55.684 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2026-02-24T22:07:55.716 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2026-02-24T22:07:55.741 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-24T22:07:55.743 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-24T22:07:55.743 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2026-02-24T22:07:55.770 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-24T22:07:55.773 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:07:55.774 DEBUG:teuthology.orchestra.run.trial046:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-24T22:07:55.819 INFO:tasks.workunit:Running workunits matching cls/test_cls_lock.sh on client.0... 2026-02-24T22:07:55.822 INFO:tasks.workunit:Running workunit cls/test_cls_lock.sh... 2026-02-24T22:07:55.822 DEBUG:teuthology.orchestra.run.trial046:workunit test cls/test_cls_lock.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7e05f4fdea5af173284f247d6b0935b337d4ccf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_lock.sh 2026-02-24T22:07:55.876 INFO:tasks.workunit.client.0.trial046.stdout:Running main() from gmock_main.cc 2026-02-24T22:07:55.876 INFO:tasks.workunit.client.0.trial046.stdout:[==========] Running 11 tests from 1 test suite. 2026-02-24T22:07:55.876 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment set-up. 2026-02-24T22:07:55.876 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 11 tests from ClsLock 2026-02-24T22:07:55.876 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] ClsLock.TestMultiLocking 2026-02-24T22:07:57.750 INFO:tasks.workunit.client.0.trial046.stdout:lock: mylock 2026-02-24T22:07:57.751 INFO:tasks.workunit.client.0.trial046.stdout: lock_type: exclusive 2026-02-24T22:07:57.751 INFO:tasks.workunit.client.0.trial046.stdout: tag: 2026-02-24T22:07:57.751 INFO:tasks.workunit.client.0.trial046.stdout: lockers: 2026-02-24T22:07:57.751 INFO:tasks.workunit.client.0.trial046.stdout: client.5292 expiration=2026-02-24T22:09:57.751209+0000 addr=v1:10.20.193.46:0/927558226 cookie= 2026-02-24T22:07:57.753 INFO:tasks.workunit.client.0.trial046.stdout:lock: mylock 2026-02-24T22:07:57.753 INFO:tasks.workunit.client.0.trial046.stdout: lock_type: shared 2026-02-24T22:07:57.753 INFO:tasks.workunit.client.0.trial046.stdout: tag: 2026-02-24T22:07:57.753 INFO:tasks.workunit.client.0.trial046.stdout: lockers: 2026-02-24T22:07:57.753 INFO:tasks.workunit.client.0.trial046.stdout: client.5111 expiration=0.000000 addr=v1:10.20.193.46:0/935821959 cookie= 2026-02-24T22:07:57.753 INFO:tasks.workunit.client.0.trial046.stdout: client.5292 expiration=2026-02-24T22:09:57.755356+0000 addr=v1:10.20.193.46:0/927558226 cookie= 2026-02-24T22:07:57.754 INFO:tasks.workunit.client.0.trial046.stdout:lock: mylock 2026-02-24T22:07:57.754 INFO:tasks.workunit.client.0.trial046.stdout: lock_type: shared 2026-02-24T22:07:57.754 INFO:tasks.workunit.client.0.trial046.stdout: tag: 2026-02-24T22:07:57.754 INFO:tasks.workunit.client.0.trial046.stdout: lockers: 2026-02-24T22:07:57.754 INFO:tasks.workunit.client.0.trial046.stdout: client.5111 expiration=0.000000 addr=v1:10.20.193.46:0/935821959 cookie= 2026-02-24T22:07:58.733 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] ClsLock.TestMultiLocking (2856 ms) 2026-02-24T22:07:58.733 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] ClsLock.TestMeta 2026-02-24T22:08:00.759 INFO:tasks.workunit.client.0.trial046.stdout:lock: mylock 2026-02-24T22:08:00.759 INFO:tasks.workunit.client.0.trial046.stdout: lock_type: shared 2026-02-24T22:08:00.759 INFO:tasks.workunit.client.0.trial046.stdout: tag: 2026-02-24T22:08:00.759 INFO:tasks.workunit.client.0.trial046.stdout: lockers: 2026-02-24T22:08:00.759 INFO:tasks.workunit.client.0.trial046.stdout: client.5117 expiration=0.000000 addr=v1:10.20.193.46:0/2263577777 cookie= 2026-02-24T22:08:00.761 INFO:tasks.workunit.client.0.trial046.stdout:lock: mylock 2026-02-24T22:08:00.761 INFO:tasks.workunit.client.0.trial046.stdout: lock_type: exclusive 2026-02-24T22:08:00.761 INFO:tasks.workunit.client.0.trial046.stdout: tag: new_tag 2026-02-24T22:08:00.761 INFO:tasks.workunit.client.0.trial046.stdout: lockers: 2026-02-24T22:08:00.761 INFO:tasks.workunit.client.0.trial046.stdout: client.5304 expiration=0.000000 addr=v1:10.20.193.46:0/872333057 cookie= 2026-02-24T22:08:01.747 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] ClsLock.TestMeta (3014 ms) 2026-02-24T22:08:01.748 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] ClsLock.TestCookie 2026-02-24T22:08:03.760 INFO:tasks.workunit.client.0.trial046.stdout:lock: mylock 2026-02-24T22:08:03.760 INFO:tasks.workunit.client.0.trial046.stdout: lock_type: exclusive 2026-02-24T22:08:03.760 INFO:tasks.workunit.client.0.trial046.stdout: tag: 2026-02-24T22:08:03.760 INFO:tasks.workunit.client.0.trial046.stdout: lockers: 2026-02-24T22:08:03.761 INFO:tasks.workunit.client.0.trial046.stdout:lock: mylock 2026-02-24T22:08:03.761 INFO:tasks.workunit.client.0.trial046.stdout: lock_type: shared 2026-02-24T22:08:03.761 INFO:tasks.workunit.client.0.trial046.stdout: tag: 2026-02-24T22:08:03.761 INFO:tasks.workunit.client.0.trial046.stdout: lockers: 2026-02-24T22:08:03.761 INFO:tasks.workunit.client.0.trial046.stdout: client.5316 expiration=0.000000 addr=v1:10.20.193.46:0/1257115429 cookie= 2026-02-24T22:08:03.761 INFO:tasks.workunit.client.0.trial046.stdout: client.5316 expiration=0.000000 addr=v1:10.20.193.46:0/1257115429 cookie=new cookie 2026-02-24T22:08:04.756 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] ClsLock.TestCookie (3008 ms) 2026-02-24T22:08:04.757 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] ClsLock.TestMultipleLocks 2026-02-24T22:08:07.769 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] ClsLock.TestMultipleLocks (3012 ms) 2026-02-24T22:08:07.770 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] ClsLock.TestLockDuration 2026-02-24T22:08:14.839 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] ClsLock.TestLockDuration (7069 ms) 2026-02-24T22:08:14.840 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] ClsLock.TestAssertLocked 2026-02-24T22:08:17.853 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] ClsLock.TestAssertLocked (3013 ms) 2026-02-24T22:08:17.853 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] ClsLock.TestSetCookie 2026-02-24T22:08:20.863 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] ClsLock.TestSetCookie (3009 ms) 2026-02-24T22:08:20.863 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] ClsLock.TestRenew 2026-02-24T22:08:40.941 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] ClsLock.TestRenew (20078 ms) 2026-02-24T22:08:40.941 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralBasic 2026-02-24T22:08:47.012 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] ClsLock.TestExclusiveEphemeralBasic (6071 ms) 2026-02-24T22:08:47.013 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealEphemeral 2026-02-24T22:08:54.731 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealEphemeral (7718 ms) 2026-02-24T22:08:54.732 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealExclusive 2026-02-24T22:09:02.730 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealExclusive (7998 ms) 2026-02-24T22:09:02.730 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 11 tests from ClsLock (66854 ms total) 2026-02-24T22:09:02.730 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:09:02.730 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment tear-down 2026-02-24T22:09:02.730 INFO:tasks.workunit.client.0.trial046.stdout:[==========] 11 tests from 1 test suite ran. (66854 ms total) 2026-02-24T22:09:02.730 INFO:tasks.workunit.client.0.trial046.stdout:[ PASSED ] 11 tests. 2026-02-24T22:09:02.734 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T22:09:02.734 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T22:09:02.787 INFO:tasks.workunit:Running workunits matching cls/test_cls_log.sh on client.0... 2026-02-24T22:09:02.789 INFO:tasks.workunit:Running workunit cls/test_cls_log.sh... 2026-02-24T22:09:02.789 DEBUG:teuthology.orchestra.run.trial046:workunit test cls/test_cls_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7e05f4fdea5af173284f247d6b0935b337d4ccf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_log.sh 2026-02-24T22:09:02.847 INFO:tasks.workunit.client.0.trial046.stdout:Running main() from gmock_main.cc 2026-02-24T22:09:02.848 INFO:tasks.workunit.client.0.trial046.stdout:[==========] Running 4 tests from 1 test suite. 2026-02-24T22:09:02.848 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment set-up. 2026-02-24T22:09:02.848 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 4 tests from cls_log 2026-02-24T22:09:02.848 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_log.test_log_add_same_time 2026-02-24T22:09:05.742 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_log.test_log_add_same_time (2894 ms) 2026-02-24T22:09:05.742 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_log.test_log_add_different_time 2026-02-24T22:09:08.753 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_log.test_log_add_different_time (3010 ms) 2026-02-24T22:09:08.753 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_log.trim_by_time 2026-02-24T22:09:11.770 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_log.trim_by_time (3017 ms) 2026-02-24T22:09:11.770 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_log.trim_by_marker 2026-02-24T22:09:14.781 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_log.trim_by_marker (3011 ms) 2026-02-24T22:09:14.781 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 4 tests from cls_log (11933 ms total) 2026-02-24T22:09:14.781 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:09:14.781 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment tear-down 2026-02-24T22:09:14.781 INFO:tasks.workunit.client.0.trial046.stdout:[==========] 4 tests from 1 test suite ran. (11933 ms total) 2026-02-24T22:09:14.781 INFO:tasks.workunit.client.0.trial046.stdout:[ PASSED ] 4 tests. 2026-02-24T22:09:14.784 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T22:09:14.784 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T22:09:14.834 INFO:tasks.workunit:Running workunits matching cls/test_cls_refcount.sh on client.0... 2026-02-24T22:09:14.837 INFO:tasks.workunit:Running workunit cls/test_cls_refcount.sh... 2026-02-24T22:09:14.837 DEBUG:teuthology.orchestra.run.trial046:workunit test cls/test_cls_refcount.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7e05f4fdea5af173284f247d6b0935b337d4ccf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_refcount.sh 2026-02-24T22:09:14.891 INFO:tasks.workunit.client.0.trial046.stdout:Running main() from gmock_main.cc 2026-02-24T22:09:14.891 INFO:tasks.workunit.client.0.trial046.stdout:[==========] Running 10 tests from 1 test suite. 2026-02-24T22:09:14.891 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment set-up. 2026-02-24T22:09:14.891 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 10 tests from cls_refcount 2026-02-24T22:09:14.892 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_refcount.test_implicit 2026-02-24T22:09:17.797 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_refcount.test_implicit (2905 ms) 2026-02-24T22:09:17.797 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_refcount.test_implicit_idempotent 2026-02-24T22:09:20.809 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_refcount.test_implicit_idempotent (3011 ms) 2026-02-24T22:09:20.809 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_refcount.test_put_snap 2026-02-24T22:09:29.839 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_refcount.test_put_snap (9030 ms) 2026-02-24T22:09:29.840 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_refcount.test_explicit 2026-02-24T22:09:32.853 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_refcount.test_explicit (3013 ms) 2026-02-24T22:09:32.854 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_refcount.set 2026-02-24T22:09:35.867 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_refcount.set (3014 ms) 2026-02-24T22:09:35.868 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_refcount.test_implicit_ec 2026-02-24T22:09:42.879 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_refcount.test_implicit_ec (7012 ms) 2026-02-24T22:09:42.880 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_refcount.test_implicit_idempotent_ec 2026-02-24T22:09:49.892 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_refcount.test_implicit_idempotent_ec (7012 ms) 2026-02-24T22:09:49.892 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_refcount.test_put_snap_ec 2026-02-24T22:10:01.983 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_refcount.test_put_snap_ec (12090 ms) 2026-02-24T22:10:01.983 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_refcount.test_explicit_ec 2026-02-24T22:10:09.010 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_refcount.test_explicit_ec (7027 ms) 2026-02-24T22:10:09.010 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_refcount.set_ec 2026-02-24T22:10:16.030 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_refcount.set_ec (7020 ms) 2026-02-24T22:10:16.030 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 10 tests from cls_refcount (61139 ms total) 2026-02-24T22:10:16.031 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:10:16.031 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment tear-down 2026-02-24T22:10:16.031 INFO:tasks.workunit.client.0.trial046.stdout:[==========] 10 tests from 1 test suite ran. (61139 ms total) 2026-02-24T22:10:16.031 INFO:tasks.workunit.client.0.trial046.stdout:[ PASSED ] 10 tests. 2026-02-24T22:10:16.034 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T22:10:16.034 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T22:10:16.086 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw.sh on client.0... 2026-02-24T22:10:16.089 INFO:tasks.workunit:Running workunit cls/test_cls_rgw.sh... 2026-02-24T22:10:16.089 DEBUG:teuthology.orchestra.run.trial046:workunit test cls/test_cls_rgw.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7e05f4fdea5af173284f247d6b0935b337d4ccf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh 2026-02-24T22:10:16.147 INFO:tasks.workunit.client.0.trial046.stdout:Running main() from gmock_main.cc 2026-02-24T22:10:16.147 INFO:tasks.workunit.client.0.trial046.stdout:[==========] Running 20 tests from 1 test suite. 2026-02-24T22:10:16.147 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment set-up. 2026-02-24T22:10:16.147 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 20 tests from cls_rgw 2026-02-24T22:10:18.040 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw.index_basic 2026-02-24T22:10:18.060 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw.index_basic (19 ms) 2026-02-24T22:10:18.060 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw.index_multiple_obj_writers 2026-02-24T22:10:18.080 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw.index_multiple_obj_writers (20 ms) 2026-02-24T22:10:18.080 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw.index_remove_object 2026-02-24T22:10:18.104 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw.index_remove_object (24 ms) 2026-02-24T22:10:18.104 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw.index_suggest 2026-02-24T22:10:19.419 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw.index_suggest (1314 ms) 2026-02-24T22:10:19.419 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw.index_suggest_complete 2026-02-24T22:10:19.423 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw.index_suggest_complete (4 ms) 2026-02-24T22:10:19.424 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw.index_list 2026-02-24T22:10:19.437 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw.index_list (13 ms) 2026-02-24T22:10:19.437 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw.index_list_delimited 2026-02-24T22:10:31.018 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw.index_list_delimited (11580 ms) 2026-02-24T22:10:31.018 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw.bi_list 2026-02-24T22:10:31.066 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw.bi_list (47 ms) 2026-02-24T22:10:31.066 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw.gc_set 2026-02-24T22:10:31.073 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw.gc_set (7 ms) 2026-02-24T22:10:31.073 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw.gc_list 2026-02-24T22:10:31.080 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw.gc_list (6 ms) 2026-02-24T22:10:31.080 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw.gc_defer 2026-02-24T22:10:37.224 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw.gc_defer (6144 ms) 2026-02-24T22:10:37.225 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw.usage_key_transition 2026-02-24T22:10:37.446 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw.usage_key_transition (221 ms) 2026-02-24T22:10:37.446 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw.usage_basic 2026-02-24T22:10:37.525 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw.usage_basic (79 ms) 2026-02-24T22:10:37.525 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw.usage_clear_no_obj 2026-02-24T22:10:37.526 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw.usage_clear_no_obj (0 ms) 2026-02-24T22:10:37.526 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw.usage_clear 2026-02-24T22:10:37.616 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw.usage_clear (90 ms) 2026-02-24T22:10:37.616 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw.bi_log_trim 2026-02-24T22:10:37.639 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw.bi_log_trim (23 ms) 2026-02-24T22:10:37.639 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw.index_racing_removes 2026-02-24T22:10:37.650 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw.index_racing_removes (10 ms) 2026-02-24T22:10:37.650 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw.reshardlog_list 2026-02-24T22:10:37.659 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw.reshardlog_list (8 ms) 2026-02-24T22:10:37.659 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw.reshardlog_num 2026-02-24T22:10:37.665 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw.reshardlog_num (6 ms) 2026-02-24T22:10:37.665 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw.bi_put_entries 2026-02-24T22:10:37.679 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw.bi_put_entries (14 ms) 2026-02-24T22:10:38.228 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 20 tests from cls_rgw (19639 ms total) 2026-02-24T22:10:38.228 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:10:38.228 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment tear-down 2026-02-24T22:10:38.229 INFO:tasks.workunit.client.0.trial046.stdout:[==========] 20 tests from 1 test suite ran. (22081 ms total) 2026-02-24T22:10:38.229 INFO:tasks.workunit.client.0.trial046.stdout:[ PASSED ] 20 tests. 2026-02-24T22:10:38.232 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T22:10:38.232 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T22:10:38.282 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw_gc.sh on client.0... 2026-02-24T22:10:38.285 INFO:tasks.workunit:Running workunit cls/test_cls_rgw_gc.sh... 2026-02-24T22:10:38.285 DEBUG:teuthology.orchestra.run.trial046:workunit test cls/test_cls_rgw_gc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7e05f4fdea5af173284f247d6b0935b337d4ccf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw_gc.sh 2026-02-24T22:10:38.339 INFO:tasks.workunit.client.0.trial046.stdout:Running main() from gmock_main.cc 2026-02-24T22:10:38.339 INFO:tasks.workunit.client.0.trial046.stdout:[==========] Running 9 tests from 1 test suite. 2026-02-24T22:10:38.339 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment set-up. 2026-02-24T22:10:38.339 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 9 tests from cls_rgw_gc 2026-02-24T22:10:38.339 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw_gc.init 2026-02-24T22:10:40.236 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw_gc.init (1897 ms) 2026-02-24T22:10:40.237 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops1 2026-02-24T22:10:40.244 INFO:tasks.workunit.client.0.trial046.stderr:[ ] list info tag = chain-0 2026-02-24T22:10:40.244 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw_gc.gc_queue_ops1 (7 ms) 2026-02-24T22:10:40.244 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops2 2026-02-24T22:10:40.247 INFO:tasks.workunit.client.0.trial046.stderr:[ ] next_marker is: = 0/4263 2026-02-24T22:10:40.248 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw_gc.gc_queue_ops2 (3 ms) 2026-02-24T22:10:40.248 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops5 2026-02-24T22:10:40.252 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw_gc.gc_queue_ops5 (4 ms) 2026-02-24T22:10:40.252 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops6 2026-02-24T22:10:41.131 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw_gc.gc_queue_ops6 (879 ms) 2026-02-24T22:10:41.132 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops7 2026-02-24T22:10:41.137 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw_gc.gc_queue_ops7 (5 ms) 2026-02-24T22:10:41.137 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops8 2026-02-24T22:10:41.142 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw_gc.gc_queue_ops8 (4 ms) 2026-02-24T22:10:41.142 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops9 2026-02-24T22:10:41.146 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw_gc.gc_queue_ops9 (4 ms) 2026-02-24T22:10:41.146 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw_gc.finalize 2026-02-24T22:10:41.239 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw_gc.finalize (92 ms) 2026-02-24T22:10:41.239 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 9 tests from cls_rgw_gc (2899 ms total) 2026-02-24T22:10:41.239 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:10:41.239 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment tear-down 2026-02-24T22:10:41.239 INFO:tasks.workunit.client.0.trial046.stdout:[==========] 9 tests from 1 test suite ran. (2899 ms total) 2026-02-24T22:10:41.239 INFO:tasks.workunit.client.0.trial046.stdout:[ PASSED ] 9 tests. 2026-02-24T22:10:41.242 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T22:10:41.242 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T22:10:41.294 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw_stats.sh on client.0... 2026-02-24T22:10:41.296 INFO:tasks.workunit:Running workunit cls/test_cls_rgw_stats.sh... 2026-02-24T22:10:41.297 DEBUG:teuthology.orchestra.run.trial046:workunit test cls/test_cls_rgw_stats.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7e05f4fdea5af173284f247d6b0935b337d4ccf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw_stats.sh 2026-02-24T22:10:41.354 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:10:41.354 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:10:41.355 INFO:tasks.workunit.client.0.trial046.stdout:[==========] Running 1 test from 1 test suite. 2026-02-24T22:10:41.355 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment set-up. 2026-02-24T22:10:43.248 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 1 test from cls_rgw_stats 2026-02-24T22:10:43.248 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_rgw_stats.simulate 2026-02-24T22:10:43.252 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.249+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=U02ON5ATGES8 type=del size=3438742 2026-02-24T22:10:43.253 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.249+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=A0UMWITLDIS5 type=del size=1676688 2026-02-24T22:10:43.254 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.253+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=NPG2ISQLJW6A type=del size=3558231 2026-02-24T22:10:43.254 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.253+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=Q397MCP0F3KW type=del size=791774 2026-02-24T22:10:43.256 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.253+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~VTHQU92VQTYU.0 size=1048576 2026-02-24T22:10:43.256 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.253+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~VTHQU92VQTYU.1 size=1009638 2026-02-24T22:10:43.256 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.253+0000 7f7e64854980 1 > prepared multipart upload key=4Y3974PYYT78 upload=VTHQU92VQTYU tag=UKYLGOSXMHK0 type=write size=3106790 2026-02-24T22:10:43.257 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.253+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=Z21BBVOR2K6I type=del size=3875695 2026-02-24T22:10:43.258 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.257+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~AQQQMV2WHHQ4.0 size=1048576 2026-02-24T22:10:43.259 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.257+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~AQQQMV2WHHQ4.1 size=968739 2026-02-24T22:10:43.259 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.257+0000 7f7e64854980 1 > prepared multipart upload key=78GUTC5THI2P upload=AQQQMV2WHHQ4 tag=BAHM4OU4H7OK type=write size=3065891 2026-02-24T22:10:43.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.257+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=M0WZ0KFQCEPH type=del size=621777 2026-02-24T22:10:43.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.257+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=YB92ZA11KARA type=del size=729680 2026-02-24T22:10:43.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.261+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~GO6Z0SPMB1DY.0 size=706264 2026-02-24T22:10:43.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.261+0000 7f7e64854980 1 > prepared multipart upload key=N3ZUNSI5K8HS upload=GO6Z0SPMB1DY tag=7DJA9O1L83EN type=write size=1754840 2026-02-24T22:10:43.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.261+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=BW3CZQ0EJAKJ type=del size=332439 2026-02-24T22:10:43.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.261+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~Y68BGTFM86RW.0 size=610687 2026-02-24T22:10:43.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.261+0000 7f7e64854980 1 > prepared multipart upload key=4Y3974PYYT78 upload=Y68BGTFM86RW tag=WCE3DO4X2Z9R type=write size=1659263 2026-02-24T22:10:43.265 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.261+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~FADQO352YN88.0 size=1048576 2026-02-24T22:10:43.265 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.265+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~FADQO352YN88.1 size=1048576 2026-02-24T22:10:43.266 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.265+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~FADQO352YN88.2 size=84447 2026-02-24T22:10:43.266 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.265+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=FADQO352YN88 tag=N2PC2QG3H3WB type=write size=3230175 2026-02-24T22:10:43.266 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.265+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=EN41SUFXINI1 type=del size=254778 2026-02-24T22:10:43.267 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.265+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=KWH4QR9ITQ3U type=del size=3305804 2026-02-24T22:10:43.268 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.265+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=GK9BM16ZWXIU type=del size=3821870 2026-02-24T22:10:43.268 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.265+0000 7f7e64854980 1 < completed delete operation key=PKRHWM69GBJI tag=U02ON5ATGES8 type=del 2026-02-24T22:10:43.269 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.265+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=HP7WZ4POBSF6 type=del size=457879 2026-02-24T22:10:43.270 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.269+0000 7f7e64854980 1 < suggested update operation key=EJCX06TE5RTZ tag=A0UMWITLDIS5 type=del size=1676688 2026-02-24T22:10:43.271 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.269+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=TYB96IPHH66P type=write size=804180 2026-02-24T22:10:43.272 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.269+0000 7f7e64854980 1 < suggested update operation key=2KKQKHXO7K3K tag=NPG2ISQLJW6A type=del size=3558231 2026-02-24T22:10:43.273 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.273+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~I7SYCV25AI1X.0 size=445237 2026-02-24T22:10:43.274 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.273+0000 7f7e64854980 1 > prepared multipart upload key=UQLUT9ZAI61V upload=I7SYCV25AI1X tag=1NB9AC84U6FM type=write size=1493813 2026-02-24T22:10:43.275 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.273+0000 7f7e64854980 1 < suggested update operation key=CI8DA17DX8LU tag=Q397MCP0F3KW type=del size=791774 2026-02-24T22:10:43.276 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.273+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~5QYD9C5X7TEQ.0 size=1025028 2026-02-24T22:10:43.276 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.273+0000 7f7e64854980 1 > prepared multipart upload key=A6PH77Q78VEN upload=5QYD9C5X7TEQ tag=U2JMAEL8S58K type=write size=2073604 2026-02-24T22:10:43.276 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.273+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~VTHQU92VQTYU.0 size=1048576 2026-02-24T22:10:43.277 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.273+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~VTHQU92VQTYU.1 size=1009638 2026-02-24T22:10:43.278 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.277+0000 7f7e64854980 1 < completed multipart upload key=4Y3974PYYT78 upload=VTHQU92VQTYU tag=UKYLGOSXMHK0 type=write size=3106790 2026-02-24T22:10:43.279 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.277+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=YT6TL2WA5BHB type=del size=1999861 2026-02-24T22:10:43.279 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.277+0000 7f7e64854980 1 < completed delete operation key=L60HQ7M3E3LL tag=Z21BBVOR2K6I type=del 2026-02-24T22:10:43.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.277+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~RIGM88Z3JQ21.0 size=1048576 2026-02-24T22:10:43.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.277+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~RIGM88Z3JQ21.1 size=797289 2026-02-24T22:10:43.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.277+0000 7f7e64854980 1 > prepared multipart upload key=Q7BP8FXTARDW upload=RIGM88Z3JQ21 tag=71V6EGBCJXA0 type=write size=2894441 2026-02-24T22:10:43.282 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.281+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~AQQQMV2WHHQ4.0 size=1048576 2026-02-24T22:10:43.282 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.281+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~AQQQMV2WHHQ4.1 size=968739 2026-02-24T22:10:43.283 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.281+0000 7f7e64854980 1 < completed multipart upload key=78GUTC5THI2P upload=AQQQMV2WHHQ4 tag=BAHM4OU4H7OK type=write size=3065891 2026-02-24T22:10:43.284 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.281+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=Z65UCL7NGDRY type=del size=1189826 2026-02-24T22:10:43.285 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.281+0000 7f7e64854980 -1 < failed to complete operation key=CI8DA17DX8LU tag=M0WZ0KFQCEPH type=del size=621777: (22) Invalid argument 2026-02-24T22:10:43.286 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.285+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~UQEGUKAGGWMW.0 size=1045862 2026-02-24T22:10:43.286 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.285+0000 7f7e64854980 1 > prepared multipart upload key=UQLUT9ZAI61V upload=UQEGUKAGGWMW tag=S7MDMYL8JI2I type=write size=2094438 2026-02-24T22:10:43.287 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.285+0000 7f7e64854980 1 < suggested remove operation key=SGQYQ2TIVJPS tag=YB92ZA11KARA type=del 2026-02-24T22:10:43.288 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.285+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=5VC9T919IUAX type=del size=4120814 2026-02-24T22:10:43.288 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.285+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~GO6Z0SPMB1DY.0 size=706264 2026-02-24T22:10:43.289 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.289+0000 7f7e64854980 1 < completed multipart upload key=N3ZUNSI5K8HS upload=GO6Z0SPMB1DY tag=7DJA9O1L83EN type=write size=1754840 2026-02-24T22:10:43.290 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.289+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=OBIJASPRFKV4 type=del size=3887525 2026-02-24T22:10:43.292 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.289+0000 7f7e64854980 1 < suggested remove operation key=GH7723GNI2WM tag=BW3CZQ0EJAKJ type=del 2026-02-24T22:10:43.293 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.289+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=6XWY8B72JO1I type=del size=2665886 2026-02-24T22:10:43.293 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.289+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~Y68BGTFM86RW.0 size=610687 2026-02-24T22:10:43.294 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.293+0000 7f7e64854980 1 < completed multipart upload key=4Y3974PYYT78 upload=Y68BGTFM86RW tag=WCE3DO4X2Z9R type=write size=1659263 2026-02-24T22:10:43.295 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.293+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~VPII07WW1HR6.0 size=512262 2026-02-24T22:10:43.296 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.293+0000 7f7e64854980 1 > prepared multipart upload key=NL60Q5QOSP8E upload=VPII07WW1HR6 tag=5LV2PO4PW14D type=write size=1560838 2026-02-24T22:10:43.296 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.293+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~FADQO352YN88.0 size=1048576 2026-02-24T22:10:43.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.293+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~FADQO352YN88.1 size=1048576 2026-02-24T22:10:43.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.293+0000 7f7e64854980 1 < canceled part key=_multipart_ZESKGBWZXM6O.2~FADQO352YN88.2 size=84447 2026-02-24T22:10:43.298 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.297+0000 7f7e64854980 1 < canceled multipart upload key=ZESKGBWZXM6O upload=FADQO352YN88 tag=N2PC2QG3H3WB type=write size=3230175 2026-02-24T22:10:43.299 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.297+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~17KWZK3BEOPK.0 size=344290 2026-02-24T22:10:43.299 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.297+0000 7f7e64854980 1 > prepared multipart upload key=0B2V0MHWSOC9 upload=17KWZK3BEOPK tag=DDCQ7K22LXN8 type=write size=1392866 2026-02-24T22:10:43.300 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.297+0000 7f7e64854980 1 < completed delete operation key=0B2V0MHWSOC9 tag=EN41SUFXINI1 type=del 2026-02-24T22:10:43.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.297+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~LHJET3IDDA1C.0 size=1048576 2026-02-24T22:10:43.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.301+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~LHJET3IDDA1C.1 size=1048576 2026-02-24T22:10:43.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.301+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~LHJET3IDDA1C.2 size=670752 2026-02-24T22:10:43.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.301+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=LHJET3IDDA1C tag=E03EN2UM7RL6 type=write size=3816480 2026-02-24T22:10:43.303 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.301+0000 7f7e64854980 1 < suggested update operation key=Z0OXREAUJVA8 tag=KWH4QR9ITQ3U type=del size=3305804 2026-02-24T22:10:43.304 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.301+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=BIL6EAXBFKY5 type=del size=2613508 2026-02-24T22:10:43.305 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.301+0000 7f7e64854980 1 < suggested update operation key=TXRUQQ456DD8 tag=GK9BM16ZWXIU type=del size=3821870 2026-02-24T22:10:43.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.305+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~89H3RRIYN9DK.0 size=232464 2026-02-24T22:10:43.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.305+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=89H3RRIYN9DK tag=WS35F1PM7D8H type=write size=1281040 2026-02-24T22:10:43.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.305+0000 7f7e64854980 1 < suggested remove operation key=HKOQRSF6XEXB tag=HP7WZ4POBSF6 type=del 2026-02-24T22:10:43.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.305+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=N31TVXNIYMWV type=del size=437264 2026-02-24T22:10:43.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.305+0000 7f7e64854980 1 < completed write operation key=D0MKGNRDW6L9 tag=TYB96IPHH66P type=write size=804180 2026-02-24T22:10:43.309 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.305+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=ISM6VJYCHB9I type=del size=514008 2026-02-24T22:10:43.310 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.309+0000 7f7e64854980 1 < completed part key=_multipart_UQLUT9ZAI61V.2~I7SYCV25AI1X.0 size=445237 2026-02-24T22:10:43.311 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.309+0000 7f7e64854980 1 < completed multipart upload key=UQLUT9ZAI61V upload=I7SYCV25AI1X tag=1NB9AC84U6FM type=write size=1493813 2026-02-24T22:10:43.311 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.309+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=K8VV3Z11DULC type=del size=2702489 2026-02-24T22:10:43.312 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.309+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~5QYD9C5X7TEQ.0 size=1025028 2026-02-24T22:10:43.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.309+0000 7f7e64854980 1 < completed multipart upload key=A6PH77Q78VEN upload=5QYD9C5X7TEQ tag=U2JMAEL8S58K type=write size=2073604 2026-02-24T22:10:43.314 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.313+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=MB2NJW8ZO5ZP type=del size=2603490 2026-02-24T22:10:43.314 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.313+0000 7f7e64854980 -1 < failed to complete operation key=HKOQRSF6XEXB tag=YT6TL2WA5BHB type=del size=1999861: (22) Invalid argument 2026-02-24T22:10:43.315 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.313+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=GYQPS3Q7ULZ2 type=del size=3727426 2026-02-24T22:10:43.315 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.313+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~RIGM88Z3JQ21.0 size=1048576 2026-02-24T22:10:43.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.517+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~RIGM88Z3JQ21.1 size=797289 2026-02-24T22:10:43.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.517+0000 7f7e64854980 1 < completed multipart upload key=Q7BP8FXTARDW upload=RIGM88Z3JQ21 tag=71V6EGBCJXA0 type=write size=2894441 2026-02-24T22:10:43.521 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.521+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~PX9XTSAK4B7V.0 size=1048576 2026-02-24T22:10:43.523 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.521+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~PX9XTSAK4B7V.1 size=1048576 2026-02-24T22:10:43.523 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.521+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~PX9XTSAK4B7V.2 size=18642 2026-02-24T22:10:43.524 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.521+0000 7f7e64854980 1 > prepared multipart upload key=3900SFJK1YQU upload=PX9XTSAK4B7V tag=YDX250QQKOHY type=write size=3164370 2026-02-24T22:10:43.525 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.521+0000 7f7e64854980 1 < suggested update operation key=PKRHWM69GBJI tag=Z65UCL7NGDRY type=del size=1189826 2026-02-24T22:10:43.526 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.525+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=5U2LO7WYQODX type=del size=484547 2026-02-24T22:10:43.526 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.525+0000 7f7e64854980 1 < completed part key=_multipart_UQLUT9ZAI61V.2~UQEGUKAGGWMW.0 size=1045862 2026-02-24T22:10:43.527 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.525+0000 7f7e64854980 1 < completed multipart upload key=UQLUT9ZAI61V upload=UQEGUKAGGWMW tag=S7MDMYL8JI2I type=write size=2094438 2026-02-24T22:10:43.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.525+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=99PYVFUVGWB9 type=del size=1166796 2026-02-24T22:10:43.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.525+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=5VC9T919IUAX type=del 2026-02-24T22:10:43.530 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.529+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~Y3F81HXQ78OT.0 size=1035589 2026-02-24T22:10:43.530 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.529+0000 7f7e64854980 1 > prepared multipart upload key=Q7BP8FXTARDW upload=Y3F81HXQ78OT tag=UE5EAUC349LP type=write size=2084165 2026-02-24T22:10:43.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.529+0000 7f7e64854980 1 < suggested update operation key=3900SFJK1YQU tag=OBIJASPRFKV4 type=del size=3887525 2026-02-24T22:10:43.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.529+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=GKMEYQLXIV6O type=del size=3822657 2026-02-24T22:10:43.532 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.529+0000 7f7e64854980 -1 < failed to complete operation key=PKRHWM69GBJI tag=6XWY8B72JO1I type=del size=2665886: (22) Invalid argument 2026-02-24T22:10:43.533 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.529+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=HIR1346G03WZ type=del size=3553394 2026-02-24T22:10:43.533 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.529+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~VPII07WW1HR6.0 size=512262 2026-02-24T22:10:43.534 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.533+0000 7f7e64854980 1 < completed multipart upload key=NL60Q5QOSP8E upload=VPII07WW1HR6 tag=5LV2PO4PW14D type=write size=1560838 2026-02-24T22:10:43.535 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.533+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=LVPQWM9T7CMG type=write size=575263 2026-02-24T22:10:43.535 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.533+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~17KWZK3BEOPK.0 size=344290 2026-02-24T22:10:43.536 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.533+0000 7f7e64854980 1 < completed multipart upload key=0B2V0MHWSOC9 upload=17KWZK3BEOPK tag=DDCQ7K22LXN8 type=write size=1392866 2026-02-24T22:10:43.538 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.537+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~F5SH6KKNWJ83.0 size=1048576 2026-02-24T22:10:43.538 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.537+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~F5SH6KKNWJ83.1 size=1048576 2026-02-24T22:10:43.539 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.537+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~F5SH6KKNWJ83.2 size=53521 2026-02-24T22:10:43.539 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.537+0000 7f7e64854980 1 > prepared multipart upload key=Z0OXREAUJVA8 upload=F5SH6KKNWJ83 tag=MSVS0ZQFC2SO type=write size=3199249 2026-02-24T22:10:43.539 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.537+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~LHJET3IDDA1C.0 size=1048576 2026-02-24T22:10:43.540 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.537+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~LHJET3IDDA1C.1 size=1048576 2026-02-24T22:10:43.540 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.537+0000 7f7e64854980 1 < canceled part key=_multipart_D0MKGNRDW6L9.2~LHJET3IDDA1C.2 size=670752 2026-02-24T22:10:43.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.537+0000 7f7e64854980 1 < canceled multipart upload key=D0MKGNRDW6L9 upload=LHJET3IDDA1C tag=E03EN2UM7RL6 type=write size=3816480 2026-02-24T22:10:43.542 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.541+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~TGL28P3ID24W.0 size=263724 2026-02-24T22:10:43.542 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.541+0000 7f7e64854980 1 > prepared multipart upload key=Q7BP8FXTARDW upload=TGL28P3ID24W tag=FWC43G64T09P type=write size=1312300 2026-02-24T22:10:43.543 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.541+0000 7f7e64854980 1 < suggested remove operation key=Q7BP8FXTARDW tag=BIL6EAXBFKY5 type=del 2026-02-24T22:10:43.543 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.541+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=EKU6O701FYRW type=write size=699574 2026-02-24T22:10:43.544 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.541+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~89H3RRIYN9DK.0 size=232464 2026-02-24T22:10:43.545 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.541+0000 7f7e64854980 1 < completed multipart upload key=TXRUQQ456DD8 upload=89H3RRIYN9DK tag=WS35F1PM7D8H type=write size=1281040 2026-02-24T22:10:43.546 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.545+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=WSXR8CYJC59Y type=del size=1085282 2026-02-24T22:10:43.546 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.545+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=N31TVXNIYMWV type=del 2026-02-24T22:10:43.547 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.545+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=GW1SAQA7JEVK type=del size=3505884 2026-02-24T22:10:43.547 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.545+0000 7f7e64854980 -1 < failed to complete operation key=PKRHWM69GBJI tag=ISM6VJYCHB9I type=del size=514008: (22) Invalid argument 2026-02-24T22:10:43.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.545+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~07G8IA217HKV.0 size=1048576 2026-02-24T22:10:43.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.545+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~07G8IA217HKV.1 size=64746 2026-02-24T22:10:43.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.545+0000 7f7e64854980 1 > prepared multipart upload key=0B2V0MHWSOC9 upload=07G8IA217HKV tag=DP9SJVT885XA type=write size=2161898 2026-02-24T22:10:43.550 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.549+0000 7f7e64854980 1 < canceled operation key=A6PH77Q78VEN tag=K8VV3Z11DULC type=del size=2702489 2026-02-24T22:10:43.551 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.549+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~J450E74Q1EWN.0 size=712257 2026-02-24T22:10:43.551 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.549+0000 7f7e64854980 1 > prepared multipart upload key=Y8GMOUA977H3 upload=J450E74Q1EWN tag=43IDOQI11K0I type=write size=1760833 2026-02-24T22:10:43.551 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.549+0000 7f7e64854980 1 < canceled operation key=TXRUQQ456DD8 tag=MB2NJW8ZO5ZP type=del size=2603490 2026-02-24T22:10:43.553 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.549+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~L8OGFCHCLQGP.0 size=1048576 2026-02-24T22:10:43.553 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.549+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~L8OGFCHCLQGP.1 size=714176 2026-02-24T22:10:43.553 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.549+0000 7f7e64854980 1 > prepared multipart upload key=3900SFJK1YQU upload=L8OGFCHCLQGP tag=Z4JDIMEY88KP type=write size=2811328 2026-02-24T22:10:43.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.553+0000 7f7e64854980 1 < suggested remove operation key=36MKR2OZN7K9 tag=GYQPS3Q7ULZ2 type=del 2026-02-24T22:10:43.555 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.553+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=48QXZ4X6ET4L type=del size=1352450 2026-02-24T22:10:43.555 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.553+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~PX9XTSAK4B7V.0 size=1048576 2026-02-24T22:10:43.556 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.553+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~PX9XTSAK4B7V.1 size=1048576 2026-02-24T22:10:43.556 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.553+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~PX9XTSAK4B7V.2 size=18642 2026-02-24T22:10:43.558 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.557+0000 7f7e64854980 1 < completed multipart upload key=3900SFJK1YQU upload=PX9XTSAK4B7V tag=YDX250QQKOHY type=write size=3164370 2026-02-24T22:10:43.558 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.557+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=XR13Z23L8921 type=del size=2186083 2026-02-24T22:10:43.559 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.557+0000 7f7e64854980 1 < completed delete operation key=Z0OXREAUJVA8 tag=5U2LO7WYQODX type=del 2026-02-24T22:10:43.560 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.557+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~5HFYXADUHMBQ.0 size=1048576 2026-02-24T22:10:43.560 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.557+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~5HFYXADUHMBQ.1 size=935793 2026-02-24T22:10:43.561 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.557+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=5HFYXADUHMBQ tag=02GWIB10YWKE type=write size=3032945 2026-02-24T22:10:43.561 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.557+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=99PYVFUVGWB9 type=del 2026-02-24T22:10:43.562 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.561+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=T9Q5NZTL03NF type=del size=1693837 2026-02-24T22:10:43.562 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.561+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~Y3F81HXQ78OT.0 size=1035589 2026-02-24T22:10:43.563 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.561+0000 7f7e64854980 1 < completed multipart upload key=Q7BP8FXTARDW upload=Y3F81HXQ78OT tag=UE5EAUC349LP type=write size=2084165 2026-02-24T22:10:43.564 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.561+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=JSNIX7MN9DWO type=del size=2009341 2026-02-24T22:10:43.565 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.561+0000 7f7e64854980 1 < completed delete operation key=TF5B43G7KCYT tag=GKMEYQLXIV6O type=del 2026-02-24T22:10:43.565 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.561+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=MJXHO2BLJ35Q type=del size=1590051 2026-02-24T22:10:43.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.565+0000 7f7e64854980 1 < canceled operation key=4Y3974PYYT78 tag=HIR1346G03WZ type=del size=3553394 2026-02-24T22:10:43.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.565+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=2PI4E8CDBTZU type=del size=2602998 2026-02-24T22:10:43.568 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.565+0000 7f7e64854980 1 < suggested update operation key=78GUTC5THI2P tag=LVPQWM9T7CMG type=write size=575263 2026-02-24T22:10:43.569 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.565+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~U0POC2JKVDBM.0 size=1048576 2026-02-24T22:10:43.569 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.569+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~U0POC2JKVDBM.1 size=1048576 2026-02-24T22:10:43.570 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.569+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~U0POC2JKVDBM.2 size=569197 2026-02-24T22:10:43.570 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.569+0000 7f7e64854980 1 > prepared multipart upload key=A6PH77Q78VEN upload=U0POC2JKVDBM tag=M92FCGCPGKPT type=write size=3714925 2026-02-24T22:10:43.570 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.569+0000 7f7e64854980 1 < canceled part key=_multipart_Z0OXREAUJVA8.2~F5SH6KKNWJ83.0 size=1048576 2026-02-24T22:10:43.571 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.569+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~F5SH6KKNWJ83.1 size=1048576 2026-02-24T22:10:43.571 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.569+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~F5SH6KKNWJ83.2 size=53521 2026-02-24T22:10:43.572 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.569+0000 7f7e64854980 1 < canceled multipart upload key=Z0OXREAUJVA8 upload=F5SH6KKNWJ83 tag=MSVS0ZQFC2SO type=write size=3199249 2026-02-24T22:10:43.573 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.569+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=KA5JFNWGMM3Z type=del size=336504 2026-02-24T22:10:43.574 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.573+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~TGL28P3ID24W.0 size=263724 2026-02-24T22:10:43.575 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.573+0000 7f7e64854980 1 < completed multipart upload key=Q7BP8FXTARDW upload=TGL28P3ID24W tag=FWC43G64T09P type=write size=1312300 2026-02-24T22:10:43.576 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.573+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=0XS3VPHJ1EFB type=del size=667549 2026-02-24T22:10:43.576 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.573+0000 7f7e64854980 1 < completed write operation key=UQLUT9ZAI61V tag=EKU6O701FYRW type=write size=699574 2026-02-24T22:10:43.577 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.577+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~L3ARCMFWMPY5.0 size=653188 2026-02-24T22:10:43.578 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.577+0000 7f7e64854980 1 > prepared multipart upload key=TF5B43G7KCYT upload=L3ARCMFWMPY5 tag=5PVC120IWMM9 type=write size=1701764 2026-02-24T22:10:43.578 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.577+0000 7f7e64854980 1 < completed delete operation key=HKOQRSF6XEXB tag=WSXR8CYJC59Y type=del 2026-02-24T22:10:43.579 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.577+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=TVGZ2VKATQZ9 type=write size=299691 2026-02-24T22:10:43.579 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.577+0000 7f7e64854980 1 < completed delete operation key=HKOQRSF6XEXB tag=GW1SAQA7JEVK type=del 2026-02-24T22:10:43.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.577+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=EMMMLNTYCYNV type=del size=3502173 2026-02-24T22:10:43.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.577+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~07G8IA217HKV.0 size=1048576 2026-02-24T22:10:43.582 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.581+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~07G8IA217HKV.1 size=64746 2026-02-24T22:10:43.584 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.581+0000 7f7e64854980 1 < completed multipart upload key=0B2V0MHWSOC9 upload=07G8IA217HKV tag=DP9SJVT885XA type=write size=2161898 2026-02-24T22:10:43.584 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.581+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=83RO6BAT6WNV type=del size=1688726 2026-02-24T22:10:43.585 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.581+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~J450E74Q1EWN.0 size=712257 2026-02-24T22:10:43.586 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.585+0000 7f7e64854980 1 < completed multipart upload key=Y8GMOUA977H3 upload=J450E74Q1EWN tag=43IDOQI11K0I type=write size=1760833 2026-02-24T22:10:43.587 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.585+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=8KH1XICWRM95 type=del size=3648731 2026-02-24T22:10:43.587 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.585+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~L8OGFCHCLQGP.0 size=1048576 2026-02-24T22:10:43.588 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.585+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~L8OGFCHCLQGP.1 size=714176 2026-02-24T22:10:43.589 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.585+0000 7f7e64854980 1 < completed multipart upload key=3900SFJK1YQU upload=L8OGFCHCLQGP tag=Z4JDIMEY88KP type=write size=2811328 2026-02-24T22:10:43.590 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.589+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~TACIAWS7SCV5.0 size=1048576 2026-02-24T22:10:43.591 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.589+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~TACIAWS7SCV5.1 size=202897 2026-02-24T22:10:43.591 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.589+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=TACIAWS7SCV5 tag=CL5NH0K38A9I type=write size=2300049 2026-02-24T22:10:43.592 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.589+0000 7f7e64854980 1 < completed delete operation key=TF5B43G7KCYT tag=48QXZ4X6ET4L type=del 2026-02-24T22:10:43.592 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.589+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~ENXGY3FQPU6T.0 size=1048576 2026-02-24T22:10:43.593 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.589+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~ENXGY3FQPU6T.1 size=1048576 2026-02-24T22:10:43.593 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.593+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~ENXGY3FQPU6T.2 size=558193 2026-02-24T22:10:43.594 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.593+0000 7f7e64854980 1 > prepared multipart upload key=4Y3974PYYT78 upload=ENXGY3FQPU6T tag=R8S6OYTW99KL type=write size=3703921 2026-02-24T22:10:43.594 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.593+0000 7f7e64854980 1 < completed delete operation key=2KKQKHXO7K3K tag=XR13Z23L8921 type=del 2026-02-24T22:10:43.595 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.593+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=V5JVQCQUELNP type=del size=2354011 2026-02-24T22:10:43.595 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.593+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~5HFYXADUHMBQ.0 size=1048576 2026-02-24T22:10:43.596 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.593+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~5HFYXADUHMBQ.1 size=935793 2026-02-24T22:10:43.597 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.593+0000 7f7e64854980 1 < completed multipart upload key=CI8DA17DX8LU upload=5HFYXADUHMBQ tag=02GWIB10YWKE type=write size=3032945 2026-02-24T22:10:43.598 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.597+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~01AUGMDBSNNV.0 size=76059 2026-02-24T22:10:43.598 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.597+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=01AUGMDBSNNV tag=KD5CU0XDAB8P type=write size=1124635 2026-02-24T22:10:43.598 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.597+0000 7f7e64854980 1 < completed delete operation key=4Y3974PYYT78 tag=T9Q5NZTL03NF type=del 2026-02-24T22:10:43.599 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.597+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=1XMFZAASZNSF type=del size=763283 2026-02-24T22:10:43.600 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.597+0000 7f7e64854980 1 < completed delete operation key=GH7723GNI2WM tag=JSNIX7MN9DWO type=del 2026-02-24T22:10:43.600 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.597+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=FE02RH9FBHQJ type=del size=2349976 2026-02-24T22:10:43.601 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.601+0000 7f7e64854980 1 < suggested update operation key=HKOQRSF6XEXB tag=MJXHO2BLJ35Q type=del size=1590051 2026-02-24T22:10:43.602 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.601+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=L13LRCTL1YGU type=write size=842418 2026-02-24T22:10:43.603 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.601+0000 7f7e64854980 1 < canceled operation key=TF5B43G7KCYT tag=2PI4E8CDBTZU type=del size=2602998 2026-02-24T22:10:43.604 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.601+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~3QALSO5XOCFP.0 size=1048576 2026-02-24T22:10:43.604 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.601+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~3QALSO5XOCFP.1 size=1048576 2026-02-24T22:10:43.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.601+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~3QALSO5XOCFP.2 size=800323 2026-02-24T22:10:43.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.601+0000 7f7e64854980 1 > prepared multipart upload key=36MKR2OZN7K9 upload=3QALSO5XOCFP tag=EE78KY6HA90Q type=write size=3946051 2026-02-24T22:10:43.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.605+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~U0POC2JKVDBM.0 size=1048576 2026-02-24T22:10:43.606 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.605+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~U0POC2JKVDBM.1 size=1048576 2026-02-24T22:10:43.606 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.605+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~U0POC2JKVDBM.2 size=569197 2026-02-24T22:10:43.607 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.605+0000 7f7e64854980 1 < completed multipart upload key=A6PH77Q78VEN upload=U0POC2JKVDBM tag=M92FCGCPGKPT type=write size=3714925 2026-02-24T22:10:43.608 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.605+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=CZ03G2D73ES5 type=del size=3346251 2026-02-24T22:10:43.609 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.605+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=KA5JFNWGMM3Z type=del 2026-02-24T22:10:43.610 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.609+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~VUI0MW40J332.0 size=927899 2026-02-24T22:10:43.610 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.609+0000 7f7e64854980 1 > prepared multipart upload key=HKOQRSF6XEXB upload=VUI0MW40J332 tag=TCAD7WWCI8VJ type=write size=1976475 2026-02-24T22:10:43.610 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.609+0000 7f7e64854980 1 < completed delete operation key=NL60Q5QOSP8E tag=0XS3VPHJ1EFB type=del 2026-02-24T22:10:43.611 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.609+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=MSEOCSAX9I1U type=del size=1304739 2026-02-24T22:10:43.611 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.609+0000 7f7e64854980 1 < canceled part key=_multipart_TF5B43G7KCYT.2~L3ARCMFWMPY5.0 size=653188 2026-02-24T22:10:43.612 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.609+0000 7f7e64854980 1 < canceled multipart upload key=TF5B43G7KCYT upload=L3ARCMFWMPY5 tag=5PVC120IWMM9 type=write size=1701764 2026-02-24T22:10:43.613 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.609+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=SBNFYOQG3Q0R type=del size=849365 2026-02-24T22:10:43.613 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.613+0000 7f7e64854980 1 < completed write operation key=EB04ETNYM087 tag=TVGZ2VKATQZ9 type=write size=299691 2026-02-24T22:10:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.613+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=S2MNVAJOM3AA type=write size=557221 2026-02-24T22:10:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.613+0000 7f7e64854980 1 < completed delete operation key=Y3L3CJ51C1WR tag=EMMMLNTYCYNV type=del 2026-02-24T22:10:43.615 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.613+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=RMGJUQ1358MO type=del size=4165469 2026-02-24T22:10:43.616 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.613+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=83RO6BAT6WNV type=del 2026-02-24T22:10:43.617 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.613+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~3WCAAQR5S2LL.0 size=678331 2026-02-24T22:10:43.617 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.613+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=3WCAAQR5S2LL tag=DHTVADE4N0LX type=write size=1726907 2026-02-24T22:10:43.618 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.617+0000 7f7e64854980 1 < completed delete operation key=GH7723GNI2WM tag=8KH1XICWRM95 type=del 2026-02-24T22:10:43.619 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.617+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~IYFQWXDLQEEC.0 size=100051 2026-02-24T22:10:43.619 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.617+0000 7f7e64854980 1 > prepared multipart upload key=EB04ETNYM087 upload=IYFQWXDLQEEC tag=R6BB42452YPL type=write size=1148627 2026-02-24T22:10:43.619 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.617+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~TACIAWS7SCV5.0 size=1048576 2026-02-24T22:10:43.620 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.617+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~TACIAWS7SCV5.1 size=202897 2026-02-24T22:10:43.621 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.617+0000 7f7e64854980 1 < completed multipart upload key=CI8DA17DX8LU upload=TACIAWS7SCV5 tag=CL5NH0K38A9I type=write size=2300049 2026-02-24T22:10:43.622 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.621+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=HH50KHPRERNY type=del size=1417198 2026-02-24T22:10:43.622 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.621+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~ENXGY3FQPU6T.0 size=1048576 2026-02-24T22:10:43.623 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.621+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~ENXGY3FQPU6T.1 size=1048576 2026-02-24T22:10:43.623 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.621+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~ENXGY3FQPU6T.2 size=558193 2026-02-24T22:10:43.624 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.621+0000 7f7e64854980 1 < completed multipart upload key=4Y3974PYYT78 upload=ENXGY3FQPU6T tag=R8S6OYTW99KL type=write size=3703921 2026-02-24T22:10:43.625 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.625+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~D83WP0TNH48F.0 size=1048576 2026-02-24T22:10:43.626 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.625+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~D83WP0TNH48F.1 size=703762 2026-02-24T22:10:43.626 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.625+0000 7f7e64854980 1 > prepared multipart upload key=SGQYQ2TIVJPS upload=D83WP0TNH48F tag=MWICJ33CSU8X type=write size=2800914 2026-02-24T22:10:43.626 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.625+0000 7f7e64854980 1 < completed delete operation key=EJCX06TE5RTZ tag=V5JVQCQUELNP type=del 2026-02-24T22:10:43.627 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.625+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=ZL4VLZ0PFUKS type=del size=3340921 2026-02-24T22:10:43.628 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.625+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~01AUGMDBSNNV.0 size=76059 2026-02-24T22:10:43.629 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.625+0000 7f7e64854980 1 < completed multipart upload key=ZESKGBWZXM6O upload=01AUGMDBSNNV tag=KD5CU0XDAB8P type=write size=1124635 2026-02-24T22:10:43.629 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.629+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=3YOBSPAQ8I1R type=del size=2722411 2026-02-24T22:10:43.630 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.629+0000 7f7e64854980 1 < completed delete operation key=0B2V0MHWSOC9 tag=1XMFZAASZNSF type=del 2026-02-24T22:10:43.630 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.629+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=EQYX60VKROCD type=write size=383584 2026-02-24T22:10:43.631 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.629+0000 7f7e64854980 1 < canceled operation key=0B2V0MHWSOC9 tag=FE02RH9FBHQJ type=del size=2349976 2026-02-24T22:10:43.632 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.629+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=356WYXPSXYIK type=del size=1649353 2026-02-24T22:10:43.633 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.629+0000 7f7e64854980 1 < suggested remove operation key=3900SFJK1YQU tag=L13LRCTL1YGU type=write 2026-02-24T22:10:43.634 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.633+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=80CT085IUPQ0 type=del size=2674608 2026-02-24T22:10:43.634 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.633+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~3QALSO5XOCFP.0 size=1048576 2026-02-24T22:10:43.635 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.633+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~3QALSO5XOCFP.1 size=1048576 2026-02-24T22:10:43.635 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.633+0000 7f7e64854980 1 < canceled part key=_multipart_36MKR2OZN7K9.2~3QALSO5XOCFP.2 size=800323 2026-02-24T22:10:43.636 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.633+0000 7f7e64854980 1 < canceled multipart upload key=36MKR2OZN7K9 upload=3QALSO5XOCFP tag=EE78KY6HA90Q type=write size=3946051 2026-02-24T22:10:43.637 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.633+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~S0S2RLMXRCY2.0 size=1002822 2026-02-24T22:10:43.637 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.633+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=S0S2RLMXRCY2 tag=GJ9A7YTDKET7 type=write size=2051398 2026-02-24T22:10:43.638 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.637+0000 7f7e64854980 1 < suggested update operation key=N3ZUNSI5K8HS tag=CZ03G2D73ES5 type=del size=3346251 2026-02-24T22:10:43.639 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.637+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~6WMNO6WJHBAF.0 size=670853 2026-02-24T22:10:43.639 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.637+0000 7f7e64854980 1 > prepared multipart upload key=0B2V0MHWSOC9 upload=6WMNO6WJHBAF tag=FE0EGKPO95EN type=write size=1719429 2026-02-24T22:10:43.640 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.637+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~VUI0MW40J332.0 size=927899 2026-02-24T22:10:43.641 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.641+0000 7f7e64854980 1 < completed multipart upload key=HKOQRSF6XEXB upload=VUI0MW40J332 tag=TCAD7WWCI8VJ type=write size=1976475 2026-02-24T22:10:43.642 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.641+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=RNHB2VUUOVRI type=write size=25750 2026-02-24T22:10:43.643 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.641+0000 7f7e64854980 1 < completed delete operation key=L60HQ7M3E3LL tag=MSEOCSAX9I1U type=del 2026-02-24T22:10:43.643 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.641+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=G6UYAT62NIPP type=del size=3916940 2026-02-24T22:10:43.644 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.641+0000 7f7e64854980 1 < completed delete operation key=TF5B43G7KCYT tag=SBNFYOQG3Q0R type=del 2026-02-24T22:10:43.645 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.641+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~9CX2X976IDDF.0 size=1048576 2026-02-24T22:10:43.646 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.645+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~9CX2X976IDDF.1 size=1048576 2026-02-24T22:10:43.646 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.645+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~9CX2X976IDDF.2 size=208620 2026-02-24T22:10:43.646 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.645+0000 7f7e64854980 1 > prepared multipart upload key=0B2V0MHWSOC9 upload=9CX2X976IDDF tag=O0W4Z367DZVC type=write size=3354348 2026-02-24T22:10:43.647 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.645+0000 7f7e64854980 1 < completed write operation key=R2893NES3WSF tag=S2MNVAJOM3AA type=write size=557221 2026-02-24T22:10:43.647 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.645+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=V35UY2190I9Y type=del size=1641811 2026-02-24T22:10:43.648 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.645+0000 7f7e64854980 1 < canceled operation key=LSRPN33Z2B9N tag=RMGJUQ1358MO type=del size=4165469 2026-02-24T22:10:43.649 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.645+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=20BFERMF49RP type=del size=3309486 2026-02-24T22:10:43.649 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.649+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~3WCAAQR5S2LL.0 size=678331 2026-02-24T22:10:43.650 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.649+0000 7f7e64854980 1 < completed multipart upload key=HOHKR92SN25M upload=3WCAAQR5S2LL tag=DHTVADE4N0LX type=write size=1726907 2026-02-24T22:10:43.651 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.649+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=160JQSCHRXKW type=del size=1509385 2026-02-24T22:10:43.651 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.649+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~IYFQWXDLQEEC.0 size=100051 2026-02-24T22:10:43.652 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.649+0000 7f7e64854980 1 < completed multipart upload key=EB04ETNYM087 upload=IYFQWXDLQEEC tag=R6BB42452YPL type=write size=1148627 2026-02-24T22:10:43.653 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.649+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=75Z70678H7KY type=del size=976625 2026-02-24T22:10:43.654 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.653+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=HH50KHPRERNY type=del 2026-02-24T22:10:43.654 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.653+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=M30DZPMA23RV type=del size=3609541 2026-02-24T22:10:43.655 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.653+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~D83WP0TNH48F.0 size=1048576 2026-02-24T22:10:43.656 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.653+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~D83WP0TNH48F.1 size=703762 2026-02-24T22:10:43.657 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.653+0000 7f7e64854980 1 < completed multipart upload key=SGQYQ2TIVJPS upload=D83WP0TNH48F tag=MWICJ33CSU8X type=write size=2800914 2026-02-24T22:10:43.657 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.653+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=LDD4T995E14P type=del size=1433886 2026-02-24T22:10:43.658 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.657+0000 7f7e64854980 1 < completed delete operation key=Z0OXREAUJVA8 tag=ZL4VLZ0PFUKS type=del 2026-02-24T22:10:43.659 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.657+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~00W8JIA45N38.0 size=1048576 2026-02-24T22:10:43.659 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.657+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~00W8JIA45N38.1 size=779894 2026-02-24T22:10:43.660 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.657+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=00W8JIA45N38 tag=DSJ3HWQKNAZL type=write size=2877046 2026-02-24T22:10:43.660 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.657+0000 7f7e64854980 1 < canceled operation key=36MKR2OZN7K9 tag=3YOBSPAQ8I1R type=del size=2722411 2026-02-24T22:10:43.661 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.657+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=0IWE92WM2OTM type=del size=2909247 2026-02-24T22:10:43.661 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.661+0000 7f7e64854980 1 < completed write operation key=HOHKR92SN25M tag=EQYX60VKROCD type=write size=383584 2026-02-24T22:10:43.662 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.661+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~RFIPIZS6UY9K.0 size=1048576 2026-02-24T22:10:43.663 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.661+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~RFIPIZS6UY9K.1 size=375541 2026-02-24T22:10:43.663 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.661+0000 7f7e64854980 1 > prepared multipart upload key=L60HQ7M3E3LL upload=RFIPIZS6UY9K tag=5ER62HD5DLZJ type=write size=2472693 2026-02-24T22:10:43.664 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.661+0000 7f7e64854980 1 < completed delete operation key=D0MKGNRDW6L9 tag=356WYXPSXYIK type=del 2026-02-24T22:10:43.664 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.661+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=5MXAGKJ3MIPK type=del size=1975868 2026-02-24T22:10:43.665 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.661+0000 7f7e64854980 1 < canceled operation key=TF5B43G7KCYT tag=80CT085IUPQ0 type=del size=2674608 2026-02-24T22:10:43.665 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.665+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=Q80TCA36EQ7P type=del size=609587 2026-02-24T22:10:43.666 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.665+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~S0S2RLMXRCY2.0 size=1002822 2026-02-24T22:10:43.667 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.665+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=S0S2RLMXRCY2 tag=GJ9A7YTDKET7 type=write size=2051398 2026-02-24T22:10:43.668 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.665+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=7PN5G3GHUGLS type=del size=3729588 2026-02-24T22:10:43.668 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.665+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~6WMNO6WJHBAF.0 size=670853 2026-02-24T22:10:43.669 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.669+0000 7f7e64854980 1 < completed multipart upload key=0B2V0MHWSOC9 upload=6WMNO6WJHBAF tag=FE0EGKPO95EN type=write size=1719429 2026-02-24T22:10:43.670 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.669+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=N4F62A3KUB65 type=del size=2240455 2026-02-24T22:10:43.671 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.669+0000 7f7e64854980 1 < suggested update operation key=4Y3974PYYT78 tag=RNHB2VUUOVRI type=write size=25750 2026-02-24T22:10:43.672 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.669+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=IQ49BJALMFM9 type=del size=295127 2026-02-24T22:10:43.673 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.673+0000 7f7e64854980 1 < suggested remove operation key=SGQYQ2TIVJPS tag=G6UYAT62NIPP type=del 2026-02-24T22:10:43.674 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.673+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~7ES4ZQ7Z8YMA.0 size=1048576 2026-02-24T22:10:43.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.673+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~7ES4ZQ7Z8YMA.1 size=541189 2026-02-24T22:10:43.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.673+0000 7f7e64854980 1 > prepared multipart upload key=NL60Q5QOSP8E upload=7ES4ZQ7Z8YMA tag=JGT62VFS64AJ type=write size=2638341 2026-02-24T22:10:43.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.673+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~9CX2X976IDDF.0 size=1048576 2026-02-24T22:10:43.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.673+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~9CX2X976IDDF.1 size=1048576 2026-02-24T22:10:43.677 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.673+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~9CX2X976IDDF.2 size=208620 2026-02-24T22:10:43.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.677+0000 7f7e64854980 1 < completed multipart upload key=0B2V0MHWSOC9 upload=9CX2X976IDDF tag=O0W4Z367DZVC type=write size=3354348 2026-02-24T22:10:43.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.677+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=4ZZJ7HRO3OLL type=del size=3569046 2026-02-24T22:10:43.679 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.677+0000 7f7e64854980 1 < completed delete operation key=Z0OXREAUJVA8 tag=V35UY2190I9Y type=del 2026-02-24T22:10:43.680 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.677+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~2CCJVFASXZAQ.0 size=1048576 2026-02-24T22:10:43.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.677+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~2CCJVFASXZAQ.1 size=103543 2026-02-24T22:10:43.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.677+0000 7f7e64854980 1 > prepared multipart upload key=78GUTC5THI2P upload=2CCJVFASXZAQ tag=VX35FPJGP7GV type=write size=2200695 2026-02-24T22:10:43.682 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.681+0000 7f7e64854980 1 < suggested update operation key=0AXS7DY23UZU tag=20BFERMF49RP type=del size=3309486 2026-02-24T22:10:43.682 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.681+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=SOCI3CNCPAL8 type=write size=96416 2026-02-24T22:10:43.683 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.681+0000 7f7e64854980 -1 < failed to complete operation key=SGQYQ2TIVJPS tag=160JQSCHRXKW type=del size=1509385: (22) Invalid argument 2026-02-24T22:10:43.684 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.681+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=MBXH3BA2W15S type=del size=1364090 2026-02-24T22:10:43.684 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.681+0000 7f7e64854980 1 < completed delete operation key=TF5B43G7KCYT tag=75Z70678H7KY type=del 2026-02-24T22:10:43.685 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.681+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=TI4YK4R4F7YR type=del size=1731809 2026-02-24T22:10:43.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.685+0000 7f7e64854980 1 < completed delete operation key=2KKQKHXO7K3K tag=M30DZPMA23RV type=del 2026-02-24T22:10:43.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.685+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~C6M9LF4B9ADI.0 size=739803 2026-02-24T22:10:43.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.685+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=C6M9LF4B9ADI tag=3DCXCC2DP3MB type=write size=1788379 2026-02-24T22:10:43.688 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.685+0000 7f7e64854980 1 < suggested remove operation key=L9NEDH44BIGA tag=LDD4T995E14P type=del 2026-02-24T22:10:43.689 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.685+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=JJSOMK93OT28 type=del size=1833925 2026-02-24T22:10:43.689 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.689+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~00W8JIA45N38.0 size=1048576 2026-02-24T22:10:43.690 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.689+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~00W8JIA45N38.1 size=779894 2026-02-24T22:10:43.691 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.689+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=00W8JIA45N38 tag=DSJ3HWQKNAZL type=write size=2877046 2026-02-24T22:10:43.692 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.689+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=IEZH62IXONFJ type=del size=3313255 2026-02-24T22:10:43.692 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.689+0000 7f7e64854980 1 < completed delete operation key=A6PH77Q78VEN tag=0IWE92WM2OTM type=del 2026-02-24T22:10:43.693 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.689+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=MFG81X99GEUF type=del size=3796838 2026-02-24T22:10:43.693 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.693+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~RFIPIZS6UY9K.0 size=1048576 2026-02-24T22:10:43.694 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.693+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~RFIPIZS6UY9K.1 size=375541 2026-02-24T22:10:43.695 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.693+0000 7f7e64854980 1 < completed multipart upload key=L60HQ7M3E3LL upload=RFIPIZS6UY9K tag=5ER62HD5DLZJ type=write size=2472693 2026-02-24T22:10:43.696 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.693+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=45WLTHCQ16CQ type=del size=2606496 2026-02-24T22:10:43.696 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.693+0000 7f7e64854980 1 < canceled operation key=ON16L53JUSTW tag=5MXAGKJ3MIPK type=del size=1975868 2026-02-24T22:10:43.697 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.693+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=I469PQMDIA1T type=write size=607330 2026-02-24T22:10:43.698 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.697+0000 7f7e64854980 1 < completed delete operation key=GH7723GNI2WM tag=Q80TCA36EQ7P type=del 2026-02-24T22:10:43.699 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.697+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~3RJGE56UN93C.0 size=1048576 2026-02-24T22:10:43.700 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.697+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~3RJGE56UN93C.1 size=1048576 2026-02-24T22:10:43.700 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.697+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~3RJGE56UN93C.2 size=38988 2026-02-24T22:10:43.701 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.697+0000 7f7e64854980 1 > prepared multipart upload key=3900SFJK1YQU upload=3RJGE56UN93C tag=I0MWT9Z7SMIU type=write size=3184716 2026-02-24T22:10:43.701 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.697+0000 7f7e64854980 1 < completed delete operation key=LSRPN33Z2B9N tag=7PN5G3GHUGLS type=del 2026-02-24T22:10:43.702 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.701+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=63UGPQ69K90G type=del size=3832367 2026-02-24T22:10:43.702 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.701+0000 7f7e64854980 1 < completed delete operation key=LSRPN33Z2B9N tag=N4F62A3KUB65 type=del 2026-02-24T22:10:43.703 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.701+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=G92T6QQHLENV type=write size=830004 2026-02-24T22:10:43.704 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.701+0000 7f7e64854980 1 < completed delete operation key=D0MKGNRDW6L9 tag=IQ49BJALMFM9 type=del 2026-02-24T22:10:43.705 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.701+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~LAB8SCSUOZFU.0 size=1048576 2026-02-24T22:10:43.706 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.705+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~LAB8SCSUOZFU.1 size=1048576 2026-02-24T22:10:43.706 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.705+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~LAB8SCSUOZFU.2 size=677792 2026-02-24T22:10:43.706 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.705+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=LAB8SCSUOZFU tag=C8IXOJ4N8YWC type=write size=3823520 2026-02-24T22:10:43.707 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.705+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~7ES4ZQ7Z8YMA.0 size=1048576 2026-02-24T22:10:43.707 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.705+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~7ES4ZQ7Z8YMA.1 size=541189 2026-02-24T22:10:43.708 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.705+0000 7f7e64854980 1 < completed multipart upload key=NL60Q5QOSP8E upload=7ES4ZQ7Z8YMA tag=JGT62VFS64AJ type=write size=2638341 2026-02-24T22:10:43.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.705+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=43OU9H92S9RY type=del size=2047257 2026-02-24T22:10:43.710 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.709+0000 7f7e64854980 1 < completed delete operation key=A6PH77Q78VEN tag=4ZZJ7HRO3OLL type=del 2026-02-24T22:10:43.711 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.709+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~22MX7SI3L2NK.0 size=1048576 2026-02-24T22:10:43.712 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.709+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~22MX7SI3L2NK.1 size=1048576 2026-02-24T22:10:43.712 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.709+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~22MX7SI3L2NK.2 size=525243 2026-02-24T22:10:43.712 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.709+0000 7f7e64854980 1 > prepared multipart upload key=Y3L3CJ51C1WR upload=22MX7SI3L2NK tag=ZR5G629KS03Z type=write size=3670971 2026-02-24T22:10:43.713 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.709+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~2CCJVFASXZAQ.0 size=1048576 2026-02-24T22:10:43.713 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.709+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~2CCJVFASXZAQ.1 size=103543 2026-02-24T22:10:43.714 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.713+0000 7f7e64854980 1 < completed multipart upload key=78GUTC5THI2P upload=2CCJVFASXZAQ tag=VX35FPJGP7GV type=write size=2200695 2026-02-24T22:10:43.715 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.713+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~5IGA1YRZKAW5.0 size=1048576 2026-02-24T22:10:43.716 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.713+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~5IGA1YRZKAW5.1 size=1048576 2026-02-24T22:10:43.716 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.713+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~5IGA1YRZKAW5.2 size=429936 2026-02-24T22:10:43.716 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.713+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=5IGA1YRZKAW5 tag=9UC12X3R31V0 type=write size=3575664 2026-02-24T22:10:43.717 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.713+0000 7f7e64854980 1 < canceled operation key=TF5B43G7KCYT tag=SOCI3CNCPAL8 type=write size=96416 2026-02-24T22:10:43.717 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.717+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=H6ZJSVV23OHS type=del size=536680 2026-02-24T22:10:43.718 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.717+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=MBXH3BA2W15S type=del 2026-02-24T22:10:43.719 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.717+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=56EXXOWM64FB type=del size=1589041 2026-02-24T22:10:43.719 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.717+0000 7f7e64854980 1 < canceled operation key=EB04ETNYM087 tag=TI4YK4R4F7YR type=del size=1731809 2026-02-24T22:10:43.720 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.717+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=YBCDCPBKV041 type=del size=222502 2026-02-24T22:10:43.721 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.717+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~C6M9LF4B9ADI.0 size=739803 2026-02-24T22:10:43.722 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.721+0000 7f7e64854980 1 < completed multipart upload key=0AXS7DY23UZU upload=C6M9LF4B9ADI tag=3DCXCC2DP3MB type=write size=1788379 2026-02-24T22:10:43.722 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.721+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=S8DWENXGQUTN type=del size=728249 2026-02-24T22:10:43.723 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.721+0000 7f7e64854980 1 < completed delete operation key=TXRUQQ456DD8 tag=JJSOMK93OT28 type=del 2026-02-24T22:10:43.724 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.721+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~TAKM937W7I52.0 size=1048576 2026-02-24T22:10:43.725 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.721+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~TAKM937W7I52.1 size=1048576 2026-02-24T22:10:43.725 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.721+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~TAKM937W7I52.2 size=396424 2026-02-24T22:10:43.725 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.721+0000 7f7e64854980 1 > prepared multipart upload key=LSRPN33Z2B9N upload=TAKM937W7I52 tag=QPFB3I61NV5C type=write size=3542152 2026-02-24T22:10:43.726 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.725+0000 7f7e64854980 1 < canceled operation key=TF5B43G7KCYT tag=IEZH62IXONFJ type=del size=3313255 2026-02-24T22:10:43.726 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.725+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=B2ZBABT147SE type=del size=516004 2026-02-24T22:10:43.727 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.725+0000 7f7e64854980 1 < completed delete operation key=UQLUT9ZAI61V tag=MFG81X99GEUF type=del 2026-02-24T22:10:43.728 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.725+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=WTHLZWE1GIQ9 type=del size=2257299 2026-02-24T22:10:43.728 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.725+0000 7f7e64854980 1 < completed delete operation key=0AXS7DY23UZU tag=45WLTHCQ16CQ type=del 2026-02-24T22:10:43.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.729+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~W7NJOFMPIO36.0 size=1048576 2026-02-24T22:10:43.730 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.729+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~W7NJOFMPIO36.1 size=1048576 2026-02-24T22:10:43.730 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.729+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~W7NJOFMPIO36.2 size=1042727 2026-02-24T22:10:43.730 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.729+0000 7f7e64854980 1 > prepared multipart upload key=EB04ETNYM087 upload=W7NJOFMPIO36 tag=0FHFTEB7XB9X type=write size=4188455 2026-02-24T22:10:43.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.729+0000 7f7e64854980 1 < completed write operation key=EJCX06TE5RTZ tag=I469PQMDIA1T type=write size=607330 2026-02-24T22:10:43.732 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.729+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=2NSZT43UQK33 type=write size=710024 2026-02-24T22:10:43.732 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.729+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~3RJGE56UN93C.0 size=1048576 2026-02-24T22:10:43.733 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.729+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~3RJGE56UN93C.1 size=1048576 2026-02-24T22:10:43.733 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.733+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~3RJGE56UN93C.2 size=38988 2026-02-24T22:10:43.735 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.733+0000 7f7e64854980 1 < completed multipart upload key=3900SFJK1YQU upload=3RJGE56UN93C tag=I0MWT9Z7SMIU type=write size=3184716 2026-02-24T22:10:43.736 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.733+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~SD4LIJIO8AMO.0 size=1048576 2026-02-24T22:10:43.736 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.733+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~SD4LIJIO8AMO.1 size=279377 2026-02-24T22:10:43.736 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.733+0000 7f7e64854980 1 > prepared multipart upload key=N3ZUNSI5K8HS upload=SD4LIJIO8AMO tag=YK2ESHUJP978 type=write size=2376529 2026-02-24T22:10:43.737 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.733+0000 7f7e64854980 1 < completed delete operation key=36MKR2OZN7K9 tag=63UGPQ69K90G type=del 2026-02-24T22:10:43.738 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.737+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~WNR7L881FJWC.0 size=1048576 2026-02-24T22:10:43.739 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.737+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~WNR7L881FJWC.1 size=446605 2026-02-24T22:10:43.739 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.737+0000 7f7e64854980 1 > prepared multipart upload key=GH7723GNI2WM upload=WNR7L881FJWC tag=VTFFEN61XOOM type=write size=2543757 2026-02-24T22:10:43.740 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.737+0000 7f7e64854980 1 < suggested update operation key=0AXS7DY23UZU tag=G92T6QQHLENV type=write size=830004 2026-02-24T22:10:43.741 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.737+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=LDKUQWGIQK0G type=del size=2083684 2026-02-24T22:10:43.742 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.741+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~LAB8SCSUOZFU.0 size=1048576 2026-02-24T22:10:43.742 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.741+0000 7f7e64854980 1 < canceled part key=_multipart_EJCX06TE5RTZ.2~LAB8SCSUOZFU.1 size=1048576 2026-02-24T22:10:43.742 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.741+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~LAB8SCSUOZFU.2 size=677792 2026-02-24T22:10:43.743 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.741+0000 7f7e64854980 1 < canceled multipart upload key=EJCX06TE5RTZ upload=LAB8SCSUOZFU tag=C8IXOJ4N8YWC type=write size=3823520 2026-02-24T22:10:43.744 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.741+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=3P3QYV041GSD type=del size=2691448 2026-02-24T22:10:43.744 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.741+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=43OU9H92S9RY type=del 2026-02-24T22:10:43.746 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.745+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~YYHSVSYCEVSK.0 size=865313 2026-02-24T22:10:43.746 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.745+0000 7f7e64854980 1 > prepared multipart upload key=GH7723GNI2WM upload=YYHSVSYCEVSK tag=HH2FHJ9HHM4K type=write size=1913889 2026-02-24T22:10:43.746 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.745+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~22MX7SI3L2NK.0 size=1048576 2026-02-24T22:10:43.747 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.745+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~22MX7SI3L2NK.1 size=1048576 2026-02-24T22:10:43.747 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.745+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~22MX7SI3L2NK.2 size=525243 2026-02-24T22:10:43.748 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.745+0000 7f7e64854980 1 < completed multipart upload key=Y3L3CJ51C1WR upload=22MX7SI3L2NK tag=ZR5G629KS03Z type=write size=3670971 2026-02-24T22:10:43.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.749+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~Z7PI501KT38G.0 size=629284 2026-02-24T22:10:43.750 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.749+0000 7f7e64854980 1 > prepared multipart upload key=SGQYQ2TIVJPS upload=Z7PI501KT38G tag=G03I2OFB5UWU type=write size=1677860 2026-02-24T22:10:43.750 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.749+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~5IGA1YRZKAW5.0 size=1048576 2026-02-24T22:10:43.751 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.749+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~5IGA1YRZKAW5.1 size=1048576 2026-02-24T22:10:43.751 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.749+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~5IGA1YRZKAW5.2 size=429936 2026-02-24T22:10:43.752 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.749+0000 7f7e64854980 1 < completed multipart upload key=HOHKR92SN25M upload=5IGA1YRZKAW5 tag=9UC12X3R31V0 type=write size=3575664 2026-02-24T22:10:43.753 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.753+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~REDJX3E3FM1T.0 size=1048576 2026-02-24T22:10:43.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.753+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~REDJX3E3FM1T.1 size=1048576 2026-02-24T22:10:43.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.753+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~REDJX3E3FM1T.2 size=557349 2026-02-24T22:10:43.755 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.753+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=REDJX3E3FM1T tag=E4IR72XXXZVQ type=write size=3703077 2026-02-24T22:10:43.755 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.753+0000 7f7e64854980 1 < completed delete operation key=NL60Q5QOSP8E tag=H6ZJSVV23OHS type=del 2026-02-24T22:10:43.756 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.753+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=E6IZ40SI1059 type=del size=4105294 2026-02-24T22:10:43.757 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.753+0000 7f7e64854980 1 < suggested update operation key=TXRUQQ456DD8 tag=56EXXOWM64FB type=del size=1589041 2026-02-24T22:10:43.758 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.757+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=RYODWA3HO74R type=write size=537881 2026-02-24T22:10:43.759 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.757+0000 7f7e64854980 1 < suggested update operation key=GH7723GNI2WM tag=YBCDCPBKV041 type=del size=222502 2026-02-24T22:10:43.760 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.757+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=OUJH43UV9BJA type=del size=3230613 2026-02-24T22:10:43.761 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.761+0000 7f7e64854980 1 < suggested update operation key=L60HQ7M3E3LL tag=S8DWENXGQUTN type=del size=728249 2026-02-24T22:10:43.762 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.761+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=HC5ZM0ZI9S2K type=write size=685599 2026-02-24T22:10:43.763 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.761+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~TAKM937W7I52.0 size=1048576 2026-02-24T22:10:43.763 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.761+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~TAKM937W7I52.1 size=1048576 2026-02-24T22:10:43.764 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.761+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~TAKM937W7I52.2 size=396424 2026-02-24T22:10:43.765 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.761+0000 7f7e64854980 1 < completed multipart upload key=LSRPN33Z2B9N upload=TAKM937W7I52 tag=QPFB3I61NV5C type=write size=3542152 2026-02-24T22:10:43.765 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.765+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=BFGMH6M89PMS type=del size=2596391 2026-02-24T22:10:43.766 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.765+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=B2ZBABT147SE type=del 2026-02-24T22:10:43.767 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.765+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=ROVJXH01I2AB type=del size=1576512 2026-02-24T22:10:43.767 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.765+0000 7f7e64854980 1 < completed delete operation key=4Y3974PYYT78 tag=WTHLZWE1GIQ9 type=del 2026-02-24T22:10:43.768 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.765+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=T2DIF9S8L6FV type=del size=369996 2026-02-24T22:10:43.769 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.765+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~W7NJOFMPIO36.0 size=1048576 2026-02-24T22:10:43.769 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.765+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~W7NJOFMPIO36.1 size=1048576 2026-02-24T22:10:43.770 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.769+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~W7NJOFMPIO36.2 size=1042727 2026-02-24T22:10:43.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.769+0000 7f7e64854980 1 < completed multipart upload key=EB04ETNYM087 upload=W7NJOFMPIO36 tag=0FHFTEB7XB9X type=write size=4188455 2026-02-24T22:10:43.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.769+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~EC27E3L0XCXK.0 size=1048576 2026-02-24T22:10:43.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.769+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~EC27E3L0XCXK.1 size=1048576 2026-02-24T22:10:43.773 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.769+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~EC27E3L0XCXK.2 size=529349 2026-02-24T22:10:43.773 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.769+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=EC27E3L0XCXK tag=7LPNUE7FQJ9Y type=write size=3675077 2026-02-24T22:10:43.773 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.773+0000 7f7e64854980 1 < completed write operation key=4Y3974PYYT78 tag=2NSZT43UQK33 type=write size=710024 2026-02-24T22:10:43.775 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.773+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~1TYVCOP0THGE.0 size=447631 2026-02-24T22:10:43.775 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.773+0000 7f7e64854980 1 > prepared multipart upload key=Z0OXREAUJVA8 upload=1TYVCOP0THGE tag=UDEW386CJX9A type=write size=1496207 2026-02-24T22:10:43.775 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.773+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~SD4LIJIO8AMO.0 size=1048576 2026-02-24T22:10:43.776 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.773+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~SD4LIJIO8AMO.1 size=279377 2026-02-24T22:10:43.777 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.773+0000 7f7e64854980 1 < completed multipart upload key=N3ZUNSI5K8HS upload=SD4LIJIO8AMO tag=YK2ESHUJP978 type=write size=2376529 2026-02-24T22:10:43.778 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.777+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~6Q5OC2MNPW3T.0 size=995969 2026-02-24T22:10:43.778 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.777+0000 7f7e64854980 1 > prepared multipart upload key=LSRPN33Z2B9N upload=6Q5OC2MNPW3T tag=U92GT4GCCJPT type=write size=2044545 2026-02-24T22:10:43.779 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.777+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~WNR7L881FJWC.0 size=1048576 2026-02-24T22:10:43.779 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.777+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~WNR7L881FJWC.1 size=446605 2026-02-24T22:10:43.780 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.777+0000 7f7e64854980 1 < completed multipart upload key=GH7723GNI2WM upload=WNR7L881FJWC tag=VTFFEN61XOOM type=write size=2543757 2026-02-24T22:10:43.781 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.777+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=YZZ7AS0MM4UL type=write size=2126 2026-02-24T22:10:43.781 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.781+0000 7f7e64854980 1 < completed delete operation key=UQLUT9ZAI61V tag=LDKUQWGIQK0G type=del 2026-02-24T22:10:43.782 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.781+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=4B8L5RKG38VO type=del size=3371698 2026-02-24T22:10:43.783 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.781+0000 7f7e64854980 1 < completed delete operation key=L9NEDH44BIGA tag=3P3QYV041GSD type=del 2026-02-24T22:10:43.783 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.781+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=18EIQXFKNVDD type=del size=670104 2026-02-24T22:10:43.784 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.781+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~YYHSVSYCEVSK.0 size=865313 2026-02-24T22:10:43.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.781+0000 7f7e64854980 1 < completed multipart upload key=GH7723GNI2WM upload=YYHSVSYCEVSK tag=HH2FHJ9HHM4K type=write size=1913889 2026-02-24T22:10:43.786 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.785+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=QAGGGOZB8Q43 type=del size=2858463 2026-02-24T22:10:43.786 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.785+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~Z7PI501KT38G.0 size=629284 2026-02-24T22:10:43.787 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.785+0000 7f7e64854980 1 < completed multipart upload key=SGQYQ2TIVJPS upload=Z7PI501KT38G tag=G03I2OFB5UWU type=write size=1677860 2026-02-24T22:10:43.788 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.785+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=Q6ZSXYT5NS7R type=del size=3575675 2026-02-24T22:10:43.788 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.785+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~REDJX3E3FM1T.0 size=1048576 2026-02-24T22:10:43.789 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.785+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~REDJX3E3FM1T.1 size=1048576 2026-02-24T22:10:43.790 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.789+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~REDJX3E3FM1T.2 size=557349 2026-02-24T22:10:43.791 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.789+0000 7f7e64854980 1 < completed multipart upload key=D0MKGNRDW6L9 upload=REDJX3E3FM1T tag=E4IR72XXXZVQ type=write size=3703077 2026-02-24T22:10:43.791 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.789+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=62IJSOWLCCST type=del size=1798819 2026-02-24T22:10:43.792 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.789+0000 7f7e64854980 1 < completed delete operation key=HKOQRSF6XEXB tag=E6IZ40SI1059 type=del 2026-02-24T22:10:43.793 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.789+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~INM6GSM70N4P.0 size=543313 2026-02-24T22:10:43.793 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.789+0000 7f7e64854980 1 > prepared multipart upload key=N3ZUNSI5K8HS upload=INM6GSM70N4P tag=OS9BVTQ878XJ type=write size=1591889 2026-02-24T22:10:43.794 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.793+0000 7f7e64854980 1 < completed write operation key=CI8DA17DX8LU tag=RYODWA3HO74R type=write size=537881 2026-02-24T22:10:43.795 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.793+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~KVC8FTMK397Y.0 size=1048576 2026-02-24T22:10:43.795 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.793+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~KVC8FTMK397Y.1 size=1048576 2026-02-24T22:10:43.796 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.793+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~KVC8FTMK397Y.2 size=60519 2026-02-24T22:10:43.796 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.793+0000 7f7e64854980 1 > prepared multipart upload key=SGQYQ2TIVJPS upload=KVC8FTMK397Y tag=8PSRBGZGJ68W type=write size=3206247 2026-02-24T22:10:43.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.793+0000 7f7e64854980 1 < completed delete operation key=78GUTC5THI2P tag=OUJH43UV9BJA type=del 2026-02-24T22:10:43.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.793+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=C8YIIQT20BKT type=del size=2408215 2026-02-24T22:10:43.798 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.797+0000 7f7e64854980 1 < suggested update operation key=LSRPN33Z2B9N tag=HC5ZM0ZI9S2K type=write size=685599 2026-02-24T22:10:43.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.797+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~7GKL23QCUJ5J.0 size=1048576 2026-02-24T22:10:43.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.797+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~7GKL23QCUJ5J.1 size=1048576 2026-02-24T22:10:43.801 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.797+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~7GKL23QCUJ5J.2 size=541822 2026-02-24T22:10:43.801 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.797+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=7GKL23QCUJ5J tag=124A7ZL80CSW type=write size=3687550 2026-02-24T22:10:43.801 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.801+0000 7f7e64854980 -1 < failed to complete operation key=LSRPN33Z2B9N tag=BFGMH6M89PMS type=del size=2596391: (22) Invalid argument 2026-02-24T22:10:43.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.801+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=79ASFPMC7HHB type=del size=1715252 2026-02-24T22:10:43.803 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.801+0000 7f7e64854980 1 < completed delete operation key=L9NEDH44BIGA tag=ROVJXH01I2AB type=del 2026-02-24T22:10:43.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.801+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~KZX203563JWX.0 size=557777 2026-02-24T22:10:43.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.801+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=KZX203563JWX tag=GWXL3HOXQO96 type=write size=1606353 2026-02-24T22:10:43.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.801+0000 7f7e64854980 1 < suggested remove operation key=78GUTC5THI2P tag=T2DIF9S8L6FV type=del 2026-02-24T22:10:43.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.805+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=OIDZ9PVTEEGB type=del size=3513983 2026-02-24T22:10:43.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.805+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~EC27E3L0XCXK.0 size=1048576 2026-02-24T22:10:43.807 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.805+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~EC27E3L0XCXK.1 size=1048576 2026-02-24T22:10:43.807 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.805+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~EC27E3L0XCXK.2 size=529349 2026-02-24T22:10:43.808 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.805+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=EC27E3L0XCXK tag=7LPNUE7FQJ9Y type=write size=3675077 2026-02-24T22:10:43.809 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.805+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=U056JW8G23Z7 type=del size=2813202 2026-02-24T22:10:43.810 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.809+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~1TYVCOP0THGE.0 size=447631 2026-02-24T22:10:43.811 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.809+0000 7f7e64854980 1 < completed multipart upload key=Z0OXREAUJVA8 upload=1TYVCOP0THGE tag=UDEW386CJX9A type=write size=1496207 2026-02-24T22:10:43.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.809+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~V22BWQ0KQPX4.0 size=1048576 2026-02-24T22:10:43.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.809+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~V22BWQ0KQPX4.1 size=515092 2026-02-24T22:10:43.813 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.809+0000 7f7e64854980 1 > prepared multipart upload key=Q7BP8FXTARDW upload=V22BWQ0KQPX4 tag=A50AZLALHIWM type=write size=2612244 2026-02-24T22:10:43.813 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.809+0000 7f7e64854980 1 < canceled part key=_multipart_LSRPN33Z2B9N.2~6Q5OC2MNPW3T.0 size=995969 2026-02-24T22:10:43.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.813+0000 7f7e64854980 1 < canceled multipart upload key=LSRPN33Z2B9N upload=6Q5OC2MNPW3T tag=U92GT4GCCJPT type=write size=2044545 2026-02-24T22:10:43.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.813+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=PBWATXM8VKA9 type=del size=3839874 2026-02-24T22:10:43.815 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.813+0000 7f7e64854980 1 < completed write operation key=Y3L3CJ51C1WR tag=YZZ7AS0MM4UL type=write size=2126 2026-02-24T22:10:43.815 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.813+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=KWCOK948RRQH type=del size=2430530 2026-02-24T22:10:43.816 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.813+0000 7f7e64854980 1 < completed delete operation key=EJCX06TE5RTZ tag=4B8L5RKG38VO type=del 2026-02-24T22:10:43.817 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.813+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~CCZRECE43PCB.0 size=1048576 2026-02-24T22:10:43.818 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.817+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~CCZRECE43PCB.1 size=1048576 2026-02-24T22:10:43.819 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.817+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~CCZRECE43PCB.2 size=241961 2026-02-24T22:10:43.819 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.817+0000 7f7e64854980 1 > prepared multipart upload key=L60HQ7M3E3LL upload=CCZRECE43PCB tag=0ELMBEL7EKQ4 type=write size=3387689 2026-02-24T22:10:43.820 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.817+0000 7f7e64854980 1 < suggested remove operation key=TF5B43G7KCYT tag=18EIQXFKNVDD type=del 2026-02-24T22:10:43.821 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.817+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~WUE5NM2DJQ03.0 size=329752 2026-02-24T22:10:43.821 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.817+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=WUE5NM2DJQ03 tag=TAAFNFF2V184 type=write size=1378328 2026-02-24T22:10:43.822 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.821+0000 7f7e64854980 1 < completed delete operation key=N3ZUNSI5K8HS tag=QAGGGOZB8Q43 type=del 2026-02-24T22:10:43.823 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.821+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~RCYBDBCSWUJX.0 size=1048576 2026-02-24T22:10:43.823 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.821+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~RCYBDBCSWUJX.1 size=1048576 2026-02-24T22:10:43.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.821+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~RCYBDBCSWUJX.2 size=244468 2026-02-24T22:10:43.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.821+0000 7f7e64854980 1 > prepared multipart upload key=ON16L53JUSTW upload=RCYBDBCSWUJX tag=9SO3AUOY10N9 type=write size=3390196 2026-02-24T22:10:43.825 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.821+0000 7f7e64854980 1 < completed delete operation key=EB04ETNYM087 tag=Q6ZSXYT5NS7R type=del 2026-02-24T22:10:43.825 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.821+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=ECS5P9PQYG5M type=del size=2295406 2026-02-24T22:10:43.826 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.825+0000 7f7e64854980 1 < completed delete operation key=HKOQRSF6XEXB tag=62IJSOWLCCST type=del 2026-02-24T22:10:43.826 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.825+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=P30PGBPVNXF7 type=del size=2995872 2026-02-24T22:10:43.827 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.825+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~INM6GSM70N4P.0 size=543313 2026-02-24T22:10:43.828 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.825+0000 7f7e64854980 1 < completed multipart upload key=N3ZUNSI5K8HS upload=INM6GSM70N4P tag=OS9BVTQ878XJ type=write size=1591889 2026-02-24T22:10:43.829 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.825+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=BWYKQX2BUJ7Q type=del size=4023581 2026-02-24T22:10:43.829 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.829+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~KVC8FTMK397Y.0 size=1048576 2026-02-24T22:10:43.830 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.829+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~KVC8FTMK397Y.1 size=1048576 2026-02-24T22:10:43.830 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.829+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~KVC8FTMK397Y.2 size=60519 2026-02-24T22:10:43.831 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.829+0000 7f7e64854980 1 < completed multipart upload key=SGQYQ2TIVJPS upload=KVC8FTMK397Y tag=8PSRBGZGJ68W type=write size=3206247 2026-02-24T22:10:43.833 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.829+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~HO8ULVW3BHFP.0 size=1048576 2026-02-24T22:10:43.833 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.829+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~HO8ULVW3BHFP.1 size=1048576 2026-02-24T22:10:43.834 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.833+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~HO8ULVW3BHFP.2 size=547647 2026-02-24T22:10:43.834 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.833+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=HO8ULVW3BHFP tag=T2ZR3TWPU3AC type=write size=3693375 2026-02-24T22:10:43.834 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.833+0000 7f7e64854980 1 < completed delete operation key=L9NEDH44BIGA tag=C8YIIQT20BKT type=del 2026-02-24T22:10:43.835 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.833+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=XYBO3Q5NYN6F type=del size=280314 2026-02-24T22:10:43.835 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.833+0000 7f7e64854980 1 < canceled part key=_multipart_2KKQKHXO7K3K.2~7GKL23QCUJ5J.0 size=1048576 2026-02-24T22:10:43.835 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.833+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~7GKL23QCUJ5J.1 size=1048576 2026-02-24T22:10:43.836 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.833+0000 7f7e64854980 1 < canceled part key=_multipart_2KKQKHXO7K3K.2~7GKL23QCUJ5J.2 size=541822 2026-02-24T22:10:43.836 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.833+0000 7f7e64854980 1 < canceled multipart upload key=2KKQKHXO7K3K upload=7GKL23QCUJ5J tag=124A7ZL80CSW type=write size=3687550 2026-02-24T22:10:43.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.833+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=O6NRFE8C6EL4 type=del size=4087862 2026-02-24T22:10:43.838 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.837+0000 7f7e64854980 1 < completed delete operation key=3900SFJK1YQU tag=79ASFPMC7HHB type=del 2026-02-24T22:10:43.839 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.837+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~CZVJLIMIZT5S.0 size=1048576 2026-02-24T22:10:43.839 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.837+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~CZVJLIMIZT5S.1 size=1032650 2026-02-24T22:10:43.839 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.837+0000 7f7e64854980 1 > prepared multipart upload key=GH7723GNI2WM upload=CZVJLIMIZT5S tag=10ZSL3ZAAJSN type=write size=3129802 2026-02-24T22:10:43.840 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.837+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~KZX203563JWX.0 size=557777 2026-02-24T22:10:43.841 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.837+0000 7f7e64854980 1 < completed multipart upload key=TXRUQQ456DD8 upload=KZX203563JWX tag=GWXL3HOXQO96 type=write size=1606353 2026-02-24T22:10:43.841 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.841+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=YP2RQIHPQVFL type=del size=1214989 2026-02-24T22:10:43.842 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.841+0000 7f7e64854980 1 < canceled operation key=TXRUQQ456DD8 tag=OIDZ9PVTEEGB type=del size=3513983 2026-02-24T22:10:43.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.841+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~TEBHPZV4CUZT.0 size=1048576 2026-02-24T22:10:43.844 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.841+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~TEBHPZV4CUZT.1 size=308318 2026-02-24T22:10:43.844 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.841+0000 7f7e64854980 1 > prepared multipart upload key=NL60Q5QOSP8E upload=TEBHPZV4CUZT tag=L9DB4V9UZV7M type=write size=2405470 2026-02-24T22:10:43.845 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.841+0000 7f7e64854980 1 < suggested update operation key=4Y3974PYYT78 tag=U056JW8G23Z7 type=del size=2813202 2026-02-24T22:10:43.846 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.845+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~9DWHTYYFQBCZ.0 size=1048576 2026-02-24T22:10:43.847 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.845+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~9DWHTYYFQBCZ.1 size=1048576 2026-02-24T22:10:43.847 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.845+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~9DWHTYYFQBCZ.2 size=265612 2026-02-24T22:10:43.847 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.845+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=9DWHTYYFQBCZ tag=ZD6XZJ1XIVW2 type=write size=3411340 2026-02-24T22:10:43.848 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.845+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~V22BWQ0KQPX4.0 size=1048576 2026-02-24T22:10:43.848 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.845+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~V22BWQ0KQPX4.1 size=515092 2026-02-24T22:10:43.849 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.849+0000 7f7e64854980 1 < completed multipart upload key=Q7BP8FXTARDW upload=V22BWQ0KQPX4 tag=A50AZLALHIWM type=write size=2612244 2026-02-24T22:10:43.850 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.849+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~2BD0R21R59O8.0 size=315979 2026-02-24T22:10:43.851 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.849+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=2BD0R21R59O8 tag=V233KQGZNI2V type=write size=1364555 2026-02-24T22:10:43.852 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.849+0000 7f7e64854980 1 < suggested remove operation key=L60HQ7M3E3LL tag=PBWATXM8VKA9 type=del 2026-02-24T22:10:43.852 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.849+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=DHRMZQSU6R4R type=del size=1639750 2026-02-24T22:10:43.853 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.849+0000 7f7e64854980 1 < completed delete operation key=HKOQRSF6XEXB tag=KWCOK948RRQH type=del 2026-02-24T22:10:43.853 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.853+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=MI45HDG0274R type=del size=1210131 2026-02-24T22:10:43.854 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.853+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~CCZRECE43PCB.0 size=1048576 2026-02-24T22:10:43.855 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.853+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~CCZRECE43PCB.1 size=1048576 2026-02-24T22:10:43.855 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.853+0000 7f7e64854980 1 < canceled part key=_multipart_L60HQ7M3E3LL.2~CCZRECE43PCB.2 size=241961 2026-02-24T22:10:43.856 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.853+0000 7f7e64854980 1 < canceled multipart upload key=L60HQ7M3E3LL upload=CCZRECE43PCB tag=0ELMBEL7EKQ4 type=write size=3387689 2026-02-24T22:10:43.856 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.853+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=R2MUO2UIATO8 type=del size=1779657 2026-02-24T22:10:43.857 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.853+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~WUE5NM2DJQ03.0 size=329752 2026-02-24T22:10:43.858 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.857+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=WUE5NM2DJQ03 tag=TAAFNFF2V184 type=write size=1378328 2026-02-24T22:10:43.858 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.857+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=P6VF9FQ53HCK type=del size=2791952 2026-02-24T22:10:43.859 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.857+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~RCYBDBCSWUJX.0 size=1048576 2026-02-24T22:10:43.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.857+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~RCYBDBCSWUJX.1 size=1048576 2026-02-24T22:10:43.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.857+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~RCYBDBCSWUJX.2 size=244468 2026-02-24T22:10:43.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.857+0000 7f7e64854980 1 < completed multipart upload key=ON16L53JUSTW upload=RCYBDBCSWUJX tag=9SO3AUOY10N9 type=write size=3390196 2026-02-24T22:10:43.862 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.861+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~L8LNN11K6GZ7.0 size=944222 2026-02-24T22:10:43.862 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.861+0000 7f7e64854980 1 > prepared multipart upload key=0B2V0MHWSOC9 upload=L8LNN11K6GZ7 tag=TIUE8QPFN9G4 type=write size=1992798 2026-02-24T22:10:43.863 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.861+0000 7f7e64854980 -1 < failed to complete operation key=L60HQ7M3E3LL tag=ECS5P9PQYG5M type=del size=2295406: (22) Invalid argument 2026-02-24T22:10:43.864 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.861+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~F9OIXWJAOL4N.0 size=435565 2026-02-24T22:10:43.864 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.861+0000 7f7e64854980 1 > prepared multipart upload key=Q7BP8FXTARDW upload=F9OIXWJAOL4N tag=KSB80516ZP26 type=write size=1484141 2026-02-24T22:10:43.865 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.861+0000 7f7e64854980 1 < completed delete operation key=LSRPN33Z2B9N tag=P30PGBPVNXF7 type=del 2026-02-24T22:10:43.865 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.865+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=MRYWWZ17HMTW type=del size=2613191 2026-02-24T22:10:43.866 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.865+0000 7f7e64854980 -1 < failed to complete operation key=L60HQ7M3E3LL tag=BWYKQX2BUJ7Q type=del size=4023581: (22) Invalid argument 2026-02-24T22:10:43.867 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.865+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=1DJW89JYHG13 type=del size=2203537 2026-02-24T22:10:43.867 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.865+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~HO8ULVW3BHFP.0 size=1048576 2026-02-24T22:10:43.868 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.865+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~HO8ULVW3BHFP.1 size=1048576 2026-02-24T22:10:43.868 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.865+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~HO8ULVW3BHFP.2 size=547647 2026-02-24T22:10:43.869 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.869+0000 7f7e64854980 1 < completed multipart upload key=TXRUQQ456DD8 upload=HO8ULVW3BHFP tag=T2ZR3TWPU3AC type=write size=3693375 2026-02-24T22:10:43.870 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.869+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=R3KJUEHGBQ2P type=del size=978155 2026-02-24T22:10:43.871 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.869+0000 7f7e64854980 1 < completed delete operation key=EB04ETNYM087 tag=XYBO3Q5NYN6F type=del 2026-02-24T22:10:43.871 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.869+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=4112CVCSUV36 type=del size=2992932 2026-02-24T22:10:43.872 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.869+0000 7f7e64854980 1 < completed delete operation key=NL60Q5QOSP8E tag=O6NRFE8C6EL4 type=del 2026-02-24T22:10:43.873 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.869+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=UWE13N1J6ILP type=del size=2842969 2026-02-24T22:10:43.873 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.869+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~CZVJLIMIZT5S.0 size=1048576 2026-02-24T22:10:43.874 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.873+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~CZVJLIMIZT5S.1 size=1032650 2026-02-24T22:10:43.875 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.873+0000 7f7e64854980 1 < completed multipart upload key=GH7723GNI2WM upload=CZVJLIMIZT5S tag=10ZSL3ZAAJSN type=write size=3129802 2026-02-24T22:10:43.876 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.873+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=ROP7E6EP1IWE type=del size=1833457 2026-02-24T22:10:43.877 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.873+0000 7f7e64854980 1 < completed delete operation key=3900SFJK1YQU tag=YP2RQIHPQVFL type=del 2026-02-24T22:10:43.878 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.877+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~T0TDA1A0QU6H.0 size=1048576 2026-02-24T22:10:43.878 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.877+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~T0TDA1A0QU6H.1 size=207674 2026-02-24T22:10:43.878 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.877+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=T0TDA1A0QU6H tag=5CENW2XPSSOO type=write size=2304826 2026-02-24T22:10:43.879 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.877+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~TEBHPZV4CUZT.0 size=1048576 2026-02-24T22:10:43.879 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.877+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~TEBHPZV4CUZT.1 size=308318 2026-02-24T22:10:43.880 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.877+0000 7f7e64854980 1 < completed multipart upload key=NL60Q5QOSP8E upload=TEBHPZV4CUZT tag=L9DB4V9UZV7M type=write size=2405470 2026-02-24T22:10:43.881 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.877+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=OCEOO30MOSQY type=del size=3191220 2026-02-24T22:10:43.882 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.881+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~9DWHTYYFQBCZ.0 size=1048576 2026-02-24T22:10:43.882 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.881+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~9DWHTYYFQBCZ.1 size=1048576 2026-02-24T22:10:43.883 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.881+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~9DWHTYYFQBCZ.2 size=265612 2026-02-24T22:10:43.884 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.881+0000 7f7e64854980 1 < completed multipart upload key=D0MKGNRDW6L9 upload=9DWHTYYFQBCZ tag=ZD6XZJ1XIVW2 type=write size=3411340 2026-02-24T22:10:43.885 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.881+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~0AU7RKDU1MD1.0 size=956393 2026-02-24T22:10:43.885 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.881+0000 7f7e64854980 1 > prepared multipart upload key=ON16L53JUSTW upload=0AU7RKDU1MD1 tag=2STTIV6SSL9N type=write size=2004969 2026-02-24T22:10:43.885 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.885+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~2BD0R21R59O8.0 size=315979 2026-02-24T22:10:43.886 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.885+0000 7f7e64854980 1 < completed multipart upload key=ZESKGBWZXM6O upload=2BD0R21R59O8 tag=V233KQGZNI2V type=write size=1364555 2026-02-24T22:10:43.888 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.885+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~8QOOZOUJQB4T.0 size=366751 2026-02-24T22:10:43.888 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.885+0000 7f7e64854980 1 > prepared multipart upload key=LSRPN33Z2B9N upload=8QOOZOUJQB4T tag=MFPUUHCE6NE4 type=write size=1415327 2026-02-24T22:10:43.889 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.885+0000 7f7e64854980 1 < suggested remove operation key=EJCX06TE5RTZ tag=DHRMZQSU6R4R type=del 2026-02-24T22:10:43.889 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.889+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=GJEO6D6RKGG1 type=del size=1016 2026-02-24T22:10:43.890 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.889+0000 7f7e64854980 1 < canceled operation key=A6PH77Q78VEN tag=MI45HDG0274R type=del size=1210131 2026-02-24T22:10:43.891 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.889+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~E21BRIOG9MZL.0 size=1048576 2026-02-24T22:10:43.892 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.889+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~E21BRIOG9MZL.1 size=1048576 2026-02-24T22:10:43.892 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.889+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~E21BRIOG9MZL.2 size=506456 2026-02-24T22:10:43.892 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.889+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=E21BRIOG9MZL tag=PMIDSL5UR1GO type=write size=3652184 2026-02-24T22:10:43.893 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.889+0000 7f7e64854980 1 < canceled operation key=3900SFJK1YQU tag=R2MUO2UIATO8 type=del size=1779657 2026-02-24T22:10:43.893 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.889+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=7YI4LI99SI70 type=del size=3719891 2026-02-24T22:10:43.894 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.893+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=P6VF9FQ53HCK type=del 2026-02-24T22:10:43.894 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.893+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=6V5N6E9YJG6U type=del size=286398 2026-02-24T22:10:43.895 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.893+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~L8LNN11K6GZ7.0 size=944222 2026-02-24T22:10:43.896 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.893+0000 7f7e64854980 1 < completed multipart upload key=0B2V0MHWSOC9 upload=L8LNN11K6GZ7 tag=TIUE8QPFN9G4 type=write size=1992798 2026-02-24T22:10:43.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.897+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~QH90U0JFHRFM.0 size=1048576 2026-02-24T22:10:43.898 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.897+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~QH90U0JFHRFM.1 size=1048576 2026-02-24T22:10:43.898 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.897+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~QH90U0JFHRFM.2 size=78065 2026-02-24T22:10:43.898 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.897+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=QH90U0JFHRFM tag=BZELTWBWVE96 type=write size=3223793 2026-02-24T22:10:43.899 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.897+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~F9OIXWJAOL4N.0 size=435565 2026-02-24T22:10:43.900 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.897+0000 7f7e64854980 1 < completed multipart upload key=Q7BP8FXTARDW upload=F9OIXWJAOL4N tag=KSB80516ZP26 type=write size=1484141 2026-02-24T22:10:43.901 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.897+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=U3BWIAEOTSU6 type=del size=2916052 2026-02-24T22:10:43.901 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.901+0000 7f7e64854980 1 < canceled operation key=4Y3974PYYT78 tag=MRYWWZ17HMTW type=del size=2613191 2026-02-24T22:10:43.902 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.901+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=PWJPCYLW4RPH type=del size=1650203 2026-02-24T22:10:43.903 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.901+0000 7f7e64854980 1 < completed delete operation key=GH7723GNI2WM tag=1DJW89JYHG13 type=del 2026-02-24T22:10:43.904 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.901+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~5HWHJ0V07YRV.0 size=1048576 2026-02-24T22:10:43.904 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.901+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~5HWHJ0V07YRV.1 size=1048576 2026-02-24T22:10:43.905 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.901+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~5HWHJ0V07YRV.2 size=762924 2026-02-24T22:10:43.905 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.901+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=5HWHJ0V07YRV tag=2TJ9R4NDIZO0 type=write size=3908652 2026-02-24T22:10:43.905 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.905+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=R3KJUEHGBQ2P type=del 2026-02-24T22:10:43.906 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.905+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=WOVEZFV3FTUI type=del size=3696961 2026-02-24T22:10:43.907 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.905+0000 7f7e64854980 1 < suggested update operation key=GH7723GNI2WM tag=4112CVCSUV36 type=del size=2992932 2026-02-24T22:10:43.908 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.905+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=S4MOKP699VH6 type=del size=1507955 2026-02-24T22:10:43.908 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.905+0000 7f7e64854980 1 < completed delete operation key=NL60Q5QOSP8E tag=UWE13N1J6ILP type=del 2026-02-24T22:10:43.909 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.905+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=R3UF3SU5Z8Y7 type=del size=3266504 2026-02-24T22:10:43.910 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.909+0000 7f7e64854980 1 < completed delete operation key=Y3L3CJ51C1WR tag=ROP7E6EP1IWE type=del 2026-02-24T22:10:43.910 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.909+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=KLPH1ZS5439N type=write size=650744 2026-02-24T22:10:43.911 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.909+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~T0TDA1A0QU6H.0 size=1048576 2026-02-24T22:10:43.911 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.909+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~T0TDA1A0QU6H.1 size=207674 2026-02-24T22:10:43.912 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.909+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=T0TDA1A0QU6H tag=5CENW2XPSSOO type=write size=2304826 2026-02-24T22:10:43.913 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.909+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=GDCJPHQL31G4 type=del size=3908319 2026-02-24T22:10:43.914 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.913+0000 7f7e64854980 1 < completed delete operation key=0B2V0MHWSOC9 tag=OCEOO30MOSQY type=del 2026-02-24T22:10:43.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.913+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~UTQ3759JDKIE.0 size=1048576 2026-02-24T22:10:43.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.913+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~UTQ3759JDKIE.1 size=1048576 2026-02-24T22:10:43.916 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.913+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~UTQ3759JDKIE.2 size=377102 2026-02-24T22:10:43.916 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.913+0000 7f7e64854980 1 > prepared multipart upload key=N3ZUNSI5K8HS upload=UTQ3759JDKIE tag=ZX8D2FGH53SW type=write size=3522830 2026-02-24T22:10:43.916 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.913+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~0AU7RKDU1MD1.0 size=956393 2026-02-24T22:10:43.917 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.917+0000 7f7e64854980 1 < completed multipart upload key=ON16L53JUSTW upload=0AU7RKDU1MD1 tag=2STTIV6SSL9N type=write size=2004969 2026-02-24T22:10:43.918 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.917+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~9S2A0ZGE66TV.0 size=1048576 2026-02-24T22:10:43.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.917+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~9S2A0ZGE66TV.1 size=529719 2026-02-24T22:10:43.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.917+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=9S2A0ZGE66TV tag=NGTIK8IT36QX type=write size=2626871 2026-02-24T22:10:43.920 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.917+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~8QOOZOUJQB4T.0 size=366751 2026-02-24T22:10:43.921 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.917+0000 7f7e64854980 1 < completed multipart upload key=LSRPN33Z2B9N upload=8QOOZOUJQB4T tag=MFPUUHCE6NE4 type=write size=1415327 2026-02-24T22:10:43.922 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.921+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~LRZ8CEJN97KT.0 size=1048576 2026-02-24T22:10:43.922 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.921+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~LRZ8CEJN97KT.1 size=1048576 2026-02-24T22:10:43.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.921+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~LRZ8CEJN97KT.2 size=338323 2026-02-24T22:10:43.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.921+0000 7f7e64854980 1 > prepared multipart upload key=3900SFJK1YQU upload=LRZ8CEJN97KT tag=LWE0SXRTOALE type=write size=3484051 2026-02-24T22:10:43.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.921+0000 7f7e64854980 1 < completed delete operation key=3900SFJK1YQU tag=GJEO6D6RKGG1 type=del 2026-02-24T22:10:43.925 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.921+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~4JQ3AYN10KNI.0 size=380143 2026-02-24T22:10:43.925 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.921+0000 7f7e64854980 1 > prepared multipart upload key=Y8GMOUA977H3 upload=4JQ3AYN10KNI tag=KSSVPWS5POYT type=write size=1428719 2026-02-24T22:10:43.925 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.921+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~E21BRIOG9MZL.0 size=1048576 2026-02-24T22:10:43.926 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.925+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~E21BRIOG9MZL.1 size=1048576 2026-02-24T22:10:43.926 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.925+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~E21BRIOG9MZL.2 size=506456 2026-02-24T22:10:43.927 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.925+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=E21BRIOG9MZL tag=PMIDSL5UR1GO type=write size=3652184 2026-02-24T22:10:43.928 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.925+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~SL33A9TD7JCV.0 size=1048576 2026-02-24T22:10:43.929 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.925+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~SL33A9TD7JCV.1 size=1048576 2026-02-24T22:10:43.929 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.929+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~SL33A9TD7JCV.2 size=518399 2026-02-24T22:10:43.929 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.929+0000 7f7e64854980 1 > prepared multipart upload key=HKOQRSF6XEXB upload=SL33A9TD7JCV tag=CQ0BC1FH02DC type=write size=3664127 2026-02-24T22:10:43.930 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.929+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=7YI4LI99SI70 type=del 2026-02-24T22:10:43.931 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.929+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=75LUSOY4Z61M type=del size=1078929 2026-02-24T22:10:43.931 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.929+0000 7f7e64854980 1 < completed delete operation key=HKOQRSF6XEXB tag=6V5N6E9YJG6U type=del 2026-02-24T22:10:43.932 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.929+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=78JDJ3KURMJ5 type=del size=444833 2026-02-24T22:10:43.932 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.929+0000 7f7e64854980 1 < canceled part key=_multipart_2KKQKHXO7K3K.2~QH90U0JFHRFM.0 size=1048576 2026-02-24T22:10:43.932 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.929+0000 7f7e64854980 1 < canceled part key=_multipart_2KKQKHXO7K3K.2~QH90U0JFHRFM.1 size=1048576 2026-02-24T22:10:43.933 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.929+0000 7f7e64854980 1 < canceled part key=_multipart_2KKQKHXO7K3K.2~QH90U0JFHRFM.2 size=78065 2026-02-24T22:10:43.934 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.933+0000 7f7e64854980 1 < canceled multipart upload key=2KKQKHXO7K3K upload=QH90U0JFHRFM tag=BZELTWBWVE96 type=write size=3223793 2026-02-24T22:10:43.934 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.933+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=F6NMEQNV25Q9 type=del size=3424328 2026-02-24T22:10:43.935 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.933+0000 7f7e64854980 1 < completed delete operation key=3900SFJK1YQU tag=U3BWIAEOTSU6 type=del 2026-02-24T22:10:43.936 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.933+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=YF4KXUKNNW8N type=del size=3041642 2026-02-24T22:10:43.936 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.933+0000 7f7e64854980 1 < completed delete operation key=L60HQ7M3E3LL tag=PWJPCYLW4RPH type=del 2026-02-24T22:10:43.937 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.933+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=QX9C33ERV68H type=write size=652864 2026-02-24T22:10:43.938 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.937+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~5HWHJ0V07YRV.0 size=1048576 2026-02-24T22:10:43.938 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.937+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~5HWHJ0V07YRV.1 size=1048576 2026-02-24T22:10:43.939 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.937+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~5HWHJ0V07YRV.2 size=762924 2026-02-24T22:10:43.940 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.937+0000 7f7e64854980 1 < completed multipart upload key=TXRUQQ456DD8 upload=5HWHJ0V07YRV tag=2TJ9R4NDIZO0 type=write size=3908652 2026-02-24T22:10:43.940 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.937+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=J16L6SFJOS0A type=del size=2734997 2026-02-24T22:10:43.941 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.937+0000 7f7e64854980 1 < canceled operation key=2KKQKHXO7K3K tag=WOVEZFV3FTUI type=del size=3696961 2026-02-24T22:10:43.942 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.941+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~BOW80XXOKS0E.0 size=698306 2026-02-24T22:10:43.942 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.941+0000 7f7e64854980 1 > prepared multipart upload key=78GUTC5THI2P upload=BOW80XXOKS0E tag=JJKCO3KHJ5L0 type=write size=1746882 2026-02-24T22:10:43.943 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.941+0000 7f7e64854980 1 < suggested remove operation key=HOHKR92SN25M tag=S4MOKP699VH6 type=del 2026-02-24T22:10:43.944 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.941+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=EVI1BNGDFMBM type=del size=1668489 2026-02-24T22:10:43.945 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.941+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=R3UF3SU5Z8Y7 type=del 2026-02-24T22:10:43.945 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.941+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=WV3GUMZZ6SHU type=write size=832690 2026-02-24T22:10:43.946 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.945+0000 7f7e64854980 1 < suggested update operation key=CI8DA17DX8LU tag=KLPH1ZS5439N type=write size=650744 2026-02-24T22:10:43.947 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.945+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=GI2D0229IF83 type=del size=1284784 2026-02-24T22:10:43.948 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.945+0000 7f7e64854980 1 < suggested remove operation key=L60HQ7M3E3LL tag=GDCJPHQL31G4 type=del 2026-02-24T22:10:43.949 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.945+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=AOY4OD8E92CO type=del size=4112097 2026-02-24T22:10:43.950 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.949+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~UTQ3759JDKIE.0 size=1048576 2026-02-24T22:10:43.950 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.949+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~UTQ3759JDKIE.1 size=1048576 2026-02-24T22:10:43.950 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.949+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~UTQ3759JDKIE.2 size=377102 2026-02-24T22:10:43.952 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.949+0000 7f7e64854980 1 < completed multipart upload key=N3ZUNSI5K8HS upload=UTQ3759JDKIE tag=ZX8D2FGH53SW type=write size=3522830 2026-02-24T22:10:43.952 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.949+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=YPJT3QK1DZGA type=del size=2530869 2026-02-24T22:10:43.953 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.949+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~9S2A0ZGE66TV.0 size=1048576 2026-02-24T22:10:43.953 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.953+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~9S2A0ZGE66TV.1 size=529719 2026-02-24T22:10:43.954 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.953+0000 7f7e64854980 1 < completed multipart upload key=EJCX06TE5RTZ upload=9S2A0ZGE66TV tag=NGTIK8IT36QX type=write size=2626871 2026-02-24T22:10:43.955 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.953+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=L2LWFQAY4AQ0 type=del size=2031065 2026-02-24T22:10:43.956 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.953+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~LRZ8CEJN97KT.0 size=1048576 2026-02-24T22:10:43.956 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.953+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~LRZ8CEJN97KT.1 size=1048576 2026-02-24T22:10:43.957 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.953+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~LRZ8CEJN97KT.2 size=338323 2026-02-24T22:10:43.958 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.957+0000 7f7e64854980 1 < completed multipart upload key=3900SFJK1YQU upload=LRZ8CEJN97KT tag=LWE0SXRTOALE type=write size=3484051 2026-02-24T22:10:43.958 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.957+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=25U0B95D3QQ4 type=del size=4054645 2026-02-24T22:10:43.959 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.957+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~4JQ3AYN10KNI.0 size=380143 2026-02-24T22:10:43.960 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.957+0000 7f7e64854980 1 < completed multipart upload key=Y8GMOUA977H3 upload=4JQ3AYN10KNI tag=KSSVPWS5POYT type=write size=1428719 2026-02-24T22:10:43.961 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.961+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~P2WOKTD40L3E.0 size=1048576 2026-02-24T22:10:43.962 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.961+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~P2WOKTD40L3E.1 size=1048576 2026-02-24T22:10:43.962 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.961+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~P2WOKTD40L3E.2 size=457754 2026-02-24T22:10:43.962 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.961+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=P2WOKTD40L3E tag=43M8UKA8CU39 type=write size=3603482 2026-02-24T22:10:43.963 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.961+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~SL33A9TD7JCV.0 size=1048576 2026-02-24T22:10:43.963 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.961+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~SL33A9TD7JCV.1 size=1048576 2026-02-24T22:10:43.963 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.961+0000 7f7e64854980 1 < canceled part key=_multipart_HKOQRSF6XEXB.2~SL33A9TD7JCV.2 size=518399 2026-02-24T22:10:43.964 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.961+0000 7f7e64854980 1 < canceled multipart upload key=HKOQRSF6XEXB upload=SL33A9TD7JCV tag=CQ0BC1FH02DC type=write size=3664127 2026-02-24T22:10:43.965 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.961+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=BK8Z39K1Z9FE type=del size=615196 2026-02-24T22:10:43.966 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.965+0000 7f7e64854980 1 < completed delete operation key=NL60Q5QOSP8E tag=75LUSOY4Z61M type=del 2026-02-24T22:10:43.967 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.965+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~DFIY40JMM8QC.0 size=1048576 2026-02-24T22:10:43.967 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.965+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~DFIY40JMM8QC.1 size=1048576 2026-02-24T22:10:43.968 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.965+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~DFIY40JMM8QC.2 size=396279 2026-02-24T22:10:43.968 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.965+0000 7f7e64854980 1 > prepared multipart upload key=LSRPN33Z2B9N upload=DFIY40JMM8QC tag=JH359TZQ0MMO type=write size=3542007 2026-02-24T22:10:43.968 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.965+0000 7f7e64854980 1 < completed delete operation key=HKOQRSF6XEXB tag=78JDJ3KURMJ5 type=del 2026-02-24T22:10:43.970 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.969+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~M5U6XSLUIXK5.0 size=1048576 2026-02-24T22:10:43.970 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.969+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~M5U6XSLUIXK5.1 size=1048576 2026-02-24T22:10:43.971 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.969+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~M5U6XSLUIXK5.2 size=318595 2026-02-24T22:10:43.971 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.969+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=M5U6XSLUIXK5 tag=NDDAQT5BPKR3 type=write size=3464323 2026-02-24T22:10:43.971 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.969+0000 7f7e64854980 1 < completed delete operation key=ZESKGBWZXM6O tag=F6NMEQNV25Q9 type=del 2026-02-24T22:10:43.972 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.969+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=BNW0Y7UQXMHQ type=del size=404595 2026-02-24T22:10:43.972 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.969+0000 7f7e64854980 -1 < failed to complete operation key=HOHKR92SN25M tag=YF4KXUKNNW8N type=del size=3041642: (22) Invalid argument 2026-02-24T22:10:43.974 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.973+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~HX3OZXS48WU5.0 size=1048576 2026-02-24T22:10:43.974 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.973+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~HX3OZXS48WU5.1 size=1048576 2026-02-24T22:10:43.975 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.973+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~HX3OZXS48WU5.2 size=142449 2026-02-24T22:10:43.975 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.973+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=HX3OZXS48WU5 tag=1Y5NX5OEOHXT type=write size=3288177 2026-02-24T22:10:43.975 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.973+0000 7f7e64854980 1 < completed write operation key=EB04ETNYM087 tag=QX9C33ERV68H type=write size=652864 2026-02-24T22:10:43.976 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.973+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~BI8L0YNMKWY5.0 size=1048576 2026-02-24T22:10:43.977 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.973+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~BI8L0YNMKWY5.1 size=1048576 2026-02-24T22:10:43.977 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.977+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~BI8L0YNMKWY5.2 size=573103 2026-02-24T22:10:43.977 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.977+0000 7f7e64854980 1 > prepared multipart upload key=N3ZUNSI5K8HS upload=BI8L0YNMKWY5 tag=I9X3XGAQPFEX type=write size=3718831 2026-02-24T22:10:43.978 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.977+0000 7f7e64854980 1 < completed delete operation key=SGQYQ2TIVJPS tag=J16L6SFJOS0A type=del 2026-02-24T22:10:43.979 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.977+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=AUSTCYEYYX1S type=del size=1918944 2026-02-24T22:10:43.979 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.977+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~BOW80XXOKS0E.0 size=698306 2026-02-24T22:10:43.980 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.977+0000 7f7e64854980 1 < completed multipart upload key=78GUTC5THI2P upload=BOW80XXOKS0E tag=JJKCO3KHJ5L0 type=write size=1746882 2026-02-24T22:10:43.981 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.977+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=7C01TFXW33ES type=del size=3133045 2026-02-24T22:10:43.982 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.981+0000 7f7e64854980 1 < completed delete operation key=36MKR2OZN7K9 tag=EVI1BNGDFMBM type=del 2026-02-24T22:10:43.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.981+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~IM9J3EU5ZGTN.0 size=1048576 2026-02-24T22:10:43.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.981+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~IM9J3EU5ZGTN.1 size=781951 2026-02-24T22:10:43.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.981+0000 7f7e64854980 1 > prepared multipart upload key=78GUTC5THI2P upload=IM9J3EU5ZGTN tag=LQSLVQ6PIS4X type=write size=2879103 2026-02-24T22:10:43.984 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.981+0000 7f7e64854980 1 < completed write operation key=GH7723GNI2WM tag=WV3GUMZZ6SHU type=write size=832690 2026-02-24T22:10:43.985 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.981+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=AY0V6IP4CGZQ type=del size=962436 2026-02-24T22:10:43.985 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.981+0000 7f7e64854980 1 < canceled operation key=4Y3974PYYT78 tag=GI2D0229IF83 type=del size=1284784 2026-02-24T22:10:43.986 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.985+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=9N9QDSLO99A8 type=del size=1397860 2026-02-24T22:10:43.986 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.985+0000 7f7e64854980 1 < completed delete operation key=0AXS7DY23UZU tag=AOY4OD8E92CO type=del 2026-02-24T22:10:43.988 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.985+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~PM72IW7P6V2K.0 size=1048576 2026-02-24T22:10:43.988 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.985+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~PM72IW7P6V2K.1 size=1048576 2026-02-24T22:10:43.989 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.985+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~PM72IW7P6V2K.2 size=891662 2026-02-24T22:10:43.989 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.985+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=PM72IW7P6V2K tag=9B2P9ZR3ZMR0 type=write size=4037390 2026-02-24T22:10:43.989 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.989+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=YPJT3QK1DZGA type=del 2026-02-24T22:10:43.990 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.989+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=LN2TD350MQHQ type=del size=3656384 2026-02-24T22:10:43.991 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.989+0000 7f7e64854980 1 < completed delete operation key=LSRPN33Z2B9N tag=L2LWFQAY4AQ0 type=del 2026-02-24T22:10:43.992 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.989+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~RY34IG921G08.0 size=1048576 2026-02-24T22:10:43.993 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.989+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~RY34IG921G08.1 size=752340 2026-02-24T22:10:43.993 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.989+0000 7f7e64854980 1 > prepared multipart upload key=ON16L53JUSTW upload=RY34IG921G08 tag=N2RWBSG55Q32 type=write size=2849492 2026-02-24T22:10:43.993 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.993+0000 7f7e64854980 1 < completed delete operation key=Y3L3CJ51C1WR tag=25U0B95D3QQ4 type=del 2026-02-24T22:10:43.994 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.993+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=7NSIN8W1ILAJ type=del size=2968919 2026-02-24T22:10:43.995 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.993+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~P2WOKTD40L3E.0 size=1048576 2026-02-24T22:10:43.995 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.993+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~P2WOKTD40L3E.1 size=1048576 2026-02-24T22:10:43.996 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.993+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~P2WOKTD40L3E.2 size=457754 2026-02-24T22:10:43.997 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.993+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=P2WOKTD40L3E tag=43M8UKA8CU39 type=write size=3603482 2026-02-24T22:10:43.998 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.997+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~DEBD5TN71KZX.0 size=379952 2026-02-24T22:10:43.998 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.997+0000 7f7e64854980 1 > prepared multipart upload key=Y8GMOUA977H3 upload=DEBD5TN71KZX tag=TNRDLE6JAHM9 type=write size=1428528 2026-02-24T22:10:43.999 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.997+0000 7f7e64854980 1 < canceled operation key=2KKQKHXO7K3K tag=BK8Z39K1Z9FE type=del size=615196 2026-02-24T22:10:43.999 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.997+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=70ZHCZPWC0IM type=del size=3630513 2026-02-24T22:10:44.000 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.997+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~DFIY40JMM8QC.0 size=1048576 2026-02-24T22:10:44.000 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.997+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~DFIY40JMM8QC.1 size=1048576 2026-02-24T22:10:44.001 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:43.997+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~DFIY40JMM8QC.2 size=396279 2026-02-24T22:10:44.002 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.001+0000 7f7e64854980 1 < completed multipart upload key=LSRPN33Z2B9N upload=DFIY40JMM8QC tag=JH359TZQ0MMO type=write size=3542007 2026-02-24T22:10:44.002 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.001+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=ZVAM58HTV0JM type=del size=2906488 2026-02-24T22:10:44.003 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.001+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~M5U6XSLUIXK5.0 size=1048576 2026-02-24T22:10:44.004 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.001+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~M5U6XSLUIXK5.1 size=1048576 2026-02-24T22:10:44.004 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.001+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~M5U6XSLUIXK5.2 size=318595 2026-02-24T22:10:44.005 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.005+0000 7f7e64854980 1 < completed multipart upload key=D0MKGNRDW6L9 upload=M5U6XSLUIXK5 tag=NDDAQT5BPKR3 type=write size=3464323 2026-02-24T22:10:44.006 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.005+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~125AFPZGCDC6.0 size=841740 2026-02-24T22:10:44.007 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.005+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=125AFPZGCDC6 tag=CH3B5R3AXYP2 type=write size=1890316 2026-02-24T22:10:44.007 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.005+0000 7f7e64854980 1 < completed delete operation key=4Y3974PYYT78 tag=BNW0Y7UQXMHQ type=del 2026-02-24T22:10:44.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.005+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=24ZCDAU1XLN8 type=write size=392768 2026-02-24T22:10:44.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.005+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~HX3OZXS48WU5.0 size=1048576 2026-02-24T22:10:44.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.005+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~HX3OZXS48WU5.1 size=1048576 2026-02-24T22:10:44.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.005+0000 7f7e64854980 1 < canceled part key=_multipart_R2893NES3WSF.2~HX3OZXS48WU5.2 size=142449 2026-02-24T22:10:44.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.009+0000 7f7e64854980 1 < canceled multipart upload key=R2893NES3WSF upload=HX3OZXS48WU5 tag=1Y5NX5OEOHXT type=write size=3288177 2026-02-24T22:10:44.011 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.009+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~EJZHBP359PT7.0 size=1010814 2026-02-24T22:10:44.011 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.009+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=EJZHBP359PT7 tag=A6QVZKWFVR03 type=write size=2059390 2026-02-24T22:10:44.011 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.009+0000 7f7e64854980 1 < canceled part key=_multipart_N3ZUNSI5K8HS.2~BI8L0YNMKWY5.0 size=1048576 2026-02-24T22:10:44.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.009+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~BI8L0YNMKWY5.1 size=1048576 2026-02-24T22:10:44.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.009+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~BI8L0YNMKWY5.2 size=573103 2026-02-24T22:10:44.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.009+0000 7f7e64854980 1 < canceled multipart upload key=N3ZUNSI5K8HS upload=BI8L0YNMKWY5 tag=I9X3XGAQPFEX type=write size=3718831 2026-02-24T22:10:44.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.013+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=QB8QU5M1PHLE type=del size=4083272 2026-02-24T22:10:44.015 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.013+0000 7f7e64854980 1 < suggested update operation key=EB04ETNYM087 tag=AUSTCYEYYX1S type=del size=1918944 2026-02-24T22:10:44.016 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.013+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~WZ0YX9A6YC29.0 size=925050 2026-02-24T22:10:44.016 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.013+0000 7f7e64854980 1 > prepared multipart upload key=Y8GMOUA977H3 upload=WZ0YX9A6YC29 tag=MFTO4D088R0U type=write size=1973626 2026-02-24T22:10:44.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.013+0000 7f7e64854980 1 < canceled operation key=CI8DA17DX8LU tag=7C01TFXW33ES type=del size=3133045 2026-02-24T22:10:44.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.017+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=BKMEVXVBCGIX type=del size=3204846 2026-02-24T22:10:44.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.017+0000 7f7e64854980 1 < canceled part key=_multipart_78GUTC5THI2P.2~IM9J3EU5ZGTN.0 size=1048576 2026-02-24T22:10:44.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.017+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~IM9J3EU5ZGTN.1 size=781951 2026-02-24T22:10:44.019 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.017+0000 7f7e64854980 1 < canceled multipart upload key=78GUTC5THI2P upload=IM9J3EU5ZGTN tag=LQSLVQ6PIS4X type=write size=2879103 2026-02-24T22:10:44.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.017+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=HHZ0UUZALQTJ type=write size=224640 2026-02-24T22:10:44.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.017+0000 7f7e64854980 1 < completed delete operation key=D0MKGNRDW6L9 tag=AY0V6IP4CGZQ type=del 2026-02-24T22:10:44.021 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.017+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=A1XH4QD8KNPR type=write size=157405 2026-02-24T22:10:44.022 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.021+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=9N9QDSLO99A8 type=del 2026-02-24T22:10:44.022 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.021+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=KV2D0F614ZRA type=write size=863444 2026-02-24T22:10:44.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.021+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~PM72IW7P6V2K.0 size=1048576 2026-02-24T22:10:44.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.021+0000 7f7e64854980 1 < canceled part key=_multipart_HOHKR92SN25M.2~PM72IW7P6V2K.1 size=1048576 2026-02-24T22:10:44.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.021+0000 7f7e64854980 1 < canceled part key=_multipart_HOHKR92SN25M.2~PM72IW7P6V2K.2 size=891662 2026-02-24T22:10:44.024 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.021+0000 7f7e64854980 1 < canceled multipart upload key=HOHKR92SN25M upload=PM72IW7P6V2K tag=9B2P9ZR3ZMR0 type=write size=4037390 2026-02-24T22:10:44.025 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.021+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=Y4D2665PULY0 type=del size=3862820 2026-02-24T22:10:44.025 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.025+0000 7f7e64854980 1 < completed delete operation key=3900SFJK1YQU tag=LN2TD350MQHQ type=del 2026-02-24T22:10:44.026 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.025+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=ZZIIONKEVSZ2 type=del size=1390770 2026-02-24T22:10:44.026 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.025+0000 7f7e64854980 1 < canceled part key=_multipart_ON16L53JUSTW.2~RY34IG921G08.0 size=1048576 2026-02-24T22:10:44.027 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.025+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~RY34IG921G08.1 size=752340 2026-02-24T22:10:44.028 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.025+0000 7f7e64854980 1 < canceled multipart upload key=ON16L53JUSTW upload=RY34IG921G08 tag=N2RWBSG55Q32 type=write size=2849492 2026-02-24T22:10:44.028 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.025+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=K6ESM0RF33M0 type=del size=3675960 2026-02-24T22:10:44.030 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.029+0000 7f7e64854980 1 < suggested update operation key=LSRPN33Z2B9N tag=7NSIN8W1ILAJ type=del size=2968919 2026-02-24T22:10:44.030 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.029+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=PO7YZD6QAAKF type=del size=3732892 2026-02-24T22:10:44.031 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.029+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~DEBD5TN71KZX.0 size=379952 2026-02-24T22:10:44.032 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.029+0000 7f7e64854980 1 < completed multipart upload key=Y8GMOUA977H3 upload=DEBD5TN71KZX tag=TNRDLE6JAHM9 type=write size=1428528 2026-02-24T22:10:44.033 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.029+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=OQ9EH9ERJYSM type=del size=4178529 2026-02-24T22:10:44.033 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.033+0000 7f7e64854980 1 < canceled operation key=N3ZUNSI5K8HS tag=70ZHCZPWC0IM type=del size=3630513 2026-02-24T22:10:44.034 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.033+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=IC3ZXNIV9W6R type=del size=1954477 2026-02-24T22:10:44.035 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.033+0000 7f7e64854980 1 < completed delete operation key=PKRHWM69GBJI tag=ZVAM58HTV0JM type=del 2026-02-24T22:10:44.035 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.033+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=2WZPX3CM2KBZ type=del size=3682214 2026-02-24T22:10:44.036 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.033+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~125AFPZGCDC6.0 size=841740 2026-02-24T22:10:44.037 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.033+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=125AFPZGCDC6 tag=CH3B5R3AXYP2 type=write size=1890316 2026-02-24T22:10:44.038 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.037+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=8RRVWFN7O9IK type=del size=3965944 2026-02-24T22:10:44.038 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.037+0000 7f7e64854980 1 < completed write operation key=NL60Q5QOSP8E tag=24ZCDAU1XLN8 type=write size=392768 2026-02-24T22:10:44.039 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.037+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~B8HHJPDO5NO9.0 size=1048576 2026-02-24T22:10:44.040 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.037+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~B8HHJPDO5NO9.1 size=208868 2026-02-24T22:10:44.040 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.037+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=B8HHJPDO5NO9 tag=2L3QQC0LLVJI type=write size=2306020 2026-02-24T22:10:44.041 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.037+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~EJZHBP359PT7.0 size=1010814 2026-02-24T22:10:44.042 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.041+0000 7f7e64854980 1 < completed multipart upload key=ZESKGBWZXM6O upload=EJZHBP359PT7 tag=A6QVZKWFVR03 type=write size=2059390 2026-02-24T22:10:44.042 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.041+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=26JTC8U0N5MK type=del size=1287959 2026-02-24T22:10:44.043 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.041+0000 7f7e64854980 1 < completed delete operation key=L9NEDH44BIGA tag=QB8QU5M1PHLE type=del 2026-02-24T22:10:44.044 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.041+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=XBQH08R8NLOO type=del size=4162730 2026-02-24T22:10:44.044 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.041+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~WZ0YX9A6YC29.0 size=925050 2026-02-24T22:10:44.045 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.045+0000 7f7e64854980 1 < completed multipart upload key=Y8GMOUA977H3 upload=WZ0YX9A6YC29 tag=MFTO4D088R0U type=write size=1973626 2026-02-24T22:10:44.047 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.045+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~EDQYFREFX75O.0 size=1048576 2026-02-24T22:10:44.047 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.045+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~EDQYFREFX75O.1 size=1048576 2026-02-24T22:10:44.047 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.045+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~EDQYFREFX75O.2 size=659107 2026-02-24T22:10:44.048 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.045+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=EDQYFREFX75O tag=FGW3AYLPHWG5 type=write size=3804835 2026-02-24T22:10:44.048 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.045+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=BKMEVXVBCGIX type=del 2026-02-24T22:10:44.049 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.049+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~EWRQ8S8I3N1V.0 size=1048576 2026-02-24T22:10:44.050 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.049+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~EWRQ8S8I3N1V.1 size=1048576 2026-02-24T22:10:44.051 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.049+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~EWRQ8S8I3N1V.2 size=377705 2026-02-24T22:10:44.051 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.049+0000 7f7e64854980 1 > prepared multipart upload key=4Y3974PYYT78 upload=EWRQ8S8I3N1V tag=RPDT2GIR0T0F type=write size=3523433 2026-02-24T22:10:44.052 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.049+0000 7f7e64854980 1 < suggested update operation key=PKRHWM69GBJI tag=HHZ0UUZALQTJ type=write size=224640 2026-02-24T22:10:44.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.053+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~FOJ0SCZ0H4K3.0 size=1048576 2026-02-24T22:10:44.054 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.053+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~FOJ0SCZ0H4K3.1 size=1048576 2026-02-24T22:10:44.054 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.053+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~FOJ0SCZ0H4K3.2 size=807998 2026-02-24T22:10:44.054 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.053+0000 7f7e64854980 1 > prepared multipart upload key=36MKR2OZN7K9 upload=FOJ0SCZ0H4K3 tag=F4XFIYIBXAOI type=write size=3953726 2026-02-24T22:10:44.055 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.053+0000 7f7e64854980 1 < canceled operation key=Y3L3CJ51C1WR tag=A1XH4QD8KNPR type=write size=157405 2026-02-24T22:10:44.056 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.053+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=RLIF2PDFL8HA type=del size=2148967 2026-02-24T22:10:44.056 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.053+0000 7f7e64854980 1 < completed write operation key=L60HQ7M3E3LL tag=KV2D0F614ZRA type=write size=863444 2026-02-24T22:10:44.057 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.053+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=ZX2052JAXNQ6 type=del size=326700 2026-02-24T22:10:44.058 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.057+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=Y4D2665PULY0 type=del 2026-02-24T22:10:44.059 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.057+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~ZUICRUFUVBIM.0 size=1048576 2026-02-24T22:10:44.059 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.057+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~ZUICRUFUVBIM.1 size=768324 2026-02-24T22:10:44.059 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.057+0000 7f7e64854980 1 > prepared multipart upload key=HKOQRSF6XEXB upload=ZUICRUFUVBIM tag=4RJBYMB2U9NA type=write size=2865476 2026-02-24T22:10:44.060 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.057+0000 7f7e64854980 1 < completed delete operation key=Z0OXREAUJVA8 tag=ZZIIONKEVSZ2 type=del 2026-02-24T22:10:44.060 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.057+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=FR6C12NE0DE7 type=del size=3785818 2026-02-24T22:10:44.061 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.057+0000 7f7e64854980 1 < completed delete operation key=D0MKGNRDW6L9 tag=K6ESM0RF33M0 type=del 2026-02-24T22:10:44.062 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.061+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=I7ANIEOLVPP6 type=write size=219334 2026-02-24T22:10:44.063 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.061+0000 7f7e64854980 1 < suggested update operation key=EB04ETNYM087 tag=PO7YZD6QAAKF type=del size=3732892 2026-02-24T22:10:44.064 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.061+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=C5SXSU9YGV48 type=del size=327075 2026-02-24T22:10:44.064 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.061+0000 7f7e64854980 1 < canceled operation key=0B2V0MHWSOC9 tag=OQ9EH9ERJYSM type=del size=4178529 2026-02-24T22:10:44.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.065+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~8P79YPX12GXS.0 size=1048576 2026-02-24T22:10:44.066 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.065+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~8P79YPX12GXS.1 size=1048576 2026-02-24T22:10:44.066 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.065+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~8P79YPX12GXS.2 size=1031161 2026-02-24T22:10:44.066 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.065+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=8P79YPX12GXS tag=5U0FH40EN0NJ type=write size=4176889 2026-02-24T22:10:44.067 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.065+0000 7f7e64854980 1 < completed delete operation key=78GUTC5THI2P tag=IC3ZXNIV9W6R type=del 2026-02-24T22:10:44.068 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.065+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=K43LOYU6SZJI type=del size=1392401 2026-02-24T22:10:44.068 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.065+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=2WZPX3CM2KBZ type=del 2026-02-24T22:10:44.069 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.065+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=YJGGKB3R1D8A type=del size=4064371 2026-02-24T22:10:44.070 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.069+0000 7f7e64854980 1 < completed delete operation key=0AXS7DY23UZU tag=8RRVWFN7O9IK type=del 2026-02-24T22:10:44.071 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.069+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~0WU2XL02AQ53.0 size=1048576 2026-02-24T22:10:44.071 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.069+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~0WU2XL02AQ53.1 size=614472 2026-02-24T22:10:44.071 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.069+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=0WU2XL02AQ53 tag=6VATQ0LFBEV5 type=write size=2711624 2026-02-24T22:10:44.072 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.069+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~B8HHJPDO5NO9.0 size=1048576 2026-02-24T22:10:44.072 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.069+0000 7f7e64854980 1 < canceled part key=_multipart_CI8DA17DX8LU.2~B8HHJPDO5NO9.1 size=208868 2026-02-24T22:10:44.073 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.069+0000 7f7e64854980 1 < canceled multipart upload key=CI8DA17DX8LU upload=B8HHJPDO5NO9 tag=2L3QQC0LLVJI type=write size=2306020 2026-02-24T22:10:44.073 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.073+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=EHW30C1MDFMN type=del size=2935228 2026-02-24T22:10:44.074 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.073+0000 7f7e64854980 1 < completed delete operation key=R2893NES3WSF tag=26JTC8U0N5MK type=del 2026-02-24T22:10:44.075 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.073+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=1JDG1IVOPG33 type=del size=50899 2026-02-24T22:10:44.075 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.073+0000 7f7e64854980 1 < completed delete operation key=TF5B43G7KCYT tag=XBQH08R8NLOO type=del 2026-02-24T22:10:44.076 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.073+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=1F1KPF0X04RL type=del size=518593 2026-02-24T22:10:44.077 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.073+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~EDQYFREFX75O.0 size=1048576 2026-02-24T22:10:44.077 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.073+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~EDQYFREFX75O.1 size=1048576 2026-02-24T22:10:44.078 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.077+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~EDQYFREFX75O.2 size=659107 2026-02-24T22:10:44.079 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.077+0000 7f7e64854980 1 < completed multipart upload key=D0MKGNRDW6L9 upload=EDQYFREFX75O tag=FGW3AYLPHWG5 type=write size=3804835 2026-02-24T22:10:44.080 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.077+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~F6UMY96DVQ89.0 size=1048576 2026-02-24T22:10:44.081 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.077+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~F6UMY96DVQ89.1 size=852701 2026-02-24T22:10:44.081 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.077+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=F6UMY96DVQ89 tag=IB1X670JMD9O type=write size=2949853 2026-02-24T22:10:44.081 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.077+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~EWRQ8S8I3N1V.0 size=1048576 2026-02-24T22:10:44.082 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.081+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~EWRQ8S8I3N1V.1 size=1048576 2026-02-24T22:10:44.082 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.081+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~EWRQ8S8I3N1V.2 size=377705 2026-02-24T22:10:44.083 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.081+0000 7f7e64854980 1 < completed multipart upload key=4Y3974PYYT78 upload=EWRQ8S8I3N1V tag=RPDT2GIR0T0F type=write size=3523433 2026-02-24T22:10:44.084 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.081+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=9WCAUK01HSB0 type=write size=672653 2026-02-24T22:10:44.085 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.081+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~FOJ0SCZ0H4K3.0 size=1048576 2026-02-24T22:10:44.085 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.081+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~FOJ0SCZ0H4K3.1 size=1048576 2026-02-24T22:10:44.086 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.085+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~FOJ0SCZ0H4K3.2 size=807998 2026-02-24T22:10:44.087 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.085+0000 7f7e64854980 1 < completed multipart upload key=36MKR2OZN7K9 upload=FOJ0SCZ0H4K3 tag=F4XFIYIBXAOI type=write size=3953726 2026-02-24T22:10:44.088 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.085+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~VDM6ATKEGJAQ.0 size=1048576 2026-02-24T22:10:44.088 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.085+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~VDM6ATKEGJAQ.1 size=867385 2026-02-24T22:10:44.088 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.085+0000 7f7e64854980 1 > prepared multipart upload key=SGQYQ2TIVJPS upload=VDM6ATKEGJAQ tag=XTLJFEM7H2SH type=write size=2964537 2026-02-24T22:10:44.089 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.085+0000 7f7e64854980 1 < completed delete operation key=L60HQ7M3E3LL tag=RLIF2PDFL8HA type=del 2026-02-24T22:10:44.090 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.089+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~DVYYFUIZKO80.0 size=339281 2026-02-24T22:10:44.090 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.089+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=DVYYFUIZKO80 tag=21H9HCCM5OIJ type=write size=1387857 2026-02-24T22:10:44.091 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.089+0000 7f7e64854980 1 < completed delete operation key=TF5B43G7KCYT tag=ZX2052JAXNQ6 type=del 2026-02-24T22:10:44.091 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.089+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=4KQ84AG91NPD type=del size=3379856 2026-02-24T22:10:44.092 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.089+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~ZUICRUFUVBIM.0 size=1048576 2026-02-24T22:10:44.093 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.089+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~ZUICRUFUVBIM.1 size=768324 2026-02-24T22:10:44.094 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.093+0000 7f7e64854980 1 < completed multipart upload key=HKOQRSF6XEXB upload=ZUICRUFUVBIM tag=4RJBYMB2U9NA type=write size=2865476 2026-02-24T22:10:44.094 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.093+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=Y0W90QXM37O3 type=del size=1456059 2026-02-24T22:10:44.096 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.093+0000 7f7e64854980 1 < suggested update operation key=36MKR2OZN7K9 tag=FR6C12NE0DE7 type=del size=3785818 2026-02-24T22:10:44.096 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.093+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=VK111ODU8D81 type=del size=459763 2026-02-24T22:10:44.097 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.093+0000 7f7e64854980 1 < completed write operation key=Y3L3CJ51C1WR tag=I7ANIEOLVPP6 type=write size=219334 2026-02-24T22:10:44.097 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.097+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=C4JGAALZTMHM type=del size=2514208 2026-02-24T22:10:44.098 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.097+0000 7f7e64854980 1 < canceled operation key=EJCX06TE5RTZ tag=C5SXSU9YGV48 type=del size=327075 2026-02-24T22:10:44.099 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.097+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=7ONJFMGF0DHP type=write size=289423 2026-02-24T22:10:44.099 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.097+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~8P79YPX12GXS.0 size=1048576 2026-02-24T22:10:44.100 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.097+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~8P79YPX12GXS.1 size=1048576 2026-02-24T22:10:44.101 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.097+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~8P79YPX12GXS.2 size=1031161 2026-02-24T22:10:44.102 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.101+0000 7f7e64854980 1 < completed multipart upload key=CI8DA17DX8LU upload=8P79YPX12GXS tag=5U0FH40EN0NJ type=write size=4176889 2026-02-24T22:10:44.102 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.101+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=T30DADK0HWOK type=del size=3795866 2026-02-24T22:10:44.103 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.101+0000 7f7e64854980 1 < completed delete operation key=4Y3974PYYT78 tag=K43LOYU6SZJI type=del 2026-02-24T22:10:44.104 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.101+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=KY7VX73MGWVZ type=write size=997054 2026-02-24T22:10:44.104 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.101+0000 7f7e64854980 1 < completed delete operation key=4Y3974PYYT78 tag=YJGGKB3R1D8A type=del 2026-02-24T22:10:44.105 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.105+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~S8FAGQMJ9BF7.0 size=1048576 2026-02-24T22:10:44.106 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.105+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~S8FAGQMJ9BF7.1 size=461586 2026-02-24T22:10:44.106 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.105+0000 7f7e64854980 1 > prepared multipart upload key=Y8GMOUA977H3 upload=S8FAGQMJ9BF7 tag=9D8K4MJUTK07 type=write size=2558738 2026-02-24T22:10:44.106 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.105+0000 7f7e64854980 1 < canceled part key=_multipart_HOHKR92SN25M.2~0WU2XL02AQ53.0 size=1048576 2026-02-24T22:10:44.107 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.105+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~0WU2XL02AQ53.1 size=614472 2026-02-24T22:10:44.108 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.105+0000 7f7e64854980 1 < canceled multipart upload key=HOHKR92SN25M upload=0WU2XL02AQ53 tag=6VATQ0LFBEV5 type=write size=2711624 2026-02-24T22:10:44.109 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.105+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~2II75FIY7X1N.0 size=762988 2026-02-24T22:10:44.109 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.105+0000 7f7e64854980 1 > prepared multipart upload key=36MKR2OZN7K9 upload=2II75FIY7X1N tag=F74VZD0GPP5W type=write size=1811564 2026-02-24T22:10:44.109 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.109+0000 7f7e64854980 1 < completed delete operation key=GH7723GNI2WM tag=EHW30C1MDFMN type=del 2026-02-24T22:10:44.110 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.109+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=N1ZET81UXHC2 type=del size=3203043 2026-02-24T22:10:44.111 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.109+0000 7f7e64854980 1 < completed delete operation key=PKRHWM69GBJI tag=1JDG1IVOPG33 type=del 2026-02-24T22:10:44.112 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.109+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~W9KOJ2FOMZ4P.0 size=781205 2026-02-24T22:10:44.112 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.109+0000 7f7e64854980 1 > prepared multipart upload key=L60HQ7M3E3LL upload=W9KOJ2FOMZ4P tag=2A9P141S2VCJ type=write size=1829781 2026-02-24T22:10:44.113 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.109+0000 7f7e64854980 1 < completed delete operation key=A6PH77Q78VEN tag=1F1KPF0X04RL type=del 2026-02-24T22:10:44.113 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.113+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=1JH56XW24CDX type=del size=3087752 2026-02-24T22:10:44.114 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.113+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~F6UMY96DVQ89.0 size=1048576 2026-02-24T22:10:44.115 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.113+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~F6UMY96DVQ89.1 size=852701 2026-02-24T22:10:44.115 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.113+0000 7f7e64854980 1 < completed multipart upload key=0AXS7DY23UZU upload=F6UMY96DVQ89 tag=IB1X670JMD9O type=write size=2949853 2026-02-24T22:10:44.116 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.113+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=1J3G8DZV7RF0 type=del size=2045423 2026-02-24T22:10:44.117 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.117+0000 7f7e64854980 1 < suggested remove operation key=Y8GMOUA977H3 tag=9WCAUK01HSB0 type=write 2026-02-24T22:10:44.118 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.117+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=NZJB5HR8CGOC type=write size=29251 2026-02-24T22:10:44.119 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.117+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~VDM6ATKEGJAQ.0 size=1048576 2026-02-24T22:10:44.119 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.117+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~VDM6ATKEGJAQ.1 size=867385 2026-02-24T22:10:44.120 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.117+0000 7f7e64854980 1 < completed multipart upload key=SGQYQ2TIVJPS upload=VDM6ATKEGJAQ tag=XTLJFEM7H2SH type=write size=2964537 2026-02-24T22:10:44.121 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.117+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=KPINBNVZXY8S type=write size=344667 2026-02-24T22:10:44.121 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.121+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~DVYYFUIZKO80.0 size=339281 2026-02-24T22:10:44.122 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.121+0000 7f7e64854980 1 < completed multipart upload key=TXRUQQ456DD8 upload=DVYYFUIZKO80 tag=21H9HCCM5OIJ type=write size=1387857 2026-02-24T22:10:44.123 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.121+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=S752FTBTV155 type=del size=354640 2026-02-24T22:10:44.124 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.121+0000 7f7e64854980 1 < suggested remove operation key=R2893NES3WSF tag=4KQ84AG91NPD type=del 2026-02-24T22:10:44.125 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.121+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=30M3C83V1HRV type=write size=788180 2026-02-24T22:10:44.126 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.125+0000 7f7e64854980 1 < suggested update operation key=Q7BP8FXTARDW tag=Y0W90QXM37O3 type=del size=1456059 2026-02-24T22:10:44.126 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.125+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=876EQQJZSOZ5 type=del size=1028484 2026-02-24T22:10:44.127 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.125+0000 7f7e64854980 1 < suggested update operation key=NL60Q5QOSP8E tag=VK111ODU8D81 type=del size=459763 2026-02-24T22:10:44.128 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.125+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=L499AGU5A880 type=write size=97239 2026-02-24T22:10:44.129 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.125+0000 7f7e64854980 1 < completed delete operation key=LSRPN33Z2B9N tag=C4JGAALZTMHM type=del 2026-02-24T22:10:44.129 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.125+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=W5DAT747BK3M type=del size=2142098 2026-02-24T22:10:44.130 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.129+0000 7f7e64854980 1 < completed write operation key=ZESKGBWZXM6O tag=7ONJFMGF0DHP type=write size=289423 2026-02-24T22:10:44.130 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.129+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=4UIYVCDG0ZEH type=del size=3609897 2026-02-24T22:10:44.131 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.129+0000 7f7e64854980 1 < completed delete operation key=UQLUT9ZAI61V tag=T30DADK0HWOK type=del 2026-02-24T22:10:44.132 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.129+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=9NLFUN8P4UU5 type=del size=1286287 2026-02-24T22:10:44.133 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.129+0000 7f7e64854980 1 < suggested remove operation key=L60HQ7M3E3LL tag=KY7VX73MGWVZ type=write 2026-02-24T22:10:44.133 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.133+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=PK2JSRRYFCS5 type=del size=358866 2026-02-24T22:10:44.134 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.133+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~S8FAGQMJ9BF7.0 size=1048576 2026-02-24T22:10:44.134 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.133+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~S8FAGQMJ9BF7.1 size=461586 2026-02-24T22:10:44.135 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.133+0000 7f7e64854980 1 < completed multipart upload key=Y8GMOUA977H3 upload=S8FAGQMJ9BF7 tag=9D8K4MJUTK07 type=write size=2558738 2026-02-24T22:10:44.136 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.133+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=H4HF77W1N8WL type=del size=3183752 2026-02-24T22:10:44.137 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.133+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~2II75FIY7X1N.0 size=762988 2026-02-24T22:10:44.138 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.137+0000 7f7e64854980 1 < completed multipart upload key=36MKR2OZN7K9 upload=2II75FIY7X1N tag=F74VZD0GPP5W type=write size=1811564 2026-02-24T22:10:44.139 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.137+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=7UEV7VAMM3JJ type=del size=3251778 2026-02-24T22:10:44.139 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.137+0000 7f7e64854980 1 < completed delete operation key=Y3L3CJ51C1WR tag=N1ZET81UXHC2 type=del 2026-02-24T22:10:44.140 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.137+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~L7R5OY36P6K0.0 size=1048576 2026-02-24T22:10:44.141 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.137+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~L7R5OY36P6K0.1 size=32238 2026-02-24T22:10:44.141 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.137+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=L7R5OY36P6K0 tag=586ENP8SRSQ6 type=write size=2129390 2026-02-24T22:10:44.141 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.137+0000 7f7e64854980 1 < canceled part key=_multipart_L60HQ7M3E3LL.2~W9KOJ2FOMZ4P.0 size=781205 2026-02-24T22:10:44.142 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.141+0000 7f7e64854980 1 < canceled multipart upload key=L60HQ7M3E3LL upload=W9KOJ2FOMZ4P tag=2A9P141S2VCJ type=write size=1829781 2026-02-24T22:10:44.143 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.141+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~CN7G2UY6LML6.0 size=470180 2026-02-24T22:10:44.143 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.141+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=CN7G2UY6LML6 tag=7F9ZAPI9XWTL type=write size=1518756 2026-02-24T22:10:44.144 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.141+0000 7f7e64854980 1 < completed delete operation key=D0MKGNRDW6L9 tag=1JH56XW24CDX type=del 2026-02-24T22:10:44.145 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.141+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~8QSTH3JRVEDW.0 size=1048576 2026-02-24T22:10:44.146 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.145+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~8QSTH3JRVEDW.1 size=1048576 2026-02-24T22:10:44.146 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.145+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~8QSTH3JRVEDW.2 size=227229 2026-02-24T22:10:44.146 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.145+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=8QSTH3JRVEDW tag=QYZ3JD21KUFK type=write size=3372957 2026-02-24T22:10:44.147 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.145+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=1J3G8DZV7RF0 type=del 2026-02-24T22:10:44.147 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.145+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=P8FBDQJ9EA8V type=del size=3223233 2026-02-24T22:10:44.148 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.145+0000 7f7e64854980 1 < completed write operation key=78GUTC5THI2P tag=NZJB5HR8CGOC type=write size=29251 2026-02-24T22:10:44.149 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.145+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~GV7HO5V1O2O2.0 size=1048576 2026-02-24T22:10:44.150 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.149+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~GV7HO5V1O2O2.1 size=1048576 2026-02-24T22:10:44.150 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.149+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~GV7HO5V1O2O2.2 size=62002 2026-02-24T22:10:44.150 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.149+0000 7f7e64854980 1 > prepared multipart upload key=A6PH77Q78VEN upload=GV7HO5V1O2O2 tag=EVNS7NG4H60Y type=write size=3207730 2026-02-24T22:10:44.151 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.149+0000 7f7e64854980 1 < completed write operation key=L9NEDH44BIGA tag=KPINBNVZXY8S type=write size=344667 2026-02-24T22:10:44.151 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.149+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=PXEJXM160Z52 type=write size=367267 2026-02-24T22:10:44.152 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.149+0000 7f7e64854980 1 < completed delete operation key=GH7723GNI2WM tag=S752FTBTV155 type=del 2026-02-24T22:10:44.153 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.149+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=TF933AOB3KII type=write size=971953 2026-02-24T22:10:44.154 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.153+0000 7f7e64854980 1 < suggested update operation key=HKOQRSF6XEXB tag=30M3C83V1HRV type=write size=788180 2026-02-24T22:10:44.155 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.153+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=7T74XY1VRHDD type=del size=3727997 2026-02-24T22:10:44.156 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.153+0000 7f7e64854980 1 < suggested remove operation key=78GUTC5THI2P tag=876EQQJZSOZ5 type=del 2026-02-24T22:10:44.157 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.153+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=BCAK37CWTVKQ type=write size=811455 2026-02-24T22:10:44.158 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.157+0000 7f7e64854980 1 < suggested remove operation key=Y8GMOUA977H3 tag=L499AGU5A880 type=write 2026-02-24T22:10:44.159 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.157+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~AZXKQL6D5E8X.0 size=1048576 2026-02-24T22:10:44.160 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.157+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~AZXKQL6D5E8X.1 size=97417 2026-02-24T22:10:44.160 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.157+0000 7f7e64854980 1 > prepared multipart upload key=N3ZUNSI5K8HS upload=AZXKQL6D5E8X tag=YKL314PJXJ6F type=write size=2194569 2026-02-24T22:10:44.160 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.157+0000 7f7e64854980 1 < completed delete operation key=0B2V0MHWSOC9 tag=W5DAT747BK3M type=del 2026-02-24T22:10:44.161 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.157+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=C9X5TOJ4ZYQ7 type=write size=228878 2026-02-24T22:10:44.162 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.161+0000 7f7e64854980 1 < completed delete operation key=GH7723GNI2WM tag=4UIYVCDG0ZEH type=del 2026-02-24T22:10:44.162 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.161+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=HH25YA3HJSAZ type=del size=1119895 2026-02-24T22:10:44.163 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.161+0000 7f7e64854980 1 < completed delete operation key=TF5B43G7KCYT tag=9NLFUN8P4UU5 type=del 2026-02-24T22:10:44.164 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.161+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=AS8Z0M4PRPGE type=del size=2074689 2026-02-24T22:10:44.165 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.161+0000 7f7e64854980 1 < suggested remove operation key=TXRUQQ456DD8 tag=PK2JSRRYFCS5 type=del 2026-02-24T22:10:44.166 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.165+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=RW5174HI1T9Q type=del size=2438087 2026-02-24T22:10:44.166 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.165+0000 7f7e64854980 1 < completed delete operation key=TF5B43G7KCYT tag=H4HF77W1N8WL type=del 2026-02-24T22:10:44.167 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.165+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=N59W51KETCP7 type=del size=2325547 2026-02-24T22:10:44.167 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.165+0000 7f7e64854980 -1 < failed to complete operation key=78GUTC5THI2P tag=7UEV7VAMM3JJ type=del size=3251778: (22) Invalid argument 2026-02-24T22:10:44.168 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.165+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=JA53RAKYJPFR type=del size=2754641 2026-02-24T22:10:44.169 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.165+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~L7R5OY36P6K0.0 size=1048576 2026-02-24T22:10:44.169 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.169+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~L7R5OY36P6K0.1 size=32238 2026-02-24T22:10:44.171 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.169+0000 7f7e64854980 1 < completed multipart upload key=0AXS7DY23UZU upload=L7R5OY36P6K0 tag=586ENP8SRSQ6 type=write size=2129390 2026-02-24T22:10:44.172 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.169+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=KL2EMMLKOBBB type=del size=1635033 2026-02-24T22:10:44.172 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.169+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~CN7G2UY6LML6.0 size=470180 2026-02-24T22:10:44.173 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.173+0000 7f7e64854980 1 < completed multipart upload key=CI8DA17DX8LU upload=CN7G2UY6LML6 tag=7F9ZAPI9XWTL type=write size=1518756 2026-02-24T22:10:44.174 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.173+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=AMHC3MIR5202 type=del size=956713 2026-02-24T22:10:44.175 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.173+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~8QSTH3JRVEDW.0 size=1048576 2026-02-24T22:10:44.175 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.173+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~8QSTH3JRVEDW.1 size=1048576 2026-02-24T22:10:44.176 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.173+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~8QSTH3JRVEDW.2 size=227229 2026-02-24T22:10:44.177 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.173+0000 7f7e64854980 1 < completed multipart upload key=EJCX06TE5RTZ upload=8QSTH3JRVEDW tag=QYZ3JD21KUFK type=write size=3372957 2026-02-24T22:10:44.178 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.177+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~NGZD1KMVAI4T.0 size=331628 2026-02-24T22:10:44.178 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.177+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=NGZD1KMVAI4T tag=KC2A52YGXVDG type=write size=1380204 2026-02-24T22:10:44.179 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.177+0000 7f7e64854980 1 < completed delete operation key=0AXS7DY23UZU tag=P8FBDQJ9EA8V type=del 2026-02-24T22:10:44.179 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.177+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=RQXMQL3AVWBX type=del size=2197000 2026-02-24T22:10:44.180 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.177+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~GV7HO5V1O2O2.0 size=1048576 2026-02-24T22:10:44.180 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.177+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~GV7HO5V1O2O2.1 size=1048576 2026-02-24T22:10:44.181 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.177+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~GV7HO5V1O2O2.2 size=62002 2026-02-24T22:10:44.182 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.181+0000 7f7e64854980 1 < completed multipart upload key=A6PH77Q78VEN upload=GV7HO5V1O2O2 tag=EVNS7NG4H60Y type=write size=3207730 2026-02-24T22:10:44.183 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.181+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=84A2EG46LBPJ type=write size=533589 2026-02-24T22:10:44.184 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.181+0000 7f7e64854980 1 < completed write operation key=36MKR2OZN7K9 tag=PXEJXM160Z52 type=write size=367267 2026-02-24T22:10:44.185 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.181+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~BEF39XBK4YGC.0 size=303545 2026-02-24T22:10:44.185 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.181+0000 7f7e64854980 1 > prepared multipart upload key=Q7BP8FXTARDW upload=BEF39XBK4YGC tag=P24V3I7CDYAH type=write size=1352121 2026-02-24T22:10:44.186 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.185+0000 7f7e64854980 -1 < failed to complete operation key=HKOQRSF6XEXB tag=TF933AOB3KII type=write size=971953: (22) Invalid argument 2026-02-24T22:10:44.187 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.185+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~P8WBVZ384WIQ.0 size=1048576 2026-02-24T22:10:44.188 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.185+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~P8WBVZ384WIQ.1 size=656994 2026-02-24T22:10:44.188 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.185+0000 7f7e64854980 1 > prepared multipart upload key=SGQYQ2TIVJPS upload=P8WBVZ384WIQ tag=B0FIC1SJKM96 type=write size=2754146 2026-02-24T22:10:44.189 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.185+0000 7f7e64854980 1 < completed delete operation key=UQLUT9ZAI61V tag=7T74XY1VRHDD type=del 2026-02-24T22:10:44.190 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.189+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=3LCGFQ9I1B70 type=del size=2627747 2026-02-24T22:10:44.190 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.189+0000 7f7e64854980 1 < canceled operation key=3900SFJK1YQU tag=BCAK37CWTVKQ type=write size=811455 2026-02-24T22:10:44.192 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.189+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~M0TJJ64KJHYN.0 size=1048576 2026-02-24T22:10:44.192 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.189+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~M0TJJ64KJHYN.1 size=1048576 2026-02-24T22:10:44.193 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.189+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~M0TJJ64KJHYN.2 size=188917 2026-02-24T22:10:44.193 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.189+0000 7f7e64854980 1 > prepared multipart upload key=78GUTC5THI2P upload=M0TJJ64KJHYN tag=3ACZ6QI7QINH type=write size=3334645 2026-02-24T22:10:44.193 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.193+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~AZXKQL6D5E8X.0 size=1048576 2026-02-24T22:10:44.194 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.193+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~AZXKQL6D5E8X.1 size=97417 2026-02-24T22:10:44.195 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.193+0000 7f7e64854980 1 < completed multipart upload key=N3ZUNSI5K8HS upload=AZXKQL6D5E8X tag=YKL314PJXJ6F type=write size=2194569 2026-02-24T22:10:44.196 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.193+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=AJTJ9WYLCTLG type=write size=811968 2026-02-24T22:10:44.198 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.197+0000 7f7e64854980 1 < suggested remove operation key=Q7BP8FXTARDW tag=C9X5TOJ4ZYQ7 type=write 2026-02-24T22:10:44.198 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.197+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=QBZFZFGZP61L type=write size=127076 2026-02-24T22:10:44.200 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.197+0000 7f7e64854980 1 < suggested remove operation key=36MKR2OZN7K9 tag=HH25YA3HJSAZ type=del 2026-02-24T22:10:44.201 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.197+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=ELRRV2W9GEKB type=del size=2002847 2026-02-24T22:10:44.202 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.201+0000 7f7e64854980 1 < completed delete operation key=0AXS7DY23UZU tag=AS8Z0M4PRPGE type=del 2026-02-24T22:10:44.203 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.201+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~3U45DM2PJKUS.0 size=1048576 2026-02-24T22:10:44.204 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.201+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~3U45DM2PJKUS.1 size=1048576 2026-02-24T22:10:44.205 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.201+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~3U45DM2PJKUS.2 size=328359 2026-02-24T22:10:44.205 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.201+0000 7f7e64854980 1 > prepared multipart upload key=NL60Q5QOSP8E upload=3U45DM2PJKUS tag=0Q2PLJNO9PGD type=write size=3474087 2026-02-24T22:10:44.205 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.205+0000 7f7e64854980 1 < completed delete operation key=Z0OXREAUJVA8 tag=RW5174HI1T9Q type=del 2026-02-24T22:10:44.206 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.205+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=HWCGYPUOSFB1 type=del size=4154022 2026-02-24T22:10:44.207 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.205+0000 7f7e64854980 1 < canceled operation key=TF5B43G7KCYT tag=N59W51KETCP7 type=del size=2325547 2026-02-24T22:10:44.208 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.205+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~P806EQDV1NR7.0 size=940229 2026-02-24T22:10:44.208 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.205+0000 7f7e64854980 1 > prepared multipart upload key=A6PH77Q78VEN upload=P806EQDV1NR7 tag=4CB3C1FZTIMC type=write size=1988805 2026-02-24T22:10:44.209 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.205+0000 7f7e64854980 1 < completed delete operation key=L9NEDH44BIGA tag=JA53RAKYJPFR type=del 2026-02-24T22:10:44.209 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.209+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=4BP0X8PQREXL type=del size=1142130 2026-02-24T22:10:44.210 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.209+0000 7f7e64854980 1 < completed delete operation key=0AXS7DY23UZU tag=KL2EMMLKOBBB type=del 2026-02-24T22:10:44.212 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.209+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~FITXEONDO1V3.0 size=940046 2026-02-24T22:10:44.212 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.209+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=FITXEONDO1V3 tag=DJKJ1BUL5UNB type=write size=1988622 2026-02-24T22:10:44.212 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.209+0000 7f7e64854980 1 < canceled operation key=Z0OXREAUJVA8 tag=AMHC3MIR5202 type=del size=956713 2026-02-24T22:10:44.213 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.209+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=UUSP60ZHZRR4 type=del size=1526584 2026-02-24T22:10:44.214 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.213+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~NGZD1KMVAI4T.0 size=331628 2026-02-24T22:10:44.215 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.213+0000 7f7e64854980 1 < completed multipart upload key=0AXS7DY23UZU upload=NGZD1KMVAI4T tag=KC2A52YGXVDG type=write size=1380204 2026-02-24T22:10:44.216 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.213+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=L3XZAN695GH8 type=write size=954505 2026-02-24T22:10:44.216 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.213+0000 7f7e64854980 1 < completed delete operation key=ZESKGBWZXM6O tag=RQXMQL3AVWBX type=del 2026-02-24T22:10:44.217 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.217+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~WF2RT8IJOPU2.0 size=1048576 2026-02-24T22:10:44.218 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.217+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~WF2RT8IJOPU2.1 size=251874 2026-02-24T22:10:44.218 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.217+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=WF2RT8IJOPU2 tag=O05CX3G9MR34 type=write size=2349026 2026-02-24T22:10:44.219 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.217+0000 7f7e64854980 1 < completed write operation key=ZESKGBWZXM6O tag=84A2EG46LBPJ type=write size=533589 2026-02-24T22:10:44.219 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.217+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=M6QGRAPR9M2D type=del size=1278832 2026-02-24T22:10:44.220 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.217+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~BEF39XBK4YGC.0 size=303545 2026-02-24T22:10:44.221 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.221+0000 7f7e64854980 1 < completed multipart upload key=Q7BP8FXTARDW upload=BEF39XBK4YGC tag=P24V3I7CDYAH type=write size=1352121 2026-02-24T22:10:44.222 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.221+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=VDIN4YDOCPCQ type=write size=135942 2026-02-24T22:10:44.223 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.221+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~P8WBVZ384WIQ.0 size=1048576 2026-02-24T22:10:44.223 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.221+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~P8WBVZ384WIQ.1 size=656994 2026-02-24T22:10:44.224 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.221+0000 7f7e64854980 1 < completed multipart upload key=SGQYQ2TIVJPS upload=P8WBVZ384WIQ tag=B0FIC1SJKM96 type=write size=2754146 2026-02-24T22:10:44.225 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.221+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=50WJDGZB4EGY type=del size=1714966 2026-02-24T22:10:44.226 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.225+0000 7f7e64854980 1 < completed delete operation key=EB04ETNYM087 tag=3LCGFQ9I1B70 type=del 2026-02-24T22:10:44.227 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.225+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~99Y38B6LIX97.0 size=267861 2026-02-24T22:10:44.227 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.225+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=99Y38B6LIX97 tag=IP0B9PX904ZT type=write size=1316437 2026-02-24T22:10:44.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.225+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~M0TJJ64KJHYN.0 size=1048576 2026-02-24T22:10:44.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.225+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~M0TJJ64KJHYN.1 size=1048576 2026-02-24T22:10:44.229 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.225+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~M0TJJ64KJHYN.2 size=188917 2026-02-24T22:10:44.230 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.229+0000 7f7e64854980 1 < completed multipart upload key=78GUTC5THI2P upload=M0TJJ64KJHYN tag=3ACZ6QI7QINH type=write size=3334645 2026-02-24T22:10:44.230 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.229+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=IBKB4B98L8TG type=del size=1032985 2026-02-24T22:10:44.232 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.229+0000 7f7e64854980 1 < suggested update operation key=LSRPN33Z2B9N tag=AJTJ9WYLCTLG type=write size=811968 2026-02-24T22:10:44.232 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.229+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=E2PI6MIMQLT5 type=del size=2257224 2026-02-24T22:10:44.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.233+0000 7f7e64854980 1 < suggested remove operation key=N3ZUNSI5K8HS tag=QBZFZFGZP61L type=write 2026-02-24T22:10:44.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.233+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=F2KYS1RSI7D0 type=del size=464666 2026-02-24T22:10:44.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.233+0000 7f7e64854980 1 < completed delete operation key=GH7723GNI2WM tag=ELRRV2W9GEKB type=del 2026-02-24T22:10:44.236 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.233+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=GCEDED80OZ87 type=write size=633135 2026-02-24T22:10:44.236 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.233+0000 7f7e64854980 1 < canceled part key=_multipart_NL60Q5QOSP8E.2~3U45DM2PJKUS.0 size=1048576 2026-02-24T22:10:44.237 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.233+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~3U45DM2PJKUS.1 size=1048576 2026-02-24T22:10:44.238 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.237+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~3U45DM2PJKUS.2 size=328359 2026-02-24T22:10:44.239 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.237+0000 7f7e64854980 1 < canceled multipart upload key=NL60Q5QOSP8E upload=3U45DM2PJKUS tag=0Q2PLJNO9PGD type=write size=3474087 2026-02-24T22:10:44.240 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.237+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~H7DNG99OIMLJ.0 size=1048576 2026-02-24T22:10:44.240 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.237+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~H7DNG99OIMLJ.1 size=869983 2026-02-24T22:10:44.241 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.237+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=H7DNG99OIMLJ tag=1PFIQOW15748 type=write size=2967135 2026-02-24T22:10:44.241 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.237+0000 7f7e64854980 1 < completed delete operation key=R2893NES3WSF tag=HWCGYPUOSFB1 type=del 2026-02-24T22:10:44.242 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.241+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=XH9T9VMIZLVG type=del size=2990673 2026-02-24T22:10:44.242 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.241+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~P806EQDV1NR7.0 size=940229 2026-02-24T22:10:44.244 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.241+0000 7f7e64854980 1 < completed multipart upload key=A6PH77Q78VEN upload=P806EQDV1NR7 tag=4CB3C1FZTIMC type=write size=1988805 2026-02-24T22:10:44.244 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.241+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=E50FIY802JPS type=del size=2412153 2026-02-24T22:10:44.246 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.245+0000 7f7e64854980 1 < completed delete operation key=SGQYQ2TIVJPS tag=4BP0X8PQREXL type=del 2026-02-24T22:10:44.250 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.249+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=VHLXIOAY2UUM type=write size=113450 2026-02-24T22:10:44.252 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.249+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~FITXEONDO1V3.0 size=940046 2026-02-24T22:10:44.254 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.253+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=FITXEONDO1V3 tag=DJKJ1BUL5UNB type=write size=1988622 2026-02-24T22:10:44.255 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.253+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~T6FJBVX34SQ3.0 size=1048576 2026-02-24T22:10:44.256 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.253+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~T6FJBVX34SQ3.1 size=961356 2026-02-24T22:10:44.256 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.253+0000 7f7e64854980 1 > prepared multipart upload key=Y8GMOUA977H3 upload=T6FJBVX34SQ3 tag=UTI2ENBF4Y93 type=write size=3058508 2026-02-24T22:10:44.257 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.253+0000 7f7e64854980 1 < completed delete operation key=TXRUQQ456DD8 tag=UUSP60ZHZRR4 type=del 2026-02-24T22:10:44.258 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.257+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~W8714TIASVJ2.0 size=343031 2026-02-24T22:10:44.258 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.257+0000 7f7e64854980 1 > prepared multipart upload key=A6PH77Q78VEN upload=W8714TIASVJ2 tag=99LFGTIKMUE0 type=write size=1391607 2026-02-24T22:10:44.258 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.257+0000 7f7e64854980 1 < completed write operation key=Z0OXREAUJVA8 tag=L3XZAN695GH8 type=write size=954505 2026-02-24T22:10:44.259 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.257+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=EWQFWFGSK0BR type=del size=3850045 2026-02-24T22:10:44.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.257+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~WF2RT8IJOPU2.0 size=1048576 2026-02-24T22:10:44.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.257+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~WF2RT8IJOPU2.1 size=251874 2026-02-24T22:10:44.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.261+0000 7f7e64854980 1 < completed multipart upload key=0AXS7DY23UZU upload=WF2RT8IJOPU2 tag=O05CX3G9MR34 type=write size=2349026 2026-02-24T22:10:44.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.261+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=7HSK9332GMQL type=del size=1200972 2026-02-24T22:10:44.263 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.261+0000 7f7e64854980 1 < canceled operation key=PKRHWM69GBJI tag=M6QGRAPR9M2D type=del size=1278832 2026-02-24T22:10:44.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.261+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=7ZI4WKQC79QY type=del size=260176 2026-02-24T22:10:44.265 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.261+0000 7f7e64854980 1 < completed write operation key=PKRHWM69GBJI tag=VDIN4YDOCPCQ type=write size=135942 2026-02-24T22:10:44.266 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.265+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~O242XY63R7R6.0 size=1048576 2026-02-24T22:10:44.267 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.265+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~O242XY63R7R6.1 size=1048576 2026-02-24T22:10:44.267 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.265+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~O242XY63R7R6.2 size=15286 2026-02-24T22:10:44.267 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.265+0000 7f7e64854980 1 > prepared multipart upload key=78GUTC5THI2P upload=O242XY63R7R6 tag=RM7NDNMCB17S type=write size=3161014 2026-02-24T22:10:44.268 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.265+0000 7f7e64854980 1 < completed delete operation key=EB04ETNYM087 tag=50WJDGZB4EGY type=del 2026-02-24T22:10:44.269 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.265+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=F1OOF41YCMHL type=del size=584362 2026-02-24T22:10:44.269 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.265+0000 7f7e64854980 1 < canceled part key=_multipart_HOHKR92SN25M.2~99Y38B6LIX97.0 size=267861 2026-02-24T22:10:44.270 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.269+0000 7f7e64854980 1 < canceled multipart upload key=HOHKR92SN25M upload=99Y38B6LIX97 tag=IP0B9PX904ZT type=write size=1316437 2026-02-24T22:10:44.272 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.269+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~RY2H2STRG1H5.0 size=162644 2026-02-24T22:10:44.272 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.269+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=RY2H2STRG1H5 tag=7CM947B6R1ZX type=write size=1211220 2026-02-24T22:10:44.272 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.269+0000 7f7e64854980 1 < completed delete operation key=3900SFJK1YQU tag=IBKB4B98L8TG type=del 2026-02-24T22:10:44.274 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.273+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~1D1KW14UXCH1.0 size=707387 2026-02-24T22:10:44.274 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.273+0000 7f7e64854980 1 > prepared multipart upload key=78GUTC5THI2P upload=1D1KW14UXCH1 tag=6QHMCIX4HK2L type=write size=1755963 2026-02-24T22:10:44.275 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.273+0000 7f7e64854980 1 < completed delete operation key=TXRUQQ456DD8 tag=E2PI6MIMQLT5 type=del 2026-02-24T22:10:44.276 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.273+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=XQQ1I9AN0A8G type=write size=327975 2026-02-24T22:10:44.277 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.273+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=F2KYS1RSI7D0 type=del 2026-02-24T22:10:44.278 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.277+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=XNUNNBPTQV68 type=del size=1715449 2026-02-24T22:10:44.278 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.277+0000 7f7e64854980 1 < completed write operation key=LSRPN33Z2B9N tag=GCEDED80OZ87 type=write size=633135 2026-02-24T22:10:44.280 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.277+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~SGUP04ZPVC12.0 size=1048576 2026-02-24T22:10:44.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.277+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~SGUP04ZPVC12.1 size=207620 2026-02-24T22:10:44.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.277+0000 7f7e64854980 1 > prepared multipart upload key=PKRHWM69GBJI upload=SGUP04ZPVC12 tag=K4LQRYOOES33 type=write size=2304772 2026-02-24T22:10:44.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.281+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~H7DNG99OIMLJ.0 size=1048576 2026-02-24T22:10:44.282 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.281+0000 7f7e64854980 1 < canceled part key=_multipart_D0MKGNRDW6L9.2~H7DNG99OIMLJ.1 size=869983 2026-02-24T22:10:44.283 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.281+0000 7f7e64854980 1 < canceled multipart upload key=D0MKGNRDW6L9 upload=H7DNG99OIMLJ tag=1PFIQOW15748 type=write size=2967135 2026-02-24T22:10:44.283 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.281+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=C2Q9HQ01U7M4 type=del size=3259107 2026-02-24T22:10:44.284 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.281+0000 7f7e64854980 1 < canceled operation key=36MKR2OZN7K9 tag=XH9T9VMIZLVG type=del size=2990673 2026-02-24T22:10:44.285 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.281+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=GLEC7A2VFNRJ type=del size=832069 2026-02-24T22:10:44.285 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.285+0000 7f7e64854980 1 < completed delete operation key=LSRPN33Z2B9N tag=E50FIY802JPS type=del 2026-02-24T22:10:44.287 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.285+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~EE1F2ADYO1EU.0 size=1048576 2026-02-24T22:10:44.287 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.285+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~EE1F2ADYO1EU.1 size=1048576 2026-02-24T22:10:44.288 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.285+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~EE1F2ADYO1EU.2 size=913054 2026-02-24T22:10:44.288 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.285+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=EE1F2ADYO1EU tag=8Q1FMM8WN5E3 type=write size=4058782 2026-02-24T22:10:44.289 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.285+0000 7f7e64854980 1 < suggested update operation key=HKOQRSF6XEXB tag=VHLXIOAY2UUM type=write size=113450 2026-02-24T22:10:44.290 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.289+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=IEV1CFZCRZ7B type=del size=3935343 2026-02-24T22:10:44.291 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.289+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~T6FJBVX34SQ3.0 size=1048576 2026-02-24T22:10:44.291 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.289+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~T6FJBVX34SQ3.1 size=961356 2026-02-24T22:10:44.292 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.289+0000 7f7e64854980 1 < completed multipart upload key=Y8GMOUA977H3 upload=T6FJBVX34SQ3 tag=UTI2ENBF4Y93 type=write size=3058508 2026-02-24T22:10:44.293 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.293+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~DJC68QOHJ9EF.0 size=1048576 2026-02-24T22:10:44.294 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.293+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~DJC68QOHJ9EF.1 size=193126 2026-02-24T22:10:44.294 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.293+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=DJC68QOHJ9EF tag=UAUHBQ2RDCBO type=write size=2290278 2026-02-24T22:10:44.295 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.293+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~W8714TIASVJ2.0 size=343031 2026-02-24T22:10:44.296 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.293+0000 7f7e64854980 1 < completed multipart upload key=A6PH77Q78VEN upload=W8714TIASVJ2 tag=99LFGTIKMUE0 type=write size=1391607 2026-02-24T22:10:44.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.293+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=GCYPZDR2QJ54 type=del size=4008097 2026-02-24T22:10:44.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.297+0000 7f7e64854980 1 < completed delete operation key=SGQYQ2TIVJPS tag=EWQFWFGSK0BR type=del 2026-02-24T22:10:44.298 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.297+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=FK5GTZSAL5GW type=write size=789547 2026-02-24T22:10:44.299 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.297+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=7HSK9332GMQL type=del 2026-02-24T22:10:44.300 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.297+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=7S2QSVB8E3DL type=del size=577949 2026-02-24T22:10:44.300 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.297+0000 7f7e64854980 1 < completed delete operation key=R2893NES3WSF tag=7ZI4WKQC79QY type=del 2026-02-24T22:10:44.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.301+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=IHNRNW4BUBRH type=del size=2342200 2026-02-24T22:10:44.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.301+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~O242XY63R7R6.0 size=1048576 2026-02-24T22:10:44.303 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.301+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~O242XY63R7R6.1 size=1048576 2026-02-24T22:10:44.303 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.301+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~O242XY63R7R6.2 size=15286 2026-02-24T22:10:44.305 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.301+0000 7f7e64854980 1 < completed multipart upload key=78GUTC5THI2P upload=O242XY63R7R6 tag=RM7NDNMCB17S type=write size=3161014 2026-02-24T22:10:44.305 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.305+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=80QBT2PKSV02 type=del size=2544476 2026-02-24T22:10:44.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.305+0000 7f7e64854980 1 < canceled operation key=R2893NES3WSF tag=F1OOF41YCMHL type=del size=584362 2026-02-24T22:10:44.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.305+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=19Z3UKDASQ0H type=del size=3941287 2026-02-24T22:10:44.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.305+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~RY2H2STRG1H5.0 size=162644 2026-02-24T22:10:44.309 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.305+0000 7f7e64854980 1 < completed multipart upload key=CI8DA17DX8LU upload=RY2H2STRG1H5 tag=7CM947B6R1ZX type=write size=1211220 2026-02-24T22:10:44.310 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.309+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=79T8FD9D22UQ type=del size=3417745 2026-02-24T22:10:44.310 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.309+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~1D1KW14UXCH1.0 size=707387 2026-02-24T22:10:44.312 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.309+0000 7f7e64854980 1 < completed multipart upload key=78GUTC5THI2P upload=1D1KW14UXCH1 tag=6QHMCIX4HK2L type=write size=1755963 2026-02-24T22:10:44.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.309+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=LXNC6AG9DI75 type=del size=2859577 2026-02-24T22:10:44.315 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.313+0000 7f7e64854980 1 < suggested remove operation key=Z0OXREAUJVA8 tag=XQQ1I9AN0A8G type=write 2026-02-24T22:10:44.316 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.313+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~HNCSEEQ70SKN.0 size=1048576 2026-02-24T22:10:44.317 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.313+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~HNCSEEQ70SKN.1 size=158812 2026-02-24T22:10:44.317 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.313+0000 7f7e64854980 1 > prepared multipart upload key=3900SFJK1YQU upload=HNCSEEQ70SKN tag=JEV39NEXXW9S type=write size=2255964 2026-02-24T22:10:44.318 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.317+0000 7f7e64854980 1 < suggested remove operation key=UQLUT9ZAI61V tag=XNUNNBPTQV68 type=del 2026-02-24T22:10:44.319 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.317+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=Z3HKWTSCME5M type=write size=176743 2026-02-24T22:10:44.320 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.317+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~SGUP04ZPVC12.0 size=1048576 2026-02-24T22:10:44.320 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.317+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~SGUP04ZPVC12.1 size=207620 2026-02-24T22:10:44.321 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.321+0000 7f7e64854980 1 < completed multipart upload key=PKRHWM69GBJI upload=SGUP04ZPVC12 tag=K4LQRYOOES33 type=write size=2304772 2026-02-24T22:10:44.322 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.321+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=KMOPTHZK73GH type=del size=3588837 2026-02-24T22:10:44.323 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.321+0000 7f7e64854980 1 < completed delete operation key=EJCX06TE5RTZ tag=C2Q9HQ01U7M4 type=del 2026-02-24T22:10:44.324 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.321+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~38UX93KRMB9Y.0 size=532545 2026-02-24T22:10:44.324 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.321+0000 7f7e64854980 1 > prepared multipart upload key=A6PH77Q78VEN upload=38UX93KRMB9Y tag=WMSJC3YR2BCH type=write size=1581121 2026-02-24T22:10:44.325 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.321+0000 7f7e64854980 1 < completed delete operation key=ZESKGBWZXM6O tag=GLEC7A2VFNRJ type=del 2026-02-24T22:10:44.326 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.325+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~49BSCP66YOCL.0 size=1048576 2026-02-24T22:10:44.327 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.325+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~49BSCP66YOCL.1 size=169728 2026-02-24T22:10:44.327 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.325+0000 7f7e64854980 1 > prepared multipart upload key=SGQYQ2TIVJPS upload=49BSCP66YOCL tag=7TYKE76L3YYD type=write size=2266880 2026-02-24T22:10:44.328 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.325+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~EE1F2ADYO1EU.0 size=1048576 2026-02-24T22:10:44.328 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.325+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~EE1F2ADYO1EU.1 size=1048576 2026-02-24T22:10:44.328 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.325+0000 7f7e64854980 1 < canceled part key=_multipart_TXRUQQ456DD8.2~EE1F2ADYO1EU.2 size=913054 2026-02-24T22:10:44.330 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.329+0000 7f7e64854980 1 < canceled multipart upload key=TXRUQQ456DD8 upload=EE1F2ADYO1EU tag=8Q1FMM8WN5E3 type=write size=4058782 2026-02-24T22:10:44.331 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.329+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~M0VFR3LFANQS.0 size=1048576 2026-02-24T22:10:44.332 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.329+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~M0VFR3LFANQS.1 size=371017 2026-02-24T22:10:44.332 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.329+0000 7f7e64854980 1 > prepared multipart upload key=LSRPN33Z2B9N upload=M0VFR3LFANQS tag=0IJZJY75X33O type=write size=2468169 2026-02-24T22:10:44.333 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.329+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=IEV1CFZCRZ7B type=del 2026-02-24T22:10:44.333 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.333+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=Y1C6BTB0LSDR type=del size=2137741 2026-02-24T22:10:44.334 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.333+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~DJC68QOHJ9EF.0 size=1048576 2026-02-24T22:10:44.334 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.333+0000 7f7e64854980 1 < canceled part key=_multipart_2KKQKHXO7K3K.2~DJC68QOHJ9EF.1 size=193126 2026-02-24T22:10:44.335 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.333+0000 7f7e64854980 1 < canceled multipart upload key=2KKQKHXO7K3K upload=DJC68QOHJ9EF tag=UAUHBQ2RDCBO type=write size=2290278 2026-02-24T22:10:44.336 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.333+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=F4XZ672ZZJSH type=del size=2865496 2026-02-24T22:10:44.337 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.333+0000 7f7e64854980 1 < completed delete operation key=R2893NES3WSF tag=GCYPZDR2QJ54 type=del 2026-02-24T22:10:44.338 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.337+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=F1V7T631UX5W type=del size=2730781 2026-02-24T22:10:44.339 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.337+0000 7f7e64854980 1 < suggested update operation key=A6PH77Q78VEN tag=FK5GTZSAL5GW type=write size=789547 2026-02-24T22:10:44.341 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.337+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~XY21L7T52TZ5.0 size=368361 2026-02-24T22:10:44.341 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.337+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=XY21L7T52TZ5 tag=UKL5TLTHWKIT type=write size=1416937 2026-02-24T22:10:44.341 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.341+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=7S2QSVB8E3DL type=del 2026-02-24T22:10:44.342 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.341+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=4URSBY7DQS3L type=del size=179335 2026-02-24T22:10:44.344 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.341+0000 7f7e64854980 1 < suggested remove operation key=LSRPN33Z2B9N tag=IHNRNW4BUBRH type=del 2026-02-24T22:10:44.345 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.341+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~2CKV1JR5DDW5.0 size=495886 2026-02-24T22:10:44.345 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.341+0000 7f7e64854980 1 > prepared multipart upload key=NL60Q5QOSP8E upload=2CKV1JR5DDW5 tag=YI8DA3X0F6U9 type=write size=1544462 2026-02-24T22:10:44.346 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.345+0000 7f7e64854980 1 < completed delete operation key=4Y3974PYYT78 tag=80QBT2PKSV02 type=del 2026-02-24T22:10:44.347 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.345+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=CBA7BXX4C5OP type=del size=1944208 2026-02-24T22:10:44.347 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.345+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=19Z3UKDASQ0H type=del 2026-02-24T22:10:44.348 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.345+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=ZZ6EYB759C9X type=del size=5898 2026-02-24T22:10:44.349 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.345+0000 7f7e64854980 1 < completed delete operation key=R2893NES3WSF tag=79T8FD9D22UQ type=del 2026-02-24T22:10:44.350 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.349+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=0CY2RQLTZBM6 type=del size=3950891 2026-02-24T22:10:44.350 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.349+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=LXNC6AG9DI75 type=del 2026-02-24T22:10:44.351 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.349+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=TCGQTRERW934 type=del size=3244276 2026-02-24T22:10:44.352 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.349+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~HNCSEEQ70SKN.0 size=1048576 2026-02-24T22:10:44.353 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.349+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~HNCSEEQ70SKN.1 size=158812 2026-02-24T22:10:44.354 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.353+0000 7f7e64854980 1 < completed multipart upload key=3900SFJK1YQU upload=HNCSEEQ70SKN tag=JEV39NEXXW9S type=write size=2255964 2026-02-24T22:10:44.355 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.353+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~TJWAON7SIQ50.0 size=888906 2026-02-24T22:10:44.355 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.353+0000 7f7e64854980 1 > prepared multipart upload key=NL60Q5QOSP8E upload=TJWAON7SIQ50 tag=VDLW2POF0957 type=write size=1937482 2026-02-24T22:10:44.356 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.353+0000 7f7e64854980 1 < completed write operation key=UQLUT9ZAI61V tag=Z3HKWTSCME5M type=write size=176743 2026-02-24T22:10:44.357 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.353+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=0T2QHVTWBLZH type=del size=730614 2026-02-24T22:10:44.357 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.357+0000 7f7e64854980 1 < completed delete operation key=3900SFJK1YQU tag=KMOPTHZK73GH type=del 2026-02-24T22:10:44.358 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.357+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=OIKWM84BBAMQ type=del size=226540 2026-02-24T22:10:44.359 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.357+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~38UX93KRMB9Y.0 size=532545 2026-02-24T22:10:44.360 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.357+0000 7f7e64854980 1 < completed multipart upload key=A6PH77Q78VEN upload=38UX93KRMB9Y tag=WMSJC3YR2BCH type=write size=1581121 2026-02-24T22:10:44.361 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.357+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=KFIINYRZW088 type=write size=662972 2026-02-24T22:10:44.362 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.361+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~49BSCP66YOCL.0 size=1048576 2026-02-24T22:10:44.362 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.361+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~49BSCP66YOCL.1 size=169728 2026-02-24T22:10:44.363 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.361+0000 7f7e64854980 1 < completed multipart upload key=SGQYQ2TIVJPS upload=49BSCP66YOCL tag=7TYKE76L3YYD type=write size=2266880 2026-02-24T22:10:44.364 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.361+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=2MPJK3362X0R type=del size=4114260 2026-02-24T22:10:44.364 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.361+0000 7f7e64854980 1 < canceled part key=_multipart_LSRPN33Z2B9N.2~M0VFR3LFANQS.0 size=1048576 2026-02-24T22:10:44.365 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.361+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~M0VFR3LFANQS.1 size=371017 2026-02-24T22:10:44.366 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.365+0000 7f7e64854980 1 < canceled multipart upload key=LSRPN33Z2B9N upload=M0VFR3LFANQS tag=0IJZJY75X33O type=write size=2468169 2026-02-24T22:10:44.368 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.365+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~672UVSKXW17F.0 size=659789 2026-02-24T22:10:44.368 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.365+0000 7f7e64854980 1 > prepared multipart upload key=3900SFJK1YQU upload=672UVSKXW17F tag=D1792BSHHV7Z type=write size=1708365 2026-02-24T22:10:44.369 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.365+0000 7f7e64854980 1 < suggested remove operation key=HOHKR92SN25M tag=Y1C6BTB0LSDR type=del 2026-02-24T22:10:44.370 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.369+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~1S56MXEYN9TZ.0 size=990895 2026-02-24T22:10:44.370 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.369+0000 7f7e64854980 1 > prepared multipart upload key=SGQYQ2TIVJPS upload=1S56MXEYN9TZ tag=GWA8C64QXLV4 type=write size=2039471 2026-02-24T22:10:44.371 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.369+0000 7f7e64854980 1 < completed delete operation key=R2893NES3WSF tag=F4XZ672ZZJSH type=del 2026-02-24T22:10:44.372 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.369+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~YQHGB4OQFE4L.0 size=24041 2026-02-24T22:10:44.373 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.369+0000 7f7e64854980 1 > prepared multipart upload key=L60HQ7M3E3LL upload=YQHGB4OQFE4L tag=XP983FJR5ZQW type=write size=1072617 2026-02-24T22:10:44.373 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.369+0000 7f7e64854980 1 < completed delete operation key=HKOQRSF6XEXB tag=F1V7T631UX5W type=del 2026-02-24T22:10:44.374 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.373+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=1OA137522FQM type=del size=1423352 2026-02-24T22:10:44.375 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.373+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~XY21L7T52TZ5.0 size=368361 2026-02-24T22:10:44.376 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.373+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=XY21L7T52TZ5 tag=UKL5TLTHWKIT type=write size=1416937 2026-02-24T22:10:44.377 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.377+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~IQ4L2G6WSXM0.0 size=412442 2026-02-24T22:10:44.377 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.377+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=IQ4L2G6WSXM0 tag=OMTIOOCAVQZ7 type=write size=1461018 2026-02-24T22:10:44.378 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.377+0000 7f7e64854980 1 < canceled operation key=EJCX06TE5RTZ tag=4URSBY7DQS3L type=del size=179335 2026-02-24T22:10:44.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.377+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=YETKLKPCRXPL type=write size=830797 2026-02-24T22:10:44.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.377+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~2CKV1JR5DDW5.0 size=495886 2026-02-24T22:10:44.381 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.377+0000 7f7e64854980 1 < completed multipart upload key=NL60Q5QOSP8E upload=2CKV1JR5DDW5 tag=YI8DA3X0F6U9 type=write size=1544462 2026-02-24T22:10:44.382 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.381+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~1085I138BL6Y.0 size=1048576 2026-02-24T22:10:44.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.381+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~1085I138BL6Y.1 size=576565 2026-02-24T22:10:44.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.381+0000 7f7e64854980 1 > prepared multipart upload key=TF5B43G7KCYT upload=1085I138BL6Y tag=8SQZ9XVISPUM type=write size=2673717 2026-02-24T22:10:44.384 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.381+0000 7f7e64854980 1 < completed delete operation key=0AXS7DY23UZU tag=CBA7BXX4C5OP type=del 2026-02-24T22:10:44.385 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.381+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~5Z5P2INP5W3K.0 size=544320 2026-02-24T22:10:44.385 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.381+0000 7f7e64854980 1 > prepared multipart upload key=Q7BP8FXTARDW upload=5Z5P2INP5W3K tag=X9XPXM6SV9WI type=write size=1592896 2026-02-24T22:10:44.386 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.385+0000 7f7e64854980 1 < completed delete operation key=HKOQRSF6XEXB tag=ZZ6EYB759C9X type=del 2026-02-24T22:10:44.387 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.385+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=25V43KO5V5ZL type=write size=1015071 2026-02-24T22:10:44.387 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.385+0000 7f7e64854980 1 < completed delete operation key=PKRHWM69GBJI tag=0CY2RQLTZBM6 type=del 2026-02-24T22:10:44.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.385+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=ONX3JCJPSQUC type=del size=1943152 2026-02-24T22:10:44.389 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.389+0000 7f7e64854980 1 < suggested update operation key=D0MKGNRDW6L9 tag=TCGQTRERW934 type=del size=3244276 2026-02-24T22:10:44.390 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.389+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=N81B57AH06T4 type=del size=3503472 2026-02-24T22:10:44.391 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.389+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~TJWAON7SIQ50.0 size=888906 2026-02-24T22:10:44.392 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.389+0000 7f7e64854980 1 < completed multipart upload key=NL60Q5QOSP8E upload=TJWAON7SIQ50 tag=VDLW2POF0957 type=write size=1937482 2026-02-24T22:10:44.393 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.389+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~8XO0ZCHMFURR.0 size=1048576 2026-02-24T22:10:44.394 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.393+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~8XO0ZCHMFURR.1 size=559938 2026-02-24T22:10:44.394 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.393+0000 7f7e64854980 1 > prepared multipart upload key=ON16L53JUSTW upload=8XO0ZCHMFURR tag=3V75SXBZNXXR type=write size=2657090 2026-02-24T22:10:44.395 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.393+0000 7f7e64854980 1 < completed delete operation key=SGQYQ2TIVJPS tag=0T2QHVTWBLZH type=del 2026-02-24T22:10:44.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.393+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~N4Q55884OZNC.0 size=149340 2026-02-24T22:10:44.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.393+0000 7f7e64854980 1 > prepared multipart upload key=NL60Q5QOSP8E upload=N4Q55884OZNC tag=QFM4N9PMLI1H type=write size=1197916 2026-02-24T22:10:44.397 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.393+0000 7f7e64854980 1 < completed delete operation key=EB04ETNYM087 tag=OIKWM84BBAMQ type=del 2026-02-24T22:10:44.397 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.397+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=HJ07IK3JVUP4 type=del size=1041617 2026-02-24T22:10:44.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.397+0000 7f7e64854980 1 < completed write operation key=Q7BP8FXTARDW tag=KFIINYRZW088 type=write size=662972 2026-02-24T22:10:44.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.397+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=92JSJELJHXZV type=del size=1570065 2026-02-24T22:10:44.400 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.397+0000 7f7e64854980 1 < suggested update operation key=GH7723GNI2WM tag=2MPJK3362X0R type=del size=4114260 2026-02-24T22:10:44.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.401+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~GXXWWJT5BYZK.0 size=292825 2026-02-24T22:10:44.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.401+0000 7f7e64854980 1 > prepared multipart upload key=GH7723GNI2WM upload=GXXWWJT5BYZK tag=4IT3612X08DR type=write size=1341401 2026-02-24T22:10:44.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.401+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~672UVSKXW17F.0 size=659789 2026-02-24T22:10:44.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.401+0000 7f7e64854980 1 < completed multipart upload key=3900SFJK1YQU upload=672UVSKXW17F tag=D1792BSHHV7Z type=write size=1708365 2026-02-24T22:10:44.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.401+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=CQURFA2IMT5K type=del size=4043822 2026-02-24T22:10:44.405 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.401+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~1S56MXEYN9TZ.0 size=990895 2026-02-24T22:10:44.406 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.405+0000 7f7e64854980 1 < completed multipart upload key=SGQYQ2TIVJPS upload=1S56MXEYN9TZ tag=GWA8C64QXLV4 type=write size=2039471 2026-02-24T22:10:44.407 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.405+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=ZU5RB9NPSW7R type=write size=799366 2026-02-24T22:10:44.408 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.405+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~YQHGB4OQFE4L.0 size=24041 2026-02-24T22:10:44.409 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.405+0000 7f7e64854980 1 < completed multipart upload key=L60HQ7M3E3LL upload=YQHGB4OQFE4L tag=XP983FJR5ZQW type=write size=1072617 2026-02-24T22:10:44.410 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.409+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=G2AUJNUAKQ2K type=del size=2976208 2026-02-24T22:10:44.411 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.409+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=1OA137522FQM type=del 2026-02-24T22:10:44.412 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.409+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~BMGNQ0C5GB13.0 size=223114 2026-02-24T22:10:44.412 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.409+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=BMGNQ0C5GB13 tag=6E7XU2FSY6PV type=write size=1271690 2026-02-24T22:10:44.413 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.409+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~IQ4L2G6WSXM0.0 size=412442 2026-02-24T22:10:44.414 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.413+0000 7f7e64854980 1 < completed multipart upload key=EJCX06TE5RTZ upload=IQ4L2G6WSXM0 tag=OMTIOOCAVQZ7 type=write size=1461018 2026-02-24T22:10:44.415 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.413+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=XNCJ5GD4CAOG type=del size=782851 2026-02-24T22:10:44.415 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.413+0000 7f7e64854980 1 < completed write operation key=PKRHWM69GBJI tag=YETKLKPCRXPL type=write size=830797 2026-02-24T22:10:44.416 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.413+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=JXD3RKA6X4HI type=write size=581842 2026-02-24T22:10:44.417 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.413+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~1085I138BL6Y.0 size=1048576 2026-02-24T22:10:44.417 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.417+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~1085I138BL6Y.1 size=576565 2026-02-24T22:10:44.419 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.417+0000 7f7e64854980 1 < completed multipart upload key=TF5B43G7KCYT upload=1085I138BL6Y tag=8SQZ9XVISPUM type=write size=2673717 2026-02-24T22:10:44.419 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.417+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=ZVX1NO4GV831 type=del size=932102 2026-02-24T22:10:44.420 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.417+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~5Z5P2INP5W3K.0 size=544320 2026-02-24T22:10:44.422 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.421+0000 7f7e64854980 1 < completed multipart upload key=Q7BP8FXTARDW upload=5Z5P2INP5W3K tag=X9XPXM6SV9WI type=write size=1592896 2026-02-24T22:10:44.422 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.421+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=VBPL5OQF9X94 type=del size=308969 2026-02-24T22:10:44.423 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.421+0000 7f7e64854980 1 < completed write operation key=4Y3974PYYT78 tag=25V43KO5V5ZL type=write size=1015071 2026-02-24T22:10:44.424 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.421+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~1VBOFMFCP5OJ.0 size=1048576 2026-02-24T22:10:44.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.421+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~1VBOFMFCP5OJ.1 size=1048576 2026-02-24T22:10:44.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.425+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~1VBOFMFCP5OJ.2 size=153824 2026-02-24T22:10:44.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.425+0000 7f7e64854980 1 > prepared multipart upload key=EB04ETNYM087 upload=1VBOFMFCP5OJ tag=VY2Y8NHRMB28 type=write size=3299552 2026-02-24T22:10:44.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.425+0000 7f7e64854980 1 < completed delete operation key=HKOQRSF6XEXB tag=ONX3JCJPSQUC type=del 2026-02-24T22:10:44.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.425+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=LMEAW4BK28Y1 type=del size=3395880 2026-02-24T22:10:44.428 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.425+0000 7f7e64854980 1 < canceled operation key=TXRUQQ456DD8 tag=N81B57AH06T4 type=del size=3503472 2026-02-24T22:10:44.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.425+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=QBVL2C5HXN6I type=del size=1565196 2026-02-24T22:10:44.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.429+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~8XO0ZCHMFURR.0 size=1048576 2026-02-24T22:10:44.430 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.429+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~8XO0ZCHMFURR.1 size=559938 2026-02-24T22:10:44.431 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.429+0000 7f7e64854980 1 < completed multipart upload key=ON16L53JUSTW upload=8XO0ZCHMFURR tag=3V75SXBZNXXR type=write size=2657090 2026-02-24T22:10:44.432 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.429+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~969FY8LUH7BJ.0 size=1048576 2026-02-24T22:10:44.433 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.429+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~969FY8LUH7BJ.1 size=1048576 2026-02-24T22:10:44.434 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.433+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~969FY8LUH7BJ.2 size=166341 2026-02-24T22:10:44.434 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.433+0000 7f7e64854980 1 > prepared multipart upload key=Y3L3CJ51C1WR upload=969FY8LUH7BJ tag=3XU0QDZPNTOY type=write size=3312069 2026-02-24T22:10:44.434 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.433+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~N4Q55884OZNC.0 size=149340 2026-02-24T22:10:44.436 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.433+0000 7f7e64854980 1 < completed multipart upload key=NL60Q5QOSP8E upload=N4Q55884OZNC tag=QFM4N9PMLI1H type=write size=1197916 2026-02-24T22:10:44.436 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.433+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=AQL8V4HDX4E0 type=del size=309793 2026-02-24T22:10:44.437 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.437+0000 7f7e64854980 1 < completed delete operation key=TF5B43G7KCYT tag=HJ07IK3JVUP4 type=del 2026-02-24T22:10:44.438 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.437+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=08QV9I4QGOD5 type=del size=1688475 2026-02-24T22:10:44.439 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.437+0000 7f7e64854980 1 < completed delete operation key=N3ZUNSI5K8HS tag=92JSJELJHXZV type=del 2026-02-24T22:10:44.440 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.437+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~5PEWWREEAKK0.0 size=1048576 2026-02-24T22:10:44.441 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.437+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~5PEWWREEAKK0.1 size=1048576 2026-02-24T22:10:44.441 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.437+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~5PEWWREEAKK0.2 size=156855 2026-02-24T22:10:44.441 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.437+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=5PEWWREEAKK0 tag=1PWI10XD73IW type=write size=3302583 2026-02-24T22:10:44.442 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.441+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~GXXWWJT5BYZK.0 size=292825 2026-02-24T22:10:44.443 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.441+0000 7f7e64854980 1 < completed multipart upload key=GH7723GNI2WM upload=GXXWWJT5BYZK tag=4IT3612X08DR type=write size=1341401 2026-02-24T22:10:44.444 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.441+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=HB33IZOTEKKR type=del size=2038453 2026-02-24T22:10:44.445 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.441+0000 7f7e64854980 1 < completed delete operation key=D0MKGNRDW6L9 tag=CQURFA2IMT5K type=del 2026-02-24T22:10:44.445 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.445+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=97NHKX32WLJS type=del size=3846373 2026-02-24T22:10:44.446 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.445+0000 7f7e64854980 1 < completed write operation key=A6PH77Q78VEN tag=ZU5RB9NPSW7R type=write size=799366 2026-02-24T22:10:44.447 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.445+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=N2M8VUMLWZDC type=del size=2487217 2026-02-24T22:10:44.448 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.445+0000 7f7e64854980 1 < canceled operation key=L60HQ7M3E3LL tag=G2AUJNUAKQ2K type=del size=2976208 2026-02-24T22:10:44.448 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.445+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=NJANGVT01JB0 type=del size=1976526 2026-02-24T22:10:44.449 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.445+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~BMGNQ0C5GB13.0 size=223114 2026-02-24T22:10:44.450 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.449+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=BMGNQ0C5GB13 tag=6E7XU2FSY6PV type=write size=1271690 2026-02-24T22:10:44.452 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.449+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~6BM9Q2LZPI9H.0 size=1048576 2026-02-24T22:10:44.453 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.449+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~6BM9Q2LZPI9H.1 size=789209 2026-02-24T22:10:44.453 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.449+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=6BM9Q2LZPI9H tag=Z07JEDV59DVL type=write size=2886361 2026-02-24T22:10:44.454 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.453+0000 7f7e64854980 1 < completed delete operation key=UQLUT9ZAI61V tag=XNCJ5GD4CAOG type=del 2026-02-24T22:10:44.454 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.453+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=7NS0AH9TZ7AA type=del size=1559568 2026-02-24T22:10:44.456 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.453+0000 7f7e64854980 1 < suggested update operation key=LSRPN33Z2B9N tag=JXD3RKA6X4HI type=write size=581842 2026-02-24T22:10:44.456 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.453+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=2U7EKD0FU836 type=write size=1029251 2026-02-24T22:10:44.458 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.457+0000 7f7e64854980 1 < suggested remove operation key=PKRHWM69GBJI tag=ZVX1NO4GV831 type=del 2026-02-24T22:10:44.459 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.457+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~XYT2NB3NLPDZ.0 size=7390 2026-02-24T22:10:44.459 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.457+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=XYT2NB3NLPDZ tag=RSTMSNDX4S6F type=write size=1055966 2026-02-24T22:10:44.460 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.457+0000 7f7e64854980 1 < canceled operation key=GH7723GNI2WM tag=VBPL5OQF9X94 type=del size=308969 2026-02-24T22:10:44.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.457+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=D14JZFE8XFRE type=del size=3380217 2026-02-24T22:10:44.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.461+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~1VBOFMFCP5OJ.0 size=1048576 2026-02-24T22:10:44.462 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.461+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~1VBOFMFCP5OJ.1 size=1048576 2026-02-24T22:10:44.463 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.461+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~1VBOFMFCP5OJ.2 size=153824 2026-02-24T22:10:44.464 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.461+0000 7f7e64854980 1 < completed multipart upload key=EB04ETNYM087 upload=1VBOFMFCP5OJ tag=VY2Y8NHRMB28 type=write size=3299552 2026-02-24T22:10:44.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.461+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=9SSQ35JP0SPE type=write size=318257 2026-02-24T22:10:44.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.465+0000 7f7e64854980 1 < completed delete operation key=NL60Q5QOSP8E tag=LMEAW4BK28Y1 type=del 2026-02-24T22:10:44.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.465+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~EGC0X20AIHHE.0 size=1048576 2026-02-24T22:10:44.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.465+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~EGC0X20AIHHE.1 size=255471 2026-02-24T22:10:44.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.465+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=EGC0X20AIHHE tag=B7VLF58JO3SU type=write size=2352623 2026-02-24T22:10:44.468 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.465+0000 7f7e64854980 -1 < failed to complete operation key=LSRPN33Z2B9N tag=QBVL2C5HXN6I type=del size=1565196: (22) Invalid argument 2026-02-24T22:10:44.469 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.465+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~L6807RF65Y41.0 size=1048576 2026-02-24T22:10:44.470 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.469+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~L6807RF65Y41.1 size=1048576 2026-02-24T22:10:44.470 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.469+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~L6807RF65Y41.2 size=931278 2026-02-24T22:10:44.470 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.469+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=L6807RF65Y41 tag=YDALAY9199X1 type=write size=4077006 2026-02-24T22:10:44.471 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.469+0000 7f7e64854980 1 < canceled part key=_multipart_Y3L3CJ51C1WR.2~969FY8LUH7BJ.0 size=1048576 2026-02-24T22:10:44.471 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.469+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~969FY8LUH7BJ.1 size=1048576 2026-02-24T22:10:44.472 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.469+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~969FY8LUH7BJ.2 size=166341 2026-02-24T22:10:44.473 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.469+0000 7f7e64854980 1 < canceled multipart upload key=Y3L3CJ51C1WR upload=969FY8LUH7BJ tag=3XU0QDZPNTOY type=write size=3312069 2026-02-24T22:10:44.474 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.473+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=7H9I5P0NT4R7 type=del size=3308125 2026-02-24T22:10:44.474 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.473+0000 7f7e64854980 1 < canceled operation key=Y3L3CJ51C1WR tag=AQL8V4HDX4E0 type=del size=309793 2026-02-24T22:10:44.476 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.473+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~ACCVM27ORSSS.0 size=800703 2026-02-24T22:10:44.476 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.473+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=ACCVM27ORSSS tag=0PKDNZV2D24D type=write size=1849279 2026-02-24T22:10:44.476 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.473+0000 7f7e64854980 1 < completed delete operation key=EB04ETNYM087 tag=08QV9I4QGOD5 type=del 2026-02-24T22:10:44.477 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.473+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=NTGCFKRXSXIT type=del size=1931530 2026-02-24T22:10:44.478 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.477+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~5PEWWREEAKK0.0 size=1048576 2026-02-24T22:10:44.478 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.477+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~5PEWWREEAKK0.1 size=1048576 2026-02-24T22:10:44.479 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.477+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~5PEWWREEAKK0.2 size=156855 2026-02-24T22:10:44.480 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.477+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=5PEWWREEAKK0 tag=1PWI10XD73IW type=write size=3302583 2026-02-24T22:10:44.481 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.477+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=2TFOTXONLVUZ type=del size=3021690 2026-02-24T22:10:44.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.481+0000 7f7e64854980 1 < completed delete operation key=Y3L3CJ51C1WR tag=HB33IZOTEKKR type=del 2026-02-24T22:10:44.483 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.481+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=TC1D5PLGWVKY type=del size=1442443 2026-02-24T22:10:44.483 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.481+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=97NHKX32WLJS type=del 2026-02-24T22:10:44.484 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.481+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=DBVZVNV72WGX type=write size=900890 2026-02-24T22:10:44.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.481+0000 7f7e64854980 1 < completed delete operation key=36MKR2OZN7K9 tag=N2M8VUMLWZDC type=del 2026-02-24T22:10:44.486 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.485+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~GNTK0GJ3DXX1.0 size=1048576 2026-02-24T22:10:44.487 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.485+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~GNTK0GJ3DXX1.1 size=707986 2026-02-24T22:10:44.487 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.485+0000 7f7e64854980 1 > prepared multipart upload key=Z0OXREAUJVA8 upload=GNTK0GJ3DXX1 tag=FEJW49QCPB3S type=write size=2805138 2026-02-24T22:10:44.487 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.485+0000 7f7e64854980 1 < completed delete operation key=3900SFJK1YQU tag=NJANGVT01JB0 type=del 2026-02-24T22:10:44.489 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.485+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~X11V8LLHB1W6.0 size=849611 2026-02-24T22:10:44.489 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.485+0000 7f7e64854980 1 > prepared multipart upload key=4Y3974PYYT78 upload=X11V8LLHB1W6 tag=L8KW6GL4BCL6 type=write size=1898187 2026-02-24T22:10:44.489 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.489+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~6BM9Q2LZPI9H.0 size=1048576 2026-02-24T22:10:44.490 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.489+0000 7f7e64854980 1 < canceled part key=_multipart_CI8DA17DX8LU.2~6BM9Q2LZPI9H.1 size=789209 2026-02-24T22:10:44.491 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.489+0000 7f7e64854980 1 < canceled multipart upload key=CI8DA17DX8LU upload=6BM9Q2LZPI9H tag=Z07JEDV59DVL type=write size=2886361 2026-02-24T22:10:44.491 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.489+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=I2GPOU6CS5IG type=del size=3459703 2026-02-24T22:10:44.492 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.489+0000 7f7e64854980 1 < completed delete operation key=4Y3974PYYT78 tag=7NS0AH9TZ7AA type=del 2026-02-24T22:10:44.493 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.489+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=JOL54Z1C59C7 type=write size=129003 2026-02-24T22:10:44.494 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.493+0000 7f7e64854980 1 < completed write operation key=HKOQRSF6XEXB tag=2U7EKD0FU836 type=write size=1029251 2026-02-24T22:10:44.495 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.493+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~FGX6NBP7UB9O.0 size=1048576 2026-02-24T22:10:44.495 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.493+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~FGX6NBP7UB9O.1 size=140253 2026-02-24T22:10:44.496 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.493+0000 7f7e64854980 1 > prepared multipart upload key=A6PH77Q78VEN upload=FGX6NBP7UB9O tag=EGW0ACKF5YQ7 type=write size=2237405 2026-02-24T22:10:44.496 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.493+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~XYT2NB3NLPDZ.0 size=7390 2026-02-24T22:10:44.497 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.497+0000 7f7e64854980 1 < completed multipart upload key=HOHKR92SN25M upload=XYT2NB3NLPDZ tag=RSTMSNDX4S6F type=write size=1055966 2026-02-24T22:10:44.499 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.497+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~3BS0M3WBTQKY.0 size=1048576 2026-02-24T22:10:44.499 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.497+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~3BS0M3WBTQKY.1 size=922103 2026-02-24T22:10:44.499 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.497+0000 7f7e64854980 1 > prepared multipart upload key=HKOQRSF6XEXB upload=3BS0M3WBTQKY tag=EPHXW4Q5BROL type=write size=3019255 2026-02-24T22:10:44.500 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.497+0000 7f7e64854980 1 < completed delete operation key=UQLUT9ZAI61V tag=D14JZFE8XFRE type=del 2026-02-24T22:10:44.501 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.501+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~VQS2ZSGYVDZM.0 size=1048576 2026-02-24T22:10:44.502 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.501+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~VQS2ZSGYVDZM.1 size=304658 2026-02-24T22:10:44.502 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.501+0000 7f7e64854980 1 > prepared multipart upload key=Y3L3CJ51C1WR upload=VQS2ZSGYVDZM tag=HHV2IP48CH9M type=write size=2401810 2026-02-24T22:10:44.503 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.501+0000 7f7e64854980 1 < completed write operation key=36MKR2OZN7K9 tag=9SSQ35JP0SPE type=write size=318257 2026-02-24T22:10:44.503 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.501+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=VC0ZQCBUKAJA type=del size=898631 2026-02-24T22:10:44.504 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.501+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~EGC0X20AIHHE.0 size=1048576 2026-02-24T22:10:44.505 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.501+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~EGC0X20AIHHE.1 size=255471 2026-02-24T22:10:44.506 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.505+0000 7f7e64854980 1 < completed multipart upload key=TXRUQQ456DD8 upload=EGC0X20AIHHE tag=B7VLF58JO3SU type=write size=2352623 2026-02-24T22:10:44.507 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.505+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~2N1YHGON0682.0 size=1048576 2026-02-24T22:10:44.508 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.505+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~2N1YHGON0682.1 size=773611 2026-02-24T22:10:44.508 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.505+0000 7f7e64854980 1 > prepared multipart upload key=N3ZUNSI5K8HS upload=2N1YHGON0682 tag=GUIVTBIXRIAC type=write size=2870763 2026-02-24T22:10:44.509 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.505+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~L6807RF65Y41.0 size=1048576 2026-02-24T22:10:44.509 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.505+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~L6807RF65Y41.1 size=1048576 2026-02-24T22:10:44.510 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.509+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~L6807RF65Y41.2 size=931278 2026-02-24T22:10:44.511 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.509+0000 7f7e64854980 1 < completed multipart upload key=CI8DA17DX8LU upload=L6807RF65Y41 tag=YDALAY9199X1 type=write size=4077006 2026-02-24T22:10:44.512 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.509+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~DOJ8U4X7M1TM.0 size=1048576 2026-02-24T22:10:44.513 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.509+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~DOJ8U4X7M1TM.1 size=678983 2026-02-24T22:10:44.513 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.509+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=DOJ8U4X7M1TM tag=FL28P9A80A6O type=write size=2776135 2026-02-24T22:10:44.514 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.513+0000 7f7e64854980 1 < completed delete operation key=D0MKGNRDW6L9 tag=7H9I5P0NT4R7 type=del 2026-02-24T22:10:44.514 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.513+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=Q1WOMH373GKH type=del size=21740 2026-02-24T22:10:44.515 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.513+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~ACCVM27ORSSS.0 size=800703 2026-02-24T22:10:44.516 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.513+0000 7f7e64854980 1 < completed multipart upload key=TXRUQQ456DD8 upload=ACCVM27ORSSS tag=0PKDNZV2D24D type=write size=1849279 2026-02-24T22:10:44.517 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.513+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=31UI1KHBDBS7 type=del size=1987227 2026-02-24T22:10:44.518 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.517+0000 7f7e64854980 1 < completed delete operation key=2KKQKHXO7K3K tag=NTGCFKRXSXIT type=del 2026-02-24T22:10:44.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.517+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~ZRJITQLBQ5VW.0 size=1048576 2026-02-24T22:10:44.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.517+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~ZRJITQLBQ5VW.1 size=1048576 2026-02-24T22:10:44.521 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.517+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~ZRJITQLBQ5VW.2 size=394766 2026-02-24T22:10:44.521 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.517+0000 7f7e64854980 1 > prepared multipart upload key=0B2V0MHWSOC9 upload=ZRJITQLBQ5VW tag=JVPJ4T52W8IR type=write size=3540494 2026-02-24T22:10:44.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.521+0000 7f7e64854980 1 < completed delete operation key=L9NEDH44BIGA tag=2TFOTXONLVUZ type=del 2026-02-24T22:10:44.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.521+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=DIFC6LHJZC81 type=del size=3577460 2026-02-24T22:10:44.524 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.521+0000 7f7e64854980 1 < suggested remove operation key=2KKQKHXO7K3K tag=TC1D5PLGWVKY type=del 2026-02-24T22:10:44.524 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.521+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=0N761SDJILG1 type=del size=697740 2026-02-24T22:10:44.525 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.525+0000 7f7e64854980 1 < completed write operation key=78GUTC5THI2P tag=DBVZVNV72WGX type=write size=900890 2026-02-24T22:10:44.526 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.525+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=KPHOC069JVOF type=del size=2752618 2026-02-24T22:10:44.527 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.525+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~GNTK0GJ3DXX1.0 size=1048576 2026-02-24T22:10:44.527 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.525+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~GNTK0GJ3DXX1.1 size=707986 2026-02-24T22:10:44.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.525+0000 7f7e64854980 1 < completed multipart upload key=Z0OXREAUJVA8 upload=GNTK0GJ3DXX1 tag=FEJW49QCPB3S type=write size=2805138 2026-02-24T22:10:44.530 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.529+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~0S6J42A5OQ5M.0 size=1048576 2026-02-24T22:10:44.530 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.529+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~0S6J42A5OQ5M.1 size=1048576 2026-02-24T22:10:44.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.529+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~0S6J42A5OQ5M.2 size=22086 2026-02-24T22:10:44.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.529+0000 7f7e64854980 1 > prepared multipart upload key=36MKR2OZN7K9 upload=0S6J42A5OQ5M tag=9NIN9LQIU3QA type=write size=3167814 2026-02-24T22:10:44.532 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.529+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~X11V8LLHB1W6.0 size=849611 2026-02-24T22:10:44.533 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.529+0000 7f7e64854980 1 < completed multipart upload key=4Y3974PYYT78 upload=X11V8LLHB1W6 tag=L8KW6GL4BCL6 type=write size=1898187 2026-02-24T22:10:44.534 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.533+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~2VELL6MVHWSV.0 size=1048576 2026-02-24T22:10:44.535 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.533+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~2VELL6MVHWSV.1 size=1048576 2026-02-24T22:10:44.535 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.533+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~2VELL6MVHWSV.2 size=549798 2026-02-24T22:10:44.536 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.533+0000 7f7e64854980 1 > prepared multipart upload key=3900SFJK1YQU upload=2VELL6MVHWSV tag=6PH403NCGS6U type=write size=3695526 2026-02-24T22:10:44.536 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.533+0000 7f7e64854980 1 < completed delete operation key=Z0OXREAUJVA8 tag=I2GPOU6CS5IG type=del 2026-02-24T22:10:44.537 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.533+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=K2TNEUG62J8O type=write size=217200 2026-02-24T22:10:44.538 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.537+0000 7f7e64854980 1 < completed write operation key=GH7723GNI2WM tag=JOL54Z1C59C7 type=write size=129003 2026-02-24T22:10:44.539 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.537+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~1TVNUB5ZE556.0 size=1048576 2026-02-24T22:10:44.540 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.537+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~1TVNUB5ZE556.1 size=1048576 2026-02-24T22:10:44.540 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.537+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~1TVNUB5ZE556.2 size=1039358 2026-02-24T22:10:44.540 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.537+0000 7f7e64854980 1 > prepared multipart upload key=UQLUT9ZAI61V upload=1TVNUB5ZE556 tag=5O3AEM2OAKK2 type=write size=4185086 2026-02-24T22:10:44.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.537+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~FGX6NBP7UB9O.0 size=1048576 2026-02-24T22:10:44.542 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.541+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~FGX6NBP7UB9O.1 size=140253 2026-02-24T22:10:44.543 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.541+0000 7f7e64854980 1 < completed multipart upload key=A6PH77Q78VEN upload=FGX6NBP7UB9O tag=EGW0ACKF5YQ7 type=write size=2237405 2026-02-24T22:10:44.544 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.541+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~35P25JSN2LS8.0 size=1048576 2026-02-24T22:10:44.545 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.541+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~35P25JSN2LS8.1 size=1048576 2026-02-24T22:10:44.545 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.545+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~35P25JSN2LS8.2 size=800249 2026-02-24T22:10:44.545 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.545+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=35P25JSN2LS8 tag=QRNFCDP1SONH type=write size=3945977 2026-02-24T22:10:44.546 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.545+0000 7f7e64854980 1 < canceled part key=_multipart_HKOQRSF6XEXB.2~3BS0M3WBTQKY.0 size=1048576 2026-02-24T22:10:44.546 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.545+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~3BS0M3WBTQKY.1 size=922103 2026-02-24T22:10:44.547 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.545+0000 7f7e64854980 1 < canceled multipart upload key=HKOQRSF6XEXB upload=3BS0M3WBTQKY tag=EPHXW4Q5BROL type=write size=3019255 2026-02-24T22:10:44.548 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.545+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=TK2BDHOZU634 type=del size=1639811 2026-02-24T22:10:44.548 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.545+0000 7f7e64854980 1 < canceled part key=_multipart_Y3L3CJ51C1WR.2~VQS2ZSGYVDZM.0 size=1048576 2026-02-24T22:10:44.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.545+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~VQS2ZSGYVDZM.1 size=304658 2026-02-24T22:10:44.550 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.549+0000 7f7e64854980 1 < canceled multipart upload key=Y3L3CJ51C1WR upload=VQS2ZSGYVDZM tag=HHV2IP48CH9M type=write size=2401810 2026-02-24T22:10:44.551 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.549+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~FDOQYR1X0B1O.0 size=55100 2026-02-24T22:10:44.551 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.549+0000 7f7e64854980 1 > prepared multipart upload key=ON16L53JUSTW upload=FDOQYR1X0B1O tag=OMNI75GMD4IX type=write size=1103676 2026-02-24T22:10:44.552 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.549+0000 7f7e64854980 1 < completed delete operation key=D0MKGNRDW6L9 tag=VC0ZQCBUKAJA type=del 2026-02-24T22:10:44.553 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.549+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=FMPINU8KR5O3 type=del size=2865090 2026-02-24T22:10:44.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.553+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~2N1YHGON0682.0 size=1048576 2026-02-24T22:10:44.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.553+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~2N1YHGON0682.1 size=773611 2026-02-24T22:10:44.555 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.553+0000 7f7e64854980 1 < completed multipart upload key=N3ZUNSI5K8HS upload=2N1YHGON0682 tag=GUIVTBIXRIAC type=write size=2870763 2026-02-24T22:10:44.557 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.553+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~NYGJUFEVMBK3.0 size=1048576 2026-02-24T22:10:44.557 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.553+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~NYGJUFEVMBK3.1 size=229423 2026-02-24T22:10:44.557 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.553+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=NYGJUFEVMBK3 tag=M32LB4KTPZFX type=write size=2326575 2026-02-24T22:10:44.558 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.557+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~DOJ8U4X7M1TM.0 size=1048576 2026-02-24T22:10:44.559 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.557+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~DOJ8U4X7M1TM.1 size=678983 2026-02-24T22:10:44.560 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.557+0000 7f7e64854980 1 < completed multipart upload key=0AXS7DY23UZU upload=DOJ8U4X7M1TM tag=FL28P9A80A6O type=write size=2776135 2026-02-24T22:10:44.561 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.561+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~92MP4AGIEH5W.0 size=898148 2026-02-24T22:10:44.561 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.561+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=92MP4AGIEH5W tag=GFLFYWLR4FR4 type=write size=1946724 2026-02-24T22:10:44.562 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.561+0000 7f7e64854980 1 < completed delete operation key=Q7BP8FXTARDW tag=Q1WOMH373GKH type=del 2026-02-24T22:10:44.563 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.561+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~9BKWPWO20VEF.0 size=1048576 2026-02-24T22:10:44.564 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.561+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~9BKWPWO20VEF.1 size=436683 2026-02-24T22:10:44.564 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.561+0000 7f7e64854980 1 > prepared multipart upload key=TF5B43G7KCYT upload=9BKWPWO20VEF tag=PT4M5KPKZ6AQ type=write size=2533835 2026-02-24T22:10:44.565 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.561+0000 7f7e64854980 1 < completed delete operation key=TF5B43G7KCYT tag=31UI1KHBDBS7 type=del 2026-02-24T22:10:44.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.565+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~74FNX5Y4T8VF.0 size=1048576 2026-02-24T22:10:44.567 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.565+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~74FNX5Y4T8VF.1 size=1048576 2026-02-24T22:10:44.567 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.565+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~74FNX5Y4T8VF.2 size=377538 2026-02-24T22:10:44.567 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.565+0000 7f7e64854980 1 > prepared multipart upload key=Q7BP8FXTARDW upload=74FNX5Y4T8VF tag=REBW0PLYE5AO type=write size=3523266 2026-02-24T22:10:44.568 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.565+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~ZRJITQLBQ5VW.0 size=1048576 2026-02-24T22:10:44.569 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.565+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~ZRJITQLBQ5VW.1 size=1048576 2026-02-24T22:10:44.569 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.565+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~ZRJITQLBQ5VW.2 size=394766 2026-02-24T22:10:44.570 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.569+0000 7f7e64854980 1 < completed multipart upload key=0B2V0MHWSOC9 upload=ZRJITQLBQ5VW tag=JVPJ4T52W8IR type=write size=3540494 2026-02-24T22:10:44.572 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.569+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~TNHIYOK5KQDW.0 size=241436 2026-02-24T22:10:44.572 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.569+0000 7f7e64854980 1 > prepared multipart upload key=Z0OXREAUJVA8 upload=TNHIYOK5KQDW tag=3ATR3QCRT022 type=write size=1290012 2026-02-24T22:10:44.573 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.569+0000 7f7e64854980 1 < suggested update operation key=L60HQ7M3E3LL tag=DIFC6LHJZC81 type=del size=3577460 2026-02-24T22:10:44.574 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.573+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=H0FRJQ2WXJZ3 type=del size=418726 2026-02-24T22:10:44.575 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.573+0000 7f7e64854980 1 < completed delete operation key=A6PH77Q78VEN tag=0N761SDJILG1 type=del 2026-02-24T22:10:44.575 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.573+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=IRIUCW941WZG type=del size=417742 2026-02-24T22:10:44.576 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.573+0000 7f7e64854980 1 < completed delete operation key=Z0OXREAUJVA8 tag=KPHOC069JVOF type=del 2026-02-24T22:10:44.577 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.577+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~IABEB2S8NRDM.0 size=1048576 2026-02-24T22:10:44.578 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.577+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~IABEB2S8NRDM.1 size=1048576 2026-02-24T22:10:44.579 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.577+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~IABEB2S8NRDM.2 size=652680 2026-02-24T22:10:44.579 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.577+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=IABEB2S8NRDM tag=DVUFPAOB65B5 type=write size=3798408 2026-02-24T22:10:44.579 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.577+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~0S6J42A5OQ5M.0 size=1048576 2026-02-24T22:10:44.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.577+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~0S6J42A5OQ5M.1 size=1048576 2026-02-24T22:10:44.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.577+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~0S6J42A5OQ5M.2 size=22086 2026-02-24T22:10:44.582 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.581+0000 7f7e64854980 1 < completed multipart upload key=36MKR2OZN7K9 upload=0S6J42A5OQ5M tag=9NIN9LQIU3QA type=write size=3167814 2026-02-24T22:10:44.583 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.581+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=GJZJL2OL9S80 type=del size=884465 2026-02-24T22:10:44.583 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.581+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~2VELL6MVHWSV.0 size=1048576 2026-02-24T22:10:44.584 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.581+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~2VELL6MVHWSV.1 size=1048576 2026-02-24T22:10:44.584 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.581+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~2VELL6MVHWSV.2 size=549798 2026-02-24T22:10:44.586 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.585+0000 7f7e64854980 1 < completed multipart upload key=3900SFJK1YQU upload=2VELL6MVHWSV tag=6PH403NCGS6U type=write size=3695526 2026-02-24T22:10:44.587 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.585+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=H1DRQBVPO442 type=del size=3161193 2026-02-24T22:10:44.588 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.585+0000 7f7e64854980 1 < completed write operation key=2KKQKHXO7K3K tag=K2TNEUG62J8O type=write size=217200 2026-02-24T22:10:44.589 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.585+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~3QTPZY9QF0S1.0 size=1048576 2026-02-24T22:10:44.590 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.589+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~3QTPZY9QF0S1.1 size=1048576 2026-02-24T22:10:44.590 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.589+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~3QTPZY9QF0S1.2 size=366469 2026-02-24T22:10:44.590 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.589+0000 7f7e64854980 1 > prepared multipart upload key=Q7BP8FXTARDW upload=3QTPZY9QF0S1 tag=BF1LMCYYWCD3 type=write size=3512197 2026-02-24T22:10:44.591 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.589+0000 7f7e64854980 1 < completed part key=_multipart_UQLUT9ZAI61V.2~1TVNUB5ZE556.0 size=1048576 2026-02-24T22:10:44.592 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.589+0000 7f7e64854980 1 < completed part key=_multipart_UQLUT9ZAI61V.2~1TVNUB5ZE556.1 size=1048576 2026-02-24T22:10:44.592 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.589+0000 7f7e64854980 1 < completed part key=_multipart_UQLUT9ZAI61V.2~1TVNUB5ZE556.2 size=1039358 2026-02-24T22:10:44.593 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.593+0000 7f7e64854980 1 < completed multipart upload key=UQLUT9ZAI61V upload=1TVNUB5ZE556 tag=5O3AEM2OAKK2 type=write size=4185086 2026-02-24T22:10:44.594 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.593+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=NH0MXYQ3HS9P type=del size=1142002 2026-02-24T22:10:44.595 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.593+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~35P25JSN2LS8.0 size=1048576 2026-02-24T22:10:44.595 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.593+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~35P25JSN2LS8.1 size=1048576 2026-02-24T22:10:44.596 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.593+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~35P25JSN2LS8.2 size=800249 2026-02-24T22:10:44.597 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.593+0000 7f7e64854980 1 < completed multipart upload key=EJCX06TE5RTZ upload=35P25JSN2LS8 tag=QRNFCDP1SONH type=write size=3945977 2026-02-24T22:10:44.598 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.597+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=V1VR17X7CI6X type=del size=1905346 2026-02-24T22:10:44.599 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.597+0000 7f7e64854980 1 < canceled operation key=78GUTC5THI2P tag=TK2BDHOZU634 type=del size=1639811 2026-02-24T22:10:44.599 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.597+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=698C3E6EUNR4 type=write size=675720 2026-02-24T22:10:44.600 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.597+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~FDOQYR1X0B1O.0 size=55100 2026-02-24T22:10:44.601 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.601+0000 7f7e64854980 1 < completed multipart upload key=ON16L53JUSTW upload=FDOQYR1X0B1O tag=OMNI75GMD4IX type=write size=1103676 2026-02-24T22:10:44.603 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.601+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~5GTOI4G0Y8EZ.0 size=1048576 2026-02-24T22:10:44.603 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.601+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~5GTOI4G0Y8EZ.1 size=23615 2026-02-24T22:10:44.603 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.601+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=5GTOI4G0Y8EZ tag=1UPEL1NND25L type=write size=2120767 2026-02-24T22:10:44.604 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.601+0000 7f7e64854980 1 < completed delete operation key=TXRUQQ456DD8 tag=FMPINU8KR5O3 type=del 2026-02-24T22:10:44.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.601+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~KADLBEIS2Z33.0 size=1048576 2026-02-24T22:10:44.606 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.605+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~KADLBEIS2Z33.1 size=1048576 2026-02-24T22:10:44.606 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.605+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~KADLBEIS2Z33.2 size=892748 2026-02-24T22:10:44.606 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.605+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=KADLBEIS2Z33 tag=LQE3K71G0JYM type=write size=4038476 2026-02-24T22:10:44.607 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.605+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~NYGJUFEVMBK3.0 size=1048576 2026-02-24T22:10:44.608 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.605+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~NYGJUFEVMBK3.1 size=229423 2026-02-24T22:10:44.609 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.605+0000 7f7e64854980 1 < completed multipart upload key=0AXS7DY23UZU upload=NYGJUFEVMBK3 tag=M32LB4KTPZFX type=write size=2326575 2026-02-24T22:10:44.610 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.609+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~5HC0ZFHL4MGJ.0 size=1048576 2026-02-24T22:10:44.611 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.609+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~5HC0ZFHL4MGJ.1 size=725206 2026-02-24T22:10:44.611 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.609+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=5HC0ZFHL4MGJ tag=VWBDMR34AILP type=write size=2822358 2026-02-24T22:10:44.611 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.609+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~92MP4AGIEH5W.0 size=898148 2026-02-24T22:10:44.613 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.609+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=92MP4AGIEH5W tag=GFLFYWLR4FR4 type=write size=1946724 2026-02-24T22:10:44.613 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.613+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=95RE1FC7C6B3 type=del size=2275121 2026-02-24T22:10:44.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.613+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~9BKWPWO20VEF.0 size=1048576 2026-02-24T22:10:44.615 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.613+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~9BKWPWO20VEF.1 size=436683 2026-02-24T22:10:44.616 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.613+0000 7f7e64854980 1 < completed multipart upload key=TF5B43G7KCYT upload=9BKWPWO20VEF tag=PT4M5KPKZ6AQ type=write size=2533835 2026-02-24T22:10:44.618 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.617+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~K2CAC0BDTHZS.0 size=582222 2026-02-24T22:10:44.618 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.617+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=K2CAC0BDTHZS tag=N8G5ZMDM9L0D type=write size=1630798 2026-02-24T22:10:44.618 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.617+0000 7f7e64854980 1 < canceled part key=_multipart_Q7BP8FXTARDW.2~74FNX5Y4T8VF.0 size=1048576 2026-02-24T22:10:44.618 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.617+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~74FNX5Y4T8VF.1 size=1048576 2026-02-24T22:10:44.619 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.617+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~74FNX5Y4T8VF.2 size=377538 2026-02-24T22:10:44.620 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.617+0000 7f7e64854980 1 < canceled multipart upload key=Q7BP8FXTARDW upload=74FNX5Y4T8VF tag=REBW0PLYE5AO type=write size=3523266 2026-02-24T22:10:44.621 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.617+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=J1SBNI9VIQX5 type=write size=766623 2026-02-24T22:10:44.622 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.621+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~TNHIYOK5KQDW.0 size=241436 2026-02-24T22:10:44.623 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.621+0000 7f7e64854980 1 < completed multipart upload key=Z0OXREAUJVA8 upload=TNHIYOK5KQDW tag=3ATR3QCRT022 type=write size=1290012 2026-02-24T22:10:44.624 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.621+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~CSPMPLW3LL0I.0 size=1048576 2026-02-24T22:10:44.625 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.621+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~CSPMPLW3LL0I.1 size=1039910 2026-02-24T22:10:44.625 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.621+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=CSPMPLW3LL0I tag=DNHC6SJ3KDKD type=write size=3137062 2026-02-24T22:10:44.626 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.625+0000 7f7e64854980 1 < completed delete operation key=EB04ETNYM087 tag=H0FRJQ2WXJZ3 type=del 2026-02-24T22:10:44.627 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.625+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~XEGXJSIP1F63.0 size=1048576 2026-02-24T22:10:44.628 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.625+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~XEGXJSIP1F63.1 size=493616 2026-02-24T22:10:44.628 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.625+0000 7f7e64854980 1 > prepared multipart upload key=78GUTC5THI2P upload=XEGXJSIP1F63 tag=IY7XJM0QNJB2 type=write size=2590768 2026-02-24T22:10:44.629 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.625+0000 7f7e64854980 1 < suggested update operation key=EJCX06TE5RTZ tag=IRIUCW941WZG type=del size=417742 2026-02-24T22:10:44.630 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.629+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=BRM8YYGGW9PA type=write size=1028767 2026-02-24T22:10:44.631 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.629+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~IABEB2S8NRDM.0 size=1048576 2026-02-24T22:10:44.631 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.629+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~IABEB2S8NRDM.1 size=1048576 2026-02-24T22:10:44.631 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.629+0000 7f7e64854980 1 < canceled part key=_multipart_HOHKR92SN25M.2~IABEB2S8NRDM.2 size=652680 2026-02-24T22:10:44.632 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.629+0000 7f7e64854980 1 < canceled multipart upload key=HOHKR92SN25M upload=IABEB2S8NRDM tag=DVUFPAOB65B5 type=write size=3798408 2026-02-24T22:10:44.634 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.633+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~1H61R8JACW52.0 size=849087 2026-02-24T22:10:44.634 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.633+0000 7f7e64854980 1 > prepared multipart upload key=EB04ETNYM087 upload=1H61R8JACW52 tag=O3WH8PRH32ZD type=write size=1897663 2026-02-24T22:10:44.635 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.633+0000 7f7e64854980 1 < suggested remove operation key=78GUTC5THI2P tag=GJZJL2OL9S80 type=del 2026-02-24T22:10:44.636 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.633+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~FVSC20M0XDRC.0 size=1048576 2026-02-24T22:10:44.637 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.633+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~FVSC20M0XDRC.1 size=425482 2026-02-24T22:10:44.637 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.633+0000 7f7e64854980 1 > prepared multipart upload key=3900SFJK1YQU upload=FVSC20M0XDRC tag=HJ65ZHAI5W6T type=write size=2522634 2026-02-24T22:10:44.638 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.637+0000 7f7e64854980 1 < completed delete operation key=36MKR2OZN7K9 tag=H1DRQBVPO442 type=del 2026-02-24T22:10:44.639 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.637+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=C8BZ83SK48I3 type=del size=924284 2026-02-24T22:10:44.639 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.637+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~3QTPZY9QF0S1.0 size=1048576 2026-02-24T22:10:44.640 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.637+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~3QTPZY9QF0S1.1 size=1048576 2026-02-24T22:10:44.641 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.637+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~3QTPZY9QF0S1.2 size=366469 2026-02-24T22:10:44.642 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.641+0000 7f7e64854980 1 < completed multipart upload key=Q7BP8FXTARDW upload=3QTPZY9QF0S1 tag=BF1LMCYYWCD3 type=write size=3512197 2026-02-24T22:10:44.643 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.641+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~3SSD0VD89U82.0 size=1048576 2026-02-24T22:10:44.644 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.641+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~3SSD0VD89U82.1 size=1048576 2026-02-24T22:10:44.644 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.641+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~3SSD0VD89U82.2 size=693246 2026-02-24T22:10:44.644 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.641+0000 7f7e64854980 1 > prepared multipart upload key=EB04ETNYM087 upload=3SSD0VD89U82 tag=4V3M4YH37O7M type=write size=3838974 2026-02-24T22:10:44.646 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.645+0000 7f7e64854980 1 < suggested update operation key=L9NEDH44BIGA tag=NH0MXYQ3HS9P type=del size=1142002 2026-02-24T22:10:44.647 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.645+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~IZD8CLVXDWWC.0 size=1048576 2026-02-24T22:10:44.648 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.645+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~IZD8CLVXDWWC.1 size=1048576 2026-02-24T22:10:44.648 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.645+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~IZD8CLVXDWWC.2 size=581687 2026-02-24T22:10:44.648 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.645+0000 7f7e64854980 1 > prepared multipart upload key=SGQYQ2TIVJPS upload=IZD8CLVXDWWC tag=DWKFI4QQY05U type=write size=3727415 2026-02-24T22:10:44.649 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.645+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=V1VR17X7CI6X type=del 2026-02-24T22:10:44.650 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.649+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~ZG5VG23AM5YY.0 size=456792 2026-02-24T22:10:44.650 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.649+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=ZG5VG23AM5YY tag=F6H01I7M9SO9 type=write size=1505368 2026-02-24T22:10:44.651 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.649+0000 7f7e64854980 1 < completed write operation key=4Y3974PYYT78 tag=698C3E6EUNR4 type=write size=675720 2026-02-24T22:10:44.652 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.649+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=UTJ5CXS6D9ZP type=write size=226251 2026-02-24T22:10:44.653 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.649+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~5GTOI4G0Y8EZ.0 size=1048576 2026-02-24T22:10:44.654 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.653+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~5GTOI4G0Y8EZ.1 size=23615 2026-02-24T22:10:44.655 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.653+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=5GTOI4G0Y8EZ tag=1UPEL1NND25L type=write size=2120767 2026-02-24T22:10:44.656 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.653+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=GSD0M4GZZY3B type=write size=158931 2026-02-24T22:10:44.657 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.653+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~KADLBEIS2Z33.0 size=1048576 2026-02-24T22:10:44.657 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.653+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~KADLBEIS2Z33.1 size=1048576 2026-02-24T22:10:44.658 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.657+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~KADLBEIS2Z33.2 size=892748 2026-02-24T22:10:44.659 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.657+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=KADLBEIS2Z33 tag=LQE3K71G0JYM type=write size=4038476 2026-02-24T22:10:44.660 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.657+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=2RIPJB2MJDF1 type=write size=146998 2026-02-24T22:10:44.660 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.657+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~5HC0ZFHL4MGJ.0 size=1048576 2026-02-24T22:10:44.661 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.657+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~5HC0ZFHL4MGJ.1 size=725206 2026-02-24T22:10:44.662 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.661+0000 7f7e64854980 1 < completed multipart upload key=0AXS7DY23UZU upload=5HC0ZFHL4MGJ tag=VWBDMR34AILP type=write size=2822358 2026-02-24T22:10:44.664 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.661+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~9WAJSMV5KAB5.0 size=821169 2026-02-24T22:10:44.664 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.661+0000 7f7e64854980 1 > prepared multipart upload key=EB04ETNYM087 upload=9WAJSMV5KAB5 tag=ODFDUBEE0JA0 type=write size=1869745 2026-02-24T22:10:44.664 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.661+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=95RE1FC7C6B3 type=del 2026-02-24T22:10:44.665 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.661+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=5DSGI15HD22W type=del size=2010664 2026-02-24T22:10:44.666 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.665+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~K2CAC0BDTHZS.0 size=582222 2026-02-24T22:10:44.667 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.665+0000 7f7e64854980 1 < completed multipart upload key=ZESKGBWZXM6O upload=K2CAC0BDTHZS tag=N8G5ZMDM9L0D type=write size=1630798 2026-02-24T22:10:44.668 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.665+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~2XVOLETN9IX8.0 size=801518 2026-02-24T22:10:44.669 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.665+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=2XVOLETN9IX8 tag=9AC5RGVJE89M type=write size=1850094 2026-02-24T22:10:44.669 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.665+0000 7f7e64854980 -1 < failed to complete operation key=EJCX06TE5RTZ tag=J1SBNI9VIQX5 type=write size=766623: (22) Invalid argument 2026-02-24T22:10:44.670 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.669+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=C35CBQFUVNLI type=del size=1000857 2026-02-24T22:10:44.671 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.669+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~CSPMPLW3LL0I.0 size=1048576 2026-02-24T22:10:44.671 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.669+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~CSPMPLW3LL0I.1 size=1039910 2026-02-24T22:10:44.673 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.669+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=CSPMPLW3LL0I tag=DNHC6SJ3KDKD type=write size=3137062 2026-02-24T22:10:44.673 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.673+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=FQV8CUPMILNB type=del size=2340017 2026-02-24T22:10:44.674 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.673+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~XEGXJSIP1F63.0 size=1048576 2026-02-24T22:10:44.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.673+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~XEGXJSIP1F63.1 size=493616 2026-02-24T22:10:44.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.673+0000 7f7e64854980 1 < completed multipart upload key=78GUTC5THI2P upload=XEGXJSIP1F63 tag=IY7XJM0QNJB2 type=write size=2590768 2026-02-24T22:10:44.677 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.677+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~P1SENFFGVYYY.0 size=1048576 2026-02-24T22:10:44.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.677+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~P1SENFFGVYYY.1 size=1048576 2026-02-24T22:10:44.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.677+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~P1SENFFGVYYY.2 size=354948 2026-02-24T22:10:44.679 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.677+0000 7f7e64854980 1 > prepared multipart upload key=NL60Q5QOSP8E upload=P1SENFFGVYYY tag=7X5T637I7DB0 type=write size=3500676 2026-02-24T22:10:44.680 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.677+0000 7f7e64854980 1 < suggested update operation key=N3ZUNSI5K8HS tag=BRM8YYGGW9PA type=write size=1028767 2026-02-24T22:10:44.682 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.681+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~O8JY2JMITSAE.0 size=789393 2026-02-24T22:10:44.682 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.681+0000 7f7e64854980 1 > prepared multipart upload key=PKRHWM69GBJI upload=O8JY2JMITSAE tag=5FOS8CHQI42X type=write size=1837969 2026-02-24T22:10:44.683 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.681+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~1H61R8JACW52.0 size=849087 2026-02-24T22:10:44.684 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.681+0000 7f7e64854980 1 < completed multipart upload key=EB04ETNYM087 upload=1H61R8JACW52 tag=O3WH8PRH32ZD type=write size=1897663 2026-02-24T22:10:44.685 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.681+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~D8PYEC4NCIYB.0 size=968112 2026-02-24T22:10:44.685 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.681+0000 7f7e64854980 1 > prepared multipart upload key=ON16L53JUSTW upload=D8PYEC4NCIYB tag=WKITVA0THXN2 type=write size=2016688 2026-02-24T22:10:44.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.685+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~FVSC20M0XDRC.0 size=1048576 2026-02-24T22:10:44.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.685+0000 7f7e64854980 1 < canceled part key=_multipart_3900SFJK1YQU.2~FVSC20M0XDRC.1 size=425482 2026-02-24T22:10:44.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.685+0000 7f7e64854980 1 < canceled multipart upload key=3900SFJK1YQU upload=FVSC20M0XDRC tag=HJ65ZHAI5W6T type=write size=2522634 2026-02-24T22:10:44.688 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.685+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=5DHD19MPZBQM type=write size=218398 2026-02-24T22:10:44.689 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.685+0000 7f7e64854980 1 < completed delete operation key=D0MKGNRDW6L9 tag=C8BZ83SK48I3 type=del 2026-02-24T22:10:44.690 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.689+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=LE8QQFLY57XV type=del size=971344 2026-02-24T22:10:44.690 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.689+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~3SSD0VD89U82.0 size=1048576 2026-02-24T22:10:44.690 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.689+0000 7f7e64854980 1 < canceled part key=_multipart_EB04ETNYM087.2~3SSD0VD89U82.1 size=1048576 2026-02-24T22:10:44.691 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.689+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~3SSD0VD89U82.2 size=693246 2026-02-24T22:10:44.692 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.689+0000 7f7e64854980 1 < canceled multipart upload key=EB04ETNYM087 upload=3SSD0VD89U82 tag=4V3M4YH37O7M type=write size=3838974 2026-02-24T22:10:44.693 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.689+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=EHL6GJTSOSOK type=del size=2289793 2026-02-24T22:10:44.694 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.693+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~IZD8CLVXDWWC.0 size=1048576 2026-02-24T22:10:44.694 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.693+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~IZD8CLVXDWWC.1 size=1048576 2026-02-24T22:10:44.695 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.693+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~IZD8CLVXDWWC.2 size=581687 2026-02-24T22:10:44.696 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.693+0000 7f7e64854980 1 < completed multipart upload key=SGQYQ2TIVJPS upload=IZD8CLVXDWWC tag=DWKFI4QQY05U type=write size=3727415 2026-02-24T22:10:44.698 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.697+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~FOGSRS39EJX8.0 size=774741 2026-02-24T22:10:44.698 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.697+0000 7f7e64854980 1 > prepared multipart upload key=Z0OXREAUJVA8 upload=FOGSRS39EJX8 tag=OWP99I9CWU1O type=write size=1823317 2026-02-24T22:10:44.699 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.697+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~ZG5VG23AM5YY.0 size=456792 2026-02-24T22:10:44.700 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.697+0000 7f7e64854980 1 < completed multipart upload key=CI8DA17DX8LU upload=ZG5VG23AM5YY tag=F6H01I7M9SO9 type=write size=1505368 2026-02-24T22:10:44.701 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.701+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~JJ5TVSZ1W14N.0 size=671740 2026-02-24T22:10:44.702 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.701+0000 7f7e64854980 1 > prepared multipart upload key=Z0OXREAUJVA8 upload=JJ5TVSZ1W14N tag=QKD72AN9N54C type=write size=1720316 2026-02-24T22:10:44.702 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.701+0000 7f7e64854980 1 < canceled operation key=L60HQ7M3E3LL tag=UTJ5CXS6D9ZP type=write size=226251 2026-02-24T22:10:44.703 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.701+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=OOC93BC4JUIV type=del size=1922233 2026-02-24T22:10:44.704 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.701+0000 7f7e64854980 1 < completed write operation key=HKOQRSF6XEXB tag=GSD0M4GZZY3B type=write size=158931 2026-02-24T22:10:44.705 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.705+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~A4ICRR2UJHC6.0 size=1048576 2026-02-24T22:10:44.706 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.705+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~A4ICRR2UJHC6.1 size=1048576 2026-02-24T22:10:44.706 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.705+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~A4ICRR2UJHC6.2 size=13719 2026-02-24T22:10:44.707 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.705+0000 7f7e64854980 1 > prepared multipart upload key=36MKR2OZN7K9 upload=A4ICRR2UJHC6 tag=Z11NVQLKEG2Q type=write size=3159447 2026-02-24T22:10:44.707 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.705+0000 7f7e64854980 1 < completed write operation key=36MKR2OZN7K9 tag=2RIPJB2MJDF1 type=write size=146998 2026-02-24T22:10:44.708 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.705+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=LLBAQ0WW0DE0 type=write size=414652 2026-02-24T22:10:44.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.705+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~9WAJSMV5KAB5.0 size=821169 2026-02-24T22:10:44.710 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.709+0000 7f7e64854980 1 < completed multipart upload key=EB04ETNYM087 upload=9WAJSMV5KAB5 tag=ODFDUBEE0JA0 type=write size=1869745 2026-02-24T22:10:44.711 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.709+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=2WJF9RWTZXAP type=write size=531391 2026-02-24T22:10:44.712 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.709+0000 7f7e64854980 1 < suggested update operation key=EJCX06TE5RTZ tag=5DSGI15HD22W type=del size=2010664 2026-02-24T22:10:44.714 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.713+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~YYEG728XWMMZ.0 size=1048576 2026-02-24T22:10:44.714 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.713+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~YYEG728XWMMZ.1 size=1048576 2026-02-24T22:10:44.715 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.713+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~YYEG728XWMMZ.2 size=794027 2026-02-24T22:10:44.715 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.713+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=YYEG728XWMMZ tag=567JSY0335GN type=write size=3939755 2026-02-24T22:10:44.716 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.713+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~2XVOLETN9IX8.0 size=801518 2026-02-24T22:10:44.717 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.713+0000 7f7e64854980 1 < completed multipart upload key=TXRUQQ456DD8 upload=2XVOLETN9IX8 tag=9AC5RGVJE89M type=write size=1850094 2026-02-24T22:10:44.718 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.717+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=LBKDXEF6NLB9 type=del size=788964 2026-02-24T22:10:44.719 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.717+0000 7f7e64854980 1 < suggested remove operation key=L9NEDH44BIGA tag=C35CBQFUVNLI type=del 2026-02-24T22:10:44.721 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.717+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~A2UHBUW248UC.0 size=1048576 2026-02-24T22:10:44.721 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.721+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~A2UHBUW248UC.1 size=1048576 2026-02-24T22:10:44.722 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.721+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~A2UHBUW248UC.2 size=325469 2026-02-24T22:10:44.722 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.721+0000 7f7e64854980 1 > prepared multipart upload key=GH7723GNI2WM upload=A2UHBUW248UC tag=W48CDYG9B2A3 type=write size=3471197 2026-02-24T22:10:44.724 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.721+0000 7f7e64854980 1 < suggested remove operation key=NL60Q5QOSP8E tag=FQV8CUPMILNB type=del 2026-02-24T22:10:44.725 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.725+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=BUKVC3K8AYDK type=del size=380542 2026-02-24T22:10:44.726 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.725+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~P1SENFFGVYYY.0 size=1048576 2026-02-24T22:10:44.727 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.725+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~P1SENFFGVYYY.1 size=1048576 2026-02-24T22:10:44.727 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.725+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~P1SENFFGVYYY.2 size=354948 2026-02-24T22:10:44.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.725+0000 7f7e64854980 1 < completed multipart upload key=NL60Q5QOSP8E upload=P1SENFFGVYYY tag=7X5T637I7DB0 type=write size=3500676 2026-02-24T22:10:44.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.729+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=S61P9C1FEKUM type=del size=983754 2026-02-24T22:10:44.730 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.729+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~O8JY2JMITSAE.0 size=789393 2026-02-24T22:10:44.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.729+0000 7f7e64854980 1 < completed multipart upload key=PKRHWM69GBJI upload=O8JY2JMITSAE tag=5FOS8CHQI42X type=write size=1837969 2026-02-24T22:10:44.732 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.729+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=V5US1FRRWJJ2 type=write size=517596 2026-02-24T22:10:44.733 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.729+0000 7f7e64854980 1 < canceled part key=_multipart_ON16L53JUSTW.2~D8PYEC4NCIYB.0 size=968112 2026-02-24T22:10:44.734 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.733+0000 7f7e64854980 1 < canceled multipart upload key=ON16L53JUSTW upload=D8PYEC4NCIYB tag=WKITVA0THXN2 type=write size=2016688 2026-02-24T22:10:44.734 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.733+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=F7LX7Y1SM8S2 type=del size=4037776 2026-02-24T22:10:44.736 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.733+0000 7f7e64854980 1 < suggested update operation key=TF5B43G7KCYT tag=5DHD19MPZBQM type=write size=218398 2026-02-24T22:10:44.737 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.737+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~ZWKTKROUHXZT.0 size=1048576 2026-02-24T22:10:44.738 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.737+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~ZWKTKROUHXZT.1 size=1048576 2026-02-24T22:10:44.739 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.737+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~ZWKTKROUHXZT.2 size=69037 2026-02-24T22:10:44.739 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.737+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=ZWKTKROUHXZT tag=MSHD0URJNYB2 type=write size=3214765 2026-02-24T22:10:44.740 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.737+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=LE8QQFLY57XV type=del 2026-02-24T22:10:44.740 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.737+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=QTI8H799T50E type=del size=3648676 2026-02-24T22:10:44.741 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.737+0000 7f7e64854980 1 < canceled operation key=D0MKGNRDW6L9 tag=EHL6GJTSOSOK type=del size=2289793 2026-02-24T22:10:44.742 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.741+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=ETWRKSBIURCF type=del size=3341259 2026-02-24T22:10:44.743 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.741+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~FOGSRS39EJX8.0 size=774741 2026-02-24T22:10:44.744 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.741+0000 7f7e64854980 1 < completed multipart upload key=Z0OXREAUJVA8 upload=FOGSRS39EJX8 tag=OWP99I9CWU1O type=write size=1823317 2026-02-24T22:10:44.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.741+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=28Q8UVV8B2E2 type=del size=660809 2026-02-24T22:10:44.746 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.745+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~JJ5TVSZ1W14N.0 size=671740 2026-02-24T22:10:44.747 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.745+0000 7f7e64854980 1 < completed multipart upload key=Z0OXREAUJVA8 upload=JJ5TVSZ1W14N tag=QKD72AN9N54C type=write size=1720316 2026-02-24T22:10:44.748 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.745+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~4RUNRY39K1NH.0 size=145094 2026-02-24T22:10:44.748 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.745+0000 7f7e64854980 1 > prepared multipart upload key=4Y3974PYYT78 upload=4RUNRY39K1NH tag=O2FZ11WXTT94 type=write size=1193670 2026-02-24T22:10:44.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.745+0000 7f7e64854980 1 < completed delete operation key=0B2V0MHWSOC9 tag=OOC93BC4JUIV type=del 2026-02-24T22:10:44.750 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.749+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=ODAU28N65EB5 type=del size=2133483 2026-02-24T22:10:44.751 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.749+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~A4ICRR2UJHC6.0 size=1048576 2026-02-24T22:10:44.751 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.749+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~A4ICRR2UJHC6.1 size=1048576 2026-02-24T22:10:44.752 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.749+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~A4ICRR2UJHC6.2 size=13719 2026-02-24T22:10:44.753 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.753+0000 7f7e64854980 1 < completed multipart upload key=36MKR2OZN7K9 upload=A4ICRR2UJHC6 tag=Z11NVQLKEG2Q type=write size=3159447 2026-02-24T22:10:44.755 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.753+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~W9443H727Y5E.0 size=697743 2026-02-24T22:10:44.755 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.753+0000 7f7e64854980 1 > prepared multipart upload key=Y8GMOUA977H3 upload=W9443H727Y5E tag=JSCYY8ZUFJ7C type=write size=1746319 2026-02-24T22:10:44.755 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.753+0000 7f7e64854980 1 < canceled operation key=CI8DA17DX8LU tag=LLBAQ0WW0DE0 type=write size=414652 2026-02-24T22:10:44.757 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.753+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~Z9P3MYLSURF3.0 size=1048576 2026-02-24T22:10:44.757 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.757+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~Z9P3MYLSURF3.1 size=246957 2026-02-24T22:10:44.758 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.757+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=Z9P3MYLSURF3 tag=X2VUMAK81OFV type=write size=2344109 2026-02-24T22:10:44.758 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.757+0000 7f7e64854980 1 < completed write operation key=0AXS7DY23UZU tag=2WJF9RWTZXAP type=write size=531391 2026-02-24T22:10:44.759 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.757+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=036D3SPNH39H type=write size=313241 2026-02-24T22:10:44.760 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.757+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~YYEG728XWMMZ.0 size=1048576 2026-02-24T22:10:44.760 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.757+0000 7f7e64854980 1 < canceled part key=_multipart_CI8DA17DX8LU.2~YYEG728XWMMZ.1 size=1048576 2026-02-24T22:10:44.760 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.757+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~YYEG728XWMMZ.2 size=794027 2026-02-24T22:10:44.762 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.761+0000 7f7e64854980 1 < canceled multipart upload key=CI8DA17DX8LU upload=YYEG728XWMMZ tag=567JSY0335GN type=write size=3939755 2026-02-24T22:10:44.763 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.761+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=RVT5Y6KVQ5BD type=del size=1031914 2026-02-24T22:10:44.763 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.761+0000 7f7e64854980 1 < completed delete operation key=4Y3974PYYT78 tag=LBKDXEF6NLB9 type=del 2026-02-24T22:10:44.764 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.761+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=NKT6RTAVU7C3 type=del size=1218587 2026-02-24T22:10:44.765 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.761+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~A2UHBUW248UC.0 size=1048576 2026-02-24T22:10:44.765 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.765+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~A2UHBUW248UC.1 size=1048576 2026-02-24T22:10:44.766 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.765+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~A2UHBUW248UC.2 size=325469 2026-02-24T22:10:44.768 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.765+0000 7f7e64854980 1 < completed multipart upload key=GH7723GNI2WM upload=A2UHBUW248UC tag=W48CDYG9B2A3 type=write size=3471197 2026-02-24T22:10:44.768 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.765+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=8NEREE0FT08S type=del size=1927520 2026-02-24T22:10:44.769 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.765+0000 7f7e64854980 1 < completed delete operation key=2KKQKHXO7K3K tag=BUKVC3K8AYDK type=del 2026-02-24T22:10:44.770 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.769+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=NSTK822927HG type=write size=697157 2026-02-24T22:10:44.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.769+0000 7f7e64854980 1 < suggested update operation key=A6PH77Q78VEN tag=S61P9C1FEKUM type=del size=983754 2026-02-24T22:10:44.773 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.769+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~4Z0DD21CH9PS.0 size=1048576 2026-02-24T22:10:44.774 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.773+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~4Z0DD21CH9PS.1 size=1048576 2026-02-24T22:10:44.774 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.773+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~4Z0DD21CH9PS.2 size=105159 2026-02-24T22:10:44.774 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.773+0000 7f7e64854980 1 > prepared multipart upload key=UQLUT9ZAI61V upload=4Z0DD21CH9PS tag=UM70MHB9H4QD type=write size=3250887 2026-02-24T22:10:44.776 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.773+0000 7f7e64854980 1 < suggested update operation key=NL60Q5QOSP8E tag=V5US1FRRWJJ2 type=write size=517596 2026-02-24T22:10:44.777 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.773+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~WI2CNCWRJNJG.0 size=702842 2026-02-24T22:10:44.777 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.773+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=WI2CNCWRJNJG tag=6QFOK2BE6SV0 type=write size=1751418 2026-02-24T22:10:44.778 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.777+0000 7f7e64854980 1 < completed delete operation key=Y3L3CJ51C1WR tag=F7LX7Y1SM8S2 type=del 2026-02-24T22:10:44.779 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.777+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=SM6VN3RMIBI4 type=del size=3993944 2026-02-24T22:10:44.779 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.777+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~ZWKTKROUHXZT.0 size=1048576 2026-02-24T22:10:44.780 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.777+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~ZWKTKROUHXZT.1 size=1048576 2026-02-24T22:10:44.781 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.777+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~ZWKTKROUHXZT.2 size=69037 2026-02-24T22:10:44.782 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.781+0000 7f7e64854980 1 < completed multipart upload key=TXRUQQ456DD8 upload=ZWKTKROUHXZT tag=MSHD0URJNYB2 type=write size=3214765 2026-02-24T22:10:44.783 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.781+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=PTL51SSPXL51 type=del size=800250 2026-02-24T22:10:44.784 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.781+0000 7f7e64854980 -1 < failed to complete operation key=NL60Q5QOSP8E tag=QTI8H799T50E type=del size=3648676: (22) Invalid argument 2026-02-24T22:10:44.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.781+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~QC1N4NH2DZF0.0 size=1048576 2026-02-24T22:10:44.786 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.785+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~QC1N4NH2DZF0.1 size=313239 2026-02-24T22:10:44.786 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.785+0000 7f7e64854980 1 > prepared multipart upload key=UQLUT9ZAI61V upload=QC1N4NH2DZF0 tag=DCXBOCPPYYFQ type=write size=2410391 2026-02-24T22:10:44.786 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.785+0000 7f7e64854980 1 < completed delete operation key=0B2V0MHWSOC9 tag=ETWRKSBIURCF type=del 2026-02-24T22:10:44.788 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.785+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~LO8LWAXSGRPG.0 size=175828 2026-02-24T22:10:44.788 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.785+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=LO8LWAXSGRPG tag=5HCEPNJBO22F type=write size=1224404 2026-02-24T22:10:44.789 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.785+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=28Q8UVV8B2E2 type=del 2026-02-24T22:10:44.790 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.789+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~EVF8JK9WO8XD.0 size=1048576 2026-02-24T22:10:44.791 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.789+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~EVF8JK9WO8XD.1 size=100507 2026-02-24T22:10:44.791 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.789+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=EVF8JK9WO8XD tag=M9BPZNL9E2Y8 type=write size=2197659 2026-02-24T22:10:44.791 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.789+0000 7f7e64854980 1 < canceled part key=_multipart_4Y3974PYYT78.2~4RUNRY39K1NH.0 size=145094 2026-02-24T22:10:44.792 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.789+0000 7f7e64854980 1 < canceled multipart upload key=4Y3974PYYT78 upload=4RUNRY39K1NH tag=O2FZ11WXTT94 type=write size=1193670 2026-02-24T22:10:44.793 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.789+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=60Z3JBEDM6OE type=del size=2261541 2026-02-24T22:10:44.793 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.793+0000 7f7e64854980 1 < completed delete operation key=4Y3974PYYT78 tag=ODAU28N65EB5 type=del 2026-02-24T22:10:44.794 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.793+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=WFJH1EDQYDXR type=del size=2495903 2026-02-24T22:10:44.795 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.793+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~W9443H727Y5E.0 size=697743 2026-02-24T22:10:44.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.793+0000 7f7e64854980 1 < completed multipart upload key=Y8GMOUA977H3 upload=W9443H727Y5E tag=JSCYY8ZUFJ7C type=write size=1746319 2026-02-24T22:10:44.798 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.797+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~R6798PX5Q4IZ.0 size=1048576 2026-02-24T22:10:44.799 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.797+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~R6798PX5Q4IZ.1 size=140168 2026-02-24T22:10:44.799 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.797+0000 7f7e64854980 1 > prepared multipart upload key=TF5B43G7KCYT upload=R6798PX5Q4IZ tag=53K8U0PLUS79 type=write size=2237320 2026-02-24T22:10:44.799 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.797+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~Z9P3MYLSURF3.0 size=1048576 2026-02-24T22:10:44.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.797+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~Z9P3MYLSURF3.1 size=246957 2026-02-24T22:10:44.801 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.801+0000 7f7e64854980 1 < completed multipart upload key=HOHKR92SN25M upload=Z9P3MYLSURF3 tag=X2VUMAK81OFV type=write size=2344109 2026-02-24T22:10:44.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.801+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=4DERX2MRXSD9 type=write size=700817 2026-02-24T22:10:44.803 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.801+0000 7f7e64854980 1 < completed write operation key=Q7BP8FXTARDW tag=036D3SPNH39H type=write size=313241 2026-02-24T22:10:44.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.801+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=7Y82807HD77B type=write size=987905 2026-02-24T22:10:44.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.801+0000 7f7e64854980 1 < completed delete operation key=0B2V0MHWSOC9 tag=RVT5Y6KVQ5BD type=del 2026-02-24T22:10:44.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.805+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=VQCFW1JCN2A3 type=del size=2625459 2026-02-24T22:10:44.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.805+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=NKT6RTAVU7C3 type=del 2026-02-24T22:10:44.807 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.805+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=1WQVJ7YSIVH3 type=del size=2174490 2026-02-24T22:10:44.808 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.805+0000 7f7e64854980 1 < completed delete operation key=L9NEDH44BIGA tag=8NEREE0FT08S type=del 2026-02-24T22:10:44.808 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.805+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=F8LF8CU07H2M type=del size=986221 2026-02-24T22:10:44.810 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.809+0000 7f7e64854980 1 < suggested update operation key=HKOQRSF6XEXB tag=NSTK822927HG type=write size=697157 2026-02-24T22:10:44.811 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.809+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=EL6R33QDUKX9 type=del size=1259894 2026-02-24T22:10:44.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.809+0000 7f7e64854980 1 < completed part key=_multipart_UQLUT9ZAI61V.2~4Z0DD21CH9PS.0 size=1048576 2026-02-24T22:10:44.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.809+0000 7f7e64854980 1 < completed part key=_multipart_UQLUT9ZAI61V.2~4Z0DD21CH9PS.1 size=1048576 2026-02-24T22:10:44.813 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.809+0000 7f7e64854980 1 < completed part key=_multipart_UQLUT9ZAI61V.2~4Z0DD21CH9PS.2 size=105159 2026-02-24T22:10:44.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.813+0000 7f7e64854980 1 < completed multipart upload key=UQLUT9ZAI61V upload=4Z0DD21CH9PS tag=UM70MHB9H4QD type=write size=3250887 2026-02-24T22:10:44.815 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.813+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=UYIN8S8MHHB6 type=write size=834665 2026-02-24T22:10:44.815 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.813+0000 7f7e64854980 1 < canceled part key=_multipart_0AXS7DY23UZU.2~WI2CNCWRJNJG.0 size=702842 2026-02-24T22:10:44.816 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.813+0000 7f7e64854980 1 < canceled multipart upload key=0AXS7DY23UZU upload=WI2CNCWRJNJG tag=6QFOK2BE6SV0 type=write size=1751418 2026-02-24T22:10:44.818 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.817+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~S1BFEB63PC6O.0 size=1048576 2026-02-24T22:10:44.818 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.817+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~S1BFEB63PC6O.1 size=69332 2026-02-24T22:10:44.818 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.817+0000 7f7e64854980 1 > prepared multipart upload key=0B2V0MHWSOC9 upload=S1BFEB63PC6O tag=CLV3MUMRNJHB type=write size=2166484 2026-02-24T22:10:44.819 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.817+0000 7f7e64854980 1 < completed delete operation key=TXRUQQ456DD8 tag=SM6VN3RMIBI4 type=del 2026-02-24T22:10:44.820 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.817+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=IZLAMVW0PRJA type=del size=1653183 2026-02-24T22:10:44.820 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.817+0000 7f7e64854980 1 < completed delete operation key=0B2V0MHWSOC9 tag=PTL51SSPXL51 type=del 2026-02-24T22:10:44.821 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.821+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=6I36Z54A8XCF type=write size=743972 2026-02-24T22:10:44.822 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.821+0000 7f7e64854980 1 < completed part key=_multipart_UQLUT9ZAI61V.2~QC1N4NH2DZF0.0 size=1048576 2026-02-24T22:10:44.823 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.821+0000 7f7e64854980 1 < completed part key=_multipart_UQLUT9ZAI61V.2~QC1N4NH2DZF0.1 size=313239 2026-02-24T22:10:44.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.821+0000 7f7e64854980 1 < completed multipart upload key=UQLUT9ZAI61V upload=QC1N4NH2DZF0 tag=DCXBOCPPYYFQ type=write size=2410391 2026-02-24T22:10:44.825 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.821+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=JT7RM23IX5CS type=del size=1887267 2026-02-24T22:10:44.826 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.825+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~LO8LWAXSGRPG.0 size=175828 2026-02-24T22:10:44.827 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.825+0000 7f7e64854980 1 < completed multipart upload key=ZESKGBWZXM6O upload=LO8LWAXSGRPG tag=5HCEPNJBO22F type=write size=1224404 2026-02-24T22:10:44.828 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.825+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=K3YDITV53TCU type=write size=412216 2026-02-24T22:10:44.828 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.825+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~EVF8JK9WO8XD.0 size=1048576 2026-02-24T22:10:44.829 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.825+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~EVF8JK9WO8XD.1 size=100507 2026-02-24T22:10:44.830 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.829+0000 7f7e64854980 1 < completed multipart upload key=EJCX06TE5RTZ upload=EVF8JK9WO8XD tag=M9BPZNL9E2Y8 type=write size=2197659 2026-02-24T22:10:44.831 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.829+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=3OQ93W4J5CMF type=del size=3853627 2026-02-24T22:10:44.832 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.829+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=60Z3JBEDM6OE type=del 2026-02-24T22:10:44.832 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.829+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=A458UOEW7SF5 type=del size=2957054 2026-02-24T22:10:44.833 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.833+0000 7f7e64854980 1 < canceled operation key=GH7723GNI2WM tag=WFJH1EDQYDXR type=del size=2495903 2026-02-24T22:10:44.834 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.833+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=X2IR6QNPOU53 type=del size=3699184 2026-02-24T22:10:44.834 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.833+0000 7f7e64854980 1 < canceled part key=_multipart_TF5B43G7KCYT.2~R6798PX5Q4IZ.0 size=1048576 2026-02-24T22:10:44.835 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.833+0000 7f7e64854980 1 < canceled part key=_multipart_TF5B43G7KCYT.2~R6798PX5Q4IZ.1 size=140168 2026-02-24T22:10:44.835 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.833+0000 7f7e64854980 1 < canceled multipart upload key=TF5B43G7KCYT upload=R6798PX5Q4IZ tag=53K8U0PLUS79 type=write size=2237320 2026-02-24T22:10:44.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.833+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~WKC90MT7VB4W.0 size=1048576 2026-02-24T22:10:44.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.837+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~WKC90MT7VB4W.1 size=793485 2026-02-24T22:10:44.838 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.837+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=WKC90MT7VB4W tag=XQKKD47JTCY4 type=write size=2890637 2026-02-24T22:10:44.838 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.837+0000 7f7e64854980 1 < completed write operation key=3900SFJK1YQU tag=4DERX2MRXSD9 type=write size=700817 2026-02-24T22:10:44.839 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.837+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=8JMO2HWIVD63 type=del size=599770 2026-02-24T22:10:44.840 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.837+0000 7f7e64854980 -1 < failed to complete operation key=HKOQRSF6XEXB tag=7Y82807HD77B type=write size=987905: (22) Invalid argument 2026-02-24T22:10:44.841 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.837+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~LUBCBUMS7LOI.0 size=134781 2026-02-24T22:10:44.841 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.837+0000 7f7e64854980 1 > prepared multipart upload key=L60HQ7M3E3LL upload=LUBCBUMS7LOI tag=KZP2L3MI850N type=write size=1183357 2026-02-24T22:10:44.842 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.841+0000 7f7e64854980 1 < completed delete operation key=Y3L3CJ51C1WR tag=VQCFW1JCN2A3 type=del 2026-02-24T22:10:44.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.841+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=8Q4HP6IUL5LE type=write size=244301 2026-02-24T22:10:44.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.841+0000 7f7e64854980 1 < canceled operation key=R2893NES3WSF tag=1WQVJ7YSIVH3 type=del size=2174490 2026-02-24T22:10:44.844 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.841+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=UOWST2LVH7AE type=del size=2408397 2026-02-24T22:10:44.845 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.841+0000 7f7e64854980 1 < completed delete operation key=Q7BP8FXTARDW tag=F8LF8CU07H2M type=del 2026-02-24T22:10:44.846 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.845+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~GDJ0IRJQKLIT.0 size=157451 2026-02-24T22:10:44.846 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.845+0000 7f7e64854980 1 > prepared multipart upload key=Y8GMOUA977H3 upload=GDJ0IRJQKLIT tag=8W9UJKK4MON4 type=write size=1206027 2026-02-24T22:10:44.848 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.845+0000 7f7e64854980 1 < suggested remove operation key=Y8GMOUA977H3 tag=EL6R33QDUKX9 type=del 2026-02-24T22:10:44.849 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.845+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=8GGWEB2WHR9N type=write size=279689 2026-02-24T22:10:44.850 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.849+0000 7f7e64854980 1 < completed write operation key=4Y3974PYYT78 tag=UYIN8S8MHHB6 type=write size=834665 2026-02-24T22:10:44.851 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.849+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~9YW560H4OEZ1.0 size=1048576 2026-02-24T22:10:44.852 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.849+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~9YW560H4OEZ1.1 size=1048576 2026-02-24T22:10:44.852 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.849+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~9YW560H4OEZ1.2 size=742712 2026-02-24T22:10:44.852 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.849+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=9YW560H4OEZ1 tag=RH11YJ7DIDKP type=write size=3888440 2026-02-24T22:10:44.853 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.849+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~S1BFEB63PC6O.0 size=1048576 2026-02-24T22:10:44.854 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.853+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~S1BFEB63PC6O.1 size=69332 2026-02-24T22:10:44.855 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.853+0000 7f7e64854980 1 < completed multipart upload key=0B2V0MHWSOC9 upload=S1BFEB63PC6O tag=CLV3MUMRNJHB type=write size=2166484 2026-02-24T22:10:44.856 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.853+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~U3D30JSZ6NMT.0 size=1048576 2026-02-24T22:10:44.857 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.853+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~U3D30JSZ6NMT.1 size=1048576 2026-02-24T22:10:44.858 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.857+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~U3D30JSZ6NMT.2 size=141907 2026-02-24T22:10:44.858 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.857+0000 7f7e64854980 1 > prepared multipart upload key=Y8GMOUA977H3 upload=U3D30JSZ6NMT tag=2YUU5SRHLKI0 type=write size=3287635 2026-02-24T22:10:44.859 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.857+0000 7f7e64854980 1 < suggested update operation key=4Y3974PYYT78 tag=IZLAMVW0PRJA type=del size=1653183 2026-02-24T22:10:44.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.857+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=1Q1TTAVRS624 type=del size=1457638 2026-02-24T22:10:44.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.857+0000 7f7e64854980 1 < completed write operation key=78GUTC5THI2P tag=6I36Z54A8XCF type=write size=743972 2026-02-24T22:10:44.862 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.861+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~ZN36X6XK8MFJ.0 size=22758 2026-02-24T22:10:44.862 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.861+0000 7f7e64854980 1 > prepared multipart upload key=Y8GMOUA977H3 upload=ZN36X6XK8MFJ tag=UTXXU76CXF3G type=write size=1071334 2026-02-24T22:10:44.863 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.861+0000 7f7e64854980 1 < completed delete operation key=L9NEDH44BIGA tag=JT7RM23IX5CS type=del 2026-02-24T22:10:44.864 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.861+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~320EKJIXSKIY.0 size=105658 2026-02-24T22:10:44.865 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.861+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=320EKJIXSKIY tag=E7K9ZHV6S6I8 type=write size=1154234 2026-02-24T22:10:44.865 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.861+0000 7f7e64854980 1 < canceled operation key=78GUTC5THI2P tag=K3YDITV53TCU type=write size=412216 2026-02-24T22:10:44.866 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.865+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=POQNFBL1SQW6 type=del size=1952134 2026-02-24T22:10:44.867 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.865+0000 7f7e64854980 1 < suggested update operation key=PKRHWM69GBJI tag=3OQ93W4J5CMF type=del size=3853627 2026-02-24T22:10:44.870 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.869+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~Q9IG9KP35TDH.0 size=1048576 2026-02-24T22:10:44.871 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.869+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~Q9IG9KP35TDH.1 size=325915 2026-02-24T22:10:44.871 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.869+0000 7f7e64854980 1 > prepared multipart upload key=N3ZUNSI5K8HS upload=Q9IG9KP35TDH tag=2DRC62H5AQV7 type=write size=2423067 2026-02-24T22:10:44.871 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.869+0000 7f7e64854980 1 < completed delete operation key=Y3L3CJ51C1WR tag=A458UOEW7SF5 type=del 2026-02-24T22:10:44.872 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.869+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=XMY6X8DKC4IC type=write size=317587 2026-02-24T22:10:44.874 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.873+0000 7f7e64854980 1 < suggested update operation key=L9NEDH44BIGA tag=X2IR6QNPOU53 type=del size=3699184 2026-02-24T22:10:44.874 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.873+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=IWL278H01LJL type=del size=2004876 2026-02-24T22:10:44.875 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.873+0000 7f7e64854980 1 < canceled part key=_multipart_HOHKR92SN25M.2~WKC90MT7VB4W.0 size=1048576 2026-02-24T22:10:44.875 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.873+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~WKC90MT7VB4W.1 size=793485 2026-02-24T22:10:44.877 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.873+0000 7f7e64854980 1 < canceled multipart upload key=HOHKR92SN25M upload=WKC90MT7VB4W tag=XQKKD47JTCY4 type=write size=2890637 2026-02-24T22:10:44.878 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.877+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~S6FE8RCFKC8C.0 size=1048576 2026-02-24T22:10:44.879 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.877+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~S6FE8RCFKC8C.1 size=140790 2026-02-24T22:10:44.879 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.877+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=S6FE8RCFKC8C tag=9R6VB4DRKPZ6 type=write size=2237942 2026-02-24T22:10:44.879 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.877+0000 7f7e64854980 -1 < no bi entry to suggest for operation key=Y8GMOUA977H3 tag=8JMO2HWIVD63 type=del size=599770: (2) No such file or directory 2026-02-24T22:10:44.880 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.877+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=LUJ5N3U7OIGE type=del size=1795671 2026-02-24T22:10:44.880 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.877+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~LUBCBUMS7LOI.0 size=134781 2026-02-24T22:10:44.882 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.881+0000 7f7e64854980 1 < completed multipart upload key=L60HQ7M3E3LL upload=LUBCBUMS7LOI tag=KZP2L3MI850N type=write size=1183357 2026-02-24T22:10:44.883 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.881+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=HZN8JGAN05K6 type=del size=2412672 2026-02-24T22:10:44.883 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.881+0000 7f7e64854980 -1 < failed to complete operation key=4Y3974PYYT78 tag=8Q4HP6IUL5LE type=write size=244301: (22) Invalid argument 2026-02-24T22:10:44.884 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.881+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=D4NPLFN32Q9H type=del size=3400206 2026-02-24T22:10:44.885 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.881+0000 7f7e64854980 1 < completed delete operation key=78GUTC5THI2P tag=UOWST2LVH7AE type=del 2026-02-24T22:10:44.886 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.885+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~LKBNKDVP2TO0.0 size=251635 2026-02-24T22:10:44.887 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.885+0000 7f7e64854980 1 > prepared multipart upload key=EB04ETNYM087 upload=LKBNKDVP2TO0 tag=QJS8OQ4EGRKW type=write size=1300211 2026-02-24T22:10:44.887 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.885+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~GDJ0IRJQKLIT.0 size=157451 2026-02-24T22:10:44.889 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.885+0000 7f7e64854980 1 < completed multipart upload key=Y8GMOUA977H3 upload=GDJ0IRJQKLIT tag=8W9UJKK4MON4 type=write size=1206027 2026-02-24T22:10:44.889 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.889+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=EGN6Y9PZM2MV type=del size=2527783 2026-02-24T22:10:44.891 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.889+0000 7f7e64854980 1 < suggested remove operation key=36MKR2OZN7K9 tag=8GGWEB2WHR9N type=write 2026-02-24T22:10:44.892 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.889+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=ZEK7F2Z46MQA type=del size=47120 2026-02-24T22:10:44.892 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.889+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~9YW560H4OEZ1.0 size=1048576 2026-02-24T22:10:44.893 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.889+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~9YW560H4OEZ1.1 size=1048576 2026-02-24T22:10:44.894 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.893+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~9YW560H4OEZ1.2 size=742712 2026-02-24T22:10:44.895 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.893+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=9YW560H4OEZ1 tag=RH11YJ7DIDKP type=write size=3888440 2026-02-24T22:10:44.896 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.893+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=QLBBN0V14ZIO type=write size=943423 2026-02-24T22:10:44.896 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.893+0000 7f7e64854980 1 < canceled part key=_multipart_Y8GMOUA977H3.2~U3D30JSZ6NMT.0 size=1048576 2026-02-24T22:10:44.896 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.893+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~U3D30JSZ6NMT.1 size=1048576 2026-02-24T22:10:44.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.893+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~U3D30JSZ6NMT.2 size=141907 2026-02-24T22:10:44.898 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.897+0000 7f7e64854980 1 < canceled multipart upload key=Y8GMOUA977H3 upload=U3D30JSZ6NMT tag=2YUU5SRHLKI0 type=write size=3287635 2026-02-24T22:10:44.899 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.897+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=G167SV81YQJ6 type=write size=320370 2026-02-24T22:10:44.901 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.897+0000 7f7e64854980 1 < suggested remove operation key=EB04ETNYM087 tag=1Q1TTAVRS624 type=del 2026-02-24T22:10:44.901 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.901+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=0IID7VU7K2GM type=del size=3898169 2026-02-24T22:10:44.902 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.901+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~ZN36X6XK8MFJ.0 size=22758 2026-02-24T22:10:44.903 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.901+0000 7f7e64854980 1 < completed multipart upload key=Y8GMOUA977H3 upload=ZN36X6XK8MFJ tag=UTXXU76CXF3G type=write size=1071334 2026-02-24T22:10:44.904 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.901+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=ZR3VP43138NF type=del size=3836496 2026-02-24T22:10:44.905 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.901+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~320EKJIXSKIY.0 size=105658 2026-02-24T22:10:44.906 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.905+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=320EKJIXSKIY tag=E7K9ZHV6S6I8 type=write size=1154234 2026-02-24T22:10:44.907 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.905+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=KF4F1PU02UD3 type=del size=2553081 2026-02-24T22:10:44.908 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.905+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=POQNFBL1SQW6 type=del 2026-02-24T22:10:44.909 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.905+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~ZFBHKRYYNE33.0 size=1048576 2026-02-24T22:10:44.910 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.909+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~ZFBHKRYYNE33.1 size=922643 2026-02-24T22:10:44.910 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.909+0000 7f7e64854980 1 > prepared multipart upload key=Y8GMOUA977H3 upload=ZFBHKRYYNE33 tag=Z50YKQEN0FR3 type=write size=3019795 2026-02-24T22:10:44.911 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.909+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~Q9IG9KP35TDH.0 size=1048576 2026-02-24T22:10:44.911 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.909+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~Q9IG9KP35TDH.1 size=325915 2026-02-24T22:10:44.912 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.909+0000 7f7e64854980 1 < completed multipart upload key=N3ZUNSI5K8HS upload=Q9IG9KP35TDH tag=2DRC62H5AQV7 type=write size=2423067 2026-02-24T22:10:44.913 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.913+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=YO8A2XE4Z81D type=del size=3436479 2026-02-24T22:10:44.914 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.913+0000 7f7e64854980 1 < completed write operation key=GH7723GNI2WM tag=XMY6X8DKC4IC type=write size=317587 2026-02-24T22:10:44.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.913+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=EIJF1FYRU4Y6 type=del size=706267 2026-02-24T22:10:44.916 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.913+0000 7f7e64854980 1 < completed delete operation key=0AXS7DY23UZU tag=IWL278H01LJL type=del 2026-02-24T22:10:44.917 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.913+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~P7HMXAIGISQZ.0 size=728421 2026-02-24T22:10:44.917 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.913+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=P7HMXAIGISQZ tag=KGBL088CHIS5 type=write size=1776997 2026-02-24T22:10:44.918 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.917+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~S6FE8RCFKC8C.0 size=1048576 2026-02-24T22:10:44.918 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.917+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~S6FE8RCFKC8C.1 size=140790 2026-02-24T22:10:44.920 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.917+0000 7f7e64854980 1 < completed multipart upload key=HOHKR92SN25M upload=S6FE8RCFKC8C tag=9R6VB4DRKPZ6 type=write size=2237942 2026-02-24T22:10:44.921 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.917+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~BKYR1J9KXLBC.0 size=1048576 2026-02-24T22:10:44.922 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.921+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~BKYR1J9KXLBC.1 size=1048576 2026-02-24T22:10:44.922 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.921+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~BKYR1J9KXLBC.2 size=695014 2026-02-24T22:10:44.922 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.921+0000 7f7e64854980 1 > prepared multipart upload key=EB04ETNYM087 upload=BKYR1J9KXLBC tag=4LSFJX352PCX type=write size=3840742 2026-02-24T22:10:44.924 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.921+0000 7f7e64854980 1 < suggested update operation key=SGQYQ2TIVJPS tag=LUJ5N3U7OIGE type=del size=1795671 2026-02-24T22:10:44.925 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.921+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~91Y2OD9VVYBC.0 size=33140 2026-02-24T22:10:44.925 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.921+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=91Y2OD9VVYBC tag=9CB7GV1AD4M3 type=write size=1081716 2026-02-24T22:10:44.926 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.925+0000 7f7e64854980 1 < canceled operation key=NL60Q5QOSP8E tag=HZN8JGAN05K6 type=del size=2412672 2026-02-24T22:10:44.927 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.925+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=VTAVK8QV8C0U type=write size=729652 2026-02-24T22:10:44.927 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.925+0000 7f7e64854980 1 < canceled operation key=PKRHWM69GBJI tag=D4NPLFN32Q9H type=del size=3400206 2026-02-24T22:10:44.928 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.925+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=QV382S022WMP type=del size=770180 2026-02-24T22:10:44.929 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.925+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~LKBNKDVP2TO0.0 size=251635 2026-02-24T22:10:44.930 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.929+0000 7f7e64854980 1 < completed multipart upload key=EB04ETNYM087 upload=LKBNKDVP2TO0 tag=QJS8OQ4EGRKW type=write size=1300211 2026-02-24T22:10:44.931 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.929+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~TZH9SVJTE30J.0 size=1048576 2026-02-24T22:10:44.932 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.929+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~TZH9SVJTE30J.1 size=341705 2026-02-24T22:10:44.932 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.929+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=TZH9SVJTE30J tag=YSD4AQMWT7DN type=write size=2438857 2026-02-24T22:10:44.933 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.929+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=EGN6Y9PZM2MV type=del 2026-02-24T22:10:44.934 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.933+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=1GH3XERT0UIK type=del size=3623447 2026-02-24T22:10:44.935 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.933+0000 7f7e64854980 1 < suggested update operation key=A6PH77Q78VEN tag=ZEK7F2Z46MQA type=del size=47120 2026-02-24T22:10:44.936 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.933+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~Y7DXKRRPT8QY.0 size=1048576 2026-02-24T22:10:44.937 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.933+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~Y7DXKRRPT8QY.1 size=1048576 2026-02-24T22:10:44.938 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.937+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~Y7DXKRRPT8QY.2 size=80516 2026-02-24T22:10:44.938 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.937+0000 7f7e64854980 1 > prepared multipart upload key=UQLUT9ZAI61V upload=Y7DXKRRPT8QY tag=FDUZ4U2HZMWB type=write size=3226244 2026-02-24T22:10:44.938 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.937+0000 7f7e64854980 1 < completed write operation key=HKOQRSF6XEXB tag=QLBBN0V14ZIO type=write size=943423 2026-02-24T22:10:44.939 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.937+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=RMIUUX8RI6D9 type=write size=852013 2026-02-24T22:10:44.940 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.937+0000 7f7e64854980 1 < completed write operation key=EJCX06TE5RTZ tag=G167SV81YQJ6 type=write size=320370 2026-02-24T22:10:44.942 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.941+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~40MDU5RT2XQC.0 size=1048576 2026-02-24T22:10:44.943 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.941+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~40MDU5RT2XQC.1 size=1048576 2026-02-24T22:10:44.943 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.941+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~40MDU5RT2XQC.2 size=150562 2026-02-24T22:10:44.943 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.941+0000 7f7e64854980 1 > prepared multipart upload key=N3ZUNSI5K8HS upload=40MDU5RT2XQC tag=JXY1VZRPIYQ3 type=write size=3296290 2026-02-24T22:10:44.944 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.941+0000 7f7e64854980 1 < completed delete operation key=0B2V0MHWSOC9 tag=0IID7VU7K2GM type=del 2026-02-24T22:10:44.945 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.941+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=S0OA6NRE8Z4S type=del size=505878 2026-02-24T22:10:44.946 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.945+0000 7f7e64854980 1 < suggested update operation key=3900SFJK1YQU tag=ZR3VP43138NF type=del size=3836496 2026-02-24T22:10:44.947 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.945+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=W7S0KSM6P828 type=del size=1019894 2026-02-24T22:10:44.948 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.945+0000 7f7e64854980 1 < completed delete operation key=L60HQ7M3E3LL tag=KF4F1PU02UD3 type=del 2026-02-24T22:10:44.949 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.945+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=FD83US2BPHG1 type=del size=939015 2026-02-24T22:10:44.950 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.949+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~ZFBHKRYYNE33.0 size=1048576 2026-02-24T22:10:44.950 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.949+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~ZFBHKRYYNE33.1 size=922643 2026-02-24T22:10:44.951 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.949+0000 7f7e64854980 1 < completed multipart upload key=Y8GMOUA977H3 upload=ZFBHKRYYNE33 tag=Z50YKQEN0FR3 type=write size=3019795 2026-02-24T22:10:44.953 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.949+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~1D8VQ3C2LRIA.0 size=822772 2026-02-24T22:10:44.953 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.949+0000 7f7e64854980 1 > prepared multipart upload key=SGQYQ2TIVJPS upload=1D8VQ3C2LRIA tag=MEKQSQX16CVU type=write size=1871348 2026-02-24T22:10:44.953 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.953+0000 7f7e64854980 1 < canceled operation key=EB04ETNYM087 tag=YO8A2XE4Z81D type=del size=3436479 2026-02-24T22:10:44.954 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.953+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=F6VZLGUWQXQJ type=del size=3873665 2026-02-24T22:10:44.955 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.953+0000 7f7e64854980 1 < completed delete operation key=2KKQKHXO7K3K tag=EIJF1FYRU4Y6 type=del 2026-02-24T22:10:44.956 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.953+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~FKCHAKUZI1QB.0 size=409071 2026-02-24T22:10:44.956 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.953+0000 7f7e64854980 1 > prepared multipart upload key=A6PH77Q78VEN upload=FKCHAKUZI1QB tag=OZ8HOPDZKPUX type=write size=1457647 2026-02-24T22:10:44.957 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.953+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~P7HMXAIGISQZ.0 size=728421 2026-02-24T22:10:44.958 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.957+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=P7HMXAIGISQZ tag=KGBL088CHIS5 type=write size=1776997 2026-02-24T22:10:44.960 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.957+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~FBPLF6RRNKVA.0 size=620965 2026-02-24T22:10:44.960 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.957+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=FBPLF6RRNKVA tag=XUK5KKSZREGU type=write size=1669541 2026-02-24T22:10:44.960 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.957+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~BKYR1J9KXLBC.0 size=1048576 2026-02-24T22:10:44.961 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.957+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~BKYR1J9KXLBC.1 size=1048576 2026-02-24T22:10:44.961 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.961+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~BKYR1J9KXLBC.2 size=695014 2026-02-24T22:10:44.963 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.961+0000 7f7e64854980 1 < completed multipart upload key=EB04ETNYM087 upload=BKYR1J9KXLBC tag=4LSFJX352PCX type=write size=3840742 2026-02-24T22:10:44.964 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.961+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=CMH1EHPN4ULC type=del size=3515681 2026-02-24T22:10:44.964 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.961+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~91Y2OD9VVYBC.0 size=33140 2026-02-24T22:10:44.966 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.965+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=91Y2OD9VVYBC tag=9CB7GV1AD4M3 type=write size=1081716 2026-02-24T22:10:44.966 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.965+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=DQ650MNQ2JZO type=del size=252571 2026-02-24T22:10:44.967 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.965+0000 7f7e64854980 1 < canceled operation key=Z0OXREAUJVA8 tag=VTAVK8QV8C0U type=write size=729652 2026-02-24T22:10:44.968 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.965+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=H3B5U88NU9A4 type=del size=3761833 2026-02-24T22:10:44.969 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.965+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=QV382S022WMP type=del 2026-02-24T22:10:44.969 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.969+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=B9KCI19CORL9 type=write size=747742 2026-02-24T22:10:44.970 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.969+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~TZH9SVJTE30J.0 size=1048576 2026-02-24T22:10:44.970 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.969+0000 7f7e64854980 1 < canceled part key=_multipart_L9NEDH44BIGA.2~TZH9SVJTE30J.1 size=341705 2026-02-24T22:10:44.971 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.969+0000 7f7e64854980 1 < canceled multipart upload key=L9NEDH44BIGA upload=TZH9SVJTE30J tag=YSD4AQMWT7DN type=write size=2438857 2026-02-24T22:10:44.972 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.969+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=DCF8YTFPL89E type=del size=2867812 2026-02-24T22:10:44.974 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.973+0000 7f7e64854980 1 < suggested update operation key=CI8DA17DX8LU tag=1GH3XERT0UIK type=del size=3623447 2026-02-24T22:10:44.974 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.973+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=8WLWMPOSD50A type=write size=370091 2026-02-24T22:10:44.975 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.973+0000 7f7e64854980 1 < canceled part key=_multipart_UQLUT9ZAI61V.2~Y7DXKRRPT8QY.0 size=1048576 2026-02-24T22:10:44.975 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.973+0000 7f7e64854980 1 < completed part key=_multipart_UQLUT9ZAI61V.2~Y7DXKRRPT8QY.1 size=1048576 2026-02-24T22:10:44.976 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.973+0000 7f7e64854980 1 < completed part key=_multipart_UQLUT9ZAI61V.2~Y7DXKRRPT8QY.2 size=80516 2026-02-24T22:10:44.977 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.973+0000 7f7e64854980 1 < canceled multipart upload key=UQLUT9ZAI61V upload=Y7DXKRRPT8QY tag=FDUZ4U2HZMWB type=write size=3226244 2026-02-24T22:10:44.978 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.977+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=OUDCZG2M1Y6Z type=del size=2675592 2026-02-24T22:10:44.978 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.977+0000 7f7e64854980 1 < completed write operation key=HKOQRSF6XEXB tag=RMIUUX8RI6D9 type=write size=852013 2026-02-24T22:10:44.980 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.977+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~FJXUA8RJYMA6.0 size=1048576 2026-02-24T22:10:44.980 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.977+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~FJXUA8RJYMA6.1 size=314099 2026-02-24T22:10:44.980 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.977+0000 7f7e64854980 1 > prepared multipart upload key=36MKR2OZN7K9 upload=FJXUA8RJYMA6 tag=3DEENA49H5BS type=write size=2411251 2026-02-24T22:10:44.981 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.977+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~40MDU5RT2XQC.0 size=1048576 2026-02-24T22:10:44.982 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.981+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~40MDU5RT2XQC.1 size=1048576 2026-02-24T22:10:44.982 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.981+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~40MDU5RT2XQC.2 size=150562 2026-02-24T22:10:44.984 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.981+0000 7f7e64854980 1 < completed multipart upload key=N3ZUNSI5K8HS upload=40MDU5RT2XQC tag=JXY1VZRPIYQ3 type=write size=3296290 2026-02-24T22:10:44.985 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.981+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~W8GNJMONU1AX.0 size=1048576 2026-02-24T22:10:44.985 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.985+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~W8GNJMONU1AX.1 size=1048576 2026-02-24T22:10:44.986 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.985+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~W8GNJMONU1AX.2 size=909399 2026-02-24T22:10:44.986 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.985+0000 7f7e64854980 1 > prepared multipart upload key=Y3L3CJ51C1WR upload=W8GNJMONU1AX tag=65E1QB2KGAGA type=write size=4055127 2026-02-24T22:10:44.987 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.985+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=S0OA6NRE8Z4S type=del 2026-02-24T22:10:44.988 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.985+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=AM6I5DB4N2H3 type=del size=736851 2026-02-24T22:10:44.989 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.985+0000 7f7e64854980 1 < completed delete operation key=EB04ETNYM087 tag=W7S0KSM6P828 type=del 2026-02-24T22:10:44.990 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.989+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~SLDO7W5J4MOK.0 size=1048576 2026-02-24T22:10:44.991 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.989+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~SLDO7W5J4MOK.1 size=1048576 2026-02-24T22:10:44.991 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.989+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~SLDO7W5J4MOK.2 size=26496 2026-02-24T22:10:44.991 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.989+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=SLDO7W5J4MOK tag=O324LJV1IZL4 type=write size=3172224 2026-02-24T22:10:44.992 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.989+0000 7f7e64854980 1 < completed delete operation key=Z0OXREAUJVA8 tag=FD83US2BPHG1 type=del 2026-02-24T22:10:44.993 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.989+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=9O75PIV7GLA1 type=write size=304120 2026-02-24T22:10:44.993 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.993+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~1D8VQ3C2LRIA.0 size=822772 2026-02-24T22:10:44.994 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.993+0000 7f7e64854980 1 < completed multipart upload key=SGQYQ2TIVJPS upload=1D8VQ3C2LRIA tag=MEKQSQX16CVU type=write size=1871348 2026-02-24T22:10:44.995 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.993+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=35CLGI0KLZFE type=del size=3350170 2026-02-24T22:10:44.997 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.993+0000 7f7e64854980 1 < suggested remove operation key=TF5B43G7KCYT tag=F6VZLGUWQXQJ type=del 2026-02-24T22:10:44.997 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.997+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=TYS0E5TPZACF type=write size=342254 2026-02-24T22:10:44.998 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.997+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~FKCHAKUZI1QB.0 size=409071 2026-02-24T22:10:45.000 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.997+0000 7f7e64854980 1 < completed multipart upload key=A6PH77Q78VEN upload=FKCHAKUZI1QB tag=OZ8HOPDZKPUX type=write size=1457647 2026-02-24T22:10:45.000 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:44.997+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=KRE82MDTX9PH type=del size=408392 2026-02-24T22:10:45.001 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.001+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~FBPLF6RRNKVA.0 size=620965 2026-02-24T22:10:45.003 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.001+0000 7f7e64854980 1 < completed multipart upload key=EJCX06TE5RTZ upload=FBPLF6RRNKVA tag=XUK5KKSZREGU type=write size=1669541 2026-02-24T22:10:45.004 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.001+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~SN3EHC0HU3PQ.0 size=1048576 2026-02-24T22:10:45.005 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.001+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~SN3EHC0HU3PQ.1 size=1048576 2026-02-24T22:10:45.005 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.001+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~SN3EHC0HU3PQ.2 size=309068 2026-02-24T22:10:45.005 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.001+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=SN3EHC0HU3PQ tag=4KWCAK07MTL2 type=write size=3454796 2026-02-24T22:10:45.006 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.005+0000 7f7e64854980 -1 < failed to complete operation key=CI8DA17DX8LU tag=CMH1EHPN4ULC type=del size=3515681: (22) Invalid argument 2026-02-24T22:10:45.007 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.005+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~5H2NHSKD2DNI.0 size=1048576 2026-02-24T22:10:45.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.005+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~5H2NHSKD2DNI.1 size=201010 2026-02-24T22:10:45.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.005+0000 7f7e64854980 1 > prepared multipart upload key=PKRHWM69GBJI upload=5H2NHSKD2DNI tag=PLSQY9TDI3NI type=write size=2298162 2026-02-24T22:10:45.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.005+0000 7f7e64854980 1 < completed delete operation key=36MKR2OZN7K9 tag=DQ650MNQ2JZO type=del 2026-02-24T22:10:45.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.009+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=HORE8XSCOP1D type=del size=1874457 2026-02-24T22:10:45.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.009+0000 7f7e64854980 1 < completed delete operation key=LSRPN33Z2B9N tag=H3B5U88NU9A4 type=del 2026-02-24T22:10:45.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.009+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~FFHHEMPSGSR7.0 size=1048576 2026-02-24T22:10:45.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.009+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~FFHHEMPSGSR7.1 size=5719 2026-02-24T22:10:45.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.009+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=FFHHEMPSGSR7 tag=8TLAJCW8LDME type=write size=2102871 2026-02-24T22:10:45.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.013+0000 7f7e64854980 1 < completed write operation key=Q7BP8FXTARDW tag=B9KCI19CORL9 type=write size=747742 2026-02-24T22:10:45.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.013+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=K08FYF96TDNU type=del size=478406 2026-02-24T22:10:45.016 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.013+0000 7f7e64854980 1 < suggested remove operation key=0AXS7DY23UZU tag=DCF8YTFPL89E type=del 2026-02-24T22:10:45.016 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.013+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=3QVY7TRB1S4S type=write size=745821 2026-02-24T22:10:45.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.013+0000 7f7e64854980 1 < completed write operation key=D0MKGNRDW6L9 tag=8WLWMPOSD50A type=write size=370091 2026-02-24T22:10:45.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.017+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=ROCVBVRAOQP6 type=del size=3079957 2026-02-24T22:10:45.019 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.017+0000 7f7e64854980 1 < completed delete operation key=PKRHWM69GBJI tag=OUDCZG2M1Y6Z type=del 2026-02-24T22:10:45.019 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.017+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=3T3LWMPI6M5O type=del size=79497 2026-02-24T22:10:45.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.017+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~FJXUA8RJYMA6.0 size=1048576 2026-02-24T22:10:45.021 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.017+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~FJXUA8RJYMA6.1 size=314099 2026-02-24T22:10:45.022 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.021+0000 7f7e64854980 1 < completed multipart upload key=36MKR2OZN7K9 upload=FJXUA8RJYMA6 tag=3DEENA49H5BS type=write size=2411251 2026-02-24T22:10:45.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.021+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=0YHJYEDVDCQ7 type=del size=2753584 2026-02-24T22:10:45.024 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.021+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~W8GNJMONU1AX.0 size=1048576 2026-02-24T22:10:45.024 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.021+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~W8GNJMONU1AX.1 size=1048576 2026-02-24T22:10:45.025 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.021+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~W8GNJMONU1AX.2 size=909399 2026-02-24T22:10:45.026 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.025+0000 7f7e64854980 1 < completed multipart upload key=Y3L3CJ51C1WR upload=W8GNJMONU1AX tag=65E1QB2KGAGA type=write size=4055127 2026-02-24T22:10:45.027 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.025+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~X07IRHGN1T2Q.0 size=1048576 2026-02-24T22:10:45.028 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.025+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~X07IRHGN1T2Q.1 size=1048576 2026-02-24T22:10:45.028 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.025+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~X07IRHGN1T2Q.2 size=398029 2026-02-24T22:10:45.029 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.025+0000 7f7e64854980 1 > prepared multipart upload key=TF5B43G7KCYT upload=X07IRHGN1T2Q tag=JB38HNBOTXXQ type=write size=3543757 2026-02-24T22:10:45.029 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.029+0000 7f7e64854980 1 < completed delete operation key=D0MKGNRDW6L9 tag=AM6I5DB4N2H3 type=del 2026-02-24T22:10:45.031 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.029+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~YR6KYNZR34Q3.0 size=1048576 2026-02-24T22:10:45.031 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.029+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~YR6KYNZR34Q3.1 size=1048576 2026-02-24T22:10:45.032 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.029+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~YR6KYNZR34Q3.2 size=896766 2026-02-24T22:10:45.032 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.029+0000 7f7e64854980 1 > prepared multipart upload key=78GUTC5THI2P upload=YR6KYNZR34Q3 tag=2SQCJ2HURCH4 type=write size=4042494 2026-02-24T22:10:45.032 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.029+0000 7f7e64854980 1 < canceled part key=_multipart_EJCX06TE5RTZ.2~SLDO7W5J4MOK.0 size=1048576 2026-02-24T22:10:45.032 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.029+0000 7f7e64854980 1 < canceled part key=_multipart_EJCX06TE5RTZ.2~SLDO7W5J4MOK.1 size=1048576 2026-02-24T22:10:45.033 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.029+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~SLDO7W5J4MOK.2 size=26496 2026-02-24T22:10:45.034 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.033+0000 7f7e64854980 1 < canceled multipart upload key=EJCX06TE5RTZ upload=SLDO7W5J4MOK tag=O324LJV1IZL4 type=write size=3172224 2026-02-24T22:10:45.035 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.033+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~8BXX75WFQDDJ.0 size=1030017 2026-02-24T22:10:45.035 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.033+0000 7f7e64854980 1 > prepared multipart upload key=78GUTC5THI2P upload=8BXX75WFQDDJ tag=N9QI8F9BI0VU type=write size=2078593 2026-02-24T22:10:45.036 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.033+0000 7f7e64854980 1 < completed write operation key=GH7723GNI2WM tag=9O75PIV7GLA1 type=write size=304120 2026-02-24T22:10:45.037 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.033+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=BN7XI7MAHC1Y type=del size=3122730 2026-02-24T22:10:45.037 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.037+0000 7f7e64854980 1 < completed delete operation key=Z0OXREAUJVA8 tag=35CLGI0KLZFE type=del 2026-02-24T22:10:45.038 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.037+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=B7A1ZQLVNY3B type=del size=309097 2026-02-24T22:10:45.040 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.037+0000 7f7e64854980 1 < suggested update operation key=L60HQ7M3E3LL tag=TYS0E5TPZACF type=write size=342254 2026-02-24T22:10:45.040 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.037+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=ADHARJZKE3X3 type=write size=948598 2026-02-24T22:10:45.041 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.041+0000 7f7e64854980 1 < completed delete operation key=GH7723GNI2WM tag=KRE82MDTX9PH type=del 2026-02-24T22:10:45.042 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.041+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~9C3DH9F4OQH1.0 size=1048576 2026-02-24T22:10:45.043 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.041+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~9C3DH9F4OQH1.1 size=1048576 2026-02-24T22:10:45.044 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.041+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~9C3DH9F4OQH1.2 size=13407 2026-02-24T22:10:45.044 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.041+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=9C3DH9F4OQH1 tag=1BQV3DURJH5S type=write size=3159135 2026-02-24T22:10:45.045 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.041+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~SN3EHC0HU3PQ.0 size=1048576 2026-02-24T22:10:45.045 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.041+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~SN3EHC0HU3PQ.1 size=1048576 2026-02-24T22:10:45.046 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.045+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~SN3EHC0HU3PQ.2 size=309068 2026-02-24T22:10:45.047 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.045+0000 7f7e64854980 1 < completed multipart upload key=TXRUQQ456DD8 upload=SN3EHC0HU3PQ tag=4KWCAK07MTL2 type=write size=3454796 2026-02-24T22:10:45.048 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.045+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=HURY010RMNY9 type=del size=2784525 2026-02-24T22:10:45.048 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.045+0000 7f7e64854980 1 < canceled part key=_multipart_PKRHWM69GBJI.2~5H2NHSKD2DNI.0 size=1048576 2026-02-24T22:10:45.049 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.045+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~5H2NHSKD2DNI.1 size=201010 2026-02-24T22:10:45.050 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.049+0000 7f7e64854980 1 < canceled multipart upload key=PKRHWM69GBJI upload=5H2NHSKD2DNI tag=PLSQY9TDI3NI type=write size=2298162 2026-02-24T22:10:45.051 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.049+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=R3A40SH8JUK0 type=del size=1142999 2026-02-24T22:10:45.052 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.049+0000 7f7e64854980 1 < suggested update operation key=Q7BP8FXTARDW tag=HORE8XSCOP1D type=del size=1874457 2026-02-24T22:10:45.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.049+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=I4MSB3TQEIWU type=write size=715502 2026-02-24T22:10:45.054 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.053+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~FFHHEMPSGSR7.0 size=1048576 2026-02-24T22:10:45.055 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.053+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~FFHHEMPSGSR7.1 size=5719 2026-02-24T22:10:45.056 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.053+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=FFHHEMPSGSR7 tag=8TLAJCW8LDME type=write size=2102871 2026-02-24T22:10:45.058 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.057+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~X8Q0LI5CS2QE.0 size=1048576 2026-02-24T22:10:45.058 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.057+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~X8Q0LI5CS2QE.1 size=823136 2026-02-24T22:10:45.058 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.057+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=X8Q0LI5CS2QE tag=3YKE3RNZVWKR type=write size=2920288 2026-02-24T22:10:45.059 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.057+0000 7f7e64854980 1 < canceled operation key=LSRPN33Z2B9N tag=K08FYF96TDNU type=del size=478406 2026-02-24T22:10:45.060 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.057+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=52QXF0GABV7Y type=del size=731432 2026-02-24T22:10:45.061 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.061+0000 7f7e64854980 1 < suggested update operation key=L9NEDH44BIGA tag=3QVY7TRB1S4S type=write size=745821 2026-02-24T22:10:45.063 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.061+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~1MZRCUY9J3AT.0 size=1048576 2026-02-24T22:10:45.063 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.061+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~1MZRCUY9J3AT.1 size=104418 2026-02-24T22:10:45.064 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.061+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=1MZRCUY9J3AT tag=M135PY5D2SYV type=write size=2201570 2026-02-24T22:10:45.064 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.061+0000 7f7e64854980 1 < completed delete operation key=EJCX06TE5RTZ tag=ROCVBVRAOQP6 type=del 2026-02-24T22:10:45.066 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.065+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~NHN3300GFOXW.0 size=1048576 2026-02-24T22:10:45.066 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.065+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~NHN3300GFOXW.1 size=1048576 2026-02-24T22:10:45.067 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.065+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~NHN3300GFOXW.2 size=203019 2026-02-24T22:10:45.067 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.065+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=NHN3300GFOXW tag=JFSWDSBUSGLX type=write size=3348747 2026-02-24T22:10:45.068 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.065+0000 7f7e64854980 1 < completed delete operation key=36MKR2OZN7K9 tag=3T3LWMPI6M5O type=del 2026-02-24T22:10:45.069 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.069+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~VRZSOK4I9QX5.0 size=1048576 2026-02-24T22:10:45.070 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.069+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~VRZSOK4I9QX5.1 size=1048576 2026-02-24T22:10:45.071 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.069+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~VRZSOK4I9QX5.2 size=420369 2026-02-24T22:10:45.071 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.069+0000 7f7e64854980 1 > prepared multipart upload key=TF5B43G7KCYT upload=VRZSOK4I9QX5 tag=AMK75WTEHQ2H type=write size=3566097 2026-02-24T22:10:45.072 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.069+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=0YHJYEDVDCQ7 type=del 2026-02-24T22:10:45.073 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.069+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~I8ZITRK6UZUF.0 size=507868 2026-02-24T22:10:45.073 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.069+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=I8ZITRK6UZUF tag=114WVAVBEBYP type=write size=1556444 2026-02-24T22:10:45.074 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.073+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~X07IRHGN1T2Q.0 size=1048576 2026-02-24T22:10:45.075 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.073+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~X07IRHGN1T2Q.1 size=1048576 2026-02-24T22:10:45.075 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.073+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~X07IRHGN1T2Q.2 size=398029 2026-02-24T22:10:45.077 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.073+0000 7f7e64854980 1 < completed multipart upload key=TF5B43G7KCYT upload=X07IRHGN1T2Q tag=JB38HNBOTXXQ type=write size=3543757 2026-02-24T22:10:45.078 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.077+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~K3BWHBJL1R61.0 size=1048576 2026-02-24T22:10:45.079 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.077+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~K3BWHBJL1R61.1 size=1048576 2026-02-24T22:10:45.079 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.077+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~K3BWHBJL1R61.2 size=504225 2026-02-24T22:10:45.080 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.077+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=K3BWHBJL1R61 tag=BXFVPCDG4XOU type=write size=3649953 2026-02-24T22:10:45.080 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.077+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~YR6KYNZR34Q3.0 size=1048576 2026-02-24T22:10:45.081 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.077+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~YR6KYNZR34Q3.1 size=1048576 2026-02-24T22:10:45.082 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.081+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~YR6KYNZR34Q3.2 size=896766 2026-02-24T22:10:45.084 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.081+0000 7f7e64854980 1 < completed multipart upload key=78GUTC5THI2P upload=YR6KYNZR34Q3 tag=2SQCJ2HURCH4 type=write size=4042494 2026-02-24T22:10:45.084 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.081+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=7PXCJ8C7S16Z type=del size=2441309 2026-02-24T22:10:45.085 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.081+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~8BXX75WFQDDJ.0 size=1030017 2026-02-24T22:10:45.086 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.085+0000 7f7e64854980 1 < completed multipart upload key=78GUTC5THI2P upload=8BXX75WFQDDJ tag=N9QI8F9BI0VU type=write size=2078593 2026-02-24T22:10:45.087 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.085+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=5WB5Q631QI8G type=del size=488745 2026-02-24T22:10:45.088 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.085+0000 7f7e64854980 1 < completed delete operation key=EJCX06TE5RTZ tag=BN7XI7MAHC1Y type=del 2026-02-24T22:10:45.090 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.089+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~JS932SSNR9G0.0 size=1048576 2026-02-24T22:10:45.090 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.089+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~JS932SSNR9G0.1 size=79056 2026-02-24T22:10:45.091 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.089+0000 7f7e64854980 1 > prepared multipart upload key=PKRHWM69GBJI upload=JS932SSNR9G0 tag=4LF3VD7QCC44 type=write size=2176208 2026-02-24T22:10:45.092 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.089+0000 7f7e64854980 1 < suggested remove operation key=3900SFJK1YQU tag=B7A1ZQLVNY3B type=del 2026-02-24T22:10:45.093 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.089+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=V2SEIUAPDF1P type=del size=875381 2026-02-24T22:10:45.094 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.093+0000 7f7e64854980 1 < completed write operation key=A6PH77Q78VEN tag=ADHARJZKE3X3 type=write size=948598 2026-02-24T22:10:45.095 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.093+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=IU1MPWU610H2 type=del size=2155862 2026-02-24T22:10:45.095 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.093+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~9C3DH9F4OQH1.0 size=1048576 2026-02-24T22:10:45.096 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.093+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~9C3DH9F4OQH1.1 size=1048576 2026-02-24T22:10:45.097 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.093+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~9C3DH9F4OQH1.2 size=13407 2026-02-24T22:10:45.098 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.097+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=9C3DH9F4OQH1 tag=1BQV3DURJH5S type=write size=3159135 2026-02-24T22:10:45.099 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.097+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~QBBI1XGLPPX5.0 size=1048576 2026-02-24T22:10:45.100 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.097+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~QBBI1XGLPPX5.1 size=187480 2026-02-24T22:10:45.100 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.097+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=QBBI1XGLPPX5 tag=U9K3HTLZ67ZM type=write size=2284632 2026-02-24T22:10:45.101 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.097+0000 7f7e64854980 1 < completed delete operation key=2KKQKHXO7K3K tag=HURY010RMNY9 type=del 2026-02-24T22:10:45.102 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.101+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~7IYWFDRYQT87.0 size=243551 2026-02-24T22:10:45.102 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.101+0000 7f7e64854980 1 > prepared multipart upload key=LSRPN33Z2B9N upload=7IYWFDRYQT87 tag=DTWS90YQ3DQU type=write size=1292127 2026-02-24T22:10:45.103 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.101+0000 7f7e64854980 1 < completed delete operation key=TXRUQQ456DD8 tag=R3A40SH8JUK0 type=del 2026-02-24T22:10:45.104 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.101+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=XQ1BUHMCAZ1Q type=del size=2543344 2026-02-24T22:10:45.105 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.101+0000 7f7e64854980 1 < completed write operation key=R2893NES3WSF tag=I4MSB3TQEIWU type=write size=715502 2026-02-24T22:10:45.106 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.105+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=EPCVBGNOY5IU type=del size=3739165 2026-02-24T22:10:45.106 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.105+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~X8Q0LI5CS2QE.0 size=1048576 2026-02-24T22:10:45.107 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.105+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~X8Q0LI5CS2QE.1 size=823136 2026-02-24T22:10:45.108 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.105+0000 7f7e64854980 1 < completed multipart upload key=EJCX06TE5RTZ upload=X8Q0LI5CS2QE tag=3YKE3RNZVWKR type=write size=2920288 2026-02-24T22:10:45.109 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.105+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=11KUDSDTDL0B type=del size=2728554 2026-02-24T22:10:45.111 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.109+0000 7f7e64854980 1 < suggested remove operation key=78GUTC5THI2P tag=52QXF0GABV7Y type=del 2026-02-24T22:10:45.112 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.109+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=DIULSFUCTAMS type=del size=1607298 2026-02-24T22:10:45.112 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.109+0000 7f7e64854980 1 < canceled part key=_multipart_2KKQKHXO7K3K.2~1MZRCUY9J3AT.0 size=1048576 2026-02-24T22:10:45.112 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.109+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~1MZRCUY9J3AT.1 size=104418 2026-02-24T22:10:45.114 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.113+0000 7f7e64854980 1 < canceled multipart upload key=2KKQKHXO7K3K upload=1MZRCUY9J3AT tag=M135PY5D2SYV type=write size=2201570 2026-02-24T22:10:45.115 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.113+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=2MGYJDC5XJTD type=write size=169765 2026-02-24T22:10:45.115 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.113+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~NHN3300GFOXW.0 size=1048576 2026-02-24T22:10:45.116 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.113+0000 7f7e64854980 1 < canceled part key=_multipart_EJCX06TE5RTZ.2~NHN3300GFOXW.1 size=1048576 2026-02-24T22:10:45.116 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.113+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~NHN3300GFOXW.2 size=203019 2026-02-24T22:10:45.117 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.117+0000 7f7e64854980 1 < canceled multipart upload key=EJCX06TE5RTZ upload=NHN3300GFOXW tag=JFSWDSBUSGLX type=write size=3348747 2026-02-24T22:10:45.119 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.117+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~BGHCI1Q2XJC5.0 size=1048576 2026-02-24T22:10:45.119 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.117+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~BGHCI1Q2XJC5.1 size=1032782 2026-02-24T22:10:45.120 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.117+0000 7f7e64854980 1 > prepared multipart upload key=ON16L53JUSTW upload=BGHCI1Q2XJC5 tag=W7DAH949D8P4 type=write size=3129934 2026-02-24T22:10:45.120 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.117+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~VRZSOK4I9QX5.0 size=1048576 2026-02-24T22:10:45.121 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.117+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~VRZSOK4I9QX5.1 size=1048576 2026-02-24T22:10:45.122 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.121+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~VRZSOK4I9QX5.2 size=420369 2026-02-24T22:10:45.123 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.121+0000 7f7e64854980 1 < completed multipart upload key=TF5B43G7KCYT upload=VRZSOK4I9QX5 tag=AMK75WTEHQ2H type=write size=3566097 2026-02-24T22:10:45.124 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.121+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=4ERXYR6Z754Y type=del size=1574032 2026-02-24T22:10:45.125 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.121+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~I8ZITRK6UZUF.0 size=507868 2026-02-24T22:10:45.126 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.125+0000 7f7e64854980 1 < completed multipart upload key=D0MKGNRDW6L9 upload=I8ZITRK6UZUF tag=114WVAVBEBYP type=write size=1556444 2026-02-24T22:10:45.127 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.125+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=GMMQ24IIBB7R type=del size=2625422 2026-02-24T22:10:45.128 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.125+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~K3BWHBJL1R61.0 size=1048576 2026-02-24T22:10:45.128 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.125+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~K3BWHBJL1R61.1 size=1048576 2026-02-24T22:10:45.129 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.125+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~K3BWHBJL1R61.2 size=504225 2026-02-24T22:10:45.130 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.129+0000 7f7e64854980 1 < completed multipart upload key=TXRUQQ456DD8 upload=K3BWHBJL1R61 tag=BXFVPCDG4XOU type=write size=3649953 2026-02-24T22:10:45.132 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.129+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~0JQXL1KWVITY.0 size=1048576 2026-02-24T22:10:45.132 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.129+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~0JQXL1KWVITY.1 size=185610 2026-02-24T22:10:45.132 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.129+0000 7f7e64854980 1 > prepared multipart upload key=A6PH77Q78VEN upload=0JQXL1KWVITY tag=66U45J77QX7Y type=write size=2282762 2026-02-24T22:10:45.133 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.129+0000 7f7e64854980 1 < completed delete operation key=L9NEDH44BIGA tag=7PXCJ8C7S16Z type=del 2026-02-24T22:10:45.135 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.133+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~IR29GX90A7Q2.0 size=1048576 2026-02-24T22:10:45.135 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.133+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~IR29GX90A7Q2.1 size=936741 2026-02-24T22:10:45.135 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.133+0000 7f7e64854980 1 > prepared multipart upload key=Y3L3CJ51C1WR upload=IR29GX90A7Q2 tag=HTZJNQHL4TET type=write size=3033893 2026-02-24T22:10:45.136 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.133+0000 7f7e64854980 1 < completed delete operation key=L9NEDH44BIGA tag=5WB5Q631QI8G type=del 2026-02-24T22:10:45.138 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.137+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~4R98W2MNLB8D.0 size=888221 2026-02-24T22:10:45.138 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.137+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=4R98W2MNLB8D tag=Q9WTB1ZB7HBH type=write size=1936797 2026-02-24T22:10:45.138 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.137+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~JS932SSNR9G0.0 size=1048576 2026-02-24T22:10:45.139 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.137+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~JS932SSNR9G0.1 size=79056 2026-02-24T22:10:45.140 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.137+0000 7f7e64854980 1 < completed multipart upload key=PKRHWM69GBJI upload=JS932SSNR9G0 tag=4LF3VD7QCC44 type=write size=2176208 2026-02-24T22:10:45.141 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.137+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=WQN11PT13MP7 type=write size=361495 2026-02-24T22:10:45.142 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.141+0000 7f7e64854980 1 < completed delete operation key=EB04ETNYM087 tag=V2SEIUAPDF1P type=del 2026-02-24T22:10:45.143 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.141+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=YNMULHWHCZI9 type=del size=3219175 2026-02-24T22:10:45.144 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.141+0000 7f7e64854980 1 < completed delete operation key=A6PH77Q78VEN tag=IU1MPWU610H2 type=del 2026-02-24T22:10:45.145 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.141+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=A6IC5E2I5DVZ type=write size=884529 2026-02-24T22:10:45.145 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.145+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~QBBI1XGLPPX5.0 size=1048576 2026-02-24T22:10:45.145 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.145+0000 7f7e64854980 1 < canceled part key=_multipart_2KKQKHXO7K3K.2~QBBI1XGLPPX5.1 size=187480 2026-02-24T22:10:45.147 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.145+0000 7f7e64854980 1 < canceled multipart upload key=2KKQKHXO7K3K upload=QBBI1XGLPPX5 tag=U9K3HTLZ67ZM type=write size=2284632 2026-02-24T22:10:45.147 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.145+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=4HQJWGPOKX1W type=del size=854289 2026-02-24T22:10:45.149 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.145+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~7IYWFDRYQT87.0 size=243551 2026-02-24T22:10:45.150 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.149+0000 7f7e64854980 1 < completed multipart upload key=LSRPN33Z2B9N upload=7IYWFDRYQT87 tag=DTWS90YQ3DQU type=write size=1292127 2026-02-24T22:10:45.151 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.149+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~HR1728IWJG7I.0 size=1048576 2026-02-24T22:10:45.152 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.149+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~HR1728IWJG7I.1 size=1048576 2026-02-24T22:10:45.153 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.149+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~HR1728IWJG7I.2 size=934727 2026-02-24T22:10:45.153 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.149+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=HR1728IWJG7I tag=ASS8FCTVH9BT type=write size=4080455 2026-02-24T22:10:45.154 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.153+0000 7f7e64854980 1 < completed delete operation key=UQLUT9ZAI61V tag=XQ1BUHMCAZ1Q type=del 2026-02-24T22:10:45.154 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.153+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=63H7WIY47NWY type=del size=3014760 2026-02-24T22:10:45.155 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.153+0000 7f7e64854980 1 < completed delete operation key=UQLUT9ZAI61V tag=EPCVBGNOY5IU type=del 2026-02-24T22:10:45.157 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.153+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~V8HR85R3F5F0.0 size=1048576 2026-02-24T22:10:45.158 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.157+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~V8HR85R3F5F0.1 size=117347 2026-02-24T22:10:45.158 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.157+0000 7f7e64854980 1 > prepared multipart upload key=ON16L53JUSTW upload=V8HR85R3F5F0 tag=XH31JGKBFK6R type=write size=2214499 2026-02-24T22:10:45.159 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.157+0000 7f7e64854980 1 < completed delete operation key=Y3L3CJ51C1WR tag=11KUDSDTDL0B type=del 2026-02-24T22:10:45.160 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.157+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=UKXMT9RLTHBT type=del size=3946224 2026-02-24T22:10:45.161 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.157+0000 7f7e64854980 1 < canceled operation key=CI8DA17DX8LU tag=DIULSFUCTAMS type=del size=1607298 2026-02-24T22:10:45.162 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.161+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~KIJ4844LS3P9.0 size=452028 2026-02-24T22:10:45.162 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.161+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=KIJ4844LS3P9 tag=S0ZGCR2WE3MS type=write size=1500604 2026-02-24T22:10:45.163 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.161+0000 7f7e64854980 1 < completed write operation key=78GUTC5THI2P tag=2MGYJDC5XJTD type=write size=169765 2026-02-24T22:10:45.164 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.161+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~2XJ2M5HFWROP.0 size=751717 2026-02-24T22:10:45.164 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.161+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=2XJ2M5HFWROP tag=HV483Q976VJ6 type=write size=1800293 2026-02-24T22:10:45.165 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.161+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~BGHCI1Q2XJC5.0 size=1048576 2026-02-24T22:10:45.165 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.161+0000 7f7e64854980 1 < canceled part key=_multipart_ON16L53JUSTW.2~BGHCI1Q2XJC5.1 size=1032782 2026-02-24T22:10:45.166 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.165+0000 7f7e64854980 1 < canceled multipart upload key=ON16L53JUSTW upload=BGHCI1Q2XJC5 tag=W7DAH949D8P4 type=write size=3129934 2026-02-24T22:10:45.167 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.165+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=K6FCQZC9PWPW type=write size=398334 2026-02-24T22:10:45.168 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.165+0000 7f7e64854980 1 < canceled operation key=A6PH77Q78VEN tag=4ERXYR6Z754Y type=del size=1574032 2026-02-24T22:10:45.169 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.165+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=TAHHND6SDJMU type=del size=2564565 2026-02-24T22:10:45.170 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.169+0000 7f7e64854980 1 < completed delete operation key=78GUTC5THI2P tag=GMMQ24IIBB7R type=del 2026-02-24T22:10:45.171 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.169+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~OX6FLVDX71AQ.0 size=1048576 2026-02-24T22:10:45.172 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.169+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~OX6FLVDX71AQ.1 size=1048576 2026-02-24T22:10:45.172 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.169+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~OX6FLVDX71AQ.2 size=526297 2026-02-24T22:10:45.172 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.169+0000 7f7e64854980 1 > prepared multipart upload key=Y8GMOUA977H3 upload=OX6FLVDX71AQ tag=BJVZY4XL8N01 type=write size=3672025 2026-02-24T22:10:45.173 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.173+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~0JQXL1KWVITY.0 size=1048576 2026-02-24T22:10:45.174 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.173+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~0JQXL1KWVITY.1 size=185610 2026-02-24T22:10:45.175 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.173+0000 7f7e64854980 1 < completed multipart upload key=A6PH77Q78VEN upload=0JQXL1KWVITY tag=66U45J77QX7Y type=write size=2282762 2026-02-24T22:10:45.177 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.173+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~NX5XAEW2LUVM.0 size=475395 2026-02-24T22:10:45.177 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.173+0000 7f7e64854980 1 > prepared multipart upload key=L60HQ7M3E3LL upload=NX5XAEW2LUVM tag=DKEXBR1FNROK type=write size=1523971 2026-02-24T22:10:45.177 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.177+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~IR29GX90A7Q2.0 size=1048576 2026-02-24T22:10:45.178 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.177+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~IR29GX90A7Q2.1 size=936741 2026-02-24T22:10:45.179 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.177+0000 7f7e64854980 1 < completed multipart upload key=Y3L3CJ51C1WR upload=IR29GX90A7Q2 tag=HTZJNQHL4TET type=write size=3033893 2026-02-24T22:10:45.180 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.177+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=GTNYWM8Q8J6A type=del size=3431095 2026-02-24T22:10:45.181 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.177+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~4R98W2MNLB8D.0 size=888221 2026-02-24T22:10:45.182 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.181+0000 7f7e64854980 1 < completed multipart upload key=EJCX06TE5RTZ upload=4R98W2MNLB8D tag=Q9WTB1ZB7HBH type=write size=1936797 2026-02-24T22:10:45.183 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.181+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=JAJ0HBZ345NS type=del size=64775 2026-02-24T22:10:45.184 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.181+0000 7f7e64854980 1 < canceled operation key=0B2V0MHWSOC9 tag=WQN11PT13MP7 type=write size=361495 2026-02-24T22:10:45.186 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.185+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~D2TVA5OESV3Q.0 size=1048576 2026-02-24T22:10:45.186 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.185+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~D2TVA5OESV3Q.1 size=1048576 2026-02-24T22:10:45.187 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.185+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~D2TVA5OESV3Q.2 size=1185 2026-02-24T22:10:45.187 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.185+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=D2TVA5OESV3Q tag=96FJ12EVP2MD type=write size=3146913 2026-02-24T22:10:45.188 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.185+0000 7f7e64854980 1 < completed delete operation key=TF5B43G7KCYT tag=YNMULHWHCZI9 type=del 2026-02-24T22:10:45.189 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.185+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~2GBB4ZZ457DU.0 size=1048576 2026-02-24T22:10:45.190 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.189+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~2GBB4ZZ457DU.1 size=385132 2026-02-24T22:10:45.190 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.189+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=2GBB4ZZ457DU tag=9G1VZ8Z5K9FW type=write size=2482284 2026-02-24T22:10:45.190 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.189+0000 7f7e64854980 1 < completed write operation key=CI8DA17DX8LU tag=A6IC5E2I5DVZ type=write size=884529 2026-02-24T22:10:45.192 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.189+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~1SFV0WI11J8D.0 size=1048576 2026-02-24T22:10:45.193 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.189+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~1SFV0WI11J8D.1 size=813336 2026-02-24T22:10:45.193 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.189+0000 7f7e64854980 1 > prepared multipart upload key=UQLUT9ZAI61V upload=1SFV0WI11J8D tag=9Y1WBQDCNLTO type=write size=2910488 2026-02-24T22:10:45.193 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.193+0000 7f7e64854980 1 < completed delete operation key=Q7BP8FXTARDW tag=4HQJWGPOKX1W type=del 2026-02-24T22:10:45.195 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.193+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~5RW9IHLBNVTE.0 size=426178 2026-02-24T22:10:45.195 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.193+0000 7f7e64854980 1 > prepared multipart upload key=Y8GMOUA977H3 upload=5RW9IHLBNVTE tag=K13Z9RQJ9NZB type=write size=1474754 2026-02-24T22:10:45.196 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.193+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~HR1728IWJG7I.0 size=1048576 2026-02-24T22:10:45.196 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.193+0000 7f7e64854980 1 < canceled part key=_multipart_ZESKGBWZXM6O.2~HR1728IWJG7I.1 size=1048576 2026-02-24T22:10:45.196 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.193+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~HR1728IWJG7I.2 size=934727 2026-02-24T22:10:45.198 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.197+0000 7f7e64854980 1 < canceled multipart upload key=ZESKGBWZXM6O upload=HR1728IWJG7I tag=ASS8FCTVH9BT type=write size=4080455 2026-02-24T22:10:45.199 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.197+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~VOUKXB79FBFE.0 size=210848 2026-02-24T22:10:45.199 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.197+0000 7f7e64854980 1 > prepared multipart upload key=N3ZUNSI5K8HS upload=VOUKXB79FBFE tag=Z9BBUWNV3W97 type=write size=1259424 2026-02-24T22:10:45.200 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.197+0000 7f7e64854980 1 < completed delete operation key=N3ZUNSI5K8HS tag=63H7WIY47NWY type=del 2026-02-24T22:10:45.201 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.201+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~M0XFVNU6FROB.0 size=1048576 2026-02-24T22:10:45.202 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.201+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~M0XFVNU6FROB.1 size=90663 2026-02-24T22:10:45.202 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.201+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=M0XFVNU6FROB tag=AB96SCX6FN7G type=write size=2187815 2026-02-24T22:10:45.203 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.201+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~V8HR85R3F5F0.0 size=1048576 2026-02-24T22:10:45.204 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.201+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~V8HR85R3F5F0.1 size=117347 2026-02-24T22:10:45.205 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.201+0000 7f7e64854980 1 < completed multipart upload key=ON16L53JUSTW upload=V8HR85R3F5F0 tag=XH31JGKBFK6R type=write size=2214499 2026-02-24T22:10:45.206 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.205+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=ZLO21PG7K60C type=del size=3571542 2026-02-24T22:10:45.206 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.205+0000 7f7e64854980 1 < completed delete operation key=HKOQRSF6XEXB tag=UKXMT9RLTHBT type=del 2026-02-24T22:10:45.207 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.205+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=2ZOHYCBW3MQC type=del size=2342121 2026-02-24T22:10:45.208 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.205+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~KIJ4844LS3P9.0 size=452028 2026-02-24T22:10:45.209 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.209+0000 7f7e64854980 1 < completed multipart upload key=EJCX06TE5RTZ upload=KIJ4844LS3P9 tag=S0ZGCR2WE3MS type=write size=1500604 2026-02-24T22:10:45.210 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.209+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=3KIRX5GN1HUI type=del size=3342130 2026-02-24T22:10:45.211 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.209+0000 7f7e64854980 1 < canceled part key=_multipart_HOHKR92SN25M.2~2XJ2M5HFWROP.0 size=751717 2026-02-24T22:10:45.212 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.209+0000 7f7e64854980 1 < canceled multipart upload key=HOHKR92SN25M upload=2XJ2M5HFWROP tag=HV483Q976VJ6 type=write size=1800293 2026-02-24T22:10:45.213 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.209+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=DX7B1DGMT1S3 type=del size=1832305 2026-02-24T22:10:45.213 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.213+0000 7f7e64854980 1 < completed write operation key=NL60Q5QOSP8E tag=K6FCQZC9PWPW type=write size=398334 2026-02-24T22:10:45.214 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.213+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=J661L2CGHXO7 type=del size=844865 2026-02-24T22:10:45.215 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.213+0000 7f7e64854980 1 < completed delete operation key=ZESKGBWZXM6O tag=TAHHND6SDJMU type=del 2026-02-24T22:10:45.216 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.213+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=XOTZZV9DBS7M type=write size=48936 2026-02-24T22:10:45.217 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.213+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~OX6FLVDX71AQ.0 size=1048576 2026-02-24T22:10:45.217 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.217+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~OX6FLVDX71AQ.1 size=1048576 2026-02-24T22:10:45.218 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.217+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~OX6FLVDX71AQ.2 size=526297 2026-02-24T22:10:45.219 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.217+0000 7f7e64854980 1 < completed multipart upload key=Y8GMOUA977H3 upload=OX6FLVDX71AQ tag=BJVZY4XL8N01 type=write size=3672025 2026-02-24T22:10:45.221 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.217+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~U9VI9Q5NPA9T.0 size=604013 2026-02-24T22:10:45.221 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.217+0000 7f7e64854980 1 > prepared multipart upload key=3900SFJK1YQU upload=U9VI9Q5NPA9T tag=A3ALIA03O2TK type=write size=1652589 2026-02-24T22:10:45.222 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.221+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~NX5XAEW2LUVM.0 size=475395 2026-02-24T22:10:45.223 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.221+0000 7f7e64854980 1 < completed multipart upload key=L60HQ7M3E3LL upload=NX5XAEW2LUVM tag=DKEXBR1FNROK type=write size=1523971 2026-02-24T22:10:45.224 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.221+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=GFKMHYJ5P0NW type=del size=3468663 2026-02-24T22:10:45.225 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.225+0000 7f7e64854980 1 < suggested update operation key=PKRHWM69GBJI tag=GTNYWM8Q8J6A type=del size=3431095 2026-02-24T22:10:45.226 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.225+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=C516ONH8X8YF type=del size=3685297 2026-02-24T22:10:45.227 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.225+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=JAJ0HBZ345NS type=del 2026-02-24T22:10:45.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.225+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=Q4V60IGQH7DI type=write size=442820 2026-02-24T22:10:45.229 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.225+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~D2TVA5OESV3Q.0 size=1048576 2026-02-24T22:10:45.229 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.229+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~D2TVA5OESV3Q.1 size=1048576 2026-02-24T22:10:45.230 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.229+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~D2TVA5OESV3Q.2 size=1185 2026-02-24T22:10:45.231 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.229+0000 7f7e64854980 1 < completed multipart upload key=ZESKGBWZXM6O upload=D2TVA5OESV3Q tag=96FJ12EVP2MD type=write size=3146913 2026-02-24T22:10:45.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.233+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~STOUIZ3EHV4O.0 size=1048576 2026-02-24T22:10:45.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.233+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~STOUIZ3EHV4O.1 size=698613 2026-02-24T22:10:45.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.233+0000 7f7e64854980 1 > prepared multipart upload key=4Y3974PYYT78 upload=STOUIZ3EHV4O tag=L5USQE1NEOSX type=write size=2795765 2026-02-24T22:10:45.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.233+0000 7f7e64854980 1 < canceled part key=_multipart_R2893NES3WSF.2~2GBB4ZZ457DU.0 size=1048576 2026-02-24T22:10:45.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.233+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~2GBB4ZZ457DU.1 size=385132 2026-02-24T22:10:45.236 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.233+0000 7f7e64854980 1 < canceled multipart upload key=R2893NES3WSF upload=2GBB4ZZ457DU tag=9G1VZ8Z5K9FW type=write size=2482284 2026-02-24T22:10:45.237 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.233+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=ZXAIAT430PVT type=del size=4124671 2026-02-24T22:10:45.238 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.237+0000 7f7e64854980 1 < completed part key=_multipart_UQLUT9ZAI61V.2~1SFV0WI11J8D.0 size=1048576 2026-02-24T22:10:45.239 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.237+0000 7f7e64854980 1 < completed part key=_multipart_UQLUT9ZAI61V.2~1SFV0WI11J8D.1 size=813336 2026-02-24T22:10:45.240 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.237+0000 7f7e64854980 1 < completed multipart upload key=UQLUT9ZAI61V upload=1SFV0WI11J8D tag=9Y1WBQDCNLTO type=write size=2910488 2026-02-24T22:10:45.241 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.237+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=TO3TP82L0E6R type=write size=307542 2026-02-24T22:10:45.242 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.241+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~5RW9IHLBNVTE.0 size=426178 2026-02-24T22:10:45.243 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.241+0000 7f7e64854980 1 < completed multipart upload key=Y8GMOUA977H3 upload=5RW9IHLBNVTE tag=K13Z9RQJ9NZB type=write size=1474754 2026-02-24T22:10:45.244 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.241+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~GDXBH7VRLKY6.0 size=1048576 2026-02-24T22:10:45.245 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.241+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~GDXBH7VRLKY6.1 size=1048576 2026-02-24T22:10:45.246 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.245+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~GDXBH7VRLKY6.2 size=1041212 2026-02-24T22:10:45.246 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.245+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=GDXBH7VRLKY6 tag=T3DDZ1GA9PB8 type=write size=4186940 2026-02-24T22:10:45.246 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.245+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~VOUKXB79FBFE.0 size=210848 2026-02-24T22:10:45.248 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.245+0000 7f7e64854980 1 < completed multipart upload key=N3ZUNSI5K8HS upload=VOUKXB79FBFE tag=Z9BBUWNV3W97 type=write size=1259424 2026-02-24T22:10:45.249 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.245+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=VILPI53ADGDT type=del size=555753 2026-02-24T22:10:45.249 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.249+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~M0XFVNU6FROB.0 size=1048576 2026-02-24T22:10:45.250 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.249+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~M0XFVNU6FROB.1 size=90663 2026-02-24T22:10:45.251 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.249+0000 7f7e64854980 1 < completed multipart upload key=ZESKGBWZXM6O upload=M0XFVNU6FROB tag=AB96SCX6FN7G type=write size=2187815 2026-02-24T22:10:45.252 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.249+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=8XZDMXG7J77E type=write size=520726 2026-02-24T22:10:45.253 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.253+0000 7f7e64854980 1 < completed delete operation key=78GUTC5THI2P tag=ZLO21PG7K60C type=del 2026-02-24T22:10:45.254 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.253+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=Q4Q83KULU3CX type=del size=3496798 2026-02-24T22:10:45.255 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.253+0000 7f7e64854980 1 < completed delete operation key=D0MKGNRDW6L9 tag=2ZOHYCBW3MQC type=del 2026-02-24T22:10:45.256 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.253+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~AUSKKV8VGUHA.0 size=1048576 2026-02-24T22:10:45.257 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.253+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~AUSKKV8VGUHA.1 size=353310 2026-02-24T22:10:45.257 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.253+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=AUSKKV8VGUHA tag=VPBEKQ6ESZH5 type=write size=2450462 2026-02-24T22:10:45.258 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.257+0000 7f7e64854980 1 < canceled operation key=Q7BP8FXTARDW tag=3KIRX5GN1HUI type=del size=3342130 2026-02-24T22:10:45.259 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.257+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=QT3ZUNVNI9CR type=del size=2125326 2026-02-24T22:10:45.259 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.257+0000 7f7e64854980 1 < completed delete operation key=N3ZUNSI5K8HS tag=DX7B1DGMT1S3 type=del 2026-02-24T22:10:45.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.257+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~PMS64A7DGEFY.0 size=123162 2026-02-24T22:10:45.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.257+0000 7f7e64854980 1 > prepared multipart upload key=Y3L3CJ51C1WR upload=PMS64A7DGEFY tag=BLR6BY3QHB1Y type=write size=1171738 2026-02-24T22:10:45.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.261+0000 7f7e64854980 1 < canceled operation key=D0MKGNRDW6L9 tag=J661L2CGHXO7 type=del size=844865 2026-02-24T22:10:45.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.261+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=5T2UM4FIEJMB type=del size=4193026 2026-02-24T22:10:45.263 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.261+0000 7f7e64854980 1 < canceled operation key=D0MKGNRDW6L9 tag=XOTZZV9DBS7M type=write size=48936 2026-02-24T22:10:45.265 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.261+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~F43IATRG0WZP.0 size=1048576 2026-02-24T22:10:45.265 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.265+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~F43IATRG0WZP.1 size=190932 2026-02-24T22:10:45.265 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.265+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=F43IATRG0WZP tag=HH0KOYT2QGZZ type=write size=2288084 2026-02-24T22:10:45.266 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.265+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~U9VI9Q5NPA9T.0 size=604013 2026-02-24T22:10:45.267 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.265+0000 7f7e64854980 1 < completed multipart upload key=3900SFJK1YQU upload=U9VI9Q5NPA9T tag=A3ALIA03O2TK type=write size=1652589 2026-02-24T22:10:45.269 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.265+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~LMCTP1TAR2LJ.0 size=930711 2026-02-24T22:10:45.269 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.265+0000 7f7e64854980 1 > prepared multipart upload key=L60HQ7M3E3LL upload=LMCTP1TAR2LJ tag=45H2QUTGFQIB type=write size=1979287 2026-02-24T22:10:45.270 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.269+0000 7f7e64854980 1 < completed delete operation key=78GUTC5THI2P tag=GFKMHYJ5P0NW type=del 2026-02-24T22:10:45.271 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.269+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~ILMG411Z1MMB.0 size=1048576 2026-02-24T22:10:45.272 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.269+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~ILMG411Z1MMB.1 size=150495 2026-02-24T22:10:45.272 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.269+0000 7f7e64854980 1 > prepared multipart upload key=Y3L3CJ51C1WR upload=ILMG411Z1MMB tag=17A7L1S5ENBU type=write size=2247647 2026-02-24T22:10:45.272 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.269+0000 7f7e64854980 1 < completed delete operation key=Z0OXREAUJVA8 tag=C516ONH8X8YF type=del 2026-02-24T22:10:45.273 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.273+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=UW443KHXPUC3 type=del size=902457 2026-02-24T22:10:45.275 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.273+0000 7f7e64854980 1 < suggested update operation key=3900SFJK1YQU tag=Q4V60IGQH7DI type=write size=442820 2026-02-24T22:10:45.276 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.273+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~5ISR1K0ZTXI9.0 size=1048576 2026-02-24T22:10:45.277 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.273+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~5ISR1K0ZTXI9.1 size=118699 2026-02-24T22:10:45.277 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.273+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=5ISR1K0ZTXI9 tag=OXM44ME3H1FG type=write size=2215851 2026-02-24T22:10:45.278 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.277+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~STOUIZ3EHV4O.0 size=1048576 2026-02-24T22:10:45.279 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.277+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~STOUIZ3EHV4O.1 size=698613 2026-02-24T22:10:45.280 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.277+0000 7f7e64854980 1 < completed multipart upload key=4Y3974PYYT78 upload=STOUIZ3EHV4O tag=L5USQE1NEOSX type=write size=2795765 2026-02-24T22:10:45.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.277+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=521Y09R2AVGM type=del size=1267129 2026-02-24T22:10:45.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.281+0000 7f7e64854980 1 < completed delete operation key=NL60Q5QOSP8E tag=ZXAIAT430PVT type=del 2026-02-24T22:10:45.283 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.281+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~YOLH5PFMGLNR.0 size=1048576 2026-02-24T22:10:45.284 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.281+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~YOLH5PFMGLNR.1 size=751662 2026-02-24T22:10:45.284 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.281+0000 7f7e64854980 1 > prepared multipart upload key=Z0OXREAUJVA8 upload=YOLH5PFMGLNR tag=IG1VLL2VBC0I type=write size=2848814 2026-02-24T22:10:45.285 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.281+0000 7f7e64854980 1 < suggested update operation key=A6PH77Q78VEN tag=TO3TP82L0E6R type=write size=307542 2026-02-24T22:10:45.286 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.285+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=FQ1N9G1KS7RY type=del size=3694828 2026-02-24T22:10:45.287 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.285+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~GDXBH7VRLKY6.0 size=1048576 2026-02-24T22:10:45.287 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.285+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~GDXBH7VRLKY6.1 size=1048576 2026-02-24T22:10:45.288 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.285+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~GDXBH7VRLKY6.2 size=1041212 2026-02-24T22:10:45.289 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.289+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=GDXBH7VRLKY6 tag=T3DDZ1GA9PB8 type=write size=4186940 2026-02-24T22:10:45.290 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.289+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=HRHYQ0AIVBF2 type=del size=1348640 2026-02-24T22:10:45.291 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.289+0000 7f7e64854980 1 < completed delete operation key=PKRHWM69GBJI tag=VILPI53ADGDT type=del 2026-02-24T22:10:45.292 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.289+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=CWH8YC949O0U type=write size=11365 2026-02-24T22:10:45.293 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.289+0000 7f7e64854980 1 < completed write operation key=Z0OXREAUJVA8 tag=8XZDMXG7J77E type=write size=520726 2026-02-24T22:10:45.294 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.293+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~GXRCP0MKBFHZ.0 size=1048576 2026-02-24T22:10:45.295 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.293+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~GXRCP0MKBFHZ.1 size=1048576 2026-02-24T22:10:45.295 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.293+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~GXRCP0MKBFHZ.2 size=839094 2026-02-24T22:10:45.295 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.293+0000 7f7e64854980 1 > prepared multipart upload key=N3ZUNSI5K8HS upload=GXRCP0MKBFHZ tag=GBKOSRPVETP6 type=write size=3984822 2026-02-24T22:10:45.296 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.293+0000 7f7e64854980 1 < completed delete operation key=NL60Q5QOSP8E tag=Q4Q83KULU3CX type=del 2026-02-24T22:10:45.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.297+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~HYUOAGPUXSNV.0 size=1048576 2026-02-24T22:10:45.298 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.297+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~HYUOAGPUXSNV.1 size=1048576 2026-02-24T22:10:45.299 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.297+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~HYUOAGPUXSNV.2 size=871961 2026-02-24T22:10:45.299 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.297+0000 7f7e64854980 1 > prepared multipart upload key=36MKR2OZN7K9 upload=HYUOAGPUXSNV tag=SY51QRQNIMTI type=write size=4017689 2026-02-24T22:10:45.300 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.297+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~AUSKKV8VGUHA.0 size=1048576 2026-02-24T22:10:45.300 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.297+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~AUSKKV8VGUHA.1 size=353310 2026-02-24T22:10:45.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.301+0000 7f7e64854980 1 < completed multipart upload key=HOHKR92SN25M upload=AUSKKV8VGUHA tag=VPBEKQ6ESZH5 type=write size=2450462 2026-02-24T22:10:45.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.301+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=9JPIYC0850CO type=del size=3082702 2026-02-24T22:10:45.303 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.301+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=QT3ZUNVNI9CR type=del 2026-02-24T22:10:45.304 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.301+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=HAUXD8YHA7CK type=del size=3489521 2026-02-24T22:10:45.305 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.301+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~PMS64A7DGEFY.0 size=123162 2026-02-24T22:10:45.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.305+0000 7f7e64854980 1 < completed multipart upload key=Y3L3CJ51C1WR upload=PMS64A7DGEFY tag=BLR6BY3QHB1Y type=write size=1171738 2026-02-24T22:10:45.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.305+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~D2S1YPT5ZUUK.0 size=407664 2026-02-24T22:10:45.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.305+0000 7f7e64854980 1 > prepared multipart upload key=PKRHWM69GBJI upload=D2S1YPT5ZUUK tag=3YHYFFMMJRVI type=write size=1456240 2026-02-24T22:10:45.309 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.305+0000 7f7e64854980 1 < completed delete operation key=Z0OXREAUJVA8 tag=5T2UM4FIEJMB type=del 2026-02-24T22:10:45.310 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.309+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~FTY2C29K4E3W.0 size=812652 2026-02-24T22:10:45.311 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.309+0000 7f7e64854980 1 > prepared multipart upload key=GH7723GNI2WM upload=FTY2C29K4E3W tag=X7YK593O9GG2 type=write size=1861228 2026-02-24T22:10:45.311 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.309+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~F43IATRG0WZP.0 size=1048576 2026-02-24T22:10:45.312 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.309+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~F43IATRG0WZP.1 size=190932 2026-02-24T22:10:45.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.309+0000 7f7e64854980 1 < completed multipart upload key=ZESKGBWZXM6O upload=F43IATRG0WZP tag=HH0KOYT2QGZZ type=write size=2288084 2026-02-24T22:10:45.315 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.313+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~KF3MHZRE1JWM.0 size=1048576 2026-02-24T22:10:45.315 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.313+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~KF3MHZRE1JWM.1 size=1048576 2026-02-24T22:10:45.316 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.313+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~KF3MHZRE1JWM.2 size=70419 2026-02-24T22:10:45.316 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.313+0000 7f7e64854980 1 > prepared multipart upload key=TF5B43G7KCYT upload=KF3MHZRE1JWM tag=BY62PVPYPKO2 type=write size=3216147 2026-02-24T22:10:45.317 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.313+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~LMCTP1TAR2LJ.0 size=930711 2026-02-24T22:10:45.318 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.317+0000 7f7e64854980 1 < completed multipart upload key=L60HQ7M3E3LL upload=LMCTP1TAR2LJ tag=45H2QUTGFQIB type=write size=1979287 2026-02-24T22:10:45.319 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.317+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=WZQF3VSB1RM7 type=del size=3112530 2026-02-24T22:10:45.319 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.317+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~ILMG411Z1MMB.0 size=1048576 2026-02-24T22:10:45.320 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.317+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~ILMG411Z1MMB.1 size=150495 2026-02-24T22:10:45.321 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.321+0000 7f7e64854980 1 < completed multipart upload key=Y3L3CJ51C1WR upload=ILMG411Z1MMB tag=17A7L1S5ENBU type=write size=2247647 2026-02-24T22:10:45.323 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.321+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~D8BD0HGQB5KB.0 size=1048576 2026-02-24T22:10:45.324 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.321+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~D8BD0HGQB5KB.1 size=918376 2026-02-24T22:10:45.324 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.321+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=D8BD0HGQB5KB tag=9243B0AOMKGA type=write size=3015528 2026-02-24T22:10:45.324 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.321+0000 7f7e64854980 1 < completed delete operation key=78GUTC5THI2P tag=UW443KHXPUC3 type=del 2026-02-24T22:10:45.325 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.321+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=M5BFCWSAQZV2 type=del size=996019 2026-02-24T22:10:45.326 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.325+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~5ISR1K0ZTXI9.0 size=1048576 2026-02-24T22:10:45.327 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.325+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~5ISR1K0ZTXI9.1 size=118699 2026-02-24T22:10:45.328 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.325+0000 7f7e64854980 1 < completed multipart upload key=0AXS7DY23UZU upload=5ISR1K0ZTXI9 tag=OXM44ME3H1FG type=write size=2215851 2026-02-24T22:10:45.329 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.325+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~A2MVHR6KDVCY.0 size=1048576 2026-02-24T22:10:45.330 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.329+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~A2MVHR6KDVCY.1 size=174454 2026-02-24T22:10:45.330 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.329+0000 7f7e64854980 1 > prepared multipart upload key=TF5B43G7KCYT upload=A2MVHR6KDVCY tag=M36Q5501PE9U type=write size=2271606 2026-02-24T22:10:45.331 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.329+0000 7f7e64854980 1 < completed delete operation key=EJCX06TE5RTZ tag=521Y09R2AVGM type=del 2026-02-24T22:10:45.332 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.329+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~VD0YKETR9WJL.0 size=1048576 2026-02-24T22:10:45.333 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.329+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~VD0YKETR9WJL.1 size=1048576 2026-02-24T22:10:45.333 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.333+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~VD0YKETR9WJL.2 size=762920 2026-02-24T22:10:45.333 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.333+0000 7f7e64854980 1 > prepared multipart upload key=NL60Q5QOSP8E upload=VD0YKETR9WJL tag=J1HR18FAYM2X type=write size=3908648 2026-02-24T22:10:45.334 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.333+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~YOLH5PFMGLNR.0 size=1048576 2026-02-24T22:10:45.335 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.333+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~YOLH5PFMGLNR.1 size=751662 2026-02-24T22:10:45.336 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.333+0000 7f7e64854980 1 < completed multipart upload key=Z0OXREAUJVA8 upload=YOLH5PFMGLNR tag=IG1VLL2VBC0I type=write size=2848814 2026-02-24T22:10:45.337 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.333+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=ADRYN5IST33P type=del size=1541604 2026-02-24T22:10:45.339 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.337+0000 7f7e64854980 1 < suggested remove operation key=Q7BP8FXTARDW tag=FQ1N9G1KS7RY type=del 2026-02-24T22:10:45.340 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.337+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~97UDHJNDJ9YG.0 size=1048576 2026-02-24T22:10:45.341 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.337+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~97UDHJNDJ9YG.1 size=1048576 2026-02-24T22:10:45.341 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.337+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~97UDHJNDJ9YG.2 size=316187 2026-02-24T22:10:45.341 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.337+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=97UDHJNDJ9YG tag=U4HE0EIJD52D type=write size=3461915 2026-02-24T22:10:45.343 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.341+0000 7f7e64854980 1 < suggested update operation key=D0MKGNRDW6L9 tag=HRHYQ0AIVBF2 type=del size=1348640 2026-02-24T22:10:45.344 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.341+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~RO73EX30O6DD.0 size=1048576 2026-02-24T22:10:45.345 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.341+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~RO73EX30O6DD.1 size=791543 2026-02-24T22:10:45.345 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.341+0000 7f7e64854980 1 > prepared multipart upload key=78GUTC5THI2P upload=RO73EX30O6DD tag=VH4DE7DBJ0T2 type=write size=2888695 2026-02-24T22:10:45.346 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.345+0000 7f7e64854980 1 < completed write operation key=A6PH77Q78VEN tag=CWH8YC949O0U type=write size=11365 2026-02-24T22:10:45.346 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.345+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=S3KTFDZJ2EG5 type=del size=4014288 2026-02-24T22:10:45.347 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.345+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~GXRCP0MKBFHZ.0 size=1048576 2026-02-24T22:10:45.348 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.345+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~GXRCP0MKBFHZ.1 size=1048576 2026-02-24T22:10:45.349 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.345+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~GXRCP0MKBFHZ.2 size=839094 2026-02-24T22:10:45.350 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.349+0000 7f7e64854980 1 < completed multipart upload key=N3ZUNSI5K8HS upload=GXRCP0MKBFHZ tag=GBKOSRPVETP6 type=write size=3984822 2026-02-24T22:10:45.351 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.349+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=4JIV7RH12HRN type=del size=2926139 2026-02-24T22:10:45.352 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.349+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~HYUOAGPUXSNV.0 size=1048576 2026-02-24T22:10:45.352 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.349+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~HYUOAGPUXSNV.1 size=1048576 2026-02-24T22:10:45.353 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.349+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~HYUOAGPUXSNV.2 size=871961 2026-02-24T22:10:45.354 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.353+0000 7f7e64854980 1 < completed multipart upload key=36MKR2OZN7K9 upload=HYUOAGPUXSNV tag=SY51QRQNIMTI type=write size=4017689 2026-02-24T22:10:45.356 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.353+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~N3OEM14RS06E.0 size=1048576 2026-02-24T22:10:45.356 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.353+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~N3OEM14RS06E.1 size=199275 2026-02-24T22:10:45.356 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.353+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=N3OEM14RS06E tag=NYUZL1N082XT type=write size=2296427 2026-02-24T22:10:45.357 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.353+0000 7f7e64854980 1 < canceled operation key=R2893NES3WSF tag=9JPIYC0850CO type=del size=3082702 2026-02-24T22:10:45.358 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.357+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=UNWVCG46BSLH type=del size=3379508 2026-02-24T22:10:45.359 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.357+0000 7f7e64854980 1 < completed delete operation key=78GUTC5THI2P tag=HAUXD8YHA7CK type=del 2026-02-24T22:10:45.360 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.357+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~ZRZ1KHTSHJ0C.0 size=1048576 2026-02-24T22:10:45.361 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.357+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~ZRZ1KHTSHJ0C.1 size=1048576 2026-02-24T22:10:45.361 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.357+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~ZRZ1KHTSHJ0C.2 size=299568 2026-02-24T22:10:45.361 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.357+0000 7f7e64854980 1 > prepared multipart upload key=UQLUT9ZAI61V upload=ZRZ1KHTSHJ0C tag=OF2TXTFIYD3F type=write size=3445296 2026-02-24T22:10:45.362 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.361+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~D2S1YPT5ZUUK.0 size=407664 2026-02-24T22:10:45.363 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.361+0000 7f7e64854980 1 < completed multipart upload key=PKRHWM69GBJI upload=D2S1YPT5ZUUK tag=3YHYFFMMJRVI type=write size=1456240 2026-02-24T22:10:45.365 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.361+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~K3NAS0HZHOFB.0 size=1048576 2026-02-24T22:10:45.365 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.365+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~K3NAS0HZHOFB.1 size=431415 2026-02-24T22:10:45.365 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.365+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=K3NAS0HZHOFB tag=1867M0E7K90C type=write size=2528567 2026-02-24T22:10:45.366 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.365+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~FTY2C29K4E3W.0 size=812652 2026-02-24T22:10:45.367 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.365+0000 7f7e64854980 1 < completed multipart upload key=GH7723GNI2WM upload=FTY2C29K4E3W tag=X7YK593O9GG2 type=write size=1861228 2026-02-24T22:10:45.369 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.365+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~TAI0PLB9ECK9.0 size=1031771 2026-02-24T22:10:45.369 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.365+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=TAI0PLB9ECK9 tag=MJUDA4E0EL9T type=write size=2080347 2026-02-24T22:10:45.370 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.369+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~KF3MHZRE1JWM.0 size=1048576 2026-02-24T22:10:45.370 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.369+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~KF3MHZRE1JWM.1 size=1048576 2026-02-24T22:10:45.371 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.369+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~KF3MHZRE1JWM.2 size=70419 2026-02-24T22:10:45.372 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.369+0000 7f7e64854980 1 < completed multipart upload key=TF5B43G7KCYT upload=KF3MHZRE1JWM tag=BY62PVPYPKO2 type=write size=3216147 2026-02-24T22:10:45.374 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.373+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~2LBY5UBDFUT3.0 size=934894 2026-02-24T22:10:45.374 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.373+0000 7f7e64854980 1 > prepared multipart upload key=Z0OXREAUJVA8 upload=2LBY5UBDFUT3 tag=YOHBFL7FW42X type=write size=1983470 2026-02-24T22:10:45.374 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.373+0000 7f7e64854980 1 < completed delete operation key=Y3L3CJ51C1WR tag=WZQF3VSB1RM7 type=del 2026-02-24T22:10:45.375 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.373+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=ML6BPV13AF1X type=del size=2635789 2026-02-24T22:10:45.376 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.373+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~D8BD0HGQB5KB.0 size=1048576 2026-02-24T22:10:45.377 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.373+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~D8BD0HGQB5KB.1 size=918376 2026-02-24T22:10:45.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.377+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=D8BD0HGQB5KB tag=9243B0AOMKGA type=write size=3015528 2026-02-24T22:10:45.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.377+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~NJI5TQRZBFX3.0 size=1048576 2026-02-24T22:10:45.381 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.377+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~NJI5TQRZBFX3.1 size=680305 2026-02-24T22:10:45.381 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.377+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=NJI5TQRZBFX3 tag=ZKC4T7GCRX1T type=write size=2777457 2026-02-24T22:10:45.381 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.381+0000 7f7e64854980 1 < completed delete operation key=ZESKGBWZXM6O tag=M5BFCWSAQZV2 type=del 2026-02-24T22:10:45.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.381+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~WGD06XYXTF0A.0 size=1048576 2026-02-24T22:10:45.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.381+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~WGD06XYXTF0A.1 size=861287 2026-02-24T22:10:45.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.381+0000 7f7e64854980 1 > prepared multipart upload key=PKRHWM69GBJI upload=WGD06XYXTF0A tag=GV6NPH60RL0U type=write size=2958439 2026-02-24T22:10:45.384 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.381+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~A2MVHR6KDVCY.0 size=1048576 2026-02-24T22:10:45.385 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.381+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~A2MVHR6KDVCY.1 size=174454 2026-02-24T22:10:45.386 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.385+0000 7f7e64854980 1 < completed multipart upload key=TF5B43G7KCYT upload=A2MVHR6KDVCY tag=M36Q5501PE9U type=write size=2271606 2026-02-24T22:10:45.387 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.385+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~VF0A262ENYRM.0 size=1048576 2026-02-24T22:10:45.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.385+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~VF0A262ENYRM.1 size=1048576 2026-02-24T22:10:45.389 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.385+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~VF0A262ENYRM.2 size=625872 2026-02-24T22:10:45.389 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.385+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=VF0A262ENYRM tag=RZ27EANJT4G2 type=write size=3771600 2026-02-24T22:10:45.389 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.389+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~VD0YKETR9WJL.0 size=1048576 2026-02-24T22:10:45.390 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.389+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~VD0YKETR9WJL.1 size=1048576 2026-02-24T22:10:45.391 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.389+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~VD0YKETR9WJL.2 size=762920 2026-02-24T22:10:45.392 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.389+0000 7f7e64854980 1 < completed multipart upload key=NL60Q5QOSP8E upload=VD0YKETR9WJL tag=J1HR18FAYM2X type=write size=3908648 2026-02-24T22:10:45.393 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.389+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=6ZEU2MQ8G9GX type=del size=225789 2026-02-24T22:10:45.393 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.393+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=ADRYN5IST33P type=del 2026-02-24T22:10:45.394 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.393+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=IPCK08QFUIG4 type=del size=1561835 2026-02-24T22:10:45.395 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.393+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~97UDHJNDJ9YG.0 size=1048576 2026-02-24T22:10:45.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.393+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~97UDHJNDJ9YG.1 size=1048576 2026-02-24T22:10:45.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.393+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~97UDHJNDJ9YG.2 size=316187 2026-02-24T22:10:45.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.397+0000 7f7e64854980 1 < completed multipart upload key=D0MKGNRDW6L9 upload=97UDHJNDJ9YG tag=U4HE0EIJD52D type=write size=3461915 2026-02-24T22:10:45.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.397+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=DK2MYWC14RDA type=del size=570572 2026-02-24T22:10:45.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.397+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~RO73EX30O6DD.0 size=1048576 2026-02-24T22:10:45.400 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.397+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~RO73EX30O6DD.1 size=791543 2026-02-24T22:10:45.401 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.397+0000 7f7e64854980 1 < completed multipart upload key=78GUTC5THI2P upload=RO73EX30O6DD tag=VH4DE7DBJ0T2 type=write size=2888695 2026-02-24T22:10:45.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.401+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=DIMMCEV1URCR type=del size=790712 2026-02-24T22:10:45.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.401+0000 7f7e64854980 1 < suggested remove operation key=UQLUT9ZAI61V tag=S3KTFDZJ2EG5 type=del 2026-02-24T22:10:45.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.401+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=YO7CGQO5LU2G type=write size=588170 2026-02-24T22:10:45.406 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.405+0000 7f7e64854980 1 < suggested remove operation key=HOHKR92SN25M tag=4JIV7RH12HRN type=del 2026-02-24T22:10:45.407 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.405+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=J0HWIQ8PT0C1 type=del size=921103 2026-02-24T22:10:45.408 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.405+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~N3OEM14RS06E.0 size=1048576 2026-02-24T22:10:45.408 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.405+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~N3OEM14RS06E.1 size=199275 2026-02-24T22:10:45.410 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.409+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=N3OEM14RS06E tag=NYUZL1N082XT type=write size=2296427 2026-02-24T22:10:45.410 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.409+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=PU766M225UYC type=del size=87241 2026-02-24T22:10:45.411 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.409+0000 7f7e64854980 1 < completed delete operation key=3900SFJK1YQU tag=UNWVCG46BSLH type=del 2026-02-24T22:10:45.412 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.409+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=RSRS7RLTHTUU type=del size=2185037 2026-02-24T22:10:45.413 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.409+0000 7f7e64854980 1 < completed part key=_multipart_UQLUT9ZAI61V.2~ZRZ1KHTSHJ0C.0 size=1048576 2026-02-24T22:10:45.413 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.409+0000 7f7e64854980 1 < completed part key=_multipart_UQLUT9ZAI61V.2~ZRZ1KHTSHJ0C.1 size=1048576 2026-02-24T22:10:45.414 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.413+0000 7f7e64854980 1 < completed part key=_multipart_UQLUT9ZAI61V.2~ZRZ1KHTSHJ0C.2 size=299568 2026-02-24T22:10:45.415 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.413+0000 7f7e64854980 1 < completed multipart upload key=UQLUT9ZAI61V upload=ZRZ1KHTSHJ0C tag=OF2TXTFIYD3F type=write size=3445296 2026-02-24T22:10:45.416 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.413+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=VK8VRW5J3WL2 type=del size=202004 2026-02-24T22:10:45.417 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.413+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~K3NAS0HZHOFB.0 size=1048576 2026-02-24T22:10:45.417 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.413+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~K3NAS0HZHOFB.1 size=431415 2026-02-24T22:10:45.418 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.417+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=K3NAS0HZHOFB tag=1867M0E7K90C type=write size=2528567 2026-02-24T22:10:45.419 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.417+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=1UMAB27R0KVW type=del size=4142253 2026-02-24T22:10:45.420 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.417+0000 7f7e64854980 1 < canceled part key=_multipart_R2893NES3WSF.2~TAI0PLB9ECK9.0 size=1031771 2026-02-24T22:10:45.421 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.417+0000 7f7e64854980 1 < canceled multipart upload key=R2893NES3WSF upload=TAI0PLB9ECK9 tag=MJUDA4E0EL9T type=write size=2080347 2026-02-24T22:10:45.422 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.421+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~0J35JM7823YV.0 size=1048576 2026-02-24T22:10:45.422 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.421+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~0J35JM7823YV.1 size=1048576 2026-02-24T22:10:45.423 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.421+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~0J35JM7823YV.2 size=931028 2026-02-24T22:10:45.423 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.421+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=0J35JM7823YV tag=198PHTAAMBDU type=write size=4076756 2026-02-24T22:10:45.423 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.421+0000 7f7e64854980 1 < canceled part key=_multipart_Z0OXREAUJVA8.2~2LBY5UBDFUT3.0 size=934894 2026-02-24T22:10:45.424 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.421+0000 7f7e64854980 1 < canceled multipart upload key=Z0OXREAUJVA8 upload=2LBY5UBDFUT3 tag=YOHBFL7FW42X type=write size=1983470 2026-02-24T22:10:45.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.425+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~5U3W1WS6TMKE.0 size=1048576 2026-02-24T22:10:45.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.425+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~5U3W1WS6TMKE.1 size=1048576 2026-02-24T22:10:45.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.425+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~5U3W1WS6TMKE.2 size=672729 2026-02-24T22:10:45.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.425+0000 7f7e64854980 1 > prepared multipart upload key=A6PH77Q78VEN upload=5U3W1WS6TMKE tag=O4T2P2CN2U68 type=write size=3818457 2026-02-24T22:10:45.428 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.425+0000 7f7e64854980 1 < canceled operation key=L9NEDH44BIGA tag=ML6BPV13AF1X type=del size=2635789 2026-02-24T22:10:45.428 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.425+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=QRAY05DD2PV0 type=del size=3097511 2026-02-24T22:10:45.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.425+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~NJI5TQRZBFX3.0 size=1048576 2026-02-24T22:10:45.430 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.429+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~NJI5TQRZBFX3.1 size=680305 2026-02-24T22:10:45.431 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.429+0000 7f7e64854980 1 < completed multipart upload key=TXRUQQ456DD8 upload=NJI5TQRZBFX3 tag=ZKC4T7GCRX1T type=write size=2777457 2026-02-24T22:10:45.432 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.429+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=R0MYABTPDVKA type=del size=2251499 2026-02-24T22:10:45.432 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.429+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~WGD06XYXTF0A.0 size=1048576 2026-02-24T22:10:45.433 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.429+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~WGD06XYXTF0A.1 size=861287 2026-02-24T22:10:45.434 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.433+0000 7f7e64854980 1 < completed multipart upload key=PKRHWM69GBJI upload=WGD06XYXTF0A tag=GV6NPH60RL0U type=write size=2958439 2026-02-24T22:10:45.435 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.433+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=LRQAOQS030C6 type=del size=2875497 2026-02-24T22:10:45.436 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.433+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~VF0A262ENYRM.0 size=1048576 2026-02-24T22:10:45.437 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.433+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~VF0A262ENYRM.1 size=1048576 2026-02-24T22:10:45.437 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.433+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~VF0A262ENYRM.2 size=625872 2026-02-24T22:10:45.438 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.437+0000 7f7e64854980 1 < completed multipart upload key=ZESKGBWZXM6O upload=VF0A262ENYRM tag=RZ27EANJT4G2 type=write size=3771600 2026-02-24T22:10:45.439 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.437+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=0B9PJ5Z1TSQQ type=write size=304513 2026-02-24T22:10:45.440 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.437+0000 7f7e64854980 1 < completed delete operation key=EB04ETNYM087 tag=6ZEU2MQ8G9GX type=del 2026-02-24T22:10:45.441 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.437+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=E9K5KJO39R07 type=write size=626599 2026-02-24T22:10:45.441 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.441+0000 7f7e64854980 1 < completed delete operation key=Y3L3CJ51C1WR tag=IPCK08QFUIG4 type=del 2026-02-24T22:10:45.443 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.441+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~VAPTBE72X5WY.0 size=1048576 2026-02-24T22:10:45.444 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.441+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~VAPTBE72X5WY.1 size=1038678 2026-02-24T22:10:45.444 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.441+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=VAPTBE72X5WY tag=ZB8NNU36ZZ02 type=write size=3135830 2026-02-24T22:10:45.444 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.441+0000 7f7e64854980 1 < canceled operation key=Y3L3CJ51C1WR tag=DK2MYWC14RDA type=del size=570572 2026-02-24T22:10:45.445 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.441+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=TLWML9LAH7WB type=del size=681460 2026-02-24T22:10:45.446 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.445+0000 7f7e64854980 1 < completed delete operation key=GH7723GNI2WM tag=DIMMCEV1URCR type=del 2026-02-24T22:10:45.447 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.445+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=TLMI8EL918FA type=write size=676471 2026-02-24T22:10:45.448 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.445+0000 7f7e64854980 1 < completed write operation key=Z0OXREAUJVA8 tag=YO7CGQO5LU2G type=write size=588170 2026-02-24T22:10:45.449 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.449+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~EAW6DA4EH9ML.0 size=235268 2026-02-24T22:10:45.450 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.449+0000 7f7e64854980 1 > prepared multipart upload key=NL60Q5QOSP8E upload=EAW6DA4EH9ML tag=Y7XDCBMCHCV9 type=write size=1283844 2026-02-24T22:10:45.450 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.449+0000 7f7e64854980 1 < completed delete operation key=SGQYQ2TIVJPS tag=J0HWIQ8PT0C1 type=del 2026-02-24T22:10:45.451 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.449+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~LLB4EMCQ2FWH.0 size=1048576 2026-02-24T22:10:45.452 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.449+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~LLB4EMCQ2FWH.1 size=892008 2026-02-24T22:10:45.452 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.449+0000 7f7e64854980 1 > prepared multipart upload key=4Y3974PYYT78 upload=LLB4EMCQ2FWH tag=0D2TY4S1LWT6 type=write size=2989160 2026-02-24T22:10:45.453 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.449+0000 7f7e64854980 1 < completed delete operation key=LSRPN33Z2B9N tag=PU766M225UYC type=del 2026-02-24T22:10:45.454 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.453+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=CFIIIZXD0ULQ type=write size=939459 2026-02-24T22:10:45.455 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.453+0000 7f7e64854980 1 < suggested update operation key=A6PH77Q78VEN tag=RSRS7RLTHTUU type=del size=2185037 2026-02-24T22:10:45.456 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.453+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=C88TKZNI9TI7 type=write size=811649 2026-02-24T22:10:45.456 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.453+0000 7f7e64854980 1 < canceled operation key=CI8DA17DX8LU tag=VK8VRW5J3WL2 type=del size=202004 2026-02-24T22:10:45.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.453+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=29WX9N5BK8M7 type=del size=3340835 2026-02-24T22:10:45.458 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.457+0000 7f7e64854980 -1 < failed to complete operation key=A6PH77Q78VEN tag=1UMAB27R0KVW type=del size=4142253: (22) Invalid argument 2026-02-24T22:10:45.459 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.457+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~E4FP234B4323.0 size=1048576 2026-02-24T22:10:45.460 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.457+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~E4FP234B4323.1 size=955588 2026-02-24T22:10:45.460 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.457+0000 7f7e64854980 1 > prepared multipart upload key=Y3L3CJ51C1WR upload=E4FP234B4323 tag=L9MUQODFGE14 type=write size=3052740 2026-02-24T22:10:45.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.457+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~0J35JM7823YV.0 size=1048576 2026-02-24T22:10:45.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.457+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~0J35JM7823YV.1 size=1048576 2026-02-24T22:10:45.462 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.461+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~0J35JM7823YV.2 size=931028 2026-02-24T22:10:45.463 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.461+0000 7f7e64854980 1 < completed multipart upload key=EJCX06TE5RTZ upload=0J35JM7823YV tag=198PHTAAMBDU type=write size=4076756 2026-02-24T22:10:45.464 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.461+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~4DCU9OE9VXN6.0 size=183420 2026-02-24T22:10:45.464 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.461+0000 7f7e64854980 1 > prepared multipart upload key=ON16L53JUSTW upload=4DCU9OE9VXN6 tag=PURR2QXOQSGY type=write size=1231996 2026-02-24T22:10:45.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.461+0000 7f7e64854980 1 < canceled part key=_multipart_A6PH77Q78VEN.2~5U3W1WS6TMKE.0 size=1048576 2026-02-24T22:10:45.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.461+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~5U3W1WS6TMKE.1 size=1048576 2026-02-24T22:10:45.466 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.465+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~5U3W1WS6TMKE.2 size=672729 2026-02-24T22:10:45.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.465+0000 7f7e64854980 1 < canceled multipart upload key=A6PH77Q78VEN upload=5U3W1WS6TMKE tag=O4T2P2CN2U68 type=write size=3818457 2026-02-24T22:10:45.468 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.465+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~Q01MVTO8H3NS.0 size=1015425 2026-02-24T22:10:45.468 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.465+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=Q01MVTO8H3NS tag=9UR453Z03WLP type=write size=2064001 2026-02-24T22:10:45.469 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.465+0000 7f7e64854980 1 < completed delete operation key=4Y3974PYYT78 tag=QRAY05DD2PV0 type=del 2026-02-24T22:10:45.470 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.469+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~11JIXYWR45JH.0 size=198503 2026-02-24T22:10:45.470 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.469+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=11JIXYWR45JH tag=DI0C7SQXNC4J type=write size=1247079 2026-02-24T22:10:45.471 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.469+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=R0MYABTPDVKA type=del 2026-02-24T22:10:45.472 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.469+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=H7H470DNNWK9 type=write size=1036025 2026-02-24T22:10:45.472 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.469+0000 7f7e64854980 1 < completed delete operation key=SGQYQ2TIVJPS tag=LRQAOQS030C6 type=del 2026-02-24T22:10:45.474 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.473+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~MODSNZW7IRBQ.0 size=1048576 2026-02-24T22:10:45.474 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.473+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~MODSNZW7IRBQ.1 size=836746 2026-02-24T22:10:45.475 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.473+0000 7f7e64854980 1 > prepared multipart upload key=ON16L53JUSTW upload=MODSNZW7IRBQ tag=1AQM3854OB9Z type=write size=2933898 2026-02-24T22:10:45.475 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.473+0000 7f7e64854980 1 < completed write operation key=EB04ETNYM087 tag=0B9PJ5Z1TSQQ type=write size=304513 2026-02-24T22:10:45.476 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.473+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=O3YWCI360O7V type=del size=2552095 2026-02-24T22:10:45.477 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.473+0000 7f7e64854980 1 < completed write operation key=PKRHWM69GBJI tag=E9K5KJO39R07 type=write size=626599 2026-02-24T22:10:45.477 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.477+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=E9428N8F8SED type=del size=1770778 2026-02-24T22:10:45.478 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.477+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~VAPTBE72X5WY.0 size=1048576 2026-02-24T22:10:45.478 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.477+0000 7f7e64854980 1 < canceled part key=_multipart_CI8DA17DX8LU.2~VAPTBE72X5WY.1 size=1038678 2026-02-24T22:10:45.480 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.477+0000 7f7e64854980 1 < canceled multipart upload key=CI8DA17DX8LU upload=VAPTBE72X5WY tag=ZB8NNU36ZZ02 type=write size=3135830 2026-02-24T22:10:45.481 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.477+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~U8Z2BWHEC47N.0 size=1048576 2026-02-24T22:10:45.481 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.481+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~U8Z2BWHEC47N.1 size=1048576 2026-02-24T22:10:45.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.481+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~U8Z2BWHEC47N.2 size=454976 2026-02-24T22:10:45.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.481+0000 7f7e64854980 1 > prepared multipart upload key=HKOQRSF6XEXB upload=U8Z2BWHEC47N tag=W1JK2IARMRUI type=write size=3600704 2026-02-24T22:10:45.484 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.481+0000 7f7e64854980 1 < suggested remove operation key=CI8DA17DX8LU tag=TLWML9LAH7WB type=del 2026-02-24T22:10:45.484 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.481+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=4NM7YPSYDQGB type=del size=925088 2026-02-24T22:10:45.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.481+0000 7f7e64854980 1 < completed write operation key=TF5B43G7KCYT tag=TLMI8EL918FA type=write size=676471 2026-02-24T22:10:45.486 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.485+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~6CL7R0HIA9OD.0 size=1048576 2026-02-24T22:10:45.487 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.485+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~6CL7R0HIA9OD.1 size=276703 2026-02-24T22:10:45.487 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.485+0000 7f7e64854980 1 > prepared multipart upload key=Y8GMOUA977H3 upload=6CL7R0HIA9OD tag=ASJNWWSWY9AM type=write size=2373855 2026-02-24T22:10:45.488 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.485+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~EAW6DA4EH9ML.0 size=235268 2026-02-24T22:10:45.489 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.485+0000 7f7e64854980 1 < completed multipart upload key=NL60Q5QOSP8E upload=EAW6DA4EH9ML tag=Y7XDCBMCHCV9 type=write size=1283844 2026-02-24T22:10:45.490 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.489+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=OZ0XXIORN8N8 type=del size=3527072 2026-02-24T22:10:45.490 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.489+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~LLB4EMCQ2FWH.0 size=1048576 2026-02-24T22:10:45.491 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.489+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~LLB4EMCQ2FWH.1 size=892008 2026-02-24T22:10:45.492 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.489+0000 7f7e64854980 1 < completed multipart upload key=4Y3974PYYT78 upload=LLB4EMCQ2FWH tag=0D2TY4S1LWT6 type=write size=2989160 2026-02-24T22:10:45.494 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.493+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~6GWX49V3UMP1.0 size=6442 2026-02-24T22:10:45.494 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.493+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=6GWX49V3UMP1 tag=LGQTRSQJCWO7 type=write size=1055018 2026-02-24T22:10:45.494 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.493+0000 7f7e64854980 1 < completed write operation key=ZESKGBWZXM6O tag=CFIIIZXD0ULQ type=write size=939459 2026-02-24T22:10:45.496 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.493+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~JRY79YRUIF93.0 size=1048576 2026-02-24T22:10:45.496 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.493+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~JRY79YRUIF93.1 size=1048576 2026-02-24T22:10:45.497 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.493+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~JRY79YRUIF93.2 size=660564 2026-02-24T22:10:45.497 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.493+0000 7f7e64854980 1 > prepared multipart upload key=Y8GMOUA977H3 upload=JRY79YRUIF93 tag=7MMZJ77GKRO9 type=write size=3806292 2026-02-24T22:10:45.498 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.497+0000 7f7e64854980 1 < completed write operation key=Q7BP8FXTARDW tag=C88TKZNI9TI7 type=write size=811649 2026-02-24T22:10:45.498 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.497+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=DA50XV8TDZKB type=del size=2929839 2026-02-24T22:10:45.500 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.497+0000 7f7e64854980 1 < suggested remove operation key=2KKQKHXO7K3K tag=29WX9N5BK8M7 type=del 2026-02-24T22:10:45.501 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.497+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=040CUCG4RUP3 type=write size=67168 2026-02-24T22:10:45.501 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.501+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~E4FP234B4323.0 size=1048576 2026-02-24T22:10:45.502 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.501+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~E4FP234B4323.1 size=955588 2026-02-24T22:10:45.503 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.501+0000 7f7e64854980 1 < completed multipart upload key=Y3L3CJ51C1WR upload=E4FP234B4323 tag=L9MUQODFGE14 type=write size=3052740 2026-02-24T22:10:45.505 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.501+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~CGLW23UEQK1J.0 size=501798 2026-02-24T22:10:45.505 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.501+0000 7f7e64854980 1 > prepared multipart upload key=HKOQRSF6XEXB upload=CGLW23UEQK1J tag=3GS3I4CX4VYN type=write size=1550374 2026-02-24T22:10:45.505 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.505+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~4DCU9OE9VXN6.0 size=183420 2026-02-24T22:10:45.507 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.505+0000 7f7e64854980 1 < completed multipart upload key=ON16L53JUSTW upload=4DCU9OE9VXN6 tag=PURR2QXOQSGY type=write size=1231996 2026-02-24T22:10:45.508 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.505+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=0PEGU0XVTF30 type=del size=2048414 2026-02-24T22:10:45.508 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.505+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~Q01MVTO8H3NS.0 size=1015425 2026-02-24T22:10:45.510 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.509+0000 7f7e64854980 1 < completed multipart upload key=ZESKGBWZXM6O upload=Q01MVTO8H3NS tag=9UR453Z03WLP type=write size=2064001 2026-02-24T22:10:45.510 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.509+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=B7DUUDZWPKYV type=del size=3592258 2026-02-24T22:10:45.511 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.509+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~11JIXYWR45JH.0 size=198503 2026-02-24T22:10:45.512 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.509+0000 7f7e64854980 1 < completed multipart upload key=EJCX06TE5RTZ upload=11JIXYWR45JH tag=DI0C7SQXNC4J type=write size=1247079 2026-02-24T22:10:45.513 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.513+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=EOAC293USC2S type=write size=997717 2026-02-24T22:10:45.515 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.513+0000 7f7e64854980 1 < suggested update operation key=78GUTC5THI2P tag=H7H470DNNWK9 type=write size=1036025 2026-02-24T22:10:45.515 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.513+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=USQND8JI8YON type=write size=73061 2026-02-24T22:10:45.516 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.513+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~MODSNZW7IRBQ.0 size=1048576 2026-02-24T22:10:45.518 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.517+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~MODSNZW7IRBQ.1 size=836746 2026-02-24T22:10:45.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.517+0000 7f7e64854980 1 < completed multipart upload key=ON16L53JUSTW upload=MODSNZW7IRBQ tag=1AQM3854OB9Z type=write size=2933898 2026-02-24T22:10:45.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.517+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~P92MGSKA7SW3.0 size=966979 2026-02-24T22:10:45.521 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.517+0000 7f7e64854980 1 > prepared multipart upload key=LSRPN33Z2B9N upload=P92MGSKA7SW3 tag=SJGW0PAZZNUR type=write size=2015555 2026-02-24T22:10:45.521 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.521+0000 7f7e64854980 1 < completed delete operation key=L60HQ7M3E3LL tag=O3YWCI360O7V type=del 2026-02-24T22:10:45.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.521+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=IIFPE46C761N type=del size=1452679 2026-02-24T22:10:45.523 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.521+0000 7f7e64854980 1 < completed delete operation key=UQLUT9ZAI61V tag=E9428N8F8SED type=del 2026-02-24T22:10:45.524 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.521+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=IYTWE7JS5KMD type=del size=2114579 2026-02-24T22:10:45.524 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.521+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~U8Z2BWHEC47N.0 size=1048576 2026-02-24T22:10:45.525 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.521+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~U8Z2BWHEC47N.1 size=1048576 2026-02-24T22:10:45.525 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.521+0000 7f7e64854980 1 < canceled part key=_multipart_HKOQRSF6XEXB.2~U8Z2BWHEC47N.2 size=454976 2026-02-24T22:10:45.526 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.525+0000 7f7e64854980 1 < canceled multipart upload key=HKOQRSF6XEXB upload=U8Z2BWHEC47N tag=W1JK2IARMRUI type=write size=3600704 2026-02-24T22:10:45.527 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.525+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=4COYE80CGDZK type=del size=4090675 2026-02-24T22:10:45.527 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.525+0000 7f7e64854980 1 < completed delete operation key=D0MKGNRDW6L9 tag=4NM7YPSYDQGB type=del 2026-02-24T22:10:45.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.525+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=CD0LYQ3F54I1 type=write size=99759 2026-02-24T22:10:45.529 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.525+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~6CL7R0HIA9OD.0 size=1048576 2026-02-24T22:10:45.530 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.529+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~6CL7R0HIA9OD.1 size=276703 2026-02-24T22:10:45.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.529+0000 7f7e64854980 1 < completed multipart upload key=Y8GMOUA977H3 upload=6CL7R0HIA9OD tag=ASJNWWSWY9AM type=write size=2373855 2026-02-24T22:10:45.532 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.529+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~STTLLS0WJWZF.0 size=1048576 2026-02-24T22:10:45.533 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.529+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~STTLLS0WJWZF.1 size=270785 2026-02-24T22:10:45.533 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.529+0000 7f7e64854980 1 > prepared multipart upload key=4Y3974PYYT78 upload=STTLLS0WJWZF tag=TKEGLZD0X866 type=write size=2367937 2026-02-24T22:10:45.533 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.533+0000 7f7e64854980 1 < canceled operation key=R2893NES3WSF tag=OZ0XXIORN8N8 type=del size=3527072 2026-02-24T22:10:45.535 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.533+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~2YCZKXXF81IM.0 size=1048576 2026-02-24T22:10:45.535 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.533+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~2YCZKXXF81IM.1 size=453689 2026-02-24T22:10:45.536 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.533+0000 7f7e64854980 1 > prepared multipart upload key=Z0OXREAUJVA8 upload=2YCZKXXF81IM tag=LHU2SRCSVH5N type=write size=2550841 2026-02-24T22:10:45.536 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.533+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~6GWX49V3UMP1.0 size=6442 2026-02-24T22:10:45.537 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.537+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=6GWX49V3UMP1 tag=LGQTRSQJCWO7 type=write size=1055018 2026-02-24T22:10:45.538 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.537+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=EZQ5I2IG0FGQ type=del size=3677231 2026-02-24T22:10:45.539 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.537+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~JRY79YRUIF93.0 size=1048576 2026-02-24T22:10:45.540 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.537+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~JRY79YRUIF93.1 size=1048576 2026-02-24T22:10:45.540 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.537+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~JRY79YRUIF93.2 size=660564 2026-02-24T22:10:45.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.541+0000 7f7e64854980 1 < completed multipart upload key=Y8GMOUA977H3 upload=JRY79YRUIF93 tag=7MMZJ77GKRO9 type=write size=3806292 2026-02-24T22:10:45.543 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.541+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~6A21BDW97I2U.0 size=1048576 2026-02-24T22:10:45.543 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.541+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~6A21BDW97I2U.1 size=481994 2026-02-24T22:10:45.543 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.541+0000 7f7e64854980 1 > prepared multipart upload key=0B2V0MHWSOC9 upload=6A21BDW97I2U tag=10NL29M7JWTH type=write size=2579146 2026-02-24T22:10:45.544 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.541+0000 7f7e64854980 1 < completed delete operation key=Z0OXREAUJVA8 tag=DA50XV8TDZKB type=del 2026-02-24T22:10:45.545 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.541+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=05ZSL9ZBB5Y0 type=del size=1290663 2026-02-24T22:10:45.546 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.545+0000 7f7e64854980 1 < completed write operation key=Z0OXREAUJVA8 tag=040CUCG4RUP3 type=write size=67168 2026-02-24T22:10:45.547 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.545+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~X0Q3IPTP8MOG.0 size=1048576 2026-02-24T22:10:45.547 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.545+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~X0Q3IPTP8MOG.1 size=770927 2026-02-24T22:10:45.548 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.545+0000 7f7e64854980 1 > prepared multipart upload key=Q7BP8FXTARDW upload=X0Q3IPTP8MOG tag=0B0DK9F11UXK type=write size=2868079 2026-02-24T22:10:45.548 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.545+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~CGLW23UEQK1J.0 size=501798 2026-02-24T22:10:45.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.549+0000 7f7e64854980 1 < completed multipart upload key=HKOQRSF6XEXB upload=CGLW23UEQK1J tag=3GS3I4CX4VYN type=write size=1550374 2026-02-24T22:10:45.550 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.549+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=WZLDIKAJ0BXO type=del size=496409 2026-02-24T22:10:45.552 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.549+0000 7f7e64854980 1 < suggested remove operation key=R2893NES3WSF tag=0PEGU0XVTF30 type=del 2026-02-24T22:10:45.552 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.549+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=KEUBTVSXJ3ZN type=del size=3092077 2026-02-24T22:10:45.553 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.549+0000 7f7e64854980 1 < canceled operation key=EJCX06TE5RTZ tag=B7DUUDZWPKYV type=del size=3592258 2026-02-24T22:10:45.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.553+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~PKK79BU5VD2I.0 size=698155 2026-02-24T22:10:45.555 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.553+0000 7f7e64854980 1 > prepared multipart upload key=Y3L3CJ51C1WR upload=PKK79BU5VD2I tag=4VOCANGFO2WK type=write size=1746731 2026-02-24T22:10:45.555 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.553+0000 7f7e64854980 1 < completed write operation key=ON16L53JUSTW tag=EOAC293USC2S type=write size=997717 2026-02-24T22:10:45.556 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.553+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=FKOICGEZGMP8 type=write size=720580 2026-02-24T22:10:45.557 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.553+0000 7f7e64854980 1 < completed write operation key=TF5B43G7KCYT tag=USQND8JI8YON type=write size=73061 2026-02-24T22:10:45.558 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.557+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~42RJYZDCEU2Z.0 size=339938 2026-02-24T22:10:45.558 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.557+0000 7f7e64854980 1 > prepared multipart upload key=TF5B43G7KCYT upload=42RJYZDCEU2Z tag=GSFAH3N3SF30 type=write size=1388514 2026-02-24T22:10:45.559 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.557+0000 7f7e64854980 1 < canceled part key=_multipart_LSRPN33Z2B9N.2~P92MGSKA7SW3.0 size=966979 2026-02-24T22:10:45.559 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.557+0000 7f7e64854980 1 < canceled multipart upload key=LSRPN33Z2B9N upload=P92MGSKA7SW3 tag=SJGW0PAZZNUR type=write size=2015555 2026-02-24T22:10:45.560 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.557+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=DIN1XFE5DY6N type=del size=1356295 2026-02-24T22:10:45.562 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.561+0000 7f7e64854980 1 < suggested update operation key=Y8GMOUA977H3 tag=IIFPE46C761N type=del size=1452679 2026-02-24T22:10:45.562 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.561+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=TJSW1JW6NOYT type=del size=766536 2026-02-24T22:10:45.563 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.561+0000 7f7e64854980 -1 < failed to complete operation key=R2893NES3WSF tag=IYTWE7JS5KMD type=del size=2114579: (22) Invalid argument 2026-02-24T22:10:45.564 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.561+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=LK1Y9NS2PEKF type=del size=3110020 2026-02-24T22:10:45.565 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.561+0000 7f7e64854980 1 < completed delete operation key=78GUTC5THI2P tag=4COYE80CGDZK type=del 2026-02-24T22:10:45.565 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.565+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=SM631DJZQO8G type=del size=2093084 2026-02-24T22:10:45.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.565+0000 7f7e64854980 1 < completed write operation key=UQLUT9ZAI61V tag=CD0LYQ3F54I1 type=write size=99759 2026-02-24T22:10:45.567 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.565+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=HF84EJLQ14WA type=del size=2684100 2026-02-24T22:10:45.568 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.565+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~STTLLS0WJWZF.0 size=1048576 2026-02-24T22:10:45.568 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.565+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~STTLLS0WJWZF.1 size=270785 2026-02-24T22:10:45.570 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.569+0000 7f7e64854980 1 < completed multipart upload key=4Y3974PYYT78 upload=STTLLS0WJWZF tag=TKEGLZD0X866 type=write size=2367937 2026-02-24T22:10:45.570 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.569+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=Z5J0HO7IMWX7 type=write size=563617 2026-02-24T22:10:45.571 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.569+0000 7f7e64854980 1 < canceled part key=_multipart_Z0OXREAUJVA8.2~2YCZKXXF81IM.0 size=1048576 2026-02-24T22:10:45.571 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.569+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~2YCZKXXF81IM.1 size=453689 2026-02-24T22:10:45.572 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.569+0000 7f7e64854980 1 < canceled multipart upload key=Z0OXREAUJVA8 upload=2YCZKXXF81IM tag=LHU2SRCSVH5N type=write size=2550841 2026-02-24T22:10:45.574 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.573+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~YUEWA9BTJB0J.0 size=1048576 2026-02-24T22:10:45.575 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.573+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~YUEWA9BTJB0J.1 size=970210 2026-02-24T22:10:45.575 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.573+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=YUEWA9BTJB0J tag=S369K8E6NVI1 type=write size=3067362 2026-02-24T22:10:45.576 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.573+0000 7f7e64854980 1 < suggested remove operation key=0B2V0MHWSOC9 tag=EZQ5I2IG0FGQ type=del 2026-02-24T22:10:45.577 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.573+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=8BF8FJVVYCQ4 type=del size=2134684 2026-02-24T22:10:45.578 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.577+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~6A21BDW97I2U.0 size=1048576 2026-02-24T22:10:45.579 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.577+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~6A21BDW97I2U.1 size=481994 2026-02-24T22:10:45.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.577+0000 7f7e64854980 1 < completed multipart upload key=0B2V0MHWSOC9 upload=6A21BDW97I2U tag=10NL29M7JWTH type=write size=2579146 2026-02-24T22:10:45.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.577+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=P12T405I6BXQ type=del size=2084765 2026-02-24T22:10:45.582 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.581+0000 7f7e64854980 -1 < failed to complete operation key=0B2V0MHWSOC9 tag=05ZSL9ZBB5Y0 type=del size=1290663: (22) Invalid argument 2026-02-24T22:10:45.583 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.581+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~IHIJX46Y0C9Z.0 size=1048576 2026-02-24T22:10:45.584 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.581+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~IHIJX46Y0C9Z.1 size=1048576 2026-02-24T22:10:45.585 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.581+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~IHIJX46Y0C9Z.2 size=184462 2026-02-24T22:10:45.585 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.581+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=IHIJX46Y0C9Z tag=Y8XX2SCIP8P1 type=write size=3330190 2026-02-24T22:10:45.585 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.585+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~X0Q3IPTP8MOG.0 size=1048576 2026-02-24T22:10:45.586 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.585+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~X0Q3IPTP8MOG.1 size=770927 2026-02-24T22:10:45.587 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.585+0000 7f7e64854980 1 < completed multipart upload key=Q7BP8FXTARDW upload=X0Q3IPTP8MOG tag=0B0DK9F11UXK type=write size=2868079 2026-02-24T22:10:45.590 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.589+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~TVWH3PWC42DB.0 size=1048576 2026-02-24T22:10:45.590 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.589+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~TVWH3PWC42DB.1 size=828307 2026-02-24T22:10:45.591 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.589+0000 7f7e64854980 1 > prepared multipart upload key=TF5B43G7KCYT upload=TVWH3PWC42DB tag=TR2S2NTOZ9AR type=write size=2925459 2026-02-24T22:10:45.591 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.589+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=WZLDIKAJ0BXO type=del 2026-02-24T22:10:45.592 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.589+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=SX32TCINUF90 type=write size=462721 2026-02-24T22:10:45.593 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.589+0000 7f7e64854980 1 < completed delete operation key=A6PH77Q78VEN tag=KEUBTVSXJ3ZN type=del 2026-02-24T22:10:45.595 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.593+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~PI3AQ1S1UH2Q.0 size=478768 2026-02-24T22:10:45.595 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.593+0000 7f7e64854980 1 > prepared multipart upload key=ON16L53JUSTW upload=PI3AQ1S1UH2Q tag=7QSYAXABIAG8 type=write size=1527344 2026-02-24T22:10:45.595 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.593+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~PKK79BU5VD2I.0 size=698155 2026-02-24T22:10:45.597 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.593+0000 7f7e64854980 1 < completed multipart upload key=Y3L3CJ51C1WR upload=PKK79BU5VD2I tag=4VOCANGFO2WK type=write size=1746731 2026-02-24T22:10:45.598 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.597+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~UBI7BOZ45OXH.0 size=264484 2026-02-24T22:10:45.598 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.597+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=UBI7BOZ45OXH tag=E9ZT5KLTLIL3 type=write size=1313060 2026-02-24T22:10:45.600 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.597+0000 7f7e64854980 1 < suggested remove operation key=HKOQRSF6XEXB tag=FKOICGEZGMP8 type=write 2026-02-24T22:10:45.601 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.601+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~GYFSJ5WQ75S2.0 size=1033995 2026-02-24T22:10:45.601 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.601+0000 7f7e64854980 1 > prepared multipart upload key=0B2V0MHWSOC9 upload=GYFSJ5WQ75S2 tag=VE1N56UE3P67 type=write size=2082571 2026-02-24T22:10:45.602 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.601+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~42RJYZDCEU2Z.0 size=339938 2026-02-24T22:10:45.603 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.601+0000 7f7e64854980 1 < completed multipart upload key=TF5B43G7KCYT upload=42RJYZDCEU2Z tag=GSFAH3N3SF30 type=write size=1388514 2026-02-24T22:10:45.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.601+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~IA71YT5SUEZL.0 size=1048576 2026-02-24T22:10:45.606 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.605+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~IA71YT5SUEZL.1 size=1048576 2026-02-24T22:10:45.606 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.605+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~IA71YT5SUEZL.2 size=434612 2026-02-24T22:10:45.606 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.605+0000 7f7e64854980 1 > prepared multipart upload key=A6PH77Q78VEN upload=IA71YT5SUEZL tag=LXYEKHAW6LZ8 type=write size=3580340 2026-02-24T22:10:45.607 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.605+0000 7f7e64854980 1 < completed delete operation key=L60HQ7M3E3LL tag=DIN1XFE5DY6N type=del 2026-02-24T22:10:45.608 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.605+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=2XTRJCXTQ50U type=del size=1580300 2026-02-24T22:10:45.609 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.605+0000 7f7e64854980 1 < canceled operation key=4Y3974PYYT78 tag=TJSW1JW6NOYT type=del size=766536 2026-02-24T22:10:45.609 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.609+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=KY1Y934ZHWZ5 type=del size=495868 2026-02-24T22:10:45.611 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.609+0000 7f7e64854980 1 < suggested update operation key=ON16L53JUSTW tag=LK1Y9NS2PEKF type=del size=3110020 2026-02-24T22:10:45.612 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.609+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=J48B26FFZGVU type=del size=3401345 2026-02-24T22:10:45.613 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.609+0000 7f7e64854980 1 < completed delete operation key=ZESKGBWZXM6O tag=SM631DJZQO8G type=del 2026-02-24T22:10:45.613 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.613+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=41354H54CVKN type=del size=1369565 2026-02-24T22:10:45.615 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.613+0000 7f7e64854980 1 < suggested remove operation key=0B2V0MHWSOC9 tag=HF84EJLQ14WA type=del 2026-02-24T22:10:45.617 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.613+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~B54WZC32GCHW.0 size=1048576 2026-02-24T22:10:45.618 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.617+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~B54WZC32GCHW.1 size=864065 2026-02-24T22:10:45.618 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.617+0000 7f7e64854980 1 > prepared multipart upload key=Q7BP8FXTARDW upload=B54WZC32GCHW tag=VYY53D2NMPAW type=write size=2961217 2026-02-24T22:10:45.618 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.617+0000 7f7e64854980 1 < completed write operation key=CI8DA17DX8LU tag=Z5J0HO7IMWX7 type=write size=563617 2026-02-24T22:10:45.620 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.617+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~M72UELPF8537.0 size=1048576 2026-02-24T22:10:45.620 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.617+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~M72UELPF8537.1 size=1048576 2026-02-24T22:10:45.621 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.617+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~M72UELPF8537.2 size=198926 2026-02-24T22:10:45.621 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.617+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=M72UELPF8537 tag=3D686TVZD7LA type=write size=3344654 2026-02-24T22:10:45.622 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.621+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~YUEWA9BTJB0J.0 size=1048576 2026-02-24T22:10:45.623 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.621+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~YUEWA9BTJB0J.1 size=970210 2026-02-24T22:10:45.624 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.621+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=YUEWA9BTJB0J tag=S369K8E6NVI1 type=write size=3067362 2026-02-24T22:10:45.625 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.621+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=9NQYGKG78BFZ type=del size=2170879 2026-02-24T22:10:45.625 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.625+0000 7f7e64854980 1 < completed delete operation key=EJCX06TE5RTZ tag=8BF8FJVVYCQ4 type=del 2026-02-24T22:10:45.626 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.625+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=8FID2ARPIA1J type=del size=1529389 2026-02-24T22:10:45.627 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.625+0000 7f7e64854980 1 < completed delete operation key=78GUTC5THI2P tag=P12T405I6BXQ type=del 2026-02-24T22:10:45.628 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.625+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=PN1JM64FNYKA type=del size=4175436 2026-02-24T22:10:45.629 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.625+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~IHIJX46Y0C9Z.0 size=1048576 2026-02-24T22:10:45.629 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.629+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~IHIJX46Y0C9Z.1 size=1048576 2026-02-24T22:10:45.630 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.629+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~IHIJX46Y0C9Z.2 size=184462 2026-02-24T22:10:45.631 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.629+0000 7f7e64854980 1 < completed multipart upload key=ZESKGBWZXM6O upload=IHIJX46Y0C9Z tag=Y8XX2SCIP8P1 type=write size=3330190 2026-02-24T22:10:45.632 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.629+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=6KE5GMRECFMF type=del size=3180038 2026-02-24T22:10:45.633 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.629+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~TVWH3PWC42DB.0 size=1048576 2026-02-24T22:10:45.634 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.633+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~TVWH3PWC42DB.1 size=828307 2026-02-24T22:10:45.635 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.633+0000 7f7e64854980 1 < completed multipart upload key=TF5B43G7KCYT upload=TVWH3PWC42DB tag=TR2S2NTOZ9AR type=write size=2925459 2026-02-24T22:10:45.636 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.633+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~HYAD03D5LXP5.0 size=434125 2026-02-24T22:10:45.637 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.633+0000 7f7e64854980 1 > prepared multipart upload key=36MKR2OZN7K9 upload=HYAD03D5LXP5 tag=OWKLVU0AJHVM type=write size=1482701 2026-02-24T22:10:45.637 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.637+0000 7f7e64854980 1 < completed write operation key=Q7BP8FXTARDW tag=SX32TCINUF90 type=write size=462721 2026-02-24T22:10:45.638 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.637+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=NY3ZBQ9H21O6 type=del size=3594050 2026-02-24T22:10:45.639 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.637+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~PI3AQ1S1UH2Q.0 size=478768 2026-02-24T22:10:45.640 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.637+0000 7f7e64854980 1 < completed multipart upload key=ON16L53JUSTW upload=PI3AQ1S1UH2Q tag=7QSYAXABIAG8 type=write size=1527344 2026-02-24T22:10:45.641 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.637+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=FPLZ5ZVC9WI6 type=del size=650018 2026-02-24T22:10:45.642 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.641+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~UBI7BOZ45OXH.0 size=264484 2026-02-24T22:10:45.643 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.641+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=UBI7BOZ45OXH tag=E9ZT5KLTLIL3 type=write size=1313060 2026-02-24T22:10:45.645 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.641+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~5FKR5HHHX77V.0 size=1048576 2026-02-24T22:10:45.645 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.645+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~5FKR5HHHX77V.1 size=289259 2026-02-24T22:10:45.645 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.645+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=5FKR5HHHX77V tag=HZ1JK4KRZ42O type=write size=2386411 2026-02-24T22:10:45.646 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.645+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~GYFSJ5WQ75S2.0 size=1033995 2026-02-24T22:10:45.647 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.645+0000 7f7e64854980 1 < completed multipart upload key=0B2V0MHWSOC9 upload=GYFSJ5WQ75S2 tag=VE1N56UE3P67 type=write size=2082571 2026-02-24T22:10:45.648 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.645+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=M54U00NJM8D5 type=del size=487956 2026-02-24T22:10:45.649 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.645+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~IA71YT5SUEZL.0 size=1048576 2026-02-24T22:10:45.650 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.649+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~IA71YT5SUEZL.1 size=1048576 2026-02-24T22:10:45.650 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.649+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~IA71YT5SUEZL.2 size=434612 2026-02-24T22:10:45.652 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.649+0000 7f7e64854980 1 < completed multipart upload key=A6PH77Q78VEN upload=IA71YT5SUEZL tag=LXYEKHAW6LZ8 type=write size=3580340 2026-02-24T22:10:45.653 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.649+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~VS4PB9DMAQQ4.0 size=1048576 2026-02-24T22:10:45.654 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.653+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~VS4PB9DMAQQ4.1 size=657632 2026-02-24T22:10:45.654 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.653+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=VS4PB9DMAQQ4 tag=23GD79S56GJE type=write size=2754784 2026-02-24T22:10:45.655 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.653+0000 7f7e64854980 1 < suggested update operation key=NL60Q5QOSP8E tag=2XTRJCXTQ50U type=del size=1580300 2026-02-24T22:10:45.657 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.653+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~Q19GY1D9ZRFQ.0 size=578695 2026-02-24T22:10:45.657 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.653+0000 7f7e64854980 1 > prepared multipart upload key=HKOQRSF6XEXB upload=Q19GY1D9ZRFQ tag=NZBCBDNP2X0E type=write size=1627271 2026-02-24T22:10:45.658 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.657+0000 7f7e64854980 1 < completed delete operation key=A6PH77Q78VEN tag=KY1Y934ZHWZ5 type=del 2026-02-24T22:10:45.659 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.657+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=J7HTD5TYEN0J type=del size=2236295 2026-02-24T22:10:45.659 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.657+0000 7f7e64854980 1 < completed delete operation key=TXRUQQ456DD8 tag=J48B26FFZGVU type=del 2026-02-24T22:10:45.661 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.657+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~1EYRIRBNM4QY.0 size=96086 2026-02-24T22:10:45.661 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.657+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=1EYRIRBNM4QY tag=NOFBFHBRFH2G type=write size=1144662 2026-02-24T22:10:45.662 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.661+0000 7f7e64854980 1 < completed delete operation key=3900SFJK1YQU tag=41354H54CVKN type=del 2026-02-24T22:10:45.663 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.661+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~2PFGFP3TLO07.0 size=1048576 2026-02-24T22:10:45.665 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.661+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~2PFGFP3TLO07.1 size=1048576 2026-02-24T22:10:45.665 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.665+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~2PFGFP3TLO07.2 size=1019262 2026-02-24T22:10:45.665 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.665+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=2PFGFP3TLO07 tag=MC8977QI1PRT type=write size=4164990 2026-02-24T22:10:45.666 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.665+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~B54WZC32GCHW.0 size=1048576 2026-02-24T22:10:45.667 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.665+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~B54WZC32GCHW.1 size=864065 2026-02-24T22:10:45.668 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.665+0000 7f7e64854980 1 < completed multipart upload key=Q7BP8FXTARDW upload=B54WZC32GCHW tag=VYY53D2NMPAW type=write size=2961217 2026-02-24T22:10:45.669 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.665+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=QG2GI8KYXIDC type=del size=38827 2026-02-24T22:10:45.670 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.669+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~M72UELPF8537.0 size=1048576 2026-02-24T22:10:45.670 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.669+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~M72UELPF8537.1 size=1048576 2026-02-24T22:10:45.671 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.669+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~M72UELPF8537.2 size=198926 2026-02-24T22:10:45.672 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.669+0000 7f7e64854980 1 < completed multipart upload key=HOHKR92SN25M upload=M72UELPF8537 tag=3D686TVZD7LA type=write size=3344654 2026-02-24T22:10:45.674 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.673+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~E60Y1H9TSSHC.0 size=1048576 2026-02-24T22:10:45.674 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.673+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~E60Y1H9TSSHC.1 size=1048576 2026-02-24T22:10:45.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.673+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~E60Y1H9TSSHC.2 size=526821 2026-02-24T22:10:45.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.673+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=E60Y1H9TSSHC tag=LZC7ENICZR79 type=write size=3672549 2026-02-24T22:10:45.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.673+0000 7f7e64854980 1 < completed delete operation key=EB04ETNYM087 tag=9NQYGKG78BFZ type=del 2026-02-24T22:10:45.677 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.673+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~1S1ZCVYIEYK0.0 size=983288 2026-02-24T22:10:45.677 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.673+0000 7f7e64854980 1 > prepared multipart upload key=UQLUT9ZAI61V upload=1S1ZCVYIEYK0 tag=DOTMYV9W8MGZ type=write size=2031864 2026-02-24T22:10:45.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.677+0000 7f7e64854980 1 < completed delete operation key=3900SFJK1YQU tag=8FID2ARPIA1J type=del 2026-02-24T22:10:45.679 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.677+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=C37IF1OFIA0M type=del size=139720 2026-02-24T22:10:45.680 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.677+0000 7f7e64854980 -1 < failed to complete operation key=NL60Q5QOSP8E tag=PN1JM64FNYKA type=del size=4175436: (22) Invalid argument 2026-02-24T22:10:45.680 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.677+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=96NQTSEBJ5WK type=del size=1367554 2026-02-24T22:10:45.682 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.681+0000 7f7e64854980 1 < suggested remove operation key=4Y3974PYYT78 tag=6KE5GMRECFMF type=del 2026-02-24T22:10:45.683 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.681+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=N4GOCU46HH75 type=del size=865513 2026-02-24T22:10:45.684 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.681+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~HYAD03D5LXP5.0 size=434125 2026-02-24T22:10:45.685 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.681+0000 7f7e64854980 1 < completed multipart upload key=36MKR2OZN7K9 upload=HYAD03D5LXP5 tag=OWKLVU0AJHVM type=write size=1482701 2026-02-24T22:10:45.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.685+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=82ASGLPTSUNL type=del size=115255 2026-02-24T22:10:45.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.685+0000 7f7e64854980 -1 < failed to complete operation key=4Y3974PYYT78 tag=NY3ZBQ9H21O6 type=del size=3594050: (22) Invalid argument 2026-02-24T22:10:45.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.685+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=YA5T6N66CPYK type=write size=254311 2026-02-24T22:10:45.688 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.685+0000 7f7e64854980 1 < canceled operation key=ON16L53JUSTW tag=FPLZ5ZVC9WI6 type=del size=650018 2026-02-24T22:10:45.690 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.689+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~6CWPMNKY9ZTQ.0 size=1048576 2026-02-24T22:10:45.690 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.689+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~6CWPMNKY9ZTQ.1 size=138953 2026-02-24T22:10:45.690 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.689+0000 7f7e64854980 1 > prepared multipart upload key=78GUTC5THI2P upload=6CWPMNKY9ZTQ tag=2CZCZAX46ADJ type=write size=2236105 2026-02-24T22:10:45.691 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.689+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~5FKR5HHHX77V.0 size=1048576 2026-02-24T22:10:45.692 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.689+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~5FKR5HHHX77V.1 size=289259 2026-02-24T22:10:45.693 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.689+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=5FKR5HHHX77V tag=HZ1JK4KRZ42O type=write size=2386411 2026-02-24T22:10:45.694 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.693+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=5ZA51J7GS0KN type=del size=2983496 2026-02-24T22:10:45.695 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.693+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=M54U00NJM8D5 type=del 2026-02-24T22:10:45.696 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.693+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=YU62QTZUCIRZ type=del size=173193 2026-02-24T22:10:45.696 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.693+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~VS4PB9DMAQQ4.0 size=1048576 2026-02-24T22:10:45.697 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.693+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~VS4PB9DMAQQ4.1 size=657632 2026-02-24T22:10:45.698 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.697+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=VS4PB9DMAQQ4 tag=23GD79S56GJE type=write size=2754784 2026-02-24T22:10:45.699 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.697+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=UPII3HG37TWZ type=del size=3643276 2026-02-24T22:10:45.700 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.697+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~Q19GY1D9ZRFQ.0 size=578695 2026-02-24T22:10:45.701 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.701+0000 7f7e64854980 1 < completed multipart upload key=HKOQRSF6XEXB upload=Q19GY1D9ZRFQ tag=NZBCBDNP2X0E type=write size=1627271 2026-02-24T22:10:45.702 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.701+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=0AXTD1M3NLHO type=del size=3141300 2026-02-24T22:10:45.703 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.701+0000 7f7e64854980 1 < completed delete operation key=L60HQ7M3E3LL tag=J7HTD5TYEN0J type=del 2026-02-24T22:10:45.704 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.701+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=CV658V3F4H4V type=write size=421409 2026-02-24T22:10:45.704 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.701+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~1EYRIRBNM4QY.0 size=96086 2026-02-24T22:10:45.706 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.705+0000 7f7e64854980 1 < completed multipart upload key=HOHKR92SN25M upload=1EYRIRBNM4QY tag=NOFBFHBRFH2G type=write size=1144662 2026-02-24T22:10:45.707 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.705+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=6REKUF9KW1NF type=del size=733623 2026-02-24T22:10:45.708 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.705+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~2PFGFP3TLO07.0 size=1048576 2026-02-24T22:10:45.708 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.705+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~2PFGFP3TLO07.1 size=1048576 2026-02-24T22:10:45.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.705+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~2PFGFP3TLO07.2 size=1019262 2026-02-24T22:10:45.710 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.709+0000 7f7e64854980 1 < completed multipart upload key=HOHKR92SN25M upload=2PFGFP3TLO07 tag=MC8977QI1PRT type=write size=4164990 2026-02-24T22:10:45.711 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.709+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=PWGKVKLPOM2Z type=write size=239560 2026-02-24T22:10:45.712 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.709+0000 7f7e64854980 1 < canceled operation key=36MKR2OZN7K9 tag=QG2GI8KYXIDC type=del size=38827 2026-02-24T22:10:45.713 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.709+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~YBP4YJC8PD63.0 size=1048576 2026-02-24T22:10:45.714 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.713+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~YBP4YJC8PD63.1 size=1048576 2026-02-24T22:10:45.714 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.713+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~YBP4YJC8PD63.2 size=853787 2026-02-24T22:10:45.715 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.713+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=YBP4YJC8PD63 tag=A68BVOZ9F8JY type=write size=3999515 2026-02-24T22:10:45.715 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.713+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~E60Y1H9TSSHC.0 size=1048576 2026-02-24T22:10:45.716 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.713+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~E60Y1H9TSSHC.1 size=1048576 2026-02-24T22:10:45.716 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.713+0000 7f7e64854980 1 < canceled part key=_multipart_D0MKGNRDW6L9.2~E60Y1H9TSSHC.2 size=526821 2026-02-24T22:10:45.717 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.713+0000 7f7e64854980 1 < canceled multipart upload key=D0MKGNRDW6L9 upload=E60Y1H9TSSHC tag=LZC7ENICZR79 type=write size=3672549 2026-02-24T22:10:45.718 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.717+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=TYETDQKYI5H3 type=del size=1078754 2026-02-24T22:10:45.719 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.717+0000 7f7e64854980 1 < completed part key=_multipart_UQLUT9ZAI61V.2~1S1ZCVYIEYK0.0 size=983288 2026-02-24T22:10:45.720 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.717+0000 7f7e64854980 1 < completed multipart upload key=UQLUT9ZAI61V upload=1S1ZCVYIEYK0 tag=DOTMYV9W8MGZ type=write size=2031864 2026-02-24T22:10:45.721 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.717+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~D276RCJCM3DW.0 size=1048576 2026-02-24T22:10:45.722 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.721+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~D276RCJCM3DW.1 size=97200 2026-02-24T22:10:45.722 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.721+0000 7f7e64854980 1 > prepared multipart upload key=LSRPN33Z2B9N upload=D276RCJCM3DW tag=J2Y2EMSPKJJE type=write size=2194352 2026-02-24T22:10:45.723 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.721+0000 7f7e64854980 1 < completed delete operation key=36MKR2OZN7K9 tag=C37IF1OFIA0M type=del 2026-02-24T22:10:45.724 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.721+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~ZPXH2CJMJZC5.0 size=1048576 2026-02-24T22:10:45.724 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.721+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~ZPXH2CJMJZC5.1 size=889218 2026-02-24T22:10:45.725 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.721+0000 7f7e64854980 1 > prepared multipart upload key=Q7BP8FXTARDW upload=ZPXH2CJMJZC5 tag=FOONYCM61V0Z type=write size=2986370 2026-02-24T22:10:45.726 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.725+0000 7f7e64854980 1 < completed delete operation key=D0MKGNRDW6L9 tag=96NQTSEBJ5WK type=del 2026-02-24T22:10:45.726 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.725+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=I0QVC83ZPQW7 type=del size=1176218 2026-02-24T22:10:45.727 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.725+0000 7f7e64854980 1 < canceled operation key=0AXS7DY23UZU tag=N4GOCU46HH75 type=del size=865513 2026-02-24T22:10:45.728 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.725+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=FECP18TNQ1CZ type=del size=2812283 2026-02-24T22:10:45.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.725+0000 7f7e64854980 1 < canceled operation key=TF5B43G7KCYT tag=82ASGLPTSUNL type=del size=115255 2026-02-24T22:10:45.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.729+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=EYNVJNLZ9G03 type=del size=3791751 2026-02-24T22:10:45.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.729+0000 7f7e64854980 1 < suggested remove operation key=0AXS7DY23UZU tag=YA5T6N66CPYK type=write 2026-02-24T22:10:45.732 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.729+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~YLVVWWF0T9ON.0 size=1048576 2026-02-24T22:10:45.733 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.729+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~YLVVWWF0T9ON.1 size=1048576 2026-02-24T22:10:45.734 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.733+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~YLVVWWF0T9ON.2 size=191069 2026-02-24T22:10:45.734 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.733+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=YLVVWWF0T9ON tag=MDL3TPL13L0B type=write size=3336797 2026-02-24T22:10:45.735 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.733+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~6CWPMNKY9ZTQ.0 size=1048576 2026-02-24T22:10:45.735 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.733+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~6CWPMNKY9ZTQ.1 size=138953 2026-02-24T22:10:45.737 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.733+0000 7f7e64854980 1 < completed multipart upload key=78GUTC5THI2P upload=6CWPMNKY9ZTQ tag=2CZCZAX46ADJ type=write size=2236105 2026-02-24T22:10:45.738 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.737+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=Q7QBCMEGSA10 type=del size=1800720 2026-02-24T22:10:45.739 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.737+0000 7f7e64854980 1 < completed delete operation key=D0MKGNRDW6L9 tag=5ZA51J7GS0KN type=del 2026-02-24T22:10:45.740 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.737+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=XFVW8G835C2Q type=del size=589285 2026-02-24T22:10:45.741 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.737+0000 7f7e64854980 1 < completed delete operation key=TXRUQQ456DD8 tag=YU62QTZUCIRZ type=del 2026-02-24T22:10:45.741 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.741+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=5MMEN8CMB27I type=write size=488200 2026-02-24T22:10:45.742 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.741+0000 7f7e64854980 1 < completed delete operation key=R2893NES3WSF tag=UPII3HG37TWZ type=del 2026-02-24T22:10:45.743 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.741+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=5WF8XUX1DS1H type=write size=492639 2026-02-24T22:10:45.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.741+0000 7f7e64854980 1 < suggested update operation key=HOHKR92SN25M tag=0AXTD1M3NLHO type=del size=3141300 2026-02-24T22:10:45.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.745+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=EZQ1ACG2RX1O type=del size=982584 2026-02-24T22:10:45.746 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.745+0000 7f7e64854980 -1 < failed to complete operation key=HOHKR92SN25M tag=CV658V3F4H4V type=write size=421409: (22) Invalid argument 2026-02-24T22:10:45.748 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.745+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~J751JNJ2COVS.0 size=1048576 2026-02-24T22:10:45.748 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.745+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~J751JNJ2COVS.1 size=677907 2026-02-24T22:10:45.748 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.745+0000 7f7e64854980 1 > prepared multipart upload key=EB04ETNYM087 upload=J751JNJ2COVS tag=5HNGOY5YJ8MX type=write size=2775059 2026-02-24T22:10:45.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.745+0000 7f7e64854980 -1 < no bi entry to suggest for operation key=0AXS7DY23UZU tag=6REKUF9KW1NF type=del size=733623: (2) No such file or directory 2026-02-24T22:10:45.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.749+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=LELRL9ZTSUWQ type=del size=557658 2026-02-24T22:10:45.751 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.749+0000 7f7e64854980 1 < suggested remove operation key=N3ZUNSI5K8HS tag=PWGKVKLPOM2Z type=write 2026-02-24T22:10:45.752 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.749+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~XU1XHTFMTBP2.0 size=1048576 2026-02-24T22:10:45.753 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.749+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~XU1XHTFMTBP2.1 size=1048576 2026-02-24T22:10:45.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.753+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~XU1XHTFMTBP2.2 size=88218 2026-02-24T22:10:45.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.753+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=XU1XHTFMTBP2 tag=FMD3X8Q31AY8 type=write size=3233946 2026-02-24T22:10:45.755 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.753+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~YBP4YJC8PD63.0 size=1048576 2026-02-24T22:10:45.755 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.753+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~YBP4YJC8PD63.1 size=1048576 2026-02-24T22:10:45.756 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.753+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~YBP4YJC8PD63.2 size=853787 2026-02-24T22:10:45.757 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.753+0000 7f7e64854980 1 < completed multipart upload key=D0MKGNRDW6L9 upload=YBP4YJC8PD63 tag=A68BVOZ9F8JY type=write size=3999515 2026-02-24T22:10:45.759 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.757+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~6DTV590020WU.0 size=751152 2026-02-24T22:10:45.759 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.757+0000 7f7e64854980 1 > prepared multipart upload key=78GUTC5THI2P upload=6DTV590020WU tag=8AW0VQ3RQORX type=write size=1799728 2026-02-24T22:10:45.759 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.757+0000 7f7e64854980 1 < completed delete operation key=HKOQRSF6XEXB tag=TYETDQKYI5H3 type=del 2026-02-24T22:10:45.761 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.757+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~SPZI918YKK43.0 size=1048576 2026-02-24T22:10:45.762 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.761+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~SPZI918YKK43.1 size=1048576 2026-02-24T22:10:45.762 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.761+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~SPZI918YKK43.2 size=349896 2026-02-24T22:10:45.762 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.761+0000 7f7e64854980 1 > prepared multipart upload key=LSRPN33Z2B9N upload=SPZI918YKK43 tag=5K3P0N8UHVHF type=write size=3495624 2026-02-24T22:10:45.763 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.761+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~D276RCJCM3DW.0 size=1048576 2026-02-24T22:10:45.764 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.761+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~D276RCJCM3DW.1 size=97200 2026-02-24T22:10:45.765 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.761+0000 7f7e64854980 1 < completed multipart upload key=LSRPN33Z2B9N upload=D276RCJCM3DW tag=J2Y2EMSPKJJE type=write size=2194352 2026-02-24T22:10:45.766 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.765+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=JN8MNZUZDKIR type=del size=706903 2026-02-24T22:10:45.766 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.765+0000 7f7e64854980 1 < canceled part key=_multipart_Q7BP8FXTARDW.2~ZPXH2CJMJZC5.0 size=1048576 2026-02-24T22:10:45.767 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.765+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~ZPXH2CJMJZC5.1 size=889218 2026-02-24T22:10:45.768 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.765+0000 7f7e64854980 1 < canceled multipart upload key=Q7BP8FXTARDW upload=ZPXH2CJMJZC5 tag=FOONYCM61V0Z type=write size=2986370 2026-02-24T22:10:45.769 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.765+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=6M8HMXDVKG2M type=del size=3232511 2026-02-24T22:10:45.770 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.769+0000 7f7e64854980 1 < completed delete operation key=78GUTC5THI2P tag=I0QVC83ZPQW7 type=del 2026-02-24T22:10:45.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.769+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=VG50NS6VSKW4 type=write size=799879 2026-02-24T22:10:45.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.769+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=FECP18TNQ1CZ type=del 2026-02-24T22:10:45.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.769+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=ZDHDLB866FZQ type=del size=1160405 2026-02-24T22:10:45.773 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.769+0000 7f7e64854980 -1 < failed to complete operation key=HOHKR92SN25M tag=EYNVJNLZ9G03 type=del size=3791751: (22) Invalid argument 2026-02-24T22:10:45.774 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.773+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~JK8SBQKHT9TE.0 size=1005336 2026-02-24T22:10:45.775 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.773+0000 7f7e64854980 1 > prepared multipart upload key=UQLUT9ZAI61V upload=JK8SBQKHT9TE tag=1U6CL8DK3KJU type=write size=2053912 2026-02-24T22:10:45.775 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.773+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~YLVVWWF0T9ON.0 size=1048576 2026-02-24T22:10:45.776 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.773+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~YLVVWWF0T9ON.1 size=1048576 2026-02-24T22:10:45.776 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.773+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~YLVVWWF0T9ON.2 size=191069 2026-02-24T22:10:45.778 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.777+0000 7f7e64854980 1 < completed multipart upload key=CI8DA17DX8LU upload=YLVVWWF0T9ON tag=MDL3TPL13L0B type=write size=3336797 2026-02-24T22:10:45.779 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.777+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~6U25F7TDZOO7.0 size=1048576 2026-02-24T22:10:45.780 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.777+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~6U25F7TDZOO7.1 size=1048576 2026-02-24T22:10:45.780 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.777+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~6U25F7TDZOO7.2 size=762143 2026-02-24T22:10:45.781 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.777+0000 7f7e64854980 1 > prepared multipart upload key=LSRPN33Z2B9N upload=6U25F7TDZOO7 tag=2V8TUUDZ41K7 type=write size=3907871 2026-02-24T22:10:45.781 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.781+0000 7f7e64854980 1 < completed delete operation key=LSRPN33Z2B9N tag=Q7QBCMEGSA10 type=del 2026-02-24T22:10:45.783 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.781+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~8V5XWQK7NNZV.0 size=1048576 2026-02-24T22:10:45.783 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.781+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~8V5XWQK7NNZV.1 size=1048576 2026-02-24T22:10:45.784 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.781+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~8V5XWQK7NNZV.2 size=752710 2026-02-24T22:10:45.784 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.781+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=8V5XWQK7NNZV tag=IVDFGF7AT9UP type=write size=3898438 2026-02-24T22:10:45.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.781+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=XFVW8G835C2Q type=del 2026-02-24T22:10:45.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.785+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=O1AHEKILUGIF type=del size=1823453 2026-02-24T22:10:45.786 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.785+0000 7f7e64854980 1 < completed write operation key=EJCX06TE5RTZ tag=5MMEN8CMB27I type=write size=488200 2026-02-24T22:10:45.787 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.785+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=KIT8303LRY0X type=del size=2588180 2026-02-24T22:10:45.788 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.785+0000 7f7e64854980 1 < completed write operation key=ZESKGBWZXM6O tag=5WF8XUX1DS1H type=write size=492639 2026-02-24T22:10:45.789 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.785+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=2A9GY7XKWRHK type=write size=607161 2026-02-24T22:10:45.790 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.789+0000 7f7e64854980 1 < suggested update operation key=Y8GMOUA977H3 tag=EZQ1ACG2RX1O type=del size=982584 2026-02-24T22:10:45.792 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.789+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~9UEFYSNAIUJS.0 size=1048576 2026-02-24T22:10:45.792 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.789+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~9UEFYSNAIUJS.1 size=26208 2026-02-24T22:10:45.792 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.789+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=9UEFYSNAIUJS tag=G01WHT3KD6YC type=write size=2123360 2026-02-24T22:10:45.793 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.789+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~J751JNJ2COVS.0 size=1048576 2026-02-24T22:10:45.794 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.793+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~J751JNJ2COVS.1 size=677907 2026-02-24T22:10:45.795 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.793+0000 7f7e64854980 1 < completed multipart upload key=EB04ETNYM087 upload=J751JNJ2COVS tag=5HNGOY5YJ8MX type=write size=2775059 2026-02-24T22:10:45.796 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.793+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=7ZFDV910AYK1 type=del size=1037684 2026-02-24T22:10:45.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.793+0000 7f7e64854980 1 < completed delete operation key=SGQYQ2TIVJPS tag=LELRL9ZTSUWQ type=del 2026-02-24T22:10:45.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.797+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=XA67R8PBYQ28 type=del size=4142788 2026-02-24T22:10:45.798 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.797+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~XU1XHTFMTBP2.0 size=1048576 2026-02-24T22:10:45.799 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.797+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~XU1XHTFMTBP2.1 size=1048576 2026-02-24T22:10:45.799 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.797+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~XU1XHTFMTBP2.2 size=88218 2026-02-24T22:10:45.801 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.797+0000 7f7e64854980 1 < completed multipart upload key=D0MKGNRDW6L9 upload=XU1XHTFMTBP2 tag=FMD3X8Q31AY8 type=write size=3233946 2026-02-24T22:10:45.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.801+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=HA4HLGNU32SF type=del size=2046038 2026-02-24T22:10:45.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.801+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~6DTV590020WU.0 size=751152 2026-02-24T22:10:45.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.801+0000 7f7e64854980 1 < completed multipart upload key=78GUTC5THI2P upload=6DTV590020WU tag=8AW0VQ3RQORX type=write size=1799728 2026-02-24T22:10:45.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.801+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~CVXHG3Z326CN.0 size=1048576 2026-02-24T22:10:45.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.805+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~CVXHG3Z326CN.1 size=1048576 2026-02-24T22:10:45.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.805+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~CVXHG3Z326CN.2 size=897280 2026-02-24T22:10:45.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.805+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=CVXHG3Z326CN tag=2LJ9D1BZ955J type=write size=4043008 2026-02-24T22:10:45.807 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.805+0000 7f7e64854980 1 < canceled part key=_multipart_LSRPN33Z2B9N.2~SPZI918YKK43.0 size=1048576 2026-02-24T22:10:45.807 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.805+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~SPZI918YKK43.1 size=1048576 2026-02-24T22:10:45.808 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.805+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~SPZI918YKK43.2 size=349896 2026-02-24T22:10:45.809 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.805+0000 7f7e64854980 1 < canceled multipart upload key=LSRPN33Z2B9N upload=SPZI918YKK43 tag=5K3P0N8UHVHF type=write size=3495624 2026-02-24T22:10:45.810 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.809+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~OGB8645NGJGB.0 size=1048576 2026-02-24T22:10:45.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.809+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~OGB8645NGJGB.1 size=766290 2026-02-24T22:10:45.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.809+0000 7f7e64854980 1 > prepared multipart upload key=SGQYQ2TIVJPS upload=OGB8645NGJGB tag=GD4191F6FN1W type=write size=2863442 2026-02-24T22:10:45.813 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.809+0000 7f7e64854980 1 < completed delete operation key=TF5B43G7KCYT tag=JN8MNZUZDKIR type=del 2026-02-24T22:10:45.813 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.813+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=1J2123AI6KLO type=del size=4133886 2026-02-24T22:10:45.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.813+0000 7f7e64854980 1 < completed delete operation key=2KKQKHXO7K3K tag=6M8HMXDVKG2M type=del 2026-02-24T22:10:45.815 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.813+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=R0LKOPBD2JAI type=del size=3981812 2026-02-24T22:10:45.816 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.813+0000 7f7e64854980 1 < completed write operation key=LSRPN33Z2B9N tag=VG50NS6VSKW4 type=write size=799879 2026-02-24T22:10:45.816 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.813+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=C7KY2STOD2PD type=del size=2026851 2026-02-24T22:10:45.817 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.817+0000 7f7e64854980 1 < completed delete operation key=N3ZUNSI5K8HS tag=ZDHDLB866FZQ type=del 2026-02-24T22:10:45.818 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.817+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=BYM1K4J46FD0 type=write size=59016 2026-02-24T22:10:45.818 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.817+0000 7f7e64854980 1 < canceled part key=_multipart_UQLUT9ZAI61V.2~JK8SBQKHT9TE.0 size=1005336 2026-02-24T22:10:45.819 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.817+0000 7f7e64854980 1 < canceled multipart upload key=UQLUT9ZAI61V upload=JK8SBQKHT9TE tag=1U6CL8DK3KJU type=write size=2053912 2026-02-24T22:10:45.820 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.817+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=CSDEFT0IMKGD type=write size=153334 2026-02-24T22:10:45.821 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.817+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~6U25F7TDZOO7.0 size=1048576 2026-02-24T22:10:45.822 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.821+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~6U25F7TDZOO7.1 size=1048576 2026-02-24T22:10:45.822 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.821+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~6U25F7TDZOO7.2 size=762143 2026-02-24T22:10:45.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.821+0000 7f7e64854980 1 < completed multipart upload key=LSRPN33Z2B9N upload=6U25F7TDZOO7 tag=2V8TUUDZ41K7 type=write size=3907871 2026-02-24T22:10:45.825 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.821+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~FIE51DT75WRR.0 size=673077 2026-02-24T22:10:45.825 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.821+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=FIE51DT75WRR tag=2UO0ZYQBDI21 type=write size=1721653 2026-02-24T22:10:45.826 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.825+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~8V5XWQK7NNZV.0 size=1048576 2026-02-24T22:10:45.826 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.825+0000 7f7e64854980 1 < canceled part key=_multipart_0AXS7DY23UZU.2~8V5XWQK7NNZV.1 size=1048576 2026-02-24T22:10:45.826 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.825+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~8V5XWQK7NNZV.2 size=752710 2026-02-24T22:10:45.828 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.825+0000 7f7e64854980 1 < canceled multipart upload key=0AXS7DY23UZU upload=8V5XWQK7NNZV tag=IVDFGF7AT9UP type=write size=3898438 2026-02-24T22:10:45.828 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.825+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=1T0LANGH9JA9 type=del size=3754268 2026-02-24T22:10:45.829 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.825+0000 7f7e64854980 1 < completed delete operation key=HKOQRSF6XEXB tag=O1AHEKILUGIF type=del 2026-02-24T22:10:45.830 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.829+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=DQKZBM2PYXW6 type=del size=3576668 2026-02-24T22:10:45.831 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.829+0000 7f7e64854980 1 < canceled operation key=EB04ETNYM087 tag=KIT8303LRY0X type=del size=2588180 2026-02-24T22:10:45.832 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.829+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~AGNUIXVVSVD0.0 size=842709 2026-02-24T22:10:45.832 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.829+0000 7f7e64854980 1 > prepared multipart upload key=4Y3974PYYT78 upload=AGNUIXVVSVD0 tag=XG7NS4TM46P4 type=write size=1891285 2026-02-24T22:10:45.833 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.829+0000 7f7e64854980 1 < completed write operation key=SGQYQ2TIVJPS tag=2A9GY7XKWRHK type=write size=607161 2026-02-24T22:10:45.834 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.833+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=EK01838O2707 type=del size=3409598 2026-02-24T22:10:45.835 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.833+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~9UEFYSNAIUJS.0 size=1048576 2026-02-24T22:10:45.835 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.833+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~9UEFYSNAIUJS.1 size=26208 2026-02-24T22:10:45.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.833+0000 7f7e64854980 1 < completed multipart upload key=ZESKGBWZXM6O upload=9UEFYSNAIUJS tag=G01WHT3KD6YC type=write size=2123360 2026-02-24T22:10:45.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.837+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=MHYD31UXVUF0 type=del size=567551 2026-02-24T22:10:45.838 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.837+0000 7f7e64854980 1 < completed delete operation key=SGQYQ2TIVJPS tag=7ZFDV910AYK1 type=del 2026-02-24T22:10:45.840 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.837+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~UBHOT79W1JYB.0 size=1048576 2026-02-24T22:10:45.840 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.837+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~UBHOT79W1JYB.1 size=491763 2026-02-24T22:10:45.840 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.837+0000 7f7e64854980 1 > prepared multipart upload key=78GUTC5THI2P upload=UBHOT79W1JYB tag=JRY1K5GWMI92 type=write size=2588915 2026-02-24T22:10:45.841 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.837+0000 7f7e64854980 1 < completed delete operation key=UQLUT9ZAI61V tag=XA67R8PBYQ28 type=del 2026-02-24T22:10:45.842 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.841+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=4GSDGDGILKBB type=del size=1925410 2026-02-24T22:10:45.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.841+0000 7f7e64854980 1 < suggested remove operation key=UQLUT9ZAI61V tag=HA4HLGNU32SF type=del 2026-02-24T22:10:45.845 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.841+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~8JVCRYZVYPOU.0 size=1048576 2026-02-24T22:10:45.845 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.845+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~8JVCRYZVYPOU.1 size=1048576 2026-02-24T22:10:45.846 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.845+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~8JVCRYZVYPOU.2 size=835140 2026-02-24T22:10:45.846 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.845+0000 7f7e64854980 1 > prepared multipart upload key=3900SFJK1YQU upload=8JVCRYZVYPOU tag=5TH4Q8WFDS9O type=write size=3980868 2026-02-24T22:10:45.847 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.845+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~CVXHG3Z326CN.0 size=1048576 2026-02-24T22:10:45.847 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.845+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~CVXHG3Z326CN.1 size=1048576 2026-02-24T22:10:45.848 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.845+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~CVXHG3Z326CN.2 size=897280 2026-02-24T22:10:45.849 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.845+0000 7f7e64854980 1 < completed multipart upload key=ZESKGBWZXM6O upload=CVXHG3Z326CN tag=2LJ9D1BZ955J type=write size=4043008 2026-02-24T22:10:45.851 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.849+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~3UYE4DZ6TSS6.0 size=447906 2026-02-24T22:10:45.851 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.849+0000 7f7e64854980 1 > prepared multipart upload key=LSRPN33Z2B9N upload=3UYE4DZ6TSS6 tag=CII92FA36HX0 type=write size=1496482 2026-02-24T22:10:45.851 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.849+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~OGB8645NGJGB.0 size=1048576 2026-02-24T22:10:45.852 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.849+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~OGB8645NGJGB.1 size=766290 2026-02-24T22:10:45.853 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.853+0000 7f7e64854980 1 < completed multipart upload key=SGQYQ2TIVJPS upload=OGB8645NGJGB tag=GD4191F6FN1W type=write size=2863442 2026-02-24T22:10:45.855 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.853+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~K9SUL1F8K6I1.0 size=1048576 2026-02-24T22:10:45.855 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.853+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~K9SUL1F8K6I1.1 size=1048576 2026-02-24T22:10:45.856 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.853+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~K9SUL1F8K6I1.2 size=151396 2026-02-24T22:10:45.856 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.853+0000 7f7e64854980 1 > prepared multipart upload key=PKRHWM69GBJI upload=K9SUL1F8K6I1 tag=BKYHTAORNSBI type=write size=3297124 2026-02-24T22:10:45.857 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.857+0000 7f7e64854980 1 < suggested update operation key=EJCX06TE5RTZ tag=1J2123AI6KLO type=del size=4133886 2026-02-24T22:10:45.859 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.857+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~RLX4C8QYDWC0.0 size=1048576 2026-02-24T22:10:45.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.857+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~RLX4C8QYDWC0.1 size=1048576 2026-02-24T22:10:45.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.857+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~RLX4C8QYDWC0.2 size=781319 2026-02-24T22:10:45.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.857+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=RLX4C8QYDWC0 tag=VR5G6HUWE4F1 type=write size=3927047 2026-02-24T22:10:45.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.857+0000 7f7e64854980 1 < completed delete operation key=TF5B43G7KCYT tag=R0LKOPBD2JAI type=del 2026-02-24T22:10:45.862 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.861+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=AEXJ34U6K4VS type=del size=2079396 2026-02-24T22:10:45.863 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.861+0000 7f7e64854980 1 < canceled operation key=GH7723GNI2WM tag=C7KY2STOD2PD type=del size=2026851 2026-02-24T22:10:45.863 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.861+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=2QQD6D9QSA6B type=del size=655441 2026-02-24T22:10:45.865 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.861+0000 7f7e64854980 1 < suggested remove operation key=HKOQRSF6XEXB tag=BYM1K4J46FD0 type=write 2026-02-24T22:10:45.866 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.865+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=ZTC67EYI7DYT type=write size=286641 2026-02-24T22:10:45.866 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.865+0000 7f7e64854980 1 < completed write operation key=PKRHWM69GBJI tag=CSDEFT0IMKGD type=write size=153334 2026-02-24T22:10:45.868 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.865+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~MS6OERMX453H.0 size=1048576 2026-02-24T22:10:45.868 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.865+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~MS6OERMX453H.1 size=680903 2026-02-24T22:10:45.869 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.865+0000 7f7e64854980 1 > prepared multipart upload key=GH7723GNI2WM upload=MS6OERMX453H tag=XUV7QTSOEH05 type=write size=2778055 2026-02-24T22:10:45.869 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.869+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~FIE51DT75WRR.0 size=673077 2026-02-24T22:10:45.871 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.869+0000 7f7e64854980 1 < completed multipart upload key=ZESKGBWZXM6O upload=FIE51DT75WRR tag=2UO0ZYQBDI21 type=write size=1721653 2026-02-24T22:10:45.872 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.869+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~EOR4JH16XRZB.0 size=794581 2026-02-24T22:10:45.872 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.869+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=EOR4JH16XRZB tag=KAZ4MV9JVOEF type=write size=1843157 2026-02-24T22:10:45.874 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.873+0000 7f7e64854980 1 < suggested remove operation key=L9NEDH44BIGA tag=1T0LANGH9JA9 type=del 2026-02-24T22:10:45.874 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.873+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=U3GQMN3NU9CM type=del size=2544729 2026-02-24T22:10:45.875 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.873+0000 7f7e64854980 1 < canceled operation key=0AXS7DY23UZU tag=DQKZBM2PYXW6 type=del size=3576668 2026-02-24T22:10:45.877 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.873+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~1LYSIO7ZL1QE.0 size=1048576 2026-02-24T22:10:45.877 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.873+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~1LYSIO7ZL1QE.1 size=116793 2026-02-24T22:10:45.877 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.873+0000 7f7e64854980 1 > prepared multipart upload key=36MKR2OZN7K9 upload=1LYSIO7ZL1QE tag=46D9XPB2DH0Y type=write size=2213945 2026-02-24T22:10:45.878 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.877+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~AGNUIXVVSVD0.0 size=842709 2026-02-24T22:10:45.879 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.877+0000 7f7e64854980 1 < completed multipart upload key=4Y3974PYYT78 upload=AGNUIXVVSVD0 tag=XG7NS4TM46P4 type=write size=1891285 2026-02-24T22:10:45.880 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.877+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=51BRLRV2GP7N type=del size=2323658 2026-02-24T22:10:45.881 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.877+0000 7f7e64854980 1 < completed delete operation key=ZESKGBWZXM6O tag=EK01838O2707 type=del 2026-02-24T22:10:45.882 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.881+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~47CSA2DFL5V7.0 size=1048576 2026-02-24T22:10:45.883 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.881+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~47CSA2DFL5V7.1 size=959526 2026-02-24T22:10:45.883 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.881+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=47CSA2DFL5V7 tag=WIDHPMPBM7PO type=write size=3056678 2026-02-24T22:10:45.885 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.881+0000 7f7e64854980 1 < completed delete operation key=N3ZUNSI5K8HS tag=MHYD31UXVUF0 type=del 2026-02-24T22:10:45.886 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.885+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~HLGC50I6CRTM.0 size=1048576 2026-02-24T22:10:45.887 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.885+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~HLGC50I6CRTM.1 size=587672 2026-02-24T22:10:45.887 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.885+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=HLGC50I6CRTM tag=111ZBS9OM7AY type=write size=2684824 2026-02-24T22:10:45.888 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.885+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~UBHOT79W1JYB.0 size=1048576 2026-02-24T22:10:45.888 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.885+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~UBHOT79W1JYB.1 size=491763 2026-02-24T22:10:45.890 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.889+0000 7f7e64854980 1 < completed multipart upload key=78GUTC5THI2P upload=UBHOT79W1JYB tag=JRY1K5GWMI92 type=write size=2588915 2026-02-24T22:10:45.890 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.889+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=9TUXTO56V8Q9 type=del size=2029339 2026-02-24T22:10:45.891 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.889+0000 7f7e64854980 1 < completed delete operation key=A6PH77Q78VEN tag=4GSDGDGILKBB type=del 2026-02-24T22:10:45.893 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.889+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~JJHS128UUHZ3.0 size=1048576 2026-02-24T22:10:45.893 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.893+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~JJHS128UUHZ3.1 size=1048576 2026-02-24T22:10:45.894 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.893+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~JJHS128UUHZ3.2 size=81911 2026-02-24T22:10:45.894 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.893+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=JJHS128UUHZ3 tag=YZVPU8T464KJ type=write size=3227639 2026-02-24T22:10:45.895 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.893+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~8JVCRYZVYPOU.0 size=1048576 2026-02-24T22:10:45.895 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.893+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~8JVCRYZVYPOU.1 size=1048576 2026-02-24T22:10:45.896 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.893+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~8JVCRYZVYPOU.2 size=835140 2026-02-24T22:10:45.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.893+0000 7f7e64854980 1 < completed multipart upload key=3900SFJK1YQU upload=8JVCRYZVYPOU tag=5TH4Q8WFDS9O type=write size=3980868 2026-02-24T22:10:45.898 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.897+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=SV78V28IFBFC type=write size=758087 2026-02-24T22:10:45.899 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.897+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~3UYE4DZ6TSS6.0 size=447906 2026-02-24T22:10:45.900 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.897+0000 7f7e64854980 1 < completed multipart upload key=LSRPN33Z2B9N upload=3UYE4DZ6TSS6 tag=CII92FA36HX0 type=write size=1496482 2026-02-24T22:10:45.901 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.897+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=1EOLG0DWJ5NC type=del size=2965392 2026-02-24T22:10:45.902 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.901+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~K9SUL1F8K6I1.0 size=1048576 2026-02-24T22:10:45.902 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.901+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~K9SUL1F8K6I1.1 size=1048576 2026-02-24T22:10:45.903 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.901+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~K9SUL1F8K6I1.2 size=151396 2026-02-24T22:10:45.904 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.901+0000 7f7e64854980 1 < completed multipart upload key=PKRHWM69GBJI upload=K9SUL1F8K6I1 tag=BKYHTAORNSBI type=write size=3297124 2026-02-24T22:10:45.905 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.901+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=Q5AKJZDLJ4M3 type=del size=2191374 2026-02-24T22:10:45.905 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.905+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~RLX4C8QYDWC0.0 size=1048576 2026-02-24T22:10:45.906 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.905+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~RLX4C8QYDWC0.1 size=1048576 2026-02-24T22:10:45.907 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.905+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~RLX4C8QYDWC0.2 size=781319 2026-02-24T22:10:45.908 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.905+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=RLX4C8QYDWC0 tag=VR5G6HUWE4F1 type=write size=3927047 2026-02-24T22:10:45.909 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.905+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=P8BLYM3GQA4Z type=del size=2756292 2026-02-24T22:10:45.910 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.909+0000 7f7e64854980 1 < suggested update operation key=UQLUT9ZAI61V tag=AEXJ34U6K4VS type=del size=2079396 2026-02-24T22:10:45.911 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.909+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=AY8ZNBPWS0HG type=del size=2491418 2026-02-24T22:10:45.912 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.909+0000 7f7e64854980 1 < completed delete operation key=D0MKGNRDW6L9 tag=2QQD6D9QSA6B type=del 2026-02-24T22:10:45.913 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.909+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=JLYVNWFY040M type=del size=770370 2026-02-24T22:10:45.913 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.913+0000 7f7e64854980 1 < completed write operation key=D0MKGNRDW6L9 tag=ZTC67EYI7DYT type=write size=286641 2026-02-24T22:10:45.914 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.913+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=YTPP7SN33JRW type=del size=1752048 2026-02-24T22:10:45.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.913+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~MS6OERMX453H.0 size=1048576 2026-02-24T22:10:45.916 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.913+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~MS6OERMX453H.1 size=680903 2026-02-24T22:10:45.917 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.913+0000 7f7e64854980 1 < completed multipart upload key=GH7723GNI2WM upload=MS6OERMX453H tag=XUV7QTSOEH05 type=write size=2778055 2026-02-24T22:10:45.918 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.917+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=JRE03LSJFP4W type=write size=852061 2026-02-24T22:10:45.918 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.917+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~EOR4JH16XRZB.0 size=794581 2026-02-24T22:10:45.920 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.917+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=EOR4JH16XRZB tag=KAZ4MV9JVOEF type=write size=1843157 2026-02-24T22:10:45.921 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.917+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=ZYATKJAG4QKR type=del size=218241 2026-02-24T22:10:45.921 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.921+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=U3GQMN3NU9CM type=del 2026-02-24T22:10:45.922 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.921+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=E4B16JA01OV1 type=del size=3134186 2026-02-24T22:10:45.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.921+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~1LYSIO7ZL1QE.0 size=1048576 2026-02-24T22:10:45.924 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.921+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~1LYSIO7ZL1QE.1 size=116793 2026-02-24T22:10:45.925 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.921+0000 7f7e64854980 1 < completed multipart upload key=36MKR2OZN7K9 upload=1LYSIO7ZL1QE tag=46D9XPB2DH0Y type=write size=2213945 2026-02-24T22:10:45.926 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.925+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~X5Y5WSB9YMX2.0 size=1048576 2026-02-24T22:10:45.927 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.925+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~X5Y5WSB9YMX2.1 size=1048576 2026-02-24T22:10:45.928 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.925+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~X5Y5WSB9YMX2.2 size=284816 2026-02-24T22:10:45.928 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.925+0000 7f7e64854980 1 > prepared multipart upload key=Z0OXREAUJVA8 upload=X5Y5WSB9YMX2 tag=JLX13GGKDV8K type=write size=3430544 2026-02-24T22:10:45.928 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.925+0000 7f7e64854980 1 < completed delete operation key=EB04ETNYM087 tag=51BRLRV2GP7N type=del 2026-02-24T22:10:45.930 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.929+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~1U2L94RW440F.0 size=1048576 2026-02-24T22:10:45.930 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.929+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~1U2L94RW440F.1 size=1048576 2026-02-24T22:10:45.931 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.929+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~1U2L94RW440F.2 size=215700 2026-02-24T22:10:45.931 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.929+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=1U2L94RW440F tag=VIEEMWP9DCM2 type=write size=3361428 2026-02-24T22:10:45.932 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.929+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~47CSA2DFL5V7.0 size=1048576 2026-02-24T22:10:45.932 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.929+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~47CSA2DFL5V7.1 size=959526 2026-02-24T22:10:45.934 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.933+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=47CSA2DFL5V7 tag=WIDHPMPBM7PO type=write size=3056678 2026-02-24T22:10:45.934 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.933+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=SYLZ7TLA36WT type=del size=2657267 2026-02-24T22:10:45.935 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.933+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~HLGC50I6CRTM.0 size=1048576 2026-02-24T22:10:45.936 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.933+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~HLGC50I6CRTM.1 size=587672 2026-02-24T22:10:45.937 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.933+0000 7f7e64854980 1 < completed multipart upload key=EJCX06TE5RTZ upload=HLGC50I6CRTM tag=111ZBS9OM7AY type=write size=2684824 2026-02-24T22:10:45.938 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.937+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=GS1P06FWRM42 type=write size=379932 2026-02-24T22:10:45.938 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.937+0000 7f7e64854980 1 < completed delete operation key=L9NEDH44BIGA tag=9TUXTO56V8Q9 type=del 2026-02-24T22:10:45.939 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.937+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=9E2J0S4K0Z7B type=write size=274113 2026-02-24T22:10:45.940 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.937+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~JJHS128UUHZ3.0 size=1048576 2026-02-24T22:10:45.941 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.937+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~JJHS128UUHZ3.1 size=1048576 2026-02-24T22:10:45.941 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.941+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~JJHS128UUHZ3.2 size=81911 2026-02-24T22:10:45.943 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.941+0000 7f7e64854980 1 < completed multipart upload key=EJCX06TE5RTZ upload=JJHS128UUHZ3 tag=YZVPU8T464KJ type=write size=3227639 2026-02-24T22:10:45.944 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.941+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~YDX9CZ0GDHHW.0 size=601552 2026-02-24T22:10:45.944 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.941+0000 7f7e64854980 1 > prepared multipart upload key=Z0OXREAUJVA8 upload=YDX9CZ0GDHHW tag=5ITLJT4XBBHC type=write size=1650128 2026-02-24T22:10:45.945 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.941+0000 7f7e64854980 1 < completed write operation key=3900SFJK1YQU tag=SV78V28IFBFC type=write size=758087 2026-02-24T22:10:45.946 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.945+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=8LMT11LVXADB type=del size=3406675 2026-02-24T22:10:45.947 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.945+0000 7f7e64854980 1 < suggested remove operation key=Q7BP8FXTARDW tag=1EOLG0DWJ5NC type=del 2026-02-24T22:10:45.948 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.945+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=YCVPPRD580WZ type=del size=971176 2026-02-24T22:10:45.949 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.945+0000 7f7e64854980 1 < completed delete operation key=0AXS7DY23UZU tag=Q5AKJZDLJ4M3 type=del 2026-02-24T22:10:45.950 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.949+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~3RNL6SHVO93M.0 size=1048576 2026-02-24T22:10:45.951 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.949+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~3RNL6SHVO93M.1 size=281859 2026-02-24T22:10:45.951 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.949+0000 7f7e64854980 1 > prepared multipart upload key=GH7723GNI2WM upload=3RNL6SHVO93M tag=QG9QH9QQZ49O type=write size=2379011 2026-02-24T22:10:45.951 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.949+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=P8BLYM3GQA4Z type=del 2026-02-24T22:10:45.952 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.949+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=4DR553ZC6ZF8 type=del size=2381471 2026-02-24T22:10:45.954 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.953+0000 7f7e64854980 1 < suggested update operation key=Y8GMOUA977H3 tag=AY8ZNBPWS0HG type=del size=2491418 2026-02-24T22:10:45.955 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.953+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=ZG8DG1LB6DSO type=write size=160570 2026-02-24T22:10:45.957 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.953+0000 7f7e64854980 1 < suggested update operation key=EB04ETNYM087 tag=JLYVNWFY040M type=del size=770370 2026-02-24T22:10:45.958 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.957+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=TSS2VVUJ11NU type=del size=869401 2026-02-24T22:10:45.959 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.957+0000 7f7e64854980 -1 < failed to complete operation key=Y8GMOUA977H3 tag=YTPP7SN33JRW type=del size=1752048: (22) Invalid argument 2026-02-24T22:10:45.960 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.957+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~1OA6LKRPHBME.0 size=1048576 2026-02-24T22:10:45.961 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.957+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~1OA6LKRPHBME.1 size=1048576 2026-02-24T22:10:45.961 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.961+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~1OA6LKRPHBME.2 size=788078 2026-02-24T22:10:45.961 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.961+0000 7f7e64854980 1 > prepared multipart upload key=Q7BP8FXTARDW upload=1OA6LKRPHBME tag=R2TGO51XI6UX type=write size=3933806 2026-02-24T22:10:45.962 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.961+0000 7f7e64854980 1 < completed write operation key=0B2V0MHWSOC9 tag=JRE03LSJFP4W type=write size=852061 2026-02-24T22:10:45.963 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.961+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~AGEEZG3MNA61.0 size=1048576 2026-02-24T22:10:45.964 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.961+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~AGEEZG3MNA61.1 size=481872 2026-02-24T22:10:45.964 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.961+0000 7f7e64854980 1 > prepared multipart upload key=A6PH77Q78VEN upload=AGEEZG3MNA61 tag=CWYDX4HT1IK4 type=write size=2579024 2026-02-24T22:10:45.965 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.965+0000 7f7e64854980 1 < suggested remove operation key=78GUTC5THI2P tag=ZYATKJAG4QKR type=del 2026-02-24T22:10:45.966 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.965+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=2TDU7RRHIICZ type=del size=2140791 2026-02-24T22:10:45.967 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.965+0000 7f7e64854980 1 < completed delete operation key=NL60Q5QOSP8E tag=E4B16JA01OV1 type=del 2026-02-24T22:10:45.968 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.965+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=9OJWJ9ARAABK type=del size=3913115 2026-02-24T22:10:45.969 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.965+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~X5Y5WSB9YMX2.0 size=1048576 2026-02-24T22:10:45.969 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.965+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~X5Y5WSB9YMX2.1 size=1048576 2026-02-24T22:10:45.970 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.969+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~X5Y5WSB9YMX2.2 size=284816 2026-02-24T22:10:45.971 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.969+0000 7f7e64854980 1 < completed multipart upload key=Z0OXREAUJVA8 upload=X5Y5WSB9YMX2 tag=JLX13GGKDV8K type=write size=3430544 2026-02-24T22:10:45.972 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.969+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~O2WLD5FT0J6L.0 size=1048576 2026-02-24T22:10:45.973 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.969+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~O2WLD5FT0J6L.1 size=399315 2026-02-24T22:10:45.973 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.969+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=O2WLD5FT0J6L tag=4M2U61SVAAV3 type=write size=2496467 2026-02-24T22:10:45.973 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.969+0000 7f7e64854980 1 < canceled part key=_multipart_2KKQKHXO7K3K.2~1U2L94RW440F.0 size=1048576 2026-02-24T22:10:45.974 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.973+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~1U2L94RW440F.1 size=1048576 2026-02-24T22:10:45.974 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.973+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~1U2L94RW440F.2 size=215700 2026-02-24T22:10:45.976 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.973+0000 7f7e64854980 1 < canceled multipart upload key=2KKQKHXO7K3K upload=1U2L94RW440F tag=VIEEMWP9DCM2 type=write size=3361428 2026-02-24T22:10:45.977 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.973+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~LL0R3ZFLWOLB.0 size=1048576 2026-02-24T22:10:45.978 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.977+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~LL0R3ZFLWOLB.1 size=1048576 2026-02-24T22:10:45.978 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.977+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~LL0R3ZFLWOLB.2 size=313776 2026-02-24T22:10:45.978 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.977+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=LL0R3ZFLWOLB tag=S2KIOVDPXV14 type=write size=3459504 2026-02-24T22:10:45.979 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.977+0000 7f7e64854980 1 < completed delete operation key=L9NEDH44BIGA tag=SYLZ7TLA36WT type=del 2026-02-24T22:10:45.980 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.977+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=8E8JXTAXUZXP type=write size=455541 2026-02-24T22:10:45.980 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.977+0000 7f7e64854980 1 < completed write operation key=SGQYQ2TIVJPS tag=GS1P06FWRM42 type=write size=379932 2026-02-24T22:10:45.982 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.981+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~UMFWO3IVCP7H.0 size=1048576 2026-02-24T22:10:45.982 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.981+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~UMFWO3IVCP7H.1 size=1048576 2026-02-24T22:10:45.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.981+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~UMFWO3IVCP7H.2 size=860851 2026-02-24T22:10:45.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.981+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=UMFWO3IVCP7H tag=EZI1L38AX6P7 type=write size=4006579 2026-02-24T22:10:45.984 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.981+0000 7f7e64854980 1 < completed write operation key=SGQYQ2TIVJPS tag=9E2J0S4K0Z7B type=write size=274113 2026-02-24T22:10:45.985 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.981+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~34MMENUGO9O6.0 size=1048576 2026-02-24T22:10:45.986 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.985+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~34MMENUGO9O6.1 size=1007347 2026-02-24T22:10:45.986 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.985+0000 7f7e64854980 1 > prepared multipart upload key=HKOQRSF6XEXB upload=34MMENUGO9O6 tag=85CO0LU2ONT6 type=write size=3104499 2026-02-24T22:10:45.987 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.985+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~YDX9CZ0GDHHW.0 size=601552 2026-02-24T22:10:45.988 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.985+0000 7f7e64854980 1 < completed multipart upload key=Z0OXREAUJVA8 upload=YDX9CZ0GDHHW tag=5ITLJT4XBBHC type=write size=1650128 2026-02-24T22:10:45.989 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.985+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=P3KKIWUM2D6B type=write size=699749 2026-02-24T22:10:45.989 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.989+0000 7f7e64854980 1 < completed delete operation key=PKRHWM69GBJI tag=8LMT11LVXADB type=del 2026-02-24T22:10:45.990 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.989+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=TOF3U8AXQRTK type=del size=3002944 2026-02-24T22:10:45.991 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.989+0000 7f7e64854980 1 < completed delete operation key=A6PH77Q78VEN tag=YCVPPRD580WZ type=del 2026-02-24T22:10:45.992 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.989+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=ATZV001PFW3I type=del size=691080 2026-02-24T22:10:45.992 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.989+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~3RNL6SHVO93M.0 size=1048576 2026-02-24T22:10:45.993 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.989+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~3RNL6SHVO93M.1 size=281859 2026-02-24T22:10:45.994 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.993+0000 7f7e64854980 1 < completed multipart upload key=GH7723GNI2WM upload=3RNL6SHVO93M tag=QG9QH9QQZ49O type=write size=2379011 2026-02-24T22:10:45.996 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.993+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~1T25M3ZOUJXA.0 size=1048576 2026-02-24T22:10:45.996 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.993+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~1T25M3ZOUJXA.1 size=883062 2026-02-24T22:10:45.996 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.993+0000 7f7e64854980 1 > prepared multipart upload key=Z0OXREAUJVA8 upload=1T25M3ZOUJXA tag=FA81GL90FT5X type=write size=2980214 2026-02-24T22:10:45.997 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.993+0000 7f7e64854980 1 < completed delete operation key=D0MKGNRDW6L9 tag=4DR553ZC6ZF8 type=del 2026-02-24T22:10:45.999 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.997+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~CTPZ4Q4OB7M7.0 size=763154 2026-02-24T22:10:45.999 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.997+0000 7f7e64854980 1 > prepared multipart upload key=0B2V0MHWSOC9 upload=CTPZ4Q4OB7M7 tag=594MPUQMH604 type=write size=1811730 2026-02-24T22:10:45.999 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.997+0000 7f7e64854980 1 < completed write operation key=TXRUQQ456DD8 tag=ZG8DG1LB6DSO type=write size=160570 2026-02-24T22:10:46.000 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.997+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=KIBXP2278K2Y type=del size=1015079 2026-02-24T22:10:46.001 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:45.997+0000 7f7e64854980 1 < completed delete operation key=D0MKGNRDW6L9 tag=TSS2VVUJ11NU type=del 2026-02-24T22:10:46.002 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.001+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=QY6KLO2T9DWS type=del size=4123260 2026-02-24T22:10:46.002 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.001+0000 7f7e64854980 1 < canceled part key=_multipart_Q7BP8FXTARDW.2~1OA6LKRPHBME.0 size=1048576 2026-02-24T22:10:46.002 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.001+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~1OA6LKRPHBME.1 size=1048576 2026-02-24T22:10:46.003 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.001+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~1OA6LKRPHBME.2 size=788078 2026-02-24T22:10:46.004 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.001+0000 7f7e64854980 1 < canceled multipart upload key=Q7BP8FXTARDW upload=1OA6LKRPHBME tag=R2TGO51XI6UX type=write size=3933806 2026-02-24T22:10:46.005 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.001+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=WM5H8Q2541KM type=del size=1359056 2026-02-24T22:10:46.006 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.005+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~AGEEZG3MNA61.0 size=1048576 2026-02-24T22:10:46.006 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.005+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~AGEEZG3MNA61.1 size=481872 2026-02-24T22:10:46.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.005+0000 7f7e64854980 1 < completed multipart upload key=A6PH77Q78VEN upload=AGEEZG3MNA61 tag=CWYDX4HT1IK4 type=write size=2579024 2026-02-24T22:10:46.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.005+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=DE5WS8BKDWO2 type=del size=743647 2026-02-24T22:10:46.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.009+0000 7f7e64854980 1 < canceled operation key=EB04ETNYM087 tag=2TDU7RRHIICZ type=del size=2140791 2026-02-24T22:10:46.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.009+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=IUY3Z6X6BE9U type=del size=976810 2026-02-24T22:10:46.011 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.009+0000 7f7e64854980 1 < completed delete operation key=0AXS7DY23UZU tag=9OJWJ9ARAABK type=del 2026-02-24T22:10:46.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.009+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~RCRC3YHPN2Z3.0 size=133008 2026-02-24T22:10:46.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.009+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=RCRC3YHPN2Z3 tag=1V716F9ICBOR type=write size=1181584 2026-02-24T22:10:46.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.009+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~O2WLD5FT0J6L.0 size=1048576 2026-02-24T22:10:46.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.013+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~O2WLD5FT0J6L.1 size=399315 2026-02-24T22:10:46.015 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.013+0000 7f7e64854980 1 < completed multipart upload key=TXRUQQ456DD8 upload=O2WLD5FT0J6L tag=4M2U61SVAAV3 type=write size=2496467 2026-02-24T22:10:46.016 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.013+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=DT7OC0W3549M type=del size=1030966 2026-02-24T22:10:46.016 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.013+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~LL0R3ZFLWOLB.0 size=1048576 2026-02-24T22:10:46.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.013+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~LL0R3ZFLWOLB.1 size=1048576 2026-02-24T22:10:46.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.017+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~LL0R3ZFLWOLB.2 size=313776 2026-02-24T22:10:46.019 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.017+0000 7f7e64854980 1 < completed multipart upload key=CI8DA17DX8LU upload=LL0R3ZFLWOLB tag=S2KIOVDPXV14 type=write size=3459504 2026-02-24T22:10:46.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.017+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=ESNBWS3BWVJF type=del size=2457468 2026-02-24T22:10:46.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.017+0000 7f7e64854980 1 < canceled operation key=78GUTC5THI2P tag=8E8JXTAXUZXP type=write size=455541 2026-02-24T22:10:46.021 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.021+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=6C767SJCR17H type=del size=1078051 2026-02-24T22:10:46.022 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.021+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~UMFWO3IVCP7H.0 size=1048576 2026-02-24T22:10:46.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.021+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~UMFWO3IVCP7H.1 size=1048576 2026-02-24T22:10:46.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.021+0000 7f7e64854980 1 < canceled part key=_multipart_CI8DA17DX8LU.2~UMFWO3IVCP7H.2 size=860851 2026-02-24T22:10:46.024 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.021+0000 7f7e64854980 1 < canceled multipart upload key=CI8DA17DX8LU upload=UMFWO3IVCP7H tag=EZI1L38AX6P7 type=write size=4006579 2026-02-24T22:10:46.025 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.025+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~0MQM2SEMGT8Z.0 size=1048576 2026-02-24T22:10:46.027 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.025+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~0MQM2SEMGT8Z.1 size=530704 2026-02-24T22:10:46.027 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.025+0000 7f7e64854980 1 > prepared multipart upload key=GH7723GNI2WM upload=0MQM2SEMGT8Z tag=ABBX55CKTKFN type=write size=2627856 2026-02-24T22:10:46.028 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.025+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~34MMENUGO9O6.0 size=1048576 2026-02-24T22:10:46.028 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.025+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~34MMENUGO9O6.1 size=1007347 2026-02-24T22:10:46.030 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.029+0000 7f7e64854980 1 < completed multipart upload key=HKOQRSF6XEXB upload=34MMENUGO9O6 tag=85CO0LU2ONT6 type=write size=3104499 2026-02-24T22:10:46.031 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.029+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~800G09A7T27D.0 size=1048576 2026-02-24T22:10:46.032 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.029+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~800G09A7T27D.1 size=1048576 2026-02-24T22:10:46.032 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.029+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~800G09A7T27D.2 size=47579 2026-02-24T22:10:46.032 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.029+0000 7f7e64854980 1 > prepared multipart upload key=HKOQRSF6XEXB upload=800G09A7T27D tag=BXLBCHJJUIN0 type=write size=3193307 2026-02-24T22:10:46.033 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.029+0000 7f7e64854980 1 < completed write operation key=HKOQRSF6XEXB tag=P3KKIWUM2D6B type=write size=699749 2026-02-24T22:10:46.034 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.033+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~2152IP0XDRXY.0 size=1048576 2026-02-24T22:10:46.035 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.033+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~2152IP0XDRXY.1 size=1048576 2026-02-24T22:10:46.035 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.033+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~2152IP0XDRXY.2 size=68530 2026-02-24T22:10:46.036 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.033+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=2152IP0XDRXY tag=O01P2QZ8ZZLE type=write size=3214258 2026-02-24T22:10:46.036 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.033+0000 7f7e64854980 1 < completed delete operation key=SGQYQ2TIVJPS tag=TOF3U8AXQRTK type=del 2026-02-24T22:10:46.037 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.033+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=7YER2QG6WMW3 type=del size=2849248 2026-02-24T22:10:46.038 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.037+0000 7f7e64854980 1 < completed delete operation key=PKRHWM69GBJI tag=ATZV001PFW3I type=del 2026-02-24T22:10:46.038 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.037+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=AMYWE6UU8KSS type=write size=558590 2026-02-24T22:10:46.039 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.037+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~1T25M3ZOUJXA.0 size=1048576 2026-02-24T22:10:46.040 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.037+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~1T25M3ZOUJXA.1 size=883062 2026-02-24T22:10:46.041 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.037+0000 7f7e64854980 1 < completed multipart upload key=Z0OXREAUJVA8 upload=1T25M3ZOUJXA tag=FA81GL90FT5X type=write size=2980214 2026-02-24T22:10:46.042 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.041+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=5KS7RERL2YID type=del size=450365 2026-02-24T22:10:46.043 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.041+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~CTPZ4Q4OB7M7.0 size=763154 2026-02-24T22:10:46.044 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.041+0000 7f7e64854980 1 < completed multipart upload key=0B2V0MHWSOC9 upload=CTPZ4Q4OB7M7 tag=594MPUQMH604 type=write size=1811730 2026-02-24T22:10:46.045 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.041+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=CLP1Y9LMSO7N type=del size=3420949 2026-02-24T22:10:46.045 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.045+0000 7f7e64854980 1 < completed delete operation key=N3ZUNSI5K8HS tag=KIBXP2278K2Y type=del 2026-02-24T22:10:46.046 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.045+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=86W34ZNRY2C8 type=del size=3443905 2026-02-24T22:10:46.048 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.045+0000 7f7e64854980 1 < suggested update operation key=N3ZUNSI5K8HS tag=QY6KLO2T9DWS type=del size=4123260 2026-02-24T22:10:46.049 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.045+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=IGV6OGVUK49N type=del size=2296920 2026-02-24T22:10:46.049 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.049+0000 7f7e64854980 1 < canceled operation key=NL60Q5QOSP8E tag=WM5H8Q2541KM type=del size=1359056 2026-02-24T22:10:46.050 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.049+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=KHVD7592O4QR type=del size=559519 2026-02-24T22:10:46.051 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.049+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=DE5WS8BKDWO2 type=del 2026-02-24T22:10:46.052 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.049+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=Z5X3Z5AL43C7 type=del size=1231694 2026-02-24T22:10:46.052 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.049+0000 7f7e64854980 1 < completed delete operation key=Q7BP8FXTARDW tag=IUY3Z6X6BE9U type=del 2026-02-24T22:10:46.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.049+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=WGN68L1ZBAHS type=del size=891215 2026-02-24T22:10:46.054 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.053+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~RCRC3YHPN2Z3.0 size=133008 2026-02-24T22:10:46.055 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.053+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=RCRC3YHPN2Z3 tag=1V716F9ICBOR type=write size=1181584 2026-02-24T22:10:46.056 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.053+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=P7PB3G7VJ0G3 type=del size=576656 2026-02-24T22:10:46.058 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.057+0000 7f7e64854980 1 < suggested remove operation key=A6PH77Q78VEN tag=DT7OC0W3549M type=del 2026-02-24T22:10:46.058 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.057+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=B3CF2QJVU1U5 type=del size=2028405 2026-02-24T22:10:46.059 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.057+0000 7f7e64854980 -1 < no bi entry to suggest for operation key=A6PH77Q78VEN tag=ESNBWS3BWVJF type=del size=2457468: (2) No such file or directory 2026-02-24T22:10:46.060 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.057+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~59IWAO411Y1E.0 size=1048576 2026-02-24T22:10:46.061 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.057+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~59IWAO411Y1E.1 size=689825 2026-02-24T22:10:46.061 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.057+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=59IWAO411Y1E tag=PUDAJPAB6RE8 type=write size=2786977 2026-02-24T22:10:46.061 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.061+0000 7f7e64854980 1 < completed delete operation key=TF5B43G7KCYT tag=6C767SJCR17H type=del 2026-02-24T22:10:46.063 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.061+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~8JTA3A4L3GFO.0 size=182982 2026-02-24T22:10:46.063 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.061+0000 7f7e64854980 1 > prepared multipart upload key=Q7BP8FXTARDW upload=8JTA3A4L3GFO tag=TE30F2JFXMDB type=write size=1231558 2026-02-24T22:10:46.064 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.061+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~0MQM2SEMGT8Z.0 size=1048576 2026-02-24T22:10:46.064 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.061+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~0MQM2SEMGT8Z.1 size=530704 2026-02-24T22:10:46.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.065+0000 7f7e64854980 1 < completed multipart upload key=GH7723GNI2WM upload=0MQM2SEMGT8Z tag=ABBX55CKTKFN type=write size=2627856 2026-02-24T22:10:46.067 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.065+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~7JUOR714R7PJ.0 size=1048576 2026-02-24T22:10:46.067 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.065+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~7JUOR714R7PJ.1 size=1048576 2026-02-24T22:10:46.068 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.065+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~7JUOR714R7PJ.2 size=327962 2026-02-24T22:10:46.068 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.065+0000 7f7e64854980 1 > prepared multipart upload key=4Y3974PYYT78 upload=7JUOR714R7PJ tag=ST3JZE3F3S11 type=write size=3473690 2026-02-24T22:10:46.069 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.065+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~800G09A7T27D.0 size=1048576 2026-02-24T22:10:46.069 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.069+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~800G09A7T27D.1 size=1048576 2026-02-24T22:10:46.069 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.069+0000 7f7e64854980 1 < canceled part key=_multipart_HKOQRSF6XEXB.2~800G09A7T27D.2 size=47579 2026-02-24T22:10:46.071 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.069+0000 7f7e64854980 1 < canceled multipart upload key=HKOQRSF6XEXB upload=800G09A7T27D tag=BXLBCHJJUIN0 type=write size=3193307 2026-02-24T22:10:46.071 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.069+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=37IX3F9BZ3L6 type=del size=3850516 2026-02-24T22:10:46.072 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.069+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~2152IP0XDRXY.0 size=1048576 2026-02-24T22:10:46.073 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.069+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~2152IP0XDRXY.1 size=1048576 2026-02-24T22:10:46.073 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.073+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~2152IP0XDRXY.2 size=68530 2026-02-24T22:10:46.075 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.073+0000 7f7e64854980 1 < completed multipart upload key=ZESKGBWZXM6O upload=2152IP0XDRXY tag=O01P2QZ8ZZLE type=write size=3214258 2026-02-24T22:10:46.076 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.073+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~AWP0QPS7SS5V.0 size=1048576 2026-02-24T22:10:46.077 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.073+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~AWP0QPS7SS5V.1 size=1048576 2026-02-24T22:10:46.077 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.073+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~AWP0QPS7SS5V.2 size=630796 2026-02-24T22:10:46.077 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.073+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=AWP0QPS7SS5V tag=T6XLGRNRSAPH type=write size=3776524 2026-02-24T22:10:46.078 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.077+0000 7f7e64854980 1 < completed delete operation key=GH7723GNI2WM tag=7YER2QG6WMW3 type=del 2026-02-24T22:10:46.079 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.077+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~37XMY5NJHYR3.0 size=457889 2026-02-24T22:10:46.079 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.077+0000 7f7e64854980 1 > prepared multipart upload key=0B2V0MHWSOC9 upload=37XMY5NJHYR3 tag=SGO2KPQSN833 type=write size=1506465 2026-02-24T22:10:46.080 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.077+0000 7f7e64854980 1 < completed write operation key=L9NEDH44BIGA tag=AMYWE6UU8KSS type=write size=558590 2026-02-24T22:10:46.081 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.077+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=LSYJ6QT5TA0D type=del size=296820 2026-02-24T22:10:46.082 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.081+0000 7f7e64854980 1 < canceled operation key=ZESKGBWZXM6O tag=5KS7RERL2YID type=del size=450365 2026-02-24T22:10:46.082 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.081+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=I8238GUZ04H1 type=del size=2966131 2026-02-24T22:10:46.083 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.081+0000 7f7e64854980 -1 < failed to complete operation key=A6PH77Q78VEN tag=CLP1Y9LMSO7N type=del size=3420949: (22) Invalid argument 2026-02-24T22:10:46.084 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.081+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~7WJSRNX8W6CJ.0 size=1048576 2026-02-24T22:10:46.085 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.081+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~7WJSRNX8W6CJ.1 size=393772 2026-02-24T22:10:46.085 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.081+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=7WJSRNX8W6CJ tag=7SWJ8NSJ43N0 type=write size=2490924 2026-02-24T22:10:46.086 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.085+0000 7f7e64854980 -1 < failed to complete operation key=N3ZUNSI5K8HS tag=86W34ZNRY2C8 type=del size=3443905: (22) Invalid argument 2026-02-24T22:10:46.087 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.085+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~RX31CAPK8CSN.0 size=1048576 2026-02-24T22:10:46.088 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.085+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~RX31CAPK8CSN.1 size=1048576 2026-02-24T22:10:46.088 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.085+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~RX31CAPK8CSN.2 size=590638 2026-02-24T22:10:46.089 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.085+0000 7f7e64854980 1 > prepared multipart upload key=SGQYQ2TIVJPS upload=RX31CAPK8CSN tag=NJ5LVEYVCT7E type=write size=3736366 2026-02-24T22:10:46.089 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.089+0000 7f7e64854980 1 < completed delete operation key=SGQYQ2TIVJPS tag=IGV6OGVUK49N type=del 2026-02-24T22:10:46.091 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.089+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~1EOJIJ8GU72R.0 size=566671 2026-02-24T22:10:46.091 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.089+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=1EOJIJ8GU72R tag=CSP2KX766AZB type=write size=1615247 2026-02-24T22:10:46.091 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.089+0000 7f7e64854980 1 < completed delete operation key=SGQYQ2TIVJPS tag=KHVD7592O4QR type=del 2026-02-24T22:10:46.093 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.089+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~PWLM33L6O8OC.0 size=27748 2026-02-24T22:10:46.093 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.089+0000 7f7e64854980 1 > prepared multipart upload key=GH7723GNI2WM upload=PWLM33L6O8OC tag=6GDJKQI486EC type=write size=1076324 2026-02-24T22:10:46.094 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.093+0000 7f7e64854980 1 < completed delete operation key=TF5B43G7KCYT tag=Z5X3Z5AL43C7 type=del 2026-02-24T22:10:46.095 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.093+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~FSGX4NZAP9AG.0 size=113241 2026-02-24T22:10:46.095 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.093+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=FSGX4NZAP9AG tag=3R4968982AVQ type=write size=1161817 2026-02-24T22:10:46.096 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.093+0000 7f7e64854980 1 < completed delete operation key=SGQYQ2TIVJPS tag=WGN68L1ZBAHS type=del 2026-02-24T22:10:46.098 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.097+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~25S5IUI186IH.0 size=1048576 2026-02-24T22:10:46.099 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.097+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~25S5IUI186IH.1 size=1048576 2026-02-24T22:10:46.099 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.097+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~25S5IUI186IH.2 size=119923 2026-02-24T22:10:46.099 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.097+0000 7f7e64854980 1 > prepared multipart upload key=PKRHWM69GBJI upload=25S5IUI186IH tag=KOUAXW0SXIMN type=write size=3265651 2026-02-24T22:10:46.101 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.097+0000 7f7e64854980 1 < suggested remove operation key=EB04ETNYM087 tag=P7PB3G7VJ0G3 type=del 2026-02-24T22:10:46.102 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.101+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=3UODRXARN2AR type=del size=1040893 2026-02-24T22:10:46.102 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.101+0000 7f7e64854980 1 < completed delete operation key=36MKR2OZN7K9 tag=B3CF2QJVU1U5 type=del 2026-02-24T22:10:46.104 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.101+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~4RQ2FZ6Z3PWL.0 size=841607 2026-02-24T22:10:46.104 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.101+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=4RQ2FZ6Z3PWL tag=QS5QPZN8NE8M type=write size=1890183 2026-02-24T22:10:46.105 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.101+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~59IWAO411Y1E.0 size=1048576 2026-02-24T22:10:46.105 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.101+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~59IWAO411Y1E.1 size=689825 2026-02-24T22:10:46.106 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.105+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=59IWAO411Y1E tag=PUDAJPAB6RE8 type=write size=2786977 2026-02-24T22:10:46.107 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.105+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=CXJZFEP7TJR6 type=del size=2583447 2026-02-24T22:10:46.108 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.105+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~8JTA3A4L3GFO.0 size=182982 2026-02-24T22:10:46.109 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.105+0000 7f7e64854980 1 < completed multipart upload key=Q7BP8FXTARDW upload=8JTA3A4L3GFO tag=TE30F2JFXMDB type=write size=1231558 2026-02-24T22:10:46.110 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.109+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=V7F70P23UWCL type=del size=3020153 2026-02-24T22:10:46.111 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.109+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~7JUOR714R7PJ.0 size=1048576 2026-02-24T22:10:46.111 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.109+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~7JUOR714R7PJ.1 size=1048576 2026-02-24T22:10:46.112 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.109+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~7JUOR714R7PJ.2 size=327962 2026-02-24T22:10:46.113 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.109+0000 7f7e64854980 1 < completed multipart upload key=4Y3974PYYT78 upload=7JUOR714R7PJ tag=ST3JZE3F3S11 type=write size=3473690 2026-02-24T22:10:46.114 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.113+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=IKG4LBAA1M6G type=del size=3023464 2026-02-24T22:10:46.115 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.113+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=37IX3F9BZ3L6 type=del 2026-02-24T22:10:46.116 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.113+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~LF0S9HPF6F87.0 size=1048576 2026-02-24T22:10:46.116 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.113+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~LF0S9HPF6F87.1 size=413930 2026-02-24T22:10:46.117 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.113+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=LF0S9HPF6F87 tag=76QAMQWAU6B1 type=write size=2511082 2026-02-24T22:10:46.117 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.117+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~AWP0QPS7SS5V.0 size=1048576 2026-02-24T22:10:46.118 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.117+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~AWP0QPS7SS5V.1 size=1048576 2026-02-24T22:10:46.118 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.117+0000 7f7e64854980 1 < canceled part key=_multipart_0AXS7DY23UZU.2~AWP0QPS7SS5V.2 size=630796 2026-02-24T22:10:46.119 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.117+0000 7f7e64854980 1 < canceled multipart upload key=0AXS7DY23UZU upload=AWP0QPS7SS5V tag=T6XLGRNRSAPH type=write size=3776524 2026-02-24T22:10:46.121 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.117+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~OV6YY0Y5XDHJ.0 size=966590 2026-02-24T22:10:46.121 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.117+0000 7f7e64854980 1 > prepared multipart upload key=UQLUT9ZAI61V upload=OV6YY0Y5XDHJ tag=OK7VG31ZF5YD type=write size=2015166 2026-02-24T22:10:46.121 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.121+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~37XMY5NJHYR3.0 size=457889 2026-02-24T22:10:46.122 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.121+0000 7f7e64854980 1 < completed multipart upload key=0B2V0MHWSOC9 upload=37XMY5NJHYR3 tag=SGO2KPQSN833 type=write size=1506465 2026-02-24T22:10:46.124 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.121+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~HYKWBHPAMYKG.0 size=468702 2026-02-24T22:10:46.124 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.121+0000 7f7e64854980 1 > prepared multipart upload key=3900SFJK1YQU upload=HYKWBHPAMYKG tag=EE44TSYCVCEB type=write size=1517278 2026-02-24T22:10:46.125 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.121+0000 7f7e64854980 1 < canceled operation key=PKRHWM69GBJI tag=LSYJ6QT5TA0D type=del size=296820 2026-02-24T22:10:46.125 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.125+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=5EIVCL1XYF5R type=del size=1410452 2026-02-24T22:10:46.126 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.125+0000 7f7e64854980 1 < completed delete operation key=0AXS7DY23UZU tag=I8238GUZ04H1 type=del 2026-02-24T22:10:46.127 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.125+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=NFVY669E1U8J type=del size=2761365 2026-02-24T22:10:46.128 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.125+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~7WJSRNX8W6CJ.0 size=1048576 2026-02-24T22:10:46.128 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.125+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~7WJSRNX8W6CJ.1 size=393772 2026-02-24T22:10:46.129 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.129+0000 7f7e64854980 1 < completed multipart upload key=HOHKR92SN25M upload=7WJSRNX8W6CJ tag=7SWJ8NSJ43N0 type=write size=2490924 2026-02-24T22:10:46.130 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.129+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=35ATR5VFH0V8 type=write size=648743 2026-02-24T22:10:46.131 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.129+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~RX31CAPK8CSN.0 size=1048576 2026-02-24T22:10:46.132 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.129+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~RX31CAPK8CSN.1 size=1048576 2026-02-24T22:10:46.132 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.129+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~RX31CAPK8CSN.2 size=590638 2026-02-24T22:10:46.133 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.133+0000 7f7e64854980 1 < completed multipart upload key=SGQYQ2TIVJPS upload=RX31CAPK8CSN tag=NJ5LVEYVCT7E type=write size=3736366 2026-02-24T22:10:46.134 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.133+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=QFNDJ43XOG93 type=del size=2451037 2026-02-24T22:10:46.135 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.133+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~1EOJIJ8GU72R.0 size=566671 2026-02-24T22:10:46.136 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.133+0000 7f7e64854980 1 < completed multipart upload key=D0MKGNRDW6L9 upload=1EOJIJ8GU72R tag=CSP2KX766AZB type=write size=1615247 2026-02-24T22:10:46.137 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.133+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=1UVARQZUGSCU type=del size=2165383 2026-02-24T22:10:46.138 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.137+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~PWLM33L6O8OC.0 size=27748 2026-02-24T22:10:46.140 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.137+0000 7f7e64854980 1 < completed multipart upload key=GH7723GNI2WM upload=PWLM33L6O8OC tag=6GDJKQI486EC type=write size=1076324 2026-02-24T22:10:46.140 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.137+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=55Y7SPDJSVJB type=del size=2222176 2026-02-24T22:10:46.141 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.137+0000 7f7e64854980 1 < canceled part key=_multipart_0AXS7DY23UZU.2~FSGX4NZAP9AG.0 size=113241 2026-02-24T22:10:46.142 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.141+0000 7f7e64854980 1 < canceled multipart upload key=0AXS7DY23UZU upload=FSGX4NZAP9AG tag=3R4968982AVQ type=write size=1161817 2026-02-24T22:10:46.142 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.141+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=PP5LUVSPTC38 type=del size=2056083 2026-02-24T22:10:46.143 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.141+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~25S5IUI186IH.0 size=1048576 2026-02-24T22:10:46.144 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.141+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~25S5IUI186IH.1 size=1048576 2026-02-24T22:10:46.144 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.141+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~25S5IUI186IH.2 size=119923 2026-02-24T22:10:46.146 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.145+0000 7f7e64854980 1 < completed multipart upload key=PKRHWM69GBJI upload=25S5IUI186IH tag=KOUAXW0SXIMN type=write size=3265651 2026-02-24T22:10:46.146 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.145+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=ZMNS2MRYK7OI type=del size=101018 2026-02-24T22:10:46.147 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.145+0000 7f7e64854980 1 < completed delete operation key=0B2V0MHWSOC9 tag=3UODRXARN2AR type=del 2026-02-24T22:10:46.148 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.145+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~PMFNE5678WM2.0 size=487483 2026-02-24T22:10:46.149 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.145+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=PMFNE5678WM2 tag=BK6L27T9QNUT type=write size=1536059 2026-02-24T22:10:46.149 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.145+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~4RQ2FZ6Z3PWL.0 size=841607 2026-02-24T22:10:46.150 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.149+0000 7f7e64854980 1 < completed multipart upload key=TXRUQQ456DD8 upload=4RQ2FZ6Z3PWL tag=QS5QPZN8NE8M type=write size=1890183 2026-02-24T22:10:46.151 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.149+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=QTGW6VA61I7B type=del size=56187 2026-02-24T22:10:46.152 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.149+0000 7f7e64854980 1 < completed delete operation key=UQLUT9ZAI61V tag=CXJZFEP7TJR6 type=del 2026-02-24T22:10:46.153 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.149+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~9BXCTGTGTW1K.0 size=523006 2026-02-24T22:10:46.153 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.149+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=9BXCTGTGTW1K tag=I0CBNDX7WNGX type=write size=1571582 2026-02-24T22:10:46.154 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.153+0000 7f7e64854980 1 < suggested update operation key=EB04ETNYM087 tag=V7F70P23UWCL type=del size=3020153 2026-02-24T22:10:46.155 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.153+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=TEMX06132SQU type=del size=4100050 2026-02-24T22:10:46.157 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.153+0000 7f7e64854980 1 < suggested remove operation key=TF5B43G7KCYT tag=IKG4LBAA1M6G type=del 2026-02-24T22:10:46.157 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.157+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=DOV9WU4L696T type=del size=2783148 2026-02-24T22:10:46.158 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.157+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~LF0S9HPF6F87.0 size=1048576 2026-02-24T22:10:46.158 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.157+0000 7f7e64854980 1 < canceled part key=_multipart_HOHKR92SN25M.2~LF0S9HPF6F87.1 size=413930 2026-02-24T22:10:46.159 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.157+0000 7f7e64854980 1 < canceled multipart upload key=HOHKR92SN25M upload=LF0S9HPF6F87 tag=76QAMQWAU6B1 type=write size=2511082 2026-02-24T22:10:46.161 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.157+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~HG8X95LDLWLI.0 size=1048576 2026-02-24T22:10:46.161 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.157+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~HG8X95LDLWLI.1 size=329814 2026-02-24T22:10:46.161 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.157+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=HG8X95LDLWLI tag=V0NORZT1PGWI type=write size=2426966 2026-02-24T22:10:46.162 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.161+0000 7f7e64854980 1 < completed part key=_multipart_UQLUT9ZAI61V.2~OV6YY0Y5XDHJ.0 size=966590 2026-02-24T22:10:46.163 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.161+0000 7f7e64854980 1 < completed multipart upload key=UQLUT9ZAI61V upload=OV6YY0Y5XDHJ tag=OK7VG31ZF5YD type=write size=2015166 2026-02-24T22:10:46.164 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.161+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=PCZX9JJPEQRY type=del size=2928263 2026-02-24T22:10:46.165 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.161+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~HYKWBHPAMYKG.0 size=468702 2026-02-24T22:10:46.166 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.165+0000 7f7e64854980 1 < completed multipart upload key=3900SFJK1YQU upload=HYKWBHPAMYKG tag=EE44TSYCVCEB type=write size=1517278 2026-02-24T22:10:46.167 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.165+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=FQZPXRANO589 type=del size=234244 2026-02-24T22:10:46.168 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.165+0000 7f7e64854980 1 < completed delete operation key=A6PH77Q78VEN tag=5EIVCL1XYF5R type=del 2026-02-24T22:10:46.169 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.165+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=758GSA5J43K4 type=del size=1372157 2026-02-24T22:10:46.170 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.169+0000 7f7e64854980 1 < completed delete operation key=L60HQ7M3E3LL tag=NFVY669E1U8J type=del 2026-02-24T22:10:46.170 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.169+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=W8XU2ZTX3JNO type=write size=727781 2026-02-24T22:10:46.171 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.169+0000 7f7e64854980 1 < completed write operation key=36MKR2OZN7K9 tag=35ATR5VFH0V8 type=write size=648743 2026-02-24T22:10:46.172 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.169+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=212BE9DGE7YE type=write size=614452 2026-02-24T22:10:46.173 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.169+0000 7f7e64854980 1 < completed delete operation key=R2893NES3WSF tag=QFNDJ43XOG93 type=del 2026-02-24T22:10:46.174 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.173+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~DWJ6AIAEUK54.0 size=1048576 2026-02-24T22:10:46.174 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.173+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~DWJ6AIAEUK54.1 size=1048576 2026-02-24T22:10:46.175 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.173+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~DWJ6AIAEUK54.2 size=51899 2026-02-24T22:10:46.175 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.173+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=DWJ6AIAEUK54 tag=Y6J8O0BWHK0B type=write size=3197627 2026-02-24T22:10:46.176 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.173+0000 7f7e64854980 1 < suggested update operation key=0AXS7DY23UZU tag=1UVARQZUGSCU type=del size=2165383 2026-02-24T22:10:46.177 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.173+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=461V5TOPIOZ5 type=write size=814871 2026-02-24T22:10:46.178 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.177+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=55Y7SPDJSVJB type=del 2026-02-24T22:10:46.179 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.177+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~F3C8FI1QZC71.0 size=1048576 2026-02-24T22:10:46.180 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.177+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~F3C8FI1QZC71.1 size=1048576 2026-02-24T22:10:46.180 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.177+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~F3C8FI1QZC71.2 size=64086 2026-02-24T22:10:46.180 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.177+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=F3C8FI1QZC71 tag=F2IE08EVODJY type=write size=3209814 2026-02-24T22:10:46.181 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.177+0000 7f7e64854980 1 < completed delete operation key=LSRPN33Z2B9N tag=PP5LUVSPTC38 type=del 2026-02-24T22:10:46.182 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.181+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~ZOK1QRJIONDA.0 size=1048576 2026-02-24T22:10:46.183 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.181+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~ZOK1QRJIONDA.1 size=35069 2026-02-24T22:10:46.183 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.181+0000 7f7e64854980 1 > prepared multipart upload key=GH7723GNI2WM upload=ZOK1QRJIONDA tag=AMLYEP0VI8TR type=write size=2132221 2026-02-24T22:10:46.184 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.181+0000 7f7e64854980 1 < completed delete operation key=LSRPN33Z2B9N tag=ZMNS2MRYK7OI type=del 2026-02-24T22:10:46.184 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.181+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=DICYKKK5CCJS type=del size=3142395 2026-02-24T22:10:46.185 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.181+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~PMFNE5678WM2.0 size=487483 2026-02-24T22:10:46.186 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.185+0000 7f7e64854980 1 < completed multipart upload key=TXRUQQ456DD8 upload=PMFNE5678WM2 tag=BK6L27T9QNUT type=write size=1536059 2026-02-24T22:10:46.187 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.185+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=5UADNAPK6WJF type=write size=94796 2026-02-24T22:10:46.189 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.185+0000 7f7e64854980 1 < suggested update operation key=NL60Q5QOSP8E tag=QTGW6VA61I7B type=del size=56187 2026-02-24T22:10:46.189 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.189+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=SCSQVTKL7JKD type=del size=3571955 2026-02-24T22:10:46.190 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.189+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~9BXCTGTGTW1K.0 size=523006 2026-02-24T22:10:46.191 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.189+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=9BXCTGTGTW1K tag=I0CBNDX7WNGX type=write size=1571582 2026-02-24T22:10:46.192 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.189+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=26KG4EPQWYO8 type=del size=596676 2026-02-24T22:10:46.194 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.193+0000 7f7e64854980 1 < suggested remove operation key=EB04ETNYM087 tag=TEMX06132SQU type=del 2026-02-24T22:10:46.196 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.193+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~ZD3THMX8WBUB.0 size=1048576 2026-02-24T22:10:46.196 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.193+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~ZD3THMX8WBUB.1 size=1042726 2026-02-24T22:10:46.196 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.193+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=ZD3THMX8WBUB tag=K105O3R6OJJN type=write size=3139878 2026-02-24T22:10:46.198 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.197+0000 7f7e64854980 1 < suggested remove operation key=HOHKR92SN25M tag=DOV9WU4L696T type=del 2026-02-24T22:10:46.198 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.197+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=K5ZTA0033JPF type=del size=3341447 2026-02-24T22:10:46.199 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.197+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~HG8X95LDLWLI.0 size=1048576 2026-02-24T22:10:46.200 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.197+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~HG8X95LDLWLI.1 size=329814 2026-02-24T22:10:46.201 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.197+0000 7f7e64854980 1 < completed multipart upload key=CI8DA17DX8LU upload=HG8X95LDLWLI tag=V0NORZT1PGWI type=write size=2426966 2026-02-24T22:10:46.202 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.201+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=CZS9ZF42UZD0 type=del size=300444 2026-02-24T22:10:46.202 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.201+0000 7f7e64854980 1 < completed delete operation key=4Y3974PYYT78 tag=PCZX9JJPEQRY type=del 2026-02-24T22:10:46.203 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.201+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=3PPGVTI34S2I type=write size=431998 2026-02-24T22:10:46.204 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.201+0000 7f7e64854980 1 < completed delete operation key=Z0OXREAUJVA8 tag=FQZPXRANO589 type=del 2026-02-24T22:10:46.205 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.201+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~RR2VF15BKPB2.0 size=1048576 2026-02-24T22:10:46.206 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.205+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~RR2VF15BKPB2.1 size=1048576 2026-02-24T22:10:46.206 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.205+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~RR2VF15BKPB2.2 size=710875 2026-02-24T22:10:46.206 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.205+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=RR2VF15BKPB2 tag=LH85AUY4SP2U type=write size=3856603 2026-02-24T22:10:46.207 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.205+0000 7f7e64854980 -1 < failed to complete operation key=HOHKR92SN25M tag=758GSA5J43K4 type=del size=1372157: (22) Invalid argument 2026-02-24T22:10:46.208 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.205+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=1JQA4649PPXI type=del size=369997 2026-02-24T22:10:46.208 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.205+0000 7f7e64854980 1 < completed write operation key=HKOQRSF6XEXB tag=W8XU2ZTX3JNO type=write size=727781 2026-02-24T22:10:46.209 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.205+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=D3KLMY7IJBDF type=del size=600835 2026-02-24T22:10:46.210 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.209+0000 7f7e64854980 1 < completed write operation key=SGQYQ2TIVJPS tag=212BE9DGE7YE type=write size=614452 2026-02-24T22:10:46.210 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.209+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=3J13EPPR4876 type=del size=2428492 2026-02-24T22:10:46.211 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.209+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~DWJ6AIAEUK54.0 size=1048576 2026-02-24T22:10:46.211 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.209+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~DWJ6AIAEUK54.1 size=1048576 2026-02-24T22:10:46.212 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.209+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~DWJ6AIAEUK54.2 size=51899 2026-02-24T22:10:46.213 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.209+0000 7f7e64854980 1 < completed multipart upload key=TXRUQQ456DD8 upload=DWJ6AIAEUK54 tag=Y6J8O0BWHK0B type=write size=3197627 2026-02-24T22:10:46.214 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.213+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=NDO3BU8LJRN9 type=del size=1644430 2026-02-24T22:10:46.214 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.213+0000 7f7e64854980 1 < completed write operation key=L9NEDH44BIGA tag=461V5TOPIOZ5 type=write size=814871 2026-02-24T22:10:46.216 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.213+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~G5EHDCLM7GVR.0 size=1048576 2026-02-24T22:10:46.216 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.213+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~G5EHDCLM7GVR.1 size=821190 2026-02-24T22:10:46.216 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.213+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=G5EHDCLM7GVR tag=61VNOJ8ACDHD type=write size=2918342 2026-02-24T22:10:46.217 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.213+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~F3C8FI1QZC71.0 size=1048576 2026-02-24T22:10:46.217 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.213+0000 7f7e64854980 1 < canceled part key=_multipart_2KKQKHXO7K3K.2~F3C8FI1QZC71.1 size=1048576 2026-02-24T22:10:46.217 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.217+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~F3C8FI1QZC71.2 size=64086 2026-02-24T22:10:46.218 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.217+0000 7f7e64854980 1 < canceled multipart upload key=2KKQKHXO7K3K upload=F3C8FI1QZC71 tag=F2IE08EVODJY type=write size=3209814 2026-02-24T22:10:46.220 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.217+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~SR1TYCXX553U.0 size=1048576 2026-02-24T22:10:46.220 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.217+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~SR1TYCXX553U.1 size=1048576 2026-02-24T22:10:46.221 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.217+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~SR1TYCXX553U.2 size=776282 2026-02-24T22:10:46.221 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.217+0000 7f7e64854980 1 > prepared multipart upload key=4Y3974PYYT78 upload=SR1TYCXX553U tag=2DI0RUGGT5H8 type=write size=3922010 2026-02-24T22:10:46.221 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.221+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~ZOK1QRJIONDA.0 size=1048576 2026-02-24T22:10:46.222 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.221+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~ZOK1QRJIONDA.1 size=35069 2026-02-24T22:10:46.223 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.221+0000 7f7e64854980 1 < completed multipart upload key=GH7723GNI2WM upload=ZOK1QRJIONDA tag=AMLYEP0VI8TR type=write size=2132221 2026-02-24T22:10:46.224 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.221+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~1V0IZBS3PRGI.0 size=1048576 2026-02-24T22:10:46.224 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.221+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~1V0IZBS3PRGI.1 size=966546 2026-02-24T22:10:46.225 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.221+0000 7f7e64854980 1 > prepared multipart upload key=N3ZUNSI5K8HS upload=1V0IZBS3PRGI tag=GZMIX7BS7AON type=write size=3063698 2026-02-24T22:10:46.225 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.221+0000 7f7e64854980 1 < completed delete operation key=TF5B43G7KCYT tag=DICYKKK5CCJS type=del 2026-02-24T22:10:46.226 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.225+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~5VDOPQMMJWF1.0 size=1048576 2026-02-24T22:10:46.227 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.225+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~5VDOPQMMJWF1.1 size=55987 2026-02-24T22:10:46.227 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.225+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=5VDOPQMMJWF1 tag=OKHLS4DROXMT type=write size=2153139 2026-02-24T22:10:46.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.225+0000 7f7e64854980 1 < completed write operation key=3900SFJK1YQU tag=5UADNAPK6WJF type=write size=94796 2026-02-24T22:10:46.229 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.225+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~ASN88M07Q1SY.0 size=1048576 2026-02-24T22:10:46.229 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.229+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~ASN88M07Q1SY.1 size=1048576 2026-02-24T22:10:46.230 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.229+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~ASN88M07Q1SY.2 size=612544 2026-02-24T22:10:46.230 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.229+0000 7f7e64854980 1 > prepared multipart upload key=SGQYQ2TIVJPS upload=ASN88M07Q1SY tag=AC5J02414Q5J type=write size=3758272 2026-02-24T22:10:46.230 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.229+0000 7f7e64854980 1 < completed delete operation key=L9NEDH44BIGA tag=SCSQVTKL7JKD type=del 2026-02-24T22:10:46.232 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.229+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~KKST2KZRWV41.0 size=1048576 2026-02-24T22:10:46.233 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.229+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~KKST2KZRWV41.1 size=1048576 2026-02-24T22:10:46.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.233+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~KKST2KZRWV41.2 size=547741 2026-02-24T22:10:46.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.233+0000 7f7e64854980 1 > prepared multipart upload key=N3ZUNSI5K8HS upload=KKST2KZRWV41 tag=CRRG5Y7QGLE9 type=write size=3693469 2026-02-24T22:10:46.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.233+0000 7f7e64854980 1 < suggested update operation key=D0MKGNRDW6L9 tag=26KG4EPQWYO8 type=del size=596676 2026-02-24T22:10:46.236 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.233+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~QRHQH60L7D2C.0 size=1048576 2026-02-24T22:10:46.237 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.233+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~QRHQH60L7D2C.1 size=186809 2026-02-24T22:10:46.237 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.233+0000 7f7e64854980 1 > prepared multipart upload key=EB04ETNYM087 upload=QRHQH60L7D2C tag=VGIL7U12Z38D type=write size=2283961 2026-02-24T22:10:46.238 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.237+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~ZD3THMX8WBUB.0 size=1048576 2026-02-24T22:10:46.238 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.237+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~ZD3THMX8WBUB.1 size=1042726 2026-02-24T22:10:46.239 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.237+0000 7f7e64854980 1 < completed multipart upload key=D0MKGNRDW6L9 upload=ZD3THMX8WBUB tag=K105O3R6OJJN type=write size=3139878 2026-02-24T22:10:46.241 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.237+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~LMI7J9O4RI95.0 size=1048576 2026-02-24T22:10:46.241 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.237+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~LMI7J9O4RI95.1 size=787273 2026-02-24T22:10:46.241 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.237+0000 7f7e64854980 1 > prepared multipart upload key=ON16L53JUSTW upload=LMI7J9O4RI95 tag=ITIWZGYFCENQ type=write size=2884425 2026-02-24T22:10:46.242 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.241+0000 7f7e64854980 1 < completed delete operation key=Q7BP8FXTARDW tag=K5ZTA0033JPF type=del 2026-02-24T22:10:46.243 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.241+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=SLE03IVD5O7Q type=write size=189507 2026-02-24T22:10:46.243 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.241+0000 7f7e64854980 1 < completed delete operation key=EJCX06TE5RTZ tag=CZS9ZF42UZD0 type=del 2026-02-24T22:10:46.244 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.241+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=8A3UM4C8I833 type=write size=732705 2026-02-24T22:10:46.245 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.245+0000 7f7e64854980 1 < suggested remove operation key=TXRUQQ456DD8 tag=3PPGVTI34S2I type=write 2026-02-24T22:10:46.247 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.245+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~03UBYG9ZBHUG.0 size=1048576 2026-02-24T22:10:46.247 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.245+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~03UBYG9ZBHUG.1 size=1048576 2026-02-24T22:10:46.248 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.245+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~03UBYG9ZBHUG.2 size=69268 2026-02-24T22:10:46.248 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.245+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=03UBYG9ZBHUG tag=PLWJ7G46VWAY type=write size=3214996 2026-02-24T22:10:46.248 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.245+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~RR2VF15BKPB2.0 size=1048576 2026-02-24T22:10:46.249 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.245+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~RR2VF15BKPB2.1 size=1048576 2026-02-24T22:10:46.249 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.245+0000 7f7e64854980 1 < canceled part key=_multipart_ZESKGBWZXM6O.2~RR2VF15BKPB2.2 size=710875 2026-02-24T22:10:46.250 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.249+0000 7f7e64854980 1 < canceled multipart upload key=ZESKGBWZXM6O upload=RR2VF15BKPB2 tag=LH85AUY4SP2U type=write size=3856603 2026-02-24T22:10:46.251 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.249+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=55KXNU0K39Q8 type=write size=154094 2026-02-24T22:10:46.252 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.249+0000 7f7e64854980 1 < suggested remove operation key=GH7723GNI2WM tag=1JQA4649PPXI type=del 2026-02-24T22:10:46.253 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.249+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=V8W1OB22KXEQ type=del size=3080726 2026-02-24T22:10:46.253 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.253+0000 7f7e64854980 1 < completed delete operation key=A6PH77Q78VEN tag=D3KLMY7IJBDF type=del 2026-02-24T22:10:46.254 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.253+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=MVGUIS7GTY10 type=del size=13268 2026-02-24T22:10:46.254 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.253+0000 7f7e64854980 1 < canceled operation key=2KKQKHXO7K3K tag=3J13EPPR4876 type=del size=2428492 2026-02-24T22:10:46.255 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.253+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=J674YVNBYCCL type=del size=2870127 2026-02-24T22:10:46.256 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.253+0000 7f7e64854980 1 < completed delete operation key=36MKR2OZN7K9 tag=NDO3BU8LJRN9 type=del 2026-02-24T22:10:46.256 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.253+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=15V0MKN3AK1V type=del size=1892390 2026-02-24T22:10:46.257 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.253+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~G5EHDCLM7GVR.0 size=1048576 2026-02-24T22:10:46.258 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.257+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~G5EHDCLM7GVR.1 size=821190 2026-02-24T22:10:46.259 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.257+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=G5EHDCLM7GVR tag=61VNOJ8ACDHD type=write size=2918342 2026-02-24T22:10:46.259 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.257+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=XNCLXI998PEC type=write size=811515 2026-02-24T22:10:46.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.257+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~SR1TYCXX553U.0 size=1048576 2026-02-24T22:10:46.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.257+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~SR1TYCXX553U.1 size=1048576 2026-02-24T22:10:46.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.257+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~SR1TYCXX553U.2 size=776282 2026-02-24T22:10:46.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.261+0000 7f7e64854980 1 < completed multipart upload key=4Y3974PYYT78 upload=SR1TYCXX553U tag=2DI0RUGGT5H8 type=write size=3922010 2026-02-24T22:10:46.263 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.261+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=YRT2D3KRF681 type=del size=1847616 2026-02-24T22:10:46.263 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.261+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~1V0IZBS3PRGI.0 size=1048576 2026-02-24T22:10:46.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.261+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~1V0IZBS3PRGI.1 size=966546 2026-02-24T22:10:46.265 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.261+0000 7f7e64854980 1 < completed multipart upload key=N3ZUNSI5K8HS upload=1V0IZBS3PRGI tag=GZMIX7BS7AON type=write size=3063698 2026-02-24T22:10:46.265 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.265+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=WOOD7WHWW3WD type=del size=1786453 2026-02-24T22:10:46.266 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.265+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~5VDOPQMMJWF1.0 size=1048576 2026-02-24T22:10:46.267 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.265+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~5VDOPQMMJWF1.1 size=55987 2026-02-24T22:10:46.268 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.265+0000 7f7e64854980 1 < completed multipart upload key=ZESKGBWZXM6O upload=5VDOPQMMJWF1 tag=OKHLS4DROXMT type=write size=2153139 2026-02-24T22:10:46.268 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.265+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=X1UMHF266TMB type=write size=301006 2026-02-24T22:10:46.269 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.265+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~ASN88M07Q1SY.0 size=1048576 2026-02-24T22:10:46.270 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.269+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~ASN88M07Q1SY.1 size=1048576 2026-02-24T22:10:46.270 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.269+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~ASN88M07Q1SY.2 size=612544 2026-02-24T22:10:46.271 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.269+0000 7f7e64854980 1 < completed multipart upload key=SGQYQ2TIVJPS upload=ASN88M07Q1SY tag=AC5J02414Q5J type=write size=3758272 2026-02-24T22:10:46.272 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.269+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=NQW0I0VL7F6U type=write size=954535 2026-02-24T22:10:46.272 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.269+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~KKST2KZRWV41.0 size=1048576 2026-02-24T22:10:46.273 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.269+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~KKST2KZRWV41.1 size=1048576 2026-02-24T22:10:46.273 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.273+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~KKST2KZRWV41.2 size=547741 2026-02-24T22:10:46.274 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.273+0000 7f7e64854980 1 < completed multipart upload key=N3ZUNSI5K8HS upload=KKST2KZRWV41 tag=CRRG5Y7QGLE9 type=write size=3693469 2026-02-24T22:10:46.275 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.273+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=EYQJF5L5TI1S type=write size=852081 2026-02-24T22:10:46.276 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.273+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~QRHQH60L7D2C.0 size=1048576 2026-02-24T22:10:46.276 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.273+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~QRHQH60L7D2C.1 size=186809 2026-02-24T22:10:46.277 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.277+0000 7f7e64854980 1 < completed multipart upload key=EB04ETNYM087 upload=QRHQH60L7D2C tag=VGIL7U12Z38D type=write size=2283961 2026-02-24T22:10:46.278 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.277+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~CZHSNNOOSO5I.0 size=813750 2026-02-24T22:10:46.279 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.277+0000 7f7e64854980 1 > prepared multipart upload key=L60HQ7M3E3LL upload=CZHSNNOOSO5I tag=6471SSG543IR type=write size=1862326 2026-02-24T22:10:46.279 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.277+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~LMI7J9O4RI95.0 size=1048576 2026-02-24T22:10:46.280 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.277+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~LMI7J9O4RI95.1 size=787273 2026-02-24T22:10:46.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.277+0000 7f7e64854980 1 < completed multipart upload key=ON16L53JUSTW upload=LMI7J9O4RI95 tag=ITIWZGYFCENQ type=write size=2884425 2026-02-24T22:10:46.282 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.281+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~9GT15WK3MIOQ.0 size=384171 2026-02-24T22:10:46.282 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.281+0000 7f7e64854980 1 > prepared multipart upload key=TF5B43G7KCYT upload=9GT15WK3MIOQ tag=9UYCORZ37DHV type=write size=1432747 2026-02-24T22:10:46.283 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.281+0000 7f7e64854980 1 < completed write operation key=L60HQ7M3E3LL tag=SLE03IVD5O7Q type=write size=189507 2026-02-24T22:10:46.284 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.281+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~U1857ESWP7FM.0 size=970131 2026-02-24T22:10:46.284 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.281+0000 7f7e64854980 1 > prepared multipart upload key=PKRHWM69GBJI upload=U1857ESWP7FM tag=ONTYSAKO9Q00 type=write size=2018707 2026-02-24T22:10:46.285 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.281+0000 7f7e64854980 1 < completed write operation key=4Y3974PYYT78 tag=8A3UM4C8I833 type=write size=732705 2026-02-24T22:10:46.285 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.285+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=E15EVN7S95PG type=del size=3006197 2026-02-24T22:10:46.286 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.285+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~03UBYG9ZBHUG.0 size=1048576 2026-02-24T22:10:46.286 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.285+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~03UBYG9ZBHUG.1 size=1048576 2026-02-24T22:10:46.287 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.285+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~03UBYG9ZBHUG.2 size=69268 2026-02-24T22:10:46.288 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.285+0000 7f7e64854980 1 < completed multipart upload key=CI8DA17DX8LU upload=03UBYG9ZBHUG tag=PLWJ7G46VWAY type=write size=3214996 2026-02-24T22:10:46.289 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.285+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=NHL18T8MLSL2 type=write size=698394 2026-02-24T22:10:46.290 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.289+0000 7f7e64854980 1 < canceled operation key=L9NEDH44BIGA tag=55KXNU0K39Q8 type=write size=154094 2026-02-24T22:10:46.291 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.289+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~ZEYXOD6JWSRL.0 size=1048576 2026-02-24T22:10:46.292 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.289+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~ZEYXOD6JWSRL.1 size=1048576 2026-02-24T22:10:46.292 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.289+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~ZEYXOD6JWSRL.2 size=531838 2026-02-24T22:10:46.292 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.289+0000 7f7e64854980 1 > prepared multipart upload key=TF5B43G7KCYT upload=ZEYXOD6JWSRL tag=WOLPVQ1T5967 type=write size=3677566 2026-02-24T22:10:46.293 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.289+0000 7f7e64854980 1 < completed delete operation key=78GUTC5THI2P tag=V8W1OB22KXEQ type=del 2026-02-24T22:10:46.294 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.293+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=LNU01A2O48T5 type=del size=4045430 2026-02-24T22:10:46.295 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.293+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=MVGUIS7GTY10 type=del 2026-02-24T22:10:46.296 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.293+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=WL4J22TODD2O type=del size=838087 2026-02-24T22:10:46.296 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.293+0000 7f7e64854980 1 < completed delete operation key=LSRPN33Z2B9N tag=J674YVNBYCCL type=del 2026-02-24T22:10:46.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.293+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=QRT0ID24IR37 type=del size=1267486 2026-02-24T22:10:46.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.297+0000 7f7e64854980 1 < canceled operation key=4Y3974PYYT78 tag=15V0MKN3AK1V type=del size=1892390 2026-02-24T22:10:46.298 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.297+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=2J3FCGQ76O14 type=del size=2193025 2026-02-24T22:10:46.299 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.297+0000 7f7e64854980 1 < completed write operation key=Y3L3CJ51C1WR tag=XNCLXI998PEC type=write size=811515 2026-02-24T22:10:46.300 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.297+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~LGYH0KTTVIY9.0 size=1048576 2026-02-24T22:10:46.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.297+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~LGYH0KTTVIY9.1 size=288038 2026-02-24T22:10:46.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.297+0000 7f7e64854980 1 > prepared multipart upload key=N3ZUNSI5K8HS upload=LGYH0KTTVIY9 tag=BXDL4JW8NI8B type=write size=2385190 2026-02-24T22:10:46.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.297+0000 7f7e64854980 1 < canceled operation key=R2893NES3WSF tag=YRT2D3KRF681 type=del size=1847616 2026-02-24T22:10:46.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.301+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~R1UBLKB6MFBH.0 size=1048576 2026-02-24T22:10:46.303 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.301+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~R1UBLKB6MFBH.1 size=233165 2026-02-24T22:10:46.303 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.301+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=R1UBLKB6MFBH tag=N068JBOVHBYM type=write size=2330317 2026-02-24T22:10:46.304 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.301+0000 7f7e64854980 1 < completed delete operation key=4Y3974PYYT78 tag=WOOD7WHWW3WD type=del 2026-02-24T22:10:46.304 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.301+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=Y8JNA43SA0O2 type=write size=530697 2026-02-24T22:10:46.305 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.301+0000 7f7e64854980 1 < completed write operation key=ZESKGBWZXM6O tag=X1UMHF266TMB type=write size=301006 2026-02-24T22:10:46.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.305+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~K11ZIEJG8JYL.0 size=87635 2026-02-24T22:10:46.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.305+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=K11ZIEJG8JYL tag=4H894X7S4C4F type=write size=1136211 2026-02-24T22:10:46.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.305+0000 7f7e64854980 1 < canceled operation key=SGQYQ2TIVJPS tag=NQW0I0VL7F6U type=write size=954535 2026-02-24T22:10:46.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.305+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~M0CUQ6ASL9X2.0 size=964678 2026-02-24T22:10:46.309 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.305+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=M0CUQ6ASL9X2 tag=MU8FUZJWOU86 type=write size=2013254 2026-02-24T22:10:46.309 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.305+0000 7f7e64854980 1 < completed write operation key=Z0OXREAUJVA8 tag=EYQJF5L5TI1S type=write size=852081 2026-02-24T22:10:46.310 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.309+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=X424S073VLHU type=write size=76295 2026-02-24T22:10:46.311 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.309+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~CZHSNNOOSO5I.0 size=813750 2026-02-24T22:10:46.312 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.309+0000 7f7e64854980 1 < completed multipart upload key=L60HQ7M3E3LL upload=CZHSNNOOSO5I tag=6471SSG543IR type=write size=1862326 2026-02-24T22:10:46.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.309+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=GAPFSIYX55JP type=del size=377057 2026-02-24T22:10:46.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.313+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~9GT15WK3MIOQ.0 size=384171 2026-02-24T22:10:46.314 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.313+0000 7f7e64854980 1 < completed multipart upload key=TF5B43G7KCYT upload=9GT15WK3MIOQ tag=9UYCORZ37DHV type=write size=1432747 2026-02-24T22:10:46.315 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.313+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=IWF8RBS07G4I type=del size=783190 2026-02-24T22:10:46.316 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.313+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~U1857ESWP7FM.0 size=970131 2026-02-24T22:10:46.317 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.313+0000 7f7e64854980 1 < completed multipart upload key=PKRHWM69GBJI upload=U1857ESWP7FM tag=ONTYSAKO9Q00 type=write size=2018707 2026-02-24T22:10:46.319 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.317+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~JIC0FPR3IDJ5.0 size=1048576 2026-02-24T22:10:46.319 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.317+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~JIC0FPR3IDJ5.1 size=601508 2026-02-24T22:10:46.319 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.317+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=JIC0FPR3IDJ5 tag=3DEZW2V0PAMZ type=write size=2698660 2026-02-24T22:10:46.320 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.317+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=E15EVN7S95PG type=del 2026-02-24T22:10:46.321 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.317+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~5BQDWSKQ1OJ7.0 size=1048576 2026-02-24T22:10:46.322 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.321+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~5BQDWSKQ1OJ7.1 size=1048576 2026-02-24T22:10:46.322 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.321+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~5BQDWSKQ1OJ7.2 size=697234 2026-02-24T22:10:46.322 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.321+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=5BQDWSKQ1OJ7 tag=TRQ40IJRA5LL type=write size=3842962 2026-02-24T22:10:46.323 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.321+0000 7f7e64854980 1 < canceled operation key=L9NEDH44BIGA tag=NHL18T8MLSL2 type=write size=698394 2026-02-24T22:10:46.324 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.321+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=OA3NFTIQX3SI type=del size=3686511 2026-02-24T22:10:46.324 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.321+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~ZEYXOD6JWSRL.0 size=1048576 2026-02-24T22:10:46.325 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.321+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~ZEYXOD6JWSRL.1 size=1048576 2026-02-24T22:10:46.326 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.325+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~ZEYXOD6JWSRL.2 size=531838 2026-02-24T22:10:46.327 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.325+0000 7f7e64854980 1 < completed multipart upload key=TF5B43G7KCYT upload=ZEYXOD6JWSRL tag=WOLPVQ1T5967 type=write size=3677566 2026-02-24T22:10:46.328 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.325+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~F2T6DE9I3QNQ.0 size=480186 2026-02-24T22:10:46.328 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.325+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=F2T6DE9I3QNQ tag=LV2ADBUKPSOQ type=write size=1528762 2026-02-24T22:10:46.329 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.325+0000 7f7e64854980 1 < completed delete operation key=0AXS7DY23UZU tag=LNU01A2O48T5 type=del 2026-02-24T22:10:46.330 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.329+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=DRO018547101 type=write size=13285 2026-02-24T22:10:46.331 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.329+0000 7f7e64854980 1 < suggested update operation key=36MKR2OZN7K9 tag=WL4J22TODD2O type=del size=838087 2026-02-24T22:10:46.332 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.329+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~IX1KB0CWHKR0.0 size=1048576 2026-02-24T22:10:46.333 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.329+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~IX1KB0CWHKR0.1 size=1048576 2026-02-24T22:10:46.333 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.333+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~IX1KB0CWHKR0.2 size=1022565 2026-02-24T22:10:46.333 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.333+0000 7f7e64854980 1 > prepared multipart upload key=Q7BP8FXTARDW upload=IX1KB0CWHKR0 tag=ZTYFUSX7QGI8 type=write size=4168293 2026-02-24T22:10:46.335 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.333+0000 7f7e64854980 1 < suggested update operation key=HKOQRSF6XEXB tag=QRT0ID24IR37 type=del size=1267486 2026-02-24T22:10:46.335 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.333+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=WTRCW9TZG0ER type=del size=1401964 2026-02-24T22:10:46.336 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.333+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=2J3FCGQ76O14 type=del 2026-02-24T22:10:46.337 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.337+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~KGRHIW4WYGAV.0 size=228825 2026-02-24T22:10:46.337 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.337+0000 7f7e64854980 1 > prepared multipart upload key=N3ZUNSI5K8HS upload=KGRHIW4WYGAV tag=8D39TD13S8G0 type=write size=1277401 2026-02-24T22:10:46.338 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.337+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~LGYH0KTTVIY9.0 size=1048576 2026-02-24T22:10:46.339 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.337+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~LGYH0KTTVIY9.1 size=288038 2026-02-24T22:10:46.340 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.337+0000 7f7e64854980 1 < completed multipart upload key=N3ZUNSI5K8HS upload=LGYH0KTTVIY9 tag=BXDL4JW8NI8B type=write size=2385190 2026-02-24T22:10:46.341 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.337+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=3NOXGTLU3W2V type=del size=378780 2026-02-24T22:10:46.342 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.341+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~R1UBLKB6MFBH.0 size=1048576 2026-02-24T22:10:46.342 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.341+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~R1UBLKB6MFBH.1 size=233165 2026-02-24T22:10:46.343 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.341+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=R1UBLKB6MFBH tag=N068JBOVHBYM type=write size=2330317 2026-02-24T22:10:46.344 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.341+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=EFZ940Q8YYK2 type=del size=1935241 2026-02-24T22:10:46.345 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.341+0000 7f7e64854980 -1 < failed to complete operation key=36MKR2OZN7K9 tag=Y8JNA43SA0O2 type=write size=530697: (22) Invalid argument 2026-02-24T22:10:46.345 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.345+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=5YDUP41WE14N type=del size=4132541 2026-02-24T22:10:46.346 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.345+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~K11ZIEJG8JYL.0 size=87635 2026-02-24T22:10:46.347 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.345+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=K11ZIEJG8JYL tag=4H894X7S4C4F type=write size=1136211 2026-02-24T22:10:46.348 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.345+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=80489BGWPABW type=del size=2963379 2026-02-24T22:10:46.349 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.345+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~M0CUQ6ASL9X2.0 size=964678 2026-02-24T22:10:46.350 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.349+0000 7f7e64854980 1 < completed multipart upload key=EJCX06TE5RTZ upload=M0CUQ6ASL9X2 tag=MU8FUZJWOU86 type=write size=2013254 2026-02-24T22:10:46.351 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.349+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=YCRP7ECAEJZ1 type=write size=200262 2026-02-24T22:10:46.351 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.349+0000 7f7e64854980 1 < completed write operation key=EB04ETNYM087 tag=X424S073VLHU type=write size=76295 2026-02-24T22:10:46.352 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.349+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=BVUA5YR0OYVT type=del size=3531299 2026-02-24T22:10:46.353 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.349+0000 7f7e64854980 1 < completed delete operation key=UQLUT9ZAI61V tag=GAPFSIYX55JP type=del 2026-02-24T22:10:46.354 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.353+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~KGUJ0FYG12NC.0 size=132556 2026-02-24T22:10:46.354 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.353+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=KGUJ0FYG12NC tag=AGTUF1L34EX7 type=write size=1181132 2026-02-24T22:10:46.355 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.353+0000 7f7e64854980 1 < completed delete operation key=R2893NES3WSF tag=IWF8RBS07G4I type=del 2026-02-24T22:10:46.356 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.353+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~DBMASA9PDLG8.0 size=440515 2026-02-24T22:10:46.356 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.353+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=DBMASA9PDLG8 tag=Y0HLDHDW7X0F type=write size=1489091 2026-02-24T22:10:46.356 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.353+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~JIC0FPR3IDJ5.0 size=1048576 2026-02-24T22:10:46.357 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.353+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~JIC0FPR3IDJ5.1 size=601508 2026-02-24T22:10:46.358 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.357+0000 7f7e64854980 1 < completed multipart upload key=D0MKGNRDW6L9 upload=JIC0FPR3IDJ5 tag=3DEZW2V0PAMZ type=write size=2698660 2026-02-24T22:10:46.360 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.357+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~071WQ6BOH16L.0 size=1048576 2026-02-24T22:10:46.361 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.357+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~071WQ6BOH16L.1 size=730344 2026-02-24T22:10:46.361 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.357+0000 7f7e64854980 1 > prepared multipart upload key=NL60Q5QOSP8E upload=071WQ6BOH16L tag=L7KO24JCO2QG type=write size=2827496 2026-02-24T22:10:46.362 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.361+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~5BQDWSKQ1OJ7.0 size=1048576 2026-02-24T22:10:46.362 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.361+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~5BQDWSKQ1OJ7.1 size=1048576 2026-02-24T22:10:46.363 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.361+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~5BQDWSKQ1OJ7.2 size=697234 2026-02-24T22:10:46.364 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.361+0000 7f7e64854980 1 < completed multipart upload key=EJCX06TE5RTZ upload=5BQDWSKQ1OJ7 tag=TRQ40IJRA5LL type=write size=3842962 2026-02-24T22:10:46.365 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.361+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=NWW740AUBAQO type=del size=3222982 2026-02-24T22:10:46.366 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.365+0000 7f7e64854980 1 < completed delete operation key=SGQYQ2TIVJPS tag=OA3NFTIQX3SI type=del 2026-02-24T22:10:46.366 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.365+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=GK8PGBLJK3BG type=del size=590482 2026-02-24T22:10:46.367 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.365+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~F2T6DE9I3QNQ.0 size=480186 2026-02-24T22:10:46.368 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.365+0000 7f7e64854980 1 < completed multipart upload key=HOHKR92SN25M upload=F2T6DE9I3QNQ tag=LV2ADBUKPSOQ type=write size=1528762 2026-02-24T22:10:46.369 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.369+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=LVFG2J6AA13S type=del size=2038359 2026-02-24T22:10:46.370 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.369+0000 7f7e64854980 1 < completed write operation key=EB04ETNYM087 tag=DRO018547101 type=write size=13285 2026-02-24T22:10:46.371 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.369+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=5VFB6CPQ3FJP type=del size=2503903 2026-02-24T22:10:46.371 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.369+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~IX1KB0CWHKR0.0 size=1048576 2026-02-24T22:10:46.372 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.369+0000 7f7e64854980 1 < canceled part key=_multipart_Q7BP8FXTARDW.2~IX1KB0CWHKR0.1 size=1048576 2026-02-24T22:10:46.372 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.369+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~IX1KB0CWHKR0.2 size=1022565 2026-02-24T22:10:46.373 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.373+0000 7f7e64854980 1 < canceled multipart upload key=Q7BP8FXTARDW upload=IX1KB0CWHKR0 tag=ZTYFUSX7QGI8 type=write size=4168293 2026-02-24T22:10:46.374 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.373+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=8YWTG9GV95AQ type=del size=217248 2026-02-24T22:10:46.375 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.373+0000 7f7e64854980 1 < completed delete operation key=TF5B43G7KCYT tag=WTRCW9TZG0ER type=del 2026-02-24T22:10:46.376 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.373+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~41RJZSJGAXY2.0 size=1048576 2026-02-24T22:10:46.377 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.373+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~41RJZSJGAXY2.1 size=555904 2026-02-24T22:10:46.377 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.373+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=41RJZSJGAXY2 tag=HM711GUYOV0P type=write size=2653056 2026-02-24T22:10:46.377 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.377+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~KGRHIW4WYGAV.0 size=228825 2026-02-24T22:10:46.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.377+0000 7f7e64854980 1 < completed multipart upload key=N3ZUNSI5K8HS upload=KGRHIW4WYGAV tag=8D39TD13S8G0 type=write size=1277401 2026-02-24T22:10:46.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.377+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~61QE2BFS7IE0.0 size=158056 2026-02-24T22:10:46.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.377+0000 7f7e64854980 1 > prepared multipart upload key=LSRPN33Z2B9N upload=61QE2BFS7IE0 tag=JBQIDDFP81QE type=write size=1206632 2026-02-24T22:10:46.381 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.377+0000 7f7e64854980 1 < completed delete operation key=Q7BP8FXTARDW tag=3NOXGTLU3W2V type=del 2026-02-24T22:10:46.381 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.381+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=1OSGQHMPU7F5 type=del size=583553 2026-02-24T22:10:46.382 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.381+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=EFZ940Q8YYK2 type=del 2026-02-24T22:10:46.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.381+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=2582N53P0033 type=del size=2398963 2026-02-24T22:10:46.385 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.381+0000 7f7e64854980 1 < suggested update operation key=HKOQRSF6XEXB tag=5YDUP41WE14N type=del size=4132541 2026-02-24T22:10:46.385 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.385+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=YECIP0I26WPP type=write size=389490 2026-02-24T22:10:46.386 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.385+0000 7f7e64854980 1 < completed delete operation key=A6PH77Q78VEN tag=80489BGWPABW type=del 2026-02-24T22:10:46.387 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.385+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=PV3I5KECBS1N type=del size=288193 2026-02-24T22:10:46.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.385+0000 7f7e64854980 1 < completed write operation key=0AXS7DY23UZU tag=YCRP7ECAEJZ1 type=write size=200262 2026-02-24T22:10:46.389 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.385+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~7RR84EM0YMNR.0 size=85341 2026-02-24T22:10:46.389 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.385+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=7RR84EM0YMNR tag=U8K0FETC9L9M type=write size=1133917 2026-02-24T22:10:46.389 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.389+0000 7f7e64854980 1 < completed delete operation key=L9NEDH44BIGA tag=BVUA5YR0OYVT type=del 2026-02-24T22:10:46.390 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.389+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=2LMKUAQQ8J59 type=del size=3304639 2026-02-24T22:10:46.391 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.389+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~KGUJ0FYG12NC.0 size=132556 2026-02-24T22:10:46.392 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.389+0000 7f7e64854980 1 < completed multipart upload key=HOHKR92SN25M upload=KGUJ0FYG12NC tag=AGTUF1L34EX7 type=write size=1181132 2026-02-24T22:10:46.393 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.389+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=BVNP71F4Y84H type=write size=904721 2026-02-24T22:10:46.393 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.393+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~DBMASA9PDLG8.0 size=440515 2026-02-24T22:10:46.394 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.393+0000 7f7e64854980 1 < completed multipart upload key=D0MKGNRDW6L9 upload=DBMASA9PDLG8 tag=Y0HLDHDW7X0F type=write size=1489091 2026-02-24T22:10:46.395 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.393+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=PF6TPCJZOXJC type=del size=2815739 2026-02-24T22:10:46.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.393+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~071WQ6BOH16L.0 size=1048576 2026-02-24T22:10:46.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.393+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~071WQ6BOH16L.1 size=730344 2026-02-24T22:10:46.397 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.397+0000 7f7e64854980 1 < completed multipart upload key=NL60Q5QOSP8E upload=071WQ6BOH16L tag=L7KO24JCO2QG type=write size=2827496 2026-02-24T22:10:46.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.397+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~AVWJOG2MUKV1.0 size=1048576 2026-02-24T22:10:46.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.397+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~AVWJOG2MUKV1.1 size=50349 2026-02-24T22:10:46.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.397+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=AVWJOG2MUKV1 tag=8OI12JNESRCR type=write size=2147501 2026-02-24T22:10:46.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.397+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=NWW740AUBAQO type=del 2026-02-24T22:10:46.400 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.397+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=AOTN63HXPQEU type=del size=314227 2026-02-24T22:10:46.401 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.397+0000 7f7e64854980 1 < completed delete operation key=SGQYQ2TIVJPS tag=GK8PGBLJK3BG type=del 2026-02-24T22:10:46.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.401+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~H9QKI11C5UWX.0 size=555216 2026-02-24T22:10:46.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.401+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=H9QKI11C5UWX tag=0RCESAO3JCCS type=write size=1603792 2026-02-24T22:10:46.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.401+0000 7f7e64854980 1 < canceled operation key=Y3L3CJ51C1WR tag=LVFG2J6AA13S type=del size=2038359 2026-02-24T22:10:46.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.401+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~2ER0CGMK3X24.0 size=1048576 2026-02-24T22:10:46.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.401+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~2ER0CGMK3X24.1 size=472524 2026-02-24T22:10:46.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.401+0000 7f7e64854980 1 > prepared multipart upload key=GH7723GNI2WM upload=2ER0CGMK3X24 tag=LXK1V9354MBK type=write size=2569676 2026-02-24T22:10:46.405 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.401+0000 7f7e64854980 1 < completed delete operation key=ZESKGBWZXM6O tag=5VFB6CPQ3FJP type=del 2026-02-24T22:10:46.406 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.405+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~W2XT0U27WQW0.0 size=1048576 2026-02-24T22:10:46.406 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.405+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~W2XT0U27WQW0.1 size=918147 2026-02-24T22:10:46.406 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.405+0000 7f7e64854980 1 > prepared multipart upload key=L60HQ7M3E3LL upload=W2XT0U27WQW0 tag=H4TLKFS472KS type=write size=3015299 2026-02-24T22:10:46.407 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.405+0000 7f7e64854980 1 < canceled operation key=Q7BP8FXTARDW tag=8YWTG9GV95AQ type=del size=217248 2026-02-24T22:10:46.408 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.405+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=KJB24W5VADSS type=write size=976646 2026-02-24T22:10:46.409 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.405+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~41RJZSJGAXY2.0 size=1048576 2026-02-24T22:10:46.409 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.405+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~41RJZSJGAXY2.1 size=555904 2026-02-24T22:10:46.410 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.409+0000 7f7e64854980 1 < completed multipart upload key=HOHKR92SN25M upload=41RJZSJGAXY2 tag=HM711GUYOV0P type=write size=2653056 2026-02-24T22:10:46.411 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.409+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~RMCKPTAWMNWL.0 size=153184 2026-02-24T22:10:46.411 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.409+0000 7f7e64854980 1 > prepared multipart upload key=Z0OXREAUJVA8 upload=RMCKPTAWMNWL tag=21TZAI4FMB1V type=write size=1201760 2026-02-24T22:10:46.412 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.409+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~61QE2BFS7IE0.0 size=158056 2026-02-24T22:10:46.413 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.409+0000 7f7e64854980 1 < completed multipart upload key=LSRPN33Z2B9N upload=61QE2BFS7IE0 tag=JBQIDDFP81QE type=write size=1206632 2026-02-24T22:10:46.414 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.413+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~G5HIMJP6JKSY.0 size=1048576 2026-02-24T22:10:46.414 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.413+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~G5HIMJP6JKSY.1 size=129866 2026-02-24T22:10:46.414 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.413+0000 7f7e64854980 1 > prepared multipart upload key=UQLUT9ZAI61V upload=G5HIMJP6JKSY tag=VEYP5PNFYVRW type=write size=2227018 2026-02-24T22:10:46.415 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.413+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=1OSGQHMPU7F5 type=del 2026-02-24T22:10:46.416 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.413+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=YS6705BD18QC type=del size=1419047 2026-02-24T22:10:46.417 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.413+0000 7f7e64854980 1 < suggested update operation key=TXRUQQ456DD8 tag=2582N53P0033 type=del size=2398963 2026-02-24T22:10:46.418 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.417+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=UXQX6DKKAQKU type=del size=2837834 2026-02-24T22:10:46.419 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.417+0000 7f7e64854980 1 < suggested remove operation key=HOHKR92SN25M tag=YECIP0I26WPP type=write 2026-02-24T22:10:46.420 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.417+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=DWQSWKJ4E01U type=del size=1505571 2026-02-24T22:10:46.421 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.417+0000 7f7e64854980 1 < suggested update operation key=SGQYQ2TIVJPS tag=PV3I5KECBS1N type=del size=288193 2026-02-24T22:10:46.422 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.421+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~87CV126IQZQ6.0 size=1048576 2026-02-24T22:10:46.423 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.421+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~87CV126IQZQ6.1 size=1048576 2026-02-24T22:10:46.424 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.421+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~87CV126IQZQ6.2 size=386734 2026-02-24T22:10:46.424 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.421+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=87CV126IQZQ6 tag=B1TY0BB2X3KZ type=write size=3532462 2026-02-24T22:10:46.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.421+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~7RR84EM0YMNR.0 size=85341 2026-02-24T22:10:46.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.425+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=7RR84EM0YMNR tag=U8K0FETC9L9M type=write size=1133917 2026-02-24T22:10:46.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.425+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=8A66WJDO25X5 type=write size=9115 2026-02-24T22:10:46.428 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.425+0000 7f7e64854980 1 < completed delete operation key=NL60Q5QOSP8E tag=2LMKUAQQ8J59 type=del 2026-02-24T22:10:46.428 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.425+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=TM6K3074PX0K type=write size=774568 2026-02-24T22:10:46.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.425+0000 7f7e64854980 1 < completed write operation key=D0MKGNRDW6L9 tag=BVNP71F4Y84H type=write size=904721 2026-02-24T22:10:46.430 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.429+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~229062R3EDDM.0 size=196853 2026-02-24T22:10:46.431 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.429+0000 7f7e64854980 1 > prepared multipart upload key=LSRPN33Z2B9N upload=229062R3EDDM tag=SAF332QKT7Q4 type=write size=1245429 2026-02-24T22:10:46.431 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.429+0000 7f7e64854980 1 < completed delete operation key=Y3L3CJ51C1WR tag=PF6TPCJZOXJC type=del 2026-02-24T22:10:46.432 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.429+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~FZNZWOCF72D6.0 size=525088 2026-02-24T22:10:46.432 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.429+0000 7f7e64854980 1 > prepared multipart upload key=4Y3974PYYT78 upload=FZNZWOCF72D6 tag=SHP4B2S8SWBV type=write size=1573664 2026-02-24T22:10:46.433 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.429+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~AVWJOG2MUKV1.0 size=1048576 2026-02-24T22:10:46.433 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.433+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~AVWJOG2MUKV1.1 size=50349 2026-02-24T22:10:46.435 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.433+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=AVWJOG2MUKV1 tag=8OI12JNESRCR type=write size=2147501 2026-02-24T22:10:46.436 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.433+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~3W2MY8OW51D8.0 size=953278 2026-02-24T22:10:46.436 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.433+0000 7f7e64854980 1 > prepared multipart upload key=Q7BP8FXTARDW upload=3W2MY8OW51D8 tag=T7CGPXN53L7R type=write size=2001854 2026-02-24T22:10:46.436 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.433+0000 7f7e64854980 1 < completed delete operation key=N3ZUNSI5K8HS tag=AOTN63HXPQEU type=del 2026-02-24T22:10:46.438 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.437+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~1B5W3IBY3PYB.0 size=142743 2026-02-24T22:10:46.438 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.437+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=1B5W3IBY3PYB tag=E5WB51R7PBDD type=write size=1191319 2026-02-24T22:10:46.438 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.437+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~H9QKI11C5UWX.0 size=555216 2026-02-24T22:10:46.439 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.437+0000 7f7e64854980 1 < completed multipart upload key=CI8DA17DX8LU upload=H9QKI11C5UWX tag=0RCESAO3JCCS type=write size=1603792 2026-02-24T22:10:46.440 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.437+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=V9A3HLGCSBU3 type=write size=250070 2026-02-24T22:10:46.440 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.437+0000 7f7e64854980 1 < canceled part key=_multipart_GH7723GNI2WM.2~2ER0CGMK3X24.0 size=1048576 2026-02-24T22:10:46.441 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.437+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~2ER0CGMK3X24.1 size=472524 2026-02-24T22:10:46.442 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.441+0000 7f7e64854980 1 < canceled multipart upload key=GH7723GNI2WM upload=2ER0CGMK3X24 tag=LXK1V9354MBK type=write size=2569676 2026-02-24T22:10:46.442 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.441+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=H5HXR4O962SS type=del size=783611 2026-02-24T22:10:46.443 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.441+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~W2XT0U27WQW0.0 size=1048576 2026-02-24T22:10:46.444 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.441+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~W2XT0U27WQW0.1 size=918147 2026-02-24T22:10:46.445 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.441+0000 7f7e64854980 1 < completed multipart upload key=L60HQ7M3E3LL upload=W2XT0U27WQW0 tag=H4TLKFS472KS type=write size=3015299 2026-02-24T22:10:46.445 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.445+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=1C87YCAA94YU type=del size=1257427 2026-02-24T22:10:46.446 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.445+0000 7f7e64854980 1 < completed write operation key=3900SFJK1YQU tag=KJB24W5VADSS type=write size=976646 2026-02-24T22:10:46.447 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.445+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=EVOP7IL2C4SW type=write size=767462 2026-02-24T22:10:46.448 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.445+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~RMCKPTAWMNWL.0 size=153184 2026-02-24T22:10:46.449 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.445+0000 7f7e64854980 1 < completed multipart upload key=Z0OXREAUJVA8 upload=RMCKPTAWMNWL tag=21TZAI4FMB1V type=write size=1201760 2026-02-24T22:10:46.450 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.449+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=Z9PKNCJ0N1AL type=del size=309736 2026-02-24T22:10:46.450 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.449+0000 7f7e64854980 1 < completed part key=_multipart_UQLUT9ZAI61V.2~G5HIMJP6JKSY.0 size=1048576 2026-02-24T22:10:46.451 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.449+0000 7f7e64854980 1 < completed part key=_multipart_UQLUT9ZAI61V.2~G5HIMJP6JKSY.1 size=129866 2026-02-24T22:10:46.452 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.449+0000 7f7e64854980 1 < completed multipart upload key=UQLUT9ZAI61V upload=G5HIMJP6JKSY tag=VEYP5PNFYVRW type=write size=2227018 2026-02-24T22:10:46.452 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.449+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=0TGXAGG9IYYS type=del size=2152946 2026-02-24T22:10:46.454 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.453+0000 7f7e64854980 1 < suggested remove operation key=R2893NES3WSF tag=YS6705BD18QC type=del 2026-02-24T22:10:46.455 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.453+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~F8UIVJ8DOG5O.0 size=1048576 2026-02-24T22:10:46.455 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.453+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~F8UIVJ8DOG5O.1 size=299046 2026-02-24T22:10:46.456 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.453+0000 7f7e64854980 1 > prepared multipart upload key=3900SFJK1YQU upload=F8UIVJ8DOG5O tag=4GMNNSDY1R8R type=write size=2396198 2026-02-24T22:10:46.456 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.453+0000 7f7e64854980 1 < canceled operation key=Q7BP8FXTARDW tag=UXQX6DKKAQKU type=del size=2837834 2026-02-24T22:10:46.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.453+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~CBZWMPY20XYH.0 size=269678 2026-02-24T22:10:46.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.453+0000 7f7e64854980 1 > prepared multipart upload key=EB04ETNYM087 upload=CBZWMPY20XYH tag=ZG5KE9JBWSVO type=write size=1318254 2026-02-24T22:10:46.458 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.457+0000 7f7e64854980 1 < canceled operation key=78GUTC5THI2P tag=DWQSWKJ4E01U type=del size=1505571 2026-02-24T22:10:46.458 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.457+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=E0KAZY3E3W0R type=del size=3434965 2026-02-24T22:10:46.459 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.457+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~87CV126IQZQ6.0 size=1048576 2026-02-24T22:10:46.460 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.457+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~87CV126IQZQ6.1 size=1048576 2026-02-24T22:10:46.460 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.457+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~87CV126IQZQ6.2 size=386734 2026-02-24T22:10:46.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.461+0000 7f7e64854980 1 < completed multipart upload key=D0MKGNRDW6L9 upload=87CV126IQZQ6 tag=B1TY0BB2X3KZ type=write size=3532462 2026-02-24T22:10:46.462 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.461+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=03W5IBU84NWC type=del size=1926283 2026-02-24T22:10:46.463 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.461+0000 7f7e64854980 1 < completed write operation key=EJCX06TE5RTZ tag=8A66WJDO25X5 type=write size=9115 2026-02-24T22:10:46.464 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.461+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~X1W4C03K8IHH.0 size=991587 2026-02-24T22:10:46.464 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.461+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=X1W4C03K8IHH tag=6HRHEADO5BA3 type=write size=2040163 2026-02-24T22:10:46.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.461+0000 7f7e64854980 1 < suggested update operation key=0B2V0MHWSOC9 tag=TM6K3074PX0K type=write size=774568 2026-02-24T22:10:46.466 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.465+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~0ISELPVXKSZC.0 size=1048576 2026-02-24T22:10:46.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.465+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~0ISELPVXKSZC.1 size=978234 2026-02-24T22:10:46.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.465+0000 7f7e64854980 1 > prepared multipart upload key=N3ZUNSI5K8HS upload=0ISELPVXKSZC tag=ZQU04DMCOWGA type=write size=3075386 2026-02-24T22:10:46.468 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.465+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~229062R3EDDM.0 size=196853 2026-02-24T22:10:46.469 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.465+0000 7f7e64854980 1 < completed multipart upload key=LSRPN33Z2B9N upload=229062R3EDDM tag=SAF332QKT7Q4 type=write size=1245429 2026-02-24T22:10:46.470 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.469+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~AF4S0W3C7Y2R.0 size=798553 2026-02-24T22:10:46.470 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.469+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=AF4S0W3C7Y2R tag=EUG2ZTV2YF3J type=write size=1847129 2026-02-24T22:10:46.470 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.469+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~FZNZWOCF72D6.0 size=525088 2026-02-24T22:10:46.472 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.469+0000 7f7e64854980 1 < completed multipart upload key=4Y3974PYYT78 upload=FZNZWOCF72D6 tag=SHP4B2S8SWBV type=write size=1573664 2026-02-24T22:10:46.473 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.469+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~X4THK7AS2671.0 size=730261 2026-02-24T22:10:46.473 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.469+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=X4THK7AS2671 tag=Q7PY5S48F0OD type=write size=1778837 2026-02-24T22:10:46.473 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.469+0000 7f7e64854980 1 < canceled part key=_multipart_Q7BP8FXTARDW.2~3W2MY8OW51D8.0 size=953278 2026-02-24T22:10:46.474 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.473+0000 7f7e64854980 1 < canceled multipart upload key=Q7BP8FXTARDW upload=3W2MY8OW51D8 tag=T7CGPXN53L7R type=write size=2001854 2026-02-24T22:10:46.475 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.473+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=ZLK9YJS7TVVJ type=del size=3901101 2026-02-24T22:10:46.476 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.473+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~1B5W3IBY3PYB.0 size=142743 2026-02-24T22:10:46.477 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.473+0000 7f7e64854980 1 < completed multipart upload key=HOHKR92SN25M upload=1B5W3IBY3PYB tag=E5WB51R7PBDD type=write size=1191319 2026-02-24T22:10:46.478 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.477+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~FK94QZJTVDJ0.0 size=941428 2026-02-24T22:10:46.478 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.477+0000 7f7e64854980 1 > prepared multipart upload key=3900SFJK1YQU upload=FK94QZJTVDJ0 tag=D74C3FO4SMV1 type=write size=1990004 2026-02-24T22:10:46.479 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.477+0000 7f7e64854980 1 < completed write operation key=NL60Q5QOSP8E tag=V9A3HLGCSBU3 type=write size=250070 2026-02-24T22:10:46.480 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.477+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~YXCWQ4KX2QSG.0 size=1048576 2026-02-24T22:10:46.480 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.477+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~YXCWQ4KX2QSG.1 size=1048576 2026-02-24T22:10:46.481 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.477+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~YXCWQ4KX2QSG.2 size=882667 2026-02-24T22:10:46.481 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.477+0000 7f7e64854980 1 > prepared multipart upload key=SGQYQ2TIVJPS upload=YXCWQ4KX2QSG tag=YO2NY1BH8ZSC type=write size=4028395 2026-02-24T22:10:46.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.481+0000 7f7e64854980 1 < completed delete operation key=HKOQRSF6XEXB tag=H5HXR4O962SS type=del 2026-02-24T22:10:46.483 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.481+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~VWGCVHA5II35.0 size=550048 2026-02-24T22:10:46.483 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.481+0000 7f7e64854980 1 > prepared multipart upload key=Z0OXREAUJVA8 upload=VWGCVHA5II35 tag=7YSUJRZ7KU3W type=write size=1598624 2026-02-24T22:10:46.483 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.481+0000 7f7e64854980 1 < completed delete operation key=UQLUT9ZAI61V tag=1C87YCAA94YU type=del 2026-02-24T22:10:46.484 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.481+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=NVJTQCL3X2D7 type=del size=3051715 2026-02-24T22:10:46.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.481+0000 7f7e64854980 1 < completed write operation key=ON16L53JUSTW tag=EVOP7IL2C4SW type=write size=767462 2026-02-24T22:10:46.486 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.485+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~YJSP1IHZLZLI.0 size=1048576 2026-02-24T22:10:46.487 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.485+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~YJSP1IHZLZLI.1 size=1048576 2026-02-24T22:10:46.488 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.485+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~YJSP1IHZLZLI.2 size=273156 2026-02-24T22:10:46.488 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.485+0000 7f7e64854980 1 > prepared multipart upload key=Q7BP8FXTARDW upload=YJSP1IHZLZLI tag=963GU6BSYJDB type=write size=3418884 2026-02-24T22:10:46.488 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.485+0000 7f7e64854980 -1 < failed to complete operation key=R2893NES3WSF tag=Z9PKNCJ0N1AL type=del size=309736: (22) Invalid argument 2026-02-24T22:10:46.489 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.489+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~SG9Y18GTBJK6.0 size=711870 2026-02-24T22:10:46.490 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.489+0000 7f7e64854980 1 > prepared multipart upload key=TF5B43G7KCYT upload=SG9Y18GTBJK6 tag=I2Z5ACLZ1HSO type=write size=1760446 2026-02-24T22:10:46.490 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.489+0000 7f7e64854980 1 < completed delete operation key=ZESKGBWZXM6O tag=0TGXAGG9IYYS type=del 2026-02-24T22:10:46.491 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.489+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=YQNNWFKWDUHX type=del size=3492847 2026-02-24T22:10:46.492 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.489+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~F8UIVJ8DOG5O.0 size=1048576 2026-02-24T22:10:46.492 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.489+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~F8UIVJ8DOG5O.1 size=299046 2026-02-24T22:10:46.493 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.493+0000 7f7e64854980 1 < completed multipart upload key=3900SFJK1YQU upload=F8UIVJ8DOG5O tag=4GMNNSDY1R8R type=write size=2396198 2026-02-24T22:10:46.495 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.493+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~4LOR1SQGI87N.0 size=275484 2026-02-24T22:10:46.495 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.493+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=4LOR1SQGI87N tag=8MP26H60MN6A type=write size=1324060 2026-02-24T22:10:46.495 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.493+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~CBZWMPY20XYH.0 size=269678 2026-02-24T22:10:46.496 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.493+0000 7f7e64854980 1 < completed multipart upload key=EB04ETNYM087 upload=CBZWMPY20XYH tag=ZG5KE9JBWSVO type=write size=1318254 2026-02-24T22:10:46.497 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.493+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=BJQJW6UCYI68 type=del size=502288 2026-02-24T22:10:46.498 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.497+0000 7f7e64854980 1 < completed delete operation key=78GUTC5THI2P tag=E0KAZY3E3W0R type=del 2026-02-24T22:10:46.499 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.497+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~UR50W2JG38V6.0 size=959246 2026-02-24T22:10:46.499 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.497+0000 7f7e64854980 1 > prepared multipart upload key=SGQYQ2TIVJPS upload=UR50W2JG38V6 tag=GCMRU2AA8Q98 type=write size=2007822 2026-02-24T22:10:46.500 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.497+0000 7f7e64854980 1 < completed delete operation key=0AXS7DY23UZU tag=03W5IBU84NWC type=del 2026-02-24T22:10:46.501 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.497+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~6FBGNNZBMQOO.0 size=1048576 2026-02-24T22:10:46.502 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.501+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~6FBGNNZBMQOO.1 size=729329 2026-02-24T22:10:46.502 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.501+0000 7f7e64854980 1 > prepared multipart upload key=NL60Q5QOSP8E upload=6FBGNNZBMQOO tag=PN900XXCD5ZU type=write size=2826481 2026-02-24T22:10:46.502 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.501+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~X1W4C03K8IHH.0 size=991587 2026-02-24T22:10:46.503 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.501+0000 7f7e64854980 1 < completed multipart upload key=HOHKR92SN25M upload=X1W4C03K8IHH tag=6HRHEADO5BA3 type=write size=2040163 2026-02-24T22:10:46.505 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.501+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~W86C09RZDCHZ.0 size=1048576 2026-02-24T22:10:46.505 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.501+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~W86C09RZDCHZ.1 size=1048576 2026-02-24T22:10:46.506 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.505+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~W86C09RZDCHZ.2 size=429744 2026-02-24T22:10:46.506 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.505+0000 7f7e64854980 1 > prepared multipart upload key=Y3L3CJ51C1WR upload=W86C09RZDCHZ tag=KV443C82D1KU type=write size=3575472 2026-02-24T22:10:46.506 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.505+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~0ISELPVXKSZC.0 size=1048576 2026-02-24T22:10:46.507 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.505+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~0ISELPVXKSZC.1 size=978234 2026-02-24T22:10:46.508 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.505+0000 7f7e64854980 1 < completed multipart upload key=N3ZUNSI5K8HS upload=0ISELPVXKSZC tag=ZQU04DMCOWGA type=write size=3075386 2026-02-24T22:10:46.509 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.505+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=P1KDAFTIPNJM type=del size=2117177 2026-02-24T22:10:46.509 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.509+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~AF4S0W3C7Y2R.0 size=798553 2026-02-24T22:10:46.510 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.509+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=AF4S0W3C7Y2R tag=EUG2ZTV2YF3J type=write size=1847129 2026-02-24T22:10:46.511 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.509+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=2D9HETCYTX03 type=del size=514192 2026-02-24T22:10:46.512 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.509+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~X4THK7AS2671.0 size=730261 2026-02-24T22:10:46.513 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.509+0000 7f7e64854980 1 < completed multipart upload key=HOHKR92SN25M upload=X4THK7AS2671 tag=Q7PY5S48F0OD type=write size=1778837 2026-02-24T22:10:46.513 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.513+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=8JYPD53EZYDZ type=del size=1315206 2026-02-24T22:10:46.514 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.513+0000 7f7e64854980 1 < completed delete operation key=EB04ETNYM087 tag=ZLK9YJS7TVVJ type=del 2026-02-24T22:10:46.515 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.513+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~ENKHZURSTMP0.0 size=580107 2026-02-24T22:10:46.516 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.513+0000 7f7e64854980 1 > prepared multipart upload key=4Y3974PYYT78 upload=ENKHZURSTMP0 tag=AY044UVBX8R0 type=write size=1628683 2026-02-24T22:10:46.516 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.513+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~FK94QZJTVDJ0.0 size=941428 2026-02-24T22:10:46.517 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.517+0000 7f7e64854980 1 < completed multipart upload key=3900SFJK1YQU upload=FK94QZJTVDJ0 tag=D74C3FO4SMV1 type=write size=1990004 2026-02-24T22:10:46.518 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.517+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=BJIP6NK9ZHVZ type=del size=2535168 2026-02-24T22:10:46.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.517+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~YXCWQ4KX2QSG.0 size=1048576 2026-02-24T22:10:46.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.517+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~YXCWQ4KX2QSG.1 size=1048576 2026-02-24T22:10:46.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.517+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~YXCWQ4KX2QSG.2 size=882667 2026-02-24T22:10:46.521 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.517+0000 7f7e64854980 1 < completed multipart upload key=SGQYQ2TIVJPS upload=YXCWQ4KX2QSG tag=YO2NY1BH8ZSC type=write size=4028395 2026-02-24T22:10:46.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.521+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=RK7ZTMIWDQEY type=del size=939922 2026-02-24T22:10:46.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.521+0000 7f7e64854980 1 < canceled part key=_multipart_Z0OXREAUJVA8.2~VWGCVHA5II35.0 size=550048 2026-02-24T22:10:46.523 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.521+0000 7f7e64854980 1 < canceled multipart upload key=Z0OXREAUJVA8 upload=VWGCVHA5II35 tag=7YSUJRZ7KU3W type=write size=1598624 2026-02-24T22:10:46.524 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.521+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~VGKEGYDUH0OW.0 size=759875 2026-02-24T22:10:46.524 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.521+0000 7f7e64854980 1 > prepared multipart upload key=GH7723GNI2WM upload=VGKEGYDUH0OW tag=JHV6TS35DLLU type=write size=1808451 2026-02-24T22:10:46.525 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.521+0000 7f7e64854980 1 < completed delete operation key=UQLUT9ZAI61V tag=NVJTQCL3X2D7 type=del 2026-02-24T22:10:46.526 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.525+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=FA1ZO9LD37AW type=del size=2389623 2026-02-24T22:10:46.526 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.525+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~YJSP1IHZLZLI.0 size=1048576 2026-02-24T22:10:46.527 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.525+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~YJSP1IHZLZLI.1 size=1048576 2026-02-24T22:10:46.527 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.525+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~YJSP1IHZLZLI.2 size=273156 2026-02-24T22:10:46.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.525+0000 7f7e64854980 1 < completed multipart upload key=Q7BP8FXTARDW upload=YJSP1IHZLZLI tag=963GU6BSYJDB type=write size=3418884 2026-02-24T22:10:46.530 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.529+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~NOZG8N8RUFMG.0 size=1048576 2026-02-24T22:10:46.530 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.529+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~NOZG8N8RUFMG.1 size=1048576 2026-02-24T22:10:46.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.529+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~NOZG8N8RUFMG.2 size=879162 2026-02-24T22:10:46.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.529+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=NOZG8N8RUFMG tag=S73ST1Y6XW1C type=write size=4024890 2026-02-24T22:10:46.532 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.529+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~SG9Y18GTBJK6.0 size=711870 2026-02-24T22:10:46.533 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.529+0000 7f7e64854980 1 < completed multipart upload key=TF5B43G7KCYT upload=SG9Y18GTBJK6 tag=I2Z5ACLZ1HSO type=write size=1760446 2026-02-24T22:10:46.533 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.533+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=DAFVH6JZPTXD type=del size=1900817 2026-02-24T22:10:46.534 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.533+0000 7f7e64854980 1 < completed delete operation key=TXRUQQ456DD8 tag=YQNNWFKWDUHX type=del 2026-02-24T22:10:46.535 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.533+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=LHAF3EA13CAW type=del size=1715090 2026-02-24T22:10:46.535 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.533+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~4LOR1SQGI87N.0 size=275484 2026-02-24T22:10:46.536 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.533+0000 7f7e64854980 1 < completed multipart upload key=0AXS7DY23UZU upload=4LOR1SQGI87N tag=8MP26H60MN6A type=write size=1324060 2026-02-24T22:10:46.538 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.537+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~9UNJ5AUUWVEG.0 size=1048576 2026-02-24T22:10:46.538 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.537+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~9UNJ5AUUWVEG.1 size=122609 2026-02-24T22:10:46.538 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.537+0000 7f7e64854980 1 > prepared multipart upload key=L60HQ7M3E3LL upload=9UNJ5AUUWVEG tag=8E0GGLE1J2VI type=write size=2219761 2026-02-24T22:10:46.539 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.537+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=BJQJW6UCYI68 type=del 2026-02-24T22:10:46.540 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.537+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~CH6L1UDKI106.0 size=851861 2026-02-24T22:10:46.540 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.537+0000 7f7e64854980 1 > prepared multipart upload key=L60HQ7M3E3LL upload=CH6L1UDKI106 tag=BKY2QMV90QII type=write size=1900437 2026-02-24T22:10:46.540 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.537+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~UR50W2JG38V6.0 size=959246 2026-02-24T22:10:46.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.537+0000 7f7e64854980 1 < completed multipart upload key=SGQYQ2TIVJPS upload=UR50W2JG38V6 tag=GCMRU2AA8Q98 type=write size=2007822 2026-02-24T22:10:46.542 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.541+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=1RAPQQXC85WY type=del size=2355768 2026-02-24T22:10:46.542 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.541+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~6FBGNNZBMQOO.0 size=1048576 2026-02-24T22:10:46.543 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.541+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~6FBGNNZBMQOO.1 size=729329 2026-02-24T22:10:46.544 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.541+0000 7f7e64854980 1 < completed multipart upload key=NL60Q5QOSP8E upload=6FBGNNZBMQOO tag=PN900XXCD5ZU type=write size=2826481 2026-02-24T22:10:46.544 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.541+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=X3NSHOG2IVR7 type=del size=3026370 2026-02-24T22:10:46.545 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.541+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~W86C09RZDCHZ.0 size=1048576 2026-02-24T22:10:46.545 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.545+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~W86C09RZDCHZ.1 size=1048576 2026-02-24T22:10:46.546 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.545+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~W86C09RZDCHZ.2 size=429744 2026-02-24T22:10:46.548 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.545+0000 7f7e64854980 1 < completed multipart upload key=Y3L3CJ51C1WR upload=W86C09RZDCHZ tag=KV443C82D1KU type=write size=3575472 2026-02-24T22:10:46.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.545+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=HFLLYF32L4YX type=del size=3690656 2026-02-24T22:10:46.550 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.549+0000 7f7e64854980 1 < completed delete operation key=LSRPN33Z2B9N tag=P1KDAFTIPNJM type=del 2026-02-24T22:10:46.550 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.549+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=OZPBR6C53R94 type=del size=1686635 2026-02-24T22:10:46.551 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.549+0000 7f7e64854980 1 < completed delete operation key=0B2V0MHWSOC9 tag=2D9HETCYTX03 type=del 2026-02-24T22:10:46.552 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.549+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=6NR7HTTG4MZ5 type=del size=2551876 2026-02-24T22:10:46.552 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.549+0000 7f7e64854980 1 < completed delete operation key=EB04ETNYM087 tag=8JYPD53EZYDZ type=del 2026-02-24T22:10:46.553 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.553+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~ARAJZMR6SAHV.0 size=1048576 2026-02-24T22:10:46.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.553+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~ARAJZMR6SAHV.1 size=991296 2026-02-24T22:10:46.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.553+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=ARAJZMR6SAHV tag=3ZJYT8A3YCN3 type=write size=3088448 2026-02-24T22:10:46.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.553+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~ENKHZURSTMP0.0 size=580107 2026-02-24T22:10:46.556 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.553+0000 7f7e64854980 1 < completed multipart upload key=4Y3974PYYT78 upload=ENKHZURSTMP0 tag=AY044UVBX8R0 type=write size=1628683 2026-02-24T22:10:46.556 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.553+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=1UOCMYJ0GXXN type=write size=69043 2026-02-24T22:10:46.557 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.557+0000 7f7e64854980 1 < suggested update operation key=Y3L3CJ51C1WR tag=BJIP6NK9ZHVZ type=del size=2535168 2026-02-24T22:10:46.558 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.557+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=0QOIS13Y3PB8 type=del size=294579 2026-02-24T22:10:46.559 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.557+0000 7f7e64854980 1 < canceled operation key=PKRHWM69GBJI tag=RK7ZTMIWDQEY type=del size=939922 2026-02-24T22:10:46.559 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.557+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=MN93XIEUFD1P type=del size=3645696 2026-02-24T22:10:46.560 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.557+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~VGKEGYDUH0OW.0 size=759875 2026-02-24T22:10:46.561 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.557+0000 7f7e64854980 1 < completed multipart upload key=GH7723GNI2WM upload=VGKEGYDUH0OW tag=JHV6TS35DLLU type=write size=1808451 2026-02-24T22:10:46.562 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.561+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~O40E1H4BEM2L.0 size=1048576 2026-02-24T22:10:46.562 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.561+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~O40E1H4BEM2L.1 size=1048576 2026-02-24T22:10:46.563 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.561+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~O40E1H4BEM2L.2 size=318324 2026-02-24T22:10:46.563 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.561+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=O40E1H4BEM2L tag=GSV6HGBKYGSA type=write size=3464052 2026-02-24T22:10:46.563 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.561+0000 7f7e64854980 1 < completed delete operation key=NL60Q5QOSP8E tag=FA1ZO9LD37AW type=del 2026-02-24T22:10:46.564 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.561+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=4LG4CQAUA3OZ type=del size=3254809 2026-02-24T22:10:46.564 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.561+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~NOZG8N8RUFMG.0 size=1048576 2026-02-24T22:10:46.564 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.561+0000 7f7e64854980 1 < canceled part key=_multipart_2KKQKHXO7K3K.2~NOZG8N8RUFMG.1 size=1048576 2026-02-24T22:10:46.565 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.561+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~NOZG8N8RUFMG.2 size=879162 2026-02-24T22:10:46.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.565+0000 7f7e64854980 1 < canceled multipart upload key=2KKQKHXO7K3K upload=NOZG8N8RUFMG tag=S73ST1Y6XW1C type=write size=4024890 2026-02-24T22:10:46.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.565+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=K2EQ5UH791ZP type=del size=538783 2026-02-24T22:10:46.567 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.565+0000 7f7e64854980 1 < completed delete operation key=0B2V0MHWSOC9 tag=DAFVH6JZPTXD type=del 2026-02-24T22:10:46.568 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.565+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~Q2Z9ZNV5XIFQ.0 size=681134 2026-02-24T22:10:46.568 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.565+0000 7f7e64854980 1 > prepared multipart upload key=Y8GMOUA977H3 upload=Q2Z9ZNV5XIFQ tag=5T4LA464NVU1 type=write size=1729710 2026-02-24T22:10:46.569 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.565+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=LHAF3EA13CAW type=del 2026-02-24T22:10:46.569 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.565+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=6SFTVNIBTNAY type=del size=1898885 2026-02-24T22:10:46.570 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.569+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~9UNJ5AUUWVEG.0 size=1048576 2026-02-24T22:10:46.570 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.569+0000 7f7e64854980 1 < canceled part key=_multipart_L60HQ7M3E3LL.2~9UNJ5AUUWVEG.1 size=122609 2026-02-24T22:10:46.571 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.569+0000 7f7e64854980 1 < canceled multipart upload key=L60HQ7M3E3LL upload=9UNJ5AUUWVEG tag=8E0GGLE1J2VI type=write size=2219761 2026-02-24T22:10:46.571 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.569+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=IY0VGCCFHHW1 type=write size=91640 2026-02-24T22:10:46.572 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.569+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~CH6L1UDKI106.0 size=851861 2026-02-24T22:10:46.573 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.569+0000 7f7e64854980 1 < completed multipart upload key=L60HQ7M3E3LL upload=CH6L1UDKI106 tag=BKY2QMV90QII type=write size=1900437 2026-02-24T22:10:46.573 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.573+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=T2D5AWCNODOJ type=write size=382970 2026-02-24T22:10:46.574 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.573+0000 7f7e64854980 1 < completed delete operation key=0B2V0MHWSOC9 tag=1RAPQQXC85WY type=del 2026-02-24T22:10:46.575 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.573+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=JNJ50CLW1MI6 type=del size=1184792 2026-02-24T22:10:46.575 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.573+0000 7f7e64854980 1 < completed delete operation key=4Y3974PYYT78 tag=X3NSHOG2IVR7 type=del 2026-02-24T22:10:46.576 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.573+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=562KMH01GF9W type=del size=2417934 2026-02-24T22:10:46.577 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.573+0000 7f7e64854980 1 < completed delete operation key=0B2V0MHWSOC9 tag=HFLLYF32L4YX type=del 2026-02-24T22:10:46.578 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.577+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~OJ395XYJZRDY.0 size=1048576 2026-02-24T22:10:46.578 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.577+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~OJ395XYJZRDY.1 size=1048576 2026-02-24T22:10:46.579 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.577+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~OJ395XYJZRDY.2 size=290041 2026-02-24T22:10:46.579 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.577+0000 7f7e64854980 1 > prepared multipart upload key=Q7BP8FXTARDW upload=OJ395XYJZRDY tag=VQ4QQFFI6FG2 type=write size=3435769 2026-02-24T22:10:46.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.577+0000 7f7e64854980 1 < completed delete operation key=4Y3974PYYT78 tag=OZPBR6C53R94 type=del 2026-02-24T22:10:46.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.577+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~UX2RUMI753HL.0 size=1048576 2026-02-24T22:10:46.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.581+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~UX2RUMI753HL.1 size=1048576 2026-02-24T22:10:46.582 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.581+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~UX2RUMI753HL.2 size=167897 2026-02-24T22:10:46.582 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.581+0000 7f7e64854980 1 > prepared multipart upload key=EB04ETNYM087 upload=UX2RUMI753HL tag=6596PBL808QK type=write size=3313625 2026-02-24T22:10:46.582 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.581+0000 7f7e64854980 1 < canceled operation key=EJCX06TE5RTZ tag=6NR7HTTG4MZ5 type=del size=2551876 2026-02-24T22:10:46.583 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.581+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=PYEN8KP5NE70 type=del size=1501559 2026-02-24T22:10:46.584 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.581+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~ARAJZMR6SAHV.0 size=1048576 2026-02-24T22:10:46.584 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.581+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~ARAJZMR6SAHV.1 size=991296 2026-02-24T22:10:46.585 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.585+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=ARAJZMR6SAHV tag=3ZJYT8A3YCN3 type=write size=3088448 2026-02-24T22:10:46.586 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.585+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=SDFCK1EBJ57V type=del size=549591 2026-02-24T22:10:46.587 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.585+0000 7f7e64854980 1 < completed write operation key=EB04ETNYM087 tag=1UOCMYJ0GXXN type=write size=69043 2026-02-24T22:10:46.588 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.585+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=H23VWS1VJZQW type=del size=4040364 2026-02-24T22:10:46.588 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.585+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=0QOIS13Y3PB8 type=del 2026-02-24T22:10:46.589 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.589+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~KW21TJFSK77I.0 size=1048576 2026-02-24T22:10:46.590 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.589+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~KW21TJFSK77I.1 size=404653 2026-02-24T22:10:46.590 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.589+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=KW21TJFSK77I tag=AEYTHCKJ2M1L type=write size=2501805 2026-02-24T22:10:46.591 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.589+0000 7f7e64854980 1 < completed delete operation key=4Y3974PYYT78 tag=MN93XIEUFD1P type=del 2026-02-24T22:10:46.592 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.589+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~82O2I7VN7UAI.0 size=1048576 2026-02-24T22:10:46.592 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.589+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~82O2I7VN7UAI.1 size=1048576 2026-02-24T22:10:46.593 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.589+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~82O2I7VN7UAI.2 size=704999 2026-02-24T22:10:46.593 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.589+0000 7f7e64854980 1 > prepared multipart upload key=L60HQ7M3E3LL upload=82O2I7VN7UAI tag=CEO4CFA0OJ83 type=write size=3850727 2026-02-24T22:10:46.593 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.589+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~O40E1H4BEM2L.0 size=1048576 2026-02-24T22:10:46.594 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.593+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~O40E1H4BEM2L.1 size=1048576 2026-02-24T22:10:46.594 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.593+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~O40E1H4BEM2L.2 size=318324 2026-02-24T22:10:46.596 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.593+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=O40E1H4BEM2L tag=GSV6HGBKYGSA type=write size=3464052 2026-02-24T22:10:46.596 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.593+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=OO2ZT62469FX type=del size=2376405 2026-02-24T22:10:46.597 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.593+0000 7f7e64854980 1 < completed delete operation key=36MKR2OZN7K9 tag=4LG4CQAUA3OZ type=del 2026-02-24T22:10:46.598 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.597+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=FGB8AMCXH5SZ type=del size=563876 2026-02-24T22:10:46.599 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.597+0000 7f7e64854980 1 < canceled operation key=Z0OXREAUJVA8 tag=K2EQ5UH791ZP type=del size=538783 2026-02-24T22:10:46.599 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.597+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=OWAX8AKJ4I9P type=del size=2994307 2026-02-24T22:10:46.600 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.597+0000 7f7e64854980 1 < canceled part key=_multipart_Y8GMOUA977H3.2~Q2Z9ZNV5XIFQ.0 size=681134 2026-02-24T22:10:46.601 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.597+0000 7f7e64854980 1 < canceled multipart upload key=Y8GMOUA977H3 upload=Q2Z9ZNV5XIFQ tag=5T4LA464NVU1 type=write size=1729710 2026-02-24T22:10:46.601 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.601+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=90XXOR5JCKSM type=del size=1720405 2026-02-24T22:10:46.602 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.601+0000 7f7e64854980 1 < completed delete operation key=LSRPN33Z2B9N tag=6SFTVNIBTNAY type=del 2026-02-24T22:10:46.604 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.601+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~XKR8OMB2LG2R.0 size=1048576 2026-02-24T22:10:46.604 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.601+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~XKR8OMB2LG2R.1 size=211459 2026-02-24T22:10:46.604 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.601+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=XKR8OMB2LG2R tag=BKOU5VTTY23X type=write size=2308611 2026-02-24T22:10:46.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.601+0000 7f7e64854980 1 < completed write operation key=4Y3974PYYT78 tag=IY0VGCCFHHW1 type=write size=91640 2026-02-24T22:10:46.606 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.605+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=7PGYEZE2BZUP type=write size=1036086 2026-02-24T22:10:46.607 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.605+0000 7f7e64854980 1 < completed write operation key=ON16L53JUSTW tag=T2D5AWCNODOJ type=write size=382970 2026-02-24T22:10:46.607 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.605+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=X3O7K8V6D2FX type=write size=673279 2026-02-24T22:10:46.608 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.605+0000 7f7e64854980 1 < completed delete operation key=36MKR2OZN7K9 tag=JNJ50CLW1MI6 type=del 2026-02-24T22:10:46.610 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.609+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=4OUERVI9S0ER type=write size=438423 2026-02-24T22:10:46.611 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.609+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=562KMH01GF9W type=del 2026-02-24T22:10:46.612 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.609+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~0021KPL97RD1.0 size=1048576 2026-02-24T22:10:46.613 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.609+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~0021KPL97RD1.1 size=1048576 2026-02-24T22:10:46.613 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.613+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~0021KPL97RD1.2 size=113682 2026-02-24T22:10:46.613 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.613+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=0021KPL97RD1 tag=CWP1T30FQXA2 type=write size=3259410 2026-02-24T22:10:46.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.613+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~OJ395XYJZRDY.0 size=1048576 2026-02-24T22:10:46.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.613+0000 7f7e64854980 1 < canceled part key=_multipart_Q7BP8FXTARDW.2~OJ395XYJZRDY.1 size=1048576 2026-02-24T22:10:46.615 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.613+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~OJ395XYJZRDY.2 size=290041 2026-02-24T22:10:46.616 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.613+0000 7f7e64854980 1 < canceled multipart upload key=Q7BP8FXTARDW upload=OJ395XYJZRDY tag=VQ4QQFFI6FG2 type=write size=3435769 2026-02-24T22:10:46.617 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.613+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=MQ407PIINXGR type=del size=961594 2026-02-24T22:10:46.617 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.617+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~UX2RUMI753HL.0 size=1048576 2026-02-24T22:10:46.618 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.617+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~UX2RUMI753HL.1 size=1048576 2026-02-24T22:10:46.618 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.617+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~UX2RUMI753HL.2 size=167897 2026-02-24T22:10:46.619 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.617+0000 7f7e64854980 1 < completed multipart upload key=EB04ETNYM087 upload=UX2RUMI753HL tag=6596PBL808QK type=write size=3313625 2026-02-24T22:10:46.620 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.617+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=7XCK8OMTMLG4 type=write size=706640 2026-02-24T22:10:46.620 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.617+0000 7f7e64854980 1 < canceled operation key=78GUTC5THI2P tag=PYEN8KP5NE70 type=del size=1501559 2026-02-24T22:10:46.621 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.621+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~HR1X0Y15143V.0 size=1048576 2026-02-24T22:10:46.622 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.621+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~HR1X0Y15143V.1 size=1048576 2026-02-24T22:10:46.622 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.621+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~HR1X0Y15143V.2 size=374296 2026-02-24T22:10:46.622 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.621+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=HR1X0Y15143V tag=WT5VNYY3JBP4 type=write size=3520024 2026-02-24T22:10:46.623 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.621+0000 7f7e64854980 1 < completed delete operation key=N3ZUNSI5K8HS tag=SDFCK1EBJ57V type=del 2026-02-24T22:10:46.624 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.621+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~44U08JW5KGV9.0 size=1048576 2026-02-24T22:10:46.624 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.621+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~44U08JW5KGV9.1 size=858498 2026-02-24T22:10:46.625 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.621+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=44U08JW5KGV9 tag=YC9Q45Y24K40 type=write size=2955650 2026-02-24T22:10:46.625 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.621+0000 7f7e64854980 1 < completed delete operation key=A6PH77Q78VEN tag=H23VWS1VJZQW type=del 2026-02-24T22:10:46.626 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.625+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=0ZTB779GICYA type=del size=4164174 2026-02-24T22:10:46.626 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.625+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~KW21TJFSK77I.0 size=1048576 2026-02-24T22:10:46.627 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.625+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~KW21TJFSK77I.1 size=404653 2026-02-24T22:10:46.628 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.625+0000 7f7e64854980 1 < completed multipart upload key=HOHKR92SN25M upload=KW21TJFSK77I tag=AEYTHCKJ2M1L type=write size=2501805 2026-02-24T22:10:46.628 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.625+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=C9CURB43W72V type=write size=192323 2026-02-24T22:10:46.629 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.625+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~82O2I7VN7UAI.0 size=1048576 2026-02-24T22:10:46.629 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.629+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~82O2I7VN7UAI.1 size=1048576 2026-02-24T22:10:46.630 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.629+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~82O2I7VN7UAI.2 size=704999 2026-02-24T22:10:46.631 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.629+0000 7f7e64854980 1 < completed multipart upload key=L60HQ7M3E3LL upload=82O2I7VN7UAI tag=CEO4CFA0OJ83 type=write size=3850727 2026-02-24T22:10:46.632 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.629+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~3E7ICZVWO9G3.0 size=1048576 2026-02-24T22:10:46.633 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.629+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~3E7ICZVWO9G3.1 size=1048576 2026-02-24T22:10:46.633 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.633+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~3E7ICZVWO9G3.2 size=69925 2026-02-24T22:10:46.633 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.633+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=3E7ICZVWO9G3 tag=5SSUMDKZE3Q8 type=write size=3215653 2026-02-24T22:10:46.634 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.633+0000 7f7e64854980 1 < suggested remove operation key=L60HQ7M3E3LL tag=OO2ZT62469FX type=del 2026-02-24T22:10:46.636 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.633+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~2SAIZ2TC1254.0 size=579678 2026-02-24T22:10:46.636 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.633+0000 7f7e64854980 1 > prepared multipart upload key=Y8GMOUA977H3 upload=2SAIZ2TC1254 tag=9B0HQW8BLRBY type=write size=1628254 2026-02-24T22:10:46.636 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.633+0000 7f7e64854980 1 < completed delete operation key=78GUTC5THI2P tag=FGB8AMCXH5SZ type=del 2026-02-24T22:10:46.637 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.633+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~VKCO332USS2H.0 size=1048576 2026-02-24T22:10:46.638 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.637+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~VKCO332USS2H.1 size=1048576 2026-02-24T22:10:46.638 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.637+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~VKCO332USS2H.2 size=891148 2026-02-24T22:10:46.638 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.637+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=VKCO332USS2H tag=PU8UKGW6G2TT type=write size=4036876 2026-02-24T22:10:46.639 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.637+0000 7f7e64854980 1 < completed delete operation key=D0MKGNRDW6L9 tag=OWAX8AKJ4I9P type=del 2026-02-24T22:10:46.640 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.637+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~FIDB46JVK2KY.0 size=1048576 2026-02-24T22:10:46.640 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.637+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~FIDB46JVK2KY.1 size=295301 2026-02-24T22:10:46.640 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.637+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=FIDB46JVK2KY tag=O75P33D2KIIQ type=write size=2392453 2026-02-24T22:10:46.641 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.637+0000 7f7e64854980 1 < canceled operation key=TXRUQQ456DD8 tag=90XXOR5JCKSM type=del size=1720405 2026-02-24T22:10:46.641 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.641+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=ATUOVSHM1JNM type=del size=4139161 2026-02-24T22:10:46.642 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.641+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~XKR8OMB2LG2R.0 size=1048576 2026-02-24T22:10:46.642 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.641+0000 7f7e64854980 1 < canceled part key=_multipart_2KKQKHXO7K3K.2~XKR8OMB2LG2R.1 size=211459 2026-02-24T22:10:46.643 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.641+0000 7f7e64854980 1 < canceled multipart upload key=2KKQKHXO7K3K upload=XKR8OMB2LG2R tag=BKOU5VTTY23X type=write size=2308611 2026-02-24T22:10:46.644 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.641+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~PCZL323QEI3Y.0 size=1048576 2026-02-24T22:10:46.644 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.641+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~PCZL323QEI3Y.1 size=631673 2026-02-24T22:10:46.644 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.641+0000 7f7e64854980 1 > prepared multipart upload key=ON16L53JUSTW upload=PCZL323QEI3Y tag=48OFYEPMJEM2 type=write size=2728825 2026-02-24T22:10:46.645 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.641+0000 7f7e64854980 1 < completed write operation key=TF5B43G7KCYT tag=7PGYEZE2BZUP type=write size=1036086 2026-02-24T22:10:46.645 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.645+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=AVNAMREQ6D6I type=write size=650650 2026-02-24T22:10:46.647 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.645+0000 7f7e64854980 1 < suggested update operation key=N3ZUNSI5K8HS tag=X3O7K8V6D2FX type=write size=673279 2026-02-24T22:10:46.647 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.645+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=TW1E8D26P3V6 type=del size=44448 2026-02-24T22:10:46.648 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.645+0000 7f7e64854980 1 < completed write operation key=TF5B43G7KCYT tag=4OUERVI9S0ER type=write size=438423 2026-02-24T22:10:46.648 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.645+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=3XEOMY042XKJ type=del size=957146 2026-02-24T22:10:46.649 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.645+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~0021KPL97RD1.0 size=1048576 2026-02-24T22:10:46.649 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.645+0000 7f7e64854980 1 < canceled part key=_multipart_ZESKGBWZXM6O.2~0021KPL97RD1.1 size=1048576 2026-02-24T22:10:46.649 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.649+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~0021KPL97RD1.2 size=113682 2026-02-24T22:10:46.650 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.649+0000 7f7e64854980 1 < canceled multipart upload key=ZESKGBWZXM6O upload=0021KPL97RD1 tag=CWP1T30FQXA2 type=write size=3259410 2026-02-24T22:10:46.651 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.649+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=652DLRRX1HYF type=del size=1915796 2026-02-24T22:10:46.652 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.649+0000 7f7e64854980 1 < completed delete operation key=HKOQRSF6XEXB tag=MQ407PIINXGR type=del 2026-02-24T22:10:46.652 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.649+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=2ZWMGZ9IFXBC type=del size=2843353 2026-02-24T22:10:46.653 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.649+0000 7f7e64854980 1 < completed write operation key=36MKR2OZN7K9 tag=7XCK8OMTMLG4 type=write size=706640 2026-02-24T22:10:46.653 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.653+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=WF5FC5C0ZRHN type=del size=2693438 2026-02-24T22:10:46.654 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.653+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~HR1X0Y15143V.0 size=1048576 2026-02-24T22:10:46.655 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.653+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~HR1X0Y15143V.1 size=1048576 2026-02-24T22:10:46.655 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.653+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~HR1X0Y15143V.2 size=374296 2026-02-24T22:10:46.657 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.653+0000 7f7e64854980 1 < completed multipart upload key=CI8DA17DX8LU upload=HR1X0Y15143V tag=WT5VNYY3JBP4 type=write size=3520024 2026-02-24T22:10:46.657 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.657+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=5SCXY5B52VGU type=del size=2156595 2026-02-24T22:10:46.658 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.657+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~44U08JW5KGV9.0 size=1048576 2026-02-24T22:10:46.659 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.657+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~44U08JW5KGV9.1 size=858498 2026-02-24T22:10:46.660 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.657+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=44U08JW5KGV9 tag=YC9Q45Y24K40 type=write size=2955650 2026-02-24T22:10:46.661 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.657+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=BIC3KHHNZXZ6 type=write size=707123 2026-02-24T22:10:46.661 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.661+0000 7f7e64854980 1 < completed delete operation key=ZESKGBWZXM6O tag=0ZTB779GICYA type=del 2026-02-24T22:10:46.663 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.661+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~3GICK5TR1T0P.0 size=1048576 2026-02-24T22:10:46.663 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.661+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~3GICK5TR1T0P.1 size=1048576 2026-02-24T22:10:46.664 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.661+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~3GICK5TR1T0P.2 size=228063 2026-02-24T22:10:46.664 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.661+0000 7f7e64854980 1 > prepared multipart upload key=PKRHWM69GBJI upload=3GICK5TR1T0P tag=GAZFYTGG7G8X type=write size=3373791 2026-02-24T22:10:46.665 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.661+0000 7f7e64854980 1 < completed write operation key=LSRPN33Z2B9N tag=C9CURB43W72V type=write size=192323 2026-02-24T22:10:46.666 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.665+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=Y0IUTKB322JW type=write size=166860 2026-02-24T22:10:46.666 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.665+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~3E7ICZVWO9G3.0 size=1048576 2026-02-24T22:10:46.667 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.665+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~3E7ICZVWO9G3.1 size=1048576 2026-02-24T22:10:46.667 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.665+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~3E7ICZVWO9G3.2 size=69925 2026-02-24T22:10:46.670 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.669+0000 7f7e64854980 1 < completed multipart upload key=0AXS7DY23UZU upload=3E7ICZVWO9G3 tag=5SSUMDKZE3Q8 type=write size=3215653 2026-02-24T22:10:46.670 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.669+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=CURPHGJH6CPP type=del size=699760 2026-02-24T22:10:46.671 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.669+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~2SAIZ2TC1254.0 size=579678 2026-02-24T22:10:46.673 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.669+0000 7f7e64854980 1 < completed multipart upload key=Y8GMOUA977H3 upload=2SAIZ2TC1254 tag=9B0HQW8BLRBY type=write size=1628254 2026-02-24T22:10:46.673 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.673+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=IRLJQYZWQ0D4 type=del size=3211313 2026-02-24T22:10:46.674 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.673+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~VKCO332USS2H.0 size=1048576 2026-02-24T22:10:46.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.673+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~VKCO332USS2H.1 size=1048576 2026-02-24T22:10:46.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.673+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~VKCO332USS2H.2 size=891148 2026-02-24T22:10:46.677 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.673+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=VKCO332USS2H tag=PU8UKGW6G2TT type=write size=4036876 2026-02-24T22:10:46.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.677+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~DOQZS1Y734BS.0 size=1048576 2026-02-24T22:10:46.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.677+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~DOQZS1Y734BS.1 size=916640 2026-02-24T22:10:46.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.677+0000 7f7e64854980 1 > prepared multipart upload key=3900SFJK1YQU upload=DOQZS1Y734BS tag=X0E0N2NIVNFC type=write size=3013792 2026-02-24T22:10:46.679 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.677+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~FIDB46JVK2KY.0 size=1048576 2026-02-24T22:10:46.680 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.677+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~FIDB46JVK2KY.1 size=295301 2026-02-24T22:10:46.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.677+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=FIDB46JVK2KY tag=O75P33D2KIIQ type=write size=2392453 2026-02-24T22:10:46.682 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.681+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=D9TEDODA3H30 type=del size=87642 2026-02-24T22:10:46.683 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.681+0000 7f7e64854980 1 < completed delete operation key=Z0OXREAUJVA8 tag=ATUOVSHM1JNM type=del 2026-02-24T22:10:46.684 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.681+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~JQ8RO3NOBHU5.0 size=1048576 2026-02-24T22:10:46.685 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.681+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~JQ8RO3NOBHU5.1 size=1048576 2026-02-24T22:10:46.685 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.681+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~JQ8RO3NOBHU5.2 size=75781 2026-02-24T22:10:46.685 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.681+0000 7f7e64854980 1 > prepared multipart upload key=SGQYQ2TIVJPS upload=JQ8RO3NOBHU5 tag=C51SWNJ5L2C0 type=write size=3221509 2026-02-24T22:10:46.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.685+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~PCZL323QEI3Y.0 size=1048576 2026-02-24T22:10:46.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.685+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~PCZL323QEI3Y.1 size=631673 2026-02-24T22:10:46.688 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.685+0000 7f7e64854980 1 < completed multipart upload key=ON16L53JUSTW upload=PCZL323QEI3Y tag=48OFYEPMJEM2 type=write size=2728825 2026-02-24T22:10:46.689 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.685+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~QZFDXKZ3WS1Y.0 size=1048576 2026-02-24T22:10:46.689 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.689+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~QZFDXKZ3WS1Y.1 size=192604 2026-02-24T22:10:46.689 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.689+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=QZFDXKZ3WS1Y tag=ZAA9EIQG4J0Y type=write size=2289756 2026-02-24T22:10:46.690 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.689+0000 7f7e64854980 1 < completed write operation key=SGQYQ2TIVJPS tag=AVNAMREQ6D6I type=write size=650650 2026-02-24T22:10:46.691 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.689+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=KNS0YORLLMDI type=del size=538000 2026-02-24T22:10:46.691 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.689+0000 7f7e64854980 1 < completed delete operation key=NL60Q5QOSP8E tag=TW1E8D26P3V6 type=del 2026-02-24T22:10:46.692 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.689+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=8ZYM134L20U7 type=del size=2834228 2026-02-24T22:10:46.693 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.689+0000 7f7e64854980 1 < completed delete operation key=N3ZUNSI5K8HS tag=3XEOMY042XKJ type=del 2026-02-24T22:10:46.694 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.693+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~FDM3PWAD318B.0 size=1048576 2026-02-24T22:10:46.695 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.693+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~FDM3PWAD318B.1 size=208656 2026-02-24T22:10:46.695 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.693+0000 7f7e64854980 1 > prepared multipart upload key=0B2V0MHWSOC9 upload=FDM3PWAD318B tag=YO3ACHZZWSEB type=write size=2305808 2026-02-24T22:10:46.696 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.693+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=652DLRRX1HYF type=del 2026-02-24T22:10:46.697 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.693+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~86UERD9KZHA8.0 size=1038890 2026-02-24T22:10:46.697 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.693+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=86UERD9KZHA8 tag=ZKX5D1TPJSNZ type=write size=2087466 2026-02-24T22:10:46.698 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.697+0000 7f7e64854980 1 < completed delete operation key=ZESKGBWZXM6O tag=2ZWMGZ9IFXBC type=del 2026-02-24T22:10:46.699 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.697+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~1NRK2UAGM8B2.0 size=1048576 2026-02-24T22:10:46.700 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.697+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~1NRK2UAGM8B2.1 size=645784 2026-02-24T22:10:46.700 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.697+0000 7f7e64854980 1 > prepared multipart upload key=36MKR2OZN7K9 upload=1NRK2UAGM8B2 tag=2CE6QTT1R513 type=write size=2742936 2026-02-24T22:10:46.700 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.697+0000 7f7e64854980 1 < completed delete operation key=R2893NES3WSF tag=WF5FC5C0ZRHN type=del 2026-02-24T22:10:46.701 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.697+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=AX3D0BBD7395 type=del size=125652 2026-02-24T22:10:46.702 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.701+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=5SCXY5B52VGU type=del 2026-02-24T22:10:46.703 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.701+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~QG6G8EAJ879R.0 size=1048576 2026-02-24T22:10:46.704 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.701+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~QG6G8EAJ879R.1 size=1048576 2026-02-24T22:10:46.704 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.701+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~QG6G8EAJ879R.2 size=457284 2026-02-24T22:10:46.705 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.701+0000 7f7e64854980 1 > prepared multipart upload key=EB04ETNYM087 upload=QG6G8EAJ879R tag=225GM5MU4PWL type=write size=3603012 2026-02-24T22:10:46.705 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.705+0000 7f7e64854980 1 < completed write operation key=EJCX06TE5RTZ tag=BIC3KHHNZXZ6 type=write size=707123 2026-02-24T22:10:46.706 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.705+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=2ZDY7MX2CC1Q type=write size=153223 2026-02-24T22:10:46.707 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.705+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~3GICK5TR1T0P.0 size=1048576 2026-02-24T22:10:46.707 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.705+0000 7f7e64854980 1 < canceled part key=_multipart_PKRHWM69GBJI.2~3GICK5TR1T0P.1 size=1048576 2026-02-24T22:10:46.707 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.705+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~3GICK5TR1T0P.2 size=228063 2026-02-24T22:10:46.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.705+0000 7f7e64854980 1 < canceled multipart upload key=PKRHWM69GBJI upload=3GICK5TR1T0P tag=GAZFYTGG7G8X type=write size=3373791 2026-02-24T22:10:46.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.709+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=6D7MJQSIFUAY type=del size=2644972 2026-02-24T22:10:46.710 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.709+0000 7f7e64854980 1 < completed write operation key=UQLUT9ZAI61V tag=Y0IUTKB322JW type=write size=166860 2026-02-24T22:10:46.711 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.709+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=T4GSZ524WIEW type=del size=2488500 2026-02-24T22:10:46.713 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.709+0000 7f7e64854980 1 < suggested update operation key=36MKR2OZN7K9 tag=CURPHGJH6CPP type=del size=699760 2026-02-24T22:10:46.714 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.713+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~CCBSJ9RDC9VJ.0 size=1048576 2026-02-24T22:10:46.715 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.713+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~CCBSJ9RDC9VJ.1 size=1048576 2026-02-24T22:10:46.715 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.713+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~CCBSJ9RDC9VJ.2 size=1029363 2026-02-24T22:10:46.715 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.713+0000 7f7e64854980 1 > prepared multipart upload key=Q7BP8FXTARDW upload=CCBSJ9RDC9VJ tag=PG8HXJ5I3WCV type=write size=4175091 2026-02-24T22:10:46.716 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.713+0000 7f7e64854980 1 < completed delete operation key=2KKQKHXO7K3K tag=IRLJQYZWQ0D4 type=del 2026-02-24T22:10:46.717 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.713+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=U0I08SMOSNGU type=del size=2260227 2026-02-24T22:10:46.718 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.717+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~DOQZS1Y734BS.0 size=1048576 2026-02-24T22:10:46.718 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.717+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~DOQZS1Y734BS.1 size=916640 2026-02-24T22:10:46.720 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.717+0000 7f7e64854980 1 < completed multipart upload key=3900SFJK1YQU upload=DOQZS1Y734BS tag=X0E0N2NIVNFC type=write size=3013792 2026-02-24T22:10:46.720 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.717+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=MP50HNO0AU5T type=del size=2386544 2026-02-24T22:10:46.721 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.721+0000 7f7e64854980 1 < completed delete operation key=PKRHWM69GBJI tag=D9TEDODA3H30 type=del 2026-02-24T22:10:46.722 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.721+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=TBELRQDVHRHI type=del size=9889 2026-02-24T22:10:46.723 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.721+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~JQ8RO3NOBHU5.0 size=1048576 2026-02-24T22:10:46.724 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.721+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~JQ8RO3NOBHU5.1 size=1048576 2026-02-24T22:10:46.724 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.721+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~JQ8RO3NOBHU5.2 size=75781 2026-02-24T22:10:46.726 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.725+0000 7f7e64854980 1 < completed multipart upload key=SGQYQ2TIVJPS upload=JQ8RO3NOBHU5 tag=C51SWNJ5L2C0 type=write size=3221509 2026-02-24T22:10:46.727 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.725+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~LPP30VH81KVG.0 size=1048576 2026-02-24T22:10:46.728 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.725+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~LPP30VH81KVG.1 size=1048576 2026-02-24T22:10:46.728 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.725+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~LPP30VH81KVG.2 size=1043170 2026-02-24T22:10:46.728 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.725+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=LPP30VH81KVG tag=QR9A4WM10EWR type=write size=4188898 2026-02-24T22:10:46.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.725+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~QZFDXKZ3WS1Y.0 size=1048576 2026-02-24T22:10:46.730 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.729+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~QZFDXKZ3WS1Y.1 size=192604 2026-02-24T22:10:46.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.729+0000 7f7e64854980 1 < completed multipart upload key=EJCX06TE5RTZ upload=QZFDXKZ3WS1Y tag=ZAA9EIQG4J0Y type=write size=2289756 2026-02-24T22:10:46.732 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.729+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=TY0K1NMS0SOE type=del size=476314 2026-02-24T22:10:46.733 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.729+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=KNS0YORLLMDI type=del 2026-02-24T22:10:46.733 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.733+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=D9NFT32LVIHF type=write size=344060 2026-02-24T22:10:46.734 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.733+0000 7f7e64854980 1 < completed delete operation key=EJCX06TE5RTZ tag=8ZYM134L20U7 type=del 2026-02-24T22:10:46.736 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.733+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~4Z9AKE0RYI1K.0 size=666204 2026-02-24T22:10:46.736 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.733+0000 7f7e64854980 1 > prepared multipart upload key=ON16L53JUSTW upload=4Z9AKE0RYI1K tag=TW8HMP6A7FJ4 type=write size=1714780 2026-02-24T22:10:46.737 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.733+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~FDM3PWAD318B.0 size=1048576 2026-02-24T22:10:46.737 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.733+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~FDM3PWAD318B.1 size=208656 2026-02-24T22:10:46.740 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.737+0000 7f7e64854980 1 < completed multipart upload key=0B2V0MHWSOC9 upload=FDM3PWAD318B tag=YO3ACHZZWSEB type=write size=2305808 2026-02-24T22:10:46.740 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.737+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=ZGQJX8FBCKAI type=del size=3540467 2026-02-24T22:10:46.741 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.741+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~86UERD9KZHA8.0 size=1038890 2026-02-24T22:10:46.743 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.741+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=86UERD9KZHA8 tag=ZKX5D1TPJSNZ type=write size=2087466 2026-02-24T22:10:46.744 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.741+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~STCYUP1M8L3T.0 size=1048576 2026-02-24T22:10:46.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.741+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~STCYUP1M8L3T.1 size=41608 2026-02-24T22:10:46.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.741+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=STCYUP1M8L3T tag=1MNNODE9158L type=write size=2138760 2026-02-24T22:10:46.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.745+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~1NRK2UAGM8B2.0 size=1048576 2026-02-24T22:10:46.746 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.745+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~1NRK2UAGM8B2.1 size=645784 2026-02-24T22:10:46.747 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.745+0000 7f7e64854980 1 < completed multipart upload key=36MKR2OZN7K9 upload=1NRK2UAGM8B2 tag=2CE6QTT1R513 type=write size=2742936 2026-02-24T22:10:46.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.745+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~EYKT364IBFEM.0 size=1048576 2026-02-24T22:10:46.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.749+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~EYKT364IBFEM.1 size=768431 2026-02-24T22:10:46.750 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.749+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=EYKT364IBFEM tag=EVYZQ9KSWUH7 type=write size=2865583 2026-02-24T22:10:46.751 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.749+0000 7f7e64854980 1 < suggested remove operation key=L60HQ7M3E3LL tag=AX3D0BBD7395 type=del 2026-02-24T22:10:46.752 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.749+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~XPGWB954DY1R.0 size=1048576 2026-02-24T22:10:46.753 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.749+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~XPGWB954DY1R.1 size=1048576 2026-02-24T22:10:46.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.753+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~XPGWB954DY1R.2 size=590830 2026-02-24T22:10:46.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.753+0000 7f7e64854980 1 > prepared multipart upload key=N3ZUNSI5K8HS upload=XPGWB954DY1R tag=O079RWFFIYM5 type=write size=3736558 2026-02-24T22:10:46.755 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.753+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~QG6G8EAJ879R.0 size=1048576 2026-02-24T22:10:46.755 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.753+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~QG6G8EAJ879R.1 size=1048576 2026-02-24T22:10:46.756 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.753+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~QG6G8EAJ879R.2 size=457284 2026-02-24T22:10:46.757 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.753+0000 7f7e64854980 1 < completed multipart upload key=EB04ETNYM087 upload=QG6G8EAJ879R tag=225GM5MU4PWL type=write size=3603012 2026-02-24T22:10:46.758 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.757+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~MSND42SQVMZX.0 size=1048576 2026-02-24T22:10:46.759 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.757+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~MSND42SQVMZX.1 size=1048576 2026-02-24T22:10:46.760 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.757+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~MSND42SQVMZX.2 size=419840 2026-02-24T22:10:46.760 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.757+0000 7f7e64854980 1 > prepared multipart upload key=HKOQRSF6XEXB upload=MSND42SQVMZX tag=UWGOYNDTZGKA type=write size=3565568 2026-02-24T22:10:46.761 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.757+0000 7f7e64854980 1 < completed write operation key=78GUTC5THI2P tag=2ZDY7MX2CC1Q type=write size=153223 2026-02-24T22:10:46.762 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.761+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~W9SS7GKBB88B.0 size=1048576 2026-02-24T22:10:46.763 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.761+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~W9SS7GKBB88B.1 size=174538 2026-02-24T22:10:46.763 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.761+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=W9SS7GKBB88B tag=APBK2QVBFHZ5 type=write size=2271690 2026-02-24T22:10:46.764 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.761+0000 7f7e64854980 1 < suggested update operation key=EJCX06TE5RTZ tag=6D7MJQSIFUAY type=del size=2644972 2026-02-24T22:10:46.766 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.765+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~PMLBPVW15KR2.0 size=620458 2026-02-24T22:10:46.766 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.765+0000 7f7e64854980 1 > prepared multipart upload key=TF5B43G7KCYT upload=PMLBPVW15KR2 tag=4II2ZJCBWK1B type=write size=1669034 2026-02-24T22:10:46.766 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.765+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=T4GSZ524WIEW type=del 2026-02-24T22:10:46.767 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.765+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=3GH8QE94ZLB9 type=del size=2424332 2026-02-24T22:10:46.768 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.765+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~CCBSJ9RDC9VJ.0 size=1048576 2026-02-24T22:10:46.769 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.765+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~CCBSJ9RDC9VJ.1 size=1048576 2026-02-24T22:10:46.769 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.769+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~CCBSJ9RDC9VJ.2 size=1029363 2026-02-24T22:10:46.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.769+0000 7f7e64854980 1 < completed multipart upload key=Q7BP8FXTARDW upload=CCBSJ9RDC9VJ tag=PG8HXJ5I3WCV type=write size=4175091 2026-02-24T22:10:46.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.769+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~D7ZP3IYV9QG2.0 size=1048576 2026-02-24T22:10:46.773 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.769+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~D7ZP3IYV9QG2.1 size=1048576 2026-02-24T22:10:46.773 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.773+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~D7ZP3IYV9QG2.2 size=1021707 2026-02-24T22:10:46.773 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.773+0000 7f7e64854980 1 > prepared multipart upload key=Y8GMOUA977H3 upload=D7ZP3IYV9QG2 tag=TQH98CIGWKIQ type=write size=4167435 2026-02-24T22:10:46.774 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.773+0000 7f7e64854980 -1 < failed to complete operation key=EJCX06TE5RTZ tag=U0I08SMOSNGU type=del size=2260227: (22) Invalid argument 2026-02-24T22:10:46.775 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.773+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~T6HNYLXBVDJQ.0 size=1048576 2026-02-24T22:10:46.776 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.773+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~T6HNYLXBVDJQ.1 size=1048576 2026-02-24T22:10:46.777 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.773+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~T6HNYLXBVDJQ.2 size=977974 2026-02-24T22:10:46.777 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.773+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=T6HNYLXBVDJQ tag=AO57IQ53YOHT type=write size=4123702 2026-02-24T22:10:46.778 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.777+0000 7f7e64854980 1 < completed delete operation key=A6PH77Q78VEN tag=MP50HNO0AU5T type=del 2026-02-24T22:10:46.778 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.777+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=TABM45A3B45T type=del size=788245 2026-02-24T22:10:46.779 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.777+0000 7f7e64854980 1 < completed delete operation key=78GUTC5THI2P tag=TBELRQDVHRHI type=del 2026-02-24T22:10:46.781 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.777+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~OB3SFKVJKE0F.0 size=1048576 2026-02-24T22:10:46.781 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.781+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~OB3SFKVJKE0F.1 size=89619 2026-02-24T22:10:46.781 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.781+0000 7f7e64854980 1 > prepared multipart upload key=L60HQ7M3E3LL upload=OB3SFKVJKE0F tag=S19MOWV0KIND type=write size=2186771 2026-02-24T22:10:46.782 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.781+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~LPP30VH81KVG.0 size=1048576 2026-02-24T22:10:46.783 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.781+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~LPP30VH81KVG.1 size=1048576 2026-02-24T22:10:46.783 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.781+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~LPP30VH81KVG.2 size=1043170 2026-02-24T22:10:46.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.781+0000 7f7e64854980 1 < completed multipart upload key=0AXS7DY23UZU upload=LPP30VH81KVG tag=QR9A4WM10EWR type=write size=4188898 2026-02-24T22:10:46.786 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.785+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~5LUYOWZ34F15.0 size=503574 2026-02-24T22:10:46.786 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.785+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=5LUYOWZ34F15 tag=CDDUR3HVCVQK type=write size=1552150 2026-02-24T22:10:46.787 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.785+0000 7f7e64854980 1 < completed delete operation key=GH7723GNI2WM tag=TY0K1NMS0SOE type=del 2026-02-24T22:10:46.788 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.785+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~0LR62N8S4NYS.0 size=1048576 2026-02-24T22:10:46.789 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.785+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~0LR62N8S4NYS.1 size=1048576 2026-02-24T22:10:46.790 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.789+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~0LR62N8S4NYS.2 size=402425 2026-02-24T22:10:46.790 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.789+0000 7f7e64854980 1 > prepared multipart upload key=PKRHWM69GBJI upload=0LR62N8S4NYS tag=BSGS2YEPF5YS type=write size=3548153 2026-02-24T22:10:46.791 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.789+0000 7f7e64854980 -1 < failed to complete operation key=EJCX06TE5RTZ tag=D9NFT32LVIHF type=write size=344060: (22) Invalid argument 2026-02-24T22:10:46.792 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.789+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~4GBSL6OZW1ZR.0 size=1048576 2026-02-24T22:10:46.793 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.789+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~4GBSL6OZW1ZR.1 size=1048576 2026-02-24T22:10:46.793 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.793+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~4GBSL6OZW1ZR.2 size=651057 2026-02-24T22:10:46.793 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.793+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=4GBSL6OZW1ZR tag=95X3VCXG47ET type=write size=3796785 2026-02-24T22:10:46.794 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.793+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~4Z9AKE0RYI1K.0 size=666204 2026-02-24T22:10:46.795 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.793+0000 7f7e64854980 1 < completed multipart upload key=ON16L53JUSTW upload=4Z9AKE0RYI1K tag=TW8HMP6A7FJ4 type=write size=1714780 2026-02-24T22:10:46.796 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.793+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=DJDY6FM1LHDH type=del size=1968696 2026-02-24T22:10:46.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.793+0000 7f7e64854980 1 < completed delete operation key=GH7723GNI2WM tag=ZGQJX8FBCKAI type=del 2026-02-24T22:10:46.798 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.797+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=5N0YK6PLP7R9 type=del size=1919309 2026-02-24T22:10:46.799 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.797+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~STCYUP1M8L3T.0 size=1048576 2026-02-24T22:10:46.799 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.797+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~STCYUP1M8L3T.1 size=41608 2026-02-24T22:10:46.801 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.797+0000 7f7e64854980 1 < completed multipart upload key=CI8DA17DX8LU upload=STCYUP1M8L3T tag=1MNNODE9158L type=write size=2138760 2026-02-24T22:10:46.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.801+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~A33JYV6W946H.0 size=1048576 2026-02-24T22:10:46.803 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.801+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~A33JYV6W946H.1 size=1048576 2026-02-24T22:10:46.803 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.801+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~A33JYV6W946H.2 size=180403 2026-02-24T22:10:46.803 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.801+0000 7f7e64854980 1 > prepared multipart upload key=PKRHWM69GBJI upload=A33JYV6W946H tag=8HZJV4XE2UA3 type=write size=3326131 2026-02-24T22:10:46.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.801+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~EYKT364IBFEM.0 size=1048576 2026-02-24T22:10:46.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.801+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~EYKT364IBFEM.1 size=768431 2026-02-24T22:10:46.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.805+0000 7f7e64854980 1 < completed multipart upload key=EJCX06TE5RTZ upload=EYKT364IBFEM tag=EVYZQ9KSWUH7 type=write size=2865583 2026-02-24T22:10:46.807 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.805+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=05G611HURJDW type=write size=240871 2026-02-24T22:10:46.808 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.805+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~XPGWB954DY1R.0 size=1048576 2026-02-24T22:10:46.808 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.805+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~XPGWB954DY1R.1 size=1048576 2026-02-24T22:10:46.809 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.805+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~XPGWB954DY1R.2 size=590830 2026-02-24T22:10:46.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.809+0000 7f7e64854980 1 < completed multipart upload key=N3ZUNSI5K8HS upload=XPGWB954DY1R tag=O079RWFFIYM5 type=write size=3736558 2026-02-24T22:10:46.813 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.809+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~4Y1F87X2ZY3H.0 size=1048576 2026-02-24T22:10:46.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.813+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~4Y1F87X2ZY3H.1 size=267013 2026-02-24T22:10:46.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.813+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=4Y1F87X2ZY3H tag=NKXN0CCCCSTN type=write size=2364165 2026-02-24T22:10:46.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.813+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~MSND42SQVMZX.0 size=1048576 2026-02-24T22:10:46.815 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.813+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~MSND42SQVMZX.1 size=1048576 2026-02-24T22:10:46.816 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.813+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~MSND42SQVMZX.2 size=419840 2026-02-24T22:10:46.817 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.813+0000 7f7e64854980 1 < completed multipart upload key=HKOQRSF6XEXB upload=MSND42SQVMZX tag=UWGOYNDTZGKA type=write size=3565568 2026-02-24T22:10:46.818 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.817+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=DV8R847NSKE7 type=del size=2839335 2026-02-24T22:10:46.819 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.817+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~W9SS7GKBB88B.0 size=1048576 2026-02-24T22:10:46.819 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.817+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~W9SS7GKBB88B.1 size=174538 2026-02-24T22:10:46.821 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.817+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=W9SS7GKBB88B tag=APBK2QVBFHZ5 type=write size=2271690 2026-02-24T22:10:46.822 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.821+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=FDPC4KLH36UR type=del size=1485688 2026-02-24T22:10:46.822 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.821+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~PMLBPVW15KR2.0 size=620458 2026-02-24T22:10:46.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.821+0000 7f7e64854980 1 < completed multipart upload key=TF5B43G7KCYT upload=PMLBPVW15KR2 tag=4II2ZJCBWK1B type=write size=1669034 2026-02-24T22:10:46.825 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.821+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~CJ02EJN4GZ1K.0 size=1048576 2026-02-24T22:10:46.826 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.825+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~CJ02EJN4GZ1K.1 size=346670 2026-02-24T22:10:46.826 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.825+0000 7f7e64854980 1 > prepared multipart upload key=EB04ETNYM087 upload=CJ02EJN4GZ1K tag=HU6EVEWI1U42 type=write size=2443822 2026-02-24T22:10:46.827 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.825+0000 7f7e64854980 1 < completed delete operation key=EJCX06TE5RTZ tag=3GH8QE94ZLB9 type=del 2026-02-24T22:10:46.828 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.825+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~H1XM2D2CD2D5.0 size=19010 2026-02-24T22:10:46.828 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.825+0000 7f7e64854980 1 > prepared multipart upload key=0B2V0MHWSOC9 upload=H1XM2D2CD2D5 tag=0SK9LL0I10XX type=write size=1067586 2026-02-24T22:10:46.829 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.825+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~D7ZP3IYV9QG2.0 size=1048576 2026-02-24T22:10:46.829 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.829+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~D7ZP3IYV9QG2.1 size=1048576 2026-02-24T22:10:46.830 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.829+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~D7ZP3IYV9QG2.2 size=1021707 2026-02-24T22:10:46.832 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.829+0000 7f7e64854980 1 < completed multipart upload key=Y8GMOUA977H3 upload=D7ZP3IYV9QG2 tag=TQH98CIGWKIQ type=write size=4167435 2026-02-24T22:10:46.832 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.829+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=ZGVB6TF9QLMW type=del size=3383230 2026-02-24T22:10:46.833 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.829+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~T6HNYLXBVDJQ.0 size=1048576 2026-02-24T22:10:46.834 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.833+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~T6HNYLXBVDJQ.1 size=1048576 2026-02-24T22:10:46.834 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.833+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~T6HNYLXBVDJQ.2 size=977974 2026-02-24T22:10:46.836 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.833+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=T6HNYLXBVDJQ tag=AO57IQ53YOHT type=write size=4123702 2026-02-24T22:10:46.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.837+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~WTEJ6263P1JS.0 size=790158 2026-02-24T22:10:46.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.837+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=WTEJ6263P1JS tag=5AGJ0GZS2DUR type=write size=1838734 2026-02-24T22:10:46.838 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.837+0000 7f7e64854980 1 < completed delete operation key=4Y3974PYYT78 tag=TABM45A3B45T type=del 2026-02-24T22:10:46.840 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.837+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~QXRHCGSK0VCP.0 size=1048576 2026-02-24T22:10:46.840 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.837+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~QXRHCGSK0VCP.1 size=387188 2026-02-24T22:10:46.840 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.837+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=QXRHCGSK0VCP tag=YQXFUFT4O7VO type=write size=2484340 2026-02-24T22:10:46.841 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.837+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~OB3SFKVJKE0F.0 size=1048576 2026-02-24T22:10:46.842 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.841+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~OB3SFKVJKE0F.1 size=89619 2026-02-24T22:10:46.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.841+0000 7f7e64854980 1 < completed multipart upload key=L60HQ7M3E3LL upload=OB3SFKVJKE0F tag=S19MOWV0KIND type=write size=2186771 2026-02-24T22:10:46.844 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.841+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=UGQMFBC7AC4N type=write size=8950 2026-02-24T22:10:46.845 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.841+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~5LUYOWZ34F15.0 size=503574 2026-02-24T22:10:46.846 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.845+0000 7f7e64854980 1 < completed multipart upload key=ZESKGBWZXM6O upload=5LUYOWZ34F15 tag=CDDUR3HVCVQK type=write size=1552150 2026-02-24T22:10:46.847 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.845+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~NMCF77679Q5S.0 size=633380 2026-02-24T22:10:46.847 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.845+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=NMCF77679Q5S tag=4E7N7MBTH2MT type=write size=1681956 2026-02-24T22:10:46.848 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.845+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~0LR62N8S4NYS.0 size=1048576 2026-02-24T22:10:46.849 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.845+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~0LR62N8S4NYS.1 size=1048576 2026-02-24T22:10:46.849 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.849+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~0LR62N8S4NYS.2 size=402425 2026-02-24T22:10:46.851 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.849+0000 7f7e64854980 1 < completed multipart upload key=PKRHWM69GBJI upload=0LR62N8S4NYS tag=BSGS2YEPF5YS type=write size=3548153 2026-02-24T22:10:46.852 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.849+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~HZMZBXCMTNKE.0 size=1048576 2026-02-24T22:10:46.853 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.849+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~HZMZBXCMTNKE.1 size=546916 2026-02-24T22:10:46.853 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.849+0000 7f7e64854980 1 > prepared multipart upload key=GH7723GNI2WM upload=HZMZBXCMTNKE tag=FO5MKN2YF63T type=write size=2644068 2026-02-24T22:10:46.854 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.853+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~4GBSL6OZW1ZR.0 size=1048576 2026-02-24T22:10:46.854 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.853+0000 7f7e64854980 1 < canceled part key=_multipart_D0MKGNRDW6L9.2~4GBSL6OZW1ZR.1 size=1048576 2026-02-24T22:10:46.854 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.853+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~4GBSL6OZW1ZR.2 size=651057 2026-02-24T22:10:46.856 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.853+0000 7f7e64854980 1 < canceled multipart upload key=D0MKGNRDW6L9 upload=4GBSL6OZW1ZR tag=95X3VCXG47ET type=write size=3796785 2026-02-24T22:10:46.857 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.853+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=XL5CEX6JB6PR type=write size=304182 2026-02-24T22:10:46.857 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.857+0000 7f7e64854980 1 < completed delete operation key=PKRHWM69GBJI tag=DJDY6FM1LHDH type=del 2026-02-24T22:10:46.858 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.857+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=NZZVUMOMWSQQ type=write size=24783 2026-02-24T22:10:46.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.857+0000 7f7e64854980 1 < suggested update operation key=2KKQKHXO7K3K tag=5N0YK6PLP7R9 type=del size=1919309 2026-02-24T22:10:46.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.857+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~L11KP85FFSZW.0 size=1048576 2026-02-24T22:10:46.862 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.861+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~L11KP85FFSZW.1 size=1048576 2026-02-24T22:10:46.862 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.861+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~L11KP85FFSZW.2 size=232914 2026-02-24T22:10:46.863 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.861+0000 7f7e64854980 1 > prepared multipart upload key=EB04ETNYM087 upload=L11KP85FFSZW tag=9FIH1RGRST6S type=write size=3378642 2026-02-24T22:10:46.863 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.861+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~A33JYV6W946H.0 size=1048576 2026-02-24T22:10:46.864 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.861+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~A33JYV6W946H.1 size=1048576 2026-02-24T22:10:46.865 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.861+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~A33JYV6W946H.2 size=180403 2026-02-24T22:10:46.866 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.865+0000 7f7e64854980 1 < completed multipart upload key=PKRHWM69GBJI upload=A33JYV6W946H tag=8HZJV4XE2UA3 type=write size=3326131 2026-02-24T22:10:46.867 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.865+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=N0DAGAXIFVWA type=write size=641796 2026-02-24T22:10:46.868 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.865+0000 7f7e64854980 1 < completed write operation key=NL60Q5QOSP8E tag=05G611HURJDW type=write size=240871 2026-02-24T22:10:46.868 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.865+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=DB21QQYC7F1S type=del size=3132243 2026-02-24T22:10:46.869 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.865+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~4Y1F87X2ZY3H.0 size=1048576 2026-02-24T22:10:46.870 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.869+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~4Y1F87X2ZY3H.1 size=267013 2026-02-24T22:10:46.871 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.869+0000 7f7e64854980 1 < completed multipart upload key=D0MKGNRDW6L9 upload=4Y1F87X2ZY3H tag=NKXN0CCCCSTN type=write size=2364165 2026-02-24T22:10:46.872 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.869+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=VXGRQKWTAAZ7 type=del size=930803 2026-02-24T22:10:46.873 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.869+0000 7f7e64854980 1 < completed delete operation key=EJCX06TE5RTZ tag=DV8R847NSKE7 type=del 2026-02-24T22:10:46.874 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.873+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=KZUQ1KKARUWN type=write size=617052 2026-02-24T22:10:46.874 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.873+0000 7f7e64854980 1 < completed delete operation key=Y3L3CJ51C1WR tag=FDPC4KLH36UR type=del 2026-02-24T22:10:46.875 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.873+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=B1P7B19DVC03 type=del size=3555750 2026-02-24T22:10:46.876 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.873+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~CJ02EJN4GZ1K.0 size=1048576 2026-02-24T22:10:46.877 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.873+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~CJ02EJN4GZ1K.1 size=346670 2026-02-24T22:10:46.878 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.877+0000 7f7e64854980 1 < completed multipart upload key=EB04ETNYM087 upload=CJ02EJN4GZ1K tag=HU6EVEWI1U42 type=write size=2443822 2026-02-24T22:10:46.880 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.877+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~DIY44I7QHN0A.0 size=1048576 2026-02-24T22:10:46.880 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.877+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~DIY44I7QHN0A.1 size=202827 2026-02-24T22:10:46.880 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.877+0000 7f7e64854980 1 > prepared multipart upload key=L60HQ7M3E3LL upload=DIY44I7QHN0A tag=M9VNWD6KBMML type=write size=2299979 2026-02-24T22:10:46.881 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.877+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~H1XM2D2CD2D5.0 size=19010 2026-02-24T22:10:46.882 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.881+0000 7f7e64854980 1 < completed multipart upload key=0B2V0MHWSOC9 upload=H1XM2D2CD2D5 tag=0SK9LL0I10XX type=write size=1067586 2026-02-24T22:10:46.883 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.881+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=PN88175PVN5L type=del size=1591506 2026-02-24T22:10:46.885 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.885+0000 7f7e64854980 -1 < failed to complete operation key=2KKQKHXO7K3K tag=ZGVB6TF9QLMW type=del size=3383230: (22) Invalid argument 2026-02-24T22:10:46.886 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.885+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=6PCPO7SVX248 type=del size=851259 2026-02-24T22:10:46.887 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.885+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~WTEJ6263P1JS.0 size=790158 2026-02-24T22:10:46.888 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.885+0000 7f7e64854980 1 < completed multipart upload key=ZESKGBWZXM6O upload=WTEJ6263P1JS tag=5AGJ0GZS2DUR type=write size=1838734 2026-02-24T22:10:46.890 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.889+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~VNLJVM4CV3XV.0 size=60948 2026-02-24T22:10:46.890 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.889+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=VNLJVM4CV3XV tag=77VQPZQF3PBA type=write size=1109524 2026-02-24T22:10:46.891 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.889+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~QXRHCGSK0VCP.0 size=1048576 2026-02-24T22:10:46.891 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.889+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~QXRHCGSK0VCP.1 size=387188 2026-02-24T22:10:46.892 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.889+0000 7f7e64854980 1 < completed multipart upload key=0AXS7DY23UZU upload=QXRHCGSK0VCP tag=YQXFUFT4O7VO type=write size=2484340 2026-02-24T22:10:46.894 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.893+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~UAYOPF44WRCF.0 size=1048576 2026-02-24T22:10:46.895 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.893+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~UAYOPF44WRCF.1 size=1048576 2026-02-24T22:10:46.895 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.893+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~UAYOPF44WRCF.2 size=766504 2026-02-24T22:10:46.895 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.893+0000 7f7e64854980 1 > prepared multipart upload key=Q7BP8FXTARDW upload=UAYOPF44WRCF tag=5TA822YYF6F2 type=write size=3912232 2026-02-24T22:10:46.896 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.893+0000 7f7e64854980 1 < canceled operation key=4Y3974PYYT78 tag=UGQMFBC7AC4N type=write size=8950 2026-02-24T22:10:46.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.893+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=B4CH78Q6O5R1 type=del size=3613103 2026-02-24T22:10:46.898 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.897+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~NMCF77679Q5S.0 size=633380 2026-02-24T22:10:46.899 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.897+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=NMCF77679Q5S tag=4E7N7MBTH2MT type=write size=1681956 2026-02-24T22:10:46.900 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.897+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=7YL4QAHQ8X4J type=del size=1129301 2026-02-24T22:10:46.901 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.897+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~HZMZBXCMTNKE.0 size=1048576 2026-02-24T22:10:46.901 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.897+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~HZMZBXCMTNKE.1 size=546916 2026-02-24T22:10:46.902 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.901+0000 7f7e64854980 1 < completed multipart upload key=GH7723GNI2WM upload=HZMZBXCMTNKE tag=FO5MKN2YF63T type=write size=2644068 2026-02-24T22:10:46.903 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.901+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=0U8TUQ27VR9S type=del size=943135 2026-02-24T22:10:46.904 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.901+0000 7f7e64854980 1 < completed write operation key=TXRUQQ456DD8 tag=XL5CEX6JB6PR type=write size=304182 2026-02-24T22:10:46.905 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.901+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=YF8X1IDIFTYR type=del size=2901024 2026-02-24T22:10:46.906 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.905+0000 7f7e64854980 1 < completed write operation key=3900SFJK1YQU tag=NZZVUMOMWSQQ type=write size=24783 2026-02-24T22:10:46.907 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.905+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=6EH4AMI0HMZ5 type=del size=2673369 2026-02-24T22:10:46.907 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.905+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~L11KP85FFSZW.0 size=1048576 2026-02-24T22:10:46.908 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.905+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~L11KP85FFSZW.1 size=1048576 2026-02-24T22:10:46.909 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.905+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~L11KP85FFSZW.2 size=232914 2026-02-24T22:10:46.910 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.909+0000 7f7e64854980 1 < completed multipart upload key=EB04ETNYM087 upload=L11KP85FFSZW tag=9FIH1RGRST6S type=write size=3378642 2026-02-24T22:10:46.911 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.909+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~3DE0KYY6PRQ3.0 size=1048576 2026-02-24T22:10:46.912 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.909+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~3DE0KYY6PRQ3.1 size=192129 2026-02-24T22:10:46.912 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.909+0000 7f7e64854980 1 > prepared multipart upload key=36MKR2OZN7K9 upload=3DE0KYY6PRQ3 tag=CPX8Q23NXU4R type=write size=2289281 2026-02-24T22:10:46.913 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.909+0000 7f7e64854980 1 < completed write operation key=4Y3974PYYT78 tag=N0DAGAXIFVWA type=write size=641796 2026-02-24T22:10:46.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.913+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~5YTVICD00API.0 size=1048576 2026-02-24T22:10:46.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.913+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~5YTVICD00API.1 size=793246 2026-02-24T22:10:46.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.913+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=5YTVICD00API tag=2BN06DYJD4Z3 type=write size=2890398 2026-02-24T22:10:46.916 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.913+0000 7f7e64854980 1 < completed delete operation key=ZESKGBWZXM6O tag=DB21QQYC7F1S type=del 2026-02-24T22:10:46.917 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.913+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=C8I7J0VWSM53 type=write size=665495 2026-02-24T22:10:46.917 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.917+0000 7f7e64854980 1 < completed delete operation key=EB04ETNYM087 tag=VXGRQKWTAAZ7 type=del 2026-02-24T22:10:46.918 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.917+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=2ISQCU992JRS type=del size=3921878 2026-02-24T22:10:46.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.917+0000 7f7e64854980 1 < completed write operation key=0AXS7DY23UZU tag=KZUQ1KKARUWN type=write size=617052 2026-02-24T22:10:46.920 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.917+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~MW3B43R2TOLN.0 size=86791 2026-02-24T22:10:46.921 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.917+0000 7f7e64854980 1 > prepared multipart upload key=TF5B43G7KCYT upload=MW3B43R2TOLN tag=XH04UU5C0WDC type=write size=1135367 2026-02-24T22:10:46.921 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.917+0000 7f7e64854980 1 < completed delete operation key=HKOQRSF6XEXB tag=B1P7B19DVC03 type=del 2026-02-24T22:10:46.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.921+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~R4Y9ZFGUX92U.0 size=1048576 2026-02-24T22:10:46.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.921+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~R4Y9ZFGUX92U.1 size=1048576 2026-02-24T22:10:46.924 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.921+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~R4Y9ZFGUX92U.2 size=267334 2026-02-24T22:10:46.924 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.921+0000 7f7e64854980 1 > prepared multipart upload key=GH7723GNI2WM upload=R4Y9ZFGUX92U tag=61SJYIWL1LRE type=write size=3413062 2026-02-24T22:10:46.925 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.921+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~DIY44I7QHN0A.0 size=1048576 2026-02-24T22:10:46.925 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.921+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~DIY44I7QHN0A.1 size=202827 2026-02-24T22:10:46.926 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.925+0000 7f7e64854980 1 < completed multipart upload key=L60HQ7M3E3LL upload=DIY44I7QHN0A tag=M9VNWD6KBMML type=write size=2299979 2026-02-24T22:10:46.927 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.925+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=JPFV34IZKPZ1 type=del size=2742404 2026-02-24T22:10:46.928 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.925+0000 7f7e64854980 1 < completed delete operation key=TF5B43G7KCYT tag=PN88175PVN5L type=del 2026-02-24T22:10:46.929 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.925+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=7S3ZFHTRTQ49 type=del size=2109684 2026-02-24T22:10:46.930 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.929+0000 7f7e64854980 1 < suggested remove operation key=L60HQ7M3E3LL tag=6PCPO7SVX248 type=del 2026-02-24T22:10:46.931 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.929+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=PZ460R8VCVDA type=del size=1450630 2026-02-24T22:10:46.932 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.929+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~VNLJVM4CV3XV.0 size=60948 2026-02-24T22:10:46.933 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.929+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=VNLJVM4CV3XV tag=77VQPZQF3PBA type=write size=1109524 2026-02-24T22:10:46.934 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.933+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=8VKAWVMI9UYD type=del size=966358 2026-02-24T22:10:46.935 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.933+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~UAYOPF44WRCF.0 size=1048576 2026-02-24T22:10:46.935 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.933+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~UAYOPF44WRCF.1 size=1048576 2026-02-24T22:10:46.936 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.933+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~UAYOPF44WRCF.2 size=766504 2026-02-24T22:10:46.937 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.937+0000 7f7e64854980 1 < completed multipart upload key=Q7BP8FXTARDW upload=UAYOPF44WRCF tag=5TA822YYF6F2 type=write size=3912232 2026-02-24T22:10:46.938 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.937+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=VL0TUBGBSOCA type=del size=449141 2026-02-24T22:10:46.940 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.937+0000 7f7e64854980 1 < suggested remove operation key=EB04ETNYM087 tag=B4CH78Q6O5R1 type=del 2026-02-24T22:10:46.940 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.937+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=TTNA3VR18YBO type=del size=1985968 2026-02-24T22:10:46.941 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.937+0000 7f7e64854980 1 < completed delete operation key=Z0OXREAUJVA8 tag=7YL4QAHQ8X4J type=del 2026-02-24T22:10:46.942 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.941+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=YQOWQNUOBBLC type=write size=889543 2026-02-24T22:10:46.943 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.941+0000 7f7e64854980 1 < completed delete operation key=GH7723GNI2WM tag=0U8TUQ27VR9S type=del 2026-02-24T22:10:46.944 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.941+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=5J2RJNDZ33JF type=del size=2525252 2026-02-24T22:10:46.944 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.941+0000 7f7e64854980 -1 < no bi entry to suggest for operation key=EB04ETNYM087 tag=YF8X1IDIFTYR type=del size=2901024: (2) No such file or directory 2026-02-24T22:10:46.945 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.941+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=WQNEOT7GA0TY type=del size=1887445 2026-02-24T22:10:46.946 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.945+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=6EH4AMI0HMZ5 type=del 2026-02-24T22:10:46.946 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.945+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=W4QMXPWRK760 type=del size=2607111 2026-02-24T22:10:46.947 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.945+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~3DE0KYY6PRQ3.0 size=1048576 2026-02-24T22:10:46.948 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.945+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~3DE0KYY6PRQ3.1 size=192129 2026-02-24T22:10:46.949 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.945+0000 7f7e64854980 1 < completed multipart upload key=36MKR2OZN7K9 upload=3DE0KYY6PRQ3 tag=CPX8Q23NXU4R type=write size=2289281 2026-02-24T22:10:46.950 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.949+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=00LOSMSJPBI3 type=del size=4098090 2026-02-24T22:10:46.951 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.949+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~5YTVICD00API.0 size=1048576 2026-02-24T22:10:46.952 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.949+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~5YTVICD00API.1 size=793246 2026-02-24T22:10:46.953 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.949+0000 7f7e64854980 1 < completed multipart upload key=HOHKR92SN25M upload=5YTVICD00API tag=2BN06DYJD4Z3 type=write size=2890398 2026-02-24T22:10:46.954 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.953+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=GEWNV0KSIKIU type=del size=4103011 2026-02-24T22:10:46.955 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.953+0000 7f7e64854980 1 < canceled operation key=LSRPN33Z2B9N tag=C8I7J0VWSM53 type=write size=665495 2026-02-24T22:10:46.955 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.953+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=1Z4E2Q7VLAL5 type=write size=903067 2026-02-24T22:10:46.956 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.953+0000 7f7e64854980 1 < completed delete operation key=Y3L3CJ51C1WR tag=2ISQCU992JRS type=del 2026-02-24T22:10:46.957 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.957+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~76W0NFQ45G2O.0 size=1048576 2026-02-24T22:10:46.958 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.957+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~76W0NFQ45G2O.1 size=1048576 2026-02-24T22:10:46.960 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.957+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~76W0NFQ45G2O.2 size=44472 2026-02-24T22:10:46.960 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.957+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=76W0NFQ45G2O tag=JHDYB9P8WBXO type=write size=3190200 2026-02-24T22:10:46.960 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.957+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~MW3B43R2TOLN.0 size=86791 2026-02-24T22:10:46.962 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.961+0000 7f7e64854980 1 < completed multipart upload key=TF5B43G7KCYT upload=MW3B43R2TOLN tag=XH04UU5C0WDC type=write size=1135367 2026-02-24T22:10:46.963 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.961+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=NH0I1UTPIFVM type=del size=3648432 2026-02-24T22:10:46.963 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.961+0000 7f7e64854980 1 < canceled part key=_multipart_GH7723GNI2WM.2~R4Y9ZFGUX92U.0 size=1048576 2026-02-24T22:10:46.963 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.961+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~R4Y9ZFGUX92U.1 size=1048576 2026-02-24T22:10:46.964 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.961+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~R4Y9ZFGUX92U.2 size=267334 2026-02-24T22:10:46.965 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.965+0000 7f7e64854980 1 < canceled multipart upload key=GH7723GNI2WM upload=R4Y9ZFGUX92U tag=61SJYIWL1LRE type=write size=3413062 2026-02-24T22:10:46.966 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.965+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=O2FTZXSQADJN type=del size=3581378 2026-02-24T22:10:46.967 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.965+0000 7f7e64854980 1 < completed delete operation key=Y3L3CJ51C1WR tag=JPFV34IZKPZ1 type=del 2026-02-24T22:10:46.968 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.965+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~E3CKPU5YZJIP.0 size=1048576 2026-02-24T22:10:46.969 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.965+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~E3CKPU5YZJIP.1 size=1048576 2026-02-24T22:10:46.969 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.969+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~E3CKPU5YZJIP.2 size=362168 2026-02-24T22:10:46.970 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.969+0000 7f7e64854980 1 > prepared multipart upload key=L60HQ7M3E3LL upload=E3CKPU5YZJIP tag=NQOLY6WEGU12 type=write size=3507896 2026-02-24T22:10:46.970 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.969+0000 7f7e64854980 1 < canceled operation key=GH7723GNI2WM tag=7S3ZFHTRTQ49 type=del size=2109684 2026-02-24T22:10:46.971 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.969+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=7SKZN2S5SE4T type=del size=3196765 2026-02-24T22:10:46.972 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.969+0000 7f7e64854980 1 < completed delete operation key=0B2V0MHWSOC9 tag=PZ460R8VCVDA type=del 2026-02-24T22:10:46.973 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.969+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=DKLKXTUQ349Q type=del size=4164382 2026-02-24T22:10:46.973 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.973+0000 7f7e64854980 1 < completed delete operation key=36MKR2OZN7K9 tag=8VKAWVMI9UYD type=del 2026-02-24T22:10:46.974 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.973+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=K591SYTAY2RD type=del size=1836684 2026-02-24T22:10:46.975 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.973+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=VL0TUBGBSOCA type=del 2026-02-24T22:10:46.976 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.973+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=34P3B6FEYOAA type=del size=1544067 2026-02-24T22:10:46.977 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.977+0000 7f7e64854980 1 < suggested update operation key=NL60Q5QOSP8E tag=TTNA3VR18YBO type=del size=1985968 2026-02-24T22:10:46.978 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.977+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=DWTVHZLFI6KG type=del size=2812576 2026-02-24T22:10:46.980 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.977+0000 7f7e64854980 1 < suggested remove operation key=TF5B43G7KCYT tag=YQOWQNUOBBLC type=write 2026-02-24T22:10:46.980 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.977+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=UGPQBU6RTVEC type=write size=285648 2026-02-24T22:10:46.981 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.977+0000 7f7e64854980 1 < completed delete operation key=N3ZUNSI5K8HS tag=5J2RJNDZ33JF type=del 2026-02-24T22:10:46.982 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.981+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=BDL4XH05PM7H type=del size=3215685 2026-02-24T22:10:46.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.981+0000 7f7e64854980 1 < completed delete operation key=D0MKGNRDW6L9 tag=WQNEOT7GA0TY type=del 2026-02-24T22:10:46.984 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.981+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~T4ZSA9JJ635W.0 size=1048576 2026-02-24T22:10:46.985 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.981+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~T4ZSA9JJ635W.1 size=593719 2026-02-24T22:10:46.985 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.981+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=T4ZSA9JJ635W tag=WGSPSDD3Q35Z type=write size=2690871 2026-02-24T22:10:46.986 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.985+0000 7f7e64854980 1 < completed delete operation key=EJCX06TE5RTZ tag=W4QMXPWRK760 type=del 2026-02-24T22:10:46.986 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.985+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=X3FSFZVF0E20 type=del size=2836848 2026-02-24T22:10:46.987 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.985+0000 7f7e64854980 1 < completed delete operation key=0AXS7DY23UZU tag=00LOSMSJPBI3 type=del 2026-02-24T22:10:46.988 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.985+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~NR01FNYJUXTF.0 size=656176 2026-02-24T22:10:46.989 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.985+0000 7f7e64854980 1 > prepared multipart upload key=Z0OXREAUJVA8 upload=NR01FNYJUXTF tag=97MB5U3TXV91 type=write size=1704752 2026-02-24T22:10:46.989 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.989+0000 7f7e64854980 1 < completed delete operation key=3900SFJK1YQU tag=GEWNV0KSIKIU type=del 2026-02-24T22:10:46.991 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.989+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~EG58D9D8ILM3.0 size=1048576 2026-02-24T22:10:46.991 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.989+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~EG58D9D8ILM3.1 size=561695 2026-02-24T22:10:46.991 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.989+0000 7f7e64854980 1 > prepared multipart upload key=ON16L53JUSTW upload=EG58D9D8ILM3 tag=HBFR2PMGSS8L type=write size=2658847 2026-02-24T22:10:46.992 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.989+0000 7f7e64854980 1 < completed write operation key=LSRPN33Z2B9N tag=1Z4E2Q7VLAL5 type=write size=903067 2026-02-24T22:10:46.993 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.989+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=M69XVEUH91WC type=del size=3938511 2026-02-24T22:10:46.994 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.993+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~76W0NFQ45G2O.0 size=1048576 2026-02-24T22:10:46.994 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.993+0000 7f7e64854980 1 < canceled part key=_multipart_0AXS7DY23UZU.2~76W0NFQ45G2O.1 size=1048576 2026-02-24T22:10:46.994 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.993+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~76W0NFQ45G2O.2 size=44472 2026-02-24T22:10:46.996 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.993+0000 7f7e64854980 1 < canceled multipart upload key=0AXS7DY23UZU upload=76W0NFQ45G2O tag=JHDYB9P8WBXO type=write size=3190200 2026-02-24T22:10:46.996 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.993+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=W1YAI9L96OJ3 type=del size=2302897 2026-02-24T22:10:46.997 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.993+0000 7f7e64854980 1 < completed delete operation key=GH7723GNI2WM tag=NH0I1UTPIFVM type=del 2026-02-24T22:10:46.999 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.997+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~EHVR6HS95VI1.0 size=1048576 2026-02-24T22:10:46.999 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.997+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~EHVR6HS95VI1.1 size=1048576 2026-02-24T22:10:47.000 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.997+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~EHVR6HS95VI1.2 size=162329 2026-02-24T22:10:47.000 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.997+0000 7f7e64854980 1 > prepared multipart upload key=N3ZUNSI5K8HS upload=EHVR6HS95VI1 tag=9DL1XFQY3VRT type=write size=3308057 2026-02-24T22:10:47.001 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:46.997+0000 7f7e64854980 1 < completed delete operation key=TXRUQQ456DD8 tag=O2FTZXSQADJN type=del 2026-02-24T22:10:47.002 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.001+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=SBTMGQM6TRCP type=del size=1745936 2026-02-24T22:10:47.002 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.001+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~E3CKPU5YZJIP.0 size=1048576 2026-02-24T22:10:47.003 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.001+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~E3CKPU5YZJIP.1 size=1048576 2026-02-24T22:10:47.004 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.001+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~E3CKPU5YZJIP.2 size=362168 2026-02-24T22:10:47.005 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.001+0000 7f7e64854980 1 < completed multipart upload key=L60HQ7M3E3LL upload=E3CKPU5YZJIP tag=NQOLY6WEGU12 type=write size=3507896 2026-02-24T22:10:47.006 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.005+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~ICXWH6YII3HK.0 size=582484 2026-02-24T22:10:47.006 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.005+0000 7f7e64854980 1 > prepared multipart upload key=78GUTC5THI2P upload=ICXWH6YII3HK tag=PEAZMZCE0Y1W type=write size=1631060 2026-02-24T22:10:47.007 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.005+0000 7f7e64854980 1 < canceled operation key=LSRPN33Z2B9N tag=7SKZN2S5SE4T type=del size=3196765 2026-02-24T22:10:47.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.005+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~JSLKZPTF7HNZ.0 size=1048576 2026-02-24T22:10:47.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.005+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~JSLKZPTF7HNZ.1 size=464588 2026-02-24T22:10:47.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.005+0000 7f7e64854980 1 > prepared multipart upload key=L60HQ7M3E3LL upload=JSLKZPTF7HNZ tag=46A98OCFDQ5D type=write size=2561740 2026-02-24T22:10:47.011 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.009+0000 7f7e64854980 1 < suggested update operation key=EJCX06TE5RTZ tag=DKLKXTUQ349Q type=del size=4164382 2026-02-24T22:10:47.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.009+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=WUOMS0ZVA98O type=del size=1875806 2026-02-24T22:10:47.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.009+0000 7f7e64854980 1 < completed delete operation key=2KKQKHXO7K3K tag=K591SYTAY2RD type=del 2026-02-24T22:10:47.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.009+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=C9I0ROSG3MIK type=del size=2202029 2026-02-24T22:10:47.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.013+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=34P3B6FEYOAA type=del 2026-02-24T22:10:47.015 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.013+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~QR7M2WO2YLP5.0 size=1048576 2026-02-24T22:10:47.016 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.013+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~QR7M2WO2YLP5.1 size=197549 2026-02-24T22:10:47.016 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.013+0000 7f7e64854980 1 > prepared multipart upload key=PKRHWM69GBJI upload=QR7M2WO2YLP5 tag=T6SQCNXG6YGR type=write size=2294701 2026-02-24T22:10:47.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.017+0000 7f7e64854980 1 < suggested update operation key=A6PH77Q78VEN tag=DWTVHZLFI6KG type=del size=2812576 2026-02-24T22:10:47.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.017+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=LKY2OYA19WTL type=write size=79816 2026-02-24T22:10:47.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.017+0000 7f7e64854980 1 < suggested remove operation key=D0MKGNRDW6L9 tag=UGPQBU6RTVEC type=write 2026-02-24T22:10:47.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.017+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=I0HRBYNR1V1B type=del size=448375 2026-02-24T22:10:47.022 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.021+0000 7f7e64854980 1 < suggested update operation key=GH7723GNI2WM tag=BDL4XH05PM7H type=del size=3215685 2026-02-24T22:10:47.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.021+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=SEPDIK1B4QWB type=write size=785375 2026-02-24T22:10:47.024 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.021+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~T4ZSA9JJ635W.0 size=1048576 2026-02-24T22:10:47.025 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.021+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~T4ZSA9JJ635W.1 size=593719 2026-02-24T22:10:47.026 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.025+0000 7f7e64854980 1 < completed multipart upload key=0AXS7DY23UZU upload=T4ZSA9JJ635W tag=WGSPSDD3Q35Z type=write size=2690871 2026-02-24T22:10:47.027 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.025+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=IB14T2QRCYD7 type=del size=3994278 2026-02-24T22:10:47.028 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.025+0000 7f7e64854980 1 < suggested remove operation key=L9NEDH44BIGA tag=X3FSFZVF0E20 type=del 2026-02-24T22:10:47.029 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.029+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=LAX1VNYI34XJ type=del size=3489138 2026-02-24T22:10:47.030 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.029+0000 7f7e64854980 1 < canceled part key=_multipart_Z0OXREAUJVA8.2~NR01FNYJUXTF.0 size=656176 2026-02-24T22:10:47.031 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.029+0000 7f7e64854980 1 < canceled multipart upload key=Z0OXREAUJVA8 upload=NR01FNYJUXTF tag=97MB5U3TXV91 type=write size=1704752 2026-02-24T22:10:47.032 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.029+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~3DED3JJ7ZPK4.0 size=1048576 2026-02-24T22:10:47.033 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.029+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~3DED3JJ7ZPK4.1 size=227877 2026-02-24T22:10:47.033 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.029+0000 7f7e64854980 1 > prepared multipart upload key=GH7723GNI2WM upload=3DED3JJ7ZPK4 tag=G6LFMYEQJPVT type=write size=2325029 2026-02-24T22:10:47.035 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.033+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~EG58D9D8ILM3.0 size=1048576 2026-02-24T22:10:47.035 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.033+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~EG58D9D8ILM3.1 size=561695 2026-02-24T22:10:47.037 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.033+0000 7f7e64854980 1 < completed multipart upload key=ON16L53JUSTW upload=EG58D9D8ILM3 tag=HBFR2PMGSS8L type=write size=2658847 2026-02-24T22:10:47.037 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.037+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=LY9A950SJXSS type=del size=1769254 2026-02-24T22:10:47.038 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.037+0000 7f7e64854980 1 < completed delete operation key=36MKR2OZN7K9 tag=M69XVEUH91WC type=del 2026-02-24T22:10:47.039 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.037+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=RRGN4QFH419B type=del size=187045 2026-02-24T22:10:47.040 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.037+0000 7f7e64854980 1 < canceled operation key=L60HQ7M3E3LL tag=W1YAI9L96OJ3 type=del size=2302897 2026-02-24T22:10:47.041 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.037+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=Z4KD2XNK0H6K type=del size=341369 2026-02-24T22:10:47.042 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.041+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~EHVR6HS95VI1.0 size=1048576 2026-02-24T22:10:47.042 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.041+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~EHVR6HS95VI1.1 size=1048576 2026-02-24T22:10:47.043 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.041+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~EHVR6HS95VI1.2 size=162329 2026-02-24T22:10:47.044 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.041+0000 7f7e64854980 1 < completed multipart upload key=N3ZUNSI5K8HS upload=EHVR6HS95VI1 tag=9DL1XFQY3VRT type=write size=3308057 2026-02-24T22:10:47.046 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.045+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~61FM9I3FSQ4W.0 size=1048576 2026-02-24T22:10:47.046 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.045+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~61FM9I3FSQ4W.1 size=161092 2026-02-24T22:10:47.046 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.045+0000 7f7e64854980 1 > prepared multipart upload key=PKRHWM69GBJI upload=61FM9I3FSQ4W tag=LRWJWN9Q3F9F type=write size=2258244 2026-02-24T22:10:47.047 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.045+0000 7f7e64854980 1 < completed delete operation key=LSRPN33Z2B9N tag=SBTMGQM6TRCP type=del 2026-02-24T22:10:47.048 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.045+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~ENI0G0WVXKMS.0 size=1048576 2026-02-24T22:10:47.049 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.045+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~ENI0G0WVXKMS.1 size=533139 2026-02-24T22:10:47.049 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.045+0000 7f7e64854980 1 > prepared multipart upload key=78GUTC5THI2P upload=ENI0G0WVXKMS tag=62GA7URUDQT6 type=write size=2630291 2026-02-24T22:10:47.050 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.049+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~ICXWH6YII3HK.0 size=582484 2026-02-24T22:10:47.051 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.049+0000 7f7e64854980 1 < completed multipart upload key=78GUTC5THI2P upload=ICXWH6YII3HK tag=PEAZMZCE0Y1W type=write size=1631060 2026-02-24T22:10:47.052 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.049+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=FBRYOLU00PK5 type=del size=3865629 2026-02-24T22:10:47.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.049+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~JSLKZPTF7HNZ.0 size=1048576 2026-02-24T22:10:47.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.053+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~JSLKZPTF7HNZ.1 size=464588 2026-02-24T22:10:47.055 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.053+0000 7f7e64854980 1 < completed multipart upload key=L60HQ7M3E3LL upload=JSLKZPTF7HNZ tag=46A98OCFDQ5D type=write size=2561740 2026-02-24T22:10:47.056 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.053+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=P7UAFZD33F2M type=write size=203979 2026-02-24T22:10:47.056 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.053+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=WUOMS0ZVA98O type=del 2026-02-24T22:10:47.058 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.057+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~6GJMEP26RFYS.0 size=1048576 2026-02-24T22:10:47.058 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.057+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~6GJMEP26RFYS.1 size=1048576 2026-02-24T22:10:47.059 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.057+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~6GJMEP26RFYS.2 size=382556 2026-02-24T22:10:47.059 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.057+0000 7f7e64854980 1 > prepared multipart upload key=36MKR2OZN7K9 upload=6GJMEP26RFYS tag=05KDM29V12U3 type=write size=3528284 2026-02-24T22:10:47.060 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.057+0000 7f7e64854980 1 < completed delete operation key=TXRUQQ456DD8 tag=C9I0ROSG3MIK type=del 2026-02-24T22:10:47.061 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.061+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~GT43E76QDL97.0 size=183771 2026-02-24T22:10:47.061 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.061+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=GT43E76QDL97 tag=4Z5ZUM2ZQEI3 type=write size=1232347 2026-02-24T22:10:47.062 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.061+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~QR7M2WO2YLP5.0 size=1048576 2026-02-24T22:10:47.063 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.061+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~QR7M2WO2YLP5.1 size=197549 2026-02-24T22:10:47.064 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.061+0000 7f7e64854980 1 < completed multipart upload key=PKRHWM69GBJI upload=QR7M2WO2YLP5 tag=T6SQCNXG6YGR type=write size=2294701 2026-02-24T22:10:47.066 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.065+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~NLNWDTQT9YWJ.0 size=1048576 2026-02-24T22:10:47.066 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.065+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~NLNWDTQT9YWJ.1 size=1048576 2026-02-24T22:10:47.067 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.065+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~NLNWDTQT9YWJ.2 size=201068 2026-02-24T22:10:47.067 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.065+0000 7f7e64854980 1 > prepared multipart upload key=LSRPN33Z2B9N upload=NLNWDTQT9YWJ tag=L6JZ28E1NYW6 type=write size=3346796 2026-02-24T22:10:47.068 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.065+0000 7f7e64854980 1 < completed write operation key=TF5B43G7KCYT tag=LKY2OYA19WTL type=write size=79816 2026-02-24T22:10:47.068 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.065+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=T7T5A02WUZSB type=del size=2897117 2026-02-24T22:10:47.069 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.065+0000 7f7e64854980 1 < completed delete operation key=R2893NES3WSF tag=I0HRBYNR1V1B type=del 2026-02-24T22:10:47.071 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.069+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~P7GPQS5TZG97.0 size=1048576 2026-02-24T22:10:47.071 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.069+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~P7GPQS5TZG97.1 size=1048576 2026-02-24T22:10:47.072 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.069+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~P7GPQS5TZG97.2 size=371333 2026-02-24T22:10:47.072 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.069+0000 7f7e64854980 1 > prepared multipart upload key=HKOQRSF6XEXB upload=P7GPQS5TZG97 tag=7DTODH8WORHR type=write size=3517061 2026-02-24T22:10:47.073 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.069+0000 7f7e64854980 1 < completed write operation key=EB04ETNYM087 tag=SEPDIK1B4QWB type=write size=785375 2026-02-24T22:10:47.073 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.073+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=46B7YSYTML2B type=write size=150004 2026-02-24T22:10:47.074 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.073+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=IB14T2QRCYD7 type=del 2026-02-24T22:10:47.075 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.073+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=T8TVBPDMNLKJ type=del size=1490652 2026-02-24T22:10:47.076 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.073+0000 7f7e64854980 1 < completed delete operation key=78GUTC5THI2P tag=LAX1VNYI34XJ type=del 2026-02-24T22:10:47.077 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.073+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~8AFJ8WPJNL3F.0 size=1048576 2026-02-24T22:10:47.078 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.077+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~8AFJ8WPJNL3F.1 size=1048576 2026-02-24T22:10:47.078 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.077+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~8AFJ8WPJNL3F.2 size=185747 2026-02-24T22:10:47.078 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.077+0000 7f7e64854980 1 > prepared multipart upload key=A6PH77Q78VEN upload=8AFJ8WPJNL3F tag=HWWKZUCVEU0N type=write size=3331475 2026-02-24T22:10:47.079 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.077+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~3DED3JJ7ZPK4.0 size=1048576 2026-02-24T22:10:47.080 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.077+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~3DED3JJ7ZPK4.1 size=227877 2026-02-24T22:10:47.081 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.077+0000 7f7e64854980 1 < completed multipart upload key=GH7723GNI2WM upload=3DED3JJ7ZPK4 tag=G6LFMYEQJPVT type=write size=2325029 2026-02-24T22:10:47.082 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.081+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~1DJA7PAW2262.0 size=1048576 2026-02-24T22:10:47.083 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.081+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~1DJA7PAW2262.1 size=500218 2026-02-24T22:10:47.083 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.081+0000 7f7e64854980 1 > prepared multipart upload key=N3ZUNSI5K8HS upload=1DJA7PAW2262 tag=NA98WPLA7YAA type=write size=2597370 2026-02-24T22:10:47.084 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.081+0000 7f7e64854980 1 < completed delete operation key=EB04ETNYM087 tag=LY9A950SJXSS type=del 2026-02-24T22:10:47.085 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.081+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~6AKDBKPZJGRB.0 size=1048576 2026-02-24T22:10:47.086 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.085+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~6AKDBKPZJGRB.1 size=81984 2026-02-24T22:10:47.086 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.085+0000 7f7e64854980 1 > prepared multipart upload key=4Y3974PYYT78 upload=6AKDBKPZJGRB tag=HBWI5UYN3EQ5 type=write size=2179136 2026-02-24T22:10:47.087 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.085+0000 7f7e64854980 1 < completed delete operation key=EB04ETNYM087 tag=RRGN4QFH419B type=del 2026-02-24T22:10:47.087 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.085+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=6QGDSL39JDYI type=write size=1012931 2026-02-24T22:10:47.088 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.085+0000 7f7e64854980 1 < completed delete operation key=Q7BP8FXTARDW tag=Z4KD2XNK0H6K type=del 2026-02-24T22:10:47.089 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.085+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=VF6HX4I55HMU type=del size=2693922 2026-02-24T22:10:47.090 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.089+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~61FM9I3FSQ4W.0 size=1048576 2026-02-24T22:10:47.090 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.089+0000 7f7e64854980 1 < canceled part key=_multipart_PKRHWM69GBJI.2~61FM9I3FSQ4W.1 size=161092 2026-02-24T22:10:47.091 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.089+0000 7f7e64854980 1 < canceled multipart upload key=PKRHWM69GBJI upload=61FM9I3FSQ4W tag=LRWJWN9Q3F9F type=write size=2258244 2026-02-24T22:10:47.093 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.089+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~0R31RIKRG6M9.0 size=1048576 2026-02-24T22:10:47.093 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.089+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~0R31RIKRG6M9.1 size=1048576 2026-02-24T22:10:47.094 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.093+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~0R31RIKRG6M9.2 size=942106 2026-02-24T22:10:47.094 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.093+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=0R31RIKRG6M9 tag=Y4TP8A7Y37UK type=write size=4087834 2026-02-24T22:10:47.095 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.093+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~ENI0G0WVXKMS.0 size=1048576 2026-02-24T22:10:47.095 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.093+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~ENI0G0WVXKMS.1 size=533139 2026-02-24T22:10:47.096 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.093+0000 7f7e64854980 1 < completed multipart upload key=78GUTC5THI2P upload=ENI0G0WVXKMS tag=62GA7URUDQT6 type=write size=2630291 2026-02-24T22:10:47.097 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.097+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=ZTGYMHG1B5HL type=write size=939712 2026-02-24T22:10:47.099 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.097+0000 7f7e64854980 1 < suggested update operation key=SGQYQ2TIVJPS tag=FBRYOLU00PK5 type=del size=3865629 2026-02-24T22:10:47.100 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.097+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=RKDJ333MLGIU type=del size=2022477 2026-02-24T22:10:47.100 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.097+0000 7f7e64854980 1 < completed write operation key=HKOQRSF6XEXB tag=P7UAFZD33F2M type=write size=203979 2026-02-24T22:10:47.102 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.101+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~1R88ISU9HO6P.0 size=1048576 2026-02-24T22:10:47.102 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.101+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~1R88ISU9HO6P.1 size=1048576 2026-02-24T22:10:47.103 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.101+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~1R88ISU9HO6P.2 size=416084 2026-02-24T22:10:47.103 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.101+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=1R88ISU9HO6P tag=OIM725K7YPPM type=write size=3561812 2026-02-24T22:10:47.104 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.101+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~6GJMEP26RFYS.0 size=1048576 2026-02-24T22:10:47.104 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.101+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~6GJMEP26RFYS.1 size=1048576 2026-02-24T22:10:47.105 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.101+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~6GJMEP26RFYS.2 size=382556 2026-02-24T22:10:47.108 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.105+0000 7f7e64854980 1 < completed multipart upload key=36MKR2OZN7K9 upload=6GJMEP26RFYS tag=05KDM29V12U3 type=write size=3528284 2026-02-24T22:10:47.108 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.105+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=V9IR4YZJPL55 type=del size=59629 2026-02-24T22:10:47.109 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.105+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~GT43E76QDL97.0 size=183771 2026-02-24T22:10:47.110 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.109+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=GT43E76QDL97 tag=4Z5ZUM2ZQEI3 type=write size=1232347 2026-02-24T22:10:47.111 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.109+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=0B4OT0HSMITY type=del size=1962376 2026-02-24T22:10:47.112 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.109+0000 7f7e64854980 1 < canceled part key=_multipart_LSRPN33Z2B9N.2~NLNWDTQT9YWJ.0 size=1048576 2026-02-24T22:10:47.112 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.109+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~NLNWDTQT9YWJ.1 size=1048576 2026-02-24T22:10:47.113 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.109+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~NLNWDTQT9YWJ.2 size=201068 2026-02-24T22:10:47.114 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.113+0000 7f7e64854980 1 < canceled multipart upload key=LSRPN33Z2B9N upload=NLNWDTQT9YWJ tag=L6JZ28E1NYW6 type=write size=3346796 2026-02-24T22:10:47.115 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.113+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=12AM0872XF7X type=del size=3467349 2026-02-24T22:10:47.115 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.113+0000 7f7e64854980 1 < completed delete operation key=L60HQ7M3E3LL tag=T7T5A02WUZSB type=del 2026-02-24T22:10:47.117 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.113+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~ONV15VLW9Q02.0 size=1048576 2026-02-24T22:10:47.117 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.117+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~ONV15VLW9Q02.1 size=1048576 2026-02-24T22:10:47.118 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.117+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~ONV15VLW9Q02.2 size=1047320 2026-02-24T22:10:47.118 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.117+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=ONV15VLW9Q02 tag=N3UT1GUQ5TAC type=write size=4193048 2026-02-24T22:10:47.119 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.117+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~P7GPQS5TZG97.0 size=1048576 2026-02-24T22:10:47.119 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.117+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~P7GPQS5TZG97.1 size=1048576 2026-02-24T22:10:47.120 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.117+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~P7GPQS5TZG97.2 size=371333 2026-02-24T22:10:47.121 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.121+0000 7f7e64854980 1 < completed multipart upload key=HKOQRSF6XEXB upload=P7GPQS5TZG97 tag=7DTODH8WORHR type=write size=3517061 2026-02-24T22:10:47.123 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.121+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~LZ5NTATRBPZV.0 size=1048576 2026-02-24T22:10:47.124 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.121+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~LZ5NTATRBPZV.1 size=582842 2026-02-24T22:10:47.124 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.121+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=LZ5NTATRBPZV tag=6KU3QYSEGCG8 type=write size=2679994 2026-02-24T22:10:47.124 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.121+0000 7f7e64854980 1 < canceled operation key=L9NEDH44BIGA tag=46B7YSYTML2B type=write size=150004 2026-02-24T22:10:47.125 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.125+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=RZQ1JXJ59C49 type=del size=1803212 2026-02-24T22:10:47.126 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.125+0000 7f7e64854980 1 < completed delete operation key=0B2V0MHWSOC9 tag=T8TVBPDMNLKJ type=del 2026-02-24T22:10:47.127 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.125+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~32223JZAN408.0 size=1048576 2026-02-24T22:10:47.128 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.125+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~32223JZAN408.1 size=1048576 2026-02-24T22:10:47.129 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.125+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~32223JZAN408.2 size=611665 2026-02-24T22:10:47.129 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.125+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=32223JZAN408 tag=K694HVT7ADO4 type=write size=3757393 2026-02-24T22:10:47.130 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.129+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~8AFJ8WPJNL3F.0 size=1048576 2026-02-24T22:10:47.130 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.129+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~8AFJ8WPJNL3F.1 size=1048576 2026-02-24T22:10:47.131 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.129+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~8AFJ8WPJNL3F.2 size=185747 2026-02-24T22:10:47.132 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.129+0000 7f7e64854980 1 < completed multipart upload key=A6PH77Q78VEN upload=8AFJ8WPJNL3F tag=HWWKZUCVEU0N type=write size=3331475 2026-02-24T22:10:47.133 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.129+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=Z0MP3GOZPPB1 type=del size=300120 2026-02-24T22:10:47.134 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.133+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~1DJA7PAW2262.0 size=1048576 2026-02-24T22:10:47.134 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.133+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~1DJA7PAW2262.1 size=500218 2026-02-24T22:10:47.136 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.133+0000 7f7e64854980 1 < completed multipart upload key=N3ZUNSI5K8HS upload=1DJA7PAW2262 tag=NA98WPLA7YAA type=write size=2597370 2026-02-24T22:10:47.137 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.133+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~96UOL4NEAIJK.0 size=846396 2026-02-24T22:10:47.137 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.133+0000 7f7e64854980 1 > prepared multipart upload key=A6PH77Q78VEN upload=96UOL4NEAIJK tag=0KYJMDQTWFUL type=write size=1894972 2026-02-24T22:10:47.138 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.137+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~6AKDBKPZJGRB.0 size=1048576 2026-02-24T22:10:47.138 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.137+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~6AKDBKPZJGRB.1 size=81984 2026-02-24T22:10:47.140 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.137+0000 7f7e64854980 1 < completed multipart upload key=4Y3974PYYT78 upload=6AKDBKPZJGRB tag=HBWI5UYN3EQ5 type=write size=2179136 2026-02-24T22:10:47.141 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.137+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~1QEBZKB018FH.0 size=1048576 2026-02-24T22:10:47.142 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.141+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~1QEBZKB018FH.1 size=1048576 2026-02-24T22:10:47.142 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.141+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~1QEBZKB018FH.2 size=739519 2026-02-24T22:10:47.142 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.141+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=1QEBZKB018FH tag=IKKAKEUONCOD type=write size=3885247 2026-02-24T22:10:47.144 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.141+0000 7f7e64854980 1 < suggested update operation key=D0MKGNRDW6L9 tag=6QGDSL39JDYI type=write size=1012931 2026-02-24T22:10:47.144 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.141+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=656K3IGF57H4 type=del size=1020971 2026-02-24T22:10:47.146 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.145+0000 7f7e64854980 1 < suggested update operation key=Q7BP8FXTARDW tag=VF6HX4I55HMU type=del size=2693922 2026-02-24T22:10:47.147 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.145+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=TDLVTI0NT2FD type=write size=668540 2026-02-24T22:10:47.148 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.145+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~0R31RIKRG6M9.0 size=1048576 2026-02-24T22:10:47.148 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.145+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~0R31RIKRG6M9.1 size=1048576 2026-02-24T22:10:47.149 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.145+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~0R31RIKRG6M9.2 size=942106 2026-02-24T22:10:47.150 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.149+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=0R31RIKRG6M9 tag=Y4TP8A7Y37UK type=write size=4087834 2026-02-24T22:10:47.151 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.149+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=F3GVBJLVDDJ0 type=del size=2661492 2026-02-24T22:10:47.152 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.149+0000 7f7e64854980 1 < canceled operation key=PKRHWM69GBJI tag=ZTGYMHG1B5HL type=write size=939712 2026-02-24T22:10:47.153 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.149+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~DTNA2CWITXI1.0 size=1048576 2026-02-24T22:10:47.154 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.153+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~DTNA2CWITXI1.1 size=1048576 2026-02-24T22:10:47.154 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.153+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~DTNA2CWITXI1.2 size=823758 2026-02-24T22:10:47.154 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.153+0000 7f7e64854980 1 > prepared multipart upload key=78GUTC5THI2P upload=DTNA2CWITXI1 tag=J9MMRAJECZIJ type=write size=3969486 2026-02-24T22:10:47.155 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.153+0000 7f7e64854980 1 < canceled operation key=EJCX06TE5RTZ tag=RKDJ333MLGIU type=del size=2022477 2026-02-24T22:10:47.156 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.153+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=9YV0GJ2IEAOH type=del size=1270186 2026-02-24T22:10:47.157 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.153+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~1R88ISU9HO6P.0 size=1048576 2026-02-24T22:10:47.157 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.157+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~1R88ISU9HO6P.1 size=1048576 2026-02-24T22:10:47.158 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.157+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~1R88ISU9HO6P.2 size=416084 2026-02-24T22:10:47.159 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.157+0000 7f7e64854980 1 < completed multipart upload key=CI8DA17DX8LU upload=1R88ISU9HO6P tag=OIM725K7YPPM type=write size=3561812 2026-02-24T22:10:47.160 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.157+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=V218UX0DHT94 type=del size=3384053 2026-02-24T22:10:47.161 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.157+0000 7f7e64854980 1 < completed delete operation key=36MKR2OZN7K9 tag=V9IR4YZJPL55 type=del 2026-02-24T22:10:47.162 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.161+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=CX8CE42BAN74 type=del size=128839 2026-02-24T22:10:47.163 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.161+0000 7f7e64854980 1 < suggested remove operation key=HKOQRSF6XEXB tag=0B4OT0HSMITY type=del 2026-02-24T22:10:47.164 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.161+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=N066YW8WED4L type=del size=2447736 2026-02-24T22:10:47.165 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.165+0000 7f7e64854980 1 < suggested update operation key=3900SFJK1YQU tag=12AM0872XF7X type=del size=3467349 2026-02-24T22:10:47.167 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.165+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~NLB5XCC8K0R8.0 size=1048576 2026-02-24T22:10:47.168 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.165+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~NLB5XCC8K0R8.1 size=1048576 2026-02-24T22:10:47.168 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.165+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~NLB5XCC8K0R8.2 size=768411 2026-02-24T22:10:47.168 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.165+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=NLB5XCC8K0R8 tag=XRTZAG249IEF type=write size=3914139 2026-02-24T22:10:47.169 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.165+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~ONV15VLW9Q02.0 size=1048576 2026-02-24T22:10:47.169 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.169+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~ONV15VLW9Q02.1 size=1048576 2026-02-24T22:10:47.170 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.169+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~ONV15VLW9Q02.2 size=1047320 2026-02-24T22:10:47.171 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.169+0000 7f7e64854980 1 < completed multipart upload key=HOHKR92SN25M upload=ONV15VLW9Q02 tag=N3UT1GUQ5TAC type=write size=4193048 2026-02-24T22:10:47.172 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.169+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=DFKIHU4GKY7N type=del size=1050637 2026-02-24T22:10:47.173 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.169+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~LZ5NTATRBPZV.0 size=1048576 2026-02-24T22:10:47.174 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.173+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~LZ5NTATRBPZV.1 size=582842 2026-02-24T22:10:47.175 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.173+0000 7f7e64854980 1 < completed multipart upload key=D0MKGNRDW6L9 upload=LZ5NTATRBPZV tag=6KU3QYSEGCG8 type=write size=2679994 2026-02-24T22:10:47.176 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.173+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=2KZH4YG08W1G type=del size=717262 2026-02-24T22:10:47.176 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.173+0000 7f7e64854980 1 < completed delete operation key=4Y3974PYYT78 tag=RZQ1JXJ59C49 type=del 2026-02-24T22:10:47.178 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.177+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=TAKELN71IDQ0 type=del size=3610755 2026-02-24T22:10:47.179 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.177+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~32223JZAN408.0 size=1048576 2026-02-24T22:10:47.180 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.177+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~32223JZAN408.1 size=1048576 2026-02-24T22:10:47.180 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.177+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~32223JZAN408.2 size=611665 2026-02-24T22:10:47.182 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.181+0000 7f7e64854980 1 < completed multipart upload key=0AXS7DY23UZU upload=32223JZAN408 tag=K694HVT7ADO4 type=write size=3757393 2026-02-24T22:10:47.183 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.181+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=SOXE4VX5ELOM type=del size=1351015 2026-02-24T22:10:47.183 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.181+0000 7f7e64854980 1 < canceled operation key=L9NEDH44BIGA tag=Z0MP3GOZPPB1 type=del size=300120 2026-02-24T22:10:47.184 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.181+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=WY1EHJZSJH4F type=del size=4008923 2026-02-24T22:10:47.185 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.181+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~96UOL4NEAIJK.0 size=846396 2026-02-24T22:10:47.186 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.185+0000 7f7e64854980 1 < completed multipart upload key=A6PH77Q78VEN upload=96UOL4NEAIJK tag=0KYJMDQTWFUL type=write size=1894972 2026-02-24T22:10:47.187 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.185+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=IG6WOOF9F43A type=del size=1872810 2026-02-24T22:10:47.188 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.185+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~1QEBZKB018FH.0 size=1048576 2026-02-24T22:10:47.188 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.185+0000 7f7e64854980 1 < canceled part key=_multipart_TXRUQQ456DD8.2~1QEBZKB018FH.1 size=1048576 2026-02-24T22:10:47.188 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.185+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~1QEBZKB018FH.2 size=739519 2026-02-24T22:10:47.189 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.189+0000 7f7e64854980 1 < canceled multipart upload key=TXRUQQ456DD8 upload=1QEBZKB018FH tag=IKKAKEUONCOD type=write size=3885247 2026-02-24T22:10:47.191 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.189+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~DAFPH2CV3M25.0 size=1048576 2026-02-24T22:10:47.191 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.189+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~DAFPH2CV3M25.1 size=849659 2026-02-24T22:10:47.192 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.189+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=DAFPH2CV3M25 tag=Y0ZSYD9CWJ9L type=write size=2946811 2026-02-24T22:10:47.193 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.189+0000 7f7e64854980 1 < suggested update operation key=LSRPN33Z2B9N tag=656K3IGF57H4 type=del size=1020971 2026-02-24T22:10:47.194 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.193+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~FD9HIUVYMV8N.0 size=1048576 2026-02-24T22:10:47.195 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.193+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~FD9HIUVYMV8N.1 size=996468 2026-02-24T22:10:47.195 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.193+0000 7f7e64854980 1 > prepared multipart upload key=0B2V0MHWSOC9 upload=FD9HIUVYMV8N tag=E2AA7HUJ5U4J type=write size=3093620 2026-02-24T22:10:47.196 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.193+0000 7f7e64854980 1 < suggested remove operation key=Q7BP8FXTARDW tag=TDLVTI0NT2FD type=write 2026-02-24T22:10:47.198 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.197+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~J71VN8UCK937.0 size=1048576 2026-02-24T22:10:47.198 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.197+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~J71VN8UCK937.1 size=1048576 2026-02-24T22:10:47.199 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.197+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~J71VN8UCK937.2 size=577404 2026-02-24T22:10:47.199 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.197+0000 7f7e64854980 1 > prepared multipart upload key=SGQYQ2TIVJPS upload=J71VN8UCK937 tag=56E203DBJZVV type=write size=3723132 2026-02-24T22:10:47.200 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.197+0000 7f7e64854980 1 < completed delete operation key=L9NEDH44BIGA tag=F3GVBJLVDDJ0 type=del 2026-02-24T22:10:47.200 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.197+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=N6VKW987IDYP type=del size=1810914 2026-02-24T22:10:47.201 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.201+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~DTNA2CWITXI1.0 size=1048576 2026-02-24T22:10:47.201 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.201+0000 7f7e64854980 1 < canceled part key=_multipart_78GUTC5THI2P.2~DTNA2CWITXI1.1 size=1048576 2026-02-24T22:10:47.202 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.201+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~DTNA2CWITXI1.2 size=823758 2026-02-24T22:10:47.203 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.201+0000 7f7e64854980 1 < canceled multipart upload key=78GUTC5THI2P upload=DTNA2CWITXI1 tag=J9MMRAJECZIJ type=write size=3969486 2026-02-24T22:10:47.204 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.201+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~IQX6JUPUYF9K.0 size=1048576 2026-02-24T22:10:47.205 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.201+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~IQX6JUPUYF9K.1 size=1048576 2026-02-24T22:10:47.206 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.205+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~IQX6JUPUYF9K.2 size=85720 2026-02-24T22:10:47.206 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.205+0000 7f7e64854980 1 > prepared multipart upload key=4Y3974PYYT78 upload=IQX6JUPUYF9K tag=3313KEOR3Q94 type=write size=3231448 2026-02-24T22:10:47.206 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.205+0000 7f7e64854980 1 < completed delete operation key=2KKQKHXO7K3K tag=9YV0GJ2IEAOH type=del 2026-02-24T22:10:47.207 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.205+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=Y4RDVT05Y12T type=del size=2916155 2026-02-24T22:10:47.208 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.205+0000 7f7e64854980 1 < completed delete operation key=PKRHWM69GBJI tag=V218UX0DHT94 type=del 2026-02-24T22:10:47.209 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.205+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=H81YII2TR2E0 type=del size=3750643 2026-02-24T22:10:47.209 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.209+0000 7f7e64854980 1 < completed delete operation key=EJCX06TE5RTZ tag=CX8CE42BAN74 type=del 2026-02-24T22:10:47.211 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.209+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~VJMXKCL2NJ80.0 size=1048576 2026-02-24T22:10:47.211 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.209+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~VJMXKCL2NJ80.1 size=1048576 2026-02-24T22:10:47.212 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.209+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~VJMXKCL2NJ80.2 size=142924 2026-02-24T22:10:47.212 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.209+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=VJMXKCL2NJ80 tag=VJIQDQZCNWEQ type=write size=3288652 2026-02-24T22:10:47.213 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.213+0000 7f7e64854980 1 < suggested remove operation key=NL60Q5QOSP8E tag=N066YW8WED4L type=del 2026-02-24T22:10:47.214 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.213+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=48B6WDGXOO3D type=write size=1046909 2026-02-24T22:10:47.215 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.213+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~NLB5XCC8K0R8.0 size=1048576 2026-02-24T22:10:47.216 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.213+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~NLB5XCC8K0R8.1 size=1048576 2026-02-24T22:10:47.216 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.213+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~NLB5XCC8K0R8.2 size=768411 2026-02-24T22:10:47.218 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.217+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=NLB5XCC8K0R8 tag=XRTZAG249IEF type=write size=3914139 2026-02-24T22:10:47.218 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.217+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=PZZ36F93XCIC type=del size=48028 2026-02-24T22:10:47.219 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.217+0000 7f7e64854980 1 < canceled operation key=N3ZUNSI5K8HS tag=DFKIHU4GKY7N type=del size=1050637 2026-02-24T22:10:47.220 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.217+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=75AXQ1WZXAP1 type=write size=525742 2026-02-24T22:10:47.221 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.217+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=2KZH4YG08W1G type=del 2026-02-24T22:10:47.221 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.221+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=Q6PI9TGDW783 type=del size=1042409 2026-02-24T22:10:47.222 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.221+0000 7f7e64854980 1 < completed delete operation key=SGQYQ2TIVJPS tag=TAKELN71IDQ0 type=del 2026-02-24T22:10:47.223 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.221+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=NU3FDCZW4XWO type=del size=4043701 2026-02-24T22:10:47.224 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.221+0000 7f7e64854980 1 < completed delete operation key=Z0OXREAUJVA8 tag=SOXE4VX5ELOM type=del 2026-02-24T22:10:47.225 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.221+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~WV7VS5ERWO5Z.0 size=389040 2026-02-24T22:10:47.225 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.221+0000 7f7e64854980 1 > prepared multipart upload key=Q7BP8FXTARDW upload=WV7VS5ERWO5Z tag=1T65OOKKKHAG type=write size=1437616 2026-02-24T22:10:47.226 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.225+0000 7f7e64854980 1 < completed delete operation key=0AXS7DY23UZU tag=WY1EHJZSJH4F type=del 2026-02-24T22:10:47.227 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.225+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=18CK50BKN5DS type=del size=2696492 2026-02-24T22:10:47.227 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.225+0000 7f7e64854980 1 < canceled operation key=Z0OXREAUJVA8 tag=IG6WOOF9F43A type=del size=1872810 2026-02-24T22:10:47.229 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.225+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~4K62RQA0WCGO.0 size=19210 2026-02-24T22:10:47.229 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.225+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=4K62RQA0WCGO tag=L5MLL3C3NCH0 type=write size=1067786 2026-02-24T22:10:47.230 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.229+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~DAFPH2CV3M25.0 size=1048576 2026-02-24T22:10:47.230 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.229+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~DAFPH2CV3M25.1 size=849659 2026-02-24T22:10:47.231 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.229+0000 7f7e64854980 1 < completed multipart upload key=TXRUQQ456DD8 upload=DAFPH2CV3M25 tag=Y0ZSYD9CWJ9L type=write size=2946811 2026-02-24T22:10:47.232 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.229+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=OLJQS4FC2K8Y type=del size=133049 2026-02-24T22:10:47.233 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.229+0000 7f7e64854980 1 < canceled part key=_multipart_0B2V0MHWSOC9.2~FD9HIUVYMV8N.0 size=1048576 2026-02-24T22:10:47.233 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.229+0000 7f7e64854980 1 < canceled part key=_multipart_0B2V0MHWSOC9.2~FD9HIUVYMV8N.1 size=996468 2026-02-24T22:10:47.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.233+0000 7f7e64854980 1 < canceled multipart upload key=0B2V0MHWSOC9 upload=FD9HIUVYMV8N tag=E2AA7HUJ5U4J type=write size=3093620 2026-02-24T22:10:47.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.233+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=XRKAID27AGDA type=del size=822092 2026-02-24T22:10:47.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.233+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~J71VN8UCK937.0 size=1048576 2026-02-24T22:10:47.236 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.233+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~J71VN8UCK937.1 size=1048576 2026-02-24T22:10:47.236 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.233+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~J71VN8UCK937.2 size=577404 2026-02-24T22:10:47.238 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.237+0000 7f7e64854980 1 < completed multipart upload key=SGQYQ2TIVJPS upload=J71VN8UCK937 tag=56E203DBJZVV type=write size=3723132 2026-02-24T22:10:47.239 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.237+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~QAGVIRTV1TO8.0 size=1012426 2026-02-24T22:10:47.239 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.237+0000 7f7e64854980 1 > prepared multipart upload key=Z0OXREAUJVA8 upload=QAGVIRTV1TO8 tag=29MPSLBIZ1H4 type=write size=2061002 2026-02-24T22:10:47.240 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.237+0000 7f7e64854980 1 < completed delete operation key=A6PH77Q78VEN tag=N6VKW987IDYP type=del 2026-02-24T22:10:47.241 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.241+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~1HFRUV646M2W.0 size=1048576 2026-02-24T22:10:47.242 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.241+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~1HFRUV646M2W.1 size=1048576 2026-02-24T22:10:47.242 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.241+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~1HFRUV646M2W.2 size=335574 2026-02-24T22:10:47.243 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.241+0000 7f7e64854980 1 > prepared multipart upload key=LSRPN33Z2B9N upload=1HFRUV646M2W tag=GWDNXNMXGZSW type=write size=3481302 2026-02-24T22:10:47.243 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.241+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~IQX6JUPUYF9K.0 size=1048576 2026-02-24T22:10:47.243 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.241+0000 7f7e64854980 1 < canceled part key=_multipart_4Y3974PYYT78.2~IQX6JUPUYF9K.1 size=1048576 2026-02-24T22:10:47.244 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.241+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~IQX6JUPUYF9K.2 size=85720 2026-02-24T22:10:47.245 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.241+0000 7f7e64854980 1 < canceled multipart upload key=4Y3974PYYT78 upload=IQX6JUPUYF9K tag=3313KEOR3Q94 type=write size=3231448 2026-02-24T22:10:47.246 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.245+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=F11E1XCZGWPV type=write size=701976 2026-02-24T22:10:47.247 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.245+0000 7f7e64854980 1 < completed delete operation key=ZESKGBWZXM6O tag=Y4RDVT05Y12T type=del 2026-02-24T22:10:47.247 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.245+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=HATEXB5UEF80 type=del size=2307239 2026-02-24T22:10:47.250 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.249+0000 7f7e64854980 1 < suggested remove operation key=Y3L3CJ51C1WR tag=H81YII2TR2E0 type=del 2026-02-24T22:10:47.251 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.249+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=0IQLRE2QYRYN type=del size=1419301 2026-02-24T22:10:47.252 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.249+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~VJMXKCL2NJ80.0 size=1048576 2026-02-24T22:10:47.252 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.249+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~VJMXKCL2NJ80.1 size=1048576 2026-02-24T22:10:47.253 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.249+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~VJMXKCL2NJ80.2 size=142924 2026-02-24T22:10:47.254 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.253+0000 7f7e64854980 1 < completed multipart upload key=EJCX06TE5RTZ upload=VJMXKCL2NJ80 tag=VJIQDQZCNWEQ type=write size=3288652 2026-02-24T22:10:47.255 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.253+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=D72TDXDKXH4J type=del size=2080634 2026-02-24T22:10:47.256 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.253+0000 7f7e64854980 1 < canceled operation key=78GUTC5THI2P tag=48B6WDGXOO3D type=write size=1046909 2026-02-24T22:10:47.256 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.253+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=SBB2ZALYCI3S type=del size=3145272 2026-02-24T22:10:47.257 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.253+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=PZZ36F93XCIC type=del 2026-02-24T22:10:47.258 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.257+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=WVJ2RL734FDW type=del size=2605477 2026-02-24T22:10:47.259 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.257+0000 7f7e64854980 1 < canceled operation key=36MKR2OZN7K9 tag=75AXQ1WZXAP1 type=write size=525742 2026-02-24T22:10:47.259 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.257+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=I1B8VEHRINH1 type=del size=2618160 2026-02-24T22:10:47.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.257+0000 7f7e64854980 1 < completed delete operation key=HKOQRSF6XEXB tag=Q6PI9TGDW783 type=del 2026-02-24T22:10:47.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.261+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~CU047CDNE5AS.0 size=1048576 2026-02-24T22:10:47.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.261+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~CU047CDNE5AS.1 size=268443 2026-02-24T22:10:47.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.261+0000 7f7e64854980 1 > prepared multipart upload key=78GUTC5THI2P upload=CU047CDNE5AS tag=I0VIWGK3D55P type=write size=2365595 2026-02-24T22:10:47.263 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.261+0000 7f7e64854980 1 < completed delete operation key=A6PH77Q78VEN tag=NU3FDCZW4XWO type=del 2026-02-24T22:10:47.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.261+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=AYX60TKS2KCO type=del size=65198 2026-02-24T22:10:47.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.261+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~WV7VS5ERWO5Z.0 size=389040 2026-02-24T22:10:47.266 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.265+0000 7f7e64854980 1 < completed multipart upload key=Q7BP8FXTARDW upload=WV7VS5ERWO5Z tag=1T65OOKKKHAG type=write size=1437616 2026-02-24T22:10:47.267 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.265+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~7YZLSW9KL2H7.0 size=966819 2026-02-24T22:10:47.267 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.265+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=7YZLSW9KL2H7 tag=3EGZ0QEERL7A type=write size=2015395 2026-02-24T22:10:47.269 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.265+0000 7f7e64854980 1 < suggested update operation key=A6PH77Q78VEN tag=18CK50BKN5DS type=del size=2696492 2026-02-24T22:10:47.270 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.269+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~EX0A80TLXY30.0 size=1048576 2026-02-24T22:10:47.270 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.269+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~EX0A80TLXY30.1 size=1048576 2026-02-24T22:10:47.271 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.269+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~EX0A80TLXY30.2 size=735751 2026-02-24T22:10:47.271 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.269+0000 7f7e64854980 1 > prepared multipart upload key=4Y3974PYYT78 upload=EX0A80TLXY30 tag=Q00QR53ML57K type=write size=3881479 2026-02-24T22:10:47.272 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.269+0000 7f7e64854980 1 < canceled part key=_multipart_D0MKGNRDW6L9.2~4K62RQA0WCGO.0 size=19210 2026-02-24T22:10:47.272 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.269+0000 7f7e64854980 1 < canceled multipart upload key=D0MKGNRDW6L9 upload=4K62RQA0WCGO tag=L5MLL3C3NCH0 type=write size=1067786 2026-02-24T22:10:47.273 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.273+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=JBTM6DAWX995 type=del size=3946717 2026-02-24T22:10:47.274 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.273+0000 7f7e64854980 1 < canceled operation key=PKRHWM69GBJI tag=OLJQS4FC2K8Y type=del size=133049 2026-02-24T22:10:47.275 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.273+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=8NJP20I3VNX0 type=del size=1279099 2026-02-24T22:10:47.275 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.273+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=XRKAID27AGDA type=del 2026-02-24T22:10:47.277 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.273+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~NL4T6NVEFOAT.0 size=1048576 2026-02-24T22:10:47.277 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.277+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~NL4T6NVEFOAT.1 size=963680 2026-02-24T22:10:47.278 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.277+0000 7f7e64854980 1 > prepared multipart upload key=N3ZUNSI5K8HS upload=NL4T6NVEFOAT tag=QYH52N19KHK1 type=write size=3060832 2026-02-24T22:10:47.278 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.277+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~QAGVIRTV1TO8.0 size=1012426 2026-02-24T22:10:47.280 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.277+0000 7f7e64854980 1 < completed multipart upload key=Z0OXREAUJVA8 upload=QAGVIRTV1TO8 tag=29MPSLBIZ1H4 type=write size=2061002 2026-02-24T22:10:47.280 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.277+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=IOWFPH3GDPJX type=del size=1524654 2026-02-24T22:10:47.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.277+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~1HFRUV646M2W.0 size=1048576 2026-02-24T22:10:47.282 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.281+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~1HFRUV646M2W.1 size=1048576 2026-02-24T22:10:47.282 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.281+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~1HFRUV646M2W.2 size=335574 2026-02-24T22:10:47.284 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.281+0000 7f7e64854980 1 < completed multipart upload key=LSRPN33Z2B9N upload=1HFRUV646M2W tag=GWDNXNMXGZSW type=write size=3481302 2026-02-24T22:10:47.284 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.281+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=FS6J2TJQYVFD type=del size=467277 2026-02-24T22:10:47.285 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.285+0000 7f7e64854980 1 < completed write operation key=4Y3974PYYT78 tag=F11E1XCZGWPV type=write size=701976 2026-02-24T22:10:47.286 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.285+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=OV1GZOO0SN5Z type=del size=1667125 2026-02-24T22:10:47.288 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.285+0000 7f7e64854980 1 < suggested remove operation key=TXRUQQ456DD8 tag=HATEXB5UEF80 type=del 2026-02-24T22:10:47.288 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.285+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=P08NIXTW2MY1 type=del size=457842 2026-02-24T22:10:47.289 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.285+0000 7f7e64854980 1 < canceled operation key=HOHKR92SN25M tag=0IQLRE2QYRYN type=del size=1419301 2026-02-24T22:10:47.290 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.289+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~IJL65AC5T3YL.0 size=457293 2026-02-24T22:10:47.290 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.289+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=IJL65AC5T3YL tag=FXIECOTXJA23 type=write size=1505869 2026-02-24T22:10:47.291 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.289+0000 7f7e64854980 1 < completed delete operation key=UQLUT9ZAI61V tag=D72TDXDKXH4J type=del 2026-02-24T22:10:47.293 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.289+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~XWJ03NAMGVJ1.0 size=1048576 2026-02-24T22:10:47.293 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.289+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~XWJ03NAMGVJ1.1 size=1048576 2026-02-24T22:10:47.294 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.293+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~XWJ03NAMGVJ1.2 size=832904 2026-02-24T22:10:47.294 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.293+0000 7f7e64854980 1 > prepared multipart upload key=3900SFJK1YQU upload=XWJ03NAMGVJ1 tag=OXQGU7ZHEN15 type=write size=3978632 2026-02-24T22:10:47.295 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.293+0000 7f7e64854980 1 < completed delete operation key=UQLUT9ZAI61V tag=SBB2ZALYCI3S type=del 2026-02-24T22:10:47.295 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.293+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=ZQ2E0A3LYLSG type=del size=3486869 2026-02-24T22:10:47.296 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.293+0000 7f7e64854980 1 < completed delete operation key=3900SFJK1YQU tag=WVJ2RL734FDW type=del 2026-02-24T22:10:47.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.297+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~C4B9AW5RVO39.0 size=1048576 2026-02-24T22:10:47.298 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.297+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~C4B9AW5RVO39.1 size=787164 2026-02-24T22:10:47.298 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.297+0000 7f7e64854980 1 > prepared multipart upload key=NL60Q5QOSP8E upload=C4B9AW5RVO39 tag=J8N80DH820L9 type=write size=2884316 2026-02-24T22:10:47.300 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.297+0000 7f7e64854980 1 < suggested update operation key=ZESKGBWZXM6O tag=I1B8VEHRINH1 type=del size=2618160 2026-02-24T22:10:47.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.297+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=QO4RM1T137JW type=write size=468843 2026-02-24T22:10:47.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.297+0000 7f7e64854980 1 < canceled part key=_multipart_78GUTC5THI2P.2~CU047CDNE5AS.0 size=1048576 2026-02-24T22:10:47.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.297+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~CU047CDNE5AS.1 size=268443 2026-02-24T22:10:47.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.301+0000 7f7e64854980 1 < canceled multipart upload key=78GUTC5THI2P upload=CU047CDNE5AS tag=I0VIWGK3D55P type=write size=2365595 2026-02-24T22:10:47.304 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.301+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~N7J8WN534HOS.0 size=1048576 2026-02-24T22:10:47.304 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.301+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~N7J8WN534HOS.1 size=1048576 2026-02-24T22:10:47.305 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.301+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~N7J8WN534HOS.2 size=943597 2026-02-24T22:10:47.305 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.301+0000 7f7e64854980 1 > prepared multipart upload key=GH7723GNI2WM upload=N7J8WN534HOS tag=OI0FYDMR58MD type=write size=4089325 2026-02-24T22:10:47.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.305+0000 7f7e64854980 1 < completed delete operation key=SGQYQ2TIVJPS tag=AYX60TKS2KCO type=del 2026-02-24T22:10:47.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.305+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=PGNBSJ4PCD5N type=del size=1289036 2026-02-24T22:10:47.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.305+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~7YZLSW9KL2H7.0 size=966819 2026-02-24T22:10:47.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.305+0000 7f7e64854980 1 < completed multipart upload key=CI8DA17DX8LU upload=7YZLSW9KL2H7 tag=3EGZ0QEERL7A type=write size=2015395 2026-02-24T22:10:47.310 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.309+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~ZGK3WAXXKG8P.0 size=1048576 2026-02-24T22:10:47.310 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.309+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~ZGK3WAXXKG8P.1 size=464892 2026-02-24T22:10:47.311 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.309+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=ZGK3WAXXKG8P tag=DM0KOLR1WP82 type=write size=2562044 2026-02-24T22:10:47.311 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.309+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~EX0A80TLXY30.0 size=1048576 2026-02-24T22:10:47.312 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.309+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~EX0A80TLXY30.1 size=1048576 2026-02-24T22:10:47.312 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.309+0000 7f7e64854980 1 < canceled part key=_multipart_4Y3974PYYT78.2~EX0A80TLXY30.2 size=735751 2026-02-24T22:10:47.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.309+0000 7f7e64854980 1 < canceled multipart upload key=4Y3974PYYT78 upload=EX0A80TLXY30 tag=Q00QR53ML57K type=write size=3881479 2026-02-24T22:10:47.314 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.313+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~MFDUN11F774R.0 size=1048576 2026-02-24T22:10:47.315 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.313+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~MFDUN11F774R.1 size=1048576 2026-02-24T22:10:47.316 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.313+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~MFDUN11F774R.2 size=370112 2026-02-24T22:10:47.316 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.313+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=MFDUN11F774R tag=9ER20FYGEHGZ type=write size=3515840 2026-02-24T22:10:47.316 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.313+0000 7f7e64854980 -1 < failed to complete operation key=ZESKGBWZXM6O tag=JBTM6DAWX995 type=del size=3946717: (22) Invalid argument 2026-02-24T22:10:47.317 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.317+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=UWQ9KWXV3Z4E type=del size=2240125 2026-02-24T22:10:47.318 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.317+0000 7f7e64854980 1 < completed delete operation key=0B2V0MHWSOC9 tag=8NJP20I3VNX0 type=del 2026-02-24T22:10:47.319 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.317+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=OBR7IGFLGROA type=write size=266490 2026-02-24T22:10:47.321 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.317+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~NL4T6NVEFOAT.0 size=1048576 2026-02-24T22:10:47.321 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.321+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~NL4T6NVEFOAT.1 size=963680 2026-02-24T22:10:47.323 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.321+0000 7f7e64854980 1 < completed multipart upload key=N3ZUNSI5K8HS upload=NL4T6NVEFOAT tag=QYH52N19KHK1 type=write size=3060832 2026-02-24T22:10:47.323 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.321+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=XHKEVEY89KQC type=del size=2648138 2026-02-24T22:10:47.324 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.321+0000 7f7e64854980 1 < completed delete operation key=UQLUT9ZAI61V tag=IOWFPH3GDPJX type=del 2026-02-24T22:10:47.325 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.321+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=MP6FQQJ0YWOI type=del size=1711365 2026-02-24T22:10:47.327 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.325+0000 7f7e64854980 1 < suggested remove operation key=EB04ETNYM087 tag=FS6J2TJQYVFD type=del 2026-02-24T22:10:47.328 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.325+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~LZJL7BA8NW7Z.0 size=1048576 2026-02-24T22:10:47.329 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.325+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~LZJL7BA8NW7Z.1 size=376378 2026-02-24T22:10:47.329 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.325+0000 7f7e64854980 1 > prepared multipart upload key=ON16L53JUSTW upload=LZJL7BA8NW7Z tag=8TTG3228LCFC type=write size=2473530 2026-02-24T22:10:47.329 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.329+0000 7f7e64854980 1 < completed delete operation key=L9NEDH44BIGA tag=OV1GZOO0SN5Z type=del 2026-02-24T22:10:47.331 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.329+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~G0L88BCQ5U93.0 size=11730 2026-02-24T22:10:47.331 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.329+0000 7f7e64854980 1 > prepared multipart upload key=Q7BP8FXTARDW upload=G0L88BCQ5U93 tag=N8F0A0KXO7JW type=write size=1060306 2026-02-24T22:10:47.331 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.329+0000 7f7e64854980 -1 < failed to complete operation key=ZESKGBWZXM6O tag=P08NIXTW2MY1 type=del size=457842: (22) Invalid argument 2026-02-24T22:10:47.333 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.329+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~DTNLOPPANT3J.0 size=1048576 2026-02-24T22:10:47.333 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.333+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~DTNLOPPANT3J.1 size=804659 2026-02-24T22:10:47.333 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.333+0000 7f7e64854980 1 > prepared multipart upload key=EB04ETNYM087 upload=DTNLOPPANT3J tag=1JKNWAU2TCOT type=write size=2901811 2026-02-24T22:10:47.334 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.333+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~IJL65AC5T3YL.0 size=457293 2026-02-24T22:10:47.335 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.333+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=IJL65AC5T3YL tag=FXIECOTXJA23 type=write size=1505869 2026-02-24T22:10:47.336 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.333+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=2ISOB65F4YND type=del size=2357323 2026-02-24T22:10:47.337 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.333+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~XWJ03NAMGVJ1.0 size=1048576 2026-02-24T22:10:47.337 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.333+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~XWJ03NAMGVJ1.1 size=1048576 2026-02-24T22:10:47.338 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.337+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~XWJ03NAMGVJ1.2 size=832904 2026-02-24T22:10:47.339 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.337+0000 7f7e64854980 1 < completed multipart upload key=3900SFJK1YQU upload=XWJ03NAMGVJ1 tag=OXQGU7ZHEN15 type=write size=3978632 2026-02-24T22:10:47.340 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.337+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=4L8UGM3F7370 type=del size=3718107 2026-02-24T22:10:47.340 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.337+0000 7f7e64854980 1 < completed delete operation key=NL60Q5QOSP8E tag=ZQ2E0A3LYLSG type=del 2026-02-24T22:10:47.342 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.341+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~BIHL4YL3NXLA.0 size=1048576 2026-02-24T22:10:47.342 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.341+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~BIHL4YL3NXLA.1 size=1048576 2026-02-24T22:10:47.343 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.341+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~BIHL4YL3NXLA.2 size=891042 2026-02-24T22:10:47.343 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.341+0000 7f7e64854980 1 > prepared multipart upload key=Y3L3CJ51C1WR upload=BIHL4YL3NXLA tag=505FZ7Z8NXF0 type=write size=4036770 2026-02-24T22:10:47.343 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.341+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~C4B9AW5RVO39.0 size=1048576 2026-02-24T22:10:47.344 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.341+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~C4B9AW5RVO39.1 size=787164 2026-02-24T22:10:47.345 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.341+0000 7f7e64854980 1 < completed multipart upload key=NL60Q5QOSP8E upload=C4B9AW5RVO39 tag=J8N80DH820L9 type=write size=2884316 2026-02-24T22:10:47.346 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.345+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=DR3WEVX35UWZ type=del size=40334 2026-02-24T22:10:47.347 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.345+0000 7f7e64854980 1 < completed write operation key=N3ZUNSI5K8HS tag=QO4RM1T137JW type=write size=468843 2026-02-24T22:10:47.347 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.345+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=POUPKBSP9B45 type=del size=959920 2026-02-24T22:10:47.348 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.345+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~N7J8WN534HOS.0 size=1048576 2026-02-24T22:10:47.349 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.345+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~N7J8WN534HOS.1 size=1048576 2026-02-24T22:10:47.349 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.349+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~N7J8WN534HOS.2 size=943597 2026-02-24T22:10:47.350 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.349+0000 7f7e64854980 1 < completed multipart upload key=GH7723GNI2WM upload=N7J8WN534HOS tag=OI0FYDMR58MD type=write size=4089325 2026-02-24T22:10:47.351 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.349+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=WSYBUWIE3Z0I type=del size=1153971 2026-02-24T22:10:47.352 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.349+0000 7f7e64854980 1 < canceled operation key=ZESKGBWZXM6O tag=PGNBSJ4PCD5N type=del size=1289036 2026-02-24T22:10:47.353 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.349+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=L3KZ0S1Z502G type=del size=1765326 2026-02-24T22:10:47.353 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.349+0000 7f7e64854980 1 < canceled part key=_multipart_TXRUQQ456DD8.2~ZGK3WAXXKG8P.0 size=1048576 2026-02-24T22:10:47.353 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.353+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~ZGK3WAXXKG8P.1 size=464892 2026-02-24T22:10:47.354 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.353+0000 7f7e64854980 1 < canceled multipart upload key=TXRUQQ456DD8 upload=ZGK3WAXXKG8P tag=DM0KOLR1WP82 type=write size=2562044 2026-02-24T22:10:47.355 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.353+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=ZZW1U0RFF3SZ type=del size=3114735 2026-02-24T22:10:47.356 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.353+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~MFDUN11F774R.0 size=1048576 2026-02-24T22:10:47.356 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.353+0000 7f7e64854980 1 < canceled part key=_multipart_TXRUQQ456DD8.2~MFDUN11F774R.1 size=1048576 2026-02-24T22:10:47.356 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.353+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~MFDUN11F774R.2 size=370112 2026-02-24T22:10:47.358 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.357+0000 7f7e64854980 1 < canceled multipart upload key=TXRUQQ456DD8 upload=MFDUN11F774R tag=9ER20FYGEHGZ type=write size=3515840 2026-02-24T22:10:47.358 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.357+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=1KE35I4VTS1K type=write size=345333 2026-02-24T22:10:47.359 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.357+0000 7f7e64854980 1 < canceled operation key=HKOQRSF6XEXB tag=UWQ9KWXV3Z4E type=del size=2240125 2026-02-24T22:10:47.360 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.357+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=5GWF77KZZYFT type=write size=169535 2026-02-24T22:10:47.361 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.357+0000 7f7e64854980 1 < canceled operation key=NL60Q5QOSP8E tag=OBR7IGFLGROA type=write size=266490 2026-02-24T22:10:47.361 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.361+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=49K10UHNQBBV type=write size=518099 2026-02-24T22:10:47.362 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.361+0000 7f7e64854980 -1 < failed to complete operation key=EB04ETNYM087 tag=XHKEVEY89KQC type=del size=2648138: (22) Invalid argument 2026-02-24T22:10:47.363 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.361+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=HIKG2174PXJQ type=del size=2652651 2026-02-24T22:10:47.363 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.361+0000 7f7e64854980 1 < completed delete operation key=Q7BP8FXTARDW tag=MP6FQQJ0YWOI type=del 2026-02-24T22:10:47.365 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.361+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~UKZR4ZBPE7QK.0 size=1048576 2026-02-24T22:10:47.365 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.365+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~UKZR4ZBPE7QK.1 size=1048576 2026-02-24T22:10:47.366 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.365+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~UKZR4ZBPE7QK.2 size=156796 2026-02-24T22:10:47.366 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.365+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=UKZR4ZBPE7QK tag=O9PDRPKWJ37R type=write size=3302524 2026-02-24T22:10:47.366 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.365+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~LZJL7BA8NW7Z.0 size=1048576 2026-02-24T22:10:47.367 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.365+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~LZJL7BA8NW7Z.1 size=376378 2026-02-24T22:10:47.368 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.365+0000 7f7e64854980 1 < completed multipart upload key=ON16L53JUSTW upload=LZJL7BA8NW7Z tag=8TTG3228LCFC type=write size=2473530 2026-02-24T22:10:47.369 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.369+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~HXQWWH7KUGIL.0 size=888458 2026-02-24T22:10:47.370 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.369+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=HXQWWH7KUGIL tag=ORN5N1866NUD type=write size=1937034 2026-02-24T22:10:47.370 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.369+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~G0L88BCQ5U93.0 size=11730 2026-02-24T22:10:47.371 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.369+0000 7f7e64854980 1 < completed multipart upload key=Q7BP8FXTARDW upload=G0L88BCQ5U93 tag=N8F0A0KXO7JW type=write size=1060306 2026-02-24T22:10:47.373 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.369+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~5OT0SJVGWAT1.0 size=1048576 2026-02-24T22:10:47.373 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.369+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~5OT0SJVGWAT1.1 size=908529 2026-02-24T22:10:47.373 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.369+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=5OT0SJVGWAT1 tag=3YIP7UU2UTOX type=write size=3005681 2026-02-24T22:10:47.374 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.373+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~DTNLOPPANT3J.0 size=1048576 2026-02-24T22:10:47.374 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.373+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~DTNLOPPANT3J.1 size=804659 2026-02-24T22:10:47.376 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.373+0000 7f7e64854980 1 < completed multipart upload key=EB04ETNYM087 upload=DTNLOPPANT3J tag=1JKNWAU2TCOT type=write size=2901811 2026-02-24T22:10:47.376 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.373+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=51DS2WQE2S5L type=del size=4096738 2026-02-24T22:10:47.377 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.373+0000 7f7e64854980 1 < completed delete operation key=PKRHWM69GBJI tag=2ISOB65F4YND type=del 2026-02-24T22:10:47.378 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.377+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~ZOGH00G52VR7.0 size=1029918 2026-02-24T22:10:47.378 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.377+0000 7f7e64854980 1 > prepared multipart upload key=EB04ETNYM087 upload=ZOGH00G52VR7 tag=EUHHOKPYG2QP type=write size=2078494 2026-02-24T22:10:47.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.377+0000 7f7e64854980 1 < completed delete operation key=HKOQRSF6XEXB tag=4L8UGM3F7370 type=del 2026-02-24T22:10:47.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.377+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~DHVJBSG441PH.0 size=754326 2026-02-24T22:10:47.381 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.377+0000 7f7e64854980 1 > prepared multipart upload key=NL60Q5QOSP8E upload=DHVJBSG441PH tag=DQTDGF6ZW1D3 type=write size=1802902 2026-02-24T22:10:47.381 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.377+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~BIHL4YL3NXLA.0 size=1048576 2026-02-24T22:10:47.382 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.381+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~BIHL4YL3NXLA.1 size=1048576 2026-02-24T22:10:47.382 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.381+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~BIHL4YL3NXLA.2 size=891042 2026-02-24T22:10:47.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.381+0000 7f7e64854980 1 < completed multipart upload key=Y3L3CJ51C1WR upload=BIHL4YL3NXLA tag=505FZ7Z8NXF0 type=write size=4036770 2026-02-24T22:10:47.384 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.381+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=B8SEMQ0FJBP0 type=write size=624475 2026-02-24T22:10:47.385 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.381+0000 7f7e64854980 1 < completed delete operation key=ZESKGBWZXM6O tag=DR3WEVX35UWZ type=del 2026-02-24T22:10:47.387 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.385+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~DEK2EAH2Q744.0 size=1048576 2026-02-24T22:10:47.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.385+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~DEK2EAH2Q744.1 size=1048576 2026-02-24T22:10:47.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.385+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~DEK2EAH2Q744.2 size=894646 2026-02-24T22:10:47.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.385+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=DEK2EAH2Q744 tag=A9C7HJ1MA3YB type=write size=4040374 2026-02-24T22:10:47.389 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.385+0000 7f7e64854980 1 < completed delete operation key=78GUTC5THI2P tag=POUPKBSP9B45 type=del 2026-02-24T22:10:47.390 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.389+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=E648D72AM8EB type=del size=139714 2026-02-24T22:10:47.390 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.389+0000 7f7e64854980 1 < completed delete operation key=Q7BP8FXTARDW tag=WSYBUWIE3Z0I type=del 2026-02-24T22:10:47.391 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.389+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=62EV94YWGHTO type=del size=3465652 2026-02-24T22:10:47.392 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.389+0000 7f7e64854980 1 < canceled operation key=NL60Q5QOSP8E tag=L3KZ0S1Z502G type=del size=1765326 2026-02-24T22:10:47.392 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.389+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=7J5U5TVN64YI type=del size=550067 2026-02-24T22:10:47.393 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.389+0000 7f7e64854980 1 < completed delete operation key=78GUTC5THI2P tag=ZZW1U0RFF3SZ type=del 2026-02-24T22:10:47.394 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.393+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=AJI8CRF0TZOI type=del size=2456788 2026-02-24T22:10:47.394 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.393+0000 7f7e64854980 1 < completed write operation key=GH7723GNI2WM tag=1KE35I4VTS1K type=write size=345333 2026-02-24T22:10:47.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.393+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~SKWG3EM7MZZ9.0 size=743595 2026-02-24T22:10:47.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.393+0000 7f7e64854980 1 > prepared multipart upload key=TF5B43G7KCYT upload=SKWG3EM7MZZ9 tag=NF4LCU1N788C type=write size=1792171 2026-02-24T22:10:47.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.393+0000 7f7e64854980 1 < completed write operation key=4Y3974PYYT78 tag=5GWF77KZZYFT type=write size=169535 2026-02-24T22:10:47.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.397+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~6UP1DEHWTROX.0 size=1048576 2026-02-24T22:10:47.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.397+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~6UP1DEHWTROX.1 size=1048576 2026-02-24T22:10:47.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.397+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~6UP1DEHWTROX.2 size=207670 2026-02-24T22:10:47.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.397+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=6UP1DEHWTROX tag=IYAKT0RRK6KE type=write size=3353398 2026-02-24T22:10:47.400 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.397+0000 7f7e64854980 1 < suggested update operation key=D0MKGNRDW6L9 tag=49K10UHNQBBV type=write size=518099 2026-02-24T22:10:47.401 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.397+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=5EG21BGITVQF type=del size=2847048 2026-02-24T22:10:47.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.401+0000 7f7e64854980 1 < suggested update operation key=L60HQ7M3E3LL tag=HIKG2174PXJQ type=del size=2652651 2026-02-24T22:10:47.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.401+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=EI2X7IHCY1TW type=write size=823523 2026-02-24T22:10:47.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.401+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~UKZR4ZBPE7QK.0 size=1048576 2026-02-24T22:10:47.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.401+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~UKZR4ZBPE7QK.1 size=1048576 2026-02-24T22:10:47.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.401+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~UKZR4ZBPE7QK.2 size=156796 2026-02-24T22:10:47.406 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.405+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=UKZR4ZBPE7QK tag=O9PDRPKWJ37R type=write size=3302524 2026-02-24T22:10:47.407 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.405+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~N5JV6941SXGD.0 size=814659 2026-02-24T22:10:47.407 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.405+0000 7f7e64854980 1 > prepared multipart upload key=Y3L3CJ51C1WR upload=N5JV6941SXGD tag=G41UWYTS4NCX type=write size=1863235 2026-02-24T22:10:47.408 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.405+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~HXQWWH7KUGIL.0 size=888458 2026-02-24T22:10:47.409 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.405+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=HXQWWH7KUGIL tag=ORN5N1866NUD type=write size=1937034 2026-02-24T22:10:47.409 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.409+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=30DFGGRN93V2 type=del size=1306134 2026-02-24T22:10:47.410 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.409+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~5OT0SJVGWAT1.0 size=1048576 2026-02-24T22:10:47.411 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.409+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~5OT0SJVGWAT1.1 size=908529 2026-02-24T22:10:47.412 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.409+0000 7f7e64854980 1 < completed multipart upload key=TXRUQQ456DD8 upload=5OT0SJVGWAT1 tag=3YIP7UU2UTOX type=write size=3005681 2026-02-24T22:10:47.412 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.409+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=YUH1QMVB306A type=del size=1617151 2026-02-24T22:10:47.413 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.409+0000 7f7e64854980 1 < completed delete operation key=Q7BP8FXTARDW tag=51DS2WQE2S5L type=del 2026-02-24T22:10:47.414 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.413+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~XLVNS41FEN82.0 size=1048576 2026-02-24T22:10:47.415 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.413+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~XLVNS41FEN82.1 size=19557 2026-02-24T22:10:47.415 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.413+0000 7f7e64854980 1 > prepared multipart upload key=Y3L3CJ51C1WR upload=XLVNS41FEN82 tag=MI48CP8SM3X3 type=write size=2116709 2026-02-24T22:10:47.415 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.413+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~ZOGH00G52VR7.0 size=1029918 2026-02-24T22:10:47.417 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.413+0000 7f7e64854980 1 < completed multipart upload key=EB04ETNYM087 upload=ZOGH00G52VR7 tag=EUHHOKPYG2QP type=write size=2078494 2026-02-24T22:10:47.417 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.417+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=5XGRMO6AJZT8 type=del size=323124 2026-02-24T22:10:47.418 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.417+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~DHVJBSG441PH.0 size=754326 2026-02-24T22:10:47.419 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.417+0000 7f7e64854980 1 < completed multipart upload key=NL60Q5QOSP8E upload=DHVJBSG441PH tag=DQTDGF6ZW1D3 type=write size=1802902 2026-02-24T22:10:47.420 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.417+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=NCN15G45YIXM type=del size=262339 2026-02-24T22:10:47.421 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.417+0000 7f7e64854980 1 < suggested update operation key=Q7BP8FXTARDW tag=B8SEMQ0FJBP0 type=write size=624475 2026-02-24T22:10:47.422 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.421+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=LOPC08KYX11L type=del size=3250224 2026-02-24T22:10:47.422 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.421+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~DEK2EAH2Q744.0 size=1048576 2026-02-24T22:10:47.423 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.421+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~DEK2EAH2Q744.1 size=1048576 2026-02-24T22:10:47.424 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.421+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~DEK2EAH2Q744.2 size=894646 2026-02-24T22:10:47.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.421+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=DEK2EAH2Q744 tag=A9C7HJ1MA3YB type=write size=4040374 2026-02-24T22:10:47.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.425+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=TXAVTEQ1ZOPJ type=del size=506126 2026-02-24T22:10:47.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.425+0000 7f7e64854980 1 < suggested update operation key=CI8DA17DX8LU tag=E648D72AM8EB type=del size=139714 2026-02-24T22:10:47.428 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.425+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=POYS6MI6K8WX type=del size=3487718 2026-02-24T22:10:47.428 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.425+0000 7f7e64854980 1 < completed delete operation key=0B2V0MHWSOC9 tag=62EV94YWGHTO type=del 2026-02-24T22:10:47.430 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.429+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~IMM8CB1FBIBN.0 size=950586 2026-02-24T22:10:47.430 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.429+0000 7f7e64854980 1 > prepared multipart upload key=L60HQ7M3E3LL upload=IMM8CB1FBIBN tag=74P8KH8NCM66 type=write size=1999162 2026-02-24T22:10:47.431 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.429+0000 7f7e64854980 1 < suggested update operation key=UQLUT9ZAI61V tag=7J5U5TVN64YI type=del size=550067 2026-02-24T22:10:47.432 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.429+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=LRDEMV1QU18I type=del size=3966540 2026-02-24T22:10:47.433 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.429+0000 7f7e64854980 1 < suggested remove operation key=L9NEDH44BIGA tag=AJI8CRF0TZOI type=del 2026-02-24T22:10:47.434 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.433+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~3BZT7B5J4V22.0 size=1048576 2026-02-24T22:10:47.435 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.433+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~3BZT7B5J4V22.1 size=887365 2026-02-24T22:10:47.435 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.433+0000 7f7e64854980 1 > prepared multipart upload key=N3ZUNSI5K8HS upload=3BZT7B5J4V22 tag=BN72MVKIDFRW type=write size=2984517 2026-02-24T22:10:47.436 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.433+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~SKWG3EM7MZZ9.0 size=743595 2026-02-24T22:10:47.437 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.433+0000 7f7e64854980 1 < completed multipart upload key=TF5B43G7KCYT upload=SKWG3EM7MZZ9 tag=NF4LCU1N788C type=write size=1792171 2026-02-24T22:10:47.438 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.437+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~EME4Z4SJNNVX.0 size=453071 2026-02-24T22:10:47.438 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.437+0000 7f7e64854980 1 > prepared multipart upload key=78GUTC5THI2P upload=EME4Z4SJNNVX tag=U26AVSF37RDT type=write size=1501647 2026-02-24T22:10:47.439 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.437+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~6UP1DEHWTROX.0 size=1048576 2026-02-24T22:10:47.439 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.437+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~6UP1DEHWTROX.1 size=1048576 2026-02-24T22:10:47.440 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.437+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~6UP1DEHWTROX.2 size=207670 2026-02-24T22:10:47.441 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.437+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=6UP1DEHWTROX tag=IYAKT0RRK6KE type=write size=3353398 2026-02-24T22:10:47.442 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.441+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=73E9RJ20ZNJ6 type=del size=1830668 2026-02-24T22:10:47.442 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.441+0000 7f7e64854980 1 < completed delete operation key=PKRHWM69GBJI tag=5EG21BGITVQF type=del 2026-02-24T22:10:47.443 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.441+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=CLCMYU6T6TS7 type=del size=1468957 2026-02-24T22:10:47.444 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.441+0000 7f7e64854980 1 < completed write operation key=HOHKR92SN25M tag=EI2X7IHCY1TW type=write size=823523 2026-02-24T22:10:47.445 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.441+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=6YBACJUM79LN type=del size=2153632 2026-02-24T22:10:47.445 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.445+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~N5JV6941SXGD.0 size=814659 2026-02-24T22:10:47.446 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.445+0000 7f7e64854980 1 < completed multipart upload key=Y3L3CJ51C1WR upload=N5JV6941SXGD tag=G41UWYTS4NCX type=write size=1863235 2026-02-24T22:10:47.448 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.445+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~4JJ55ATULAMC.0 size=113375 2026-02-24T22:10:47.448 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.445+0000 7f7e64854980 1 > prepared multipart upload key=N3ZUNSI5K8HS upload=4JJ55ATULAMC tag=597R306MWJ74 type=write size=1161951 2026-02-24T22:10:47.448 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.445+0000 7f7e64854980 1 < completed delete operation key=D0MKGNRDW6L9 tag=30DFGGRN93V2 type=del 2026-02-24T22:10:47.449 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.445+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=I5U430QNZBBZ type=del size=1686934 2026-02-24T22:10:47.450 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.449+0000 7f7e64854980 1 < completed delete operation key=A6PH77Q78VEN tag=YUH1QMVB306A type=del 2026-02-24T22:10:47.450 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.449+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=W033KR40PSGI type=del size=1747458 2026-02-24T22:10:47.452 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.449+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~XLVNS41FEN82.0 size=1048576 2026-02-24T22:10:47.453 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.449+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~XLVNS41FEN82.1 size=19557 2026-02-24T22:10:47.454 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.453+0000 7f7e64854980 1 < completed multipart upload key=Y3L3CJ51C1WR upload=XLVNS41FEN82 tag=MI48CP8SM3X3 type=write size=2116709 2026-02-24T22:10:47.455 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.453+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=EZ2Q9TIUN6LV type=del size=3225767 2026-02-24T22:10:47.455 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.453+0000 7f7e64854980 1 < completed delete operation key=D0MKGNRDW6L9 tag=5XGRMO6AJZT8 type=del 2026-02-24T22:10:47.456 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.453+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=0B1XK6SOTWS8 type=write size=474189 2026-02-24T22:10:47.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.453+0000 7f7e64854980 1 < completed delete operation key=Y3L3CJ51C1WR tag=NCN15G45YIXM type=del 2026-02-24T22:10:47.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.457+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=W0XM7VHRAUV2 type=del size=2910643 2026-02-24T22:10:47.458 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.457+0000 7f7e64854980 1 < completed delete operation key=LSRPN33Z2B9N tag=LOPC08KYX11L type=del 2026-02-24T22:10:47.459 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.457+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=MYZSP2HCF7WS type=del size=718332 2026-02-24T22:10:47.460 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.457+0000 7f7e64854980 1 < completed delete operation key=EJCX06TE5RTZ tag=TXAVTEQ1ZOPJ type=del 2026-02-24T22:10:47.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.457+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~C0R7Z0W9GH8L.0 size=1048576 2026-02-24T22:10:47.462 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.461+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~C0R7Z0W9GH8L.1 size=1048576 2026-02-24T22:10:47.462 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.461+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~C0R7Z0W9GH8L.2 size=73666 2026-02-24T22:10:47.462 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.461+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=C0R7Z0W9GH8L tag=7H8S7R76A1FF type=write size=3219394 2026-02-24T22:10:47.463 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.461+0000 7f7e64854980 1 < suggested remove operation key=GH7723GNI2WM tag=POYS6MI6K8WX type=del 2026-02-24T22:10:47.464 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.461+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=GWR9WLGA81E9 type=del size=1809937 2026-02-24T22:10:47.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.461+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~IMM8CB1FBIBN.0 size=950586 2026-02-24T22:10:47.466 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.465+0000 7f7e64854980 1 < completed multipart upload key=L60HQ7M3E3LL upload=IMM8CB1FBIBN tag=74P8KH8NCM66 type=write size=1999162 2026-02-24T22:10:47.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.465+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=F2RHVEM3JTTM type=write size=586096 2026-02-24T22:10:47.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.465+0000 7f7e64854980 1 < completed delete operation key=NL60Q5QOSP8E tag=LRDEMV1QU18I type=del 2026-02-24T22:10:47.469 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.465+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~6N3ZZUXWTUTY.0 size=1048576 2026-02-24T22:10:47.469 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.465+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~6N3ZZUXWTUTY.1 size=1048576 2026-02-24T22:10:47.470 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.469+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~6N3ZZUXWTUTY.2 size=886833 2026-02-24T22:10:47.470 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.469+0000 7f7e64854980 1 > prepared multipart upload key=ON16L53JUSTW upload=6N3ZZUXWTUTY tag=U5LY55O2S6J3 type=write size=4032561 2026-02-24T22:10:47.470 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.469+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~3BZT7B5J4V22.0 size=1048576 2026-02-24T22:10:47.471 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.469+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~3BZT7B5J4V22.1 size=887365 2026-02-24T22:10:47.472 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.469+0000 7f7e64854980 1 < completed multipart upload key=N3ZUNSI5K8HS upload=3BZT7B5J4V22 tag=BN72MVKIDFRW type=write size=2984517 2026-02-24T22:10:47.473 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.469+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=F5HS1L7CC6HT type=del size=1539971 2026-02-24T22:10:47.473 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.473+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~EME4Z4SJNNVX.0 size=453071 2026-02-24T22:10:47.475 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.473+0000 7f7e64854980 1 < completed multipart upload key=78GUTC5THI2P upload=EME4Z4SJNNVX tag=U26AVSF37RDT type=write size=1501647 2026-02-24T22:10:47.475 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.473+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=009PRNR02ABN type=write size=991490 2026-02-24T22:10:47.477 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.473+0000 7f7e64854980 1 < suggested remove operation key=UQLUT9ZAI61V tag=73E9RJ20ZNJ6 type=del 2026-02-24T22:10:47.478 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.477+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~XZMF6MY47JIB.0 size=299933 2026-02-24T22:10:47.478 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.477+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=XZMF6MY47JIB tag=XN5BECDH36MH type=write size=1348509 2026-02-24T22:10:47.479 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.477+0000 7f7e64854980 1 < canceled operation key=HKOQRSF6XEXB tag=CLCMYU6T6TS7 type=del size=1468957 2026-02-24T22:10:47.479 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.477+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=V1EKSF0E5C0Q type=del size=3417263 2026-02-24T22:10:47.480 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.477+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=6YBACJUM79LN type=del 2026-02-24T22:10:47.481 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.481+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~SP34EMHK4IIA.0 size=355170 2026-02-24T22:10:47.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.481+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=SP34EMHK4IIA tag=KLYU25JCU27U type=write size=1403746 2026-02-24T22:10:47.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.481+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~4JJ55ATULAMC.0 size=113375 2026-02-24T22:10:47.483 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.481+0000 7f7e64854980 1 < completed multipart upload key=N3ZUNSI5K8HS upload=4JJ55ATULAMC tag=597R306MWJ74 type=write size=1161951 2026-02-24T22:10:47.484 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.481+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=POSJ33DHGF2I type=del size=1328429 2026-02-24T22:10:47.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.481+0000 7f7e64854980 1 < completed delete operation key=78GUTC5THI2P tag=I5U430QNZBBZ type=del 2026-02-24T22:10:47.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.485+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=ZADDVSV4XP5W type=del size=3052343 2026-02-24T22:10:47.487 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.485+0000 7f7e64854980 1 < suggested update operation key=L60HQ7M3E3LL tag=W033KR40PSGI type=del size=1747458 2026-02-24T22:10:47.488 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.485+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~X5LFJIZ28SZD.0 size=1048576 2026-02-24T22:10:47.488 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.485+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~X5LFJIZ28SZD.1 size=184651 2026-02-24T22:10:47.488 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.485+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=X5LFJIZ28SZD tag=H90X9C2I5GQ1 type=write size=2281803 2026-02-24T22:10:47.489 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.485+0000 7f7e64854980 1 < completed delete operation key=R2893NES3WSF tag=EZ2Q9TIUN6LV type=del 2026-02-24T22:10:47.490 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.489+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=1N1FTII6AHUX type=del size=3693039 2026-02-24T22:10:47.491 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.489+0000 7f7e64854980 1 < completed write operation key=D0MKGNRDW6L9 tag=0B1XK6SOTWS8 type=write size=474189 2026-02-24T22:10:47.491 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.489+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=VLCXTTF6VD5S type=del size=2739099 2026-02-24T22:10:47.492 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.489+0000 7f7e64854980 1 < completed delete operation key=3900SFJK1YQU tag=W0XM7VHRAUV2 type=del 2026-02-24T22:10:47.493 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.489+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=WBRMLA1IMLO9 type=del size=2468768 2026-02-24T22:10:47.494 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.493+0000 7f7e64854980 1 < suggested remove operation key=R2893NES3WSF tag=MYZSP2HCF7WS type=del 2026-02-24T22:10:47.495 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.493+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=ZKBGT8XOY2A5 type=write size=500882 2026-02-24T22:10:47.496 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.493+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~C0R7Z0W9GH8L.0 size=1048576 2026-02-24T22:10:47.496 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.493+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~C0R7Z0W9GH8L.1 size=1048576 2026-02-24T22:10:47.497 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.493+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~C0R7Z0W9GH8L.2 size=73666 2026-02-24T22:10:47.498 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.497+0000 7f7e64854980 1 < completed multipart upload key=CI8DA17DX8LU upload=C0R7Z0W9GH8L tag=7H8S7R76A1FF type=write size=3219394 2026-02-24T22:10:47.499 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.497+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=5IDZDUZEBZB6 type=del size=146158 2026-02-24T22:10:47.499 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.497+0000 7f7e64854980 -1 < failed to complete operation key=R2893NES3WSF tag=GWR9WLGA81E9 type=del size=1809937: (22) Invalid argument 2026-02-24T22:10:47.500 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.497+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=KA9QFO7ZV93Y type=del size=2852133 2026-02-24T22:10:47.501 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.497+0000 7f7e64854980 1 < completed write operation key=78GUTC5THI2P tag=F2RHVEM3JTTM type=write size=586096 2026-02-24T22:10:47.501 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.501+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=6V2YT7RTJJ8D type=del size=1985791 2026-02-24T22:10:47.502 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.501+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~6N3ZZUXWTUTY.0 size=1048576 2026-02-24T22:10:47.503 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.501+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~6N3ZZUXWTUTY.1 size=1048576 2026-02-24T22:10:47.503 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.501+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~6N3ZZUXWTUTY.2 size=886833 2026-02-24T22:10:47.504 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.501+0000 7f7e64854980 1 < completed multipart upload key=ON16L53JUSTW upload=6N3ZZUXWTUTY tag=U5LY55O2S6J3 type=write size=4032561 2026-02-24T22:10:47.506 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.505+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~0DG6ORHHL76M.0 size=1048576 2026-02-24T22:10:47.506 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.505+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~0DG6ORHHL76M.1 size=1020735 2026-02-24T22:10:47.506 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.505+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=0DG6ORHHL76M tag=PRVMBJZ2GNH6 type=write size=3117887 2026-02-24T22:10:47.508 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.505+0000 7f7e64854980 1 < suggested remove operation key=Y8GMOUA977H3 tag=F5HS1L7CC6HT type=del 2026-02-24T22:10:47.508 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.505+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=SHFY3FM6WEXL type=del size=4132062 2026-02-24T22:10:47.510 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.509+0000 7f7e64854980 1 < suggested remove operation key=L9NEDH44BIGA tag=009PRNR02ABN type=write 2026-02-24T22:10:47.511 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.509+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~ZDPV0R7HQL16.0 size=350811 2026-02-24T22:10:47.511 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.509+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=ZDPV0R7HQL16 tag=1LUKAE3DRRU9 type=write size=1399387 2026-02-24T22:10:47.512 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.509+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~XZMF6MY47JIB.0 size=299933 2026-02-24T22:10:47.513 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.509+0000 7f7e64854980 1 < completed multipart upload key=TXRUQQ456DD8 upload=XZMF6MY47JIB tag=XN5BECDH36MH type=write size=1348509 2026-02-24T22:10:47.514 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.513+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~3MB5NZ9GFOFQ.0 size=1048576 2026-02-24T22:10:47.515 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.513+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~3MB5NZ9GFOFQ.1 size=197436 2026-02-24T22:10:47.515 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.513+0000 7f7e64854980 1 > prepared multipart upload key=PKRHWM69GBJI upload=3MB5NZ9GFOFQ tag=XA09WOX7AMSQ type=write size=2294588 2026-02-24T22:10:47.515 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.513+0000 7f7e64854980 1 < completed delete operation key=N3ZUNSI5K8HS tag=V1EKSF0E5C0Q type=del 2026-02-24T22:10:47.517 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.513+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~X0WZDQNW3D32.0 size=986883 2026-02-24T22:10:47.517 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.513+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=X0WZDQNW3D32 tag=F26T1NF9E8YB type=write size=2035459 2026-02-24T22:10:47.518 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.517+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~SP34EMHK4IIA.0 size=355170 2026-02-24T22:10:47.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.517+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=SP34EMHK4IIA tag=KLYU25JCU27U type=write size=1403746 2026-02-24T22:10:47.521 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.517+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=6H394UHH4KW7 type=del size=2229047 2026-02-24T22:10:47.521 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.521+0000 7f7e64854980 1 < completed delete operation key=LSRPN33Z2B9N tag=POSJ33DHGF2I type=del 2026-02-24T22:10:47.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.521+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=MT740AG6DXTK type=del size=1788715 2026-02-24T22:10:47.523 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.521+0000 7f7e64854980 1 < completed delete operation key=PKRHWM69GBJI tag=ZADDVSV4XP5W type=del 2026-02-24T22:10:47.524 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.521+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~0UVBQXS71I94.0 size=1048576 2026-02-24T22:10:47.525 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.521+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~0UVBQXS71I94.1 size=343968 2026-02-24T22:10:47.525 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.521+0000 7f7e64854980 1 > prepared multipart upload key=SGQYQ2TIVJPS upload=0UVBQXS71I94 tag=FIBU749I476V type=write size=2441120 2026-02-24T22:10:47.525 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.525+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~X5LFJIZ28SZD.0 size=1048576 2026-02-24T22:10:47.526 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.525+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~X5LFJIZ28SZD.1 size=184651 2026-02-24T22:10:47.527 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.525+0000 7f7e64854980 1 < completed multipart upload key=HOHKR92SN25M upload=X5LFJIZ28SZD tag=H90X9C2I5GQ1 type=write size=2281803 2026-02-24T22:10:47.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.525+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~6EPFKKY41SLL.0 size=1048576 2026-02-24T22:10:47.529 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.525+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~6EPFKKY41SLL.1 size=1048576 2026-02-24T22:10:47.529 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.529+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~6EPFKKY41SLL.2 size=1029264 2026-02-24T22:10:47.529 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.529+0000 7f7e64854980 1 > prepared multipart upload key=4Y3974PYYT78 upload=6EPFKKY41SLL tag=7AZG9WV7TSAG type=write size=4174992 2026-02-24T22:10:47.530 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.529+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=1N1FTII6AHUX type=del 2026-02-24T22:10:47.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.529+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=ZHUFJB9TNRJ3 type=del size=3349132 2026-02-24T22:10:47.532 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.529+0000 7f7e64854980 1 < canceled operation key=EB04ETNYM087 tag=VLCXTTF6VD5S type=del size=2739099 2026-02-24T22:10:47.532 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.529+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=GZDN9DNUUOUM type=del size=629712 2026-02-24T22:10:47.533 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.529+0000 7f7e64854980 -1 < failed to complete operation key=L9NEDH44BIGA tag=WBRMLA1IMLO9 type=del size=2468768: (22) Invalid argument 2026-02-24T22:10:47.534 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.533+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=ZPLBP4WLKARW type=del size=3228099 2026-02-24T22:10:47.534 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.533+0000 7f7e64854980 1 < canceled operation key=Q7BP8FXTARDW tag=ZKBGT8XOY2A5 type=write size=500882 2026-02-24T22:10:47.536 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.533+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~09Y5NU57ZICK.0 size=438544 2026-02-24T22:10:47.536 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.533+0000 7f7e64854980 1 > prepared multipart upload key=A6PH77Q78VEN upload=09Y5NU57ZICK tag=YMY86WBN8EM4 type=write size=1487120 2026-02-24T22:10:47.536 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.533+0000 7f7e64854980 1 < completed delete operation key=4Y3974PYYT78 tag=5IDZDUZEBZB6 type=del 2026-02-24T22:10:47.537 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.533+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=H1H66CLTBTR1 type=del size=1599661 2026-02-24T22:10:47.538 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.537+0000 7f7e64854980 1 < completed delete operation key=SGQYQ2TIVJPS tag=KA9QFO7ZV93Y type=del 2026-02-24T22:10:47.539 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.537+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=ZN55FNJ5072G type=del size=1210379 2026-02-24T22:10:47.539 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.537+0000 7f7e64854980 1 < completed delete operation key=PKRHWM69GBJI tag=6V2YT7RTJJ8D type=del 2026-02-24T22:10:47.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.537+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~RRGR45OGU6WN.0 size=1048576 2026-02-24T22:10:47.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.537+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~RRGR45OGU6WN.1 size=137379 2026-02-24T22:10:47.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.537+0000 7f7e64854980 1 > prepared multipart upload key=NL60Q5QOSP8E upload=RRGR45OGU6WN tag=PD3R13SEPNBK type=write size=2234531 2026-02-24T22:10:47.542 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.541+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~0DG6ORHHL76M.0 size=1048576 2026-02-24T22:10:47.542 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.541+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~0DG6ORHHL76M.1 size=1020735 2026-02-24T22:10:47.544 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.541+0000 7f7e64854980 1 < completed multipart upload key=TXRUQQ456DD8 upload=0DG6ORHHL76M tag=PRVMBJZ2GNH6 type=write size=3117887 2026-02-24T22:10:47.544 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.541+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=P17UQSNUEXFO type=del size=2851920 2026-02-24T22:10:47.545 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.541+0000 7f7e64854980 1 < completed delete operation key=LSRPN33Z2B9N tag=SHFY3FM6WEXL type=del 2026-02-24T22:10:47.546 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.545+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=IRAXDHXHFC94 type=del size=2747451 2026-02-24T22:10:47.546 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.545+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~ZDPV0R7HQL16.0 size=350811 2026-02-24T22:10:47.548 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.545+0000 7f7e64854980 1 < completed multipart upload key=ZESKGBWZXM6O upload=ZDPV0R7HQL16 tag=1LUKAE3DRRU9 type=write size=1399387 2026-02-24T22:10:47.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.545+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~14D2LG0WJIS3.0 size=364429 2026-02-24T22:10:47.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.545+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=14D2LG0WJIS3 tag=YLO50P59K0GL type=write size=1413005 2026-02-24T22:10:47.550 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.549+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~3MB5NZ9GFOFQ.0 size=1048576 2026-02-24T22:10:47.550 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.549+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~3MB5NZ9GFOFQ.1 size=197436 2026-02-24T22:10:47.551 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.549+0000 7f7e64854980 1 < completed multipart upload key=PKRHWM69GBJI upload=3MB5NZ9GFOFQ tag=XA09WOX7AMSQ type=write size=2294588 2026-02-24T22:10:47.552 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.549+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=WDC369JTEUQF type=del size=1307388 2026-02-24T22:10:47.553 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.549+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~X0WZDQNW3D32.0 size=986883 2026-02-24T22:10:47.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.553+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=X0WZDQNW3D32 tag=F26T1NF9E8YB type=write size=2035459 2026-02-24T22:10:47.555 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.553+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=VJB2XOXT4O29 type=del size=3180912 2026-02-24T22:10:47.555 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.553+0000 7f7e64854980 1 < completed delete operation key=0B2V0MHWSOC9 tag=6H394UHH4KW7 type=del 2026-02-24T22:10:47.556 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.553+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=RF4IUMP8J6YN type=del size=2572021 2026-02-24T22:10:47.558 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.557+0000 7f7e64854980 1 < suggested remove operation key=Q7BP8FXTARDW tag=MT740AG6DXTK type=del 2026-02-24T22:10:47.558 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.557+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=4RZQVNALSCEN type=del size=397354 2026-02-24T22:10:47.559 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.557+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~0UVBQXS71I94.0 size=1048576 2026-02-24T22:10:47.559 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.557+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~0UVBQXS71I94.1 size=343968 2026-02-24T22:10:47.561 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.557+0000 7f7e64854980 1 < completed multipart upload key=SGQYQ2TIVJPS upload=0UVBQXS71I94 tag=FIBU749I476V type=write size=2441120 2026-02-24T22:10:47.561 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.561+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=5VDTM9Q7WBO0 type=del size=3055475 2026-02-24T22:10:47.562 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.561+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~6EPFKKY41SLL.0 size=1048576 2026-02-24T22:10:47.563 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.561+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~6EPFKKY41SLL.1 size=1048576 2026-02-24T22:10:47.563 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.561+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~6EPFKKY41SLL.2 size=1029264 2026-02-24T22:10:47.565 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.561+0000 7f7e64854980 1 < completed multipart upload key=4Y3974PYYT78 upload=6EPFKKY41SLL tag=7AZG9WV7TSAG type=write size=4174992 2026-02-24T22:10:47.565 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.565+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=9CH33YJ4VB21 type=del size=1579247 2026-02-24T22:10:47.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.565+0000 7f7e64854980 1 < completed delete operation key=N3ZUNSI5K8HS tag=ZHUFJB9TNRJ3 type=del 2026-02-24T22:10:47.568 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.565+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~TY77K5VP8LM5.0 size=1048576 2026-02-24T22:10:47.569 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.565+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~TY77K5VP8LM5.1 size=277992 2026-02-24T22:10:47.569 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.565+0000 7f7e64854980 1 > prepared multipart upload key=EB04ETNYM087 upload=TY77K5VP8LM5 tag=WUTYY6RDKEXH type=write size=2375144 2026-02-24T22:10:47.570 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.569+0000 7f7e64854980 1 < completed delete operation key=HKOQRSF6XEXB tag=GZDN9DNUUOUM type=del 2026-02-24T22:10:47.571 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.569+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~6O5GQBDKN2Y6.0 size=1048576 2026-02-24T22:10:47.572 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.569+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~6O5GQBDKN2Y6.1 size=577709 2026-02-24T22:10:47.572 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.569+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=6O5GQBDKN2Y6 tag=4CD320VCBW3K type=write size=2674861 2026-02-24T22:10:47.573 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.569+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=ZPLBP4WLKARW type=del 2026-02-24T22:10:47.573 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.573+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=5WM6A51CB396 type=del size=1763487 2026-02-24T22:10:47.574 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.573+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~09Y5NU57ZICK.0 size=438544 2026-02-24T22:10:47.575 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.573+0000 7f7e64854980 1 < completed multipart upload key=A6PH77Q78VEN upload=09Y5NU57ZICK tag=YMY86WBN8EM4 type=write size=1487120 2026-02-24T22:10:47.576 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.573+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=GAKR7KPY3XBB type=del size=1770611 2026-02-24T22:10:47.578 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.577+0000 7f7e64854980 1 < suggested update operation key=L60HQ7M3E3LL tag=H1H66CLTBTR1 type=del size=1599661 2026-02-24T22:10:47.578 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.577+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=OFG203Z1Q478 type=del size=3127487 2026-02-24T22:10:47.579 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.577+0000 7f7e64854980 1 < completed delete operation key=4Y3974PYYT78 tag=ZN55FNJ5072G type=del 2026-02-24T22:10:47.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.577+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=CJ10YPGJC469 type=del size=2566956 2026-02-24T22:10:47.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.577+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~RRGR45OGU6WN.0 size=1048576 2026-02-24T22:10:47.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.581+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~RRGR45OGU6WN.1 size=137379 2026-02-24T22:10:47.583 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.581+0000 7f7e64854980 1 < completed multipart upload key=NL60Q5QOSP8E upload=RRGR45OGU6WN tag=PD3R13SEPNBK type=write size=2234531 2026-02-24T22:10:47.583 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.581+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=38F5IXDJYLGR type=del size=2546372 2026-02-24T22:10:47.584 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.581+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=P17UQSNUEXFO type=del 2026-02-24T22:10:47.585 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.581+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=0QLZ0TNNT1XW type=del size=1760889 2026-02-24T22:10:47.586 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.585+0000 7f7e64854980 1 < suggested remove operation key=ON16L53JUSTW tag=IRAXDHXHFC94 type=del 2026-02-24T22:10:47.589 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.585+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~XQXB2SPO83DI.0 size=1048576 2026-02-24T22:10:47.590 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.589+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~XQXB2SPO83DI.1 size=345672 2026-02-24T22:10:47.590 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.589+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=XQXB2SPO83DI tag=V19H0P8GA8D3 type=write size=2442824 2026-02-24T22:10:47.590 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.589+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~14D2LG0WJIS3.0 size=364429 2026-02-24T22:10:47.592 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.589+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=14D2LG0WJIS3 tag=YLO50P59K0GL type=write size=1413005 2026-02-24T22:10:47.592 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.589+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=JB3QUOH324LR type=del size=1242252 2026-02-24T22:10:47.594 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.593+0000 7f7e64854980 1 < suggested update operation key=HKOQRSF6XEXB tag=WDC369JTEUQF type=del size=1307388 2026-02-24T22:10:47.596 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.593+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~GKKE93T6H5BA.0 size=1048576 2026-02-24T22:10:47.596 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.593+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~GKKE93T6H5BA.1 size=1048576 2026-02-24T22:10:47.597 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.593+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~GKKE93T6H5BA.2 size=764422 2026-02-24T22:10:47.597 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.593+0000 7f7e64854980 1 > prepared multipart upload key=0B2V0MHWSOC9 upload=GKKE93T6H5BA tag=0F375MAFL364 type=write size=3910150 2026-02-24T22:10:47.597 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.597+0000 7f7e64854980 1 < completed delete operation key=SGQYQ2TIVJPS tag=VJB2XOXT4O29 type=del 2026-02-24T22:10:47.598 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.597+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=DV1E58J74JK6 type=del size=345872 2026-02-24T22:10:47.600 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.597+0000 7f7e64854980 1 < suggested remove operation key=0AXS7DY23UZU tag=RF4IUMP8J6YN type=del 2026-02-24T22:10:47.601 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.597+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~MZE8PT0I6GN3.0 size=313968 2026-02-24T22:10:47.602 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.597+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=MZE8PT0I6GN3 tag=XPVLONA9W9LY type=write size=1362544 2026-02-24T22:10:47.602 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.601+0000 7f7e64854980 -1 < no bi entry to suggest for operation key=ON16L53JUSTW tag=4RZQVNALSCEN type=del size=397354: (2) No such file or directory 2026-02-24T22:10:47.602 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.601+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=VX0VU3JJ0W3L type=del size=2694093 2026-02-24T22:10:47.603 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.601+0000 7f7e64854980 1 < completed delete operation key=0B2V0MHWSOC9 tag=5VDTM9Q7WBO0 type=del 2026-02-24T22:10:47.604 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.601+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=0NSSIRNC7IFU type=write size=620546 2026-02-24T22:10:47.604 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.601+0000 7f7e64854980 1 < completed delete operation key=SGQYQ2TIVJPS tag=9CH33YJ4VB21 type=del 2026-02-24T22:10:47.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.601+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=FQ6D8PZRAR0X type=del size=1399382 2026-02-24T22:10:47.606 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.605+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~TY77K5VP8LM5.0 size=1048576 2026-02-24T22:10:47.606 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.605+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~TY77K5VP8LM5.1 size=277992 2026-02-24T22:10:47.608 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.605+0000 7f7e64854980 1 < completed multipart upload key=EB04ETNYM087 upload=TY77K5VP8LM5 tag=WUTYY6RDKEXH type=write size=2375144 2026-02-24T22:10:47.609 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.605+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=B2I6TCM1W4CF type=del size=2819763 2026-02-24T22:10:47.609 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.605+0000 7f7e64854980 1 < canceled part key=_multipart_L9NEDH44BIGA.2~6O5GQBDKN2Y6.0 size=1048576 2026-02-24T22:10:47.609 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.605+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~6O5GQBDKN2Y6.1 size=577709 2026-02-24T22:10:47.610 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.609+0000 7f7e64854980 1 < canceled multipart upload key=L9NEDH44BIGA upload=6O5GQBDKN2Y6 tag=4CD320VCBW3K type=write size=2674861 2026-02-24T22:10:47.612 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.609+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~UUJK2JE59AZK.0 size=1048576 2026-02-24T22:10:47.612 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.609+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~UUJK2JE59AZK.1 size=1048576 2026-02-24T22:10:47.613 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.609+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~UUJK2JE59AZK.2 size=545750 2026-02-24T22:10:47.613 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.609+0000 7f7e64854980 1 > prepared multipart upload key=3900SFJK1YQU upload=UUJK2JE59AZK tag=2V7KL7CYZ3PN type=write size=3691478 2026-02-24T22:10:47.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.613+0000 7f7e64854980 1 < suggested remove operation key=Y3L3CJ51C1WR tag=5WM6A51CB396 type=del 2026-02-24T22:10:47.615 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.613+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=PO2FYC36SSQB type=del size=3147947 2026-02-24T22:10:47.616 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.613+0000 7f7e64854980 1 < completed delete operation key=2KKQKHXO7K3K tag=GAKR7KPY3XBB type=del 2026-02-24T22:10:47.617 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.617+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~ZA0YOWDUF3L9.0 size=1048576 2026-02-24T22:10:47.618 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.617+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~ZA0YOWDUF3L9.1 size=443771 2026-02-24T22:10:47.618 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.617+0000 7f7e64854980 1 > prepared multipart upload key=N3ZUNSI5K8HS upload=ZA0YOWDUF3L9 tag=82ZYK1W0LIPW type=write size=2540923 2026-02-24T22:10:47.619 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.617+0000 7f7e64854980 1 < completed delete operation key=Q7BP8FXTARDW tag=OFG203Z1Q478 type=del 2026-02-24T22:10:47.620 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.617+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=T0HGCSOEX4PG type=del size=182812 2026-02-24T22:10:47.621 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.617+0000 7f7e64854980 1 < suggested update operation key=78GUTC5THI2P tag=CJ10YPGJC469 type=del size=2566956 2026-02-24T22:10:47.622 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.621+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~TB9ZLOP5KZLD.0 size=1048576 2026-02-24T22:10:47.623 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.621+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~TB9ZLOP5KZLD.1 size=1048576 2026-02-24T22:10:47.624 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.621+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~TB9ZLOP5KZLD.2 size=409090 2026-02-24T22:10:47.624 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.621+0000 7f7e64854980 1 > prepared multipart upload key=A6PH77Q78VEN upload=TB9ZLOP5KZLD tag=EHHMRANEDIL4 type=write size=3554818 2026-02-24T22:10:47.624 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.621+0000 7f7e64854980 1 < completed delete operation key=L60HQ7M3E3LL tag=38F5IXDJYLGR type=del 2026-02-24T22:10:47.626 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.625+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~22ZX6QAGYVIS.0 size=481263 2026-02-24T22:10:47.626 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.625+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=22ZX6QAGYVIS tag=BU1MI5QHELSN type=write size=1529839 2026-02-24T22:10:47.627 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.625+0000 7f7e64854980 1 < suggested remove operation key=ZESKGBWZXM6O tag=0QLZ0TNNT1XW type=del 2026-02-24T22:10:47.628 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.625+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~D4HOAGMA61LT.0 size=428038 2026-02-24T22:10:47.629 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.625+0000 7f7e64854980 1 > prepared multipart upload key=HKOQRSF6XEXB upload=D4HOAGMA61LT tag=VT6L5VW5XOHW type=write size=1476614 2026-02-24T22:10:47.629 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.629+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~XQXB2SPO83DI.0 size=1048576 2026-02-24T22:10:47.630 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.629+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~XQXB2SPO83DI.1 size=345672 2026-02-24T22:10:47.631 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.629+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=XQXB2SPO83DI tag=V19H0P8GA8D3 type=write size=2442824 2026-02-24T22:10:47.633 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.629+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~5Q6KMM44S6KV.0 size=1048576 2026-02-24T22:10:47.633 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.629+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~5Q6KMM44S6KV.1 size=1048576 2026-02-24T22:10:47.634 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.633+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~5Q6KMM44S6KV.2 size=414604 2026-02-24T22:10:47.634 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.633+0000 7f7e64854980 1 > prepared multipart upload key=PKRHWM69GBJI upload=5Q6KMM44S6KV tag=1KYK22F9YL35 type=write size=3560332 2026-02-24T22:10:47.634 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.633+0000 7f7e64854980 1 < completed delete operation key=0B2V0MHWSOC9 tag=JB3QUOH324LR type=del 2026-02-24T22:10:47.636 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.633+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~MCREBWKLFF98.0 size=1048576 2026-02-24T22:10:47.636 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.633+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~MCREBWKLFF98.1 size=551269 2026-02-24T22:10:47.637 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.633+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=MCREBWKLFF98 tag=2L6VV7E4IOUP type=write size=2648421 2026-02-24T22:10:47.637 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.633+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~GKKE93T6H5BA.0 size=1048576 2026-02-24T22:10:47.638 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.637+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~GKKE93T6H5BA.1 size=1048576 2026-02-24T22:10:47.638 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.637+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~GKKE93T6H5BA.2 size=764422 2026-02-24T22:10:47.640 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.637+0000 7f7e64854980 1 < completed multipart upload key=0B2V0MHWSOC9 upload=GKKE93T6H5BA tag=0F375MAFL364 type=write size=3910150 2026-02-24T22:10:47.641 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.637+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~K1XO85KCQ8Z1.0 size=1048576 2026-02-24T22:10:47.642 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.641+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~K1XO85KCQ8Z1.1 size=1048576 2026-02-24T22:10:47.642 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.641+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~K1XO85KCQ8Z1.2 size=135832 2026-02-24T22:10:47.642 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.641+0000 7f7e64854980 1 > prepared multipart upload key=36MKR2OZN7K9 upload=K1XO85KCQ8Z1 tag=8XQ502XG6RJ5 type=write size=3281560 2026-02-24T22:10:47.643 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.641+0000 7f7e64854980 1 < completed delete operation key=EB04ETNYM087 tag=DV1E58J74JK6 type=del 2026-02-24T22:10:47.644 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.641+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~NEZRWAD5WKFM.0 size=367214 2026-02-24T22:10:47.644 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.641+0000 7f7e64854980 1 > prepared multipart upload key=Y8GMOUA977H3 upload=NEZRWAD5WKFM tag=2UZQW26B6873 type=write size=1415790 2026-02-24T22:10:47.645 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.641+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~MZE8PT0I6GN3.0 size=313968 2026-02-24T22:10:47.646 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.645+0000 7f7e64854980 1 < completed multipart upload key=CI8DA17DX8LU upload=MZE8PT0I6GN3 tag=XPVLONA9W9LY type=write size=1362544 2026-02-24T22:10:47.647 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.645+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=VPMHY5ZKO6TB type=del size=3718216 2026-02-24T22:10:47.649 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.645+0000 7f7e64854980 1 < suggested update operation key=78GUTC5THI2P tag=VX0VU3JJ0W3L type=del size=2694093 2026-02-24T22:10:47.650 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.649+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~9YS51JBI5OF3.0 size=1048576 2026-02-24T22:10:47.650 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.649+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~9YS51JBI5OF3.1 size=1048576 2026-02-24T22:10:47.651 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.649+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~9YS51JBI5OF3.2 size=104398 2026-02-24T22:10:47.651 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.649+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=9YS51JBI5OF3 tag=V0WOQNVHZTE5 type=write size=3250126 2026-02-24T22:10:47.652 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.649+0000 7f7e64854980 1 < suggested update operation key=3900SFJK1YQU tag=0NSSIRNC7IFU type=write size=620546 2026-02-24T22:10:47.653 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.649+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=FUW9KIZ0FVY3 type=del size=2581510 2026-02-24T22:10:47.655 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.653+0000 7f7e64854980 1 < suggested update operation key=NL60Q5QOSP8E tag=FQ6D8PZRAR0X type=del size=1399382 2026-02-24T22:10:47.656 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.653+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~GN9EIAN7K8J8.0 size=553844 2026-02-24T22:10:47.656 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.653+0000 7f7e64854980 1 > prepared multipart upload key=A6PH77Q78VEN upload=GN9EIAN7K8J8 tag=WL3HZIPWTYWP type=write size=1602420 2026-02-24T22:10:47.657 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.653+0000 7f7e64854980 1 < completed delete operation key=LSRPN33Z2B9N tag=B2I6TCM1W4CF type=del 2026-02-24T22:10:47.659 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.657+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~ZWE73FO1OTOG.0 size=833653 2026-02-24T22:10:47.659 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.657+0000 7f7e64854980 1 > prepared multipart upload key=NL60Q5QOSP8E upload=ZWE73FO1OTOG tag=5F5J3BZ4KKR3 type=write size=1882229 2026-02-24T22:10:47.660 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.657+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~UUJK2JE59AZK.0 size=1048576 2026-02-24T22:10:47.661 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.657+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~UUJK2JE59AZK.1 size=1048576 2026-02-24T22:10:47.661 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.657+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~UUJK2JE59AZK.2 size=545750 2026-02-24T22:10:47.662 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.661+0000 7f7e64854980 1 < completed multipart upload key=3900SFJK1YQU upload=UUJK2JE59AZK tag=2V7KL7CYZ3PN type=write size=3691478 2026-02-24T22:10:47.663 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.661+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=MK4GNIKIQ9M2 type=del size=1600979 2026-02-24T22:10:47.664 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.661+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=PO2FYC36SSQB type=del 2026-02-24T22:10:47.664 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.661+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=HEGH558W7CQJ type=del size=4115640 2026-02-24T22:10:47.665 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.661+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~ZA0YOWDUF3L9.0 size=1048576 2026-02-24T22:10:47.666 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.665+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~ZA0YOWDUF3L9.1 size=443771 2026-02-24T22:10:47.667 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.665+0000 7f7e64854980 1 < completed multipart upload key=N3ZUNSI5K8HS upload=ZA0YOWDUF3L9 tag=82ZYK1W0LIPW type=write size=2540923 2026-02-24T22:10:47.667 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.665+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=J3DZ1LFVPSWL type=del size=1640269 2026-02-24T22:10:47.668 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.665+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=T0HGCSOEX4PG type=del 2026-02-24T22:10:47.669 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.665+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~NT753RK2KCKF.0 size=1048576 2026-02-24T22:10:47.670 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.669+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~NT753RK2KCKF.1 size=1005619 2026-02-24T22:10:47.670 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.669+0000 7f7e64854980 1 > prepared multipart upload key=A6PH77Q78VEN upload=NT753RK2KCKF tag=XJP3BZZKXIFZ type=write size=3102771 2026-02-24T22:10:47.670 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.669+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~TB9ZLOP5KZLD.0 size=1048576 2026-02-24T22:10:47.671 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.669+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~TB9ZLOP5KZLD.1 size=1048576 2026-02-24T22:10:47.672 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.669+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~TB9ZLOP5KZLD.2 size=409090 2026-02-24T22:10:47.673 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.669+0000 7f7e64854980 1 < completed multipart upload key=A6PH77Q78VEN upload=TB9ZLOP5KZLD tag=EHHMRANEDIL4 type=write size=3554818 2026-02-24T22:10:47.674 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.673+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~04U4R8VZP5HR.0 size=221118 2026-02-24T22:10:47.674 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.673+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=04U4R8VZP5HR tag=3JULH1LOQD3H type=write size=1269694 2026-02-24T22:10:47.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.673+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~22ZX6QAGYVIS.0 size=481263 2026-02-24T22:10:47.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.673+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=22ZX6QAGYVIS tag=BU1MI5QHELSN type=write size=1529839 2026-02-24T22:10:47.677 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.673+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=LKGCYZTJVJBJ type=write size=568829 2026-02-24T22:10:47.677 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.677+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~D4HOAGMA61LT.0 size=428038 2026-02-24T22:10:47.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.677+0000 7f7e64854980 1 < completed multipart upload key=HKOQRSF6XEXB upload=D4HOAGMA61LT tag=VT6L5VW5XOHW type=write size=1476614 2026-02-24T22:10:47.680 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.677+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~Z273RWBRF75Q.0 size=411925 2026-02-24T22:10:47.680 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.677+0000 7f7e64854980 1 > prepared multipart upload key=PKRHWM69GBJI upload=Z273RWBRF75Q tag=T6V8T22K2UH0 type=write size=1460501 2026-02-24T22:10:47.680 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.677+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~5Q6KMM44S6KV.0 size=1048576 2026-02-24T22:10:47.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.677+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~5Q6KMM44S6KV.1 size=1048576 2026-02-24T22:10:47.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.681+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~5Q6KMM44S6KV.2 size=414604 2026-02-24T22:10:47.683 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.681+0000 7f7e64854980 1 < completed multipart upload key=PKRHWM69GBJI upload=5Q6KMM44S6KV tag=1KYK22F9YL35 type=write size=3560332 2026-02-24T22:10:47.683 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.681+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=W0ASFOV9LIJU type=del size=1271312 2026-02-24T22:10:47.684 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.681+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~MCREBWKLFF98.0 size=1048576 2026-02-24T22:10:47.685 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.681+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~MCREBWKLFF98.1 size=551269 2026-02-24T22:10:47.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.685+0000 7f7e64854980 1 < completed multipart upload key=HOHKR92SN25M upload=MCREBWKLFF98 tag=2L6VV7E4IOUP type=write size=2648421 2026-02-24T22:10:47.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.685+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=Y2O96JEP8SJ5 type=del size=131222 2026-02-24T22:10:47.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.685+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~K1XO85KCQ8Z1.0 size=1048576 2026-02-24T22:10:47.688 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.685+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~K1XO85KCQ8Z1.1 size=1048576 2026-02-24T22:10:47.688 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.685+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~K1XO85KCQ8Z1.2 size=135832 2026-02-24T22:10:47.690 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.689+0000 7f7e64854980 1 < completed multipart upload key=36MKR2OZN7K9 upload=K1XO85KCQ8Z1 tag=8XQ502XG6RJ5 type=write size=3281560 2026-02-24T22:10:47.691 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.689+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~ML47G0S8XLPG.0 size=1048576 2026-02-24T22:10:47.691 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.689+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~ML47G0S8XLPG.1 size=1048576 2026-02-24T22:10:47.692 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.689+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~ML47G0S8XLPG.2 size=921462 2026-02-24T22:10:47.692 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.689+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=ML47G0S8XLPG tag=LLALK00Y26M0 type=write size=4067190 2026-02-24T22:10:47.692 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.689+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~NEZRWAD5WKFM.0 size=367214 2026-02-24T22:10:47.694 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.693+0000 7f7e64854980 1 < completed multipart upload key=Y8GMOUA977H3 upload=NEZRWAD5WKFM tag=2UZQW26B6873 type=write size=1415790 2026-02-24T22:10:47.694 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.693+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=3KEMFK8Z9VAM type=del size=3581739 2026-02-24T22:10:47.695 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.693+0000 7f7e64854980 1 < completed delete operation key=2KKQKHXO7K3K tag=VPMHY5ZKO6TB type=del 2026-02-24T22:10:47.696 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.693+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=VYZ5KJ45390O type=del size=2819640 2026-02-24T22:10:47.696 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.693+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~9YS51JBI5OF3.0 size=1048576 2026-02-24T22:10:47.697 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.693+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~9YS51JBI5OF3.1 size=1048576 2026-02-24T22:10:47.697 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.697+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~9YS51JBI5OF3.2 size=104398 2026-02-24T22:10:47.699 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.697+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=9YS51JBI5OF3 tag=V0WOQNVHZTE5 type=write size=3250126 2026-02-24T22:10:47.700 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.697+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~1MR90UVY8TW6.0 size=830960 2026-02-24T22:10:47.700 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.697+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=1MR90UVY8TW6 tag=O7HPS9X92YC8 type=write size=1879536 2026-02-24T22:10:47.701 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.697+0000 7f7e64854980 1 < completed delete operation key=Q7BP8FXTARDW tag=FUW9KIZ0FVY3 type=del 2026-02-24T22:10:47.701 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.697+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=K7PW4FWYHOK2 type=del size=3565529 2026-02-24T22:10:47.702 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.701+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~GN9EIAN7K8J8.0 size=553844 2026-02-24T22:10:47.703 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.701+0000 7f7e64854980 1 < completed multipart upload key=A6PH77Q78VEN upload=GN9EIAN7K8J8 tag=WL3HZIPWTYWP type=write size=1602420 2026-02-24T22:10:47.704 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.701+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~TJ32MM6XQODT.0 size=101552 2026-02-24T22:10:47.704 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.701+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=TJ32MM6XQODT tag=THF4VK45HPP9 type=write size=1150128 2026-02-24T22:10:47.705 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.701+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~ZWE73FO1OTOG.0 size=833653 2026-02-24T22:10:47.706 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.705+0000 7f7e64854980 1 < completed multipart upload key=NL60Q5QOSP8E upload=ZWE73FO1OTOG tag=5F5J3BZ4KKR3 type=write size=1882229 2026-02-24T22:10:47.707 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.705+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=89ACVNSC2K68 type=write size=52453 2026-02-24T22:10:47.708 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.705+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=MK4GNIKIQ9M2 type=del 2026-02-24T22:10:47.708 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.705+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=71F5Z2RTRBHG type=del size=1131286 2026-02-24T22:10:47.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.705+0000 7f7e64854980 1 < completed delete operation key=GH7723GNI2WM tag=HEGH558W7CQJ type=del 2026-02-24T22:10:47.710 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.709+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~UI1TKUMCMOTN.0 size=1048576 2026-02-24T22:10:47.711 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.709+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~UI1TKUMCMOTN.1 size=1048576 2026-02-24T22:10:47.711 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.709+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~UI1TKUMCMOTN.2 size=335790 2026-02-24T22:10:47.712 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.709+0000 7f7e64854980 1 > prepared multipart upload key=SGQYQ2TIVJPS upload=UI1TKUMCMOTN tag=QBHGTHGKYPZS type=write size=3481518 2026-02-24T22:10:47.712 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.709+0000 7f7e64854980 1 < completed delete operation key=3900SFJK1YQU tag=J3DZ1LFVPSWL type=del 2026-02-24T22:10:47.713 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.709+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=A3MCSKB8RRTT type=write size=696619 2026-02-24T22:10:47.714 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.713+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~NT753RK2KCKF.0 size=1048576 2026-02-24T22:10:47.714 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.713+0000 7f7e64854980 1 < canceled part key=_multipart_A6PH77Q78VEN.2~NT753RK2KCKF.1 size=1005619 2026-02-24T22:10:47.715 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.713+0000 7f7e64854980 1 < canceled multipart upload key=A6PH77Q78VEN upload=NT753RK2KCKF tag=XJP3BZZKXIFZ type=write size=3102771 2026-02-24T22:10:47.716 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.713+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~84PWTQ751BJF.0 size=1048576 2026-02-24T22:10:47.716 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.713+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~84PWTQ751BJF.1 size=1048576 2026-02-24T22:10:47.717 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.713+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~84PWTQ751BJF.2 size=739056 2026-02-24T22:10:47.717 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.713+0000 7f7e64854980 1 > prepared multipart upload key=L60HQ7M3E3LL upload=84PWTQ751BJF tag=O7UGK50HNIHF type=write size=3884784 2026-02-24T22:10:47.718 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.717+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~04U4R8VZP5HR.0 size=221118 2026-02-24T22:10:47.719 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.717+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=04U4R8VZP5HR tag=3JULH1LOQD3H type=write size=1269694 2026-02-24T22:10:47.720 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.717+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=2BEBDR0DQGUM type=write size=875095 2026-02-24T22:10:47.720 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.717+0000 7f7e64854980 1 < completed write operation key=CI8DA17DX8LU tag=LKGCYZTJVJBJ type=write size=568829 2026-02-24T22:10:47.722 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.721+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=L4HB1TJD4XL2 type=del size=2335408 2026-02-24T22:10:47.723 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.721+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~Z273RWBRF75Q.0 size=411925 2026-02-24T22:10:47.724 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.721+0000 7f7e64854980 1 < completed multipart upload key=PKRHWM69GBJI upload=Z273RWBRF75Q tag=T6V8T22K2UH0 type=write size=1460501 2026-02-24T22:10:47.725 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.721+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=GG0RHR8TFXVQ type=del size=3569783 2026-02-24T22:10:47.725 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.725+0000 7f7e64854980 1 < completed delete operation key=LSRPN33Z2B9N tag=W0ASFOV9LIJU type=del 2026-02-24T22:10:47.726 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.725+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=VDHZ5ZV8T4B3 type=del size=722338 2026-02-24T22:10:47.727 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.725+0000 7f7e64854980 1 < completed delete operation key=L60HQ7M3E3LL tag=Y2O96JEP8SJ5 type=del 2026-02-24T22:10:47.728 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.725+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=FBKFXK9PU22Y type=del size=1454418 2026-02-24T22:10:47.728 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.725+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~ML47G0S8XLPG.0 size=1048576 2026-02-24T22:10:47.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.725+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~ML47G0S8XLPG.1 size=1048576 2026-02-24T22:10:47.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.729+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~ML47G0S8XLPG.2 size=921462 2026-02-24T22:10:47.730 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.729+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=ML47G0S8XLPG tag=LLALK00Y26M0 type=write size=4067190 2026-02-24T22:10:47.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.729+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=5HHPA36I6UK1 type=del size=2777924 2026-02-24T22:10:47.732 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.729+0000 7f7e64854980 1 < completed delete operation key=Y3L3CJ51C1WR tag=3KEMFK8Z9VAM type=del 2026-02-24T22:10:47.733 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.733+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~0FWBGTT9DJN0.0 size=1048576 2026-02-24T22:10:47.734 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.733+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~0FWBGTT9DJN0.1 size=1048576 2026-02-24T22:10:47.734 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.733+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~0FWBGTT9DJN0.2 size=177810 2026-02-24T22:10:47.734 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.733+0000 7f7e64854980 1 > prepared multipart upload key=TF5B43G7KCYT upload=0FWBGTT9DJN0 tag=2ZVOUDR9504T type=write size=3323538 2026-02-24T22:10:47.735 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.733+0000 7f7e64854980 1 < completed delete operation key=NL60Q5QOSP8E tag=VYZ5KJ45390O type=del 2026-02-24T22:10:47.736 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.733+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=5A0LWFO5EPUF type=del size=2527785 2026-02-24T22:10:47.736 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.733+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~1MR90UVY8TW6.0 size=830960 2026-02-24T22:10:47.738 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.737+0000 7f7e64854980 1 < completed multipart upload key=EJCX06TE5RTZ upload=1MR90UVY8TW6 tag=O7HPS9X92YC8 type=write size=1879536 2026-02-24T22:10:47.739 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.737+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~T2X913DI6E1V.0 size=841959 2026-02-24T22:10:47.739 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.737+0000 7f7e64854980 1 > prepared multipart upload key=SGQYQ2TIVJPS upload=T2X913DI6E1V tag=WHUN193EPY11 type=write size=1890535 2026-02-24T22:10:47.739 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.737+0000 7f7e64854980 1 < completed delete operation key=EJCX06TE5RTZ tag=K7PW4FWYHOK2 type=del 2026-02-24T22:10:47.740 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.737+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=0JVN4F8WYT4R type=del size=1415997 2026-02-24T22:10:47.741 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.737+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~TJ32MM6XQODT.0 size=101552 2026-02-24T22:10:47.742 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.741+0000 7f7e64854980 1 < completed multipart upload key=CI8DA17DX8LU upload=TJ32MM6XQODT tag=THF4VK45HPP9 type=write size=1150128 2026-02-24T22:10:47.743 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.741+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=ULO7FG13P6Y7 type=del size=2088461 2026-02-24T22:10:47.743 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.741+0000 7f7e64854980 1 < completed write operation key=EJCX06TE5RTZ tag=89ACVNSC2K68 type=write size=52453 2026-02-24T22:10:47.744 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.741+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=OK4KLNJQDORH type=del size=707172 2026-02-24T22:10:47.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.745+0000 7f7e64854980 1 < suggested update operation key=PKRHWM69GBJI tag=71F5Z2RTRBHG type=del size=1131286 2026-02-24T22:10:47.746 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.745+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=30Q30ZV7FELU type=del size=2193238 2026-02-24T22:10:47.747 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.745+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~UI1TKUMCMOTN.0 size=1048576 2026-02-24T22:10:47.748 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.745+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~UI1TKUMCMOTN.1 size=1048576 2026-02-24T22:10:47.748 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.745+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~UI1TKUMCMOTN.2 size=335790 2026-02-24T22:10:47.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.749+0000 7f7e64854980 1 < completed multipart upload key=SGQYQ2TIVJPS upload=UI1TKUMCMOTN tag=QBHGTHGKYPZS type=write size=3481518 2026-02-24T22:10:47.750 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.749+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~PYPGMWE0E052.0 size=1048576 2026-02-24T22:10:47.751 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.749+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~PYPGMWE0E052.1 size=1048576 2026-02-24T22:10:47.752 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.749+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~PYPGMWE0E052.2 size=985633 2026-02-24T22:10:47.752 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.749+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=PYPGMWE0E052 tag=53FESM9C9JXP type=write size=4131361 2026-02-24T22:10:47.752 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.749+0000 7f7e64854980 1 < completed write operation key=HOHKR92SN25M tag=A3MCSKB8RRTT type=write size=696619 2026-02-24T22:10:47.753 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.749+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=SYO9ME31NRNX type=del size=3090885 2026-02-24T22:10:47.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.753+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~84PWTQ751BJF.0 size=1048576 2026-02-24T22:10:47.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.753+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~84PWTQ751BJF.1 size=1048576 2026-02-24T22:10:47.755 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.753+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~84PWTQ751BJF.2 size=739056 2026-02-24T22:10:47.756 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.753+0000 7f7e64854980 1 < completed multipart upload key=L60HQ7M3E3LL upload=84PWTQ751BJF tag=O7UGK50HNIHF type=write size=3884784 2026-02-24T22:10:47.757 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.753+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=URDWLONVT2CJ type=del size=2469635 2026-02-24T22:10:47.757 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.757+0000 7f7e64854980 1 < completed write operation key=HKOQRSF6XEXB tag=2BEBDR0DQGUM type=write size=875095 2026-02-24T22:10:47.758 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.757+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=1OICWCMQ0L3U type=write size=598300 2026-02-24T22:10:47.759 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.757+0000 7f7e64854980 1 < completed delete operation key=36MKR2OZN7K9 tag=L4HB1TJD4XL2 type=del 2026-02-24T22:10:47.760 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.757+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=PAC86RAVF6M2 type=del size=3707942 2026-02-24T22:10:47.760 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.757+0000 7f7e64854980 1 < canceled operation key=HKOQRSF6XEXB tag=GG0RHR8TFXVQ type=del size=3569783 2026-02-24T22:10:47.762 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.761+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~SYC2IS1VKUFC.0 size=1048576 2026-02-24T22:10:47.762 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.761+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~SYC2IS1VKUFC.1 size=1048576 2026-02-24T22:10:47.763 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.761+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~SYC2IS1VKUFC.2 size=874938 2026-02-24T22:10:47.763 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.761+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=SYC2IS1VKUFC tag=YR5ZPC0HFNJ0 type=write size=4020666 2026-02-24T22:10:47.763 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.761+0000 7f7e64854980 1 < completed delete operation key=N3ZUNSI5K8HS tag=VDHZ5ZV8T4B3 type=del 2026-02-24T22:10:47.765 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.761+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~58INIEUPG5LA.0 size=1048576 2026-02-24T22:10:47.765 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.761+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~58INIEUPG5LA.1 size=1048576 2026-02-24T22:10:47.766 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.765+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~58INIEUPG5LA.2 size=583217 2026-02-24T22:10:47.766 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.765+0000 7f7e64854980 1 > prepared multipart upload key=3900SFJK1YQU upload=58INIEUPG5LA tag=LDNH1JN0KKIY type=write size=3728945 2026-02-24T22:10:47.767 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.765+0000 7f7e64854980 1 < canceled operation key=SGQYQ2TIVJPS tag=FBKFXK9PU22Y type=del size=1454418 2026-02-24T22:10:47.768 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.765+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~4P70X82VYOQX.0 size=1048576 2026-02-24T22:10:47.768 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.765+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~4P70X82VYOQX.1 size=1048576 2026-02-24T22:10:47.769 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.765+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~4P70X82VYOQX.2 size=998760 2026-02-24T22:10:47.769 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.765+0000 7f7e64854980 1 > prepared multipart upload key=TF5B43G7KCYT upload=4P70X82VYOQX tag=LWTSS35GMZFW type=write size=4144488 2026-02-24T22:10:47.770 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.769+0000 7f7e64854980 1 < completed delete operation key=3900SFJK1YQU tag=5HHPA36I6UK1 type=del 2026-02-24T22:10:47.770 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.769+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=KDOF82IEL0UZ type=write size=92341 2026-02-24T22:10:47.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.769+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~0FWBGTT9DJN0.0 size=1048576 2026-02-24T22:10:47.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.769+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~0FWBGTT9DJN0.1 size=1048576 2026-02-24T22:10:47.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.769+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~0FWBGTT9DJN0.2 size=177810 2026-02-24T22:10:47.773 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.773+0000 7f7e64854980 1 < completed multipart upload key=TF5B43G7KCYT upload=0FWBGTT9DJN0 tag=2ZVOUDR9504T type=write size=3323538 2026-02-24T22:10:47.774 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.773+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=HSW3W0UI92A8 type=del size=2947571 2026-02-24T22:10:47.775 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.773+0000 7f7e64854980 1 < suggested update operation key=R2893NES3WSF tag=5A0LWFO5EPUF type=del size=2527785 2026-02-24T22:10:47.776 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.773+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=UJH3YEL0JKS1 type=del size=2007908 2026-02-24T22:10:47.777 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.773+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~T2X913DI6E1V.0 size=841959 2026-02-24T22:10:47.778 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.777+0000 7f7e64854980 1 < completed multipart upload key=SGQYQ2TIVJPS upload=T2X913DI6E1V tag=WHUN193EPY11 type=write size=1890535 2026-02-24T22:10:47.779 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.777+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=VMEJ9ZSZ1QNZ type=del size=474777 2026-02-24T22:10:47.779 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.777+0000 7f7e64854980 1 < completed delete operation key=D0MKGNRDW6L9 tag=0JVN4F8WYT4R type=del 2026-02-24T22:10:47.780 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.777+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=M7UUMPYZTU02 type=del size=2579481 2026-02-24T22:10:47.781 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.777+0000 7f7e64854980 -1 < failed to complete operation key=PKRHWM69GBJI tag=ULO7FG13P6Y7 type=del size=2088461: (22) Invalid argument 2026-02-24T22:10:47.781 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.781+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=8QRTKXABFI8Q type=del size=3887281 2026-02-24T22:10:47.782 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.781+0000 7f7e64854980 -1 < failed to complete operation key=R2893NES3WSF tag=OK4KLNJQDORH type=del size=707172: (22) Invalid argument 2026-02-24T22:10:47.783 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.781+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=3IB7YPGW771T type=del size=3355454 2026-02-24T22:10:47.784 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.781+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=30Q30ZV7FELU type=del 2026-02-24T22:10:47.784 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.781+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=WKDPB4MUPA0H type=del size=986271 2026-02-24T22:10:47.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.781+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~PYPGMWE0E052.0 size=1048576 2026-02-24T22:10:47.786 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.785+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~PYPGMWE0E052.1 size=1048576 2026-02-24T22:10:47.786 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.785+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~PYPGMWE0E052.2 size=985633 2026-02-24T22:10:47.788 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.785+0000 7f7e64854980 1 < completed multipart upload key=CI8DA17DX8LU upload=PYPGMWE0E052 tag=53FESM9C9JXP type=write size=4131361 2026-02-24T22:10:47.790 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.789+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~ID2DDZGQWCIK.0 size=1048576 2026-02-24T22:10:47.790 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.789+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~ID2DDZGQWCIK.1 size=103803 2026-02-24T22:10:47.790 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.789+0000 7f7e64854980 1 > prepared multipart upload key=0B2V0MHWSOC9 upload=ID2DDZGQWCIK tag=L73GNK9B8D5T type=write size=2200955 2026-02-24T22:10:47.792 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.789+0000 7f7e64854980 1 < suggested remove operation key=A6PH77Q78VEN tag=SYO9ME31NRNX type=del 2026-02-24T22:10:47.792 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.789+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=V0N5LH64PMPP type=write size=95953 2026-02-24T22:10:47.793 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.789+0000 7f7e64854980 1 < completed delete operation key=Y3L3CJ51C1WR tag=URDWLONVT2CJ type=del 2026-02-24T22:10:47.794 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.793+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=V55463HRDTY4 type=del size=2073069 2026-02-24T22:10:47.794 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.793+0000 7f7e64854980 1 < completed write operation key=CI8DA17DX8LU tag=1OICWCMQ0L3U type=write size=598300 2026-02-24T22:10:47.795 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.793+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=EZZWPR8GHBBA type=del size=2340935 2026-02-24T22:10:47.796 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.793+0000 7f7e64854980 1 < canceled operation key=ZESKGBWZXM6O tag=PAC86RAVF6M2 type=del size=3707942 2026-02-24T22:10:47.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.793+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=JVCJ8BBIREDR type=del size=4027610 2026-02-24T22:10:47.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.793+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~SYC2IS1VKUFC.0 size=1048576 2026-02-24T22:10:47.798 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.797+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~SYC2IS1VKUFC.1 size=1048576 2026-02-24T22:10:47.798 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.797+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~SYC2IS1VKUFC.2 size=874938 2026-02-24T22:10:47.799 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.797+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=SYC2IS1VKUFC tag=YR5ZPC0HFNJ0 type=write size=4020666 2026-02-24T22:10:47.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.797+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=YJVRYEQMMEHK type=del size=3538464 2026-02-24T22:10:47.801 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.797+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~58INIEUPG5LA.0 size=1048576 2026-02-24T22:10:47.801 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.801+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~58INIEUPG5LA.1 size=1048576 2026-02-24T22:10:47.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.801+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~58INIEUPG5LA.2 size=583217 2026-02-24T22:10:47.803 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.801+0000 7f7e64854980 1 < completed multipart upload key=3900SFJK1YQU upload=58INIEUPG5LA tag=LDNH1JN0KKIY type=write size=3728945 2026-02-24T22:10:47.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.801+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~DCXWE2LAPUSS.0 size=1048576 2026-02-24T22:10:47.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.801+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~DCXWE2LAPUSS.1 size=1048576 2026-02-24T22:10:47.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.805+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~DCXWE2LAPUSS.2 size=583557 2026-02-24T22:10:47.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.805+0000 7f7e64854980 1 > prepared multipart upload key=PKRHWM69GBJI upload=DCXWE2LAPUSS tag=BF1TRUDJ7C4N type=write size=3729285 2026-02-24T22:10:47.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.805+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~4P70X82VYOQX.0 size=1048576 2026-02-24T22:10:47.807 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.805+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~4P70X82VYOQX.1 size=1048576 2026-02-24T22:10:47.807 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.805+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~4P70X82VYOQX.2 size=998760 2026-02-24T22:10:47.808 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.805+0000 7f7e64854980 1 < completed multipart upload key=TF5B43G7KCYT upload=4P70X82VYOQX tag=LWTSS35GMZFW type=write size=4144488 2026-02-24T22:10:47.809 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.805+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=ORKXC4KPMFFS type=del size=1158181 2026-02-24T22:10:47.810 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.809+0000 7f7e64854980 1 < completed write operation key=EB04ETNYM087 tag=KDOF82IEL0UZ type=write size=92341 2026-02-24T22:10:47.811 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.809+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=U27A68JM2K2Q type=del size=88680 2026-02-24T22:10:47.835 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.809+0000 7f7e64854980 1 < completed delete operation key=4Y3974PYYT78 tag=HSW3W0UI92A8 type=del 2026-02-24T22:10:47.835 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.809+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=SR5D0W3TPBLE type=del size=1448008 2026-02-24T22:10:47.835 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.809+0000 7f7e64854980 1 < completed delete operation key=Q7BP8FXTARDW tag=UJH3YEL0JKS1 type=del 2026-02-24T22:10:47.835 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.813+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=BR2ZTT24FKM6 type=del size=1326063 2026-02-24T22:10:47.835 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.813+0000 7f7e64854980 1 < suggested remove operation key=0AXS7DY23UZU tag=VMEJ9ZSZ1QNZ type=del 2026-02-24T22:10:47.836 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.813+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~INGDQU7Q7TMP.0 size=1048576 2026-02-24T22:10:47.836 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.813+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~INGDQU7Q7TMP.1 size=1048576 2026-02-24T22:10:47.836 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.817+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~INGDQU7Q7TMP.2 size=535746 2026-02-24T22:10:47.836 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.817+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=INGDQU7Q7TMP tag=U7LXOSNR263U type=write size=3681474 2026-02-24T22:10:47.836 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.817+0000 7f7e64854980 1 < suggested remove operation key=D0MKGNRDW6L9 tag=M7UUMPYZTU02 type=del 2026-02-24T22:10:47.836 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.817+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=YJNMFVN6TWJN type=del size=4030536 2026-02-24T22:10:47.836 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.817+0000 7f7e64854980 1 < suggested remove operation key=UQLUT9ZAI61V tag=8QRTKXABFI8Q type=del 2026-02-24T22:10:47.836 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.821+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=0544HRSHBBYE type=del size=791809 2026-02-24T22:10:47.836 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.821+0000 7f7e64854980 1 < completed delete operation key=EJCX06TE5RTZ tag=3IB7YPGW771T type=del 2026-02-24T22:10:47.836 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.821+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=FN8GSX7C797C type=del size=2676185 2026-02-24T22:10:47.836 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.821+0000 7f7e64854980 -1 < failed to complete operation key=UQLUT9ZAI61V tag=WKDPB4MUPA0H type=del size=986271: (22) Invalid argument 2026-02-24T22:10:47.836 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.821+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=5YDO9V6GF0QH type=del size=507498 2026-02-24T22:10:47.836 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.825+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~ID2DDZGQWCIK.0 size=1048576 2026-02-24T22:10:47.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.825+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~ID2DDZGQWCIK.1 size=103803 2026-02-24T22:10:47.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.825+0000 7f7e64854980 1 < completed multipart upload key=0B2V0MHWSOC9 upload=ID2DDZGQWCIK tag=L73GNK9B8D5T type=write size=2200955 2026-02-24T22:10:47.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.825+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=5DRWJT13NNPZ type=del size=1351785 2026-02-24T22:10:47.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.829+0000 7f7e64854980 1 < suggested update operation key=ZESKGBWZXM6O tag=V0N5LH64PMPP type=write size=95953 2026-02-24T22:10:47.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.829+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~2FI3GPSNR0XR.0 size=1048576 2026-02-24T22:10:47.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.829+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~2FI3GPSNR0XR.1 size=519260 2026-02-24T22:10:47.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.829+0000 7f7e64854980 1 > prepared multipart upload key=PKRHWM69GBJI upload=2FI3GPSNR0XR tag=Y9MW8MLA5LCV type=write size=2616412 2026-02-24T22:10:47.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.829+0000 7f7e64854980 1 < completed delete operation key=GH7723GNI2WM tag=V55463HRDTY4 type=del 2026-02-24T22:10:47.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.829+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=1LP153GNC4FQ type=del size=1811002 2026-02-24T22:10:47.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.833+0000 7f7e64854980 1 < completed delete operation key=R2893NES3WSF tag=EZZWPR8GHBBA type=del 2026-02-24T22:10:47.838 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.833+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~K6ND076TPFXI.0 size=1048576 2026-02-24T22:10:47.838 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.833+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~K6ND076TPFXI.1 size=525030 2026-02-24T22:10:47.838 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.833+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=K6ND076TPFXI tag=GTQ1IJPBTMZG type=write size=2622182 2026-02-24T22:10:47.839 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.833+0000 7f7e64854980 -1 < failed to complete operation key=ZESKGBWZXM6O tag=JVCJ8BBIREDR type=del size=4027610: (22) Invalid argument 2026-02-24T22:10:47.839 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.837+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=J2MEFSNSEAZ8 type=del size=3281111 2026-02-24T22:10:47.839 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.837+0000 7f7e64854980 1 < canceled operation key=TF5B43G7KCYT tag=YJVRYEQMMEHK type=del size=3538464 2026-02-24T22:10:47.839 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.837+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=PD7Y5MCRH4TG type=write size=177544 2026-02-24T22:10:47.839 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.837+0000 7f7e64854980 1 < canceled part key=_multipart_PKRHWM69GBJI.2~DCXWE2LAPUSS.0 size=1048576 2026-02-24T22:10:47.840 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.837+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~DCXWE2LAPUSS.1 size=1048576 2026-02-24T22:10:47.840 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.837+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~DCXWE2LAPUSS.2 size=583557 2026-02-24T22:10:47.842 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.841+0000 7f7e64854980 1 < canceled multipart upload key=PKRHWM69GBJI upload=DCXWE2LAPUSS tag=BF1TRUDJ7C4N type=write size=3729285 2026-02-24T22:10:47.844 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.841+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~5T665AN10819.0 size=1048576 2026-02-24T22:10:47.844 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.841+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~5T665AN10819.1 size=329255 2026-02-24T22:10:47.844 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.841+0000 7f7e64854980 1 > prepared multipart upload key=GH7723GNI2WM upload=5T665AN10819 tag=WGD5FWWI07YW type=write size=2426407 2026-02-24T22:10:47.845 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.841+0000 7f7e64854980 1 < completed delete operation key=L9NEDH44BIGA tag=ORKXC4KPMFFS type=del 2026-02-24T22:10:47.847 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.845+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~RJ4I0VX09T94.0 size=601252 2026-02-24T22:10:47.847 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.845+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=RJ4I0VX09T94 tag=NY4Z7SLE8VNA type=write size=1649828 2026-02-24T22:10:47.847 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.845+0000 7f7e64854980 -1 < no bi entry to suggest for operation key=D0MKGNRDW6L9 tag=U27A68JM2K2Q type=del size=88680: (2) No such file or directory 2026-02-24T22:10:47.848 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.845+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=DAQTGHGBP9NW type=write size=667792 2026-02-24T22:10:47.848 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.845+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=SR5D0W3TPBLE type=del 2026-02-24T22:10:47.850 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.849+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~UTDBZVLXCNAZ.0 size=974308 2026-02-24T22:10:47.850 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.849+0000 7f7e64854980 1 > prepared multipart upload key=Y3L3CJ51C1WR upload=UTDBZVLXCNAZ tag=MPQS96DZGZDZ type=write size=2022884 2026-02-24T22:10:47.851 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.849+0000 7f7e64854980 1 < canceled operation key=R2893NES3WSF tag=BR2ZTT24FKM6 type=del size=1326063 2026-02-24T22:10:47.852 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.849+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=F2EE9H11V0A6 type=del size=3538897 2026-02-24T22:10:47.852 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.849+0000 7f7e64854980 1 < canceled part key=_multipart_CI8DA17DX8LU.2~INGDQU7Q7TMP.0 size=1048576 2026-02-24T22:10:47.853 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.849+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~INGDQU7Q7TMP.1 size=1048576 2026-02-24T22:10:47.853 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.853+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~INGDQU7Q7TMP.2 size=535746 2026-02-24T22:10:47.854 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.853+0000 7f7e64854980 1 < canceled multipart upload key=CI8DA17DX8LU upload=INGDQU7Q7TMP tag=U7LXOSNR263U type=write size=3681474 2026-02-24T22:10:47.856 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.853+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~IFOYU7QYCDMW.0 size=1048576 2026-02-24T22:10:47.856 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.853+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~IFOYU7QYCDMW.1 size=466216 2026-02-24T22:10:47.856 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.853+0000 7f7e64854980 1 > prepared multipart upload key=NL60Q5QOSP8E upload=IFOYU7QYCDMW tag=K9498QBN81MH type=write size=2563368 2026-02-24T22:10:47.858 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.857+0000 7f7e64854980 1 < completed delete operation key=Q7BP8FXTARDW tag=YJNMFVN6TWJN type=del 2026-02-24T22:10:47.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.857+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~IFQV089T2LUD.0 size=350174 2026-02-24T22:10:47.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.857+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=IFQV089T2LUD tag=LFDN1G307NRH type=write size=1398750 2026-02-24T22:10:47.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.857+0000 7f7e64854980 1 < completed delete operation key=TF5B43G7KCYT tag=0544HRSHBBYE type=del 2026-02-24T22:10:47.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.861+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~33YYL0PTQMDT.0 size=1048576 2026-02-24T22:10:47.862 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.861+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~33YYL0PTQMDT.1 size=1048576 2026-02-24T22:10:47.862 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.861+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~33YYL0PTQMDT.2 size=221101 2026-02-24T22:10:47.863 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.861+0000 7f7e64854980 1 > prepared multipart upload key=Y8GMOUA977H3 upload=33YYL0PTQMDT tag=T3NK0UVP15NS type=write size=3366829 2026-02-24T22:10:47.863 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.861+0000 7f7e64854980 1 < completed delete operation key=Z0OXREAUJVA8 tag=FN8GSX7C797C type=del 2026-02-24T22:10:47.864 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.861+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=JSMLQLISH9AB type=del size=2047611 2026-02-24T22:10:47.864 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.861+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=5YDO9V6GF0QH type=del 2026-02-24T22:10:47.865 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.861+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=GFJCAA57SN88 type=del size=4165982 2026-02-24T22:10:47.866 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.865+0000 7f7e64854980 -1 < failed to complete operation key=ZESKGBWZXM6O tag=5DRWJT13NNPZ type=del size=1351785: (22) Invalid argument 2026-02-24T22:10:47.866 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.865+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=FQV5AAOAOHZN type=del size=612091 2026-02-24T22:10:47.867 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.865+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~2FI3GPSNR0XR.0 size=1048576 2026-02-24T22:10:47.868 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.865+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~2FI3GPSNR0XR.1 size=519260 2026-02-24T22:10:47.869 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.865+0000 7f7e64854980 1 < completed multipart upload key=PKRHWM69GBJI upload=2FI3GPSNR0XR tag=Y9MW8MLA5LCV type=write size=2616412 2026-02-24T22:10:47.869 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.869+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=0XTGJEKE1MB2 type=del size=1337957 2026-02-24T22:10:47.870 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.869+0000 7f7e64854980 1 < completed delete operation key=0B2V0MHWSOC9 tag=1LP153GNC4FQ type=del 2026-02-24T22:10:47.871 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.869+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=OWFI0KT66XXA type=write size=569364 2026-02-24T22:10:47.871 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.869+0000 7f7e64854980 1 < canceled part key=_multipart_2KKQKHXO7K3K.2~K6ND076TPFXI.0 size=1048576 2026-02-24T22:10:47.872 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.869+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~K6ND076TPFXI.1 size=525030 2026-02-24T22:10:47.873 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.869+0000 7f7e64854980 1 < canceled multipart upload key=2KKQKHXO7K3K upload=K6ND076TPFXI tag=GTQ1IJPBTMZG type=write size=2622182 2026-02-24T22:10:47.873 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.873+0000 7f7e64854980 1 > prepared operation key=TF5B43G7KCYT tag=0JLHTS3EUCQD type=del size=3220411 2026-02-24T22:10:47.874 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.873+0000 7f7e64854980 1 < completed delete operation key=N3ZUNSI5K8HS tag=J2MEFSNSEAZ8 type=del 2026-02-24T22:10:47.876 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.873+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~BJIU3BJPZKF5.0 size=1048576 2026-02-24T22:10:47.876 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.873+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~BJIU3BJPZKF5.1 size=1048576 2026-02-24T22:10:47.877 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.873+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~BJIU3BJPZKF5.2 size=592413 2026-02-24T22:10:47.877 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.873+0000 7f7e64854980 1 > prepared multipart upload key=Y8GMOUA977H3 upload=BJIU3BJPZKF5 tag=8C61B76JITXC type=write size=3738141 2026-02-24T22:10:47.877 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.877+0000 7f7e64854980 1 < completed write operation key=3900SFJK1YQU tag=PD7Y5MCRH4TG type=write size=177544 2026-02-24T22:10:47.878 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.877+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=PFVSPYWB0RUZ type=del size=3514648 2026-02-24T22:10:47.878 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.877+0000 7f7e64854980 1 < canceled part key=_multipart_GH7723GNI2WM.2~5T665AN10819.0 size=1048576 2026-02-24T22:10:47.879 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.877+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~5T665AN10819.1 size=329255 2026-02-24T22:10:47.880 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.877+0000 7f7e64854980 1 < canceled multipart upload key=GH7723GNI2WM upload=5T665AN10819 tag=WGD5FWWI07YW type=write size=2426407 2026-02-24T22:10:47.881 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.877+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=LNYYZHL1J4JA type=del size=2693545 2026-02-24T22:10:47.882 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.881+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~RJ4I0VX09T94.0 size=601252 2026-02-24T22:10:47.883 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.881+0000 7f7e64854980 1 < completed multipart upload key=D0MKGNRDW6L9 upload=RJ4I0VX09T94 tag=NY4Z7SLE8VNA type=write size=1649828 2026-02-24T22:10:47.883 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.881+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=N00VNHB2VRHV type=del size=3147863 2026-02-24T22:10:47.885 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.881+0000 7f7e64854980 1 < suggested update operation key=HOHKR92SN25M tag=DAQTGHGBP9NW type=write size=667792 2026-02-24T22:10:47.885 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.885+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=1ZNXDAFFEAW6 type=del size=769739 2026-02-24T22:10:47.886 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.885+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~UTDBZVLXCNAZ.0 size=974308 2026-02-24T22:10:47.887 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.885+0000 7f7e64854980 1 < completed multipart upload key=Y3L3CJ51C1WR upload=UTDBZVLXCNAZ tag=MPQS96DZGZDZ type=write size=2022884 2026-02-24T22:10:47.888 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.885+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=3TQ64MRQE465 type=del size=748319 2026-02-24T22:10:47.889 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.885+0000 7f7e64854980 1 < completed delete operation key=LSRPN33Z2B9N tag=F2EE9H11V0A6 type=del 2026-02-24T22:10:47.889 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.889+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=7SY4IHVJH62V type=write size=163261 2026-02-24T22:10:47.890 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.889+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~IFOYU7QYCDMW.0 size=1048576 2026-02-24T22:10:47.890 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.889+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~IFOYU7QYCDMW.1 size=466216 2026-02-24T22:10:47.892 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.889+0000 7f7e64854980 1 < completed multipart upload key=NL60Q5QOSP8E upload=IFOYU7QYCDMW tag=K9498QBN81MH type=write size=2563368 2026-02-24T22:10:47.893 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.889+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~T2QNLC1DAMWQ.0 size=231060 2026-02-24T22:10:47.893 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.889+0000 7f7e64854980 1 > prepared multipart upload key=TF5B43G7KCYT upload=T2QNLC1DAMWQ tag=85Z36UJ6WXJS type=write size=1279636 2026-02-24T22:10:47.894 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.893+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~IFQV089T2LUD.0 size=350174 2026-02-24T22:10:47.895 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.893+0000 7f7e64854980 1 < completed multipart upload key=D0MKGNRDW6L9 upload=IFQV089T2LUD tag=LFDN1G307NRH type=write size=1398750 2026-02-24T22:10:47.896 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.893+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~NKUVPZBJQUPI.0 size=1048576 2026-02-24T22:10:47.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.893+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~NKUVPZBJQUPI.1 size=311309 2026-02-24T22:10:47.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.893+0000 7f7e64854980 1 > prepared multipart upload key=L60HQ7M3E3LL upload=NKUVPZBJQUPI tag=23SI3MYGX8QW type=write size=2408461 2026-02-24T22:10:47.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.893+0000 7f7e64854980 1 < canceled part key=_multipart_Y8GMOUA977H3.2~33YYL0PTQMDT.0 size=1048576 2026-02-24T22:10:47.898 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.897+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~33YYL0PTQMDT.1 size=1048576 2026-02-24T22:10:47.898 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.897+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~33YYL0PTQMDT.2 size=221101 2026-02-24T22:10:47.899 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.897+0000 7f7e64854980 1 < canceled multipart upload key=Y8GMOUA977H3 upload=33YYL0PTQMDT tag=T3NK0UVP15NS type=write size=3366829 2026-02-24T22:10:47.900 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.897+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=I20J7B4R4RHJ type=del size=414857 2026-02-24T22:10:47.901 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.901+0000 7f7e64854980 1 < suggested update operation key=R2893NES3WSF tag=JSMLQLISH9AB type=del size=2047611 2026-02-24T22:10:47.902 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.901+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=578A73CZ1SIR type=write size=373926 2026-02-24T22:10:47.903 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.901+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=GFJCAA57SN88 type=del 2026-02-24T22:10:47.904 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.901+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~81RXVZF12JU1.0 size=1048576 2026-02-24T22:10:47.905 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.901+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~81RXVZF12JU1.1 size=1048576 2026-02-24T22:10:47.905 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.905+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~81RXVZF12JU1.2 size=34158 2026-02-24T22:10:47.906 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.905+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=81RXVZF12JU1 tag=5B56C3J20T7Y type=write size=3179886 2026-02-24T22:10:47.907 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.905+0000 7f7e64854980 1 < suggested remove operation key=Z0OXREAUJVA8 tag=FQV5AAOAOHZN type=del 2026-02-24T22:10:47.908 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.905+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~0JJNKLXK481C.0 size=1048576 2026-02-24T22:10:47.909 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.905+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~0JJNKLXK481C.1 size=1048576 2026-02-24T22:10:47.909 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.905+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~0JJNKLXK481C.2 size=53859 2026-02-24T22:10:47.909 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.905+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=0JJNKLXK481C tag=XXTSK2O8DRY4 type=write size=3199587 2026-02-24T22:10:47.910 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.909+0000 7f7e64854980 1 < canceled operation key=GH7723GNI2WM tag=0XTGJEKE1MB2 type=del size=1337957 2026-02-24T22:10:47.911 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.909+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~EHA93E60L8WZ.0 size=1048576 2026-02-24T22:10:47.912 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.909+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~EHA93E60L8WZ.1 size=150563 2026-02-24T22:10:47.912 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.909+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=EHA93E60L8WZ tag=Z767DIX1HQL8 type=write size=2247715 2026-02-24T22:10:47.913 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.909+0000 7f7e64854980 1 < suggested remove operation key=Y8GMOUA977H3 tag=OWFI0KT66XXA type=write 2026-02-24T22:10:47.914 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.913+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~N0T0XE8W6W3S.0 size=1048576 2026-02-24T22:10:47.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.913+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~N0T0XE8W6W3S.1 size=832201 2026-02-24T22:10:47.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.913+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=N0T0XE8W6W3S tag=3CO34TJFXZJK type=write size=2929353 2026-02-24T22:10:47.916 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.913+0000 7f7e64854980 1 < suggested remove operation key=TF5B43G7KCYT tag=0JLHTS3EUCQD type=del 2026-02-24T22:10:47.917 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.913+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=0R4CJXG64PUT type=del size=505416 2026-02-24T22:10:47.917 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.917+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~BJIU3BJPZKF5.0 size=1048576 2026-02-24T22:10:47.918 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.917+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~BJIU3BJPZKF5.1 size=1048576 2026-02-24T22:10:47.918 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.917+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~BJIU3BJPZKF5.2 size=592413 2026-02-24T22:10:47.920 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.917+0000 7f7e64854980 1 < completed multipart upload key=Y8GMOUA977H3 upload=BJIU3BJPZKF5 tag=8C61B76JITXC type=write size=3738141 2026-02-24T22:10:47.920 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.917+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=EI40NQKYUL6Y type=del size=2280719 2026-02-24T22:10:47.922 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.921+0000 7f7e64854980 1 < suggested remove operation key=SGQYQ2TIVJPS tag=PFVSPYWB0RUZ type=del 2026-02-24T22:10:47.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.921+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=64J2E18CNVW3 type=del size=191166 2026-02-24T22:10:47.924 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.921+0000 7f7e64854980 -1 < failed to complete operation key=HOHKR92SN25M tag=LNYYZHL1J4JA type=del size=2693545: (22) Invalid argument 2026-02-24T22:10:47.925 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.921+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=4I2AK0S73089 type=del size=1376276 2026-02-24T22:10:47.926 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.925+0000 7f7e64854980 1 < suggested update operation key=4Y3974PYYT78 tag=N00VNHB2VRHV type=del size=3147863 2026-02-24T22:10:47.927 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.925+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=ZIXXN88VS35L type=del size=1996695 2026-02-24T22:10:47.928 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.925+0000 7f7e64854980 1 < completed delete operation key=Q7BP8FXTARDW tag=1ZNXDAFFEAW6 type=del 2026-02-24T22:10:47.928 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.925+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=875BRPDLA6Q6 type=del size=158262 2026-02-24T22:10:47.929 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.925+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=3TQ64MRQE465 type=del 2026-02-24T22:10:47.930 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.929+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~UFE6MF7URD3J.0 size=1048576 2026-02-24T22:10:47.931 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.929+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~UFE6MF7URD3J.1 size=660483 2026-02-24T22:10:47.931 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.929+0000 7f7e64854980 1 > prepared multipart upload key=78GUTC5THI2P upload=UFE6MF7URD3J tag=UI59WGZYW50B type=write size=2757635 2026-02-24T22:10:47.931 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.929+0000 7f7e64854980 1 < completed write operation key=3900SFJK1YQU tag=7SY4IHVJH62V type=write size=163261 2026-02-24T22:10:47.932 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.929+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=GCY4O61JLHMC type=del size=4065346 2026-02-24T22:10:47.933 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.929+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~T2QNLC1DAMWQ.0 size=231060 2026-02-24T22:10:47.934 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.933+0000 7f7e64854980 1 < completed multipart upload key=TF5B43G7KCYT upload=T2QNLC1DAMWQ tag=85Z36UJ6WXJS type=write size=1279636 2026-02-24T22:10:47.935 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.933+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~OHQHR6SLO2JC.0 size=1048576 2026-02-24T22:10:47.936 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.933+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~OHQHR6SLO2JC.1 size=1048576 2026-02-24T22:10:47.936 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.933+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~OHQHR6SLO2JC.2 size=1046017 2026-02-24T22:10:47.936 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.933+0000 7f7e64854980 1 > prepared multipart upload key=HKOQRSF6XEXB upload=OHQHR6SLO2JC tag=7XP3JHMDQP0O type=write size=4191745 2026-02-24T22:10:47.937 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.933+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~NKUVPZBJQUPI.0 size=1048576 2026-02-24T22:10:47.937 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.937+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~NKUVPZBJQUPI.1 size=311309 2026-02-24T22:10:47.938 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.937+0000 7f7e64854980 1 < completed multipart upload key=L60HQ7M3E3LL upload=NKUVPZBJQUPI tag=23SI3MYGX8QW type=write size=2408461 2026-02-24T22:10:47.939 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.937+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=K4SLTIBMF8WC type=write size=426626 2026-02-24T22:10:47.941 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.937+0000 7f7e64854980 1 < suggested update operation key=ON16L53JUSTW tag=I20J7B4R4RHJ type=del size=414857 2026-02-24T22:10:47.941 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.941+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=L9LS6H4SN774 type=write size=269503 2026-02-24T22:10:47.942 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.941+0000 7f7e64854980 1 < completed write operation key=L9NEDH44BIGA tag=578A73CZ1SIR type=write size=373926 2026-02-24T22:10:47.943 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.941+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=QDHM5ZY3KNGO type=del size=958867 2026-02-24T22:10:47.944 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.941+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~81RXVZF12JU1.0 size=1048576 2026-02-24T22:10:47.944 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.941+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~81RXVZF12JU1.1 size=1048576 2026-02-24T22:10:47.945 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.941+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~81RXVZF12JU1.2 size=34158 2026-02-24T22:10:47.946 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.945+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=81RXVZF12JU1 tag=5B56C3J20T7Y type=write size=3179886 2026-02-24T22:10:47.947 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.945+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=2402E01R7ODI type=del size=4033719 2026-02-24T22:10:47.947 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.945+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~0JJNKLXK481C.0 size=1048576 2026-02-24T22:10:47.948 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.945+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~0JJNKLXK481C.1 size=1048576 2026-02-24T22:10:47.948 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.945+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~0JJNKLXK481C.2 size=53859 2026-02-24T22:10:47.950 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.949+0000 7f7e64854980 1 < completed multipart upload key=ZESKGBWZXM6O upload=0JJNKLXK481C tag=XXTSK2O8DRY4 type=write size=3199587 2026-02-24T22:10:47.950 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.949+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=0GVPCZ9H9CI9 type=write size=212593 2026-02-24T22:10:47.951 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.949+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~EHA93E60L8WZ.0 size=1048576 2026-02-24T22:10:47.951 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.949+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~EHA93E60L8WZ.1 size=150563 2026-02-24T22:10:47.953 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.949+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=EHA93E60L8WZ tag=Z767DIX1HQL8 type=write size=2247715 2026-02-24T22:10:47.954 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.953+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~CY9MVGUT4DTC.0 size=1048576 2026-02-24T22:10:47.954 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.953+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~CY9MVGUT4DTC.1 size=1048576 2026-02-24T22:10:47.955 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.953+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~CY9MVGUT4DTC.2 size=404397 2026-02-24T22:10:47.955 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.953+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=CY9MVGUT4DTC tag=MQVPTVYOX8LO type=write size=3550125 2026-02-24T22:10:47.956 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.953+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~N0T0XE8W6W3S.0 size=1048576 2026-02-24T22:10:47.956 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.953+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~N0T0XE8W6W3S.1 size=832201 2026-02-24T22:10:47.957 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.957+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=N0T0XE8W6W3S tag=3CO34TJFXZJK type=write size=2929353 2026-02-24T22:10:47.958 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.957+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=SR8ZZW50C6WK type=del size=3068416 2026-02-24T22:10:47.959 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.957+0000 7f7e64854980 1 < completed delete operation key=3900SFJK1YQU tag=0R4CJXG64PUT type=del 2026-02-24T22:10:47.959 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.957+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=GMFDUSV55PX7 type=del size=2977771 2026-02-24T22:10:47.960 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.957+0000 7f7e64854980 1 < completed delete operation key=HKOQRSF6XEXB tag=EI40NQKYUL6Y type=del 2026-02-24T22:10:47.961 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.957+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~7DGZG3GE1H4E.0 size=395303 2026-02-24T22:10:47.961 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.957+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=7DGZG3GE1H4E tag=WQDWFZVWAOAZ type=write size=1443879 2026-02-24T22:10:47.962 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.961+0000 7f7e64854980 1 < completed delete operation key=36MKR2OZN7K9 tag=64J2E18CNVW3 type=del 2026-02-24T22:10:47.963 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.961+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=HDVPPV2CHAWG type=del size=1284674 2026-02-24T22:10:47.963 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.961+0000 7f7e64854980 1 < completed delete operation key=Q7BP8FXTARDW tag=4I2AK0S73089 type=del 2026-02-24T22:10:47.965 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.961+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~VB0E6ZWT70X1.0 size=1048576 2026-02-24T22:10:47.965 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.961+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~VB0E6ZWT70X1.1 size=1048576 2026-02-24T22:10:47.966 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.965+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~VB0E6ZWT70X1.2 size=422802 2026-02-24T22:10:47.966 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.965+0000 7f7e64854980 1 > prepared multipart upload key=SGQYQ2TIVJPS upload=VB0E6ZWT70X1 tag=28AA71XLHQ1D type=write size=3568530 2026-02-24T22:10:47.966 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.965+0000 7f7e64854980 1 < completed delete operation key=R2893NES3WSF tag=ZIXXN88VS35L type=del 2026-02-24T22:10:47.967 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.965+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=M9JT0C14S0SK type=del size=2982233 2026-02-24T22:10:47.968 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.965+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=875BRPDLA6Q6 type=del 2026-02-24T22:10:47.969 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.965+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~F58O7BGJJRGE.0 size=1048576 2026-02-24T22:10:47.969 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.969+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~F58O7BGJJRGE.1 size=990281 2026-02-24T22:10:47.970 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.969+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=F58O7BGJJRGE tag=US3KSW78S4H5 type=write size=3087433 2026-02-24T22:10:47.970 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.969+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~UFE6MF7URD3J.0 size=1048576 2026-02-24T22:10:47.971 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.969+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~UFE6MF7URD3J.1 size=660483 2026-02-24T22:10:47.972 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.969+0000 7f7e64854980 1 < completed multipart upload key=78GUTC5THI2P upload=UFE6MF7URD3J tag=UI59WGZYW50B type=write size=2757635 2026-02-24T22:10:47.973 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.969+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=E8RVZFM1LU9M type=del size=2151394 2026-02-24T22:10:47.973 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.973+0000 7f7e64854980 1 < canceled operation key=78GUTC5THI2P tag=GCY4O61JLHMC type=del size=4065346 2026-02-24T22:10:47.974 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.973+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=DJL362FZ7L1U type=del size=3355067 2026-02-24T22:10:47.975 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.973+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~OHQHR6SLO2JC.0 size=1048576 2026-02-24T22:10:47.975 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.973+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~OHQHR6SLO2JC.1 size=1048576 2026-02-24T22:10:47.976 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.973+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~OHQHR6SLO2JC.2 size=1046017 2026-02-24T22:10:47.977 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.973+0000 7f7e64854980 1 < completed multipart upload key=HKOQRSF6XEXB upload=OHQHR6SLO2JC tag=7XP3JHMDQP0O type=write size=4191745 2026-02-24T22:10:47.978 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.977+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=ZEDDJ8HQ66YQ type=del size=625241 2026-02-24T22:10:47.978 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.977+0000 7f7e64854980 1 < completed write operation key=EB04ETNYM087 tag=K4SLTIBMF8WC type=write size=426626 2026-02-24T22:10:47.979 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.977+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=YHL222QNDHU1 type=write size=151646 2026-02-24T22:10:47.980 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.977+0000 7f7e64854980 1 < completed write operation key=L9NEDH44BIGA tag=L9LS6H4SN774 type=write size=269503 2026-02-24T22:10:47.981 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.977+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~BI5GWP6315PX.0 size=1048576 2026-02-24T22:10:47.981 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.981+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~BI5GWP6315PX.1 size=448616 2026-02-24T22:10:47.982 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.981+0000 7f7e64854980 1 > prepared multipart upload key=SGQYQ2TIVJPS upload=BI5GWP6315PX tag=HTKO0E7R0NYB type=write size=2545768 2026-02-24T22:10:47.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.981+0000 7f7e64854980 1 < suggested remove operation key=D0MKGNRDW6L9 tag=QDHM5ZY3KNGO type=del 2026-02-24T22:10:47.984 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.981+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=CNHCDEXBL6SC type=del size=488115 2026-02-24T22:10:47.985 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.981+0000 7f7e64854980 1 < suggested update operation key=R2893NES3WSF tag=2402E01R7ODI type=del size=4033719 2026-02-24T22:10:47.986 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.985+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=AAWEH7YEJ9KT type=del size=165721 2026-02-24T22:10:47.986 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.985+0000 7f7e64854980 1 < completed write operation key=0AXS7DY23UZU tag=0GVPCZ9H9CI9 type=write size=212593 2026-02-24T22:10:47.988 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.985+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=QDNH99C082W5 type=del size=327480 2026-02-24T22:10:47.989 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.989+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~CY9MVGUT4DTC.0 size=1048576 2026-02-24T22:10:47.990 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.989+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~CY9MVGUT4DTC.1 size=1048576 2026-02-24T22:10:47.990 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.989+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~CY9MVGUT4DTC.2 size=404397 2026-02-24T22:10:47.992 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.989+0000 7f7e64854980 1 < completed multipart upload key=TXRUQQ456DD8 upload=CY9MVGUT4DTC tag=MQVPTVYOX8LO type=write size=3550125 2026-02-24T22:10:47.992 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.989+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=81IGFGSQAIQV type=write size=705185 2026-02-24T22:10:47.994 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.993+0000 7f7e64854980 1 < suggested remove operation key=CI8DA17DX8LU tag=SR8ZZW50C6WK type=del 2026-02-24T22:10:47.994 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.993+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=M0VAN805O20I type=del size=1890189 2026-02-24T22:10:47.995 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.993+0000 7f7e64854980 1 < completed delete operation key=HKOQRSF6XEXB tag=GMFDUSV55PX7 type=del 2026-02-24T22:10:47.996 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.993+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=8KEHSUC7NPGF type=del size=2058453 2026-02-24T22:10:47.996 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.993+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~7DGZG3GE1H4E.0 size=395303 2026-02-24T22:10:47.997 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.997+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=7DGZG3GE1H4E tag=WQDWFZVWAOAZ type=write size=1443879 2026-02-24T22:10:47.998 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.997+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=GAQ54D8SWNYF type=del size=288445 2026-02-24T22:10:47.999 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.997+0000 7f7e64854980 1 < completed delete operation key=EJCX06TE5RTZ tag=HDVPPV2CHAWG type=del 2026-02-24T22:10:48.000 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.997+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=XWRGEC4TRC3B type=write size=507224 2026-02-24T22:10:48.000 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.997+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~VB0E6ZWT70X1.0 size=1048576 2026-02-24T22:10:48.001 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:47.997+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~VB0E6ZWT70X1.1 size=1048576 2026-02-24T22:10:48.001 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.001+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~VB0E6ZWT70X1.2 size=422802 2026-02-24T22:10:48.003 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.001+0000 7f7e64854980 1 < completed multipart upload key=SGQYQ2TIVJPS upload=VB0E6ZWT70X1 tag=28AA71XLHQ1D type=write size=3568530 2026-02-24T22:10:48.004 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.001+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~QG1ZR2NBAIQI.0 size=327848 2026-02-24T22:10:48.004 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.001+0000 7f7e64854980 1 > prepared multipart upload key=3900SFJK1YQU upload=QG1ZR2NBAIQI tag=QZMTHY16FABB type=write size=1376424 2026-02-24T22:10:48.005 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.001+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=M9JT0C14S0SK type=del 2026-02-24T22:10:48.005 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.005+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=8FYC4FK2IHOH type=del size=1371076 2026-02-24T22:10:48.006 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.005+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~F58O7BGJJRGE.0 size=1048576 2026-02-24T22:10:48.006 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.005+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~F58O7BGJJRGE.1 size=990281 2026-02-24T22:10:48.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.005+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=F58O7BGJJRGE tag=US3KSW78S4H5 type=write size=3087433 2026-02-24T22:10:48.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.005+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~8J3RBT5IOB3M.0 size=1048576 2026-02-24T22:10:48.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.009+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~8J3RBT5IOB3M.1 size=1048576 2026-02-24T22:10:48.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.009+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~8J3RBT5IOB3M.2 size=217912 2026-02-24T22:10:48.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.009+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=8J3RBT5IOB3M tag=KDQWKIMJWZE9 type=write size=3363640 2026-02-24T22:10:48.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.009+0000 7f7e64854980 -1 < no bi entry to suggest for operation key=CI8DA17DX8LU tag=E8RVZFM1LU9M type=del size=2151394: (2) No such file or directory 2026-02-24T22:10:48.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.009+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~4AAV26GISJG5.0 size=23326 2026-02-24T22:10:48.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.009+0000 7f7e64854980 1 > prepared multipart upload key=TF5B43G7KCYT upload=4AAV26GISJG5 tag=WX9ITQVO0W5U type=write size=1071902 2026-02-24T22:10:48.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.009+0000 7f7e64854980 1 < completed delete operation key=SGQYQ2TIVJPS tag=DJL362FZ7L1U type=del 2026-02-24T22:10:48.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.013+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~15Q2DDKNWB3K.0 size=1048576 2026-02-24T22:10:48.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.013+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~15Q2DDKNWB3K.1 size=1048576 2026-02-24T22:10:48.015 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.013+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~15Q2DDKNWB3K.2 size=749827 2026-02-24T22:10:48.015 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.013+0000 7f7e64854980 1 > prepared multipart upload key=Z0OXREAUJVA8 upload=15Q2DDKNWB3K tag=Z29OC6T63WX0 type=write size=3895555 2026-02-24T22:10:48.015 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.013+0000 7f7e64854980 1 < completed delete operation key=UQLUT9ZAI61V tag=ZEDDJ8HQ66YQ type=del 2026-02-24T22:10:48.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.013+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~Y1C6AESSVFHG.0 size=1048576 2026-02-24T22:10:48.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.017+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~Y1C6AESSVFHG.1 size=523979 2026-02-24T22:10:48.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.017+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=Y1C6AESSVFHG tag=HIBXHNUXJ2LH type=write size=2621131 2026-02-24T22:10:48.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.017+0000 7f7e64854980 1 < completed write operation key=36MKR2OZN7K9 tag=YHL222QNDHU1 type=write size=151646 2026-02-24T22:10:48.019 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.017+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=7Y87WC5CTW0P type=del size=1321141 2026-02-24T22:10:48.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.017+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~BI5GWP6315PX.0 size=1048576 2026-02-24T22:10:48.021 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.017+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~BI5GWP6315PX.1 size=448616 2026-02-24T22:10:48.022 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.021+0000 7f7e64854980 1 < completed multipart upload key=SGQYQ2TIVJPS upload=BI5GWP6315PX tag=HTKO0E7R0NYB type=write size=2545768 2026-02-24T22:10:48.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.021+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=7N09G9XQ5ROW type=del size=212209 2026-02-24T22:10:48.024 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.021+0000 7f7e64854980 1 < completed delete operation key=EJCX06TE5RTZ tag=CNHCDEXBL6SC type=del 2026-02-24T22:10:48.025 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.021+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=M9KUE4Q7JZU6 type=del size=1165604 2026-02-24T22:10:48.025 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.025+0000 7f7e64854980 1 < completed delete operation key=PKRHWM69GBJI tag=AAWEH7YEJ9KT type=del 2026-02-24T22:10:48.026 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.025+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=J4KXIYBG75IV type=del size=2314966 2026-02-24T22:10:48.028 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.025+0000 7f7e64854980 1 < suggested update operation key=2KKQKHXO7K3K tag=QDNH99C082W5 type=del size=327480 2026-02-24T22:10:48.029 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.025+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=D4Q9YTES3P53 type=del size=2271 2026-02-24T22:10:48.030 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.029+0000 7f7e64854980 1 < canceled operation key=N3ZUNSI5K8HS tag=81IGFGSQAIQV type=write size=705185 2026-02-24T22:10:48.030 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.029+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=4FWU73U6UWKZ type=del size=1787508 2026-02-24T22:10:48.031 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.029+0000 7f7e64854980 1 < completed delete operation key=3900SFJK1YQU tag=M0VAN805O20I type=del 2026-02-24T22:10:48.032 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.029+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=8EFAKTGF1W3Q type=del size=3125243 2026-02-24T22:10:48.033 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.029+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=8KEHSUC7NPGF type=del 2026-02-24T22:10:48.033 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.033+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=8L9XZRUPM6KO type=del size=2186197 2026-02-24T22:10:48.035 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.033+0000 7f7e64854980 1 < suggested remove operation key=UQLUT9ZAI61V tag=GAQ54D8SWNYF type=del 2026-02-24T22:10:48.036 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.033+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~RIWDVENEMPEL.0 size=1048576 2026-02-24T22:10:48.036 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.033+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~RIWDVENEMPEL.1 size=150599 2026-02-24T22:10:48.037 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.033+0000 7f7e64854980 1 > prepared multipart upload key=PKRHWM69GBJI upload=RIWDVENEMPEL tag=F1OF3Y2XIDU4 type=write size=2247751 2026-02-24T22:10:48.037 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.033+0000 7f7e64854980 1 < completed write operation key=36MKR2OZN7K9 tag=XWRGEC4TRC3B type=write size=507224 2026-02-24T22:10:48.038 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.037+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=DGD3OCICUDJB type=write size=324275 2026-02-24T22:10:48.039 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.037+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~QG1ZR2NBAIQI.0 size=327848 2026-02-24T22:10:48.040 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.037+0000 7f7e64854980 1 < completed multipart upload key=3900SFJK1YQU upload=QG1ZR2NBAIQI tag=QZMTHY16FABB type=write size=1376424 2026-02-24T22:10:48.040 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.037+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=O16TYYAKW730 type=del size=4147268 2026-02-24T22:10:48.041 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.037+0000 7f7e64854980 1 < canceled operation key=TXRUQQ456DD8 tag=8FYC4FK2IHOH type=del size=1371076 2026-02-24T22:10:48.042 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.041+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=1C01GHIVPVWU type=write size=288226 2026-02-24T22:10:48.043 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.041+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~8J3RBT5IOB3M.0 size=1048576 2026-02-24T22:10:48.043 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.041+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~8J3RBT5IOB3M.1 size=1048576 2026-02-24T22:10:48.044 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.041+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~8J3RBT5IOB3M.2 size=217912 2026-02-24T22:10:48.045 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.041+0000 7f7e64854980 1 < completed multipart upload key=D0MKGNRDW6L9 upload=8J3RBT5IOB3M tag=KDQWKIMJWZE9 type=write size=3363640 2026-02-24T22:10:48.045 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.045+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=KZJGKCX48BAV type=del size=3811258 2026-02-24T22:10:48.046 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.045+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~4AAV26GISJG5.0 size=23326 2026-02-24T22:10:48.048 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.045+0000 7f7e64854980 1 < completed multipart upload key=TF5B43G7KCYT upload=4AAV26GISJG5 tag=WX9ITQVO0W5U type=write size=1071902 2026-02-24T22:10:48.048 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.045+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=B3YX46JPBB31 type=del size=3759482 2026-02-24T22:10:48.049 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.045+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~15Q2DDKNWB3K.0 size=1048576 2026-02-24T22:10:48.050 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.049+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~15Q2DDKNWB3K.1 size=1048576 2026-02-24T22:10:48.050 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.049+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~15Q2DDKNWB3K.2 size=749827 2026-02-24T22:10:48.051 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.049+0000 7f7e64854980 1 < completed multipart upload key=Z0OXREAUJVA8 upload=15Q2DDKNWB3K tag=Z29OC6T63WX0 type=write size=3895555 2026-02-24T22:10:48.052 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.049+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=EQTOPXNKEWWL type=write size=332186 2026-02-24T22:10:48.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.049+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~Y1C6AESSVFHG.0 size=1048576 2026-02-24T22:10:48.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.053+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~Y1C6AESSVFHG.1 size=523979 2026-02-24T22:10:48.054 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.053+0000 7f7e64854980 1 < completed multipart upload key=HOHKR92SN25M upload=Y1C6AESSVFHG tag=HIBXHNUXJ2LH type=write size=2621131 2026-02-24T22:10:48.055 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.053+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=OTCXSHKGK504 type=del size=1561574 2026-02-24T22:10:48.058 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.057+0000 7f7e64854980 1 < suggested update operation key=SGQYQ2TIVJPS tag=7Y87WC5CTW0P type=del size=1321141 2026-02-24T22:10:48.059 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.057+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=D4XKR10EJAJV type=del size=3816634 2026-02-24T22:10:48.059 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.057+0000 7f7e64854980 1 < completed delete operation key=D0MKGNRDW6L9 tag=7N09G9XQ5ROW type=del 2026-02-24T22:10:48.060 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.057+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=OHI35786AT1C type=del size=1659477 2026-02-24T22:10:48.061 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.057+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=M9KUE4Q7JZU6 type=del 2026-02-24T22:10:48.062 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.061+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=BTKONDN6O3EN type=del size=1053982 2026-02-24T22:10:48.063 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.061+0000 7f7e64854980 1 < suggested update operation key=78GUTC5THI2P tag=J4KXIYBG75IV type=del size=2314966 2026-02-24T22:10:48.064 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.061+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=ILX73KB0V2UH type=del size=2683401 2026-02-24T22:10:48.064 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.061+0000 7f7e64854980 1 < completed delete operation key=L9NEDH44BIGA tag=D4Q9YTES3P53 type=del 2026-02-24T22:10:48.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.061+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=Q02ERGTQ7UZ7 type=write size=1009989 2026-02-24T22:10:48.066 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.065+0000 7f7e64854980 1 < suggested remove operation key=Y8GMOUA977H3 tag=4FWU73U6UWKZ type=del 2026-02-24T22:10:48.067 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.065+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=GNWWRMBHNRRF type=del size=3073597 2026-02-24T22:10:48.068 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.065+0000 7f7e64854980 1 < suggested update operation key=EJCX06TE5RTZ tag=8EFAKTGF1W3Q type=del size=3125243 2026-02-24T22:10:48.070 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.069+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~X1245AW5WRUH.0 size=221082 2026-02-24T22:10:48.070 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.069+0000 7f7e64854980 1 > prepared multipart upload key=LSRPN33Z2B9N upload=X1245AW5WRUH tag=1DJHZRQ8ZN28 type=write size=1269658 2026-02-24T22:10:48.070 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.069+0000 7f7e64854980 1 < completed delete operation key=Y3L3CJ51C1WR tag=8L9XZRUPM6KO type=del 2026-02-24T22:10:48.071 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.069+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=HOCZ5FF8DSNG type=write size=14036 2026-02-24T22:10:48.072 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.069+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~RIWDVENEMPEL.0 size=1048576 2026-02-24T22:10:48.072 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.069+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~RIWDVENEMPEL.1 size=150599 2026-02-24T22:10:48.073 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.073+0000 7f7e64854980 1 < completed multipart upload key=PKRHWM69GBJI upload=RIWDVENEMPEL tag=F1OF3Y2XIDU4 type=write size=2247751 2026-02-24T22:10:48.074 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.073+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=0BI9WAJ0ST6H type=write size=430524 2026-02-24T22:10:48.075 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.073+0000 7f7e64854980 -1 < failed to complete operation key=EJCX06TE5RTZ tag=DGD3OCICUDJB type=write size=324275: (22) Invalid argument 2026-02-24T22:10:48.076 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.073+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=93HF01V6UYHE type=del size=4008851 2026-02-24T22:10:48.076 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.073+0000 7f7e64854980 1 < completed delete operation key=36MKR2OZN7K9 tag=O16TYYAKW730 type=del 2026-02-24T22:10:48.077 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.073+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=EV9363RGMZH2 type=del size=701284 2026-02-24T22:10:48.078 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.077+0000 7f7e64854980 1 < completed write operation key=Y3L3CJ51C1WR tag=1C01GHIVPVWU type=write size=288226 2026-02-24T22:10:48.079 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.077+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~BQTKRS4N1F22.0 size=1048576 2026-02-24T22:10:48.079 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.077+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~BQTKRS4N1F22.1 size=610156 2026-02-24T22:10:48.079 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.077+0000 7f7e64854980 1 > prepared multipart upload key=Y8GMOUA977H3 upload=BQTKRS4N1F22 tag=FVBCGHF8ZBSX type=write size=2707308 2026-02-24T22:10:48.081 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.077+0000 7f7e64854980 1 < suggested remove operation key=UQLUT9ZAI61V tag=KZJGKCX48BAV type=del 2026-02-24T22:10:48.082 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.081+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~750B00GTHGP7.0 size=1048576 2026-02-24T22:10:48.082 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.081+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~750B00GTHGP7.1 size=966630 2026-02-24T22:10:48.083 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.081+0000 7f7e64854980 1 > prepared multipart upload key=Z0OXREAUJVA8 upload=750B00GTHGP7 tag=9IT0FSGBPXUS type=write size=3063782 2026-02-24T22:10:48.083 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.081+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=B3YX46JPBB31 type=del 2026-02-24T22:10:48.084 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.081+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=NPLEJTR0F0V4 type=del size=1681246 2026-02-24T22:10:48.084 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.081+0000 7f7e64854980 1 < completed write operation key=N3ZUNSI5K8HS tag=EQTOPXNKEWWL type=write size=332186 2026-02-24T22:10:48.086 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.085+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~VDB5U5N1HLHL.0 size=1048576 2026-02-24T22:10:48.086 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.085+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~VDB5U5N1HLHL.1 size=976337 2026-02-24T22:10:48.086 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.085+0000 7f7e64854980 1 > prepared multipart upload key=PKRHWM69GBJI upload=VDB5U5N1HLHL tag=71FC7GW6U18M type=write size=3073489 2026-02-24T22:10:48.087 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.085+0000 7f7e64854980 -1 < failed to complete operation key=Y8GMOUA977H3 tag=OTCXSHKGK504 type=del size=1561574: (22) Invalid argument 2026-02-24T22:10:48.088 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.085+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~7LXM25PTTMRY.0 size=1048576 2026-02-24T22:10:48.089 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.085+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~7LXM25PTTMRY.1 size=724043 2026-02-24T22:10:48.089 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.085+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=7LXM25PTTMRY tag=0CJZ53A0MT7A type=write size=2821195 2026-02-24T22:10:48.089 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.089+0000 7f7e64854980 1 < completed delete operation key=0B2V0MHWSOC9 tag=D4XKR10EJAJV type=del 2026-02-24T22:10:48.090 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.089+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=MS4TUURP9BMF type=del size=2885749 2026-02-24T22:10:48.091 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.089+0000 7f7e64854980 1 < completed delete operation key=0AXS7DY23UZU tag=OHI35786AT1C type=del 2026-02-24T22:10:48.092 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.089+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~1FJ2SZ05I3AS.0 size=1048576 2026-02-24T22:10:48.093 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.089+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~1FJ2SZ05I3AS.1 size=1048576 2026-02-24T22:10:48.093 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.093+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~1FJ2SZ05I3AS.2 size=783218 2026-02-24T22:10:48.094 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.093+0000 7f7e64854980 1 > prepared multipart upload key=Q7BP8FXTARDW upload=1FJ2SZ05I3AS tag=PH0U2UVN0L2M type=write size=3928946 2026-02-24T22:10:48.094 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.093+0000 7f7e64854980 1 < completed delete operation key=3900SFJK1YQU tag=BTKONDN6O3EN type=del 2026-02-24T22:10:48.095 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.093+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=P6KNXE8MS9YP type=del size=3336103 2026-02-24T22:10:48.095 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.093+0000 7f7e64854980 1 < completed delete operation key=EB04ETNYM087 tag=ILX73KB0V2UH type=del 2026-02-24T22:10:48.097 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.093+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~33WACJE8R7E3.0 size=1048576 2026-02-24T22:10:48.097 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.097+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~33WACJE8R7E3.1 size=80898 2026-02-24T22:10:48.097 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.097+0000 7f7e64854980 1 > prepared multipart upload key=GH7723GNI2WM upload=33WACJE8R7E3 tag=AKVP0QKT16KZ type=write size=2178050 2026-02-24T22:10:48.098 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.097+0000 7f7e64854980 1 < completed write operation key=Y3L3CJ51C1WR tag=Q02ERGTQ7UZ7 type=write size=1009989 2026-02-24T22:10:48.099 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.097+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=EQZDUAX4V356 type=del size=1823696 2026-02-24T22:10:48.099 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.097+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=GNWWRMBHNRRF type=del 2026-02-24T22:10:48.100 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.097+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=BJ5UWMV0QD48 type=del size=3726307 2026-02-24T22:10:48.101 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.097+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~X1245AW5WRUH.0 size=221082 2026-02-24T22:10:48.102 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.101+0000 7f7e64854980 1 < completed multipart upload key=LSRPN33Z2B9N upload=X1245AW5WRUH tag=1DJHZRQ8ZN28 type=write size=1269658 2026-02-24T22:10:48.103 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.101+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~WW91LQASVV07.0 size=1048576 2026-02-24T22:10:48.104 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.101+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~WW91LQASVV07.1 size=20015 2026-02-24T22:10:48.104 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.101+0000 7f7e64854980 1 > prepared multipart upload key=78GUTC5THI2P upload=WW91LQASVV07 tag=XS11PM1YZXZS type=write size=2117167 2026-02-24T22:10:48.105 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.101+0000 7f7e64854980 1 < suggested update operation key=SGQYQ2TIVJPS tag=HOCZ5FF8DSNG type=write size=14036 2026-02-24T22:10:48.106 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.105+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~D3B2IWCFX626.0 size=805231 2026-02-24T22:10:48.106 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.105+0000 7f7e64854980 1 > prepared multipart upload key=36MKR2OZN7K9 upload=D3B2IWCFX626 tag=0RGNOKX8AT3K type=write size=1853807 2026-02-24T22:10:48.107 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.105+0000 7f7e64854980 1 < completed write operation key=Q7BP8FXTARDW tag=0BI9WAJ0ST6H type=write size=430524 2026-02-24T22:10:48.108 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.105+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=CMWK8KOECXD8 type=del size=153325 2026-02-24T22:10:48.109 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.105+0000 7f7e64854980 1 < suggested update operation key=NL60Q5QOSP8E tag=93HF01V6UYHE type=del size=4008851 2026-02-24T22:10:48.110 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.109+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=GCSGR6BVIV1D type=write size=1001700 2026-02-24T22:10:48.110 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.109+0000 7f7e64854980 1 < canceled operation key=EB04ETNYM087 tag=EV9363RGMZH2 type=del size=701284 2026-02-24T22:10:48.111 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.109+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=X5HTSJ27PQQJ type=write size=727765 2026-02-24T22:10:48.112 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.109+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~BQTKRS4N1F22.0 size=1048576 2026-02-24T22:10:48.112 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.109+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~BQTKRS4N1F22.1 size=610156 2026-02-24T22:10:48.113 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.113+0000 7f7e64854980 1 < completed multipart upload key=Y8GMOUA977H3 upload=BQTKRS4N1F22 tag=FVBCGHF8ZBSX type=write size=2707308 2026-02-24T22:10:48.115 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.113+0000 7f7e64854980 1 > prepared part key=_multipart_HOHKR92SN25M.2~733PJOU9FEG0.0 size=918006 2026-02-24T22:10:48.115 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.113+0000 7f7e64854980 1 > prepared multipart upload key=HOHKR92SN25M upload=733PJOU9FEG0 tag=45C6T5KN1C0B type=write size=1966582 2026-02-24T22:10:48.115 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.113+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~750B00GTHGP7.0 size=1048576 2026-02-24T22:10:48.116 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.113+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~750B00GTHGP7.1 size=966630 2026-02-24T22:10:48.117 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.113+0000 7f7e64854980 1 < completed multipart upload key=Z0OXREAUJVA8 upload=750B00GTHGP7 tag=9IT0FSGBPXUS type=write size=3063782 2026-02-24T22:10:48.118 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.117+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~SYLI300JJGV8.0 size=788069 2026-02-24T22:10:48.118 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.117+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=SYLI300JJGV8 tag=17HJ43GBV7IL type=write size=1836645 2026-02-24T22:10:48.120 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.117+0000 7f7e64854980 1 < suggested remove operation key=ON16L53JUSTW tag=NPLEJTR0F0V4 type=del 2026-02-24T22:10:48.121 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.117+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=CJELMUWGTTJP type=del size=3937782 2026-02-24T22:10:48.121 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.121+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~VDB5U5N1HLHL.0 size=1048576 2026-02-24T22:10:48.122 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.121+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~VDB5U5N1HLHL.1 size=976337 2026-02-24T22:10:48.125 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.121+0000 7f7e64854980 1 < completed multipart upload key=PKRHWM69GBJI upload=VDB5U5N1HLHL tag=71FC7GW6U18M type=write size=3073489 2026-02-24T22:10:48.125 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.125+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=KYYCLMYNQA9N type=write size=896836 2026-02-24T22:10:48.126 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.125+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~7LXM25PTTMRY.0 size=1048576 2026-02-24T22:10:48.127 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.125+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~7LXM25PTTMRY.1 size=724043 2026-02-24T22:10:48.128 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.125+0000 7f7e64854980 1 < completed multipart upload key=0AXS7DY23UZU upload=7LXM25PTTMRY tag=0CJZ53A0MT7A type=write size=2821195 2026-02-24T22:10:48.129 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.125+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=F9OAAZXRCRYX type=del size=2168114 2026-02-24T22:10:48.130 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.129+0000 7f7e64854980 1 < completed delete operation key=EJCX06TE5RTZ tag=MS4TUURP9BMF type=del 2026-02-24T22:10:48.130 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.129+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=V3Y5ZWO3HDDH type=write size=908204 2026-02-24T22:10:48.131 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.129+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~1FJ2SZ05I3AS.0 size=1048576 2026-02-24T22:10:48.131 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.129+0000 7f7e64854980 1 < canceled part key=_multipart_Q7BP8FXTARDW.2~1FJ2SZ05I3AS.1 size=1048576 2026-02-24T22:10:48.132 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.129+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~1FJ2SZ05I3AS.2 size=783218 2026-02-24T22:10:48.133 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.129+0000 7f7e64854980 1 < canceled multipart upload key=Q7BP8FXTARDW upload=1FJ2SZ05I3AS tag=PH0U2UVN0L2M type=write size=3928946 2026-02-24T22:10:48.133 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.133+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=9PLR41XSZ7W5 type=write size=355795 2026-02-24T22:10:48.134 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.133+0000 7f7e64854980 1 < completed delete operation key=TXRUQQ456DD8 tag=P6KNXE8MS9YP type=del 2026-02-24T22:10:48.135 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.133+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=9GHBH2KN034T type=del size=503999 2026-02-24T22:10:48.136 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.133+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~33WACJE8R7E3.0 size=1048576 2026-02-24T22:10:48.136 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.133+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~33WACJE8R7E3.1 size=80898 2026-02-24T22:10:48.137 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.137+0000 7f7e64854980 1 < completed multipart upload key=GH7723GNI2WM upload=33WACJE8R7E3 tag=AKVP0QKT16KZ type=write size=2178050 2026-02-24T22:10:48.138 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.137+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=NMAIXXXC83VV type=write size=988989 2026-02-24T22:10:48.140 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.137+0000 7f7e64854980 1 < suggested remove operation key=LSRPN33Z2B9N tag=EQZDUAX4V356 type=del 2026-02-24T22:10:48.141 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.137+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~T5K3FZ0KTW2A.0 size=1048576 2026-02-24T22:10:48.141 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.141+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~T5K3FZ0KTW2A.1 size=658430 2026-02-24T22:10:48.142 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.141+0000 7f7e64854980 1 > prepared multipart upload key=HKOQRSF6XEXB upload=T5K3FZ0KTW2A tag=010RRGL5P4LG type=write size=2755582 2026-02-24T22:10:48.142 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.141+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=BJ5UWMV0QD48 type=del 2026-02-24T22:10:48.144 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.141+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~36BTROD5RXTR.0 size=856411 2026-02-24T22:10:48.144 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.141+0000 7f7e64854980 1 > prepared multipart upload key=EB04ETNYM087 upload=36BTROD5RXTR tag=9P3B61R0R3G8 type=write size=1904987 2026-02-24T22:10:48.144 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.141+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~WW91LQASVV07.0 size=1048576 2026-02-24T22:10:48.145 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.141+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~WW91LQASVV07.1 size=20015 2026-02-24T22:10:48.146 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.145+0000 7f7e64854980 1 < completed multipart upload key=78GUTC5THI2P upload=WW91LQASVV07 tag=XS11PM1YZXZS type=write size=2117167 2026-02-24T22:10:48.147 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.145+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~R83RL9GUX21M.0 size=1048576 2026-02-24T22:10:48.148 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.145+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~R83RL9GUX21M.1 size=600374 2026-02-24T22:10:48.148 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.145+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=R83RL9GUX21M tag=MC0EHVN8AS2L type=write size=2697526 2026-02-24T22:10:48.149 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.145+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~D3B2IWCFX626.0 size=805231 2026-02-24T22:10:48.150 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.149+0000 7f7e64854980 1 < completed multipart upload key=36MKR2OZN7K9 upload=D3B2IWCFX626 tag=0RGNOKX8AT3K type=write size=1853807 2026-02-24T22:10:48.151 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.149+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=3XQRIRO1RC15 type=del size=2896110 2026-02-24T22:10:48.152 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.149+0000 7f7e64854980 1 < completed delete operation key=UQLUT9ZAI61V tag=CMWK8KOECXD8 type=del 2026-02-24T22:10:48.153 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.149+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~YMD3C91BT7WV.0 size=1048576 2026-02-24T22:10:48.153 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.153+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~YMD3C91BT7WV.1 size=43691 2026-02-24T22:10:48.153 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.153+0000 7f7e64854980 1 > prepared multipart upload key=GH7723GNI2WM upload=YMD3C91BT7WV tag=9F98XI2IPJTP type=write size=2140843 2026-02-24T22:10:48.154 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.153+0000 7f7e64854980 1 < completed write operation key=Q7BP8FXTARDW tag=GCSGR6BVIV1D type=write size=1001700 2026-02-24T22:10:48.155 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.153+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=FLDZ1R8YE540 type=del size=1779370 2026-02-24T22:10:48.156 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.153+0000 7f7e64854980 1 < suggested update operation key=TXRUQQ456DD8 tag=X5HTSJ27PQQJ type=write size=727765 2026-02-24T22:10:48.157 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.153+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~Y4PRHVE3NKFD.0 size=1048576 2026-02-24T22:10:48.157 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.157+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~Y4PRHVE3NKFD.1 size=229403 2026-02-24T22:10:48.157 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.157+0000 7f7e64854980 1 > prepared multipart upload key=UQLUT9ZAI61V upload=Y4PRHVE3NKFD tag=NROSHWUFDEGY type=write size=2326555 2026-02-24T22:10:48.158 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.157+0000 7f7e64854980 1 < completed part key=_multipart_HOHKR92SN25M.2~733PJOU9FEG0.0 size=918006 2026-02-24T22:10:48.159 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.157+0000 7f7e64854980 1 < completed multipart upload key=HOHKR92SN25M upload=733PJOU9FEG0 tag=45C6T5KN1C0B type=write size=1966582 2026-02-24T22:10:48.160 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.157+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=J4V82OP3D293 type=del size=3885064 2026-02-24T22:10:48.160 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.157+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~SYLI300JJGV8.0 size=788069 2026-02-24T22:10:48.161 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.161+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=SYLI300JJGV8 tag=17HJ43GBV7IL type=write size=1836645 2026-02-24T22:10:48.163 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.161+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~OCNBXAUYPDX6.0 size=1048576 2026-02-24T22:10:48.163 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.161+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~OCNBXAUYPDX6.1 size=322891 2026-02-24T22:10:48.163 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.161+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=OCNBXAUYPDX6 tag=Y5Z4X7EXEN7H type=write size=2420043 2026-02-24T22:10:48.164 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.161+0000 7f7e64854980 1 < completed delete operation key=Z0OXREAUJVA8 tag=CJELMUWGTTJP type=del 2026-02-24T22:10:48.164 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.161+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=6FZS8DEZHET7 type=del size=525663 2026-02-24T22:10:48.166 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.165+0000 7f7e64854980 1 < suggested remove operation key=EB04ETNYM087 tag=KYYCLMYNQA9N type=write 2026-02-24T22:10:48.167 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.165+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~U0MBOYQ8PEVK.0 size=409166 2026-02-24T22:10:48.167 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.165+0000 7f7e64854980 1 > prepared multipart upload key=Y8GMOUA977H3 upload=U0MBOYQ8PEVK tag=Z5E57G7E6JX5 type=write size=1457742 2026-02-24T22:10:48.168 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.165+0000 7f7e64854980 1 < completed delete operation key=3900SFJK1YQU tag=F9OAAZXRCRYX type=del 2026-02-24T22:10:48.168 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.165+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=X9G6RLIJDGEW type=del size=113653 2026-02-24T22:10:48.169 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.165+0000 7f7e64854980 1 < completed write operation key=2KKQKHXO7K3K tag=V3Y5ZWO3HDDH type=write size=908204 2026-02-24T22:10:48.169 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.169+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=U7BFHNSS4M39 type=del size=3358714 2026-02-24T22:10:48.170 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.169+0000 7f7e64854980 1 < completed write operation key=36MKR2OZN7K9 tag=9PLR41XSZ7W5 type=write size=355795 2026-02-24T22:10:48.171 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.169+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=MEGUNEQPGIJW type=del size=1523584 2026-02-24T22:10:48.172 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.169+0000 7f7e64854980 1 < suggested remove operation key=EJCX06TE5RTZ tag=9GHBH2KN034T type=del 2026-02-24T22:10:48.173 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.169+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=QL37KCIQYPW8 type=del size=2410082 2026-02-24T22:10:48.173 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.173+0000 7f7e64854980 1 < completed write operation key=D0MKGNRDW6L9 tag=NMAIXXXC83VV type=write size=988989 2026-02-24T22:10:48.174 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.173+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=4U8P8K0NAMW8 type=del size=340304 2026-02-24T22:10:48.175 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.173+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~T5K3FZ0KTW2A.0 size=1048576 2026-02-24T22:10:48.175 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.173+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~T5K3FZ0KTW2A.1 size=658430 2026-02-24T22:10:48.176 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.173+0000 7f7e64854980 1 < completed multipart upload key=HKOQRSF6XEXB upload=T5K3FZ0KTW2A tag=010RRGL5P4LG type=write size=2755582 2026-02-24T22:10:48.177 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.173+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=G7XBQPG3F338 type=del size=4074209 2026-02-24T22:10:48.178 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.177+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~36BTROD5RXTR.0 size=856411 2026-02-24T22:10:48.179 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.177+0000 7f7e64854980 1 < completed multipart upload key=EB04ETNYM087 upload=36BTROD5RXTR tag=9P3B61R0R3G8 type=write size=1904987 2026-02-24T22:10:48.180 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.177+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~8D16FHE0N4F2.0 size=1048576 2026-02-24T22:10:48.180 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.177+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~8D16FHE0N4F2.1 size=1048576 2026-02-24T22:10:48.181 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.177+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~8D16FHE0N4F2.2 size=529619 2026-02-24T22:10:48.181 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.177+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=8D16FHE0N4F2 tag=8BT7QRKK4ED7 type=write size=3675347 2026-02-24T22:10:48.182 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.181+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~R83RL9GUX21M.0 size=1048576 2026-02-24T22:10:48.182 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.181+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~R83RL9GUX21M.1 size=600374 2026-02-24T22:10:48.183 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.181+0000 7f7e64854980 1 < completed multipart upload key=TXRUQQ456DD8 upload=R83RL9GUX21M tag=MC0EHVN8AS2L type=write size=2697526 2026-02-24T22:10:48.184 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.181+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=OV4MKNVL9CVG type=del size=124639 2026-02-24T22:10:48.185 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.181+0000 7f7e64854980 1 < completed delete operation key=L60HQ7M3E3LL tag=3XQRIRO1RC15 type=del 2026-02-24T22:10:48.185 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.185+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=935DK23YKNFA type=del size=3068410 2026-02-24T22:10:48.186 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.185+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~YMD3C91BT7WV.0 size=1048576 2026-02-24T22:10:48.186 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.185+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~YMD3C91BT7WV.1 size=43691 2026-02-24T22:10:48.188 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.185+0000 7f7e64854980 1 < completed multipart upload key=GH7723GNI2WM upload=YMD3C91BT7WV tag=9F98XI2IPJTP type=write size=2140843 2026-02-24T22:10:48.189 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.185+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=MXX1RWDNMLYU type=del size=818018 2026-02-24T22:10:48.190 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.189+0000 7f7e64854980 1 < completed delete operation key=N3ZUNSI5K8HS tag=FLDZ1R8YE540 type=del 2026-02-24T22:10:48.191 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.189+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=WN9R1TTLM4H3 type=del size=1308544 2026-02-24T22:10:48.192 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.189+0000 7f7e64854980 1 < completed part key=_multipart_UQLUT9ZAI61V.2~Y4PRHVE3NKFD.0 size=1048576 2026-02-24T22:10:48.192 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.189+0000 7f7e64854980 1 < completed part key=_multipart_UQLUT9ZAI61V.2~Y4PRHVE3NKFD.1 size=229403 2026-02-24T22:10:48.193 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.189+0000 7f7e64854980 1 < completed multipart upload key=UQLUT9ZAI61V upload=Y4PRHVE3NKFD tag=NROSHWUFDEGY type=write size=2326555 2026-02-24T22:10:48.194 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.193+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~1407ITUUF2WQ.0 size=521238 2026-02-24T22:10:48.194 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.193+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=1407ITUUF2WQ tag=4936BK29ZGG1 type=write size=1569814 2026-02-24T22:10:48.196 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.193+0000 7f7e64854980 1 < suggested update operation key=4Y3974PYYT78 tag=J4V82OP3D293 type=del size=3885064 2026-02-24T22:10:48.197 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.193+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~1XUOHC7SMWU2.0 size=1048576 2026-02-24T22:10:48.197 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.197+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~1XUOHC7SMWU2.1 size=1048576 2026-02-24T22:10:48.198 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.197+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~1XUOHC7SMWU2.2 size=1041350 2026-02-24T22:10:48.198 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.197+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=1XUOHC7SMWU2 tag=K7BB5559NKKZ type=write size=4187078 2026-02-24T22:10:48.199 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.197+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~OCNBXAUYPDX6.0 size=1048576 2026-02-24T22:10:48.199 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.197+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~OCNBXAUYPDX6.1 size=322891 2026-02-24T22:10:48.200 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.197+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=OCNBXAUYPDX6 tag=Y5Z4X7EXEN7H type=write size=2420043 2026-02-24T22:10:48.201 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.201+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~SIT9KCM3DX8E.0 size=1048576 2026-02-24T22:10:48.202 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.201+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~SIT9KCM3DX8E.1 size=70135 2026-02-24T22:10:48.202 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.201+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=SIT9KCM3DX8E tag=TH84XCKNU91V type=write size=2167287 2026-02-24T22:10:48.203 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.201+0000 7f7e64854980 1 < completed delete operation key=LSRPN33Z2B9N tag=6FZS8DEZHET7 type=del 2026-02-24T22:10:48.204 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.201+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~S43H5B20K1P1.0 size=1048576 2026-02-24T22:10:48.204 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.201+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~S43H5B20K1P1.1 size=562243 2026-02-24T22:10:48.204 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.201+0000 7f7e64854980 1 > prepared multipart upload key=0B2V0MHWSOC9 upload=S43H5B20K1P1 tag=S0LI3AINYNGM type=write size=2659395 2026-02-24T22:10:48.205 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.201+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~U0MBOYQ8PEVK.0 size=409166 2026-02-24T22:10:48.206 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.205+0000 7f7e64854980 1 < completed multipart upload key=Y8GMOUA977H3 upload=U0MBOYQ8PEVK tag=Z5E57G7E6JX5 type=write size=1457742 2026-02-24T22:10:48.207 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.205+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=3M5XTZJLCZVK type=del size=1283385 2026-02-24T22:10:48.207 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.205+0000 7f7e64854980 1 < completed delete operation key=R2893NES3WSF tag=X9G6RLIJDGEW type=del 2026-02-24T22:10:48.208 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.205+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=E5K8TOFQYXBT type=del size=3697624 2026-02-24T22:10:48.208 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.205+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=U7BFHNSS4M39 type=del 2026-02-24T22:10:48.210 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.209+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~4PPWXSU3D3ON.0 size=1048576 2026-02-24T22:10:48.211 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.209+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~4PPWXSU3D3ON.1 size=843996 2026-02-24T22:10:48.211 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.209+0000 7f7e64854980 1 > prepared multipart upload key=Y3L3CJ51C1WR upload=4PPWXSU3D3ON tag=XB1USUUPJBF4 type=write size=2941148 2026-02-24T22:10:48.211 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.209+0000 7f7e64854980 1 < completed delete operation key=EB04ETNYM087 tag=MEGUNEQPGIJW type=del 2026-02-24T22:10:48.212 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.209+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=M8ZX7L77LCGA type=del size=2429105 2026-02-24T22:10:48.213 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.209+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=QL37KCIQYPW8 type=del 2026-02-24T22:10:48.213 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.213+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=C03HU5FX5NX9 type=del size=4037141 2026-02-24T22:10:48.214 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.213+0000 7f7e64854980 1 < completed delete operation key=SGQYQ2TIVJPS tag=4U8P8K0NAMW8 type=del 2026-02-24T22:10:48.215 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.213+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~0ZUNXPPDFM3Q.0 size=1048576 2026-02-24T22:10:48.216 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.213+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~0ZUNXPPDFM3Q.1 size=1048576 2026-02-24T22:10:48.216 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.213+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~0ZUNXPPDFM3Q.2 size=683606 2026-02-24T22:10:48.216 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.213+0000 7f7e64854980 1 > prepared multipart upload key=3900SFJK1YQU upload=0ZUNXPPDFM3Q tag=GCY89VG6D0PN type=write size=3829334 2026-02-24T22:10:48.217 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.213+0000 7f7e64854980 1 < completed delete operation key=N3ZUNSI5K8HS tag=G7XBQPG3F338 type=del 2026-02-24T22:10:48.218 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.217+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=KQLWH81IMJOV type=del size=2768983 2026-02-24T22:10:48.218 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.217+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~8D16FHE0N4F2.0 size=1048576 2026-02-24T22:10:48.219 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.217+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~8D16FHE0N4F2.1 size=1048576 2026-02-24T22:10:48.219 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.217+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~8D16FHE0N4F2.2 size=529619 2026-02-24T22:10:48.221 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.217+0000 7f7e64854980 1 < completed multipart upload key=D0MKGNRDW6L9 upload=8D16FHE0N4F2 tag=8BT7QRKK4ED7 type=write size=3675347 2026-02-24T22:10:48.221 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.221+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=O8ROIK111LV9 type=del size=3481278 2026-02-24T22:10:48.222 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.221+0000 7f7e64854980 1 < completed delete operation key=UQLUT9ZAI61V tag=OV4MKNVL9CVG type=del 2026-02-24T22:10:48.222 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.221+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=53EL5LVSYPR6 type=write size=966361 2026-02-24T22:10:48.223 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.221+0000 7f7e64854980 1 < completed delete operation key=GH7723GNI2WM tag=935DK23YKNFA type=del 2026-02-24T22:10:48.224 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.221+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=CN7S8319OJ10 type=del size=94437 2026-02-24T22:10:48.224 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.221+0000 7f7e64854980 1 < completed delete operation key=3900SFJK1YQU tag=MXX1RWDNMLYU type=del 2026-02-24T22:10:48.225 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.221+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=SMO0A5RGTWIP type=del size=3111376 2026-02-24T22:10:48.226 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.225+0000 7f7e64854980 1 < suggested remove operation key=HOHKR92SN25M tag=WN9R1TTLM4H3 type=del 2026-02-24T22:10:48.227 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.225+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~Q4KVCXKAJV1A.0 size=1036956 2026-02-24T22:10:48.227 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.225+0000 7f7e64854980 1 > prepared multipart upload key=36MKR2OZN7K9 upload=Q4KVCXKAJV1A tag=7JFGA989R5IQ type=write size=2085532 2026-02-24T22:10:48.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.225+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~1407ITUUF2WQ.0 size=521238 2026-02-24T22:10:48.229 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.225+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=1407ITUUF2WQ tag=4936BK29ZGG1 type=write size=1569814 2026-02-24T22:10:48.230 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.229+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~XBQMXVSMSO4K.0 size=675775 2026-02-24T22:10:48.230 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.229+0000 7f7e64854980 1 > prepared multipart upload key=HKOQRSF6XEXB upload=XBQMXVSMSO4K tag=DCBY79X15FWP type=write size=1724351 2026-02-24T22:10:48.230 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.229+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~1XUOHC7SMWU2.0 size=1048576 2026-02-24T22:10:48.231 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.229+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~1XUOHC7SMWU2.1 size=1048576 2026-02-24T22:10:48.231 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.229+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~1XUOHC7SMWU2.2 size=1041350 2026-02-24T22:10:48.232 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.229+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=1XUOHC7SMWU2 tag=K7BB5559NKKZ type=write size=4187078 2026-02-24T22:10:48.233 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.229+0000 7f7e64854980 1 > prepared operation key=3900SFJK1YQU tag=0S6AVP54CA2U type=del size=728577 2026-02-24T22:10:48.233 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.233+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~SIT9KCM3DX8E.0 size=1048576 2026-02-24T22:10:48.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.233+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~SIT9KCM3DX8E.1 size=70135 2026-02-24T22:10:48.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.233+0000 7f7e64854980 1 < completed multipart upload key=0AXS7DY23UZU upload=SIT9KCM3DX8E tag=TH84XCKNU91V type=write size=2167287 2026-02-24T22:10:48.236 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.233+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~AH0T87R5OWWW.0 size=1048576 2026-02-24T22:10:48.237 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.233+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~AH0T87R5OWWW.1 size=739025 2026-02-24T22:10:48.237 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.233+0000 7f7e64854980 1 > prepared multipart upload key=0B2V0MHWSOC9 upload=AH0T87R5OWWW tag=X7TK01M9JBX2 type=write size=2836177 2026-02-24T22:10:48.237 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.237+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~S43H5B20K1P1.0 size=1048576 2026-02-24T22:10:48.238 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.237+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~S43H5B20K1P1.1 size=562243 2026-02-24T22:10:48.239 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.237+0000 7f7e64854980 1 < completed multipart upload key=0B2V0MHWSOC9 upload=S43H5B20K1P1 tag=S0LI3AINYNGM type=write size=2659395 2026-02-24T22:10:48.240 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.237+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~K94TFEFGIOAE.0 size=551291 2026-02-24T22:10:48.240 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.237+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=K94TFEFGIOAE tag=VS2SL5H411GC type=write size=1599867 2026-02-24T22:10:48.241 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.237+0000 7f7e64854980 1 < completed delete operation key=PKRHWM69GBJI tag=3M5XTZJLCZVK type=del 2026-02-24T22:10:48.241 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.241+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=P9GU818YXIFX type=write size=653251 2026-02-24T22:10:48.242 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.241+0000 7f7e64854980 1 < completed delete operation key=A6PH77Q78VEN tag=E5K8TOFQYXBT type=del 2026-02-24T22:10:48.243 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.241+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=QL84UKCWDFDR type=del size=3602256 2026-02-24T22:10:48.243 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.241+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~4PPWXSU3D3ON.0 size=1048576 2026-02-24T22:10:48.244 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.241+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~4PPWXSU3D3ON.1 size=843996 2026-02-24T22:10:48.244 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.241+0000 7f7e64854980 1 < completed multipart upload key=Y3L3CJ51C1WR upload=4PPWXSU3D3ON tag=XB1USUUPJBF4 type=write size=2941148 2026-02-24T22:10:48.245 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.241+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=I6QBVJJSMISP type=del size=1300756 2026-02-24T22:10:48.246 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.245+0000 7f7e64854980 1 < canceled operation key=LSRPN33Z2B9N tag=M8ZX7L77LCGA type=del size=2429105 2026-02-24T22:10:48.246 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.245+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=M7ELN0P73NDP type=del size=1333158 2026-02-24T22:10:48.248 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.245+0000 7f7e64854980 1 < completed delete operation key=ZESKGBWZXM6O tag=C03HU5FX5NX9 type=del 2026-02-24T22:10:48.248 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.245+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=P67MLIMOSGG9 type=del size=2858850 2026-02-24T22:10:48.249 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.245+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~0ZUNXPPDFM3Q.0 size=1048576 2026-02-24T22:10:48.249 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.249+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~0ZUNXPPDFM3Q.1 size=1048576 2026-02-24T22:10:48.250 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.249+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~0ZUNXPPDFM3Q.2 size=683606 2026-02-24T22:10:48.251 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.249+0000 7f7e64854980 1 < completed multipart upload key=3900SFJK1YQU upload=0ZUNXPPDFM3Q tag=GCY89VG6D0PN type=write size=3829334 2026-02-24T22:10:48.251 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.249+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=2HDE6QS8LLQ0 type=del size=732867 2026-02-24T22:10:48.252 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.249+0000 7f7e64854980 1 < completed delete operation key=HKOQRSF6XEXB tag=KQLWH81IMJOV type=del 2026-02-24T22:10:48.253 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.249+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=OZ9GW2CZZIA0 type=write size=486320 2026-02-24T22:10:48.253 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.249+0000 7f7e64854980 1 < completed delete operation key=3900SFJK1YQU tag=O8ROIK111LV9 type=del 2026-02-24T22:10:48.254 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.253+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=4X1B1UTCBRFS type=write size=427902 2026-02-24T22:10:48.255 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.253+0000 7f7e64854980 1 < suggested remove operation key=UQLUT9ZAI61V tag=53EL5LVSYPR6 type=write 2026-02-24T22:10:48.256 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.253+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~QEVK8KDQ3213.0 size=1048576 2026-02-24T22:10:48.256 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.253+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~QEVK8KDQ3213.1 size=1048576 2026-02-24T22:10:48.257 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.253+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~QEVK8KDQ3213.2 size=60239 2026-02-24T22:10:48.257 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.253+0000 7f7e64854980 1 > prepared multipart upload key=Q7BP8FXTARDW upload=QEVK8KDQ3213 tag=MDVUX8BZBPW9 type=write size=3205967 2026-02-24T22:10:48.258 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.257+0000 7f7e64854980 1 < completed delete operation key=ON16L53JUSTW tag=CN7S8319OJ10 type=del 2026-02-24T22:10:48.259 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.257+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~UEZTSY7YDZPH.0 size=795390 2026-02-24T22:10:48.259 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.257+0000 7f7e64854980 1 > prepared multipart upload key=3900SFJK1YQU upload=UEZTSY7YDZPH tag=UDW9HAMCUX3Y type=write size=1843966 2026-02-24T22:10:48.259 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.257+0000 7f7e64854980 1 < completed delete operation key=TXRUQQ456DD8 tag=SMO0A5RGTWIP type=del 2026-02-24T22:10:48.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.257+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~3GLF0KGRSF5P.0 size=1048576 2026-02-24T22:10:48.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.257+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~3GLF0KGRSF5P.1 size=1048576 2026-02-24T22:10:48.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.261+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~3GLF0KGRSF5P.2 size=797951 2026-02-24T22:10:48.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.261+0000 7f7e64854980 1 > prepared multipart upload key=Q7BP8FXTARDW upload=3GLF0KGRSF5P tag=9HWCH1OU5IOY type=write size=3943679 2026-02-24T22:10:48.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.261+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~Q4KVCXKAJV1A.0 size=1036956 2026-02-24T22:10:48.263 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.261+0000 7f7e64854980 1 < completed multipart upload key=36MKR2OZN7K9 upload=Q4KVCXKAJV1A tag=7JFGA989R5IQ type=write size=2085532 2026-02-24T22:10:48.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.261+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=E3MS5ALD7TZN type=del size=3969730 2026-02-24T22:10:48.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.261+0000 7f7e64854980 1 < canceled part key=_multipart_HKOQRSF6XEXB.2~XBQMXVSMSO4K.0 size=675775 2026-02-24T22:10:48.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.261+0000 7f7e64854980 1 < canceled multipart upload key=HKOQRSF6XEXB upload=XBQMXVSMSO4K tag=DCBY79X15FWP type=write size=1724351 2026-02-24T22:10:48.265 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.265+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~KPGOYCGWY3A0.0 size=520601 2026-02-24T22:10:48.266 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.265+0000 7f7e64854980 1 > prepared multipart upload key=3900SFJK1YQU upload=KPGOYCGWY3A0 tag=7SJ56Q6XX1FH type=write size=1569177 2026-02-24T22:10:48.266 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.265+0000 7f7e64854980 1 < completed delete operation key=3900SFJK1YQU tag=0S6AVP54CA2U type=del 2026-02-24T22:10:48.267 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.265+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~LY8L6BJWCS0L.0 size=1048576 2026-02-24T22:10:48.268 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.265+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~LY8L6BJWCS0L.1 size=691493 2026-02-24T22:10:48.268 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.265+0000 7f7e64854980 1 > prepared multipart upload key=A6PH77Q78VEN upload=LY8L6BJWCS0L tag=13R58JLJ7CC9 type=write size=2788645 2026-02-24T22:10:48.268 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.265+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~AH0T87R5OWWW.0 size=1048576 2026-02-24T22:10:48.269 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.265+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~AH0T87R5OWWW.1 size=739025 2026-02-24T22:10:48.270 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.269+0000 7f7e64854980 1 < completed multipart upload key=0B2V0MHWSOC9 upload=AH0T87R5OWWW tag=X7TK01M9JBX2 type=write size=2836177 2026-02-24T22:10:48.270 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.269+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=MQRNQUZVF1W7 type=del size=1187456 2026-02-24T22:10:48.271 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.269+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~K94TFEFGIOAE.0 size=551291 2026-02-24T22:10:48.272 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.269+0000 7f7e64854980 1 < completed multipart upload key=CI8DA17DX8LU upload=K94TFEFGIOAE tag=VS2SL5H411GC type=write size=1599867 2026-02-24T22:10:48.272 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.269+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=7RFHX1YHZ9T1 type=write size=445124 2026-02-24T22:10:48.273 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.273+0000 7f7e64854980 1 < suggested update operation key=R2893NES3WSF tag=P9GU818YXIFX type=write size=653251 2026-02-24T22:10:48.274 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.273+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=I696WK2XYC25 type=del size=698297 2026-02-24T22:10:48.275 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.273+0000 7f7e64854980 -1 < failed to complete operation key=UQLUT9ZAI61V tag=QL84UKCWDFDR type=del size=3602256: (22) Invalid argument 2026-02-24T22:10:48.275 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.273+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=PUK08H0WF1CG type=del size=3098538 2026-02-24T22:10:48.276 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.273+0000 7f7e64854980 1 < completed delete operation key=A6PH77Q78VEN tag=I6QBVJJSMISP type=del 2026-02-24T22:10:48.277 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.273+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=29ERODOPJ9SB type=del size=3192341 2026-02-24T22:10:48.277 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.277+0000 7f7e64854980 1 < canceled operation key=GH7723GNI2WM tag=M7ELN0P73NDP type=del size=1333158 2026-02-24T22:10:48.278 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.277+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=XK292L5C5063 type=del size=2334308 2026-02-24T22:10:48.278 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.277+0000 7f7e64854980 1 < completed delete operation key=Y3L3CJ51C1WR tag=P67MLIMOSGG9 type=del 2026-02-24T22:10:48.280 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.277+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~0L4AQARB8MND.0 size=1048576 2026-02-24T22:10:48.280 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.277+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~0L4AQARB8MND.1 size=501500 2026-02-24T22:10:48.280 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.277+0000 7f7e64854980 1 > prepared multipart upload key=GH7723GNI2WM upload=0L4AQARB8MND tag=A5TXHXP8D1R3 type=write size=2598652 2026-02-24T22:10:48.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.277+0000 7f7e64854980 1 < completed delete operation key=EJCX06TE5RTZ tag=2HDE6QS8LLQ0 type=del 2026-02-24T22:10:48.282 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.281+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~CZYFOOYGB3E3.0 size=1048576 2026-02-24T22:10:48.282 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.281+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~CZYFOOYGB3E3.1 size=1048576 2026-02-24T22:10:48.283 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.281+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~CZYFOOYGB3E3.2 size=620947 2026-02-24T22:10:48.283 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.281+0000 7f7e64854980 1 > prepared multipart upload key=SGQYQ2TIVJPS upload=CZYFOOYGB3E3 tag=WZC56CJL7PUM type=write size=3766675 2026-02-24T22:10:48.283 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.281+0000 7f7e64854980 1 < completed write operation key=N3ZUNSI5K8HS tag=OZ9GW2CZZIA0 type=write size=486320 2026-02-24T22:10:48.284 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.281+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~XKCIPL505Z5T.0 size=1048576 2026-02-24T22:10:48.285 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.281+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~XKCIPL505Z5T.1 size=1048576 2026-02-24T22:10:48.285 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.285+0000 7f7e64854980 1 > prepared part key=_multipart_4Y3974PYYT78.2~XKCIPL505Z5T.2 size=853861 2026-02-24T22:10:48.285 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.285+0000 7f7e64854980 1 > prepared multipart upload key=4Y3974PYYT78 upload=XKCIPL505Z5T tag=PZRPBPOE08PR type=write size=3999589 2026-02-24T22:10:48.286 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.285+0000 7f7e64854980 1 < completed write operation key=78GUTC5THI2P tag=4X1B1UTCBRFS type=write size=427902 2026-02-24T22:10:48.287 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.285+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~7OYR9RTC3ZOZ.0 size=1048576 2026-02-24T22:10:48.287 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.285+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~7OYR9RTC3ZOZ.1 size=1048576 2026-02-24T22:10:48.288 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.285+0000 7f7e64854980 1 > prepared part key=_multipart_N3ZUNSI5K8HS.2~7OYR9RTC3ZOZ.2 size=406717 2026-02-24T22:10:48.288 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.285+0000 7f7e64854980 1 > prepared multipart upload key=N3ZUNSI5K8HS upload=7OYR9RTC3ZOZ tag=X0889PBMQPMG type=write size=3552445 2026-02-24T22:10:48.288 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.285+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~QEVK8KDQ3213.0 size=1048576 2026-02-24T22:10:48.289 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.285+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~QEVK8KDQ3213.1 size=1048576 2026-02-24T22:10:48.290 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.289+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~QEVK8KDQ3213.2 size=60239 2026-02-24T22:10:48.291 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.289+0000 7f7e64854980 1 < completed multipart upload key=Q7BP8FXTARDW upload=QEVK8KDQ3213 tag=MDVUX8BZBPW9 type=write size=3205967 2026-02-24T22:10:48.292 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.289+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~GCFS4ZCCLWXW.0 size=1048576 2026-02-24T22:10:48.292 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.289+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~GCFS4ZCCLWXW.1 size=729220 2026-02-24T22:10:48.292 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.289+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=GCFS4ZCCLWXW tag=2LNM5GJX39EE type=write size=2826372 2026-02-24T22:10:48.293 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.289+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~UEZTSY7YDZPH.0 size=795390 2026-02-24T22:10:48.294 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.293+0000 7f7e64854980 1 < completed multipart upload key=3900SFJK1YQU upload=UEZTSY7YDZPH tag=UDW9HAMCUX3Y type=write size=1843966 2026-02-24T22:10:48.294 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.293+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=AXWEGQO0DL1N type=del size=2788075 2026-02-24T22:10:48.295 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.293+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~3GLF0KGRSF5P.0 size=1048576 2026-02-24T22:10:48.295 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.293+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~3GLF0KGRSF5P.1 size=1048576 2026-02-24T22:10:48.296 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.293+0000 7f7e64854980 1 < completed part key=_multipart_Q7BP8FXTARDW.2~3GLF0KGRSF5P.2 size=797951 2026-02-24T22:10:48.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.293+0000 7f7e64854980 1 < completed multipart upload key=Q7BP8FXTARDW upload=3GLF0KGRSF5P tag=9HWCH1OU5IOY type=write size=3943679 2026-02-24T22:10:48.298 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.297+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~R67967X02M4Y.0 size=1048576 2026-02-24T22:10:48.298 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.297+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~R67967X02M4Y.1 size=433848 2026-02-24T22:10:48.299 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.297+0000 7f7e64854980 1 > prepared multipart upload key=LSRPN33Z2B9N upload=R67967X02M4Y tag=BOVCG7UJWDPG type=write size=2531000 2026-02-24T22:10:48.300 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.297+0000 7f7e64854980 1 < suggested update operation key=TXRUQQ456DD8 tag=E3MS5ALD7TZN type=del size=3969730 2026-02-24T22:10:48.300 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.297+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=2C9KPQ78R42H type=del size=3448125 2026-02-24T22:10:48.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.297+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~KPGOYCGWY3A0.0 size=520601 2026-02-24T22:10:48.303 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.301+0000 7f7e64854980 1 < completed multipart upload key=3900SFJK1YQU upload=KPGOYCGWY3A0 tag=7SJ56Q6XX1FH type=write size=1569177 2026-02-24T22:10:48.304 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.301+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=NKJQ733CV638 type=del size=437354 2026-02-24T22:10:48.304 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.301+0000 7f7e64854980 1 < canceled part key=_multipart_A6PH77Q78VEN.2~LY8L6BJWCS0L.0 size=1048576 2026-02-24T22:10:48.304 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.301+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~LY8L6BJWCS0L.1 size=691493 2026-02-24T22:10:48.305 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.301+0000 7f7e64854980 1 < canceled multipart upload key=A6PH77Q78VEN upload=LY8L6BJWCS0L tag=13R58JLJ7CC9 type=write size=2788645 2026-02-24T22:10:48.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.305+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=3U1OJ2DEETIZ type=del size=1261279 2026-02-24T22:10:48.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.305+0000 7f7e64854980 1 < suggested update operation key=NL60Q5QOSP8E tag=MQRNQUZVF1W7 type=del size=1187456 2026-02-24T22:10:48.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.305+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~4BNNWZB617WC.0 size=1048576 2026-02-24T22:10:48.309 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.305+0000 7f7e64854980 1 > prepared part key=_multipart_SGQYQ2TIVJPS.2~4BNNWZB617WC.1 size=951403 2026-02-24T22:10:48.309 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.305+0000 7f7e64854980 1 > prepared multipart upload key=SGQYQ2TIVJPS upload=4BNNWZB617WC tag=4JLF7ZV9VVBH type=write size=3048555 2026-02-24T22:10:48.309 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.309+0000 7f7e64854980 1 < canceled operation key=UQLUT9ZAI61V tag=7RFHX1YHZ9T1 type=write size=445124 2026-02-24T22:10:48.310 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.309+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=9R0PT75ZU9X6 type=del size=3226594 2026-02-24T22:10:48.311 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.309+0000 7f7e64854980 -1 < failed to complete operation key=TXRUQQ456DD8 tag=I696WK2XYC25 type=del size=698297: (22) Invalid argument 2026-02-24T22:10:48.312 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.309+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~JHLWAP0R2FED.0 size=39414 2026-02-24T22:10:48.312 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.309+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=JHLWAP0R2FED tag=7MW9IKC515OA type=write size=1087990 2026-02-24T22:10:48.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.309+0000 7f7e64854980 1 < suggested update operation key=78GUTC5THI2P tag=PUK08H0WF1CG type=del size=3098538 2026-02-24T22:10:48.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.313+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=5X3VYWCBWCVE type=del size=1803197 2026-02-24T22:10:48.314 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.313+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=29ERODOPJ9SB type=del 2026-02-24T22:10:48.315 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.313+0000 7f7e64854980 1 > prepared part key=_multipart_GH7723GNI2WM.2~L2BI1LVL2PJG.0 size=669337 2026-02-24T22:10:48.315 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.313+0000 7f7e64854980 1 > prepared multipart upload key=GH7723GNI2WM upload=L2BI1LVL2PJG tag=ZT0NVMLHB27S type=write size=1717913 2026-02-24T22:10:48.316 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.313+0000 7f7e64854980 1 < completed delete operation key=PKRHWM69GBJI tag=XK292L5C5063 type=del 2026-02-24T22:10:48.317 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.313+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~2RGFRR08Z231.0 size=800536 2026-02-24T22:10:48.317 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.313+0000 7f7e64854980 1 > prepared multipart upload key=PKRHWM69GBJI upload=2RGFRR08Z231 tag=GULWRDHZXYBB type=write size=1849112 2026-02-24T22:10:48.318 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.317+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~0L4AQARB8MND.0 size=1048576 2026-02-24T22:10:48.318 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.317+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~0L4AQARB8MND.1 size=501500 2026-02-24T22:10:48.319 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.317+0000 7f7e64854980 1 < completed multipart upload key=GH7723GNI2WM upload=0L4AQARB8MND tag=A5TXHXP8D1R3 type=write size=2598652 2026-02-24T22:10:48.320 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.317+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~UPIU1CPVK78X.0 size=1048576 2026-02-24T22:10:48.321 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.317+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~UPIU1CPVK78X.1 size=600564 2026-02-24T22:10:48.321 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.317+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=UPIU1CPVK78X tag=NSJ0RQBWZFHW type=write size=2697716 2026-02-24T22:10:48.321 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.321+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~CZYFOOYGB3E3.0 size=1048576 2026-02-24T22:10:48.322 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.321+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~CZYFOOYGB3E3.1 size=1048576 2026-02-24T22:10:48.322 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.321+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~CZYFOOYGB3E3.2 size=620947 2026-02-24T22:10:48.323 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.321+0000 7f7e64854980 1 < completed multipart upload key=SGQYQ2TIVJPS upload=CZYFOOYGB3E3 tag=WZC56CJL7PUM type=write size=3766675 2026-02-24T22:10:48.324 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.321+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=D3NJFZNLPPW0 type=write size=256579 2026-02-24T22:10:48.324 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.321+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~XKCIPL505Z5T.0 size=1048576 2026-02-24T22:10:48.325 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.321+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~XKCIPL505Z5T.1 size=1048576 2026-02-24T22:10:48.325 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.325+0000 7f7e64854980 1 < completed part key=_multipart_4Y3974PYYT78.2~XKCIPL505Z5T.2 size=853861 2026-02-24T22:10:48.326 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.325+0000 7f7e64854980 1 < completed multipart upload key=4Y3974PYYT78 upload=XKCIPL505Z5T tag=PZRPBPOE08PR type=write size=3999589 2026-02-24T22:10:48.327 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.325+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=M2EQTTH23RYU type=del size=2122656 2026-02-24T22:10:48.327 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.325+0000 7f7e64854980 1 < canceled part key=_multipart_N3ZUNSI5K8HS.2~7OYR9RTC3ZOZ.0 size=1048576 2026-02-24T22:10:48.327 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.325+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~7OYR9RTC3ZOZ.1 size=1048576 2026-02-24T22:10:48.328 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.325+0000 7f7e64854980 1 < completed part key=_multipart_N3ZUNSI5K8HS.2~7OYR9RTC3ZOZ.2 size=406717 2026-02-24T22:10:48.329 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.325+0000 7f7e64854980 1 < canceled multipart upload key=N3ZUNSI5K8HS upload=7OYR9RTC3ZOZ tag=X0889PBMQPMG type=write size=3552445 2026-02-24T22:10:48.330 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.329+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~HO8GLRC33T9X.0 size=181293 2026-02-24T22:10:48.330 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.329+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=HO8GLRC33T9X tag=CCK6BOYOY1GW type=write size=1229869 2026-02-24T22:10:48.331 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.329+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~GCFS4ZCCLWXW.0 size=1048576 2026-02-24T22:10:48.331 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.329+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~GCFS4ZCCLWXW.1 size=729220 2026-02-24T22:10:48.332 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.329+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=GCFS4ZCCLWXW tag=2LNM5GJX39EE type=write size=2826372 2026-02-24T22:10:48.333 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.329+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~WJ9U0SV4FHLN.0 size=80728 2026-02-24T22:10:48.333 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.329+0000 7f7e64854980 1 > prepared multipart upload key=EB04ETNYM087 upload=WJ9U0SV4FHLN tag=3KOPFZM0ZJWI type=write size=1129304 2026-02-24T22:10:48.334 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.333+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=AXWEGQO0DL1N type=del 2026-02-24T22:10:48.335 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.333+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~WA1XOP8EBBEA.0 size=1048576 2026-02-24T22:10:48.335 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.333+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~WA1XOP8EBBEA.1 size=227913 2026-02-24T22:10:48.335 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.333+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=WA1XOP8EBBEA tag=HO5KMK4X6N7B type=write size=2325065 2026-02-24T22:10:48.336 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.333+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~R67967X02M4Y.0 size=1048576 2026-02-24T22:10:48.336 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.333+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~R67967X02M4Y.1 size=433848 2026-02-24T22:10:48.337 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.333+0000 7f7e64854980 1 < completed multipart upload key=LSRPN33Z2B9N upload=R67967X02M4Y tag=BOVCG7UJWDPG type=write size=2531000 2026-02-24T22:10:48.338 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.337+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~A4OS50T4YGZI.0 size=1048576 2026-02-24T22:10:48.339 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.337+0000 7f7e64854980 1 > prepared part key=_multipart_R2893NES3WSF.2~A4OS50T4YGZI.1 size=889549 2026-02-24T22:10:48.339 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.337+0000 7f7e64854980 1 > prepared multipart upload key=R2893NES3WSF upload=A4OS50T4YGZI tag=JSMYBBC2SDLE type=write size=2986701 2026-02-24T22:10:48.339 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.337+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=2C9KPQ78R42H type=del 2026-02-24T22:10:48.340 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.337+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=Z0K9H25TE3BO type=del size=293954 2026-02-24T22:10:48.340 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.337+0000 7f7e64854980 1 < completed delete operation key=Z0OXREAUJVA8 tag=NKJQ733CV638 type=del 2026-02-24T22:10:48.341 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.337+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=2JF9TM0H164R type=del size=1904068 2026-02-24T22:10:48.341 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.341+0000 7f7e64854980 1 < completed delete operation key=N3ZUNSI5K8HS tag=3U1OJ2DEETIZ type=del 2026-02-24T22:10:48.342 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.341+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~ZNZ0HM0779G0.0 size=591866 2026-02-24T22:10:48.342 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.341+0000 7f7e64854980 1 > prepared multipart upload key=78GUTC5THI2P upload=ZNZ0HM0779G0 tag=Y25ZAFPX022K type=write size=1640442 2026-02-24T22:10:48.343 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.341+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~4BNNWZB617WC.0 size=1048576 2026-02-24T22:10:48.343 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.341+0000 7f7e64854980 1 < completed part key=_multipart_SGQYQ2TIVJPS.2~4BNNWZB617WC.1 size=951403 2026-02-24T22:10:48.344 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.341+0000 7f7e64854980 1 < completed multipart upload key=SGQYQ2TIVJPS upload=4BNNWZB617WC tag=4JLF7ZV9VVBH type=write size=3048555 2026-02-24T22:10:48.345 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.341+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=D9APHNN43SV0 type=del size=2591135 2026-02-24T22:10:48.345 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.345+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=9R0PT75ZU9X6 type=del 2026-02-24T22:10:48.346 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.345+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=NNA7PGYIPW8Z type=del size=3861170 2026-02-24T22:10:48.347 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.345+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~JHLWAP0R2FED.0 size=39414 2026-02-24T22:10:48.348 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.345+0000 7f7e64854980 1 < completed multipart upload key=D0MKGNRDW6L9 upload=JHLWAP0R2FED tag=7MW9IKC515OA type=write size=1087990 2026-02-24T22:10:48.349 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.345+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~OEE7S5E4N1KS.0 size=1048576 2026-02-24T22:10:48.349 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.349+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~OEE7S5E4N1KS.1 size=328741 2026-02-24T22:10:48.349 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.349+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=OEE7S5E4N1KS tag=XO8CRP6FCQ8X type=write size=2425893 2026-02-24T22:10:48.350 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.349+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=5X3VYWCBWCVE type=del 2026-02-24T22:10:48.350 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.349+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=YV8WJPLKZK5H type=del size=703457 2026-02-24T22:10:48.351 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.349+0000 7f7e64854980 1 < completed part key=_multipart_GH7723GNI2WM.2~L2BI1LVL2PJG.0 size=669337 2026-02-24T22:10:48.352 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.349+0000 7f7e64854980 1 < completed multipart upload key=GH7723GNI2WM upload=L2BI1LVL2PJG tag=ZT0NVMLHB27S type=write size=1717913 2026-02-24T22:10:48.353 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.349+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=4RCWIVBZGFTW type=del size=438356 2026-02-24T22:10:48.353 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.349+0000 7f7e64854980 1 < completed part key=_multipart_PKRHWM69GBJI.2~2RGFRR08Z231.0 size=800536 2026-02-24T22:10:48.354 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.353+0000 7f7e64854980 1 < completed multipart upload key=PKRHWM69GBJI upload=2RGFRR08Z231 tag=GULWRDHZXYBB type=write size=1849112 2026-02-24T22:10:48.355 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.353+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=CIEATYXY1TW0 type=del size=2344953 2026-02-24T22:10:48.355 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.353+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~UPIU1CPVK78X.0 size=1048576 2026-02-24T22:10:48.356 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.353+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~UPIU1CPVK78X.1 size=600564 2026-02-24T22:10:48.357 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.357+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=UPIU1CPVK78X tag=NSJ0RQBWZFHW type=write size=2697716 2026-02-24T22:10:48.358 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.357+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=TM6UUNNSVGI3 type=del size=3631361 2026-02-24T22:10:48.359 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.357+0000 7f7e64854980 1 < completed write operation key=2KKQKHXO7K3K tag=D3NJFZNLPPW0 type=write size=256579 2026-02-24T22:10:48.359 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.357+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=HCHGKJDLTA10 type=del size=4008975 2026-02-24T22:10:48.360 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.357+0000 7f7e64854980 1 < completed delete operation key=UQLUT9ZAI61V tag=M2EQTTH23RYU type=del 2026-02-24T22:10:48.360 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.357+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=O5RQSWKUWVH1 type=del size=3689888 2026-02-24T22:10:48.361 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.357+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~HO8GLRC33T9X.0 size=181293 2026-02-24T22:10:48.362 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.361+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=HO8GLRC33T9X tag=CCK6BOYOY1GW type=write size=1229869 2026-02-24T22:10:48.362 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.361+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=I04NJH31ZKQ2 type=del size=3246767 2026-02-24T22:10:48.363 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.361+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~WJ9U0SV4FHLN.0 size=80728 2026-02-24T22:10:48.364 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.361+0000 7f7e64854980 1 < completed multipart upload key=EB04ETNYM087 upload=WJ9U0SV4FHLN tag=3KOPFZM0ZJWI type=write size=1129304 2026-02-24T22:10:48.364 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.361+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=GD8GXMPQJKIK type=write size=1015871 2026-02-24T22:10:48.365 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.361+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~WA1XOP8EBBEA.0 size=1048576 2026-02-24T22:10:48.365 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.365+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~WA1XOP8EBBEA.1 size=227913 2026-02-24T22:10:48.366 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.365+0000 7f7e64854980 1 < completed multipart upload key=TXRUQQ456DD8 upload=WA1XOP8EBBEA tag=HO5KMK4X6N7B type=write size=2325065 2026-02-24T22:10:48.367 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.365+0000 7f7e64854980 1 > prepared operation key=Q7BP8FXTARDW tag=RC7U0B5FJ4FV type=del size=3050255 2026-02-24T22:10:48.367 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.365+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~A4OS50T4YGZI.0 size=1048576 2026-02-24T22:10:48.368 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.365+0000 7f7e64854980 1 < completed part key=_multipart_R2893NES3WSF.2~A4OS50T4YGZI.1 size=889549 2026-02-24T22:10:48.369 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.365+0000 7f7e64854980 1 < completed multipart upload key=R2893NES3WSF upload=A4OS50T4YGZI tag=JSMYBBC2SDLE type=write size=2986701 2026-02-24T22:10:48.369 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.369+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=TZYF3C7XGGLL type=del size=3841916 2026-02-24T22:10:48.370 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.369+0000 7f7e64854980 1 < completed delete operation key=R2893NES3WSF tag=Z0K9H25TE3BO type=del 2026-02-24T22:10:48.371 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.369+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~PKVAD7EADYPP.0 size=365786 2026-02-24T22:10:48.371 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.369+0000 7f7e64854980 1 > prepared multipart upload key=NL60Q5QOSP8E upload=PKVAD7EADYPP tag=XNM7DFLGQ2WN type=write size=1414362 2026-02-24T22:10:48.372 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.369+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=2JF9TM0H164R type=del 2026-02-24T22:10:48.372 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.369+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=T6O3P0S3OKY8 type=write size=15734 2026-02-24T22:10:48.373 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.369+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~ZNZ0HM0779G0.0 size=591866 2026-02-24T22:10:48.374 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.373+0000 7f7e64854980 1 < completed multipart upload key=78GUTC5THI2P upload=ZNZ0HM0779G0 tag=Y25ZAFPX022K type=write size=1640442 2026-02-24T22:10:48.374 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.373+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=0KJXHXHPSB3G type=del size=27347 2026-02-24T22:10:48.375 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.373+0000 7f7e64854980 1 < completed delete operation key=PKRHWM69GBJI tag=D9APHNN43SV0 type=del 2026-02-24T22:10:48.376 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.373+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~DY8PMQ7WRRED.0 size=1048576 2026-02-24T22:10:48.376 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.373+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~DY8PMQ7WRRED.1 size=829201 2026-02-24T22:10:48.376 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.373+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=DY8PMQ7WRRED tag=V2EO6PEK588O type=write size=2926353 2026-02-24T22:10:48.377 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.377+0000 7f7e64854980 1 < suggested remove operation key=LSRPN33Z2B9N tag=NNA7PGYIPW8Z type=del 2026-02-24T22:10:48.378 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.377+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=AOUXQJW0K8K7 type=write size=860342 2026-02-24T22:10:48.378 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.377+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~OEE7S5E4N1KS.0 size=1048576 2026-02-24T22:10:48.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.377+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~OEE7S5E4N1KS.1 size=328741 2026-02-24T22:10:48.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.377+0000 7f7e64854980 1 < completed multipart upload key=2KKQKHXO7K3K upload=OEE7S5E4N1KS tag=XO8CRP6FCQ8X type=write size=2425893 2026-02-24T22:10:48.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.377+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=GEF8GGER3O0V type=del size=3232356 2026-02-24T22:10:48.381 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.377+0000 7f7e64854980 1 < canceled operation key=78GUTC5THI2P tag=YV8WJPLKZK5H type=del size=703457 2026-02-24T22:10:48.381 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.381+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=SATQB4A0QK60 type=del size=192172 2026-02-24T22:10:48.382 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.381+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=4RCWIVBZGFTW type=del 2026-02-24T22:10:48.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.381+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=PFSMMCHMF906 type=del size=2367101 2026-02-24T22:10:48.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.381+0000 7f7e64854980 1 < completed delete operation key=N3ZUNSI5K8HS tag=CIEATYXY1TW0 type=del 2026-02-24T22:10:48.384 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.381+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=YZ4YBLAKNI5J type=del size=1199234 2026-02-24T22:10:48.384 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.381+0000 7f7e64854980 1 < canceled operation key=L60HQ7M3E3LL tag=TM6UUNNSVGI3 type=del size=3631361 2026-02-24T22:10:48.385 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.381+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=IF1VW2WFY9L2 type=write size=691148 2026-02-24T22:10:48.386 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.385+0000 7f7e64854980 1 < canceled operation key=2KKQKHXO7K3K tag=HCHGKJDLTA10 type=del size=4008975 2026-02-24T22:10:48.386 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.385+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=QWEEUED8AG1B type=del size=2188153 2026-02-24T22:10:48.387 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.385+0000 7f7e64854980 1 < canceled operation key=0AXS7DY23UZU tag=O5RQSWKUWVH1 type=del size=3689888 2026-02-24T22:10:48.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.385+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~T0WTFCC1863Y.0 size=738967 2026-02-24T22:10:48.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.385+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=T0WTFCC1863Y tag=1S2OL32LK23O type=write size=1787543 2026-02-24T22:10:48.389 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.385+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=I04NJH31ZKQ2 type=del 2026-02-24T22:10:48.389 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.385+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=BEDKRR2IA6U6 type=del size=2445682 2026-02-24T22:10:48.390 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.389+0000 7f7e64854980 1 < completed write operation key=HKOQRSF6XEXB tag=GD8GXMPQJKIK type=write size=1015871 2026-02-24T22:10:48.391 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.389+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~AB0NQWWD50J1.0 size=1048576 2026-02-24T22:10:48.391 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.389+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~AB0NQWWD50J1.1 size=1048576 2026-02-24T22:10:48.392 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.389+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~AB0NQWWD50J1.2 size=118189 2026-02-24T22:10:48.392 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.389+0000 7f7e64854980 1 > prepared multipart upload key=A6PH77Q78VEN upload=AB0NQWWD50J1 tag=EM2OV4HJTWK6 type=write size=3263917 2026-02-24T22:10:48.393 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.389+0000 7f7e64854980 1 < suggested remove operation key=Q7BP8FXTARDW tag=RC7U0B5FJ4FV type=del 2026-02-24T22:10:48.394 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.393+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~6GF9107TWYJ9.0 size=13289 2026-02-24T22:10:48.394 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.393+0000 7f7e64854980 1 > prepared multipart upload key=0B2V0MHWSOC9 upload=6GF9107TWYJ9 tag=UR14XD6X33DN type=write size=1061865 2026-02-24T22:10:48.394 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.393+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=TZYF3C7XGGLL type=del 2026-02-24T22:10:48.395 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.393+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=EDVNMSC08QIX type=del size=2567206 2026-02-24T22:10:48.395 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.393+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~PKVAD7EADYPP.0 size=365786 2026-02-24T22:10:48.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.393+0000 7f7e64854980 1 < completed multipart upload key=NL60Q5QOSP8E upload=PKVAD7EADYPP tag=XNM7DFLGQ2WN type=write size=1414362 2026-02-24T22:10:48.397 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.393+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=B6433XTO68GR type=del size=1020963 2026-02-24T22:10:48.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.397+0000 7f7e64854980 1 < suggested update operation key=D0MKGNRDW6L9 tag=T6O3P0S3OKY8 type=write size=15734 2026-02-24T22:10:48.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.397+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~1VAVBC15THLQ.0 size=539753 2026-02-24T22:10:48.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.397+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=1VAVBC15THLQ tag=ISSJCDN51GR0 type=write size=1588329 2026-02-24T22:10:48.400 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.397+0000 7f7e64854980 1 < suggested update operation key=PKRHWM69GBJI tag=0KJXHXHPSB3G type=del size=27347 2026-02-24T22:10:48.401 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.397+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=0CPVJ9N9O9F5 type=write size=1008573 2026-02-24T22:10:48.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.401+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~DY8PMQ7WRRED.0 size=1048576 2026-02-24T22:10:48.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.401+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~DY8PMQ7WRRED.1 size=829201 2026-02-24T22:10:48.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.401+0000 7f7e64854980 1 < completed multipart upload key=TXRUQQ456DD8 upload=DY8PMQ7WRRED tag=V2EO6PEK588O type=write size=2926353 2026-02-24T22:10:48.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.401+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~Z6V0JKBJP4QU.0 size=1048576 2026-02-24T22:10:48.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.401+0000 7f7e64854980 1 > prepared part key=_multipart_UQLUT9ZAI61V.2~Z6V0JKBJP4QU.1 size=578867 2026-02-24T22:10:48.405 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.401+0000 7f7e64854980 1 > prepared multipart upload key=UQLUT9ZAI61V upload=Z6V0JKBJP4QU tag=56CTPY9XCL8N type=write size=2676019 2026-02-24T22:10:48.406 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.405+0000 7f7e64854980 1 < suggested update operation key=NL60Q5QOSP8E tag=AOUXQJW0K8K7 type=write size=860342 2026-02-24T22:10:48.407 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.405+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~9XH3P6W7BKAA.0 size=1021593 2026-02-24T22:10:48.407 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.405+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=9XH3P6W7BKAA tag=FOB6TX6N29PK type=write size=2070169 2026-02-24T22:10:48.408 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.405+0000 7f7e64854980 1 < suggested update operation key=CI8DA17DX8LU tag=GEF8GGER3O0V type=del size=3232356 2026-02-24T22:10:48.408 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.405+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=Q6KF728FSJN3 type=del size=887769 2026-02-24T22:10:48.409 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.405+0000 7f7e64854980 1 < completed delete operation key=0AXS7DY23UZU tag=SATQB4A0QK60 type=del 2026-02-24T22:10:48.410 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.409+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~2AMGD0D612I8.0 size=1048576 2026-02-24T22:10:48.410 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.409+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~2AMGD0D612I8.1 size=1048576 2026-02-24T22:10:48.412 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.409+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~2AMGD0D612I8.2 size=460070 2026-02-24T22:10:48.412 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.409+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=2AMGD0D612I8 tag=7WZVTDYYN475 type=write size=3605798 2026-02-24T22:10:48.412 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.409+0000 7f7e64854980 1 < completed delete operation key=L9NEDH44BIGA tag=PFSMMCHMF906 type=del 2026-02-24T22:10:48.413 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.409+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=3TT65HUGXA8E type=del size=4023634 2026-02-24T22:10:48.413 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.413+0000 7f7e64854980 1 < completed delete operation key=HKOQRSF6XEXB tag=YZ4YBLAKNI5J type=del 2026-02-24T22:10:48.414 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.413+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~H3AVZCLSB842.0 size=1048576 2026-02-24T22:10:48.415 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.413+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~H3AVZCLSB842.1 size=1048576 2026-02-24T22:10:48.415 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.413+0000 7f7e64854980 1 > prepared part key=_multipart_L9NEDH44BIGA.2~H3AVZCLSB842.2 size=238301 2026-02-24T22:10:48.415 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.413+0000 7f7e64854980 1 > prepared multipart upload key=L9NEDH44BIGA upload=H3AVZCLSB842 tag=NWSHASIIPOQF type=write size=3384029 2026-02-24T22:10:48.416 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.413+0000 7f7e64854980 1 < completed write operation key=Y3L3CJ51C1WR tag=IF1VW2WFY9L2 type=write size=691148 2026-02-24T22:10:48.416 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.413+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=SHPH5RG1IR5V type=del size=1368245 2026-02-24T22:10:48.417 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.413+0000 7f7e64854980 1 < completed delete operation key=L60HQ7M3E3LL tag=QWEEUED8AG1B type=del 2026-02-24T22:10:48.418 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.417+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=OW4V6GPGMBC1 type=del size=2381136 2026-02-24T22:10:48.418 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.417+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~T0WTFCC1863Y.0 size=738967 2026-02-24T22:10:48.419 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.417+0000 7f7e64854980 1 < completed multipart upload key=EJCX06TE5RTZ upload=T0WTFCC1863Y tag=1S2OL32LK23O type=write size=1787543 2026-02-24T22:10:48.420 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.417+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=XFBC658SASW3 type=del size=2441393 2026-02-24T22:10:48.420 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.417+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=BEDKRR2IA6U6 type=del 2026-02-24T22:10:48.421 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.417+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=8M6YCWTOPL8O type=del size=3432344 2026-02-24T22:10:48.421 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.421+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~AB0NQWWD50J1.0 size=1048576 2026-02-24T22:10:48.422 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.421+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~AB0NQWWD50J1.1 size=1048576 2026-02-24T22:10:48.422 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.421+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~AB0NQWWD50J1.2 size=118189 2026-02-24T22:10:48.423 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.421+0000 7f7e64854980 1 < completed multipart upload key=A6PH77Q78VEN upload=AB0NQWWD50J1 tag=EM2OV4HJTWK6 type=write size=3263917 2026-02-24T22:10:48.424 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.421+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=WCYFI88Q3XCA type=write size=165666 2026-02-24T22:10:48.424 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.421+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~6GF9107TWYJ9.0 size=13289 2026-02-24T22:10:48.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.425+0000 7f7e64854980 1 < completed multipart upload key=0B2V0MHWSOC9 upload=6GF9107TWYJ9 tag=UR14XD6X33DN type=write size=1061865 2026-02-24T22:10:48.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.425+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=I3X6JCWN7RNI type=del size=2437647 2026-02-24T22:10:48.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.425+0000 7f7e64854980 1 < completed delete operation key=4Y3974PYYT78 tag=EDVNMSC08QIX type=del 2026-02-24T22:10:48.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.425+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=4Y0SYGZ2Z5GF type=write size=141455 2026-02-24T22:10:48.428 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.425+0000 7f7e64854980 1 < completed delete operation key=2KKQKHXO7K3K tag=B6433XTO68GR type=del 2026-02-24T22:10:48.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.425+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~Y0HCZWK3RTFF.0 size=1048576 2026-02-24T22:10:48.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.429+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~Y0HCZWK3RTFF.1 size=1048576 2026-02-24T22:10:48.430 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.429+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~Y0HCZWK3RTFF.2 size=2493 2026-02-24T22:10:48.430 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.429+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=Y0HCZWK3RTFF tag=I9V8ZVIKWF34 type=write size=3148221 2026-02-24T22:10:48.430 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.429+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~1VAVBC15THLQ.0 size=539753 2026-02-24T22:10:48.431 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.429+0000 7f7e64854980 1 < completed multipart upload key=CI8DA17DX8LU upload=1VAVBC15THLQ tag=ISSJCDN51GR0 type=write size=1588329 2026-02-24T22:10:48.432 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.429+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=TGU48NEWKCRT type=del size=3537300 2026-02-24T22:10:48.432 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.429+0000 7f7e64854980 -1 < failed to complete operation key=CI8DA17DX8LU tag=0CPVJ9N9O9F5 type=write size=1008573: (22) Invalid argument 2026-02-24T22:10:48.433 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.429+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=XIOE40XNUY4K type=del size=1839337 2026-02-24T22:10:48.433 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.429+0000 7f7e64854980 1 < canceled part key=_multipart_UQLUT9ZAI61V.2~Z6V0JKBJP4QU.0 size=1048576 2026-02-24T22:10:48.434 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.433+0000 7f7e64854980 1 < completed part key=_multipart_UQLUT9ZAI61V.2~Z6V0JKBJP4QU.1 size=578867 2026-02-24T22:10:48.434 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.433+0000 7f7e64854980 1 < canceled multipart upload key=UQLUT9ZAI61V upload=Z6V0JKBJP4QU tag=56CTPY9XCL8N type=write size=2676019 2026-02-24T22:10:48.435 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.433+0000 7f7e64854980 1 > prepared operation key=R2893NES3WSF tag=7F5KQ6OYA3VN type=write size=493919 2026-02-24T22:10:48.436 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.433+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~9XH3P6W7BKAA.0 size=1021593 2026-02-24T22:10:48.437 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.433+0000 7f7e64854980 1 < completed multipart upload key=ZESKGBWZXM6O upload=9XH3P6W7BKAA tag=FOB6TX6N29PK type=write size=2070169 2026-02-24T22:10:48.437 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.433+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=6T3J4VGROGQV type=write size=950376 2026-02-24T22:10:48.438 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.437+0000 7f7e64854980 1 < completed delete operation key=LSRPN33Z2B9N tag=Q6KF728FSJN3 type=del 2026-02-24T22:10:48.439 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.437+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~0OULBNBUQF5W.0 size=1048576 2026-02-24T22:10:48.439 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.437+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~0OULBNBUQF5W.1 size=1048576 2026-02-24T22:10:48.440 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.437+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~0OULBNBUQF5W.2 size=576188 2026-02-24T22:10:48.440 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.437+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=0OULBNBUQF5W tag=LPMLHXMEHV17 type=write size=3721916 2026-02-24T22:10:48.440 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.437+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~2AMGD0D612I8.0 size=1048576 2026-02-24T22:10:48.440 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.437+0000 7f7e64854980 1 < canceled part key=_multipart_TXRUQQ456DD8.2~2AMGD0D612I8.1 size=1048576 2026-02-24T22:10:48.441 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.437+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~2AMGD0D612I8.2 size=460070 2026-02-24T22:10:48.442 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.441+0000 7f7e64854980 1 < canceled multipart upload key=TXRUQQ456DD8 upload=2AMGD0D612I8 tag=7WZVTDYYN475 type=write size=3605798 2026-02-24T22:10:48.443 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.441+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~QAMETVX3HXDI.0 size=1048576 2026-02-24T22:10:48.443 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.441+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~QAMETVX3HXDI.1 size=1048576 2026-02-24T22:10:48.444 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.441+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~QAMETVX3HXDI.2 size=316129 2026-02-24T22:10:48.444 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.441+0000 7f7e64854980 1 > prepared multipart upload key=Y8GMOUA977H3 upload=QAMETVX3HXDI tag=VUKK58E3YHXD type=write size=3461857 2026-02-24T22:10:48.444 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.441+0000 7f7e64854980 1 < completed delete operation key=SGQYQ2TIVJPS tag=3TT65HUGXA8E type=del 2026-02-24T22:10:48.445 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.441+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~5180LV21TLEM.0 size=1048576 2026-02-24T22:10:48.446 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.445+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~5180LV21TLEM.1 size=1048576 2026-02-24T22:10:48.446 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.445+0000 7f7e64854980 1 > prepared part key=_multipart_EB04ETNYM087.2~5180LV21TLEM.2 size=127704 2026-02-24T22:10:48.446 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.445+0000 7f7e64854980 1 > prepared multipart upload key=EB04ETNYM087 upload=5180LV21TLEM tag=YX627Y1XEE0E type=write size=3273432 2026-02-24T22:10:48.447 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.445+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~H3AVZCLSB842.0 size=1048576 2026-02-24T22:10:48.447 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.445+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~H3AVZCLSB842.1 size=1048576 2026-02-24T22:10:48.448 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.445+0000 7f7e64854980 1 < completed part key=_multipart_L9NEDH44BIGA.2~H3AVZCLSB842.2 size=238301 2026-02-24T22:10:48.449 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.445+0000 7f7e64854980 1 < completed multipart upload key=L9NEDH44BIGA upload=H3AVZCLSB842 tag=NWSHASIIPOQF type=write size=3384029 2026-02-24T22:10:48.449 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.445+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=OA3385MCOZC6 type=write size=603011 2026-02-24T22:10:48.450 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.449+0000 7f7e64854980 1 < completed delete operation key=L9NEDH44BIGA tag=SHPH5RG1IR5V type=del 2026-02-24T22:10:48.451 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.449+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~CY8CSOB3QGTA.0 size=1048576 2026-02-24T22:10:48.451 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.449+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~CY8CSOB3QGTA.1 size=15414 2026-02-24T22:10:48.451 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.449+0000 7f7e64854980 1 > prepared multipart upload key=TF5B43G7KCYT upload=CY8CSOB3QGTA tag=SQ3HR57J17K0 type=write size=2112566 2026-02-24T22:10:48.452 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.449+0000 7f7e64854980 1 < completed delete operation key=0AXS7DY23UZU tag=OW4V6GPGMBC1 type=del 2026-02-24T22:10:48.452 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.449+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=0HZEPICOIAMY type=del size=4124489 2026-02-24T22:10:48.453 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.449+0000 7f7e64854980 1 < completed delete operation key=0B2V0MHWSOC9 tag=XFBC658SASW3 type=del 2026-02-24T22:10:48.454 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.453+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~OBY9GXF0553Y.0 size=532132 2026-02-24T22:10:48.454 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.453+0000 7f7e64854980 1 > prepared multipart upload key=NL60Q5QOSP8E upload=OBY9GXF0553Y tag=ULUKBSWR02H5 type=write size=1580708 2026-02-24T22:10:48.455 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.453+0000 7f7e64854980 1 < completed delete operation key=TXRUQQ456DD8 tag=8M6YCWTOPL8O type=del 2026-02-24T22:10:48.455 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.453+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=KM2S2B31A91T type=del size=1807094 2026-02-24T22:10:48.456 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.453+0000 7f7e64854980 1 < canceled operation key=EB04ETNYM087 tag=WCYFI88Q3XCA type=write size=165666 2026-02-24T22:10:48.456 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.453+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=YKCOAXHXMD72 type=del size=1970005 2026-02-24T22:10:48.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.453+0000 7f7e64854980 1 < completed delete operation key=ZESKGBWZXM6O tag=I3X6JCWN7RNI type=del 2026-02-24T22:10:48.458 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.457+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=L1M21XNDG3HZ type=write size=932787 2026-02-24T22:10:48.458 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.457+0000 7f7e64854980 1 < completed write operation key=2KKQKHXO7K3K tag=4Y0SYGZ2Z5GF type=write size=141455 2026-02-24T22:10:48.459 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.457+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=007B1100AZL4 type=del size=361191 2026-02-24T22:10:48.459 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.457+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~Y0HCZWK3RTFF.0 size=1048576 2026-02-24T22:10:48.460 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.457+0000 7f7e64854980 1 < completed part key=_multipart_D0MKGNRDW6L9.2~Y0HCZWK3RTFF.1 size=1048576 2026-02-24T22:10:48.460 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.457+0000 7f7e64854980 1 < canceled part key=_multipart_D0MKGNRDW6L9.2~Y0HCZWK3RTFF.2 size=2493 2026-02-24T22:10:48.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.457+0000 7f7e64854980 1 < canceled multipart upload key=D0MKGNRDW6L9 upload=Y0HCZWK3RTFF tag=I9V8ZVIKWF34 type=write size=3148221 2026-02-24T22:10:48.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.461+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=V8BSA1C02KA5 type=del size=453101 2026-02-24T22:10:48.462 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.461+0000 7f7e64854980 1 < canceled operation key=ON16L53JUSTW tag=TGU48NEWKCRT type=del size=3537300 2026-02-24T22:10:48.462 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.461+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=5B828232P5VN type=del size=1668954 2026-02-24T22:10:48.463 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.461+0000 7f7e64854980 1 < completed delete operation key=ZESKGBWZXM6O tag=XIOE40XNUY4K type=del 2026-02-24T22:10:48.463 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.461+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=EIC6DCI0WB6L type=del size=3015660 2026-02-24T22:10:48.464 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.461+0000 7f7e64854980 1 < completed write operation key=R2893NES3WSF tag=7F5KQ6OYA3VN type=write size=493919 2026-02-24T22:10:48.466 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.465+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~ZLW80BEFY913.0 size=1007535 2026-02-24T22:10:48.466 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.465+0000 7f7e64854980 1 > prepared multipart upload key=L60HQ7M3E3LL upload=ZLW80BEFY913 tag=TX0QY2AC2ZST type=write size=2056111 2026-02-24T22:10:48.466 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.465+0000 7f7e64854980 1 < completed write operation key=L9NEDH44BIGA tag=6T3J4VGROGQV type=write size=950376 2026-02-24T22:10:48.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.465+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=VPGOJUCHS63N type=del size=806842 2026-02-24T22:10:48.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.465+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~0OULBNBUQF5W.0 size=1048576 2026-02-24T22:10:48.468 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.465+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~0OULBNBUQF5W.1 size=1048576 2026-02-24T22:10:48.468 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.465+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~0OULBNBUQF5W.2 size=576188 2026-02-24T22:10:48.469 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.469+0000 7f7e64854980 1 < completed multipart upload key=0AXS7DY23UZU upload=0OULBNBUQF5W tag=LPMLHXMEHV17 type=write size=3721916 2026-02-24T22:10:48.470 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.469+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=3ISYUTTGWKFX type=del size=2710966 2026-02-24T22:10:48.470 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.469+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~QAMETVX3HXDI.0 size=1048576 2026-02-24T22:10:48.471 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.469+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~QAMETVX3HXDI.1 size=1048576 2026-02-24T22:10:48.471 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.469+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~QAMETVX3HXDI.2 size=316129 2026-02-24T22:10:48.472 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.469+0000 7f7e64854980 1 < completed multipart upload key=Y8GMOUA977H3 upload=QAMETVX3HXDI tag=VUKK58E3YHXD type=write size=3461857 2026-02-24T22:10:48.473 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.469+0000 7f7e64854980 1 > prepared operation key=TXRUQQ456DD8 tag=HZUN3UDHC4GF type=del size=3831173 2026-02-24T22:10:48.473 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.473+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~5180LV21TLEM.0 size=1048576 2026-02-24T22:10:48.474 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.473+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~5180LV21TLEM.1 size=1048576 2026-02-24T22:10:48.474 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.473+0000 7f7e64854980 1 < completed part key=_multipart_EB04ETNYM087.2~5180LV21TLEM.2 size=127704 2026-02-24T22:10:48.476 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.473+0000 7f7e64854980 1 < completed multipart upload key=EB04ETNYM087 upload=5180LV21TLEM tag=YX627Y1XEE0E type=write size=3273432 2026-02-24T22:10:48.476 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.473+0000 7f7e64854980 1 > prepared operation key=ZESKGBWZXM6O tag=Z3BG7G9ALKXT type=del size=1321085 2026-02-24T22:10:48.477 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.473+0000 7f7e64854980 1 < completed write operation key=78GUTC5THI2P tag=OA3385MCOZC6 type=write size=603011 2026-02-24T22:10:48.478 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.477+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~JDNNBXYHSJ7V.0 size=1048576 2026-02-24T22:10:48.478 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.477+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~JDNNBXYHSJ7V.1 size=404746 2026-02-24T22:10:48.478 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.477+0000 7f7e64854980 1 > prepared multipart upload key=Z0OXREAUJVA8 upload=JDNNBXYHSJ7V tag=DH5GF4IGN0RE type=write size=2501898 2026-02-24T22:10:48.479 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.477+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~CY8CSOB3QGTA.0 size=1048576 2026-02-24T22:10:48.479 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.477+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~CY8CSOB3QGTA.1 size=15414 2026-02-24T22:10:48.480 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.477+0000 7f7e64854980 1 < completed multipart upload key=TF5B43G7KCYT upload=CY8CSOB3QGTA tag=SQ3HR57J17K0 type=write size=2112566 2026-02-24T22:10:48.481 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.477+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=5R11UAJOV3Q1 type=del size=2870174 2026-02-24T22:10:48.481 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.477+0000 7f7e64854980 1 < completed delete operation key=ZESKGBWZXM6O tag=0HZEPICOIAMY type=del 2026-02-24T22:10:48.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.481+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~3AH7HVYD0N5B.0 size=1048576 2026-02-24T22:10:48.483 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.481+0000 7f7e64854980 1 > prepared part key=_multipart_NL60Q5QOSP8E.2~3AH7HVYD0N5B.1 size=500390 2026-02-24T22:10:48.483 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.481+0000 7f7e64854980 1 > prepared multipart upload key=NL60Q5QOSP8E upload=3AH7HVYD0N5B tag=403S5JQKOBN7 type=write size=2597542 2026-02-24T22:10:48.483 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.481+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~OBY9GXF0553Y.0 size=532132 2026-02-24T22:10:48.484 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.481+0000 7f7e64854980 1 < completed multipart upload key=NL60Q5QOSP8E upload=OBY9GXF0553Y tag=ULUKBSWR02H5 type=write size=1580708 2026-02-24T22:10:48.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.481+0000 7f7e64854980 1 > prepared part key=_multipart_0B2V0MHWSOC9.2~L5UX7Q040IXC.0 size=815483 2026-02-24T22:10:48.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.481+0000 7f7e64854980 1 > prepared multipart upload key=0B2V0MHWSOC9 upload=L5UX7Q040IXC tag=VLGQA2UP00G1 type=write size=1864059 2026-02-24T22:10:48.486 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.485+0000 7f7e64854980 1 < completed delete operation key=L9NEDH44BIGA tag=KM2S2B31A91T type=del 2026-02-24T22:10:48.486 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.485+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=16IZ2KHSJCST type=del size=2759072 2026-02-24T22:10:48.487 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.485+0000 7f7e64854980 1 < suggested remove operation key=Y3L3CJ51C1WR tag=YKCOAXHXMD72 type=del 2026-02-24T22:10:48.488 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.485+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~W7E3IQLFGCTD.0 size=1048576 2026-02-24T22:10:48.489 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.485+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~W7E3IQLFGCTD.1 size=1048576 2026-02-24T22:10:48.489 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.489+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~W7E3IQLFGCTD.2 size=997632 2026-02-24T22:10:48.489 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.489+0000 7f7e64854980 1 > prepared multipart upload key=Y3L3CJ51C1WR upload=W7E3IQLFGCTD tag=OS86U10OK49O type=write size=4143360 2026-02-24T22:10:48.490 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.489+0000 7f7e64854980 1 < completed write operation key=36MKR2OZN7K9 tag=L1M21XNDG3HZ type=write size=932787 2026-02-24T22:10:48.491 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.489+0000 7f7e64854980 1 > prepared part key=_multipart_TXRUQQ456DD8.2~XQZQ9RQHFRY6.0 size=105470 2026-02-24T22:10:48.491 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.489+0000 7f7e64854980 1 > prepared multipart upload key=TXRUQQ456DD8 upload=XQZQ9RQHFRY6 tag=4AWRVNDMGWMO type=write size=1154046 2026-02-24T22:10:48.491 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.489+0000 7f7e64854980 1 < completed delete operation key=L9NEDH44BIGA tag=007B1100AZL4 type=del 2026-02-24T22:10:48.492 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.489+0000 7f7e64854980 1 > prepared part key=_multipart_Z0OXREAUJVA8.2~FD22V1CTREKW.0 size=335849 2026-02-24T22:10:48.492 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.489+0000 7f7e64854980 1 > prepared multipart upload key=Z0OXREAUJVA8 upload=FD22V1CTREKW tag=FAGV9ZAW4KGS type=write size=1384425 2026-02-24T22:10:48.493 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.489+0000 7f7e64854980 1 < completed delete operation key=0AXS7DY23UZU tag=V8BSA1C02KA5 type=del 2026-02-24T22:10:48.493 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.493+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=XZF7O3L4DHEK type=del size=351188 2026-02-24T22:10:48.494 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.493+0000 7f7e64854980 1 < completed delete operation key=4Y3974PYYT78 tag=5B828232P5VN type=del 2026-02-24T22:10:48.495 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.493+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=BXEHEFRTRY83 type=del size=2885322 2026-02-24T22:10:48.496 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.493+0000 7f7e64854980 1 < suggested update operation key=HOHKR92SN25M tag=EIC6DCI0WB6L type=del size=3015660 2026-02-24T22:10:48.497 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.493+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=SNF97FXJOWDC type=write size=131421 2026-02-24T22:10:48.497 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.497+0000 7f7e64854980 1 < completed part key=_multipart_L60HQ7M3E3LL.2~ZLW80BEFY913.0 size=1007535 2026-02-24T22:10:48.498 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.497+0000 7f7e64854980 1 < completed multipart upload key=L60HQ7M3E3LL upload=ZLW80BEFY913 tag=TX0QY2AC2ZST type=write size=2056111 2026-02-24T22:10:48.500 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.497+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~DGUYF62X0N9A.0 size=1048576 2026-02-24T22:10:48.500 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.497+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~DGUYF62X0N9A.1 size=223620 2026-02-24T22:10:48.500 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.497+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=DGUYF62X0N9A tag=DH3GO2PRN8AJ type=write size=2320772 2026-02-24T22:10:48.501 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.497+0000 7f7e64854980 1 < completed delete operation key=ZESKGBWZXM6O tag=VPGOJUCHS63N type=del 2026-02-24T22:10:48.501 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.501+0000 7f7e64854980 1 > prepared operation key=GH7723GNI2WM tag=RAFDWZLF80VN type=write size=131644 2026-02-24T22:10:48.502 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.501+0000 7f7e64854980 1 < completed delete operation key=Z0OXREAUJVA8 tag=3ISYUTTGWKFX type=del 2026-02-24T22:10:48.503 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.501+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~X2NOJ21VZORX.0 size=621082 2026-02-24T22:10:48.504 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.501+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=X2NOJ21VZORX tag=T1FMRPH3MBDS type=write size=1669658 2026-02-24T22:10:48.504 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.501+0000 7f7e64854980 1 < completed delete operation key=TXRUQQ456DD8 tag=HZUN3UDHC4GF type=del 2026-02-24T22:10:48.505 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.501+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=WNFUFODJFP0K type=del size=1605324 2026-02-24T22:10:48.506 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.505+0000 7f7e64854980 1 < suggested update operation key=ZESKGBWZXM6O tag=Z3BG7G9ALKXT type=del size=1321085 2026-02-24T22:10:48.507 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.505+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~6LJEP3DALC4M.0 size=1048576 2026-02-24T22:10:48.507 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.505+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~6LJEP3DALC4M.1 size=1042083 2026-02-24T22:10:48.507 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.505+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=6LJEP3DALC4M tag=R51K4M788KIY type=write size=3139235 2026-02-24T22:10:48.508 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.505+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~JDNNBXYHSJ7V.0 size=1048576 2026-02-24T22:10:48.508 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.505+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~JDNNBXYHSJ7V.1 size=404746 2026-02-24T22:10:48.509 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.505+0000 7f7e64854980 1 < completed multipart upload key=Z0OXREAUJVA8 upload=JDNNBXYHSJ7V tag=DH5GF4IGN0RE type=write size=2501898 2026-02-24T22:10:48.510 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.509+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~4ADCLLGT43XD.0 size=588857 2026-02-24T22:10:48.510 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.509+0000 7f7e64854980 1 > prepared multipart upload key=ON16L53JUSTW upload=4ADCLLGT43XD tag=KTPNWP3KF85T type=write size=1637433 2026-02-24T22:10:48.511 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.509+0000 7f7e64854980 -1 < failed to complete operation key=HOHKR92SN25M tag=5R11UAJOV3Q1 type=del size=2870174: (22) Invalid argument 2026-02-24T22:10:48.512 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.509+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~JTWF0DDXO2KE.0 size=1048576 2026-02-24T22:10:48.512 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.509+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~JTWF0DDXO2KE.1 size=259478 2026-02-24T22:10:48.512 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.509+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=JTWF0DDXO2KE tag=WD3ZJW9YDFWF type=write size=2356630 2026-02-24T22:10:48.513 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.509+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~3AH7HVYD0N5B.0 size=1048576 2026-02-24T22:10:48.513 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.509+0000 7f7e64854980 1 < completed part key=_multipart_NL60Q5QOSP8E.2~3AH7HVYD0N5B.1 size=500390 2026-02-24T22:10:48.514 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.513+0000 7f7e64854980 1 < completed multipart upload key=NL60Q5QOSP8E upload=3AH7HVYD0N5B tag=403S5JQKOBN7 type=write size=2597542 2026-02-24T22:10:48.515 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.513+0000 7f7e64854980 1 > prepared operation key=Y8GMOUA977H3 tag=SFS6XVYHBNHJ type=del size=3391821 2026-02-24T22:10:48.515 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.513+0000 7f7e64854980 1 < completed part key=_multipart_0B2V0MHWSOC9.2~L5UX7Q040IXC.0 size=815483 2026-02-24T22:10:48.516 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.513+0000 7f7e64854980 1 < completed multipart upload key=0B2V0MHWSOC9 upload=L5UX7Q040IXC tag=VLGQA2UP00G1 type=write size=1864059 2026-02-24T22:10:48.517 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.513+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=4SY3J8JIRSLW type=del size=3252793 2026-02-24T22:10:48.517 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.517+0000 7f7e64854980 1 < completed delete operation key=EB04ETNYM087 tag=16IZ2KHSJCST type=del 2026-02-24T22:10:48.518 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.517+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=N23CN7AIHI0L type=del size=1069788 2026-02-24T22:10:48.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.517+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~W7E3IQLFGCTD.0 size=1048576 2026-02-24T22:10:48.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.517+0000 7f7e64854980 1 < canceled part key=_multipart_Y3L3CJ51C1WR.2~W7E3IQLFGCTD.1 size=1048576 2026-02-24T22:10:48.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.517+0000 7f7e64854980 1 < canceled part key=_multipart_Y3L3CJ51C1WR.2~W7E3IQLFGCTD.2 size=997632 2026-02-24T22:10:48.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.517+0000 7f7e64854980 1 < canceled multipart upload key=Y3L3CJ51C1WR upload=W7E3IQLFGCTD tag=OS86U10OK49O type=write size=4143360 2026-02-24T22:10:48.521 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.517+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~SEVJ4LCJDGWY.0 size=1048576 2026-02-24T22:10:48.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.521+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~SEVJ4LCJDGWY.1 size=777775 2026-02-24T22:10:48.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.521+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=SEVJ4LCJDGWY tag=R79KHSPEZWE7 type=write size=2874927 2026-02-24T22:10:48.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.521+0000 7f7e64854980 1 < completed part key=_multipart_TXRUQQ456DD8.2~XQZQ9RQHFRY6.0 size=105470 2026-02-24T22:10:48.523 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.521+0000 7f7e64854980 1 < completed multipart upload key=TXRUQQ456DD8 upload=XQZQ9RQHFRY6 tag=4AWRVNDMGWMO type=write size=1154046 2026-02-24T22:10:48.524 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.521+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~IC75MM60US0I.0 size=48232 2026-02-24T22:10:48.524 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.521+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=IC75MM60US0I tag=IT6V0F1OMY4E type=write size=1096808 2026-02-24T22:10:48.525 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.521+0000 7f7e64854980 1 < completed part key=_multipart_Z0OXREAUJVA8.2~FD22V1CTREKW.0 size=335849 2026-02-24T22:10:48.526 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.525+0000 7f7e64854980 1 < completed multipart upload key=Z0OXREAUJVA8 upload=FD22V1CTREKW tag=FAGV9ZAW4KGS type=write size=1384425 2026-02-24T22:10:48.526 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.525+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=GRWTT06BU8SF type=del size=1000957 2026-02-24T22:10:48.527 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.525+0000 7f7e64854980 1 < canceled operation key=L9NEDH44BIGA tag=XZF7O3L4DHEK type=del size=351188 2026-02-24T22:10:48.527 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.525+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=AUEHD6DTH0PL type=del size=960493 2026-02-24T22:10:48.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.525+0000 7f7e64854980 1 < suggested remove operation key=ON16L53JUSTW tag=BXEHEFRTRY83 type=del 2026-02-24T22:10:48.529 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.525+0000 7f7e64854980 1 > prepared part key=_multipart_78GUTC5THI2P.2~X4UZV5S5PIRH.0 size=466295 2026-02-24T22:10:48.529 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.525+0000 7f7e64854980 1 > prepared multipart upload key=78GUTC5THI2P upload=X4UZV5S5PIRH tag=K8PYRNLB9NV5 type=write size=1514871 2026-02-24T22:10:48.530 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.529+0000 7f7e64854980 1 < completed write operation key=0AXS7DY23UZU tag=SNF97FXJOWDC type=write size=131421 2026-02-24T22:10:48.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.529+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~JY0M9UT06NAB.0 size=1048576 2026-02-24T22:10:48.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.529+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~JY0M9UT06NAB.1 size=738527 2026-02-24T22:10:48.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.529+0000 7f7e64854980 1 > prepared multipart upload key=A6PH77Q78VEN upload=JY0M9UT06NAB tag=WKACU4CT2HDD type=write size=2835679 2026-02-24T22:10:48.532 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.529+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~DGUYF62X0N9A.0 size=1048576 2026-02-24T22:10:48.532 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.529+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~DGUYF62X0N9A.1 size=223620 2026-02-24T22:10:48.533 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.533+0000 7f7e64854980 1 < completed multipart upload key=ZESKGBWZXM6O upload=DGUYF62X0N9A tag=DH3GO2PRN8AJ type=write size=2320772 2026-02-24T22:10:48.534 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.533+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=M7SDCMLN16UK type=del size=3839738 2026-02-24T22:10:48.535 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.533+0000 7f7e64854980 1 < suggested update operation key=GH7723GNI2WM tag=RAFDWZLF80VN type=write size=131644 2026-02-24T22:10:48.536 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.533+0000 7f7e64854980 1 > prepared part key=_multipart_3900SFJK1YQU.2~KISYUA9L3NPJ.0 size=354329 2026-02-24T22:10:48.536 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.533+0000 7f7e64854980 1 > prepared multipart upload key=3900SFJK1YQU upload=KISYUA9L3NPJ tag=HLGXV811HRZ4 type=write size=1402905 2026-02-24T22:10:48.536 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.533+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~X2NOJ21VZORX.0 size=621082 2026-02-24T22:10:48.537 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.537+0000 7f7e64854980 1 < completed multipart upload key=EJCX06TE5RTZ upload=X2NOJ21VZORX tag=T1FMRPH3MBDS type=write size=1669658 2026-02-24T22:10:48.538 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.537+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=FA7I7DBDTCV7 type=del size=2753779 2026-02-24T22:10:48.539 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.537+0000 7f7e64854980 1 < suggested update operation key=NL60Q5QOSP8E tag=WNFUFODJFP0K type=del size=1605324 2026-02-24T22:10:48.540 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.537+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=INM6OZYPQFK1 type=write size=1027734 2026-02-24T22:10:48.540 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.537+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~6LJEP3DALC4M.0 size=1048576 2026-02-24T22:10:48.540 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.537+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~6LJEP3DALC4M.1 size=1042083 2026-02-24T22:10:48.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.541+0000 7f7e64854980 1 < completed multipart upload key=0AXS7DY23UZU upload=6LJEP3DALC4M tag=R51K4M788KIY type=write size=3139235 2026-02-24T22:10:48.542 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.541+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~6XG69RXY494G.0 size=1048576 2026-02-24T22:10:48.543 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.541+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~6XG69RXY494G.1 size=1048576 2026-02-24T22:10:48.543 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.541+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~6XG69RXY494G.2 size=396517 2026-02-24T22:10:48.543 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.541+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=6XG69RXY494G tag=37DP5EMUNQ9Z type=write size=3542245 2026-02-24T22:10:48.544 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.541+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~4ADCLLGT43XD.0 size=588857 2026-02-24T22:10:48.545 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.541+0000 7f7e64854980 1 < completed multipart upload key=ON16L53JUSTW upload=4ADCLLGT43XD tag=KTPNWP3KF85T type=write size=1637433 2026-02-24T22:10:48.545 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.545+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=N2VJRC0561US type=write size=204809 2026-02-24T22:10:48.546 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.545+0000 7f7e64854980 1 < canceled part key=_multipart_0AXS7DY23UZU.2~JTWF0DDXO2KE.0 size=1048576 2026-02-24T22:10:48.546 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.545+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~JTWF0DDXO2KE.1 size=259478 2026-02-24T22:10:48.547 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.545+0000 7f7e64854980 1 < canceled multipart upload key=0AXS7DY23UZU upload=JTWF0DDXO2KE tag=WD3ZJW9YDFWF type=write size=2356630 2026-02-24T22:10:48.547 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.545+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=JXX5YUUMYJIC type=del size=1102109 2026-02-24T22:10:48.548 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.545+0000 7f7e64854980 1 < completed delete operation key=Y8GMOUA977H3 tag=SFS6XVYHBNHJ type=del 2026-02-24T22:10:48.548 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.545+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=POCRC6MQ3O3J type=del size=1079561 2026-02-24T22:10:48.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.545+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=4SY3J8JIRSLW type=del 2026-02-24T22:10:48.550 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.549+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~5EEB84DX2LNR.0 size=190074 2026-02-24T22:10:48.550 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.549+0000 7f7e64854980 1 > prepared multipart upload key=TF5B43G7KCYT upload=5EEB84DX2LNR tag=VQAL2XL7OH58 type=write size=1238650 2026-02-24T22:10:48.551 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.549+0000 7f7e64854980 1 < completed delete operation key=A6PH77Q78VEN tag=N23CN7AIHI0L type=del 2026-02-24T22:10:48.552 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.549+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~9BKGX833EZFD.0 size=1048576 2026-02-24T22:10:48.552 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.549+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~9BKGX833EZFD.1 size=1048576 2026-02-24T22:10:48.552 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.549+0000 7f7e64854980 1 > prepared part key=_multipart_TF5B43G7KCYT.2~9BKGX833EZFD.2 size=677336 2026-02-24T22:10:48.553 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.549+0000 7f7e64854980 1 > prepared multipart upload key=TF5B43G7KCYT upload=9BKGX833EZFD tag=5Q6KWKP9XIT3 type=write size=3823064 2026-02-24T22:10:48.553 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.549+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~SEVJ4LCJDGWY.0 size=1048576 2026-02-24T22:10:48.553 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.553+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~SEVJ4LCJDGWY.1 size=777775 2026-02-24T22:10:48.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.553+0000 7f7e64854980 1 < completed multipart upload key=ZESKGBWZXM6O upload=SEVJ4LCJDGWY tag=R79KHSPEZWE7 type=write size=2874927 2026-02-24T22:10:48.555 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.553+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~5O7NIFG4ZXT5.0 size=787581 2026-02-24T22:10:48.555 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.553+0000 7f7e64854980 1 > prepared multipart upload key=LSRPN33Z2B9N upload=5O7NIFG4ZXT5 tag=XMRZTRSK0V82 type=write size=1836157 2026-02-24T22:10:48.556 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.553+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~IC75MM60US0I.0 size=48232 2026-02-24T22:10:48.557 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.553+0000 7f7e64854980 1 < completed multipart upload key=EJCX06TE5RTZ upload=IC75MM60US0I tag=IT6V0F1OMY4E type=write size=1096808 2026-02-24T22:10:48.558 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.557+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~PPB9ADVYAKEV.0 size=1048576 2026-02-24T22:10:48.558 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.557+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~PPB9ADVYAKEV.1 size=20083 2026-02-24T22:10:48.558 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.557+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=PPB9ADVYAKEV tag=9ATOVBD72UNP type=write size=2117235 2026-02-24T22:10:48.559 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.557+0000 7f7e64854980 1 < completed delete operation key=L9NEDH44BIGA tag=GRWTT06BU8SF type=del 2026-02-24T22:10:48.559 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.557+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=61NSJTBVUGB5 type=del size=1515712 2026-02-24T22:10:48.560 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.557+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=AUEHD6DTH0PL type=del 2026-02-24T22:10:48.560 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.557+0000 7f7e64854980 1 > prepared operation key=PKRHWM69GBJI tag=1WP68RTH26MZ type=del size=1248601 2026-02-24T22:10:48.561 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.557+0000 7f7e64854980 1 < completed part key=_multipart_78GUTC5THI2P.2~X4UZV5S5PIRH.0 size=466295 2026-02-24T22:10:48.562 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.561+0000 7f7e64854980 1 < completed multipart upload key=78GUTC5THI2P upload=X4UZV5S5PIRH tag=K8PYRNLB9NV5 type=write size=1514871 2026-02-24T22:10:48.563 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.561+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~M2I0EV7XXC8H.0 size=1048576 2026-02-24T22:10:48.564 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.561+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~M2I0EV7XXC8H.1 size=640099 2026-02-24T22:10:48.564 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.561+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=M2I0EV7XXC8H tag=N89TYT51LU49 type=write size=2737251 2026-02-24T22:10:48.564 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.561+0000 7f7e64854980 1 < canceled part key=_multipart_A6PH77Q78VEN.2~JY0M9UT06NAB.0 size=1048576 2026-02-24T22:10:48.564 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.561+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~JY0M9UT06NAB.1 size=738527 2026-02-24T22:10:48.565 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.561+0000 7f7e64854980 1 < canceled multipart upload key=A6PH77Q78VEN upload=JY0M9UT06NAB tag=WKACU4CT2HDD type=write size=2835679 2026-02-24T22:10:48.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.565+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~5WHF0AENI2MY.0 size=1048576 2026-02-24T22:10:48.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.565+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~5WHF0AENI2MY.1 size=1048576 2026-02-24T22:10:48.567 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.565+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~5WHF0AENI2MY.2 size=261762 2026-02-24T22:10:48.567 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.565+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=5WHF0AENI2MY tag=7X615BAJTZ59 type=write size=3407490 2026-02-24T22:10:48.567 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.565+0000 7f7e64854980 1 < completed delete operation key=L9NEDH44BIGA tag=M7SDCMLN16UK type=del 2026-02-24T22:10:48.568 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.565+0000 7f7e64854980 1 > prepared operation key=36MKR2OZN7K9 tag=EPZX7X6HDCF1 type=del size=1182929 2026-02-24T22:10:48.569 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.565+0000 7f7e64854980 1 < completed part key=_multipart_3900SFJK1YQU.2~KISYUA9L3NPJ.0 size=354329 2026-02-24T22:10:48.570 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.569+0000 7f7e64854980 1 < completed multipart upload key=3900SFJK1YQU upload=KISYUA9L3NPJ tag=HLGXV811HRZ4 type=write size=1402905 2026-02-24T22:10:48.571 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.569+0000 7f7e64854980 1 > prepared operation key=L60HQ7M3E3LL tag=6JQ3EONXHP14 type=write size=994870 2026-02-24T22:10:48.571 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.569+0000 7f7e64854980 1 < completed delete operation key=CI8DA17DX8LU tag=FA7I7DBDTCV7 type=del 2026-02-24T22:10:48.572 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.569+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~FJK5DDCAT637.0 size=1048576 2026-02-24T22:10:48.573 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.569+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~FJK5DDCAT637.1 size=780506 2026-02-24T22:10:48.573 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.569+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=FJK5DDCAT637 tag=ZO81IYPTFLC8 type=write size=2877658 2026-02-24T22:10:48.573 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.573+0000 7f7e64854980 1 < completed write operation key=CI8DA17DX8LU tag=INM6OZYPQFK1 type=write size=1027734 2026-02-24T22:10:48.575 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.573+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~AV0TX87FPSG5.0 size=1048576 2026-02-24T22:10:48.575 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.573+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~AV0TX87FPSG5.1 size=755594 2026-02-24T22:10:48.575 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.573+0000 7f7e64854980 1 > prepared multipart upload key=HKOQRSF6XEXB upload=AV0TX87FPSG5 tag=KSW5ALAMLPT6 type=write size=2852746 2026-02-24T22:10:48.576 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.573+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~6XG69RXY494G.0 size=1048576 2026-02-24T22:10:48.576 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.573+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~6XG69RXY494G.1 size=1048576 2026-02-24T22:10:48.577 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.573+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~6XG69RXY494G.2 size=396517 2026-02-24T22:10:48.578 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.577+0000 7f7e64854980 1 < completed multipart upload key=CI8DA17DX8LU upload=6XG69RXY494G tag=37DP5EMUNQ9Z type=write size=3542245 2026-02-24T22:10:48.579 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.577+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=PGCYHBLJMY8K type=del size=306636 2026-02-24T22:10:48.579 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.577+0000 7f7e64854980 1 < completed write operation key=HOHKR92SN25M tag=N2VJRC0561US type=write size=204809 2026-02-24T22:10:48.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.577+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~RK11T3ZLE5D9.0 size=1048576 2026-02-24T22:10:48.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.577+0000 7f7e64854980 1 > prepared part key=_multipart_Y3L3CJ51C1WR.2~RK11T3ZLE5D9.1 size=209632 2026-02-24T22:10:48.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.577+0000 7f7e64854980 1 > prepared multipart upload key=Y3L3CJ51C1WR upload=RK11T3ZLE5D9 tag=0GOUFC9K7IKS type=write size=2306784 2026-02-24T22:10:48.582 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.581+0000 7f7e64854980 1 < completed delete operation key=EJCX06TE5RTZ tag=JXX5YUUMYJIC type=del 2026-02-24T22:10:48.584 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.581+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~DCLOG40WE7RX.0 size=1048576 2026-02-24T22:10:48.584 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.581+0000 7f7e64854980 1 > prepared part key=_multipart_ON16L53JUSTW.2~DCLOG40WE7RX.1 size=755179 2026-02-24T22:10:48.584 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.581+0000 7f7e64854980 1 > prepared multipart upload key=ON16L53JUSTW upload=DCLOG40WE7RX tag=L8ARWV3GTSFD type=write size=2852331 2026-02-24T22:10:48.586 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.585+0000 7f7e64854980 1 < suggested remove operation key=A6PH77Q78VEN tag=POCRC6MQ3O3J type=del 2026-02-24T22:10:48.586 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.585+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=5H1WBKOG1NZL type=del size=954336 2026-02-24T22:10:48.587 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.585+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~5EEB84DX2LNR.0 size=190074 2026-02-24T22:10:48.588 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.585+0000 7f7e64854980 1 < completed multipart upload key=TF5B43G7KCYT upload=5EEB84DX2LNR tag=VQAL2XL7OH58 type=write size=1238650 2026-02-24T22:10:48.590 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.589+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~T7MDNYWLIF7M.0 size=1048576 2026-02-24T22:10:48.591 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.589+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~T7MDNYWLIF7M.1 size=774383 2026-02-24T22:10:48.591 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.589+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=T7MDNYWLIF7M tag=FZHVE35H2X7Y type=write size=2871535 2026-02-24T22:10:48.591 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.589+0000 7f7e64854980 1 < canceled part key=_multipart_TF5B43G7KCYT.2~9BKGX833EZFD.0 size=1048576 2026-02-24T22:10:48.592 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.589+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~9BKGX833EZFD.1 size=1048576 2026-02-24T22:10:48.592 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.589+0000 7f7e64854980 1 < completed part key=_multipart_TF5B43G7KCYT.2~9BKGX833EZFD.2 size=677336 2026-02-24T22:10:48.594 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.593+0000 7f7e64854980 1 < canceled multipart upload key=TF5B43G7KCYT upload=9BKGX833EZFD tag=5Q6KWKP9XIT3 type=write size=3823064 2026-02-24T22:10:48.594 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.593+0000 7f7e64854980 1 > prepared operation key=78GUTC5THI2P tag=LFB682ZJGXE1 type=del size=3039120 2026-02-24T22:10:48.595 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.593+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~5O7NIFG4ZXT5.0 size=787581 2026-02-24T22:10:48.596 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.593+0000 7f7e64854980 1 < completed multipart upload key=LSRPN33Z2B9N upload=5O7NIFG4ZXT5 tag=XMRZTRSK0V82 type=write size=1836157 2026-02-24T22:10:48.597 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.593+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=9N64LCRM0KJ2 type=del size=281035 2026-02-24T22:10:48.598 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.597+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~PPB9ADVYAKEV.0 size=1048576 2026-02-24T22:10:48.598 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.597+0000 7f7e64854980 1 < canceled part key=_multipart_EJCX06TE5RTZ.2~PPB9ADVYAKEV.1 size=20083 2026-02-24T22:10:48.599 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.597+0000 7f7e64854980 1 < canceled multipart upload key=EJCX06TE5RTZ upload=PPB9ADVYAKEV tag=9ATOVBD72UNP type=write size=2117235 2026-02-24T22:10:48.601 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.597+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~KZR6LP3N4PH8.0 size=1048576 2026-02-24T22:10:48.602 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.597+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~KZR6LP3N4PH8.1 size=1048576 2026-02-24T22:10:48.602 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.601+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~KZR6LP3N4PH8.2 size=851151 2026-02-24T22:10:48.602 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.601+0000 7f7e64854980 1 > prepared multipart upload key=HKOQRSF6XEXB upload=KZR6LP3N4PH8 tag=GIIYDEEZ1MD2 type=write size=3996879 2026-02-24T22:10:48.603 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.601+0000 7f7e64854980 1 < completed delete operation key=4Y3974PYYT78 tag=61NSJTBVUGB5 type=del 2026-02-24T22:10:48.604 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.601+0000 7f7e64854980 1 > prepared part key=_multipart_Y8GMOUA977H3.2~Q39BXCEUIB1N.0 size=210027 2026-02-24T22:10:48.604 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.601+0000 7f7e64854980 1 > prepared multipart upload key=Y8GMOUA977H3 upload=Q39BXCEUIB1N tag=UG5AZY3YPZ5N type=write size=1258603 2026-02-24T22:10:48.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.601+0000 7f7e64854980 1 < completed delete operation key=PKRHWM69GBJI tag=1WP68RTH26MZ type=del 2026-02-24T22:10:48.606 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.605+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=PJ493R1WAIN6 type=del size=606554 2026-02-24T22:10:48.607 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.605+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~M2I0EV7XXC8H.0 size=1048576 2026-02-24T22:10:48.607 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.605+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~M2I0EV7XXC8H.1 size=640099 2026-02-24T22:10:48.608 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.605+0000 7f7e64854980 1 < completed multipart upload key=EJCX06TE5RTZ upload=M2I0EV7XXC8H tag=N89TYT51LU49 type=write size=2737251 2026-02-24T22:10:48.609 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.605+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=3O3E97VVSXZS type=del size=3483892 2026-02-24T22:10:48.609 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.605+0000 7f7e64854980 1 < canceled part key=_multipart_ZESKGBWZXM6O.2~5WHF0AENI2MY.0 size=1048576 2026-02-24T22:10:48.610 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.609+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~5WHF0AENI2MY.1 size=1048576 2026-02-24T22:10:48.610 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.609+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~5WHF0AENI2MY.2 size=261762 2026-02-24T22:10:48.612 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.609+0000 7f7e64854980 1 < canceled multipart upload key=ZESKGBWZXM6O upload=5WHF0AENI2MY tag=7X615BAJTZ59 type=write size=3407490 2026-02-24T22:10:48.612 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.609+0000 7f7e64854980 1 > prepared operation key=LSRPN33Z2B9N tag=IAQMXD5FYOTG type=del size=2654187 2026-02-24T22:10:48.613 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.609+0000 7f7e64854980 1 < completed delete operation key=36MKR2OZN7K9 tag=EPZX7X6HDCF1 type=del 2026-02-24T22:10:48.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.613+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~WSV93U781Q7U.0 size=1048576 2026-02-24T22:10:48.615 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.613+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~WSV93U781Q7U.1 size=1048576 2026-02-24T22:10:48.615 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.613+0000 7f7e64854980 1 > prepared part key=_multipart_ZESKGBWZXM6O.2~WSV93U781Q7U.2 size=562148 2026-02-24T22:10:48.615 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.613+0000 7f7e64854980 1 > prepared multipart upload key=ZESKGBWZXM6O upload=WSV93U781Q7U tag=FSEW30YXOLO1 type=write size=3707876 2026-02-24T22:10:48.616 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.613+0000 7f7e64854980 1 < completed write operation key=L60HQ7M3E3LL tag=6JQ3EONXHP14 type=write size=994870 2026-02-24T22:10:48.617 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.617+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~0PMR2X5SMHHG.0 size=1048576 2026-02-24T22:10:48.618 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.617+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~0PMR2X5SMHHG.1 size=1048576 2026-02-24T22:10:48.618 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.617+0000 7f7e64854980 1 > prepared part key=_multipart_CI8DA17DX8LU.2~0PMR2X5SMHHG.2 size=158090 2026-02-24T22:10:48.618 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.617+0000 7f7e64854980 1 > prepared multipart upload key=CI8DA17DX8LU upload=0PMR2X5SMHHG tag=DTDZP9S33XBT type=write size=3303818 2026-02-24T22:10:48.619 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.617+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~FJK5DDCAT637.0 size=1048576 2026-02-24T22:10:48.620 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.617+0000 7f7e64854980 1 < completed part key=_multipart_0AXS7DY23UZU.2~FJK5DDCAT637.1 size=780506 2026-02-24T22:10:48.621 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.617+0000 7f7e64854980 1 < completed multipart upload key=0AXS7DY23UZU upload=FJK5DDCAT637 tag=ZO81IYPTFLC8 type=write size=2877658 2026-02-24T22:10:48.622 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.621+0000 7f7e64854980 1 > prepared operation key=A6PH77Q78VEN tag=FGIC1CGR6BD8 type=write size=980525 2026-02-24T22:10:48.622 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.621+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~AV0TX87FPSG5.0 size=1048576 2026-02-24T22:10:48.622 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.621+0000 7f7e64854980 1 < canceled part key=_multipart_HKOQRSF6XEXB.2~AV0TX87FPSG5.1 size=755594 2026-02-24T22:10:48.624 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.621+0000 7f7e64854980 1 < canceled multipart upload key=HKOQRSF6XEXB upload=AV0TX87FPSG5 tag=KSW5ALAMLPT6 type=write size=2852746 2026-02-24T22:10:48.624 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.621+0000 7f7e64854980 1 > prepared operation key=NL60Q5QOSP8E tag=TSECKBGQZNKE type=del size=1597303 2026-02-24T22:10:48.625 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.621+0000 7f7e64854980 1 < canceled operation key=4Y3974PYYT78 tag=PGCYHBLJMY8K type=del size=306636 2026-02-24T22:10:48.626 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.625+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=L7NW5B892OPT type=del size=2210120 2026-02-24T22:10:48.626 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.625+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~RK11T3ZLE5D9.0 size=1048576 2026-02-24T22:10:48.627 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.625+0000 7f7e64854980 1 < completed part key=_multipart_Y3L3CJ51C1WR.2~RK11T3ZLE5D9.1 size=209632 2026-02-24T22:10:48.628 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.625+0000 7f7e64854980 1 < completed multipart upload key=Y3L3CJ51C1WR upload=RK11T3ZLE5D9 tag=0GOUFC9K7IKS type=write size=2306784 2026-02-24T22:10:48.629 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.625+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=CQKCN82J8JZA type=write size=852693 2026-02-24T22:10:48.630 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.629+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~DCLOG40WE7RX.0 size=1048576 2026-02-24T22:10:48.630 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.629+0000 7f7e64854980 1 < completed part key=_multipart_ON16L53JUSTW.2~DCLOG40WE7RX.1 size=755179 2026-02-24T22:10:48.632 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.629+0000 7f7e64854980 1 < completed multipart upload key=ON16L53JUSTW upload=DCLOG40WE7RX tag=L8ARWV3GTSFD type=write size=2852331 2026-02-24T22:10:48.632 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.629+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=DLX67IHL6CN6 type=del size=2528826 2026-02-24T22:10:48.633 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.629+0000 7f7e64854980 1 < completed delete operation key=Z0OXREAUJVA8 tag=5H1WBKOG1NZL type=del 2026-02-24T22:10:48.634 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.633+0000 7f7e64854980 1 > prepared operation key=L9NEDH44BIGA tag=9VTOFGWE1ZA2 type=del size=114554 2026-02-24T22:10:48.635 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.633+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~T7MDNYWLIF7M.0 size=1048576 2026-02-24T22:10:48.635 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.633+0000 7f7e64854980 1 < completed part key=_multipart_EJCX06TE5RTZ.2~T7MDNYWLIF7M.1 size=774383 2026-02-24T22:10:48.637 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.637+0000 7f7e64854980 1 < completed multipart upload key=EJCX06TE5RTZ upload=T7MDNYWLIF7M tag=FZHVE35H2X7Y type=write size=2871535 2026-02-24T22:10:48.639 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.637+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~1HTT1TMLZ35P.0 size=1048576 2026-02-24T22:10:48.639 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.637+0000 7f7e64854980 1 > prepared part key=_multipart_2KKQKHXO7K3K.2~1HTT1TMLZ35P.1 size=410805 2026-02-24T22:10:48.639 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.637+0000 7f7e64854980 1 > prepared multipart upload key=2KKQKHXO7K3K upload=1HTT1TMLZ35P tag=3XN48D86T3VZ type=write size=2507957 2026-02-24T22:10:48.640 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.637+0000 7f7e64854980 1 < canceled operation key=78GUTC5THI2P tag=LFB682ZJGXE1 type=del size=3039120 2026-02-24T22:10:48.641 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.641+0000 7f7e64854980 1 > prepared part key=_multipart_A6PH77Q78VEN.2~Y42JPWQ124YC.0 size=17968 2026-02-24T22:10:48.641 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.641+0000 7f7e64854980 1 > prepared multipart upload key=A6PH77Q78VEN upload=Y42JPWQ124YC tag=QZFST7W0LF3O type=write size=1066544 2026-02-24T22:10:48.642 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.641+0000 7f7e64854980 1 < completed delete operation key=UQLUT9ZAI61V tag=9N64LCRM0KJ2 type=del 2026-02-24T22:10:48.643 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.641+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~OQQN79RX3SPQ.0 size=1048576 2026-02-24T22:10:48.644 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.641+0000 7f7e64854980 1 > prepared part key=_multipart_LSRPN33Z2B9N.2~OQQN79RX3SPQ.1 size=216865 2026-02-24T22:10:48.644 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.641+0000 7f7e64854980 1 > prepared multipart upload key=LSRPN33Z2B9N upload=OQQN79RX3SPQ tag=HXB3S3JD0MT1 type=write size=2314017 2026-02-24T22:10:48.645 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.641+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~KZR6LP3N4PH8.0 size=1048576 2026-02-24T22:10:48.645 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.641+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~KZR6LP3N4PH8.1 size=1048576 2026-02-24T22:10:48.646 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.645+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~KZR6LP3N4PH8.2 size=851151 2026-02-24T22:10:48.647 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.645+0000 7f7e64854980 1 < completed multipart upload key=HKOQRSF6XEXB upload=KZR6LP3N4PH8 tag=GIIYDEEZ1MD2 type=write size=3996879 2026-02-24T22:10:48.648 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.645+0000 7f7e64854980 1 > prepared operation key=CI8DA17DX8LU tag=Q21N0FVN84JK type=write size=610524 2026-02-24T22:10:48.648 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.645+0000 7f7e64854980 1 < completed part key=_multipart_Y8GMOUA977H3.2~Q39BXCEUIB1N.0 size=210027 2026-02-24T22:10:48.650 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.649+0000 7f7e64854980 1 < completed multipart upload key=Y8GMOUA977H3 upload=Q39BXCEUIB1N tag=UG5AZY3YPZ5N type=write size=1258603 2026-02-24T22:10:48.650 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.649+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=TKL7HN21KNJ1 type=del size=2268311 2026-02-24T22:10:48.651 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.649+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=PJ493R1WAIN6 type=del 2026-02-24T22:10:48.652 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.649+0000 7f7e64854980 1 > prepared operation key=HKOQRSF6XEXB tag=BOJ7J554MK76 type=del size=2099320 2026-02-24T22:10:48.653 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.649+0000 7f7e64854980 1 < completed delete operation key=A6PH77Q78VEN tag=3O3E97VVSXZS type=del 2026-02-24T22:10:48.653 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.653+0000 7f7e64854980 1 > prepared operation key=Y3L3CJ51C1WR tag=IID0S05ANKT7 type=del size=2869573 2026-02-24T22:10:48.655 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.653+0000 7f7e64854980 1 < suggested remove operation key=LSRPN33Z2B9N tag=IAQMXD5FYOTG type=del 2026-02-24T22:10:48.656 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.653+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=6YPRAK5AT7P5 type=del size=3273864 2026-02-24T22:10:48.656 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.653+0000 7f7e64854980 1 < canceled part key=_multipart_ZESKGBWZXM6O.2~WSV93U781Q7U.0 size=1048576 2026-02-24T22:10:48.656 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.653+0000 7f7e64854980 1 < canceled part key=_multipart_ZESKGBWZXM6O.2~WSV93U781Q7U.1 size=1048576 2026-02-24T22:10:48.656 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.653+0000 7f7e64854980 1 < completed part key=_multipart_ZESKGBWZXM6O.2~WSV93U781Q7U.2 size=562148 2026-02-24T22:10:48.657 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.657+0000 7f7e64854980 1 < canceled multipart upload key=ZESKGBWZXM6O upload=WSV93U781Q7U tag=FSEW30YXOLO1 type=write size=3707876 2026-02-24T22:10:48.658 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.657+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=R7VR07QIFQH3 type=del size=1483453 2026-02-24T22:10:48.659 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.657+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~0PMR2X5SMHHG.0 size=1048576 2026-02-24T22:10:48.659 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.657+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~0PMR2X5SMHHG.1 size=1048576 2026-02-24T22:10:48.660 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.657+0000 7f7e64854980 1 < completed part key=_multipart_CI8DA17DX8LU.2~0PMR2X5SMHHG.2 size=158090 2026-02-24T22:10:48.661 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.661+0000 7f7e64854980 1 < completed multipart upload key=CI8DA17DX8LU upload=0PMR2X5SMHHG tag=DTDZP9S33XBT type=write size=3303818 2026-02-24T22:10:48.662 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.661+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=7ERXCL6X4T0P type=del size=3171410 2026-02-24T22:10:48.663 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.661+0000 7f7e64854980 1 < completed write operation key=A6PH77Q78VEN tag=FGIC1CGR6BD8 type=write size=980525 2026-02-24T22:10:48.663 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.661+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=HHR14F20ODBM type=del size=919200 2026-02-24T22:10:48.665 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.661+0000 7f7e64854980 1 < suggested update operation key=NL60Q5QOSP8E tag=TSECKBGQZNKE type=del size=1597303 2026-02-24T22:10:48.666 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.665+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=AJ3JCLNFW8H9 type=del size=672872 2026-02-24T22:10:48.666 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.665+0000 7f7e64854980 1 < completed delete operation key=2KKQKHXO7K3K tag=L7NW5B892OPT type=del 2026-02-24T22:10:48.667 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.665+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=JC38XFBE2TME type=del size=3842913 2026-02-24T22:10:48.668 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.665+0000 7f7e64854980 1 < completed write operation key=HKOQRSF6XEXB tag=CQKCN82J8JZA type=write size=852693 2026-02-24T22:10:48.669 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.665+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=IK3E8F0UY3J1 type=del size=375016 2026-02-24T22:10:48.670 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.669+0000 7f7e64854980 1 < completed delete operation key=HOHKR92SN25M tag=DLX67IHL6CN6 type=del 2026-02-24T22:10:48.671 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.669+0000 7f7e64854980 1 > prepared part key=_multipart_HKOQRSF6XEXB.2~9341KVWLB2QR.0 size=792559 2026-02-24T22:10:48.671 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.669+0000 7f7e64854980 1 > prepared multipart upload key=HKOQRSF6XEXB upload=9341KVWLB2QR tag=R291ZTK1N3T0 type=write size=1841135 2026-02-24T22:10:48.672 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.669+0000 7f7e64854980 1 < suggested remove operation key=L9NEDH44BIGA tag=9VTOFGWE1ZA2 type=del 2026-02-24T22:10:48.673 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.669+0000 7f7e64854980 1 > prepared operation key=EB04ETNYM087 tag=KIQNQLWL8Z45 type=del size=766162 2026-02-24T22:10:48.673 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.669+0000 7f7e64854980 1 < canceled part key=_multipart_2KKQKHXO7K3K.2~1HTT1TMLZ35P.0 size=1048576 2026-02-24T22:10:48.674 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.673+0000 7f7e64854980 1 < completed part key=_multipart_2KKQKHXO7K3K.2~1HTT1TMLZ35P.1 size=410805 2026-02-24T22:10:48.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.673+0000 7f7e64854980 1 < canceled multipart upload key=2KKQKHXO7K3K upload=1HTT1TMLZ35P tag=3XN48D86T3VZ type=write size=2507957 2026-02-24T22:10:48.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.673+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~SDILAG9S0R8Y.0 size=1048576 2026-02-24T22:10:48.677 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.673+0000 7f7e64854980 1 > prepared part key=_multipart_36MKR2OZN7K9.2~SDILAG9S0R8Y.1 size=151677 2026-02-24T22:10:48.677 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.673+0000 7f7e64854980 1 > prepared multipart upload key=36MKR2OZN7K9 upload=SDILAG9S0R8Y tag=BYTS1Q6IA614 type=write size=2248829 2026-02-24T22:10:48.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.677+0000 7f7e64854980 1 < completed part key=_multipart_A6PH77Q78VEN.2~Y42JPWQ124YC.0 size=17968 2026-02-24T22:10:48.679 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.677+0000 7f7e64854980 1 < completed multipart upload key=A6PH77Q78VEN upload=Y42JPWQ124YC tag=QZFST7W0LF3O type=write size=1066544 2026-02-24T22:10:48.679 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.677+0000 7f7e64854980 1 > prepared operation key=0B2V0MHWSOC9 tag=1PW461JKEC57 type=del size=3471256 2026-02-24T22:10:48.680 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.677+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~OQQN79RX3SPQ.0 size=1048576 2026-02-24T22:10:48.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.677+0000 7f7e64854980 1 < completed part key=_multipart_LSRPN33Z2B9N.2~OQQN79RX3SPQ.1 size=216865 2026-02-24T22:10:48.682 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.681+0000 7f7e64854980 1 < completed multipart upload key=LSRPN33Z2B9N upload=OQQN79RX3SPQ tag=HXB3S3JD0MT1 type=write size=2314017 2026-02-24T22:10:48.683 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.681+0000 7f7e64854980 1 > prepared operation key=ON16L53JUSTW tag=6PFGZYW3JEHA type=write size=397920 2026-02-24T22:10:48.683 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.681+0000 7f7e64854980 1 < completed write operation key=CI8DA17DX8LU tag=Q21N0FVN84JK type=write size=610524 2026-02-24T22:10:48.685 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.681+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~WF9XU25YOKEZ.0 size=1048576 2026-02-24T22:10:48.685 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.685+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~WF9XU25YOKEZ.1 size=1048576 2026-02-24T22:10:48.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.685+0000 7f7e64854980 1 > prepared part key=_multipart_L60HQ7M3E3LL.2~WF9XU25YOKEZ.2 size=561115 2026-02-24T22:10:48.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.685+0000 7f7e64854980 1 > prepared multipart upload key=L60HQ7M3E3LL upload=WF9XU25YOKEZ tag=PWUGDDML0S1S type=write size=3706843 2026-02-24T22:10:48.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.685+0000 7f7e64854980 1 < completed delete operation key=D0MKGNRDW6L9 tag=TKL7HN21KNJ1 type=del 2026-02-24T22:10:48.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.685+0000 7f7e64854980 1 > prepared operation key=N3ZUNSI5K8HS tag=NYMUTRWSQOKR type=del size=3940273 2026-02-24T22:10:48.689 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.685+0000 7f7e64854980 1 < suggested remove operation key=HKOQRSF6XEXB tag=BOJ7J554MK76 type=del 2026-02-24T22:10:48.689 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.689+0000 7f7e64854980 1 > prepared operation key=4Y3974PYYT78 tag=F3N06GEYH2YZ type=write size=969928 2026-02-24T22:10:48.690 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.689+0000 7f7e64854980 1 < canceled operation key=Y3L3CJ51C1WR tag=IID0S05ANKT7 type=del size=2869573 2026-02-24T22:10:48.691 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.689+0000 7f7e64854980 1 > prepared operation key=EJCX06TE5RTZ tag=FU704V8XM8FM type=del size=1995651 2026-02-24T22:10:48.693 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.689+0000 7f7e64854980 1 < suggested update operation key=D0MKGNRDW6L9 tag=6YPRAK5AT7P5 type=del size=3273864 2026-02-24T22:10:48.694 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.693+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~V9DPMFOA894J.0 size=1048576 2026-02-24T22:10:48.694 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.693+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~V9DPMFOA894J.1 size=1048576 2026-02-24T22:10:48.695 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.693+0000 7f7e64854980 1 > prepared part key=_multipart_Q7BP8FXTARDW.2~V9DPMFOA894J.2 size=67126 2026-02-24T22:10:48.695 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.693+0000 7f7e64854980 1 > prepared multipart upload key=Q7BP8FXTARDW upload=V9DPMFOA894J tag=EADDG5ETDT6O type=write size=3212854 2026-02-24T22:10:48.696 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.693+0000 7f7e64854980 1 < canceled operation key=EB04ETNYM087 tag=R7VR07QIFQH3 type=del size=1483453 2026-02-24T22:10:48.697 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.693+0000 7f7e64854980 1 > prepared operation key=UQLUT9ZAI61V tag=TLC1875MUNT6 type=write size=466031 2026-02-24T22:10:48.698 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.697+0000 7f7e64854980 1 < completed delete operation key=0B2V0MHWSOC9 tag=7ERXCL6X4T0P type=del 2026-02-24T22:10:48.698 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.697+0000 7f7e64854980 1 > prepared operation key=0AXS7DY23UZU tag=JWD24FK39W3L type=del size=3193050 2026-02-24T22:10:48.699 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.697+0000 7f7e64854980 1 < completed delete operation key=4Y3974PYYT78 tag=HHR14F20ODBM type=del 2026-02-24T22:10:48.700 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.697+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~HY2Z7QGGGGVP.0 size=1048576 2026-02-24T22:10:48.701 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.697+0000 7f7e64854980 1 > prepared part key=_multipart_EJCX06TE5RTZ.2~HY2Z7QGGGGVP.1 size=911585 2026-02-24T22:10:48.701 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.697+0000 7f7e64854980 1 > prepared multipart upload key=EJCX06TE5RTZ upload=HY2Z7QGGGGVP tag=3SWPS9JGBND0 type=write size=3008737 2026-02-24T22:10:48.701 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.701+0000 7f7e64854980 -1 < failed to complete operation key=D0MKGNRDW6L9 tag=AJ3JCLNFW8H9 type=del size=672872: (22) Invalid argument 2026-02-24T22:10:48.702 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.701+0000 7f7e64854980 1 > prepared operation key=SGQYQ2TIVJPS tag=6WGXPLKVYKJZ type=del size=256677 2026-02-24T22:10:48.703 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.701+0000 7f7e64854980 1 < completed delete operation key=2KKQKHXO7K3K tag=JC38XFBE2TME type=del 2026-02-24T22:10:48.704 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.701+0000 7f7e64854980 1 > prepared operation key=2KKQKHXO7K3K tag=D6EIDON8VTZJ type=del size=1964345 2026-02-24T22:10:48.704 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.701+0000 7f7e64854980 1 < completed delete operation key=EJCX06TE5RTZ tag=IK3E8F0UY3J1 type=del 2026-02-24T22:10:48.707 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.705+0000 7f7e64854980 1 > prepared part key=_multipart_D0MKGNRDW6L9.2~53Z6NX4L6LR0.0 size=222010 2026-02-24T22:10:48.707 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.705+0000 7f7e64854980 1 > prepared multipart upload key=D0MKGNRDW6L9 upload=53Z6NX4L6LR0 tag=D5UGYYZOL4AO type=write size=1270586 2026-02-24T22:10:48.707 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.705+0000 7f7e64854980 1 < completed part key=_multipart_HKOQRSF6XEXB.2~9341KVWLB2QR.0 size=792559 2026-02-24T22:10:48.708 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.705+0000 7f7e64854980 1 < completed multipart upload key=HKOQRSF6XEXB upload=9341KVWLB2QR tag=R291ZTK1N3T0 type=write size=1841135 2026-02-24T22:10:48.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.705+0000 7f7e64854980 1 > prepared operation key=D0MKGNRDW6L9 tag=QJQQ121HTZ2U type=del size=4076686 2026-02-24T22:10:48.710 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.709+0000 7f7e64854980 1 < completed delete operation key=EB04ETNYM087 tag=KIQNQLWL8Z45 type=del 2026-02-24T22:10:48.711 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.709+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~LRL65EO510JI.0 size=1048576 2026-02-24T22:10:48.712 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.709+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~LRL65EO510JI.1 size=1048576 2026-02-24T22:10:48.712 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.709+0000 7f7e64854980 1 > prepared part key=_multipart_0AXS7DY23UZU.2~LRL65EO510JI.2 size=333929 2026-02-24T22:10:48.712 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.709+0000 7f7e64854980 1 > prepared multipart upload key=0AXS7DY23UZU upload=LRL65EO510JI tag=A316WQFQ7HPP type=write size=3479657 2026-02-24T22:10:48.713 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.709+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~SDILAG9S0R8Y.0 size=1048576 2026-02-24T22:10:48.713 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.713+0000 7f7e64854980 1 < completed part key=_multipart_36MKR2OZN7K9.2~SDILAG9S0R8Y.1 size=151677 2026-02-24T22:10:48.715 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.713+0000 7f7e64854980 1 < completed multipart upload key=36MKR2OZN7K9 upload=SDILAG9S0R8Y tag=BYTS1Q6IA614 type=write size=2248829 2026-02-24T22:10:48.715 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.713+0000 7f7e64854980 1 > prepared operation key=Z0OXREAUJVA8 tag=YRWDUIARR7CT type=write size=332684 2026-02-24T22:10:48.716 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.713+0000 7f7e64854980 1 < completed delete operation key=0B2V0MHWSOC9 tag=1PW461JKEC57 type=del 2026-02-24T22:10:48.717 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.713+0000 7f7e64854980 1 > prepared operation key=HOHKR92SN25M tag=XQNISNWM7ZHA type=write size=963139 2026-02-24T22:10:48.717 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.717+0000 7f7e64854980 1 < completed write operation key=ON16L53JUSTW tag=6PFGZYW3JEHA type=write size=397920 2026-02-24T22:10:48.719 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.717+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~1LPQGHMDICL9.0 size=1048576 2026-02-24T22:10:48.719 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.717+0000 7f7e64854980 1 > prepared part key=_multipart_PKRHWM69GBJI.2~1LPQGHMDICL9.1 size=371384 2026-02-24T22:10:48.719 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:10:48.717+0000 7f7e64854980 1 > prepared multipart upload key=PKRHWM69GBJI upload=1LPQGHMDICL9 tag=YOKP1Z8VSFJL type=write size=2468536 2026-02-24T22:10:48.720 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_rgw_stats.simulate (5471 ms) 2026-02-24T22:10:48.720 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 1 test from cls_rgw_stats (5471 ms total) 2026-02-24T22:10:48.720 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:10:48.720 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment tear-down 2026-02-24T22:10:48.738 INFO:tasks.workunit.client.0.trial046.stdout:[==========] 1 test from 1 test suite ran. (7383 ms total) 2026-02-24T22:10:48.738 INFO:tasks.workunit.client.0.trial046.stdout:[ PASSED ] 1 test. 2026-02-24T22:10:48.742 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T22:10:48.743 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T22:10:48.794 INFO:tasks.workunit:Running workunits matching cls/test_cls_cmpomap.sh on client.0... 2026-02-24T22:10:48.796 INFO:tasks.workunit:Running workunit cls/test_cls_cmpomap.sh... 2026-02-24T22:10:48.797 DEBUG:teuthology.orchestra.run.trial046:workunit test cls/test_cls_cmpomap.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7e05f4fdea5af173284f247d6b0935b337d4ccf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_cmpomap.sh 2026-02-24T22:10:48.851 INFO:tasks.workunit.client.0.trial046.stdout:Running main() from gmock_main.cc 2026-02-24T22:10:48.851 INFO:tasks.workunit.client.0.trial046.stdout:[==========] Running 31 tests from 1 test suite. 2026-02-24T22:10:48.851 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment set-up. 2026-02-24T22:10:50.745 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 31 tests from CmpOmap 2026-02-24T22:10:50.745 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_str 2026-02-24T22:10:50.751 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_vals_noexist_str (6 ms) 2026-02-24T22:10:50.752 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_str_default 2026-02-24T22:10:50.755 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_vals_noexist_str_default (3 ms) 2026-02-24T22:10:50.755 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_u64 2026-02-24T22:10:50.758 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_vals_noexist_u64 (2 ms) 2026-02-24T22:10:50.758 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_u64_default 2026-02-24T22:10:50.760 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_vals_noexist_u64_default (2 ms) 2026-02-24T22:10:50.760 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_vals_str 2026-02-24T22:10:50.766 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_vals_str (5 ms) 2026-02-24T22:10:50.766 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_vals_u64 2026-02-24T22:10:50.770 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_vals_u64 (4 ms) 2026-02-24T22:10:50.770 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_input 2026-02-24T22:10:50.772 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_input (2 ms) 2026-02-24T22:10:50.772 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_default 2026-02-24T22:10:50.775 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_default (2 ms) 2026-02-24T22:10:50.775 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_vals_u64_empty_default 2026-02-24T22:10:50.779 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_vals_u64_empty_default (3 ms) 2026-02-24T22:10:50.779 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_value 2026-02-24T22:10:50.783 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_value (3 ms) 2026-02-24T22:10:50.783 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_vals_at_max_keys 2026-02-24T22:10:50.783 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_vals_at_max_keys (0 ms) 2026-02-24T22:10:50.783 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_vals_over_max_keys 2026-02-24T22:10:50.784 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_vals_over_max_keys (0 ms) 2026-02-24T22:10:50.784 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_str 2026-02-24T22:10:50.788 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_str (4 ms) 2026-02-24T22:10:50.788 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_str_default 2026-02-24T22:10:50.792 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_str_default (4 ms) 2026-02-24T22:10:50.793 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_u64 2026-02-24T22:10:50.796 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_u64 (3 ms) 2026-02-24T22:10:50.796 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_u64_default 2026-02-24T22:10:50.800 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_u64_default (3 ms) 2026-02-24T22:10:50.800 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_set_vals_str 2026-02-24T22:10:50.804 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_set_vals_str (4 ms) 2026-02-24T22:10:50.805 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64 2026-02-24T22:10:50.810 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_set_vals_u64 (5 ms) 2026-02-24T22:10:50.810 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64_einval 2026-02-24T22:10:50.812 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_set_vals_u64_einval (2 ms) 2026-02-24T22:10:50.812 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64_eio 2026-02-24T22:10:50.814 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_set_vals_u64_eio (1 ms) 2026-02-24T22:10:50.814 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_set_vals_at_max_keys 2026-02-24T22:10:50.815 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_set_vals_at_max_keys (0 ms) 2026-02-24T22:10:50.815 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_set_vals_over_max_keys 2026-02-24T22:10:50.815 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_set_vals_over_max_keys (0 ms) 2026-02-24T22:10:50.815 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_rm_keys_noexist_str 2026-02-24T22:10:50.819 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_rm_keys_noexist_str (3 ms) 2026-02-24T22:10:50.819 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_rm_keys_noexist_u64 2026-02-24T22:10:50.823 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_rm_keys_noexist_u64 (3 ms) 2026-02-24T22:10:50.823 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_rm_keys_str 2026-02-24T22:10:50.827 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_rm_keys_str (4 ms) 2026-02-24T22:10:50.827 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64 2026-02-24T22:10:50.832 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64 (5 ms) 2026-02-24T22:10:50.832 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_einval 2026-02-24T22:10:50.833 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_einval (1 ms) 2026-02-24T22:10:50.834 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_eio 2026-02-24T22:10:50.835 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_eio (1 ms) 2026-02-24T22:10:50.835 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_rm_keys_at_max_keys 2026-02-24T22:10:50.836 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_rm_keys_at_max_keys (0 ms) 2026-02-24T22:10:50.836 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_rm_keys_over_max_keys 2026-02-24T22:10:50.836 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_rm_keys_over_max_keys (0 ms) 2026-02-24T22:10:50.837 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_empty 2026-02-24T22:10:50.841 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_empty (4 ms) 2026-02-24T22:10:50.841 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 31 tests from CmpOmap (96 ms total) 2026-02-24T22:10:50.841 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:10:50.841 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment tear-down 2026-02-24T22:10:51.750 INFO:tasks.workunit.client.0.trial046.stdout:[==========] 31 tests from 1 test suite ran. (2899 ms total) 2026-02-24T22:10:51.751 INFO:tasks.workunit.client.0.trial046.stdout:[ PASSED ] 31 tests. 2026-02-24T22:10:51.753 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T22:10:51.753 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T22:10:51.802 INFO:tasks.workunit:Running workunits matching cls/test_cls_2pc_queue.sh on client.0... 2026-02-24T22:10:51.804 INFO:tasks.workunit:Running workunit cls/test_cls_2pc_queue.sh... 2026-02-24T22:10:51.805 DEBUG:teuthology.orchestra.run.trial046:workunit test cls/test_cls_2pc_queue.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7e05f4fdea5af173284f247d6b0935b337d4ccf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_2pc_queue.sh 2026-02-24T22:10:51.858 INFO:tasks.workunit.client.0.trial046.stdout:Running main() from gmock_main.cc 2026-02-24T22:10:51.859 INFO:tasks.workunit.client.0.trial046.stdout:[==========] Running 24 tests from 1 test suite. 2026-02-24T22:10:51.859 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment set-up. 2026-02-24T22:10:51.859 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 24 tests from TestCls2PCQueue 2026-02-24T22:10:51.859 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestCls2PCQueue.GetCapacity 2026-02-24T22:10:54.755 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestCls2PCQueue.GetCapacity (2896 ms) 2026-02-24T22:10:54.755 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestCls2PCQueue.AsyncGetCapacity 2026-02-24T22:10:57.767 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestCls2PCQueue.AsyncGetCapacity (3012 ms) 2026-02-24T22:10:57.767 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestCls2PCQueue.Reserve 2026-02-24T22:11:00.777 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestCls2PCQueue.Reserve (3010 ms) 2026-02-24T22:11:00.777 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestCls2PCQueue.AsyncReserve 2026-02-24T22:11:03.792 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestCls2PCQueue.AsyncReserve (3015 ms) 2026-02-24T22:11:03.792 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestCls2PCQueue.Commit 2026-02-24T22:11:06.803 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestCls2PCQueue.Commit (3010 ms) 2026-02-24T22:11:06.803 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestCls2PCQueue.Stats 2026-02-24T22:11:09.815 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestCls2PCQueue.Stats (3012 ms) 2026-02-24T22:11:09.816 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestCls2PCQueue.UpgradeFromReef 2026-02-24T22:11:12.827 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestCls2PCQueue.UpgradeFromReef (3011 ms) 2026-02-24T22:11:12.827 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestCls2PCQueue.Abort 2026-02-24T22:11:15.840 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestCls2PCQueue.Abort (3013 ms) 2026-02-24T22:11:15.841 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestCls2PCQueue.ReserveError 2026-02-24T22:11:18.848 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestCls2PCQueue.ReserveError (3008 ms) 2026-02-24T22:11:18.849 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestCls2PCQueue.CommitError 2026-02-24T22:11:21.865 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestCls2PCQueue.CommitError (3016 ms) 2026-02-24T22:11:21.866 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestCls2PCQueue.AbortError 2026-02-24T22:11:24.877 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestCls2PCQueue.AbortError (3011 ms) 2026-02-24T22:11:24.877 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestCls2PCQueue.MultiReserve 2026-02-24T22:11:27.891 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestCls2PCQueue.MultiReserve (3014 ms) 2026-02-24T22:11:27.892 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestCls2PCQueue.MultiCommit 2026-02-24T22:11:30.904 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestCls2PCQueue.MultiCommit (3012 ms) 2026-02-24T22:11:30.904 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestCls2PCQueue.MultiAbort 2026-02-24T22:11:33.920 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestCls2PCQueue.MultiAbort (3016 ms) 2026-02-24T22:11:33.920 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestCls2PCQueue.ReserveCommit 2026-02-24T22:11:36.932 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestCls2PCQueue.ReserveCommit (3012 ms) 2026-02-24T22:11:36.933 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestCls2PCQueue.ReserveAbort 2026-02-24T22:11:39.949 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestCls2PCQueue.ReserveAbort (3016 ms) 2026-02-24T22:11:39.949 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestCls2PCQueue.ManualCleanup 2026-02-24T22:11:42.960 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestCls2PCQueue.ManualCleanup (3011 ms) 2026-02-24T22:11:42.960 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestCls2PCQueue.Cleanup 2026-02-24T22:11:45.969 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestCls2PCQueue.Cleanup (3008 ms) 2026-02-24T22:11:45.969 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestCls2PCQueue.MultiProducer 2026-02-24T22:11:52.909 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestCls2PCQueue.MultiProducer (6940 ms) 2026-02-24T22:11:52.909 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestCls2PCQueue.AsyncConsumer 2026-02-24T22:11:55.922 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestCls2PCQueue.AsyncConsumer (3012 ms) 2026-02-24T22:11:55.922 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestCls2PCQueue.MultiProducerConsumer 2026-02-24T22:12:04.744 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestCls2PCQueue.MultiProducerConsumer (8822 ms) 2026-02-24T22:12:04.744 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestCls2PCQueue.ReserveSpillover 2026-02-24T22:12:07.758 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestCls2PCQueue.ReserveSpillover (3014 ms) 2026-02-24T22:12:07.759 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestCls2PCQueue.CommitSpillover 2026-02-24T22:12:11.773 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestCls2PCQueue.CommitSpillover (4015 ms) 2026-02-24T22:12:11.774 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestCls2PCQueue.AbortSpillover 2026-02-24T22:12:15.789 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestCls2PCQueue.AbortSpillover (4015 ms) 2026-02-24T22:12:15.789 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 24 tests from TestCls2PCQueue (83931 ms total) 2026-02-24T22:12:15.790 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:12:15.790 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment tear-down 2026-02-24T22:12:15.790 INFO:tasks.workunit.client.0.trial046.stdout:[==========] 24 tests from 1 test suite ran. (83931 ms total) 2026-02-24T22:12:15.790 INFO:tasks.workunit.client.0.trial046.stdout:[ PASSED ] 24 tests. 2026-02-24T22:12:15.793 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T22:12:15.794 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T22:12:15.845 INFO:tasks.workunit:Running workunits matching cls/test_cls_user.sh on client.0... 2026-02-24T22:12:15.847 INFO:tasks.workunit:Running workunit cls/test_cls_user.sh... 2026-02-24T22:12:15.848 DEBUG:teuthology.orchestra.run.trial046:workunit test cls/test_cls_user.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7e05f4fdea5af173284f247d6b0935b337d4ccf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh 2026-02-24T22:12:15.906 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:12:15.906 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:12:15.906 INFO:tasks.workunit.client.0.trial046.stdout:[==========] Running 4 tests from 1 test suite. 2026-02-24T22:12:15.906 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment set-up. 2026-02-24T22:12:17.800 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 4 tests from ClsAccount 2026-02-24T22:12:17.800 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] ClsAccount.add 2026-02-24T22:12:17.809 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] ClsAccount.add (9 ms) 2026-02-24T22:12:17.810 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] ClsAccount.get 2026-02-24T22:12:17.812 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] ClsAccount.get (3 ms) 2026-02-24T22:12:17.813 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] ClsAccount.rm 2026-02-24T22:12:18.140 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] ClsAccount.rm (326 ms) 2026-02-24T22:12:18.140 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] ClsAccount.list 2026-02-24T22:12:18.148 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] ClsAccount.list (8 ms) 2026-02-24T22:12:18.148 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 4 tests from ClsAccount (347 ms total) 2026-02-24T22:12:18.148 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:12:18.148 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment tear-down 2026-02-24T22:12:18.803 INFO:tasks.workunit.client.0.trial046.stdout:[==========] 4 tests from 1 test suite ran. (2896 ms total) 2026-02-24T22:12:18.803 INFO:tasks.workunit.client.0.trial046.stdout:[ PASSED ] 4 tests. 2026-02-24T22:12:18.806 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T22:12:18.807 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T22:12:18.857 INFO:tasks.workunit:Running workunits matching cls/test_cls_sem_set.sh on client.0... 2026-02-24T22:12:18.860 INFO:tasks.workunit:Running workunit cls/test_cls_sem_set.sh... 2026-02-24T22:12:18.860 DEBUG:teuthology.orchestra.run.trial046:workunit test cls/test_cls_sem_set.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7e05f4fdea5af173284f247d6b0935b337d4ccf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_sem_set.sh 2026-02-24T22:12:18.914 INFO:tasks.workunit.client.0.trial046.stdout:Running main() from gmock_main.cc 2026-02-24T22:12:18.914 INFO:tasks.workunit.client.0.trial046.stdout:[==========] Running 5 tests from 1 test suite. 2026-02-24T22:12:18.914 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment set-up. 2026-02-24T22:12:18.914 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 5 tests from cls_sem_set 2026-02-24T22:12:18.914 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_sem_set.inc_dec 2026-02-24T22:12:21.818 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_sem_set.inc_dec (2904 ms) 2026-02-24T22:12:21.819 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_sem_set.inc_dec_overflow 2026-02-24T22:12:24.826 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_sem_set.inc_dec_overflow (3007 ms) 2026-02-24T22:12:24.827 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_sem_set.list_small 2026-02-24T22:12:27.842 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_sem_set.list_small (3015 ms) 2026-02-24T22:12:27.842 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_sem_set.list_large 2026-02-24T22:12:30.853 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_sem_set.list_large (3011 ms) 2026-02-24T22:12:30.854 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] cls_sem_set.inc_dec_time 2026-02-24T22:12:34.869 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] cls_sem_set.inc_dec_time (4015 ms) 2026-02-24T22:12:34.869 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 5 tests from cls_sem_set (15954 ms total) 2026-02-24T22:12:34.869 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:12:34.869 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment tear-down 2026-02-24T22:12:34.869 INFO:tasks.workunit.client.0.trial046.stdout:[==========] 5 tests from 1 test suite ran. (15955 ms total) 2026-02-24T22:12:34.869 INFO:tasks.workunit.client.0.trial046.stdout:[ PASSED ] 5 tests. 2026-02-24T22:12:34.873 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T22:12:34.873 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T22:12:34.925 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_gc_log.sh on client.0... 2026-02-24T22:12:34.927 INFO:tasks.workunit:Running workunit rgw/test_rgw_gc_log.sh... 2026-02-24T22:12:34.928 DEBUG:teuthology.orchestra.run.trial046:workunit test rgw/test_rgw_gc_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7e05f4fdea5af173284f247d6b0935b337d4ccf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_gc_log.sh 2026-02-24T22:12:34.985 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:12:34.985 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:12:34.986 INFO:tasks.workunit.client.0.trial046.stdout:[==========] Running 6 tests from 1 test suite. 2026-02-24T22:12:34.986 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment set-up. 2026-02-24T22:12:36.880 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 6 tests from rgw_gc_log 2026-02-24T22:12:36.880 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] rgw_gc_log.init_existing_queue 2026-02-24T22:12:36.883 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] rgw_gc_log.init_existing_queue (3 ms) 2026-02-24T22:12:36.883 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] rgw_gc_log.init_existing_omap 2026-02-24T22:12:36.893 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] rgw_gc_log.init_existing_omap (9 ms) 2026-02-24T22:12:36.893 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] rgw_gc_log.enqueue1_after_init 2026-02-24T22:12:36.894 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] rgw_gc_log.enqueue1_after_init (1 ms) 2026-02-24T22:12:36.894 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] rgw_gc_log.enqueue2_before_init 2026-02-24T22:12:36.895 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] rgw_gc_log.enqueue2_before_init (1 ms) 2026-02-24T22:12:36.895 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] rgw_gc_log.defer1_after_init 2026-02-24T22:12:36.897 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] rgw_gc_log.defer1_after_init (1 ms) 2026-02-24T22:12:36.897 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] rgw_gc_log.defer2_before_init 2026-02-24T22:12:36.898 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] rgw_gc_log.defer2_before_init (0 ms) 2026-02-24T22:12:36.898 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 6 tests from rgw_gc_log (18 ms total) 2026-02-24T22:12:36.898 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:12:36.898 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment tear-down 2026-02-24T22:12:37.880 INFO:tasks.workunit.client.0.trial046.stdout:[==========] 6 tests from 1 test suite ran. (2894 ms total) 2026-02-24T22:12:37.880 INFO:tasks.workunit.client.0.trial046.stdout:[ PASSED ] 6 tests. 2026-02-24T22:12:37.884 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T22:12:37.884 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T22:12:37.933 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_obj.sh on client.0... 2026-02-24T22:12:37.935 INFO:tasks.workunit:Running workunit rgw/test_rgw_obj.sh... 2026-02-24T22:12:37.935 DEBUG:teuthology.orchestra.run.trial046:workunit test rgw/test_rgw_obj.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7e05f4fdea5af173284f247d6b0935b337d4ccf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_obj.sh 2026-02-24T22:12:37.996 INFO:tasks.workunit.client.0.trial046.stdout:Running main() from gmock_main.cc 2026-02-24T22:12:37.996 INFO:tasks.workunit.client.0.trial046.stdout:[==========] Running 4 tests from 1 test suite. 2026-02-24T22:12:37.996 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment set-up. 2026-02-24T22:12:37.996 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 4 tests from TestRGWObj 2026-02-24T22:12:37.996 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestRGWObj.underscore 2026-02-24T22:12:37.996 INFO:tasks.workunit.client.0.trial046.stdout:parsed: _obj 2026-02-24T22:12:37.996 INFO:tasks.workunit.client.0.trial046.stdout:parsed: _obj 2026-02-24T22:12:37.996 INFO:tasks.workunit.client.0.trial046.stdout:stripped: _obj ns= i= 2026-02-24T22:12:37.997 INFO:tasks.workunit.client.0.trial046.stdout:parsed: _obj 2026-02-24T22:12:37.997 INFO:tasks.workunit.client.0.trial046.stdout:parsed: _obj 2026-02-24T22:12:37.997 INFO:tasks.workunit.client.0.trial046.stdout:stripped: _obj ns= i= 2026-02-24T22:12:37.997 INFO:tasks.workunit.client.0.trial046.stdout:parsed: _obj 2026-02-24T22:12:37.997 INFO:tasks.workunit.client.0.trial046.stdout:parsed: _obj 2026-02-24T22:12:37.997 INFO:tasks.workunit.client.0.trial046.stdout:stripped: _obj ns= i= 2026-02-24T22:12:37.997 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:37.997 INFO:tasks.workunit.client.0.trial046.stdout: "obj1": { 2026-02-24T22:12:37.997 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:37.997 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:37.997 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:37.997 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:37.998 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:37.998 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:37.998 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:37.998 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:37.998 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:37.998 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:37.998 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:37.998 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:37.998 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_obj", 2026-02-24T22:12:37.998 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:37.998 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:37.998 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:37.999 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:37.999 INFO:tasks.workunit.client.0.trial046.stdout: "obj2": { 2026-02-24T22:12:37.999 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:37.999 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:37.999 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:37.999 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:37.999 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:37.999 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:37.999 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:37.999 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:37.999 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:37.999 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:37.999 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.000 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.000 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_obj", 2026-02-24T22:12:38.000 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.000 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.000 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.000 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.000 INFO:tasks.workunit.client.0.trial046.stdout: "obj3": { 2026-02-24T22:12:38.000 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.000 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.000 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.000 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.000 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.000 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.001 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.001 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.001 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.001 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.001 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.001 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.001 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_obj", 2026-02-24T22:12:38.001 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.001 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.001 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.001 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.001 INFO:tasks.workunit.client.0.trial046.stdout: "obj3-2": { 2026-02-24T22:12:38.002 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.002 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.002 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.002 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.002 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.002 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.002 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.002 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.002 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.002 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.002 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.002 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.003 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_obj", 2026-02-24T22:12:38.003 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.003 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.003 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.003 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.003 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.003 INFO:tasks.workunit.client.0.trial046.stdout:parsed: _obj 2026-02-24T22:12:38.003 INFO:tasks.workunit.client.0.trial046.stdout:parsed: _obj 2026-02-24T22:12:38.003 INFO:tasks.workunit.client.0.trial046.stdout:stripped: _obj ns=ns i= 2026-02-24T22:12:38.003 INFO:tasks.workunit.client.0.trial046.stdout:parsed: _obj 2026-02-24T22:12:38.003 INFO:tasks.workunit.client.0.trial046.stdout:parsed: _obj 2026-02-24T22:12:38.003 INFO:tasks.workunit.client.0.trial046.stdout:stripped: _obj ns=ns i= 2026-02-24T22:12:38.004 INFO:tasks.workunit.client.0.trial046.stdout:parsed: _obj 2026-02-24T22:12:38.004 INFO:tasks.workunit.client.0.trial046.stdout:parsed: _obj 2026-02-24T22:12:38.004 INFO:tasks.workunit.client.0.trial046.stdout:stripped: _obj ns=ns i= 2026-02-24T22:12:38.004 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.004 INFO:tasks.workunit.client.0.trial046.stdout: "obj1": { 2026-02-24T22:12:38.004 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.004 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.004 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.004 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.004 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.004 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.004 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.004 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.005 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.005 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.005 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.005 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.005 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_obj", 2026-02-24T22:12:38.005 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.005 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.005 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.005 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.005 INFO:tasks.workunit.client.0.trial046.stdout: "obj2": { 2026-02-24T22:12:38.005 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.005 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.006 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.006 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.006 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.006 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.006 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.006 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.006 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.006 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.006 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.006 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.006 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_obj", 2026-02-24T22:12:38.006 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.007 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.007 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.007 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.007 INFO:tasks.workunit.client.0.trial046.stdout: "obj3": { 2026-02-24T22:12:38.007 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.007 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.007 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.007 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.007 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.007 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.007 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.007 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.008 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.008 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.008 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.008 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.008 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_obj", 2026-02-24T22:12:38.008 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.008 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.008 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.008 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.008 INFO:tasks.workunit.client.0.trial046.stdout: "obj3-2": { 2026-02-24T22:12:38.008 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.008 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.009 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.009 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.009 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.009 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.009 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.009 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.009 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.009 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.009 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.009 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.009 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_obj", 2026-02-24T22:12:38.009 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.009 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.010 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.010 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.010 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.010 INFO:tasks.workunit.client.0.trial046.stdout:parsed: _obj[v1] 2026-02-24T22:12:38.010 INFO:tasks.workunit.client.0.trial046.stdout:parsed: _obj[v1] 2026-02-24T22:12:38.010 INFO:tasks.workunit.client.0.trial046.stdout:stripped: _obj ns= i=v1 2026-02-24T22:12:38.010 INFO:tasks.workunit.client.0.trial046.stdout:parsed: _obj[v1] 2026-02-24T22:12:38.010 INFO:tasks.workunit.client.0.trial046.stdout:parsed: _obj[v1] 2026-02-24T22:12:38.010 INFO:tasks.workunit.client.0.trial046.stdout:stripped: _obj ns= i=v1 2026-02-24T22:12:38.010 INFO:tasks.workunit.client.0.trial046.stdout:parsed: _obj[v1] 2026-02-24T22:12:38.010 INFO:tasks.workunit.client.0.trial046.stdout:parsed: _obj[v1] 2026-02-24T22:12:38.010 INFO:tasks.workunit.client.0.trial046.stdout:stripped: _obj ns= i=v1 2026-02-24T22:12:38.010 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.011 INFO:tasks.workunit.client.0.trial046.stdout: "obj1": { 2026-02-24T22:12:38.011 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.011 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.011 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.011 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.011 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.011 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.011 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.011 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.011 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.011 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.011 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.012 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.012 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_obj", 2026-02-24T22:12:38.012 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "v1", 2026-02-24T22:12:38.012 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.012 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.012 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.012 INFO:tasks.workunit.client.0.trial046.stdout: "obj2": { 2026-02-24T22:12:38.012 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.012 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.012 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.012 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.012 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.012 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.013 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.013 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.013 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.013 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.013 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.013 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.013 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_obj", 2026-02-24T22:12:38.013 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "v1", 2026-02-24T22:12:38.013 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.013 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.013 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.013 INFO:tasks.workunit.client.0.trial046.stdout: "obj3": { 2026-02-24T22:12:38.013 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.014 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.014 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.014 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.014 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.014 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.014 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.014 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.014 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.014 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.014 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.014 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.014 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_obj", 2026-02-24T22:12:38.015 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.015 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.015 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.015 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.015 INFO:tasks.workunit.client.0.trial046.stdout: "obj3-2": { 2026-02-24T22:12:38.015 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.015 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.015 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.015 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.015 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.015 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.015 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.015 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.016 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.016 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.016 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.016 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.016 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_obj", 2026-02-24T22:12:38.016 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "v1", 2026-02-24T22:12:38.016 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.016 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.016 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.016 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.016 INFO:tasks.workunit.client.0.trial046.stdout:parsed: _obj[v1] 2026-02-24T22:12:38.016 INFO:tasks.workunit.client.0.trial046.stdout:parsed: _obj[v1] 2026-02-24T22:12:38.016 INFO:tasks.workunit.client.0.trial046.stdout:stripped: _obj ns=ns i=v1 2026-02-24T22:12:38.017 INFO:tasks.workunit.client.0.trial046.stdout:parsed: _obj[v1] 2026-02-24T22:12:38.017 INFO:tasks.workunit.client.0.trial046.stdout:parsed: _obj[v1] 2026-02-24T22:12:38.017 INFO:tasks.workunit.client.0.trial046.stdout:stripped: _obj ns=ns i=v1 2026-02-24T22:12:38.017 INFO:tasks.workunit.client.0.trial046.stdout:parsed: _obj[v1] 2026-02-24T22:12:38.017 INFO:tasks.workunit.client.0.trial046.stdout:parsed: _obj[v1] 2026-02-24T22:12:38.017 INFO:tasks.workunit.client.0.trial046.stdout:stripped: _obj ns=ns i=v1 2026-02-24T22:12:38.017 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.017 INFO:tasks.workunit.client.0.trial046.stdout: "obj1": { 2026-02-24T22:12:38.017 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.017 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.017 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.017 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.017 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.018 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.018 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.018 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.018 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.018 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.018 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.018 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.018 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_obj", 2026-02-24T22:12:38.018 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "v1", 2026-02-24T22:12:38.018 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.018 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.018 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.018 INFO:tasks.workunit.client.0.trial046.stdout: "obj2": { 2026-02-24T22:12:38.019 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.019 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.019 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.019 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.019 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.019 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.019 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.019 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.019 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.019 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.019 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.019 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.019 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_obj", 2026-02-24T22:12:38.020 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "v1", 2026-02-24T22:12:38.020 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.020 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.020 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.020 INFO:tasks.workunit.client.0.trial046.stdout: "obj3": { 2026-02-24T22:12:38.020 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.020 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.020 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.020 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.020 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.020 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.020 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.020 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.020 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.021 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.021 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.021 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.021 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_obj", 2026-02-24T22:12:38.021 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.021 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.021 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.021 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.021 INFO:tasks.workunit.client.0.trial046.stdout: "obj3-2": { 2026-02-24T22:12:38.021 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.021 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.021 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.021 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.022 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.022 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.022 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.022 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.022 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.022 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.022 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.022 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.022 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_obj", 2026-02-24T22:12:38.022 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "v1", 2026-02-24T22:12:38.022 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.022 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.022 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.023 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.023 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestRGWObj.underscore (0 ms) 2026-02-24T22:12:38.023 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestRGWObj.no_underscore 2026-02-24T22:12:38.023 INFO:tasks.workunit.client.0.trial046.stdout:parsed: obj 2026-02-24T22:12:38.023 INFO:tasks.workunit.client.0.trial046.stdout:parsed: obj 2026-02-24T22:12:38.023 INFO:tasks.workunit.client.0.trial046.stdout:stripped: obj ns= i= 2026-02-24T22:12:38.023 INFO:tasks.workunit.client.0.trial046.stdout:parsed: obj 2026-02-24T22:12:38.023 INFO:tasks.workunit.client.0.trial046.stdout:parsed: obj 2026-02-24T22:12:38.023 INFO:tasks.workunit.client.0.trial046.stdout:stripped: obj ns= i= 2026-02-24T22:12:38.023 INFO:tasks.workunit.client.0.trial046.stdout:parsed: obj 2026-02-24T22:12:38.023 INFO:tasks.workunit.client.0.trial046.stdout:parsed: obj 2026-02-24T22:12:38.023 INFO:tasks.workunit.client.0.trial046.stdout:stripped: obj ns= i= 2026-02-24T22:12:38.023 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.024 INFO:tasks.workunit.client.0.trial046.stdout: "obj1": { 2026-02-24T22:12:38.024 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.024 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.024 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.024 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.024 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.024 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.024 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.024 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.024 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.024 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.024 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.025 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.025 INFO:tasks.workunit.client.0.trial046.stdout: "name": "obj", 2026-02-24T22:12:38.025 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.025 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.025 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.025 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.025 INFO:tasks.workunit.client.0.trial046.stdout: "obj2": { 2026-02-24T22:12:38.025 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.025 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.025 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.025 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.025 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.025 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.026 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.026 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.026 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.026 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.026 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.026 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.026 INFO:tasks.workunit.client.0.trial046.stdout: "name": "obj", 2026-02-24T22:12:38.026 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.026 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.026 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.026 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.026 INFO:tasks.workunit.client.0.trial046.stdout: "obj3": { 2026-02-24T22:12:38.026 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.027 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.027 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.027 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.027 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.027 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.027 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.027 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.027 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.027 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.027 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.027 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.027 INFO:tasks.workunit.client.0.trial046.stdout: "name": "obj", 2026-02-24T22:12:38.027 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.027 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.027 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.028 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.028 INFO:tasks.workunit.client.0.trial046.stdout: "obj3-2": { 2026-02-24T22:12:38.028 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.028 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.028 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.028 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.028 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.028 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.028 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.028 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.028 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.028 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.028 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.028 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.028 INFO:tasks.workunit.client.0.trial046.stdout: "name": "obj", 2026-02-24T22:12:38.029 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.029 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.029 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.029 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.029 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.029 INFO:tasks.workunit.client.0.trial046.stdout:parsed: obj 2026-02-24T22:12:38.029 INFO:tasks.workunit.client.0.trial046.stdout:parsed: obj 2026-02-24T22:12:38.029 INFO:tasks.workunit.client.0.trial046.stdout:stripped: obj ns=ns i= 2026-02-24T22:12:38.029 INFO:tasks.workunit.client.0.trial046.stdout:parsed: obj 2026-02-24T22:12:38.029 INFO:tasks.workunit.client.0.trial046.stdout:parsed: obj 2026-02-24T22:12:38.029 INFO:tasks.workunit.client.0.trial046.stdout:stripped: obj ns=ns i= 2026-02-24T22:12:38.029 INFO:tasks.workunit.client.0.trial046.stdout:parsed: obj 2026-02-24T22:12:38.029 INFO:tasks.workunit.client.0.trial046.stdout:parsed: obj 2026-02-24T22:12:38.029 INFO:tasks.workunit.client.0.trial046.stdout:stripped: obj ns=ns i= 2026-02-24T22:12:38.030 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.030 INFO:tasks.workunit.client.0.trial046.stdout: "obj1": { 2026-02-24T22:12:38.030 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.030 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.030 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.030 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.030 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.030 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.030 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.030 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.030 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.030 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.030 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.030 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.030 INFO:tasks.workunit.client.0.trial046.stdout: "name": "obj", 2026-02-24T22:12:38.031 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.031 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.031 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.031 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.031 INFO:tasks.workunit.client.0.trial046.stdout: "obj2": { 2026-02-24T22:12:38.031 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.031 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.031 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.031 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.031 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.031 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.031 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.031 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.031 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.031 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.031 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.032 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.032 INFO:tasks.workunit.client.0.trial046.stdout: "name": "obj", 2026-02-24T22:12:38.032 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.032 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.032 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.032 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.032 INFO:tasks.workunit.client.0.trial046.stdout: "obj3": { 2026-02-24T22:12:38.032 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.032 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.032 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.032 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.032 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.032 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.032 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.032 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.032 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.032 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.033 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.033 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.033 INFO:tasks.workunit.client.0.trial046.stdout: "name": "obj", 2026-02-24T22:12:38.033 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.033 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.033 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.033 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.033 INFO:tasks.workunit.client.0.trial046.stdout: "obj3-2": { 2026-02-24T22:12:38.033 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.033 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.033 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.033 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.033 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.033 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.033 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.033 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.033 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.034 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.034 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.034 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.034 INFO:tasks.workunit.client.0.trial046.stdout: "name": "obj", 2026-02-24T22:12:38.034 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.034 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.034 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.034 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.034 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.034 INFO:tasks.workunit.client.0.trial046.stdout:parsed: obj[v1] 2026-02-24T22:12:38.034 INFO:tasks.workunit.client.0.trial046.stdout:parsed: obj[v1] 2026-02-24T22:12:38.034 INFO:tasks.workunit.client.0.trial046.stdout:stripped: obj ns= i=v1 2026-02-24T22:12:38.034 INFO:tasks.workunit.client.0.trial046.stdout:parsed: obj[v1] 2026-02-24T22:12:38.034 INFO:tasks.workunit.client.0.trial046.stdout:parsed: obj[v1] 2026-02-24T22:12:38.034 INFO:tasks.workunit.client.0.trial046.stdout:stripped: obj ns= i=v1 2026-02-24T22:12:38.034 INFO:tasks.workunit.client.0.trial046.stdout:parsed: obj[v1] 2026-02-24T22:12:38.034 INFO:tasks.workunit.client.0.trial046.stdout:parsed: obj[v1] 2026-02-24T22:12:38.034 INFO:tasks.workunit.client.0.trial046.stdout:stripped: obj ns= i=v1 2026-02-24T22:12:38.035 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.035 INFO:tasks.workunit.client.0.trial046.stdout: "obj1": { 2026-02-24T22:12:38.035 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.035 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.035 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.035 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.035 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.035 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.035 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.035 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.035 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.035 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.035 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.035 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.035 INFO:tasks.workunit.client.0.trial046.stdout: "name": "obj", 2026-02-24T22:12:38.035 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "v1", 2026-02-24T22:12:38.035 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.035 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.036 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.036 INFO:tasks.workunit.client.0.trial046.stdout: "obj2": { 2026-02-24T22:12:38.036 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.036 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.036 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.036 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.036 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.036 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.036 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.036 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.036 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.036 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.036 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.036 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.036 INFO:tasks.workunit.client.0.trial046.stdout: "name": "obj", 2026-02-24T22:12:38.036 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "v1", 2026-02-24T22:12:38.036 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.036 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.036 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.037 INFO:tasks.workunit.client.0.trial046.stdout: "obj3": { 2026-02-24T22:12:38.037 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.037 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.037 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.037 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.037 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.037 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.037 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.037 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.037 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.037 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.037 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.037 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.037 INFO:tasks.workunit.client.0.trial046.stdout: "name": "obj", 2026-02-24T22:12:38.037 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.037 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.037 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.037 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.037 INFO:tasks.workunit.client.0.trial046.stdout: "obj3-2": { 2026-02-24T22:12:38.037 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.038 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.038 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.038 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.038 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.038 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.038 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.038 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.038 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.038 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.038 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.038 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.038 INFO:tasks.workunit.client.0.trial046.stdout: "name": "obj", 2026-02-24T22:12:38.038 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "v1", 2026-02-24T22:12:38.038 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.038 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.038 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.038 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.038 INFO:tasks.workunit.client.0.trial046.stdout:parsed: obj[v1] 2026-02-24T22:12:38.039 INFO:tasks.workunit.client.0.trial046.stdout:parsed: obj[v1] 2026-02-24T22:12:38.039 INFO:tasks.workunit.client.0.trial046.stdout:stripped: obj ns=ns i=v1 2026-02-24T22:12:38.039 INFO:tasks.workunit.client.0.trial046.stdout:parsed: obj[v1] 2026-02-24T22:12:38.039 INFO:tasks.workunit.client.0.trial046.stdout:parsed: obj[v1] 2026-02-24T22:12:38.039 INFO:tasks.workunit.client.0.trial046.stdout:stripped: obj ns=ns i=v1 2026-02-24T22:12:38.039 INFO:tasks.workunit.client.0.trial046.stdout:parsed: obj[v1] 2026-02-24T22:12:38.039 INFO:tasks.workunit.client.0.trial046.stdout:parsed: obj[v1] 2026-02-24T22:12:38.039 INFO:tasks.workunit.client.0.trial046.stdout:stripped: obj ns=ns i=v1 2026-02-24T22:12:38.039 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.039 INFO:tasks.workunit.client.0.trial046.stdout: "obj1": { 2026-02-24T22:12:38.039 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.039 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.039 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.039 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.039 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.039 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.039 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.039 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.039 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.039 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.039 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.040 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.040 INFO:tasks.workunit.client.0.trial046.stdout: "name": "obj", 2026-02-24T22:12:38.040 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "v1", 2026-02-24T22:12:38.040 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.040 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.040 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.040 INFO:tasks.workunit.client.0.trial046.stdout: "obj2": { 2026-02-24T22:12:38.040 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.040 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.040 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.040 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.040 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.040 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.040 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.040 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.040 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.040 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.040 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.040 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.040 INFO:tasks.workunit.client.0.trial046.stdout: "name": "obj", 2026-02-24T22:12:38.040 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "v1", 2026-02-24T22:12:38.040 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.041 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.041 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.041 INFO:tasks.workunit.client.0.trial046.stdout: "obj3": { 2026-02-24T22:12:38.041 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.041 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.041 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.041 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.041 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.041 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.041 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.041 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.041 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.041 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.041 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.041 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.041 INFO:tasks.workunit.client.0.trial046.stdout: "name": "obj", 2026-02-24T22:12:38.041 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.041 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.041 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.041 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.042 INFO:tasks.workunit.client.0.trial046.stdout: "obj3-2": { 2026-02-24T22:12:38.042 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.042 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.042 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.042 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.042 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.042 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.042 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.042 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.042 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.042 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.042 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.042 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.042 INFO:tasks.workunit.client.0.trial046.stdout: "name": "obj", 2026-02-24T22:12:38.042 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "v1", 2026-02-24T22:12:38.042 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.042 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.042 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.042 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.042 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestRGWObj.no_underscore (0 ms) 2026-02-24T22:12:38.042 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestRGWObj.obj_to_raw 2026-02-24T22:12:38.042 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.042 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.043 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.043 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.043 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.043 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.043 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.043 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.043 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.043 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.043 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.043 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.043 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.043 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.043 INFO:tasks.workunit.client.0.trial046.stdout: "obj": { 2026-02-24T22:12:38.043 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.043 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.043 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.043 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.043 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.043 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.043 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.043 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.043 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.044 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.044 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.044 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.044 INFO:tasks.workunit.client.0.trial046.stdout: "name": "myobj", 2026-02-24T22:12:38.044 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.044 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.044 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.044 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.044 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.044 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.044 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.044 INFO:tasks.workunit.client.0.trial046.stdout: "pool": "default-placement.data", 2026-02-24T22:12:38.044 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker_myobj", 2026-02-24T22:12:38.044 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.044 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.044 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.044 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.044 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.044 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.044 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.044 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.045 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.045 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.045 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.045 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.045 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.045 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.045 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.045 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.045 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.045 INFO:tasks.workunit.client.0.trial046.stdout: "name": "myobj", 2026-02-24T22:12:38.045 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.045 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.045 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.045 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.045 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.045 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.045 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.045 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.045 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.045 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.045 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.045 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.045 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.046 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.046 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.046 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.046 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.046 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.046 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.046 INFO:tasks.workunit.client.0.trial046.stdout: "obj": { 2026-02-24T22:12:38.046 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.046 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.046 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.046 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.046 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.046 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.046 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.046 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.046 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.046 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.046 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.046 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.046 INFO:tasks.workunit.client.0.trial046.stdout: "name": "myobj", 2026-02-24T22:12:38.046 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.046 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.046 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.047 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.047 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.047 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.047 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.047 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.047 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker_myobj", 2026-02-24T22:12:38.047 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.047 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.047 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.047 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.047 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.047 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.047 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.047 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.047 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.047 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.047 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.047 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.047 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.047 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.047 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.047 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.047 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.047 INFO:tasks.workunit.client.0.trial046.stdout: "name": "myobj", 2026-02-24T22:12:38.048 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.048 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.048 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.048 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.048 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.048 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.048 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.048 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.048 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.048 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.048 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.048 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.048 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.048 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.048 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.048 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.048 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.048 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.048 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.048 INFO:tasks.workunit.client.0.trial046.stdout: "obj": { 2026-02-24T22:12:38.048 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.048 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.048 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.048 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.049 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.049 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.049 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.049 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.049 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.049 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.049 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.049 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.049 INFO:tasks.workunit.client.0.trial046.stdout: "name": "myobj", 2026-02-24T22:12:38.049 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.049 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.049 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.049 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.049 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.049 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.049 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.049 INFO:tasks.workunit.client.0.trial046.stdout: "pool": "default-placement.data", 2026-02-24T22:12:38.049 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__ns_myobj", 2026-02-24T22:12:38.049 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.049 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.049 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.049 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.049 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.049 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.050 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.050 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.050 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.050 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.050 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.050 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.050 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.050 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.050 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.050 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.050 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.050 INFO:tasks.workunit.client.0.trial046.stdout: "name": "myobj", 2026-02-24T22:12:38.050 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.050 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.050 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.050 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.050 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.050 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.050 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.050 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.050 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.050 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.050 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.050 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.050 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.050 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.051 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.051 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.051 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.051 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.051 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.051 INFO:tasks.workunit.client.0.trial046.stdout: "obj": { 2026-02-24T22:12:38.051 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.051 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.051 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.051 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.051 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.051 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.051 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.051 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.051 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.051 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.051 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.051 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.051 INFO:tasks.workunit.client.0.trial046.stdout: "name": "myobj", 2026-02-24T22:12:38.051 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.051 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.051 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.051 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.051 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.051 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.051 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__ns_myobj", 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout: "name": "myobj", 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.052 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout: "obj": { 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout: "name": "myobj", 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.053 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.054 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.054 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.054 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.054 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.054 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.054 INFO:tasks.workunit.client.0.trial046.stdout: "pool": "default-placement.data", 2026-02-24T22:12:38.054 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__:inst_myobj", 2026-02-24T22:12:38.054 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.054 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.054 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.054 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.054 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.054 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.054 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.054 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.054 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.054 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.054 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.054 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.054 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.054 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.054 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.054 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.054 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.054 INFO:tasks.workunit.client.0.trial046.stdout: "name": "myobj", 2026-02-24T22:12:38.054 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout: "obj": { 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.055 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout: "name": "myobj", 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__:inst_myobj", 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.056 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout: "name": "myobj", 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout: "obj": { 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.057 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout: "name": "myobj", 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout: "pool": "default-placement.data", 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__ns:inst_myobj", 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.058 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout: "name": "myobj", 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout: "obj": { 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.059 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: "name": "myobj", 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__ns:inst_myobj", 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.060 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: "name": "myobj", 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: "obj": { 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.061 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj", 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: "pool": "default-placement.data", 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker___myobj", 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "marker__myobj" 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj", 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.062 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: "obj": { 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj", 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.063 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker___myobj", 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "marker__myobj" 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj", 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.064 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: "obj": { 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj", 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: "pool": "default-placement.data", 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__ns__myobj", 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.065 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj", 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.066 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: "obj": { 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj", 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__ns__myobj", 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.067 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj", 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: "obj": { 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.068 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj", 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: "pool": "default-placement.data", 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__:inst__myobj", 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "marker__myobj" 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj", 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.069 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: "obj": { 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj", 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.070 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__:inst__myobj", 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "marker__myobj" 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj", 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.071 INFO:tasks.workunit.client.0.trial046.stdout: "obj": { 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj", 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "pool": "default-placement.data", 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__ns:inst__myobj", 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.072 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj", 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "obj": { 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj", 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.073 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__ns:inst__myobj", 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj", 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.074 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "obj": { 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj_", 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "pool": "default-placement.data", 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker___myobj_", 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "marker__myobj_" 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.075 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj_", 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "obj": { 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj_", 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.076 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker___myobj_", 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "marker__myobj_" 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj_", 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "obj": { 2026-02-24T22:12:38.077 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj_", 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "pool": "default-placement.data", 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__ns__myobj_", 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj_", 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.078 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "obj": { 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj_", 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__ns__myobj_", 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.079 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj_", 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "obj": { 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.080 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj_", 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "pool": "default-placement.data", 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__:inst__myobj_", 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "marker__myobj_" 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj_", 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.081 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "obj": { 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj_", 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__:inst__myobj_", 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "marker__myobj_" 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.082 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj_", 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "obj": { 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj_", 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.083 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "pool": "default-placement.data", 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__ns:inst__myobj_", 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "name": "test", 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": "", 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": "" 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj_", 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "obj": { 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.084 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj_", 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__ns:inst__myobj_", 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj_", 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestRGWObj.obj_to_raw (0 ms) 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] TestRGWObj.old_to_raw 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.085 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker_myobj", 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "name": "myobj", 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout:raw=.data-pool:marker_myobj 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj2": { 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker_myobj", 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj2": { 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "name": "myobj", 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.086 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout:raw2=.data-pool:marker_myobj 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__ns_myobj", 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "name": "myobj", 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout:raw=.data-pool:marker__ns_myobj 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj2": { 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__ns_myobj", 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj2": { 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.087 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "name": "myobj", 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout:raw2=.data-pool:marker__ns_myobj 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__:inst_myobj", 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "name": "myobj", 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout:raw=.data-pool:marker__:inst_myobj 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj2": { 2026-02-24T22:12:38.088 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__:inst_myobj", 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj2": { 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "name": "myobj", 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout:raw2=.data-pool:marker__:inst_myobj 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__ns:inst_myobj", 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.089 INFO:tasks.workunit.client.0.trial046.stdout: "name": "myobj", 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout:raw=.data-pool:marker__ns:inst_myobj 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj2": { 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__ns:inst_myobj", 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj2": { 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "name": "myobj", 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout:raw2=.data-pool:marker__ns:inst_myobj 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker___myobj", 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "marker__myobj" 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.090 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj", 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout:raw=.data-pool:marker___myobj 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj2": { 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker___myobj", 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "marker__myobj" 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj2": { 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj", 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout:raw2=.data-pool:marker___myobj 2026-02-24T22:12:38.091 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__ns__myobj", 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj", 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout:raw=.data-pool:marker__ns__myobj 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj2": { 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__ns__myobj", 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj2": { 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.092 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj", 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout:raw2=.data-pool:marker__ns__myobj 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__:inst__myobj", 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "marker__myobj" 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj", 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout:raw=.data-pool:marker__:inst__myobj 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj2": { 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__:inst__myobj", 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "marker__myobj" 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.093 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj2": { 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj", 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout:raw2=.data-pool:marker__:inst__myobj 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__ns:inst__myobj", 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj", 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.094 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout:raw=.data-pool:marker__ns:inst__myobj 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj2": { 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__ns:inst__myobj", 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj2": { 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj", 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout:raw2=.data-pool:marker__ns:inst__myobj 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker___myobj_", 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "marker__myobj_" 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.095 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj_", 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout:raw=.data-pool:marker___myobj_ 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj2": { 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker___myobj_", 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "marker__myobj_" 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj2": { 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj_", 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout:raw2=.data-pool:marker___myobj_ 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.096 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__ns__myobj_", 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj_", 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout:raw=.data-pool:marker__ns__myobj_ 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj2": { 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__ns__myobj_", 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj2": { 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj_", 2026-02-24T22:12:38.097 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "", 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout:raw2=.data-pool:marker__ns__myobj_ 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__:inst__myobj_", 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "marker__myobj_" 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj_", 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout:raw=.data-pool:marker__:inst__myobj_ 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj2": { 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__:inst__myobj_", 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "marker__myobj_" 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj2": { 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.098 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj_", 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "" 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout:raw2=.data-pool:marker__:inst__myobj_ 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj": { 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__ns:inst__myobj_", 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj": { 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj_", 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout:raw=.data-pool:marker__ns:inst__myobj_ 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.099 INFO:tasks.workunit.client.0.trial046.stdout: "raw_obj2": { 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout: "pool": ".data-pool", 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout: "oid": "marker__ns:inst__myobj_", 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout: "loc": "" 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout: "new_obj2": { 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout: "bucket": { 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout: "name": "ebtest", 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout: "marker": "marker", 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_id": "bucket-id", 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout: "tenant": "", 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout: "explicit_placement": { 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout: "data_pool": ".data-pool", 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout: "data_extra_pool": "", 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout: "index_pool": ".index-pool" 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout: "key": { 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout: "name": "_myobj_", 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout: "instance": "inst", 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout: "ns": "ns" 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout:raw2=.data-pool:marker__ns:inst__myobj_ 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] TestRGWObj.old_to_raw (0 ms) 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 4 tests from TestRGWObj (0 ms total) 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment tear-down 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout:[==========] 4 tests from 1 test suite ran. (0 ms total) 2026-02-24T22:12:38.100 INFO:tasks.workunit.client.0.trial046.stdout:[ PASSED ] 4 tests. 2026-02-24T22:12:38.100 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T22:12:38.100 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T22:12:38.107 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_datalog.sh on client.0... 2026-02-24T22:12:38.108 INFO:tasks.workunit:Running workunit rgw/test_rgw_datalog.sh... 2026-02-24T22:12:38.108 DEBUG:teuthology.orchestra.run.trial046:workunit test rgw/test_rgw_datalog.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7e05f4fdea5af173284f247d6b0935b337d4ccf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_datalog.sh 2026-02-24T22:12:38.160 INFO:tasks.workunit.client.0.trial046.stdout:Running main() from gmock_main.cc 2026-02-24T22:12:38.160 INFO:tasks.workunit.client.0.trial046.stdout:[==========] Running 10 tests from 4 test suites. 2026-02-24T22:12:38.160 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment set-up. 2026-02-24T22:12:38.160 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 1 test from DataLogBG 2026-02-24T22:12:38.160 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] DataLogBG.TestRoundTrip 2026-02-24T22:12:38.161 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] DataLogBG.TestRoundTrip (0 ms) 2026-02-24T22:12:38.161 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 1 test from DataLogBG (0 ms total) 2026-02-24T22:12:38.161 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:12:38.161 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 4 tests from DataLog 2026-02-24T22:12:38.161 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] DataLog.TestSem 2026-02-24T22:12:40.896 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] DataLog.TestSem (2735 ms) 2026-02-24T22:12:40.896 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] DataLog.SimpleRecovery 2026-02-24T22:12:43.913 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] DataLog.SimpleRecovery (3016 ms) 2026-02-24T22:12:43.913 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] DataLog.CycleRecovery 2026-02-24T22:12:46.925 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] DataLog.CycleRecovery (3012 ms) 2026-02-24T22:12:46.926 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] DataLog.SemaphoresRecovery 2026-02-24T22:12:49.939 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] DataLog.SemaphoresRecovery (3013 ms) 2026-02-24T22:12:49.939 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 4 tests from DataLog (11778 ms total) 2026-02-24T22:12:49.940 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:12:49.940 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 1 test from DataLogWatchless 2026-02-24T22:12:49.940 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] DataLogWatchless.NotWatching 2026-02-24T22:12:52.951 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] DataLogWatchless.NotWatching (3011 ms) 2026-02-24T22:12:52.951 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 1 test from DataLogWatchless (3011 ms total) 2026-02-24T22:12:52.951 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:12:52.951 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 4 tests from DataLogBulky 2026-02-24T22:12:52.951 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] DataLogBulky.TestSemBulky 2026-02-24T22:12:55.968 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] DataLogBulky.TestSemBulky (3017 ms) 2026-02-24T22:12:55.969 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] DataLogBulky.BulkyRecovery 2026-02-24T22:12:58.981 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] DataLogBulky.BulkyRecovery (3012 ms) 2026-02-24T22:12:58.981 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] DataLogBulky.BulkyCycleRecovery 2026-02-24T22:13:01.995 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] DataLogBulky.BulkyCycleRecovery (3014 ms) 2026-02-24T22:13:01.996 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] DataLogBulky.BulkySemaphoresRecovery 2026-02-24T22:13:05.005 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] DataLogBulky.BulkySemaphoresRecovery (3010 ms) 2026-02-24T22:13:05.006 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 4 tests from DataLogBulky (12055 ms total) 2026-02-24T22:13:05.006 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:13:05.006 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment tear-down 2026-02-24T22:13:05.006 INFO:tasks.workunit.client.0.trial046.stdout:[==========] 10 tests from 4 test suites ran. (26845 ms total) 2026-02-24T22:13:05.006 INFO:tasks.workunit.client.0.trial046.stdout:[ PASSED ] 10 tests. 2026-02-24T22:13:05.012 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T22:13:05.012 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T22:13:05.061 INFO:tasks.workunit:Running workunits matching rgw/test_librgw_file.sh on client.0... 2026-02-24T22:13:05.064 INFO:tasks.workunit:Running workunit rgw/test_librgw_file.sh... 2026-02-24T22:13:05.064 DEBUG:teuthology.orchestra.run.trial046:workunit test rgw/test_librgw_file.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7e05f4fdea5af173284f247d6b0935b337d4ccf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_librgw_file.sh 2026-02-24T22:13:05.134 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:13:05.135 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:13:05.152 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.150+0000 7f2dfbf9fa80 4 Realm: () 2026-02-24T22:13:05.152 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.150+0000 7f2dfbf9fa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:13:05.152 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.150+0000 7f2dfbf9fa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:13:05.152 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.150+0000 7f2dfbf9fa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:13:05.152 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.150+0000 7f2dfbf9fa80 20 zonegroup default 2026-02-24T22:13:05.153 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.154+0000 7f2dfbf9fa80 20 rados->read ofs=0 len=0 2026-02-24T22:13:05.359 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.358+0000 7f2dfbf9fa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:13:05.359 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.358+0000 7f2dfbf9fa80 20 rados->read ofs=0 len=0 2026-02-24T22:13:05.360 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.358+0000 7f2dfbf9fa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:13:05.360 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.358+0000 7f2dfbf9fa80 20 started sync module instance, tier type = 2026-02-24T22:13:05.360 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.358+0000 7f2dfbf9fa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:13:05.360 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.362+0000 7f2df9272640 0 -- 10.20.193.46:0/4018519777 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x564641fd0e50 msgr2=0x564641ff12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:13:05.362 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.362+0000 7f2dfbf9fa80 20 add_watcher() i=3 2026-02-24T22:13:05.362 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.362+0000 7f2dfbf9fa80 20 add_watcher() i=1 2026-02-24T22:13:05.362 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.362+0000 7f2dfbf9fa80 20 add_watcher() i=6 2026-02-24T22:13:05.363 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.362+0000 7f2dfbf9fa80 20 add_watcher() i=0 2026-02-24T22:13:05.363 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.362+0000 7f2dfbf9fa80 20 add_watcher() i=2 2026-02-24T22:13:05.363 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.362+0000 7f2dfbf9fa80 20 add_watcher() i=5 2026-02-24T22:13:05.363 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.362+0000 7f2dfbf9fa80 20 add_watcher() i=7 2026-02-24T22:13:05.568 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.566+0000 7f2dfbf9fa80 20 add_watcher() i=4 2026-02-24T22:13:05.568 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.566+0000 7f2dfbf9fa80 2 all 8 watchers are set, enabling cache 2026-02-24T22:13:05.569 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.570+0000 7f2df9272640 0 -- 10.20.193.46:0/536968901 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f2d18008af0 msgr2=0x7f2d18028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:13:05.571 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.570+0000 7f2dfbf9fa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:13:05.571 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.570+0000 7f2dfbf9fa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:13:05.571 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.570+0000 7f2dfbf9fa80 5 note: GC not initialized 2026-02-24T22:13:05.571 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.570+0000 7f2d2a7fc640 20 reqs_thread_entry: start 2026-02-24T22:13:05.810 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.810+0000 7f2dfbf9fa80 20 init_complete bucket index max shards: 1 2026-02-24T22:13:05.811 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.810+0000 7f2dfbf9fa80 20 Filter name: none 2026-02-24T22:13:05.811 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.810+0000 7f2d28ff9640 20 reqs_thread_entry: start 2026-02-24T22:13:05.811 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.810+0000 7f2dfbf9fa80 10 cache get: name=default.rgw.meta+users.uid+ceph-test-librgw-file : miss 2026-02-24T22:13:05.811 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.810+0000 7f2dfbf9fa80 20 rados->read ofs=0 len=0 2026-02-24T22:13:05.811 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.810+0000 7f2dfbf9fa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:13:05.811 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.810+0000 7f2dfbf9fa80 10 cache put: name=default.rgw.meta+users.uid+ceph-test-librgw-file info.flags=0x0 2026-02-24T22:13:05.811 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.810+0000 7f2dfbf9fa80 10 adding default.rgw.meta+users.uid+ceph-test-librgw-file to cache LRU end 2026-02-24T22:13:05.811 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.810+0000 7f2dfbf9fa80 10 cache get: name=default.rgw.meta+users.email+librgw@example.com : miss 2026-02-24T22:13:05.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.810+0000 7f2dfbf9fa80 20 rados->read ofs=0 len=0 2026-02-24T22:13:05.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.810+0000 7f2dfbf9fa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:13:05.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.810+0000 7f2dfbf9fa80 10 cache put: name=default.rgw.meta+users.email+librgw@example.com info.flags=0x0 2026-02-24T22:13:05.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.810+0000 7f2dfbf9fa80 10 adding default.rgw.meta+users.email+librgw@example.com to cache LRU end 2026-02-24T22:13:05.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.810+0000 7f2dfbf9fa80 10 cache get: name=default.rgw.meta+users.keys+WtCxft5UPJMpM9VLVp3DFIDX1tU= : miss 2026-02-24T22:13:05.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.810+0000 7f2dfbf9fa80 20 rados->read ofs=0 len=0 2026-02-24T22:13:05.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.810+0000 7f2dfbf9fa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:13:05.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.810+0000 7f2dfbf9fa80 10 cache put: name=default.rgw.meta+users.keys+WtCxft5UPJMpM9VLVp3DFIDX1tU= info.flags=0x0 2026-02-24T22:13:05.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.810+0000 7f2dfbf9fa80 10 adding default.rgw.meta+users.keys+WtCxft5UPJMpM9VLVp3DFIDX1tU= to cache LRU end 2026-02-24T22:13:05.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.810+0000 7f2dfbf9fa80 10 cache get: name=default.rgw.meta+users.keys+WtCxft5UPJMpM9VLVp3DFIDX1tU= : hit (negative entry) 2026-02-24T22:13:05.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.810+0000 7f2dfbf9fa80 10 cache get: name=default.rgw.meta+users.keys+WtCxft5UPJMpM9VLVp3DFIDX1tU= : hit (negative entry) 2026-02-24T22:13:05.813 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.810+0000 7f2dfbf9fa80 10 cache put: name=default.rgw.meta+users.uid+ceph-test-librgw-file info.flags=0x17 2026-02-24T22:13:05.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.810+0000 7f2dfbf9fa80 10 moving default.rgw.meta+users.uid+ceph-test-librgw-file to cache LRU end 2026-02-24T22:13:05.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.810+0000 7f2dfbf9fa80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:ceph-test-librgw-file, ofs0, ns] 2026-02-24T22:13:05.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.810+0000 7f2dd4ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1236950581252 cookie 94859755626224 notifier 5769 bl.length()=697 2026-02-24T22:13:05.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.810+0000 7f2dd4ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ceph-test-librgw-file info.flags=0x17 2026-02-24T22:13:05.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.810+0000 7f2dd4ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ceph-test-librgw-file to cache LRU end 2026-02-24T22:13:05.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.814+0000 7f2dfbf9fa80 10 cache put: name=default.rgw.meta+users.email+librgw@example.com info.flags=0x7 2026-02-24T22:13:05.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.814+0000 7f2dfbf9fa80 10 moving default.rgw.meta+users.email+librgw@example.com to cache LRU end 2026-02-24T22:13:05.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.814+0000 7f2dfbf9fa80 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.email:librgw@example.com, ofs0, ns] 2026-02-24T22:13:05.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.814+0000 7f2dd4ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1236950581258 cookie 94859754259952 notifier 5769 bl.length()=194 2026-02-24T22:13:05.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.814+0000 7f2dd4ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+librgw@example.com info.flags=0x7 2026-02-24T22:13:05.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.814+0000 7f2dd4ff9640 10 rgw watcher librados: moving default.rgw.meta+users.email+librgw@example.com to cache LRU end 2026-02-24T22:13:05.815 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.814+0000 7f2dfbf9fa80 10 cache put: name=default.rgw.meta+users.keys+WtCxft5UPJMpM9VLVp3DFIDX1tU= info.flags=0x7 2026-02-24T22:13:05.815 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.814+0000 7f2dfbf9fa80 10 moving default.rgw.meta+users.keys+WtCxft5UPJMpM9VLVp3DFIDX1tU= to cache LRU end 2026-02-24T22:13:05.815 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.814+0000 7f2dfbf9fa80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.keys:WtCxft5UPJMpM9VLVp3DFIDX1tU=, ofs0, ns] 2026-02-24T22:13:05.815 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.814+0000 7f2dd57fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1236950581265 cookie 94859755620640 notifier 5769 bl.length()=203 2026-02-24T22:13:05.815 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.814+0000 7f2dd57fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+WtCxft5UPJMpM9VLVp3DFIDX1tU= info.flags=0x7 2026-02-24T22:13:05.816 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.814+0000 7f2dd57fa640 10 rgw watcher librados: moving default.rgw.meta+users.keys+WtCxft5UPJMpM9VLVp3DFIDX1tU= to cache LRU end 2026-02-24T22:13:05.816 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:13:05.816 INFO:tasks.workunit.client.0.trial046.stdout: "user_id": "ceph-test-librgw-file", 2026-02-24T22:13:05.816 INFO:tasks.workunit.client.0.trial046.stdout: "display_name": "librgw test user", 2026-02-24T22:13:05.816 INFO:tasks.workunit.client.0.trial046.stdout: "email": "librgw@example.com", 2026-02-24T22:13:05.816 INFO:tasks.workunit.client.0.trial046.stdout: "suspended": 0, 2026-02-24T22:13:05.816 INFO:tasks.workunit.client.0.trial046.stdout: "max_buckets": 1000, 2026-02-24T22:13:05.816 INFO:tasks.workunit.client.0.trial046.stdout: "subusers": [], 2026-02-24T22:13:05.817 INFO:tasks.workunit.client.0.trial046.stdout: "keys": [ 2026-02-24T22:13:05.817 INFO:tasks.workunit.client.0.trial046.stdout: { 2026-02-24T22:13:05.817 INFO:tasks.workunit.client.0.trial046.stdout: "user": "ceph-test-librgw-file", 2026-02-24T22:13:05.817 INFO:tasks.workunit.client.0.trial046.stdout: "access_key": "WtCxft5UPJMpM9VLVp3DFIDX1tU=", 2026-02-24T22:13:05.817 INFO:tasks.workunit.client.0.trial046.stdout: "secret_key": "fG0JMvNRBQTwhl1RAepyGjVKMuG2erHlor6+fxmklh+h4GxthiNYyQ==", 2026-02-24T22:13:05.817 INFO:tasks.workunit.client.0.trial046.stdout: "active": true, 2026-02-24T22:13:05.817 INFO:tasks.workunit.client.0.trial046.stdout: "create_date": "2026-02-24T22:13:05.814108Z" 2026-02-24T22:13:05.817 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:13:05.817 INFO:tasks.workunit.client.0.trial046.stdout: ], 2026-02-24T22:13:05.817 INFO:tasks.workunit.client.0.trial046.stdout: "swift_keys": [], 2026-02-24T22:13:05.817 INFO:tasks.workunit.client.0.trial046.stdout: "caps": [], 2026-02-24T22:13:05.818 INFO:tasks.workunit.client.0.trial046.stdout: "op_mask": "read, write, delete", 2026-02-24T22:13:05.818 INFO:tasks.workunit.client.0.trial046.stdout: "default_placement": "", 2026-02-24T22:13:05.818 INFO:tasks.workunit.client.0.trial046.stdout: "default_storage_class": "", 2026-02-24T22:13:05.818 INFO:tasks.workunit.client.0.trial046.stdout: "placement_tags": [], 2026-02-24T22:13:05.818 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_quota": { 2026-02-24T22:13:05.818 INFO:tasks.workunit.client.0.trial046.stdout: "enabled": false, 2026-02-24T22:13:05.818 INFO:tasks.workunit.client.0.trial046.stdout: "check_on_raw": false, 2026-02-24T22:13:05.818 INFO:tasks.workunit.client.0.trial046.stdout: "max_size": -1, 2026-02-24T22:13:05.818 INFO:tasks.workunit.client.0.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:13:05.818 INFO:tasks.workunit.client.0.trial046.stdout: "max_objects": -1 2026-02-24T22:13:05.818 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:13:05.818 INFO:tasks.workunit.client.0.trial046.stdout: "user_quota": { 2026-02-24T22:13:05.819 INFO:tasks.workunit.client.0.trial046.stdout: "enabled": false, 2026-02-24T22:13:05.819 INFO:tasks.workunit.client.0.trial046.stdout: "check_on_raw": false, 2026-02-24T22:13:05.819 INFO:tasks.workunit.client.0.trial046.stdout: "max_size": -1, 2026-02-24T22:13:05.819 INFO:tasks.workunit.client.0.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:13:05.819 INFO:tasks.workunit.client.0.trial046.stdout: "max_objects": -1 2026-02-24T22:13:05.819 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:13:05.819 INFO:tasks.workunit.client.0.trial046.stdout: "temp_url_keys": [], 2026-02-24T22:13:05.819 INFO:tasks.workunit.client.0.trial046.stdout: "type": "rgw", 2026-02-24T22:13:05.819 INFO:tasks.workunit.client.0.trial046.stdout: "mfa_ids": [], 2026-02-24T22:13:05.819 INFO:tasks.workunit.client.0.trial046.stdout: "account_id": "", 2026-02-24T22:13:05.819 INFO:tasks.workunit.client.0.trial046.stdout: "path": "/", 2026-02-24T22:13:05.819 INFO:tasks.workunit.client.0.trial046.stdout: "create_date": "2026-02-24T22:13:05.814098Z", 2026-02-24T22:13:05.819 INFO:tasks.workunit.client.0.trial046.stdout: "tags": [], 2026-02-24T22:13:05.820 INFO:tasks.workunit.client.0.trial046.stdout: "group_ids": [] 2026-02-24T22:13:05.820 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:13:05.820 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:13:05.821 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.818+0000 7f2dfbf9fa80 20 remove_watcher() i=3 2026-02-24T22:13:05.821 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.818+0000 7f2dfbf9fa80 2 removed watcher, disabling cache 2026-02-24T22:13:05.821 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.818+0000 7f2dfbf9fa80 20 remove_watcher() i=2 2026-02-24T22:13:05.821 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.818+0000 7f2dfbf9fa80 20 remove_watcher() i=6 2026-02-24T22:13:05.821 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.818+0000 7f2dfbf9fa80 20 remove_watcher() i=4 2026-02-24T22:13:05.821 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.818+0000 7f2dfbf9fa80 20 remove_watcher() i=1 2026-02-24T22:13:05.821 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.818+0000 7f2dfbf9fa80 20 remove_watcher() i=7 2026-02-24T22:13:05.821 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.818+0000 7f2dfbf9fa80 20 remove_watcher() i=5 2026-02-24T22:13:05.822 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:05.818+0000 7f2dfbf9fa80 20 remove_watcher() i=0 2026-02-24T22:13:05.824 INFO:tasks.workunit.client.0.trial046.stdout:phase 1.1 2026-02-24T22:13:05.837 INFO:tasks.workunit.client.0.trial046.stdout:[==========] Running 24 tests from 1 test suite. 2026-02-24T22:13:05.838 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment set-up. 2026-02-24T22:13:05.838 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 24 tests from LibRGW 2026-02-24T22:13:05.838 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.INIT 2026-02-24T22:13:05.842 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:13:05.842 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:13:05.968 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.INIT (130 ms) 2026-02-24T22:13:05.969 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MOUNT_NOROOT 2026-02-24T22:13:05.969 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MOUNT_NOROOT (0 ms) 2026-02-24T22:13:05.969 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MOUNT 2026-02-24T22:13:05.969 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2026-02-24T22:13:05.970 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.SETUP_HIER1 2026-02-24T22:13:06.003 INFO:tasks.workunit.client.0.trial046.stdout:creating: nfsroot:foo/bar/baz/quux 2026-02-24T22:13:06.006 INFO:tasks.workunit.client.0.trial046.stdout:creating: nfsroot:foo/f1 2026-02-24T22:13:06.009 INFO:tasks.workunit.client.0.trial046.stdout:creating: nfsroot:foo/f2 2026-02-24T22:13:06.011 INFO:tasks.workunit.client.0.trial046.stdout:creating: nfsroot:foo/bar/f1 2026-02-24T22:13:06.012 INFO:tasks.workunit.client.0.trial046.stdout:creating: nfsroot:foo/bar/d1/ 2026-02-24T22:13:06.014 INFO:tasks.workunit.client.0.trial046.stdout:creating: nfsroot:foo/bar/baz/hungry 2026-02-24T22:13:06.016 INFO:tasks.workunit.client.0.trial046.stdout:creating: nfsroot:foo/bar/baz/hungry/ 2026-02-24T22:13:06.018 INFO:tasks.workunit.client.0.trial046.stdout:creating: nfsroot:foo/bar/baz/momma 2026-02-24T22:13:06.020 INFO:tasks.workunit.client.0.trial046.stdout:creating: nfsroot:foo/bar/baz/bear/ 2026-02-24T22:13:06.022 INFO:tasks.workunit.client.0.trial046.stdout:creating: nfsroot:foo/bar/baz/sasquatch 2026-02-24T22:13:06.024 INFO:tasks.workunit.client.0.trial046.stdout:creating: nfsroot:foo/bar/baz/sasquatch/ 2026-02-24T22:13:06.026 INFO:tasks.workunit.client.0.trial046.stdout:creating: nfsroot:foo/bar/baz/frobozz 2026-02-24T22:13:06.028 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.SETUP_HIER1 (58 ms) 2026-02-24T22:13:06.028 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.SETUP_DIRS1 2026-02-24T22:13:06.098 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.SETUP_DIRS1 (69 ms) 2026-02-24T22:13:06.098 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.SETATTR 2026-02-24T22:13:06.098 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.SETATTR (0 ms) 2026-02-24T22:13:06.098 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.RGW_CREATE_DIRS1 2026-02-24T22:13:06.108 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.RGW_CREATE_DIRS1 (10 ms) 2026-02-24T22:13:06.108 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.RGW_SETUP_RENAME1 2026-02-24T22:13:06.134 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.RGW_SETUP_RENAME1 (26 ms) 2026-02-24T22:13:06.135 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.RGW_INTRABUCKET_RENAME1 2026-02-24T22:13:06.135 INFO:tasks.workunit.client.0.trial046.stdout:rename file rfile_0 to rfile_r0 (bucket brename0) 2026-02-24T22:13:06.140 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.RGW_INTRABUCKET_RENAME1 (4 ms) 2026-02-24T22:13:06.140 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.RGW_CROSSBUCKET_RENAME1 2026-02-24T22:13:06.141 INFO:tasks.workunit.client.0.trial046.stdout:rename file rfile_1 (bucket brename0) to rfile_rhilldog (bucket brename1) 2026-02-24T22:13:06.142 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.RGW_CROSSBUCKET_RENAME1 (3 ms) 2026-02-24T22:13:06.143 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.GETATTR_DIRS1 2026-02-24T22:13:06.143 INFO:tasks.workunit.client.0.trial046.stdout:scanning objects in /bdirs1/dir_0 2026-02-24T22:13:06.143 INFO:tasks.workunit.client.0.trial046.stdout: /bdirs1/dir_0/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2026-02-24T22:13:06.143 INFO:tasks.workunit.client.0.trial046.stdout: /bdirs1/dir_0/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2026-02-24T22:13:06.143 INFO:tasks.workunit.client.0.trial046.stdout: /bdirs1/dir_0/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2026-02-24T22:13:06.143 INFO:tasks.workunit.client.0.trial046.stdout: /bdirs1/dir_0/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2026-02-24T22:13:06.143 INFO:tasks.workunit.client.0.trial046.stdout:scanning objects in /bdirs1/dir_1 2026-02-24T22:13:06.143 INFO:tasks.workunit.client.0.trial046.stdout: /bdirs1/dir_1/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2026-02-24T22:13:06.143 INFO:tasks.workunit.client.0.trial046.stdout: /bdirs1/dir_1/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2026-02-24T22:13:06.143 INFO:tasks.workunit.client.0.trial046.stdout: /bdirs1/dir_1/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2026-02-24T22:13:06.143 INFO:tasks.workunit.client.0.trial046.stdout: /bdirs1/dir_1/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2026-02-24T22:13:06.143 INFO:tasks.workunit.client.0.trial046.stdout:scanning objects in /bdirs1/dir_2 2026-02-24T22:13:06.143 INFO:tasks.workunit.client.0.trial046.stdout: /bdirs1/dir_2/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2026-02-24T22:13:06.143 INFO:tasks.workunit.client.0.trial046.stdout: /bdirs1/dir_2/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2026-02-24T22:13:06.143 INFO:tasks.workunit.client.0.trial046.stdout: /bdirs1/dir_2/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2026-02-24T22:13:06.143 INFO:tasks.workunit.client.0.trial046.stdout: /bdirs1/dir_2/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2026-02-24T22:13:06.143 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.GETATTR_DIRS1 (0 ms) 2026-02-24T22:13:06.143 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.READ_DIRS1 2026-02-24T22:13:06.143 INFO:tasks.workunit.client.0.trial046.stdout:read back objects in /bdirs1/dir_0 2026-02-24T22:13:06.143 INFO:tasks.workunit.client.0.trial046.stdout:reading 0,256 dir_0/sfile_0 2026-02-24T22:13:06.144 INFO:tasks.workunit.client.0.trial046.stdout: read back from sfile_0 : "data for sfile_0" 2026-02-24T22:13:06.144 INFO:tasks.workunit.client.0.trial046.stdout:reading 0,256 dir_0/sfile_1 2026-02-24T22:13:06.144 INFO:tasks.workunit.client.0.trial046.stdout: read back from sfile_1 : "data for sfile_1" 2026-02-24T22:13:06.144 INFO:tasks.workunit.client.0.trial046.stdout:read back objects in /bdirs1/dir_1 2026-02-24T22:13:06.144 INFO:tasks.workunit.client.0.trial046.stdout:reading 0,256 dir_1/sfile_0 2026-02-24T22:13:06.145 INFO:tasks.workunit.client.0.trial046.stdout: read back from sfile_0 : "data for sfile_0" 2026-02-24T22:13:06.145 INFO:tasks.workunit.client.0.trial046.stdout:reading 0,256 dir_1/sfile_1 2026-02-24T22:13:06.146 INFO:tasks.workunit.client.0.trial046.stdout: read back from sfile_1 : "data for sfile_1" 2026-02-24T22:13:06.146 INFO:tasks.workunit.client.0.trial046.stdout:read back objects in /bdirs1/dir_2 2026-02-24T22:13:06.146 INFO:tasks.workunit.client.0.trial046.stdout:reading 0,256 dir_2/sfile_0 2026-02-24T22:13:06.147 INFO:tasks.workunit.client.0.trial046.stdout: read back from sfile_0 : "data for sfile_0" 2026-02-24T22:13:06.147 INFO:tasks.workunit.client.0.trial046.stdout:reading 0,256 dir_2/sfile_1 2026-02-24T22:13:06.147 INFO:tasks.workunit.client.0.trial046.stdout: read back from sfile_1 : "data for sfile_1" 2026-02-24T22:13:06.147 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.READ_DIRS1 (4 ms) 2026-02-24T22:13:06.147 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.READF_SETUP1 2026-02-24T22:13:06.261 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.READF_SETUP1 (113 ms) 2026-02-24T22:13:06.261 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.READF_DIRS1 2026-02-24T22:13:06.261 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.READF_DIRS1 (0 ms) 2026-02-24T22:13:06.261 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.WRITEF_DIRS1 2026-02-24T22:13:06.261 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.WRITEF_DIRS1 (0 ms) 2026-02-24T22:13:06.262 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.RELEASE_DIRS1 2026-02-24T22:13:06.262 INFO:tasks.workunit.client.0.trial046.stdout:release sdir_0 type: DIR refs: 2 2026-02-24T22:13:06.262 INFO:tasks.workunit.client.0.trial046.stdout:release sfile_0 type: FILE refs: 2 2026-02-24T22:13:06.262 INFO:tasks.workunit.client.0.trial046.stdout:release sdir_1 type: DIR refs: 2 2026-02-24T22:13:06.262 INFO:tasks.workunit.client.0.trial046.stdout:release sfile_1 type: FILE refs: 2 2026-02-24T22:13:06.262 INFO:tasks.workunit.client.0.trial046.stdout:release sdir_0 type: DIR refs: 2 2026-02-24T22:13:06.262 INFO:tasks.workunit.client.0.trial046.stdout:release sfile_0 type: FILE refs: 2 2026-02-24T22:13:06.262 INFO:tasks.workunit.client.0.trial046.stdout:release sdir_1 type: DIR refs: 2 2026-02-24T22:13:06.262 INFO:tasks.workunit.client.0.trial046.stdout:release sfile_1 type: FILE refs: 2 2026-02-24T22:13:06.262 INFO:tasks.workunit.client.0.trial046.stdout:release sdir_0 type: DIR refs: 2 2026-02-24T22:13:06.262 INFO:tasks.workunit.client.0.trial046.stdout:release sfile_0 type: FILE refs: 2 2026-02-24T22:13:06.262 INFO:tasks.workunit.client.0.trial046.stdout:release sdir_1 type: DIR refs: 2 2026-02-24T22:13:06.262 INFO:tasks.workunit.client.0.trial046.stdout:release sfile_1 type: FILE refs: 2 2026-02-24T22:13:06.263 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.RELEASE_DIRS1 (0 ms) 2026-02-24T22:13:06.263 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.HIER1 2026-02-24T22:13:06.265 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/f2 (f2): FILE 2026-02-24T22:13:06.266 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/f1 (f1): FILE 2026-02-24T22:13:06.267 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/bar/f1 (f1): FILE 2026-02-24T22:13:06.268 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/bar/d1 (d1): DIR 2026-02-24T22:13:06.269 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2026-02-24T22:13:06.270 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2026-02-24T22:13:06.270 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/bar/baz/quux (quux): FILE 2026-02-24T22:13:06.270 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/bar/baz/momma (momma): FILE 2026-02-24T22:13:06.271 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2026-02-24T22:13:06.271 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2026-02-24T22:13:06.271 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/bar/baz/frobozz (frobozz): FILE 2026-02-24T22:13:06.272 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/bar/baz/bear (bear): DIR 2026-02-24T22:13:06.272 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/bar/baz (baz): DIR 2026-02-24T22:13:06.273 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/bar (bar): DIR 2026-02-24T22:13:06.273 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo (foo): DIR 2026-02-24T22:13:06.273 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot (nfsroot): DIR 2026-02-24T22:13:06.273 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.HIER1 (11 ms) 2026-02-24T22:13:06.273 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2026-02-24T22:13:06.273 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (0 ms) 2026-02-24T22:13:06.273 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2026-02-24T22:13:06.273 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2026-02-24T22:13:06.273 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2026-02-24T22:13:06.273 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2026-02-24T22:13:06.273 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2026-02-24T22:13:06.273 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2026-02-24T22:13:06.274 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.CLEANUP 2026-02-24T22:13:06.274 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2026-02-24T22:13:06.274 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.UMOUNT 2026-02-24T22:13:06.274 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2026-02-24T22:13:06.274 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.SHUTDOWN 2026-02-24T22:13:06.274 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:06.274+0000 7f7300fa5b80 -1 shutting down 2026-02-24T22:13:06.274 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:06.274+0000 7f71dacd6640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 29741, si_uid : 1000, si_addr0x3e80000742d, si_status0 2026-02-24T22:13:07.939 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.SHUTDOWN (1666 ms) 2026-02-24T22:13:07.939 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 24 tests from LibRGW (2101 ms total) 2026-02-24T22:13:07.939 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:13:07.939 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment tear-down 2026-02-24T22:13:07.939 INFO:tasks.workunit.client.0.trial046.stdout:[==========] 24 tests from 1 test suite ran. (2101 ms total) 2026-02-24T22:13:07.939 INFO:tasks.workunit.client.0.trial046.stdout:[ PASSED ] 24 tests. 2026-02-24T22:13:07.948 INFO:tasks.workunit.client.0.trial046.stdout:phase 1.2 2026-02-24T22:13:07.959 INFO:tasks.workunit.client.0.trial046.stdout:[==========] Running 24 tests from 1 test suite. 2026-02-24T22:13:07.959 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment set-up. 2026-02-24T22:13:07.959 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 24 tests from LibRGW 2026-02-24T22:13:07.959 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.INIT 2026-02-24T22:13:07.963 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:13:07.963 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:13:08.088 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.INIT (128 ms) 2026-02-24T22:13:08.088 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MOUNT_NOROOT 2026-02-24T22:13:08.090 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MOUNT_NOROOT (0 ms) 2026-02-24T22:13:08.090 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MOUNT 2026-02-24T22:13:08.090 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2026-02-24T22:13:08.090 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.SETUP_HIER1 2026-02-24T22:13:08.090 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.SETUP_HIER1 (0 ms) 2026-02-24T22:13:08.090 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.SETUP_DIRS1 2026-02-24T22:13:08.090 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.SETUP_DIRS1 (0 ms) 2026-02-24T22:13:08.091 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.SETATTR 2026-02-24T22:13:08.091 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.SETATTR (0 ms) 2026-02-24T22:13:08.091 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.RGW_CREATE_DIRS1 2026-02-24T22:13:08.091 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.RGW_CREATE_DIRS1 (0 ms) 2026-02-24T22:13:08.091 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.RGW_SETUP_RENAME1 2026-02-24T22:13:08.091 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.RGW_SETUP_RENAME1 (0 ms) 2026-02-24T22:13:08.091 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.RGW_INTRABUCKET_RENAME1 2026-02-24T22:13:08.091 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.RGW_INTRABUCKET_RENAME1 (0 ms) 2026-02-24T22:13:08.091 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.RGW_CROSSBUCKET_RENAME1 2026-02-24T22:13:08.091 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.RGW_CROSSBUCKET_RENAME1 (0 ms) 2026-02-24T22:13:08.091 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.GETATTR_DIRS1 2026-02-24T22:13:08.091 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.GETATTR_DIRS1 (0 ms) 2026-02-24T22:13:08.092 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.READ_DIRS1 2026-02-24T22:13:08.092 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.READ_DIRS1 (0 ms) 2026-02-24T22:13:08.092 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.READF_SETUP1 2026-02-24T22:13:08.092 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.READF_SETUP1 (0 ms) 2026-02-24T22:13:08.092 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.READF_DIRS1 2026-02-24T22:13:08.092 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.READF_DIRS1 (0 ms) 2026-02-24T22:13:08.092 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.WRITEF_DIRS1 2026-02-24T22:13:08.092 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.WRITEF_DIRS1 (0 ms) 2026-02-24T22:13:08.092 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.RELEASE_DIRS1 2026-02-24T22:13:08.092 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.RELEASE_DIRS1 (0 ms) 2026-02-24T22:13:08.092 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.HIER1 2026-02-24T22:13:08.092 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.HIER1 (0 ms) 2026-02-24T22:13:08.092 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2026-02-24T22:13:08.093 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (0 ms) 2026-02-24T22:13:08.093 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2026-02-24T22:13:08.093 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2026-02-24T22:13:08.093 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2026-02-24T22:13:08.093 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2026-02-24T22:13:08.093 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2026-02-24T22:13:08.093 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2026-02-24T22:13:08.093 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.CLEANUP 2026-02-24T22:13:08.093 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2026-02-24T22:13:08.093 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.UMOUNT 2026-02-24T22:13:08.093 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2026-02-24T22:13:08.093 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.SHUTDOWN 2026-02-24T22:13:08.094 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:08.090+0000 7fda49168b80 -1 shutting down 2026-02-24T22:13:08.094 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:08.090+0000 7fd922e99640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 30398, si_uid : 1000, si_addr0x3e8000076be, si_status0 2026-02-24T22:13:10.063 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.SHUTDOWN (1973 ms) 2026-02-24T22:13:10.063 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 24 tests from LibRGW (2103 ms total) 2026-02-24T22:13:10.063 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:13:10.063 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment tear-down 2026-02-24T22:13:10.063 INFO:tasks.workunit.client.0.trial046.stdout:[==========] 24 tests from 1 test suite ran. (2103 ms total) 2026-02-24T22:13:10.063 INFO:tasks.workunit.client.0.trial046.stdout:[ PASSED ] 24 tests. 2026-02-24T22:13:10.071 INFO:tasks.workunit.client.0.trial046.stdout:phase 1.3 2026-02-24T22:13:10.083 INFO:tasks.workunit.client.0.trial046.stdout:[==========] Running 24 tests from 1 test suite. 2026-02-24T22:13:10.083 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment set-up. 2026-02-24T22:13:10.083 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 24 tests from LibRGW 2026-02-24T22:13:10.083 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.INIT 2026-02-24T22:13:10.087 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:13:10.088 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:13:10.632 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.INIT (549 ms) 2026-02-24T22:13:10.632 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MOUNT_NOROOT 2026-02-24T22:13:10.633 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MOUNT_NOROOT (0 ms) 2026-02-24T22:13:10.633 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MOUNT 2026-02-24T22:13:10.633 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2026-02-24T22:13:10.633 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.SETUP_HIER1 2026-02-24T22:13:10.637 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.SETUP_HIER1 (4 ms) 2026-02-24T22:13:10.637 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.SETUP_DIRS1 2026-02-24T22:13:10.675 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.SETUP_DIRS1 (37 ms) 2026-02-24T22:13:10.675 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.SETATTR 2026-02-24T22:13:10.675 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.SETATTR (0 ms) 2026-02-24T22:13:10.675 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.RGW_CREATE_DIRS1 2026-02-24T22:13:10.675 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.RGW_CREATE_DIRS1 (0 ms) 2026-02-24T22:13:10.675 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.RGW_SETUP_RENAME1 2026-02-24T22:13:10.675 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.RGW_SETUP_RENAME1 (0 ms) 2026-02-24T22:13:10.675 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.RGW_INTRABUCKET_RENAME1 2026-02-24T22:13:10.676 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.RGW_INTRABUCKET_RENAME1 (0 ms) 2026-02-24T22:13:10.676 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.RGW_CROSSBUCKET_RENAME1 2026-02-24T22:13:10.676 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.RGW_CROSSBUCKET_RENAME1 (0 ms) 2026-02-24T22:13:10.676 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.GETATTR_DIRS1 2026-02-24T22:13:10.676 INFO:tasks.workunit.client.0.trial046.stdout:scanning objects in /bdirs1/dir_0 2026-02-24T22:13:10.676 INFO:tasks.workunit.client.0.trial046.stdout: /bdirs1/dir_0/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2026-02-24T22:13:10.676 INFO:tasks.workunit.client.0.trial046.stdout: /bdirs1/dir_0/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2026-02-24T22:13:10.676 INFO:tasks.workunit.client.0.trial046.stdout: /bdirs1/dir_0/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2026-02-24T22:13:10.676 INFO:tasks.workunit.client.0.trial046.stdout: /bdirs1/dir_0/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2026-02-24T22:13:10.676 INFO:tasks.workunit.client.0.trial046.stdout:scanning objects in /bdirs1/dir_1 2026-02-24T22:13:10.676 INFO:tasks.workunit.client.0.trial046.stdout: /bdirs1/dir_1/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2026-02-24T22:13:10.676 INFO:tasks.workunit.client.0.trial046.stdout: /bdirs1/dir_1/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2026-02-24T22:13:10.676 INFO:tasks.workunit.client.0.trial046.stdout: /bdirs1/dir_1/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2026-02-24T22:13:10.677 INFO:tasks.workunit.client.0.trial046.stdout: /bdirs1/dir_1/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2026-02-24T22:13:10.677 INFO:tasks.workunit.client.0.trial046.stdout:scanning objects in /bdirs1/dir_2 2026-02-24T22:13:10.677 INFO:tasks.workunit.client.0.trial046.stdout: /bdirs1/dir_2/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2026-02-24T22:13:10.677 INFO:tasks.workunit.client.0.trial046.stdout: /bdirs1/dir_2/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2026-02-24T22:13:10.677 INFO:tasks.workunit.client.0.trial046.stdout: /bdirs1/dir_2/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2026-02-24T22:13:10.677 INFO:tasks.workunit.client.0.trial046.stdout: /bdirs1/dir_2/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2026-02-24T22:13:10.677 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.GETATTR_DIRS1 (0 ms) 2026-02-24T22:13:10.677 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.READ_DIRS1 2026-02-24T22:13:10.677 INFO:tasks.workunit.client.0.trial046.stdout:read back objects in /bdirs1/dir_0 2026-02-24T22:13:10.677 INFO:tasks.workunit.client.0.trial046.stdout:reading 0,256 dir_0/sfile_0 2026-02-24T22:13:10.701 INFO:tasks.workunit.client.0.trial046.stdout: read back from sfile_0 : "data for sfile_0" 2026-02-24T22:13:10.701 INFO:tasks.workunit.client.0.trial046.stdout:reading 0,256 dir_0/sfile_1 2026-02-24T22:13:10.701 INFO:tasks.workunit.client.0.trial046.stdout: read back from sfile_1 : "data for sfile_1" 2026-02-24T22:13:10.701 INFO:tasks.workunit.client.0.trial046.stdout:read back objects in /bdirs1/dir_1 2026-02-24T22:13:10.701 INFO:tasks.workunit.client.0.trial046.stdout:reading 0,256 dir_1/sfile_0 2026-02-24T22:13:10.702 INFO:tasks.workunit.client.0.trial046.stdout: read back from sfile_0 : "data for sfile_0" 2026-02-24T22:13:10.702 INFO:tasks.workunit.client.0.trial046.stdout:reading 0,256 dir_1/sfile_1 2026-02-24T22:13:10.703 INFO:tasks.workunit.client.0.trial046.stdout: read back from sfile_1 : "data for sfile_1" 2026-02-24T22:13:10.703 INFO:tasks.workunit.client.0.trial046.stdout:read back objects in /bdirs1/dir_2 2026-02-24T22:13:10.703 INFO:tasks.workunit.client.0.trial046.stdout:reading 0,256 dir_2/sfile_0 2026-02-24T22:13:10.704 INFO:tasks.workunit.client.0.trial046.stdout: read back from sfile_0 : "data for sfile_0" 2026-02-24T22:13:10.704 INFO:tasks.workunit.client.0.trial046.stdout:reading 0,256 dir_2/sfile_1 2026-02-24T22:13:10.705 INFO:tasks.workunit.client.0.trial046.stdout: read back from sfile_1 : "data for sfile_1" 2026-02-24T22:13:10.705 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.READ_DIRS1 (29 ms) 2026-02-24T22:13:10.705 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.READF_SETUP1 2026-02-24T22:13:10.705 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.READF_SETUP1 (0 ms) 2026-02-24T22:13:10.705 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.READF_DIRS1 2026-02-24T22:13:10.705 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.READF_DIRS1 (0 ms) 2026-02-24T22:13:10.705 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.WRITEF_DIRS1 2026-02-24T22:13:10.705 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.WRITEF_DIRS1 (0 ms) 2026-02-24T22:13:10.706 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.RELEASE_DIRS1 2026-02-24T22:13:10.706 INFO:tasks.workunit.client.0.trial046.stdout:release sdir_0 type: DIR refs: 2 2026-02-24T22:13:10.706 INFO:tasks.workunit.client.0.trial046.stdout:release sfile_0 type: FILE refs: 2 2026-02-24T22:13:10.706 INFO:tasks.workunit.client.0.trial046.stdout:release sdir_1 type: DIR refs: 2 2026-02-24T22:13:10.706 INFO:tasks.workunit.client.0.trial046.stdout:release sfile_1 type: FILE refs: 2 2026-02-24T22:13:10.706 INFO:tasks.workunit.client.0.trial046.stdout:release sdir_0 type: DIR refs: 2 2026-02-24T22:13:10.706 INFO:tasks.workunit.client.0.trial046.stdout:release sfile_0 type: FILE refs: 2 2026-02-24T22:13:10.706 INFO:tasks.workunit.client.0.trial046.stdout:release sdir_1 type: DIR refs: 2 2026-02-24T22:13:10.706 INFO:tasks.workunit.client.0.trial046.stdout:release sfile_1 type: FILE refs: 2 2026-02-24T22:13:10.706 INFO:tasks.workunit.client.0.trial046.stdout:release sdir_0 type: DIR refs: 2 2026-02-24T22:13:10.706 INFO:tasks.workunit.client.0.trial046.stdout:release sfile_0 type: FILE refs: 2 2026-02-24T22:13:10.706 INFO:tasks.workunit.client.0.trial046.stdout:release sdir_1 type: DIR refs: 2 2026-02-24T22:13:10.707 INFO:tasks.workunit.client.0.trial046.stdout:release sfile_1 type: FILE refs: 2 2026-02-24T22:13:10.707 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.RELEASE_DIRS1 (0 ms) 2026-02-24T22:13:10.707 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.HIER1 2026-02-24T22:13:10.709 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/f2 (f2): FILE 2026-02-24T22:13:10.709 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/f1 (f1): FILE 2026-02-24T22:13:10.711 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/bar/f1 (f1): FILE 2026-02-24T22:13:10.712 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/bar/d1 (d1): DIR 2026-02-24T22:13:10.713 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2026-02-24T22:13:10.714 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2026-02-24T22:13:10.714 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/bar/baz/quux (quux): FILE 2026-02-24T22:13:10.714 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/bar/baz/momma (momma): FILE 2026-02-24T22:13:10.715 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2026-02-24T22:13:10.715 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2026-02-24T22:13:10.716 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/bar/baz/frobozz (frobozz): FILE 2026-02-24T22:13:10.717 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/bar/baz/bear (bear): DIR 2026-02-24T22:13:10.717 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/bar/baz (baz): DIR 2026-02-24T22:13:10.717 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo/bar (bar): DIR 2026-02-24T22:13:10.717 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot/foo (foo): DIR 2026-02-24T22:13:10.717 INFO:tasks.workunit.client.0.trial046.stdout:/nfsroot (nfsroot): DIR 2026-02-24T22:13:10.718 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.HIER1 (12 ms) 2026-02-24T22:13:10.718 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2026-02-24T22:13:10.718 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (0 ms) 2026-02-24T22:13:10.718 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2026-02-24T22:13:10.718 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2026-02-24T22:13:10.718 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2026-02-24T22:13:10.718 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2026-02-24T22:13:10.718 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2026-02-24T22:13:10.718 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2026-02-24T22:13:10.718 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.CLEANUP 2026-02-24T22:13:10.718 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2026-02-24T22:13:10.718 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.UMOUNT 2026-02-24T22:13:10.719 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2026-02-24T22:13:10.719 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.SHUTDOWN 2026-02-24T22:13:10.719 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:10.718+0000 7f3626cf5b80 -1 shutting down 2026-02-24T22:13:10.719 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:10.718+0000 7f3500a26640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 31007, si_uid : 1000, si_addr0x3e80000791f, si_status0 2026-02-24T22:13:12.600 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.SHUTDOWN (1883 ms) 2026-02-24T22:13:12.600 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 24 tests from LibRGW (2516 ms total) 2026-02-24T22:13:12.600 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:13:12.600 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment tear-down 2026-02-24T22:13:12.601 INFO:tasks.workunit.client.0.trial046.stdout:[==========] 24 tests from 1 test suite ran. (2516 ms total) 2026-02-24T22:13:12.601 INFO:tasks.workunit.client.0.trial046.stdout:[ PASSED ] 24 tests. 2026-02-24T22:13:12.608 INFO:tasks.workunit.client.0.trial046.stdout:phase 2.1 2026-02-24T22:13:12.621 INFO:tasks.workunit.client.0.trial046.stdout:[==========] Running 9 tests from 1 test suite. 2026-02-24T22:13:12.621 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment set-up. 2026-02-24T22:13:12.621 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 9 tests from LibRGW 2026-02-24T22:13:12.621 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.INIT 2026-02-24T22:13:12.625 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:13:12.625 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:13:12.748 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.INIT (127 ms) 2026-02-24T22:13:12.748 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MOUNT 2026-02-24T22:13:12.748 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2026-02-24T22:13:12.749 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2026-02-24T22:13:12.774 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.CREATE_BUCKET (26 ms) 2026-02-24T22:13:12.775 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2026-02-24T22:13:12.775 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.DELETE_BUCKET (0 ms) 2026-02-24T22:13:12.775 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.CREATE_BUCKET_MULTI 2026-02-24T22:13:12.780 INFO:tasks.workunit.client.0.trial046.stdout:created: sorrydave0 2026-02-24T22:13:12.785 INFO:tasks.workunit.client.0.trial046.stdout:created: sorrydave1 2026-02-24T22:13:12.790 INFO:tasks.workunit.client.0.trial046.stdout:created: sorrydave2 2026-02-24T22:13:12.796 INFO:tasks.workunit.client.0.trial046.stdout:created: sorrydave3 2026-02-24T22:13:12.801 INFO:tasks.workunit.client.0.trial046.stdout:created: sorrydave4 2026-02-24T22:13:12.806 INFO:tasks.workunit.client.0.trial046.stdout:created: sorrydave5 2026-02-24T22:13:12.811 INFO:tasks.workunit.client.0.trial046.stdout:created: sorrydave6 2026-02-24T22:13:12.816 INFO:tasks.workunit.client.0.trial046.stdout:created: sorrydave7 2026-02-24T22:13:12.822 INFO:tasks.workunit.client.0.trial046.stdout:created: sorrydave8 2026-02-24T22:13:12.827 INFO:tasks.workunit.client.0.trial046.stdout:created: sorrydave9 2026-02-24T22:13:12.827 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.CREATE_BUCKET_MULTI (52 ms) 2026-02-24T22:13:12.827 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.DELETE_BUCKET_MULTI 2026-02-24T22:13:12.930 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.DELETE_BUCKET_MULTI (103 ms) 2026-02-24T22:13:12.931 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.CLEANUP 2026-02-24T22:13:12.931 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2026-02-24T22:13:12.931 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.UMOUNT 2026-02-24T22:13:12.931 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2026-02-24T22:13:12.931 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.SHUTDOWN 2026-02-24T22:13:12.931 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:12.930+0000 7f8bac259b80 -1 shutting down 2026-02-24T22:13:12.932 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:12.930+0000 7f8a85f8a640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 31664, si_uid : 1000, si_addr0x3e800007bb0, si_status0 2026-02-24T22:13:14.722 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.SHUTDOWN (1792 ms) 2026-02-24T22:13:14.723 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 9 tests from LibRGW (2101 ms total) 2026-02-24T22:13:14.723 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:13:14.723 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment tear-down 2026-02-24T22:13:14.723 INFO:tasks.workunit.client.0.trial046.stdout:[==========] 9 tests from 1 test suite ran. (2101 ms total) 2026-02-24T22:13:14.723 INFO:tasks.workunit.client.0.trial046.stdout:[ PASSED ] 9 tests. 2026-02-24T22:13:14.731 INFO:tasks.workunit.client.0.trial046.stdout:phase 2.2 2026-02-24T22:13:14.743 INFO:tasks.workunit.client.0.trial046.stdout:[==========] Running 9 tests from 1 test suite. 2026-02-24T22:13:14.743 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment set-up. 2026-02-24T22:13:14.743 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 9 tests from LibRGW 2026-02-24T22:13:14.743 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.INIT 2026-02-24T22:13:14.747 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:13:14.747 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:13:14.876 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.INIT (133 ms) 2026-02-24T22:13:14.876 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MOUNT 2026-02-24T22:13:14.876 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2026-02-24T22:13:14.876 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2026-02-24T22:13:14.877 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.CREATE_BUCKET (0 ms) 2026-02-24T22:13:14.877 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2026-02-24T22:13:14.889 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.DELETE_BUCKET (13 ms) 2026-02-24T22:13:14.890 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.CREATE_BUCKET_MULTI 2026-02-24T22:13:14.894 INFO:tasks.workunit.client.0.trial046.stdout:created: sorrydave0 2026-02-24T22:13:14.900 INFO:tasks.workunit.client.0.trial046.stdout:created: sorrydave1 2026-02-24T22:13:14.904 INFO:tasks.workunit.client.0.trial046.stdout:created: sorrydave2 2026-02-24T22:13:14.910 INFO:tasks.workunit.client.0.trial046.stdout:created: sorrydave3 2026-02-24T22:13:14.915 INFO:tasks.workunit.client.0.trial046.stdout:created: sorrydave4 2026-02-24T22:13:14.920 INFO:tasks.workunit.client.0.trial046.stdout:created: sorrydave5 2026-02-24T22:13:14.925 INFO:tasks.workunit.client.0.trial046.stdout:created: sorrydave6 2026-02-24T22:13:14.930 INFO:tasks.workunit.client.0.trial046.stdout:created: sorrydave7 2026-02-24T22:13:14.935 INFO:tasks.workunit.client.0.trial046.stdout:created: sorrydave8 2026-02-24T22:13:14.941 INFO:tasks.workunit.client.0.trial046.stdout:created: sorrydave9 2026-02-24T22:13:14.941 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.CREATE_BUCKET_MULTI (51 ms) 2026-02-24T22:13:14.941 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.DELETE_BUCKET_MULTI 2026-02-24T22:13:15.047 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.DELETE_BUCKET_MULTI (106 ms) 2026-02-24T22:13:15.048 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.CLEANUP 2026-02-24T22:13:15.048 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2026-02-24T22:13:15.048 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.UMOUNT 2026-02-24T22:13:15.048 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2026-02-24T22:13:15.048 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.SHUTDOWN 2026-02-24T22:13:15.048 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:15.046+0000 7f50c06deb80 -1 shutting down 2026-02-24T22:13:15.049 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:15.050+0000 7f4f9a40f640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 32321, si_uid : 1000, si_addr0x3e800007e41, si_status0 2026-02-24T22:13:16.843 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.SHUTDOWN (1795 ms) 2026-02-24T22:13:16.843 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 9 tests from LibRGW (2100 ms total) 2026-02-24T22:13:16.844 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:13:16.844 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment tear-down 2026-02-24T22:13:16.844 INFO:tasks.workunit.client.0.trial046.stdout:[==========] 9 tests from 1 test suite ran. (2100 ms total) 2026-02-24T22:13:16.844 INFO:tasks.workunit.client.0.trial046.stdout:[ PASSED ] 9 tests. 2026-02-24T22:13:16.852 INFO:tasks.workunit.client.0.trial046.stdout:phase 3.1 2026-02-24T22:13:17.040 INFO:tasks.workunit.client.0.trial046.stdout:[==========] Running 13 tests from 1 test suite. 2026-02-24T22:13:17.041 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment set-up. 2026-02-24T22:13:17.041 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 13 tests from LibRGW 2026-02-24T22:13:17.041 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.INIT 2026-02-24T22:13:17.045 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:13:17.045 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:13:17.176 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.INIT (135 ms) 2026-02-24T22:13:17.176 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MOUNT 2026-02-24T22:13:17.177 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2026-02-24T22:13:17.177 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2026-02-24T22:13:17.206 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.CREATE_BUCKET (29 ms) 2026-02-24T22:13:17.206 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2026-02-24T22:13:17.208 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (1 ms) 2026-02-24T22:13:17.208 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2026-02-24T22:13:17.209 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (1 ms) 2026-02-24T22:13:17.209 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.LARGE1 2026-02-24T22:13:17.259 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.LARGE1 (50 ms) 2026-02-24T22:13:17.259 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.LARGE2 2026-02-24T22:13:17.285 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.LARGE2 (25 ms) 2026-02-24T22:13:17.285 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.STAT_OBJECT 2026-02-24T22:13:17.286 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2026-02-24T22:13:17.286 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2026-02-24T22:13:17.286 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.DELETE_OBJECT (0 ms) 2026-02-24T22:13:17.286 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2026-02-24T22:13:17.286 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.DELETE_BUCKET (0 ms) 2026-02-24T22:13:17.286 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.CLEANUP 2026-02-24T22:13:17.286 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2026-02-24T22:13:17.286 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.UMOUNT 2026-02-24T22:13:17.286 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2026-02-24T22:13:17.286 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.SHUTDOWN 2026-02-24T22:13:17.287 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:17.286+0000 7f5d7aa5db80 -1 shutting down 2026-02-24T22:13:17.287 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:17.286+0000 7f5c5478e640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 32978, si_uid : 1000, si_addr0x3e8000080d2, si_status0 2026-02-24T22:13:19.146 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.SHUTDOWN (1861 ms) 2026-02-24T22:13:19.147 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 13 tests from LibRGW (2105 ms total) 2026-02-24T22:13:19.147 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:13:19.147 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment tear-down 2026-02-24T22:13:19.147 INFO:tasks.workunit.client.0.trial046.stdout:[==========] 13 tests from 1 test suite ran. (2105 ms total) 2026-02-24T22:13:19.147 INFO:tasks.workunit.client.0.trial046.stdout:[ PASSED ] 13 tests. 2026-02-24T22:13:19.158 INFO:tasks.workunit.client.0.trial046.stdout:phase 3.2 2026-02-24T22:13:19.353 INFO:tasks.workunit.client.0.trial046.stdout:[==========] Running 13 tests from 1 test suite. 2026-02-24T22:13:19.354 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment set-up. 2026-02-24T22:13:19.354 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 13 tests from LibRGW 2026-02-24T22:13:19.354 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.INIT 2026-02-24T22:13:19.358 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:13:19.358 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:13:19.484 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.INIT (130 ms) 2026-02-24T22:13:19.484 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MOUNT 2026-02-24T22:13:19.484 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2026-02-24T22:13:19.485 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2026-02-24T22:13:19.485 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.CREATE_BUCKET (0 ms) 2026-02-24T22:13:19.485 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2026-02-24T22:13:19.489 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (4 ms) 2026-02-24T22:13:19.489 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2026-02-24T22:13:19.491 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (2 ms) 2026-02-24T22:13:19.492 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.LARGE1 2026-02-24T22:13:19.569 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.LARGE1 (77 ms) 2026-02-24T22:13:19.569 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.LARGE2 2026-02-24T22:13:19.569 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.LARGE2 (0 ms) 2026-02-24T22:13:19.569 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.STAT_OBJECT 2026-02-24T22:13:19.569 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2026-02-24T22:13:19.569 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2026-02-24T22:13:19.572 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.DELETE_OBJECT (3 ms) 2026-02-24T22:13:19.572 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2026-02-24T22:13:19.583 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.DELETE_BUCKET (10 ms) 2026-02-24T22:13:19.583 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.CLEANUP 2026-02-24T22:13:19.583 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2026-02-24T22:13:19.583 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.UMOUNT 2026-02-24T22:13:19.583 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2026-02-24T22:13:19.583 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.SHUTDOWN 2026-02-24T22:13:19.583 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:19.582+0000 7f4fba709b80 -1 shutting down 2026-02-24T22:13:19.584 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:19.586+0000 7f4e9443a640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 33645, si_uid : 1000, si_addr0x3e80000836d, si_status0 2026-02-24T22:13:21.459 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.SHUTDOWN (1876 ms) 2026-02-24T22:13:21.459 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 13 tests from LibRGW (2105 ms total) 2026-02-24T22:13:21.459 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:13:21.459 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment tear-down 2026-02-24T22:13:21.460 INFO:tasks.workunit.client.0.trial046.stdout:[==========] 13 tests from 1 test suite ran. (2105 ms total) 2026-02-24T22:13:21.460 INFO:tasks.workunit.client.0.trial046.stdout:[ PASSED ] 13 tests. 2026-02-24T22:13:21.471 INFO:tasks.workunit.client.0.trial046.stdout:phase 4.1 2026-02-24T22:13:21.483 INFO:tasks.workunit.client.0.trial046.stdout:[==========] Running 11 tests from 1 test suite. 2026-02-24T22:13:21.483 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment set-up. 2026-02-24T22:13:21.483 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 11 tests from LibRGW 2026-02-24T22:13:21.484 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.TVAR 2026-02-24T22:13:21.484 INFO:tasks.workunit.client.0.trial046.stdout:read i1: 64001 2026-02-24T22:13:21.484 INFO:tasks.workunit.client.0.trial046.stdout:read s1: blunderbuss 2026-02-24T22:13:21.484 INFO:tasks.workunit.client.0.trial046.stdout:read s3: [ OK ] LibRGW.TVAR (0 ms) 2026-02-24T22:13:21.484 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.INIT 2026-02-24T22:13:21.488 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:13:21.488 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:13:21.612 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.INIT (128 ms) 2026-02-24T22:13:21.612 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MOUNT 2026-02-24T22:13:21.612 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2026-02-24T22:13:21.613 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2026-02-24T22:13:21.663 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (50 ms) 2026-02-24T22:13:21.663 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2026-02-24T22:13:22.041 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_0 2026-02-24T22:13:22.041 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_1 2026-02-24T22:13:22.041 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_2 2026-02-24T22:13:22.042 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_3 2026-02-24T22:13:22.042 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_4 2026-02-24T22:13:22.042 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_5 2026-02-24T22:13:22.042 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_6 2026-02-24T22:13:22.042 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_7 2026-02-24T22:13:22.042 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_8 2026-02-24T22:13:22.042 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_9 2026-02-24T22:13:22.042 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_10 2026-02-24T22:13:22.042 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_11 2026-02-24T22:13:22.042 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_12 2026-02-24T22:13:22.042 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_13 2026-02-24T22:13:22.042 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_14 2026-02-24T22:13:22.042 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_15 2026-02-24T22:13:22.043 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_16 2026-02-24T22:13:22.043 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_17 2026-02-24T22:13:22.043 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_18 2026-02-24T22:13:22.043 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_19 2026-02-24T22:13:22.043 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_20 2026-02-24T22:13:22.043 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_21 2026-02-24T22:13:22.043 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_22 2026-02-24T22:13:22.043 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_23 2026-02-24T22:13:22.043 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_24 2026-02-24T22:13:22.043 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_25 2026-02-24T22:13:22.043 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_26 2026-02-24T22:13:22.043 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_27 2026-02-24T22:13:22.044 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_28 2026-02-24T22:13:22.044 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_29 2026-02-24T22:13:22.044 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_30 2026-02-24T22:13:22.044 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_31 2026-02-24T22:13:22.044 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_32 2026-02-24T22:13:22.044 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_33 2026-02-24T22:13:22.044 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_34 2026-02-24T22:13:22.044 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_35 2026-02-24T22:13:22.044 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_36 2026-02-24T22:13:22.044 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_37 2026-02-24T22:13:22.044 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_38 2026-02-24T22:13:22.044 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_39 2026-02-24T22:13:22.045 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_40 2026-02-24T22:13:22.045 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_41 2026-02-24T22:13:22.045 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_42 2026-02-24T22:13:22.045 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_43 2026-02-24T22:13:22.045 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_44 2026-02-24T22:13:22.045 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_45 2026-02-24T22:13:22.045 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_46 2026-02-24T22:13:22.045 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_47 2026-02-24T22:13:22.045 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_48 2026-02-24T22:13:22.045 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_49 2026-02-24T22:13:22.045 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_50 2026-02-24T22:13:22.045 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_51 2026-02-24T22:13:22.045 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_52 2026-02-24T22:13:22.045 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_53 2026-02-24T22:13:22.046 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_54 2026-02-24T22:13:22.046 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_55 2026-02-24T22:13:22.046 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_56 2026-02-24T22:13:22.046 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_57 2026-02-24T22:13:22.046 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_58 2026-02-24T22:13:22.046 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_59 2026-02-24T22:13:22.046 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_60 2026-02-24T22:13:22.046 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_61 2026-02-24T22:13:22.046 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_62 2026-02-24T22:13:22.046 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_63 2026-02-24T22:13:22.046 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_64 2026-02-24T22:13:22.046 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_65 2026-02-24T22:13:22.046 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_66 2026-02-24T22:13:22.046 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_67 2026-02-24T22:13:22.047 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_68 2026-02-24T22:13:22.047 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_69 2026-02-24T22:13:22.047 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_70 2026-02-24T22:13:22.047 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_71 2026-02-24T22:13:22.047 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_72 2026-02-24T22:13:22.047 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_73 2026-02-24T22:13:22.047 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_74 2026-02-24T22:13:22.047 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_75 2026-02-24T22:13:22.047 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_76 2026-02-24T22:13:22.047 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_77 2026-02-24T22:13:22.047 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_78 2026-02-24T22:13:22.047 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_79 2026-02-24T22:13:22.047 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_80 2026-02-24T22:13:22.048 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_81 2026-02-24T22:13:22.048 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_82 2026-02-24T22:13:22.048 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_83 2026-02-24T22:13:22.048 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_84 2026-02-24T22:13:22.048 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_85 2026-02-24T22:13:22.048 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_86 2026-02-24T22:13:22.048 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_87 2026-02-24T22:13:22.048 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_88 2026-02-24T22:13:22.048 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_89 2026-02-24T22:13:22.048 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_90 2026-02-24T22:13:22.048 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_91 2026-02-24T22:13:22.048 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_92 2026-02-24T22:13:22.048 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_93 2026-02-24T22:13:22.048 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_94 2026-02-24T22:13:22.049 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_95 2026-02-24T22:13:22.049 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_96 2026-02-24T22:13:22.049 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_97 2026-02-24T22:13:22.049 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_98 2026-02-24T22:13:22.049 INFO:tasks.workunit.client.0.trial046.stdout:created: dmarker:f_99 2026-02-24T22:13:22.049 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (378 ms) 2026-02-24T22:13:22.049 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2026-02-24T22:13:22.050 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=. flags=4 2026-02-24T22:13:22.050 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=.. flags=4 2026-02-24T22:13:22.050 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_0 flags=8 2026-02-24T22:13:22.051 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_1 flags=8 2026-02-24T22:13:22.051 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_10 flags=8 2026-02-24T22:13:22.051 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_11 flags=8 2026-02-24T22:13:22.051 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_12 flags=8 2026-02-24T22:13:22.051 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_13 flags=8 2026-02-24T22:13:22.051 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_14 flags=8 2026-02-24T22:13:22.051 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_15 flags=8 2026-02-24T22:13:22.051 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_16 flags=8 2026-02-24T22:13:22.051 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_17 flags=8 2026-02-24T22:13:22.051 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_18 flags=8 2026-02-24T22:13:22.051 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_19 flags=8 2026-02-24T22:13:22.051 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_2 flags=8 2026-02-24T22:13:22.051 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_20 flags=8 2026-02-24T22:13:22.052 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_21 flags=8 2026-02-24T22:13:22.052 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_22 flags=8 2026-02-24T22:13:22.052 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_23 flags=8 2026-02-24T22:13:22.052 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_24 flags=8 2026-02-24T22:13:22.052 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_25 flags=8 2026-02-24T22:13:22.052 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_26 flags=8 2026-02-24T22:13:22.052 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_27 flags=8 2026-02-24T22:13:22.052 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_28 flags=8 2026-02-24T22:13:22.052 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_29 flags=8 2026-02-24T22:13:22.052 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_3 flags=8 2026-02-24T22:13:22.052 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_30 flags=8 2026-02-24T22:13:22.052 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_31 flags=8 2026-02-24T22:13:22.053 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_32 flags=8 2026-02-24T22:13:22.053 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_33 flags=8 2026-02-24T22:13:22.053 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_34 flags=8 2026-02-24T22:13:22.053 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_35 flags=8 2026-02-24T22:13:22.053 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_36 flags=8 2026-02-24T22:13:22.053 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_37 flags=8 2026-02-24T22:13:22.053 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_38 flags=8 2026-02-24T22:13:22.053 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_39 flags=8 2026-02-24T22:13:22.053 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_4 flags=8 2026-02-24T22:13:22.053 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_40 flags=8 2026-02-24T22:13:22.054 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_41 flags=8 2026-02-24T22:13:22.054 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_42 flags=8 2026-02-24T22:13:22.054 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_43 flags=8 2026-02-24T22:13:22.054 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_44 flags=8 2026-02-24T22:13:22.054 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_45 flags=8 2026-02-24T22:13:22.054 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_46 flags=8 2026-02-24T22:13:22.054 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_47 flags=8 2026-02-24T22:13:22.054 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_48 flags=8 2026-02-24T22:13:22.054 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_49 flags=8 2026-02-24T22:13:22.054 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_5 flags=8 2026-02-24T22:13:22.054 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_50 flags=8 2026-02-24T22:13:22.054 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_51 flags=8 2026-02-24T22:13:22.054 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_52 flags=8 2026-02-24T22:13:22.055 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_53 flags=8 2026-02-24T22:13:22.055 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_54 flags=8 2026-02-24T22:13:22.055 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_55 flags=8 2026-02-24T22:13:22.055 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_56 flags=8 2026-02-24T22:13:22.055 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_57 flags=8 2026-02-24T22:13:22.055 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_58 flags=8 2026-02-24T22:13:22.055 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_59 flags=8 2026-02-24T22:13:22.055 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_6 flags=8 2026-02-24T22:13:22.055 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_60 flags=8 2026-02-24T22:13:22.055 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_61 flags=8 2026-02-24T22:13:22.055 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_62 flags=8 2026-02-24T22:13:22.055 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_63 flags=8 2026-02-24T22:13:22.055 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_64 flags=8 2026-02-24T22:13:22.055 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_65 flags=8 2026-02-24T22:13:22.056 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_66 flags=8 2026-02-24T22:13:22.056 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_67 flags=8 2026-02-24T22:13:22.056 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_68 flags=8 2026-02-24T22:13:22.056 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_69 flags=8 2026-02-24T22:13:22.056 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_7 flags=8 2026-02-24T22:13:22.056 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_70 flags=8 2026-02-24T22:13:22.056 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_71 flags=8 2026-02-24T22:13:22.056 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_72 flags=8 2026-02-24T22:13:22.056 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_73 flags=8 2026-02-24T22:13:22.056 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_74 flags=8 2026-02-24T22:13:22.056 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_75 flags=8 2026-02-24T22:13:22.056 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_76 flags=8 2026-02-24T22:13:22.056 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_77 flags=8 2026-02-24T22:13:22.056 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_78 flags=8 2026-02-24T22:13:22.057 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_79 flags=8 2026-02-24T22:13:22.057 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_8 flags=8 2026-02-24T22:13:22.057 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_80 flags=8 2026-02-24T22:13:22.057 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_81 flags=8 2026-02-24T22:13:22.057 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_82 flags=8 2026-02-24T22:13:22.057 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_83 flags=8 2026-02-24T22:13:22.057 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_84 flags=8 2026-02-24T22:13:22.057 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_85 flags=8 2026-02-24T22:13:22.057 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_86 flags=8 2026-02-24T22:13:22.057 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_87 flags=8 2026-02-24T22:13:22.057 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_88 flags=8 2026-02-24T22:13:22.057 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_89 flags=8 2026-02-24T22:13:22.057 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_9 flags=8 2026-02-24T22:13:22.058 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_90 flags=8 2026-02-24T22:13:22.058 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_91 flags=8 2026-02-24T22:13:22.058 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_92 flags=8 2026-02-24T22:13:22.058 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_93 flags=8 2026-02-24T22:13:22.058 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_94 flags=8 2026-02-24T22:13:22.058 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_95 flags=8 2026-02-24T22:13:22.058 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_96 flags=8 2026-02-24T22:13:22.058 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_97 flags=8 2026-02-24T22:13:22.058 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_98 flags=8 2026-02-24T22:13:22.058 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_99 flags=8 2026-02-24T22:13:22.058 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MARKER1_READDIR (2 ms) 2026-02-24T22:13:22.058 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MARKER2_READDIR 2026-02-24T22:13:22.058 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_0 flags=8 2026-02-24T22:13:22.059 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_1 flags=8 2026-02-24T22:13:22.059 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_10 flags=8 2026-02-24T22:13:22.059 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_11 flags=8 2026-02-24T22:13:22.059 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_12 flags=8 2026-02-24T22:13:22.059 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_13 flags=8 2026-02-24T22:13:22.059 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_14 flags=8 2026-02-24T22:13:22.059 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_15 flags=8 2026-02-24T22:13:22.059 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_16 flags=8 2026-02-24T22:13:22.059 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_17 flags=8 2026-02-24T22:13:22.059 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_18 flags=8 2026-02-24T22:13:22.059 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_19 flags=8 2026-02-24T22:13:22.059 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_2 flags=8 2026-02-24T22:13:22.059 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_20 flags=8 2026-02-24T22:13:22.060 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_21 flags=8 2026-02-24T22:13:22.060 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_22 flags=8 2026-02-24T22:13:22.060 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_23 flags=8 2026-02-24T22:13:22.060 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_24 flags=8 2026-02-24T22:13:22.060 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_25 flags=8 2026-02-24T22:13:22.060 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_26 flags=8 2026-02-24T22:13:22.060 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_27 flags=8 2026-02-24T22:13:22.060 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_28 flags=8 2026-02-24T22:13:22.060 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_29 flags=8 2026-02-24T22:13:22.060 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_3 flags=8 2026-02-24T22:13:22.060 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_30 flags=8 2026-02-24T22:13:22.060 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_31 flags=8 2026-02-24T22:13:22.060 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_32 flags=8 2026-02-24T22:13:22.061 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_33 flags=8 2026-02-24T22:13:22.061 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_34 flags=8 2026-02-24T22:13:22.061 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_35 flags=8 2026-02-24T22:13:22.061 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_36 flags=8 2026-02-24T22:13:22.061 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_37 flags=8 2026-02-24T22:13:22.061 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_38 flags=8 2026-02-24T22:13:22.061 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_39 flags=8 2026-02-24T22:13:22.061 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_4 flags=8 2026-02-24T22:13:22.061 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_40 flags=8 2026-02-24T22:13:22.061 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_41 flags=8 2026-02-24T22:13:22.061 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_42 flags=8 2026-02-24T22:13:22.061 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_43 flags=8 2026-02-24T22:13:22.061 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_44 flags=8 2026-02-24T22:13:22.061 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_45 flags=8 2026-02-24T22:13:22.062 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_46 flags=8 2026-02-24T22:13:22.062 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_47 flags=8 2026-02-24T22:13:22.062 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_48 flags=8 2026-02-24T22:13:22.062 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_49 flags=8 2026-02-24T22:13:22.062 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_5 flags=8 2026-02-24T22:13:22.062 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_50 flags=8 2026-02-24T22:13:22.062 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_51 flags=8 2026-02-24T22:13:22.062 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_52 flags=8 2026-02-24T22:13:22.062 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_53 flags=8 2026-02-24T22:13:22.062 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_54 flags=8 2026-02-24T22:13:22.062 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_55 flags=8 2026-02-24T22:13:22.062 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_56 flags=8 2026-02-24T22:13:22.062 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_57 flags=8 2026-02-24T22:13:22.063 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_58 flags=8 2026-02-24T22:13:22.063 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_59 flags=8 2026-02-24T22:13:22.063 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_6 flags=8 2026-02-24T22:13:22.063 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_60 flags=8 2026-02-24T22:13:22.063 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_61 flags=8 2026-02-24T22:13:22.063 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_62 flags=8 2026-02-24T22:13:22.063 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_63 flags=8 2026-02-24T22:13:22.063 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_64 flags=8 2026-02-24T22:13:22.063 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_65 flags=8 2026-02-24T22:13:22.063 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_66 flags=8 2026-02-24T22:13:22.063 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_67 flags=8 2026-02-24T22:13:22.063 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_68 flags=8 2026-02-24T22:13:22.063 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_69 flags=8 2026-02-24T22:13:22.063 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_7 flags=8 2026-02-24T22:13:22.064 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_70 flags=8 2026-02-24T22:13:22.064 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_71 flags=8 2026-02-24T22:13:22.064 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_72 flags=8 2026-02-24T22:13:22.064 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_73 flags=8 2026-02-24T22:13:22.064 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_74 flags=8 2026-02-24T22:13:22.064 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_75 flags=8 2026-02-24T22:13:22.064 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_76 flags=8 2026-02-24T22:13:22.064 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_77 flags=8 2026-02-24T22:13:22.064 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_78 flags=8 2026-02-24T22:13:22.064 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_79 flags=8 2026-02-24T22:13:22.064 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_8 flags=8 2026-02-24T22:13:22.064 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_80 flags=8 2026-02-24T22:13:22.064 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_81 flags=8 2026-02-24T22:13:22.065 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_82 flags=8 2026-02-24T22:13:22.065 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_83 flags=8 2026-02-24T22:13:22.065 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_84 flags=8 2026-02-24T22:13:22.065 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_85 flags=8 2026-02-24T22:13:22.065 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_86 flags=8 2026-02-24T22:13:22.065 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_87 flags=8 2026-02-24T22:13:22.065 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_88 flags=8 2026-02-24T22:13:22.065 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_89 flags=8 2026-02-24T22:13:22.065 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_9 flags=8 2026-02-24T22:13:22.065 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_90 flags=8 2026-02-24T22:13:22.065 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_91 flags=8 2026-02-24T22:13:22.065 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_92 flags=8 2026-02-24T22:13:22.065 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_93 flags=8 2026-02-24T22:13:22.065 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_94 flags=8 2026-02-24T22:13:22.066 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_95 flags=8 2026-02-24T22:13:22.066 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_96 flags=8 2026-02-24T22:13:22.066 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_97 flags=8 2026-02-24T22:13:22.066 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_98 flags=8 2026-02-24T22:13:22.066 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_99 flags=8 2026-02-24T22:13:22.066 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MARKER2_READDIR (1 ms) 2026-02-24T22:13:22.066 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2026-02-24T22:13:22.066 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2026-02-24T22:13:22.066 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.CLEANUP 2026-02-24T22:13:22.066 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2026-02-24T22:13:22.066 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.UMOUNT 2026-02-24T22:13:22.066 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2026-02-24T22:13:22.066 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.SHUTDOWN 2026-02-24T22:13:22.067 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:22.046+0000 7febe72eab80 -1 shutting down 2026-02-24T22:13:22.067 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:22.046+0000 7feac101b640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 34309, si_uid : 1000, si_addr0x3e800008605, si_status0 2026-02-24T22:13:23.588 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.SHUTDOWN (1543 ms) 2026-02-24T22:13:23.588 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 11 tests from LibRGW (2104 ms total) 2026-02-24T22:13:23.588 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:13:23.588 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment tear-down 2026-02-24T22:13:23.588 INFO:tasks.workunit.client.0.trial046.stdout:[==========] 11 tests from 1 test suite ran. (2104 ms total) 2026-02-24T22:13:23.588 INFO:tasks.workunit.client.0.trial046.stdout:[ PASSED ] 11 tests. 2026-02-24T22:13:23.597 INFO:tasks.workunit.client.0.trial046.stdout:phase 4.2 2026-02-24T22:13:23.610 INFO:tasks.workunit.client.0.trial046.stdout:[==========] Running 11 tests from 1 test suite. 2026-02-24T22:13:23.610 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment set-up. 2026-02-24T22:13:23.610 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 11 tests from LibRGW 2026-02-24T22:13:23.610 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.TVAR 2026-02-24T22:13:23.610 INFO:tasks.workunit.client.0.trial046.stdout:read i1: 64001 2026-02-24T22:13:23.610 INFO:tasks.workunit.client.0.trial046.stdout:read s1: blunderbuss 2026-02-24T22:13:23.610 INFO:tasks.workunit.client.0.trial046.stdout:read s3: [ OK ] LibRGW.TVAR (0 ms) 2026-02-24T22:13:23.611 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.INIT 2026-02-24T22:13:23.614 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:13:23.614 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:13:23.752 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.INIT (142 ms) 2026-02-24T22:13:23.752 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MOUNT 2026-02-24T22:13:23.753 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2026-02-24T22:13:23.753 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2026-02-24T22:13:23.761 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (8 ms) 2026-02-24T22:13:23.761 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2026-02-24T22:13:23.761 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2026-02-24T22:13:23.761 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2026-02-24T22:13:23.761 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2026-02-24T22:13:23.761 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MARKER2_READDIR 2026-02-24T22:13:23.763 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_0 flags=8 2026-02-24T22:13:23.763 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_1 flags=8 2026-02-24T22:13:23.763 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_10 flags=8 2026-02-24T22:13:23.763 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_11 flags=8 2026-02-24T22:13:23.763 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_12 flags=8 2026-02-24T22:13:23.764 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_13 flags=8 2026-02-24T22:13:23.764 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_14 flags=8 2026-02-24T22:13:23.764 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_15 flags=8 2026-02-24T22:13:23.764 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_16 flags=8 2026-02-24T22:13:23.764 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_17 flags=8 2026-02-24T22:13:23.764 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_18 flags=8 2026-02-24T22:13:23.764 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_19 flags=8 2026-02-24T22:13:23.764 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_2 flags=8 2026-02-24T22:13:23.764 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_20 flags=8 2026-02-24T22:13:23.764 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_21 flags=8 2026-02-24T22:13:23.764 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_22 flags=8 2026-02-24T22:13:23.764 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_23 flags=8 2026-02-24T22:13:23.765 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_24 flags=8 2026-02-24T22:13:23.765 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_25 flags=8 2026-02-24T22:13:23.765 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_26 flags=8 2026-02-24T22:13:23.765 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_27 flags=8 2026-02-24T22:13:23.765 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_28 flags=8 2026-02-24T22:13:23.765 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_29 flags=8 2026-02-24T22:13:23.765 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_3 flags=8 2026-02-24T22:13:23.765 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_30 flags=8 2026-02-24T22:13:23.765 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_31 flags=8 2026-02-24T22:13:23.765 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_32 flags=8 2026-02-24T22:13:23.765 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_33 flags=8 2026-02-24T22:13:23.765 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_34 flags=8 2026-02-24T22:13:23.765 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_35 flags=8 2026-02-24T22:13:23.766 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_36 flags=8 2026-02-24T22:13:23.766 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_37 flags=8 2026-02-24T22:13:23.766 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_38 flags=8 2026-02-24T22:13:23.766 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_39 flags=8 2026-02-24T22:13:23.766 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_4 flags=8 2026-02-24T22:13:23.766 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_40 flags=8 2026-02-24T22:13:23.766 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_41 flags=8 2026-02-24T22:13:23.766 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_42 flags=8 2026-02-24T22:13:23.766 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_43 flags=8 2026-02-24T22:13:23.766 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_44 flags=8 2026-02-24T22:13:23.766 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_45 flags=8 2026-02-24T22:13:23.766 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_46 flags=8 2026-02-24T22:13:23.766 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_47 flags=8 2026-02-24T22:13:23.767 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_48 flags=8 2026-02-24T22:13:23.767 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_49 flags=8 2026-02-24T22:13:23.767 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_5 flags=8 2026-02-24T22:13:23.767 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_50 flags=8 2026-02-24T22:13:23.767 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_51 flags=8 2026-02-24T22:13:23.767 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_52 flags=8 2026-02-24T22:13:23.767 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_53 flags=8 2026-02-24T22:13:23.767 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_54 flags=8 2026-02-24T22:13:23.767 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_55 flags=8 2026-02-24T22:13:23.767 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_56 flags=8 2026-02-24T22:13:23.767 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_57 flags=8 2026-02-24T22:13:23.767 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_58 flags=8 2026-02-24T22:13:23.767 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_59 flags=8 2026-02-24T22:13:23.768 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_6 flags=8 2026-02-24T22:13:23.768 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_60 flags=8 2026-02-24T22:13:23.768 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_61 flags=8 2026-02-24T22:13:23.768 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_62 flags=8 2026-02-24T22:13:23.768 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_63 flags=8 2026-02-24T22:13:23.768 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_64 flags=8 2026-02-24T22:13:23.768 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_65 flags=8 2026-02-24T22:13:23.768 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_66 flags=8 2026-02-24T22:13:23.768 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_67 flags=8 2026-02-24T22:13:23.768 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_68 flags=8 2026-02-24T22:13:23.768 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_69 flags=8 2026-02-24T22:13:23.768 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_7 flags=8 2026-02-24T22:13:23.768 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_70 flags=8 2026-02-24T22:13:23.769 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_71 flags=8 2026-02-24T22:13:23.769 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_72 flags=8 2026-02-24T22:13:23.769 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_73 flags=8 2026-02-24T22:13:23.769 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_74 flags=8 2026-02-24T22:13:23.769 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_75 flags=8 2026-02-24T22:13:23.769 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_76 flags=8 2026-02-24T22:13:23.769 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_77 flags=8 2026-02-24T22:13:23.769 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_78 flags=8 2026-02-24T22:13:23.769 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_79 flags=8 2026-02-24T22:13:23.769 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_8 flags=8 2026-02-24T22:13:23.769 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_80 flags=8 2026-02-24T22:13:23.769 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_81 flags=8 2026-02-24T22:13:23.769 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_82 flags=8 2026-02-24T22:13:23.769 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_83 flags=8 2026-02-24T22:13:23.770 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_84 flags=8 2026-02-24T22:13:23.770 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_85 flags=8 2026-02-24T22:13:23.770 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_86 flags=8 2026-02-24T22:13:23.770 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_87 flags=8 2026-02-24T22:13:23.770 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_88 flags=8 2026-02-24T22:13:23.770 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_89 flags=8 2026-02-24T22:13:23.770 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_9 flags=8 2026-02-24T22:13:23.770 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_90 flags=8 2026-02-24T22:13:23.770 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_91 flags=8 2026-02-24T22:13:23.770 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_92 flags=8 2026-02-24T22:13:23.770 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_93 flags=8 2026-02-24T22:13:23.770 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_94 flags=8 2026-02-24T22:13:23.770 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_95 flags=8 2026-02-24T22:13:23.771 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_96 flags=8 2026-02-24T22:13:23.771 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_97 flags=8 2026-02-24T22:13:23.771 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_98 flags=8 2026-02-24T22:13:23.771 INFO:tasks.workunit.client.0.trial046.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_99 flags=8 2026-02-24T22:13:23.771 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MARKER2_READDIR (1 ms) 2026-02-24T22:13:23.771 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2026-02-24T22:13:23.771 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2026-02-24T22:13:23.771 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.CLEANUP 2026-02-24T22:13:23.771 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2026-02-24T22:13:23.771 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.UMOUNT 2026-02-24T22:13:23.771 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2026-02-24T22:13:23.771 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.SHUTDOWN 2026-02-24T22:13:23.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:23.762+0000 7f64c52a2b80 -1 shutting down 2026-02-24T22:13:23.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:23.766+0000 7f639efd3640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 34966, si_uid : 1000, si_addr0x3e800008896, si_status0 2026-02-24T22:13:25.720 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.SHUTDOWN (1956 ms) 2026-02-24T22:13:25.720 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 11 tests from LibRGW (2109 ms total) 2026-02-24T22:13:25.720 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:13:25.720 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment tear-down 2026-02-24T22:13:25.720 INFO:tasks.workunit.client.0.trial046.stdout:[==========] 11 tests from 1 test suite ran. (2109 ms total) 2026-02-24T22:13:25.720 INFO:tasks.workunit.client.0.trial046.stdout:[ PASSED ] 11 tests. 2026-02-24T22:13:25.728 INFO:tasks.workunit.client.0.trial046.stdout:phase 5.1 2026-02-24T22:13:25.747 INFO:tasks.workunit.client.0.trial046.stdout:[==========] Running 19 tests from 1 test suite. 2026-02-24T22:13:25.747 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment set-up. 2026-02-24T22:13:25.747 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 19 tests from LibRGW 2026-02-24T22:13:25.747 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.INIT 2026-02-24T22:13:25.751 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:13:25.751 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:13:25.884 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.INIT (137 ms) 2026-02-24T22:13:25.884 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MOUNT 2026-02-24T22:13:25.885 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2026-02-24T22:13:25.885 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2026-02-24T22:13:25.918 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.CREATE_BUCKET (33 ms) 2026-02-24T22:13:25.918 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2026-02-24T22:13:25.920 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (2 ms) 2026-02-24T22:13:25.920 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.LIST_OBJECTS 2026-02-24T22:13:25.920 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.LIST_OBJECTS (0 ms) 2026-02-24T22:13:25.921 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2026-02-24T22:13:25.921 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (1 ms) 2026-02-24T22:13:25.921 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.OBJ_OPEN 2026-02-24T22:13:25.921 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.OBJ_OPEN (0 ms) 2026-02-24T22:13:25.922 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.PUT_OBJECT 2026-02-24T22:13:25.924 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.PUT_OBJECT (2 ms) 2026-02-24T22:13:25.924 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.GET_OBJECT 2026-02-24T22:13:25.925 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.GET_OBJECT (0 ms) 2026-02-24T22:13:25.926 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.STAT_OBJECT 2026-02-24T22:13:25.926 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2026-02-24T22:13:25.926 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.WRITE_READ_VERIFY 2026-02-24T22:13:25.926 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.WRITE_READ_VERIFY (0 ms) 2026-02-24T22:13:25.926 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.WRITEV 2026-02-24T22:13:25.926 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.WRITEV (0 ms) 2026-02-24T22:13:25.926 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.READV 2026-02-24T22:13:25.926 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.READV (0 ms) 2026-02-24T22:13:25.926 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.READV_AFTER_WRITEV 2026-02-24T22:13:25.926 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.READV_AFTER_WRITEV (0 ms) 2026-02-24T22:13:25.926 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2026-02-24T22:13:25.927 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.DELETE_OBJECT (0 ms) 2026-02-24T22:13:25.927 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2026-02-24T22:13:25.927 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.DELETE_BUCKET (0 ms) 2026-02-24T22:13:25.927 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.CLEANUP 2026-02-24T22:13:25.927 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2026-02-24T22:13:25.927 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.UMOUNT 2026-02-24T22:13:25.927 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2026-02-24T22:13:25.927 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.SHUTDOWN 2026-02-24T22:13:25.928 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:25.926+0000 7f8f45512b80 -1 shutting down 2026-02-24T22:13:25.928 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:25.926+0000 7f8e1f243640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 35623, si_uid : 1000, si_addr0x3e800008b27, si_status0 2026-02-24T22:13:27.850 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.SHUTDOWN (1925 ms) 2026-02-24T22:13:27.850 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 19 tests from LibRGW (2103 ms total) 2026-02-24T22:13:27.850 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:13:27.850 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment tear-down 2026-02-24T22:13:27.850 INFO:tasks.workunit.client.0.trial046.stdout:[==========] 19 tests from 1 test suite ran. (2103 ms total) 2026-02-24T22:13:27.850 INFO:tasks.workunit.client.0.trial046.stdout:[ PASSED ] 19 tests. 2026-02-24T22:13:27.859 INFO:tasks.workunit.client.0.trial046.stdout:phase 5.2 2026-02-24T22:13:27.876 INFO:tasks.workunit.client.0.trial046.stdout:[==========] Running 19 tests from 1 test suite. 2026-02-24T22:13:27.876 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment set-up. 2026-02-24T22:13:27.876 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 19 tests from LibRGW 2026-02-24T22:13:27.876 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.INIT 2026-02-24T22:13:27.879 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:13:27.880 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:13:28.012 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.INIT (135 ms) 2026-02-24T22:13:28.012 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MOUNT 2026-02-24T22:13:28.012 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2026-02-24T22:13:28.012 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2026-02-24T22:13:28.012 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.CREATE_BUCKET (0 ms) 2026-02-24T22:13:28.012 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2026-02-24T22:13:28.017 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (4 ms) 2026-02-24T22:13:28.017 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.LIST_OBJECTS 2026-02-24T22:13:28.017 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.LIST_OBJECTS (0 ms) 2026-02-24T22:13:28.017 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2026-02-24T22:13:28.017 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (0 ms) 2026-02-24T22:13:28.017 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.OBJ_OPEN 2026-02-24T22:13:28.017 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.OBJ_OPEN (0 ms) 2026-02-24T22:13:28.017 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.PUT_OBJECT 2026-02-24T22:13:28.017 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.PUT_OBJECT (0 ms) 2026-02-24T22:13:28.017 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.GET_OBJECT 2026-02-24T22:13:28.017 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.GET_OBJECT (0 ms) 2026-02-24T22:13:28.017 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.STAT_OBJECT 2026-02-24T22:13:28.017 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2026-02-24T22:13:28.017 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.WRITE_READ_VERIFY 2026-02-24T22:13:28.017 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.WRITE_READ_VERIFY (0 ms) 2026-02-24T22:13:28.017 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.WRITEV 2026-02-24T22:13:28.017 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.WRITEV (0 ms) 2026-02-24T22:13:28.017 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.READV 2026-02-24T22:13:28.017 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.READV (0 ms) 2026-02-24T22:13:28.017 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.READV_AFTER_WRITEV 2026-02-24T22:13:28.017 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.READV_AFTER_WRITEV (0 ms) 2026-02-24T22:13:28.017 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2026-02-24T22:13:28.020 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.DELETE_OBJECT (3 ms) 2026-02-24T22:13:28.021 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2026-02-24T22:13:28.030 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.DELETE_BUCKET (9 ms) 2026-02-24T22:13:28.030 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.CLEANUP 2026-02-24T22:13:28.030 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2026-02-24T22:13:28.030 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.UMOUNT 2026-02-24T22:13:28.030 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2026-02-24T22:13:28.030 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.SHUTDOWN 2026-02-24T22:13:28.030 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:28.030+0000 7fc02c279b80 -1 shutting down 2026-02-24T22:13:28.032 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:28.034+0000 7fbf05faa640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 36280, si_uid : 1000, si_addr0x3e800008db8, si_status0 2026-02-24T22:13:29.982 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.SHUTDOWN (1951 ms) 2026-02-24T22:13:29.983 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 19 tests from LibRGW (2105 ms total) 2026-02-24T22:13:29.983 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:13:29.983 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment tear-down 2026-02-24T22:13:29.983 INFO:tasks.workunit.client.0.trial046.stdout:[==========] 19 tests from 1 test suite ran. (2105 ms total) 2026-02-24T22:13:29.983 INFO:tasks.workunit.client.0.trial046.stdout:[ PASSED ] 19 tests. 2026-02-24T22:13:29.991 INFO:tasks.workunit.client.0.trial046.stdout:phase 6.1 2026-02-24T22:13:30.002 INFO:tasks.workunit.client.0.trial046.stdout:[==========] Running 10 tests from 1 test suite. 2026-02-24T22:13:30.002 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment set-up. 2026-02-24T22:13:30.003 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 10 tests from LibRGW 2026-02-24T22:13:30.003 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.INIT 2026-02-24T22:13:30.006 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:13:30.006 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:13:30.136 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.INIT (133 ms) 2026-02-24T22:13:30.136 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.MOUNT 2026-02-24T22:13:30.136 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2026-02-24T22:13:30.137 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.CREATE_BUCKETS 2026-02-24T22:13:30.179 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.CREATE_BUCKETS (42 ms) 2026-02-24T22:13:30.179 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.LOOKUP_BUCKETS 2026-02-24T22:13:30.184 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.LOOKUP_BUCKETS (4 ms) 2026-02-24T22:13:30.184 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.TOPDIR_RENAME 2026-02-24T22:13:30.196 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.TOPDIR_RENAME (12 ms) 2026-02-24T22:13:30.196 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.SUBDIR_RENAME 2026-02-24T22:13:30.211 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.SUBDIR_RENAME (14 ms) 2026-02-24T22:13:30.211 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.CROSS_BUCKET_RENAME 2026-02-24T22:13:30.226 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.CROSS_BUCKET_RENAME (15 ms) 2026-02-24T22:13:30.227 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.CLEANUP 2026-02-24T22:13:30.227 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2026-02-24T22:13:30.227 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.UMOUNT 2026-02-24T22:13:30.227 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2026-02-24T22:13:30.227 INFO:tasks.workunit.client.0.trial046.stdout:[ RUN ] LibRGW.SHUTDOWN 2026-02-24T22:13:30.227 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:30.226+0000 7ff51e781b80 -1 shutting down 2026-02-24T22:13:30.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:30.230+0000 7ff3f84b2640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 36937, si_uid : 1000, si_addr0x3e800009049, si_status0 2026-02-24T22:13:32.107 INFO:tasks.workunit.client.0.trial046.stdout:[ OK ] LibRGW.SHUTDOWN (1880 ms) 2026-02-24T22:13:32.107 INFO:tasks.workunit.client.0.trial046.stdout:[----------] 10 tests from LibRGW (2104 ms total) 2026-02-24T22:13:32.108 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:13:32.108 INFO:tasks.workunit.client.0.trial046.stdout:[----------] Global test environment tear-down 2026-02-24T22:13:32.108 INFO:tasks.workunit.client.0.trial046.stdout:[==========] 10 tests from 1 test suite ran. (2104 ms total) 2026-02-24T22:13:32.108 INFO:tasks.workunit.client.0.trial046.stdout:[ PASSED ] 10 tests. 2026-02-24T22:13:32.116 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T22:13:32.117 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T22:13:32.173 INFO:tasks.workunit:Running workunits matching rgw/test_awssdkv4_sig.sh on client.0... 2026-02-24T22:13:32.176 INFO:tasks.workunit:Running workunit rgw/test_awssdkv4_sig.sh... 2026-02-24T22:13:32.176 DEBUG:teuthology.orchestra.run.trial046:workunit test rgw/test_awssdkv4_sig.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7e05f4fdea5af173284f247d6b0935b337d4ccf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_awssdkv4_sig.sh 2026-02-24T22:13:32.222 INFO:tasks.workunit.client.0.trial046.stderr:+ [ -z ] 2026-02-24T22:13:32.223 INFO:tasks.workunit.client.0.trial046.stderr:+ export AWS_ACCESS_KEY_ID=lNCnR47C2g+ZidCWBAUuwfSAA7Q= 2026-02-24T22:13:32.223 INFO:tasks.workunit.client.0.trial046.stderr:+ export AWS_SECRET_ACCESS_KEY=tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== 2026-02-24T22:13:32.223 INFO:tasks.workunit.client.0.trial046.stderr:+ radosgw-admin user create --uid ceph-test-maven --access-key lNCnR47C2g+ZidCWBAUuwfSAA7Q= --secret tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== --display-name maven test user --email sigv4@example.com 2026-02-24T22:13:32.238 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:13:32.239 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:13:32.256 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.258+0000 7fbcb015ea80 4 Realm: () 2026-02-24T22:13:32.256 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.258+0000 7fbcb015ea80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:13:32.256 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.258+0000 7fbcb015ea80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:13:32.256 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.258+0000 7fbcb015ea80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:13:32.256 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.258+0000 7fbcb015ea80 20 zonegroup default 2026-02-24T22:13:32.257 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.258+0000 7fbcb015ea80 20 rados->read ofs=0 len=0 2026-02-24T22:13:32.257 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.258+0000 7fbcb015ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:13:32.257 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.258+0000 7fbcb015ea80 20 rados->read ofs=0 len=0 2026-02-24T22:13:32.258 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.258+0000 7fbcb015ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:13:32.258 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.258+0000 7fbcb015ea80 20 started sync module instance, tier type = 2026-02-24T22:13:32.258 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.258+0000 7fbcb015ea80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:13:32.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.258+0000 7fbcae433640 0 -- 10.20.193.46:0/807821659 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55acfa3499d0 msgr2=0x55acfa369db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:13:32.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.258+0000 7fbcb015ea80 20 add_watcher() i=1 2026-02-24T22:13:32.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.258+0000 7fbcb015ea80 20 add_watcher() i=5 2026-02-24T22:13:32.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.258+0000 7fbcb015ea80 20 add_watcher() i=6 2026-02-24T22:13:32.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.262+0000 7fbcb015ea80 20 add_watcher() i=4 2026-02-24T22:13:32.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.262+0000 7fbcb015ea80 20 add_watcher() i=3 2026-02-24T22:13:32.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.262+0000 7fbcb015ea80 20 add_watcher() i=7 2026-02-24T22:13:32.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.262+0000 7fbcb015ea80 20 add_watcher() i=0 2026-02-24T22:13:32.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.262+0000 7fbcb015ea80 20 add_watcher() i=2 2026-02-24T22:13:32.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.262+0000 7fbcb015ea80 2 all 8 watchers are set, enabling cache 2026-02-24T22:13:32.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.262+0000 7fbcb015ea80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:13:32.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.262+0000 7fbcb015ea80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:13:32.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.262+0000 7fbcb015ea80 5 note: GC not initialized 2026-02-24T22:13:32.263 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.262+0000 7fbbde7fc640 20 reqs_thread_entry: start 2026-02-24T22:13:32.263 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.262+0000 7fbcae433640 0 -- 10.20.193.46:0/2165894914 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55acfa36a800 msgr2=0x55acfa36abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:13:32.296 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbcb015ea80 20 init_complete bucket index max shards: 1 2026-02-24T22:13:32.296 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbcb015ea80 20 Filter name: none 2026-02-24T22:13:32.296 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbbdcff9640 20 reqs_thread_entry: start 2026-02-24T22:13:32.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbcb015ea80 10 cache get: name=default.rgw.meta+users.uid+ceph-test-maven : miss 2026-02-24T22:13:32.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbcb015ea80 20 rados->read ofs=0 len=0 2026-02-24T22:13:32.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbcb015ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:13:32.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbcb015ea80 10 cache put: name=default.rgw.meta+users.uid+ceph-test-maven info.flags=0x0 2026-02-24T22:13:32.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbcb015ea80 10 adding default.rgw.meta+users.uid+ceph-test-maven to cache LRU end 2026-02-24T22:13:32.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbcb015ea80 10 cache get: name=default.rgw.meta+users.email+sigv4@example.com : miss 2026-02-24T22:13:32.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbcb015ea80 20 rados->read ofs=0 len=0 2026-02-24T22:13:32.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbcb015ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:13:32.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbcb015ea80 10 cache put: name=default.rgw.meta+users.email+sigv4@example.com info.flags=0x0 2026-02-24T22:13:32.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbcb015ea80 10 adding default.rgw.meta+users.email+sigv4@example.com to cache LRU end 2026-02-24T22:13:32.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbcb015ea80 10 cache get: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= : miss 2026-02-24T22:13:32.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbcb015ea80 20 rados->read ofs=0 len=0 2026-02-24T22:13:32.298 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbcb015ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:13:32.298 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbcb015ea80 10 cache put: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= info.flags=0x0 2026-02-24T22:13:32.299 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbcb015ea80 10 adding default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= to cache LRU end 2026-02-24T22:13:32.299 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbcb015ea80 10 cache get: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= : hit (negative entry) 2026-02-24T22:13:32.299 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbcb015ea80 10 cache get: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= : hit (negative entry) 2026-02-24T22:13:32.299 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbcb015ea80 10 cache put: name=default.rgw.meta+users.uid+ceph-test-maven info.flags=0x17 2026-02-24T22:13:32.299 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbcb015ea80 10 moving default.rgw.meta+users.uid+ceph-test-maven to cache LRU end 2026-02-24T22:13:32.299 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbcb015ea80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:ceph-test-maven, ofs0, ns] 2026-02-24T22:13:32.299 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbc897fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1236950581268 cookie 94201422119200 notifier 5684 bl.length()=677 2026-02-24T22:13:32.299 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbc897fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ceph-test-maven info.flags=0x17 2026-02-24T22:13:32.299 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbc897fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ceph-test-maven to cache LRU end 2026-02-24T22:13:32.300 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbcb015ea80 10 cache put: name=default.rgw.meta+users.email+sigv4@example.com info.flags=0x7 2026-02-24T22:13:32.300 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbcb015ea80 10 moving default.rgw.meta+users.email+sigv4@example.com to cache LRU end 2026-02-24T22:13:32.300 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbcb015ea80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.email:sigv4@example.com, ofs0, ns] 2026-02-24T22:13:32.300 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbc897fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1236950581262 cookie 94201420887664 notifier 5684 bl.length()=187 2026-02-24T22:13:32.300 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbc897fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+sigv4@example.com info.flags=0x7 2026-02-24T22:13:32.300 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.298+0000 7fbc897fa640 10 rgw watcher librados: moving default.rgw.meta+users.email+sigv4@example.com to cache LRU end 2026-02-24T22:13:32.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.302+0000 7fbcb015ea80 10 cache put: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= info.flags=0x7 2026-02-24T22:13:32.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.302+0000 7fbcb015ea80 10 moving default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= to cache LRU end 2026-02-24T22:13:32.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.302+0000 7fbcb015ea80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:lNCnR47C2g+ZidCWBAUuwfSAA7Q=, ofs0, ns] 2026-02-24T22:13:32.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.302+0000 7fbc88ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1236950581263 cookie 94201420887664 notifier 5684 bl.length()=197 2026-02-24T22:13:32.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.302+0000 7fbc88ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= info.flags=0x7 2026-02-24T22:13:32.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.302+0000 7fbc88ff9640 10 rgw watcher librados: moving default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= to cache LRU end 2026-02-24T22:13:32.302 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:13:32.302 INFO:tasks.workunit.client.0.trial046.stdout: "user_id": "ceph-test-maven", 2026-02-24T22:13:32.302 INFO:tasks.workunit.client.0.trial046.stdout: "display_name": "maven test user", 2026-02-24T22:13:32.302 INFO:tasks.workunit.client.0.trial046.stdout: "email": "sigv4@example.com", 2026-02-24T22:13:32.302 INFO:tasks.workunit.client.0.trial046.stdout: "suspended": 0, 2026-02-24T22:13:32.302 INFO:tasks.workunit.client.0.trial046.stdout: "max_buckets": 1000, 2026-02-24T22:13:32.302 INFO:tasks.workunit.client.0.trial046.stdout: "subusers": [], 2026-02-24T22:13:32.303 INFO:tasks.workunit.client.0.trial046.stdout: "keys": [ 2026-02-24T22:13:32.303 INFO:tasks.workunit.client.0.trial046.stdout: { 2026-02-24T22:13:32.303 INFO:tasks.workunit.client.0.trial046.stdout: "user": "ceph-test-maven", 2026-02-24T22:13:32.303 INFO:tasks.workunit.client.0.trial046.stdout: "access_key": "lNCnR47C2g+ZidCWBAUuwfSAA7Q=", 2026-02-24T22:13:32.303 INFO:tasks.workunit.client.0.trial046.stdout: "secret_key": "tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g==", 2026-02-24T22:13:32.303 INFO:tasks.workunit.client.0.trial046.stdout: "active": true, 2026-02-24T22:13:32.303 INFO:tasks.workunit.client.0.trial046.stdout: "create_date": "2026-02-24T22:13:32.300654Z" 2026-02-24T22:13:32.303 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:13:32.303 INFO:tasks.workunit.client.0.trial046.stdout: ], 2026-02-24T22:13:32.303 INFO:tasks.workunit.client.0.trial046.stdout: "swift_keys": [], 2026-02-24T22:13:32.303 INFO:tasks.workunit.client.0.trial046.stdout: "caps": [], 2026-02-24T22:13:32.303 INFO:tasks.workunit.client.0.trial046.stdout: "op_mask": "read, write, delete", 2026-02-24T22:13:32.304 INFO:tasks.workunit.client.0.trial046.stdout: "default_placement": "", 2026-02-24T22:13:32.304 INFO:tasks.workunit.client.0.trial046.stdout: "default_storage_class": "", 2026-02-24T22:13:32.304 INFO:tasks.workunit.client.0.trial046.stdout: "placement_tags": [], 2026-02-24T22:13:32.304 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_quota": { 2026-02-24T22:13:32.304 INFO:tasks.workunit.client.0.trial046.stdout: "enabled": false, 2026-02-24T22:13:32.304 INFO:tasks.workunit.client.0.trial046.stdout: "check_on_raw": false, 2026-02-24T22:13:32.304 INFO:tasks.workunit.client.0.trial046.stdout: "max_size": -1, 2026-02-24T22:13:32.304 INFO:tasks.workunit.client.0.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:13:32.304 INFO:tasks.workunit.client.0.trial046.stdout: "max_objects": -1 2026-02-24T22:13:32.304 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:13:32.304 INFO:tasks.workunit.client.0.trial046.stdout: "user_quota": { 2026-02-24T22:13:32.304 INFO:tasks.workunit.client.0.trial046.stdout: "enabled": false, 2026-02-24T22:13:32.304 INFO:tasks.workunit.client.0.trial046.stdout: "check_on_raw": false, 2026-02-24T22:13:32.305 INFO:tasks.workunit.client.0.trial046.stdout: "max_size": -1, 2026-02-24T22:13:32.305 INFO:tasks.workunit.client.0.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:13:32.305 INFO:tasks.workunit.client.0.trial046.stdout: "max_objects": -1 2026-02-24T22:13:32.305 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:13:32.305 INFO:tasks.workunit.client.0.trial046.stdout: "temp_url_keys": [], 2026-02-24T22:13:32.305 INFO:tasks.workunit.client.0.trial046.stdout: "type": "rgw", 2026-02-24T22:13:32.305 INFO:tasks.workunit.client.0.trial046.stdout: "mfa_ids": [], 2026-02-24T22:13:32.305 INFO:tasks.workunit.client.0.trial046.stdout: "account_id": "", 2026-02-24T22:13:32.305 INFO:tasks.workunit.client.0.trial046.stdout: "path": "/", 2026-02-24T22:13:32.305 INFO:tasks.workunit.client.0.trial046.stdout: "create_date": "2026-02-24T22:13:32.300643Z", 2026-02-24T22:13:32.305 INFO:tasks.workunit.client.0.trial046.stdout: "tags": [], 2026-02-24T22:13:32.305 INFO:tasks.workunit.client.0.trial046.stdout: "group_ids": [] 2026-02-24T22:13:32.305 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:13:32.306 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:13:32.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.306+0000 7fbcb015ea80 20 remove_watcher() i=3 2026-02-24T22:13:32.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.306+0000 7fbcb015ea80 2 removed watcher, disabling cache 2026-02-24T22:13:32.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.306+0000 7fbcb015ea80 20 remove_watcher() i=4 2026-02-24T22:13:32.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.306+0000 7fbcb015ea80 20 remove_watcher() i=2 2026-02-24T22:13:32.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.306+0000 7fbcb015ea80 20 remove_watcher() i=5 2026-02-24T22:13:32.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.306+0000 7fbcb015ea80 20 remove_watcher() i=6 2026-02-24T22:13:32.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.306+0000 7fbcb015ea80 20 remove_watcher() i=1 2026-02-24T22:13:32.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.306+0000 7fbcb015ea80 20 remove_watcher() i=7 2026-02-24T22:13:32.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:32.306+0000 7fbcb015ea80 20 remove_watcher() i=0 2026-02-24T22:13:32.311 INFO:tasks.workunit.client.0.trial046.stderr:+ [ -z ] 2026-02-24T22:13:32.311 INFO:tasks.workunit.client.0.trial046.stderr:+ [ -z /home/ubuntu/cephtest ] 2026-02-24T22:13:32.311 INFO:tasks.workunit.client.0.trial046.stderr:+ cat /home/ubuntu/cephtest/url_file 2026-02-24T22:13:32.312 INFO:tasks.workunit.client.0.trial046.stderr:+ export RGW_HTTP_ENDPOINT_URL=https://trial046.front.sepia.ceph.com:443 2026-02-24T22:13:32.312 INFO:tasks.workunit.client.0.trial046.stderr:+ [ -z /home/ubuntu/cephtest/clone.client.0 ] 2026-02-24T22:13:32.312 INFO:tasks.workunit.client.0.trial046.stderr:+ cd /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum 2026-02-24T22:13:32.312 INFO:tasks.workunit.client.0.trial046.stderr:+ ./mvnw clean package 2026-02-24T22:13:32.318 INFO:tasks.workunit.client.0.trial046.stdout:Warning: JAVA_HOME environment variable is not set. 2026-02-24T22:13:33.236 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] Scanning for projects... 2026-02-24T22:13:33.294 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.1/junit-bom-5.10.1.pom 2026-02-24T22:13:33.349 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 908 B Progress (1): 2.4 kB Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.1/junit-bom-5.10.1.pom (5.6 kB at 101 kB/s) 2026-02-24T22:13:33.355 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom/2.20.43/bom-2.20.43.pom 2026-02-24T22:13:33.389 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 720 B Progress (1): 1.7 kB Progress (1): 17 kB Progress (1): 34 kB Progress (1): 46 kB Progress (1): 61 kB Progress (1): 75 kB Progress (1): 83 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom/2.20.43/bom-2.20.43.pom (83 kB at 2.4 MB/s) 2026-02-24T22:13:33.393 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-sdk-java-pom/2.20.43/aws-sdk-java-pom-2.20.43.pom 2026-02-24T22:13:33.419 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 791 B Progress (1): 1.7 kB Progress (1): 4.0 kB Progress (1): 5.7 kB Progress (1): 7.1 kB Progress (1): 9.1 kB Progress (1): 12 kB Progress (1): 15 kB Progress (1): 21 kB Progress (1): 26 kB Progress (1): 29 kB Progress (1): 33 kB Progress (1): 37 kB Progress (1): 40 kB Progress (1): 50 kB Progress (1): 55 kB Progress (1): 60 kB Progress (1): 69 kB Progress (1): 75 kB Progress (1): 77 kB Progress (1): 77 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-sdk-java-pom/2.20.43/aws-sdk-java-pom-2.20.43.pom (77 kB at 3.0 MB/s) 2026-02-24T22:13:33.437 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] 2026-02-24T22:13:33.437 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] ---------------------------< io.ceph:jcksum >--------------------------- 2026-02-24T22:13:33.437 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] Building jcksum 1.0 2026-02-24T22:13:33.437 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] from pom.xml 2026-02-24T22:13:33.438 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] --------------------------------[ jar ]--------------------------------- 2026-02-24T22:13:33.439 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.pom 2026-02-24T22:13:33.464 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 780 B Progress (1): 1.9 kB Progress (1): 4.3 kB Progress (1): 5.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.pom (5.3 kB at 221 kB/s) 2026-02-24T22:13:33.467 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/35/maven-plugins-35.pom 2026-02-24T22:13:33.491 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 752 B Progress (1): 1.8 kB Progress (1): 3.8 kB Progress (1): 6.5 kB Progress (1): 8.3 kB Progress (1): 9.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/35/maven-plugins-35.pom (9.9 kB at 413 kB/s) 2026-02-24T22:13:33.493 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/35/maven-parent-35.pom 2026-02-24T22:13:33.519 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 726 B Progress (1): 1.7 kB Progress (1): 4.5 kB Progress (1): 8.7 kB Progress (1): 12 kB Progress (1): 16 kB Progress (1): 21 kB Progress (1): 24 kB Progress (1): 26 kB Progress (1): 28 kB Progress (1): 30 kB Progress (1): 33 kB Progress (1): 37 kB Progress (1): 39 kB Progress (1): 42 kB Progress (1): 45 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/35/maven-parent-35.pom (45 kB at 1.7 MB/s) 2026-02-24T22:13:33.525 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/25/apache-25.pom 2026-02-24T22:13:33.550 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 743 B Progress (1): 1.7 kB Progress (1): 3.6 kB Progress (1): 5.6 kB Progress (1): 9.4 kB Progress (1): 14 kB Progress (1): 17 kB Progress (1): 19 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/25/apache-25.pom (21 kB at 822 kB/s) 2026-02-24T22:13:33.559 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.jar 2026-02-24T22:13:33.586 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 0.9/36 kB Progress (1): 2.3/36 kB Progress (1): 3.7/36 kB Progress (1): 5.0/36 kB Progress (1): 6.4/36 kB Progress (1): 7.8/36 kB Progress (1): 9.1/36 kB Progress (1): 11/36 kB Progress (1): 12/36 kB Progress (1): 13/36 kB Progress (1): 17/36 kB Progress (1): 22/36 kB Progress (1): 26/36 kB Progress (1): 30/36 kB Progress (1): 34/36 kB Progress (1): 36 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.jar (36 kB at 1.3 MB/s) 2026-02-24T22:13:33.592 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.pom 2026-02-24T22:13:33.617 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 767 B Progress (1): 2.2 kB Progress (1): 4.9 kB Progress (1): 7.5 kB Progress (1): 8.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.pom (8.2 kB at 314 kB/s) 2026-02-24T22:13:33.619 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/39/maven-plugins-39.pom 2026-02-24T22:13:33.644 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 773 B Progress (1): 2.3 kB Progress (1): 4.9 kB Progress (1): 7.4 kB Progress (1): 8.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/39/maven-plugins-39.pom (8.1 kB at 324 kB/s) 2026-02-24T22:13:33.646 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/39/maven-parent-39.pom 2026-02-24T22:13:33.672 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 724 B Progress (1): 1.9 kB Progress (1): 5.5 kB Progress (1): 9.8 kB Progress (1): 14 kB Progress (1): 19 kB Progress (1): 23 kB Progress (1): 25 kB Progress (1): 27 kB Progress (1): 29 kB Progress (1): 32 kB Progress (1): 35 kB Progress (1): 38 kB Progress (1): 41 kB Progress (1): 45 kB Progress (1): 48 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/39/maven-parent-39.pom (48 kB at 1.9 MB/s) 2026-02-24T22:13:33.675 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/29/apache-29.pom 2026-02-24T22:13:33.697 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 764 B Progress (1): 2.1 kB Progress (1): 4.0 kB Progress (1): 7.4 kB Progress (1): 12 kB Progress (1): 15 kB Progress (1): 18 kB Progress (1): 20 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/29/apache-29.pom (21 kB at 942 kB/s) 2026-02-24T22:13:33.702 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.jar 2026-02-24T22:13:33.724 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 3.8/31 kB Progress (1): 8.0/31 kB Progress (1): 12/31 kB Progress (1): 16/31 kB Progress (1): 21/31 kB Progress (1): 24/31 kB Progress (1): 28/31 kB Progress (1): 31 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.jar (31 kB at 1.4 MB/s) 2026-02-24T22:13:33.728 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.pom 2026-02-24T22:13:33.751 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 763 B Progress (1): 2.2 kB Progress (1): 3.9 kB Progress (1): 7.6 kB Progress (1): 9.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.pom (9.8 kB at 427 kB/s) 2026-02-24T22:13:33.756 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.jar 2026-02-24T22:13:33.779 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 3.8/66 kB Progress (1): 8.0/66 kB Progress (1): 12/66 kB Progress (1): 16/66 kB Progress (1): 21/66 kB Progress (1): 25/66 kB Progress (1): 28/66 kB Progress (1): 32/66 kB Progress (1): 36/66 kB Progress (1): 41/66 kB Progress (1): 45/66 kB Progress (1): 49/66 kB Progress (1): 53/66 kB Progress (1): 58/66 kB Progress (1): 62/66 kB Progress (1): 66 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.jar (66 kB at 2.9 MB/s) 2026-02-24T22:13:33.784 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.pom 2026-02-24T22:13:33.807 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 833 B Progress (1): 2.7 kB Progress (1): 5.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.pom (5.3 kB at 231 kB/s) 2026-02-24T22:13:33.809 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire/3.2.3/surefire-3.2.3.pom 2026-02-24T22:13:33.833 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 791 B Progress (1): 2.4 kB Progress (1): 4.1 kB Progress (1): 8.0 kB Progress (1): 12 kB Progress (1): 14 kB Progress (1): 17 kB Progress (1): 20 kB Progress (1): 22 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire/3.2.3/surefire-3.2.3.pom (22 kB at 914 kB/s) 2026-02-24T22:13:33.837 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/41/maven-parent-41.pom 2026-02-24T22:13:33.859 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 724 B Progress (1): 1.9 kB Progress (1): 5.5 kB Progress (1): 9.9 kB Progress (1): 14 kB Progress (1): 19 kB Progress (1): 23 kB Progress (1): 27 kB Progress (1): 28 kB Progress (1): 31 kB Progress (1): 34 kB Progress (1): 37 kB Progress (1): 40 kB Progress (1): 43 kB Progress (1): 48 kB Progress (1): 48 kB Progress (1): 50 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/41/maven-parent-41.pom (50 kB at 2.3 MB/s) 2026-02-24T22:13:33.863 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/31/apache-31.pom 2026-02-24T22:13:33.884 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 739 B Progress (1): 2.1 kB Progress (1): 3.9 kB Progress (1): 6.2 kB Progress (1): 11 kB Progress (1): 16 kB Progress (1): 20 kB Progress (1): 22 kB Progress (1): 24 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/31/apache-31.pom (24 kB at 1.1 MB/s) 2026-02-24T22:13:33.889 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.3/junit-bom-5.9.3.pom 2026-02-24T22:13:33.907 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 907 B Progress (1): 4.1 kB Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.3/junit-bom-5.9.3.pom (5.6 kB at 296 kB/s) 2026-02-24T22:13:33.910 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.jar 2026-02-24T22:13:33.928 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/45 kB Progress (1): 16/45 kB Progress (1): 32/45 kB Progress (1): 45 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.jar (45 kB at 2.6 MB/s) 2026-02-24T22:13:33.934 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.pom 2026-02-24T22:13:33.953 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 787 B Progress (1): 2.2 kB Progress (1): 5.1 kB Progress (1): 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.pom (6.8 kB at 338 kB/s) 2026-02-24T22:13:33.955 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/37/maven-plugins-37.pom 2026-02-24T22:13:33.976 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 753 B Progress (1): 2.2 kB Progress (1): 4.5 kB Progress (1): 7.3 kB Progress (1): 9.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/37/maven-plugins-37.pom (9.9 kB at 495 kB/s) 2026-02-24T22:13:33.977 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/37/maven-parent-37.pom 2026-02-24T22:13:33.998 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 721 B Progress (1): 1.9 kB Progress (1): 5.4 kB Progress (1): 9.5 kB Progress (1): 14 kB Progress (1): 19 kB Progress (1): 23 kB Progress (1): 26 kB Progress (1): 28 kB Progress (1): 30 kB Progress (1): 34 kB Progress (1): 37 kB Progress (1): 40 kB Progress (1): 45 kB Progress (1): 46 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/37/maven-parent-37.pom (46 kB at 2.3 MB/s) 2026-02-24T22:13:34.000 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/27/apache-27.pom 2026-02-24T22:13:34.021 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 745 B Progress (1): 2.1 kB Progress (1): 3.9 kB Progress (1): 7.3 kB Progress (1): 12 kB Progress (1): 16 kB Progress (1): 19 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/27/apache-27.pom (20 kB at 970 kB/s) 2026-02-24T22:13:34.025 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.jar 2026-02-24T22:13:34.049 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/27 kB Progress (1): 16/27 kB Progress (1): 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.jar (27 kB at 1.1 MB/s) 2026-02-24T22:13:34.060 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.pom 2026-02-24T22:13:34.083 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 875 B Progress (1): 3.6 kB Progress (1): 6.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.pom (6.1 kB at 277 kB/s) 2026-02-24T22:13:34.084 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/services/2.20.43/services-2.20.43.pom 2026-02-24T22:13:34.108 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 730 B Progress (1): 3.1 kB Progress (1): 6.3 kB Progress (1): 9.4 kB Progress (1): 13 kB Progress (1): 17 kB Progress (1): 21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/services/2.20.43/services-2.20.43.pom (21 kB at 888 kB/s) 2026-02-24T22:13:34.112 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom-internal/2.20.43/bom-internal-2.20.43.pom 2026-02-24T22:13:34.141 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 757 B Progress (1): 6.3 kB Progress (1): 12 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom-internal/2.20.43/bom-internal-2.20.43.pom (20 kB at 667 kB/s) 2026-02-24T22:13:34.147 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.pom 2026-02-24T22:13:34.170 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 863 B Progress (1): 4.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.pom (4.0 kB at 167 kB/s) 2026-02-24T22:13:34.172 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocols/2.20.43/protocols-2.20.43.pom 2026-02-24T22:13:34.192 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 891 B Progress (1): 1.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocols/2.20.43/protocols-2.20.43.pom (1.6 kB at 74 kB/s) 2026-02-24T22:13:34.193 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/core/2.20.43/core-2.20.43.pom 2026-02-24T22:13:34.216 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 902 B Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/core/2.20.43/core-2.20.43.pom (2.2 kB at 97 kB/s) 2026-02-24T22:13:34.219 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.pom 2026-02-24T22:13:34.239 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 865 B Progress (1): 3.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.pom (3.6 kB at 191 kB/s) 2026-02-24T22:13:34.242 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.pom 2026-02-24T22:13:34.263 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 866 B Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.pom (3.2 kB at 146 kB/s) 2026-02-24T22:13:34.266 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.pom 2026-02-24T22:13:34.288 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 842 B Progress (1): 4.0 kB Progress (1): 8.8 kB Progress (1): 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.pom (11 kB at 476 kB/s) 2026-02-24T22:13:34.291 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.pom 2026-02-24T22:13:34.314 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 893 B Progress (1): 1.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.pom (1.7 kB at 73 kB/s) 2026-02-24T22:13:34.317 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.pom 2026-02-24T22:13:34.340 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 861 B Progress (1): 3.8 kB Progress (1): 3.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.pom (3.8 kB at 166 kB/s) 2026-02-24T22:13:34.342 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.pom 2026-02-24T22:13:34.362 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 853 B Progress (1): 4.6 kB Progress (1): 4.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.pom (4.9 kB at 246 kB/s) 2026-02-24T22:13:34.365 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.pom 2026-02-24T22:13:34.386 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.pom (1.2 kB at 53 kB/s) 2026-02-24T22:13:34.388 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.30/slf4j-api-1.7.30.pom 2026-02-24T22:13:34.411 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.0 kB Progress (1): 3.3 kB Progress (1): 3.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.30/slf4j-api-1.7.30.pom (3.8 kB at 167 kB/s) 2026-02-24T22:13:34.412 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.30/slf4j-parent-1.7.30.pom 2026-02-24T22:13:34.434 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.0 kB Progress (1): 3.2 kB Progress (1): 6.0 kB Progress (1): 9.1 kB Progress (1): 12 kB Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.30/slf4j-parent-1.7.30.pom (14 kB at 628 kB/s) 2026-02-24T22:13:34.437 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.pom 2026-02-24T22:13:34.457 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.8 kB Progress (1): 3.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.pom (3.7 kB at 184 kB/s) 2026-02-24T22:13:34.460 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.pom 2026-02-24T22:13:34.484 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 866 B Progress (1): 2.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.pom (2.5 kB at 98 kB/s) 2026-02-24T22:13:34.487 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.pom 2026-02-24T22:13:34.507 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 870 B Progress (1): 2.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.pom (2.9 kB at 139 kB/s) 2026-02-24T22:13:34.510 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.pom 2026-02-24T22:13:34.531 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 852 B Progress (1): 4.6 kB Progress (1): 7.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.pom (7.4 kB at 353 kB/s) 2026-02-24T22:13:34.534 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.pom 2026-02-24T22:13:34.553 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 848 B Progress (1): 4.5 kB Progress (1): 5.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.pom (5.5 kB at 303 kB/s) 2026-02-24T22:13:34.556 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.pom 2026-02-24T22:13:34.579 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 860 B Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.pom (3.5 kB at 152 kB/s) 2026-02-24T22:13:34.582 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.pom 2026-02-24T22:13:34.604 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.2 kB Progress (1): 3.8 kB Progress (1): 4.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.pom (4.4 kB at 191 kB/s) 2026-02-24T22:13:34.605 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party/2.20.43/third-party-2.20.43.pom 2026-02-24T22:13:34.628 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 898 B Progress (1): 2.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party/2.20.43/third-party-2.20.43.pom (2.4 kB at 109 kB/s) 2026-02-24T22:13:34.630 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.pom 2026-02-24T22:13:34.650 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 852 B Progress (1): 5.0 kB Progress (1): 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.pom (6.4 kB at 339 kB/s) 2026-02-24T22:13:34.653 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.pom 2026-02-24T22:13:34.673 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.2 kB Progress (1): 4.4 kB Progress (1): 6.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.pom (6.1 kB at 303 kB/s) 2026-02-24T22:13:34.676 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.pom 2026-02-24T22:13:34.694 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 866 B Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.pom (3.0 kB at 157 kB/s) 2026-02-24T22:13:34.697 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.pom 2026-02-24T22:13:34.716 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 862 B Progress (1): 3.5 kB Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.pom (3.5 kB at 183 kB/s) 2026-02-24T22:13:34.719 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.pom 2026-02-24T22:13:34.739 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 862 B Progress (1): 4.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.pom (4.1 kB at 215 kB/s) 2026-02-24T22:13:34.740 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-clients/2.20.43/http-clients-2.20.43.pom 2026-02-24T22:13:34.761 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 886 B Progress (1): 1.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-clients/2.20.43/http-clients-2.20.43.pom (1.8 kB at 89 kB/s) 2026-02-24T22:13:34.763 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom 2026-02-24T22:13:34.792 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 863 B Progress (1): 2.8 kB Progress (1): 5.1 kB Progress (1): 6.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom (6.6 kB at 236 kB/s) 2026-02-24T22:13:34.793 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.5.13/httpcomponents-client-4.5.13.pom 2026-02-24T22:13:34.817 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 838 B Progress (1): 2.3 kB Progress (1): 5.1 kB Progress (1): 8.5 kB Progress (1): 11 kB Progress (1): 15 kB Progress (1): 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.5.13/httpcomponents-client-4.5.13.pom (16 kB at 683 kB/s) 2026-02-24T22:13:34.818 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/11/httpcomponents-parent-11.pom 2026-02-24T22:13:34.842 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 803 B Progress (1): 2.2 kB Progress (1): 3.7 kB Progress (1): 5.3 kB Progress (1): 6.5 kB Progress (1): 8.9 kB Progress (1): 12 kB Progress (1): 15 kB Progress (1): 18 kB Progress (1): 23 kB Progress (1): 26 kB Progress (1): 28 kB Progress (1): 32 kB Progress (1): 34 kB Progress (1): 35 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/11/httpcomponents-parent-11.pom (35 kB at 1.5 MB/s) 2026-02-24T22:13:34.843 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/21/apache-21.pom 2026-02-24T22:13:34.862 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 753 B Progress (1): 2.1 kB Progress (1): 3.9 kB Progress (1): 8.3 kB Progress (1): 13 kB Progress (1): 16 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/21/apache-21.pom (17 kB at 900 kB/s) 2026-02-24T22:13:34.865 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.pom 2026-02-24T22:13:34.890 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 869 B Progress (1): 2.5 kB Progress (1): 4.7 kB Progress (1): 5.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.pom (5.0 kB at 199 kB/s) 2026-02-24T22:13:34.892 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-core/4.4.13/httpcomponents-core-4.4.13.pom 2026-02-24T22:13:34.913 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 846 B Progress (1): 2.3 kB Progress (1): 4.5 kB Progress (1): 7.1 kB Progress (1): 11 kB Progress (1): 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-core/4.4.13/httpcomponents-core-4.4.13.pom (13 kB at 627 kB/s) 2026-02-24T22:13:34.916 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.11/commons-codec-1.11.pom 2026-02-24T22:13:34.938 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 752 B Progress (1): 2.0 kB Progress (1): 4.1 kB Progress (1): 6.5 kB Progress (1): 8.4 kB Progress (1): 11 kB Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.11/commons-codec-1.11.pom (14 kB at 635 kB/s) 2026-02-24T22:13:34.939 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/42/commons-parent-42.pom 2026-02-24T22:13:34.963 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 707 B Progress (1): 1.9 kB Progress (1): 3.1 kB Progress (1): 4.5 kB Progress (1): 5.9 kB Progress (1): 7.5 kB Progress (1): 11 kB Progress (1): 14 kB Progress (1): 17 kB Progress (1): 21 kB Progress (1): 24 kB Progress (1): 26 kB Progress (1): 29 kB Progress (1): 31 kB Progress (1): 34 kB Progress (1): 34 kB Progress (1): 38 kB Progress (1): 40 kB Progress (1): 44 kB Progress (1): 49 kB Progress (1): 53 kB Progress (1): 56 kB Progress (1): 59 kB Progress (1): 61 kB Progress (1): 63 kB Progress (1): 65 kB Progress (1): 66 kB Progress (1): 68 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/42/commons-parent-42.pom (68 kB at 3.0 MB/s) 2026-02-24T22:13:34.968 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/18/apache-18.pom 2026-02-24T22:13:34.987 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 748 B Progress (1): 2.1 kB Progress (1): 3.9 kB Progress (1): 8.0 kB Progress (1): 12 kB Progress (1): 15 kB Progress (1): 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/18/apache-18.pom (16 kB at 870 kB/s) 2026-02-24T22:13:34.990 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.pom 2026-02-24T22:13:35.014 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 754 B Progress (1): 2.0 kB Progress (1): 4.0 kB Progress (1): 6.4 kB Progress (1): 8.3 kB Progress (1): 10 kB Progress (1): 13 kB Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.pom (15 kB at 643 kB/s) 2026-02-24T22:13:35.015 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/52/commons-parent-52.pom 2026-02-24T22:13:35.041 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 703 B Progress (1): 1.8 kB Progress (1): 3.1 kB Progress (1): 4.6 kB Progress (1): 6.7 kB Progress (1): 8.1 kB Progress (1): 10 kB Progress (1): 12 kB Progress (1): 15 kB Progress (1): 17 kB Progress (1): 20 kB Progress (1): 22 kB Progress (1): 25 kB Progress (1): 29 kB Progress (1): 32 kB Progress (1): 33 kB Progress (1): 36 kB Progress (1): 40 kB Progress (1): 42 kB Progress (1): 44 kB Progress (1): 47 kB Progress (1): 51 kB Progress (1): 54 kB Progress (1): 56 kB Progress (1): 61 kB Progress (1): 66 kB Progress (1): 73 kB Progress (1): 76 kB Progress (1): 79 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/52/commons-parent-52.pom (79 kB at 3.2 MB/s) 2026-02-24T22:13:35.044 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/23/apache-23.pom 2026-02-24T22:13:35.063 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 749 B Progress (1): 2.1 kB Progress (1): 3.9 kB Progress (1): 7.9 kB Progress (1): 13 kB Progress (1): 16 kB Progress (1): 18 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/23/apache-23.pom (18 kB at 970 kB/s) 2026-02-24T22:13:35.065 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.pom 2026-02-24T22:13:35.087 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 765 B Progress (1): 2.1 kB Progress (1): 5.2 kB Progress (1): 7.6 kB Progress (1): 10 kB Progress (1): 12 kB Progress (1): 15 kB Progress (1): 18 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.pom (20 kB at 951 kB/s) 2026-02-24T22:13:35.088 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/65/commons-parent-65.pom 2026-02-24T22:13:35.114 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 705 B Progress (1): 1.9 kB Progress (1): 3.1 kB Progress (1): 4.7 kB Progress (1): 6.4 kB Progress (1): 8.4 kB Progress (1): 10 kB Progress (1): 13 kB Progress (1): 15 kB Progress (1): 18 kB Progress (1): 19 kB Progress (1): 23 kB Progress (1): 25 kB Progress (1): 28 kB Progress (1): 32 kB Progress (1): 32 kB Progress (1): 36 kB Progress (1): 41 kB Progress (1): 44 kB Progress (1): 46 kB Progress (1): 49 kB Progress (1): 51 kB Progress (1): 55 kB Progress (1): 58 kB Progress (1): 60 kB Progress (1): 64 kB Progress (1): 68 kB Progress (1): 71 kB Progress (1): 74 kB Progress (1): 76 kB Progress (1): 78 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/65/commons-parent-65.pom (78 kB at 3.2 MB/s) 2026-02-24T22:13:35.117 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.pom 2026-02-24T22:13:35.140 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.0 kB Progress (1): 3.3 kB Progress (1): 3.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.pom (3.8 kB at 167 kB/s) 2026-02-24T22:13:35.141 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.28/slf4j-parent-1.7.28.pom 2026-02-24T22:13:35.163 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.0 kB Progress (1): 3.2 kB Progress (1): 6.0 kB Progress (1): 9.1 kB Progress (1): 12 kB Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.28/slf4j-parent-1.7.28.pom (14 kB at 628 kB/s) 2026-02-24T22:13:35.165 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.pom 2026-02-24T22:13:35.187 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 944 B Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.pom (3.2 kB at 146 kB/s) 2026-02-24T22:13:35.190 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.pom 2026-02-24T22:13:35.211 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 954 B Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.pom (3.2 kB at 145 kB/s) 2026-02-24T22:13:35.212 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.pom 2026-02-24T22:13:35.233 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 996 B Progress (1): 2.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.pom (2.0 kB at 102 kB/s) 2026-02-24T22:13:35.234 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.pom 2026-02-24T22:13:35.254 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 966 B Progress (1): 2.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.pom (2.8 kB at 149 kB/s) 2026-02-24T22:13:35.255 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.pom 2026-02-24T22:13:35.278 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.0 kB Progress (1): 1.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.pom (1.5 kB at 69 kB/s) 2026-02-24T22:13:35.279 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.pom 2026-02-24T22:13:35.300 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 964 B Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.pom (3.0 kB at 143 kB/s) 2026-02-24T22:13:35.302 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.pom 2026-02-24T22:13:35.321 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 963 B Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.pom (3.2 kB at 169 kB/s) 2026-02-24T22:13:35.322 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.pom 2026-02-24T22:13:35.344 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 958 B Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.pom (3.2 kB at 146 kB/s) 2026-02-24T22:13:35.346 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.pom 2026-02-24T22:13:35.372 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.pom (1.0 kB at 41 kB/s) 2026-02-24T22:13:35.374 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.pom 2026-02-24T22:13:35.395 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 964 B Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.pom (964 B at 46 kB/s) 2026-02-24T22:13:35.397 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.pom 2026-02-24T22:13:35.421 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 949 B Progress (1): 3.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.pom (3.1 kB at 127 kB/s) 2026-02-24T22:13:35.423 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.pom 2026-02-24T22:13:35.451 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 962 B Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.pom (3.0 kB at 108 kB/s) 2026-02-24T22:13:35.452 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.pom 2026-02-24T22:13:35.472 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 967 B Progress (1): 3.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.pom (3.4 kB at 181 kB/s) 2026-02-24T22:13:35.473 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.pom 2026-02-24T22:13:35.495 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 968 B Progress (1): 3.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.pom (3.4 kB at 164 kB/s) 2026-02-24T22:13:35.496 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.pom 2026-02-24T22:13:35.517 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 967 B Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.pom (3.0 kB at 144 kB/s) 2026-02-24T22:13:35.532 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.jar 2026-02-24T22:13:35.606 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 0/3.3 MB Progress (1): 0/3.3 MB Progress (1): 0/3.3 MB Progress (1): 0/3.3 MB Progress (1): 0/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.3/3.3 MB Progress (1): 3.3/3.3 MB Progress (1): 3.3 MB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.jar (3.3 MB at 44 MB/s) 2026-02-24T22:13:35.608 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.jar 2026-02-24T22:13:35.608 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.jar 2026-02-24T22:13:35.608 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.jar 2026-02-24T22:13:35.608 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.jar 2026-02-24T22:13:35.608 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.jar 2026-02-24T22:13:35.645 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/16 kB Progress (1): 16/16 kB Progress (1): 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.jar (16 kB at 427 kB/s) 2026-02-24T22:13:35.645 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.jar 2026-02-24T22:13:35.662 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/16 kB Progress (1): 16/16 kB Progress (1): 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.jar (16 kB at 292 kB/s) 2026-02-24T22:13:35.663 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.jar 2026-02-24T22:13:35.678 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 0.9/35 kB Progress (1): 2.3/35 kB Progress (1): 3.7/35 kB Progress (1): 5.0/35 kB Progress (1): 6.4/35 kB Progress (1): 7.8/35 kB Progress (1): 9.1/35 kB Progress (1): 11/35 kB Progress (1): 12/35 kB Progress (1): 13/35 kB Progress (1): 15/35 kB Progress (1): 16/35 kB Progress (2): 16/35 kB | 0.9/49 kB Progress (2): 17/35 kB | 0.9/49 kB Progress (2): 19/35 kB | 0.9/49 kB Progress (2): 19/35 kB | 2.3/49 kB Progress (2): 20/35 kB | 2.3/49 kB Progress (2): 20/35 kB | 3.7/49 kB Progress (2): 21/35 kB | 3.7/49 kB Progress (2): 21/35 kB | 5.0/49 kB Progress (2): 23/35 kB | 5.0/49 kB Progress (2): 23/35 kB | 6.4/49 kB Progress (2): 24/35 kB | 6.4/49 kB Progress (2): 24/35 kB | 7.8/49 kB Progress (2): 26/35 kB | 7.8/49 kB Progress (2): 26/35 kB | 9.1/49 kB Progress (2): 27/35 kB | 9.1/49 kB Progress (2): 27/35 kB | 11/49 kB Progress (2): 28/35 kB | 11/49 kB Progress (2): 28/35 kB | 12/49 kB Progress (2): 30/35 kB | 12/49 kB Progress (2): 30/35 kB | 13/49 kB Progress (2): 31/35 kB | 13/49 kB Progress (2): 31/35 kB | 15/49 kB Progress (2): 32/35 kB | 15/49 kB Progress (2): 32/35 kB | 16/49 kB Progress (2): 34/35 kB | 16/49 kB Progress (2): 34/35 kB | 17/49 kB Progress (2): 35/35 kB | 17/49 kB Progress (2): 35 kB | 17/49 kB Progress (2): 35 kB | 19/49 kB Progress (2): 35 kB | 20/49 kB Progress (2): 35 kB | 21/49 kB Progress (2): 35 kB | 23/49 kB Progress (2): 35 kB | 24/49 kB Progress (2): 35 kB | 26/49 kB Progress (2): 35 kB | 27/49 kB Progress (2): 35 kB | 28/49 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.jar (35 kB at 525 kB/s) 2026-02-24T22:13:35.678 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 30/49 kB Downloading from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.jar 2026-02-24T22:13:35.678 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 31/49 kB Progress (1): 32/49 kB Progress (1): 34/49 kB Progress (1): 35/49 kB Progress (1): 37/49 kB Progress (1): 38/49 kB Progress (1): 39/49 kB Progress (1): 41/49 kB Progress (1): 42/49 kB Progress (1): 43/49 kB Progress (1): 45/49 kB Progress (1): 46/49 kB Progress (1): 47/49 kB Progress (1): 49/49 kB Progress (1): 49 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.jar (49 kB at 724 kB/s) 2026-02-24T22:13:35.678 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.jar 2026-02-24T22:13:35.683 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 0.9/71 kB Progress (1): 2.3/71 kB Progress (1): 3.7/71 kB Progress (1): 5.0/71 kB Progress (1): 6.4/71 kB Progress (1): 7.8/71 kB Progress (1): 9.1/71 kB Progress (1): 11/71 kB Progress (1): 12/71 kB Progress (1): 13/71 kB Progress (1): 15/71 kB Progress (1): 16/71 kB Progress (1): 17/71 kB Progress (1): 18/71 kB Progress (1): 20/71 kB Progress (1): 21/71 kB Progress (1): 23/71 kB Progress (1): 24/71 kB Progress (1): 25/71 kB Progress (1): 27/71 kB Progress (1): 28/71 kB Progress (1): 29/71 kB Progress (1): 31/71 kB Progress (1): 32/71 kB Progress (1): 33/71 kB Progress (1): 35/71 kB Progress (1): 36/71 kB Progress (1): 38/71 kB Progress (1): 39/71 kB Progress (1): 40/71 kB Progress (1): 42/71 kB Progress (1): 43/71 kB Progress (1): 44/71 kB Progress (1): 46/71 kB Progress (1): 47/71 kB Progress (1): 49/71 kB Progress (1): 50/71 kB Progress (1): 51/71 kB Progress (1): 53/71 kB Progress (1): 54/71 kB Progress (1): 55/71 kB Progress (1): 60/71 kB Progress (1): 64/71 kB Progress (1): 68/71 kB Progress (1): 71 kB Progress (2): 71 kB | 7.7/782 kB Progress (2): 71 kB | 16/782 kB Progress (2): 71 kB | 32/782 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.jar (71 kB at 928 kB/s) 2026-02-24T22:13:35.683 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 49/782 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.jar 2026-02-24T22:13:35.693 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 65/782 kB Progress (1): 81/782 kB Progress (1): 98/782 kB Progress (1): 114/782 kB Progress (1): 131/782 kB Progress (1): 147/782 kB Progress (1): 163/782 kB Progress (1): 180/782 kB Progress (1): 196/782 kB Progress (1): 213/782 kB Progress (1): 229/782 kB Progress (1): 245/782 kB Progress (1): 262/782 kB Progress (1): 278/782 kB Progress (1): 294/782 kB Progress (1): 311/782 kB Progress (1): 327/782 kB Progress (1): 344/782 kB Progress (1): 360/782 kB Progress (1): 376/782 kB Progress (1): 393/782 kB Progress (1): 409/782 kB Progress (1): 426/782 kB Progress (1): 442/782 kB Progress (1): 458/782 kB Progress (1): 475/782 kB Progress (1): 491/782 kB Progress (1): 507/782 kB Progress (1): 524/782 kB Progress (1): 540/782 kB Progress (1): 557/782 kB Progress (1): 573/782 kB Progress (1): 589/782 kB Progress (1): 606/782 kB Progress (1): 622/782 kB Progress (1): 639/782 kB Progress (1): 655/782 kB Progress (1): 671/782 kB Progress (1): 688/782 kB Progress (1): 704/782 kB Progress (1): 720/782 kB Progress (1): 737/782 kB Progress (1): 753/782 kB Progress (2): 753/782 kB | 0.9/98 kB Progress (2): 753/782 kB | 2.3/98 kB Progress (2): 770/782 kB | 2.3/98 kB Progress (2): 770/782 kB | 3.7/98 kB Progress (2): 770/782 kB | 5.0/98 kB Progress (2): 770/782 kB | 6.4/98 kB Progress (2): 782 kB | 6.4/98 kB Progress (2): 782 kB | 7.8/98 kB Progress (2): 782 kB | 9.1/98 kB Progress (2): 782 kB | 11/98 kB Progress (2): 782 kB | 12/98 kB Progress (2): 782 kB | 13/98 kB Progress (2): 782 kB | 15/98 kB Progress (2): 782 kB | 16/98 kB Progress (2): 782 kB | 17/98 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.jar (782 kB at 9.5 MB/s) 2026-02-24T22:13:35.694 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 19/98 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.jar 2026-02-24T22:13:35.695 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 20/98 kB Progress (1): 21/98 kB Progress (1): 23/98 kB Progress (1): 24/98 kB Progress (1): 26/98 kB Progress (1): 27/98 kB Progress (1): 28/98 kB Progress (1): 30/98 kB Progress (1): 31/98 kB Progress (1): 32/98 kB Progress (1): 34/98 kB Progress (1): 35/98 kB Progress (1): 37/98 kB Progress (1): 38/98 kB Progress (1): 39/98 kB Progress (1): 41/98 kB Progress (1): 42/98 kB Progress (1): 43/98 kB Progress (1): 45/98 kB Progress (1): 46/98 kB Progress (1): 47/98 kB Progress (1): 49/98 kB Progress (1): 50/98 kB Progress (1): 52/98 kB Progress (1): 53/98 kB Progress (1): 54/98 kB Progress (1): 56/98 kB Progress (1): 60/98 kB Progress (1): 61/98 kB Progress (1): 65/98 kB Progress (1): 69/98 kB Progress (1): 73/98 kB Progress (1): 78/98 kB Progress (1): 82/98 kB Progress (1): 86/98 kB Progress (1): 90/98 kB Progress (1): 95/98 kB Progress (1): 98 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.jar (98 kB at 1.1 MB/s) 2026-02-24T22:13:35.695 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.jar 2026-02-24T22:13:35.704 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 0.9/11 kB Progress (1): 2.3/11 kB Progress (1): 3.7/11 kB Progress (1): 5.0/11 kB Progress (1): 6.4/11 kB Progress (1): 7.8/11 kB Progress (1): 9.1/11 kB Progress (1): 11/11 kB Progress (1): 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.jar (11 kB at 118 kB/s) 2026-02-24T22:13:35.704 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.jar 2026-02-24T22:13:35.706 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 3.8/30 kB Progress (1): 8.0/30 kB Progress (1): 12/30 kB Progress (1): 16/30 kB Progress (1): 20/30 kB Progress (1): 24/30 kB Progress (1): 28/30 kB Progress (1): 30 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.jar (30 kB at 308 kB/s) 2026-02-24T22:13:35.706 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.jar 2026-02-24T22:13:35.709 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/81 kB Progress (1): 16/81 kB Progress (1): 29/81 kB Progress (1): 45/81 kB Progress (1): 61/81 kB Progress (1): 78/81 kB Progress (1): 81 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.jar (81 kB at 805 kB/s) 2026-02-24T22:13:35.709 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.jar 2026-02-24T22:13:35.743 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 3.8/798 kB Progress (1): 8.0/798 kB Progress (1): 12/798 kB Progress (1): 16/798 kB Progress (1): 21/798 kB Progress (1): 25/798 kB Progress (1): 29/798 kB Progress (1): 33/798 kB Progress (1): 37/798 kB Progress (1): 41/798 kB Progress (1): 46/798 kB Progress (1): 50/798 kB Progress (1): 54/798 kB Progress (1): 58/798 kB Progress (1): 63/798 kB Progress (1): 67/798 kB Progress (1): 71/798 kB Progress (1): 75/798 kB Progress (1): 79/798 kB Progress (1): 84/798 kB Progress (1): 88/798 kB Progress (1): 92/798 kB Progress (1): 96/798 kB Progress (1): 101/798 kB Progress (1): 105/798 kB Progress (1): 109/798 kB Progress (1): 113/798 kB Progress (1): 117/798 kB Progress (1): 122/798 kB Progress (2): 122/798 kB | 0.9/13 kB Progress (2): 122/798 kB | 2.3/13 kB Progress (2): 122/798 kB | 3.7/13 kB Progress (2): 122/798 kB | 5.0/13 kB Progress (2): 122/798 kB | 6.4/13 kB Progress (2): 122/798 kB | 11/13 kB Progress (2): 138/798 kB | 11/13 kB Progress (2): 138/798 kB | 13 kB Progress (2): 154/798 kB | 13 kB Progress (2): 160/798 kB | 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.jar (13 kB at 105 kB/s) 2026-02-24T22:13:35.743 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 176/798 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.jar 2026-02-24T22:13:35.743 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 193/798 kB Progress (2): 193/798 kB | 3.8/182 kB Progress (2): 209/798 kB | 3.8/182 kB Progress (3): 209/798 kB | 3.8/182 kB | 7.7/151 kB Progress (3): 213/798 kB | 3.8/182 kB | 7.7/151 kB Progress (3): 213/798 kB | 8.0/182 kB | 7.7/151 kB Progress (3): 213/798 kB | 8.0/182 kB | 16/151 kB Progress (3): 229/798 kB | 8.0/182 kB | 16/151 kB Progress (3): 229/798 kB | 12/182 kB | 16/151 kB Progress (3): 246/798 kB | 12/182 kB | 16/151 kB Progress (3): 246/798 kB | 12/182 kB | 32/151 kB Progress (3): 246/798 kB | 16/182 kB | 32/151 kB Progress (3): 262/798 kB | 16/182 kB | 32/151 kB Progress (3): 262/798 kB | 21/182 kB | 32/151 kB Progress (3): 262/798 kB | 21/182 kB | 49/151 kB Progress (3): 279/798 kB | 21/182 kB | 49/151 kB Progress (3): 279/798 kB | 25/182 kB | 49/151 kB Progress (3): 279/798 kB | 25/182 kB | 65/151 kB Progress (3): 295/798 kB | 25/182 kB | 65/151 kB Progress (3): 295/798 kB | 29/182 kB | 65/151 kB Progress (3): 295/798 kB | 29/182 kB | 66/151 kB Progress (3): 311/798 kB | 29/182 kB | 66/151 kB Progress (3): 311/798 kB | 33/182 kB | 66/151 kB Progress (3): 311/798 kB | 33/182 kB | 82/151 kB Progress (3): 328/798 kB | 33/182 kB | 82/151 kB Progress (3): 328/798 kB | 38/182 kB | 82/151 kB Progress (3): 328/798 kB | 38/182 kB | 98/151 kB Progress (3): 344/798 kB | 38/182 kB | 98/151 kB Progress (3): 344/798 kB | 42/182 kB | 98/151 kB Progress (3): 344/798 kB | 42/182 kB | 115/151 kB Progress (3): 360/798 kB | 42/182 kB | 115/151 kB Progress (3): 360/798 kB | 46/182 kB | 115/151 kB Progress (3): 360/798 kB | 46/182 kB | 131/151 kB Progress (3): 377/798 kB | 46/182 kB | 131/151 kB Progress (3): 377/798 kB | 50/182 kB | 131/151 kB Progress (3): 377/798 kB | 50/182 kB | 147/151 kB Progress (3): 393/798 kB | 50/182 kB | 147/151 kB Progress (3): 393/798 kB | 50/182 kB | 151 kB Progress (3): 393/798 kB | 55/182 kB | 151 kB Progress (3): 410/798 kB | 55/182 kB | 151 kB Progress (3): 410/798 kB | 59/182 kB | 151 kB Progress (3): 426/798 kB | 59/182 kB | 151 kB Progress (3): 426/798 kB | 63/182 kB | 151 kB Progress (3): 442/798 kB | 63/182 kB | 151 kB Progress (3): 442/798 kB | 67/182 kB | 151 kB Progress (3): 459/798 kB | 67/182 kB | 151 kB Progress (3): 459/798 kB | 71/182 kB | 151 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.jar (151 kB at 1.2 MB/s) 2026-02-24T22:13:35.743 INFO:tasks.workunit.client.0.trial046.stdout:Progress (2): 459/798 kB | 76/182 kB Progress (2): 475/798 kB | 76/182 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.jar 2026-02-24T22:13:35.743 INFO:tasks.workunit.client.0.trial046.stdout:Progress (2): 475/798 kB | 80/182 kB Progress (2): 492/798 kB | 80/182 kB Progress (2): 492/798 kB | 84/182 kB Progress (2): 508/798 kB | 84/182 kB Progress (2): 508/798 kB | 88/182 kB Progress (2): 524/798 kB | 88/182 kB Progress (2): 524/798 kB | 93/182 kB Progress (2): 541/798 kB | 93/182 kB Progress (2): 541/798 kB | 97/182 kB Progress (2): 541/798 kB | 101/182 kB Progress (2): 557/798 kB | 101/182 kB Progress (2): 557/798 kB | 105/182 kB Progress (2): 557/798 kB | 110/182 kB Progress (2): 573/798 kB | 110/182 kB Progress (2): 573/798 kB | 114/182 kB Progress (2): 573/798 kB | 118/182 kB Progress (2): 590/798 kB | 118/182 kB Progress (2): 590/798 kB | 131/182 kB Progress (2): 606/798 kB | 131/182 kB Progress (2): 606/798 kB | 147/182 kB Progress (2): 623/798 kB | 147/182 kB Progress (2): 623/798 kB | 163/182 kB Progress (2): 639/798 kB | 163/182 kB Progress (2): 639/798 kB | 180/182 kB Progress (2): 655/798 kB | 180/182 kB Progress (2): 655/798 kB | 182 kB Progress (2): 672/798 kB | 182 kB Progress (2): 688/798 kB | 182 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.jar (182 kB at 1.4 MB/s) 2026-02-24T22:13:35.743 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 705/798 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.jar 2026-02-24T22:13:35.743 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 721/798 kB Progress (1): 737/798 kB Progress (1): 754/798 kB Progress (1): 770/798 kB Progress (1): 786/798 kB Progress (1): 798 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.jar (798 kB at 6.0 MB/s) 2026-02-24T22:13:35.744 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.jar 2026-02-24T22:13:35.756 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/30 kB Progress (1): 16/30 kB Progress (1): 30 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.jar (30 kB at 204 kB/s) 2026-02-24T22:13:35.756 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.jar 2026-02-24T22:13:35.759 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 3.8/27 kB Progress (1): 8.0/27 kB Progress (1): 12/27 kB Progress (1): 16/27 kB Progress (1): 21/27 kB Progress (1): 24/27 kB Progress (1): 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.jar (27 kB at 182 kB/s) 2026-02-24T22:13:35.759 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar 2026-02-24T22:13:35.766 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/13 kB Progress (1): 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.jar (13 kB at 83 kB/s) 2026-02-24T22:13:35.766 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.jar 2026-02-24T22:13:35.771 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/385 kB Progress (1): 16/385 kB Progress (1): 32/385 kB Progress (1): 48/385 kB Progress (1): 65/385 kB Progress (1): 81/385 kB Progress (1): 98/385 kB Progress (1): 114/385 kB Progress (1): 130/385 kB Progress (1): 131/385 kB Progress (1): 147/385 kB Progress (1): 164/385 kB Progress (1): 180/385 kB Progress (1): 197/385 kB Progress (1): 213/385 kB Progress (1): 229/385 kB Progress (1): 246/385 kB Progress (1): 262/385 kB Progress (1): 279/385 kB Progress (1): 295/385 kB Progress (1): 311/385 kB Progress (1): 328/385 kB Progress (1): 344/385 kB Progress (1): 360/385 kB Progress (1): 377/385 kB Progress (1): 385 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.jar (385 kB at 2.4 MB/s) 2026-02-24T22:13:35.771 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.jar 2026-02-24T22:13:35.773 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/75 kB Progress (1): 16/75 kB Progress (1): 32/75 kB Progress (1): 49/75 kB Progress (1): 57/75 kB Progress (1): 73/75 kB Progress (1): 75 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.jar (75 kB at 452 kB/s) 2026-02-24T22:13:35.774 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.jar 2026-02-24T22:13:35.794 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 3.8/780 kB Progress (1): 8.0/780 kB Progress (1): 8.2/780 kB Progress (1): 12/780 kB Progress (1): 17/780 kB Progress (1): 21/780 kB Progress (1): 25/780 kB Progress (1): 29/780 kB Progress (1): 34/780 kB Progress (1): 38/780 kB Progress (1): 42/780 kB Progress (1): 46/780 kB Progress (1): 50/780 kB Progress (1): 55/780 kB Progress (1): 59/780 kB Progress (1): 63/780 kB Progress (1): 67/780 kB Progress (1): 72/780 kB Progress (1): 76/780 kB Progress (1): 80/780 kB Progress (1): 84/780 kB Progress (1): 89/780 kB Progress (1): 93/780 kB Progress (1): 97/780 kB Progress (1): 101/780 kB Progress (1): 105/780 kB Progress (1): 110/780 kB Progress (1): 114/780 kB Progress (1): 118/780 kB Progress (1): 122/780 kB Progress (1): 123/780 kB Progress (1): 139/780 kB Progress (1): 156/780 kB Progress (1): 172/780 kB Progress (1): 188/780 kB Progress (1): 205/780 kB Progress (1): 221/780 kB Progress (1): 238/780 kB Progress (1): 254/780 kB Progress (1): 270/780 kB Progress (1): 287/780 kB Progress (1): 303/780 kB Progress (1): 319/780 kB Progress (1): 335/780 kB Progress (1): 352/780 kB Progress (2): 352/780 kB | 7.7/354 kB Progress (2): 352/780 kB | 12/354 kB Progress (2): 368/780 kB | 12/354 kB Progress (2): 368/780 kB | 29/354 kB Progress (2): 384/780 kB | 29/354 kB Progress (2): 384/780 kB | 45/354 kB Progress (2): 401/780 kB | 45/354 kB Progress (2): 401/780 kB | 61/354 kB Progress (2): 417/780 kB | 61/354 kB Progress (2): 417/780 kB | 78/354 kB Progress (2): 417/780 kB | 94/354 kB Progress (2): 417/780 kB | 111/354 kB Progress (2): 434/780 kB | 111/354 kB Progress (2): 434/780 kB | 127/354 kB Progress (2): 450/780 kB | 127/354 kB Progress (2): 450/780 kB | 143/354 kB Progress (2): 466/780 kB | 143/354 kB Progress (2): 466/780 kB | 160/354 kB Progress (2): 483/780 kB | 160/354 kB Progress (2): 483/780 kB | 176/354 kB Progress (2): 499/780 kB | 176/354 kB Progress (2): 499/780 kB | 193/354 kB Progress (2): 499/780 kB | 197/354 kB Progress (2): 515/780 kB | 197/354 kB Progress (2): 515/780 kB | 213/354 kB Progress (2): 532/780 kB | 213/354 kB Progress (2): 532/780 kB | 229/354 kB Progress (2): 548/780 kB | 229/354 kB Progress (2): 548/780 kB | 246/354 kB Progress (2): 565/780 kB | 246/354 kB Progress (2): 565/780 kB | 262/354 kB Progress (2): 581/780 kB | 262/354 kB Progress (2): 581/780 kB | 279/354 kB Progress (2): 597/780 kB | 279/354 kB Progress (2): 597/780 kB | 295/354 kB Progress (2): 614/780 kB | 295/354 kB Progress (2): 614/780 kB | 311/354 kB Progress (2): 630/780 kB | 311/354 kB Progress (2): 630/780 kB | 328/354 kB Progress (2): 647/780 kB | 328/354 kB Progress (2): 647/780 kB | 344/354 kB Progress (2): 663/780 kB | 344/354 kB Progress (2): 663/780 kB | 354 kB Progress (2): 679/780 kB | 354 kB Progress (2): 696/780 kB | 354 kB Progress (2): 712/780 kB | 354 kB Progress (2): 728/780 kB | 354 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.jar (354 kB at 1.9 MB/s) 2026-02-24T22:13:35.794 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.jar 2026-02-24T22:13:35.794 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 745/780 kB Progress (1): 761/780 kB Progress (1): 778/780 kB Progress (1): 780 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar (780 kB at 4.2 MB/s) 2026-02-24T22:13:35.794 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.jar 2026-02-24T22:13:35.801 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/501 kB Progress (1): 16/501 kB Progress (1): 32/501 kB Progress (1): 49/501 kB Progress (1): 65/501 kB Progress (1): 81/501 kB Progress (2): 81/501 kB | 7.7/329 kB Progress (2): 98/501 kB | 7.7/329 kB Progress (2): 98/501 kB | 16/329 kB Progress (2): 114/501 kB | 16/329 kB Progress (2): 114/501 kB | 32/329 kB Progress (2): 131/501 kB | 32/329 kB Progress (2): 131/501 kB | 49/329 kB Progress (2): 147/501 kB | 49/329 kB Progress (2): 147/501 kB | 65/329 kB Progress (2): 163/501 kB | 65/329 kB Progress (2): 163/501 kB | 81/329 kB Progress (2): 180/501 kB | 81/329 kB Progress (2): 180/501 kB | 98/329 kB Progress (2): 196/501 kB | 98/329 kB Progress (2): 196/501 kB | 114/329 kB Progress (2): 213/501 kB | 114/329 kB Progress (2): 213/501 kB | 131/329 kB Progress (2): 229/501 kB | 131/329 kB Progress (2): 229/501 kB | 147/329 kB Progress (2): 245/501 kB | 147/329 kB Progress (2): 245/501 kB | 163/329 kB Progress (2): 262/501 kB | 163/329 kB Progress (2): 262/501 kB | 180/329 kB Progress (2): 278/501 kB | 180/329 kB Progress (2): 278/501 kB | 196/329 kB Progress (2): 294/501 kB | 196/329 kB Progress (2): 294/501 kB | 213/329 kB Progress (2): 311/501 kB | 213/329 kB Progress (2): 311/501 kB | 229/329 kB Progress (2): 327/501 kB | 229/329 kB Progress (2): 327/501 kB | 245/329 kB Progress (2): 344/501 kB | 245/329 kB Progress (2): 344/501 kB | 262/329 kB Progress (2): 360/501 kB | 262/329 kB Progress (2): 360/501 kB | 278/329 kB Progress (2): 376/501 kB | 278/329 kB Progress (2): 376/501 kB | 294/329 kB Progress (2): 393/501 kB | 294/329 kB Progress (2): 393/501 kB | 311/329 kB Progress (2): 409/501 kB | 311/329 kB Progress (2): 409/501 kB | 327/329 kB Progress (2): 426/501 kB | 327/329 kB Progress (2): 426/501 kB | 329 kB Progress (2): 442/501 kB | 329 kB Progress (2): 458/501 kB | 329 kB Progress (2): 475/501 kB | 329 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.jar (329 kB at 1.7 MB/s) 2026-02-24T22:13:35.801 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.jar 2026-02-24T22:13:35.802 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 491/501 kB Progress (1): 501 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.jar (501 kB at 2.6 MB/s) 2026-02-24T22:13:35.802 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.jar 2026-02-24T22:13:35.813 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 3.4/41 kB Progress (1): 20/41 kB Progress (1): 28/41 kB Progress (1): 41 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.jar (41 kB at 202 kB/s) 2026-02-24T22:13:35.813 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.jar 2026-02-24T22:13:35.821 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.jar (6.4 kB at 30 kB/s) 2026-02-24T22:13:35.821 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.jar 2026-02-24T22:13:35.830 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/211 kB Progress (1): 16/211 kB Progress (1): 32/211 kB Progress (2): 32/211 kB | 7.7/14 kB Progress (2): 49/211 kB | 7.7/14 kB Progress (2): 49/211 kB | 14 kB Progress (2): 65/211 kB | 14 kB Progress (2): 81/211 kB | 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.jar (14 kB at 65 kB/s) 2026-02-24T22:13:35.830 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 98/211 kB Downloading from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar 2026-02-24T22:13:35.831 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 114/211 kB Progress (1): 131/211 kB Progress (1): 147/211 kB Progress (1): 163/211 kB Progress (1): 180/211 kB Progress (1): 196/211 kB Progress (1): 211 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.jar (211 kB at 955 kB/s) 2026-02-24T22:13:35.831 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.jar 2026-02-24T22:13:35.838 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 8.2/586 kB Progress (1): 25/586 kB Progress (1): 33/586 kB Progress (1): 49/586 kB Progress (1): 66/586 kB Progress (1): 82/586 kB Progress (1): 98/586 kB Progress (1): 115/586 kB Progress (1): 131/586 kB Progress (1): 147/586 kB Progress (1): 164/586 kB Progress (1): 180/586 kB Progress (1): 197/586 kB Progress (1): 213/586 kB Progress (1): 229/586 kB Progress (1): 246/586 kB Progress (1): 262/586 kB Progress (1): 279/586 kB Progress (1): 295/586 kB Progress (1): 311/586 kB Progress (1): 328/586 kB Progress (1): 344/586 kB Progress (1): 360/586 kB Progress (1): 377/586 kB Progress (1): 393/586 kB Progress (1): 410/586 kB Progress (1): 426/586 kB Progress (1): 442/586 kB Progress (1): 459/586 kB Progress (1): 475/586 kB Progress (1): 492/586 kB Progress (1): 508/586 kB Progress (1): 524/586 kB Progress (1): 541/586 kB Progress (1): 557/586 kB Progress (1): 573/586 kB Progress (1): 586 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.jar (586 kB at 2.6 MB/s) 2026-02-24T22:13:35.839 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.jar 2026-02-24T22:13:35.847 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/106 kB Progress (1): 16/106 kB Progress (1): 32/106 kB Progress (1): 49/106 kB Progress (1): 65/106 kB Progress (1): 81/106 kB Progress (1): 98/106 kB Progress (1): 106 kB Progress (2): 106 kB | 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.jar (106 kB at 448 kB/s) 2026-02-24T22:13:35.847 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.jar 2026-02-24T22:13:35.847 INFO:tasks.workunit.client.0.trial046.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar (6.8 kB at 29 kB/s) 2026-02-24T22:13:35.847 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.jar 2026-02-24T22:13:35.856 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/245 kB Progress (1): 16/245 kB Progress (1): 28/245 kB Progress (1): 44/245 kB Progress (1): 61/245 kB Progress (1): 77/245 kB Progress (1): 94/245 kB Progress (1): 110/245 kB Progress (1): 126/245 kB Progress (1): 131/245 kB Progress (1): 147/245 kB Progress (1): 164/245 kB Progress (1): 180/245 kB Progress (1): 197/245 kB Progress (1): 213/245 kB Progress (1): 229/245 kB Progress (1): 245 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.jar (245 kB at 991 kB/s) 2026-02-24T22:13:35.857 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.jar 2026-02-24T22:13:35.864 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/205 kB Progress (1): 16/205 kB Progress (1): 32/205 kB Progress (1): 49/205 kB Progress (1): 65/205 kB Progress (1): 81/205 kB Progress (1): 82/205 kB Progress (1): 98/205 kB Progress (1): 115/205 kB Progress (1): 131/205 kB Progress (1): 147/205 kB Progress (1): 164/205 kB Progress (1): 180/205 kB Progress (1): 197/205 kB Progress (1): 205 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.jar (205 kB at 803 kB/s) 2026-02-24T22:13:35.865 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.jar 2026-02-24T22:13:35.865 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/15 kB Progress (1): 12/15 kB Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.jar (15 kB at 60 kB/s) 2026-02-24T22:13:35.866 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.jar 2026-02-24T22:13:35.873 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/16 kB Progress (1): 16/16 kB Progress (2): 16/16 kB | 1.4/205 kB Progress (2): 16/16 kB | 2.7/205 kB Progress (2): 16 kB | 2.7/205 kB Progress (2): 16 kB | 4.1/205 kB Progress (2): 16 kB | 8.2/205 kB Progress (2): 16 kB | 12/205 kB Progress (2): 16 kB | 16/205 kB Progress (2): 16 kB | 21/205 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.jar (16 kB at 63 kB/s) 2026-02-24T22:13:35.873 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 25/205 kB Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.jar 2026-02-24T22:13:35.887 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 29/205 kB Progress (1): 33/205 kB Progress (1): 38/205 kB Progress (1): 41/205 kB Progress (1): 45/205 kB Progress (1): 49/205 kB Progress (1): 53/205 kB Progress (1): 58/205 kB Progress (1): 62/205 kB Progress (1): 66/205 kB Progress (2): 66/205 kB | 7.7/24 kB Progress (2): 66/205 kB | 16/24 kB Progress (2): 66/205 kB | 24 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.jar (24 kB at 87 kB/s) 2026-02-24T22:13:35.887 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.jar 2026-02-24T22:13:35.887 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 70/205 kB Progress (1): 74/205 kB Progress (1): 78/205 kB Progress (1): 82/205 kB Progress (2): 82/205 kB | 6.4 kB Progress (2): 86/205 kB | 6.4 kB Progress (2): 90/205 kB | 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.jar (6.4 kB at 23 kB/s) 2026-02-24T22:13:35.889 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 95/205 kB Progress (1): 99/205 kB Progress (1): 103/205 kB Progress (1): 107/205 kB Progress (1): 112/205 kB Progress (1): 115/205 kB Progress (1): 119/205 kB Progress (1): 123/205 kB Progress (1): 127/205 kB Progress (1): 131/205 kB Progress (2): 131/205 kB | 7.7/23 kB Progress (2): 131/205 kB | 8.2/23 kB Progress (2): 131/205 kB | 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.jar (23 kB at 81 kB/s) 2026-02-24T22:13:35.901 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 135/205 kB Progress (1): 140/205 kB Progress (1): 144/205 kB Progress (1): 148/205 kB Progress (1): 152/205 kB Progress (1): 156/205 kB Progress (1): 161/205 kB Progress (1): 165/205 kB Progress (1): 169/205 kB Progress (1): 180/205 kB Progress (1): 197/205 kB Progress (2): 197/205 kB | 7.7/17 kB Progress (2): 197/205 kB | 16/17 kB Progress (2): 197/205 kB | 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.jar (17 kB at 59 kB/s) 2026-02-24T22:13:35.904 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 205 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.jar (205 kB at 690 kB/s) 2026-02-24T22:13:35.906 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/184 kB Progress (1): 16/184 kB Progress (1): 32/184 kB Progress (1): 49/184 kB Progress (1): 65/184 kB Progress (1): 81/184 kB Progress (1): 98/184 kB Progress (1): 114/184 kB Progress (1): 131/184 kB Progress (1): 147/184 kB Progress (1): 163/184 kB Progress (1): 180/184 kB Progress (1): 184 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.jar (184 kB at 621 kB/s) 2026-02-24T22:13:35.923 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] 2026-02-24T22:13:35.923 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] --- clean:3.2.0:clean (default-clean) @ jcksum --- 2026-02-24T22:13:35.928 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.pom 2026-02-24T22:13:35.950 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 790 B Progress (1): 2.3 kB Progress (1): 4.3 kB Progress (1): 5.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.pom (5.8 kB at 265 kB/s) 2026-02-24T22:13:35.952 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/34/maven-shared-components-34.pom 2026-02-24T22:13:35.970 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 781 B Progress (1): 2.3 kB Progress (1): 4.6 kB Progress (1): 5.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/34/maven-shared-components-34.pom (5.1 kB at 268 kB/s) 2026-02-24T22:13:35.972 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/34/maven-parent-34.pom 2026-02-24T22:13:35.991 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 727 B Progress (1): 1.9 kB Progress (1): 5.4 kB Progress (1): 9.8 kB Progress (1): 14 kB Progress (1): 19 kB Progress (1): 23 kB Progress (1): 26 kB Progress (1): 27 kB Progress (1): 30 kB Progress (1): 34 kB Progress (1): 37 kB Progress (1): 41 kB Progress (1): 43 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/34/maven-parent-34.pom (43 kB at 2.1 MB/s) 2026-02-24T22:13:35.994 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.pom 2026-02-24T22:13:36.012 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 782 B Progress (1): 2.1 kB Progress (1): 5.3 kB Progress (1): 7.5 kB Progress (1): 9.5 kB Progress (1): 12 kB Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.pom (14 kB at 792 kB/s) 2026-02-24T22:13:36.016 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.jar 2026-02-24T22:13:36.037 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/153 kB Progress (1): 16/153 kB Progress (1): 32/153 kB Progress (1): 49/153 kB Progress (1): 65/153 kB Progress (1): 81/153 kB Progress (1): 98/153 kB Progress (1): 114/153 kB Progress (1): 131/153 kB Progress (1): 147/153 kB Progress (1): 153 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.jar (153 kB at 7.0 MB/s) 2026-02-24T22:13:36.038 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.jar 2026-02-24T22:13:36.055 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/215 kB Progress (1): 12/215 kB Progress (1): 29/215 kB Progress (1): 45/215 kB Progress (1): 61/215 kB Progress (1): 78/215 kB Progress (1): 94/215 kB Progress (1): 111/215 kB Progress (1): 127/215 kB Progress (1): 143/215 kB Progress (1): 160/215 kB Progress (1): 176/215 kB Progress (1): 193/215 kB Progress (1): 209/215 kB Progress (1): 215 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.jar (215 kB at 12 MB/s) 2026-02-24T22:13:36.085 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] 2026-02-24T22:13:36.085 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] --- resources:3.3.1:resources (default-resources) @ jcksum --- 2026-02-24T22:13:36.088 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.pom 2026-02-24T22:13:36.110 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.4 kB Progress (1): 2.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.pom (2.7 kB at 115 kB/s) 2026-02-24T22:13:36.111 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/5.1/plexus-5.1.pom 2026-02-24T22:13:36.134 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 700 B Progress (1): 2.7 kB Progress (1): 6.5 kB Progress (1): 9.1 kB Progress (1): 12 kB Progress (1): 16 kB Progress (1): 21 kB Progress (1): 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/5.1/plexus-5.1.pom (23 kB at 1.0 MB/s) 2026-02-24T22:13:36.134 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.pom 2026-02-24T22:13:36.158 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 726 B Progress (1): 2.4 kB Progress (1): 4.4 kB Progress (1): 8.6 kB Progress (1): 8.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.pom (8.8 kB at 365 kB/s) 2026-02-24T22:13:36.160 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/10/plexus-10.pom 2026-02-24T22:13:36.178 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 696 B Progress (1): 2.7 kB Progress (1): 6.4 kB Progress (1): 9.2 kB Progress (1): 11 kB Progress (1): 16 kB Progress (1): 20 kB Progress (1): 24 kB Progress (1): 25 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/10/plexus-10.pom (25 kB at 1.4 MB/s) 2026-02-24T22:13:36.180 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.pom 2026-02-24T22:13:36.198 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 793 B Progress (1): 2.3 kB Progress (1): 5.6 kB Progress (1): 6.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.pom (6.0 kB at 318 kB/s) 2026-02-24T22:13:36.200 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/39/maven-shared-components-39.pom 2026-02-24T22:13:36.219 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 812 B Progress (1): 2.2 kB Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/39/maven-shared-components-39.pom (3.2 kB at 161 kB/s) 2026-02-24T22:13:36.221 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.pom 2026-02-24T22:13:36.242 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 612 B Downloaded from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.pom (612 B at 31 kB/s) 2026-02-24T22:13:36.243 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.pom 2026-02-24T22:13:36.258 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.0 kB Progress (1): 2.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.pom (2.7 kB at 171 kB/s) 2026-02-24T22:13:36.259 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.36/slf4j-parent-1.7.36.pom 2026-02-24T22:13:36.274 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.0 kB Progress (1): 3.0 kB Progress (1): 5.9 kB Progress (1): 8.7 kB Progress (1): 12 kB Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.36/slf4j-parent-1.7.36.pom (14 kB at 1.0 MB/s) 2026-02-24T22:13:36.275 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.pom 2026-02-24T22:13:36.298 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 910 B Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.pom (3.2 kB at 139 kB/s) 2026-02-24T22:13:36.299 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/15/spice-parent-15.pom 2026-02-24T22:13:36.321 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.2 kB Progress (1): 3.7 kB Progress (1): 7.3 kB Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/15/spice-parent-15.pom (8.4 kB at 398 kB/s) 2026-02-24T22:13:36.322 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/5/forge-parent-5.pom 2026-02-24T22:13:36.344 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.3 kB Progress (1): 5.5 kB Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/5/forge-parent-5.pom (8.4 kB at 364 kB/s) 2026-02-24T22:13:36.346 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.pom 2026-02-24T22:13:36.364 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 725 B Progress (1): 2.4 kB Progress (1): 4.5 kB Progress (1): 8.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.pom (8.0 kB at 422 kB/s) 2026-02-24T22:13:36.366 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.pom 2026-02-24T22:13:36.386 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 764 B Progress (1): 2.1 kB Progress (1): 5.2 kB Progress (1): 7.8 kB Progress (1): 10 kB Progress (1): 12 kB Progress (1): 15 kB Progress (1): 18 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.pom (20 kB at 986 kB/s) 2026-02-24T22:13:36.388 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.2/junit-bom-5.7.2.pom 2026-02-24T22:13:36.405 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 912 B Progress (1): 4.1 kB Progress (1): 5.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.2/junit-bom-5.7.2.pom (5.1 kB at 319 kB/s) 2026-02-24T22:13:36.406 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.pom 2026-02-24T22:13:36.427 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 740 B Progress (1): 2.1 kB Progress (1): 5.4 kB Progress (1): 8.8 kB Progress (1): 12 kB Progress (1): 15 kB Progress (1): 17 kB Progress (1): 19 kB Progress (1): 21 kB Progress (1): 24 kB Progress (1): 28 kB Progress (1): 30 kB Progress (1): 31 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.pom (31 kB at 1.5 MB/s) 2026-02-24T22:13:36.430 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.1/junit-bom-5.7.1.pom 2026-02-24T22:13:36.446 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 912 B Progress (1): 4.1 kB Progress (1): 5.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.1/junit-bom-5.7.1.pom (5.1 kB at 300 kB/s) 2026-02-24T22:13:36.449 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.jar 2026-02-24T22:13:36.469 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/85 kB Progress (1): 16/85 kB Progress (1): 32/85 kB Progress (1): 49/85 kB Progress (1): 65/85 kB Progress (1): 81/85 kB Progress (1): 85 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.jar (85 kB at 4.5 MB/s) 2026-02-24T22:13:36.469 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.jar 2026-02-24T22:13:36.469 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.jar 2026-02-24T22:13:36.470 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.jar 2026-02-24T22:13:36.470 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.jar 2026-02-24T22:13:36.470 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.jar 2026-02-24T22:13:36.488 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/269 kB Progress (1): 16/269 kB Progress (1): 32/269 kB Progress (1): 48/269 kB Progress (1): 65/269 kB Progress (1): 81/269 kB Progress (1): 98/269 kB Progress (1): 114/269 kB Progress (1): 130/269 kB Progress (1): 147/269 kB Progress (1): 163/269 kB Progress (1): 179/269 kB Progress (1): 196/269 kB Progress (1): 212/269 kB Progress (1): 229/269 kB Progress (1): 245/269 kB Progress (1): 261/269 kB Progress (1): 269 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.jar (269 kB at 14 MB/s) 2026-02-24T22:13:36.488 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.jar 2026-02-24T22:13:36.491 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/55 kB Progress (1): 16/55 kB Progress (1): 32/55 kB Progress (1): 49/55 kB Progress (1): 55 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.jar (55 kB at 2.5 MB/s) 2026-02-24T22:13:36.491 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.jar 2026-02-24T22:13:36.496 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/41 kB Progress (1): 16/41 kB Progress (1): 32/41 kB Progress (1): 41 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.jar (41 kB at 1.6 MB/s) 2026-02-24T22:13:36.496 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/8.5 kB Progress (1): 8.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.jar (8.5 kB at 314 kB/s) 2026-02-24T22:13:36.502 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 2.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.jar (2.5 kB at 78 kB/s) 2026-02-24T22:13:36.507 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/327 kB Progress (1): 7.8/327 kB Progress (1): 24/327 kB Progress (1): 41/327 kB Progress (1): 57/327 kB Progress (1): 73/327 kB Progress (1): 90/327 kB Progress (1): 106/327 kB Progress (1): 122/327 kB Progress (1): 139/327 kB Progress (1): 155/327 kB Progress (1): 172/327 kB Progress (1): 188/327 kB Progress (1): 204/327 kB Progress (1): 221/327 kB Progress (1): 233/327 kB Progress (1): 249/327 kB Progress (1): 265/327 kB Progress (1): 282/327 kB Progress (1): 298/327 kB Progress (1): 314/327 kB Progress (1): 327 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.jar (327 kB at 8.6 MB/s) 2026-02-24T22:13:36.519 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/587 kB Progress (1): 16/587 kB Progress (1): 32/587 kB Progress (1): 49/587 kB Progress (1): 65/587 kB Progress (1): 81/587 kB Progress (1): 98/587 kB Progress (1): 114/587 kB Progress (1): 131/587 kB Progress (1): 147/587 kB Progress (1): 163/587 kB Progress (1): 180/587 kB Progress (1): 196/587 kB Progress (1): 213/587 kB Progress (1): 229/587 kB Progress (1): 245/587 kB Progress (1): 262/587 kB Progress (1): 278/587 kB Progress (1): 294/587 kB Progress (1): 311/587 kB Progress (1): 327/587 kB Progress (1): 344/587 kB Progress (1): 360/587 kB Progress (1): 376/587 kB Progress (1): 393/587 kB Progress (1): 409/587 kB Progress (1): 426/587 kB Progress (1): 442/587 kB Progress (1): 458/587 kB Progress (1): 475/587 kB Progress (1): 491/587 kB Progress (1): 507/587 kB Progress (1): 524/587 kB Progress (1): 540/587 kB Progress (1): 557/587 kB Progress (1): 573/587 kB Progress (1): 587 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.jar (587 kB at 12 MB/s) 2026-02-24T22:13:36.546 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/main/resources 2026-02-24T22:13:36.547 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] 2026-02-24T22:13:36.547 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] --- compiler:3.11.0:compile (default-compile) @ jcksum --- 2026-02-24T22:13:36.550 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.pom 2026-02-24T22:13:36.570 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 806 B Progress (1): 2.3 kB Progress (1): 4.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.pom (4.7 kB at 237 kB/s) 2026-02-24T22:13:36.572 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/19/maven-shared-components-19.pom 2026-02-24T22:13:36.591 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 773 B Progress (1): 2.4 kB Progress (1): 4.2 kB Progress (1): 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/19/maven-shared-components-19.pom (6.4 kB at 335 kB/s) 2026-02-24T22:13:36.592 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/23/maven-parent-23.pom 2026-02-24T22:13:36.614 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 726 B Progress (1): 2.0 kB Progress (1): 5.9 kB Progress (1): 10 kB Progress (1): 15 kB Progress (1): 19 kB Progress (1): 22 kB Progress (1): 25 kB Progress (1): 27 kB Progress (1): 31 kB Progress (1): 33 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/23/maven-parent-23.pom (33 kB at 1.6 MB/s) 2026-02-24T22:13:36.615 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/13/apache-13.pom 2026-02-24T22:13:36.633 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 749 B Progress (1): 2.1 kB Progress (1): 4.1 kB Progress (1): 9.0 kB Progress (1): 11 kB Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/13/apache-13.pom (14 kB at 822 kB/s) 2026-02-24T22:13:36.634 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.pom 2026-02-24T22:13:36.653 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 815 B Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.pom (815 B at 43 kB/s) 2026-02-24T22:13:36.654 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/1.5.5/plexus-containers-1.5.5.pom 2026-02-24T22:13:36.679 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.3 kB Progress (1): 4.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/1.5.5/plexus-containers-1.5.5.pom (4.2 kB at 177 kB/s) 2026-02-24T22:13:36.679 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/2.0.7/plexus-2.0.7.pom 2026-02-24T22:13:36.696 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 711 B Progress (1): 2.8 kB Progress (1): 6.4 kB Progress (1): 9.1 kB Progress (1): 13 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/2.0.7/plexus-2.0.7.pom (17 kB at 1.0 MB/s) 2026-02-24T22:13:36.698 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.pom 2026-02-24T22:13:36.717 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.3 kB Progress (1): 4.2 kB Progress (1): 5.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.pom (5.0 kB at 262 kB/s) 2026-02-24T22:13:36.719 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.1.2/plexus-languages-1.1.2.pom 2026-02-24T22:13:36.738 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.3 kB Progress (1): 4.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.1.2/plexus-languages-1.1.2.pom (4.1 kB at 207 kB/s) 2026-02-24T22:13:36.740 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.pom 2026-02-24T22:13:36.761 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.3 kB Progress (1): 2.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.pom (2.4 kB at 108 kB/s) 2026-02-24T22:13:36.762 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/ow2/1.5.1/ow2-1.5.1.pom 2026-02-24T22:13:36.783 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 709 B Progress (1): 2.1 kB Progress (1): 3.6 kB Progress (1): 5.7 kB Progress (1): 8.1 kB Progress (1): 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/ow2/1.5.1/ow2-1.5.1.pom (11 kB at 538 kB/s) 2026-02-24T22:13:36.785 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.pom 2026-02-24T22:13:36.805 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 872 B Progress (1): 3.4 kB Progress (1): 6.6 kB Progress (1): 11 kB Progress (1): 13 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.pom (17 kB at 821 kB/s) 2026-02-24T22:13:36.806 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/9/oss-parent-9.pom 2026-02-24T22:13:36.825 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 838 B Progress (1): 2.7 kB Progress (1): 6.5 kB Progress (1): 6.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/9/oss-parent-9.pom (6.6 kB at 346 kB/s) 2026-02-24T22:13:36.827 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.pom 2026-02-24T22:13:36.845 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.pom (1.1 kB at 56 kB/s) 2026-02-24T22:13:36.846 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler/2.13.0/plexus-compiler-2.13.0.pom 2026-02-24T22:13:36.864 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.2 kB Progress (1): 4.1 kB Progress (1): 7.3 kB Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler/2.13.0/plexus-compiler-2.13.0.pom (8.4 kB at 466 kB/s) 2026-02-24T22:13:36.865 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/10.0/plexus-components-10.0.pom 2026-02-24T22:13:36.884 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.3 kB Progress (1): 2.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/10.0/plexus-components-10.0.pom (2.7 kB at 141 kB/s) 2026-02-24T22:13:36.886 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.1/junit-bom-5.9.1.pom 2026-02-24T22:13:36.906 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 907 B Progress (1): 4.1 kB Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.1/junit-bom-5.9.1.pom (5.6 kB at 268 kB/s) 2026-02-24T22:13:36.908 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.pom 2026-02-24T22:13:36.925 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.pom (1.1 kB at 67 kB/s) 2026-02-24T22:13:36.927 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.1/plexus-component-annotations-2.1.1.pom 2026-02-24T22:13:36.943 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 770 B Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.1/plexus-component-annotations-2.1.1.pom (770 B at 48 kB/s) 2026-02-24T22:13:36.944 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.1/plexus-containers-2.1.1.pom 2026-02-24T22:13:36.962 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.3 kB Progress (1): 4.9 kB Progress (1): 6.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.1/plexus-containers-2.1.1.pom (6.0 kB at 355 kB/s) 2026-02-24T22:13:36.963 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/6.5/plexus-6.5.pom 2026-02-24T22:13:36.983 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 696 B Progress (1): 2.7 kB Progress (1): 6.4 kB Progress (1): 9.2 kB Progress (1): 11 kB Progress (1): 16 kB Progress (1): 20 kB Progress (1): 24 kB Progress (1): 26 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/6.5/plexus-6.5.pom (26 kB at 1.3 MB/s) 2026-02-24T22:13:36.984 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.pom 2026-02-24T22:13:37.007 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.pom (1.2 kB at 52 kB/s) 2026-02-24T22:13:37.008 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compilers/2.13.0/plexus-compilers-2.13.0.pom 2026-02-24T22:13:37.028 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compilers/2.13.0/plexus-compilers-2.13.0.pom (1.3 kB at 70 kB/s) 2026-02-24T22:13:37.032 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.jar 2026-02-24T22:13:37.048 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/14 kB Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.jar (14 kB at 846 kB/s) 2026-02-24T22:13:37.048 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.jar 2026-02-24T22:13:37.048 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.jar 2026-02-24T22:13:37.048 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.jar 2026-02-24T22:13:37.048 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.jar 2026-02-24T22:13:37.048 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.jar 2026-02-24T22:13:37.066 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 4.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.jar (4.2 kB at 234 kB/s) 2026-02-24T22:13:37.066 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.jar 2026-02-24T22:13:37.070 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/55 kB Progress (1): 16/55 kB Progress (1): 32/55 kB Progress (1): 49/55 kB Progress (1): 55 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.jar (55 kB at 2.4 MB/s) 2026-02-24T22:13:37.070 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.jar 2026-02-24T22:13:37.073 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/334 kB Progress (1): 16/334 kB Progress (1): 32/334 kB Progress (1): 48/334 kB Progress (1): 65/334 kB Progress (1): 81/334 kB Progress (1): 98/334 kB Progress (1): 114/334 kB Progress (1): 130/334 kB Progress (1): 147/334 kB Progress (1): 163/334 kB Progress (1): 180/334 kB Progress (1): 196/334 kB Progress (1): 212/334 kB Progress (1): 229/334 kB Progress (1): 245/334 kB Progress (1): 261/334 kB Progress (1): 278/334 kB Progress (1): 294/334 kB Progress (1): 311/334 kB Progress (1): 327/334 kB Progress (1): 334 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.jar (334 kB at 13 MB/s) 2026-02-24T22:13:37.073 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.jar 2026-02-24T22:13:37.077 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/122 kB Progress (1): 16/122 kB Progress (1): 32/122 kB Progress (1): 48/122 kB Progress (1): 65/122 kB Progress (1): 81/122 kB Progress (1): 98/122 kB Progress (1): 114/122 kB Progress (1): 122 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.jar (122 kB at 4.1 MB/s) 2026-02-24T22:13:37.082 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/27 kB Progress (1): 16/27 kB Progress (1): 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.jar (27 kB at 772 kB/s) 2026-02-24T22:13:37.089 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/267 kB Progress (1): 12/267 kB Progress (1): 28/267 kB Progress (1): 44/267 kB Progress (1): 61/267 kB Progress (1): 77/267 kB Progress (1): 94/267 kB Progress (1): 110/267 kB Progress (1): 126/267 kB Progress (1): 143/267 kB Progress (1): 159/267 kB Progress (1): 175/267 kB Progress (1): 192/267 kB Progress (1): 197/267 kB Progress (1): 213/267 kB Progress (1): 229/267 kB Progress (1): 246/267 kB Progress (1): 262/267 kB Progress (1): 267 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.jar (267 kB at 6.5 MB/s) 2026-02-24T22:13:37.091 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 4.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.jar (4.7 kB at 106 kB/s) 2026-02-24T22:13:37.101 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/23 kB Progress (1): 16/23 kB Progress (1): 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.jar (23 kB at 422 kB/s) 2026-02-24T22:13:37.142 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] Changes detected - recompiling the module! :source 2026-02-24T22:13:37.143 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] Compiling 1 source file with javac [debug target 1.9] to target/classes 2026-02-24T22:13:37.488 INFO:tasks.workunit.client.0.trial046.stdout:[WARNING] bootstrap class path not set in conjunction with -source 9 2026-02-24T22:13:37.488 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] 2026-02-24T22:13:37.488 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] --- resources:3.3.1:testResources (default-testResources) @ jcksum --- 2026-02-24T22:13:37.490 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/test/resources 2026-02-24T22:13:37.490 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] 2026-02-24T22:13:37.490 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] --- compiler:3.11.0:testCompile (default-testCompile) @ jcksum --- 2026-02-24T22:13:37.492 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] Changes detected - recompiling the module! :dependency 2026-02-24T22:13:37.493 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] Compiling 1 source file with javac [debug target 1.9] to target/test-classes 2026-02-24T22:13:37.564 INFO:tasks.workunit.client.0.trial046.stdout:[WARNING] bootstrap class path not set in conjunction with -source 9 2026-02-24T22:13:37.564 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] 2026-02-24T22:13:37.565 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] --- surefire:3.2.3:test (default-test) @ jcksum --- 2026-02-24T22:13:37.567 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.pom 2026-02-24T22:13:37.592 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 834 B Progress (1): 3.3 kB Progress (1): 6.2 kB Progress (1): 6.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.pom (6.2 kB at 260 kB/s) 2026-02-24T22:13:37.593 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.pom 2026-02-24T22:13:37.622 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 843 B Progress (1): 2.7 kB Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.pom (3.5 kB at 122 kB/s) 2026-02-24T22:13:37.624 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.pom 2026-02-24T22:13:37.648 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 834 B Progress (1): 2.3 kB Progress (1): 3.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.pom (3.3 kB at 136 kB/s) 2026-02-24T22:13:37.650 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.pom 2026-02-24T22:13:37.673 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.1 kB Progress (1): 4.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.pom (4.1 kB at 184 kB/s) 2026-02-24T22:13:37.674 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.pom 2026-02-24T22:13:37.697 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 843 B Progress (1): 2.8 kB Progress (1): 3.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.pom (3.3 kB at 144 kB/s) 2026-02-24T22:13:37.699 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.pom 2026-02-24T22:13:37.728 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 834 B Progress (1): 3.0 kB Progress (1): 4.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.pom (4.5 kB at 159 kB/s) 2026-02-24T22:13:37.730 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.pom 2026-02-24T22:13:37.759 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 878 B Progress (1): 1.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.pom (1.8 kB at 63 kB/s) 2026-02-24T22:13:37.761 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.pom 2026-02-24T22:13:37.792 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 937 B Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.pom (2.2 kB at 73 kB/s) 2026-02-24T22:13:37.793 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether/1.0.0.v20140518/aether-1.0.0.v20140518.pom 2026-02-24T22:13:37.820 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 798 B Progress (1): 3.2 kB Progress (1): 5.1 kB Progress (1): 9.1 kB Progress (1): 12 kB Progress (1): 17 kB Progress (1): 20 kB Progress (1): 23 kB Progress (1): 28 kB Progress (1): 30 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether/1.0.0.v20140518/aether-1.0.0.v20140518.pom (30 kB at 1.2 MB/s) 2026-02-24T22:13:37.821 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.pom 2026-02-24T22:13:37.845 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 938 B Progress (1): 1.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.pom (1.9 kB at 82 kB/s) 2026-02-24T22:13:37.846 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.pom 2026-02-24T22:13:37.872 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 798 B Progress (1): 2.3 kB Progress (1): 5.7 kB Progress (1): 5.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.pom (5.8 kB at 223 kB/s) 2026-02-24T22:13:37.874 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-artifact/3.2.5/maven-artifact-3.2.5.pom 2026-02-24T22:13:37.898 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 843 B Progress (1): 2.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-artifact/3.2.5/maven-artifact-3.2.5.pom (2.3 kB at 98 kB/s) 2026-02-24T22:13:37.899 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven/3.2.5/maven-3.2.5.pom 2026-02-24T22:13:37.923 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 759 B Progress (1): 1.9 kB Progress (1): 3.4 kB Progress (1): 5.3 kB Progress (1): 11 kB Progress (1): 15 kB Progress (1): 18 kB Progress (1): 21 kB Progress (1): 22 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven/3.2.5/maven-3.2.5.pom (22 kB at 932 kB/s) 2026-02-24T22:13:37.924 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/25/maven-parent-25.pom 2026-02-24T22:13:37.950 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 730 B Progress (1): 2.0 kB Progress (1): 6.0 kB Progress (1): 11 kB Progress (1): 15 kB Progress (1): 19 kB Progress (1): 23 kB Progress (1): 24 kB Progress (1): 27 kB Progress (1): 30 kB Progress (1): 32 kB Progress (1): 37 kB Progress (1): 37 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/25/maven-parent-25.pom (37 kB at 1.5 MB/s) 2026-02-24T22:13:37.951 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/15/apache-15.pom 2026-02-24T22:13:37.975 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 754 B Progress (1): 2.1 kB Progress (1): 3.9 kB Progress (1): 7.9 kB Progress (1): 11 kB Progress (1): 14 kB Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/15/apache-15.pom (15 kB at 635 kB/s) 2026-02-24T22:13:37.977 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/4.0.0/plexus-utils-4.0.0.pom 2026-02-24T22:13:38.001 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 724 B Progress (1): 2.3 kB Progress (1): 4.5 kB Progress (1): 8.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/4.0.0/plexus-utils-4.0.0.pom (8.7 kB at 346 kB/s) 2026-02-24T22:13:38.002 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/13/plexus-13.pom 2026-02-24T22:13:38.035 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 798 B Progress (1): 2.8 kB Progress (1): 6.8 kB Progress (1): 7.5 kB Progress (1): 9.1 kB Progress (1): 11 kB Progress (1): 15 kB Progress (1): 19 kB Progress (1): 22 kB Progress (1): 26 kB Progress (1): 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/13/plexus-13.pom (27 kB at 830 kB/s) 2026-02-24T22:13:38.037 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-core/3.2.5/maven-core-3.2.5.pom 2026-02-24T22:13:38.061 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 794 B Progress (1): 2.2 kB Progress (1): 5.3 kB Progress (1): 8.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-core/3.2.5/maven-core-3.2.5.pom (8.1 kB at 351 kB/s) 2026-02-24T22:13:38.062 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings/3.2.5/maven-settings-3.2.5.pom 2026-02-24T22:13:38.087 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 823 B Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings/3.2.5/maven-settings-3.2.5.pom (2.2 kB at 87 kB/s) 2026-02-24T22:13:38.089 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings-builder/3.2.5/maven-settings-builder-3.2.5.pom 2026-02-24T22:13:38.119 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 823 B Progress (1): 2.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings-builder/3.2.5/maven-settings-builder-3.2.5.pom (2.6 kB at 84 kB/s) 2026-02-24T22:13:38.121 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.21/plexus-interpolation-1.21.pom 2026-02-24T22:13:38.142 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.3 kB Progress (1): 1.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.21/plexus-interpolation-1.21.pom (1.5 kB at 70 kB/s) 2026-02-24T22:13:38.143 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/1.3.1/plexus-components-1.3.1.pom 2026-02-24T22:13:38.167 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.3 kB Progress (1): 3.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/1.3.1/plexus-components-1.3.1.pom (3.1 kB at 133 kB/s) 2026-02-24T22:13:38.167 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/3.3.1/plexus-3.3.1.pom 2026-02-24T22:13:38.191 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 707 B Progress (1): 2.6 kB Progress (1): 6.1 kB Progress (1): 8.6 kB Progress (1): 11 kB Progress (1): 16 kB Progress (1): 20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/3.3.1/plexus-3.3.1.pom (20 kB at 852 kB/s) 2026-02-24T22:13:38.192 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/17/spice-parent-17.pom 2026-02-24T22:13:38.218 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.2 kB Progress (1): 3.4 kB Progress (1): 6.7 kB Progress (1): 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/17/spice-parent-17.pom (6.8 kB at 260 kB/s) 2026-02-24T22:13:38.219 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/10/forge-parent-10.pom 2026-02-24T22:13:38.243 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 735 B Progress (1): 2.6 kB Progress (1): 8.6 kB Progress (1): 12 kB Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/10/forge-parent-10.pom (14 kB at 542 kB/s) 2026-02-24T22:13:38.245 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.pom 2026-02-24T22:13:38.269 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.3 kB Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.pom (3.0 kB at 123 kB/s) 2026-02-24T22:13:38.270 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/12/spice-parent-12.pom 2026-02-24T22:13:38.294 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.2 kB Progress (1): 4.2 kB Progress (1): 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/12/spice-parent-12.pom (6.8 kB at 283 kB/s) 2026-02-24T22:13:38.295 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/4/forge-parent-4.pom 2026-02-24T22:13:38.316 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.4 kB Progress (1): 5.8 kB Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/4/forge-parent-4.pom (8.4 kB at 400 kB/s) 2026-02-24T22:13:38.317 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-cipher/1.4/plexus-cipher-1.4.pom 2026-02-24T22:13:38.338 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.4 kB Progress (1): 2.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-cipher/1.4/plexus-cipher-1.4.pom (2.1 kB at 98 kB/s) 2026-02-24T22:13:38.339 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-repository-metadata/3.2.5/maven-repository-metadata-3.2.5.pom 2026-02-24T22:13:38.364 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 823 B Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-repository-metadata/3.2.5/maven-repository-metadata-3.2.5.pom (2.2 kB at 89 kB/s) 2026-02-24T22:13:38.366 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-plugin-api/3.2.5/maven-plugin-api-3.2.5.pom 2026-02-24T22:13:38.389 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 816 B Progress (1): 2.5 kB Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-plugin-api/3.2.5/maven-plugin-api-3.2.5.pom (3.0 kB at 132 kB/s) 2026-02-24T22:13:38.391 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.plexus/0.9.0.M2/org.eclipse.sisu.plexus-0.9.0.M2.pom 2026-02-24T22:13:38.417 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 827 B Progress (1): 3.5 kB Progress (1): 8.7 kB Progress (1): 15 kB Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.plexus/0.9.0.M2/org.eclipse.sisu.plexus-0.9.0.M2.pom (15 kB at 576 kB/s) 2026-02-24T22:13:38.418 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-plexus/0.9.0.M2/sisu-plexus-0.9.0.M2.pom 2026-02-24T22:13:38.443 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 811 B Progress (1): 2.8 kB Progress (1): 4.9 kB Progress (1): 8.5 kB Progress (1): 12 kB Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-plexus/0.9.0.M2/sisu-plexus-0.9.0.M2.pom (15 kB at 604 kB/s) 2026-02-24T22:13:38.444 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/annotation/javax.annotation-api/1.2/javax.annotation-api-1.2.pom 2026-02-24T22:13:38.468 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 702 B Progress (1): 2.0 kB Progress (1): 3.5 kB Progress (1): 5.5 kB Progress (1): 8.7 kB Progress (1): 12 kB Progress (1): 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/javax/annotation/javax.annotation-api/1.2/javax.annotation-api-1.2.pom (13 kB at 584 kB/s) 2026-02-24T22:13:38.469 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/net/java/jvnet-parent/3/jvnet-parent-3.pom 2026-02-24T22:13:38.496 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 838 B Progress (1): 2.5 kB Progress (1): 4.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/net/java/jvnet-parent/3/jvnet-parent-3.pom (4.8 kB at 171 kB/s) 2026-02-24T22:13:38.497 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/enterprise/cdi-api/1.2/cdi-api-1.2.pom 2026-02-24T22:13:38.520 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.0 kB Progress (1): 3.3 kB Progress (1): 5.8 kB Progress (1): 6.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/javax/enterprise/cdi-api/1.2/cdi-api-1.2.pom (6.3 kB at 285 kB/s) 2026-02-24T22:13:38.521 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/jboss/weld/weld-parent/26/weld-parent-26.pom 2026-02-24T22:13:38.546 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.0 kB Progress (1): 2.9 kB Progress (1): 6.3 kB Progress (1): 11 kB Progress (1): 16 kB Progress (1): 19 kB Progress (1): 22 kB Progress (1): 26 kB Progress (1): 31 kB Progress (1): 32 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/jboss/weld/weld-parent/26/weld-parent-26.pom (32 kB at 1.3 MB/s) 2026-02-24T22:13:38.548 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.inject/0.9.0.M2/org.eclipse.sisu.inject-0.9.0.M2.pom 2026-02-24T22:13:38.573 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 804 B Progress (1): 4.1 kB Progress (1): 10 kB Progress (1): 16 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.inject/0.9.0.M2/org.eclipse.sisu.inject-0.9.0.M2.pom (17 kB at 694 kB/s) 2026-02-24T22:13:38.574 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-inject/0.9.0.M2/sisu-inject-0.9.0.M2.pom 2026-02-24T22:13:38.598 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 819 B Progress (1): 2.8 kB Progress (1): 4.9 kB Progress (1): 8.5 kB Progress (1): 12 kB Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-inject/0.9.0.M2/sisu-inject-0.9.0.M2.pom (15 kB at 617 kB/s) 2026-02-24T22:13:38.600 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.0/plexus-component-annotations-2.1.0.pom 2026-02-24T22:13:38.622 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 750 B Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.0/plexus-component-annotations-2.1.0.pom (750 B at 34 kB/s) 2026-02-24T22:13:38.623 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.0/plexus-containers-2.1.0.pom 2026-02-24T22:13:38.643 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.3 kB Progress (1): 4.7 kB Progress (1): 4.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.0/plexus-containers-2.1.0.pom (4.8 kB at 229 kB/s) 2026-02-24T22:13:38.644 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.6.0/plexus-classworlds-2.6.0.pom 2026-02-24T22:13:38.666 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 932 B Progress (1): 2.6 kB Progress (1): 5.5 kB Progress (1): 7.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.6.0/plexus-classworlds-2.6.0.pom (7.9 kB at 377 kB/s) 2026-02-24T22:13:38.667 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-model-builder/3.2.5/maven-model-builder-3.2.5.pom 2026-02-24T22:13:38.689 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 836 B Progress (1): 2.7 kB Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-model-builder/3.2.5/maven-model-builder-3.2.5.pom (3.0 kB at 136 kB/s) 2026-02-24T22:13:38.691 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-aether-provider/3.2.5/maven-aether-provider-3.2.5.pom 2026-02-24T22:13:38.720 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 811 B Progress (1): 3.0 kB Progress (1): 4.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-aether-provider/3.2.5/maven-aether-provider-3.2.5.pom (4.2 kB at 152 kB/s) 2026-02-24T22:13:38.721 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-spi/1.0.0.v20140518/aether-spi-1.0.0.v20140518.pom 2026-02-24T22:13:38.749 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 934 B Progress (1): 2.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-spi/1.0.0.v20140518/aether-spi-1.0.0.v20140518.pom (2.1 kB at 73 kB/s) 2026-02-24T22:13:38.750 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-impl/1.0.0.v20140518/aether-impl-1.0.0.v20140518.pom 2026-02-24T22:13:38.776 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 922 B Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-impl/1.0.0.v20140518/aether-impl-1.0.0.v20140518.pom (3.5 kB at 134 kB/s) 2026-02-24T22:13:38.778 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/sisu-guice/3.2.3/sisu-guice-3.2.3.pom 2026-02-24T22:13:38.802 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.3 kB Progress (1): 3.8 kB Progress (1): 6.0 kB Progress (1): 9.7 kB Progress (1): 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/sisu-guice/3.2.3/sisu-guice-3.2.3.pom (11 kB at 475 kB/s) 2026-02-24T22:13:38.803 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/inject/guice-parent/3.2.3/guice-parent-3.2.3.pom 2026-02-24T22:13:38.828 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 799 B Progress (1): 2.7 kB Progress (1): 5.1 kB Progress (1): 7.6 kB Progress (1): 10 kB Progress (1): 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/inject/guice-parent/3.2.3/guice-parent-3.2.3.pom (13 kB at 518 kB/s) 2026-02-24T22:13:38.829 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/38/forge-parent-38.pom 2026-02-24T22:13:38.851 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 812 B Progress (1): 2.5 kB Progress (1): 5.0 kB Progress (1): 8.7 kB Progress (1): 12 kB Progress (1): 16 kB Progress (1): 19 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/38/forge-parent-38.pom (19 kB at 892 kB/s) 2026-02-24T22:13:38.852 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/aopalliance/aopalliance/1.0/aopalliance-1.0.pom 2026-02-24T22:13:38.878 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 363 B Downloaded from central: https://repo.maven.apache.org/maven2/aopalliance/aopalliance/1.0/aopalliance-1.0.pom (363 B at 14 kB/s) 2026-02-24T22:13:38.879 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/google/guava/guava/16.0.1/guava-16.0.1.pom 2026-02-24T22:13:38.907 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 956 B Progress (1): 2.8 kB Progress (1): 5.3 kB Progress (1): 6.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/google/guava/guava/16.0.1/guava-16.0.1.pom (6.1 kB at 218 kB/s) 2026-02-24T22:13:38.908 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/google/guava/guava-parent/16.0.1/guava-parent-16.0.1.pom 2026-02-24T22:13:38.933 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.1 kB Progress (1): 2.9 kB Progress (1): 6.3 kB Progress (1): 7.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/google/guava/guava-parent/16.0.1/guava-parent-16.0.1.pom (7.3 kB at 293 kB/s) 2026-02-24T22:13:38.934 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/7/oss-parent-7.pom 2026-02-24T22:13:38.959 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 848 B Progress (1): 2.5 kB Progress (1): 4.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/7/oss-parent-7.pom (4.8 kB at 193 kB/s) 2026-02-24T22:13:38.960 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.5.2/plexus-classworlds-2.5.2.pom 2026-02-24T22:13:38.987 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 943 B Progress (1): 3.1 kB Progress (1): 5.6 kB Progress (1): 7.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.5.2/plexus-classworlds-2.5.2.pom (7.3 kB at 281 kB/s) 2026-02-24T22:13:38.988 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.pom 2026-02-24T22:13:39.017 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 765 B Progress (1): 2.1 kB Progress (1): 5.2 kB Progress (1): 7.5 kB Progress (1): 10 kB Progress (1): 12 kB Progress (1): 15 kB Progress (1): 18 kB Progress (1): 21 kB Progress (1): 22 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.pom (22 kB at 746 kB/s) 2026-02-24T22:13:39.018 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/64/commons-parent-64.pom 2026-02-24T22:13:39.050 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 706 B Progress (1): 1.9 kB Progress (1): 3.1 kB Progress (1): 4.7 kB Progress (1): 6.4 kB Progress (1): 8.4 kB Progress (1): 10 kB Progress (1): 13 kB Progress (1): 15 kB Progress (1): 18 kB Progress (1): 19 kB Progress (1): 23 kB Progress (1): 25 kB Progress (1): 28 kB Progress (1): 32 kB Progress (1): 32 kB Progress (1): 36 kB Progress (1): 41 kB Progress (1): 44 kB Progress (1): 46 kB Progress (1): 49 kB Progress (1): 51 kB Progress (1): 56 kB Progress (1): 58 kB Progress (1): 60 kB Progress (1): 64 kB Progress (1): 68 kB Progress (1): 71 kB Progress (1): 74 kB Progress (1): 76 kB Progress (1): 78 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/64/commons-parent-64.pom (78 kB at 2.6 MB/s) 2026-02-24T22:13:39.051 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/30/apache-30.pom 2026-02-24T22:13:39.078 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 743 B Progress (1): 2.1 kB Progress (1): 4.0 kB Progress (1): 6.3 kB Progress (1): 11 kB Progress (1): 16 kB Progress (1): 20 kB Progress (1): 22 kB Progress (1): 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/30/apache-30.pom (23 kB at 861 kB/s) 2026-02-24T22:13:39.080 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.0/junit-bom-5.10.0.pom 2026-02-24T22:13:39.104 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 913 B Progress (1): 4.1 kB Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.0/junit-bom-5.10.0.pom (5.6 kB at 226 kB/s) 2026-02-24T22:13:39.105 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.pom 2026-02-24T22:13:39.132 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.9 kB Progress (1): 4.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.pom (4.3 kB at 158 kB/s) 2026-02-24T22:13:39.133 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.2.0/plexus-languages-1.2.0.pom 2026-02-24T22:13:39.156 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.3 kB Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.2.0/plexus-languages-1.2.0.pom (3.2 kB at 139 kB/s) 2026-02-24T22:13:39.157 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/15/plexus-15.pom 2026-02-24T22:13:39.184 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 692 B Progress (1): 2.7 kB Progress (1): 6.4 kB Progress (1): 8.9 kB Progress (1): 11 kB Progress (1): 14 kB Progress (1): 18 kB Progress (1): 21 kB Progress (1): 25 kB Progress (1): 28 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/15/plexus-15.pom (28 kB at 1.1 MB/s) 2026-02-24T22:13:39.185 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.pom 2026-02-24T22:13:39.213 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.3 kB Progress (1): 2.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.pom (2.4 kB at 88 kB/s) 2026-02-24T22:13:39.216 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.jar 2026-02-24T22:13:39.241 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/308 kB Progress (1): 16/308 kB Progress (1): 32/308 kB Progress (1): 49/308 kB Progress (1): 65/308 kB Progress (1): 81/308 kB Progress (1): 98/308 kB Progress (1): 114/308 kB Progress (1): 131/308 kB Progress (1): 147/308 kB Progress (1): 163/308 kB Progress (1): 180/308 kB Progress (1): 196/308 kB Progress (1): 213/308 kB Progress (1): 229/308 kB Progress (1): 245/308 kB Progress (1): 262/308 kB Progress (1): 278/308 kB Progress (1): 294/308 kB Progress (1): 308 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.jar (308 kB at 13 MB/s) 2026-02-24T22:13:39.242 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.jar 2026-02-24T22:13:39.242 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.jar 2026-02-24T22:13:39.242 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.jar 2026-02-24T22:13:39.242 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.jar 2026-02-24T22:13:39.242 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.jar 2026-02-24T22:13:39.265 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/14 kB Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.jar (14 kB at 591 kB/s) 2026-02-24T22:13:39.265 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.jar 2026-02-24T22:13:39.270 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/171 kB Progress (1): 16/171 kB Progress (1): 29/171 kB Progress (1): 45/171 kB Progress (1): 61/171 kB Progress (1): 66/171 kB Progress (1): 82/171 kB Progress (1): 98/171 kB Progress (1): 115/171 kB Progress (1): 131/171 kB Progress (1): 147/171 kB Progress (1): 164/171 kB Progress (1): 171 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.jar (171 kB at 6.1 MB/s) 2026-02-24T22:13:39.270 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.jar 2026-02-24T22:13:39.271 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/118 kB Progress (1): 16/118 kB Progress (1): 32/118 kB Progress (1): 49/118 kB Progress (1): 65/118 kB Progress (1): 81/118 kB Progress (1): 98/118 kB Progress (1): 114/118 kB Progress (1): 118 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.jar (118 kB at 4.1 MB/s) 2026-02-24T22:13:39.271 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.jar 2026-02-24T22:13:39.273 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/26 kB Progress (1): 16/26 kB Progress (1): 24/26 kB Progress (1): 26 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.jar (26 kB at 834 kB/s) 2026-02-24T22:13:39.273 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.jar 2026-02-24T22:13:39.277 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/8.2 kB Progress (1): 8.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.jar (8.2 kB at 234 kB/s) 2026-02-24T22:13:39.277 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.jar 2026-02-24T22:13:39.287 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/146 kB Progress (1): 16/146 kB Progress (1): 32/146 kB Progress (1): 48/146 kB Progress (1): 65/146 kB Progress (1): 81/146 kB Progress (1): 98/146 kB Progress (1): 114/146 kB Progress (1): 130/146 kB Progress (1): 146 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.jar (146 kB at 3.2 MB/s) 2026-02-24T22:13:39.287 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.jar 2026-02-24T22:13:39.290 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/61 kB Progress (1): 16/61 kB Progress (1): 32/61 kB Progress (1): 49/61 kB Progress (1): 61 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.jar (61 kB at 1.3 MB/s) 2026-02-24T22:13:39.290 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.jar 2026-02-24T22:13:39.293 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/136 kB Progress (1): 12/136 kB Progress (1): 28/136 kB Progress (1): 44/136 kB Progress (1): 61/136 kB Progress (1): 77/136 kB Progress (1): 94/136 kB Progress (1): 110/136 kB Progress (1): 126/136 kB Progress (1): 136 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.jar (136 kB at 2.6 MB/s) 2026-02-24T22:13:39.298 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/501 kB Progress (1): 16/501 kB Progress (1): 32/501 kB Progress (1): 49/501 kB Progress (1): 57/501 kB Progress (1): 73/501 kB Progress (1): 90/501 kB Progress (1): 106/501 kB Progress (1): 122/501 kB Progress (1): 139/501 kB Progress (1): 155/501 kB Progress (1): 172/501 kB Progress (1): 188/501 kB Progress (1): 204/501 kB Progress (1): 221/501 kB Progress (1): 229/501 kB Progress (1): 246/501 kB Progress (1): 262/501 kB Progress (1): 279/501 kB Progress (1): 295/501 kB Progress (1): 311/501 kB Progress (1): 328/501 kB Progress (1): 344/501 kB Progress (1): 360/501 kB Progress (1): 377/501 kB Progress (1): 393/501 kB Progress (1): 410/501 kB Progress (1): 426/501 kB Progress (1): 442/501 kB Progress (1): 459/501 kB Progress (1): 475/501 kB Progress (1): 492/501 kB Progress (1): 501 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.jar (501 kB at 8.9 MB/s) 2026-02-24T22:13:39.305 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/58 kB Progress (1): 16/58 kB Progress (1): 32/58 kB Progress (1): 48/58 kB Progress (1): 58 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.jar (58 kB at 899 kB/s) 2026-02-24T22:13:39.309 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/124 kB Progress (1): 16/124 kB Progress (1): 32/124 kB Progress (1): 49/124 kB Progress (1): 65/124 kB Progress (1): 81/124 kB Progress (1): 98/124 kB Progress (1): 114/124 kB Progress (1): 124 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.jar (124 kB at 1.8 MB/s) 2026-02-24T22:13:39.325 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 0/2.4 MB Progress (1): 0/2.4 MB Progress (1): 0/2.4 MB Progress (1): 0/2.4 MB Progress (1): 0.1/2.4 MB Progress (1): 0.1/2.4 MB Progress (1): 0.1/2.4 MB Progress (1): 0.1/2.4 MB Progress (1): 0.1/2.4 MB Progress (1): 0.1/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4 MB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.jar (2.4 MB at 29 MB/s) 2026-02-24T22:13:39.381 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] 2026-02-24T22:13:39.381 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] --- jar:3.3.0:jar (default-jar) @ jcksum --- 2026-02-24T22:13:39.384 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.pom 2026-02-24T22:13:39.403 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 800 B Progress (1): 2.2 kB Progress (1): 4.3 kB Progress (1): 4.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.pom (4.5 kB at 236 kB/s) 2026-02-24T22:13:39.404 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/36/maven-shared-components-36.pom 2026-02-24T22:13:39.428 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 781 B Progress (1): 2.3 kB Progress (1): 4.5 kB Progress (1): 4.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/36/maven-shared-components-36.pom (4.9 kB at 204 kB/s) 2026-02-24T22:13:39.429 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/36/maven-parent-36.pom 2026-02-24T22:13:39.452 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 725 B Progress (1): 1.9 kB Progress (1): 5.4 kB Progress (1): 9.7 kB Progress (1): 14 kB Progress (1): 19 kB Progress (1): 23 kB Progress (1): 26 kB Progress (1): 28 kB Progress (1): 30 kB Progress (1): 34 kB Progress (1): 37 kB Progress (1): 40 kB Progress (1): 45 kB Progress (1): 45 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/36/maven-parent-36.pom (45 kB at 2.0 MB/s) 2026-02-24T22:13:39.454 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/26/apache-26.pom 2026-02-24T22:13:39.475 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 744 B Progress (1): 2.1 kB Progress (1): 3.9 kB Progress (1): 7.3 kB Progress (1): 12 kB Progress (1): 16 kB Progress (1): 19 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/26/apache-26.pom (21 kB at 978 kB/s) 2026-02-24T22:13:39.476 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.3.0/plexus-utils-3.3.0.pom 2026-02-24T22:13:39.496 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 801 B Progress (1): 2.6 kB Progress (1): 4.9 kB Progress (1): 5.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.3.0/plexus-utils-3.3.0.pom (5.2 kB at 259 kB/s) 2026-02-24T22:13:39.498 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.pom 2026-02-24T22:13:39.519 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 796 B Progress (1): 2.2 kB Progress (1): 3.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.pom (3.9 kB at 178 kB/s) 2026-02-24T22:13:39.524 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.pom 2026-02-24T22:13:39.544 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.2 kB Progress (1): 3.8 kB Progress (1): 6.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.pom (6.0 kB at 301 kB/s) 2026-02-24T22:13:39.546 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.pom 2026-02-24T22:13:39.565 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.2 kB Progress (1): 4.0 kB Progress (1): 6.2 kB Progress (1): 6.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.pom (6.3 kB at 313 kB/s) 2026-02-24T22:13:39.566 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.pom 2026-02-24T22:13:39.585 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 753 B Progress (1): 1.9 kB Progress (1): 3.5 kB Progress (1): 6.8 kB Progress (1): 9.1 kB Progress (1): 11 kB Progress (1): 14 kB Progress (1): 18 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.pom (20 kB at 1.1 MB/s) 2026-02-24T22:13:39.587 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.pom 2026-02-24T22:13:39.609 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.2 kB Progress (1): 3.7 kB Progress (1): 8.5 kB Progress (1): 12 kB Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.pom (15 kB at 692 kB/s) 2026-02-24T22:13:39.610 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.pom 2026-02-24T22:13:39.634 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.1 kB Progress (1): 2.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.pom (2.0 kB at 85 kB/s) 2026-02-24T22:13:39.635 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.pom 2026-02-24T22:13:39.660 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 729 B Progress (1): 2.3 kB Progress (1): 4.7 kB Progress (1): 8.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.pom (8.2 kB at 329 kB/s) 2026-02-24T22:13:39.661 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/8/plexus-8.pom 2026-02-24T22:13:39.687 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 767 B Progress (1): 2.9 kB Progress (1): 6.7 kB Progress (1): 7.8 kB Progress (1): 9.7 kB Progress (1): 13 kB Progress (1): 17 kB Progress (1): 22 kB Progress (1): 25 kB Progress (1): 25 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/8/plexus-8.pom (25 kB at 978 kB/s) 2026-02-24T22:13:39.690 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.jar 2026-02-24T22:13:39.706 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/36 kB Progress (1): 16/36 kB Progress (1): 32/36 kB Progress (1): 36 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.jar (36 kB at 2.3 MB/s) 2026-02-24T22:13:39.707 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.jar 2026-02-24T22:13:39.707 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.jar 2026-02-24T22:13:39.707 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.jar 2026-02-24T22:13:39.707 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.jar 2026-02-24T22:13:39.707 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.jar 2026-02-24T22:13:39.726 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/26 kB Progress (1): 16/26 kB Progress (1): 24/26 kB Progress (1): 26 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.jar (26 kB at 1.3 MB/s) 2026-02-24T22:13:39.726 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.jar 2026-02-24T22:13:39.738 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/211 kB Progress (1): 16/211 kB Progress (1): 32/211 kB Progress (1): 48/211 kB Progress (1): 65/211 kB Progress (1): 81/211 kB Progress (1): 98/211 kB Progress (1): 114/211 kB Progress (1): 115/211 kB Progress (2): 115/211 kB | 0/1.0 MB Progress (2): 131/211 kB | 0/1.0 MB Progress (2): 131/211 kB | 0/1.0 MB Progress (2): 147/211 kB | 0/1.0 MB Progress (2): 147/211 kB | 0/1.0 MB Progress (2): 164/211 kB | 0/1.0 MB Progress (2): 180/211 kB | 0/1.0 MB Progress (2): 180/211 kB | 0/1.0 MB Progress (2): 197/211 kB | 0/1.0 MB Progress (2): 211 kB | 0/1.0 MB Progress (2): 211 kB | 0.1/1.0 MB Progress (2): 211 kB | 0.1/1.0 MB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.jar (211 kB at 7.8 MB/s) 2026-02-24T22:13:39.738 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.jar 2026-02-24T22:13:39.738 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 0.1/1.0 MB Progress (1): 0.1/1.0 MB Progress (1): 0.1/1.0 MB Progress (1): 0.1/1.0 MB Progress (1): 0.2/1.0 MB Progress (1): 0.2/1.0 MB Progress (1): 0.2/1.0 MB Progress (1): 0.2/1.0 MB Progress (1): 0.2/1.0 MB Progress (1): 0.2/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0 MB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.jar (1.0 MB at 34 MB/s) 2026-02-24T22:13:39.739 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/58 kB Progress (1): 16/58 kB Progress (1): 32/58 kB Progress (1): 49/58 kB Progress (1): 58 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.jar (58 kB at 1.8 MB/s) 2026-02-24T22:13:39.742 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/79 kB Progress (1): 16/79 kB Progress (1): 32/79 kB Progress (1): 49/79 kB Progress (1): 65/79 kB Progress (1): 79 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.jar (79 kB at 2.2 MB/s) 2026-02-24T22:13:39.750 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/116 kB Progress (1): 16/116 kB Progress (1): 24/116 kB Progress (1): 41/116 kB Progress (1): 57/116 kB Progress (1): 73/116 kB Progress (1): 90/116 kB Progress (1): 106/116 kB Progress (1): 116 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.jar (116 kB at 2.6 MB/s) 2026-02-24T22:13:39.753 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 7.7/267 kB Progress (1): 16/267 kB Progress (1): 24/267 kB Progress (1): 41/267 kB Progress (1): 57/267 kB Progress (1): 73/267 kB Progress (1): 90/267 kB Progress (1): 106/267 kB Progress (1): 122/267 kB Progress (1): 139/267 kB Progress (1): 155/267 kB Progress (1): 172/267 kB Progress (1): 188/267 kB Progress (1): 204/267 kB Progress (1): 221/267 kB Progress (1): 229/267 kB Progress (1): 246/267 kB Progress (1): 262/267 kB Progress (1): 267 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.jar (267 kB at 5.7 MB/s) 2026-02-24T22:13:39.836 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] Building jar: /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/target/jcksum-1.0.jar 2026-02-24T22:13:39.847 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] ------------------------------------------------------------------------ 2026-02-24T22:13:39.848 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] BUILD SUCCESS 2026-02-24T22:13:39.848 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] ------------------------------------------------------------------------ 2026-02-24T22:13:39.848 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] Total time: 6.636 s 2026-02-24T22:13:39.848 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] Finished at: 2026-02-24T22:13:39Z 2026-02-24T22:13:39.849 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] ------------------------------------------------------------------------ 2026-02-24T22:13:39.890 INFO:tasks.workunit.client.0.trial046.stderr:+ ./mvnw test -Dtest=PutObjects 2026-02-24T22:13:39.898 INFO:tasks.workunit.client.0.trial046.stdout:Warning: JAVA_HOME environment variable is not set. 2026-02-24T22:13:40.411 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] Scanning for projects... 2026-02-24T22:13:40.453 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] 2026-02-24T22:13:40.453 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] ---------------------------< io.ceph:jcksum >--------------------------- 2026-02-24T22:13:40.453 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] Building jcksum 1.0 2026-02-24T22:13:40.453 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] from pom.xml 2026-02-24T22:13:40.453 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] --------------------------------[ jar ]--------------------------------- 2026-02-24T22:13:40.580 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] 2026-02-24T22:13:40.580 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] --- resources:3.3.1:resources (default-resources) @ jcksum --- 2026-02-24T22:13:40.623 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/main/resources 2026-02-24T22:13:40.624 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] 2026-02-24T22:13:40.624 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] --- compiler:3.11.0:compile (default-compile) @ jcksum --- 2026-02-24T22:13:40.697 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] Nothing to compile - all classes are up to date 2026-02-24T22:13:40.698 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] 2026-02-24T22:13:40.698 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] --- resources:3.3.1:testResources (default-testResources) @ jcksum --- 2026-02-24T22:13:40.699 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/test/resources 2026-02-24T22:13:40.699 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] 2026-02-24T22:13:40.699 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] --- compiler:3.11.0:testCompile (default-testCompile) @ jcksum --- 2026-02-24T22:13:40.701 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] Nothing to compile - all classes are up to date 2026-02-24T22:13:40.701 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] 2026-02-24T22:13:40.701 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] --- surefire:3.2.3:test (default-test) @ jcksum --- 2026-02-24T22:13:40.785 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider 2026-02-24T22:13:40.887 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.pom 2026-02-24T22:13:41.007 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 846 B Progress (1): 2.8 kB Progress (1): 4.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.pom (4.7 kB at 38 kB/s) 2026-02-24T22:13:41.011 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-providers/3.2.3/surefire-providers-3.2.3.pom 2026-02-24T22:13:41.037 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 849 B Progress (1): 2.4 kB Progress (1): 2.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-providers/3.2.3/surefire-providers-3.2.3.pom (2.6 kB at 104 kB/s) 2026-02-24T22:13:41.039 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.pom 2026-02-24T22:13:41.065 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 849 B Progress (1): 2.1 kB Progress (1): 2.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.pom (2.8 kB at 113 kB/s) 2026-02-24T22:13:41.067 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.pom 2026-02-24T22:13:41.094 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 966 B Progress (1): 2.6 kB Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.pom (3.0 kB at 108 kB/s) 2026-02-24T22:13:41.095 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.pom 2026-02-24T22:13:41.121 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 957 B Progress (1): 2.6 kB Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.pom (3.2 kB at 123 kB/s) 2026-02-24T22:13:41.123 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.pom 2026-02-24T22:13:41.150 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 1.2 kB Progress (1): 1.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.pom (1.7 kB at 62 kB/s) 2026-02-24T22:13:41.152 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.pom 2026-02-24T22:13:41.178 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 971 B Progress (1): 2.6 kB Progress (1): 2.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.pom (2.8 kB at 109 kB/s) 2026-02-24T22:13:41.183 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.jar 2026-02-24T22:13:41.213 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 0.9/27 kB Progress (1): 2.3/27 kB Progress (1): 3.7/27 kB Progress (1): 5.0/27 kB Progress (1): 6.4/27 kB Progress (1): 7.8/27 kB Progress (1): 9.1/27 kB Progress (1): 11/27 kB Progress (1): 12/27 kB Progress (1): 13/27 kB Progress (1): 15/27 kB Progress (1): 16/27 kB Progress (1): 17/27 kB Progress (1): 19/27 kB Progress (1): 20/27 kB Progress (1): 21/27 kB Progress (1): 23/27 kB Progress (1): 24/27 kB Progress (1): 26/27 kB Progress (1): 27/27 kB Progress (1): 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.jar (27 kB at 876 kB/s) 2026-02-24T22:13:41.215 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.jar 2026-02-24T22:13:41.215 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.jar 2026-02-24T22:13:41.215 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.jar 2026-02-24T22:13:41.216 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar 2026-02-24T22:13:41.216 INFO:tasks.workunit.client.0.trial046.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.jar 2026-02-24T22:13:41.250 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 0.9/18 kB Progress (1): 2.3/18 kB Progress (1): 3.7/18 kB Progress (1): 5.0/18 kB Progress (1): 9.3/18 kB Progress (1): 13/18 kB Progress (1): 18 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.jar (18 kB at 488 kB/s) 2026-02-24T22:13:41.259 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 0.9/7.7 kB Progress (1): 2.3/7.7 kB Progress (1): 3.7/7.7 kB Progress (1): 5.0/7.7 kB Progress (1): 6.4/7.7 kB Progress (1): 7.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar (7.7 kB at 174 kB/s) 2026-02-24T22:13:41.273 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 0.9/103 kB Progress (1): 2.3/103 kB Progress (1): 3.7/103 kB Progress (1): 5.0/103 kB Progress (1): 6.4/103 kB Progress (1): 7.8/103 kB Progress (1): 9.1/103 kB Progress (1): 11/103 kB Progress (1): 12/103 kB Progress (1): 13/103 kB Progress (1): 15/103 kB Progress (1): 16/103 kB Progress (1): 17/103 kB Progress (1): 19/103 kB Progress (1): 20/103 kB Progress (1): 21/103 kB Progress (1): 23/103 kB Progress (1): 24/103 kB Progress (1): 26/103 kB Progress (1): 27/103 kB Progress (1): 28/103 kB Progress (1): 30/103 kB Progress (1): 31/103 kB Progress (1): 32/103 kB Progress (1): 34/103 kB Progress (1): 35/103 kB Progress (1): 37/103 kB Progress (1): 38/103 kB Progress (1): 39/103 kB Progress (1): 41/103 kB Progress (1): 42/103 kB Progress (1): 43/103 kB Progress (1): 45/103 kB Progress (1): 46/103 kB Progress (1): 47/103 kB Progress (1): 49/103 kB Progress (1): 50/103 kB Progress (1): 52/103 kB Progress (1): 53/103 kB Progress (1): 54/103 kB Progress (1): 56/103 kB Progress (1): 60/103 kB Progress (1): 64/103 kB Progress (1): 65/103 kB Progress (1): 69/103 kB Progress (1): 73/103 kB Progress (1): 78/103 kB Progress (1): 82/103 kB Progress (1): 86/103 kB Progress (1): 90/103 kB Progress (1): 94/103 kB Progress (1): 99/103 kB Progress (1): 103/103 kB Progress (1): 103 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.jar (103 kB at 1.8 MB/s) 2026-02-24T22:13:41.290 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 0.9/169 kB Progress (2): 0.9/169 kB | 0.9/189 kB Progress (2): 2.3/169 kB | 0.9/189 kB Progress (2): 2.3/169 kB | 2.3/189 kB Progress (2): 3.4/169 kB | 2.3/189 kB Progress (2): 3.4/169 kB | 3.7/189 kB Progress (2): 4.8/169 kB | 3.7/189 kB Progress (2): 4.8/169 kB | 5.0/189 kB Progress (2): 6.1/169 kB | 5.0/189 kB Progress (2): 6.1/169 kB | 6.4/189 kB Progress (2): 7.5/169 kB | 6.4/189 kB Progress (2): 7.5/169 kB | 7.8/189 kB Progress (2): 8.9/169 kB | 7.8/189 kB Progress (2): 8.9/169 kB | 9.1/189 kB Progress (2): 10/169 kB | 9.1/189 kB Progress (2): 10/169 kB | 11/189 kB Progress (2): 12/169 kB | 11/189 kB Progress (2): 12/169 kB | 12/189 kB Progress (2): 13/169 kB | 12/189 kB Progress (2): 13/169 kB | 13/189 kB Progress (2): 14/169 kB | 13/189 kB Progress (2): 14/169 kB | 15/189 kB Progress (2): 16/169 kB | 15/189 kB Progress (2): 16/169 kB | 16/189 kB Progress (2): 17/169 kB | 16/189 kB Progress (2): 18/169 kB | 16/189 kB Progress (2): 18/169 kB | 17/189 kB Progress (2): 20/169 kB | 17/189 kB Progress (2): 20/169 kB | 19/189 kB Progress (2): 21/169 kB | 19/189 kB Progress (2): 21/169 kB | 20/189 kB Progress (2): 23/169 kB | 20/189 kB Progress (2): 23/169 kB | 21/189 kB Progress (2): 24/169 kB | 21/189 kB Progress (2): 25/169 kB | 21/189 kB Progress (2): 25/169 kB | 23/189 kB Progress (2): 27/169 kB | 23/189 kB Progress (2): 28/169 kB | 23/189 kB Progress (2): 28/169 kB | 24/189 kB Progress (2): 29/169 kB | 24/189 kB Progress (2): 29/169 kB | 26/189 kB Progress (2): 31/169 kB | 26/189 kB Progress (2): 31/169 kB | 27/189 kB Progress (2): 32/169 kB | 27/189 kB Progress (2): 32/169 kB | 28/189 kB Progress (2): 33/169 kB | 28/189 kB Progress (2): 35/169 kB | 28/189 kB Progress (2): 35/169 kB | 30/189 kB Progress (2): 36/169 kB | 30/189 kB Progress (2): 38/169 kB | 30/189 kB Progress (2): 38/169 kB | 31/189 kB Progress (2): 39/169 kB | 31/189 kB Progress (2): 39/169 kB | 32/189 kB Progress (2): 40/169 kB | 32/189 kB Progress (2): 40/169 kB | 34/189 kB Progress (2): 42/169 kB | 34/189 kB Progress (2): 42/169 kB | 35/189 kB Progress (2): 43/169 kB | 35/189 kB Progress (2): 43/169 kB | 37/189 kB Progress (2): 44/169 kB | 37/189 kB Progress (2): 44/169 kB | 38/189 kB Progress (2): 46/169 kB | 38/189 kB Progress (2): 46/169 kB | 39/189 kB Progress (2): 47/169 kB | 39/189 kB Progress (2): 47/169 kB | 41/189 kB Progress (2): 49/169 kB | 41/189 kB Progress (2): 49/169 kB | 42/189 kB Progress (2): 50/169 kB | 42/189 kB Progress (2): 50/169 kB | 43/189 kB Progress (2): 51/169 kB | 43/189 kB Progress (2): 53/169 kB | 43/189 kB Progress (2): 53/169 kB | 45/189 kB Progress (2): 54/169 kB | 45/189 kB Progress (2): 54/169 kB | 46/189 kB Progress (2): 55/169 kB | 46/189 kB Progress (2): 55/169 kB | 47/189 kB Progress (2): 60/169 kB | 47/189 kB Progress (2): 60/169 kB | 49/189 kB Progress (2): 64/169 kB | 49/189 kB Progress (2): 64/169 kB | 50/189 kB Progress (2): 68/169 kB | 50/189 kB Progress (2): 68/169 kB | 52/189 kB Progress (2): 72/169 kB | 52/189 kB Progress (2): 72/169 kB | 53/189 kB Progress (2): 77/169 kB | 53/189 kB Progress (2): 77/169 kB | 54/189 kB Progress (2): 81/169 kB | 54/189 kB Progress (2): 81/169 kB | 56/189 kB Progress (2): 85/169 kB | 56/189 kB Progress (2): 85/169 kB | 60/189 kB Progress (2): 89/169 kB | 60/189 kB Progress (2): 89/169 kB | 64/189 kB Progress (2): 89/169 kB | 68/189 kB Progress (2): 93/169 kB | 68/189 kB Progress (2): 93/169 kB | 73/189 kB Progress (2): 98/169 kB | 73/189 kB Progress (2): 98/169 kB | 77/189 kB Progress (2): 102/169 kB | 77/189 kB Progress (2): 102/169 kB | 81/189 kB Progress (2): 106/169 kB | 81/189 kB Progress (2): 106/169 kB | 85/189 kB Progress (2): 106/169 kB | 90/189 kB Progress (2): 110/169 kB | 90/189 kB Progress (2): 110/169 kB | 94/189 kB Progress (2): 115/169 kB | 94/189 kB Progress (2): 115/169 kB | 98/189 kB Progress (2): 119/169 kB | 98/189 kB Progress (2): 119/169 kB | 102/189 kB Progress (2): 119/169 kB | 106/189 kB Progress (2): 123/169 kB | 106/189 kB Progress (2): 123/169 kB | 111/189 kB Progress (2): 126/169 kB | 111/189 kB Progress (2): 126/169 kB | 115/189 kB Progress (2): 126/169 kB | 119/189 kB Progress (2): 130/169 kB | 119/189 kB Progress (2): 130/169 kB | 123/189 kB Progress (2): 135/169 kB | 123/189 kB Progress (2): 135/169 kB | 128/189 kB Progress (2): 139/169 kB | 128/189 kB Progress (2): 143/169 kB | 128/189 kB Progress (2): 143/169 kB | 132/189 kB Progress (2): 147/169 kB | 132/189 kB Progress (2): 147/169 kB | 136/189 kB Progress (2): 152/169 kB | 136/189 kB Progress (2): 152/169 kB | 140/189 kB Progress (2): 156/169 kB | 140/189 kB Progress (2): 156/169 kB | 144/189 kB Progress (2): 160/169 kB | 144/189 kB Progress (2): 160/169 kB | 149/189 kB Progress (2): 164/169 kB | 149/189 kB Progress (2): 164/169 kB | 153/189 kB Progress (2): 169/169 kB | 153/189 kB Progress (2): 169/169 kB | 157/189 kB Progress (2): 169 kB | 157/189 kB Progress (2): 169 kB | 161/189 kB Progress (2): 169 kB | 166/189 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.jar (169 kB at 2.2 MB/s) 2026-02-24T22:13:41.292 INFO:tasks.workunit.client.0.trial046.stdout:Progress (1): 170/189 kB Progress (1): 174/189 kB Progress (1): 178/189 kB Progress (1): 183/189 kB Progress (1): 187/189 kB Progress (1): 189 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.jar (189 kB at 2.4 MB/s) 2026-02-24T22:13:41.305 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] 2026-02-24T22:13:41.305 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] ------------------------------------------------------- 2026-02-24T22:13:41.305 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] T E S T S 2026-02-24T22:13:41.305 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] ------------------------------------------------------- 2026-02-24T22:13:41.551 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] Running io.ceph.jcksum.PutObjects 2026-02-24T22:13:41.616 INFO:tasks.workunit.client.0.trial046.stdout:PutObjects.java: starting test run: 2026-02-24T22:13:41.616 INFO:tasks.workunit.client.0.trial046.stdout: AccessKey=lNCnR47C2g+ZidCWBAUuwfSAA7Q= 2026-02-24T22:13:41.616 INFO:tasks.workunit.client.0.trial046.stdout: SecretKey=tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== 2026-02-24T22:13:41.617 INFO:tasks.workunit.client.0.trial046.stdout: EndpointUrl=https://trial046.front.sepia.ceph.com:443 2026-02-24T22:13:41.617 INFO:tasks.workunit.client.0.trial046.stderr:[main] WARN software.amazon.awssdk.http.apache.ApacheHttpClient - SSL Certificate verification is disabled. This is not a safe setting and should only be used for testing. 2026-02-24T22:13:41.714 INFO:tasks.workunit.client.0.trial046.stdout:DEBUG: Environment Variables Read 2026-02-24T22:13:41.814 INFO:tasks.workunit.client.0.trial046.stdout:DEBUG: S3 Client Initialized 2026-02-24T22:13:41.814 INFO:tasks.workunit.client.0.trial046.stdout:DEBUG: Generating File 2026-02-24T22:13:41.815 INFO:tasks.workunit.client.0.trial046.stdout:DEBUG: File Generated 2026-02-24T22:13:41.815 INFO:tasks.workunit.client.0.trial046.stdout:DEBUG: Generating File 2026-02-24T22:13:41.815 INFO:tasks.workunit.client.0.trial046.stdout:DEBUG: File Generated 2026-02-24T22:13:41.815 INFO:tasks.workunit.client.0.trial046.stdout:DEBUG: Generating File 2026-02-24T22:13:41.915 INFO:tasks.workunit.client.0.trial046.stdout:DEBUG: File Generated 2026-02-24T22:13:41.917 INFO:tasks.workunit.client.0.trial046.stdout:DEBUG: Generating File 2026-02-24T22:13:41.917 INFO:tasks.workunit.client.0.trial046.stdout:DEBUG: File Generated 2026-02-24T22:13:41.917 INFO:tasks.workunit.client.0.trial046.stdout:DEBUG: Generating File 2026-02-24T22:13:41.918 INFO:tasks.workunit.client.0.trial046.stdout:DEBUG: File Generated 2026-02-24T22:13:41.918 INFO:tasks.workunit.client.0.trial046.stdout:DEBUG: Generating File 2026-02-24T22:13:41.918 INFO:tasks.workunit.client.0.trial046.stdout:DEBUG: File Generated 2026-02-24T22:13:41.918 INFO:tasks.workunit.client.0.trial046.stdout:DEBUG: Generated Big Files 2026-02-24T22:13:42.012 INFO:tasks.workunit.client.0.trial046.stdout:HeadBucketResponse() 2026-02-24T22:13:42.013 INFO:tasks.workunit.client.0.trial046.stdout:sheik is ready 2026-02-24T22:13:42.013 INFO:tasks.workunit.client.0.trial046.stdout:DEBUG: Test Bucket Created 2026-02-24T22:13:42.105 INFO:tasks.workunit.client.0.trial046.stdout:putObjectFromFileNoCksum called with file-0b 2026-02-24T22:13:42.114 INFO:tasks.workunit.client.0.trial046.stdout:putObjectFromFileNoCksum called with file-8b 2026-02-24T22:13:42.214 INFO:tasks.workunit.client.0.trial046.stdout:putObjectFromFileNoCksum called with file-200b 2026-02-24T22:13:42.284 INFO:tasks.workunit.client.0.trial046.stdout:putObjectFromFileNoCksum called with file-21983b 2026-02-24T22:13:42.314 INFO:tasks.workunit.client.0.trial046.stdout:putObjectFromFileNoCksum called with file-5519b 2026-02-24T22:13:42.381 INFO:tasks.workunit.client.0.trial046.stdout:putObjectFromFileNoCksum called with file-204329b 2026-02-24T22:13:42.414 INFO:tasks.workunit.client.0.trial046.stdout:putObjectFromFileNoCksum called with file-256k 2026-02-24T22:13:42.468 INFO:tasks.workunit.client.0.trial046.stdout:putObjectFromFileNoCksum called with file-1m 2026-02-24T22:13:42.514 INFO:tasks.workunit.client.0.trial046.stdout:putObjectFromFileNoCksum called with file-1038757b 2026-02-24T22:13:42.614 INFO:tasks.workunit.client.0.trial046.stdout:mpuObjectFromFileNoCksum called with file-5m 2026-02-24T22:13:44.116 INFO:tasks.workunit.client.0.trial046.stdout:mpuObjectFromFileNoCksum called with file-10m 2026-02-24T22:13:44.416 INFO:tasks.workunit.client.0.trial046.stdout:mpuObjectFromFileNoCksum called with file-100m 2026-02-24T22:13:45.837 INFO:tasks.workunit.client.0.trial046.stdout:arg: file-0b 2026-02-24T22:13:45.838 INFO:tasks.workunit.client.0.trial046.stdout:arg: file-8b 2026-02-24T22:13:45.839 INFO:tasks.workunit.client.0.trial046.stdout:arg: file-200b 2026-02-24T22:13:45.839 INFO:tasks.workunit.client.0.trial046.stdout:arg: file-21983b 2026-02-24T22:13:45.839 INFO:tasks.workunit.client.0.trial046.stdout:arg: file-5519b 2026-02-24T22:13:45.839 INFO:tasks.workunit.client.0.trial046.stdout:arg: file-204329b 2026-02-24T22:13:45.840 INFO:tasks.workunit.client.0.trial046.stdout:arg: file-256k 2026-02-24T22:13:45.840 INFO:tasks.workunit.client.0.trial046.stdout:arg: file-1m 2026-02-24T22:13:45.841 INFO:tasks.workunit.client.0.trial046.stdout:arg: file-1038757b 2026-02-24T22:13:45.867 INFO:tasks.workunit.client.0.trial046.stdout:putObjectFromFileCksum called with file-0b 2026-02-24T22:13:45.867 INFO:tasks.workunit.client.0.trial046.stdout:PutObjectResponse 2026-02-24T22:13:45.920 INFO:tasks.workunit.client.0.trial046.stdout:putObjectFromFileCksum called with file-8b 2026-02-24T22:13:45.920 INFO:tasks.workunit.client.0.trial046.stdout:PutObjectResponse 2026-02-24T22:13:45.926 INFO:tasks.workunit.client.0.trial046.stdout:putObjectFromFileCksum called with file-200b 2026-02-24T22:13:45.976 INFO:tasks.workunit.client.0.trial046.stdout:PutObjectResponse 2026-02-24T22:13:46.026 INFO:tasks.workunit.client.0.trial046.stdout:putObjectFromFileCksum called with file-21983b 2026-02-24T22:13:46.026 INFO:tasks.workunit.client.0.trial046.stdout:PutObjectResponse 2026-02-24T22:13:46.088 INFO:tasks.workunit.client.0.trial046.stdout:putObjectFromFileCksum called with file-5519b 2026-02-24T22:13:46.088 INFO:tasks.workunit.client.0.trial046.stdout:PutObjectResponse 2026-02-24T22:13:46.126 INFO:tasks.workunit.client.0.trial046.stdout:putObjectFromFileCksum called with file-204329b 2026-02-24T22:13:46.127 INFO:tasks.workunit.client.0.trial046.stdout:PutObjectResponse 2026-02-24T22:13:46.175 INFO:tasks.workunit.client.0.trial046.stdout:putObjectFromFileCksum called with file-256k 2026-02-24T22:13:46.176 INFO:tasks.workunit.client.0.trial046.stdout:PutObjectResponse 2026-02-24T22:13:46.208 INFO:tasks.workunit.client.0.trial046.stdout:putObjectFromFileCksum called with file-1m 2026-02-24T22:13:46.208 INFO:tasks.workunit.client.0.trial046.stdout:PutObjectResponse 2026-02-24T22:13:46.226 INFO:tasks.workunit.client.0.trial046.stdout:putObjectFromFileCksum called with file-1038757b 2026-02-24T22:13:46.302 INFO:tasks.workunit.client.0.trial046.stdout:PutObjectResponse 2026-02-24T22:13:46.326 INFO:tasks.workunit.client.0.trial046.stdout:mpuObjectFromFileCksum called with file-5m 2026-02-24T22:13:46.527 INFO:tasks.workunit.client.0.trial046.stdout:mpuObjectFromFileCksum called with file-10m 2026-02-24T22:13:46.827 INFO:tasks.workunit.client.0.trial046.stdout:mpuObjectFromFileCksum called with file-100m 2026-02-24T22:13:48.277 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] Tests run: 33, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.716 s -- in io.ceph.jcksum.PutObjects 2026-02-24T22:13:48.298 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] 2026-02-24T22:13:48.299 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] Results: 2026-02-24T22:13:48.299 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] 2026-02-24T22:13:48.299 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] Tests run: 33, Failures: 0, Errors: 0, Skipped: 0 2026-02-24T22:13:48.299 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] 2026-02-24T22:13:48.301 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] ------------------------------------------------------------------------ 2026-02-24T22:13:48.301 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] BUILD SUCCESS 2026-02-24T22:13:48.301 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] ------------------------------------------------------------------------ 2026-02-24T22:13:48.301 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] Total time: 7.900 s 2026-02-24T22:13:48.302 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] Finished at: 2026-02-24T22:13:48Z 2026-02-24T22:13:48.302 INFO:tasks.workunit.client.0.trial046.stdout:[INFO] ------------------------------------------------------------------------ 2026-02-24T22:13:48.341 INFO:tasks.workunit.client.0.trial046.stderr:+ exit 2026-02-24T22:13:48.342 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T22:13:48.342 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T22:13:48.392 INFO:tasks.workunit:Running workunits matching rgw/test_gosdk2.sh on client.0... 2026-02-24T22:13:48.395 INFO:tasks.workunit:Running workunit rgw/test_gosdk2.sh... 2026-02-24T22:13:48.395 DEBUG:teuthology.orchestra.run.trial046:workunit test rgw/test_gosdk2.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7e05f4fdea5af173284f247d6b0935b337d4ccf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_gosdk2.sh 2026-02-24T22:13:48.442 INFO:tasks.workunit.client.0.trial046.stderr:+ [ -z ] 2026-02-24T22:13:48.442 INFO:tasks.workunit.client.0.trial046.stderr:+ export AWS_ACCESS_KEY_ID=lNCnR47C2g+ZidCWBAUuwfSAA7Q= 2026-02-24T22:13:48.442 INFO:tasks.workunit.client.0.trial046.stderr:+ export AWS_SECRET_ACCESS_KEY=tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== 2026-02-24T22:13:48.442 INFO:tasks.workunit.client.0.trial046.stderr:+ radosgw-admin user create --uid ceph-test-gosdk2 --access-key lNCnR47C2g+ZidCWBAUuwfSAA7Q= --secret tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== --display-name gosdk2 test user --email gosdk2@example.com 2026-02-24T22:13:48.458 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:13:48.458 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:13:48.476 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.477+0000 7f33df12ea80 4 Realm: () 2026-02-24T22:13:48.476 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.477+0000 7f33df12ea80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:13:48.476 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.477+0000 7f33df12ea80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:13:48.476 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.477+0000 7f33df12ea80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:13:48.476 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.477+0000 7f33df12ea80 20 zonegroup default 2026-02-24T22:13:48.477 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.477+0000 7f33df12ea80 20 rados->read ofs=0 len=0 2026-02-24T22:13:48.478 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.477+0000 7f33df12ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:13:48.478 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.477+0000 7f33df12ea80 20 rados->read ofs=0 len=0 2026-02-24T22:13:48.478 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.477+0000 7f33df12ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:13:48.479 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.477+0000 7f33df12ea80 20 started sync module instance, tier type = 2026-02-24T22:13:48.479 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.477+0000 7f33df12ea80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:13:48.481 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.481+0000 7f33dcc02640 0 -- 10.20.193.46:0/1572939682 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x561e44e11e80 msgr2=0x561e44e32300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:13:48.481 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.481+0000 7f33df12ea80 20 add_watcher() i=1 2026-02-24T22:13:48.481 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.481+0000 7f33df12ea80 20 add_watcher() i=3 2026-02-24T22:13:48.481 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.481+0000 7f33df12ea80 20 add_watcher() i=6 2026-02-24T22:13:48.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.481+0000 7f33df12ea80 20 add_watcher() i=0 2026-02-24T22:13:48.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.481+0000 7f33df12ea80 20 add_watcher() i=5 2026-02-24T22:13:48.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.481+0000 7f33df12ea80 20 add_watcher() i=2 2026-02-24T22:13:48.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.481+0000 7f33df12ea80 20 add_watcher() i=4 2026-02-24T22:13:48.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.481+0000 7f33df12ea80 20 add_watcher() i=7 2026-02-24T22:13:48.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.481+0000 7f33df12ea80 2 all 8 watchers are set, enabling cache 2026-02-24T22:13:48.484 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.481+0000 7f33df12ea80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:13:48.484 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.481+0000 7f33df12ea80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:13:48.484 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.485+0000 7f33df12ea80 5 note: GC not initialized 2026-02-24T22:13:48.484 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.485+0000 7f3389fe3640 20 reqs_thread_entry: start 2026-02-24T22:13:48.484 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.485+0000 7f33dd403640 0 -- 10.20.193.46:0/3328561589 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x561e44d9eee0 msgr2=0x561e44d8df60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:13:48.726 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.725+0000 7f33df12ea80 20 init_complete bucket index max shards: 1 2026-02-24T22:13:48.727 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.725+0000 7f33df12ea80 20 Filter name: none 2026-02-24T22:13:48.727 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.725+0000 7f32e3fff640 20 reqs_thread_entry: start 2026-02-24T22:13:48.727 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.725+0000 7f33df12ea80 10 cache get: name=default.rgw.meta+users.uid+ceph-test-gosdk2 : miss 2026-02-24T22:13:48.727 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.725+0000 7f33df12ea80 20 rados->read ofs=0 len=0 2026-02-24T22:13:48.727 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.725+0000 7f33df12ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:13:48.727 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.725+0000 7f33df12ea80 10 cache put: name=default.rgw.meta+users.uid+ceph-test-gosdk2 info.flags=0x0 2026-02-24T22:13:48.727 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.725+0000 7f33df12ea80 10 adding default.rgw.meta+users.uid+ceph-test-gosdk2 to cache LRU end 2026-02-24T22:13:48.727 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.725+0000 7f33df12ea80 10 cache get: name=default.rgw.meta+users.email+gosdk2@example.com : miss 2026-02-24T22:13:48.727 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.725+0000 7f33df12ea80 20 rados->read ofs=0 len=0 2026-02-24T22:13:48.727 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.725+0000 7f33df12ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:13:48.727 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.725+0000 7f33df12ea80 10 cache put: name=default.rgw.meta+users.email+gosdk2@example.com info.flags=0x0 2026-02-24T22:13:48.728 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.725+0000 7f33df12ea80 10 adding default.rgw.meta+users.email+gosdk2@example.com to cache LRU end 2026-02-24T22:13:48.728 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.725+0000 7f33df12ea80 10 cache get: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= : miss 2026-02-24T22:13:48.728 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.725+0000 7f33df12ea80 20 rados->read ofs=0 len=0 2026-02-24T22:13:48.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.725+0000 7f33df12ea80 20 rados_obj.operate() r=0 bl.length=19 2026-02-24T22:13:48.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.725+0000 7f33df12ea80 10 cache put: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= info.flags=0x5 2026-02-24T22:13:48.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.725+0000 7f33df12ea80 10 adding default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= to cache LRU end 2026-02-24T22:13:48.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.725+0000 7f33df12ea80 10 cache get: name=default.rgw.meta+users.uid+ceph-test-maven : miss 2026-02-24T22:13:48.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.725+0000 7f33df12ea80 20 rados->read ofs=0 len=0 2026-02-24T22:13:48.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.725+0000 7f33df12ea80 20 rados_obj.operate() r=0 bl.length=489 2026-02-24T22:13:48.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.725+0000 7f33df12ea80 10 cache put: name=default.rgw.meta+users.uid+ceph-test-maven info.flags=0x13 2026-02-24T22:13:48.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.725+0000 7f33df12ea80 10 adding default.rgw.meta+users.uid+ceph-test-maven to cache LRU end 2026-02-24T22:13:48.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.725+0000 7f33df12ea80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:13:48.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.725+0000 7f33df12ea80 10 chain_cache_entry: cache_locator=default.rgw.meta+users.uid+ceph-test-maven 2026-02-24T22:13:48.730 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.725+0000 7f33df12ea80 20 get_user_info_from_index found 0 xattrs 2026-02-24T22:13:48.730 INFO:tasks.workunit.client.0.trial046.stderr:could not create user: unable to parse parameters, duplicate key provided 2026-02-24T22:13:48.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.729+0000 7f33df12ea80 20 remove_watcher() i=4 2026-02-24T22:13:48.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.729+0000 7f33df12ea80 2 removed watcher, disabling cache 2026-02-24T22:13:48.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.729+0000 7f33df12ea80 20 remove_watcher() i=3 2026-02-24T22:13:48.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.729+0000 7f33df12ea80 20 remove_watcher() i=0 2026-02-24T22:13:48.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.729+0000 7f33df12ea80 20 remove_watcher() i=2 2026-02-24T22:13:48.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.729+0000 7f33df12ea80 20 remove_watcher() i=1 2026-02-24T22:13:48.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.729+0000 7f33df12ea80 20 remove_watcher() i=7 2026-02-24T22:13:48.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.729+0000 7f33df12ea80 20 remove_watcher() i=6 2026-02-24T22:13:48.732 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:13:48.729+0000 7f33df12ea80 20 remove_watcher() i=5 2026-02-24T22:13:48.738 INFO:tasks.workunit.client.0.trial046.stderr:+ echo gosdk2 user exists 2026-02-24T22:13:48.738 INFO:tasks.workunit.client.0.trial046.stdout:gosdk2 user exists 2026-02-24T22:13:48.739 INFO:tasks.workunit.client.0.trial046.stderr:+ [ -z ] 2026-02-24T22:13:48.739 INFO:tasks.workunit.client.0.trial046.stderr:+ [ -z /home/ubuntu/cephtest ] 2026-02-24T22:13:48.739 INFO:tasks.workunit.client.0.trial046.stderr:+ cat /home/ubuntu/cephtest/url_file 2026-02-24T22:13:48.739 INFO:tasks.workunit.client.0.trial046.stderr:+ export RGW_HTTP_ENDPOINT_URL=https://trial046.front.sepia.ceph.com:443 2026-02-24T22:13:48.740 INFO:tasks.workunit.client.0.trial046.stderr:+ [ -z /home/ubuntu/cephtest/clone.client.0 ] 2026-02-24T22:13:48.740 INFO:tasks.workunit.client.0.trial046.stderr:+ cd /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/gcksum 2026-02-24T22:13:48.740 INFO:tasks.workunit.client.0.trial046.stderr:+ go mod download github.com/aws/aws-sdk-go-v2 2026-02-24T22:13:48.741 INFO:tasks.workunit.client.0.trial046.stderr:go: errors parsing go.mod: 2026-02-24T22:13:48.741 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/gcksum/go.mod:3: invalid go version '1.23.4': must match format 1.23 2026-02-24T22:13:48.742 INFO:tasks.workunit.client.0.trial046.stderr:+ go mod download github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream 2026-02-24T22:13:48.743 INFO:tasks.workunit.client.0.trial046.stderr:go: errors parsing go.mod: 2026-02-24T22:13:48.743 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/gcksum/go.mod:3: invalid go version '1.23.4': must match format 1.23 2026-02-24T22:13:48.744 INFO:tasks.workunit.client.0.trial046.stderr:+ go mod download github.com/aws/aws-sdk-go-v2/config 2026-02-24T22:13:48.746 INFO:tasks.workunit.client.0.trial046.stderr:go: errors parsing go.mod: 2026-02-24T22:13:48.746 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/gcksum/go.mod:3: invalid go version '1.23.4': must match format 1.23 2026-02-24T22:13:48.746 INFO:tasks.workunit.client.0.trial046.stderr:+ go mod download github.com/aws/aws-sdk-go-v2/credentials 2026-02-24T22:13:48.749 INFO:tasks.workunit.client.0.trial046.stderr:go: errors parsing go.mod: 2026-02-24T22:13:48.749 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/gcksum/go.mod:3: invalid go version '1.23.4': must match format 1.23 2026-02-24T22:13:48.749 INFO:tasks.workunit.client.0.trial046.stderr:+ go mod download github.com/aws/aws-sdk-go-v2/feature/ec2/imds 2026-02-24T22:13:48.752 INFO:tasks.workunit.client.0.trial046.stderr:go: errors parsing go.mod: 2026-02-24T22:13:48.752 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/gcksum/go.mod:3: invalid go version '1.23.4': must match format 1.23 2026-02-24T22:13:48.752 INFO:tasks.workunit.client.0.trial046.stderr:+ go mod download github.com/aws/aws-sdk-go-v2/internal/configsources 2026-02-24T22:13:48.755 INFO:tasks.workunit.client.0.trial046.stderr:go: errors parsing go.mod: 2026-02-24T22:13:48.755 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/gcksum/go.mod:3: invalid go version '1.23.4': must match format 1.23 2026-02-24T22:13:48.755 INFO:tasks.workunit.client.0.trial046.stderr:+ go mod download github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 2026-02-24T22:13:48.758 INFO:tasks.workunit.client.0.trial046.stderr:go: errors parsing go.mod: 2026-02-24T22:13:48.758 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/gcksum/go.mod:3: invalid go version '1.23.4': must match format 1.23 2026-02-24T22:13:48.758 INFO:tasks.workunit.client.0.trial046.stderr:+ go mod download github.com/aws/aws-sdk-go-v2/internal/ini 2026-02-24T22:13:48.761 INFO:tasks.workunit.client.0.trial046.stderr:go: errors parsing go.mod: 2026-02-24T22:13:48.761 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/gcksum/go.mod:3: invalid go version '1.23.4': must match format 1.23 2026-02-24T22:13:48.762 INFO:tasks.workunit.client.0.trial046.stderr:+ go mod download github.com/aws/aws-sdk-go-v2/internal/v4a 2026-02-24T22:13:48.764 INFO:tasks.workunit.client.0.trial046.stderr:go: errors parsing go.mod: 2026-02-24T22:13:48.764 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/gcksum/go.mod:3: invalid go version '1.23.4': must match format 1.23 2026-02-24T22:13:48.765 INFO:tasks.workunit.client.0.trial046.stderr:+ go mod download github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding 2026-02-24T22:13:48.767 INFO:tasks.workunit.client.0.trial046.stderr:go: errors parsing go.mod: 2026-02-24T22:13:48.767 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/gcksum/go.mod:3: invalid go version '1.23.4': must match format 1.23 2026-02-24T22:13:48.768 INFO:tasks.workunit.client.0.trial046.stderr:+ go mod download github.com/aws/aws-sdk-go-v2/service/internal/checksum 2026-02-24T22:13:48.770 INFO:tasks.workunit.client.0.trial046.stderr:go: errors parsing go.mod: 2026-02-24T22:13:48.770 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/gcksum/go.mod:3: invalid go version '1.23.4': must match format 1.23 2026-02-24T22:13:48.771 INFO:tasks.workunit.client.0.trial046.stderr:+ go mod download github.com/aws/aws-sdk-go-v2/service/internal/presigned-url 2026-02-24T22:13:48.773 INFO:tasks.workunit.client.0.trial046.stderr:go: errors parsing go.mod: 2026-02-24T22:13:48.773 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/gcksum/go.mod:3: invalid go version '1.23.4': must match format 1.23 2026-02-24T22:13:48.774 INFO:tasks.workunit.client.0.trial046.stderr:+ go mod download github.com/aws/aws-sdk-go-v2/service/internal/s3shared 2026-02-24T22:13:48.776 INFO:tasks.workunit.client.0.trial046.stderr:go: errors parsing go.mod: 2026-02-24T22:13:48.776 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/gcksum/go.mod:3: invalid go version '1.23.4': must match format 1.23 2026-02-24T22:13:48.777 INFO:tasks.workunit.client.0.trial046.stderr:+ go mod download github.com/aws/aws-sdk-go-v2/service/s3 2026-02-24T22:13:48.778 INFO:tasks.workunit.client.0.trial046.stderr:go: errors parsing go.mod: 2026-02-24T22:13:48.778 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/gcksum/go.mod:3: invalid go version '1.23.4': must match format 1.23 2026-02-24T22:13:48.779 INFO:tasks.workunit.client.0.trial046.stderr:+ go mod download github.com/aws/aws-sdk-go-v2/service/sso 2026-02-24T22:13:48.780 INFO:tasks.workunit.client.0.trial046.stderr:go: errors parsing go.mod: 2026-02-24T22:13:48.780 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/gcksum/go.mod:3: invalid go version '1.23.4': must match format 1.23 2026-02-24T22:13:48.781 INFO:tasks.workunit.client.0.trial046.stderr:+ go mod download github.com/aws/aws-sdk-go-v2/service/ssooidc 2026-02-24T22:13:48.783 INFO:tasks.workunit.client.0.trial046.stderr:go: errors parsing go.mod: 2026-02-24T22:13:48.784 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/gcksum/go.mod:3: invalid go version '1.23.4': must match format 1.23 2026-02-24T22:13:48.784 INFO:tasks.workunit.client.0.trial046.stderr:+ go mod download github.com/aws/aws-sdk-go-v2/service/sts 2026-02-24T22:13:48.786 INFO:tasks.workunit.client.0.trial046.stderr:go: errors parsing go.mod: 2026-02-24T22:13:48.786 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/gcksum/go.mod:3: invalid go version '1.23.4': must match format 1.23 2026-02-24T22:13:48.787 INFO:tasks.workunit.client.0.trial046.stderr:+ go mod download github.com/aws/smithy-go 2026-02-24T22:13:48.788 INFO:tasks.workunit.client.0.trial046.stderr:go: errors parsing go.mod: 2026-02-24T22:13:48.788 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/gcksum/go.mod:3: invalid go version '1.23.4': must match format 1.23 2026-02-24T22:13:48.789 INFO:tasks.workunit.client.0.trial046.stderr:+ go test 2026-02-24T22:13:48.791 INFO:tasks.workunit.client.0.trial046.stderr:go: errors parsing go.mod: 2026-02-24T22:13:48.791 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/gcksum/go.mod:3: invalid go version '1.23.4': must match format 1.23 2026-02-24T22:13:48.792 INFO:tasks.workunit.client.0.trial046.stderr:+ exit 0 2026-02-24T22:13:48.793 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T22:13:48.793 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T22:13:48.843 INFO:tasks.workunit:Stopping ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'cls/test_cls_rgw_gc.sh', 'cls/test_cls_rgw_stats.sh', 'cls/test_cls_cmpomap.sh', 'cls/test_cls_2pc_queue.sh', 'cls/test_cls_user.sh', 'cls/test_cls_sem_set.sh', 'rgw/test_rgw_gc_log.sh', 'rgw/test_rgw_obj.sh', 'rgw/test_rgw_datalog.sh', 'rgw/test_librgw_file.sh', 'rgw/test_awssdkv4_sig.sh', 'rgw/test_gosdk2.sh'] on client.0... 2026-02-24T22:13:48.844 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-24T22:13:49.251 DEBUG:teuthology.parallel:result is None 2026-02-24T22:13:49.252 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-24T22:13:49.261 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-24T22:13:49.261 DEBUG:teuthology.orchestra.run.trial046:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-24T22:13:49.311 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-24T22:13:49.311 INFO:teuthology.run_tasks:Running task workunit... 2026-02-24T22:13:49.326 INFO:tasks.workunit:Pulling workunits from ref f7e05f4fdea5af173284f247d6b0935b337d4ccf 2026-02-24T22:13:49.327 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-24T22:13:49.327 DEBUG:teuthology.orchestra.run.trial046:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-24T22:13:49.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T22:13:49.355 INFO:teuthology.orchestra.run.trial046.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-24T22:13:49.356 DEBUG:teuthology.orchestra.run.trial046:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-24T22:13:49.399 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-24T22:13:49.399 DEBUG:teuthology.orchestra.run.trial046:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-24T22:13:49.447 INFO:tasks.workunit:timeout=3h 2026-02-24T22:13:49.447 INFO:tasks.workunit:cleanup=True 2026-02-24T22:13:49.448 DEBUG:teuthology.orchestra.run.trial046:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/samarahu/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout f7e05f4fdea5af173284f247d6b0935b337d4ccf 2026-02-24T22:13:49.492 INFO:tasks.workunit.client.0.trial046.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-24T22:14:11.783 INFO:tasks.workunit.client.0.trial046.stderr:Note: switching to 'f7e05f4fdea5af173284f247d6b0935b337d4ccf'. 2026-02-24T22:14:11.784 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:14:11.784 INFO:tasks.workunit.client.0.trial046.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-24T22:14:11.784 INFO:tasks.workunit.client.0.trial046.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-24T22:14:11.784 INFO:tasks.workunit.client.0.trial046.stderr:state without impacting any branches by switching back to a branch. 2026-02-24T22:14:11.784 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:14:11.784 INFO:tasks.workunit.client.0.trial046.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-24T22:14:11.784 INFO:tasks.workunit.client.0.trial046.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-24T22:14:11.784 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:14:11.784 INFO:tasks.workunit.client.0.trial046.stderr: git switch -c 2026-02-24T22:14:11.784 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:14:11.785 INFO:tasks.workunit.client.0.trial046.stderr:Or undo this operation with: 2026-02-24T22:14:11.785 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:14:11.785 INFO:tasks.workunit.client.0.trial046.stderr: git switch - 2026-02-24T22:14:11.785 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:14:11.785 INFO:tasks.workunit.client.0.trial046.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-24T22:14:11.785 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:14:11.785 INFO:tasks.workunit.client.0.trial046.stderr:HEAD is now at f7e05f4fdea rgw/d4n: Implement bucket check_empty and remove methods 2026-02-24T22:14:11.790 DEBUG:teuthology.orchestra.run.trial046:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-24T22:14:11.836 INFO:tasks.workunit.client.0.trial046.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2026-02-24T22:14:11.837 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-24T22:14:11.837 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2026-02-24T22:14:11.865 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2026-02-24T22:14:11.898 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2026-02-24T22:14:11.923 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-24T22:14:11.924 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-24T22:14:11.924 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2026-02-24T22:14:11.946 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-24T22:14:11.950 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:14:11.950 DEBUG:teuthology.orchestra.run.trial046:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-24T22:14:11.996 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_s3_mp_reupload.sh on client.0... 2026-02-24T22:14:11.998 INFO:tasks.workunit:Running workunit rgw/test_rgw_s3_mp_reupload.sh... 2026-02-24T22:14:11.999 DEBUG:teuthology.orchestra.run.trial046:workunit test rgw/test_rgw_s3_mp_reupload.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7e05f4fdea5af173284f247d6b0935b337d4ccf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_s3_mp_reupload.sh 2026-02-24T22:14:12.048 INFO:tasks.workunit.client.0.trial046.stdout:INFO: fully qualified domain name: trial046 2026-02-24T22:14:12.048 INFO:tasks.workunit.client.0.trial046.stdout:== TESTING *ABORTING* MULTIPART UPLOAD WITH RE-UPLOADS == 2026-02-24T22:14:12.048 INFO:tasks.workunit.client.0.trial046.stdout:== TESTING VERSIONED BUCKET == 2026-02-24T22:14:13.552 INFO:tasks.workunit.client.0.trial046.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2026-02-24T22:14:13.610 INFO:tasks.workunit.client.0.trial046.stdout:Collecting pip 2026-02-24T22:14:13.614 INFO:tasks.workunit.client.0.trial046.stdout: Using cached pip-26.0.1-py3-none-any.whl (1.8 MB) 2026-02-24T22:14:13.630 INFO:tasks.workunit.client.0.trial046.stdout:Installing collected packages: pip 2026-02-24T22:14:13.630 INFO:tasks.workunit.client.0.trial046.stdout: Attempting uninstall: pip 2026-02-24T22:14:13.630 INFO:tasks.workunit.client.0.trial046.stdout: Found existing installation: pip 22.0.2 2026-02-24T22:14:13.803 INFO:tasks.workunit.client.0.trial046.stdout: Uninstalling pip-22.0.2: 2026-02-24T22:14:13.806 INFO:tasks.workunit.client.0.trial046.stdout: Successfully uninstalled pip-22.0.2 2026-02-24T22:14:14.148 INFO:tasks.workunit.client.0.trial046.stdout:Successfully installed pip-26.0.1 2026-02-24T22:14:14.499 INFO:tasks.workunit.client.0.trial046.stdout:Collecting boto3 2026-02-24T22:14:14.500 INFO:tasks.workunit.client.0.trial046.stdout: Using cached boto3-1.42.56-py3-none-any.whl.metadata (6.7 kB) 2026-02-24T22:14:14.626 INFO:tasks.workunit.client.0.trial046.stdout:Collecting botocore<1.43.0,>=1.42.56 (from boto3) 2026-02-24T22:14:14.626 INFO:tasks.workunit.client.0.trial046.stdout: Using cached botocore-1.42.56-py3-none-any.whl.metadata (5.9 kB) 2026-02-24T22:14:14.855 INFO:tasks.workunit.client.0.trial046.stdout:Collecting jmespath<2.0.0,>=0.7.1 (from boto3) 2026-02-24T22:14:14.855 INFO:tasks.workunit.client.0.trial046.stdout: Using cached jmespath-1.1.0-py3-none-any.whl.metadata (7.6 kB) 2026-02-24T22:14:14.867 INFO:tasks.workunit.client.0.trial046.stdout:Collecting s3transfer<0.17.0,>=0.16.0 (from boto3) 2026-02-24T22:14:14.868 INFO:tasks.workunit.client.0.trial046.stdout: Using cached s3transfer-0.16.0-py3-none-any.whl.metadata (1.7 kB) 2026-02-24T22:14:14.880 INFO:tasks.workunit.client.0.trial046.stdout:Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.43.0,>=1.42.56->boto3) 2026-02-24T22:14:14.880 INFO:tasks.workunit.client.0.trial046.stdout: Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) 2026-02-24T22:14:14.900 INFO:tasks.workunit.client.0.trial046.stdout:Collecting urllib3!=2.2.0,<3,>=1.25.4 (from botocore<1.43.0,>=1.42.56->boto3) 2026-02-24T22:14:14.901 INFO:tasks.workunit.client.0.trial046.stdout: Using cached urllib3-2.6.3-py3-none-any.whl.metadata (6.9 kB) 2026-02-24T22:14:14.922 INFO:tasks.workunit.client.0.trial046.stdout:Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.43.0,>=1.42.56->boto3) 2026-02-24T22:14:14.923 INFO:tasks.workunit.client.0.trial046.stdout: Using cached six-1.17.0-py2.py3-none-any.whl.metadata (1.7 kB) 2026-02-24T22:14:14.950 INFO:tasks.workunit.client.0.trial046.stdout:Using cached boto3-1.42.56-py3-none-any.whl (140 kB) 2026-02-24T22:14:14.951 INFO:tasks.workunit.client.0.trial046.stdout:Using cached botocore-1.42.56-py3-none-any.whl (14.6 MB) 2026-02-24T22:14:14.960 INFO:tasks.workunit.client.0.trial046.stdout:Using cached jmespath-1.1.0-py3-none-any.whl (20 kB) 2026-02-24T22:14:14.961 INFO:tasks.workunit.client.0.trial046.stdout:Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2026-02-24T22:14:14.962 INFO:tasks.workunit.client.0.trial046.stdout:Using cached s3transfer-0.16.0-py3-none-any.whl (86 kB) 2026-02-24T22:14:14.962 INFO:tasks.workunit.client.0.trial046.stdout:Using cached urllib3-2.6.3-py3-none-any.whl (131 kB) 2026-02-24T22:14:14.963 INFO:tasks.workunit.client.0.trial046.stdout:Using cached six-1.17.0-py2.py3-none-any.whl (11 kB) 2026-02-24T22:14:14.988 INFO:tasks.workunit.client.0.trial046.stdout:Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3 2026-02-24T22:14:15.314 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:14:15.315 INFO:tasks.workunit.client.0.trial046.stdout:Successfully installed boto3-1.42.56 botocore-1.42.56 jmespath-1.1.0 python-dateutil-2.9.0.post0 s3transfer-0.16.0 six-1.17.0 urllib3-2.6.3 2026-02-24T22:14:15.845 INFO:tasks.workunit.client.0.trial046.stdout:INFO: creating user 2026-02-24T22:14:15.921 INFO:tasks.workunit.client.0.trial046.stdout:{ 2026-02-24T22:14:15.921 INFO:tasks.workunit.client.0.trial046.stdout: "user_id": "testid", 2026-02-24T22:14:15.921 INFO:tasks.workunit.client.0.trial046.stdout: "display_name": "M. Tester", 2026-02-24T22:14:15.921 INFO:tasks.workunit.client.0.trial046.stdout: "email": "tester@ceph.com", 2026-02-24T22:14:15.922 INFO:tasks.workunit.client.0.trial046.stdout: "suspended": 0, 2026-02-24T22:14:15.922 INFO:tasks.workunit.client.0.trial046.stdout: "max_buckets": 1000, 2026-02-24T22:14:15.922 INFO:tasks.workunit.client.0.trial046.stdout: "subusers": [], 2026-02-24T22:14:15.922 INFO:tasks.workunit.client.0.trial046.stdout: "keys": [ 2026-02-24T22:14:15.922 INFO:tasks.workunit.client.0.trial046.stdout: { 2026-02-24T22:14:15.922 INFO:tasks.workunit.client.0.trial046.stdout: "user": "testid", 2026-02-24T22:14:15.922 INFO:tasks.workunit.client.0.trial046.stdout: "access_key": "0555b35654ad1656d804", 2026-02-24T22:14:15.922 INFO:tasks.workunit.client.0.trial046.stdout: "secret_key": "h7GhxuBLTrlhVUyxSPUKUV8r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q==", 2026-02-24T22:14:15.922 INFO:tasks.workunit.client.0.trial046.stdout: "active": true, 2026-02-24T22:14:15.922 INFO:tasks.workunit.client.0.trial046.stdout: "create_date": "2026-02-24T22:14:15.921279Z" 2026-02-24T22:14:15.922 INFO:tasks.workunit.client.0.trial046.stdout: } 2026-02-24T22:14:15.922 INFO:tasks.workunit.client.0.trial046.stdout: ], 2026-02-24T22:14:15.923 INFO:tasks.workunit.client.0.trial046.stdout: "swift_keys": [], 2026-02-24T22:14:15.923 INFO:tasks.workunit.client.0.trial046.stdout: "caps": [], 2026-02-24T22:14:15.923 INFO:tasks.workunit.client.0.trial046.stdout: "op_mask": "read, write, delete", 2026-02-24T22:14:15.923 INFO:tasks.workunit.client.0.trial046.stdout: "default_placement": "", 2026-02-24T22:14:15.923 INFO:tasks.workunit.client.0.trial046.stdout: "default_storage_class": "", 2026-02-24T22:14:15.923 INFO:tasks.workunit.client.0.trial046.stdout: "placement_tags": [], 2026-02-24T22:14:15.923 INFO:tasks.workunit.client.0.trial046.stdout: "bucket_quota": { 2026-02-24T22:14:15.923 INFO:tasks.workunit.client.0.trial046.stdout: "enabled": false, 2026-02-24T22:14:15.923 INFO:tasks.workunit.client.0.trial046.stdout: "check_on_raw": false, 2026-02-24T22:14:15.923 INFO:tasks.workunit.client.0.trial046.stdout: "max_size": -1, 2026-02-24T22:14:15.923 INFO:tasks.workunit.client.0.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:14:15.923 INFO:tasks.workunit.client.0.trial046.stdout: "max_objects": -1 2026-02-24T22:14:15.923 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:14:15.924 INFO:tasks.workunit.client.0.trial046.stdout: "user_quota": { 2026-02-24T22:14:15.924 INFO:tasks.workunit.client.0.trial046.stdout: "enabled": false, 2026-02-24T22:14:15.924 INFO:tasks.workunit.client.0.trial046.stdout: "check_on_raw": false, 2026-02-24T22:14:15.924 INFO:tasks.workunit.client.0.trial046.stdout: "max_size": -1, 2026-02-24T22:14:15.924 INFO:tasks.workunit.client.0.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:14:15.924 INFO:tasks.workunit.client.0.trial046.stdout: "max_objects": -1 2026-02-24T22:14:15.924 INFO:tasks.workunit.client.0.trial046.stdout: }, 2026-02-24T22:14:15.924 INFO:tasks.workunit.client.0.trial046.stdout: "temp_url_keys": [], 2026-02-24T22:14:15.924 INFO:tasks.workunit.client.0.trial046.stdout: "type": "rgw", 2026-02-24T22:14:15.924 INFO:tasks.workunit.client.0.trial046.stdout: "mfa_ids": [], 2026-02-24T22:14:15.924 INFO:tasks.workunit.client.0.trial046.stdout: "account_id": "", 2026-02-24T22:14:15.924 INFO:tasks.workunit.client.0.trial046.stdout: "path": "/", 2026-02-24T22:14:15.924 INFO:tasks.workunit.client.0.trial046.stdout: "create_date": "2026-02-24T22:14:15.921272Z", 2026-02-24T22:14:15.925 INFO:tasks.workunit.client.0.trial046.stdout: "tags": [], 2026-02-24T22:14:15.925 INFO:tasks.workunit.client.0.trial046.stdout: "group_ids": [] 2026-02-24T22:14:15.925 INFO:tasks.workunit.client.0.trial046.stdout:} 2026-02-24T22:14:15.925 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:14:25.925 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'trial046'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:25.925 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:25.935 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'trial046'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:25.935 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:25.952 INFO:tasks.workunit.client.0.trial046.stderr:5+0 records in 2026-02-24T22:14:25.952 INFO:tasks.workunit.client.0.trial046.stderr:5+0 records out 2026-02-24T22:14:25.952 INFO:tasks.workunit.client.0.trial046.stderr:5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.0130399 s, 402 MB/s 2026-02-24T22:14:25.953 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'trial046'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:25.953 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:26.001 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'trial046'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:26.001 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:26.077 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'trial046'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:26.077 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:26.147 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'trial046'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:26.147 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:26.220 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'trial046'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:26.220 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:26.290 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'trial046'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:26.290 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:26.358 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'trial046'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:26.358 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:26.432 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'trial046'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:26.432 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:26.502 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'trial046'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:26.502 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:26.573 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'trial046'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:26.573 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:26.645 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'trial046'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:26.645 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:26.717 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'trial046'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:26.717 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:26.787 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'trial046'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:26.787 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:26.864 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'trial046'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:26.864 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:26.934 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'trial046'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:26.934 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:26.942 INFO:tasks.workunit.client.0.trial046.stdout:bucket nams is reupload-bkt-100895 2026-02-24T22:14:26.942 INFO:tasks.workunit.client.0.trial046.stdout:endpoint is https://trial046:443 2026-02-24T22:14:26.942 INFO:tasks.workunit.client.0.trial046.stdout:start UploadId=2~45uAeJrF45tEptlUawfJZ8Gbozq8JeH 2026-02-24T22:14:26.942 INFO:tasks.workunit.client.0.trial046.stdout:phase 1 uploaded part {'ETag': '"8800ac79301b2b413ec03da879db920c"', 'PartNumber': 1} 2026-02-24T22:14:26.942 INFO:tasks.workunit.client.0.trial046.stdout:phase 1 uploaded part {'ETag': '"8800ac79301b2b413ec03da879db920c"', 'PartNumber': 2} 2026-02-24T22:14:26.943 INFO:tasks.workunit.client.0.trial046.stdout:phase 2 uploaded part {'ETag': '"8800ac79301b2b413ec03da879db920c"', 'PartNumber': 1} 2026-02-24T22:14:26.943 INFO:tasks.workunit.client.0.trial046.stdout:phase 2 uploaded part {'ETag': '"8800ac79301b2b413ec03da879db920c"', 'PartNumber': 1} 2026-02-24T22:14:26.943 INFO:tasks.workunit.client.0.trial046.stdout:phase 2 uploaded part {'ETag': '"8800ac79301b2b413ec03da879db920c"', 'PartNumber': 1} 2026-02-24T22:14:26.943 INFO:tasks.workunit.client.0.trial046.stdout:phase 2 uploaded part {'ETag': '"8800ac79301b2b413ec03da879db920c"', 'PartNumber': 1} 2026-02-24T22:14:26.943 INFO:tasks.workunit.client.0.trial046.stdout:phase 2 uploaded part {'ETag': '"8800ac79301b2b413ec03da879db920c"', 'PartNumber': 1} 2026-02-24T22:14:26.943 INFO:tasks.workunit.client.0.trial046.stdout:phase 2 uploaded part {'ETag': '"8800ac79301b2b413ec03da879db920c"', 'PartNumber': 1} 2026-02-24T22:14:26.943 INFO:tasks.workunit.client.0.trial046.stdout:phase 2 uploaded part {'ETag': '"8800ac79301b2b413ec03da879db920c"', 'PartNumber': 1} 2026-02-24T22:14:26.943 INFO:tasks.workunit.client.0.trial046.stdout:phase 2 uploaded part {'ETag': '"8800ac79301b2b413ec03da879db920c"', 'PartNumber': 1} 2026-02-24T22:14:26.943 INFO:tasks.workunit.client.0.trial046.stdout:phase 2 uploaded part {'ETag': '"8800ac79301b2b413ec03da879db920c"', 'PartNumber': 1} 2026-02-24T22:14:26.943 INFO:tasks.workunit.client.0.trial046.stdout:phase 2 uploaded part {'ETag': '"8800ac79301b2b413ec03da879db920c"', 'PartNumber': 1} 2026-02-24T22:14:26.943 INFO:tasks.workunit.client.0.trial046.stdout:phase 2 uploaded part {'ETag': '"8800ac79301b2b413ec03da879db920c"', 'PartNumber': 1} 2026-02-24T22:14:26.943 INFO:tasks.workunit.client.0.trial046.stdout:aborting multipart upload, parts=[{'ETag': '"8800ac79301b2b413ec03da879db920c"', 'PartNumber': 1}, {'ETag': '"8800ac79301b2b413ec03da879db920c"', 'PartNumber': 2}] 2026-02-24T22:14:27.373 INFO:tasks.workunit.client.0.trial046.stdout:radosgw-admin bucket radoslist: 2026-02-24T22:14:27.631 INFO:tasks.workunit.client.0.trial046.stdout:rados ls: 2026-02-24T22:14:27.636 INFO:tasks.workunit.client.0.trial046.stdout:== PASSED == 2026-02-24T22:14:27.637 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T22:14:27.637 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T22:14:27.688 INFO:tasks.workunit:Stopping ['rgw/test_rgw_s3_mp_reupload.sh'] on client.0... 2026-02-24T22:14:27.688 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-24T22:14:28.105 DEBUG:teuthology.parallel:result is None 2026-02-24T22:14:28.106 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-24T22:14:28.114 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-24T22:14:28.115 DEBUG:teuthology.orchestra.run.trial046:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-24T22:14:28.159 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-24T22:14:28.159 INFO:teuthology.run_tasks:Running task workunit... 2026-02-24T22:14:28.174 INFO:tasks.workunit:Pulling workunits from ref f7e05f4fdea5af173284f247d6b0935b337d4ccf 2026-02-24T22:14:28.175 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-24T22:14:28.175 DEBUG:teuthology.orchestra.run.trial046:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-24T22:14:28.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T22:14:28.207 INFO:teuthology.orchestra.run.trial046.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-24T22:14:28.207 DEBUG:teuthology.orchestra.run.trial046:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-24T22:14:28.255 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-24T22:14:28.255 DEBUG:teuthology.orchestra.run.trial046:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-24T22:14:28.298 INFO:tasks.workunit:timeout=3h 2026-02-24T22:14:28.299 INFO:tasks.workunit:cleanup=True 2026-02-24T22:14:28.299 DEBUG:teuthology.orchestra.run.trial046:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/samarahu/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout f7e05f4fdea5af173284f247d6b0935b337d4ccf 2026-02-24T22:14:28.343 INFO:tasks.workunit.client.0.trial046.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-24T22:14:51.649 INFO:tasks.workunit.client.0.trial046.stderr:Note: switching to 'f7e05f4fdea5af173284f247d6b0935b337d4ccf'. 2026-02-24T22:14:51.649 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:14:51.649 INFO:tasks.workunit.client.0.trial046.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-24T22:14:51.649 INFO:tasks.workunit.client.0.trial046.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-24T22:14:51.649 INFO:tasks.workunit.client.0.trial046.stderr:state without impacting any branches by switching back to a branch. 2026-02-24T22:14:51.649 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:14:51.649 INFO:tasks.workunit.client.0.trial046.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-24T22:14:51.650 INFO:tasks.workunit.client.0.trial046.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-24T22:14:51.650 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:14:51.650 INFO:tasks.workunit.client.0.trial046.stderr: git switch -c 2026-02-24T22:14:51.650 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:14:51.650 INFO:tasks.workunit.client.0.trial046.stderr:Or undo this operation with: 2026-02-24T22:14:51.650 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:14:51.650 INFO:tasks.workunit.client.0.trial046.stderr: git switch - 2026-02-24T22:14:51.650 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:14:51.650 INFO:tasks.workunit.client.0.trial046.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-24T22:14:51.650 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:14:51.650 INFO:tasks.workunit.client.0.trial046.stderr:HEAD is now at f7e05f4fdea rgw/d4n: Implement bucket check_empty and remove methods 2026-02-24T22:14:51.656 DEBUG:teuthology.orchestra.run.trial046:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-24T22:14:51.703 INFO:tasks.workunit.client.0.trial046.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2026-02-24T22:14:51.705 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-24T22:14:51.705 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2026-02-24T22:14:51.735 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2026-02-24T22:14:51.767 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2026-02-24T22:14:51.794 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-24T22:14:51.795 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-24T22:14:51.795 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2026-02-24T22:14:51.817 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-24T22:14:51.821 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:14:51.821 DEBUG:teuthology.orchestra.run.trial046:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-24T22:14:51.868 INFO:tasks.workunit:Running workunits matching rgw/run-rados-pool-quota.sh on client.0... 2026-02-24T22:14:51.870 INFO:tasks.workunit:Running workunit rgw/run-rados-pool-quota.sh... 2026-02-24T22:14:51.871 DEBUG:teuthology.orchestra.run.trial046:workunit test rgw/run-rados-pool-quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7e05f4fdea5af173284f247d6b0935b337d4ccf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-rados-pool-quota.sh 2026-02-24T22:14:51.919 INFO:tasks.workunit.client.0.trial046.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-rados-pool-quota.sh 2026-02-24T22:14:51.920 INFO:tasks.workunit.client.0.trial046.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2026-02-24T22:14:51.920 INFO:tasks.workunit.client.0.trial046.stderr:+ python3 -m venv /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2026-02-24T22:14:53.256 INFO:tasks.workunit.client.0.trial046.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/activate 2026-02-24T22:14:53.256 INFO:tasks.workunit.client.0.trial046.stderr:++ deactivate nondestructive 2026-02-24T22:14:53.256 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n '' ']' 2026-02-24T22:14:53.256 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n '' ']' 2026-02-24T22:14:53.256 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2026-02-24T22:14:53.257 INFO:tasks.workunit.client.0.trial046.stderr:++ hash -r 2026-02-24T22:14:53.257 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n '' ']' 2026-02-24T22:14:53.257 INFO:tasks.workunit.client.0.trial046.stderr:++ unset VIRTUAL_ENV 2026-02-24T22:14:53.257 INFO:tasks.workunit.client.0.trial046.stderr:++ unset VIRTUAL_ENV_PROMPT 2026-02-24T22:14:53.257 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' '!' nondestructive = nondestructive ']' 2026-02-24T22:14:53.257 INFO:tasks.workunit.client.0.trial046.stderr:++ VIRTUAL_ENV=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2026-02-24T22:14:53.257 INFO:tasks.workunit.client.0.trial046.stderr:++ export VIRTUAL_ENV 2026-02-24T22:14:53.257 INFO:tasks.workunit.client.0.trial046.stderr:++ _OLD_VIRTUAL_PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T22:14:53.257 INFO:tasks.workunit.client.0.trial046.stderr:++ PATH=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T22:14:53.257 INFO:tasks.workunit.client.0.trial046.stderr:++ export PATH 2026-02-24T22:14:53.257 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n '' ']' 2026-02-24T22:14:53.257 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -z '' ']' 2026-02-24T22:14:53.258 INFO:tasks.workunit.client.0.trial046.stderr:++ _OLD_VIRTUAL_PS1= 2026-02-24T22:14:53.258 INFO:tasks.workunit.client.0.trial046.stderr:++ PS1='(rgw) ' 2026-02-24T22:14:53.258 INFO:tasks.workunit.client.0.trial046.stderr:++ export PS1 2026-02-24T22:14:53.258 INFO:tasks.workunit.client.0.trial046.stderr:++ VIRTUAL_ENV_PROMPT='(rgw) ' 2026-02-24T22:14:53.258 INFO:tasks.workunit.client.0.trial046.stderr:++ export VIRTUAL_ENV_PROMPT 2026-02-24T22:14:53.258 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2026-02-24T22:14:53.258 INFO:tasks.workunit.client.0.trial046.stderr:++ hash -r 2026-02-24T22:14:53.258 INFO:tasks.workunit.client.0.trial046.stderr:+ pip install pip --upgrade 2026-02-24T22:14:53.432 INFO:tasks.workunit.client.0.trial046.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2026-02-24T22:14:53.491 INFO:tasks.workunit.client.0.trial046.stdout:Collecting pip 2026-02-24T22:14:53.495 INFO:tasks.workunit.client.0.trial046.stdout: Using cached pip-26.0.1-py3-none-any.whl (1.8 MB) 2026-02-24T22:14:53.511 INFO:tasks.workunit.client.0.trial046.stdout:Installing collected packages: pip 2026-02-24T22:14:53.511 INFO:tasks.workunit.client.0.trial046.stdout: Attempting uninstall: pip 2026-02-24T22:14:53.511 INFO:tasks.workunit.client.0.trial046.stdout: Found existing installation: pip 22.0.2 2026-02-24T22:14:53.687 INFO:tasks.workunit.client.0.trial046.stdout: Uninstalling pip-22.0.2: 2026-02-24T22:14:53.690 INFO:tasks.workunit.client.0.trial046.stdout: Successfully uninstalled pip-22.0.2 2026-02-24T22:14:54.038 INFO:tasks.workunit.client.0.trial046.stdout:Successfully installed pip-26.0.1 2026-02-24T22:14:54.067 INFO:tasks.workunit.client.0.trial046.stderr:+ pip install boto3 2026-02-24T22:14:54.392 INFO:tasks.workunit.client.0.trial046.stdout:Collecting boto3 2026-02-24T22:14:54.393 INFO:tasks.workunit.client.0.trial046.stdout: Using cached boto3-1.42.56-py3-none-any.whl.metadata (6.7 kB) 2026-02-24T22:14:54.524 INFO:tasks.workunit.client.0.trial046.stdout:Collecting botocore<1.43.0,>=1.42.56 (from boto3) 2026-02-24T22:14:54.525 INFO:tasks.workunit.client.0.trial046.stdout: Using cached botocore-1.42.56-py3-none-any.whl.metadata (5.9 kB) 2026-02-24T22:14:54.534 INFO:tasks.workunit.client.0.trial046.stdout:Collecting jmespath<2.0.0,>=0.7.1 (from boto3) 2026-02-24T22:14:54.535 INFO:tasks.workunit.client.0.trial046.stdout: Using cached jmespath-1.1.0-py3-none-any.whl.metadata (7.6 kB) 2026-02-24T22:14:54.543 INFO:tasks.workunit.client.0.trial046.stdout:Collecting s3transfer<0.17.0,>=0.16.0 (from boto3) 2026-02-24T22:14:54.543 INFO:tasks.workunit.client.0.trial046.stdout: Using cached s3transfer-0.16.0-py3-none-any.whl.metadata (1.7 kB) 2026-02-24T22:14:54.551 INFO:tasks.workunit.client.0.trial046.stdout:Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.43.0,>=1.42.56->boto3) 2026-02-24T22:14:54.551 INFO:tasks.workunit.client.0.trial046.stdout: Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) 2026-02-24T22:14:54.564 INFO:tasks.workunit.client.0.trial046.stdout:Collecting urllib3!=2.2.0,<3,>=1.25.4 (from botocore<1.43.0,>=1.42.56->boto3) 2026-02-24T22:14:54.564 INFO:tasks.workunit.client.0.trial046.stdout: Using cached urllib3-2.6.3-py3-none-any.whl.metadata (6.9 kB) 2026-02-24T22:14:54.571 INFO:tasks.workunit.client.0.trial046.stdout:Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.43.0,>=1.42.56->boto3) 2026-02-24T22:14:54.572 INFO:tasks.workunit.client.0.trial046.stdout: Using cached six-1.17.0-py2.py3-none-any.whl.metadata (1.7 kB) 2026-02-24T22:14:54.605 INFO:tasks.workunit.client.0.trial046.stdout:Using cached boto3-1.42.56-py3-none-any.whl (140 kB) 2026-02-24T22:14:54.606 INFO:tasks.workunit.client.0.trial046.stdout:Using cached botocore-1.42.56-py3-none-any.whl (14.6 MB) 2026-02-24T22:14:54.615 INFO:tasks.workunit.client.0.trial046.stdout:Using cached jmespath-1.1.0-py3-none-any.whl (20 kB) 2026-02-24T22:14:54.615 INFO:tasks.workunit.client.0.trial046.stdout:Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2026-02-24T22:14:54.616 INFO:tasks.workunit.client.0.trial046.stdout:Using cached s3transfer-0.16.0-py3-none-any.whl (86 kB) 2026-02-24T22:14:54.616 INFO:tasks.workunit.client.0.trial046.stdout:Using cached urllib3-2.6.3-py3-none-any.whl (131 kB) 2026-02-24T22:14:54.617 INFO:tasks.workunit.client.0.trial046.stdout:Using cached six-1.17.0-py2.py3-none-any.whl (11 kB) 2026-02-24T22:14:54.643 INFO:tasks.workunit.client.0.trial046.stdout:Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3 2026-02-24T22:14:54.975 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:14:54.976 INFO:tasks.workunit.client.0.trial046.stdout:Successfully installed boto3-1.42.56 botocore-1.42.56 jmespath-1.1.0 python-dateutil-2.9.0.post0 s3transfer-0.16.0 six-1.17.0 urllib3-2.6.3 2026-02-24T22:14:55.017 INFO:tasks.workunit.client.0.trial046.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/python3 /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_rados_pool_quota.py 2026-02-24T22:14:55.088 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin user create --uid quota-tester --display-name "Quota Tester" --access-key LTA662PVVDTDWX6M2AB0 --secret pvtchqajgzqx5581t6qbddbkj0bgf3a69qdkjcea 2026-02-24T22:14:55.105 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:14:55.105 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:14:55.124 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.124+0000 7f4825b1aa80 4 Realm: () 2026-02-24T22:14:55.124 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.124+0000 7f4825b1aa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:14:55.124 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.124+0000 7f4825b1aa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:14:55.124 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.124+0000 7f4825b1aa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:14:55.124 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.124+0000 7f4825b1aa80 20 zonegroup default 2026-02-24T22:14:55.125 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.124+0000 7f4825b1aa80 20 rados->read ofs=0 len=0 2026-02-24T22:14:55.126 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.124+0000 7f4825b1aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:14:55.126 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.124+0000 7f4825b1aa80 20 rados->read ofs=0 len=0 2026-02-24T22:14:55.126 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.124+0000 7f4825b1aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:14:55.126 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.124+0000 7f4825b1aa80 20 started sync module instance, tier type = 2026-02-24T22:14:55.126 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.124+0000 7f4825b1aa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:14:55.127 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.128+0000 7f481effd640 0 -- 10.20.193.46:0/3440425973 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x5607a7926070 msgr2=0x5607a79464f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:14:55.128 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.128+0000 7f4825b1aa80 20 add_watcher() i=3 2026-02-24T22:14:55.128 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.128+0000 7f4825b1aa80 20 add_watcher() i=1 2026-02-24T22:14:55.128 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.128+0000 7f4825b1aa80 20 add_watcher() i=0 2026-02-24T22:14:55.129 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.128+0000 7f4825b1aa80 20 add_watcher() i=6 2026-02-24T22:14:55.129 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.128+0000 7f4825b1aa80 20 add_watcher() i=2 2026-02-24T22:14:55.129 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.128+0000 7f4825b1aa80 20 add_watcher() i=5 2026-02-24T22:14:55.129 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.128+0000 7f4825b1aa80 20 add_watcher() i=4 2026-02-24T22:14:55.334 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.332+0000 7f4825b1aa80 20 add_watcher() i=7 2026-02-24T22:14:55.334 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.332+0000 7f4825b1aa80 2 all 8 watchers are set, enabling cache 2026-02-24T22:14:55.337 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.336+0000 7f4825b1aa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:14:55.337 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.336+0000 7f4825b1aa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:14:55.337 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.336+0000 7f4825b1aa80 5 note: GC not initialized 2026-02-24T22:14:55.337 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.336+0000 7f473bfff640 20 reqs_thread_entry: start 2026-02-24T22:14:55.338 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.336+0000 7f481f7fe640 0 -- 10.20.193.46:0/156729612 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5607a780c370 msgr2=0x5607a780c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:14:55.579 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.576+0000 7f4825b1aa80 20 init_complete bucket index max shards: 1 2026-02-24T22:14:55.579 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.576+0000 7f4825b1aa80 20 Filter name: none 2026-02-24T22:14:55.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.576+0000 7f473a7fc640 20 reqs_thread_entry: start 2026-02-24T22:14:55.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.576+0000 7f4825b1aa80 10 cache get: name=default.rgw.meta+users.uid+quota-tester : miss 2026-02-24T22:14:55.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.576+0000 7f4825b1aa80 20 rados->read ofs=0 len=0 2026-02-24T22:14:55.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.576+0000 7f4825b1aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:14:55.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.576+0000 7f4825b1aa80 10 cache put: name=default.rgw.meta+users.uid+quota-tester info.flags=0x0 2026-02-24T22:14:55.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.576+0000 7f4825b1aa80 10 adding default.rgw.meta+users.uid+quota-tester to cache LRU end 2026-02-24T22:14:55.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.576+0000 7f4825b1aa80 10 cache get: name=default.rgw.meta+users.keys+LTA662PVVDTDWX6M2AB0 : miss 2026-02-24T22:14:55.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.576+0000 7f4825b1aa80 20 rados->read ofs=0 len=0 2026-02-24T22:14:55.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.580+0000 7f4825b1aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:14:55.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.580+0000 7f4825b1aa80 10 cache put: name=default.rgw.meta+users.keys+LTA662PVVDTDWX6M2AB0 info.flags=0x0 2026-02-24T22:14:55.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.580+0000 7f4825b1aa80 10 adding default.rgw.meta+users.keys+LTA662PVVDTDWX6M2AB0 to cache LRU end 2026-02-24T22:14:55.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.580+0000 7f4825b1aa80 10 cache get: name=default.rgw.meta+users.keys+LTA662PVVDTDWX6M2AB0 : hit (negative entry) 2026-02-24T22:14:55.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.580+0000 7f4825b1aa80 10 cache get: name=default.rgw.meta+users.keys+LTA662PVVDTDWX6M2AB0 : hit (negative entry) 2026-02-24T22:14:55.582 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.580+0000 7f4825b1aa80 10 cache put: name=default.rgw.meta+users.uid+quota-tester info.flags=0x17 2026-02-24T22:14:55.582 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.580+0000 7f4825b1aa80 10 moving default.rgw.meta+users.uid+quota-tester to cache LRU end 2026-02-24T22:14:55.582 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.580+0000 7f4825b1aa80 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:quota-tester, ofs0, ns] 2026-02-24T22:14:55.582 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.580+0000 7f47fe7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1258425417832 cookie 94590874928960 notifier 5810 bl.length()=592 2026-02-24T22:14:55.582 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.580+0000 7f47fe7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+quota-tester info.flags=0x17 2026-02-24T22:14:55.582 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.580+0000 7f47fe7fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+quota-tester to cache LRU end 2026-02-24T22:14:55.582 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.580+0000 7f4825b1aa80 10 cache put: name=default.rgw.meta+users.keys+LTA662PVVDTDWX6M2AB0 info.flags=0x7 2026-02-24T22:14:55.582 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.580+0000 7f4825b1aa80 10 moving default.rgw.meta+users.keys+LTA662PVVDTDWX6M2AB0 to cache LRU end 2026-02-24T22:14:55.582 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.580+0000 7f4825b1aa80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:users.keys:LTA662PVVDTDWX6M2AB0, ofs0, ns] 2026-02-24T22:14:55.583 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.580+0000 7f47fdffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1258425417739 cookie 94590875333760 notifier 5810 bl.length()=186 2026-02-24T22:14:55.583 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.580+0000 7f47fdffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+LTA662PVVDTDWX6M2AB0 info.flags=0x7 2026-02-24T22:14:55.583 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.580+0000 7f47fdffb640 10 rgw watcher librados: moving default.rgw.meta+users.keys+LTA662PVVDTDWX6M2AB0 to cache LRU end 2026-02-24T22:14:55.585 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.584+0000 7f4825b1aa80 20 remove_watcher() i=3 2026-02-24T22:14:55.585 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.584+0000 7f4825b1aa80 2 removed watcher, disabling cache 2026-02-24T22:14:55.586 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.584+0000 7f4825b1aa80 20 remove_watcher() i=0 2026-02-24T22:14:55.586 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.584+0000 7f4825b1aa80 20 remove_watcher() i=6 2026-02-24T22:14:55.586 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.584+0000 7f4825b1aa80 20 remove_watcher() i=4 2026-02-24T22:14:55.586 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.584+0000 7f4825b1aa80 20 remove_watcher() i=5 2026-02-24T22:14:55.586 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.584+0000 7f4825b1aa80 20 remove_watcher() i=1 2026-02-24T22:14:55.586 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.584+0000 7f4825b1aa80 20 remove_watcher() i=7 2026-02-24T22:14:55.586 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:14:55.584+0000 7f4825b1aa80 20 remove_watcher() i=2 2026-02-24T22:14:55.664 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:55.664 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:55.708 INFO:tasks.workunit.client.0.trial046.stderr:ceph osd pool set-quota default.rgw.buckets.data max_objects 0 2026-02-24T22:14:57.770 INFO:tasks.workunit.client.0.trial046.stderr:set-quota max_objects = 0 for pool default.rgw.buckets.data 2026-02-24T22:14:57.786 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:57.786 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:57.790 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:57.790 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:57.803 INFO:tasks.workunit.client.0.trial046.stderr:TEST: verify that objects can be deleted with rados pool at quota 2026-02-24T22:14:57.805 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:57.806 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:57.815 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:57.815 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:57.821 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:57.821 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:57.827 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:57.827 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:57.833 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:57.833 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:57.839 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:57.839 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:57.845 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:57.845 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:57.851 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:57.851 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:57.858 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:57.858 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:57.864 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:14:57.864 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:14:57.870 INFO:tasks.workunit.client.0.trial046.stderr:ceph osd pool set-quota default.rgw.buckets.data max_objects 1 2026-02-24T22:14:59.777 INFO:tasks.workunit.client.0.trial046.stderr:set-quota max_objects = 1 for pool default.rgw.buckets.data 2026-02-24T22:14:59.792 INFO:tasks.workunit.client.0.trial046.stderr:forcing quota to propagate 2026-02-24T22:15:09.802 INFO:tasks.workunit.client.0.trial046.stderr:rados -p default.rgw.buckets.data ls 2026-02-24T22:15:09.814 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:15:09.814 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:15:09.840 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:15:09.839+0000 7f5825cf2640 0 -- 10.20.193.46:0/1420127092 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x7f5810090b50 msgr2=0x7f581007f970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:15:09.844 INFO:tasks.workunit.client.0.trial046.stderr:forced quota to propagate 2026-02-24T22:15:09.845 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:15:09.845 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:15:09.893 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:15:09.893 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:15:09.899 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:15:09.899 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:15:09.904 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:15:09.904 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:15:09.910 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:15:09.910 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:15:09.916 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:15:09.916 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:15:09.921 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:15:09.921 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:15:09.926 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:15:09.927 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:15:09.932 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:15:09.932 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:15:09.937 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:15:09.937 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:15:09.943 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:15:09.943 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:15:09.947 INFO:tasks.workunit.client.0.trial046.stderr:rados -p default.rgw.buckets.data ls | grep afae9b4f-b302-4e9f-bbe8-3e359a17e53f 2026-02-24T22:15:09.960 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:15:09.961 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:15:09.984 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:15:09.983+0000 7f1366f46640 0 -- 10.20.193.46:0/3254526650 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x7f13500f3600 msgr2=0x7f13501198a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:15:09.991 INFO:tasks.workunit.client.0.trial046.stderr:ceph osd pool set-quota default.rgw.buckets.data max_objects 0 2026-02-24T22:15:11.790 INFO:tasks.workunit.client.0.trial046.stderr:set-quota max_objects = 0 for pool default.rgw.buckets.data 2026-02-24T22:15:11.803 INFO:tasks.workunit.client.0.trial046.stderr:Completed rados pool quota tests 2026-02-24T22:15:11.804 INFO:tasks.workunit.client.0.trial046.stdout:Could not connect to the endpoint URL: "http://localhost:80/" 2026-02-24T22:15:11.804 INFO:tasks.workunit.client.0.trial046.stdout:Could not connect to the endpoint URL: "http://localhost:8000/" 2026-02-24T22:15:11.804 INFO:tasks.workunit.client.0.trial046.stdout:connected to https://localhost:443 2026-02-24T22:15:11.842 INFO:tasks.workunit.client.0.trial046.stderr:+ deactivate 2026-02-24T22:15:11.842 INFO:tasks.workunit.client.0.trial046.stderr:+ '[' -n /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin ']' 2026-02-24T22:15:11.842 INFO:tasks.workunit.client.0.trial046.stderr:+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T22:15:11.842 INFO:tasks.workunit.client.0.trial046.stderr:+ export PATH 2026-02-24T22:15:11.842 INFO:tasks.workunit.client.0.trial046.stderr:+ unset _OLD_VIRTUAL_PATH 2026-02-24T22:15:11.842 INFO:tasks.workunit.client.0.trial046.stderr:+ '[' -n '' ']' 2026-02-24T22:15:11.842 INFO:tasks.workunit.client.0.trial046.stderr:+ '[' -n /usr/bin/bash -o -n '' ']' 2026-02-24T22:15:11.842 INFO:tasks.workunit.client.0.trial046.stderr:+ hash -r 2026-02-24T22:15:11.842 INFO:tasks.workunit.client.0.trial046.stderr:+ '[' -n '' ']' 2026-02-24T22:15:11.842 INFO:tasks.workunit.client.0.trial046.stderr:+ unset VIRTUAL_ENV 2026-02-24T22:15:11.843 INFO:tasks.workunit.client.0.trial046.stderr:+ unset VIRTUAL_ENV_PROMPT 2026-02-24T22:15:11.843 INFO:tasks.workunit.client.0.trial046.stderr:+ '[' '!' '' = nondestructive ']' 2026-02-24T22:15:11.843 INFO:tasks.workunit.client.0.trial046.stderr:+ unset -f deactivate 2026-02-24T22:15:11.843 INFO:tasks.workunit.client.0.trial046.stderr:+ echo OK. 2026-02-24T22:15:11.844 INFO:tasks.workunit.client.0.trial046.stdout:OK. 2026-02-24T22:15:11.844 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T22:15:11.844 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T22:15:11.895 INFO:tasks.workunit:Stopping ['rgw/run-rados-pool-quota.sh'] on client.0... 2026-02-24T22:15:11.896 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-24T22:15:12.315 DEBUG:teuthology.parallel:result is None 2026-02-24T22:15:12.316 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-24T22:15:12.324 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-24T22:15:12.324 DEBUG:teuthology.orchestra.run.trial046:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-24T22:15:12.370 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-24T22:15:12.371 INFO:teuthology.run_tasks:Running task ragweed... 2026-02-24T22:15:12.395 DEBUG:tasks.ragweed:ragweed config is {'client.0': {'default-branch': 'ceph-master', 'rgw_server': 'client.0', 'stages': 'prepare,check'}} 2026-02-24T22:15:12.395 INFO:tasks.ragweed:Downloading ragweed... 2026-02-24T22:15:12.395 INFO:tasks.ragweed:Using branch 'wip-d4n-remove-bucket' for ragweed 2026-02-24T22:15:12.396 DEBUG:teuthology.orchestra.run.trial046:> git clone -b wip-d4n-remove-bucket https://git.ceph.com/ragweed.git /home/ubuntu/cephtest/ragweed.client.0 2026-02-24T22:15:12.414 INFO:teuthology.orchestra.run.trial046.stderr:Cloning into '/home/ubuntu/cephtest/ragweed.client.0'... 2026-02-24T22:15:12.464 INFO:teuthology.orchestra.run.trial046.stderr:fatal: Remote branch wip-d4n-remove-bucket not found in upstream origin 2026-02-24T22:15:12.465 DEBUG:teuthology.orchestra.run:got remote process result: 128 2026-02-24T22:15:12.467 INFO:tasks.ragweed:Using branch 'ceph-master' for ragweed 2026-02-24T22:15:12.467 DEBUG:teuthology.orchestra.run.trial046:> git clone -b ceph-master https://git.ceph.com/ragweed.git /home/ubuntu/cephtest/ragweed.client.0 2026-02-24T22:15:12.514 INFO:teuthology.orchestra.run.trial046.stderr:Cloning into '/home/ubuntu/cephtest/ragweed.client.0'... 2026-02-24T22:15:12.585 INFO:tasks.ragweed:Creating rgw users... 2026-02-24T22:15:12.586 DEBUG:tasks.ragweed:Creating user ragweed.client.0 on client.0 2026-02-24T22:15:12.586 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid ragweed.client.0 --display-name 'Mr. ragweed.client.0' --access-key IVLSXKBUNCVSHAJHFPEJ --secret luy2E/mYgJjnexQCwz8xxcn5S0jSnY5aDDG+gy/AJE0WreGp/daNlg== --email ragweed.client.0+test@test.test --system false 2026-02-24T22:15:12.648 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:15:12.648 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:15:12.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.663+0000 7f28d7d5aa80 4 Realm: () 2026-02-24T22:15:12.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.663+0000 7f28d7d5aa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:15:12.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.663+0000 7f28d7d5aa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:15:12.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.663+0000 7f28d7d5aa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:15:12.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.663+0000 7f28d7d5aa80 20 zonegroup default 2026-02-24T22:15:12.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.663+0000 7f28d7d5aa80 20 rados->read ofs=0 len=0 2026-02-24T22:15:12.666 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.663+0000 7f28d7d5aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:15:12.666 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.663+0000 7f28d7d5aa80 20 rados->read ofs=0 len=0 2026-02-24T22:15:12.667 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.663+0000 7f28d7d5aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:15:12.667 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.663+0000 7f28d7d5aa80 20 started sync module instance, tier type = 2026-02-24T22:15:12.667 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.663+0000 7f28d7d5aa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:15:12.668 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.667+0000 7f28d602f640 0 -- 10.20.193.46:0/1155783825 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55bfd995f590 msgr2=0x55bfd995fa00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:15:12.669 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.667+0000 7f28d7d5aa80 20 add_watcher() i=6 2026-02-24T22:15:12.669 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.667+0000 7f28d7d5aa80 20 add_watcher() i=1 2026-02-24T22:15:12.669 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.667+0000 7f28d7d5aa80 20 add_watcher() i=3 2026-02-24T22:15:12.670 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.667+0000 7f28d7d5aa80 20 add_watcher() i=5 2026-02-24T22:15:12.670 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.667+0000 7f28d7d5aa80 20 add_watcher() i=0 2026-02-24T22:15:12.670 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.667+0000 7f28d7d5aa80 20 add_watcher() i=2 2026-02-24T22:15:12.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.667+0000 7f28d7d5aa80 20 add_watcher() i=7 2026-02-24T22:15:12.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.667+0000 7f28d7d5aa80 20 add_watcher() i=4 2026-02-24T22:15:12.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.667+0000 7f28d7d5aa80 2 all 8 watchers are set, enabling cache 2026-02-24T22:15:12.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.671+0000 7f28d7d5aa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:15:12.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.671+0000 7f28d7d5aa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:15:12.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.671+0000 7f28d7d5aa80 5 note: GC not initialized 2026-02-24T22:15:12.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.671+0000 7f28067fc640 20 reqs_thread_entry: start 2026-02-24T22:15:12.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.671+0000 7f28d602f640 0 -- 10.20.193.46:0/1441544723 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55bfd9b231a0 msgr2=0x55bfd9b23610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:15:12.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.911+0000 7f28d7d5aa80 20 init_complete bucket index max shards: 1 2026-02-24T22:15:12.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.911+0000 7f28d7d5aa80 20 Filter name: none 2026-02-24T22:15:12.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.911+0000 7f2804ff9640 20 reqs_thread_entry: start 2026-02-24T22:15:12.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.911+0000 7f28d7d5aa80 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2026-02-24T22:15:12.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.911+0000 7f28d7d5aa80 20 rados->read ofs=0 len=0 2026-02-24T22:15:12.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.911+0000 7f28d7d5aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:15:12.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.911+0000 7f28d7d5aa80 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x0 2026-02-24T22:15:12.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.911+0000 7f28d7d5aa80 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2026-02-24T22:15:12.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.911+0000 7f28d7d5aa80 10 cache get: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test : miss 2026-02-24T22:15:12.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.911+0000 7f28d7d5aa80 20 rados->read ofs=0 len=0 2026-02-24T22:15:12.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.911+0000 7f28d7d5aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:15:12.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.911+0000 7f28d7d5aa80 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x0 2026-02-24T22:15:12.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.911+0000 7f28d7d5aa80 10 adding default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2026-02-24T22:15:12.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.911+0000 7f28d7d5aa80 10 cache get: name=default.rgw.meta+users.keys+IVLSXKBUNCVSHAJHFPEJ : miss 2026-02-24T22:15:12.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.911+0000 7f28d7d5aa80 20 rados->read ofs=0 len=0 2026-02-24T22:15:12.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.911+0000 7f28d7d5aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:15:12.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.911+0000 7f28d7d5aa80 10 cache put: name=default.rgw.meta+users.keys+IVLSXKBUNCVSHAJHFPEJ info.flags=0x0 2026-02-24T22:15:12.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.911+0000 7f28d7d5aa80 10 adding default.rgw.meta+users.keys+IVLSXKBUNCVSHAJHFPEJ to cache LRU end 2026-02-24T22:15:12.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.911+0000 7f28d7d5aa80 10 cache get: name=default.rgw.meta+users.keys+IVLSXKBUNCVSHAJHFPEJ : hit (negative entry) 2026-02-24T22:15:12.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.911+0000 7f28d7d5aa80 10 cache get: name=default.rgw.meta+users.keys+IVLSXKBUNCVSHAJHFPEJ : hit (negative entry) 2026-02-24T22:15:12.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.911+0000 7f28d7d5aa80 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2026-02-24T22:15:12.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.911+0000 7f28d7d5aa80 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2026-02-24T22:15:12.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.911+0000 7f28d7d5aa80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ragweed.client.0, ofs0, ns] 2026-02-24T22:15:12.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.915+0000 7f28b0ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1292785156220 cookie 94282477618016 notifier 5852 bl.length()=675 2026-02-24T22:15:12.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.915+0000 7f28b0ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2026-02-24T22:15:12.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.915+0000 7f28b0ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2026-02-24T22:15:12.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.915+0000 7f28b0ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1292785156221 cookie 94282477618016 notifier 5852 bl.length()=675 2026-02-24T22:15:12.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.915+0000 7f28b0ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2026-02-24T22:15:12.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.915+0000 7f28b0ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2026-02-24T22:15:12.918 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.915+0000 7f28d7d5aa80 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2026-02-24T22:15:12.918 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.915+0000 7f28d7d5aa80 10 moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2026-02-24T22:15:12.918 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.915+0000 7f28d7d5aa80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.email:ragweed.client.0+test@test.test, ofs0, ns] 2026-02-24T22:15:12.918 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.915+0000 7f28b0ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1292785156112 cookie 94282477605904 notifier 5852 bl.length()=202 2026-02-24T22:15:12.919 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.915+0000 7f28b0ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2026-02-24T22:15:12.919 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.915+0000 7f28b0ff9640 10 rgw watcher librados: moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2026-02-24T22:15:12.920 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.919+0000 7f28d7d5aa80 10 cache put: name=default.rgw.meta+users.keys+IVLSXKBUNCVSHAJHFPEJ info.flags=0x7 2026-02-24T22:15:12.920 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.919+0000 7f28d7d5aa80 10 moving default.rgw.meta+users.keys+IVLSXKBUNCVSHAJHFPEJ to cache LRU end 2026-02-24T22:15:12.920 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.919+0000 7f28d7d5aa80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:users.keys:IVLSXKBUNCVSHAJHFPEJ, ofs0, ns] 2026-02-24T22:15:12.920 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.919+0000 7f28b17fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1292785156108 cookie 94282477634976 notifier 5852 bl.length()=190 2026-02-24T22:15:12.920 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.919+0000 7f28b17fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+IVLSXKBUNCVSHAJHFPEJ info.flags=0x7 2026-02-24T22:15:12.920 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.919+0000 7f28b17fa640 10 rgw watcher librados: moving default.rgw.meta+users.keys+IVLSXKBUNCVSHAJHFPEJ to cache LRU end 2026-02-24T22:15:12.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.919+0000 7f28b0ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1292785156109 cookie 94282477634976 notifier 5852 bl.length()=190 2026-02-24T22:15:12.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.919+0000 7f28b0ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+IVLSXKBUNCVSHAJHFPEJ info.flags=0x7 2026-02-24T22:15:12.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.919+0000 7f28b0ff9640 10 rgw watcher librados: moving default.rgw.meta+users.keys+IVLSXKBUNCVSHAJHFPEJ to cache LRU end 2026-02-24T22:15:12.922 INFO:teuthology.orchestra.run.trial046.stdout:{ 2026-02-24T22:15:12.922 INFO:teuthology.orchestra.run.trial046.stdout: "user_id": "ragweed.client.0", 2026-02-24T22:15:12.922 INFO:teuthology.orchestra.run.trial046.stdout: "display_name": "Mr. ragweed.client.0", 2026-02-24T22:15:12.922 INFO:teuthology.orchestra.run.trial046.stdout: "email": "ragweed.client.0+test@test.test", 2026-02-24T22:15:12.922 INFO:teuthology.orchestra.run.trial046.stdout: "suspended": 0, 2026-02-24T22:15:12.922 INFO:teuthology.orchestra.run.trial046.stdout: "max_buckets": 1000, 2026-02-24T22:15:12.922 INFO:teuthology.orchestra.run.trial046.stdout: "subusers": [], 2026-02-24T22:15:12.922 INFO:teuthology.orchestra.run.trial046.stdout: "keys": [ 2026-02-24T22:15:12.922 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:15:12.923 INFO:teuthology.orchestra.run.trial046.stdout: "user": "ragweed.client.0", 2026-02-24T22:15:12.923 INFO:teuthology.orchestra.run.trial046.stdout: "access_key": "IVLSXKBUNCVSHAJHFPEJ", 2026-02-24T22:15:12.923 INFO:teuthology.orchestra.run.trial046.stdout: "secret_key": "luy2E/mYgJjnexQCwz8xxcn5S0jSnY5aDDG+gy/AJE0WreGp/daNlg==", 2026-02-24T22:15:12.923 INFO:teuthology.orchestra.run.trial046.stdout: "active": true, 2026-02-24T22:15:12.923 INFO:teuthology.orchestra.run.trial046.stdout: "create_date": "2026-02-24T22:15:12.918577Z" 2026-02-24T22:15:12.923 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:15:12.923 INFO:teuthology.orchestra.run.trial046.stdout: ], 2026-02-24T22:15:12.923 INFO:teuthology.orchestra.run.trial046.stdout: "swift_keys": [], 2026-02-24T22:15:12.923 INFO:teuthology.orchestra.run.trial046.stdout: "caps": [], 2026-02-24T22:15:12.923 INFO:teuthology.orchestra.run.trial046.stdout: "op_mask": "read, write, delete", 2026-02-24T22:15:12.923 INFO:teuthology.orchestra.run.trial046.stdout: "default_placement": "", 2026-02-24T22:15:12.924 INFO:teuthology.orchestra.run.trial046.stdout: "default_storage_class": "", 2026-02-24T22:15:12.924 INFO:teuthology.orchestra.run.trial046.stdout: "placement_tags": [], 2026-02-24T22:15:12.924 INFO:teuthology.orchestra.run.trial046.stdout: "bucket_quota": { 2026-02-24T22:15:12.924 INFO:teuthology.orchestra.run.trial046.stdout: "enabled": false, 2026-02-24T22:15:12.924 INFO:teuthology.orchestra.run.trial046.stdout: "check_on_raw": false, 2026-02-24T22:15:12.924 INFO:teuthology.orchestra.run.trial046.stdout: "max_size": -1, 2026-02-24T22:15:12.924 INFO:teuthology.orchestra.run.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:15:12.924 INFO:teuthology.orchestra.run.trial046.stdout: "max_objects": -1 2026-02-24T22:15:12.924 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:15:12.924 INFO:teuthology.orchestra.run.trial046.stdout: "user_quota": { 2026-02-24T22:15:12.924 INFO:teuthology.orchestra.run.trial046.stdout: "enabled": false, 2026-02-24T22:15:12.924 INFO:teuthology.orchestra.run.trial046.stdout: "check_on_raw": false, 2026-02-24T22:15:12.925 INFO:teuthology.orchestra.run.trial046.stdout: "max_size": -1, 2026-02-24T22:15:12.925 INFO:teuthology.orchestra.run.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:15:12.925 INFO:teuthology.orchestra.run.trial046.stdout: "max_objects": -1 2026-02-24T22:15:12.925 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:15:12.925 INFO:teuthology.orchestra.run.trial046.stdout: "temp_url_keys": [], 2026-02-24T22:15:12.925 INFO:teuthology.orchestra.run.trial046.stdout: "type": "rgw", 2026-02-24T22:15:12.925 INFO:teuthology.orchestra.run.trial046.stdout: "mfa_ids": [], 2026-02-24T22:15:12.925 INFO:teuthology.orchestra.run.trial046.stdout: "account_id": "", 2026-02-24T22:15:12.925 INFO:teuthology.orchestra.run.trial046.stdout: "path": "/", 2026-02-24T22:15:12.925 INFO:teuthology.orchestra.run.trial046.stdout: "create_date": "2026-02-24T22:15:12.918566Z", 2026-02-24T22:15:12.925 INFO:teuthology.orchestra.run.trial046.stdout: "tags": [], 2026-02-24T22:15:12.925 INFO:teuthology.orchestra.run.trial046.stdout: "group_ids": [] 2026-02-24T22:15:12.926 INFO:teuthology.orchestra.run.trial046.stdout:} 2026-02-24T22:15:12.926 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:15:12.927 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.923+0000 7f28d7d5aa80 20 remove_watcher() i=0 2026-02-24T22:15:12.927 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.923+0000 7f28d7d5aa80 2 removed watcher, disabling cache 2026-02-24T22:15:12.927 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.923+0000 7f28d7d5aa80 20 remove_watcher() i=3 2026-02-24T22:15:12.927 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.923+0000 7f28d7d5aa80 20 remove_watcher() i=6 2026-02-24T22:15:12.927 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.923+0000 7f28d7d5aa80 20 remove_watcher() i=5 2026-02-24T22:15:12.927 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.923+0000 7f28d7d5aa80 20 remove_watcher() i=7 2026-02-24T22:15:12.927 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.923+0000 7f28d7d5aa80 20 remove_watcher() i=1 2026-02-24T22:15:12.927 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.923+0000 7f28d7d5aa80 20 remove_watcher() i=2 2026-02-24T22:15:12.928 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:12.923+0000 7f28d7d5aa80 20 remove_watcher() i=4 2026-02-24T22:15:12.933 DEBUG:tasks.ragweed:Creating user sysuser.client.0 on client.0 2026-02-24T22:15:12.933 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid sysuser.client.0 --display-name 'Mr. sysuser.client.0' --access-key YVXCPDVBWJHBKZOBYKCH --secret rYeBt+OTPB9JBLoSDwU+ovS20xAc0vcN9TV0hCU4pZ0hZs084eALvw== --email sysuser.client.0+test@test.test --system true 2026-02-24T22:15:12.996 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:15:12.996 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:15:13.012 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.011+0000 7fcdb8eeaa80 4 Realm: () 2026-02-24T22:15:13.012 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.011+0000 7fcdb8eeaa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:15:13.012 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.011+0000 7fcdb8eeaa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:15:13.012 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.011+0000 7fcdb8eeaa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:15:13.013 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.011+0000 7fcdb8eeaa80 20 zonegroup default 2026-02-24T22:15:13.013 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.011+0000 7fcdb8eeaa80 20 rados->read ofs=0 len=0 2026-02-24T22:15:13.014 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.011+0000 7fcdb8eeaa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:15:13.014 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.011+0000 7fcdb8eeaa80 20 rados->read ofs=0 len=0 2026-02-24T22:15:13.015 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.011+0000 7fcdb8eeaa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:15:13.015 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.011+0000 7fcdb8eeaa80 20 started sync module instance, tier type = 2026-02-24T22:15:13.015 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.011+0000 7fcdb8eeaa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:15:13.015 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.015+0000 7fcdb69be640 0 -- 10.20.193.46:0/56491044 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x555b03db7c80 msgr2=0x555b03dd8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:15:13.016 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.015+0000 7fcdb8eeaa80 20 add_watcher() i=1 2026-02-24T22:15:13.017 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.015+0000 7fcdb8eeaa80 20 add_watcher() i=0 2026-02-24T22:15:13.017 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.015+0000 7fcdb8eeaa80 20 add_watcher() i=3 2026-02-24T22:15:13.017 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.015+0000 7fcdb8eeaa80 20 add_watcher() i=5 2026-02-24T22:15:13.017 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.015+0000 7fcdb8eeaa80 20 add_watcher() i=6 2026-02-24T22:15:13.017 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.015+0000 7fcdb8eeaa80 20 add_watcher() i=2 2026-02-24T22:15:13.018 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.015+0000 7fcdb8eeaa80 20 add_watcher() i=4 2026-02-24T22:15:13.223 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.219+0000 7fcdb8eeaa80 20 add_watcher() i=7 2026-02-24T22:15:13.223 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.219+0000 7fcdb8eeaa80 2 all 8 watchers are set, enabling cache 2026-02-24T22:15:13.223 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.223+0000 7fcdb61bd640 0 -- 10.20.193.46:0/1134025458 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7fccd8008af0 msgr2=0x7fccd8028fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:15:13.425 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.427+0000 7fcdb8eeaa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:15:13.425 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.427+0000 7fcdb8eeaa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:15:13.425 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.427+0000 7fcdb8eeaa80 5 note: GC not initialized 2026-02-24T22:15:13.425 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.427+0000 7fcd64fe1640 20 reqs_thread_entry: start 2026-02-24T22:15:13.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.455+0000 7fcdb8eeaa80 20 init_complete bucket index max shards: 1 2026-02-24T22:15:13.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.455+0000 7fcdb8eeaa80 20 Filter name: none 2026-02-24T22:15:13.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fccceffd640 20 reqs_thread_entry: start 2026-02-24T22:15:13.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcdb8eeaa80 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2026-02-24T22:15:13.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcdb8eeaa80 20 rados->read ofs=0 len=0 2026-02-24T22:15:13.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcdb8eeaa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:15:13.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcdb8eeaa80 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x0 2026-02-24T22:15:13.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcdb8eeaa80 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2026-02-24T22:15:13.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcdb8eeaa80 10 cache get: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test : miss 2026-02-24T22:15:13.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcdb8eeaa80 20 rados->read ofs=0 len=0 2026-02-24T22:15:13.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcdb8eeaa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:15:13.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcdb8eeaa80 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x0 2026-02-24T22:15:13.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcdb8eeaa80 10 adding default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2026-02-24T22:15:13.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcdb8eeaa80 10 cache get: name=default.rgw.meta+users.keys+YVXCPDVBWJHBKZOBYKCH : miss 2026-02-24T22:15:13.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcdb8eeaa80 20 rados->read ofs=0 len=0 2026-02-24T22:15:13.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcdb8eeaa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:15:13.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcdb8eeaa80 10 cache put: name=default.rgw.meta+users.keys+YVXCPDVBWJHBKZOBYKCH info.flags=0x0 2026-02-24T22:15:13.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcdb8eeaa80 10 adding default.rgw.meta+users.keys+YVXCPDVBWJHBKZOBYKCH to cache LRU end 2026-02-24T22:15:13.462 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcdb8eeaa80 10 cache get: name=default.rgw.meta+users.keys+YVXCPDVBWJHBKZOBYKCH : hit (negative entry) 2026-02-24T22:15:13.462 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcdb8eeaa80 10 cache get: name=default.rgw.meta+users.keys+YVXCPDVBWJHBKZOBYKCH : hit (negative entry) 2026-02-24T22:15:13.463 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcdb8eeaa80 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2026-02-24T22:15:13.463 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcdb8eeaa80 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2026-02-24T22:15:13.463 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcdb8eeaa80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:sysuser.client.0, ofs0, ns] 2026-02-24T22:15:13.463 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcd927fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1292785156113 cookie 93849395246752 notifier 5867 bl.length()=675 2026-02-24T22:15:13.463 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcd927fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2026-02-24T22:15:13.463 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcd927fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2026-02-24T22:15:13.463 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcdb8eeaa80 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2026-02-24T22:15:13.463 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcdb8eeaa80 10 moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2026-02-24T22:15:13.463 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcdb8eeaa80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.email:sysuser.client.0+test@test.test, ofs0, ns] 2026-02-24T22:15:13.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcd92ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1292785156222 cookie 93849392845264 notifier 5867 bl.length()=202 2026-02-24T22:15:13.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcd92ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2026-02-24T22:15:13.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.459+0000 7fcd92ffd640 10 rgw watcher librados: moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2026-02-24T22:15:13.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.463+0000 7fcdb8eeaa80 10 cache put: name=default.rgw.meta+users.keys+YVXCPDVBWJHBKZOBYKCH info.flags=0x7 2026-02-24T22:15:13.465 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.463+0000 7fcdb8eeaa80 10 moving default.rgw.meta+users.keys+YVXCPDVBWJHBKZOBYKCH to cache LRU end 2026-02-24T22:15:13.465 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.463+0000 7fcdb8eeaa80 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.keys:YVXCPDVBWJHBKZOBYKCH, ofs0, ns] 2026-02-24T22:15:13.465 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.463+0000 7fcd927fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1292785156201 cookie 93849395235776 notifier 5867 bl.length()=190 2026-02-24T22:15:13.465 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.463+0000 7fcd927fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+YVXCPDVBWJHBKZOBYKCH info.flags=0x7 2026-02-24T22:15:13.465 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.463+0000 7fcd927fc640 10 rgw watcher librados: moving default.rgw.meta+users.keys+YVXCPDVBWJHBKZOBYKCH to cache LRU end 2026-02-24T22:15:13.465 INFO:teuthology.orchestra.run.trial046.stdout:{ 2026-02-24T22:15:13.465 INFO:teuthology.orchestra.run.trial046.stdout: "user_id": "sysuser.client.0", 2026-02-24T22:15:13.465 INFO:teuthology.orchestra.run.trial046.stdout: "display_name": "Mr. sysuser.client.0", 2026-02-24T22:15:13.466 INFO:teuthology.orchestra.run.trial046.stdout: "email": "sysuser.client.0+test@test.test", 2026-02-24T22:15:13.466 INFO:teuthology.orchestra.run.trial046.stdout: "suspended": 0, 2026-02-24T22:15:13.466 INFO:teuthology.orchestra.run.trial046.stdout: "max_buckets": 1000, 2026-02-24T22:15:13.466 INFO:teuthology.orchestra.run.trial046.stdout: "subusers": [], 2026-02-24T22:15:13.466 INFO:teuthology.orchestra.run.trial046.stdout: "keys": [ 2026-02-24T22:15:13.466 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:15:13.466 INFO:teuthology.orchestra.run.trial046.stdout: "user": "sysuser.client.0", 2026-02-24T22:15:13.466 INFO:teuthology.orchestra.run.trial046.stdout: "access_key": "YVXCPDVBWJHBKZOBYKCH", 2026-02-24T22:15:13.466 INFO:teuthology.orchestra.run.trial046.stdout: "secret_key": "rYeBt+OTPB9JBLoSDwU+ovS20xAc0vcN9TV0hCU4pZ0hZs084eALvw==", 2026-02-24T22:15:13.466 INFO:teuthology.orchestra.run.trial046.stdout: "active": true, 2026-02-24T22:15:13.466 INFO:teuthology.orchestra.run.trial046.stdout: "create_date": "2026-02-24T22:15:13.464617Z" 2026-02-24T22:15:13.467 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:15:13.467 INFO:teuthology.orchestra.run.trial046.stdout: ], 2026-02-24T22:15:13.467 INFO:teuthology.orchestra.run.trial046.stdout: "swift_keys": [], 2026-02-24T22:15:13.467 INFO:teuthology.orchestra.run.trial046.stdout: "caps": [], 2026-02-24T22:15:13.467 INFO:teuthology.orchestra.run.trial046.stdout: "op_mask": "read, write, delete", 2026-02-24T22:15:13.467 INFO:teuthology.orchestra.run.trial046.stdout: "system": true, 2026-02-24T22:15:13.467 INFO:teuthology.orchestra.run.trial046.stdout: "default_placement": "", 2026-02-24T22:15:13.467 INFO:teuthology.orchestra.run.trial046.stdout: "default_storage_class": "", 2026-02-24T22:15:13.467 INFO:teuthology.orchestra.run.trial046.stdout: "placement_tags": [], 2026-02-24T22:15:13.467 INFO:teuthology.orchestra.run.trial046.stdout: "bucket_quota": { 2026-02-24T22:15:13.467 INFO:teuthology.orchestra.run.trial046.stdout: "enabled": false, 2026-02-24T22:15:13.467 INFO:teuthology.orchestra.run.trial046.stdout: "check_on_raw": false, 2026-02-24T22:15:13.468 INFO:teuthology.orchestra.run.trial046.stdout: "max_size": -1, 2026-02-24T22:15:13.468 INFO:teuthology.orchestra.run.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:15:13.468 INFO:teuthology.orchestra.run.trial046.stdout: "max_objects": -1 2026-02-24T22:15:13.468 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:15:13.468 INFO:teuthology.orchestra.run.trial046.stdout: "user_quota": { 2026-02-24T22:15:13.468 INFO:teuthology.orchestra.run.trial046.stdout: "enabled": false, 2026-02-24T22:15:13.468 INFO:teuthology.orchestra.run.trial046.stdout: "check_on_raw": false, 2026-02-24T22:15:13.468 INFO:teuthology.orchestra.run.trial046.stdout: "max_size": -1, 2026-02-24T22:15:13.468 INFO:teuthology.orchestra.run.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:15:13.468 INFO:teuthology.orchestra.run.trial046.stdout: "max_objects": -1 2026-02-24T22:15:13.468 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:15:13.468 INFO:teuthology.orchestra.run.trial046.stdout: "temp_url_keys": [], 2026-02-24T22:15:13.468 INFO:teuthology.orchestra.run.trial046.stdout: "type": "rgw", 2026-02-24T22:15:13.468 INFO:teuthology.orchestra.run.trial046.stdout: "mfa_ids": [], 2026-02-24T22:15:13.469 INFO:teuthology.orchestra.run.trial046.stdout: "account_id": "", 2026-02-24T22:15:13.469 INFO:teuthology.orchestra.run.trial046.stdout: "path": "/", 2026-02-24T22:15:13.469 INFO:teuthology.orchestra.run.trial046.stdout: "create_date": "2026-02-24T22:15:13.464606Z", 2026-02-24T22:15:13.469 INFO:teuthology.orchestra.run.trial046.stdout: "tags": [], 2026-02-24T22:15:13.469 INFO:teuthology.orchestra.run.trial046.stdout: "group_ids": [] 2026-02-24T22:15:13.469 INFO:teuthology.orchestra.run.trial046.stdout:} 2026-02-24T22:15:13.469 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:15:13.470 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.467+0000 7fcdb8eeaa80 20 remove_watcher() i=3 2026-02-24T22:15:13.470 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.467+0000 7fcdb8eeaa80 2 removed watcher, disabling cache 2026-02-24T22:15:13.470 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.467+0000 7fcdb8eeaa80 20 remove_watcher() i=6 2026-02-24T22:15:13.471 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.467+0000 7fcdb8eeaa80 20 remove_watcher() i=1 2026-02-24T22:15:13.471 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.467+0000 7fcdb8eeaa80 20 remove_watcher() i=2 2026-02-24T22:15:13.471 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.467+0000 7fcdb8eeaa80 20 remove_watcher() i=7 2026-02-24T22:15:13.471 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.467+0000 7fcdb8eeaa80 20 remove_watcher() i=0 2026-02-24T22:15:13.471 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.467+0000 7fcdb8eeaa80 20 remove_watcher() i=5 2026-02-24T22:15:13.471 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:13.467+0000 7fcdb8eeaa80 20 remove_watcher() i=4 2026-02-24T22:15:13.474 INFO:tasks.ragweed:Configuring ragweed... 2026-02-24T22:15:13.475 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:15:13.475 DEBUG:teuthology.orchestra.run.trial046:> dd of=/home/ubuntu/cephtest/archive/ragweed.client.0.conf 2026-02-24T22:15:13.519 INFO:tasks.ragweed:Configuring boto... 2026-02-24T22:15:13.519 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:15:13.519 DEBUG:teuthology.orchestra.run.trial046:> dd of=/home/ubuntu/cephtest/boto.cfg 2026-02-24T22:15:13.567 DEBUG:teuthology.orchestra.run.trial046:ragweed tests against rgw> source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/ragweed.client.0 && RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg tox --sitepackages -- -v -m 'not fails_on_rgw' 2026-02-24T22:15:13.855 INFO:teuthology.orchestra.run.trial046.stdout:py: install_deps> python -I -m pip install -r requirements.txt 2026-02-24T22:15:14.695 INFO:teuthology.orchestra.run.trial046.stdout:.pkg: install_requires> python -I -m pip install 'setuptools>=61.0' 2026-02-24T22:15:15.061 INFO:teuthology.orchestra.run.trial046.stdout:.pkg: _optional_hooks> python /home/ubuntu/cephtest/tox-venv/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta 2026-02-24T22:15:15.276 INFO:teuthology.orchestra.run.trial046.stdout:.pkg: get_requires_for_build_sdist> python /home/ubuntu/cephtest/tox-venv/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta 2026-02-24T22:15:15.507 INFO:teuthology.orchestra.run.trial046.stdout:.pkg: build_sdist> python /home/ubuntu/cephtest/tox-venv/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta 2026-02-24T22:15:15.698 INFO:teuthology.orchestra.run.trial046.stdout:py: install_package_deps> python -I -m pip install PyYAML boto gevent isodate munch 2026-02-24T22:15:16.000 INFO:teuthology.orchestra.run.trial046.stdout:py: install_package> python -I -m pip install --force-reinstall --no-deps /home/ubuntu/cephtest/ragweed.client.0/.tox/.tmp/package/1/ragweed-0.0.1.tar.gz 2026-02-24T22:15:17.043 INFO:teuthology.orchestra.run.trial046.stdout:py: commands[0]> python -m pytest -v -m 'not fails_on_rgw' 2026-02-24T22:15:17.116 INFO:teuthology.orchestra.run.trial046.stdout:============================= test session starts ============================== 2026-02-24T22:15:17.116 INFO:teuthology.orchestra.run.trial046.stdout:platform linux -- Python 3.10.12, pytest-6.2.5, py-1.10.0, pluggy-0.13.0 -- /home/ubuntu/cephtest/ragweed.client.0/.tox/py/bin/python 2026-02-24T22:15:17.116 INFO:teuthology.orchestra.run.trial046.stdout:cachedir: .tox/py/.pytest_cache 2026-02-24T22:15:17.116 INFO:teuthology.orchestra.run.trial046.stdout:rootdir: /home/ubuntu/cephtest/ragweed.client.0, configfile: pytest.ini 2026-02-24T22:15:17.185 INFO:teuthology.orchestra.run.trial046.stdout:collecting ... collected 9 items 2026-02-24T22:15:17.185 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:15:18.801 INFO:teuthology.orchestra.run.trial046.stdout:ragweed/tests/tests.py::r_test_small_obj_data::test PASSED [ 11%] 2026-02-24T22:15:19.494 INFO:teuthology.orchestra.run.trial046.stdout:ragweed/tests/tests.py::r_test_multipart_simple::test PASSED [ 22%] 2026-02-24T22:15:20.136 INFO:teuthology.orchestra.run.trial046.stdout:ragweed/tests/tests.py::r_test_multipart_index_versioning::test PASSED [ 33%] 2026-02-24T22:15:20.820 INFO:teuthology.orchestra.run.trial046.stdout:ragweed/tests/tests.py::r_test_multipart_defer_complete::test PASSED [ 44%] 2026-02-24T22:15:21.517 INFO:teuthology.orchestra.run.trial046.stdout:ragweed/tests/tests.py::r_test_multipart_defer_update_complete::test PASSED [ 55%] 2026-02-24T22:15:27.691 INFO:teuthology.orchestra.run.trial046.stdout:ragweed/tests/tests.py::r_test_obj_storage_class::test PASSED [ 66%] 2026-02-24T22:15:29.168 INFO:teuthology.orchestra.run.trial046.stdout:ragweed/tests/tests.py::r_test_obj_storage_class_multipart::test PASSED [ 77%] 2026-02-24T22:15:31.139 INFO:teuthology.orchestra.run.trial046.stdout:ragweed/tests/tests.py::r_test_obj_storage_class_copy::test PASSED [ 88%] 2026-02-24T22:15:34.011 INFO:teuthology.orchestra.run.trial046.stdout:ragweed/tests/tests.py::r_test_obj_storage_class_multipart_copy::test PASSED [100%] 2026-02-24T22:15:34.011 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:15:34.012 INFO:teuthology.orchestra.run.trial046.stdout:=============================== warnings summary =============================== 2026-02-24T22:15:34.012 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/boto/plugin.py:40 2026-02-24T22:15:34.012 INFO:teuthology.orchestra.run.trial046.stdout: /home/ubuntu/cephtest/ragweed.client.0/.tox/py/lib/python3.10/site-packages/boto/plugin.py:40: DeprecationWarning: the imp module is deprecated in favour of importlib and slated for removal in Python 3.12; see the module's documentation for alternative uses 2026-02-24T22:15:34.012 INFO:teuthology.orchestra.run.trial046.stdout: import imp 2026-02-24T22:15:34.012 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:15:34.012 INFO:teuthology.orchestra.run.trial046.stdout:-- Docs: https://docs.pytest.org/en/stable/warnings.html 2026-02-24T22:15:34.012 INFO:teuthology.orchestra.run.trial046.stdout:======================== 9 passed, 1 warning in 16.90s ========================= 2026-02-24T22:15:34.035 INFO:teuthology.orchestra.run.trial046.stdout: py: OK (20.34=setup[3.35]+cmd[16.99] seconds) 2026-02-24T22:15:34.035 INFO:teuthology.orchestra.run.trial046.stdout: congratulations :) (20.35 seconds) 2026-02-24T22:15:34.051 INFO:tasks.ragweed:Cleaning up boto... 2026-02-24T22:15:34.051 DEBUG:teuthology.orchestra.run.trial046:> rm -f /home/ubuntu/cephtest/boto.cfg 2026-02-24T22:15:34.056 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid ragweed.client.0 --purge-data 2026-02-24T22:15:34.116 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:15:34.116 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:15:34.335 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.339+0000 7f29cf777a80 4 Realm: () 2026-02-24T22:15:34.336 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.339+0000 7f29cf777a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:15:34.336 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.339+0000 7f29cf777a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:15:34.336 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.339+0000 7f29cf777a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:15:34.336 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.339+0000 7f29cf777a80 20 zonegroup default 2026-02-24T22:15:34.336 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.339+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.337 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.339+0000 7f29cf777a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:15:34.337 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.339+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.338 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.339+0000 7f29cf777a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:15:34.338 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.339+0000 7f29cf777a80 20 started sync module instance, tier type = 2026-02-24T22:15:34.338 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.339+0000 7f29cf777a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:15:34.339 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.343+0000 7f29cca4a640 0 -- 10.20.193.46:0/4094398572 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x55dc9a3bf930 msgr2=0x55dc9a3bfda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:15:34.340 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.343+0000 7f29cf777a80 20 add_watcher() i=1 2026-02-24T22:15:34.340 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.343+0000 7f29cf777a80 20 add_watcher() i=6 2026-02-24T22:15:34.340 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.343+0000 7f29cf777a80 20 add_watcher() i=3 2026-02-24T22:15:34.341 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.343+0000 7f29cf777a80 20 add_watcher() i=5 2026-02-24T22:15:34.341 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.343+0000 7f29cf777a80 20 add_watcher() i=0 2026-02-24T22:15:34.341 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.343+0000 7f29cf777a80 20 add_watcher() i=7 2026-02-24T22:15:34.341 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.343+0000 7f29cf777a80 20 add_watcher() i=2 2026-02-24T22:15:34.342 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.343+0000 7f29cf777a80 20 add_watcher() i=4 2026-02-24T22:15:34.342 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.343+0000 7f29cf777a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:15:34.342 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.343+0000 7f29cca4a640 0 -- 10.20.193.46:0/2840741800 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f28ec008af0 msgr2=0x7f28ec028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:15:34.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.547+0000 7f29cf777a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:15:34.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.547+0000 7f29cf777a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:15:34.572 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.575+0000 7f297a7e4640 20 reqs_thread_entry: start 2026-02-24T22:15:34.607 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 20 init_complete bucket index max shards: 1 2026-02-24T22:15:34.608 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 20 Filter name: none 2026-02-24T22:15:34.608 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f2978fe1640 20 reqs_thread_entry: start 2026-02-24T22:15:34.608 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2026-02-24T22:15:34.608 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.608 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 20 rados_obj.operate() r=0 bl.length=486 2026-02-24T22:15:34.609 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x13 2026-02-24T22:15:34.609 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2026-02-24T22:15:34.609 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:15:34.609 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506 : miss 2026-02-24T22:15:34.609 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 20 rados_obj.operate() r=0 bl.length=440 2026-02-24T22:15:34.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506 info.flags=0x17 2026-02-24T22:15:34.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-conf:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506 to cache LRU end 2026-02-24T22:15:34.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:15:34.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 10 updating xattr: name=user.rgw.acl bl.length()=209 2026-02-24T22:15:34.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-conf:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506 2026-02-24T22:15:34.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 10 cls_bucket_list_unordered: got tests/r_test_multipart_defer_complete 2026-02-24T22:15:34.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 10 cls_bucket_list_unordered: got tests/r_test_multipart_defer_update_complete 2026-02-24T22:15:34.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 10 cls_bucket_list_unordered: got tests/r_test_multipart_index_versioning 2026-02-24T22:15:34.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 10 cls_bucket_list_unordered: got tests/r_test_multipart_simple 2026-02-24T22:15:34.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class 2026-02-24T22:15:34.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class_copy 2026-02-24T22:15:34.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart 2026-02-24T22:15:34.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart_copy 2026-02-24T22:15:34.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 10 cls_bucket_list_unordered: got tests/r_test_small_obj_data 2026-02-24T22:15:34.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a56e910 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x55dc9a57c1e8 s->prefetch_data=0 2026-02-24T22:15:34.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 10 manifest: total_size = 5275 2026-02-24T22:15:34.612 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14580771443633498059 2026-02-24T22:15:34.612 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.611+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506.0 2026-02-24T22:15:34.612 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.615+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a56e910 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x55dc9a57c1e8 s->prefetch_data=0 2026-02-24T22:15:34.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.615+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.615+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=5275 stripe_ofs=5275 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.615+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14580771443633498059 2026-02-24T22:15:34.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.615+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.615+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506__shadow_.XKxo8uUMDUo-InHXqbai9FtIq0UvkWQ_0 2026-02-24T22:15:34.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.615+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 335 2026-02-24T22:15:34.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.615+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.615+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.20tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14580771443633498059 2026-02-24T22:15:34.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.615+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_defer_complete 2026-02-24T22:15:34.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.615+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a56e910 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x55dc9a57c0e8 s->prefetch_data=0 2026-02-24T22:15:34.614 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.615+0000 7f29cf777a80 10 manifest: total_size = 5057 2026-02-24T22:15:34.614 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.615+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10995261666208502301 2026-02-24T22:15:34.614 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.615+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506.0 2026-02-24T22:15:34.615 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.615+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a56e910 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x55dc9a57c0e8 s->prefetch_data=0 2026-02-24T22:15:34.615 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.615+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.615 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.615+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=5057 stripe_ofs=5057 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.615 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.615+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10995261666208502301 2026-02-24T22:15:34.615 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.615+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.615 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.615+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506__shadow_.qE75D0gZJzHACOSf5Kud3X72AsgZOrF_0 2026-02-24T22:15:34.615 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.615+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 335 2026-02-24T22:15:34.615 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.615+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.615 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.615+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.13tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10995261666208502301 2026-02-24T22:15:34.616 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.619+0000 7f2997fff640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_defer_update_complete 2026-02-24T22:15:34.616 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.619+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a56e910 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x55dc9a57bbe8 s->prefetch_data=0 2026-02-24T22:15:34.616 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.619+0000 7f29cf777a80 10 manifest: total_size = 13054 2026-02-24T22:15:34.616 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.619+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3246403053267063527 2026-02-24T22:15:34.616 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.619+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506.0 2026-02-24T22:15:34.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.619+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a56e910 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x55dc9a57bbe8 s->prefetch_data=0 2026-02-24T22:15:34.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.619+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.619+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=13054 stripe_ofs=13054 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.619+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3246403053267063527 2026-02-24T22:15:34.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.619+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.619+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506__shadow_.-n722HI8LwjS-atk1OepQ7HDrxQWHgl_0 2026-02-24T22:15:34.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.619+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 334 2026-02-24T22:15:34.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.619+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.619+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.12tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3246403053267063527 2026-02-24T22:15:34.619 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.619+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_index_versioning 2026-02-24T22:15:34.619 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.619+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a56e910 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x55dc9a4a5298 s->prefetch_data=0 2026-02-24T22:15:34.619 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.623+0000 7f29cf777a80 10 manifest: total_size = 13057 2026-02-24T22:15:34.619 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.623+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.852628313330632671 2026-02-24T22:15:34.619 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.623+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506.0 2026-02-24T22:15:34.621 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.623+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a56e910 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x55dc9a4a5298 s->prefetch_data=0 2026-02-24T22:15:34.621 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.623+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.621 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.623+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=13057 stripe_ofs=13057 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.621 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.623+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.852628313330632671 2026-02-24T22:15:34.621 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.623+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.621 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.623+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506__shadow_.kRtf1G5_yWIFNoAWmj_mAn1ou7HB-yn_0 2026-02-24T22:15:34.621 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.623+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 333 2026-02-24T22:15:34.621 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.623+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.621 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.623+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.2tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.852628313330632671 2026-02-24T22:15:34.622 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.623+0000 7f2997fff640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_simple 2026-02-24T22:15:34.622 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.623+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a56e910 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x55dc9a4a5298 s->prefetch_data=0 2026-02-24T22:15:34.622 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.623+0000 7f29cf777a80 10 manifest: total_size = 13793 2026-02-24T22:15:34.622 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.623+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11492922603019535792 2026-02-24T22:15:34.622 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.623+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506.0 2026-02-24T22:15:34.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.627+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a56e910 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x55dc9a4a5298 s->prefetch_data=0 2026-02-24T22:15:34.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.627+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.627+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=13793 stripe_ofs=13793 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.627+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11492922603019535792 2026-02-24T22:15:34.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.627+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.627+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506__shadow_.oKPbSpKRnPnnn0nvf11HifYFmNwFH_N_0 2026-02-24T22:15:34.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.627+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 335 2026-02-24T22:15:34.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.627+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.627+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.25tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11492922603019535792 2026-02-24T22:15:34.625 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.627+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class 2026-02-24T22:15:34.625 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.627+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a56e910 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x55dc9a4a5298 s->prefetch_data=0 2026-02-24T22:15:34.625 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.627+0000 7f29cf777a80 10 manifest: total_size = 16148 2026-02-24T22:15:34.625 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.627+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14210244994683381690 2026-02-24T22:15:34.625 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.627+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506.0 2026-02-24T22:15:34.627 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.627+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a56e910 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x55dc9a4a5298 s->prefetch_data=0 2026-02-24T22:15:34.627 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.627+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.627 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.627+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=16148 stripe_ofs=16148 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.627 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.627+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14210244994683381690 2026-02-24T22:15:34.627 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.627+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.627 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.627+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506__shadow_.JixsIATasPZSVbPoHz513AK0FF1w3lG_0 2026-02-24T22:15:34.627 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.627+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 335 2026-02-24T22:15:34.627 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.627+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.627 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.627+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.16tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14210244994683381690 2026-02-24T22:15:34.628 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.631+0000 7f2997fff640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class_copy 2026-02-24T22:15:34.628 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.631+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a56e910 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x55dc9a4a5298 s->prefetch_data=0 2026-02-24T22:15:34.628 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.631+0000 7f29cf777a80 10 manifest: total_size = 5450 2026-02-24T22:15:34.628 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.631+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1044560560734127319 2026-02-24T22:15:34.628 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.631+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506.0 2026-02-24T22:15:34.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.631+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a56e910 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x55dc9a4a5298 s->prefetch_data=0 2026-02-24T22:15:34.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.631+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.631+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=5450 stripe_ofs=5450 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.631+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1044560560734127319 2026-02-24T22:15:34.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.631+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.631+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506__shadow_.t8VES_JBHmVWgluUpXu3wfjP4xUnpqp_0 2026-02-24T22:15:34.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.631+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 334 2026-02-24T22:15:34.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.631+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.631+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.13tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1044560560734127319 2026-02-24T22:15:34.631 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.631+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class_multipart 2026-02-24T22:15:34.631 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.631+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a56e910 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x55dc9a4a5298 s->prefetch_data=0 2026-02-24T22:15:34.631 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.635+0000 7f29cf777a80 10 manifest: total_size = 16180 2026-02-24T22:15:34.631 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.635+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9413642548407789474 2026-02-24T22:15:34.631 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.635+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506.0 2026-02-24T22:15:34.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.635+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a56e910 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x55dc9a4a5298 s->prefetch_data=0 2026-02-24T22:15:34.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.635+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.633 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.635+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=16180 stripe_ofs=16180 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.633 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.635+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9413642548407789474 2026-02-24T22:15:34.633 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.635+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.633 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.635+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506__shadow_.ub6QlRpEa6_Iap2pmdfBUlE6y26uQVV_0 2026-02-24T22:15:34.633 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.635+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 334 2026-02-24T22:15:34.633 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.635+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.633 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.635+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.6tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9413642548407789474 2026-02-24T22:15:34.634 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.635+0000 7f2997fff640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class_multipart_copy 2026-02-24T22:15:34.634 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.635+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a57f240 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x55dc9a4a5298 s->prefetch_data=0 2026-02-24T22:15:34.634 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.635+0000 7f29cf777a80 10 manifest: total_size = 8895 2026-02-24T22:15:34.634 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.635+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17135096283144166018 2026-02-24T22:15:34.634 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.635+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506.0 2026-02-24T22:15:34.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.639+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a57f240 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x55dc9a4a5298 s->prefetch_data=0 2026-02-24T22:15:34.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.639+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.639+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=8895 stripe_ofs=8895 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.639+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17135096283144166018 2026-02-24T22:15:34.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.639+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.639+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506__shadow_.pDOXUX8a_f8l6cGjUX2faN9weGnPaQZ_0 2026-02-24T22:15:34.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.639+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 335 2026-02-24T22:15:34.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.639+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.639+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.8tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17135096283144166018 2026-02-24T22:15:34.637 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.639+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=tests/r_test_small_obj_data 2026-02-24T22:15:34.637 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.639+0000 7f29cf777a80 20 list_objects_ordered: starting attempt 1 2026-02-24T22:15:34.637 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.639+0000 7f29cf777a80 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2026-02-24T22:15:34.638 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.639+0000 7f29cf777a80 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2026-02-24T22:15:34.638 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.639+0000 7f29cf777a80 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2026-02-24T22:15:34.638 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.639+0000 7f29cf777a80 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2026-02-24T22:15:34.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.639+0000 7f29cf777a80 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2026-02-24T22:15:34.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.639+0000 7f29cf777a80 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-conf[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506])"; uploads.size()=0; is_truncated=0 2026-02-24T22:15:34.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.643+0000 7f29cf777a80 10 cache get: name=default.rgw.meta+root+test-client.0-conf : miss 2026-02-24T22:15:34.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.643+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.643+0000 7f29cf777a80 20 rados_obj.operate() r=0 bl.length=210 2026-02-24T22:15:34.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.643+0000 7f29cf777a80 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x11 2026-02-24T22:15:34.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.643+0000 7f29cf777a80 10 adding default.rgw.meta+root+test-client.0-conf to cache LRU end 2026-02-24T22:15:34.641 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.643+0000 7f29cf777a80 10 removing default.rgw.meta+root+test-client.0-conf from cache 2026-02-24T22:15:34.641 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.643+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-conf, ofs0, ns] 2026-02-24T22:15:34.641 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.643+0000 7f2997fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961312 cookie 94405970386160 notifier 6228 bl.length()=162 2026-02-24T22:15:34.642 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.643+0000 7f29cf777a80 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-conf:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506 from cache 2026-02-24T22:15:34.642 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.643+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-conf:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506, ofs0, ns] 2026-02-24T22:15:34.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.643+0000 7f29a8ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961382 cookie 94405968749408 notifier 6228 bl.length()=222 2026-02-24T22:15:34.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.647+0000 7f29cf777a80 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-conf/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506 : miss 2026-02-24T22:15:34.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.647+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.647+0000 7f29cf777a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:15:34.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.647+0000 7f29cf777a80 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-conf/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506 info.flags=0x0 2026-02-24T22:15:34.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.647+0000 7f29cf777a80 10 adding default.rgw.log++pubsub..bucket.test-client.0-conf/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506 to cache LRU end 2026-02-24T22:15:34.645 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.647+0000 7f29cf777a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1512 : miss 2026-02-24T22:15:34.645 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.647+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.646 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.647+0000 7f29cf777a80 20 rados_obj.operate() r=0 bl.length=469 2026-02-24T22:15:34.646 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.647+0000 7f29cf777a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1512 info.flags=0x17 2026-02-24T22:15:34.646 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.647+0000 7f29cf777a80 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1512 to cache LRU end 2026-02-24T22:15:34.646 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.647+0000 7f29cf777a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:15:34.646 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.647+0000 7f29cf777a80 10 updating xattr: name=user.rgw.acl bl.length()=209 2026-02-24T22:15:34.646 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.647+0000 7f29cf777a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1512 2026-02-24T22:15:34.646 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.647+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo 2026-02-24T22:15:34.647 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.647+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a5820b0 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x55dc9a57d098 s->prefetch_data=0 2026-02-24T22:15:34.647 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.647+0000 7f29cf777a80 10 manifest: total_size = 886297 2026-02-24T22:15:34.647 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.647+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7954056344309016737 2026-02-24T22:15:34.647 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.647+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1512.0 2026-02-24T22:15:34.648 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a5820b0 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x55dc9a57d098 s->prefetch_data=0 2026-02-24T22:15:34.648 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2026-02-24T22:15:34.649 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=246247 2026-02-24T22:15:34.649 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=246247 stripe_ofs=246247 part_ofs=246247 rule->part_size=246247 2026-02-24T22:15:34.649 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2026-02-24T22:15:34.649 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6537703 part_ofs=246247 rule->part_size=246247 2026-02-24T22:15:34.649 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=492494 stripe_ofs=492494 part_ofs=492494 rule->part_size=246247 2026-02-24T22:15:34.649 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2026-02-24T22:15:34.649 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6783950 part_ofs=492494 rule->part_size=246247 2026-02-24T22:15:34.649 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=738741 stripe_ofs=738741 part_ofs=738741 rule->part_size=147556 2026-02-24T22:15:34.649 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=147556 rules.size()=2 2026-02-24T22:15:34.649 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=7030197 part_ofs=738741 rule->part_size=147556 2026-02-24T22:15:34.649 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=886297 stripe_ofs=886297 part_ofs=886297 rule->part_size=147556 2026-02-24T22:15:34.649 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7954056344309016737 2026-02-24T22:15:34.650 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.650 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1512__multipart_foo.2~GzM7_FGELiRE827ZarTvcbTNPjj0TKX.1 2026-02-24T22:15:34.650 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 350 2026-02-24T22:15:34.650 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1512__multipart_foo.2~GzM7_FGELiRE827ZarTvcbTNPjj0TKX.2 2026-02-24T22:15:34.650 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 600 2026-02-24T22:15:34.650 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1512__multipart_foo.2~GzM7_FGELiRE827ZarTvcbTNPjj0TKX.3 2026-02-24T22:15:34.650 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 850 2026-02-24T22:15:34.650 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1512__multipart_foo.2~GzM7_FGELiRE827ZarTvcbTNPjj0TKX.4 2026-02-24T22:15:34.650 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1100 2026-02-24T22:15:34.650 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.650 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.24tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7954056344309016737 2026-02-24T22:15:34.650 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f2997fff640 20 handle_completion(): completion ok for obj=foo 2026-02-24T22:15:34.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 list_objects_ordered: starting attempt 1 2026-02-24T22:15:34.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2026-02-24T22:15:34.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2026-02-24T22:15:34.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2026-02-24T22:15:34.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2026-02-24T22:15:34.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2026-02-24T22:15:34.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-complete.1[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1512])"; uploads.size()=0; is_truncated=0 2026-02-24T22:15:34.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : miss 2026-02-24T22:15:34.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 20 rados_obj.operate() r=0 bl.length=239 2026-02-24T22:15:34.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x11 2026-02-24T22:15:34.653 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.651+0000 7f29cf777a80 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2026-02-24T22:15:34.653 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.655+0000 7f29cf777a80 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 from cache 2026-02-24T22:15:34.653 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.655+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-defer-complete.1, ofs0, ns] 2026-02-24T22:15:34.653 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.655+0000 7f29a8ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959985 cookie 94405970382176 notifier 6228 bl.length()=191 2026-02-24T22:15:34.653 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.655+0000 7f29cf777a80 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1512 from cache 2026-02-24T22:15:34.653 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.655+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1512, ofs0, ns] 2026-02-24T22:15:34.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.655+0000 7f2997fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959890 cookie 94405968768752 notifier 6228 bl.length()=251 2026-02-24T22:15:34.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.655+0000 7f29cf777a80 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-complete.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1512 : miss 2026-02-24T22:15:34.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.655+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.655+0000 7f29cf777a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:15:34.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.655+0000 7f29cf777a80 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-complete.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1512 info.flags=0x0 2026-02-24T22:15:34.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.655+0000 7f29cf777a80 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-complete.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1512 to cache LRU end 2026-02-24T22:15:34.655 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1513 : miss 2026-02-24T22:15:34.655 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.656 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 rados_obj.operate() r=0 bl.length=476 2026-02-24T22:15:34.656 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1513 info.flags=0x17 2026-02-24T22:15:34.656 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1513 to cache LRU end 2026-02-24T22:15:34.656 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:15:34.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 10 updating xattr: name=user.rgw.acl bl.length()=209 2026-02-24T22:15:34.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1513 2026-02-24T22:15:34.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo 2026-02-24T22:15:34.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a5820b0 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x55dc9a4ad108 s->prefetch_data=0 2026-02-24T22:15:34.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 10 manifest: total_size = 886297 2026-02-24T22:15:34.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7590825751120846387 2026-02-24T22:15:34.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1513.0 2026-02-24T22:15:34.659 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a5820b0 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x55dc9a4ad108 s->prefetch_data=0 2026-02-24T22:15:34.659 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2026-02-24T22:15:34.659 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=246247 2026-02-24T22:15:34.659 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=246247 stripe_ofs=246247 part_ofs=246247 rule->part_size=246247 2026-02-24T22:15:34.659 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2026-02-24T22:15:34.659 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6537703 part_ofs=246247 rule->part_size=246247 2026-02-24T22:15:34.659 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=492494 stripe_ofs=492494 part_ofs=492494 rule->part_size=246247 2026-02-24T22:15:34.659 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2026-02-24T22:15:34.659 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6783950 part_ofs=492494 rule->part_size=246247 2026-02-24T22:15:34.659 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=738741 stripe_ofs=738741 part_ofs=738741 rule->part_size=147556 2026-02-24T22:15:34.659 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=147556 rules.size()=2 2026-02-24T22:15:34.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=7030197 part_ofs=738741 rule->part_size=147556 2026-02-24T22:15:34.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=886297 stripe_ofs=886297 part_ofs=886297 rule->part_size=147556 2026-02-24T22:15:34.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7590825751120846387 2026-02-24T22:15:34.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1513__multipart_foo.2~ATtL2jemudp08pGeq99v6rRNAWXMFEL.1 2026-02-24T22:15:34.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 350 2026-02-24T22:15:34.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1513__multipart_foo.2~ATtL2jemudp08pGeq99v6rRNAWXMFEL.2 2026-02-24T22:15:34.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 600 2026-02-24T22:15:34.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1513__multipart_foo.2~ATtL2jemudp08pGeq99v6rRNAWXMFEL.3 2026-02-24T22:15:34.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 850 2026-02-24T22:15:34.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1513__multipart_foo.2~ATtL2jemudp08pGeq99v6rRNAWXMFEL.4 2026-02-24T22:15:34.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1100 2026-02-24T22:15:34.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.661 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.659+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.16tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7590825751120846387 2026-02-24T22:15:34.661 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.663+0000 7f2997fff640 20 handle_completion(): completion ok for obj=foo 2026-02-24T22:15:34.662 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.663+0000 7f29cf777a80 20 list_objects_ordered: starting attempt 1 2026-02-24T22:15:34.662 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.663+0000 7f29cf777a80 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2026-02-24T22:15:34.662 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.663+0000 7f29cf777a80 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2026-02-24T22:15:34.662 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.663+0000 7f29cf777a80 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2026-02-24T22:15:34.662 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.663+0000 7f29cf777a80 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2026-02-24T22:15:34.662 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.663+0000 7f29cf777a80 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2026-02-24T22:15:34.662 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.663+0000 7f29cf777a80 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-update-complete.1[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1513])"; uploads.size()=0; is_truncated=0 2026-02-24T22:15:34.662 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.663+0000 7f29cf777a80 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : miss 2026-02-24T22:15:34.662 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.663+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.662 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.663+0000 7f29cf777a80 20 rados_obj.operate() r=0 bl.length=246 2026-02-24T22:15:34.662 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.663+0000 7f29cf777a80 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x11 2026-02-24T22:15:34.663 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.663+0000 7f29cf777a80 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2026-02-24T22:15:34.663 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.663+0000 7f29cf777a80 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 from cache 2026-02-24T22:15:34.663 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.663+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-defer-update-complete.1, ofs0, ns] 2026-02-24T22:15:34.663 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.663+0000 7f29a8ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959894 cookie 94405968772512 notifier 6228 bl.length()=198 2026-02-24T22:15:34.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.667+0000 7f29cf777a80 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1513 from cache 2026-02-24T22:15:34.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.667+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1513, ofs0, ns] 2026-02-24T22:15:34.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.667+0000 7f2997fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961313 cookie 94405970389184 notifier 6228 bl.length()=258 2026-02-24T22:15:34.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.667+0000 7f29cf777a80 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-update-complete.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1513 : miss 2026-02-24T22:15:34.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.667+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.667+0000 7f29cf777a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:15:34.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.667+0000 7f29cf777a80 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-update-complete.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1513 info.flags=0x0 2026-02-24T22:15:34.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.667+0000 7f29cf777a80 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-update-complete.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1513 to cache LRU end 2026-02-24T22:15:34.666 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.667+0000 7f29cf777a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1511 : miss 2026-02-24T22:15:34.667 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.667+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.667 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.671+0000 7f29cf777a80 20 rados_obj.operate() r=0 bl.length=471 2026-02-24T22:15:34.667 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.671+0000 7f29cf777a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1511 info.flags=0x17 2026-02-24T22:15:34.668 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.671+0000 7f29cf777a80 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1511 to cache LRU end 2026-02-24T22:15:34.668 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.671+0000 7f29cf777a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:15:34.668 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.671+0000 7f29cf777a80 10 updating xattr: name=user.rgw.acl bl.length()=209 2026-02-24T22:15:34.668 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.671+0000 7f29cf777a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1511 2026-02-24T22:15:34.668 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.671+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo[unyDnMBGuSCwaCn.6LwLgIaKxQ3ZGVC] 2026-02-24T22:15:34.668 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.671+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1511.0 2026-02-24T22:15:34.668 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.671+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a57ff90 obj=test-client.0-r-test-multipart-index-versioning.1:foo state=0x55dc9a4acc08 s->prefetch_data=0 2026-02-24T22:15:34.669 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.671+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to d4secomvtkz7ihzq2onwt1otchyhbxz6 2026-02-24T22:15:34.669 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.671+0000 7f29cf777a80 20 get_obj_state_impl: setting s->olh_tag to bpwcap07iqypughrq8irj7nlam0x21jb 2026-02-24T22:15:34.669 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.671+0000 7f29cf777a80 20 bucket_index_guard_olh_op(): olh_state.olh_tag=bpwcap07iqypughrq8irj7nlam0x21jb 2026-02-24T22:15:34.669 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.671+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1511.0 2026-02-24T22:15:34.670 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.671+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1511.0 2026-02-24T22:15:34.670 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.671+0000 7f29cf777a80 20 olh_log_entry: epoch=1771971334673687722 op=2 key=foo[] 2026-02-24T22:15:34.670 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.671+0000 7f29cf777a80 20 olh_log_entry: epoch=1771971334673687722 op=3 key=foo[unyDnMBGuSCwaCn.6LwLgIaKxQ3ZGVC] 2026-02-24T22:15:34.670 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.671+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a57ff90 obj=test-client.0-r-test-multipart-index-versioning.1:_:unyDnMBGuSCwaCn.6LwLgIaKxQ3ZGVC_foo state=0x55dc9a4b0868 s->prefetch_data=0 2026-02-24T22:15:34.670 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.671+0000 7f29cf777a80 10 manifest: total_size = 74198 2026-02-24T22:15:34.670 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.671+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7117079776643466823 2026-02-24T22:15:34.670 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.671+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1511.0 2026-02-24T22:15:34.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a57ff90 obj=test-client.0-r-test-multipart-index-versioning.1:_:unyDnMBGuSCwaCn.6LwLgIaKxQ3ZGVC_foo state=0x55dc9a4b0868 s->prefetch_data=0 2026-02-24T22:15:34.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2026-02-24T22:15:34.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=20613 2026-02-24T22:15:34.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=20613 stripe_ofs=20613 part_ofs=20613 rule->part_size=20613 2026-02-24T22:15:34.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2026-02-24T22:15:34.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6312069 part_ofs=20613 rule->part_size=20613 2026-02-24T22:15:34.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=41226 stripe_ofs=41226 part_ofs=41226 rule->part_size=20613 2026-02-24T22:15:34.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2026-02-24T22:15:34.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6332682 part_ofs=41226 rule->part_size=20613 2026-02-24T22:15:34.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=61839 stripe_ofs=61839 part_ofs=61839 rule->part_size=12359 2026-02-24T22:15:34.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=12359 rules.size()=2 2026-02-24T22:15:34.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6353295 part_ofs=61839 rule->part_size=12359 2026-02-24T22:15:34.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=74198 stripe_ofs=74198 part_ofs=74198 rule->part_size=12359 2026-02-24T22:15:34.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7117079776643466823 2026-02-24T22:15:34.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1511__multipart_foo.2~NF5_B9ScYFVByojVZs4d1K2b1N7WHkE.1 2026-02-24T22:15:34.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 350 2026-02-24T22:15:34.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1511__multipart_foo.2~NF5_B9ScYFVByojVZs4d1K2b1N7WHkE.2 2026-02-24T22:15:34.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 600 2026-02-24T22:15:34.674 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1511__multipart_foo.2~NF5_B9ScYFVByojVZs4d1K2b1N7WHkE.3 2026-02-24T22:15:34.674 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 850 2026-02-24T22:15:34.674 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1511__multipart_foo.2~NF5_B9ScYFVByojVZs4d1K2b1N7WHkE.4 2026-02-24T22:15:34.674 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1100 2026-02-24T22:15:34.674 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.674 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.25tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7117079776643466823 2026-02-24T22:15:34.675 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=foo[unyDnMBGuSCwaCn.6LwLgIaKxQ3ZGVC] 2026-02-24T22:15:34.675 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a57ff90 obj=test-client.0-r-test-multipart-index-versioning.1:foo state=0x55dc9a4acc08 s->prefetch_data=0 2026-02-24T22:15:34.675 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1511.0 2026-02-24T22:15:34.675 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1511.0 2026-02-24T22:15:34.675 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 20 list_objects_ordered: starting attempt 1 2026-02-24T22:15:34.675 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.675+0000 7f29cf777a80 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2026-02-24T22:15:34.675 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.679+0000 7f29cf777a80 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2026-02-24T22:15:34.675 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.679+0000 7f29cf777a80 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2026-02-24T22:15:34.675 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.679+0000 7f29cf777a80 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2026-02-24T22:15:34.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.679+0000 7f29cf777a80 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2026-02-24T22:15:34.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.679+0000 7f29cf777a80 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-index-versioning.1[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1511])"; uploads.size()=0; is_truncated=0 2026-02-24T22:15:34.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.679+0000 7f29cf777a80 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : miss 2026-02-24T22:15:34.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.679+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.679+0000 7f29cf777a80 20 rados_obj.operate() r=0 bl.length=241 2026-02-24T22:15:34.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.679+0000 7f29cf777a80 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x11 2026-02-24T22:15:34.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.679+0000 7f29cf777a80 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2026-02-24T22:15:34.677 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.679+0000 7f29cf777a80 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 from cache 2026-02-24T22:15:34.677 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.679+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-index-versioning.1, ofs0, ns] 2026-02-24T22:15:34.677 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.679+0000 7f2997fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959895 cookie 94405968772512 notifier 6228 bl.length()=193 2026-02-24T22:15:34.678 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.679+0000 7f29cf777a80 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1511 from cache 2026-02-24T22:15:34.678 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.679+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1511, ofs0, ns] 2026-02-24T22:15:34.678 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.679+0000 7f29a8ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959986 cookie 94405970382176 notifier 6228 bl.length()=253 2026-02-24T22:15:34.679 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.683+0000 7f29cf777a80 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-index-versioning.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1511 : miss 2026-02-24T22:15:34.679 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.683+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.679 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.683+0000 7f29cf777a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:15:34.680 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.683+0000 7f29cf777a80 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-index-versioning.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1511 info.flags=0x0 2026-02-24T22:15:34.680 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.683+0000 7f29cf777a80 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-index-versioning.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1511 to cache LRU end 2026-02-24T22:15:34.681 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.683+0000 7f29cf777a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1510 : miss 2026-02-24T22:15:34.681 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.683+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.682 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.683+0000 7f29cf777a80 20 rados_obj.operate() r=0 bl.length=461 2026-02-24T22:15:34.682 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.683+0000 7f29cf777a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1510 info.flags=0x17 2026-02-24T22:15:34.682 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.683+0000 7f29cf777a80 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1510 to cache LRU end 2026-02-24T22:15:34.682 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.683+0000 7f29cf777a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:15:34.682 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.683+0000 7f29cf777a80 10 updating xattr: name=user.rgw.acl bl.length()=209 2026-02-24T22:15:34.682 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.683+0000 7f29cf777a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1510 2026-02-24T22:15:34.683 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.683+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo 2026-02-24T22:15:34.683 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.683+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a57f240 obj=test-client.0-r-test-multipart-simple.1:foo state=0x55dc9a4adf68 s->prefetch_data=0 2026-02-24T22:15:34.683 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.683+0000 7f29cf777a80 10 manifest: total_size = 74198 2026-02-24T22:15:34.683 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.683+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14943672638169667292 2026-02-24T22:15:34.683 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.683+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1510.0 2026-02-24T22:15:34.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a57f240 obj=test-client.0-r-test-multipart-simple.1:foo state=0x55dc9a4adf68 s->prefetch_data=0 2026-02-24T22:15:34.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2026-02-24T22:15:34.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=20613 2026-02-24T22:15:34.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=20613 stripe_ofs=20613 part_ofs=20613 rule->part_size=20613 2026-02-24T22:15:34.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2026-02-24T22:15:34.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6312069 part_ofs=20613 rule->part_size=20613 2026-02-24T22:15:34.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=41226 stripe_ofs=41226 part_ofs=41226 rule->part_size=20613 2026-02-24T22:15:34.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2026-02-24T22:15:34.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6332682 part_ofs=41226 rule->part_size=20613 2026-02-24T22:15:34.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=61839 stripe_ofs=61839 part_ofs=61839 rule->part_size=12359 2026-02-24T22:15:34.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=12359 rules.size()=2 2026-02-24T22:15:34.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6353295 part_ofs=61839 rule->part_size=12359 2026-02-24T22:15:34.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=74198 stripe_ofs=74198 part_ofs=74198 rule->part_size=12359 2026-02-24T22:15:34.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14943672638169667292 2026-02-24T22:15:34.686 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.686 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1510__multipart_foo.2~C_s_8O1FVHE1j3RXfNFuXkknTNKZslW.1 2026-02-24T22:15:34.686 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 351 2026-02-24T22:15:34.686 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1510__multipart_foo.2~C_s_8O1FVHE1j3RXfNFuXkknTNKZslW.2 2026-02-24T22:15:34.686 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 601 2026-02-24T22:15:34.686 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1510__multipart_foo.2~C_s_8O1FVHE1j3RXfNFuXkknTNKZslW.3 2026-02-24T22:15:34.686 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 851 2026-02-24T22:15:34.686 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1510__multipart_foo.2~C_s_8O1FVHE1j3RXfNFuXkknTNKZslW.4 2026-02-24T22:15:34.686 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1101 2026-02-24T22:15:34.686 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.686 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.5tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14943672638169667292 2026-02-24T22:15:34.687 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=foo 2026-02-24T22:15:34.687 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 list_objects_ordered: starting attempt 1 2026-02-24T22:15:34.687 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2026-02-24T22:15:34.688 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2026-02-24T22:15:34.688 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2026-02-24T22:15:34.688 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2026-02-24T22:15:34.688 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2026-02-24T22:15:34.688 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-simple.1[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1510])"; uploads.size()=0; is_truncated=0 2026-02-24T22:15:34.688 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : miss 2026-02-24T22:15:34.688 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.687+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.688 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.691+0000 7f29cf777a80 20 rados_obj.operate() r=0 bl.length=231 2026-02-24T22:15:34.688 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.691+0000 7f29cf777a80 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x11 2026-02-24T22:15:34.688 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.691+0000 7f29cf777a80 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2026-02-24T22:15:34.689 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.691+0000 7f29cf777a80 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 from cache 2026-02-24T22:15:34.689 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.691+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-simple.1, ofs0, ns] 2026-02-24T22:15:34.689 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.691+0000 7f2997fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554960002 cookie 94405970383248 notifier 6228 bl.length()=183 2026-02-24T22:15:34.689 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.691+0000 7f29cf777a80 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1510 from cache 2026-02-24T22:15:34.689 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.691+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-simple.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1510, ofs0, ns] 2026-02-24T22:15:34.689 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.691+0000 7f29a8ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959895 cookie 94405968788768 notifier 6228 bl.length()=243 2026-02-24T22:15:34.690 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.691+0000 7f29cf777a80 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-simple.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1510 : miss 2026-02-24T22:15:34.690 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.691+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.691 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.691+0000 7f29cf777a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:15:34.691 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.691+0000 7f29cf777a80 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-simple.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1510 info.flags=0x0 2026-02-24T22:15:34.691 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.691+0000 7f29cf777a80 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-simple.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1510 to cache LRU end 2026-02-24T22:15:34.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.695+0000 7f29cf777a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516 : miss 2026-02-24T22:15:34.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.695+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.695+0000 7f29cf777a80 20 rados_obj.operate() r=0 bl.length=467 2026-02-24T22:15:34.693 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.695+0000 7f29cf777a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516 info.flags=0x17 2026-02-24T22:15:34.693 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.695+0000 7f29cf777a80 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516 to cache LRU end 2026-02-24T22:15:34.693 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.695+0000 7f29cf777a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:15:34.693 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.695+0000 7f29cf777a80 10 updating xattr: name=user.rgw.acl bl.length()=209 2026-02-24T22:15:34.693 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.695+0000 7f29cf777a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516 2026-02-24T22:15:34.693 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.695+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-FROZEN 2026-02-24T22:15:34.693 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.695+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-FROZEN-FROZEN 2026-02-24T22:15:34.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.695+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM 2026-02-24T22:15:34.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.695+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-FROZEN-STANDARD 2026-02-24T22:15:34.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.695+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-LUKEWARM 2026-02-24T22:15:34.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.695+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN 2026-02-24T22:15:34.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.695+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM 2026-02-24T22:15:34.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.695+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD 2026-02-24T22:15:34.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.695+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-STANDARD 2026-02-24T22:15:34.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.695+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-STANDARD-FROZEN 2026-02-24T22:15:34.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.695+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM 2026-02-24T22:15:34.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.695+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-STANDARD-STANDARD 2026-02-24T22:15:34.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.695+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a57ff90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x55dc9a4af858 s->prefetch_data=0 2026-02-24T22:15:34.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.695+0000 7f29cf777a80 10 manifest: total_size = 5242880 2026-02-24T22:15:34.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.695+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6535627183964551577 2026-02-24T22:15:34.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.695+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516.0 2026-02-24T22:15:34.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.699+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a57ff90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x55dc9a4af858 s->prefetch_data=0 2026-02-24T22:15:34.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.699+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.699+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.699+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6535627183964551577 2026-02-24T22:15:34.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.699+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.699+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516__shadow_.sWqJtHRKb8-jjjuiy67dcn_vIAyyWJ5_0 2026-02-24T22:15:34.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.699+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 341 2026-02-24T22:15:34.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.699+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.699+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.23tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6535627183964551577 2026-02-24T22:15:34.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.699+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=foo-FROZEN 2026-02-24T22:15:34.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.699+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a57ff90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x55dc9a4af858 s->prefetch_data=0 2026-02-24T22:15:34.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.699+0000 7f29cf777a80 10 manifest: total_size = 5242880 2026-02-24T22:15:34.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.699+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18405830696094412580 2026-02-24T22:15:34.697 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.699+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516.0 2026-02-24T22:15:34.698 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.699+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a57ff90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x55dc9a4af858 s->prefetch_data=0 2026-02-24T22:15:34.698 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.699+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.698 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.699+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.698 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.699+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18405830696094412580 2026-02-24T22:15:34.698 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.699+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.699 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.699+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516__shadow_.sWqJtHRKb8-jjjuiy67dcn_vIAyyWJ5_0 2026-02-24T22:15:34.699 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.699+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 342 2026-02-24T22:15:34.699 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.699+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.699 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.699+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.18tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18405830696094412580 2026-02-24T22:15:34.699 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.703+0000 7f2997fff640 20 handle_completion(): completion ok for obj=foo-FROZEN-FROZEN 2026-02-24T22:15:34.699 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.703+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b80c6f80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x55dc9a4af858 s->prefetch_data=0 2026-02-24T22:15:34.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.703+0000 7f29cf777a80 10 manifest: total_size = 5242880 2026-02-24T22:15:34.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.703+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to _nCCGDm0rkN93CCQ-QNE6qTxPVqof9O0 2026-02-24T22:15:34.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.703+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516.0 2026-02-24T22:15:34.701 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.703+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b80c6f80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x55dc9a4af858 s->prefetch_data=0 2026-02-24T22:15:34.701 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.703+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.701 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.703+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.702 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.703+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: _nCCGDm0rkN93CCQ-QNE6qTxPVqof9O0 2026-02-24T22:15:34.702 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.703+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.702 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.703+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516__shadow_.uhuY4WSjo13TdnwcizzSKCVTzCu2KNn_0 2026-02-24T22:15:34.702 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.703+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 314 2026-02-24T22:15:34.702 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.703+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.702 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.703+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.2tag is: _nCCGDm0rkN93CCQ-QNE6qTxPVqof9O0 2026-02-24T22:15:34.702 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.703+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=foo-FROZEN-LUKEWARM 2026-02-24T22:15:34.702 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.703+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b80c6f80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x55dc9a4af858 s->prefetch_data=0 2026-02-24T22:15:34.703 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.703+0000 7f29cf777a80 10 manifest: total_size = 2044 2026-02-24T22:15:34.703 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.703+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to _tOBSvPkQmPQ23t_Bs9nDZLtArbY2MN0 2026-02-24T22:15:34.703 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.703+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516.0 2026-02-24T22:15:34.704 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.707+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b80c6f80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x55dc9a4af858 s->prefetch_data=0 2026-02-24T22:15:34.704 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.707+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.704 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.707+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=2044 stripe_ofs=2044 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.704 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.707+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: _tOBSvPkQmPQ23t_Bs9nDZLtArbY2MN0 2026-02-24T22:15:34.704 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.707+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.704 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.707+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516__shadow_.F-HaF-nU-REigcj3r1Z_eoaX9ZwtJ3K_0 2026-02-24T22:15:34.705 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.707+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 305 2026-02-24T22:15:34.705 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.707+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.705 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.707+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.2tag is: _tOBSvPkQmPQ23t_Bs9nDZLtArbY2MN0 2026-02-24T22:15:34.705 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.707+0000 7f2997fff640 20 handle_completion(): completion ok for obj=foo-FROZEN-STANDARD 2026-02-24T22:15:34.706 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.707+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b80c6f80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x55dc9a4af858 s->prefetch_data=0 2026-02-24T22:15:34.706 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.707+0000 7f29cf777a80 10 manifest: total_size = 5242880 2026-02-24T22:15:34.706 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.707+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13109709138848217368 2026-02-24T22:15:34.707 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.707+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516.0 2026-02-24T22:15:34.708 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.711+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b80c6f80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x55dc9a4af858 s->prefetch_data=0 2026-02-24T22:15:34.708 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.711+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.708 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.711+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.708 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.711+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13109709138848217368 2026-02-24T22:15:34.708 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.711+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.708 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.711+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516__shadow_.CdgduzJ1eKGKrr2KZaGcK9i4YvqxGVS_0 2026-02-24T22:15:34.709 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.711+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 344 2026-02-24T22:15:34.709 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.711+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.709 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.711+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.15tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13109709138848217368 2026-02-24T22:15:34.709 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.711+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2026-02-24T22:15:34.709 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.711+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c0070150 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x55dc9a4af858 s->prefetch_data=0 2026-02-24T22:15:34.709 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.711+0000 7f29cf777a80 10 manifest: total_size = 5242880 2026-02-24T22:15:34.710 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.711+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to _Wt1Ru69B-lzXu5V57DiCvWsM7R-nYyD 2026-02-24T22:15:34.710 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.711+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516.0 2026-02-24T22:15:34.711 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.715+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c0070150 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x55dc9a4af858 s->prefetch_data=0 2026-02-24T22:15:34.711 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.715+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.711 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.715+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.711 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.715+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: _Wt1Ru69B-lzXu5V57DiCvWsM7R-nYyD 2026-02-24T22:15:34.711 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.715+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.712 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.715+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516__shadow_.D8W143s3I_9bUC-zDmO5dBf-ExUdgqz_0 2026-02-24T22:15:34.712 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.715+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 312 2026-02-24T22:15:34.712 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.715+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.712 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.715+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.9tag is: _Wt1Ru69B-lzXu5V57DiCvWsM7R-nYyD 2026-02-24T22:15:34.712 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.715+0000 7f2997fff640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-FROZEN 2026-02-24T22:15:34.712 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.715+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c0070150 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x55dc9a4af858 s->prefetch_data=0 2026-02-24T22:15:34.713 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.715+0000 7f29cf777a80 10 manifest: total_size = 5242880 2026-02-24T22:15:34.713 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.715+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10304081461979102702 2026-02-24T22:15:34.713 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.715+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516.0 2026-02-24T22:15:34.714 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.715+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c0070150 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x55dc9a4af858 s->prefetch_data=0 2026-02-24T22:15:34.714 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.715+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.714 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.715+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.714 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.715+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10304081461979102702 2026-02-24T22:15:34.714 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.715+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.714 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.715+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516__shadow_.CdgduzJ1eKGKrr2KZaGcK9i4YvqxGVS_0 2026-02-24T22:15:34.714 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.715+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 344 2026-02-24T22:15:34.714 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.715+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.715 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.715+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.31tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10304081461979102702 2026-02-24T22:15:34.715 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.715+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-LUKEWARM 2026-02-24T22:15:34.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.719+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c0040c90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x55dc9a4af858 s->prefetch_data=0 2026-02-24T22:15:34.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.719+0000 7f29cf777a80 10 manifest: total_size = 22651 2026-02-24T22:15:34.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.719+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to _W5viz0oj3FO2M-LaumlMS9NAsjzcb7c 2026-02-24T22:15:34.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.719+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516.0 2026-02-24T22:15:34.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.719+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c0040c90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x55dc9a4af858 s->prefetch_data=0 2026-02-24T22:15:34.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.719+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.719+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=22651 stripe_ofs=22651 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.719+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: _W5viz0oj3FO2M-LaumlMS9NAsjzcb7c 2026-02-24T22:15:34.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.719+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.719+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516__shadow_.15O0tu_ni1pbUws8wBwFTuroGmO0gDt_0 2026-02-24T22:15:34.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.719+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 305 2026-02-24T22:15:34.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.719+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.719+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.18tag is: _W5viz0oj3FO2M-LaumlMS9NAsjzcb7c 2026-02-24T22:15:34.719 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.719+0000 7f2997fff640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-STANDARD 2026-02-24T22:15:34.719 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.723+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c0070150 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x55dc9a4af858 s->prefetch_data=0 2026-02-24T22:15:34.719 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.723+0000 7f29cf777a80 10 manifest: total_size = 324647 2026-02-24T22:15:34.719 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.723+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9358090815558633539 2026-02-24T22:15:34.719 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.723+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516.0 2026-02-24T22:15:34.721 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.723+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c0070150 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x55dc9a4af858 s->prefetch_data=0 2026-02-24T22:15:34.721 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.723+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.721 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.723+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=324647 stripe_ofs=324647 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.721 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.723+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9358090815558633539 2026-02-24T22:15:34.721 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.723+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.721 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.723+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516__shadow_.xxehxqX99AfpMEclRDOFmiTORUyCPIY_0 2026-02-24T22:15:34.721 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.723+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 334 2026-02-24T22:15:34.721 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.723+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.721 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.723+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.13tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9358090815558633539 2026-02-24T22:15:34.722 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.723+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=foo-STANDARD 2026-02-24T22:15:34.722 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.723+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b80c6f80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x55dc9a4af858 s->prefetch_data=0 2026-02-24T22:15:34.722 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.723+0000 7f29cf777a80 10 manifest: total_size = 5242880 2026-02-24T22:15:34.722 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.723+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to _RhxOYsClOISq4Qa0MqZpqN0V19U2jYQ 2026-02-24T22:15:34.722 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.723+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516.0 2026-02-24T22:15:34.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.727+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b80c6f80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x55dc9a4af858 s->prefetch_data=0 2026-02-24T22:15:34.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.727+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.727+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.727+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: _RhxOYsClOISq4Qa0MqZpqN0V19U2jYQ 2026-02-24T22:15:34.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.727+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.727+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516__shadow_.-XjTT5X1d7UuY3Uf7qRWM3a15gOiNVj_0 2026-02-24T22:15:34.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.727+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 312 2026-02-24T22:15:34.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.727+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.727+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.12tag is: _RhxOYsClOISq4Qa0MqZpqN0V19U2jYQ 2026-02-24T22:15:34.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.727+0000 7f2997fff640 20 handle_completion(): completion ok for obj=foo-STANDARD-FROZEN 2026-02-24T22:15:34.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.727+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c0040c90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x55dc9a4af858 s->prefetch_data=0 2026-02-24T22:15:34.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.727+0000 7f29cf777a80 10 manifest: total_size = 5242880 2026-02-24T22:15:34.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.727+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to _wRdzuuKc7-rNqVuziPyNzmkO3KRpsXv 2026-02-24T22:15:34.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.727+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516.0 2026-02-24T22:15:34.726 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.727+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c0040c90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x55dc9a4af858 s->prefetch_data=0 2026-02-24T22:15:34.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.727+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.727+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.727+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: _wRdzuuKc7-rNqVuziPyNzmkO3KRpsXv 2026-02-24T22:15:34.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.727+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.727+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516__shadow_.qgVdRUKpPtJp93cwY89LYuQAR3QEpgp_0 2026-02-24T22:15:34.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.727+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 314 2026-02-24T22:15:34.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.727+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.727+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.12tag is: _wRdzuuKc7-rNqVuziPyNzmkO3KRpsXv 2026-02-24T22:15:34.728 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.731+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b80c6f80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x55dc9a4af858 s->prefetch_data=0 2026-02-24T22:15:34.728 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.731+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=foo-STANDARD-LUKEWARM 2026-02-24T22:15:34.728 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.731+0000 7f29cf777a80 10 manifest: total_size = 324647 2026-02-24T22:15:34.728 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.731+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6571258558727922323 2026-02-24T22:15:34.728 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.731+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516.0 2026-02-24T22:15:34.729 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.731+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b80c6f80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x55dc9a4af858 s->prefetch_data=0 2026-02-24T22:15:34.729 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.731+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.729 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.731+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=324647 stripe_ofs=324647 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.729 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.731+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6571258558727922323 2026-02-24T22:15:34.729 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.731+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.729 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.731+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516__shadow_.xxehxqX99AfpMEclRDOFmiTORUyCPIY_0 2026-02-24T22:15:34.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.731+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 334 2026-02-24T22:15:34.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.731+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.731+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.22tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6571258558727922323 2026-02-24T22:15:34.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.731+0000 7f2997fff640 20 handle_completion(): completion ok for obj=foo-STANDARD-STANDARD 2026-02-24T22:15:34.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.731+0000 7f29cf777a80 20 list_objects_ordered: starting attempt 1 2026-02-24T22:15:34.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.731+0000 7f29cf777a80 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2026-02-24T22:15:34.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.731+0000 7f29cf777a80 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2026-02-24T22:15:34.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.731+0000 7f29cf777a80 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2026-02-24T22:15:34.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.731+0000 7f29cf777a80 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2026-02-24T22:15:34.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.731+0000 7f29cf777a80 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2026-02-24T22:15:34.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.731+0000 7f29cf777a80 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-copy.1[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516])"; uploads.size()=0; is_truncated=0 2026-02-24T22:15:34.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.735+0000 7f29cf777a80 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : miss 2026-02-24T22:15:34.732 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.735+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.732 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.735+0000 7f29cf777a80 20 rados_obj.operate() r=0 bl.length=237 2026-02-24T22:15:34.732 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.735+0000 7f29cf777a80 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x11 2026-02-24T22:15:34.732 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.735+0000 7f29cf777a80 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2026-02-24T22:15:34.733 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.735+0000 7f29cf777a80 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 from cache 2026-02-24T22:15:34.733 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.735+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class-copy.1, ofs0, ns] 2026-02-24T22:15:34.733 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.735+0000 7f29a8ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959896 cookie 94405968772512 notifier 6228 bl.length()=189 2026-02-24T22:15:34.734 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.735+0000 7f29cf777a80 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516 from cache 2026-02-24T22:15:34.734 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.735+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516, ofs0, ns] 2026-02-24T22:15:34.734 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.735+0000 7f2997fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959987 cookie 94405970382176 notifier 6228 bl.length()=249 2026-02-24T22:15:34.735 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-copy.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516 : miss 2026-02-24T22:15:34.735 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.736 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:15:34.736 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-copy.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516 info.flags=0x0 2026-02-24T22:15:34.736 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-copy.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516 to cache LRU end 2026-02-24T22:15:34.737 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517 : miss 2026-02-24T22:15:34.737 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.738 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 20 rados_obj.operate() r=0 bl.length=477 2026-02-24T22:15:34.738 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517 info.flags=0x17 2026-02-24T22:15:34.738 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517 to cache LRU end 2026-02-24T22:15:34.738 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:15:34.738 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 10 updating xattr: name=user.rgw.acl bl.length()=209 2026-02-24T22:15:34.738 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517 2026-02-24T22:15:34.739 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-FROZEN 2026-02-24T22:15:34.739 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-FROZEN-FROZEN 2026-02-24T22:15:34.739 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM 2026-02-24T22:15:34.739 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-FROZEN-STANDARD 2026-02-24T22:15:34.739 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-LUKEWARM 2026-02-24T22:15:34.739 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN 2026-02-24T22:15:34.739 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM 2026-02-24T22:15:34.739 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD 2026-02-24T22:15:34.739 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-STANDARD 2026-02-24T22:15:34.739 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-STANDARD-FROZEN 2026-02-24T22:15:34.739 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM 2026-02-24T22:15:34.739 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-STANDARD-STANDARD 2026-02-24T22:15:34.739 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c40473b0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x55dc9a4ae388 s->prefetch_data=0 2026-02-24T22:15:34.740 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 10 manifest: total_size = 11534336 2026-02-24T22:15:34.740 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18294110152583415713 2026-02-24T22:15:34.740 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.739+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517.0 2026-02-24T22:15:34.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.743+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c40473b0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x55dc9a4ae388 s->prefetch_data=0 2026-02-24T22:15:34.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.743+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2026-02-24T22:15:34.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.743+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2026-02-24T22:15:34.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.743+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2026-02-24T22:15:34.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.743+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2026-02-24T22:15:34.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.743+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2026-02-24T22:15:34.742 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.743+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2026-02-24T22:15:34.742 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.743+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2026-02-24T22:15:34.742 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.743+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2026-02-24T22:15:34.742 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.743+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2026-02-24T22:15:34.742 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.743+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18294110152583415713 2026-02-24T22:15:34.742 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.743+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.742 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.743+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-FROZEN.2~5YnNWY2srlSZqsTYGvvi4fm2BZ8_YNh.1 2026-02-24T22:15:34.742 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.743+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 372 2026-02-24T22:15:34.742 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.743+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-FROZEN.2~5YnNWY2srlSZqsTYGvvi4fm2BZ8_YNh.2 2026-02-24T22:15:34.742 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.743+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 643 2026-02-24T22:15:34.742 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.743+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-FROZEN.2~5YnNWY2srlSZqsTYGvvi4fm2BZ8_YNh.3 2026-02-24T22:15:34.743 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.743+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 914 2026-02-24T22:15:34.743 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.743+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.743 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.743+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.24tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18294110152583415713 2026-02-24T22:15:34.744 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.743+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=foo-FROZEN 2026-02-24T22:15:34.744 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.743+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b80a5040 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x55dc9a4adf68 s->prefetch_data=0 2026-02-24T22:15:34.744 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.743+0000 7f29cf777a80 10 manifest: total_size = 11534336 2026-02-24T22:15:34.744 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.743+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15636809563206940919 2026-02-24T22:15:34.744 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.743+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517.0 2026-02-24T22:15:34.744 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b80a5040 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x55dc9a4adf68 s->prefetch_data=0 2026-02-24T22:15:34.744 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2026-02-24T22:15:34.744 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2026-02-24T22:15:34.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2026-02-24T22:15:34.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2026-02-24T22:15:34.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2026-02-24T22:15:34.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2026-02-24T22:15:34.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2026-02-24T22:15:34.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2026-02-24T22:15:34.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2026-02-24T22:15:34.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15636809563206940919 2026-02-24T22:15:34.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-FROZEN.2~5YnNWY2srlSZqsTYGvvi4fm2BZ8_YNh.1 2026-02-24T22:15:34.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 372 2026-02-24T22:15:34.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-FROZEN.2~5YnNWY2srlSZqsTYGvvi4fm2BZ8_YNh.2 2026-02-24T22:15:34.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 643 2026-02-24T22:15:34.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-FROZEN.2~5YnNWY2srlSZqsTYGvvi4fm2BZ8_YNh.3 2026-02-24T22:15:34.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 914 2026-02-24T22:15:34.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.20tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15636809563206940919 2026-02-24T22:15:34.746 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f2997fff640 20 handle_completion(): completion ok for obj=foo-FROZEN-FROZEN 2026-02-24T22:15:34.746 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a179d80 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x55dc9a4adf68 s->prefetch_data=0 2026-02-24T22:15:34.746 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 10 manifest: total_size = 11534336 2026-02-24T22:15:34.746 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to _fsAoz_6UOtDohCTsbCNwbDGWOgo4Lig 2026-02-24T22:15:34.746 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517.0 2026-02-24T22:15:34.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a179d80 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x55dc9a4adf68 s->prefetch_data=0 2026-02-24T22:15:34.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: _fsAoz_6UOtDohCTsbCNwbDGWOgo4Lig 2026-02-24T22:15:34.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__shadow_.VKLnlZFWfqVGhJjB9E_MozyOXaKiE-l_0 2026-02-24T22:15:34.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 314 2026-02-24T22:15:34.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__shadow_.VKLnlZFWfqVGhJjB9E_MozyOXaKiE-l_1 2026-02-24T22:15:34.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 557 2026-02-24T22:15:34.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.747+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.23tag is: _fsAoz_6UOtDohCTsbCNwbDGWOgo4Lig 2026-02-24T22:15:34.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.751+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=foo-FROZEN-LUKEWARM 2026-02-24T22:15:34.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.751+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a581420 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x55dc9a4adf68 s->prefetch_data=0 2026-02-24T22:15:34.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.751+0000 7f29cf777a80 10 manifest: total_size = 45360 2026-02-24T22:15:34.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.751+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to _eMxk_RffcSV4X2zgMvC2RgaeyyGFAMg 2026-02-24T22:15:34.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.751+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517.0 2026-02-24T22:15:34.749 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.751+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a581420 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x55dc9a4adf68 s->prefetch_data=0 2026-02-24T22:15:34.749 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.751+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.749 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.751+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=45360 stripe_ofs=45360 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.750 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.751+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: _eMxk_RffcSV4X2zgMvC2RgaeyyGFAMg 2026-02-24T22:15:34.750 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.751+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.750 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.751+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__shadow_.QKLJaehx_iyyTi0XhEEtK594VLJ3McC_0 2026-02-24T22:15:34.750 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.751+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 305 2026-02-24T22:15:34.750 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.751+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.750 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.751+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.22tag is: _eMxk_RffcSV4X2zgMvC2RgaeyyGFAMg 2026-02-24T22:15:34.750 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.751+0000 7f2997fff640 20 handle_completion(): completion ok for obj=foo-FROZEN-STANDARD 2026-02-24T22:15:34.750 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.751+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c40473b0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x55dc9a4adf68 s->prefetch_data=0 2026-02-24T22:15:34.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.751+0000 7f29cf777a80 10 manifest: total_size = 11534336 2026-02-24T22:15:34.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.751+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1281654688979473632 2026-02-24T22:15:34.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.755+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517.0 2026-02-24T22:15:34.752 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.755+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c40473b0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x55dc9a4adf68 s->prefetch_data=0 2026-02-24T22:15:34.752 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.755+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2026-02-24T22:15:34.752 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.755+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2026-02-24T22:15:34.752 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.755+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2026-02-24T22:15:34.752 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.755+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2026-02-24T22:15:34.752 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.755+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2026-02-24T22:15:34.753 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.755+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2026-02-24T22:15:34.753 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.755+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2026-02-24T22:15:34.753 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.755+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2026-02-24T22:15:34.753 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.755+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2026-02-24T22:15:34.753 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.755+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1281654688979473632 2026-02-24T22:15:34.753 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.755+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.753 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.755+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-LUKEWARM.2~eOtMH0vmpN_usdE1y1Q_d9UHSttRW0T.1 2026-02-24T22:15:34.753 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.755+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 377 2026-02-24T22:15:34.753 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.755+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-LUKEWARM.2~eOtMH0vmpN_usdE1y1Q_d9UHSttRW0T.2 2026-02-24T22:15:34.753 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.755+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 654 2026-02-24T22:15:34.753 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.755+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-LUKEWARM.2~eOtMH0vmpN_usdE1y1Q_d9UHSttRW0T.3 2026-02-24T22:15:34.753 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.755+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 931 2026-02-24T22:15:34.753 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.755+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.753 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.755+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1281654688979473632 2026-02-24T22:15:34.754 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.755+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2026-02-24T22:15:34.754 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.755+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b812e3f0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x55dc9a4adf68 s->prefetch_data=0 2026-02-24T22:15:34.754 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.755+0000 7f29cf777a80 10 manifest: total_size = 11534336 2026-02-24T22:15:34.754 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.755+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to _ts56h4efjrgjs2u92KKZkl03X4Y-q0M 2026-02-24T22:15:34.754 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.755+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517.0 2026-02-24T22:15:34.755 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b812e3f0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x55dc9a4adf68 s->prefetch_data=0 2026-02-24T22:15:34.755 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.755 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.755 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.755 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.755 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: _ts56h4efjrgjs2u92KKZkl03X4Y-q0M 2026-02-24T22:15:34.755 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.755 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__shadow_.0YMmA6C8ADUnevp6GXGSxQcYwmX1W7K_0 2026-02-24T22:15:34.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 312 2026-02-24T22:15:34.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__shadow_.0YMmA6C8ADUnevp6GXGSxQcYwmX1W7K_1 2026-02-24T22:15:34.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 553 2026-02-24T22:15:34.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.11tag is: _ts56h4efjrgjs2u92KKZkl03X4Y-q0M 2026-02-24T22:15:34.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f2997fff640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-FROZEN 2026-02-24T22:15:34.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c0073c40 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x55dc9a4adf68 s->prefetch_data=0 2026-02-24T22:15:34.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 10 manifest: total_size = 11534336 2026-02-24T22:15:34.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9762428589688093258 2026-02-24T22:15:34.757 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517.0 2026-02-24T22:15:34.758 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c0073c40 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x55dc9a4adf68 s->prefetch_data=0 2026-02-24T22:15:34.758 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2026-02-24T22:15:34.758 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2026-02-24T22:15:34.758 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2026-02-24T22:15:34.758 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2026-02-24T22:15:34.758 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2026-02-24T22:15:34.758 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2026-02-24T22:15:34.758 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2026-02-24T22:15:34.759 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2026-02-24T22:15:34.759 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2026-02-24T22:15:34.759 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9762428589688093258 2026-02-24T22:15:34.759 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.759 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-LUKEWARM.2~eOtMH0vmpN_usdE1y1Q_d9UHSttRW0T.1 2026-02-24T22:15:34.759 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 377 2026-02-24T22:15:34.759 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-LUKEWARM.2~eOtMH0vmpN_usdE1y1Q_d9UHSttRW0T.2 2026-02-24T22:15:34.759 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 654 2026-02-24T22:15:34.759 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-LUKEWARM.2~eOtMH0vmpN_usdE1y1Q_d9UHSttRW0T.3 2026-02-24T22:15:34.759 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 931 2026-02-24T22:15:34.759 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.759 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.759+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.5tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9762428589688093258 2026-02-24T22:15:34.760 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.763+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-LUKEWARM 2026-02-24T22:15:34.760 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.763+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a576180 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x55dc9a4adf68 s->prefetch_data=0 2026-02-24T22:15:34.760 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.763+0000 7f29cf777a80 10 manifest: total_size = 11534336 2026-02-24T22:15:34.760 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.763+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to _B2FfFxI9qhwogYvcRSKsvoB7bA6j04b 2026-02-24T22:15:34.760 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.763+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517.0 2026-02-24T22:15:34.761 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.763+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a576180 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x55dc9a4adf68 s->prefetch_data=0 2026-02-24T22:15:34.761 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.763+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.761 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.763+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.761 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.763+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.761 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.763+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.761 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.763+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: _B2FfFxI9qhwogYvcRSKsvoB7bA6j04b 2026-02-24T22:15:34.761 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.763+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.763+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__shadow_.gZ3g481IEgRHeOYtYzixIcbqGp57sGE_0 2026-02-24T22:15:34.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.763+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 305 2026-02-24T22:15:34.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.763+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__shadow_.gZ3g481IEgRHeOYtYzixIcbqGp57sGE_1 2026-02-24T22:15:34.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.763+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 539 2026-02-24T22:15:34.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.763+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.763+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.31tag is: _B2FfFxI9qhwogYvcRSKsvoB7bA6j04b 2026-02-24T22:15:34.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.763+0000 7f2997fff640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-STANDARD 2026-02-24T22:15:34.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.763+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b81124b0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x55dc9a4adf68 s->prefetch_data=0 2026-02-24T22:15:34.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.763+0000 7f29cf777a80 10 manifest: total_size = 45360 2026-02-24T22:15:34.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.763+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.376449562292424880 2026-02-24T22:15:34.763 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.763+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517.0 2026-02-24T22:15:34.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.767+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b81124b0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x55dc9a4adf68 s->prefetch_data=0 2026-02-24T22:15:34.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.767+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2026-02-24T22:15:34.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.767+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=20613 2026-02-24T22:15:34.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.767+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=20613 stripe_ofs=20613 part_ofs=20613 rule->part_size=20613 2026-02-24T22:15:34.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.767+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2026-02-24T22:15:34.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.767+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6312069 part_ofs=20613 rule->part_size=20613 2026-02-24T22:15:34.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.767+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=41226 stripe_ofs=41226 part_ofs=41226 rule->part_size=4134 2026-02-24T22:15:34.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.767+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=4134 rules.size()=2 2026-02-24T22:15:34.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.767+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6332682 part_ofs=41226 rule->part_size=4134 2026-02-24T22:15:34.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.767+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=45360 stripe_ofs=45360 part_ofs=45360 rule->part_size=4134 2026-02-24T22:15:34.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.767+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.376449562292424880 2026-02-24T22:15:34.765 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.767+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.765 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.767+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-STANDARD.2~jB8VjG_eXaI_0rIVLXOTPSldrI2t5vd.1 2026-02-24T22:15:34.765 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.767+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 367 2026-02-24T22:15:34.765 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.767+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-STANDARD.2~jB8VjG_eXaI_0rIVLXOTPSldrI2t5vd.2 2026-02-24T22:15:34.765 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.767+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 635 2026-02-24T22:15:34.765 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.767+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-STANDARD.2~jB8VjG_eXaI_0rIVLXOTPSldrI2t5vd.3 2026-02-24T22:15:34.765 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.767+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 903 2026-02-24T22:15:34.765 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.767+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.765 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.767+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.8tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.376449562292424880 2026-02-24T22:15:34.765 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.767+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=foo-STANDARD 2026-02-24T22:15:34.765 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.767+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a4ac5b0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x55dc9a4adf68 s->prefetch_data=0 2026-02-24T22:15:34.766 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.767+0000 7f29cf777a80 10 manifest: total_size = 11534336 2026-02-24T22:15:34.766 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.767+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to _pkoGCCK1k5qeAAjR3H0FufucfZZXwL5 2026-02-24T22:15:34.766 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.767+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517.0 2026-02-24T22:15:34.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a4ac5b0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x55dc9a4adf68 s->prefetch_data=0 2026-02-24T22:15:34.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: _pkoGCCK1k5qeAAjR3H0FufucfZZXwL5 2026-02-24T22:15:34.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__shadow_.4c_YiLaIb-mvcfuglGE64eqdsF08fxH_0 2026-02-24T22:15:34.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 312 2026-02-24T22:15:34.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__shadow_.4c_YiLaIb-mvcfuglGE64eqdsF08fxH_1 2026-02-24T22:15:34.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 553 2026-02-24T22:15:34.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: _pkoGCCK1k5qeAAjR3H0FufucfZZXwL5 2026-02-24T22:15:34.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f2997fff640 20 handle_completion(): completion ok for obj=foo-STANDARD-FROZEN 2026-02-24T22:15:34.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b80c6f80 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x55dc9a4adf68 s->prefetch_data=0 2026-02-24T22:15:34.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 10 manifest: total_size = 11534336 2026-02-24T22:15:34.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to _Ifx-HNEmzwwadjGmce14gvFB1L2Q25d 2026-02-24T22:15:34.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517.0 2026-02-24T22:15:34.770 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b80c6f80 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x55dc9a4adf68 s->prefetch_data=0 2026-02-24T22:15:34.770 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.770 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.770 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.770 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: _Ifx-HNEmzwwadjGmce14gvFB1L2Q25d 2026-02-24T22:15:34.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__shadow_.KnK3Gb3aUR1Rg2m1qiZ5r5bTDgHzkTU_0 2026-02-24T22:15:34.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 314 2026-02-24T22:15:34.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__shadow_.KnK3Gb3aUR1Rg2m1qiZ5r5bTDgHzkTU_1 2026-02-24T22:15:34.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 557 2026-02-24T22:15:34.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.1tag is: _Ifx-HNEmzwwadjGmce14gvFB1L2Q25d 2026-02-24T22:15:34.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.771+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=foo-STANDARD-LUKEWARM 2026-02-24T22:15:34.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c0073fb0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x55dc9a4adf68 s->prefetch_data=0 2026-02-24T22:15:34.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 10 manifest: total_size = 45360 2026-02-24T22:15:34.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17574378362889513698 2026-02-24T22:15:34.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517.0 2026-02-24T22:15:34.773 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c0073fb0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x55dc9a4adf68 s->prefetch_data=0 2026-02-24T22:15:34.773 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2026-02-24T22:15:34.773 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=20613 2026-02-24T22:15:34.773 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=20613 stripe_ofs=20613 part_ofs=20613 rule->part_size=20613 2026-02-24T22:15:34.773 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2026-02-24T22:15:34.773 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6312069 part_ofs=20613 rule->part_size=20613 2026-02-24T22:15:34.774 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=41226 stripe_ofs=41226 part_ofs=41226 rule->part_size=4134 2026-02-24T22:15:34.774 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=4134 rules.size()=2 2026-02-24T22:15:34.774 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6332682 part_ofs=41226 rule->part_size=4134 2026-02-24T22:15:34.774 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=45360 stripe_ofs=45360 part_ofs=45360 rule->part_size=4134 2026-02-24T22:15:34.774 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17574378362889513698 2026-02-24T22:15:34.774 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.774 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-STANDARD.2~jB8VjG_eXaI_0rIVLXOTPSldrI2t5vd.1 2026-02-24T22:15:34.774 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 369 2026-02-24T22:15:34.774 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-STANDARD.2~jB8VjG_eXaI_0rIVLXOTPSldrI2t5vd.2 2026-02-24T22:15:34.775 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 637 2026-02-24T22:15:34.775 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-STANDARD.2~jB8VjG_eXaI_0rIVLXOTPSldrI2t5vd.3 2026-02-24T22:15:34.775 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 905 2026-02-24T22:15:34.775 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.775 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.11tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17574378362889513698 2026-02-24T22:15:34.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f2997fff640 20 handle_completion(): completion ok for obj=foo-STANDARD-STANDARD 2026-02-24T22:15:34.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 list_objects_ordered: starting attempt 1 2026-02-24T22:15:34.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2026-02-24T22:15:34.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2026-02-24T22:15:34.777 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2026-02-24T22:15:34.777 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2026-02-24T22:15:34.777 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2026-02-24T22:15:34.777 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.775+0000 7f29cf777a80 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart-copy.1[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517])"; uploads.size()=0; is_truncated=0 2026-02-24T22:15:34.777 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.779+0000 7f29cf777a80 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : miss 2026-02-24T22:15:34.777 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.779+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.777 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.779+0000 7f29cf777a80 20 rados_obj.operate() r=0 bl.length=247 2026-02-24T22:15:34.777 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.779+0000 7f29cf777a80 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x11 2026-02-24T22:15:34.777 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.779+0000 7f29cf777a80 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2026-02-24T22:15:34.777 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.779+0000 7f29cf777a80 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 from cache 2026-02-24T22:15:34.777 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.779+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart-copy.1, ofs0, ns] 2026-02-24T22:15:34.777 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.779+0000 7f29a8ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961314 cookie 94405970389184 notifier 6228 bl.length()=199 2026-02-24T22:15:34.778 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.779+0000 7f29cf777a80 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517 from cache 2026-02-24T22:15:34.778 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.779+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517, ofs0, ns] 2026-02-24T22:15:34.778 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.779+0000 7f2997fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554960003 cookie 94405970383248 notifier 6228 bl.length()=259 2026-02-24T22:15:34.779 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.783+0000 7f29cf777a80 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart-copy.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517 : miss 2026-02-24T22:15:34.779 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.783+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.779 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.783+0000 7f29cf777a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:15:34.779 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.783+0000 7f29cf777a80 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart-copy.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517 info.flags=0x0 2026-02-24T22:15:34.779 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.783+0000 7f29cf777a80 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart-copy.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517 to cache LRU end 2026-02-24T22:15:34.781 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.783+0000 7f29cf777a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515 : miss 2026-02-24T22:15:34.781 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.783+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.781 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.783+0000 7f29cf777a80 20 rados_obj.operate() r=0 bl.length=472 2026-02-24T22:15:34.781 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.783+0000 7f29cf777a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515 info.flags=0x17 2026-02-24T22:15:34.781 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.783+0000 7f29cf777a80 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515 to cache LRU end 2026-02-24T22:15:34.781 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.783+0000 7f29cf777a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:15:34.782 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.783+0000 7f29cf777a80 10 updating xattr: name=user.rgw.acl bl.length()=209 2026-02-24T22:15:34.782 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.783+0000 7f29cf777a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515 2026-02-24T22:15:34.782 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.783+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-FROZEN 2026-02-24T22:15:34.782 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.783+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-LUKEWARM 2026-02-24T22:15:34.782 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.783+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-STANDARD 2026-02-24T22:15:34.783 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.783+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c401a040 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x55dc9a175f28 s->prefetch_data=0 2026-02-24T22:15:34.783 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.783+0000 7f29cf777a80 10 manifest: total_size = 11534336 2026-02-24T22:15:34.783 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.783+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15874434673824327812 2026-02-24T22:15:34.783 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.783+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515.0 2026-02-24T22:15:34.784 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c401a040 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x55dc9a175f28 s->prefetch_data=0 2026-02-24T22:15:34.784 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2026-02-24T22:15:34.784 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2026-02-24T22:15:34.784 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2026-02-24T22:15:34.784 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2026-02-24T22:15:34.784 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2026-02-24T22:15:34.784 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2026-02-24T22:15:34.784 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2026-02-24T22:15:34.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2026-02-24T22:15:34.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2026-02-24T22:15:34.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15874434673824327812 2026-02-24T22:15:34.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515__multipart_foo-FROZEN.2~v_f2anZgPC1pddb-uul3GH8WZsvkJ8F.1 2026-02-24T22:15:34.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 372 2026-02-24T22:15:34.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515__multipart_foo-FROZEN.2~v_f2anZgPC1pddb-uul3GH8WZsvkJ8F.2 2026-02-24T22:15:34.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 643 2026-02-24T22:15:34.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515__multipart_foo-FROZEN.2~v_f2anZgPC1pddb-uul3GH8WZsvkJ8F.3 2026-02-24T22:15:34.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 914 2026-02-24T22:15:34.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.786 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.26tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15874434673824327812 2026-02-24T22:15:34.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f2997fff640 20 handle_completion(): completion ok for obj=foo-FROZEN 2026-02-24T22:15:34.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b812e3f0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x55dc9a175f28 s->prefetch_data=0 2026-02-24T22:15:34.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 10 manifest: total_size = 11534336 2026-02-24T22:15:34.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2409765189372994385 2026-02-24T22:15:34.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515.0 2026-02-24T22:15:34.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b812e3f0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x55dc9a175f28 s->prefetch_data=0 2026-02-24T22:15:34.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2026-02-24T22:15:34.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2026-02-24T22:15:34.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2026-02-24T22:15:34.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2026-02-24T22:15:34.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2026-02-24T22:15:34.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2026-02-24T22:15:34.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2026-02-24T22:15:34.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2026-02-24T22:15:34.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2026-02-24T22:15:34.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2409765189372994385 2026-02-24T22:15:34.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515__multipart_foo-LUKEWARM.2~UQF9y4W3iByiBZqrzUJprUfj83wj_ro.1 2026-02-24T22:15:34.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 377 2026-02-24T22:15:34.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515__multipart_foo-LUKEWARM.2~UQF9y4W3iByiBZqrzUJprUfj83wj_ro.2 2026-02-24T22:15:34.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 654 2026-02-24T22:15:34.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515__multipart_foo-LUKEWARM.2~UQF9y4W3iByiBZqrzUJprUfj83wj_ro.3 2026-02-24T22:15:34.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 931 2026-02-24T22:15:34.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.787+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.22tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2409765189372994385 2026-02-24T22:15:34.790 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2026-02-24T22:15:34.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b80a5040 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x55dc9a1786d8 s->prefetch_data=0 2026-02-24T22:15:34.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 10 manifest: total_size = 541681 2026-02-24T22:15:34.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18117921011956775801 2026-02-24T22:15:34.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515.0 2026-02-24T22:15:34.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b80a5040 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x55dc9a1786d8 s->prefetch_data=0 2026-02-24T22:15:34.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2026-02-24T22:15:34.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=246247 2026-02-24T22:15:34.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=246247 stripe_ofs=246247 part_ofs=246247 rule->part_size=246247 2026-02-24T22:15:34.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2026-02-24T22:15:34.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6537703 part_ofs=246247 rule->part_size=246247 2026-02-24T22:15:34.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=492494 stripe_ofs=492494 part_ofs=492494 rule->part_size=49187 2026-02-24T22:15:34.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=49187 rules.size()=2 2026-02-24T22:15:34.792 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 RGWObjManifest::operator++(): stripe_ofs=6783950 part_ofs=492494 rule->part_size=49187 2026-02-24T22:15:34.792 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=541681 stripe_ofs=541681 part_ofs=541681 rule->part_size=49187 2026-02-24T22:15:34.792 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18117921011956775801 2026-02-24T22:15:34.792 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.792 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515__multipart_foo-STANDARD.2~FbIvQzmk1UO-cPN9QEsQGk3xJ3CxYDu.1 2026-02-24T22:15:34.792 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 369 2026-02-24T22:15:34.792 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515__multipart_foo-STANDARD.2~FbIvQzmk1UO-cPN9QEsQGk3xJ3CxYDu.2 2026-02-24T22:15:34.792 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 637 2026-02-24T22:15:34.792 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515__multipart_foo-STANDARD.2~FbIvQzmk1UO-cPN9QEsQGk3xJ3CxYDu.3 2026-02-24T22:15:34.792 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 905 2026-02-24T22:15:34.792 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.792 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.8tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18117921011956775801 2026-02-24T22:15:34.793 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f2997fff640 20 handle_completion(): completion ok for obj=foo-STANDARD 2026-02-24T22:15:34.793 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 list_objects_ordered: starting attempt 1 2026-02-24T22:15:34.793 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2026-02-24T22:15:34.793 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2026-02-24T22:15:34.793 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2026-02-24T22:15:34.793 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2026-02-24T22:15:34.793 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2026-02-24T22:15:34.793 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.791+0000 7f29cf777a80 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart.1[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515])"; uploads.size()=0; is_truncated=0 2026-02-24T22:15:34.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.795+0000 7f29cf777a80 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : miss 2026-02-24T22:15:34.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.795+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.795+0000 7f29cf777a80 20 rados_obj.operate() r=0 bl.length=242 2026-02-24T22:15:34.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.795+0000 7f29cf777a80 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x11 2026-02-24T22:15:34.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.795+0000 7f29cf777a80 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2026-02-24T22:15:34.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.795+0000 7f29cf777a80 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 from cache 2026-02-24T22:15:34.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.795+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart.1, ofs0, ns] 2026-02-24T22:15:34.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.795+0000 7f29a8ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961383 cookie 94405968749408 notifier 6228 bl.length()=194 2026-02-24T22:15:34.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.795+0000 7f29cf777a80 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515 from cache 2026-02-24T22:15:34.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.795+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515, ofs0, ns] 2026-02-24T22:15:34.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.795+0000 7f2997fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554960004 cookie 94405970383248 notifier 6228 bl.length()=254 2026-02-24T22:15:34.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.799+0000 7f29cf777a80 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515 : miss 2026-02-24T22:15:34.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.799+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.799+0000 7f29cf777a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:15:34.796 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.799+0000 7f29cf777a80 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515 info.flags=0x0 2026-02-24T22:15:34.796 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.799+0000 7f29cf777a80 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515 to cache LRU end 2026-02-24T22:15:34.797 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.799+0000 7f29cf777a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1514 : miss 2026-02-24T22:15:34.797 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.799+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.797 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.799+0000 7f29cf777a80 20 rados_obj.operate() r=0 bl.length=462 2026-02-24T22:15:34.797 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.799+0000 7f29cf777a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1514 info.flags=0x17 2026-02-24T22:15:34.798 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.799+0000 7f29cf777a80 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1514 to cache LRU end 2026-02-24T22:15:34.798 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.799+0000 7f29cf777a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:15:34.798 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.799+0000 7f29cf777a80 10 updating xattr: name=user.rgw.acl bl.length()=209 2026-02-24T22:15:34.798 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.799+0000 7f29cf777a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1514 2026-02-24T22:15:34.798 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.799+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-FROZEN 2026-02-24T22:15:34.798 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.799+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-LUKEWARM 2026-02-24T22:15:34.798 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.799+0000 7f29cf777a80 10 cls_bucket_list_unordered: got foo-STANDARD 2026-02-24T22:15:34.798 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.799+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a4a8980 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x55dc9a176428 s->prefetch_data=0 2026-02-24T22:15:34.799 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.799+0000 7f29cf777a80 10 manifest: total_size = 10485760 2026-02-24T22:15:34.799 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.799+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4740371126170548554 2026-02-24T22:15:34.799 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.799+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1514.0 2026-02-24T22:15:34.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.803+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a4a8980 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x55dc9a176428 s->prefetch_data=0 2026-02-24T22:15:34.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.803+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.803+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.803+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.803+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.803+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4740371126170548554 2026-02-24T22:15:34.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.803+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.801 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.803+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1514__shadow_.N46S_0BhEFOUya4Zwxt-QeVsF39DQtA_0 2026-02-24T22:15:34.801 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.803+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 341 2026-02-24T22:15:34.801 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.803+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1514__shadow_.N46S_0BhEFOUya4Zwxt-QeVsF39DQtA_1 2026-02-24T22:15:34.801 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.803+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 582 2026-02-24T22:15:34.801 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.803+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.801 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.803+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.29tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4740371126170548554 2026-02-24T22:15:34.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.803+0000 7f2997fff640 20 handle_completion(): completion ok for obj=foo-FROZEN 2026-02-24T22:15:34.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.803+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b80a4770 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x55dc9a176428 s->prefetch_data=0 2026-02-24T22:15:34.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.803+0000 7f29cf777a80 10 manifest: total_size = 10485760 2026-02-24T22:15:34.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.803+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15541942487287088756 2026-02-24T22:15:34.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.803+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1514.0 2026-02-24T22:15:34.803 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b80a4770 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x55dc9a176428 s->prefetch_data=0 2026-02-24T22:15:34.803 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.803 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.803 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.803 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.803 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15541942487287088756 2026-02-24T22:15:34.803 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.803 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1514__shadow_.PvSGPD7Ox7xQx-3vb1DSf09f7jCO-zh_0 2026-02-24T22:15:34.803 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 344 2026-02-24T22:15:34.804 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1514__shadow_.PvSGPD7Ox7xQx-3vb1DSf09f7jCO-zh_1 2026-02-24T22:15:34.804 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 587 2026-02-24T22:15:34.804 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.804 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.21tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15541942487287088756 2026-02-24T22:15:34.804 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2026-02-24T22:15:34.804 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c0070150 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x55dc9a176428 s->prefetch_data=0 2026-02-24T22:15:34.805 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29cf777a80 10 manifest: total_size = 44258 2026-02-24T22:15:34.805 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7566596603028383589 2026-02-24T22:15:34.805 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1514.0 2026-02-24T22:15:34.806 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c0070150 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x55dc9a176428 s->prefetch_data=0 2026-02-24T22:15:34.806 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.806 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=44258 stripe_ofs=44258 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.806 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7566596603028383589 2026-02-24T22:15:34.806 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.806 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1514__shadow_.MfeKSHY3n6XK6cjaWNSd-2QVQOvM8H9_0 2026-02-24T22:15:34.806 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 334 2026-02-24T22:15:34.806 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.807 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.807+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.13tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7566596603028383589 2026-02-24T22:15:34.807 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.811+0000 7f2997fff640 20 handle_completion(): completion ok for obj=foo-STANDARD 2026-02-24T22:15:34.807 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.811+0000 7f29cf777a80 20 list_objects_ordered: starting attempt 1 2026-02-24T22:15:34.807 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.811+0000 7f29cf777a80 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2026-02-24T22:15:34.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.811+0000 7f29cf777a80 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2026-02-24T22:15:34.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.811+0000 7f29cf777a80 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2026-02-24T22:15:34.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.811+0000 7f29cf777a80 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2026-02-24T22:15:34.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.811+0000 7f29cf777a80 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2026-02-24T22:15:34.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.811+0000 7f29cf777a80 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class.1[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1514])"; uploads.size()=0; is_truncated=0 2026-02-24T22:15:34.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.811+0000 7f29cf777a80 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : miss 2026-02-24T22:15:34.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.811+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.811+0000 7f29cf777a80 20 rados_obj.operate() r=0 bl.length=232 2026-02-24T22:15:34.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.811+0000 7f29cf777a80 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x11 2026-02-24T22:15:34.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.811+0000 7f29cf777a80 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2026-02-24T22:15:34.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.811+0000 7f29cf777a80 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 from cache 2026-02-24T22:15:34.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.811+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class.1, ofs0, ns] 2026-02-24T22:15:34.810 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.811+0000 7f29a8ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959897 cookie 94405968772512 notifier 6228 bl.length()=184 2026-02-24T22:15:34.811 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.815+0000 7f29cf777a80 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1514 from cache 2026-02-24T22:15:34.811 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.815+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1514, ofs0, ns] 2026-02-24T22:15:34.811 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.815+0000 7f2997fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961315 cookie 94405970389184 notifier 6228 bl.length()=244 2026-02-24T22:15:34.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.815+0000 7f29cf777a80 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1514 : miss 2026-02-24T22:15:34.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.815+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.815+0000 7f29cf777a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:15:34.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.815+0000 7f29cf777a80 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1514 info.flags=0x0 2026-02-24T22:15:34.813 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.815+0000 7f29cf777a80 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1514 to cache LRU end 2026-02-24T22:15:34.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.815+0000 7f29cf777a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1507 : miss 2026-02-24T22:15:34.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.815+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.815+0000 7f29cf777a80 20 rados_obj.operate() r=0 bl.length=459 2026-02-24T22:15:34.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.815+0000 7f29cf777a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1507 info.flags=0x17 2026-02-24T22:15:34.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.815+0000 7f29cf777a80 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1507 to cache LRU end 2026-02-24T22:15:34.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.815+0000 7f29cf777a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:15:34.815 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.815+0000 7f29cf777a80 10 updating xattr: name=user.rgw.acl bl.length()=209 2026-02-24T22:15:34.815 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.815+0000 7f29cf777a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1507 2026-02-24T22:15:34.815 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.815+0000 7f29cf777a80 10 cls_bucket_list_unordered: got __ 2026-02-24T22:15:34.815 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.815+0000 7f29cf777a80 10 cls_bucket_list_unordered: got __ _ 2026-02-24T22:15:34.815 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.815+0000 7f29cf777a80 10 cls_bucket_list_unordered: got ___ 2026-02-24T22:15:34.815 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.815+0000 7f29cf777a80 10 cls_bucket_list_unordered: got obj 2026-02-24T22:15:34.815 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.815+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a4ad140 obj=test-client.0-r-test-small-obj-data.1:__ state=0x55dc9a1786d8 s->prefetch_data=0 2026-02-24T22:15:34.816 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.819+0000 7f29cf777a80 10 manifest: total_size = 0 2026-02-24T22:15:34.816 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.819+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9574520592776221013 2026-02-24T22:15:34.816 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.819+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1507.0 2026-02-24T22:15:34.816 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.819+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a4ad140 obj=test-client.0-r-test-small-obj-data.1:__ state=0x55dc9a1786d8 s->prefetch_data=0 2026-02-24T22:15:34.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.819+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a4ad140 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x55dc9a4bacc8 s->prefetch_data=0 2026-02-24T22:15:34.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.819+0000 7f29cf777a80 10 manifest: total_size = 0 2026-02-24T22:15:34.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.819+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2012709011602489571 2026-02-24T22:15:34.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.819+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1507.0 2026-02-24T22:15:34.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.819+0000 7f2997fff640 20 handle_completion(): completion ok for obj=__ 2026-02-24T22:15:34.818 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.819+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a4ad140 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x55dc9a4bacc8 s->prefetch_data=0 2026-02-24T22:15:34.818 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.819+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c402cc30 obj=test-client.0-r-test-small-obj-data.1:___ state=0x55dc9a4bacc8 s->prefetch_data=0 2026-02-24T22:15:34.819 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.823+0000 7f29cf777a80 10 manifest: total_size = 0 2026-02-24T22:15:34.819 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.823+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8490448534132486380 2026-02-24T22:15:34.819 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.823+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1507.0 2026-02-24T22:15:34.819 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.823+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=__ _ 2026-02-24T22:15:34.820 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.823+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c402cc30 obj=test-client.0-r-test-small-obj-data.1:___ state=0x55dc9a4bacc8 s->prefetch_data=0 2026-02-24T22:15:34.820 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.823+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c00878f0 obj=test-client.0-r-test-small-obj-data.1:obj state=0x55dc9a4bacc8 s->prefetch_data=0 2026-02-24T22:15:34.821 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.823+0000 7f29cf777a80 10 manifest: total_size = 0 2026-02-24T22:15:34.821 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.823+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3118164009285580896 2026-02-24T22:15:34.821 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.823+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1507.0 2026-02-24T22:15:34.821 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.823+0000 7f2997fff640 20 handle_completion(): completion ok for obj=___ 2026-02-24T22:15:34.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.823+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c00878f0 obj=test-client.0-r-test-small-obj-data.1:obj state=0x55dc9a4bacc8 s->prefetch_data=0 2026-02-24T22:15:34.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.823+0000 7f29cf777a80 20 list_objects_ordered: starting attempt 1 2026-02-24T22:15:34.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.823+0000 7f29cf777a80 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2026-02-24T22:15:34.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.827+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=obj 2026-02-24T22:15:34.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.827+0000 7f29cf777a80 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2026-02-24T22:15:34.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.827+0000 7f29cf777a80 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2026-02-24T22:15:34.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.827+0000 7f29cf777a80 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2026-02-24T22:15:34.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.827+0000 7f29cf777a80 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2026-02-24T22:15:34.824 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.827+0000 7f29cf777a80 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.1[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1507])"; uploads.size()=0; is_truncated=0 2026-02-24T22:15:34.824 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.827+0000 7f29cf777a80 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : miss 2026-02-24T22:15:34.824 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.827+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.824 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.827+0000 7f29cf777a80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:15:34.825 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.827+0000 7f29cf777a80 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x11 2026-02-24T22:15:34.825 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.827+0000 7f29cf777a80 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2026-02-24T22:15:34.825 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.827+0000 7f29cf777a80 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 from cache 2026-02-24T22:15:34.825 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.827+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-small-obj-data.1, ofs0, ns] 2026-02-24T22:15:34.826 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.827+0000 7f2997fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959988 cookie 94405970382176 notifier 6228 bl.length()=181 2026-02-24T22:15:34.827 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.831+0000 7f29cf777a80 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1507 from cache 2026-02-24T22:15:34.827 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.831+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.1:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1507, ofs0, ns] 2026-02-24T22:15:34.827 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.831+0000 7f29a8ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959891 cookie 94405968768752 notifier 6228 bl.length()=241 2026-02-24T22:15:34.828 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.831+0000 7f29cf777a80 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1507 : miss 2026-02-24T22:15:34.828 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.831+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.828 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.831+0000 7f29cf777a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:15:34.828 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.831+0000 7f29cf777a80 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1507 info.flags=0x0 2026-02-24T22:15:34.829 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.831+0000 7f29cf777a80 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.1/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1507 to cache LRU end 2026-02-24T22:15:34.830 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.831+0000 7f29cf777a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1508 : miss 2026-02-24T22:15:34.830 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.831+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.830 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.831+0000 7f29cf777a80 20 rados_obj.operate() r=0 bl.length=459 2026-02-24T22:15:34.831 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.831+0000 7f29cf777a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1508 info.flags=0x17 2026-02-24T22:15:34.831 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.831+0000 7f29cf777a80 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1508 to cache LRU end 2026-02-24T22:15:34.831 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.831+0000 7f29cf777a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:15:34.831 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.831+0000 7f29cf777a80 10 updating xattr: name=user.rgw.acl bl.length()=209 2026-02-24T22:15:34.831 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.831+0000 7f29cf777a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1508 2026-02-24T22:15:34.831 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.835+0000 7f29cf777a80 10 cls_bucket_list_unordered: got __ 2026-02-24T22:15:34.831 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.835+0000 7f29cf777a80 10 cls_bucket_list_unordered: got __ _ 2026-02-24T22:15:34.831 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.835+0000 7f29cf777a80 10 cls_bucket_list_unordered: got ___ 2026-02-24T22:15:34.831 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.835+0000 7f29cf777a80 10 cls_bucket_list_unordered: got obj 2026-02-24T22:15:34.831 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.835+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c006f4e0 obj=test-client.0-r-test-small-obj-data.2:__ state=0x55dc9a4aaa78 s->prefetch_data=0 2026-02-24T22:15:34.831 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.835+0000 7f29cf777a80 10 manifest: total_size = 38 2026-02-24T22:15:34.832 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.835+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11343189274247675259 2026-02-24T22:15:34.832 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.835+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1508.0 2026-02-24T22:15:34.833 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.835+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c006f4e0 obj=test-client.0-r-test-small-obj-data.2:__ state=0x55dc9a4aaa78 s->prefetch_data=0 2026-02-24T22:15:34.833 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.835+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.833 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.835+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=38 stripe_ofs=38 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.833 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.835+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11343189274247675259 2026-02-24T22:15:34.833 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.835+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.833 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.835+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1508__shadow_.kBc0WU5eDwqW85uZimjNJv1eLL83L_j_0 2026-02-24T22:15:34.833 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.835+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 335 2026-02-24T22:15:34.833 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.835+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.833 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.835+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11343189274247675259 2026-02-24T22:15:34.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.835+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=__ 2026-02-24T22:15:34.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.835+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c006f4e0 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x55dc9a1786d8 s->prefetch_data=0 2026-02-24T22:15:34.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.835+0000 7f29cf777a80 10 manifest: total_size = 2078 2026-02-24T22:15:34.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.835+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8640604134508455727 2026-02-24T22:15:34.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.835+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1508.0 2026-02-24T22:15:34.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.839+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c006f4e0 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x55dc9a1786d8 s->prefetch_data=0 2026-02-24T22:15:34.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.839+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.839+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=2078 stripe_ofs=2078 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.839+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8640604134508455727 2026-02-24T22:15:34.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.839+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.839+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1508__shadow_.b6CswsPtCBa2VmIODmGbByFBjOUtXWy_0 2026-02-24T22:15:34.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.839+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 334 2026-02-24T22:15:34.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.839+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.839+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.3tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8640604134508455727 2026-02-24T22:15:34.837 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.839+0000 7f2997fff640 20 handle_completion(): completion ok for obj=__ _ 2026-02-24T22:15:34.837 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.839+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b80a4770 obj=test-client.0-r-test-small-obj-data.2:___ state=0x55dc9a4aaa78 s->prefetch_data=0 2026-02-24T22:15:34.837 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.839+0000 7f29cf777a80 10 manifest: total_size = 38 2026-02-24T22:15:34.837 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.839+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17966126535872513209 2026-02-24T22:15:34.837 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.839+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1508.0 2026-02-24T22:15:34.839 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.843+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b80a4770 obj=test-client.0-r-test-small-obj-data.2:___ state=0x55dc9a4aaa78 s->prefetch_data=0 2026-02-24T22:15:34.839 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.843+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.839 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.843+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=38 stripe_ofs=38 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.839 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.843+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17966126535872513209 2026-02-24T22:15:34.839 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.843+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.839 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.843+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1508__shadow_.lYElLIF4Mtlfsvm-eHTqZl6S-7ZT7RQ_0 2026-02-24T22:15:34.839 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.843+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 335 2026-02-24T22:15:34.839 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.843+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.839 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.843+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.15tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17966126535872513209 2026-02-24T22:15:34.840 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.843+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=___ 2026-02-24T22:15:34.840 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.843+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c006f4e0 obj=test-client.0-r-test-small-obj-data.2:obj state=0x55dc9a4aaa78 s->prefetch_data=0 2026-02-24T22:15:34.840 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.843+0000 7f29cf777a80 10 manifest: total_size = 2078 2026-02-24T22:15:34.840 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.843+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15417456784467582125 2026-02-24T22:15:34.840 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.843+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1508.0 2026-02-24T22:15:34.842 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.843+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c006f4e0 obj=test-client.0-r-test-small-obj-data.2:obj state=0x55dc9a4aaa78 s->prefetch_data=0 2026-02-24T22:15:34.842 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.843+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.842 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.843+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=2078 stripe_ofs=2078 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.842 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.843+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15417456784467582125 2026-02-24T22:15:34.842 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.843+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.842 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.843+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1508__shadow_.euiyy4tPIEvRNg9E7wHR8g4NX9Qp2EJ_0 2026-02-24T22:15:34.842 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.843+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 335 2026-02-24T22:15:34.842 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.843+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.842 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.843+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.11tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15417456784467582125 2026-02-24T22:15:34.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.847+0000 7f2997fff640 20 handle_completion(): completion ok for obj=obj 2026-02-24T22:15:34.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.847+0000 7f29cf777a80 20 list_objects_ordered: starting attempt 1 2026-02-24T22:15:34.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.847+0000 7f29cf777a80 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2026-02-24T22:15:34.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.847+0000 7f29cf777a80 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2026-02-24T22:15:34.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.847+0000 7f29cf777a80 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2026-02-24T22:15:34.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.847+0000 7f29cf777a80 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2026-02-24T22:15:34.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.847+0000 7f29cf777a80 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2026-02-24T22:15:34.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.847+0000 7f29cf777a80 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.2[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1508])"; uploads.size()=0; is_truncated=0 2026-02-24T22:15:34.844 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.847+0000 7f29cf777a80 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : miss 2026-02-24T22:15:34.845 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.847+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.845 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.847+0000 7f29cf777a80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:15:34.845 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.847+0000 7f29cf777a80 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x11 2026-02-24T22:15:34.845 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.847+0000 7f29cf777a80 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2026-02-24T22:15:34.846 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.847+0000 7f29cf777a80 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 from cache 2026-02-24T22:15:34.846 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.847+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-small-obj-data.2, ofs0, ns] 2026-02-24T22:15:34.846 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.847+0000 7f29a8ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959989 cookie 94405970382176 notifier 6228 bl.length()=181 2026-02-24T22:15:34.847 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.851+0000 7f29cf777a80 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1508 from cache 2026-02-24T22:15:34.847 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.851+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.2:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1508, ofs0, ns] 2026-02-24T22:15:34.847 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.851+0000 7f2997fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959892 cookie 94405968768752 notifier 6228 bl.length()=241 2026-02-24T22:15:34.848 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.851+0000 7f29cf777a80 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.2/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1508 : miss 2026-02-24T22:15:34.848 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.851+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.849 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.851+0000 7f29cf777a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:15:34.849 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.851+0000 7f29cf777a80 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.2/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1508 info.flags=0x0 2026-02-24T22:15:34.849 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.851+0000 7f29cf777a80 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.2/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1508 to cache LRU end 2026-02-24T22:15:34.850 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.855+0000 7f29cf777a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1509 : miss 2026-02-24T22:15:34.850 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.855+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.851 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.855+0000 7f29cf777a80 20 rados_obj.operate() r=0 bl.length=459 2026-02-24T22:15:34.851 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.855+0000 7f29cf777a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1509 info.flags=0x17 2026-02-24T22:15:34.851 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.855+0000 7f29cf777a80 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1509 to cache LRU end 2026-02-24T22:15:34.851 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.855+0000 7f29cf777a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:15:34.851 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.855+0000 7f29cf777a80 10 updating xattr: name=user.rgw.acl bl.length()=209 2026-02-24T22:15:34.851 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.855+0000 7f29cf777a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1509 2026-02-24T22:15:34.851 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.855+0000 7f29cf777a80 10 cls_bucket_list_unordered: got __ 2026-02-24T22:15:34.852 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.855+0000 7f29cf777a80 10 cls_bucket_list_unordered: got __ _ 2026-02-24T22:15:34.852 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.855+0000 7f29cf777a80 10 cls_bucket_list_unordered: got ___ 2026-02-24T22:15:34.852 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.855+0000 7f29cf777a80 10 cls_bucket_list_unordered: got obj 2026-02-24T22:15:34.852 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.855+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c0073c40 obj=test-client.0-r-test-small-obj-data.3:__ state=0x55dc9a4aaa78 s->prefetch_data=0 2026-02-24T22:15:34.852 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.855+0000 7f29cf777a80 10 manifest: total_size = 49187 2026-02-24T22:15:34.852 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.855+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14401360298554497472 2026-02-24T22:15:34.852 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.855+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1509.0 2026-02-24T22:15:34.854 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.855+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c0073c40 obj=test-client.0-r-test-small-obj-data.3:__ state=0x55dc9a4aaa78 s->prefetch_data=0 2026-02-24T22:15:34.854 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.855+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.854 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.855+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=49187 stripe_ofs=49187 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.854 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.855+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14401360298554497472 2026-02-24T22:15:34.854 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.855+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.854 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.855+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1509__shadow_.e7u9i7S9dKl2B_a6pDZisvf8QwJe33f_0 2026-02-24T22:15:34.854 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.855+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 335 2026-02-24T22:15:34.854 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.855+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.854 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.855+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14401360298554497472 2026-02-24T22:15:34.854 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.855+0000 7f2997fff640 20 handle_completion(): completion ok for obj=__ 2026-02-24T22:15:34.855 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.859+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c0073c40 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x55dc9a4aaa78 s->prefetch_data=0 2026-02-24T22:15:34.855 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.859+0000 7f29cf777a80 10 manifest: total_size = 1035 2026-02-24T22:15:34.855 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.859+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4180207770643741057 2026-02-24T22:15:34.855 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.859+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1509.0 2026-02-24T22:15:34.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.859+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29c0073c40 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x55dc9a4aaa78 s->prefetch_data=0 2026-02-24T22:15:34.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.859+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.859+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=1035 stripe_ofs=1035 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.859+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4180207770643741057 2026-02-24T22:15:34.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.859+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.859+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1509__shadow_.TtaC2q81tR68GthRWZMpiyQYGJZVmAg_0 2026-02-24T22:15:34.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.859+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 334 2026-02-24T22:15:34.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.859+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.859+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.19tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4180207770643741057 2026-02-24T22:15:34.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.859+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=__ _ 2026-02-24T22:15:34.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.859+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a57f140 obj=test-client.0-r-test-small-obj-data.3:___ state=0x55dc9a4aaa78 s->prefetch_data=0 2026-02-24T22:15:34.858 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.859+0000 7f29cf777a80 10 manifest: total_size = 1035 2026-02-24T22:15:34.858 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.859+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15953924534094524185 2026-02-24T22:15:34.858 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.859+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1509.0 2026-02-24T22:15:34.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.863+0000 7f29cf777a80 20 get_obj_state: octx=0x55dc9a57f140 obj=test-client.0-r-test-small-obj-data.3:___ state=0x55dc9a4aaa78 s->prefetch_data=0 2026-02-24T22:15:34.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.863+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.863+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=1035 stripe_ofs=1035 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.863+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15953924534094524185 2026-02-24T22:15:34.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.863+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.863+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1509__shadow_.YoLUFnLK5ZRrNRyZOMBdYRic6VB-wPW_0 2026-02-24T22:15:34.860 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.863+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 335 2026-02-24T22:15:34.860 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.863+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.860 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.863+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.12tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15953924534094524185 2026-02-24T22:15:34.860 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.863+0000 7f2997fff640 20 handle_completion(): completion ok for obj=___ 2026-02-24T22:15:34.860 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.863+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b812e3f0 obj=test-client.0-r-test-small-obj-data.3:obj state=0x55dc9a4aaa78 s->prefetch_data=0 2026-02-24T22:15:34.860 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.863+0000 7f29cf777a80 10 manifest: total_size = 49187 2026-02-24T22:15:34.861 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.863+0000 7f29cf777a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17046905199865487336 2026-02-24T22:15:34.861 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.863+0000 7f29cf777a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1509.0 2026-02-24T22:15:34.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.863+0000 7f29cf777a80 20 get_obj_state: octx=0x7f29b812e3f0 obj=test-client.0-r-test-small-obj-data.3:obj state=0x55dc9a4aaa78 s->prefetch_data=0 2026-02-24T22:15:34.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.863+0000 7f29cf777a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:15:34.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.863+0000 7f29cf777a80 20 RGWObjManifest::operator++(): result: ofs=49187 stripe_ofs=49187 part_ofs=0 rule->part_size=0 2026-02-24T22:15:34.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.863+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17046905199865487336 2026-02-24T22:15:34.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.863+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:15:34.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.863+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1509__shadow_.p4oMCcql17T6nBiRW_VJfDwhMxy7Jxf_0 2026-02-24T22:15:34.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.863+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 335 2026-02-24T22:15:34.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.863+0000 7f29cf777a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:15:34.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.863+0000 7f29cf777a80 20 garbage collection: RGWGC::send_chain - on object name: gc.15tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17046905199865487336 2026-02-24T22:15:34.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.867+0000 7f29a8ff9640 20 handle_completion(): completion ok for obj=obj 2026-02-24T22:15:34.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.867+0000 7f29cf777a80 20 list_objects_ordered: starting attempt 1 2026-02-24T22:15:34.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.867+0000 7f29cf777a80 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2026-02-24T22:15:34.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.867+0000 7f29cf777a80 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2026-02-24T22:15:34.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.867+0000 7f29cf777a80 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2026-02-24T22:15:34.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.867+0000 7f29cf777a80 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2026-02-24T22:15:34.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.867+0000 7f29cf777a80 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2026-02-24T22:15:34.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.867+0000 7f29cf777a80 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.3[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1509])"; uploads.size()=0; is_truncated=0 2026-02-24T22:15:34.864 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.867+0000 7f29cf777a80 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : miss 2026-02-24T22:15:34.864 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.867+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.867+0000 7f29cf777a80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:15:34.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.867+0000 7f29cf777a80 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x11 2026-02-24T22:15:34.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.867+0000 7f29cf777a80 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2026-02-24T22:15:34.866 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.867+0000 7f29cf777a80 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 from cache 2026-02-24T22:15:34.866 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.867+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-small-obj-data.3, ofs0, ns] 2026-02-24T22:15:34.866 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.867+0000 7f2997fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959990 cookie 94405970382176 notifier 6228 bl.length()=181 2026-02-24T22:15:34.867 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.871+0000 7f29cf777a80 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1509 from cache 2026-02-24T22:15:34.867 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.871+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.3:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1509, ofs0, ns] 2026-02-24T22:15:34.867 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.871+0000 7f2997fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959893 cookie 94405968768752 notifier 6228 bl.length()=241 2026-02-24T22:15:34.868 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.871+0000 7f29cf777a80 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.3/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1509 : miss 2026-02-24T22:15:34.868 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.871+0000 7f29cf777a80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.869 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.871+0000 7f29cf777a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:15:34.869 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.871+0000 7f29cf777a80 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.3/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1509 info.flags=0x0 2026-02-24T22:15:34.869 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.871+0000 7f29cf777a80 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.3/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1509 to cache LRU end 2026-02-24T22:15:34.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.871+0000 7f29cf777a80 10 removing key index: IVLSXKBUNCVSHAJHFPEJ 2026-02-24T22:15:34.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.875+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.keys:IVLSXKBUNCVSHAJHFPEJ, ofs0, ns] 2026-02-24T22:15:34.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.875+0000 7f2997fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959894 cookie 94405968768752 notifier 6228 bl.length()=170 2026-02-24T22:15:34.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.875+0000 7f29cf777a80 10 removing email index: ragweed.client.0+test@test.test 2026-02-24T22:15:34.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.875+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.email:ragweed.client.0+test@test.test, ofs0, ns] 2026-02-24T22:15:34.873 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.875+0000 7f29a8ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959898 cookie 94405968772512 notifier 6228 bl.length()=182 2026-02-24T22:15:34.873 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.875+0000 7f29cf777a80 10 removing user buckets index 2026-02-24T22:15:34.874 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.875+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:ragweed.client.0.buckets, ofs0, ns] 2026-02-24T22:15:34.874 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.875+0000 7f2997fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961384 cookie 94405968749408 notifier 6228 bl.length()=173 2026-02-24T22:15:34.874 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.875+0000 7f29cf777a80 10 removing user index: ragweed.client.0 2026-02-24T22:15:34.875 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.879+0000 7f29cf777a80 10 removing default.rgw.meta+users.uid+ragweed.client.0 from cache 2026-02-24T22:15:34.875 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.879+0000 7f29cf777a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.uid:ragweed.client.0, ofs0, ns] 2026-02-24T22:15:34.875 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.879+0000 7f29a8ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554960005 cookie 94405970383248 notifier 6228 bl.length()=165 2026-02-24T22:15:34.879 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.883+0000 7f29cf777a80 20 remove_watcher() i=3 2026-02-24T22:15:34.879 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.883+0000 7f29cf777a80 2 removed watcher, disabling cache 2026-02-24T22:15:34.879 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.883+0000 7f29cf777a80 20 remove_watcher() i=1 2026-02-24T22:15:34.879 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.883+0000 7f29cf777a80 20 remove_watcher() i=4 2026-02-24T22:15:34.879 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.883+0000 7f29cf777a80 20 remove_watcher() i=6 2026-02-24T22:15:34.879 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.883+0000 7f29cf777a80 20 remove_watcher() i=0 2026-02-24T22:15:34.879 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.883+0000 7f29cf777a80 20 remove_watcher() i=5 2026-02-24T22:15:34.879 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.883+0000 7f29cf777a80 20 remove_watcher() i=7 2026-02-24T22:15:34.879 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.883+0000 7f29cf777a80 20 remove_watcher() i=2 2026-02-24T22:15:34.886 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid sysuser.client.0 --purge-data 2026-02-24T22:15:34.944 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:15:34.944 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:15:34.960 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.963+0000 7f693068da80 4 Realm: () 2026-02-24T22:15:34.960 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.963+0000 7f693068da80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:15:34.960 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.963+0000 7f693068da80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:15:34.960 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.963+0000 7f693068da80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:15:34.960 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.963+0000 7f693068da80 20 zonegroup default 2026-02-24T22:15:34.960 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.963+0000 7f693068da80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.962 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.963+0000 7f693068da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:15:34.962 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.963+0000 7f693068da80 20 rados->read ofs=0 len=0 2026-02-24T22:15:34.962 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.963+0000 7f693068da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:15:34.962 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.963+0000 7f693068da80 20 started sync module instance, tier type = 2026-02-24T22:15:34.962 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.963+0000 7f693068da80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:15:34.963 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.967+0000 7f692e161640 0 -- 10.20.193.46:0/2417009535 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55e2caa42de0 msgr2=0x55e2caa43260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:15:34.964 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.967+0000 7f693068da80 20 add_watcher() i=1 2026-02-24T22:15:34.964 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.967+0000 7f693068da80 20 add_watcher() i=3 2026-02-24T22:15:34.964 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.967+0000 7f693068da80 20 add_watcher() i=5 2026-02-24T22:15:34.964 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.967+0000 7f693068da80 20 add_watcher() i=0 2026-02-24T22:15:34.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.967+0000 7f693068da80 20 add_watcher() i=4 2026-02-24T22:15:34.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.967+0000 7f693068da80 20 add_watcher() i=2 2026-02-24T22:15:34.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.967+0000 7f693068da80 20 add_watcher() i=7 2026-02-24T22:15:34.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.967+0000 7f693068da80 20 add_watcher() i=6 2026-02-24T22:15:34.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.967+0000 7f693068da80 2 all 8 watchers are set, enabling cache 2026-02-24T22:15:34.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.967+0000 7f692d960640 0 -- 10.20.193.46:0/47417360 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f6850008af0 msgr2=0x7f6850028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:15:34.967 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.971+0000 7f693068da80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:15:34.967 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.971+0000 7f693068da80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:15:34.990 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:34.991+0000 7f68477fe640 20 reqs_thread_entry: start 2026-02-24T22:15:35.023 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.027+0000 7f693068da80 20 init_complete bucket index max shards: 1 2026-02-24T22:15:35.023 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.027+0000 7f693068da80 20 Filter name: none 2026-02-24T22:15:35.024 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.027+0000 7f6845ffb640 20 reqs_thread_entry: start 2026-02-24T22:15:35.024 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.027+0000 7f693068da80 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2026-02-24T22:15:35.024 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.027+0000 7f693068da80 20 rados->read ofs=0 len=0 2026-02-24T22:15:35.024 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.027+0000 7f693068da80 20 rados_obj.operate() r=0 bl.length=486 2026-02-24T22:15:35.024 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.027+0000 7f693068da80 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x13 2026-02-24T22:15:35.025 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.027+0000 7f693068da80 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2026-02-24T22:15:35.025 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.027+0000 7f693068da80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:15:35.025 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.027+0000 7f693068da80 10 removing key index: YVXCPDVBWJHBKZOBYKCH 2026-02-24T22:15:35.025 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.027+0000 7f693068da80 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:users.keys:YVXCPDVBWJHBKZOBYKCH, ofs0, ns] 2026-02-24T22:15:35.026 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.027+0000 7f69097fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959991 cookie 94432550742976 notifier 6249 bl.length()=170 2026-02-24T22:15:35.026 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.027+0000 7f693068da80 10 removing email index: sysuser.client.0+test@test.test 2026-02-24T22:15:35.026 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.027+0000 7f693068da80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.email:sysuser.client.0+test@test.test, ofs0, ns] 2026-02-24T22:15:35.026 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.027+0000 7f6909ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554960006 cookie 94432551053728 notifier 6249 bl.length()=182 2026-02-24T22:15:35.026 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.027+0000 7f693068da80 10 removing user buckets index 2026-02-24T22:15:35.027 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.031+0000 7f693068da80 10 removing user index: sysuser.client.0 2026-02-24T22:15:35.028 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.031+0000 7f693068da80 10 removing default.rgw.meta+users.uid+sysuser.client.0 from cache 2026-02-24T22:15:35.028 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.031+0000 7f693068da80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.uid:sysuser.client.0, ofs0, ns] 2026-02-24T22:15:35.028 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.031+0000 7f6909ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959899 cookie 94432550746736 notifier 6249 bl.length()=165 2026-02-24T22:15:35.031 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.035+0000 7f693068da80 20 remove_watcher() i=3 2026-02-24T22:15:35.031 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.035+0000 7f693068da80 2 removed watcher, disabling cache 2026-02-24T22:15:35.031 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.035+0000 7f693068da80 20 remove_watcher() i=0 2026-02-24T22:15:35.032 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.035+0000 7f693068da80 20 remove_watcher() i=6 2026-02-24T22:15:35.032 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.035+0000 7f693068da80 20 remove_watcher() i=4 2026-02-24T22:15:35.032 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.035+0000 7f693068da80 20 remove_watcher() i=5 2026-02-24T22:15:35.032 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.035+0000 7f693068da80 20 remove_watcher() i=1 2026-02-24T22:15:35.032 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.035+0000 7f693068da80 20 remove_watcher() i=2 2026-02-24T22:15:35.032 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:15:35.035+0000 7f693068da80 20 remove_watcher() i=7 2026-02-24T22:15:35.038 INFO:tasks.ragweed:Removing ragweed... 2026-02-24T22:15:35.038 DEBUG:teuthology.orchestra.run.trial046:> rm -rf /home/ubuntu/cephtest/ragweed.client.0 2026-02-24T22:15:35.132 INFO:teuthology.run_tasks:Running task workunit... 2026-02-24T22:15:35.143 INFO:tasks.workunit:Pulling workunits from ref f7e05f4fdea5af173284f247d6b0935b337d4ccf 2026-02-24T22:15:35.143 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-24T22:15:35.144 DEBUG:teuthology.orchestra.run.trial046:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-24T22:15:35.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T22:15:35.147 INFO:teuthology.orchestra.run.trial046.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-24T22:15:35.148 DEBUG:teuthology.orchestra.run.trial046:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-24T22:15:35.194 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-24T22:15:35.195 DEBUG:teuthology.orchestra.run.trial046:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-24T22:15:35.238 INFO:tasks.workunit:timeout=3h 2026-02-24T22:15:35.238 INFO:tasks.workunit:cleanup=True 2026-02-24T22:15:35.239 DEBUG:teuthology.orchestra.run.trial046:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/samarahu/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout f7e05f4fdea5af173284f247d6b0935b337d4ccf 2026-02-24T22:15:35.283 INFO:tasks.workunit.client.0.trial046.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-24T22:15:58.184 INFO:tasks.workunit.client.0.trial046.stderr:Note: switching to 'f7e05f4fdea5af173284f247d6b0935b337d4ccf'. 2026-02-24T22:15:58.185 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:15:58.185 INFO:tasks.workunit.client.0.trial046.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-24T22:15:58.185 INFO:tasks.workunit.client.0.trial046.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-24T22:15:58.185 INFO:tasks.workunit.client.0.trial046.stderr:state without impacting any branches by switching back to a branch. 2026-02-24T22:15:58.185 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:15:58.185 INFO:tasks.workunit.client.0.trial046.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-24T22:15:58.185 INFO:tasks.workunit.client.0.trial046.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-24T22:15:58.185 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:15:58.185 INFO:tasks.workunit.client.0.trial046.stderr: git switch -c 2026-02-24T22:15:58.186 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:15:58.186 INFO:tasks.workunit.client.0.trial046.stderr:Or undo this operation with: 2026-02-24T22:15:58.186 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:15:58.186 INFO:tasks.workunit.client.0.trial046.stderr: git switch - 2026-02-24T22:15:58.186 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:15:58.186 INFO:tasks.workunit.client.0.trial046.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-24T22:15:58.186 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:15:58.186 INFO:tasks.workunit.client.0.trial046.stderr:HEAD is now at f7e05f4fdea rgw/d4n: Implement bucket check_empty and remove methods 2026-02-24T22:15:58.191 DEBUG:teuthology.orchestra.run.trial046:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-24T22:15:58.242 INFO:tasks.workunit.client.0.trial046.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2026-02-24T22:15:58.244 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-24T22:15:58.244 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2026-02-24T22:15:58.273 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2026-02-24T22:15:58.302 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2026-02-24T22:15:58.328 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-24T22:15:58.330 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-24T22:15:58.330 INFO:tasks.workunit.client.0.trial046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2026-02-24T22:15:58.355 INFO:tasks.workunit.client.0.trial046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-24T22:15:58.358 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:15:58.358 DEBUG:teuthology.orchestra.run.trial046:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-24T22:15:58.403 INFO:tasks.workunit:Running workunits matching rgw/run-reshard.sh on client.0... 2026-02-24T22:15:58.406 INFO:tasks.workunit:Running workunit rgw/run-reshard.sh... 2026-02-24T22:15:58.406 DEBUG:teuthology.orchestra.run.trial046:workunit test rgw/run-reshard.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7e05f4fdea5af173284f247d6b0935b337d4ccf TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-reshard.sh 2026-02-24T22:15:58.454 INFO:tasks.workunit.client.0.trial046.stderr:+ ulimit -c 0 2026-02-24T22:15:58.454 INFO:tasks.workunit.client.0.trial046.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-reshard.sh 2026-02-24T22:15:58.455 INFO:tasks.workunit.client.0.trial046.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2026-02-24T22:15:58.455 INFO:tasks.workunit.client.0.trial046.stderr:+ python3 -m venv /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2026-02-24T22:15:59.797 INFO:tasks.workunit.client.0.trial046.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/activate 2026-02-24T22:15:59.797 INFO:tasks.workunit.client.0.trial046.stderr:++ deactivate nondestructive 2026-02-24T22:15:59.797 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n '' ']' 2026-02-24T22:15:59.798 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n '' ']' 2026-02-24T22:15:59.798 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2026-02-24T22:15:59.798 INFO:tasks.workunit.client.0.trial046.stderr:++ hash -r 2026-02-24T22:15:59.798 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n '' ']' 2026-02-24T22:15:59.798 INFO:tasks.workunit.client.0.trial046.stderr:++ unset VIRTUAL_ENV 2026-02-24T22:15:59.798 INFO:tasks.workunit.client.0.trial046.stderr:++ unset VIRTUAL_ENV_PROMPT 2026-02-24T22:15:59.798 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' '!' nondestructive = nondestructive ']' 2026-02-24T22:15:59.798 INFO:tasks.workunit.client.0.trial046.stderr:++ VIRTUAL_ENV=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2026-02-24T22:15:59.798 INFO:tasks.workunit.client.0.trial046.stderr:++ export VIRTUAL_ENV 2026-02-24T22:15:59.798 INFO:tasks.workunit.client.0.trial046.stderr:++ _OLD_VIRTUAL_PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T22:15:59.798 INFO:tasks.workunit.client.0.trial046.stderr:++ PATH=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T22:15:59.798 INFO:tasks.workunit.client.0.trial046.stderr:++ export PATH 2026-02-24T22:15:59.799 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n '' ']' 2026-02-24T22:15:59.799 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -z '' ']' 2026-02-24T22:15:59.799 INFO:tasks.workunit.client.0.trial046.stderr:++ _OLD_VIRTUAL_PS1= 2026-02-24T22:15:59.799 INFO:tasks.workunit.client.0.trial046.stderr:++ PS1='(rgw) ' 2026-02-24T22:15:59.799 INFO:tasks.workunit.client.0.trial046.stderr:++ export PS1 2026-02-24T22:15:59.799 INFO:tasks.workunit.client.0.trial046.stderr:++ VIRTUAL_ENV_PROMPT='(rgw) ' 2026-02-24T22:15:59.799 INFO:tasks.workunit.client.0.trial046.stderr:++ export VIRTUAL_ENV_PROMPT 2026-02-24T22:15:59.799 INFO:tasks.workunit.client.0.trial046.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2026-02-24T22:15:59.799 INFO:tasks.workunit.client.0.trial046.stderr:++ hash -r 2026-02-24T22:15:59.799 INFO:tasks.workunit.client.0.trial046.stderr:+ pip install pip --upgrade 2026-02-24T22:15:59.973 INFO:tasks.workunit.client.0.trial046.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2026-02-24T22:16:00.043 INFO:tasks.workunit.client.0.trial046.stdout:Collecting pip 2026-02-24T22:16:00.047 INFO:tasks.workunit.client.0.trial046.stdout: Using cached pip-26.0.1-py3-none-any.whl (1.8 MB) 2026-02-24T22:16:00.063 INFO:tasks.workunit.client.0.trial046.stdout:Installing collected packages: pip 2026-02-24T22:16:00.063 INFO:tasks.workunit.client.0.trial046.stdout: Attempting uninstall: pip 2026-02-24T22:16:00.063 INFO:tasks.workunit.client.0.trial046.stdout: Found existing installation: pip 22.0.2 2026-02-24T22:16:00.238 INFO:tasks.workunit.client.0.trial046.stdout: Uninstalling pip-22.0.2: 2026-02-24T22:16:00.242 INFO:tasks.workunit.client.0.trial046.stdout: Successfully uninstalled pip-22.0.2 2026-02-24T22:16:00.587 INFO:tasks.workunit.client.0.trial046.stdout:Successfully installed pip-26.0.1 2026-02-24T22:16:00.615 INFO:tasks.workunit.client.0.trial046.stderr:+ pip install boto3 2026-02-24T22:16:00.944 INFO:tasks.workunit.client.0.trial046.stdout:Collecting boto3 2026-02-24T22:16:00.945 INFO:tasks.workunit.client.0.trial046.stdout: Using cached boto3-1.42.56-py3-none-any.whl.metadata (6.7 kB) 2026-02-24T22:16:01.079 INFO:tasks.workunit.client.0.trial046.stdout:Collecting botocore<1.43.0,>=1.42.56 (from boto3) 2026-02-24T22:16:01.080 INFO:tasks.workunit.client.0.trial046.stdout: Using cached botocore-1.42.56-py3-none-any.whl.metadata (5.9 kB) 2026-02-24T22:16:01.087 INFO:tasks.workunit.client.0.trial046.stdout:Collecting jmespath<2.0.0,>=0.7.1 (from boto3) 2026-02-24T22:16:01.087 INFO:tasks.workunit.client.0.trial046.stdout: Using cached jmespath-1.1.0-py3-none-any.whl.metadata (7.6 kB) 2026-02-24T22:16:01.096 INFO:tasks.workunit.client.0.trial046.stdout:Collecting s3transfer<0.17.0,>=0.16.0 (from boto3) 2026-02-24T22:16:01.096 INFO:tasks.workunit.client.0.trial046.stdout: Using cached s3transfer-0.16.0-py3-none-any.whl.metadata (1.7 kB) 2026-02-24T22:16:01.104 INFO:tasks.workunit.client.0.trial046.stdout:Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.43.0,>=1.42.56->boto3) 2026-02-24T22:16:01.105 INFO:tasks.workunit.client.0.trial046.stdout: Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) 2026-02-24T22:16:01.118 INFO:tasks.workunit.client.0.trial046.stdout:Collecting urllib3!=2.2.0,<3,>=1.25.4 (from botocore<1.43.0,>=1.42.56->boto3) 2026-02-24T22:16:01.119 INFO:tasks.workunit.client.0.trial046.stdout: Using cached urllib3-2.6.3-py3-none-any.whl.metadata (6.9 kB) 2026-02-24T22:16:01.127 INFO:tasks.workunit.client.0.trial046.stdout:Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.43.0,>=1.42.56->boto3) 2026-02-24T22:16:01.128 INFO:tasks.workunit.client.0.trial046.stdout: Using cached six-1.17.0-py2.py3-none-any.whl.metadata (1.7 kB) 2026-02-24T22:16:01.149 INFO:tasks.workunit.client.0.trial046.stdout:Using cached boto3-1.42.56-py3-none-any.whl (140 kB) 2026-02-24T22:16:01.150 INFO:tasks.workunit.client.0.trial046.stdout:Using cached botocore-1.42.56-py3-none-any.whl (14.6 MB) 2026-02-24T22:16:01.159 INFO:tasks.workunit.client.0.trial046.stdout:Using cached jmespath-1.1.0-py3-none-any.whl (20 kB) 2026-02-24T22:16:01.160 INFO:tasks.workunit.client.0.trial046.stdout:Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2026-02-24T22:16:01.160 INFO:tasks.workunit.client.0.trial046.stdout:Using cached s3transfer-0.16.0-py3-none-any.whl (86 kB) 2026-02-24T22:16:01.161 INFO:tasks.workunit.client.0.trial046.stdout:Using cached urllib3-2.6.3-py3-none-any.whl (131 kB) 2026-02-24T22:16:01.161 INFO:tasks.workunit.client.0.trial046.stdout:Using cached six-1.17.0-py2.py3-none-any.whl (11 kB) 2026-02-24T22:16:01.188 INFO:tasks.workunit.client.0.trial046.stdout:Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3 2026-02-24T22:16:01.521 INFO:tasks.workunit.client.0.trial046.stdout: 2026-02-24T22:16:01.522 INFO:tasks.workunit.client.0.trial046.stdout:Successfully installed boto3-1.42.56 botocore-1.42.56 jmespath-1.1.0 python-dateutil-2.9.0.post0 s3transfer-0.16.0 six-1.17.0 urllib3-2.6.3 2026-02-24T22:16:01.562 INFO:tasks.workunit.client.0.trial046.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/python3 /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_reshard.py 2026-02-24T22:16:01.633 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin user create --uid tester --display-name "Testing" --access-key NX5QOQKC6BH2IDN8HC7A --secret LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn 2026-02-24T22:16:01.650 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:16:01.650 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:16:01.668 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:01.670+0000 7f616fe42a80 4 Realm: () 2026-02-24T22:16:01.668 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:01.670+0000 7f616fe42a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:16:01.668 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:01.670+0000 7f616fe42a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:16:01.668 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:01.670+0000 7f616fe42a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:16:01.668 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:01.670+0000 7f616fe42a80 20 zonegroup default 2026-02-24T22:16:01.669 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:01.670+0000 7f616fe42a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:01.670 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:01.670+0000 7f616fe42a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:01.670 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:01.670+0000 7f616fe42a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:01.670 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:01.670+0000 7f616fe42a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:01.670 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:01.670+0000 7f616fe42a80 20 started sync module instance, tier type = 2026-02-24T22:16:01.670 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:01.670+0000 7f616fe42a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:16:01.671 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:01.674+0000 7f616d916640 0 -- 10.20.193.46:0/2457146103 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x556d264f8090 msgr2=0x556d26518510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:16:01.672 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:01.674+0000 7f616fe42a80 20 add_watcher() i=6 2026-02-24T22:16:01.672 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:01.674+0000 7f616fe42a80 20 add_watcher() i=1 2026-02-24T22:16:01.672 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:01.674+0000 7f616fe42a80 20 add_watcher() i=3 2026-02-24T22:16:01.672 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:01.674+0000 7f616fe42a80 20 add_watcher() i=0 2026-02-24T22:16:01.673 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:01.674+0000 7f616fe42a80 20 add_watcher() i=5 2026-02-24T22:16:01.673 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:01.674+0000 7f616fe42a80 20 add_watcher() i=7 2026-02-24T22:16:01.673 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:01.674+0000 7f616fe42a80 20 add_watcher() i=4 2026-02-24T22:16:01.877 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:01.878+0000 7f616fe42a80 20 add_watcher() i=2 2026-02-24T22:16:01.878 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:01.878+0000 7f616fe42a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:16:01.878 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:01.878+0000 7f616d115640 0 -- 10.20.193.46:0/2391391752 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f608c008af0 msgr2=0x7f608c028fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:02.081 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.082+0000 7f616fe42a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:16:02.081 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.082+0000 7f616fe42a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:16:02.081 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.082+0000 7f616fe42a80 5 note: GC not initialized 2026-02-24T22:16:02.082 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.082+0000 7f609e7fc640 20 reqs_thread_entry: start 2026-02-24T22:16:02.118 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.118+0000 7f616fe42a80 20 init_complete bucket index max shards: 1 2026-02-24T22:16:02.119 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.118+0000 7f616fe42a80 20 Filter name: none 2026-02-24T22:16:02.119 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.118+0000 7f609cff9640 20 reqs_thread_entry: start 2026-02-24T22:16:02.119 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.118+0000 7f616fe42a80 10 cache get: name=default.rgw.meta+users.uid+tester : miss 2026-02-24T22:16:02.119 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.118+0000 7f616fe42a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:02.119 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.118+0000 7f616fe42a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:02.119 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.118+0000 7f616fe42a80 10 cache put: name=default.rgw.meta+users.uid+tester info.flags=0x0 2026-02-24T22:16:02.119 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.118+0000 7f616fe42a80 10 adding default.rgw.meta+users.uid+tester to cache LRU end 2026-02-24T22:16:02.119 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.118+0000 7f616fe42a80 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : miss 2026-02-24T22:16:02.119 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.118+0000 7f616fe42a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:02.120 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.122+0000 7f616fe42a80 20 rados_obj.operate() r=0 bl.length=22 2026-02-24T22:16:02.120 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.122+0000 7f616fe42a80 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x5 2026-02-24T22:16:02.120 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.122+0000 7f616fe42a80 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2026-02-24T22:16:02.121 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.122+0000 7f616fe42a80 10 cache get: name=default.rgw.meta+users.uid+rgw_datacache_user : miss 2026-02-24T22:16:02.121 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.122+0000 7f616fe42a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:02.121 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.122+0000 7f616fe42a80 20 rados_obj.operate() r=0 bl.length=420 2026-02-24T22:16:02.121 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.122+0000 7f616fe42a80 10 cache put: name=default.rgw.meta+users.uid+rgw_datacache_user info.flags=0x13 2026-02-24T22:16:02.121 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.122+0000 7f616fe42a80 10 adding default.rgw.meta+users.uid+rgw_datacache_user to cache LRU end 2026-02-24T22:16:02.121 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.122+0000 7f616fe42a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:16:02.121 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.122+0000 7f616fe42a80 10 chain_cache_entry: cache_locator=default.rgw.meta+users.uid+rgw_datacache_user 2026-02-24T22:16:02.121 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.122+0000 7f616fe42a80 20 get_user_info_from_index found 0 xattrs 2026-02-24T22:16:02.121 INFO:tasks.workunit.client.0.trial046.stderr:could not create user: unable to parse parameters, duplicate key provided 2026-02-24T22:16:02.122 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.122+0000 7f616fe42a80 20 remove_watcher() i=0 2026-02-24T22:16:02.122 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.122+0000 7f616fe42a80 2 removed watcher, disabling cache 2026-02-24T22:16:02.122 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.122+0000 7f616fe42a80 20 remove_watcher() i=4 2026-02-24T22:16:02.123 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.122+0000 7f616fe42a80 20 remove_watcher() i=3 2026-02-24T22:16:02.123 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.122+0000 7f616fe42a80 20 remove_watcher() i=1 2026-02-24T22:16:02.123 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.122+0000 7f616fe42a80 20 remove_watcher() i=5 2026-02-24T22:16:02.123 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.122+0000 7f616fe42a80 20 remove_watcher() i=2 2026-02-24T22:16:02.123 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.122+0000 7f616fe42a80 20 remove_watcher() i=6 2026-02-24T22:16:02.123 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:02.122+0000 7f616fe42a80 20 remove_watcher() i=7 2026-02-24T22:16:22.528 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:22.529 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:22.573 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:22.573 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:22.584 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:22.584 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:22.593 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:22.593 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:22.597 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:22.597 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:22.639 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:22.640 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:22.683 INFO:tasks.workunit.client.0.trial046.stderr:TEST: reshard add 2026-02-24T22:16:22.683 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:22.683 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket a-bucket 2026-02-24T22:16:22.698 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:16:22.698 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:16:22.716 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.718+0000 7faa4da84a80 4 Realm: () 2026-02-24T22:16:22.716 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.718+0000 7faa4da84a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:16:22.716 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.718+0000 7faa4da84a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:16:22.716 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.718+0000 7faa4da84a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:16:22.716 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.718+0000 7faa4da84a80 20 zonegroup default 2026-02-24T22:16:22.716 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.718+0000 7faa4da84a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:22.718 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.718+0000 7faa4da84a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:22.718 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.718+0000 7faa4da84a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:22.718 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.722+0000 7faa4da84a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:22.719 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.722+0000 7faa4da84a80 20 started sync module instance, tier type = 2026-02-24T22:16:22.719 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.722+0000 7faa4da84a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:16:22.720 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.722+0000 7faa4da84a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:16:22.720 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.722+0000 7faa4da84a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:16:22.720 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.722+0000 7faa4da84a80 5 note: GC not initialized 2026-02-24T22:16:22.721 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.722+0000 7fa9f8fe1640 20 reqs_thread_entry: start 2026-02-24T22:16:22.724 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.726+0000 7faa477fe640 0 -- 10.20.193.46:0/1452916201 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x5643ea2ffaa0 msgr2=0x5643ea306210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:22.935 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.938+0000 7faa46ffd640 0 -- 10.20.193.46:0/1452916201 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5643ea301c90 msgr2=0x5643ea2fca00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:22.963 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.966+0000 7faa4da84a80 20 init_complete bucket index max shards: 1 2026-02-24T22:16:22.963 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.966+0000 7faa4da84a80 20 Filter name: none 2026-02-24T22:16:22.963 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.966+0000 7fa96affd640 20 reqs_thread_entry: start 2026-02-24T22:16:22.963 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.966+0000 7faa4da84a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:22.964 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.966+0000 7faa4da84a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:16:22.965 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.966+0000 7faa4da84a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:22.965 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.966+0000 7faa4da84a80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:16:22.965 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.966+0000 7faa4da84a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:16:22.965 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.966+0000 7faa4da84a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:22.965 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.966+0000 7faa4da84a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:16:22.965 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.966+0000 7faa4da84a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:22.965 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.966+0000 7faa4da84a80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:16:22.965 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:22.966+0000 7faa4da84a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:16:22.975 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:22.975 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:16:22.975 INFO:tasks.workunit.client.0.trial046.stderr:bucket: a-bucket id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 num_objects: 0 size_kb: 0 num_shards: 1 2026-02-24T22:16:22.975 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:22.976 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin reshard add --bucket a-bucket --num-shards 2 2026-02-24T22:16:22.991 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:16:22.991 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:16:23.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.010+0000 7f3686ac7a80 4 Realm: () 2026-02-24T22:16:23.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.010+0000 7f3686ac7a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:16:23.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.010+0000 7f3686ac7a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:16:23.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.010+0000 7f3686ac7a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:16:23.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.010+0000 7f3686ac7a80 20 zonegroup default 2026-02-24T22:16:23.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.010+0000 7f3686ac7a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:23.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.010+0000 7f3686ac7a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:23.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.010+0000 7f3686ac7a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:23.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.014+0000 7f3686ac7a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:23.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.014+0000 7f3686ac7a80 20 started sync module instance, tier type = 2026-02-24T22:16:23.011 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.014+0000 7f3686ac7a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:16:23.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.014+0000 7f3684d9c640 0 -- 10.20.193.46:0/4022091575 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55f76725e3c0 msgr2=0x55f76727e7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:23.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.014+0000 7f3686ac7a80 20 add_watcher() i=6 2026-02-24T22:16:23.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.014+0000 7f3686ac7a80 20 add_watcher() i=1 2026-02-24T22:16:23.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.014+0000 7f3686ac7a80 20 add_watcher() i=3 2026-02-24T22:16:23.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.014+0000 7f3686ac7a80 20 add_watcher() i=5 2026-02-24T22:16:23.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.014+0000 7f3686ac7a80 20 add_watcher() i=0 2026-02-24T22:16:23.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.014+0000 7f3686ac7a80 20 add_watcher() i=7 2026-02-24T22:16:23.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.014+0000 7f3686ac7a80 20 add_watcher() i=4 2026-02-24T22:16:23.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.014+0000 7f3686ac7a80 20 add_watcher() i=2 2026-02-24T22:16:23.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.014+0000 7f3686ac7a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:16:23.015 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.018+0000 7f3686ac7a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:16:23.015 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.018+0000 7f3686ac7a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:16:23.015 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.018+0000 7f3686ac7a80 5 note: GC not initialized 2026-02-24T22:16:23.015 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.018+0000 7f36317e2640 20 reqs_thread_entry: start 2026-02-24T22:16:23.016 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.018+0000 7f3684d9c640 0 -- 10.20.193.46:0/3950705724 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55f767273770 msgr2=0x55f767273b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:16:23.259 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.262+0000 7f3686ac7a80 20 init_complete bucket index max shards: 1 2026-02-24T22:16:23.259 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.262+0000 7f3686ac7a80 20 Filter name: none 2026-02-24T22:16:23.259 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.262+0000 7f359b7fe640 20 reqs_thread_entry: start 2026-02-24T22:16:23.259 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.262+0000 7f3686ac7a80 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2026-02-24T22:16:23.259 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.262+0000 7f3686ac7a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:23.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.262+0000 7f3686ac7a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:16:23.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.262+0000 7f3686ac7a80 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2026-02-24T22:16:23.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.262+0000 7f3686ac7a80 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2026-02-24T22:16:23.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.262+0000 7f3686ac7a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : miss 2026-02-24T22:16:23.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.262+0000 7f3686ac7a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:23.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.262+0000 7f3686ac7a80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:16:23.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.262+0000 7f3686ac7a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:16:23.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.262+0000 7f3686ac7a80 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:16:23.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.262+0000 7f3686ac7a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:16:23.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.262+0000 7f3686ac7a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:16:23.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.262+0000 7f3686ac7a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:16:23.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.266+0000 7f3686ac7a80 20 remove_watcher() i=3 2026-02-24T22:16:23.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.266+0000 7f3686ac7a80 2 removed watcher, disabling cache 2026-02-24T22:16:23.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.266+0000 7f3686ac7a80 20 remove_watcher() i=6 2026-02-24T22:16:23.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.266+0000 7f3686ac7a80 20 remove_watcher() i=4 2026-02-24T22:16:23.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.266+0000 7f3686ac7a80 20 remove_watcher() i=2 2026-02-24T22:16:23.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.266+0000 7f3686ac7a80 20 remove_watcher() i=5 2026-02-24T22:16:23.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.266+0000 7f3686ac7a80 20 remove_watcher() i=1 2026-02-24T22:16:23.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.266+0000 7f3686ac7a80 20 remove_watcher() i=0 2026-02-24T22:16:23.265 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.266+0000 7f3686ac7a80 20 remove_watcher() i=7 2026-02-24T22:16:23.270 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin reshard list 2026-02-24T22:16:23.283 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:16:23.283 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:16:23.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.302+0000 7f0b08047a80 4 Realm: () 2026-02-24T22:16:23.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.302+0000 7f0b08047a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:16:23.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.302+0000 7f0b08047a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:16:23.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.302+0000 7f0b08047a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:16:23.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.302+0000 7f0b08047a80 20 zonegroup default 2026-02-24T22:16:23.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.302+0000 7f0b08047a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:23.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.302+0000 7f0b08047a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:23.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.302+0000 7f0b08047a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:23.303 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.306+0000 7f0b08047a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:23.303 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.306+0000 7f0b08047a80 20 started sync module instance, tier type = 2026-02-24T22:16:23.303 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.306+0000 7f0b08047a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:16:23.304 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.306+0000 7f0b08047a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:16:23.304 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.306+0000 7f0b08047a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:16:23.304 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.306+0000 7f0b08047a80 5 note: GC not initialized 2026-02-24T22:16:23.304 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.306+0000 7f0a26ffd640 20 reqs_thread_entry: start 2026-02-24T22:16:23.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.310+0000 7f0b05b1b640 0 -- 10.20.193.46:0/1123446113 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55df835a9e00 msgr2=0x55df835aa250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:23.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.522+0000 7f0b0631c640 0 -- 10.20.193.46:0/1123446113 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x55df83597d10 msgr2=0x55df8359b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:23.748 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.750+0000 7f0b08047a80 20 init_complete bucket index max shards: 1 2026-02-24T22:16:23.748 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.750+0000 7f0b08047a80 20 Filter name: none 2026-02-24T22:16:23.748 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.750+0000 7f0a257fa640 20 reqs_thread_entry: start 2026-02-24T22:16:23.762 INFO:tasks.workunit.client.0.trial046.stderr:bucket name a-bucket 2026-02-24T22:16:23.762 INFO:tasks.workunit.client.0.trial046.stderr:TEST: reshard process 2026-02-24T22:16:23.762 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:23.762 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin reshard process 2026-02-24T22:16:23.775 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:16:23.775 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:16:23.792 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.794+0000 7f763773ba80 4 Realm: () 2026-02-24T22:16:23.792 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.794+0000 7f763773ba80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:16:23.792 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.794+0000 7f763773ba80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:16:23.792 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.794+0000 7f763773ba80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:16:23.792 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.794+0000 7f763773ba80 20 zonegroup default 2026-02-24T22:16:23.793 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.794+0000 7f763773ba80 20 rados->read ofs=0 len=0 2026-02-24T22:16:23.793 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.794+0000 7f763773ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:23.793 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.794+0000 7f763773ba80 20 rados->read ofs=0 len=0 2026-02-24T22:16:23.794 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.794+0000 7f763773ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:23.794 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.794+0000 7f763773ba80 20 started sync module instance, tier type = 2026-02-24T22:16:23.794 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.794+0000 7f763773ba80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:16:23.795 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.798+0000 7f7635a10640 0 -- 10.20.193.46:0/1679999159 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x557bb2f8ece0 msgr2=0x557bb2faf160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:23.796 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.798+0000 7f763773ba80 20 add_watcher() i=3 2026-02-24T22:16:23.796 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.798+0000 7f763773ba80 20 add_watcher() i=1 2026-02-24T22:16:23.796 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.798+0000 7f763773ba80 20 add_watcher() i=6 2026-02-24T22:16:23.796 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.798+0000 7f763773ba80 20 add_watcher() i=0 2026-02-24T22:16:23.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.798+0000 7f763773ba80 20 add_watcher() i=4 2026-02-24T22:16:23.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.798+0000 7f763773ba80 20 add_watcher() i=7 2026-02-24T22:16:23.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.798+0000 7f763773ba80 20 add_watcher() i=5 2026-02-24T22:16:23.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.798+0000 7f763773ba80 20 add_watcher() i=2 2026-02-24T22:16:23.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.798+0000 7f763773ba80 2 all 8 watchers are set, enabling cache 2026-02-24T22:16:23.798 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.798+0000 7f763773ba80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:16:23.798 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.798+0000 7f763773ba80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:16:23.798 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.798+0000 7f763773ba80 5 note: GC not initialized 2026-02-24T22:16:23.798 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.802+0000 7f75e27e4640 20 reqs_thread_entry: start 2026-02-24T22:16:23.799 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:23.802+0000 7f7635a10640 0 -- 10.20.193.46:0/232065253 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x557bb2fbdbb0 msgr2=0x557bb2fb89b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:16:24.040 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.042+0000 7f763773ba80 20 init_complete bucket index max shards: 1 2026-02-24T22:16:24.040 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.042+0000 7f763773ba80 20 Filter name: none 2026-02-24T22:16:24.040 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.042+0000 7f75e0fe1640 20 reqs_thread_entry: start 2026-02-24T22:16:24.040 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.042+0000 7f763773ba80 20 processing logshard = reshard.0000000000 2026-02-24T22:16:24.041 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.042+0000 7f763773ba80 20 finish processing logshard = reshard.0000000000 , ret = 0 2026-02-24T22:16:24.042 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.042+0000 7f763773ba80 20 processing logshard = reshard.0000000001 2026-02-24T22:16:24.043 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.046+0000 7f763773ba80 20 finish processing logshard = reshard.0000000001 , ret = 0 2026-02-24T22:16:24.044 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.046+0000 7f763773ba80 20 processing logshard = reshard.0000000002 2026-02-24T22:16:24.045 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.046+0000 7f763773ba80 20 finish processing logshard = reshard.0000000002 , ret = 0 2026-02-24T22:16:24.045 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.046+0000 7f763773ba80 20 processing logshard = reshard.0000000003 2026-02-24T22:16:24.047 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.050+0000 7f763773ba80 20 finish processing logshard = reshard.0000000003 , ret = 0 2026-02-24T22:16:24.047 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.050+0000 7f763773ba80 20 processing logshard = reshard.0000000004 2026-02-24T22:16:24.049 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.050+0000 7f763773ba80 20 finish processing logshard = reshard.0000000004 , ret = 0 2026-02-24T22:16:24.049 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.050+0000 7f763773ba80 20 processing logshard = reshard.0000000005 2026-02-24T22:16:24.051 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.054+0000 7f763773ba80 20 finish processing logshard = reshard.0000000005 , ret = 0 2026-02-24T22:16:24.051 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.054+0000 7f763773ba80 20 processing logshard = reshard.0000000006 2026-02-24T22:16:24.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.054+0000 7f763773ba80 20 process_entry resharding a-bucket 2026-02-24T22:16:24.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.054+0000 7f763773ba80 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2026-02-24T22:16:24.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.054+0000 7f763773ba80 20 rados->read ofs=0 len=0 2026-02-24T22:16:24.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.054+0000 7f763773ba80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:16:24.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.054+0000 7f763773ba80 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x1 2026-02-24T22:16:24.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.054+0000 7f763773ba80 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2026-02-24T22:16:24.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.054+0000 7f763773ba80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : miss 2026-02-24T22:16:24.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.054+0000 7f763773ba80 20 rados->read ofs=0 len=0 2026-02-24T22:16:24.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.054+0000 7f763773ba80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:16:24.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.054+0000 7f763773ba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:16:24.054 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.054+0000 7f763773ba80 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:16:24.054 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.054+0000 7f763773ba80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:16:24.054 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.054+0000 7f763773ba80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:16:24.054 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.054+0000 7f763773ba80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:16:24.058 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.058+0000 7f763773ba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:16:24.058 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.058+0000 7f763773ba80 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:16:24.058 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.058+0000 7f763773ba80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:16:24.058 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.058+0000 7f763773ba80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518, ofs0, ns] 2026-02-24T22:16:24.059 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f75fffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959896 cookie 93989772263904 notifier 6336 bl.length()=929 2026-02-24T22:16:24.059 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f75fffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:16:24.059 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f75fffff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:16:24.059 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f75fffff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:16:24.059 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f763773ba80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_sources= new_sources= 2026-02-24T22:16:24.059 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f763773ba80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential sources added=[] removed=[] 2026-02-24T22:16:24.059 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f763773ba80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_dests= new_dests= 2026-02-24T22:16:24.060 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f763773ba80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential dests added=[] removed=[] 2026-02-24T22:16:24.060 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f763773ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.1.0 2026-02-24T22:16:24.060 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f763773ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.1.0 2026-02-24T22:16:24.060 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f763773ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.1.1 2026-02-24T22:16:24.060 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f763773ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.1.1 2026-02-24T22:16:24.061 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f763773ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.0 2026-02-24T22:16:24.061 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f763773ba80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : hit (requested=0x17, cached=0x17) 2026-02-24T22:16:24.061 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f763773ba80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:16:24.062 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f763773ba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:16:24.062 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f763773ba80 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:16:24.062 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f763773ba80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:16:24.062 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f763773ba80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518, ofs0, ns] 2026-02-24T22:16:24.062 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f75fffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959897 cookie 93989772263904 notifier 6336 bl.length()=929 2026-02-24T22:16:24.062 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f75fffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:16:24.062 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f75fffff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:16:24.063 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f75fffff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:16:24.063 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f763773ba80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_sources= new_sources= 2026-02-24T22:16:24.063 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f763773ba80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential sources added=[] removed=[] 2026-02-24T22:16:24.063 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f763773ba80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_dests= new_dests= 2026-02-24T22:16:24.063 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.062+0000 7f763773ba80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential dests added=[] removed=[] 2026-02-24T22:16:24.064 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.066+0000 7f763773ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.0 2026-02-24T22:16:24.064 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.066+0000 7f763773ba80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : hit (requested=0x17, cached=0x17) 2026-02-24T22:16:24.064 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.066+0000 7f763773ba80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:16:24.064 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.066+0000 7f763773ba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:16:24.064 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.066+0000 7f763773ba80 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:16:24.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.066+0000 7f763773ba80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:16:24.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.066+0000 7f763773ba80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518, ofs0, ns] 2026-02-24T22:16:24.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.066+0000 7f75fffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959898 cookie 93989772263904 notifier 6336 bl.length()=893 2026-02-24T22:16:24.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.066+0000 7f75fffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:16:24.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.066+0000 7f75fffff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:16:24.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.066+0000 7f75fffff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:16:24.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.066+0000 7f763773ba80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_sources= new_sources= 2026-02-24T22:16:24.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.066+0000 7f763773ba80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential sources added=[] removed=[] 2026-02-24T22:16:24.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.066+0000 7f763773ba80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_dests= new_dests= 2026-02-24T22:16:24.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.066+0000 7f763773ba80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential dests added=[] removed=[] 2026-02-24T22:16:24.066 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.066+0000 7f763773ba80 1 execute INFO: reshard of bucket "a-bucket" from 1 shards to 2 shards completed successfully 2026-02-24T22:16:24.066 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.066+0000 7f763773ba80 20 process_entry removing reshard queue entry for bucket a-bucket 2026-02-24T22:16:24.067 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.070+0000 7f763773ba80 20 finish processing logshard = reshard.0000000006 , ret = 0 2026-02-24T22:16:24.067 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.070+0000 7f763773ba80 20 processing logshard = reshard.0000000007 2026-02-24T22:16:24.068 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.070+0000 7f763773ba80 20 finish processing logshard = reshard.0000000007 , ret = 0 2026-02-24T22:16:24.068 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.070+0000 7f763773ba80 20 processing logshard = reshard.0000000008 2026-02-24T22:16:24.070 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.070+0000 7f763773ba80 20 finish processing logshard = reshard.0000000008 , ret = 0 2026-02-24T22:16:24.070 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.070+0000 7f763773ba80 20 processing logshard = reshard.0000000009 2026-02-24T22:16:24.072 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.074+0000 7f763773ba80 20 finish processing logshard = reshard.0000000009 , ret = 0 2026-02-24T22:16:24.072 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.074+0000 7f763773ba80 20 processing logshard = reshard.0000000010 2026-02-24T22:16:24.073 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.074+0000 7f763773ba80 20 finish processing logshard = reshard.0000000010 , ret = 0 2026-02-24T22:16:24.073 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.074+0000 7f763773ba80 20 processing logshard = reshard.0000000011 2026-02-24T22:16:24.075 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.078+0000 7f763773ba80 20 finish processing logshard = reshard.0000000011 , ret = 0 2026-02-24T22:16:24.075 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.078+0000 7f763773ba80 20 processing logshard = reshard.0000000012 2026-02-24T22:16:24.076 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.078+0000 7f763773ba80 20 finish processing logshard = reshard.0000000012 , ret = 0 2026-02-24T22:16:24.077 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.078+0000 7f763773ba80 20 processing logshard = reshard.0000000013 2026-02-24T22:16:24.078 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.082+0000 7f763773ba80 20 finish processing logshard = reshard.0000000013 , ret = 0 2026-02-24T22:16:24.078 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.082+0000 7f763773ba80 20 processing logshard = reshard.0000000014 2026-02-24T22:16:24.080 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.082+0000 7f763773ba80 20 finish processing logshard = reshard.0000000014 , ret = 0 2026-02-24T22:16:24.080 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.082+0000 7f763773ba80 20 processing logshard = reshard.0000000015 2026-02-24T22:16:24.081 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.082+0000 7f763773ba80 20 finish processing logshard = reshard.0000000015 , ret = 0 2026-02-24T22:16:24.085 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.086+0000 7f763773ba80 20 remove_watcher() i=3 2026-02-24T22:16:24.085 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.086+0000 7f763773ba80 2 removed watcher, disabling cache 2026-02-24T22:16:24.085 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.086+0000 7f763773ba80 20 remove_watcher() i=0 2026-02-24T22:16:24.085 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.086+0000 7f763773ba80 20 remove_watcher() i=1 2026-02-24T22:16:24.085 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.086+0000 7f763773ba80 20 remove_watcher() i=6 2026-02-24T22:16:24.085 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.086+0000 7f763773ba80 20 remove_watcher() i=5 2026-02-24T22:16:24.086 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.086+0000 7f763773ba80 20 remove_watcher() i=2 2026-02-24T22:16:24.086 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.086+0000 7f763773ba80 20 remove_watcher() i=4 2026-02-24T22:16:24.086 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:24.086+0000 7f763773ba80 20 remove_watcher() i=7 2026-02-24T22:16:29.098 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket a-bucket 2026-02-24T22:16:29.114 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:16:29.114 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:16:29.132 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.133+0000 7fa3e9ffaa80 4 Realm: () 2026-02-24T22:16:29.132 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.133+0000 7fa3e9ffaa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:16:29.132 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.133+0000 7fa3e9ffaa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:16:29.132 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.133+0000 7fa3e9ffaa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:16:29.133 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.133+0000 7fa3e9ffaa80 20 zonegroup default 2026-02-24T22:16:29.133 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.133+0000 7fa3e9ffaa80 20 rados->read ofs=0 len=0 2026-02-24T22:16:29.134 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.133+0000 7fa3e9ffaa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:29.134 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.133+0000 7fa3e9ffaa80 20 rados->read ofs=0 len=0 2026-02-24T22:16:29.134 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.133+0000 7fa3e9ffaa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:29.134 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.133+0000 7fa3e9ffaa80 20 started sync module instance, tier type = 2026-02-24T22:16:29.134 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.133+0000 7fa3e9ffaa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:16:29.136 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.137+0000 7fa3e9ffaa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:16:29.136 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.137+0000 7fa3e9ffaa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:16:29.136 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.137+0000 7fa3e9ffaa80 5 note: GC not initialized 2026-02-24T22:16:29.136 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.137+0000 7fa3957e2640 20 reqs_thread_entry: start 2026-02-24T22:16:29.141 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.141+0000 7fa3e3fff640 0 -- 10.20.193.46:0/2948914962 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55c7d36726e0 msgr2=0x55c7d366c080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:29.147 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.149+0000 7fa3e37fe640 0 -- 10.20.193.46:0/2948914962 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55c7d3646d70 msgr2=0x55c7d3674890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:29.170 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.169+0000 7fa3e9ffaa80 20 init_complete bucket index max shards: 1 2026-02-24T22:16:29.170 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.169+0000 7fa3e9ffaa80 20 Filter name: none 2026-02-24T22:16:29.170 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.169+0000 7fa2ff7fe640 20 reqs_thread_entry: start 2026-02-24T22:16:29.170 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.169+0000 7fa3e9ffaa80 20 rados->read ofs=0 len=0 2026-02-24T22:16:29.171 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.169+0000 7fa3e9ffaa80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:16:29.171 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.169+0000 7fa3e9ffaa80 20 rados->read ofs=0 len=0 2026-02-24T22:16:29.171 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.173+0000 7fa3e9ffaa80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:16:29.171 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.173+0000 7fa3e9ffaa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:16:29.171 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.173+0000 7fa3e9ffaa80 20 rados->read ofs=0 len=0 2026-02-24T22:16:29.171 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.173+0000 7fa3e9ffaa80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:16:29.171 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.173+0000 7fa3e9ffaa80 20 rados->read ofs=0 len=0 2026-02-24T22:16:29.171 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.173+0000 7fa3e9ffaa80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:16:29.172 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.173+0000 7fa3e9ffaa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:16:29.182 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:29.182 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:16:29.182 INFO:tasks.workunit.client.0.trial046.stderr:bucket: a-bucket id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 num_objects: 0 size_kb: 0 num_shards: 2 2026-02-24T22:16:29.182 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:29.182 INFO:tasks.workunit.client.0.trial046.stderr:TEST: reshard add non empty bucket 2026-02-24T22:16:29.182 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:29.185 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:29.185 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:29.192 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:29.192 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:29.199 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:29.199 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:29.204 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:29.204 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:29.210 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:29.210 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:29.215 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:29.216 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:29.222 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:29.222 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:29.229 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:29.229 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:29.233 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket a-bucket 2026-02-24T22:16:29.248 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:16:29.248 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:16:29.265 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.265+0000 7fc86eff5a80 4 Realm: () 2026-02-24T22:16:29.265 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.265+0000 7fc86eff5a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:16:29.265 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.265+0000 7fc86eff5a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:16:29.265 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.265+0000 7fc86eff5a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:16:29.265 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.265+0000 7fc86eff5a80 20 zonegroup default 2026-02-24T22:16:29.266 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.265+0000 7fc86eff5a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:29.266 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.269+0000 7fc86eff5a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:29.267 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.269+0000 7fc86eff5a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:29.267 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.269+0000 7fc86eff5a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:29.267 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.269+0000 7fc86eff5a80 20 started sync module instance, tier type = 2026-02-24T22:16:29.267 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.269+0000 7fc86eff5a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:16:29.269 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.269+0000 7fc86eff5a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:16:29.269 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.269+0000 7fc86eff5a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:16:29.269 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.269+0000 7fc86eff5a80 5 note: GC not initialized 2026-02-24T22:16:29.269 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.269+0000 7fc81a7e4640 20 reqs_thread_entry: start 2026-02-24T22:16:29.273 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.273+0000 7fc86cac9640 0 -- 10.20.193.46:0/764861127 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55b8a48a76a0 msgr2=0x55b8a48a4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:16:29.484 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.485+0000 7fc86d2ca640 0 -- 10.20.193.46:0/764861127 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x55b8a4895940 msgr2=0x55b8a4895dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:16:29.712 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.713+0000 7fc86eff5a80 20 init_complete bucket index max shards: 1 2026-02-24T22:16:29.713 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.713+0000 7fc86eff5a80 20 Filter name: none 2026-02-24T22:16:29.713 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.713+0000 7fc818fe1640 20 reqs_thread_entry: start 2026-02-24T22:16:29.713 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.713+0000 7fc86eff5a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:29.714 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.713+0000 7fc86eff5a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:16:29.714 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.713+0000 7fc86eff5a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:29.714 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.713+0000 7fc86eff5a80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:16:29.714 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.713+0000 7fc86eff5a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:16:29.714 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.713+0000 7fc86eff5a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:29.714 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.713+0000 7fc86eff5a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:16:29.714 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.713+0000 7fc86eff5a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:29.714 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.713+0000 7fc86eff5a80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:16:29.715 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.713+0000 7fc86eff5a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:16:29.725 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:29.725 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:16:29.725 INFO:tasks.workunit.client.0.trial046.stderr:bucket: a-bucket id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 num_objects: 8 size_kb: 1 num_shards: 2 2026-02-24T22:16:29.725 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:29.726 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin reshard add --bucket a-bucket --num-shards 3 2026-02-24T22:16:29.740 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:16:29.740 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:16:29.759 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.761+0000 7f916d6fea80 4 Realm: () 2026-02-24T22:16:29.759 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.761+0000 7f916d6fea80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:16:29.759 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.761+0000 7f916d6fea80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:16:29.759 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.761+0000 7f916d6fea80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:16:29.759 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.761+0000 7f916d6fea80 20 zonegroup default 2026-02-24T22:16:29.760 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.761+0000 7f916d6fea80 20 rados->read ofs=0 len=0 2026-02-24T22:16:29.760 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.761+0000 7f916d6fea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:29.760 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.761+0000 7f916d6fea80 20 rados->read ofs=0 len=0 2026-02-24T22:16:29.761 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.761+0000 7f916d6fea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:29.761 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.761+0000 7f916d6fea80 20 started sync module instance, tier type = 2026-02-24T22:16:29.761 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.761+0000 7f916d6fea80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:16:29.763 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.761+0000 7f91677fe640 0 -- 10.20.193.46:0/4278694312 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x561877bf7400 msgr2=0x561877c17880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:29.763 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.765+0000 7f916d6fea80 20 add_watcher() i=1 2026-02-24T22:16:29.763 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.765+0000 7f916d6fea80 20 add_watcher() i=6 2026-02-24T22:16:29.763 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.765+0000 7f916d6fea80 20 add_watcher() i=0 2026-02-24T22:16:29.763 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.765+0000 7f916d6fea80 20 add_watcher() i=3 2026-02-24T22:16:29.764 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.765+0000 7f916d6fea80 20 add_watcher() i=2 2026-02-24T22:16:29.764 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.765+0000 7f916d6fea80 20 add_watcher() i=4 2026-02-24T22:16:29.764 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.765+0000 7f916d6fea80 20 add_watcher() i=5 2026-02-24T22:16:29.764 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.765+0000 7f916d6fea80 20 add_watcher() i=7 2026-02-24T22:16:29.764 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.765+0000 7f916d6fea80 2 all 8 watchers are set, enabling cache 2026-02-24T22:16:29.765 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.765+0000 7f916d6fea80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:16:29.765 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.765+0000 7f916d6fea80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:16:29.765 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.765+0000 7f916d6fea80 5 note: GC not initialized 2026-02-24T22:16:29.766 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.765+0000 7f9083fff640 20 reqs_thread_entry: start 2026-02-24T22:16:29.766 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:29.765+0000 7f91677fe640 0 -- 10.20.193.46:0/3133493931 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x561877c2a660 msgr2=0x561877c2aa30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:30.007 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.009+0000 7f916d6fea80 20 init_complete bucket index max shards: 1 2026-02-24T22:16:30.007 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.009+0000 7f916d6fea80 20 Filter name: none 2026-02-24T22:16:30.007 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.009+0000 7f90827fc640 20 reqs_thread_entry: start 2026-02-24T22:16:30.007 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.009+0000 7f916d6fea80 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2026-02-24T22:16:30.007 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.009+0000 7f916d6fea80 20 rados->read ofs=0 len=0 2026-02-24T22:16:30.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.009+0000 7f916d6fea80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:16:30.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.009+0000 7f916d6fea80 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2026-02-24T22:16:30.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.009+0000 7f916d6fea80 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2026-02-24T22:16:30.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.009+0000 7f916d6fea80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : miss 2026-02-24T22:16:30.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.009+0000 7f916d6fea80 20 rados->read ofs=0 len=0 2026-02-24T22:16:30.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.009+0000 7f916d6fea80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:16:30.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.009+0000 7f916d6fea80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:16:30.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.009+0000 7f916d6fea80 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:16:30.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.009+0000 7f916d6fea80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:16:30.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.009+0000 7f916d6fea80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:16:30.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.009+0000 7f916d6fea80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:16:30.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.013+0000 7f916d6fea80 20 remove_watcher() i=3 2026-02-24T22:16:30.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.013+0000 7f916d6fea80 2 removed watcher, disabling cache 2026-02-24T22:16:30.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.013+0000 7f916d6fea80 20 remove_watcher() i=1 2026-02-24T22:16:30.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.013+0000 7f916d6fea80 20 remove_watcher() i=2 2026-02-24T22:16:30.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.013+0000 7f916d6fea80 20 remove_watcher() i=0 2026-02-24T22:16:30.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.013+0000 7f916d6fea80 20 remove_watcher() i=4 2026-02-24T22:16:30.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.013+0000 7f916d6fea80 20 remove_watcher() i=6 2026-02-24T22:16:30.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.013+0000 7f916d6fea80 20 remove_watcher() i=7 2026-02-24T22:16:30.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.013+0000 7f916d6fea80 20 remove_watcher() i=5 2026-02-24T22:16:30.018 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin reshard list 2026-02-24T22:16:30.033 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:16:30.033 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:16:30.052 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.053+0000 7f49d0e01a80 4 Realm: () 2026-02-24T22:16:30.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.053+0000 7f49d0e01a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:16:30.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.053+0000 7f49d0e01a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:16:30.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.053+0000 7f49d0e01a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:16:30.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.053+0000 7f49d0e01a80 20 zonegroup default 2026-02-24T22:16:30.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.053+0000 7f49d0e01a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:30.054 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.053+0000 7f49d0e01a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:30.054 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.053+0000 7f49d0e01a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:30.055 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.057+0000 7f49d0e01a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:30.055 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.057+0000 7f49d0e01a80 20 started sync module instance, tier type = 2026-02-24T22:16:30.055 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.057+0000 7f49d0e01a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:16:30.056 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.057+0000 7f49d0e01a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:16:30.056 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.057+0000 7f49d0e01a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:16:30.056 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.057+0000 7f49d0e01a80 5 note: GC not initialized 2026-02-24T22:16:30.056 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.057+0000 7f497d7e2640 20 reqs_thread_entry: start 2026-02-24T22:16:30.058 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.057+0000 7f49ce0d4640 0 -- 10.20.193.46:0/2100415254 >> [v2:10.20.193.46:6826/3065971976,v1:10.20.193.46:6827/3065971976] conn(0x55868fec9d30 msgr2=0x55868feca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:16:30.265 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.265+0000 7f49ce0d4640 0 -- 10.20.193.46:0/2100415254 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x55868feb45d0 msgr2=0x55868feb4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:16:30.497 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.497+0000 7f49d0e01a80 20 init_complete bucket index max shards: 1 2026-02-24T22:16:30.497 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.497+0000 7f49d0e01a80 20 Filter name: none 2026-02-24T22:16:30.497 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.497+0000 7f48e77fe640 20 reqs_thread_entry: start 2026-02-24T22:16:30.512 INFO:tasks.workunit.client.0.trial046.stderr:bucket name a-bucket 2026-02-24T22:16:30.512 INFO:tasks.workunit.client.0.trial046.stderr:TEST: reshard process non empty bucket 2026-02-24T22:16:30.512 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:30.512 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin reshard process 2026-02-24T22:16:30.526 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:16:30.526 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:16:30.544 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.545+0000 7fb8a0506a80 4 Realm: () 2026-02-24T22:16:30.544 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.545+0000 7fb8a0506a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:16:30.544 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.545+0000 7fb8a0506a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:16:30.544 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.545+0000 7fb8a0506a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:16:30.544 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.545+0000 7fb8a0506a80 20 zonegroup default 2026-02-24T22:16:30.545 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.545+0000 7fb8a0506a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:30.546 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.545+0000 7fb8a0506a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:30.546 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.545+0000 7fb8a0506a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:30.546 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.549+0000 7fb8a0506a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:30.546 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.549+0000 7fb8a0506a80 20 started sync module instance, tier type = 2026-02-24T22:16:30.547 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.549+0000 7fb8a0506a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:16:30.547 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.549+0000 7fb89dfda640 0 -- 10.20.193.46:0/3074362282 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x55ae04edefa0 msgr2=0x55ae04eff420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:16:30.548 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.549+0000 7fb8a0506a80 20 add_watcher() i=1 2026-02-24T22:16:30.548 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.549+0000 7fb8a0506a80 20 add_watcher() i=6 2026-02-24T22:16:30.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.549+0000 7fb8a0506a80 20 add_watcher() i=0 2026-02-24T22:16:30.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.549+0000 7fb8a0506a80 20 add_watcher() i=3 2026-02-24T22:16:30.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.549+0000 7fb8a0506a80 20 add_watcher() i=5 2026-02-24T22:16:30.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.549+0000 7fb8a0506a80 20 add_watcher() i=4 2026-02-24T22:16:30.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.549+0000 7fb8a0506a80 20 add_watcher() i=2 2026-02-24T22:16:30.753 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.753+0000 7fb8a0506a80 20 add_watcher() i=7 2026-02-24T22:16:30.753 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.753+0000 7fb8a0506a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:16:30.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.753+0000 7fb89d7d9640 0 -- 10.20.193.46:0/3749433888 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7fb7bc008af0 msgr2=0x7fb7bc028fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:30.956 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.957+0000 7fb8a0506a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:16:30.956 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.957+0000 7fb8a0506a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:16:30.956 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.957+0000 7fb8a0506a80 5 note: GC not initialized 2026-02-24T22:16:30.956 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.957+0000 7fb7ce7fc640 20 reqs_thread_entry: start 2026-02-24T22:16:30.992 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.993+0000 7fb8a0506a80 20 init_complete bucket index max shards: 1 2026-02-24T22:16:30.992 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.993+0000 7fb8a0506a80 20 Filter name: none 2026-02-24T22:16:30.992 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.993+0000 7fb7ccff9640 20 reqs_thread_entry: start 2026-02-24T22:16:30.992 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.993+0000 7fb8a0506a80 20 processing logshard = reshard.0000000000 2026-02-24T22:16:30.993 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.993+0000 7fb8a0506a80 20 finish processing logshard = reshard.0000000000 , ret = 0 2026-02-24T22:16:30.994 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.993+0000 7fb8a0506a80 20 processing logshard = reshard.0000000001 2026-02-24T22:16:30.995 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.997+0000 7fb8a0506a80 20 finish processing logshard = reshard.0000000001 , ret = 0 2026-02-24T22:16:30.995 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.997+0000 7fb8a0506a80 20 processing logshard = reshard.0000000002 2026-02-24T22:16:30.996 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.997+0000 7fb8a0506a80 20 finish processing logshard = reshard.0000000002 , ret = 0 2026-02-24T22:16:30.997 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:30.997+0000 7fb8a0506a80 20 processing logshard = reshard.0000000003 2026-02-24T22:16:30.998 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.001+0000 7fb8a0506a80 20 finish processing logshard = reshard.0000000003 , ret = 0 2026-02-24T22:16:30.998 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.001+0000 7fb8a0506a80 20 processing logshard = reshard.0000000004 2026-02-24T22:16:31.000 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.001+0000 7fb8a0506a80 20 finish processing logshard = reshard.0000000004 , ret = 0 2026-02-24T22:16:31.000 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.001+0000 7fb8a0506a80 20 processing logshard = reshard.0000000005 2026-02-24T22:16:31.001 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.001+0000 7fb8a0506a80 20 finish processing logshard = reshard.0000000005 , ret = 0 2026-02-24T22:16:31.001 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.001+0000 7fb8a0506a80 20 processing logshard = reshard.0000000006 2026-02-24T22:16:31.003 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.005+0000 7fb8a0506a80 20 process_entry resharding a-bucket 2026-02-24T22:16:31.003 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.005+0000 7fb8a0506a80 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2026-02-24T22:16:31.003 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.005+0000 7fb8a0506a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:31.003 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.005+0000 7fb8a0506a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:16:31.003 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.005+0000 7fb8a0506a80 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x1 2026-02-24T22:16:31.004 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.005+0000 7fb8a0506a80 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2026-02-24T22:16:31.004 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.005+0000 7fb8a0506a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : miss 2026-02-24T22:16:31.004 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.005+0000 7fb8a0506a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:31.004 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.005+0000 7fb8a0506a80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:16:31.004 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.005+0000 7fb8a0506a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:16:31.004 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.005+0000 7fb8a0506a80 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:16:31.004 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.005+0000 7fb8a0506a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:16:31.004 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.005+0000 7fb8a0506a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:16:31.004 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.005+0000 7fb8a0506a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:16:31.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.009+0000 7fb8a0506a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:16:31.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.009+0000 7fb8a0506a80 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:16:31.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.009+0000 7fb8a0506a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:16:31.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.009+0000 7fb8a0506a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518, ofs0, ns] 2026-02-24T22:16:31.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.009+0000 7fb8797fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959899 cookie 94205895510416 notifier 6065 bl.length()=929 2026-02-24T22:16:31.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.009+0000 7fb8797fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:16:31.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.009+0000 7fb8797fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:16:31.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.009+0000 7fb8797fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:16:31.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.009+0000 7fb8a0506a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_sources= new_sources= 2026-02-24T22:16:31.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.009+0000 7fb8a0506a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential sources added=[] removed=[] 2026-02-24T22:16:31.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.009+0000 7fb8a0506a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_dests= new_dests= 2026-02-24T22:16:31.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.009+0000 7fb8a0506a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential dests added=[] removed=[] 2026-02-24T22:16:31.011 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.013+0000 7fb8a0506a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 2026-02-24T22:16:31.011 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.013+0000 7fb8a0506a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 2026-02-24T22:16:31.011 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.013+0000 7fb8a0506a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1 2026-02-24T22:16:31.011 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.013+0000 7fb8a0506a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1 2026-02-24T22:16:31.011 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.013+0000 7fb8a0506a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.2 2026-02-24T22:16:31.011 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.013+0000 7fb8a0506a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.2 2026-02-24T22:16:31.011 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.013+0000 7fb8a0506a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.1.0 2026-02-24T22:16:31.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.013+0000 7fb8a0506a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.1.0 2026-02-24T22:16:31.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.013+0000 7fb8a0506a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.1.1 2026-02-24T22:16:31.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.013+0000 7fb8a0506a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.1.1 2026-02-24T22:16:31.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.013+0000 7fb8a0506a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : hit (requested=0x17, cached=0x17) 2026-02-24T22:16:31.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.013+0000 7fb8a0506a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:16:31.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.013+0000 7fb8a0506a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:16:31.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.013+0000 7fb8a0506a80 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:16:31.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.013+0000 7fb8a0506a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:16:31.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.013+0000 7fb8a0506a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518, ofs0, ns] 2026-02-24T22:16:31.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.013+0000 7fb878ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959900 cookie 94205895510416 notifier 6065 bl.length()=929 2026-02-24T22:16:31.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.013+0000 7fb878ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:16:31.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.013+0000 7fb878ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:16:31.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.013+0000 7fb878ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:16:31.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.013+0000 7fb8a0506a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_sources= new_sources= 2026-02-24T22:16:31.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.013+0000 7fb8a0506a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential sources added=[] removed=[] 2026-02-24T22:16:31.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.013+0000 7fb8a0506a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_dests= new_dests= 2026-02-24T22:16:31.015 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.013+0000 7fb8a0506a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential dests added=[] removed=[] 2026-02-24T22:16:31.015 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.017+0000 7fb8a0506a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.1.0 2026-02-24T22:16:31.015 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.017+0000 7fb8a0506a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.1.0 2026-02-24T22:16:31.015 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.017+0000 7fb8a0506a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.1.1 2026-02-24T22:16:31.015 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.017+0000 7fb8a0506a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.1.1 2026-02-24T22:16:31.016 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.017+0000 7fb8a0506a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : hit (requested=0x17, cached=0x17) 2026-02-24T22:16:31.016 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.017+0000 7fb8a0506a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:16:31.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.017+0000 7fb8a0506a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:16:31.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.017+0000 7fb8a0506a80 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:16:31.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.017+0000 7fb8a0506a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:16:31.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.017+0000 7fb8a0506a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518, ofs0, ns] 2026-02-24T22:16:31.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.017+0000 7fb8797fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959901 cookie 94205895510416 notifier 6065 bl.length()=893 2026-02-24T22:16:31.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.017+0000 7fb8797fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:16:31.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.017+0000 7fb8797fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:16:31.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.017+0000 7fb8797fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:16:31.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.017+0000 7fb8a0506a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_sources= new_sources= 2026-02-24T22:16:31.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.017+0000 7fb8a0506a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential sources added=[] removed=[] 2026-02-24T22:16:31.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.017+0000 7fb8a0506a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_dests= new_dests= 2026-02-24T22:16:31.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.017+0000 7fb8a0506a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential dests added=[] removed=[] 2026-02-24T22:16:31.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.017+0000 7fb8a0506a80 1 execute INFO: reshard of bucket "a-bucket" from 2 shards to 3 shards completed successfully 2026-02-24T22:16:31.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.017+0000 7fb8a0506a80 20 process_entry removing reshard queue entry for bucket a-bucket 2026-02-24T22:16:31.019 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.021+0000 7fb8a0506a80 20 finish processing logshard = reshard.0000000006 , ret = 0 2026-02-24T22:16:31.019 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.021+0000 7fb8a0506a80 20 processing logshard = reshard.0000000007 2026-02-24T22:16:31.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.021+0000 7fb8a0506a80 20 finish processing logshard = reshard.0000000007 , ret = 0 2026-02-24T22:16:31.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.021+0000 7fb8a0506a80 20 processing logshard = reshard.0000000008 2026-02-24T22:16:31.022 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.021+0000 7fb8a0506a80 20 finish processing logshard = reshard.0000000008 , ret = 0 2026-02-24T22:16:31.022 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.021+0000 7fb8a0506a80 20 processing logshard = reshard.0000000009 2026-02-24T22:16:31.024 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.025+0000 7fb8a0506a80 20 finish processing logshard = reshard.0000000009 , ret = 0 2026-02-24T22:16:31.024 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.025+0000 7fb8a0506a80 20 processing logshard = reshard.0000000010 2026-02-24T22:16:31.025 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.025+0000 7fb8a0506a80 20 finish processing logshard = reshard.0000000010 , ret = 0 2026-02-24T22:16:31.025 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.025+0000 7fb8a0506a80 20 processing logshard = reshard.0000000011 2026-02-24T22:16:31.027 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.029+0000 7fb8a0506a80 20 finish processing logshard = reshard.0000000011 , ret = 0 2026-02-24T22:16:31.027 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.029+0000 7fb8a0506a80 20 processing logshard = reshard.0000000012 2026-02-24T22:16:31.028 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.029+0000 7fb8a0506a80 20 finish processing logshard = reshard.0000000012 , ret = 0 2026-02-24T22:16:31.029 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.029+0000 7fb8a0506a80 20 processing logshard = reshard.0000000013 2026-02-24T22:16:31.030 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.033+0000 7fb8a0506a80 20 finish processing logshard = reshard.0000000013 , ret = 0 2026-02-24T22:16:31.030 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.033+0000 7fb8a0506a80 20 processing logshard = reshard.0000000014 2026-02-24T22:16:31.032 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.033+0000 7fb8a0506a80 20 finish processing logshard = reshard.0000000014 , ret = 0 2026-02-24T22:16:31.032 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.033+0000 7fb8a0506a80 20 processing logshard = reshard.0000000015 2026-02-24T22:16:31.033 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.033+0000 7fb8a0506a80 20 finish processing logshard = reshard.0000000015 , ret = 0 2026-02-24T22:16:31.037 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.037+0000 7fb8a0506a80 20 remove_watcher() i=3 2026-02-24T22:16:31.037 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.037+0000 7fb8a0506a80 2 removed watcher, disabling cache 2026-02-24T22:16:31.037 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.037+0000 7fb8a0506a80 20 remove_watcher() i=4 2026-02-24T22:16:31.037 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.037+0000 7fb8a0506a80 20 remove_watcher() i=0 2026-02-24T22:16:31.037 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.037+0000 7fb8a0506a80 20 remove_watcher() i=2 2026-02-24T22:16:31.038 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.037+0000 7fb8a0506a80 20 remove_watcher() i=5 2026-02-24T22:16:31.038 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.037+0000 7fb8a0506a80 20 remove_watcher() i=6 2026-02-24T22:16:31.038 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.037+0000 7fb8a0506a80 20 remove_watcher() i=7 2026-02-24T22:16:31.038 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.037+0000 7fb8a0506a80 20 remove_watcher() i=1 2026-02-24T22:16:31.044 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket a-bucket 2026-02-24T22:16:31.058 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:16:31.059 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:16:31.077 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.077+0000 7f42f5dfea80 4 Realm: () 2026-02-24T22:16:31.077 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.077+0000 7f42f5dfea80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:16:31.077 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.077+0000 7f42f5dfea80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:16:31.077 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.077+0000 7f42f5dfea80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:16:31.077 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.077+0000 7f42f5dfea80 20 zonegroup default 2026-02-24T22:16:31.078 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.077+0000 7f42f5dfea80 20 rados->read ofs=0 len=0 2026-02-24T22:16:31.079 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.081+0000 7f42f5dfea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:31.079 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.081+0000 7f42f5dfea80 20 rados->read ofs=0 len=0 2026-02-24T22:16:31.080 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.081+0000 7f42f5dfea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:31.080 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.081+0000 7f42f5dfea80 20 started sync module instance, tier type = 2026-02-24T22:16:31.080 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.081+0000 7f42f5dfea80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:16:31.081 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.081+0000 7f42f5dfea80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:16:31.081 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.081+0000 7f42f5dfea80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:16:31.081 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.081+0000 7f42f5dfea80 5 note: GC not initialized 2026-02-24T22:16:31.082 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.081+0000 7f42a17e2640 20 reqs_thread_entry: start 2026-02-24T22:16:31.085 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.085+0000 7f42ef7fe640 0 -- 10.20.193.46:0/1241370918 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55730d98e530 msgr2=0x55730d98e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:31.296 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.297+0000 7f42effff640 0 -- 10.20.193.46:0/1241370918 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x55730d97f5d0 msgr2=0x55730d97fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:31.523 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.525+0000 7f42f5dfea80 20 init_complete bucket index max shards: 1 2026-02-24T22:16:31.523 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.525+0000 7f42f5dfea80 20 Filter name: none 2026-02-24T22:16:31.523 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.525+0000 7f420b7fe640 20 reqs_thread_entry: start 2026-02-24T22:16:31.523 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.525+0000 7f42f5dfea80 20 rados->read ofs=0 len=0 2026-02-24T22:16:31.524 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.525+0000 7f42f5dfea80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:16:31.524 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.525+0000 7f42f5dfea80 20 rados->read ofs=0 len=0 2026-02-24T22:16:31.524 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.525+0000 7f42f5dfea80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:16:31.525 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.525+0000 7f42f5dfea80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:16:31.525 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.525+0000 7f42f5dfea80 20 rados->read ofs=0 len=0 2026-02-24T22:16:31.525 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.525+0000 7f42f5dfea80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:16:31.525 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.525+0000 7f42f5dfea80 20 rados->read ofs=0 len=0 2026-02-24T22:16:31.525 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.525+0000 7f42f5dfea80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:16:31.525 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.525+0000 7f42f5dfea80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:16:31.535 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:31.535 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:16:31.535 INFO:tasks.workunit.client.0.trial046.stderr:bucket: a-bucket id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 num_objects: 8 size_kb: 1 num_shards: 3 2026-02-24T22:16:31.536 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:31.536 INFO:tasks.workunit.client.0.trial046.stderr:TEST: reshard bucket with EIO injected at init_index 2026-02-24T22:16:31.536 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:31.536 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:31.537 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:31.545 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:31.545 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:31.591 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:31.592 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:31.598 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:31.598 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:31.604 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:31.604 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:31.611 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:31.611 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:31.617 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:31.617 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:31.623 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:31.623 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:31.628 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:31.628 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:31.635 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:31.635 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:31.640 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:31.641 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:31.647 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:31.647 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:31.653 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:31.653 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:31.659 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:31.659 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:31.665 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:31.665 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:31.671 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:31.671 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:31.676 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:31.676 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:31.682 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:31.683 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:31.689 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:31.689 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:31.696 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:31.696 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:31.702 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:31.702 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:31.708 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:31.708 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:31.713 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-init-index 2026-02-24T22:16:31.727 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:16:31.727 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:16:31.744 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.745+0000 7fa317f84a80 4 Realm: () 2026-02-24T22:16:31.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.745+0000 7fa317f84a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:16:31.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.745+0000 7fa317f84a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:16:31.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.745+0000 7fa317f84a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:16:31.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.745+0000 7fa317f84a80 20 zonegroup default 2026-02-24T22:16:31.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.745+0000 7fa317f84a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:31.746 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.749+0000 7fa317f84a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:31.747 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.749+0000 7fa317f84a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:31.747 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.749+0000 7fa317f84a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:31.747 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.749+0000 7fa317f84a80 20 started sync module instance, tier type = 2026-02-24T22:16:31.747 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.749+0000 7fa317f84a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:16:31.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.749+0000 7fa317f84a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:16:31.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.749+0000 7fa317f84a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:16:31.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.749+0000 7fa317f84a80 5 note: GC not initialized 2026-02-24T22:16:31.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.749+0000 7fa236ffd640 20 reqs_thread_entry: start 2026-02-24T22:16:31.753 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.753+0000 7fa316259640 0 -- 10.20.193.46:0/2576497905 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55da232a55e0 msgr2=0x55da23298850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:16:31.759 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.761+0000 7fa315a58640 0 -- 10.20.193.46:0/2576497905 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55da232a1060 msgr2=0x55da2329bdd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:31.783 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.785+0000 7fa317f84a80 20 init_complete bucket index max shards: 1 2026-02-24T22:16:31.784 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.785+0000 7fa317f84a80 20 Filter name: none 2026-02-24T22:16:31.784 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.785+0000 7fa2357fa640 20 reqs_thread_entry: start 2026-02-24T22:16:31.784 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.785+0000 7fa317f84a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:31.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.785+0000 7fa317f84a80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:16:31.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.785+0000 7fa317f84a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:31.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.785+0000 7fa317f84a80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:16:31.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.785+0000 7fa317f84a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:16:31.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.785+0000 7fa317f84a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:31.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.785+0000 7fa317f84a80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:16:31.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.785+0000 7fa317f84a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:31.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.785+0000 7fa317f84a80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:16:31.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.785+0000 7fa317f84a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:16:31.795 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:31.795 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:16:31.795 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-init-index id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:16:31.795 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:31.796 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket error-at-init-index --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at init_index 2026-02-24T22:16:31.812 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:16:31.812 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:16:31.829 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.829+0000 7f3e88152a80 4 Realm: () 2026-02-24T22:16:31.830 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.829+0000 7f3e88152a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:16:31.830 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.829+0000 7f3e88152a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:16:31.830 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.829+0000 7f3e88152a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:16:31.830 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.829+0000 7f3e88152a80 20 zonegroup default 2026-02-24T22:16:31.830 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.829+0000 7f3e88152a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:31.831 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.833+0000 7f3e88152a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:31.831 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.833+0000 7f3e88152a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:31.832 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.833+0000 7f3e88152a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:31.832 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.833+0000 7f3e88152a80 20 started sync module instance, tier type = 2026-02-24T22:16:31.832 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.833+0000 7f3e88152a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:16:31.833 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.833+0000 7f3e86427640 0 -- 10.20.193.46:0/2427904373 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x560df1ca49d0 msgr2=0x560df1cc4db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:16:31.834 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.833+0000 7f3e88152a80 20 add_watcher() i=1 2026-02-24T22:16:31.834 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.833+0000 7f3e88152a80 20 add_watcher() i=6 2026-02-24T22:16:31.834 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.833+0000 7f3e88152a80 20 add_watcher() i=5 2026-02-24T22:16:31.834 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.833+0000 7f3e88152a80 20 add_watcher() i=3 2026-02-24T22:16:31.834 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.833+0000 7f3e88152a80 20 add_watcher() i=0 2026-02-24T22:16:31.834 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.833+0000 7f3e88152a80 20 add_watcher() i=2 2026-02-24T22:16:31.834 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.833+0000 7f3e88152a80 20 add_watcher() i=7 2026-02-24T22:16:31.835 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.837+0000 7f3e88152a80 20 add_watcher() i=4 2026-02-24T22:16:31.835 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.837+0000 7f3e88152a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:16:31.835 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.837+0000 7f3e85425640 0 -- 10.20.193.46:0/4182249574 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f3da4008af0 msgr2=0x7f3da4028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:31.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.837+0000 7f3e88152a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:16:31.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.837+0000 7f3e88152a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:16:31.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.837+0000 7f3e88152a80 5 note: GC not initialized 2026-02-24T22:16:31.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.837+0000 7f3db67fc640 20 reqs_thread_entry: start 2026-02-24T22:16:31.870 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.869+0000 7f3e88152a80 20 init_complete bucket index max shards: 1 2026-02-24T22:16:31.870 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.869+0000 7f3e88152a80 20 Filter name: none 2026-02-24T22:16:31.870 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.869+0000 7f3db4ff9640 20 reqs_thread_entry: start 2026-02-24T22:16:31.870 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.869+0000 7f3e88152a80 10 cache get: name=default.rgw.meta+root+error-at-init-index : miss 2026-02-24T22:16:31.870 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.869+0000 7f3e88152a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:31.870 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.869+0000 7f3e88152a80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:16:31.871 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.869+0000 7f3e88152a80 10 cache put: name=default.rgw.meta+root+error-at-init-index info.flags=0x11 2026-02-24T22:16:31.871 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.869+0000 7f3e88152a80 10 adding default.rgw.meta+root+error-at-init-index to cache LRU end 2026-02-24T22:16:31.871 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.869+0000 7f3e88152a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 : miss 2026-02-24T22:16:31.871 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.869+0000 7f3e88152a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:31.871 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.869+0000 7f3e88152a80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:16:31.871 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.869+0000 7f3e88152a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 info.flags=0x17 2026-02-24T22:16:31.871 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.869+0000 7f3e88152a80 10 adding default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 to cache LRU end 2026-02-24T22:16:31.871 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.869+0000 7f3e88152a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:16:31.871 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.869+0000 7f3e88152a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:16:31.871 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.869+0000 7f3e88152a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 2026-02-24T22:16:31.872 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.873+0000 7f3e88152a80 -1 Injecting error=-5 at location=init_index 2026-02-24T22:16:31.872 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.873+0000 7f3e88152a80 0 ERROR: init_target_index failed to initialize target index shard objects: (5) Input/output error 2026-02-24T22:16:31.874 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.873+0000 7f3e88152a80 20 remove_watcher() i=3 2026-02-24T22:16:31.874 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.873+0000 7f3e88152a80 2 removed watcher, disabling cache 2026-02-24T22:16:31.874 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.873+0000 7f3e88152a80 20 remove_watcher() i=4 2026-02-24T22:16:31.874 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.873+0000 7f3e88152a80 20 remove_watcher() i=6 2026-02-24T22:16:31.874 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.873+0000 7f3e88152a80 20 remove_watcher() i=1 2026-02-24T22:16:31.875 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.873+0000 7f3e88152a80 20 remove_watcher() i=2 2026-02-24T22:16:31.875 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.873+0000 7f3e88152a80 20 remove_watcher() i=5 2026-02-24T22:16:31.875 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.873+0000 7f3e88152a80 20 remove_watcher() i=0 2026-02-24T22:16:31.875 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.873+0000 7f3e88152a80 20 remove_watcher() i=7 2026-02-24T22:16:31.880 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-init-index 2026-02-24T22:16:31.895 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:16:31.895 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:16:31.911 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.913+0000 7f8032018a80 4 Realm: () 2026-02-24T22:16:31.911 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.913+0000 7f8032018a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:16:31.911 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.913+0000 7f8032018a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:16:31.911 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.913+0000 7f8032018a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:16:31.911 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.913+0000 7f8032018a80 20 zonegroup default 2026-02-24T22:16:31.912 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.913+0000 7f8032018a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:31.913 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.913+0000 7f8032018a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:31.913 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.913+0000 7f8032018a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:31.913 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.913+0000 7f8032018a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:31.913 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.913+0000 7f8032018a80 20 started sync module instance, tier type = 2026-02-24T22:16:31.913 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.913+0000 7f8032018a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:16:31.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.917+0000 7f8032018a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:16:31.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.917+0000 7f8032018a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:16:31.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.917+0000 7f8032018a80 5 note: GC not initialized 2026-02-24T22:16:31.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.917+0000 7f7fdd7e2640 20 reqs_thread_entry: start 2026-02-24T22:16:31.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.921+0000 7f802bfff640 0 -- 10.20.193.46:0/3734382125 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x557a9edf6720 msgr2=0x557a9ede9990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:31.924 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.925+0000 7f802b7fe640 0 -- 10.20.193.46:0/3734382125 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x557a9edf21a0 msgr2=0x557a9edecf10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:31.948 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.949+0000 7f8032018a80 20 init_complete bucket index max shards: 1 2026-02-24T22:16:31.948 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.949+0000 7f8032018a80 20 Filter name: none 2026-02-24T22:16:31.948 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.949+0000 7f7f477fe640 20 reqs_thread_entry: start 2026-02-24T22:16:31.948 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.949+0000 7f8032018a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:31.948 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.949+0000 7f8032018a80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:16:31.948 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.949+0000 7f8032018a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:31.949 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.949+0000 7f8032018a80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:16:31.949 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.949+0000 7f8032018a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:16:31.949 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.949+0000 7f8032018a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:31.949 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.949+0000 7f8032018a80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:16:31.950 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.949+0000 7f8032018a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:31.950 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.949+0000 7f8032018a80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:16:31.950 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:31.949+0000 7f8032018a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:16:31.958 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:31.958 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:16:31.958 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-init-index id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:16:31.958 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:31.960 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:31.960 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:31.966 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:31.966 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:32.011 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket error-at-init-index --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2026-02-24T22:16:32.027 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:16:32.027 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:16:32.046 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.045+0000 7f61cf54fa80 4 Realm: () 2026-02-24T22:16:32.046 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.045+0000 7f61cf54fa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:16:32.046 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.045+0000 7f61cf54fa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:16:32.046 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.045+0000 7f61cf54fa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:16:32.046 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.045+0000 7f61cf54fa80 20 zonegroup default 2026-02-24T22:16:32.046 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.045+0000 7f61cf54fa80 20 rados->read ofs=0 len=0 2026-02-24T22:16:32.048 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.049+0000 7f61cf54fa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:32.048 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.049+0000 7f61cf54fa80 20 rados->read ofs=0 len=0 2026-02-24T22:16:32.048 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.049+0000 7f61cf54fa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:32.049 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.049+0000 7f61cf54fa80 20 started sync module instance, tier type = 2026-02-24T22:16:32.049 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.049+0000 7f61cf54fa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:16:32.049 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.049+0000 7f61cc822640 0 -- 10.20.193.46:0/3797797588 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x55ac8fa78880 msgr2=0x55ac8fa98d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:32.050 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.049+0000 7f61cf54fa80 20 add_watcher() i=1 2026-02-24T22:16:32.050 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.049+0000 7f61cf54fa80 20 add_watcher() i=3 2026-02-24T22:16:32.051 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.053+0000 7f61cf54fa80 20 add_watcher() i=0 2026-02-24T22:16:32.051 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.053+0000 7f61cf54fa80 20 add_watcher() i=6 2026-02-24T22:16:32.051 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.053+0000 7f61cf54fa80 20 add_watcher() i=5 2026-02-24T22:16:32.051 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.053+0000 7f61cf54fa80 20 add_watcher() i=7 2026-02-24T22:16:32.051 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.053+0000 7f61cf54fa80 20 add_watcher() i=2 2026-02-24T22:16:32.253 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.253+0000 7f61cf54fa80 20 add_watcher() i=4 2026-02-24T22:16:32.253 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.253+0000 7f61cf54fa80 2 all 8 watchers are set, enabling cache 2026-02-24T22:16:32.254 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.253+0000 7f61cc822640 0 -- 10.20.193.46:0/1544514125 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f60ec008af0 msgr2=0x7f60ec028fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:32.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.457+0000 7f61cf54fa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:16:32.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.457+0000 7f61cf54fa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:16:32.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.457+0000 7f61cf54fa80 5 note: GC not initialized 2026-02-24T22:16:32.458 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.457+0000 7f617a7e4640 20 reqs_thread_entry: start 2026-02-24T22:16:32.494 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.493+0000 7f61cf54fa80 20 init_complete bucket index max shards: 1 2026-02-24T22:16:32.494 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.493+0000 7f61cf54fa80 20 Filter name: none 2026-02-24T22:16:32.495 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.493+0000 7f6178fe1640 20 reqs_thread_entry: start 2026-02-24T22:16:32.495 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.493+0000 7f61cf54fa80 10 cache get: name=default.rgw.meta+root+error-at-init-index : miss 2026-02-24T22:16:32.495 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.493+0000 7f61cf54fa80 20 rados->read ofs=0 len=0 2026-02-24T22:16:32.495 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.493+0000 7f61cf54fa80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:16:32.495 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.493+0000 7f61cf54fa80 10 cache put: name=default.rgw.meta+root+error-at-init-index info.flags=0x11 2026-02-24T22:16:32.495 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.493+0000 7f61cf54fa80 10 adding default.rgw.meta+root+error-at-init-index to cache LRU end 2026-02-24T22:16:32.495 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.493+0000 7f61cf54fa80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 : miss 2026-02-24T22:16:32.495 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.493+0000 7f61cf54fa80 20 rados->read ofs=0 len=0 2026-02-24T22:16:32.495 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.497+0000 7f61cf54fa80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:16:32.495 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.497+0000 7f61cf54fa80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 info.flags=0x17 2026-02-24T22:16:32.495 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.497+0000 7f61cf54fa80 10 adding default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 to cache LRU end 2026-02-24T22:16:32.496 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.497+0000 7f61cf54fa80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:16:32.496 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.497+0000 7f61cf54fa80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:16:32.496 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.497+0000 7f61cf54fa80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 2026-02-24T22:16:32.500 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.501+0000 7f61cf54fa80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 info.flags=0x17 2026-02-24T22:16:32.500 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.501+0000 7f61cf54fa80 10 moving default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 to cache LRU end 2026-02-24T22:16:32.501 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.501+0000 7f61cf54fa80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:16:32.501 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.501+0000 7f61cf54fa80 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520, ofs0, ns] 2026-02-24T22:16:32.501 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.501+0000 7f6197fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961318 cookie 94199633676448 notifier 6522 bl.length()=1013 2026-02-24T22:16:32.501 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.501+0000 7f6197fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 info.flags=0x17 2026-02-24T22:16:32.501 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.501+0000 7f6197fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 to cache LRU end 2026-02-24T22:16:32.501 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.501+0000 7f6197fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:16:32.501 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.501+0000 7f61cf54fa80 20 handle_bi_update(): bucket=:error-at-init-index[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520]): orig_sources= new_sources= 2026-02-24T22:16:32.501 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.501+0000 7f61cf54fa80 20 handle_bi_update(): bucket=:error-at-init-index[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520]): potential sources added=[] removed=[] 2026-02-24T22:16:32.501 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.501+0000 7f61cf54fa80 20 handle_bi_update(): bucket=:error-at-init-index[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520]): orig_dests= new_dests= 2026-02-24T22:16:32.501 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.501+0000 7f61cf54fa80 20 handle_bi_update(): bucket=:error-at-init-index[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520]): potential dests added=[] removed=[] 2026-02-24T22:16:32.502 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.501+0000 7f61cf54fa80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520.1.0 2026-02-24T22:16:32.502 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.501+0000 7f61cf54fa80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520.1.0 2026-02-24T22:16:32.502 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.501+0000 7f61cf54fa80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520.1.1 2026-02-24T22:16:32.502 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.501+0000 7f61cf54fa80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520.1.1 2026-02-24T22:16:32.502 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.501+0000 7f61cf54fa80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520.0 2026-02-24T22:16:32.504 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.505+0000 7f61cf54fa80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 : hit (requested=0x17, cached=0x17) 2026-02-24T22:16:32.504 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.505+0000 7f61cf54fa80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 2026-02-24T22:16:32.506 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.505+0000 7f61cf54fa80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 info.flags=0x17 2026-02-24T22:16:32.506 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.505+0000 7f61cf54fa80 10 moving default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 to cache LRU end 2026-02-24T22:16:32.506 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.505+0000 7f61cf54fa80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:16:32.506 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.505+0000 7f61cf54fa80 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520, ofs0, ns] 2026-02-24T22:16:32.506 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.505+0000 7f61a8ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961319 cookie 94199633676448 notifier 6522 bl.length()=1013 2026-02-24T22:16:32.506 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.505+0000 7f61a8ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 info.flags=0x17 2026-02-24T22:16:32.506 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.505+0000 7f61a8ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 to cache LRU end 2026-02-24T22:16:32.507 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.505+0000 7f61a8ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:16:32.507 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.505+0000 7f61cf54fa80 20 handle_bi_update(): bucket=:error-at-init-index[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520]): orig_sources= new_sources= 2026-02-24T22:16:32.507 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.505+0000 7f61cf54fa80 20 handle_bi_update(): bucket=:error-at-init-index[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520]): potential sources added=[] removed=[] 2026-02-24T22:16:32.507 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.505+0000 7f61cf54fa80 20 handle_bi_update(): bucket=:error-at-init-index[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520]): orig_dests= new_dests= 2026-02-24T22:16:32.507 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.505+0000 7f61cf54fa80 20 handle_bi_update(): bucket=:error-at-init-index[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520]): potential dests added=[] removed=[] 2026-02-24T22:16:32.508 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.509+0000 7f61cf54fa80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520.0 2026-02-24T22:16:32.508 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.509+0000 7f61cf54fa80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 : hit (requested=0x17, cached=0x17) 2026-02-24T22:16:32.508 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.509+0000 7f61cf54fa80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 2026-02-24T22:16:32.508 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.509+0000 7f61cf54fa80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 info.flags=0x17 2026-02-24T22:16:32.509 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.509+0000 7f61cf54fa80 10 moving default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 to cache LRU end 2026-02-24T22:16:32.509 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.509+0000 7f61cf54fa80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:16:32.509 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.509+0000 7f61cf54fa80 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520, ofs0, ns] 2026-02-24T22:16:32.509 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.509+0000 7f6197fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961320 cookie 94199633676448 notifier 6522 bl.length()=977 2026-02-24T22:16:32.509 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.509+0000 7f6197fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 info.flags=0x17 2026-02-24T22:16:32.509 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.509+0000 7f6197fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 to cache LRU end 2026-02-24T22:16:32.509 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.509+0000 7f6197fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:16:32.509 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.509+0000 7f61cf54fa80 20 handle_bi_update(): bucket=:error-at-init-index[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520]): orig_sources= new_sources= 2026-02-24T22:16:32.509 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.509+0000 7f61cf54fa80 20 handle_bi_update(): bucket=:error-at-init-index[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520]): potential sources added=[] removed=[] 2026-02-24T22:16:32.509 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.509+0000 7f61cf54fa80 20 handle_bi_update(): bucket=:error-at-init-index[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520]): orig_dests= new_dests= 2026-02-24T22:16:32.509 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.509+0000 7f61cf54fa80 20 handle_bi_update(): bucket=:error-at-init-index[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520]): potential dests added=[] removed=[] 2026-02-24T22:16:32.510 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.509+0000 7f61cf54fa80 1 execute INFO: reshard of bucket "error-at-init-index" from 1 shards to 2 shards completed successfully 2026-02-24T22:16:32.513 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.513+0000 7f61cf54fa80 20 remove_watcher() i=3 2026-02-24T22:16:32.513 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.513+0000 7f61cf54fa80 2 removed watcher, disabling cache 2026-02-24T22:16:32.513 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.513+0000 7f61cf54fa80 20 remove_watcher() i=4 2026-02-24T22:16:32.513 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.513+0000 7f61cf54fa80 20 remove_watcher() i=5 2026-02-24T22:16:32.513 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.513+0000 7f61cf54fa80 20 remove_watcher() i=0 2026-02-24T22:16:32.513 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.513+0000 7f61cf54fa80 20 remove_watcher() i=2 2026-02-24T22:16:32.513 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.513+0000 7f61cf54fa80 20 remove_watcher() i=7 2026-02-24T22:16:32.513 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.513+0000 7f61cf54fa80 20 remove_watcher() i=1 2026-02-24T22:16:32.514 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.513+0000 7f61cf54fa80 20 remove_watcher() i=6 2026-02-24T22:16:32.519 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-init-index 2026-02-24T22:16:32.535 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:16:32.535 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:16:32.553 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.553+0000 7f947d25aa80 4 Realm: () 2026-02-24T22:16:32.553 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.553+0000 7f947d25aa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:16:32.553 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.553+0000 7f947d25aa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:16:32.553 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.553+0000 7f947d25aa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:16:32.553 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.553+0000 7f947d25aa80 20 zonegroup default 2026-02-24T22:16:32.553 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.553+0000 7f947d25aa80 20 rados->read ofs=0 len=0 2026-02-24T22:16:32.555 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.557+0000 7f947d25aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:32.555 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.557+0000 7f947d25aa80 20 rados->read ofs=0 len=0 2026-02-24T22:16:32.556 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.557+0000 7f947d25aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:32.556 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.557+0000 7f947d25aa80 20 started sync module instance, tier type = 2026-02-24T22:16:32.556 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.557+0000 7f947d25aa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:16:32.557 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.557+0000 7f947d25aa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:16:32.557 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.557+0000 7f947d25aa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:16:32.558 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.557+0000 7f947d25aa80 5 note: GC not initialized 2026-02-24T22:16:32.558 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.557+0000 7f94297e2640 20 reqs_thread_entry: start 2026-02-24T22:16:32.561 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.561+0000 7f947b52f640 0 -- 10.20.193.46:0/1996641485 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x56096be4a1a0 msgr2=0x56096be3d410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:32.567 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.569+0000 7f947ad2e640 0 -- 10.20.193.46:0/1996641485 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x56096be45c20 msgr2=0x56096be40990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:32.591 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.593+0000 7f947d25aa80 20 init_complete bucket index max shards: 1 2026-02-24T22:16:32.591 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.593+0000 7f947d25aa80 20 Filter name: none 2026-02-24T22:16:32.591 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.593+0000 7f93937fe640 20 reqs_thread_entry: start 2026-02-24T22:16:32.591 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.593+0000 7f947d25aa80 20 rados->read ofs=0 len=0 2026-02-24T22:16:32.592 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.593+0000 7f947d25aa80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:16:32.592 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.593+0000 7f947d25aa80 20 rados->read ofs=0 len=0 2026-02-24T22:16:32.592 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.593+0000 7f947d25aa80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:16:32.592 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.593+0000 7f947d25aa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:16:32.592 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.593+0000 7f947d25aa80 20 rados->read ofs=0 len=0 2026-02-24T22:16:32.592 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.593+0000 7f947d25aa80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:16:32.593 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.593+0000 7f947d25aa80 20 rados->read ofs=0 len=0 2026-02-24T22:16:32.593 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.593+0000 7f947d25aa80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:16:32.593 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.593+0000 7f947d25aa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:16:32.602 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:32.602 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:16:32.602 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-init-index id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520 num_objects: 19 size_kb: 1 num_shards: 2 2026-02-24T22:16:32.602 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:32.603 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:32.603 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:32.606 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:32.606 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:32.652 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:32.652 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:32.664 INFO:tasks.workunit.client.0.trial046.stderr:TEST: reshard bucket with EOPNOTSUPP injected at init_index 2026-02-24T22:16:32.664 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:32.664 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:32.665 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:32.672 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:32.673 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:32.715 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:32.715 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:32.721 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:32.721 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:32.727 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:32.727 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:32.732 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:32.733 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:32.738 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:32.738 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:32.743 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:32.744 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:32.749 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:32.749 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:32.754 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:32.755 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:32.759 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:32.759 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:32.765 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:32.765 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:32.770 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:32.771 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:32.777 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:32.777 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:32.783 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:32.783 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:32.789 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:32.789 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:32.795 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:32.795 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:32.800 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:32.800 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:32.806 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:32.806 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:32.811 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:32.812 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:32.817 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:32.817 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:32.823 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:32.823 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:32.828 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-init-index 2026-02-24T22:16:32.843 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:16:32.843 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:16:32.859 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.861+0000 7f44e5c7aa80 4 Realm: () 2026-02-24T22:16:32.859 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.861+0000 7f44e5c7aa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:16:32.859 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.861+0000 7f44e5c7aa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:16:32.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.861+0000 7f44e5c7aa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:16:32.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.861+0000 7f44e5c7aa80 20 zonegroup default 2026-02-24T22:16:32.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.861+0000 7f44e5c7aa80 20 rados->read ofs=0 len=0 2026-02-24T22:16:32.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.861+0000 7f44e5c7aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:32.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.861+0000 7f44e5c7aa80 20 rados->read ofs=0 len=0 2026-02-24T22:16:32.862 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.861+0000 7f44e5c7aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:32.862 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.861+0000 7f44e5c7aa80 20 started sync module instance, tier type = 2026-02-24T22:16:32.862 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.861+0000 7f44e5c7aa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:16:32.863 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.865+0000 7f44e5c7aa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:16:32.863 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.865+0000 7f44e5c7aa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:16:32.864 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.865+0000 7f44e5c7aa80 5 note: GC not initialized 2026-02-24T22:16:32.864 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.865+0000 7f4490fe1640 20 reqs_thread_entry: start 2026-02-24T22:16:32.867 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.869+0000 7f44df7fe640 0 -- 10.20.193.46:0/2672105635 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x558ae2037a60 msgr2=0x558ae20313e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:32.873 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:32.873+0000 7f44deffd640 0 -- 10.20.193.46:0/2672105635 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x558ae200c0d0 msgr2=0x558ae2039bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:16:33.102 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.101+0000 7f44e5c7aa80 20 init_complete bucket index max shards: 1 2026-02-24T22:16:33.102 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.101+0000 7f44e5c7aa80 20 Filter name: none 2026-02-24T22:16:33.102 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.101+0000 7f4402ffd640 20 reqs_thread_entry: start 2026-02-24T22:16:33.102 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.101+0000 7f44e5c7aa80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.103 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.105+0000 7f44e5c7aa80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:16:33.104 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.105+0000 7f44e5c7aa80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.104 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.105+0000 7f44e5c7aa80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:16:33.104 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.105+0000 7f44e5c7aa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:16:33.104 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.105+0000 7f44e5c7aa80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.104 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.105+0000 7f44e5c7aa80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:16:33.104 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.105+0000 7f44e5c7aa80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.104 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.105+0000 7f44e5c7aa80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:16:33.104 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.105+0000 7f44e5c7aa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:16:33.114 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.114 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:16:33.114 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-init-index id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:16:33.114 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.115 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket error-at-init-index --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at init_index --inject-error-code 95 2026-02-24T22:16:33.132 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:16:33.132 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:16:33.150 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.149+0000 7f744fa29a80 4 Realm: () 2026-02-24T22:16:33.150 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.149+0000 7f744fa29a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:16:33.150 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.149+0000 7f744fa29a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:16:33.151 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.149+0000 7f744fa29a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:16:33.151 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.149+0000 7f744fa29a80 20 zonegroup default 2026-02-24T22:16:33.151 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.153+0000 7f744fa29a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.152 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.153+0000 7f744fa29a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:33.152 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.153+0000 7f744fa29a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.152 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.153+0000 7f744fa29a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:33.152 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.153+0000 7f744fa29a80 20 started sync module instance, tier type = 2026-02-24T22:16:33.152 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.153+0000 7f744fa29a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:16:33.153 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.153+0000 7f744d4fd640 0 -- 10.20.193.46:0/479528954 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x55f4801c8910 msgr2=0x55f4801e8d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:33.154 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.153+0000 7f744fa29a80 20 add_watcher() i=1 2026-02-24T22:16:33.154 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.153+0000 7f744fa29a80 20 add_watcher() i=6 2026-02-24T22:16:33.154 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.153+0000 7f744fa29a80 20 add_watcher() i=3 2026-02-24T22:16:33.155 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.153+0000 7f744fa29a80 20 add_watcher() i=0 2026-02-24T22:16:33.155 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.153+0000 7f744fa29a80 20 add_watcher() i=5 2026-02-24T22:16:33.155 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.157+0000 7f744fa29a80 20 add_watcher() i=2 2026-02-24T22:16:33.155 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.157+0000 7f744fa29a80 20 add_watcher() i=4 2026-02-24T22:16:33.155 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.157+0000 7f744fa29a80 20 add_watcher() i=7 2026-02-24T22:16:33.155 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.157+0000 7f744fa29a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:16:33.157 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.157+0000 7f744fa29a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:16:33.157 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.157+0000 7f744fa29a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:16:33.157 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.157+0000 7f744fa29a80 5 note: GC not initialized 2026-02-24T22:16:33.157 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.157+0000 7f73fa7e4640 20 reqs_thread_entry: start 2026-02-24T22:16:33.157 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.157+0000 7f744dcfe640 0 -- 10.20.193.46:0/2206987277 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55f4801cded0 msgr2=0x55f4801ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:33.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.397+0000 7f744fa29a80 20 init_complete bucket index max shards: 1 2026-02-24T22:16:33.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.397+0000 7f744fa29a80 20 Filter name: none 2026-02-24T22:16:33.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.397+0000 7f73f8fe1640 20 reqs_thread_entry: start 2026-02-24T22:16:33.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.397+0000 7f744fa29a80 10 cache get: name=default.rgw.meta+root+error-at-init-index : miss 2026-02-24T22:16:33.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.397+0000 7f744fa29a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.397+0000 7f744fa29a80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:16:33.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.397+0000 7f744fa29a80 10 cache put: name=default.rgw.meta+root+error-at-init-index info.flags=0x11 2026-02-24T22:16:33.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.397+0000 7f744fa29a80 10 adding default.rgw.meta+root+error-at-init-index to cache LRU end 2026-02-24T22:16:33.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.397+0000 7f744fa29a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521 : miss 2026-02-24T22:16:33.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.397+0000 7f744fa29a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.397+0000 7f744fa29a80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:16:33.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.397+0000 7f744fa29a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521 info.flags=0x17 2026-02-24T22:16:33.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.397+0000 7f744fa29a80 10 adding default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521 to cache LRU end 2026-02-24T22:16:33.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.397+0000 7f744fa29a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:16:33.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.397+0000 7f744fa29a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:16:33.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.397+0000 7f744fa29a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521 2026-02-24T22:16:33.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.401+0000 7f744fa29a80 -1 Injecting error=-95 at location=init_index 2026-02-24T22:16:33.400 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.401+0000 7f744fa29a80 0 WARNING: init_index() does not supported logrecord, falling back to block reshard mode. 2026-02-24T22:16:33.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f744fa29a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521 info.flags=0x17 2026-02-24T22:16:33.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f744fa29a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521 to cache LRU end 2026-02-24T22:16:33.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f744fa29a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:16:33.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f744fa29a80 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521, ofs0, ns] 2026-02-24T22:16:33.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f7417fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961323 cookie 94508607710208 notifier 6191 bl.length()=1013 2026-02-24T22:16:33.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f7417fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521 info.flags=0x17 2026-02-24T22:16:33.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f7417fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521 to cache LRU end 2026-02-24T22:16:33.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f7417fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:16:33.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f744fa29a80 20 handle_bi_update(): bucket=:error-at-init-index[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521]): orig_sources= new_sources= 2026-02-24T22:16:33.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f744fa29a80 20 handle_bi_update(): bucket=:error-at-init-index[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521]): potential sources added=[] removed=[] 2026-02-24T22:16:33.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f744fa29a80 20 handle_bi_update(): bucket=:error-at-init-index[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521]): orig_dests= new_dests= 2026-02-24T22:16:33.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f744fa29a80 20 handle_bi_update(): bucket=:error-at-init-index[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521]): potential dests added=[] removed=[] 2026-02-24T22:16:33.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f744fa29a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521.1.0 2026-02-24T22:16:33.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f744fa29a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521.1.0 2026-02-24T22:16:33.405 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f744fa29a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521.1.1 2026-02-24T22:16:33.405 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f744fa29a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521.1.1 2026-02-24T22:16:33.405 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f744fa29a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521.0 2026-02-24T22:16:33.405 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f744fa29a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521 : hit (requested=0x17, cached=0x17) 2026-02-24T22:16:33.405 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f744fa29a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521 2026-02-24T22:16:33.406 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f744fa29a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521 info.flags=0x17 2026-02-24T22:16:33.406 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f744fa29a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521 to cache LRU end 2026-02-24T22:16:33.406 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f744fa29a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:16:33.407 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f744fa29a80 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521, ofs0, ns] 2026-02-24T22:16:33.407 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f7428ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961324 cookie 94508607710208 notifier 6191 bl.length()=977 2026-02-24T22:16:33.407 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f7428ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521 info.flags=0x17 2026-02-24T22:16:33.407 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f7428ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-init-index:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521 to cache LRU end 2026-02-24T22:16:33.407 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.405+0000 7f7428ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:16:33.407 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.409+0000 7f744fa29a80 20 handle_bi_update(): bucket=:error-at-init-index[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521]): orig_sources= new_sources= 2026-02-24T22:16:33.407 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.409+0000 7f744fa29a80 20 handle_bi_update(): bucket=:error-at-init-index[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521]): potential sources added=[] removed=[] 2026-02-24T22:16:33.407 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.409+0000 7f744fa29a80 20 handle_bi_update(): bucket=:error-at-init-index[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521]): orig_dests= new_dests= 2026-02-24T22:16:33.407 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.409+0000 7f744fa29a80 20 handle_bi_update(): bucket=:error-at-init-index[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521]): potential dests added=[] removed=[] 2026-02-24T22:16:33.408 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.409+0000 7f744fa29a80 1 execute INFO: reshard of bucket "error-at-init-index" from 1 shards to 2 shards completed successfully 2026-02-24T22:16:33.411 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.413+0000 7f744fa29a80 20 remove_watcher() i=3 2026-02-24T22:16:33.411 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.413+0000 7f744fa29a80 2 removed watcher, disabling cache 2026-02-24T22:16:33.411 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.413+0000 7f744fa29a80 20 remove_watcher() i=0 2026-02-24T22:16:33.411 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.413+0000 7f744fa29a80 20 remove_watcher() i=2 2026-02-24T22:16:33.411 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.413+0000 7f744fa29a80 20 remove_watcher() i=6 2026-02-24T22:16:33.411 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.413+0000 7f744fa29a80 20 remove_watcher() i=4 2026-02-24T22:16:33.411 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.413+0000 7f744fa29a80 20 remove_watcher() i=7 2026-02-24T22:16:33.412 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.413+0000 7f744fa29a80 20 remove_watcher() i=5 2026-02-24T22:16:33.412 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.413+0000 7f744fa29a80 20 remove_watcher() i=1 2026-02-24T22:16:33.418 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-init-index 2026-02-24T22:16:33.434 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:16:33.434 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:16:33.452 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.453+0000 7f9585400a80 4 Realm: () 2026-02-24T22:16:33.452 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.453+0000 7f9585400a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:16:33.452 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.453+0000 7f9585400a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:16:33.452 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.453+0000 7f9585400a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:16:33.452 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.453+0000 7f9585400a80 20 zonegroup default 2026-02-24T22:16:33.453 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.453+0000 7f9585400a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.454 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.453+0000 7f9585400a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:33.454 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.453+0000 7f9585400a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.454 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.457+0000 7f9585400a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:33.455 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.457+0000 7f9585400a80 20 started sync module instance, tier type = 2026-02-24T22:16:33.455 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.457+0000 7f9585400a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:16:33.456 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.457+0000 7f9585400a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:16:33.456 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.457+0000 7f9585400a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:16:33.456 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.457+0000 7f9585400a80 5 note: GC not initialized 2026-02-24T22:16:33.456 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.457+0000 7f95317e2640 20 reqs_thread_entry: start 2026-02-24T22:16:33.460 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.461+0000 7f95836d5640 0 -- 10.20.193.46:0/281283631 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55eeefef2af0 msgr2=0x55eeefee5d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:33.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.465+0000 7f9582ed4640 0 -- 10.20.193.46:0/281283631 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55eeefeee570 msgr2=0x55eeefee92e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:33.489 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.489+0000 7f9585400a80 20 init_complete bucket index max shards: 1 2026-02-24T22:16:33.489 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.489+0000 7f9585400a80 20 Filter name: none 2026-02-24T22:16:33.490 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.489+0000 7f949b7fe640 20 reqs_thread_entry: start 2026-02-24T22:16:33.490 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.489+0000 7f9585400a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.491 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.489+0000 7f9585400a80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:16:33.491 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.489+0000 7f9585400a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.491 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.493+0000 7f9585400a80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:16:33.491 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.493+0000 7f9585400a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:16:33.491 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.493+0000 7f9585400a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.491 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.493+0000 7f9585400a80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:16:33.491 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.493+0000 7f9585400a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.491 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.493+0000 7f9585400a80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:16:33.491 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.493+0000 7f9585400a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:16:33.502 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.502 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:16:33.502 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-init-index id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521 num_objects: 20 size_kb: 1 num_shards: 2 2026-02-24T22:16:33.502 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.503 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.503 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:33.506 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.506 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:33.551 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.552 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:33.563 INFO:tasks.workunit.client.0.trial046.stderr:TEST: reshard bucket with abort at init_index 2026-02-24T22:16:33.563 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.564 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.564 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:33.573 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.573 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:33.616 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.616 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:33.621 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.621 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:33.628 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.628 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:33.634 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.634 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:33.640 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.640 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:33.646 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.647 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:33.652 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.653 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:33.659 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.659 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:33.665 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.665 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:33.672 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.672 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:33.678 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.678 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:33.684 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.684 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:33.690 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.690 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:33.696 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.696 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:33.702 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.702 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:33.708 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.708 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:33.714 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.714 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:33.720 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.720 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:33.726 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.726 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:33.733 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.733 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:33.738 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket abort-at-init-indext 2026-02-24T22:16:33.753 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:16:33.754 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:16:33.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.773+0000 7fb1bfc50a80 4 Realm: () 2026-02-24T22:16:33.773 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.773+0000 7fb1bfc50a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:16:33.773 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.773+0000 7fb1bfc50a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:16:33.773 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.773+0000 7fb1bfc50a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:16:33.773 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.773+0000 7fb1bfc50a80 20 zonegroup default 2026-02-24T22:16:33.773 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.773+0000 7fb1bfc50a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.774 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.773+0000 7fb1bfc50a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:33.774 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.773+0000 7fb1bfc50a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.775 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.777+0000 7fb1bfc50a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:33.775 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.777+0000 7fb1bfc50a80 20 started sync module instance, tier type = 2026-02-24T22:16:33.775 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.777+0000 7fb1bfc50a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:16:33.776 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.777+0000 7fb1bfc50a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:16:33.776 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.777+0000 7fb1bfc50a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:16:33.776 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.777+0000 7fb1bfc50a80 5 note: GC not initialized 2026-02-24T22:16:33.776 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.777+0000 7fb16afe5640 20 reqs_thread_entry: start 2026-02-24T22:16:33.780 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.781+0000 7fb1bdf25640 0 -- 10.20.193.46:0/1985865011 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55634963fc60 msgr2=0x556349636450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:16:33.786 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.785+0000 7fb1bd724640 0 -- 10.20.193.46:0/1985865011 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x556349611060 msgr2=0x55634963ec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:33.809 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.809+0000 7fb1bfc50a80 20 init_complete bucket index max shards: 1 2026-02-24T22:16:33.809 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.809+0000 7fb1bfc50a80 20 Filter name: none 2026-02-24T22:16:33.810 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.809+0000 7fb1697e2640 20 reqs_thread_entry: start 2026-02-24T22:16:33.810 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.809+0000 7fb1bfc50a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.810 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.813+0000 7fb1bfc50a80 20 rados_obj.operate() r=0 bl.length=216 2026-02-24T22:16:33.810 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.813+0000 7fb1bfc50a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.811 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.813+0000 7fb1bfc50a80 20 rados_obj.operate() r=0 bl.length=446 2026-02-24T22:16:33.811 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.813+0000 7fb1bfc50a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:16:33.811 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.813+0000 7fb1bfc50a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.813+0000 7fb1bfc50a80 20 rados_obj.operate() r=0 bl.length=216 2026-02-24T22:16:33.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.813+0000 7fb1bfc50a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.813+0000 7fb1bfc50a80 20 rados_obj.operate() r=0 bl.length=446 2026-02-24T22:16:33.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.813+0000 7fb1bfc50a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:16:33.821 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.822 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:16:33.822 INFO:tasks.workunit.client.0.trial046.stderr:bucket: abort-at-init-indext id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:16:33.822 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.822 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket abort-at-init-indext --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at init_index 2026-02-24T22:16:33.839 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:16:33.839 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:16:33.856 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.857+0000 7f2bf102ba80 4 Realm: () 2026-02-24T22:16:33.857 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.857+0000 7f2bf102ba80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:16:33.857 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.857+0000 7f2bf102ba80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:16:33.857 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:16:33.857 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.857+0000 7f2bf102ba80 20 zonegroup default 2026-02-24T22:16:33.857 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.857+0000 7f2bf102ba80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.858 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.857+0000 7f2bf102ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:33.858 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.857+0000 7f2bf102ba80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.858 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:33.858 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 started sync module instance, tier type = 2026-02-24T22:16:33.858 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:16:33.859 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.861+0000 7f2beeaff640 0 -- 10.20.193.46:0/4246297721 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x5566959d8390 msgr2=0x5566959f8810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:33.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 add_watcher() i=1 2026-02-24T22:16:33.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 add_watcher() i=3 2026-02-24T22:16:33.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 add_watcher() i=0 2026-02-24T22:16:33.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 add_watcher() i=6 2026-02-24T22:16:33.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 add_watcher() i=5 2026-02-24T22:16:33.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 add_watcher() i=2 2026-02-24T22:16:33.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 add_watcher() i=4 2026-02-24T22:16:33.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 add_watcher() i=7 2026-02-24T22:16:33.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.861+0000 7f2bf102ba80 2 all 8 watchers are set, enabling cache 2026-02-24T22:16:33.862 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.861+0000 7f2bee2fe640 0 -- 10.20.193.46:0/4102099818 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f2b10008af0 msgr2=0x7f2b10028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:33.863 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.865+0000 7f2bf102ba80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:16:33.863 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.865+0000 7f2bf102ba80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:16:33.863 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 note: GC not initialized 2026-02-24T22:16:33.863 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.865+0000 7f2b9cfe1640 20 reqs_thread_entry: start 2026-02-24T22:16:33.894 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.893+0000 7f2bf102ba80 20 init_complete bucket index max shards: 1 2026-02-24T22:16:33.894 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.893+0000 7f2bf102ba80 20 Filter name: none 2026-02-24T22:16:33.894 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.893+0000 7f2b06ffd640 20 reqs_thread_entry: start 2026-02-24T22:16:33.894 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.893+0000 7f2bf102ba80 10 cache get: name=default.rgw.meta+root+abort-at-init-indext : miss 2026-02-24T22:16:33.894 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.893+0000 7f2bf102ba80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.895 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.897+0000 7f2bf102ba80 20 rados_obj.operate() r=0 bl.length=216 2026-02-24T22:16:33.895 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.897+0000 7f2bf102ba80 10 cache put: name=default.rgw.meta+root+abort-at-init-indext info.flags=0x11 2026-02-24T22:16:33.896 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.897+0000 7f2bf102ba80 10 adding default.rgw.meta+root+abort-at-init-indext to cache LRU end 2026-02-24T22:16:33.896 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.897+0000 7f2bf102ba80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 : miss 2026-02-24T22:16:33.896 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.897+0000 7f2bf102ba80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.896 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.897+0000 7f2bf102ba80 20 rados_obj.operate() r=0 bl.length=446 2026-02-24T22:16:33.896 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.897+0000 7f2bf102ba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 info.flags=0x17 2026-02-24T22:16:33.896 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.897+0000 7f2bf102ba80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 to cache LRU end 2026-02-24T22:16:33.896 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.897+0000 7f2bf102ba80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:16:33.896 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.897+0000 7f2bf102ba80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:16:33.896 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.897+0000 7f2bf102ba80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 2026-02-24T22:16:33.896 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f2bf102ba80 time 2026-02-24T22:16:33.900926+0000 2026-02-24T22:16:33.896 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:16:33.931 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:16:33.931 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f2bf446deb1] 2026-02-24T22:16:33.931 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x605b01) [0x55667cc58b01] 2026-02-24T22:16:33.931 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x55667d2c4202] 2026-02-24T22:16:33.932 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:16:33.932 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f2bf376dd90] 2026-02-24T22:16:33.932 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:16:33.932 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:16:33.932 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.897+0000 7f2bf102ba80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f2bf102ba80 time 2026-02-24T22:16:33.900926+0000 2026-02-24T22:16:33.932 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:16:33.932 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.932 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:16:33.932 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f2bf446deb1] 2026-02-24T22:16:33.932 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x605b01) [0x55667cc58b01] 2026-02-24T22:16:33.932 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x55667d2c4202] 2026-02-24T22:16:33.932 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:16:33.932 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f2bf376dd90] 2026-02-24T22:16:33.932 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:16:33.932 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:16:33.933 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.933 INFO:tasks.workunit.client.0.trial046.stderr:*** Caught signal (Aborted) ** 2026-02-24T22:16:33.933 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f2bf102ba80 thread_name:radosgw-admin 2026-02-24T22:16:33.933 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:16:33.933 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f2bf3786520] 2026-02-24T22:16:33.933 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:16:33.933 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:16:33.933 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:16:33.933 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f2bf446df10] 2026-02-24T22:16:33.933 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x605b01) [0x55667cc58b01] 2026-02-24T22:16:33.933 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x55667d2c4202] 2026-02-24T22:16:33.933 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:16:33.933 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f2bf376dd90] 2026-02-24T22:16:33.933 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:16:33.933 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:16:33.933 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.897+0000 7f2bf102ba80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:16:33.933 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f2bf102ba80 thread_name:radosgw-admin 2026-02-24T22:16:33.934 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.934 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:16:33.934 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f2bf3786520] 2026-02-24T22:16:33.934 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:16:33.934 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:16:33.934 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:16:33.934 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f2bf446df10] 2026-02-24T22:16:33.934 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x605b01) [0x55667cc58b01] 2026-02-24T22:16:33.934 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x55667d2c4202] 2026-02-24T22:16:33.934 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:16:33.934 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f2bf376dd90] 2026-02-24T22:16:33.934 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:16:33.934 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:16:33.934 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:16:33.934 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.934 INFO:tasks.workunit.client.0.trial046.stderr:--- begin dump of recent events --- 2026-02-24T22:16:33.934 INFO:tasks.workunit.client.0.trial046.stderr: -499> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30700) adding con mode: secure 2026-02-24T22:16:33.935 INFO:tasks.workunit.client.0.trial046.stderr: -498> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30700) adding con mode: crc 2026-02-24T22:16:33.935 INFO:tasks.workunit.client.0.trial046.stderr: -497> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30700) adding con mode: secure 2026-02-24T22:16:33.935 INFO:tasks.workunit.client.0.trial046.stderr: -496> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.935 INFO:tasks.workunit.client.0.trial046.stderr: -495> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.935 INFO:tasks.workunit.client.0.trial046.stderr: -494> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command rotate-key hook 0x7ffc81d30848 2026-02-24T22:16:33.935 INFO:tasks.workunit.client.0.trial046.stderr: -493> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient: _reopen_session rank -1 2026-02-24T22:16:33.935 INFO:tasks.workunit.client.0.trial046.stderr: -492> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:16:33.935 INFO:tasks.workunit.client.0.trial046.stderr: -491> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-c con 0x556695411fe0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:16:33.935 INFO:tasks.workunit.client.0.trial046.stderr: -490> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-a con 0x5566954bab00 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:16:33.935 INFO:tasks.workunit.client.0.trial046.stderr: -489> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-b con 0x5566954bdcf0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:16:33.935 INFO:tasks.workunit.client.0.trial046.stderr: -488> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.935 INFO:tasks.workunit.client.0.trial046.stderr: -487> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.935 INFO:tasks.workunit.client.0.trial046.stderr: -486> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.935 INFO:tasks.workunit.client.0.trial046.stderr: -485> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): _renew_subs 2026-02-24T22:16:33.935 INFO:tasks.workunit.client.0.trial046.stderr: -484> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): authenticate will time out at 1763.953538s 2026-02-24T22:16:33.935 INFO:tasks.workunit.client.0.trial046.stderr: -483> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): get_auth_request con 0x5566954bab00 auth_method 0 2026-02-24T22:16:33.935 INFO:tasks.workunit.client.0.trial046.stderr: -482> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:16:33.935 INFO:tasks.workunit.client.0.trial046.stderr: -481> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:16:33.936 INFO:tasks.workunit.client.0.trial046.stderr: -480> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:16:33.936 INFO:tasks.workunit.client.0.trial046.stderr: -479> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): get_auth_request con 0x556695411fe0 auth_method 0 2026-02-24T22:16:33.936 INFO:tasks.workunit.client.0.trial046.stderr: -478> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:16:33.936 INFO:tasks.workunit.client.0.trial046.stderr: -477> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:16:33.936 INFO:tasks.workunit.client.0.trial046.stderr: -476> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:16:33.936 INFO:tasks.workunit.client.0.trial046.stderr: -475> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:16:33.936 INFO:tasks.workunit.client.0.trial046.stderr: -474> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:16:33.936 INFO:tasks.workunit.client.0.trial046.stderr: -473> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:16:33.936 INFO:tasks.workunit.client.0.trial046.stderr: -472> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:16:33.936 INFO:tasks.workunit.client.0.trial046.stderr: -471> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:16:33.936 INFO:tasks.workunit.client.0.trial046.stderr: -470> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:16:33.936 INFO:tasks.workunit.client.0.trial046.stderr: -469> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_done global_id 6630 payload 274 2026-02-24T22:16:33.936 INFO:tasks.workunit.client.0.trial046.stderr: -468> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient: _finish_hunting 0 2026-02-24T22:16:33.936 INFO:tasks.workunit.client.0.trial046.stderr: -467> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 1 monclient: found mon.noname-a 2026-02-24T22:16:33.936 INFO:tasks.workunit.client.0.trial046.stderr: -466> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:16:33.936 INFO:tasks.workunit.client.0.trial046.stderr: -465> 2026-02-24T22:16:33.845+0000 7f2bd57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:16:33.936 INFO:tasks.workunit.client.0.trial046.stderr: -464> 2026-02-24T22:16:33.845+0000 7f2bd57fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:16:33.936 INFO:tasks.workunit.client.0.trial046.stderr: -463> 2026-02-24T22:16:33.845+0000 7f2bd57fa640 10 monclient: dump: 2026-02-24T22:16:33.936 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:16:33.936 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:16:33.937 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.937 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.937 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:16:33.937 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.937 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:16:33.937 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:16:33.937 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:16:33.937 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.937 INFO:tasks.workunit.client.0.trial046.stderr: -462> 2026-02-24T22:16:33.845+0000 7f2bd57fa640 10 monclient: _finish_auth 0 2026-02-24T22:16:33.937 INFO:tasks.workunit.client.0.trial046.stderr: -461> 2026-02-24T22:16:33.845+0000 7f2bd57fa640 10 monclient: _check_auth_tickets 2026-02-24T22:16:33.937 INFO:tasks.workunit.client.0.trial046.stderr: -460> 2026-02-24T22:16:33.845+0000 7f2bd57fa640 10 monclient: handle_config config(0 keys) 2026-02-24T22:16:33.937 INFO:tasks.workunit.client.0.trial046.stderr: -459> 2026-02-24T22:16:33.845+0000 7f2bd57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:16:33.937 INFO:tasks.workunit.client.0.trial046.stderr: -458> 2026-02-24T22:16:33.845+0000 7f2bd57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:16:33.937 INFO:tasks.workunit.client.0.trial046.stderr: -457> 2026-02-24T22:16:33.845+0000 7f2bd57fa640 10 monclient: dump: 2026-02-24T22:16:33.937 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:16:33.937 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:16:33.937 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.937 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.938 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:16:33.938 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.938 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:16:33.938 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:16:33.938 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:16:33.938 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.938 INFO:tasks.workunit.client.0.trial046.stderr: -456> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 monclient: authenticate success, global_id 6630 2026-02-24T22:16:33.938 INFO:tasks.workunit.client.0.trial046.stderr: -455> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient: get_monmap_and_config success 2026-02-24T22:16:33.938 INFO:tasks.workunit.client.0.trial046.stderr: -454> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 4 set_mon_vals no callback set 2026-02-24T22:16:33.938 INFO:tasks.workunit.client.0.trial046.stderr: -453> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient: shutdown 2026-02-24T22:16:33.938 INFO:tasks.workunit.client.0.trial046.stderr: -452> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 asok(0x556695383f00) unregister_commands rotate-key 2026-02-24T22:16:33.938 INFO:tasks.workunit.client.0.trial046.stderr: -451> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient: build_initial_monmap 2026-02-24T22:16:33.938 INFO:tasks.workunit.client.0.trial046.stderr: -450> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 1 build_initial for_mkfs: 0 2026-02-24T22:16:33.938 INFO:tasks.workunit.client.0.trial046.stderr: -449> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient: monmap: 2026-02-24T22:16:33.938 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:16:33.938 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:16:33.938 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:16:33.938 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:16:33.938 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:16:33.938 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.939 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:16:33.939 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:16:33.939 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:16:33.939 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.939 INFO:tasks.workunit.client.0.trial046.stderr: -448> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding auth protocol: cephx 2026-02-24T22:16:33.939 INFO:tasks.workunit.client.0.trial046.stderr: -447> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding auth protocol: cephx 2026-02-24T22:16:33.939 INFO:tasks.workunit.client.0.trial046.stderr: -446> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding auth protocol: cephx 2026-02-24T22:16:33.939 INFO:tasks.workunit.client.0.trial046.stderr: -445> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding con mode: secure 2026-02-24T22:16:33.939 INFO:tasks.workunit.client.0.trial046.stderr: -444> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding con mode: crc 2026-02-24T22:16:33.939 INFO:tasks.workunit.client.0.trial046.stderr: -443> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding con mode: secure 2026-02-24T22:16:33.939 INFO:tasks.workunit.client.0.trial046.stderr: -442> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding con mode: crc 2026-02-24T22:16:33.939 INFO:tasks.workunit.client.0.trial046.stderr: -441> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding con mode: secure 2026-02-24T22:16:33.939 INFO:tasks.workunit.client.0.trial046.stderr: -440> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding con mode: crc 2026-02-24T22:16:33.939 INFO:tasks.workunit.client.0.trial046.stderr: -439> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding con mode: crc 2026-02-24T22:16:33.939 INFO:tasks.workunit.client.0.trial046.stderr: -438> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding con mode: secure 2026-02-24T22:16:33.939 INFO:tasks.workunit.client.0.trial046.stderr: -437> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding con mode: crc 2026-02-24T22:16:33.939 INFO:tasks.workunit.client.0.trial046.stderr: -436> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding con mode: secure 2026-02-24T22:16:33.939 INFO:tasks.workunit.client.0.trial046.stderr: -435> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding con mode: crc 2026-02-24T22:16:33.939 INFO:tasks.workunit.client.0.trial046.stderr: -434> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding con mode: secure 2026-02-24T22:16:33.939 INFO:tasks.workunit.client.0.trial046.stderr: -433> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.939 INFO:tasks.workunit.client.0.trial046.stderr: -432> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 1 librados: starting msgr at 2026-02-24T22:16:33.940 INFO:tasks.workunit.client.0.trial046.stderr: -431> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 1 librados: starting objecter 2026-02-24T22:16:33.940 INFO:tasks.workunit.client.0.trial046.stderr: -430> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command objecter_requests hook 0x556695484b90 2026-02-24T22:16:33.940 INFO:tasks.workunit.client.0.trial046.stderr: -429> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 1 librados: setting wanted keys 2026-02-24T22:16:33.940 INFO:tasks.workunit.client.0.trial046.stderr: -428> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 1 librados: calling monclient init 2026-02-24T22:16:33.940 INFO:tasks.workunit.client.0.trial046.stderr: -427> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient: init 2026-02-24T22:16:33.940 INFO:tasks.workunit.client.0.trial046.stderr: -426> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding auth protocol: cephx 2026-02-24T22:16:33.940 INFO:tasks.workunit.client.0.trial046.stderr: -425> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding auth protocol: cephx 2026-02-24T22:16:33.940 INFO:tasks.workunit.client.0.trial046.stderr: -424> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding auth protocol: cephx 2026-02-24T22:16:33.940 INFO:tasks.workunit.client.0.trial046.stderr: -423> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding con mode: secure 2026-02-24T22:16:33.940 INFO:tasks.workunit.client.0.trial046.stderr: -422> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding con mode: crc 2026-02-24T22:16:33.940 INFO:tasks.workunit.client.0.trial046.stderr: -421> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding con mode: secure 2026-02-24T22:16:33.940 INFO:tasks.workunit.client.0.trial046.stderr: -420> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding con mode: crc 2026-02-24T22:16:33.940 INFO:tasks.workunit.client.0.trial046.stderr: -419> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding con mode: secure 2026-02-24T22:16:33.940 INFO:tasks.workunit.client.0.trial046.stderr: -418> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding con mode: crc 2026-02-24T22:16:33.940 INFO:tasks.workunit.client.0.trial046.stderr: -417> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding con mode: crc 2026-02-24T22:16:33.940 INFO:tasks.workunit.client.0.trial046.stderr: -416> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding con mode: secure 2026-02-24T22:16:33.940 INFO:tasks.workunit.client.0.trial046.stderr: -415> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding con mode: crc 2026-02-24T22:16:33.940 INFO:tasks.workunit.client.0.trial046.stderr: -414> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding con mode: secure 2026-02-24T22:16:33.940 INFO:tasks.workunit.client.0.trial046.stderr: -413> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding con mode: crc 2026-02-24T22:16:33.940 INFO:tasks.workunit.client.0.trial046.stderr: -412> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding con mode: secure 2026-02-24T22:16:33.940 INFO:tasks.workunit.client.0.trial046.stderr: -411> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.940 INFO:tasks.workunit.client.0.trial046.stderr: -410> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.940 INFO:tasks.workunit.client.0.trial046.stderr: -409> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command rotate-key hook 0x556695483538 2026-02-24T22:16:33.941 INFO:tasks.workunit.client.0.trial046.stderr: -408> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient: _reopen_session rank -1 2026-02-24T22:16:33.941 INFO:tasks.workunit.client.0.trial046.stderr: -407> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:16:33.941 INFO:tasks.workunit.client.0.trial046.stderr: -406> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-c con 0x556695411fe0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:16:33.941 INFO:tasks.workunit.client.0.trial046.stderr: -405> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-a con 0x5566954bab00 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:16:33.941 INFO:tasks.workunit.client.0.trial046.stderr: -404> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-b con 0x5566954bdcf0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:16:33.941 INFO:tasks.workunit.client.0.trial046.stderr: -403> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.941 INFO:tasks.workunit.client.0.trial046.stderr: -402> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.941 INFO:tasks.workunit.client.0.trial046.stderr: -401> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.941 INFO:tasks.workunit.client.0.trial046.stderr: -400> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): _renew_subs 2026-02-24T22:16:33.941 INFO:tasks.workunit.client.0.trial046.stderr: -399> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): authenticate will time out at 1763.954845s 2026-02-24T22:16:33.941 INFO:tasks.workunit.client.0.trial046.stderr: -398> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): get_auth_request con 0x5566954bab00 auth_method 0 2026-02-24T22:16:33.941 INFO:tasks.workunit.client.0.trial046.stderr: -397> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:16:33.941 INFO:tasks.workunit.client.0.trial046.stderr: -396> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:16:33.941 INFO:tasks.workunit.client.0.trial046.stderr: -395> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:16:33.941 INFO:tasks.workunit.client.0.trial046.stderr: -394> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): get_auth_request con 0x556695411fe0 auth_method 0 2026-02-24T22:16:33.941 INFO:tasks.workunit.client.0.trial046.stderr: -393> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:16:33.941 INFO:tasks.workunit.client.0.trial046.stderr: -392> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:16:33.941 INFO:tasks.workunit.client.0.trial046.stderr: -391> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:16:33.941 INFO:tasks.workunit.client.0.trial046.stderr: -390> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:16:33.941 INFO:tasks.workunit.client.0.trial046.stderr: -389> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:16:33.941 INFO:tasks.workunit.client.0.trial046.stderr: -388> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:16:33.941 INFO:tasks.workunit.client.0.trial046.stderr: -387> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:16:33.941 INFO:tasks.workunit.client.0.trial046.stderr: -386> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:16:33.942 INFO:tasks.workunit.client.0.trial046.stderr: -385> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:16:33.942 INFO:tasks.workunit.client.0.trial046.stderr: -384> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_done global_id 6633 payload 824 2026-02-24T22:16:33.942 INFO:tasks.workunit.client.0.trial046.stderr: -383> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient: _finish_hunting 0 2026-02-24T22:16:33.942 INFO:tasks.workunit.client.0.trial046.stderr: -382> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 1 monclient: found mon.noname-a 2026-02-24T22:16:33.942 INFO:tasks.workunit.client.0.trial046.stderr: -381> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:16:33.942 INFO:tasks.workunit.client.0.trial046.stderr: -380> 2026-02-24T22:16:33.845+0000 7f2bef300640 10 monclient: get_auth_request con 0x5566954bdcf0 auth_method 0 2026-02-24T22:16:33.942 INFO:tasks.workunit.client.0.trial046.stderr: -379> 2026-02-24T22:16:33.845+0000 7f2bd3ff7640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:16:33.942 INFO:tasks.workunit.client.0.trial046.stderr: -378> 2026-02-24T22:16:33.845+0000 7f2bd3ff7640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:16:33.942 INFO:tasks.workunit.client.0.trial046.stderr: -377> 2026-02-24T22:16:33.845+0000 7f2bd3ff7640 10 monclient: dump: 2026-02-24T22:16:33.942 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:16:33.942 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:16:33.942 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.942 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.942 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:16:33.942 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.942 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:16:33.942 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:16:33.942 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:16:33.942 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.942 INFO:tasks.workunit.client.0.trial046.stderr: -376> 2026-02-24T22:16:33.845+0000 7f2bd3ff7640 10 monclient: _finish_auth 0 2026-02-24T22:16:33.942 INFO:tasks.workunit.client.0.trial046.stderr: -375> 2026-02-24T22:16:33.845+0000 7f2bd3ff7640 10 monclient: _check_auth_tickets 2026-02-24T22:16:33.942 INFO:tasks.workunit.client.0.trial046.stderr: -374> 2026-02-24T22:16:33.845+0000 7f2bd3ff7640 10 monclient: handle_config config(0 keys) 2026-02-24T22:16:33.943 INFO:tasks.workunit.client.0.trial046.stderr: -373> 2026-02-24T22:16:33.845+0000 7f2bd3ff7640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:16:33.943 INFO:tasks.workunit.client.0.trial046.stderr: -372> 2026-02-24T22:16:33.845+0000 7f2bd3ff7640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:16:33.943 INFO:tasks.workunit.client.0.trial046.stderr: -371> 2026-02-24T22:16:33.845+0000 7f2bd3ff7640 10 monclient: dump: 2026-02-24T22:16:33.943 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:16:33.943 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:16:33.943 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.943 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.943 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:16:33.943 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.943 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:16:33.943 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:16:33.943 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:16:33.943 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.943 INFO:tasks.workunit.client.0.trial046.stderr: -370> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 monclient: authenticate success, global_id 6633 2026-02-24T22:16:33.943 INFO:tasks.workunit.client.0.trial046.stderr: -369> 2026-02-24T22:16:33.845+0000 7f2bd57fa640 4 set_mon_vals no callback set 2026-02-24T22:16:33.943 INFO:tasks.workunit.client.0.trial046.stderr: -368> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient: _renew_subs 2026-02-24T22:16:33.943 INFO:tasks.workunit.client.0.trial046.stderr: -367> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:16:33.943 INFO:tasks.workunit.client.0.trial046.stderr: -366> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient: _renew_subs 2026-02-24T22:16:33.943 INFO:tasks.workunit.client.0.trial046.stderr: -365> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:16:33.943 INFO:tasks.workunit.client.0.trial046.stderr: -364> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 1 librados: init done 2026-02-24T22:16:33.943 INFO:tasks.workunit.client.0.trial046.stderr: -363> 2026-02-24T22:16:33.849+0000 7f2bd3ff7640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:16:33.943 INFO:tasks.workunit.client.0.trial046.stderr: -362> 2026-02-24T22:16:33.849+0000 7f2bd3ff7640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:16:33.943 INFO:tasks.workunit.client.0.trial046.stderr: -361> 2026-02-24T22:16:33.849+0000 7f2bd3ff7640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:16:33.943 INFO:tasks.workunit.client.0.trial046.stderr: -360> 2026-02-24T22:16:33.849+0000 7f2beeaff640 10 monclient: get_auth_request con 0x7f2bcc0707a0 auth_method 0 2026-02-24T22:16:33.944 INFO:tasks.workunit.client.0.trial046.stderr: -359> 2026-02-24T22:16:33.849+0000 7f2bef300640 10 monclient: get_auth_request con 0x5566954ad2f0 auth_method 0 2026-02-24T22:16:33.944 INFO:tasks.workunit.client.0.trial046.stderr: -358> 2026-02-24T22:16:33.849+0000 7f2beeaff640 10 monclient: get_auth_request con 0x5566955114b0 auth_method 0 2026-02-24T22:16:33.944 INFO:tasks.workunit.client.0.trial046.stderr: -357> 2026-02-24T22:16:33.849+0000 7f2bee2fe640 10 monclient: get_auth_request con 0x556695512c90 auth_method 0 2026-02-24T22:16:33.944 INFO:tasks.workunit.client.0.trial046.stderr: -356> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 10 monclient: build_initial_monmap 2026-02-24T22:16:33.944 INFO:tasks.workunit.client.0.trial046.stderr: -355> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 1 build_initial for_mkfs: 0 2026-02-24T22:16:33.944 INFO:tasks.workunit.client.0.trial046.stderr: -354> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 10 monclient: monmap: 2026-02-24T22:16:33.944 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:16:33.944 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:16:33.944 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:16:33.944 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:16:33.944 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:16:33.944 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.944 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:16:33.944 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:16:33.944 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:16:33.944 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.944 INFO:tasks.workunit.client.0.trial046.stderr: -353> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding auth protocol: cephx 2026-02-24T22:16:33.944 INFO:tasks.workunit.client.0.trial046.stderr: -352> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding auth protocol: cephx 2026-02-24T22:16:33.944 INFO:tasks.workunit.client.0.trial046.stderr: -351> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding auth protocol: cephx 2026-02-24T22:16:33.944 INFO:tasks.workunit.client.0.trial046.stderr: -350> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding con mode: secure 2026-02-24T22:16:33.944 INFO:tasks.workunit.client.0.trial046.stderr: -349> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding con mode: crc 2026-02-24T22:16:33.944 INFO:tasks.workunit.client.0.trial046.stderr: -348> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding con mode: secure 2026-02-24T22:16:33.944 INFO:tasks.workunit.client.0.trial046.stderr: -347> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding con mode: crc 2026-02-24T22:16:33.944 INFO:tasks.workunit.client.0.trial046.stderr: -346> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding con mode: secure 2026-02-24T22:16:33.945 INFO:tasks.workunit.client.0.trial046.stderr: -345> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding con mode: crc 2026-02-24T22:16:33.945 INFO:tasks.workunit.client.0.trial046.stderr: -344> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding con mode: crc 2026-02-24T22:16:33.945 INFO:tasks.workunit.client.0.trial046.stderr: -343> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding con mode: secure 2026-02-24T22:16:33.945 INFO:tasks.workunit.client.0.trial046.stderr: -342> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding con mode: crc 2026-02-24T22:16:33.945 INFO:tasks.workunit.client.0.trial046.stderr: -341> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding con mode: secure 2026-02-24T22:16:33.945 INFO:tasks.workunit.client.0.trial046.stderr: -340> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding con mode: crc 2026-02-24T22:16:33.945 INFO:tasks.workunit.client.0.trial046.stderr: -339> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding con mode: secure 2026-02-24T22:16:33.945 INFO:tasks.workunit.client.0.trial046.stderr: -338> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.945 INFO:tasks.workunit.client.0.trial046.stderr: -337> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command objecter_requests cmddesc objecter_requests hook 0x55669551a520 EEXIST 2026-02-24T22:16:33.945 INFO:tasks.workunit.client.0.trial046.stderr: -336> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: init 2026-02-24T22:16:33.945 INFO:tasks.workunit.client.0.trial046.stderr: -335> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding auth protocol: cephx 2026-02-24T22:16:33.945 INFO:tasks.workunit.client.0.trial046.stderr: -334> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding auth protocol: cephx 2026-02-24T22:16:33.945 INFO:tasks.workunit.client.0.trial046.stderr: -333> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding auth protocol: cephx 2026-02-24T22:16:33.945 INFO:tasks.workunit.client.0.trial046.stderr: -332> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding con mode: secure 2026-02-24T22:16:33.945 INFO:tasks.workunit.client.0.trial046.stderr: -331> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding con mode: crc 2026-02-24T22:16:33.945 INFO:tasks.workunit.client.0.trial046.stderr: -330> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding con mode: secure 2026-02-24T22:16:33.945 INFO:tasks.workunit.client.0.trial046.stderr: -329> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding con mode: crc 2026-02-24T22:16:33.945 INFO:tasks.workunit.client.0.trial046.stderr: -328> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding con mode: secure 2026-02-24T22:16:33.945 INFO:tasks.workunit.client.0.trial046.stderr: -327> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding con mode: crc 2026-02-24T22:16:33.945 INFO:tasks.workunit.client.0.trial046.stderr: -326> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding con mode: crc 2026-02-24T22:16:33.945 INFO:tasks.workunit.client.0.trial046.stderr: -325> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding con mode: secure 2026-02-24T22:16:33.945 INFO:tasks.workunit.client.0.trial046.stderr: -324> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding con mode: crc 2026-02-24T22:16:33.945 INFO:tasks.workunit.client.0.trial046.stderr: -323> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding con mode: secure 2026-02-24T22:16:33.945 INFO:tasks.workunit.client.0.trial046.stderr: -322> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding con mode: crc 2026-02-24T22:16:33.945 INFO:tasks.workunit.client.0.trial046.stderr: -321> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding con mode: secure 2026-02-24T22:16:33.945 INFO:tasks.workunit.client.0.trial046.stderr: -320> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.946 INFO:tasks.workunit.client.0.trial046.stderr: -319> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.946 INFO:tasks.workunit.client.0.trial046.stderr: -318> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command rotate-key cmddesc rotate-key hook 0x556695513810 EEXIST 2026-02-24T22:16:33.946 INFO:tasks.workunit.client.0.trial046.stderr: -317> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: _reopen_session rank -1 2026-02-24T22:16:33.946 INFO:tasks.workunit.client.0.trial046.stderr: -316> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:16:33.946 INFO:tasks.workunit.client.0.trial046.stderr: -315> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-c con 0x55669566e4b0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:16:33.946 INFO:tasks.workunit.client.0.trial046.stderr: -314> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-a con 0x55669566c0e0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:16:33.946 INFO:tasks.workunit.client.0.trial046.stderr: -313> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-b con 0x556695669ad0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:16:33.946 INFO:tasks.workunit.client.0.trial046.stderr: -312> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.946 INFO:tasks.workunit.client.0.trial046.stderr: -311> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.946 INFO:tasks.workunit.client.0.trial046.stderr: -310> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.946 INFO:tasks.workunit.client.0.trial046.stderr: -309> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): _renew_subs 2026-02-24T22:16:33.946 INFO:tasks.workunit.client.0.trial046.stderr: -308> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): authenticate will time out at 1763.960536s 2026-02-24T22:16:33.946 INFO:tasks.workunit.client.0.trial046.stderr: -307> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): get_auth_request con 0x55669566e4b0 auth_method 0 2026-02-24T22:16:33.946 INFO:tasks.workunit.client.0.trial046.stderr: -306> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:16:33.946 INFO:tasks.workunit.client.0.trial046.stderr: -305> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:16:33.946 INFO:tasks.workunit.client.0.trial046.stderr: -304> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:16:33.946 INFO:tasks.workunit.client.0.trial046.stderr: -303> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:16:33.946 INFO:tasks.workunit.client.0.trial046.stderr: -302> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:16:33.946 INFO:tasks.workunit.client.0.trial046.stderr: -301> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:16:33.946 INFO:tasks.workunit.client.0.trial046.stderr: -300> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): get_auth_request con 0x55669566c0e0 auth_method 0 2026-02-24T22:16:33.946 INFO:tasks.workunit.client.0.trial046.stderr: -299> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:16:33.946 INFO:tasks.workunit.client.0.trial046.stderr: -298> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:16:33.946 INFO:tasks.workunit.client.0.trial046.stderr: -297> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr: -296> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr: -295> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr: -294> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr: -293> 2026-02-24T22:16:33.853+0000 7f2bef300640 10 monclient(hunting): get_auth_request con 0x556695669ad0 auth_method 0 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr: -292> 2026-02-24T22:16:33.853+0000 7f2bef300640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr: -291> 2026-02-24T22:16:33.853+0000 7f2bef300640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr: -290> 2026-02-24T22:16:33.853+0000 7f2bef300640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr: -289> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): handle_auth_done global_id 6233 payload 824 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr: -288> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient: _finish_hunting 0 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr: -287> 2026-02-24T22:16:33.853+0000 7f2beeaff640 1 monclient: found mon.noname-c 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr: -286> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr: -285> 2026-02-24T22:16:33.853+0000 7f2bd17f2640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr: -284> 2026-02-24T22:16:33.853+0000 7f2bd17f2640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr: -283> 2026-02-24T22:16:33.853+0000 7f2bd17f2640 10 monclient: dump: 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr: -282> 2026-02-24T22:16:33.853+0000 7f2bd17f2640 10 monclient: _finish_auth 0 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr: -281> 2026-02-24T22:16:33.853+0000 7f2bd17f2640 10 monclient: _check_auth_tickets 2026-02-24T22:16:33.947 INFO:tasks.workunit.client.0.trial046.stderr: -280> 2026-02-24T22:16:33.853+0000 7f2bd17f2640 10 monclient: handle_config config(0 keys) 2026-02-24T22:16:33.948 INFO:tasks.workunit.client.0.trial046.stderr: -279> 2026-02-24T22:16:33.853+0000 7f2bd17f2640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:16:33.948 INFO:tasks.workunit.client.0.trial046.stderr: -278> 2026-02-24T22:16:33.853+0000 7f2bd17f2640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:16:33.948 INFO:tasks.workunit.client.0.trial046.stderr: -277> 2026-02-24T22:16:33.853+0000 7f2bd17f2640 10 monclient: dump: 2026-02-24T22:16:33.948 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:16:33.948 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:16:33.948 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.948 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.948 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:16:33.948 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.948 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:16:33.948 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:16:33.948 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:16:33.948 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.948 INFO:tasks.workunit.client.0.trial046.stderr: -276> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 monclient: authenticate success, global_id 6233 2026-02-24T22:16:33.948 INFO:tasks.workunit.client.0.trial046.stderr: -275> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: _renew_subs 2026-02-24T22:16:33.948 INFO:tasks.workunit.client.0.trial046.stderr: -274> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:16:33.948 INFO:tasks.workunit.client.0.trial046.stderr: -273> 2026-02-24T22:16:33.853+0000 7f2befb01640 4 set_mon_vals no callback set 2026-02-24T22:16:33.948 INFO:tasks.workunit.client.0.trial046.stderr: -272> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: _renew_subs 2026-02-24T22:16:33.948 INFO:tasks.workunit.client.0.trial046.stderr: -271> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:16:33.948 INFO:tasks.workunit.client.0.trial046.stderr: -270> 2026-02-24T22:16:33.853+0000 7f2bd17f2640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:16:33.948 INFO:tasks.workunit.client.0.trial046.stderr: -269> 2026-02-24T22:16:33.853+0000 7f2bd17f2640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:16:33.948 INFO:tasks.workunit.client.0.trial046.stderr: -268> 2026-02-24T22:16:33.853+0000 7f2bd17f2640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:16:33.948 INFO:tasks.workunit.client.0.trial046.stderr: -267> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient: get_auth_request con 0x7f2bc00706b0 auth_method 0 2026-02-24T22:16:33.948 INFO:tasks.workunit.client.0.trial046.stderr: -266> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: get_monmap_and_config 2026-02-24T22:16:33.948 INFO:tasks.workunit.client.0.trial046.stderr: -265> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: build_initial_monmap 2026-02-24T22:16:33.948 INFO:tasks.workunit.client.0.trial046.stderr: -264> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 1 build_initial for_mkfs: 0 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr: -263> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: monmap: 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr: -262> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding auth protocol: cephx 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr: -261> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding auth protocol: cephx 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr: -260> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding auth protocol: cephx 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr: -259> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding con mode: secure 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr: -258> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding con mode: crc 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr: -257> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding con mode: secure 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr: -256> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding con mode: crc 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr: -255> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding con mode: secure 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr: -254> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding con mode: crc 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr: -253> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding con mode: crc 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr: -252> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding con mode: secure 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr: -251> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding con mode: crc 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr: -250> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding con mode: secure 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr: -249> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding con mode: crc 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr: -248> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding con mode: secure 2026-02-24T22:16:33.949 INFO:tasks.workunit.client.0.trial046.stderr: -247> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -246> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: init 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -245> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding auth protocol: cephx 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -244> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding auth protocol: cephx 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -243> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding auth protocol: cephx 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -242> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding con mode: secure 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -241> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding con mode: crc 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -240> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding con mode: secure 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -239> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding con mode: crc 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -238> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding con mode: secure 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -237> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding con mode: crc 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -236> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding con mode: crc 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -235> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding con mode: secure 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -234> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding con mode: crc 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -233> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding con mode: secure 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -232> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding con mode: crc 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -231> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding con mode: secure 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -230> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -229> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -228> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command rotate-key cmddesc rotate-key hook 0x7ffc81d30718 EEXIST 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -227> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: _reopen_session rank -1 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -226> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -225> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-c con 0x556695726690 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -224> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-a con 0x556695700610 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -223> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-b con 0x556695701220 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -222> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -221> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -220> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.950 INFO:tasks.workunit.client.0.trial046.stderr: -219> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): _renew_subs 2026-02-24T22:16:33.951 INFO:tasks.workunit.client.0.trial046.stderr: -218> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): authenticate will time out at 1763.962996s 2026-02-24T22:16:33.951 INFO:tasks.workunit.client.0.trial046.stderr: -217> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): get_auth_request con 0x556695726690 auth_method 0 2026-02-24T22:16:33.951 INFO:tasks.workunit.client.0.trial046.stderr: -216> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:16:33.951 INFO:tasks.workunit.client.0.trial046.stderr: -215> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:16:33.951 INFO:tasks.workunit.client.0.trial046.stderr: -214> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:16:33.951 INFO:tasks.workunit.client.0.trial046.stderr: -213> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): get_auth_request con 0x556695700610 auth_method 0 2026-02-24T22:16:33.951 INFO:tasks.workunit.client.0.trial046.stderr: -212> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:16:33.951 INFO:tasks.workunit.client.0.trial046.stderr: -211> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:16:33.951 INFO:tasks.workunit.client.0.trial046.stderr: -210> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:16:33.951 INFO:tasks.workunit.client.0.trial046.stderr: -209> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:16:33.951 INFO:tasks.workunit.client.0.trial046.stderr: -208> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:16:33.951 INFO:tasks.workunit.client.0.trial046.stderr: -207> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:16:33.951 INFO:tasks.workunit.client.0.trial046.stderr: -206> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:16:33.951 INFO:tasks.workunit.client.0.trial046.stderr: -205> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:16:33.951 INFO:tasks.workunit.client.0.trial046.stderr: -204> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:16:33.951 INFO:tasks.workunit.client.0.trial046.stderr: -203> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): handle_auth_done global_id 6236 payload 274 2026-02-24T22:16:33.951 INFO:tasks.workunit.client.0.trial046.stderr: -202> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient: _finish_hunting 0 2026-02-24T22:16:33.951 INFO:tasks.workunit.client.0.trial046.stderr: -201> 2026-02-24T22:16:33.853+0000 7f2beeaff640 1 monclient: found mon.noname-c 2026-02-24T22:16:33.951 INFO:tasks.workunit.client.0.trial046.stderr: -200> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:16:33.951 INFO:tasks.workunit.client.0.trial046.stderr: -199> 2026-02-24T22:16:33.853+0000 7f2bcaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:16:33.951 INFO:tasks.workunit.client.0.trial046.stderr: -198> 2026-02-24T22:16:33.853+0000 7f2bcaffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:16:33.951 INFO:tasks.workunit.client.0.trial046.stderr: -197> 2026-02-24T22:16:33.853+0000 7f2bcaffd640 10 monclient: dump: 2026-02-24T22:16:33.951 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:16:33.951 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:16:33.951 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.951 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr: -196> 2026-02-24T22:16:33.853+0000 7f2bcaffd640 10 monclient: _finish_auth 0 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr: -195> 2026-02-24T22:16:33.853+0000 7f2bcaffd640 10 monclient: _check_auth_tickets 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr: -194> 2026-02-24T22:16:33.853+0000 7f2bcaffd640 10 monclient: handle_config config(0 keys) 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr: -193> 2026-02-24T22:16:33.853+0000 7f2bcaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr: -192> 2026-02-24T22:16:33.853+0000 7f2bcaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr: -191> 2026-02-24T22:16:33.853+0000 7f2bcaffd640 10 monclient: dump: 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr: -190> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 monclient: authenticate success, global_id 6236 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr: -189> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: get_monmap_and_config success 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr: -188> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 4 set_mon_vals no callback set 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr: -187> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: shutdown 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr: -186> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: build_initial_monmap 2026-02-24T22:16:33.952 INFO:tasks.workunit.client.0.trial046.stderr: -185> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 1 build_initial for_mkfs: 0 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr: -184> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: monmap: 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr: -183> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding auth protocol: cephx 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr: -182> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding auth protocol: cephx 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr: -181> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding auth protocol: cephx 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr: -180> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding con mode: secure 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr: -179> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding con mode: crc 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr: -178> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding con mode: secure 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr: -177> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding con mode: crc 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr: -176> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding con mode: secure 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr: -175> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding con mode: crc 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr: -174> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding con mode: crc 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr: -173> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding con mode: secure 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr: -172> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding con mode: crc 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr: -171> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding con mode: secure 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr: -170> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding con mode: crc 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr: -169> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding con mode: secure 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr: -168> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr: -167> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 1 librados: starting msgr at 2026-02-24T22:16:33.953 INFO:tasks.workunit.client.0.trial046.stderr: -166> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 1 librados: starting objecter 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -165> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command objecter_requests cmddesc objecter_requests hook 0x5566953e57c0 EEXIST 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -164> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 1 librados: setting wanted keys 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -163> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 1 librados: calling monclient init 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -162> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient: init 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -161> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding auth protocol: cephx 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -160> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding auth protocol: cephx 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -159> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding auth protocol: cephx 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -158> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding con mode: secure 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -157> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding con mode: crc 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -156> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding con mode: secure 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -155> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding con mode: crc 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -154> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding con mode: secure 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -153> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding con mode: crc 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -152> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding con mode: crc 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -151> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding con mode: secure 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -150> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding con mode: crc 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -149> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding con mode: secure 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -148> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding con mode: crc 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -147> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding con mode: secure 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -146> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -145> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -144> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command rotate-key cmddesc rotate-key hook 0x5566956fe308 EEXIST 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -143> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient: _reopen_session rank -1 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -142> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -141> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-b con 0x556695701220 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -140> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-c con 0x556695726690 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -139> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-a con 0x55669572c650 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -138> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -137> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.954 INFO:tasks.workunit.client.0.trial046.stderr: -136> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -135> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient(hunting): _renew_subs 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -134> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient(hunting): authenticate will time out at 1763.964478s 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -133> 2026-02-24T22:16:33.857+0000 7f2bef300640 10 monclient(hunting): get_auth_request con 0x55669572c650 auth_method 0 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -132> 2026-02-24T22:16:33.857+0000 7f2bef300640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -131> 2026-02-24T22:16:33.857+0000 7f2bef300640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -130> 2026-02-24T22:16:33.857+0000 7f2bef300640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -129> 2026-02-24T22:16:33.857+0000 7f2bee2fe640 10 monclient(hunting): get_auth_request con 0x556695726690 auth_method 0 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -128> 2026-02-24T22:16:33.857+0000 7f2bee2fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -127> 2026-02-24T22:16:33.857+0000 7f2bee2fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -126> 2026-02-24T22:16:33.857+0000 7f2bee2fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -125> 2026-02-24T22:16:33.857+0000 7f2bef300640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -124> 2026-02-24T22:16:33.857+0000 7f2bef300640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -123> 2026-02-24T22:16:33.857+0000 7f2bef300640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -122> 2026-02-24T22:16:33.857+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -121> 2026-02-24T22:16:33.857+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -120> 2026-02-24T22:16:33.857+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -119> 2026-02-24T22:16:33.857+0000 7f2beeaff640 10 monclient(hunting): get_auth_request con 0x556695701220 auth_method 0 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -118> 2026-02-24T22:16:33.857+0000 7f2beeaff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -117> 2026-02-24T22:16:33.857+0000 7f2beeaff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -116> 2026-02-24T22:16:33.857+0000 7f2beeaff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -115> 2026-02-24T22:16:33.857+0000 7f2bef300640 10 monclient(hunting): handle_auth_done global_id 6642 payload 824 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -114> 2026-02-24T22:16:33.857+0000 7f2bef300640 10 monclient: _finish_hunting 0 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -113> 2026-02-24T22:16:33.857+0000 7f2bef300640 1 monclient: found mon.noname-a 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -112> 2026-02-24T22:16:33.857+0000 7f2bef300640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -111> 2026-02-24T22:16:33.857+0000 7f2bc97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -110> 2026-02-24T22:16:33.857+0000 7f2bc97fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr: -109> 2026-02-24T22:16:33.857+0000 7f2bc97fa640 10 monclient: dump: 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.955 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr: -108> 2026-02-24T22:16:33.857+0000 7f2bc97fa640 10 monclient: _finish_auth 0 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr: -107> 2026-02-24T22:16:33.857+0000 7f2bc97fa640 10 monclient: _check_auth_tickets 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr: -106> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 monclient: authenticate success, global_id 6642 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr: -105> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient: _renew_subs 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr: -104> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr: -103> 2026-02-24T22:16:33.857+0000 7f2bc97fa640 10 monclient: handle_config config(0 keys) 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr: -102> 2026-02-24T22:16:33.857+0000 7f2bc97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr: -101> 2026-02-24T22:16:33.857+0000 7f2bc97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr: -100> 2026-02-24T22:16:33.857+0000 7f2bc97fa640 10 monclient: dump: 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr: -99> 2026-02-24T22:16:33.857+0000 7f2bcaffd640 4 set_mon_vals no callback set 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr: -98> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient: _renew_subs 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr: -97> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr: -96> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 1 librados: init done 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr: -95> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command cr dump hook 0x55669575f8d8 2026-02-24T22:16:33.956 INFO:tasks.workunit.client.0.trial046.stderr: -94> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 4 Realm: () 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -93> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -92> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -91> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -90> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 20 zonegroup default 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -89> 2026-02-24T22:16:33.857+0000 7f2beeaff640 10 monclient: get_auth_request con 0x5566957c7b40 auth_method 0 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -88> 2026-02-24T22:16:33.857+0000 7f2bc97fa640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -87> 2026-02-24T22:16:33.857+0000 7f2bc97fa640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -86> 2026-02-24T22:16:33.857+0000 7f2bc97fa640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -85> 2026-02-24T22:16:33.857+0000 7f2bee2fe640 10 monclient: get_auth_request con 0x7f2bb0070730 auth_method 0 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -84> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -83> 2026-02-24T22:16:33.857+0000 7f2beeaff640 10 monclient: get_auth_request con 0x5566957eae60 auth_method 0 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -82> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -81> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -80> 2026-02-24T22:16:33.857+0000 7f2bee2fe640 10 monclient: get_auth_request con 0x55669580c480 auth_method 0 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -79> 2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -78> 2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 started sync module instance, tier type = 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -77> 2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -76> 2026-02-24T22:16:33.861+0000 7f2bef300640 10 monclient: get_auth_request con 0x55669586d8d0 auth_method 0 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -75> 2026-02-24T22:16:33.861+0000 7f2bef300640 10 monclient: get_auth_request con 0x556695975390 auth_method 0 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -74> 2026-02-24T22:16:33.861+0000 7f2beeaff640 10 monclient: get_auth_request con 0x5566958d02c0 auth_method 0 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -73> 2026-02-24T22:16:33.861+0000 7f2bee2fe640 10 monclient: get_auth_request con 0x556695933250 auth_method 0 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -72> 2026-02-24T22:16:33.861+0000 7f2beeaff640 10 monclient: get_auth_request con 0x5566959d8390 auth_method 0 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -71> 2026-02-24T22:16:33.861+0000 7f2beeaff640 0 -- 10.20.193.46:0/4246297721 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x5566959d8390 msgr2=0x5566959f8810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -70> 2026-02-24T22:16:33.861+0000 7f2bc97fa640 1 client.6642.objecter ms_handle_reset 0x5566959d8390 session 0x5566959d7c80 osd.6 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -69> 2026-02-24T22:16:33.861+0000 7f2bc97fa640 10 monclient: _renew_subs 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -68> 2026-02-24T22:16:33.861+0000 7f2bc97fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -67> 2026-02-24T22:16:33.861+0000 7f2beeaff640 10 monclient: get_auth_request con 0x7f2bb009d460 auth_method 0 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -66> 2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 add_watcher() i=1 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -65> 2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 add_watcher() i=3 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -64> 2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 add_watcher() i=0 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -63> 2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 add_watcher() i=6 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -62> 2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 add_watcher() i=5 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -61> 2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 add_watcher() i=2 2026-02-24T22:16:33.957 INFO:tasks.workunit.client.0.trial046.stderr: -60> 2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 add_watcher() i=4 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -59> 2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 add_watcher() i=7 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -58> 2026-02-24T22:16:33.861+0000 7f2bf102ba80 2 all 8 watchers are set, enabling cache 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -57> 2026-02-24T22:16:33.861+0000 7f2bee2fe640 10 monclient: get_auth_request con 0x7f2b10008af0 auth_method 0 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -56> 2026-02-24T22:16:33.861+0000 7f2bee2fe640 0 -- 10.20.193.46:0/4102099818 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f2b10008af0 msgr2=0x7f2b10028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -55> 2026-02-24T22:16:33.865+0000 7f2bd17f2640 1 client.6233.objecter ms_handle_reset 0x7f2b10008af0 session 0x7f2b10008470 osd.1 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -54> 2026-02-24T22:16:33.865+0000 7f2bd17f2640 10 monclient: _renew_subs 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -53> 2026-02-24T22:16:33.865+0000 7f2bd17f2640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -52> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding auth protocol: cephx 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -51> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding auth protocol: cephx 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -50> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding auth protocol: cephx 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -49> 2026-02-24T22:16:33.865+0000 7f2bee2fe640 10 monclient: get_auth_request con 0x7f2bc009d440 auth_method 0 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -48> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding con mode: secure 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -47> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding con mode: crc 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -46> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding con mode: secure 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -45> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding con mode: crc 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -44> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding con mode: secure 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -43> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding con mode: crc 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -42> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding con mode: crc 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -41> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding con mode: secure 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -40> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding con mode: crc 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -39> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding con mode: secure 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -38> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding con mode: crc 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -37> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding con mode: secure 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -36> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -35> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -34> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -33> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command cache list hook 0x55669572d060 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -32> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command cache inspect hook 0x55669572d060 2026-02-24T22:16:33.958 INFO:tasks.workunit.client.0.trial046.stderr: -31> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command cache erase hook 0x55669572d060 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -30> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command cache zap hook 0x55669572d060 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -29> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 note: GC not initialized 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -28> 2026-02-24T22:16:33.865+0000 7f2b9cfe1640 20 reqs_thread_entry: start 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -27> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command sync trace show hook 0x7f2be40cb030 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -26> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command sync trace history hook 0x7f2be40cb030 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -25> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command sync trace active hook 0x7f2be40cb030 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -24> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command sync trace active_short hook 0x7f2be40cb030 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -23> 2026-02-24T22:16:33.865+0000 7f2bef300640 10 monclient: get_auth_request con 0x5566959aa080 auth_method 0 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -22> 2026-02-24T22:16:33.865+0000 7f2beeaff640 10 monclient: get_auth_request con 0x5566959f13b0 auth_method 0 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -21> 2026-02-24T22:16:33.865+0000 7f2bee2fe640 10 monclient: get_auth_request con 0x5566959f0fb0 auth_method 0 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -20> 2026-02-24T22:16:33.869+0000 7f2bef300640 10 monclient: get_auth_request con 0x5566959a9aa0 auth_method 0 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -19> 2026-02-24T22:16:33.869+0000 7f2beeaff640 10 monclient: get_auth_request con 0x5566959f4200 auth_method 0 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -18> 2026-02-24T22:16:33.873+0000 7f2bee2fe640 10 monclient: get_auth_request con 0x5566959f70a0 auth_method 0 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -17> 2026-02-24T22:16:33.893+0000 7f2bf102ba80 20 init_complete bucket index max shards: 1 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -16> 2026-02-24T22:16:33.893+0000 7f2bf102ba80 20 Filter name: none 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -15> 2026-02-24T22:16:33.893+0000 7f2b06ffd640 20 reqs_thread_entry: start 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -14> 2026-02-24T22:16:33.893+0000 7f2bf102ba80 10 cache get: name=default.rgw.meta+root+abort-at-init-indext : miss 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -13> 2026-02-24T22:16:33.893+0000 7f2bf102ba80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -12> 2026-02-24T22:16:33.897+0000 7f2bf102ba80 20 rados_obj.operate() r=0 bl.length=216 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -11> 2026-02-24T22:16:33.897+0000 7f2bf102ba80 10 cache put: name=default.rgw.meta+root+abort-at-init-indext info.flags=0x11 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -10> 2026-02-24T22:16:33.897+0000 7f2bf102ba80 10 adding default.rgw.meta+root+abort-at-init-indext to cache LRU end 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -9> 2026-02-24T22:16:33.897+0000 7f2bf102ba80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 : miss 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -8> 2026-02-24T22:16:33.897+0000 7f2bf102ba80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -7> 2026-02-24T22:16:33.897+0000 7f2bf102ba80 20 rados_obj.operate() r=0 bl.length=446 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -6> 2026-02-24T22:16:33.897+0000 7f2bf102ba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 info.flags=0x17 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -5> 2026-02-24T22:16:33.897+0000 7f2bf102ba80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 to cache LRU end 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -4> 2026-02-24T22:16:33.897+0000 7f2bf102ba80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -3> 2026-02-24T22:16:33.897+0000 7f2bf102ba80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -2> 2026-02-24T22:16:33.897+0000 7f2bf102ba80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: -1> 2026-02-24T22:16:33.897+0000 7f2bf102ba80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f2bf102ba80 time 2026-02-24T22:16:33.900926+0000 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:16:33.959 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f2bf446deb1] 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x605b01) [0x55667cc58b01] 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x55667d2c4202] 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f2bf376dd90] 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 0> 2026-02-24T22:16:33.897+0000 7f2bf102ba80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f2bf102ba80 thread_name:radosgw-admin 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f2bf3786520] 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f2bf446df10] 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x605b01) [0x55667cc58b01] 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x55667d2c4202] 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f2bf376dd90] 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr:--- logging levels --- 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 none 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 lockdep 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 context 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 crush 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds 2026-02-24T22:16:33.960 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_balancer 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_locker 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log_expire 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_migrator 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 3/ 5 mds_quiesce 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 buffer 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 timer 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 filer 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 striper 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 objecter 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rados 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_mirror 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_replay 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_pwl 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 journaler 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objectcacher 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 immutable_obj_cache 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 client 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 osd 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 optracker 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objclass 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 filestore 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 journal 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 ms 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mon 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 0/10 monc 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 paxos 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 tp 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 auth 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 crypto 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 finisher 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 reserver 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 heartbeatmap 2026-02-24T22:16:33.961 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 perfcounter 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_sync 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_datacache 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_access 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dbstore 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_flight 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_lifecycle 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_restore 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw_notification 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_bucket_logging 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 javaclient 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 asok 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 throttle 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 refs 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 compressor 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore_compression 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluefs 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 bdev 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 kstore 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 4/ 5 rocksdb 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 fuse 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 2/ 5 mgr 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mgrc 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 dpdk 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 eventtrace 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 prioritycache 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 test 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephfs_mirror 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephsqlite 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 crimson_interrupt 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_onode 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_odata 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_omap 2026-02-24T22:16:33.962 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_tm 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_t 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cleaner 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_epm 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_lba 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_fixedkv_tree 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cache 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_journal 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_device 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_backref 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 alienstore 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mclock 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dedup 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cyanstore 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 ceph_exporter 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 memstore 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 trace 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 ceph_dedup 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: -2/-2 (syslog threshold) 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 99/99 (stderr threshold) 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr:--- pthread ID / name mapping for recent threads --- 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 7f2b06ffd640 / http_manager 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 7f2b9cfe1640 / http_manager 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 7f2bc97fa640 / ms_dispatch 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 7f2bcaffd640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 7f2bd17f2640 / ms_dispatch 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 7f2bd3ff7640 / ms_dispatch 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 7f2bd57fa640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 7f2bd5ffb640 / admin_socket 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 7f2bedafd640 / ms_dispatch 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 7f2bee2fe640 / msgr-worker-2 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 7f2beeaff640 / msgr-worker-1 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 7f2bef300640 / msgr-worker-0 2026-02-24T22:16:33.963 INFO:tasks.workunit.client.0.trial046.stderr: 7f2befb01640 / io_context_pool 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: 7f2bf102ba80 / radosgw-admin 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: max_recent 500 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: max_new 1000 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: log_file /var/log/ceph/ceph-client.admin.42148.log 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr:--- end dump of recent events --- 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr:--- begin dump of recent events --- 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -499> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30700) adding con mode: secure 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -498> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30700) adding con mode: crc 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -497> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30700) adding con mode: secure 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -496> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -495> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -494> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command rotate-key hook 0x7ffc81d30848 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -493> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient: _reopen_session rank -1 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -492> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -491> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-c con 0x556695411fe0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -490> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-a con 0x5566954bab00 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -489> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-b con 0x5566954bdcf0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -488> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -487> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -486> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -485> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): _renew_subs 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -484> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): authenticate will time out at 1763.953538s 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -483> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): get_auth_request con 0x5566954bab00 auth_method 0 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -482> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -481> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -480> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -479> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): get_auth_request con 0x556695411fe0 auth_method 0 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -478> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -477> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -476> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -475> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -474> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -473> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -472> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -471> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:16:33.964 INFO:tasks.workunit.client.0.trial046.stderr: -470> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr: -469> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_done global_id 6630 payload 274 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr: -468> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient: _finish_hunting 0 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr: -467> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 1 monclient: found mon.noname-a 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr: -466> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr: -465> 2026-02-24T22:16:33.845+0000 7f2bd57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr: -464> 2026-02-24T22:16:33.845+0000 7f2bd57fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr: -463> 2026-02-24T22:16:33.845+0000 7f2bd57fa640 10 monclient: dump: 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr: -462> 2026-02-24T22:16:33.845+0000 7f2bd57fa640 10 monclient: _finish_auth 0 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr: -461> 2026-02-24T22:16:33.845+0000 7f2bd57fa640 10 monclient: _check_auth_tickets 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr: -460> 2026-02-24T22:16:33.845+0000 7f2bd57fa640 10 monclient: handle_config config(0 keys) 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr: -459> 2026-02-24T22:16:33.845+0000 7f2bd57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr: -458> 2026-02-24T22:16:33.845+0000 7f2bd57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr: -457> 2026-02-24T22:16:33.845+0000 7f2bd57fa640 10 monclient: dump: 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr: -456> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 monclient: authenticate success, global_id 6630 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr: -455> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient: get_monmap_and_config success 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr: -454> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 4 set_mon_vals no callback set 2026-02-24T22:16:33.965 INFO:tasks.workunit.client.0.trial046.stderr: -453> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient: shutdown 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -452> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 asok(0x556695383f00) unregister_commands rotate-key 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -451> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient: build_initial_monmap 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -450> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 1 build_initial for_mkfs: 0 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -449> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient: monmap: 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -448> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding auth protocol: cephx 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -447> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding auth protocol: cephx 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -446> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding auth protocol: cephx 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -445> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding con mode: secure 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -444> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding con mode: crc 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -443> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding con mode: secure 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -442> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding con mode: crc 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -441> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding con mode: secure 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -440> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding con mode: crc 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -439> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding con mode: crc 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -438> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding con mode: secure 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -437> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding con mode: crc 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -436> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding con mode: secure 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -435> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding con mode: crc 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -434> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954b69b0) adding con mode: secure 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -433> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -432> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 1 librados: starting msgr at 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -431> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 1 librados: starting objecter 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -430> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command objecter_requests hook 0x556695484b90 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -429> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 1 librados: setting wanted keys 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -428> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 1 librados: calling monclient init 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -427> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient: init 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -426> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding auth protocol: cephx 2026-02-24T22:16:33.966 INFO:tasks.workunit.client.0.trial046.stderr: -425> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding auth protocol: cephx 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -424> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding auth protocol: cephx 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -423> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding con mode: secure 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -422> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding con mode: crc 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -421> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding con mode: secure 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -420> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding con mode: crc 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -419> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding con mode: secure 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -418> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding con mode: crc 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -417> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding con mode: crc 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -416> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding con mode: secure 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -415> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding con mode: crc 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -414> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding con mode: secure 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -413> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding con mode: crc 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -412> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 AuthRegistry(0x5566954833f0) adding con mode: secure 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -411> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -410> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -409> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command rotate-key hook 0x556695483538 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -408> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient: _reopen_session rank -1 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -407> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -406> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-c con 0x556695411fe0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -405> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-a con 0x5566954bab00 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -404> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-b con 0x5566954bdcf0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -403> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -402> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -401> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -400> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): _renew_subs 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -399> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient(hunting): authenticate will time out at 1763.954845s 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -398> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): get_auth_request con 0x5566954bab00 auth_method 0 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -397> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -396> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -395> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -394> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): get_auth_request con 0x556695411fe0 auth_method 0 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -393> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -392> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -391> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -390> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -389> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -388> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -387> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:16:33.967 INFO:tasks.workunit.client.0.trial046.stderr: -386> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr: -385> 2026-02-24T22:16:33.845+0000 7f2beeaff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr: -384> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_done global_id 6633 payload 824 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr: -383> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient: _finish_hunting 0 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr: -382> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 1 monclient: found mon.noname-a 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr: -381> 2026-02-24T22:16:33.845+0000 7f2bee2fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr: -380> 2026-02-24T22:16:33.845+0000 7f2bef300640 10 monclient: get_auth_request con 0x5566954bdcf0 auth_method 0 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr: -379> 2026-02-24T22:16:33.845+0000 7f2bd3ff7640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr: -378> 2026-02-24T22:16:33.845+0000 7f2bd3ff7640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr: -377> 2026-02-24T22:16:33.845+0000 7f2bd3ff7640 10 monclient: dump: 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr: -376> 2026-02-24T22:16:33.845+0000 7f2bd3ff7640 10 monclient: _finish_auth 0 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr: -375> 2026-02-24T22:16:33.845+0000 7f2bd3ff7640 10 monclient: _check_auth_tickets 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr: -374> 2026-02-24T22:16:33.845+0000 7f2bd3ff7640 10 monclient: handle_config config(0 keys) 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr: -373> 2026-02-24T22:16:33.845+0000 7f2bd3ff7640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr: -372> 2026-02-24T22:16:33.845+0000 7f2bd3ff7640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr: -371> 2026-02-24T22:16:33.845+0000 7f2bd3ff7640 10 monclient: dump: 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr: -370> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 5 monclient: authenticate success, global_id 6633 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr: -369> 2026-02-24T22:16:33.845+0000 7f2bd57fa640 4 set_mon_vals no callback set 2026-02-24T22:16:33.968 INFO:tasks.workunit.client.0.trial046.stderr: -368> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient: _renew_subs 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -367> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -366> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient: _renew_subs 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -365> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -364> 2026-02-24T22:16:33.845+0000 7f2bf102ba80 1 librados: init done 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -363> 2026-02-24T22:16:33.849+0000 7f2bd3ff7640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -362> 2026-02-24T22:16:33.849+0000 7f2bd3ff7640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -361> 2026-02-24T22:16:33.849+0000 7f2bd3ff7640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -360> 2026-02-24T22:16:33.849+0000 7f2beeaff640 10 monclient: get_auth_request con 0x7f2bcc0707a0 auth_method 0 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -359> 2026-02-24T22:16:33.849+0000 7f2bef300640 10 monclient: get_auth_request con 0x5566954ad2f0 auth_method 0 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -358> 2026-02-24T22:16:33.849+0000 7f2beeaff640 10 monclient: get_auth_request con 0x5566955114b0 auth_method 0 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -357> 2026-02-24T22:16:33.849+0000 7f2bee2fe640 10 monclient: get_auth_request con 0x556695512c90 auth_method 0 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -356> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 10 monclient: build_initial_monmap 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -355> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 1 build_initial for_mkfs: 0 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -354> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 10 monclient: monmap: 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -353> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding auth protocol: cephx 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -352> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding auth protocol: cephx 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -351> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding auth protocol: cephx 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -350> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding con mode: secure 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -349> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding con mode: crc 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -348> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding con mode: secure 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -347> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding con mode: crc 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -346> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding con mode: secure 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -345> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding con mode: crc 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -344> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding con mode: crc 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -343> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding con mode: secure 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -342> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding con mode: crc 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -341> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding con mode: secure 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -340> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding con mode: crc 2026-02-24T22:16:33.969 INFO:tasks.workunit.client.0.trial046.stderr: -339> 2026-02-24T22:16:33.849+0000 7f2bf102ba80 5 AuthRegistry(0x556695515c20) adding con mode: secure 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -338> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -337> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command objecter_requests cmddesc objecter_requests hook 0x55669551a520 EEXIST 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -336> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: init 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -335> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding auth protocol: cephx 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -334> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding auth protocol: cephx 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -333> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding auth protocol: cephx 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -332> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding con mode: secure 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -331> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding con mode: crc 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -330> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding con mode: secure 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -329> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding con mode: crc 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -328> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding con mode: secure 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -327> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding con mode: crc 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -326> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding con mode: crc 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -325> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding con mode: secure 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -324> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding con mode: crc 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -323> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding con mode: secure 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -322> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding con mode: crc 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -321> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566955136c8) adding con mode: secure 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -320> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -319> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -318> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command rotate-key cmddesc rotate-key hook 0x556695513810 EEXIST 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -317> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: _reopen_session rank -1 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -316> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -315> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-c con 0x55669566e4b0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -314> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-a con 0x55669566c0e0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -313> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-b con 0x556695669ad0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -312> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -311> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -310> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -309> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): _renew_subs 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -308> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): authenticate will time out at 1763.960536s 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -307> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): get_auth_request con 0x55669566e4b0 auth_method 0 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -306> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -305> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -304> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -303> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -302> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -301> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -300> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): get_auth_request con 0x55669566c0e0 auth_method 0 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -299> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:16:33.970 INFO:tasks.workunit.client.0.trial046.stderr: -298> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr: -297> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr: -296> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr: -295> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr: -294> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr: -293> 2026-02-24T22:16:33.853+0000 7f2bef300640 10 monclient(hunting): get_auth_request con 0x556695669ad0 auth_method 0 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr: -292> 2026-02-24T22:16:33.853+0000 7f2bef300640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr: -291> 2026-02-24T22:16:33.853+0000 7f2bef300640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr: -290> 2026-02-24T22:16:33.853+0000 7f2bef300640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr: -289> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): handle_auth_done global_id 6233 payload 824 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr: -288> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient: _finish_hunting 0 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr: -287> 2026-02-24T22:16:33.853+0000 7f2beeaff640 1 monclient: found mon.noname-c 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr: -286> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr: -285> 2026-02-24T22:16:33.853+0000 7f2bd17f2640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr: -284> 2026-02-24T22:16:33.853+0000 7f2bd17f2640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr: -283> 2026-02-24T22:16:33.853+0000 7f2bd17f2640 10 monclient: dump: 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr: -282> 2026-02-24T22:16:33.853+0000 7f2bd17f2640 10 monclient: _finish_auth 0 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr: -281> 2026-02-24T22:16:33.853+0000 7f2bd17f2640 10 monclient: _check_auth_tickets 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr: -280> 2026-02-24T22:16:33.853+0000 7f2bd17f2640 10 monclient: handle_config config(0 keys) 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr: -279> 2026-02-24T22:16:33.853+0000 7f2bd17f2640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr: -278> 2026-02-24T22:16:33.853+0000 7f2bd17f2640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr: -277> 2026-02-24T22:16:33.853+0000 7f2bd17f2640 10 monclient: dump: 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:16:33.971 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -276> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 monclient: authenticate success, global_id 6233 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -275> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: _renew_subs 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -274> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -273> 2026-02-24T22:16:33.853+0000 7f2befb01640 4 set_mon_vals no callback set 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -272> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: _renew_subs 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -271> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -270> 2026-02-24T22:16:33.853+0000 7f2bd17f2640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -269> 2026-02-24T22:16:33.853+0000 7f2bd17f2640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -268> 2026-02-24T22:16:33.853+0000 7f2bd17f2640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -267> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient: get_auth_request con 0x7f2bc00706b0 auth_method 0 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -266> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: get_monmap_and_config 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -265> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: build_initial_monmap 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -264> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 1 build_initial for_mkfs: 0 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -263> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: monmap: 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -262> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding auth protocol: cephx 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -261> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding auth protocol: cephx 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -260> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding auth protocol: cephx 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -259> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding con mode: secure 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -258> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding con mode: crc 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -257> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding con mode: secure 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -256> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding con mode: crc 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -255> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding con mode: secure 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -254> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding con mode: crc 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -253> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding con mode: crc 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -252> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding con mode: secure 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -251> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding con mode: crc 2026-02-24T22:16:33.972 INFO:tasks.workunit.client.0.trial046.stderr: -250> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding con mode: secure 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -249> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding con mode: crc 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -248> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566956ff5a0) adding con mode: secure 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -247> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -246> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: init 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -245> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding auth protocol: cephx 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -244> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding auth protocol: cephx 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -243> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding auth protocol: cephx 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -242> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding con mode: secure 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -241> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding con mode: crc 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -240> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding con mode: secure 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -239> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding con mode: crc 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -238> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding con mode: secure 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -237> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding con mode: crc 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -236> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding con mode: crc 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -235> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding con mode: secure 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -234> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding con mode: crc 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -233> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding con mode: secure 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -232> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding con mode: crc 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -231> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d305d0) adding con mode: secure 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -230> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -229> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -228> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command rotate-key cmddesc rotate-key hook 0x7ffc81d30718 EEXIST 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -227> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: _reopen_session rank -1 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -226> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -225> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-c con 0x556695726690 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -224> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-a con 0x556695700610 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -223> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-b con 0x556695701220 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -222> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -221> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -220> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -219> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): _renew_subs 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -218> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient(hunting): authenticate will time out at 1763.962996s 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -217> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): get_auth_request con 0x556695726690 auth_method 0 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -216> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -215> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -214> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -213> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): get_auth_request con 0x556695700610 auth_method 0 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -212> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -211> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -210> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:16:33.973 INFO:tasks.workunit.client.0.trial046.stderr: -209> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr: -208> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr: -207> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr: -206> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr: -205> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr: -204> 2026-02-24T22:16:33.853+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr: -203> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient(hunting): handle_auth_done global_id 6236 payload 274 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr: -202> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient: _finish_hunting 0 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr: -201> 2026-02-24T22:16:33.853+0000 7f2beeaff640 1 monclient: found mon.noname-c 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr: -200> 2026-02-24T22:16:33.853+0000 7f2beeaff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr: -199> 2026-02-24T22:16:33.853+0000 7f2bcaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr: -198> 2026-02-24T22:16:33.853+0000 7f2bcaffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr: -197> 2026-02-24T22:16:33.853+0000 7f2bcaffd640 10 monclient: dump: 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr: -196> 2026-02-24T22:16:33.853+0000 7f2bcaffd640 10 monclient: _finish_auth 0 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr: -195> 2026-02-24T22:16:33.853+0000 7f2bcaffd640 10 monclient: _check_auth_tickets 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr: -194> 2026-02-24T22:16:33.853+0000 7f2bcaffd640 10 monclient: handle_config config(0 keys) 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr: -193> 2026-02-24T22:16:33.853+0000 7f2bcaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr: -192> 2026-02-24T22:16:33.853+0000 7f2bcaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr: -191> 2026-02-24T22:16:33.853+0000 7f2bcaffd640 10 monclient: dump: 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr: -190> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 monclient: authenticate success, global_id 6236 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr: -189> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: get_monmap_and_config success 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr: -188> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 4 set_mon_vals no callback set 2026-02-24T22:16:33.974 INFO:tasks.workunit.client.0.trial046.stderr: -187> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: shutdown 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -186> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: build_initial_monmap 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -185> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 1 build_initial for_mkfs: 0 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -184> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 10 monclient: monmap: 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -183> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding auth protocol: cephx 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -182> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding auth protocol: cephx 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -181> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding auth protocol: cephx 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -180> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding con mode: secure 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -179> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding con mode: crc 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -178> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding con mode: secure 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -177> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding con mode: crc 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -176> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding con mode: secure 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -175> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding con mode: crc 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -174> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding con mode: crc 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -173> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding con mode: secure 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -172> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding con mode: crc 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -171> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding con mode: secure 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -170> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding con mode: crc 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -169> 2026-02-24T22:16:33.853+0000 7f2bf102ba80 5 AuthRegistry(0x5566957324b0) adding con mode: secure 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -168> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -167> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 1 librados: starting msgr at 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -166> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 1 librados: starting objecter 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -165> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command objecter_requests cmddesc objecter_requests hook 0x5566953e57c0 EEXIST 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -164> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 1 librados: setting wanted keys 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -163> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 1 librados: calling monclient init 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -162> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient: init 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -161> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding auth protocol: cephx 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -160> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding auth protocol: cephx 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -159> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding auth protocol: cephx 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -158> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding con mode: secure 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -157> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding con mode: crc 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -156> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding con mode: secure 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -155> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding con mode: crc 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -154> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding con mode: secure 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -153> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding con mode: crc 2026-02-24T22:16:33.975 INFO:tasks.workunit.client.0.trial046.stderr: -152> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding con mode: crc 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -151> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding con mode: secure 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -150> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding con mode: crc 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -149> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding con mode: secure 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -148> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding con mode: crc 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -147> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 AuthRegistry(0x5566956fe1c0) adding con mode: secure 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -146> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -145> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -144> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command rotate-key cmddesc rotate-key hook 0x5566956fe308 EEXIST 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -143> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient: _reopen_session rank -1 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -142> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -141> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-b con 0x556695701220 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -140> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-c con 0x556695726690 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -139> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient(hunting): picked mon.noname-a con 0x55669572c650 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -138> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -137> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -136> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -135> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient(hunting): _renew_subs 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -134> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient(hunting): authenticate will time out at 1763.964478s 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -133> 2026-02-24T22:16:33.857+0000 7f2bef300640 10 monclient(hunting): get_auth_request con 0x55669572c650 auth_method 0 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -132> 2026-02-24T22:16:33.857+0000 7f2bef300640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -131> 2026-02-24T22:16:33.857+0000 7f2bef300640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -130> 2026-02-24T22:16:33.857+0000 7f2bef300640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -129> 2026-02-24T22:16:33.857+0000 7f2bee2fe640 10 monclient(hunting): get_auth_request con 0x556695726690 auth_method 0 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -128> 2026-02-24T22:16:33.857+0000 7f2bee2fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -127> 2026-02-24T22:16:33.857+0000 7f2bee2fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -126> 2026-02-24T22:16:33.857+0000 7f2bee2fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -125> 2026-02-24T22:16:33.857+0000 7f2bef300640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -124> 2026-02-24T22:16:33.857+0000 7f2bef300640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -123> 2026-02-24T22:16:33.857+0000 7f2bef300640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -122> 2026-02-24T22:16:33.857+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -121> 2026-02-24T22:16:33.857+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -120> 2026-02-24T22:16:33.857+0000 7f2bee2fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -119> 2026-02-24T22:16:33.857+0000 7f2beeaff640 10 monclient(hunting): get_auth_request con 0x556695701220 auth_method 0 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -118> 2026-02-24T22:16:33.857+0000 7f2beeaff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -117> 2026-02-24T22:16:33.857+0000 7f2beeaff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -116> 2026-02-24T22:16:33.857+0000 7f2beeaff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -115> 2026-02-24T22:16:33.857+0000 7f2bef300640 10 monclient(hunting): handle_auth_done global_id 6642 payload 824 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -114> 2026-02-24T22:16:33.857+0000 7f2bef300640 10 monclient: _finish_hunting 0 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -113> 2026-02-24T22:16:33.857+0000 7f2bef300640 1 monclient: found mon.noname-a 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -112> 2026-02-24T22:16:33.857+0000 7f2bef300640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -111> 2026-02-24T22:16:33.857+0000 7f2bc97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -110> 2026-02-24T22:16:33.857+0000 7f2bc97fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr: -109> 2026-02-24T22:16:33.857+0000 7f2bc97fa640 10 monclient: dump: 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:16:33.976 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -108> 2026-02-24T22:16:33.857+0000 7f2bc97fa640 10 monclient: _finish_auth 0 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -107> 2026-02-24T22:16:33.857+0000 7f2bc97fa640 10 monclient: _check_auth_tickets 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -106> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 monclient: authenticate success, global_id 6642 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -105> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient: _renew_subs 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -104> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -103> 2026-02-24T22:16:33.857+0000 7f2bc97fa640 10 monclient: handle_config config(0 keys) 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -102> 2026-02-24T22:16:33.857+0000 7f2bc97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -101> 2026-02-24T22:16:33.857+0000 7f2bc97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -100> 2026-02-24T22:16:33.857+0000 7f2bc97fa640 10 monclient: dump: 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -99> 2026-02-24T22:16:33.857+0000 7f2bcaffd640 4 set_mon_vals no callback set 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -98> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient: _renew_subs 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -97> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -96> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 1 librados: init done 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -95> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command cr dump hook 0x55669575f8d8 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -94> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 4 Realm: () 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -93> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -92> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -91> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -90> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 20 zonegroup default 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -89> 2026-02-24T22:16:33.857+0000 7f2beeaff640 10 monclient: get_auth_request con 0x5566957c7b40 auth_method 0 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -88> 2026-02-24T22:16:33.857+0000 7f2bc97fa640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -87> 2026-02-24T22:16:33.857+0000 7f2bc97fa640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -86> 2026-02-24T22:16:33.857+0000 7f2bc97fa640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -85> 2026-02-24T22:16:33.857+0000 7f2bee2fe640 10 monclient: get_auth_request con 0x7f2bb0070730 auth_method 0 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -84> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -83> 2026-02-24T22:16:33.857+0000 7f2beeaff640 10 monclient: get_auth_request con 0x5566957eae60 auth_method 0 2026-02-24T22:16:33.977 INFO:tasks.workunit.client.0.trial046.stderr: -82> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -81> 2026-02-24T22:16:33.857+0000 7f2bf102ba80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -80> 2026-02-24T22:16:33.857+0000 7f2bee2fe640 10 monclient: get_auth_request con 0x55669580c480 auth_method 0 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -79> 2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -78> 2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 started sync module instance, tier type = 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -77> 2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -76> 2026-02-24T22:16:33.861+0000 7f2bef300640 10 monclient: get_auth_request con 0x55669586d8d0 auth_method 0 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -75> 2026-02-24T22:16:33.861+0000 7f2bef300640 10 monclient: get_auth_request con 0x556695975390 auth_method 0 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -74> 2026-02-24T22:16:33.861+0000 7f2beeaff640 10 monclient: get_auth_request con 0x5566958d02c0 auth_method 0 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -73> 2026-02-24T22:16:33.861+0000 7f2bee2fe640 10 monclient: get_auth_request con 0x556695933250 auth_method 0 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -72> 2026-02-24T22:16:33.861+0000 7f2beeaff640 10 monclient: get_auth_request con 0x5566959d8390 auth_method 0 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -71> 2026-02-24T22:16:33.861+0000 7f2beeaff640 0 -- 10.20.193.46:0/4246297721 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x5566959d8390 msgr2=0x5566959f8810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -70> 2026-02-24T22:16:33.861+0000 7f2bc97fa640 1 client.6642.objecter ms_handle_reset 0x5566959d8390 session 0x5566959d7c80 osd.6 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -69> 2026-02-24T22:16:33.861+0000 7f2bc97fa640 10 monclient: _renew_subs 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -68> 2026-02-24T22:16:33.861+0000 7f2bc97fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -67> 2026-02-24T22:16:33.861+0000 7f2beeaff640 10 monclient: get_auth_request con 0x7f2bb009d460 auth_method 0 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -66> 2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 add_watcher() i=1 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -65> 2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 add_watcher() i=3 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -64> 2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 add_watcher() i=0 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -63> 2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 add_watcher() i=6 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -62> 2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 add_watcher() i=5 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -61> 2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 add_watcher() i=2 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -60> 2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 add_watcher() i=4 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -59> 2026-02-24T22:16:33.861+0000 7f2bf102ba80 20 add_watcher() i=7 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -58> 2026-02-24T22:16:33.861+0000 7f2bf102ba80 2 all 8 watchers are set, enabling cache 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -57> 2026-02-24T22:16:33.861+0000 7f2bee2fe640 10 monclient: get_auth_request con 0x7f2b10008af0 auth_method 0 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -56> 2026-02-24T22:16:33.861+0000 7f2bee2fe640 0 -- 10.20.193.46:0/4102099818 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f2b10008af0 msgr2=0x7f2b10028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -55> 2026-02-24T22:16:33.865+0000 7f2bd17f2640 1 client.6233.objecter ms_handle_reset 0x7f2b10008af0 session 0x7f2b10008470 osd.1 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -54> 2026-02-24T22:16:33.865+0000 7f2bd17f2640 10 monclient: _renew_subs 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -53> 2026-02-24T22:16:33.865+0000 7f2bd17f2640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -52> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding auth protocol: cephx 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -51> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding auth protocol: cephx 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -50> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding auth protocol: cephx 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -49> 2026-02-24T22:16:33.865+0000 7f2bee2fe640 10 monclient: get_auth_request con 0x7f2bc009d440 auth_method 0 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -48> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding con mode: secure 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -47> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding con mode: crc 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -46> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding con mode: secure 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -45> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding con mode: crc 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -44> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding con mode: secure 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -43> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding con mode: crc 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -42> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding con mode: crc 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -41> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding con mode: secure 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -40> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding con mode: crc 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -39> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding con mode: secure 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -38> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding con mode: crc 2026-02-24T22:16:33.978 INFO:tasks.workunit.client.0.trial046.stderr: -37> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 AuthRegistry(0x7ffc81d30a40) adding con mode: secure 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -36> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -35> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -34> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -33> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command cache list hook 0x55669572d060 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -32> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command cache inspect hook 0x55669572d060 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -31> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command cache erase hook 0x55669572d060 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -30> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command cache zap hook 0x55669572d060 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -29> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 note: GC not initialized 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -28> 2026-02-24T22:16:33.865+0000 7f2b9cfe1640 20 reqs_thread_entry: start 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -27> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command sync trace show hook 0x7f2be40cb030 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -26> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command sync trace history hook 0x7f2be40cb030 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -25> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command sync trace active hook 0x7f2be40cb030 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -24> 2026-02-24T22:16:33.865+0000 7f2bf102ba80 5 asok(0x556695383f00) register_command sync trace active_short hook 0x7f2be40cb030 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -23> 2026-02-24T22:16:33.865+0000 7f2bef300640 10 monclient: get_auth_request con 0x5566959aa080 auth_method 0 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -22> 2026-02-24T22:16:33.865+0000 7f2beeaff640 10 monclient: get_auth_request con 0x5566959f13b0 auth_method 0 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -21> 2026-02-24T22:16:33.865+0000 7f2bee2fe640 10 monclient: get_auth_request con 0x5566959f0fb0 auth_method 0 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -20> 2026-02-24T22:16:33.869+0000 7f2bef300640 10 monclient: get_auth_request con 0x5566959a9aa0 auth_method 0 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -19> 2026-02-24T22:16:33.869+0000 7f2beeaff640 10 monclient: get_auth_request con 0x5566959f4200 auth_method 0 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -18> 2026-02-24T22:16:33.873+0000 7f2bee2fe640 10 monclient: get_auth_request con 0x5566959f70a0 auth_method 0 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -17> 2026-02-24T22:16:33.893+0000 7f2bf102ba80 20 init_complete bucket index max shards: 1 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -16> 2026-02-24T22:16:33.893+0000 7f2bf102ba80 20 Filter name: none 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -15> 2026-02-24T22:16:33.893+0000 7f2b06ffd640 20 reqs_thread_entry: start 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -14> 2026-02-24T22:16:33.893+0000 7f2bf102ba80 10 cache get: name=default.rgw.meta+root+abort-at-init-indext : miss 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -13> 2026-02-24T22:16:33.893+0000 7f2bf102ba80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -12> 2026-02-24T22:16:33.897+0000 7f2bf102ba80 20 rados_obj.operate() r=0 bl.length=216 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -11> 2026-02-24T22:16:33.897+0000 7f2bf102ba80 10 cache put: name=default.rgw.meta+root+abort-at-init-indext info.flags=0x11 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -10> 2026-02-24T22:16:33.897+0000 7f2bf102ba80 10 adding default.rgw.meta+root+abort-at-init-indext to cache LRU end 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -9> 2026-02-24T22:16:33.897+0000 7f2bf102ba80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 : miss 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -8> 2026-02-24T22:16:33.897+0000 7f2bf102ba80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -7> 2026-02-24T22:16:33.897+0000 7f2bf102ba80 20 rados_obj.operate() r=0 bl.length=446 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -6> 2026-02-24T22:16:33.897+0000 7f2bf102ba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 info.flags=0x17 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -5> 2026-02-24T22:16:33.897+0000 7f2bf102ba80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 to cache LRU end 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -4> 2026-02-24T22:16:33.897+0000 7f2bf102ba80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -3> 2026-02-24T22:16:33.897+0000 7f2bf102ba80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -2> 2026-02-24T22:16:33.897+0000 7f2bf102ba80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: -1> 2026-02-24T22:16:33.897+0000 7f2bf102ba80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f2bf102ba80 time 2026-02-24T22:16:33.900926+0000 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f2bf446deb1] 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x605b01) [0x55667cc58b01] 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x55667d2c4202] 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f2bf376dd90] 2026-02-24T22:16:33.979 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 0> 2026-02-24T22:16:33.897+0000 7f2bf102ba80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f2bf102ba80 thread_name:radosgw-admin 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f2bf3786520] 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f2bf446df10] 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x605b01) [0x55667cc58b01] 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x55667d2c4202] 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f2bf376dd90] 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr:--- logging levels --- 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 none 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 lockdep 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 context 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 crush 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_balancer 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_locker 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log_expire 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_migrator 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 3/ 5 mds_quiesce 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 buffer 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 timer 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 filer 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 striper 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 objecter 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rados 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_mirror 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_replay 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_pwl 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 journaler 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objectcacher 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 immutable_obj_cache 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 client 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 osd 2026-02-24T22:16:33.980 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 optracker 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objclass 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 filestore 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 journal 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 ms 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mon 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 0/10 monc 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 paxos 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 tp 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 auth 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 crypto 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 finisher 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 reserver 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 heartbeatmap 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 perfcounter 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_sync 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_datacache 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_access 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dbstore 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_flight 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_lifecycle 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_restore 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw_notification 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_bucket_logging 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 javaclient 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 asok 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 throttle 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 refs 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 compressor 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore_compression 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluefs 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 bdev 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 kstore 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 4/ 5 rocksdb 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 fuse 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 2/ 5 mgr 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mgrc 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 dpdk 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 eventtrace 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 prioritycache 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 test 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephfs_mirror 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephsqlite 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 crimson_interrupt 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_onode 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_odata 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_omap 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_tm 2026-02-24T22:16:33.981 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_t 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cleaner 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_epm 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_lba 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_fixedkv_tree 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cache 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_journal 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_device 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_backref 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 alienstore 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mclock 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dedup 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cyanstore 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 ceph_exporter 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 memstore 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 trace 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 ceph_dedup 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: -2/-2 (syslog threshold) 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 99/99 (stderr threshold) 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr:--- pthread ID / name mapping for recent threads --- 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 7f2b06ffd640 / http_manager 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 7f2b9cfe1640 / http_manager 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 7f2bc97fa640 / ms_dispatch 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 7f2bcaffd640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 7f2bd17f2640 / ms_dispatch 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 7f2bd3ff7640 / ms_dispatch 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 7f2bd57fa640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 7f2bd5ffb640 / admin_socket 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 7f2bedafd640 / ms_dispatch 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 7f2bee2fe640 / msgr-worker-2 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 7f2beeaff640 / msgr-worker-1 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 7f2bef300640 / msgr-worker-0 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 7f2befb01640 / io_context_pool 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: 7f2bf102ba80 / radosgw-admin 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: max_recent 500 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: max_new 1000 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr: log_file /var/lib/ceph/crash/2026-02-24T22:16:33.902925Z_3cd20bac-5a91-46ff-8607-e26754aa38b5/log 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr:--- end dump of recent events --- 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr:Aborted 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket abort-at-init-indext 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:16:33.982 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:16:33.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.937+0000 7fee40ca9a80 4 Realm: () 2026-02-24T22:16:33.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.937+0000 7fee40ca9a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:16:33.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.937+0000 7fee40ca9a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:16:33.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.937+0000 7fee40ca9a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:16:33.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.937+0000 7fee40ca9a80 20 zonegroup default 2026-02-24T22:16:33.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.937+0000 7fee40ca9a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.937+0000 7fee40ca9a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:33.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.937+0000 7fee40ca9a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.937+0000 7fee40ca9a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:33.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.937+0000 7fee40ca9a80 20 started sync module instance, tier type = 2026-02-24T22:16:33.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.937+0000 7fee40ca9a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:16:33.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.941+0000 7fee40ca9a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:16:33.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.941+0000 7fee40ca9a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:16:33.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.941+0000 7fee40ca9a80 5 note: GC not initialized 2026-02-24T22:16:33.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.941+0000 7fed5ffff640 20 reqs_thread_entry: start 2026-02-24T22:16:33.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.945+0000 7fee3ef7e640 0 -- 10.20.193.46:0/3591767411 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x556a3c0d2c40 msgr2=0x556a3c0c5eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:33.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.949+0000 7fee3e77d640 0 -- 10.20.193.46:0/3591767411 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x556a3c0ce6c0 msgr2=0x556a3c0c9430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:33.983 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.973+0000 7fee40ca9a80 20 init_complete bucket index max shards: 1 2026-02-24T22:16:33.984 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.973+0000 7fee40ca9a80 20 Filter name: none 2026-02-24T22:16:33.984 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.973+0000 7fed5e7fc640 20 reqs_thread_entry: start 2026-02-24T22:16:33.984 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.973+0000 7fee40ca9a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.984 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.973+0000 7fee40ca9a80 20 rados_obj.operate() r=0 bl.length=216 2026-02-24T22:16:33.984 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.973+0000 7fee40ca9a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.984 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.973+0000 7fee40ca9a80 20 rados_obj.operate() r=0 bl.length=446 2026-02-24T22:16:33.984 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.973+0000 7fee40ca9a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:16:33.984 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.973+0000 7fee40ca9a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.984 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.973+0000 7fee40ca9a80 20 rados_obj.operate() r=0 bl.length=216 2026-02-24T22:16:33.984 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.973+0000 7fee40ca9a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:33.984 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.973+0000 7fee40ca9a80 20 rados_obj.operate() r=0 bl.length=446 2026-02-24T22:16:33.984 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:33.973+0000 7fee40ca9a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:16:33.984 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.984 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:16:33.984 INFO:tasks.workunit.client.0.trial046.stderr:bucket: abort-at-init-indext id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:16:33.984 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:16:33.985 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.985 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:33.990 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:16:33.990 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:16:34.035 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket abort-at-init-indext --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2026-02-24T22:16:34.051 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:16:34.051 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:16:34.068 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.069+0000 7f3847bf7a80 4 Realm: () 2026-02-24T22:16:34.068 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.069+0000 7f3847bf7a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:16:34.068 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.069+0000 7f3847bf7a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:16:34.068 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.069+0000 7f3847bf7a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:16:34.068 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.069+0000 7f3847bf7a80 20 zonegroup default 2026-02-24T22:16:34.068 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.069+0000 7f3847bf7a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:34.069 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.069+0000 7f3847bf7a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:34.069 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.069+0000 7f3847bf7a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:34.070 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.069+0000 7f3847bf7a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:16:34.070 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.069+0000 7f3847bf7a80 20 started sync module instance, tier type = 2026-02-24T22:16:34.070 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.069+0000 7f3847bf7a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:16:34.071 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.073+0000 7f3845ecc640 0 -- 10.20.193.46:0/2994713901 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x55c3b1c17060 msgr2=0x55c3b1c374e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:16:34.071 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.073+0000 7f3847bf7a80 20 add_watcher() i=1 2026-02-24T22:16:34.072 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.073+0000 7f3847bf7a80 20 add_watcher() i=3 2026-02-24T22:16:34.072 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.073+0000 7f3847bf7a80 20 add_watcher() i=6 2026-02-24T22:16:34.072 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.073+0000 7f3847bf7a80 20 add_watcher() i=0 2026-02-24T22:16:34.072 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.073+0000 7f3847bf7a80 20 add_watcher() i=5 2026-02-24T22:16:34.073 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.073+0000 7f3847bf7a80 20 add_watcher() i=4 2026-02-24T22:16:34.073 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.073+0000 7f3847bf7a80 20 add_watcher() i=2 2026-02-24T22:16:34.073 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.073+0000 7f3847bf7a80 20 add_watcher() i=7 2026-02-24T22:16:34.073 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.073+0000 7f3847bf7a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:16:34.074 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.077+0000 7f3847bf7a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:16:34.074 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.077+0000 7f3847bf7a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:16:34.075 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.077+0000 7f3847bf7a80 5 note: GC not initialized 2026-02-24T22:16:34.075 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.077+0000 7f37f27e4640 20 reqs_thread_entry: start 2026-02-24T22:16:34.075 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.077+0000 7f3845ecc640 0 -- 10.20.193.46:0/2551268226 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55c3b1b60d70 msgr2=0x55c3b1b62f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:16:34.317 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.317+0000 7f3847bf7a80 20 init_complete bucket index max shards: 1 2026-02-24T22:16:34.317 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.317+0000 7f3847bf7a80 20 Filter name: none 2026-02-24T22:16:34.318 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.317+0000 7f37f0fe1640 20 reqs_thread_entry: start 2026-02-24T22:16:34.318 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.317+0000 7f3847bf7a80 10 cache get: name=default.rgw.meta+root+abort-at-init-indext : miss 2026-02-24T22:16:34.318 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.317+0000 7f3847bf7a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:34.318 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.317+0000 7f3847bf7a80 20 rados_obj.operate() r=0 bl.length=216 2026-02-24T22:16:34.318 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.317+0000 7f3847bf7a80 10 cache put: name=default.rgw.meta+root+abort-at-init-indext info.flags=0x11 2026-02-24T22:16:34.318 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.317+0000 7f3847bf7a80 10 adding default.rgw.meta+root+abort-at-init-indext to cache LRU end 2026-02-24T22:16:34.318 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.317+0000 7f3847bf7a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 : miss 2026-02-24T22:16:34.318 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.317+0000 7f3847bf7a80 20 rados->read ofs=0 len=0 2026-02-24T22:16:34.318 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.317+0000 7f3847bf7a80 20 rados_obj.operate() r=0 bl.length=446 2026-02-24T22:16:34.318 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.317+0000 7f3847bf7a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 info.flags=0x17 2026-02-24T22:16:34.318 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.317+0000 7f3847bf7a80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 to cache LRU end 2026-02-24T22:16:34.318 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.317+0000 7f3847bf7a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:16:34.319 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.317+0000 7f3847bf7a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:16:34.319 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.317+0000 7f3847bf7a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 2026-02-24T22:16:34.319 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.317+0000 7f3847bf7a80 0 INFO: RGWReshardLock::lock found lock on abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 to be held by another RGW process; skipping for now 2026-02-24T22:16:34.321 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.321+0000 7f3847bf7a80 20 remove_watcher() i=3 2026-02-24T22:16:34.321 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.321+0000 7f3847bf7a80 2 removed watcher, disabling cache 2026-02-24T22:16:34.322 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.321+0000 7f3847bf7a80 20 remove_watcher() i=2 2026-02-24T22:16:34.322 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.321+0000 7f3847bf7a80 20 remove_watcher() i=1 2026-02-24T22:16:34.322 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.321+0000 7f3847bf7a80 20 remove_watcher() i=6 2026-02-24T22:16:34.322 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.321+0000 7f3847bf7a80 20 remove_watcher() i=4 2026-02-24T22:16:34.322 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.321+0000 7f3847bf7a80 20 remove_watcher() i=0 2026-02-24T22:16:34.322 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.321+0000 7f3847bf7a80 20 remove_watcher() i=5 2026-02-24T22:16:34.322 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:16:34.321+0000 7f3847bf7a80 20 remove_watcher() i=7 2026-02-24T22:16:34.327 INFO:tasks.workunit.client.0.trial046.stderr:waiting 30 seconds for reshard lock to expire... 2026-02-24T22:17:04.358 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket abort-at-init-indext --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2026-02-24T22:17:04.375 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:04.376 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:04.394 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.393+0000 7f2d9c8c7a80 4 Realm: () 2026-02-24T22:17:04.394 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.393+0000 7f2d9c8c7a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:04.394 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.393+0000 7f2d9c8c7a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:04.394 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.393+0000 7f2d9c8c7a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:04.394 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.393+0000 7f2d9c8c7a80 20 zonegroup default 2026-02-24T22:17:04.394 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.397+0000 7f2d9c8c7a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:04.395 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.397+0000 7f2d9c8c7a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:04.395 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.397+0000 7f2d9c8c7a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:04.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.397+0000 7f2d9c8c7a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:04.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.397+0000 7f2d9c8c7a80 20 started sync module instance, tier type = 2026-02-24T22:17:04.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.397+0000 7f2d9c8c7a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:04.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.397+0000 7f2d99b9a640 0 -- 10.20.193.46:0/4202748556 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x564f4cb34ba0 msgr2=0x564f4cb55020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:17:04.397 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.397+0000 7f2d9c8c7a80 20 add_watcher() i=1 2026-02-24T22:17:04.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.397+0000 7f2d9c8c7a80 20 add_watcher() i=5 2026-02-24T22:17:04.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.397+0000 7f2d9c8c7a80 20 add_watcher() i=3 2026-02-24T22:17:04.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.397+0000 7f2d9c8c7a80 20 add_watcher() i=0 2026-02-24T22:17:04.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.397+0000 7f2d9c8c7a80 20 add_watcher() i=6 2026-02-24T22:17:04.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.401+0000 7f2d9c8c7a80 20 add_watcher() i=2 2026-02-24T22:17:04.600 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.601+0000 7f2d99b9a640 0 -- 10.20.193.46:0/4202748556 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x564f4cb34ba0 msgr2=0x564f4cb55020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:04.601 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.601+0000 7f2d9c8c7a80 20 add_watcher() i=7 2026-02-24T22:17:04.602 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.605+0000 7f2d9c8c7a80 20 add_watcher() i=4 2026-02-24T22:17:04.602 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.605+0000 7f2d9c8c7a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:17:04.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.605+0000 7f2d9c8c7a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:04.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.605+0000 7f2d9c8c7a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:04.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.605+0000 7f2d9c8c7a80 5 note: GC not initialized 2026-02-24T22:17:04.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.605+0000 7f2cb37fe640 20 reqs_thread_entry: start 2026-02-24T22:17:04.642 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.641+0000 7f2d9c8c7a80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:04.642 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.641+0000 7f2d9c8c7a80 20 Filter name: none 2026-02-24T22:17:04.642 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.641+0000 7f2cb1ffb640 20 reqs_thread_entry: start 2026-02-24T22:17:04.642 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.641+0000 7f2d9c8c7a80 10 cache get: name=default.rgw.meta+root+abort-at-init-indext : miss 2026-02-24T22:17:04.642 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.641+0000 7f2d9c8c7a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:04.642 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.641+0000 7f2d9c8c7a80 20 rados_obj.operate() r=0 bl.length=216 2026-02-24T22:17:04.642 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.641+0000 7f2d9c8c7a80 10 cache put: name=default.rgw.meta+root+abort-at-init-indext info.flags=0x11 2026-02-24T22:17:04.642 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.641+0000 7f2d9c8c7a80 10 adding default.rgw.meta+root+abort-at-init-indext to cache LRU end 2026-02-24T22:17:04.642 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.641+0000 7f2d9c8c7a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 : miss 2026-02-24T22:17:04.642 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.641+0000 7f2d9c8c7a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:04.643 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.641+0000 7f2d9c8c7a80 20 rados_obj.operate() r=0 bl.length=446 2026-02-24T22:17:04.643 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.641+0000 7f2d9c8c7a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 info.flags=0x17 2026-02-24T22:17:04.643 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.641+0000 7f2d9c8c7a80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 to cache LRU end 2026-02-24T22:17:04.643 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.641+0000 7f2d9c8c7a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:17:04.643 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.641+0000 7f2d9c8c7a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:04.643 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.641+0000 7f2d9c8c7a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 2026-02-24T22:17:04.646 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.649+0000 7f2d9c8c7a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 info.flags=0x17 2026-02-24T22:17:04.647 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.649+0000 7f2d9c8c7a80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 to cache LRU end 2026-02-24T22:17:04.647 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.649+0000 7f2d9c8c7a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:04.647 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.649+0000 7f2d9c8c7a80 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522, ofs0, ns] 2026-02-24T22:17:04.647 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.649+0000 7f2d757fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961327 cookie 94898589491360 notifier 6696 bl.length()=1015 2026-02-24T22:17:04.647 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.649+0000 7f2d757fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 info.flags=0x17 2026-02-24T22:17:04.647 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.649+0000 7f2d757fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 to cache LRU end 2026-02-24T22:17:04.647 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.649+0000 7f2d757fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:04.647 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.649+0000 7f2d9c8c7a80 20 handle_bi_update(): bucket=:abort-at-init-indext[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522]): orig_sources= new_sources= 2026-02-24T22:17:04.647 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.649+0000 7f2d9c8c7a80 20 handle_bi_update(): bucket=:abort-at-init-indext[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522]): potential sources added=[] removed=[] 2026-02-24T22:17:04.647 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.649+0000 7f2d9c8c7a80 20 handle_bi_update(): bucket=:abort-at-init-indext[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522]): orig_dests= new_dests= 2026-02-24T22:17:04.648 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.649+0000 7f2d9c8c7a80 20 handle_bi_update(): bucket=:abort-at-init-indext[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522]): potential dests added=[] removed=[] 2026-02-24T22:17:04.648 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.649+0000 7f2d9c8c7a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522.1.0 2026-02-24T22:17:04.648 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.649+0000 7f2d9c8c7a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522.1.0 2026-02-24T22:17:04.648 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.649+0000 7f2d9c8c7a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522.1.1 2026-02-24T22:17:04.649 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.649+0000 7f2d9c8c7a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522.1.1 2026-02-24T22:17:04.649 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.649+0000 7f2d9c8c7a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522.0 2026-02-24T22:17:04.650 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d9c8c7a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 : hit (requested=0x17, cached=0x17) 2026-02-24T22:17:04.650 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d9c8c7a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 2026-02-24T22:17:04.651 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d9c8c7a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 info.flags=0x17 2026-02-24T22:17:04.651 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d9c8c7a80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 to cache LRU end 2026-02-24T22:17:04.652 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d9c8c7a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:04.652 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d9c8c7a80 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522, ofs0, ns] 2026-02-24T22:17:04.652 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d757fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961328 cookie 94898589491360 notifier 6696 bl.length()=1015 2026-02-24T22:17:04.652 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d757fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 info.flags=0x17 2026-02-24T22:17:04.652 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d757fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 to cache LRU end 2026-02-24T22:17:04.652 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d757fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:04.652 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d9c8c7a80 20 handle_bi_update(): bucket=:abort-at-init-indext[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522]): orig_sources= new_sources= 2026-02-24T22:17:04.652 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d9c8c7a80 20 handle_bi_update(): bucket=:abort-at-init-indext[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522]): potential sources added=[] removed=[] 2026-02-24T22:17:04.652 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d9c8c7a80 20 handle_bi_update(): bucket=:abort-at-init-indext[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522]): orig_dests= new_dests= 2026-02-24T22:17:04.652 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d9c8c7a80 20 handle_bi_update(): bucket=:abort-at-init-indext[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522]): potential dests added=[] removed=[] 2026-02-24T22:17:04.654 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d9c8c7a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522.0 2026-02-24T22:17:04.654 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d9c8c7a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 : hit (requested=0x17, cached=0x17) 2026-02-24T22:17:04.654 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d9c8c7a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 2026-02-24T22:17:04.654 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d9c8c7a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 info.flags=0x17 2026-02-24T22:17:04.654 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d9c8c7a80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 to cache LRU end 2026-02-24T22:17:04.654 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d9c8c7a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:04.654 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d9c8c7a80 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522, ofs0, ns] 2026-02-24T22:17:04.654 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d75ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961329 cookie 94898589491360 notifier 6696 bl.length()=979 2026-02-24T22:17:04.654 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d75ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 info.flags=0x17 2026-02-24T22:17:04.655 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d75ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-init-indext:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 to cache LRU end 2026-02-24T22:17:04.655 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d75ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:04.655 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d9c8c7a80 20 handle_bi_update(): bucket=:abort-at-init-indext[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522]): orig_sources= new_sources= 2026-02-24T22:17:04.655 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d9c8c7a80 20 handle_bi_update(): bucket=:abort-at-init-indext[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522]): potential sources added=[] removed=[] 2026-02-24T22:17:04.655 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d9c8c7a80 20 handle_bi_update(): bucket=:abort-at-init-indext[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522]): orig_dests= new_dests= 2026-02-24T22:17:04.655 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.653+0000 7f2d9c8c7a80 20 handle_bi_update(): bucket=:abort-at-init-indext[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522]): potential dests added=[] removed=[] 2026-02-24T22:17:04.655 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.657+0000 7f2d9c8c7a80 1 execute INFO: reshard of bucket "abort-at-init-indext" from 1 shards to 2 shards completed successfully 2026-02-24T22:17:04.658 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.657+0000 7f2d9c8c7a80 20 remove_watcher() i=3 2026-02-24T22:17:04.658 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.657+0000 7f2d9c8c7a80 2 removed watcher, disabling cache 2026-02-24T22:17:04.658 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.657+0000 7f2d9c8c7a80 20 remove_watcher() i=0 2026-02-24T22:17:04.658 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.657+0000 7f2d9c8c7a80 20 remove_watcher() i=2 2026-02-24T22:17:04.658 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.657+0000 7f2d9c8c7a80 20 remove_watcher() i=6 2026-02-24T22:17:04.658 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.657+0000 7f2d9c8c7a80 20 remove_watcher() i=4 2026-02-24T22:17:04.658 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.657+0000 7f2d9c8c7a80 20 remove_watcher() i=1 2026-02-24T22:17:04.658 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.657+0000 7f2d9c8c7a80 20 remove_watcher() i=5 2026-02-24T22:17:04.658 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.657+0000 7f2d9c8c7a80 20 remove_watcher() i=7 2026-02-24T22:17:04.664 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket abort-at-init-indext 2026-02-24T22:17:04.679 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:04.679 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:04.696 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.697+0000 7f7bcc897a80 4 Realm: () 2026-02-24T22:17:04.696 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.697+0000 7f7bcc897a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:04.696 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.697+0000 7f7bcc897a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:04.696 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.697+0000 7f7bcc897a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:04.696 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.697+0000 7f7bcc897a80 20 zonegroup default 2026-02-24T22:17:04.697 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.697+0000 7f7bcc897a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:04.698 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.701+0000 7f7bcc897a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:04.698 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.701+0000 7f7bcc897a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:04.699 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.701+0000 7f7bcc897a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:04.699 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.701+0000 7f7bcc897a80 20 started sync module instance, tier type = 2026-02-24T22:17:04.699 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.701+0000 7f7bcc897a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:04.701 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.701+0000 7f7bcc897a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:04.701 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.701+0000 7f7bcc897a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:04.701 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.701+0000 7f7bcc897a80 5 note: GC not initialized 2026-02-24T22:17:04.701 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.701+0000 7f7aebfff640 20 reqs_thread_entry: start 2026-02-24T22:17:04.705 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.705+0000 7f7bca36b640 0 -- 10.20.193.46:0/1676084087 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x559b4a2c58d0 msgr2=0x559b4a2c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:04.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:04.917+0000 7f7bcab6c640 0 -- 10.20.193.46:0/1676084087 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x559b4a2b6c90 msgr2=0x559b4a2b70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:17:05.144 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.145+0000 7f7bcc897a80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:05.144 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.145+0000 7f7bcc897a80 20 Filter name: none 2026-02-24T22:17:05.144 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.145+0000 7f7aea7fc640 20 reqs_thread_entry: start 2026-02-24T22:17:05.144 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.145+0000 7f7bcc897a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:05.145 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.145+0000 7f7bcc897a80 20 rados_obj.operate() r=0 bl.length=216 2026-02-24T22:17:05.145 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.145+0000 7f7bcc897a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:05.146 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.145+0000 7f7bcc897a80 20 rados_obj.operate() r=0 bl.length=446 2026-02-24T22:17:05.146 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.145+0000 7f7bcc897a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:05.146 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.145+0000 7f7bcc897a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:05.147 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.149+0000 7f7bcc897a80 20 rados_obj.operate() r=0 bl.length=216 2026-02-24T22:17:05.147 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.149+0000 7f7bcc897a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:05.147 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.149+0000 7f7bcc897a80 20 rados_obj.operate() r=0 bl.length=446 2026-02-24T22:17:05.147 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.149+0000 7f7bcc897a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:05.158 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:05.158 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:17:05.158 INFO:tasks.workunit.client.0.trial046.stderr:bucket: abort-at-init-indext id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522 num_objects: 19 size_kb: 1 num_shards: 2 2026-02-24T22:17:05.158 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:05.159 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:05.159 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:05.162 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:05.162 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:05.207 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:05.207 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:05.219 INFO:tasks.workunit.client.0.trial046.stderr:TEST: reshard bucket with EIO injected at set_target_layout 2026-02-24T22:17:05.219 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:05.220 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:05.220 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:05.228 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:05.228 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:05.271 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:05.271 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:05.279 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:05.279 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:05.284 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:05.284 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:05.290 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:05.291 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:05.297 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:05.297 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:05.302 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:05.302 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:05.308 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:05.309 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:05.314 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:05.315 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:05.320 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:05.320 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:05.326 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:05.326 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:05.332 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:05.332 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:05.338 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:05.338 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:05.344 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:05.345 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:05.351 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:05.351 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:05.357 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:05.357 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:05.364 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:05.364 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:05.370 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:05.370 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:05.375 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:05.376 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:05.381 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:05.381 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:05.387 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:05.387 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:05.392 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2026-02-24T22:17:05.407 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:05.407 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:05.424 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.425+0000 7f86554dda80 4 Realm: () 2026-02-24T22:17:05.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.425+0000 7f86554dda80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:05.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.425+0000 7f86554dda80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:05.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.425+0000 7f86554dda80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:05.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.425+0000 7f86554dda80 20 zonegroup default 2026-02-24T22:17:05.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.425+0000 7f86554dda80 20 rados->read ofs=0 len=0 2026-02-24T22:17:05.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.429+0000 7f86554dda80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:05.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.429+0000 7f86554dda80 20 rados->read ofs=0 len=0 2026-02-24T22:17:05.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.429+0000 7f86554dda80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:05.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.429+0000 7f86554dda80 20 started sync module instance, tier type = 2026-02-24T22:17:05.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.429+0000 7f86554dda80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:05.428 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.429+0000 7f86554dda80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:05.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.429+0000 7f86554dda80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:05.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.429+0000 7f86554dda80 5 note: GC not initialized 2026-02-24T22:17:05.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.429+0000 7f86017e2640 20 reqs_thread_entry: start 2026-02-24T22:17:05.431 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.433+0000 7f86537b2640 0 -- 10.20.193.46:0/2384773032 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55dc68fe4e40 msgr2=0x55dc68fd80b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:05.437 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.437+0000 7f8652fb1640 0 -- 10.20.193.46:0/2384773032 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55dc68fe08c0 msgr2=0x55dc68fdb630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:17:05.460 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.461+0000 7f86554dda80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:05.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.461+0000 7f86554dda80 20 Filter name: none 2026-02-24T22:17:05.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.461+0000 7f856b7fe640 20 reqs_thread_entry: start 2026-02-24T22:17:05.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.461+0000 7f86554dda80 20 rados->read ofs=0 len=0 2026-02-24T22:17:05.462 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.461+0000 7f86554dda80 20 rados_obj.operate() r=0 bl.length=222 2026-02-24T22:17:05.462 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.461+0000 7f86554dda80 20 rados->read ofs=0 len=0 2026-02-24T22:17:05.462 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.461+0000 7f86554dda80 20 rados_obj.operate() r=0 bl.length=452 2026-02-24T22:17:05.462 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.461+0000 7f86554dda80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:05.462 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.461+0000 7f86554dda80 20 rados->read ofs=0 len=0 2026-02-24T22:17:05.462 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.461+0000 7f86554dda80 20 rados_obj.operate() r=0 bl.length=222 2026-02-24T22:17:05.462 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.461+0000 7f86554dda80 20 rados->read ofs=0 len=0 2026-02-24T22:17:05.462 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.465+0000 7f86554dda80 20 rados_obj.operate() r=0 bl.length=452 2026-02-24T22:17:05.463 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.465+0000 7f86554dda80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:05.473 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:05.473 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:17:05.473 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-set-target-layout id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:17:05.473 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:05.474 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket error-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at set_target_layout 2026-02-24T22:17:05.489 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:05.489 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:05.508 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.509+0000 7fd72d1cca80 4 Realm: () 2026-02-24T22:17:05.508 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.509+0000 7fd72d1cca80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:05.508 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.509+0000 7fd72d1cca80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:05.508 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.509+0000 7fd72d1cca80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:05.508 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.509+0000 7fd72d1cca80 20 zonegroup default 2026-02-24T22:17:05.509 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.509+0000 7fd72d1cca80 20 rados->read ofs=0 len=0 2026-02-24T22:17:05.510 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.509+0000 7fd72d1cca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:05.510 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.509+0000 7fd72d1cca80 20 rados->read ofs=0 len=0 2026-02-24T22:17:05.510 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.513+0000 7fd72d1cca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:05.510 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.513+0000 7fd72d1cca80 20 started sync module instance, tier type = 2026-02-24T22:17:05.510 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.513+0000 7fd72d1cca80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:05.511 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.513+0000 7fd72aca0640 0 -- 10.20.193.46:0/440973319 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5640a85d5240 msgr2=0x5640a85f56c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:05.512 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.513+0000 7fd72d1cca80 20 add_watcher() i=1 2026-02-24T22:17:05.512 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.513+0000 7fd72d1cca80 20 add_watcher() i=0 2026-02-24T22:17:05.512 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.513+0000 7fd72d1cca80 20 add_watcher() i=5 2026-02-24T22:17:05.513 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.513+0000 7fd72d1cca80 20 add_watcher() i=3 2026-02-24T22:17:05.513 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.513+0000 7fd72d1cca80 20 add_watcher() i=2 2026-02-24T22:17:05.513 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.513+0000 7fd72d1cca80 20 add_watcher() i=7 2026-02-24T22:17:05.514 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.513+0000 7fd72d1cca80 20 add_watcher() i=6 2026-02-24T22:17:05.514 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.513+0000 7fd72d1cca80 20 add_watcher() i=4 2026-02-24T22:17:05.514 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.513+0000 7fd72d1cca80 2 all 8 watchers are set, enabling cache 2026-02-24T22:17:05.514 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.513+0000 7fd72a49f640 0 -- 10.20.193.46:0/1257367459 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7fd6fc09d3f0 msgr2=0x7fd6fc0bd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:05.716 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.717+0000 7fd72d1cca80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:05.716 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.717+0000 7fd72d1cca80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:05.716 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.717+0000 7fd72d1cca80 5 note: GC not initialized 2026-02-24T22:17:05.716 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.717+0000 7fd6d8fe1640 20 reqs_thread_entry: start 2026-02-24T22:17:05.752 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.753+0000 7fd72d1cca80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:05.752 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.753+0000 7fd72d1cca80 20 Filter name: none 2026-02-24T22:17:05.752 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.753+0000 7fd642ffd640 20 reqs_thread_entry: start 2026-02-24T22:17:05.753 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.753+0000 7fd72d1cca80 10 cache get: name=default.rgw.meta+root+error-at-set-target-layout : miss 2026-02-24T22:17:05.753 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.753+0000 7fd72d1cca80 20 rados->read ofs=0 len=0 2026-02-24T22:17:05.753 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.753+0000 7fd72d1cca80 20 rados_obj.operate() r=0 bl.length=222 2026-02-24T22:17:05.753 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.753+0000 7fd72d1cca80 10 cache put: name=default.rgw.meta+root+error-at-set-target-layout info.flags=0x11 2026-02-24T22:17:05.753 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.753+0000 7fd72d1cca80 10 adding default.rgw.meta+root+error-at-set-target-layout to cache LRU end 2026-02-24T22:17:05.753 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.753+0000 7fd72d1cca80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 : miss 2026-02-24T22:17:05.753 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.753+0000 7fd72d1cca80 20 rados->read ofs=0 len=0 2026-02-24T22:17:05.753 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.753+0000 7fd72d1cca80 20 rados_obj.operate() r=0 bl.length=452 2026-02-24T22:17:05.753 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.753+0000 7fd72d1cca80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 info.flags=0x17 2026-02-24T22:17:05.753 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.753+0000 7fd72d1cca80 10 adding default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 to cache LRU end 2026-02-24T22:17:05.753 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.753+0000 7fd72d1cca80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:17:05.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.753+0000 7fd72d1cca80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:05.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.753+0000 7fd72d1cca80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 2026-02-24T22:17:05.756 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.757+0000 7fd72d1cca80 -1 Injecting error=-5 at location=set_target_layout 2026-02-24T22:17:05.756 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.757+0000 7fd72d1cca80 0 ERROR: init_target_layout failed to write target index layout to bucket info: (5) Input/output error 2026-02-24T22:17:05.761 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.761+0000 7fd72d1cca80 20 remove_watcher() i=3 2026-02-24T22:17:05.761 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.761+0000 7fd72d1cca80 2 removed watcher, disabling cache 2026-02-24T22:17:05.761 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.761+0000 7fd72d1cca80 20 remove_watcher() i=2 2026-02-24T22:17:05.761 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.761+0000 7fd72d1cca80 20 remove_watcher() i=0 2026-02-24T22:17:05.761 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.761+0000 7fd72d1cca80 20 remove_watcher() i=1 2026-02-24T22:17:05.761 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.761+0000 7fd72d1cca80 20 remove_watcher() i=4 2026-02-24T22:17:05.761 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.761+0000 7fd72d1cca80 20 remove_watcher() i=7 2026-02-24T22:17:05.761 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.761+0000 7fd72d1cca80 20 remove_watcher() i=6 2026-02-24T22:17:05.762 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:05.761+0000 7fd72d1cca80 20 remove_watcher() i=5 2026-02-24T22:17:05.766 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2026-02-24T22:17:05.782 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:05.782 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:05.999 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.001+0000 7f2fa6270640 0 --2- 10.20.193.46:0/1429167305 >> [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] conn(0x7f2f78001fb0 0x7f2f780223d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=8 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2026-02-24T22:17:06.005 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.005+0000 7f2fa879ca80 4 Realm: () 2026-02-24T22:17:06.006 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.005+0000 7f2fa879ca80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:06.006 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.005+0000 7f2fa879ca80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:06.006 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.005+0000 7f2fa879ca80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:06.006 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.005+0000 7f2fa879ca80 20 zonegroup default 2026-02-24T22:17:06.006 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.009+0000 7f2fa879ca80 20 rados->read ofs=0 len=0 2026-02-24T22:17:06.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.009+0000 7f2fa879ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:06.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.009+0000 7f2fa879ca80 20 rados->read ofs=0 len=0 2026-02-24T22:17:06.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.009+0000 7f2fa879ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:06.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.009+0000 7f2fa879ca80 20 started sync module instance, tier type = 2026-02-24T22:17:06.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.009+0000 7f2fa879ca80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:06.011 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.013+0000 7f2fa879ca80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:06.011 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.013+0000 7f2fa879ca80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:06.011 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.013+0000 7f2fa879ca80 5 note: GC not initialized 2026-02-24T22:17:06.011 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.013+0000 7f2ec7fff640 20 reqs_thread_entry: start 2026-02-24T22:17:06.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.013+0000 7f2fa6270640 0 -- 10.20.193.46:0/1429167305 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55d8955f7440 msgr2=0x55d8955f21b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:06.016 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.017+0000 7f2fa6270640 0 -- 10.20.193.46:0/1429167305 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55d8955efc40 msgr2=0x55d8955f00a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:06.046 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.045+0000 7f2fa879ca80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:06.046 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.045+0000 7f2fa879ca80 20 Filter name: none 2026-02-24T22:17:06.046 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.045+0000 7f2ec67fc640 20 reqs_thread_entry: start 2026-02-24T22:17:06.046 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.045+0000 7f2fa879ca80 20 rados->read ofs=0 len=0 2026-02-24T22:17:06.046 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.045+0000 7f2fa879ca80 20 rados_obj.operate() r=0 bl.length=222 2026-02-24T22:17:06.046 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.045+0000 7f2fa879ca80 20 rados->read ofs=0 len=0 2026-02-24T22:17:06.047 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.049+0000 7f2fa879ca80 20 rados_obj.operate() r=0 bl.length=452 2026-02-24T22:17:06.047 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.049+0000 7f2fa879ca80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:06.047 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.049+0000 7f2fa879ca80 20 rados->read ofs=0 len=0 2026-02-24T22:17:06.047 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.049+0000 7f2fa879ca80 20 rados_obj.operate() r=0 bl.length=222 2026-02-24T22:17:06.047 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.049+0000 7f2fa879ca80 20 rados->read ofs=0 len=0 2026-02-24T22:17:06.047 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.049+0000 7f2fa879ca80 20 rados_obj.operate() r=0 bl.length=452 2026-02-24T22:17:06.047 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.049+0000 7f2fa879ca80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:06.057 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:06.058 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:17:06.058 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-set-target-layout id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:17:06.058 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:06.059 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:06.059 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:06.064 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:06.065 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:06.106 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket error-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2026-02-24T22:17:06.123 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:06.123 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:06.140 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.141+0000 7fbf9ad2ba80 4 Realm: () 2026-02-24T22:17:06.140 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.141+0000 7fbf9ad2ba80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:06.140 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.141+0000 7fbf9ad2ba80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:06.140 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.141+0000 7fbf9ad2ba80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:06.140 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.141+0000 7fbf9ad2ba80 20 zonegroup default 2026-02-24T22:17:06.141 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.141+0000 7fbf9ad2ba80 20 rados->read ofs=0 len=0 2026-02-24T22:17:06.142 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.141+0000 7fbf9ad2ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:06.142 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.141+0000 7fbf9ad2ba80 20 rados->read ofs=0 len=0 2026-02-24T22:17:06.142 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.145+0000 7fbf9ad2ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:06.142 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.145+0000 7fbf9ad2ba80 20 started sync module instance, tier type = 2026-02-24T22:17:06.142 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.145+0000 7fbf9ad2ba80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:06.143 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.145+0000 7fbf99000640 0 -- 10.20.193.46:0/3808771572 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x5599819d1de0 msgr2=0x5599819f21c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:06.144 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.145+0000 7fbf9ad2ba80 20 add_watcher() i=1 2026-02-24T22:17:06.144 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.145+0000 7fbf9ad2ba80 20 add_watcher() i=6 2026-02-24T22:17:06.145 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.145+0000 7fbf9ad2ba80 20 add_watcher() i=5 2026-02-24T22:17:06.145 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.145+0000 7fbf9ad2ba80 20 add_watcher() i=3 2026-02-24T22:17:06.145 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.145+0000 7fbf9ad2ba80 20 add_watcher() i=0 2026-02-24T22:17:06.145 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.145+0000 7fbf9ad2ba80 20 add_watcher() i=4 2026-02-24T22:17:06.145 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.145+0000 7fbf9ad2ba80 20 add_watcher() i=7 2026-02-24T22:17:06.146 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.145+0000 7fbf9ad2ba80 20 add_watcher() i=2 2026-02-24T22:17:06.146 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.145+0000 7fbf9ad2ba80 2 all 8 watchers are set, enabling cache 2026-02-24T22:17:06.146 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.149+0000 7fbf9ad2ba80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:06.147 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.149+0000 7fbf9ad2ba80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:06.147 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.149+0000 7fbf9ad2ba80 5 note: GC not initialized 2026-02-24T22:17:06.147 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.149+0000 7fbf45fdb640 20 reqs_thread_entry: start 2026-02-24T22:17:06.147 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.149+0000 7fbf99000640 0 -- 10.20.193.46:0/881061079 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x559981afa9a0 msgr2=0x559981afad70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:17:06.387 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.389+0000 7fbf9ad2ba80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:06.387 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.389+0000 7fbf9ad2ba80 20 Filter name: none 2026-02-24T22:17:06.387 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.389+0000 7fbe9ffff640 20 reqs_thread_entry: start 2026-02-24T22:17:06.387 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.389+0000 7fbf9ad2ba80 10 cache get: name=default.rgw.meta+root+error-at-set-target-layout : miss 2026-02-24T22:17:06.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.389+0000 7fbf9ad2ba80 20 rados->read ofs=0 len=0 2026-02-24T22:17:06.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.389+0000 7fbf9ad2ba80 20 rados_obj.operate() r=0 bl.length=222 2026-02-24T22:17:06.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.389+0000 7fbf9ad2ba80 10 cache put: name=default.rgw.meta+root+error-at-set-target-layout info.flags=0x11 2026-02-24T22:17:06.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.389+0000 7fbf9ad2ba80 10 adding default.rgw.meta+root+error-at-set-target-layout to cache LRU end 2026-02-24T22:17:06.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.389+0000 7fbf9ad2ba80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 : miss 2026-02-24T22:17:06.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.389+0000 7fbf9ad2ba80 20 rados->read ofs=0 len=0 2026-02-24T22:17:06.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.389+0000 7fbf9ad2ba80 20 rados_obj.operate() r=0 bl.length=452 2026-02-24T22:17:06.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.389+0000 7fbf9ad2ba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 info.flags=0x17 2026-02-24T22:17:06.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.389+0000 7fbf9ad2ba80 10 adding default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 to cache LRU end 2026-02-24T22:17:06.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.389+0000 7fbf9ad2ba80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:17:06.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.389+0000 7fbf9ad2ba80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:06.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.389+0000 7fbf9ad2ba80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 2026-02-24T22:17:06.392 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.393+0000 7fbf9ad2ba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 info.flags=0x17 2026-02-24T22:17:06.392 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.393+0000 7fbf9ad2ba80 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 to cache LRU end 2026-02-24T22:17:06.392 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.393+0000 7fbf9ad2ba80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:06.392 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.393+0000 7fbf9ad2ba80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523, ofs0, ns] 2026-02-24T22:17:06.392 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.393+0000 7fbf66ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959903 cookie 94117794262144 notifier 6795 bl.length()=1027 2026-02-24T22:17:06.393 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.393+0000 7fbf66ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 info.flags=0x17 2026-02-24T22:17:06.393 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.393+0000 7fbf66ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 to cache LRU end 2026-02-24T22:17:06.393 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.393+0000 7fbf66ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:06.393 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.393+0000 7fbf9ad2ba80 20 handle_bi_update(): bucket=:error-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523]): orig_sources= new_sources= 2026-02-24T22:17:06.393 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.393+0000 7fbf9ad2ba80 20 handle_bi_update(): bucket=:error-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523]): potential sources added=[] removed=[] 2026-02-24T22:17:06.393 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.393+0000 7fbf9ad2ba80 20 handle_bi_update(): bucket=:error-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523]): orig_dests= new_dests= 2026-02-24T22:17:06.393 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.393+0000 7fbf9ad2ba80 20 handle_bi_update(): bucket=:error-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523]): potential dests added=[] removed=[] 2026-02-24T22:17:06.597 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.597+0000 7fbf9ad2ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523.1.0 2026-02-24T22:17:06.597 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.597+0000 7fbf9ad2ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523.1.0 2026-02-24T22:17:06.597 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.597+0000 7fbf9ad2ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523.1.1 2026-02-24T22:17:06.598 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.597+0000 7fbf9ad2ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523.1.1 2026-02-24T22:17:06.598 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.597+0000 7fbf9ad2ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523.0 2026-02-24T22:17:06.599 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.601+0000 7fbf9ad2ba80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 : hit (requested=0x17, cached=0x17) 2026-02-24T22:17:06.599 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.601+0000 7fbf9ad2ba80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 2026-02-24T22:17:06.600 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.601+0000 7fbf9ad2ba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 info.flags=0x17 2026-02-24T22:17:06.600 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.601+0000 7fbf9ad2ba80 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 to cache LRU end 2026-02-24T22:17:06.600 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.601+0000 7fbf9ad2ba80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:06.600 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.601+0000 7fbf9ad2ba80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523, ofs0, ns] 2026-02-24T22:17:06.601 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.601+0000 7fbf677fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959904 cookie 94117794262144 notifier 6795 bl.length()=1027 2026-02-24T22:17:06.601 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.601+0000 7fbf677fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 info.flags=0x17 2026-02-24T22:17:06.601 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.601+0000 7fbf677fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 to cache LRU end 2026-02-24T22:17:06.601 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.601+0000 7fbf677fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:06.601 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.601+0000 7fbf9ad2ba80 20 handle_bi_update(): bucket=:error-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523]): orig_sources= new_sources= 2026-02-24T22:17:06.601 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.601+0000 7fbf9ad2ba80 20 handle_bi_update(): bucket=:error-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523]): potential sources added=[] removed=[] 2026-02-24T22:17:06.601 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.601+0000 7fbf9ad2ba80 20 handle_bi_update(): bucket=:error-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523]): orig_dests= new_dests= 2026-02-24T22:17:06.601 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.601+0000 7fbf9ad2ba80 20 handle_bi_update(): bucket=:error-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523]): potential dests added=[] removed=[] 2026-02-24T22:17:06.602 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.601+0000 7fbf9ad2ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523.0 2026-02-24T22:17:06.602 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.601+0000 7fbf9ad2ba80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 : hit (requested=0x17, cached=0x17) 2026-02-24T22:17:06.602 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.601+0000 7fbf9ad2ba80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 2026-02-24T22:17:06.603 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.605+0000 7fbf9ad2ba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 info.flags=0x17 2026-02-24T22:17:06.603 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.605+0000 7fbf9ad2ba80 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 to cache LRU end 2026-02-24T22:17:06.603 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.605+0000 7fbf9ad2ba80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:06.603 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.605+0000 7fbf9ad2ba80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523, ofs0, ns] 2026-02-24T22:17:06.603 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.605+0000 7fbf677fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959905 cookie 94117794262144 notifier 6795 bl.length()=991 2026-02-24T22:17:06.603 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.605+0000 7fbf677fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 info.flags=0x17 2026-02-24T22:17:06.604 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.605+0000 7fbf677fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 to cache LRU end 2026-02-24T22:17:06.604 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.605+0000 7fbf677fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:06.604 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.605+0000 7fbf9ad2ba80 20 handle_bi_update(): bucket=:error-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523]): orig_sources= new_sources= 2026-02-24T22:17:06.604 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.605+0000 7fbf9ad2ba80 20 handle_bi_update(): bucket=:error-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523]): potential sources added=[] removed=[] 2026-02-24T22:17:06.604 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.605+0000 7fbf9ad2ba80 20 handle_bi_update(): bucket=:error-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523]): orig_dests= new_dests= 2026-02-24T22:17:06.604 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.605+0000 7fbf9ad2ba80 20 handle_bi_update(): bucket=:error-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523]): potential dests added=[] removed=[] 2026-02-24T22:17:06.604 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.605+0000 7fbf9ad2ba80 1 execute INFO: reshard of bucket "error-at-set-target-layout" from 1 shards to 2 shards completed successfully 2026-02-24T22:17:06.608 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.609+0000 7fbf9ad2ba80 20 remove_watcher() i=3 2026-02-24T22:17:06.608 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.609+0000 7fbf9ad2ba80 2 removed watcher, disabling cache 2026-02-24T22:17:06.608 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.609+0000 7fbf9ad2ba80 20 remove_watcher() i=2 2026-02-24T22:17:06.609 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.609+0000 7fbf9ad2ba80 20 remove_watcher() i=0 2026-02-24T22:17:06.609 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.609+0000 7fbf9ad2ba80 20 remove_watcher() i=6 2026-02-24T22:17:06.609 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.609+0000 7fbf9ad2ba80 20 remove_watcher() i=1 2026-02-24T22:17:06.609 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.609+0000 7fbf9ad2ba80 20 remove_watcher() i=4 2026-02-24T22:17:06.609 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.609+0000 7fbf9ad2ba80 20 remove_watcher() i=5 2026-02-24T22:17:06.609 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.609+0000 7fbf9ad2ba80 20 remove_watcher() i=7 2026-02-24T22:17:06.616 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2026-02-24T22:17:06.631 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:06.631 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:06.647 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.649+0000 7fcca8f81a80 4 Realm: () 2026-02-24T22:17:06.648 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.649+0000 7fcca8f81a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:06.648 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.649+0000 7fcca8f81a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:06.648 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.649+0000 7fcca8f81a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:06.648 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.649+0000 7fcca8f81a80 20 zonegroup default 2026-02-24T22:17:06.648 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.649+0000 7fcca8f81a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:06.650 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.649+0000 7fcca8f81a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:06.650 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.653+0000 7fcca8f81a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:06.650 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.653+0000 7fcca8f81a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:06.651 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.653+0000 7fcca8f81a80 20 started sync module instance, tier type = 2026-02-24T22:17:06.651 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.653+0000 7fcca8f81a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:06.652 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.653+0000 7fcca8f81a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:06.652 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.653+0000 7fcca8f81a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:06.652 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.653+0000 7fcca8f81a80 5 note: GC not initialized 2026-02-24T22:17:06.652 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.653+0000 7fcc557e2640 20 reqs_thread_entry: start 2026-02-24T22:17:06.657 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.657+0000 7fcca6a55640 0 -- 10.20.193.46:0/370031568 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55f0c9dd3af0 msgr2=0x55f0c9dd0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:17:06.867 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:06.869+0000 7fcca7256640 0 -- 10.20.193.46:0/370031568 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x55f0c9dbfb70 msgr2=0x55f0c9dc1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:07.095 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.097+0000 7fcca8f81a80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:07.096 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.097+0000 7fcca8f81a80 20 Filter name: none 2026-02-24T22:17:07.096 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.097+0000 7fcbbf7fe640 20 reqs_thread_entry: start 2026-02-24T22:17:07.096 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.097+0000 7fcca8f81a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:07.096 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.097+0000 7fcca8f81a80 20 rados_obj.operate() r=0 bl.length=222 2026-02-24T22:17:07.096 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.097+0000 7fcca8f81a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:07.097 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.097+0000 7fcca8f81a80 20 rados_obj.operate() r=0 bl.length=452 2026-02-24T22:17:07.097 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.097+0000 7fcca8f81a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:07.097 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.097+0000 7fcca8f81a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:07.097 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.097+0000 7fcca8f81a80 20 rados_obj.operate() r=0 bl.length=222 2026-02-24T22:17:07.097 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.097+0000 7fcca8f81a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:07.097 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.097+0000 7fcca8f81a80 20 rados_obj.operate() r=0 bl.length=452 2026-02-24T22:17:07.098 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.097+0000 7fcca8f81a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:07.108 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:07.108 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:17:07.108 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-set-target-layout id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523 num_objects: 19 size_kb: 1 num_shards: 2 2026-02-24T22:17:07.108 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:07.109 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:07.109 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:07.112 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:07.112 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:07.159 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:07.159 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:07.171 INFO:tasks.workunit.client.0.trial046.stderr:TEST: reshard bucket with ECANCELED injected at set_target_layout 2026-02-24T22:17:07.171 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:07.172 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:07.172 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:07.179 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:07.179 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:07.223 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:07.223 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:07.229 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:07.229 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:07.235 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:07.235 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:07.241 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:07.241 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:07.246 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:07.246 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:07.252 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:07.252 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:07.257 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:07.257 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:07.264 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:07.264 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:07.269 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:07.269 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:07.275 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:07.275 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:07.280 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:07.280 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:07.286 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:07.286 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:07.291 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:07.292 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:07.297 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:07.297 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:07.303 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:07.303 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:07.309 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:07.309 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:07.315 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:07.315 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:07.320 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:07.320 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:07.326 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:07.326 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:07.331 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:07.332 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:07.337 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2026-02-24T22:17:07.351 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:07.351 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:07.368 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.369+0000 7f27e267fa80 4 Realm: () 2026-02-24T22:17:07.368 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.369+0000 7f27e267fa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:07.368 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.369+0000 7f27e267fa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:07.368 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.369+0000 7f27e267fa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:07.368 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.369+0000 7f27e267fa80 20 zonegroup default 2026-02-24T22:17:07.368 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.369+0000 7f27e267fa80 20 rados->read ofs=0 len=0 2026-02-24T22:17:07.369 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.369+0000 7f27e267fa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:07.370 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.369+0000 7f27e267fa80 20 rados->read ofs=0 len=0 2026-02-24T22:17:07.370 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.373+0000 7f27e267fa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:07.370 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.373+0000 7f27e267fa80 20 started sync module instance, tier type = 2026-02-24T22:17:07.370 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.373+0000 7f27e267fa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:07.372 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.373+0000 7f27e267fa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:07.372 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.373+0000 7f27e267fa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:07.372 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.373+0000 7f27e267fa80 5 note: GC not initialized 2026-02-24T22:17:07.372 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.373+0000 7f278e7dc640 20 reqs_thread_entry: start 2026-02-24T22:17:07.375 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.377+0000 7f27e0954640 0 -- 10.20.193.46:0/2883105505 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55ab4cfbfb40 msgr2=0x55ab4cfbff80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:07.381 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.381+0000 7f27dbfff640 0 -- 10.20.193.46:0/2883105505 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55ab4cf8de40 msgr2=0x55ab4cfbbad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:07.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.405+0000 7f27e267fa80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:07.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.405+0000 7f27e267fa80 20 Filter name: none 2026-02-24T22:17:07.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.405+0000 7f278cfd9640 20 reqs_thread_entry: start 2026-02-24T22:17:07.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.405+0000 7f27e267fa80 20 rados->read ofs=0 len=0 2026-02-24T22:17:07.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.405+0000 7f27e267fa80 20 rados_obj.operate() r=0 bl.length=222 2026-02-24T22:17:07.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.405+0000 7f27e267fa80 20 rados->read ofs=0 len=0 2026-02-24T22:17:07.405 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.405+0000 7f27e267fa80 20 rados_obj.operate() r=0 bl.length=452 2026-02-24T22:17:07.405 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.405+0000 7f27e267fa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:07.405 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.405+0000 7f27e267fa80 20 rados->read ofs=0 len=0 2026-02-24T22:17:07.405 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.405+0000 7f27e267fa80 20 rados_obj.operate() r=0 bl.length=222 2026-02-24T22:17:07.405 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.405+0000 7f27e267fa80 20 rados->read ofs=0 len=0 2026-02-24T22:17:07.405 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.405+0000 7f27e267fa80 20 rados_obj.operate() r=0 bl.length=452 2026-02-24T22:17:07.405 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.405+0000 7f27e267fa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:07.413 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:07.413 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:17:07.413 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-set-target-layout id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:17:07.413 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:07.414 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket error-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at set_target_layout --inject-error-code 125 2026-02-24T22:17:07.430 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:07.430 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:07.447 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.449+0000 7f3513205a80 4 Realm: () 2026-02-24T22:17:07.447 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.449+0000 7f3513205a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:07.447 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.449+0000 7f3513205a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:07.447 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.449+0000 7f3513205a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:07.448 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.449+0000 7f3513205a80 20 zonegroup default 2026-02-24T22:17:07.448 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.449+0000 7f3513205a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:07.449 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.449+0000 7f3513205a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:07.449 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.449+0000 7f3513205a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:07.449 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.449+0000 7f3513205a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:07.449 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.449+0000 7f3513205a80 20 started sync module instance, tier type = 2026-02-24T22:17:07.449 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.449+0000 7f3513205a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:07.450 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.453+0000 7f3510cd9640 0 -- 10.20.193.46:0/1064984451 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x5611bccea220 msgr2=0x5611bcd0a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:07.451 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.453+0000 7f3513205a80 20 add_watcher() i=1 2026-02-24T22:17:07.451 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.453+0000 7f3513205a80 20 add_watcher() i=5 2026-02-24T22:17:07.451 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.453+0000 7f3513205a80 20 add_watcher() i=3 2026-02-24T22:17:07.451 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.453+0000 7f3513205a80 20 add_watcher() i=0 2026-02-24T22:17:07.452 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.453+0000 7f3513205a80 20 add_watcher() i=6 2026-02-24T22:17:07.452 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.453+0000 7f3513205a80 20 add_watcher() i=2 2026-02-24T22:17:07.452 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.453+0000 7f3513205a80 20 add_watcher() i=4 2026-02-24T22:17:07.656 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.657+0000 7f3513205a80 20 add_watcher() i=7 2026-02-24T22:17:07.657 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.657+0000 7f3513205a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:17:07.657 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.657+0000 7f350bfff640 0 -- 10.20.193.46:0/1343795982 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f3430008af0 msgr2=0x7f3430028fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:07.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.861+0000 7f3513205a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:07.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.861+0000 7f3513205a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:07.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.861+0000 7f3513205a80 5 note: GC not initialized 2026-02-24T22:17:07.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.861+0000 7f34bdfe3640 20 reqs_thread_entry: start 2026-02-24T22:17:07.896 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.897+0000 7f3513205a80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:07.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.897+0000 7f3513205a80 20 Filter name: none 2026-02-24T22:17:07.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.897+0000 7f3417fff640 20 reqs_thread_entry: start 2026-02-24T22:17:07.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.897+0000 7f3513205a80 10 cache get: name=default.rgw.meta+root+error-at-set-target-layout : miss 2026-02-24T22:17:07.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.897+0000 7f3513205a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:07.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.897+0000 7f3513205a80 20 rados_obj.operate() r=0 bl.length=222 2026-02-24T22:17:07.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.897+0000 7f3513205a80 10 cache put: name=default.rgw.meta+root+error-at-set-target-layout info.flags=0x11 2026-02-24T22:17:07.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.897+0000 7f3513205a80 10 adding default.rgw.meta+root+error-at-set-target-layout to cache LRU end 2026-02-24T22:17:07.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.897+0000 7f3513205a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524 : miss 2026-02-24T22:17:07.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.897+0000 7f3513205a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:07.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.897+0000 7f3513205a80 20 rados_obj.operate() r=0 bl.length=452 2026-02-24T22:17:07.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.897+0000 7f3513205a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524 info.flags=0x17 2026-02-24T22:17:07.898 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.897+0000 7f3513205a80 10 adding default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524 to cache LRU end 2026-02-24T22:17:07.898 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.897+0000 7f3513205a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:17:07.898 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.897+0000 7f3513205a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:07.898 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.897+0000 7f3513205a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524 2026-02-24T22:17:07.900 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.901+0000 7f3513205a80 -1 Injecting error=-125 at location=set_target_layout 2026-02-24T22:17:07.901 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.901+0000 7f3513205a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524 info.flags=0x17 2026-02-24T22:17:07.901 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.901+0000 7f3513205a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524 to cache LRU end 2026-02-24T22:17:07.901 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.901+0000 7f3513205a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:07.901 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.901+0000 7f3513205a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524, ofs0, ns] 2026-02-24T22:17:07.901 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.901+0000 7f34df7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959908 cookie 94634182229568 notifier 6419 bl.length()=1027 2026-02-24T22:17:07.901 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.901+0000 7f34df7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524 info.flags=0x17 2026-02-24T22:17:07.901 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.901+0000 7f34df7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524 to cache LRU end 2026-02-24T22:17:07.902 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.901+0000 7f34df7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:07.902 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.901+0000 7f3513205a80 20 handle_bi_update(): bucket=:error-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524]): orig_sources= new_sources= 2026-02-24T22:17:07.902 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.901+0000 7f3513205a80 20 handle_bi_update(): bucket=:error-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524]): potential sources added=[] removed=[] 2026-02-24T22:17:07.902 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.901+0000 7f3513205a80 20 handle_bi_update(): bucket=:error-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524]): orig_dests= new_dests= 2026-02-24T22:17:07.902 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.901+0000 7f3513205a80 20 handle_bi_update(): bucket=:error-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524]): potential dests added=[] removed=[] 2026-02-24T22:17:07.903 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.905+0000 7f3513205a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524.1.0 2026-02-24T22:17:07.903 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.905+0000 7f3513205a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524.1.0 2026-02-24T22:17:07.903 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.905+0000 7f3513205a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524.1.1 2026-02-24T22:17:07.903 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.905+0000 7f3513205a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524.1.1 2026-02-24T22:17:07.903 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.905+0000 7f3513205a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524.0 2026-02-24T22:17:07.904 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.905+0000 7f3513205a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524 : hit (requested=0x17, cached=0x17) 2026-02-24T22:17:07.904 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.905+0000 7f3513205a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524 2026-02-24T22:17:07.905 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.905+0000 7f3513205a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524 info.flags=0x17 2026-02-24T22:17:07.906 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.905+0000 7f3513205a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524 to cache LRU end 2026-02-24T22:17:07.906 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.905+0000 7f3513205a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:07.906 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.905+0000 7f3513205a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524, ofs0, ns] 2026-02-24T22:17:07.906 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.905+0000 7f34df7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959909 cookie 94634182229568 notifier 6419 bl.length()=1027 2026-02-24T22:17:07.906 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.905+0000 7f34df7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524 info.flags=0x17 2026-02-24T22:17:07.906 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.905+0000 7f34df7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524 to cache LRU end 2026-02-24T22:17:07.906 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.905+0000 7f34df7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:07.906 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.905+0000 7f3513205a80 20 handle_bi_update(): bucket=:error-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524]): orig_sources= new_sources= 2026-02-24T22:17:07.906 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.905+0000 7f3513205a80 20 handle_bi_update(): bucket=:error-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524]): potential sources added=[] removed=[] 2026-02-24T22:17:07.906 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.905+0000 7f3513205a80 20 handle_bi_update(): bucket=:error-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524]): orig_dests= new_dests= 2026-02-24T22:17:07.906 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.905+0000 7f3513205a80 20 handle_bi_update(): bucket=:error-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524]): potential dests added=[] removed=[] 2026-02-24T22:17:07.908 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.909+0000 7f3513205a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524.0 2026-02-24T22:17:07.908 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.909+0000 7f3513205a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524 : hit (requested=0x17, cached=0x17) 2026-02-24T22:17:07.908 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.909+0000 7f3513205a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524 2026-02-24T22:17:07.908 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.909+0000 7f3513205a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524 info.flags=0x17 2026-02-24T22:17:07.908 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.909+0000 7f3513205a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524 to cache LRU end 2026-02-24T22:17:07.908 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.909+0000 7f3513205a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:07.908 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.909+0000 7f3513205a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524, ofs0, ns] 2026-02-24T22:17:07.908 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.909+0000 7f34df7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959910 cookie 94634182229568 notifier 6419 bl.length()=991 2026-02-24T22:17:07.909 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.909+0000 7f34df7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524 info.flags=0x17 2026-02-24T22:17:07.909 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.909+0000 7f34df7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524 to cache LRU end 2026-02-24T22:17:07.909 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.909+0000 7f34df7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:07.909 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.909+0000 7f3513205a80 20 handle_bi_update(): bucket=:error-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524]): orig_sources= new_sources= 2026-02-24T22:17:07.909 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.909+0000 7f3513205a80 20 handle_bi_update(): bucket=:error-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524]): potential sources added=[] removed=[] 2026-02-24T22:17:07.909 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.909+0000 7f3513205a80 20 handle_bi_update(): bucket=:error-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524]): orig_dests= new_dests= 2026-02-24T22:17:07.909 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.909+0000 7f3513205a80 20 handle_bi_update(): bucket=:error-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524]): potential dests added=[] removed=[] 2026-02-24T22:17:07.909 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.909+0000 7f3513205a80 1 execute INFO: reshard of bucket "error-at-set-target-layout" from 1 shards to 2 shards completed successfully 2026-02-24T22:17:07.912 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.913+0000 7f3513205a80 20 remove_watcher() i=4 2026-02-24T22:17:07.912 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.913+0000 7f3513205a80 2 removed watcher, disabling cache 2026-02-24T22:17:07.912 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.913+0000 7f3513205a80 20 remove_watcher() i=1 2026-02-24T22:17:07.912 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.913+0000 7f3513205a80 20 remove_watcher() i=3 2026-02-24T22:17:07.912 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.913+0000 7f3513205a80 20 remove_watcher() i=6 2026-02-24T22:17:07.913 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.913+0000 7f3513205a80 20 remove_watcher() i=2 2026-02-24T22:17:07.913 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.913+0000 7f3513205a80 20 remove_watcher() i=7 2026-02-24T22:17:07.913 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.913+0000 7f3513205a80 20 remove_watcher() i=5 2026-02-24T22:17:07.913 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.913+0000 7f3513205a80 20 remove_watcher() i=0 2026-02-24T22:17:07.919 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2026-02-24T22:17:07.934 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:07.935 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:07.951 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.953+0000 7f738eaa1a80 4 Realm: () 2026-02-24T22:17:07.951 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.953+0000 7f738eaa1a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:07.951 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.953+0000 7f738eaa1a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:07.951 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.953+0000 7f738eaa1a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:07.951 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.953+0000 7f738eaa1a80 20 zonegroup default 2026-02-24T22:17:07.952 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.953+0000 7f738eaa1a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:07.952 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.953+0000 7f738eaa1a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:07.952 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.953+0000 7f738eaa1a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:07.953 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.953+0000 7f738eaa1a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:07.953 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.953+0000 7f738eaa1a80 20 started sync module instance, tier type = 2026-02-24T22:17:07.953 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.953+0000 7f738eaa1a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:07.955 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.957+0000 7f738eaa1a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:07.955 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.957+0000 7f738eaa1a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:07.955 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.957+0000 7f738eaa1a80 5 note: GC not initialized 2026-02-24T22:17:07.955 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.957+0000 7f7339fe3640 20 reqs_thread_entry: start 2026-02-24T22:17:07.958 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.961+0000 7f738cd76640 0 -- 10.20.193.46:0/3805051347 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x5619d1b3ef90 msgr2=0x5619d1b32200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:07.963 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.965+0000 7f7387fff640 0 -- 10.20.193.46:0/3805051347 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5619d1b3aa10 msgr2=0x5619d1b35780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:17:07.988 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.989+0000 7f738eaa1a80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:07.989 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.989+0000 7f738eaa1a80 20 Filter name: none 2026-02-24T22:17:07.989 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.989+0000 7f729bfff640 20 reqs_thread_entry: start 2026-02-24T22:17:07.989 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.989+0000 7f738eaa1a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:07.989 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.989+0000 7f738eaa1a80 20 rados_obj.operate() r=0 bl.length=222 2026-02-24T22:17:07.989 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.989+0000 7f738eaa1a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:07.990 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.989+0000 7f738eaa1a80 20 rados_obj.operate() r=0 bl.length=452 2026-02-24T22:17:07.990 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.989+0000 7f738eaa1a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:07.990 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.989+0000 7f738eaa1a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:07.990 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.989+0000 7f738eaa1a80 20 rados_obj.operate() r=0 bl.length=222 2026-02-24T22:17:07.990 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.989+0000 7f738eaa1a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:07.990 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.989+0000 7f738eaa1a80 20 rados_obj.operate() r=0 bl.length=452 2026-02-24T22:17:07.990 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:07.989+0000 7f738eaa1a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:07.999 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:07.999 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:17:07.999 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-set-target-layout id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524 num_objects: 20 size_kb: 1 num_shards: 2 2026-02-24T22:17:07.999 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.000 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:08.000 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:08.002 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:08.003 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:08.051 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:08.051 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:08.062 INFO:tasks.workunit.client.0.trial046.stderr:TEST: reshard bucket with abort at set_target_layout 2026-02-24T22:17:08.062 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.063 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:08.063 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:08.071 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:08.071 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:08.138 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:08.138 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:08.144 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:08.144 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:08.150 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:08.150 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:08.156 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:08.156 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:08.162 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:08.163 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:08.168 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:08.168 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:08.174 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:08.174 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:08.180 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:08.180 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:08.185 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:08.185 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:08.191 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:08.192 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:08.197 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:08.197 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:08.204 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:08.204 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:08.210 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:08.210 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:08.216 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:08.216 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:08.222 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:08.223 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:08.228 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:08.229 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:08.234 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:08.234 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:08.239 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:08.239 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:08.245 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:08.245 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:08.251 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:08.251 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:08.256 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket abort-at-set-target-layout 2026-02-24T22:17:08.271 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:08.271 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:08.288 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.289+0000 7f24d3f79a80 4 Realm: () 2026-02-24T22:17:08.288 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.289+0000 7f24d3f79a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:08.289 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.289+0000 7f24d3f79a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:08.289 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.289+0000 7f24d3f79a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:08.289 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.289+0000 7f24d3f79a80 20 zonegroup default 2026-02-24T22:17:08.289 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.289+0000 7f24d3f79a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:08.290 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.293+0000 7f24d3f79a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:08.290 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.293+0000 7f24d3f79a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:08.291 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.293+0000 7f24d3f79a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:08.291 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.293+0000 7f24d3f79a80 20 started sync module instance, tier type = 2026-02-24T22:17:08.291 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.293+0000 7f24d3f79a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:08.292 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.293+0000 7f24d3f79a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:08.293 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.293+0000 7f24d3f79a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:08.293 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.293+0000 7f24d3f79a80 5 note: GC not initialized 2026-02-24T22:17:08.293 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.293+0000 7f23f2ffd640 20 reqs_thread_entry: start 2026-02-24T22:17:08.294 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.297+0000 7f24d124c640 0 -- 10.20.193.46:0/2693403885 >> [v2:10.20.193.46:6826/3065971976,v1:10.20.193.46:6827/3065971976] conn(0x5562e2513c20 msgr2=0x5562e2514010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:17:08.299 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.301+0000 7f24d124c640 0 -- 10.20.193.46:0/2693403885 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x5562e2512380 msgr2=0x5562e24fed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:17:08.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.533+0000 7f24d3f79a80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:08.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.533+0000 7f24d3f79a80 20 Filter name: none 2026-02-24T22:17:08.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.533+0000 7f23f17fa640 20 reqs_thread_entry: start 2026-02-24T22:17:08.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.533+0000 7f24d3f79a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:08.533 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.533+0000 7f24d3f79a80 20 rados_obj.operate() r=0 bl.length=222 2026-02-24T22:17:08.533 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.533+0000 7f24d3f79a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:08.533 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.533+0000 7f24d3f79a80 20 rados_obj.operate() r=0 bl.length=452 2026-02-24T22:17:08.533 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.533+0000 7f24d3f79a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:08.533 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.533+0000 7f24d3f79a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:08.533 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.533+0000 7f24d3f79a80 20 rados_obj.operate() r=0 bl.length=222 2026-02-24T22:17:08.533 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.533+0000 7f24d3f79a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:08.533 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.533+0000 7f24d3f79a80 20 rados_obj.operate() r=0 bl.length=452 2026-02-24T22:17:08.533 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.533+0000 7f24d3f79a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:08.543 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.543 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:17:08.543 INFO:tasks.workunit.client.0.trial046.stderr:bucket: abort-at-set-target-layout id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:17:08.543 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.544 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket abort-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at set_target_layout 2026-02-24T22:17:08.560 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:08.560 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:08.576 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.577+0000 7f160cb7da80 4 Realm: () 2026-02-24T22:17:08.576 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.577+0000 7f160cb7da80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:08.577 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.577+0000 7f160cb7da80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:08.577 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.577+0000 7f160cb7da80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:08.577 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.577+0000 7f160cb7da80 20 zonegroup default 2026-02-24T22:17:08.577 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.577+0000 7f160cb7da80 20 rados->read ofs=0 len=0 2026-02-24T22:17:08.578 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.577+0000 7f160cb7da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:08.578 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.577+0000 7f160cb7da80 20 rados->read ofs=0 len=0 2026-02-24T22:17:08.578 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.581+0000 7f160cb7da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:08.578 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.581+0000 7f160cb7da80 20 started sync module instance, tier type = 2026-02-24T22:17:08.578 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.581+0000 7f160cb7da80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:08.579 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.581+0000 7f160a651640 0 -- 10.20.193.46:0/1925587481 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x556d1179cbd0 msgr2=0x556d117bd030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:17:08.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.581+0000 7f160cb7da80 20 add_watcher() i=1 2026-02-24T22:17:08.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.581+0000 7f160cb7da80 20 add_watcher() i=6 2026-02-24T22:17:08.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.581+0000 7f160cb7da80 20 add_watcher() i=5 2026-02-24T22:17:08.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.581+0000 7f160cb7da80 20 add_watcher() i=3 2026-02-24T22:17:08.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.581+0000 7f160cb7da80 20 add_watcher() i=0 2026-02-24T22:17:08.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.581+0000 7f160cb7da80 20 add_watcher() i=2 2026-02-24T22:17:08.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.581+0000 7f160cb7da80 20 add_watcher() i=4 2026-02-24T22:17:08.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.581+0000 7f160cb7da80 20 add_watcher() i=7 2026-02-24T22:17:08.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.581+0000 7f160cb7da80 2 all 8 watchers are set, enabling cache 2026-02-24T22:17:08.583 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.585+0000 7f160cb7da80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:08.583 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.585+0000 7f160cb7da80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:08.583 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.585+0000 7f160cb7da80 5 note: GC not initialized 2026-02-24T22:17:08.583 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.585+0000 7f15237fe640 20 reqs_thread_entry: start 2026-02-24T22:17:08.584 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.585+0000 7f160ae52640 0 -- 10.20.193.46:0/3793593507 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x556d117b38d0 msgr2=0x556d117b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:08.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.825+0000 7f160cb7da80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:08.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.825+0000 7f160cb7da80 20 Filter name: none 2026-02-24T22:17:08.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.825+0000 7f1521ffb640 20 reqs_thread_entry: start 2026-02-24T22:17:08.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.825+0000 7f160cb7da80 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2026-02-24T22:17:08.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.825+0000 7f160cb7da80 20 rados->read ofs=0 len=0 2026-02-24T22:17:08.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.825+0000 7f160cb7da80 20 rados_obj.operate() r=0 bl.length=222 2026-02-24T22:17:08.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.825+0000 7f160cb7da80 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2026-02-24T22:17:08.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.825+0000 7f160cb7da80 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2026-02-24T22:17:08.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.825+0000 7f160cb7da80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 : miss 2026-02-24T22:17:08.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.825+0000 7f160cb7da80 20 rados->read ofs=0 len=0 2026-02-24T22:17:08.825 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.825+0000 7f160cb7da80 20 rados_obj.operate() r=0 bl.length=452 2026-02-24T22:17:08.825 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.825+0000 7f160cb7da80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 info.flags=0x17 2026-02-24T22:17:08.825 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.825+0000 7f160cb7da80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 to cache LRU end 2026-02-24T22:17:08.825 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.825+0000 7f160cb7da80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:17:08.825 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.825+0000 7f160cb7da80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:08.825 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.825+0000 7f160cb7da80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 2026-02-24T22:17:08.827 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f160cb7da80 time 2026-02-24T22:17:08.833071+0000 2026-02-24T22:17:08.827 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:17:08.828 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:17:08.828 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f160ffbfeb1] 2026-02-24T22:17:08.829 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x1713e63) [0x556cef4f7e63] 2026-02-24T22:17:08.829 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x556ceea55202] 2026-02-24T22:17:08.829 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:17:08.829 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f160f2bfd90] 2026-02-24T22:17:08.829 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:17:08.829 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:17:08.829 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.829+0000 7f160cb7da80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f160cb7da80 time 2026-02-24T22:17:08.833071+0000 2026-02-24T22:17:08.829 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:17:08.829 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.830 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:17:08.830 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f160ffbfeb1] 2026-02-24T22:17:08.830 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x1713e63) [0x556cef4f7e63] 2026-02-24T22:17:08.830 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x556ceea55202] 2026-02-24T22:17:08.830 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:17:08.830 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f160f2bfd90] 2026-02-24T22:17:08.830 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:17:08.830 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:17:08.830 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.830 INFO:tasks.workunit.client.0.trial046.stderr:*** Caught signal (Aborted) ** 2026-02-24T22:17:08.831 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f160cb7da80 thread_name:radosgw-admin 2026-02-24T22:17:08.851 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:17:08.852 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f160f2d8520] 2026-02-24T22:17:08.852 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:17:08.852 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:17:08.852 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:17:08.852 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f160ffbff10] 2026-02-24T22:17:08.852 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x1713e63) [0x556cef4f7e63] 2026-02-24T22:17:08.852 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x556ceea55202] 2026-02-24T22:17:08.852 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:17:08.852 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f160f2bfd90] 2026-02-24T22:17:08.853 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:17:08.853 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:17:08.853 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.829+0000 7f160cb7da80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:17:08.853 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f160cb7da80 thread_name:radosgw-admin 2026-02-24T22:17:08.853 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.853 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:17:08.853 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f160f2d8520] 2026-02-24T22:17:08.853 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:17:08.853 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:17:08.853 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:17:08.853 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f160ffbff10] 2026-02-24T22:17:08.853 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x1713e63) [0x556cef4f7e63] 2026-02-24T22:17:08.854 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x556ceea55202] 2026-02-24T22:17:08.854 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:17:08.854 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f160f2bfd90] 2026-02-24T22:17:08.854 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:17:08.854 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:17:08.854 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:17:08.854 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.854 INFO:tasks.workunit.client.0.trial046.stderr:--- begin dump of recent events --- 2026-02-24T22:17:08.854 INFO:tasks.workunit.client.0.trial046.stderr: -499> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f0f0) adding con mode: secure 2026-02-24T22:17:08.854 INFO:tasks.workunit.client.0.trial046.stderr: -498> 2026-02-24T22:17:08.565+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.854 INFO:tasks.workunit.client.0.trial046.stderr: -497> 2026-02-24T22:17:08.565+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.854 INFO:tasks.workunit.client.0.trial046.stderr: -496> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command rotate-key hook 0x7ffc2c48f238 2026-02-24T22:17:08.855 INFO:tasks.workunit.client.0.trial046.stderr: -495> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient: _reopen_session rank -1 2026-02-24T22:17:08.855 INFO:tasks.workunit.client.0.trial046.stderr: -494> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T22:17:08.855 INFO:tasks.workunit.client.0.trial046.stderr: -493> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-a con 0x556d111d7fe0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:17:08.855 INFO:tasks.workunit.client.0.trial046.stderr: -492> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-b con 0x556d11280b00 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:17:08.855 INFO:tasks.workunit.client.0.trial046.stderr: -491> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-c con 0x556d11283cf0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:17:08.855 INFO:tasks.workunit.client.0.trial046.stderr: -490> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.855 INFO:tasks.workunit.client.0.trial046.stderr: -489> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.855 INFO:tasks.workunit.client.0.trial046.stderr: -488> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.855 INFO:tasks.workunit.client.0.trial046.stderr: -487> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): _renew_subs 2026-02-24T22:17:08.855 INFO:tasks.workunit.client.0.trial046.stderr: -486> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): authenticate will time out at 1798.674511s 2026-02-24T22:17:08.855 INFO:tasks.workunit.client.0.trial046.stderr: -485> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient(hunting): get_auth_request con 0x556d111d7fe0 auth_method 0 2026-02-24T22:17:08.855 INFO:tasks.workunit.client.0.trial046.stderr: -484> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.856 INFO:tasks.workunit.client.0.trial046.stderr: -483> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.856 INFO:tasks.workunit.client.0.trial046.stderr: -482> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.856 INFO:tasks.workunit.client.0.trial046.stderr: -481> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:08.856 INFO:tasks.workunit.client.0.trial046.stderr: -480> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:08.856 INFO:tasks.workunit.client.0.trial046.stderr: -479> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:08.856 INFO:tasks.workunit.client.0.trial046.stderr: -478> 2026-02-24T22:17:08.565+0000 7f160ae52640 10 monclient(hunting): get_auth_request con 0x556d11283cf0 auth_method 0 2026-02-24T22:17:08.856 INFO:tasks.workunit.client.0.trial046.stderr: -477> 2026-02-24T22:17:08.565+0000 7f160ae52640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.856 INFO:tasks.workunit.client.0.trial046.stderr: -476> 2026-02-24T22:17:08.565+0000 7f160ae52640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.856 INFO:tasks.workunit.client.0.trial046.stderr: -475> 2026-02-24T22:17:08.565+0000 7f160ae52640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.856 INFO:tasks.workunit.client.0.trial046.stderr: -474> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient(hunting): handle_auth_done global_id 6894 payload 274 2026-02-24T22:17:08.856 INFO:tasks.workunit.client.0.trial046.stderr: -473> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient: _finish_hunting 0 2026-02-24T22:17:08.856 INFO:tasks.workunit.client.0.trial046.stderr: -472> 2026-02-24T22:17:08.565+0000 7f160a651640 1 monclient: found mon.noname-a 2026-02-24T22:17:08.856 INFO:tasks.workunit.client.0.trial046.stderr: -471> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:17:08.857 INFO:tasks.workunit.client.0.trial046.stderr: -470> 2026-02-24T22:17:08.565+0000 7f15f8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:08.857 INFO:tasks.workunit.client.0.trial046.stderr: -469> 2026-02-24T22:17:08.565+0000 7f15f8ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:17:08.857 INFO:tasks.workunit.client.0.trial046.stderr: -468> 2026-02-24T22:17:08.565+0000 7f15f8ff9640 10 monclient: dump: 2026-02-24T22:17:08.857 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:08.857 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:08.857 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.857 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.857 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:08.857 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.857 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:08.857 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:08.857 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:08.857 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.858 INFO:tasks.workunit.client.0.trial046.stderr: -467> 2026-02-24T22:17:08.565+0000 7f15f8ff9640 10 monclient: _finish_auth 0 2026-02-24T22:17:08.858 INFO:tasks.workunit.client.0.trial046.stderr: -466> 2026-02-24T22:17:08.565+0000 7f15f8ff9640 10 monclient: _check_auth_tickets 2026-02-24T22:17:08.858 INFO:tasks.workunit.client.0.trial046.stderr: -465> 2026-02-24T22:17:08.565+0000 7f15f8ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T22:17:08.858 INFO:tasks.workunit.client.0.trial046.stderr: -464> 2026-02-24T22:17:08.565+0000 7f15f8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:08.858 INFO:tasks.workunit.client.0.trial046.stderr: -463> 2026-02-24T22:17:08.565+0000 7f15f8ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:17:08.858 INFO:tasks.workunit.client.0.trial046.stderr: -462> 2026-02-24T22:17:08.565+0000 7f15f8ff9640 10 monclient: dump: 2026-02-24T22:17:08.858 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:08.858 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:08.858 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.858 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.858 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:08.858 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.858 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:08.859 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:08.859 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:08.859 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.859 INFO:tasks.workunit.client.0.trial046.stderr: -461> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 monclient: authenticate success, global_id 6894 2026-02-24T22:17:08.859 INFO:tasks.workunit.client.0.trial046.stderr: -460> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient: get_monmap_and_config success 2026-02-24T22:17:08.859 INFO:tasks.workunit.client.0.trial046.stderr: -459> 2026-02-24T22:17:08.565+0000 7f160cb7da80 4 set_mon_vals no callback set 2026-02-24T22:17:08.859 INFO:tasks.workunit.client.0.trial046.stderr: -458> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient: shutdown 2026-02-24T22:17:08.859 INFO:tasks.workunit.client.0.trial046.stderr: -457> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 asok(0x556d11149f00) unregister_commands rotate-key 2026-02-24T22:17:08.859 INFO:tasks.workunit.client.0.trial046.stderr: -456> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient: build_initial_monmap 2026-02-24T22:17:08.859 INFO:tasks.workunit.client.0.trial046.stderr: -455> 2026-02-24T22:17:08.565+0000 7f160cb7da80 1 build_initial for_mkfs: 0 2026-02-24T22:17:08.859 INFO:tasks.workunit.client.0.trial046.stderr: -454> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient: monmap: 2026-02-24T22:17:08.859 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:17:08.859 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:17:08.860 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:17:08.860 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:17:08.860 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:17:08.860 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.860 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:17:08.860 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:17:08.860 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:17:08.860 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.860 INFO:tasks.workunit.client.0.trial046.stderr: -453> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding auth protocol: cephx 2026-02-24T22:17:08.860 INFO:tasks.workunit.client.0.trial046.stderr: -452> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding auth protocol: cephx 2026-02-24T22:17:08.860 INFO:tasks.workunit.client.0.trial046.stderr: -451> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding auth protocol: cephx 2026-02-24T22:17:08.860 INFO:tasks.workunit.client.0.trial046.stderr: -450> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding con mode: secure 2026-02-24T22:17:08.860 INFO:tasks.workunit.client.0.trial046.stderr: -449> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding con mode: crc 2026-02-24T22:17:08.860 INFO:tasks.workunit.client.0.trial046.stderr: -448> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding con mode: secure 2026-02-24T22:17:08.861 INFO:tasks.workunit.client.0.trial046.stderr: -447> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding con mode: crc 2026-02-24T22:17:08.861 INFO:tasks.workunit.client.0.trial046.stderr: -446> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding con mode: secure 2026-02-24T22:17:08.861 INFO:tasks.workunit.client.0.trial046.stderr: -445> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding con mode: crc 2026-02-24T22:17:08.861 INFO:tasks.workunit.client.0.trial046.stderr: -444> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding con mode: crc 2026-02-24T22:17:08.861 INFO:tasks.workunit.client.0.trial046.stderr: -443> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding con mode: secure 2026-02-24T22:17:08.861 INFO:tasks.workunit.client.0.trial046.stderr: -442> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding con mode: crc 2026-02-24T22:17:08.861 INFO:tasks.workunit.client.0.trial046.stderr: -441> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding con mode: secure 2026-02-24T22:17:08.861 INFO:tasks.workunit.client.0.trial046.stderr: -440> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding con mode: crc 2026-02-24T22:17:08.861 INFO:tasks.workunit.client.0.trial046.stderr: -439> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding con mode: secure 2026-02-24T22:17:08.861 INFO:tasks.workunit.client.0.trial046.stderr: -438> 2026-02-24T22:17:08.565+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.861 INFO:tasks.workunit.client.0.trial046.stderr: -437> 2026-02-24T22:17:08.565+0000 7f160cb7da80 1 librados: starting msgr at 2026-02-24T22:17:08.861 INFO:tasks.workunit.client.0.trial046.stderr: -436> 2026-02-24T22:17:08.565+0000 7f160cb7da80 1 librados: starting objecter 2026-02-24T22:17:08.861 INFO:tasks.workunit.client.0.trial046.stderr: -435> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command objecter_requests hook 0x556d1124bbf0 2026-02-24T22:17:08.861 INFO:tasks.workunit.client.0.trial046.stderr: -434> 2026-02-24T22:17:08.565+0000 7f160cb7da80 1 librados: setting wanted keys 2026-02-24T22:17:08.862 INFO:tasks.workunit.client.0.trial046.stderr: -433> 2026-02-24T22:17:08.565+0000 7f160cb7da80 1 librados: calling monclient init 2026-02-24T22:17:08.862 INFO:tasks.workunit.client.0.trial046.stderr: -432> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient: init 2026-02-24T22:17:08.862 INFO:tasks.workunit.client.0.trial046.stderr: -431> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding auth protocol: cephx 2026-02-24T22:17:08.862 INFO:tasks.workunit.client.0.trial046.stderr: -430> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding auth protocol: cephx 2026-02-24T22:17:08.862 INFO:tasks.workunit.client.0.trial046.stderr: -429> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding auth protocol: cephx 2026-02-24T22:17:08.862 INFO:tasks.workunit.client.0.trial046.stderr: -428> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding con mode: secure 2026-02-24T22:17:08.862 INFO:tasks.workunit.client.0.trial046.stderr: -427> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding con mode: crc 2026-02-24T22:17:08.862 INFO:tasks.workunit.client.0.trial046.stderr: -426> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding con mode: secure 2026-02-24T22:17:08.862 INFO:tasks.workunit.client.0.trial046.stderr: -425> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding con mode: crc 2026-02-24T22:17:08.862 INFO:tasks.workunit.client.0.trial046.stderr: -424> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding con mode: secure 2026-02-24T22:17:08.862 INFO:tasks.workunit.client.0.trial046.stderr: -423> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding con mode: crc 2026-02-24T22:17:08.862 INFO:tasks.workunit.client.0.trial046.stderr: -422> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding con mode: crc 2026-02-24T22:17:08.862 INFO:tasks.workunit.client.0.trial046.stderr: -421> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding con mode: secure 2026-02-24T22:17:08.862 INFO:tasks.workunit.client.0.trial046.stderr: -420> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding con mode: crc 2026-02-24T22:17:08.862 INFO:tasks.workunit.client.0.trial046.stderr: -419> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding con mode: secure 2026-02-24T22:17:08.863 INFO:tasks.workunit.client.0.trial046.stderr: -418> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding con mode: crc 2026-02-24T22:17:08.863 INFO:tasks.workunit.client.0.trial046.stderr: -417> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding con mode: secure 2026-02-24T22:17:08.863 INFO:tasks.workunit.client.0.trial046.stderr: -416> 2026-02-24T22:17:08.565+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.863 INFO:tasks.workunit.client.0.trial046.stderr: -415> 2026-02-24T22:17:08.565+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.863 INFO:tasks.workunit.client.0.trial046.stderr: -414> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command rotate-key hook 0x556d112494b8 2026-02-24T22:17:08.863 INFO:tasks.workunit.client.0.trial046.stderr: -413> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient: _reopen_session rank -1 2026-02-24T22:17:08.863 INFO:tasks.workunit.client.0.trial046.stderr: -412> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T22:17:08.863 INFO:tasks.workunit.client.0.trial046.stderr: -411> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-a con 0x556d111d7fe0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:17:08.863 INFO:tasks.workunit.client.0.trial046.stderr: -410> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-c con 0x556d11280b00 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:17:08.863 INFO:tasks.workunit.client.0.trial046.stderr: -409> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-b con 0x556d11283cf0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:17:08.863 INFO:tasks.workunit.client.0.trial046.stderr: -408> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.863 INFO:tasks.workunit.client.0.trial046.stderr: -407> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.863 INFO:tasks.workunit.client.0.trial046.stderr: -406> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.863 INFO:tasks.workunit.client.0.trial046.stderr: -405> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): _renew_subs 2026-02-24T22:17:08.863 INFO:tasks.workunit.client.0.trial046.stderr: -404> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): authenticate will time out at 1798.675850s 2026-02-24T22:17:08.864 INFO:tasks.workunit.client.0.trial046.stderr: -403> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient(hunting): get_auth_request con 0x556d111d7fe0 auth_method 0 2026-02-24T22:17:08.864 INFO:tasks.workunit.client.0.trial046.stderr: -402> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.864 INFO:tasks.workunit.client.0.trial046.stderr: -401> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.864 INFO:tasks.workunit.client.0.trial046.stderr: -400> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.864 INFO:tasks.workunit.client.0.trial046.stderr: -399> 2026-02-24T22:17:08.565+0000 7f1609e50640 10 monclient(hunting): get_auth_request con 0x556d11280b00 auth_method 0 2026-02-24T22:17:08.864 INFO:tasks.workunit.client.0.trial046.stderr: -398> 2026-02-24T22:17:08.565+0000 7f1609e50640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.864 INFO:tasks.workunit.client.0.trial046.stderr: -397> 2026-02-24T22:17:08.569+0000 7f1609e50640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.864 INFO:tasks.workunit.client.0.trial046.stderr: -396> 2026-02-24T22:17:08.569+0000 7f1609e50640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.864 INFO:tasks.workunit.client.0.trial046.stderr: -395> 2026-02-24T22:17:08.569+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:08.864 INFO:tasks.workunit.client.0.trial046.stderr: -394> 2026-02-24T22:17:08.569+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:08.864 INFO:tasks.workunit.client.0.trial046.stderr: -393> 2026-02-24T22:17:08.569+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:08.864 INFO:tasks.workunit.client.0.trial046.stderr: -392> 2026-02-24T22:17:08.569+0000 7f160ae52640 10 monclient(hunting): get_auth_request con 0x556d11283cf0 auth_method 0 2026-02-24T22:17:08.864 INFO:tasks.workunit.client.0.trial046.stderr: -391> 2026-02-24T22:17:08.569+0000 7f160ae52640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.864 INFO:tasks.workunit.client.0.trial046.stderr: -390> 2026-02-24T22:17:08.569+0000 7f160ae52640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.864 INFO:tasks.workunit.client.0.trial046.stderr: -389> 2026-02-24T22:17:08.569+0000 7f160ae52640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.864 INFO:tasks.workunit.client.0.trial046.stderr: -388> 2026-02-24T22:17:08.569+0000 7f1609e50640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:08.865 INFO:tasks.workunit.client.0.trial046.stderr: -387> 2026-02-24T22:17:08.569+0000 7f1609e50640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:08.865 INFO:tasks.workunit.client.0.trial046.stderr: -386> 2026-02-24T22:17:08.569+0000 7f1609e50640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:08.865 INFO:tasks.workunit.client.0.trial046.stderr: -385> 2026-02-24T22:17:08.569+0000 7f160a651640 10 monclient(hunting): handle_auth_done global_id 6897 payload 824 2026-02-24T22:17:08.865 INFO:tasks.workunit.client.0.trial046.stderr: -384> 2026-02-24T22:17:08.569+0000 7f160a651640 10 monclient: _finish_hunting 0 2026-02-24T22:17:08.865 INFO:tasks.workunit.client.0.trial046.stderr: -383> 2026-02-24T22:17:08.569+0000 7f160a651640 1 monclient: found mon.noname-a 2026-02-24T22:17:08.865 INFO:tasks.workunit.client.0.trial046.stderr: -382> 2026-02-24T22:17:08.569+0000 7f160a651640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:17:08.865 INFO:tasks.workunit.client.0.trial046.stderr: -381> 2026-02-24T22:17:08.569+0000 7f15eeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:08.865 INFO:tasks.workunit.client.0.trial046.stderr: -380> 2026-02-24T22:17:08.569+0000 7f15eeffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:17:08.865 INFO:tasks.workunit.client.0.trial046.stderr: -379> 2026-02-24T22:17:08.569+0000 7f15eeffd640 10 monclient: dump: 2026-02-24T22:17:08.865 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:08.865 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:08.865 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.865 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.865 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:08.865 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.865 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:08.866 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:08.866 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:08.866 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.866 INFO:tasks.workunit.client.0.trial046.stderr: -378> 2026-02-24T22:17:08.569+0000 7f15eeffd640 10 monclient: _finish_auth 0 2026-02-24T22:17:08.866 INFO:tasks.workunit.client.0.trial046.stderr: -377> 2026-02-24T22:17:08.569+0000 7f15eeffd640 10 monclient: _check_auth_tickets 2026-02-24T22:17:08.866 INFO:tasks.workunit.client.0.trial046.stderr: -376> 2026-02-24T22:17:08.569+0000 7f15eeffd640 10 monclient: handle_config config(0 keys) 2026-02-24T22:17:08.866 INFO:tasks.workunit.client.0.trial046.stderr: -375> 2026-02-24T22:17:08.569+0000 7f15eeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:08.866 INFO:tasks.workunit.client.0.trial046.stderr: -374> 2026-02-24T22:17:08.569+0000 7f15eeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:17:08.866 INFO:tasks.workunit.client.0.trial046.stderr: -373> 2026-02-24T22:17:08.569+0000 7f15eeffd640 10 monclient: dump: 2026-02-24T22:17:08.866 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:08.866 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:08.866 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.866 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.866 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:08.866 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.866 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:08.867 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:08.867 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:08.867 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.867 INFO:tasks.workunit.client.0.trial046.stderr: -372> 2026-02-24T22:17:08.569+0000 7f160cb7da80 5 monclient: authenticate success, global_id 6897 2026-02-24T22:17:08.867 INFO:tasks.workunit.client.0.trial046.stderr: -371> 2026-02-24T22:17:08.569+0000 7f160cb7da80 10 monclient: _renew_subs 2026-02-24T22:17:08.867 INFO:tasks.workunit.client.0.trial046.stderr: -370> 2026-02-24T22:17:08.569+0000 7f160cb7da80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:17:08.867 INFO:tasks.workunit.client.0.trial046.stderr: -369> 2026-02-24T22:17:08.569+0000 7f15f8ff9640 4 set_mon_vals no callback set 2026-02-24T22:17:08.867 INFO:tasks.workunit.client.0.trial046.stderr: -368> 2026-02-24T22:17:08.569+0000 7f160cb7da80 10 monclient: _renew_subs 2026-02-24T22:17:08.867 INFO:tasks.workunit.client.0.trial046.stderr: -367> 2026-02-24T22:17:08.569+0000 7f160cb7da80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:17:08.867 INFO:tasks.workunit.client.0.trial046.stderr: -366> 2026-02-24T22:17:08.569+0000 7f160cb7da80 1 librados: init done 2026-02-24T22:17:08.867 INFO:tasks.workunit.client.0.trial046.stderr: -365> 2026-02-24T22:17:08.569+0000 7f15eeffd640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:17:08.867 INFO:tasks.workunit.client.0.trial046.stderr: -364> 2026-02-24T22:17:08.569+0000 7f15eeffd640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:17:08.867 INFO:tasks.workunit.client.0.trial046.stderr: -363> 2026-02-24T22:17:08.569+0000 7f15eeffd640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:17:08.867 INFO:tasks.workunit.client.0.trial046.stderr: -362> 2026-02-24T22:17:08.569+0000 7f1609e50640 10 monclient: get_auth_request con 0x7f15e80707b0 auth_method 0 2026-02-24T22:17:08.867 INFO:tasks.workunit.client.0.trial046.stderr: -361> 2026-02-24T22:17:08.569+0000 7f160ae52640 10 monclient: get_auth_request con 0x556d11276210 auth_method 0 2026-02-24T22:17:08.868 INFO:tasks.workunit.client.0.trial046.stderr: -360> 2026-02-24T22:17:08.569+0000 7f160a651640 10 monclient: get_auth_request con 0x556d112d7a20 auth_method 0 2026-02-24T22:17:08.868 INFO:tasks.workunit.client.0.trial046.stderr: -359> 2026-02-24T22:17:08.573+0000 7f1609e50640 10 monclient: get_auth_request con 0x556d112d9100 auth_method 0 2026-02-24T22:17:08.868 INFO:tasks.workunit.client.0.trial046.stderr: -358> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: build_initial_monmap 2026-02-24T22:17:08.868 INFO:tasks.workunit.client.0.trial046.stderr: -357> 2026-02-24T22:17:08.573+0000 7f160cb7da80 1 build_initial for_mkfs: 0 2026-02-24T22:17:08.868 INFO:tasks.workunit.client.0.trial046.stderr: -356> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: monmap: 2026-02-24T22:17:08.868 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:17:08.868 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:17:08.868 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:17:08.868 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:17:08.868 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:17:08.868 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.868 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:17:08.868 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:17:08.868 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:17:08.868 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.869 INFO:tasks.workunit.client.0.trial046.stderr: -355> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding auth protocol: cephx 2026-02-24T22:17:08.869 INFO:tasks.workunit.client.0.trial046.stderr: -354> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding auth protocol: cephx 2026-02-24T22:17:08.869 INFO:tasks.workunit.client.0.trial046.stderr: -353> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding auth protocol: cephx 2026-02-24T22:17:08.869 INFO:tasks.workunit.client.0.trial046.stderr: -352> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding con mode: secure 2026-02-24T22:17:08.869 INFO:tasks.workunit.client.0.trial046.stderr: -351> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding con mode: crc 2026-02-24T22:17:08.869 INFO:tasks.workunit.client.0.trial046.stderr: -350> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding con mode: secure 2026-02-24T22:17:08.869 INFO:tasks.workunit.client.0.trial046.stderr: -349> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding con mode: crc 2026-02-24T22:17:08.869 INFO:tasks.workunit.client.0.trial046.stderr: -348> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding con mode: secure 2026-02-24T22:17:08.869 INFO:tasks.workunit.client.0.trial046.stderr: -347> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding con mode: crc 2026-02-24T22:17:08.869 INFO:tasks.workunit.client.0.trial046.stderr: -346> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding con mode: crc 2026-02-24T22:17:08.869 INFO:tasks.workunit.client.0.trial046.stderr: -345> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding con mode: secure 2026-02-24T22:17:08.869 INFO:tasks.workunit.client.0.trial046.stderr: -344> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding con mode: crc 2026-02-24T22:17:08.869 INFO:tasks.workunit.client.0.trial046.stderr: -343> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding con mode: secure 2026-02-24T22:17:08.869 INFO:tasks.workunit.client.0.trial046.stderr: -342> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding con mode: crc 2026-02-24T22:17:08.869 INFO:tasks.workunit.client.0.trial046.stderr: -341> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding con mode: secure 2026-02-24T22:17:08.869 INFO:tasks.workunit.client.0.trial046.stderr: -340> 2026-02-24T22:17:08.573+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.869 INFO:tasks.workunit.client.0.trial046.stderr: -339> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command objecter_requests cmddesc objecter_requests hook 0x556d112e01b0 EEXIST 2026-02-24T22:17:08.870 INFO:tasks.workunit.client.0.trial046.stderr: -338> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: init 2026-02-24T22:17:08.870 INFO:tasks.workunit.client.0.trial046.stderr: -337> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding auth protocol: cephx 2026-02-24T22:17:08.870 INFO:tasks.workunit.client.0.trial046.stderr: -336> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding auth protocol: cephx 2026-02-24T22:17:08.870 INFO:tasks.workunit.client.0.trial046.stderr: -335> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding auth protocol: cephx 2026-02-24T22:17:08.870 INFO:tasks.workunit.client.0.trial046.stderr: -334> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding con mode: secure 2026-02-24T22:17:08.870 INFO:tasks.workunit.client.0.trial046.stderr: -333> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding con mode: crc 2026-02-24T22:17:08.870 INFO:tasks.workunit.client.0.trial046.stderr: -332> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding con mode: secure 2026-02-24T22:17:08.870 INFO:tasks.workunit.client.0.trial046.stderr: -331> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding con mode: crc 2026-02-24T22:17:08.870 INFO:tasks.workunit.client.0.trial046.stderr: -330> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding con mode: secure 2026-02-24T22:17:08.870 INFO:tasks.workunit.client.0.trial046.stderr: -329> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding con mode: crc 2026-02-24T22:17:08.870 INFO:tasks.workunit.client.0.trial046.stderr: -328> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding con mode: crc 2026-02-24T22:17:08.870 INFO:tasks.workunit.client.0.trial046.stderr: -327> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding con mode: secure 2026-02-24T22:17:08.870 INFO:tasks.workunit.client.0.trial046.stderr: -326> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding con mode: crc 2026-02-24T22:17:08.870 INFO:tasks.workunit.client.0.trial046.stderr: -325> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding con mode: secure 2026-02-24T22:17:08.870 INFO:tasks.workunit.client.0.trial046.stderr: -324> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding con mode: crc 2026-02-24T22:17:08.870 INFO:tasks.workunit.client.0.trial046.stderr: -323> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding con mode: secure 2026-02-24T22:17:08.870 INFO:tasks.workunit.client.0.trial046.stderr: -322> 2026-02-24T22:17:08.573+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.870 INFO:tasks.workunit.client.0.trial046.stderr: -321> 2026-02-24T22:17:08.573+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.871 INFO:tasks.workunit.client.0.trial046.stderr: -320> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command rotate-key cmddesc rotate-key hook 0x556d112d9e00 EEXIST 2026-02-24T22:17:08.871 INFO:tasks.workunit.client.0.trial046.stderr: -319> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: _reopen_session rank -1 2026-02-24T22:17:08.871 INFO:tasks.workunit.client.0.trial046.stderr: -318> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T22:17:08.871 INFO:tasks.workunit.client.0.trial046.stderr: -317> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-a con 0x556d11433d90 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:17:08.871 INFO:tasks.workunit.client.0.trial046.stderr: -316> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-c con 0x556d11431640 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:17:08.871 INFO:tasks.workunit.client.0.trial046.stderr: -315> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-b con 0x556d1142ade0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:17:08.871 INFO:tasks.workunit.client.0.trial046.stderr: -314> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.871 INFO:tasks.workunit.client.0.trial046.stderr: -313> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.871 INFO:tasks.workunit.client.0.trial046.stderr: -312> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.871 INFO:tasks.workunit.client.0.trial046.stderr: -311> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): _renew_subs 2026-02-24T22:17:08.871 INFO:tasks.workunit.client.0.trial046.stderr: -310> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): authenticate will time out at 1798.681241s 2026-02-24T22:17:08.871 INFO:tasks.workunit.client.0.trial046.stderr: -309> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): get_auth_request con 0x556d11433d90 auth_method 0 2026-02-24T22:17:08.871 INFO:tasks.workunit.client.0.trial046.stderr: -308> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.871 INFO:tasks.workunit.client.0.trial046.stderr: -307> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.871 INFO:tasks.workunit.client.0.trial046.stderr: -306> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.871 INFO:tasks.workunit.client.0.trial046.stderr: -305> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:08.871 INFO:tasks.workunit.client.0.trial046.stderr: -304> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:08.872 INFO:tasks.workunit.client.0.trial046.stderr: -303> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:08.872 INFO:tasks.workunit.client.0.trial046.stderr: -302> 2026-02-24T22:17:08.573+0000 7f1609e50640 10 monclient(hunting): get_auth_request con 0x556d11431640 auth_method 0 2026-02-24T22:17:08.872 INFO:tasks.workunit.client.0.trial046.stderr: -301> 2026-02-24T22:17:08.573+0000 7f1609e50640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.872 INFO:tasks.workunit.client.0.trial046.stderr: -300> 2026-02-24T22:17:08.573+0000 7f1609e50640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.872 INFO:tasks.workunit.client.0.trial046.stderr: -299> 2026-02-24T22:17:08.573+0000 7f1609e50640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.872 INFO:tasks.workunit.client.0.trial046.stderr: -298> 2026-02-24T22:17:08.573+0000 7f1609e50640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:08.872 INFO:tasks.workunit.client.0.trial046.stderr: -297> 2026-02-24T22:17:08.573+0000 7f1609e50640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:08.872 INFO:tasks.workunit.client.0.trial046.stderr: -296> 2026-02-24T22:17:08.573+0000 7f1609e50640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:08.872 INFO:tasks.workunit.client.0.trial046.stderr: -295> 2026-02-24T22:17:08.573+0000 7f160ae52640 10 monclient(hunting): get_auth_request con 0x556d1142ade0 auth_method 0 2026-02-24T22:17:08.872 INFO:tasks.workunit.client.0.trial046.stderr: -294> 2026-02-24T22:17:08.573+0000 7f160ae52640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.872 INFO:tasks.workunit.client.0.trial046.stderr: -293> 2026-02-24T22:17:08.573+0000 7f160ae52640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.872 INFO:tasks.workunit.client.0.trial046.stderr: -292> 2026-02-24T22:17:08.573+0000 7f160ae52640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.872 INFO:tasks.workunit.client.0.trial046.stderr: -291> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): handle_auth_done global_id 6900 payload 824 2026-02-24T22:17:08.872 INFO:tasks.workunit.client.0.trial046.stderr: -290> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient: _finish_hunting 0 2026-02-24T22:17:08.872 INFO:tasks.workunit.client.0.trial046.stderr: -289> 2026-02-24T22:17:08.573+0000 7f160a651640 1 monclient: found mon.noname-a 2026-02-24T22:17:08.872 INFO:tasks.workunit.client.0.trial046.stderr: -288> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:17:08.872 INFO:tasks.workunit.client.0.trial046.stderr: -287> 2026-02-24T22:17:08.573+0000 7f15e7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:08.873 INFO:tasks.workunit.client.0.trial046.stderr: -286> 2026-02-24T22:17:08.573+0000 7f15e7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:17:08.873 INFO:tasks.workunit.client.0.trial046.stderr: -285> 2026-02-24T22:17:08.573+0000 7f15e7fff640 10 monclient: dump: 2026-02-24T22:17:08.873 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:08.873 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:08.873 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.873 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.873 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:08.873 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.873 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:08.873 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:08.873 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:08.873 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.873 INFO:tasks.workunit.client.0.trial046.stderr: -284> 2026-02-24T22:17:08.573+0000 7f15e7fff640 10 monclient: _finish_auth 0 2026-02-24T22:17:08.873 INFO:tasks.workunit.client.0.trial046.stderr: -283> 2026-02-24T22:17:08.573+0000 7f15e7fff640 10 monclient: _check_auth_tickets 2026-02-24T22:17:08.873 INFO:tasks.workunit.client.0.trial046.stderr: -282> 2026-02-24T22:17:08.573+0000 7f15e7fff640 10 monclient: handle_config config(0 keys) 2026-02-24T22:17:08.873 INFO:tasks.workunit.client.0.trial046.stderr: -281> 2026-02-24T22:17:08.573+0000 7f15e7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:08.873 INFO:tasks.workunit.client.0.trial046.stderr: -280> 2026-02-24T22:17:08.573+0000 7f15e7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:17:08.873 INFO:tasks.workunit.client.0.trial046.stderr: -279> 2026-02-24T22:17:08.573+0000 7f15e7fff640 10 monclient: dump: 2026-02-24T22:17:08.873 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:08.874 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:08.874 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.874 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.874 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:08.874 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.874 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:08.874 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:08.874 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:08.874 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.874 INFO:tasks.workunit.client.0.trial046.stderr: -278> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 monclient: authenticate success, global_id 6900 2026-02-24T22:17:08.874 INFO:tasks.workunit.client.0.trial046.stderr: -277> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: _renew_subs 2026-02-24T22:17:08.874 INFO:tasks.workunit.client.0.trial046.stderr: -276> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:17:08.874 INFO:tasks.workunit.client.0.trial046.stderr: -275> 2026-02-24T22:17:08.573+0000 7f160b653640 4 set_mon_vals no callback set 2026-02-24T22:17:08.874 INFO:tasks.workunit.client.0.trial046.stderr: -274> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: _renew_subs 2026-02-24T22:17:08.874 INFO:tasks.workunit.client.0.trial046.stderr: -273> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:17:08.874 INFO:tasks.workunit.client.0.trial046.stderr: -272> 2026-02-24T22:17:08.573+0000 7f15e7fff640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:17:08.874 INFO:tasks.workunit.client.0.trial046.stderr: -271> 2026-02-24T22:17:08.573+0000 7f15e7fff640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:17:08.874 INFO:tasks.workunit.client.0.trial046.stderr: -270> 2026-02-24T22:17:08.573+0000 7f15e7fff640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:17:08.874 INFO:tasks.workunit.client.0.trial046.stderr: -269> 2026-02-24T22:17:08.573+0000 7f1609e50640 10 monclient: get_auth_request con 0x7f15dc0706b0 auth_method 0 2026-02-24T22:17:08.875 INFO:tasks.workunit.client.0.trial046.stderr: -268> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: get_monmap_and_config 2026-02-24T22:17:08.875 INFO:tasks.workunit.client.0.trial046.stderr: -267> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: build_initial_monmap 2026-02-24T22:17:08.875 INFO:tasks.workunit.client.0.trial046.stderr: -266> 2026-02-24T22:17:08.573+0000 7f160cb7da80 1 build_initial for_mkfs: 0 2026-02-24T22:17:08.875 INFO:tasks.workunit.client.0.trial046.stderr: -265> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: monmap: 2026-02-24T22:17:08.875 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:17:08.875 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:17:08.875 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:17:08.875 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:17:08.875 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:17:08.875 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.875 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:17:08.875 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:17:08.875 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:17:08.875 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.875 INFO:tasks.workunit.client.0.trial046.stderr: -264> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding auth protocol: cephx 2026-02-24T22:17:08.875 INFO:tasks.workunit.client.0.trial046.stderr: -263> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding auth protocol: cephx 2026-02-24T22:17:08.875 INFO:tasks.workunit.client.0.trial046.stderr: -262> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding auth protocol: cephx 2026-02-24T22:17:08.875 INFO:tasks.workunit.client.0.trial046.stderr: -261> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding con mode: secure 2026-02-24T22:17:08.876 INFO:tasks.workunit.client.0.trial046.stderr: -260> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding con mode: crc 2026-02-24T22:17:08.876 INFO:tasks.workunit.client.0.trial046.stderr: -259> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding con mode: secure 2026-02-24T22:17:08.876 INFO:tasks.workunit.client.0.trial046.stderr: -258> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding con mode: crc 2026-02-24T22:17:08.876 INFO:tasks.workunit.client.0.trial046.stderr: -257> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding con mode: secure 2026-02-24T22:17:08.876 INFO:tasks.workunit.client.0.trial046.stderr: -256> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding con mode: crc 2026-02-24T22:17:08.876 INFO:tasks.workunit.client.0.trial046.stderr: -255> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding con mode: crc 2026-02-24T22:17:08.876 INFO:tasks.workunit.client.0.trial046.stderr: -254> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding con mode: secure 2026-02-24T22:17:08.876 INFO:tasks.workunit.client.0.trial046.stderr: -253> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding con mode: crc 2026-02-24T22:17:08.876 INFO:tasks.workunit.client.0.trial046.stderr: -252> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding con mode: secure 2026-02-24T22:17:08.876 INFO:tasks.workunit.client.0.trial046.stderr: -251> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding con mode: crc 2026-02-24T22:17:08.876 INFO:tasks.workunit.client.0.trial046.stderr: -250> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding con mode: secure 2026-02-24T22:17:08.876 INFO:tasks.workunit.client.0.trial046.stderr: -249> 2026-02-24T22:17:08.573+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.876 INFO:tasks.workunit.client.0.trial046.stderr: -248> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: init 2026-02-24T22:17:08.876 INFO:tasks.workunit.client.0.trial046.stderr: -247> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding auth protocol: cephx 2026-02-24T22:17:08.876 INFO:tasks.workunit.client.0.trial046.stderr: -246> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding auth protocol: cephx 2026-02-24T22:17:08.876 INFO:tasks.workunit.client.0.trial046.stderr: -245> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding auth protocol: cephx 2026-02-24T22:17:08.876 INFO:tasks.workunit.client.0.trial046.stderr: -244> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding con mode: secure 2026-02-24T22:17:08.876 INFO:tasks.workunit.client.0.trial046.stderr: -243> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding con mode: crc 2026-02-24T22:17:08.876 INFO:tasks.workunit.client.0.trial046.stderr: -242> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding con mode: secure 2026-02-24T22:17:08.876 INFO:tasks.workunit.client.0.trial046.stderr: -241> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding con mode: crc 2026-02-24T22:17:08.877 INFO:tasks.workunit.client.0.trial046.stderr: -240> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding con mode: secure 2026-02-24T22:17:08.877 INFO:tasks.workunit.client.0.trial046.stderr: -239> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding con mode: crc 2026-02-24T22:17:08.877 INFO:tasks.workunit.client.0.trial046.stderr: -238> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding con mode: crc 2026-02-24T22:17:08.877 INFO:tasks.workunit.client.0.trial046.stderr: -237> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding con mode: secure 2026-02-24T22:17:08.877 INFO:tasks.workunit.client.0.trial046.stderr: -236> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding con mode: crc 2026-02-24T22:17:08.877 INFO:tasks.workunit.client.0.trial046.stderr: -235> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding con mode: secure 2026-02-24T22:17:08.877 INFO:tasks.workunit.client.0.trial046.stderr: -234> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding con mode: crc 2026-02-24T22:17:08.877 INFO:tasks.workunit.client.0.trial046.stderr: -233> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding con mode: secure 2026-02-24T22:17:08.877 INFO:tasks.workunit.client.0.trial046.stderr: -232> 2026-02-24T22:17:08.573+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.877 INFO:tasks.workunit.client.0.trial046.stderr: -231> 2026-02-24T22:17:08.573+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.877 INFO:tasks.workunit.client.0.trial046.stderr: -230> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command rotate-key cmddesc rotate-key hook 0x7ffc2c48f108 EEXIST 2026-02-24T22:17:08.877 INFO:tasks.workunit.client.0.trial046.stderr: -229> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: _reopen_session rank -1 2026-02-24T22:17:08.877 INFO:tasks.workunit.client.0.trial046.stderr: -228> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T22:17:08.877 INFO:tasks.workunit.client.0.trial046.stderr: -227> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-c con 0x556d114ec010 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:17:08.877 INFO:tasks.workunit.client.0.trial046.stderr: -226> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-b con 0x556d114ef780 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:17:08.877 INFO:tasks.workunit.client.0.trial046.stderr: -225> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-a con 0x556d114ca540 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:17:08.877 INFO:tasks.workunit.client.0.trial046.stderr: -224> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.877 INFO:tasks.workunit.client.0.trial046.stderr: -223> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.877 INFO:tasks.workunit.client.0.trial046.stderr: -222> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.878 INFO:tasks.workunit.client.0.trial046.stderr: -221> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): _renew_subs 2026-02-24T22:17:08.878 INFO:tasks.workunit.client.0.trial046.stderr: -220> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): authenticate will time out at 1798.683485s 2026-02-24T22:17:08.878 INFO:tasks.workunit.client.0.trial046.stderr: -219> 2026-02-24T22:17:08.573+0000 7f160ae52640 10 monclient(hunting): get_auth_request con 0x556d114ca540 auth_method 0 2026-02-24T22:17:08.878 INFO:tasks.workunit.client.0.trial046.stderr: -218> 2026-02-24T22:17:08.573+0000 7f160ae52640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.878 INFO:tasks.workunit.client.0.trial046.stderr: -217> 2026-02-24T22:17:08.573+0000 7f160ae52640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.878 INFO:tasks.workunit.client.0.trial046.stderr: -216> 2026-02-24T22:17:08.573+0000 7f160ae52640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.878 INFO:tasks.workunit.client.0.trial046.stderr: -215> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): get_auth_request con 0x556d114ec010 auth_method 0 2026-02-24T22:17:08.878 INFO:tasks.workunit.client.0.trial046.stderr: -214> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.878 INFO:tasks.workunit.client.0.trial046.stderr: -213> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.878 INFO:tasks.workunit.client.0.trial046.stderr: -212> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.878 INFO:tasks.workunit.client.0.trial046.stderr: -211> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:08.878 INFO:tasks.workunit.client.0.trial046.stderr: -210> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:08.878 INFO:tasks.workunit.client.0.trial046.stderr: -209> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:08.878 INFO:tasks.workunit.client.0.trial046.stderr: -208> 2026-02-24T22:17:08.573+0000 7f160ae52640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:08.878 INFO:tasks.workunit.client.0.trial046.stderr: -207> 2026-02-24T22:17:08.573+0000 7f160ae52640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:08.878 INFO:tasks.workunit.client.0.trial046.stderr: -206> 2026-02-24T22:17:08.573+0000 7f160ae52640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:08.878 INFO:tasks.workunit.client.0.trial046.stderr: -205> 2026-02-24T22:17:08.573+0000 7f1609e50640 10 monclient(hunting): get_auth_request con 0x556d114ef780 auth_method 0 2026-02-24T22:17:08.878 INFO:tasks.workunit.client.0.trial046.stderr: -204> 2026-02-24T22:17:08.573+0000 7f1609e50640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.878 INFO:tasks.workunit.client.0.trial046.stderr: -203> 2026-02-24T22:17:08.573+0000 7f1609e50640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.878 INFO:tasks.workunit.client.0.trial046.stderr: -202> 2026-02-24T22:17:08.573+0000 7f1609e50640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.878 INFO:tasks.workunit.client.0.trial046.stderr: -201> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): handle_auth_done global_id 6464 payload 274 2026-02-24T22:17:08.878 INFO:tasks.workunit.client.0.trial046.stderr: -200> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient: _finish_hunting 0 2026-02-24T22:17:08.879 INFO:tasks.workunit.client.0.trial046.stderr: -199> 2026-02-24T22:17:08.573+0000 7f160a651640 1 monclient: found mon.noname-c 2026-02-24T22:17:08.879 INFO:tasks.workunit.client.0.trial046.stderr: -198> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:17:08.879 INFO:tasks.workunit.client.0.trial046.stderr: -197> 2026-02-24T22:17:08.573+0000 7f15e5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:08.879 INFO:tasks.workunit.client.0.trial046.stderr: -196> 2026-02-24T22:17:08.573+0000 7f15e5ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:17:08.879 INFO:tasks.workunit.client.0.trial046.stderr: -195> 2026-02-24T22:17:08.573+0000 7f15e5ffb640 10 monclient: dump: 2026-02-24T22:17:08.879 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:08.879 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:08.879 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.879 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.879 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:08.879 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.879 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:08.879 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:08.879 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:08.879 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.879 INFO:tasks.workunit.client.0.trial046.stderr: -194> 2026-02-24T22:17:08.573+0000 7f15e5ffb640 10 monclient: _finish_auth 0 2026-02-24T22:17:08.879 INFO:tasks.workunit.client.0.trial046.stderr: -193> 2026-02-24T22:17:08.573+0000 7f15e5ffb640 10 monclient: _check_auth_tickets 2026-02-24T22:17:08.879 INFO:tasks.workunit.client.0.trial046.stderr: -192> 2026-02-24T22:17:08.573+0000 7f15e5ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T22:17:08.879 INFO:tasks.workunit.client.0.trial046.stderr: -191> 2026-02-24T22:17:08.573+0000 7f15e5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:08.879 INFO:tasks.workunit.client.0.trial046.stderr: -190> 2026-02-24T22:17:08.573+0000 7f15e5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:17:08.879 INFO:tasks.workunit.client.0.trial046.stderr: -189> 2026-02-24T22:17:08.573+0000 7f15e5ffb640 10 monclient: dump: 2026-02-24T22:17:08.879 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:08.880 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:08.880 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.880 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.880 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:08.880 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.880 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:08.880 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:08.880 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:08.880 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.880 INFO:tasks.workunit.client.0.trial046.stderr: -188> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 monclient: authenticate success, global_id 6464 2026-02-24T22:17:08.880 INFO:tasks.workunit.client.0.trial046.stderr: -187> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: get_monmap_and_config success 2026-02-24T22:17:08.880 INFO:tasks.workunit.client.0.trial046.stderr: -186> 2026-02-24T22:17:08.573+0000 7f160cb7da80 4 set_mon_vals no callback set 2026-02-24T22:17:08.880 INFO:tasks.workunit.client.0.trial046.stderr: -185> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient: shutdown 2026-02-24T22:17:08.880 INFO:tasks.workunit.client.0.trial046.stderr: -184> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient: build_initial_monmap 2026-02-24T22:17:08.880 INFO:tasks.workunit.client.0.trial046.stderr: -183> 2026-02-24T22:17:08.577+0000 7f160cb7da80 1 build_initial for_mkfs: 0 2026-02-24T22:17:08.880 INFO:tasks.workunit.client.0.trial046.stderr: -182> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient: monmap: 2026-02-24T22:17:08.880 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:17:08.880 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:17:08.880 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:17:08.880 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:17:08.880 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:17:08.880 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.880 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:17:08.880 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:17:08.881 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:17:08.881 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.881 INFO:tasks.workunit.client.0.trial046.stderr: -181> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding auth protocol: cephx 2026-02-24T22:17:08.881 INFO:tasks.workunit.client.0.trial046.stderr: -180> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding auth protocol: cephx 2026-02-24T22:17:08.881 INFO:tasks.workunit.client.0.trial046.stderr: -179> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding auth protocol: cephx 2026-02-24T22:17:08.881 INFO:tasks.workunit.client.0.trial046.stderr: -178> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding con mode: secure 2026-02-24T22:17:08.881 INFO:tasks.workunit.client.0.trial046.stderr: -177> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding con mode: crc 2026-02-24T22:17:08.881 INFO:tasks.workunit.client.0.trial046.stderr: -176> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding con mode: secure 2026-02-24T22:17:08.881 INFO:tasks.workunit.client.0.trial046.stderr: -175> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding con mode: crc 2026-02-24T22:17:08.881 INFO:tasks.workunit.client.0.trial046.stderr: -174> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding con mode: secure 2026-02-24T22:17:08.881 INFO:tasks.workunit.client.0.trial046.stderr: -173> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding con mode: crc 2026-02-24T22:17:08.881 INFO:tasks.workunit.client.0.trial046.stderr: -172> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding con mode: crc 2026-02-24T22:17:08.881 INFO:tasks.workunit.client.0.trial046.stderr: -171> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding con mode: secure 2026-02-24T22:17:08.881 INFO:tasks.workunit.client.0.trial046.stderr: -170> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding con mode: crc 2026-02-24T22:17:08.881 INFO:tasks.workunit.client.0.trial046.stderr: -169> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding con mode: secure 2026-02-24T22:17:08.881 INFO:tasks.workunit.client.0.trial046.stderr: -168> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding con mode: crc 2026-02-24T22:17:08.881 INFO:tasks.workunit.client.0.trial046.stderr: -167> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding con mode: secure 2026-02-24T22:17:08.881 INFO:tasks.workunit.client.0.trial046.stderr: -166> 2026-02-24T22:17:08.577+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.881 INFO:tasks.workunit.client.0.trial046.stderr: -165> 2026-02-24T22:17:08.577+0000 7f160cb7da80 1 librados: starting msgr at 2026-02-24T22:17:08.881 INFO:tasks.workunit.client.0.trial046.stderr: -164> 2026-02-24T22:17:08.577+0000 7f160cb7da80 1 librados: starting objecter 2026-02-24T22:17:08.881 INFO:tasks.workunit.client.0.trial046.stderr: -163> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command objecter_requests cmddesc objecter_requests hook 0x556d1142cca0 EEXIST 2026-02-24T22:17:08.881 INFO:tasks.workunit.client.0.trial046.stderr: -162> 2026-02-24T22:17:08.577+0000 7f160cb7da80 1 librados: setting wanted keys 2026-02-24T22:17:08.882 INFO:tasks.workunit.client.0.trial046.stderr: -161> 2026-02-24T22:17:08.577+0000 7f160cb7da80 1 librados: calling monclient init 2026-02-24T22:17:08.882 INFO:tasks.workunit.client.0.trial046.stderr: -160> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient: init 2026-02-24T22:17:08.882 INFO:tasks.workunit.client.0.trial046.stderr: -159> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding auth protocol: cephx 2026-02-24T22:17:08.882 INFO:tasks.workunit.client.0.trial046.stderr: -158> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding auth protocol: cephx 2026-02-24T22:17:08.882 INFO:tasks.workunit.client.0.trial046.stderr: -157> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding auth protocol: cephx 2026-02-24T22:17:08.882 INFO:tasks.workunit.client.0.trial046.stderr: -156> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding con mode: secure 2026-02-24T22:17:08.882 INFO:tasks.workunit.client.0.trial046.stderr: -155> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding con mode: crc 2026-02-24T22:17:08.882 INFO:tasks.workunit.client.0.trial046.stderr: -154> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding con mode: secure 2026-02-24T22:17:08.882 INFO:tasks.workunit.client.0.trial046.stderr: -153> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding con mode: crc 2026-02-24T22:17:08.882 INFO:tasks.workunit.client.0.trial046.stderr: -152> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding con mode: secure 2026-02-24T22:17:08.882 INFO:tasks.workunit.client.0.trial046.stderr: -151> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding con mode: crc 2026-02-24T22:17:08.882 INFO:tasks.workunit.client.0.trial046.stderr: -150> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding con mode: crc 2026-02-24T22:17:08.882 INFO:tasks.workunit.client.0.trial046.stderr: -149> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding con mode: secure 2026-02-24T22:17:08.882 INFO:tasks.workunit.client.0.trial046.stderr: -148> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding con mode: crc 2026-02-24T22:17:08.882 INFO:tasks.workunit.client.0.trial046.stderr: -147> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding con mode: secure 2026-02-24T22:17:08.882 INFO:tasks.workunit.client.0.trial046.stderr: -146> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding con mode: crc 2026-02-24T22:17:08.882 INFO:tasks.workunit.client.0.trial046.stderr: -145> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding con mode: secure 2026-02-24T22:17:08.882 INFO:tasks.workunit.client.0.trial046.stderr: -144> 2026-02-24T22:17:08.577+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.882 INFO:tasks.workunit.client.0.trial046.stderr: -143> 2026-02-24T22:17:08.577+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.882 INFO:tasks.workunit.client.0.trial046.stderr: -142> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command rotate-key cmddesc rotate-key hook 0x556d114c3f28 EEXIST 2026-02-24T22:17:08.882 INFO:tasks.workunit.client.0.trial046.stderr: -141> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient: _reopen_session rank -1 2026-02-24T22:17:08.882 INFO:tasks.workunit.client.0.trial046.stderr: -140> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T22:17:08.882 INFO:tasks.workunit.client.0.trial046.stderr: -139> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-a con 0x556d114ec010 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:17:08.882 INFO:tasks.workunit.client.0.trial046.stderr: -138> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-b con 0x556d114ef780 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:17:08.883 INFO:tasks.workunit.client.0.trial046.stderr: -137> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-c con 0x556d1151c050 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:17:08.883 INFO:tasks.workunit.client.0.trial046.stderr: -136> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.883 INFO:tasks.workunit.client.0.trial046.stderr: -135> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.883 INFO:tasks.workunit.client.0.trial046.stderr: -134> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.883 INFO:tasks.workunit.client.0.trial046.stderr: -133> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient(hunting): _renew_subs 2026-02-24T22:17:08.883 INFO:tasks.workunit.client.0.trial046.stderr: -132> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient(hunting): authenticate will time out at 1798.684809s 2026-02-24T22:17:08.883 INFO:tasks.workunit.client.0.trial046.stderr: -131> 2026-02-24T22:17:08.577+0000 7f160ae52640 10 monclient(hunting): get_auth_request con 0x556d1151c050 auth_method 0 2026-02-24T22:17:08.883 INFO:tasks.workunit.client.0.trial046.stderr: -130> 2026-02-24T22:17:08.577+0000 7f160ae52640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.883 INFO:tasks.workunit.client.0.trial046.stderr: -129> 2026-02-24T22:17:08.577+0000 7f160ae52640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.883 INFO:tasks.workunit.client.0.trial046.stderr: -128> 2026-02-24T22:17:08.577+0000 7f160ae52640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.883 INFO:tasks.workunit.client.0.trial046.stderr: -127> 2026-02-24T22:17:08.577+0000 7f160a651640 10 monclient(hunting): get_auth_request con 0x556d114ec010 auth_method 0 2026-02-24T22:17:08.883 INFO:tasks.workunit.client.0.trial046.stderr: -126> 2026-02-24T22:17:08.577+0000 7f160a651640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.883 INFO:tasks.workunit.client.0.trial046.stderr: -125> 2026-02-24T22:17:08.577+0000 7f160a651640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.883 INFO:tasks.workunit.client.0.trial046.stderr: -124> 2026-02-24T22:17:08.577+0000 7f160a651640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.883 INFO:tasks.workunit.client.0.trial046.stderr: -123> 2026-02-24T22:17:08.577+0000 7f160ae52640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:08.883 INFO:tasks.workunit.client.0.trial046.stderr: -122> 2026-02-24T22:17:08.577+0000 7f160ae52640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:08.883 INFO:tasks.workunit.client.0.trial046.stderr: -121> 2026-02-24T22:17:08.577+0000 7f160ae52640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:08.883 INFO:tasks.workunit.client.0.trial046.stderr: -120> 2026-02-24T22:17:08.577+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:08.883 INFO:tasks.workunit.client.0.trial046.stderr: -119> 2026-02-24T22:17:08.577+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:08.883 INFO:tasks.workunit.client.0.trial046.stderr: -118> 2026-02-24T22:17:08.577+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:08.883 INFO:tasks.workunit.client.0.trial046.stderr: -117> 2026-02-24T22:17:08.577+0000 7f1609e50640 10 monclient(hunting): get_auth_request con 0x556d114ef780 auth_method 0 2026-02-24T22:17:08.883 INFO:tasks.workunit.client.0.trial046.stderr: -116> 2026-02-24T22:17:08.577+0000 7f1609e50640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.883 INFO:tasks.workunit.client.0.trial046.stderr: -115> 2026-02-24T22:17:08.577+0000 7f1609e50640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.883 INFO:tasks.workunit.client.0.trial046.stderr: -114> 2026-02-24T22:17:08.577+0000 7f1609e50640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.883 INFO:tasks.workunit.client.0.trial046.stderr: -113> 2026-02-24T22:17:08.577+0000 7f160a651640 10 monclient(hunting): handle_auth_done global_id 6906 payload 824 2026-02-24T22:17:08.884 INFO:tasks.workunit.client.0.trial046.stderr: -112> 2026-02-24T22:17:08.577+0000 7f160a651640 10 monclient: _finish_hunting 0 2026-02-24T22:17:08.884 INFO:tasks.workunit.client.0.trial046.stderr: -111> 2026-02-24T22:17:08.577+0000 7f160a651640 1 monclient: found mon.noname-a 2026-02-24T22:17:08.884 INFO:tasks.workunit.client.0.trial046.stderr: -110> 2026-02-24T22:17:08.577+0000 7f160a651640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:17:08.884 INFO:tasks.workunit.client.0.trial046.stderr: -109> 2026-02-24T22:17:08.577+0000 7f15d3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:08.884 INFO:tasks.workunit.client.0.trial046.stderr: -108> 2026-02-24T22:17:08.577+0000 7f15d3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:17:08.884 INFO:tasks.workunit.client.0.trial046.stderr: -107> 2026-02-24T22:17:08.577+0000 7f15d3fff640 10 monclient: dump: 2026-02-24T22:17:08.884 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:08.884 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:08.884 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.884 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.884 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:08.884 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.884 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:08.884 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:08.884 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:08.884 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.884 INFO:tasks.workunit.client.0.trial046.stderr: -106> 2026-02-24T22:17:08.577+0000 7f15d3fff640 10 monclient: _finish_auth 0 2026-02-24T22:17:08.884 INFO:tasks.workunit.client.0.trial046.stderr: -105> 2026-02-24T22:17:08.577+0000 7f15d3fff640 10 monclient: _check_auth_tickets 2026-02-24T22:17:08.884 INFO:tasks.workunit.client.0.trial046.stderr: -104> 2026-02-24T22:17:08.577+0000 7f15d3fff640 10 monclient: handle_config config(0 keys) 2026-02-24T22:17:08.884 INFO:tasks.workunit.client.0.trial046.stderr: -103> 2026-02-24T22:17:08.577+0000 7f15d3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:08.884 INFO:tasks.workunit.client.0.trial046.stderr: -102> 2026-02-24T22:17:08.577+0000 7f15d3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:17:08.884 INFO:tasks.workunit.client.0.trial046.stderr: -101> 2026-02-24T22:17:08.577+0000 7f15d3fff640 10 monclient: dump: 2026-02-24T22:17:08.884 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:08.884 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:08.884 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.885 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.885 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:08.885 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.885 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:08.885 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:08.885 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:08.885 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.885 INFO:tasks.workunit.client.0.trial046.stderr: -100> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 monclient: authenticate success, global_id 6906 2026-02-24T22:17:08.885 INFO:tasks.workunit.client.0.trial046.stderr: -99> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient: _renew_subs 2026-02-24T22:17:08.885 INFO:tasks.workunit.client.0.trial046.stderr: -98> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:17:08.885 INFO:tasks.workunit.client.0.trial046.stderr: -97> 2026-02-24T22:17:08.577+0000 7f15e5ffb640 4 set_mon_vals no callback set 2026-02-24T22:17:08.885 INFO:tasks.workunit.client.0.trial046.stderr: -96> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient: _renew_subs 2026-02-24T22:17:08.885 INFO:tasks.workunit.client.0.trial046.stderr: -95> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:17:08.885 INFO:tasks.workunit.client.0.trial046.stderr: -94> 2026-02-24T22:17:08.577+0000 7f160cb7da80 1 librados: init done 2026-02-24T22:17:08.885 INFO:tasks.workunit.client.0.trial046.stderr: -93> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command cr dump hook 0x556d1142c6b8 2026-02-24T22:17:08.885 INFO:tasks.workunit.client.0.trial046.stderr: -92> 2026-02-24T22:17:08.577+0000 7f160cb7da80 4 Realm: () 2026-02-24T22:17:08.885 INFO:tasks.workunit.client.0.trial046.stderr: -91> 2026-02-24T22:17:08.577+0000 7f160cb7da80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:08.885 INFO:tasks.workunit.client.0.trial046.stderr: -90> 2026-02-24T22:17:08.577+0000 7f160cb7da80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:08.885 INFO:tasks.workunit.client.0.trial046.stderr: -89> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:08.885 INFO:tasks.workunit.client.0.trial046.stderr: -88> 2026-02-24T22:17:08.577+0000 7f160cb7da80 20 zonegroup default 2026-02-24T22:17:08.885 INFO:tasks.workunit.client.0.trial046.stderr: -87> 2026-02-24T22:17:08.577+0000 7f1609e50640 10 monclient: get_auth_request con 0x556d1158c620 auth_method 0 2026-02-24T22:17:08.885 INFO:tasks.workunit.client.0.trial046.stderr: -86> 2026-02-24T22:17:08.577+0000 7f15d3fff640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:17:08.885 INFO:tasks.workunit.client.0.trial046.stderr: -85> 2026-02-24T22:17:08.577+0000 7f15d3fff640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:17:08.885 INFO:tasks.workunit.client.0.trial046.stderr: -84> 2026-02-24T22:17:08.577+0000 7f15d3fff640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:17:08.886 INFO:tasks.workunit.client.0.trial046.stderr: -83> 2026-02-24T22:17:08.577+0000 7f160ae52640 10 monclient: get_auth_request con 0x7f15cc0707a0 auth_method 0 2026-02-24T22:17:08.886 INFO:tasks.workunit.client.0.trial046.stderr: -82> 2026-02-24T22:17:08.577+0000 7f160cb7da80 20 rados->read ofs=0 len=0 2026-02-24T22:17:08.886 INFO:tasks.workunit.client.0.trial046.stderr: -81> 2026-02-24T22:17:08.577+0000 7f160a651640 10 monclient: get_auth_request con 0x556d115af940 auth_method 0 2026-02-24T22:17:08.886 INFO:tasks.workunit.client.0.trial046.stderr: -80> 2026-02-24T22:17:08.577+0000 7f160cb7da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:08.886 INFO:tasks.workunit.client.0.trial046.stderr: -79> 2026-02-24T22:17:08.577+0000 7f160cb7da80 20 rados->read ofs=0 len=0 2026-02-24T22:17:08.886 INFO:tasks.workunit.client.0.trial046.stderr: -78> 2026-02-24T22:17:08.577+0000 7f1609e50640 10 monclient: get_auth_request con 0x556d115d0fe0 auth_method 0 2026-02-24T22:17:08.886 INFO:tasks.workunit.client.0.trial046.stderr: -77> 2026-02-24T22:17:08.581+0000 7f160cb7da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:08.886 INFO:tasks.workunit.client.0.trial046.stderr: -76> 2026-02-24T22:17:08.581+0000 7f160cb7da80 20 started sync module instance, tier type = 2026-02-24T22:17:08.886 INFO:tasks.workunit.client.0.trial046.stderr: -75> 2026-02-24T22:17:08.581+0000 7f160cb7da80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:08.886 INFO:tasks.workunit.client.0.trial046.stderr: -74> 2026-02-24T22:17:08.581+0000 7f160ae52640 10 monclient: get_auth_request con 0x556d11632490 auth_method 0 2026-02-24T22:17:08.886 INFO:tasks.workunit.client.0.trial046.stderr: -73> 2026-02-24T22:17:08.581+0000 7f160ae52640 10 monclient: get_auth_request con 0x556d11739bd0 auth_method 0 2026-02-24T22:17:08.886 INFO:tasks.workunit.client.0.trial046.stderr: -72> 2026-02-24T22:17:08.581+0000 7f160a651640 10 monclient: get_auth_request con 0x556d11694c90 auth_method 0 2026-02-24T22:17:08.886 INFO:tasks.workunit.client.0.trial046.stderr: -71> 2026-02-24T22:17:08.581+0000 7f1609e50640 10 monclient: get_auth_request con 0x556d116f7a90 auth_method 0 2026-02-24T22:17:08.886 INFO:tasks.workunit.client.0.trial046.stderr: -70> 2026-02-24T22:17:08.581+0000 7f160a651640 10 monclient: get_auth_request con 0x556d1179cbd0 auth_method 0 2026-02-24T22:17:08.886 INFO:tasks.workunit.client.0.trial046.stderr: -69> 2026-02-24T22:17:08.581+0000 7f160a651640 0 -- 10.20.193.46:0/1925587481 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x556d1179cbd0 msgr2=0x556d117bd030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:17:08.886 INFO:tasks.workunit.client.0.trial046.stderr: -68> 2026-02-24T22:17:08.581+0000 7f15d3fff640 1 client.6906.objecter ms_handle_reset 0x556d1179cbd0 session 0x556d1179c4c0 osd.6 2026-02-24T22:17:08.886 INFO:tasks.workunit.client.0.trial046.stderr: -67> 2026-02-24T22:17:08.581+0000 7f15d3fff640 10 monclient: _renew_subs 2026-02-24T22:17:08.886 INFO:tasks.workunit.client.0.trial046.stderr: -66> 2026-02-24T22:17:08.581+0000 7f15d3fff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:17:08.886 INFO:tasks.workunit.client.0.trial046.stderr: -65> 2026-02-24T22:17:08.581+0000 7f160a651640 10 monclient: get_auth_request con 0x7f15cc09d450 auth_method 0 2026-02-24T22:17:08.886 INFO:tasks.workunit.client.0.trial046.stderr: -64> 2026-02-24T22:17:08.581+0000 7f160cb7da80 20 add_watcher() i=1 2026-02-24T22:17:08.886 INFO:tasks.workunit.client.0.trial046.stderr: -63> 2026-02-24T22:17:08.581+0000 7f160cb7da80 20 add_watcher() i=6 2026-02-24T22:17:08.886 INFO:tasks.workunit.client.0.trial046.stderr: -62> 2026-02-24T22:17:08.581+0000 7f160cb7da80 20 add_watcher() i=5 2026-02-24T22:17:08.886 INFO:tasks.workunit.client.0.trial046.stderr: -61> 2026-02-24T22:17:08.581+0000 7f160cb7da80 20 add_watcher() i=3 2026-02-24T22:17:08.886 INFO:tasks.workunit.client.0.trial046.stderr: -60> 2026-02-24T22:17:08.581+0000 7f160cb7da80 20 add_watcher() i=0 2026-02-24T22:17:08.886 INFO:tasks.workunit.client.0.trial046.stderr: -59> 2026-02-24T22:17:08.581+0000 7f160cb7da80 20 add_watcher() i=2 2026-02-24T22:17:08.886 INFO:tasks.workunit.client.0.trial046.stderr: -58> 2026-02-24T22:17:08.581+0000 7f160cb7da80 20 add_watcher() i=4 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -57> 2026-02-24T22:17:08.581+0000 7f160cb7da80 20 add_watcher() i=7 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -56> 2026-02-24T22:17:08.581+0000 7f160cb7da80 2 all 8 watchers are set, enabling cache 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -55> 2026-02-24T22:17:08.581+0000 7f1609e50640 10 monclient: get_auth_request con 0x7f152c008af0 auth_method 0 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -54> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding auth protocol: cephx 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -53> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding auth protocol: cephx 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -52> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding auth protocol: cephx 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -51> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding con mode: secure 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -50> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding con mode: crc 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -49> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding con mode: secure 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -48> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding con mode: crc 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -47> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding con mode: secure 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -46> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding con mode: crc 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -45> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding con mode: crc 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -44> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding con mode: secure 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -43> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding con mode: crc 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -42> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding con mode: secure 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -41> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding con mode: crc 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -40> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding con mode: secure 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -39> 2026-02-24T22:17:08.585+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -38> 2026-02-24T22:17:08.585+0000 7f160cb7da80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -37> 2026-02-24T22:17:08.585+0000 7f160cb7da80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -36> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command cache list hook 0x556d114efb90 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -35> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command cache inspect hook 0x556d114efb90 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -34> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command cache erase hook 0x556d114efb90 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -33> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command cache zap hook 0x556d114efb90 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -32> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 note: GC not initialized 2026-02-24T22:17:08.887 INFO:tasks.workunit.client.0.trial046.stderr: -31> 2026-02-24T22:17:08.585+0000 7f15237fe640 20 reqs_thread_entry: start 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -30> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command sync trace show hook 0x7f15fc0062d0 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -29> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command sync trace history hook 0x7f15fc0062d0 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -28> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command sync trace active hook 0x7f15fc0062d0 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -27> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command sync trace active_short hook 0x7f15fc0062d0 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -26> 2026-02-24T22:17:08.585+0000 7f160ae52640 0 -- 10.20.193.46:0/3793593507 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x556d117b38d0 msgr2=0x556d117b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -25> 2026-02-24T22:17:08.585+0000 7f160ae52640 10 monclient: get_auth_request con 0x556d117b38d0 auth_method 0 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -24> 2026-02-24T22:17:08.789+0000 7f160ae52640 10 monclient: get_auth_request con 0x556d117b38d0 auth_method 0 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -23> 2026-02-24T22:17:08.789+0000 7f160a651640 10 monclient: get_auth_request con 0x556d117b51f0 auth_method 0 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -22> 2026-02-24T22:17:08.793+0000 7f1609e50640 10 monclient: get_auth_request con 0x556d117b4df0 auth_method 0 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -21> 2026-02-24T22:17:08.793+0000 7f160ae52640 10 monclient: get_auth_request con 0x556d116c8520 auth_method 0 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -20> 2026-02-24T22:17:08.797+0000 7f160a651640 10 monclient: get_auth_request con 0x556d117b9eb0 auth_method 0 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -19> 2026-02-24T22:17:08.801+0000 7f1609e50640 10 monclient: get_auth_request con 0x556d117bcd50 auth_method 0 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -18> 2026-02-24T22:17:08.825+0000 7f160cb7da80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -17> 2026-02-24T22:17:08.825+0000 7f160cb7da80 20 Filter name: none 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -16> 2026-02-24T22:17:08.825+0000 7f1521ffb640 20 reqs_thread_entry: start 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -15> 2026-02-24T22:17:08.825+0000 7f160cb7da80 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -14> 2026-02-24T22:17:08.825+0000 7f160cb7da80 20 rados->read ofs=0 len=0 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -13> 2026-02-24T22:17:08.825+0000 7f160cb7da80 20 rados_obj.operate() r=0 bl.length=222 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -12> 2026-02-24T22:17:08.825+0000 7f160cb7da80 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -11> 2026-02-24T22:17:08.825+0000 7f160cb7da80 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -10> 2026-02-24T22:17:08.825+0000 7f160cb7da80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 : miss 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -9> 2026-02-24T22:17:08.825+0000 7f160cb7da80 20 rados->read ofs=0 len=0 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -8> 2026-02-24T22:17:08.825+0000 7f160cb7da80 20 rados_obj.operate() r=0 bl.length=452 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -7> 2026-02-24T22:17:08.825+0000 7f160cb7da80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 info.flags=0x17 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -6> 2026-02-24T22:17:08.825+0000 7f160cb7da80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 to cache LRU end 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -5> 2026-02-24T22:17:08.825+0000 7f160cb7da80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:17:08.888 INFO:tasks.workunit.client.0.trial046.stderr: -4> 2026-02-24T22:17:08.825+0000 7f160cb7da80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr: -3> 2026-02-24T22:17:08.825+0000 7f160cb7da80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr: -2> 2026-02-24T22:17:08.825+0000 7f160ae52640 10 monclient: get_auth_request con 0x556d116d3930 auth_method 0 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr: -1> 2026-02-24T22:17:08.829+0000 7f160cb7da80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f160cb7da80 time 2026-02-24T22:17:08.833071+0000 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f160ffbfeb1] 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x1713e63) [0x556cef4f7e63] 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x556ceea55202] 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f160f2bfd90] 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr: 0> 2026-02-24T22:17:08.829+0000 7f160cb7da80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f160cb7da80 thread_name:radosgw-admin 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f160f2d8520] 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f160ffbff10] 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x1713e63) [0x556cef4f7e63] 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x556ceea55202] 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:17:08.889 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f160f2bfd90] 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr:--- logging levels --- 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 none 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 lockdep 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 context 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 crush 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_balancer 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_locker 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log_expire 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_migrator 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 3/ 5 mds_quiesce 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 buffer 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 timer 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 filer 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 striper 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 objecter 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rados 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_mirror 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_replay 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_pwl 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 journaler 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objectcacher 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 immutable_obj_cache 2026-02-24T22:17:08.890 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 client 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 osd 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 optracker 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objclass 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 filestore 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 journal 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 ms 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mon 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 0/10 monc 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 paxos 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 tp 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 auth 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 crypto 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 finisher 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 reserver 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 heartbeatmap 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 perfcounter 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_sync 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_datacache 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_access 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dbstore 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_flight 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_lifecycle 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_restore 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw_notification 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_bucket_logging 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 javaclient 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 asok 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 throttle 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 refs 2026-02-24T22:17:08.891 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 compressor 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore_compression 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluefs 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 bdev 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 kstore 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 4/ 5 rocksdb 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 fuse 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 2/ 5 mgr 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mgrc 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 dpdk 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 eventtrace 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 prioritycache 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 test 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephfs_mirror 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephsqlite 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 crimson_interrupt 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_onode 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_odata 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_omap 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_tm 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_t 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cleaner 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_epm 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_lba 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_fixedkv_tree 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cache 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_journal 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_device 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_backref 2026-02-24T22:17:08.892 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 alienstore 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mclock 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dedup 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cyanstore 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 ceph_exporter 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 memstore 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 trace 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 ceph_dedup 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: -2/-2 (syslog threshold) 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: 99/99 (stderr threshold) 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr:--- pthread ID / name mapping for recent threads --- 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: 7f1521ffb640 / http_manager 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: 7f15237fe640 / http_manager 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: 7f15d3fff640 / ms_dispatch 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: 7f15e5ffb640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: 7f15e7fff640 / ms_dispatch 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: 7f15eeffd640 / ms_dispatch 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: 7f15f8ff9640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: 7f15f97fa640 / admin_socket 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: 7f160964f640 / ms_dispatch 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: 7f1609e50640 / msgr-worker-2 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: 7f160a651640 / msgr-worker-1 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: 7f160ae52640 / msgr-worker-0 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: 7f160b653640 / io_context_pool 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: 7f160cb7da80 / radosgw-admin 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: max_recent 500 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: max_new 1000 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: log_file /var/log/ceph/ceph-client.admin.43909.log 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr:--- end dump of recent events --- 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr:--- begin dump of recent events --- 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: -499> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f0f0) adding con mode: secure 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: -498> 2026-02-24T22:17:08.565+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: -497> 2026-02-24T22:17:08.565+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.893 INFO:tasks.workunit.client.0.trial046.stderr: -496> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command rotate-key hook 0x7ffc2c48f238 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -495> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient: _reopen_session rank -1 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -494> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -493> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-a con 0x556d111d7fe0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -492> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-b con 0x556d11280b00 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -491> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-c con 0x556d11283cf0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -490> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -489> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -488> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -487> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): _renew_subs 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -486> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): authenticate will time out at 1798.674511s 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -485> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient(hunting): get_auth_request con 0x556d111d7fe0 auth_method 0 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -484> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -483> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -482> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -481> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -480> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -479> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -478> 2026-02-24T22:17:08.565+0000 7f160ae52640 10 monclient(hunting): get_auth_request con 0x556d11283cf0 auth_method 0 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -477> 2026-02-24T22:17:08.565+0000 7f160ae52640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -476> 2026-02-24T22:17:08.565+0000 7f160ae52640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -475> 2026-02-24T22:17:08.565+0000 7f160ae52640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -474> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient(hunting): handle_auth_done global_id 6894 payload 274 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -473> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient: _finish_hunting 0 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -472> 2026-02-24T22:17:08.565+0000 7f160a651640 1 monclient: found mon.noname-a 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -471> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -470> 2026-02-24T22:17:08.565+0000 7f15f8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -469> 2026-02-24T22:17:08.565+0000 7f15f8ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr: -468> 2026-02-24T22:17:08.565+0000 7f15f8ff9640 10 monclient: dump: 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:08.894 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr: -467> 2026-02-24T22:17:08.565+0000 7f15f8ff9640 10 monclient: _finish_auth 0 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr: -466> 2026-02-24T22:17:08.565+0000 7f15f8ff9640 10 monclient: _check_auth_tickets 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr: -465> 2026-02-24T22:17:08.565+0000 7f15f8ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr: -464> 2026-02-24T22:17:08.565+0000 7f15f8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr: -463> 2026-02-24T22:17:08.565+0000 7f15f8ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr: -462> 2026-02-24T22:17:08.565+0000 7f15f8ff9640 10 monclient: dump: 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr: -461> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 monclient: authenticate success, global_id 6894 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr: -460> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient: get_monmap_and_config success 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr: -459> 2026-02-24T22:17:08.565+0000 7f160cb7da80 4 set_mon_vals no callback set 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr: -458> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient: shutdown 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr: -457> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 asok(0x556d11149f00) unregister_commands rotate-key 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr: -456> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient: build_initial_monmap 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr: -455> 2026-02-24T22:17:08.565+0000 7f160cb7da80 1 build_initial for_mkfs: 0 2026-02-24T22:17:08.895 INFO:tasks.workunit.client.0.trial046.stderr: -454> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient: monmap: 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr: -453> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding auth protocol: cephx 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr: -452> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding auth protocol: cephx 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr: -451> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding auth protocol: cephx 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr: -450> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding con mode: secure 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr: -449> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding con mode: crc 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr: -448> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding con mode: secure 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr: -447> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding con mode: crc 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr: -446> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding con mode: secure 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr: -445> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding con mode: crc 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr: -444> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding con mode: crc 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr: -443> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding con mode: secure 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr: -442> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding con mode: crc 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr: -441> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding con mode: secure 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr: -440> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding con mode: crc 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr: -439> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d1124e3a0) adding con mode: secure 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr: -438> 2026-02-24T22:17:08.565+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr: -437> 2026-02-24T22:17:08.565+0000 7f160cb7da80 1 librados: starting msgr at 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr: -436> 2026-02-24T22:17:08.565+0000 7f160cb7da80 1 librados: starting objecter 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr: -435> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command objecter_requests hook 0x556d1124bbf0 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr: -434> 2026-02-24T22:17:08.565+0000 7f160cb7da80 1 librados: setting wanted keys 2026-02-24T22:17:08.896 INFO:tasks.workunit.client.0.trial046.stderr: -433> 2026-02-24T22:17:08.565+0000 7f160cb7da80 1 librados: calling monclient init 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -432> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient: init 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -431> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding auth protocol: cephx 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -430> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding auth protocol: cephx 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -429> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding auth protocol: cephx 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -428> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding con mode: secure 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -427> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding con mode: crc 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -426> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding con mode: secure 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -425> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding con mode: crc 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -424> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding con mode: secure 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -423> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding con mode: crc 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -422> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding con mode: crc 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -421> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding con mode: secure 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -420> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding con mode: crc 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -419> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding con mode: secure 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -418> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding con mode: crc 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -417> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 AuthRegistry(0x556d11249370) adding con mode: secure 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -416> 2026-02-24T22:17:08.565+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -415> 2026-02-24T22:17:08.565+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -414> 2026-02-24T22:17:08.565+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command rotate-key hook 0x556d112494b8 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -413> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient: _reopen_session rank -1 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -412> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -411> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-a con 0x556d111d7fe0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -410> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-c con 0x556d11280b00 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -409> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-b con 0x556d11283cf0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -408> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -407> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -406> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -405> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): _renew_subs 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -404> 2026-02-24T22:17:08.565+0000 7f160cb7da80 10 monclient(hunting): authenticate will time out at 1798.675850s 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -403> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient(hunting): get_auth_request con 0x556d111d7fe0 auth_method 0 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -402> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -401> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.897 INFO:tasks.workunit.client.0.trial046.stderr: -400> 2026-02-24T22:17:08.565+0000 7f160a651640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr: -399> 2026-02-24T22:17:08.565+0000 7f1609e50640 10 monclient(hunting): get_auth_request con 0x556d11280b00 auth_method 0 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr: -398> 2026-02-24T22:17:08.565+0000 7f1609e50640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr: -397> 2026-02-24T22:17:08.569+0000 7f1609e50640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr: -396> 2026-02-24T22:17:08.569+0000 7f1609e50640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr: -395> 2026-02-24T22:17:08.569+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr: -394> 2026-02-24T22:17:08.569+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr: -393> 2026-02-24T22:17:08.569+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr: -392> 2026-02-24T22:17:08.569+0000 7f160ae52640 10 monclient(hunting): get_auth_request con 0x556d11283cf0 auth_method 0 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr: -391> 2026-02-24T22:17:08.569+0000 7f160ae52640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr: -390> 2026-02-24T22:17:08.569+0000 7f160ae52640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr: -389> 2026-02-24T22:17:08.569+0000 7f160ae52640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr: -388> 2026-02-24T22:17:08.569+0000 7f1609e50640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr: -387> 2026-02-24T22:17:08.569+0000 7f1609e50640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr: -386> 2026-02-24T22:17:08.569+0000 7f1609e50640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr: -385> 2026-02-24T22:17:08.569+0000 7f160a651640 10 monclient(hunting): handle_auth_done global_id 6897 payload 824 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr: -384> 2026-02-24T22:17:08.569+0000 7f160a651640 10 monclient: _finish_hunting 0 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr: -383> 2026-02-24T22:17:08.569+0000 7f160a651640 1 monclient: found mon.noname-a 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr: -382> 2026-02-24T22:17:08.569+0000 7f160a651640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr: -381> 2026-02-24T22:17:08.569+0000 7f15eeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr: -380> 2026-02-24T22:17:08.569+0000 7f15eeffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr: -379> 2026-02-24T22:17:08.569+0000 7f15eeffd640 10 monclient: dump: 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr: -378> 2026-02-24T22:17:08.569+0000 7f15eeffd640 10 monclient: _finish_auth 0 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr: -377> 2026-02-24T22:17:08.569+0000 7f15eeffd640 10 monclient: _check_auth_tickets 2026-02-24T22:17:08.898 INFO:tasks.workunit.client.0.trial046.stderr: -376> 2026-02-24T22:17:08.569+0000 7f15eeffd640 10 monclient: handle_config config(0 keys) 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr: -375> 2026-02-24T22:17:08.569+0000 7f15eeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr: -374> 2026-02-24T22:17:08.569+0000 7f15eeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr: -373> 2026-02-24T22:17:08.569+0000 7f15eeffd640 10 monclient: dump: 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr: -372> 2026-02-24T22:17:08.569+0000 7f160cb7da80 5 monclient: authenticate success, global_id 6897 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr: -371> 2026-02-24T22:17:08.569+0000 7f160cb7da80 10 monclient: _renew_subs 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr: -370> 2026-02-24T22:17:08.569+0000 7f160cb7da80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr: -369> 2026-02-24T22:17:08.569+0000 7f15f8ff9640 4 set_mon_vals no callback set 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr: -368> 2026-02-24T22:17:08.569+0000 7f160cb7da80 10 monclient: _renew_subs 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr: -367> 2026-02-24T22:17:08.569+0000 7f160cb7da80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr: -366> 2026-02-24T22:17:08.569+0000 7f160cb7da80 1 librados: init done 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr: -365> 2026-02-24T22:17:08.569+0000 7f15eeffd640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr: -364> 2026-02-24T22:17:08.569+0000 7f15eeffd640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr: -363> 2026-02-24T22:17:08.569+0000 7f15eeffd640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr: -362> 2026-02-24T22:17:08.569+0000 7f1609e50640 10 monclient: get_auth_request con 0x7f15e80707b0 auth_method 0 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr: -361> 2026-02-24T22:17:08.569+0000 7f160ae52640 10 monclient: get_auth_request con 0x556d11276210 auth_method 0 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr: -360> 2026-02-24T22:17:08.569+0000 7f160a651640 10 monclient: get_auth_request con 0x556d112d7a20 auth_method 0 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr: -359> 2026-02-24T22:17:08.573+0000 7f1609e50640 10 monclient: get_auth_request con 0x556d112d9100 auth_method 0 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr: -358> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: build_initial_monmap 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr: -357> 2026-02-24T22:17:08.573+0000 7f160cb7da80 1 build_initial for_mkfs: 0 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr: -356> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: monmap: 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:17:08.899 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -355> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding auth protocol: cephx 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -354> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding auth protocol: cephx 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -353> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding auth protocol: cephx 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -352> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding con mode: secure 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -351> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding con mode: crc 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -350> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding con mode: secure 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -349> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding con mode: crc 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -348> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding con mode: secure 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -347> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding con mode: crc 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -346> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding con mode: crc 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -345> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding con mode: secure 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -344> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding con mode: crc 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -343> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding con mode: secure 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -342> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding con mode: crc 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -341> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112dbcd0) adding con mode: secure 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -340> 2026-02-24T22:17:08.573+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -339> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command objecter_requests cmddesc objecter_requests hook 0x556d112e01b0 EEXIST 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -338> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: init 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -337> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding auth protocol: cephx 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -336> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding auth protocol: cephx 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -335> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding auth protocol: cephx 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -334> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding con mode: secure 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -333> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding con mode: crc 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -332> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding con mode: secure 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -331> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding con mode: crc 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -330> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding con mode: secure 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -329> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding con mode: crc 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -328> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding con mode: crc 2026-02-24T22:17:08.900 INFO:tasks.workunit.client.0.trial046.stderr: -327> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding con mode: secure 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -326> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding con mode: crc 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -325> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding con mode: secure 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -324> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding con mode: crc 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -323> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d112d9cb8) adding con mode: secure 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -322> 2026-02-24T22:17:08.573+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -321> 2026-02-24T22:17:08.573+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -320> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command rotate-key cmddesc rotate-key hook 0x556d112d9e00 EEXIST 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -319> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: _reopen_session rank -1 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -318> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -317> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-a con 0x556d11433d90 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -316> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-c con 0x556d11431640 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -315> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-b con 0x556d1142ade0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -314> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -313> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -312> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -311> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): _renew_subs 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -310> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): authenticate will time out at 1798.681241s 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -309> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): get_auth_request con 0x556d11433d90 auth_method 0 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -308> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -307> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -306> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -305> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -304> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -303> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -302> 2026-02-24T22:17:08.573+0000 7f1609e50640 10 monclient(hunting): get_auth_request con 0x556d11431640 auth_method 0 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -301> 2026-02-24T22:17:08.573+0000 7f1609e50640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -300> 2026-02-24T22:17:08.573+0000 7f1609e50640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -299> 2026-02-24T22:17:08.573+0000 7f1609e50640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -298> 2026-02-24T22:17:08.573+0000 7f1609e50640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -297> 2026-02-24T22:17:08.573+0000 7f1609e50640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -296> 2026-02-24T22:17:08.573+0000 7f1609e50640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -295> 2026-02-24T22:17:08.573+0000 7f160ae52640 10 monclient(hunting): get_auth_request con 0x556d1142ade0 auth_method 0 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -294> 2026-02-24T22:17:08.573+0000 7f160ae52640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -293> 2026-02-24T22:17:08.573+0000 7f160ae52640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.901 INFO:tasks.workunit.client.0.trial046.stderr: -292> 2026-02-24T22:17:08.573+0000 7f160ae52640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr: -291> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): handle_auth_done global_id 6900 payload 824 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr: -290> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient: _finish_hunting 0 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr: -289> 2026-02-24T22:17:08.573+0000 7f160a651640 1 monclient: found mon.noname-a 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr: -288> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr: -287> 2026-02-24T22:17:08.573+0000 7f15e7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr: -286> 2026-02-24T22:17:08.573+0000 7f15e7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr: -285> 2026-02-24T22:17:08.573+0000 7f15e7fff640 10 monclient: dump: 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr: -284> 2026-02-24T22:17:08.573+0000 7f15e7fff640 10 monclient: _finish_auth 0 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr: -283> 2026-02-24T22:17:08.573+0000 7f15e7fff640 10 monclient: _check_auth_tickets 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr: -282> 2026-02-24T22:17:08.573+0000 7f15e7fff640 10 monclient: handle_config config(0 keys) 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr: -281> 2026-02-24T22:17:08.573+0000 7f15e7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr: -280> 2026-02-24T22:17:08.573+0000 7f15e7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr: -279> 2026-02-24T22:17:08.573+0000 7f15e7fff640 10 monclient: dump: 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr: -278> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 monclient: authenticate success, global_id 6900 2026-02-24T22:17:08.902 INFO:tasks.workunit.client.0.trial046.stderr: -277> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: _renew_subs 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -276> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -275> 2026-02-24T22:17:08.573+0000 7f160b653640 4 set_mon_vals no callback set 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -274> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: _renew_subs 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -273> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -272> 2026-02-24T22:17:08.573+0000 7f15e7fff640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -271> 2026-02-24T22:17:08.573+0000 7f15e7fff640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -270> 2026-02-24T22:17:08.573+0000 7f15e7fff640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -269> 2026-02-24T22:17:08.573+0000 7f1609e50640 10 monclient: get_auth_request con 0x7f15dc0706b0 auth_method 0 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -268> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: get_monmap_and_config 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -267> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: build_initial_monmap 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -266> 2026-02-24T22:17:08.573+0000 7f160cb7da80 1 build_initial for_mkfs: 0 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -265> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: monmap: 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -264> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding auth protocol: cephx 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -263> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding auth protocol: cephx 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -262> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding auth protocol: cephx 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -261> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding con mode: secure 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -260> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding con mode: crc 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -259> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding con mode: secure 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -258> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding con mode: crc 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -257> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding con mode: secure 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -256> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding con mode: crc 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -255> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding con mode: crc 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -254> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding con mode: secure 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -253> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding con mode: crc 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -252> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding con mode: secure 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -251> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding con mode: crc 2026-02-24T22:17:08.903 INFO:tasks.workunit.client.0.trial046.stderr: -250> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4fc0) adding con mode: secure 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -249> 2026-02-24T22:17:08.573+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -248> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: init 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -247> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding auth protocol: cephx 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -246> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding auth protocol: cephx 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -245> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding auth protocol: cephx 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -244> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding con mode: secure 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -243> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding con mode: crc 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -242> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding con mode: secure 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -241> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding con mode: crc 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -240> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding con mode: secure 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -239> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding con mode: crc 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -238> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding con mode: crc 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -237> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding con mode: secure 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -236> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding con mode: crc 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -235> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding con mode: secure 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -234> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding con mode: crc 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -233> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48efc0) adding con mode: secure 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -232> 2026-02-24T22:17:08.573+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -231> 2026-02-24T22:17:08.573+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -230> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command rotate-key cmddesc rotate-key hook 0x7ffc2c48f108 EEXIST 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -229> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: _reopen_session rank -1 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -228> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -227> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-c con 0x556d114ec010 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -226> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-b con 0x556d114ef780 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -225> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-a con 0x556d114ca540 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -224> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -223> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -222> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -221> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): _renew_subs 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -220> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient(hunting): authenticate will time out at 1798.683485s 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -219> 2026-02-24T22:17:08.573+0000 7f160ae52640 10 monclient(hunting): get_auth_request con 0x556d114ca540 auth_method 0 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -218> 2026-02-24T22:17:08.573+0000 7f160ae52640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -217> 2026-02-24T22:17:08.573+0000 7f160ae52640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -216> 2026-02-24T22:17:08.573+0000 7f160ae52640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -215> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): get_auth_request con 0x556d114ec010 auth_method 0 2026-02-24T22:17:08.904 INFO:tasks.workunit.client.0.trial046.stderr: -214> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr: -213> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr: -212> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr: -211> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr: -210> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr: -209> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr: -208> 2026-02-24T22:17:08.573+0000 7f160ae52640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr: -207> 2026-02-24T22:17:08.573+0000 7f160ae52640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr: -206> 2026-02-24T22:17:08.573+0000 7f160ae52640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr: -205> 2026-02-24T22:17:08.573+0000 7f1609e50640 10 monclient(hunting): get_auth_request con 0x556d114ef780 auth_method 0 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr: -204> 2026-02-24T22:17:08.573+0000 7f1609e50640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr: -203> 2026-02-24T22:17:08.573+0000 7f1609e50640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr: -202> 2026-02-24T22:17:08.573+0000 7f1609e50640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr: -201> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient(hunting): handle_auth_done global_id 6464 payload 274 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr: -200> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient: _finish_hunting 0 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr: -199> 2026-02-24T22:17:08.573+0000 7f160a651640 1 monclient: found mon.noname-c 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr: -198> 2026-02-24T22:17:08.573+0000 7f160a651640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr: -197> 2026-02-24T22:17:08.573+0000 7f15e5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr: -196> 2026-02-24T22:17:08.573+0000 7f15e5ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr: -195> 2026-02-24T22:17:08.573+0000 7f15e5ffb640 10 monclient: dump: 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr: -194> 2026-02-24T22:17:08.573+0000 7f15e5ffb640 10 monclient: _finish_auth 0 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr: -193> 2026-02-24T22:17:08.573+0000 7f15e5ffb640 10 monclient: _check_auth_tickets 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr: -192> 2026-02-24T22:17:08.573+0000 7f15e5ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr: -191> 2026-02-24T22:17:08.573+0000 7f15e5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr: -190> 2026-02-24T22:17:08.573+0000 7f15e5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr: -189> 2026-02-24T22:17:08.573+0000 7f15e5ffb640 10 monclient: dump: 2026-02-24T22:17:08.905 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr: -188> 2026-02-24T22:17:08.573+0000 7f160cb7da80 5 monclient: authenticate success, global_id 6464 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr: -187> 2026-02-24T22:17:08.573+0000 7f160cb7da80 10 monclient: get_monmap_and_config success 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr: -186> 2026-02-24T22:17:08.573+0000 7f160cb7da80 4 set_mon_vals no callback set 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr: -185> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient: shutdown 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr: -184> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient: build_initial_monmap 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr: -183> 2026-02-24T22:17:08.577+0000 7f160cb7da80 1 build_initial for_mkfs: 0 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr: -182> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient: monmap: 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr: -181> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding auth protocol: cephx 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr: -180> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding auth protocol: cephx 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr: -179> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding auth protocol: cephx 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr: -178> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding con mode: secure 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr: -177> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding con mode: crc 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr: -176> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding con mode: secure 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr: -175> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding con mode: crc 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr: -174> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding con mode: secure 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr: -173> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding con mode: crc 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr: -172> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding con mode: crc 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr: -171> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding con mode: secure 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr: -170> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding con mode: crc 2026-02-24T22:17:08.906 INFO:tasks.workunit.client.0.trial046.stderr: -169> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding con mode: secure 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -168> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding con mode: crc 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -167> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c4f00) adding con mode: secure 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -166> 2026-02-24T22:17:08.577+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -165> 2026-02-24T22:17:08.577+0000 7f160cb7da80 1 librados: starting msgr at 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -164> 2026-02-24T22:17:08.577+0000 7f160cb7da80 1 librados: starting objecter 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -163> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command objecter_requests cmddesc objecter_requests hook 0x556d1142cca0 EEXIST 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -162> 2026-02-24T22:17:08.577+0000 7f160cb7da80 1 librados: setting wanted keys 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -161> 2026-02-24T22:17:08.577+0000 7f160cb7da80 1 librados: calling monclient init 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -160> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient: init 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -159> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding auth protocol: cephx 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -158> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding auth protocol: cephx 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -157> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding auth protocol: cephx 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -156> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding con mode: secure 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -155> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding con mode: crc 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -154> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding con mode: secure 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -153> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding con mode: crc 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -152> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding con mode: secure 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -151> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding con mode: crc 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -150> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding con mode: crc 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -149> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding con mode: secure 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -148> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding con mode: crc 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -147> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding con mode: secure 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -146> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding con mode: crc 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -145> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 AuthRegistry(0x556d114c3de0) adding con mode: secure 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -144> 2026-02-24T22:17:08.577+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -143> 2026-02-24T22:17:08.577+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -142> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command rotate-key cmddesc rotate-key hook 0x556d114c3f28 EEXIST 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -141> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient: _reopen_session rank -1 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -140> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -139> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-a con 0x556d114ec010 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -138> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-b con 0x556d114ef780 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -137> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient(hunting): picked mon.noname-c con 0x556d1151c050 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -136> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -135> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -134> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -133> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient(hunting): _renew_subs 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -132> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient(hunting): authenticate will time out at 1798.684809s 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -131> 2026-02-24T22:17:08.577+0000 7f160ae52640 10 monclient(hunting): get_auth_request con 0x556d1151c050 auth_method 0 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -130> 2026-02-24T22:17:08.577+0000 7f160ae52640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.907 INFO:tasks.workunit.client.0.trial046.stderr: -129> 2026-02-24T22:17:08.577+0000 7f160ae52640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -128> 2026-02-24T22:17:08.577+0000 7f160ae52640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -127> 2026-02-24T22:17:08.577+0000 7f160a651640 10 monclient(hunting): get_auth_request con 0x556d114ec010 auth_method 0 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -126> 2026-02-24T22:17:08.577+0000 7f160a651640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -125> 2026-02-24T22:17:08.577+0000 7f160a651640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -124> 2026-02-24T22:17:08.577+0000 7f160a651640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -123> 2026-02-24T22:17:08.577+0000 7f160ae52640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -122> 2026-02-24T22:17:08.577+0000 7f160ae52640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -121> 2026-02-24T22:17:08.577+0000 7f160ae52640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -120> 2026-02-24T22:17:08.577+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -119> 2026-02-24T22:17:08.577+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -118> 2026-02-24T22:17:08.577+0000 7f160a651640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -117> 2026-02-24T22:17:08.577+0000 7f1609e50640 10 monclient(hunting): get_auth_request con 0x556d114ef780 auth_method 0 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -116> 2026-02-24T22:17:08.577+0000 7f1609e50640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -115> 2026-02-24T22:17:08.577+0000 7f1609e50640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -114> 2026-02-24T22:17:08.577+0000 7f1609e50640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -113> 2026-02-24T22:17:08.577+0000 7f160a651640 10 monclient(hunting): handle_auth_done global_id 6906 payload 824 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -112> 2026-02-24T22:17:08.577+0000 7f160a651640 10 monclient: _finish_hunting 0 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -111> 2026-02-24T22:17:08.577+0000 7f160a651640 1 monclient: found mon.noname-a 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -110> 2026-02-24T22:17:08.577+0000 7f160a651640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -109> 2026-02-24T22:17:08.577+0000 7f15d3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -108> 2026-02-24T22:17:08.577+0000 7f15d3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -107> 2026-02-24T22:17:08.577+0000 7f15d3fff640 10 monclient: dump: 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -106> 2026-02-24T22:17:08.577+0000 7f15d3fff640 10 monclient: _finish_auth 0 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -105> 2026-02-24T22:17:08.577+0000 7f15d3fff640 10 monclient: _check_auth_tickets 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -104> 2026-02-24T22:17:08.577+0000 7f15d3fff640 10 monclient: handle_config config(0 keys) 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -103> 2026-02-24T22:17:08.577+0000 7f15d3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -102> 2026-02-24T22:17:08.577+0000 7f15d3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr: -101> 2026-02-24T22:17:08.577+0000 7f15d3fff640 10 monclient: dump: 2026-02-24T22:17:08.908 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -100> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 monclient: authenticate success, global_id 6906 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -99> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient: _renew_subs 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -98> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -97> 2026-02-24T22:17:08.577+0000 7f15e5ffb640 4 set_mon_vals no callback set 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -96> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient: _renew_subs 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -95> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -94> 2026-02-24T22:17:08.577+0000 7f160cb7da80 1 librados: init done 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -93> 2026-02-24T22:17:08.577+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command cr dump hook 0x556d1142c6b8 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -92> 2026-02-24T22:17:08.577+0000 7f160cb7da80 4 Realm: () 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -91> 2026-02-24T22:17:08.577+0000 7f160cb7da80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -90> 2026-02-24T22:17:08.577+0000 7f160cb7da80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -89> 2026-02-24T22:17:08.577+0000 7f160cb7da80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -88> 2026-02-24T22:17:08.577+0000 7f160cb7da80 20 zonegroup default 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -87> 2026-02-24T22:17:08.577+0000 7f1609e50640 10 monclient: get_auth_request con 0x556d1158c620 auth_method 0 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -86> 2026-02-24T22:17:08.577+0000 7f15d3fff640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -85> 2026-02-24T22:17:08.577+0000 7f15d3fff640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -84> 2026-02-24T22:17:08.577+0000 7f15d3fff640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -83> 2026-02-24T22:17:08.577+0000 7f160ae52640 10 monclient: get_auth_request con 0x7f15cc0707a0 auth_method 0 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -82> 2026-02-24T22:17:08.577+0000 7f160cb7da80 20 rados->read ofs=0 len=0 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -81> 2026-02-24T22:17:08.577+0000 7f160a651640 10 monclient: get_auth_request con 0x556d115af940 auth_method 0 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -80> 2026-02-24T22:17:08.577+0000 7f160cb7da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -79> 2026-02-24T22:17:08.577+0000 7f160cb7da80 20 rados->read ofs=0 len=0 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -78> 2026-02-24T22:17:08.577+0000 7f1609e50640 10 monclient: get_auth_request con 0x556d115d0fe0 auth_method 0 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -77> 2026-02-24T22:17:08.581+0000 7f160cb7da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -76> 2026-02-24T22:17:08.581+0000 7f160cb7da80 20 started sync module instance, tier type = 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -75> 2026-02-24T22:17:08.581+0000 7f160cb7da80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -74> 2026-02-24T22:17:08.581+0000 7f160ae52640 10 monclient: get_auth_request con 0x556d11632490 auth_method 0 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -73> 2026-02-24T22:17:08.581+0000 7f160ae52640 10 monclient: get_auth_request con 0x556d11739bd0 auth_method 0 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -72> 2026-02-24T22:17:08.581+0000 7f160a651640 10 monclient: get_auth_request con 0x556d11694c90 auth_method 0 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -71> 2026-02-24T22:17:08.581+0000 7f1609e50640 10 monclient: get_auth_request con 0x556d116f7a90 auth_method 0 2026-02-24T22:17:08.909 INFO:tasks.workunit.client.0.trial046.stderr: -70> 2026-02-24T22:17:08.581+0000 7f160a651640 10 monclient: get_auth_request con 0x556d1179cbd0 auth_method 0 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -69> 2026-02-24T22:17:08.581+0000 7f160a651640 0 -- 10.20.193.46:0/1925587481 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x556d1179cbd0 msgr2=0x556d117bd030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -68> 2026-02-24T22:17:08.581+0000 7f15d3fff640 1 client.6906.objecter ms_handle_reset 0x556d1179cbd0 session 0x556d1179c4c0 osd.6 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -67> 2026-02-24T22:17:08.581+0000 7f15d3fff640 10 monclient: _renew_subs 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -66> 2026-02-24T22:17:08.581+0000 7f15d3fff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -65> 2026-02-24T22:17:08.581+0000 7f160a651640 10 monclient: get_auth_request con 0x7f15cc09d450 auth_method 0 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -64> 2026-02-24T22:17:08.581+0000 7f160cb7da80 20 add_watcher() i=1 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -63> 2026-02-24T22:17:08.581+0000 7f160cb7da80 20 add_watcher() i=6 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -62> 2026-02-24T22:17:08.581+0000 7f160cb7da80 20 add_watcher() i=5 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -61> 2026-02-24T22:17:08.581+0000 7f160cb7da80 20 add_watcher() i=3 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -60> 2026-02-24T22:17:08.581+0000 7f160cb7da80 20 add_watcher() i=0 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -59> 2026-02-24T22:17:08.581+0000 7f160cb7da80 20 add_watcher() i=2 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -58> 2026-02-24T22:17:08.581+0000 7f160cb7da80 20 add_watcher() i=4 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -57> 2026-02-24T22:17:08.581+0000 7f160cb7da80 20 add_watcher() i=7 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -56> 2026-02-24T22:17:08.581+0000 7f160cb7da80 2 all 8 watchers are set, enabling cache 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -55> 2026-02-24T22:17:08.581+0000 7f1609e50640 10 monclient: get_auth_request con 0x7f152c008af0 auth_method 0 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -54> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding auth protocol: cephx 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -53> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding auth protocol: cephx 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -52> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding auth protocol: cephx 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -51> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding con mode: secure 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -50> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding con mode: crc 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -49> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding con mode: secure 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -48> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding con mode: crc 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -47> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding con mode: secure 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -46> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding con mode: crc 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -45> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding con mode: crc 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -44> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding con mode: secure 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -43> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding con mode: crc 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -42> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding con mode: secure 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -41> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding con mode: crc 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -40> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 AuthRegistry(0x7ffc2c48f430) adding con mode: secure 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -39> 2026-02-24T22:17:08.585+0000 7f160cb7da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -38> 2026-02-24T22:17:08.585+0000 7f160cb7da80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -37> 2026-02-24T22:17:08.585+0000 7f160cb7da80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -36> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command cache list hook 0x556d114efb90 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -35> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command cache inspect hook 0x556d114efb90 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -34> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command cache erase hook 0x556d114efb90 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -33> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command cache zap hook 0x556d114efb90 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -32> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 note: GC not initialized 2026-02-24T22:17:08.910 INFO:tasks.workunit.client.0.trial046.stderr: -31> 2026-02-24T22:17:08.585+0000 7f15237fe640 20 reqs_thread_entry: start 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -30> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command sync trace show hook 0x7f15fc0062d0 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -29> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command sync trace history hook 0x7f15fc0062d0 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -28> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command sync trace active hook 0x7f15fc0062d0 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -27> 2026-02-24T22:17:08.585+0000 7f160cb7da80 5 asok(0x556d11149f00) register_command sync trace active_short hook 0x7f15fc0062d0 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -26> 2026-02-24T22:17:08.585+0000 7f160ae52640 0 -- 10.20.193.46:0/3793593507 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x556d117b38d0 msgr2=0x556d117b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -25> 2026-02-24T22:17:08.585+0000 7f160ae52640 10 monclient: get_auth_request con 0x556d117b38d0 auth_method 0 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -24> 2026-02-24T22:17:08.789+0000 7f160ae52640 10 monclient: get_auth_request con 0x556d117b38d0 auth_method 0 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -23> 2026-02-24T22:17:08.789+0000 7f160a651640 10 monclient: get_auth_request con 0x556d117b51f0 auth_method 0 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -22> 2026-02-24T22:17:08.793+0000 7f1609e50640 10 monclient: get_auth_request con 0x556d117b4df0 auth_method 0 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -21> 2026-02-24T22:17:08.793+0000 7f160ae52640 10 monclient: get_auth_request con 0x556d116c8520 auth_method 0 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -20> 2026-02-24T22:17:08.797+0000 7f160a651640 10 monclient: get_auth_request con 0x556d117b9eb0 auth_method 0 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -19> 2026-02-24T22:17:08.801+0000 7f1609e50640 10 monclient: get_auth_request con 0x556d117bcd50 auth_method 0 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -18> 2026-02-24T22:17:08.825+0000 7f160cb7da80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -17> 2026-02-24T22:17:08.825+0000 7f160cb7da80 20 Filter name: none 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -16> 2026-02-24T22:17:08.825+0000 7f1521ffb640 20 reqs_thread_entry: start 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -15> 2026-02-24T22:17:08.825+0000 7f160cb7da80 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -14> 2026-02-24T22:17:08.825+0000 7f160cb7da80 20 rados->read ofs=0 len=0 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -13> 2026-02-24T22:17:08.825+0000 7f160cb7da80 20 rados_obj.operate() r=0 bl.length=222 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -12> 2026-02-24T22:17:08.825+0000 7f160cb7da80 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -11> 2026-02-24T22:17:08.825+0000 7f160cb7da80 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -10> 2026-02-24T22:17:08.825+0000 7f160cb7da80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 : miss 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -9> 2026-02-24T22:17:08.825+0000 7f160cb7da80 20 rados->read ofs=0 len=0 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -8> 2026-02-24T22:17:08.825+0000 7f160cb7da80 20 rados_obj.operate() r=0 bl.length=452 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -7> 2026-02-24T22:17:08.825+0000 7f160cb7da80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 info.flags=0x17 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -6> 2026-02-24T22:17:08.825+0000 7f160cb7da80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 to cache LRU end 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -5> 2026-02-24T22:17:08.825+0000 7f160cb7da80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -4> 2026-02-24T22:17:08.825+0000 7f160cb7da80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -3> 2026-02-24T22:17:08.825+0000 7f160cb7da80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -2> 2026-02-24T22:17:08.825+0000 7f160ae52640 10 monclient: get_auth_request con 0x556d116d3930 auth_method 0 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: -1> 2026-02-24T22:17:08.829+0000 7f160cb7da80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f160cb7da80 time 2026-02-24T22:17:08.833071+0000 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f160ffbfeb1] 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x1713e63) [0x556cef4f7e63] 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x556ceea55202] 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f160f2bfd90] 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:17:08.911 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 0> 2026-02-24T22:17:08.829+0000 7f160cb7da80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f160cb7da80 thread_name:radosgw-admin 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f160f2d8520] 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f160ffbff10] 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x1713e63) [0x556cef4f7e63] 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x556ceea55202] 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f160f2bfd90] 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr:--- logging levels --- 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 none 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 lockdep 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 context 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 crush 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_balancer 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_locker 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log_expire 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_migrator 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 3/ 5 mds_quiesce 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 buffer 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 timer 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 filer 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 striper 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 objecter 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rados 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_mirror 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_replay 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_pwl 2026-02-24T22:17:08.912 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 journaler 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objectcacher 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 immutable_obj_cache 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 client 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 osd 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 optracker 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objclass 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 filestore 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 journal 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 ms 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mon 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 0/10 monc 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 paxos 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 tp 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 auth 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 crypto 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 finisher 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 reserver 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 heartbeatmap 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 perfcounter 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_sync 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_datacache 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_access 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dbstore 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_flight 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_lifecycle 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_restore 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw_notification 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_bucket_logging 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 javaclient 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 asok 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 throttle 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 refs 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 compressor 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore_compression 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluefs 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 bdev 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 kstore 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 4/ 5 rocksdb 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 fuse 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 2/ 5 mgr 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mgrc 2026-02-24T22:17:08.913 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 dpdk 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 eventtrace 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 prioritycache 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 test 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephfs_mirror 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephsqlite 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 crimson_interrupt 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_onode 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_odata 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_omap 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_tm 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_t 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cleaner 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_epm 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_lba 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_fixedkv_tree 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cache 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_journal 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_device 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_backref 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 alienstore 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mclock 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dedup 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cyanstore 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 ceph_exporter 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 memstore 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 trace 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 ceph_dedup 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: -2/-2 (syslog threshold) 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 99/99 (stderr threshold) 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr:--- pthread ID / name mapping for recent threads --- 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 7f1521ffb640 / http_manager 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 7f15237fe640 / http_manager 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 7f15d3fff640 / ms_dispatch 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 7f15e5ffb640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 7f15e7fff640 / ms_dispatch 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 7f15eeffd640 / ms_dispatch 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 7f15f8ff9640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 7f15f97fa640 / admin_socket 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 7f160964f640 / ms_dispatch 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 7f1609e50640 / msgr-worker-2 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 7f160a651640 / msgr-worker-1 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 7f160ae52640 / msgr-worker-0 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 7f160b653640 / io_context_pool 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: 7f160cb7da80 / radosgw-admin 2026-02-24T22:17:08.914 INFO:tasks.workunit.client.0.trial046.stderr: max_recent 500 2026-02-24T22:17:08.915 INFO:tasks.workunit.client.0.trial046.stderr: max_new 1000 2026-02-24T22:17:08.915 INFO:tasks.workunit.client.0.trial046.stderr: log_file /var/lib/ceph/crash/2026-02-24T22:17:08.835337Z_dbd41aea-e7c0-42be-8f6c-78ce63f6baea/log 2026-02-24T22:17:08.915 INFO:tasks.workunit.client.0.trial046.stderr:--- end dump of recent events --- 2026-02-24T22:17:08.915 INFO:tasks.workunit.client.0.trial046.stderr:Aborted 2026-02-24T22:17:08.915 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket abort-at-set-target-layout 2026-02-24T22:17:08.915 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:08.915 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:08.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.869+0000 7fc93a999a80 4 Realm: () 2026-02-24T22:17:08.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.869+0000 7fc93a999a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:08.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.869+0000 7fc93a999a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:08.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.869+0000 7fc93a999a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:08.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.869+0000 7fc93a999a80 20 zonegroup default 2026-02-24T22:17:08.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.869+0000 7fc93a999a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:08.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.873+0000 7fc93a999a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:08.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.873+0000 7fc93a999a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:08.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.873+0000 7fc93a999a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:08.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.873+0000 7fc93a999a80 20 started sync module instance, tier type = 2026-02-24T22:17:08.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.873+0000 7fc93a999a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:08.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.873+0000 7fc93a999a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:08.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.873+0000 7fc93a999a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:08.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.873+0000 7fc93a999a80 5 note: GC not initialized 2026-02-24T22:17:08.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.873+0000 7fc8e67dc640 20 reqs_thread_entry: start 2026-02-24T22:17:08.916 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:08.877+0000 7fc938c6e640 0 -- 10.20.193.46:0/2562360705 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x56133682a690 msgr2=0x561336823ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:09.085 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.085+0000 7fc933fff640 0 -- 10.20.193.46:0/2562360705 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5613367fec50 msgr2=0x56133682c800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:09.111 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.113+0000 7fc93a999a80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:09.111 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.113+0000 7fc93a999a80 20 Filter name: none 2026-02-24T22:17:09.111 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.113+0000 7fc8e4fd9640 20 reqs_thread_entry: start 2026-02-24T22:17:09.111 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.113+0000 7fc93a999a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:09.112 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.113+0000 7fc93a999a80 20 rados_obj.operate() r=0 bl.length=222 2026-02-24T22:17:09.112 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.113+0000 7fc93a999a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:09.112 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.113+0000 7fc93a999a80 20 rados_obj.operate() r=0 bl.length=452 2026-02-24T22:17:09.112 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.113+0000 7fc93a999a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:09.113 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.113+0000 7fc93a999a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:09.113 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.113+0000 7fc93a999a80 20 rados_obj.operate() r=0 bl.length=222 2026-02-24T22:17:09.113 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.113+0000 7fc93a999a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:09.113 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.113+0000 7fc93a999a80 20 rados_obj.operate() r=0 bl.length=452 2026-02-24T22:17:09.113 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.113+0000 7fc93a999a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:09.122 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:09.123 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:17:09.123 INFO:tasks.workunit.client.0.trial046.stderr:bucket: abort-at-set-target-layout id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:17:09.123 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:09.124 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:09.124 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:09.129 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:09.130 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:09.174 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket abort-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2026-02-24T22:17:09.192 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:09.192 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:09.209 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.209+0000 7fd9dfb25a80 4 Realm: () 2026-02-24T22:17:09.209 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.209+0000 7fd9dfb25a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:09.209 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.209+0000 7fd9dfb25a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:09.209 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.209+0000 7fd9dfb25a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:09.209 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.209+0000 7fd9dfb25a80 20 zonegroup default 2026-02-24T22:17:09.209 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.209+0000 7fd9dfb25a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:09.210 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.213+0000 7fd9dfb25a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:09.210 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.213+0000 7fd9dfb25a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:09.211 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.213+0000 7fd9dfb25a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:09.211 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.213+0000 7fd9dfb25a80 20 started sync module instance, tier type = 2026-02-24T22:17:09.211 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.213+0000 7fd9dfb25a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:09.211 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.213+0000 7fd9dd5f9640 0 -- 10.20.193.46:0/3607409110 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55f0351cbe50 msgr2=0x55f0351ec230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:09.212 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.213+0000 7fd9dfb25a80 20 add_watcher() i=1 2026-02-24T22:17:09.213 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.213+0000 7fd9dfb25a80 20 add_watcher() i=5 2026-02-24T22:17:09.213 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.213+0000 7fd9dfb25a80 20 add_watcher() i=3 2026-02-24T22:17:09.213 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.213+0000 7fd9dfb25a80 20 add_watcher() i=0 2026-02-24T22:17:09.213 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.213+0000 7fd9dfb25a80 20 add_watcher() i=7 2026-02-24T22:17:09.213 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.213+0000 7fd9dfb25a80 20 add_watcher() i=2 2026-02-24T22:17:09.214 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.213+0000 7fd9dfb25a80 20 add_watcher() i=6 2026-02-24T22:17:09.214 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.213+0000 7fd9dfb25a80 20 add_watcher() i=4 2026-02-24T22:17:09.214 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.213+0000 7fd9dfb25a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:17:09.214 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.217+0000 7fd9dfb25a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:09.215 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.217+0000 7fd9dfb25a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:09.215 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.217+0000 7fd9dfb25a80 5 note: GC not initialized 2026-02-24T22:17:09.215 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.217+0000 7fd98a7e4640 20 reqs_thread_entry: start 2026-02-24T22:17:09.215 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.217+0000 7fd9dddfa640 0 -- 10.20.193.46:0/1077648825 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55f03539fc30 msgr2=0x55f0353a0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:17:09.456 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.457+0000 7fd9dfb25a80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:09.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.457+0000 7fd9dfb25a80 20 Filter name: none 2026-02-24T22:17:09.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.457+0000 7fd988fe1640 20 reqs_thread_entry: start 2026-02-24T22:17:09.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.457+0000 7fd9dfb25a80 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2026-02-24T22:17:09.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.457+0000 7fd9dfb25a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:09.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.457+0000 7fd9dfb25a80 20 rados_obj.operate() r=0 bl.length=222 2026-02-24T22:17:09.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.457+0000 7fd9dfb25a80 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2026-02-24T22:17:09.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.457+0000 7fd9dfb25a80 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2026-02-24T22:17:09.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.457+0000 7fd9dfb25a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 : miss 2026-02-24T22:17:09.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.457+0000 7fd9dfb25a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:09.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.457+0000 7fd9dfb25a80 20 rados_obj.operate() r=0 bl.length=452 2026-02-24T22:17:09.458 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.457+0000 7fd9dfb25a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 info.flags=0x17 2026-02-24T22:17:09.458 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.457+0000 7fd9dfb25a80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 to cache LRU end 2026-02-24T22:17:09.458 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.457+0000 7fd9dfb25a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:17:09.458 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.457+0000 7fd9dfb25a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:09.458 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.457+0000 7fd9dfb25a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 2026-02-24T22:17:09.458 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.457+0000 7fd9dfb25a80 0 INFO: RGWReshardLock::lock found lock on abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 to be held by another RGW process; skipping for now 2026-02-24T22:17:09.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.461+0000 7fd9dfb25a80 20 remove_watcher() i=3 2026-02-24T22:17:09.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.461+0000 7fd9dfb25a80 2 removed watcher, disabling cache 2026-02-24T22:17:09.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.461+0000 7fd9dfb25a80 20 remove_watcher() i=6 2026-02-24T22:17:09.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.461+0000 7fd9dfb25a80 20 remove_watcher() i=4 2026-02-24T22:17:09.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.461+0000 7fd9dfb25a80 20 remove_watcher() i=2 2026-02-24T22:17:09.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.461+0000 7fd9dfb25a80 20 remove_watcher() i=0 2026-02-24T22:17:09.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.461+0000 7fd9dfb25a80 20 remove_watcher() i=5 2026-02-24T22:17:09.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.461+0000 7fd9dfb25a80 20 remove_watcher() i=7 2026-02-24T22:17:09.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:09.461+0000 7fd9dfb25a80 20 remove_watcher() i=1 2026-02-24T22:17:09.467 INFO:tasks.workunit.client.0.trial046.stderr:waiting 30 seconds for reshard lock to expire... 2026-02-24T22:17:39.478 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket abort-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2026-02-24T22:17:39.496 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:39.497 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:39.515 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.516+0000 7f1142374a80 4 Realm: () 2026-02-24T22:17:39.515 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.516+0000 7f1142374a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:39.515 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.516+0000 7f1142374a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:39.515 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.516+0000 7f1142374a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:39.515 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.516+0000 7f1142374a80 20 zonegroup default 2026-02-24T22:17:39.515 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.516+0000 7f1142374a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:39.516 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.516+0000 7f1142374a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:39.516 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.516+0000 7f1142374a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:39.517 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.516+0000 7f1142374a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:39.517 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.516+0000 7f1142374a80 20 started sync module instance, tier type = 2026-02-24T22:17:39.517 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.516+0000 7f1142374a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:39.518 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.520+0000 7f113bfff640 0 -- 10.20.193.46:0/1373427970 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55987da5f890 msgr2=0x55987da7fc70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:17:39.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.520+0000 7f1142374a80 20 add_watcher() i=1 2026-02-24T22:17:39.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.520+0000 7f1142374a80 20 add_watcher() i=6 2026-02-24T22:17:39.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.520+0000 7f1142374a80 20 add_watcher() i=5 2026-02-24T22:17:39.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.520+0000 7f1142374a80 20 add_watcher() i=3 2026-02-24T22:17:39.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.520+0000 7f1142374a80 20 add_watcher() i=0 2026-02-24T22:17:39.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.520+0000 7f1142374a80 20 add_watcher() i=4 2026-02-24T22:17:39.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.520+0000 7f1142374a80 20 add_watcher() i=7 2026-02-24T22:17:39.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.520+0000 7f1142374a80 20 add_watcher() i=2 2026-02-24T22:17:39.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.520+0000 7f1142374a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:17:39.523 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.524+0000 7f1142374a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:39.523 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.524+0000 7f1142374a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:39.523 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.524+0000 7f1142374a80 5 note: GC not initialized 2026-02-24T22:17:39.523 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.524+0000 7f10ecfe1640 20 reqs_thread_entry: start 2026-02-24T22:17:39.524 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.524+0000 7f113bfff640 0 -- 10.20.193.46:0/1593875154 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55987db161e0 msgr2=0x55987db0d9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:17:39.765 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.764+0000 7f1142374a80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:39.765 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.764+0000 7f1142374a80 20 Filter name: none 2026-02-24T22:17:39.766 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.764+0000 7f1056ffd640 20 reqs_thread_entry: start 2026-02-24T22:17:39.766 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.764+0000 7f1142374a80 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2026-02-24T22:17:39.766 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.764+0000 7f1142374a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:39.766 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.764+0000 7f1142374a80 20 rados_obj.operate() r=0 bl.length=222 2026-02-24T22:17:39.766 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.764+0000 7f1142374a80 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2026-02-24T22:17:39.766 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.764+0000 7f1142374a80 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2026-02-24T22:17:39.766 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.764+0000 7f1142374a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 : miss 2026-02-24T22:17:39.766 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.764+0000 7f1142374a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:39.767 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.768+0000 7f1142374a80 20 rados_obj.operate() r=0 bl.length=452 2026-02-24T22:17:39.767 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.768+0000 7f1142374a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 info.flags=0x17 2026-02-24T22:17:39.767 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.768+0000 7f1142374a80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 to cache LRU end 2026-02-24T22:17:39.767 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.768+0000 7f1142374a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:17:39.767 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.768+0000 7f1142374a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:39.767 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.768+0000 7f1142374a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 2026-02-24T22:17:39.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.772+0000 7f1142374a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 info.flags=0x17 2026-02-24T22:17:39.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.772+0000 7f1142374a80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 to cache LRU end 2026-02-24T22:17:39.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.772+0000 7f1142374a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:39.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.772+0000 7f1142374a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525, ofs0, ns] 2026-02-24T22:17:39.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.772+0000 7f1116ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961336 cookie 94113430879456 notifier 6521 bl.length()=1027 2026-02-24T22:17:39.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.772+0000 7f1116ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 info.flags=0x17 2026-02-24T22:17:39.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.772+0000 7f1116ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 to cache LRU end 2026-02-24T22:17:39.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.772+0000 7f1116ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:39.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.772+0000 7f1142374a80 20 handle_bi_update(): bucket=:abort-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525]): orig_sources= new_sources= 2026-02-24T22:17:39.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.772+0000 7f1142374a80 20 handle_bi_update(): bucket=:abort-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525]): potential sources added=[] removed=[] 2026-02-24T22:17:39.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.772+0000 7f1142374a80 20 handle_bi_update(): bucket=:abort-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525]): orig_dests= new_dests= 2026-02-24T22:17:39.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.772+0000 7f1142374a80 20 handle_bi_update(): bucket=:abort-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525]): potential dests added=[] removed=[] 2026-02-24T22:17:39.773 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.772+0000 7f1142374a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525.1.0 2026-02-24T22:17:39.773 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.772+0000 7f1142374a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525.1.0 2026-02-24T22:17:39.773 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.772+0000 7f1142374a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525.1.1 2026-02-24T22:17:39.773 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.772+0000 7f1142374a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525.1.1 2026-02-24T22:17:39.773 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.772+0000 7f1142374a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525.0 2026-02-24T22:17:39.774 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.776+0000 7f1142374a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 : hit (requested=0x17, cached=0x17) 2026-02-24T22:17:39.774 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.776+0000 7f1142374a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 2026-02-24T22:17:39.775 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.776+0000 7f1142374a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 info.flags=0x17 2026-02-24T22:17:39.776 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.776+0000 7f1142374a80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 to cache LRU end 2026-02-24T22:17:39.776 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.776+0000 7f1142374a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:39.776 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.776+0000 7f1142374a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525, ofs0, ns] 2026-02-24T22:17:39.776 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.776+0000 7f11167fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961337 cookie 94113430879456 notifier 6521 bl.length()=1027 2026-02-24T22:17:39.776 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.776+0000 7f11167fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 info.flags=0x17 2026-02-24T22:17:39.776 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.776+0000 7f11167fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 to cache LRU end 2026-02-24T22:17:39.776 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.776+0000 7f11167fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:39.776 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.776+0000 7f1142374a80 20 handle_bi_update(): bucket=:abort-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525]): orig_sources= new_sources= 2026-02-24T22:17:39.776 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.776+0000 7f1142374a80 20 handle_bi_update(): bucket=:abort-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525]): potential sources added=[] removed=[] 2026-02-24T22:17:39.776 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.776+0000 7f1142374a80 20 handle_bi_update(): bucket=:abort-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525]): orig_dests= new_dests= 2026-02-24T22:17:39.776 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.776+0000 7f1142374a80 20 handle_bi_update(): bucket=:abort-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525]): potential dests added=[] removed=[] 2026-02-24T22:17:39.778 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.776+0000 7f1142374a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525.0 2026-02-24T22:17:39.778 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.776+0000 7f1142374a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 : hit (requested=0x17, cached=0x17) 2026-02-24T22:17:39.778 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.776+0000 7f1142374a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 2026-02-24T22:17:39.778 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.776+0000 7f1142374a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 info.flags=0x17 2026-02-24T22:17:39.778 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.776+0000 7f1142374a80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 to cache LRU end 2026-02-24T22:17:39.778 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.776+0000 7f1142374a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:39.778 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.776+0000 7f1142374a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525, ofs0, ns] 2026-02-24T22:17:39.778 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.776+0000 7f11167fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961338 cookie 94113430879456 notifier 6521 bl.length()=991 2026-02-24T22:17:39.778 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.776+0000 7f11167fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 info.flags=0x17 2026-02-24T22:17:39.778 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.776+0000 7f11167fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 to cache LRU end 2026-02-24T22:17:39.779 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.776+0000 7f11167fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:39.779 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.780+0000 7f1142374a80 20 handle_bi_update(): bucket=:abort-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525]): orig_sources= new_sources= 2026-02-24T22:17:39.779 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.780+0000 7f1142374a80 20 handle_bi_update(): bucket=:abort-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525]): potential sources added=[] removed=[] 2026-02-24T22:17:39.779 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.780+0000 7f1142374a80 20 handle_bi_update(): bucket=:abort-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525]): orig_dests= new_dests= 2026-02-24T22:17:39.779 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.780+0000 7f1142374a80 20 handle_bi_update(): bucket=:abort-at-set-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525]): potential dests added=[] removed=[] 2026-02-24T22:17:39.779 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.780+0000 7f1142374a80 1 execute INFO: reshard of bucket "abort-at-set-target-layout" from 1 shards to 2 shards completed successfully 2026-02-24T22:17:39.782 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.784+0000 7f1142374a80 20 remove_watcher() i=3 2026-02-24T22:17:39.783 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.784+0000 7f1142374a80 2 removed watcher, disabling cache 2026-02-24T22:17:39.783 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.784+0000 7f1142374a80 20 remove_watcher() i=5 2026-02-24T22:17:39.783 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.784+0000 7f1142374a80 20 remove_watcher() i=6 2026-02-24T22:17:39.783 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.784+0000 7f1142374a80 20 remove_watcher() i=0 2026-02-24T22:17:39.783 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.784+0000 7f1142374a80 20 remove_watcher() i=4 2026-02-24T22:17:39.783 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.784+0000 7f1142374a80 20 remove_watcher() i=2 2026-02-24T22:17:39.783 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.784+0000 7f1142374a80 20 remove_watcher() i=1 2026-02-24T22:17:39.783 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.784+0000 7f1142374a80 20 remove_watcher() i=7 2026-02-24T22:17:39.789 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket abort-at-set-target-layout 2026-02-24T22:17:39.803 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:39.803 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:39.820 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.820+0000 7f5a1a01ca80 4 Realm: () 2026-02-24T22:17:39.820 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.820+0000 7f5a1a01ca80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:39.820 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.820+0000 7f5a1a01ca80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:39.821 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.820+0000 7f5a1a01ca80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:39.821 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.820+0000 7f5a1a01ca80 20 zonegroup default 2026-02-24T22:17:39.821 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.820+0000 7f5a1a01ca80 20 rados->read ofs=0 len=0 2026-02-24T22:17:39.822 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.824+0000 7f5a1a01ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:39.822 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.824+0000 7f5a1a01ca80 20 rados->read ofs=0 len=0 2026-02-24T22:17:39.823 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.824+0000 7f5a1a01ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:39.823 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.824+0000 7f5a1a01ca80 20 started sync module instance, tier type = 2026-02-24T22:17:39.823 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.824+0000 7f5a1a01ca80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:39.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.824+0000 7f5a1a01ca80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:39.825 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.824+0000 7f5a1a01ca80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:39.825 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.824+0000 7f5a1a01ca80 5 note: GC not initialized 2026-02-24T22:17:39.825 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.824+0000 7f59c57e2640 20 reqs_thread_entry: start 2026-02-24T22:17:39.827 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.828+0000 7f5a13fff640 0 -- 10.20.193.46:0/157926099 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x555fa8953c00 msgr2=0x555fa8946e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:17:39.833 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.832+0000 7f5a137fe640 0 -- 10.20.193.46:0/157926099 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x555fa894f680 msgr2=0x555fa894a3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:39.856 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.856+0000 7f5a1a01ca80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:39.857 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.856+0000 7f5a1a01ca80 20 Filter name: none 2026-02-24T22:17:39.857 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.856+0000 7f592f7fe640 20 reqs_thread_entry: start 2026-02-24T22:17:39.857 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.856+0000 7f5a1a01ca80 20 rados->read ofs=0 len=0 2026-02-24T22:17:39.858 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.856+0000 7f5a1a01ca80 20 rados_obj.operate() r=0 bl.length=222 2026-02-24T22:17:39.858 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.856+0000 7f5a1a01ca80 20 rados->read ofs=0 len=0 2026-02-24T22:17:39.858 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.856+0000 7f5a1a01ca80 20 rados_obj.operate() r=0 bl.length=452 2026-02-24T22:17:39.858 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.856+0000 7f5a1a01ca80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:39.858 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.856+0000 7f5a1a01ca80 20 rados->read ofs=0 len=0 2026-02-24T22:17:39.858 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.856+0000 7f5a1a01ca80 20 rados_obj.operate() r=0 bl.length=222 2026-02-24T22:17:39.858 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.856+0000 7f5a1a01ca80 20 rados->read ofs=0 len=0 2026-02-24T22:17:39.858 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.856+0000 7f5a1a01ca80 20 rados_obj.operate() r=0 bl.length=452 2026-02-24T22:17:39.858 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:39.856+0000 7f5a1a01ca80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:39.867 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:39.867 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:17:39.867 INFO:tasks.workunit.client.0.trial046.stderr:bucket: abort-at-set-target-layout id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525 num_objects: 19 size_kb: 1 num_shards: 2 2026-02-24T22:17:39.867 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:39.868 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:39.869 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:39.871 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:39.871 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:39.915 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:39.915 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:39.926 INFO:tasks.workunit.client.0.trial046.stderr:TEST: reshard bucket with EIO injected at trim_reshard_log_entries 2026-02-24T22:17:39.926 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:39.927 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:39.927 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:39.936 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:39.936 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:39.979 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:39.979 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:39.985 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:39.985 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:39.991 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:39.991 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:39.997 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:39.997 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:40.004 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:40.004 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:40.010 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:40.010 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:40.016 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:40.016 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:40.021 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:40.022 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:40.027 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:40.028 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:40.033 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:40.033 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:40.039 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:40.039 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:40.045 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:40.045 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:40.051 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:40.052 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:40.057 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:40.057 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:40.063 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:40.063 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:40.069 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:40.069 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:40.075 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:40.075 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:40.082 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:40.082 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:40.088 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:40.088 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:40.094 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:40.094 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:40.099 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-trim-reshard-log-entries 2026-02-24T22:17:40.114 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:40.114 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:40.131 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.132+0000 7f45e62d3a80 4 Realm: () 2026-02-24T22:17:40.131 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.132+0000 7f45e62d3a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:40.131 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.132+0000 7f45e62d3a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:40.131 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.132+0000 7f45e62d3a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:40.131 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.132+0000 7f45e62d3a80 20 zonegroup default 2026-02-24T22:17:40.132 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.132+0000 7f45e62d3a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:40.132 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.132+0000 7f45e62d3a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:40.133 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.132+0000 7f45e62d3a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:40.133 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.132+0000 7f45e62d3a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:40.133 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.132+0000 7f45e62d3a80 20 started sync module instance, tier type = 2026-02-24T22:17:40.133 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.132+0000 7f45e62d3a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:40.135 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.136+0000 7f45e62d3a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:40.135 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.136+0000 7f45e62d3a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:40.135 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.136+0000 7f45e62d3a80 5 note: GC not initialized 2026-02-24T22:17:40.135 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.136+0000 7f45917e2640 20 reqs_thread_entry: start 2026-02-24T22:17:40.138 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.140+0000 7f45dffff640 0 -- 10.20.193.46:0/3675136169 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x5607e59bc3e0 msgr2=0x5607e59b2bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:17:40.350 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.352+0000 7f45df7fe640 0 -- 10.20.193.46:0/3675136169 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5607e598d7e0 msgr2=0x5607e59bb3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:40.376 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.376+0000 7f45e62d3a80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:40.377 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.376+0000 7f45e62d3a80 20 Filter name: none 2026-02-24T22:17:40.377 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.376+0000 7f44fb7fe640 20 reqs_thread_entry: start 2026-02-24T22:17:40.377 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.376+0000 7f45e62d3a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:40.378 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.380+0000 7f45e62d3a80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:17:40.378 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.380+0000 7f45e62d3a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:40.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.380+0000 7f45e62d3a80 20 rados_obj.operate() r=0 bl.length=459 2026-02-24T22:17:40.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.380+0000 7f45e62d3a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:40.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.380+0000 7f45e62d3a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:40.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.380+0000 7f45e62d3a80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:17:40.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.380+0000 7f45e62d3a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:40.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.380+0000 7f45e62d3a80 20 rados_obj.operate() r=0 bl.length=459 2026-02-24T22:17:40.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.380+0000 7f45e62d3a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:40.389 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:40.390 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:17:40.390 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-trim-reshard-log-entries id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:17:40.390 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:40.390 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket error-at-trim-reshard-log-entries --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at trim_reshard_log_entries 2026-02-24T22:17:40.407 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:40.407 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:40.424 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.424+0000 7f81b3db4a80 4 Realm: () 2026-02-24T22:17:40.424 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.424+0000 7f81b3db4a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:40.424 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.424+0000 7f81b3db4a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:40.424 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.424+0000 7f81b3db4a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:40.424 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.424+0000 7f81b3db4a80 20 zonegroup default 2026-02-24T22:17:40.424 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.424+0000 7f81b3db4a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:40.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.424+0000 7f81b3db4a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:40.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.424+0000 7f81b3db4a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:40.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.428+0000 7f81b3db4a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:40.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.428+0000 7f81b3db4a80 20 started sync module instance, tier type = 2026-02-24T22:17:40.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.428+0000 7f81b3db4a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:40.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.428+0000 7f81b2089640 0 -- 10.20.193.46:0/1700752476 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x55f75cd05320 msgr2=0x55f75cd257a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:40.428 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.428+0000 7f81b3db4a80 20 add_watcher() i=1 2026-02-24T22:17:40.428 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.428+0000 7f81b3db4a80 20 add_watcher() i=3 2026-02-24T22:17:40.428 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.428+0000 7f81b3db4a80 20 add_watcher() i=6 2026-02-24T22:17:40.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.428+0000 7f81b3db4a80 20 add_watcher() i=0 2026-02-24T22:17:40.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.428+0000 7f81b3db4a80 20 add_watcher() i=2 2026-02-24T22:17:40.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.428+0000 7f81b3db4a80 20 add_watcher() i=5 2026-02-24T22:17:40.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.428+0000 7f81b3db4a80 20 add_watcher() i=7 2026-02-24T22:17:40.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.428+0000 7f81b3db4a80 20 add_watcher() i=4 2026-02-24T22:17:40.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.428+0000 7f81b3db4a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:17:40.430 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.432+0000 7f81b3db4a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:40.430 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.432+0000 7f81b3db4a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:40.430 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.432+0000 7f81b3db4a80 5 note: GC not initialized 2026-02-24T22:17:40.430 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.432+0000 7f80e27fc640 20 reqs_thread_entry: start 2026-02-24T22:17:40.430 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.432+0000 7f81b2089640 0 -- 10.20.193.46:0/3701022290 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55f75cd1b730 msgr2=0x55f75cd1bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:40.670 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.672+0000 7f81b3db4a80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:40.671 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.672+0000 7f81b3db4a80 20 Filter name: none 2026-02-24T22:17:40.671 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.672+0000 7f80e0ff9640 20 reqs_thread_entry: start 2026-02-24T22:17:40.671 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.672+0000 7f81b3db4a80 10 cache get: name=default.rgw.meta+root+error-at-trim-reshard-log-entries : miss 2026-02-24T22:17:40.671 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.672+0000 7f81b3db4a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:40.672 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.672+0000 7f81b3db4a80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:17:40.672 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.672+0000 7f81b3db4a80 10 cache put: name=default.rgw.meta+root+error-at-trim-reshard-log-entries info.flags=0x11 2026-02-24T22:17:40.672 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.672+0000 7f81b3db4a80 10 adding default.rgw.meta+root+error-at-trim-reshard-log-entries to cache LRU end 2026-02-24T22:17:40.672 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.672+0000 7f81b3db4a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 : miss 2026-02-24T22:17:40.672 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.672+0000 7f81b3db4a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:40.672 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.672+0000 7f81b3db4a80 20 rados_obj.operate() r=0 bl.length=459 2026-02-24T22:17:40.672 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.672+0000 7f81b3db4a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 info.flags=0x17 2026-02-24T22:17:40.672 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.672+0000 7f81b3db4a80 10 adding default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 to cache LRU end 2026-02-24T22:17:40.672 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.672+0000 7f81b3db4a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:17:40.673 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.672+0000 7f81b3db4a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:40.673 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.672+0000 7f81b3db4a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 2026-02-24T22:17:40.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.676+0000 7f81b3db4a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 info.flags=0x17 2026-02-24T22:17:40.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.676+0000 7f81b3db4a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 to cache LRU end 2026-02-24T22:17:40.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.676+0000 7f81b3db4a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:40.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.676+0000 7f81b3db4a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526, ofs0, ns] 2026-02-24T22:17:40.677 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.676+0000 7f818cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554960009 cookie 94520902451616 notifier 6569 bl.length()=1041 2026-02-24T22:17:40.677 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.676+0000 7f818cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 info.flags=0x17 2026-02-24T22:17:40.677 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.676+0000 7f818cff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 to cache LRU end 2026-02-24T22:17:40.677 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.676+0000 7f818cff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:40.677 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.676+0000 7f81b3db4a80 20 handle_bi_update(): bucket=:error-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526]): orig_sources= new_sources= 2026-02-24T22:17:40.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.676+0000 7f81b3db4a80 20 handle_bi_update(): bucket=:error-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526]): potential sources added=[] removed=[] 2026-02-24T22:17:40.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.676+0000 7f81b3db4a80 20 handle_bi_update(): bucket=:error-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526]): orig_dests= new_dests= 2026-02-24T22:17:40.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.676+0000 7f81b3db4a80 20 handle_bi_update(): bucket=:error-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526]): potential dests added=[] removed=[] 2026-02-24T22:17:40.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.676+0000 7f81b3db4a80 -1 Injecting error=-5 at location=trim_reshard_log_entries 2026-02-24T22:17:40.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.676+0000 7f81b3db4a80 0 ERROR: init_reshard failed to trim reshard log entries: (5) Input/output error 2026-02-24T22:17:40.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.680+0000 7f81b3db4a80 20 remove_watcher() i=3 2026-02-24T22:17:40.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.680+0000 7f81b3db4a80 2 removed watcher, disabling cache 2026-02-24T22:17:40.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.680+0000 7f81b3db4a80 20 remove_watcher() i=2 2026-02-24T22:17:40.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.680+0000 7f81b3db4a80 20 remove_watcher() i=0 2026-02-24T22:17:40.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.680+0000 7f81b3db4a80 20 remove_watcher() i=4 2026-02-24T22:17:40.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.680+0000 7f81b3db4a80 20 remove_watcher() i=7 2026-02-24T22:17:40.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.680+0000 7f81b3db4a80 20 remove_watcher() i=6 2026-02-24T22:17:40.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.680+0000 7f81b3db4a80 20 remove_watcher() i=5 2026-02-24T22:17:40.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.680+0000 7f81b3db4a80 20 remove_watcher() i=1 2026-02-24T22:17:40.688 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-trim-reshard-log-entries 2026-02-24T22:17:40.702 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:40.702 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:40.719 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.720+0000 7fe8eaf47a80 4 Realm: () 2026-02-24T22:17:40.719 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.720+0000 7fe8eaf47a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:40.719 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.720+0000 7fe8eaf47a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:40.719 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.720+0000 7fe8eaf47a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:40.719 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.720+0000 7fe8eaf47a80 20 zonegroup default 2026-02-24T22:17:40.720 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.720+0000 7fe8eaf47a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:40.721 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.720+0000 7fe8eaf47a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:40.721 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.720+0000 7fe8eaf47a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:40.721 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.724+0000 7fe8eaf47a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:40.722 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.724+0000 7fe8eaf47a80 20 started sync module instance, tier type = 2026-02-24T22:17:40.722 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.724+0000 7fe8eaf47a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:40.723 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.724+0000 7fe8eaf47a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:40.723 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.724+0000 7fe8eaf47a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:40.724 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.724+0000 7fe8eaf47a80 5 note: GC not initialized 2026-02-24T22:17:40.724 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.724+0000 7fe8967e4640 20 reqs_thread_entry: start 2026-02-24T22:17:40.727 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.728+0000 7fe8e921c640 0 -- 10.20.193.46:0/2367011503 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x5617727dc4a0 msgr2=0x5617727cf710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:40.937 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.936+0000 7fe8e8a1b640 0 -- 10.20.193.46:0/2367011503 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5617727d7f20 msgr2=0x5617727d2c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:17:40.965 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.964+0000 7fe8eaf47a80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:40.965 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.964+0000 7fe8eaf47a80 20 Filter name: none 2026-02-24T22:17:40.965 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.964+0000 7fe894fe1640 20 reqs_thread_entry: start 2026-02-24T22:17:40.965 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.964+0000 7fe8eaf47a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:40.966 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.968+0000 7fe8eaf47a80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:17:40.966 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.968+0000 7fe8eaf47a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:40.967 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.968+0000 7fe8eaf47a80 20 rados_obj.operate() r=0 bl.length=495 2026-02-24T22:17:40.967 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.968+0000 7fe8eaf47a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:40.967 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.968+0000 7fe8eaf47a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:40.967 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.968+0000 7fe8eaf47a80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:17:40.967 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.968+0000 7fe8eaf47a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:40.967 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.968+0000 7fe8eaf47a80 20 rados_obj.operate() r=0 bl.length=495 2026-02-24T22:17:40.967 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:40.968+0000 7fe8eaf47a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:40.978 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:40.978 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:17:40.978 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-trim-reshard-log-entries id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:17:40.978 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:40.979 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:40.979 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:40.985 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:40.985 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:41.026 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket error-at-trim-reshard-log-entries --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2026-02-24T22:17:41.043 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:41.043 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:41.061 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.060+0000 7f05570e9a80 4 Realm: () 2026-02-24T22:17:41.061 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.060+0000 7f05570e9a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:41.061 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.060+0000 7f05570e9a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:41.061 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.060+0000 7f05570e9a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:41.061 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.060+0000 7f05570e9a80 20 zonegroup default 2026-02-24T22:17:41.062 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.064+0000 7f05570e9a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:41.062 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.064+0000 7f05570e9a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:41.062 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.064+0000 7f05570e9a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:41.063 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.064+0000 7f05570e9a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:41.063 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.064+0000 7f05570e9a80 20 started sync module instance, tier type = 2026-02-24T22:17:41.063 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.064+0000 7f05570e9a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:41.064 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.064+0000 7f05553be640 0 -- 10.20.193.46:0/2767638624 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55e50e112e50 msgr2=0x55e50e133230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:41.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.064+0000 7f05570e9a80 20 add_watcher() i=1 2026-02-24T22:17:41.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.064+0000 7f05570e9a80 20 add_watcher() i=6 2026-02-24T22:17:41.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.064+0000 7f05570e9a80 20 add_watcher() i=5 2026-02-24T22:17:41.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.064+0000 7f05570e9a80 20 add_watcher() i=3 2026-02-24T22:17:41.066 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.064+0000 7f05570e9a80 20 add_watcher() i=2 2026-02-24T22:17:41.066 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.064+0000 7f05570e9a80 20 add_watcher() i=4 2026-02-24T22:17:41.066 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.068+0000 7f05570e9a80 20 add_watcher() i=0 2026-02-24T22:17:41.066 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.068+0000 7f05570e9a80 20 add_watcher() i=7 2026-02-24T22:17:41.067 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.068+0000 7f05570e9a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:17:41.067 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.068+0000 7f054ffff640 0 -- 10.20.193.46:0/168494631 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f052409d3e0 msgr2=0x7f05240bd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:41.271 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.272+0000 7f05570e9a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:41.272 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.272+0000 7f05570e9a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:41.272 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.272+0000 7f05570e9a80 5 note: GC not initialized 2026-02-24T22:17:41.272 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.272+0000 7f0501fe3640 20 reqs_thread_entry: start 2026-02-24T22:17:41.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.308+0000 7f05570e9a80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:41.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.308+0000 7f05570e9a80 20 Filter name: none 2026-02-24T22:17:41.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.308+0000 7f045bfff640 20 reqs_thread_entry: start 2026-02-24T22:17:41.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.308+0000 7f05570e9a80 10 cache get: name=default.rgw.meta+root+error-at-trim-reshard-log-entries : miss 2026-02-24T22:17:41.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.308+0000 7f05570e9a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:41.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.308+0000 7f05570e9a80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:17:41.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.308+0000 7f05570e9a80 10 cache put: name=default.rgw.meta+root+error-at-trim-reshard-log-entries info.flags=0x11 2026-02-24T22:17:41.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.308+0000 7f05570e9a80 10 adding default.rgw.meta+root+error-at-trim-reshard-log-entries to cache LRU end 2026-02-24T22:17:41.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.308+0000 7f05570e9a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 : miss 2026-02-24T22:17:41.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.308+0000 7f05570e9a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:41.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.308+0000 7f05570e9a80 20 rados_obj.operate() r=0 bl.length=495 2026-02-24T22:17:41.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.308+0000 7f05570e9a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 info.flags=0x17 2026-02-24T22:17:41.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.308+0000 7f05570e9a80 10 adding default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 to cache LRU end 2026-02-24T22:17:41.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.308+0000 7f05570e9a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:17:41.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.308+0000 7f05570e9a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:41.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.308+0000 7f05570e9a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 2026-02-24T22:17:41.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.308+0000 7f05570e9a80 10 init_target_layout removing existing target index objects from a previous reshard attempt 2026-02-24T22:17:41.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.312+0000 7f05570e9a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 info.flags=0x17 2026-02-24T22:17:41.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.312+0000 7f05570e9a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 to cache LRU end 2026-02-24T22:17:41.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.312+0000 7f05570e9a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:41.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.312+0000 7f05570e9a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526, ofs0, ns] 2026-02-24T22:17:41.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.312+0000 7f0523fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554960010 cookie 94442272684880 notifier 6602 bl.length()=1041 2026-02-24T22:17:41.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.312+0000 7f0523fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 info.flags=0x17 2026-02-24T22:17:41.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.312+0000 7f0523fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 to cache LRU end 2026-02-24T22:17:41.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.312+0000 7f0523fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:41.314 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.312+0000 7f05570e9a80 20 handle_bi_update(): bucket=:error-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526]): orig_sources= new_sources= 2026-02-24T22:17:41.314 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.312+0000 7f05570e9a80 20 handle_bi_update(): bucket=:error-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526]): potential sources added=[] removed=[] 2026-02-24T22:17:41.314 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.312+0000 7f05570e9a80 20 handle_bi_update(): bucket=:error-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526]): orig_dests= new_dests= 2026-02-24T22:17:41.314 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.312+0000 7f05570e9a80 20 handle_bi_update(): bucket=:error-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526]): potential dests added=[] removed=[] 2026-02-24T22:17:41.315 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.316+0000 7f05570e9a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526.2.0 2026-02-24T22:17:41.315 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.316+0000 7f05570e9a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526.2.0 2026-02-24T22:17:41.315 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.316+0000 7f05570e9a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526.2.1 2026-02-24T22:17:41.315 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.316+0000 7f05570e9a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526.2.1 2026-02-24T22:17:41.315 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.316+0000 7f05570e9a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526.0 2026-02-24T22:17:41.317 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.316+0000 7f05570e9a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 : hit (requested=0x17, cached=0x17) 2026-02-24T22:17:41.317 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.316+0000 7f05570e9a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 2026-02-24T22:17:41.318 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05570e9a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 info.flags=0x17 2026-02-24T22:17:41.319 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05570e9a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 to cache LRU end 2026-02-24T22:17:41.319 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05570e9a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:41.319 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05570e9a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526, ofs0, ns] 2026-02-24T22:17:41.319 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05237fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554960011 cookie 94442272684880 notifier 6602 bl.length()=1041 2026-02-24T22:17:41.319 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05237fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 info.flags=0x17 2026-02-24T22:17:41.319 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05237fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 to cache LRU end 2026-02-24T22:17:41.319 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05237fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:41.320 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05570e9a80 20 handle_bi_update(): bucket=:error-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526]): orig_sources= new_sources= 2026-02-24T22:17:41.320 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05570e9a80 20 handle_bi_update(): bucket=:error-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526]): potential sources added=[] removed=[] 2026-02-24T22:17:41.320 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05570e9a80 20 handle_bi_update(): bucket=:error-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526]): orig_dests= new_dests= 2026-02-24T22:17:41.320 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05570e9a80 20 handle_bi_update(): bucket=:error-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526]): potential dests added=[] removed=[] 2026-02-24T22:17:41.320 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05570e9a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526.0 2026-02-24T22:17:41.320 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05570e9a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 : hit (requested=0x17, cached=0x17) 2026-02-24T22:17:41.321 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05570e9a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 2026-02-24T22:17:41.321 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05570e9a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 info.flags=0x17 2026-02-24T22:17:41.321 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05570e9a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 to cache LRU end 2026-02-24T22:17:41.322 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05570e9a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:41.322 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05570e9a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526, ofs0, ns] 2026-02-24T22:17:41.322 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05237fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554960012 cookie 94442272684880 notifier 6602 bl.length()=1005 2026-02-24T22:17:41.322 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05237fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 info.flags=0x17 2026-02-24T22:17:41.322 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05237fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 to cache LRU end 2026-02-24T22:17:41.322 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05237fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:41.322 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05570e9a80 20 handle_bi_update(): bucket=:error-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526]): orig_sources= new_sources= 2026-02-24T22:17:41.322 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05570e9a80 20 handle_bi_update(): bucket=:error-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526]): potential sources added=[] removed=[] 2026-02-24T22:17:41.322 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05570e9a80 20 handle_bi_update(): bucket=:error-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526]): orig_dests= new_dests= 2026-02-24T22:17:41.322 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.320+0000 7f05570e9a80 20 handle_bi_update(): bucket=:error-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526]): potential dests added=[] removed=[] 2026-02-24T22:17:41.323 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.324+0000 7f05570e9a80 1 execute INFO: reshard of bucket "error-at-trim-reshard-log-entries" from 1 shards to 2 shards completed successfully 2026-02-24T22:17:41.326 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.328+0000 7f05570e9a80 20 remove_watcher() i=2 2026-02-24T22:17:41.326 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.328+0000 7f05570e9a80 2 removed watcher, disabling cache 2026-02-24T22:17:41.327 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.328+0000 7f05570e9a80 20 remove_watcher() i=5 2026-02-24T22:17:41.327 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.328+0000 7f05570e9a80 20 remove_watcher() i=4 2026-02-24T22:17:41.327 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.328+0000 7f05570e9a80 20 remove_watcher() i=7 2026-02-24T22:17:41.327 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.328+0000 7f05570e9a80 20 remove_watcher() i=6 2026-02-24T22:17:41.327 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.328+0000 7f05570e9a80 20 remove_watcher() i=1 2026-02-24T22:17:41.327 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.328+0000 7f05570e9a80 20 remove_watcher() i=3 2026-02-24T22:17:41.327 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.328+0000 7f05570e9a80 20 remove_watcher() i=0 2026-02-24T22:17:41.333 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-trim-reshard-log-entries 2026-02-24T22:17:41.348 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:41.348 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:41.366 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.368+0000 7fab1b63fa80 4 Realm: () 2026-02-24T22:17:41.366 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.368+0000 7fab1b63fa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:41.366 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.368+0000 7fab1b63fa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:41.366 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.368+0000 7fab1b63fa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:41.367 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.368+0000 7fab1b63fa80 20 zonegroup default 2026-02-24T22:17:41.367 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.368+0000 7fab1b63fa80 20 rados->read ofs=0 len=0 2026-02-24T22:17:41.368 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.368+0000 7fab1b63fa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:41.368 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.368+0000 7fab1b63fa80 20 rados->read ofs=0 len=0 2026-02-24T22:17:41.369 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.368+0000 7fab1b63fa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:41.369 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.368+0000 7fab1b63fa80 20 started sync module instance, tier type = 2026-02-24T22:17:41.369 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.368+0000 7fab1b63fa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:41.371 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.372+0000 7fab1b63fa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:41.371 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.372+0000 7fab1b63fa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:41.371 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.372+0000 7fab1b63fa80 5 note: GC not initialized 2026-02-24T22:17:41.371 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.372+0000 7faac6fe5640 20 reqs_thread_entry: start 2026-02-24T22:17:41.374 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.376+0000 7fab19914640 0 -- 10.20.193.46:0/1636585321 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x563e47b49a00 msgr2=0x563e47b433c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:41.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.380+0000 7fab19113640 0 -- 10.20.193.46:0/1636585321 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x563e47b1e000 msgr2=0x563e47b4bbd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:41.401 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.400+0000 7fab1b63fa80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:41.401 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.400+0000 7fab1b63fa80 20 Filter name: none 2026-02-24T22:17:41.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.400+0000 7faac57e2640 20 reqs_thread_entry: start 2026-02-24T22:17:41.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.400+0000 7fab1b63fa80 20 rados->read ofs=0 len=0 2026-02-24T22:17:41.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.404+0000 7fab1b63fa80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:17:41.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.404+0000 7fab1b63fa80 20 rados->read ofs=0 len=0 2026-02-24T22:17:41.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.404+0000 7fab1b63fa80 20 rados_obj.operate() r=0 bl.length=459 2026-02-24T22:17:41.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.404+0000 7fab1b63fa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:41.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.404+0000 7fab1b63fa80 20 rados->read ofs=0 len=0 2026-02-24T22:17:41.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.404+0000 7fab1b63fa80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:17:41.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.404+0000 7fab1b63fa80 20 rados->read ofs=0 len=0 2026-02-24T22:17:41.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.404+0000 7fab1b63fa80 20 rados_obj.operate() r=0 bl.length=459 2026-02-24T22:17:41.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.404+0000 7fab1b63fa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:41.415 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:41.415 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:17:41.415 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-trim-reshard-log-entries id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526 num_objects: 19 size_kb: 1 num_shards: 2 2026-02-24T22:17:41.415 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:41.416 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:41.416 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:41.418 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:41.418 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:41.463 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:41.463 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:41.473 INFO:tasks.workunit.client.0.trial046.stderr:TEST: reshard bucket with EOPNOTSUPP injected at trim_reshard_log_entries 2026-02-24T22:17:41.473 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:41.474 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:41.474 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:41.482 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:41.482 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:41.527 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:41.527 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:41.533 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:41.533 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:41.538 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:41.538 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:41.544 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:41.544 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:41.550 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:41.550 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:41.556 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:41.556 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:41.562 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:41.562 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:41.568 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:41.568 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:41.573 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:41.574 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:41.580 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:41.580 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:41.586 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:41.586 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:41.591 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:41.591 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:41.597 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:41.597 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:41.603 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:41.603 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:41.609 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:41.609 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:41.615 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:41.615 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:41.620 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:41.621 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:41.626 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:41.626 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:41.632 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:41.632 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:41.638 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:41.638 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:41.643 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-trim-reshard-log-entries 2026-02-24T22:17:41.659 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:41.659 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:41.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.676+0000 7f2d593bba80 4 Realm: () 2026-02-24T22:17:41.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.676+0000 7f2d593bba80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:41.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.676+0000 7f2d593bba80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:41.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.676+0000 7f2d593bba80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:41.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.676+0000 7f2d593bba80 20 zonegroup default 2026-02-24T22:17:41.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.676+0000 7f2d593bba80 20 rados->read ofs=0 len=0 2026-02-24T22:17:41.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.680+0000 7f2d593bba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:41.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.680+0000 7f2d593bba80 20 rados->read ofs=0 len=0 2026-02-24T22:17:41.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.680+0000 7f2d593bba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:41.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.680+0000 7f2d593bba80 20 started sync module instance, tier type = 2026-02-24T22:17:41.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.680+0000 7f2d593bba80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:41.680 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.680+0000 7f2d593bba80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:41.680 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.680+0000 7f2d593bba80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:41.680 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.680+0000 7f2d593bba80 5 note: GC not initialized 2026-02-24T22:17:41.680 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.680+0000 7f2d057e2640 20 reqs_thread_entry: start 2026-02-24T22:17:41.684 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.684+0000 7f2d56e8f640 0 -- 10.20.193.46:0/3097570988 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x5569d34ceca0 msgr2=0x5569d34cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:17:41.894 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:41.896+0000 7f2d57690640 0 -- 10.20.193.46:0/3097570988 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x5569d34c0170 msgr2=0x5569d34c05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:17:42.123 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.124+0000 7f2d593bba80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:42.123 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.124+0000 7f2d593bba80 20 Filter name: none 2026-02-24T22:17:42.123 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.124+0000 7f2c6f7fe640 20 reqs_thread_entry: start 2026-02-24T22:17:42.123 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.124+0000 7f2d593bba80 20 rados->read ofs=0 len=0 2026-02-24T22:17:42.124 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.124+0000 7f2d593bba80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:17:42.124 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.124+0000 7f2d593bba80 20 rados->read ofs=0 len=0 2026-02-24T22:17:42.125 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.124+0000 7f2d593bba80 20 rados_obj.operate() r=0 bl.length=459 2026-02-24T22:17:42.125 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.124+0000 7f2d593bba80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:42.125 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.124+0000 7f2d593bba80 20 rados->read ofs=0 len=0 2026-02-24T22:17:42.125 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.124+0000 7f2d593bba80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:17:42.125 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.124+0000 7f2d593bba80 20 rados->read ofs=0 len=0 2026-02-24T22:17:42.126 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.124+0000 7f2d593bba80 20 rados_obj.operate() r=0 bl.length=459 2026-02-24T22:17:42.126 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.124+0000 7f2d593bba80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:42.136 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:42.137 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:17:42.137 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-trim-reshard-log-entries id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:17:42.137 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:42.137 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket error-at-trim-reshard-log-entries --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at trim_reshard_log_entries --inject-error-code 95 2026-02-24T22:17:42.153 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:42.153 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:42.172 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.172+0000 7facfffbba80 4 Realm: () 2026-02-24T22:17:42.172 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.172+0000 7facfffbba80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:42.172 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.172+0000 7facfffbba80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:42.172 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.172+0000 7facfffbba80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:42.172 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.172+0000 7facfffbba80 20 zonegroup default 2026-02-24T22:17:42.173 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.172+0000 7facfffbba80 20 rados->read ofs=0 len=0 2026-02-24T22:17:42.174 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.176+0000 7facfffbba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:42.174 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.176+0000 7facfffbba80 20 rados->read ofs=0 len=0 2026-02-24T22:17:42.174 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.176+0000 7facfffbba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:42.175 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.176+0000 7facfffbba80 20 started sync module instance, tier type = 2026-02-24T22:17:42.175 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.176+0000 7facfffbba80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:42.175 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.176+0000 7facfda8f640 0 -- 10.20.193.46:0/642153419 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x558b5d12a830 msgr2=0x558b5d14ac50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:42.176 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.176+0000 7facfffbba80 20 add_watcher() i=3 2026-02-24T22:17:42.176 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.176+0000 7facfffbba80 20 add_watcher() i=1 2026-02-24T22:17:42.176 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.176+0000 7facfffbba80 20 add_watcher() i=5 2026-02-24T22:17:42.176 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.176+0000 7facfffbba80 20 add_watcher() i=0 2026-02-24T22:17:42.177 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.176+0000 7facfffbba80 20 add_watcher() i=7 2026-02-24T22:17:42.177 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.176+0000 7facfffbba80 20 add_watcher() i=2 2026-02-24T22:17:42.177 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.176+0000 7facfffbba80 20 add_watcher() i=6 2026-02-24T22:17:42.177 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.176+0000 7facfffbba80 20 add_watcher() i=4 2026-02-24T22:17:42.177 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.176+0000 7facfffbba80 2 all 8 watchers are set, enabling cache 2026-02-24T22:17:42.179 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.180+0000 7facfffbba80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:42.179 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.180+0000 7facfffbba80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:42.179 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.180+0000 7facfffbba80 5 note: GC not initialized 2026-02-24T22:17:42.179 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.180+0000 7fac2e7fc640 20 reqs_thread_entry: start 2026-02-24T22:17:42.179 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.180+0000 7facfe290640 0 -- 10.20.193.46:0/1174327103 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x558b5d2a4b40 msgr2=0x558b5d2a4f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:42.419 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.420+0000 7facfffbba80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:42.420 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.420+0000 7facfffbba80 20 Filter name: none 2026-02-24T22:17:42.420 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.420+0000 7fac2cff9640 20 reqs_thread_entry: start 2026-02-24T22:17:42.420 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.420+0000 7facfffbba80 10 cache get: name=default.rgw.meta+root+error-at-trim-reshard-log-entries : miss 2026-02-24T22:17:42.420 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.420+0000 7facfffbba80 20 rados->read ofs=0 len=0 2026-02-24T22:17:42.420 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.420+0000 7facfffbba80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:17:42.420 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.420+0000 7facfffbba80 10 cache put: name=default.rgw.meta+root+error-at-trim-reshard-log-entries info.flags=0x11 2026-02-24T22:17:42.420 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.420+0000 7facfffbba80 10 adding default.rgw.meta+root+error-at-trim-reshard-log-entries to cache LRU end 2026-02-24T22:17:42.420 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.420+0000 7facfffbba80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527 : miss 2026-02-24T22:17:42.420 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.420+0000 7facfffbba80 20 rados->read ofs=0 len=0 2026-02-24T22:17:42.421 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.420+0000 7facfffbba80 20 rados_obj.operate() r=0 bl.length=459 2026-02-24T22:17:42.421 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.420+0000 7facfffbba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527 info.flags=0x17 2026-02-24T22:17:42.421 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.420+0000 7facfffbba80 10 adding default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527 to cache LRU end 2026-02-24T22:17:42.421 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.420+0000 7facfffbba80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:17:42.421 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.420+0000 7facfffbba80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:42.421 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.420+0000 7facfffbba80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527 2026-02-24T22:17:42.424 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.424+0000 7facfffbba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527 info.flags=0x17 2026-02-24T22:17:42.424 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.424+0000 7facfffbba80 10 moving default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527 to cache LRU end 2026-02-24T22:17:42.424 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.424+0000 7facfffbba80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:42.424 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.424+0000 7facfffbba80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527, ofs0, ns] 2026-02-24T22:17:42.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.424+0000 7facd8ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554960015 cookie 94057052479728 notifier 7104 bl.length()=1041 2026-02-24T22:17:42.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.424+0000 7facd8ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527 info.flags=0x17 2026-02-24T22:17:42.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.424+0000 7facd8ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527 to cache LRU end 2026-02-24T22:17:42.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.424+0000 7facd8ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:42.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.424+0000 7facfffbba80 20 handle_bi_update(): bucket=:error-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527]): orig_sources= new_sources= 2026-02-24T22:17:42.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.424+0000 7facfffbba80 20 handle_bi_update(): bucket=:error-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527]): potential sources added=[] removed=[] 2026-02-24T22:17:42.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.424+0000 7facfffbba80 20 handle_bi_update(): bucket=:error-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527]): orig_dests= new_dests= 2026-02-24T22:17:42.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.424+0000 7facfffbba80 20 handle_bi_update(): bucket=:error-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527]): potential dests added=[] removed=[] 2026-02-24T22:17:42.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.424+0000 7facfffbba80 -1 Injecting error=-95 at location=trim_reshard_log_entries 2026-02-24T22:17:42.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.424+0000 7facfffbba80 0 WARNING: trim_reshard_log() does not supported logrecord, falling back to block reshard mode. 2026-02-24T22:17:42.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.428+0000 7facfffbba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527.1.0 2026-02-24T22:17:42.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.428+0000 7facfffbba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527.1.0 2026-02-24T22:17:42.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.428+0000 7facfffbba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527.1.1 2026-02-24T22:17:42.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.428+0000 7facfffbba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527.1.1 2026-02-24T22:17:42.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.428+0000 7facfffbba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527.0 2026-02-24T22:17:42.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.428+0000 7facfffbba80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527 : hit (requested=0x17, cached=0x17) 2026-02-24T22:17:42.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.428+0000 7facfffbba80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527 2026-02-24T22:17:42.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.428+0000 7facfffbba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527 info.flags=0x17 2026-02-24T22:17:42.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.428+0000 7facfffbba80 10 moving default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527 to cache LRU end 2026-02-24T22:17:42.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.428+0000 7facfffbba80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:42.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.428+0000 7facfffbba80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527, ofs0, ns] 2026-02-24T22:17:42.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.428+0000 7facd97fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554960016 cookie 94057052479728 notifier 7104 bl.length()=1005 2026-02-24T22:17:42.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.428+0000 7facd97fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527 info.flags=0x17 2026-02-24T22:17:42.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.428+0000 7facd97fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527 to cache LRU end 2026-02-24T22:17:42.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.428+0000 7facd97fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:42.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.428+0000 7facfffbba80 20 handle_bi_update(): bucket=:error-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527]): orig_sources= new_sources= 2026-02-24T22:17:42.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.428+0000 7facfffbba80 20 handle_bi_update(): bucket=:error-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527]): potential sources added=[] removed=[] 2026-02-24T22:17:42.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.428+0000 7facfffbba80 20 handle_bi_update(): bucket=:error-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527]): orig_dests= new_dests= 2026-02-24T22:17:42.430 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.428+0000 7facfffbba80 20 handle_bi_update(): bucket=:error-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527]): potential dests added=[] removed=[] 2026-02-24T22:17:42.430 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.428+0000 7facfffbba80 1 execute INFO: reshard of bucket "error-at-trim-reshard-log-entries" from 1 shards to 2 shards completed successfully 2026-02-24T22:17:42.434 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.436+0000 7facfffbba80 20 remove_watcher() i=3 2026-02-24T22:17:42.435 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.436+0000 7facfffbba80 2 removed watcher, disabling cache 2026-02-24T22:17:42.435 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.436+0000 7facfffbba80 20 remove_watcher() i=2 2026-02-24T22:17:42.435 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.436+0000 7facfffbba80 20 remove_watcher() i=1 2026-02-24T22:17:42.435 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.436+0000 7facfffbba80 20 remove_watcher() i=0 2026-02-24T22:17:42.435 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.436+0000 7facfffbba80 20 remove_watcher() i=4 2026-02-24T22:17:42.435 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.436+0000 7facfffbba80 20 remove_watcher() i=5 2026-02-24T22:17:42.435 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.436+0000 7facfffbba80 20 remove_watcher() i=7 2026-02-24T22:17:42.435 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.436+0000 7facfffbba80 20 remove_watcher() i=6 2026-02-24T22:17:42.442 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-trim-reshard-log-entries 2026-02-24T22:17:42.457 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:42.457 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:42.474 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.476+0000 7f23e12bda80 4 Realm: () 2026-02-24T22:17:42.475 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.476+0000 7f23e12bda80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:42.475 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.476+0000 7f23e12bda80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:42.475 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.476+0000 7f23e12bda80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:42.475 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.476+0000 7f23e12bda80 20 zonegroup default 2026-02-24T22:17:42.475 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.476+0000 7f23e12bda80 20 rados->read ofs=0 len=0 2026-02-24T22:17:42.476 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.476+0000 7f23e12bda80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:42.477 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.476+0000 7f23e12bda80 20 rados->read ofs=0 len=0 2026-02-24T22:17:42.477 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.476+0000 7f23e12bda80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:42.477 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.476+0000 7f23e12bda80 20 started sync module instance, tier type = 2026-02-24T22:17:42.477 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.476+0000 7f23e12bda80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:42.479 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.480+0000 7f23e12bda80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:42.479 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.480+0000 7f23e12bda80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:42.479 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.480+0000 7f23e12bda80 5 note: GC not initialized 2026-02-24T22:17:42.479 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.480+0000 7f238d7e2640 20 reqs_thread_entry: start 2026-02-24T22:17:42.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.484+0000 7f23df592640 0 -- 10.20.193.46:0/2052858244 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x5626f7935540 msgr2=0x5626f79287b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:42.488 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.488+0000 7f23ded91640 0 -- 10.20.193.46:0/2052858244 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5626f7930fc0 msgr2=0x5626f792bd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:42.510 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.512+0000 7f23e12bda80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:42.510 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.512+0000 7f23e12bda80 20 Filter name: none 2026-02-24T22:17:42.511 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.512+0000 7f22f77fe640 20 reqs_thread_entry: start 2026-02-24T22:17:42.511 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.512+0000 7f23e12bda80 20 rados->read ofs=0 len=0 2026-02-24T22:17:42.512 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.512+0000 7f23e12bda80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:17:42.512 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.512+0000 7f23e12bda80 20 rados->read ofs=0 len=0 2026-02-24T22:17:42.513 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.512+0000 7f23e12bda80 20 rados_obj.operate() r=0 bl.length=459 2026-02-24T22:17:42.513 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.512+0000 7f23e12bda80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:42.513 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.512+0000 7f23e12bda80 20 rados->read ofs=0 len=0 2026-02-24T22:17:42.513 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.512+0000 7f23e12bda80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:17:42.513 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.512+0000 7f23e12bda80 20 rados->read ofs=0 len=0 2026-02-24T22:17:42.513 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.512+0000 7f23e12bda80 20 rados_obj.operate() r=0 bl.length=459 2026-02-24T22:17:42.513 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.512+0000 7f23e12bda80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:42.524 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:42.524 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:17:42.524 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-trim-reshard-log-entries id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527 num_objects: 20 size_kb: 1 num_shards: 2 2026-02-24T22:17:42.524 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:42.525 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:42.526 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:42.528 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:42.528 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:42.571 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:42.571 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:42.581 INFO:tasks.workunit.client.0.trial046.stderr:TEST: reshard bucket with abort at trim_reshard_log_entries 2026-02-24T22:17:42.582 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:42.582 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:42.583 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:42.591 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:42.591 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:42.635 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:42.635 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:42.641 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:42.641 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:42.649 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:42.649 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:42.655 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:42.655 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:42.660 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:42.660 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:42.666 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:42.666 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:42.671 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:42.671 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:42.677 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:42.677 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:42.682 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:42.683 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:42.688 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:42.688 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:42.694 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:42.694 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:42.701 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:42.701 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:42.707 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:42.707 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:42.713 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:42.713 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:42.719 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:42.719 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:42.726 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:42.726 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:42.732 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:42.732 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:42.738 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:42.738 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:42.743 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:42.743 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:42.749 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:42.749 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:42.754 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket abort-at-trim-reshard-log-entries 2026-02-24T22:17:42.770 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:42.770 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:42.787 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.788+0000 7fd522418a80 4 Realm: () 2026-02-24T22:17:42.788 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.788+0000 7fd522418a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:42.788 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.788+0000 7fd522418a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:42.788 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.788+0000 7fd522418a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:42.788 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.788+0000 7fd522418a80 20 zonegroup default 2026-02-24T22:17:42.788 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.788+0000 7fd522418a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:42.991 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.992+0000 7fd522418a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:42.991 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.992+0000 7fd522418a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:42.992 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.992+0000 7fd522418a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:42.992 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.992+0000 7fd522418a80 20 started sync module instance, tier type = 2026-02-24T22:17:42.992 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.992+0000 7fd522418a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:42.994 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.996+0000 7fd522418a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:42.994 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.996+0000 7fd522418a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:42.994 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.996+0000 7fd522418a80 5 note: GC not initialized 2026-02-24T22:17:42.995 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:42.996+0000 7fd4cd7e2640 20 reqs_thread_entry: start 2026-02-24T22:17:42.998 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.000+0000 7fd51bfff640 0 -- 10.20.193.46:0/3909380959 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55b1cba71810 msgr2=0x55b1cba6b1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:43.412 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.412+0000 7fd513fff640 0 -- 10.20.193.46:0/3909380959 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x55b1cba5ac80 msgr2=0x55b1cba5b0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:43.439 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.440+0000 7fd522418a80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:43.439 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.440+0000 7fd522418a80 20 Filter name: none 2026-02-24T22:17:43.439 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.440+0000 7fd4377fe640 20 reqs_thread_entry: start 2026-02-24T22:17:43.440 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.440+0000 7fd522418a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:43.440 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.440+0000 7fd522418a80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:17:43.440 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.440+0000 7fd522418a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:43.441 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.440+0000 7fd522418a80 20 rados_obj.operate() r=0 bl.length=459 2026-02-24T22:17:43.441 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.440+0000 7fd522418a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:43.441 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.440+0000 7fd522418a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:43.441 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.440+0000 7fd522418a80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:17:43.441 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.440+0000 7fd522418a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:43.441 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.440+0000 7fd522418a80 20 rados_obj.operate() r=0 bl.length=459 2026-02-24T22:17:43.441 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.440+0000 7fd522418a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:43.451 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.452 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:17:43.452 INFO:tasks.workunit.client.0.trial046.stderr:bucket: abort-at-trim-reshard-log-entries id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:17:43.452 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.452 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket abort-at-trim-reshard-log-entries --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at trim_reshard_log_entries 2026-02-24T22:17:43.468 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:43.468 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:43.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.484+0000 7f8da3bf1a80 4 Realm: () 2026-02-24T22:17:43.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.484+0000 7f8da3bf1a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:43.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.484+0000 7f8da3bf1a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:43.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:43.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.484+0000 7f8da3bf1a80 20 zonegroup default 2026-02-24T22:17:43.486 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:43.487 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:43.487 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:43.487 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:43.488 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 started sync module instance, tier type = 2026-02-24T22:17:43.488 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:43.488 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.488+0000 7f8da16c5640 0 -- 10.20.193.46:0/3123648294 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x558d512114b0 msgr2=0x558d51231910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:43.489 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 add_watcher() i=1 2026-02-24T22:17:43.490 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 add_watcher() i=5 2026-02-24T22:17:43.490 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 add_watcher() i=3 2026-02-24T22:17:43.490 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 add_watcher() i=0 2026-02-24T22:17:43.490 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 add_watcher() i=6 2026-02-24T22:17:43.490 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.492+0000 7f8da3bf1a80 20 add_watcher() i=2 2026-02-24T22:17:43.490 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.492+0000 7f8da3bf1a80 20 add_watcher() i=4 2026-02-24T22:17:43.491 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.492+0000 7f8da3bf1a80 20 add_watcher() i=7 2026-02-24T22:17:43.491 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.492+0000 7f8da3bf1a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:17:43.491 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.492+0000 7f8da0ec4640 0 -- 10.20.193.46:0/2381496400 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f8cc0008af0 msgr2=0x7f8cc0028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:43.492 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.492+0000 7f8da3bf1a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:43.492 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.492+0000 7f8da3bf1a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:43.492 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 note: GC not initialized 2026-02-24T22:17:43.493 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.492+0000 7f8d4e7e4640 20 reqs_thread_entry: start 2026-02-24T22:17:43.526 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.528+0000 7f8da3bf1a80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:43.526 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.528+0000 7f8da3bf1a80 20 Filter name: none 2026-02-24T22:17:43.526 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.528+0000 7f8d4cfe1640 20 reqs_thread_entry: start 2026-02-24T22:17:43.526 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 cache get: name=default.rgw.meta+root+abort-at-trim-reshard-log-entries : miss 2026-02-24T22:17:43.526 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.528+0000 7f8da3bf1a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:43.527 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.528+0000 7f8da3bf1a80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:17:43.527 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 cache put: name=default.rgw.meta+root+abort-at-trim-reshard-log-entries info.flags=0x11 2026-02-24T22:17:43.527 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 adding default.rgw.meta+root+abort-at-trim-reshard-log-entries to cache LRU end 2026-02-24T22:17:43.527 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 : miss 2026-02-24T22:17:43.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.528+0000 7f8da3bf1a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:43.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.528+0000 7f8da3bf1a80 20 rados_obj.operate() r=0 bl.length=459 2026-02-24T22:17:43.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 info.flags=0x17 2026-02-24T22:17:43.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 to cache LRU end 2026-02-24T22:17:43.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:17:43.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:43.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 2026-02-24T22:17:43.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.532+0000 7f8da3bf1a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 info.flags=0x17 2026-02-24T22:17:43.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.532+0000 7f8da3bf1a80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 to cache LRU end 2026-02-24T22:17:43.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.532+0000 7f8da3bf1a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:43.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.532+0000 7f8da3bf1a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528, ofs0, ns] 2026-02-24T22:17:43.532 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.532+0000 7f8d7cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961346 cookie 94065439058112 notifier 6704 bl.length()=1041 2026-02-24T22:17:43.532 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.532+0000 7f8d7cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 info.flags=0x17 2026-02-24T22:17:43.532 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.532+0000 7f8d7cff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 to cache LRU end 2026-02-24T22:17:43.532 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.532+0000 7f8d7cff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:43.532 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.532+0000 7f8da3bf1a80 20 handle_bi_update(): bucket=:abort-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528]): orig_sources= new_sources= 2026-02-24T22:17:43.532 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.532+0000 7f8da3bf1a80 20 handle_bi_update(): bucket=:abort-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528]): potential sources added=[] removed=[] 2026-02-24T22:17:43.532 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.532+0000 7f8da3bf1a80 20 handle_bi_update(): bucket=:abort-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528]): orig_dests= new_dests= 2026-02-24T22:17:43.532 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.532+0000 7f8da3bf1a80 20 handle_bi_update(): bucket=:abort-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528]): potential dests added=[] removed=[] 2026-02-24T22:17:43.532 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f8da3bf1a80 time 2026-02-24T22:17:43.537470+0000 2026-02-24T22:17:43.532 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:17:43.544 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:17:43.545 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f8da7033eb1] 2026-02-24T22:17:43.545 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x558d0fbfab9f] 2026-02-24T22:17:43.545 INFO:tasks.workunit.client.0.trial046.stderr: 3: radosgw-admin(+0x17143e1) [0x558d10e093e1] 2026-02-24T22:17:43.545 INFO:tasks.workunit.client.0.trial046.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x558d10366202] 2026-02-24T22:17:43.545 INFO:tasks.workunit.client.0.trial046.stderr: 5: main() 2026-02-24T22:17:43.545 INFO:tasks.workunit.client.0.trial046.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f8da6333d90] 2026-02-24T22:17:43.545 INFO:tasks.workunit.client.0.trial046.stderr: 7: __libc_start_main() 2026-02-24T22:17:43.545 INFO:tasks.workunit.client.0.trial046.stderr: 8: _start() 2026-02-24T22:17:43.545 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.532+0000 7f8da3bf1a80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f8da3bf1a80 time 2026-02-24T22:17:43.537470+0000 2026-02-24T22:17:43.545 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:17:43.546 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.546 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:17:43.546 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f8da7033eb1] 2026-02-24T22:17:43.546 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x558d0fbfab9f] 2026-02-24T22:17:43.546 INFO:tasks.workunit.client.0.trial046.stderr: 3: radosgw-admin(+0x17143e1) [0x558d10e093e1] 2026-02-24T22:17:43.546 INFO:tasks.workunit.client.0.trial046.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x558d10366202] 2026-02-24T22:17:43.546 INFO:tasks.workunit.client.0.trial046.stderr: 5: main() 2026-02-24T22:17:43.546 INFO:tasks.workunit.client.0.trial046.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f8da6333d90] 2026-02-24T22:17:43.546 INFO:tasks.workunit.client.0.trial046.stderr: 7: __libc_start_main() 2026-02-24T22:17:43.546 INFO:tasks.workunit.client.0.trial046.stderr: 8: _start() 2026-02-24T22:17:43.546 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.546 INFO:tasks.workunit.client.0.trial046.stderr:*** Caught signal (Aborted) ** 2026-02-24T22:17:43.547 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f8da3bf1a80 thread_name:radosgw-admin 2026-02-24T22:17:43.547 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:17:43.547 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f8da634c520] 2026-02-24T22:17:43.547 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:17:43.547 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:17:43.547 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:17:43.547 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f8da7033f10] 2026-02-24T22:17:43.547 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x558d0fbfab9f] 2026-02-24T22:17:43.547 INFO:tasks.workunit.client.0.trial046.stderr: 7: radosgw-admin(+0x17143e1) [0x558d10e093e1] 2026-02-24T22:17:43.547 INFO:tasks.workunit.client.0.trial046.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x558d10366202] 2026-02-24T22:17:43.547 INFO:tasks.workunit.client.0.trial046.stderr: 9: main() 2026-02-24T22:17:43.547 INFO:tasks.workunit.client.0.trial046.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f8da6333d90] 2026-02-24T22:17:43.548 INFO:tasks.workunit.client.0.trial046.stderr: 11: __libc_start_main() 2026-02-24T22:17:43.548 INFO:tasks.workunit.client.0.trial046.stderr: 12: _start() 2026-02-24T22:17:43.548 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.532+0000 7f8da3bf1a80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:17:43.548 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f8da3bf1a80 thread_name:radosgw-admin 2026-02-24T22:17:43.548 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.548 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:17:43.548 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f8da634c520] 2026-02-24T22:17:43.548 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:17:43.548 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:17:43.548 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:17:43.548 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f8da7033f10] 2026-02-24T22:17:43.548 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x558d0fbfab9f] 2026-02-24T22:17:43.548 INFO:tasks.workunit.client.0.trial046.stderr: 7: radosgw-admin(+0x17143e1) [0x558d10e093e1] 2026-02-24T22:17:43.549 INFO:tasks.workunit.client.0.trial046.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x558d10366202] 2026-02-24T22:17:43.549 INFO:tasks.workunit.client.0.trial046.stderr: 9: main() 2026-02-24T22:17:43.549 INFO:tasks.workunit.client.0.trial046.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f8da6333d90] 2026-02-24T22:17:43.549 INFO:tasks.workunit.client.0.trial046.stderr: 11: __libc_start_main() 2026-02-24T22:17:43.549 INFO:tasks.workunit.client.0.trial046.stderr: 12: _start() 2026-02-24T22:17:43.549 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:17:43.549 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.549 INFO:tasks.workunit.client.0.trial046.stderr:--- begin dump of recent events --- 2026-02-24T22:17:43.549 INFO:tasks.workunit.client.0.trial046.stderr: -499> 2026-02-24T22:17:43.472+0000 7f8da16c5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:43.549 INFO:tasks.workunit.client.0.trial046.stderr: -498> 2026-02-24T22:17:43.472+0000 7f8da16c5640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:43.549 INFO:tasks.workunit.client.0.trial046.stderr: -497> 2026-02-24T22:17:43.472+0000 7f8da16c5640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:43.549 INFO:tasks.workunit.client.0.trial046.stderr: -496> 2026-02-24T22:17:43.472+0000 7f8da1ec6640 10 monclient(hunting): get_auth_request con 0x558d50cf2c20 auth_method 0 2026-02-24T22:17:43.550 INFO:tasks.workunit.client.0.trial046.stderr: -495> 2026-02-24T22:17:43.472+0000 7f8da1ec6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:43.550 INFO:tasks.workunit.client.0.trial046.stderr: -494> 2026-02-24T22:17:43.472+0000 7f8da1ec6640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:43.550 INFO:tasks.workunit.client.0.trial046.stderr: -493> 2026-02-24T22:17:43.472+0000 7f8da1ec6640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:43.550 INFO:tasks.workunit.client.0.trial046.stderr: -492> 2026-02-24T22:17:43.472+0000 7f8da16c5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:43.550 INFO:tasks.workunit.client.0.trial046.stderr: -491> 2026-02-24T22:17:43.472+0000 7f8da16c5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:43.550 INFO:tasks.workunit.client.0.trial046.stderr: -490> 2026-02-24T22:17:43.472+0000 7f8da16c5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:43.550 INFO:tasks.workunit.client.0.trial046.stderr: -489> 2026-02-24T22:17:43.472+0000 7f8da1ec6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:43.550 INFO:tasks.workunit.client.0.trial046.stderr: -488> 2026-02-24T22:17:43.472+0000 7f8da1ec6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:43.550 INFO:tasks.workunit.client.0.trial046.stderr: -487> 2026-02-24T22:17:43.472+0000 7f8da1ec6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:43.550 INFO:tasks.workunit.client.0.trial046.stderr: -486> 2026-02-24T22:17:43.472+0000 7f8da16c5640 10 monclient(hunting): handle_auth_done global_id 6692 payload 274 2026-02-24T22:17:43.550 INFO:tasks.workunit.client.0.trial046.stderr: -485> 2026-02-24T22:17:43.472+0000 7f8da16c5640 10 monclient: _finish_hunting 0 2026-02-24T22:17:43.550 INFO:tasks.workunit.client.0.trial046.stderr: -484> 2026-02-24T22:17:43.472+0000 7f8da16c5640 1 monclient: found mon.noname-c 2026-02-24T22:17:43.550 INFO:tasks.workunit.client.0.trial046.stderr: -483> 2026-02-24T22:17:43.472+0000 7f8da16c5640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:17:43.551 INFO:tasks.workunit.client.0.trial046.stderr: -482> 2026-02-24T22:17:43.472+0000 7f8d877fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:43.551 INFO:tasks.workunit.client.0.trial046.stderr: -481> 2026-02-24T22:17:43.472+0000 7f8d877fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:17:43.551 INFO:tasks.workunit.client.0.trial046.stderr: -480> 2026-02-24T22:17:43.472+0000 7f8d877fe640 10 monclient: dump: 2026-02-24T22:17:43.551 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:43.551 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:43.551 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.551 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.551 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:43.551 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.551 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:43.551 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:43.551 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:43.552 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.552 INFO:tasks.workunit.client.0.trial046.stderr: -479> 2026-02-24T22:17:43.472+0000 7f8d877fe640 10 monclient: _finish_auth 0 2026-02-24T22:17:43.552 INFO:tasks.workunit.client.0.trial046.stderr: -478> 2026-02-24T22:17:43.472+0000 7f8d877fe640 10 monclient: _check_auth_tickets 2026-02-24T22:17:43.552 INFO:tasks.workunit.client.0.trial046.stderr: -477> 2026-02-24T22:17:43.472+0000 7f8d877fe640 10 monclient: handle_config config(0 keys) 2026-02-24T22:17:43.552 INFO:tasks.workunit.client.0.trial046.stderr: -476> 2026-02-24T22:17:43.472+0000 7f8d877fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:43.552 INFO:tasks.workunit.client.0.trial046.stderr: -475> 2026-02-24T22:17:43.472+0000 7f8d877fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:17:43.552 INFO:tasks.workunit.client.0.trial046.stderr: -474> 2026-02-24T22:17:43.472+0000 7f8d877fe640 10 monclient: dump: 2026-02-24T22:17:43.552 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:43.552 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:43.552 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.552 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.552 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:43.552 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.553 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:43.553 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:43.553 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:43.553 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.553 INFO:tasks.workunit.client.0.trial046.stderr: -473> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 monclient: authenticate success, global_id 6692 2026-02-24T22:17:43.553 INFO:tasks.workunit.client.0.trial046.stderr: -472> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 10 monclient: get_monmap_and_config success 2026-02-24T22:17:43.553 INFO:tasks.workunit.client.0.trial046.stderr: -471> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 4 set_mon_vals no callback set 2026-02-24T22:17:43.553 INFO:tasks.workunit.client.0.trial046.stderr: -470> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 10 monclient: shutdown 2026-02-24T22:17:43.553 INFO:tasks.workunit.client.0.trial046.stderr: -469> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) unregister_commands rotate-key 2026-02-24T22:17:43.553 INFO:tasks.workunit.client.0.trial046.stderr: -468> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 10 monclient: build_initial_monmap 2026-02-24T22:17:43.553 INFO:tasks.workunit.client.0.trial046.stderr: -467> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 1 build_initial for_mkfs: 0 2026-02-24T22:17:43.553 INFO:tasks.workunit.client.0.trial046.stderr: -466> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 10 monclient: monmap: 2026-02-24T22:17:43.553 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:17:43.554 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:17:43.554 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:17:43.554 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:17:43.554 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:17:43.554 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.554 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:17:43.554 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:17:43.554 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:17:43.554 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.554 INFO:tasks.workunit.client.0.trial046.stderr: -465> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding auth protocol: cephx 2026-02-24T22:17:43.554 INFO:tasks.workunit.client.0.trial046.stderr: -464> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding auth protocol: cephx 2026-02-24T22:17:43.555 INFO:tasks.workunit.client.0.trial046.stderr: -463> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding auth protocol: cephx 2026-02-24T22:17:43.555 INFO:tasks.workunit.client.0.trial046.stderr: -462> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding con mode: secure 2026-02-24T22:17:43.555 INFO:tasks.workunit.client.0.trial046.stderr: -461> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding con mode: crc 2026-02-24T22:17:43.555 INFO:tasks.workunit.client.0.trial046.stderr: -460> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding con mode: secure 2026-02-24T22:17:43.555 INFO:tasks.workunit.client.0.trial046.stderr: -459> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding con mode: crc 2026-02-24T22:17:43.555 INFO:tasks.workunit.client.0.trial046.stderr: -458> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding con mode: secure 2026-02-24T22:17:43.555 INFO:tasks.workunit.client.0.trial046.stderr: -457> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding con mode: crc 2026-02-24T22:17:43.555 INFO:tasks.workunit.client.0.trial046.stderr: -456> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding con mode: crc 2026-02-24T22:17:43.555 INFO:tasks.workunit.client.0.trial046.stderr: -455> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding con mode: secure 2026-02-24T22:17:43.555 INFO:tasks.workunit.client.0.trial046.stderr: -454> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding con mode: crc 2026-02-24T22:17:43.555 INFO:tasks.workunit.client.0.trial046.stderr: -453> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding con mode: secure 2026-02-24T22:17:43.555 INFO:tasks.workunit.client.0.trial046.stderr: -452> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding con mode: crc 2026-02-24T22:17:43.556 INFO:tasks.workunit.client.0.trial046.stderr: -451> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding con mode: secure 2026-02-24T22:17:43.556 INFO:tasks.workunit.client.0.trial046.stderr: -450> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:43.556 INFO:tasks.workunit.client.0.trial046.stderr: -449> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 1 librados: starting msgr at 2026-02-24T22:17:43.556 INFO:tasks.workunit.client.0.trial046.stderr: -448> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 1 librados: starting objecter 2026-02-24T22:17:43.556 INFO:tasks.workunit.client.0.trial046.stderr: -447> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command objecter_requests hook 0x558d50cc29e0 2026-02-24T22:17:43.556 INFO:tasks.workunit.client.0.trial046.stderr: -446> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 1 librados: setting wanted keys 2026-02-24T22:17:43.556 INFO:tasks.workunit.client.0.trial046.stderr: -445> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 1 librados: calling monclient init 2026-02-24T22:17:43.556 INFO:tasks.workunit.client.0.trial046.stderr: -444> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 10 monclient: init 2026-02-24T22:17:43.556 INFO:tasks.workunit.client.0.trial046.stderr: -443> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding auth protocol: cephx 2026-02-24T22:17:43.556 INFO:tasks.workunit.client.0.trial046.stderr: -442> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding auth protocol: cephx 2026-02-24T22:17:43.556 INFO:tasks.workunit.client.0.trial046.stderr: -441> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding auth protocol: cephx 2026-02-24T22:17:43.556 INFO:tasks.workunit.client.0.trial046.stderr: -440> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding con mode: secure 2026-02-24T22:17:43.556 INFO:tasks.workunit.client.0.trial046.stderr: -439> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding con mode: crc 2026-02-24T22:17:43.557 INFO:tasks.workunit.client.0.trial046.stderr: -438> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding con mode: secure 2026-02-24T22:17:43.557 INFO:tasks.workunit.client.0.trial046.stderr: -437> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding con mode: crc 2026-02-24T22:17:43.557 INFO:tasks.workunit.client.0.trial046.stderr: -436> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding con mode: secure 2026-02-24T22:17:43.557 INFO:tasks.workunit.client.0.trial046.stderr: -435> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding con mode: crc 2026-02-24T22:17:43.557 INFO:tasks.workunit.client.0.trial046.stderr: -434> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding con mode: crc 2026-02-24T22:17:43.557 INFO:tasks.workunit.client.0.trial046.stderr: -433> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding con mode: secure 2026-02-24T22:17:43.557 INFO:tasks.workunit.client.0.trial046.stderr: -432> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding con mode: crc 2026-02-24T22:17:43.557 INFO:tasks.workunit.client.0.trial046.stderr: -431> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding con mode: secure 2026-02-24T22:17:43.557 INFO:tasks.workunit.client.0.trial046.stderr: -430> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding con mode: crc 2026-02-24T22:17:43.557 INFO:tasks.workunit.client.0.trial046.stderr: -429> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding con mode: secure 2026-02-24T22:17:43.557 INFO:tasks.workunit.client.0.trial046.stderr: -428> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:43.557 INFO:tasks.workunit.client.0.trial046.stderr: -427> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:43.557 INFO:tasks.workunit.client.0.trial046.stderr: -426> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command rotate-key hook 0x558d50cbe838 2026-02-24T22:17:43.558 INFO:tasks.workunit.client.0.trial046.stderr: -425> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient: _reopen_session rank -1 2026-02-24T22:17:43.558 INFO:tasks.workunit.client.0.trial046.stderr: -424> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T22:17:43.558 INFO:tasks.workunit.client.0.trial046.stderr: -423> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient(hunting): picked mon.noname-a con 0x558d50c487a0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:17:43.558 INFO:tasks.workunit.client.0.trial046.stderr: -422> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient(hunting): picked mon.noname-c con 0x558d50c4b010 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:17:43.558 INFO:tasks.workunit.client.0.trial046.stderr: -421> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient(hunting): picked mon.noname-b con 0x558d50cf2c20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:17:43.558 INFO:tasks.workunit.client.0.trial046.stderr: -420> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:43.558 INFO:tasks.workunit.client.0.trial046.stderr: -419> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:43.558 INFO:tasks.workunit.client.0.trial046.stderr: -418> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:43.558 INFO:tasks.workunit.client.0.trial046.stderr: -417> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient(hunting): _renew_subs 2026-02-24T22:17:43.558 INFO:tasks.workunit.client.0.trial046.stderr: -416> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient(hunting): authenticate will time out at 1833.584129s 2026-02-24T22:17:43.558 INFO:tasks.workunit.client.0.trial046.stderr: -415> 2026-02-24T22:17:43.476+0000 7f8da16c5640 10 monclient(hunting): get_auth_request con 0x558d50c487a0 auth_method 0 2026-02-24T22:17:43.558 INFO:tasks.workunit.client.0.trial046.stderr: -414> 2026-02-24T22:17:43.476+0000 7f8da16c5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:43.558 INFO:tasks.workunit.client.0.trial046.stderr: -413> 2026-02-24T22:17:43.476+0000 7f8da16c5640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:43.559 INFO:tasks.workunit.client.0.trial046.stderr: -412> 2026-02-24T22:17:43.476+0000 7f8da16c5640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:43.559 INFO:tasks.workunit.client.0.trial046.stderr: -411> 2026-02-24T22:17:43.476+0000 7f8da0ec4640 10 monclient(hunting): get_auth_request con 0x558d50c4b010 auth_method 0 2026-02-24T22:17:43.559 INFO:tasks.workunit.client.0.trial046.stderr: -410> 2026-02-24T22:17:43.476+0000 7f8da0ec4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:43.559 INFO:tasks.workunit.client.0.trial046.stderr: -409> 2026-02-24T22:17:43.476+0000 7f8da0ec4640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:43.559 INFO:tasks.workunit.client.0.trial046.stderr: -408> 2026-02-24T22:17:43.476+0000 7f8da0ec4640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:43.559 INFO:tasks.workunit.client.0.trial046.stderr: -407> 2026-02-24T22:17:43.476+0000 7f8da16c5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:43.559 INFO:tasks.workunit.client.0.trial046.stderr: -406> 2026-02-24T22:17:43.476+0000 7f8da16c5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:43.559 INFO:tasks.workunit.client.0.trial046.stderr: -405> 2026-02-24T22:17:43.476+0000 7f8da16c5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:43.559 INFO:tasks.workunit.client.0.trial046.stderr: -404> 2026-02-24T22:17:43.476+0000 7f8da0ec4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:43.559 INFO:tasks.workunit.client.0.trial046.stderr: -403> 2026-02-24T22:17:43.476+0000 7f8da0ec4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:43.559 INFO:tasks.workunit.client.0.trial046.stderr: -402> 2026-02-24T22:17:43.476+0000 7f8da0ec4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:43.559 INFO:tasks.workunit.client.0.trial046.stderr: -401> 2026-02-24T22:17:43.476+0000 7f8da1ec6640 10 monclient(hunting): get_auth_request con 0x558d50cf2c20 auth_method 0 2026-02-24T22:17:43.559 INFO:tasks.workunit.client.0.trial046.stderr: -400> 2026-02-24T22:17:43.476+0000 7f8da1ec6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:43.559 INFO:tasks.workunit.client.0.trial046.stderr: -399> 2026-02-24T22:17:43.476+0000 7f8da1ec6640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:43.560 INFO:tasks.workunit.client.0.trial046.stderr: -398> 2026-02-24T22:17:43.476+0000 7f8da1ec6640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:43.560 INFO:tasks.workunit.client.0.trial046.stderr: -397> 2026-02-24T22:17:43.476+0000 7f8da16c5640 10 monclient(hunting): handle_auth_done global_id 7149 payload 824 2026-02-24T22:17:43.560 INFO:tasks.workunit.client.0.trial046.stderr: -396> 2026-02-24T22:17:43.476+0000 7f8da16c5640 10 monclient: _finish_hunting 0 2026-02-24T22:17:43.560 INFO:tasks.workunit.client.0.trial046.stderr: -395> 2026-02-24T22:17:43.476+0000 7f8da16c5640 1 monclient: found mon.noname-a 2026-02-24T22:17:43.560 INFO:tasks.workunit.client.0.trial046.stderr: -394> 2026-02-24T22:17:43.476+0000 7f8da16c5640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:17:43.560 INFO:tasks.workunit.client.0.trial046.stderr: -393> 2026-02-24T22:17:43.476+0000 7f8d85ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:43.560 INFO:tasks.workunit.client.0.trial046.stderr: -392> 2026-02-24T22:17:43.476+0000 7f8d85ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:17:43.560 INFO:tasks.workunit.client.0.trial046.stderr: -391> 2026-02-24T22:17:43.476+0000 7f8d85ffb640 10 monclient: dump: 2026-02-24T22:17:43.560 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:43.560 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:43.560 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.560 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.560 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:43.561 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.561 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:43.561 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:43.561 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:43.561 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.561 INFO:tasks.workunit.client.0.trial046.stderr: -390> 2026-02-24T22:17:43.476+0000 7f8d85ffb640 10 monclient: _finish_auth 0 2026-02-24T22:17:43.561 INFO:tasks.workunit.client.0.trial046.stderr: -389> 2026-02-24T22:17:43.476+0000 7f8d85ffb640 10 monclient: _check_auth_tickets 2026-02-24T22:17:43.561 INFO:tasks.workunit.client.0.trial046.stderr: -388> 2026-02-24T22:17:43.476+0000 7f8d85ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T22:17:43.561 INFO:tasks.workunit.client.0.trial046.stderr: -387> 2026-02-24T22:17:43.476+0000 7f8d85ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:43.561 INFO:tasks.workunit.client.0.trial046.stderr: -386> 2026-02-24T22:17:43.476+0000 7f8d85ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:17:43.561 INFO:tasks.workunit.client.0.trial046.stderr: -385> 2026-02-24T22:17:43.476+0000 7f8d85ffb640 10 monclient: dump: 2026-02-24T22:17:43.561 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:43.561 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:43.562 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.562 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.562 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:43.562 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.562 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:43.562 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:43.562 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:43.562 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.562 INFO:tasks.workunit.client.0.trial046.stderr: -384> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 5 monclient: authenticate success, global_id 7149 2026-02-24T22:17:43.562 INFO:tasks.workunit.client.0.trial046.stderr: -383> 2026-02-24T22:17:43.476+0000 7f8d877fe640 4 set_mon_vals no callback set 2026-02-24T22:17:43.562 INFO:tasks.workunit.client.0.trial046.stderr: -382> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient: _renew_subs 2026-02-24T22:17:43.562 INFO:tasks.workunit.client.0.trial046.stderr: -381> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:17:43.562 INFO:tasks.workunit.client.0.trial046.stderr: -380> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient: _renew_subs 2026-02-24T22:17:43.562 INFO:tasks.workunit.client.0.trial046.stderr: -379> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:17:43.563 INFO:tasks.workunit.client.0.trial046.stderr: -378> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 1 librados: init done 2026-02-24T22:17:43.563 INFO:tasks.workunit.client.0.trial046.stderr: -377> 2026-02-24T22:17:43.476+0000 7f8d85ffb640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:17:43.563 INFO:tasks.workunit.client.0.trial046.stderr: -376> 2026-02-24T22:17:43.476+0000 7f8d85ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:17:43.563 INFO:tasks.workunit.client.0.trial046.stderr: -375> 2026-02-24T22:17:43.476+0000 7f8d85ffb640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:17:43.563 INFO:tasks.workunit.client.0.trial046.stderr: -374> 2026-02-24T22:17:43.476+0000 7f8da0ec4640 10 monclient: get_auth_request con 0x7f8d800707b0 auth_method 0 2026-02-24T22:17:43.563 INFO:tasks.workunit.client.0.trial046.stderr: -373> 2026-02-24T22:17:43.476+0000 7f8da1ec6640 10 monclient: get_auth_request con 0x558d50d20570 auth_method 0 2026-02-24T22:17:43.563 INFO:tasks.workunit.client.0.trial046.stderr: -372> 2026-02-24T22:17:43.476+0000 7f8da16c5640 10 monclient: get_auth_request con 0x558d50d4a0f0 auth_method 0 2026-02-24T22:17:43.563 INFO:tasks.workunit.client.0.trial046.stderr: -371> 2026-02-24T22:17:43.480+0000 7f8da0ec4640 10 monclient: get_auth_request con 0x558d50d4ba60 auth_method 0 2026-02-24T22:17:43.563 INFO:tasks.workunit.client.0.trial046.stderr: -370> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient: build_initial_monmap 2026-02-24T22:17:43.563 INFO:tasks.workunit.client.0.trial046.stderr: -369> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 1 build_initial for_mkfs: 0 2026-02-24T22:17:43.563 INFO:tasks.workunit.client.0.trial046.stderr: -368> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient: monmap: 2026-02-24T22:17:43.563 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:17:43.563 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:17:43.563 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:17:43.563 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:17:43.564 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:17:43.564 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.564 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:17:43.564 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:17:43.564 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:17:43.564 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.564 INFO:tasks.workunit.client.0.trial046.stderr: -367> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding auth protocol: cephx 2026-02-24T22:17:43.564 INFO:tasks.workunit.client.0.trial046.stderr: -366> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding auth protocol: cephx 2026-02-24T22:17:43.564 INFO:tasks.workunit.client.0.trial046.stderr: -365> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding auth protocol: cephx 2026-02-24T22:17:43.564 INFO:tasks.workunit.client.0.trial046.stderr: -364> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding con mode: secure 2026-02-24T22:17:43.564 INFO:tasks.workunit.client.0.trial046.stderr: -363> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding con mode: crc 2026-02-24T22:17:43.564 INFO:tasks.workunit.client.0.trial046.stderr: -362> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding con mode: secure 2026-02-24T22:17:43.564 INFO:tasks.workunit.client.0.trial046.stderr: -361> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding con mode: crc 2026-02-24T22:17:43.564 INFO:tasks.workunit.client.0.trial046.stderr: -360> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding con mode: secure 2026-02-24T22:17:43.564 INFO:tasks.workunit.client.0.trial046.stderr: -359> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding con mode: crc 2026-02-24T22:17:43.565 INFO:tasks.workunit.client.0.trial046.stderr: -358> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding con mode: crc 2026-02-24T22:17:43.565 INFO:tasks.workunit.client.0.trial046.stderr: -357> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding con mode: secure 2026-02-24T22:17:43.565 INFO:tasks.workunit.client.0.trial046.stderr: -356> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding con mode: crc 2026-02-24T22:17:43.565 INFO:tasks.workunit.client.0.trial046.stderr: -355> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding con mode: secure 2026-02-24T22:17:43.565 INFO:tasks.workunit.client.0.trial046.stderr: -354> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding con mode: crc 2026-02-24T22:17:43.565 INFO:tasks.workunit.client.0.trial046.stderr: -353> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding con mode: secure 2026-02-24T22:17:43.565 INFO:tasks.workunit.client.0.trial046.stderr: -352> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:43.565 INFO:tasks.workunit.client.0.trial046.stderr: -351> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command objecter_requests cmddesc objecter_requests hook 0x558d50d53e10 EEXIST 2026-02-24T22:17:43.565 INFO:tasks.workunit.client.0.trial046.stderr: -350> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient: init 2026-02-24T22:17:43.565 INFO:tasks.workunit.client.0.trial046.stderr: -349> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding auth protocol: cephx 2026-02-24T22:17:43.565 INFO:tasks.workunit.client.0.trial046.stderr: -348> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding auth protocol: cephx 2026-02-24T22:17:43.565 INFO:tasks.workunit.client.0.trial046.stderr: -347> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding auth protocol: cephx 2026-02-24T22:17:43.565 INFO:tasks.workunit.client.0.trial046.stderr: -346> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding con mode: secure 2026-02-24T22:17:43.565 INFO:tasks.workunit.client.0.trial046.stderr: -345> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding con mode: crc 2026-02-24T22:17:43.565 INFO:tasks.workunit.client.0.trial046.stderr: -344> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding con mode: secure 2026-02-24T22:17:43.565 INFO:tasks.workunit.client.0.trial046.stderr: -343> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding con mode: crc 2026-02-24T22:17:43.566 INFO:tasks.workunit.client.0.trial046.stderr: -342> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding con mode: secure 2026-02-24T22:17:43.566 INFO:tasks.workunit.client.0.trial046.stderr: -341> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding con mode: crc 2026-02-24T22:17:43.566 INFO:tasks.workunit.client.0.trial046.stderr: -340> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding con mode: crc 2026-02-24T22:17:43.566 INFO:tasks.workunit.client.0.trial046.stderr: -339> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding con mode: secure 2026-02-24T22:17:43.566 INFO:tasks.workunit.client.0.trial046.stderr: -338> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding con mode: crc 2026-02-24T22:17:43.566 INFO:tasks.workunit.client.0.trial046.stderr: -337> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding con mode: secure 2026-02-24T22:17:43.566 INFO:tasks.workunit.client.0.trial046.stderr: -336> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding con mode: crc 2026-02-24T22:17:43.566 INFO:tasks.workunit.client.0.trial046.stderr: -335> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding con mode: secure 2026-02-24T22:17:43.566 INFO:tasks.workunit.client.0.trial046.stderr: -334> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:43.566 INFO:tasks.workunit.client.0.trial046.stderr: -333> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:43.566 INFO:tasks.workunit.client.0.trial046.stderr: -332> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command rotate-key cmddesc rotate-key hook 0x558d50d4c510 EEXIST 2026-02-24T22:17:43.566 INFO:tasks.workunit.client.0.trial046.stderr: -331> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient: _reopen_session rank -1 2026-02-24T22:17:43.566 INFO:tasks.workunit.client.0.trial046.stderr: -330> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T22:17:43.566 INFO:tasks.workunit.client.0.trial046.stderr: -329> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient(hunting): picked mon.noname-c con 0x558d50ea7700 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:17:43.566 INFO:tasks.workunit.client.0.trial046.stderr: -328> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient(hunting): picked mon.noname-b con 0x558d50ea48f0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:17:43.566 INFO:tasks.workunit.client.0.trial046.stderr: -327> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient(hunting): picked mon.noname-a con 0x558d50ea0880 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:17:43.566 INFO:tasks.workunit.client.0.trial046.stderr: -326> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:43.567 INFO:tasks.workunit.client.0.trial046.stderr: -325> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:43.567 INFO:tasks.workunit.client.0.trial046.stderr: -324> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:43.567 INFO:tasks.workunit.client.0.trial046.stderr: -323> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient(hunting): _renew_subs 2026-02-24T22:17:43.567 INFO:tasks.workunit.client.0.trial046.stderr: -322> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient(hunting): authenticate will time out at 1833.589779s 2026-02-24T22:17:43.567 INFO:tasks.workunit.client.0.trial046.stderr: -321> 2026-02-24T22:17:43.480+0000 7f8da16c5640 10 monclient(hunting): get_auth_request con 0x558d50ea7700 auth_method 0 2026-02-24T22:17:43.567 INFO:tasks.workunit.client.0.trial046.stderr: -320> 2026-02-24T22:17:43.480+0000 7f8da16c5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:43.567 INFO:tasks.workunit.client.0.trial046.stderr: -319> 2026-02-24T22:17:43.480+0000 7f8da16c5640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:43.567 INFO:tasks.workunit.client.0.trial046.stderr: -318> 2026-02-24T22:17:43.480+0000 7f8da16c5640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:43.567 INFO:tasks.workunit.client.0.trial046.stderr: -317> 2026-02-24T22:17:43.480+0000 7f8da1ec6640 10 monclient(hunting): get_auth_request con 0x558d50ea0880 auth_method 0 2026-02-24T22:17:43.567 INFO:tasks.workunit.client.0.trial046.stderr: -316> 2026-02-24T22:17:43.480+0000 7f8da1ec6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:43.567 INFO:tasks.workunit.client.0.trial046.stderr: -315> 2026-02-24T22:17:43.480+0000 7f8da1ec6640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:43.567 INFO:tasks.workunit.client.0.trial046.stderr: -314> 2026-02-24T22:17:43.480+0000 7f8da1ec6640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:43.567 INFO:tasks.workunit.client.0.trial046.stderr: -313> 2026-02-24T22:17:43.480+0000 7f8da16c5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:43.567 INFO:tasks.workunit.client.0.trial046.stderr: -312> 2026-02-24T22:17:43.480+0000 7f8da16c5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:43.567 INFO:tasks.workunit.client.0.trial046.stderr: -311> 2026-02-24T22:17:43.480+0000 7f8da16c5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:43.567 INFO:tasks.workunit.client.0.trial046.stderr: -310> 2026-02-24T22:17:43.480+0000 7f8da1ec6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:43.568 INFO:tasks.workunit.client.0.trial046.stderr: -309> 2026-02-24T22:17:43.480+0000 7f8da1ec6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:43.568 INFO:tasks.workunit.client.0.trial046.stderr: -308> 2026-02-24T22:17:43.480+0000 7f8da1ec6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:43.568 INFO:tasks.workunit.client.0.trial046.stderr: -307> 2026-02-24T22:17:43.480+0000 7f8da0ec4640 10 monclient(hunting): get_auth_request con 0x558d50ea48f0 auth_method 0 2026-02-24T22:17:43.568 INFO:tasks.workunit.client.0.trial046.stderr: -306> 2026-02-24T22:17:43.480+0000 7f8da0ec4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:43.568 INFO:tasks.workunit.client.0.trial046.stderr: -305> 2026-02-24T22:17:43.480+0000 7f8da0ec4640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:43.568 INFO:tasks.workunit.client.0.trial046.stderr: -304> 2026-02-24T22:17:43.480+0000 7f8da0ec4640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:43.568 INFO:tasks.workunit.client.0.trial046.stderr: -303> 2026-02-24T22:17:43.480+0000 7f8da16c5640 10 monclient(hunting): handle_auth_done global_id 6698 payload 824 2026-02-24T22:17:43.568 INFO:tasks.workunit.client.0.trial046.stderr: -302> 2026-02-24T22:17:43.480+0000 7f8da16c5640 10 monclient: _finish_hunting 0 2026-02-24T22:17:43.568 INFO:tasks.workunit.client.0.trial046.stderr: -301> 2026-02-24T22:17:43.480+0000 7f8da16c5640 1 monclient: found mon.noname-c 2026-02-24T22:17:43.568 INFO:tasks.workunit.client.0.trial046.stderr: -300> 2026-02-24T22:17:43.480+0000 7f8da16c5640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:17:43.568 INFO:tasks.workunit.client.0.trial046.stderr: -299> 2026-02-24T22:17:43.480+0000 7f8d7effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:43.568 INFO:tasks.workunit.client.0.trial046.stderr: -298> 2026-02-24T22:17:43.480+0000 7f8d7effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:17:43.568 INFO:tasks.workunit.client.0.trial046.stderr: -297> 2026-02-24T22:17:43.480+0000 7f8d7effd640 10 monclient: dump: 2026-02-24T22:17:43.568 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:43.568 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:43.568 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.568 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.568 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:43.569 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.569 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:43.569 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:43.569 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:43.569 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.569 INFO:tasks.workunit.client.0.trial046.stderr: -296> 2026-02-24T22:17:43.480+0000 7f8d7effd640 10 monclient: _finish_auth 0 2026-02-24T22:17:43.569 INFO:tasks.workunit.client.0.trial046.stderr: -295> 2026-02-24T22:17:43.480+0000 7f8d7effd640 10 monclient: _check_auth_tickets 2026-02-24T22:17:43.569 INFO:tasks.workunit.client.0.trial046.stderr: -294> 2026-02-24T22:17:43.480+0000 7f8d7effd640 10 monclient: handle_config config(0 keys) 2026-02-24T22:17:43.569 INFO:tasks.workunit.client.0.trial046.stderr: -293> 2026-02-24T22:17:43.480+0000 7f8d7effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:43.569 INFO:tasks.workunit.client.0.trial046.stderr: -292> 2026-02-24T22:17:43.480+0000 7f8d7effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:17:43.569 INFO:tasks.workunit.client.0.trial046.stderr: -291> 2026-02-24T22:17:43.480+0000 7f8d7effd640 10 monclient: dump: 2026-02-24T22:17:43.569 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:43.569 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:43.569 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.569 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.569 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:43.569 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.570 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:43.570 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:43.570 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:43.570 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.570 INFO:tasks.workunit.client.0.trial046.stderr: -290> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 monclient: authenticate success, global_id 6698 2026-02-24T22:17:43.570 INFO:tasks.workunit.client.0.trial046.stderr: -289> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient: _renew_subs 2026-02-24T22:17:43.570 INFO:tasks.workunit.client.0.trial046.stderr: -288> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:17:43.570 INFO:tasks.workunit.client.0.trial046.stderr: -287> 2026-02-24T22:17:43.480+0000 7f8da26c7640 4 set_mon_vals no callback set 2026-02-24T22:17:43.570 INFO:tasks.workunit.client.0.trial046.stderr: -286> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient: _renew_subs 2026-02-24T22:17:43.570 INFO:tasks.workunit.client.0.trial046.stderr: -285> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:17:43.570 INFO:tasks.workunit.client.0.trial046.stderr: -284> 2026-02-24T22:17:43.480+0000 7f8d7effd640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:17:43.570 INFO:tasks.workunit.client.0.trial046.stderr: -283> 2026-02-24T22:17:43.480+0000 7f8d7effd640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:17:43.570 INFO:tasks.workunit.client.0.trial046.stderr: -282> 2026-02-24T22:17:43.480+0000 7f8d7effd640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:17:43.570 INFO:tasks.workunit.client.0.trial046.stderr: -281> 2026-02-24T22:17:43.480+0000 7f8da0ec4640 10 monclient: get_auth_request con 0x7f8d740706c0 auth_method 0 2026-02-24T22:17:43.570 INFO:tasks.workunit.client.0.trial046.stderr: -280> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient: get_monmap_and_config 2026-02-24T22:17:43.570 INFO:tasks.workunit.client.0.trial046.stderr: -279> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient: build_initial_monmap 2026-02-24T22:17:43.570 INFO:tasks.workunit.client.0.trial046.stderr: -278> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 1 build_initial for_mkfs: 0 2026-02-24T22:17:43.570 INFO:tasks.workunit.client.0.trial046.stderr: -277> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient: monmap: 2026-02-24T22:17:43.570 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:17:43.571 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:17:43.571 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:17:43.571 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:17:43.571 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:17:43.571 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.571 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:17:43.571 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:17:43.571 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:17:43.571 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.571 INFO:tasks.workunit.client.0.trial046.stderr: -276> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding auth protocol: cephx 2026-02-24T22:17:43.571 INFO:tasks.workunit.client.0.trial046.stderr: -275> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding auth protocol: cephx 2026-02-24T22:17:43.571 INFO:tasks.workunit.client.0.trial046.stderr: -274> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding auth protocol: cephx 2026-02-24T22:17:43.571 INFO:tasks.workunit.client.0.trial046.stderr: -273> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding con mode: secure 2026-02-24T22:17:43.571 INFO:tasks.workunit.client.0.trial046.stderr: -272> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding con mode: crc 2026-02-24T22:17:43.571 INFO:tasks.workunit.client.0.trial046.stderr: -271> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding con mode: secure 2026-02-24T22:17:43.571 INFO:tasks.workunit.client.0.trial046.stderr: -270> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding con mode: crc 2026-02-24T22:17:43.571 INFO:tasks.workunit.client.0.trial046.stderr: -269> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding con mode: secure 2026-02-24T22:17:43.571 INFO:tasks.workunit.client.0.trial046.stderr: -268> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding con mode: crc 2026-02-24T22:17:43.571 INFO:tasks.workunit.client.0.trial046.stderr: -267> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding con mode: crc 2026-02-24T22:17:43.572 INFO:tasks.workunit.client.0.trial046.stderr: -266> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding con mode: secure 2026-02-24T22:17:43.572 INFO:tasks.workunit.client.0.trial046.stderr: -265> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding con mode: crc 2026-02-24T22:17:43.572 INFO:tasks.workunit.client.0.trial046.stderr: -264> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding con mode: secure 2026-02-24T22:17:43.572 INFO:tasks.workunit.client.0.trial046.stderr: -263> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding con mode: crc 2026-02-24T22:17:43.572 INFO:tasks.workunit.client.0.trial046.stderr: -262> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding con mode: secure 2026-02-24T22:17:43.572 INFO:tasks.workunit.client.0.trial046.stderr: -261> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:43.572 INFO:tasks.workunit.client.0.trial046.stderr: -260> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: init 2026-02-24T22:17:43.572 INFO:tasks.workunit.client.0.trial046.stderr: -259> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding auth protocol: cephx 2026-02-24T22:17:43.572 INFO:tasks.workunit.client.0.trial046.stderr: -258> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding auth protocol: cephx 2026-02-24T22:17:43.572 INFO:tasks.workunit.client.0.trial046.stderr: -257> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding auth protocol: cephx 2026-02-24T22:17:43.572 INFO:tasks.workunit.client.0.trial046.stderr: -256> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding con mode: secure 2026-02-24T22:17:43.572 INFO:tasks.workunit.client.0.trial046.stderr: -255> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding con mode: crc 2026-02-24T22:17:43.572 INFO:tasks.workunit.client.0.trial046.stderr: -254> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding con mode: secure 2026-02-24T22:17:43.572 INFO:tasks.workunit.client.0.trial046.stderr: -253> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding con mode: crc 2026-02-24T22:17:43.572 INFO:tasks.workunit.client.0.trial046.stderr: -252> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding con mode: secure 2026-02-24T22:17:43.572 INFO:tasks.workunit.client.0.trial046.stderr: -251> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding con mode: crc 2026-02-24T22:17:43.572 INFO:tasks.workunit.client.0.trial046.stderr: -250> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding con mode: crc 2026-02-24T22:17:43.572 INFO:tasks.workunit.client.0.trial046.stderr: -249> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding con mode: secure 2026-02-24T22:17:43.572 INFO:tasks.workunit.client.0.trial046.stderr: -248> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding con mode: crc 2026-02-24T22:17:43.573 INFO:tasks.workunit.client.0.trial046.stderr: -247> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding con mode: secure 2026-02-24T22:17:43.573 INFO:tasks.workunit.client.0.trial046.stderr: -246> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding con mode: crc 2026-02-24T22:17:43.573 INFO:tasks.workunit.client.0.trial046.stderr: -245> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding con mode: secure 2026-02-24T22:17:43.573 INFO:tasks.workunit.client.0.trial046.stderr: -244> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:43.573 INFO:tasks.workunit.client.0.trial046.stderr: -243> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:43.573 INFO:tasks.workunit.client.0.trial046.stderr: -242> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command rotate-key cmddesc rotate-key hook 0x7ffe3af4d808 EEXIST 2026-02-24T22:17:43.573 INFO:tasks.workunit.client.0.trial046.stderr: -241> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: _reopen_session rank -1 2026-02-24T22:17:43.573 INFO:tasks.workunit.client.0.trial046.stderr: -240> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T22:17:43.573 INFO:tasks.workunit.client.0.trial046.stderr: -239> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): picked mon.noname-b con 0x558d50f5f760 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:17:43.573 INFO:tasks.workunit.client.0.trial046.stderr: -238> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): picked mon.noname-a con 0x558d50f62ed0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:17:43.573 INFO:tasks.workunit.client.0.trial046.stderr: -237> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): picked mon.noname-c con 0x558d50f3dc90 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:17:43.573 INFO:tasks.workunit.client.0.trial046.stderr: -236> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:43.573 INFO:tasks.workunit.client.0.trial046.stderr: -235> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:43.573 INFO:tasks.workunit.client.0.trial046.stderr: -234> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:43.573 INFO:tasks.workunit.client.0.trial046.stderr: -233> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): _renew_subs 2026-02-24T22:17:43.573 INFO:tasks.workunit.client.0.trial046.stderr: -232> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): authenticate will time out at 1833.592444s 2026-02-24T22:17:43.573 INFO:tasks.workunit.client.0.trial046.stderr: -231> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient(hunting): get_auth_request con 0x558d50f3dc90 auth_method 0 2026-02-24T22:17:43.573 INFO:tasks.workunit.client.0.trial046.stderr: -230> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:43.573 INFO:tasks.workunit.client.0.trial046.stderr: -229> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:43.573 INFO:tasks.workunit.client.0.trial046.stderr: -228> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:43.574 INFO:tasks.workunit.client.0.trial046.stderr: -227> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): get_auth_request con 0x558d50f62ed0 auth_method 0 2026-02-24T22:17:43.574 INFO:tasks.workunit.client.0.trial046.stderr: -226> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:43.574 INFO:tasks.workunit.client.0.trial046.stderr: -225> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:43.574 INFO:tasks.workunit.client.0.trial046.stderr: -224> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:43.574 INFO:tasks.workunit.client.0.trial046.stderr: -223> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:43.574 INFO:tasks.workunit.client.0.trial046.stderr: -222> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:43.574 INFO:tasks.workunit.client.0.trial046.stderr: -221> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:43.574 INFO:tasks.workunit.client.0.trial046.stderr: -220> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:43.574 INFO:tasks.workunit.client.0.trial046.stderr: -219> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:43.574 INFO:tasks.workunit.client.0.trial046.stderr: -218> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:43.574 INFO:tasks.workunit.client.0.trial046.stderr: -217> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient(hunting): handle_auth_done global_id 6701 payload 274 2026-02-24T22:17:43.574 INFO:tasks.workunit.client.0.trial046.stderr: -216> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient: _finish_hunting 0 2026-02-24T22:17:43.574 INFO:tasks.workunit.client.0.trial046.stderr: -215> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 1 monclient: found mon.noname-c 2026-02-24T22:17:43.574 INFO:tasks.workunit.client.0.trial046.stderr: -214> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:17:43.574 INFO:tasks.workunit.client.0.trial046.stderr: -213> 2026-02-24T22:17:43.484+0000 7f8da16c5640 10 monclient: get_auth_request con 0x558d50f5f760 auth_method 0 2026-02-24T22:17:43.574 INFO:tasks.workunit.client.0.trial046.stderr: -212> 2026-02-24T22:17:43.484+0000 7f8d7cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:43.574 INFO:tasks.workunit.client.0.trial046.stderr: -211> 2026-02-24T22:17:43.484+0000 7f8d7cff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:17:43.574 INFO:tasks.workunit.client.0.trial046.stderr: -210> 2026-02-24T22:17:43.484+0000 7f8d7cff9640 10 monclient: dump: 2026-02-24T22:17:43.574 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:43.574 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:43.575 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.575 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.575 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:43.575 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.575 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:43.575 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:43.575 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:43.575 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.575 INFO:tasks.workunit.client.0.trial046.stderr: -209> 2026-02-24T22:17:43.484+0000 7f8d7cff9640 10 monclient: _finish_auth 0 2026-02-24T22:17:43.575 INFO:tasks.workunit.client.0.trial046.stderr: -208> 2026-02-24T22:17:43.484+0000 7f8d7cff9640 10 monclient: _check_auth_tickets 2026-02-24T22:17:43.575 INFO:tasks.workunit.client.0.trial046.stderr: -207> 2026-02-24T22:17:43.484+0000 7f8d7cff9640 10 monclient: handle_config config(0 keys) 2026-02-24T22:17:43.575 INFO:tasks.workunit.client.0.trial046.stderr: -206> 2026-02-24T22:17:43.484+0000 7f8d7cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:43.575 INFO:tasks.workunit.client.0.trial046.stderr: -205> 2026-02-24T22:17:43.484+0000 7f8d7cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:17:43.575 INFO:tasks.workunit.client.0.trial046.stderr: -204> 2026-02-24T22:17:43.484+0000 7f8d7cff9640 10 monclient: dump: 2026-02-24T22:17:43.575 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:43.575 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:43.575 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.575 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.575 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:43.576 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.576 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:43.576 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:43.576 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:43.576 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.576 INFO:tasks.workunit.client.0.trial046.stderr: -203> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 monclient: authenticate success, global_id 6701 2026-02-24T22:17:43.576 INFO:tasks.workunit.client.0.trial046.stderr: -202> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: get_monmap_and_config success 2026-02-24T22:17:43.576 INFO:tasks.workunit.client.0.trial046.stderr: -201> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 4 set_mon_vals no callback set 2026-02-24T22:17:43.576 INFO:tasks.workunit.client.0.trial046.stderr: -200> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: shutdown 2026-02-24T22:17:43.576 INFO:tasks.workunit.client.0.trial046.stderr: -199> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: build_initial_monmap 2026-02-24T22:17:43.576 INFO:tasks.workunit.client.0.trial046.stderr: -198> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 1 build_initial for_mkfs: 0 2026-02-24T22:17:43.576 INFO:tasks.workunit.client.0.trial046.stderr: -197> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: monmap: 2026-02-24T22:17:43.576 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:17:43.576 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:17:43.576 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:17:43.576 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:17:43.576 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:17:43.576 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.576 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:17:43.576 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:17:43.576 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:17:43.577 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.577 INFO:tasks.workunit.client.0.trial046.stderr: -196> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding auth protocol: cephx 2026-02-24T22:17:43.577 INFO:tasks.workunit.client.0.trial046.stderr: -195> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding auth protocol: cephx 2026-02-24T22:17:43.577 INFO:tasks.workunit.client.0.trial046.stderr: -194> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding auth protocol: cephx 2026-02-24T22:17:43.577 INFO:tasks.workunit.client.0.trial046.stderr: -193> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding con mode: secure 2026-02-24T22:17:43.577 INFO:tasks.workunit.client.0.trial046.stderr: -192> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding con mode: crc 2026-02-24T22:17:43.577 INFO:tasks.workunit.client.0.trial046.stderr: -191> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding con mode: secure 2026-02-24T22:17:43.577 INFO:tasks.workunit.client.0.trial046.stderr: -190> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding con mode: crc 2026-02-24T22:17:43.577 INFO:tasks.workunit.client.0.trial046.stderr: -189> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding con mode: secure 2026-02-24T22:17:43.577 INFO:tasks.workunit.client.0.trial046.stderr: -188> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding con mode: crc 2026-02-24T22:17:43.577 INFO:tasks.workunit.client.0.trial046.stderr: -187> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding con mode: crc 2026-02-24T22:17:43.577 INFO:tasks.workunit.client.0.trial046.stderr: -186> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding con mode: secure 2026-02-24T22:17:43.577 INFO:tasks.workunit.client.0.trial046.stderr: -185> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding con mode: crc 2026-02-24T22:17:43.577 INFO:tasks.workunit.client.0.trial046.stderr: -184> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding con mode: secure 2026-02-24T22:17:43.577 INFO:tasks.workunit.client.0.trial046.stderr: -183> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding con mode: crc 2026-02-24T22:17:43.577 INFO:tasks.workunit.client.0.trial046.stderr: -182> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding con mode: secure 2026-02-24T22:17:43.577 INFO:tasks.workunit.client.0.trial046.stderr: -181> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:43.577 INFO:tasks.workunit.client.0.trial046.stderr: -180> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 1 librados: starting msgr at 2026-02-24T22:17:43.577 INFO:tasks.workunit.client.0.trial046.stderr: -179> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 1 librados: starting objecter 2026-02-24T22:17:43.577 INFO:tasks.workunit.client.0.trial046.stderr: -178> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command objecter_requests cmddesc objecter_requests hook 0x558d50f6a5f0 EEXIST 2026-02-24T22:17:43.577 INFO:tasks.workunit.client.0.trial046.stderr: -177> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 1 librados: setting wanted keys 2026-02-24T22:17:43.578 INFO:tasks.workunit.client.0.trial046.stderr: -176> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 1 librados: calling monclient init 2026-02-24T22:17:43.578 INFO:tasks.workunit.client.0.trial046.stderr: -175> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: init 2026-02-24T22:17:43.578 INFO:tasks.workunit.client.0.trial046.stderr: -174> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding auth protocol: cephx 2026-02-24T22:17:43.578 INFO:tasks.workunit.client.0.trial046.stderr: -173> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding auth protocol: cephx 2026-02-24T22:17:43.578 INFO:tasks.workunit.client.0.trial046.stderr: -172> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding auth protocol: cephx 2026-02-24T22:17:43.578 INFO:tasks.workunit.client.0.trial046.stderr: -171> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding con mode: secure 2026-02-24T22:17:43.578 INFO:tasks.workunit.client.0.trial046.stderr: -170> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding con mode: crc 2026-02-24T22:17:43.578 INFO:tasks.workunit.client.0.trial046.stderr: -169> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding con mode: secure 2026-02-24T22:17:43.578 INFO:tasks.workunit.client.0.trial046.stderr: -168> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding con mode: crc 2026-02-24T22:17:43.578 INFO:tasks.workunit.client.0.trial046.stderr: -167> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding con mode: secure 2026-02-24T22:17:43.578 INFO:tasks.workunit.client.0.trial046.stderr: -166> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding con mode: crc 2026-02-24T22:17:43.578 INFO:tasks.workunit.client.0.trial046.stderr: -165> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding con mode: crc 2026-02-24T22:17:43.578 INFO:tasks.workunit.client.0.trial046.stderr: -164> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding con mode: secure 2026-02-24T22:17:43.578 INFO:tasks.workunit.client.0.trial046.stderr: -163> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding con mode: crc 2026-02-24T22:17:43.578 INFO:tasks.workunit.client.0.trial046.stderr: -162> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding con mode: secure 2026-02-24T22:17:43.578 INFO:tasks.workunit.client.0.trial046.stderr: -161> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding con mode: crc 2026-02-24T22:17:43.578 INFO:tasks.workunit.client.0.trial046.stderr: -160> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding con mode: secure 2026-02-24T22:17:43.578 INFO:tasks.workunit.client.0.trial046.stderr: -159> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:43.578 INFO:tasks.workunit.client.0.trial046.stderr: -158> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:43.578 INFO:tasks.workunit.client.0.trial046.stderr: -157> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command rotate-key cmddesc rotate-key hook 0x558d50ed6448 EEXIST 2026-02-24T22:17:43.578 INFO:tasks.workunit.client.0.trial046.stderr: -156> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: _reopen_session rank -1 2026-02-24T22:17:43.579 INFO:tasks.workunit.client.0.trial046.stderr: -155> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T22:17:43.579 INFO:tasks.workunit.client.0.trial046.stderr: -154> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): picked mon.noname-a con 0x558d50f5f760 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:17:43.579 INFO:tasks.workunit.client.0.trial046.stderr: -153> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): picked mon.noname-c con 0x558d50f62ed0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:17:43.579 INFO:tasks.workunit.client.0.trial046.stderr: -152> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): picked mon.noname-b con 0x558d50f8d4e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:17:43.579 INFO:tasks.workunit.client.0.trial046.stderr: -151> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:43.579 INFO:tasks.workunit.client.0.trial046.stderr: -150> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:43.579 INFO:tasks.workunit.client.0.trial046.stderr: -149> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:43.579 INFO:tasks.workunit.client.0.trial046.stderr: -148> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): _renew_subs 2026-02-24T22:17:43.579 INFO:tasks.workunit.client.0.trial046.stderr: -147> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): authenticate will time out at 1833.593790s 2026-02-24T22:17:43.579 INFO:tasks.workunit.client.0.trial046.stderr: -146> 2026-02-24T22:17:43.484+0000 7f8da16c5640 10 monclient(hunting): get_auth_request con 0x558d50f5f760 auth_method 0 2026-02-24T22:17:43.579 INFO:tasks.workunit.client.0.trial046.stderr: -145> 2026-02-24T22:17:43.484+0000 7f8da16c5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:43.579 INFO:tasks.workunit.client.0.trial046.stderr: -144> 2026-02-24T22:17:43.484+0000 7f8da16c5640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:43.579 INFO:tasks.workunit.client.0.trial046.stderr: -143> 2026-02-24T22:17:43.484+0000 7f8da16c5640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:43.579 INFO:tasks.workunit.client.0.trial046.stderr: -142> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): get_auth_request con 0x558d50f62ed0 auth_method 0 2026-02-24T22:17:43.579 INFO:tasks.workunit.client.0.trial046.stderr: -141> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:43.579 INFO:tasks.workunit.client.0.trial046.stderr: -140> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:43.579 INFO:tasks.workunit.client.0.trial046.stderr: -139> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:43.579 INFO:tasks.workunit.client.0.trial046.stderr: -138> 2026-02-24T22:17:43.484+0000 7f8da16c5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:43.579 INFO:tasks.workunit.client.0.trial046.stderr: -137> 2026-02-24T22:17:43.484+0000 7f8da16c5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:43.579 INFO:tasks.workunit.client.0.trial046.stderr: -136> 2026-02-24T22:17:43.484+0000 7f8da16c5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:43.579 INFO:tasks.workunit.client.0.trial046.stderr: -135> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:43.579 INFO:tasks.workunit.client.0.trial046.stderr: -134> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:43.579 INFO:tasks.workunit.client.0.trial046.stderr: -133> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:43.579 INFO:tasks.workunit.client.0.trial046.stderr: -132> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient(hunting): get_auth_request con 0x558d50f8d4e0 auth_method 0 2026-02-24T22:17:43.580 INFO:tasks.workunit.client.0.trial046.stderr: -131> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:43.580 INFO:tasks.workunit.client.0.trial046.stderr: -130> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:43.580 INFO:tasks.workunit.client.0.trial046.stderr: -129> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:43.580 INFO:tasks.workunit.client.0.trial046.stderr: -128> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): handle_auth_done global_id 6704 payload 824 2026-02-24T22:17:43.580 INFO:tasks.workunit.client.0.trial046.stderr: -127> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient: _finish_hunting 0 2026-02-24T22:17:43.580 INFO:tasks.workunit.client.0.trial046.stderr: -126> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 1 monclient: found mon.noname-c 2026-02-24T22:17:43.580 INFO:tasks.workunit.client.0.trial046.stderr: -125> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:17:43.580 INFO:tasks.workunit.client.0.trial046.stderr: -124> 2026-02-24T22:17:43.484+0000 7f8d6affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:43.580 INFO:tasks.workunit.client.0.trial046.stderr: -123> 2026-02-24T22:17:43.484+0000 7f8d6affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:17:43.580 INFO:tasks.workunit.client.0.trial046.stderr: -122> 2026-02-24T22:17:43.484+0000 7f8d6affd640 10 monclient: dump: 2026-02-24T22:17:43.580 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:43.580 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:43.580 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.580 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.580 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:43.580 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.580 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:43.580 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:43.580 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:43.580 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.580 INFO:tasks.workunit.client.0.trial046.stderr: -121> 2026-02-24T22:17:43.484+0000 7f8d6affd640 10 monclient: _finish_auth 0 2026-02-24T22:17:43.580 INFO:tasks.workunit.client.0.trial046.stderr: -120> 2026-02-24T22:17:43.484+0000 7f8d6affd640 10 monclient: _check_auth_tickets 2026-02-24T22:17:43.580 INFO:tasks.workunit.client.0.trial046.stderr: -119> 2026-02-24T22:17:43.484+0000 7f8d6affd640 10 monclient: handle_config config(0 keys) 2026-02-24T22:17:43.581 INFO:tasks.workunit.client.0.trial046.stderr: -118> 2026-02-24T22:17:43.484+0000 7f8d6affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:43.581 INFO:tasks.workunit.client.0.trial046.stderr: -117> 2026-02-24T22:17:43.484+0000 7f8d6affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:17:43.581 INFO:tasks.workunit.client.0.trial046.stderr: -116> 2026-02-24T22:17:43.484+0000 7f8d6affd640 10 monclient: dump: 2026-02-24T22:17:43.581 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:43.581 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:43.581 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.581 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.581 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:43.581 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.581 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:43.581 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:43.581 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:43.581 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.581 INFO:tasks.workunit.client.0.trial046.stderr: -115> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 monclient: authenticate success, global_id 6704 2026-02-24T22:17:43.581 INFO:tasks.workunit.client.0.trial046.stderr: -114> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: _renew_subs 2026-02-24T22:17:43.581 INFO:tasks.workunit.client.0.trial046.stderr: -113> 2026-02-24T22:17:43.484+0000 7f8d7cff9640 4 set_mon_vals no callback set 2026-02-24T22:17:43.581 INFO:tasks.workunit.client.0.trial046.stderr: -112> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:17:43.581 INFO:tasks.workunit.client.0.trial046.stderr: -111> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: _renew_subs 2026-02-24T22:17:43.581 INFO:tasks.workunit.client.0.trial046.stderr: -110> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:17:43.581 INFO:tasks.workunit.client.0.trial046.stderr: -109> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 1 librados: init done 2026-02-24T22:17:43.581 INFO:tasks.workunit.client.0.trial046.stderr: -108> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command cr dump hook 0x558d50f66f88 2026-02-24T22:17:43.581 INFO:tasks.workunit.client.0.trial046.stderr: -107> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 4 Realm: () 2026-02-24T22:17:43.581 INFO:tasks.workunit.client.0.trial046.stderr: -106> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:43.582 INFO:tasks.workunit.client.0.trial046.stderr: -105> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:43.582 INFO:tasks.workunit.client.0.trial046.stderr: -104> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:43.582 INFO:tasks.workunit.client.0.trial046.stderr: -103> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 20 zonegroup default 2026-02-24T22:17:43.582 INFO:tasks.workunit.client.0.trial046.stderr: -102> 2026-02-24T22:17:43.484+0000 7f8da16c5640 10 monclient: get_auth_request con 0x558d51000c00 auth_method 0 2026-02-24T22:17:43.582 INFO:tasks.workunit.client.0.trial046.stderr: -101> 2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:43.582 INFO:tasks.workunit.client.0.trial046.stderr: -100> 2026-02-24T22:17:43.488+0000 7f8d6affd640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:17:43.582 INFO:tasks.workunit.client.0.trial046.stderr: -99> 2026-02-24T22:17:43.488+0000 7f8d6affd640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:17:43.582 INFO:tasks.workunit.client.0.trial046.stderr: -98> 2026-02-24T22:17:43.488+0000 7f8d6affd640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:17:43.582 INFO:tasks.workunit.client.0.trial046.stderr: -97> 2026-02-24T22:17:43.488+0000 7f8da1ec6640 10 monclient: get_auth_request con 0x7f8d640707b0 auth_method 0 2026-02-24T22:17:43.582 INFO:tasks.workunit.client.0.trial046.stderr: -96> 2026-02-24T22:17:43.488+0000 7f8da16c5640 10 monclient: get_auth_request con 0x558d51023f70 auth_method 0 2026-02-24T22:17:43.582 INFO:tasks.workunit.client.0.trial046.stderr: -95> 2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:43.582 INFO:tasks.workunit.client.0.trial046.stderr: -94> 2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:43.582 INFO:tasks.workunit.client.0.trial046.stderr: -93> 2026-02-24T22:17:43.488+0000 7f8da0ec4640 10 monclient: get_auth_request con 0x558d51045620 auth_method 0 2026-02-24T22:17:43.582 INFO:tasks.workunit.client.0.trial046.stderr: -92> 2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:43.582 INFO:tasks.workunit.client.0.trial046.stderr: -91> 2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 started sync module instance, tier type = 2026-02-24T22:17:43.582 INFO:tasks.workunit.client.0.trial046.stderr: -90> 2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:43.582 INFO:tasks.workunit.client.0.trial046.stderr: -89> 2026-02-24T22:17:43.488+0000 7f8da1ec6640 10 monclient: get_auth_request con 0x558d511ae4b0 auth_method 0 2026-02-24T22:17:43.582 INFO:tasks.workunit.client.0.trial046.stderr: -88> 2026-02-24T22:17:43.488+0000 7f8da1ec6640 10 monclient: get_auth_request con 0x558d510a6a70 auth_method 0 2026-02-24T22:17:43.582 INFO:tasks.workunit.client.0.trial046.stderr: -87> 2026-02-24T22:17:43.488+0000 7f8da16c5640 10 monclient: get_auth_request con 0x558d511093f0 auth_method 0 2026-02-24T22:17:43.582 INFO:tasks.workunit.client.0.trial046.stderr: -86> 2026-02-24T22:17:43.488+0000 7f8da0ec4640 10 monclient: get_auth_request con 0x558d5116c370 auth_method 0 2026-02-24T22:17:43.582 INFO:tasks.workunit.client.0.trial046.stderr: -85> 2026-02-24T22:17:43.488+0000 7f8da16c5640 10 monclient: get_auth_request con 0x558d512114b0 auth_method 0 2026-02-24T22:17:43.582 INFO:tasks.workunit.client.0.trial046.stderr: -84> 2026-02-24T22:17:43.488+0000 7f8da16c5640 0 -- 10.20.193.46:0/3123648294 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x558d512114b0 msgr2=0x558d51231910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:43.582 INFO:tasks.workunit.client.0.trial046.stderr: -83> 2026-02-24T22:17:43.488+0000 7f8d6affd640 1 client.6704.objecter ms_handle_reset 0x558d512114b0 session 0x558d51210da0 osd.6 2026-02-24T22:17:43.582 INFO:tasks.workunit.client.0.trial046.stderr: -82> 2026-02-24T22:17:43.488+0000 7f8d6affd640 10 monclient: _renew_subs 2026-02-24T22:17:43.582 INFO:tasks.workunit.client.0.trial046.stderr: -81> 2026-02-24T22:17:43.488+0000 7f8d6affd640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:17:43.582 INFO:tasks.workunit.client.0.trial046.stderr: -80> 2026-02-24T22:17:43.488+0000 7f8da16c5640 10 monclient: get_auth_request con 0x7f8d6409d460 auth_method 0 2026-02-24T22:17:43.583 INFO:tasks.workunit.client.0.trial046.stderr: -79> 2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 add_watcher() i=1 2026-02-24T22:17:43.583 INFO:tasks.workunit.client.0.trial046.stderr: -78> 2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 add_watcher() i=5 2026-02-24T22:17:43.583 INFO:tasks.workunit.client.0.trial046.stderr: -77> 2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 add_watcher() i=3 2026-02-24T22:17:43.583 INFO:tasks.workunit.client.0.trial046.stderr: -76> 2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 add_watcher() i=0 2026-02-24T22:17:43.583 INFO:tasks.workunit.client.0.trial046.stderr: -75> 2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 add_watcher() i=6 2026-02-24T22:17:43.583 INFO:tasks.workunit.client.0.trial046.stderr: -74> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 20 add_watcher() i=2 2026-02-24T22:17:43.583 INFO:tasks.workunit.client.0.trial046.stderr: -73> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 20 add_watcher() i=4 2026-02-24T22:17:43.583 INFO:tasks.workunit.client.0.trial046.stderr: -72> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 20 add_watcher() i=7 2026-02-24T22:17:43.583 INFO:tasks.workunit.client.0.trial046.stderr: -71> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:17:43.583 INFO:tasks.workunit.client.0.trial046.stderr: -70> 2026-02-24T22:17:43.492+0000 7f8da0ec4640 10 monclient: get_auth_request con 0x7f8cc0008af0 auth_method 0 2026-02-24T22:17:43.583 INFO:tasks.workunit.client.0.trial046.stderr: -69> 2026-02-24T22:17:43.492+0000 7f8da0ec4640 0 -- 10.20.193.46:0/2381496400 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f8cc0008af0 msgr2=0x7f8cc0028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:43.583 INFO:tasks.workunit.client.0.trial046.stderr: -68> 2026-02-24T22:17:43.492+0000 7f8d7effd640 1 client.6698.objecter ms_handle_reset 0x7f8cc0008af0 session 0x7f8cc0008470 osd.1 2026-02-24T22:17:43.583 INFO:tasks.workunit.client.0.trial046.stderr: -67> 2026-02-24T22:17:43.492+0000 7f8d7effd640 10 monclient: _renew_subs 2026-02-24T22:17:43.583 INFO:tasks.workunit.client.0.trial046.stderr: -66> 2026-02-24T22:17:43.492+0000 7f8d7effd640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:17:43.583 INFO:tasks.workunit.client.0.trial046.stderr: -65> 2026-02-24T22:17:43.492+0000 7f8da0ec4640 10 monclient: get_auth_request con 0x7f8d7409d3f0 auth_method 0 2026-02-24T22:17:43.583 INFO:tasks.workunit.client.0.trial046.stderr: -64> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding auth protocol: cephx 2026-02-24T22:17:43.583 INFO:tasks.workunit.client.0.trial046.stderr: -63> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding auth protocol: cephx 2026-02-24T22:17:43.583 INFO:tasks.workunit.client.0.trial046.stderr: -62> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding auth protocol: cephx 2026-02-24T22:17:43.583 INFO:tasks.workunit.client.0.trial046.stderr: -61> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding con mode: secure 2026-02-24T22:17:43.583 INFO:tasks.workunit.client.0.trial046.stderr: -60> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding con mode: crc 2026-02-24T22:17:43.583 INFO:tasks.workunit.client.0.trial046.stderr: -59> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding con mode: secure 2026-02-24T22:17:43.583 INFO:tasks.workunit.client.0.trial046.stderr: -58> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding con mode: crc 2026-02-24T22:17:43.583 INFO:tasks.workunit.client.0.trial046.stderr: -57> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding con mode: secure 2026-02-24T22:17:43.583 INFO:tasks.workunit.client.0.trial046.stderr: -56> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding con mode: crc 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -55> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding con mode: crc 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -54> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding con mode: secure 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -53> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding con mode: crc 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -52> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding con mode: secure 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -51> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding con mode: crc 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -50> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding con mode: secure 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -49> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -48> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -47> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -46> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command cache list hook 0x558d50f6c960 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -45> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command cache inspect hook 0x558d50f6c960 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -44> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command cache erase hook 0x558d50f6c960 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -43> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command cache zap hook 0x558d50f6c960 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -42> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 note: GC not initialized 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -41> 2026-02-24T22:17:43.492+0000 7f8d4e7e4640 20 reqs_thread_entry: start 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -40> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command sync trace show hook 0x7f8d940a6740 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -39> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command sync trace history hook 0x7f8d940a6740 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -38> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command sync trace active hook 0x7f8d940a6740 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -37> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command sync trace active_short hook 0x7f8d940a6740 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -36> 2026-02-24T22:17:43.492+0000 7f8da1ec6640 10 monclient: get_auth_request con 0x558d511a1940 auth_method 0 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -35> 2026-02-24T22:17:43.496+0000 7f8da16c5640 10 monclient: get_auth_request con 0x558d5119a400 auth_method 0 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -34> 2026-02-24T22:17:43.496+0000 7f8da0ec4640 10 monclient: get_auth_request con 0x558d5119df90 auth_method 0 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -33> 2026-02-24T22:17:43.496+0000 7f8da1ec6640 10 monclient: get_auth_request con 0x558d5119d680 auth_method 0 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -32> 2026-02-24T22:17:43.500+0000 7f8da16c5640 10 monclient: get_auth_request con 0x558d51198110 auth_method 0 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -31> 2026-02-24T22:17:43.504+0000 7f8da0ec4640 10 monclient: get_auth_request con 0x558d51198c40 auth_method 0 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -30> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:43.584 INFO:tasks.workunit.client.0.trial046.stderr: -29> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 20 Filter name: none 2026-02-24T22:17:43.585 INFO:tasks.workunit.client.0.trial046.stderr: -28> 2026-02-24T22:17:43.528+0000 7f8d4cfe1640 20 reqs_thread_entry: start 2026-02-24T22:17:43.585 INFO:tasks.workunit.client.0.trial046.stderr: -27> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 cache get: name=default.rgw.meta+root+abort-at-trim-reshard-log-entries : miss 2026-02-24T22:17:43.585 INFO:tasks.workunit.client.0.trial046.stderr: -26> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:43.585 INFO:tasks.workunit.client.0.trial046.stderr: -25> 2026-02-24T22:17:43.528+0000 7f8da1ec6640 10 monclient: get_auth_request con 0x558d5122aa60 auth_method 0 2026-02-24T22:17:43.585 INFO:tasks.workunit.client.0.trial046.stderr: -24> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:17:43.585 INFO:tasks.workunit.client.0.trial046.stderr: -23> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 cache put: name=default.rgw.meta+root+abort-at-trim-reshard-log-entries info.flags=0x11 2026-02-24T22:17:43.585 INFO:tasks.workunit.client.0.trial046.stderr: -22> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 adding default.rgw.meta+root+abort-at-trim-reshard-log-entries to cache LRU end 2026-02-24T22:17:43.585 INFO:tasks.workunit.client.0.trial046.stderr: -21> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 : miss 2026-02-24T22:17:43.585 INFO:tasks.workunit.client.0.trial046.stderr: -20> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:43.585 INFO:tasks.workunit.client.0.trial046.stderr: -19> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 20 rados_obj.operate() r=0 bl.length=459 2026-02-24T22:17:43.585 INFO:tasks.workunit.client.0.trial046.stderr: -18> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 info.flags=0x17 2026-02-24T22:17:43.585 INFO:tasks.workunit.client.0.trial046.stderr: -17> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 to cache LRU end 2026-02-24T22:17:43.585 INFO:tasks.workunit.client.0.trial046.stderr: -16> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:17:43.585 INFO:tasks.workunit.client.0.trial046.stderr: -15> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:43.585 INFO:tasks.workunit.client.0.trial046.stderr: -14> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 2026-02-24T22:17:43.585 INFO:tasks.workunit.client.0.trial046.stderr: -13> 2026-02-24T22:17:43.532+0000 7f8da3bf1a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 info.flags=0x17 2026-02-24T22:17:43.585 INFO:tasks.workunit.client.0.trial046.stderr: -12> 2026-02-24T22:17:43.532+0000 7f8da3bf1a80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 to cache LRU end 2026-02-24T22:17:43.585 INFO:tasks.workunit.client.0.trial046.stderr: -11> 2026-02-24T22:17:43.532+0000 7f8da3bf1a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:43.585 INFO:tasks.workunit.client.0.trial046.stderr: -10> 2026-02-24T22:17:43.532+0000 7f8da3bf1a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528, ofs0, ns] 2026-02-24T22:17:43.585 INFO:tasks.workunit.client.0.trial046.stderr: -9> 2026-02-24T22:17:43.532+0000 7f8d7cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961346 cookie 94065439058112 notifier 6704 bl.length()=1041 2026-02-24T22:17:43.585 INFO:tasks.workunit.client.0.trial046.stderr: -8> 2026-02-24T22:17:43.532+0000 7f8d7cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 info.flags=0x17 2026-02-24T22:17:43.585 INFO:tasks.workunit.client.0.trial046.stderr: -7> 2026-02-24T22:17:43.532+0000 7f8d7cff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 to cache LRU end 2026-02-24T22:17:43.585 INFO:tasks.workunit.client.0.trial046.stderr: -6> 2026-02-24T22:17:43.532+0000 7f8d7cff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:43.585 INFO:tasks.workunit.client.0.trial046.stderr: -5> 2026-02-24T22:17:43.532+0000 7f8da3bf1a80 20 handle_bi_update(): bucket=:abort-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528]): orig_sources= new_sources= 2026-02-24T22:17:43.585 INFO:tasks.workunit.client.0.trial046.stderr: -4> 2026-02-24T22:17:43.532+0000 7f8da3bf1a80 20 handle_bi_update(): bucket=:abort-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528]): potential sources added=[] removed=[] 2026-02-24T22:17:43.585 INFO:tasks.workunit.client.0.trial046.stderr: -3> 2026-02-24T22:17:43.532+0000 7f8da3bf1a80 20 handle_bi_update(): bucket=:abort-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528]): orig_dests= new_dests= 2026-02-24T22:17:43.586 INFO:tasks.workunit.client.0.trial046.stderr: -2> 2026-02-24T22:17:43.532+0000 7f8da3bf1a80 20 handle_bi_update(): bucket=:abort-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528]): potential dests added=[] removed=[] 2026-02-24T22:17:43.586 INFO:tasks.workunit.client.0.trial046.stderr: -1> 2026-02-24T22:17:43.532+0000 7f8da3bf1a80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f8da3bf1a80 time 2026-02-24T22:17:43.537470+0000 2026-02-24T22:17:43.586 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:17:43.586 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.586 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:17:43.586 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f8da7033eb1] 2026-02-24T22:17:43.586 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x558d0fbfab9f] 2026-02-24T22:17:43.586 INFO:tasks.workunit.client.0.trial046.stderr: 3: radosgw-admin(+0x17143e1) [0x558d10e093e1] 2026-02-24T22:17:43.586 INFO:tasks.workunit.client.0.trial046.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x558d10366202] 2026-02-24T22:17:43.586 INFO:tasks.workunit.client.0.trial046.stderr: 5: main() 2026-02-24T22:17:43.586 INFO:tasks.workunit.client.0.trial046.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f8da6333d90] 2026-02-24T22:17:43.586 INFO:tasks.workunit.client.0.trial046.stderr: 7: __libc_start_main() 2026-02-24T22:17:43.586 INFO:tasks.workunit.client.0.trial046.stderr: 8: _start() 2026-02-24T22:17:43.586 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.586 INFO:tasks.workunit.client.0.trial046.stderr: 0> 2026-02-24T22:17:43.532+0000 7f8da3bf1a80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:17:43.586 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f8da3bf1a80 thread_name:radosgw-admin 2026-02-24T22:17:43.586 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.586 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:17:43.586 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f8da634c520] 2026-02-24T22:17:43.586 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:17:43.586 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:17:43.586 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:17:43.586 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f8da7033f10] 2026-02-24T22:17:43.586 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x558d0fbfab9f] 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr: 7: radosgw-admin(+0x17143e1) [0x558d10e093e1] 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x558d10366202] 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr: 9: main() 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f8da6333d90] 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr: 11: __libc_start_main() 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr: 12: _start() 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr:--- logging levels --- 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 none 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 lockdep 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 context 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 crush 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_balancer 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_locker 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log_expire 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_migrator 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr: 3/ 5 mds_quiesce 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 buffer 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 timer 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 filer 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 striper 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 objecter 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rados 2026-02-24T22:17:43.587 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_mirror 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_replay 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_pwl 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 journaler 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objectcacher 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 immutable_obj_cache 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 client 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 osd 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 optracker 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objclass 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 filestore 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 journal 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 ms 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mon 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 0/10 monc 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 paxos 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 tp 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 auth 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 crypto 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 finisher 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 reserver 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 heartbeatmap 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 perfcounter 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_sync 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_datacache 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_access 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dbstore 2026-02-24T22:17:43.588 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_flight 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_lifecycle 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_restore 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw_notification 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_bucket_logging 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 javaclient 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 asok 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 throttle 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 refs 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 compressor 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore_compression 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluefs 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 bdev 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 kstore 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 4/ 5 rocksdb 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 fuse 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 2/ 5 mgr 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mgrc 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 dpdk 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 eventtrace 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 prioritycache 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 test 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephfs_mirror 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephsqlite 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 crimson_interrupt 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore 2026-02-24T22:17:43.589 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_onode 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_odata 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_omap 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_tm 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_t 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cleaner 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_epm 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_lba 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_fixedkv_tree 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cache 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_journal 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_device 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_backref 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 alienstore 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mclock 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dedup 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cyanstore 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 ceph_exporter 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 memstore 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 trace 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 ceph_dedup 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: -2/-2 (syslog threshold) 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 99/99 (stderr threshold) 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr:--- pthread ID / name mapping for recent threads --- 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 7f8d4cfe1640 / http_manager 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 7f8d4e7e4640 / http_manager 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 7f8d6affd640 / ms_dispatch 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 7f8d7cff9640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 7f8d7effd640 / ms_dispatch 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 7f8d85ffb640 / ms_dispatch 2026-02-24T22:17:43.590 INFO:tasks.workunit.client.0.trial046.stderr: 7f8d877fe640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: 7f8d87fff640 / admin_socket 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: 7f8d93fff640 / ms_dispatch 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: 7f8da0ec4640 / msgr-worker-2 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: 7f8da16c5640 / msgr-worker-1 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: 7f8da1ec6640 / msgr-worker-0 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: 7f8da26c7640 / io_context_pool 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: 7f8da3bf1a80 / radosgw-admin 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: max_recent 500 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: max_new 1000 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: log_file /var/log/ceph/ceph-client.admin.45717.log 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr:--- end dump of recent events --- 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr:--- begin dump of recent events --- 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: -499> 2026-02-24T22:17:43.472+0000 7f8da16c5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: -498> 2026-02-24T22:17:43.472+0000 7f8da16c5640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: -497> 2026-02-24T22:17:43.472+0000 7f8da16c5640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: -496> 2026-02-24T22:17:43.472+0000 7f8da1ec6640 10 monclient(hunting): get_auth_request con 0x558d50cf2c20 auth_method 0 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: -495> 2026-02-24T22:17:43.472+0000 7f8da1ec6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: -494> 2026-02-24T22:17:43.472+0000 7f8da1ec6640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: -493> 2026-02-24T22:17:43.472+0000 7f8da1ec6640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: -492> 2026-02-24T22:17:43.472+0000 7f8da16c5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: -491> 2026-02-24T22:17:43.472+0000 7f8da16c5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: -490> 2026-02-24T22:17:43.472+0000 7f8da16c5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: -489> 2026-02-24T22:17:43.472+0000 7f8da1ec6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: -488> 2026-02-24T22:17:43.472+0000 7f8da1ec6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: -487> 2026-02-24T22:17:43.472+0000 7f8da1ec6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: -486> 2026-02-24T22:17:43.472+0000 7f8da16c5640 10 monclient(hunting): handle_auth_done global_id 6692 payload 274 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: -485> 2026-02-24T22:17:43.472+0000 7f8da16c5640 10 monclient: _finish_hunting 0 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: -484> 2026-02-24T22:17:43.472+0000 7f8da16c5640 1 monclient: found mon.noname-c 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: -483> 2026-02-24T22:17:43.472+0000 7f8da16c5640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:17:43.591 INFO:tasks.workunit.client.0.trial046.stderr: -482> 2026-02-24T22:17:43.472+0000 7f8d877fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr: -481> 2026-02-24T22:17:43.472+0000 7f8d877fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr: -480> 2026-02-24T22:17:43.472+0000 7f8d877fe640 10 monclient: dump: 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr: -479> 2026-02-24T22:17:43.472+0000 7f8d877fe640 10 monclient: _finish_auth 0 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr: -478> 2026-02-24T22:17:43.472+0000 7f8d877fe640 10 monclient: _check_auth_tickets 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr: -477> 2026-02-24T22:17:43.472+0000 7f8d877fe640 10 monclient: handle_config config(0 keys) 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr: -476> 2026-02-24T22:17:43.472+0000 7f8d877fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr: -475> 2026-02-24T22:17:43.472+0000 7f8d877fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr: -474> 2026-02-24T22:17:43.472+0000 7f8d877fe640 10 monclient: dump: 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.592 INFO:tasks.workunit.client.0.trial046.stderr: -473> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 monclient: authenticate success, global_id 6692 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr: -472> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 10 monclient: get_monmap_and_config success 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr: -471> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 4 set_mon_vals no callback set 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr: -470> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 10 monclient: shutdown 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr: -469> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) unregister_commands rotate-key 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr: -468> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 10 monclient: build_initial_monmap 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr: -467> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 1 build_initial for_mkfs: 0 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr: -466> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 10 monclient: monmap: 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr: -465> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding auth protocol: cephx 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr: -464> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding auth protocol: cephx 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr: -463> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding auth protocol: cephx 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr: -462> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding con mode: secure 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr: -461> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding con mode: crc 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr: -460> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding con mode: secure 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr: -459> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding con mode: crc 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr: -458> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding con mode: secure 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr: -457> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding con mode: crc 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr: -456> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding con mode: crc 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr: -455> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding con mode: secure 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr: -454> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding con mode: crc 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr: -453> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding con mode: secure 2026-02-24T22:17:43.593 INFO:tasks.workunit.client.0.trial046.stderr: -452> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding con mode: crc 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -451> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cc3840) adding con mode: secure 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -450> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -449> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 1 librados: starting msgr at 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -448> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 1 librados: starting objecter 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -447> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command objecter_requests hook 0x558d50cc29e0 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -446> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 1 librados: setting wanted keys 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -445> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 1 librados: calling monclient init 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -444> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 10 monclient: init 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -443> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding auth protocol: cephx 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -442> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding auth protocol: cephx 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -441> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding auth protocol: cephx 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -440> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding con mode: secure 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -439> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding con mode: crc 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -438> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding con mode: secure 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -437> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding con mode: crc 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -436> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding con mode: secure 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -435> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding con mode: crc 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -434> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding con mode: crc 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -433> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding con mode: secure 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -432> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding con mode: crc 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -431> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding con mode: secure 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -430> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding con mode: crc 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -429> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50cbe6f0) adding con mode: secure 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -428> 2026-02-24T22:17:43.472+0000 7f8da3bf1a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -427> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -426> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command rotate-key hook 0x558d50cbe838 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -425> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient: _reopen_session rank -1 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -424> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -423> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient(hunting): picked mon.noname-a con 0x558d50c487a0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -422> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient(hunting): picked mon.noname-c con 0x558d50c4b010 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:17:43.594 INFO:tasks.workunit.client.0.trial046.stderr: -421> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient(hunting): picked mon.noname-b con 0x558d50cf2c20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -420> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -419> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -418> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -417> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient(hunting): _renew_subs 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -416> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient(hunting): authenticate will time out at 1833.584129s 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -415> 2026-02-24T22:17:43.476+0000 7f8da16c5640 10 monclient(hunting): get_auth_request con 0x558d50c487a0 auth_method 0 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -414> 2026-02-24T22:17:43.476+0000 7f8da16c5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -413> 2026-02-24T22:17:43.476+0000 7f8da16c5640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -412> 2026-02-24T22:17:43.476+0000 7f8da16c5640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -411> 2026-02-24T22:17:43.476+0000 7f8da0ec4640 10 monclient(hunting): get_auth_request con 0x558d50c4b010 auth_method 0 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -410> 2026-02-24T22:17:43.476+0000 7f8da0ec4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -409> 2026-02-24T22:17:43.476+0000 7f8da0ec4640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -408> 2026-02-24T22:17:43.476+0000 7f8da0ec4640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -407> 2026-02-24T22:17:43.476+0000 7f8da16c5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -406> 2026-02-24T22:17:43.476+0000 7f8da16c5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -405> 2026-02-24T22:17:43.476+0000 7f8da16c5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -404> 2026-02-24T22:17:43.476+0000 7f8da0ec4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -403> 2026-02-24T22:17:43.476+0000 7f8da0ec4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -402> 2026-02-24T22:17:43.476+0000 7f8da0ec4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -401> 2026-02-24T22:17:43.476+0000 7f8da1ec6640 10 monclient(hunting): get_auth_request con 0x558d50cf2c20 auth_method 0 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -400> 2026-02-24T22:17:43.476+0000 7f8da1ec6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -399> 2026-02-24T22:17:43.476+0000 7f8da1ec6640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -398> 2026-02-24T22:17:43.476+0000 7f8da1ec6640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -397> 2026-02-24T22:17:43.476+0000 7f8da16c5640 10 monclient(hunting): handle_auth_done global_id 7149 payload 824 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -396> 2026-02-24T22:17:43.476+0000 7f8da16c5640 10 monclient: _finish_hunting 0 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -395> 2026-02-24T22:17:43.476+0000 7f8da16c5640 1 monclient: found mon.noname-a 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -394> 2026-02-24T22:17:43.476+0000 7f8da16c5640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -393> 2026-02-24T22:17:43.476+0000 7f8d85ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -392> 2026-02-24T22:17:43.476+0000 7f8d85ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr: -391> 2026-02-24T22:17:43.476+0000 7f8d85ffb640 10 monclient: dump: 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:43.595 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr: -390> 2026-02-24T22:17:43.476+0000 7f8d85ffb640 10 monclient: _finish_auth 0 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr: -389> 2026-02-24T22:17:43.476+0000 7f8d85ffb640 10 monclient: _check_auth_tickets 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr: -388> 2026-02-24T22:17:43.476+0000 7f8d85ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr: -387> 2026-02-24T22:17:43.476+0000 7f8d85ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr: -386> 2026-02-24T22:17:43.476+0000 7f8d85ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr: -385> 2026-02-24T22:17:43.476+0000 7f8d85ffb640 10 monclient: dump: 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr: -384> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 5 monclient: authenticate success, global_id 7149 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr: -383> 2026-02-24T22:17:43.476+0000 7f8d877fe640 4 set_mon_vals no callback set 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr: -382> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient: _renew_subs 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr: -381> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr: -380> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient: _renew_subs 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr: -379> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr: -378> 2026-02-24T22:17:43.476+0000 7f8da3bf1a80 1 librados: init done 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr: -377> 2026-02-24T22:17:43.476+0000 7f8d85ffb640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:17:43.596 INFO:tasks.workunit.client.0.trial046.stderr: -376> 2026-02-24T22:17:43.476+0000 7f8d85ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr: -375> 2026-02-24T22:17:43.476+0000 7f8d85ffb640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr: -374> 2026-02-24T22:17:43.476+0000 7f8da0ec4640 10 monclient: get_auth_request con 0x7f8d800707b0 auth_method 0 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr: -373> 2026-02-24T22:17:43.476+0000 7f8da1ec6640 10 monclient: get_auth_request con 0x558d50d20570 auth_method 0 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr: -372> 2026-02-24T22:17:43.476+0000 7f8da16c5640 10 monclient: get_auth_request con 0x558d50d4a0f0 auth_method 0 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr: -371> 2026-02-24T22:17:43.480+0000 7f8da0ec4640 10 monclient: get_auth_request con 0x558d50d4ba60 auth_method 0 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr: -370> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient: build_initial_monmap 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr: -369> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 1 build_initial for_mkfs: 0 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr: -368> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient: monmap: 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr: -367> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding auth protocol: cephx 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr: -366> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding auth protocol: cephx 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr: -365> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding auth protocol: cephx 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr: -364> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding con mode: secure 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr: -363> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding con mode: crc 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr: -362> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding con mode: secure 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr: -361> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding con mode: crc 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr: -360> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding con mode: secure 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr: -359> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding con mode: crc 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr: -358> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding con mode: crc 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr: -357> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding con mode: secure 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr: -356> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding con mode: crc 2026-02-24T22:17:43.597 INFO:tasks.workunit.client.0.trial046.stderr: -355> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding con mode: secure 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -354> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding con mode: crc 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -353> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4ef30) adding con mode: secure 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -352> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -351> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command objecter_requests cmddesc objecter_requests hook 0x558d50d53e10 EEXIST 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -350> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient: init 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -349> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding auth protocol: cephx 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -348> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding auth protocol: cephx 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -347> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding auth protocol: cephx 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -346> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding con mode: secure 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -345> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding con mode: crc 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -344> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding con mode: secure 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -343> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding con mode: crc 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -342> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding con mode: secure 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -341> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding con mode: crc 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -340> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding con mode: crc 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -339> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding con mode: secure 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -338> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding con mode: crc 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -337> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding con mode: secure 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -336> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding con mode: crc 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -335> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50d4c3c8) adding con mode: secure 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -334> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -333> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -332> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command rotate-key cmddesc rotate-key hook 0x558d50d4c510 EEXIST 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -331> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient: _reopen_session rank -1 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -330> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -329> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient(hunting): picked mon.noname-c con 0x558d50ea7700 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -328> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient(hunting): picked mon.noname-b con 0x558d50ea48f0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -327> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient(hunting): picked mon.noname-a con 0x558d50ea0880 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -326> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -325> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -324> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -323> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient(hunting): _renew_subs 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -322> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient(hunting): authenticate will time out at 1833.589779s 2026-02-24T22:17:43.598 INFO:tasks.workunit.client.0.trial046.stderr: -321> 2026-02-24T22:17:43.480+0000 7f8da16c5640 10 monclient(hunting): get_auth_request con 0x558d50ea7700 auth_method 0 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr: -320> 2026-02-24T22:17:43.480+0000 7f8da16c5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr: -319> 2026-02-24T22:17:43.480+0000 7f8da16c5640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr: -318> 2026-02-24T22:17:43.480+0000 7f8da16c5640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr: -317> 2026-02-24T22:17:43.480+0000 7f8da1ec6640 10 monclient(hunting): get_auth_request con 0x558d50ea0880 auth_method 0 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr: -316> 2026-02-24T22:17:43.480+0000 7f8da1ec6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr: -315> 2026-02-24T22:17:43.480+0000 7f8da1ec6640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr: -314> 2026-02-24T22:17:43.480+0000 7f8da1ec6640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr: -313> 2026-02-24T22:17:43.480+0000 7f8da16c5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr: -312> 2026-02-24T22:17:43.480+0000 7f8da16c5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr: -311> 2026-02-24T22:17:43.480+0000 7f8da16c5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr: -310> 2026-02-24T22:17:43.480+0000 7f8da1ec6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr: -309> 2026-02-24T22:17:43.480+0000 7f8da1ec6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr: -308> 2026-02-24T22:17:43.480+0000 7f8da1ec6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr: -307> 2026-02-24T22:17:43.480+0000 7f8da0ec4640 10 monclient(hunting): get_auth_request con 0x558d50ea48f0 auth_method 0 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr: -306> 2026-02-24T22:17:43.480+0000 7f8da0ec4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr: -305> 2026-02-24T22:17:43.480+0000 7f8da0ec4640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr: -304> 2026-02-24T22:17:43.480+0000 7f8da0ec4640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr: -303> 2026-02-24T22:17:43.480+0000 7f8da16c5640 10 monclient(hunting): handle_auth_done global_id 6698 payload 824 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr: -302> 2026-02-24T22:17:43.480+0000 7f8da16c5640 10 monclient: _finish_hunting 0 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr: -301> 2026-02-24T22:17:43.480+0000 7f8da16c5640 1 monclient: found mon.noname-c 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr: -300> 2026-02-24T22:17:43.480+0000 7f8da16c5640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr: -299> 2026-02-24T22:17:43.480+0000 7f8d7effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr: -298> 2026-02-24T22:17:43.480+0000 7f8d7effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr: -297> 2026-02-24T22:17:43.480+0000 7f8d7effd640 10 monclient: dump: 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:43.599 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr: -296> 2026-02-24T22:17:43.480+0000 7f8d7effd640 10 monclient: _finish_auth 0 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr: -295> 2026-02-24T22:17:43.480+0000 7f8d7effd640 10 monclient: _check_auth_tickets 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr: -294> 2026-02-24T22:17:43.480+0000 7f8d7effd640 10 monclient: handle_config config(0 keys) 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr: -293> 2026-02-24T22:17:43.480+0000 7f8d7effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr: -292> 2026-02-24T22:17:43.480+0000 7f8d7effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr: -291> 2026-02-24T22:17:43.480+0000 7f8d7effd640 10 monclient: dump: 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr: -290> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 monclient: authenticate success, global_id 6698 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr: -289> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient: _renew_subs 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr: -288> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr: -287> 2026-02-24T22:17:43.480+0000 7f8da26c7640 4 set_mon_vals no callback set 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr: -286> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient: _renew_subs 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr: -285> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr: -284> 2026-02-24T22:17:43.480+0000 7f8d7effd640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr: -283> 2026-02-24T22:17:43.480+0000 7f8d7effd640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr: -282> 2026-02-24T22:17:43.480+0000 7f8d7effd640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr: -281> 2026-02-24T22:17:43.480+0000 7f8da0ec4640 10 monclient: get_auth_request con 0x7f8d740706c0 auth_method 0 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr: -280> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient: get_monmap_and_config 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr: -279> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient: build_initial_monmap 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr: -278> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 1 build_initial for_mkfs: 0 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr: -277> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 10 monclient: monmap: 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:17:43.600 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -276> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding auth protocol: cephx 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -275> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding auth protocol: cephx 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -274> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding auth protocol: cephx 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -273> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding con mode: secure 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -272> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding con mode: crc 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -271> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding con mode: secure 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -270> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding con mode: crc 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -269> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding con mode: secure 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -268> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding con mode: crc 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -267> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding con mode: crc 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -266> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding con mode: secure 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -265> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding con mode: crc 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -264> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding con mode: secure 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -263> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding con mode: crc 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -262> 2026-02-24T22:17:43.480+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f38670) adding con mode: secure 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -261> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -260> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: init 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -259> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding auth protocol: cephx 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -258> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding auth protocol: cephx 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -257> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding auth protocol: cephx 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -256> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding con mode: secure 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -255> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding con mode: crc 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -254> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding con mode: secure 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -253> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding con mode: crc 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -252> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding con mode: secure 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -251> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding con mode: crc 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -250> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding con mode: crc 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -249> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding con mode: secure 2026-02-24T22:17:43.601 INFO:tasks.workunit.client.0.trial046.stderr: -248> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding con mode: crc 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -247> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding con mode: secure 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -246> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding con mode: crc 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -245> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4d6c0) adding con mode: secure 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -244> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -243> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -242> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command rotate-key cmddesc rotate-key hook 0x7ffe3af4d808 EEXIST 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -241> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: _reopen_session rank -1 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -240> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -239> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): picked mon.noname-b con 0x558d50f5f760 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -238> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): picked mon.noname-a con 0x558d50f62ed0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -237> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): picked mon.noname-c con 0x558d50f3dc90 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -236> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -235> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -234> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -233> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): _renew_subs 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -232> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): authenticate will time out at 1833.592444s 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -231> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient(hunting): get_auth_request con 0x558d50f3dc90 auth_method 0 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -230> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -229> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -228> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -227> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): get_auth_request con 0x558d50f62ed0 auth_method 0 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -226> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -225> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -224> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -223> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -222> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -221> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -220> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -219> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -218> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -217> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient(hunting): handle_auth_done global_id 6701 payload 274 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -216> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient: _finish_hunting 0 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -215> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 1 monclient: found mon.noname-c 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -214> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -213> 2026-02-24T22:17:43.484+0000 7f8da16c5640 10 monclient: get_auth_request con 0x558d50f5f760 auth_method 0 2026-02-24T22:17:43.602 INFO:tasks.workunit.client.0.trial046.stderr: -212> 2026-02-24T22:17:43.484+0000 7f8d7cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr: -211> 2026-02-24T22:17:43.484+0000 7f8d7cff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr: -210> 2026-02-24T22:17:43.484+0000 7f8d7cff9640 10 monclient: dump: 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr: -209> 2026-02-24T22:17:43.484+0000 7f8d7cff9640 10 monclient: _finish_auth 0 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr: -208> 2026-02-24T22:17:43.484+0000 7f8d7cff9640 10 monclient: _check_auth_tickets 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr: -207> 2026-02-24T22:17:43.484+0000 7f8d7cff9640 10 monclient: handle_config config(0 keys) 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr: -206> 2026-02-24T22:17:43.484+0000 7f8d7cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr: -205> 2026-02-24T22:17:43.484+0000 7f8d7cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr: -204> 2026-02-24T22:17:43.484+0000 7f8d7cff9640 10 monclient: dump: 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr: -203> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 monclient: authenticate success, global_id 6701 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr: -202> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: get_monmap_and_config success 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr: -201> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 4 set_mon_vals no callback set 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr: -200> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: shutdown 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr: -199> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: build_initial_monmap 2026-02-24T22:17:43.603 INFO:tasks.workunit.client.0.trial046.stderr: -198> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 1 build_initial for_mkfs: 0 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -197> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: monmap: 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -196> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding auth protocol: cephx 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -195> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding auth protocol: cephx 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -194> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding auth protocol: cephx 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -193> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding con mode: secure 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -192> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding con mode: crc 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -191> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding con mode: secure 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -190> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding con mode: crc 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -189> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding con mode: secure 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -188> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding con mode: crc 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -187> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding con mode: crc 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -186> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding con mode: secure 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -185> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding con mode: crc 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -184> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding con mode: secure 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -183> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding con mode: crc 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -182> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50f8f970) adding con mode: secure 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -181> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -180> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 1 librados: starting msgr at 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -179> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 1 librados: starting objecter 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -178> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command objecter_requests cmddesc objecter_requests hook 0x558d50f6a5f0 EEXIST 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -177> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 1 librados: setting wanted keys 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -176> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 1 librados: calling monclient init 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -175> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: init 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -174> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding auth protocol: cephx 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -173> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding auth protocol: cephx 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -172> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding auth protocol: cephx 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -171> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding con mode: secure 2026-02-24T22:17:43.604 INFO:tasks.workunit.client.0.trial046.stderr: -170> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding con mode: crc 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -169> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding con mode: secure 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -168> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding con mode: crc 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -167> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding con mode: secure 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -166> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding con mode: crc 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -165> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding con mode: crc 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -164> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding con mode: secure 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -163> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding con mode: crc 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -162> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding con mode: secure 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -161> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding con mode: crc 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -160> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 AuthRegistry(0x558d50ed6300) adding con mode: secure 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -159> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -158> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -157> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command rotate-key cmddesc rotate-key hook 0x558d50ed6448 EEXIST 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -156> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: _reopen_session rank -1 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -155> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -154> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): picked mon.noname-a con 0x558d50f5f760 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -153> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): picked mon.noname-c con 0x558d50f62ed0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -152> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): picked mon.noname-b con 0x558d50f8d4e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -151> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -150> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -149> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -148> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): _renew_subs 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -147> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient(hunting): authenticate will time out at 1833.593790s 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -146> 2026-02-24T22:17:43.484+0000 7f8da16c5640 10 monclient(hunting): get_auth_request con 0x558d50f5f760 auth_method 0 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -145> 2026-02-24T22:17:43.484+0000 7f8da16c5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -144> 2026-02-24T22:17:43.484+0000 7f8da16c5640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -143> 2026-02-24T22:17:43.484+0000 7f8da16c5640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -142> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): get_auth_request con 0x558d50f62ed0 auth_method 0 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -141> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -140> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -139> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -138> 2026-02-24T22:17:43.484+0000 7f8da16c5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -137> 2026-02-24T22:17:43.484+0000 7f8da16c5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -136> 2026-02-24T22:17:43.484+0000 7f8da16c5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -135> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -134> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:17:43.605 INFO:tasks.workunit.client.0.trial046.stderr: -133> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr: -132> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient(hunting): get_auth_request con 0x558d50f8d4e0 auth_method 0 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr: -131> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr: -130> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr: -129> 2026-02-24T22:17:43.484+0000 7f8da1ec6640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr: -128> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient(hunting): handle_auth_done global_id 6704 payload 824 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr: -127> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient: _finish_hunting 0 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr: -126> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 1 monclient: found mon.noname-c 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr: -125> 2026-02-24T22:17:43.484+0000 7f8da0ec4640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr: -124> 2026-02-24T22:17:43.484+0000 7f8d6affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr: -123> 2026-02-24T22:17:43.484+0000 7f8d6affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr: -122> 2026-02-24T22:17:43.484+0000 7f8d6affd640 10 monclient: dump: 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr: -121> 2026-02-24T22:17:43.484+0000 7f8d6affd640 10 monclient: _finish_auth 0 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr: -120> 2026-02-24T22:17:43.484+0000 7f8d6affd640 10 monclient: _check_auth_tickets 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr: -119> 2026-02-24T22:17:43.484+0000 7f8d6affd640 10 monclient: handle_config config(0 keys) 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr: -118> 2026-02-24T22:17:43.484+0000 7f8d6affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr: -117> 2026-02-24T22:17:43.484+0000 7f8d6affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr: -116> 2026-02-24T22:17:43.484+0000 7f8d6affd640 10 monclient: dump: 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr: -115> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 monclient: authenticate success, global_id 6704 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr: -114> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: _renew_subs 2026-02-24T22:17:43.606 INFO:tasks.workunit.client.0.trial046.stderr: -113> 2026-02-24T22:17:43.484+0000 7f8d7cff9640 4 set_mon_vals no callback set 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -112> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -111> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: _renew_subs 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -110> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -109> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 1 librados: init done 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -108> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command cr dump hook 0x558d50f66f88 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -107> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 4 Realm: () 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -106> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -105> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -104> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -103> 2026-02-24T22:17:43.484+0000 7f8da3bf1a80 20 zonegroup default 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -102> 2026-02-24T22:17:43.484+0000 7f8da16c5640 10 monclient: get_auth_request con 0x558d51000c00 auth_method 0 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -101> 2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -100> 2026-02-24T22:17:43.488+0000 7f8d6affd640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -99> 2026-02-24T22:17:43.488+0000 7f8d6affd640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -98> 2026-02-24T22:17:43.488+0000 7f8d6affd640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -97> 2026-02-24T22:17:43.488+0000 7f8da1ec6640 10 monclient: get_auth_request con 0x7f8d640707b0 auth_method 0 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -96> 2026-02-24T22:17:43.488+0000 7f8da16c5640 10 monclient: get_auth_request con 0x558d51023f70 auth_method 0 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -95> 2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -94> 2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -93> 2026-02-24T22:17:43.488+0000 7f8da0ec4640 10 monclient: get_auth_request con 0x558d51045620 auth_method 0 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -92> 2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -91> 2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 started sync module instance, tier type = 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -90> 2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -89> 2026-02-24T22:17:43.488+0000 7f8da1ec6640 10 monclient: get_auth_request con 0x558d511ae4b0 auth_method 0 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -88> 2026-02-24T22:17:43.488+0000 7f8da1ec6640 10 monclient: get_auth_request con 0x558d510a6a70 auth_method 0 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -87> 2026-02-24T22:17:43.488+0000 7f8da16c5640 10 monclient: get_auth_request con 0x558d511093f0 auth_method 0 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -86> 2026-02-24T22:17:43.488+0000 7f8da0ec4640 10 monclient: get_auth_request con 0x558d5116c370 auth_method 0 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -85> 2026-02-24T22:17:43.488+0000 7f8da16c5640 10 monclient: get_auth_request con 0x558d512114b0 auth_method 0 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -84> 2026-02-24T22:17:43.488+0000 7f8da16c5640 0 -- 10.20.193.46:0/3123648294 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x558d512114b0 msgr2=0x558d51231910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -83> 2026-02-24T22:17:43.488+0000 7f8d6affd640 1 client.6704.objecter ms_handle_reset 0x558d512114b0 session 0x558d51210da0 osd.6 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -82> 2026-02-24T22:17:43.488+0000 7f8d6affd640 10 monclient: _renew_subs 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -81> 2026-02-24T22:17:43.488+0000 7f8d6affd640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -80> 2026-02-24T22:17:43.488+0000 7f8da16c5640 10 monclient: get_auth_request con 0x7f8d6409d460 auth_method 0 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -79> 2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 add_watcher() i=1 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -78> 2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 add_watcher() i=5 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -77> 2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 add_watcher() i=3 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -76> 2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 add_watcher() i=0 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -75> 2026-02-24T22:17:43.488+0000 7f8da3bf1a80 20 add_watcher() i=6 2026-02-24T22:17:43.607 INFO:tasks.workunit.client.0.trial046.stderr: -74> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 20 add_watcher() i=2 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -73> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 20 add_watcher() i=4 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -72> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 20 add_watcher() i=7 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -71> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -70> 2026-02-24T22:17:43.492+0000 7f8da0ec4640 10 monclient: get_auth_request con 0x7f8cc0008af0 auth_method 0 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -69> 2026-02-24T22:17:43.492+0000 7f8da0ec4640 0 -- 10.20.193.46:0/2381496400 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f8cc0008af0 msgr2=0x7f8cc0028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -68> 2026-02-24T22:17:43.492+0000 7f8d7effd640 1 client.6698.objecter ms_handle_reset 0x7f8cc0008af0 session 0x7f8cc0008470 osd.1 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -67> 2026-02-24T22:17:43.492+0000 7f8d7effd640 10 monclient: _renew_subs 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -66> 2026-02-24T22:17:43.492+0000 7f8d7effd640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -65> 2026-02-24T22:17:43.492+0000 7f8da0ec4640 10 monclient: get_auth_request con 0x7f8d7409d3f0 auth_method 0 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -64> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding auth protocol: cephx 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -63> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding auth protocol: cephx 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -62> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding auth protocol: cephx 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -61> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding con mode: secure 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -60> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding con mode: crc 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -59> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding con mode: secure 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -58> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding con mode: crc 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -57> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding con mode: secure 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -56> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding con mode: crc 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -55> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding con mode: crc 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -54> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding con mode: secure 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -53> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding con mode: crc 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -52> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding con mode: secure 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -51> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding con mode: crc 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -50> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 AuthRegistry(0x7ffe3af4db30) adding con mode: secure 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -49> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -48> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -47> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -46> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command cache list hook 0x558d50f6c960 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -45> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command cache inspect hook 0x558d50f6c960 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -44> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command cache erase hook 0x558d50f6c960 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -43> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command cache zap hook 0x558d50f6c960 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -42> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 note: GC not initialized 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -41> 2026-02-24T22:17:43.492+0000 7f8d4e7e4640 20 reqs_thread_entry: start 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -40> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command sync trace show hook 0x7f8d940a6740 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -39> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command sync trace history hook 0x7f8d940a6740 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -38> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command sync trace active hook 0x7f8d940a6740 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -37> 2026-02-24T22:17:43.492+0000 7f8da3bf1a80 5 asok(0x558d50bbcf00) register_command sync trace active_short hook 0x7f8d940a6740 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -36> 2026-02-24T22:17:43.492+0000 7f8da1ec6640 10 monclient: get_auth_request con 0x558d511a1940 auth_method 0 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -35> 2026-02-24T22:17:43.496+0000 7f8da16c5640 10 monclient: get_auth_request con 0x558d5119a400 auth_method 0 2026-02-24T22:17:43.608 INFO:tasks.workunit.client.0.trial046.stderr: -34> 2026-02-24T22:17:43.496+0000 7f8da0ec4640 10 monclient: get_auth_request con 0x558d5119df90 auth_method 0 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -33> 2026-02-24T22:17:43.496+0000 7f8da1ec6640 10 monclient: get_auth_request con 0x558d5119d680 auth_method 0 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -32> 2026-02-24T22:17:43.500+0000 7f8da16c5640 10 monclient: get_auth_request con 0x558d51198110 auth_method 0 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -31> 2026-02-24T22:17:43.504+0000 7f8da0ec4640 10 monclient: get_auth_request con 0x558d51198c40 auth_method 0 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -30> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -29> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 20 Filter name: none 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -28> 2026-02-24T22:17:43.528+0000 7f8d4cfe1640 20 reqs_thread_entry: start 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -27> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 cache get: name=default.rgw.meta+root+abort-at-trim-reshard-log-entries : miss 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -26> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -25> 2026-02-24T22:17:43.528+0000 7f8da1ec6640 10 monclient: get_auth_request con 0x558d5122aa60 auth_method 0 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -24> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -23> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 cache put: name=default.rgw.meta+root+abort-at-trim-reshard-log-entries info.flags=0x11 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -22> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 adding default.rgw.meta+root+abort-at-trim-reshard-log-entries to cache LRU end 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -21> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 : miss 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -20> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 20 rados->read ofs=0 len=0 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -19> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 20 rados_obj.operate() r=0 bl.length=459 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -18> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 info.flags=0x17 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -17> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 to cache LRU end 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -16> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -15> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -14> 2026-02-24T22:17:43.528+0000 7f8da3bf1a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -13> 2026-02-24T22:17:43.532+0000 7f8da3bf1a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 info.flags=0x17 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -12> 2026-02-24T22:17:43.532+0000 7f8da3bf1a80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 to cache LRU end 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -11> 2026-02-24T22:17:43.532+0000 7f8da3bf1a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -10> 2026-02-24T22:17:43.532+0000 7f8da3bf1a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528, ofs0, ns] 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -9> 2026-02-24T22:17:43.532+0000 7f8d7cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961346 cookie 94065439058112 notifier 6704 bl.length()=1041 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -8> 2026-02-24T22:17:43.532+0000 7f8d7cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 info.flags=0x17 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -7> 2026-02-24T22:17:43.532+0000 7f8d7cff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 to cache LRU end 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -6> 2026-02-24T22:17:43.532+0000 7f8d7cff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -5> 2026-02-24T22:17:43.532+0000 7f8da3bf1a80 20 handle_bi_update(): bucket=:abort-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528]): orig_sources= new_sources= 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -4> 2026-02-24T22:17:43.532+0000 7f8da3bf1a80 20 handle_bi_update(): bucket=:abort-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528]): potential sources added=[] removed=[] 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -3> 2026-02-24T22:17:43.532+0000 7f8da3bf1a80 20 handle_bi_update(): bucket=:abort-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528]): orig_dests= new_dests= 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -2> 2026-02-24T22:17:43.532+0000 7f8da3bf1a80 20 handle_bi_update(): bucket=:abort-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528]): potential dests added=[] removed=[] 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: -1> 2026-02-24T22:17:43.532+0000 7f8da3bf1a80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f8da3bf1a80 time 2026-02-24T22:17:43.537470+0000 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f8da7033eb1] 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x558d0fbfab9f] 2026-02-24T22:17:43.609 INFO:tasks.workunit.client.0.trial046.stderr: 3: radosgw-admin(+0x17143e1) [0x558d10e093e1] 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x558d10366202] 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 5: main() 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f8da6333d90] 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 7: __libc_start_main() 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 8: _start() 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 0> 2026-02-24T22:17:43.532+0000 7f8da3bf1a80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f8da3bf1a80 thread_name:radosgw-admin 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f8da634c520] 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f8da7033f10] 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x558d0fbfab9f] 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 7: radosgw-admin(+0x17143e1) [0x558d10e093e1] 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x558d10366202] 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 9: main() 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f8da6333d90] 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 11: __libc_start_main() 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 12: _start() 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr:--- logging levels --- 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 none 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 lockdep 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 context 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 crush 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_balancer 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_locker 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log_expire 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_migrator 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 3/ 5 mds_quiesce 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 buffer 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 timer 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 filer 2026-02-24T22:17:43.610 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 striper 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 objecter 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rados 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_mirror 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_replay 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_pwl 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 journaler 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objectcacher 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 immutable_obj_cache 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 client 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 osd 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 optracker 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objclass 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 filestore 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 journal 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 ms 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mon 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 0/10 monc 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 paxos 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 tp 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 auth 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 crypto 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 finisher 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 reserver 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 heartbeatmap 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 perfcounter 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_sync 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_datacache 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_access 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dbstore 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_flight 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_lifecycle 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_restore 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw_notification 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_bucket_logging 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 javaclient 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 asok 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 throttle 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 refs 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 compressor 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore 2026-02-24T22:17:43.611 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore_compression 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluefs 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 bdev 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 kstore 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 4/ 5 rocksdb 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 fuse 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 2/ 5 mgr 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mgrc 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 dpdk 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 eventtrace 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 prioritycache 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 test 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephfs_mirror 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephsqlite 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 crimson_interrupt 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_onode 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_odata 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_omap 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_tm 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_t 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cleaner 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_epm 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_lba 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_fixedkv_tree 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cache 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_journal 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_device 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_backref 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 alienstore 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mclock 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dedup 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cyanstore 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 ceph_exporter 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 memstore 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 trace 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 ceph_dedup 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: -2/-2 (syslog threshold) 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 99/99 (stderr threshold) 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr:--- pthread ID / name mapping for recent threads --- 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 7f8d4cfe1640 / http_manager 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 7f8d4e7e4640 / http_manager 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 7f8d6affd640 / ms_dispatch 2026-02-24T22:17:43.612 INFO:tasks.workunit.client.0.trial046.stderr: 7f8d7cff9640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:17:43.613 INFO:tasks.workunit.client.0.trial046.stderr: 7f8d7effd640 / ms_dispatch 2026-02-24T22:17:43.613 INFO:tasks.workunit.client.0.trial046.stderr: 7f8d85ffb640 / ms_dispatch 2026-02-24T22:17:43.613 INFO:tasks.workunit.client.0.trial046.stderr: 7f8d877fe640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:17:43.613 INFO:tasks.workunit.client.0.trial046.stderr: 7f8d87fff640 / admin_socket 2026-02-24T22:17:43.613 INFO:tasks.workunit.client.0.trial046.stderr: 7f8d93fff640 / ms_dispatch 2026-02-24T22:17:43.613 INFO:tasks.workunit.client.0.trial046.stderr: 7f8da0ec4640 / msgr-worker-2 2026-02-24T22:17:43.613 INFO:tasks.workunit.client.0.trial046.stderr: 7f8da16c5640 / msgr-worker-1 2026-02-24T22:17:43.613 INFO:tasks.workunit.client.0.trial046.stderr: 7f8da1ec6640 / msgr-worker-0 2026-02-24T22:17:43.613 INFO:tasks.workunit.client.0.trial046.stderr: 7f8da26c7640 / io_context_pool 2026-02-24T22:17:43.613 INFO:tasks.workunit.client.0.trial046.stderr: 7f8da3bf1a80 / radosgw-admin 2026-02-24T22:17:43.613 INFO:tasks.workunit.client.0.trial046.stderr: max_recent 500 2026-02-24T22:17:43.613 INFO:tasks.workunit.client.0.trial046.stderr: max_new 1000 2026-02-24T22:17:43.613 INFO:tasks.workunit.client.0.trial046.stderr: log_file /var/lib/ceph/crash/2026-02-24T22:17:43.538901Z_3509e76b-258f-49c6-a3bd-77b3bb913d87/log 2026-02-24T22:17:43.613 INFO:tasks.workunit.client.0.trial046.stderr:--- end dump of recent events --- 2026-02-24T22:17:43.613 INFO:tasks.workunit.client.0.trial046.stderr:Aborted 2026-02-24T22:17:43.613 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket abort-at-trim-reshard-log-entries 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.572+0000 7f101527ea80 4 Realm: () 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.572+0000 7f101527ea80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.572+0000 7f101527ea80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.572+0000 7f101527ea80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.572+0000 7f101527ea80 20 zonegroup default 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.572+0000 7f101527ea80 20 rados->read ofs=0 len=0 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.572+0000 7f101527ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.572+0000 7f101527ea80 20 rados->read ofs=0 len=0 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.572+0000 7f101527ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.572+0000 7f101527ea80 20 started sync module instance, tier type = 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.572+0000 7f101527ea80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.576+0000 7f101527ea80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.576+0000 7f101527ea80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.576+0000 7f101527ea80 5 note: GC not initialized 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.576+0000 7f0fc17e2640 20 reqs_thread_entry: start 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.580+0000 7f1013553640 0 -- 10.20.193.46:0/3504959142 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x56016519dbc0 msgr2=0x5601651943b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.584+0000 7f1012d52640 0 -- 10.20.193.46:0/3504959142 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x56016516f080 msgr2=0x56016519cbc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.608+0000 7f101527ea80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.608+0000 7f101527ea80 20 Filter name: none 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.608+0000 7f0f2b7fe640 20 reqs_thread_entry: start 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.608+0000 7f101527ea80 20 rados->read ofs=0 len=0 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.608+0000 7f101527ea80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.608+0000 7f101527ea80 20 rados->read ofs=0 len=0 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.608+0000 7f101527ea80 20 rados_obj.operate() r=0 bl.length=495 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.608+0000 7f101527ea80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.608+0000 7f101527ea80 20 rados->read ofs=0 len=0 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.608+0000 7f101527ea80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.608+0000 7f101527ea80 20 rados->read ofs=0 len=0 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.608+0000 7f101527ea80 20 rados_obj.operate() r=0 bl.length=495 2026-02-24T22:17:43.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.608+0000 7f101527ea80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:17:43.618 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.618 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:17:43.619 INFO:tasks.workunit.client.0.trial046.stderr:bucket: abort-at-trim-reshard-log-entries id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:17:43.619 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:17:43.619 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:43.619 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:43.625 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:17:43.625 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:17:43.670 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket abort-at-trim-reshard-log-entries --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2026-02-24T22:17:43.686 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:17:43.686 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:17:43.705 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.704+0000 7f6dc7e6ea80 4 Realm: () 2026-02-24T22:17:43.705 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.704+0000 7f6dc7e6ea80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:17:43.705 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.704+0000 7f6dc7e6ea80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:17:43.705 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.704+0000 7f6dc7e6ea80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:17:43.705 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.704+0000 7f6dc7e6ea80 20 zonegroup default 2026-02-24T22:17:43.705 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.704+0000 7f6dc7e6ea80 20 rados->read ofs=0 len=0 2026-02-24T22:17:43.706 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.708+0000 7f6dc7e6ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:43.706 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.708+0000 7f6dc7e6ea80 20 rados->read ofs=0 len=0 2026-02-24T22:17:43.706 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.708+0000 7f6dc7e6ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:17:43.706 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.708+0000 7f6dc7e6ea80 20 started sync module instance, tier type = 2026-02-24T22:17:43.707 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.708+0000 7f6dc7e6ea80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:17:43.708 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.708+0000 7f6dc6143640 0 -- 10.20.193.46:0/3464697313 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x55a0364f0fe0 msgr2=0x55a036511460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:43.708 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.708+0000 7f6dc7e6ea80 20 add_watcher() i=1 2026-02-24T22:17:43.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.708+0000 7f6dc7e6ea80 20 add_watcher() i=3 2026-02-24T22:17:43.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.708+0000 7f6dc7e6ea80 20 add_watcher() i=0 2026-02-24T22:17:43.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.708+0000 7f6dc7e6ea80 20 add_watcher() i=6 2026-02-24T22:17:43.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.708+0000 7f6dc7e6ea80 20 add_watcher() i=5 2026-02-24T22:17:43.710 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.708+0000 7f6dc7e6ea80 20 add_watcher() i=7 2026-02-24T22:17:43.710 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.708+0000 7f6dc7e6ea80 20 add_watcher() i=4 2026-02-24T22:17:43.710 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.712+0000 7f6dc7e6ea80 20 add_watcher() i=2 2026-02-24T22:17:43.710 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.712+0000 7f6dc7e6ea80 2 all 8 watchers are set, enabling cache 2026-02-24T22:17:43.710 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.712+0000 7f6dc5141640 0 -- 10.20.193.46:0/4179799581 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f6ce4008af0 msgr2=0x7f6ce4028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:17:43.711 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.712+0000 7f6dc7e6ea80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:17:43.711 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.712+0000 7f6dc7e6ea80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:17:43.712 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.712+0000 7f6dc7e6ea80 5 note: GC not initialized 2026-02-24T22:17:43.712 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.712+0000 7f6cf67fc640 20 reqs_thread_entry: start 2026-02-24T22:17:43.742 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.744+0000 7f6dc7e6ea80 20 init_complete bucket index max shards: 1 2026-02-24T22:17:43.743 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.744+0000 7f6dc7e6ea80 20 Filter name: none 2026-02-24T22:17:43.743 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.744+0000 7f6cf4ff9640 20 reqs_thread_entry: start 2026-02-24T22:17:43.743 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.744+0000 7f6dc7e6ea80 10 cache get: name=default.rgw.meta+root+abort-at-trim-reshard-log-entries : miss 2026-02-24T22:17:43.743 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.744+0000 7f6dc7e6ea80 20 rados->read ofs=0 len=0 2026-02-24T22:17:43.744 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.744+0000 7f6dc7e6ea80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:17:43.744 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.744+0000 7f6dc7e6ea80 10 cache put: name=default.rgw.meta+root+abort-at-trim-reshard-log-entries info.flags=0x11 2026-02-24T22:17:43.744 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.744+0000 7f6dc7e6ea80 10 adding default.rgw.meta+root+abort-at-trim-reshard-log-entries to cache LRU end 2026-02-24T22:17:43.744 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.744+0000 7f6dc7e6ea80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 : miss 2026-02-24T22:17:43.744 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.744+0000 7f6dc7e6ea80 20 rados->read ofs=0 len=0 2026-02-24T22:17:43.744 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.744+0000 7f6dc7e6ea80 20 rados_obj.operate() r=0 bl.length=495 2026-02-24T22:17:43.744 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.744+0000 7f6dc7e6ea80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 info.flags=0x17 2026-02-24T22:17:43.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.744+0000 7f6dc7e6ea80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 to cache LRU end 2026-02-24T22:17:43.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.744+0000 7f6dc7e6ea80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:17:43.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.744+0000 7f6dc7e6ea80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:17:43.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.744+0000 7f6dc7e6ea80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 2026-02-24T22:17:43.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.744+0000 7f6dc7e6ea80 0 INFO: RGWReshardLock::lock found lock on abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 to be held by another RGW process; skipping for now 2026-02-24T22:17:43.747 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.748+0000 7f6dc7e6ea80 20 remove_watcher() i=3 2026-02-24T22:17:43.747 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.748+0000 7f6dc7e6ea80 2 removed watcher, disabling cache 2026-02-24T22:17:43.747 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.748+0000 7f6dc7e6ea80 20 remove_watcher() i=5 2026-02-24T22:17:43.747 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.748+0000 7f6dc7e6ea80 20 remove_watcher() i=0 2026-02-24T22:17:43.747 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.748+0000 7f6dc7e6ea80 20 remove_watcher() i=2 2026-02-24T22:17:43.748 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.748+0000 7f6dc7e6ea80 20 remove_watcher() i=1 2026-02-24T22:17:43.748 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.748+0000 7f6dc7e6ea80 20 remove_watcher() i=6 2026-02-24T22:17:43.748 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.748+0000 7f6dc7e6ea80 20 remove_watcher() i=4 2026-02-24T22:17:43.748 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:17:43.748+0000 7f6dc7e6ea80 20 remove_watcher() i=7 2026-02-24T22:17:43.752 INFO:tasks.workunit.client.0.trial046.stderr:waiting 30 seconds for reshard lock to expire... 2026-02-24T22:18:13.766 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket abort-at-trim-reshard-log-entries --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2026-02-24T22:18:13.783 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:18:13.783 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:18:13.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.803+0000 7f9e2415ca80 4 Realm: () 2026-02-24T22:18:13.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.803+0000 7f9e2415ca80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:13.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.803+0000 7f9e2415ca80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:13.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.803+0000 7f9e2415ca80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:13.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.803+0000 7f9e2415ca80 20 zonegroup default 2026-02-24T22:18:13.801 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.803+0000 7f9e2415ca80 20 rados->read ofs=0 len=0 2026-02-24T22:18:13.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.807+0000 7f9e2415ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:13.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.807+0000 7f9e2415ca80 20 rados->read ofs=0 len=0 2026-02-24T22:18:13.803 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.807+0000 7f9e2415ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:13.803 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.807+0000 7f9e2415ca80 20 started sync module instance, tier type = 2026-02-24T22:18:13.803 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.807+0000 7f9e2415ca80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:13.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.807+0000 7f9e2415ca80 20 add_watcher() i=1 2026-02-24T22:18:13.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.807+0000 7f9e22431640 0 -- 10.20.193.46:0/2972897565 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x555f6f599250 msgr2=0x555f6f5b9630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:13.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.807+0000 7f9e2415ca80 20 add_watcher() i=3 2026-02-24T22:18:13.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.807+0000 7f9e2415ca80 20 add_watcher() i=5 2026-02-24T22:18:13.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.807+0000 7f9e2415ca80 20 add_watcher() i=6 2026-02-24T22:18:13.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.807+0000 7f9e2415ca80 20 add_watcher() i=0 2026-02-24T22:18:13.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.811+0000 7f9e2415ca80 20 add_watcher() i=4 2026-02-24T22:18:13.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.811+0000 7f9e2415ca80 20 add_watcher() i=7 2026-02-24T22:18:13.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.811+0000 7f9e2415ca80 20 add_watcher() i=2 2026-02-24T22:18:13.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.811+0000 7f9e2415ca80 2 all 8 watchers are set, enabling cache 2026-02-24T22:18:13.807 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.811+0000 7f9e2415ca80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:13.807 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.811+0000 7f9e2415ca80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:13.807 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.811+0000 7f9e2415ca80 5 note: GC not initialized 2026-02-24T22:18:13.807 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.811+0000 7f9d527fc640 20 reqs_thread_entry: start 2026-02-24T22:18:13.808 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.811+0000 7f9e22431640 0 -- 10.20.193.46:0/1333768213 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x555f6f68e9d0 msgr2=0x555f6f68eda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:13.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.847+0000 7f9e2415ca80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:13.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.847+0000 7f9e2415ca80 20 Filter name: none 2026-02-24T22:18:13.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.847+0000 7f9d50ff9640 20 reqs_thread_entry: start 2026-02-24T22:18:13.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.847+0000 7f9e2415ca80 10 cache get: name=default.rgw.meta+root+abort-at-trim-reshard-log-entries : miss 2026-02-24T22:18:13.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:13.847+0000 7f9e2415ca80 20 rados->read ofs=0 len=0 2026-02-24T22:18:14.046 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.051+0000 7f9e2415ca80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:18:14.047 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.051+0000 7f9e2415ca80 10 cache put: name=default.rgw.meta+root+abort-at-trim-reshard-log-entries info.flags=0x11 2026-02-24T22:18:14.047 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.051+0000 7f9e2415ca80 10 adding default.rgw.meta+root+abort-at-trim-reshard-log-entries to cache LRU end 2026-02-24T22:18:14.047 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.051+0000 7f9e2415ca80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 : miss 2026-02-24T22:18:14.047 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.051+0000 7f9e2415ca80 20 rados->read ofs=0 len=0 2026-02-24T22:18:14.047 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.051+0000 7f9e2415ca80 20 rados_obj.operate() r=0 bl.length=495 2026-02-24T22:18:14.047 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.051+0000 7f9e2415ca80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 info.flags=0x17 2026-02-24T22:18:14.047 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.051+0000 7f9e2415ca80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 to cache LRU end 2026-02-24T22:18:14.047 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.051+0000 7f9e2415ca80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:18:14.047 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.051+0000 7f9e2415ca80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:14.047 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.051+0000 7f9e2415ca80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 2026-02-24T22:18:14.048 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.051+0000 7f9e2415ca80 10 init_target_layout removing existing target index objects from a previous reshard attempt 2026-02-24T22:18:14.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.055+0000 7f9e2415ca80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 info.flags=0x17 2026-02-24T22:18:14.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.055+0000 7f9e2415ca80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 to cache LRU end 2026-02-24T22:18:14.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.055+0000 7f9e2415ca80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:14.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.055+0000 7f9e2415ca80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528, ofs0, ns] 2026-02-24T22:18:14.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.055+0000 7f9dfd7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961347 cookie 93868379472032 notifier 7212 bl.length()=1041 2026-02-24T22:18:14.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.055+0000 7f9dfd7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 info.flags=0x17 2026-02-24T22:18:14.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.055+0000 7f9dfd7fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 to cache LRU end 2026-02-24T22:18:14.054 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.055+0000 7f9dfd7fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:14.054 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.055+0000 7f9e2415ca80 20 handle_bi_update(): bucket=:abort-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528]): orig_sources= new_sources= 2026-02-24T22:18:14.054 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.055+0000 7f9e2415ca80 20 handle_bi_update(): bucket=:abort-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528]): potential sources added=[] removed=[] 2026-02-24T22:18:14.054 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.055+0000 7f9e2415ca80 20 handle_bi_update(): bucket=:abort-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528]): orig_dests= new_dests= 2026-02-24T22:18:14.054 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.055+0000 7f9e2415ca80 20 handle_bi_update(): bucket=:abort-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528]): potential dests added=[] removed=[] 2026-02-24T22:18:14.055 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.059+0000 7f9e2415ca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528.2.0 2026-02-24T22:18:14.055 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.059+0000 7f9e2415ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528.2.0 2026-02-24T22:18:14.055 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.059+0000 7f9e2415ca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528.2.1 2026-02-24T22:18:14.055 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.059+0000 7f9e2415ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528.2.1 2026-02-24T22:18:14.055 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.059+0000 7f9e2415ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528.0 2026-02-24T22:18:14.056 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.059+0000 7f9e2415ca80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 : hit (requested=0x17, cached=0x17) 2026-02-24T22:18:14.056 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.059+0000 7f9e2415ca80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 2026-02-24T22:18:14.058 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9e2415ca80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 info.flags=0x17 2026-02-24T22:18:14.058 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9e2415ca80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 to cache LRU end 2026-02-24T22:18:14.058 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9e2415ca80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:14.058 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9e2415ca80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528, ofs0, ns] 2026-02-24T22:18:14.058 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9dfd7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961348 cookie 93868379472032 notifier 7212 bl.length()=1041 2026-02-24T22:18:14.058 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9dfd7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 info.flags=0x17 2026-02-24T22:18:14.058 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9dfd7fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 to cache LRU end 2026-02-24T22:18:14.058 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9dfd7fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:14.058 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9e2415ca80 20 handle_bi_update(): bucket=:abort-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528]): orig_sources= new_sources= 2026-02-24T22:18:14.059 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9e2415ca80 20 handle_bi_update(): bucket=:abort-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528]): potential sources added=[] removed=[] 2026-02-24T22:18:14.059 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9e2415ca80 20 handle_bi_update(): bucket=:abort-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528]): orig_dests= new_dests= 2026-02-24T22:18:14.059 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9e2415ca80 20 handle_bi_update(): bucket=:abort-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528]): potential dests added=[] removed=[] 2026-02-24T22:18:14.059 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9e2415ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528.0 2026-02-24T22:18:14.059 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9e2415ca80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 : hit (requested=0x17, cached=0x17) 2026-02-24T22:18:14.059 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9e2415ca80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 2026-02-24T22:18:14.060 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9e2415ca80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 info.flags=0x17 2026-02-24T22:18:14.060 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9e2415ca80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 to cache LRU end 2026-02-24T22:18:14.060 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9e2415ca80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:14.060 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9e2415ca80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528, ofs0, ns] 2026-02-24T22:18:14.061 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9dfd7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961349 cookie 93868379472032 notifier 7212 bl.length()=1005 2026-02-24T22:18:14.061 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9dfd7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 info.flags=0x17 2026-02-24T22:18:14.061 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9dfd7fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-trim-reshard-log-entries:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 to cache LRU end 2026-02-24T22:18:14.061 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9dfd7fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:14.061 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9e2415ca80 20 handle_bi_update(): bucket=:abort-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528]): orig_sources= new_sources= 2026-02-24T22:18:14.061 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9e2415ca80 20 handle_bi_update(): bucket=:abort-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528]): potential sources added=[] removed=[] 2026-02-24T22:18:14.061 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9e2415ca80 20 handle_bi_update(): bucket=:abort-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528]): orig_dests= new_dests= 2026-02-24T22:18:14.061 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9e2415ca80 20 handle_bi_update(): bucket=:abort-at-trim-reshard-log-entries[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528]): potential dests added=[] removed=[] 2026-02-24T22:18:14.062 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.063+0000 7f9e2415ca80 1 execute INFO: reshard of bucket "abort-at-trim-reshard-log-entries" from 1 shards to 2 shards completed successfully 2026-02-24T22:18:14.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.067+0000 7f9e2415ca80 20 remove_watcher() i=3 2026-02-24T22:18:14.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.067+0000 7f9e2415ca80 2 removed watcher, disabling cache 2026-02-24T22:18:14.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.067+0000 7f9e2415ca80 20 remove_watcher() i=1 2026-02-24T22:18:14.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.067+0000 7f9e2415ca80 20 remove_watcher() i=6 2026-02-24T22:18:14.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.067+0000 7f9e2415ca80 20 remove_watcher() i=0 2026-02-24T22:18:14.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.067+0000 7f9e2415ca80 20 remove_watcher() i=7 2026-02-24T22:18:14.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.067+0000 7f9e2415ca80 20 remove_watcher() i=4 2026-02-24T22:18:14.065 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.067+0000 7f9e2415ca80 20 remove_watcher() i=5 2026-02-24T22:18:14.066 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.067+0000 7f9e2415ca80 20 remove_watcher() i=2 2026-02-24T22:18:14.071 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket abort-at-trim-reshard-log-entries 2026-02-24T22:18:14.086 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:18:14.086 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:18:14.102 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.107+0000 7fbe98c08a80 4 Realm: () 2026-02-24T22:18:14.102 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.107+0000 7fbe98c08a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:14.103 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.107+0000 7fbe98c08a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:14.103 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.107+0000 7fbe98c08a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:14.103 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.107+0000 7fbe98c08a80 20 zonegroup default 2026-02-24T22:18:14.103 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.107+0000 7fbe98c08a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:14.104 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.107+0000 7fbe98c08a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:14.104 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.107+0000 7fbe98c08a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:14.105 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.107+0000 7fbe98c08a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:14.105 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.107+0000 7fbe98c08a80 20 started sync module instance, tier type = 2026-02-24T22:18:14.105 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.107+0000 7fbe98c08a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:14.107 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.111+0000 7fbe98c08a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:14.107 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.111+0000 7fbe98c08a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:14.107 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.111+0000 7fbe98c08a80 5 note: GC not initialized 2026-02-24T22:18:14.107 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.111+0000 7fbdb7fff640 20 reqs_thread_entry: start 2026-02-24T22:18:14.112 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.115+0000 7fbe966dc640 0 -- 10.20.193.46:0/417524266 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55592b062b80 msgr2=0x55592b062ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:14.323 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.327+0000 7fbe96edd640 0 -- 10.20.193.46:0/417524266 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x55592b053b90 msgr2=0x55592b054000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:18:14.550 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.555+0000 7fbe98c08a80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:14.551 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.555+0000 7fbe98c08a80 20 Filter name: none 2026-02-24T22:18:14.551 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.555+0000 7fbdb67fc640 20 reqs_thread_entry: start 2026-02-24T22:18:14.551 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.555+0000 7fbe98c08a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:14.551 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.555+0000 7fbe98c08a80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:18:14.552 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.555+0000 7fbe98c08a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:14.552 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.555+0000 7fbe98c08a80 20 rados_obj.operate() r=0 bl.length=459 2026-02-24T22:18:14.552 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.555+0000 7fbe98c08a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:14.552 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.555+0000 7fbe98c08a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:14.553 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.555+0000 7fbe98c08a80 20 rados_obj.operate() r=0 bl.length=229 2026-02-24T22:18:14.553 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.555+0000 7fbe98c08a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:14.553 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.555+0000 7fbe98c08a80 20 rados_obj.operate() r=0 bl.length=459 2026-02-24T22:18:14.553 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.555+0000 7fbe98c08a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:14.564 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:14.564 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:18:14.564 INFO:tasks.workunit.client.0.trial046.stderr:bucket: abort-at-trim-reshard-log-entries id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528 num_objects: 19 size_kb: 1 num_shards: 2 2026-02-24T22:18:14.564 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:14.565 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:14.565 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:14.568 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:14.568 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:14.610 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:14.611 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:14.621 INFO:tasks.workunit.client.0.trial046.stderr:TEST: reshard bucket with EIO injected at block_writes 2026-02-24T22:18:14.621 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:14.622 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:14.622 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:14.631 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:14.631 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:14.679 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:14.679 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:14.684 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:14.685 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:14.690 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:14.691 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:14.696 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:14.696 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:14.702 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:14.702 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:14.707 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:14.707 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:14.713 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:14.713 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:14.720 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:14.720 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:14.726 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:14.726 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:14.733 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:14.733 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:14.739 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:14.739 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:14.745 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:14.745 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:14.750 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:14.750 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:14.757 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:14.757 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:14.762 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:14.762 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:14.768 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:14.769 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:14.774 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:14.775 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:14.781 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:14.781 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:14.787 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:14.787 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:14.792 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:14.792 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:14.798 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-block-writes 2026-02-24T22:18:14.813 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:18:14.814 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:18:14.830 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.835+0000 7ff44a56da80 4 Realm: () 2026-02-24T22:18:14.830 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.835+0000 7ff44a56da80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:14.830 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.835+0000 7ff44a56da80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:14.830 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.835+0000 7ff44a56da80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:14.830 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.835+0000 7ff44a56da80 20 zonegroup default 2026-02-24T22:18:14.831 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.835+0000 7ff44a56da80 20 rados->read ofs=0 len=0 2026-02-24T22:18:14.831 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.835+0000 7ff44a56da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:14.831 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.835+0000 7ff44a56da80 20 rados->read ofs=0 len=0 2026-02-24T22:18:14.832 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.835+0000 7ff44a56da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:14.832 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.835+0000 7ff44a56da80 20 started sync module instance, tier type = 2026-02-24T22:18:14.832 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.835+0000 7ff44a56da80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:14.833 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.835+0000 7ff44a56da80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:14.833 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.835+0000 7ff44a56da80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:14.833 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.839+0000 7ff44a56da80 5 note: GC not initialized 2026-02-24T22:18:14.834 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.839+0000 7ff3f67dc640 20 reqs_thread_entry: start 2026-02-24T22:18:14.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:14.843+0000 7ff443fff640 0 -- 10.20.193.46:0/3724853317 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55933611d100 msgr2=0x55933611d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:15.051 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.055+0000 7ff448842640 0 -- 10.20.193.46:0/3724853317 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x55933610e870 msgr2=0x55933610ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:15.278 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.283+0000 7ff44a56da80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:15.279 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.283+0000 7ff44a56da80 20 Filter name: none 2026-02-24T22:18:15.279 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.283+0000 7ff3f4fd9640 20 reqs_thread_entry: start 2026-02-24T22:18:15.279 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.283+0000 7ff44a56da80 20 rados->read ofs=0 len=0 2026-02-24T22:18:15.279 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.283+0000 7ff44a56da80 20 rados_obj.operate() r=0 bl.length=217 2026-02-24T22:18:15.280 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.283+0000 7ff44a56da80 20 rados->read ofs=0 len=0 2026-02-24T22:18:15.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.283+0000 7ff44a56da80 20 rados_obj.operate() r=0 bl.length=447 2026-02-24T22:18:15.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.283+0000 7ff44a56da80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:15.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.283+0000 7ff44a56da80 20 rados->read ofs=0 len=0 2026-02-24T22:18:15.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.283+0000 7ff44a56da80 20 rados_obj.operate() r=0 bl.length=217 2026-02-24T22:18:15.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.283+0000 7ff44a56da80 20 rados->read ofs=0 len=0 2026-02-24T22:18:15.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.283+0000 7ff44a56da80 20 rados_obj.operate() r=0 bl.length=447 2026-02-24T22:18:15.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.283+0000 7ff44a56da80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:15.292 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:15.292 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:18:15.292 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-block-writes id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:18:15.292 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:15.292 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket error-at-block-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at block_writes 2026-02-24T22:18:15.308 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:18:15.308 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:18:15.327 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.331+0000 7fe956839a80 4 Realm: () 2026-02-24T22:18:15.327 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.331+0000 7fe956839a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:15.327 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.331+0000 7fe956839a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:15.327 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.331+0000 7fe956839a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:15.327 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.331+0000 7fe956839a80 20 zonegroup default 2026-02-24T22:18:15.328 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.331+0000 7fe956839a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:15.328 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.331+0000 7fe956839a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:15.328 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.331+0000 7fe956839a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:15.329 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.331+0000 7fe956839a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:15.329 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.331+0000 7fe956839a80 20 started sync module instance, tier type = 2026-02-24T22:18:15.329 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.331+0000 7fe956839a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:15.330 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.335+0000 7fe94ffff640 0 -- 10.20.193.46:0/933250873 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x55f5802f83b0 msgr2=0x55f580318810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:18:15.331 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.335+0000 7fe956839a80 20 add_watcher() i=1 2026-02-24T22:18:15.331 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.335+0000 7fe956839a80 20 add_watcher() i=3 2026-02-24T22:18:15.331 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.335+0000 7fe956839a80 20 add_watcher() i=5 2026-02-24T22:18:15.331 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.335+0000 7fe956839a80 20 add_watcher() i=6 2026-02-24T22:18:15.332 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.335+0000 7fe956839a80 20 add_watcher() i=2 2026-02-24T22:18:15.332 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.335+0000 7fe956839a80 20 add_watcher() i=4 2026-02-24T22:18:15.332 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.335+0000 7fe956839a80 20 add_watcher() i=0 2026-02-24T22:18:15.332 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.335+0000 7fe956839a80 20 add_watcher() i=7 2026-02-24T22:18:15.332 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.335+0000 7fe956839a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:18:15.333 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.335+0000 7fe947fff640 0 -- 10.20.193.46:0/2153294722 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7fe874008af0 msgr2=0x7fe874028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:15.334 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.339+0000 7fe956839a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:15.334 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.339+0000 7fe956839a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:15.334 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.339+0000 7fe956839a80 5 note: GC not initialized 2026-02-24T22:18:15.334 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.339+0000 7fe901fdb640 20 reqs_thread_entry: start 2026-02-24T22:18:15.368 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.371+0000 7fe956839a80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:15.368 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.371+0000 7fe956839a80 20 Filter name: none 2026-02-24T22:18:15.368 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.371+0000 7fe85bfff640 20 reqs_thread_entry: start 2026-02-24T22:18:15.369 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.371+0000 7fe956839a80 10 cache get: name=default.rgw.meta+root+error-at-block-writes : miss 2026-02-24T22:18:15.369 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.371+0000 7fe956839a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:15.369 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.371+0000 7fe956839a80 20 rados_obj.operate() r=0 bl.length=217 2026-02-24T22:18:15.369 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.371+0000 7fe956839a80 10 cache put: name=default.rgw.meta+root+error-at-block-writes info.flags=0x11 2026-02-24T22:18:15.369 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.371+0000 7fe956839a80 10 adding default.rgw.meta+root+error-at-block-writes to cache LRU end 2026-02-24T22:18:15.369 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.371+0000 7fe956839a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 : miss 2026-02-24T22:18:15.369 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.371+0000 7fe956839a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:15.370 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.371+0000 7fe956839a80 20 rados_obj.operate() r=0 bl.length=447 2026-02-24T22:18:15.370 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.371+0000 7fe956839a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 info.flags=0x17 2026-02-24T22:18:15.370 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.371+0000 7fe956839a80 10 adding default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 to cache LRU end 2026-02-24T22:18:15.370 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.371+0000 7fe956839a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:18:15.370 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.371+0000 7fe956839a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:15.370 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.371+0000 7fe956839a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 2026-02-24T22:18:15.373 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.379+0000 7fe956839a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 info.flags=0x17 2026-02-24T22:18:15.374 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.379+0000 7fe956839a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 to cache LRU end 2026-02-24T22:18:15.374 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.379+0000 7fe956839a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:15.374 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.379+0000 7fe956839a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529, ofs0, ns] 2026-02-24T22:18:15.374 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.379+0000 7fe9237fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554960019 cookie 94512906081024 notifier 7263 bl.length()=1017 2026-02-24T22:18:15.375 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.379+0000 7fe9237fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 info.flags=0x17 2026-02-24T22:18:15.375 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.379+0000 7fe9237fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 to cache LRU end 2026-02-24T22:18:15.375 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.379+0000 7fe9237fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:15.375 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.379+0000 7fe956839a80 20 handle_bi_update(): bucket=:error-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529]): orig_sources= new_sources= 2026-02-24T22:18:15.375 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.379+0000 7fe956839a80 20 handle_bi_update(): bucket=:error-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529]): potential sources added=[] removed=[] 2026-02-24T22:18:15.375 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.379+0000 7fe956839a80 20 handle_bi_update(): bucket=:error-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529]): orig_dests= new_dests= 2026-02-24T22:18:15.375 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.379+0000 7fe956839a80 20 handle_bi_update(): bucket=:error-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529]): potential dests added=[] removed=[] 2026-02-24T22:18:15.376 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.379+0000 7fe956839a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529.1.0 2026-02-24T22:18:15.376 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.379+0000 7fe956839a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529.1.0 2026-02-24T22:18:15.376 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.379+0000 7fe956839a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529.1.1 2026-02-24T22:18:15.376 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.379+0000 7fe956839a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529.1.1 2026-02-24T22:18:15.376 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.379+0000 7fe956839a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529.0 2026-02-24T22:18:15.377 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.383+0000 7fe956839a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 : hit (requested=0x17, cached=0x17) 2026-02-24T22:18:15.377 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.383+0000 7fe956839a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 2026-02-24T22:18:15.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.383+0000 7fe956839a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 info.flags=0x17 2026-02-24T22:18:15.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.383+0000 7fe956839a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 to cache LRU end 2026-02-24T22:18:15.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.383+0000 7fe956839a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:15.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.383+0000 7fe956839a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529, ofs0, ns] 2026-02-24T22:18:15.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.383+0000 7fe922ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554960020 cookie 94512906081024 notifier 7263 bl.length()=1017 2026-02-24T22:18:15.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.383+0000 7fe922ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 info.flags=0x17 2026-02-24T22:18:15.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.383+0000 7fe922ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 to cache LRU end 2026-02-24T22:18:15.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.383+0000 7fe922ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:15.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.383+0000 7fe956839a80 20 handle_bi_update(): bucket=:error-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529]): orig_sources= new_sources= 2026-02-24T22:18:15.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.383+0000 7fe956839a80 20 handle_bi_update(): bucket=:error-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529]): potential sources added=[] removed=[] 2026-02-24T22:18:15.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.383+0000 7fe956839a80 20 handle_bi_update(): bucket=:error-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529]): orig_dests= new_dests= 2026-02-24T22:18:15.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.383+0000 7fe956839a80 20 handle_bi_update(): bucket=:error-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529]): potential dests added=[] removed=[] 2026-02-24T22:18:15.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.383+0000 7fe956839a80 -1 Injecting error=-5 at location=block_writes 2026-02-24T22:18:15.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.383+0000 7fe956839a80 0 ERROR: change_reshard_state failed to pause writes to the current index: (5) Input/output error 2026-02-24T22:18:15.381 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.383+0000 7fe956839a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 : hit (requested=0x17, cached=0x17) 2026-02-24T22:18:15.381 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.383+0000 7fe956839a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 2026-02-24T22:18:15.382 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.387+0000 7fe956839a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 info.flags=0x17 2026-02-24T22:18:15.382 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.387+0000 7fe956839a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 to cache LRU end 2026-02-24T22:18:15.382 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.387+0000 7fe956839a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:15.382 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.387+0000 7fe956839a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529, ofs0, ns] 2026-02-24T22:18:15.382 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.387+0000 7fe922ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554960021 cookie 94512906081024 notifier 7263 bl.length()=981 2026-02-24T22:18:15.382 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.387+0000 7fe922ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 info.flags=0x17 2026-02-24T22:18:15.382 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.387+0000 7fe922ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 to cache LRU end 2026-02-24T22:18:15.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.387+0000 7fe922ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:15.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.387+0000 7fe956839a80 20 handle_bi_update(): bucket=:error-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529]): orig_sources= new_sources= 2026-02-24T22:18:15.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.387+0000 7fe956839a80 20 handle_bi_update(): bucket=:error-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529]): potential sources added=[] removed=[] 2026-02-24T22:18:15.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.387+0000 7fe956839a80 20 handle_bi_update(): bucket=:error-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529]): orig_dests= new_dests= 2026-02-24T22:18:15.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.387+0000 7fe956839a80 20 handle_bi_update(): bucket=:error-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529]): potential dests added=[] removed=[] 2026-02-24T22:18:15.384 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.387+0000 7fe956839a80 1 execute INFO: reshard of bucket "error-at-block-writes" canceled due to errors 2026-02-24T22:18:15.387 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.391+0000 7fe956839a80 20 remove_watcher() i=3 2026-02-24T22:18:15.387 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.391+0000 7fe956839a80 2 removed watcher, disabling cache 2026-02-24T22:18:15.387 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.391+0000 7fe956839a80 20 remove_watcher() i=2 2026-02-24T22:18:15.387 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.391+0000 7fe956839a80 20 remove_watcher() i=4 2026-02-24T22:18:15.387 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.391+0000 7fe956839a80 20 remove_watcher() i=1 2026-02-24T22:18:15.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.391+0000 7fe956839a80 20 remove_watcher() i=6 2026-02-24T22:18:15.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.391+0000 7fe956839a80 20 remove_watcher() i=7 2026-02-24T22:18:15.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.391+0000 7fe956839a80 20 remove_watcher() i=0 2026-02-24T22:18:15.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.391+0000 7fe956839a80 20 remove_watcher() i=5 2026-02-24T22:18:15.393 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-block-writes 2026-02-24T22:18:15.407 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:18:15.407 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:18:15.423 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.427+0000 7ff3f7db9a80 4 Realm: () 2026-02-24T22:18:15.423 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.427+0000 7ff3f7db9a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:15.423 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.427+0000 7ff3f7db9a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:15.423 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.427+0000 7ff3f7db9a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:15.423 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.427+0000 7ff3f7db9a80 20 zonegroup default 2026-02-24T22:18:15.424 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.427+0000 7ff3f7db9a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:15.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.427+0000 7ff3f7db9a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:15.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.427+0000 7ff3f7db9a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:15.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.431+0000 7ff3f7db9a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:15.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.431+0000 7ff3f7db9a80 20 started sync module instance, tier type = 2026-02-24T22:18:15.425 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.431+0000 7ff3f7db9a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:15.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.431+0000 7ff3f7db9a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:15.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.431+0000 7ff3f7db9a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:15.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.431+0000 7ff3f7db9a80 5 note: GC not initialized 2026-02-24T22:18:15.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.431+0000 7ff316ffd640 20 reqs_thread_entry: start 2026-02-24T22:18:15.431 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.435+0000 7ff3f608e640 0 -- 10.20.193.46:0/1443252083 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x5581ebdea800 msgr2=0x5581ebdf0f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:18:15.436 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.439+0000 7ff3f588d640 0 -- 10.20.193.46:0/1443252083 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5581ebdbede0 msgr2=0x5581ebdec9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:15.459 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.463+0000 7ff3f7db9a80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:15.459 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.463+0000 7ff3f7db9a80 20 Filter name: none 2026-02-24T22:18:15.459 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.463+0000 7ff3157fa640 20 reqs_thread_entry: start 2026-02-24T22:18:15.459 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.463+0000 7ff3f7db9a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:15.460 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.463+0000 7ff3f7db9a80 20 rados_obj.operate() r=0 bl.length=217 2026-02-24T22:18:15.460 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.463+0000 7ff3f7db9a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:15.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.463+0000 7ff3f7db9a80 20 rados_obj.operate() r=0 bl.length=447 2026-02-24T22:18:15.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.463+0000 7ff3f7db9a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:15.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.463+0000 7ff3f7db9a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:15.462 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.463+0000 7ff3f7db9a80 20 rados_obj.operate() r=0 bl.length=217 2026-02-24T22:18:15.462 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.463+0000 7ff3f7db9a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:15.462 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.467+0000 7ff3f7db9a80 20 rados_obj.operate() r=0 bl.length=447 2026-02-24T22:18:15.462 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.467+0000 7ff3f7db9a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:15.472 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:15.472 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:18:15.472 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-block-writes id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:18:15.472 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:15.473 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:15.473 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:15.478 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:15.479 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:15.522 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket error-at-block-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2026-02-24T22:18:15.537 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:18:15.538 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:18:15.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.559+0000 7f323230ba80 4 Realm: () 2026-02-24T22:18:15.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.559+0000 7f323230ba80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:15.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.559+0000 7f323230ba80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:15.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.559+0000 7f323230ba80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:15.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.559+0000 7f323230ba80 20 zonegroup default 2026-02-24T22:18:15.555 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.559+0000 7f323230ba80 20 rados->read ofs=0 len=0 2026-02-24T22:18:15.555 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.559+0000 7f323230ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:15.556 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.559+0000 7f323230ba80 20 rados->read ofs=0 len=0 2026-02-24T22:18:15.556 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.559+0000 7f323230ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:15.556 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.559+0000 7f323230ba80 20 started sync module instance, tier type = 2026-02-24T22:18:15.556 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.559+0000 7f323230ba80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:15.557 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.559+0000 7f322b7fe640 0 -- 10.20.193.46:0/1724834028 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x55aef7de49f0 msgr2=0x55aef7e04e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:15.558 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.563+0000 7f323230ba80 20 add_watcher() i=1 2026-02-24T22:18:15.558 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.563+0000 7f323230ba80 20 add_watcher() i=0 2026-02-24T22:18:15.558 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.563+0000 7f323230ba80 20 add_watcher() i=3 2026-02-24T22:18:15.558 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.563+0000 7f323230ba80 20 add_watcher() i=6 2026-02-24T22:18:15.558 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.563+0000 7f323230ba80 20 add_watcher() i=5 2026-02-24T22:18:15.559 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.563+0000 7f323230ba80 20 add_watcher() i=2 2026-02-24T22:18:15.559 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.563+0000 7f323230ba80 20 add_watcher() i=4 2026-02-24T22:18:15.559 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.563+0000 7f323230ba80 20 add_watcher() i=7 2026-02-24T22:18:15.559 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.563+0000 7f323230ba80 2 all 8 watchers are set, enabling cache 2026-02-24T22:18:15.560 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.563+0000 7f323230ba80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:15.561 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.563+0000 7f323230ba80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:15.561 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.563+0000 7f323230ba80 5 note: GC not initialized 2026-02-24T22:18:15.561 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.563+0000 7f31dcfe1640 20 reqs_thread_entry: start 2026-02-24T22:18:15.561 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.567+0000 7f322bfff640 0 -- 10.20.193.46:0/815993488 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55aef7d0ee40 msgr2=0x55aef7d06630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:15.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.807+0000 7f323230ba80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:15.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.807+0000 7f323230ba80 20 Filter name: none 2026-02-24T22:18:15.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.807+0000 7f3146ffd640 20 reqs_thread_entry: start 2026-02-24T22:18:15.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.807+0000 7f323230ba80 10 cache get: name=default.rgw.meta+root+error-at-block-writes : miss 2026-02-24T22:18:15.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.807+0000 7f323230ba80 20 rados->read ofs=0 len=0 2026-02-24T22:18:15.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.807+0000 7f323230ba80 20 rados_obj.operate() r=0 bl.length=217 2026-02-24T22:18:15.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.807+0000 7f323230ba80 10 cache put: name=default.rgw.meta+root+error-at-block-writes info.flags=0x11 2026-02-24T22:18:15.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.807+0000 7f323230ba80 10 adding default.rgw.meta+root+error-at-block-writes to cache LRU end 2026-02-24T22:18:15.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.807+0000 7f323230ba80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 : miss 2026-02-24T22:18:15.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.807+0000 7f323230ba80 20 rados->read ofs=0 len=0 2026-02-24T22:18:15.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.807+0000 7f323230ba80 20 rados_obj.operate() r=0 bl.length=447 2026-02-24T22:18:15.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.807+0000 7f323230ba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 info.flags=0x17 2026-02-24T22:18:15.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.807+0000 7f323230ba80 10 adding default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 to cache LRU end 2026-02-24T22:18:15.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.807+0000 7f323230ba80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:18:15.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.807+0000 7f323230ba80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:15.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.807+0000 7f323230ba80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 2026-02-24T22:18:15.809 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.811+0000 7f323230ba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 info.flags=0x17 2026-02-24T22:18:15.809 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.811+0000 7f323230ba80 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 to cache LRU end 2026-02-24T22:18:15.809 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.811+0000 7f323230ba80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:15.809 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.811+0000 7f323230ba80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529, ofs0, ns] 2026-02-24T22:18:15.809 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.811+0000 7f32067fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554960022 cookie 94209970531440 notifier 7299 bl.length()=1017 2026-02-24T22:18:15.809 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.811+0000 7f32067fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 info.flags=0x17 2026-02-24T22:18:15.810 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.811+0000 7f32067fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 to cache LRU end 2026-02-24T22:18:15.810 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.811+0000 7f32067fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:15.810 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.815+0000 7f323230ba80 20 handle_bi_update(): bucket=:error-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529]): orig_sources= new_sources= 2026-02-24T22:18:15.810 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.815+0000 7f323230ba80 20 handle_bi_update(): bucket=:error-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529]): potential sources added=[] removed=[] 2026-02-24T22:18:15.810 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.815+0000 7f323230ba80 20 handle_bi_update(): bucket=:error-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529]): orig_dests= new_dests= 2026-02-24T22:18:15.811 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.815+0000 7f323230ba80 20 handle_bi_update(): bucket=:error-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529]): potential dests added=[] removed=[] 2026-02-24T22:18:15.811 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.815+0000 7f323230ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529.1.0 2026-02-24T22:18:15.811 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.815+0000 7f323230ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529.1.0 2026-02-24T22:18:15.811 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.815+0000 7f323230ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529.1.1 2026-02-24T22:18:15.811 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.815+0000 7f323230ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529.1.1 2026-02-24T22:18:15.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.815+0000 7f323230ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529.0 2026-02-24T22:18:15.813 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.815+0000 7f323230ba80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 : hit (requested=0x17, cached=0x17) 2026-02-24T22:18:15.813 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.815+0000 7f323230ba80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 2026-02-24T22:18:15.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.819+0000 7f323230ba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 info.flags=0x17 2026-02-24T22:18:15.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.819+0000 7f323230ba80 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 to cache LRU end 2026-02-24T22:18:15.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.819+0000 7f323230ba80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:15.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.819+0000 7f323230ba80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529, ofs0, ns] 2026-02-24T22:18:15.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.819+0000 7f32067fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554960023 cookie 94209970531440 notifier 7299 bl.length()=1017 2026-02-24T22:18:15.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.819+0000 7f32067fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 info.flags=0x17 2026-02-24T22:18:15.815 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.819+0000 7f32067fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 to cache LRU end 2026-02-24T22:18:15.815 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.819+0000 7f32067fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:15.815 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.819+0000 7f323230ba80 20 handle_bi_update(): bucket=:error-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529]): orig_sources= new_sources= 2026-02-24T22:18:15.815 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.819+0000 7f323230ba80 20 handle_bi_update(): bucket=:error-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529]): potential sources added=[] removed=[] 2026-02-24T22:18:15.815 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.819+0000 7f323230ba80 20 handle_bi_update(): bucket=:error-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529]): orig_dests= new_dests= 2026-02-24T22:18:15.815 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.819+0000 7f323230ba80 20 handle_bi_update(): bucket=:error-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529]): potential dests added=[] removed=[] 2026-02-24T22:18:15.816 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.819+0000 7f323230ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529.0 2026-02-24T22:18:15.816 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.819+0000 7f323230ba80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 : hit (requested=0x17, cached=0x17) 2026-02-24T22:18:15.816 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.819+0000 7f323230ba80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 2026-02-24T22:18:15.817 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.819+0000 7f323230ba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 info.flags=0x17 2026-02-24T22:18:15.817 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.819+0000 7f323230ba80 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 to cache LRU end 2026-02-24T22:18:15.817 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.819+0000 7f323230ba80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:15.817 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.819+0000 7f323230ba80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529, ofs0, ns] 2026-02-24T22:18:15.817 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.819+0000 7f32067fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554960024 cookie 94209970531440 notifier 7299 bl.length()=981 2026-02-24T22:18:15.817 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.819+0000 7f32067fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 info.flags=0x17 2026-02-24T22:18:15.817 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.819+0000 7f32067fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 to cache LRU end 2026-02-24T22:18:15.817 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.819+0000 7f32067fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:15.818 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.823+0000 7f323230ba80 20 handle_bi_update(): bucket=:error-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529]): orig_sources= new_sources= 2026-02-24T22:18:15.818 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.823+0000 7f323230ba80 20 handle_bi_update(): bucket=:error-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529]): potential sources added=[] removed=[] 2026-02-24T22:18:15.818 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.823+0000 7f323230ba80 20 handle_bi_update(): bucket=:error-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529]): orig_dests= new_dests= 2026-02-24T22:18:15.818 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.823+0000 7f323230ba80 20 handle_bi_update(): bucket=:error-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529]): potential dests added=[] removed=[] 2026-02-24T22:18:15.818 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.823+0000 7f323230ba80 1 execute INFO: reshard of bucket "error-at-block-writes" from 1 shards to 2 shards completed successfully 2026-02-24T22:18:15.822 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.827+0000 7f323230ba80 20 remove_watcher() i=3 2026-02-24T22:18:15.822 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.827+0000 7f323230ba80 2 removed watcher, disabling cache 2026-02-24T22:18:15.822 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.827+0000 7f323230ba80 20 remove_watcher() i=0 2026-02-24T22:18:15.822 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.827+0000 7f323230ba80 20 remove_watcher() i=4 2026-02-24T22:18:15.822 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.827+0000 7f323230ba80 20 remove_watcher() i=7 2026-02-24T22:18:15.822 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.827+0000 7f323230ba80 20 remove_watcher() i=1 2026-02-24T22:18:15.823 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.827+0000 7f323230ba80 20 remove_watcher() i=2 2026-02-24T22:18:15.823 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.827+0000 7f323230ba80 20 remove_watcher() i=6 2026-02-24T22:18:15.823 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.827+0000 7f323230ba80 20 remove_watcher() i=5 2026-02-24T22:18:15.829 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-block-writes 2026-02-24T22:18:15.843 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:18:15.843 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:18:15.859 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.863+0000 7f14d701da80 4 Realm: () 2026-02-24T22:18:15.859 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.863+0000 7f14d701da80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:15.859 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.863+0000 7f14d701da80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:15.859 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.863+0000 7f14d701da80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:15.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.863+0000 7f14d701da80 20 zonegroup default 2026-02-24T22:18:15.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.863+0000 7f14d701da80 20 rados->read ofs=0 len=0 2026-02-24T22:18:15.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.863+0000 7f14d701da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:15.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.867+0000 7f14d701da80 20 rados->read ofs=0 len=0 2026-02-24T22:18:15.862 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.867+0000 7f14d701da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:15.862 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.867+0000 7f14d701da80 20 started sync module instance, tier type = 2026-02-24T22:18:15.862 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.867+0000 7f14d701da80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:15.863 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.867+0000 7f14d701da80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:15.863 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.867+0000 7f14d701da80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:15.863 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.867+0000 7f14d701da80 5 note: GC not initialized 2026-02-24T22:18:15.863 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.867+0000 7f14827e4640 20 reqs_thread_entry: start 2026-02-24T22:18:15.866 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.871+0000 7f14cffff640 0 -- 10.20.193.46:0/4127387187 >> [v2:10.20.193.46:6826/3065971976,v1:10.20.193.46:6827/3065971976] conn(0x5559f3100710 msgr2=0x5559f30f7b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:15.870 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:15.875+0000 7f14cffff640 0 -- 10.20.193.46:0/4127387187 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x5559f30f6070 msgr2=0x5559f30f64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:16.103 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.107+0000 7f14d701da80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:16.103 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.107+0000 7f14d701da80 20 Filter name: none 2026-02-24T22:18:16.103 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.107+0000 7f1480fe1640 20 reqs_thread_entry: start 2026-02-24T22:18:16.103 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.107+0000 7f14d701da80 20 rados->read ofs=0 len=0 2026-02-24T22:18:16.104 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.107+0000 7f14d701da80 20 rados_obj.operate() r=0 bl.length=217 2026-02-24T22:18:16.104 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.107+0000 7f14d701da80 20 rados->read ofs=0 len=0 2026-02-24T22:18:16.105 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.107+0000 7f14d701da80 20 rados_obj.operate() r=0 bl.length=447 2026-02-24T22:18:16.105 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.107+0000 7f14d701da80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:16.105 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.107+0000 7f14d701da80 20 rados->read ofs=0 len=0 2026-02-24T22:18:16.105 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.107+0000 7f14d701da80 20 rados_obj.operate() r=0 bl.length=217 2026-02-24T22:18:16.105 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.107+0000 7f14d701da80 20 rados->read ofs=0 len=0 2026-02-24T22:18:16.105 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.107+0000 7f14d701da80 20 rados_obj.operate() r=0 bl.length=447 2026-02-24T22:18:16.105 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.107+0000 7f14d701da80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:16.116 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.116 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:18:16.116 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-block-writes id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529 num_objects: 19 size_kb: 1 num_shards: 2 2026-02-24T22:18:16.117 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.118 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:16.118 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:16.120 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:16.120 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:16.162 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:16.162 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:16.173 INFO:tasks.workunit.client.0.trial046.stderr:TEST: reshard bucket with abort at block_writes 2026-02-24T22:18:16.173 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.175 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:16.175 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:16.183 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:16.183 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:16.226 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:16.226 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:16.233 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:16.233 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:16.239 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:16.239 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:16.245 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:16.245 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:16.251 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:16.251 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:16.257 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:16.257 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:16.262 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:16.263 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:16.269 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:16.269 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:16.275 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:16.275 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:16.281 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:16.281 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:16.287 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:16.287 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:16.293 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:16.293 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:16.300 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:16.300 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:16.305 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:16.306 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:16.312 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:16.312 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:16.318 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:16.318 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:16.324 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:16.324 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:16.330 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:16.330 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:16.336 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:16.336 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:16.341 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:16.342 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:16.347 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket abort-at-block-writes 2026-02-24T22:18:16.362 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:18:16.362 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:18:16.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.383+0000 7f04b3b78a80 4 Realm: () 2026-02-24T22:18:16.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.383+0000 7f04b3b78a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:16.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.383+0000 7f04b3b78a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:16.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.383+0000 7f04b3b78a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:16.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.383+0000 7f04b3b78a80 20 zonegroup default 2026-02-24T22:18:16.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.383+0000 7f04b3b78a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:16.381 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.383+0000 7f04b3b78a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:16.381 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.383+0000 7f04b3b78a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:16.381 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.387+0000 7f04b3b78a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:16.381 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.387+0000 7f04b3b78a80 20 started sync module instance, tier type = 2026-02-24T22:18:16.382 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.387+0000 7f04b3b78a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:16.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.387+0000 7f04b3b78a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:16.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.387+0000 7f04b3b78a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:16.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.387+0000 7f04b3b78a80 5 note: GC not initialized 2026-02-24T22:18:16.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.387+0000 7f045efe5640 20 reqs_thread_entry: start 2026-02-24T22:18:16.386 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.391+0000 7f04b1e4d640 0 -- 10.20.193.46:0/2665277239 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x564710403270 msgr2=0x5647104099c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:16.597 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.599+0000 7f04b0e4b640 0 -- 10.20.193.46:0/2665277239 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x5647103ebd20 msgr2=0x5647103ec1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:16.624 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.627+0000 7f04b3b78a80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:16.624 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.627+0000 7f04b3b78a80 20 Filter name: none 2026-02-24T22:18:16.624 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.627+0000 7f045d7e2640 20 reqs_thread_entry: start 2026-02-24T22:18:16.624 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.627+0000 7f04b3b78a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:16.625 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.627+0000 7f04b3b78a80 20 rados_obj.operate() r=0 bl.length=217 2026-02-24T22:18:16.625 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.627+0000 7f04b3b78a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:16.625 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.627+0000 7f04b3b78a80 20 rados_obj.operate() r=0 bl.length=447 2026-02-24T22:18:16.625 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.627+0000 7f04b3b78a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:16.625 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.627+0000 7f04b3b78a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:16.626 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.627+0000 7f04b3b78a80 20 rados_obj.operate() r=0 bl.length=217 2026-02-24T22:18:16.626 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.627+0000 7f04b3b78a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:16.626 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.627+0000 7f04b3b78a80 20 rados_obj.operate() r=0 bl.length=447 2026-02-24T22:18:16.626 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.627+0000 7f04b3b78a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:16.636 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.636 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:18:16.636 INFO:tasks.workunit.client.0.trial046.stderr:bucket: abort-at-block-writes id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:18:16.636 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.637 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket abort-at-block-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at block_writes 2026-02-24T22:18:16.652 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:18:16.652 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:18:16.670 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.675+0000 7f961646aa80 4 Realm: () 2026-02-24T22:18:16.670 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.675+0000 7f961646aa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:16.670 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.675+0000 7f961646aa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:16.670 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.675+0000 7f961646aa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:16.670 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.675+0000 7f961646aa80 20 zonegroup default 2026-02-24T22:18:16.670 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.675+0000 7f961646aa80 20 rados->read ofs=0 len=0 2026-02-24T22:18:16.671 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.675+0000 7f961646aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:16.671 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.675+0000 7f961646aa80 20 rados->read ofs=0 len=0 2026-02-24T22:18:16.672 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.675+0000 7f961646aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:16.672 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.675+0000 7f961646aa80 20 started sync module instance, tier type = 2026-02-24T22:18:16.672 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.675+0000 7f961646aa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:16.673 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.675+0000 7f960ffff640 0 -- 10.20.193.46:0/3578363307 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x558d22f8b4d0 msgr2=0x558d22fab8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:18:16.674 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.679+0000 7f961646aa80 20 add_watcher() i=6 2026-02-24T22:18:16.674 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.679+0000 7f961646aa80 20 add_watcher() i=3 2026-02-24T22:18:16.674 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.679+0000 7f961646aa80 20 add_watcher() i=2 2026-02-24T22:18:16.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.679+0000 7f961646aa80 20 add_watcher() i=0 2026-02-24T22:18:16.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.679+0000 7f961646aa80 20 add_watcher() i=1 2026-02-24T22:18:16.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.679+0000 7f961646aa80 20 add_watcher() i=4 2026-02-24T22:18:16.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.679+0000 7f961646aa80 20 add_watcher() i=5 2026-02-24T22:18:16.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.679+0000 7f961646aa80 20 add_watcher() i=7 2026-02-24T22:18:16.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.679+0000 7f961646aa80 2 all 8 watchers are set, enabling cache 2026-02-24T22:18:16.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.679+0000 7f961646aa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:16.677 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.679+0000 7f961646aa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:16.677 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.679+0000 7f961646aa80 5 note: GC not initialized 2026-02-24T22:18:16.677 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.679+0000 7f95c0fe1640 20 reqs_thread_entry: start 2026-02-24T22:18:16.677 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.679+0000 7f960ffff640 0 -- 10.20.193.46:0/3295766240 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x558d22fd6bb0 msgr2=0x558d22fd6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:16.913 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.915+0000 7f961646aa80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:16.914 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.915+0000 7f961646aa80 20 Filter name: none 2026-02-24T22:18:16.914 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.915+0000 7f952affd640 20 reqs_thread_entry: start 2026-02-24T22:18:16.914 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.915+0000 7f961646aa80 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2026-02-24T22:18:16.914 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.915+0000 7f961646aa80 20 rados->read ofs=0 len=0 2026-02-24T22:18:16.914 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.919+0000 7f961646aa80 20 rados_obj.operate() r=0 bl.length=217 2026-02-24T22:18:16.914 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.919+0000 7f961646aa80 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2026-02-24T22:18:16.914 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.919+0000 7f961646aa80 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2026-02-24T22:18:16.914 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.919+0000 7f961646aa80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 : miss 2026-02-24T22:18:16.914 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.919+0000 7f961646aa80 20 rados->read ofs=0 len=0 2026-02-24T22:18:16.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.919+0000 7f961646aa80 20 rados_obj.operate() r=0 bl.length=447 2026-02-24T22:18:16.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.919+0000 7f961646aa80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 info.flags=0x17 2026-02-24T22:18:16.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.919+0000 7f961646aa80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 to cache LRU end 2026-02-24T22:18:16.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.919+0000 7f961646aa80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:18:16.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.919+0000 7f961646aa80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:16.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.919+0000 7f961646aa80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 2026-02-24T22:18:16.918 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.923+0000 7f961646aa80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 info.flags=0x17 2026-02-24T22:18:16.918 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.923+0000 7f961646aa80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 to cache LRU end 2026-02-24T22:18:16.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.923+0000 7f961646aa80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:16.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.923+0000 7f961646aa80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530, ofs0, ns] 2026-02-24T22:18:16.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.923+0000 7f95eaffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961356 cookie 94064666145152 notifier 7359 bl.length()=1017 2026-02-24T22:18:16.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.923+0000 7f95eaffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 info.flags=0x17 2026-02-24T22:18:16.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.923+0000 7f95eaffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 to cache LRU end 2026-02-24T22:18:16.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.923+0000 7f95eaffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:16.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.923+0000 7f961646aa80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): orig_sources= new_sources= 2026-02-24T22:18:16.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.923+0000 7f961646aa80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): potential sources added=[] removed=[] 2026-02-24T22:18:16.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.923+0000 7f961646aa80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): orig_dests= new_dests= 2026-02-24T22:18:16.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.923+0000 7f961646aa80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): potential dests added=[] removed=[] 2026-02-24T22:18:16.920 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.923+0000 7f961646aa80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530.1.0 2026-02-24T22:18:16.920 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.923+0000 7f961646aa80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530.1.0 2026-02-24T22:18:16.920 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.923+0000 7f961646aa80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530.1.1 2026-02-24T22:18:16.920 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.923+0000 7f961646aa80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530.1.1 2026-02-24T22:18:16.920 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.923+0000 7f961646aa80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530.0 2026-02-24T22:18:16.921 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.927+0000 7f961646aa80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 : hit (requested=0x17, cached=0x17) 2026-02-24T22:18:16.922 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.927+0000 7f961646aa80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 2026-02-24T22:18:16.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.927+0000 7f961646aa80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 info.flags=0x17 2026-02-24T22:18:16.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.927+0000 7f961646aa80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 to cache LRU end 2026-02-24T22:18:16.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.927+0000 7f961646aa80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:16.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.927+0000 7f961646aa80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530, ofs0, ns] 2026-02-24T22:18:16.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.927+0000 7f95eaffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961357 cookie 94064666145152 notifier 7359 bl.length()=1017 2026-02-24T22:18:16.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.927+0000 7f95eaffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 info.flags=0x17 2026-02-24T22:18:16.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.927+0000 7f95eaffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 to cache LRU end 2026-02-24T22:18:16.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.927+0000 7f95eaffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:16.924 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.927+0000 7f961646aa80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): orig_sources= new_sources= 2026-02-24T22:18:16.924 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.927+0000 7f961646aa80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): potential sources added=[] removed=[] 2026-02-24T22:18:16.924 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.927+0000 7f961646aa80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): orig_dests= new_dests= 2026-02-24T22:18:16.924 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.927+0000 7f961646aa80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): potential dests added=[] removed=[] 2026-02-24T22:18:16.924 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f961646aa80 time 2026-02-24T22:18:16.929305+0000 2026-02-24T22:18:16.924 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:18:16.925 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:18:16.925 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f96198aceb1] 2026-02-24T22:18:16.925 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x558d12970b9f] 2026-02-24T22:18:16.925 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::do_reshard(rgw::bucket_index_layout_generation const&, rgw::bucket_index_layout_generation const&, int, bool, bool, std::ostream*, ceph::Formatter*, FaultInjector > >&, DoutPrefixProvider const*, optional_yield)+0x20d6) [0x558d130db576] 2026-02-24T22:18:16.925 INFO:tasks.workunit.client.0.trial046.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xe6b) [0x558d130dcf6b] 2026-02-24T22:18:16.925 INFO:tasks.workunit.client.0.trial046.stderr: 5: main() 2026-02-24T22:18:16.925 INFO:tasks.workunit.client.0.trial046.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f9618bacd90] 2026-02-24T22:18:16.925 INFO:tasks.workunit.client.0.trial046.stderr: 7: __libc_start_main() 2026-02-24T22:18:16.925 INFO:tasks.workunit.client.0.trial046.stderr: 8: _start() 2026-02-24T22:18:16.925 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.927+0000 7f961646aa80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f961646aa80 time 2026-02-24T22:18:16.929305+0000 2026-02-24T22:18:16.925 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:18:16.926 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.926 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:18:16.926 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f96198aceb1] 2026-02-24T22:18:16.926 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x558d12970b9f] 2026-02-24T22:18:16.926 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::do_reshard(rgw::bucket_index_layout_generation const&, rgw::bucket_index_layout_generation const&, int, bool, bool, std::ostream*, ceph::Formatter*, FaultInjector > >&, DoutPrefixProvider const*, optional_yield)+0x20d6) [0x558d130db576] 2026-02-24T22:18:16.926 INFO:tasks.workunit.client.0.trial046.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xe6b) [0x558d130dcf6b] 2026-02-24T22:18:16.926 INFO:tasks.workunit.client.0.trial046.stderr: 5: main() 2026-02-24T22:18:16.926 INFO:tasks.workunit.client.0.trial046.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f9618bacd90] 2026-02-24T22:18:16.926 INFO:tasks.workunit.client.0.trial046.stderr: 7: __libc_start_main() 2026-02-24T22:18:16.926 INFO:tasks.workunit.client.0.trial046.stderr: 8: _start() 2026-02-24T22:18:16.926 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.926 INFO:tasks.workunit.client.0.trial046.stderr:*** Caught signal (Aborted) ** 2026-02-24T22:18:16.927 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f961646aa80 thread_name:radosgw-admin 2026-02-24T22:18:16.949 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:18:16.949 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f9618bc5520] 2026-02-24T22:18:16.950 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:18:16.950 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:18:16.950 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:18:16.950 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f96198acf10] 2026-02-24T22:18:16.950 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x558d12970b9f] 2026-02-24T22:18:16.950 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::do_reshard(rgw::bucket_index_layout_generation const&, rgw::bucket_index_layout_generation const&, int, bool, bool, std::ostream*, ceph::Formatter*, FaultInjector > >&, DoutPrefixProvider const*, optional_yield)+0x20d6) [0x558d130db576] 2026-02-24T22:18:16.950 INFO:tasks.workunit.client.0.trial046.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xe6b) [0x558d130dcf6b] 2026-02-24T22:18:16.950 INFO:tasks.workunit.client.0.trial046.stderr: 9: main() 2026-02-24T22:18:16.950 INFO:tasks.workunit.client.0.trial046.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f9618bacd90] 2026-02-24T22:18:16.950 INFO:tasks.workunit.client.0.trial046.stderr: 11: __libc_start_main() 2026-02-24T22:18:16.950 INFO:tasks.workunit.client.0.trial046.stderr: 12: _start() 2026-02-24T22:18:16.951 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.931+0000 7f961646aa80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:18:16.951 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f961646aa80 thread_name:radosgw-admin 2026-02-24T22:18:16.951 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.951 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:18:16.951 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f9618bc5520] 2026-02-24T22:18:16.951 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:18:16.951 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:18:16.951 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:18:16.951 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f96198acf10] 2026-02-24T22:18:16.951 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x558d12970b9f] 2026-02-24T22:18:16.951 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::do_reshard(rgw::bucket_index_layout_generation const&, rgw::bucket_index_layout_generation const&, int, bool, bool, std::ostream*, ceph::Formatter*, FaultInjector > >&, DoutPrefixProvider const*, optional_yield)+0x20d6) [0x558d130db576] 2026-02-24T22:18:16.951 INFO:tasks.workunit.client.0.trial046.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xe6b) [0x558d130dcf6b] 2026-02-24T22:18:16.951 INFO:tasks.workunit.client.0.trial046.stderr: 9: main() 2026-02-24T22:18:16.952 INFO:tasks.workunit.client.0.trial046.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f9618bacd90] 2026-02-24T22:18:16.952 INFO:tasks.workunit.client.0.trial046.stderr: 11: __libc_start_main() 2026-02-24T22:18:16.952 INFO:tasks.workunit.client.0.trial046.stderr: 12: _start() 2026-02-24T22:18:16.952 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:18:16.952 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.952 INFO:tasks.workunit.client.0.trial046.stderr:--- begin dump of recent events --- 2026-02-24T22:18:16.952 INFO:tasks.workunit.client.0.trial046.stderr: -499> 2026-02-24T22:18:16.659+0000 7f960ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:16.952 INFO:tasks.workunit.client.0.trial046.stderr: -498> 2026-02-24T22:18:16.659+0000 7f960ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:16.952 INFO:tasks.workunit.client.0.trial046.stderr: -497> 2026-02-24T22:18:16.659+0000 7f960f7fe640 10 monclient(hunting): get_auth_request con 0x558d22b2d7a0 auth_method 0 2026-02-24T22:18:16.952 INFO:tasks.workunit.client.0.trial046.stderr: -496> 2026-02-24T22:18:16.659+0000 7f960f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:16.952 INFO:tasks.workunit.client.0.trial046.stderr: -495> 2026-02-24T22:18:16.659+0000 7f960f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:16.952 INFO:tasks.workunit.client.0.trial046.stderr: -494> 2026-02-24T22:18:16.659+0000 7f960f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:16.952 INFO:tasks.workunit.client.0.trial046.stderr: -493> 2026-02-24T22:18:16.659+0000 7f960effd640 10 monclient(hunting): handle_auth_done global_id 7347 payload 274 2026-02-24T22:18:16.953 INFO:tasks.workunit.client.0.trial046.stderr: -492> 2026-02-24T22:18:16.659+0000 7f960effd640 10 monclient: _finish_hunting 0 2026-02-24T22:18:16.953 INFO:tasks.workunit.client.0.trial046.stderr: -491> 2026-02-24T22:18:16.659+0000 7f960effd640 1 monclient: found mon.noname-a 2026-02-24T22:18:16.953 INFO:tasks.workunit.client.0.trial046.stderr: -490> 2026-02-24T22:18:16.659+0000 7f960effd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:18:16.953 INFO:tasks.workunit.client.0.trial046.stderr: -489> 2026-02-24T22:18:16.659+0000 7f95fdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:16.953 INFO:tasks.workunit.client.0.trial046.stderr: -488> 2026-02-24T22:18:16.659+0000 7f95fdffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:18:16.953 INFO:tasks.workunit.client.0.trial046.stderr: -487> 2026-02-24T22:18:16.659+0000 7f95fdffb640 10 monclient: dump: 2026-02-24T22:18:16.953 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:16.953 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:16.953 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.953 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.953 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:16.953 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:16.953 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:16.953 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:16.954 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:16.954 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.954 INFO:tasks.workunit.client.0.trial046.stderr: -486> 2026-02-24T22:18:16.659+0000 7f95fdffb640 10 monclient: _finish_auth 0 2026-02-24T22:18:16.954 INFO:tasks.workunit.client.0.trial046.stderr: -485> 2026-02-24T22:18:16.659+0000 7f95fdffb640 10 monclient: _check_auth_tickets 2026-02-24T22:18:16.954 INFO:tasks.workunit.client.0.trial046.stderr: -484> 2026-02-24T22:18:16.659+0000 7f95fdffb640 10 monclient: handle_config config(0 keys) 2026-02-24T22:18:16.954 INFO:tasks.workunit.client.0.trial046.stderr: -483> 2026-02-24T22:18:16.659+0000 7f95fdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:16.954 INFO:tasks.workunit.client.0.trial046.stderr: -482> 2026-02-24T22:18:16.659+0000 7f95fdffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:18:16.954 INFO:tasks.workunit.client.0.trial046.stderr: -481> 2026-02-24T22:18:16.659+0000 7f95fdffb640 10 monclient: dump: 2026-02-24T22:18:16.954 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:16.954 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:16.954 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.954 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.954 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:16.954 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:16.955 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:16.955 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:16.955 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:16.955 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.955 INFO:tasks.workunit.client.0.trial046.stderr: -480> 2026-02-24T22:18:16.659+0000 7f961646aa80 5 monclient: authenticate success, global_id 7347 2026-02-24T22:18:16.955 INFO:tasks.workunit.client.0.trial046.stderr: -479> 2026-02-24T22:18:16.659+0000 7f961646aa80 10 monclient: get_monmap_and_config success 2026-02-24T22:18:16.955 INFO:tasks.workunit.client.0.trial046.stderr: -478> 2026-02-24T22:18:16.659+0000 7f961646aa80 4 set_mon_vals no callback set 2026-02-24T22:18:16.955 INFO:tasks.workunit.client.0.trial046.stderr: -477> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient: shutdown 2026-02-24T22:18:16.955 INFO:tasks.workunit.client.0.trial046.stderr: -476> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 asok(0x558d22aa1f00) unregister_commands rotate-key 2026-02-24T22:18:16.955 INFO:tasks.workunit.client.0.trial046.stderr: -475> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient: build_initial_monmap 2026-02-24T22:18:16.955 INFO:tasks.workunit.client.0.trial046.stderr: -474> 2026-02-24T22:18:16.663+0000 7f961646aa80 1 build_initial for_mkfs: 0 2026-02-24T22:18:16.955 INFO:tasks.workunit.client.0.trial046.stderr: -473> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient: monmap: 2026-02-24T22:18:16.955 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:18:16.955 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:18:16.955 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:18:16.956 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:18:16.956 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:18:16.956 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:16.956 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:18:16.956 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:18:16.956 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:18:16.956 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.956 INFO:tasks.workunit.client.0.trial046.stderr: -472> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding auth protocol: cephx 2026-02-24T22:18:16.956 INFO:tasks.workunit.client.0.trial046.stderr: -471> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding auth protocol: cephx 2026-02-24T22:18:16.956 INFO:tasks.workunit.client.0.trial046.stderr: -470> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding auth protocol: cephx 2026-02-24T22:18:16.956 INFO:tasks.workunit.client.0.trial046.stderr: -469> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding con mode: secure 2026-02-24T22:18:16.956 INFO:tasks.workunit.client.0.trial046.stderr: -468> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding con mode: crc 2026-02-24T22:18:16.956 INFO:tasks.workunit.client.0.trial046.stderr: -467> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding con mode: secure 2026-02-24T22:18:16.956 INFO:tasks.workunit.client.0.trial046.stderr: -466> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding con mode: crc 2026-02-24T22:18:16.956 INFO:tasks.workunit.client.0.trial046.stderr: -465> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding con mode: secure 2026-02-24T22:18:16.956 INFO:tasks.workunit.client.0.trial046.stderr: -464> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding con mode: crc 2026-02-24T22:18:16.957 INFO:tasks.workunit.client.0.trial046.stderr: -463> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding con mode: crc 2026-02-24T22:18:16.957 INFO:tasks.workunit.client.0.trial046.stderr: -462> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding con mode: secure 2026-02-24T22:18:16.957 INFO:tasks.workunit.client.0.trial046.stderr: -461> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding con mode: crc 2026-02-24T22:18:16.957 INFO:tasks.workunit.client.0.trial046.stderr: -460> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding con mode: secure 2026-02-24T22:18:16.957 INFO:tasks.workunit.client.0.trial046.stderr: -459> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding con mode: crc 2026-02-24T22:18:16.957 INFO:tasks.workunit.client.0.trial046.stderr: -458> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding con mode: secure 2026-02-24T22:18:16.957 INFO:tasks.workunit.client.0.trial046.stderr: -457> 2026-02-24T22:18:16.663+0000 7f961646aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:16.957 INFO:tasks.workunit.client.0.trial046.stderr: -456> 2026-02-24T22:18:16.663+0000 7f961646aa80 1 librados: starting msgr at 2026-02-24T22:18:16.957 INFO:tasks.workunit.client.0.trial046.stderr: -455> 2026-02-24T22:18:16.663+0000 7f961646aa80 1 librados: starting objecter 2026-02-24T22:18:16.957 INFO:tasks.workunit.client.0.trial046.stderr: -454> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command objecter_requests hook 0x558d22bd3560 2026-02-24T22:18:16.957 INFO:tasks.workunit.client.0.trial046.stderr: -453> 2026-02-24T22:18:16.663+0000 7f961646aa80 1 librados: setting wanted keys 2026-02-24T22:18:16.957 INFO:tasks.workunit.client.0.trial046.stderr: -452> 2026-02-24T22:18:16.663+0000 7f961646aa80 1 librados: calling monclient init 2026-02-24T22:18:16.957 INFO:tasks.workunit.client.0.trial046.stderr: -451> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient: init 2026-02-24T22:18:16.957 INFO:tasks.workunit.client.0.trial046.stderr: -450> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding auth protocol: cephx 2026-02-24T22:18:16.957 INFO:tasks.workunit.client.0.trial046.stderr: -449> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding auth protocol: cephx 2026-02-24T22:18:16.957 INFO:tasks.workunit.client.0.trial046.stderr: -448> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding auth protocol: cephx 2026-02-24T22:18:16.958 INFO:tasks.workunit.client.0.trial046.stderr: -447> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding con mode: secure 2026-02-24T22:18:16.958 INFO:tasks.workunit.client.0.trial046.stderr: -446> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding con mode: crc 2026-02-24T22:18:16.958 INFO:tasks.workunit.client.0.trial046.stderr: -445> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding con mode: secure 2026-02-24T22:18:16.958 INFO:tasks.workunit.client.0.trial046.stderr: -444> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding con mode: crc 2026-02-24T22:18:16.958 INFO:tasks.workunit.client.0.trial046.stderr: -443> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding con mode: secure 2026-02-24T22:18:16.958 INFO:tasks.workunit.client.0.trial046.stderr: -442> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding con mode: crc 2026-02-24T22:18:16.958 INFO:tasks.workunit.client.0.trial046.stderr: -441> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding con mode: crc 2026-02-24T22:18:16.958 INFO:tasks.workunit.client.0.trial046.stderr: -440> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding con mode: secure 2026-02-24T22:18:16.958 INFO:tasks.workunit.client.0.trial046.stderr: -439> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding con mode: crc 2026-02-24T22:18:16.958 INFO:tasks.workunit.client.0.trial046.stderr: -438> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding con mode: secure 2026-02-24T22:18:16.958 INFO:tasks.workunit.client.0.trial046.stderr: -437> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding con mode: crc 2026-02-24T22:18:16.958 INFO:tasks.workunit.client.0.trial046.stderr: -436> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding con mode: secure 2026-02-24T22:18:16.958 INFO:tasks.workunit.client.0.trial046.stderr: -435> 2026-02-24T22:18:16.663+0000 7f961646aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:16.958 INFO:tasks.workunit.client.0.trial046.stderr: -434> 2026-02-24T22:18:16.663+0000 7f961646aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:16.958 INFO:tasks.workunit.client.0.trial046.stderr: -433> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command rotate-key hook 0x558d22ba36f8 2026-02-24T22:18:16.958 INFO:tasks.workunit.client.0.trial046.stderr: -432> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient: _reopen_session rank -1 2026-02-24T22:18:16.958 INFO:tasks.workunit.client.0.trial046.stderr: -431> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:18:16.958 INFO:tasks.workunit.client.0.trial046.stderr: -430> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient(hunting): picked mon.noname-c con 0x558d22b2d7a0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:18:16.959 INFO:tasks.workunit.client.0.trial046.stderr: -429> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient(hunting): picked mon.noname-a con 0x558d22b30010 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:18:16.959 INFO:tasks.workunit.client.0.trial046.stderr: -428> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient(hunting): picked mon.noname-b con 0x558d22bd7c20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:18:16.959 INFO:tasks.workunit.client.0.trial046.stderr: -427> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:16.959 INFO:tasks.workunit.client.0.trial046.stderr: -426> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:16.959 INFO:tasks.workunit.client.0.trial046.stderr: -425> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:16.959 INFO:tasks.workunit.client.0.trial046.stderr: -424> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient(hunting): _renew_subs 2026-02-24T22:18:16.959 INFO:tasks.workunit.client.0.trial046.stderr: -423> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient(hunting): authenticate will time out at 1866.768826s 2026-02-24T22:18:16.959 INFO:tasks.workunit.client.0.trial046.stderr: -422> 2026-02-24T22:18:16.663+0000 7f960effd640 10 monclient(hunting): get_auth_request con 0x558d22b30010 auth_method 0 2026-02-24T22:18:16.959 INFO:tasks.workunit.client.0.trial046.stderr: -421> 2026-02-24T22:18:16.663+0000 7f960effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:16.959 INFO:tasks.workunit.client.0.trial046.stderr: -420> 2026-02-24T22:18:16.663+0000 7f960effd640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:16.959 INFO:tasks.workunit.client.0.trial046.stderr: -419> 2026-02-24T22:18:16.663+0000 7f960effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:16.959 INFO:tasks.workunit.client.0.trial046.stderr: -418> 2026-02-24T22:18:16.663+0000 7f960f7fe640 10 monclient(hunting): get_auth_request con 0x558d22b2d7a0 auth_method 0 2026-02-24T22:18:16.959 INFO:tasks.workunit.client.0.trial046.stderr: -417> 2026-02-24T22:18:16.663+0000 7f960f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:16.959 INFO:tasks.workunit.client.0.trial046.stderr: -416> 2026-02-24T22:18:16.663+0000 7f960f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:16.959 INFO:tasks.workunit.client.0.trial046.stderr: -415> 2026-02-24T22:18:16.663+0000 7f960f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:16.959 INFO:tasks.workunit.client.0.trial046.stderr: -414> 2026-02-24T22:18:16.663+0000 7f960effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:16.959 INFO:tasks.workunit.client.0.trial046.stderr: -413> 2026-02-24T22:18:16.663+0000 7f960effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:16.959 INFO:tasks.workunit.client.0.trial046.stderr: -412> 2026-02-24T22:18:16.663+0000 7f960effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:16.960 INFO:tasks.workunit.client.0.trial046.stderr: -411> 2026-02-24T22:18:16.663+0000 7f960f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:16.960 INFO:tasks.workunit.client.0.trial046.stderr: -410> 2026-02-24T22:18:16.663+0000 7f960f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:16.960 INFO:tasks.workunit.client.0.trial046.stderr: -409> 2026-02-24T22:18:16.663+0000 7f960f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:16.960 INFO:tasks.workunit.client.0.trial046.stderr: -408> 2026-02-24T22:18:16.663+0000 7f960ffff640 10 monclient(hunting): get_auth_request con 0x558d22bd7c20 auth_method 0 2026-02-24T22:18:16.960 INFO:tasks.workunit.client.0.trial046.stderr: -407> 2026-02-24T22:18:16.663+0000 7f960ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:16.960 INFO:tasks.workunit.client.0.trial046.stderr: -406> 2026-02-24T22:18:16.663+0000 7f960ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:16.960 INFO:tasks.workunit.client.0.trial046.stderr: -405> 2026-02-24T22:18:16.663+0000 7f960ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:16.960 INFO:tasks.workunit.client.0.trial046.stderr: -404> 2026-02-24T22:18:16.663+0000 7f960effd640 10 monclient(hunting): handle_auth_done global_id 7350 payload 824 2026-02-24T22:18:16.960 INFO:tasks.workunit.client.0.trial046.stderr: -403> 2026-02-24T22:18:16.663+0000 7f960effd640 10 monclient: _finish_hunting 0 2026-02-24T22:18:16.960 INFO:tasks.workunit.client.0.trial046.stderr: -402> 2026-02-24T22:18:16.663+0000 7f960effd640 1 monclient: found mon.noname-a 2026-02-24T22:18:16.960 INFO:tasks.workunit.client.0.trial046.stderr: -401> 2026-02-24T22:18:16.663+0000 7f960effd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:18:16.960 INFO:tasks.workunit.client.0.trial046.stderr: -400> 2026-02-24T22:18:16.663+0000 7f95f3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:16.960 INFO:tasks.workunit.client.0.trial046.stderr: -399> 2026-02-24T22:18:16.663+0000 7f95f3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:18:16.960 INFO:tasks.workunit.client.0.trial046.stderr: -398> 2026-02-24T22:18:16.663+0000 7f95f3fff640 10 monclient: dump: 2026-02-24T22:18:16.960 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:16.960 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:16.960 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.960 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.961 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:16.961 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:16.961 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:16.961 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:16.961 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:16.961 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.961 INFO:tasks.workunit.client.0.trial046.stderr: -397> 2026-02-24T22:18:16.663+0000 7f95f3fff640 10 monclient: _finish_auth 0 2026-02-24T22:18:16.961 INFO:tasks.workunit.client.0.trial046.stderr: -396> 2026-02-24T22:18:16.663+0000 7f95f3fff640 10 monclient: _check_auth_tickets 2026-02-24T22:18:16.961 INFO:tasks.workunit.client.0.trial046.stderr: -395> 2026-02-24T22:18:16.663+0000 7f95f3fff640 10 monclient: handle_config config(0 keys) 2026-02-24T22:18:16.961 INFO:tasks.workunit.client.0.trial046.stderr: -394> 2026-02-24T22:18:16.663+0000 7f95f3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:16.961 INFO:tasks.workunit.client.0.trial046.stderr: -393> 2026-02-24T22:18:16.663+0000 7f95f3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:18:16.961 INFO:tasks.workunit.client.0.trial046.stderr: -392> 2026-02-24T22:18:16.663+0000 7f95f3fff640 10 monclient: dump: 2026-02-24T22:18:16.961 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:16.961 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:16.961 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.961 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.961 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:16.961 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:16.961 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:16.962 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:16.962 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:16.962 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.962 INFO:tasks.workunit.client.0.trial046.stderr: -391> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 monclient: authenticate success, global_id 7350 2026-02-24T22:18:16.962 INFO:tasks.workunit.client.0.trial046.stderr: -390> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient: _renew_subs 2026-02-24T22:18:16.962 INFO:tasks.workunit.client.0.trial046.stderr: -389> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:18:16.962 INFO:tasks.workunit.client.0.trial046.stderr: -388> 2026-02-24T22:18:16.663+0000 7f95fdffb640 4 set_mon_vals no callback set 2026-02-24T22:18:16.962 INFO:tasks.workunit.client.0.trial046.stderr: -387> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient: _renew_subs 2026-02-24T22:18:16.962 INFO:tasks.workunit.client.0.trial046.stderr: -386> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:18:16.962 INFO:tasks.workunit.client.0.trial046.stderr: -385> 2026-02-24T22:18:16.663+0000 7f961646aa80 1 librados: init done 2026-02-24T22:18:16.962 INFO:tasks.workunit.client.0.trial046.stderr: -384> 2026-02-24T22:18:16.663+0000 7f95f3fff640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:18:16.962 INFO:tasks.workunit.client.0.trial046.stderr: -383> 2026-02-24T22:18:16.663+0000 7f95f3fff640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:18:16.962 INFO:tasks.workunit.client.0.trial046.stderr: -382> 2026-02-24T22:18:16.663+0000 7f95f3fff640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:18:16.962 INFO:tasks.workunit.client.0.trial046.stderr: -381> 2026-02-24T22:18:16.663+0000 7f960f7fe640 10 monclient: get_auth_request con 0x7f95f40707b0 auth_method 0 2026-02-24T22:18:16.962 INFO:tasks.workunit.client.0.trial046.stderr: -380> 2026-02-24T22:18:16.663+0000 7f960ffff640 10 monclient: get_auth_request con 0x558d22bd1730 auth_method 0 2026-02-24T22:18:16.962 INFO:tasks.workunit.client.0.trial046.stderr: -379> 2026-02-24T22:18:16.663+0000 7f960f7fe640 10 monclient: get_auth_request con 0x558d22bd1b00 auth_method 0 2026-02-24T22:18:16.962 INFO:tasks.workunit.client.0.trial046.stderr: -378> 2026-02-24T22:18:16.667+0000 7f960effd640 10 monclient: get_auth_request con 0x558d22c31ee0 auth_method 0 2026-02-24T22:18:16.962 INFO:tasks.workunit.client.0.trial046.stderr: -377> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient: build_initial_monmap 2026-02-24T22:18:16.963 INFO:tasks.workunit.client.0.trial046.stderr: -376> 2026-02-24T22:18:16.667+0000 7f961646aa80 1 build_initial for_mkfs: 0 2026-02-24T22:18:16.963 INFO:tasks.workunit.client.0.trial046.stderr: -375> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient: monmap: 2026-02-24T22:18:16.963 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:18:16.963 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:18:16.963 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:18:16.963 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:18:16.963 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:18:16.963 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:16.963 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:18:16.963 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:18:16.963 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:18:16.963 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.963 INFO:tasks.workunit.client.0.trial046.stderr: -374> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding auth protocol: cephx 2026-02-24T22:18:16.963 INFO:tasks.workunit.client.0.trial046.stderr: -373> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding auth protocol: cephx 2026-02-24T22:18:16.963 INFO:tasks.workunit.client.0.trial046.stderr: -372> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding auth protocol: cephx 2026-02-24T22:18:16.963 INFO:tasks.workunit.client.0.trial046.stderr: -371> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding con mode: secure 2026-02-24T22:18:16.963 INFO:tasks.workunit.client.0.trial046.stderr: -370> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding con mode: crc 2026-02-24T22:18:16.963 INFO:tasks.workunit.client.0.trial046.stderr: -369> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding con mode: secure 2026-02-24T22:18:16.963 INFO:tasks.workunit.client.0.trial046.stderr: -368> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding con mode: crc 2026-02-24T22:18:16.963 INFO:tasks.workunit.client.0.trial046.stderr: -367> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding con mode: secure 2026-02-24T22:18:16.964 INFO:tasks.workunit.client.0.trial046.stderr: -366> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding con mode: crc 2026-02-24T22:18:16.964 INFO:tasks.workunit.client.0.trial046.stderr: -365> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding con mode: crc 2026-02-24T22:18:16.964 INFO:tasks.workunit.client.0.trial046.stderr: -364> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding con mode: secure 2026-02-24T22:18:16.964 INFO:tasks.workunit.client.0.trial046.stderr: -363> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding con mode: crc 2026-02-24T22:18:16.964 INFO:tasks.workunit.client.0.trial046.stderr: -362> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding con mode: secure 2026-02-24T22:18:16.964 INFO:tasks.workunit.client.0.trial046.stderr: -361> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding con mode: crc 2026-02-24T22:18:16.964 INFO:tasks.workunit.client.0.trial046.stderr: -360> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding con mode: secure 2026-02-24T22:18:16.964 INFO:tasks.workunit.client.0.trial046.stderr: -359> 2026-02-24T22:18:16.667+0000 7f961646aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:16.964 INFO:tasks.workunit.client.0.trial046.stderr: -358> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command objecter_requests cmddesc objecter_requests hook 0x558d22c421b0 EEXIST 2026-02-24T22:18:16.964 INFO:tasks.workunit.client.0.trial046.stderr: -357> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient: init 2026-02-24T22:18:16.964 INFO:tasks.workunit.client.0.trial046.stderr: -356> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding auth protocol: cephx 2026-02-24T22:18:16.964 INFO:tasks.workunit.client.0.trial046.stderr: -355> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding auth protocol: cephx 2026-02-24T22:18:16.964 INFO:tasks.workunit.client.0.trial046.stderr: -354> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding auth protocol: cephx 2026-02-24T22:18:16.964 INFO:tasks.workunit.client.0.trial046.stderr: -353> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding con mode: secure 2026-02-24T22:18:16.964 INFO:tasks.workunit.client.0.trial046.stderr: -352> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding con mode: crc 2026-02-24T22:18:16.964 INFO:tasks.workunit.client.0.trial046.stderr: -351> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding con mode: secure 2026-02-24T22:18:16.964 INFO:tasks.workunit.client.0.trial046.stderr: -350> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding con mode: crc 2026-02-24T22:18:16.964 INFO:tasks.workunit.client.0.trial046.stderr: -349> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding con mode: secure 2026-02-24T22:18:16.964 INFO:tasks.workunit.client.0.trial046.stderr: -348> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding con mode: crc 2026-02-24T22:18:16.965 INFO:tasks.workunit.client.0.trial046.stderr: -347> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding con mode: crc 2026-02-24T22:18:16.965 INFO:tasks.workunit.client.0.trial046.stderr: -346> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding con mode: secure 2026-02-24T22:18:16.965 INFO:tasks.workunit.client.0.trial046.stderr: -345> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding con mode: crc 2026-02-24T22:18:16.965 INFO:tasks.workunit.client.0.trial046.stderr: -344> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding con mode: secure 2026-02-24T22:18:16.965 INFO:tasks.workunit.client.0.trial046.stderr: -343> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding con mode: crc 2026-02-24T22:18:16.965 INFO:tasks.workunit.client.0.trial046.stderr: -342> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding con mode: secure 2026-02-24T22:18:16.965 INFO:tasks.workunit.client.0.trial046.stderr: -341> 2026-02-24T22:18:16.667+0000 7f961646aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:16.965 INFO:tasks.workunit.client.0.trial046.stderr: -340> 2026-02-24T22:18:16.667+0000 7f961646aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:16.965 INFO:tasks.workunit.client.0.trial046.stderr: -339> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command rotate-key cmddesc rotate-key hook 0x558d22c31090 EEXIST 2026-02-24T22:18:16.965 INFO:tasks.workunit.client.0.trial046.stderr: -338> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient: _reopen_session rank -1 2026-02-24T22:18:16.965 INFO:tasks.workunit.client.0.trial046.stderr: -337> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T22:18:16.965 INFO:tasks.workunit.client.0.trial046.stderr: -336> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient(hunting): picked mon.noname-b con 0x558d22d89770 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:18:16.965 INFO:tasks.workunit.client.0.trial046.stderr: -335> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient(hunting): picked mon.noname-c con 0x558d22d89b40 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:18:16.965 INFO:tasks.workunit.client.0.trial046.stderr: -334> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient(hunting): picked mon.noname-a con 0x558d22db79c0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:18:16.965 INFO:tasks.workunit.client.0.trial046.stderr: -333> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:16.965 INFO:tasks.workunit.client.0.trial046.stderr: -332> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:16.965 INFO:tasks.workunit.client.0.trial046.stderr: -331> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:16.965 INFO:tasks.workunit.client.0.trial046.stderr: -330> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient(hunting): _renew_subs 2026-02-24T22:18:16.965 INFO:tasks.workunit.client.0.trial046.stderr: -329> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient(hunting): authenticate will time out at 1866.775009s 2026-02-24T22:18:16.965 INFO:tasks.workunit.client.0.trial046.stderr: -328> 2026-02-24T22:18:16.667+0000 7f960effd640 10 monclient(hunting): get_auth_request con 0x558d22d89b40 auth_method 0 2026-02-24T22:18:16.966 INFO:tasks.workunit.client.0.trial046.stderr: -327> 2026-02-24T22:18:16.667+0000 7f960effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:16.966 INFO:tasks.workunit.client.0.trial046.stderr: -326> 2026-02-24T22:18:16.667+0000 7f960effd640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:16.966 INFO:tasks.workunit.client.0.trial046.stderr: -325> 2026-02-24T22:18:16.667+0000 7f960effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:16.966 INFO:tasks.workunit.client.0.trial046.stderr: -324> 2026-02-24T22:18:16.667+0000 7f960ffff640 10 monclient(hunting): get_auth_request con 0x558d22db79c0 auth_method 0 2026-02-24T22:18:16.966 INFO:tasks.workunit.client.0.trial046.stderr: -323> 2026-02-24T22:18:16.667+0000 7f960ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:16.966 INFO:tasks.workunit.client.0.trial046.stderr: -322> 2026-02-24T22:18:16.667+0000 7f960ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:16.966 INFO:tasks.workunit.client.0.trial046.stderr: -321> 2026-02-24T22:18:16.667+0000 7f960ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:16.966 INFO:tasks.workunit.client.0.trial046.stderr: -320> 2026-02-24T22:18:16.667+0000 7f960effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:16.966 INFO:tasks.workunit.client.0.trial046.stderr: -319> 2026-02-24T22:18:16.667+0000 7f960effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:16.966 INFO:tasks.workunit.client.0.trial046.stderr: -318> 2026-02-24T22:18:16.667+0000 7f960effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:16.966 INFO:tasks.workunit.client.0.trial046.stderr: -317> 2026-02-24T22:18:16.667+0000 7f960ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:16.966 INFO:tasks.workunit.client.0.trial046.stderr: -316> 2026-02-24T22:18:16.667+0000 7f960ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:16.966 INFO:tasks.workunit.client.0.trial046.stderr: -315> 2026-02-24T22:18:16.667+0000 7f960ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:16.966 INFO:tasks.workunit.client.0.trial046.stderr: -314> 2026-02-24T22:18:16.667+0000 7f960f7fe640 10 monclient(hunting): get_auth_request con 0x558d22d89770 auth_method 0 2026-02-24T22:18:16.966 INFO:tasks.workunit.client.0.trial046.stderr: -313> 2026-02-24T22:18:16.667+0000 7f960f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:16.966 INFO:tasks.workunit.client.0.trial046.stderr: -312> 2026-02-24T22:18:16.667+0000 7f960f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:16.966 INFO:tasks.workunit.client.0.trial046.stderr: -311> 2026-02-24T22:18:16.667+0000 7f960f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:16.966 INFO:tasks.workunit.client.0.trial046.stderr: -310> 2026-02-24T22:18:16.667+0000 7f960effd640 10 monclient(hunting): handle_auth_done global_id 6878 payload 824 2026-02-24T22:18:16.966 INFO:tasks.workunit.client.0.trial046.stderr: -309> 2026-02-24T22:18:16.667+0000 7f960effd640 10 monclient: _finish_hunting 0 2026-02-24T22:18:16.966 INFO:tasks.workunit.client.0.trial046.stderr: -308> 2026-02-24T22:18:16.667+0000 7f960effd640 1 monclient: found mon.noname-c 2026-02-24T22:18:16.967 INFO:tasks.workunit.client.0.trial046.stderr: -307> 2026-02-24T22:18:16.667+0000 7f960effd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:18:16.967 INFO:tasks.workunit.client.0.trial046.stderr: -306> 2026-02-24T22:18:16.667+0000 7f95f17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:16.967 INFO:tasks.workunit.client.0.trial046.stderr: -305> 2026-02-24T22:18:16.667+0000 7f95f17fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:18:16.967 INFO:tasks.workunit.client.0.trial046.stderr: -304> 2026-02-24T22:18:16.667+0000 7f95f17fa640 10 monclient: dump: 2026-02-24T22:18:16.967 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:16.967 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:16.967 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.967 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.967 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:16.967 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:16.967 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:16.967 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:16.967 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:16.967 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.967 INFO:tasks.workunit.client.0.trial046.stderr: -303> 2026-02-24T22:18:16.667+0000 7f95f17fa640 10 monclient: _finish_auth 0 2026-02-24T22:18:16.967 INFO:tasks.workunit.client.0.trial046.stderr: -302> 2026-02-24T22:18:16.667+0000 7f95f17fa640 10 monclient: _check_auth_tickets 2026-02-24T22:18:16.967 INFO:tasks.workunit.client.0.trial046.stderr: -301> 2026-02-24T22:18:16.667+0000 7f95f17fa640 10 monclient: handle_config config(0 keys) 2026-02-24T22:18:16.967 INFO:tasks.workunit.client.0.trial046.stderr: -300> 2026-02-24T22:18:16.667+0000 7f95f17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:16.967 INFO:tasks.workunit.client.0.trial046.stderr: -299> 2026-02-24T22:18:16.667+0000 7f95f17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:18:16.967 INFO:tasks.workunit.client.0.trial046.stderr: -298> 2026-02-24T22:18:16.667+0000 7f95f17fa640 10 monclient: dump: 2026-02-24T22:18:16.968 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:16.968 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:16.968 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.968 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.968 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:16.968 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:16.968 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:16.968 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:16.968 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:16.968 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.968 INFO:tasks.workunit.client.0.trial046.stderr: -297> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 monclient: authenticate success, global_id 6878 2026-02-24T22:18:16.968 INFO:tasks.workunit.client.0.trial046.stderr: -296> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient: _renew_subs 2026-02-24T22:18:16.968 INFO:tasks.workunit.client.0.trial046.stderr: -295> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:18:16.968 INFO:tasks.workunit.client.0.trial046.stderr: -294> 2026-02-24T22:18:16.667+0000 7f9614f40640 4 set_mon_vals no callback set 2026-02-24T22:18:16.968 INFO:tasks.workunit.client.0.trial046.stderr: -293> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient: _renew_subs 2026-02-24T22:18:16.968 INFO:tasks.workunit.client.0.trial046.stderr: -292> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:18:16.968 INFO:tasks.workunit.client.0.trial046.stderr: -291> 2026-02-24T22:18:16.671+0000 7f95f17fa640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:18:16.968 INFO:tasks.workunit.client.0.trial046.stderr: -290> 2026-02-24T22:18:16.671+0000 7f95f17fa640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:18:16.968 INFO:tasks.workunit.client.0.trial046.stderr: -289> 2026-02-24T22:18:16.671+0000 7f95f17fa640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:18:16.968 INFO:tasks.workunit.client.0.trial046.stderr: -288> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient: get_auth_request con 0x7f95e40706c0 auth_method 0 2026-02-24T22:18:16.968 INFO:tasks.workunit.client.0.trial046.stderr: -287> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: get_monmap_and_config 2026-02-24T22:18:16.969 INFO:tasks.workunit.client.0.trial046.stderr: -286> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: build_initial_monmap 2026-02-24T22:18:16.969 INFO:tasks.workunit.client.0.trial046.stderr: -285> 2026-02-24T22:18:16.671+0000 7f961646aa80 1 build_initial for_mkfs: 0 2026-02-24T22:18:16.969 INFO:tasks.workunit.client.0.trial046.stderr: -284> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: monmap: 2026-02-24T22:18:16.969 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:18:16.969 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:18:16.969 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:18:16.969 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:18:16.969 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:18:16.969 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:16.969 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:18:16.969 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:18:16.969 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:18:16.969 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.969 INFO:tasks.workunit.client.0.trial046.stderr: -283> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding auth protocol: cephx 2026-02-24T22:18:16.969 INFO:tasks.workunit.client.0.trial046.stderr: -282> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding auth protocol: cephx 2026-02-24T22:18:16.969 INFO:tasks.workunit.client.0.trial046.stderr: -281> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding auth protocol: cephx 2026-02-24T22:18:16.969 INFO:tasks.workunit.client.0.trial046.stderr: -280> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: secure 2026-02-24T22:18:16.969 INFO:tasks.workunit.client.0.trial046.stderr: -279> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: crc 2026-02-24T22:18:16.969 INFO:tasks.workunit.client.0.trial046.stderr: -278> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: secure 2026-02-24T22:18:16.969 INFO:tasks.workunit.client.0.trial046.stderr: -277> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: crc 2026-02-24T22:18:16.969 INFO:tasks.workunit.client.0.trial046.stderr: -276> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: secure 2026-02-24T22:18:16.969 INFO:tasks.workunit.client.0.trial046.stderr: -275> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: crc 2026-02-24T22:18:16.970 INFO:tasks.workunit.client.0.trial046.stderr: -274> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: crc 2026-02-24T22:18:16.970 INFO:tasks.workunit.client.0.trial046.stderr: -273> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: secure 2026-02-24T22:18:16.970 INFO:tasks.workunit.client.0.trial046.stderr: -272> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: crc 2026-02-24T22:18:16.970 INFO:tasks.workunit.client.0.trial046.stderr: -271> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: secure 2026-02-24T22:18:16.970 INFO:tasks.workunit.client.0.trial046.stderr: -270> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: crc 2026-02-24T22:18:16.970 INFO:tasks.workunit.client.0.trial046.stderr: -269> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: secure 2026-02-24T22:18:16.970 INFO:tasks.workunit.client.0.trial046.stderr: -268> 2026-02-24T22:18:16.671+0000 7f961646aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:16.970 INFO:tasks.workunit.client.0.trial046.stderr: -267> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: init 2026-02-24T22:18:16.970 INFO:tasks.workunit.client.0.trial046.stderr: -266> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding auth protocol: cephx 2026-02-24T22:18:16.970 INFO:tasks.workunit.client.0.trial046.stderr: -265> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding auth protocol: cephx 2026-02-24T22:18:16.970 INFO:tasks.workunit.client.0.trial046.stderr: -264> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding auth protocol: cephx 2026-02-24T22:18:16.970 INFO:tasks.workunit.client.0.trial046.stderr: -263> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding con mode: secure 2026-02-24T22:18:16.970 INFO:tasks.workunit.client.0.trial046.stderr: -262> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding con mode: crc 2026-02-24T22:18:16.970 INFO:tasks.workunit.client.0.trial046.stderr: -261> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding con mode: secure 2026-02-24T22:18:16.970 INFO:tasks.workunit.client.0.trial046.stderr: -260> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding con mode: crc 2026-02-24T22:18:16.970 INFO:tasks.workunit.client.0.trial046.stderr: -259> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding con mode: secure 2026-02-24T22:18:16.970 INFO:tasks.workunit.client.0.trial046.stderr: -258> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding con mode: crc 2026-02-24T22:18:16.970 INFO:tasks.workunit.client.0.trial046.stderr: -257> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding con mode: crc 2026-02-24T22:18:16.970 INFO:tasks.workunit.client.0.trial046.stderr: -256> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding con mode: secure 2026-02-24T22:18:16.970 INFO:tasks.workunit.client.0.trial046.stderr: -255> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding con mode: crc 2026-02-24T22:18:16.970 INFO:tasks.workunit.client.0.trial046.stderr: -254> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding con mode: secure 2026-02-24T22:18:16.970 INFO:tasks.workunit.client.0.trial046.stderr: -253> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding con mode: crc 2026-02-24T22:18:16.970 INFO:tasks.workunit.client.0.trial046.stderr: -252> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding con mode: secure 2026-02-24T22:18:16.970 INFO:tasks.workunit.client.0.trial046.stderr: -251> 2026-02-24T22:18:16.671+0000 7f961646aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:16.971 INFO:tasks.workunit.client.0.trial046.stderr: -250> 2026-02-24T22:18:16.671+0000 7f961646aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:16.971 INFO:tasks.workunit.client.0.trial046.stderr: -249> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command rotate-key cmddesc rotate-key hook 0x7ffd7bdc9978 EEXIST 2026-02-24T22:18:16.971 INFO:tasks.workunit.client.0.trial046.stderr: -248> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: _reopen_session rank -1 2026-02-24T22:18:16.971 INFO:tasks.workunit.client.0.trial046.stderr: -247> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T22:18:16.971 INFO:tasks.workunit.client.0.trial046.stderr: -246> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): picked mon.noname-c con 0x558d22e43fb0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:18:16.971 INFO:tasks.workunit.client.0.trial046.stderr: -245> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): picked mon.noname-b con 0x558d22e47720 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:18:16.971 INFO:tasks.workunit.client.0.trial046.stderr: -244> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): picked mon.noname-a con 0x558d22e224e0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:18:16.971 INFO:tasks.workunit.client.0.trial046.stderr: -243> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:16.971 INFO:tasks.workunit.client.0.trial046.stderr: -242> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:16.971 INFO:tasks.workunit.client.0.trial046.stderr: -241> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:16.971 INFO:tasks.workunit.client.0.trial046.stderr: -240> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): _renew_subs 2026-02-24T22:18:16.971 INFO:tasks.workunit.client.0.trial046.stderr: -239> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): authenticate will time out at 1866.777750s 2026-02-24T22:18:16.971 INFO:tasks.workunit.client.0.trial046.stderr: -238> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): get_auth_request con 0x558d22e43fb0 auth_method 0 2026-02-24T22:18:16.971 INFO:tasks.workunit.client.0.trial046.stderr: -237> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:16.971 INFO:tasks.workunit.client.0.trial046.stderr: -236> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:16.971 INFO:tasks.workunit.client.0.trial046.stderr: -235> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:16.971 INFO:tasks.workunit.client.0.trial046.stderr: -234> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:16.971 INFO:tasks.workunit.client.0.trial046.stderr: -233> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:16.971 INFO:tasks.workunit.client.0.trial046.stderr: -232> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:16.971 INFO:tasks.workunit.client.0.trial046.stderr: -231> 2026-02-24T22:18:16.671+0000 7f960ffff640 10 monclient(hunting): get_auth_request con 0x558d22e224e0 auth_method 0 2026-02-24T22:18:16.971 INFO:tasks.workunit.client.0.trial046.stderr: -230> 2026-02-24T22:18:16.671+0000 7f960ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:16.971 INFO:tasks.workunit.client.0.trial046.stderr: -229> 2026-02-24T22:18:16.671+0000 7f960ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:16.971 INFO:tasks.workunit.client.0.trial046.stderr: -228> 2026-02-24T22:18:16.671+0000 7f960ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:16.972 INFO:tasks.workunit.client.0.trial046.stderr: -227> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): handle_auth_done global_id 6881 payload 274 2026-02-24T22:18:16.972 INFO:tasks.workunit.client.0.trial046.stderr: -226> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient: _finish_hunting 0 2026-02-24T22:18:16.972 INFO:tasks.workunit.client.0.trial046.stderr: -225> 2026-02-24T22:18:16.671+0000 7f960f7fe640 1 monclient: found mon.noname-c 2026-02-24T22:18:16.972 INFO:tasks.workunit.client.0.trial046.stderr: -224> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:18:16.972 INFO:tasks.workunit.client.0.trial046.stderr: -223> 2026-02-24T22:18:16.671+0000 7f95eaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:16.972 INFO:tasks.workunit.client.0.trial046.stderr: -222> 2026-02-24T22:18:16.671+0000 7f95eaffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:18:16.972 INFO:tasks.workunit.client.0.trial046.stderr: -221> 2026-02-24T22:18:16.671+0000 7f95eaffd640 10 monclient: dump: 2026-02-24T22:18:16.972 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:16.972 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:16.972 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.972 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.972 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:16.972 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:16.972 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:16.972 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:16.972 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:16.972 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.972 INFO:tasks.workunit.client.0.trial046.stderr: -220> 2026-02-24T22:18:16.671+0000 7f95eaffd640 10 monclient: _finish_auth 0 2026-02-24T22:18:16.972 INFO:tasks.workunit.client.0.trial046.stderr: -219> 2026-02-24T22:18:16.671+0000 7f95eaffd640 10 monclient: _check_auth_tickets 2026-02-24T22:18:16.972 INFO:tasks.workunit.client.0.trial046.stderr: -218> 2026-02-24T22:18:16.671+0000 7f95eaffd640 10 monclient: handle_config config(0 keys) 2026-02-24T22:18:16.972 INFO:tasks.workunit.client.0.trial046.stderr: -217> 2026-02-24T22:18:16.671+0000 7f95eaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:16.972 INFO:tasks.workunit.client.0.trial046.stderr: -216> 2026-02-24T22:18:16.671+0000 7f95eaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:18:16.972 INFO:tasks.workunit.client.0.trial046.stderr: -215> 2026-02-24T22:18:16.671+0000 7f95eaffd640 10 monclient: dump: 2026-02-24T22:18:16.973 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:16.973 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:16.973 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.973 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.973 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:16.973 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:16.973 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:16.973 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:16.973 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:16.973 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.973 INFO:tasks.workunit.client.0.trial046.stderr: -214> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 monclient: authenticate success, global_id 6881 2026-02-24T22:18:16.973 INFO:tasks.workunit.client.0.trial046.stderr: -213> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: get_monmap_and_config success 2026-02-24T22:18:16.973 INFO:tasks.workunit.client.0.trial046.stderr: -212> 2026-02-24T22:18:16.671+0000 7f961646aa80 4 set_mon_vals no callback set 2026-02-24T22:18:16.973 INFO:tasks.workunit.client.0.trial046.stderr: -211> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: shutdown 2026-02-24T22:18:16.973 INFO:tasks.workunit.client.0.trial046.stderr: -210> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: build_initial_monmap 2026-02-24T22:18:16.973 INFO:tasks.workunit.client.0.trial046.stderr: -209> 2026-02-24T22:18:16.671+0000 7f961646aa80 1 build_initial for_mkfs: 0 2026-02-24T22:18:16.973 INFO:tasks.workunit.client.0.trial046.stderr: -208> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: monmap: 2026-02-24T22:18:16.973 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:18:16.973 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:18:16.973 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:18:16.973 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:18:16.973 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:18:16.973 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:16.973 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr: -207> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding auth protocol: cephx 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr: -206> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding auth protocol: cephx 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr: -205> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding auth protocol: cephx 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr: -204> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: secure 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr: -203> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: crc 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr: -202> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: secure 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr: -201> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: crc 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr: -200> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: secure 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr: -199> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: crc 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr: -198> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: crc 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr: -197> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: secure 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr: -196> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: crc 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr: -195> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: secure 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr: -194> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: crc 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr: -193> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: secure 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr: -192> 2026-02-24T22:18:16.671+0000 7f961646aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr: -191> 2026-02-24T22:18:16.671+0000 7f961646aa80 1 librados: starting msgr at 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr: -190> 2026-02-24T22:18:16.671+0000 7f961646aa80 1 librados: starting objecter 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr: -189> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command objecter_requests cmddesc objecter_requests hook 0x558d22e20aa0 EEXIST 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr: -188> 2026-02-24T22:18:16.671+0000 7f961646aa80 1 librados: setting wanted keys 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr: -187> 2026-02-24T22:18:16.671+0000 7f961646aa80 1 librados: calling monclient init 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr: -186> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: init 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr: -185> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding auth protocol: cephx 2026-02-24T22:18:16.974 INFO:tasks.workunit.client.0.trial046.stderr: -184> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding auth protocol: cephx 2026-02-24T22:18:16.975 INFO:tasks.workunit.client.0.trial046.stderr: -183> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding auth protocol: cephx 2026-02-24T22:18:16.975 INFO:tasks.workunit.client.0.trial046.stderr: -182> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding con mode: secure 2026-02-24T22:18:16.975 INFO:tasks.workunit.client.0.trial046.stderr: -181> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding con mode: crc 2026-02-24T22:18:16.975 INFO:tasks.workunit.client.0.trial046.stderr: -180> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding con mode: secure 2026-02-24T22:18:16.975 INFO:tasks.workunit.client.0.trial046.stderr: -179> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding con mode: crc 2026-02-24T22:18:16.975 INFO:tasks.workunit.client.0.trial046.stderr: -178> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding con mode: secure 2026-02-24T22:18:16.975 INFO:tasks.workunit.client.0.trial046.stderr: -177> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding con mode: crc 2026-02-24T22:18:16.975 INFO:tasks.workunit.client.0.trial046.stderr: -176> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding con mode: crc 2026-02-24T22:18:16.975 INFO:tasks.workunit.client.0.trial046.stderr: -175> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding con mode: secure 2026-02-24T22:18:16.975 INFO:tasks.workunit.client.0.trial046.stderr: -174> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding con mode: crc 2026-02-24T22:18:16.975 INFO:tasks.workunit.client.0.trial046.stderr: -173> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding con mode: secure 2026-02-24T22:18:16.975 INFO:tasks.workunit.client.0.trial046.stderr: -172> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding con mode: crc 2026-02-24T22:18:16.975 INFO:tasks.workunit.client.0.trial046.stderr: -171> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding con mode: secure 2026-02-24T22:18:16.975 INFO:tasks.workunit.client.0.trial046.stderr: -170> 2026-02-24T22:18:16.671+0000 7f961646aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:16.975 INFO:tasks.workunit.client.0.trial046.stderr: -169> 2026-02-24T22:18:16.671+0000 7f961646aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:16.975 INFO:tasks.workunit.client.0.trial046.stderr: -168> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command rotate-key cmddesc rotate-key hook 0x558d22e1bc68 EEXIST 2026-02-24T22:18:16.975 INFO:tasks.workunit.client.0.trial046.stderr: -167> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: _reopen_session rank -1 2026-02-24T22:18:16.975 INFO:tasks.workunit.client.0.trial046.stderr: -166> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T22:18:16.975 INFO:tasks.workunit.client.0.trial046.stderr: -165> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): picked mon.noname-a con 0x558d22e43fb0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:18:16.975 INFO:tasks.workunit.client.0.trial046.stderr: -164> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): picked mon.noname-c con 0x558d22e47720 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:18:16.975 INFO:tasks.workunit.client.0.trial046.stderr: -163> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): picked mon.noname-b con 0x558d22e4fd40 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:18:16.975 INFO:tasks.workunit.client.0.trial046.stderr: -162> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:16.975 INFO:tasks.workunit.client.0.trial046.stderr: -161> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:16.975 INFO:tasks.workunit.client.0.trial046.stderr: -160> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:16.975 INFO:tasks.workunit.client.0.trial046.stderr: -159> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): _renew_subs 2026-02-24T22:18:16.975 INFO:tasks.workunit.client.0.trial046.stderr: -158> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): authenticate will time out at 1866.779018s 2026-02-24T22:18:16.976 INFO:tasks.workunit.client.0.trial046.stderr: -157> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): get_auth_request con 0x558d22e43fb0 auth_method 0 2026-02-24T22:18:16.976 INFO:tasks.workunit.client.0.trial046.stderr: -156> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:16.976 INFO:tasks.workunit.client.0.trial046.stderr: -155> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:16.976 INFO:tasks.workunit.client.0.trial046.stderr: -154> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:16.976 INFO:tasks.workunit.client.0.trial046.stderr: -153> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:16.976 INFO:tasks.workunit.client.0.trial046.stderr: -152> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:16.976 INFO:tasks.workunit.client.0.trial046.stderr: -151> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:16.976 INFO:tasks.workunit.client.0.trial046.stderr: -150> 2026-02-24T22:18:16.671+0000 7f960effd640 10 monclient(hunting): get_auth_request con 0x558d22e47720 auth_method 0 2026-02-24T22:18:16.976 INFO:tasks.workunit.client.0.trial046.stderr: -149> 2026-02-24T22:18:16.671+0000 7f960effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:16.976 INFO:tasks.workunit.client.0.trial046.stderr: -148> 2026-02-24T22:18:16.671+0000 7f960effd640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:16.976 INFO:tasks.workunit.client.0.trial046.stderr: -147> 2026-02-24T22:18:16.671+0000 7f960effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:16.976 INFO:tasks.workunit.client.0.trial046.stderr: -146> 2026-02-24T22:18:16.671+0000 7f960effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:16.976 INFO:tasks.workunit.client.0.trial046.stderr: -145> 2026-02-24T22:18:16.671+0000 7f960effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:16.976 INFO:tasks.workunit.client.0.trial046.stderr: -144> 2026-02-24T22:18:16.671+0000 7f960effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:16.976 INFO:tasks.workunit.client.0.trial046.stderr: -143> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): handle_auth_done global_id 7359 payload 824 2026-02-24T22:18:16.976 INFO:tasks.workunit.client.0.trial046.stderr: -142> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient: _finish_hunting 0 2026-02-24T22:18:16.976 INFO:tasks.workunit.client.0.trial046.stderr: -141> 2026-02-24T22:18:16.671+0000 7f960f7fe640 1 monclient: found mon.noname-a 2026-02-24T22:18:16.976 INFO:tasks.workunit.client.0.trial046.stderr: -140> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:18:16.976 INFO:tasks.workunit.client.0.trial046.stderr: -139> 2026-02-24T22:18:16.671+0000 7f95e97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:16.976 INFO:tasks.workunit.client.0.trial046.stderr: -138> 2026-02-24T22:18:16.671+0000 7f95e97fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:18:16.976 INFO:tasks.workunit.client.0.trial046.stderr: -137> 2026-02-24T22:18:16.671+0000 7f95e97fa640 10 monclient: dump: 2026-02-24T22:18:16.976 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:16.976 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:16.976 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.976 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.976 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:16.977 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:16.977 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:16.977 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:16.977 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:16.977 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.977 INFO:tasks.workunit.client.0.trial046.stderr: -136> 2026-02-24T22:18:16.671+0000 7f95e97fa640 10 monclient: _finish_auth 0 2026-02-24T22:18:16.977 INFO:tasks.workunit.client.0.trial046.stderr: -135> 2026-02-24T22:18:16.671+0000 7f95e97fa640 10 monclient: _check_auth_tickets 2026-02-24T22:18:16.977 INFO:tasks.workunit.client.0.trial046.stderr: -134> 2026-02-24T22:18:16.671+0000 7f95e97fa640 10 monclient: handle_config config(0 keys) 2026-02-24T22:18:16.977 INFO:tasks.workunit.client.0.trial046.stderr: -133> 2026-02-24T22:18:16.671+0000 7f95e97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:16.977 INFO:tasks.workunit.client.0.trial046.stderr: -132> 2026-02-24T22:18:16.671+0000 7f95e97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:18:16.977 INFO:tasks.workunit.client.0.trial046.stderr: -131> 2026-02-24T22:18:16.671+0000 7f95e97fa640 10 monclient: dump: 2026-02-24T22:18:16.977 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:16.977 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:16.977 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.977 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.977 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:16.977 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:16.977 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:16.977 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:16.977 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:16.977 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.977 INFO:tasks.workunit.client.0.trial046.stderr: -130> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 monclient: authenticate success, global_id 7359 2026-02-24T22:18:16.977 INFO:tasks.workunit.client.0.trial046.stderr: -129> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: _renew_subs 2026-02-24T22:18:16.977 INFO:tasks.workunit.client.0.trial046.stderr: -128> 2026-02-24T22:18:16.671+0000 7f95eaffd640 4 set_mon_vals no callback set 2026-02-24T22:18:16.977 INFO:tasks.workunit.client.0.trial046.stderr: -127> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:18:16.977 INFO:tasks.workunit.client.0.trial046.stderr: -126> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: _renew_subs 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -125> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -124> 2026-02-24T22:18:16.671+0000 7f961646aa80 1 librados: init done 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -123> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command cr dump hook 0x558d22e44cf8 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -122> 2026-02-24T22:18:16.675+0000 7f961646aa80 4 Realm: () 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -121> 2026-02-24T22:18:16.675+0000 7f961646aa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -120> 2026-02-24T22:18:16.675+0000 7f961646aa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -119> 2026-02-24T22:18:16.675+0000 7f961646aa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -118> 2026-02-24T22:18:16.675+0000 7f961646aa80 20 zonegroup default 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -117> 2026-02-24T22:18:16.675+0000 7f960effd640 10 monclient: get_auth_request con 0x558d22ee5650 auth_method 0 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -116> 2026-02-24T22:18:16.675+0000 7f961646aa80 20 rados->read ofs=0 len=0 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -115> 2026-02-24T22:18:16.675+0000 7f95e97fa640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -114> 2026-02-24T22:18:16.675+0000 7f95e97fa640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -113> 2026-02-24T22:18:16.675+0000 7f95e97fa640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -112> 2026-02-24T22:18:16.675+0000 7f960ffff640 10 monclient: get_auth_request con 0x7f95d40707b0 auth_method 0 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -111> 2026-02-24T22:18:16.675+0000 7f960f7fe640 10 monclient: get_auth_request con 0x558d22f08970 auth_method 0 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -110> 2026-02-24T22:18:16.675+0000 7f961646aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -109> 2026-02-24T22:18:16.675+0000 7f961646aa80 20 rados->read ofs=0 len=0 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -108> 2026-02-24T22:18:16.675+0000 7f960effd640 10 monclient: get_auth_request con 0x558d22f2a050 auth_method 0 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -107> 2026-02-24T22:18:16.675+0000 7f961646aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -106> 2026-02-24T22:18:16.675+0000 7f961646aa80 20 started sync module instance, tier type = 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -105> 2026-02-24T22:18:16.675+0000 7f961646aa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -104> 2026-02-24T22:18:16.675+0000 7f960ffff640 10 monclient: get_auth_request con 0x558d22f8b4d0 auth_method 0 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -103> 2026-02-24T22:18:16.675+0000 7f960ffff640 10 monclient: get_auth_request con 0x558d23092d70 auth_method 0 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -102> 2026-02-24T22:18:16.675+0000 7f960f7fe640 10 monclient: get_auth_request con 0x558d22fedcb0 auth_method 0 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -101> 2026-02-24T22:18:16.675+0000 7f960effd640 10 monclient: get_auth_request con 0x558d23050c30 auth_method 0 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -100> 2026-02-24T22:18:16.675+0000 7f960f7fe640 10 monclient: get_auth_request con 0x558d230f5d70 auth_method 0 2026-02-24T22:18:16.978 INFO:tasks.workunit.client.0.trial046.stderr: -99> 2026-02-24T22:18:16.675+0000 7f960ffff640 0 -- 10.20.193.46:0/3578363307 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x558d22f8b4d0 msgr2=0x558d22fab8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -98> 2026-02-24T22:18:16.675+0000 7f95e97fa640 1 client.7359.objecter ms_handle_reset 0x558d22f8b4d0 session 0x558d22f07f50 osd.0 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -97> 2026-02-24T22:18:16.679+0000 7f95e97fa640 10 monclient: _renew_subs 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -96> 2026-02-24T22:18:16.679+0000 7f95e97fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -95> 2026-02-24T22:18:16.679+0000 7f960ffff640 10 monclient: get_auth_request con 0x7f95d409d4e0 auth_method 0 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -94> 2026-02-24T22:18:16.679+0000 7f961646aa80 20 add_watcher() i=6 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -93> 2026-02-24T22:18:16.679+0000 7f961646aa80 20 add_watcher() i=3 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -92> 2026-02-24T22:18:16.679+0000 7f961646aa80 20 add_watcher() i=2 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -91> 2026-02-24T22:18:16.679+0000 7f961646aa80 20 add_watcher() i=0 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -90> 2026-02-24T22:18:16.679+0000 7f961646aa80 20 add_watcher() i=1 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -89> 2026-02-24T22:18:16.679+0000 7f961646aa80 20 add_watcher() i=4 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -88> 2026-02-24T22:18:16.679+0000 7f961646aa80 20 add_watcher() i=5 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -87> 2026-02-24T22:18:16.679+0000 7f961646aa80 20 add_watcher() i=7 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -86> 2026-02-24T22:18:16.679+0000 7f961646aa80 2 all 8 watchers are set, enabling cache 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -85> 2026-02-24T22:18:16.679+0000 7f960effd640 10 monclient: get_auth_request con 0x7f9534008af0 auth_method 0 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -84> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding auth protocol: cephx 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -83> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding auth protocol: cephx 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -82> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding auth protocol: cephx 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -81> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding con mode: secure 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -80> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding con mode: crc 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -79> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding con mode: secure 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -78> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding con mode: crc 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -77> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding con mode: secure 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -76> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding con mode: crc 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -75> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding con mode: crc 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -74> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding con mode: secure 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -73> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding con mode: crc 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -72> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding con mode: secure 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -71> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding con mode: crc 2026-02-24T22:18:16.979 INFO:tasks.workunit.client.0.trial046.stderr: -70> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding con mode: secure 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -69> 2026-02-24T22:18:16.679+0000 7f961646aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -68> 2026-02-24T22:18:16.679+0000 7f961646aa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -67> 2026-02-24T22:18:16.679+0000 7f961646aa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -66> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command cache list hook 0x558d22e6a290 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -65> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command cache inspect hook 0x558d22e6a290 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -64> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command cache erase hook 0x558d22e6a290 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -63> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command cache zap hook 0x558d22e6a290 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -62> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 note: GC not initialized 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -61> 2026-02-24T22:18:16.679+0000 7f95c0fe1640 20 reqs_thread_entry: start 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -60> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command sync trace show hook 0x7f9600026040 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -59> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command sync trace history hook 0x7f9600026040 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -58> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command sync trace active hook 0x7f9600026040 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -57> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command sync trace active_short hook 0x7f9600026040 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -56> 2026-02-24T22:18:16.679+0000 7f960ffff640 10 monclient: get_auth_request con 0x558d22fd6bb0 auth_method 0 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -55> 2026-02-24T22:18:16.679+0000 7f960ffff640 0 -- 10.20.193.46:0/3295766240 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x558d22fd6bb0 msgr2=0x558d22fd6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -54> 2026-02-24T22:18:16.883+0000 7f960ffff640 10 monclient: get_auth_request con 0x558d22fd6bb0 auth_method 0 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -53> 2026-02-24T22:18:16.883+0000 7f960f7fe640 10 monclient: get_auth_request con 0x558d22f8ebb0 auth_method 0 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -52> 2026-02-24T22:18:16.883+0000 7f960effd640 10 monclient: get_auth_request con 0x558d22f90a30 auth_method 0 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -51> 2026-02-24T22:18:16.887+0000 7f960ffff640 10 monclient: get_auth_request con 0x558d22fd84a0 auth_method 0 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -50> 2026-02-24T22:18:16.891+0000 7f960f7fe640 10 monclient: get_auth_request con 0x558d22fd6750 auth_method 0 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -49> 2026-02-24T22:18:16.895+0000 7f960effd640 10 monclient: get_auth_request con 0x558d22fd5390 auth_method 0 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -48> 2026-02-24T22:18:16.915+0000 7f961646aa80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -47> 2026-02-24T22:18:16.915+0000 7f961646aa80 20 Filter name: none 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -46> 2026-02-24T22:18:16.915+0000 7f952affd640 20 reqs_thread_entry: start 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -45> 2026-02-24T22:18:16.915+0000 7f961646aa80 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -44> 2026-02-24T22:18:16.915+0000 7f961646aa80 20 rados->read ofs=0 len=0 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -43> 2026-02-24T22:18:16.919+0000 7f961646aa80 20 rados_obj.operate() r=0 bl.length=217 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -42> 2026-02-24T22:18:16.919+0000 7f961646aa80 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2026-02-24T22:18:16.980 INFO:tasks.workunit.client.0.trial046.stderr: -41> 2026-02-24T22:18:16.919+0000 7f961646aa80 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -40> 2026-02-24T22:18:16.919+0000 7f961646aa80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 : miss 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -39> 2026-02-24T22:18:16.919+0000 7f961646aa80 20 rados->read ofs=0 len=0 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -38> 2026-02-24T22:18:16.919+0000 7f961646aa80 20 rados_obj.operate() r=0 bl.length=447 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -37> 2026-02-24T22:18:16.919+0000 7f961646aa80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 info.flags=0x17 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -36> 2026-02-24T22:18:16.919+0000 7f961646aa80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 to cache LRU end 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -35> 2026-02-24T22:18:16.919+0000 7f961646aa80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -34> 2026-02-24T22:18:16.919+0000 7f961646aa80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -33> 2026-02-24T22:18:16.919+0000 7f961646aa80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -32> 2026-02-24T22:18:16.923+0000 7f961646aa80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 info.flags=0x17 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -31> 2026-02-24T22:18:16.923+0000 7f961646aa80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 to cache LRU end 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -30> 2026-02-24T22:18:16.923+0000 7f961646aa80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -29> 2026-02-24T22:18:16.923+0000 7f961646aa80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530, ofs0, ns] 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -28> 2026-02-24T22:18:16.923+0000 7f95eaffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961356 cookie 94064666145152 notifier 7359 bl.length()=1017 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -27> 2026-02-24T22:18:16.923+0000 7f95eaffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 info.flags=0x17 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -26> 2026-02-24T22:18:16.923+0000 7f95eaffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 to cache LRU end 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -25> 2026-02-24T22:18:16.923+0000 7f95eaffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -24> 2026-02-24T22:18:16.923+0000 7f961646aa80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): orig_sources= new_sources= 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -23> 2026-02-24T22:18:16.923+0000 7f961646aa80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): potential sources added=[] removed=[] 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -22> 2026-02-24T22:18:16.923+0000 7f961646aa80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): orig_dests= new_dests= 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -21> 2026-02-24T22:18:16.923+0000 7f961646aa80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): potential dests added=[] removed=[] 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -20> 2026-02-24T22:18:16.923+0000 7f961646aa80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530.1.0 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -19> 2026-02-24T22:18:16.923+0000 7f961646aa80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530.1.0 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -18> 2026-02-24T22:18:16.923+0000 7f961646aa80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530.1.1 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -17> 2026-02-24T22:18:16.923+0000 7f961646aa80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530.1.1 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -16> 2026-02-24T22:18:16.923+0000 7f961646aa80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530.0 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -15> 2026-02-24T22:18:16.927+0000 7f961646aa80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 : hit (requested=0x17, cached=0x17) 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -14> 2026-02-24T22:18:16.927+0000 7f961646aa80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 2026-02-24T22:18:16.981 INFO:tasks.workunit.client.0.trial046.stderr: -13> 2026-02-24T22:18:16.927+0000 7f961646aa80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 info.flags=0x17 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr: -12> 2026-02-24T22:18:16.927+0000 7f961646aa80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 to cache LRU end 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr: -11> 2026-02-24T22:18:16.927+0000 7f961646aa80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr: -10> 2026-02-24T22:18:16.927+0000 7f961646aa80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530, ofs0, ns] 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr: -9> 2026-02-24T22:18:16.927+0000 7f95eaffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961357 cookie 94064666145152 notifier 7359 bl.length()=1017 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr: -8> 2026-02-24T22:18:16.927+0000 7f95eaffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 info.flags=0x17 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr: -7> 2026-02-24T22:18:16.927+0000 7f95eaffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 to cache LRU end 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr: -6> 2026-02-24T22:18:16.927+0000 7f95eaffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr: -5> 2026-02-24T22:18:16.927+0000 7f961646aa80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): orig_sources= new_sources= 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr: -4> 2026-02-24T22:18:16.927+0000 7f961646aa80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): potential sources added=[] removed=[] 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr: -3> 2026-02-24T22:18:16.927+0000 7f961646aa80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): orig_dests= new_dests= 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr: -2> 2026-02-24T22:18:16.927+0000 7f961646aa80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): potential dests added=[] removed=[] 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr: -1> 2026-02-24T22:18:16.927+0000 7f961646aa80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f961646aa80 time 2026-02-24T22:18:16.929305+0000 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f96198aceb1] 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x558d12970b9f] 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::do_reshard(rgw::bucket_index_layout_generation const&, rgw::bucket_index_layout_generation const&, int, bool, bool, std::ostream*, ceph::Formatter*, FaultInjector > >&, DoutPrefixProvider const*, optional_yield)+0x20d6) [0x558d130db576] 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xe6b) [0x558d130dcf6b] 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr: 5: main() 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f9618bacd90] 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr: 7: __libc_start_main() 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr: 8: _start() 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr: 0> 2026-02-24T22:18:16.931+0000 7f961646aa80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f961646aa80 thread_name:radosgw-admin 2026-02-24T22:18:16.982 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f9618bc5520] 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f96198acf10] 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x558d12970b9f] 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::do_reshard(rgw::bucket_index_layout_generation const&, rgw::bucket_index_layout_generation const&, int, bool, bool, std::ostream*, ceph::Formatter*, FaultInjector > >&, DoutPrefixProvider const*, optional_yield)+0x20d6) [0x558d130db576] 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xe6b) [0x558d130dcf6b] 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: 9: main() 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f9618bacd90] 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: 11: __libc_start_main() 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: 12: _start() 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr:--- logging levels --- 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 none 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 lockdep 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 context 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 crush 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_balancer 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_locker 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log_expire 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_migrator 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: 3/ 5 mds_quiesce 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 buffer 2026-02-24T22:18:16.983 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 timer 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 filer 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 striper 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 objecter 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rados 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_mirror 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_replay 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_pwl 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 journaler 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objectcacher 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 immutable_obj_cache 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 client 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 osd 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 optracker 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objclass 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 filestore 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 journal 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 ms 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mon 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 0/10 monc 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 paxos 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 tp 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 auth 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 crypto 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 finisher 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 reserver 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 heartbeatmap 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 perfcounter 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_sync 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_datacache 2026-02-24T22:18:16.984 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_access 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dbstore 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_flight 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_lifecycle 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_restore 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw_notification 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_bucket_logging 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 javaclient 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 asok 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 throttle 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 refs 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 compressor 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore_compression 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluefs 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 bdev 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 kstore 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 4/ 5 rocksdb 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 fuse 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 2/ 5 mgr 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mgrc 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 dpdk 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 eventtrace 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 prioritycache 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 test 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephfs_mirror 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephsqlite 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 crimson_interrupt 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_onode 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_odata 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_omap 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_tm 2026-02-24T22:18:16.985 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_t 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cleaner 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_epm 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_lba 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_fixedkv_tree 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cache 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_journal 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_device 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_backref 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 alienstore 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mclock 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dedup 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cyanstore 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 ceph_exporter 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 memstore 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 trace 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 ceph_dedup 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: -2/-2 (syslog threshold) 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 99/99 (stderr threshold) 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr:--- pthread ID / name mapping for recent threads --- 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 7f952affd640 / http_manager 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 7f95c0fe1640 / http_manager 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 7f95e97fa640 / ms_dispatch 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 7f95eaffd640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 7f95f17fa640 / ms_dispatch 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 7f95f3fff640 / ms_dispatch 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 7f95fdffb640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 7f95fe7fc640 / admin_socket 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 7f960e7fc640 / ms_dispatch 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 7f960effd640 / msgr-worker-2 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 7f960f7fe640 / msgr-worker-1 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 7f960ffff640 / msgr-worker-0 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 7f9614f40640 / io_context_pool 2026-02-24T22:18:16.986 INFO:tasks.workunit.client.0.trial046.stderr: 7f961646aa80 / radosgw-admin 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr: max_recent 500 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr: max_new 1000 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr: log_file /var/log/ceph/ceph-client.admin.47126.log 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr:--- end dump of recent events --- 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr:--- begin dump of recent events --- 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr: -499> 2026-02-24T22:18:16.659+0000 7f960ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr: -498> 2026-02-24T22:18:16.659+0000 7f960ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr: -497> 2026-02-24T22:18:16.659+0000 7f960f7fe640 10 monclient(hunting): get_auth_request con 0x558d22b2d7a0 auth_method 0 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr: -496> 2026-02-24T22:18:16.659+0000 7f960f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr: -495> 2026-02-24T22:18:16.659+0000 7f960f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr: -494> 2026-02-24T22:18:16.659+0000 7f960f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr: -493> 2026-02-24T22:18:16.659+0000 7f960effd640 10 monclient(hunting): handle_auth_done global_id 7347 payload 274 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr: -492> 2026-02-24T22:18:16.659+0000 7f960effd640 10 monclient: _finish_hunting 0 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr: -491> 2026-02-24T22:18:16.659+0000 7f960effd640 1 monclient: found mon.noname-a 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr: -490> 2026-02-24T22:18:16.659+0000 7f960effd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr: -489> 2026-02-24T22:18:16.659+0000 7f95fdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr: -488> 2026-02-24T22:18:16.659+0000 7f95fdffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr: -487> 2026-02-24T22:18:16.659+0000 7f95fdffb640 10 monclient: dump: 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr: -486> 2026-02-24T22:18:16.659+0000 7f95fdffb640 10 monclient: _finish_auth 0 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr: -485> 2026-02-24T22:18:16.659+0000 7f95fdffb640 10 monclient: _check_auth_tickets 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr: -484> 2026-02-24T22:18:16.659+0000 7f95fdffb640 10 monclient: handle_config config(0 keys) 2026-02-24T22:18:16.987 INFO:tasks.workunit.client.0.trial046.stderr: -483> 2026-02-24T22:18:16.659+0000 7f95fdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr: -482> 2026-02-24T22:18:16.659+0000 7f95fdffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr: -481> 2026-02-24T22:18:16.659+0000 7f95fdffb640 10 monclient: dump: 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr: -480> 2026-02-24T22:18:16.659+0000 7f961646aa80 5 monclient: authenticate success, global_id 7347 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr: -479> 2026-02-24T22:18:16.659+0000 7f961646aa80 10 monclient: get_monmap_and_config success 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr: -478> 2026-02-24T22:18:16.659+0000 7f961646aa80 4 set_mon_vals no callback set 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr: -477> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient: shutdown 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr: -476> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 asok(0x558d22aa1f00) unregister_commands rotate-key 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr: -475> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient: build_initial_monmap 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr: -474> 2026-02-24T22:18:16.663+0000 7f961646aa80 1 build_initial for_mkfs: 0 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr: -473> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient: monmap: 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr: -472> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding auth protocol: cephx 2026-02-24T22:18:16.988 INFO:tasks.workunit.client.0.trial046.stderr: -471> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding auth protocol: cephx 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -470> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding auth protocol: cephx 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -469> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding con mode: secure 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -468> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding con mode: crc 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -467> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding con mode: secure 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -466> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding con mode: crc 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -465> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding con mode: secure 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -464> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding con mode: crc 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -463> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding con mode: crc 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -462> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding con mode: secure 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -461> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding con mode: crc 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -460> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding con mode: secure 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -459> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding con mode: crc 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -458> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba8970) adding con mode: secure 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -457> 2026-02-24T22:18:16.663+0000 7f961646aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -456> 2026-02-24T22:18:16.663+0000 7f961646aa80 1 librados: starting msgr at 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -455> 2026-02-24T22:18:16.663+0000 7f961646aa80 1 librados: starting objecter 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -454> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command objecter_requests hook 0x558d22bd3560 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -453> 2026-02-24T22:18:16.663+0000 7f961646aa80 1 librados: setting wanted keys 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -452> 2026-02-24T22:18:16.663+0000 7f961646aa80 1 librados: calling monclient init 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -451> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient: init 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -450> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding auth protocol: cephx 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -449> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding auth protocol: cephx 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -448> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding auth protocol: cephx 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -447> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding con mode: secure 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -446> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding con mode: crc 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -445> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding con mode: secure 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -444> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding con mode: crc 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -443> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding con mode: secure 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -442> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding con mode: crc 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -441> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding con mode: crc 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -440> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding con mode: secure 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -439> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding con mode: crc 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -438> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding con mode: secure 2026-02-24T22:18:16.989 INFO:tasks.workunit.client.0.trial046.stderr: -437> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding con mode: crc 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -436> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 AuthRegistry(0x558d22ba35b0) adding con mode: secure 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -435> 2026-02-24T22:18:16.663+0000 7f961646aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -434> 2026-02-24T22:18:16.663+0000 7f961646aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -433> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command rotate-key hook 0x558d22ba36f8 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -432> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient: _reopen_session rank -1 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -431> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -430> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient(hunting): picked mon.noname-c con 0x558d22b2d7a0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -429> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient(hunting): picked mon.noname-a con 0x558d22b30010 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -428> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient(hunting): picked mon.noname-b con 0x558d22bd7c20 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -427> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -426> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -425> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -424> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient(hunting): _renew_subs 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -423> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient(hunting): authenticate will time out at 1866.768826s 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -422> 2026-02-24T22:18:16.663+0000 7f960effd640 10 monclient(hunting): get_auth_request con 0x558d22b30010 auth_method 0 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -421> 2026-02-24T22:18:16.663+0000 7f960effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -420> 2026-02-24T22:18:16.663+0000 7f960effd640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -419> 2026-02-24T22:18:16.663+0000 7f960effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -418> 2026-02-24T22:18:16.663+0000 7f960f7fe640 10 monclient(hunting): get_auth_request con 0x558d22b2d7a0 auth_method 0 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -417> 2026-02-24T22:18:16.663+0000 7f960f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -416> 2026-02-24T22:18:16.663+0000 7f960f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -415> 2026-02-24T22:18:16.663+0000 7f960f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -414> 2026-02-24T22:18:16.663+0000 7f960effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -413> 2026-02-24T22:18:16.663+0000 7f960effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -412> 2026-02-24T22:18:16.663+0000 7f960effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -411> 2026-02-24T22:18:16.663+0000 7f960f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -410> 2026-02-24T22:18:16.663+0000 7f960f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -409> 2026-02-24T22:18:16.663+0000 7f960f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -408> 2026-02-24T22:18:16.663+0000 7f960ffff640 10 monclient(hunting): get_auth_request con 0x558d22bd7c20 auth_method 0 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -407> 2026-02-24T22:18:16.663+0000 7f960ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -406> 2026-02-24T22:18:16.663+0000 7f960ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -405> 2026-02-24T22:18:16.663+0000 7f960ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -404> 2026-02-24T22:18:16.663+0000 7f960effd640 10 monclient(hunting): handle_auth_done global_id 7350 payload 824 2026-02-24T22:18:16.990 INFO:tasks.workunit.client.0.trial046.stderr: -403> 2026-02-24T22:18:16.663+0000 7f960effd640 10 monclient: _finish_hunting 0 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr: -402> 2026-02-24T22:18:16.663+0000 7f960effd640 1 monclient: found mon.noname-a 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr: -401> 2026-02-24T22:18:16.663+0000 7f960effd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr: -400> 2026-02-24T22:18:16.663+0000 7f95f3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr: -399> 2026-02-24T22:18:16.663+0000 7f95f3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr: -398> 2026-02-24T22:18:16.663+0000 7f95f3fff640 10 monclient: dump: 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr: -397> 2026-02-24T22:18:16.663+0000 7f95f3fff640 10 monclient: _finish_auth 0 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr: -396> 2026-02-24T22:18:16.663+0000 7f95f3fff640 10 monclient: _check_auth_tickets 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr: -395> 2026-02-24T22:18:16.663+0000 7f95f3fff640 10 monclient: handle_config config(0 keys) 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr: -394> 2026-02-24T22:18:16.663+0000 7f95f3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr: -393> 2026-02-24T22:18:16.663+0000 7f95f3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr: -392> 2026-02-24T22:18:16.663+0000 7f95f3fff640 10 monclient: dump: 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr: -391> 2026-02-24T22:18:16.663+0000 7f961646aa80 5 monclient: authenticate success, global_id 7350 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr: -390> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient: _renew_subs 2026-02-24T22:18:16.991 INFO:tasks.workunit.client.0.trial046.stderr: -389> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr: -388> 2026-02-24T22:18:16.663+0000 7f95fdffb640 4 set_mon_vals no callback set 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr: -387> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient: _renew_subs 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr: -386> 2026-02-24T22:18:16.663+0000 7f961646aa80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr: -385> 2026-02-24T22:18:16.663+0000 7f961646aa80 1 librados: init done 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr: -384> 2026-02-24T22:18:16.663+0000 7f95f3fff640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr: -383> 2026-02-24T22:18:16.663+0000 7f95f3fff640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr: -382> 2026-02-24T22:18:16.663+0000 7f95f3fff640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr: -381> 2026-02-24T22:18:16.663+0000 7f960f7fe640 10 monclient: get_auth_request con 0x7f95f40707b0 auth_method 0 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr: -380> 2026-02-24T22:18:16.663+0000 7f960ffff640 10 monclient: get_auth_request con 0x558d22bd1730 auth_method 0 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr: -379> 2026-02-24T22:18:16.663+0000 7f960f7fe640 10 monclient: get_auth_request con 0x558d22bd1b00 auth_method 0 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr: -378> 2026-02-24T22:18:16.667+0000 7f960effd640 10 monclient: get_auth_request con 0x558d22c31ee0 auth_method 0 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr: -377> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient: build_initial_monmap 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr: -376> 2026-02-24T22:18:16.667+0000 7f961646aa80 1 build_initial for_mkfs: 0 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr: -375> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient: monmap: 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr: -374> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding auth protocol: cephx 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr: -373> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding auth protocol: cephx 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr: -372> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding auth protocol: cephx 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr: -371> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding con mode: secure 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr: -370> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding con mode: crc 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr: -369> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding con mode: secure 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr: -368> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding con mode: crc 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr: -367> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding con mode: secure 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr: -366> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding con mode: crc 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr: -365> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding con mode: crc 2026-02-24T22:18:16.992 INFO:tasks.workunit.client.0.trial046.stderr: -364> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding con mode: secure 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -363> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding con mode: crc 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -362> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding con mode: secure 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -361> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding con mode: crc 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -360> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c338d0) adding con mode: secure 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -359> 2026-02-24T22:18:16.667+0000 7f961646aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -358> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command objecter_requests cmddesc objecter_requests hook 0x558d22c421b0 EEXIST 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -357> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient: init 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -356> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding auth protocol: cephx 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -355> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding auth protocol: cephx 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -354> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding auth protocol: cephx 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -353> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding con mode: secure 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -352> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding con mode: crc 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -351> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding con mode: secure 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -350> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding con mode: crc 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -349> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding con mode: secure 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -348> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding con mode: crc 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -347> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding con mode: crc 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -346> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding con mode: secure 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -345> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding con mode: crc 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -344> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding con mode: secure 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -343> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding con mode: crc 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -342> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 AuthRegistry(0x558d22c30f48) adding con mode: secure 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -341> 2026-02-24T22:18:16.667+0000 7f961646aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -340> 2026-02-24T22:18:16.667+0000 7f961646aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -339> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command rotate-key cmddesc rotate-key hook 0x558d22c31090 EEXIST 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -338> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient: _reopen_session rank -1 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -337> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -336> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient(hunting): picked mon.noname-b con 0x558d22d89770 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -335> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient(hunting): picked mon.noname-c con 0x558d22d89b40 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -334> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient(hunting): picked mon.noname-a con 0x558d22db79c0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -333> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -332> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -331> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -330> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient(hunting): _renew_subs 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -329> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient(hunting): authenticate will time out at 1866.775009s 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -328> 2026-02-24T22:18:16.667+0000 7f960effd640 10 monclient(hunting): get_auth_request con 0x558d22d89b40 auth_method 0 2026-02-24T22:18:16.993 INFO:tasks.workunit.client.0.trial046.stderr: -327> 2026-02-24T22:18:16.667+0000 7f960effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -326> 2026-02-24T22:18:16.667+0000 7f960effd640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -325> 2026-02-24T22:18:16.667+0000 7f960effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -324> 2026-02-24T22:18:16.667+0000 7f960ffff640 10 monclient(hunting): get_auth_request con 0x558d22db79c0 auth_method 0 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -323> 2026-02-24T22:18:16.667+0000 7f960ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -322> 2026-02-24T22:18:16.667+0000 7f960ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -321> 2026-02-24T22:18:16.667+0000 7f960ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -320> 2026-02-24T22:18:16.667+0000 7f960effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -319> 2026-02-24T22:18:16.667+0000 7f960effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -318> 2026-02-24T22:18:16.667+0000 7f960effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -317> 2026-02-24T22:18:16.667+0000 7f960ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -316> 2026-02-24T22:18:16.667+0000 7f960ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -315> 2026-02-24T22:18:16.667+0000 7f960ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -314> 2026-02-24T22:18:16.667+0000 7f960f7fe640 10 monclient(hunting): get_auth_request con 0x558d22d89770 auth_method 0 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -313> 2026-02-24T22:18:16.667+0000 7f960f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -312> 2026-02-24T22:18:16.667+0000 7f960f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -311> 2026-02-24T22:18:16.667+0000 7f960f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -310> 2026-02-24T22:18:16.667+0000 7f960effd640 10 monclient(hunting): handle_auth_done global_id 6878 payload 824 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -309> 2026-02-24T22:18:16.667+0000 7f960effd640 10 monclient: _finish_hunting 0 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -308> 2026-02-24T22:18:16.667+0000 7f960effd640 1 monclient: found mon.noname-c 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -307> 2026-02-24T22:18:16.667+0000 7f960effd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -306> 2026-02-24T22:18:16.667+0000 7f95f17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -305> 2026-02-24T22:18:16.667+0000 7f95f17fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -304> 2026-02-24T22:18:16.667+0000 7f95f17fa640 10 monclient: dump: 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -303> 2026-02-24T22:18:16.667+0000 7f95f17fa640 10 monclient: _finish_auth 0 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -302> 2026-02-24T22:18:16.667+0000 7f95f17fa640 10 monclient: _check_auth_tickets 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -301> 2026-02-24T22:18:16.667+0000 7f95f17fa640 10 monclient: handle_config config(0 keys) 2026-02-24T22:18:16.994 INFO:tasks.workunit.client.0.trial046.stderr: -300> 2026-02-24T22:18:16.667+0000 7f95f17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr: -299> 2026-02-24T22:18:16.667+0000 7f95f17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr: -298> 2026-02-24T22:18:16.667+0000 7f95f17fa640 10 monclient: dump: 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr: -297> 2026-02-24T22:18:16.667+0000 7f961646aa80 5 monclient: authenticate success, global_id 6878 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr: -296> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient: _renew_subs 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr: -295> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr: -294> 2026-02-24T22:18:16.667+0000 7f9614f40640 4 set_mon_vals no callback set 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr: -293> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient: _renew_subs 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr: -292> 2026-02-24T22:18:16.667+0000 7f961646aa80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr: -291> 2026-02-24T22:18:16.671+0000 7f95f17fa640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr: -290> 2026-02-24T22:18:16.671+0000 7f95f17fa640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr: -289> 2026-02-24T22:18:16.671+0000 7f95f17fa640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr: -288> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient: get_auth_request con 0x7f95e40706c0 auth_method 0 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr: -287> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: get_monmap_and_config 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr: -286> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: build_initial_monmap 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr: -285> 2026-02-24T22:18:16.671+0000 7f961646aa80 1 build_initial for_mkfs: 0 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr: -284> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: monmap: 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.995 INFO:tasks.workunit.client.0.trial046.stderr: -283> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding auth protocol: cephx 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -282> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding auth protocol: cephx 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -281> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding auth protocol: cephx 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -280> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: secure 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -279> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: crc 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -278> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: secure 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -277> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: crc 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -276> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: secure 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -275> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: crc 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -274> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: crc 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -273> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: secure 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -272> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: crc 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -271> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: secure 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -270> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: crc 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -269> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: secure 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -268> 2026-02-24T22:18:16.671+0000 7f961646aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -267> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: init 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -266> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding auth protocol: cephx 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -265> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding auth protocol: cephx 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -264> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding auth protocol: cephx 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -263> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding con mode: secure 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -262> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding con mode: crc 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -261> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding con mode: secure 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -260> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding con mode: crc 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -259> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding con mode: secure 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -258> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding con mode: crc 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -257> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding con mode: crc 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -256> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding con mode: secure 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -255> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding con mode: crc 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -254> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding con mode: secure 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -253> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding con mode: crc 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -252> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9830) adding con mode: secure 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -251> 2026-02-24T22:18:16.671+0000 7f961646aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -250> 2026-02-24T22:18:16.671+0000 7f961646aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -249> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command rotate-key cmddesc rotate-key hook 0x7ffd7bdc9978 EEXIST 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -248> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: _reopen_session rank -1 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -247> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -246> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): picked mon.noname-c con 0x558d22e43fb0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -245> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): picked mon.noname-b con 0x558d22e47720 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -244> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): picked mon.noname-a con 0x558d22e224e0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:18:16.996 INFO:tasks.workunit.client.0.trial046.stderr: -243> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -242> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -241> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -240> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): _renew_subs 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -239> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): authenticate will time out at 1866.777750s 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -238> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): get_auth_request con 0x558d22e43fb0 auth_method 0 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -237> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -236> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -235> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -234> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -233> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -232> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -231> 2026-02-24T22:18:16.671+0000 7f960ffff640 10 monclient(hunting): get_auth_request con 0x558d22e224e0 auth_method 0 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -230> 2026-02-24T22:18:16.671+0000 7f960ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -229> 2026-02-24T22:18:16.671+0000 7f960ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -228> 2026-02-24T22:18:16.671+0000 7f960ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -227> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): handle_auth_done global_id 6881 payload 274 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -226> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient: _finish_hunting 0 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -225> 2026-02-24T22:18:16.671+0000 7f960f7fe640 1 monclient: found mon.noname-c 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -224> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -223> 2026-02-24T22:18:16.671+0000 7f95eaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -222> 2026-02-24T22:18:16.671+0000 7f95eaffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -221> 2026-02-24T22:18:16.671+0000 7f95eaffd640 10 monclient: dump: 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -220> 2026-02-24T22:18:16.671+0000 7f95eaffd640 10 monclient: _finish_auth 0 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -219> 2026-02-24T22:18:16.671+0000 7f95eaffd640 10 monclient: _check_auth_tickets 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -218> 2026-02-24T22:18:16.671+0000 7f95eaffd640 10 monclient: handle_config config(0 keys) 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -217> 2026-02-24T22:18:16.671+0000 7f95eaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -216> 2026-02-24T22:18:16.671+0000 7f95eaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:18:16.997 INFO:tasks.workunit.client.0.trial046.stderr: -215> 2026-02-24T22:18:16.671+0000 7f95eaffd640 10 monclient: dump: 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr: -214> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 monclient: authenticate success, global_id 6881 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr: -213> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: get_monmap_and_config success 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr: -212> 2026-02-24T22:18:16.671+0000 7f961646aa80 4 set_mon_vals no callback set 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr: -211> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: shutdown 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr: -210> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: build_initial_monmap 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr: -209> 2026-02-24T22:18:16.671+0000 7f961646aa80 1 build_initial for_mkfs: 0 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr: -208> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: monmap: 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr: -207> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding auth protocol: cephx 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr: -206> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding auth protocol: cephx 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr: -205> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding auth protocol: cephx 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr: -204> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: secure 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr: -203> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: crc 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr: -202> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: secure 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr: -201> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: crc 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr: -200> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: secure 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr: -199> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: crc 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr: -198> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: crc 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr: -197> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: secure 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr: -196> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: crc 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr: -195> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: secure 2026-02-24T22:18:16.998 INFO:tasks.workunit.client.0.trial046.stderr: -194> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: crc 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -193> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1cf80) adding con mode: secure 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -192> 2026-02-24T22:18:16.671+0000 7f961646aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -191> 2026-02-24T22:18:16.671+0000 7f961646aa80 1 librados: starting msgr at 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -190> 2026-02-24T22:18:16.671+0000 7f961646aa80 1 librados: starting objecter 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -189> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command objecter_requests cmddesc objecter_requests hook 0x558d22e20aa0 EEXIST 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -188> 2026-02-24T22:18:16.671+0000 7f961646aa80 1 librados: setting wanted keys 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -187> 2026-02-24T22:18:16.671+0000 7f961646aa80 1 librados: calling monclient init 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -186> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: init 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -185> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding auth protocol: cephx 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -184> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding auth protocol: cephx 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -183> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding auth protocol: cephx 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -182> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding con mode: secure 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -181> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding con mode: crc 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -180> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding con mode: secure 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -179> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding con mode: crc 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -178> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding con mode: secure 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -177> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding con mode: crc 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -176> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding con mode: crc 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -175> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding con mode: secure 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -174> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding con mode: crc 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -173> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding con mode: secure 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -172> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding con mode: crc 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -171> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 AuthRegistry(0x558d22e1bb20) adding con mode: secure 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -170> 2026-02-24T22:18:16.671+0000 7f961646aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -169> 2026-02-24T22:18:16.671+0000 7f961646aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -168> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command rotate-key cmddesc rotate-key hook 0x558d22e1bc68 EEXIST 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -167> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: _reopen_session rank -1 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -166> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -165> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): picked mon.noname-a con 0x558d22e43fb0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -164> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): picked mon.noname-c con 0x558d22e47720 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -163> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): picked mon.noname-b con 0x558d22e4fd40 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -162> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -161> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -160> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -159> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): _renew_subs 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -158> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient(hunting): authenticate will time out at 1866.779018s 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -157> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): get_auth_request con 0x558d22e43fb0 auth_method 0 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -156> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -155> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -154> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:16.999 INFO:tasks.workunit.client.0.trial046.stderr: -153> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr: -152> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr: -151> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr: -150> 2026-02-24T22:18:16.671+0000 7f960effd640 10 monclient(hunting): get_auth_request con 0x558d22e47720 auth_method 0 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr: -149> 2026-02-24T22:18:16.671+0000 7f960effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr: -148> 2026-02-24T22:18:16.671+0000 7f960effd640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr: -147> 2026-02-24T22:18:16.671+0000 7f960effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr: -146> 2026-02-24T22:18:16.671+0000 7f960effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr: -145> 2026-02-24T22:18:16.671+0000 7f960effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr: -144> 2026-02-24T22:18:16.671+0000 7f960effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr: -143> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient(hunting): handle_auth_done global_id 7359 payload 824 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr: -142> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient: _finish_hunting 0 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr: -141> 2026-02-24T22:18:16.671+0000 7f960f7fe640 1 monclient: found mon.noname-a 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr: -140> 2026-02-24T22:18:16.671+0000 7f960f7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr: -139> 2026-02-24T22:18:16.671+0000 7f95e97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr: -138> 2026-02-24T22:18:16.671+0000 7f95e97fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr: -137> 2026-02-24T22:18:16.671+0000 7f95e97fa640 10 monclient: dump: 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr: -136> 2026-02-24T22:18:16.671+0000 7f95e97fa640 10 monclient: _finish_auth 0 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr: -135> 2026-02-24T22:18:16.671+0000 7f95e97fa640 10 monclient: _check_auth_tickets 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr: -134> 2026-02-24T22:18:16.671+0000 7f95e97fa640 10 monclient: handle_config config(0 keys) 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr: -133> 2026-02-24T22:18:16.671+0000 7f95e97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr: -132> 2026-02-24T22:18:16.671+0000 7f95e97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr: -131> 2026-02-24T22:18:16.671+0000 7f95e97fa640 10 monclient: dump: 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:17.000 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -130> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 monclient: authenticate success, global_id 7359 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -129> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: _renew_subs 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -128> 2026-02-24T22:18:16.671+0000 7f95eaffd640 4 set_mon_vals no callback set 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -127> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -126> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: _renew_subs 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -125> 2026-02-24T22:18:16.671+0000 7f961646aa80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -124> 2026-02-24T22:18:16.671+0000 7f961646aa80 1 librados: init done 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -123> 2026-02-24T22:18:16.671+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command cr dump hook 0x558d22e44cf8 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -122> 2026-02-24T22:18:16.675+0000 7f961646aa80 4 Realm: () 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -121> 2026-02-24T22:18:16.675+0000 7f961646aa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -120> 2026-02-24T22:18:16.675+0000 7f961646aa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -119> 2026-02-24T22:18:16.675+0000 7f961646aa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -118> 2026-02-24T22:18:16.675+0000 7f961646aa80 20 zonegroup default 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -117> 2026-02-24T22:18:16.675+0000 7f960effd640 10 monclient: get_auth_request con 0x558d22ee5650 auth_method 0 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -116> 2026-02-24T22:18:16.675+0000 7f961646aa80 20 rados->read ofs=0 len=0 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -115> 2026-02-24T22:18:16.675+0000 7f95e97fa640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -114> 2026-02-24T22:18:16.675+0000 7f95e97fa640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -113> 2026-02-24T22:18:16.675+0000 7f95e97fa640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -112> 2026-02-24T22:18:16.675+0000 7f960ffff640 10 monclient: get_auth_request con 0x7f95d40707b0 auth_method 0 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -111> 2026-02-24T22:18:16.675+0000 7f960f7fe640 10 monclient: get_auth_request con 0x558d22f08970 auth_method 0 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -110> 2026-02-24T22:18:16.675+0000 7f961646aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -109> 2026-02-24T22:18:16.675+0000 7f961646aa80 20 rados->read ofs=0 len=0 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -108> 2026-02-24T22:18:16.675+0000 7f960effd640 10 monclient: get_auth_request con 0x558d22f2a050 auth_method 0 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -107> 2026-02-24T22:18:16.675+0000 7f961646aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -106> 2026-02-24T22:18:16.675+0000 7f961646aa80 20 started sync module instance, tier type = 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -105> 2026-02-24T22:18:16.675+0000 7f961646aa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -104> 2026-02-24T22:18:16.675+0000 7f960ffff640 10 monclient: get_auth_request con 0x558d22f8b4d0 auth_method 0 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -103> 2026-02-24T22:18:16.675+0000 7f960ffff640 10 monclient: get_auth_request con 0x558d23092d70 auth_method 0 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -102> 2026-02-24T22:18:16.675+0000 7f960f7fe640 10 monclient: get_auth_request con 0x558d22fedcb0 auth_method 0 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -101> 2026-02-24T22:18:16.675+0000 7f960effd640 10 monclient: get_auth_request con 0x558d23050c30 auth_method 0 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -100> 2026-02-24T22:18:16.675+0000 7f960f7fe640 10 monclient: get_auth_request con 0x558d230f5d70 auth_method 0 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -99> 2026-02-24T22:18:16.675+0000 7f960ffff640 0 -- 10.20.193.46:0/3578363307 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x558d22f8b4d0 msgr2=0x558d22fab8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -98> 2026-02-24T22:18:16.675+0000 7f95e97fa640 1 client.7359.objecter ms_handle_reset 0x558d22f8b4d0 session 0x558d22f07f50 osd.0 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -97> 2026-02-24T22:18:16.679+0000 7f95e97fa640 10 monclient: _renew_subs 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -96> 2026-02-24T22:18:16.679+0000 7f95e97fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -95> 2026-02-24T22:18:16.679+0000 7f960ffff640 10 monclient: get_auth_request con 0x7f95d409d4e0 auth_method 0 2026-02-24T22:18:17.001 INFO:tasks.workunit.client.0.trial046.stderr: -94> 2026-02-24T22:18:16.679+0000 7f961646aa80 20 add_watcher() i=6 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -93> 2026-02-24T22:18:16.679+0000 7f961646aa80 20 add_watcher() i=3 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -92> 2026-02-24T22:18:16.679+0000 7f961646aa80 20 add_watcher() i=2 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -91> 2026-02-24T22:18:16.679+0000 7f961646aa80 20 add_watcher() i=0 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -90> 2026-02-24T22:18:16.679+0000 7f961646aa80 20 add_watcher() i=1 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -89> 2026-02-24T22:18:16.679+0000 7f961646aa80 20 add_watcher() i=4 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -88> 2026-02-24T22:18:16.679+0000 7f961646aa80 20 add_watcher() i=5 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -87> 2026-02-24T22:18:16.679+0000 7f961646aa80 20 add_watcher() i=7 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -86> 2026-02-24T22:18:16.679+0000 7f961646aa80 2 all 8 watchers are set, enabling cache 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -85> 2026-02-24T22:18:16.679+0000 7f960effd640 10 monclient: get_auth_request con 0x7f9534008af0 auth_method 0 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -84> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding auth protocol: cephx 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -83> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding auth protocol: cephx 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -82> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding auth protocol: cephx 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -81> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding con mode: secure 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -80> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding con mode: crc 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -79> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding con mode: secure 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -78> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding con mode: crc 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -77> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding con mode: secure 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -76> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding con mode: crc 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -75> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding con mode: crc 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -74> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding con mode: secure 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -73> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding con mode: crc 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -72> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding con mode: secure 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -71> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding con mode: crc 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -70> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 AuthRegistry(0x7ffd7bdc9ca0) adding con mode: secure 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -69> 2026-02-24T22:18:16.679+0000 7f961646aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -68> 2026-02-24T22:18:16.679+0000 7f961646aa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -67> 2026-02-24T22:18:16.679+0000 7f961646aa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -66> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command cache list hook 0x558d22e6a290 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -65> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command cache inspect hook 0x558d22e6a290 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -64> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command cache erase hook 0x558d22e6a290 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -63> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command cache zap hook 0x558d22e6a290 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -62> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 note: GC not initialized 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -61> 2026-02-24T22:18:16.679+0000 7f95c0fe1640 20 reqs_thread_entry: start 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -60> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command sync trace show hook 0x7f9600026040 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -59> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command sync trace history hook 0x7f9600026040 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -58> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command sync trace active hook 0x7f9600026040 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -57> 2026-02-24T22:18:16.679+0000 7f961646aa80 5 asok(0x558d22aa1f00) register_command sync trace active_short hook 0x7f9600026040 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -56> 2026-02-24T22:18:16.679+0000 7f960ffff640 10 monclient: get_auth_request con 0x558d22fd6bb0 auth_method 0 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -55> 2026-02-24T22:18:16.679+0000 7f960ffff640 0 -- 10.20.193.46:0/3295766240 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x558d22fd6bb0 msgr2=0x558d22fd6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -54> 2026-02-24T22:18:16.883+0000 7f960ffff640 10 monclient: get_auth_request con 0x558d22fd6bb0 auth_method 0 2026-02-24T22:18:17.002 INFO:tasks.workunit.client.0.trial046.stderr: -53> 2026-02-24T22:18:16.883+0000 7f960f7fe640 10 monclient: get_auth_request con 0x558d22f8ebb0 auth_method 0 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -52> 2026-02-24T22:18:16.883+0000 7f960effd640 10 monclient: get_auth_request con 0x558d22f90a30 auth_method 0 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -51> 2026-02-24T22:18:16.887+0000 7f960ffff640 10 monclient: get_auth_request con 0x558d22fd84a0 auth_method 0 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -50> 2026-02-24T22:18:16.891+0000 7f960f7fe640 10 monclient: get_auth_request con 0x558d22fd6750 auth_method 0 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -49> 2026-02-24T22:18:16.895+0000 7f960effd640 10 monclient: get_auth_request con 0x558d22fd5390 auth_method 0 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -48> 2026-02-24T22:18:16.915+0000 7f961646aa80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -47> 2026-02-24T22:18:16.915+0000 7f961646aa80 20 Filter name: none 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -46> 2026-02-24T22:18:16.915+0000 7f952affd640 20 reqs_thread_entry: start 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -45> 2026-02-24T22:18:16.915+0000 7f961646aa80 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -44> 2026-02-24T22:18:16.915+0000 7f961646aa80 20 rados->read ofs=0 len=0 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -43> 2026-02-24T22:18:16.919+0000 7f961646aa80 20 rados_obj.operate() r=0 bl.length=217 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -42> 2026-02-24T22:18:16.919+0000 7f961646aa80 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -41> 2026-02-24T22:18:16.919+0000 7f961646aa80 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -40> 2026-02-24T22:18:16.919+0000 7f961646aa80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 : miss 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -39> 2026-02-24T22:18:16.919+0000 7f961646aa80 20 rados->read ofs=0 len=0 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -38> 2026-02-24T22:18:16.919+0000 7f961646aa80 20 rados_obj.operate() r=0 bl.length=447 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -37> 2026-02-24T22:18:16.919+0000 7f961646aa80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 info.flags=0x17 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -36> 2026-02-24T22:18:16.919+0000 7f961646aa80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 to cache LRU end 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -35> 2026-02-24T22:18:16.919+0000 7f961646aa80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -34> 2026-02-24T22:18:16.919+0000 7f961646aa80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -33> 2026-02-24T22:18:16.919+0000 7f961646aa80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -32> 2026-02-24T22:18:16.923+0000 7f961646aa80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 info.flags=0x17 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -31> 2026-02-24T22:18:16.923+0000 7f961646aa80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 to cache LRU end 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -30> 2026-02-24T22:18:16.923+0000 7f961646aa80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -29> 2026-02-24T22:18:16.923+0000 7f961646aa80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530, ofs0, ns] 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -28> 2026-02-24T22:18:16.923+0000 7f95eaffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961356 cookie 94064666145152 notifier 7359 bl.length()=1017 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -27> 2026-02-24T22:18:16.923+0000 7f95eaffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 info.flags=0x17 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -26> 2026-02-24T22:18:16.923+0000 7f95eaffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 to cache LRU end 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -25> 2026-02-24T22:18:16.923+0000 7f95eaffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -24> 2026-02-24T22:18:16.923+0000 7f961646aa80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): orig_sources= new_sources= 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -23> 2026-02-24T22:18:16.923+0000 7f961646aa80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): potential sources added=[] removed=[] 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -22> 2026-02-24T22:18:16.923+0000 7f961646aa80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): orig_dests= new_dests= 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -21> 2026-02-24T22:18:16.923+0000 7f961646aa80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): potential dests added=[] removed=[] 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -20> 2026-02-24T22:18:16.923+0000 7f961646aa80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530.1.0 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -19> 2026-02-24T22:18:16.923+0000 7f961646aa80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530.1.0 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -18> 2026-02-24T22:18:16.923+0000 7f961646aa80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530.1.1 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -17> 2026-02-24T22:18:16.923+0000 7f961646aa80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530.1.1 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -16> 2026-02-24T22:18:16.923+0000 7f961646aa80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530.0 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -15> 2026-02-24T22:18:16.927+0000 7f961646aa80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 : hit (requested=0x17, cached=0x17) 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -14> 2026-02-24T22:18:16.927+0000 7f961646aa80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -13> 2026-02-24T22:18:16.927+0000 7f961646aa80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 info.flags=0x17 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -12> 2026-02-24T22:18:16.927+0000 7f961646aa80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 to cache LRU end 2026-02-24T22:18:17.003 INFO:tasks.workunit.client.0.trial046.stderr: -11> 2026-02-24T22:18:16.927+0000 7f961646aa80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: -10> 2026-02-24T22:18:16.927+0000 7f961646aa80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530, ofs0, ns] 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: -9> 2026-02-24T22:18:16.927+0000 7f95eaffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961357 cookie 94064666145152 notifier 7359 bl.length()=1017 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: -8> 2026-02-24T22:18:16.927+0000 7f95eaffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 info.flags=0x17 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: -7> 2026-02-24T22:18:16.927+0000 7f95eaffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 to cache LRU end 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: -6> 2026-02-24T22:18:16.927+0000 7f95eaffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: -5> 2026-02-24T22:18:16.927+0000 7f961646aa80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): orig_sources= new_sources= 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: -4> 2026-02-24T22:18:16.927+0000 7f961646aa80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): potential sources added=[] removed=[] 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: -3> 2026-02-24T22:18:16.927+0000 7f961646aa80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): orig_dests= new_dests= 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: -2> 2026-02-24T22:18:16.927+0000 7f961646aa80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): potential dests added=[] removed=[] 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: -1> 2026-02-24T22:18:16.927+0000 7f961646aa80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f961646aa80 time 2026-02-24T22:18:16.929305+0000 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f96198aceb1] 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x558d12970b9f] 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::do_reshard(rgw::bucket_index_layout_generation const&, rgw::bucket_index_layout_generation const&, int, bool, bool, std::ostream*, ceph::Formatter*, FaultInjector > >&, DoutPrefixProvider const*, optional_yield)+0x20d6) [0x558d130db576] 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xe6b) [0x558d130dcf6b] 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: 5: main() 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f9618bacd90] 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: 7: __libc_start_main() 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: 8: _start() 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: 0> 2026-02-24T22:18:16.931+0000 7f961646aa80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f961646aa80 thread_name:radosgw-admin 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f9618bc5520] 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f96198acf10] 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x558d12970b9f] 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::do_reshard(rgw::bucket_index_layout_generation const&, rgw::bucket_index_layout_generation const&, int, bool, bool, std::ostream*, ceph::Formatter*, FaultInjector > >&, DoutPrefixProvider const*, optional_yield)+0x20d6) [0x558d130db576] 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xe6b) [0x558d130dcf6b] 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: 9: main() 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f9618bacd90] 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: 11: __libc_start_main() 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: 12: _start() 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:17.004 INFO:tasks.workunit.client.0.trial046.stderr:--- logging levels --- 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 none 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 lockdep 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 context 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 crush 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_balancer 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_locker 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log_expire 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_migrator 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 3/ 5 mds_quiesce 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 buffer 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 timer 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 filer 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 striper 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 objecter 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rados 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_mirror 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_replay 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_pwl 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 journaler 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objectcacher 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 immutable_obj_cache 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 client 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 osd 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 optracker 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objclass 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 filestore 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 journal 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 ms 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mon 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 0/10 monc 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 paxos 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 tp 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 auth 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 crypto 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 finisher 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 reserver 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 heartbeatmap 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 perfcounter 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_sync 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_datacache 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_access 2026-02-24T22:18:17.005 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dbstore 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_flight 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_lifecycle 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_restore 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw_notification 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_bucket_logging 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 javaclient 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 asok 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 throttle 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 refs 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 compressor 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore_compression 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluefs 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 bdev 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 kstore 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 4/ 5 rocksdb 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 fuse 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 2/ 5 mgr 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mgrc 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 dpdk 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 eventtrace 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 prioritycache 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 test 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephfs_mirror 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephsqlite 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 crimson_interrupt 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_onode 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_odata 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_omap 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_tm 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_t 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cleaner 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_epm 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_lba 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_fixedkv_tree 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cache 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_journal 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_device 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_backref 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 alienstore 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mclock 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dedup 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cyanstore 2026-02-24T22:18:17.006 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 ceph_exporter 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 memstore 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 trace 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 ceph_dedup 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr: -2/-2 (syslog threshold) 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr: 99/99 (stderr threshold) 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr:--- pthread ID / name mapping for recent threads --- 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr: 7f952affd640 / http_manager 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr: 7f95c0fe1640 / http_manager 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr: 7f95e97fa640 / ms_dispatch 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr: 7f95eaffd640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr: 7f95f17fa640 / ms_dispatch 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr: 7f95f3fff640 / ms_dispatch 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr: 7f95fdffb640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr: 7f95fe7fc640 / admin_socket 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr: 7f960e7fc640 / ms_dispatch 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr: 7f960effd640 / msgr-worker-2 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr: 7f960f7fe640 / msgr-worker-1 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr: 7f960ffff640 / msgr-worker-0 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr: 7f9614f40640 / io_context_pool 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr: 7f961646aa80 / radosgw-admin 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr: max_recent 500 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr: max_new 1000 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr: log_file /var/lib/ceph/crash/2026-02-24T22:18:16.931703Z_143d6446-de4f-4292-baca-718b78520527/log 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr:--- end dump of recent events --- 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr:Aborted 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket abort-at-block-writes 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:18:17.007 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:18:17.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.967+0000 7fe34a39ea80 4 Realm: () 2026-02-24T22:18:17.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.967+0000 7fe34a39ea80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:17.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.967+0000 7fe34a39ea80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:17.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.967+0000 7fe34a39ea80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:17.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.967+0000 7fe34a39ea80 20 zonegroup default 2026-02-24T22:18:17.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.967+0000 7fe34a39ea80 20 rados->read ofs=0 len=0 2026-02-24T22:18:17.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.971+0000 7fe34a39ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:17.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.971+0000 7fe34a39ea80 20 rados->read ofs=0 len=0 2026-02-24T22:18:17.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.971+0000 7fe34a39ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:17.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.971+0000 7fe34a39ea80 20 started sync module instance, tier type = 2026-02-24T22:18:17.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.971+0000 7fe34a39ea80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:17.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.971+0000 7fe34a39ea80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:17.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.971+0000 7fe34a39ea80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:17.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.971+0000 7fe34a39ea80 5 note: GC not initialized 2026-02-24T22:18:17.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.971+0000 7fe2f57e2640 20 reqs_thread_entry: start 2026-02-24T22:18:17.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:16.975+0000 7fe343fff640 0 -- 10.20.193.46:0/1544704358 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x56524c8d5d10 msgr2=0x56524c8c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:18:17.182 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.187+0000 7fe3437fe640 0 -- 10.20.193.46:0/1544704358 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x56524c8d1790 msgr2=0x56524c8cc500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:17.208 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.211+0000 7fe34a39ea80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:17.208 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.211+0000 7fe34a39ea80 20 Filter name: none 2026-02-24T22:18:17.209 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.211+0000 7fe25f7fe640 20 reqs_thread_entry: start 2026-02-24T22:18:17.209 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.211+0000 7fe34a39ea80 20 rados->read ofs=0 len=0 2026-02-24T22:18:17.209 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.211+0000 7fe34a39ea80 20 rados_obj.operate() r=0 bl.length=217 2026-02-24T22:18:17.209 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.211+0000 7fe34a39ea80 20 rados->read ofs=0 len=0 2026-02-24T22:18:17.210 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.215+0000 7fe34a39ea80 20 rados_obj.operate() r=0 bl.length=483 2026-02-24T22:18:17.210 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.215+0000 7fe34a39ea80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:17.210 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.215+0000 7fe34a39ea80 20 rados->read ofs=0 len=0 2026-02-24T22:18:17.210 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.215+0000 7fe34a39ea80 20 rados_obj.operate() r=0 bl.length=217 2026-02-24T22:18:17.210 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.215+0000 7fe34a39ea80 20 rados->read ofs=0 len=0 2026-02-24T22:18:17.210 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.215+0000 7fe34a39ea80 20 rados_obj.operate() r=0 bl.length=483 2026-02-24T22:18:17.210 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.215+0000 7fe34a39ea80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:17.221 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:17.221 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:18:17.221 INFO:tasks.workunit.client.0.trial046.stderr:bucket: abort-at-block-writes id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:18:17.221 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:17.222 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:17.222 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:17.227 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:17.228 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:17.270 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket abort-at-block-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2026-02-24T22:18:17.286 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:18:17.286 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:18:17.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.307+0000 7f7517ff1a80 4 Realm: () 2026-02-24T22:18:17.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.307+0000 7f7517ff1a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:17.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.307+0000 7f7517ff1a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:17.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.307+0000 7f7517ff1a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:17.303 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.307+0000 7f7517ff1a80 20 zonegroup default 2026-02-24T22:18:17.303 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.307+0000 7f7517ff1a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:17.304 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.307+0000 7f7517ff1a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:17.304 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.307+0000 7f7517ff1a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:17.304 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.307+0000 7f7517ff1a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:17.304 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.307+0000 7f7517ff1a80 20 started sync module instance, tier type = 2026-02-24T22:18:17.304 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.307+0000 7f7517ff1a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:17.305 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.311+0000 7f75162c6640 0 -- 10.20.193.46:0/1741028523 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x557a4364bf90 msgr2=0x557a4366c410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:17.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.311+0000 7f7517ff1a80 20 add_watcher() i=1 2026-02-24T22:18:17.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.311+0000 7f7517ff1a80 20 add_watcher() i=3 2026-02-24T22:18:17.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.311+0000 7f7517ff1a80 20 add_watcher() i=0 2026-02-24T22:18:17.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.311+0000 7f7517ff1a80 20 add_watcher() i=6 2026-02-24T22:18:17.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.311+0000 7f7517ff1a80 20 add_watcher() i=5 2026-02-24T22:18:17.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.311+0000 7f7517ff1a80 20 add_watcher() i=2 2026-02-24T22:18:17.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.311+0000 7f7517ff1a80 20 add_watcher() i=4 2026-02-24T22:18:17.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.311+0000 7f7517ff1a80 20 add_watcher() i=7 2026-02-24T22:18:17.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.311+0000 7f7517ff1a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:18:17.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.311+0000 7f7517ff1a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:17.309 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.311+0000 7f7517ff1a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:17.309 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.311+0000 7f7517ff1a80 5 note: GC not initialized 2026-02-24T22:18:17.309 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.311+0000 7f74467fc640 20 reqs_thread_entry: start 2026-02-24T22:18:17.309 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.311+0000 7f75162c6640 0 -- 10.20.193.46:0/3819046228 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x557a43662160 msgr2=0x557a43662530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:18:17.547 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.551+0000 7f7517ff1a80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:17.547 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.551+0000 7f7517ff1a80 20 Filter name: none 2026-02-24T22:18:17.548 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.551+0000 7f7444ff9640 20 reqs_thread_entry: start 2026-02-24T22:18:17.548 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.551+0000 7f7517ff1a80 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2026-02-24T22:18:17.548 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.551+0000 7f7517ff1a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:17.548 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.551+0000 7f7517ff1a80 20 rados_obj.operate() r=0 bl.length=217 2026-02-24T22:18:17.548 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.551+0000 7f7517ff1a80 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2026-02-24T22:18:17.548 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.551+0000 7f7517ff1a80 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2026-02-24T22:18:17.548 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.551+0000 7f7517ff1a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 : miss 2026-02-24T22:18:17.548 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.551+0000 7f7517ff1a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:17.548 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.551+0000 7f7517ff1a80 20 rados_obj.operate() r=0 bl.length=483 2026-02-24T22:18:17.548 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.551+0000 7f7517ff1a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 info.flags=0x17 2026-02-24T22:18:17.548 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.551+0000 7f7517ff1a80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 to cache LRU end 2026-02-24T22:18:17.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.551+0000 7f7517ff1a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:18:17.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.551+0000 7f7517ff1a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:17.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.551+0000 7f7517ff1a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 2026-02-24T22:18:17.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.551+0000 7f7517ff1a80 0 INFO: RGWReshardLock::lock found lock on abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 to be held by another RGW process; skipping for now 2026-02-24T22:18:17.551 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.555+0000 7f7517ff1a80 20 remove_watcher() i=3 2026-02-24T22:18:17.552 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.555+0000 7f7517ff1a80 2 removed watcher, disabling cache 2026-02-24T22:18:17.552 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.555+0000 7f7517ff1a80 20 remove_watcher() i=2 2026-02-24T22:18:17.552 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.555+0000 7f7517ff1a80 20 remove_watcher() i=4 2026-02-24T22:18:17.552 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.555+0000 7f7517ff1a80 20 remove_watcher() i=6 2026-02-24T22:18:17.552 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.555+0000 7f7517ff1a80 20 remove_watcher() i=7 2026-02-24T22:18:17.552 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.555+0000 7f7517ff1a80 20 remove_watcher() i=1 2026-02-24T22:18:17.552 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.555+0000 7f7517ff1a80 20 remove_watcher() i=5 2026-02-24T22:18:17.552 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:17.555+0000 7f7517ff1a80 20 remove_watcher() i=0 2026-02-24T22:18:17.558 INFO:tasks.workunit.client.0.trial046.stderr:waiting 30 seconds for reshard lock to expire... 2026-02-24T22:18:47.596 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket abort-at-block-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2026-02-24T22:18:47.601 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:18:47.602 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:18:47.619 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:47.622+0000 7f3f4e983a80 4 Realm: () 2026-02-24T22:18:47.619 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:47.622+0000 7f3f4e983a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:47.619 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:47.622+0000 7f3f4e983a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:47.619 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:47.622+0000 7f3f4e983a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:47.619 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:47.622+0000 7f3f4e983a80 20 zonegroup default 2026-02-24T22:18:47.619 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:47.622+0000 7f3f4e983a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:47.620 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:47.622+0000 7f3f4e983a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:47.620 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:47.622+0000 7f3f4e983a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:47.621 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:47.626+0000 7f3f4e983a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:47.621 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:47.626+0000 7f3f4e983a80 20 started sync module instance, tier type = 2026-02-24T22:18:47.621 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:47.626+0000 7f3f4e983a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:47.622 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:47.626+0000 7f3f3ffff640 0 -- 10.20.193.46:0/1829183339 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x559d506332b0 msgr2=0x559d50653730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:47.622 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:47.626+0000 7f3f4e983a80 20 add_watcher() i=1 2026-02-24T22:18:47.623 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:47.626+0000 7f3f4e983a80 20 add_watcher() i=3 2026-02-24T22:18:47.623 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:47.626+0000 7f3f4e983a80 20 add_watcher() i=0 2026-02-24T22:18:47.623 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:47.626+0000 7f3f4e983a80 20 add_watcher() i=6 2026-02-24T22:18:47.623 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:47.626+0000 7f3f4e983a80 20 add_watcher() i=5 2026-02-24T22:18:47.624 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:47.626+0000 7f3f4e983a80 20 add_watcher() i=2 2026-02-24T22:18:47.624 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:47.626+0000 7f3f4e983a80 20 add_watcher() i=7 2026-02-24T22:18:47.827 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:47.830+0000 7f3f4e983a80 20 add_watcher() i=4 2026-02-24T22:18:47.827 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:47.830+0000 7f3f4e983a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:18:47.829 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:47.834+0000 7f3f3ffff640 0 -- 10.20.193.46:0/3930982925 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f3e6c008af0 msgr2=0x7f3e6c028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:47.829 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:47.834+0000 7f3f4e983a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:47.830 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:47.834+0000 7f3f4e983a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:47.830 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:47.834+0000 7f3f4e983a80 5 note: GC not initialized 2026-02-24T22:18:47.830 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:47.834+0000 7f3ef97e2640 20 reqs_thread_entry: start 2026-02-24T22:18:48.073 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.074+0000 7f3f4e983a80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:48.073 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.074+0000 7f3f4e983a80 20 Filter name: none 2026-02-24T22:18:48.073 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.074+0000 7f3e637fe640 20 reqs_thread_entry: start 2026-02-24T22:18:48.073 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.074+0000 7f3f4e983a80 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2026-02-24T22:18:48.073 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.074+0000 7f3f4e983a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:48.073 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.074+0000 7f3f4e983a80 20 rados_obj.operate() r=0 bl.length=217 2026-02-24T22:18:48.073 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.074+0000 7f3f4e983a80 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2026-02-24T22:18:48.073 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.074+0000 7f3f4e983a80 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2026-02-24T22:18:48.073 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.074+0000 7f3f4e983a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 : miss 2026-02-24T22:18:48.073 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.074+0000 7f3f4e983a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:48.074 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.078+0000 7f3f4e983a80 20 rados_obj.operate() r=0 bl.length=483 2026-02-24T22:18:48.074 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.078+0000 7f3f4e983a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 info.flags=0x17 2026-02-24T22:18:48.074 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.078+0000 7f3f4e983a80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 to cache LRU end 2026-02-24T22:18:48.074 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.078+0000 7f3f4e983a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:18:48.075 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.078+0000 7f3f4e983a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:48.075 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.078+0000 7f3f4e983a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 2026-02-24T22:18:48.075 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.078+0000 7f3f4e983a80 10 init_target_layout removing existing target index objects from a previous reshard attempt 2026-02-24T22:18:48.079 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.082+0000 7f3f4e983a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 info.flags=0x17 2026-02-24T22:18:48.079 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.082+0000 7f3f4e983a80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 to cache LRU end 2026-02-24T22:18:48.079 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.082+0000 7f3f4e983a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:48.079 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.082+0000 7f3f4e983a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530, ofs0, ns] 2026-02-24T22:18:48.079 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.082+0000 7f3f1affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961358 cookie 94134147722880 notifier 6935 bl.length()=1017 2026-02-24T22:18:48.079 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.082+0000 7f3f1affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 info.flags=0x17 2026-02-24T22:18:48.079 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.082+0000 7f3f1affd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 to cache LRU end 2026-02-24T22:18:48.079 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.082+0000 7f3f1affd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:48.079 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.082+0000 7f3f4e983a80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): orig_sources= new_sources= 2026-02-24T22:18:48.079 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.082+0000 7f3f4e983a80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): potential sources added=[] removed=[] 2026-02-24T22:18:48.080 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.082+0000 7f3f4e983a80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): orig_dests= new_dests= 2026-02-24T22:18:48.080 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.082+0000 7f3f4e983a80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): potential dests added=[] removed=[] 2026-02-24T22:18:48.080 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.082+0000 7f3f1b7fe640 20 trim reshard logs: write on '.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530.0' needs retry: Success 2026-02-24T22:18:48.081 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.086+0000 7f3f4e983a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530.2.0 2026-02-24T22:18:48.081 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.086+0000 7f3f4e983a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530.2.0 2026-02-24T22:18:48.081 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.086+0000 7f3f4e983a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530.2.1 2026-02-24T22:18:48.081 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.086+0000 7f3f4e983a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530.2.1 2026-02-24T22:18:48.081 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.086+0000 7f3f4e983a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530.0 2026-02-24T22:18:48.083 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.086+0000 7f3f4e983a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 : hit (requested=0x17, cached=0x17) 2026-02-24T22:18:48.083 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.086+0000 7f3f4e983a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 2026-02-24T22:18:48.084 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.086+0000 7f3f4e983a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 info.flags=0x17 2026-02-24T22:18:48.084 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.086+0000 7f3f4e983a80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 to cache LRU end 2026-02-24T22:18:48.084 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.086+0000 7f3f4e983a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:48.084 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.086+0000 7f3f4e983a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530, ofs0, ns] 2026-02-24T22:18:48.084 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.086+0000 7f3f1affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961359 cookie 94134147722880 notifier 6935 bl.length()=1017 2026-02-24T22:18:48.084 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.086+0000 7f3f1affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 info.flags=0x17 2026-02-24T22:18:48.084 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.086+0000 7f3f1affd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 to cache LRU end 2026-02-24T22:18:48.084 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.086+0000 7f3f1affd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:48.085 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.086+0000 7f3f4e983a80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): orig_sources= new_sources= 2026-02-24T22:18:48.085 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.086+0000 7f3f4e983a80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): potential sources added=[] removed=[] 2026-02-24T22:18:48.085 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.086+0000 7f3f4e983a80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): orig_dests= new_dests= 2026-02-24T22:18:48.085 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.086+0000 7f3f4e983a80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): potential dests added=[] removed=[] 2026-02-24T22:18:48.085 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.086+0000 7f3f4e983a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530.0 2026-02-24T22:18:48.085 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.090+0000 7f3f4e983a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 : hit (requested=0x17, cached=0x17) 2026-02-24T22:18:48.086 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.090+0000 7f3f4e983a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 2026-02-24T22:18:48.086 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.090+0000 7f3f4e983a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 info.flags=0x17 2026-02-24T22:18:48.086 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.090+0000 7f3f4e983a80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 to cache LRU end 2026-02-24T22:18:48.087 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.090+0000 7f3f4e983a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:48.087 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.090+0000 7f3f4e983a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530, ofs0, ns] 2026-02-24T22:18:48.087 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.090+0000 7f3f1b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961360 cookie 94134147722880 notifier 6935 bl.length()=981 2026-02-24T22:18:48.087 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.090+0000 7f3f1b7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 info.flags=0x17 2026-02-24T22:18:48.087 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.090+0000 7f3f1b7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 to cache LRU end 2026-02-24T22:18:48.087 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.090+0000 7f3f1b7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:48.087 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.090+0000 7f3f4e983a80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): orig_sources= new_sources= 2026-02-24T22:18:48.087 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.090+0000 7f3f4e983a80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): potential sources added=[] removed=[] 2026-02-24T22:18:48.087 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.090+0000 7f3f4e983a80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): orig_dests= new_dests= 2026-02-24T22:18:48.087 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.090+0000 7f3f4e983a80 20 handle_bi_update(): bucket=:abort-at-block-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530]): potential dests added=[] removed=[] 2026-02-24T22:18:48.088 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.090+0000 7f3f4e983a80 1 execute INFO: reshard of bucket "abort-at-block-writes" from 1 shards to 2 shards completed successfully 2026-02-24T22:18:48.091 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.094+0000 7f3f4e983a80 20 remove_watcher() i=3 2026-02-24T22:18:48.091 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.094+0000 7f3f4e983a80 2 removed watcher, disabling cache 2026-02-24T22:18:48.091 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.094+0000 7f3f4e983a80 20 remove_watcher() i=4 2026-02-24T22:18:48.091 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.094+0000 7f3f4e983a80 20 remove_watcher() i=0 2026-02-24T22:18:48.091 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.094+0000 7f3f4e983a80 20 remove_watcher() i=5 2026-02-24T22:18:48.091 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.094+0000 7f3f4e983a80 20 remove_watcher() i=1 2026-02-24T22:18:48.091 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.094+0000 7f3f4e983a80 20 remove_watcher() i=2 2026-02-24T22:18:48.091 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.094+0000 7f3f4e983a80 20 remove_watcher() i=7 2026-02-24T22:18:48.092 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.094+0000 7f3f4e983a80 20 remove_watcher() i=6 2026-02-24T22:18:48.098 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket abort-at-block-writes 2026-02-24T22:18:48.112 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:18:48.113 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:18:48.129 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.134+0000 7fe2ac570a80 4 Realm: () 2026-02-24T22:18:48.129 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.134+0000 7fe2ac570a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:48.129 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.134+0000 7fe2ac570a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:48.130 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.134+0000 7fe2ac570a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:48.130 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.134+0000 7fe2ac570a80 20 zonegroup default 2026-02-24T22:18:48.130 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.134+0000 7fe2ac570a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:48.131 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.134+0000 7fe2ac570a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:48.131 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.134+0000 7fe2ac570a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:48.132 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.134+0000 7fe2ac570a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:48.132 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.134+0000 7fe2ac570a80 20 started sync module instance, tier type = 2026-02-24T22:18:48.132 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.134+0000 7fe2ac570a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:48.133 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.138+0000 7fe2ac570a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:48.133 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.138+0000 7fe2ac570a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:48.133 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.138+0000 7fe2ac570a80 5 note: GC not initialized 2026-02-24T22:18:48.134 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.138+0000 7fe1cbfff640 20 reqs_thread_entry: start 2026-02-24T22:18:48.137 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.142+0000 7fe2aa845640 0 -- 10.20.193.46:0/4217920295 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x557c336dd620 msgr2=0x557c336d0890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:48.143 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.146+0000 7fe2aa044640 0 -- 10.20.193.46:0/4217920295 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x557c336d90a0 msgr2=0x557c336d3e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:48.167 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.170+0000 7fe2ac570a80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:48.167 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.170+0000 7fe2ac570a80 20 Filter name: none 2026-02-24T22:18:48.167 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.170+0000 7fe1ca7fc640 20 reqs_thread_entry: start 2026-02-24T22:18:48.167 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.170+0000 7fe2ac570a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:48.167 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.170+0000 7fe2ac570a80 20 rados_obj.operate() r=0 bl.length=217 2026-02-24T22:18:48.167 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.170+0000 7fe2ac570a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:48.167 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.170+0000 7fe2ac570a80 20 rados_obj.operate() r=0 bl.length=447 2026-02-24T22:18:48.167 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.170+0000 7fe2ac570a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:48.168 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.170+0000 7fe2ac570a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:48.168 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.170+0000 7fe2ac570a80 20 rados_obj.operate() r=0 bl.length=217 2026-02-24T22:18:48.168 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.170+0000 7fe2ac570a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:48.168 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.170+0000 7fe2ac570a80 20 rados_obj.operate() r=0 bl.length=447 2026-02-24T22:18:48.168 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.170+0000 7fe2ac570a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:48.177 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:48.177 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:18:48.177 INFO:tasks.workunit.client.0.trial046.stderr:bucket: abort-at-block-writes id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530 num_objects: 19 size_kb: 1 num_shards: 2 2026-02-24T22:18:48.177 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:48.178 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.178 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.181 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.181 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.226 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.226 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.237 INFO:tasks.workunit.client.0.trial046.stderr:TEST: reshard bucket with EIO injected at commit_target_layout 2026-02-24T22:18:48.237 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:48.238 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.238 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.247 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.247 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.290 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.290 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.297 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.297 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.303 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.303 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.309 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.309 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.315 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.315 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.321 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.321 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.326 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.327 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.332 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.332 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.338 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.338 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.344 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.344 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.350 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.350 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.356 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.356 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.362 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.362 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.369 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.369 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.374 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.374 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.380 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.380 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.386 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.386 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.393 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.393 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.421 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.421 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.426 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.426 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.431 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2026-02-24T22:18:48.446 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:18:48.446 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:18:48.464 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.466+0000 7fd7060e9a80 4 Realm: () 2026-02-24T22:18:48.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.466+0000 7fd7060e9a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:48.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.466+0000 7fd7060e9a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:48.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.466+0000 7fd7060e9a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:48.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.466+0000 7fd7060e9a80 20 zonegroup default 2026-02-24T22:18:48.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.470+0000 7fd7060e9a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:48.466 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.470+0000 7fd7060e9a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:48.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.470+0000 7fd7060e9a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:48.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.470+0000 7fd7060e9a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:48.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.470+0000 7fd7060e9a80 20 started sync module instance, tier type = 2026-02-24T22:18:48.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.470+0000 7fd7060e9a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:48.469 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.470+0000 7fd7060e9a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:48.469 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.470+0000 7fd7060e9a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:48.469 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.474+0000 7fd7060e9a80 5 note: GC not initialized 2026-02-24T22:18:48.469 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.474+0000 7fd6b17e2640 20 reqs_thread_entry: start 2026-02-24T22:18:48.473 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.478+0000 7fd6fffff640 0 -- 10.20.193.46:0/3229329297 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x556847042280 msgr2=0x5568470489f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:48.479 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.482+0000 7fd6ff7fe640 0 -- 10.20.193.46:0/3229329297 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x556847044470 msgr2=0x55684703f1e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:48.502 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.506+0000 7fd7060e9a80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:48.503 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.506+0000 7fd7060e9a80 20 Filter name: none 2026-02-24T22:18:48.503 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.506+0000 7fd61b7fe640 20 reqs_thread_entry: start 2026-02-24T22:18:48.503 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.506+0000 7fd7060e9a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:48.503 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.506+0000 7fd7060e9a80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:18:48.503 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.506+0000 7fd7060e9a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:48.504 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.506+0000 7fd7060e9a80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:18:48.505 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.506+0000 7fd7060e9a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:48.505 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.506+0000 7fd7060e9a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:48.505 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.506+0000 7fd7060e9a80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:18:48.505 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.506+0000 7fd7060e9a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:48.505 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.510+0000 7fd7060e9a80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:18:48.505 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.510+0000 7fd7060e9a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:48.515 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:48.515 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:18:48.515 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-commit-target-layout id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:18:48.515 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:48.515 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket error-at-commit-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at commit_target_layout 2026-02-24T22:18:48.531 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:18:48.531 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:18:48.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.554+0000 7ff197feca80 4 Realm: () 2026-02-24T22:18:48.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.554+0000 7ff197feca80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:48.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.554+0000 7ff197feca80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:48.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.554+0000 7ff197feca80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:48.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.554+0000 7ff197feca80 20 zonegroup default 2026-02-24T22:18:48.550 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.554+0000 7ff197feca80 20 rados->read ofs=0 len=0 2026-02-24T22:18:48.551 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.554+0000 7ff197feca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:48.551 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.554+0000 7ff197feca80 20 rados->read ofs=0 len=0 2026-02-24T22:18:48.552 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.554+0000 7ff197feca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:48.552 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.554+0000 7ff197feca80 20 started sync module instance, tier type = 2026-02-24T22:18:48.552 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.554+0000 7ff197feca80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:48.553 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.558+0000 7ff1962c1640 0 -- 10.20.193.46:0/2341902397 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x555d7a39a380 msgr2=0x555d7a3ba800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:18:48.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.558+0000 7ff197feca80 20 add_watcher() i=1 2026-02-24T22:18:48.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.558+0000 7ff197feca80 20 add_watcher() i=3 2026-02-24T22:18:48.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.558+0000 7ff197feca80 20 add_watcher() i=0 2026-02-24T22:18:48.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.558+0000 7ff197feca80 20 add_watcher() i=6 2026-02-24T22:18:48.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.558+0000 7ff197feca80 20 add_watcher() i=5 2026-02-24T22:18:48.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.558+0000 7ff197feca80 20 add_watcher() i=7 2026-02-24T22:18:48.555 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.558+0000 7ff197feca80 20 add_watcher() i=4 2026-02-24T22:18:48.555 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.558+0000 7ff197feca80 20 add_watcher() i=2 2026-02-24T22:18:48.555 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.558+0000 7ff197feca80 2 all 8 watchers are set, enabling cache 2026-02-24T22:18:48.557 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.558+0000 7ff197feca80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:48.557 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.558+0000 7ff197feca80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:48.557 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.558+0000 7ff197feca80 5 note: GC not initialized 2026-02-24T22:18:48.557 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.558+0000 7ff0c67fc640 20 reqs_thread_entry: start 2026-02-24T22:18:48.557 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.562+0000 7ff1962c1640 0 -- 10.20.193.46:0/648304492 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x555d7a3cc610 msgr2=0x555d7a3af240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:18:48.799 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.802+0000 7ff197feca80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:48.799 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.802+0000 7ff197feca80 20 Filter name: none 2026-02-24T22:18:48.799 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.802+0000 7ff0c4ff9640 20 reqs_thread_entry: start 2026-02-24T22:18:48.799 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.802+0000 7ff197feca80 10 cache get: name=default.rgw.meta+root+error-at-commit-target-layout : miss 2026-02-24T22:18:48.799 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.802+0000 7ff197feca80 20 rados->read ofs=0 len=0 2026-02-24T22:18:48.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.802+0000 7ff197feca80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:18:48.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.802+0000 7ff197feca80 10 cache put: name=default.rgw.meta+root+error-at-commit-target-layout info.flags=0x11 2026-02-24T22:18:48.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.802+0000 7ff197feca80 10 adding default.rgw.meta+root+error-at-commit-target-layout to cache LRU end 2026-02-24T22:18:48.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.802+0000 7ff197feca80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 : miss 2026-02-24T22:18:48.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.802+0000 7ff197feca80 20 rados->read ofs=0 len=0 2026-02-24T22:18:48.801 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.802+0000 7ff197feca80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:18:48.801 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.802+0000 7ff197feca80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 info.flags=0x17 2026-02-24T22:18:48.801 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.802+0000 7ff197feca80 10 adding default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 to cache LRU end 2026-02-24T22:18:48.801 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.802+0000 7ff197feca80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:18:48.801 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.802+0000 7ff197feca80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:48.801 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.802+0000 7ff197feca80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 2026-02-24T22:18:48.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.806+0000 7ff197feca80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 info.flags=0x17 2026-02-24T22:18:48.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.806+0000 7ff197feca80 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 to cache LRU end 2026-02-24T22:18:48.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.806+0000 7ff197feca80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:48.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.806+0000 7ff197feca80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531, ofs0, ns] 2026-02-24T22:18:48.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.810+0000 7ff170ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959903 cookie 93859971319616 notifier 7470 bl.length()=1033 2026-02-24T22:18:48.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.810+0000 7ff170ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 info.flags=0x17 2026-02-24T22:18:48.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.810+0000 7ff170ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 to cache LRU end 2026-02-24T22:18:48.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.810+0000 7ff170ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:48.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.810+0000 7ff197feca80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531]): orig_sources= new_sources= 2026-02-24T22:18:48.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.810+0000 7ff197feca80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531]): potential sources added=[] removed=[] 2026-02-24T22:18:48.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.810+0000 7ff197feca80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531]): orig_dests= new_dests= 2026-02-24T22:18:48.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.810+0000 7ff197feca80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531]): potential dests added=[] removed=[] 2026-02-24T22:18:48.807 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.810+0000 7ff197feca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531.1.0 2026-02-24T22:18:48.807 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.810+0000 7ff197feca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531.1.0 2026-02-24T22:18:48.807 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.810+0000 7ff197feca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531.1.1 2026-02-24T22:18:48.807 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.810+0000 7ff197feca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531.1.1 2026-02-24T22:18:48.807 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.810+0000 7ff197feca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531.0 2026-02-24T22:18:48.809 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.810+0000 7ff197feca80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 : hit (requested=0x17, cached=0x17) 2026-02-24T22:18:48.809 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.810+0000 7ff197feca80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 2026-02-24T22:18:48.810 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.814+0000 7ff197feca80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 info.flags=0x17 2026-02-24T22:18:48.810 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.814+0000 7ff197feca80 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 to cache LRU end 2026-02-24T22:18:48.810 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.814+0000 7ff197feca80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:48.810 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.814+0000 7ff197feca80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531, ofs0, ns] 2026-02-24T22:18:48.811 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.814+0000 7ff170ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959904 cookie 93859971319616 notifier 7470 bl.length()=1033 2026-02-24T22:18:48.811 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.814+0000 7ff170ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 info.flags=0x17 2026-02-24T22:18:48.811 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.814+0000 7ff170ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 to cache LRU end 2026-02-24T22:18:48.811 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.814+0000 7ff170ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:48.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.814+0000 7ff197feca80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531]): orig_sources= new_sources= 2026-02-24T22:18:48.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.814+0000 7ff197feca80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531]): potential sources added=[] removed=[] 2026-02-24T22:18:48.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.814+0000 7ff197feca80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531]): orig_dests= new_dests= 2026-02-24T22:18:48.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.814+0000 7ff197feca80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531]): potential dests added=[] removed=[] 2026-02-24T22:18:48.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.814+0000 7ff197feca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531.0 2026-02-24T22:18:48.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.814+0000 7ff197feca80 -1 Injecting error=-5 at location=commit_target_layout 2026-02-24T22:18:48.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.814+0000 7ff197feca80 0 ERROR: commit_reshard failed to commit target index layout: (5) Input/output error 2026-02-24T22:18:48.816 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.818+0000 7ff197feca80 20 remove_watcher() i=1 2026-02-24T22:18:48.816 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.818+0000 7ff197feca80 2 removed watcher, disabling cache 2026-02-24T22:18:48.816 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.818+0000 7ff197feca80 20 remove_watcher() i=3 2026-02-24T22:18:48.817 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.818+0000 7ff197feca80 20 remove_watcher() i=4 2026-02-24T22:18:48.817 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.818+0000 7ff197feca80 20 remove_watcher() i=2 2026-02-24T22:18:48.817 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.818+0000 7ff197feca80 20 remove_watcher() i=0 2026-02-24T22:18:48.817 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.818+0000 7ff197feca80 20 remove_watcher() i=5 2026-02-24T22:18:48.817 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.818+0000 7ff197feca80 20 remove_watcher() i=6 2026-02-24T22:18:48.817 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.818+0000 7ff197feca80 20 remove_watcher() i=7 2026-02-24T22:18:48.823 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2026-02-24T22:18:48.838 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:18:48.838 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:18:48.855 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.858+0000 7fb903568a80 4 Realm: () 2026-02-24T22:18:48.855 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.858+0000 7fb903568a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:48.855 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.858+0000 7fb903568a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:48.856 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.858+0000 7fb903568a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:48.856 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.858+0000 7fb903568a80 20 zonegroup default 2026-02-24T22:18:48.856 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.858+0000 7fb903568a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:48.857 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.862+0000 7fb903568a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:48.857 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.862+0000 7fb903568a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:48.858 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.862+0000 7fb903568a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:48.858 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.862+0000 7fb903568a80 20 started sync module instance, tier type = 2026-02-24T22:18:48.858 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.862+0000 7fb903568a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:48.859 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.862+0000 7fb903568a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:48.859 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.862+0000 7fb903568a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:48.859 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.862+0000 7fb903568a80 5 note: GC not initialized 2026-02-24T22:18:48.859 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.862+0000 7fb8aefe5640 20 reqs_thread_entry: start 2026-02-24T22:18:48.863 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.866+0000 7fb90183d640 0 -- 10.20.193.46:0/3284208058 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55c99709b880 msgr2=0x55c99708eaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:48.869 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.874+0000 7fb90103c640 0 -- 10.20.193.46:0/3284208058 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55c997097300 msgr2=0x55c997092070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:18:48.894 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.898+0000 7fb903568a80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:48.894 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.898+0000 7fb903568a80 20 Filter name: none 2026-02-24T22:18:48.895 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.898+0000 7fb8ad7e2640 20 reqs_thread_entry: start 2026-02-24T22:18:48.895 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.898+0000 7fb903568a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:48.895 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.898+0000 7fb903568a80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:18:48.895 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.898+0000 7fb903568a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:48.896 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.898+0000 7fb903568a80 20 rados_obj.operate() r=0 bl.length=491 2026-02-24T22:18:48.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.898+0000 7fb903568a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:48.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.898+0000 7fb903568a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:48.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.898+0000 7fb903568a80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:18:48.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.898+0000 7fb903568a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:48.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.902+0000 7fb903568a80 20 rados_obj.operate() r=0 bl.length=491 2026-02-24T22:18:48.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.902+0000 7fb903568a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:48.907 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:48.908 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:18:48.908 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-commit-target-layout id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:18:48.908 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:48.909 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.909 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.915 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:48.915 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:48.957 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket error-at-commit-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2026-02-24T22:18:48.974 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:18:48.974 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:18:48.990 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.994+0000 7fad307fea80 4 Realm: () 2026-02-24T22:18:48.990 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.994+0000 7fad307fea80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:48.990 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.994+0000 7fad307fea80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:48.991 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.994+0000 7fad307fea80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:48.991 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.994+0000 7fad307fea80 20 zonegroup default 2026-02-24T22:18:48.991 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.994+0000 7fad307fea80 20 rados->read ofs=0 len=0 2026-02-24T22:18:48.992 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.994+0000 7fad307fea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:48.993 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.994+0000 7fad307fea80 20 rados->read ofs=0 len=0 2026-02-24T22:18:48.993 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.998+0000 7fad307fea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:48.993 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.998+0000 7fad307fea80 20 started sync module instance, tier type = 2026-02-24T22:18:48.993 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.998+0000 7fad307fea80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:48.994 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.998+0000 7fad2e2d2640 0 -- 10.20.193.46:0/3460050526 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x5644d65aafc0 msgr2=0x5644d65cb440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:18:48.995 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.998+0000 7fad307fea80 20 add_watcher() i=1 2026-02-24T22:18:48.995 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.998+0000 7fad307fea80 20 add_watcher() i=3 2026-02-24T22:18:48.995 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.998+0000 7fad307fea80 20 add_watcher() i=5 2026-02-24T22:18:48.995 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.998+0000 7fad307fea80 20 add_watcher() i=6 2026-02-24T22:18:48.995 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.998+0000 7fad307fea80 20 add_watcher() i=0 2026-02-24T22:18:48.996 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.998+0000 7fad307fea80 20 add_watcher() i=7 2026-02-24T22:18:48.996 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:48.998+0000 7fad307fea80 20 add_watcher() i=4 2026-02-24T22:18:49.200 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.202+0000 7fad307fea80 20 add_watcher() i=2 2026-02-24T22:18:49.200 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.202+0000 7fad307fea80 2 all 8 watchers are set, enabling cache 2026-02-24T22:18:49.202 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.206+0000 7fad307fea80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:49.202 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.206+0000 7fad307fea80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:49.202 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.206+0000 7fad307fea80 5 note: GC not initialized 2026-02-24T22:18:49.202 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.206+0000 7fac477fe640 20 reqs_thread_entry: start 2026-02-24T22:18:49.203 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.206+0000 7fad2ead3640 0 -- 10.20.193.46:0/2537826229 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5644d663fc80 msgr2=0x5644d66374e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:18:49.445 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.446+0000 7fad307fea80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:49.445 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.446+0000 7fad307fea80 20 Filter name: none 2026-02-24T22:18:49.446 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.446+0000 7fac45ffb640 20 reqs_thread_entry: start 2026-02-24T22:18:49.446 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.450+0000 7fad307fea80 10 cache get: name=default.rgw.meta+root+error-at-commit-target-layout : miss 2026-02-24T22:18:49.446 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.450+0000 7fad307fea80 20 rados->read ofs=0 len=0 2026-02-24T22:18:49.446 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.450+0000 7fad307fea80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:18:49.446 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.450+0000 7fad307fea80 10 cache put: name=default.rgw.meta+root+error-at-commit-target-layout info.flags=0x11 2026-02-24T22:18:49.446 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.450+0000 7fad307fea80 10 adding default.rgw.meta+root+error-at-commit-target-layout to cache LRU end 2026-02-24T22:18:49.446 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.450+0000 7fad307fea80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 : miss 2026-02-24T22:18:49.446 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.450+0000 7fad307fea80 20 rados->read ofs=0 len=0 2026-02-24T22:18:49.447 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.450+0000 7fad307fea80 20 rados_obj.operate() r=0 bl.length=491 2026-02-24T22:18:49.447 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.450+0000 7fad307fea80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 info.flags=0x17 2026-02-24T22:18:49.447 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.450+0000 7fad307fea80 10 adding default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 to cache LRU end 2026-02-24T22:18:49.447 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.450+0000 7fad307fea80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:18:49.447 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.450+0000 7fad307fea80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:49.447 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.450+0000 7fad307fea80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 2026-02-24T22:18:49.448 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.450+0000 7fad307fea80 10 init_target_layout removing existing target index objects from a previous reshard attempt 2026-02-24T22:18:49.452 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.454+0000 7fad307fea80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 info.flags=0x17 2026-02-24T22:18:49.452 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.454+0000 7fad307fea80 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 to cache LRU end 2026-02-24T22:18:49.452 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.454+0000 7fad307fea80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:49.452 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.454+0000 7fad307fea80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531, ofs0, ns] 2026-02-24T22:18:49.452 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.454+0000 7fad09ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959905 cookie 94853655262000 notifier 7506 bl.length()=1033 2026-02-24T22:18:49.452 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.454+0000 7fad09ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 info.flags=0x17 2026-02-24T22:18:49.452 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.454+0000 7fad09ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 to cache LRU end 2026-02-24T22:18:49.452 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.454+0000 7fad09ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:49.453 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.454+0000 7fad307fea80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531]): orig_sources= new_sources= 2026-02-24T22:18:49.453 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.454+0000 7fad307fea80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531]): potential sources added=[] removed=[] 2026-02-24T22:18:49.453 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.454+0000 7fad307fea80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531]): orig_dests= new_dests= 2026-02-24T22:18:49.453 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.454+0000 7fad307fea80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531]): potential dests added=[] removed=[] 2026-02-24T22:18:49.454 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.458+0000 7fad307fea80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531.2.0 2026-02-24T22:18:49.454 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.458+0000 7fad307fea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531.2.0 2026-02-24T22:18:49.454 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.458+0000 7fad307fea80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531.2.1 2026-02-24T22:18:49.454 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.458+0000 7fad307fea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531.2.1 2026-02-24T22:18:49.454 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.458+0000 7fad307fea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531.0 2026-02-24T22:18:49.455 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.458+0000 7fad307fea80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 : hit (requested=0x17, cached=0x17) 2026-02-24T22:18:49.455 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.458+0000 7fad307fea80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 2026-02-24T22:18:49.456 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.458+0000 7fad307fea80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 info.flags=0x17 2026-02-24T22:18:49.456 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.458+0000 7fad307fea80 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 to cache LRU end 2026-02-24T22:18:49.456 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.458+0000 7fad307fea80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:49.456 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.458+0000 7fad307fea80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531, ofs0, ns] 2026-02-24T22:18:49.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.458+0000 7fad09ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959906 cookie 94853655262000 notifier 7506 bl.length()=1033 2026-02-24T22:18:49.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.458+0000 7fad09ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 info.flags=0x17 2026-02-24T22:18:49.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.458+0000 7fad09ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 to cache LRU end 2026-02-24T22:18:49.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.458+0000 7fad09ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:49.458 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.462+0000 7fad307fea80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531]): orig_sources= new_sources= 2026-02-24T22:18:49.458 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.462+0000 7fad307fea80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531]): potential sources added=[] removed=[] 2026-02-24T22:18:49.458 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.462+0000 7fad307fea80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531]): orig_dests= new_dests= 2026-02-24T22:18:49.458 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.462+0000 7fad307fea80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531]): potential dests added=[] removed=[] 2026-02-24T22:18:49.458 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.462+0000 7fad307fea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531.0 2026-02-24T22:18:49.459 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.462+0000 7fad307fea80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 : hit (requested=0x17, cached=0x17) 2026-02-24T22:18:49.459 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.462+0000 7fad307fea80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 2026-02-24T22:18:49.459 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.462+0000 7fad307fea80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 info.flags=0x17 2026-02-24T22:18:49.459 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.462+0000 7fad307fea80 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 to cache LRU end 2026-02-24T22:18:49.459 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.462+0000 7fad307fea80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:49.459 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.462+0000 7fad307fea80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531, ofs0, ns] 2026-02-24T22:18:49.460 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.462+0000 7fad09ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959907 cookie 94853655262000 notifier 7506 bl.length()=997 2026-02-24T22:18:49.460 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.462+0000 7fad09ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 info.flags=0x17 2026-02-24T22:18:49.460 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.462+0000 7fad09ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 to cache LRU end 2026-02-24T22:18:49.460 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.462+0000 7fad09ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:49.460 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.462+0000 7fad307fea80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531]): orig_sources= new_sources= 2026-02-24T22:18:49.460 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.462+0000 7fad307fea80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531]): potential sources added=[] removed=[] 2026-02-24T22:18:49.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.462+0000 7fad307fea80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531]): orig_dests= new_dests= 2026-02-24T22:18:49.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.462+0000 7fad307fea80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531]): potential dests added=[] removed=[] 2026-02-24T22:18:49.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.462+0000 7fad307fea80 1 execute INFO: reshard of bucket "error-at-commit-target-layout" from 1 shards to 2 shards completed successfully 2026-02-24T22:18:49.464 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.466+0000 7fad307fea80 20 remove_watcher() i=3 2026-02-24T22:18:49.464 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.466+0000 7fad307fea80 2 removed watcher, disabling cache 2026-02-24T22:18:49.464 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.466+0000 7fad307fea80 20 remove_watcher() i=5 2026-02-24T22:18:49.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.466+0000 7fad307fea80 20 remove_watcher() i=6 2026-02-24T22:18:49.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.466+0000 7fad307fea80 20 remove_watcher() i=1 2026-02-24T22:18:49.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.466+0000 7fad307fea80 20 remove_watcher() i=4 2026-02-24T22:18:49.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.466+0000 7fad307fea80 20 remove_watcher() i=0 2026-02-24T22:18:49.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.466+0000 7fad307fea80 20 remove_watcher() i=7 2026-02-24T22:18:49.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.466+0000 7fad307fea80 20 remove_watcher() i=2 2026-02-24T22:18:49.471 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2026-02-24T22:18:49.486 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:18:49.486 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:18:49.504 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.506+0000 7fe4d5fbda80 4 Realm: () 2026-02-24T22:18:49.504 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.506+0000 7fe4d5fbda80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:49.504 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.506+0000 7fe4d5fbda80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:49.504 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.506+0000 7fe4d5fbda80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:49.504 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.506+0000 7fe4d5fbda80 20 zonegroup default 2026-02-24T22:18:49.505 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.506+0000 7fe4d5fbda80 20 rados->read ofs=0 len=0 2026-02-24T22:18:49.506 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.510+0000 7fe4d5fbda80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:49.506 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.510+0000 7fe4d5fbda80 20 rados->read ofs=0 len=0 2026-02-24T22:18:49.506 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.510+0000 7fe4d5fbda80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:49.507 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.510+0000 7fe4d5fbda80 20 started sync module instance, tier type = 2026-02-24T22:18:49.507 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.510+0000 7fe4d5fbda80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:49.508 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.510+0000 7fe4d5fbda80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:49.508 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.510+0000 7fe4d5fbda80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:49.508 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.510+0000 7fe4d5fbda80 5 note: GC not initialized 2026-02-24T22:18:49.508 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.510+0000 7fe4817e2640 20 reqs_thread_entry: start 2026-02-24T22:18:49.512 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.514+0000 7fe4cffff640 0 -- 10.20.193.46:0/3896427998 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x5582b477c910 msgr2=0x5582b4783080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:49.720 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.722+0000 7fe4cf7fe640 0 -- 10.20.193.46:0/3896427998 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5582b4750f00 msgr2=0x5582b477eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:49.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.754+0000 7fe4d5fbda80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:49.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.754+0000 7fe4d5fbda80 20 Filter name: none 2026-02-24T22:18:49.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.754+0000 7fe3eb7fe640 20 reqs_thread_entry: start 2026-02-24T22:18:49.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.754+0000 7fe4d5fbda80 20 rados->read ofs=0 len=0 2026-02-24T22:18:49.750 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.754+0000 7fe4d5fbda80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:18:49.750 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.754+0000 7fe4d5fbda80 20 rados->read ofs=0 len=0 2026-02-24T22:18:49.751 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.754+0000 7fe4d5fbda80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:18:49.751 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.754+0000 7fe4d5fbda80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:49.751 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.754+0000 7fe4d5fbda80 20 rados->read ofs=0 len=0 2026-02-24T22:18:49.751 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.754+0000 7fe4d5fbda80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:18:49.751 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.754+0000 7fe4d5fbda80 20 rados->read ofs=0 len=0 2026-02-24T22:18:49.752 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.754+0000 7fe4d5fbda80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:18:49.752 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:49.754+0000 7fe4d5fbda80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:49.761 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:49.761 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:18:49.762 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-commit-target-layout id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531 num_objects: 19 size_kb: 1 num_shards: 2 2026-02-24T22:18:49.762 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:49.763 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:49.763 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:49.765 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:49.765 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:49.810 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:49.810 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:49.821 INFO:tasks.workunit.client.0.trial046.stderr:TEST: reshard bucket with ECANCELED injected at commit_target_layout 2026-02-24T22:18:49.821 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:49.822 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:49.822 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:49.831 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:49.831 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:49.874 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:49.874 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:49.879 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:49.880 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:49.885 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:49.885 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:49.891 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:49.891 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:49.897 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:49.897 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:49.903 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:49.903 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:49.909 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:49.909 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:49.914 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:49.914 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:49.920 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:49.920 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:49.926 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:49.926 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:49.932 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:49.932 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:49.937 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:49.937 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:49.943 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:49.943 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:49.948 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:49.948 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:49.954 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:49.954 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:49.960 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:49.960 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:49.966 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:49.966 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:49.972 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:49.972 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:49.978 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:49.978 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:49.983 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:49.984 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:49.989 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2026-02-24T22:18:50.004 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:18:50.004 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:18:50.022 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.026+0000 7f335d9fba80 4 Realm: () 2026-02-24T22:18:50.022 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.026+0000 7f335d9fba80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:50.022 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.026+0000 7f335d9fba80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:50.022 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.026+0000 7f335d9fba80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:50.022 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.026+0000 7f335d9fba80 20 zonegroup default 2026-02-24T22:18:50.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.026+0000 7f335d9fba80 20 rados->read ofs=0 len=0 2026-02-24T22:18:50.024 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.026+0000 7f335d9fba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:50.024 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.026+0000 7f335d9fba80 20 rados->read ofs=0 len=0 2026-02-24T22:18:50.025 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.030+0000 7f335d9fba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:50.025 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.030+0000 7f335d9fba80 20 started sync module instance, tier type = 2026-02-24T22:18:50.025 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.030+0000 7f335d9fba80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:50.026 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.030+0000 7f335d9fba80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:50.027 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.030+0000 7f335d9fba80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:50.027 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.030+0000 7f335d9fba80 5 note: GC not initialized 2026-02-24T22:18:50.027 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.030+0000 7f3308fe1640 20 reqs_thread_entry: start 2026-02-24T22:18:50.031 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.034+0000 7f3356ffd640 0 -- 10.20.193.46:0/1118944665 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55c4590cb3a0 msgr2=0x55c4590cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:50.242 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.246+0000 7f33577fe640 0 -- 10.20.193.46:0/1118944665 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x55c4590b9dc0 msgr2=0x55c4590bc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:50.470 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.474+0000 7f335d9fba80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:50.471 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.474+0000 7f335d9fba80 20 Filter name: none 2026-02-24T22:18:50.471 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.474+0000 7f327affd640 20 reqs_thread_entry: start 2026-02-24T22:18:50.471 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.474+0000 7f335d9fba80 20 rados->read ofs=0 len=0 2026-02-24T22:18:50.472 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.474+0000 7f335d9fba80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:18:50.472 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.474+0000 7f335d9fba80 20 rados->read ofs=0 len=0 2026-02-24T22:18:50.472 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.474+0000 7f335d9fba80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:18:50.472 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.474+0000 7f335d9fba80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:50.472 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.474+0000 7f335d9fba80 20 rados->read ofs=0 len=0 2026-02-24T22:18:50.473 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.474+0000 7f335d9fba80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:18:50.473 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.474+0000 7f335d9fba80 20 rados->read ofs=0 len=0 2026-02-24T22:18:50.473 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.474+0000 7f335d9fba80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:18:50.473 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.474+0000 7f335d9fba80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:50.483 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:50.483 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:18:50.483 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-commit-target-layout id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:18:50.483 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:50.483 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket error-at-commit-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at commit_target_layout --inject-error-code 125 2026-02-24T22:18:50.500 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:18:50.500 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:18:50.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.522+0000 7fdb0c2a8a80 4 Realm: () 2026-02-24T22:18:50.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.522+0000 7fdb0c2a8a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:50.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.522+0000 7fdb0c2a8a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:50.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.522+0000 7fdb0c2a8a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:50.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.522+0000 7fdb0c2a8a80 20 zonegroup default 2026-02-24T22:18:50.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.522+0000 7fdb0c2a8a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:50.521 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.526+0000 7fdb0c2a8a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:50.521 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.526+0000 7fdb0c2a8a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:50.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.526+0000 7fdb0c2a8a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:50.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.526+0000 7fdb0c2a8a80 20 started sync module instance, tier type = 2026-02-24T22:18:50.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.526+0000 7fdb0c2a8a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:50.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.526+0000 7fdb0a57d640 0 -- 10.20.193.46:0/4247646049 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x55aefc874300 msgr2=0x55aefc894780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:50.523 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.526+0000 7fdb0c2a8a80 20 add_watcher() i=3 2026-02-24T22:18:50.523 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.526+0000 7fdb0c2a8a80 20 add_watcher() i=1 2026-02-24T22:18:50.523 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.526+0000 7fdb0c2a8a80 20 add_watcher() i=0 2026-02-24T22:18:50.524 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.526+0000 7fdb0c2a8a80 20 add_watcher() i=6 2026-02-24T22:18:50.524 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.526+0000 7fdb0c2a8a80 20 add_watcher() i=5 2026-02-24T22:18:50.524 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.526+0000 7fdb0c2a8a80 20 add_watcher() i=2 2026-02-24T22:18:50.524 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.526+0000 7fdb0c2a8a80 20 add_watcher() i=4 2026-02-24T22:18:50.525 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.530+0000 7fdb0c2a8a80 20 add_watcher() i=7 2026-02-24T22:18:50.525 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.530+0000 7fdb0c2a8a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:18:50.527 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.530+0000 7fdb0c2a8a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:50.527 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.530+0000 7fdb0c2a8a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:50.527 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.530+0000 7fdb0c2a8a80 5 note: GC not initialized 2026-02-24T22:18:50.527 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.530+0000 7fda3a7fc640 20 reqs_thread_entry: start 2026-02-24T22:18:50.527 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.530+0000 7fdb0a57d640 0 -- 10.20.193.46:0/3950779280 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55aefc79c410 msgr2=0x55aefc79c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:50.767 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.770+0000 7fdb0c2a8a80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:50.767 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.770+0000 7fdb0c2a8a80 20 Filter name: none 2026-02-24T22:18:50.767 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.770+0000 7fda38ff9640 20 reqs_thread_entry: start 2026-02-24T22:18:50.767 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.770+0000 7fdb0c2a8a80 10 cache get: name=default.rgw.meta+root+error-at-commit-target-layout : miss 2026-02-24T22:18:50.767 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.770+0000 7fdb0c2a8a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:50.767 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.770+0000 7fdb0c2a8a80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:18:50.767 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.770+0000 7fdb0c2a8a80 10 cache put: name=default.rgw.meta+root+error-at-commit-target-layout info.flags=0x11 2026-02-24T22:18:50.767 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.770+0000 7fdb0c2a8a80 10 adding default.rgw.meta+root+error-at-commit-target-layout to cache LRU end 2026-02-24T22:18:50.767 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.770+0000 7fdb0c2a8a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532 : miss 2026-02-24T22:18:50.768 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.770+0000 7fdb0c2a8a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:50.768 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.770+0000 7fdb0c2a8a80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:18:50.768 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.770+0000 7fdb0c2a8a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532 info.flags=0x17 2026-02-24T22:18:50.769 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.770+0000 7fdb0c2a8a80 10 adding default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532 to cache LRU end 2026-02-24T22:18:50.769 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.770+0000 7fdb0c2a8a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:18:50.769 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.770+0000 7fdb0c2a8a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:50.769 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.770+0000 7fdb0c2a8a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532 2026-02-24T22:18:50.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.774+0000 7fdb0c2a8a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532 info.flags=0x17 2026-02-24T22:18:50.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.774+0000 7fdb0c2a8a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532 to cache LRU end 2026-02-24T22:18:50.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.774+0000 7fdb0c2a8a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:50.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.774+0000 7fdb0c2a8a80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532, ofs0, ns] 2026-02-24T22:18:50.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.774+0000 7fdae4ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959910 cookie 94210049395488 notifier 7076 bl.length()=1033 2026-02-24T22:18:50.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.774+0000 7fdae4ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532 info.flags=0x17 2026-02-24T22:18:50.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.774+0000 7fdae4ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532 to cache LRU end 2026-02-24T22:18:50.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.774+0000 7fdae4ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:50.773 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.774+0000 7fdb0c2a8a80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532]): orig_sources= new_sources= 2026-02-24T22:18:50.773 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.774+0000 7fdb0c2a8a80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532]): potential sources added=[] removed=[] 2026-02-24T22:18:50.773 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.774+0000 7fdb0c2a8a80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532]): orig_dests= new_dests= 2026-02-24T22:18:50.773 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.774+0000 7fdb0c2a8a80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532]): potential dests added=[] removed=[] 2026-02-24T22:18:50.774 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.778+0000 7fdb0c2a8a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532.1.0 2026-02-24T22:18:50.774 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.778+0000 7fdb0c2a8a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532.1.0 2026-02-24T22:18:50.774 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.778+0000 7fdb0c2a8a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532.1.1 2026-02-24T22:18:50.774 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.778+0000 7fdb0c2a8a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532.1.1 2026-02-24T22:18:50.774 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.778+0000 7fdb0c2a8a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532.0 2026-02-24T22:18:50.776 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.778+0000 7fdb0c2a8a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532 : hit (requested=0x17, cached=0x17) 2026-02-24T22:18:50.776 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.778+0000 7fdb0c2a8a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532 2026-02-24T22:18:50.777 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.778+0000 7fdb0c2a8a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532 info.flags=0x17 2026-02-24T22:18:50.777 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.778+0000 7fdb0c2a8a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532 to cache LRU end 2026-02-24T22:18:50.777 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.778+0000 7fdb0c2a8a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:50.777 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.778+0000 7fdb0c2a8a80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532, ofs0, ns] 2026-02-24T22:18:50.777 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.778+0000 7fdae57fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959911 cookie 94210049395488 notifier 7076 bl.length()=1033 2026-02-24T22:18:50.777 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.778+0000 7fdae57fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532 info.flags=0x17 2026-02-24T22:18:50.777 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.778+0000 7fdae57fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532 to cache LRU end 2026-02-24T22:18:50.777 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.778+0000 7fdae57fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:50.778 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.782+0000 7fdb0c2a8a80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532]): orig_sources= new_sources= 2026-02-24T22:18:50.778 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.782+0000 7fdb0c2a8a80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532]): potential sources added=[] removed=[] 2026-02-24T22:18:50.778 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.782+0000 7fdb0c2a8a80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532]): orig_dests= new_dests= 2026-02-24T22:18:50.778 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.782+0000 7fdb0c2a8a80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532]): potential dests added=[] removed=[] 2026-02-24T22:18:50.778 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.782+0000 7fdb0c2a8a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532.0 2026-02-24T22:18:50.779 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.782+0000 7fdb0c2a8a80 -1 Injecting error=-125 at location=commit_target_layout 2026-02-24T22:18:50.779 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.782+0000 7fdb0c2a8a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532 : hit (requested=0x17, cached=0x17) 2026-02-24T22:18:50.779 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.782+0000 7fdb0c2a8a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532 2026-02-24T22:18:50.780 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.782+0000 7fdb0c2a8a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532 info.flags=0x17 2026-02-24T22:18:50.780 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.782+0000 7fdb0c2a8a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532 to cache LRU end 2026-02-24T22:18:50.780 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.782+0000 7fdb0c2a8a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:50.780 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.782+0000 7fdb0c2a8a80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532, ofs0, ns] 2026-02-24T22:18:50.780 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.782+0000 7fdae57fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959912 cookie 94210049395488 notifier 7076 bl.length()=997 2026-02-24T22:18:50.780 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.782+0000 7fdae57fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532 info.flags=0x17 2026-02-24T22:18:50.780 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.782+0000 7fdae57fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532 to cache LRU end 2026-02-24T22:18:50.780 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.782+0000 7fdae57fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:50.780 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.782+0000 7fdb0c2a8a80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532]): orig_sources= new_sources= 2026-02-24T22:18:50.780 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.782+0000 7fdb0c2a8a80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532]): potential sources added=[] removed=[] 2026-02-24T22:18:50.781 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.782+0000 7fdb0c2a8a80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532]): orig_dests= new_dests= 2026-02-24T22:18:50.781 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.782+0000 7fdb0c2a8a80 20 handle_bi_update(): bucket=:error-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532]): potential dests added=[] removed=[] 2026-02-24T22:18:50.781 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.782+0000 7fdb0c2a8a80 1 execute INFO: reshard of bucket "error-at-commit-target-layout" from 1 shards to 2 shards completed successfully 2026-02-24T22:18:50.784 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.786+0000 7fdb0c2a8a80 20 remove_watcher() i=1 2026-02-24T22:18:50.784 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.786+0000 7fdb0c2a8a80 2 removed watcher, disabling cache 2026-02-24T22:18:50.784 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.786+0000 7fdb0c2a8a80 20 remove_watcher() i=3 2026-02-24T22:18:50.784 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.786+0000 7fdb0c2a8a80 20 remove_watcher() i=0 2026-02-24T22:18:50.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.786+0000 7fdb0c2a8a80 20 remove_watcher() i=2 2026-02-24T22:18:50.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.786+0000 7fdb0c2a8a80 20 remove_watcher() i=4 2026-02-24T22:18:50.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.786+0000 7fdb0c2a8a80 20 remove_watcher() i=5 2026-02-24T22:18:50.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.786+0000 7fdb0c2a8a80 20 remove_watcher() i=6 2026-02-24T22:18:50.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.786+0000 7fdb0c2a8a80 20 remove_watcher() i=7 2026-02-24T22:18:50.791 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2026-02-24T22:18:50.805 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:18:50.805 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:18:50.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.826+0000 7f263fccaa80 4 Realm: () 2026-02-24T22:18:50.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.826+0000 7f263fccaa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:50.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.826+0000 7f263fccaa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:50.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.826+0000 7f263fccaa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:50.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.826+0000 7f263fccaa80 20 zonegroup default 2026-02-24T22:18:50.825 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.826+0000 7f263fccaa80 20 rados->read ofs=0 len=0 2026-02-24T22:18:50.826 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.830+0000 7f263fccaa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:50.826 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.830+0000 7f263fccaa80 20 rados->read ofs=0 len=0 2026-02-24T22:18:50.827 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.830+0000 7f263fccaa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:50.827 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.830+0000 7f263fccaa80 20 started sync module instance, tier type = 2026-02-24T22:18:50.827 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.830+0000 7f263fccaa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:50.829 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.830+0000 7f263fccaa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:50.829 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.830+0000 7f263fccaa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:50.829 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.830+0000 7f263fccaa80 5 note: GC not initialized 2026-02-24T22:18:50.829 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.830+0000 7f25eafe5640 20 reqs_thread_entry: start 2026-02-24T22:18:50.832 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:50.834+0000 7f263df9f640 0 -- 10.20.193.46:0/3279638507 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55e6eb235030 msgr2=0x55e6eb2282a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:51.042 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.046+0000 7f263d79e640 0 -- 10.20.193.46:0/3279638507 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55e6eb230ab0 msgr2=0x55e6eb22b820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:51.069 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.070+0000 7f263fccaa80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:51.069 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.070+0000 7f263fccaa80 20 Filter name: none 2026-02-24T22:18:51.069 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.070+0000 7f25e97e2640 20 reqs_thread_entry: start 2026-02-24T22:18:51.069 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.070+0000 7f263fccaa80 20 rados->read ofs=0 len=0 2026-02-24T22:18:51.070 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.074+0000 7f263fccaa80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:18:51.070 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.074+0000 7f263fccaa80 20 rados->read ofs=0 len=0 2026-02-24T22:18:51.070 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.074+0000 7f263fccaa80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:18:51.070 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.074+0000 7f263fccaa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:51.070 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.074+0000 7f263fccaa80 20 rados->read ofs=0 len=0 2026-02-24T22:18:51.070 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.074+0000 7f263fccaa80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:18:51.070 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.074+0000 7f263fccaa80 20 rados->read ofs=0 len=0 2026-02-24T22:18:51.071 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.074+0000 7f263fccaa80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:18:51.071 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.074+0000 7f263fccaa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:51.080 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.081 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:18:51.081 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-commit-target-layout id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532 num_objects: 20 size_kb: 1 num_shards: 2 2026-02-24T22:18:51.081 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.082 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:51.082 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:51.084 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:51.084 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:51.126 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:51.126 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:51.137 INFO:tasks.workunit.client.0.trial046.stderr:TEST: reshard bucket with abort at commit_target_layout 2026-02-24T22:18:51.137 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.139 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:51.139 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:51.147 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:51.147 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:51.190 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:51.190 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:51.197 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:51.197 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:51.203 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:51.204 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:51.210 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:51.210 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:51.216 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:51.216 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:51.222 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:51.222 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:51.228 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:51.228 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:51.234 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:51.234 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:51.240 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:51.240 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:51.246 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:51.246 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:51.252 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:51.252 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:51.259 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:51.259 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:51.264 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:51.264 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:51.270 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:51.270 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:51.276 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:51.276 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:51.282 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:51.282 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:51.289 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:51.289 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:51.295 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:51.295 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:51.301 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:51.301 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:51.307 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:51.307 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:18:51.312 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket abort-at-commit-target-layout 2026-02-24T22:18:51.326 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:18:51.326 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:18:51.342 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.346+0000 7f1debfe5a80 4 Realm: () 2026-02-24T22:18:51.342 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.346+0000 7f1debfe5a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:51.342 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.346+0000 7f1debfe5a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:51.342 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.346+0000 7f1debfe5a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:51.342 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.346+0000 7f1debfe5a80 20 zonegroup default 2026-02-24T22:18:51.342 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.346+0000 7f1debfe5a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:51.343 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.346+0000 7f1debfe5a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:51.343 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.346+0000 7f1debfe5a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:51.344 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.346+0000 7f1debfe5a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:51.344 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.346+0000 7f1debfe5a80 20 started sync module instance, tier type = 2026-02-24T22:18:51.344 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.346+0000 7f1debfe5a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:51.345 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.350+0000 7f1debfe5a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:51.345 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.350+0000 7f1debfe5a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:51.346 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.350+0000 7f1debfe5a80 5 note: GC not initialized 2026-02-24T22:18:51.346 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.350+0000 7f1d0affd640 20 reqs_thread_entry: start 2026-02-24T22:18:51.350 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.354+0000 7f1dea2ba640 0 -- 10.20.193.46:0/1883579916 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55c623ba8860 msgr2=0x55c623b9bad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:51.355 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.358+0000 7f1de9ab9640 0 -- 10.20.193.46:0/1883579916 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55c623ba42e0 msgr2=0x55c623b9f050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:18:51.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.382+0000 7f1debfe5a80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:51.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.382+0000 7f1debfe5a80 20 Filter name: none 2026-02-24T22:18:51.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.382+0000 7f1d097fa640 20 reqs_thread_entry: start 2026-02-24T22:18:51.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.382+0000 7f1debfe5a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:51.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.382+0000 7f1debfe5a80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:18:51.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.382+0000 7f1debfe5a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:51.381 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.386+0000 7f1debfe5a80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:18:51.381 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.386+0000 7f1debfe5a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:51.381 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.386+0000 7f1debfe5a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:51.381 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.386+0000 7f1debfe5a80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:18:51.381 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.386+0000 7f1debfe5a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:51.382 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.386+0000 7f1debfe5a80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:18:51.382 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.386+0000 7f1debfe5a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:51.391 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.392 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:18:51.392 INFO:tasks.workunit.client.0.trial046.stderr:bucket: abort-at-commit-target-layout id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:18:51.392 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.392 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket abort-at-commit-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at commit_target_layout 2026-02-24T22:18:51.408 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:18:51.408 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:18:51.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.430+0000 7f3ffd6dda80 4 Realm: () 2026-02-24T22:18:51.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.430+0000 7f3ffd6dda80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:51.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.430+0000 7f3ffd6dda80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:51.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.430+0000 7f3ffd6dda80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:51.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.430+0000 7f3ffd6dda80 20 zonegroup default 2026-02-24T22:18:51.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.430+0000 7f3ffd6dda80 20 rados->read ofs=0 len=0 2026-02-24T22:18:51.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.430+0000 7f3ffd6dda80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:51.428 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.430+0000 7f3ffd6dda80 20 rados->read ofs=0 len=0 2026-02-24T22:18:51.428 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.430+0000 7f3ffd6dda80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:51.428 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.430+0000 7f3ffd6dda80 20 started sync module instance, tier type = 2026-02-24T22:18:51.428 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.430+0000 7f3ffd6dda80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:51.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.434+0000 7f3ff77fe640 0 -- 10.20.193.46:0/1574263675 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x556e149f3f20 msgr2=0x556e14a14300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:51.430 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 add_watcher() i=1 2026-02-24T22:18:51.430 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 add_watcher() i=6 2026-02-24T22:18:51.431 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 add_watcher() i=3 2026-02-24T22:18:51.431 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 add_watcher() i=0 2026-02-24T22:18:51.431 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 add_watcher() i=5 2026-02-24T22:18:51.431 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 add_watcher() i=2 2026-02-24T22:18:51.431 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 add_watcher() i=7 2026-02-24T22:18:51.431 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 add_watcher() i=4 2026-02-24T22:18:51.431 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.434+0000 7f3ffd6dda80 2 all 8 watchers are set, enabling cache 2026-02-24T22:18:51.432 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:51.432 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:51.432 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 note: GC not initialized 2026-02-24T22:18:51.433 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.434+0000 7f3f13fff640 20 reqs_thread_entry: start 2026-02-24T22:18:51.433 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.438+0000 7f3ff77fe640 0 -- 10.20.193.46:0/1786546684 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x556e14a873b0 msgr2=0x556e14a11ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:51.674 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.678+0000 7f3ffd6dda80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:51.674 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.678+0000 7f3ffd6dda80 20 Filter name: none 2026-02-24T22:18:51.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.678+0000 7f3f127fc640 20 reqs_thread_entry: start 2026-02-24T22:18:51.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2026-02-24T22:18:51.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.678+0000 7f3ffd6dda80 20 rados->read ofs=0 len=0 2026-02-24T22:18:51.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.678+0000 7f3ffd6dda80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:18:51.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2026-02-24T22:18:51.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2026-02-24T22:18:51.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 : miss 2026-02-24T22:18:51.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.678+0000 7f3ffd6dda80 20 rados->read ofs=0 len=0 2026-02-24T22:18:51.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.678+0000 7f3ffd6dda80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:18:51.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 info.flags=0x17 2026-02-24T22:18:51.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 to cache LRU end 2026-02-24T22:18:51.676 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:18:51.677 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:51.677 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 2026-02-24T22:18:51.680 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.682+0000 7f3ffd6dda80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 info.flags=0x17 2026-02-24T22:18:51.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.682+0000 7f3ffd6dda80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 to cache LRU end 2026-02-24T22:18:51.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.682+0000 7f3ffd6dda80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:51.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.682+0000 7f3ffd6dda80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533, ofs0, ns] 2026-02-24T22:18:51.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.682+0000 7f3fd5ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959913 cookie 93931282091568 notifier 7121 bl.length()=1033 2026-02-24T22:18:51.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.682+0000 7f3fd5ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 info.flags=0x17 2026-02-24T22:18:51.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.682+0000 7f3fd5ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 to cache LRU end 2026-02-24T22:18:51.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.682+0000 7f3fd5ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:51.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.682+0000 7f3ffd6dda80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): orig_sources= new_sources= 2026-02-24T22:18:51.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.682+0000 7f3ffd6dda80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): potential sources added=[] removed=[] 2026-02-24T22:18:51.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.682+0000 7f3ffd6dda80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): orig_dests= new_dests= 2026-02-24T22:18:51.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.682+0000 7f3ffd6dda80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): potential dests added=[] removed=[] 2026-02-24T22:18:51.682 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.686+0000 7f3ffd6dda80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533.1.0 2026-02-24T22:18:51.682 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.686+0000 7f3ffd6dda80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533.1.0 2026-02-24T22:18:51.683 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.686+0000 7f3ffd6dda80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533.1.1 2026-02-24T22:18:51.683 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.686+0000 7f3ffd6dda80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533.1.1 2026-02-24T22:18:51.683 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.686+0000 7f3ffd6dda80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533.0 2026-02-24T22:18:51.684 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.686+0000 7f3ffd6dda80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 : hit (requested=0x17, cached=0x17) 2026-02-24T22:18:51.684 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.686+0000 7f3ffd6dda80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 2026-02-24T22:18:51.685 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.690+0000 7f3ffd6dda80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 info.flags=0x17 2026-02-24T22:18:51.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.690+0000 7f3ffd6dda80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 to cache LRU end 2026-02-24T22:18:51.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.690+0000 7f3ffd6dda80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:51.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.690+0000 7f3ffd6dda80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533, ofs0, ns] 2026-02-24T22:18:51.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.690+0000 7f3fd5ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959914 cookie 93931282091568 notifier 7121 bl.length()=1033 2026-02-24T22:18:51.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.690+0000 7f3fd5ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 info.flags=0x17 2026-02-24T22:18:51.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.690+0000 7f3fd5ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 to cache LRU end 2026-02-24T22:18:51.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.690+0000 7f3fd5ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:51.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.690+0000 7f3ffd6dda80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): orig_sources= new_sources= 2026-02-24T22:18:51.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.690+0000 7f3ffd6dda80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): potential sources added=[] removed=[] 2026-02-24T22:18:51.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.690+0000 7f3ffd6dda80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): orig_dests= new_dests= 2026-02-24T22:18:51.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.690+0000 7f3ffd6dda80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): potential dests added=[] removed=[] 2026-02-24T22:18:51.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.690+0000 7f3ffd6dda80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533.0 2026-02-24T22:18:51.688 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f3ffd6dda80 time 2026-02-24T22:18:51.693432+0000 2026-02-24T22:18:51.688 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:18:51.688 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:18:51.688 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f4000b1feb1] 2026-02-24T22:18:51.688 INFO:tasks.workunit.client.0.trial046.stderr: 2: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xce6) [0x556df6828de6] 2026-02-24T22:18:51.688 INFO:tasks.workunit.client.0.trial046.stderr: 3: main() 2026-02-24T22:18:51.688 INFO:tasks.workunit.client.0.trial046.stderr: 4: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f3fffe1fd90] 2026-02-24T22:18:51.688 INFO:tasks.workunit.client.0.trial046.stderr: 5: __libc_start_main() 2026-02-24T22:18:51.688 INFO:tasks.workunit.client.0.trial046.stderr: 6: _start() 2026-02-24T22:18:51.688 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.690+0000 7f3ffd6dda80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f3ffd6dda80 time 2026-02-24T22:18:51.693432+0000 2026-02-24T22:18:51.689 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:18:51.689 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.689 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:18:51.689 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f4000b1feb1] 2026-02-24T22:18:51.689 INFO:tasks.workunit.client.0.trial046.stderr: 2: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xce6) [0x556df6828de6] 2026-02-24T22:18:51.689 INFO:tasks.workunit.client.0.trial046.stderr: 3: main() 2026-02-24T22:18:51.689 INFO:tasks.workunit.client.0.trial046.stderr: 4: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f3fffe1fd90] 2026-02-24T22:18:51.689 INFO:tasks.workunit.client.0.trial046.stderr: 5: __libc_start_main() 2026-02-24T22:18:51.689 INFO:tasks.workunit.client.0.trial046.stderr: 6: _start() 2026-02-24T22:18:51.689 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.689 INFO:tasks.workunit.client.0.trial046.stderr:*** Caught signal (Aborted) ** 2026-02-24T22:18:51.689 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f3ffd6dda80 thread_name:radosgw-admin 2026-02-24T22:18:51.713 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:18:51.713 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f3fffe38520] 2026-02-24T22:18:51.713 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:18:51.713 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:18:51.713 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:18:51.713 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f4000b1ff10] 2026-02-24T22:18:51.713 INFO:tasks.workunit.client.0.trial046.stderr: 6: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xce6) [0x556df6828de6] 2026-02-24T22:18:51.713 INFO:tasks.workunit.client.0.trial046.stderr: 7: main() 2026-02-24T22:18:51.713 INFO:tasks.workunit.client.0.trial046.stderr: 8: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f3fffe1fd90] 2026-02-24T22:18:51.714 INFO:tasks.workunit.client.0.trial046.stderr: 9: __libc_start_main() 2026-02-24T22:18:51.714 INFO:tasks.workunit.client.0.trial046.stderr: 10: _start() 2026-02-24T22:18:51.714 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.690+0000 7f3ffd6dda80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:18:51.714 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f3ffd6dda80 thread_name:radosgw-admin 2026-02-24T22:18:51.714 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.714 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:18:51.714 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f3fffe38520] 2026-02-24T22:18:51.714 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:18:51.714 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:18:51.714 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:18:51.714 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f4000b1ff10] 2026-02-24T22:18:51.714 INFO:tasks.workunit.client.0.trial046.stderr: 6: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xce6) [0x556df6828de6] 2026-02-24T22:18:51.714 INFO:tasks.workunit.client.0.trial046.stderr: 7: main() 2026-02-24T22:18:51.715 INFO:tasks.workunit.client.0.trial046.stderr: 8: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f3fffe1fd90] 2026-02-24T22:18:51.715 INFO:tasks.workunit.client.0.trial046.stderr: 9: __libc_start_main() 2026-02-24T22:18:51.715 INFO:tasks.workunit.client.0.trial046.stderr: 10: _start() 2026-02-24T22:18:51.715 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:18:51.715 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.715 INFO:tasks.workunit.client.0.trial046.stderr:--- begin dump of recent events --- 2026-02-24T22:18:51.715 INFO:tasks.workunit.client.0.trial046.stderr: -499> 2026-02-24T22:18:51.418+0000 7f3feffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:51.715 INFO:tasks.workunit.client.0.trial046.stderr: -498> 2026-02-24T22:18:51.418+0000 7f3feffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:51.715 INFO:tasks.workunit.client.0.trial046.stderr: -497> 2026-02-24T22:18:51.418+0000 7f3feffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:51.715 INFO:tasks.workunit.client.0.trial046.stderr: -496> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_done global_id 7608 payload 274 2026-02-24T22:18:51.715 INFO:tasks.workunit.client.0.trial046.stderr: -495> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 10 monclient: _finish_hunting 0 2026-02-24T22:18:51.715 INFO:tasks.workunit.client.0.trial046.stderr: -494> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 1 monclient: found mon.noname-a 2026-02-24T22:18:51.715 INFO:tasks.workunit.client.0.trial046.stderr: -493> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:18:51.715 INFO:tasks.workunit.client.0.trial046.stderr: -492> 2026-02-24T22:18:51.418+0000 7f3fed7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:51.715 INFO:tasks.workunit.client.0.trial046.stderr: -491> 2026-02-24T22:18:51.418+0000 7f3fed7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:18:51.716 INFO:tasks.workunit.client.0.trial046.stderr: -490> 2026-02-24T22:18:51.418+0000 7f3fed7fa640 10 monclient: dump: 2026-02-24T22:18:51.716 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:51.716 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:51.716 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.716 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.716 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:51.716 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.716 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:51.716 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:51.716 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:51.716 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.716 INFO:tasks.workunit.client.0.trial046.stderr: -489> 2026-02-24T22:18:51.418+0000 7f3fed7fa640 10 monclient: _finish_auth 0 2026-02-24T22:18:51.716 INFO:tasks.workunit.client.0.trial046.stderr: -488> 2026-02-24T22:18:51.418+0000 7f3fed7fa640 10 monclient: _check_auth_tickets 2026-02-24T22:18:51.716 INFO:tasks.workunit.client.0.trial046.stderr: -487> 2026-02-24T22:18:51.418+0000 7f3fed7fa640 10 monclient: handle_config config(0 keys) 2026-02-24T22:18:51.717 INFO:tasks.workunit.client.0.trial046.stderr: -486> 2026-02-24T22:18:51.418+0000 7f3fed7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:51.717 INFO:tasks.workunit.client.0.trial046.stderr: -485> 2026-02-24T22:18:51.418+0000 7f3fed7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:18:51.717 INFO:tasks.workunit.client.0.trial046.stderr: -484> 2026-02-24T22:18:51.418+0000 7f3fed7fa640 10 monclient: dump: 2026-02-24T22:18:51.717 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:51.717 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:51.717 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.717 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.717 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:51.717 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.717 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:51.717 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:51.717 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:51.717 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.717 INFO:tasks.workunit.client.0.trial046.stderr: -483> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 monclient: authenticate success, global_id 7608 2026-02-24T22:18:51.718 INFO:tasks.workunit.client.0.trial046.stderr: -482> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient: get_monmap_and_config success 2026-02-24T22:18:51.718 INFO:tasks.workunit.client.0.trial046.stderr: -481> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 4 set_mon_vals no callback set 2026-02-24T22:18:51.718 INFO:tasks.workunit.client.0.trial046.stderr: -480> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient: shutdown 2026-02-24T22:18:51.718 INFO:tasks.workunit.client.0.trial046.stderr: -479> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 asok(0x556e14509f00) unregister_commands rotate-key 2026-02-24T22:18:51.718 INFO:tasks.workunit.client.0.trial046.stderr: -478> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient: build_initial_monmap 2026-02-24T22:18:51.718 INFO:tasks.workunit.client.0.trial046.stderr: -477> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 1 build_initial for_mkfs: 0 2026-02-24T22:18:51.718 INFO:tasks.workunit.client.0.trial046.stderr: -476> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient: monmap: 2026-02-24T22:18:51.718 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:18:51.718 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:18:51.718 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:18:51.718 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:18:51.718 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:18:51.718 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.718 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:18:51.719 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:18:51.719 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:18:51.719 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.719 INFO:tasks.workunit.client.0.trial046.stderr: -475> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding auth protocol: cephx 2026-02-24T22:18:51.719 INFO:tasks.workunit.client.0.trial046.stderr: -474> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding auth protocol: cephx 2026-02-24T22:18:51.719 INFO:tasks.workunit.client.0.trial046.stderr: -473> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding auth protocol: cephx 2026-02-24T22:18:51.719 INFO:tasks.workunit.client.0.trial046.stderr: -472> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding con mode: secure 2026-02-24T22:18:51.719 INFO:tasks.workunit.client.0.trial046.stderr: -471> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding con mode: crc 2026-02-24T22:18:51.719 INFO:tasks.workunit.client.0.trial046.stderr: -470> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding con mode: secure 2026-02-24T22:18:51.719 INFO:tasks.workunit.client.0.trial046.stderr: -469> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding con mode: crc 2026-02-24T22:18:51.719 INFO:tasks.workunit.client.0.trial046.stderr: -468> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding con mode: secure 2026-02-24T22:18:51.719 INFO:tasks.workunit.client.0.trial046.stderr: -467> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding con mode: crc 2026-02-24T22:18:51.719 INFO:tasks.workunit.client.0.trial046.stderr: -466> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding con mode: crc 2026-02-24T22:18:51.719 INFO:tasks.workunit.client.0.trial046.stderr: -465> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding con mode: secure 2026-02-24T22:18:51.719 INFO:tasks.workunit.client.0.trial046.stderr: -464> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding con mode: crc 2026-02-24T22:18:51.720 INFO:tasks.workunit.client.0.trial046.stderr: -463> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding con mode: secure 2026-02-24T22:18:51.720 INFO:tasks.workunit.client.0.trial046.stderr: -462> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding con mode: crc 2026-02-24T22:18:51.720 INFO:tasks.workunit.client.0.trial046.stderr: -461> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding con mode: secure 2026-02-24T22:18:51.720 INFO:tasks.workunit.client.0.trial046.stderr: -460> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:51.720 INFO:tasks.workunit.client.0.trial046.stderr: -459> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 1 librados: starting msgr at 2026-02-24T22:18:51.720 INFO:tasks.workunit.client.0.trial046.stderr: -458> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 1 librados: starting objecter 2026-02-24T22:18:51.720 INFO:tasks.workunit.client.0.trial046.stderr: -457> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command objecter_requests hook 0x556e1466ab90 2026-02-24T22:18:51.720 INFO:tasks.workunit.client.0.trial046.stderr: -456> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 1 librados: setting wanted keys 2026-02-24T22:18:51.720 INFO:tasks.workunit.client.0.trial046.stderr: -455> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 1 librados: calling monclient init 2026-02-24T22:18:51.720 INFO:tasks.workunit.client.0.trial046.stderr: -454> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient: init 2026-02-24T22:18:51.720 INFO:tasks.workunit.client.0.trial046.stderr: -453> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding auth protocol: cephx 2026-02-24T22:18:51.720 INFO:tasks.workunit.client.0.trial046.stderr: -452> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding auth protocol: cephx 2026-02-24T22:18:51.720 INFO:tasks.workunit.client.0.trial046.stderr: -451> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding auth protocol: cephx 2026-02-24T22:18:51.720 INFO:tasks.workunit.client.0.trial046.stderr: -450> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding con mode: secure 2026-02-24T22:18:51.720 INFO:tasks.workunit.client.0.trial046.stderr: -449> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding con mode: crc 2026-02-24T22:18:51.721 INFO:tasks.workunit.client.0.trial046.stderr: -448> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding con mode: secure 2026-02-24T22:18:51.721 INFO:tasks.workunit.client.0.trial046.stderr: -447> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding con mode: crc 2026-02-24T22:18:51.721 INFO:tasks.workunit.client.0.trial046.stderr: -446> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding con mode: secure 2026-02-24T22:18:51.721 INFO:tasks.workunit.client.0.trial046.stderr: -445> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding con mode: crc 2026-02-24T22:18:51.721 INFO:tasks.workunit.client.0.trial046.stderr: -444> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding con mode: crc 2026-02-24T22:18:51.721 INFO:tasks.workunit.client.0.trial046.stderr: -443> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding con mode: secure 2026-02-24T22:18:51.721 INFO:tasks.workunit.client.0.trial046.stderr: -442> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding con mode: crc 2026-02-24T22:18:51.721 INFO:tasks.workunit.client.0.trial046.stderr: -441> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding con mode: secure 2026-02-24T22:18:51.721 INFO:tasks.workunit.client.0.trial046.stderr: -440> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding con mode: crc 2026-02-24T22:18:51.721 INFO:tasks.workunit.client.0.trial046.stderr: -439> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding con mode: secure 2026-02-24T22:18:51.721 INFO:tasks.workunit.client.0.trial046.stderr: -438> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:51.721 INFO:tasks.workunit.client.0.trial046.stderr: -437> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:51.721 INFO:tasks.workunit.client.0.trial046.stderr: -436> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command rotate-key hook 0x556e14609918 2026-02-24T22:18:51.721 INFO:tasks.workunit.client.0.trial046.stderr: -435> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient: _reopen_session rank -1 2026-02-24T22:18:51.721 INFO:tasks.workunit.client.0.trial046.stderr: -434> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T22:18:51.722 INFO:tasks.workunit.client.0.trial046.stderr: -433> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient(hunting): picked mon.noname-a con 0x556e14597fe0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:18:51.722 INFO:tasks.workunit.client.0.trial046.stderr: -432> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient(hunting): picked mon.noname-b con 0x556e14640b00 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:18:51.722 INFO:tasks.workunit.client.0.trial046.stderr: -431> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient(hunting): picked mon.noname-c con 0x556e14643cf0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:18:51.722 INFO:tasks.workunit.client.0.trial046.stderr: -430> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:51.722 INFO:tasks.workunit.client.0.trial046.stderr: -429> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:51.722 INFO:tasks.workunit.client.0.trial046.stderr: -428> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:51.722 INFO:tasks.workunit.client.0.trial046.stderr: -427> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient(hunting): _renew_subs 2026-02-24T22:18:51.722 INFO:tasks.workunit.client.0.trial046.stderr: -426> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient(hunting): authenticate will time out at 1901.525863s 2026-02-24T22:18:51.722 INFO:tasks.workunit.client.0.trial046.stderr: -425> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 10 monclient(hunting): get_auth_request con 0x556e14597fe0 auth_method 0 2026-02-24T22:18:51.722 INFO:tasks.workunit.client.0.trial046.stderr: -424> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:51.722 INFO:tasks.workunit.client.0.trial046.stderr: -423> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:51.722 INFO:tasks.workunit.client.0.trial046.stderr: -422> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:51.722 INFO:tasks.workunit.client.0.trial046.stderr: -421> 2026-02-24T22:18:51.418+0000 7f3ff77fe640 10 monclient(hunting): get_auth_request con 0x556e14643cf0 auth_method 0 2026-02-24T22:18:51.722 INFO:tasks.workunit.client.0.trial046.stderr: -420> 2026-02-24T22:18:51.418+0000 7f3ff77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:51.722 INFO:tasks.workunit.client.0.trial046.stderr: -419> 2026-02-24T22:18:51.418+0000 7f3ff77fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:51.722 INFO:tasks.workunit.client.0.trial046.stderr: -418> 2026-02-24T22:18:51.418+0000 7f3ff77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:51.723 INFO:tasks.workunit.client.0.trial046.stderr: -417> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:51.723 INFO:tasks.workunit.client.0.trial046.stderr: -416> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:51.723 INFO:tasks.workunit.client.0.trial046.stderr: -415> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:51.723 INFO:tasks.workunit.client.0.trial046.stderr: -414> 2026-02-24T22:18:51.418+0000 7f3ff77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:51.723 INFO:tasks.workunit.client.0.trial046.stderr: -413> 2026-02-24T22:18:51.418+0000 7f3ff77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:51.723 INFO:tasks.workunit.client.0.trial046.stderr: -412> 2026-02-24T22:18:51.418+0000 7f3ff77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:51.723 INFO:tasks.workunit.client.0.trial046.stderr: -411> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_done global_id 7611 payload 824 2026-02-24T22:18:51.723 INFO:tasks.workunit.client.0.trial046.stderr: -410> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 10 monclient: _finish_hunting 0 2026-02-24T22:18:51.723 INFO:tasks.workunit.client.0.trial046.stderr: -409> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 1 monclient: found mon.noname-a 2026-02-24T22:18:51.723 INFO:tasks.workunit.client.0.trial046.stderr: -408> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:18:51.723 INFO:tasks.workunit.client.0.trial046.stderr: -407> 2026-02-24T22:18:51.418+0000 7f3fdf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:51.723 INFO:tasks.workunit.client.0.trial046.stderr: -406> 2026-02-24T22:18:51.418+0000 7f3fdf7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:18:51.723 INFO:tasks.workunit.client.0.trial046.stderr: -405> 2026-02-24T22:18:51.418+0000 7f3fdf7fe640 10 monclient: dump: 2026-02-24T22:18:51.723 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:51.723 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:51.723 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.723 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.724 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:51.724 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.724 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:51.724 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:51.724 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:51.724 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.724 INFO:tasks.workunit.client.0.trial046.stderr: -404> 2026-02-24T22:18:51.418+0000 7f3fdf7fe640 10 monclient: _finish_auth 0 2026-02-24T22:18:51.724 INFO:tasks.workunit.client.0.trial046.stderr: -403> 2026-02-24T22:18:51.418+0000 7f3fdf7fe640 10 monclient: _check_auth_tickets 2026-02-24T22:18:51.724 INFO:tasks.workunit.client.0.trial046.stderr: -402> 2026-02-24T22:18:51.418+0000 7f3fdf7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T22:18:51.724 INFO:tasks.workunit.client.0.trial046.stderr: -401> 2026-02-24T22:18:51.418+0000 7f3fdf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:51.724 INFO:tasks.workunit.client.0.trial046.stderr: -400> 2026-02-24T22:18:51.418+0000 7f3fdf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:18:51.724 INFO:tasks.workunit.client.0.trial046.stderr: -399> 2026-02-24T22:18:51.418+0000 7f3fdf7fe640 10 monclient: dump: 2026-02-24T22:18:51.724 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:51.724 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:51.724 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.724 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.724 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:51.725 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.725 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:51.725 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:51.725 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:51.725 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.725 INFO:tasks.workunit.client.0.trial046.stderr: -398> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 monclient: authenticate success, global_id 7611 2026-02-24T22:18:51.725 INFO:tasks.workunit.client.0.trial046.stderr: -397> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient: _renew_subs 2026-02-24T22:18:51.725 INFO:tasks.workunit.client.0.trial046.stderr: -396> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:18:51.725 INFO:tasks.workunit.client.0.trial046.stderr: -395> 2026-02-24T22:18:51.418+0000 7f3fed7fa640 4 set_mon_vals no callback set 2026-02-24T22:18:51.725 INFO:tasks.workunit.client.0.trial046.stderr: -394> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient: _renew_subs 2026-02-24T22:18:51.725 INFO:tasks.workunit.client.0.trial046.stderr: -393> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:18:51.725 INFO:tasks.workunit.client.0.trial046.stderr: -392> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 1 librados: init done 2026-02-24T22:18:51.725 INFO:tasks.workunit.client.0.trial046.stderr: -391> 2026-02-24T22:18:51.418+0000 7f3fdf7fe640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:18:51.725 INFO:tasks.workunit.client.0.trial046.stderr: -390> 2026-02-24T22:18:51.418+0000 7f3fdf7fe640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:18:51.725 INFO:tasks.workunit.client.0.trial046.stderr: -389> 2026-02-24T22:18:51.418+0000 7f3fdf7fe640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:18:51.725 INFO:tasks.workunit.client.0.trial046.stderr: -388> 2026-02-24T22:18:51.418+0000 7f3feffff640 10 monclient: get_auth_request con 0x7f3fd80707b0 auth_method 0 2026-02-24T22:18:51.725 INFO:tasks.workunit.client.0.trial046.stderr: -387> 2026-02-24T22:18:51.422+0000 7f3ff77fe640 10 monclient: get_auth_request con 0x556e14635620 auth_method 0 2026-02-24T22:18:51.725 INFO:tasks.workunit.client.0.trial046.stderr: -386> 2026-02-24T22:18:51.422+0000 7f3ff6ffd640 10 monclient: get_auth_request con 0x556e14695a40 auth_method 0 2026-02-24T22:18:51.726 INFO:tasks.workunit.client.0.trial046.stderr: -385> 2026-02-24T22:18:51.422+0000 7f3feffff640 10 monclient: get_auth_request con 0x556e14697080 auth_method 0 2026-02-24T22:18:51.726 INFO:tasks.workunit.client.0.trial046.stderr: -384> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 10 monclient: build_initial_monmap 2026-02-24T22:18:51.726 INFO:tasks.workunit.client.0.trial046.stderr: -383> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 1 build_initial for_mkfs: 0 2026-02-24T22:18:51.726 INFO:tasks.workunit.client.0.trial046.stderr: -382> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 10 monclient: monmap: 2026-02-24T22:18:51.726 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:18:51.726 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:18:51.726 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:18:51.726 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:18:51.726 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:18:51.726 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.726 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:18:51.726 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:18:51.726 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:18:51.726 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.726 INFO:tasks.workunit.client.0.trial046.stderr: -381> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding auth protocol: cephx 2026-02-24T22:18:51.726 INFO:tasks.workunit.client.0.trial046.stderr: -380> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding auth protocol: cephx 2026-02-24T22:18:51.726 INFO:tasks.workunit.client.0.trial046.stderr: -379> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding auth protocol: cephx 2026-02-24T22:18:51.726 INFO:tasks.workunit.client.0.trial046.stderr: -378> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding con mode: secure 2026-02-24T22:18:51.727 INFO:tasks.workunit.client.0.trial046.stderr: -377> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding con mode: crc 2026-02-24T22:18:51.727 INFO:tasks.workunit.client.0.trial046.stderr: -376> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding con mode: secure 2026-02-24T22:18:51.727 INFO:tasks.workunit.client.0.trial046.stderr: -375> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding con mode: crc 2026-02-24T22:18:51.727 INFO:tasks.workunit.client.0.trial046.stderr: -374> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding con mode: secure 2026-02-24T22:18:51.727 INFO:tasks.workunit.client.0.trial046.stderr: -373> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding con mode: crc 2026-02-24T22:18:51.727 INFO:tasks.workunit.client.0.trial046.stderr: -372> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding con mode: crc 2026-02-24T22:18:51.727 INFO:tasks.workunit.client.0.trial046.stderr: -371> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding con mode: secure 2026-02-24T22:18:51.727 INFO:tasks.workunit.client.0.trial046.stderr: -370> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding con mode: crc 2026-02-24T22:18:51.727 INFO:tasks.workunit.client.0.trial046.stderr: -369> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding con mode: secure 2026-02-24T22:18:51.727 INFO:tasks.workunit.client.0.trial046.stderr: -368> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding con mode: crc 2026-02-24T22:18:51.727 INFO:tasks.workunit.client.0.trial046.stderr: -367> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding con mode: secure 2026-02-24T22:18:51.727 INFO:tasks.workunit.client.0.trial046.stderr: -366> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:51.727 INFO:tasks.workunit.client.0.trial046.stderr: -365> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command objecter_requests cmddesc objecter_requests hook 0x556e1469ec30 EEXIST 2026-02-24T22:18:51.727 INFO:tasks.workunit.client.0.trial046.stderr: -364> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 10 monclient: init 2026-02-24T22:18:51.727 INFO:tasks.workunit.client.0.trial046.stderr: -363> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding auth protocol: cephx 2026-02-24T22:18:51.727 INFO:tasks.workunit.client.0.trial046.stderr: -362> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding auth protocol: cephx 2026-02-24T22:18:51.727 INFO:tasks.workunit.client.0.trial046.stderr: -361> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding auth protocol: cephx 2026-02-24T22:18:51.727 INFO:tasks.workunit.client.0.trial046.stderr: -360> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding con mode: secure 2026-02-24T22:18:51.727 INFO:tasks.workunit.client.0.trial046.stderr: -359> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding con mode: crc 2026-02-24T22:18:51.728 INFO:tasks.workunit.client.0.trial046.stderr: -358> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding con mode: secure 2026-02-24T22:18:51.728 INFO:tasks.workunit.client.0.trial046.stderr: -357> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding con mode: crc 2026-02-24T22:18:51.728 INFO:tasks.workunit.client.0.trial046.stderr: -356> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding con mode: secure 2026-02-24T22:18:51.728 INFO:tasks.workunit.client.0.trial046.stderr: -355> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding con mode: crc 2026-02-24T22:18:51.728 INFO:tasks.workunit.client.0.trial046.stderr: -354> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding con mode: crc 2026-02-24T22:18:51.728 INFO:tasks.workunit.client.0.trial046.stderr: -353> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding con mode: secure 2026-02-24T22:18:51.728 INFO:tasks.workunit.client.0.trial046.stderr: -352> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding con mode: crc 2026-02-24T22:18:51.728 INFO:tasks.workunit.client.0.trial046.stderr: -351> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding con mode: secure 2026-02-24T22:18:51.728 INFO:tasks.workunit.client.0.trial046.stderr: -350> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding con mode: crc 2026-02-24T22:18:51.728 INFO:tasks.workunit.client.0.trial046.stderr: -349> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding con mode: secure 2026-02-24T22:18:51.728 INFO:tasks.workunit.client.0.trial046.stderr: -348> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:51.728 INFO:tasks.workunit.client.0.trial046.stderr: -347> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:51.728 INFO:tasks.workunit.client.0.trial046.stderr: -346> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command rotate-key cmddesc rotate-key hook 0x556e14697da0 EEXIST 2026-02-24T22:18:51.728 INFO:tasks.workunit.client.0.trial046.stderr: -345> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 10 monclient: _reopen_session rank -1 2026-02-24T22:18:51.728 INFO:tasks.workunit.client.0.trial046.stderr: -344> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:18:51.728 INFO:tasks.workunit.client.0.trial046.stderr: -343> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 10 monclient(hunting): picked mon.noname-c con 0x556e147f2490 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:18:51.728 INFO:tasks.workunit.client.0.trial046.stderr: -342> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 10 monclient(hunting): picked mon.noname-a con 0x556e147ef630 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:18:51.728 INFO:tasks.workunit.client.0.trial046.stderr: -341> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 10 monclient(hunting): picked mon.noname-b con 0x556e147eb610 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:18:51.728 INFO:tasks.workunit.client.0.trial046.stderr: -340> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:51.728 INFO:tasks.workunit.client.0.trial046.stderr: -339> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:51.729 INFO:tasks.workunit.client.0.trial046.stderr: -338> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:51.729 INFO:tasks.workunit.client.0.trial046.stderr: -337> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 10 monclient(hunting): _renew_subs 2026-02-24T22:18:51.729 INFO:tasks.workunit.client.0.trial046.stderr: -336> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 10 monclient(hunting): authenticate will time out at 1901.531774s 2026-02-24T22:18:51.729 INFO:tasks.workunit.client.0.trial046.stderr: -335> 2026-02-24T22:18:51.422+0000 7f3feffff640 10 monclient(hunting): get_auth_request con 0x556e147ef630 auth_method 0 2026-02-24T22:18:51.729 INFO:tasks.workunit.client.0.trial046.stderr: -334> 2026-02-24T22:18:51.422+0000 7f3feffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:51.729 INFO:tasks.workunit.client.0.trial046.stderr: -333> 2026-02-24T22:18:51.422+0000 7f3feffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:51.729 INFO:tasks.workunit.client.0.trial046.stderr: -332> 2026-02-24T22:18:51.422+0000 7f3feffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:51.729 INFO:tasks.workunit.client.0.trial046.stderr: -331> 2026-02-24T22:18:51.422+0000 7f3ff6ffd640 10 monclient(hunting): get_auth_request con 0x556e147f2490 auth_method 0 2026-02-24T22:18:51.729 INFO:tasks.workunit.client.0.trial046.stderr: -330> 2026-02-24T22:18:51.422+0000 7f3ff6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:51.729 INFO:tasks.workunit.client.0.trial046.stderr: -329> 2026-02-24T22:18:51.422+0000 7f3ff6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:51.729 INFO:tasks.workunit.client.0.trial046.stderr: -328> 2026-02-24T22:18:51.422+0000 7f3ff6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:51.729 INFO:tasks.workunit.client.0.trial046.stderr: -327> 2026-02-24T22:18:51.422+0000 7f3feffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:51.729 INFO:tasks.workunit.client.0.trial046.stderr: -326> 2026-02-24T22:18:51.422+0000 7f3feffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:51.729 INFO:tasks.workunit.client.0.trial046.stderr: -325> 2026-02-24T22:18:51.422+0000 7f3feffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:51.729 INFO:tasks.workunit.client.0.trial046.stderr: -324> 2026-02-24T22:18:51.422+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:51.729 INFO:tasks.workunit.client.0.trial046.stderr: -323> 2026-02-24T22:18:51.422+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:51.729 INFO:tasks.workunit.client.0.trial046.stderr: -322> 2026-02-24T22:18:51.422+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:51.729 INFO:tasks.workunit.client.0.trial046.stderr: -321> 2026-02-24T22:18:51.422+0000 7f3feffff640 10 monclient(hunting): handle_auth_done global_id 7614 payload 824 2026-02-24T22:18:51.729 INFO:tasks.workunit.client.0.trial046.stderr: -320> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient: _finish_hunting 0 2026-02-24T22:18:51.729 INFO:tasks.workunit.client.0.trial046.stderr: -319> 2026-02-24T22:18:51.426+0000 7f3feffff640 1 monclient: found mon.noname-a 2026-02-24T22:18:51.730 INFO:tasks.workunit.client.0.trial046.stderr: -318> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:18:51.730 INFO:tasks.workunit.client.0.trial046.stderr: -317> 2026-02-24T22:18:51.426+0000 7f3ff77fe640 10 monclient: get_auth_request con 0x556e147eb610 auth_method 0 2026-02-24T22:18:51.730 INFO:tasks.workunit.client.0.trial046.stderr: -316> 2026-02-24T22:18:51.426+0000 7f3fdcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:51.730 INFO:tasks.workunit.client.0.trial046.stderr: -315> 2026-02-24T22:18:51.426+0000 7f3fdcff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:18:51.730 INFO:tasks.workunit.client.0.trial046.stderr: -314> 2026-02-24T22:18:51.426+0000 7f3fdcff9640 10 monclient: dump: 2026-02-24T22:18:51.730 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:51.730 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:51.730 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.730 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.730 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:51.730 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.730 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:51.730 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:51.730 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:51.730 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.730 INFO:tasks.workunit.client.0.trial046.stderr: -313> 2026-02-24T22:18:51.426+0000 7f3fdcff9640 10 monclient: _finish_auth 0 2026-02-24T22:18:51.730 INFO:tasks.workunit.client.0.trial046.stderr: -312> 2026-02-24T22:18:51.426+0000 7f3fdcff9640 10 monclient: _check_auth_tickets 2026-02-24T22:18:51.730 INFO:tasks.workunit.client.0.trial046.stderr: -311> 2026-02-24T22:18:51.426+0000 7f3fdcff9640 10 monclient: handle_config config(0 keys) 2026-02-24T22:18:51.730 INFO:tasks.workunit.client.0.trial046.stderr: -310> 2026-02-24T22:18:51.426+0000 7f3fdcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:51.730 INFO:tasks.workunit.client.0.trial046.stderr: -309> 2026-02-24T22:18:51.426+0000 7f3fdcff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:18:51.730 INFO:tasks.workunit.client.0.trial046.stderr: -308> 2026-02-24T22:18:51.426+0000 7f3fdcff9640 10 monclient: dump: 2026-02-24T22:18:51.731 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:51.731 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:51.731 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.731 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.731 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:51.731 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.731 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:51.731 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:51.731 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:51.731 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.731 INFO:tasks.workunit.client.0.trial046.stderr: -307> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 monclient: authenticate success, global_id 7614 2026-02-24T22:18:51.731 INFO:tasks.workunit.client.0.trial046.stderr: -306> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: _renew_subs 2026-02-24T22:18:51.731 INFO:tasks.workunit.client.0.trial046.stderr: -305> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:18:51.731 INFO:tasks.workunit.client.0.trial046.stderr: -304> 2026-02-24T22:18:51.426+0000 7f3ff7fff640 4 set_mon_vals no callback set 2026-02-24T22:18:51.731 INFO:tasks.workunit.client.0.trial046.stderr: -303> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: _renew_subs 2026-02-24T22:18:51.731 INFO:tasks.workunit.client.0.trial046.stderr: -302> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:18:51.731 INFO:tasks.workunit.client.0.trial046.stderr: -301> 2026-02-24T22:18:51.426+0000 7f3fdcff9640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:18:51.731 INFO:tasks.workunit.client.0.trial046.stderr: -300> 2026-02-24T22:18:51.426+0000 7f3fdcff9640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:18:51.731 INFO:tasks.workunit.client.0.trial046.stderr: -299> 2026-02-24T22:18:51.426+0000 7f3fdcff9640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:18:51.731 INFO:tasks.workunit.client.0.trial046.stderr: -298> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient: get_auth_request con 0x7f3fcc0706c0 auth_method 0 2026-02-24T22:18:51.731 INFO:tasks.workunit.client.0.trial046.stderr: -297> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: get_monmap_and_config 2026-02-24T22:18:51.732 INFO:tasks.workunit.client.0.trial046.stderr: -296> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: build_initial_monmap 2026-02-24T22:18:51.732 INFO:tasks.workunit.client.0.trial046.stderr: -295> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 1 build_initial for_mkfs: 0 2026-02-24T22:18:51.732 INFO:tasks.workunit.client.0.trial046.stderr: -294> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: monmap: 2026-02-24T22:18:51.732 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:18:51.732 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:18:51.732 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:18:51.732 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:18:51.732 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:18:51.732 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.732 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:18:51.732 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:18:51.732 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:18:51.732 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.732 INFO:tasks.workunit.client.0.trial046.stderr: -293> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding auth protocol: cephx 2026-02-24T22:18:51.732 INFO:tasks.workunit.client.0.trial046.stderr: -292> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding auth protocol: cephx 2026-02-24T22:18:51.732 INFO:tasks.workunit.client.0.trial046.stderr: -291> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding auth protocol: cephx 2026-02-24T22:18:51.732 INFO:tasks.workunit.client.0.trial046.stderr: -290> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding con mode: secure 2026-02-24T22:18:51.732 INFO:tasks.workunit.client.0.trial046.stderr: -289> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding con mode: crc 2026-02-24T22:18:51.732 INFO:tasks.workunit.client.0.trial046.stderr: -288> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding con mode: secure 2026-02-24T22:18:51.732 INFO:tasks.workunit.client.0.trial046.stderr: -287> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding con mode: crc 2026-02-24T22:18:51.732 INFO:tasks.workunit.client.0.trial046.stderr: -286> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding con mode: secure 2026-02-24T22:18:51.732 INFO:tasks.workunit.client.0.trial046.stderr: -285> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding con mode: crc 2026-02-24T22:18:51.733 INFO:tasks.workunit.client.0.trial046.stderr: -284> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding con mode: crc 2026-02-24T22:18:51.733 INFO:tasks.workunit.client.0.trial046.stderr: -283> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding con mode: secure 2026-02-24T22:18:51.733 INFO:tasks.workunit.client.0.trial046.stderr: -282> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding con mode: crc 2026-02-24T22:18:51.733 INFO:tasks.workunit.client.0.trial046.stderr: -281> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding con mode: secure 2026-02-24T22:18:51.733 INFO:tasks.workunit.client.0.trial046.stderr: -280> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding con mode: crc 2026-02-24T22:18:51.733 INFO:tasks.workunit.client.0.trial046.stderr: -279> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding con mode: secure 2026-02-24T22:18:51.733 INFO:tasks.workunit.client.0.trial046.stderr: -278> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:51.733 INFO:tasks.workunit.client.0.trial046.stderr: -277> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: init 2026-02-24T22:18:51.733 INFO:tasks.workunit.client.0.trial046.stderr: -276> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding auth protocol: cephx 2026-02-24T22:18:51.733 INFO:tasks.workunit.client.0.trial046.stderr: -275> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding auth protocol: cephx 2026-02-24T22:18:51.733 INFO:tasks.workunit.client.0.trial046.stderr: -274> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding auth protocol: cephx 2026-02-24T22:18:51.733 INFO:tasks.workunit.client.0.trial046.stderr: -273> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding con mode: secure 2026-02-24T22:18:51.733 INFO:tasks.workunit.client.0.trial046.stderr: -272> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding con mode: crc 2026-02-24T22:18:51.733 INFO:tasks.workunit.client.0.trial046.stderr: -271> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding con mode: secure 2026-02-24T22:18:51.733 INFO:tasks.workunit.client.0.trial046.stderr: -270> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding con mode: crc 2026-02-24T22:18:51.733 INFO:tasks.workunit.client.0.trial046.stderr: -269> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding con mode: secure 2026-02-24T22:18:51.733 INFO:tasks.workunit.client.0.trial046.stderr: -268> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding con mode: crc 2026-02-24T22:18:51.733 INFO:tasks.workunit.client.0.trial046.stderr: -267> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding con mode: crc 2026-02-24T22:18:51.733 INFO:tasks.workunit.client.0.trial046.stderr: -266> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding con mode: secure 2026-02-24T22:18:51.733 INFO:tasks.workunit.client.0.trial046.stderr: -265> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding con mode: crc 2026-02-24T22:18:51.733 INFO:tasks.workunit.client.0.trial046.stderr: -264> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding con mode: secure 2026-02-24T22:18:51.733 INFO:tasks.workunit.client.0.trial046.stderr: -263> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding con mode: crc 2026-02-24T22:18:51.733 INFO:tasks.workunit.client.0.trial046.stderr: -262> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding con mode: secure 2026-02-24T22:18:51.733 INFO:tasks.workunit.client.0.trial046.stderr: -261> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:51.734 INFO:tasks.workunit.client.0.trial046.stderr: -260> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:51.734 INFO:tasks.workunit.client.0.trial046.stderr: -259> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command rotate-key cmddesc rotate-key hook 0x7fff7c664a58 EEXIST 2026-02-24T22:18:51.734 INFO:tasks.workunit.client.0.trial046.stderr: -258> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: _reopen_session rank -1 2026-02-24T22:18:51.734 INFO:tasks.workunit.client.0.trial046.stderr: -257> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T22:18:51.734 INFO:tasks.workunit.client.0.trial046.stderr: -256> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): picked mon.noname-a con 0x556e148856a0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:18:51.734 INFO:tasks.workunit.client.0.trial046.stderr: -255> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): picked mon.noname-c con 0x556e1456a840 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:18:51.734 INFO:tasks.workunit.client.0.trial046.stderr: -254> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): picked mon.noname-b con 0x556e148dcf70 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:18:51.734 INFO:tasks.workunit.client.0.trial046.stderr: -253> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:51.734 INFO:tasks.workunit.client.0.trial046.stderr: -252> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:51.734 INFO:tasks.workunit.client.0.trial046.stderr: -251> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:51.734 INFO:tasks.workunit.client.0.trial046.stderr: -250> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): _renew_subs 2026-02-24T22:18:51.734 INFO:tasks.workunit.client.0.trial046.stderr: -249> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): authenticate will time out at 1901.534301s 2026-02-24T22:18:51.734 INFO:tasks.workunit.client.0.trial046.stderr: -248> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient(hunting): get_auth_request con 0x556e1456a840 auth_method 0 2026-02-24T22:18:51.734 INFO:tasks.workunit.client.0.trial046.stderr: -247> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:51.734 INFO:tasks.workunit.client.0.trial046.stderr: -246> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:51.734 INFO:tasks.workunit.client.0.trial046.stderr: -245> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:51.734 INFO:tasks.workunit.client.0.trial046.stderr: -244> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): get_auth_request con 0x556e148856a0 auth_method 0 2026-02-24T22:18:51.734 INFO:tasks.workunit.client.0.trial046.stderr: -243> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:51.734 INFO:tasks.workunit.client.0.trial046.stderr: -242> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:51.734 INFO:tasks.workunit.client.0.trial046.stderr: -241> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:51.734 INFO:tasks.workunit.client.0.trial046.stderr: -240> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:51.734 INFO:tasks.workunit.client.0.trial046.stderr: -239> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:51.734 INFO:tasks.workunit.client.0.trial046.stderr: -238> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:51.735 INFO:tasks.workunit.client.0.trial046.stderr: -237> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:51.735 INFO:tasks.workunit.client.0.trial046.stderr: -236> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:51.735 INFO:tasks.workunit.client.0.trial046.stderr: -235> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:51.735 INFO:tasks.workunit.client.0.trial046.stderr: -234> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient(hunting): handle_auth_done global_id 7118 payload 274 2026-02-24T22:18:51.735 INFO:tasks.workunit.client.0.trial046.stderr: -233> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient: _finish_hunting 0 2026-02-24T22:18:51.735 INFO:tasks.workunit.client.0.trial046.stderr: -232> 2026-02-24T22:18:51.426+0000 7f3feffff640 1 monclient: found mon.noname-c 2026-02-24T22:18:51.735 INFO:tasks.workunit.client.0.trial046.stderr: -231> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:18:51.735 INFO:tasks.workunit.client.0.trial046.stderr: -230> 2026-02-24T22:18:51.426+0000 7f3ff77fe640 10 monclient: get_auth_request con 0x556e148dcf70 auth_method 0 2026-02-24T22:18:51.735 INFO:tasks.workunit.client.0.trial046.stderr: -229> 2026-02-24T22:18:51.426+0000 7f3fd67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:51.735 INFO:tasks.workunit.client.0.trial046.stderr: -228> 2026-02-24T22:18:51.426+0000 7f3fd67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:18:51.735 INFO:tasks.workunit.client.0.trial046.stderr: -227> 2026-02-24T22:18:51.426+0000 7f3fd67fc640 10 monclient: dump: 2026-02-24T22:18:51.735 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:51.735 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:51.735 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.735 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.735 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:51.735 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.735 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:51.735 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:51.735 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:51.735 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.735 INFO:tasks.workunit.client.0.trial046.stderr: -226> 2026-02-24T22:18:51.426+0000 7f3fd67fc640 10 monclient: _finish_auth 0 2026-02-24T22:18:51.736 INFO:tasks.workunit.client.0.trial046.stderr: -225> 2026-02-24T22:18:51.426+0000 7f3fd67fc640 10 monclient: _check_auth_tickets 2026-02-24T22:18:51.736 INFO:tasks.workunit.client.0.trial046.stderr: -224> 2026-02-24T22:18:51.426+0000 7f3fd67fc640 10 monclient: handle_config config(0 keys) 2026-02-24T22:18:51.736 INFO:tasks.workunit.client.0.trial046.stderr: -223> 2026-02-24T22:18:51.426+0000 7f3fd67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:51.736 INFO:tasks.workunit.client.0.trial046.stderr: -222> 2026-02-24T22:18:51.426+0000 7f3fd67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:18:51.736 INFO:tasks.workunit.client.0.trial046.stderr: -221> 2026-02-24T22:18:51.426+0000 7f3fd67fc640 10 monclient: dump: 2026-02-24T22:18:51.736 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:51.736 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:51.736 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.736 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.736 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:51.736 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.736 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:51.736 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:51.736 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:51.736 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.736 INFO:tasks.workunit.client.0.trial046.stderr: -220> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 monclient: authenticate success, global_id 7118 2026-02-24T22:18:51.736 INFO:tasks.workunit.client.0.trial046.stderr: -219> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: get_monmap_and_config success 2026-02-24T22:18:51.736 INFO:tasks.workunit.client.0.trial046.stderr: -218> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 4 set_mon_vals no callback set 2026-02-24T22:18:51.736 INFO:tasks.workunit.client.0.trial046.stderr: -217> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: shutdown 2026-02-24T22:18:51.736 INFO:tasks.workunit.client.0.trial046.stderr: -216> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: build_initial_monmap 2026-02-24T22:18:51.736 INFO:tasks.workunit.client.0.trial046.stderr: -215> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 1 build_initial for_mkfs: 0 2026-02-24T22:18:51.736 INFO:tasks.workunit.client.0.trial046.stderr: -214> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: monmap: 2026-02-24T22:18:51.737 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:18:51.737 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:18:51.737 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:18:51.737 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:18:51.737 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:18:51.737 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.737 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:18:51.737 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:18:51.737 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:18:51.737 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.737 INFO:tasks.workunit.client.0.trial046.stderr: -213> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding auth protocol: cephx 2026-02-24T22:18:51.737 INFO:tasks.workunit.client.0.trial046.stderr: -212> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding auth protocol: cephx 2026-02-24T22:18:51.737 INFO:tasks.workunit.client.0.trial046.stderr: -211> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding auth protocol: cephx 2026-02-24T22:18:51.737 INFO:tasks.workunit.client.0.trial046.stderr: -210> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding con mode: secure 2026-02-24T22:18:51.737 INFO:tasks.workunit.client.0.trial046.stderr: -209> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding con mode: crc 2026-02-24T22:18:51.737 INFO:tasks.workunit.client.0.trial046.stderr: -208> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding con mode: secure 2026-02-24T22:18:51.737 INFO:tasks.workunit.client.0.trial046.stderr: -207> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding con mode: crc 2026-02-24T22:18:51.737 INFO:tasks.workunit.client.0.trial046.stderr: -206> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding con mode: secure 2026-02-24T22:18:51.737 INFO:tasks.workunit.client.0.trial046.stderr: -205> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding con mode: crc 2026-02-24T22:18:51.737 INFO:tasks.workunit.client.0.trial046.stderr: -204> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding con mode: crc 2026-02-24T22:18:51.737 INFO:tasks.workunit.client.0.trial046.stderr: -203> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding con mode: secure 2026-02-24T22:18:51.737 INFO:tasks.workunit.client.0.trial046.stderr: -202> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding con mode: crc 2026-02-24T22:18:51.737 INFO:tasks.workunit.client.0.trial046.stderr: -201> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding con mode: secure 2026-02-24T22:18:51.737 INFO:tasks.workunit.client.0.trial046.stderr: -200> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding con mode: crc 2026-02-24T22:18:51.737 INFO:tasks.workunit.client.0.trial046.stderr: -199> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding con mode: secure 2026-02-24T22:18:51.738 INFO:tasks.workunit.client.0.trial046.stderr: -198> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:51.738 INFO:tasks.workunit.client.0.trial046.stderr: -197> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 1 librados: starting msgr at 2026-02-24T22:18:51.738 INFO:tasks.workunit.client.0.trial046.stderr: -196> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 1 librados: starting objecter 2026-02-24T22:18:51.738 INFO:tasks.workunit.client.0.trial046.stderr: -195> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command objecter_requests cmddesc objecter_requests hook 0x556e14889970 EEXIST 2026-02-24T22:18:51.738 INFO:tasks.workunit.client.0.trial046.stderr: -194> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 1 librados: setting wanted keys 2026-02-24T22:18:51.738 INFO:tasks.workunit.client.0.trial046.stderr: -193> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 1 librados: calling monclient init 2026-02-24T22:18:51.738 INFO:tasks.workunit.client.0.trial046.stderr: -192> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: init 2026-02-24T22:18:51.738 INFO:tasks.workunit.client.0.trial046.stderr: -191> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding auth protocol: cephx 2026-02-24T22:18:51.738 INFO:tasks.workunit.client.0.trial046.stderr: -190> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding auth protocol: cephx 2026-02-24T22:18:51.738 INFO:tasks.workunit.client.0.trial046.stderr: -189> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding auth protocol: cephx 2026-02-24T22:18:51.738 INFO:tasks.workunit.client.0.trial046.stderr: -188> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding con mode: secure 2026-02-24T22:18:51.738 INFO:tasks.workunit.client.0.trial046.stderr: -187> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding con mode: crc 2026-02-24T22:18:51.738 INFO:tasks.workunit.client.0.trial046.stderr: -186> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding con mode: secure 2026-02-24T22:18:51.738 INFO:tasks.workunit.client.0.trial046.stderr: -185> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding con mode: crc 2026-02-24T22:18:51.738 INFO:tasks.workunit.client.0.trial046.stderr: -184> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding con mode: secure 2026-02-24T22:18:51.738 INFO:tasks.workunit.client.0.trial046.stderr: -183> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding con mode: crc 2026-02-24T22:18:51.738 INFO:tasks.workunit.client.0.trial046.stderr: -182> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding con mode: crc 2026-02-24T22:18:51.738 INFO:tasks.workunit.client.0.trial046.stderr: -181> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding con mode: secure 2026-02-24T22:18:51.738 INFO:tasks.workunit.client.0.trial046.stderr: -180> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding con mode: crc 2026-02-24T22:18:51.738 INFO:tasks.workunit.client.0.trial046.stderr: -179> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding con mode: secure 2026-02-24T22:18:51.738 INFO:tasks.workunit.client.0.trial046.stderr: -178> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding con mode: crc 2026-02-24T22:18:51.738 INFO:tasks.workunit.client.0.trial046.stderr: -177> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding con mode: secure 2026-02-24T22:18:51.738 INFO:tasks.workunit.client.0.trial046.stderr: -176> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:51.738 INFO:tasks.workunit.client.0.trial046.stderr: -175> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:51.738 INFO:tasks.workunit.client.0.trial046.stderr: -174> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command rotate-key cmddesc rotate-key hook 0x556e148824b8 EEXIST 2026-02-24T22:18:51.738 INFO:tasks.workunit.client.0.trial046.stderr: -173> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: _reopen_session rank -1 2026-02-24T22:18:51.739 INFO:tasks.workunit.client.0.trial046.stderr: -172> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T22:18:51.739 INFO:tasks.workunit.client.0.trial046.stderr: -171> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): picked mon.noname-c con 0x556e148856a0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:18:51.739 INFO:tasks.workunit.client.0.trial046.stderr: -170> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): picked mon.noname-b con 0x556e148dcf70 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:18:51.739 INFO:tasks.workunit.client.0.trial046.stderr: -169> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): picked mon.noname-a con 0x556e148e5e30 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:18:51.739 INFO:tasks.workunit.client.0.trial046.stderr: -168> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:51.739 INFO:tasks.workunit.client.0.trial046.stderr: -167> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:51.739 INFO:tasks.workunit.client.0.trial046.stderr: -166> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:51.739 INFO:tasks.workunit.client.0.trial046.stderr: -165> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): _renew_subs 2026-02-24T22:18:51.739 INFO:tasks.workunit.client.0.trial046.stderr: -164> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): authenticate will time out at 1901.535601s 2026-02-24T22:18:51.739 INFO:tasks.workunit.client.0.trial046.stderr: -163> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): get_auth_request con 0x556e148856a0 auth_method 0 2026-02-24T22:18:51.739 INFO:tasks.workunit.client.0.trial046.stderr: -162> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:51.739 INFO:tasks.workunit.client.0.trial046.stderr: -161> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:51.739 INFO:tasks.workunit.client.0.trial046.stderr: -160> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:51.739 INFO:tasks.workunit.client.0.trial046.stderr: -159> 2026-02-24T22:18:51.426+0000 7f3ff77fe640 10 monclient(hunting): get_auth_request con 0x556e148e5e30 auth_method 0 2026-02-24T22:18:51.739 INFO:tasks.workunit.client.0.trial046.stderr: -158> 2026-02-24T22:18:51.426+0000 7f3ff77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:51.739 INFO:tasks.workunit.client.0.trial046.stderr: -157> 2026-02-24T22:18:51.426+0000 7f3ff77fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:51.739 INFO:tasks.workunit.client.0.trial046.stderr: -156> 2026-02-24T22:18:51.426+0000 7f3ff77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:51.739 INFO:tasks.workunit.client.0.trial046.stderr: -155> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:51.739 INFO:tasks.workunit.client.0.trial046.stderr: -154> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:51.739 INFO:tasks.workunit.client.0.trial046.stderr: -153> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:51.739 INFO:tasks.workunit.client.0.trial046.stderr: -152> 2026-02-24T22:18:51.426+0000 7f3ff77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:51.739 INFO:tasks.workunit.client.0.trial046.stderr: -151> 2026-02-24T22:18:51.426+0000 7f3ff77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:51.739 INFO:tasks.workunit.client.0.trial046.stderr: -150> 2026-02-24T22:18:51.426+0000 7f3ff77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:51.739 INFO:tasks.workunit.client.0.trial046.stderr: -149> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient(hunting): get_auth_request con 0x556e148dcf70 auth_method 0 2026-02-24T22:18:51.740 INFO:tasks.workunit.client.0.trial046.stderr: -148> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:51.740 INFO:tasks.workunit.client.0.trial046.stderr: -147> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:51.740 INFO:tasks.workunit.client.0.trial046.stderr: -146> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:51.740 INFO:tasks.workunit.client.0.trial046.stderr: -145> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_done global_id 7121 payload 824 2026-02-24T22:18:51.740 INFO:tasks.workunit.client.0.trial046.stderr: -144> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient: _finish_hunting 0 2026-02-24T22:18:51.740 INFO:tasks.workunit.client.0.trial046.stderr: -143> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 1 monclient: found mon.noname-c 2026-02-24T22:18:51.740 INFO:tasks.workunit.client.0.trial046.stderr: -142> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:18:51.740 INFO:tasks.workunit.client.0.trial046.stderr: -141> 2026-02-24T22:18:51.430+0000 7f3fd4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:51.740 INFO:tasks.workunit.client.0.trial046.stderr: -140> 2026-02-24T22:18:51.430+0000 7f3fd4ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:18:51.740 INFO:tasks.workunit.client.0.trial046.stderr: -139> 2026-02-24T22:18:51.430+0000 7f3fd4ff9640 10 monclient: dump: 2026-02-24T22:18:51.740 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:51.740 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:51.740 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.740 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.740 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:51.740 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.740 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:51.740 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:51.740 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:51.740 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.740 INFO:tasks.workunit.client.0.trial046.stderr: -138> 2026-02-24T22:18:51.430+0000 7f3fd4ff9640 10 monclient: _finish_auth 0 2026-02-24T22:18:51.740 INFO:tasks.workunit.client.0.trial046.stderr: -137> 2026-02-24T22:18:51.430+0000 7f3fd4ff9640 10 monclient: _check_auth_tickets 2026-02-24T22:18:51.740 INFO:tasks.workunit.client.0.trial046.stderr: -136> 2026-02-24T22:18:51.430+0000 7f3fd4ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T22:18:51.740 INFO:tasks.workunit.client.0.trial046.stderr: -135> 2026-02-24T22:18:51.430+0000 7f3fd4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:51.740 INFO:tasks.workunit.client.0.trial046.stderr: -134> 2026-02-24T22:18:51.430+0000 7f3fd4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:18:51.740 INFO:tasks.workunit.client.0.trial046.stderr: -133> 2026-02-24T22:18:51.430+0000 7f3fd4ff9640 10 monclient: dump: 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr: -132> 2026-02-24T22:18:51.430+0000 7f3fd67fc640 4 set_mon_vals no callback set 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr: -131> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 5 monclient: authenticate success, global_id 7121 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr: -130> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 10 monclient: _renew_subs 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr: -129> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr: -128> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 10 monclient: _renew_subs 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr: -127> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr: -126> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 1 librados: init done 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr: -125> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command cr dump hook 0x556e148b1298 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr: -124> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 4 Realm: () 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr: -123> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr: -122> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr: -121> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr: -120> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 20 zonegroup default 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr: -119> 2026-02-24T22:18:51.430+0000 7f3feffff640 10 monclient: get_auth_request con 0x556e1494e010 auth_method 0 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr: -118> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 20 rados->read ofs=0 len=0 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr: -117> 2026-02-24T22:18:51.430+0000 7f3fd4ff9640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:18:51.741 INFO:tasks.workunit.client.0.trial046.stderr: -116> 2026-02-24T22:18:51.430+0000 7f3fd4ff9640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -115> 2026-02-24T22:18:51.430+0000 7f3fd4ff9640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -114> 2026-02-24T22:18:51.430+0000 7f3ff77fe640 10 monclient: get_auth_request con 0x7f3fbc0707b0 auth_method 0 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -113> 2026-02-24T22:18:51.430+0000 7f3ff6ffd640 10 monclient: get_auth_request con 0x556e14971380 auth_method 0 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -112> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -111> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 20 rados->read ofs=0 len=0 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -110> 2026-02-24T22:18:51.430+0000 7f3feffff640 10 monclient: get_auth_request con 0x556e14992a70 auth_method 0 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -109> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -108> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 20 started sync module instance, tier type = 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -107> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -106> 2026-02-24T22:18:51.430+0000 7f3ff77fe640 10 monclient: get_auth_request con 0x556e149f3f20 auth_method 0 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -105> 2026-02-24T22:18:51.430+0000 7f3ff77fe640 10 monclient: get_auth_request con 0x556e14afb930 auth_method 0 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -104> 2026-02-24T22:18:51.430+0000 7f3feffff640 10 monclient: get_auth_request con 0x556e14ab97f0 auth_method 0 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -103> 2026-02-24T22:18:51.430+0000 7f3ff6ffd640 10 monclient: get_auth_request con 0x556e14a56860 auth_method 0 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -102> 2026-02-24T22:18:51.430+0000 7f3ff6ffd640 10 monclient: get_auth_request con 0x556e14b5e930 auth_method 0 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -101> 2026-02-24T22:18:51.434+0000 7f3ff77fe640 0 -- 10.20.193.46:0/1574263675 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x556e149f3f20 msgr2=0x556e14a14300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -100> 2026-02-24T22:18:51.434+0000 7f3fd4ff9640 1 client.7121.objecter ms_handle_reset 0x556e149f3f20 session 0x556e14970910 osd.0 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -99> 2026-02-24T22:18:51.434+0000 7f3fd4ff9640 10 monclient: _renew_subs 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -98> 2026-02-24T22:18:51.434+0000 7f3fd4ff9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -97> 2026-02-24T22:18:51.434+0000 7f3ff77fe640 10 monclient: get_auth_request con 0x7f3fbc09d460 auth_method 0 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -96> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 add_watcher() i=1 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -95> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 add_watcher() i=6 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -94> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 add_watcher() i=3 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -93> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 add_watcher() i=0 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -92> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 add_watcher() i=5 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -91> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 add_watcher() i=2 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -90> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 add_watcher() i=7 2026-02-24T22:18:51.742 INFO:tasks.workunit.client.0.trial046.stderr: -89> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 add_watcher() i=4 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -88> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 2 all 8 watchers are set, enabling cache 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -87> 2026-02-24T22:18:51.434+0000 7f3feffff640 10 monclient: get_auth_request con 0x7f3f1c008af0 auth_method 0 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -86> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding auth protocol: cephx 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -85> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding auth protocol: cephx 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -84> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding auth protocol: cephx 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -83> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding con mode: secure 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -82> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding con mode: crc 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -81> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding con mode: secure 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -80> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding con mode: crc 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -79> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding con mode: secure 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -78> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding con mode: crc 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -77> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding con mode: crc 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -76> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding con mode: secure 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -75> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding con mode: crc 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -74> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding con mode: secure 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -73> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding con mode: crc 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -72> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding con mode: secure 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -71> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -70> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -69> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -68> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command cache list hook 0x556e148e6260 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -67> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command cache inspect hook 0x556e148e6260 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -66> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command cache erase hook 0x556e148e6260 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -65> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command cache zap hook 0x556e148e6260 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -64> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 note: GC not initialized 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -63> 2026-02-24T22:18:51.434+0000 7f3f13fff640 20 reqs_thread_entry: start 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -62> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command sync trace show hook 0x7f3fe00043a0 2026-02-24T22:18:51.743 INFO:tasks.workunit.client.0.trial046.stderr: -61> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command sync trace history hook 0x7f3fe00043a0 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -60> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command sync trace active hook 0x7f3fe00043a0 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -59> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command sync trace active_short hook 0x7f3fe00043a0 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -58> 2026-02-24T22:18:51.434+0000 7f3ff77fe640 10 monclient: get_auth_request con 0x556e14a873b0 auth_method 0 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -57> 2026-02-24T22:18:51.438+0000 7f3ff77fe640 0 -- 10.20.193.46:0/1786546684 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x556e14a873b0 msgr2=0x556e14a11ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -56> 2026-02-24T22:18:51.642+0000 7f3ff77fe640 10 monclient: get_auth_request con 0x556e14a873b0 auth_method 0 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -55> 2026-02-24T22:18:51.642+0000 7f3ff6ffd640 10 monclient: get_auth_request con 0x556e14a12c10 auth_method 0 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -54> 2026-02-24T22:18:51.646+0000 7f3feffff640 10 monclient: get_auth_request con 0x556e14a14420 auth_method 0 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -53> 2026-02-24T22:18:51.646+0000 7f3ff77fe640 10 monclient: get_auth_request con 0x556e14a0be00 auth_method 0 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -52> 2026-02-24T22:18:51.650+0000 7f3ff6ffd640 10 monclient: get_auth_request con 0x556e14a0e2e0 auth_method 0 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -51> 2026-02-24T22:18:51.654+0000 7f3feffff640 10 monclient: get_auth_request con 0x556e14a115a0 auth_method 0 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -50> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -49> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 20 Filter name: none 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -48> 2026-02-24T22:18:51.678+0000 7f3f127fc640 20 reqs_thread_entry: start 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -47> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -46> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 20 rados->read ofs=0 len=0 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -45> 2026-02-24T22:18:51.678+0000 7f3ff77fe640 10 monclient: get_auth_request con 0x556e14607bf0 auth_method 0 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -44> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -43> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -42> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -41> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 : miss 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -40> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 20 rados->read ofs=0 len=0 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -39> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -38> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 info.flags=0x17 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -37> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 to cache LRU end 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -36> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -35> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:51.744 INFO:tasks.workunit.client.0.trial046.stderr: -34> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -33> 2026-02-24T22:18:51.682+0000 7f3ffd6dda80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 info.flags=0x17 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -32> 2026-02-24T22:18:51.682+0000 7f3ffd6dda80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 to cache LRU end 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -31> 2026-02-24T22:18:51.682+0000 7f3ffd6dda80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -30> 2026-02-24T22:18:51.682+0000 7f3ffd6dda80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533, ofs0, ns] 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -29> 2026-02-24T22:18:51.682+0000 7f3fd5ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959913 cookie 93931282091568 notifier 7121 bl.length()=1033 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -28> 2026-02-24T22:18:51.682+0000 7f3fd5ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 info.flags=0x17 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -27> 2026-02-24T22:18:51.682+0000 7f3fd5ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 to cache LRU end 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -26> 2026-02-24T22:18:51.682+0000 7f3fd5ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -25> 2026-02-24T22:18:51.682+0000 7f3ffd6dda80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): orig_sources= new_sources= 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -24> 2026-02-24T22:18:51.682+0000 7f3ffd6dda80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): potential sources added=[] removed=[] 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -23> 2026-02-24T22:18:51.682+0000 7f3ffd6dda80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): orig_dests= new_dests= 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -22> 2026-02-24T22:18:51.682+0000 7f3ffd6dda80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): potential dests added=[] removed=[] 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -21> 2026-02-24T22:18:51.686+0000 7f3ffd6dda80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533.1.0 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -20> 2026-02-24T22:18:51.686+0000 7f3ffd6dda80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533.1.0 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -19> 2026-02-24T22:18:51.686+0000 7f3ffd6dda80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533.1.1 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -18> 2026-02-24T22:18:51.686+0000 7f3ffd6dda80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533.1.1 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -17> 2026-02-24T22:18:51.686+0000 7f3ffd6dda80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533.0 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -16> 2026-02-24T22:18:51.686+0000 7f3ffd6dda80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 : hit (requested=0x17, cached=0x17) 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -15> 2026-02-24T22:18:51.686+0000 7f3ffd6dda80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -14> 2026-02-24T22:18:51.690+0000 7f3ffd6dda80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 info.flags=0x17 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -13> 2026-02-24T22:18:51.690+0000 7f3ffd6dda80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 to cache LRU end 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -12> 2026-02-24T22:18:51.690+0000 7f3ffd6dda80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -11> 2026-02-24T22:18:51.690+0000 7f3ffd6dda80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533, ofs0, ns] 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -10> 2026-02-24T22:18:51.690+0000 7f3fd5ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959914 cookie 93931282091568 notifier 7121 bl.length()=1033 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -9> 2026-02-24T22:18:51.690+0000 7f3fd5ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 info.flags=0x17 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -8> 2026-02-24T22:18:51.690+0000 7f3fd5ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 to cache LRU end 2026-02-24T22:18:51.745 INFO:tasks.workunit.client.0.trial046.stderr: -7> 2026-02-24T22:18:51.690+0000 7f3fd5ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: -6> 2026-02-24T22:18:51.690+0000 7f3ffd6dda80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): orig_sources= new_sources= 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: -5> 2026-02-24T22:18:51.690+0000 7f3ffd6dda80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): potential sources added=[] removed=[] 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: -4> 2026-02-24T22:18:51.690+0000 7f3ffd6dda80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): orig_dests= new_dests= 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: -3> 2026-02-24T22:18:51.690+0000 7f3ffd6dda80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): potential dests added=[] removed=[] 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: -2> 2026-02-24T22:18:51.690+0000 7f3ffd6dda80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533.0 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: -1> 2026-02-24T22:18:51.690+0000 7f3ffd6dda80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f3ffd6dda80 time 2026-02-24T22:18:51.693432+0000 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f4000b1feb1] 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: 2: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xce6) [0x556df6828de6] 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: 3: main() 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: 4: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f3fffe1fd90] 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: 5: __libc_start_main() 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: 6: _start() 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: 0> 2026-02-24T22:18:51.690+0000 7f3ffd6dda80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f3ffd6dda80 thread_name:radosgw-admin 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f3fffe38520] 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f4000b1ff10] 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: 6: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xce6) [0x556df6828de6] 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: 7: main() 2026-02-24T22:18:51.746 INFO:tasks.workunit.client.0.trial046.stderr: 8: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f3fffe1fd90] 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 9: __libc_start_main() 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 10: _start() 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr:--- logging levels --- 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 none 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 lockdep 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 context 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 crush 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_balancer 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_locker 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log_expire 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_migrator 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 3/ 5 mds_quiesce 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 buffer 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 timer 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 filer 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 striper 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 objecter 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rados 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_mirror 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_replay 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_pwl 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 journaler 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objectcacher 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 immutable_obj_cache 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 client 2026-02-24T22:18:51.747 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 osd 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 optracker 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objclass 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 filestore 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 journal 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 ms 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mon 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 0/10 monc 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 paxos 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 tp 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 auth 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 crypto 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 finisher 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 reserver 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 heartbeatmap 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 perfcounter 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_sync 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_datacache 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_access 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dbstore 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_flight 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_lifecycle 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_restore 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw_notification 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_bucket_logging 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 javaclient 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 asok 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 throttle 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 refs 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 compressor 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore 2026-02-24T22:18:51.748 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore_compression 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluefs 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 bdev 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 kstore 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 4/ 5 rocksdb 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 fuse 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 2/ 5 mgr 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mgrc 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 dpdk 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 eventtrace 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 prioritycache 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 test 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephfs_mirror 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephsqlite 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 crimson_interrupt 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_onode 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_odata 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_omap 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_tm 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_t 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cleaner 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_epm 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_lba 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_fixedkv_tree 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cache 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_journal 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_device 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_backref 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 alienstore 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mclock 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dedup 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cyanstore 2026-02-24T22:18:51.749 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 ceph_exporter 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 memstore 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 trace 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 ceph_dedup 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: -2/-2 (syslog threshold) 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: 99/99 (stderr threshold) 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr:--- pthread ID / name mapping for recent threads --- 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: 7f3f127fc640 / http_manager 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: 7f3f13fff640 / http_manager 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: 7f3fd4ff9640 / ms_dispatch 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: 7f3fd5ffb640 / io_context_pool 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: 7f3fd67fc640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: 7f3fdcff9640 / ms_dispatch 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: 7f3fdf7fe640 / ms_dispatch 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: 7f3fed7fa640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: 7f3fedffb640 / admin_socket 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: 7f3feffff640 / msgr-worker-2 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: 7f3ff67fc640 / ms_dispatch 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: 7f3ff6ffd640 / msgr-worker-1 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: 7f3ff77fe640 / msgr-worker-0 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: 7f3ff7fff640 / io_context_pool 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: 7f3ffd6dda80 / radosgw-admin 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: max_recent 500 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: max_new 1000 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: log_file /var/log/ceph/ceph-client.admin.48937.log 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr:--- end dump of recent events --- 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr:--- begin dump of recent events --- 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: -499> 2026-02-24T22:18:51.418+0000 7f3feffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: -498> 2026-02-24T22:18:51.418+0000 7f3feffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: -497> 2026-02-24T22:18:51.418+0000 7f3feffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: -496> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_done global_id 7608 payload 274 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: -495> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 10 monclient: _finish_hunting 0 2026-02-24T22:18:51.750 INFO:tasks.workunit.client.0.trial046.stderr: -494> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 1 monclient: found mon.noname-a 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr: -493> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr: -492> 2026-02-24T22:18:51.418+0000 7f3fed7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr: -491> 2026-02-24T22:18:51.418+0000 7f3fed7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr: -490> 2026-02-24T22:18:51.418+0000 7f3fed7fa640 10 monclient: dump: 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr: -489> 2026-02-24T22:18:51.418+0000 7f3fed7fa640 10 monclient: _finish_auth 0 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr: -488> 2026-02-24T22:18:51.418+0000 7f3fed7fa640 10 monclient: _check_auth_tickets 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr: -487> 2026-02-24T22:18:51.418+0000 7f3fed7fa640 10 monclient: handle_config config(0 keys) 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr: -486> 2026-02-24T22:18:51.418+0000 7f3fed7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr: -485> 2026-02-24T22:18:51.418+0000 7f3fed7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr: -484> 2026-02-24T22:18:51.418+0000 7f3fed7fa640 10 monclient: dump: 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.751 INFO:tasks.workunit.client.0.trial046.stderr: -483> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 monclient: authenticate success, global_id 7608 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr: -482> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient: get_monmap_and_config success 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr: -481> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 4 set_mon_vals no callback set 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr: -480> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient: shutdown 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr: -479> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 asok(0x556e14509f00) unregister_commands rotate-key 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr: -478> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient: build_initial_monmap 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr: -477> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 1 build_initial for_mkfs: 0 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr: -476> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient: monmap: 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr: -475> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding auth protocol: cephx 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr: -474> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding auth protocol: cephx 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr: -473> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding auth protocol: cephx 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr: -472> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding con mode: secure 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr: -471> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding con mode: crc 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr: -470> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding con mode: secure 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr: -469> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding con mode: crc 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr: -468> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding con mode: secure 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr: -467> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding con mode: crc 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr: -466> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding con mode: crc 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr: -465> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding con mode: secure 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr: -464> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding con mode: crc 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr: -463> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding con mode: secure 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr: -462> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding con mode: crc 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr: -461> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e1460b760) adding con mode: secure 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr: -460> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:51.752 INFO:tasks.workunit.client.0.trial046.stderr: -459> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 1 librados: starting msgr at 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -458> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 1 librados: starting objecter 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -457> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command objecter_requests hook 0x556e1466ab90 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -456> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 1 librados: setting wanted keys 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -455> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 1 librados: calling monclient init 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -454> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient: init 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -453> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding auth protocol: cephx 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -452> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding auth protocol: cephx 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -451> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding auth protocol: cephx 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -450> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding con mode: secure 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -449> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding con mode: crc 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -448> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding con mode: secure 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -447> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding con mode: crc 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -446> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding con mode: secure 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -445> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding con mode: crc 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -444> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding con mode: crc 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -443> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding con mode: secure 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -442> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding con mode: crc 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -441> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding con mode: secure 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -440> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding con mode: crc 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -439> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 AuthRegistry(0x556e146097d0) adding con mode: secure 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -438> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -437> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -436> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command rotate-key hook 0x556e14609918 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -435> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient: _reopen_session rank -1 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -434> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -433> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient(hunting): picked mon.noname-a con 0x556e14597fe0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -432> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient(hunting): picked mon.noname-b con 0x556e14640b00 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -431> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient(hunting): picked mon.noname-c con 0x556e14643cf0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -430> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -429> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -428> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -427> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient(hunting): _renew_subs 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -426> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient(hunting): authenticate will time out at 1901.525863s 2026-02-24T22:18:51.753 INFO:tasks.workunit.client.0.trial046.stderr: -425> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 10 monclient(hunting): get_auth_request con 0x556e14597fe0 auth_method 0 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr: -424> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr: -423> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr: -422> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr: -421> 2026-02-24T22:18:51.418+0000 7f3ff77fe640 10 monclient(hunting): get_auth_request con 0x556e14643cf0 auth_method 0 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr: -420> 2026-02-24T22:18:51.418+0000 7f3ff77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr: -419> 2026-02-24T22:18:51.418+0000 7f3ff77fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr: -418> 2026-02-24T22:18:51.418+0000 7f3ff77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr: -417> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr: -416> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr: -415> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr: -414> 2026-02-24T22:18:51.418+0000 7f3ff77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr: -413> 2026-02-24T22:18:51.418+0000 7f3ff77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr: -412> 2026-02-24T22:18:51.418+0000 7f3ff77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr: -411> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_done global_id 7611 payload 824 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr: -410> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 10 monclient: _finish_hunting 0 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr: -409> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 1 monclient: found mon.noname-a 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr: -408> 2026-02-24T22:18:51.418+0000 7f3ff6ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr: -407> 2026-02-24T22:18:51.418+0000 7f3fdf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr: -406> 2026-02-24T22:18:51.418+0000 7f3fdf7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr: -405> 2026-02-24T22:18:51.418+0000 7f3fdf7fe640 10 monclient: dump: 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr: -404> 2026-02-24T22:18:51.418+0000 7f3fdf7fe640 10 monclient: _finish_auth 0 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr: -403> 2026-02-24T22:18:51.418+0000 7f3fdf7fe640 10 monclient: _check_auth_tickets 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr: -402> 2026-02-24T22:18:51.418+0000 7f3fdf7fe640 10 monclient: handle_config config(0 keys) 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr: -401> 2026-02-24T22:18:51.418+0000 7f3fdf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:51.754 INFO:tasks.workunit.client.0.trial046.stderr: -400> 2026-02-24T22:18:51.418+0000 7f3fdf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr: -399> 2026-02-24T22:18:51.418+0000 7f3fdf7fe640 10 monclient: dump: 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr: -398> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 5 monclient: authenticate success, global_id 7611 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr: -397> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient: _renew_subs 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr: -396> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr: -395> 2026-02-24T22:18:51.418+0000 7f3fed7fa640 4 set_mon_vals no callback set 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr: -394> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient: _renew_subs 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr: -393> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr: -392> 2026-02-24T22:18:51.418+0000 7f3ffd6dda80 1 librados: init done 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr: -391> 2026-02-24T22:18:51.418+0000 7f3fdf7fe640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr: -390> 2026-02-24T22:18:51.418+0000 7f3fdf7fe640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr: -389> 2026-02-24T22:18:51.418+0000 7f3fdf7fe640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr: -388> 2026-02-24T22:18:51.418+0000 7f3feffff640 10 monclient: get_auth_request con 0x7f3fd80707b0 auth_method 0 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr: -387> 2026-02-24T22:18:51.422+0000 7f3ff77fe640 10 monclient: get_auth_request con 0x556e14635620 auth_method 0 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr: -386> 2026-02-24T22:18:51.422+0000 7f3ff6ffd640 10 monclient: get_auth_request con 0x556e14695a40 auth_method 0 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr: -385> 2026-02-24T22:18:51.422+0000 7f3feffff640 10 monclient: get_auth_request con 0x556e14697080 auth_method 0 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr: -384> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 10 monclient: build_initial_monmap 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr: -383> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 1 build_initial for_mkfs: 0 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr: -382> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 10 monclient: monmap: 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.755 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -381> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding auth protocol: cephx 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -380> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding auth protocol: cephx 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -379> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding auth protocol: cephx 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -378> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding con mode: secure 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -377> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding con mode: crc 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -376> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding con mode: secure 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -375> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding con mode: crc 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -374> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding con mode: secure 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -373> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding con mode: crc 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -372> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding con mode: crc 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -371> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding con mode: secure 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -370> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding con mode: crc 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -369> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding con mode: secure 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -368> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding con mode: crc 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -367> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14699d90) adding con mode: secure 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -366> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -365> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command objecter_requests cmddesc objecter_requests hook 0x556e1469ec30 EEXIST 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -364> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 10 monclient: init 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -363> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding auth protocol: cephx 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -362> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding auth protocol: cephx 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -361> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding auth protocol: cephx 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -360> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding con mode: secure 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -359> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding con mode: crc 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -358> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding con mode: secure 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -357> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding con mode: crc 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -356> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding con mode: secure 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -355> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding con mode: crc 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -354> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding con mode: crc 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -353> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding con mode: secure 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -352> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding con mode: crc 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -351> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding con mode: secure 2026-02-24T22:18:51.756 INFO:tasks.workunit.client.0.trial046.stderr: -350> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding con mode: crc 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -349> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14697c58) adding con mode: secure 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -348> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -347> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -346> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command rotate-key cmddesc rotate-key hook 0x556e14697da0 EEXIST 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -345> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 10 monclient: _reopen_session rank -1 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -344> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -343> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 10 monclient(hunting): picked mon.noname-c con 0x556e147f2490 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -342> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 10 monclient(hunting): picked mon.noname-a con 0x556e147ef630 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -341> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 10 monclient(hunting): picked mon.noname-b con 0x556e147eb610 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -340> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -339> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -338> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -337> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 10 monclient(hunting): _renew_subs 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -336> 2026-02-24T22:18:51.422+0000 7f3ffd6dda80 10 monclient(hunting): authenticate will time out at 1901.531774s 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -335> 2026-02-24T22:18:51.422+0000 7f3feffff640 10 monclient(hunting): get_auth_request con 0x556e147ef630 auth_method 0 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -334> 2026-02-24T22:18:51.422+0000 7f3feffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -333> 2026-02-24T22:18:51.422+0000 7f3feffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -332> 2026-02-24T22:18:51.422+0000 7f3feffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -331> 2026-02-24T22:18:51.422+0000 7f3ff6ffd640 10 monclient(hunting): get_auth_request con 0x556e147f2490 auth_method 0 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -330> 2026-02-24T22:18:51.422+0000 7f3ff6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -329> 2026-02-24T22:18:51.422+0000 7f3ff6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -328> 2026-02-24T22:18:51.422+0000 7f3ff6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -327> 2026-02-24T22:18:51.422+0000 7f3feffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -326> 2026-02-24T22:18:51.422+0000 7f3feffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -325> 2026-02-24T22:18:51.422+0000 7f3feffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -324> 2026-02-24T22:18:51.422+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -323> 2026-02-24T22:18:51.422+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -322> 2026-02-24T22:18:51.422+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -321> 2026-02-24T22:18:51.422+0000 7f3feffff640 10 monclient(hunting): handle_auth_done global_id 7614 payload 824 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -320> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient: _finish_hunting 0 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -319> 2026-02-24T22:18:51.426+0000 7f3feffff640 1 monclient: found mon.noname-a 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -318> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -317> 2026-02-24T22:18:51.426+0000 7f3ff77fe640 10 monclient: get_auth_request con 0x556e147eb610 auth_method 0 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -316> 2026-02-24T22:18:51.426+0000 7f3fdcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -315> 2026-02-24T22:18:51.426+0000 7f3fdcff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr: -314> 2026-02-24T22:18:51.426+0000 7f3fdcff9640 10 monclient: dump: 2026-02-24T22:18:51.757 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr: -313> 2026-02-24T22:18:51.426+0000 7f3fdcff9640 10 monclient: _finish_auth 0 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr: -312> 2026-02-24T22:18:51.426+0000 7f3fdcff9640 10 monclient: _check_auth_tickets 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr: -311> 2026-02-24T22:18:51.426+0000 7f3fdcff9640 10 monclient: handle_config config(0 keys) 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr: -310> 2026-02-24T22:18:51.426+0000 7f3fdcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr: -309> 2026-02-24T22:18:51.426+0000 7f3fdcff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr: -308> 2026-02-24T22:18:51.426+0000 7f3fdcff9640 10 monclient: dump: 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr: -307> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 monclient: authenticate success, global_id 7614 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr: -306> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: _renew_subs 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr: -305> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr: -304> 2026-02-24T22:18:51.426+0000 7f3ff7fff640 4 set_mon_vals no callback set 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr: -303> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: _renew_subs 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr: -302> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr: -301> 2026-02-24T22:18:51.426+0000 7f3fdcff9640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr: -300> 2026-02-24T22:18:51.426+0000 7f3fdcff9640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr: -299> 2026-02-24T22:18:51.426+0000 7f3fdcff9640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr: -298> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient: get_auth_request con 0x7f3fcc0706c0 auth_method 0 2026-02-24T22:18:51.758 INFO:tasks.workunit.client.0.trial046.stderr: -297> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: get_monmap_and_config 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -296> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: build_initial_monmap 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -295> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 1 build_initial for_mkfs: 0 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -294> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: monmap: 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -293> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding auth protocol: cephx 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -292> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding auth protocol: cephx 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -291> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding auth protocol: cephx 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -290> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding con mode: secure 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -289> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding con mode: crc 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -288> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding con mode: secure 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -287> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding con mode: crc 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -286> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding con mode: secure 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -285> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding con mode: crc 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -284> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding con mode: crc 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -283> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding con mode: secure 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -282> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding con mode: crc 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -281> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding con mode: secure 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -280> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding con mode: crc 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -279> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e148837b0) adding con mode: secure 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -278> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -277> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: init 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -276> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding auth protocol: cephx 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -275> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding auth protocol: cephx 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -274> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding auth protocol: cephx 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -273> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding con mode: secure 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -272> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding con mode: crc 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -271> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding con mode: secure 2026-02-24T22:18:51.759 INFO:tasks.workunit.client.0.trial046.stderr: -270> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding con mode: crc 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -269> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding con mode: secure 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -268> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding con mode: crc 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -267> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding con mode: crc 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -266> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding con mode: secure 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -265> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding con mode: crc 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -264> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding con mode: secure 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -263> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding con mode: crc 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -262> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664910) adding con mode: secure 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -261> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -260> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -259> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command rotate-key cmddesc rotate-key hook 0x7fff7c664a58 EEXIST 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -258> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: _reopen_session rank -1 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -257> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: _add_conns ranks=[0,2,1] 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -256> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): picked mon.noname-a con 0x556e148856a0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -255> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): picked mon.noname-c con 0x556e1456a840 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -254> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): picked mon.noname-b con 0x556e148dcf70 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -253> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -252> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -251> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -250> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): _renew_subs 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -249> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): authenticate will time out at 1901.534301s 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -248> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient(hunting): get_auth_request con 0x556e1456a840 auth_method 0 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -247> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -246> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -245> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -244> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): get_auth_request con 0x556e148856a0 auth_method 0 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -243> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -242> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -241> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -240> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -239> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -238> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -237> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -236> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -235> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -234> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient(hunting): handle_auth_done global_id 7118 payload 274 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -233> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient: _finish_hunting 0 2026-02-24T22:18:51.760 INFO:tasks.workunit.client.0.trial046.stderr: -232> 2026-02-24T22:18:51.426+0000 7f3feffff640 1 monclient: found mon.noname-c 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr: -231> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr: -230> 2026-02-24T22:18:51.426+0000 7f3ff77fe640 10 monclient: get_auth_request con 0x556e148dcf70 auth_method 0 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr: -229> 2026-02-24T22:18:51.426+0000 7f3fd67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr: -228> 2026-02-24T22:18:51.426+0000 7f3fd67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr: -227> 2026-02-24T22:18:51.426+0000 7f3fd67fc640 10 monclient: dump: 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr: -226> 2026-02-24T22:18:51.426+0000 7f3fd67fc640 10 monclient: _finish_auth 0 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr: -225> 2026-02-24T22:18:51.426+0000 7f3fd67fc640 10 monclient: _check_auth_tickets 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr: -224> 2026-02-24T22:18:51.426+0000 7f3fd67fc640 10 monclient: handle_config config(0 keys) 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr: -223> 2026-02-24T22:18:51.426+0000 7f3fd67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr: -222> 2026-02-24T22:18:51.426+0000 7f3fd67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr: -221> 2026-02-24T22:18:51.426+0000 7f3fd67fc640 10 monclient: dump: 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr: -220> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 monclient: authenticate success, global_id 7118 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr: -219> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: get_monmap_and_config success 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr: -218> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 4 set_mon_vals no callback set 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr: -217> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: shutdown 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr: -216> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: build_initial_monmap 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr: -215> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 1 build_initial for_mkfs: 0 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr: -214> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: monmap: 2026-02-24T22:18:51.761 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -213> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding auth protocol: cephx 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -212> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding auth protocol: cephx 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -211> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding auth protocol: cephx 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -210> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding con mode: secure 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -209> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding con mode: crc 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -208> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding con mode: secure 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -207> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding con mode: crc 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -206> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding con mode: secure 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -205> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding con mode: crc 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -204> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding con mode: crc 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -203> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding con mode: secure 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -202> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding con mode: crc 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -201> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding con mode: secure 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -200> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding con mode: crc 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -199> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14883680) adding con mode: secure 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -198> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -197> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 1 librados: starting msgr at 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -196> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 1 librados: starting objecter 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -195> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command objecter_requests cmddesc objecter_requests hook 0x556e14889970 EEXIST 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -194> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 1 librados: setting wanted keys 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -193> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 1 librados: calling monclient init 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -192> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: init 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -191> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding auth protocol: cephx 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -190> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding auth protocol: cephx 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -189> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding auth protocol: cephx 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -188> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding con mode: secure 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -187> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding con mode: crc 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -186> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding con mode: secure 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -185> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding con mode: crc 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -184> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding con mode: secure 2026-02-24T22:18:51.762 INFO:tasks.workunit.client.0.trial046.stderr: -183> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding con mode: crc 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -182> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding con mode: crc 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -181> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding con mode: secure 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -180> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding con mode: crc 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -179> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding con mode: secure 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -178> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding con mode: crc 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -177> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 AuthRegistry(0x556e14882370) adding con mode: secure 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -176> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -175> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -174> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command rotate-key cmddesc rotate-key hook 0x556e148824b8 EEXIST 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -173> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: _reopen_session rank -1 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -172> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -171> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): picked mon.noname-c con 0x556e148856a0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -170> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): picked mon.noname-b con 0x556e148dcf70 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -169> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): picked mon.noname-a con 0x556e148e5e30 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -168> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -167> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -166> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -165> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): _renew_subs 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -164> 2026-02-24T22:18:51.426+0000 7f3ffd6dda80 10 monclient(hunting): authenticate will time out at 1901.535601s 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -163> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): get_auth_request con 0x556e148856a0 auth_method 0 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -162> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -161> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -160> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -159> 2026-02-24T22:18:51.426+0000 7f3ff77fe640 10 monclient(hunting): get_auth_request con 0x556e148e5e30 auth_method 0 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -158> 2026-02-24T22:18:51.426+0000 7f3ff77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -157> 2026-02-24T22:18:51.426+0000 7f3ff77fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -156> 2026-02-24T22:18:51.426+0000 7f3ff77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -155> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -154> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -153> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -152> 2026-02-24T22:18:51.426+0000 7f3ff77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -151> 2026-02-24T22:18:51.426+0000 7f3ff77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -150> 2026-02-24T22:18:51.426+0000 7f3ff77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -149> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient(hunting): get_auth_request con 0x556e148dcf70 auth_method 0 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -148> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -147> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -146> 2026-02-24T22:18:51.426+0000 7f3feffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -145> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient(hunting): handle_auth_done global_id 7121 payload 824 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -144> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient: _finish_hunting 0 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -143> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 1 monclient: found mon.noname-c 2026-02-24T22:18:51.763 INFO:tasks.workunit.client.0.trial046.stderr: -142> 2026-02-24T22:18:51.426+0000 7f3ff6ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr: -141> 2026-02-24T22:18:51.430+0000 7f3fd4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr: -140> 2026-02-24T22:18:51.430+0000 7f3fd4ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr: -139> 2026-02-24T22:18:51.430+0000 7f3fd4ff9640 10 monclient: dump: 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr: -138> 2026-02-24T22:18:51.430+0000 7f3fd4ff9640 10 monclient: _finish_auth 0 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr: -137> 2026-02-24T22:18:51.430+0000 7f3fd4ff9640 10 monclient: _check_auth_tickets 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr: -136> 2026-02-24T22:18:51.430+0000 7f3fd4ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr: -135> 2026-02-24T22:18:51.430+0000 7f3fd4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr: -134> 2026-02-24T22:18:51.430+0000 7f3fd4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr: -133> 2026-02-24T22:18:51.430+0000 7f3fd4ff9640 10 monclient: dump: 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr: -132> 2026-02-24T22:18:51.430+0000 7f3fd67fc640 4 set_mon_vals no callback set 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr: -131> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 5 monclient: authenticate success, global_id 7121 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr: -130> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 10 monclient: _renew_subs 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr: -129> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr: -128> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 10 monclient: _renew_subs 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr: -127> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr: -126> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 1 librados: init done 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr: -125> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command cr dump hook 0x556e148b1298 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr: -124> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 4 Realm: () 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr: -123> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr: -122> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:51.764 INFO:tasks.workunit.client.0.trial046.stderr: -121> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -120> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 20 zonegroup default 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -119> 2026-02-24T22:18:51.430+0000 7f3feffff640 10 monclient: get_auth_request con 0x556e1494e010 auth_method 0 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -118> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 20 rados->read ofs=0 len=0 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -117> 2026-02-24T22:18:51.430+0000 7f3fd4ff9640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -116> 2026-02-24T22:18:51.430+0000 7f3fd4ff9640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -115> 2026-02-24T22:18:51.430+0000 7f3fd4ff9640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -114> 2026-02-24T22:18:51.430+0000 7f3ff77fe640 10 monclient: get_auth_request con 0x7f3fbc0707b0 auth_method 0 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -113> 2026-02-24T22:18:51.430+0000 7f3ff6ffd640 10 monclient: get_auth_request con 0x556e14971380 auth_method 0 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -112> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -111> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 20 rados->read ofs=0 len=0 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -110> 2026-02-24T22:18:51.430+0000 7f3feffff640 10 monclient: get_auth_request con 0x556e14992a70 auth_method 0 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -109> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -108> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 20 started sync module instance, tier type = 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -107> 2026-02-24T22:18:51.430+0000 7f3ffd6dda80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -106> 2026-02-24T22:18:51.430+0000 7f3ff77fe640 10 monclient: get_auth_request con 0x556e149f3f20 auth_method 0 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -105> 2026-02-24T22:18:51.430+0000 7f3ff77fe640 10 monclient: get_auth_request con 0x556e14afb930 auth_method 0 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -104> 2026-02-24T22:18:51.430+0000 7f3feffff640 10 monclient: get_auth_request con 0x556e14ab97f0 auth_method 0 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -103> 2026-02-24T22:18:51.430+0000 7f3ff6ffd640 10 monclient: get_auth_request con 0x556e14a56860 auth_method 0 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -102> 2026-02-24T22:18:51.430+0000 7f3ff6ffd640 10 monclient: get_auth_request con 0x556e14b5e930 auth_method 0 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -101> 2026-02-24T22:18:51.434+0000 7f3ff77fe640 0 -- 10.20.193.46:0/1574263675 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x556e149f3f20 msgr2=0x556e14a14300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -100> 2026-02-24T22:18:51.434+0000 7f3fd4ff9640 1 client.7121.objecter ms_handle_reset 0x556e149f3f20 session 0x556e14970910 osd.0 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -99> 2026-02-24T22:18:51.434+0000 7f3fd4ff9640 10 monclient: _renew_subs 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -98> 2026-02-24T22:18:51.434+0000 7f3fd4ff9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -97> 2026-02-24T22:18:51.434+0000 7f3ff77fe640 10 monclient: get_auth_request con 0x7f3fbc09d460 auth_method 0 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -96> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 add_watcher() i=1 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -95> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 add_watcher() i=6 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -94> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 add_watcher() i=3 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -93> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 add_watcher() i=0 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -92> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 add_watcher() i=5 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -91> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 add_watcher() i=2 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -90> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 add_watcher() i=7 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -89> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 add_watcher() i=4 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -88> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 2 all 8 watchers are set, enabling cache 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -87> 2026-02-24T22:18:51.434+0000 7f3feffff640 10 monclient: get_auth_request con 0x7f3f1c008af0 auth_method 0 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -86> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding auth protocol: cephx 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -85> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding auth protocol: cephx 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -84> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding auth protocol: cephx 2026-02-24T22:18:51.765 INFO:tasks.workunit.client.0.trial046.stderr: -83> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding con mode: secure 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -82> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding con mode: crc 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -81> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding con mode: secure 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -80> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding con mode: crc 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -79> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding con mode: secure 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -78> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding con mode: crc 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -77> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding con mode: crc 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -76> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding con mode: secure 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -75> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding con mode: crc 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -74> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding con mode: secure 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -73> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding con mode: crc 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -72> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 AuthRegistry(0x7fff7c664d80) adding con mode: secure 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -71> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -70> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -69> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -68> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command cache list hook 0x556e148e6260 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -67> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command cache inspect hook 0x556e148e6260 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -66> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command cache erase hook 0x556e148e6260 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -65> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command cache zap hook 0x556e148e6260 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -64> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 note: GC not initialized 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -63> 2026-02-24T22:18:51.434+0000 7f3f13fff640 20 reqs_thread_entry: start 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -62> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command sync trace show hook 0x7f3fe00043a0 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -61> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command sync trace history hook 0x7f3fe00043a0 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -60> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command sync trace active hook 0x7f3fe00043a0 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -59> 2026-02-24T22:18:51.434+0000 7f3ffd6dda80 5 asok(0x556e14509f00) register_command sync trace active_short hook 0x7f3fe00043a0 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -58> 2026-02-24T22:18:51.434+0000 7f3ff77fe640 10 monclient: get_auth_request con 0x556e14a873b0 auth_method 0 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -57> 2026-02-24T22:18:51.438+0000 7f3ff77fe640 0 -- 10.20.193.46:0/1786546684 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x556e14a873b0 msgr2=0x556e14a11ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -56> 2026-02-24T22:18:51.642+0000 7f3ff77fe640 10 monclient: get_auth_request con 0x556e14a873b0 auth_method 0 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -55> 2026-02-24T22:18:51.642+0000 7f3ff6ffd640 10 monclient: get_auth_request con 0x556e14a12c10 auth_method 0 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -54> 2026-02-24T22:18:51.646+0000 7f3feffff640 10 monclient: get_auth_request con 0x556e14a14420 auth_method 0 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -53> 2026-02-24T22:18:51.646+0000 7f3ff77fe640 10 monclient: get_auth_request con 0x556e14a0be00 auth_method 0 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -52> 2026-02-24T22:18:51.650+0000 7f3ff6ffd640 10 monclient: get_auth_request con 0x556e14a0e2e0 auth_method 0 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -51> 2026-02-24T22:18:51.654+0000 7f3feffff640 10 monclient: get_auth_request con 0x556e14a115a0 auth_method 0 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -50> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -49> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 20 Filter name: none 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -48> 2026-02-24T22:18:51.678+0000 7f3f127fc640 20 reqs_thread_entry: start 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -47> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -46> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 20 rados->read ofs=0 len=0 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -45> 2026-02-24T22:18:51.678+0000 7f3ff77fe640 10 monclient: get_auth_request con 0x556e14607bf0 auth_method 0 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -44> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -43> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2026-02-24T22:18:51.766 INFO:tasks.workunit.client.0.trial046.stderr: -42> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -41> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 : miss 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -40> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 20 rados->read ofs=0 len=0 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -39> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -38> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 info.flags=0x17 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -37> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 to cache LRU end 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -36> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -35> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -34> 2026-02-24T22:18:51.678+0000 7f3ffd6dda80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -33> 2026-02-24T22:18:51.682+0000 7f3ffd6dda80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 info.flags=0x17 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -32> 2026-02-24T22:18:51.682+0000 7f3ffd6dda80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 to cache LRU end 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -31> 2026-02-24T22:18:51.682+0000 7f3ffd6dda80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -30> 2026-02-24T22:18:51.682+0000 7f3ffd6dda80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533, ofs0, ns] 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -29> 2026-02-24T22:18:51.682+0000 7f3fd5ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959913 cookie 93931282091568 notifier 7121 bl.length()=1033 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -28> 2026-02-24T22:18:51.682+0000 7f3fd5ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 info.flags=0x17 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -27> 2026-02-24T22:18:51.682+0000 7f3fd5ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 to cache LRU end 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -26> 2026-02-24T22:18:51.682+0000 7f3fd5ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -25> 2026-02-24T22:18:51.682+0000 7f3ffd6dda80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): orig_sources= new_sources= 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -24> 2026-02-24T22:18:51.682+0000 7f3ffd6dda80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): potential sources added=[] removed=[] 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -23> 2026-02-24T22:18:51.682+0000 7f3ffd6dda80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): orig_dests= new_dests= 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -22> 2026-02-24T22:18:51.682+0000 7f3ffd6dda80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): potential dests added=[] removed=[] 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -21> 2026-02-24T22:18:51.686+0000 7f3ffd6dda80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533.1.0 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -20> 2026-02-24T22:18:51.686+0000 7f3ffd6dda80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533.1.0 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -19> 2026-02-24T22:18:51.686+0000 7f3ffd6dda80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533.1.1 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -18> 2026-02-24T22:18:51.686+0000 7f3ffd6dda80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533.1.1 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -17> 2026-02-24T22:18:51.686+0000 7f3ffd6dda80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533.0 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -16> 2026-02-24T22:18:51.686+0000 7f3ffd6dda80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 : hit (requested=0x17, cached=0x17) 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -15> 2026-02-24T22:18:51.686+0000 7f3ffd6dda80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -14> 2026-02-24T22:18:51.690+0000 7f3ffd6dda80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 info.flags=0x17 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -13> 2026-02-24T22:18:51.690+0000 7f3ffd6dda80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 to cache LRU end 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -12> 2026-02-24T22:18:51.690+0000 7f3ffd6dda80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -11> 2026-02-24T22:18:51.690+0000 7f3ffd6dda80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533, ofs0, ns] 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -10> 2026-02-24T22:18:51.690+0000 7f3fd5ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959914 cookie 93931282091568 notifier 7121 bl.length()=1033 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -9> 2026-02-24T22:18:51.690+0000 7f3fd5ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 info.flags=0x17 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -8> 2026-02-24T22:18:51.690+0000 7f3fd5ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 to cache LRU end 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -7> 2026-02-24T22:18:51.690+0000 7f3fd5ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -6> 2026-02-24T22:18:51.690+0000 7f3ffd6dda80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): orig_sources= new_sources= 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -5> 2026-02-24T22:18:51.690+0000 7f3ffd6dda80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): potential sources added=[] removed=[] 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -4> 2026-02-24T22:18:51.690+0000 7f3ffd6dda80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): orig_dests= new_dests= 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -3> 2026-02-24T22:18:51.690+0000 7f3ffd6dda80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): potential dests added=[] removed=[] 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -2> 2026-02-24T22:18:51.690+0000 7f3ffd6dda80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533.0 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr: -1> 2026-02-24T22:18:51.690+0000 7f3ffd6dda80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f3ffd6dda80 time 2026-02-24T22:18:51.693432+0000 2026-02-24T22:18:51.767 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f4000b1feb1] 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 2: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xce6) [0x556df6828de6] 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 3: main() 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 4: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f3fffe1fd90] 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 5: __libc_start_main() 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 6: _start() 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 0> 2026-02-24T22:18:51.690+0000 7f3ffd6dda80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f3ffd6dda80 thread_name:radosgw-admin 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f3fffe38520] 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f4000b1ff10] 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 6: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xce6) [0x556df6828de6] 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 7: main() 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 8: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f3fffe1fd90] 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 9: __libc_start_main() 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 10: _start() 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr:--- logging levels --- 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 none 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 lockdep 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 context 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 crush 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_balancer 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_locker 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log_expire 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_migrator 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 3/ 5 mds_quiesce 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 buffer 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 timer 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 filer 2026-02-24T22:18:51.768 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 striper 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 objecter 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rados 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_mirror 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_replay 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_pwl 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 journaler 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objectcacher 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 immutable_obj_cache 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 client 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 osd 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 optracker 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objclass 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 filestore 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 journal 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 ms 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mon 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 0/10 monc 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 paxos 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 tp 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 auth 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 crypto 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 finisher 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 reserver 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 heartbeatmap 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 perfcounter 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_sync 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_datacache 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_access 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dbstore 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_flight 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_lifecycle 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_restore 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw_notification 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_bucket_logging 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 javaclient 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 asok 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 throttle 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 refs 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 compressor 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore_compression 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluefs 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 bdev 2026-02-24T22:18:51.769 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 kstore 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 4/ 5 rocksdb 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 fuse 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 2/ 5 mgr 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mgrc 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 dpdk 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 eventtrace 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 prioritycache 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 test 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephfs_mirror 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephsqlite 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 crimson_interrupt 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_onode 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_odata 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_omap 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_tm 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_t 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cleaner 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_epm 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_lba 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_fixedkv_tree 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cache 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_journal 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_device 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_backref 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 alienstore 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mclock 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dedup 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cyanstore 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 ceph_exporter 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 memstore 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 trace 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 ceph_dedup 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: -2/-2 (syslog threshold) 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 99/99 (stderr threshold) 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr:--- pthread ID / name mapping for recent threads --- 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 7f3f127fc640 / http_manager 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 7f3f13fff640 / http_manager 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 7f3fd4ff9640 / ms_dispatch 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 7f3fd5ffb640 / io_context_pool 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 7f3fd67fc640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 7f3fdcff9640 / ms_dispatch 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 7f3fdf7fe640 / ms_dispatch 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 7f3fed7fa640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 7f3fedffb640 / admin_socket 2026-02-24T22:18:51.770 INFO:tasks.workunit.client.0.trial046.stderr: 7f3feffff640 / msgr-worker-2 2026-02-24T22:18:51.771 INFO:tasks.workunit.client.0.trial046.stderr: 7f3ff67fc640 / ms_dispatch 2026-02-24T22:18:51.771 INFO:tasks.workunit.client.0.trial046.stderr: 7f3ff6ffd640 / msgr-worker-1 2026-02-24T22:18:51.771 INFO:tasks.workunit.client.0.trial046.stderr: 7f3ff77fe640 / msgr-worker-0 2026-02-24T22:18:51.771 INFO:tasks.workunit.client.0.trial046.stderr: 7f3ff7fff640 / io_context_pool 2026-02-24T22:18:51.771 INFO:tasks.workunit.client.0.trial046.stderr: 7f3ffd6dda80 / radosgw-admin 2026-02-24T22:18:51.771 INFO:tasks.workunit.client.0.trial046.stderr: max_recent 500 2026-02-24T22:18:51.771 INFO:tasks.workunit.client.0.trial046.stderr: max_new 1000 2026-02-24T22:18:51.771 INFO:tasks.workunit.client.0.trial046.stderr: log_file /var/lib/ceph/crash/2026-02-24T22:18:51.695174Z_096a9bca-4a18-45ce-831e-7eb5ef318f72/log 2026-02-24T22:18:51.771 INFO:tasks.workunit.client.0.trial046.stderr:--- end dump of recent events --- 2026-02-24T22:18:51.771 INFO:tasks.workunit.client.0.trial046.stderr:Aborted 2026-02-24T22:18:51.771 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket abort-at-commit-target-layout 2026-02-24T22:18:51.771 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:18:51.771 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:18:51.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.730+0000 7f6ee8a38a80 4 Realm: () 2026-02-24T22:18:51.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.730+0000 7f6ee8a38a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:18:51.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.730+0000 7f6ee8a38a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:18:51.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.730+0000 7f6ee8a38a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:18:51.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.730+0000 7f6ee8a38a80 20 zonegroup default 2026-02-24T22:18:51.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.730+0000 7f6ee8a38a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:51.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.730+0000 7f6ee8a38a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:51.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.730+0000 7f6ee8a38a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:51.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.730+0000 7f6ee8a38a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:18:51.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.730+0000 7f6ee8a38a80 20 started sync module instance, tier type = 2026-02-24T22:18:51.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.730+0000 7f6ee8a38a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:18:51.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.734+0000 7f6ee8a38a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:18:51.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.734+0000 7f6ee8a38a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:18:51.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.734+0000 7f6ee8a38a80 5 note: GC not initialized 2026-02-24T22:18:51.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.734+0000 7f6e07fff640 20 reqs_thread_entry: start 2026-02-24T22:18:51.937 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:51.942+0000 7f6ee6d0d640 0 -- 10.20.193.46:0/2916966285 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x562504d98950 msgr2=0x562504d8ed80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:52.349 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:52.354+0000 7f6ee650c640 0 -- 10.20.193.46:0/2916966285 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x562504d69270 msgr2=0x562504d69640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:18:52.376 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:52.378+0000 7f6ee8a38a80 20 init_complete bucket index max shards: 1 2026-02-24T22:18:52.376 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:52.378+0000 7f6ee8a38a80 20 Filter name: none 2026-02-24T22:18:52.376 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:52.378+0000 7f6e067fc640 20 reqs_thread_entry: start 2026-02-24T22:18:52.376 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:52.378+0000 7f6ee8a38a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:52.376 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:52.378+0000 7f6ee8a38a80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:18:52.376 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:52.378+0000 7f6ee8a38a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:52.378 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:52.382+0000 7f6ee8a38a80 20 rados_obj.operate() r=0 bl.length=491 2026-02-24T22:18:52.378 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:52.382+0000 7f6ee8a38a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:52.378 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:52.382+0000 7f6ee8a38a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:52.378 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:52.382+0000 7f6ee8a38a80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:18:52.378 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:52.382+0000 7f6ee8a38a80 20 rados->read ofs=0 len=0 2026-02-24T22:18:52.378 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:52.382+0000 7f6ee8a38a80 20 rados_obj.operate() r=0 bl.length=491 2026-02-24T22:18:52.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:18:52.382+0000 7f6ee8a38a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:18:52.389 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:52.389 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:18:52.390 INFO:tasks.workunit.client.0.trial046.stderr:bucket: abort-at-commit-target-layout id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:18:52.390 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:18:52.391 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:18:52.391 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:22.413 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:22.413 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:22.457 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket abort-at-commit-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2026-02-24T22:19:22.474 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:19:22.474 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:19:22.695 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.698+0000 7fcb88dd4a80 4 Realm: () 2026-02-24T22:19:22.695 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.698+0000 7fcb88dd4a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:19:22.695 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.698+0000 7fcb88dd4a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:19:22.695 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.698+0000 7fcb88dd4a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:19:22.696 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.698+0000 7fcb88dd4a80 20 zonegroup default 2026-02-24T22:19:22.696 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.698+0000 7fcb88dd4a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:22.697 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.702+0000 7fcb88dd4a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:22.697 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.702+0000 7fcb88dd4a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:22.698 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.702+0000 7fcb88dd4a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:22.698 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.702+0000 7fcb88dd4a80 20 started sync module instance, tier type = 2026-02-24T22:19:22.698 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.702+0000 7fcb88dd4a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:19:22.698 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.702+0000 7fcb870a9640 0 -- 10.20.193.46:0/2859689134 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55a9eaa17db0 msgr2=0x55a9eaa38190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:22.699 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.702+0000 7fcb88dd4a80 20 add_watcher() i=1 2026-02-24T22:19:22.700 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.702+0000 7fcb88dd4a80 20 add_watcher() i=6 2026-02-24T22:19:22.700 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.702+0000 7fcb870a9640 0 -- 10.20.193.46:0/2859689134 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x7fcb4809d4d0 msgr2=0x7fcb480bd8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:22.700 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.702+0000 7fcb88dd4a80 20 add_watcher() i=3 2026-02-24T22:19:22.700 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.702+0000 7fcb88dd4a80 20 add_watcher() i=5 2026-02-24T22:19:22.701 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.702+0000 7fcb88dd4a80 20 add_watcher() i=7 2026-02-24T22:19:22.701 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.702+0000 7fcb88dd4a80 20 add_watcher() i=2 2026-02-24T22:19:22.701 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.702+0000 7fcb88dd4a80 20 add_watcher() i=4 2026-02-24T22:19:22.701 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.702+0000 7fcb88dd4a80 20 add_watcher() i=0 2026-02-24T22:19:22.701 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.702+0000 7fcb88dd4a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:19:22.702 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.706+0000 7fcb88dd4a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:19:22.702 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.706+0000 7fcb88dd4a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:19:22.702 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.706+0000 7fcb88dd4a80 5 note: GC not initialized 2026-02-24T22:19:22.702 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.706+0000 7fcb34fe1640 20 reqs_thread_entry: start 2026-02-24T22:19:22.734 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.738+0000 7fcb88dd4a80 20 init_complete bucket index max shards: 1 2026-02-24T22:19:22.734 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.738+0000 7fcb88dd4a80 20 Filter name: none 2026-02-24T22:19:22.734 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.738+0000 7fca9effd640 20 reqs_thread_entry: start 2026-02-24T22:19:22.734 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.738+0000 7fcb88dd4a80 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2026-02-24T22:19:22.734 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.738+0000 7fcb88dd4a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:22.735 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.738+0000 7fcb88dd4a80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:19:22.735 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.738+0000 7fcb88dd4a80 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2026-02-24T22:19:22.735 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.738+0000 7fcb88dd4a80 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2026-02-24T22:19:22.735 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.738+0000 7fcb88dd4a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 : miss 2026-02-24T22:19:22.735 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.738+0000 7fcb88dd4a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:22.736 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.738+0000 7fcb88dd4a80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:19:22.736 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.738+0000 7fcb88dd4a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 info.flags=0x17 2026-02-24T22:19:22.736 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.738+0000 7fcb88dd4a80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 to cache LRU end 2026-02-24T22:19:22.736 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.738+0000 7fcb88dd4a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:19:22.736 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.738+0000 7fcb88dd4a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:22.736 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.738+0000 7fcb88dd4a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 2026-02-24T22:19:22.739 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.742+0000 7fcb88dd4a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 info.flags=0x17 2026-02-24T22:19:22.739 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.742+0000 7fcb88dd4a80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 to cache LRU end 2026-02-24T22:19:22.740 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.742+0000 7fcb88dd4a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:22.740 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.742+0000 7fcb88dd4a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533, ofs0, ns] 2026-02-24T22:19:22.740 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.742+0000 7fcb627fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959916 cookie 94188274290256 notifier 7154 bl.length()=1033 2026-02-24T22:19:22.740 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.742+0000 7fcb627fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 info.flags=0x17 2026-02-24T22:19:22.740 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.742+0000 7fcb627fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 to cache LRU end 2026-02-24T22:19:22.740 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.742+0000 7fcb627fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:22.740 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.742+0000 7fcb88dd4a80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): orig_sources= new_sources= 2026-02-24T22:19:22.741 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.742+0000 7fcb88dd4a80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): potential sources added=[] removed=[] 2026-02-24T22:19:22.741 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.742+0000 7fcb88dd4a80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): orig_dests= new_dests= 2026-02-24T22:19:22.741 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.742+0000 7fcb88dd4a80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): potential dests added=[] removed=[] 2026-02-24T22:19:22.741 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.746+0000 7fcb88dd4a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533.1.0 2026-02-24T22:19:22.742 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.746+0000 7fcb88dd4a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533.1.0 2026-02-24T22:19:22.742 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.746+0000 7fcb88dd4a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533.1.1 2026-02-24T22:19:22.742 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.746+0000 7fcb88dd4a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533.1.1 2026-02-24T22:19:22.742 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.746+0000 7fcb88dd4a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533.0 2026-02-24T22:19:22.743 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.746+0000 7fcb88dd4a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 : hit (requested=0x17, cached=0x17) 2026-02-24T22:19:22.743 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.746+0000 7fcb88dd4a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 2026-02-24T22:19:22.744 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.746+0000 7fcb88dd4a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 info.flags=0x17 2026-02-24T22:19:22.744 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.746+0000 7fcb88dd4a80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 to cache LRU end 2026-02-24T22:19:22.744 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.746+0000 7fcb88dd4a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:22.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.746+0000 7fcb88dd4a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533, ofs0, ns] 2026-02-24T22:19:22.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.746+0000 7fcb627fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959917 cookie 94188274290256 notifier 7154 bl.length()=1033 2026-02-24T22:19:22.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.746+0000 7fcb627fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 info.flags=0x17 2026-02-24T22:19:22.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.746+0000 7fcb627fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 to cache LRU end 2026-02-24T22:19:22.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.746+0000 7fcb627fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:22.746 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.750+0000 7fcb88dd4a80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): orig_sources= new_sources= 2026-02-24T22:19:22.746 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.750+0000 7fcb88dd4a80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): potential sources added=[] removed=[] 2026-02-24T22:19:22.746 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.750+0000 7fcb88dd4a80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): orig_dests= new_dests= 2026-02-24T22:19:22.746 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.750+0000 7fcb88dd4a80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): potential dests added=[] removed=[] 2026-02-24T22:19:22.746 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.750+0000 7fcb88dd4a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533.0 2026-02-24T22:19:22.746 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.750+0000 7fcb88dd4a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 : hit (requested=0x17, cached=0x17) 2026-02-24T22:19:22.746 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.750+0000 7fcb88dd4a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 2026-02-24T22:19:22.747 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.750+0000 7fcb88dd4a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 info.flags=0x17 2026-02-24T22:19:22.748 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.750+0000 7fcb88dd4a80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 to cache LRU end 2026-02-24T22:19:22.748 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.750+0000 7fcb88dd4a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:22.748 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.750+0000 7fcb88dd4a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533, ofs0, ns] 2026-02-24T22:19:22.748 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.750+0000 7fcb627fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959918 cookie 94188274290256 notifier 7154 bl.length()=997 2026-02-24T22:19:22.748 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.750+0000 7fcb627fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 info.flags=0x17 2026-02-24T22:19:22.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.750+0000 7fcb627fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 to cache LRU end 2026-02-24T22:19:22.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.750+0000 7fcb627fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:22.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.754+0000 7fcb88dd4a80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): orig_sources= new_sources= 2026-02-24T22:19:22.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.754+0000 7fcb88dd4a80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): potential sources added=[] removed=[] 2026-02-24T22:19:22.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.754+0000 7fcb88dd4a80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): orig_dests= new_dests= 2026-02-24T22:19:22.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.754+0000 7fcb88dd4a80 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533]): potential dests added=[] removed=[] 2026-02-24T22:19:22.750 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.754+0000 7fcb88dd4a80 1 execute INFO: reshard of bucket "abort-at-commit-target-layout" from 1 shards to 2 shards completed successfully 2026-02-24T22:19:22.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.758+0000 7fcb88dd4a80 20 remove_watcher() i=3 2026-02-24T22:19:22.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.758+0000 7fcb88dd4a80 2 removed watcher, disabling cache 2026-02-24T22:19:22.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.758+0000 7fcb88dd4a80 20 remove_watcher() i=1 2026-02-24T22:19:22.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.758+0000 7fcb88dd4a80 20 remove_watcher() i=4 2026-02-24T22:19:22.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.758+0000 7fcb88dd4a80 20 remove_watcher() i=2 2026-02-24T22:19:22.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.758+0000 7fcb88dd4a80 20 remove_watcher() i=0 2026-02-24T22:19:22.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.758+0000 7fcb88dd4a80 20 remove_watcher() i=5 2026-02-24T22:19:22.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.758+0000 7fcb88dd4a80 20 remove_watcher() i=7 2026-02-24T22:19:22.755 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.758+0000 7fcb88dd4a80 20 remove_watcher() i=6 2026-02-24T22:19:22.762 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket abort-at-commit-target-layout 2026-02-24T22:19:22.777 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:19:22.778 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:19:22.794 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.798+0000 7f89beb88a80 4 Realm: () 2026-02-24T22:19:22.795 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.798+0000 7f89beb88a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:19:22.795 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.798+0000 7f89beb88a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:19:22.795 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.798+0000 7f89beb88a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:19:22.795 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.798+0000 7f89beb88a80 20 zonegroup default 2026-02-24T22:19:22.795 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.798+0000 7f89beb88a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:22.796 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.798+0000 7f89beb88a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:22.796 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.798+0000 7f89beb88a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:22.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.802+0000 7f89beb88a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:22.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.802+0000 7f89beb88a80 20 started sync module instance, tier type = 2026-02-24T22:19:22.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.802+0000 7f89beb88a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:19:22.798 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.802+0000 7f89beb88a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:19:22.798 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.802+0000 7f89beb88a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:19:22.799 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.802+0000 7f89beb88a80 5 note: GC not initialized 2026-02-24T22:19:22.799 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.802+0000 7f896a7dc640 20 reqs_thread_entry: start 2026-02-24T22:19:22.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:22.806+0000 7f89b7fff640 0 -- 10.20.193.46:0/3095188695 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x5639f95d91f0 msgr2=0x5639f95d9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:23.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.018+0000 7f89b7fff640 0 -- 10.20.193.46:0/3095188695 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5639f95b2560 msgr2=0x5639f95e00a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:23.040 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.042+0000 7f89beb88a80 20 init_complete bucket index max shards: 1 2026-02-24T22:19:23.041 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.042+0000 7f89beb88a80 20 Filter name: none 2026-02-24T22:19:23.041 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.042+0000 7f8968fd9640 20 reqs_thread_entry: start 2026-02-24T22:19:23.041 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.046+0000 7f89beb88a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:23.041 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.046+0000 7f89beb88a80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:19:23.041 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.046+0000 7f89beb88a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:23.042 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.046+0000 7f89beb88a80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:19:23.043 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.046+0000 7f89beb88a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:19:23.043 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.046+0000 7f89beb88a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:23.043 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.046+0000 7f89beb88a80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:19:23.043 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.046+0000 7f89beb88a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:23.043 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.046+0000 7f89beb88a80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:19:23.043 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.046+0000 7f89beb88a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:19:23.054 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:23.054 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:19:23.054 INFO:tasks.workunit.client.0.trial046.stderr:bucket: abort-at-commit-target-layout id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533 num_objects: 19 size_kb: 1 num_shards: 2 2026-02-24T22:19:23.054 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:23.055 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:23.055 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:23.058 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:23.058 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:23.102 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:23.102 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:23.112 INFO:tasks.workunit.client.0.trial046.stderr:TEST: reshard bucket with EIO injected at do_reshard 2026-02-24T22:19:23.112 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:23.113 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:23.113 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:23.121 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:23.122 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:23.166 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:23.166 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:23.172 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:23.173 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:23.178 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:23.178 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:23.184 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:23.184 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:23.190 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:23.190 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:23.196 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:23.196 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:23.202 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:23.202 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:23.207 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:23.207 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:23.214 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:23.214 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:23.220 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:23.220 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:23.226 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:23.226 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:23.232 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:23.233 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:23.239 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:23.239 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:23.245 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:23.245 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:23.251 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:23.251 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:23.257 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:23.257 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:23.263 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:23.263 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:23.268 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:23.269 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:23.275 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:23.275 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:23.281 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:23.281 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:23.286 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-do-reshard 2026-02-24T22:19:23.301 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:19:23.301 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:19:23.319 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.322+0000 7f2600b19a80 4 Realm: () 2026-02-24T22:19:23.319 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.322+0000 7f2600b19a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:19:23.320 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.322+0000 7f2600b19a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:19:23.320 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.322+0000 7f2600b19a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:19:23.320 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.322+0000 7f2600b19a80 20 zonegroup default 2026-02-24T22:19:23.320 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.322+0000 7f2600b19a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:23.321 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.326+0000 7f2600b19a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:23.321 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.326+0000 7f2600b19a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:23.321 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.326+0000 7f2600b19a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:23.321 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.326+0000 7f2600b19a80 20 started sync module instance, tier type = 2026-02-24T22:19:23.322 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.326+0000 7f2600b19a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:19:23.323 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.326+0000 7f2600b19a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:19:23.323 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.326+0000 7f2600b19a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:19:23.323 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.326+0000 7f2600b19a80 5 note: GC not initialized 2026-02-24T22:19:23.323 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.326+0000 7f251ffff640 20 reqs_thread_entry: start 2026-02-24T22:19:23.326 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.330+0000 7f25fedee640 0 -- 10.20.193.46:0/3901190129 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x560572760330 msgr2=0x5605727535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:19:23.538 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.542+0000 7f25fe5ed640 0 -- 10.20.193.46:0/3901190129 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x56057275bdb0 msgr2=0x560572756b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:23.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.570+0000 7f2600b19a80 20 init_complete bucket index max shards: 1 2026-02-24T22:19:23.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.570+0000 7f2600b19a80 20 Filter name: none 2026-02-24T22:19:23.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.570+0000 7f251e7fc640 20 reqs_thread_entry: start 2026-02-24T22:19:23.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.570+0000 7f2600b19a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:23.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.570+0000 7f2600b19a80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:19:23.567 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.570+0000 7f2600b19a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:23.568 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.570+0000 7f2600b19a80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:19:23.568 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.570+0000 7f2600b19a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:19:23.568 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.570+0000 7f2600b19a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:23.568 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.570+0000 7f2600b19a80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:19:23.568 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.570+0000 7f2600b19a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:23.568 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.570+0000 7f2600b19a80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:19:23.568 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.570+0000 7f2600b19a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:19:23.578 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:23.578 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:19:23.578 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-do-reshard id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:19:23.579 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:23.579 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket error-at-do-reshard --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at do_reshard 2026-02-24T22:19:23.595 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:19:23.595 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:19:23.612 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.614+0000 7f0064c42a80 4 Realm: () 2026-02-24T22:19:23.612 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.614+0000 7f0064c42a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:19:23.612 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.614+0000 7f0064c42a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:19:23.612 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.614+0000 7f0064c42a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:19:23.612 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.614+0000 7f0064c42a80 20 zonegroup default 2026-02-24T22:19:23.613 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.618+0000 7f0064c42a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:23.613 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.618+0000 7f0064c42a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:23.613 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.618+0000 7f0064c42a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:23.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.618+0000 7f0064c42a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:23.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.618+0000 7f0064c42a80 20 started sync module instance, tier type = 2026-02-24T22:19:23.614 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.618+0000 7f0064c42a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:19:23.615 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.618+0000 7f0062716640 0 -- 10.20.193.46:0/2230602828 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x562d167f2430 msgr2=0x562d168128b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:23.616 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.618+0000 7f0064c42a80 20 add_watcher() i=1 2026-02-24T22:19:23.616 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.618+0000 7f0064c42a80 20 add_watcher() i=0 2026-02-24T22:19:23.616 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.618+0000 7f0064c42a80 20 add_watcher() i=3 2026-02-24T22:19:23.616 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.618+0000 7f0064c42a80 20 add_watcher() i=6 2026-02-24T22:19:23.616 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.618+0000 7f0064c42a80 20 add_watcher() i=5 2026-02-24T22:19:23.617 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.622+0000 7f0064c42a80 20 add_watcher() i=2 2026-02-24T22:19:23.617 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.622+0000 7f0064c42a80 20 add_watcher() i=4 2026-02-24T22:19:23.617 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.622+0000 7f0064c42a80 20 add_watcher() i=7 2026-02-24T22:19:23.617 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.622+0000 7f0064c42a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:19:23.619 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.622+0000 7f0064c42a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:19:23.619 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.622+0000 7f0064c42a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:19:23.619 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.622+0000 7f0064c42a80 5 note: GC not initialized 2026-02-24T22:19:23.619 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.622+0000 7eff7b7fe640 20 reqs_thread_entry: start 2026-02-24T22:19:23.619 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.622+0000 7f0062f17640 0 -- 10.20.193.46:0/4156791327 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x562d167bf4f0 msgr2=0x562d167bf8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:19:23.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.862+0000 7f0064c42a80 20 init_complete bucket index max shards: 1 2026-02-24T22:19:23.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.862+0000 7f0064c42a80 20 Filter name: none 2026-02-24T22:19:23.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.862+0000 7eff79ffb640 20 reqs_thread_entry: start 2026-02-24T22:19:23.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.862+0000 7f0064c42a80 10 cache get: name=default.rgw.meta+root+error-at-do-reshard : miss 2026-02-24T22:19:23.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.862+0000 7f0064c42a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:23.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.862+0000 7f0064c42a80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:19:23.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.862+0000 7f0064c42a80 10 cache put: name=default.rgw.meta+root+error-at-do-reshard info.flags=0x11 2026-02-24T22:19:23.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.862+0000 7f0064c42a80 10 adding default.rgw.meta+root+error-at-do-reshard to cache LRU end 2026-02-24T22:19:23.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.862+0000 7f0064c42a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 : miss 2026-02-24T22:19:23.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.862+0000 7f0064c42a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:23.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.862+0000 7f0064c42a80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:19:23.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.862+0000 7f0064c42a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 info.flags=0x17 2026-02-24T22:19:23.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.862+0000 7f0064c42a80 10 adding default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 to cache LRU end 2026-02-24T22:19:23.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.862+0000 7f0064c42a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:19:23.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.862+0000 7f0064c42a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:23.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.862+0000 7f0064c42a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 2026-02-24T22:19:23.865 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.870+0000 7f0064c42a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 info.flags=0x17 2026-02-24T22:19:23.865 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.870+0000 7f0064c42a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 to cache LRU end 2026-02-24T22:19:23.865 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.870+0000 7f0064c42a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:23.865 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.870+0000 7f0064c42a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534, ofs0, ns] 2026-02-24T22:19:23.865 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.870+0000 7f003d7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961365 cookie 94751650134080 notifier 7704 bl.length()=1013 2026-02-24T22:19:23.865 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.870+0000 7f003d7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 info.flags=0x17 2026-02-24T22:19:23.866 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.870+0000 7f003d7fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 to cache LRU end 2026-02-24T22:19:23.866 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.870+0000 7f003d7fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:23.866 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.870+0000 7f0064c42a80 20 handle_bi_update(): bucket=:error-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534]): orig_sources= new_sources= 2026-02-24T22:19:23.866 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.870+0000 7f0064c42a80 20 handle_bi_update(): bucket=:error-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534]): potential sources added=[] removed=[] 2026-02-24T22:19:23.866 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.870+0000 7f0064c42a80 20 handle_bi_update(): bucket=:error-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534]): orig_dests= new_dests= 2026-02-24T22:19:23.866 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.870+0000 7f0064c42a80 20 handle_bi_update(): bucket=:error-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534]): potential dests added=[] removed=[] 2026-02-24T22:19:23.867 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.870+0000 7f0064c42a80 -1 Injecting error=-5 at location=do_reshard 2026-02-24T22:19:23.869 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.874+0000 7f0064c42a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 : hit (requested=0x17, cached=0x17) 2026-02-24T22:19:23.869 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.874+0000 7f0064c42a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 2026-02-24T22:19:23.870 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.874+0000 7f0064c42a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 info.flags=0x17 2026-02-24T22:19:23.870 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.874+0000 7f0064c42a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 to cache LRU end 2026-02-24T22:19:23.871 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.874+0000 7f0064c42a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:23.871 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.874+0000 7f0064c42a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534, ofs0, ns] 2026-02-24T22:19:23.871 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.874+0000 7f003dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961366 cookie 94751650134080 notifier 7704 bl.length()=977 2026-02-24T22:19:23.871 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.874+0000 7f003dffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 info.flags=0x17 2026-02-24T22:19:23.871 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.874+0000 7f003dffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 to cache LRU end 2026-02-24T22:19:23.871 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.874+0000 7f003dffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:23.871 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.874+0000 7f0064c42a80 20 handle_bi_update(): bucket=:error-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534]): orig_sources= new_sources= 2026-02-24T22:19:23.872 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.874+0000 7f0064c42a80 20 handle_bi_update(): bucket=:error-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534]): potential sources added=[] removed=[] 2026-02-24T22:19:23.872 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.874+0000 7f0064c42a80 20 handle_bi_update(): bucket=:error-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534]): orig_dests= new_dests= 2026-02-24T22:19:23.872 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.874+0000 7f0064c42a80 20 handle_bi_update(): bucket=:error-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534]): potential dests added=[] removed=[] 2026-02-24T22:19:23.872 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.874+0000 7f0064c42a80 1 execute INFO: reshard of bucket "error-at-do-reshard" canceled due to errors 2026-02-24T22:19:23.875 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.878+0000 7f0064c42a80 20 remove_watcher() i=1 2026-02-24T22:19:23.876 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.878+0000 7f0064c42a80 2 removed watcher, disabling cache 2026-02-24T22:19:23.876 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.878+0000 7f0064c42a80 20 remove_watcher() i=6 2026-02-24T22:19:23.876 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.878+0000 7f0064c42a80 20 remove_watcher() i=3 2026-02-24T22:19:23.876 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.878+0000 7f0064c42a80 20 remove_watcher() i=5 2026-02-24T22:19:23.876 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.878+0000 7f0064c42a80 20 remove_watcher() i=2 2026-02-24T22:19:23.876 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.878+0000 7f0064c42a80 20 remove_watcher() i=4 2026-02-24T22:19:23.876 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.878+0000 7f0064c42a80 20 remove_watcher() i=0 2026-02-24T22:19:23.876 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.878+0000 7f0064c42a80 20 remove_watcher() i=7 2026-02-24T22:19:23.882 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-do-reshard 2026-02-24T22:19:23.897 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:19:23.897 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:19:23.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.918+0000 7feaa4ed4a80 4 Realm: () 2026-02-24T22:19:23.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.918+0000 7feaa4ed4a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:19:23.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.918+0000 7feaa4ed4a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:19:23.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.918+0000 7feaa4ed4a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:19:23.915 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.918+0000 7feaa4ed4a80 20 zonegroup default 2026-02-24T22:19:23.916 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.918+0000 7feaa4ed4a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:23.917 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.922+0000 7feaa4ed4a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:23.917 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.922+0000 7feaa4ed4a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:23.917 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.922+0000 7feaa4ed4a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:23.917 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.922+0000 7feaa4ed4a80 20 started sync module instance, tier type = 2026-02-24T22:19:23.918 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.922+0000 7feaa4ed4a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:19:23.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.922+0000 7feaa4ed4a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:19:23.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.922+0000 7feaa4ed4a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:19:23.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.922+0000 7feaa4ed4a80 5 note: GC not initialized 2026-02-24T22:19:23.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.922+0000 7fea517e2640 20 reqs_thread_entry: start 2026-02-24T22:19:23.921 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:23.926+0000 7feaa31a9640 0 -- 10.20.193.46:0/227262905 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x555cd170f8a0 msgr2=0x555cd1706090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:19:24.132 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.134+0000 7feaa29a8640 0 -- 10.20.193.46:0/227262905 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x555cd16e0c80 msgr2=0x555cd170e8a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:24.159 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.162+0000 7feaa4ed4a80 20 init_complete bucket index max shards: 1 2026-02-24T22:19:24.160 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.162+0000 7feaa4ed4a80 20 Filter name: none 2026-02-24T22:19:24.160 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.162+0000 7fe9bb7fe640 20 reqs_thread_entry: start 2026-02-24T22:19:24.160 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.162+0000 7feaa4ed4a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:24.160 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.162+0000 7feaa4ed4a80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:19:24.161 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.162+0000 7feaa4ed4a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:24.161 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.166+0000 7feaa4ed4a80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:19:24.161 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.166+0000 7feaa4ed4a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:19:24.162 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.166+0000 7feaa4ed4a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:24.162 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.166+0000 7feaa4ed4a80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:19:24.162 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.166+0000 7feaa4ed4a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:24.162 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.166+0000 7feaa4ed4a80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:19:24.162 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.166+0000 7feaa4ed4a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:19:24.171 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:24.171 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:19:24.171 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-do-reshard id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:19:24.171 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:24.172 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:24.172 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:24.178 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:24.178 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:24.225 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket error-at-do-reshard --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2026-02-24T22:19:24.241 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:19:24.241 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:19:24.259 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.262+0000 7fcd95300a80 4 Realm: () 2026-02-24T22:19:24.259 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.262+0000 7fcd95300a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:19:24.259 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.262+0000 7fcd95300a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:19:24.259 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.262+0000 7fcd95300a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:19:24.259 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.262+0000 7fcd95300a80 20 zonegroup default 2026-02-24T22:19:24.259 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.262+0000 7fcd95300a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:24.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.262+0000 7fcd95300a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:24.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.262+0000 7fcd95300a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:24.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.266+0000 7fcd95300a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:24.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.266+0000 7fcd95300a80 20 started sync module instance, tier type = 2026-02-24T22:19:24.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.266+0000 7fcd95300a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:19:24.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.266+0000 7fcd925d3640 0 -- 10.20.193.46:0/1944426940 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x564e61ef2450 msgr2=0x564e61f128d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:24.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.266+0000 7fcd95300a80 20 add_watcher() i=3 2026-02-24T22:19:24.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.266+0000 7fcd95300a80 20 add_watcher() i=0 2026-02-24T22:19:24.263 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.266+0000 7fcd95300a80 20 add_watcher() i=6 2026-02-24T22:19:24.263 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.266+0000 7fcd95300a80 20 add_watcher() i=1 2026-02-24T22:19:24.263 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.266+0000 7fcd95300a80 20 add_watcher() i=5 2026-02-24T22:19:24.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.266+0000 7fcd95300a80 20 add_watcher() i=2 2026-02-24T22:19:24.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.266+0000 7fcd95300a80 20 add_watcher() i=7 2026-02-24T22:19:24.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.266+0000 7fcd95300a80 20 add_watcher() i=4 2026-02-24T22:19:24.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.266+0000 7fcd95300a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:19:24.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.266+0000 7fcd925d3640 0 -- 10.20.193.46:0/913882283 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7fccb4008ac0 msgr2=0x7fccb4028f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:24.265 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.270+0000 7fcd95300a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:19:24.266 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.270+0000 7fcd95300a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:19:24.266 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.270+0000 7fcd95300a80 5 note: GC not initialized 2026-02-24T22:19:24.266 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.270+0000 7fcd40fe1640 20 reqs_thread_entry: start 2026-02-24T22:19:24.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.298+0000 7fcd95300a80 20 init_complete bucket index max shards: 1 2026-02-24T22:19:24.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.298+0000 7fcd95300a80 20 Filter name: none 2026-02-24T22:19:24.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.298+0000 7fccaaffd640 20 reqs_thread_entry: start 2026-02-24T22:19:24.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.298+0000 7fcd95300a80 10 cache get: name=default.rgw.meta+root+error-at-do-reshard : miss 2026-02-24T22:19:24.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.298+0000 7fcd95300a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:24.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.302+0000 7fcd95300a80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:19:24.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.302+0000 7fcd95300a80 10 cache put: name=default.rgw.meta+root+error-at-do-reshard info.flags=0x11 2026-02-24T22:19:24.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.302+0000 7fcd95300a80 10 adding default.rgw.meta+root+error-at-do-reshard to cache LRU end 2026-02-24T22:19:24.298 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.302+0000 7fcd95300a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 : miss 2026-02-24T22:19:24.298 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.302+0000 7fcd95300a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:24.298 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.302+0000 7fcd95300a80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:19:24.298 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.302+0000 7fcd95300a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 info.flags=0x17 2026-02-24T22:19:24.298 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.302+0000 7fcd95300a80 10 adding default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 to cache LRU end 2026-02-24T22:19:24.298 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.302+0000 7fcd95300a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:19:24.298 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.302+0000 7fcd95300a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:24.298 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.302+0000 7fcd95300a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 2026-02-24T22:19:24.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.306+0000 7fcd95300a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 info.flags=0x17 2026-02-24T22:19:24.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.306+0000 7fcd95300a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 to cache LRU end 2026-02-24T22:19:24.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.306+0000 7fcd95300a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:24.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.306+0000 7fcd95300a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534, ofs0, ns] 2026-02-24T22:19:24.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.306+0000 7fcd6e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961367 cookie 94894650358880 notifier 7743 bl.length()=1013 2026-02-24T22:19:24.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.306+0000 7fcd6e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 info.flags=0x17 2026-02-24T22:19:24.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.306+0000 7fcd6e7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 to cache LRU end 2026-02-24T22:19:24.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.306+0000 7fcd6e7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:24.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.306+0000 7fcd95300a80 20 handle_bi_update(): bucket=:error-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534]): orig_sources= new_sources= 2026-02-24T22:19:24.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.306+0000 7fcd95300a80 20 handle_bi_update(): bucket=:error-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534]): potential sources added=[] removed=[] 2026-02-24T22:19:24.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.306+0000 7fcd95300a80 20 handle_bi_update(): bucket=:error-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534]): orig_dests= new_dests= 2026-02-24T22:19:24.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.306+0000 7fcd95300a80 20 handle_bi_update(): bucket=:error-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534]): potential dests added=[] removed=[] 2026-02-24T22:19:24.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.306+0000 7fcd95300a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534.1.0 2026-02-24T22:19:24.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.306+0000 7fcd95300a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534.1.0 2026-02-24T22:19:24.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.306+0000 7fcd95300a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534.1.1 2026-02-24T22:19:24.303 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.306+0000 7fcd95300a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534.1.1 2026-02-24T22:19:24.303 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.306+0000 7fcd95300a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534.0 2026-02-24T22:19:24.304 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.306+0000 7fcd95300a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 : hit (requested=0x17, cached=0x17) 2026-02-24T22:19:24.304 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.306+0000 7fcd95300a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 2026-02-24T22:19:24.305 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd95300a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 info.flags=0x17 2026-02-24T22:19:24.305 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd95300a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 to cache LRU end 2026-02-24T22:19:24.305 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd95300a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:24.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd95300a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534, ofs0, ns] 2026-02-24T22:19:24.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd6e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961368 cookie 94894650358880 notifier 7743 bl.length()=1013 2026-02-24T22:19:24.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd6e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 info.flags=0x17 2026-02-24T22:19:24.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd6e7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 to cache LRU end 2026-02-24T22:19:24.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd6e7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:24.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd95300a80 20 handle_bi_update(): bucket=:error-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534]): orig_sources= new_sources= 2026-02-24T22:19:24.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd95300a80 20 handle_bi_update(): bucket=:error-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534]): potential sources added=[] removed=[] 2026-02-24T22:19:24.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd95300a80 20 handle_bi_update(): bucket=:error-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534]): orig_dests= new_dests= 2026-02-24T22:19:24.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd95300a80 20 handle_bi_update(): bucket=:error-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534]): potential dests added=[] removed=[] 2026-02-24T22:19:24.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd95300a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534.0 2026-02-24T22:19:24.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd95300a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 : hit (requested=0x17, cached=0x17) 2026-02-24T22:19:24.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd95300a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 2026-02-24T22:19:24.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd95300a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 info.flags=0x17 2026-02-24T22:19:24.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd95300a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 to cache LRU end 2026-02-24T22:19:24.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd95300a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:24.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd95300a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534, ofs0, ns] 2026-02-24T22:19:24.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd6e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961369 cookie 94894650358880 notifier 7743 bl.length()=977 2026-02-24T22:19:24.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd6e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 info.flags=0x17 2026-02-24T22:19:24.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd6e7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 to cache LRU end 2026-02-24T22:19:24.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd6e7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:24.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd95300a80 20 handle_bi_update(): bucket=:error-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534]): orig_sources= new_sources= 2026-02-24T22:19:24.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd95300a80 20 handle_bi_update(): bucket=:error-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534]): potential sources added=[] removed=[] 2026-02-24T22:19:24.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd95300a80 20 handle_bi_update(): bucket=:error-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534]): orig_dests= new_dests= 2026-02-24T22:19:24.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd95300a80 20 handle_bi_update(): bucket=:error-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534]): potential dests added=[] removed=[] 2026-02-24T22:19:24.309 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.310+0000 7fcd95300a80 1 execute INFO: reshard of bucket "error-at-do-reshard" from 1 shards to 2 shards completed successfully 2026-02-24T22:19:24.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.314+0000 7fcd95300a80 20 remove_watcher() i=0 2026-02-24T22:19:24.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.314+0000 7fcd95300a80 2 removed watcher, disabling cache 2026-02-24T22:19:24.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.314+0000 7fcd95300a80 20 remove_watcher() i=3 2026-02-24T22:19:24.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.314+0000 7fcd95300a80 20 remove_watcher() i=4 2026-02-24T22:19:24.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.314+0000 7fcd95300a80 20 remove_watcher() i=1 2026-02-24T22:19:24.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.314+0000 7fcd95300a80 20 remove_watcher() i=2 2026-02-24T22:19:24.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.314+0000 7fcd95300a80 20 remove_watcher() i=6 2026-02-24T22:19:24.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.314+0000 7fcd95300a80 20 remove_watcher() i=7 2026-02-24T22:19:24.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.314+0000 7fcd95300a80 20 remove_watcher() i=5 2026-02-24T22:19:24.319 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-do-reshard 2026-02-24T22:19:24.333 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:19:24.334 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:19:24.351 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.354+0000 7f4c51cafa80 4 Realm: () 2026-02-24T22:19:24.351 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.354+0000 7f4c51cafa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:19:24.351 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.354+0000 7f4c51cafa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:19:24.351 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.354+0000 7f4c51cafa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:19:24.352 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.354+0000 7f4c51cafa80 20 zonegroup default 2026-02-24T22:19:24.352 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.354+0000 7f4c51cafa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:24.353 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.358+0000 7f4c51cafa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:24.353 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.358+0000 7f4c51cafa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:24.354 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.358+0000 7f4c51cafa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:24.354 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.358+0000 7f4c51cafa80 20 started sync module instance, tier type = 2026-02-24T22:19:24.354 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.358+0000 7f4c51cafa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:19:24.355 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.358+0000 7f4c51cafa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:19:24.355 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.358+0000 7f4c51cafa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:19:24.355 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.358+0000 7f4c51cafa80 5 note: GC not initialized 2026-02-24T22:19:24.355 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.358+0000 7f4bfcfe1640 20 reqs_thread_entry: start 2026-02-24T22:19:24.561 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.566+0000 7f4c4b7fe640 0 -- 10.20.193.46:0/893878472 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x564261bcffb0 msgr2=0x564261bc3220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:24.972 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:24.974+0000 7f4c4affd640 0 -- 10.20.193.46:0/893878472 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x564261bcba30 msgr2=0x564261bc67a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:24.999 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.002+0000 7f4c51cafa80 20 init_complete bucket index max shards: 1 2026-02-24T22:19:24.999 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.002+0000 7f4c51cafa80 20 Filter name: none 2026-02-24T22:19:24.999 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.002+0000 7f4b6effd640 20 reqs_thread_entry: start 2026-02-24T22:19:24.999 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.002+0000 7f4c51cafa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:24.999 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.002+0000 7f4c51cafa80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:19:25.000 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.002+0000 7f4c51cafa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:25.000 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.002+0000 7f4c51cafa80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:19:25.001 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.002+0000 7f4c51cafa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:19:25.001 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.002+0000 7f4c51cafa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:25.001 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.002+0000 7f4c51cafa80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:19:25.001 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.002+0000 7f4c51cafa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:25.001 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.006+0000 7f4c51cafa80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:19:25.001 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.006+0000 7f4c51cafa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:19:25.012 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.012 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:19:25.012 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-do-reshard id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534 num_objects: 19 size_kb: 1 num_shards: 2 2026-02-24T22:19:25.012 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.013 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:25.013 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:25.016 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:25.016 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:25.066 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:25.066 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:25.077 INFO:tasks.workunit.client.0.trial046.stderr:TEST: reshard bucket with abort at do_reshard 2026-02-24T22:19:25.077 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.078 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:25.078 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:25.087 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:25.088 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:25.130 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:25.130 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:25.136 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:25.136 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:25.142 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:25.142 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:25.149 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:25.149 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:25.155 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:25.155 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:25.161 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:25.161 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:25.167 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:25.167 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:25.173 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:25.173 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:25.179 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:25.179 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:25.185 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:25.185 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:25.191 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:25.191 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:25.197 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:25.197 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:25.203 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:25.203 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:25.209 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:25.210 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:25.216 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:25.216 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:25.223 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:25.223 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:25.229 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:25.229 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:25.235 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:25.236 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:25.242 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:25.242 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:25.247 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:25.248 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:25.254 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket abort-at-do-reshard 2026-02-24T22:19:25.269 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:19:25.269 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:19:25.287 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.290+0000 7f8e29531a80 4 Realm: () 2026-02-24T22:19:25.287 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.290+0000 7f8e29531a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:19:25.287 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.290+0000 7f8e29531a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:19:25.287 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.290+0000 7f8e29531a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:19:25.288 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.290+0000 7f8e29531a80 20 zonegroup default 2026-02-24T22:19:25.288 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.290+0000 7f8e29531a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:25.289 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.294+0000 7f8e29531a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:25.289 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.294+0000 7f8e29531a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:25.289 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.294+0000 7f8e29531a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:25.289 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.294+0000 7f8e29531a80 20 started sync module instance, tier type = 2026-02-24T22:19:25.289 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.294+0000 7f8e29531a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:19:25.291 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.294+0000 7f8e29531a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:19:25.291 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.294+0000 7f8e29531a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:19:25.291 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.294+0000 7f8e29531a80 5 note: GC not initialized 2026-02-24T22:19:25.291 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.294+0000 7f8dd4fe1640 20 reqs_thread_entry: start 2026-02-24T22:19:25.294 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.298+0000 7f8e237fe640 0 -- 10.20.193.46:0/2635418185 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55881a5a1750 msgr2=0x55881a5a7e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:19:25.505 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.510+0000 7f8e22ffd640 0 -- 10.20.193.46:0/2635418185 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55881a575cc0 msgr2=0x55881a5a3900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:25.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.534+0000 7f8e29531a80 20 init_complete bucket index max shards: 1 2026-02-24T22:19:25.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.534+0000 7f8e29531a80 20 Filter name: none 2026-02-24T22:19:25.532 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.534+0000 7f8d46ffd640 20 reqs_thread_entry: start 2026-02-24T22:19:25.532 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.534+0000 7f8e29531a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:25.532 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.534+0000 7f8e29531a80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:19:25.532 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.534+0000 7f8e29531a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:25.533 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.538+0000 7f8e29531a80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:19:25.533 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.538+0000 7f8e29531a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:19:25.533 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.538+0000 7f8e29531a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:25.533 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.538+0000 7f8e29531a80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:19:25.534 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.538+0000 7f8e29531a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:25.534 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.538+0000 7f8e29531a80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:19:25.534 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.538+0000 7f8e29531a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:19:25.545 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.545 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:19:25.545 INFO:tasks.workunit.client.0.trial046.stderr:bucket: abort-at-do-reshard id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:19:25.545 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.545 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket abort-at-do-reshard --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at do_reshard 2026-02-24T22:19:25.562 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:19:25.562 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:19:25.579 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.582+0000 7f263b02aa80 4 Realm: () 2026-02-24T22:19:25.579 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.582+0000 7f263b02aa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:19:25.579 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.582+0000 7f263b02aa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:19:25.579 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.582+0000 7f263b02aa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:19:25.579 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.582+0000 7f263b02aa80 20 zonegroup default 2026-02-24T22:19:25.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.582+0000 7f263b02aa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:25.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.582+0000 7f263b02aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:25.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.582+0000 7f263b02aa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:25.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.586+0000 7f263b02aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:25.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.586+0000 7f263b02aa80 20 started sync module instance, tier type = 2026-02-24T22:19:25.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.586+0000 7f263b02aa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:19:25.582 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.586+0000 7f26392ff640 0 -- 10.20.193.46:0/2048819072 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x5560cfff3350 msgr2=0x5560d0096880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:25.582 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.586+0000 7f263b02aa80 20 add_watcher() i=1 2026-02-24T22:19:25.583 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.586+0000 7f263b02aa80 20 add_watcher() i=6 2026-02-24T22:19:25.583 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.586+0000 7f263b02aa80 20 add_watcher() i=5 2026-02-24T22:19:25.583 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.586+0000 7f263b02aa80 20 add_watcher() i=0 2026-02-24T22:19:25.584 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.586+0000 7f263b02aa80 20 add_watcher() i=3 2026-02-24T22:19:25.584 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.586+0000 7f263b02aa80 20 add_watcher() i=7 2026-02-24T22:19:25.584 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.586+0000 7f263b02aa80 20 add_watcher() i=2 2026-02-24T22:19:25.584 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.586+0000 7f263b02aa80 20 add_watcher() i=4 2026-02-24T22:19:25.584 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.586+0000 7f263b02aa80 2 all 8 watchers are set, enabling cache 2026-02-24T22:19:25.585 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.590+0000 7f263b02aa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:19:25.585 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.590+0000 7f263b02aa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:19:25.585 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.590+0000 7f263b02aa80 5 note: GC not initialized 2026-02-24T22:19:25.585 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.590+0000 7f25e5fe3640 20 reqs_thread_entry: start 2026-02-24T22:19:25.586 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.590+0000 7f26392ff640 0 -- 10.20.193.46:0/240894779 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5560d010d050 msgr2=0x5560d0108770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:25.827 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.830+0000 7f263b02aa80 20 init_complete bucket index max shards: 1 2026-02-24T22:19:25.827 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.830+0000 7f263b02aa80 20 Filter name: none 2026-02-24T22:19:25.827 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.830+0000 7f253ffff640 20 reqs_thread_entry: start 2026-02-24T22:19:25.827 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.830+0000 7f263b02aa80 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2026-02-24T22:19:25.827 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.830+0000 7f263b02aa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:25.827 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.830+0000 7f263b02aa80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:19:25.827 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.830+0000 7f263b02aa80 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2026-02-24T22:19:25.827 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.830+0000 7f263b02aa80 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2026-02-24T22:19:25.827 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.830+0000 7f263b02aa80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 : miss 2026-02-24T22:19:25.828 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.830+0000 7f263b02aa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:25.828 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.830+0000 7f263b02aa80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:19:25.828 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.830+0000 7f263b02aa80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 info.flags=0x17 2026-02-24T22:19:25.828 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.830+0000 7f263b02aa80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 to cache LRU end 2026-02-24T22:19:25.828 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.830+0000 7f263b02aa80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:19:25.828 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.830+0000 7f263b02aa80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:25.828 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.830+0000 7f263b02aa80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 2026-02-24T22:19:25.831 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.834+0000 7f263b02aa80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 info.flags=0x17 2026-02-24T22:19:25.831 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.834+0000 7f263b02aa80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 to cache LRU end 2026-02-24T22:19:25.831 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.834+0000 7f263b02aa80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:25.831 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.834+0000 7f263b02aa80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535, ofs0, ns] 2026-02-24T22:19:25.832 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.834+0000 7f2607fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959903 cookie 93874295768320 notifier 7292 bl.length()=1013 2026-02-24T22:19:25.832 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.834+0000 7f2607fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 info.flags=0x17 2026-02-24T22:19:25.832 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.834+0000 7f2607fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 to cache LRU end 2026-02-24T22:19:25.832 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.834+0000 7f2607fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:25.833 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.834+0000 7f263b02aa80 20 handle_bi_update(): bucket=:abort-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535]): orig_sources= new_sources= 2026-02-24T22:19:25.833 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.834+0000 7f263b02aa80 20 handle_bi_update(): bucket=:abort-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535]): potential sources added=[] removed=[] 2026-02-24T22:19:25.833 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.834+0000 7f263b02aa80 20 handle_bi_update(): bucket=:abort-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535]): orig_dests= new_dests= 2026-02-24T22:19:25.833 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.834+0000 7f263b02aa80 20 handle_bi_update(): bucket=:abort-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535]): potential dests added=[] removed=[] 2026-02-24T22:19:25.833 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f263b02aa80 time 2026-02-24T22:19:25.840474+0000 2026-02-24T22:19:25.833 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:19:25.834 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:19:25.834 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f263e46ceb1] 2026-02-24T22:19:25.834 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x5560cbbfdb9f] 2026-02-24T22:19:25.834 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x16a) [0x5560cc36926a] 2026-02-24T22:19:25.835 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:19:25.835 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f263d76cd90] 2026-02-24T22:19:25.835 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:19:25.835 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:19:25.835 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.838+0000 7f263b02aa80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f263b02aa80 time 2026-02-24T22:19:25.840474+0000 2026-02-24T22:19:25.835 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:19:25.835 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.835 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:19:25.835 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f263e46ceb1] 2026-02-24T22:19:25.835 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x5560cbbfdb9f] 2026-02-24T22:19:25.836 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x16a) [0x5560cc36926a] 2026-02-24T22:19:25.836 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:19:25.836 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f263d76cd90] 2026-02-24T22:19:25.836 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:19:25.836 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:19:25.836 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.836 INFO:tasks.workunit.client.0.trial046.stderr:*** Caught signal (Aborted) ** 2026-02-24T22:19:25.836 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f263b02aa80 thread_name:radosgw-admin 2026-02-24T22:19:25.861 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:19:25.861 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f263d785520] 2026-02-24T22:19:25.861 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:19:25.861 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:19:25.862 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:19:25.862 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f263e46cf10] 2026-02-24T22:19:25.862 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x5560cbbfdb9f] 2026-02-24T22:19:25.862 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x16a) [0x5560cc36926a] 2026-02-24T22:19:25.862 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:19:25.862 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f263d76cd90] 2026-02-24T22:19:25.862 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:19:25.862 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:19:25.862 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.838+0000 7f263b02aa80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:19:25.862 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f263b02aa80 thread_name:radosgw-admin 2026-02-24T22:19:25.862 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.863 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:19:25.863 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f263d785520] 2026-02-24T22:19:25.863 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:19:25.863 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:19:25.863 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:19:25.863 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f263e46cf10] 2026-02-24T22:19:25.863 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x5560cbbfdb9f] 2026-02-24T22:19:25.863 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x16a) [0x5560cc36926a] 2026-02-24T22:19:25.863 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:19:25.863 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f263d76cd90] 2026-02-24T22:19:25.863 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:19:25.863 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:19:25.864 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:19:25.864 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.864 INFO:tasks.workunit.client.0.trial046.stderr:--- begin dump of recent events --- 2026-02-24T22:19:25.864 INFO:tasks.workunit.client.0.trial046.stderr: -499> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T22:19:25.864 INFO:tasks.workunit.client.0.trial046.stderr: -498> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-a con 0x5560cfc1bfe0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:19:25.864 INFO:tasks.workunit.client.0.trial046.stderr: -497> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-b con 0x5560cfcc4b00 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:19:25.864 INFO:tasks.workunit.client.0.trial046.stderr: -496> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-c con 0x5560cfcc7cf0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:19:25.864 INFO:tasks.workunit.client.0.trial046.stderr: -495> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.864 INFO:tasks.workunit.client.0.trial046.stderr: -494> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.864 INFO:tasks.workunit.client.0.trial046.stderr: -493> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.864 INFO:tasks.workunit.client.0.trial046.stderr: -492> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): _renew_subs 2026-02-24T22:19:25.864 INFO:tasks.workunit.client.0.trial046.stderr: -491> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): authenticate will time out at 1935.678383s 2026-02-24T22:19:25.864 INFO:tasks.workunit.client.0.trial046.stderr: -490> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): get_auth_request con 0x5560cfc1bfe0 auth_method 0 2026-02-24T22:19:25.864 INFO:tasks.workunit.client.0.trial046.stderr: -489> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:25.865 INFO:tasks.workunit.client.0.trial046.stderr: -488> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:25.865 INFO:tasks.workunit.client.0.trial046.stderr: -487> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:25.865 INFO:tasks.workunit.client.0.trial046.stderr: -486> 2026-02-24T22:19:25.570+0000 7f26392ff640 10 monclient(hunting): get_auth_request con 0x5560cfcc7cf0 auth_method 0 2026-02-24T22:19:25.865 INFO:tasks.workunit.client.0.trial046.stderr: -485> 2026-02-24T22:19:25.570+0000 7f26392ff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:25.865 INFO:tasks.workunit.client.0.trial046.stderr: -484> 2026-02-24T22:19:25.570+0000 7f26392ff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:25.865 INFO:tasks.workunit.client.0.trial046.stderr: -483> 2026-02-24T22:19:25.570+0000 7f26392ff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:25.865 INFO:tasks.workunit.client.0.trial046.stderr: -482> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:25.865 INFO:tasks.workunit.client.0.trial046.stderr: -481> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:25.865 INFO:tasks.workunit.client.0.trial046.stderr: -480> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:25.865 INFO:tasks.workunit.client.0.trial046.stderr: -479> 2026-02-24T22:19:25.570+0000 7f2633fff640 10 monclient(hunting): get_auth_request con 0x5560cfcc4b00 auth_method 0 2026-02-24T22:19:25.865 INFO:tasks.workunit.client.0.trial046.stderr: -478> 2026-02-24T22:19:25.570+0000 7f2633fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:25.865 INFO:tasks.workunit.client.0.trial046.stderr: -477> 2026-02-24T22:19:25.570+0000 7f2633fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:25.865 INFO:tasks.workunit.client.0.trial046.stderr: -476> 2026-02-24T22:19:25.570+0000 7f2633fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:25.865 INFO:tasks.workunit.client.0.trial046.stderr: -475> 2026-02-24T22:19:25.570+0000 7f26392ff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:25.865 INFO:tasks.workunit.client.0.trial046.stderr: -474> 2026-02-24T22:19:25.570+0000 7f26392ff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:25.866 INFO:tasks.workunit.client.0.trial046.stderr: -473> 2026-02-24T22:19:25.570+0000 7f26392ff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:25.866 INFO:tasks.workunit.client.0.trial046.stderr: -472> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): handle_auth_done global_id 7785 payload 274 2026-02-24T22:19:25.866 INFO:tasks.workunit.client.0.trial046.stderr: -471> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient: _finish_hunting 0 2026-02-24T22:19:25.866 INFO:tasks.workunit.client.0.trial046.stderr: -470> 2026-02-24T22:19:25.570+0000 7f2638afe640 1 monclient: found mon.noname-a 2026-02-24T22:19:25.866 INFO:tasks.workunit.client.0.trial046.stderr: -469> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:19:25.866 INFO:tasks.workunit.client.0.trial046.stderr: -468> 2026-02-24T22:19:25.570+0000 7f261effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:25.866 INFO:tasks.workunit.client.0.trial046.stderr: -467> 2026-02-24T22:19:25.570+0000 7f261effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:19:25.866 INFO:tasks.workunit.client.0.trial046.stderr: -466> 2026-02-24T22:19:25.570+0000 7f261effd640 10 monclient: dump: 2026-02-24T22:19:25.866 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:25.866 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:25.866 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.866 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.866 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:25.866 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.867 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:25.867 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:25.867 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:25.867 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.867 INFO:tasks.workunit.client.0.trial046.stderr: -465> 2026-02-24T22:19:25.570+0000 7f261effd640 10 monclient: _finish_auth 0 2026-02-24T22:19:25.867 INFO:tasks.workunit.client.0.trial046.stderr: -464> 2026-02-24T22:19:25.570+0000 7f261effd640 10 monclient: _check_auth_tickets 2026-02-24T22:19:25.867 INFO:tasks.workunit.client.0.trial046.stderr: -463> 2026-02-24T22:19:25.570+0000 7f261effd640 10 monclient: handle_config config(0 keys) 2026-02-24T22:19:25.867 INFO:tasks.workunit.client.0.trial046.stderr: -462> 2026-02-24T22:19:25.570+0000 7f261effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:25.867 INFO:tasks.workunit.client.0.trial046.stderr: -461> 2026-02-24T22:19:25.570+0000 7f261effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:19:25.867 INFO:tasks.workunit.client.0.trial046.stderr: -460> 2026-02-24T22:19:25.570+0000 7f261effd640 10 monclient: dump: 2026-02-24T22:19:25.867 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:25.867 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:25.867 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.867 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.867 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:25.868 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.868 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:25.868 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:25.868 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:25.868 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.868 INFO:tasks.workunit.client.0.trial046.stderr: -459> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 monclient: authenticate success, global_id 7785 2026-02-24T22:19:25.868 INFO:tasks.workunit.client.0.trial046.stderr: -458> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient: get_monmap_and_config success 2026-02-24T22:19:25.868 INFO:tasks.workunit.client.0.trial046.stderr: -457> 2026-02-24T22:19:25.570+0000 7f263b02aa80 4 set_mon_vals no callback set 2026-02-24T22:19:25.868 INFO:tasks.workunit.client.0.trial046.stderr: -456> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient: shutdown 2026-02-24T22:19:25.868 INFO:tasks.workunit.client.0.trial046.stderr: -455> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 asok(0x5560cfb8df00) unregister_commands rotate-key 2026-02-24T22:19:25.868 INFO:tasks.workunit.client.0.trial046.stderr: -454> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient: build_initial_monmap 2026-02-24T22:19:25.868 INFO:tasks.workunit.client.0.trial046.stderr: -453> 2026-02-24T22:19:25.570+0000 7f263b02aa80 1 build_initial for_mkfs: 0 2026-02-24T22:19:25.868 INFO:tasks.workunit.client.0.trial046.stderr: -452> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient: monmap: 2026-02-24T22:19:25.868 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:19:25.868 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:19:25.868 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:19:25.869 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:19:25.869 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:19:25.869 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.869 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:19:25.869 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:19:25.869 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:19:25.869 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.869 INFO:tasks.workunit.client.0.trial046.stderr: -451> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding auth protocol: cephx 2026-02-24T22:19:25.869 INFO:tasks.workunit.client.0.trial046.stderr: -450> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding auth protocol: cephx 2026-02-24T22:19:25.869 INFO:tasks.workunit.client.0.trial046.stderr: -449> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding auth protocol: cephx 2026-02-24T22:19:25.869 INFO:tasks.workunit.client.0.trial046.stderr: -448> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding con mode: secure 2026-02-24T22:19:25.869 INFO:tasks.workunit.client.0.trial046.stderr: -447> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding con mode: crc 2026-02-24T22:19:25.869 INFO:tasks.workunit.client.0.trial046.stderr: -446> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding con mode: secure 2026-02-24T22:19:25.869 INFO:tasks.workunit.client.0.trial046.stderr: -445> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding con mode: crc 2026-02-24T22:19:25.870 INFO:tasks.workunit.client.0.trial046.stderr: -444> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding con mode: secure 2026-02-24T22:19:25.870 INFO:tasks.workunit.client.0.trial046.stderr: -443> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding con mode: crc 2026-02-24T22:19:25.870 INFO:tasks.workunit.client.0.trial046.stderr: -442> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding con mode: crc 2026-02-24T22:19:25.870 INFO:tasks.workunit.client.0.trial046.stderr: -441> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding con mode: secure 2026-02-24T22:19:25.870 INFO:tasks.workunit.client.0.trial046.stderr: -440> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding con mode: crc 2026-02-24T22:19:25.870 INFO:tasks.workunit.client.0.trial046.stderr: -439> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding con mode: secure 2026-02-24T22:19:25.870 INFO:tasks.workunit.client.0.trial046.stderr: -438> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding con mode: crc 2026-02-24T22:19:25.870 INFO:tasks.workunit.client.0.trial046.stderr: -437> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding con mode: secure 2026-02-24T22:19:25.870 INFO:tasks.workunit.client.0.trial046.stderr: -436> 2026-02-24T22:19:25.570+0000 7f263b02aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:25.870 INFO:tasks.workunit.client.0.trial046.stderr: -435> 2026-02-24T22:19:25.570+0000 7f263b02aa80 1 librados: starting msgr at 2026-02-24T22:19:25.870 INFO:tasks.workunit.client.0.trial046.stderr: -434> 2026-02-24T22:19:25.570+0000 7f263b02aa80 1 librados: starting objecter 2026-02-24T22:19:25.870 INFO:tasks.workunit.client.0.trial046.stderr: -433> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command objecter_requests hook 0x5560cfc93140 2026-02-24T22:19:25.870 INFO:tasks.workunit.client.0.trial046.stderr: -432> 2026-02-24T22:19:25.570+0000 7f263b02aa80 1 librados: setting wanted keys 2026-02-24T22:19:25.870 INFO:tasks.workunit.client.0.trial046.stderr: -431> 2026-02-24T22:19:25.570+0000 7f263b02aa80 1 librados: calling monclient init 2026-02-24T22:19:25.870 INFO:tasks.workunit.client.0.trial046.stderr: -430> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient: init 2026-02-24T22:19:25.871 INFO:tasks.workunit.client.0.trial046.stderr: -429> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding auth protocol: cephx 2026-02-24T22:19:25.871 INFO:tasks.workunit.client.0.trial046.stderr: -428> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding auth protocol: cephx 2026-02-24T22:19:25.871 INFO:tasks.workunit.client.0.trial046.stderr: -427> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding auth protocol: cephx 2026-02-24T22:19:25.871 INFO:tasks.workunit.client.0.trial046.stderr: -426> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding con mode: secure 2026-02-24T22:19:25.871 INFO:tasks.workunit.client.0.trial046.stderr: -425> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding con mode: crc 2026-02-24T22:19:25.871 INFO:tasks.workunit.client.0.trial046.stderr: -424> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding con mode: secure 2026-02-24T22:19:25.871 INFO:tasks.workunit.client.0.trial046.stderr: -423> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding con mode: crc 2026-02-24T22:19:25.871 INFO:tasks.workunit.client.0.trial046.stderr: -422> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding con mode: secure 2026-02-24T22:19:25.871 INFO:tasks.workunit.client.0.trial046.stderr: -421> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding con mode: crc 2026-02-24T22:19:25.871 INFO:tasks.workunit.client.0.trial046.stderr: -420> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding con mode: crc 2026-02-24T22:19:25.871 INFO:tasks.workunit.client.0.trial046.stderr: -419> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding con mode: secure 2026-02-24T22:19:25.871 INFO:tasks.workunit.client.0.trial046.stderr: -418> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding con mode: crc 2026-02-24T22:19:25.871 INFO:tasks.workunit.client.0.trial046.stderr: -417> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding con mode: secure 2026-02-24T22:19:25.871 INFO:tasks.workunit.client.0.trial046.stderr: -416> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding con mode: crc 2026-02-24T22:19:25.871 INFO:tasks.workunit.client.0.trial046.stderr: -415> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding con mode: secure 2026-02-24T22:19:25.871 INFO:tasks.workunit.client.0.trial046.stderr: -414> 2026-02-24T22:19:25.570+0000 7f263b02aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:25.872 INFO:tasks.workunit.client.0.trial046.stderr: -413> 2026-02-24T22:19:25.570+0000 7f263b02aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:25.872 INFO:tasks.workunit.client.0.trial046.stderr: -412> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command rotate-key hook 0x5560cfc8d908 2026-02-24T22:19:25.872 INFO:tasks.workunit.client.0.trial046.stderr: -411> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient: _reopen_session rank -1 2026-02-24T22:19:25.872 INFO:tasks.workunit.client.0.trial046.stderr: -410> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:19:25.872 INFO:tasks.workunit.client.0.trial046.stderr: -409> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-c con 0x5560cfc1bfe0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:19:25.872 INFO:tasks.workunit.client.0.trial046.stderr: -408> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-a con 0x5560cfcc4b00 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:19:25.872 INFO:tasks.workunit.client.0.trial046.stderr: -407> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-b con 0x5560cfcc7cf0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:19:25.872 INFO:tasks.workunit.client.0.trial046.stderr: -406> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.872 INFO:tasks.workunit.client.0.trial046.stderr: -405> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.872 INFO:tasks.workunit.client.0.trial046.stderr: -404> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.872 INFO:tasks.workunit.client.0.trial046.stderr: -403> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): _renew_subs 2026-02-24T22:19:25.872 INFO:tasks.workunit.client.0.trial046.stderr: -402> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): authenticate will time out at 1935.679750s 2026-02-24T22:19:25.872 INFO:tasks.workunit.client.0.trial046.stderr: -401> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): get_auth_request con 0x5560cfc1bfe0 auth_method 0 2026-02-24T22:19:25.872 INFO:tasks.workunit.client.0.trial046.stderr: -400> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:25.872 INFO:tasks.workunit.client.0.trial046.stderr: -399> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:25.872 INFO:tasks.workunit.client.0.trial046.stderr: -398> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:25.873 INFO:tasks.workunit.client.0.trial046.stderr: -397> 2026-02-24T22:19:25.570+0000 7f2633fff640 10 monclient(hunting): get_auth_request con 0x5560cfcc4b00 auth_method 0 2026-02-24T22:19:25.873 INFO:tasks.workunit.client.0.trial046.stderr: -396> 2026-02-24T22:19:25.570+0000 7f2633fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:25.873 INFO:tasks.workunit.client.0.trial046.stderr: -395> 2026-02-24T22:19:25.570+0000 7f2633fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:25.873 INFO:tasks.workunit.client.0.trial046.stderr: -394> 2026-02-24T22:19:25.570+0000 7f2633fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:25.873 INFO:tasks.workunit.client.0.trial046.stderr: -393> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:25.873 INFO:tasks.workunit.client.0.trial046.stderr: -392> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:25.873 INFO:tasks.workunit.client.0.trial046.stderr: -391> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:25.873 INFO:tasks.workunit.client.0.trial046.stderr: -390> 2026-02-24T22:19:25.570+0000 7f2633fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:25.873 INFO:tasks.workunit.client.0.trial046.stderr: -389> 2026-02-24T22:19:25.570+0000 7f2633fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:25.873 INFO:tasks.workunit.client.0.trial046.stderr: -388> 2026-02-24T22:19:25.570+0000 7f2633fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:25.873 INFO:tasks.workunit.client.0.trial046.stderr: -387> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): handle_auth_done global_id 7283 payload 824 2026-02-24T22:19:25.873 INFO:tasks.workunit.client.0.trial046.stderr: -386> 2026-02-24T22:19:25.574+0000 7f2638afe640 10 monclient: _finish_hunting 0 2026-02-24T22:19:25.873 INFO:tasks.workunit.client.0.trial046.stderr: -385> 2026-02-24T22:19:25.574+0000 7f2638afe640 1 monclient: found mon.noname-c 2026-02-24T22:19:25.873 INFO:tasks.workunit.client.0.trial046.stderr: -384> 2026-02-24T22:19:25.574+0000 7f2638afe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:19:25.873 INFO:tasks.workunit.client.0.trial046.stderr: -383> 2026-02-24T22:19:25.574+0000 7f26392ff640 10 monclient: get_auth_request con 0x5560cfcc7cf0 auth_method 0 2026-02-24T22:19:25.874 INFO:tasks.workunit.client.0.trial046.stderr: -382> 2026-02-24T22:19:25.574+0000 7f261d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:25.874 INFO:tasks.workunit.client.0.trial046.stderr: -381> 2026-02-24T22:19:25.574+0000 7f261d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:19:25.874 INFO:tasks.workunit.client.0.trial046.stderr: -380> 2026-02-24T22:19:25.574+0000 7f261d7fa640 10 monclient: dump: 2026-02-24T22:19:25.874 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:25.874 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:25.874 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.874 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.874 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:25.874 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.874 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:25.874 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:25.874 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:25.874 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.874 INFO:tasks.workunit.client.0.trial046.stderr: -379> 2026-02-24T22:19:25.574+0000 7f261d7fa640 10 monclient: _finish_auth 0 2026-02-24T22:19:25.874 INFO:tasks.workunit.client.0.trial046.stderr: -378> 2026-02-24T22:19:25.574+0000 7f261d7fa640 10 monclient: _check_auth_tickets 2026-02-24T22:19:25.874 INFO:tasks.workunit.client.0.trial046.stderr: -377> 2026-02-24T22:19:25.574+0000 7f261d7fa640 10 monclient: handle_config config(0 keys) 2026-02-24T22:19:25.875 INFO:tasks.workunit.client.0.trial046.stderr: -376> 2026-02-24T22:19:25.574+0000 7f261d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:25.875 INFO:tasks.workunit.client.0.trial046.stderr: -375> 2026-02-24T22:19:25.574+0000 7f261d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:19:25.875 INFO:tasks.workunit.client.0.trial046.stderr: -374> 2026-02-24T22:19:25.574+0000 7f261d7fa640 10 monclient: dump: 2026-02-24T22:19:25.875 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:25.875 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:25.875 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.875 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.875 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:25.875 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.875 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:25.875 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:25.875 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:25.875 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.875 INFO:tasks.workunit.client.0.trial046.stderr: -373> 2026-02-24T22:19:25.574+0000 7f263b02aa80 5 monclient: authenticate success, global_id 7283 2026-02-24T22:19:25.875 INFO:tasks.workunit.client.0.trial046.stderr: -372> 2026-02-24T22:19:25.574+0000 7f263b02aa80 10 monclient: _renew_subs 2026-02-24T22:19:25.875 INFO:tasks.workunit.client.0.trial046.stderr: -371> 2026-02-24T22:19:25.574+0000 7f263b02aa80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:19:25.876 INFO:tasks.workunit.client.0.trial046.stderr: -370> 2026-02-24T22:19:25.574+0000 7f261effd640 4 set_mon_vals no callback set 2026-02-24T22:19:25.876 INFO:tasks.workunit.client.0.trial046.stderr: -369> 2026-02-24T22:19:25.574+0000 7f263b02aa80 10 monclient: _renew_subs 2026-02-24T22:19:25.876 INFO:tasks.workunit.client.0.trial046.stderr: -368> 2026-02-24T22:19:25.574+0000 7f263b02aa80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:19:25.876 INFO:tasks.workunit.client.0.trial046.stderr: -367> 2026-02-24T22:19:25.574+0000 7f263b02aa80 1 librados: init done 2026-02-24T22:19:25.876 INFO:tasks.workunit.client.0.trial046.stderr: -366> 2026-02-24T22:19:25.574+0000 7f261d7fa640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:19:25.876 INFO:tasks.workunit.client.0.trial046.stderr: -365> 2026-02-24T22:19:25.574+0000 7f261d7fa640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:19:25.876 INFO:tasks.workunit.client.0.trial046.stderr: -364> 2026-02-24T22:19:25.574+0000 7f261d7fa640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:19:25.876 INFO:tasks.workunit.client.0.trial046.stderr: -363> 2026-02-24T22:19:25.574+0000 7f2633fff640 10 monclient: get_auth_request con 0x7f2618070730 auth_method 0 2026-02-24T22:19:25.876 INFO:tasks.workunit.client.0.trial046.stderr: -362> 2026-02-24T22:19:25.574+0000 7f26392ff640 10 monclient: get_auth_request con 0x5560cfcef720 auth_method 0 2026-02-24T22:19:25.876 INFO:tasks.workunit.client.0.trial046.stderr: -361> 2026-02-24T22:19:25.574+0000 7f2638afe640 10 monclient: get_auth_request con 0x5560cfd1b5c0 auth_method 0 2026-02-24T22:19:25.876 INFO:tasks.workunit.client.0.trial046.stderr: -360> 2026-02-24T22:19:25.574+0000 7f2633fff640 10 monclient: get_auth_request con 0x5560cfd1e310 auth_method 0 2026-02-24T22:19:25.876 INFO:tasks.workunit.client.0.trial046.stderr: -359> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: build_initial_monmap 2026-02-24T22:19:25.876 INFO:tasks.workunit.client.0.trial046.stderr: -358> 2026-02-24T22:19:25.578+0000 7f263b02aa80 1 build_initial for_mkfs: 0 2026-02-24T22:19:25.876 INFO:tasks.workunit.client.0.trial046.stderr: -357> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: monmap: 2026-02-24T22:19:25.876 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:19:25.876 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:19:25.876 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:19:25.877 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:19:25.877 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:19:25.877 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.877 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:19:25.877 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:19:25.877 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:19:25.877 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.877 INFO:tasks.workunit.client.0.trial046.stderr: -356> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding auth protocol: cephx 2026-02-24T22:19:25.877 INFO:tasks.workunit.client.0.trial046.stderr: -355> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding auth protocol: cephx 2026-02-24T22:19:25.877 INFO:tasks.workunit.client.0.trial046.stderr: -354> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding auth protocol: cephx 2026-02-24T22:19:25.877 INFO:tasks.workunit.client.0.trial046.stderr: -353> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding con mode: secure 2026-02-24T22:19:25.877 INFO:tasks.workunit.client.0.trial046.stderr: -352> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding con mode: crc 2026-02-24T22:19:25.877 INFO:tasks.workunit.client.0.trial046.stderr: -351> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding con mode: secure 2026-02-24T22:19:25.877 INFO:tasks.workunit.client.0.trial046.stderr: -350> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding con mode: crc 2026-02-24T22:19:25.877 INFO:tasks.workunit.client.0.trial046.stderr: -349> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding con mode: secure 2026-02-24T22:19:25.877 INFO:tasks.workunit.client.0.trial046.stderr: -348> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding con mode: crc 2026-02-24T22:19:25.877 INFO:tasks.workunit.client.0.trial046.stderr: -347> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding con mode: crc 2026-02-24T22:19:25.878 INFO:tasks.workunit.client.0.trial046.stderr: -346> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding con mode: secure 2026-02-24T22:19:25.878 INFO:tasks.workunit.client.0.trial046.stderr: -345> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding con mode: crc 2026-02-24T22:19:25.878 INFO:tasks.workunit.client.0.trial046.stderr: -344> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding con mode: secure 2026-02-24T22:19:25.878 INFO:tasks.workunit.client.0.trial046.stderr: -343> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding con mode: crc 2026-02-24T22:19:25.878 INFO:tasks.workunit.client.0.trial046.stderr: -342> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding con mode: secure 2026-02-24T22:19:25.878 INFO:tasks.workunit.client.0.trial046.stderr: -341> 2026-02-24T22:19:25.578+0000 7f263b02aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:25.878 INFO:tasks.workunit.client.0.trial046.stderr: -340> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command objecter_requests cmddesc objecter_requests hook 0x5560cfd24d70 EEXIST 2026-02-24T22:19:25.878 INFO:tasks.workunit.client.0.trial046.stderr: -339> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: init 2026-02-24T22:19:25.878 INFO:tasks.workunit.client.0.trial046.stderr: -338> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding auth protocol: cephx 2026-02-24T22:19:25.878 INFO:tasks.workunit.client.0.trial046.stderr: -337> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding auth protocol: cephx 2026-02-24T22:19:25.878 INFO:tasks.workunit.client.0.trial046.stderr: -336> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding auth protocol: cephx 2026-02-24T22:19:25.878 INFO:tasks.workunit.client.0.trial046.stderr: -335> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding con mode: secure 2026-02-24T22:19:25.878 INFO:tasks.workunit.client.0.trial046.stderr: -334> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding con mode: crc 2026-02-24T22:19:25.878 INFO:tasks.workunit.client.0.trial046.stderr: -333> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding con mode: secure 2026-02-24T22:19:25.878 INFO:tasks.workunit.client.0.trial046.stderr: -332> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding con mode: crc 2026-02-24T22:19:25.878 INFO:tasks.workunit.client.0.trial046.stderr: -331> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding con mode: secure 2026-02-24T22:19:25.878 INFO:tasks.workunit.client.0.trial046.stderr: -330> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding con mode: crc 2026-02-24T22:19:25.878 INFO:tasks.workunit.client.0.trial046.stderr: -329> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding con mode: crc 2026-02-24T22:19:25.878 INFO:tasks.workunit.client.0.trial046.stderr: -328> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding con mode: secure 2026-02-24T22:19:25.878 INFO:tasks.workunit.client.0.trial046.stderr: -327> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding con mode: crc 2026-02-24T22:19:25.878 INFO:tasks.workunit.client.0.trial046.stderr: -326> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding con mode: secure 2026-02-24T22:19:25.879 INFO:tasks.workunit.client.0.trial046.stderr: -325> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding con mode: crc 2026-02-24T22:19:25.879 INFO:tasks.workunit.client.0.trial046.stderr: -324> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding con mode: secure 2026-02-24T22:19:25.879 INFO:tasks.workunit.client.0.trial046.stderr: -323> 2026-02-24T22:19:25.578+0000 7f263b02aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:25.879 INFO:tasks.workunit.client.0.trial046.stderr: -322> 2026-02-24T22:19:25.578+0000 7f263b02aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:25.879 INFO:tasks.workunit.client.0.trial046.stderr: -321> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command rotate-key cmddesc rotate-key hook 0x5560cfd1c500 EEXIST 2026-02-24T22:19:25.879 INFO:tasks.workunit.client.0.trial046.stderr: -320> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: _reopen_session rank -1 2026-02-24T22:19:25.879 INFO:tasks.workunit.client.0.trial046.stderr: -319> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:19:25.879 INFO:tasks.workunit.client.0.trial046.stderr: -318> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-c con 0x5560cfe78510 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:19:25.879 INFO:tasks.workunit.client.0.trial046.stderr: -317> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-a con 0x5560cfe711b0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:19:25.879 INFO:tasks.workunit.client.0.trial046.stderr: -316> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-b con 0x5560cfe71620 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:19:25.879 INFO:tasks.workunit.client.0.trial046.stderr: -315> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.879 INFO:tasks.workunit.client.0.trial046.stderr: -314> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.879 INFO:tasks.workunit.client.0.trial046.stderr: -313> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.879 INFO:tasks.workunit.client.0.trial046.stderr: -312> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): _renew_subs 2026-02-24T22:19:25.879 INFO:tasks.workunit.client.0.trial046.stderr: -311> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): authenticate will time out at 1935.685229s 2026-02-24T22:19:25.879 INFO:tasks.workunit.client.0.trial046.stderr: -310> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): get_auth_request con 0x5560cfe78510 auth_method 0 2026-02-24T22:19:25.879 INFO:tasks.workunit.client.0.trial046.stderr: -309> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:25.880 INFO:tasks.workunit.client.0.trial046.stderr: -308> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:25.880 INFO:tasks.workunit.client.0.trial046.stderr: -307> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:25.880 INFO:tasks.workunit.client.0.trial046.stderr: -306> 2026-02-24T22:19:25.578+0000 7f2633fff640 10 monclient(hunting): get_auth_request con 0x5560cfe711b0 auth_method 0 2026-02-24T22:19:25.880 INFO:tasks.workunit.client.0.trial046.stderr: -305> 2026-02-24T22:19:25.578+0000 7f2633fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:25.880 INFO:tasks.workunit.client.0.trial046.stderr: -304> 2026-02-24T22:19:25.578+0000 7f2633fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:25.880 INFO:tasks.workunit.client.0.trial046.stderr: -303> 2026-02-24T22:19:25.578+0000 7f2633fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:25.880 INFO:tasks.workunit.client.0.trial046.stderr: -302> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:25.880 INFO:tasks.workunit.client.0.trial046.stderr: -301> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:25.880 INFO:tasks.workunit.client.0.trial046.stderr: -300> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:25.880 INFO:tasks.workunit.client.0.trial046.stderr: -299> 2026-02-24T22:19:25.578+0000 7f2633fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:25.880 INFO:tasks.workunit.client.0.trial046.stderr: -298> 2026-02-24T22:19:25.578+0000 7f2633fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:25.880 INFO:tasks.workunit.client.0.trial046.stderr: -297> 2026-02-24T22:19:25.578+0000 7f2633fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:25.880 INFO:tasks.workunit.client.0.trial046.stderr: -296> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): handle_auth_done global_id 7286 payload 824 2026-02-24T22:19:25.880 INFO:tasks.workunit.client.0.trial046.stderr: -295> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient: _finish_hunting 0 2026-02-24T22:19:25.880 INFO:tasks.workunit.client.0.trial046.stderr: -294> 2026-02-24T22:19:25.578+0000 7f2638afe640 1 monclient: found mon.noname-c 2026-02-24T22:19:25.880 INFO:tasks.workunit.client.0.trial046.stderr: -293> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:19:25.880 INFO:tasks.workunit.client.0.trial046.stderr: -292> 2026-02-24T22:19:25.578+0000 7f26392ff640 10 monclient: get_auth_request con 0x5560cfe71620 auth_method 0 2026-02-24T22:19:25.880 INFO:tasks.workunit.client.0.trial046.stderr: -291> 2026-02-24T22:19:25.578+0000 7f26167fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:25.880 INFO:tasks.workunit.client.0.trial046.stderr: -290> 2026-02-24T22:19:25.578+0000 7f26167fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:19:25.881 INFO:tasks.workunit.client.0.trial046.stderr: -289> 2026-02-24T22:19:25.578+0000 7f26167fc640 10 monclient: dump: 2026-02-24T22:19:25.881 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:25.881 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:25.881 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.881 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.881 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:25.881 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.881 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:25.881 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:25.881 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:25.881 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.881 INFO:tasks.workunit.client.0.trial046.stderr: -288> 2026-02-24T22:19:25.578+0000 7f26167fc640 10 monclient: _finish_auth 0 2026-02-24T22:19:25.881 INFO:tasks.workunit.client.0.trial046.stderr: -287> 2026-02-24T22:19:25.578+0000 7f26167fc640 10 monclient: _check_auth_tickets 2026-02-24T22:19:25.881 INFO:tasks.workunit.client.0.trial046.stderr: -286> 2026-02-24T22:19:25.578+0000 7f26167fc640 10 monclient: handle_config config(0 keys) 2026-02-24T22:19:25.881 INFO:tasks.workunit.client.0.trial046.stderr: -285> 2026-02-24T22:19:25.578+0000 7f26167fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:25.881 INFO:tasks.workunit.client.0.trial046.stderr: -284> 2026-02-24T22:19:25.578+0000 7f26167fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:19:25.881 INFO:tasks.workunit.client.0.trial046.stderr: -283> 2026-02-24T22:19:25.578+0000 7f26167fc640 10 monclient: dump: 2026-02-24T22:19:25.881 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:25.881 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:25.881 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.882 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.882 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:25.882 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.882 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:25.882 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:25.882 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:25.882 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.882 INFO:tasks.workunit.client.0.trial046.stderr: -282> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 monclient: authenticate success, global_id 7286 2026-02-24T22:19:25.882 INFO:tasks.workunit.client.0.trial046.stderr: -281> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: _renew_subs 2026-02-24T22:19:25.882 INFO:tasks.workunit.client.0.trial046.stderr: -280> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:19:25.882 INFO:tasks.workunit.client.0.trial046.stderr: -279> 2026-02-24T22:19:25.578+0000 7f2639b00640 4 set_mon_vals no callback set 2026-02-24T22:19:25.882 INFO:tasks.workunit.client.0.trial046.stderr: -278> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: _renew_subs 2026-02-24T22:19:25.882 INFO:tasks.workunit.client.0.trial046.stderr: -277> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:19:25.882 INFO:tasks.workunit.client.0.trial046.stderr: -276> 2026-02-24T22:19:25.578+0000 7f26167fc640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:19:25.882 INFO:tasks.workunit.client.0.trial046.stderr: -275> 2026-02-24T22:19:25.578+0000 7f26167fc640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:19:25.882 INFO:tasks.workunit.client.0.trial046.stderr: -274> 2026-02-24T22:19:25.578+0000 7f26167fc640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:19:25.882 INFO:tasks.workunit.client.0.trial046.stderr: -273> 2026-02-24T22:19:25.578+0000 7f2633fff640 10 monclient: get_auth_request con 0x7f26080706c0 auth_method 0 2026-02-24T22:19:25.882 INFO:tasks.workunit.client.0.trial046.stderr: -272> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: get_monmap_and_config 2026-02-24T22:19:25.882 INFO:tasks.workunit.client.0.trial046.stderr: -271> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: build_initial_monmap 2026-02-24T22:19:25.882 INFO:tasks.workunit.client.0.trial046.stderr: -270> 2026-02-24T22:19:25.578+0000 7f263b02aa80 1 build_initial for_mkfs: 0 2026-02-24T22:19:25.883 INFO:tasks.workunit.client.0.trial046.stderr: -269> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: monmap: 2026-02-24T22:19:25.883 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:19:25.883 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:19:25.883 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:19:25.883 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:19:25.883 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:19:25.883 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.883 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:19:25.883 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:19:25.883 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:19:25.883 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.883 INFO:tasks.workunit.client.0.trial046.stderr: -268> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding auth protocol: cephx 2026-02-24T22:19:25.883 INFO:tasks.workunit.client.0.trial046.stderr: -267> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding auth protocol: cephx 2026-02-24T22:19:25.883 INFO:tasks.workunit.client.0.trial046.stderr: -266> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding auth protocol: cephx 2026-02-24T22:19:25.883 INFO:tasks.workunit.client.0.trial046.stderr: -265> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding con mode: secure 2026-02-24T22:19:25.883 INFO:tasks.workunit.client.0.trial046.stderr: -264> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding con mode: crc 2026-02-24T22:19:25.883 INFO:tasks.workunit.client.0.trial046.stderr: -263> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding con mode: secure 2026-02-24T22:19:25.883 INFO:tasks.workunit.client.0.trial046.stderr: -262> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding con mode: crc 2026-02-24T22:19:25.883 INFO:tasks.workunit.client.0.trial046.stderr: -261> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding con mode: secure 2026-02-24T22:19:25.883 INFO:tasks.workunit.client.0.trial046.stderr: -260> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding con mode: crc 2026-02-24T22:19:25.883 INFO:tasks.workunit.client.0.trial046.stderr: -259> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding con mode: crc 2026-02-24T22:19:25.883 INFO:tasks.workunit.client.0.trial046.stderr: -258> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding con mode: secure 2026-02-24T22:19:25.883 INFO:tasks.workunit.client.0.trial046.stderr: -257> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding con mode: crc 2026-02-24T22:19:25.884 INFO:tasks.workunit.client.0.trial046.stderr: -256> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding con mode: secure 2026-02-24T22:19:25.884 INFO:tasks.workunit.client.0.trial046.stderr: -255> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding con mode: crc 2026-02-24T22:19:25.884 INFO:tasks.workunit.client.0.trial046.stderr: -254> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding con mode: secure 2026-02-24T22:19:25.884 INFO:tasks.workunit.client.0.trial046.stderr: -253> 2026-02-24T22:19:25.578+0000 7f263b02aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:25.884 INFO:tasks.workunit.client.0.trial046.stderr: -252> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: init 2026-02-24T22:19:25.884 INFO:tasks.workunit.client.0.trial046.stderr: -251> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding auth protocol: cephx 2026-02-24T22:19:25.884 INFO:tasks.workunit.client.0.trial046.stderr: -250> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding auth protocol: cephx 2026-02-24T22:19:25.884 INFO:tasks.workunit.client.0.trial046.stderr: -249> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding auth protocol: cephx 2026-02-24T22:19:25.884 INFO:tasks.workunit.client.0.trial046.stderr: -248> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding con mode: secure 2026-02-24T22:19:25.884 INFO:tasks.workunit.client.0.trial046.stderr: -247> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding con mode: crc 2026-02-24T22:19:25.884 INFO:tasks.workunit.client.0.trial046.stderr: -246> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding con mode: secure 2026-02-24T22:19:25.884 INFO:tasks.workunit.client.0.trial046.stderr: -245> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding con mode: crc 2026-02-24T22:19:25.884 INFO:tasks.workunit.client.0.trial046.stderr: -244> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding con mode: secure 2026-02-24T22:19:25.884 INFO:tasks.workunit.client.0.trial046.stderr: -243> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding con mode: crc 2026-02-24T22:19:25.884 INFO:tasks.workunit.client.0.trial046.stderr: -242> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding con mode: crc 2026-02-24T22:19:25.884 INFO:tasks.workunit.client.0.trial046.stderr: -241> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding con mode: secure 2026-02-24T22:19:25.884 INFO:tasks.workunit.client.0.trial046.stderr: -240> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding con mode: crc 2026-02-24T22:19:25.884 INFO:tasks.workunit.client.0.trial046.stderr: -239> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding con mode: secure 2026-02-24T22:19:25.884 INFO:tasks.workunit.client.0.trial046.stderr: -238> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding con mode: crc 2026-02-24T22:19:25.884 INFO:tasks.workunit.client.0.trial046.stderr: -237> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding con mode: secure 2026-02-24T22:19:25.884 INFO:tasks.workunit.client.0.trial046.stderr: -236> 2026-02-24T22:19:25.578+0000 7f263b02aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:25.884 INFO:tasks.workunit.client.0.trial046.stderr: -235> 2026-02-24T22:19:25.578+0000 7f263b02aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:25.885 INFO:tasks.workunit.client.0.trial046.stderr: -234> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command rotate-key cmddesc rotate-key hook 0x7ffc55abcd68 EEXIST 2026-02-24T22:19:25.885 INFO:tasks.workunit.client.0.trial046.stderr: -233> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: _reopen_session rank -1 2026-02-24T22:19:25.885 INFO:tasks.workunit.client.0.trial046.stderr: -232> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T22:19:25.885 INFO:tasks.workunit.client.0.trial046.stderr: -231> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-a con 0x5560cff30630 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:19:25.885 INFO:tasks.workunit.client.0.trial046.stderr: -230> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-b con 0x5560cff33da0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:19:25.885 INFO:tasks.workunit.client.0.trial046.stderr: -229> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-c con 0x5560cff0eb60 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:19:25.885 INFO:tasks.workunit.client.0.trial046.stderr: -228> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.885 INFO:tasks.workunit.client.0.trial046.stderr: -227> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.885 INFO:tasks.workunit.client.0.trial046.stderr: -226> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.885 INFO:tasks.workunit.client.0.trial046.stderr: -225> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): _renew_subs 2026-02-24T22:19:25.885 INFO:tasks.workunit.client.0.trial046.stderr: -224> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): authenticate will time out at 1935.687558s 2026-02-24T22:19:25.885 INFO:tasks.workunit.client.0.trial046.stderr: -223> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): get_auth_request con 0x5560cff30630 auth_method 0 2026-02-24T22:19:25.885 INFO:tasks.workunit.client.0.trial046.stderr: -222> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:25.885 INFO:tasks.workunit.client.0.trial046.stderr: -221> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:25.885 INFO:tasks.workunit.client.0.trial046.stderr: -220> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:25.885 INFO:tasks.workunit.client.0.trial046.stderr: -219> 2026-02-24T22:19:25.578+0000 7f26392ff640 10 monclient(hunting): get_auth_request con 0x5560cff0eb60 auth_method 0 2026-02-24T22:19:25.885 INFO:tasks.workunit.client.0.trial046.stderr: -218> 2026-02-24T22:19:25.578+0000 7f26392ff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:25.885 INFO:tasks.workunit.client.0.trial046.stderr: -217> 2026-02-24T22:19:25.578+0000 7f26392ff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:25.885 INFO:tasks.workunit.client.0.trial046.stderr: -216> 2026-02-24T22:19:25.578+0000 7f26392ff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:25.885 INFO:tasks.workunit.client.0.trial046.stderr: -215> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:25.885 INFO:tasks.workunit.client.0.trial046.stderr: -214> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:25.885 INFO:tasks.workunit.client.0.trial046.stderr: -213> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:25.885 INFO:tasks.workunit.client.0.trial046.stderr: -212> 2026-02-24T22:19:25.578+0000 7f26392ff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:25.886 INFO:tasks.workunit.client.0.trial046.stderr: -211> 2026-02-24T22:19:25.578+0000 7f26392ff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:25.886 INFO:tasks.workunit.client.0.trial046.stderr: -210> 2026-02-24T22:19:25.578+0000 7f26392ff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:25.886 INFO:tasks.workunit.client.0.trial046.stderr: -209> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): handle_auth_done global_id 7794 payload 274 2026-02-24T22:19:25.886 INFO:tasks.workunit.client.0.trial046.stderr: -208> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient: _finish_hunting 0 2026-02-24T22:19:25.886 INFO:tasks.workunit.client.0.trial046.stderr: -207> 2026-02-24T22:19:25.578+0000 7f2638afe640 1 monclient: found mon.noname-a 2026-02-24T22:19:25.886 INFO:tasks.workunit.client.0.trial046.stderr: -206> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:19:25.886 INFO:tasks.workunit.client.0.trial046.stderr: -205> 2026-02-24T22:19:25.578+0000 7f2607fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:25.886 INFO:tasks.workunit.client.0.trial046.stderr: -204> 2026-02-24T22:19:25.578+0000 7f2607fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:19:25.886 INFO:tasks.workunit.client.0.trial046.stderr: -203> 2026-02-24T22:19:25.578+0000 7f2607fff640 10 monclient: dump: 2026-02-24T22:19:25.886 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:25.886 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:25.886 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.886 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.886 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:25.886 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.886 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:25.886 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:25.886 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:25.886 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.886 INFO:tasks.workunit.client.0.trial046.stderr: -202> 2026-02-24T22:19:25.578+0000 7f2607fff640 10 monclient: _finish_auth 0 2026-02-24T22:19:25.886 INFO:tasks.workunit.client.0.trial046.stderr: -201> 2026-02-24T22:19:25.578+0000 7f2607fff640 10 monclient: _check_auth_tickets 2026-02-24T22:19:25.886 INFO:tasks.workunit.client.0.trial046.stderr: -200> 2026-02-24T22:19:25.578+0000 7f2607fff640 10 monclient: handle_config config(0 keys) 2026-02-24T22:19:25.886 INFO:tasks.workunit.client.0.trial046.stderr: -199> 2026-02-24T22:19:25.578+0000 7f2607fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:25.886 INFO:tasks.workunit.client.0.trial046.stderr: -198> 2026-02-24T22:19:25.578+0000 7f2607fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:19:25.887 INFO:tasks.workunit.client.0.trial046.stderr: -197> 2026-02-24T22:19:25.578+0000 7f2607fff640 10 monclient: dump: 2026-02-24T22:19:25.887 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:25.887 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:25.887 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.887 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.887 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:25.887 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.887 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:25.887 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:25.887 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:25.887 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.887 INFO:tasks.workunit.client.0.trial046.stderr: -196> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 monclient: authenticate success, global_id 7794 2026-02-24T22:19:25.887 INFO:tasks.workunit.client.0.trial046.stderr: -195> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: get_monmap_and_config success 2026-02-24T22:19:25.887 INFO:tasks.workunit.client.0.trial046.stderr: -194> 2026-02-24T22:19:25.578+0000 7f263b02aa80 4 set_mon_vals no callback set 2026-02-24T22:19:25.887 INFO:tasks.workunit.client.0.trial046.stderr: -193> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient: shutdown 2026-02-24T22:19:25.887 INFO:tasks.workunit.client.0.trial046.stderr: -192> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient: build_initial_monmap 2026-02-24T22:19:25.887 INFO:tasks.workunit.client.0.trial046.stderr: -191> 2026-02-24T22:19:25.582+0000 7f263b02aa80 1 build_initial for_mkfs: 0 2026-02-24T22:19:25.887 INFO:tasks.workunit.client.0.trial046.stderr: -190> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient: monmap: 2026-02-24T22:19:25.887 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:19:25.887 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:19:25.888 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:19:25.888 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:19:25.888 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:19:25.888 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.888 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:19:25.888 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:19:25.888 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:19:25.888 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.888 INFO:tasks.workunit.client.0.trial046.stderr: -189> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding auth protocol: cephx 2026-02-24T22:19:25.888 INFO:tasks.workunit.client.0.trial046.stderr: -188> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding auth protocol: cephx 2026-02-24T22:19:25.888 INFO:tasks.workunit.client.0.trial046.stderr: -187> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding auth protocol: cephx 2026-02-24T22:19:25.888 INFO:tasks.workunit.client.0.trial046.stderr: -186> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding con mode: secure 2026-02-24T22:19:25.888 INFO:tasks.workunit.client.0.trial046.stderr: -185> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding con mode: crc 2026-02-24T22:19:25.888 INFO:tasks.workunit.client.0.trial046.stderr: -184> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding con mode: secure 2026-02-24T22:19:25.888 INFO:tasks.workunit.client.0.trial046.stderr: -183> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding con mode: crc 2026-02-24T22:19:25.888 INFO:tasks.workunit.client.0.trial046.stderr: -182> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding con mode: secure 2026-02-24T22:19:25.888 INFO:tasks.workunit.client.0.trial046.stderr: -181> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding con mode: crc 2026-02-24T22:19:25.888 INFO:tasks.workunit.client.0.trial046.stderr: -180> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding con mode: crc 2026-02-24T22:19:25.888 INFO:tasks.workunit.client.0.trial046.stderr: -179> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding con mode: secure 2026-02-24T22:19:25.889 INFO:tasks.workunit.client.0.trial046.stderr: -178> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding con mode: crc 2026-02-24T22:19:25.889 INFO:tasks.workunit.client.0.trial046.stderr: -177> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding con mode: secure 2026-02-24T22:19:25.889 INFO:tasks.workunit.client.0.trial046.stderr: -176> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding con mode: crc 2026-02-24T22:19:25.889 INFO:tasks.workunit.client.0.trial046.stderr: -175> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding con mode: secure 2026-02-24T22:19:25.889 INFO:tasks.workunit.client.0.trial046.stderr: -174> 2026-02-24T22:19:25.582+0000 7f263b02aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:25.889 INFO:tasks.workunit.client.0.trial046.stderr: -173> 2026-02-24T22:19:25.582+0000 7f263b02aa80 1 librados: starting msgr at 2026-02-24T22:19:25.889 INFO:tasks.workunit.client.0.trial046.stderr: -172> 2026-02-24T22:19:25.582+0000 7f263b02aa80 1 librados: starting objecter 2026-02-24T22:19:25.889 INFO:tasks.workunit.client.0.trial046.stderr: -171> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command objecter_requests cmddesc objecter_requests hook 0x5560cff379c0 EEXIST 2026-02-24T22:19:25.889 INFO:tasks.workunit.client.0.trial046.stderr: -170> 2026-02-24T22:19:25.582+0000 7f263b02aa80 1 librados: setting wanted keys 2026-02-24T22:19:25.889 INFO:tasks.workunit.client.0.trial046.stderr: -169> 2026-02-24T22:19:25.582+0000 7f263b02aa80 1 librados: calling monclient init 2026-02-24T22:19:25.889 INFO:tasks.workunit.client.0.trial046.stderr: -168> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient: init 2026-02-24T22:19:25.889 INFO:tasks.workunit.client.0.trial046.stderr: -167> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding auth protocol: cephx 2026-02-24T22:19:25.890 INFO:tasks.workunit.client.0.trial046.stderr: -166> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding auth protocol: cephx 2026-02-24T22:19:25.890 INFO:tasks.workunit.client.0.trial046.stderr: -165> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding auth protocol: cephx 2026-02-24T22:19:25.890 INFO:tasks.workunit.client.0.trial046.stderr: -164> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding con mode: secure 2026-02-24T22:19:25.890 INFO:tasks.workunit.client.0.trial046.stderr: -163> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding con mode: crc 2026-02-24T22:19:25.890 INFO:tasks.workunit.client.0.trial046.stderr: -162> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding con mode: secure 2026-02-24T22:19:25.890 INFO:tasks.workunit.client.0.trial046.stderr: -161> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding con mode: crc 2026-02-24T22:19:25.890 INFO:tasks.workunit.client.0.trial046.stderr: -160> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding con mode: secure 2026-02-24T22:19:25.890 INFO:tasks.workunit.client.0.trial046.stderr: -159> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding con mode: crc 2026-02-24T22:19:25.890 INFO:tasks.workunit.client.0.trial046.stderr: -158> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding con mode: crc 2026-02-24T22:19:25.890 INFO:tasks.workunit.client.0.trial046.stderr: -157> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding con mode: secure 2026-02-24T22:19:25.890 INFO:tasks.workunit.client.0.trial046.stderr: -156> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding con mode: crc 2026-02-24T22:19:25.890 INFO:tasks.workunit.client.0.trial046.stderr: -155> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding con mode: secure 2026-02-24T22:19:25.890 INFO:tasks.workunit.client.0.trial046.stderr: -154> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding con mode: crc 2026-02-24T22:19:25.891 INFO:tasks.workunit.client.0.trial046.stderr: -153> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding con mode: secure 2026-02-24T22:19:25.891 INFO:tasks.workunit.client.0.trial046.stderr: -152> 2026-02-24T22:19:25.582+0000 7f263b02aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:25.891 INFO:tasks.workunit.client.0.trial046.stderr: -151> 2026-02-24T22:19:25.582+0000 7f263b02aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:25.891 INFO:tasks.workunit.client.0.trial046.stderr: -150> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command rotate-key cmddesc rotate-key hook 0x5560cff08678 EEXIST 2026-02-24T22:19:25.891 INFO:tasks.workunit.client.0.trial046.stderr: -149> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient: _reopen_session rank -1 2026-02-24T22:19:25.891 INFO:tasks.workunit.client.0.trial046.stderr: -148> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T22:19:25.891 INFO:tasks.workunit.client.0.trial046.stderr: -147> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-c con 0x5560cff30630 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:19:25.891 INFO:tasks.workunit.client.0.trial046.stderr: -146> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-b con 0x5560cff33da0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:19:25.891 INFO:tasks.workunit.client.0.trial046.stderr: -145> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-a con 0x5560cff3c4b0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:19:25.891 INFO:tasks.workunit.client.0.trial046.stderr: -144> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.891 INFO:tasks.workunit.client.0.trial046.stderr: -143> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.891 INFO:tasks.workunit.client.0.trial046.stderr: -142> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.891 INFO:tasks.workunit.client.0.trial046.stderr: -141> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient(hunting): _renew_subs 2026-02-24T22:19:25.891 INFO:tasks.workunit.client.0.trial046.stderr: -140> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient(hunting): authenticate will time out at 1935.688836s 2026-02-24T22:19:25.892 INFO:tasks.workunit.client.0.trial046.stderr: -139> 2026-02-24T22:19:25.582+0000 7f2638afe640 10 monclient(hunting): get_auth_request con 0x5560cff30630 auth_method 0 2026-02-24T22:19:25.892 INFO:tasks.workunit.client.0.trial046.stderr: -138> 2026-02-24T22:19:25.582+0000 7f2638afe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:25.892 INFO:tasks.workunit.client.0.trial046.stderr: -137> 2026-02-24T22:19:25.582+0000 7f2638afe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:25.892 INFO:tasks.workunit.client.0.trial046.stderr: -136> 2026-02-24T22:19:25.582+0000 7f2638afe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:25.892 INFO:tasks.workunit.client.0.trial046.stderr: -135> 2026-02-24T22:19:25.582+0000 7f26392ff640 10 monclient(hunting): get_auth_request con 0x5560cff3c4b0 auth_method 0 2026-02-24T22:19:25.892 INFO:tasks.workunit.client.0.trial046.stderr: -134> 2026-02-24T22:19:25.582+0000 7f26392ff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:25.892 INFO:tasks.workunit.client.0.trial046.stderr: -133> 2026-02-24T22:19:25.582+0000 7f26392ff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:25.892 INFO:tasks.workunit.client.0.trial046.stderr: -132> 2026-02-24T22:19:25.582+0000 7f26392ff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:25.892 INFO:tasks.workunit.client.0.trial046.stderr: -131> 2026-02-24T22:19:25.582+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:25.892 INFO:tasks.workunit.client.0.trial046.stderr: -130> 2026-02-24T22:19:25.582+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:25.892 INFO:tasks.workunit.client.0.trial046.stderr: -129> 2026-02-24T22:19:25.582+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:25.892 INFO:tasks.workunit.client.0.trial046.stderr: -128> 2026-02-24T22:19:25.582+0000 7f26392ff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:25.892 INFO:tasks.workunit.client.0.trial046.stderr: -127> 2026-02-24T22:19:25.582+0000 7f26392ff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:25.892 INFO:tasks.workunit.client.0.trial046.stderr: -126> 2026-02-24T22:19:25.582+0000 7f26392ff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:25.892 INFO:tasks.workunit.client.0.trial046.stderr: -125> 2026-02-24T22:19:25.582+0000 7f2638afe640 10 monclient(hunting): handle_auth_done global_id 7292 payload 824 2026-02-24T22:19:25.892 INFO:tasks.workunit.client.0.trial046.stderr: -124> 2026-02-24T22:19:25.582+0000 7f2638afe640 10 monclient: _finish_hunting 0 2026-02-24T22:19:25.892 INFO:tasks.workunit.client.0.trial046.stderr: -123> 2026-02-24T22:19:25.582+0000 7f2638afe640 1 monclient: found mon.noname-c 2026-02-24T22:19:25.892 INFO:tasks.workunit.client.0.trial046.stderr: -122> 2026-02-24T22:19:25.582+0000 7f2638afe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:19:25.892 INFO:tasks.workunit.client.0.trial046.stderr: -121> 2026-02-24T22:19:25.582+0000 7f2633fff640 10 monclient: get_auth_request con 0x5560cff33da0 auth_method 0 2026-02-24T22:19:25.892 INFO:tasks.workunit.client.0.trial046.stderr: -120> 2026-02-24T22:19:25.582+0000 7f26067fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:25.892 INFO:tasks.workunit.client.0.trial046.stderr: -119> 2026-02-24T22:19:25.582+0000 7f26067fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:19:25.892 INFO:tasks.workunit.client.0.trial046.stderr: -118> 2026-02-24T22:19:25.582+0000 7f26067fc640 10 monclient: dump: 2026-02-24T22:19:25.892 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:25.892 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:25.892 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.892 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.893 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:25.893 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.893 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:25.893 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:25.893 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:25.893 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.893 INFO:tasks.workunit.client.0.trial046.stderr: -117> 2026-02-24T22:19:25.582+0000 7f26067fc640 10 monclient: _finish_auth 0 2026-02-24T22:19:25.893 INFO:tasks.workunit.client.0.trial046.stderr: -116> 2026-02-24T22:19:25.582+0000 7f26067fc640 10 monclient: _check_auth_tickets 2026-02-24T22:19:25.893 INFO:tasks.workunit.client.0.trial046.stderr: -115> 2026-02-24T22:19:25.582+0000 7f26067fc640 10 monclient: handle_config config(0 keys) 2026-02-24T22:19:25.893 INFO:tasks.workunit.client.0.trial046.stderr: -114> 2026-02-24T22:19:25.582+0000 7f26067fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:25.893 INFO:tasks.workunit.client.0.trial046.stderr: -113> 2026-02-24T22:19:25.582+0000 7f26067fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:19:25.893 INFO:tasks.workunit.client.0.trial046.stderr: -112> 2026-02-24T22:19:25.582+0000 7f26067fc640 10 monclient: dump: 2026-02-24T22:19:25.893 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:25.893 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:25.893 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.893 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.893 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:25.893 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.893 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:25.893 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:25.893 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:25.893 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.893 INFO:tasks.workunit.client.0.trial046.stderr: -111> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 monclient: authenticate success, global_id 7292 2026-02-24T22:19:25.893 INFO:tasks.workunit.client.0.trial046.stderr: -110> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient: _renew_subs 2026-02-24T22:19:25.893 INFO:tasks.workunit.client.0.trial046.stderr: -109> 2026-02-24T22:19:25.582+0000 7f2607fff640 4 set_mon_vals no callback set 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -108> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -107> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient: _renew_subs 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -106> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -105> 2026-02-24T22:19:25.582+0000 7f263b02aa80 1 librados: init done 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -104> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command cr dump hook 0x5560cff32ae8 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -103> 2026-02-24T22:19:25.582+0000 7f263b02aa80 4 Realm: () 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -102> 2026-02-24T22:19:25.582+0000 7f263b02aa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -101> 2026-02-24T22:19:25.582+0000 7f263b02aa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -100> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -99> 2026-02-24T22:19:25.582+0000 7f263b02aa80 20 zonegroup default 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -98> 2026-02-24T22:19:25.582+0000 7f2633fff640 10 monclient: get_auth_request con 0x5560cffd0910 auth_method 0 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -97> 2026-02-24T22:19:25.582+0000 7f26067fc640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -96> 2026-02-24T22:19:25.582+0000 7f26067fc640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -95> 2026-02-24T22:19:25.582+0000 7f26067fc640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -94> 2026-02-24T22:19:25.582+0000 7f26392ff640 10 monclient: get_auth_request con 0x7f25fc0707b0 auth_method 0 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -93> 2026-02-24T22:19:25.582+0000 7f263b02aa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -92> 2026-02-24T22:19:25.582+0000 7f2638afe640 10 monclient: get_auth_request con 0x5560cfff3c30 auth_method 0 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -91> 2026-02-24T22:19:25.582+0000 7f263b02aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -90> 2026-02-24T22:19:25.582+0000 7f263b02aa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -89> 2026-02-24T22:19:25.582+0000 7f2633fff640 10 monclient: get_auth_request con 0x5560d0015340 auth_method 0 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -88> 2026-02-24T22:19:25.586+0000 7f263b02aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -87> 2026-02-24T22:19:25.586+0000 7f263b02aa80 20 started sync module instance, tier type = 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -86> 2026-02-24T22:19:25.586+0000 7f263b02aa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -85> 2026-02-24T22:19:25.586+0000 7f26392ff640 10 monclient: get_auth_request con 0x5560cfff3350 auth_method 0 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -84> 2026-02-24T22:19:25.586+0000 7f26392ff640 10 monclient: get_auth_request con 0x5560d017e200 auth_method 0 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -83> 2026-02-24T22:19:25.586+0000 7f2638afe640 10 monclient: get_auth_request con 0x5560d01e1200 auth_method 0 2026-02-24T22:19:25.894 INFO:tasks.workunit.client.0.trial046.stderr: -82> 2026-02-24T22:19:25.586+0000 7f2638afe640 10 monclient: get_auth_request con 0x5560d00d9140 auth_method 0 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -81> 2026-02-24T22:19:25.586+0000 7f2633fff640 10 monclient: get_auth_request con 0x5560d013c0c0 auth_method 0 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -80> 2026-02-24T22:19:25.586+0000 7f26392ff640 0 -- 10.20.193.46:0/2048819072 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x5560cfff3350 msgr2=0x5560d0096880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -79> 2026-02-24T22:19:25.586+0000 7f26067fc640 1 client.7292.objecter ms_handle_reset 0x5560cfff3350 session 0x5560cfff3190 osd.0 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -78> 2026-02-24T22:19:25.586+0000 7f26067fc640 10 monclient: _renew_subs 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -77> 2026-02-24T22:19:25.586+0000 7f26067fc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -76> 2026-02-24T22:19:25.586+0000 7f26392ff640 10 monclient: get_auth_request con 0x7f25fc09d4e0 auth_method 0 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -75> 2026-02-24T22:19:25.586+0000 7f263b02aa80 20 add_watcher() i=1 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -74> 2026-02-24T22:19:25.586+0000 7f263b02aa80 20 add_watcher() i=6 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -73> 2026-02-24T22:19:25.586+0000 7f263b02aa80 20 add_watcher() i=5 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -72> 2026-02-24T22:19:25.586+0000 7f263b02aa80 20 add_watcher() i=0 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -71> 2026-02-24T22:19:25.586+0000 7f263b02aa80 20 add_watcher() i=3 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -70> 2026-02-24T22:19:25.586+0000 7f263b02aa80 20 add_watcher() i=7 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -69> 2026-02-24T22:19:25.586+0000 7f263b02aa80 20 add_watcher() i=2 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -68> 2026-02-24T22:19:25.586+0000 7f263b02aa80 20 add_watcher() i=4 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -67> 2026-02-24T22:19:25.586+0000 7f263b02aa80 2 all 8 watchers are set, enabling cache 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -66> 2026-02-24T22:19:25.586+0000 7f2633fff640 10 monclient: get_auth_request con 0x7f2558008af0 auth_method 0 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -65> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding auth protocol: cephx 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -64> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding auth protocol: cephx 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -63> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding auth protocol: cephx 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -62> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding con mode: secure 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -61> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding con mode: crc 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -60> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding con mode: secure 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -59> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding con mode: crc 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -58> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding con mode: secure 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -57> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding con mode: crc 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -56> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding con mode: crc 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -55> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding con mode: secure 2026-02-24T22:19:25.895 INFO:tasks.workunit.client.0.trial046.stderr: -54> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding con mode: crc 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -53> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding con mode: secure 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -52> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding con mode: crc 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -51> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding con mode: secure 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -50> 2026-02-24T22:19:25.590+0000 7f263b02aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -49> 2026-02-24T22:19:25.590+0000 7f263b02aa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -48> 2026-02-24T22:19:25.590+0000 7f263b02aa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -47> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command cache list hook 0x5560cff3bf50 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -46> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command cache inspect hook 0x5560cff3bf50 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -45> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command cache erase hook 0x5560cff3bf50 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -44> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command cache zap hook 0x5560cff3bf50 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -43> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 note: GC not initialized 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -42> 2026-02-24T22:19:25.590+0000 7f25e5fe3640 20 reqs_thread_entry: start 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -41> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command sync trace show hook 0x7f262000c450 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -40> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command sync trace history hook 0x7f262000c450 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -39> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command sync trace active hook 0x7f262000c450 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -38> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command sync trace active_short hook 0x7f262000c450 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -37> 2026-02-24T22:19:25.590+0000 7f26392ff640 10 monclient: get_auth_request con 0x5560d010d050 auth_method 0 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -36> 2026-02-24T22:19:25.590+0000 7f26392ff640 0 -- 10.20.193.46:0/240894779 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5560d010d050 msgr2=0x5560d0108770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -35> 2026-02-24T22:19:25.794+0000 7f26392ff640 10 monclient: get_auth_request con 0x5560d010d050 auth_method 0 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -34> 2026-02-24T22:19:25.794+0000 7f2638afe640 10 monclient: get_auth_request con 0x5560d0104270 auth_method 0 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -33> 2026-02-24T22:19:25.798+0000 7f2633fff640 10 monclient: get_auth_request con 0x5560d0105960 auth_method 0 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -32> 2026-02-24T22:19:25.798+0000 7f26392ff640 10 monclient: get_auth_request con 0x5560d00fa9b0 auth_method 0 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -31> 2026-02-24T22:19:25.802+0000 7f2638afe640 10 monclient: get_auth_request con 0x5560d00fd130 auth_method 0 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -30> 2026-02-24T22:19:25.806+0000 7f2633fff640 10 monclient: get_auth_request con 0x5560d00ffff0 auth_method 0 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -29> 2026-02-24T22:19:25.830+0000 7f263b02aa80 20 init_complete bucket index max shards: 1 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -28> 2026-02-24T22:19:25.830+0000 7f263b02aa80 20 Filter name: none 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -27> 2026-02-24T22:19:25.830+0000 7f253ffff640 20 reqs_thread_entry: start 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -26> 2026-02-24T22:19:25.830+0000 7f263b02aa80 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -25> 2026-02-24T22:19:25.830+0000 7f263b02aa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:25.896 INFO:tasks.workunit.client.0.trial046.stderr: -24> 2026-02-24T22:19:25.830+0000 7f263b02aa80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: -23> 2026-02-24T22:19:25.830+0000 7f263b02aa80 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: -22> 2026-02-24T22:19:25.830+0000 7f263b02aa80 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: -21> 2026-02-24T22:19:25.830+0000 7f263b02aa80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 : miss 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: -20> 2026-02-24T22:19:25.830+0000 7f263b02aa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: -19> 2026-02-24T22:19:25.830+0000 7f263b02aa80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: -18> 2026-02-24T22:19:25.830+0000 7f263b02aa80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 info.flags=0x17 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: -17> 2026-02-24T22:19:25.830+0000 7f263b02aa80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 to cache LRU end 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: -16> 2026-02-24T22:19:25.830+0000 7f263b02aa80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: -15> 2026-02-24T22:19:25.830+0000 7f263b02aa80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: -14> 2026-02-24T22:19:25.830+0000 7f263b02aa80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: -13> 2026-02-24T22:19:25.834+0000 7f263b02aa80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 info.flags=0x17 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: -12> 2026-02-24T22:19:25.834+0000 7f263b02aa80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 to cache LRU end 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: -11> 2026-02-24T22:19:25.834+0000 7f263b02aa80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: -10> 2026-02-24T22:19:25.834+0000 7f263b02aa80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535, ofs0, ns] 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: -9> 2026-02-24T22:19:25.834+0000 7f2607fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959903 cookie 93874295768320 notifier 7292 bl.length()=1013 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: -8> 2026-02-24T22:19:25.834+0000 7f2607fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 info.flags=0x17 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: -7> 2026-02-24T22:19:25.834+0000 7f2607fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 to cache LRU end 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: -6> 2026-02-24T22:19:25.834+0000 7f2607fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: -5> 2026-02-24T22:19:25.834+0000 7f263b02aa80 20 handle_bi_update(): bucket=:abort-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535]): orig_sources= new_sources= 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: -4> 2026-02-24T22:19:25.834+0000 7f263b02aa80 20 handle_bi_update(): bucket=:abort-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535]): potential sources added=[] removed=[] 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: -3> 2026-02-24T22:19:25.834+0000 7f263b02aa80 20 handle_bi_update(): bucket=:abort-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535]): orig_dests= new_dests= 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: -2> 2026-02-24T22:19:25.834+0000 7f263b02aa80 20 handle_bi_update(): bucket=:abort-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535]): potential dests added=[] removed=[] 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: -1> 2026-02-24T22:19:25.838+0000 7f263b02aa80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f263b02aa80 time 2026-02-24T22:19:25.840474+0000 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f263e46ceb1] 2026-02-24T22:19:25.897 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x5560cbbfdb9f] 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x16a) [0x5560cc36926a] 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f263d76cd90] 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: 0> 2026-02-24T22:19:25.838+0000 7f263b02aa80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f263b02aa80 thread_name:radosgw-admin 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f263d785520] 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f263e46cf10] 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x5560cbbfdb9f] 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x16a) [0x5560cc36926a] 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f263d76cd90] 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr:--- logging levels --- 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 none 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 lockdep 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 context 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 crush 2026-02-24T22:19:25.898 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_balancer 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_locker 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log_expire 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_migrator 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 3/ 5 mds_quiesce 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 buffer 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 timer 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 filer 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 striper 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 objecter 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rados 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_mirror 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_replay 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_pwl 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 journaler 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objectcacher 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 immutable_obj_cache 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 client 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 osd 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 optracker 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objclass 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 filestore 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 journal 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 ms 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mon 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 0/10 monc 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 paxos 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 tp 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 auth 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 crypto 2026-02-24T22:19:25.899 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 finisher 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 reserver 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 heartbeatmap 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 perfcounter 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_sync 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_datacache 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_access 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dbstore 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_flight 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_lifecycle 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_restore 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw_notification 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_bucket_logging 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 javaclient 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 asok 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 throttle 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 refs 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 compressor 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore_compression 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluefs 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 bdev 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 kstore 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 4/ 5 rocksdb 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 fuse 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 2/ 5 mgr 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mgrc 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 dpdk 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 eventtrace 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 prioritycache 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 test 2026-02-24T22:19:25.900 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephfs_mirror 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephsqlite 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 crimson_interrupt 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_onode 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_odata 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_omap 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_tm 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_t 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cleaner 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_epm 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_lba 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_fixedkv_tree 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cache 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_journal 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_device 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_backref 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 alienstore 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mclock 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dedup 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cyanstore 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 ceph_exporter 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 memstore 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 trace 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 ceph_dedup 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: -2/-2 (syslog threshold) 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 99/99 (stderr threshold) 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr:--- pthread ID / name mapping for recent threads --- 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 7f253ffff640 / http_manager 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 7f25e5fe3640 / http_manager 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 7f26067fc640 / ms_dispatch 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 7f2607fff640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 7f26167fc640 / ms_dispatch 2026-02-24T22:19:25.901 INFO:tasks.workunit.client.0.trial046.stderr: 7f261d7fa640 / ms_dispatch 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: 7f261effd640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: 7f261f7fe640 / admin_socket 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: 7f26337fe640 / ms_dispatch 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: 7f2633fff640 / msgr-worker-2 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: 7f2638afe640 / msgr-worker-1 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: 7f26392ff640 / msgr-worker-0 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: 7f2639b00640 / io_context_pool 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: 7f263b02aa80 / radosgw-admin 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: max_recent 500 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: max_new 1000 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: log_file /var/log/ceph/ceph-client.admin.50265.log 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr:--- end dump of recent events --- 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr:--- begin dump of recent events --- 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: -499> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: -498> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-a con 0x5560cfc1bfe0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: -497> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-b con 0x5560cfcc4b00 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: -496> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-c con 0x5560cfcc7cf0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: -495> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: -494> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: -493> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: -492> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): _renew_subs 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: -491> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): authenticate will time out at 1935.678383s 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: -490> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): get_auth_request con 0x5560cfc1bfe0 auth_method 0 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: -489> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: -488> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: -487> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: -486> 2026-02-24T22:19:25.570+0000 7f26392ff640 10 monclient(hunting): get_auth_request con 0x5560cfcc7cf0 auth_method 0 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: -485> 2026-02-24T22:19:25.570+0000 7f26392ff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: -484> 2026-02-24T22:19:25.570+0000 7f26392ff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: -483> 2026-02-24T22:19:25.570+0000 7f26392ff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: -482> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:25.902 INFO:tasks.workunit.client.0.trial046.stderr: -481> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr: -480> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr: -479> 2026-02-24T22:19:25.570+0000 7f2633fff640 10 monclient(hunting): get_auth_request con 0x5560cfcc4b00 auth_method 0 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr: -478> 2026-02-24T22:19:25.570+0000 7f2633fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr: -477> 2026-02-24T22:19:25.570+0000 7f2633fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr: -476> 2026-02-24T22:19:25.570+0000 7f2633fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr: -475> 2026-02-24T22:19:25.570+0000 7f26392ff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr: -474> 2026-02-24T22:19:25.570+0000 7f26392ff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr: -473> 2026-02-24T22:19:25.570+0000 7f26392ff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr: -472> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): handle_auth_done global_id 7785 payload 274 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr: -471> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient: _finish_hunting 0 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr: -470> 2026-02-24T22:19:25.570+0000 7f2638afe640 1 monclient: found mon.noname-a 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr: -469> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr: -468> 2026-02-24T22:19:25.570+0000 7f261effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr: -467> 2026-02-24T22:19:25.570+0000 7f261effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr: -466> 2026-02-24T22:19:25.570+0000 7f261effd640 10 monclient: dump: 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr: -465> 2026-02-24T22:19:25.570+0000 7f261effd640 10 monclient: _finish_auth 0 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr: -464> 2026-02-24T22:19:25.570+0000 7f261effd640 10 monclient: _check_auth_tickets 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr: -463> 2026-02-24T22:19:25.570+0000 7f261effd640 10 monclient: handle_config config(0 keys) 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr: -462> 2026-02-24T22:19:25.570+0000 7f261effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr: -461> 2026-02-24T22:19:25.570+0000 7f261effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr: -460> 2026-02-24T22:19:25.570+0000 7f261effd640 10 monclient: dump: 2026-02-24T22:19:25.903 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr: -459> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 monclient: authenticate success, global_id 7785 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr: -458> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient: get_monmap_and_config success 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr: -457> 2026-02-24T22:19:25.570+0000 7f263b02aa80 4 set_mon_vals no callback set 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr: -456> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient: shutdown 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr: -455> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 asok(0x5560cfb8df00) unregister_commands rotate-key 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr: -454> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient: build_initial_monmap 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr: -453> 2026-02-24T22:19:25.570+0000 7f263b02aa80 1 build_initial for_mkfs: 0 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr: -452> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient: monmap: 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr: -451> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding auth protocol: cephx 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr: -450> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding auth protocol: cephx 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr: -449> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding auth protocol: cephx 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr: -448> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding con mode: secure 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr: -447> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding con mode: crc 2026-02-24T22:19:25.904 INFO:tasks.workunit.client.0.trial046.stderr: -446> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding con mode: secure 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -445> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding con mode: crc 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -444> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding con mode: secure 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -443> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding con mode: crc 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -442> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding con mode: crc 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -441> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding con mode: secure 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -440> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding con mode: crc 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -439> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding con mode: secure 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -438> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding con mode: crc 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -437> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc92840) adding con mode: secure 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -436> 2026-02-24T22:19:25.570+0000 7f263b02aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -435> 2026-02-24T22:19:25.570+0000 7f263b02aa80 1 librados: starting msgr at 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -434> 2026-02-24T22:19:25.570+0000 7f263b02aa80 1 librados: starting objecter 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -433> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command objecter_requests hook 0x5560cfc93140 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -432> 2026-02-24T22:19:25.570+0000 7f263b02aa80 1 librados: setting wanted keys 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -431> 2026-02-24T22:19:25.570+0000 7f263b02aa80 1 librados: calling monclient init 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -430> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient: init 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -429> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding auth protocol: cephx 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -428> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding auth protocol: cephx 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -427> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding auth protocol: cephx 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -426> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding con mode: secure 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -425> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding con mode: crc 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -424> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding con mode: secure 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -423> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding con mode: crc 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -422> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding con mode: secure 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -421> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding con mode: crc 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -420> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding con mode: crc 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -419> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding con mode: secure 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -418> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding con mode: crc 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -417> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding con mode: secure 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -416> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding con mode: crc 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -415> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 AuthRegistry(0x5560cfc8d7c0) adding con mode: secure 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -414> 2026-02-24T22:19:25.570+0000 7f263b02aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -413> 2026-02-24T22:19:25.570+0000 7f263b02aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:25.905 INFO:tasks.workunit.client.0.trial046.stderr: -412> 2026-02-24T22:19:25.570+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command rotate-key hook 0x5560cfc8d908 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -411> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient: _reopen_session rank -1 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -410> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -409> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-c con 0x5560cfc1bfe0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -408> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-a con 0x5560cfcc4b00 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -407> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-b con 0x5560cfcc7cf0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -406> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -405> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -404> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -403> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): _renew_subs 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -402> 2026-02-24T22:19:25.570+0000 7f263b02aa80 10 monclient(hunting): authenticate will time out at 1935.679750s 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -401> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): get_auth_request con 0x5560cfc1bfe0 auth_method 0 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -400> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -399> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -398> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -397> 2026-02-24T22:19:25.570+0000 7f2633fff640 10 monclient(hunting): get_auth_request con 0x5560cfcc4b00 auth_method 0 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -396> 2026-02-24T22:19:25.570+0000 7f2633fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -395> 2026-02-24T22:19:25.570+0000 7f2633fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -394> 2026-02-24T22:19:25.570+0000 7f2633fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -393> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -392> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -391> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -390> 2026-02-24T22:19:25.570+0000 7f2633fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -389> 2026-02-24T22:19:25.570+0000 7f2633fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -388> 2026-02-24T22:19:25.570+0000 7f2633fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -387> 2026-02-24T22:19:25.570+0000 7f2638afe640 10 monclient(hunting): handle_auth_done global_id 7283 payload 824 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -386> 2026-02-24T22:19:25.574+0000 7f2638afe640 10 monclient: _finish_hunting 0 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -385> 2026-02-24T22:19:25.574+0000 7f2638afe640 1 monclient: found mon.noname-c 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -384> 2026-02-24T22:19:25.574+0000 7f2638afe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -383> 2026-02-24T22:19:25.574+0000 7f26392ff640 10 monclient: get_auth_request con 0x5560cfcc7cf0 auth_method 0 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -382> 2026-02-24T22:19:25.574+0000 7f261d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -381> 2026-02-24T22:19:25.574+0000 7f261d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr: -380> 2026-02-24T22:19:25.574+0000 7f261d7fa640 10 monclient: dump: 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:25.906 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr: -379> 2026-02-24T22:19:25.574+0000 7f261d7fa640 10 monclient: _finish_auth 0 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr: -378> 2026-02-24T22:19:25.574+0000 7f261d7fa640 10 monclient: _check_auth_tickets 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr: -377> 2026-02-24T22:19:25.574+0000 7f261d7fa640 10 monclient: handle_config config(0 keys) 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr: -376> 2026-02-24T22:19:25.574+0000 7f261d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr: -375> 2026-02-24T22:19:25.574+0000 7f261d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr: -374> 2026-02-24T22:19:25.574+0000 7f261d7fa640 10 monclient: dump: 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr: -373> 2026-02-24T22:19:25.574+0000 7f263b02aa80 5 monclient: authenticate success, global_id 7283 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr: -372> 2026-02-24T22:19:25.574+0000 7f263b02aa80 10 monclient: _renew_subs 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr: -371> 2026-02-24T22:19:25.574+0000 7f263b02aa80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr: -370> 2026-02-24T22:19:25.574+0000 7f261effd640 4 set_mon_vals no callback set 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr: -369> 2026-02-24T22:19:25.574+0000 7f263b02aa80 10 monclient: _renew_subs 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr: -368> 2026-02-24T22:19:25.574+0000 7f263b02aa80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr: -367> 2026-02-24T22:19:25.574+0000 7f263b02aa80 1 librados: init done 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr: -366> 2026-02-24T22:19:25.574+0000 7f261d7fa640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr: -365> 2026-02-24T22:19:25.574+0000 7f261d7fa640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr: -364> 2026-02-24T22:19:25.574+0000 7f261d7fa640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:19:25.907 INFO:tasks.workunit.client.0.trial046.stderr: -363> 2026-02-24T22:19:25.574+0000 7f2633fff640 10 monclient: get_auth_request con 0x7f2618070730 auth_method 0 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: -362> 2026-02-24T22:19:25.574+0000 7f26392ff640 10 monclient: get_auth_request con 0x5560cfcef720 auth_method 0 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: -361> 2026-02-24T22:19:25.574+0000 7f2638afe640 10 monclient: get_auth_request con 0x5560cfd1b5c0 auth_method 0 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: -360> 2026-02-24T22:19:25.574+0000 7f2633fff640 10 monclient: get_auth_request con 0x5560cfd1e310 auth_method 0 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: -359> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: build_initial_monmap 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: -358> 2026-02-24T22:19:25.578+0000 7f263b02aa80 1 build_initial for_mkfs: 0 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: -357> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: monmap: 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: -356> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding auth protocol: cephx 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: -355> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding auth protocol: cephx 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: -354> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding auth protocol: cephx 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: -353> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding con mode: secure 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: -352> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding con mode: crc 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: -351> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding con mode: secure 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: -350> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding con mode: crc 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: -349> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding con mode: secure 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: -348> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding con mode: crc 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: -347> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding con mode: crc 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: -346> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding con mode: secure 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: -345> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding con mode: crc 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: -344> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding con mode: secure 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: -343> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding con mode: crc 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: -342> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1fe40) adding con mode: secure 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: -341> 2026-02-24T22:19:25.578+0000 7f263b02aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: -340> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command objecter_requests cmddesc objecter_requests hook 0x5560cfd24d70 EEXIST 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: -339> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: init 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: -338> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding auth protocol: cephx 2026-02-24T22:19:25.908 INFO:tasks.workunit.client.0.trial046.stderr: -337> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding auth protocol: cephx 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -336> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding auth protocol: cephx 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -335> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding con mode: secure 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -334> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding con mode: crc 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -333> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding con mode: secure 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -332> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding con mode: crc 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -331> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding con mode: secure 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -330> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding con mode: crc 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -329> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding con mode: crc 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -328> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding con mode: secure 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -327> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding con mode: crc 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -326> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding con mode: secure 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -325> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding con mode: crc 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -324> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cfd1c3b8) adding con mode: secure 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -323> 2026-02-24T22:19:25.578+0000 7f263b02aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -322> 2026-02-24T22:19:25.578+0000 7f263b02aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -321> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command rotate-key cmddesc rotate-key hook 0x5560cfd1c500 EEXIST 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -320> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: _reopen_session rank -1 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -319> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -318> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-c con 0x5560cfe78510 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -317> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-a con 0x5560cfe711b0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -316> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-b con 0x5560cfe71620 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -315> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -314> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -313> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -312> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): _renew_subs 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -311> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): authenticate will time out at 1935.685229s 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -310> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): get_auth_request con 0x5560cfe78510 auth_method 0 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -309> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -308> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -307> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -306> 2026-02-24T22:19:25.578+0000 7f2633fff640 10 monclient(hunting): get_auth_request con 0x5560cfe711b0 auth_method 0 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -305> 2026-02-24T22:19:25.578+0000 7f2633fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -304> 2026-02-24T22:19:25.578+0000 7f2633fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -303> 2026-02-24T22:19:25.578+0000 7f2633fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -302> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:25.909 INFO:tasks.workunit.client.0.trial046.stderr: -301> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr: -300> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr: -299> 2026-02-24T22:19:25.578+0000 7f2633fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr: -298> 2026-02-24T22:19:25.578+0000 7f2633fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr: -297> 2026-02-24T22:19:25.578+0000 7f2633fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr: -296> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): handle_auth_done global_id 7286 payload 824 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr: -295> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient: _finish_hunting 0 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr: -294> 2026-02-24T22:19:25.578+0000 7f2638afe640 1 monclient: found mon.noname-c 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr: -293> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr: -292> 2026-02-24T22:19:25.578+0000 7f26392ff640 10 monclient: get_auth_request con 0x5560cfe71620 auth_method 0 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr: -291> 2026-02-24T22:19:25.578+0000 7f26167fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr: -290> 2026-02-24T22:19:25.578+0000 7f26167fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr: -289> 2026-02-24T22:19:25.578+0000 7f26167fc640 10 monclient: dump: 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr: -288> 2026-02-24T22:19:25.578+0000 7f26167fc640 10 monclient: _finish_auth 0 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr: -287> 2026-02-24T22:19:25.578+0000 7f26167fc640 10 monclient: _check_auth_tickets 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr: -286> 2026-02-24T22:19:25.578+0000 7f26167fc640 10 monclient: handle_config config(0 keys) 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr: -285> 2026-02-24T22:19:25.578+0000 7f26167fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr: -284> 2026-02-24T22:19:25.578+0000 7f26167fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr: -283> 2026-02-24T22:19:25.578+0000 7f26167fc640 10 monclient: dump: 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:25.910 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -282> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 monclient: authenticate success, global_id 7286 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -281> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: _renew_subs 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -280> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -279> 2026-02-24T22:19:25.578+0000 7f2639b00640 4 set_mon_vals no callback set 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -278> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: _renew_subs 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -277> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -276> 2026-02-24T22:19:25.578+0000 7f26167fc640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -275> 2026-02-24T22:19:25.578+0000 7f26167fc640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -274> 2026-02-24T22:19:25.578+0000 7f26167fc640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -273> 2026-02-24T22:19:25.578+0000 7f2633fff640 10 monclient: get_auth_request con 0x7f26080706c0 auth_method 0 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -272> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: get_monmap_and_config 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -271> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: build_initial_monmap 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -270> 2026-02-24T22:19:25.578+0000 7f263b02aa80 1 build_initial for_mkfs: 0 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -269> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: monmap: 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -268> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding auth protocol: cephx 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -267> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding auth protocol: cephx 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -266> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding auth protocol: cephx 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -265> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding con mode: secure 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -264> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding con mode: crc 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -263> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding con mode: secure 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -262> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding con mode: crc 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -261> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding con mode: secure 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -260> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding con mode: crc 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -259> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding con mode: crc 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -258> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding con mode: secure 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -257> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding con mode: crc 2026-02-24T22:19:25.911 INFO:tasks.workunit.client.0.trial046.stderr: -256> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding con mode: secure 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -255> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding con mode: crc 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -254> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09560) adding con mode: secure 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -253> 2026-02-24T22:19:25.578+0000 7f263b02aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -252> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: init 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -251> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding auth protocol: cephx 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -250> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding auth protocol: cephx 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -249> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding auth protocol: cephx 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -248> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding con mode: secure 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -247> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding con mode: crc 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -246> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding con mode: secure 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -245> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding con mode: crc 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -244> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding con mode: secure 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -243> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding con mode: crc 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -242> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding con mode: crc 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -241> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding con mode: secure 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -240> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding con mode: crc 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -239> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding con mode: secure 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -238> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding con mode: crc 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -237> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abcc20) adding con mode: secure 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -236> 2026-02-24T22:19:25.578+0000 7f263b02aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -235> 2026-02-24T22:19:25.578+0000 7f263b02aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -234> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command rotate-key cmddesc rotate-key hook 0x7ffc55abcd68 EEXIST 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -233> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: _reopen_session rank -1 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -232> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -231> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-a con 0x5560cff30630 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -230> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-b con 0x5560cff33da0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -229> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-c con 0x5560cff0eb60 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -228> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -227> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -226> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -225> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): _renew_subs 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -224> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient(hunting): authenticate will time out at 1935.687558s 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -223> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): get_auth_request con 0x5560cff30630 auth_method 0 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -222> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -221> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -220> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -219> 2026-02-24T22:19:25.578+0000 7f26392ff640 10 monclient(hunting): get_auth_request con 0x5560cff0eb60 auth_method 0 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -218> 2026-02-24T22:19:25.578+0000 7f26392ff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -217> 2026-02-24T22:19:25.578+0000 7f26392ff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:25.912 INFO:tasks.workunit.client.0.trial046.stderr: -216> 2026-02-24T22:19:25.578+0000 7f26392ff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr: -215> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr: -214> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr: -213> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr: -212> 2026-02-24T22:19:25.578+0000 7f26392ff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr: -211> 2026-02-24T22:19:25.578+0000 7f26392ff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr: -210> 2026-02-24T22:19:25.578+0000 7f26392ff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr: -209> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient(hunting): handle_auth_done global_id 7794 payload 274 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr: -208> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient: _finish_hunting 0 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr: -207> 2026-02-24T22:19:25.578+0000 7f2638afe640 1 monclient: found mon.noname-a 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr: -206> 2026-02-24T22:19:25.578+0000 7f2638afe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr: -205> 2026-02-24T22:19:25.578+0000 7f2607fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr: -204> 2026-02-24T22:19:25.578+0000 7f2607fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr: -203> 2026-02-24T22:19:25.578+0000 7f2607fff640 10 monclient: dump: 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr: -202> 2026-02-24T22:19:25.578+0000 7f2607fff640 10 monclient: _finish_auth 0 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr: -201> 2026-02-24T22:19:25.578+0000 7f2607fff640 10 monclient: _check_auth_tickets 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr: -200> 2026-02-24T22:19:25.578+0000 7f2607fff640 10 monclient: handle_config config(0 keys) 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr: -199> 2026-02-24T22:19:25.578+0000 7f2607fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr: -198> 2026-02-24T22:19:25.578+0000 7f2607fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr: -197> 2026-02-24T22:19:25.578+0000 7f2607fff640 10 monclient: dump: 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:25.913 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -196> 2026-02-24T22:19:25.578+0000 7f263b02aa80 5 monclient: authenticate success, global_id 7794 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -195> 2026-02-24T22:19:25.578+0000 7f263b02aa80 10 monclient: get_monmap_and_config success 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -194> 2026-02-24T22:19:25.578+0000 7f263b02aa80 4 set_mon_vals no callback set 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -193> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient: shutdown 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -192> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient: build_initial_monmap 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -191> 2026-02-24T22:19:25.582+0000 7f263b02aa80 1 build_initial for_mkfs: 0 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -190> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient: monmap: 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -189> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding auth protocol: cephx 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -188> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding auth protocol: cephx 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -187> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding auth protocol: cephx 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -186> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding con mode: secure 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -185> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding con mode: crc 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -184> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding con mode: secure 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -183> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding con mode: crc 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -182> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding con mode: secure 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -181> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding con mode: crc 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -180> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding con mode: crc 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -179> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding con mode: secure 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -178> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding con mode: crc 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -177> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding con mode: secure 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -176> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding con mode: crc 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -175> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff09510) adding con mode: secure 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -174> 2026-02-24T22:19:25.582+0000 7f263b02aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -173> 2026-02-24T22:19:25.582+0000 7f263b02aa80 1 librados: starting msgr at 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -172> 2026-02-24T22:19:25.582+0000 7f263b02aa80 1 librados: starting objecter 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -171> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command objecter_requests cmddesc objecter_requests hook 0x5560cff379c0 EEXIST 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -170> 2026-02-24T22:19:25.582+0000 7f263b02aa80 1 librados: setting wanted keys 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -169> 2026-02-24T22:19:25.582+0000 7f263b02aa80 1 librados: calling monclient init 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -168> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient: init 2026-02-24T22:19:25.914 INFO:tasks.workunit.client.0.trial046.stderr: -167> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding auth protocol: cephx 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -166> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding auth protocol: cephx 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -165> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding auth protocol: cephx 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -164> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding con mode: secure 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -163> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding con mode: crc 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -162> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding con mode: secure 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -161> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding con mode: crc 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -160> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding con mode: secure 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -159> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding con mode: crc 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -158> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding con mode: crc 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -157> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding con mode: secure 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -156> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding con mode: crc 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -155> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding con mode: secure 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -154> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding con mode: crc 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -153> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 AuthRegistry(0x5560cff08530) adding con mode: secure 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -152> 2026-02-24T22:19:25.582+0000 7f263b02aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -151> 2026-02-24T22:19:25.582+0000 7f263b02aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -150> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command rotate-key cmddesc rotate-key hook 0x5560cff08678 EEXIST 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -149> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient: _reopen_session rank -1 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -148> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -147> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-c con 0x5560cff30630 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -146> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-b con 0x5560cff33da0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -145> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient(hunting): picked mon.noname-a con 0x5560cff3c4b0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -144> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -143> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -142> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -141> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient(hunting): _renew_subs 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -140> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient(hunting): authenticate will time out at 1935.688836s 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -139> 2026-02-24T22:19:25.582+0000 7f2638afe640 10 monclient(hunting): get_auth_request con 0x5560cff30630 auth_method 0 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -138> 2026-02-24T22:19:25.582+0000 7f2638afe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -137> 2026-02-24T22:19:25.582+0000 7f2638afe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -136> 2026-02-24T22:19:25.582+0000 7f2638afe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -135> 2026-02-24T22:19:25.582+0000 7f26392ff640 10 monclient(hunting): get_auth_request con 0x5560cff3c4b0 auth_method 0 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -134> 2026-02-24T22:19:25.582+0000 7f26392ff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -133> 2026-02-24T22:19:25.582+0000 7f26392ff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -132> 2026-02-24T22:19:25.582+0000 7f26392ff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -131> 2026-02-24T22:19:25.582+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -130> 2026-02-24T22:19:25.582+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -129> 2026-02-24T22:19:25.582+0000 7f2638afe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -128> 2026-02-24T22:19:25.582+0000 7f26392ff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -127> 2026-02-24T22:19:25.582+0000 7f26392ff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:25.915 INFO:tasks.workunit.client.0.trial046.stderr: -126> 2026-02-24T22:19:25.582+0000 7f26392ff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr: -125> 2026-02-24T22:19:25.582+0000 7f2638afe640 10 monclient(hunting): handle_auth_done global_id 7292 payload 824 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr: -124> 2026-02-24T22:19:25.582+0000 7f2638afe640 10 monclient: _finish_hunting 0 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr: -123> 2026-02-24T22:19:25.582+0000 7f2638afe640 1 monclient: found mon.noname-c 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr: -122> 2026-02-24T22:19:25.582+0000 7f2638afe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr: -121> 2026-02-24T22:19:25.582+0000 7f2633fff640 10 monclient: get_auth_request con 0x5560cff33da0 auth_method 0 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr: -120> 2026-02-24T22:19:25.582+0000 7f26067fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr: -119> 2026-02-24T22:19:25.582+0000 7f26067fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr: -118> 2026-02-24T22:19:25.582+0000 7f26067fc640 10 monclient: dump: 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr: -117> 2026-02-24T22:19:25.582+0000 7f26067fc640 10 monclient: _finish_auth 0 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr: -116> 2026-02-24T22:19:25.582+0000 7f26067fc640 10 monclient: _check_auth_tickets 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr: -115> 2026-02-24T22:19:25.582+0000 7f26067fc640 10 monclient: handle_config config(0 keys) 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr: -114> 2026-02-24T22:19:25.582+0000 7f26067fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr: -113> 2026-02-24T22:19:25.582+0000 7f26067fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr: -112> 2026-02-24T22:19:25.582+0000 7f26067fc640 10 monclient: dump: 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr: -111> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 monclient: authenticate success, global_id 7292 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr: -110> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient: _renew_subs 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr: -109> 2026-02-24T22:19:25.582+0000 7f2607fff640 4 set_mon_vals no callback set 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr: -108> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr: -107> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient: _renew_subs 2026-02-24T22:19:25.916 INFO:tasks.workunit.client.0.trial046.stderr: -106> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -105> 2026-02-24T22:19:25.582+0000 7f263b02aa80 1 librados: init done 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -104> 2026-02-24T22:19:25.582+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command cr dump hook 0x5560cff32ae8 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -103> 2026-02-24T22:19:25.582+0000 7f263b02aa80 4 Realm: () 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -102> 2026-02-24T22:19:25.582+0000 7f263b02aa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -101> 2026-02-24T22:19:25.582+0000 7f263b02aa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -100> 2026-02-24T22:19:25.582+0000 7f263b02aa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -99> 2026-02-24T22:19:25.582+0000 7f263b02aa80 20 zonegroup default 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -98> 2026-02-24T22:19:25.582+0000 7f2633fff640 10 monclient: get_auth_request con 0x5560cffd0910 auth_method 0 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -97> 2026-02-24T22:19:25.582+0000 7f26067fc640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -96> 2026-02-24T22:19:25.582+0000 7f26067fc640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -95> 2026-02-24T22:19:25.582+0000 7f26067fc640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -94> 2026-02-24T22:19:25.582+0000 7f26392ff640 10 monclient: get_auth_request con 0x7f25fc0707b0 auth_method 0 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -93> 2026-02-24T22:19:25.582+0000 7f263b02aa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -92> 2026-02-24T22:19:25.582+0000 7f2638afe640 10 monclient: get_auth_request con 0x5560cfff3c30 auth_method 0 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -91> 2026-02-24T22:19:25.582+0000 7f263b02aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -90> 2026-02-24T22:19:25.582+0000 7f263b02aa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -89> 2026-02-24T22:19:25.582+0000 7f2633fff640 10 monclient: get_auth_request con 0x5560d0015340 auth_method 0 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -88> 2026-02-24T22:19:25.586+0000 7f263b02aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -87> 2026-02-24T22:19:25.586+0000 7f263b02aa80 20 started sync module instance, tier type = 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -86> 2026-02-24T22:19:25.586+0000 7f263b02aa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -85> 2026-02-24T22:19:25.586+0000 7f26392ff640 10 monclient: get_auth_request con 0x5560cfff3350 auth_method 0 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -84> 2026-02-24T22:19:25.586+0000 7f26392ff640 10 monclient: get_auth_request con 0x5560d017e200 auth_method 0 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -83> 2026-02-24T22:19:25.586+0000 7f2638afe640 10 monclient: get_auth_request con 0x5560d01e1200 auth_method 0 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -82> 2026-02-24T22:19:25.586+0000 7f2638afe640 10 monclient: get_auth_request con 0x5560d00d9140 auth_method 0 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -81> 2026-02-24T22:19:25.586+0000 7f2633fff640 10 monclient: get_auth_request con 0x5560d013c0c0 auth_method 0 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -80> 2026-02-24T22:19:25.586+0000 7f26392ff640 0 -- 10.20.193.46:0/2048819072 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x5560cfff3350 msgr2=0x5560d0096880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -79> 2026-02-24T22:19:25.586+0000 7f26067fc640 1 client.7292.objecter ms_handle_reset 0x5560cfff3350 session 0x5560cfff3190 osd.0 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -78> 2026-02-24T22:19:25.586+0000 7f26067fc640 10 monclient: _renew_subs 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -77> 2026-02-24T22:19:25.586+0000 7f26067fc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -76> 2026-02-24T22:19:25.586+0000 7f26392ff640 10 monclient: get_auth_request con 0x7f25fc09d4e0 auth_method 0 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -75> 2026-02-24T22:19:25.586+0000 7f263b02aa80 20 add_watcher() i=1 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -74> 2026-02-24T22:19:25.586+0000 7f263b02aa80 20 add_watcher() i=6 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -73> 2026-02-24T22:19:25.586+0000 7f263b02aa80 20 add_watcher() i=5 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -72> 2026-02-24T22:19:25.586+0000 7f263b02aa80 20 add_watcher() i=0 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -71> 2026-02-24T22:19:25.586+0000 7f263b02aa80 20 add_watcher() i=3 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -70> 2026-02-24T22:19:25.586+0000 7f263b02aa80 20 add_watcher() i=7 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -69> 2026-02-24T22:19:25.586+0000 7f263b02aa80 20 add_watcher() i=2 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -68> 2026-02-24T22:19:25.586+0000 7f263b02aa80 20 add_watcher() i=4 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -67> 2026-02-24T22:19:25.586+0000 7f263b02aa80 2 all 8 watchers are set, enabling cache 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -66> 2026-02-24T22:19:25.586+0000 7f2633fff640 10 monclient: get_auth_request con 0x7f2558008af0 auth_method 0 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -65> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding auth protocol: cephx 2026-02-24T22:19:25.917 INFO:tasks.workunit.client.0.trial046.stderr: -64> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding auth protocol: cephx 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -63> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding auth protocol: cephx 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -62> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding con mode: secure 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -61> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding con mode: crc 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -60> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding con mode: secure 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -59> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding con mode: crc 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -58> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding con mode: secure 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -57> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding con mode: crc 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -56> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding con mode: crc 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -55> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding con mode: secure 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -54> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding con mode: crc 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -53> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding con mode: secure 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -52> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding con mode: crc 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -51> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 AuthRegistry(0x7ffc55abd090) adding con mode: secure 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -50> 2026-02-24T22:19:25.590+0000 7f263b02aa80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -49> 2026-02-24T22:19:25.590+0000 7f263b02aa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -48> 2026-02-24T22:19:25.590+0000 7f263b02aa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -47> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command cache list hook 0x5560cff3bf50 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -46> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command cache inspect hook 0x5560cff3bf50 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -45> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command cache erase hook 0x5560cff3bf50 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -44> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command cache zap hook 0x5560cff3bf50 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -43> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 note: GC not initialized 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -42> 2026-02-24T22:19:25.590+0000 7f25e5fe3640 20 reqs_thread_entry: start 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -41> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command sync trace show hook 0x7f262000c450 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -40> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command sync trace history hook 0x7f262000c450 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -39> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command sync trace active hook 0x7f262000c450 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -38> 2026-02-24T22:19:25.590+0000 7f263b02aa80 5 asok(0x5560cfb8df00) register_command sync trace active_short hook 0x7f262000c450 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -37> 2026-02-24T22:19:25.590+0000 7f26392ff640 10 monclient: get_auth_request con 0x5560d010d050 auth_method 0 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -36> 2026-02-24T22:19:25.590+0000 7f26392ff640 0 -- 10.20.193.46:0/240894779 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5560d010d050 msgr2=0x5560d0108770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -35> 2026-02-24T22:19:25.794+0000 7f26392ff640 10 monclient: get_auth_request con 0x5560d010d050 auth_method 0 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -34> 2026-02-24T22:19:25.794+0000 7f2638afe640 10 monclient: get_auth_request con 0x5560d0104270 auth_method 0 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -33> 2026-02-24T22:19:25.798+0000 7f2633fff640 10 monclient: get_auth_request con 0x5560d0105960 auth_method 0 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -32> 2026-02-24T22:19:25.798+0000 7f26392ff640 10 monclient: get_auth_request con 0x5560d00fa9b0 auth_method 0 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -31> 2026-02-24T22:19:25.802+0000 7f2638afe640 10 monclient: get_auth_request con 0x5560d00fd130 auth_method 0 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -30> 2026-02-24T22:19:25.806+0000 7f2633fff640 10 monclient: get_auth_request con 0x5560d00ffff0 auth_method 0 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -29> 2026-02-24T22:19:25.830+0000 7f263b02aa80 20 init_complete bucket index max shards: 1 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -28> 2026-02-24T22:19:25.830+0000 7f263b02aa80 20 Filter name: none 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -27> 2026-02-24T22:19:25.830+0000 7f253ffff640 20 reqs_thread_entry: start 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -26> 2026-02-24T22:19:25.830+0000 7f263b02aa80 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -25> 2026-02-24T22:19:25.830+0000 7f263b02aa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -24> 2026-02-24T22:19:25.830+0000 7f263b02aa80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -23> 2026-02-24T22:19:25.830+0000 7f263b02aa80 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -22> 2026-02-24T22:19:25.830+0000 7f263b02aa80 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -21> 2026-02-24T22:19:25.830+0000 7f263b02aa80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 : miss 2026-02-24T22:19:25.918 INFO:tasks.workunit.client.0.trial046.stderr: -20> 2026-02-24T22:19:25.830+0000 7f263b02aa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: -19> 2026-02-24T22:19:25.830+0000 7f263b02aa80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: -18> 2026-02-24T22:19:25.830+0000 7f263b02aa80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 info.flags=0x17 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: -17> 2026-02-24T22:19:25.830+0000 7f263b02aa80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 to cache LRU end 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: -16> 2026-02-24T22:19:25.830+0000 7f263b02aa80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: -15> 2026-02-24T22:19:25.830+0000 7f263b02aa80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: -14> 2026-02-24T22:19:25.830+0000 7f263b02aa80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: -13> 2026-02-24T22:19:25.834+0000 7f263b02aa80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 info.flags=0x17 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: -12> 2026-02-24T22:19:25.834+0000 7f263b02aa80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 to cache LRU end 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: -11> 2026-02-24T22:19:25.834+0000 7f263b02aa80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: -10> 2026-02-24T22:19:25.834+0000 7f263b02aa80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535, ofs0, ns] 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: -9> 2026-02-24T22:19:25.834+0000 7f2607fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959903 cookie 93874295768320 notifier 7292 bl.length()=1013 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: -8> 2026-02-24T22:19:25.834+0000 7f2607fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 info.flags=0x17 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: -7> 2026-02-24T22:19:25.834+0000 7f2607fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 to cache LRU end 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: -6> 2026-02-24T22:19:25.834+0000 7f2607fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: -5> 2026-02-24T22:19:25.834+0000 7f263b02aa80 20 handle_bi_update(): bucket=:abort-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535]): orig_sources= new_sources= 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: -4> 2026-02-24T22:19:25.834+0000 7f263b02aa80 20 handle_bi_update(): bucket=:abort-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535]): potential sources added=[] removed=[] 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: -3> 2026-02-24T22:19:25.834+0000 7f263b02aa80 20 handle_bi_update(): bucket=:abort-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535]): orig_dests= new_dests= 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: -2> 2026-02-24T22:19:25.834+0000 7f263b02aa80 20 handle_bi_update(): bucket=:abort-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535]): potential dests added=[] removed=[] 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: -1> 2026-02-24T22:19:25.838+0000 7f263b02aa80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f263b02aa80 time 2026-02-24T22:19:25.840474+0000 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f263e46ceb1] 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x5560cbbfdb9f] 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x16a) [0x5560cc36926a] 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f263d76cd90] 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: 0> 2026-02-24T22:19:25.838+0000 7f263b02aa80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f263b02aa80 thread_name:radosgw-admin 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f263d785520] 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f263e46cf10] 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x5560cbbfdb9f] 2026-02-24T22:19:25.919 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x16a) [0x5560cc36926a] 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f263d76cd90] 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr:--- logging levels --- 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 none 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 lockdep 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 context 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 crush 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_balancer 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_locker 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log_expire 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_migrator 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 3/ 5 mds_quiesce 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 buffer 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 timer 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 filer 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 striper 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 objecter 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rados 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_mirror 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_replay 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_pwl 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 journaler 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objectcacher 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 immutable_obj_cache 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 client 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 osd 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 optracker 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objclass 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 filestore 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 journal 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 ms 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mon 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 0/10 monc 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 paxos 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 tp 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 auth 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 crypto 2026-02-24T22:19:25.920 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 finisher 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 reserver 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 heartbeatmap 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 perfcounter 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_sync 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_datacache 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_access 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dbstore 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_flight 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_lifecycle 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_restore 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw_notification 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_bucket_logging 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 javaclient 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 asok 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 throttle 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 refs 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 compressor 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore_compression 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluefs 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 bdev 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 kstore 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 4/ 5 rocksdb 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 fuse 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 2/ 5 mgr 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mgrc 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 dpdk 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 eventtrace 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 prioritycache 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 test 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephfs_mirror 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephsqlite 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 crimson_interrupt 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_onode 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_odata 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_omap 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_tm 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_t 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cleaner 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_epm 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_lba 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_fixedkv_tree 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cache 2026-02-24T22:19:25.921 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_journal 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_device 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_backref 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 alienstore 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mclock 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dedup 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cyanstore 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 ceph_exporter 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 memstore 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 trace 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 ceph_dedup 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: -2/-2 (syslog threshold) 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: 99/99 (stderr threshold) 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr:--- pthread ID / name mapping for recent threads --- 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: 7f253ffff640 / http_manager 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: 7f25e5fe3640 / http_manager 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: 7f26067fc640 / ms_dispatch 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: 7f2607fff640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: 7f26167fc640 / ms_dispatch 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: 7f261d7fa640 / ms_dispatch 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: 7f261effd640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: 7f261f7fe640 / admin_socket 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: 7f26337fe640 / ms_dispatch 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: 7f2633fff640 / msgr-worker-2 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: 7f2638afe640 / msgr-worker-1 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: 7f26392ff640 / msgr-worker-0 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: 7f2639b00640 / io_context_pool 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: 7f263b02aa80 / radosgw-admin 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: max_recent 500 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: max_new 1000 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr: log_file /var/lib/ceph/crash/2026-02-24T22:19:25.842509Z_40e6f65f-d563-4ca5-b732-1397a5892963/log 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr:--- end dump of recent events --- 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr:Aborted 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket abort-at-do-reshard 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:19:25.922 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:19:25.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.878+0000 7f9e82c1fa80 4 Realm: () 2026-02-24T22:19:25.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.878+0000 7f9e82c1fa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:19:25.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.878+0000 7f9e82c1fa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:19:25.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.878+0000 7f9e82c1fa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:19:25.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.878+0000 7f9e82c1fa80 20 zonegroup default 2026-02-24T22:19:25.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.878+0000 7f9e82c1fa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:25.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.878+0000 7f9e82c1fa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:25.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.878+0000 7f9e82c1fa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:25.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.878+0000 7f9e82c1fa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:25.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.878+0000 7f9e82c1fa80 20 started sync module instance, tier type = 2026-02-24T22:19:25.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.878+0000 7f9e82c1fa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:19:25.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.882+0000 7f9e82c1fa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:19:25.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.882+0000 7f9e82c1fa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:19:25.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.882+0000 7f9e82c1fa80 5 note: GC not initialized 2026-02-24T22:19:25.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.882+0000 7f9e2e7dc640 20 reqs_thread_entry: start 2026-02-24T22:19:25.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.882+0000 7f9e7b7fe640 0 -- 10.20.193.46:0/3046107367 >> [v2:10.20.193.46:6826/3065971976,v1:10.20.193.46:6827/3065971976] conn(0x56162b4fc6b0 msgr2=0x56162b4fcab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:25.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:25.890+0000 7f9e7b7fe640 0 -- 10.20.193.46:0/3046107367 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x56162b4e6e70 msgr2=0x56162b4e72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:26.118 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.122+0000 7f9e82c1fa80 20 init_complete bucket index max shards: 1 2026-02-24T22:19:26.118 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.122+0000 7f9e82c1fa80 20 Filter name: none 2026-02-24T22:19:26.118 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.122+0000 7f9e2cfd9640 20 reqs_thread_entry: start 2026-02-24T22:19:26.118 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.122+0000 7f9e82c1fa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:26.118 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.122+0000 7f9e82c1fa80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:19:26.118 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.122+0000 7f9e82c1fa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:26.118 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.122+0000 7f9e82c1fa80 20 rados_obj.operate() r=0 bl.length=481 2026-02-24T22:19:26.119 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.122+0000 7f9e82c1fa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:19:26.119 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.122+0000 7f9e82c1fa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:26.119 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.122+0000 7f9e82c1fa80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:19:26.119 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.122+0000 7f9e82c1fa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:26.119 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.122+0000 7f9e82c1fa80 20 rados_obj.operate() r=0 bl.length=481 2026-02-24T22:19:26.119 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.122+0000 7f9e82c1fa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:19:26.130 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:26.130 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:19:26.131 INFO:tasks.workunit.client.0.trial046.stderr:bucket: abort-at-do-reshard id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:19:26.131 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:26.131 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:26.132 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:26.137 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:26.137 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:26.185 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket abort-at-do-reshard --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2026-02-24T22:19:26.201 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:19:26.201 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:19:26.219 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.222+0000 7f3b378dda80 4 Realm: () 2026-02-24T22:19:26.219 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.222+0000 7f3b378dda80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:19:26.220 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.222+0000 7f3b378dda80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:19:26.220 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.222+0000 7f3b378dda80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:19:26.220 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.222+0000 7f3b378dda80 20 zonegroup default 2026-02-24T22:19:26.220 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.222+0000 7f3b378dda80 20 rados->read ofs=0 len=0 2026-02-24T22:19:26.221 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.226+0000 7f3b378dda80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:26.221 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.226+0000 7f3b378dda80 20 rados->read ofs=0 len=0 2026-02-24T22:19:26.222 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.226+0000 7f3b378dda80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:26.222 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.226+0000 7f3b378dda80 20 started sync module instance, tier type = 2026-02-24T22:19:26.222 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.226+0000 7f3b378dda80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:19:26.222 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.226+0000 7f3b35bb2640 0 -- 10.20.193.46:0/4291975671 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x564b839ab0f0 msgr2=0x564b839cb4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:26.224 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.226+0000 7f3b378dda80 20 add_watcher() i=1 2026-02-24T22:19:26.224 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.226+0000 7f3b378dda80 20 add_watcher() i=3 2026-02-24T22:19:26.224 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.226+0000 7f3b378dda80 20 add_watcher() i=6 2026-02-24T22:19:26.224 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.226+0000 7f3b378dda80 20 add_watcher() i=5 2026-02-24T22:19:26.224 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.226+0000 7f3b378dda80 20 add_watcher() i=0 2026-02-24T22:19:26.224 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.226+0000 7f3b378dda80 20 add_watcher() i=2 2026-02-24T22:19:26.225 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.226+0000 7f3b378dda80 20 add_watcher() i=7 2026-02-24T22:19:26.225 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.226+0000 7f3b378dda80 20 add_watcher() i=4 2026-02-24T22:19:26.225 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.226+0000 7f3b378dda80 2 all 8 watchers are set, enabling cache 2026-02-24T22:19:26.226 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.230+0000 7f3b378dda80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:19:26.226 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.230+0000 7f3b378dda80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:19:26.226 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.230+0000 7f3b378dda80 5 note: GC not initialized 2026-02-24T22:19:26.226 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.230+0000 7f3ae27e4640 20 reqs_thread_entry: start 2026-02-24T22:19:26.226 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.230+0000 7f3b35bb2640 0 -- 10.20.193.46:0/1860243492 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x564b83a425e0 msgr2=0x564b83a39d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:26.464 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.466+0000 7f3b378dda80 20 init_complete bucket index max shards: 1 2026-02-24T22:19:26.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.466+0000 7f3b378dda80 20 Filter name: none 2026-02-24T22:19:26.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.466+0000 7f3ae0fe1640 20 reqs_thread_entry: start 2026-02-24T22:19:26.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.466+0000 7f3b378dda80 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2026-02-24T22:19:26.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.466+0000 7f3b378dda80 20 rados->read ofs=0 len=0 2026-02-24T22:19:26.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.466+0000 7f3b378dda80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:19:26.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.466+0000 7f3b378dda80 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2026-02-24T22:19:26.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.466+0000 7f3b378dda80 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2026-02-24T22:19:26.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.466+0000 7f3b378dda80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 : miss 2026-02-24T22:19:26.465 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.466+0000 7f3b378dda80 20 rados->read ofs=0 len=0 2026-02-24T22:19:26.466 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.470+0000 7f3b378dda80 20 rados_obj.operate() r=0 bl.length=481 2026-02-24T22:19:26.466 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.470+0000 7f3b378dda80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 info.flags=0x17 2026-02-24T22:19:26.466 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.470+0000 7f3b378dda80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 to cache LRU end 2026-02-24T22:19:26.466 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.470+0000 7f3b378dda80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:19:26.466 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.470+0000 7f3b378dda80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:26.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.470+0000 7f3b378dda80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 2026-02-24T22:19:26.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.470+0000 7f3b378dda80 0 INFO: RGWReshardLock::lock found lock on abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 to be held by another RGW process; skipping for now 2026-02-24T22:19:26.468 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.470+0000 7f3b378dda80 20 remove_watcher() i=3 2026-02-24T22:19:26.469 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.470+0000 7f3b378dda80 2 removed watcher, disabling cache 2026-02-24T22:19:26.469 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.470+0000 7f3b378dda80 20 remove_watcher() i=0 2026-02-24T22:19:26.469 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.470+0000 7f3b378dda80 20 remove_watcher() i=2 2026-02-24T22:19:26.469 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.470+0000 7f3b378dda80 20 remove_watcher() i=6 2026-02-24T22:19:26.469 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.470+0000 7f3b378dda80 20 remove_watcher() i=4 2026-02-24T22:19:26.469 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.470+0000 7f3b378dda80 20 remove_watcher() i=1 2026-02-24T22:19:26.469 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.470+0000 7f3b378dda80 20 remove_watcher() i=5 2026-02-24T22:19:26.469 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:26.470+0000 7f3b378dda80 20 remove_watcher() i=7 2026-02-24T22:19:26.475 INFO:tasks.workunit.client.0.trial046.stderr:waiting 30 seconds for reshard lock to expire... 2026-02-24T22:19:56.505 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket abort-at-do-reshard --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2026-02-24T22:19:56.521 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:19:56.521 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:19:56.539 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.541+0000 7f8b3f68da80 4 Realm: () 2026-02-24T22:19:56.539 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.541+0000 7f8b3f68da80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:19:56.539 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.541+0000 7f8b3f68da80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:19:56.539 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.541+0000 7f8b3f68da80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:19:56.540 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.541+0000 7f8b3f68da80 20 zonegroup default 2026-02-24T22:19:56.540 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.541+0000 7f8b3f68da80 20 rados->read ofs=0 len=0 2026-02-24T22:19:56.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.545+0000 7f8b3f68da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:56.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.545+0000 7f8b3f68da80 20 rados->read ofs=0 len=0 2026-02-24T22:19:56.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.545+0000 7f8b3f68da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:56.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.545+0000 7f8b3f68da80 20 started sync module instance, tier type = 2026-02-24T22:19:56.542 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.545+0000 7f8b3f68da80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:19:56.542 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.545+0000 7f8b3c960640 0 -- 10.20.193.46:0/3179113831 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x55a1044d85d0 msgr2=0x55a1044f8a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:56.543 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.545+0000 7f8b3f68da80 20 add_watcher() i=1 2026-02-24T22:19:56.543 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.545+0000 7f8b3f68da80 20 add_watcher() i=3 2026-02-24T22:19:56.543 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.545+0000 7f8b3f68da80 20 add_watcher() i=6 2026-02-24T22:19:56.544 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.545+0000 7f8b3f68da80 20 add_watcher() i=5 2026-02-24T22:19:56.544 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.545+0000 7f8b3f68da80 20 add_watcher() i=0 2026-02-24T22:19:56.544 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.545+0000 7f8b3f68da80 20 add_watcher() i=2 2026-02-24T22:19:56.544 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.545+0000 7f8b3f68da80 20 add_watcher() i=7 2026-02-24T22:19:56.544 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.549+0000 7f8b3f68da80 20 add_watcher() i=4 2026-02-24T22:19:56.545 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.549+0000 7f8b3f68da80 2 all 8 watchers are set, enabling cache 2026-02-24T22:19:56.545 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.549+0000 7f8b3c960640 0 -- 10.20.193.46:0/2271619310 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f8a5c008af0 msgr2=0x7f8a5c028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:56.546 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.549+0000 7f8b3f68da80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:19:56.546 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.549+0000 7f8b3f68da80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:19:56.546 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.549+0000 7f8b3f68da80 5 note: GC not initialized 2026-02-24T22:19:56.546 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.549+0000 7f8aea7e4640 20 reqs_thread_entry: start 2026-02-24T22:19:56.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.581+0000 7f8b3f68da80 20 init_complete bucket index max shards: 1 2026-02-24T22:19:56.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.581+0000 7f8b3f68da80 20 Filter name: none 2026-02-24T22:19:56.580 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.581+0000 7f8ae8fe1640 20 reqs_thread_entry: start 2026-02-24T22:19:56.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.581+0000 7f8b3f68da80 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2026-02-24T22:19:56.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.581+0000 7f8b3f68da80 20 rados->read ofs=0 len=0 2026-02-24T22:19:56.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.585+0000 7f8b3f68da80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:19:56.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.585+0000 7f8b3f68da80 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2026-02-24T22:19:56.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.585+0000 7f8b3f68da80 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2026-02-24T22:19:56.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.585+0000 7f8b3f68da80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 : miss 2026-02-24T22:19:56.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.585+0000 7f8b3f68da80 20 rados->read ofs=0 len=0 2026-02-24T22:19:56.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.585+0000 7f8b3f68da80 20 rados_obj.operate() r=0 bl.length=481 2026-02-24T22:19:56.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.585+0000 7f8b3f68da80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 info.flags=0x17 2026-02-24T22:19:56.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.585+0000 7f8b3f68da80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 to cache LRU end 2026-02-24T22:19:56.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.585+0000 7f8b3f68da80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:19:56.581 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.585+0000 7f8b3f68da80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:56.582 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.585+0000 7f8b3f68da80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 2026-02-24T22:19:56.582 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.585+0000 7f8b3f68da80 10 init_target_layout removing existing target index objects from a previous reshard attempt 2026-02-24T22:19:56.586 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.589+0000 7f8b3f68da80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 info.flags=0x17 2026-02-24T22:19:56.586 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.589+0000 7f8b3f68da80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 to cache LRU end 2026-02-24T22:19:56.586 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.589+0000 7f8b3f68da80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:56.586 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.589+0000 7f8b3f68da80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535, ofs0, ns] 2026-02-24T22:19:56.586 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.589+0000 7f8b18ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959904 cookie 94150049876480 notifier 7343 bl.length()=1013 2026-02-24T22:19:56.586 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.589+0000 7f8b18ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 info.flags=0x17 2026-02-24T22:19:56.586 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.589+0000 7f8b18ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 to cache LRU end 2026-02-24T22:19:56.586 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.589+0000 7f8b18ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:56.587 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.589+0000 7f8b3f68da80 20 handle_bi_update(): bucket=:abort-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535]): orig_sources= new_sources= 2026-02-24T22:19:56.587 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.589+0000 7f8b3f68da80 20 handle_bi_update(): bucket=:abort-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535]): potential sources added=[] removed=[] 2026-02-24T22:19:56.587 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.589+0000 7f8b3f68da80 20 handle_bi_update(): bucket=:abort-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535]): orig_dests= new_dests= 2026-02-24T22:19:56.587 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.589+0000 7f8b3f68da80 20 handle_bi_update(): bucket=:abort-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535]): potential dests added=[] removed=[] 2026-02-24T22:19:56.587 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.589+0000 7f8b18ff9640 20 trim reshard logs: write on '.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535.0' needs retry: Success 2026-02-24T22:19:56.588 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.593+0000 7f8b3f68da80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535.2.0 2026-02-24T22:19:56.588 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.593+0000 7f8b3f68da80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535.2.0 2026-02-24T22:19:56.588 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.593+0000 7f8b3f68da80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535.2.1 2026-02-24T22:19:56.588 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.593+0000 7f8b3f68da80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535.2.1 2026-02-24T22:19:56.589 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.593+0000 7f8b3f68da80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535.0 2026-02-24T22:19:56.590 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.593+0000 7f8b3f68da80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 : hit (requested=0x17, cached=0x17) 2026-02-24T22:19:56.590 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.593+0000 7f8b3f68da80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 2026-02-24T22:19:56.591 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.593+0000 7f8b3f68da80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 info.flags=0x17 2026-02-24T22:19:56.591 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.593+0000 7f8b3f68da80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 to cache LRU end 2026-02-24T22:19:56.591 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.593+0000 7f8b3f68da80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:56.591 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.593+0000 7f8b3f68da80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535, ofs0, ns] 2026-02-24T22:19:56.591 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.593+0000 7f8b07fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959905 cookie 94150049876480 notifier 7343 bl.length()=1013 2026-02-24T22:19:56.591 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.593+0000 7f8b07fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 info.flags=0x17 2026-02-24T22:19:56.591 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.593+0000 7f8b07fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 to cache LRU end 2026-02-24T22:19:56.591 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.593+0000 7f8b07fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:56.592 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.593+0000 7f8b3f68da80 20 handle_bi_update(): bucket=:abort-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535]): orig_sources= new_sources= 2026-02-24T22:19:56.592 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.593+0000 7f8b3f68da80 20 handle_bi_update(): bucket=:abort-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535]): potential sources added=[] removed=[] 2026-02-24T22:19:56.592 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.593+0000 7f8b3f68da80 20 handle_bi_update(): bucket=:abort-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535]): orig_dests= new_dests= 2026-02-24T22:19:56.593 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.593+0000 7f8b3f68da80 20 handle_bi_update(): bucket=:abort-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535]): potential dests added=[] removed=[] 2026-02-24T22:19:56.593 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.593+0000 7f8b3f68da80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535.0 2026-02-24T22:19:56.593 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.597+0000 7f8b3f68da80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 : hit (requested=0x17, cached=0x17) 2026-02-24T22:19:56.593 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.597+0000 7f8b3f68da80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 2026-02-24T22:19:56.593 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.597+0000 7f8b3f68da80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 info.flags=0x17 2026-02-24T22:19:56.594 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.597+0000 7f8b3f68da80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 to cache LRU end 2026-02-24T22:19:56.594 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.597+0000 7f8b3f68da80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:56.594 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.597+0000 7f8b3f68da80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535, ofs0, ns] 2026-02-24T22:19:56.594 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.597+0000 7f8b07fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959906 cookie 94150049876480 notifier 7343 bl.length()=977 2026-02-24T22:19:56.594 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.597+0000 7f8b07fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 info.flags=0x17 2026-02-24T22:19:56.594 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.597+0000 7f8b07fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 to cache LRU end 2026-02-24T22:19:56.594 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.597+0000 7f8b07fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:56.594 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.597+0000 7f8b3f68da80 20 handle_bi_update(): bucket=:abort-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535]): orig_sources= new_sources= 2026-02-24T22:19:56.594 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.597+0000 7f8b3f68da80 20 handle_bi_update(): bucket=:abort-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535]): potential sources added=[] removed=[] 2026-02-24T22:19:56.594 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.597+0000 7f8b3f68da80 20 handle_bi_update(): bucket=:abort-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535]): orig_dests= new_dests= 2026-02-24T22:19:56.594 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.597+0000 7f8b3f68da80 20 handle_bi_update(): bucket=:abort-at-do-reshard[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535]): potential dests added=[] removed=[] 2026-02-24T22:19:56.595 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.597+0000 7f8b3f68da80 1 execute INFO: reshard of bucket "abort-at-do-reshard" from 1 shards to 2 shards completed successfully 2026-02-24T22:19:56.598 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.601+0000 7f8b3f68da80 20 remove_watcher() i=3 2026-02-24T22:19:56.598 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.601+0000 7f8b3f68da80 2 removed watcher, disabling cache 2026-02-24T22:19:56.598 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.601+0000 7f8b3f68da80 20 remove_watcher() i=2 2026-02-24T22:19:56.598 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.601+0000 7f8b3f68da80 20 remove_watcher() i=1 2026-02-24T22:19:56.598 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.601+0000 7f8b3f68da80 20 remove_watcher() i=6 2026-02-24T22:19:56.598 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.601+0000 7f8b3f68da80 20 remove_watcher() i=4 2026-02-24T22:19:56.598 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.601+0000 7f8b3f68da80 20 remove_watcher() i=5 2026-02-24T22:19:56.599 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.601+0000 7f8b3f68da80 20 remove_watcher() i=0 2026-02-24T22:19:56.599 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.601+0000 7f8b3f68da80 20 remove_watcher() i=7 2026-02-24T22:19:56.605 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket abort-at-do-reshard 2026-02-24T22:19:56.620 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:19:56.621 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:19:56.637 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.641+0000 7f5bc703aa80 4 Realm: () 2026-02-24T22:19:56.637 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.641+0000 7f5bc703aa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:19:56.638 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.641+0000 7f5bc703aa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:19:56.638 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.641+0000 7f5bc703aa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:19:56.638 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.641+0000 7f5bc703aa80 20 zonegroup default 2026-02-24T22:19:56.638 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.641+0000 7f5bc703aa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:56.639 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.641+0000 7f5bc703aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:56.639 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.641+0000 7f5bc703aa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:56.640 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.641+0000 7f5bc703aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:56.640 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.641+0000 7f5bc703aa80 20 started sync module instance, tier type = 2026-02-24T22:19:56.640 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.641+0000 7f5bc703aa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:19:56.642 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.645+0000 7f5bc703aa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:19:56.642 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.645+0000 7f5bc703aa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:19:56.642 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.645+0000 7f5bc703aa80 5 note: GC not initialized 2026-02-24T22:19:56.642 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.645+0000 7f5b727e4640 20 reqs_thread_entry: start 2026-02-24T22:19:56.644 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.649+0000 7f5bbffff640 0 -- 10.20.193.46:0/1737199617 >> [v2:10.20.193.46:6826/3065971976,v1:10.20.193.46:6827/3065971976] conn(0x560e578f26f0 msgr2=0x560e578f2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:56.856 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:56.857+0000 7f5bbffff640 0 -- 10.20.193.46:0/1737199617 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x560e578f5860 msgr2=0x560e578dde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:57.089 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.093+0000 7f5bc703aa80 20 init_complete bucket index max shards: 1 2026-02-24T22:19:57.089 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.093+0000 7f5bc703aa80 20 Filter name: none 2026-02-24T22:19:57.089 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.093+0000 7f5b70fe1640 20 reqs_thread_entry: start 2026-02-24T22:19:57.089 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.093+0000 7f5bc703aa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:57.089 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.093+0000 7f5bc703aa80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:19:57.089 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.093+0000 7f5bc703aa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:57.090 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.093+0000 7f5bc703aa80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:19:57.090 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.093+0000 7f5bc703aa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:19:57.090 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.093+0000 7f5bc703aa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:57.090 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.093+0000 7f5bc703aa80 20 rados_obj.operate() r=0 bl.length=215 2026-02-24T22:19:57.091 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.093+0000 7f5bc703aa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:57.091 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.093+0000 7f5bc703aa80 20 rados_obj.operate() r=0 bl.length=445 2026-02-24T22:19:57.091 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.093+0000 7f5bc703aa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:19:57.102 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:57.102 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:19:57.102 INFO:tasks.workunit.client.0.trial046.stderr:bucket: abort-at-do-reshard id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535 num_objects: 19 size_kb: 1 num_shards: 2 2026-02-24T22:19:57.102 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:57.103 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.103 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.106 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.106 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.149 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.149 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.161 INFO:tasks.workunit.client.0.trial046.stderr:TEST: reshard bucket with EIO injected at logrecord_writes 2026-02-24T22:19:57.161 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:57.162 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.162 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.171 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.171 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.213 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.213 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.220 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.220 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.226 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.227 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.233 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.233 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.239 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.239 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.245 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.245 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.252 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.252 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.258 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.258 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.264 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.264 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.270 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.271 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.276 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.276 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.282 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.282 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.287 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.287 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.293 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.293 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.298 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.299 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.305 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.305 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.310 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.310 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.316 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.316 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.321 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.321 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.328 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.328 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.333 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-logrecord-writes 2026-02-24T22:19:57.348 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:19:57.348 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:19:57.365 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.369+0000 7fec5545da80 4 Realm: () 2026-02-24T22:19:57.365 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.369+0000 7fec5545da80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:19:57.365 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.369+0000 7fec5545da80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:19:57.365 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.369+0000 7fec5545da80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:19:57.366 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.369+0000 7fec5545da80 20 zonegroup default 2026-02-24T22:19:57.366 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.369+0000 7fec5545da80 20 rados->read ofs=0 len=0 2026-02-24T22:19:57.367 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.369+0000 7fec5545da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:57.367 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.369+0000 7fec5545da80 20 rados->read ofs=0 len=0 2026-02-24T22:19:57.368 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.369+0000 7fec5545da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:57.368 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.369+0000 7fec5545da80 20 started sync module instance, tier type = 2026-02-24T22:19:57.368 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.369+0000 7fec5545da80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:19:57.369 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.373+0000 7fec5545da80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:19:57.369 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.373+0000 7fec5545da80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:19:57.369 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.373+0000 7fec5545da80 5 note: GC not initialized 2026-02-24T22:19:57.369 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.373+0000 7fec017e2640 20 reqs_thread_entry: start 2026-02-24T22:19:57.373 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.377+0000 7fec53732640 0 -- 10.20.193.46:0/2446873518 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x5610670384e0 msgr2=0x56106702b750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:57.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.381+0000 7fec52f31640 0 -- 10.20.193.46:0/2446873518 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x561067033f60 msgr2=0x56106702ecd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:57.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.405+0000 7fec5545da80 20 init_complete bucket index max shards: 1 2026-02-24T22:19:57.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.405+0000 7fec5545da80 20 Filter name: none 2026-02-24T22:19:57.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.405+0000 7feb6b7fe640 20 reqs_thread_entry: start 2026-02-24T22:19:57.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.405+0000 7fec5545da80 20 rados->read ofs=0 len=0 2026-02-24T22:19:57.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.409+0000 7fec5545da80 20 rados_obj.operate() r=0 bl.length=221 2026-02-24T22:19:57.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.409+0000 7fec5545da80 20 rados->read ofs=0 len=0 2026-02-24T22:19:57.405 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.409+0000 7fec5545da80 20 rados_obj.operate() r=0 bl.length=451 2026-02-24T22:19:57.405 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.409+0000 7fec5545da80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:19:57.405 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.409+0000 7fec5545da80 20 rados->read ofs=0 len=0 2026-02-24T22:19:57.405 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.409+0000 7fec5545da80 20 rados_obj.operate() r=0 bl.length=221 2026-02-24T22:19:57.406 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.409+0000 7fec5545da80 20 rados->read ofs=0 len=0 2026-02-24T22:19:57.406 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.409+0000 7fec5545da80 20 rados_obj.operate() r=0 bl.length=451 2026-02-24T22:19:57.406 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.409+0000 7fec5545da80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:19:57.416 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:57.416 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:19:57.416 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-logrecord-writes id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:19:57.416 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:57.417 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket error-at-logrecord-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at logrecord_writes 2026-02-24T22:19:57.434 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:19:57.434 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:19:57.452 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.457+0000 7f5b5680da80 4 Realm: () 2026-02-24T22:19:57.453 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.457+0000 7f5b5680da80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:19:57.453 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.457+0000 7f5b5680da80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:19:57.453 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.457+0000 7f5b5680da80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:19:57.453 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.457+0000 7f5b5680da80 20 zonegroup default 2026-02-24T22:19:57.453 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.457+0000 7f5b5680da80 20 rados->read ofs=0 len=0 2026-02-24T22:19:57.454 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.457+0000 7f5b5680da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:57.455 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.457+0000 7f5b5680da80 20 rados->read ofs=0 len=0 2026-02-24T22:19:57.455 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.457+0000 7f5b5680da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:57.455 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.457+0000 7f5b5680da80 20 started sync module instance, tier type = 2026-02-24T22:19:57.455 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.457+0000 7f5b5680da80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:19:57.456 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.461+0000 7f5b54ae2640 0 -- 10.20.193.46:0/283489377 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55b36149e190 msgr2=0x55b3614be570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:57.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.461+0000 7f5b5680da80 20 add_watcher() i=6 2026-02-24T22:19:57.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.461+0000 7f5b5680da80 20 add_watcher() i=1 2026-02-24T22:19:57.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.461+0000 7f5b5680da80 20 add_watcher() i=5 2026-02-24T22:19:57.458 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.461+0000 7f5b5680da80 20 add_watcher() i=3 2026-02-24T22:19:57.458 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.461+0000 7f5b5680da80 20 add_watcher() i=2 2026-02-24T22:19:57.458 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.461+0000 7f5b5680da80 20 add_watcher() i=0 2026-02-24T22:19:57.458 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.461+0000 7f5b5680da80 20 add_watcher() i=4 2026-02-24T22:19:57.459 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.461+0000 7f5b5680da80 20 add_watcher() i=7 2026-02-24T22:19:57.459 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.461+0000 7f5b5680da80 2 all 8 watchers are set, enabling cache 2026-02-24T22:19:57.460 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.461+0000 7f5b5680da80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:19:57.460 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.461+0000 7f5b5680da80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:19:57.460 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.461+0000 7f5b5680da80 5 note: GC not initialized 2026-02-24T22:19:57.460 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.461+0000 7f5b017e2640 20 reqs_thread_entry: start 2026-02-24T22:19:57.460 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.465+0000 7f5b54ae2640 0 -- 10.20.193.46:0/3020927788 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55b3614c8780 msgr2=0x55b3614c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:19:57.702 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.705+0000 7f5b5680da80 20 init_complete bucket index max shards: 1 2026-02-24T22:19:57.702 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.705+0000 7f5b5680da80 20 Filter name: none 2026-02-24T22:19:57.702 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.705+0000 7f5a6b7fe640 20 reqs_thread_entry: start 2026-02-24T22:19:57.702 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.705+0000 7f5b5680da80 10 cache get: name=default.rgw.meta+root+error-at-logrecord-writes : miss 2026-02-24T22:19:57.703 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.705+0000 7f5b5680da80 20 rados->read ofs=0 len=0 2026-02-24T22:19:57.703 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.705+0000 7f5b5680da80 20 rados_obj.operate() r=0 bl.length=221 2026-02-24T22:19:57.703 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.705+0000 7f5b5680da80 10 cache put: name=default.rgw.meta+root+error-at-logrecord-writes info.flags=0x11 2026-02-24T22:19:57.703 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.705+0000 7f5b5680da80 10 adding default.rgw.meta+root+error-at-logrecord-writes to cache LRU end 2026-02-24T22:19:57.703 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.705+0000 7f5b5680da80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 : miss 2026-02-24T22:19:57.703 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.705+0000 7f5b5680da80 20 rados->read ofs=0 len=0 2026-02-24T22:19:57.704 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.705+0000 7f5b5680da80 20 rados_obj.operate() r=0 bl.length=451 2026-02-24T22:19:57.704 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.705+0000 7f5b5680da80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 info.flags=0x17 2026-02-24T22:19:57.704 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.705+0000 7f5b5680da80 10 adding default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 to cache LRU end 2026-02-24T22:19:57.704 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.705+0000 7f5b5680da80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:19:57.704 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.705+0000 7f5b5680da80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:57.704 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.705+0000 7f5b5680da80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 2026-02-24T22:19:57.707 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.709+0000 7f5b5680da80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 info.flags=0x17 2026-02-24T22:19:57.707 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.709+0000 7f5b5680da80 10 moving default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 to cache LRU end 2026-02-24T22:19:57.708 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.709+0000 7f5b5680da80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:57.708 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.709+0000 7f5b5680da80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536, ofs0, ns] 2026-02-24T22:19:57.708 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.709+0000 7f5b2affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959921 cookie 94228920004688 notifier 7394 bl.length()=1025 2026-02-24T22:19:57.708 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.709+0000 7f5b2affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 info.flags=0x17 2026-02-24T22:19:57.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.709+0000 7f5b2affd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 to cache LRU end 2026-02-24T22:19:57.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.709+0000 7f5b2affd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:57.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.713+0000 7f5b5680da80 20 handle_bi_update(): bucket=:error-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536]): orig_sources= new_sources= 2026-02-24T22:19:57.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.713+0000 7f5b5680da80 20 handle_bi_update(): bucket=:error-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536]): potential sources added=[] removed=[] 2026-02-24T22:19:57.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.713+0000 7f5b5680da80 20 handle_bi_update(): bucket=:error-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536]): orig_dests= new_dests= 2026-02-24T22:19:57.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.713+0000 7f5b5680da80 20 handle_bi_update(): bucket=:error-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536]): potential dests added=[] removed=[] 2026-02-24T22:19:57.710 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.713+0000 7f5b5680da80 -1 Injecting error=-5 at location=logrecord_writes 2026-02-24T22:19:57.710 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.713+0000 7f5b5680da80 0 ERROR: init_reshard failed to pause writes to the current index: (5) Input/output error 2026-02-24T22:19:57.710 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.713+0000 7f5b5680da80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 : hit (requested=0x17, cached=0x17) 2026-02-24T22:19:57.711 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.713+0000 7f5b5680da80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 2026-02-24T22:19:57.712 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.713+0000 7f5b5680da80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 info.flags=0x17 2026-02-24T22:19:57.712 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.713+0000 7f5b5680da80 10 moving default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 to cache LRU end 2026-02-24T22:19:57.712 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.713+0000 7f5b5680da80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:57.712 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.713+0000 7f5b5680da80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536, ofs0, ns] 2026-02-24T22:19:57.712 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.713+0000 7f5b2affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959922 cookie 94228920004688 notifier 7394 bl.length()=989 2026-02-24T22:19:57.712 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.713+0000 7f5b2affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 info.flags=0x17 2026-02-24T22:19:57.712 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.713+0000 7f5b2affd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 to cache LRU end 2026-02-24T22:19:57.712 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.713+0000 7f5b2affd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:57.713 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.717+0000 7f5b5680da80 20 handle_bi_update(): bucket=:error-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536]): orig_sources= new_sources= 2026-02-24T22:19:57.713 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.717+0000 7f5b5680da80 20 handle_bi_update(): bucket=:error-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536]): potential sources added=[] removed=[] 2026-02-24T22:19:57.713 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.717+0000 7f5b5680da80 20 handle_bi_update(): bucket=:error-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536]): orig_dests= new_dests= 2026-02-24T22:19:57.713 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.717+0000 7f5b5680da80 20 handle_bi_update(): bucket=:error-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536]): potential dests added=[] removed=[] 2026-02-24T22:19:57.716 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.717+0000 7f5b5680da80 20 remove_watcher() i=3 2026-02-24T22:19:57.716 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.717+0000 7f5b5680da80 2 removed watcher, disabling cache 2026-02-24T22:19:57.716 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.717+0000 7f5b5680da80 20 remove_watcher() i=0 2026-02-24T22:19:57.716 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.717+0000 7f5b5680da80 20 remove_watcher() i=4 2026-02-24T22:19:57.716 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.717+0000 7f5b5680da80 20 remove_watcher() i=5 2026-02-24T22:19:57.716 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.717+0000 7f5b5680da80 20 remove_watcher() i=7 2026-02-24T22:19:57.717 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.717+0000 7f5b5680da80 20 remove_watcher() i=2 2026-02-24T22:19:57.717 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.717+0000 7f5b5680da80 20 remove_watcher() i=6 2026-02-24T22:19:57.717 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.717+0000 7f5b5680da80 20 remove_watcher() i=1 2026-02-24T22:19:57.723 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-logrecord-writes 2026-02-24T22:19:57.738 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:19:57.738 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:19:57.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.757+0000 7f66e5669a80 4 Realm: () 2026-02-24T22:19:57.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.757+0000 7f66e5669a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:19:57.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.757+0000 7f66e5669a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:19:57.755 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.757+0000 7f66e5669a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:19:57.755 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.757+0000 7f66e5669a80 20 zonegroup default 2026-02-24T22:19:57.755 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.757+0000 7f66e5669a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:57.756 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.757+0000 7f66e5669a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:57.756 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.757+0000 7f66e5669a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:57.756 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.761+0000 7f66e5669a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:57.756 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.761+0000 7f66e5669a80 20 started sync module instance, tier type = 2026-02-24T22:19:57.757 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.761+0000 7f66e5669a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:19:57.758 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.761+0000 7f66e5669a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:19:57.758 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.761+0000 7f66e5669a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:19:57.758 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.761+0000 7f66e5669a80 5 note: GC not initialized 2026-02-24T22:19:57.758 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.761+0000 7f6690fe1640 20 reqs_thread_entry: start 2026-02-24T22:19:57.762 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.765+0000 7f66df7fe640 0 -- 10.20.193.46:0/2498542222 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55d7593a3010 msgr2=0x55d7593a96d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:57.768 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.773+0000 7f66deffd640 0 -- 10.20.193.46:0/2498542222 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55d759377510 msgr2=0x55d7593a5150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:57.790 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.793+0000 7f66e5669a80 20 init_complete bucket index max shards: 1 2026-02-24T22:19:57.790 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.793+0000 7f66e5669a80 20 Filter name: none 2026-02-24T22:19:57.790 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.793+0000 7f6602ffd640 20 reqs_thread_entry: start 2026-02-24T22:19:57.790 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.793+0000 7f66e5669a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:57.791 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.793+0000 7f66e5669a80 20 rados_obj.operate() r=0 bl.length=221 2026-02-24T22:19:57.791 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.793+0000 7f66e5669a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:57.793 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.797+0000 7f66e5669a80 20 rados_obj.operate() r=0 bl.length=451 2026-02-24T22:19:57.793 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.797+0000 7f66e5669a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:19:57.793 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.797+0000 7f66e5669a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:57.793 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.797+0000 7f66e5669a80 20 rados_obj.operate() r=0 bl.length=221 2026-02-24T22:19:57.793 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.797+0000 7f66e5669a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:57.793 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.797+0000 7f66e5669a80 20 rados_obj.operate() r=0 bl.length=451 2026-02-24T22:19:57.793 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.797+0000 7f66e5669a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:19:57.802 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:57.802 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:19:57.802 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-logrecord-writes id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:19:57.802 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:57.803 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.803 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.809 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:57.809 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:57.853 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket error-at-logrecord-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2026-02-24T22:19:57.869 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:19:57.869 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:19:57.886 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.889+0000 7feb20b01a80 4 Realm: () 2026-02-24T22:19:57.886 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.889+0000 7feb20b01a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:19:57.886 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.889+0000 7feb20b01a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:19:57.886 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.889+0000 7feb20b01a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:19:57.886 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.889+0000 7feb20b01a80 20 zonegroup default 2026-02-24T22:19:57.886 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.889+0000 7feb20b01a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:57.887 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.889+0000 7feb20b01a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:57.887 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.889+0000 7feb20b01a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:57.888 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.889+0000 7feb20b01a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:57.888 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.889+0000 7feb20b01a80 20 started sync module instance, tier type = 2026-02-24T22:19:57.888 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.889+0000 7feb20b01a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:19:57.888 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.893+0000 7feb1e5d5640 0 -- 10.20.193.46:0/1797868387 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x5565d4615bf0 msgr2=0x5565d4636070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:19:57.889 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.893+0000 7feb20b01a80 20 add_watcher() i=3 2026-02-24T22:19:57.890 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.893+0000 7feb20b01a80 20 add_watcher() i=1 2026-02-24T22:19:57.890 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.893+0000 7feb20b01a80 20 add_watcher() i=0 2026-02-24T22:19:57.890 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.893+0000 7feb20b01a80 20 add_watcher() i=6 2026-02-24T22:19:57.890 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.893+0000 7feb20b01a80 20 add_watcher() i=5 2026-02-24T22:19:57.890 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.893+0000 7feb20b01a80 20 add_watcher() i=4 2026-02-24T22:19:57.890 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:57.893+0000 7feb20b01a80 20 add_watcher() i=2 2026-02-24T22:19:58.095 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.097+0000 7feb20b01a80 20 add_watcher() i=7 2026-02-24T22:19:58.095 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.097+0000 7feb20b01a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:19:58.097 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.101+0000 7feb20b01a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:19:58.097 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.101+0000 7feb20b01a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:19:58.097 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.101+0000 7feb20b01a80 5 note: GC not initialized 2026-02-24T22:19:58.097 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.101+0000 7fea377fe640 20 reqs_thread_entry: start 2026-02-24T22:19:58.098 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.101+0000 7feb1edd6640 0 -- 10.20.193.46:0/2820349929 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5565d45e5100 msgr2=0x5565d45d4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:58.337 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.341+0000 7feb20b01a80 20 init_complete bucket index max shards: 1 2026-02-24T22:19:58.337 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.341+0000 7feb20b01a80 20 Filter name: none 2026-02-24T22:19:58.337 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.341+0000 7fea35ffb640 20 reqs_thread_entry: start 2026-02-24T22:19:58.337 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.341+0000 7feb20b01a80 10 cache get: name=default.rgw.meta+root+error-at-logrecord-writes : miss 2026-02-24T22:19:58.337 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.341+0000 7feb20b01a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.338 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.341+0000 7feb20b01a80 20 rados_obj.operate() r=0 bl.length=221 2026-02-24T22:19:58.338 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.341+0000 7feb20b01a80 10 cache put: name=default.rgw.meta+root+error-at-logrecord-writes info.flags=0x11 2026-02-24T22:19:58.338 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.341+0000 7feb20b01a80 10 adding default.rgw.meta+root+error-at-logrecord-writes to cache LRU end 2026-02-24T22:19:58.338 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.341+0000 7feb20b01a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 : miss 2026-02-24T22:19:58.338 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.341+0000 7feb20b01a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.338 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.341+0000 7feb20b01a80 20 rados_obj.operate() r=0 bl.length=451 2026-02-24T22:19:58.339 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.341+0000 7feb20b01a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 info.flags=0x17 2026-02-24T22:19:58.339 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.341+0000 7feb20b01a80 10 adding default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 to cache LRU end 2026-02-24T22:19:58.339 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.341+0000 7feb20b01a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:19:58.339 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.341+0000 7feb20b01a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:58.339 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.341+0000 7feb20b01a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 2026-02-24T22:19:58.342 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.345+0000 7feb20b01a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 info.flags=0x17 2026-02-24T22:19:58.342 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.345+0000 7feb20b01a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 to cache LRU end 2026-02-24T22:19:58.342 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.345+0000 7feb20b01a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:58.342 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.345+0000 7feb20b01a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536, ofs0, ns] 2026-02-24T22:19:58.342 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.345+0000 7feaf97fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959923 cookie 93895843346432 notifier 7944 bl.length()=1025 2026-02-24T22:19:58.342 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.345+0000 7feaf97fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 info.flags=0x17 2026-02-24T22:19:58.342 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.345+0000 7feaf97fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 to cache LRU end 2026-02-24T22:19:58.343 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.345+0000 7feaf97fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:58.343 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.345+0000 7feb20b01a80 20 handle_bi_update(): bucket=:error-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536]): orig_sources= new_sources= 2026-02-24T22:19:58.343 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.345+0000 7feb20b01a80 20 handle_bi_update(): bucket=:error-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536]): potential sources added=[] removed=[] 2026-02-24T22:19:58.343 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.345+0000 7feb20b01a80 20 handle_bi_update(): bucket=:error-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536]): orig_dests= new_dests= 2026-02-24T22:19:58.343 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.345+0000 7feb20b01a80 20 handle_bi_update(): bucket=:error-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536]): potential dests added=[] removed=[] 2026-02-24T22:19:58.344 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.345+0000 7feb20b01a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536.1.0 2026-02-24T22:19:58.344 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.345+0000 7feb20b01a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536.1.0 2026-02-24T22:19:58.344 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.345+0000 7feb20b01a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536.1.1 2026-02-24T22:19:58.344 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.345+0000 7feb20b01a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536.1.1 2026-02-24T22:19:58.344 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.345+0000 7feb20b01a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536.0 2026-02-24T22:19:58.345 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.349+0000 7feb20b01a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 : hit (requested=0x17, cached=0x17) 2026-02-24T22:19:58.346 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.349+0000 7feb20b01a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 2026-02-24T22:19:58.347 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.349+0000 7feb20b01a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 info.flags=0x17 2026-02-24T22:19:58.347 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.349+0000 7feb20b01a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 to cache LRU end 2026-02-24T22:19:58.347 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.349+0000 7feb20b01a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:58.347 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.349+0000 7feb20b01a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536, ofs0, ns] 2026-02-24T22:19:58.347 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.349+0000 7feaf97fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959924 cookie 93895843346432 notifier 7944 bl.length()=1025 2026-02-24T22:19:58.347 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.349+0000 7feaf97fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 info.flags=0x17 2026-02-24T22:19:58.347 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.349+0000 7feaf97fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 to cache LRU end 2026-02-24T22:19:58.347 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.349+0000 7feaf97fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:58.348 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.349+0000 7feb20b01a80 20 handle_bi_update(): bucket=:error-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536]): orig_sources= new_sources= 2026-02-24T22:19:58.348 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.349+0000 7feb20b01a80 20 handle_bi_update(): bucket=:error-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536]): potential sources added=[] removed=[] 2026-02-24T22:19:58.348 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.349+0000 7feb20b01a80 20 handle_bi_update(): bucket=:error-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536]): orig_dests= new_dests= 2026-02-24T22:19:58.348 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.349+0000 7feb20b01a80 20 handle_bi_update(): bucket=:error-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536]): potential dests added=[] removed=[] 2026-02-24T22:19:58.348 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.349+0000 7feb20b01a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536.0 2026-02-24T22:19:58.348 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.349+0000 7feb20b01a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 : hit (requested=0x17, cached=0x17) 2026-02-24T22:19:58.349 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.349+0000 7feb20b01a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 2026-02-24T22:19:58.349 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.353+0000 7feb20b01a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 info.flags=0x17 2026-02-24T22:19:58.350 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.353+0000 7feb20b01a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 to cache LRU end 2026-02-24T22:19:58.350 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.353+0000 7feb20b01a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:58.350 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.353+0000 7feb20b01a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536, ofs0, ns] 2026-02-24T22:19:58.350 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.353+0000 7feaf9ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959925 cookie 93895843346432 notifier 7944 bl.length()=989 2026-02-24T22:19:58.350 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.353+0000 7feaf9ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 info.flags=0x17 2026-02-24T22:19:58.350 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.353+0000 7feaf9ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 to cache LRU end 2026-02-24T22:19:58.350 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.353+0000 7feaf9ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:58.350 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.353+0000 7feb20b01a80 20 handle_bi_update(): bucket=:error-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536]): orig_sources= new_sources= 2026-02-24T22:19:58.350 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.353+0000 7feb20b01a80 20 handle_bi_update(): bucket=:error-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536]): potential sources added=[] removed=[] 2026-02-24T22:19:58.350 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.353+0000 7feb20b01a80 20 handle_bi_update(): bucket=:error-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536]): orig_dests= new_dests= 2026-02-24T22:19:58.350 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.353+0000 7feb20b01a80 20 handle_bi_update(): bucket=:error-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536]): potential dests added=[] removed=[] 2026-02-24T22:19:58.351 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.353+0000 7feb20b01a80 1 execute INFO: reshard of bucket "error-at-logrecord-writes" from 1 shards to 2 shards completed successfully 2026-02-24T22:19:58.354 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.357+0000 7feb20b01a80 20 remove_watcher() i=3 2026-02-24T22:19:58.354 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.357+0000 7feb20b01a80 2 removed watcher, disabling cache 2026-02-24T22:19:58.354 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.357+0000 7feb20b01a80 20 remove_watcher() i=2 2026-02-24T22:19:58.354 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.357+0000 7feb20b01a80 20 remove_watcher() i=5 2026-02-24T22:19:58.354 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.357+0000 7feb20b01a80 20 remove_watcher() i=1 2026-02-24T22:19:58.354 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.357+0000 7feb20b01a80 20 remove_watcher() i=4 2026-02-24T22:19:58.354 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.357+0000 7feb20b01a80 20 remove_watcher() i=0 2026-02-24T22:19:58.355 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.357+0000 7feb20b01a80 20 remove_watcher() i=6 2026-02-24T22:19:58.355 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.357+0000 7feb20b01a80 20 remove_watcher() i=7 2026-02-24T22:19:58.361 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-logrecord-writes 2026-02-24T22:19:58.376 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:19:58.376 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:19:58.394 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.397+0000 7f0ca1921a80 4 Realm: () 2026-02-24T22:19:58.394 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.397+0000 7f0ca1921a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:19:58.394 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.397+0000 7f0ca1921a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:19:58.394 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.397+0000 7f0ca1921a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:19:58.394 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.397+0000 7f0ca1921a80 20 zonegroup default 2026-02-24T22:19:58.394 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.397+0000 7f0ca1921a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.397+0000 7f0ca1921a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:58.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.397+0000 7f0ca1921a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.401+0000 7f0ca1921a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:58.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.401+0000 7f0ca1921a80 20 started sync module instance, tier type = 2026-02-24T22:19:58.397 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.401+0000 7f0ca1921a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:19:58.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.401+0000 7f0ca1921a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:19:58.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.401+0000 7f0ca1921a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:19:58.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.401+0000 7f0ca1921a80 5 note: GC not initialized 2026-02-24T22:19:58.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.401+0000 7f0c4cfe1640 20 reqs_thread_entry: start 2026-02-24T22:19:58.401 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.405+0000 7f0c9b7fe640 0 -- 10.20.193.46:0/1791439347 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x56439d3197a0 msgr2=0x56439d30ff90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:58.407 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.409+0000 7f0c9a7fc640 0 -- 10.20.193.46:0/1791439347 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x56439d312f60 msgr2=0x56439d2ff080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:58.431 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.433+0000 7f0ca1921a80 20 init_complete bucket index max shards: 1 2026-02-24T22:19:58.432 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.433+0000 7f0ca1921a80 20 Filter name: none 2026-02-24T22:19:58.432 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.433+0000 7f0bbeffd640 20 reqs_thread_entry: start 2026-02-24T22:19:58.432 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.433+0000 7f0ca1921a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.432 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.437+0000 7f0ca1921a80 20 rados_obj.operate() r=0 bl.length=221 2026-02-24T22:19:58.433 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.437+0000 7f0ca1921a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.434 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.437+0000 7f0ca1921a80 20 rados_obj.operate() r=0 bl.length=451 2026-02-24T22:19:58.434 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.437+0000 7f0ca1921a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:19:58.434 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.437+0000 7f0ca1921a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.434 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.437+0000 7f0ca1921a80 20 rados_obj.operate() r=0 bl.length=221 2026-02-24T22:19:58.434 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.437+0000 7f0ca1921a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.434 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.437+0000 7f0ca1921a80 20 rados_obj.operate() r=0 bl.length=451 2026-02-24T22:19:58.435 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.437+0000 7f0ca1921a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:19:58.445 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.445 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:19:58.445 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-logrecord-writes id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536 num_objects: 19 size_kb: 1 num_shards: 2 2026-02-24T22:19:58.445 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.446 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:58.446 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:58.449 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:58.449 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:58.493 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:58.493 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:58.504 INFO:tasks.workunit.client.0.trial046.stderr:TEST: reshard bucket with abort at logrecord_writes 2026-02-24T22:19:58.504 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.505 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:58.505 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:58.514 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:58.514 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:58.557 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:58.557 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:58.563 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:58.563 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:58.569 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:58.569 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:58.576 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:58.576 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:58.582 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:58.582 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:58.588 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:58.588 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:58.594 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:58.595 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:58.600 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:58.600 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:58.607 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:58.607 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:58.613 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:58.613 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:58.619 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:58.619 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:58.625 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:58.625 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:58.631 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:58.631 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:58.637 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:58.637 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:58.643 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:58.643 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:58.648 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:58.648 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:58.654 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:58.654 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:58.660 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:58.660 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:58.666 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:58.666 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:58.672 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:58.672 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:58.677 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket abort-at-logrecord-writes 2026-02-24T22:19:58.692 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:19:58.692 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:19:58.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.713+0000 7f11336cfa80 4 Realm: () 2026-02-24T22:19:58.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.713+0000 7f11336cfa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:19:58.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.713+0000 7f11336cfa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:19:58.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.713+0000 7f11336cfa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:19:58.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.713+0000 7f11336cfa80 20 zonegroup default 2026-02-24T22:19:58.710 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.713+0000 7f11336cfa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.711 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.713+0000 7f11336cfa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:58.711 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.713+0000 7f11336cfa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.712 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.713+0000 7f11336cfa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:58.712 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.713+0000 7f11336cfa80 20 started sync module instance, tier type = 2026-02-24T22:19:58.712 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.713+0000 7f11336cfa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:19:58.713 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.717+0000 7f11336cfa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:19:58.714 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.717+0000 7f11336cfa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:19:58.714 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.717+0000 7f11336cfa80 5 note: GC not initialized 2026-02-24T22:19:58.714 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.717+0000 7f10defe5640 20 reqs_thread_entry: start 2026-02-24T22:19:58.717 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.721+0000 7f11319a4640 0 -- 10.20.193.46:0/1076915861 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x5624cbf40510 msgr2=0x5624cbf33780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:19:58.723 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.725+0000 7f11311a3640 0 -- 10.20.193.46:0/1076915861 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5624cbf3bf90 msgr2=0x5624cbf36d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:58.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.753+0000 7f11336cfa80 20 init_complete bucket index max shards: 1 2026-02-24T22:19:58.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.753+0000 7f11336cfa80 20 Filter name: none 2026-02-24T22:19:58.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.753+0000 7f10dd7e2640 20 reqs_thread_entry: start 2026-02-24T22:19:58.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.753+0000 7f11336cfa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.750 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.753+0000 7f11336cfa80 20 rados_obj.operate() r=0 bl.length=221 2026-02-24T22:19:58.750 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.753+0000 7f11336cfa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.751 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.753+0000 7f11336cfa80 20 rados_obj.operate() r=0 bl.length=451 2026-02-24T22:19:58.751 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.753+0000 7f11336cfa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:19:58.751 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.753+0000 7f11336cfa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.751 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.753+0000 7f11336cfa80 20 rados_obj.operate() r=0 bl.length=221 2026-02-24T22:19:58.751 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.753+0000 7f11336cfa80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.752 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.753+0000 7f11336cfa80 20 rados_obj.operate() r=0 bl.length=451 2026-02-24T22:19:58.752 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.753+0000 7f11336cfa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:19:58.761 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.761 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:19:58.761 INFO:tasks.workunit.client.0.trial046.stderr:bucket: abort-at-logrecord-writes id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:19:58.761 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.762 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket abort-at-logrecord-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at logrecord_writes 2026-02-24T22:19:58.778 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:19:58.778 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:19:58.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.801+0000 7f01615dda80 4 Realm: () 2026-02-24T22:19:58.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.801+0000 7f01615dda80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:19:58.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.801+0000 7f01615dda80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:19:58.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.801+0000 7f01615dda80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:19:58.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.801+0000 7f01615dda80 20 zonegroup default 2026-02-24T22:19:58.798 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.801+0000 7f01615dda80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.798 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.801+0000 7f01615dda80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:58.799 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.801+0000 7f01615dda80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.799 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.801+0000 7f01615dda80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:58.799 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.801+0000 7f01615dda80 20 started sync module instance, tier type = 2026-02-24T22:19:58.799 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.801+0000 7f01615dda80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:19:58.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.801+0000 7f015a7fc640 0 -- 10.20.193.46:0/135678098 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x5619914cb510 msgr2=0x5619914eb9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:58.801 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.805+0000 7f01615dda80 20 add_watcher() i=3 2026-02-24T22:19:58.801 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.805+0000 7f01615dda80 20 add_watcher() i=1 2026-02-24T22:19:58.801 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.805+0000 7f01615dda80 20 add_watcher() i=5 2026-02-24T22:19:58.801 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.805+0000 7f01615dda80 20 add_watcher() i=0 2026-02-24T22:19:58.801 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.805+0000 7f01615dda80 20 add_watcher() i=6 2026-02-24T22:19:58.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.805+0000 7f01615dda80 20 add_watcher() i=4 2026-02-24T22:19:58.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.805+0000 7f015affd640 0 -- 10.20.193.46:0/135678098 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x561991697090 msgr2=0x5619916b7510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:58.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.805+0000 7f01615dda80 20 add_watcher() i=7 2026-02-24T22:19:58.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.805+0000 7f01615dda80 20 add_watcher() i=2 2026-02-24T22:19:58.803 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.805+0000 7f01615dda80 2 all 8 watchers are set, enabling cache 2026-02-24T22:19:58.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.805+0000 7f01615dda80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:19:58.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.805+0000 7f01615dda80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:19:58.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.805+0000 7f01615dda80 5 note: GC not initialized 2026-02-24T22:19:58.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.805+0000 7f0077fff640 20 reqs_thread_entry: start 2026-02-24T22:19:58.836 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.837+0000 7f01615dda80 20 init_complete bucket index max shards: 1 2026-02-24T22:19:58.836 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.837+0000 7f01615dda80 20 Filter name: none 2026-02-24T22:19:58.836 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.841+0000 7f00767fc640 20 reqs_thread_entry: start 2026-02-24T22:19:58.836 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.841+0000 7f01615dda80 10 cache get: name=default.rgw.meta+root+abort-at-logrecord-writes : miss 2026-02-24T22:19:58.836 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.841+0000 7f01615dda80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.841+0000 7f01615dda80 20 rados_obj.operate() r=0 bl.length=221 2026-02-24T22:19:58.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.841+0000 7f01615dda80 10 cache put: name=default.rgw.meta+root+abort-at-logrecord-writes info.flags=0x11 2026-02-24T22:19:58.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.841+0000 7f01615dda80 10 adding default.rgw.meta+root+abort-at-logrecord-writes to cache LRU end 2026-02-24T22:19:58.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.841+0000 7f01615dda80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 : miss 2026-02-24T22:19:58.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.841+0000 7f01615dda80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.838 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.841+0000 7f01615dda80 20 rados_obj.operate() r=0 bl.length=451 2026-02-24T22:19:58.838 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.841+0000 7f01615dda80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 info.flags=0x17 2026-02-24T22:19:58.838 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.841+0000 7f01615dda80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 to cache LRU end 2026-02-24T22:19:58.838 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.841+0000 7f01615dda80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:19:58.838 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.841+0000 7f01615dda80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:58.838 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.841+0000 7f01615dda80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 2026-02-24T22:19:58.842 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.845+0000 7f01615dda80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 info.flags=0x17 2026-02-24T22:19:58.842 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.845+0000 7f01615dda80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 to cache LRU end 2026-02-24T22:19:58.842 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.845+0000 7f01615dda80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:58.842 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.845+0000 7f01615dda80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537, ofs0, ns] 2026-02-24T22:19:58.842 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.845+0000 7f013a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961388 cookie 94667813368912 notifier 7478 bl.length()=1025 2026-02-24T22:19:58.842 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.845+0000 7f013a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 info.flags=0x17 2026-02-24T22:19:58.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.845+0000 7f013a7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 to cache LRU end 2026-02-24T22:19:58.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.845+0000 7f013a7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:58.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.845+0000 7f01615dda80 20 handle_bi_update(): bucket=:abort-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537]): orig_sources= new_sources= 2026-02-24T22:19:58.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.845+0000 7f01615dda80 20 handle_bi_update(): bucket=:abort-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537]): potential sources added=[] removed=[] 2026-02-24T22:19:58.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.845+0000 7f01615dda80 20 handle_bi_update(): bucket=:abort-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537]): orig_dests= new_dests= 2026-02-24T22:19:58.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.845+0000 7f01615dda80 20 handle_bi_update(): bucket=:abort-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537]): potential dests added=[] removed=[] 2026-02-24T22:19:58.843 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f01615dda80 time 2026-02-24T22:19:58.850383+0000 2026-02-24T22:19:58.843 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:19:58.844 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:19:58.844 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f0164a1feb1] 2026-02-24T22:19:58.844 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x56197c7bcb9f] 2026-02-24T22:19:58.844 INFO:tasks.workunit.client.0.trial046.stderr: 3: radosgw-admin(+0x1714451) [0x56197d9cb451] 2026-02-24T22:19:58.844 INFO:tasks.workunit.client.0.trial046.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x56197cf28202] 2026-02-24T22:19:58.844 INFO:tasks.workunit.client.0.trial046.stderr: 5: main() 2026-02-24T22:19:58.845 INFO:tasks.workunit.client.0.trial046.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f0163d1fd90] 2026-02-24T22:19:58.845 INFO:tasks.workunit.client.0.trial046.stderr: 7: __libc_start_main() 2026-02-24T22:19:58.845 INFO:tasks.workunit.client.0.trial046.stderr: 8: _start() 2026-02-24T22:19:58.845 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.845+0000 7f01615dda80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f01615dda80 time 2026-02-24T22:19:58.850383+0000 2026-02-24T22:19:58.845 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:19:58.845 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.845 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:19:58.845 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f0164a1feb1] 2026-02-24T22:19:58.845 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x56197c7bcb9f] 2026-02-24T22:19:58.845 INFO:tasks.workunit.client.0.trial046.stderr: 3: radosgw-admin(+0x1714451) [0x56197d9cb451] 2026-02-24T22:19:58.845 INFO:tasks.workunit.client.0.trial046.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x56197cf28202] 2026-02-24T22:19:58.845 INFO:tasks.workunit.client.0.trial046.stderr: 5: main() 2026-02-24T22:19:58.846 INFO:tasks.workunit.client.0.trial046.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f0163d1fd90] 2026-02-24T22:19:58.846 INFO:tasks.workunit.client.0.trial046.stderr: 7: __libc_start_main() 2026-02-24T22:19:58.846 INFO:tasks.workunit.client.0.trial046.stderr: 8: _start() 2026-02-24T22:19:58.846 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.846 INFO:tasks.workunit.client.0.trial046.stderr:*** Caught signal (Aborted) ** 2026-02-24T22:19:58.846 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f01615dda80 thread_name:radosgw-admin 2026-02-24T22:19:58.880 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:19:58.880 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f0163d38520] 2026-02-24T22:19:58.880 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:19:58.880 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:19:58.880 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:19:58.880 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f0164a1ff10] 2026-02-24T22:19:58.880 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x56197c7bcb9f] 2026-02-24T22:19:58.880 INFO:tasks.workunit.client.0.trial046.stderr: 7: radosgw-admin(+0x1714451) [0x56197d9cb451] 2026-02-24T22:19:58.880 INFO:tasks.workunit.client.0.trial046.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x56197cf28202] 2026-02-24T22:19:58.880 INFO:tasks.workunit.client.0.trial046.stderr: 9: main() 2026-02-24T22:19:58.880 INFO:tasks.workunit.client.0.trial046.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f0163d1fd90] 2026-02-24T22:19:58.880 INFO:tasks.workunit.client.0.trial046.stderr: 11: __libc_start_main() 2026-02-24T22:19:58.880 INFO:tasks.workunit.client.0.trial046.stderr: 12: _start() 2026-02-24T22:19:58.881 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.849+0000 7f01615dda80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:19:58.881 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f01615dda80 thread_name:radosgw-admin 2026-02-24T22:19:58.881 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.881 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:19:58.881 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f0163d38520] 2026-02-24T22:19:58.881 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:19:58.881 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:19:58.881 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:19:58.881 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f0164a1ff10] 2026-02-24T22:19:58.881 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x56197c7bcb9f] 2026-02-24T22:19:58.881 INFO:tasks.workunit.client.0.trial046.stderr: 7: radosgw-admin(+0x1714451) [0x56197d9cb451] 2026-02-24T22:19:58.881 INFO:tasks.workunit.client.0.trial046.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x56197cf28202] 2026-02-24T22:19:58.881 INFO:tasks.workunit.client.0.trial046.stderr: 9: main() 2026-02-24T22:19:58.881 INFO:tasks.workunit.client.0.trial046.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f0163d1fd90] 2026-02-24T22:19:58.881 INFO:tasks.workunit.client.0.trial046.stderr: 11: __libc_start_main() 2026-02-24T22:19:58.881 INFO:tasks.workunit.client.0.trial046.stderr: 12: _start() 2026-02-24T22:19:58.882 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:19:58.882 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.882 INFO:tasks.workunit.client.0.trial046.stderr:--- begin dump of recent events --- 2026-02-24T22:19:58.882 INFO:tasks.workunit.client.0.trial046.stderr: -499> 2026-02-24T22:19:58.785+0000 7f01615dda80 10 monclient(hunting): authenticate will time out at 1968.895734s 2026-02-24T22:19:58.882 INFO:tasks.workunit.client.0.trial046.stderr: -498> 2026-02-24T22:19:58.785+0000 7f015a7fc640 10 monclient(hunting): get_auth_request con 0x561991176350 auth_method 0 2026-02-24T22:19:58.882 INFO:tasks.workunit.client.0.trial046.stderr: -497> 2026-02-24T22:19:58.785+0000 7f015a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:58.882 INFO:tasks.workunit.client.0.trial046.stderr: -496> 2026-02-24T22:19:58.785+0000 7f015a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:58.882 INFO:tasks.workunit.client.0.trial046.stderr: -495> 2026-02-24T22:19:58.785+0000 7f015a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:58.882 INFO:tasks.workunit.client.0.trial046.stderr: -494> 2026-02-24T22:19:58.785+0000 7f015b7fe640 10 monclient(hunting): get_auth_request con 0x56199122fd00 auth_method 0 2026-02-24T22:19:58.882 INFO:tasks.workunit.client.0.trial046.stderr: -493> 2026-02-24T22:19:58.785+0000 7f015b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:58.882 INFO:tasks.workunit.client.0.trial046.stderr: -492> 2026-02-24T22:19:58.785+0000 7f015b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:58.882 INFO:tasks.workunit.client.0.trial046.stderr: -491> 2026-02-24T22:19:58.785+0000 7f015b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:58.882 INFO:tasks.workunit.client.0.trial046.stderr: -490> 2026-02-24T22:19:58.785+0000 7f015a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:58.882 INFO:tasks.workunit.client.0.trial046.stderr: -489> 2026-02-24T22:19:58.785+0000 7f015a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:58.882 INFO:tasks.workunit.client.0.trial046.stderr: -488> 2026-02-24T22:19:58.785+0000 7f015a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:58.882 INFO:tasks.workunit.client.0.trial046.stderr: -487> 2026-02-24T22:19:58.785+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:58.882 INFO:tasks.workunit.client.0.trial046.stderr: -486> 2026-02-24T22:19:58.785+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:58.882 INFO:tasks.workunit.client.0.trial046.stderr: -485> 2026-02-24T22:19:58.785+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:58.882 INFO:tasks.workunit.client.0.trial046.stderr: -484> 2026-02-24T22:19:58.785+0000 7f015affd640 10 monclient(hunting): get_auth_request con 0x7f01440030d0 auth_method 0 2026-02-24T22:19:58.883 INFO:tasks.workunit.client.0.trial046.stderr: -483> 2026-02-24T22:19:58.785+0000 7f015affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:58.883 INFO:tasks.workunit.client.0.trial046.stderr: -482> 2026-02-24T22:19:58.785+0000 7f015affd640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:58.883 INFO:tasks.workunit.client.0.trial046.stderr: -481> 2026-02-24T22:19:58.785+0000 7f015affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:58.883 INFO:tasks.workunit.client.0.trial046.stderr: -480> 2026-02-24T22:19:58.785+0000 7f015a7fc640 10 monclient(hunting): handle_auth_done global_id 7466 payload 274 2026-02-24T22:19:58.883 INFO:tasks.workunit.client.0.trial046.stderr: -479> 2026-02-24T22:19:58.785+0000 7f015a7fc640 10 monclient: _finish_hunting 0 2026-02-24T22:19:58.883 INFO:tasks.workunit.client.0.trial046.stderr: -478> 2026-02-24T22:19:58.785+0000 7f015a7fc640 1 monclient: found mon.noname-c 2026-02-24T22:19:58.883 INFO:tasks.workunit.client.0.trial046.stderr: -477> 2026-02-24T22:19:58.785+0000 7f015a7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:19:58.883 INFO:tasks.workunit.client.0.trial046.stderr: -476> 2026-02-24T22:19:58.789+0000 7f01497fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:58.883 INFO:tasks.workunit.client.0.trial046.stderr: -475> 2026-02-24T22:19:58.789+0000 7f01497fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:19:58.883 INFO:tasks.workunit.client.0.trial046.stderr: -474> 2026-02-24T22:19:58.789+0000 7f01497fa640 10 monclient: dump: 2026-02-24T22:19:58.883 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:58.883 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:58.883 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.883 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.883 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:58.883 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.883 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:58.883 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:58.883 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:58.883 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.884 INFO:tasks.workunit.client.0.trial046.stderr: -473> 2026-02-24T22:19:58.789+0000 7f01497fa640 10 monclient: _finish_auth 0 2026-02-24T22:19:58.884 INFO:tasks.workunit.client.0.trial046.stderr: -472> 2026-02-24T22:19:58.789+0000 7f01497fa640 10 monclient: _check_auth_tickets 2026-02-24T22:19:58.884 INFO:tasks.workunit.client.0.trial046.stderr: -471> 2026-02-24T22:19:58.789+0000 7f01497fa640 10 monclient: handle_config config(0 keys) 2026-02-24T22:19:58.884 INFO:tasks.workunit.client.0.trial046.stderr: -470> 2026-02-24T22:19:58.789+0000 7f01497fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:58.884 INFO:tasks.workunit.client.0.trial046.stderr: -469> 2026-02-24T22:19:58.789+0000 7f01497fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:19:58.884 INFO:tasks.workunit.client.0.trial046.stderr: -468> 2026-02-24T22:19:58.789+0000 7f01497fa640 10 monclient: dump: 2026-02-24T22:19:58.884 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:58.884 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:58.884 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.884 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.884 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:58.884 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.884 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:58.884 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:58.884 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:58.884 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.884 INFO:tasks.workunit.client.0.trial046.stderr: -467> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 monclient: authenticate success, global_id 7466 2026-02-24T22:19:58.884 INFO:tasks.workunit.client.0.trial046.stderr: -466> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient: get_monmap_and_config success 2026-02-24T22:19:58.884 INFO:tasks.workunit.client.0.trial046.stderr: -465> 2026-02-24T22:19:58.789+0000 7f01615dda80 4 set_mon_vals no callback set 2026-02-24T22:19:58.885 INFO:tasks.workunit.client.0.trial046.stderr: -464> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient: shutdown 2026-02-24T22:19:58.885 INFO:tasks.workunit.client.0.trial046.stderr: -463> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 asok(0x561991042f00) unregister_commands rotate-key 2026-02-24T22:19:58.885 INFO:tasks.workunit.client.0.trial046.stderr: -462> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient: build_initial_monmap 2026-02-24T22:19:58.885 INFO:tasks.workunit.client.0.trial046.stderr: -461> 2026-02-24T22:19:58.789+0000 7f01615dda80 1 build_initial for_mkfs: 0 2026-02-24T22:19:58.885 INFO:tasks.workunit.client.0.trial046.stderr: -460> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient: monmap: 2026-02-24T22:19:58.885 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:19:58.885 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:19:58.885 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:19:58.885 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:19:58.885 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:19:58.885 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.885 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:19:58.885 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:19:58.885 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:19:58.885 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.885 INFO:tasks.workunit.client.0.trial046.stderr: -459> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding auth protocol: cephx 2026-02-24T22:19:58.885 INFO:tasks.workunit.client.0.trial046.stderr: -458> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding auth protocol: cephx 2026-02-24T22:19:58.885 INFO:tasks.workunit.client.0.trial046.stderr: -457> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding auth protocol: cephx 2026-02-24T22:19:58.885 INFO:tasks.workunit.client.0.trial046.stderr: -456> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding con mode: secure 2026-02-24T22:19:58.885 INFO:tasks.workunit.client.0.trial046.stderr: -455> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding con mode: crc 2026-02-24T22:19:58.885 INFO:tasks.workunit.client.0.trial046.stderr: -454> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding con mode: secure 2026-02-24T22:19:58.886 INFO:tasks.workunit.client.0.trial046.stderr: -453> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding con mode: crc 2026-02-24T22:19:58.886 INFO:tasks.workunit.client.0.trial046.stderr: -452> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding con mode: secure 2026-02-24T22:19:58.886 INFO:tasks.workunit.client.0.trial046.stderr: -451> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding con mode: crc 2026-02-24T22:19:58.886 INFO:tasks.workunit.client.0.trial046.stderr: -450> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding con mode: crc 2026-02-24T22:19:58.886 INFO:tasks.workunit.client.0.trial046.stderr: -449> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding con mode: secure 2026-02-24T22:19:58.886 INFO:tasks.workunit.client.0.trial046.stderr: -448> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding con mode: crc 2026-02-24T22:19:58.886 INFO:tasks.workunit.client.0.trial046.stderr: -447> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding con mode: secure 2026-02-24T22:19:58.886 INFO:tasks.workunit.client.0.trial046.stderr: -446> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding con mode: crc 2026-02-24T22:19:58.886 INFO:tasks.workunit.client.0.trial046.stderr: -445> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding con mode: secure 2026-02-24T22:19:58.886 INFO:tasks.workunit.client.0.trial046.stderr: -444> 2026-02-24T22:19:58.789+0000 7f01615dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:58.886 INFO:tasks.workunit.client.0.trial046.stderr: -443> 2026-02-24T22:19:58.789+0000 7f01615dda80 1 librados: starting msgr at 2026-02-24T22:19:58.886 INFO:tasks.workunit.client.0.trial046.stderr: -442> 2026-02-24T22:19:58.789+0000 7f01615dda80 1 librados: starting objecter 2026-02-24T22:19:58.886 INFO:tasks.workunit.client.0.trial046.stderr: -441> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 asok(0x561991042f00) register_command objecter_requests hook 0x561991175eb0 2026-02-24T22:19:58.886 INFO:tasks.workunit.client.0.trial046.stderr: -440> 2026-02-24T22:19:58.789+0000 7f01615dda80 1 librados: setting wanted keys 2026-02-24T22:19:58.886 INFO:tasks.workunit.client.0.trial046.stderr: -439> 2026-02-24T22:19:58.789+0000 7f01615dda80 1 librados: calling monclient init 2026-02-24T22:19:58.886 INFO:tasks.workunit.client.0.trial046.stderr: -438> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient: init 2026-02-24T22:19:58.886 INFO:tasks.workunit.client.0.trial046.stderr: -437> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding auth protocol: cephx 2026-02-24T22:19:58.886 INFO:tasks.workunit.client.0.trial046.stderr: -436> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding auth protocol: cephx 2026-02-24T22:19:58.886 INFO:tasks.workunit.client.0.trial046.stderr: -435> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding auth protocol: cephx 2026-02-24T22:19:58.886 INFO:tasks.workunit.client.0.trial046.stderr: -434> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding con mode: secure 2026-02-24T22:19:58.886 INFO:tasks.workunit.client.0.trial046.stderr: -433> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding con mode: crc 2026-02-24T22:19:58.887 INFO:tasks.workunit.client.0.trial046.stderr: -432> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding con mode: secure 2026-02-24T22:19:58.887 INFO:tasks.workunit.client.0.trial046.stderr: -431> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding con mode: crc 2026-02-24T22:19:58.887 INFO:tasks.workunit.client.0.trial046.stderr: -430> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding con mode: secure 2026-02-24T22:19:58.887 INFO:tasks.workunit.client.0.trial046.stderr: -429> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding con mode: crc 2026-02-24T22:19:58.887 INFO:tasks.workunit.client.0.trial046.stderr: -428> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding con mode: crc 2026-02-24T22:19:58.887 INFO:tasks.workunit.client.0.trial046.stderr: -427> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding con mode: secure 2026-02-24T22:19:58.887 INFO:tasks.workunit.client.0.trial046.stderr: -426> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding con mode: crc 2026-02-24T22:19:58.887 INFO:tasks.workunit.client.0.trial046.stderr: -425> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding con mode: secure 2026-02-24T22:19:58.887 INFO:tasks.workunit.client.0.trial046.stderr: -424> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding con mode: crc 2026-02-24T22:19:58.887 INFO:tasks.workunit.client.0.trial046.stderr: -423> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding con mode: secure 2026-02-24T22:19:58.887 INFO:tasks.workunit.client.0.trial046.stderr: -422> 2026-02-24T22:19:58.789+0000 7f01615dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:58.887 INFO:tasks.workunit.client.0.trial046.stderr: -421> 2026-02-24T22:19:58.789+0000 7f01615dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:58.887 INFO:tasks.workunit.client.0.trial046.stderr: -420> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 asok(0x561991042f00) register_command rotate-key hook 0x5619911450b8 2026-02-24T22:19:58.887 INFO:tasks.workunit.client.0.trial046.stderr: -419> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient: _reopen_session rank -1 2026-02-24T22:19:58.887 INFO:tasks.workunit.client.0.trial046.stderr: -418> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T22:19:58.887 INFO:tasks.workunit.client.0.trial046.stderr: -417> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient(hunting): picked mon.noname-a con 0x561991176350 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:19:58.887 INFO:tasks.workunit.client.0.trial046.stderr: -416> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient(hunting): picked mon.noname-b con 0x56199122fd00 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:19:58.887 INFO:tasks.workunit.client.0.trial046.stderr: -415> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient(hunting): picked mon.noname-c con 0x7f01440030d0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:19:58.887 INFO:tasks.workunit.client.0.trial046.stderr: -414> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:58.887 INFO:tasks.workunit.client.0.trial046.stderr: -413> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:58.887 INFO:tasks.workunit.client.0.trial046.stderr: -412> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:58.887 INFO:tasks.workunit.client.0.trial046.stderr: -411> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient(hunting): _renew_subs 2026-02-24T22:19:58.888 INFO:tasks.workunit.client.0.trial046.stderr: -410> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient(hunting): authenticate will time out at 1968.897150s 2026-02-24T22:19:58.888 INFO:tasks.workunit.client.0.trial046.stderr: -409> 2026-02-24T22:19:58.789+0000 7f015affd640 10 monclient(hunting): get_auth_request con 0x561991176350 auth_method 0 2026-02-24T22:19:58.888 INFO:tasks.workunit.client.0.trial046.stderr: -408> 2026-02-24T22:19:58.789+0000 7f015affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:58.888 INFO:tasks.workunit.client.0.trial046.stderr: -407> 2026-02-24T22:19:58.789+0000 7f015affd640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:58.888 INFO:tasks.workunit.client.0.trial046.stderr: -406> 2026-02-24T22:19:58.789+0000 7f015affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:58.888 INFO:tasks.workunit.client.0.trial046.stderr: -405> 2026-02-24T22:19:58.789+0000 7f015b7fe640 10 monclient(hunting): get_auth_request con 0x7f01440030d0 auth_method 0 2026-02-24T22:19:58.888 INFO:tasks.workunit.client.0.trial046.stderr: -404> 2026-02-24T22:19:58.789+0000 7f015b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:58.888 INFO:tasks.workunit.client.0.trial046.stderr: -403> 2026-02-24T22:19:58.789+0000 7f015b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:58.888 INFO:tasks.workunit.client.0.trial046.stderr: -402> 2026-02-24T22:19:58.789+0000 7f015b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:58.888 INFO:tasks.workunit.client.0.trial046.stderr: -401> 2026-02-24T22:19:58.789+0000 7f015affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:58.888 INFO:tasks.workunit.client.0.trial046.stderr: -400> 2026-02-24T22:19:58.789+0000 7f015affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:58.888 INFO:tasks.workunit.client.0.trial046.stderr: -399> 2026-02-24T22:19:58.789+0000 7f015affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:58.888 INFO:tasks.workunit.client.0.trial046.stderr: -398> 2026-02-24T22:19:58.789+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:58.888 INFO:tasks.workunit.client.0.trial046.stderr: -397> 2026-02-24T22:19:58.789+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:58.888 INFO:tasks.workunit.client.0.trial046.stderr: -396> 2026-02-24T22:19:58.789+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:58.888 INFO:tasks.workunit.client.0.trial046.stderr: -395> 2026-02-24T22:19:58.789+0000 7f015a7fc640 10 monclient(hunting): get_auth_request con 0x56199122fd00 auth_method 0 2026-02-24T22:19:58.888 INFO:tasks.workunit.client.0.trial046.stderr: -394> 2026-02-24T22:19:58.789+0000 7f015a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:58.888 INFO:tasks.workunit.client.0.trial046.stderr: -393> 2026-02-24T22:19:58.789+0000 7f015a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:58.888 INFO:tasks.workunit.client.0.trial046.stderr: -392> 2026-02-24T22:19:58.789+0000 7f015a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:58.888 INFO:tasks.workunit.client.0.trial046.stderr: -391> 2026-02-24T22:19:58.789+0000 7f015affd640 10 monclient(hunting): handle_auth_done global_id 7992 payload 824 2026-02-24T22:19:58.888 INFO:tasks.workunit.client.0.trial046.stderr: -390> 2026-02-24T22:19:58.789+0000 7f015affd640 10 monclient: _finish_hunting 0 2026-02-24T22:19:58.888 INFO:tasks.workunit.client.0.trial046.stderr: -389> 2026-02-24T22:19:58.789+0000 7f015affd640 1 monclient: found mon.noname-a 2026-02-24T22:19:58.889 INFO:tasks.workunit.client.0.trial046.stderr: -388> 2026-02-24T22:19:58.789+0000 7f015affd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:19:58.889 INFO:tasks.workunit.client.0.trial046.stderr: -387> 2026-02-24T22:19:58.789+0000 7f01437fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:58.889 INFO:tasks.workunit.client.0.trial046.stderr: -386> 2026-02-24T22:19:58.789+0000 7f01437fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:19:58.889 INFO:tasks.workunit.client.0.trial046.stderr: -385> 2026-02-24T22:19:58.789+0000 7f01437fe640 10 monclient: dump: 2026-02-24T22:19:58.889 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:58.889 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:58.889 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.889 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.889 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:58.889 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.889 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:58.889 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:58.889 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:58.889 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.889 INFO:tasks.workunit.client.0.trial046.stderr: -384> 2026-02-24T22:19:58.789+0000 7f01437fe640 10 monclient: _finish_auth 0 2026-02-24T22:19:58.889 INFO:tasks.workunit.client.0.trial046.stderr: -383> 2026-02-24T22:19:58.789+0000 7f01437fe640 10 monclient: _check_auth_tickets 2026-02-24T22:19:58.889 INFO:tasks.workunit.client.0.trial046.stderr: -382> 2026-02-24T22:19:58.789+0000 7f01437fe640 10 monclient: handle_config config(0 keys) 2026-02-24T22:19:58.889 INFO:tasks.workunit.client.0.trial046.stderr: -381> 2026-02-24T22:19:58.789+0000 7f01437fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:58.889 INFO:tasks.workunit.client.0.trial046.stderr: -380> 2026-02-24T22:19:58.789+0000 7f01437fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:19:58.889 INFO:tasks.workunit.client.0.trial046.stderr: -379> 2026-02-24T22:19:58.789+0000 7f01437fe640 10 monclient: dump: 2026-02-24T22:19:58.889 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:58.889 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:58.890 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.890 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.890 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:58.890 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.890 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:58.890 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:58.890 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:58.890 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.890 INFO:tasks.workunit.client.0.trial046.stderr: -378> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 monclient: authenticate success, global_id 7992 2026-02-24T22:19:58.890 INFO:tasks.workunit.client.0.trial046.stderr: -377> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient: _renew_subs 2026-02-24T22:19:58.890 INFO:tasks.workunit.client.0.trial046.stderr: -376> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:19:58.890 INFO:tasks.workunit.client.0.trial046.stderr: -375> 2026-02-24T22:19:58.789+0000 7f01497fa640 4 set_mon_vals no callback set 2026-02-24T22:19:58.890 INFO:tasks.workunit.client.0.trial046.stderr: -374> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient: _renew_subs 2026-02-24T22:19:58.890 INFO:tasks.workunit.client.0.trial046.stderr: -373> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:19:58.890 INFO:tasks.workunit.client.0.trial046.stderr: -372> 2026-02-24T22:19:58.789+0000 7f01615dda80 1 librados: init done 2026-02-24T22:19:58.890 INFO:tasks.workunit.client.0.trial046.stderr: -371> 2026-02-24T22:19:58.789+0000 7f01437fe640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:19:58.890 INFO:tasks.workunit.client.0.trial046.stderr: -370> 2026-02-24T22:19:58.789+0000 7f01437fe640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:19:58.890 INFO:tasks.workunit.client.0.trial046.stderr: -369> 2026-02-24T22:19:58.789+0000 7f01437fe640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:19:58.890 INFO:tasks.workunit.client.0.trial046.stderr: -368> 2026-02-24T22:19:58.789+0000 7f015a7fc640 10 monclient: get_auth_request con 0x7f013c0707b0 auth_method 0 2026-02-24T22:19:58.890 INFO:tasks.workunit.client.0.trial046.stderr: -367> 2026-02-24T22:19:58.789+0000 7f015b7fe640 10 monclient: get_auth_request con 0x561991173760 auth_method 0 2026-02-24T22:19:58.890 INFO:tasks.workunit.client.0.trial046.stderr: -366> 2026-02-24T22:19:58.789+0000 7f015affd640 10 monclient: get_auth_request con 0x561991174610 auth_method 0 2026-02-24T22:19:58.890 INFO:tasks.workunit.client.0.trial046.stderr: -365> 2026-02-24T22:19:58.793+0000 7f015a7fc640 10 monclient: get_auth_request con 0x5619912dd240 auth_method 0 2026-02-24T22:19:58.891 INFO:tasks.workunit.client.0.trial046.stderr: -364> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient: build_initial_monmap 2026-02-24T22:19:58.891 INFO:tasks.workunit.client.0.trial046.stderr: -363> 2026-02-24T22:19:58.793+0000 7f01615dda80 1 build_initial for_mkfs: 0 2026-02-24T22:19:58.891 INFO:tasks.workunit.client.0.trial046.stderr: -362> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient: monmap: 2026-02-24T22:19:58.891 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:19:58.891 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:19:58.891 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:19:58.891 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:19:58.891 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:19:58.891 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.891 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:19:58.891 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:19:58.891 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:19:58.891 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.891 INFO:tasks.workunit.client.0.trial046.stderr: -361> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding auth protocol: cephx 2026-02-24T22:19:58.891 INFO:tasks.workunit.client.0.trial046.stderr: -360> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding auth protocol: cephx 2026-02-24T22:19:58.891 INFO:tasks.workunit.client.0.trial046.stderr: -359> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding auth protocol: cephx 2026-02-24T22:19:58.891 INFO:tasks.workunit.client.0.trial046.stderr: -358> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding con mode: secure 2026-02-24T22:19:58.891 INFO:tasks.workunit.client.0.trial046.stderr: -357> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding con mode: crc 2026-02-24T22:19:58.891 INFO:tasks.workunit.client.0.trial046.stderr: -356> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding con mode: secure 2026-02-24T22:19:58.891 INFO:tasks.workunit.client.0.trial046.stderr: -355> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding con mode: crc 2026-02-24T22:19:58.891 INFO:tasks.workunit.client.0.trial046.stderr: -354> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding con mode: secure 2026-02-24T22:19:58.891 INFO:tasks.workunit.client.0.trial046.stderr: -353> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding con mode: crc 2026-02-24T22:19:58.892 INFO:tasks.workunit.client.0.trial046.stderr: -352> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding con mode: crc 2026-02-24T22:19:58.892 INFO:tasks.workunit.client.0.trial046.stderr: -351> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding con mode: secure 2026-02-24T22:19:58.892 INFO:tasks.workunit.client.0.trial046.stderr: -350> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding con mode: crc 2026-02-24T22:19:58.892 INFO:tasks.workunit.client.0.trial046.stderr: -349> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding con mode: secure 2026-02-24T22:19:58.892 INFO:tasks.workunit.client.0.trial046.stderr: -348> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding con mode: crc 2026-02-24T22:19:58.892 INFO:tasks.workunit.client.0.trial046.stderr: -347> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding con mode: secure 2026-02-24T22:19:58.892 INFO:tasks.workunit.client.0.trial046.stderr: -346> 2026-02-24T22:19:58.793+0000 7f01615dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:58.892 INFO:tasks.workunit.client.0.trial046.stderr: -345> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 asok(0x561991042f00) register_command objecter_requests cmddesc objecter_requests hook 0x561991302730 EEXIST 2026-02-24T22:19:58.892 INFO:tasks.workunit.client.0.trial046.stderr: -344> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient: init 2026-02-24T22:19:58.892 INFO:tasks.workunit.client.0.trial046.stderr: -343> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding auth protocol: cephx 2026-02-24T22:19:58.892 INFO:tasks.workunit.client.0.trial046.stderr: -342> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding auth protocol: cephx 2026-02-24T22:19:58.892 INFO:tasks.workunit.client.0.trial046.stderr: -341> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding auth protocol: cephx 2026-02-24T22:19:58.892 INFO:tasks.workunit.client.0.trial046.stderr: -340> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding con mode: secure 2026-02-24T22:19:58.892 INFO:tasks.workunit.client.0.trial046.stderr: -339> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding con mode: crc 2026-02-24T22:19:58.892 INFO:tasks.workunit.client.0.trial046.stderr: -338> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding con mode: secure 2026-02-24T22:19:58.892 INFO:tasks.workunit.client.0.trial046.stderr: -337> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding con mode: crc 2026-02-24T22:19:58.892 INFO:tasks.workunit.client.0.trial046.stderr: -336> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding con mode: secure 2026-02-24T22:19:58.892 INFO:tasks.workunit.client.0.trial046.stderr: -335> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding con mode: crc 2026-02-24T22:19:58.892 INFO:tasks.workunit.client.0.trial046.stderr: -334> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding con mode: crc 2026-02-24T22:19:58.892 INFO:tasks.workunit.client.0.trial046.stderr: -333> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding con mode: secure 2026-02-24T22:19:58.892 INFO:tasks.workunit.client.0.trial046.stderr: -332> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding con mode: crc 2026-02-24T22:19:58.892 INFO:tasks.workunit.client.0.trial046.stderr: -331> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding con mode: secure 2026-02-24T22:19:58.892 INFO:tasks.workunit.client.0.trial046.stderr: -330> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding con mode: crc 2026-02-24T22:19:58.892 INFO:tasks.workunit.client.0.trial046.stderr: -329> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding con mode: secure 2026-02-24T22:19:58.892 INFO:tasks.workunit.client.0.trial046.stderr: -328> 2026-02-24T22:19:58.793+0000 7f01615dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:58.892 INFO:tasks.workunit.client.0.trial046.stderr: -327> 2026-02-24T22:19:58.793+0000 7f01615dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:58.893 INFO:tasks.workunit.client.0.trial046.stderr: -326> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 asok(0x561991042f00) register_command rotate-key cmddesc rotate-key hook 0x5619912dc360 EEXIST 2026-02-24T22:19:58.893 INFO:tasks.workunit.client.0.trial046.stderr: -325> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient: _reopen_session rank -1 2026-02-24T22:19:58.893 INFO:tasks.workunit.client.0.trial046.stderr: -324> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T22:19:58.893 INFO:tasks.workunit.client.0.trial046.stderr: -323> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient(hunting): picked mon.noname-c con 0x56199132abc0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:19:58.893 INFO:tasks.workunit.client.0.trial046.stderr: -322> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient(hunting): picked mon.noname-b con 0x56199132c900 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:19:58.893 INFO:tasks.workunit.client.0.trial046.stderr: -321> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient(hunting): picked mon.noname-a con 0x56199135c740 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:19:58.893 INFO:tasks.workunit.client.0.trial046.stderr: -320> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:58.893 INFO:tasks.workunit.client.0.trial046.stderr: -319> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:58.893 INFO:tasks.workunit.client.0.trial046.stderr: -318> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:58.893 INFO:tasks.workunit.client.0.trial046.stderr: -317> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient(hunting): _renew_subs 2026-02-24T22:19:58.893 INFO:tasks.workunit.client.0.trial046.stderr: -316> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient(hunting): authenticate will time out at 1968.903189s 2026-02-24T22:19:58.893 INFO:tasks.workunit.client.0.trial046.stderr: -315> 2026-02-24T22:19:58.793+0000 7f015affd640 10 monclient(hunting): get_auth_request con 0x56199132abc0 auth_method 0 2026-02-24T22:19:58.893 INFO:tasks.workunit.client.0.trial046.stderr: -314> 2026-02-24T22:19:58.793+0000 7f015affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:58.893 INFO:tasks.workunit.client.0.trial046.stderr: -313> 2026-02-24T22:19:58.793+0000 7f015affd640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:58.893 INFO:tasks.workunit.client.0.trial046.stderr: -312> 2026-02-24T22:19:58.793+0000 7f015affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:58.893 INFO:tasks.workunit.client.0.trial046.stderr: -311> 2026-02-24T22:19:58.793+0000 7f015b7fe640 10 monclient(hunting): get_auth_request con 0x56199135c740 auth_method 0 2026-02-24T22:19:58.893 INFO:tasks.workunit.client.0.trial046.stderr: -310> 2026-02-24T22:19:58.793+0000 7f015b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:58.893 INFO:tasks.workunit.client.0.trial046.stderr: -309> 2026-02-24T22:19:58.793+0000 7f015b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:58.893 INFO:tasks.workunit.client.0.trial046.stderr: -308> 2026-02-24T22:19:58.793+0000 7f015b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:58.893 INFO:tasks.workunit.client.0.trial046.stderr: -307> 2026-02-24T22:19:58.793+0000 7f015affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:58.893 INFO:tasks.workunit.client.0.trial046.stderr: -306> 2026-02-24T22:19:58.793+0000 7f015affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:58.893 INFO:tasks.workunit.client.0.trial046.stderr: -305> 2026-02-24T22:19:58.793+0000 7f015affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:58.893 INFO:tasks.workunit.client.0.trial046.stderr: -304> 2026-02-24T22:19:58.793+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:58.893 INFO:tasks.workunit.client.0.trial046.stderr: -303> 2026-02-24T22:19:58.793+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:58.894 INFO:tasks.workunit.client.0.trial046.stderr: -302> 2026-02-24T22:19:58.793+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:58.894 INFO:tasks.workunit.client.0.trial046.stderr: -301> 2026-02-24T22:19:58.793+0000 7f015affd640 10 monclient(hunting): handle_auth_done global_id 7472 payload 824 2026-02-24T22:19:58.894 INFO:tasks.workunit.client.0.trial046.stderr: -300> 2026-02-24T22:19:58.793+0000 7f015affd640 10 monclient: _finish_hunting 0 2026-02-24T22:19:58.894 INFO:tasks.workunit.client.0.trial046.stderr: -299> 2026-02-24T22:19:58.793+0000 7f015affd640 1 monclient: found mon.noname-c 2026-02-24T22:19:58.894 INFO:tasks.workunit.client.0.trial046.stderr: -298> 2026-02-24T22:19:58.793+0000 7f015affd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:19:58.894 INFO:tasks.workunit.client.0.trial046.stderr: -297> 2026-02-24T22:19:58.793+0000 7f0140ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:58.894 INFO:tasks.workunit.client.0.trial046.stderr: -296> 2026-02-24T22:19:58.793+0000 7f0140ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:19:58.894 INFO:tasks.workunit.client.0.trial046.stderr: -295> 2026-02-24T22:19:58.793+0000 7f0140ff9640 10 monclient: dump: 2026-02-24T22:19:58.894 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:58.894 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:58.894 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.894 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.894 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:58.894 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.894 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:58.894 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:58.894 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:58.894 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.894 INFO:tasks.workunit.client.0.trial046.stderr: -294> 2026-02-24T22:19:58.793+0000 7f0140ff9640 10 monclient: _finish_auth 0 2026-02-24T22:19:58.894 INFO:tasks.workunit.client.0.trial046.stderr: -293> 2026-02-24T22:19:58.793+0000 7f0140ff9640 10 monclient: _check_auth_tickets 2026-02-24T22:19:58.894 INFO:tasks.workunit.client.0.trial046.stderr: -292> 2026-02-24T22:19:58.793+0000 7f0140ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T22:19:58.894 INFO:tasks.workunit.client.0.trial046.stderr: -291> 2026-02-24T22:19:58.793+0000 7f0140ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:58.894 INFO:tasks.workunit.client.0.trial046.stderr: -290> 2026-02-24T22:19:58.793+0000 7f0140ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:19:58.894 INFO:tasks.workunit.client.0.trial046.stderr: -289> 2026-02-24T22:19:58.793+0000 7f0140ff9640 10 monclient: dump: 2026-02-24T22:19:58.895 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:58.895 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:58.895 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.895 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.895 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:58.895 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.895 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:58.895 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:58.895 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:58.895 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.895 INFO:tasks.workunit.client.0.trial046.stderr: -288> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 monclient: authenticate success, global_id 7472 2026-02-24T22:19:58.895 INFO:tasks.workunit.client.0.trial046.stderr: -287> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient: _renew_subs 2026-02-24T22:19:58.895 INFO:tasks.workunit.client.0.trial046.stderr: -286> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:19:58.895 INFO:tasks.workunit.client.0.trial046.stderr: -285> 2026-02-24T22:19:58.793+0000 7f015bfff640 4 set_mon_vals no callback set 2026-02-24T22:19:58.895 INFO:tasks.workunit.client.0.trial046.stderr: -284> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient: _renew_subs 2026-02-24T22:19:58.895 INFO:tasks.workunit.client.0.trial046.stderr: -283> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:19:58.895 INFO:tasks.workunit.client.0.trial046.stderr: -282> 2026-02-24T22:19:58.797+0000 7f0140ff9640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:19:58.895 INFO:tasks.workunit.client.0.trial046.stderr: -281> 2026-02-24T22:19:58.797+0000 7f0140ff9640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:19:58.895 INFO:tasks.workunit.client.0.trial046.stderr: -280> 2026-02-24T22:19:58.797+0000 7f0140ff9640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:19:58.895 INFO:tasks.workunit.client.0.trial046.stderr: -279> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient: get_auth_request con 0x7f01300706c0 auth_method 0 2026-02-24T22:19:58.895 INFO:tasks.workunit.client.0.trial046.stderr: -278> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: get_monmap_and_config 2026-02-24T22:19:58.895 INFO:tasks.workunit.client.0.trial046.stderr: -277> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: build_initial_monmap 2026-02-24T22:19:58.895 INFO:tasks.workunit.client.0.trial046.stderr: -276> 2026-02-24T22:19:58.797+0000 7f01615dda80 1 build_initial for_mkfs: 0 2026-02-24T22:19:58.895 INFO:tasks.workunit.client.0.trial046.stderr: -275> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: monmap: 2026-02-24T22:19:58.895 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:19:58.896 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:19:58.896 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:19:58.896 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:19:58.896 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:19:58.896 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.896 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:19:58.896 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:19:58.896 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:19:58.896 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.896 INFO:tasks.workunit.client.0.trial046.stderr: -274> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding auth protocol: cephx 2026-02-24T22:19:58.896 INFO:tasks.workunit.client.0.trial046.stderr: -273> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding auth protocol: cephx 2026-02-24T22:19:58.896 INFO:tasks.workunit.client.0.trial046.stderr: -272> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding auth protocol: cephx 2026-02-24T22:19:58.896 INFO:tasks.workunit.client.0.trial046.stderr: -271> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding con mode: secure 2026-02-24T22:19:58.896 INFO:tasks.workunit.client.0.trial046.stderr: -270> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding con mode: crc 2026-02-24T22:19:58.896 INFO:tasks.workunit.client.0.trial046.stderr: -269> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding con mode: secure 2026-02-24T22:19:58.896 INFO:tasks.workunit.client.0.trial046.stderr: -268> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding con mode: crc 2026-02-24T22:19:58.896 INFO:tasks.workunit.client.0.trial046.stderr: -267> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding con mode: secure 2026-02-24T22:19:58.896 INFO:tasks.workunit.client.0.trial046.stderr: -266> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding con mode: crc 2026-02-24T22:19:58.896 INFO:tasks.workunit.client.0.trial046.stderr: -265> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding con mode: crc 2026-02-24T22:19:58.896 INFO:tasks.workunit.client.0.trial046.stderr: -264> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding con mode: secure 2026-02-24T22:19:58.896 INFO:tasks.workunit.client.0.trial046.stderr: -263> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding con mode: crc 2026-02-24T22:19:58.896 INFO:tasks.workunit.client.0.trial046.stderr: -262> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding con mode: secure 2026-02-24T22:19:58.896 INFO:tasks.workunit.client.0.trial046.stderr: -261> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding con mode: crc 2026-02-24T22:19:58.896 INFO:tasks.workunit.client.0.trial046.stderr: -260> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding con mode: secure 2026-02-24T22:19:58.896 INFO:tasks.workunit.client.0.trial046.stderr: -259> 2026-02-24T22:19:58.797+0000 7f01615dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:58.896 INFO:tasks.workunit.client.0.trial046.stderr: -258> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: init 2026-02-24T22:19:58.897 INFO:tasks.workunit.client.0.trial046.stderr: -257> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding auth protocol: cephx 2026-02-24T22:19:58.897 INFO:tasks.workunit.client.0.trial046.stderr: -256> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding auth protocol: cephx 2026-02-24T22:19:58.897 INFO:tasks.workunit.client.0.trial046.stderr: -255> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding auth protocol: cephx 2026-02-24T22:19:58.897 INFO:tasks.workunit.client.0.trial046.stderr: -254> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding con mode: secure 2026-02-24T22:19:58.897 INFO:tasks.workunit.client.0.trial046.stderr: -253> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding con mode: crc 2026-02-24T22:19:58.897 INFO:tasks.workunit.client.0.trial046.stderr: -252> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding con mode: secure 2026-02-24T22:19:58.897 INFO:tasks.workunit.client.0.trial046.stderr: -251> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding con mode: crc 2026-02-24T22:19:58.897 INFO:tasks.workunit.client.0.trial046.stderr: -250> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding con mode: secure 2026-02-24T22:19:58.897 INFO:tasks.workunit.client.0.trial046.stderr: -249> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding con mode: crc 2026-02-24T22:19:58.897 INFO:tasks.workunit.client.0.trial046.stderr: -248> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding con mode: crc 2026-02-24T22:19:58.897 INFO:tasks.workunit.client.0.trial046.stderr: -247> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding con mode: secure 2026-02-24T22:19:58.897 INFO:tasks.workunit.client.0.trial046.stderr: -246> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding con mode: crc 2026-02-24T22:19:58.897 INFO:tasks.workunit.client.0.trial046.stderr: -245> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding con mode: secure 2026-02-24T22:19:58.897 INFO:tasks.workunit.client.0.trial046.stderr: -244> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding con mode: crc 2026-02-24T22:19:58.897 INFO:tasks.workunit.client.0.trial046.stderr: -243> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding con mode: secure 2026-02-24T22:19:58.897 INFO:tasks.workunit.client.0.trial046.stderr: -242> 2026-02-24T22:19:58.797+0000 7f01615dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:58.897 INFO:tasks.workunit.client.0.trial046.stderr: -241> 2026-02-24T22:19:58.797+0000 7f01615dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:58.897 INFO:tasks.workunit.client.0.trial046.stderr: -240> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 asok(0x561991042f00) register_command rotate-key cmddesc rotate-key hook 0x7ffc4c815ff8 EEXIST 2026-02-24T22:19:58.897 INFO:tasks.workunit.client.0.trial046.stderr: -239> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: _reopen_session rank -1 2026-02-24T22:19:58.897 INFO:tasks.workunit.client.0.trial046.stderr: -238> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T22:19:58.897 INFO:tasks.workunit.client.0.trial046.stderr: -237> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): picked mon.noname-b con 0x5619913e6410 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:19:58.897 INFO:tasks.workunit.client.0.trial046.stderr: -236> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): picked mon.noname-a con 0x5619913e9b80 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:19:58.897 INFO:tasks.workunit.client.0.trial046.stderr: -235> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): picked mon.noname-c con 0x5619913c4940 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:19:58.897 INFO:tasks.workunit.client.0.trial046.stderr: -234> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:58.897 INFO:tasks.workunit.client.0.trial046.stderr: -233> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:58.897 INFO:tasks.workunit.client.0.trial046.stderr: -232> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr: -231> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): _renew_subs 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr: -230> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): authenticate will time out at 1968.905832s 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr: -229> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient(hunting): get_auth_request con 0x5619913e9b80 auth_method 0 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr: -228> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr: -227> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr: -226> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr: -225> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): get_auth_request con 0x5619913c4940 auth_method 0 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr: -224> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr: -223> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr: -222> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr: -221> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr: -220> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr: -219> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr: -218> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr: -217> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr: -216> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr: -215> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient(hunting): handle_auth_done global_id 7998 payload 274 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr: -214> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient: _finish_hunting 0 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr: -213> 2026-02-24T22:19:58.797+0000 7f015a7fc640 1 monclient: found mon.noname-a 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr: -212> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr: -211> 2026-02-24T22:19:58.797+0000 7f015affd640 10 monclient: get_auth_request con 0x5619913e6410 auth_method 0 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr: -210> 2026-02-24T22:19:58.797+0000 7f013a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr: -209> 2026-02-24T22:19:58.797+0000 7f013a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr: -208> 2026-02-24T22:19:58.797+0000 7f013a7fc640 10 monclient: dump: 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.898 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr: -207> 2026-02-24T22:19:58.797+0000 7f013a7fc640 10 monclient: _finish_auth 0 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr: -206> 2026-02-24T22:19:58.797+0000 7f013a7fc640 10 monclient: _check_auth_tickets 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr: -205> 2026-02-24T22:19:58.797+0000 7f013a7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr: -204> 2026-02-24T22:19:58.797+0000 7f013a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr: -203> 2026-02-24T22:19:58.797+0000 7f013a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr: -202> 2026-02-24T22:19:58.797+0000 7f013a7fc640 10 monclient: dump: 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr: -201> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 monclient: authenticate success, global_id 7998 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr: -200> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: get_monmap_and_config success 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr: -199> 2026-02-24T22:19:58.797+0000 7f01615dda80 4 set_mon_vals no callback set 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr: -198> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: shutdown 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr: -197> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: build_initial_monmap 2026-02-24T22:19:58.899 INFO:tasks.workunit.client.0.trial046.stderr: -196> 2026-02-24T22:19:58.797+0000 7f01615dda80 1 build_initial for_mkfs: 0 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr: -195> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: monmap: 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr: -194> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding auth protocol: cephx 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr: -193> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding auth protocol: cephx 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr: -192> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding auth protocol: cephx 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr: -191> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding con mode: secure 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr: -190> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding con mode: crc 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr: -189> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding con mode: secure 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr: -188> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding con mode: crc 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr: -187> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding con mode: secure 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr: -186> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding con mode: crc 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr: -185> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding con mode: crc 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr: -184> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding con mode: secure 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr: -183> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding con mode: crc 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr: -182> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding con mode: secure 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr: -181> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding con mode: crc 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr: -180> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding con mode: secure 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr: -179> 2026-02-24T22:19:58.797+0000 7f01615dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr: -178> 2026-02-24T22:19:58.797+0000 7f01615dda80 1 librados: starting msgr at 2026-02-24T22:19:58.900 INFO:tasks.workunit.client.0.trial046.stderr: -177> 2026-02-24T22:19:58.797+0000 7f01615dda80 1 librados: starting objecter 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -176> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 asok(0x561991042f00) register_command objecter_requests cmddesc objecter_requests hook 0x5619913c32a0 EEXIST 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -175> 2026-02-24T22:19:58.797+0000 7f01615dda80 1 librados: setting wanted keys 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -174> 2026-02-24T22:19:58.797+0000 7f01615dda80 1 librados: calling monclient init 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -173> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: init 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -172> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding auth protocol: cephx 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -171> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding auth protocol: cephx 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -170> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding auth protocol: cephx 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -169> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding con mode: secure 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -168> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding con mode: crc 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -167> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding con mode: secure 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -166> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding con mode: crc 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -165> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding con mode: secure 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -164> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding con mode: crc 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -163> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding con mode: crc 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -162> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding con mode: secure 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -161> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding con mode: crc 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -160> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding con mode: secure 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -159> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding con mode: crc 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -158> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding con mode: secure 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -157> 2026-02-24T22:19:58.797+0000 7f01615dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -156> 2026-02-24T22:19:58.797+0000 7f01615dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -155> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 asok(0x561991042f00) register_command rotate-key cmddesc rotate-key hook 0x5619913bdf08 EEXIST 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -154> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: _reopen_session rank -1 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -153> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -152> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): picked mon.noname-c con 0x5619913e6410 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -151> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): picked mon.noname-b con 0x5619913e9b80 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:19:58.901 INFO:tasks.workunit.client.0.trial046.stderr: -150> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): picked mon.noname-a con 0x561991411150 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -149> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -148> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -147> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -146> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): _renew_subs 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -145> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): authenticate will time out at 1968.907171s 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -144> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): get_auth_request con 0x561991411150 auth_method 0 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -143> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -142> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -141> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -140> 2026-02-24T22:19:58.797+0000 7f015affd640 10 monclient(hunting): get_auth_request con 0x5619913e6410 auth_method 0 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -139> 2026-02-24T22:19:58.797+0000 7f015affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -138> 2026-02-24T22:19:58.797+0000 7f015affd640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -137> 2026-02-24T22:19:58.797+0000 7f015affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -136> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -135> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -134> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -133> 2026-02-24T22:19:58.797+0000 7f015affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -132> 2026-02-24T22:19:58.797+0000 7f015affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -131> 2026-02-24T22:19:58.797+0000 7f015affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -130> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient(hunting): get_auth_request con 0x5619913e9b80 auth_method 0 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -129> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -128> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -127> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -126> 2026-02-24T22:19:58.797+0000 7f015affd640 10 monclient(hunting): handle_auth_done global_id 7478 payload 824 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -125> 2026-02-24T22:19:58.797+0000 7f015affd640 10 monclient: _finish_hunting 0 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -124> 2026-02-24T22:19:58.797+0000 7f015affd640 1 monclient: found mon.noname-c 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -123> 2026-02-24T22:19:58.797+0000 7f015affd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -122> 2026-02-24T22:19:58.797+0000 7f0138ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -121> 2026-02-24T22:19:58.797+0000 7f0138ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:19:58.902 INFO:tasks.workunit.client.0.trial046.stderr: -120> 2026-02-24T22:19:58.797+0000 7f0138ff9640 10 monclient: dump: 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr: -119> 2026-02-24T22:19:58.797+0000 7f0138ff9640 10 monclient: _finish_auth 0 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr: -118> 2026-02-24T22:19:58.797+0000 7f0138ff9640 10 monclient: _check_auth_tickets 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr: -117> 2026-02-24T22:19:58.797+0000 7f0138ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr: -116> 2026-02-24T22:19:58.797+0000 7f0138ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr: -115> 2026-02-24T22:19:58.797+0000 7f0138ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr: -114> 2026-02-24T22:19:58.797+0000 7f0138ff9640 10 monclient: dump: 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr: -113> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 monclient: authenticate success, global_id 7478 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr: -112> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: _renew_subs 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr: -111> 2026-02-24T22:19:58.797+0000 7f013a7fc640 4 set_mon_vals no callback set 2026-02-24T22:19:58.903 INFO:tasks.workunit.client.0.trial046.stderr: -110> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -109> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: _renew_subs 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -108> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -107> 2026-02-24T22:19:58.797+0000 7f01615dda80 1 librados: init done 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -106> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 asok(0x561991042f00) register_command cr dump hook 0x5619913c0548 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -105> 2026-02-24T22:19:58.801+0000 7f01615dda80 4 Realm: () 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -104> 2026-02-24T22:19:58.801+0000 7f01615dda80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -103> 2026-02-24T22:19:58.801+0000 7f01615dda80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -102> 2026-02-24T22:19:58.801+0000 7f01615dda80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -101> 2026-02-24T22:19:58.801+0000 7f01615dda80 20 zonegroup default 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -100> 2026-02-24T22:19:58.801+0000 7f015a7fc640 10 monclient: get_auth_request con 0x561991486ac0 auth_method 0 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -99> 2026-02-24T22:19:58.801+0000 7f0138ff9640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -98> 2026-02-24T22:19:58.801+0000 7f0138ff9640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -97> 2026-02-24T22:19:58.801+0000 7f0138ff9640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -96> 2026-02-24T22:19:58.801+0000 7f015b7fe640 10 monclient: get_auth_request con 0x7f01200707b0 auth_method 0 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -95> 2026-02-24T22:19:58.801+0000 7f01615dda80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -94> 2026-02-24T22:19:58.801+0000 7f015affd640 10 monclient: get_auth_request con 0x5619914a9e30 auth_method 0 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -93> 2026-02-24T22:19:58.801+0000 7f01615dda80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -92> 2026-02-24T22:19:58.801+0000 7f01615dda80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -91> 2026-02-24T22:19:58.801+0000 7f015a7fc640 10 monclient: get_auth_request con 0x5619914cb510 auth_method 0 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -90> 2026-02-24T22:19:58.801+0000 7f01615dda80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -89> 2026-02-24T22:19:58.801+0000 7f01615dda80 20 started sync module instance, tier type = 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -88> 2026-02-24T22:19:58.801+0000 7f01615dda80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -87> 2026-02-24T22:19:58.801+0000 7f015b7fe640 10 monclient: get_auth_request con 0x56199152c9c0 auth_method 0 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -86> 2026-02-24T22:19:58.801+0000 7f015b7fe640 10 monclient: get_auth_request con 0x561991634090 auth_method 0 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -85> 2026-02-24T22:19:58.801+0000 7f015affd640 10 monclient: get_auth_request con 0x56199158f1f0 auth_method 0 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -84> 2026-02-24T22:19:58.801+0000 7f015a7fc640 10 monclient: get_auth_request con 0x5619915f1f50 auth_method 0 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -83> 2026-02-24T22:19:58.801+0000 7f015affd640 10 monclient: get_auth_request con 0x561991697090 auth_method 0 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -82> 2026-02-24T22:19:58.801+0000 7f015a7fc640 0 -- 10.20.193.46:0/135678098 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x5619914cb510 msgr2=0x5619914eb9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -81> 2026-02-24T22:19:58.801+0000 7f0138ff9640 1 client.7478.objecter ms_handle_reset 0x5619914cb510 session 0x5619914cae00 osd.3 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -80> 2026-02-24T22:19:58.801+0000 7f0138ff9640 10 monclient: _renew_subs 2026-02-24T22:19:58.904 INFO:tasks.workunit.client.0.trial046.stderr: -79> 2026-02-24T22:19:58.801+0000 7f0138ff9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -78> 2026-02-24T22:19:58.801+0000 7f015a7fc640 10 monclient: get_auth_request con 0x7f012009d4e0 auth_method 0 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -77> 2026-02-24T22:19:58.805+0000 7f01615dda80 20 add_watcher() i=3 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -76> 2026-02-24T22:19:58.805+0000 7f01615dda80 20 add_watcher() i=1 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -75> 2026-02-24T22:19:58.805+0000 7f01615dda80 20 add_watcher() i=5 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -74> 2026-02-24T22:19:58.805+0000 7f01615dda80 20 add_watcher() i=0 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -73> 2026-02-24T22:19:58.805+0000 7f01615dda80 20 add_watcher() i=6 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -72> 2026-02-24T22:19:58.805+0000 7f01615dda80 20 add_watcher() i=4 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -71> 2026-02-24T22:19:58.805+0000 7f015affd640 0 -- 10.20.193.46:0/135678098 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x561991697090 msgr2=0x5619916b7510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -70> 2026-02-24T22:19:58.805+0000 7f01615dda80 20 add_watcher() i=7 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -69> 2026-02-24T22:19:58.805+0000 7f0138ff9640 1 client.7478.objecter ms_handle_reset 0x561991697090 session 0x561991696980 osd.6 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -68> 2026-02-24T22:19:58.805+0000 7f01615dda80 20 add_watcher() i=2 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -67> 2026-02-24T22:19:58.805+0000 7f01615dda80 2 all 8 watchers are set, enabling cache 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -66> 2026-02-24T22:19:58.805+0000 7f015affd640 10 monclient: get_auth_request con 0x5619914cb510 auth_method 0 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -65> 2026-02-24T22:19:58.805+0000 7f015a7fc640 10 monclient: get_auth_request con 0x7f0080008af0 auth_method 0 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -64> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding auth protocol: cephx 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -63> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding auth protocol: cephx 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -62> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding auth protocol: cephx 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -61> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding con mode: secure 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -60> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding con mode: crc 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -59> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding con mode: secure 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -58> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding con mode: crc 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -57> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding con mode: secure 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -56> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding con mode: crc 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -55> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding con mode: crc 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -54> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding con mode: secure 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -53> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding con mode: crc 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -52> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding con mode: secure 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -51> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding con mode: crc 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -50> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding con mode: secure 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -49> 2026-02-24T22:19:58.805+0000 7f01615dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -48> 2026-02-24T22:19:58.805+0000 7f01615dda80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:19:58.905 INFO:tasks.workunit.client.0.trial046.stderr: -47> 2026-02-24T22:19:58.805+0000 7f01615dda80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -46> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 asok(0x561991042f00) register_command cache list hook 0x5619913e9f90 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -45> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 asok(0x561991042f00) register_command cache inspect hook 0x5619913e9f90 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -44> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 asok(0x561991042f00) register_command cache erase hook 0x5619913e9f90 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -43> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 asok(0x561991042f00) register_command cache zap hook 0x5619913e9f90 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -42> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 note: GC not initialized 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -41> 2026-02-24T22:19:58.805+0000 7f0077fff640 20 reqs_thread_entry: start 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -40> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 asok(0x561991042f00) register_command sync trace show hook 0x7f0144001d40 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -39> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 asok(0x561991042f00) register_command sync trace history hook 0x7f0144001d40 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -38> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 asok(0x561991042f00) register_command sync trace active hook 0x7f0144001d40 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -37> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 asok(0x561991042f00) register_command sync trace active_short hook 0x7f0144001d40 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -36> 2026-02-24T22:19:58.809+0000 7f015b7fe640 10 monclient: get_auth_request con 0x561991561840 auth_method 0 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -35> 2026-02-24T22:19:58.809+0000 7f015affd640 10 monclient: get_auth_request con 0x56199155a2d0 auth_method 0 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -34> 2026-02-24T22:19:58.809+0000 7f015a7fc640 10 monclient: get_auth_request con 0x56199155c650 auth_method 0 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -33> 2026-02-24T22:19:58.809+0000 7f015b7fe640 10 monclient: get_auth_request con 0x56199155da20 auth_method 0 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -32> 2026-02-24T22:19:58.813+0000 7f015affd640 10 monclient: get_auth_request con 0x561991559000 auth_method 0 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -31> 2026-02-24T22:19:58.817+0000 7f015a7fc640 10 monclient: get_auth_request con 0x5619914e6620 auth_method 0 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -30> 2026-02-24T22:19:58.837+0000 7f01615dda80 20 init_complete bucket index max shards: 1 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -29> 2026-02-24T22:19:58.837+0000 7f01615dda80 20 Filter name: none 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -28> 2026-02-24T22:19:58.841+0000 7f00767fc640 20 reqs_thread_entry: start 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -27> 2026-02-24T22:19:58.841+0000 7f01615dda80 10 cache get: name=default.rgw.meta+root+abort-at-logrecord-writes : miss 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -26> 2026-02-24T22:19:58.841+0000 7f01615dda80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -25> 2026-02-24T22:19:58.841+0000 7f01615dda80 20 rados_obj.operate() r=0 bl.length=221 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -24> 2026-02-24T22:19:58.841+0000 7f01615dda80 10 cache put: name=default.rgw.meta+root+abort-at-logrecord-writes info.flags=0x11 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -23> 2026-02-24T22:19:58.841+0000 7f01615dda80 10 adding default.rgw.meta+root+abort-at-logrecord-writes to cache LRU end 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -22> 2026-02-24T22:19:58.841+0000 7f01615dda80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 : miss 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -21> 2026-02-24T22:19:58.841+0000 7f01615dda80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -20> 2026-02-24T22:19:58.841+0000 7f01615dda80 20 rados_obj.operate() r=0 bl.length=451 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -19> 2026-02-24T22:19:58.841+0000 7f01615dda80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 info.flags=0x17 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -18> 2026-02-24T22:19:58.841+0000 7f01615dda80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 to cache LRU end 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -17> 2026-02-24T22:19:58.841+0000 7f01615dda80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -16> 2026-02-24T22:19:58.841+0000 7f01615dda80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -15> 2026-02-24T22:19:58.841+0000 7f01615dda80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 2026-02-24T22:19:58.906 INFO:tasks.workunit.client.0.trial046.stderr: -14> 2026-02-24T22:19:58.841+0000 7f015b7fe640 10 monclient: get_auth_request con 0x5619915689d0 auth_method 0 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: -13> 2026-02-24T22:19:58.845+0000 7f01615dda80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 info.flags=0x17 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: -12> 2026-02-24T22:19:58.845+0000 7f01615dda80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 to cache LRU end 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: -11> 2026-02-24T22:19:58.845+0000 7f01615dda80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: -10> 2026-02-24T22:19:58.845+0000 7f01615dda80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537, ofs0, ns] 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: -9> 2026-02-24T22:19:58.845+0000 7f013a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961388 cookie 94667813368912 notifier 7478 bl.length()=1025 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: -8> 2026-02-24T22:19:58.845+0000 7f013a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 info.flags=0x17 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: -7> 2026-02-24T22:19:58.845+0000 7f013a7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 to cache LRU end 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: -6> 2026-02-24T22:19:58.845+0000 7f013a7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: -5> 2026-02-24T22:19:58.845+0000 7f01615dda80 20 handle_bi_update(): bucket=:abort-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537]): orig_sources= new_sources= 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: -4> 2026-02-24T22:19:58.845+0000 7f01615dda80 20 handle_bi_update(): bucket=:abort-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537]): potential sources added=[] removed=[] 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: -3> 2026-02-24T22:19:58.845+0000 7f01615dda80 20 handle_bi_update(): bucket=:abort-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537]): orig_dests= new_dests= 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: -2> 2026-02-24T22:19:58.845+0000 7f01615dda80 20 handle_bi_update(): bucket=:abort-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537]): potential dests added=[] removed=[] 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: -1> 2026-02-24T22:19:58.845+0000 7f01615dda80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f01615dda80 time 2026-02-24T22:19:58.850383+0000 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f0164a1feb1] 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x56197c7bcb9f] 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: 3: radosgw-admin(+0x1714451) [0x56197d9cb451] 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x56197cf28202] 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: 5: main() 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f0163d1fd90] 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: 7: __libc_start_main() 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: 8: _start() 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: 0> 2026-02-24T22:19:58.849+0000 7f01615dda80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f01615dda80 thread_name:radosgw-admin 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f0163d38520] 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:19:58.907 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f0164a1ff10] 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x56197c7bcb9f] 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 7: radosgw-admin(+0x1714451) [0x56197d9cb451] 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x56197cf28202] 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 9: main() 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f0163d1fd90] 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 11: __libc_start_main() 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 12: _start() 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr:--- logging levels --- 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 none 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 lockdep 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 context 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 crush 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_balancer 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_locker 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log_expire 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_migrator 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 3/ 5 mds_quiesce 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 buffer 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 timer 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 filer 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 striper 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 objecter 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rados 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_mirror 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_replay 2026-02-24T22:19:58.908 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_pwl 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 journaler 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objectcacher 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 immutable_obj_cache 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 client 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 osd 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 optracker 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objclass 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 filestore 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 journal 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 ms 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mon 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 0/10 monc 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 paxos 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 tp 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 auth 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 crypto 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 finisher 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 reserver 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 heartbeatmap 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 perfcounter 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_sync 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_datacache 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_access 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dbstore 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_flight 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_lifecycle 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_restore 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw_notification 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_bucket_logging 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 javaclient 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 asok 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 throttle 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 refs 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 compressor 2026-02-24T22:19:58.909 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore_compression 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluefs 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 bdev 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 kstore 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 4/ 5 rocksdb 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 fuse 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 2/ 5 mgr 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mgrc 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 dpdk 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 eventtrace 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 prioritycache 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 test 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephfs_mirror 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephsqlite 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 crimson_interrupt 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_onode 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_odata 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_omap 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_tm 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_t 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cleaner 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_epm 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_lba 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_fixedkv_tree 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cache 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_journal 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_device 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_backref 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 alienstore 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mclock 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dedup 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cyanstore 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 ceph_exporter 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 memstore 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 trace 2026-02-24T22:19:58.910 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 ceph_dedup 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: -2/-2 (syslog threshold) 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: 99/99 (stderr threshold) 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr:--- pthread ID / name mapping for recent threads --- 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: 7f00767fc640 / http_manager 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: 7f0077fff640 / http_manager 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: 7f0138ff9640 / ms_dispatch 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: 7f013a7fc640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: 7f0140ff9640 / ms_dispatch 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: 7f01437fe640 / ms_dispatch 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: 7f01497fa640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: 7f0149ffb640 / admin_socket 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: 7f0159ffb640 / ms_dispatch 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: 7f015a7fc640 / msgr-worker-2 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: 7f015affd640 / msgr-worker-1 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: 7f015b7fe640 / msgr-worker-0 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: 7f015bfff640 / io_context_pool 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: 7f01615dda80 / radosgw-admin 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: max_recent 500 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: max_new 1000 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: log_file /var/log/ceph/ceph-client.admin.51674.log 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr:--- end dump of recent events --- 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr:--- begin dump of recent events --- 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: -499> 2026-02-24T22:19:58.785+0000 7f01615dda80 10 monclient(hunting): authenticate will time out at 1968.895734s 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: -498> 2026-02-24T22:19:58.785+0000 7f015a7fc640 10 monclient(hunting): get_auth_request con 0x561991176350 auth_method 0 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: -497> 2026-02-24T22:19:58.785+0000 7f015a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: -496> 2026-02-24T22:19:58.785+0000 7f015a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: -495> 2026-02-24T22:19:58.785+0000 7f015a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: -494> 2026-02-24T22:19:58.785+0000 7f015b7fe640 10 monclient(hunting): get_auth_request con 0x56199122fd00 auth_method 0 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: -493> 2026-02-24T22:19:58.785+0000 7f015b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: -492> 2026-02-24T22:19:58.785+0000 7f015b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: -491> 2026-02-24T22:19:58.785+0000 7f015b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: -490> 2026-02-24T22:19:58.785+0000 7f015a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: -489> 2026-02-24T22:19:58.785+0000 7f015a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: -488> 2026-02-24T22:19:58.785+0000 7f015a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:58.911 INFO:tasks.workunit.client.0.trial046.stderr: -487> 2026-02-24T22:19:58.785+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr: -486> 2026-02-24T22:19:58.785+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr: -485> 2026-02-24T22:19:58.785+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr: -484> 2026-02-24T22:19:58.785+0000 7f015affd640 10 monclient(hunting): get_auth_request con 0x7f01440030d0 auth_method 0 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr: -483> 2026-02-24T22:19:58.785+0000 7f015affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr: -482> 2026-02-24T22:19:58.785+0000 7f015affd640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr: -481> 2026-02-24T22:19:58.785+0000 7f015affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr: -480> 2026-02-24T22:19:58.785+0000 7f015a7fc640 10 monclient(hunting): handle_auth_done global_id 7466 payload 274 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr: -479> 2026-02-24T22:19:58.785+0000 7f015a7fc640 10 monclient: _finish_hunting 0 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr: -478> 2026-02-24T22:19:58.785+0000 7f015a7fc640 1 monclient: found mon.noname-c 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr: -477> 2026-02-24T22:19:58.785+0000 7f015a7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr: -476> 2026-02-24T22:19:58.789+0000 7f01497fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr: -475> 2026-02-24T22:19:58.789+0000 7f01497fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr: -474> 2026-02-24T22:19:58.789+0000 7f01497fa640 10 monclient: dump: 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr: -473> 2026-02-24T22:19:58.789+0000 7f01497fa640 10 monclient: _finish_auth 0 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr: -472> 2026-02-24T22:19:58.789+0000 7f01497fa640 10 monclient: _check_auth_tickets 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr: -471> 2026-02-24T22:19:58.789+0000 7f01497fa640 10 monclient: handle_config config(0 keys) 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr: -470> 2026-02-24T22:19:58.789+0000 7f01497fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr: -469> 2026-02-24T22:19:58.789+0000 7f01497fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr: -468> 2026-02-24T22:19:58.789+0000 7f01497fa640 10 monclient: dump: 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:58.912 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr: -467> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 monclient: authenticate success, global_id 7466 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr: -466> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient: get_monmap_and_config success 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr: -465> 2026-02-24T22:19:58.789+0000 7f01615dda80 4 set_mon_vals no callback set 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr: -464> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient: shutdown 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr: -463> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 asok(0x561991042f00) unregister_commands rotate-key 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr: -462> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient: build_initial_monmap 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr: -461> 2026-02-24T22:19:58.789+0000 7f01615dda80 1 build_initial for_mkfs: 0 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr: -460> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient: monmap: 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr: -459> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding auth protocol: cephx 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr: -458> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding auth protocol: cephx 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr: -457> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding auth protocol: cephx 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr: -456> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding con mode: secure 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr: -455> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding con mode: crc 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr: -454> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding con mode: secure 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr: -453> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding con mode: crc 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr: -452> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding con mode: secure 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr: -451> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding con mode: crc 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr: -450> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding con mode: crc 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr: -449> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding con mode: secure 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr: -448> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding con mode: crc 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr: -447> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding con mode: secure 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr: -446> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding con mode: crc 2026-02-24T22:19:58.913 INFO:tasks.workunit.client.0.trial046.stderr: -445> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x5619912301a0) adding con mode: secure 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -444> 2026-02-24T22:19:58.789+0000 7f01615dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -443> 2026-02-24T22:19:58.789+0000 7f01615dda80 1 librados: starting msgr at 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -442> 2026-02-24T22:19:58.789+0000 7f01615dda80 1 librados: starting objecter 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -441> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 asok(0x561991042f00) register_command objecter_requests hook 0x561991175eb0 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -440> 2026-02-24T22:19:58.789+0000 7f01615dda80 1 librados: setting wanted keys 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -439> 2026-02-24T22:19:58.789+0000 7f01615dda80 1 librados: calling monclient init 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -438> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient: init 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -437> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding auth protocol: cephx 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -436> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding auth protocol: cephx 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -435> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding auth protocol: cephx 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -434> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding con mode: secure 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -433> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding con mode: crc 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -432> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding con mode: secure 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -431> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding con mode: crc 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -430> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding con mode: secure 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -429> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding con mode: crc 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -428> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding con mode: crc 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -427> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding con mode: secure 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -426> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding con mode: crc 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -425> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding con mode: secure 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -424> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding con mode: crc 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -423> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 AuthRegistry(0x561991144f70) adding con mode: secure 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -422> 2026-02-24T22:19:58.789+0000 7f01615dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -421> 2026-02-24T22:19:58.789+0000 7f01615dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -420> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 asok(0x561991042f00) register_command rotate-key hook 0x5619911450b8 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -419> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient: _reopen_session rank -1 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -418> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -417> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient(hunting): picked mon.noname-a con 0x561991176350 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -416> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient(hunting): picked mon.noname-b con 0x56199122fd00 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -415> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient(hunting): picked mon.noname-c con 0x7f01440030d0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -414> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -413> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -412> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -411> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient(hunting): _renew_subs 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -410> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient(hunting): authenticate will time out at 1968.897150s 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -409> 2026-02-24T22:19:58.789+0000 7f015affd640 10 monclient(hunting): get_auth_request con 0x561991176350 auth_method 0 2026-02-24T22:19:58.914 INFO:tasks.workunit.client.0.trial046.stderr: -408> 2026-02-24T22:19:58.789+0000 7f015affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -407> 2026-02-24T22:19:58.789+0000 7f015affd640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -406> 2026-02-24T22:19:58.789+0000 7f015affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -405> 2026-02-24T22:19:58.789+0000 7f015b7fe640 10 monclient(hunting): get_auth_request con 0x7f01440030d0 auth_method 0 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -404> 2026-02-24T22:19:58.789+0000 7f015b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -403> 2026-02-24T22:19:58.789+0000 7f015b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -402> 2026-02-24T22:19:58.789+0000 7f015b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -401> 2026-02-24T22:19:58.789+0000 7f015affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -400> 2026-02-24T22:19:58.789+0000 7f015affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -399> 2026-02-24T22:19:58.789+0000 7f015affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -398> 2026-02-24T22:19:58.789+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -397> 2026-02-24T22:19:58.789+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -396> 2026-02-24T22:19:58.789+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -395> 2026-02-24T22:19:58.789+0000 7f015a7fc640 10 monclient(hunting): get_auth_request con 0x56199122fd00 auth_method 0 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -394> 2026-02-24T22:19:58.789+0000 7f015a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -393> 2026-02-24T22:19:58.789+0000 7f015a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -392> 2026-02-24T22:19:58.789+0000 7f015a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -391> 2026-02-24T22:19:58.789+0000 7f015affd640 10 monclient(hunting): handle_auth_done global_id 7992 payload 824 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -390> 2026-02-24T22:19:58.789+0000 7f015affd640 10 monclient: _finish_hunting 0 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -389> 2026-02-24T22:19:58.789+0000 7f015affd640 1 monclient: found mon.noname-a 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -388> 2026-02-24T22:19:58.789+0000 7f015affd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -387> 2026-02-24T22:19:58.789+0000 7f01437fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -386> 2026-02-24T22:19:58.789+0000 7f01437fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -385> 2026-02-24T22:19:58.789+0000 7f01437fe640 10 monclient: dump: 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -384> 2026-02-24T22:19:58.789+0000 7f01437fe640 10 monclient: _finish_auth 0 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -383> 2026-02-24T22:19:58.789+0000 7f01437fe640 10 monclient: _check_auth_tickets 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -382> 2026-02-24T22:19:58.789+0000 7f01437fe640 10 monclient: handle_config config(0 keys) 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -381> 2026-02-24T22:19:58.789+0000 7f01437fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:58.915 INFO:tasks.workunit.client.0.trial046.stderr: -380> 2026-02-24T22:19:58.789+0000 7f01437fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr: -379> 2026-02-24T22:19:58.789+0000 7f01437fe640 10 monclient: dump: 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr: -378> 2026-02-24T22:19:58.789+0000 7f01615dda80 5 monclient: authenticate success, global_id 7992 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr: -377> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient: _renew_subs 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr: -376> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr: -375> 2026-02-24T22:19:58.789+0000 7f01497fa640 4 set_mon_vals no callback set 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr: -374> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient: _renew_subs 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr: -373> 2026-02-24T22:19:58.789+0000 7f01615dda80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr: -372> 2026-02-24T22:19:58.789+0000 7f01615dda80 1 librados: init done 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr: -371> 2026-02-24T22:19:58.789+0000 7f01437fe640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr: -370> 2026-02-24T22:19:58.789+0000 7f01437fe640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr: -369> 2026-02-24T22:19:58.789+0000 7f01437fe640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr: -368> 2026-02-24T22:19:58.789+0000 7f015a7fc640 10 monclient: get_auth_request con 0x7f013c0707b0 auth_method 0 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr: -367> 2026-02-24T22:19:58.789+0000 7f015b7fe640 10 monclient: get_auth_request con 0x561991173760 auth_method 0 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr: -366> 2026-02-24T22:19:58.789+0000 7f015affd640 10 monclient: get_auth_request con 0x561991174610 auth_method 0 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr: -365> 2026-02-24T22:19:58.793+0000 7f015a7fc640 10 monclient: get_auth_request con 0x5619912dd240 auth_method 0 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr: -364> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient: build_initial_monmap 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr: -363> 2026-02-24T22:19:58.793+0000 7f01615dda80 1 build_initial for_mkfs: 0 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr: -362> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient: monmap: 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:19:58.916 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -361> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding auth protocol: cephx 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -360> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding auth protocol: cephx 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -359> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding auth protocol: cephx 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -358> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding con mode: secure 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -357> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding con mode: crc 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -356> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding con mode: secure 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -355> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding con mode: crc 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -354> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding con mode: secure 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -353> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding con mode: crc 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -352> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding con mode: crc 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -351> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding con mode: secure 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -350> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding con mode: crc 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -349> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding con mode: secure 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -348> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding con mode: crc 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -347> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912fed60) adding con mode: secure 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -346> 2026-02-24T22:19:58.793+0000 7f01615dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -345> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 asok(0x561991042f00) register_command objecter_requests cmddesc objecter_requests hook 0x561991302730 EEXIST 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -344> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient: init 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -343> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding auth protocol: cephx 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -342> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding auth protocol: cephx 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -341> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding auth protocol: cephx 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -340> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding con mode: secure 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -339> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding con mode: crc 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -338> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding con mode: secure 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -337> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding con mode: crc 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -336> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding con mode: secure 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -335> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding con mode: crc 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -334> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding con mode: crc 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -333> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding con mode: secure 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -332> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding con mode: crc 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -331> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding con mode: secure 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -330> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding con mode: crc 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -329> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 AuthRegistry(0x5619912dc218) adding con mode: secure 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -328> 2026-02-24T22:19:58.793+0000 7f01615dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -327> 2026-02-24T22:19:58.793+0000 7f01615dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -326> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 asok(0x561991042f00) register_command rotate-key cmddesc rotate-key hook 0x5619912dc360 EEXIST 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -325> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient: _reopen_session rank -1 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -324> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T22:19:58.917 INFO:tasks.workunit.client.0.trial046.stderr: -323> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient(hunting): picked mon.noname-c con 0x56199132abc0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -322> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient(hunting): picked mon.noname-b con 0x56199132c900 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -321> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient(hunting): picked mon.noname-a con 0x56199135c740 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -320> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -319> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -318> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -317> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient(hunting): _renew_subs 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -316> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient(hunting): authenticate will time out at 1968.903189s 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -315> 2026-02-24T22:19:58.793+0000 7f015affd640 10 monclient(hunting): get_auth_request con 0x56199132abc0 auth_method 0 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -314> 2026-02-24T22:19:58.793+0000 7f015affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -313> 2026-02-24T22:19:58.793+0000 7f015affd640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -312> 2026-02-24T22:19:58.793+0000 7f015affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -311> 2026-02-24T22:19:58.793+0000 7f015b7fe640 10 monclient(hunting): get_auth_request con 0x56199135c740 auth_method 0 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -310> 2026-02-24T22:19:58.793+0000 7f015b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -309> 2026-02-24T22:19:58.793+0000 7f015b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -308> 2026-02-24T22:19:58.793+0000 7f015b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -307> 2026-02-24T22:19:58.793+0000 7f015affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -306> 2026-02-24T22:19:58.793+0000 7f015affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -305> 2026-02-24T22:19:58.793+0000 7f015affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -304> 2026-02-24T22:19:58.793+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -303> 2026-02-24T22:19:58.793+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -302> 2026-02-24T22:19:58.793+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -301> 2026-02-24T22:19:58.793+0000 7f015affd640 10 monclient(hunting): handle_auth_done global_id 7472 payload 824 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -300> 2026-02-24T22:19:58.793+0000 7f015affd640 10 monclient: _finish_hunting 0 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -299> 2026-02-24T22:19:58.793+0000 7f015affd640 1 monclient: found mon.noname-c 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -298> 2026-02-24T22:19:58.793+0000 7f015affd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -297> 2026-02-24T22:19:58.793+0000 7f0140ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -296> 2026-02-24T22:19:58.793+0000 7f0140ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -295> 2026-02-24T22:19:58.793+0000 7f0140ff9640 10 monclient: dump: 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -294> 2026-02-24T22:19:58.793+0000 7f0140ff9640 10 monclient: _finish_auth 0 2026-02-24T22:19:58.918 INFO:tasks.workunit.client.0.trial046.stderr: -293> 2026-02-24T22:19:58.793+0000 7f0140ff9640 10 monclient: _check_auth_tickets 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr: -292> 2026-02-24T22:19:58.793+0000 7f0140ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr: -291> 2026-02-24T22:19:58.793+0000 7f0140ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr: -290> 2026-02-24T22:19:58.793+0000 7f0140ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr: -289> 2026-02-24T22:19:58.793+0000 7f0140ff9640 10 monclient: dump: 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr: -288> 2026-02-24T22:19:58.793+0000 7f01615dda80 5 monclient: authenticate success, global_id 7472 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr: -287> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient: _renew_subs 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr: -286> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr: -285> 2026-02-24T22:19:58.793+0000 7f015bfff640 4 set_mon_vals no callback set 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr: -284> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient: _renew_subs 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr: -283> 2026-02-24T22:19:58.793+0000 7f01615dda80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr: -282> 2026-02-24T22:19:58.797+0000 7f0140ff9640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr: -281> 2026-02-24T22:19:58.797+0000 7f0140ff9640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr: -280> 2026-02-24T22:19:58.797+0000 7f0140ff9640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr: -279> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient: get_auth_request con 0x7f01300706c0 auth_method 0 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr: -278> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: get_monmap_and_config 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr: -277> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: build_initial_monmap 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr: -276> 2026-02-24T22:19:58.797+0000 7f01615dda80 1 build_initial for_mkfs: 0 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr: -275> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: monmap: 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.919 INFO:tasks.workunit.client.0.trial046.stderr: -274> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding auth protocol: cephx 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -273> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding auth protocol: cephx 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -272> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding auth protocol: cephx 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -271> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding con mode: secure 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -270> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding con mode: crc 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -269> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding con mode: secure 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -268> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding con mode: crc 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -267> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding con mode: secure 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -266> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding con mode: crc 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -265> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding con mode: crc 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -264> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding con mode: secure 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -263> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding con mode: crc 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -262> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding con mode: secure 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -261> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding con mode: crc 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -260> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bf350) adding con mode: secure 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -259> 2026-02-24T22:19:58.797+0000 7f01615dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -258> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: init 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -257> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding auth protocol: cephx 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -256> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding auth protocol: cephx 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -255> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding auth protocol: cephx 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -254> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding con mode: secure 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -253> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding con mode: crc 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -252> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding con mode: secure 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -251> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding con mode: crc 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -250> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding con mode: secure 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -249> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding con mode: crc 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -248> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding con mode: crc 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -247> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding con mode: secure 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -246> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding con mode: crc 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -245> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding con mode: secure 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -244> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding con mode: crc 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -243> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c815eb0) adding con mode: secure 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -242> 2026-02-24T22:19:58.797+0000 7f01615dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -241> 2026-02-24T22:19:58.797+0000 7f01615dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -240> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 asok(0x561991042f00) register_command rotate-key cmddesc rotate-key hook 0x7ffc4c815ff8 EEXIST 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -239> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: _reopen_session rank -1 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -238> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -237> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): picked mon.noname-b con 0x5619913e6410 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -236> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): picked mon.noname-a con 0x5619913e9b80 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -235> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): picked mon.noname-c con 0x5619913c4940 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -234> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -233> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:58.920 INFO:tasks.workunit.client.0.trial046.stderr: -232> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -231> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): _renew_subs 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -230> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): authenticate will time out at 1968.905832s 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -229> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient(hunting): get_auth_request con 0x5619913e9b80 auth_method 0 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -228> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -227> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -226> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -225> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): get_auth_request con 0x5619913c4940 auth_method 0 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -224> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -223> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -222> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -221> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -220> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -219> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -218> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -217> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -216> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -215> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient(hunting): handle_auth_done global_id 7998 payload 274 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -214> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient: _finish_hunting 0 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -213> 2026-02-24T22:19:58.797+0000 7f015a7fc640 1 monclient: found mon.noname-a 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -212> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -211> 2026-02-24T22:19:58.797+0000 7f015affd640 10 monclient: get_auth_request con 0x5619913e6410 auth_method 0 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -210> 2026-02-24T22:19:58.797+0000 7f013a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -209> 2026-02-24T22:19:58.797+0000 7f013a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -208> 2026-02-24T22:19:58.797+0000 7f013a7fc640 10 monclient: dump: 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -207> 2026-02-24T22:19:58.797+0000 7f013a7fc640 10 monclient: _finish_auth 0 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -206> 2026-02-24T22:19:58.797+0000 7f013a7fc640 10 monclient: _check_auth_tickets 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -205> 2026-02-24T22:19:58.797+0000 7f013a7fc640 10 monclient: handle_config config(0 keys) 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -204> 2026-02-24T22:19:58.797+0000 7f013a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -203> 2026-02-24T22:19:58.797+0000 7f013a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr: -202> 2026-02-24T22:19:58.797+0000 7f013a7fc640 10 monclient: dump: 2026-02-24T22:19:58.921 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr: -201> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 monclient: authenticate success, global_id 7998 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr: -200> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: get_monmap_and_config success 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr: -199> 2026-02-24T22:19:58.797+0000 7f01615dda80 4 set_mon_vals no callback set 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr: -198> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: shutdown 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr: -197> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: build_initial_monmap 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr: -196> 2026-02-24T22:19:58.797+0000 7f01615dda80 1 build_initial for_mkfs: 0 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr: -195> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: monmap: 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr: -194> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding auth protocol: cephx 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr: -193> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding auth protocol: cephx 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr: -192> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding auth protocol: cephx 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr: -191> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding con mode: secure 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr: -190> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding con mode: crc 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr: -189> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding con mode: secure 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr: -188> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding con mode: crc 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr: -187> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding con mode: secure 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr: -186> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding con mode: crc 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr: -185> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding con mode: crc 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr: -184> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding con mode: secure 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr: -183> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding con mode: crc 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr: -182> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding con mode: secure 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr: -181> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding con mode: crc 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr: -180> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x561991416620) adding con mode: secure 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr: -179> 2026-02-24T22:19:58.797+0000 7f01615dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:58.922 INFO:tasks.workunit.client.0.trial046.stderr: -178> 2026-02-24T22:19:58.797+0000 7f01615dda80 1 librados: starting msgr at 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -177> 2026-02-24T22:19:58.797+0000 7f01615dda80 1 librados: starting objecter 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -176> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 asok(0x561991042f00) register_command objecter_requests cmddesc objecter_requests hook 0x5619913c32a0 EEXIST 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -175> 2026-02-24T22:19:58.797+0000 7f01615dda80 1 librados: setting wanted keys 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -174> 2026-02-24T22:19:58.797+0000 7f01615dda80 1 librados: calling monclient init 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -173> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: init 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -172> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding auth protocol: cephx 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -171> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding auth protocol: cephx 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -170> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding auth protocol: cephx 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -169> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding con mode: secure 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -168> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding con mode: crc 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -167> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding con mode: secure 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -166> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding con mode: crc 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -165> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding con mode: secure 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -164> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding con mode: crc 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -163> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding con mode: crc 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -162> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding con mode: secure 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -161> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding con mode: crc 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -160> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding con mode: secure 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -159> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding con mode: crc 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -158> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 AuthRegistry(0x5619913bddc0) adding con mode: secure 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -157> 2026-02-24T22:19:58.797+0000 7f01615dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -156> 2026-02-24T22:19:58.797+0000 7f01615dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -155> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 asok(0x561991042f00) register_command rotate-key cmddesc rotate-key hook 0x5619913bdf08 EEXIST 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -154> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: _reopen_session rank -1 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -153> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -152> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): picked mon.noname-c con 0x5619913e6410 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -151> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): picked mon.noname-b con 0x5619913e9b80 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -150> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): picked mon.noname-a con 0x561991411150 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -149> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -148> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -147> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): start opening mon connection 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -146> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): _renew_subs 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -145> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient(hunting): authenticate will time out at 1968.907171s 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -144> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): get_auth_request con 0x561991411150 auth_method 0 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -143> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -142> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -141> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -140> 2026-02-24T22:19:58.797+0000 7f015affd640 10 monclient(hunting): get_auth_request con 0x5619913e6410 auth_method 0 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -139> 2026-02-24T22:19:58.797+0000 7f015affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -138> 2026-02-24T22:19:58.797+0000 7f015affd640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -137> 2026-02-24T22:19:58.797+0000 7f015affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -136> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -135> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:58.923 INFO:tasks.workunit.client.0.trial046.stderr: -134> 2026-02-24T22:19:58.797+0000 7f015b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr: -133> 2026-02-24T22:19:58.797+0000 7f015affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr: -132> 2026-02-24T22:19:58.797+0000 7f015affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr: -131> 2026-02-24T22:19:58.797+0000 7f015affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr: -130> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient(hunting): get_auth_request con 0x5619913e9b80 auth_method 0 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr: -129> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr: -128> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr: -127> 2026-02-24T22:19:58.797+0000 7f015a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr: -126> 2026-02-24T22:19:58.797+0000 7f015affd640 10 monclient(hunting): handle_auth_done global_id 7478 payload 824 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr: -125> 2026-02-24T22:19:58.797+0000 7f015affd640 10 monclient: _finish_hunting 0 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr: -124> 2026-02-24T22:19:58.797+0000 7f015affd640 1 monclient: found mon.noname-c 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr: -123> 2026-02-24T22:19:58.797+0000 7f015affd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr: -122> 2026-02-24T22:19:58.797+0000 7f0138ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr: -121> 2026-02-24T22:19:58.797+0000 7f0138ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr: -120> 2026-02-24T22:19:58.797+0000 7f0138ff9640 10 monclient: dump: 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr: -119> 2026-02-24T22:19:58.797+0000 7f0138ff9640 10 monclient: _finish_auth 0 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr: -118> 2026-02-24T22:19:58.797+0000 7f0138ff9640 10 monclient: _check_auth_tickets 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr: -117> 2026-02-24T22:19:58.797+0000 7f0138ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr: -116> 2026-02-24T22:19:58.797+0000 7f0138ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr: -115> 2026-02-24T22:19:58.797+0000 7f0138ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr: -114> 2026-02-24T22:19:58.797+0000 7f0138ff9640 10 monclient: dump: 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr: -113> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 monclient: authenticate success, global_id 7478 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr: -112> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: _renew_subs 2026-02-24T22:19:58.924 INFO:tasks.workunit.client.0.trial046.stderr: -111> 2026-02-24T22:19:58.797+0000 7f013a7fc640 4 set_mon_vals no callback set 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -110> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -109> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: _renew_subs 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -108> 2026-02-24T22:19:58.797+0000 7f01615dda80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -107> 2026-02-24T22:19:58.797+0000 7f01615dda80 1 librados: init done 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -106> 2026-02-24T22:19:58.797+0000 7f01615dda80 5 asok(0x561991042f00) register_command cr dump hook 0x5619913c0548 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -105> 2026-02-24T22:19:58.801+0000 7f01615dda80 4 Realm: () 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -104> 2026-02-24T22:19:58.801+0000 7f01615dda80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -103> 2026-02-24T22:19:58.801+0000 7f01615dda80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -102> 2026-02-24T22:19:58.801+0000 7f01615dda80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -101> 2026-02-24T22:19:58.801+0000 7f01615dda80 20 zonegroup default 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -100> 2026-02-24T22:19:58.801+0000 7f015a7fc640 10 monclient: get_auth_request con 0x561991486ac0 auth_method 0 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -99> 2026-02-24T22:19:58.801+0000 7f0138ff9640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -98> 2026-02-24T22:19:58.801+0000 7f0138ff9640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -97> 2026-02-24T22:19:58.801+0000 7f0138ff9640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -96> 2026-02-24T22:19:58.801+0000 7f015b7fe640 10 monclient: get_auth_request con 0x7f01200707b0 auth_method 0 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -95> 2026-02-24T22:19:58.801+0000 7f01615dda80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -94> 2026-02-24T22:19:58.801+0000 7f015affd640 10 monclient: get_auth_request con 0x5619914a9e30 auth_method 0 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -93> 2026-02-24T22:19:58.801+0000 7f01615dda80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -92> 2026-02-24T22:19:58.801+0000 7f01615dda80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -91> 2026-02-24T22:19:58.801+0000 7f015a7fc640 10 monclient: get_auth_request con 0x5619914cb510 auth_method 0 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -90> 2026-02-24T22:19:58.801+0000 7f01615dda80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -89> 2026-02-24T22:19:58.801+0000 7f01615dda80 20 started sync module instance, tier type = 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -88> 2026-02-24T22:19:58.801+0000 7f01615dda80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -87> 2026-02-24T22:19:58.801+0000 7f015b7fe640 10 monclient: get_auth_request con 0x56199152c9c0 auth_method 0 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -86> 2026-02-24T22:19:58.801+0000 7f015b7fe640 10 monclient: get_auth_request con 0x561991634090 auth_method 0 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -85> 2026-02-24T22:19:58.801+0000 7f015affd640 10 monclient: get_auth_request con 0x56199158f1f0 auth_method 0 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -84> 2026-02-24T22:19:58.801+0000 7f015a7fc640 10 monclient: get_auth_request con 0x5619915f1f50 auth_method 0 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -83> 2026-02-24T22:19:58.801+0000 7f015affd640 10 monclient: get_auth_request con 0x561991697090 auth_method 0 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -82> 2026-02-24T22:19:58.801+0000 7f015a7fc640 0 -- 10.20.193.46:0/135678098 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x5619914cb510 msgr2=0x5619914eb9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -81> 2026-02-24T22:19:58.801+0000 7f0138ff9640 1 client.7478.objecter ms_handle_reset 0x5619914cb510 session 0x5619914cae00 osd.3 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -80> 2026-02-24T22:19:58.801+0000 7f0138ff9640 10 monclient: _renew_subs 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -79> 2026-02-24T22:19:58.801+0000 7f0138ff9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -78> 2026-02-24T22:19:58.801+0000 7f015a7fc640 10 monclient: get_auth_request con 0x7f012009d4e0 auth_method 0 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -77> 2026-02-24T22:19:58.805+0000 7f01615dda80 20 add_watcher() i=3 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -76> 2026-02-24T22:19:58.805+0000 7f01615dda80 20 add_watcher() i=1 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -75> 2026-02-24T22:19:58.805+0000 7f01615dda80 20 add_watcher() i=5 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -74> 2026-02-24T22:19:58.805+0000 7f01615dda80 20 add_watcher() i=0 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -73> 2026-02-24T22:19:58.805+0000 7f01615dda80 20 add_watcher() i=6 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -72> 2026-02-24T22:19:58.805+0000 7f01615dda80 20 add_watcher() i=4 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -71> 2026-02-24T22:19:58.805+0000 7f015affd640 0 -- 10.20.193.46:0/135678098 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x561991697090 msgr2=0x5619916b7510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -70> 2026-02-24T22:19:58.805+0000 7f01615dda80 20 add_watcher() i=7 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -69> 2026-02-24T22:19:58.805+0000 7f0138ff9640 1 client.7478.objecter ms_handle_reset 0x561991697090 session 0x561991696980 osd.6 2026-02-24T22:19:58.925 INFO:tasks.workunit.client.0.trial046.stderr: -68> 2026-02-24T22:19:58.805+0000 7f01615dda80 20 add_watcher() i=2 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -67> 2026-02-24T22:19:58.805+0000 7f01615dda80 2 all 8 watchers are set, enabling cache 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -66> 2026-02-24T22:19:58.805+0000 7f015affd640 10 monclient: get_auth_request con 0x5619914cb510 auth_method 0 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -65> 2026-02-24T22:19:58.805+0000 7f015a7fc640 10 monclient: get_auth_request con 0x7f0080008af0 auth_method 0 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -64> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding auth protocol: cephx 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -63> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding auth protocol: cephx 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -62> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding auth protocol: cephx 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -61> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding con mode: secure 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -60> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding con mode: crc 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -59> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding con mode: secure 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -58> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding con mode: crc 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -57> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding con mode: secure 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -56> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding con mode: crc 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -55> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding con mode: crc 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -54> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding con mode: secure 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -53> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding con mode: crc 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -52> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding con mode: secure 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -51> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding con mode: crc 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -50> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 AuthRegistry(0x7ffc4c816320) adding con mode: secure 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -49> 2026-02-24T22:19:58.805+0000 7f01615dda80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -48> 2026-02-24T22:19:58.805+0000 7f01615dda80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -47> 2026-02-24T22:19:58.805+0000 7f01615dda80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -46> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 asok(0x561991042f00) register_command cache list hook 0x5619913e9f90 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -45> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 asok(0x561991042f00) register_command cache inspect hook 0x5619913e9f90 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -44> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 asok(0x561991042f00) register_command cache erase hook 0x5619913e9f90 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -43> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 asok(0x561991042f00) register_command cache zap hook 0x5619913e9f90 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -42> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 note: GC not initialized 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -41> 2026-02-24T22:19:58.805+0000 7f0077fff640 20 reqs_thread_entry: start 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -40> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 asok(0x561991042f00) register_command sync trace show hook 0x7f0144001d40 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -39> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 asok(0x561991042f00) register_command sync trace history hook 0x7f0144001d40 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -38> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 asok(0x561991042f00) register_command sync trace active hook 0x7f0144001d40 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -37> 2026-02-24T22:19:58.805+0000 7f01615dda80 5 asok(0x561991042f00) register_command sync trace active_short hook 0x7f0144001d40 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -36> 2026-02-24T22:19:58.809+0000 7f015b7fe640 10 monclient: get_auth_request con 0x561991561840 auth_method 0 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -35> 2026-02-24T22:19:58.809+0000 7f015affd640 10 monclient: get_auth_request con 0x56199155a2d0 auth_method 0 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -34> 2026-02-24T22:19:58.809+0000 7f015a7fc640 10 monclient: get_auth_request con 0x56199155c650 auth_method 0 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -33> 2026-02-24T22:19:58.809+0000 7f015b7fe640 10 monclient: get_auth_request con 0x56199155da20 auth_method 0 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -32> 2026-02-24T22:19:58.813+0000 7f015affd640 10 monclient: get_auth_request con 0x561991559000 auth_method 0 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -31> 2026-02-24T22:19:58.817+0000 7f015a7fc640 10 monclient: get_auth_request con 0x5619914e6620 auth_method 0 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -30> 2026-02-24T22:19:58.837+0000 7f01615dda80 20 init_complete bucket index max shards: 1 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -29> 2026-02-24T22:19:58.837+0000 7f01615dda80 20 Filter name: none 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -28> 2026-02-24T22:19:58.841+0000 7f00767fc640 20 reqs_thread_entry: start 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -27> 2026-02-24T22:19:58.841+0000 7f01615dda80 10 cache get: name=default.rgw.meta+root+abort-at-logrecord-writes : miss 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -26> 2026-02-24T22:19:58.841+0000 7f01615dda80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -25> 2026-02-24T22:19:58.841+0000 7f01615dda80 20 rados_obj.operate() r=0 bl.length=221 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -24> 2026-02-24T22:19:58.841+0000 7f01615dda80 10 cache put: name=default.rgw.meta+root+abort-at-logrecord-writes info.flags=0x11 2026-02-24T22:19:58.926 INFO:tasks.workunit.client.0.trial046.stderr: -23> 2026-02-24T22:19:58.841+0000 7f01615dda80 10 adding default.rgw.meta+root+abort-at-logrecord-writes to cache LRU end 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: -22> 2026-02-24T22:19:58.841+0000 7f01615dda80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 : miss 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: -21> 2026-02-24T22:19:58.841+0000 7f01615dda80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: -20> 2026-02-24T22:19:58.841+0000 7f01615dda80 20 rados_obj.operate() r=0 bl.length=451 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: -19> 2026-02-24T22:19:58.841+0000 7f01615dda80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 info.flags=0x17 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: -18> 2026-02-24T22:19:58.841+0000 7f01615dda80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 to cache LRU end 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: -17> 2026-02-24T22:19:58.841+0000 7f01615dda80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: -16> 2026-02-24T22:19:58.841+0000 7f01615dda80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: -15> 2026-02-24T22:19:58.841+0000 7f01615dda80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: -14> 2026-02-24T22:19:58.841+0000 7f015b7fe640 10 monclient: get_auth_request con 0x5619915689d0 auth_method 0 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: -13> 2026-02-24T22:19:58.845+0000 7f01615dda80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 info.flags=0x17 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: -12> 2026-02-24T22:19:58.845+0000 7f01615dda80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 to cache LRU end 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: -11> 2026-02-24T22:19:58.845+0000 7f01615dda80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: -10> 2026-02-24T22:19:58.845+0000 7f01615dda80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537, ofs0, ns] 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: -9> 2026-02-24T22:19:58.845+0000 7f013a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961388 cookie 94667813368912 notifier 7478 bl.length()=1025 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: -8> 2026-02-24T22:19:58.845+0000 7f013a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 info.flags=0x17 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: -7> 2026-02-24T22:19:58.845+0000 7f013a7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 to cache LRU end 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: -6> 2026-02-24T22:19:58.845+0000 7f013a7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: -5> 2026-02-24T22:19:58.845+0000 7f01615dda80 20 handle_bi_update(): bucket=:abort-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537]): orig_sources= new_sources= 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: -4> 2026-02-24T22:19:58.845+0000 7f01615dda80 20 handle_bi_update(): bucket=:abort-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537]): potential sources added=[] removed=[] 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: -3> 2026-02-24T22:19:58.845+0000 7f01615dda80 20 handle_bi_update(): bucket=:abort-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537]): orig_dests= new_dests= 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: -2> 2026-02-24T22:19:58.845+0000 7f01615dda80 20 handle_bi_update(): bucket=:abort-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537]): potential dests added=[] removed=[] 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: -1> 2026-02-24T22:19:58.845+0000 7f01615dda80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f01615dda80 time 2026-02-24T22:19:58.850383+0000 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f0164a1feb1] 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x56197c7bcb9f] 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: 3: radosgw-admin(+0x1714451) [0x56197d9cb451] 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x56197cf28202] 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: 5: main() 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f0163d1fd90] 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: 7: __libc_start_main() 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: 8: _start() 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: 0> 2026-02-24T22:19:58.849+0000 7f01615dda80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f01615dda80 thread_name:radosgw-admin 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f0163d38520] 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:19:58.927 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f0164a1ff10] 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x56197c7bcb9f] 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 7: radosgw-admin(+0x1714451) [0x56197d9cb451] 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x56197cf28202] 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 9: main() 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f0163d1fd90] 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 11: __libc_start_main() 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 12: _start() 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr:--- logging levels --- 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 none 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 lockdep 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 context 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 crush 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_balancer 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_locker 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log_expire 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_migrator 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 3/ 5 mds_quiesce 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 buffer 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 timer 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 filer 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 striper 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 objecter 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rados 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_mirror 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_replay 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_pwl 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 journaler 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objectcacher 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 immutable_obj_cache 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 client 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 osd 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 optracker 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objclass 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 filestore 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 journal 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 ms 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mon 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 0/10 monc 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 paxos 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 tp 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 auth 2026-02-24T22:19:58.928 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 crypto 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 finisher 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 reserver 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 heartbeatmap 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 perfcounter 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_sync 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_datacache 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_access 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dbstore 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_flight 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_lifecycle 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_restore 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw_notification 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_bucket_logging 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 javaclient 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 asok 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 throttle 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 refs 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 compressor 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore_compression 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluefs 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 bdev 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 kstore 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 4/ 5 rocksdb 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 fuse 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 2/ 5 mgr 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mgrc 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 dpdk 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 eventtrace 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 prioritycache 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 test 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephfs_mirror 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephsqlite 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 crimson_interrupt 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_onode 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_odata 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_omap 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_tm 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_t 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cleaner 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_epm 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_lba 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_fixedkv_tree 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cache 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_journal 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_device 2026-02-24T22:19:58.929 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_backref 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 alienstore 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mclock 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dedup 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cyanstore 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 ceph_exporter 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 memstore 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 trace 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 ceph_dedup 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: -2/-2 (syslog threshold) 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: 99/99 (stderr threshold) 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr:--- pthread ID / name mapping for recent threads --- 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: 7f00767fc640 / http_manager 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: 7f0077fff640 / http_manager 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: 7f0138ff9640 / ms_dispatch 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: 7f013a7fc640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: 7f0140ff9640 / ms_dispatch 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: 7f01437fe640 / ms_dispatch 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: 7f01497fa640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: 7f0149ffb640 / admin_socket 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: 7f0159ffb640 / ms_dispatch 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: 7f015a7fc640 / msgr-worker-2 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: 7f015affd640 / msgr-worker-1 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: 7f015b7fe640 / msgr-worker-0 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: 7f015bfff640 / io_context_pool 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: 7f01615dda80 / radosgw-admin 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: max_recent 500 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: max_new 1000 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr: log_file /var/lib/ceph/crash/2026-02-24T22:19:58.852871Z_8feef4be-827b-4d3c-8bd5-7b2e3dd1680a/log 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr:--- end dump of recent events --- 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr:Aborted 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket abort-at-logrecord-writes 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:19:58.930 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:19:58.931 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.885+0000 7f85b57d7a80 4 Realm: () 2026-02-24T22:19:58.931 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.885+0000 7f85b57d7a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:19:58.931 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.885+0000 7f85b57d7a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:19:58.931 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.885+0000 7f85b57d7a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:19:58.931 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.885+0000 7f85b57d7a80 20 zonegroup default 2026-02-24T22:19:58.931 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.885+0000 7f85b57d7a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.931 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.889+0000 7f85b57d7a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:58.931 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.889+0000 7f85b57d7a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:58.931 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.889+0000 7f85b57d7a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:58.931 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.889+0000 7f85b57d7a80 20 started sync module instance, tier type = 2026-02-24T22:19:58.931 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.889+0000 7f85b57d7a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:19:58.931 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.889+0000 7f85b57d7a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:19:58.931 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.889+0000 7f85b57d7a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:19:58.931 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.889+0000 7f85b57d7a80 5 note: GC not initialized 2026-02-24T22:19:58.931 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.889+0000 7f8560fe1640 20 reqs_thread_entry: start 2026-02-24T22:19:58.931 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:58.893+0000 7f85aeffd640 0 -- 10.20.193.46:0/3812282250 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x560402a0f860 msgr2=0x560402a0fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:59.100 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.105+0000 7f85af7fe640 0 -- 10.20.193.46:0/3812282250 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x5604029ffb90 msgr2=0x560402a014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:19:59.328 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.333+0000 7f85b57d7a80 20 init_complete bucket index max shards: 1 2026-02-24T22:19:59.329 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.333+0000 7f85b57d7a80 20 Filter name: none 2026-02-24T22:19:59.329 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.333+0000 7f84d2ffd640 20 reqs_thread_entry: start 2026-02-24T22:19:59.329 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.333+0000 7f85b57d7a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:59.329 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.333+0000 7f85b57d7a80 20 rados_obj.operate() r=0 bl.length=221 2026-02-24T22:19:59.329 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.333+0000 7f85b57d7a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:59.331 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.333+0000 7f85b57d7a80 20 rados_obj.operate() r=0 bl.length=487 2026-02-24T22:19:59.331 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.333+0000 7f85b57d7a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:19:59.331 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.333+0000 7f85b57d7a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:59.331 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.333+0000 7f85b57d7a80 20 rados_obj.operate() r=0 bl.length=221 2026-02-24T22:19:59.331 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.333+0000 7f85b57d7a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:59.331 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.333+0000 7f85b57d7a80 20 rados_obj.operate() r=0 bl.length=487 2026-02-24T22:19:59.332 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.333+0000 7f85b57d7a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:19:59.341 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:59.341 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:19:59.341 INFO:tasks.workunit.client.0.trial046.stderr:bucket: abort-at-logrecord-writes id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:19:59.341 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:19:59.342 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:59.342 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:59.348 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:19:59.348 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:19:59.392 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket abort-at-logrecord-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2026-02-24T22:19:59.409 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:19:59.410 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:19:59.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.429+0000 7fa6ee2d9a80 4 Realm: () 2026-02-24T22:19:59.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.429+0000 7fa6ee2d9a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:19:59.428 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.429+0000 7fa6ee2d9a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:19:59.428 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.429+0000 7fa6ee2d9a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:19:59.428 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.429+0000 7fa6ee2d9a80 20 zonegroup default 2026-02-24T22:19:59.428 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.429+0000 7fa6ee2d9a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:59.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.433+0000 7fa6ee2d9a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:59.429 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.433+0000 7fa6ee2d9a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:59.430 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.433+0000 7fa6ee2d9a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:19:59.430 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.433+0000 7fa6ee2d9a80 20 started sync module instance, tier type = 2026-02-24T22:19:59.430 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.433+0000 7fa6ee2d9a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:19:59.430 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.433+0000 7fa6e6ffd640 0 -- 10.20.193.46:0/606144881 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55ccbe980730 msgr2=0x55ccbe9a0bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:19:59.432 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.433+0000 7fa6ee2d9a80 20 add_watcher() i=6 2026-02-24T22:19:59.432 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.433+0000 7fa6ee2d9a80 20 add_watcher() i=3 2026-02-24T22:19:59.432 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.433+0000 7fa6ee2d9a80 20 add_watcher() i=5 2026-02-24T22:19:59.433 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.433+0000 7fa6ee2d9a80 20 add_watcher() i=0 2026-02-24T22:19:59.433 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.433+0000 7fa6ee2d9a80 20 add_watcher() i=1 2026-02-24T22:19:59.433 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.437+0000 7fa6ee2d9a80 20 add_watcher() i=7 2026-02-24T22:19:59.433 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.437+0000 7fa6ee2d9a80 20 add_watcher() i=2 2026-02-24T22:19:59.433 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.437+0000 7fa6ee2d9a80 20 add_watcher() i=4 2026-02-24T22:19:59.433 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.437+0000 7fa6ee2d9a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:19:59.434 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.437+0000 7fa6ee2d9a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:19:59.434 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.437+0000 7fa6ee2d9a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:19:59.434 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.437+0000 7fa6ee2d9a80 5 note: GC not initialized 2026-02-24T22:19:59.434 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.437+0000 7fa698fe1640 20 reqs_thread_entry: start 2026-02-24T22:19:59.434 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.437+0000 7fa6e7fff640 0 -- 10.20.193.46:0/3971673498 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55ccbead8230 msgr2=0x55ccbe997860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:19:59.673 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.677+0000 7fa6ee2d9a80 20 init_complete bucket index max shards: 1 2026-02-24T22:19:59.673 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.677+0000 7fa6ee2d9a80 20 Filter name: none 2026-02-24T22:19:59.673 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.677+0000 7fa602ffd640 20 reqs_thread_entry: start 2026-02-24T22:19:59.673 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.677+0000 7fa6ee2d9a80 10 cache get: name=default.rgw.meta+root+abort-at-logrecord-writes : miss 2026-02-24T22:19:59.673 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.677+0000 7fa6ee2d9a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:59.674 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.677+0000 7fa6ee2d9a80 20 rados_obj.operate() r=0 bl.length=221 2026-02-24T22:19:59.674 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.677+0000 7fa6ee2d9a80 10 cache put: name=default.rgw.meta+root+abort-at-logrecord-writes info.flags=0x11 2026-02-24T22:19:59.674 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.677+0000 7fa6ee2d9a80 10 adding default.rgw.meta+root+abort-at-logrecord-writes to cache LRU end 2026-02-24T22:19:59.674 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.677+0000 7fa6ee2d9a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 : miss 2026-02-24T22:19:59.674 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.677+0000 7fa6ee2d9a80 20 rados->read ofs=0 len=0 2026-02-24T22:19:59.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.677+0000 7fa6ee2d9a80 20 rados_obj.operate() r=0 bl.length=487 2026-02-24T22:19:59.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.677+0000 7fa6ee2d9a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 info.flags=0x17 2026-02-24T22:19:59.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.677+0000 7fa6ee2d9a80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 to cache LRU end 2026-02-24T22:19:59.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.677+0000 7fa6ee2d9a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:19:59.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.677+0000 7fa6ee2d9a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:19:59.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.677+0000 7fa6ee2d9a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 2026-02-24T22:19:59.675 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.677+0000 7fa6ee2d9a80 0 INFO: RGWReshardLock::lock found lock on abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 to be held by another RGW process; skipping for now 2026-02-24T22:19:59.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.681+0000 7fa6ee2d9a80 20 remove_watcher() i=3 2026-02-24T22:19:59.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.681+0000 7fa6ee2d9a80 2 removed watcher, disabling cache 2026-02-24T22:19:59.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.681+0000 7fa6ee2d9a80 20 remove_watcher() i=4 2026-02-24T22:19:59.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.681+0000 7fa6ee2d9a80 20 remove_watcher() i=0 2026-02-24T22:19:59.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.681+0000 7fa6ee2d9a80 20 remove_watcher() i=1 2026-02-24T22:19:59.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.681+0000 7fa6ee2d9a80 20 remove_watcher() i=2 2026-02-24T22:19:59.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.681+0000 7fa6ee2d9a80 20 remove_watcher() i=6 2026-02-24T22:19:59.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.681+0000 7fa6ee2d9a80 20 remove_watcher() i=5 2026-02-24T22:19:59.678 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:19:59.681+0000 7fa6ee2d9a80 20 remove_watcher() i=7 2026-02-24T22:19:59.684 INFO:tasks.workunit.client.0.trial046.stderr:waiting 30 seconds for reshard lock to expire... 2026-02-24T22:20:29.714 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket abort-at-logrecord-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2026-02-24T22:20:29.730 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:20:29.730 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:20:29.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:29.752+0000 7f5de2a7ea80 4 Realm: () 2026-02-24T22:20:29.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:29.752+0000 7f5de2a7ea80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:20:29.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:29.752+0000 7f5de2a7ea80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:20:29.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:29.752+0000 7f5de2a7ea80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:20:29.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:29.752+0000 7f5de2a7ea80 20 zonegroup default 2026-02-24T22:20:29.750 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:29.752+0000 7f5de2a7ea80 20 rados->read ofs=0 len=0 2026-02-24T22:20:29.750 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:29.752+0000 7f5de2a7ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:29.750 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:29.752+0000 7f5de2a7ea80 20 rados->read ofs=0 len=0 2026-02-24T22:20:29.751 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:29.752+0000 7f5de2a7ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:29.751 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:29.752+0000 7f5de2a7ea80 20 started sync module instance, tier type = 2026-02-24T22:20:29.751 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:29.752+0000 7f5de2a7ea80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:20:29.752 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:29.756+0000 7f5ddbfff640 0 -- 10.20.193.46:0/929625614 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x561f386992c0 msgr2=0x561f386b9720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:29.753 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:29.756+0000 7f5de2a7ea80 20 add_watcher() i=1 2026-02-24T22:20:29.753 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:29.756+0000 7f5de2a7ea80 20 add_watcher() i=3 2026-02-24T22:20:29.753 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:29.756+0000 7f5de2a7ea80 20 add_watcher() i=0 2026-02-24T22:20:29.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:29.756+0000 7f5de2a7ea80 20 add_watcher() i=5 2026-02-24T22:20:29.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:29.756+0000 7f5de2a7ea80 20 add_watcher() i=4 2026-02-24T22:20:29.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:29.756+0000 7f5de2a7ea80 20 add_watcher() i=6 2026-02-24T22:20:29.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:29.756+0000 7f5de2a7ea80 20 add_watcher() i=2 2026-02-24T22:20:29.958 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:29.960+0000 7f5de2a7ea80 20 add_watcher() i=7 2026-02-24T22:20:29.958 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:29.960+0000 7f5de2a7ea80 2 all 8 watchers are set, enabling cache 2026-02-24T22:20:29.960 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:29.960+0000 7f5de2a7ea80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:20:29.960 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:29.960+0000 7f5de2a7ea80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:20:29.960 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:29.964+0000 7f5de2a7ea80 5 note: GC not initialized 2026-02-24T22:20:29.960 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:29.964+0000 7f5d8dfdb640 20 reqs_thread_entry: start 2026-02-24T22:20:29.961 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:29.964+0000 7f5de0d53640 0 -- 10.20.193.46:0/651698730 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x561f3857d340 msgr2=0x561f3857d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:30.201 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.204+0000 7f5de2a7ea80 20 init_complete bucket index max shards: 1 2026-02-24T22:20:30.201 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.204+0000 7f5de2a7ea80 20 Filter name: none 2026-02-24T22:20:30.201 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.204+0000 7f5ce7fff640 20 reqs_thread_entry: start 2026-02-24T22:20:30.201 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.204+0000 7f5de2a7ea80 10 cache get: name=default.rgw.meta+root+abort-at-logrecord-writes : miss 2026-02-24T22:20:30.201 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.204+0000 7f5de2a7ea80 20 rados->read ofs=0 len=0 2026-02-24T22:20:30.201 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.204+0000 7f5de2a7ea80 20 rados_obj.operate() r=0 bl.length=221 2026-02-24T22:20:30.201 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.204+0000 7f5de2a7ea80 10 cache put: name=default.rgw.meta+root+abort-at-logrecord-writes info.flags=0x11 2026-02-24T22:20:30.202 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.204+0000 7f5de2a7ea80 10 adding default.rgw.meta+root+abort-at-logrecord-writes to cache LRU end 2026-02-24T22:20:30.202 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.204+0000 7f5de2a7ea80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 : miss 2026-02-24T22:20:30.202 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.204+0000 7f5de2a7ea80 20 rados->read ofs=0 len=0 2026-02-24T22:20:30.203 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.204+0000 7f5de2a7ea80 20 rados_obj.operate() r=0 bl.length=487 2026-02-24T22:20:30.203 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.204+0000 7f5de2a7ea80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 info.flags=0x17 2026-02-24T22:20:30.203 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.204+0000 7f5de2a7ea80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 to cache LRU end 2026-02-24T22:20:30.203 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.204+0000 7f5de2a7ea80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:20:30.203 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.204+0000 7f5de2a7ea80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:30.203 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.204+0000 7f5de2a7ea80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 2026-02-24T22:20:30.203 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.204+0000 7f5de2a7ea80 10 init_target_layout removing existing target index objects from a previous reshard attempt 2026-02-24T22:20:30.207 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.208+0000 7f5de2a7ea80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 info.flags=0x17 2026-02-24T22:20:30.208 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.208+0000 7f5de2a7ea80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 to cache LRU end 2026-02-24T22:20:30.208 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.208+0000 7f5de2a7ea80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:30.208 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.208+0000 7f5de2a7ea80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537, ofs0, ns] 2026-02-24T22:20:30.208 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.208+0000 7f5daeffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961389 cookie 94692090565216 notifier 7529 bl.length()=1025 2026-02-24T22:20:30.208 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.208+0000 7f5daeffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 info.flags=0x17 2026-02-24T22:20:30.208 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.208+0000 7f5daeffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 to cache LRU end 2026-02-24T22:20:30.208 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.208+0000 7f5daeffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:30.208 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.208+0000 7f5de2a7ea80 20 handle_bi_update(): bucket=:abort-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537]): orig_sources= new_sources= 2026-02-24T22:20:30.208 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.208+0000 7f5de2a7ea80 20 handle_bi_update(): bucket=:abort-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537]): potential sources added=[] removed=[] 2026-02-24T22:20:30.208 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.208+0000 7f5de2a7ea80 20 handle_bi_update(): bucket=:abort-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537]): orig_dests= new_dests= 2026-02-24T22:20:30.208 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.208+0000 7f5de2a7ea80 20 handle_bi_update(): bucket=:abort-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537]): potential dests added=[] removed=[] 2026-02-24T22:20:30.209 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.212+0000 7f5de2a7ea80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537.2.0 2026-02-24T22:20:30.209 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.212+0000 7f5de2a7ea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537.2.0 2026-02-24T22:20:30.209 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.212+0000 7f5de2a7ea80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537.2.1 2026-02-24T22:20:30.210 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.212+0000 7f5de2a7ea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537.2.1 2026-02-24T22:20:30.210 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.212+0000 7f5de2a7ea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537.0 2026-02-24T22:20:30.211 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.212+0000 7f5de2a7ea80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 : hit (requested=0x17, cached=0x17) 2026-02-24T22:20:30.211 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.212+0000 7f5de2a7ea80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 2026-02-24T22:20:30.212 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5de2a7ea80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 info.flags=0x17 2026-02-24T22:20:30.212 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5de2a7ea80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 to cache LRU end 2026-02-24T22:20:30.212 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5de2a7ea80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:30.213 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5de2a7ea80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537, ofs0, ns] 2026-02-24T22:20:30.213 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5daf7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961390 cookie 94692090565216 notifier 7529 bl.length()=1025 2026-02-24T22:20:30.213 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5daf7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 info.flags=0x17 2026-02-24T22:20:30.213 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5daf7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 to cache LRU end 2026-02-24T22:20:30.213 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5daf7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:30.213 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5de2a7ea80 20 handle_bi_update(): bucket=:abort-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537]): orig_sources= new_sources= 2026-02-24T22:20:30.213 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5de2a7ea80 20 handle_bi_update(): bucket=:abort-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537]): potential sources added=[] removed=[] 2026-02-24T22:20:30.213 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5de2a7ea80 20 handle_bi_update(): bucket=:abort-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537]): orig_dests= new_dests= 2026-02-24T22:20:30.213 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5de2a7ea80 20 handle_bi_update(): bucket=:abort-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537]): potential dests added=[] removed=[] 2026-02-24T22:20:30.214 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5de2a7ea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537.0 2026-02-24T22:20:30.214 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5de2a7ea80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 : hit (requested=0x17, cached=0x17) 2026-02-24T22:20:30.214 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5de2a7ea80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 2026-02-24T22:20:30.215 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5de2a7ea80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 info.flags=0x17 2026-02-24T22:20:30.215 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5de2a7ea80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 to cache LRU end 2026-02-24T22:20:30.215 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5de2a7ea80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:30.215 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5de2a7ea80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537, ofs0, ns] 2026-02-24T22:20:30.215 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5daeffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961391 cookie 94692090565216 notifier 7529 bl.length()=989 2026-02-24T22:20:30.216 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5daeffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 info.flags=0x17 2026-02-24T22:20:30.216 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5daeffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-logrecord-writes:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 to cache LRU end 2026-02-24T22:20:30.216 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5daeffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:30.216 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5de2a7ea80 20 handle_bi_update(): bucket=:abort-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537]): orig_sources= new_sources= 2026-02-24T22:20:30.216 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5de2a7ea80 20 handle_bi_update(): bucket=:abort-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537]): potential sources added=[] removed=[] 2026-02-24T22:20:30.216 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5de2a7ea80 20 handle_bi_update(): bucket=:abort-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537]): orig_dests= new_dests= 2026-02-24T22:20:30.216 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.216+0000 7f5de2a7ea80 20 handle_bi_update(): bucket=:abort-at-logrecord-writes[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537]): potential dests added=[] removed=[] 2026-02-24T22:20:30.216 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.220+0000 7f5de2a7ea80 1 execute INFO: reshard of bucket "abort-at-logrecord-writes" from 1 shards to 2 shards completed successfully 2026-02-24T22:20:30.220 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.220+0000 7f5de2a7ea80 20 remove_watcher() i=2 2026-02-24T22:20:30.220 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.220+0000 7f5de2a7ea80 2 removed watcher, disabling cache 2026-02-24T22:20:30.220 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.220+0000 7f5de2a7ea80 20 remove_watcher() i=3 2026-02-24T22:20:30.220 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.220+0000 7f5de2a7ea80 20 remove_watcher() i=5 2026-02-24T22:20:30.220 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.220+0000 7f5de2a7ea80 20 remove_watcher() i=4 2026-02-24T22:20:30.220 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.220+0000 7f5de2a7ea80 20 remove_watcher() i=1 2026-02-24T22:20:30.220 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.220+0000 7f5de2a7ea80 20 remove_watcher() i=0 2026-02-24T22:20:30.221 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.220+0000 7f5de2a7ea80 20 remove_watcher() i=7 2026-02-24T22:20:30.221 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.220+0000 7f5de2a7ea80 20 remove_watcher() i=6 2026-02-24T22:20:30.227 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket abort-at-logrecord-writes 2026-02-24T22:20:30.241 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:20:30.241 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:20:30.259 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.260+0000 7f639362ca80 4 Realm: () 2026-02-24T22:20:30.259 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.260+0000 7f639362ca80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:20:30.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.260+0000 7f639362ca80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:20:30.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.260+0000 7f639362ca80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:20:30.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.260+0000 7f639362ca80 20 zonegroup default 2026-02-24T22:20:30.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.264+0000 7f639362ca80 20 rados->read ofs=0 len=0 2026-02-24T22:20:30.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.264+0000 7f639362ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:30.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.264+0000 7f639362ca80 20 rados->read ofs=0 len=0 2026-02-24T22:20:30.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.264+0000 7f639362ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:30.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.264+0000 7f639362ca80 20 started sync module instance, tier type = 2026-02-24T22:20:30.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.264+0000 7f639362ca80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:20:30.263 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.264+0000 7f639362ca80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:20:30.263 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.264+0000 7f639362ca80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:20:30.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.264+0000 7f639362ca80 5 note: GC not initialized 2026-02-24T22:20:30.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.264+0000 7f633efe5640 20 reqs_thread_entry: start 2026-02-24T22:20:30.266 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.268+0000 7f63908ff640 0 -- 10.20.193.46:0/2888474635 >> [v2:10.20.193.46:6826/3065971976,v1:10.20.193.46:6827/3065971976] conn(0x562355b6c550 msgr2=0x562355b6c920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:30.475 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.476+0000 7f63908ff640 0 -- 10.20.193.46:0/2888474635 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x562355b57a90 msgr2=0x562355b57ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:30.707 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.708+0000 7f639362ca80 20 init_complete bucket index max shards: 1 2026-02-24T22:20:30.707 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.708+0000 7f639362ca80 20 Filter name: none 2026-02-24T22:20:30.707 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.708+0000 7f633d7e2640 20 reqs_thread_entry: start 2026-02-24T22:20:30.707 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.708+0000 7f639362ca80 20 rados->read ofs=0 len=0 2026-02-24T22:20:30.707 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.712+0000 7f639362ca80 20 rados_obj.operate() r=0 bl.length=221 2026-02-24T22:20:30.708 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.712+0000 7f639362ca80 20 rados->read ofs=0 len=0 2026-02-24T22:20:30.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.712+0000 7f639362ca80 20 rados_obj.operate() r=0 bl.length=451 2026-02-24T22:20:30.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.712+0000 7f639362ca80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:20:30.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.712+0000 7f639362ca80 20 rados->read ofs=0 len=0 2026-02-24T22:20:30.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.712+0000 7f639362ca80 20 rados_obj.operate() r=0 bl.length=221 2026-02-24T22:20:30.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.712+0000 7f639362ca80 20 rados->read ofs=0 len=0 2026-02-24T22:20:30.710 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.712+0000 7f639362ca80 20 rados_obj.operate() r=0 bl.length=451 2026-02-24T22:20:30.710 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:30.712+0000 7f639362ca80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:20:30.720 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:30.720 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:20:30.720 INFO:tasks.workunit.client.0.trial046.stderr:bucket: abort-at-logrecord-writes id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537 num_objects: 19 size_kb: 1 num_shards: 2 2026-02-24T22:20:30.720 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:30.721 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:30.721 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:30.724 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:30.724 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:30.769 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:30.769 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:30.780 INFO:tasks.workunit.client.0.trial046.stderr:TEST: reshard bucket with EIO injected at change_reshard_state 2026-02-24T22:20:30.780 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:30.781 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:30.781 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:30.790 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:30.790 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:30.833 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:30.833 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:30.839 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:30.839 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:30.845 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:30.845 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:30.850 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:30.850 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:30.858 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:30.858 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:30.864 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:30.865 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:30.871 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:30.871 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:30.877 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:30.877 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:30.883 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:30.883 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:30.889 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:30.889 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:30.895 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:30.895 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:30.901 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:30.901 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:30.907 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:30.907 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:30.913 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:30.913 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:30.918 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:30.918 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:30.924 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:30.924 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:30.930 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:30.930 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:30.936 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:30.936 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:30.969 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:30.969 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:30.974 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:30.974 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:30.979 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-change-reshard-state 2026-02-24T22:20:30.993 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:20:30.993 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:20:31.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.012+0000 7f180729da80 4 Realm: () 2026-02-24T22:20:31.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.012+0000 7f180729da80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:20:31.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.012+0000 7f180729da80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:20:31.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.012+0000 7f180729da80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:20:31.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.012+0000 7f180729da80 20 zonegroup default 2026-02-24T22:20:31.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.012+0000 7f180729da80 20 rados->read ofs=0 len=0 2026-02-24T22:20:31.011 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.012+0000 7f180729da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:31.011 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.012+0000 7f180729da80 20 rados->read ofs=0 len=0 2026-02-24T22:20:31.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.016+0000 7f180729da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:31.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.016+0000 7f180729da80 20 started sync module instance, tier type = 2026-02-24T22:20:31.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.016+0000 7f180729da80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:20:31.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.016+0000 7f180729da80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:20:31.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.016+0000 7f180729da80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:20:31.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.016+0000 7f180729da80 5 note: GC not initialized 2026-02-24T22:20:31.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.016+0000 7f17b27e4640 20 reqs_thread_entry: start 2026-02-24T22:20:31.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.020+0000 7f1805572640 0 -- 10.20.193.46:0/3614668958 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55dbaade1ea0 msgr2=0x55dbaadd8700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:20:31.227 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.228+0000 7f1804d71640 0 -- 10.20.193.46:0/3614668958 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55dbaadb33f0 msgr2=0x55dbaade0f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:31.255 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.256+0000 7f180729da80 20 init_complete bucket index max shards: 1 2026-02-24T22:20:31.255 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.256+0000 7f180729da80 20 Filter name: none 2026-02-24T22:20:31.255 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.256+0000 7f17b0fe1640 20 reqs_thread_entry: start 2026-02-24T22:20:31.255 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.256+0000 7f180729da80 20 rados->read ofs=0 len=0 2026-02-24T22:20:31.256 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.256+0000 7f180729da80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:20:31.256 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.256+0000 7f180729da80 20 rados->read ofs=0 len=0 2026-02-24T22:20:31.256 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.260+0000 7f180729da80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:20:31.256 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.260+0000 7f180729da80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:20:31.257 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.260+0000 7f180729da80 20 rados->read ofs=0 len=0 2026-02-24T22:20:31.257 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.260+0000 7f180729da80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:20:31.257 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.260+0000 7f180729da80 20 rados->read ofs=0 len=0 2026-02-24T22:20:31.257 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.260+0000 7f180729da80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:20:31.257 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.260+0000 7f180729da80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:20:31.267 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:31.267 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:20:31.267 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-change-reshard-state id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:20:31.267 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:31.268 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket error-at-change-reshard-state --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at change_reshard_state 2026-02-24T22:20:31.285 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:20:31.285 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:20:31.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.304+0000 7fe9c9cc5a80 4 Realm: () 2026-02-24T22:20:31.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.304+0000 7fe9c9cc5a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:20:31.301 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.304+0000 7fe9c9cc5a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:20:31.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.304+0000 7fe9c9cc5a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:20:31.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.304+0000 7fe9c9cc5a80 20 zonegroup default 2026-02-24T22:20:31.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.304+0000 7fe9c9cc5a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:31.302 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.304+0000 7fe9c9cc5a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:31.303 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.304+0000 7fe9c9cc5a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:31.303 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.304+0000 7fe9c9cc5a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:31.303 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.304+0000 7fe9c9cc5a80 20 started sync module instance, tier type = 2026-02-24T22:20:31.303 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.304+0000 7fe9c9cc5a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:20:31.304 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.308+0000 7fe9c37fe640 0 -- 10.20.193.46:0/3499054769 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55ae0364a2b0 msgr2=0x55ae0366a690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:31.305 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.308+0000 7fe9c9cc5a80 20 add_watcher() i=1 2026-02-24T22:20:31.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.308+0000 7fe9c9cc5a80 20 add_watcher() i=5 2026-02-24T22:20:31.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.308+0000 7fe9c9cc5a80 20 add_watcher() i=0 2026-02-24T22:20:31.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.308+0000 7fe9c9cc5a80 20 add_watcher() i=3 2026-02-24T22:20:31.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.308+0000 7fe9c9cc5a80 20 add_watcher() i=2 2026-02-24T22:20:31.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.308+0000 7fe9c9cc5a80 20 add_watcher() i=7 2026-02-24T22:20:31.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.308+0000 7fe9c9cc5a80 20 add_watcher() i=6 2026-02-24T22:20:31.306 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.308+0000 7fe9c9cc5a80 20 add_watcher() i=4 2026-02-24T22:20:31.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.308+0000 7fe9c9cc5a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:20:31.307 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.308+0000 7fe9c9cc5a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:20:31.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.308+0000 7fe9c9cc5a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:20:31.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.308+0000 7fe9c9cc5a80 5 note: GC not initialized 2026-02-24T22:20:31.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.308+0000 7fe8dffff640 20 reqs_thread_entry: start 2026-02-24T22:20:31.308 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.312+0000 7fe9c37fe640 0 -- 10.20.193.46:0/3956027339 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55ae03738720 msgr2=0x55ae03738af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:31.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.548+0000 7fe9c9cc5a80 20 init_complete bucket index max shards: 1 2026-02-24T22:20:31.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.548+0000 7fe9c9cc5a80 20 Filter name: none 2026-02-24T22:20:31.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.552+0000 7fe8de7fc640 20 reqs_thread_entry: start 2026-02-24T22:20:31.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.552+0000 7fe9c9cc5a80 10 cache get: name=default.rgw.meta+root+error-at-change-reshard-state : miss 2026-02-24T22:20:31.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.552+0000 7fe9c9cc5a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:31.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.552+0000 7fe9c9cc5a80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:20:31.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.552+0000 7fe9c9cc5a80 10 cache put: name=default.rgw.meta+root+error-at-change-reshard-state info.flags=0x11 2026-02-24T22:20:31.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.552+0000 7fe9c9cc5a80 10 adding default.rgw.meta+root+error-at-change-reshard-state to cache LRU end 2026-02-24T22:20:31.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.552+0000 7fe9c9cc5a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 : miss 2026-02-24T22:20:31.549 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.552+0000 7fe9c9cc5a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:31.550 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.552+0000 7fe9c9cc5a80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:20:31.550 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.552+0000 7fe9c9cc5a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 info.flags=0x17 2026-02-24T22:20:31.550 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.552+0000 7fe9c9cc5a80 10 adding default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 to cache LRU end 2026-02-24T22:20:31.550 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.552+0000 7fe9c9cc5a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:20:31.550 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.552+0000 7fe9c9cc5a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:31.550 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.552+0000 7fe9c9cc5a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 2026-02-24T22:20:31.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.556+0000 7fe9c9cc5a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 info.flags=0x17 2026-02-24T22:20:31.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.556+0000 7fe9c9cc5a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 to cache LRU end 2026-02-24T22:20:31.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.556+0000 7fe9c9cc5a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:31.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.556+0000 7fe9c9cc5a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538, ofs0, ns] 2026-02-24T22:20:31.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.556+0000 7fe9a1ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959930 cookie 94205869604400 notifier 7580 bl.length()=1033 2026-02-24T22:20:31.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.556+0000 7fe9a1ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 info.flags=0x17 2026-02-24T22:20:31.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.556+0000 7fe9a1ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 to cache LRU end 2026-02-24T22:20:31.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.556+0000 7fe9a1ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:31.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.556+0000 7fe9c9cc5a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538]): orig_sources= new_sources= 2026-02-24T22:20:31.554 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.556+0000 7fe9c9cc5a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538]): potential sources added=[] removed=[] 2026-02-24T22:20:31.555 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.556+0000 7fe9c9cc5a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538]): orig_dests= new_dests= 2026-02-24T22:20:31.555 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.556+0000 7fe9c9cc5a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538]): potential dests added=[] removed=[] 2026-02-24T22:20:31.555 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.556+0000 7fe9c9cc5a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538.1.0 2026-02-24T22:20:31.555 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.556+0000 7fe9c9cc5a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538.1.0 2026-02-24T22:20:31.556 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.556+0000 7fe9c9cc5a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538.1.1 2026-02-24T22:20:31.556 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.556+0000 7fe9c9cc5a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538.1.1 2026-02-24T22:20:31.556 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.556+0000 7fe9c9cc5a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538.0 2026-02-24T22:20:31.557 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.560+0000 7fe9c9cc5a80 -1 Injecting error=-5 at location=change_reshard_state 2026-02-24T22:20:31.557 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.560+0000 7fe9c9cc5a80 0 ERROR: change_reshard_state failed to commit target index layout: (5) Input/output error 2026-02-24T22:20:31.560 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.564+0000 7fe9c9cc5a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 : hit (requested=0x17, cached=0x17) 2026-02-24T22:20:31.560 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.564+0000 7fe9c9cc5a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 2026-02-24T22:20:31.562 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.564+0000 7fe9c9cc5a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 info.flags=0x17 2026-02-24T22:20:31.562 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.564+0000 7fe9c9cc5a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 to cache LRU end 2026-02-24T22:20:31.562 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.564+0000 7fe9c9cc5a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:31.562 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.564+0000 7fe9c9cc5a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538, ofs0, ns] 2026-02-24T22:20:31.562 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.564+0000 7fe9a27fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959931 cookie 94205869604400 notifier 7580 bl.length()=997 2026-02-24T22:20:31.562 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.564+0000 7fe9a27fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 info.flags=0x17 2026-02-24T22:20:31.562 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.564+0000 7fe9a27fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 to cache LRU end 2026-02-24T22:20:31.562 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.564+0000 7fe9a27fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:31.563 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.564+0000 7fe9c9cc5a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538]): orig_sources= new_sources= 2026-02-24T22:20:31.563 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.564+0000 7fe9c9cc5a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538]): potential sources added=[] removed=[] 2026-02-24T22:20:31.563 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.564+0000 7fe9c9cc5a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538]): orig_dests= new_dests= 2026-02-24T22:20:31.563 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.564+0000 7fe9c9cc5a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538]): potential dests added=[] removed=[] 2026-02-24T22:20:31.563 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.564+0000 7fe9c9cc5a80 1 execute INFO: reshard of bucket "error-at-change-reshard-state" canceled due to errors 2026-02-24T22:20:31.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.568+0000 7fe9c9cc5a80 20 remove_watcher() i=3 2026-02-24T22:20:31.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.568+0000 7fe9c9cc5a80 2 removed watcher, disabling cache 2026-02-24T22:20:31.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.568+0000 7fe9c9cc5a80 20 remove_watcher() i=2 2026-02-24T22:20:31.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.568+0000 7fe9c9cc5a80 20 remove_watcher() i=6 2026-02-24T22:20:31.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.568+0000 7fe9c9cc5a80 20 remove_watcher() i=4 2026-02-24T22:20:31.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.568+0000 7fe9c9cc5a80 20 remove_watcher() i=0 2026-02-24T22:20:31.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.568+0000 7fe9c9cc5a80 20 remove_watcher() i=1 2026-02-24T22:20:31.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.568+0000 7fe9c9cc5a80 20 remove_watcher() i=5 2026-02-24T22:20:31.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.568+0000 7fe9c9cc5a80 20 remove_watcher() i=7 2026-02-24T22:20:31.573 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-change-reshard-state 2026-02-24T22:20:31.587 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:20:31.587 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:20:31.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.608+0000 7f1fd76c8a80 4 Realm: () 2026-02-24T22:20:31.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.608+0000 7f1fd76c8a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:20:31.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.608+0000 7f1fd76c8a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:20:31.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.608+0000 7f1fd76c8a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:20:31.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.608+0000 7f1fd76c8a80 20 zonegroup default 2026-02-24T22:20:31.606 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.608+0000 7f1fd76c8a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:31.606 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.608+0000 7f1fd76c8a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:31.606 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.608+0000 7f1fd76c8a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:31.607 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.608+0000 7f1fd76c8a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:31.607 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.608+0000 7f1fd76c8a80 20 started sync module instance, tier type = 2026-02-24T22:20:31.607 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.608+0000 7f1fd76c8a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:20:31.609 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.612+0000 7f1fd76c8a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:20:31.609 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.612+0000 7f1fd76c8a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:20:31.609 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.612+0000 7f1fd76c8a80 5 note: GC not initialized 2026-02-24T22:20:31.609 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.612+0000 7f1f82fe5640 20 reqs_thread_entry: start 2026-02-24T22:20:31.612 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.616+0000 7f1fd519c640 0 -- 10.20.193.46:0/2463642712 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55667a70f0c0 msgr2=0x55667a7068a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:31.824 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:31.828+0000 7f1fd599d640 0 -- 10.20.193.46:0/2463642712 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x55667a6f7d20 msgr2=0x55667a6f81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:32.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.056+0000 7f1fd76c8a80 20 init_complete bucket index max shards: 1 2026-02-24T22:20:32.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.056+0000 7f1fd76c8a80 20 Filter name: none 2026-02-24T22:20:32.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.056+0000 7f1f817e2640 20 reqs_thread_entry: start 2026-02-24T22:20:32.053 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.056+0000 7f1fd76c8a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:32.054 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.056+0000 7f1fd76c8a80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:20:32.054 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.056+0000 7f1fd76c8a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:32.054 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.056+0000 7f1fd76c8a80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:20:32.054 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.056+0000 7f1fd76c8a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:20:32.054 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.056+0000 7f1fd76c8a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:32.055 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.056+0000 7f1fd76c8a80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:20:32.055 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.056+0000 7f1fd76c8a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:32.055 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.056+0000 7f1fd76c8a80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:20:32.055 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.056+0000 7f1fd76c8a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:20:32.064 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:32.064 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:20:32.064 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-change-reshard-state id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:20:32.064 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:32.065 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.065 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.072 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.072 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.116 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket error-at-change-reshard-state --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2026-02-24T22:20:32.133 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:20:32.133 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:20:32.151 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.152+0000 7fa482739a80 4 Realm: () 2026-02-24T22:20:32.151 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.152+0000 7fa482739a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:20:32.151 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.152+0000 7fa482739a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:20:32.151 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.152+0000 7fa482739a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:20:32.151 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.152+0000 7fa482739a80 20 zonegroup default 2026-02-24T22:20:32.152 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.156+0000 7fa482739a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:32.153 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.156+0000 7fa482739a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:32.153 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.156+0000 7fa482739a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:32.153 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.156+0000 7fa482739a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:32.154 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.156+0000 7fa482739a80 20 started sync module instance, tier type = 2026-02-24T22:20:32.154 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.156+0000 7fa482739a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:20:32.154 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.156+0000 7fa47b7fe640 0 -- 10.20.193.46:0/3090923833 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x564ff699a2b0 msgr2=0x564ff69ba730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:32.155 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.156+0000 7fa482739a80 20 add_watcher() i=1 2026-02-24T22:20:32.155 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.156+0000 7fa482739a80 20 add_watcher() i=3 2026-02-24T22:20:32.156 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.156+0000 7fa482739a80 20 add_watcher() i=0 2026-02-24T22:20:32.156 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.156+0000 7fa482739a80 20 add_watcher() i=6 2026-02-24T22:20:32.156 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.156+0000 7fa482739a80 20 add_watcher() i=5 2026-02-24T22:20:32.156 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.156+0000 7fa482739a80 20 add_watcher() i=2 2026-02-24T22:20:32.156 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.156+0000 7fa482739a80 20 add_watcher() i=7 2026-02-24T22:20:32.359 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.360+0000 7fa482739a80 20 add_watcher() i=4 2026-02-24T22:20:32.359 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.360+0000 7fa482739a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:20:32.361 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.364+0000 7fa47b7fe640 0 -- 10.20.193.46:0/1846482573 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7fa3a0008af0 msgr2=0x7fa3a0028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:32.362 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.364+0000 7fa482739a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:20:32.362 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.364+0000 7fa482739a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:20:32.362 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.364+0000 7fa482739a80 5 note: GC not initialized 2026-02-24T22:20:32.362 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.364+0000 7fa42d7e2640 20 reqs_thread_entry: start 2026-02-24T22:20:32.395 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.396+0000 7fa482739a80 20 init_complete bucket index max shards: 1 2026-02-24T22:20:32.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.396+0000 7fa482739a80 20 Filter name: none 2026-02-24T22:20:32.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.396+0000 7fa3977fe640 20 reqs_thread_entry: start 2026-02-24T22:20:32.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.396+0000 7fa482739a80 10 cache get: name=default.rgw.meta+root+error-at-change-reshard-state : miss 2026-02-24T22:20:32.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.396+0000 7fa482739a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:32.397 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.400+0000 7fa482739a80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:20:32.397 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.400+0000 7fa482739a80 10 cache put: name=default.rgw.meta+root+error-at-change-reshard-state info.flags=0x11 2026-02-24T22:20:32.397 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.400+0000 7fa482739a80 10 adding default.rgw.meta+root+error-at-change-reshard-state to cache LRU end 2026-02-24T22:20:32.397 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.400+0000 7fa482739a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 : miss 2026-02-24T22:20:32.397 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.400+0000 7fa482739a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:32.397 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.400+0000 7fa482739a80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:20:32.397 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.400+0000 7fa482739a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 info.flags=0x17 2026-02-24T22:20:32.397 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.400+0000 7fa482739a80 10 adding default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 to cache LRU end 2026-02-24T22:20:32.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.400+0000 7fa482739a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:20:32.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.400+0000 7fa482739a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:32.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.400+0000 7fa482739a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 2026-02-24T22:20:32.401 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.404+0000 7fa482739a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 info.flags=0x17 2026-02-24T22:20:32.401 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.404+0000 7fa482739a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 to cache LRU end 2026-02-24T22:20:32.401 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.404+0000 7fa482739a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:32.401 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.404+0000 7fa482739a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538, ofs0, ns] 2026-02-24T22:20:32.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.404+0000 7fa4577fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959932 cookie 94901440482320 notifier 7610 bl.length()=1033 2026-02-24T22:20:32.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.404+0000 7fa4577fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 info.flags=0x17 2026-02-24T22:20:32.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.404+0000 7fa4577fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 to cache LRU end 2026-02-24T22:20:32.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.404+0000 7fa4577fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:32.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.404+0000 7fa482739a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538]): orig_sources= new_sources= 2026-02-24T22:20:32.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.404+0000 7fa482739a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538]): potential sources added=[] removed=[] 2026-02-24T22:20:32.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.404+0000 7fa482739a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538]): orig_dests= new_dests= 2026-02-24T22:20:32.402 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.404+0000 7fa482739a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538]): potential dests added=[] removed=[] 2026-02-24T22:20:32.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.404+0000 7fa482739a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538.1.0 2026-02-24T22:20:32.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.404+0000 7fa482739a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538.1.0 2026-02-24T22:20:32.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.404+0000 7fa482739a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538.1.1 2026-02-24T22:20:32.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.404+0000 7fa482739a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538.1.1 2026-02-24T22:20:32.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.404+0000 7fa482739a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538.0 2026-02-24T22:20:32.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.408+0000 7fa482739a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 : hit (requested=0x17, cached=0x17) 2026-02-24T22:20:32.404 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.408+0000 7fa482739a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 2026-02-24T22:20:32.405 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.408+0000 7fa482739a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 info.flags=0x17 2026-02-24T22:20:32.405 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.408+0000 7fa482739a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 to cache LRU end 2026-02-24T22:20:32.405 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.408+0000 7fa482739a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:32.406 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.408+0000 7fa482739a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538, ofs0, ns] 2026-02-24T22:20:32.406 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.408+0000 7fa4577fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959933 cookie 94901440482320 notifier 7610 bl.length()=1033 2026-02-24T22:20:32.406 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.408+0000 7fa4577fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 info.flags=0x17 2026-02-24T22:20:32.406 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.408+0000 7fa4577fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 to cache LRU end 2026-02-24T22:20:32.406 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.408+0000 7fa4577fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:32.406 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.408+0000 7fa482739a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538]): orig_sources= new_sources= 2026-02-24T22:20:32.407 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.408+0000 7fa482739a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538]): potential sources added=[] removed=[] 2026-02-24T22:20:32.407 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.408+0000 7fa482739a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538]): orig_dests= new_dests= 2026-02-24T22:20:32.407 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.408+0000 7fa482739a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538]): potential dests added=[] removed=[] 2026-02-24T22:20:32.407 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.408+0000 7fa482739a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538.0 2026-02-24T22:20:32.407 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.408+0000 7fa482739a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 : hit (requested=0x17, cached=0x17) 2026-02-24T22:20:32.408 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.408+0000 7fa482739a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 2026-02-24T22:20:32.408 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.412+0000 7fa482739a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 info.flags=0x17 2026-02-24T22:20:32.408 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.412+0000 7fa482739a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 to cache LRU end 2026-02-24T22:20:32.409 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.412+0000 7fa482739a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:32.409 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.412+0000 7fa482739a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538, ofs0, ns] 2026-02-24T22:20:32.409 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.412+0000 7fa4577fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959934 cookie 94901440482320 notifier 7610 bl.length()=997 2026-02-24T22:20:32.409 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.412+0000 7fa4577fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 info.flags=0x17 2026-02-24T22:20:32.409 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.412+0000 7fa4577fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 to cache LRU end 2026-02-24T22:20:32.409 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.412+0000 7fa4577fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:32.409 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.412+0000 7fa482739a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538]): orig_sources= new_sources= 2026-02-24T22:20:32.409 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.412+0000 7fa482739a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538]): potential sources added=[] removed=[] 2026-02-24T22:20:32.409 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.412+0000 7fa482739a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538]): orig_dests= new_dests= 2026-02-24T22:20:32.409 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.412+0000 7fa482739a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538]): potential dests added=[] removed=[] 2026-02-24T22:20:32.410 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.412+0000 7fa482739a80 1 execute INFO: reshard of bucket "error-at-change-reshard-state" from 1 shards to 2 shards completed successfully 2026-02-24T22:20:32.413 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.416+0000 7fa482739a80 20 remove_watcher() i=3 2026-02-24T22:20:32.413 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.416+0000 7fa482739a80 2 removed watcher, disabling cache 2026-02-24T22:20:32.413 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.416+0000 7fa482739a80 20 remove_watcher() i=6 2026-02-24T22:20:32.413 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.416+0000 7fa482739a80 20 remove_watcher() i=0 2026-02-24T22:20:32.413 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.416+0000 7fa482739a80 20 remove_watcher() i=2 2026-02-24T22:20:32.413 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.416+0000 7fa482739a80 20 remove_watcher() i=5 2026-02-24T22:20:32.414 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.416+0000 7fa482739a80 20 remove_watcher() i=7 2026-02-24T22:20:32.414 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.416+0000 7fa482739a80 20 remove_watcher() i=1 2026-02-24T22:20:32.414 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.416+0000 7fa482739a80 20 remove_watcher() i=4 2026-02-24T22:20:32.420 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-change-reshard-state 2026-02-24T22:20:32.436 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:20:32.436 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:20:32.453 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.456+0000 7f4cfa096a80 4 Realm: () 2026-02-24T22:20:32.453 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.456+0000 7f4cfa096a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:20:32.454 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.456+0000 7f4cfa096a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:20:32.454 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.456+0000 7f4cfa096a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:20:32.454 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.456+0000 7f4cfa096a80 20 zonegroup default 2026-02-24T22:20:32.454 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.456+0000 7f4cfa096a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:32.455 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.456+0000 7f4cfa096a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:32.455 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.456+0000 7f4cfa096a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:32.456 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.460+0000 7f4cfa096a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:32.456 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.460+0000 7f4cfa096a80 20 started sync module instance, tier type = 2026-02-24T22:20:32.456 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.460+0000 7f4cfa096a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:20:32.457 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.460+0000 7f4cfa096a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:20:32.458 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.460+0000 7f4cfa096a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:20:32.458 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.460+0000 7f4cfa096a80 5 note: GC not initialized 2026-02-24T22:20:32.458 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.460+0000 7f4ca57e2640 20 reqs_thread_entry: start 2026-02-24T22:20:32.461 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.464+0000 7f4cf3fff640 0 -- 10.20.193.46:0/2251529667 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x555bf40629f0 msgr2=0x555bf405c340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:32.466 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.468+0000 7f4cf37fe640 0 -- 10.20.193.46:0/2251529667 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x555bf4036f10 msgr2=0x555bf4064b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:32.491 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.492+0000 7f4cfa096a80 20 init_complete bucket index max shards: 1 2026-02-24T22:20:32.491 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.492+0000 7f4cfa096a80 20 Filter name: none 2026-02-24T22:20:32.491 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.492+0000 7f4c0f7fe640 20 reqs_thread_entry: start 2026-02-24T22:20:32.491 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.492+0000 7f4cfa096a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:32.492 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.492+0000 7f4cfa096a80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:20:32.492 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.492+0000 7f4cfa096a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:32.492 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.496+0000 7f4cfa096a80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:20:32.493 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.496+0000 7f4cfa096a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:20:32.493 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.496+0000 7f4cfa096a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:32.493 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.496+0000 7f4cfa096a80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:20:32.493 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.496+0000 7f4cfa096a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:32.493 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.496+0000 7f4cfa096a80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:20:32.493 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.496+0000 7f4cfa096a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:20:32.503 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:32.503 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:20:32.503 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-change-reshard-state id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538 num_objects: 19 size_kb: 1 num_shards: 2 2026-02-24T22:20:32.503 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:32.504 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.504 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.507 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.507 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.553 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.553 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.564 INFO:tasks.workunit.client.0.trial046.stderr:TEST: reshard bucket with ECANCELED injected at change_reshard_state 2026-02-24T22:20:32.564 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:32.564 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.565 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.573 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.573 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.617 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.617 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.623 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.623 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.629 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.629 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.635 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.635 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.641 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.641 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.647 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.647 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.653 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.653 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.659 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.659 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.665 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.665 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.671 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.671 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.677 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.677 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.683 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.683 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.688 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.689 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.694 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.694 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.700 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.700 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.706 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.706 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.712 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.712 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.718 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.718 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.724 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.724 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.730 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:32.730 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:32.735 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-change-reshard-state 2026-02-24T22:20:32.749 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:20:32.749 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:20:32.766 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.768+0000 7f621ef7aa80 4 Realm: () 2026-02-24T22:20:32.767 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.768+0000 7f621ef7aa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:20:32.767 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.768+0000 7f621ef7aa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:20:32.767 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.768+0000 7f621ef7aa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:20:32.767 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.768+0000 7f621ef7aa80 20 zonegroup default 2026-02-24T22:20:32.767 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.768+0000 7f621ef7aa80 20 rados->read ofs=0 len=0 2026-02-24T22:20:32.769 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.772+0000 7f621ef7aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:32.769 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.772+0000 7f621ef7aa80 20 rados->read ofs=0 len=0 2026-02-24T22:20:32.770 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.772+0000 7f621ef7aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:32.770 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.772+0000 7f621ef7aa80 20 started sync module instance, tier type = 2026-02-24T22:20:32.770 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.772+0000 7f621ef7aa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:20:32.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.772+0000 7f621ef7aa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:20:32.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.772+0000 7f621ef7aa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:20:32.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.772+0000 7f621ef7aa80 5 note: GC not initialized 2026-02-24T22:20:32.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.772+0000 7f61ca7e4640 20 reqs_thread_entry: start 2026-02-24T22:20:32.775 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.776+0000 7f621ca4e640 0 -- 10.20.193.46:0/2093151746 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x557c252e6a50 msgr2=0x557c252e6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:20:32.985 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:32.988+0000 7f621d24f640 0 -- 10.20.193.46:0/2093151746 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x557c252d5f10 msgr2=0x557c252d81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:33.214 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.216+0000 7f621ef7aa80 20 init_complete bucket index max shards: 1 2026-02-24T22:20:33.214 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.216+0000 7f621ef7aa80 20 Filter name: none 2026-02-24T22:20:33.214 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.216+0000 7f61c8fe1640 20 reqs_thread_entry: start 2026-02-24T22:20:33.214 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.216+0000 7f621ef7aa80 20 rados->read ofs=0 len=0 2026-02-24T22:20:33.215 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.216+0000 7f621ef7aa80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:20:33.215 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.216+0000 7f621ef7aa80 20 rados->read ofs=0 len=0 2026-02-24T22:20:33.216 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.220+0000 7f621ef7aa80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:20:33.216 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.220+0000 7f621ef7aa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:20:33.217 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.220+0000 7f621ef7aa80 20 rados->read ofs=0 len=0 2026-02-24T22:20:33.217 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.220+0000 7f621ef7aa80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:20:33.217 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.220+0000 7f621ef7aa80 20 rados->read ofs=0 len=0 2026-02-24T22:20:33.217 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.220+0000 7f621ef7aa80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:20:33.217 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.220+0000 7f621ef7aa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:20:33.227 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:33.227 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:20:33.228 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-change-reshard-state id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:20:33.228 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:33.228 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket error-at-change-reshard-state --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at change_reshard_state --inject-error-code 125 2026-02-24T22:20:33.245 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:20:33.245 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:20:33.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.264+0000 7f208e384a80 4 Realm: () 2026-02-24T22:20:33.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.264+0000 7f208e384a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:20:33.261 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.264+0000 7f208e384a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:20:33.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.264+0000 7f208e384a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:20:33.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.264+0000 7f208e384a80 20 zonegroup default 2026-02-24T22:20:33.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.264+0000 7f208e384a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:33.263 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.264+0000 7f208e384a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:33.263 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.264+0000 7f208e384a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:33.263 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.264+0000 7f208e384a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:33.263 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.264+0000 7f208e384a80 20 started sync module instance, tier type = 2026-02-24T22:20:33.263 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.264+0000 7f208e384a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:20:33.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.268+0000 7f2087fff640 0 -- 10.20.193.46:0/619699280 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55bec49b2cb0 msgr2=0x55bec49d3090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:33.265 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.268+0000 7f208e384a80 20 add_watcher() i=1 2026-02-24T22:20:33.266 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.268+0000 7f208e384a80 20 add_watcher() i=6 2026-02-24T22:20:33.266 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.268+0000 7f208e384a80 20 add_watcher() i=5 2026-02-24T22:20:33.266 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.268+0000 7f208e384a80 20 add_watcher() i=3 2026-02-24T22:20:33.266 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.268+0000 7f208e384a80 20 add_watcher() i=0 2026-02-24T22:20:33.266 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.268+0000 7f208e384a80 20 add_watcher() i=7 2026-02-24T22:20:33.266 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.268+0000 7f208e384a80 20 add_watcher() i=4 2026-02-24T22:20:33.267 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.268+0000 7f208e384a80 20 add_watcher() i=2 2026-02-24T22:20:33.267 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.268+0000 7f208e384a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:20:33.267 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.268+0000 7f208e384a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:20:33.268 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.268+0000 7f208e384a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:20:33.268 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.268+0000 7f208e384a80 5 note: GC not initialized 2026-02-24T22:20:33.268 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.268+0000 7f2038fe1640 20 reqs_thread_entry: start 2026-02-24T22:20:33.268 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.272+0000 7f2087fff640 0 -- 10.20.193.46:0/2571241660 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55bec4aa8a00 msgr2=0x55bec4aa3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:33.510 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.512+0000 7f208e384a80 20 init_complete bucket index max shards: 1 2026-02-24T22:20:33.510 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.512+0000 7f208e384a80 20 Filter name: none 2026-02-24T22:20:33.510 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.512+0000 7f1fa37fe640 20 reqs_thread_entry: start 2026-02-24T22:20:33.510 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.512+0000 7f208e384a80 10 cache get: name=default.rgw.meta+root+error-at-change-reshard-state : miss 2026-02-24T22:20:33.510 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.512+0000 7f208e384a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:33.510 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.512+0000 7f208e384a80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:20:33.510 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.512+0000 7f208e384a80 10 cache put: name=default.rgw.meta+root+error-at-change-reshard-state info.flags=0x11 2026-02-24T22:20:33.510 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.512+0000 7f208e384a80 10 adding default.rgw.meta+root+error-at-change-reshard-state to cache LRU end 2026-02-24T22:20:33.511 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.512+0000 7f208e384a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539 : miss 2026-02-24T22:20:33.511 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.512+0000 7f208e384a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:33.511 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.512+0000 7f208e384a80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:20:33.511 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.512+0000 7f208e384a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539 info.flags=0x17 2026-02-24T22:20:33.512 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.512+0000 7f208e384a80 10 adding default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539 to cache LRU end 2026-02-24T22:20:33.512 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.512+0000 7f208e384a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:20:33.512 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.512+0000 7f208e384a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:33.512 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.512+0000 7f208e384a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539 2026-02-24T22:20:33.514 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.516+0000 7f208e384a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539 info.flags=0x17 2026-02-24T22:20:33.514 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.516+0000 7f208e384a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539 to cache LRU end 2026-02-24T22:20:33.514 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.516+0000 7f208e384a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:33.515 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.516+0000 7f208e384a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539, ofs0, ns] 2026-02-24T22:20:33.515 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.516+0000 7f2062ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959937 cookie 94277832271712 notifier 7661 bl.length()=1033 2026-02-24T22:20:33.515 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.516+0000 7f2062ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539 info.flags=0x17 2026-02-24T22:20:33.515 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.516+0000 7f2062ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539 to cache LRU end 2026-02-24T22:20:33.515 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.516+0000 7f2062ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:33.515 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.516+0000 7f208e384a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539]): orig_sources= new_sources= 2026-02-24T22:20:33.516 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.516+0000 7f208e384a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539]): potential sources added=[] removed=[] 2026-02-24T22:20:33.516 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.516+0000 7f208e384a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539]): orig_dests= new_dests= 2026-02-24T22:20:33.516 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.516+0000 7f208e384a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539]): potential dests added=[] removed=[] 2026-02-24T22:20:33.517 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.520+0000 7f208e384a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539.1.0 2026-02-24T22:20:33.517 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.520+0000 7f208e384a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539.1.0 2026-02-24T22:20:33.517 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.520+0000 7f208e384a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539.1.1 2026-02-24T22:20:33.517 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.520+0000 7f208e384a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539.1.1 2026-02-24T22:20:33.517 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.520+0000 7f208e384a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539.0 2026-02-24T22:20:33.518 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.520+0000 7f208e384a80 -1 Injecting error=-125 at location=change_reshard_state 2026-02-24T22:20:33.518 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.520+0000 7f208e384a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539 : hit (requested=0x17, cached=0x17) 2026-02-24T22:20:33.518 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.520+0000 7f208e384a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539 2026-02-24T22:20:33.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.520+0000 7f208e384a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539 info.flags=0x17 2026-02-24T22:20:33.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.520+0000 7f208e384a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539 to cache LRU end 2026-02-24T22:20:33.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.520+0000 7f208e384a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:33.519 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.520+0000 7f208e384a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539, ofs0, ns] 2026-02-24T22:20:33.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.520+0000 7f20627fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959938 cookie 94277832271712 notifier 7661 bl.length()=1033 2026-02-24T22:20:33.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.520+0000 7f20627fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539 info.flags=0x17 2026-02-24T22:20:33.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.520+0000 7f20627fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539 to cache LRU end 2026-02-24T22:20:33.520 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.520+0000 7f20627fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:33.521 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.520+0000 7f208e384a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539]): orig_sources= new_sources= 2026-02-24T22:20:33.521 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.520+0000 7f208e384a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539]): potential sources added=[] removed=[] 2026-02-24T22:20:33.521 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.520+0000 7f208e384a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539]): orig_dests= new_dests= 2026-02-24T22:20:33.521 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.520+0000 7f208e384a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539]): potential dests added=[] removed=[] 2026-02-24T22:20:33.521 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.524+0000 7f208e384a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539.0 2026-02-24T22:20:33.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.524+0000 7f208e384a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539 : hit (requested=0x17, cached=0x17) 2026-02-24T22:20:33.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.524+0000 7f208e384a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539 2026-02-24T22:20:33.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.524+0000 7f208e384a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539 info.flags=0x17 2026-02-24T22:20:33.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.524+0000 7f208e384a80 10 moving default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539 to cache LRU end 2026-02-24T22:20:33.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.524+0000 7f208e384a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:33.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.524+0000 7f208e384a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539, ofs0, ns] 2026-02-24T22:20:33.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.524+0000 7f2062ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959939 cookie 94277832271712 notifier 7661 bl.length()=997 2026-02-24T22:20:33.522 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.524+0000 7f2062ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539 info.flags=0x17 2026-02-24T22:20:33.523 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.524+0000 7f2062ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539 to cache LRU end 2026-02-24T22:20:33.523 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.524+0000 7f2062ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:33.523 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.524+0000 7f208e384a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539]): orig_sources= new_sources= 2026-02-24T22:20:33.523 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.524+0000 7f208e384a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539]): potential sources added=[] removed=[] 2026-02-24T22:20:33.523 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.524+0000 7f208e384a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539]): orig_dests= new_dests= 2026-02-24T22:20:33.523 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.524+0000 7f208e384a80 20 handle_bi_update(): bucket=:error-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539]): potential dests added=[] removed=[] 2026-02-24T22:20:33.524 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.528+0000 7f208e384a80 1 execute INFO: reshard of bucket "error-at-change-reshard-state" from 1 shards to 2 shards completed successfully 2026-02-24T22:20:33.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.528+0000 7f208e384a80 20 remove_watcher() i=3 2026-02-24T22:20:33.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.528+0000 7f208e384a80 2 removed watcher, disabling cache 2026-02-24T22:20:33.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.528+0000 7f208e384a80 20 remove_watcher() i=2 2026-02-24T22:20:33.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.528+0000 7f208e384a80 20 remove_watcher() i=0 2026-02-24T22:20:33.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.528+0000 7f208e384a80 20 remove_watcher() i=4 2026-02-24T22:20:33.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.528+0000 7f208e384a80 20 remove_watcher() i=6 2026-02-24T22:20:33.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.528+0000 7f208e384a80 20 remove_watcher() i=1 2026-02-24T22:20:33.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.528+0000 7f208e384a80 20 remove_watcher() i=7 2026-02-24T22:20:33.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.528+0000 7f208e384a80 20 remove_watcher() i=5 2026-02-24T22:20:33.534 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket error-at-change-reshard-state 2026-02-24T22:20:33.548 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:20:33.549 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:20:33.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.568+0000 7f06a0900a80 4 Realm: () 2026-02-24T22:20:33.566 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.568+0000 7f06a0900a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:20:33.567 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.568+0000 7f06a0900a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:20:33.567 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.568+0000 7f06a0900a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:20:33.567 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.568+0000 7f06a0900a80 20 zonegroup default 2026-02-24T22:20:33.567 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.568+0000 7f06a0900a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:33.568 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.572+0000 7f06a0900a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:33.568 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.572+0000 7f06a0900a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:33.569 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.572+0000 7f06a0900a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:33.569 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.572+0000 7f06a0900a80 20 started sync module instance, tier type = 2026-02-24T22:20:33.569 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.572+0000 7f06a0900a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:20:33.570 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.572+0000 7f06a0900a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:20:33.571 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.572+0000 7f06a0900a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:20:33.571 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.572+0000 7f06a0900a80 5 note: GC not initialized 2026-02-24T22:20:33.571 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.572+0000 7f05bffff640 20 reqs_thread_entry: start 2026-02-24T22:20:33.572 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.576+0000 7f069dbd3640 0 -- 10.20.193.46:0/1508452018 >> [v2:10.20.193.46:6826/3065971976,v1:10.20.193.46:6827/3065971976] conn(0x55feef9fdcb0 msgr2=0x55feef9f5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:20:33.782 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:33.784+0000 7f069dbd3640 0 -- 10.20.193.46:0/1508452018 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x55feef9f8540 msgr2=0x55feef9e0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:34.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.016+0000 7f06a0900a80 20 init_complete bucket index max shards: 1 2026-02-24T22:20:34.012 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.016+0000 7f06a0900a80 20 Filter name: none 2026-02-24T22:20:34.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.016+0000 7f05be7fc640 20 reqs_thread_entry: start 2026-02-24T22:20:34.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.016+0000 7f06a0900a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:34.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.016+0000 7f06a0900a80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:20:34.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.016+0000 7f06a0900a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:34.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.016+0000 7f06a0900a80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:20:34.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.016+0000 7f06a0900a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:20:34.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.016+0000 7f06a0900a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:34.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.016+0000 7f06a0900a80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:20:34.015 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.016+0000 7f06a0900a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:34.015 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.016+0000 7f06a0900a80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:20:34.015 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.016+0000 7f06a0900a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:20:34.024 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.025 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:20:34.025 INFO:tasks.workunit.client.0.trial046.stderr:bucket: error-at-change-reshard-state id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539 num_objects: 20 size_kb: 1 num_shards: 2 2026-02-24T22:20:34.025 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.025 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:34.026 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:34.027 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:34.027 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:34.073 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:34.073 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:34.084 INFO:tasks.workunit.client.0.trial046.stderr:TEST: reshard bucket with abort at change_reshard_state 2026-02-24T22:20:34.084 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.085 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:34.085 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:34.093 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:34.093 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:34.137 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:34.137 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:34.143 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:34.143 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:34.150 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:34.150 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:34.156 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:34.156 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:34.161 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:34.162 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:34.168 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:34.168 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:34.174 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:34.174 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:34.180 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:34.180 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:34.186 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:34.186 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:34.193 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:34.193 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:34.198 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:34.198 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:34.204 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:34.204 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:34.210 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:34.210 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:34.216 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:34.216 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:34.221 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:34.222 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:34.227 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:34.227 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:34.233 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:34.233 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:34.239 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:34.239 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:34.244 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:34.244 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:34.250 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:34.250 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:34.255 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket abort-at-change-reshard-state 2026-02-24T22:20:34.271 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:20:34.271 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:20:34.289 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.292+0000 7f677f65aa80 4 Realm: () 2026-02-24T22:20:34.289 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.292+0000 7f677f65aa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:20:34.289 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.292+0000 7f677f65aa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:20:34.289 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.292+0000 7f677f65aa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:20:34.289 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.292+0000 7f677f65aa80 20 zonegroup default 2026-02-24T22:20:34.290 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.292+0000 7f677f65aa80 20 rados->read ofs=0 len=0 2026-02-24T22:20:34.291 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.292+0000 7f677f65aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:34.291 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.292+0000 7f677f65aa80 20 rados->read ofs=0 len=0 2026-02-24T22:20:34.291 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.292+0000 7f677f65aa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:34.292 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.292+0000 7f677f65aa80 20 started sync module instance, tier type = 2026-02-24T22:20:34.292 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.292+0000 7f677f65aa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:20:34.293 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.296+0000 7f677f65aa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:20:34.293 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.296+0000 7f677f65aa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:20:34.293 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.296+0000 7f677f65aa80 5 note: GC not initialized 2026-02-24T22:20:34.293 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.296+0000 7f672afe5640 20 reqs_thread_entry: start 2026-02-24T22:20:34.296 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.300+0000 7f677d92f640 0 -- 10.20.193.46:0/3749755438 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x5643f980f8a0 msgr2=0x5643f9802b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:34.507 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.508+0000 7f677d12e640 0 -- 10.20.193.46:0/3749755438 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5643f980b320 msgr2=0x5643f9806090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:34.536 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.540+0000 7f677f65aa80 20 init_complete bucket index max shards: 1 2026-02-24T22:20:34.536 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.540+0000 7f677f65aa80 20 Filter name: none 2026-02-24T22:20:34.536 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.540+0000 7f67297e2640 20 reqs_thread_entry: start 2026-02-24T22:20:34.536 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.540+0000 7f677f65aa80 20 rados->read ofs=0 len=0 2026-02-24T22:20:34.537 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.540+0000 7f677f65aa80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:20:34.537 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.540+0000 7f677f65aa80 20 rados->read ofs=0 len=0 2026-02-24T22:20:34.538 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.540+0000 7f677f65aa80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:20:34.538 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.540+0000 7f677f65aa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:20:34.538 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.540+0000 7f677f65aa80 20 rados->read ofs=0 len=0 2026-02-24T22:20:34.538 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.540+0000 7f677f65aa80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:20:34.538 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.540+0000 7f677f65aa80 20 rados->read ofs=0 len=0 2026-02-24T22:20:34.539 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.540+0000 7f677f65aa80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:20:34.539 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.540+0000 7f677f65aa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:20:34.549 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.549 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:20:34.549 INFO:tasks.workunit.client.0.trial046.stderr:bucket: abort-at-change-reshard-state id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:20:34.549 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.549 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket abort-at-change-reshard-state --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at change_reshard_state 2026-02-24T22:20:34.565 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:20:34.566 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:20:34.583 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.584+0000 7f0e06a6ea80 4 Realm: () 2026-02-24T22:20:34.583 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.584+0000 7f0e06a6ea80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:20:34.583 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.584+0000 7f0e06a6ea80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:20:34.583 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:20:34.584 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.584+0000 7f0e06a6ea80 20 zonegroup default 2026-02-24T22:20:34.584 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.588+0000 7f0e06a6ea80 20 rados->read ofs=0 len=0 2026-02-24T22:20:34.585 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.588+0000 7f0e06a6ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:34.585 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.588+0000 7f0e06a6ea80 20 rados->read ofs=0 len=0 2026-02-24T22:20:34.586 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.588+0000 7f0e06a6ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:34.586 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.588+0000 7f0e06a6ea80 20 started sync module instance, tier type = 2026-02-24T22:20:34.586 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.588+0000 7f0e06a6ea80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:20:34.587 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.588+0000 7f0df7fff640 0 -- 10.20.193.46:0/1054225154 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55ac5f4ae390 msgr2=0x55ac5f4ce840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:34.589 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 add_watcher() i=6 2026-02-24T22:20:34.589 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 add_watcher() i=1 2026-02-24T22:20:34.589 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 add_watcher() i=3 2026-02-24T22:20:34.589 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 add_watcher() i=5 2026-02-24T22:20:34.589 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 add_watcher() i=0 2026-02-24T22:20:34.589 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 add_watcher() i=7 2026-02-24T22:20:34.589 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 add_watcher() i=2 2026-02-24T22:20:34.589 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 add_watcher() i=4 2026-02-24T22:20:34.589 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.592+0000 7f0e06a6ea80 2 all 8 watchers are set, enabling cache 2026-02-24T22:20:34.590 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.592+0000 7f0df7fff640 0 -- 10.20.193.46:0/3878285961 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f0d24008af0 msgr2=0x7f0d24028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:34.590 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:20:34.591 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:20:34.591 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 note: GC not initialized 2026-02-24T22:20:34.591 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.592+0000 7f0db1fdb640 20 reqs_thread_entry: start 2026-02-24T22:20:34.625 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.628+0000 7f0e06a6ea80 20 init_complete bucket index max shards: 1 2026-02-24T22:20:34.625 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.628+0000 7f0e06a6ea80 20 Filter name: none 2026-02-24T22:20:34.625 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.628+0000 7f0d0bfff640 20 reqs_thread_entry: start 2026-02-24T22:20:34.625 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 cache get: name=default.rgw.meta+root+abort-at-change-reshard-state : miss 2026-02-24T22:20:34.625 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.628+0000 7f0e06a6ea80 20 rados->read ofs=0 len=0 2026-02-24T22:20:34.625 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.628+0000 7f0e06a6ea80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:20:34.626 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 cache put: name=default.rgw.meta+root+abort-at-change-reshard-state info.flags=0x11 2026-02-24T22:20:34.626 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 adding default.rgw.meta+root+abort-at-change-reshard-state to cache LRU end 2026-02-24T22:20:34.626 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 : miss 2026-02-24T22:20:34.626 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.628+0000 7f0e06a6ea80 20 rados->read ofs=0 len=0 2026-02-24T22:20:34.626 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.628+0000 7f0e06a6ea80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:20:34.626 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 info.flags=0x17 2026-02-24T22:20:34.626 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 to cache LRU end 2026-02-24T22:20:34.626 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:20:34.626 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:34.626 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 2026-02-24T22:20:34.630 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.632+0000 7f0e06a6ea80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 info.flags=0x17 2026-02-24T22:20:34.630 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.632+0000 7f0e06a6ea80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 to cache LRU end 2026-02-24T22:20:34.630 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.632+0000 7f0e06a6ea80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:34.630 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.632+0000 7f0e06a6ea80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540, ofs0, ns] 2026-02-24T22:20:34.630 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.632+0000 7f0dd37fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961394 cookie 94198822548960 notifier 7709 bl.length()=1033 2026-02-24T22:20:34.630 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.632+0000 7f0dd37fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 info.flags=0x17 2026-02-24T22:20:34.630 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.632+0000 7f0dd37fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 to cache LRU end 2026-02-24T22:20:34.630 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.632+0000 7f0dd37fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:34.630 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.632+0000 7f0e06a6ea80 20 handle_bi_update(): bucket=:abort-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540]): orig_sources= new_sources= 2026-02-24T22:20:34.631 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.632+0000 7f0e06a6ea80 20 handle_bi_update(): bucket=:abort-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540]): potential sources added=[] removed=[] 2026-02-24T22:20:34.631 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.632+0000 7f0e06a6ea80 20 handle_bi_update(): bucket=:abort-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540]): orig_dests= new_dests= 2026-02-24T22:20:34.631 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.632+0000 7f0e06a6ea80 20 handle_bi_update(): bucket=:abort-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540]): potential dests added=[] removed=[] 2026-02-24T22:20:34.631 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.632+0000 7f0e06a6ea80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540.1.0 2026-02-24T22:20:34.632 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.632+0000 7f0e06a6ea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540.1.0 2026-02-24T22:20:34.632 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.632+0000 7f0e06a6ea80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540.1.1 2026-02-24T22:20:34.632 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.632+0000 7f0e06a6ea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540.1.1 2026-02-24T22:20:34.632 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.632+0000 7f0e06a6ea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540.0 2026-02-24T22:20:34.633 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f0e06a6ea80 time 2026-02-24T22:20:34.641018+0000 2026-02-24T22:20:34.633 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:20:34.634 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:20:34.634 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f0e09eb0eb1] 2026-02-24T22:20:34.634 INFO:tasks.workunit.client.0.trial046.stderr: 2: (RGWBucketReshard::do_reshard(rgw::bucket_index_layout_generation const&, rgw::bucket_index_layout_generation const&, int, bool, bool, std::ostream*, ceph::Formatter*, FaultInjector > >&, DoutPrefixProvider const*, optional_yield)+0x274a) [0x55ac50f71bea] 2026-02-24T22:20:34.634 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xe6b) [0x55ac50f72f6b] 2026-02-24T22:20:34.634 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:20:34.635 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f0e091b0d90] 2026-02-24T22:20:34.635 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:20:34.635 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:20:34.635 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.636+0000 7f0e06a6ea80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f0e06a6ea80 time 2026-02-24T22:20:34.641018+0000 2026-02-24T22:20:34.635 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:20:34.635 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.635 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:20:34.635 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f0e09eb0eb1] 2026-02-24T22:20:34.635 INFO:tasks.workunit.client.0.trial046.stderr: 2: (RGWBucketReshard::do_reshard(rgw::bucket_index_layout_generation const&, rgw::bucket_index_layout_generation const&, int, bool, bool, std::ostream*, ceph::Formatter*, FaultInjector > >&, DoutPrefixProvider const*, optional_yield)+0x274a) [0x55ac50f71bea] 2026-02-24T22:20:34.636 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xe6b) [0x55ac50f72f6b] 2026-02-24T22:20:34.636 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:20:34.636 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f0e091b0d90] 2026-02-24T22:20:34.636 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:20:34.636 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:20:34.636 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.636 INFO:tasks.workunit.client.0.trial046.stderr:*** Caught signal (Aborted) ** 2026-02-24T22:20:34.636 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f0e06a6ea80 thread_name:radosgw-admin 2026-02-24T22:20:34.672 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:20:34.672 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f0e091c9520] 2026-02-24T22:20:34.672 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:20:34.673 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:20:34.673 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:20:34.673 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f0e09eb0f10] 2026-02-24T22:20:34.673 INFO:tasks.workunit.client.0.trial046.stderr: 6: (RGWBucketReshard::do_reshard(rgw::bucket_index_layout_generation const&, rgw::bucket_index_layout_generation const&, int, bool, bool, std::ostream*, ceph::Formatter*, FaultInjector > >&, DoutPrefixProvider const*, optional_yield)+0x274a) [0x55ac50f71bea] 2026-02-24T22:20:34.673 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xe6b) [0x55ac50f72f6b] 2026-02-24T22:20:34.673 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:20:34.673 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f0e091b0d90] 2026-02-24T22:20:34.673 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:20:34.673 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:20:34.673 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.636+0000 7f0e06a6ea80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:20:34.673 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f0e06a6ea80 thread_name:radosgw-admin 2026-02-24T22:20:34.673 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.673 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:20:34.673 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f0e091c9520] 2026-02-24T22:20:34.673 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:20:34.673 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:20:34.673 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:20:34.674 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f0e09eb0f10] 2026-02-24T22:20:34.674 INFO:tasks.workunit.client.0.trial046.stderr: 6: (RGWBucketReshard::do_reshard(rgw::bucket_index_layout_generation const&, rgw::bucket_index_layout_generation const&, int, bool, bool, std::ostream*, ceph::Formatter*, FaultInjector > >&, DoutPrefixProvider const*, optional_yield)+0x274a) [0x55ac50f71bea] 2026-02-24T22:20:34.674 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xe6b) [0x55ac50f72f6b] 2026-02-24T22:20:34.674 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:20:34.674 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f0e091b0d90] 2026-02-24T22:20:34.674 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:20:34.674 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:20:34.674 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:20:34.674 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.674 INFO:tasks.workunit.client.0.trial046.stderr:--- begin dump of recent events --- 2026-02-24T22:20:34.674 INFO:tasks.workunit.client.0.trial046.stderr: -499> 2026-02-24T22:20:34.572+0000 7f0dfffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:20:34.674 INFO:tasks.workunit.client.0.trial046.stderr: -498> 2026-02-24T22:20:34.572+0000 7f0e04d43640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:20:34.674 INFO:tasks.workunit.client.0.trial046.stderr: -497> 2026-02-24T22:20:34.572+0000 7f0e04d43640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:20:34.674 INFO:tasks.workunit.client.0.trial046.stderr: -496> 2026-02-24T22:20:34.572+0000 7f0e04d43640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:20:34.674 INFO:tasks.workunit.client.0.trial046.stderr: -495> 2026-02-24T22:20:34.572+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:20:34.674 INFO:tasks.workunit.client.0.trial046.stderr: -494> 2026-02-24T22:20:34.572+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:20:34.674 INFO:tasks.workunit.client.0.trial046.stderr: -493> 2026-02-24T22:20:34.572+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:20:34.674 INFO:tasks.workunit.client.0.trial046.stderr: -492> 2026-02-24T22:20:34.572+0000 7f0dfffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:20:34.674 INFO:tasks.workunit.client.0.trial046.stderr: -491> 2026-02-24T22:20:34.572+0000 7f0dfffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:20:34.675 INFO:tasks.workunit.client.0.trial046.stderr: -490> 2026-02-24T22:20:34.572+0000 7f0dfffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:20:34.675 INFO:tasks.workunit.client.0.trial046.stderr: -489> 2026-02-24T22:20:34.572+0000 7f0e04d43640 10 monclient(hunting): handle_auth_done global_id 8253 payload 274 2026-02-24T22:20:34.675 INFO:tasks.workunit.client.0.trial046.stderr: -488> 2026-02-24T22:20:34.572+0000 7f0e04d43640 10 monclient: _finish_hunting 0 2026-02-24T22:20:34.675 INFO:tasks.workunit.client.0.trial046.stderr: -487> 2026-02-24T22:20:34.572+0000 7f0e04d43640 1 monclient: found mon.noname-a 2026-02-24T22:20:34.675 INFO:tasks.workunit.client.0.trial046.stderr: -486> 2026-02-24T22:20:34.572+0000 7f0e04d43640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:20:34.675 INFO:tasks.workunit.client.0.trial046.stderr: -485> 2026-02-24T22:20:34.572+0000 7f0df67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:20:34.675 INFO:tasks.workunit.client.0.trial046.stderr: -484> 2026-02-24T22:20:34.572+0000 7f0df67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:20:34.675 INFO:tasks.workunit.client.0.trial046.stderr: -483> 2026-02-24T22:20:34.572+0000 7f0df67fc640 10 monclient: dump: 2026-02-24T22:20:34.675 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:20:34.675 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:20:34.675 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.675 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.675 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:20:34.675 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.675 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:20:34.675 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:20:34.675 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:20:34.675 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.675 INFO:tasks.workunit.client.0.trial046.stderr: -482> 2026-02-24T22:20:34.572+0000 7f0df67fc640 10 monclient: _finish_auth 0 2026-02-24T22:20:34.675 INFO:tasks.workunit.client.0.trial046.stderr: -481> 2026-02-24T22:20:34.572+0000 7f0df67fc640 10 monclient: _check_auth_tickets 2026-02-24T22:20:34.676 INFO:tasks.workunit.client.0.trial046.stderr: -480> 2026-02-24T22:20:34.572+0000 7f0df67fc640 10 monclient: handle_config config(0 keys) 2026-02-24T22:20:34.676 INFO:tasks.workunit.client.0.trial046.stderr: -479> 2026-02-24T22:20:34.572+0000 7f0df67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:20:34.676 INFO:tasks.workunit.client.0.trial046.stderr: -478> 2026-02-24T22:20:34.572+0000 7f0df67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:20:34.676 INFO:tasks.workunit.client.0.trial046.stderr: -477> 2026-02-24T22:20:34.572+0000 7f0df67fc640 10 monclient: dump: 2026-02-24T22:20:34.676 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:20:34.676 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:20:34.676 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.676 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.676 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:20:34.676 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.676 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:20:34.676 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:20:34.676 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:20:34.676 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.676 INFO:tasks.workunit.client.0.trial046.stderr: -476> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 monclient: authenticate success, global_id 8253 2026-02-24T22:20:34.676 INFO:tasks.workunit.client.0.trial046.stderr: -475> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient: get_monmap_and_config success 2026-02-24T22:20:34.676 INFO:tasks.workunit.client.0.trial046.stderr: -474> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 4 set_mon_vals no callback set 2026-02-24T22:20:34.676 INFO:tasks.workunit.client.0.trial046.stderr: -473> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient: shutdown 2026-02-24T22:20:34.676 INFO:tasks.workunit.client.0.trial046.stderr: -472> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) unregister_commands rotate-key 2026-02-24T22:20:34.676 INFO:tasks.workunit.client.0.trial046.stderr: -471> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient: build_initial_monmap 2026-02-24T22:20:34.677 INFO:tasks.workunit.client.0.trial046.stderr: -470> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 1 build_initial for_mkfs: 0 2026-02-24T22:20:34.677 INFO:tasks.workunit.client.0.trial046.stderr: -469> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient: monmap: 2026-02-24T22:20:34.677 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:20:34.677 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:20:34.677 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:20:34.677 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:20:34.677 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:20:34.677 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.677 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:20:34.677 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:20:34.677 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:20:34.677 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.677 INFO:tasks.workunit.client.0.trial046.stderr: -468> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding auth protocol: cephx 2026-02-24T22:20:34.677 INFO:tasks.workunit.client.0.trial046.stderr: -467> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding auth protocol: cephx 2026-02-24T22:20:34.677 INFO:tasks.workunit.client.0.trial046.stderr: -466> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding auth protocol: cephx 2026-02-24T22:20:34.677 INFO:tasks.workunit.client.0.trial046.stderr: -465> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding con mode: secure 2026-02-24T22:20:34.677 INFO:tasks.workunit.client.0.trial046.stderr: -464> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding con mode: crc 2026-02-24T22:20:34.677 INFO:tasks.workunit.client.0.trial046.stderr: -463> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding con mode: secure 2026-02-24T22:20:34.678 INFO:tasks.workunit.client.0.trial046.stderr: -462> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding con mode: crc 2026-02-24T22:20:34.678 INFO:tasks.workunit.client.0.trial046.stderr: -461> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding con mode: secure 2026-02-24T22:20:34.678 INFO:tasks.workunit.client.0.trial046.stderr: -460> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding con mode: crc 2026-02-24T22:20:34.678 INFO:tasks.workunit.client.0.trial046.stderr: -459> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding con mode: crc 2026-02-24T22:20:34.678 INFO:tasks.workunit.client.0.trial046.stderr: -458> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding con mode: secure 2026-02-24T22:20:34.678 INFO:tasks.workunit.client.0.trial046.stderr: -457> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding con mode: crc 2026-02-24T22:20:34.678 INFO:tasks.workunit.client.0.trial046.stderr: -456> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding con mode: secure 2026-02-24T22:20:34.678 INFO:tasks.workunit.client.0.trial046.stderr: -455> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding con mode: crc 2026-02-24T22:20:34.678 INFO:tasks.workunit.client.0.trial046.stderr: -454> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding con mode: secure 2026-02-24T22:20:34.678 INFO:tasks.workunit.client.0.trial046.stderr: -453> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:20:34.678 INFO:tasks.workunit.client.0.trial046.stderr: -452> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 1 librados: starting msgr at 2026-02-24T22:20:34.678 INFO:tasks.workunit.client.0.trial046.stderr: -451> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 1 librados: starting objecter 2026-02-24T22:20:34.678 INFO:tasks.workunit.client.0.trial046.stderr: -450> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command objecter_requests hook 0x55ac5f158b60 2026-02-24T22:20:34.678 INFO:tasks.workunit.client.0.trial046.stderr: -449> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 1 librados: setting wanted keys 2026-02-24T22:20:34.678 INFO:tasks.workunit.client.0.trial046.stderr: -448> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 1 librados: calling monclient init 2026-02-24T22:20:34.678 INFO:tasks.workunit.client.0.trial046.stderr: -447> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient: init 2026-02-24T22:20:34.678 INFO:tasks.workunit.client.0.trial046.stderr: -446> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding auth protocol: cephx 2026-02-24T22:20:34.678 INFO:tasks.workunit.client.0.trial046.stderr: -445> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding auth protocol: cephx 2026-02-24T22:20:34.678 INFO:tasks.workunit.client.0.trial046.stderr: -444> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding auth protocol: cephx 2026-02-24T22:20:34.678 INFO:tasks.workunit.client.0.trial046.stderr: -443> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding con mode: secure 2026-02-24T22:20:34.678 INFO:tasks.workunit.client.0.trial046.stderr: -442> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding con mode: crc 2026-02-24T22:20:34.678 INFO:tasks.workunit.client.0.trial046.stderr: -441> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding con mode: secure 2026-02-24T22:20:34.678 INFO:tasks.workunit.client.0.trial046.stderr: -440> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding con mode: crc 2026-02-24T22:20:34.679 INFO:tasks.workunit.client.0.trial046.stderr: -439> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding con mode: secure 2026-02-24T22:20:34.679 INFO:tasks.workunit.client.0.trial046.stderr: -438> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding con mode: crc 2026-02-24T22:20:34.679 INFO:tasks.workunit.client.0.trial046.stderr: -437> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding con mode: crc 2026-02-24T22:20:34.679 INFO:tasks.workunit.client.0.trial046.stderr: -436> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding con mode: secure 2026-02-24T22:20:34.679 INFO:tasks.workunit.client.0.trial046.stderr: -435> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding con mode: crc 2026-02-24T22:20:34.679 INFO:tasks.workunit.client.0.trial046.stderr: -434> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding con mode: secure 2026-02-24T22:20:34.679 INFO:tasks.workunit.client.0.trial046.stderr: -433> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding con mode: crc 2026-02-24T22:20:34.679 INFO:tasks.workunit.client.0.trial046.stderr: -432> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding con mode: secure 2026-02-24T22:20:34.679 INFO:tasks.workunit.client.0.trial046.stderr: -431> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:20:34.679 INFO:tasks.workunit.client.0.trial046.stderr: -430> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:20:34.679 INFO:tasks.workunit.client.0.trial046.stderr: -429> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command rotate-key hook 0x55ac5f125278 2026-02-24T22:20:34.679 INFO:tasks.workunit.client.0.trial046.stderr: -428> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient: _reopen_session rank -1 2026-02-24T22:20:34.679 INFO:tasks.workunit.client.0.trial046.stderr: -427> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T22:20:34.679 INFO:tasks.workunit.client.0.trial046.stderr: -426> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient(hunting): picked mon.noname-b con 0x55ac5f0b3fe0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:20:34.679 INFO:tasks.workunit.client.0.trial046.stderr: -425> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient(hunting): picked mon.noname-c con 0x55ac5f15cb00 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:20:34.679 INFO:tasks.workunit.client.0.trial046.stderr: -424> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient(hunting): picked mon.noname-a con 0x55ac5f15fcf0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:20:34.679 INFO:tasks.workunit.client.0.trial046.stderr: -423> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient(hunting): start opening mon connection 2026-02-24T22:20:34.679 INFO:tasks.workunit.client.0.trial046.stderr: -422> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient(hunting): start opening mon connection 2026-02-24T22:20:34.679 INFO:tasks.workunit.client.0.trial046.stderr: -421> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient(hunting): start opening mon connection 2026-02-24T22:20:34.679 INFO:tasks.workunit.client.0.trial046.stderr: -420> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient(hunting): _renew_subs 2026-02-24T22:20:34.679 INFO:tasks.workunit.client.0.trial046.stderr: -419> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient(hunting): authenticate will time out at 2004.683917s 2026-02-24T22:20:34.679 INFO:tasks.workunit.client.0.trial046.stderr: -418> 2026-02-24T22:20:34.572+0000 7f0df7fff640 10 monclient(hunting): get_auth_request con 0x55ac5f15cb00 auth_method 0 2026-02-24T22:20:34.680 INFO:tasks.workunit.client.0.trial046.stderr: -417> 2026-02-24T22:20:34.576+0000 7f0df7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:20:34.680 INFO:tasks.workunit.client.0.trial046.stderr: -416> 2026-02-24T22:20:34.576+0000 7f0df7fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:20:34.680 INFO:tasks.workunit.client.0.trial046.stderr: -415> 2026-02-24T22:20:34.576+0000 7f0df7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:20:34.680 INFO:tasks.workunit.client.0.trial046.stderr: -414> 2026-02-24T22:20:34.576+0000 7f0e04d43640 10 monclient(hunting): get_auth_request con 0x55ac5f15fcf0 auth_method 0 2026-02-24T22:20:34.680 INFO:tasks.workunit.client.0.trial046.stderr: -413> 2026-02-24T22:20:34.576+0000 7f0e04d43640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:20:34.680 INFO:tasks.workunit.client.0.trial046.stderr: -412> 2026-02-24T22:20:34.576+0000 7f0e04d43640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:20:34.680 INFO:tasks.workunit.client.0.trial046.stderr: -411> 2026-02-24T22:20:34.576+0000 7f0e04d43640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:20:34.680 INFO:tasks.workunit.client.0.trial046.stderr: -410> 2026-02-24T22:20:34.576+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:20:34.680 INFO:tasks.workunit.client.0.trial046.stderr: -409> 2026-02-24T22:20:34.576+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:20:34.680 INFO:tasks.workunit.client.0.trial046.stderr: -408> 2026-02-24T22:20:34.576+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:20:34.680 INFO:tasks.workunit.client.0.trial046.stderr: -407> 2026-02-24T22:20:34.576+0000 7f0e04d43640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:20:34.680 INFO:tasks.workunit.client.0.trial046.stderr: -406> 2026-02-24T22:20:34.576+0000 7f0e04d43640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:20:34.680 INFO:tasks.workunit.client.0.trial046.stderr: -405> 2026-02-24T22:20:34.576+0000 7f0e04d43640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:20:34.680 INFO:tasks.workunit.client.0.trial046.stderr: -404> 2026-02-24T22:20:34.576+0000 7f0dfffff640 10 monclient(hunting): get_auth_request con 0x55ac5f0b3fe0 auth_method 0 2026-02-24T22:20:34.680 INFO:tasks.workunit.client.0.trial046.stderr: -403> 2026-02-24T22:20:34.576+0000 7f0dfffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:20:34.680 INFO:tasks.workunit.client.0.trial046.stderr: -402> 2026-02-24T22:20:34.576+0000 7f0dfffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:20:34.680 INFO:tasks.workunit.client.0.trial046.stderr: -401> 2026-02-24T22:20:34.576+0000 7f0dfffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:20:34.680 INFO:tasks.workunit.client.0.trial046.stderr: -400> 2026-02-24T22:20:34.576+0000 7f0df7fff640 10 monclient(hunting): handle_auth_done global_id 7700 payload 824 2026-02-24T22:20:34.680 INFO:tasks.workunit.client.0.trial046.stderr: -399> 2026-02-24T22:20:34.576+0000 7f0df7fff640 10 monclient: _finish_hunting 0 2026-02-24T22:20:34.680 INFO:tasks.workunit.client.0.trial046.stderr: -398> 2026-02-24T22:20:34.576+0000 7f0df7fff640 1 monclient: found mon.noname-c 2026-02-24T22:20:34.680 INFO:tasks.workunit.client.0.trial046.stderr: -397> 2026-02-24T22:20:34.576+0000 7f0df7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:20:34.680 INFO:tasks.workunit.client.0.trial046.stderr: -396> 2026-02-24T22:20:34.576+0000 7f0df4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:20:34.680 INFO:tasks.workunit.client.0.trial046.stderr: -395> 2026-02-24T22:20:34.576+0000 7f0df4ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:20:34.680 INFO:tasks.workunit.client.0.trial046.stderr: -394> 2026-02-24T22:20:34.576+0000 7f0df4ff9640 10 monclient: dump: 2026-02-24T22:20:34.681 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:20:34.681 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:20:34.681 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.681 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.681 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:20:34.681 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.681 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:20:34.681 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:20:34.681 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:20:34.681 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.681 INFO:tasks.workunit.client.0.trial046.stderr: -393> 2026-02-24T22:20:34.576+0000 7f0df4ff9640 10 monclient: _finish_auth 0 2026-02-24T22:20:34.681 INFO:tasks.workunit.client.0.trial046.stderr: -392> 2026-02-24T22:20:34.576+0000 7f0df4ff9640 10 monclient: _check_auth_tickets 2026-02-24T22:20:34.681 INFO:tasks.workunit.client.0.trial046.stderr: -391> 2026-02-24T22:20:34.576+0000 7f0df4ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T22:20:34.681 INFO:tasks.workunit.client.0.trial046.stderr: -390> 2026-02-24T22:20:34.576+0000 7f0df4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:20:34.681 INFO:tasks.workunit.client.0.trial046.stderr: -389> 2026-02-24T22:20:34.576+0000 7f0df4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:20:34.681 INFO:tasks.workunit.client.0.trial046.stderr: -388> 2026-02-24T22:20:34.576+0000 7f0df4ff9640 10 monclient: dump: 2026-02-24T22:20:34.681 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:20:34.681 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:20:34.681 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.681 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.681 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:20:34.681 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.682 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:20:34.682 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:20:34.682 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:20:34.682 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.682 INFO:tasks.workunit.client.0.trial046.stderr: -387> 2026-02-24T22:20:34.576+0000 7f0df67fc640 4 set_mon_vals no callback set 2026-02-24T22:20:34.682 INFO:tasks.workunit.client.0.trial046.stderr: -386> 2026-02-24T22:20:34.576+0000 7f0e06a6ea80 5 monclient: authenticate success, global_id 7700 2026-02-24T22:20:34.682 INFO:tasks.workunit.client.0.trial046.stderr: -385> 2026-02-24T22:20:34.576+0000 7f0e06a6ea80 10 monclient: _renew_subs 2026-02-24T22:20:34.682 INFO:tasks.workunit.client.0.trial046.stderr: -384> 2026-02-24T22:20:34.576+0000 7f0e06a6ea80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:20:34.682 INFO:tasks.workunit.client.0.trial046.stderr: -383> 2026-02-24T22:20:34.576+0000 7f0e06a6ea80 10 monclient: _renew_subs 2026-02-24T22:20:34.682 INFO:tasks.workunit.client.0.trial046.stderr: -382> 2026-02-24T22:20:34.576+0000 7f0e06a6ea80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:20:34.682 INFO:tasks.workunit.client.0.trial046.stderr: -381> 2026-02-24T22:20:34.576+0000 7f0e06a6ea80 1 librados: init done 2026-02-24T22:20:34.682 INFO:tasks.workunit.client.0.trial046.stderr: -380> 2026-02-24T22:20:34.576+0000 7f0df4ff9640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:20:34.682 INFO:tasks.workunit.client.0.trial046.stderr: -379> 2026-02-24T22:20:34.576+0000 7f0df4ff9640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:20:34.682 INFO:tasks.workunit.client.0.trial046.stderr: -378> 2026-02-24T22:20:34.576+0000 7f0df4ff9640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:20:34.682 INFO:tasks.workunit.client.0.trial046.stderr: -377> 2026-02-24T22:20:34.576+0000 7f0dfffff640 10 monclient: get_auth_request con 0x7f0de00707a0 auth_method 0 2026-02-24T22:20:34.682 INFO:tasks.workunit.client.0.trial046.stderr: -376> 2026-02-24T22:20:34.576+0000 7f0e04d43640 10 monclient: get_auth_request con 0x55ac5f18edd0 auth_method 0 2026-02-24T22:20:34.682 INFO:tasks.workunit.client.0.trial046.stderr: -375> 2026-02-24T22:20:34.576+0000 7f0dfffff640 10 monclient: get_auth_request con 0x55ac5f1b34d0 auth_method 0 2026-02-24T22:20:34.682 INFO:tasks.workunit.client.0.trial046.stderr: -374> 2026-02-24T22:20:34.580+0000 7f0df7fff640 10 monclient: get_auth_request con 0x55ac5f1b4de0 auth_method 0 2026-02-24T22:20:34.682 INFO:tasks.workunit.client.0.trial046.stderr: -373> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient: build_initial_monmap 2026-02-24T22:20:34.682 INFO:tasks.workunit.client.0.trial046.stderr: -372> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 1 build_initial for_mkfs: 0 2026-02-24T22:20:34.682 INFO:tasks.workunit.client.0.trial046.stderr: -371> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient: monmap: 2026-02-24T22:20:34.682 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:20:34.682 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:20:34.683 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:20:34.683 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:20:34.683 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:20:34.683 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.683 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:20:34.683 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:20:34.683 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:20:34.683 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.683 INFO:tasks.workunit.client.0.trial046.stderr: -370> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding auth protocol: cephx 2026-02-24T22:20:34.683 INFO:tasks.workunit.client.0.trial046.stderr: -369> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding auth protocol: cephx 2026-02-24T22:20:34.683 INFO:tasks.workunit.client.0.trial046.stderr: -368> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding auth protocol: cephx 2026-02-24T22:20:34.683 INFO:tasks.workunit.client.0.trial046.stderr: -367> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding con mode: secure 2026-02-24T22:20:34.683 INFO:tasks.workunit.client.0.trial046.stderr: -366> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding con mode: crc 2026-02-24T22:20:34.683 INFO:tasks.workunit.client.0.trial046.stderr: -365> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding con mode: secure 2026-02-24T22:20:34.683 INFO:tasks.workunit.client.0.trial046.stderr: -364> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding con mode: crc 2026-02-24T22:20:34.683 INFO:tasks.workunit.client.0.trial046.stderr: -363> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding con mode: secure 2026-02-24T22:20:34.683 INFO:tasks.workunit.client.0.trial046.stderr: -362> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding con mode: crc 2026-02-24T22:20:34.683 INFO:tasks.workunit.client.0.trial046.stderr: -361> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding con mode: crc 2026-02-24T22:20:34.683 INFO:tasks.workunit.client.0.trial046.stderr: -360> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding con mode: secure 2026-02-24T22:20:34.683 INFO:tasks.workunit.client.0.trial046.stderr: -359> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding con mode: crc 2026-02-24T22:20:34.683 INFO:tasks.workunit.client.0.trial046.stderr: -358> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding con mode: secure 2026-02-24T22:20:34.683 INFO:tasks.workunit.client.0.trial046.stderr: -357> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding con mode: crc 2026-02-24T22:20:34.683 INFO:tasks.workunit.client.0.trial046.stderr: -356> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding con mode: secure 2026-02-24T22:20:34.683 INFO:tasks.workunit.client.0.trial046.stderr: -355> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:20:34.684 INFO:tasks.workunit.client.0.trial046.stderr: -354> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command objecter_requests cmddesc objecter_requests hook 0x55ac5f1bcc30 EEXIST 2026-02-24T22:20:34.684 INFO:tasks.workunit.client.0.trial046.stderr: -353> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient: init 2026-02-24T22:20:34.684 INFO:tasks.workunit.client.0.trial046.stderr: -352> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding auth protocol: cephx 2026-02-24T22:20:34.684 INFO:tasks.workunit.client.0.trial046.stderr: -351> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding auth protocol: cephx 2026-02-24T22:20:34.684 INFO:tasks.workunit.client.0.trial046.stderr: -350> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding auth protocol: cephx 2026-02-24T22:20:34.684 INFO:tasks.workunit.client.0.trial046.stderr: -349> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding con mode: secure 2026-02-24T22:20:34.684 INFO:tasks.workunit.client.0.trial046.stderr: -348> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding con mode: crc 2026-02-24T22:20:34.684 INFO:tasks.workunit.client.0.trial046.stderr: -347> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding con mode: secure 2026-02-24T22:20:34.684 INFO:tasks.workunit.client.0.trial046.stderr: -346> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding con mode: crc 2026-02-24T22:20:34.684 INFO:tasks.workunit.client.0.trial046.stderr: -345> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding con mode: secure 2026-02-24T22:20:34.684 INFO:tasks.workunit.client.0.trial046.stderr: -344> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding con mode: crc 2026-02-24T22:20:34.684 INFO:tasks.workunit.client.0.trial046.stderr: -343> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding con mode: crc 2026-02-24T22:20:34.684 INFO:tasks.workunit.client.0.trial046.stderr: -342> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding con mode: secure 2026-02-24T22:20:34.684 INFO:tasks.workunit.client.0.trial046.stderr: -341> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding con mode: crc 2026-02-24T22:20:34.684 INFO:tasks.workunit.client.0.trial046.stderr: -340> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding con mode: secure 2026-02-24T22:20:34.684 INFO:tasks.workunit.client.0.trial046.stderr: -339> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding con mode: crc 2026-02-24T22:20:34.684 INFO:tasks.workunit.client.0.trial046.stderr: -338> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding con mode: secure 2026-02-24T22:20:34.684 INFO:tasks.workunit.client.0.trial046.stderr: -337> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:20:34.684 INFO:tasks.workunit.client.0.trial046.stderr: -336> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:20:34.684 INFO:tasks.workunit.client.0.trial046.stderr: -335> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command rotate-key cmddesc rotate-key hook 0x55ac5f1b5850 EEXIST 2026-02-24T22:20:34.684 INFO:tasks.workunit.client.0.trial046.stderr: -334> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient: _reopen_session rank -1 2026-02-24T22:20:34.684 INFO:tasks.workunit.client.0.trial046.stderr: -333> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T22:20:34.684 INFO:tasks.workunit.client.0.trial046.stderr: -332> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient(hunting): picked mon.noname-b con 0x55ac5f310530 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:20:34.684 INFO:tasks.workunit.client.0.trial046.stderr: -331> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient(hunting): picked mon.noname-c con 0x55ac5f30ea40 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:20:34.684 INFO:tasks.workunit.client.0.trial046.stderr: -330> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient(hunting): picked mon.noname-a con 0x55ac5f30bc20 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:20:34.685 INFO:tasks.workunit.client.0.trial046.stderr: -329> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient(hunting): start opening mon connection 2026-02-24T22:20:34.685 INFO:tasks.workunit.client.0.trial046.stderr: -328> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient(hunting): start opening mon connection 2026-02-24T22:20:34.685 INFO:tasks.workunit.client.0.trial046.stderr: -327> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient(hunting): start opening mon connection 2026-02-24T22:20:34.685 INFO:tasks.workunit.client.0.trial046.stderr: -326> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient(hunting): _renew_subs 2026-02-24T22:20:34.685 INFO:tasks.workunit.client.0.trial046.stderr: -325> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient(hunting): authenticate will time out at 2004.689417s 2026-02-24T22:20:34.685 INFO:tasks.workunit.client.0.trial046.stderr: -324> 2026-02-24T22:20:34.580+0000 7f0e04d43640 10 monclient(hunting): get_auth_request con 0x55ac5f30bc20 auth_method 0 2026-02-24T22:20:34.685 INFO:tasks.workunit.client.0.trial046.stderr: -323> 2026-02-24T22:20:34.580+0000 7f0e04d43640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:20:34.685 INFO:tasks.workunit.client.0.trial046.stderr: -322> 2026-02-24T22:20:34.580+0000 7f0e04d43640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:20:34.685 INFO:tasks.workunit.client.0.trial046.stderr: -321> 2026-02-24T22:20:34.580+0000 7f0e04d43640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:20:34.685 INFO:tasks.workunit.client.0.trial046.stderr: -320> 2026-02-24T22:20:34.580+0000 7f0df7fff640 10 monclient(hunting): get_auth_request con 0x55ac5f30ea40 auth_method 0 2026-02-24T22:20:34.685 INFO:tasks.workunit.client.0.trial046.stderr: -319> 2026-02-24T22:20:34.580+0000 7f0df7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:20:34.685 INFO:tasks.workunit.client.0.trial046.stderr: -318> 2026-02-24T22:20:34.580+0000 7f0df7fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:20:34.685 INFO:tasks.workunit.client.0.trial046.stderr: -317> 2026-02-24T22:20:34.580+0000 7f0df7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:20:34.685 INFO:tasks.workunit.client.0.trial046.stderr: -316> 2026-02-24T22:20:34.580+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:20:34.685 INFO:tasks.workunit.client.0.trial046.stderr: -315> 2026-02-24T22:20:34.580+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:20:34.685 INFO:tasks.workunit.client.0.trial046.stderr: -314> 2026-02-24T22:20:34.580+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:20:34.685 INFO:tasks.workunit.client.0.trial046.stderr: -313> 2026-02-24T22:20:34.580+0000 7f0e04d43640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:20:34.685 INFO:tasks.workunit.client.0.trial046.stderr: -312> 2026-02-24T22:20:34.580+0000 7f0e04d43640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:20:34.685 INFO:tasks.workunit.client.0.trial046.stderr: -311> 2026-02-24T22:20:34.580+0000 7f0e04d43640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:20:34.685 INFO:tasks.workunit.client.0.trial046.stderr: -310> 2026-02-24T22:20:34.580+0000 7f0dfffff640 10 monclient(hunting): get_auth_request con 0x55ac5f310530 auth_method 0 2026-02-24T22:20:34.685 INFO:tasks.workunit.client.0.trial046.stderr: -309> 2026-02-24T22:20:34.580+0000 7f0dfffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:20:34.685 INFO:tasks.workunit.client.0.trial046.stderr: -308> 2026-02-24T22:20:34.580+0000 7f0dfffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:20:34.685 INFO:tasks.workunit.client.0.trial046.stderr: -307> 2026-02-24T22:20:34.580+0000 7f0dfffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:20:34.685 INFO:tasks.workunit.client.0.trial046.stderr: -306> 2026-02-24T22:20:34.580+0000 7f0e04d43640 10 monclient(hunting): handle_auth_done global_id 8259 payload 824 2026-02-24T22:20:34.685 INFO:tasks.workunit.client.0.trial046.stderr: -305> 2026-02-24T22:20:34.580+0000 7f0e04d43640 10 monclient: _finish_hunting 0 2026-02-24T22:20:34.685 INFO:tasks.workunit.client.0.trial046.stderr: -304> 2026-02-24T22:20:34.580+0000 7f0e04d43640 1 monclient: found mon.noname-a 2026-02-24T22:20:34.686 INFO:tasks.workunit.client.0.trial046.stderr: -303> 2026-02-24T22:20:34.580+0000 7f0e04d43640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:20:34.686 INFO:tasks.workunit.client.0.trial046.stderr: -302> 2026-02-24T22:20:34.580+0000 7f0de5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:20:34.686 INFO:tasks.workunit.client.0.trial046.stderr: -301> 2026-02-24T22:20:34.580+0000 7f0de5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:20:34.686 INFO:tasks.workunit.client.0.trial046.stderr: -300> 2026-02-24T22:20:34.580+0000 7f0de5ffb640 10 monclient: dump: 2026-02-24T22:20:34.686 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:20:34.686 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:20:34.686 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.686 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.686 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:20:34.686 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.686 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:20:34.686 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:20:34.686 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:20:34.686 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.686 INFO:tasks.workunit.client.0.trial046.stderr: -299> 2026-02-24T22:20:34.580+0000 7f0de5ffb640 10 monclient: _finish_auth 0 2026-02-24T22:20:34.686 INFO:tasks.workunit.client.0.trial046.stderr: -298> 2026-02-24T22:20:34.580+0000 7f0de5ffb640 10 monclient: _check_auth_tickets 2026-02-24T22:20:34.686 INFO:tasks.workunit.client.0.trial046.stderr: -297> 2026-02-24T22:20:34.580+0000 7f0de5ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T22:20:34.686 INFO:tasks.workunit.client.0.trial046.stderr: -296> 2026-02-24T22:20:34.580+0000 7f0de5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:20:34.686 INFO:tasks.workunit.client.0.trial046.stderr: -295> 2026-02-24T22:20:34.580+0000 7f0de5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:20:34.686 INFO:tasks.workunit.client.0.trial046.stderr: -294> 2026-02-24T22:20:34.580+0000 7f0de5ffb640 10 monclient: dump: 2026-02-24T22:20:34.686 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:20:34.686 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:20:34.686 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.686 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.686 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:20:34.686 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr: -293> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 monclient: authenticate success, global_id 8259 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr: -292> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient: _renew_subs 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr: -291> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr: -290> 2026-02-24T22:20:34.580+0000 7f0e05544640 4 set_mon_vals no callback set 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr: -289> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient: _renew_subs 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr: -288> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr: -287> 2026-02-24T22:20:34.580+0000 7f0de5ffb640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr: -286> 2026-02-24T22:20:34.580+0000 7f0de5ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr: -285> 2026-02-24T22:20:34.580+0000 7f0de5ffb640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr: -284> 2026-02-24T22:20:34.580+0000 7f0dfffff640 10 monclient: get_auth_request con 0x7f0dd40706c0 auth_method 0 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr: -283> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient: get_monmap_and_config 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr: -282> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient: build_initial_monmap 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr: -281> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 1 build_initial for_mkfs: 0 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr: -280> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient: monmap: 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:20:34.687 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -279> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding auth protocol: cephx 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -278> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding auth protocol: cephx 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -277> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding auth protocol: cephx 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -276> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding con mode: secure 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -275> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding con mode: crc 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -274> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding con mode: secure 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -273> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding con mode: crc 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -272> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding con mode: secure 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -271> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding con mode: crc 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -270> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding con mode: crc 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -269> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding con mode: secure 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -268> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding con mode: crc 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -267> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding con mode: secure 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -266> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding con mode: crc 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -265> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding con mode: secure 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -264> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -263> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: init 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -262> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding auth protocol: cephx 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -261> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding auth protocol: cephx 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -260> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding auth protocol: cephx 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -259> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding con mode: secure 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -258> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding con mode: crc 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -257> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding con mode: secure 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -256> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding con mode: crc 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -255> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding con mode: secure 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -254> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding con mode: crc 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -253> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding con mode: crc 2026-02-24T22:20:34.688 INFO:tasks.workunit.client.0.trial046.stderr: -252> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding con mode: secure 2026-02-24T22:20:34.689 INFO:tasks.workunit.client.0.trial046.stderr: -251> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding con mode: crc 2026-02-24T22:20:34.689 INFO:tasks.workunit.client.0.trial046.stderr: -250> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding con mode: secure 2026-02-24T22:20:34.689 INFO:tasks.workunit.client.0.trial046.stderr: -249> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding con mode: crc 2026-02-24T22:20:34.689 INFO:tasks.workunit.client.0.trial046.stderr: -248> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding con mode: secure 2026-02-24T22:20:34.689 INFO:tasks.workunit.client.0.trial046.stderr: -247> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:20:34.689 INFO:tasks.workunit.client.0.trial046.stderr: -246> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:20:34.689 INFO:tasks.workunit.client.0.trial046.stderr: -245> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command rotate-key cmddesc rotate-key hook 0x7fff441f2618 EEXIST 2026-02-24T22:20:34.689 INFO:tasks.workunit.client.0.trial046.stderr: -244> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: _reopen_session rank -1 2026-02-24T22:20:34.689 INFO:tasks.workunit.client.0.trial046.stderr: -243> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:20:34.689 INFO:tasks.workunit.client.0.trial046.stderr: -242> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): picked mon.noname-c con 0x55ac5f3c85f0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:20:34.689 INFO:tasks.workunit.client.0.trial046.stderr: -241> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): picked mon.noname-a con 0x55ac5f3cbd60 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:20:34.689 INFO:tasks.workunit.client.0.trial046.stderr: -240> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): picked mon.noname-b con 0x55ac5f3a3510 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:20:34.689 INFO:tasks.workunit.client.0.trial046.stderr: -239> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): start opening mon connection 2026-02-24T22:20:34.689 INFO:tasks.workunit.client.0.trial046.stderr: -238> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): start opening mon connection 2026-02-24T22:20:34.689 INFO:tasks.workunit.client.0.trial046.stderr: -237> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): start opening mon connection 2026-02-24T22:20:34.689 INFO:tasks.workunit.client.0.trial046.stderr: -236> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): _renew_subs 2026-02-24T22:20:34.689 INFO:tasks.workunit.client.0.trial046.stderr: -235> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): authenticate will time out at 2004.692684s 2026-02-24T22:20:34.689 INFO:tasks.workunit.client.0.trial046.stderr: -234> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): get_auth_request con 0x55ac5f3cbd60 auth_method 0 2026-02-24T22:20:34.689 INFO:tasks.workunit.client.0.trial046.stderr: -233> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:20:34.689 INFO:tasks.workunit.client.0.trial046.stderr: -232> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:20:34.689 INFO:tasks.workunit.client.0.trial046.stderr: -231> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:20:34.689 INFO:tasks.workunit.client.0.trial046.stderr: -230> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): get_auth_request con 0x55ac5f3c85f0 auth_method 0 2026-02-24T22:20:34.689 INFO:tasks.workunit.client.0.trial046.stderr: -229> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:20:34.689 INFO:tasks.workunit.client.0.trial046.stderr: -228> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr: -227> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr: -226> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr: -225> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr: -224> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr: -223> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr: -222> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr: -221> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr: -220> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): handle_auth_done global_id 7706 payload 274 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr: -219> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient: _finish_hunting 0 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr: -218> 2026-02-24T22:20:34.584+0000 7f0dfffff640 1 monclient: found mon.noname-c 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr: -217> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr: -216> 2026-02-24T22:20:34.584+0000 7f0dd37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr: -215> 2026-02-24T22:20:34.584+0000 7f0dd37fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr: -214> 2026-02-24T22:20:34.584+0000 7f0dd37fe640 10 monclient: dump: 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr: -213> 2026-02-24T22:20:34.584+0000 7f0dd37fe640 10 monclient: _finish_auth 0 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr: -212> 2026-02-24T22:20:34.584+0000 7f0dd37fe640 10 monclient: _check_auth_tickets 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr: -211> 2026-02-24T22:20:34.584+0000 7f0dd37fe640 10 monclient: handle_config config(0 keys) 2026-02-24T22:20:34.690 INFO:tasks.workunit.client.0.trial046.stderr: -210> 2026-02-24T22:20:34.584+0000 7f0dd37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr: -209> 2026-02-24T22:20:34.584+0000 7f0dd37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr: -208> 2026-02-24T22:20:34.584+0000 7f0dd37fe640 10 monclient: dump: 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr: -207> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 monclient: authenticate success, global_id 7706 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr: -206> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: get_monmap_and_config success 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr: -205> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 4 set_mon_vals no callback set 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr: -204> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: shutdown 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr: -203> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: build_initial_monmap 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr: -202> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 1 build_initial for_mkfs: 0 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr: -201> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: monmap: 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:20:34.691 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -200> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding auth protocol: cephx 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -199> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding auth protocol: cephx 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -198> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding auth protocol: cephx 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -197> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding con mode: secure 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -196> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding con mode: crc 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -195> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding con mode: secure 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -194> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding con mode: crc 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -193> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding con mode: secure 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -192> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding con mode: crc 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -191> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding con mode: crc 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -190> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding con mode: secure 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -189> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding con mode: crc 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -188> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding con mode: secure 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -187> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding con mode: crc 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -186> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding con mode: secure 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -185> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -184> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 1 librados: starting msgr at 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -183> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 1 librados: starting objecter 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -182> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command objecter_requests cmddesc objecter_requests hook 0x55ac5f3094d0 EEXIST 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -181> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 1 librados: setting wanted keys 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -180> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 1 librados: calling monclient init 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -179> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: init 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -178> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding auth protocol: cephx 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -177> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding auth protocol: cephx 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -176> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding auth protocol: cephx 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -175> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding con mode: secure 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -174> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding con mode: crc 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -173> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding con mode: secure 2026-02-24T22:20:34.692 INFO:tasks.workunit.client.0.trial046.stderr: -172> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding con mode: crc 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -171> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding con mode: secure 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -170> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding con mode: crc 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -169> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding con mode: crc 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -168> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding con mode: secure 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -167> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding con mode: crc 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -166> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding con mode: secure 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -165> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding con mode: crc 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -164> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding con mode: secure 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -163> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -162> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -161> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command rotate-key cmddesc rotate-key hook 0x55ac5f3a04c8 EEXIST 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -160> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: _reopen_session rank -1 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -159> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -158> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): picked mon.noname-c con 0x55ac5f3c85f0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -157> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): picked mon.noname-a con 0x55ac5f3cbd60 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -156> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): picked mon.noname-b con 0x55ac5f3ee9f0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -155> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): start opening mon connection 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -154> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): start opening mon connection 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -153> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): start opening mon connection 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -152> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): _renew_subs 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -151> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): authenticate will time out at 2004.693990s 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -150> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): get_auth_request con 0x55ac5f3c85f0 auth_method 0 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -149> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -148> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -147> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -146> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): get_auth_request con 0x55ac5f3cbd60 auth_method 0 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -145> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -144> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -143> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:20:34.693 INFO:tasks.workunit.client.0.trial046.stderr: -142> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr: -141> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr: -140> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr: -139> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr: -138> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr: -137> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr: -136> 2026-02-24T22:20:34.584+0000 7f0e04d43640 10 monclient(hunting): get_auth_request con 0x55ac5f3ee9f0 auth_method 0 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr: -135> 2026-02-24T22:20:34.584+0000 7f0e04d43640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr: -134> 2026-02-24T22:20:34.584+0000 7f0e04d43640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr: -133> 2026-02-24T22:20:34.584+0000 7f0e04d43640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr: -132> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): handle_auth_done global_id 7709 payload 824 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr: -131> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient: _finish_hunting 0 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr: -130> 2026-02-24T22:20:34.584+0000 7f0dfffff640 1 monclient: found mon.noname-c 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr: -129> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr: -128> 2026-02-24T22:20:34.584+0000 7f0dd1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr: -127> 2026-02-24T22:20:34.584+0000 7f0dd1ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr: -126> 2026-02-24T22:20:34.584+0000 7f0dd1ffb640 10 monclient: dump: 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr: -125> 2026-02-24T22:20:34.584+0000 7f0dd1ffb640 10 monclient: _finish_auth 0 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr: -124> 2026-02-24T22:20:34.584+0000 7f0dd1ffb640 10 monclient: _check_auth_tickets 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr: -123> 2026-02-24T22:20:34.584+0000 7f0dd1ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T22:20:34.694 INFO:tasks.workunit.client.0.trial046.stderr: -122> 2026-02-24T22:20:34.584+0000 7f0dd1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr: -121> 2026-02-24T22:20:34.584+0000 7f0dd1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr: -120> 2026-02-24T22:20:34.584+0000 7f0dd1ffb640 10 monclient: dump: 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr: -119> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 monclient: authenticate success, global_id 7709 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr: -118> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: _renew_subs 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr: -117> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr: -116> 2026-02-24T22:20:34.584+0000 7f0dd37fe640 4 set_mon_vals no callback set 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr: -115> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: _renew_subs 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr: -114> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr: -113> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 1 librados: init done 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr: -112> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command cr dump hook 0x55ac5f3ef578 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr: -111> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 4 Realm: () 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr: -110> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr: -109> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr: -108> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr: -107> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 20 zonegroup default 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr: -106> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient: get_auth_request con 0x55ac5f4699f0 auth_method 0 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr: -105> 2026-02-24T22:20:34.588+0000 7f0e06a6ea80 20 rados->read ofs=0 len=0 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr: -104> 2026-02-24T22:20:34.588+0000 7f0dd1ffb640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr: -103> 2026-02-24T22:20:34.588+0000 7f0dd1ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr: -102> 2026-02-24T22:20:34.588+0000 7f0dd1ffb640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:20:34.695 INFO:tasks.workunit.client.0.trial046.stderr: -101> 2026-02-24T22:20:34.588+0000 7f0e04d43640 10 monclient: get_auth_request con 0x7f0dc40707a0 auth_method 0 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -100> 2026-02-24T22:20:34.588+0000 7f0dfffff640 10 monclient: get_auth_request con 0x55ac5f48cd10 auth_method 0 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -99> 2026-02-24T22:20:34.588+0000 7f0e06a6ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -98> 2026-02-24T22:20:34.588+0000 7f0e06a6ea80 20 rados->read ofs=0 len=0 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -97> 2026-02-24T22:20:34.588+0000 7f0df7fff640 10 monclient: get_auth_request con 0x55ac5f4ae390 auth_method 0 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -96> 2026-02-24T22:20:34.588+0000 7f0e06a6ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -95> 2026-02-24T22:20:34.588+0000 7f0e06a6ea80 20 started sync module instance, tier type = 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -94> 2026-02-24T22:20:34.588+0000 7f0e06a6ea80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -93> 2026-02-24T22:20:34.588+0000 7f0e04d43640 10 monclient: get_auth_request con 0x55ac5f50f840 auth_method 0 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -92> 2026-02-24T22:20:34.588+0000 7f0e04d43640 10 monclient: get_auth_request con 0x55ac5f6171d0 auth_method 0 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -91> 2026-02-24T22:20:34.588+0000 7f0dfffff640 10 monclient: get_auth_request con 0x55ac5f572110 auth_method 0 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -90> 2026-02-24T22:20:34.588+0000 7f0dfffff640 10 monclient: get_auth_request con 0x55ac5f67a1d0 auth_method 0 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -89> 2026-02-24T22:20:34.588+0000 7f0df7fff640 10 monclient: get_auth_request con 0x55ac5f5d5090 auth_method 0 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -88> 2026-02-24T22:20:34.588+0000 7f0df7fff640 0 -- 10.20.193.46:0/1054225154 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55ac5f4ae390 msgr2=0x55ac5f4ce840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -87> 2026-02-24T22:20:34.588+0000 7f0dd1ffb640 1 client.7709.objecter ms_handle_reset 0x55ac5f4ae390 session 0x55ac5f4adc80 osd.3 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -86> 2026-02-24T22:20:34.588+0000 7f0dd1ffb640 10 monclient: _renew_subs 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -85> 2026-02-24T22:20:34.588+0000 7f0dd1ffb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -84> 2026-02-24T22:20:34.588+0000 7f0df7fff640 10 monclient: get_auth_request con 0x7f0dc409d4d0 auth_method 0 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -83> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 add_watcher() i=6 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -82> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 add_watcher() i=1 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -81> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 add_watcher() i=3 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -80> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 add_watcher() i=5 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -79> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 add_watcher() i=0 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -78> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 add_watcher() i=7 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -77> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 add_watcher() i=2 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -76> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 add_watcher() i=4 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -75> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 2 all 8 watchers are set, enabling cache 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -74> 2026-02-24T22:20:34.592+0000 7f0df7fff640 10 monclient: get_auth_request con 0x7f0d24008af0 auth_method 0 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -73> 2026-02-24T22:20:34.592+0000 7f0df7fff640 0 -- 10.20.193.46:0/3878285961 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f0d24008af0 msgr2=0x7f0d24028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -72> 2026-02-24T22:20:34.592+0000 7f0de5ffb640 1 client.8259.objecter ms_handle_reset 0x7f0d24008af0 session 0x7f0d24008470 osd.1 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -71> 2026-02-24T22:20:34.592+0000 7f0de5ffb640 10 monclient: _renew_subs 2026-02-24T22:20:34.696 INFO:tasks.workunit.client.0.trial046.stderr: -70> 2026-02-24T22:20:34.592+0000 7f0de5ffb640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -69> 2026-02-24T22:20:34.592+0000 7f0df7fff640 10 monclient: get_auth_request con 0x7f0dd409d3f0 auth_method 0 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -68> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding auth protocol: cephx 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -67> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding auth protocol: cephx 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -66> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding auth protocol: cephx 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -65> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding con mode: secure 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -64> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding con mode: crc 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -63> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding con mode: secure 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -62> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding con mode: crc 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -61> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding con mode: secure 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -60> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding con mode: crc 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -59> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding con mode: crc 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -58> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding con mode: secure 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -57> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding con mode: crc 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -56> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding con mode: secure 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -55> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding con mode: crc 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -54> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding con mode: secure 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -53> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -52> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -51> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -50> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command cache list hook 0x55ac5f4015f0 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -49> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command cache inspect hook 0x55ac5f4015f0 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -48> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command cache erase hook 0x55ac5f4015f0 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -47> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command cache zap hook 0x55ac5f4015f0 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -46> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 note: GC not initialized 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -45> 2026-02-24T22:20:34.592+0000 7f0db1fdb640 20 reqs_thread_entry: start 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -44> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command sync trace show hook 0x7f0df00cb040 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -43> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command sync trace history hook 0x7f0df00cb040 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -42> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command sync trace active hook 0x7f0df00cb040 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -41> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command sync trace active_short hook 0x7f0df00cb040 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -40> 2026-02-24T22:20:34.592+0000 7f0e04d43640 10 monclient: get_auth_request con 0x55ac5f544d40 auth_method 0 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -39> 2026-02-24T22:20:34.596+0000 7f0dfffff640 10 monclient: get_auth_request con 0x55ac5f4c83a0 auth_method 0 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -38> 2026-02-24T22:20:34.596+0000 7f0df7fff640 10 monclient: get_auth_request con 0x55ac5f4c7fa0 auth_method 0 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -37> 2026-02-24T22:20:34.596+0000 7f0e04d43640 10 monclient: get_auth_request con 0x55ac5f544760 auth_method 0 2026-02-24T22:20:34.697 INFO:tasks.workunit.client.0.trial046.stderr: -36> 2026-02-24T22:20:34.600+0000 7f0dfffff640 10 monclient: get_auth_request con 0x55ac5f4ccde0 auth_method 0 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -35> 2026-02-24T22:20:34.604+0000 7f0df7fff640 10 monclient: get_auth_request con 0x55ac5f4cd790 auth_method 0 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -34> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 20 init_complete bucket index max shards: 1 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -33> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 20 Filter name: none 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -32> 2026-02-24T22:20:34.628+0000 7f0d0bfff640 20 reqs_thread_entry: start 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -31> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 cache get: name=default.rgw.meta+root+abort-at-change-reshard-state : miss 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -30> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 20 rados->read ofs=0 len=0 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -29> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -28> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 cache put: name=default.rgw.meta+root+abort-at-change-reshard-state info.flags=0x11 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -27> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 adding default.rgw.meta+root+abort-at-change-reshard-state to cache LRU end 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -26> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 : miss 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -25> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 20 rados->read ofs=0 len=0 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -24> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -23> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 info.flags=0x17 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -22> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 to cache LRU end 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -21> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -20> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -19> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -18> 2026-02-24T22:20:34.632+0000 7f0e06a6ea80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 info.flags=0x17 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -17> 2026-02-24T22:20:34.632+0000 7f0e06a6ea80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 to cache LRU end 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -16> 2026-02-24T22:20:34.632+0000 7f0e06a6ea80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -15> 2026-02-24T22:20:34.632+0000 7f0e06a6ea80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540, ofs0, ns] 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -14> 2026-02-24T22:20:34.632+0000 7f0dd37fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961394 cookie 94198822548960 notifier 7709 bl.length()=1033 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -13> 2026-02-24T22:20:34.632+0000 7f0dd37fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 info.flags=0x17 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -12> 2026-02-24T22:20:34.632+0000 7f0dd37fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 to cache LRU end 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -11> 2026-02-24T22:20:34.632+0000 7f0dd37fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -10> 2026-02-24T22:20:34.632+0000 7f0e06a6ea80 20 handle_bi_update(): bucket=:abort-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540]): orig_sources= new_sources= 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -9> 2026-02-24T22:20:34.632+0000 7f0e06a6ea80 20 handle_bi_update(): bucket=:abort-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540]): potential sources added=[] removed=[] 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -8> 2026-02-24T22:20:34.632+0000 7f0e06a6ea80 20 handle_bi_update(): bucket=:abort-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540]): orig_dests= new_dests= 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -7> 2026-02-24T22:20:34.632+0000 7f0e06a6ea80 20 handle_bi_update(): bucket=:abort-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540]): potential dests added=[] removed=[] 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -6> 2026-02-24T22:20:34.632+0000 7f0e06a6ea80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540.1.0 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -5> 2026-02-24T22:20:34.632+0000 7f0e06a6ea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540.1.0 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -4> 2026-02-24T22:20:34.632+0000 7f0e06a6ea80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540.1.1 2026-02-24T22:20:34.698 INFO:tasks.workunit.client.0.trial046.stderr: -3> 2026-02-24T22:20:34.632+0000 7f0e06a6ea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540.1.1 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: -2> 2026-02-24T22:20:34.632+0000 7f0e06a6ea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540.0 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: -1> 2026-02-24T22:20:34.636+0000 7f0e06a6ea80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f0e06a6ea80 time 2026-02-24T22:20:34.641018+0000 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f0e09eb0eb1] 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: 2: (RGWBucketReshard::do_reshard(rgw::bucket_index_layout_generation const&, rgw::bucket_index_layout_generation const&, int, bool, bool, std::ostream*, ceph::Formatter*, FaultInjector > >&, DoutPrefixProvider const*, optional_yield)+0x274a) [0x55ac50f71bea] 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xe6b) [0x55ac50f72f6b] 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f0e091b0d90] 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: 0> 2026-02-24T22:20:34.636+0000 7f0e06a6ea80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f0e06a6ea80 thread_name:radosgw-admin 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f0e091c9520] 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f0e09eb0f10] 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: 6: (RGWBucketReshard::do_reshard(rgw::bucket_index_layout_generation const&, rgw::bucket_index_layout_generation const&, int, bool, bool, std::ostream*, ceph::Formatter*, FaultInjector > >&, DoutPrefixProvider const*, optional_yield)+0x274a) [0x55ac50f71bea] 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xe6b) [0x55ac50f72f6b] 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f0e091b0d90] 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr:--- logging levels --- 2026-02-24T22:20:34.699 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 none 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 lockdep 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 context 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 crush 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_balancer 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_locker 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log_expire 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_migrator 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 3/ 5 mds_quiesce 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 buffer 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 timer 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 filer 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 striper 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 objecter 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rados 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_mirror 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_replay 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_pwl 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 journaler 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objectcacher 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 immutable_obj_cache 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 client 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 osd 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 optracker 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objclass 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 filestore 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 journal 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 ms 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mon 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 0/10 monc 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 paxos 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 tp 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 auth 2026-02-24T22:20:34.700 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 crypto 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 finisher 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 reserver 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 heartbeatmap 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 perfcounter 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_sync 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_datacache 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_access 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dbstore 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_flight 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_lifecycle 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_restore 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw_notification 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_bucket_logging 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 javaclient 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 asok 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 throttle 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 refs 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 compressor 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore_compression 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluefs 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 bdev 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 kstore 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 4/ 5 rocksdb 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 fuse 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 2/ 5 mgr 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mgrc 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 dpdk 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 eventtrace 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 prioritycache 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 test 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephfs_mirror 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephsqlite 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 crimson_interrupt 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore 2026-02-24T22:20:34.701 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_onode 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_odata 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_omap 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_tm 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_t 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cleaner 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_epm 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_lba 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_fixedkv_tree 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cache 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_journal 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_device 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_backref 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 alienstore 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mclock 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dedup 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cyanstore 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 ceph_exporter 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 memstore 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 trace 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 ceph_dedup 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: -2/-2 (syslog threshold) 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 99/99 (stderr threshold) 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr:--- pthread ID / name mapping for recent threads --- 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 7f0d0bfff640 / http_manager 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 7f0db1fdb640 / http_manager 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 7f0dd1ffb640 / ms_dispatch 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 7f0dd37fe640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 7f0de5ffb640 / ms_dispatch 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 7f0df4ff9640 / ms_dispatch 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 7f0df67fc640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 7f0df6ffd640 / admin_socket 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 7f0df7fff640 / msgr-worker-2 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 7f0dff7fe640 / ms_dispatch 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 7f0dfffff640 / msgr-worker-1 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 7f0e04d43640 / msgr-worker-0 2026-02-24T22:20:34.702 INFO:tasks.workunit.client.0.trial046.stderr: 7f0e05544640 / io_context_pool 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr: 7f0e06a6ea80 / radosgw-admin 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr: max_recent 500 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr: max_new 1000 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr: log_file /var/log/ceph/ceph-client.admin.53480.log 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr:--- end dump of recent events --- 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr:--- begin dump of recent events --- 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr: -499> 2026-02-24T22:20:34.572+0000 7f0dfffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr: -498> 2026-02-24T22:20:34.572+0000 7f0e04d43640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr: -497> 2026-02-24T22:20:34.572+0000 7f0e04d43640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr: -496> 2026-02-24T22:20:34.572+0000 7f0e04d43640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr: -495> 2026-02-24T22:20:34.572+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr: -494> 2026-02-24T22:20:34.572+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr: -493> 2026-02-24T22:20:34.572+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr: -492> 2026-02-24T22:20:34.572+0000 7f0dfffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr: -491> 2026-02-24T22:20:34.572+0000 7f0dfffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr: -490> 2026-02-24T22:20:34.572+0000 7f0dfffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr: -489> 2026-02-24T22:20:34.572+0000 7f0e04d43640 10 monclient(hunting): handle_auth_done global_id 8253 payload 274 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr: -488> 2026-02-24T22:20:34.572+0000 7f0e04d43640 10 monclient: _finish_hunting 0 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr: -487> 2026-02-24T22:20:34.572+0000 7f0e04d43640 1 monclient: found mon.noname-a 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr: -486> 2026-02-24T22:20:34.572+0000 7f0e04d43640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr: -485> 2026-02-24T22:20:34.572+0000 7f0df67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr: -484> 2026-02-24T22:20:34.572+0000 7f0df67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr: -483> 2026-02-24T22:20:34.572+0000 7f0df67fc640 10 monclient: dump: 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr: -482> 2026-02-24T22:20:34.572+0000 7f0df67fc640 10 monclient: _finish_auth 0 2026-02-24T22:20:34.703 INFO:tasks.workunit.client.0.trial046.stderr: -481> 2026-02-24T22:20:34.572+0000 7f0df67fc640 10 monclient: _check_auth_tickets 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr: -480> 2026-02-24T22:20:34.572+0000 7f0df67fc640 10 monclient: handle_config config(0 keys) 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr: -479> 2026-02-24T22:20:34.572+0000 7f0df67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr: -478> 2026-02-24T22:20:34.572+0000 7f0df67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr: -477> 2026-02-24T22:20:34.572+0000 7f0df67fc640 10 monclient: dump: 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr: -476> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 monclient: authenticate success, global_id 8253 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr: -475> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient: get_monmap_and_config success 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr: -474> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 4 set_mon_vals no callback set 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr: -473> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient: shutdown 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr: -472> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) unregister_commands rotate-key 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr: -471> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient: build_initial_monmap 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr: -470> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 1 build_initial for_mkfs: 0 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr: -469> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient: monmap: 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr: -468> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding auth protocol: cephx 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr: -467> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding auth protocol: cephx 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr: -466> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding auth protocol: cephx 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr: -465> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding con mode: secure 2026-02-24T22:20:34.704 INFO:tasks.workunit.client.0.trial046.stderr: -464> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding con mode: crc 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -463> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding con mode: secure 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -462> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding con mode: crc 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -461> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding con mode: secure 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -460> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding con mode: crc 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -459> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding con mode: crc 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -458> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding con mode: secure 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -457> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding con mode: crc 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -456> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding con mode: secure 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -455> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding con mode: crc 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -454> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f127030) adding con mode: secure 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -453> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -452> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 1 librados: starting msgr at 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -451> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 1 librados: starting objecter 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -450> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command objecter_requests hook 0x55ac5f158b60 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -449> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 1 librados: setting wanted keys 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -448> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 1 librados: calling monclient init 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -447> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient: init 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -446> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding auth protocol: cephx 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -445> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding auth protocol: cephx 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -444> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding auth protocol: cephx 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -443> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding con mode: secure 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -442> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding con mode: crc 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -441> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding con mode: secure 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -440> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding con mode: crc 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -439> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding con mode: secure 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -438> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding con mode: crc 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -437> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding con mode: crc 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -436> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding con mode: secure 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -435> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding con mode: crc 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -434> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding con mode: secure 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -433> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding con mode: crc 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -432> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f125130) adding con mode: secure 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -431> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -430> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -429> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command rotate-key hook 0x55ac5f125278 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -428> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient: _reopen_session rank -1 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -427> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T22:20:34.705 INFO:tasks.workunit.client.0.trial046.stderr: -426> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient(hunting): picked mon.noname-b con 0x55ac5f0b3fe0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -425> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient(hunting): picked mon.noname-c con 0x55ac5f15cb00 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -424> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient(hunting): picked mon.noname-a con 0x55ac5f15fcf0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -423> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient(hunting): start opening mon connection 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -422> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient(hunting): start opening mon connection 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -421> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient(hunting): start opening mon connection 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -420> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient(hunting): _renew_subs 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -419> 2026-02-24T22:20:34.572+0000 7f0e06a6ea80 10 monclient(hunting): authenticate will time out at 2004.683917s 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -418> 2026-02-24T22:20:34.572+0000 7f0df7fff640 10 monclient(hunting): get_auth_request con 0x55ac5f15cb00 auth_method 0 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -417> 2026-02-24T22:20:34.576+0000 7f0df7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -416> 2026-02-24T22:20:34.576+0000 7f0df7fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -415> 2026-02-24T22:20:34.576+0000 7f0df7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -414> 2026-02-24T22:20:34.576+0000 7f0e04d43640 10 monclient(hunting): get_auth_request con 0x55ac5f15fcf0 auth_method 0 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -413> 2026-02-24T22:20:34.576+0000 7f0e04d43640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -412> 2026-02-24T22:20:34.576+0000 7f0e04d43640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -411> 2026-02-24T22:20:34.576+0000 7f0e04d43640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -410> 2026-02-24T22:20:34.576+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -409> 2026-02-24T22:20:34.576+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -408> 2026-02-24T22:20:34.576+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -407> 2026-02-24T22:20:34.576+0000 7f0e04d43640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -406> 2026-02-24T22:20:34.576+0000 7f0e04d43640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -405> 2026-02-24T22:20:34.576+0000 7f0e04d43640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -404> 2026-02-24T22:20:34.576+0000 7f0dfffff640 10 monclient(hunting): get_auth_request con 0x55ac5f0b3fe0 auth_method 0 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -403> 2026-02-24T22:20:34.576+0000 7f0dfffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -402> 2026-02-24T22:20:34.576+0000 7f0dfffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -401> 2026-02-24T22:20:34.576+0000 7f0dfffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -400> 2026-02-24T22:20:34.576+0000 7f0df7fff640 10 monclient(hunting): handle_auth_done global_id 7700 payload 824 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -399> 2026-02-24T22:20:34.576+0000 7f0df7fff640 10 monclient: _finish_hunting 0 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -398> 2026-02-24T22:20:34.576+0000 7f0df7fff640 1 monclient: found mon.noname-c 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -397> 2026-02-24T22:20:34.576+0000 7f0df7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -396> 2026-02-24T22:20:34.576+0000 7f0df4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -395> 2026-02-24T22:20:34.576+0000 7f0df4ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr: -394> 2026-02-24T22:20:34.576+0000 7f0df4ff9640 10 monclient: dump: 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:20:34.706 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr: -393> 2026-02-24T22:20:34.576+0000 7f0df4ff9640 10 monclient: _finish_auth 0 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr: -392> 2026-02-24T22:20:34.576+0000 7f0df4ff9640 10 monclient: _check_auth_tickets 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr: -391> 2026-02-24T22:20:34.576+0000 7f0df4ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr: -390> 2026-02-24T22:20:34.576+0000 7f0df4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr: -389> 2026-02-24T22:20:34.576+0000 7f0df4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr: -388> 2026-02-24T22:20:34.576+0000 7f0df4ff9640 10 monclient: dump: 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr: -387> 2026-02-24T22:20:34.576+0000 7f0df67fc640 4 set_mon_vals no callback set 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr: -386> 2026-02-24T22:20:34.576+0000 7f0e06a6ea80 5 monclient: authenticate success, global_id 7700 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr: -385> 2026-02-24T22:20:34.576+0000 7f0e06a6ea80 10 monclient: _renew_subs 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr: -384> 2026-02-24T22:20:34.576+0000 7f0e06a6ea80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr: -383> 2026-02-24T22:20:34.576+0000 7f0e06a6ea80 10 monclient: _renew_subs 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr: -382> 2026-02-24T22:20:34.576+0000 7f0e06a6ea80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr: -381> 2026-02-24T22:20:34.576+0000 7f0e06a6ea80 1 librados: init done 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr: -380> 2026-02-24T22:20:34.576+0000 7f0df4ff9640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr: -379> 2026-02-24T22:20:34.576+0000 7f0df4ff9640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr: -378> 2026-02-24T22:20:34.576+0000 7f0df4ff9640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr: -377> 2026-02-24T22:20:34.576+0000 7f0dfffff640 10 monclient: get_auth_request con 0x7f0de00707a0 auth_method 0 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr: -376> 2026-02-24T22:20:34.576+0000 7f0e04d43640 10 monclient: get_auth_request con 0x55ac5f18edd0 auth_method 0 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr: -375> 2026-02-24T22:20:34.576+0000 7f0dfffff640 10 monclient: get_auth_request con 0x55ac5f1b34d0 auth_method 0 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr: -374> 2026-02-24T22:20:34.580+0000 7f0df7fff640 10 monclient: get_auth_request con 0x55ac5f1b4de0 auth_method 0 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr: -373> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient: build_initial_monmap 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr: -372> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 1 build_initial for_mkfs: 0 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr: -371> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient: monmap: 2026-02-24T22:20:34.707 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -370> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding auth protocol: cephx 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -369> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding auth protocol: cephx 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -368> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding auth protocol: cephx 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -367> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding con mode: secure 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -366> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding con mode: crc 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -365> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding con mode: secure 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -364> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding con mode: crc 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -363> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding con mode: secure 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -362> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding con mode: crc 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -361> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding con mode: crc 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -360> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding con mode: secure 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -359> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding con mode: crc 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -358> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding con mode: secure 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -357> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding con mode: crc 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -356> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b7d90) adding con mode: secure 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -355> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -354> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command objecter_requests cmddesc objecter_requests hook 0x55ac5f1bcc30 EEXIST 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -353> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient: init 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -352> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding auth protocol: cephx 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -351> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding auth protocol: cephx 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -350> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding auth protocol: cephx 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -349> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding con mode: secure 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -348> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding con mode: crc 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -347> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding con mode: secure 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -346> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding con mode: crc 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -345> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding con mode: secure 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -344> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding con mode: crc 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -343> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding con mode: crc 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -342> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding con mode: secure 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -341> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding con mode: crc 2026-02-24T22:20:34.708 INFO:tasks.workunit.client.0.trial046.stderr: -340> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding con mode: secure 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -339> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding con mode: crc 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -338> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f1b5708) adding con mode: secure 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -337> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -336> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -335> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command rotate-key cmddesc rotate-key hook 0x55ac5f1b5850 EEXIST 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -334> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient: _reopen_session rank -1 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -333> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -332> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient(hunting): picked mon.noname-b con 0x55ac5f310530 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -331> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient(hunting): picked mon.noname-c con 0x55ac5f30ea40 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -330> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient(hunting): picked mon.noname-a con 0x55ac5f30bc20 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -329> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient(hunting): start opening mon connection 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -328> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient(hunting): start opening mon connection 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -327> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient(hunting): start opening mon connection 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -326> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient(hunting): _renew_subs 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -325> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient(hunting): authenticate will time out at 2004.689417s 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -324> 2026-02-24T22:20:34.580+0000 7f0e04d43640 10 monclient(hunting): get_auth_request con 0x55ac5f30bc20 auth_method 0 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -323> 2026-02-24T22:20:34.580+0000 7f0e04d43640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -322> 2026-02-24T22:20:34.580+0000 7f0e04d43640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -321> 2026-02-24T22:20:34.580+0000 7f0e04d43640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -320> 2026-02-24T22:20:34.580+0000 7f0df7fff640 10 monclient(hunting): get_auth_request con 0x55ac5f30ea40 auth_method 0 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -319> 2026-02-24T22:20:34.580+0000 7f0df7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -318> 2026-02-24T22:20:34.580+0000 7f0df7fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -317> 2026-02-24T22:20:34.580+0000 7f0df7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -316> 2026-02-24T22:20:34.580+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -315> 2026-02-24T22:20:34.580+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -314> 2026-02-24T22:20:34.580+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -313> 2026-02-24T22:20:34.580+0000 7f0e04d43640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -312> 2026-02-24T22:20:34.580+0000 7f0e04d43640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -311> 2026-02-24T22:20:34.580+0000 7f0e04d43640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -310> 2026-02-24T22:20:34.580+0000 7f0dfffff640 10 monclient(hunting): get_auth_request con 0x55ac5f310530 auth_method 0 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -309> 2026-02-24T22:20:34.580+0000 7f0dfffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -308> 2026-02-24T22:20:34.580+0000 7f0dfffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -307> 2026-02-24T22:20:34.580+0000 7f0dfffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -306> 2026-02-24T22:20:34.580+0000 7f0e04d43640 10 monclient(hunting): handle_auth_done global_id 8259 payload 824 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -305> 2026-02-24T22:20:34.580+0000 7f0e04d43640 10 monclient: _finish_hunting 0 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -304> 2026-02-24T22:20:34.580+0000 7f0e04d43640 1 monclient: found mon.noname-a 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -303> 2026-02-24T22:20:34.580+0000 7f0e04d43640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -302> 2026-02-24T22:20:34.580+0000 7f0de5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -301> 2026-02-24T22:20:34.580+0000 7f0de5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr: -300> 2026-02-24T22:20:34.580+0000 7f0de5ffb640 10 monclient: dump: 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:20:34.709 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr: -299> 2026-02-24T22:20:34.580+0000 7f0de5ffb640 10 monclient: _finish_auth 0 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr: -298> 2026-02-24T22:20:34.580+0000 7f0de5ffb640 10 monclient: _check_auth_tickets 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr: -297> 2026-02-24T22:20:34.580+0000 7f0de5ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr: -296> 2026-02-24T22:20:34.580+0000 7f0de5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr: -295> 2026-02-24T22:20:34.580+0000 7f0de5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr: -294> 2026-02-24T22:20:34.580+0000 7f0de5ffb640 10 monclient: dump: 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr: -293> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 monclient: authenticate success, global_id 8259 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr: -292> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient: _renew_subs 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr: -291> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr: -290> 2026-02-24T22:20:34.580+0000 7f0e05544640 4 set_mon_vals no callback set 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr: -289> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient: _renew_subs 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr: -288> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr: -287> 2026-02-24T22:20:34.580+0000 7f0de5ffb640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr: -286> 2026-02-24T22:20:34.580+0000 7f0de5ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr: -285> 2026-02-24T22:20:34.580+0000 7f0de5ffb640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr: -284> 2026-02-24T22:20:34.580+0000 7f0dfffff640 10 monclient: get_auth_request con 0x7f0dd40706c0 auth_method 0 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr: -283> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient: get_monmap_and_config 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr: -282> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient: build_initial_monmap 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr: -281> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 1 build_initial for_mkfs: 0 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr: -280> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 10 monclient: monmap: 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:20:34.710 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -279> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding auth protocol: cephx 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -278> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding auth protocol: cephx 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -277> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding auth protocol: cephx 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -276> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding con mode: secure 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -275> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding con mode: crc 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -274> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding con mode: secure 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -273> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding con mode: crc 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -272> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding con mode: secure 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -271> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding con mode: crc 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -270> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding con mode: crc 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -269> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding con mode: secure 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -268> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding con mode: crc 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -267> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding con mode: secure 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -266> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding con mode: crc 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -265> 2026-02-24T22:20:34.580+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1580) adding con mode: secure 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -264> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -263> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: init 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -262> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding auth protocol: cephx 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -261> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding auth protocol: cephx 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -260> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding auth protocol: cephx 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -259> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding con mode: secure 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -258> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding con mode: crc 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -257> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding con mode: secure 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -256> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding con mode: crc 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -255> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding con mode: secure 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -254> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding con mode: crc 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -253> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding con mode: crc 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -252> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding con mode: secure 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -251> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding con mode: crc 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -250> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding con mode: secure 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -249> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding con mode: crc 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -248> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f24d0) adding con mode: secure 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -247> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:20:34.711 INFO:tasks.workunit.client.0.trial046.stderr: -246> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -245> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command rotate-key cmddesc rotate-key hook 0x7fff441f2618 EEXIST 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -244> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: _reopen_session rank -1 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -243> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -242> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): picked mon.noname-c con 0x55ac5f3c85f0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -241> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): picked mon.noname-a con 0x55ac5f3cbd60 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -240> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): picked mon.noname-b con 0x55ac5f3a3510 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -239> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): start opening mon connection 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -238> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): start opening mon connection 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -237> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): start opening mon connection 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -236> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): _renew_subs 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -235> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): authenticate will time out at 2004.692684s 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -234> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): get_auth_request con 0x55ac5f3cbd60 auth_method 0 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -233> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -232> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -231> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -230> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): get_auth_request con 0x55ac5f3c85f0 auth_method 0 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -229> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -228> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -227> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -226> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -225> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -224> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -223> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -222> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -221> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -220> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): handle_auth_done global_id 7706 payload 274 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -219> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient: _finish_hunting 0 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -218> 2026-02-24T22:20:34.584+0000 7f0dfffff640 1 monclient: found mon.noname-c 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -217> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -216> 2026-02-24T22:20:34.584+0000 7f0dd37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -215> 2026-02-24T22:20:34.584+0000 7f0dd37fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: -214> 2026-02-24T22:20:34.584+0000 7f0dd37fe640 10 monclient: dump: 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:20:34.712 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr: -213> 2026-02-24T22:20:34.584+0000 7f0dd37fe640 10 monclient: _finish_auth 0 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr: -212> 2026-02-24T22:20:34.584+0000 7f0dd37fe640 10 monclient: _check_auth_tickets 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr: -211> 2026-02-24T22:20:34.584+0000 7f0dd37fe640 10 monclient: handle_config config(0 keys) 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr: -210> 2026-02-24T22:20:34.584+0000 7f0dd37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr: -209> 2026-02-24T22:20:34.584+0000 7f0dd37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr: -208> 2026-02-24T22:20:34.584+0000 7f0dd37fe640 10 monclient: dump: 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr: -207> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 monclient: authenticate success, global_id 7706 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr: -206> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: get_monmap_and_config success 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr: -205> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 4 set_mon_vals no callback set 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr: -204> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: shutdown 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr: -203> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: build_initial_monmap 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr: -202> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 1 build_initial for_mkfs: 0 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr: -201> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: monmap: 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr: -200> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding auth protocol: cephx 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr: -199> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding auth protocol: cephx 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr: -198> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding auth protocol: cephx 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr: -197> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding con mode: secure 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr: -196> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding con mode: crc 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr: -195> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding con mode: secure 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr: -194> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding con mode: crc 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr: -193> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding con mode: secure 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr: -192> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding con mode: crc 2026-02-24T22:20:34.713 INFO:tasks.workunit.client.0.trial046.stderr: -191> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding con mode: crc 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -190> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding con mode: secure 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -189> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding con mode: crc 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -188> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding con mode: secure 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -187> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding con mode: crc 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -186> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a1450) adding con mode: secure 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -185> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -184> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 1 librados: starting msgr at 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -183> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 1 librados: starting objecter 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -182> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command objecter_requests cmddesc objecter_requests hook 0x55ac5f3094d0 EEXIST 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -181> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 1 librados: setting wanted keys 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -180> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 1 librados: calling monclient init 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -179> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: init 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -178> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding auth protocol: cephx 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -177> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding auth protocol: cephx 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -176> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding auth protocol: cephx 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -175> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding con mode: secure 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -174> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding con mode: crc 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -173> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding con mode: secure 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -172> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding con mode: crc 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -171> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding con mode: secure 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -170> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding con mode: crc 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -169> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding con mode: crc 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -168> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding con mode: secure 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -167> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding con mode: crc 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -166> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding con mode: secure 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -165> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding con mode: crc 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -164> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 AuthRegistry(0x55ac5f3a0380) adding con mode: secure 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -163> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -162> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -161> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command rotate-key cmddesc rotate-key hook 0x55ac5f3a04c8 EEXIST 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -160> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: _reopen_session rank -1 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -159> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -158> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): picked mon.noname-c con 0x55ac5f3c85f0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -157> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): picked mon.noname-a con 0x55ac5f3cbd60 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -156> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): picked mon.noname-b con 0x55ac5f3ee9f0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -155> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): start opening mon connection 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -154> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): start opening mon connection 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -153> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): start opening mon connection 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -152> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): _renew_subs 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -151> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient(hunting): authenticate will time out at 2004.693990s 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -150> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): get_auth_request con 0x55ac5f3c85f0 auth_method 0 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -149> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -148> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -147> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:20:34.714 INFO:tasks.workunit.client.0.trial046.stderr: -146> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): get_auth_request con 0x55ac5f3cbd60 auth_method 0 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: -145> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: -144> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: -143> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: -142> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: -141> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: -140> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: -139> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: -138> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: -137> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: -136> 2026-02-24T22:20:34.584+0000 7f0e04d43640 10 monclient(hunting): get_auth_request con 0x55ac5f3ee9f0 auth_method 0 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: -135> 2026-02-24T22:20:34.584+0000 7f0e04d43640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: -134> 2026-02-24T22:20:34.584+0000 7f0e04d43640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: -133> 2026-02-24T22:20:34.584+0000 7f0e04d43640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: -132> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient(hunting): handle_auth_done global_id 7709 payload 824 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: -131> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient: _finish_hunting 0 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: -130> 2026-02-24T22:20:34.584+0000 7f0dfffff640 1 monclient: found mon.noname-c 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: -129> 2026-02-24T22:20:34.584+0000 7f0dfffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: -128> 2026-02-24T22:20:34.584+0000 7f0dd1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: -127> 2026-02-24T22:20:34.584+0000 7f0dd1ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: -126> 2026-02-24T22:20:34.584+0000 7f0dd1ffb640 10 monclient: dump: 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: -125> 2026-02-24T22:20:34.584+0000 7f0dd1ffb640 10 monclient: _finish_auth 0 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: -124> 2026-02-24T22:20:34.584+0000 7f0dd1ffb640 10 monclient: _check_auth_tickets 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: -123> 2026-02-24T22:20:34.584+0000 7f0dd1ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: -122> 2026-02-24T22:20:34.584+0000 7f0dd1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: -121> 2026-02-24T22:20:34.584+0000 7f0dd1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr: -120> 2026-02-24T22:20:34.584+0000 7f0dd1ffb640 10 monclient: dump: 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:20:34.715 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -119> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 monclient: authenticate success, global_id 7709 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -118> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: _renew_subs 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -117> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -116> 2026-02-24T22:20:34.584+0000 7f0dd37fe640 4 set_mon_vals no callback set 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -115> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: _renew_subs 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -114> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -113> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 1 librados: init done 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -112> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command cr dump hook 0x55ac5f3ef578 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -111> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 4 Realm: () 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -110> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -109> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -108> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -107> 2026-02-24T22:20:34.584+0000 7f0e06a6ea80 20 zonegroup default 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -106> 2026-02-24T22:20:34.584+0000 7f0df7fff640 10 monclient: get_auth_request con 0x55ac5f4699f0 auth_method 0 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -105> 2026-02-24T22:20:34.588+0000 7f0e06a6ea80 20 rados->read ofs=0 len=0 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -104> 2026-02-24T22:20:34.588+0000 7f0dd1ffb640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -103> 2026-02-24T22:20:34.588+0000 7f0dd1ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -102> 2026-02-24T22:20:34.588+0000 7f0dd1ffb640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -101> 2026-02-24T22:20:34.588+0000 7f0e04d43640 10 monclient: get_auth_request con 0x7f0dc40707a0 auth_method 0 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -100> 2026-02-24T22:20:34.588+0000 7f0dfffff640 10 monclient: get_auth_request con 0x55ac5f48cd10 auth_method 0 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -99> 2026-02-24T22:20:34.588+0000 7f0e06a6ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -98> 2026-02-24T22:20:34.588+0000 7f0e06a6ea80 20 rados->read ofs=0 len=0 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -97> 2026-02-24T22:20:34.588+0000 7f0df7fff640 10 monclient: get_auth_request con 0x55ac5f4ae390 auth_method 0 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -96> 2026-02-24T22:20:34.588+0000 7f0e06a6ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -95> 2026-02-24T22:20:34.588+0000 7f0e06a6ea80 20 started sync module instance, tier type = 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -94> 2026-02-24T22:20:34.588+0000 7f0e06a6ea80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -93> 2026-02-24T22:20:34.588+0000 7f0e04d43640 10 monclient: get_auth_request con 0x55ac5f50f840 auth_method 0 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -92> 2026-02-24T22:20:34.588+0000 7f0e04d43640 10 monclient: get_auth_request con 0x55ac5f6171d0 auth_method 0 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -91> 2026-02-24T22:20:34.588+0000 7f0dfffff640 10 monclient: get_auth_request con 0x55ac5f572110 auth_method 0 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -90> 2026-02-24T22:20:34.588+0000 7f0dfffff640 10 monclient: get_auth_request con 0x55ac5f67a1d0 auth_method 0 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -89> 2026-02-24T22:20:34.588+0000 7f0df7fff640 10 monclient: get_auth_request con 0x55ac5f5d5090 auth_method 0 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -88> 2026-02-24T22:20:34.588+0000 7f0df7fff640 0 -- 10.20.193.46:0/1054225154 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55ac5f4ae390 msgr2=0x55ac5f4ce840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -87> 2026-02-24T22:20:34.588+0000 7f0dd1ffb640 1 client.7709.objecter ms_handle_reset 0x55ac5f4ae390 session 0x55ac5f4adc80 osd.3 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -86> 2026-02-24T22:20:34.588+0000 7f0dd1ffb640 10 monclient: _renew_subs 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -85> 2026-02-24T22:20:34.588+0000 7f0dd1ffb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -84> 2026-02-24T22:20:34.588+0000 7f0df7fff640 10 monclient: get_auth_request con 0x7f0dc409d4d0 auth_method 0 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -83> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 add_watcher() i=6 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -82> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 add_watcher() i=1 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -81> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 add_watcher() i=3 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -80> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 add_watcher() i=5 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -79> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 add_watcher() i=0 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -78> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 add_watcher() i=7 2026-02-24T22:20:34.716 INFO:tasks.workunit.client.0.trial046.stderr: -77> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 add_watcher() i=2 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -76> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 add_watcher() i=4 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -75> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 2 all 8 watchers are set, enabling cache 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -74> 2026-02-24T22:20:34.592+0000 7f0df7fff640 10 monclient: get_auth_request con 0x7f0d24008af0 auth_method 0 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -73> 2026-02-24T22:20:34.592+0000 7f0df7fff640 0 -- 10.20.193.46:0/3878285961 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f0d24008af0 msgr2=0x7f0d24028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -72> 2026-02-24T22:20:34.592+0000 7f0de5ffb640 1 client.8259.objecter ms_handle_reset 0x7f0d24008af0 session 0x7f0d24008470 osd.1 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -71> 2026-02-24T22:20:34.592+0000 7f0de5ffb640 10 monclient: _renew_subs 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -70> 2026-02-24T22:20:34.592+0000 7f0de5ffb640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -69> 2026-02-24T22:20:34.592+0000 7f0df7fff640 10 monclient: get_auth_request con 0x7f0dd409d3f0 auth_method 0 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -68> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding auth protocol: cephx 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -67> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding auth protocol: cephx 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -66> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding auth protocol: cephx 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -65> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding con mode: secure 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -64> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding con mode: crc 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -63> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding con mode: secure 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -62> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding con mode: crc 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -61> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding con mode: secure 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -60> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding con mode: crc 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -59> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding con mode: crc 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -58> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding con mode: secure 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -57> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding con mode: crc 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -56> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding con mode: secure 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -55> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding con mode: crc 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -54> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 AuthRegistry(0x7fff441f2940) adding con mode: secure 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -53> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -52> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -51> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -50> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command cache list hook 0x55ac5f4015f0 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -49> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command cache inspect hook 0x55ac5f4015f0 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -48> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command cache erase hook 0x55ac5f4015f0 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -47> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command cache zap hook 0x55ac5f4015f0 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -46> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 note: GC not initialized 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -45> 2026-02-24T22:20:34.592+0000 7f0db1fdb640 20 reqs_thread_entry: start 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -44> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command sync trace show hook 0x7f0df00cb040 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -43> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command sync trace history hook 0x7f0df00cb040 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -42> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command sync trace active hook 0x7f0df00cb040 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -41> 2026-02-24T22:20:34.592+0000 7f0e06a6ea80 5 asok(0x55ac5f025f00) register_command sync trace active_short hook 0x7f0df00cb040 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -40> 2026-02-24T22:20:34.592+0000 7f0e04d43640 10 monclient: get_auth_request con 0x55ac5f544d40 auth_method 0 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -39> 2026-02-24T22:20:34.596+0000 7f0dfffff640 10 monclient: get_auth_request con 0x55ac5f4c83a0 auth_method 0 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -38> 2026-02-24T22:20:34.596+0000 7f0df7fff640 10 monclient: get_auth_request con 0x55ac5f4c7fa0 auth_method 0 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -37> 2026-02-24T22:20:34.596+0000 7f0e04d43640 10 monclient: get_auth_request con 0x55ac5f544760 auth_method 0 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -36> 2026-02-24T22:20:34.600+0000 7f0dfffff640 10 monclient: get_auth_request con 0x55ac5f4ccde0 auth_method 0 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -35> 2026-02-24T22:20:34.604+0000 7f0df7fff640 10 monclient: get_auth_request con 0x55ac5f4cd790 auth_method 0 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -34> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 20 init_complete bucket index max shards: 1 2026-02-24T22:20:34.717 INFO:tasks.workunit.client.0.trial046.stderr: -33> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 20 Filter name: none 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -32> 2026-02-24T22:20:34.628+0000 7f0d0bfff640 20 reqs_thread_entry: start 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -31> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 cache get: name=default.rgw.meta+root+abort-at-change-reshard-state : miss 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -30> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 20 rados->read ofs=0 len=0 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -29> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -28> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 cache put: name=default.rgw.meta+root+abort-at-change-reshard-state info.flags=0x11 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -27> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 adding default.rgw.meta+root+abort-at-change-reshard-state to cache LRU end 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -26> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 : miss 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -25> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 20 rados->read ofs=0 len=0 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -24> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -23> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 info.flags=0x17 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -22> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 to cache LRU end 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -21> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -20> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -19> 2026-02-24T22:20:34.628+0000 7f0e06a6ea80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -18> 2026-02-24T22:20:34.632+0000 7f0e06a6ea80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 info.flags=0x17 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -17> 2026-02-24T22:20:34.632+0000 7f0e06a6ea80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 to cache LRU end 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -16> 2026-02-24T22:20:34.632+0000 7f0e06a6ea80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -15> 2026-02-24T22:20:34.632+0000 7f0e06a6ea80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540, ofs0, ns] 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -14> 2026-02-24T22:20:34.632+0000 7f0dd37fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961394 cookie 94198822548960 notifier 7709 bl.length()=1033 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -13> 2026-02-24T22:20:34.632+0000 7f0dd37fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 info.flags=0x17 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -12> 2026-02-24T22:20:34.632+0000 7f0dd37fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 to cache LRU end 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -11> 2026-02-24T22:20:34.632+0000 7f0dd37fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -10> 2026-02-24T22:20:34.632+0000 7f0e06a6ea80 20 handle_bi_update(): bucket=:abort-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540]): orig_sources= new_sources= 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -9> 2026-02-24T22:20:34.632+0000 7f0e06a6ea80 20 handle_bi_update(): bucket=:abort-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540]): potential sources added=[] removed=[] 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -8> 2026-02-24T22:20:34.632+0000 7f0e06a6ea80 20 handle_bi_update(): bucket=:abort-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540]): orig_dests= new_dests= 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -7> 2026-02-24T22:20:34.632+0000 7f0e06a6ea80 20 handle_bi_update(): bucket=:abort-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540]): potential dests added=[] removed=[] 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -6> 2026-02-24T22:20:34.632+0000 7f0e06a6ea80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540.1.0 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -5> 2026-02-24T22:20:34.632+0000 7f0e06a6ea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540.1.0 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -4> 2026-02-24T22:20:34.632+0000 7f0e06a6ea80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540.1.1 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -3> 2026-02-24T22:20:34.632+0000 7f0e06a6ea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540.1.1 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -2> 2026-02-24T22:20:34.632+0000 7f0e06a6ea80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540.0 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: -1> 2026-02-24T22:20:34.636+0000 7f0e06a6ea80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f0e06a6ea80 time 2026-02-24T22:20:34.641018+0000 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f0e09eb0eb1] 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: 2: (RGWBucketReshard::do_reshard(rgw::bucket_index_layout_generation const&, rgw::bucket_index_layout_generation const&, int, bool, bool, std::ostream*, ceph::Formatter*, FaultInjector > >&, DoutPrefixProvider const*, optional_yield)+0x274a) [0x55ac50f71bea] 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xe6b) [0x55ac50f72f6b] 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f0e091b0d90] 2026-02-24T22:20:34.718 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 0> 2026-02-24T22:20:34.636+0000 7f0e06a6ea80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f0e06a6ea80 thread_name:radosgw-admin 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f0e091c9520] 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f0e09eb0f10] 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 6: (RGWBucketReshard::do_reshard(rgw::bucket_index_layout_generation const&, rgw::bucket_index_layout_generation const&, int, bool, bool, std::ostream*, ceph::Formatter*, FaultInjector > >&, DoutPrefixProvider const*, optional_yield)+0x274a) [0x55ac50f71bea] 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xe6b) [0x55ac50f72f6b] 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f0e091b0d90] 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr:--- logging levels --- 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 none 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 lockdep 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 context 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 crush 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_balancer 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_locker 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log_expire 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_migrator 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 3/ 5 mds_quiesce 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 buffer 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 timer 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 filer 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 striper 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 objecter 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rados 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_mirror 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_replay 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_pwl 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 journaler 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objectcacher 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 immutable_obj_cache 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 client 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 osd 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 optracker 2026-02-24T22:20:34.719 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objclass 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 filestore 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 journal 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 ms 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mon 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 0/10 monc 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 paxos 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 tp 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 auth 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 crypto 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 finisher 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 reserver 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 heartbeatmap 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 perfcounter 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_sync 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_datacache 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_access 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dbstore 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_flight 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_lifecycle 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_restore 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw_notification 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_bucket_logging 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 javaclient 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 asok 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 throttle 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 refs 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 compressor 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore_compression 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluefs 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 bdev 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 kstore 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 4/ 5 rocksdb 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 fuse 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 2/ 5 mgr 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mgrc 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 dpdk 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 eventtrace 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 prioritycache 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 test 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephfs_mirror 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephsqlite 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 crimson_interrupt 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_onode 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_odata 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_omap 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_tm 2026-02-24T22:20:34.720 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_t 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cleaner 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_epm 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_lba 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_fixedkv_tree 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cache 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_journal 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_device 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_backref 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 alienstore 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mclock 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dedup 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cyanstore 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 ceph_exporter 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 memstore 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 trace 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 ceph_dedup 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: -2/-2 (syslog threshold) 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 99/99 (stderr threshold) 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr:--- pthread ID / name mapping for recent threads --- 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 7f0d0bfff640 / http_manager 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 7f0db1fdb640 / http_manager 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 7f0dd1ffb640 / ms_dispatch 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 7f0dd37fe640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 7f0de5ffb640 / ms_dispatch 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 7f0df4ff9640 / ms_dispatch 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 7f0df67fc640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 7f0df6ffd640 / admin_socket 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 7f0df7fff640 / msgr-worker-2 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 7f0dff7fe640 / ms_dispatch 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 7f0dfffff640 / msgr-worker-1 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 7f0e04d43640 / msgr-worker-0 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 7f0e05544640 / io_context_pool 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: 7f0e06a6ea80 / radosgw-admin 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: max_recent 500 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: max_new 1000 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr: log_file /var/lib/ceph/crash/2026-02-24T22:20:34.643121Z_965dd657-4fbc-4eb2-8bbf-49ae355184a7/log 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr:--- end dump of recent events --- 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr:Aborted 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket abort-at-change-reshard-state 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:20:34.721 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:20:34.722 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.676+0000 7f1542a58a80 4 Realm: () 2026-02-24T22:20:34.722 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.676+0000 7f1542a58a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:20:34.722 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.676+0000 7f1542a58a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:20:34.722 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.676+0000 7f1542a58a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:20:34.722 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.676+0000 7f1542a58a80 20 zonegroup default 2026-02-24T22:20:34.722 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.676+0000 7f1542a58a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:34.722 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.676+0000 7f1542a58a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:34.722 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.676+0000 7f1542a58a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:34.722 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.680+0000 7f1542a58a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:34.722 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.680+0000 7f1542a58a80 20 started sync module instance, tier type = 2026-02-24T22:20:34.722 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.680+0000 7f1542a58a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:20:34.722 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.680+0000 7f1542a58a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:20:34.722 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.680+0000 7f1542a58a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:20:34.722 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.680+0000 7f1542a58a80 5 note: GC not initialized 2026-02-24T22:20:34.722 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.680+0000 7f14ee7dc640 20 reqs_thread_entry: start 2026-02-24T22:20:34.722 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.684+0000 7f1533fff640 0 -- 10.20.193.46:0/1562051086 >> [v2:10.20.193.46:6826/3065971976,v1:10.20.193.46:6827/3065971976] conn(0x563ab7c01d00 msgr2=0x563ab7c02100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:20:34.890 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:34.892+0000 7f1533fff640 0 -- 10.20.193.46:0/1562051086 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x563ab7bece50 msgr2=0x563ab7bed2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:35.121 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.124+0000 7f1542a58a80 20 init_complete bucket index max shards: 1 2026-02-24T22:20:35.122 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.124+0000 7f1542a58a80 20 Filter name: none 2026-02-24T22:20:35.122 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.124+0000 7f14ecfd9640 20 reqs_thread_entry: start 2026-02-24T22:20:35.122 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.124+0000 7f1542a58a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:35.122 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.124+0000 7f1542a58a80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:20:35.123 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.124+0000 7f1542a58a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:35.123 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.124+0000 7f1542a58a80 20 rados_obj.operate() r=0 bl.length=491 2026-02-24T22:20:35.124 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.124+0000 7f1542a58a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:20:35.124 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.124+0000 7f1542a58a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:35.124 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.128+0000 7f1542a58a80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:20:35.124 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.128+0000 7f1542a58a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:35.124 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.128+0000 7f1542a58a80 20 rados_obj.operate() r=0 bl.length=491 2026-02-24T22:20:35.125 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.128+0000 7f1542a58a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:20:35.135 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:35.136 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:20:35.136 INFO:tasks.workunit.client.0.trial046.stderr:bucket: abort-at-change-reshard-state id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 num_objects: 20 size_kb: 1 num_shards: 1 2026-02-24T22:20:35.136 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:20:35.136 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:35.137 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:35.142 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:20:35.142 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:20:35.184 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket abort-at-change-reshard-state --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2026-02-24T22:20:35.201 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:20:35.201 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:20:35.218 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.220+0000 7f7ea4ed3a80 4 Realm: () 2026-02-24T22:20:35.218 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.220+0000 7f7ea4ed3a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:20:35.218 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.220+0000 7f7ea4ed3a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:20:35.218 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.220+0000 7f7ea4ed3a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:20:35.218 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.220+0000 7f7ea4ed3a80 20 zonegroup default 2026-02-24T22:20:35.218 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.220+0000 7f7ea4ed3a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:35.219 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.220+0000 7f7ea4ed3a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:35.219 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.220+0000 7f7ea4ed3a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:35.220 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.220+0000 7f7ea4ed3a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:20:35.220 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.220+0000 7f7ea4ed3a80 20 started sync module instance, tier type = 2026-02-24T22:20:35.220 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.220+0000 7f7ea4ed3a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:20:35.221 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.224+0000 7f7ea29a7640 0 -- 10.20.193.46:0/2120753579 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55b984464340 msgr2=0x55b984484760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:20:35.221 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.224+0000 7f7ea4ed3a80 20 add_watcher() i=1 2026-02-24T22:20:35.221 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.224+0000 7f7ea4ed3a80 20 add_watcher() i=3 2026-02-24T22:20:35.222 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.224+0000 7f7ea4ed3a80 20 add_watcher() i=0 2026-02-24T22:20:35.222 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.224+0000 7f7ea4ed3a80 20 add_watcher() i=5 2026-02-24T22:20:35.222 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.224+0000 7f7ea4ed3a80 20 add_watcher() i=2 2026-02-24T22:20:35.222 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.224+0000 7f7ea4ed3a80 20 add_watcher() i=4 2026-02-24T22:20:35.223 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.224+0000 7f7ea4ed3a80 20 add_watcher() i=7 2026-02-24T22:20:35.223 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.224+0000 7f7ea4ed3a80 20 add_watcher() i=6 2026-02-24T22:20:35.223 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.224+0000 7f7ea4ed3a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:20:35.224 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.228+0000 7f7ea4ed3a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:20:35.224 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.228+0000 7f7ea4ed3a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:20:35.224 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.228+0000 7f7ea4ed3a80 5 note: GC not initialized 2026-02-24T22:20:35.224 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.228+0000 7f7e50fe1640 20 reqs_thread_entry: start 2026-02-24T22:20:35.225 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.228+0000 7f7ea31a8640 0 -- 10.20.193.46:0/4038396143 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55b98457da90 msgr2=0x55b98457de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:20:35.466 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.468+0000 7f7ea4ed3a80 20 init_complete bucket index max shards: 1 2026-02-24T22:20:35.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.468+0000 7f7ea4ed3a80 20 Filter name: none 2026-02-24T22:20:35.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.468+0000 7f7dbaffd640 20 reqs_thread_entry: start 2026-02-24T22:20:35.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.468+0000 7f7ea4ed3a80 10 cache get: name=default.rgw.meta+root+abort-at-change-reshard-state : miss 2026-02-24T22:20:35.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.468+0000 7f7ea4ed3a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:35.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.468+0000 7f7ea4ed3a80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:20:35.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.468+0000 7f7ea4ed3a80 10 cache put: name=default.rgw.meta+root+abort-at-change-reshard-state info.flags=0x11 2026-02-24T22:20:35.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.468+0000 7f7ea4ed3a80 10 adding default.rgw.meta+root+abort-at-change-reshard-state to cache LRU end 2026-02-24T22:20:35.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.468+0000 7f7ea4ed3a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 : miss 2026-02-24T22:20:35.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.468+0000 7f7ea4ed3a80 20 rados->read ofs=0 len=0 2026-02-24T22:20:35.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.468+0000 7f7ea4ed3a80 20 rados_obj.operate() r=0 bl.length=491 2026-02-24T22:20:35.467 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.468+0000 7f7ea4ed3a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 info.flags=0x17 2026-02-24T22:20:35.468 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.468+0000 7f7ea4ed3a80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 to cache LRU end 2026-02-24T22:20:35.468 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.468+0000 7f7ea4ed3a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:20:35.468 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.468+0000 7f7ea4ed3a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:20:35.468 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.468+0000 7f7ea4ed3a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 2026-02-24T22:20:35.468 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.468+0000 7f7ea4ed3a80 0 INFO: RGWReshardLock::lock found lock on abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 to be held by another RGW process; skipping for now 2026-02-24T22:20:35.470 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.472+0000 7f7ea4ed3a80 20 remove_watcher() i=3 2026-02-24T22:20:35.470 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.472+0000 7f7ea4ed3a80 2 removed watcher, disabling cache 2026-02-24T22:20:35.470 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.472+0000 7f7ea4ed3a80 20 remove_watcher() i=2 2026-02-24T22:20:35.471 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.472+0000 7f7ea4ed3a80 20 remove_watcher() i=4 2026-02-24T22:20:35.471 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.472+0000 7f7ea4ed3a80 20 remove_watcher() i=0 2026-02-24T22:20:35.471 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.472+0000 7f7ea4ed3a80 20 remove_watcher() i=1 2026-02-24T22:20:35.471 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.472+0000 7f7ea4ed3a80 20 remove_watcher() i=5 2026-02-24T22:20:35.471 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.472+0000 7f7ea4ed3a80 20 remove_watcher() i=6 2026-02-24T22:20:35.471 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:20:35.472+0000 7f7ea4ed3a80 20 remove_watcher() i=7 2026-02-24T22:20:35.476 INFO:tasks.workunit.client.0.trial046.stderr:waiting 30 seconds for reshard lock to expire... 2026-02-24T22:21:05.502 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket abort-at-change-reshard-state --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2026-02-24T22:21:05.519 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:21:05.519 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:21:05.536 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.540+0000 7fd919521a80 4 Realm: () 2026-02-24T22:21:05.536 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.540+0000 7fd919521a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:21:05.537 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.540+0000 7fd919521a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:21:05.537 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.540+0000 7fd919521a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:21:05.537 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.540+0000 7fd919521a80 20 zonegroup default 2026-02-24T22:21:05.537 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.540+0000 7fd919521a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:05.538 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.540+0000 7fd919521a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:05.538 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.540+0000 7fd919521a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:05.538 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.540+0000 7fd919521a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:05.539 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.540+0000 7fd919521a80 20 started sync module instance, tier type = 2026-02-24T22:21:05.539 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.540+0000 7fd919521a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:21:05.540 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.544+0000 7fd9177f6640 0 -- 10.20.193.46:0/1131607856 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x56071c8e7c40 msgr2=0x56071c9080c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:05.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.544+0000 7fd919521a80 20 add_watcher() i=1 2026-02-24T22:21:05.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.544+0000 7fd919521a80 20 add_watcher() i=3 2026-02-24T22:21:05.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.544+0000 7fd919521a80 20 add_watcher() i=6 2026-02-24T22:21:05.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.544+0000 7fd919521a80 20 add_watcher() i=0 2026-02-24T22:21:05.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.544+0000 7fd919521a80 20 add_watcher() i=2 2026-02-24T22:21:05.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.544+0000 7fd919521a80 20 add_watcher() i=5 2026-02-24T22:21:05.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.544+0000 7fd919521a80 20 add_watcher() i=4 2026-02-24T22:21:05.542 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.544+0000 7fd919521a80 20 add_watcher() i=7 2026-02-24T22:21:05.542 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.544+0000 7fd919521a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:21:05.543 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.544+0000 7fd919521a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:21:05.543 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.544+0000 7fd919521a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:21:05.543 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.544+0000 7fd919521a80 5 note: GC not initialized 2026-02-24T22:21:05.543 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.544+0000 7fd8c4fe1640 20 reqs_thread_entry: start 2026-02-24T22:21:05.544 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.548+0000 7fd9177f6640 0 -- 10.20.193.46:0/2015340978 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x56071c876580 msgr2=0x56071c8fe280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:05.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.788+0000 7fd919521a80 20 init_complete bucket index max shards: 1 2026-02-24T22:21:05.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.788+0000 7fd919521a80 20 Filter name: none 2026-02-24T22:21:05.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.788+0000 7fd82effd640 20 reqs_thread_entry: start 2026-02-24T22:21:05.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.788+0000 7fd919521a80 10 cache get: name=default.rgw.meta+root+abort-at-change-reshard-state : miss 2026-02-24T22:21:05.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.788+0000 7fd919521a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:05.786 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.788+0000 7fd919521a80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:21:05.786 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.788+0000 7fd919521a80 10 cache put: name=default.rgw.meta+root+abort-at-change-reshard-state info.flags=0x11 2026-02-24T22:21:05.786 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.788+0000 7fd919521a80 10 adding default.rgw.meta+root+abort-at-change-reshard-state to cache LRU end 2026-02-24T22:21:05.786 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.788+0000 7fd919521a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 : miss 2026-02-24T22:21:05.786 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.788+0000 7fd919521a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:05.787 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.788+0000 7fd919521a80 20 rados_obj.operate() r=0 bl.length=491 2026-02-24T22:21:05.787 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.788+0000 7fd919521a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 info.flags=0x17 2026-02-24T22:21:05.787 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.788+0000 7fd919521a80 10 adding default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 to cache LRU end 2026-02-24T22:21:05.787 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.788+0000 7fd919521a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:21:05.787 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.788+0000 7fd919521a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:21:05.787 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.788+0000 7fd919521a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 2026-02-24T22:21:05.787 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.788+0000 7fd919521a80 10 init_target_layout removing existing target index objects from a previous reshard attempt 2026-02-24T22:21:05.790 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.792+0000 7fd919521a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 info.flags=0x17 2026-02-24T22:21:05.791 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.792+0000 7fd919521a80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 to cache LRU end 2026-02-24T22:21:05.791 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.792+0000 7fd919521a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:21:05.791 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.792+0000 7fd919521a80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540, ofs0, ns] 2026-02-24T22:21:05.791 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.792+0000 7fd8f2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961395 cookie 94588542240512 notifier 8316 bl.length()=1033 2026-02-24T22:21:05.792 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.792+0000 7fd8f2ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 info.flags=0x17 2026-02-24T22:21:05.792 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.792+0000 7fd8f2ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 to cache LRU end 2026-02-24T22:21:05.792 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.792+0000 7fd8f2ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:21:05.792 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.792+0000 7fd919521a80 20 handle_bi_update(): bucket=:abort-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540]): orig_sources= new_sources= 2026-02-24T22:21:05.792 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.792+0000 7fd919521a80 20 handle_bi_update(): bucket=:abort-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540]): potential sources added=[] removed=[] 2026-02-24T22:21:05.792 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.792+0000 7fd919521a80 20 handle_bi_update(): bucket=:abort-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540]): orig_dests= new_dests= 2026-02-24T22:21:05.792 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.792+0000 7fd919521a80 20 handle_bi_update(): bucket=:abort-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540]): potential dests added=[] removed=[] 2026-02-24T22:21:05.793 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.796+0000 7fd8f2ffd640 20 trim reshard logs: write on '.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540.0' needs retry: Success 2026-02-24T22:21:05.793 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.796+0000 7fd919521a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540.2.0 2026-02-24T22:21:05.794 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.796+0000 7fd919521a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540.2.0 2026-02-24T22:21:05.794 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.796+0000 7fd919521a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540.2.1 2026-02-24T22:21:05.794 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.796+0000 7fd919521a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540.2.1 2026-02-24T22:21:05.794 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.796+0000 7fd919521a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540.0 2026-02-24T22:21:05.795 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.796+0000 7fd919521a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 : hit (requested=0x17, cached=0x17) 2026-02-24T22:21:05.795 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.796+0000 7fd919521a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 2026-02-24T22:21:05.796 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd919521a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 info.flags=0x17 2026-02-24T22:21:05.796 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd919521a80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 to cache LRU end 2026-02-24T22:21:05.796 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd919521a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:21:05.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd919521a80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540, ofs0, ns] 2026-02-24T22:21:05.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd8f27fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961396 cookie 94588542240512 notifier 8316 bl.length()=1033 2026-02-24T22:21:05.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd8f27fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 info.flags=0x17 2026-02-24T22:21:05.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd8f27fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 to cache LRU end 2026-02-24T22:21:05.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd8f27fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:21:05.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd919521a80 20 handle_bi_update(): bucket=:abort-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540]): orig_sources= new_sources= 2026-02-24T22:21:05.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd919521a80 20 handle_bi_update(): bucket=:abort-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540]): potential sources added=[] removed=[] 2026-02-24T22:21:05.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd919521a80 20 handle_bi_update(): bucket=:abort-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540]): orig_dests= new_dests= 2026-02-24T22:21:05.797 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd919521a80 20 handle_bi_update(): bucket=:abort-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540]): potential dests added=[] removed=[] 2026-02-24T22:21:05.798 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd919521a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540.0 2026-02-24T22:21:05.798 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd919521a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 : hit (requested=0x17, cached=0x17) 2026-02-24T22:21:05.798 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd919521a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 2026-02-24T22:21:05.799 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd919521a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 info.flags=0x17 2026-02-24T22:21:05.799 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd919521a80 10 moving default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 to cache LRU end 2026-02-24T22:21:05.799 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd919521a80 10 updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:21:05.799 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd919521a80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540, ofs0, ns] 2026-02-24T22:21:05.799 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd8f2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961397 cookie 94588542240512 notifier 8316 bl.length()=997 2026-02-24T22:21:05.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd8f2ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 info.flags=0x17 2026-02-24T22:21:05.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd8f2ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-change-reshard-state:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 to cache LRU end 2026-02-24T22:21:05.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd8f2ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2026-02-24T22:21:05.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd919521a80 20 handle_bi_update(): bucket=:abort-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540]): orig_sources= new_sources= 2026-02-24T22:21:05.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd919521a80 20 handle_bi_update(): bucket=:abort-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540]): potential sources added=[] removed=[] 2026-02-24T22:21:05.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd919521a80 20 handle_bi_update(): bucket=:abort-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540]): orig_dests= new_dests= 2026-02-24T22:21:05.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.800+0000 7fd919521a80 20 handle_bi_update(): bucket=:abort-at-change-reshard-state[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540]): potential dests added=[] removed=[] 2026-02-24T22:21:05.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.804+0000 7fd919521a80 1 execute INFO: reshard of bucket "abort-at-change-reshard-state" from 1 shards to 2 shards completed successfully 2026-02-24T22:21:05.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.804+0000 7fd919521a80 20 remove_watcher() i=3 2026-02-24T22:21:05.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.804+0000 7fd919521a80 2 removed watcher, disabling cache 2026-02-24T22:21:05.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.804+0000 7fd919521a80 20 remove_watcher() i=0 2026-02-24T22:21:05.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.804+0000 7fd919521a80 20 remove_watcher() i=2 2026-02-24T22:21:05.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.804+0000 7fd919521a80 20 remove_watcher() i=1 2026-02-24T22:21:05.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.804+0000 7fd919521a80 20 remove_watcher() i=6 2026-02-24T22:21:05.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.808+0000 7fd919521a80 20 remove_watcher() i=5 2026-02-24T22:21:05.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.808+0000 7fd919521a80 20 remove_watcher() i=7 2026-02-24T22:21:05.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.808+0000 7fd919521a80 20 remove_watcher() i=4 2026-02-24T22:21:05.811 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket abort-at-change-reshard-state 2026-02-24T22:21:05.824 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:21:05.824 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:21:05.842 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.844+0000 7fe85b39ea80 4 Realm: () 2026-02-24T22:21:05.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.844+0000 7fe85b39ea80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:21:05.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.844+0000 7fe85b39ea80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:21:05.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.844+0000 7fe85b39ea80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:21:05.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.844+0000 7fe85b39ea80 20 zonegroup default 2026-02-24T22:21:05.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.844+0000 7fe85b39ea80 20 rados->read ofs=0 len=0 2026-02-24T22:21:05.844 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.848+0000 7fe85b39ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:05.844 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.848+0000 7fe85b39ea80 20 rados->read ofs=0 len=0 2026-02-24T22:21:05.845 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.848+0000 7fe85b39ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:05.845 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.848+0000 7fe85b39ea80 20 started sync module instance, tier type = 2026-02-24T22:21:05.845 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.848+0000 7fe85b39ea80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:21:05.847 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.848+0000 7fe85b39ea80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:21:05.847 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.848+0000 7fe85b39ea80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:21:05.847 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.848+0000 7fe85b39ea80 5 note: GC not initialized 2026-02-24T22:21:05.847 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.848+0000 7fe8067e4640 20 reqs_thread_entry: start 2026-02-24T22:21:05.849 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:05.852+0000 7fe853fff640 0 -- 10.20.193.46:0/2826550779 >> [v2:10.20.193.46:6826/3065971976,v1:10.20.193.46:6827/3065971976] conn(0x555d6723caf0 msgr2=0x555d6723cec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:06.058 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.060+0000 7fe853fff640 0 -- 10.20.193.46:0/2826550779 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x555d6723b3f0 msgr2=0x555d67227b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:06.289 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.292+0000 7fe85b39ea80 20 init_complete bucket index max shards: 1 2026-02-24T22:21:06.290 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.292+0000 7fe85b39ea80 20 Filter name: none 2026-02-24T22:21:06.290 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.292+0000 7fe804fe1640 20 reqs_thread_entry: start 2026-02-24T22:21:06.290 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.292+0000 7fe85b39ea80 20 rados->read ofs=0 len=0 2026-02-24T22:21:06.291 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.292+0000 7fe85b39ea80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:21:06.291 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.292+0000 7fe85b39ea80 20 rados->read ofs=0 len=0 2026-02-24T22:21:06.291 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.292+0000 7fe85b39ea80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:21:06.291 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.296+0000 7fe85b39ea80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:21:06.292 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.296+0000 7fe85b39ea80 20 rados->read ofs=0 len=0 2026-02-24T22:21:06.292 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.296+0000 7fe85b39ea80 20 rados_obj.operate() r=0 bl.length=225 2026-02-24T22:21:06.292 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.296+0000 7fe85b39ea80 20 rados->read ofs=0 len=0 2026-02-24T22:21:06.292 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.296+0000 7fe85b39ea80 20 rados_obj.operate() r=0 bl.length=455 2026-02-24T22:21:06.292 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.296+0000 7fe85b39ea80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:21:06.303 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:06.303 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:21:06.303 INFO:tasks.workunit.client.0.trial046.stderr:bucket: abort-at-change-reshard-state id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540 num_objects: 19 size_kb: 1 num_shards: 2 2026-02-24T22:21:06.303 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:06.304 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:21:06.304 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:21:06.308 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:21:06.308 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:21:06.353 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:21:06.353 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:21:06.363 INFO:tasks.workunit.client.0.trial046.stderr: test: logrecord could be stopped after reshard failed 2026-02-24T22:21:06.364 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket a-bucket 2026-02-24T22:21:06.379 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:21:06.379 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:21:06.397 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.400+0000 7ff6f9e42a80 4 Realm: () 2026-02-24T22:21:06.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.400+0000 7ff6f9e42a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:21:06.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.400+0000 7ff6f9e42a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:21:06.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.400+0000 7ff6f9e42a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:21:06.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.400+0000 7ff6f9e42a80 20 zonegroup default 2026-02-24T22:21:06.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.400+0000 7ff6f9e42a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:06.400 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.404+0000 7ff6f9e42a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:06.400 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.404+0000 7ff6f9e42a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:06.400 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.404+0000 7ff6f9e42a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:06.401 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.404+0000 7ff6f9e42a80 20 started sync module instance, tier type = 2026-02-24T22:21:06.401 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.404+0000 7ff6f9e42a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:21:06.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.404+0000 7ff6f9e42a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:21:06.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.404+0000 7ff6f9e42a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:21:06.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.404+0000 7ff6f9e42a80 5 note: GC not initialized 2026-02-24T22:21:06.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.404+0000 7ff6a57e2640 20 reqs_thread_entry: start 2026-02-24T22:21:06.406 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.408+0000 7ff6f2ffd640 0 -- 10.20.193.46:0/3077073209 >> [v2:10.20.193.46:6826/3065971976,v1:10.20.193.46:6827/3065971976] conn(0x555b9d8c36a0 msgr2=0x555b9d8c3aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:06.410 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.412+0000 7ff6f2ffd640 0 -- 10.20.193.46:0/3077073209 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x555b9d8c2210 msgr2=0x555b9d8ae480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:06.641 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.644+0000 7ff6f9e42a80 20 init_complete bucket index max shards: 1 2026-02-24T22:21:06.641 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.644+0000 7ff6f9e42a80 20 Filter name: none 2026-02-24T22:21:06.641 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.644+0000 7ff60f7fe640 20 reqs_thread_entry: start 2026-02-24T22:21:06.641 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.644+0000 7ff6f9e42a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:06.642 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.644+0000 7ff6f9e42a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:21:06.643 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.644+0000 7ff6f9e42a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:06.643 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.644+0000 7ff6f9e42a80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:21:06.643 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.644+0000 7ff6f9e42a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:21:06.643 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.644+0000 7ff6f9e42a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:06.643 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.644+0000 7ff6f9e42a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:21:06.643 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.644+0000 7ff6f9e42a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:06.643 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.644+0000 7ff6f9e42a80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:21:06.643 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.644+0000 7ff6f9e42a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:21:06.653 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:06.653 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:21:06.653 INFO:tasks.workunit.client.0.trial046.stderr:bucket: a-bucket id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 num_objects: 8 size_kb: 1 num_shards: 3 2026-02-24T22:21:06.653 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:06.654 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket a-bucket 2026-02-24T22:21:06.668 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:21:06.668 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:21:06.685 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.688+0000 7f06ae913a80 4 Realm: () 2026-02-24T22:21:06.685 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.688+0000 7f06ae913a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:21:06.685 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.688+0000 7f06ae913a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:21:06.685 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.688+0000 7f06ae913a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:21:06.685 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.688+0000 7f06ae913a80 20 zonegroup default 2026-02-24T22:21:06.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.688+0000 7f06ae913a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:06.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.688+0000 7f06ae913a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:06.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.688+0000 7f06ae913a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:06.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.692+0000 7f06ae913a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:06.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.692+0000 7f06ae913a80 20 started sync module instance, tier type = 2026-02-24T22:21:06.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.692+0000 7f06ae913a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:21:06.689 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.692+0000 7f06ae913a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:21:06.689 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.692+0000 7f06ae913a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:21:06.689 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.692+0000 7f06ae913a80 5 note: GC not initialized 2026-02-24T22:21:06.689 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.692+0000 7f065a7dc640 20 reqs_thread_entry: start 2026-02-24T22:21:06.694 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.696+0000 7f06a7fff640 0 -- 10.20.193.46:0/3027413673 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x5591a27929f0 msgr2=0x5591a278a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:06.905 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:06.908+0000 7f06acbe8640 0 -- 10.20.193.46:0/3027413673 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x5591a277b5a0 msgr2=0x5591a277b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:07.133 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.136+0000 7f06ae913a80 20 init_complete bucket index max shards: 1 2026-02-24T22:21:07.134 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.136+0000 7f06ae913a80 20 Filter name: none 2026-02-24T22:21:07.134 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.136+0000 7f0658fd9640 20 reqs_thread_entry: start 2026-02-24T22:21:07.134 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.136+0000 7f06ae913a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:07.135 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.136+0000 7f06ae913a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:21:07.135 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.136+0000 7f06ae913a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:07.135 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.136+0000 7f06ae913a80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:21:07.135 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.136+0000 7f06ae913a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:21:07.135 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.136+0000 7f06ae913a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:07.135 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.136+0000 7f06ae913a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:21:07.135 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.136+0000 7f06ae913a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:07.135 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.136+0000 7f06ae913a80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:21:07.136 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.136+0000 7f06ae913a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:21:07.146 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket a-bucket --num-shards 4 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at change_reshard_state 2026-02-24T22:21:07.163 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:21:07.163 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:21:07.182 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.184+0000 7f857503ba80 4 Realm: () 2026-02-24T22:21:07.182 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.184+0000 7f857503ba80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:21:07.182 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.184+0000 7f857503ba80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:21:07.182 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.184+0000 7f857503ba80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:21:07.183 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.184+0000 7f857503ba80 20 zonegroup default 2026-02-24T22:21:07.183 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.184+0000 7f857503ba80 20 rados->read ofs=0 len=0 2026-02-24T22:21:07.184 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.188+0000 7f857503ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:07.184 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.188+0000 7f857503ba80 20 rados->read ofs=0 len=0 2026-02-24T22:21:07.184 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.188+0000 7f857503ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:07.185 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.188+0000 7f857503ba80 20 started sync module instance, tier type = 2026-02-24T22:21:07.185 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.188+0000 7f857503ba80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:21:07.186 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.188+0000 7f8572b0f640 0 -- 10.20.193.46:0/3692073238 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55cd62a54420 msgr2=0x55cd62a748d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:07.186 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.188+0000 7f857503ba80 20 add_watcher() i=1 2026-02-24T22:21:07.186 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.188+0000 7f857503ba80 20 add_watcher() i=3 2026-02-24T22:21:07.187 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.188+0000 7f857503ba80 20 add_watcher() i=0 2026-02-24T22:21:07.187 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.188+0000 7f857503ba80 20 add_watcher() i=5 2026-02-24T22:21:07.187 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.188+0000 7f857503ba80 20 add_watcher() i=2 2026-02-24T22:21:07.187 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.188+0000 7f857503ba80 20 add_watcher() i=4 2026-02-24T22:21:07.188 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.192+0000 7f857503ba80 20 add_watcher() i=6 2026-02-24T22:21:07.188 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.192+0000 7f857503ba80 20 add_watcher() i=7 2026-02-24T22:21:07.188 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.192+0000 7f857503ba80 2 all 8 watchers are set, enabling cache 2026-02-24T22:21:07.189 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.192+0000 7f857230e640 0 -- 10.20.193.46:0/2318160246 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f8494008ac0 msgr2=0x7f8494028f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:07.189 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.192+0000 7f857503ba80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:21:07.190 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.192+0000 7f857503ba80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:21:07.190 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.192+0000 7f857503ba80 5 note: GC not initialized 2026-02-24T22:21:07.190 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.192+0000 7f8520fe1640 20 reqs_thread_entry: start 2026-02-24T22:21:07.221 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.224+0000 7f857503ba80 20 init_complete bucket index max shards: 1 2026-02-24T22:21:07.221 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.224+0000 7f857503ba80 20 Filter name: none 2026-02-24T22:21:07.221 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.224+0000 7f848affd640 20 reqs_thread_entry: start 2026-02-24T22:21:07.221 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.224+0000 7f857503ba80 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2026-02-24T22:21:07.221 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.224+0000 7f857503ba80 20 rados->read ofs=0 len=0 2026-02-24T22:21:07.222 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.224+0000 7f857503ba80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:21:07.222 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.224+0000 7f857503ba80 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2026-02-24T22:21:07.222 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.224+0000 7f857503ba80 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2026-02-24T22:21:07.222 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.224+0000 7f857503ba80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : miss 2026-02-24T22:21:07.222 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.224+0000 7f857503ba80 20 rados->read ofs=0 len=0 2026-02-24T22:21:07.222 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.224+0000 7f857503ba80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:21:07.223 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.224+0000 7f857503ba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:21:07.223 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.224+0000 7f857503ba80 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:21:07.223 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.224+0000 7f857503ba80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:21:07.223 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.224+0000 7f857503ba80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:21:07.223 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.224+0000 7f857503ba80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:21:07.225 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.228+0000 7f857503ba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:21:07.226 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.228+0000 7f857503ba80 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:21:07.226 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.228+0000 7f857503ba80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:21:07.226 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.228+0000 7f857503ba80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518, ofs0, ns] 2026-02-24T22:21:07.226 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.228+0000 7f854e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959943 cookie 94340613537584 notifier 7826 bl.length()=929 2026-02-24T22:21:07.226 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.228+0000 7f854e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:21:07.226 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.228+0000 7f854e7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:21:07.226 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.228+0000 7f854e7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:21:07.226 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.228+0000 7f857503ba80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_sources= new_sources= 2026-02-24T22:21:07.226 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.228+0000 7f857503ba80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential sources added=[] removed=[] 2026-02-24T22:21:07.226 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.228+0000 7f857503ba80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_dests= new_dests= 2026-02-24T22:21:07.226 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.228+0000 7f857503ba80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential dests added=[] removed=[] 2026-02-24T22:21:07.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.232+0000 7f857503ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.0 2026-02-24T22:21:07.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.232+0000 7f857503ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.0 2026-02-24T22:21:07.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.232+0000 7f857503ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.1 2026-02-24T22:21:07.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.232+0000 7f857503ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.1 2026-02-24T22:21:07.229 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.232+0000 7f857503ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.2 2026-02-24T22:21:07.229 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.232+0000 7f857503ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.2 2026-02-24T22:21:07.229 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.232+0000 7f857503ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.3 2026-02-24T22:21:07.229 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.232+0000 7f857503ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.3 2026-02-24T22:21:07.229 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.232+0000 7f857503ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 2026-02-24T22:21:07.229 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.232+0000 7f857503ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 2026-02-24T22:21:07.229 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.232+0000 7f857503ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1 2026-02-24T22:21:07.229 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.232+0000 7f857503ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1 2026-02-24T22:21:07.229 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.232+0000 7f857503ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.2 2026-02-24T22:21:07.229 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.232+0000 7f857503ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.2 2026-02-24T22:21:07.230 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f857503ba80 time 2026-02-24T22:21:07.237818+0000 2026-02-24T22:21:07.230 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:21:07.231 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:21:07.231 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f857847deb1] 2026-02-24T22:21:07.231 INFO:tasks.workunit.client.0.trial046.stderr: 2: (RGWBucketReshard::do_reshard(rgw::bucket_index_layout_generation const&, rgw::bucket_index_layout_generation const&, int, bool, bool, std::ostream*, ceph::Formatter*, FaultInjector > >&, DoutPrefixProvider const*, optional_yield)+0x274a) [0x55cd42d7ebea] 2026-02-24T22:21:07.231 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xe6b) [0x55cd42d7ff6b] 2026-02-24T22:21:07.231 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:21:07.231 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f857777dd90] 2026-02-24T22:21:07.231 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:21:07.231 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:21:07.231 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.232+0000 7f857503ba80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f857503ba80 time 2026-02-24T22:21:07.237818+0000 2026-02-24T22:21:07.232 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:21:07.232 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.232 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:21:07.232 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f857847deb1] 2026-02-24T22:21:07.232 INFO:tasks.workunit.client.0.trial046.stderr: 2: (RGWBucketReshard::do_reshard(rgw::bucket_index_layout_generation const&, rgw::bucket_index_layout_generation const&, int, bool, bool, std::ostream*, ceph::Formatter*, FaultInjector > >&, DoutPrefixProvider const*, optional_yield)+0x274a) [0x55cd42d7ebea] 2026-02-24T22:21:07.232 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xe6b) [0x55cd42d7ff6b] 2026-02-24T22:21:07.232 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:21:07.232 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f857777dd90] 2026-02-24T22:21:07.232 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:21:07.232 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:21:07.232 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.232 INFO:tasks.workunit.client.0.trial046.stderr:*** Caught signal (Aborted) ** 2026-02-24T22:21:07.233 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f857503ba80 thread_name:radosgw-admin 2026-02-24T22:21:07.255 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:21:07.255 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f8577796520] 2026-02-24T22:21:07.256 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:21:07.256 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:21:07.256 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:21:07.256 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f857847df10] 2026-02-24T22:21:07.256 INFO:tasks.workunit.client.0.trial046.stderr: 6: (RGWBucketReshard::do_reshard(rgw::bucket_index_layout_generation const&, rgw::bucket_index_layout_generation const&, int, bool, bool, std::ostream*, ceph::Formatter*, FaultInjector > >&, DoutPrefixProvider const*, optional_yield)+0x274a) [0x55cd42d7ebea] 2026-02-24T22:21:07.256 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xe6b) [0x55cd42d7ff6b] 2026-02-24T22:21:07.256 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:21:07.256 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f857777dd90] 2026-02-24T22:21:07.256 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:21:07.256 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:21:07.256 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.236+0000 7f857503ba80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:21:07.256 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f857503ba80 thread_name:radosgw-admin 2026-02-24T22:21:07.257 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.257 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:21:07.257 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f8577796520] 2026-02-24T22:21:07.257 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:21:07.257 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:21:07.257 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:21:07.257 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f857847df10] 2026-02-24T22:21:07.257 INFO:tasks.workunit.client.0.trial046.stderr: 6: (RGWBucketReshard::do_reshard(rgw::bucket_index_layout_generation const&, rgw::bucket_index_layout_generation const&, int, bool, bool, std::ostream*, ceph::Formatter*, FaultInjector > >&, DoutPrefixProvider const*, optional_yield)+0x274a) [0x55cd42d7ebea] 2026-02-24T22:21:07.257 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xe6b) [0x55cd42d7ff6b] 2026-02-24T22:21:07.257 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:21:07.257 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f857777dd90] 2026-02-24T22:21:07.257 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:21:07.257 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:21:07.258 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:21:07.258 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.258 INFO:tasks.workunit.client.0.trial046.stderr:--- begin dump of recent events --- 2026-02-24T22:21:07.258 INFO:tasks.workunit.client.0.trial046.stderr: -499> 2026-02-24T22:21:07.172+0000 7f8572b0f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:07.258 INFO:tasks.workunit.client.0.trial046.stderr: -498> 2026-02-24T22:21:07.172+0000 7f8572b0f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:07.258 INFO:tasks.workunit.client.0.trial046.stderr: -497> 2026-02-24T22:21:07.172+0000 7f8572b0f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:07.258 INFO:tasks.workunit.client.0.trial046.stderr: -496> 2026-02-24T22:21:07.172+0000 7f857230e640 10 monclient(hunting): handle_auth_done global_id 8373 payload 274 2026-02-24T22:21:07.258 INFO:tasks.workunit.client.0.trial046.stderr: -495> 2026-02-24T22:21:07.172+0000 7f857230e640 10 monclient: _finish_hunting 0 2026-02-24T22:21:07.258 INFO:tasks.workunit.client.0.trial046.stderr: -494> 2026-02-24T22:21:07.172+0000 7f857230e640 1 monclient: found mon.noname-a 2026-02-24T22:21:07.258 INFO:tasks.workunit.client.0.trial046.stderr: -493> 2026-02-24T22:21:07.172+0000 7f857230e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:21:07.258 INFO:tasks.workunit.client.0.trial046.stderr: -492> 2026-02-24T22:21:07.172+0000 7f85597fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:07.258 INFO:tasks.workunit.client.0.trial046.stderr: -491> 2026-02-24T22:21:07.172+0000 7f85597fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:21:07.258 INFO:tasks.workunit.client.0.trial046.stderr: -490> 2026-02-24T22:21:07.172+0000 7f85597fa640 10 monclient: dump: 2026-02-24T22:21:07.258 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:07.258 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:07.259 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.259 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.259 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:07.259 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.259 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:07.259 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:07.259 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:07.259 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.259 INFO:tasks.workunit.client.0.trial046.stderr: -489> 2026-02-24T22:21:07.172+0000 7f85597fa640 10 monclient: _finish_auth 0 2026-02-24T22:21:07.259 INFO:tasks.workunit.client.0.trial046.stderr: -488> 2026-02-24T22:21:07.172+0000 7f85597fa640 10 monclient: _check_auth_tickets 2026-02-24T22:21:07.259 INFO:tasks.workunit.client.0.trial046.stderr: -487> 2026-02-24T22:21:07.172+0000 7f85597fa640 10 monclient: handle_config config(0 keys) 2026-02-24T22:21:07.259 INFO:tasks.workunit.client.0.trial046.stderr: -486> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 monclient: authenticate success, global_id 8373 2026-02-24T22:21:07.259 INFO:tasks.workunit.client.0.trial046.stderr: -485> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient: get_monmap_and_config success 2026-02-24T22:21:07.259 INFO:tasks.workunit.client.0.trial046.stderr: -484> 2026-02-24T22:21:07.172+0000 7f857503ba80 4 set_mon_vals no callback set 2026-02-24T22:21:07.259 INFO:tasks.workunit.client.0.trial046.stderr: -483> 2026-02-24T22:21:07.172+0000 7f85597fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:07.259 INFO:tasks.workunit.client.0.trial046.stderr: -482> 2026-02-24T22:21:07.172+0000 7f85597fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:21:07.260 INFO:tasks.workunit.client.0.trial046.stderr: -481> 2026-02-24T22:21:07.172+0000 7f85597fa640 10 monclient: dump: 2026-02-24T22:21:07.260 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:07.260 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:07.260 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.260 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.260 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:07.260 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.260 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:07.260 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:07.260 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:07.260 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.260 INFO:tasks.workunit.client.0.trial046.stderr: -480> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient: shutdown 2026-02-24T22:21:07.260 INFO:tasks.workunit.client.0.trial046.stderr: -479> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 asok(0x55cd625edf00) unregister_commands rotate-key 2026-02-24T22:21:07.260 INFO:tasks.workunit.client.0.trial046.stderr: -478> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient: build_initial_monmap 2026-02-24T22:21:07.260 INFO:tasks.workunit.client.0.trial046.stderr: -477> 2026-02-24T22:21:07.172+0000 7f857503ba80 1 build_initial for_mkfs: 0 2026-02-24T22:21:07.260 INFO:tasks.workunit.client.0.trial046.stderr: -476> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient: monmap: 2026-02-24T22:21:07.261 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:21:07.261 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:21:07.261 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:21:07.261 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:21:07.261 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:21:07.261 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.261 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:21:07.261 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:21:07.261 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:21:07.261 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.261 INFO:tasks.workunit.client.0.trial046.stderr: -475> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding auth protocol: cephx 2026-02-24T22:21:07.261 INFO:tasks.workunit.client.0.trial046.stderr: -474> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding auth protocol: cephx 2026-02-24T22:21:07.261 INFO:tasks.workunit.client.0.trial046.stderr: -473> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding auth protocol: cephx 2026-02-24T22:21:07.261 INFO:tasks.workunit.client.0.trial046.stderr: -472> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding con mode: secure 2026-02-24T22:21:07.261 INFO:tasks.workunit.client.0.trial046.stderr: -471> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding con mode: crc 2026-02-24T22:21:07.261 INFO:tasks.workunit.client.0.trial046.stderr: -470> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding con mode: secure 2026-02-24T22:21:07.262 INFO:tasks.workunit.client.0.trial046.stderr: -469> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding con mode: crc 2026-02-24T22:21:07.262 INFO:tasks.workunit.client.0.trial046.stderr: -468> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding con mode: secure 2026-02-24T22:21:07.262 INFO:tasks.workunit.client.0.trial046.stderr: -467> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding con mode: crc 2026-02-24T22:21:07.262 INFO:tasks.workunit.client.0.trial046.stderr: -466> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding con mode: crc 2026-02-24T22:21:07.262 INFO:tasks.workunit.client.0.trial046.stderr: -465> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding con mode: secure 2026-02-24T22:21:07.262 INFO:tasks.workunit.client.0.trial046.stderr: -464> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding con mode: crc 2026-02-24T22:21:07.262 INFO:tasks.workunit.client.0.trial046.stderr: -463> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding con mode: secure 2026-02-24T22:21:07.262 INFO:tasks.workunit.client.0.trial046.stderr: -462> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding con mode: crc 2026-02-24T22:21:07.262 INFO:tasks.workunit.client.0.trial046.stderr: -461> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding con mode: secure 2026-02-24T22:21:07.262 INFO:tasks.workunit.client.0.trial046.stderr: -460> 2026-02-24T22:21:07.172+0000 7f857503ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:07.262 INFO:tasks.workunit.client.0.trial046.stderr: -459> 2026-02-24T22:21:07.172+0000 7f857503ba80 1 librados: starting msgr at 2026-02-24T22:21:07.262 INFO:tasks.workunit.client.0.trial046.stderr: -458> 2026-02-24T22:21:07.172+0000 7f857503ba80 1 librados: starting objecter 2026-02-24T22:21:07.262 INFO:tasks.workunit.client.0.trial046.stderr: -457> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command objecter_requests hook 0x55cd62721160 2026-02-24T22:21:07.262 INFO:tasks.workunit.client.0.trial046.stderr: -456> 2026-02-24T22:21:07.172+0000 7f857503ba80 1 librados: setting wanted keys 2026-02-24T22:21:07.262 INFO:tasks.workunit.client.0.trial046.stderr: -455> 2026-02-24T22:21:07.172+0000 7f857503ba80 1 librados: calling monclient init 2026-02-24T22:21:07.262 INFO:tasks.workunit.client.0.trial046.stderr: -454> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient: init 2026-02-24T22:21:07.263 INFO:tasks.workunit.client.0.trial046.stderr: -453> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding auth protocol: cephx 2026-02-24T22:21:07.263 INFO:tasks.workunit.client.0.trial046.stderr: -452> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding auth protocol: cephx 2026-02-24T22:21:07.263 INFO:tasks.workunit.client.0.trial046.stderr: -451> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding auth protocol: cephx 2026-02-24T22:21:07.263 INFO:tasks.workunit.client.0.trial046.stderr: -450> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding con mode: secure 2026-02-24T22:21:07.263 INFO:tasks.workunit.client.0.trial046.stderr: -449> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding con mode: crc 2026-02-24T22:21:07.263 INFO:tasks.workunit.client.0.trial046.stderr: -448> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding con mode: secure 2026-02-24T22:21:07.263 INFO:tasks.workunit.client.0.trial046.stderr: -447> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding con mode: crc 2026-02-24T22:21:07.263 INFO:tasks.workunit.client.0.trial046.stderr: -446> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding con mode: secure 2026-02-24T22:21:07.263 INFO:tasks.workunit.client.0.trial046.stderr: -445> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding con mode: crc 2026-02-24T22:21:07.263 INFO:tasks.workunit.client.0.trial046.stderr: -444> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding con mode: crc 2026-02-24T22:21:07.263 INFO:tasks.workunit.client.0.trial046.stderr: -443> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding con mode: secure 2026-02-24T22:21:07.263 INFO:tasks.workunit.client.0.trial046.stderr: -442> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding con mode: crc 2026-02-24T22:21:07.263 INFO:tasks.workunit.client.0.trial046.stderr: -441> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding con mode: secure 2026-02-24T22:21:07.263 INFO:tasks.workunit.client.0.trial046.stderr: -440> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding con mode: crc 2026-02-24T22:21:07.263 INFO:tasks.workunit.client.0.trial046.stderr: -439> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding con mode: secure 2026-02-24T22:21:07.264 INFO:tasks.workunit.client.0.trial046.stderr: -438> 2026-02-24T22:21:07.172+0000 7f857503ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:07.264 INFO:tasks.workunit.client.0.trial046.stderr: -437> 2026-02-24T22:21:07.172+0000 7f857503ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:07.264 INFO:tasks.workunit.client.0.trial046.stderr: -436> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command rotate-key hook 0x55cd626ef538 2026-02-24T22:21:07.264 INFO:tasks.workunit.client.0.trial046.stderr: -435> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient: _reopen_session rank -1 2026-02-24T22:21:07.264 INFO:tasks.workunit.client.0.trial046.stderr: -434> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T22:21:07.264 INFO:tasks.workunit.client.0.trial046.stderr: -433> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient(hunting): picked mon.noname-b con 0x55cd626797a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:21:07.264 INFO:tasks.workunit.client.0.trial046.stderr: -432> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient(hunting): picked mon.noname-a con 0x55cd6267c010 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:21:07.264 INFO:tasks.workunit.client.0.trial046.stderr: -431> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient(hunting): picked mon.noname-c con 0x55cd62723510 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:21:07.264 INFO:tasks.workunit.client.0.trial046.stderr: -430> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:07.264 INFO:tasks.workunit.client.0.trial046.stderr: -429> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:07.264 INFO:tasks.workunit.client.0.trial046.stderr: -428> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:07.264 INFO:tasks.workunit.client.0.trial046.stderr: -427> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient(hunting): _renew_subs 2026-02-24T22:21:07.264 INFO:tasks.workunit.client.0.trial046.stderr: -426> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient(hunting): authenticate will time out at 2037.281839s 2026-02-24T22:21:07.264 INFO:tasks.workunit.client.0.trial046.stderr: -425> 2026-02-24T22:21:07.172+0000 7f857230e640 10 monclient(hunting): get_auth_request con 0x55cd6267c010 auth_method 0 2026-02-24T22:21:07.264 INFO:tasks.workunit.client.0.trial046.stderr: -424> 2026-02-24T22:21:07.172+0000 7f857230e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:07.264 INFO:tasks.workunit.client.0.trial046.stderr: -423> 2026-02-24T22:21:07.172+0000 7f857230e640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:07.264 INFO:tasks.workunit.client.0.trial046.stderr: -422> 2026-02-24T22:21:07.172+0000 7f857230e640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:07.265 INFO:tasks.workunit.client.0.trial046.stderr: -421> 2026-02-24T22:21:07.172+0000 7f8573310640 10 monclient(hunting): get_auth_request con 0x55cd62723510 auth_method 0 2026-02-24T22:21:07.265 INFO:tasks.workunit.client.0.trial046.stderr: -420> 2026-02-24T22:21:07.172+0000 7f8573310640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:07.265 INFO:tasks.workunit.client.0.trial046.stderr: -419> 2026-02-24T22:21:07.172+0000 7f8573310640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:07.265 INFO:tasks.workunit.client.0.trial046.stderr: -418> 2026-02-24T22:21:07.172+0000 7f8573310640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:07.265 INFO:tasks.workunit.client.0.trial046.stderr: -417> 2026-02-24T22:21:07.172+0000 7f857230e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:07.265 INFO:tasks.workunit.client.0.trial046.stderr: -416> 2026-02-24T22:21:07.172+0000 7f857230e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:07.265 INFO:tasks.workunit.client.0.trial046.stderr: -415> 2026-02-24T22:21:07.172+0000 7f857230e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:07.265 INFO:tasks.workunit.client.0.trial046.stderr: -414> 2026-02-24T22:21:07.172+0000 7f8573310640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:07.265 INFO:tasks.workunit.client.0.trial046.stderr: -413> 2026-02-24T22:21:07.172+0000 7f8573310640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:07.265 INFO:tasks.workunit.client.0.trial046.stderr: -412> 2026-02-24T22:21:07.172+0000 7f8573310640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:07.265 INFO:tasks.workunit.client.0.trial046.stderr: -411> 2026-02-24T22:21:07.172+0000 7f857230e640 10 monclient(hunting): handle_auth_done global_id 8376 payload 824 2026-02-24T22:21:07.265 INFO:tasks.workunit.client.0.trial046.stderr: -410> 2026-02-24T22:21:07.172+0000 7f857230e640 10 monclient: _finish_hunting 0 2026-02-24T22:21:07.265 INFO:tasks.workunit.client.0.trial046.stderr: -409> 2026-02-24T22:21:07.172+0000 7f857230e640 1 monclient: found mon.noname-a 2026-02-24T22:21:07.265 INFO:tasks.workunit.client.0.trial046.stderr: -408> 2026-02-24T22:21:07.172+0000 7f857230e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:21:07.265 INFO:tasks.workunit.client.0.trial046.stderr: -407> 2026-02-24T22:21:07.172+0000 7f8557ff7640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:07.265 INFO:tasks.workunit.client.0.trial046.stderr: -406> 2026-02-24T22:21:07.172+0000 7f8557ff7640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:21:07.265 INFO:tasks.workunit.client.0.trial046.stderr: -405> 2026-02-24T22:21:07.172+0000 7f8557ff7640 10 monclient: dump: 2026-02-24T22:21:07.265 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:07.266 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:07.266 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.266 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.266 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:07.266 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.266 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:07.266 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:07.266 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:07.266 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.266 INFO:tasks.workunit.client.0.trial046.stderr: -404> 2026-02-24T22:21:07.172+0000 7f8557ff7640 10 monclient: _finish_auth 0 2026-02-24T22:21:07.266 INFO:tasks.workunit.client.0.trial046.stderr: -403> 2026-02-24T22:21:07.172+0000 7f8557ff7640 10 monclient: _check_auth_tickets 2026-02-24T22:21:07.266 INFO:tasks.workunit.client.0.trial046.stderr: -402> 2026-02-24T22:21:07.172+0000 7f8557ff7640 10 monclient: handle_config config(0 keys) 2026-02-24T22:21:07.266 INFO:tasks.workunit.client.0.trial046.stderr: -401> 2026-02-24T22:21:07.172+0000 7f8557ff7640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:07.266 INFO:tasks.workunit.client.0.trial046.stderr: -400> 2026-02-24T22:21:07.172+0000 7f8557ff7640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:21:07.266 INFO:tasks.workunit.client.0.trial046.stderr: -399> 2026-02-24T22:21:07.172+0000 7f8557ff7640 10 monclient: dump: 2026-02-24T22:21:07.266 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:07.266 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:07.266 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.267 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.267 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:07.267 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.267 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:07.267 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:07.267 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:07.267 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.267 INFO:tasks.workunit.client.0.trial046.stderr: -398> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 monclient: authenticate success, global_id 8376 2026-02-24T22:21:07.267 INFO:tasks.workunit.client.0.trial046.stderr: -397> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient: _renew_subs 2026-02-24T22:21:07.267 INFO:tasks.workunit.client.0.trial046.stderr: -396> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:21:07.267 INFO:tasks.workunit.client.0.trial046.stderr: -395> 2026-02-24T22:21:07.172+0000 7f85597fa640 4 set_mon_vals no callback set 2026-02-24T22:21:07.267 INFO:tasks.workunit.client.0.trial046.stderr: -394> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient: _renew_subs 2026-02-24T22:21:07.267 INFO:tasks.workunit.client.0.trial046.stderr: -393> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:21:07.267 INFO:tasks.workunit.client.0.trial046.stderr: -392> 2026-02-24T22:21:07.172+0000 7f857503ba80 1 librados: init done 2026-02-24T22:21:07.267 INFO:tasks.workunit.client.0.trial046.stderr: -391> 2026-02-24T22:21:07.176+0000 7f8557ff7640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:21:07.267 INFO:tasks.workunit.client.0.trial046.stderr: -390> 2026-02-24T22:21:07.176+0000 7f8557ff7640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:21:07.267 INFO:tasks.workunit.client.0.trial046.stderr: -389> 2026-02-24T22:21:07.176+0000 7f8557ff7640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:21:07.267 INFO:tasks.workunit.client.0.trial046.stderr: -388> 2026-02-24T22:21:07.176+0000 7f8572b0f640 10 monclient: get_auth_request con 0x7f85500707b0 auth_method 0 2026-02-24T22:21:07.267 INFO:tasks.workunit.client.0.trial046.stderr: -387> 2026-02-24T22:21:07.176+0000 7f8573310640 10 monclient: get_auth_request con 0x55cd62717b50 auth_method 0 2026-02-24T22:21:07.268 INFO:tasks.workunit.client.0.trial046.stderr: -386> 2026-02-24T22:21:07.176+0000 7f8572b0f640 10 monclient: get_auth_request con 0x55cd6277b0e0 auth_method 0 2026-02-24T22:21:07.268 INFO:tasks.workunit.client.0.trial046.stderr: -385> 2026-02-24T22:21:07.176+0000 7f857230e640 10 monclient: get_auth_request con 0x55cd6277de00 auth_method 0 2026-02-24T22:21:07.268 INFO:tasks.workunit.client.0.trial046.stderr: -384> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: build_initial_monmap 2026-02-24T22:21:07.268 INFO:tasks.workunit.client.0.trial046.stderr: -383> 2026-02-24T22:21:07.180+0000 7f857503ba80 1 build_initial for_mkfs: 0 2026-02-24T22:21:07.268 INFO:tasks.workunit.client.0.trial046.stderr: -382> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: monmap: 2026-02-24T22:21:07.268 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:21:07.268 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:21:07.268 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:21:07.268 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:21:07.268 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:21:07.268 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.268 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:21:07.268 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:21:07.268 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:21:07.268 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.268 INFO:tasks.workunit.client.0.trial046.stderr: -381> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding auth protocol: cephx 2026-02-24T22:21:07.268 INFO:tasks.workunit.client.0.trial046.stderr: -380> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding auth protocol: cephx 2026-02-24T22:21:07.268 INFO:tasks.workunit.client.0.trial046.stderr: -379> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding auth protocol: cephx 2026-02-24T22:21:07.268 INFO:tasks.workunit.client.0.trial046.stderr: -378> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding con mode: secure 2026-02-24T22:21:07.269 INFO:tasks.workunit.client.0.trial046.stderr: -377> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding con mode: crc 2026-02-24T22:21:07.269 INFO:tasks.workunit.client.0.trial046.stderr: -376> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding con mode: secure 2026-02-24T22:21:07.269 INFO:tasks.workunit.client.0.trial046.stderr: -375> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding con mode: crc 2026-02-24T22:21:07.269 INFO:tasks.workunit.client.0.trial046.stderr: -374> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding con mode: secure 2026-02-24T22:21:07.269 INFO:tasks.workunit.client.0.trial046.stderr: -373> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding con mode: crc 2026-02-24T22:21:07.269 INFO:tasks.workunit.client.0.trial046.stderr: -372> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding con mode: crc 2026-02-24T22:21:07.269 INFO:tasks.workunit.client.0.trial046.stderr: -371> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding con mode: secure 2026-02-24T22:21:07.269 INFO:tasks.workunit.client.0.trial046.stderr: -370> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding con mode: crc 2026-02-24T22:21:07.269 INFO:tasks.workunit.client.0.trial046.stderr: -369> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding con mode: secure 2026-02-24T22:21:07.269 INFO:tasks.workunit.client.0.trial046.stderr: -368> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding con mode: crc 2026-02-24T22:21:07.269 INFO:tasks.workunit.client.0.trial046.stderr: -367> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding con mode: secure 2026-02-24T22:21:07.269 INFO:tasks.workunit.client.0.trial046.stderr: -366> 2026-02-24T22:21:07.180+0000 7f857503ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:07.269 INFO:tasks.workunit.client.0.trial046.stderr: -365> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command objecter_requests cmddesc objecter_requests hook 0x55cd627842c0 EEXIST 2026-02-24T22:21:07.269 INFO:tasks.workunit.client.0.trial046.stderr: -364> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: init 2026-02-24T22:21:07.269 INFO:tasks.workunit.client.0.trial046.stderr: -363> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding auth protocol: cephx 2026-02-24T22:21:07.269 INFO:tasks.workunit.client.0.trial046.stderr: -362> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding auth protocol: cephx 2026-02-24T22:21:07.269 INFO:tasks.workunit.client.0.trial046.stderr: -361> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding auth protocol: cephx 2026-02-24T22:21:07.269 INFO:tasks.workunit.client.0.trial046.stderr: -360> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding con mode: secure 2026-02-24T22:21:07.269 INFO:tasks.workunit.client.0.trial046.stderr: -359> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding con mode: crc 2026-02-24T22:21:07.270 INFO:tasks.workunit.client.0.trial046.stderr: -358> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding con mode: secure 2026-02-24T22:21:07.270 INFO:tasks.workunit.client.0.trial046.stderr: -357> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding con mode: crc 2026-02-24T22:21:07.270 INFO:tasks.workunit.client.0.trial046.stderr: -356> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding con mode: secure 2026-02-24T22:21:07.270 INFO:tasks.workunit.client.0.trial046.stderr: -355> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding con mode: crc 2026-02-24T22:21:07.270 INFO:tasks.workunit.client.0.trial046.stderr: -354> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding con mode: crc 2026-02-24T22:21:07.270 INFO:tasks.workunit.client.0.trial046.stderr: -353> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding con mode: secure 2026-02-24T22:21:07.270 INFO:tasks.workunit.client.0.trial046.stderr: -352> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding con mode: crc 2026-02-24T22:21:07.270 INFO:tasks.workunit.client.0.trial046.stderr: -351> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding con mode: secure 2026-02-24T22:21:07.270 INFO:tasks.workunit.client.0.trial046.stderr: -350> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding con mode: crc 2026-02-24T22:21:07.270 INFO:tasks.workunit.client.0.trial046.stderr: -349> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding con mode: secure 2026-02-24T22:21:07.270 INFO:tasks.workunit.client.0.trial046.stderr: -348> 2026-02-24T22:21:07.180+0000 7f857503ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:07.270 INFO:tasks.workunit.client.0.trial046.stderr: -347> 2026-02-24T22:21:07.180+0000 7f857503ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:07.270 INFO:tasks.workunit.client.0.trial046.stderr: -346> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command rotate-key cmddesc rotate-key hook 0x55cd6277c020 EEXIST 2026-02-24T22:21:07.270 INFO:tasks.workunit.client.0.trial046.stderr: -345> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: _reopen_session rank -1 2026-02-24T22:21:07.270 INFO:tasks.workunit.client.0.trial046.stderr: -344> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:21:07.270 INFO:tasks.workunit.client.0.trial046.stderr: -343> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): picked mon.noname-c con 0x55cd628d7e20 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:21:07.270 INFO:tasks.workunit.client.0.trial046.stderr: -342> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): picked mon.noname-a con 0x55cd628d5b50 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:21:07.270 INFO:tasks.workunit.client.0.trial046.stderr: -341> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): picked mon.noname-b con 0x55cd628d5f60 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:21:07.270 INFO:tasks.workunit.client.0.trial046.stderr: -340> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:07.270 INFO:tasks.workunit.client.0.trial046.stderr: -339> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:07.270 INFO:tasks.workunit.client.0.trial046.stderr: -338> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:07.271 INFO:tasks.workunit.client.0.trial046.stderr: -337> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): _renew_subs 2026-02-24T22:21:07.271 INFO:tasks.workunit.client.0.trial046.stderr: -336> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): authenticate will time out at 2037.288993s 2026-02-24T22:21:07.271 INFO:tasks.workunit.client.0.trial046.stderr: -335> 2026-02-24T22:21:07.180+0000 7f8572b0f640 10 monclient(hunting): get_auth_request con 0x55cd628d7e20 auth_method 0 2026-02-24T22:21:07.271 INFO:tasks.workunit.client.0.trial046.stderr: -334> 2026-02-24T22:21:07.180+0000 7f8572b0f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:07.271 INFO:tasks.workunit.client.0.trial046.stderr: -333> 2026-02-24T22:21:07.180+0000 7f8572b0f640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:07.271 INFO:tasks.workunit.client.0.trial046.stderr: -332> 2026-02-24T22:21:07.180+0000 7f8572b0f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:07.271 INFO:tasks.workunit.client.0.trial046.stderr: -331> 2026-02-24T22:21:07.180+0000 7f857230e640 10 monclient(hunting): get_auth_request con 0x55cd628d5b50 auth_method 0 2026-02-24T22:21:07.271 INFO:tasks.workunit.client.0.trial046.stderr: -330> 2026-02-24T22:21:07.180+0000 7f857230e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:07.271 INFO:tasks.workunit.client.0.trial046.stderr: -329> 2026-02-24T22:21:07.180+0000 7f857230e640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:07.271 INFO:tasks.workunit.client.0.trial046.stderr: -328> 2026-02-24T22:21:07.180+0000 7f857230e640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:07.271 INFO:tasks.workunit.client.0.trial046.stderr: -327> 2026-02-24T22:21:07.180+0000 7f857230e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:07.271 INFO:tasks.workunit.client.0.trial046.stderr: -326> 2026-02-24T22:21:07.180+0000 7f857230e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:07.271 INFO:tasks.workunit.client.0.trial046.stderr: -325> 2026-02-24T22:21:07.180+0000 7f857230e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:07.271 INFO:tasks.workunit.client.0.trial046.stderr: -324> 2026-02-24T22:21:07.180+0000 7f8572b0f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:07.271 INFO:tasks.workunit.client.0.trial046.stderr: -323> 2026-02-24T22:21:07.180+0000 7f8572b0f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:07.271 INFO:tasks.workunit.client.0.trial046.stderr: -322> 2026-02-24T22:21:07.180+0000 7f8572b0f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:07.271 INFO:tasks.workunit.client.0.trial046.stderr: -321> 2026-02-24T22:21:07.180+0000 7f8573310640 10 monclient(hunting): get_auth_request con 0x55cd628d5f60 auth_method 0 2026-02-24T22:21:07.271 INFO:tasks.workunit.client.0.trial046.stderr: -320> 2026-02-24T22:21:07.180+0000 7f8573310640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:07.271 INFO:tasks.workunit.client.0.trial046.stderr: -319> 2026-02-24T22:21:07.180+0000 7f8573310640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:07.271 INFO:tasks.workunit.client.0.trial046.stderr: -318> 2026-02-24T22:21:07.180+0000 7f8573310640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:07.271 INFO:tasks.workunit.client.0.trial046.stderr: -317> 2026-02-24T22:21:07.180+0000 7f857230e640 10 monclient(hunting): handle_auth_done global_id 8379 payload 824 2026-02-24T22:21:07.271 INFO:tasks.workunit.client.0.trial046.stderr: -316> 2026-02-24T22:21:07.180+0000 7f857230e640 10 monclient: _finish_hunting 0 2026-02-24T22:21:07.272 INFO:tasks.workunit.client.0.trial046.stderr: -315> 2026-02-24T22:21:07.180+0000 7f857230e640 1 monclient: found mon.noname-a 2026-02-24T22:21:07.272 INFO:tasks.workunit.client.0.trial046.stderr: -314> 2026-02-24T22:21:07.180+0000 7f857230e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:21:07.272 INFO:tasks.workunit.client.0.trial046.stderr: -313> 2026-02-24T22:21:07.180+0000 7f85557f2640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:07.272 INFO:tasks.workunit.client.0.trial046.stderr: -312> 2026-02-24T22:21:07.180+0000 7f85557f2640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:21:07.272 INFO:tasks.workunit.client.0.trial046.stderr: -311> 2026-02-24T22:21:07.180+0000 7f85557f2640 10 monclient: dump: 2026-02-24T22:21:07.272 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:07.272 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:07.272 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.272 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.272 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:07.272 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.272 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:07.272 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:07.272 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:07.272 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.272 INFO:tasks.workunit.client.0.trial046.stderr: -310> 2026-02-24T22:21:07.180+0000 7f85557f2640 10 monclient: _finish_auth 0 2026-02-24T22:21:07.272 INFO:tasks.workunit.client.0.trial046.stderr: -309> 2026-02-24T22:21:07.180+0000 7f85557f2640 10 monclient: _check_auth_tickets 2026-02-24T22:21:07.272 INFO:tasks.workunit.client.0.trial046.stderr: -308> 2026-02-24T22:21:07.180+0000 7f85557f2640 10 monclient: handle_config config(0 keys) 2026-02-24T22:21:07.272 INFO:tasks.workunit.client.0.trial046.stderr: -307> 2026-02-24T22:21:07.180+0000 7f85557f2640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:07.272 INFO:tasks.workunit.client.0.trial046.stderr: -306> 2026-02-24T22:21:07.180+0000 7f85557f2640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:21:07.272 INFO:tasks.workunit.client.0.trial046.stderr: -305> 2026-02-24T22:21:07.180+0000 7f85557f2640 10 monclient: dump: 2026-02-24T22:21:07.273 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:07.273 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:07.273 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.273 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.273 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:07.273 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.273 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:07.273 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:07.273 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:07.273 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.273 INFO:tasks.workunit.client.0.trial046.stderr: -304> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 monclient: authenticate success, global_id 8379 2026-02-24T22:21:07.273 INFO:tasks.workunit.client.0.trial046.stderr: -303> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: _renew_subs 2026-02-24T22:21:07.273 INFO:tasks.workunit.client.0.trial046.stderr: -302> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:21:07.273 INFO:tasks.workunit.client.0.trial046.stderr: -301> 2026-02-24T22:21:07.180+0000 7f8573b11640 4 set_mon_vals no callback set 2026-02-24T22:21:07.273 INFO:tasks.workunit.client.0.trial046.stderr: -300> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: _renew_subs 2026-02-24T22:21:07.273 INFO:tasks.workunit.client.0.trial046.stderr: -299> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:21:07.273 INFO:tasks.workunit.client.0.trial046.stderr: -298> 2026-02-24T22:21:07.180+0000 7f85557f2640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:21:07.273 INFO:tasks.workunit.client.0.trial046.stderr: -297> 2026-02-24T22:21:07.180+0000 7f85557f2640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:21:07.273 INFO:tasks.workunit.client.0.trial046.stderr: -296> 2026-02-24T22:21:07.180+0000 7f85557f2640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:21:07.273 INFO:tasks.workunit.client.0.trial046.stderr: -295> 2026-02-24T22:21:07.180+0000 7f8572b0f640 10 monclient: get_auth_request con 0x7f85440706c0 auth_method 0 2026-02-24T22:21:07.273 INFO:tasks.workunit.client.0.trial046.stderr: -294> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: get_monmap_and_config 2026-02-24T22:21:07.273 INFO:tasks.workunit.client.0.trial046.stderr: -293> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: build_initial_monmap 2026-02-24T22:21:07.274 INFO:tasks.workunit.client.0.trial046.stderr: -292> 2026-02-24T22:21:07.180+0000 7f857503ba80 1 build_initial for_mkfs: 0 2026-02-24T22:21:07.274 INFO:tasks.workunit.client.0.trial046.stderr: -291> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: monmap: 2026-02-24T22:21:07.274 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:21:07.274 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:21:07.274 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:21:07.274 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:21:07.274 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:21:07.274 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.274 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:21:07.274 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:21:07.274 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:21:07.274 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.274 INFO:tasks.workunit.client.0.trial046.stderr: -290> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding auth protocol: cephx 2026-02-24T22:21:07.274 INFO:tasks.workunit.client.0.trial046.stderr: -289> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding auth protocol: cephx 2026-02-24T22:21:07.274 INFO:tasks.workunit.client.0.trial046.stderr: -288> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding auth protocol: cephx 2026-02-24T22:21:07.274 INFO:tasks.workunit.client.0.trial046.stderr: -287> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding con mode: secure 2026-02-24T22:21:07.274 INFO:tasks.workunit.client.0.trial046.stderr: -286> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding con mode: crc 2026-02-24T22:21:07.274 INFO:tasks.workunit.client.0.trial046.stderr: -285> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding con mode: secure 2026-02-24T22:21:07.274 INFO:tasks.workunit.client.0.trial046.stderr: -284> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding con mode: crc 2026-02-24T22:21:07.274 INFO:tasks.workunit.client.0.trial046.stderr: -283> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding con mode: secure 2026-02-24T22:21:07.274 INFO:tasks.workunit.client.0.trial046.stderr: -282> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding con mode: crc 2026-02-24T22:21:07.274 INFO:tasks.workunit.client.0.trial046.stderr: -281> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding con mode: crc 2026-02-24T22:21:07.274 INFO:tasks.workunit.client.0.trial046.stderr: -280> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding con mode: secure 2026-02-24T22:21:07.275 INFO:tasks.workunit.client.0.trial046.stderr: -279> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding con mode: crc 2026-02-24T22:21:07.275 INFO:tasks.workunit.client.0.trial046.stderr: -278> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding con mode: secure 2026-02-24T22:21:07.275 INFO:tasks.workunit.client.0.trial046.stderr: -277> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding con mode: crc 2026-02-24T22:21:07.275 INFO:tasks.workunit.client.0.trial046.stderr: -276> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding con mode: secure 2026-02-24T22:21:07.275 INFO:tasks.workunit.client.0.trial046.stderr: -275> 2026-02-24T22:21:07.180+0000 7f857503ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:07.275 INFO:tasks.workunit.client.0.trial046.stderr: -274> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: init 2026-02-24T22:21:07.275 INFO:tasks.workunit.client.0.trial046.stderr: -273> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding auth protocol: cephx 2026-02-24T22:21:07.275 INFO:tasks.workunit.client.0.trial046.stderr: -272> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding auth protocol: cephx 2026-02-24T22:21:07.275 INFO:tasks.workunit.client.0.trial046.stderr: -271> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding auth protocol: cephx 2026-02-24T22:21:07.275 INFO:tasks.workunit.client.0.trial046.stderr: -270> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding con mode: secure 2026-02-24T22:21:07.275 INFO:tasks.workunit.client.0.trial046.stderr: -269> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding con mode: crc 2026-02-24T22:21:07.275 INFO:tasks.workunit.client.0.trial046.stderr: -268> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding con mode: secure 2026-02-24T22:21:07.275 INFO:tasks.workunit.client.0.trial046.stderr: -267> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding con mode: crc 2026-02-24T22:21:07.275 INFO:tasks.workunit.client.0.trial046.stderr: -266> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding con mode: secure 2026-02-24T22:21:07.275 INFO:tasks.workunit.client.0.trial046.stderr: -265> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding con mode: crc 2026-02-24T22:21:07.275 INFO:tasks.workunit.client.0.trial046.stderr: -264> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding con mode: crc 2026-02-24T22:21:07.275 INFO:tasks.workunit.client.0.trial046.stderr: -263> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding con mode: secure 2026-02-24T22:21:07.275 INFO:tasks.workunit.client.0.trial046.stderr: -262> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding con mode: crc 2026-02-24T22:21:07.275 INFO:tasks.workunit.client.0.trial046.stderr: -261> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding con mode: secure 2026-02-24T22:21:07.275 INFO:tasks.workunit.client.0.trial046.stderr: -260> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding con mode: crc 2026-02-24T22:21:07.275 INFO:tasks.workunit.client.0.trial046.stderr: -259> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding con mode: secure 2026-02-24T22:21:07.275 INFO:tasks.workunit.client.0.trial046.stderr: -258> 2026-02-24T22:21:07.180+0000 7f857503ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:07.275 INFO:tasks.workunit.client.0.trial046.stderr: -257> 2026-02-24T22:21:07.180+0000 7f857503ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:07.276 INFO:tasks.workunit.client.0.trial046.stderr: -256> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command rotate-key cmddesc rotate-key hook 0x7ffc7a41aa88 EEXIST 2026-02-24T22:21:07.276 INFO:tasks.workunit.client.0.trial046.stderr: -255> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: _reopen_session rank -1 2026-02-24T22:21:07.276 INFO:tasks.workunit.client.0.trial046.stderr: -254> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T22:21:07.276 INFO:tasks.workunit.client.0.trial046.stderr: -253> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): picked mon.noname-b con 0x55cd62990070 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:21:07.276 INFO:tasks.workunit.client.0.trial046.stderr: -252> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): picked mon.noname-c con 0x55cd629937e0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:21:07.276 INFO:tasks.workunit.client.0.trial046.stderr: -251> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): picked mon.noname-a con 0x55cd629c05c0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:21:07.276 INFO:tasks.workunit.client.0.trial046.stderr: -250> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:07.276 INFO:tasks.workunit.client.0.trial046.stderr: -249> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:07.276 INFO:tasks.workunit.client.0.trial046.stderr: -248> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:07.276 INFO:tasks.workunit.client.0.trial046.stderr: -247> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): _renew_subs 2026-02-24T22:21:07.276 INFO:tasks.workunit.client.0.trial046.stderr: -246> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): authenticate will time out at 2037.291965s 2026-02-24T22:21:07.276 INFO:tasks.workunit.client.0.trial046.stderr: -245> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): get_auth_request con 0x55cd629c05c0 auth_method 0 2026-02-24T22:21:07.276 INFO:tasks.workunit.client.0.trial046.stderr: -244> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:07.276 INFO:tasks.workunit.client.0.trial046.stderr: -243> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:07.276 INFO:tasks.workunit.client.0.trial046.stderr: -242> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:07.276 INFO:tasks.workunit.client.0.trial046.stderr: -241> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): get_auth_request con 0x55cd629937e0 auth_method 0 2026-02-24T22:21:07.276 INFO:tasks.workunit.client.0.trial046.stderr: -240> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:07.276 INFO:tasks.workunit.client.0.trial046.stderr: -239> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:07.276 INFO:tasks.workunit.client.0.trial046.stderr: -238> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:07.276 INFO:tasks.workunit.client.0.trial046.stderr: -237> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:07.276 INFO:tasks.workunit.client.0.trial046.stderr: -236> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:07.276 INFO:tasks.workunit.client.0.trial046.stderr: -235> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:07.277 INFO:tasks.workunit.client.0.trial046.stderr: -234> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:07.277 INFO:tasks.workunit.client.0.trial046.stderr: -233> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:07.277 INFO:tasks.workunit.client.0.trial046.stderr: -232> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:07.277 INFO:tasks.workunit.client.0.trial046.stderr: -231> 2026-02-24T22:21:07.184+0000 7f8572b0f640 10 monclient(hunting): get_auth_request con 0x55cd62990070 auth_method 0 2026-02-24T22:21:07.277 INFO:tasks.workunit.client.0.trial046.stderr: -230> 2026-02-24T22:21:07.184+0000 7f8572b0f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:07.277 INFO:tasks.workunit.client.0.trial046.stderr: -229> 2026-02-24T22:21:07.184+0000 7f8572b0f640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:07.277 INFO:tasks.workunit.client.0.trial046.stderr: -228> 2026-02-24T22:21:07.184+0000 7f8572b0f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:07.277 INFO:tasks.workunit.client.0.trial046.stderr: -227> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): handle_auth_done global_id 8382 payload 274 2026-02-24T22:21:07.277 INFO:tasks.workunit.client.0.trial046.stderr: -226> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient: _finish_hunting 0 2026-02-24T22:21:07.277 INFO:tasks.workunit.client.0.trial046.stderr: -225> 2026-02-24T22:21:07.184+0000 7f8573310640 1 monclient: found mon.noname-a 2026-02-24T22:21:07.277 INFO:tasks.workunit.client.0.trial046.stderr: -224> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:21:07.277 INFO:tasks.workunit.client.0.trial046.stderr: -223> 2026-02-24T22:21:07.184+0000 7f854effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:07.277 INFO:tasks.workunit.client.0.trial046.stderr: -222> 2026-02-24T22:21:07.184+0000 7f854effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:21:07.277 INFO:tasks.workunit.client.0.trial046.stderr: -221> 2026-02-24T22:21:07.184+0000 7f854effd640 10 monclient: dump: 2026-02-24T22:21:07.277 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:07.277 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:07.277 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.277 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.277 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:07.277 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.277 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:07.277 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:07.277 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:07.277 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.278 INFO:tasks.workunit.client.0.trial046.stderr: -220> 2026-02-24T22:21:07.184+0000 7f854effd640 10 monclient: _finish_auth 0 2026-02-24T22:21:07.278 INFO:tasks.workunit.client.0.trial046.stderr: -219> 2026-02-24T22:21:07.184+0000 7f854effd640 10 monclient: _check_auth_tickets 2026-02-24T22:21:07.278 INFO:tasks.workunit.client.0.trial046.stderr: -218> 2026-02-24T22:21:07.184+0000 7f854effd640 10 monclient: handle_config config(0 keys) 2026-02-24T22:21:07.278 INFO:tasks.workunit.client.0.trial046.stderr: -217> 2026-02-24T22:21:07.184+0000 7f854effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:07.278 INFO:tasks.workunit.client.0.trial046.stderr: -216> 2026-02-24T22:21:07.184+0000 7f854effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:21:07.278 INFO:tasks.workunit.client.0.trial046.stderr: -215> 2026-02-24T22:21:07.184+0000 7f854effd640 10 monclient: dump: 2026-02-24T22:21:07.278 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:07.278 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:07.278 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.278 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.278 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:07.278 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.278 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:07.278 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:07.278 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:07.278 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.278 INFO:tasks.workunit.client.0.trial046.stderr: -214> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 monclient: authenticate success, global_id 8382 2026-02-24T22:21:07.278 INFO:tasks.workunit.client.0.trial046.stderr: -213> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient: get_monmap_and_config success 2026-02-24T22:21:07.278 INFO:tasks.workunit.client.0.trial046.stderr: -212> 2026-02-24T22:21:07.184+0000 7f857503ba80 4 set_mon_vals no callback set 2026-02-24T22:21:07.278 INFO:tasks.workunit.client.0.trial046.stderr: -211> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient: shutdown 2026-02-24T22:21:07.278 INFO:tasks.workunit.client.0.trial046.stderr: -210> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient: build_initial_monmap 2026-02-24T22:21:07.278 INFO:tasks.workunit.client.0.trial046.stderr: -209> 2026-02-24T22:21:07.184+0000 7f857503ba80 1 build_initial for_mkfs: 0 2026-02-24T22:21:07.278 INFO:tasks.workunit.client.0.trial046.stderr: -208> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient: monmap: 2026-02-24T22:21:07.278 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:21:07.279 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:21:07.279 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:21:07.279 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:21:07.279 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:21:07.279 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.279 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:21:07.279 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:21:07.279 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:21:07.279 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.279 INFO:tasks.workunit.client.0.trial046.stderr: -207> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding auth protocol: cephx 2026-02-24T22:21:07.279 INFO:tasks.workunit.client.0.trial046.stderr: -206> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding auth protocol: cephx 2026-02-24T22:21:07.279 INFO:tasks.workunit.client.0.trial046.stderr: -205> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding auth protocol: cephx 2026-02-24T22:21:07.279 INFO:tasks.workunit.client.0.trial046.stderr: -204> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding con mode: secure 2026-02-24T22:21:07.279 INFO:tasks.workunit.client.0.trial046.stderr: -203> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding con mode: crc 2026-02-24T22:21:07.279 INFO:tasks.workunit.client.0.trial046.stderr: -202> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding con mode: secure 2026-02-24T22:21:07.279 INFO:tasks.workunit.client.0.trial046.stderr: -201> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding con mode: crc 2026-02-24T22:21:07.279 INFO:tasks.workunit.client.0.trial046.stderr: -200> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding con mode: secure 2026-02-24T22:21:07.279 INFO:tasks.workunit.client.0.trial046.stderr: -199> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding con mode: crc 2026-02-24T22:21:07.279 INFO:tasks.workunit.client.0.trial046.stderr: -198> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding con mode: crc 2026-02-24T22:21:07.279 INFO:tasks.workunit.client.0.trial046.stderr: -197> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding con mode: secure 2026-02-24T22:21:07.279 INFO:tasks.workunit.client.0.trial046.stderr: -196> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding con mode: crc 2026-02-24T22:21:07.279 INFO:tasks.workunit.client.0.trial046.stderr: -195> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding con mode: secure 2026-02-24T22:21:07.279 INFO:tasks.workunit.client.0.trial046.stderr: -194> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding con mode: crc 2026-02-24T22:21:07.279 INFO:tasks.workunit.client.0.trial046.stderr: -193> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding con mode: secure 2026-02-24T22:21:07.280 INFO:tasks.workunit.client.0.trial046.stderr: -192> 2026-02-24T22:21:07.184+0000 7f857503ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:07.280 INFO:tasks.workunit.client.0.trial046.stderr: -191> 2026-02-24T22:21:07.184+0000 7f857503ba80 1 librados: starting msgr at 2026-02-24T22:21:07.280 INFO:tasks.workunit.client.0.trial046.stderr: -190> 2026-02-24T22:21:07.184+0000 7f857503ba80 1 librados: starting objecter 2026-02-24T22:21:07.280 INFO:tasks.workunit.client.0.trial046.stderr: -189> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command objecter_requests cmddesc objecter_requests hook 0x55cd6296e9b0 EEXIST 2026-02-24T22:21:07.280 INFO:tasks.workunit.client.0.trial046.stderr: -188> 2026-02-24T22:21:07.184+0000 7f857503ba80 1 librados: setting wanted keys 2026-02-24T22:21:07.280 INFO:tasks.workunit.client.0.trial046.stderr: -187> 2026-02-24T22:21:07.184+0000 7f857503ba80 1 librados: calling monclient init 2026-02-24T22:21:07.280 INFO:tasks.workunit.client.0.trial046.stderr: -186> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient: init 2026-02-24T22:21:07.280 INFO:tasks.workunit.client.0.trial046.stderr: -185> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding auth protocol: cephx 2026-02-24T22:21:07.280 INFO:tasks.workunit.client.0.trial046.stderr: -184> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding auth protocol: cephx 2026-02-24T22:21:07.280 INFO:tasks.workunit.client.0.trial046.stderr: -183> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding auth protocol: cephx 2026-02-24T22:21:07.280 INFO:tasks.workunit.client.0.trial046.stderr: -182> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding con mode: secure 2026-02-24T22:21:07.280 INFO:tasks.workunit.client.0.trial046.stderr: -181> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding con mode: crc 2026-02-24T22:21:07.280 INFO:tasks.workunit.client.0.trial046.stderr: -180> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding con mode: secure 2026-02-24T22:21:07.280 INFO:tasks.workunit.client.0.trial046.stderr: -179> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding con mode: crc 2026-02-24T22:21:07.280 INFO:tasks.workunit.client.0.trial046.stderr: -178> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding con mode: secure 2026-02-24T22:21:07.280 INFO:tasks.workunit.client.0.trial046.stderr: -177> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding con mode: crc 2026-02-24T22:21:07.280 INFO:tasks.workunit.client.0.trial046.stderr: -176> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding con mode: crc 2026-02-24T22:21:07.280 INFO:tasks.workunit.client.0.trial046.stderr: -175> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding con mode: secure 2026-02-24T22:21:07.280 INFO:tasks.workunit.client.0.trial046.stderr: -174> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding con mode: crc 2026-02-24T22:21:07.280 INFO:tasks.workunit.client.0.trial046.stderr: -173> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding con mode: secure 2026-02-24T22:21:07.280 INFO:tasks.workunit.client.0.trial046.stderr: -172> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding con mode: crc 2026-02-24T22:21:07.280 INFO:tasks.workunit.client.0.trial046.stderr: -171> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding con mode: secure 2026-02-24T22:21:07.280 INFO:tasks.workunit.client.0.trial046.stderr: -170> 2026-02-24T22:21:07.184+0000 7f857503ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:07.280 INFO:tasks.workunit.client.0.trial046.stderr: -169> 2026-02-24T22:21:07.184+0000 7f857503ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:07.280 INFO:tasks.workunit.client.0.trial046.stderr: -168> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command rotate-key cmddesc rotate-key hook 0x55cd62968198 EEXIST 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -167> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient: _reopen_session rank -1 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -166> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -165> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient(hunting): picked mon.noname-b con 0x55cd629937e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -164> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient(hunting): picked mon.noname-a con 0x55cd629c05c0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -163> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient(hunting): picked mon.noname-c con 0x55cd629c5940 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -162> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -161> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -160> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -159> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient(hunting): _renew_subs 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -158> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient(hunting): authenticate will time out at 2037.293308s 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -157> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): get_auth_request con 0x55cd629c5940 auth_method 0 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -156> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -155> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -154> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -153> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): get_auth_request con 0x55cd629c05c0 auth_method 0 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -152> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -151> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -150> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -149> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -148> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -147> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -146> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -145> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -144> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -143> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): handle_auth_done global_id 7826 payload 824 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -142> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient: _finish_hunting 0 2026-02-24T22:21:07.281 INFO:tasks.workunit.client.0.trial046.stderr: -141> 2026-02-24T22:21:07.184+0000 7f8573310640 1 monclient: found mon.noname-c 2026-02-24T22:21:07.282 INFO:tasks.workunit.client.0.trial046.stderr: -140> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:21:07.282 INFO:tasks.workunit.client.0.trial046.stderr: -139> 2026-02-24T22:21:07.184+0000 7f8572b0f640 10 monclient: get_auth_request con 0x55cd629937e0 auth_method 0 2026-02-24T22:21:07.282 INFO:tasks.workunit.client.0.trial046.stderr: -138> 2026-02-24T22:21:07.184+0000 7f854d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:07.282 INFO:tasks.workunit.client.0.trial046.stderr: -137> 2026-02-24T22:21:07.184+0000 7f854d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:21:07.282 INFO:tasks.workunit.client.0.trial046.stderr: -136> 2026-02-24T22:21:07.184+0000 7f854d7fa640 10 monclient: dump: 2026-02-24T22:21:07.282 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:07.282 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:07.282 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.282 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.282 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:07.282 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.282 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:07.282 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:07.282 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:07.282 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.282 INFO:tasks.workunit.client.0.trial046.stderr: -135> 2026-02-24T22:21:07.184+0000 7f854d7fa640 10 monclient: _finish_auth 0 2026-02-24T22:21:07.282 INFO:tasks.workunit.client.0.trial046.stderr: -134> 2026-02-24T22:21:07.184+0000 7f854d7fa640 10 monclient: _check_auth_tickets 2026-02-24T22:21:07.282 INFO:tasks.workunit.client.0.trial046.stderr: -133> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 monclient: authenticate success, global_id 7826 2026-02-24T22:21:07.282 INFO:tasks.workunit.client.0.trial046.stderr: -132> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient: _renew_subs 2026-02-24T22:21:07.282 INFO:tasks.workunit.client.0.trial046.stderr: -131> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:21:07.282 INFO:tasks.workunit.client.0.trial046.stderr: -130> 2026-02-24T22:21:07.184+0000 7f854d7fa640 10 monclient: handle_config config(0 keys) 2026-02-24T22:21:07.282 INFO:tasks.workunit.client.0.trial046.stderr: -129> 2026-02-24T22:21:07.184+0000 7f854d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:07.282 INFO:tasks.workunit.client.0.trial046.stderr: -128> 2026-02-24T22:21:07.184+0000 7f854d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:21:07.282 INFO:tasks.workunit.client.0.trial046.stderr: -127> 2026-02-24T22:21:07.184+0000 7f854d7fa640 10 monclient: dump: 2026-02-24T22:21:07.282 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:07.282 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr: -126> 2026-02-24T22:21:07.184+0000 7f854effd640 4 set_mon_vals no callback set 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr: -125> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient: _renew_subs 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr: -124> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr: -123> 2026-02-24T22:21:07.184+0000 7f857503ba80 1 librados: init done 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr: -122> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command cr dump hook 0x55cd629bab38 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr: -121> 2026-02-24T22:21:07.184+0000 7f857503ba80 4 Realm: () 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr: -120> 2026-02-24T22:21:07.184+0000 7f857503ba80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr: -119> 2026-02-24T22:21:07.184+0000 7f857503ba80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr: -118> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr: -117> 2026-02-24T22:21:07.184+0000 7f857503ba80 20 zonegroup default 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr: -116> 2026-02-24T22:21:07.184+0000 7f8572b0f640 10 monclient: get_auth_request con 0x55cd62a31100 auth_method 0 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr: -115> 2026-02-24T22:21:07.184+0000 7f857503ba80 20 rados->read ofs=0 len=0 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr: -114> 2026-02-24T22:21:07.184+0000 7f854d7fa640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr: -113> 2026-02-24T22:21:07.184+0000 7f854d7fa640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr: -112> 2026-02-24T22:21:07.184+0000 7f854d7fa640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr: -111> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient: get_auth_request con 0x7f8534070730 auth_method 0 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr: -110> 2026-02-24T22:21:07.188+0000 7f8572b0f640 10 monclient: get_auth_request con 0x55cd62a54420 auth_method 0 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr: -109> 2026-02-24T22:21:07.188+0000 7f857503ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr: -108> 2026-02-24T22:21:07.188+0000 7f857503ba80 20 rados->read ofs=0 len=0 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr: -107> 2026-02-24T22:21:07.188+0000 7f857230e640 10 monclient: get_auth_request con 0x55cd62a75b40 auth_method 0 2026-02-24T22:21:07.283 INFO:tasks.workunit.client.0.trial046.stderr: -106> 2026-02-24T22:21:07.188+0000 7f857503ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -105> 2026-02-24T22:21:07.188+0000 7f857503ba80 20 started sync module instance, tier type = 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -104> 2026-02-24T22:21:07.188+0000 7f857503ba80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -103> 2026-02-24T22:21:07.188+0000 7f8573310640 10 monclient: get_auth_request con 0x55cd62ad6fc0 auth_method 0 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -102> 2026-02-24T22:21:07.188+0000 7f8573310640 10 monclient: get_auth_request con 0x55cd62bdea30 auth_method 0 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -101> 2026-02-24T22:21:07.188+0000 7f8572b0f640 10 monclient: get_auth_request con 0x55cd62b39960 auth_method 0 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -100> 2026-02-24T22:21:07.188+0000 7f857230e640 10 monclient: get_auth_request con 0x55cd62b9c8f0 auth_method 0 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -99> 2026-02-24T22:21:07.188+0000 7f8572b0f640 10 monclient: get_auth_request con 0x55cd62c41a30 auth_method 0 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -98> 2026-02-24T22:21:07.188+0000 7f8572b0f640 0 -- 10.20.193.46:0/3692073238 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55cd62a54420 msgr2=0x55cd62a748d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -97> 2026-02-24T22:21:07.188+0000 7f854d7fa640 1 client.7826.objecter ms_handle_reset 0x55cd62a54420 session 0x55cd62a30630 osd.4 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -96> 2026-02-24T22:21:07.188+0000 7f854d7fa640 10 monclient: _renew_subs 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -95> 2026-02-24T22:21:07.188+0000 7f854d7fa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -94> 2026-02-24T22:21:07.188+0000 7f8572b0f640 10 monclient: get_auth_request con 0x7f853409d460 auth_method 0 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -93> 2026-02-24T22:21:07.188+0000 7f857503ba80 20 add_watcher() i=1 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -92> 2026-02-24T22:21:07.188+0000 7f857503ba80 20 add_watcher() i=3 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -91> 2026-02-24T22:21:07.188+0000 7f857503ba80 20 add_watcher() i=0 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -90> 2026-02-24T22:21:07.188+0000 7f857503ba80 20 add_watcher() i=5 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -89> 2026-02-24T22:21:07.188+0000 7f857503ba80 20 add_watcher() i=2 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -88> 2026-02-24T22:21:07.188+0000 7f857503ba80 20 add_watcher() i=4 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -87> 2026-02-24T22:21:07.192+0000 7f857503ba80 20 add_watcher() i=6 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -86> 2026-02-24T22:21:07.192+0000 7f857503ba80 20 add_watcher() i=7 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -85> 2026-02-24T22:21:07.192+0000 7f857503ba80 2 all 8 watchers are set, enabling cache 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -84> 2026-02-24T22:21:07.192+0000 7f857230e640 10 monclient: get_auth_request con 0x7f8494008ac0 auth_method 0 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -83> 2026-02-24T22:21:07.192+0000 7f857230e640 0 -- 10.20.193.46:0/2318160246 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f8494008ac0 msgr2=0x7f8494028f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -82> 2026-02-24T22:21:07.192+0000 7f85557f2640 1 client.8379.objecter ms_handle_reset 0x7f8494008ac0 session 0x7f8494008440 osd.1 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -81> 2026-02-24T22:21:07.192+0000 7f85557f2640 10 monclient: _renew_subs 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -80> 2026-02-24T22:21:07.192+0000 7f85557f2640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:21:07.284 INFO:tasks.workunit.client.0.trial046.stderr: -79> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding auth protocol: cephx 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -78> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding auth protocol: cephx 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -77> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding auth protocol: cephx 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -76> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding con mode: secure 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -75> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding con mode: crc 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -74> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding con mode: secure 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -73> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding con mode: crc 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -72> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding con mode: secure 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -71> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding con mode: crc 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -70> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding con mode: crc 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -69> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding con mode: secure 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -68> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding con mode: crc 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -67> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding con mode: secure 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -66> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding con mode: crc 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -65> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding con mode: secure 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -64> 2026-02-24T22:21:07.192+0000 7f857230e640 10 monclient: get_auth_request con 0x7f854409d450 auth_method 0 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -63> 2026-02-24T22:21:07.192+0000 7f857503ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -62> 2026-02-24T22:21:07.192+0000 7f857503ba80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -61> 2026-02-24T22:21:07.192+0000 7f857503ba80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -60> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command cache list hook 0x55cd629c5770 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -59> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command cache inspect hook 0x55cd629c5770 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -58> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command cache erase hook 0x55cd629c5770 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -57> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command cache zap hook 0x55cd629c5770 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -56> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 note: GC not initialized 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -55> 2026-02-24T22:21:07.192+0000 7f8520fe1640 20 reqs_thread_entry: start 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -54> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command sync trace show hook 0x7f8564006aa0 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -53> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command sync trace history hook 0x7f8564006aa0 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -52> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command sync trace active hook 0x7f8564006aa0 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -51> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command sync trace active_short hook 0x7f8564006aa0 2026-02-24T22:21:07.285 INFO:tasks.workunit.client.0.trial046.stderr: -50> 2026-02-24T22:21:07.192+0000 7f8573310640 10 monclient: get_auth_request con 0x55cd62b6b110 auth_method 0 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -49> 2026-02-24T22:21:07.196+0000 7f8572b0f640 10 monclient: get_auth_request con 0x55cd62b671e0 auth_method 0 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -48> 2026-02-24T22:21:07.196+0000 7f857230e640 10 monclient: get_auth_request con 0x55cd62b68fb0 auth_method 0 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -47> 2026-02-24T22:21:07.196+0000 7f8573310640 10 monclient: get_auth_request con 0x55cd62b6c4f0 auth_method 0 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -46> 2026-02-24T22:21:07.200+0000 7f8572b0f640 10 monclient: get_auth_request con 0x55cd62a6f7a0 auth_method 0 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -45> 2026-02-24T22:21:07.204+0000 7f857230e640 10 monclient: get_auth_request con 0x55cd62a71e40 auth_method 0 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -44> 2026-02-24T22:21:07.224+0000 7f857503ba80 20 init_complete bucket index max shards: 1 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -43> 2026-02-24T22:21:07.224+0000 7f857503ba80 20 Filter name: none 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -42> 2026-02-24T22:21:07.224+0000 7f848affd640 20 reqs_thread_entry: start 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -41> 2026-02-24T22:21:07.224+0000 7f857503ba80 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -40> 2026-02-24T22:21:07.224+0000 7f857503ba80 20 rados->read ofs=0 len=0 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -39> 2026-02-24T22:21:07.224+0000 7f8573310640 10 monclient: get_auth_request con 0x55cd62a74880 auth_method 0 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -38> 2026-02-24T22:21:07.224+0000 7f857503ba80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -37> 2026-02-24T22:21:07.224+0000 7f857503ba80 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -36> 2026-02-24T22:21:07.224+0000 7f857503ba80 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -35> 2026-02-24T22:21:07.224+0000 7f857503ba80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : miss 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -34> 2026-02-24T22:21:07.224+0000 7f857503ba80 20 rados->read ofs=0 len=0 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -33> 2026-02-24T22:21:07.224+0000 7f857503ba80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -32> 2026-02-24T22:21:07.224+0000 7f857503ba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -31> 2026-02-24T22:21:07.224+0000 7f857503ba80 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -30> 2026-02-24T22:21:07.224+0000 7f857503ba80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -29> 2026-02-24T22:21:07.224+0000 7f857503ba80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -28> 2026-02-24T22:21:07.224+0000 7f857503ba80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -27> 2026-02-24T22:21:07.228+0000 7f857503ba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -26> 2026-02-24T22:21:07.228+0000 7f857503ba80 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -25> 2026-02-24T22:21:07.228+0000 7f857503ba80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -24> 2026-02-24T22:21:07.228+0000 7f857503ba80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518, ofs0, ns] 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -23> 2026-02-24T22:21:07.228+0000 7f854e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959943 cookie 94340613537584 notifier 7826 bl.length()=929 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -22> 2026-02-24T22:21:07.228+0000 7f854e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -21> 2026-02-24T22:21:07.228+0000 7f854e7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:21:07.286 INFO:tasks.workunit.client.0.trial046.stderr: -20> 2026-02-24T22:21:07.228+0000 7f854e7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: -19> 2026-02-24T22:21:07.228+0000 7f857503ba80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_sources= new_sources= 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: -18> 2026-02-24T22:21:07.228+0000 7f857503ba80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential sources added=[] removed=[] 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: -17> 2026-02-24T22:21:07.228+0000 7f857503ba80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_dests= new_dests= 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: -16> 2026-02-24T22:21:07.228+0000 7f857503ba80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential dests added=[] removed=[] 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: -15> 2026-02-24T22:21:07.232+0000 7f857503ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.0 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: -14> 2026-02-24T22:21:07.232+0000 7f857503ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.0 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: -13> 2026-02-24T22:21:07.232+0000 7f857503ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.1 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: -12> 2026-02-24T22:21:07.232+0000 7f857503ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.1 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: -11> 2026-02-24T22:21:07.232+0000 7f857503ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.2 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: -10> 2026-02-24T22:21:07.232+0000 7f857503ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.2 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: -9> 2026-02-24T22:21:07.232+0000 7f857503ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.3 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: -8> 2026-02-24T22:21:07.232+0000 7f857503ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.3 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: -7> 2026-02-24T22:21:07.232+0000 7f857503ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: -6> 2026-02-24T22:21:07.232+0000 7f857503ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: -5> 2026-02-24T22:21:07.232+0000 7f857503ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: -4> 2026-02-24T22:21:07.232+0000 7f857503ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: -3> 2026-02-24T22:21:07.232+0000 7f857503ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.2 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: -2> 2026-02-24T22:21:07.232+0000 7f857503ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.2 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: -1> 2026-02-24T22:21:07.232+0000 7f857503ba80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f857503ba80 time 2026-02-24T22:21:07.237818+0000 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f857847deb1] 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: 2: (RGWBucketReshard::do_reshard(rgw::bucket_index_layout_generation const&, rgw::bucket_index_layout_generation const&, int, bool, bool, std::ostream*, ceph::Formatter*, FaultInjector > >&, DoutPrefixProvider const*, optional_yield)+0x274a) [0x55cd42d7ebea] 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xe6b) [0x55cd42d7ff6b] 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f857777dd90] 2026-02-24T22:21:07.287 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: 0> 2026-02-24T22:21:07.236+0000 7f857503ba80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f857503ba80 thread_name:radosgw-admin 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f8577796520] 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f857847df10] 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: 6: (RGWBucketReshard::do_reshard(rgw::bucket_index_layout_generation const&, rgw::bucket_index_layout_generation const&, int, bool, bool, std::ostream*, ceph::Formatter*, FaultInjector > >&, DoutPrefixProvider const*, optional_yield)+0x274a) [0x55cd42d7ebea] 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xe6b) [0x55cd42d7ff6b] 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f857777dd90] 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr:--- logging levels --- 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 none 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 lockdep 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 context 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 crush 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_balancer 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_locker 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log_expire 2026-02-24T22:21:07.288 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_migrator 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 3/ 5 mds_quiesce 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 buffer 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 timer 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 filer 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 striper 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 objecter 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rados 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_mirror 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_replay 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_pwl 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 journaler 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objectcacher 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 immutable_obj_cache 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 client 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 osd 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 optracker 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objclass 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 filestore 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 journal 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 ms 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mon 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 0/10 monc 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 paxos 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 tp 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 auth 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 crypto 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 finisher 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 reserver 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 heartbeatmap 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 perfcounter 2026-02-24T22:21:07.289 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_sync 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_datacache 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_access 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dbstore 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_flight 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_lifecycle 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_restore 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw_notification 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_bucket_logging 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 javaclient 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 asok 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 throttle 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 refs 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 compressor 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore_compression 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluefs 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 bdev 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 kstore 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 4/ 5 rocksdb 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 fuse 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 2/ 5 mgr 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mgrc 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 dpdk 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 eventtrace 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 prioritycache 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 test 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephfs_mirror 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephsqlite 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 crimson_interrupt 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_onode 2026-02-24T22:21:07.290 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_odata 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_omap 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_tm 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_t 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cleaner 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_epm 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_lba 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_fixedkv_tree 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cache 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_journal 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_device 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_backref 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 alienstore 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mclock 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dedup 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cyanstore 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 ceph_exporter 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 memstore 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 trace 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 ceph_dedup 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: -2/-2 (syslog threshold) 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 99/99 (stderr threshold) 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr:--- pthread ID / name mapping for recent threads --- 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 7f848affd640 / http_manager 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 7f8520fe1640 / http_manager 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 7f854d7fa640 / ms_dispatch 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 7f854e7fc640 / io_context_pool 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 7f854effd640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 7f85557f2640 / ms_dispatch 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 7f8557ff7640 / ms_dispatch 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 7f85597fa640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 7f8559ffb640 / admin_socket 2026-02-24T22:21:07.291 INFO:tasks.workunit.client.0.trial046.stderr: 7f8571b0d640 / ms_dispatch 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr: 7f857230e640 / msgr-worker-2 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr: 7f8572b0f640 / msgr-worker-1 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr: 7f8573310640 / msgr-worker-0 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr: 7f8573b11640 / io_context_pool 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr: 7f857503ba80 / radosgw-admin 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr: max_recent 500 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr: max_new 1000 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr: log_file /var/log/ceph/ceph-client.admin.54360.log 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr:--- end dump of recent events --- 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr:--- begin dump of recent events --- 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr: -499> 2026-02-24T22:21:07.172+0000 7f8572b0f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr: -498> 2026-02-24T22:21:07.172+0000 7f8572b0f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr: -497> 2026-02-24T22:21:07.172+0000 7f8572b0f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr: -496> 2026-02-24T22:21:07.172+0000 7f857230e640 10 monclient(hunting): handle_auth_done global_id 8373 payload 274 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr: -495> 2026-02-24T22:21:07.172+0000 7f857230e640 10 monclient: _finish_hunting 0 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr: -494> 2026-02-24T22:21:07.172+0000 7f857230e640 1 monclient: found mon.noname-a 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr: -493> 2026-02-24T22:21:07.172+0000 7f857230e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr: -492> 2026-02-24T22:21:07.172+0000 7f85597fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr: -491> 2026-02-24T22:21:07.172+0000 7f85597fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr: -490> 2026-02-24T22:21:07.172+0000 7f85597fa640 10 monclient: dump: 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr: -489> 2026-02-24T22:21:07.172+0000 7f85597fa640 10 monclient: _finish_auth 0 2026-02-24T22:21:07.292 INFO:tasks.workunit.client.0.trial046.stderr: -488> 2026-02-24T22:21:07.172+0000 7f85597fa640 10 monclient: _check_auth_tickets 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr: -487> 2026-02-24T22:21:07.172+0000 7f85597fa640 10 monclient: handle_config config(0 keys) 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr: -486> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 monclient: authenticate success, global_id 8373 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr: -485> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient: get_monmap_and_config success 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr: -484> 2026-02-24T22:21:07.172+0000 7f857503ba80 4 set_mon_vals no callback set 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr: -483> 2026-02-24T22:21:07.172+0000 7f85597fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr: -482> 2026-02-24T22:21:07.172+0000 7f85597fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr: -481> 2026-02-24T22:21:07.172+0000 7f85597fa640 10 monclient: dump: 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr: -480> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient: shutdown 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr: -479> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 asok(0x55cd625edf00) unregister_commands rotate-key 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr: -478> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient: build_initial_monmap 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr: -477> 2026-02-24T22:21:07.172+0000 7f857503ba80 1 build_initial for_mkfs: 0 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr: -476> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient: monmap: 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.293 INFO:tasks.workunit.client.0.trial046.stderr: -475> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding auth protocol: cephx 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -474> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding auth protocol: cephx 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -473> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding auth protocol: cephx 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -472> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding con mode: secure 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -471> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding con mode: crc 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -470> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding con mode: secure 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -469> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding con mode: crc 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -468> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding con mode: secure 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -467> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding con mode: crc 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -466> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding con mode: crc 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -465> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding con mode: secure 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -464> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding con mode: crc 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -463> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding con mode: secure 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -462> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding con mode: crc 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -461> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626f5980) adding con mode: secure 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -460> 2026-02-24T22:21:07.172+0000 7f857503ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -459> 2026-02-24T22:21:07.172+0000 7f857503ba80 1 librados: starting msgr at 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -458> 2026-02-24T22:21:07.172+0000 7f857503ba80 1 librados: starting objecter 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -457> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command objecter_requests hook 0x55cd62721160 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -456> 2026-02-24T22:21:07.172+0000 7f857503ba80 1 librados: setting wanted keys 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -455> 2026-02-24T22:21:07.172+0000 7f857503ba80 1 librados: calling monclient init 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -454> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient: init 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -453> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding auth protocol: cephx 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -452> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding auth protocol: cephx 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -451> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding auth protocol: cephx 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -450> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding con mode: secure 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -449> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding con mode: crc 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -448> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding con mode: secure 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -447> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding con mode: crc 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -446> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding con mode: secure 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -445> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding con mode: crc 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -444> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding con mode: crc 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -443> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding con mode: secure 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -442> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding con mode: crc 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -441> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding con mode: secure 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -440> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding con mode: crc 2026-02-24T22:21:07.294 INFO:tasks.workunit.client.0.trial046.stderr: -439> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 AuthRegistry(0x55cd626ef3f0) adding con mode: secure 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -438> 2026-02-24T22:21:07.172+0000 7f857503ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -437> 2026-02-24T22:21:07.172+0000 7f857503ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -436> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command rotate-key hook 0x55cd626ef538 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -435> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient: _reopen_session rank -1 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -434> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -433> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient(hunting): picked mon.noname-b con 0x55cd626797a0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -432> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient(hunting): picked mon.noname-a con 0x55cd6267c010 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -431> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient(hunting): picked mon.noname-c con 0x55cd62723510 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -430> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -429> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -428> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -427> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient(hunting): _renew_subs 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -426> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient(hunting): authenticate will time out at 2037.281839s 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -425> 2026-02-24T22:21:07.172+0000 7f857230e640 10 monclient(hunting): get_auth_request con 0x55cd6267c010 auth_method 0 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -424> 2026-02-24T22:21:07.172+0000 7f857230e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -423> 2026-02-24T22:21:07.172+0000 7f857230e640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -422> 2026-02-24T22:21:07.172+0000 7f857230e640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -421> 2026-02-24T22:21:07.172+0000 7f8573310640 10 monclient(hunting): get_auth_request con 0x55cd62723510 auth_method 0 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -420> 2026-02-24T22:21:07.172+0000 7f8573310640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -419> 2026-02-24T22:21:07.172+0000 7f8573310640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -418> 2026-02-24T22:21:07.172+0000 7f8573310640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -417> 2026-02-24T22:21:07.172+0000 7f857230e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -416> 2026-02-24T22:21:07.172+0000 7f857230e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -415> 2026-02-24T22:21:07.172+0000 7f857230e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -414> 2026-02-24T22:21:07.172+0000 7f8573310640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -413> 2026-02-24T22:21:07.172+0000 7f8573310640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -412> 2026-02-24T22:21:07.172+0000 7f8573310640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -411> 2026-02-24T22:21:07.172+0000 7f857230e640 10 monclient(hunting): handle_auth_done global_id 8376 payload 824 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -410> 2026-02-24T22:21:07.172+0000 7f857230e640 10 monclient: _finish_hunting 0 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -409> 2026-02-24T22:21:07.172+0000 7f857230e640 1 monclient: found mon.noname-a 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -408> 2026-02-24T22:21:07.172+0000 7f857230e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -407> 2026-02-24T22:21:07.172+0000 7f8557ff7640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -406> 2026-02-24T22:21:07.172+0000 7f8557ff7640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:21:07.295 INFO:tasks.workunit.client.0.trial046.stderr: -405> 2026-02-24T22:21:07.172+0000 7f8557ff7640 10 monclient: dump: 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr: -404> 2026-02-24T22:21:07.172+0000 7f8557ff7640 10 monclient: _finish_auth 0 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr: -403> 2026-02-24T22:21:07.172+0000 7f8557ff7640 10 monclient: _check_auth_tickets 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr: -402> 2026-02-24T22:21:07.172+0000 7f8557ff7640 10 monclient: handle_config config(0 keys) 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr: -401> 2026-02-24T22:21:07.172+0000 7f8557ff7640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr: -400> 2026-02-24T22:21:07.172+0000 7f8557ff7640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr: -399> 2026-02-24T22:21:07.172+0000 7f8557ff7640 10 monclient: dump: 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr: -398> 2026-02-24T22:21:07.172+0000 7f857503ba80 5 monclient: authenticate success, global_id 8376 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr: -397> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient: _renew_subs 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr: -396> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr: -395> 2026-02-24T22:21:07.172+0000 7f85597fa640 4 set_mon_vals no callback set 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr: -394> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient: _renew_subs 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr: -393> 2026-02-24T22:21:07.172+0000 7f857503ba80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr: -392> 2026-02-24T22:21:07.172+0000 7f857503ba80 1 librados: init done 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr: -391> 2026-02-24T22:21:07.176+0000 7f8557ff7640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:21:07.296 INFO:tasks.workunit.client.0.trial046.stderr: -390> 2026-02-24T22:21:07.176+0000 7f8557ff7640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: -389> 2026-02-24T22:21:07.176+0000 7f8557ff7640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: -388> 2026-02-24T22:21:07.176+0000 7f8572b0f640 10 monclient: get_auth_request con 0x7f85500707b0 auth_method 0 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: -387> 2026-02-24T22:21:07.176+0000 7f8573310640 10 monclient: get_auth_request con 0x55cd62717b50 auth_method 0 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: -386> 2026-02-24T22:21:07.176+0000 7f8572b0f640 10 monclient: get_auth_request con 0x55cd6277b0e0 auth_method 0 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: -385> 2026-02-24T22:21:07.176+0000 7f857230e640 10 monclient: get_auth_request con 0x55cd6277de00 auth_method 0 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: -384> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: build_initial_monmap 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: -383> 2026-02-24T22:21:07.180+0000 7f857503ba80 1 build_initial for_mkfs: 0 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: -382> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: monmap: 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: -381> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding auth protocol: cephx 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: -380> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding auth protocol: cephx 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: -379> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding auth protocol: cephx 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: -378> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding con mode: secure 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: -377> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding con mode: crc 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: -376> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding con mode: secure 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: -375> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding con mode: crc 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: -374> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding con mode: secure 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: -373> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding con mode: crc 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: -372> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding con mode: crc 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: -371> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding con mode: secure 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: -370> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding con mode: crc 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: -369> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding con mode: secure 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: -368> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding con mode: crc 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: -367> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277f980) adding con mode: secure 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: -366> 2026-02-24T22:21:07.180+0000 7f857503ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: -365> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command objecter_requests cmddesc objecter_requests hook 0x55cd627842c0 EEXIST 2026-02-24T22:21:07.297 INFO:tasks.workunit.client.0.trial046.stderr: -364> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: init 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -363> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding auth protocol: cephx 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -362> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding auth protocol: cephx 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -361> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding auth protocol: cephx 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -360> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding con mode: secure 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -359> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding con mode: crc 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -358> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding con mode: secure 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -357> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding con mode: crc 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -356> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding con mode: secure 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -355> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding con mode: crc 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -354> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding con mode: crc 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -353> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding con mode: secure 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -352> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding con mode: crc 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -351> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding con mode: secure 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -350> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding con mode: crc 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -349> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd6277bed8) adding con mode: secure 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -348> 2026-02-24T22:21:07.180+0000 7f857503ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -347> 2026-02-24T22:21:07.180+0000 7f857503ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -346> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command rotate-key cmddesc rotate-key hook 0x55cd6277c020 EEXIST 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -345> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: _reopen_session rank -1 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -344> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -343> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): picked mon.noname-c con 0x55cd628d7e20 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -342> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): picked mon.noname-a con 0x55cd628d5b50 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -341> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): picked mon.noname-b con 0x55cd628d5f60 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -340> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -339> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -338> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -337> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): _renew_subs 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -336> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): authenticate will time out at 2037.288993s 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -335> 2026-02-24T22:21:07.180+0000 7f8572b0f640 10 monclient(hunting): get_auth_request con 0x55cd628d7e20 auth_method 0 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -334> 2026-02-24T22:21:07.180+0000 7f8572b0f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -333> 2026-02-24T22:21:07.180+0000 7f8572b0f640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -332> 2026-02-24T22:21:07.180+0000 7f8572b0f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -331> 2026-02-24T22:21:07.180+0000 7f857230e640 10 monclient(hunting): get_auth_request con 0x55cd628d5b50 auth_method 0 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -330> 2026-02-24T22:21:07.180+0000 7f857230e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -329> 2026-02-24T22:21:07.180+0000 7f857230e640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -328> 2026-02-24T22:21:07.180+0000 7f857230e640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:07.298 INFO:tasks.workunit.client.0.trial046.stderr: -327> 2026-02-24T22:21:07.180+0000 7f857230e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr: -326> 2026-02-24T22:21:07.180+0000 7f857230e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr: -325> 2026-02-24T22:21:07.180+0000 7f857230e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr: -324> 2026-02-24T22:21:07.180+0000 7f8572b0f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr: -323> 2026-02-24T22:21:07.180+0000 7f8572b0f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr: -322> 2026-02-24T22:21:07.180+0000 7f8572b0f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr: -321> 2026-02-24T22:21:07.180+0000 7f8573310640 10 monclient(hunting): get_auth_request con 0x55cd628d5f60 auth_method 0 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr: -320> 2026-02-24T22:21:07.180+0000 7f8573310640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr: -319> 2026-02-24T22:21:07.180+0000 7f8573310640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr: -318> 2026-02-24T22:21:07.180+0000 7f8573310640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr: -317> 2026-02-24T22:21:07.180+0000 7f857230e640 10 monclient(hunting): handle_auth_done global_id 8379 payload 824 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr: -316> 2026-02-24T22:21:07.180+0000 7f857230e640 10 monclient: _finish_hunting 0 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr: -315> 2026-02-24T22:21:07.180+0000 7f857230e640 1 monclient: found mon.noname-a 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr: -314> 2026-02-24T22:21:07.180+0000 7f857230e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr: -313> 2026-02-24T22:21:07.180+0000 7f85557f2640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr: -312> 2026-02-24T22:21:07.180+0000 7f85557f2640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr: -311> 2026-02-24T22:21:07.180+0000 7f85557f2640 10 monclient: dump: 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr: -310> 2026-02-24T22:21:07.180+0000 7f85557f2640 10 monclient: _finish_auth 0 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr: -309> 2026-02-24T22:21:07.180+0000 7f85557f2640 10 monclient: _check_auth_tickets 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr: -308> 2026-02-24T22:21:07.180+0000 7f85557f2640 10 monclient: handle_config config(0 keys) 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr: -307> 2026-02-24T22:21:07.180+0000 7f85557f2640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr: -306> 2026-02-24T22:21:07.180+0000 7f85557f2640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr: -305> 2026-02-24T22:21:07.180+0000 7f85557f2640 10 monclient: dump: 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.299 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr: -304> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 monclient: authenticate success, global_id 8379 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr: -303> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: _renew_subs 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr: -302> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr: -301> 2026-02-24T22:21:07.180+0000 7f8573b11640 4 set_mon_vals no callback set 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr: -300> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: _renew_subs 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr: -299> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr: -298> 2026-02-24T22:21:07.180+0000 7f85557f2640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr: -297> 2026-02-24T22:21:07.180+0000 7f85557f2640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr: -296> 2026-02-24T22:21:07.180+0000 7f85557f2640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr: -295> 2026-02-24T22:21:07.180+0000 7f8572b0f640 10 monclient: get_auth_request con 0x7f85440706c0 auth_method 0 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr: -294> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: get_monmap_and_config 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr: -293> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: build_initial_monmap 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr: -292> 2026-02-24T22:21:07.180+0000 7f857503ba80 1 build_initial for_mkfs: 0 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr: -291> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: monmap: 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr: -290> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding auth protocol: cephx 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr: -289> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding auth protocol: cephx 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr: -288> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding auth protocol: cephx 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr: -287> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding con mode: secure 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr: -286> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding con mode: crc 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr: -285> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding con mode: secure 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr: -284> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding con mode: crc 2026-02-24T22:21:07.300 INFO:tasks.workunit.client.0.trial046.stderr: -283> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding con mode: secure 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -282> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding con mode: crc 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -281> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding con mode: crc 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -280> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding con mode: secure 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -279> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding con mode: crc 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -278> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding con mode: secure 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -277> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding con mode: crc 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -276> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x55cd62969000) adding con mode: secure 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -275> 2026-02-24T22:21:07.180+0000 7f857503ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -274> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: init 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -273> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding auth protocol: cephx 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -272> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding auth protocol: cephx 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -271> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding auth protocol: cephx 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -270> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding con mode: secure 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -269> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding con mode: crc 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -268> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding con mode: secure 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -267> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding con mode: crc 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -266> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding con mode: secure 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -265> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding con mode: crc 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -264> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding con mode: crc 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -263> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding con mode: secure 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -262> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding con mode: crc 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -261> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding con mode: secure 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -260> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding con mode: crc 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -259> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41a940) adding con mode: secure 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -258> 2026-02-24T22:21:07.180+0000 7f857503ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -257> 2026-02-24T22:21:07.180+0000 7f857503ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -256> 2026-02-24T22:21:07.180+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command rotate-key cmddesc rotate-key hook 0x7ffc7a41aa88 EEXIST 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -255> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: _reopen_session rank -1 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -254> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -253> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): picked mon.noname-b con 0x55cd62990070 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -252> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): picked mon.noname-c con 0x55cd629937e0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -251> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): picked mon.noname-a con 0x55cd629c05c0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -250> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -249> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -248> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -247> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): _renew_subs 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -246> 2026-02-24T22:21:07.180+0000 7f857503ba80 10 monclient(hunting): authenticate will time out at 2037.291965s 2026-02-24T22:21:07.301 INFO:tasks.workunit.client.0.trial046.stderr: -245> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): get_auth_request con 0x55cd629c05c0 auth_method 0 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -244> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -243> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -242> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -241> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): get_auth_request con 0x55cd629937e0 auth_method 0 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -240> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -239> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -238> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -237> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -236> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -235> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -234> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -233> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -232> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -231> 2026-02-24T22:21:07.184+0000 7f8572b0f640 10 monclient(hunting): get_auth_request con 0x55cd62990070 auth_method 0 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -230> 2026-02-24T22:21:07.184+0000 7f8572b0f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -229> 2026-02-24T22:21:07.184+0000 7f8572b0f640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -228> 2026-02-24T22:21:07.184+0000 7f8572b0f640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -227> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): handle_auth_done global_id 8382 payload 274 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -226> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient: _finish_hunting 0 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -225> 2026-02-24T22:21:07.184+0000 7f8573310640 1 monclient: found mon.noname-a 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -224> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -223> 2026-02-24T22:21:07.184+0000 7f854effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -222> 2026-02-24T22:21:07.184+0000 7f854effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -221> 2026-02-24T22:21:07.184+0000 7f854effd640 10 monclient: dump: 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -220> 2026-02-24T22:21:07.184+0000 7f854effd640 10 monclient: _finish_auth 0 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -219> 2026-02-24T22:21:07.184+0000 7f854effd640 10 monclient: _check_auth_tickets 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -218> 2026-02-24T22:21:07.184+0000 7f854effd640 10 monclient: handle_config config(0 keys) 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -217> 2026-02-24T22:21:07.184+0000 7f854effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:07.302 INFO:tasks.workunit.client.0.trial046.stderr: -216> 2026-02-24T22:21:07.184+0000 7f854effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr: -215> 2026-02-24T22:21:07.184+0000 7f854effd640 10 monclient: dump: 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr: -214> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 monclient: authenticate success, global_id 8382 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr: -213> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient: get_monmap_and_config success 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr: -212> 2026-02-24T22:21:07.184+0000 7f857503ba80 4 set_mon_vals no callback set 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr: -211> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient: shutdown 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr: -210> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient: build_initial_monmap 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr: -209> 2026-02-24T22:21:07.184+0000 7f857503ba80 1 build_initial for_mkfs: 0 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr: -208> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient: monmap: 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr: -207> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding auth protocol: cephx 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr: -206> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding auth protocol: cephx 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr: -205> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding auth protocol: cephx 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr: -204> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding con mode: secure 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr: -203> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding con mode: crc 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr: -202> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding con mode: secure 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr: -201> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding con mode: crc 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr: -200> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding con mode: secure 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr: -199> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding con mode: crc 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr: -198> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding con mode: crc 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr: -197> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding con mode: secure 2026-02-24T22:21:07.303 INFO:tasks.workunit.client.0.trial046.stderr: -196> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding con mode: crc 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -195> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding con mode: secure 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -194> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding con mode: crc 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -193> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968f90) adding con mode: secure 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -192> 2026-02-24T22:21:07.184+0000 7f857503ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -191> 2026-02-24T22:21:07.184+0000 7f857503ba80 1 librados: starting msgr at 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -190> 2026-02-24T22:21:07.184+0000 7f857503ba80 1 librados: starting objecter 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -189> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command objecter_requests cmddesc objecter_requests hook 0x55cd6296e9b0 EEXIST 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -188> 2026-02-24T22:21:07.184+0000 7f857503ba80 1 librados: setting wanted keys 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -187> 2026-02-24T22:21:07.184+0000 7f857503ba80 1 librados: calling monclient init 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -186> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient: init 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -185> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding auth protocol: cephx 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -184> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding auth protocol: cephx 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -183> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding auth protocol: cephx 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -182> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding con mode: secure 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -181> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding con mode: crc 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -180> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding con mode: secure 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -179> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding con mode: crc 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -178> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding con mode: secure 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -177> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding con mode: crc 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -176> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding con mode: crc 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -175> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding con mode: secure 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -174> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding con mode: crc 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -173> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding con mode: secure 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -172> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding con mode: crc 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -171> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 AuthRegistry(0x55cd62968050) adding con mode: secure 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -170> 2026-02-24T22:21:07.184+0000 7f857503ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -169> 2026-02-24T22:21:07.184+0000 7f857503ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -168> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command rotate-key cmddesc rotate-key hook 0x55cd62968198 EEXIST 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -167> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient: _reopen_session rank -1 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -166> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -165> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient(hunting): picked mon.noname-b con 0x55cd629937e0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -164> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient(hunting): picked mon.noname-a con 0x55cd629c05c0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -163> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient(hunting): picked mon.noname-c con 0x55cd629c5940 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -162> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -161> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -160> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -159> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient(hunting): _renew_subs 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -158> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient(hunting): authenticate will time out at 2037.293308s 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -157> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): get_auth_request con 0x55cd629c5940 auth_method 0 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -156> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -155> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -154> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:07.304 INFO:tasks.workunit.client.0.trial046.stderr: -153> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): get_auth_request con 0x55cd629c05c0 auth_method 0 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: -152> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: -151> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: -150> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: -149> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: -148> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: -147> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: -146> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: -145> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: -144> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: -143> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient(hunting): handle_auth_done global_id 7826 payload 824 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: -142> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient: _finish_hunting 0 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: -141> 2026-02-24T22:21:07.184+0000 7f8573310640 1 monclient: found mon.noname-c 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: -140> 2026-02-24T22:21:07.184+0000 7f8573310640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: -139> 2026-02-24T22:21:07.184+0000 7f8572b0f640 10 monclient: get_auth_request con 0x55cd629937e0 auth_method 0 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: -138> 2026-02-24T22:21:07.184+0000 7f854d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: -137> 2026-02-24T22:21:07.184+0000 7f854d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: -136> 2026-02-24T22:21:07.184+0000 7f854d7fa640 10 monclient: dump: 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: -135> 2026-02-24T22:21:07.184+0000 7f854d7fa640 10 monclient: _finish_auth 0 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: -134> 2026-02-24T22:21:07.184+0000 7f854d7fa640 10 monclient: _check_auth_tickets 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: -133> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 monclient: authenticate success, global_id 7826 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: -132> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient: _renew_subs 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: -131> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: -130> 2026-02-24T22:21:07.184+0000 7f854d7fa640 10 monclient: handle_config config(0 keys) 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: -129> 2026-02-24T22:21:07.184+0000 7f854d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: -128> 2026-02-24T22:21:07.184+0000 7f854d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr: -127> 2026-02-24T22:21:07.184+0000 7f854d7fa640 10 monclient: dump: 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:07.305 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -126> 2026-02-24T22:21:07.184+0000 7f854effd640 4 set_mon_vals no callback set 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -125> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient: _renew_subs 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -124> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -123> 2026-02-24T22:21:07.184+0000 7f857503ba80 1 librados: init done 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -122> 2026-02-24T22:21:07.184+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command cr dump hook 0x55cd629bab38 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -121> 2026-02-24T22:21:07.184+0000 7f857503ba80 4 Realm: () 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -120> 2026-02-24T22:21:07.184+0000 7f857503ba80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -119> 2026-02-24T22:21:07.184+0000 7f857503ba80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -118> 2026-02-24T22:21:07.184+0000 7f857503ba80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -117> 2026-02-24T22:21:07.184+0000 7f857503ba80 20 zonegroup default 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -116> 2026-02-24T22:21:07.184+0000 7f8572b0f640 10 monclient: get_auth_request con 0x55cd62a31100 auth_method 0 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -115> 2026-02-24T22:21:07.184+0000 7f857503ba80 20 rados->read ofs=0 len=0 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -114> 2026-02-24T22:21:07.184+0000 7f854d7fa640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -113> 2026-02-24T22:21:07.184+0000 7f854d7fa640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -112> 2026-02-24T22:21:07.184+0000 7f854d7fa640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -111> 2026-02-24T22:21:07.184+0000 7f857230e640 10 monclient: get_auth_request con 0x7f8534070730 auth_method 0 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -110> 2026-02-24T22:21:07.188+0000 7f8572b0f640 10 monclient: get_auth_request con 0x55cd62a54420 auth_method 0 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -109> 2026-02-24T22:21:07.188+0000 7f857503ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -108> 2026-02-24T22:21:07.188+0000 7f857503ba80 20 rados->read ofs=0 len=0 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -107> 2026-02-24T22:21:07.188+0000 7f857230e640 10 monclient: get_auth_request con 0x55cd62a75b40 auth_method 0 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -106> 2026-02-24T22:21:07.188+0000 7f857503ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -105> 2026-02-24T22:21:07.188+0000 7f857503ba80 20 started sync module instance, tier type = 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -104> 2026-02-24T22:21:07.188+0000 7f857503ba80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -103> 2026-02-24T22:21:07.188+0000 7f8573310640 10 monclient: get_auth_request con 0x55cd62ad6fc0 auth_method 0 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -102> 2026-02-24T22:21:07.188+0000 7f8573310640 10 monclient: get_auth_request con 0x55cd62bdea30 auth_method 0 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -101> 2026-02-24T22:21:07.188+0000 7f8572b0f640 10 monclient: get_auth_request con 0x55cd62b39960 auth_method 0 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -100> 2026-02-24T22:21:07.188+0000 7f857230e640 10 monclient: get_auth_request con 0x55cd62b9c8f0 auth_method 0 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -99> 2026-02-24T22:21:07.188+0000 7f8572b0f640 10 monclient: get_auth_request con 0x55cd62c41a30 auth_method 0 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -98> 2026-02-24T22:21:07.188+0000 7f8572b0f640 0 -- 10.20.193.46:0/3692073238 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55cd62a54420 msgr2=0x55cd62a748d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -97> 2026-02-24T22:21:07.188+0000 7f854d7fa640 1 client.7826.objecter ms_handle_reset 0x55cd62a54420 session 0x55cd62a30630 osd.4 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -96> 2026-02-24T22:21:07.188+0000 7f854d7fa640 10 monclient: _renew_subs 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -95> 2026-02-24T22:21:07.188+0000 7f854d7fa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -94> 2026-02-24T22:21:07.188+0000 7f8572b0f640 10 monclient: get_auth_request con 0x7f853409d460 auth_method 0 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -93> 2026-02-24T22:21:07.188+0000 7f857503ba80 20 add_watcher() i=1 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -92> 2026-02-24T22:21:07.188+0000 7f857503ba80 20 add_watcher() i=3 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -91> 2026-02-24T22:21:07.188+0000 7f857503ba80 20 add_watcher() i=0 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -90> 2026-02-24T22:21:07.188+0000 7f857503ba80 20 add_watcher() i=5 2026-02-24T22:21:07.306 INFO:tasks.workunit.client.0.trial046.stderr: -89> 2026-02-24T22:21:07.188+0000 7f857503ba80 20 add_watcher() i=2 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -88> 2026-02-24T22:21:07.188+0000 7f857503ba80 20 add_watcher() i=4 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -87> 2026-02-24T22:21:07.192+0000 7f857503ba80 20 add_watcher() i=6 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -86> 2026-02-24T22:21:07.192+0000 7f857503ba80 20 add_watcher() i=7 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -85> 2026-02-24T22:21:07.192+0000 7f857503ba80 2 all 8 watchers are set, enabling cache 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -84> 2026-02-24T22:21:07.192+0000 7f857230e640 10 monclient: get_auth_request con 0x7f8494008ac0 auth_method 0 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -83> 2026-02-24T22:21:07.192+0000 7f857230e640 0 -- 10.20.193.46:0/2318160246 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f8494008ac0 msgr2=0x7f8494028f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -82> 2026-02-24T22:21:07.192+0000 7f85557f2640 1 client.8379.objecter ms_handle_reset 0x7f8494008ac0 session 0x7f8494008440 osd.1 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -81> 2026-02-24T22:21:07.192+0000 7f85557f2640 10 monclient: _renew_subs 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -80> 2026-02-24T22:21:07.192+0000 7f85557f2640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -79> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding auth protocol: cephx 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -78> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding auth protocol: cephx 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -77> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding auth protocol: cephx 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -76> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding con mode: secure 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -75> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding con mode: crc 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -74> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding con mode: secure 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -73> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding con mode: crc 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -72> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding con mode: secure 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -71> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding con mode: crc 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -70> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding con mode: crc 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -69> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding con mode: secure 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -68> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding con mode: crc 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -67> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding con mode: secure 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -66> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding con mode: crc 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -65> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 AuthRegistry(0x7ffc7a41adb0) adding con mode: secure 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -64> 2026-02-24T22:21:07.192+0000 7f857230e640 10 monclient: get_auth_request con 0x7f854409d450 auth_method 0 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -63> 2026-02-24T22:21:07.192+0000 7f857503ba80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -62> 2026-02-24T22:21:07.192+0000 7f857503ba80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -61> 2026-02-24T22:21:07.192+0000 7f857503ba80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -60> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command cache list hook 0x55cd629c5770 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -59> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command cache inspect hook 0x55cd629c5770 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -58> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command cache erase hook 0x55cd629c5770 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -57> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command cache zap hook 0x55cd629c5770 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -56> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 note: GC not initialized 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -55> 2026-02-24T22:21:07.192+0000 7f8520fe1640 20 reqs_thread_entry: start 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -54> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command sync trace show hook 0x7f8564006aa0 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -53> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command sync trace history hook 0x7f8564006aa0 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -52> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command sync trace active hook 0x7f8564006aa0 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -51> 2026-02-24T22:21:07.192+0000 7f857503ba80 5 asok(0x55cd625edf00) register_command sync trace active_short hook 0x7f8564006aa0 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -50> 2026-02-24T22:21:07.192+0000 7f8573310640 10 monclient: get_auth_request con 0x55cd62b6b110 auth_method 0 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -49> 2026-02-24T22:21:07.196+0000 7f8572b0f640 10 monclient: get_auth_request con 0x55cd62b671e0 auth_method 0 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -48> 2026-02-24T22:21:07.196+0000 7f857230e640 10 monclient: get_auth_request con 0x55cd62b68fb0 auth_method 0 2026-02-24T22:21:07.307 INFO:tasks.workunit.client.0.trial046.stderr: -47> 2026-02-24T22:21:07.196+0000 7f8573310640 10 monclient: get_auth_request con 0x55cd62b6c4f0 auth_method 0 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -46> 2026-02-24T22:21:07.200+0000 7f8572b0f640 10 monclient: get_auth_request con 0x55cd62a6f7a0 auth_method 0 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -45> 2026-02-24T22:21:07.204+0000 7f857230e640 10 monclient: get_auth_request con 0x55cd62a71e40 auth_method 0 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -44> 2026-02-24T22:21:07.224+0000 7f857503ba80 20 init_complete bucket index max shards: 1 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -43> 2026-02-24T22:21:07.224+0000 7f857503ba80 20 Filter name: none 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -42> 2026-02-24T22:21:07.224+0000 7f848affd640 20 reqs_thread_entry: start 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -41> 2026-02-24T22:21:07.224+0000 7f857503ba80 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -40> 2026-02-24T22:21:07.224+0000 7f857503ba80 20 rados->read ofs=0 len=0 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -39> 2026-02-24T22:21:07.224+0000 7f8573310640 10 monclient: get_auth_request con 0x55cd62a74880 auth_method 0 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -38> 2026-02-24T22:21:07.224+0000 7f857503ba80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -37> 2026-02-24T22:21:07.224+0000 7f857503ba80 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -36> 2026-02-24T22:21:07.224+0000 7f857503ba80 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -35> 2026-02-24T22:21:07.224+0000 7f857503ba80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : miss 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -34> 2026-02-24T22:21:07.224+0000 7f857503ba80 20 rados->read ofs=0 len=0 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -33> 2026-02-24T22:21:07.224+0000 7f857503ba80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -32> 2026-02-24T22:21:07.224+0000 7f857503ba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -31> 2026-02-24T22:21:07.224+0000 7f857503ba80 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -30> 2026-02-24T22:21:07.224+0000 7f857503ba80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -29> 2026-02-24T22:21:07.224+0000 7f857503ba80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -28> 2026-02-24T22:21:07.224+0000 7f857503ba80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -27> 2026-02-24T22:21:07.228+0000 7f857503ba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -26> 2026-02-24T22:21:07.228+0000 7f857503ba80 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -25> 2026-02-24T22:21:07.228+0000 7f857503ba80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -24> 2026-02-24T22:21:07.228+0000 7f857503ba80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518, ofs0, ns] 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -23> 2026-02-24T22:21:07.228+0000 7f854e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959943 cookie 94340613537584 notifier 7826 bl.length()=929 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -22> 2026-02-24T22:21:07.228+0000 7f854e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -21> 2026-02-24T22:21:07.228+0000 7f854e7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -20> 2026-02-24T22:21:07.228+0000 7f854e7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -19> 2026-02-24T22:21:07.228+0000 7f857503ba80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_sources= new_sources= 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -18> 2026-02-24T22:21:07.228+0000 7f857503ba80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential sources added=[] removed=[] 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -17> 2026-02-24T22:21:07.228+0000 7f857503ba80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_dests= new_dests= 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -16> 2026-02-24T22:21:07.228+0000 7f857503ba80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential dests added=[] removed=[] 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -15> 2026-02-24T22:21:07.232+0000 7f857503ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.0 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -14> 2026-02-24T22:21:07.232+0000 7f857503ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.0 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -13> 2026-02-24T22:21:07.232+0000 7f857503ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.1 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -12> 2026-02-24T22:21:07.232+0000 7f857503ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.1 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -11> 2026-02-24T22:21:07.232+0000 7f857503ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.2 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -10> 2026-02-24T22:21:07.232+0000 7f857503ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.2 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -9> 2026-02-24T22:21:07.232+0000 7f857503ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.3 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -8> 2026-02-24T22:21:07.232+0000 7f857503ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.3 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -7> 2026-02-24T22:21:07.232+0000 7f857503ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -6> 2026-02-24T22:21:07.232+0000 7f857503ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -5> 2026-02-24T22:21:07.232+0000 7f857503ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -4> 2026-02-24T22:21:07.232+0000 7f857503ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1 2026-02-24T22:21:07.308 INFO:tasks.workunit.client.0.trial046.stderr: -3> 2026-02-24T22:21:07.232+0000 7f857503ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.2 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: -2> 2026-02-24T22:21:07.232+0000 7f857503ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.2 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: -1> 2026-02-24T22:21:07.232+0000 7f857503ba80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f857503ba80 time 2026-02-24T22:21:07.237818+0000 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f857847deb1] 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 2: (RGWBucketReshard::do_reshard(rgw::bucket_index_layout_generation const&, rgw::bucket_index_layout_generation const&, int, bool, bool, std::ostream*, ceph::Formatter*, FaultInjector > >&, DoutPrefixProvider const*, optional_yield)+0x274a) [0x55cd42d7ebea] 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xe6b) [0x55cd42d7ff6b] 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f857777dd90] 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 0> 2026-02-24T22:21:07.236+0000 7f857503ba80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f857503ba80 thread_name:radosgw-admin 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f8577796520] 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f857847df10] 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 6: (RGWBucketReshard::do_reshard(rgw::bucket_index_layout_generation const&, rgw::bucket_index_layout_generation const&, int, bool, bool, std::ostream*, ceph::Formatter*, FaultInjector > >&, DoutPrefixProvider const*, optional_yield)+0x274a) [0x55cd42d7ebea] 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0xe6b) [0x55cd42d7ff6b] 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f857777dd90] 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr:--- logging levels --- 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 none 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 lockdep 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 context 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 crush 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_balancer 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_locker 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log_expire 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_migrator 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 3/ 5 mds_quiesce 2026-02-24T22:21:07.309 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 buffer 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 timer 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 filer 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 striper 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 objecter 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rados 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_mirror 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_replay 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_pwl 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 journaler 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objectcacher 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 immutable_obj_cache 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 client 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 osd 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 optracker 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objclass 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 filestore 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 journal 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 ms 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mon 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 0/10 monc 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 paxos 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 tp 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 auth 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 crypto 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 finisher 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 reserver 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 heartbeatmap 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 perfcounter 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_sync 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_datacache 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_access 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dbstore 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_flight 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_lifecycle 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_restore 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw_notification 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_bucket_logging 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 javaclient 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 asok 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 throttle 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 refs 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 compressor 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore_compression 2026-02-24T22:21:07.310 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluefs 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 bdev 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 kstore 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 4/ 5 rocksdb 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 fuse 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 2/ 5 mgr 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mgrc 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 dpdk 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 eventtrace 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 prioritycache 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 test 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephfs_mirror 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephsqlite 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 crimson_interrupt 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_onode 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_odata 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_omap 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_tm 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_t 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cleaner 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_epm 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_lba 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_fixedkv_tree 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cache 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_journal 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_device 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_backref 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 alienstore 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mclock 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dedup 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cyanstore 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 ceph_exporter 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 memstore 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 trace 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 ceph_dedup 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: -2/-2 (syslog threshold) 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 99/99 (stderr threshold) 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr:--- pthread ID / name mapping for recent threads --- 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 7f848affd640 / http_manager 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 7f8520fe1640 / http_manager 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 7f854d7fa640 / ms_dispatch 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 7f854e7fc640 / io_context_pool 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 7f854effd640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 7f85557f2640 / ms_dispatch 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 7f8557ff7640 / ms_dispatch 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 7f85597fa640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:21:07.311 INFO:tasks.workunit.client.0.trial046.stderr: 7f8559ffb640 / admin_socket 2026-02-24T22:21:07.312 INFO:tasks.workunit.client.0.trial046.stderr: 7f8571b0d640 / ms_dispatch 2026-02-24T22:21:07.312 INFO:tasks.workunit.client.0.trial046.stderr: 7f857230e640 / msgr-worker-2 2026-02-24T22:21:07.312 INFO:tasks.workunit.client.0.trial046.stderr: 7f8572b0f640 / msgr-worker-1 2026-02-24T22:21:07.312 INFO:tasks.workunit.client.0.trial046.stderr: 7f8573310640 / msgr-worker-0 2026-02-24T22:21:07.312 INFO:tasks.workunit.client.0.trial046.stderr: 7f8573b11640 / io_context_pool 2026-02-24T22:21:07.312 INFO:tasks.workunit.client.0.trial046.stderr: 7f857503ba80 / radosgw-admin 2026-02-24T22:21:07.312 INFO:tasks.workunit.client.0.trial046.stderr: max_recent 500 2026-02-24T22:21:07.312 INFO:tasks.workunit.client.0.trial046.stderr: max_new 1000 2026-02-24T22:21:07.312 INFO:tasks.workunit.client.0.trial046.stderr: log_file /var/lib/ceph/crash/2026-02-24T22:21:07.239932Z_05bc2af0-ef11-449e-af51-cf7c061ce14a/log 2026-02-24T22:21:07.312 INFO:tasks.workunit.client.0.trial046.stderr:--- end dump of recent events --- 2026-02-24T22:21:07.312 INFO:tasks.workunit.client.0.trial046.stderr:Aborted 2026-02-24T22:21:07.312 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket a-bucket 2026-02-24T22:21:07.312 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:21:07.312 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:21:07.312 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.272+0000 7fe803c0da80 4 Realm: () 2026-02-24T22:21:07.312 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.272+0000 7fe803c0da80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:21:07.312 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.272+0000 7fe803c0da80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:21:07.312 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.272+0000 7fe803c0da80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:21:07.312 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.272+0000 7fe803c0da80 20 zonegroup default 2026-02-24T22:21:07.472 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.476+0000 7fe803c0da80 20 rados->read ofs=0 len=0 2026-02-24T22:21:07.473 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.476+0000 7fe803c0da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:07.473 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.476+0000 7fe803c0da80 20 rados->read ofs=0 len=0 2026-02-24T22:21:07.474 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.476+0000 7fe803c0da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:07.474 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.476+0000 7fe803c0da80 20 started sync module instance, tier type = 2026-02-24T22:21:07.474 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.476+0000 7fe803c0da80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:21:07.476 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.480+0000 7fe803c0da80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:21:07.477 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.480+0000 7fe803c0da80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:21:07.477 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.480+0000 7fe803c0da80 5 note: GC not initialized 2026-02-24T22:21:07.477 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.480+0000 7fe7aefe5640 20 reqs_thread_entry: start 2026-02-24T22:21:07.480 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.484+0000 7fe801ee2640 0 -- 10.20.193.46:0/4247208607 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x56007a47cd00 msgr2=0x56007a46ff70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:21:07.692 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.696+0000 7fe8016e1640 0 -- 10.20.193.46:0/4247208607 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x56007a478780 msgr2=0x56007a4734f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:07.718 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.720+0000 7fe803c0da80 20 init_complete bucket index max shards: 1 2026-02-24T22:21:07.718 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.720+0000 7fe803c0da80 20 Filter name: none 2026-02-24T22:21:07.718 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.720+0000 7fe7ad7e2640 20 reqs_thread_entry: start 2026-02-24T22:21:07.718 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.720+0000 7fe803c0da80 20 rados->read ofs=0 len=0 2026-02-24T22:21:07.719 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.720+0000 7fe803c0da80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:21:07.719 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.720+0000 7fe803c0da80 20 rados->read ofs=0 len=0 2026-02-24T22:21:07.720 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.720+0000 7fe803c0da80 20 rados_obj.operate() r=0 bl.length=470 2026-02-24T22:21:07.720 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.720+0000 7fe803c0da80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:21:07.720 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.720+0000 7fe803c0da80 20 rados->read ofs=0 len=0 2026-02-24T22:21:07.720 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.724+0000 7fe803c0da80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:21:07.720 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.724+0000 7fe803c0da80 20 rados->read ofs=0 len=0 2026-02-24T22:21:07.720 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.724+0000 7fe803c0da80 20 rados_obj.operate() r=0 bl.length=470 2026-02-24T22:21:07.720 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.724+0000 7fe803c0da80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:21:07.732 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:21:07.732 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:21:07.739 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin reshardlog list --bucket a-bucket 2026-02-24T22:21:07.753 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:21:07.753 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:21:07.770 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.772+0000 7ff57b82ca80 4 Realm: () 2026-02-24T22:21:07.770 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.772+0000 7ff57b82ca80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:21:07.770 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.772+0000 7ff57b82ca80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:21:07.770 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.772+0000 7ff57b82ca80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:21:07.770 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.772+0000 7ff57b82ca80 20 zonegroup default 2026-02-24T22:21:07.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.772+0000 7ff57b82ca80 20 rados->read ofs=0 len=0 2026-02-24T22:21:07.771 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.776+0000 7ff57b82ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:07.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.776+0000 7ff57b82ca80 20 rados->read ofs=0 len=0 2026-02-24T22:21:07.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.776+0000 7ff57b82ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:07.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.776+0000 7ff57b82ca80 20 started sync module instance, tier type = 2026-02-24T22:21:07.772 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.776+0000 7ff57b82ca80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:21:07.773 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.776+0000 7ff579300640 0 -- 10.20.193.46:0/4001031137 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x55629f71c110 msgr2=0x55629f73c590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:21:07.774 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.776+0000 7ff57b82ca80 20 add_watcher() i=1 2026-02-24T22:21:07.774 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.776+0000 7ff57b82ca80 20 add_watcher() i=3 2026-02-24T22:21:07.775 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.776+0000 7ff57b82ca80 20 add_watcher() i=0 2026-02-24T22:21:07.775 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.776+0000 7ff57b82ca80 20 add_watcher() i=6 2026-02-24T22:21:07.775 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.776+0000 7ff57b82ca80 20 add_watcher() i=5 2026-02-24T22:21:07.775 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.776+0000 7ff57b82ca80 20 add_watcher() i=2 2026-02-24T22:21:07.775 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.776+0000 7ff57b82ca80 20 add_watcher() i=4 2026-02-24T22:21:07.776 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.780+0000 7ff57b82ca80 20 add_watcher() i=7 2026-02-24T22:21:07.776 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.780+0000 7ff57b82ca80 2 all 8 watchers are set, enabling cache 2026-02-24T22:21:07.777 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.780+0000 7ff57b82ca80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:21:07.777 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.780+0000 7ff57b82ca80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:21:07.778 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.780+0000 7ff57b82ca80 5 note: GC not initialized 2026-02-24T22:21:07.778 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.780+0000 7ff5267e4640 20 reqs_thread_entry: start 2026-02-24T22:21:07.778 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:07.780+0000 7ff579b01640 0 -- 10.20.193.46:0/4064768428 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55629f643310 msgr2=0x55629f63ab70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:21:08.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.020+0000 7ff57b82ca80 20 init_complete bucket index max shards: 1 2026-02-24T22:21:08.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.020+0000 7ff57b82ca80 20 Filter name: none 2026-02-24T22:21:08.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.020+0000 7ff524fe1640 20 reqs_thread_entry: start 2026-02-24T22:21:08.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.020+0000 7ff57b82ca80 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2026-02-24T22:21:08.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.020+0000 7ff57b82ca80 20 rados->read ofs=0 len=0 2026-02-24T22:21:08.019 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.020+0000 7ff57b82ca80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:21:08.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.020+0000 7ff57b82ca80 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2026-02-24T22:21:08.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.020+0000 7ff57b82ca80 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2026-02-24T22:21:08.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.020+0000 7ff57b82ca80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : miss 2026-02-24T22:21:08.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.020+0000 7ff57b82ca80 20 rados->read ofs=0 len=0 2026-02-24T22:21:08.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.020+0000 7ff57b82ca80 20 rados_obj.operate() r=0 bl.length=470 2026-02-24T22:21:08.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.020+0000 7ff57b82ca80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:21:08.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.020+0000 7ff57b82ca80 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:21:08.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.020+0000 7ff57b82ca80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:21:08.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.020+0000 7ff57b82ca80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:21:08.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.020+0000 7ff57b82ca80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:21:08.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.020+0000 7ff57b82ca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 2026-02-24T22:21:08.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.020+0000 7ff57b82ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 2026-02-24T22:21:08.021 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.020+0000 7ff57b82ca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1 2026-02-24T22:21:08.021 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.020+0000 7ff57b82ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1 2026-02-24T22:21:08.021 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.020+0000 7ff57b82ca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.2 2026-02-24T22:21:08.021 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.020+0000 7ff57b82ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.2 2026-02-24T22:21:08.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.024+0000 7ff57b82ca80 20 remove_watcher() i=2 2026-02-24T22:21:08.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.024+0000 7ff57b82ca80 2 removed watcher, disabling cache 2026-02-24T22:21:08.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.024+0000 7ff57b82ca80 20 remove_watcher() i=5 2026-02-24T22:21:08.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.024+0000 7ff57b82ca80 20 remove_watcher() i=4 2026-02-24T22:21:08.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.024+0000 7ff57b82ca80 20 remove_watcher() i=3 2026-02-24T22:21:08.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.024+0000 7ff57b82ca80 20 remove_watcher() i=0 2026-02-24T22:21:08.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.024+0000 7ff57b82ca80 20 remove_watcher() i=6 2026-02-24T22:21:08.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.024+0000 7ff57b82ca80 20 remove_watcher() i=1 2026-02-24T22:21:08.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:08.024+0000 7ff57b82ca80 20 remove_watcher() i=7 2026-02-24T22:21:38.058 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket a-bucket 2026-02-24T22:21:38.074 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:21:38.074 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:21:38.092 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.099+0000 7f8981a10a80 4 Realm: () 2026-02-24T22:21:38.092 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.099+0000 7f8981a10a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:21:38.093 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.099+0000 7f8981a10a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:21:38.093 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.099+0000 7f8981a10a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:21:38.093 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.099+0000 7f8981a10a80 20 zonegroup default 2026-02-24T22:21:38.093 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.099+0000 7f8981a10a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:38.094 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.099+0000 7f8981a10a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:38.094 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.099+0000 7f8981a10a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:38.095 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.103+0000 7f8981a10a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:38.095 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.103+0000 7f8981a10a80 20 started sync module instance, tier type = 2026-02-24T22:21:38.095 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.103+0000 7f8981a10a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:21:38.097 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.103+0000 7f8981a10a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:21:38.097 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.103+0000 7f8981a10a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:21:38.097 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.103+0000 7f8981a10a80 5 note: GC not initialized 2026-02-24T22:21:38.098 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.103+0000 7f892cfe1640 20 reqs_thread_entry: start 2026-02-24T22:21:38.101 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.107+0000 7f897affd640 0 -- 10.20.193.46:0/1329147913 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x562c3df84b10 msgr2=0x562c3df84f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:38.313 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.319+0000 7f897b7fe640 0 -- 10.20.193.46:0/1329147913 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x562c3df761d0 msgr2=0x562c3df76640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:38.539 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.547+0000 7f8981a10a80 20 init_complete bucket index max shards: 1 2026-02-24T22:21:38.540 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.547+0000 7f8981a10a80 20 Filter name: none 2026-02-24T22:21:38.540 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.547+0000 7f889effd640 20 reqs_thread_entry: start 2026-02-24T22:21:38.540 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.547+0000 7f8981a10a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:38.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.547+0000 7f8981a10a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:21:38.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.547+0000 7f8981a10a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:38.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.547+0000 7f8981a10a80 20 rados_obj.operate() r=0 bl.length=470 2026-02-24T22:21:38.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.547+0000 7f8981a10a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:21:38.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.547+0000 7f8981a10a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:38.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.547+0000 7f8981a10a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:21:38.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.547+0000 7f8981a10a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:38.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.547+0000 7f8981a10a80 20 rados_obj.operate() r=0 bl.length=470 2026-02-24T22:21:38.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.547+0000 7f8981a10a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:21:38.554 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:21:38.554 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:21:38.567 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin reshardlog list --bucket a-bucket 2026-02-24T22:21:38.583 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:21:38.583 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:21:38.600 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.607+0000 7fe51340ca80 4 Realm: () 2026-02-24T22:21:38.600 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.607+0000 7fe51340ca80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:21:38.600 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.607+0000 7fe51340ca80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:21:38.600 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.607+0000 7fe51340ca80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:21:38.600 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.607+0000 7fe51340ca80 20 zonegroup default 2026-02-24T22:21:38.601 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.607+0000 7fe51340ca80 20 rados->read ofs=0 len=0 2026-02-24T22:21:38.602 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.607+0000 7fe51340ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:38.602 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.607+0000 7fe51340ca80 20 rados->read ofs=0 len=0 2026-02-24T22:21:38.602 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.607+0000 7fe51340ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:38.602 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.607+0000 7fe51340ca80 20 started sync module instance, tier type = 2026-02-24T22:21:38.602 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.607+0000 7fe51340ca80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:21:38.603 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.611+0000 7fe510ee0640 0 -- 10.20.193.46:0/3608189209 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55805381a610 msgr2=0x55805383aa50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:38.604 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.611+0000 7fe51340ca80 20 add_watcher() i=1 2026-02-24T22:21:38.604 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.611+0000 7fe51340ca80 20 add_watcher() i=3 2026-02-24T22:21:38.604 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.611+0000 7fe51340ca80 20 add_watcher() i=0 2026-02-24T22:21:38.604 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.611+0000 7fe51340ca80 20 add_watcher() i=5 2026-02-24T22:21:38.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.611+0000 7fe51340ca80 20 add_watcher() i=4 2026-02-24T22:21:38.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.611+0000 7fe51340ca80 20 add_watcher() i=2 2026-02-24T22:21:38.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.611+0000 7fe51340ca80 20 add_watcher() i=6 2026-02-24T22:21:38.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.611+0000 7fe51340ca80 20 add_watcher() i=7 2026-02-24T22:21:38.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.611+0000 7fe51340ca80 2 all 8 watchers are set, enabling cache 2026-02-24T22:21:38.607 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.615+0000 7fe51340ca80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:21:38.607 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.615+0000 7fe51340ca80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:21:38.607 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.615+0000 7fe51340ca80 5 note: GC not initialized 2026-02-24T22:21:38.607 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.615+0000 7fe4bdfe3640 20 reqs_thread_entry: start 2026-02-24T22:21:38.607 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.615+0000 7fe5116e1640 0 -- 10.20.193.46:0/1513973745 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x558053952580 msgr2=0x558053949de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:38.849 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.855+0000 7fe51340ca80 20 init_complete bucket index max shards: 1 2026-02-24T22:21:38.849 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.855+0000 7fe51340ca80 20 Filter name: none 2026-02-24T22:21:38.849 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.855+0000 7fe417fff640 20 reqs_thread_entry: start 2026-02-24T22:21:38.849 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.855+0000 7fe51340ca80 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2026-02-24T22:21:38.849 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.855+0000 7fe51340ca80 20 rados->read ofs=0 len=0 2026-02-24T22:21:38.850 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.855+0000 7fe51340ca80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:21:38.851 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.855+0000 7fe51340ca80 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2026-02-24T22:21:38.851 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.855+0000 7fe51340ca80 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2026-02-24T22:21:38.851 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.855+0000 7fe51340ca80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : miss 2026-02-24T22:21:38.851 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.855+0000 7fe51340ca80 20 rados->read ofs=0 len=0 2026-02-24T22:21:38.851 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.855+0000 7fe51340ca80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:21:38.851 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.855+0000 7fe51340ca80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:21:38.851 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.855+0000 7fe51340ca80 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:21:38.851 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.855+0000 7fe51340ca80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:21:38.851 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.855+0000 7fe51340ca80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:21:38.852 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.855+0000 7fe51340ca80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:21:38.852 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.855+0000 7fe51340ca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 2026-02-24T22:21:38.852 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.855+0000 7fe51340ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 2026-02-24T22:21:38.852 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.855+0000 7fe51340ca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1 2026-02-24T22:21:38.852 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.855+0000 7fe51340ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1 2026-02-24T22:21:38.852 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.855+0000 7fe51340ca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.2 2026-02-24T22:21:38.852 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.855+0000 7fe51340ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.2 2026-02-24T22:21:38.854 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.859+0000 7fe51340ca80 20 remove_watcher() i=1 2026-02-24T22:21:38.854 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.859+0000 7fe51340ca80 2 removed watcher, disabling cache 2026-02-24T22:21:38.854 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.859+0000 7fe51340ca80 20 remove_watcher() i=2 2026-02-24T22:21:38.854 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.859+0000 7fe51340ca80 20 remove_watcher() i=5 2026-02-24T22:21:38.854 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.859+0000 7fe51340ca80 20 remove_watcher() i=3 2026-02-24T22:21:38.854 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.859+0000 7fe51340ca80 20 remove_watcher() i=6 2026-02-24T22:21:38.854 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.859+0000 7fe51340ca80 20 remove_watcher() i=4 2026-02-24T22:21:38.854 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.859+0000 7fe51340ca80 20 remove_watcher() i=0 2026-02-24T22:21:38.855 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.859+0000 7fe51340ca80 20 remove_watcher() i=7 2026-02-24T22:21:38.860 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket a-bucket 2026-02-24T22:21:38.874 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:21:38.874 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:21:38.893 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.899+0000 7f2263a8ea80 4 Realm: () 2026-02-24T22:21:38.893 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.899+0000 7f2263a8ea80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:21:38.893 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.899+0000 7f2263a8ea80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:21:38.893 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.899+0000 7f2263a8ea80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:21:38.893 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.899+0000 7f2263a8ea80 20 zonegroup default 2026-02-24T22:21:38.893 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.899+0000 7f2263a8ea80 20 rados->read ofs=0 len=0 2026-02-24T22:21:38.895 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.903+0000 7f2263a8ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:38.895 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.903+0000 7f2263a8ea80 20 rados->read ofs=0 len=0 2026-02-24T22:21:38.895 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.903+0000 7f2263a8ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:38.896 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.903+0000 7f2263a8ea80 20 started sync module instance, tier type = 2026-02-24T22:21:38.896 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.903+0000 7f2263a8ea80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:21:38.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.903+0000 7f2263a8ea80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:21:38.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.903+0000 7f2263a8ea80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:21:38.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.903+0000 7f2263a8ea80 5 note: GC not initialized 2026-02-24T22:21:38.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.903+0000 7f220efe5640 20 reqs_thread_entry: start 2026-02-24T22:21:38.899 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:38.907+0000 7f2260d61640 0 -- 10.20.193.46:0/2349206310 >> [v2:10.20.193.46:6826/3065971976,v1:10.20.193.46:6827/3065971976] conn(0x565328acd980 msgr2=0x565328acdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:39.111 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.119+0000 7f2260d61640 0 -- 10.20.193.46:0/2349206310 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x565328ad0c00 msgr2=0x565328ab9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:39.345 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.351+0000 7f2263a8ea80 20 init_complete bucket index max shards: 1 2026-02-24T22:21:39.345 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.351+0000 7f2263a8ea80 20 Filter name: none 2026-02-24T22:21:39.346 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.351+0000 7f220d7e2640 20 reqs_thread_entry: start 2026-02-24T22:21:39.346 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.351+0000 7f2263a8ea80 20 rados->read ofs=0 len=0 2026-02-24T22:21:39.347 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.351+0000 7f2263a8ea80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:21:39.347 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.351+0000 7f2263a8ea80 20 rados->read ofs=0 len=0 2026-02-24T22:21:39.347 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.351+0000 7f2263a8ea80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:21:39.347 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.351+0000 7f2263a8ea80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:21:39.347 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.351+0000 7f2263a8ea80 20 rados->read ofs=0 len=0 2026-02-24T22:21:39.347 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.351+0000 7f2263a8ea80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:21:39.347 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.351+0000 7f2263a8ea80 20 rados->read ofs=0 len=0 2026-02-24T22:21:39.348 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.351+0000 7f2263a8ea80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:21:39.348 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.351+0000 7f2263a8ea80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:21:39.357 INFO:tasks.workunit.client.0.trial046.stderr: test: duplicated entries should be purged before reshard 2026-02-24T22:21:39.358 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket a-bucket 2026-02-24T22:21:39.372 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:21:39.372 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:21:39.594 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.599+0000 7efc6a51fa80 4 Realm: () 2026-02-24T22:21:39.594 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.599+0000 7efc6a51fa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:21:39.594 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.599+0000 7efc6a51fa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:21:39.594 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.599+0000 7efc6a51fa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:21:39.594 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.599+0000 7efc6a51fa80 20 zonegroup default 2026-02-24T22:21:39.594 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.599+0000 7efc6a51fa80 20 rados->read ofs=0 len=0 2026-02-24T22:21:39.595 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.603+0000 7efc6a51fa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:39.595 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.603+0000 7efc6a51fa80 20 rados->read ofs=0 len=0 2026-02-24T22:21:39.596 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.603+0000 7efc6a51fa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:39.596 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.603+0000 7efc6a51fa80 20 started sync module instance, tier type = 2026-02-24T22:21:39.596 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.603+0000 7efc6a51fa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:21:39.598 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.603+0000 7efc6a51fa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:21:39.598 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.603+0000 7efc6a51fa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:21:39.598 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.603+0000 7efc6a51fa80 5 note: GC not initialized 2026-02-24T22:21:39.598 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.603+0000 7efc157e2640 20 reqs_thread_entry: start 2026-02-24T22:21:39.601 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.607+0000 7efc63fff640 0 -- 10.20.193.46:0/199282318 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x5654d4d37370 msgr2=0x5654d4d2a5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:39.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.819+0000 7efc637fe640 0 -- 10.20.193.46:0/199282318 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5654d4d32df0 msgr2=0x5654d4d2db60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:39.838 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.843+0000 7efc6a51fa80 20 init_complete bucket index max shards: 1 2026-02-24T22:21:39.838 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.843+0000 7efc6a51fa80 20 Filter name: none 2026-02-24T22:21:39.838 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.843+0000 7efb7f7fe640 20 reqs_thread_entry: start 2026-02-24T22:21:39.839 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.843+0000 7efc6a51fa80 20 rados->read ofs=0 len=0 2026-02-24T22:21:39.840 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.847+0000 7efc6a51fa80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:21:39.840 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.847+0000 7efc6a51fa80 20 rados->read ofs=0 len=0 2026-02-24T22:21:39.840 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.847+0000 7efc6a51fa80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:21:39.840 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.847+0000 7efc6a51fa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:21:39.840 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.847+0000 7efc6a51fa80 20 rados->read ofs=0 len=0 2026-02-24T22:21:39.840 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.847+0000 7efc6a51fa80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:21:39.840 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.847+0000 7efc6a51fa80 20 rados->read ofs=0 len=0 2026-02-24T22:21:39.840 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.847+0000 7efc6a51fa80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:21:39.840 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.847+0000 7efc6a51fa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:21:39.851 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:39.851 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:21:39.851 INFO:tasks.workunit.client.0.trial046.stderr:bucket: a-bucket id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 num_objects: 10 size_kb: 1 num_shards: 3 2026-02-24T22:21:39.851 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:39.851 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket a-bucket --num-shards 4 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at do_reshard 2026-02-24T22:21:39.868 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:21:39.868 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:21:39.885 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.891+0000 7f803667da80 4 Realm: () 2026-02-24T22:21:39.885 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.891+0000 7f803667da80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:21:39.885 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.891+0000 7f803667da80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:21:39.885 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.891+0000 7f803667da80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:21:39.885 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.891+0000 7f803667da80 20 zonegroup default 2026-02-24T22:21:39.886 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.891+0000 7f803667da80 20 rados->read ofs=0 len=0 2026-02-24T22:21:39.886 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.891+0000 7f803667da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:39.886 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.891+0000 7f803667da80 20 rados->read ofs=0 len=0 2026-02-24T22:21:39.887 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.891+0000 7f803667da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:39.887 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.891+0000 7f803667da80 20 started sync module instance, tier type = 2026-02-24T22:21:39.887 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.891+0000 7f803667da80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:21:39.888 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.895+0000 7f8034952640 0 -- 10.20.193.46:0/1916772995 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x555cf989bd60 msgr2=0x555cf98bc140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:39.888 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.895+0000 7f803667da80 20 add_watcher() i=1 2026-02-24T22:21:39.889 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.895+0000 7f803667da80 20 add_watcher() i=6 2026-02-24T22:21:39.889 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.895+0000 7f803667da80 20 add_watcher() i=5 2026-02-24T22:21:39.889 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.895+0000 7f803667da80 20 add_watcher() i=2 2026-02-24T22:21:39.889 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.895+0000 7f803667da80 20 add_watcher() i=0 2026-02-24T22:21:39.889 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.895+0000 7f803667da80 20 add_watcher() i=3 2026-02-24T22:21:39.890 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.895+0000 7f803667da80 20 add_watcher() i=4 2026-02-24T22:21:39.890 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.895+0000 7f803667da80 20 add_watcher() i=7 2026-02-24T22:21:39.890 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.895+0000 7f803667da80 2 all 8 watchers are set, enabling cache 2026-02-24T22:21:39.891 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.899+0000 7f803667da80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:21:39.891 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.899+0000 7f803667da80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:21:39.891 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.899+0000 7f803667da80 5 note: GC not initialized 2026-02-24T22:21:39.891 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.899+0000 7f7fe1fdb640 20 reqs_thread_entry: start 2026-02-24T22:21:39.892 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:39.899+0000 7f8034952640 0 -- 10.20.193.46:0/2371875498 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x555cf99918b0 msgr2=0x555cf998c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:21:40.133 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.139+0000 7f803667da80 20 init_complete bucket index max shards: 1 2026-02-24T22:21:40.133 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.139+0000 7f803667da80 20 Filter name: none 2026-02-24T22:21:40.133 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.139+0000 7f7f3bfff640 20 reqs_thread_entry: start 2026-02-24T22:21:40.133 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.139+0000 7f803667da80 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2026-02-24T22:21:40.133 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.139+0000 7f803667da80 20 rados->read ofs=0 len=0 2026-02-24T22:21:40.134 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.139+0000 7f803667da80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:21:40.134 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.139+0000 7f803667da80 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2026-02-24T22:21:40.134 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.139+0000 7f803667da80 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2026-02-24T22:21:40.134 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.139+0000 7f803667da80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : miss 2026-02-24T22:21:40.134 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.139+0000 7f803667da80 20 rados->read ofs=0 len=0 2026-02-24T22:21:40.135 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.139+0000 7f803667da80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:21:40.135 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.139+0000 7f803667da80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:21:40.135 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.139+0000 7f803667da80 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:21:40.135 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.139+0000 7f803667da80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:21:40.135 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.139+0000 7f803667da80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:21:40.135 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.139+0000 7f803667da80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:21:40.139 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.143+0000 7f803667da80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:21:40.139 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.143+0000 7f803667da80 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:21:40.139 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.143+0000 7f803667da80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:21:40.139 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.143+0000 7f803667da80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518, ofs0, ns] 2026-02-24T22:21:40.139 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.143+0000 7f8002ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959945 cookie 93857813528032 notifier 7943 bl.length()=929 2026-02-24T22:21:40.139 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.143+0000 7f8002ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:21:40.139 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.143+0000 7f8002ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:21:40.139 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.143+0000 7f8002ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:21:40.140 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.147+0000 7f803667da80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_sources= new_sources= 2026-02-24T22:21:40.140 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.147+0000 7f803667da80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential sources added=[] removed=[] 2026-02-24T22:21:40.140 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.147+0000 7f803667da80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_dests= new_dests= 2026-02-24T22:21:40.140 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.147+0000 7f803667da80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential dests added=[] removed=[] 2026-02-24T22:21:40.141 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f803667da80 time 2026-02-24T22:21:40.149765+0000 2026-02-24T22:21:40.141 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:21:40.142 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:21:40.142 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f8039abfeb1] 2026-02-24T22:21:40.142 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x555ceb63eb9f] 2026-02-24T22:21:40.142 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x16a) [0x555cebdaa26a] 2026-02-24T22:21:40.142 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:21:40.142 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f8038dbfd90] 2026-02-24T22:21:40.142 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:21:40.142 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:21:40.143 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.147+0000 7f803667da80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f803667da80 time 2026-02-24T22:21:40.149765+0000 2026-02-24T22:21:40.143 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:21:40.143 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.143 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:21:40.143 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f8039abfeb1] 2026-02-24T22:21:40.143 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x555ceb63eb9f] 2026-02-24T22:21:40.143 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x16a) [0x555cebdaa26a] 2026-02-24T22:21:40.143 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:21:40.143 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f8038dbfd90] 2026-02-24T22:21:40.143 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:21:40.143 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:21:40.144 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.144 INFO:tasks.workunit.client.0.trial046.stderr:*** Caught signal (Aborted) ** 2026-02-24T22:21:40.144 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f803667da80 thread_name:radosgw-admin 2026-02-24T22:21:40.164 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:21:40.164 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f8038dd8520] 2026-02-24T22:21:40.165 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:21:40.165 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:21:40.165 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:21:40.165 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f8039abff10] 2026-02-24T22:21:40.165 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x555ceb63eb9f] 2026-02-24T22:21:40.165 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x16a) [0x555cebdaa26a] 2026-02-24T22:21:40.165 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:21:40.165 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f8038dbfd90] 2026-02-24T22:21:40.165 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:21:40.165 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:21:40.165 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.151+0000 7f803667da80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:21:40.166 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f803667da80 thread_name:radosgw-admin 2026-02-24T22:21:40.166 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.166 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:21:40.166 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f8038dd8520] 2026-02-24T22:21:40.166 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:21:40.166 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:21:40.166 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:21:40.166 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f8039abff10] 2026-02-24T22:21:40.166 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x555ceb63eb9f] 2026-02-24T22:21:40.166 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x16a) [0x555cebdaa26a] 2026-02-24T22:21:40.166 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:21:40.166 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f8038dbfd90] 2026-02-24T22:21:40.167 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:21:40.167 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:21:40.167 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:21:40.167 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.167 INFO:tasks.workunit.client.0.trial046.stderr:--- begin dump of recent events --- 2026-02-24T22:21:40.167 INFO:tasks.workunit.client.0.trial046.stderr: -499> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.167 INFO:tasks.workunit.client.0.trial046.stderr: -498> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.167 INFO:tasks.workunit.client.0.trial046.stderr: -497> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient(hunting): _renew_subs 2026-02-24T22:21:40.167 INFO:tasks.workunit.client.0.trial046.stderr: -496> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient(hunting): authenticate will time out at 2069.986078s 2026-02-24T22:21:40.167 INFO:tasks.workunit.client.0.trial046.stderr: -495> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient(hunting): get_auth_request con 0x555cf94e3b00 auth_method 0 2026-02-24T22:21:40.167 INFO:tasks.workunit.client.0.trial046.stderr: -494> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:40.167 INFO:tasks.workunit.client.0.trial046.stderr: -493> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:40.168 INFO:tasks.workunit.client.0.trial046.stderr: -492> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:40.168 INFO:tasks.workunit.client.0.trial046.stderr: -491> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): get_auth_request con 0x555cf94e6cf0 auth_method 0 2026-02-24T22:21:40.168 INFO:tasks.workunit.client.0.trial046.stderr: -490> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:40.168 INFO:tasks.workunit.client.0.trial046.stderr: -489> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:40.168 INFO:tasks.workunit.client.0.trial046.stderr: -488> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:40.168 INFO:tasks.workunit.client.0.trial046.stderr: -487> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:40.168 INFO:tasks.workunit.client.0.trial046.stderr: -486> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:40.168 INFO:tasks.workunit.client.0.trial046.stderr: -485> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:40.168 INFO:tasks.workunit.client.0.trial046.stderr: -484> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:40.168 INFO:tasks.workunit.client.0.trial046.stderr: -483> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:40.168 INFO:tasks.workunit.client.0.trial046.stderr: -482> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:40.168 INFO:tasks.workunit.client.0.trial046.stderr: -481> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient(hunting): handle_auth_done global_id 8505 payload 274 2026-02-24T22:21:40.168 INFO:tasks.workunit.client.0.trial046.stderr: -480> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient: _finish_hunting 0 2026-02-24T22:21:40.169 INFO:tasks.workunit.client.0.trial046.stderr: -479> 2026-02-24T22:21:39.879+0000 7f8027fff640 1 monclient: found mon.noname-a 2026-02-24T22:21:40.169 INFO:tasks.workunit.client.0.trial046.stderr: -478> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:21:40.169 INFO:tasks.workunit.client.0.trial046.stderr: -477> 2026-02-24T22:21:39.879+0000 7f80267fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:40.169 INFO:tasks.workunit.client.0.trial046.stderr: -476> 2026-02-24T22:21:39.879+0000 7f80267fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:21:40.169 INFO:tasks.workunit.client.0.trial046.stderr: -475> 2026-02-24T22:21:39.879+0000 7f80267fc640 10 monclient: dump: 2026-02-24T22:21:40.169 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:40.169 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:40.169 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.169 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.169 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:40.169 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.169 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:40.169 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:40.170 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:40.170 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.170 INFO:tasks.workunit.client.0.trial046.stderr: -474> 2026-02-24T22:21:39.879+0000 7f80267fc640 10 monclient: _finish_auth 0 2026-02-24T22:21:40.170 INFO:tasks.workunit.client.0.trial046.stderr: -473> 2026-02-24T22:21:39.879+0000 7f80267fc640 10 monclient: _check_auth_tickets 2026-02-24T22:21:40.170 INFO:tasks.workunit.client.0.trial046.stderr: -472> 2026-02-24T22:21:39.879+0000 7f80267fc640 10 monclient: handle_config config(0 keys) 2026-02-24T22:21:40.170 INFO:tasks.workunit.client.0.trial046.stderr: -471> 2026-02-24T22:21:39.879+0000 7f80267fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:40.170 INFO:tasks.workunit.client.0.trial046.stderr: -470> 2026-02-24T22:21:39.879+0000 7f80267fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:21:40.170 INFO:tasks.workunit.client.0.trial046.stderr: -469> 2026-02-24T22:21:39.879+0000 7f80267fc640 10 monclient: dump: 2026-02-24T22:21:40.170 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:40.170 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:40.170 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.170 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.170 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:40.171 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.171 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:40.171 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:40.171 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:40.171 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.171 INFO:tasks.workunit.client.0.trial046.stderr: -468> 2026-02-24T22:21:39.879+0000 7f803667da80 5 monclient: authenticate success, global_id 8505 2026-02-24T22:21:40.171 INFO:tasks.workunit.client.0.trial046.stderr: -467> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient: get_monmap_and_config success 2026-02-24T22:21:40.171 INFO:tasks.workunit.client.0.trial046.stderr: -466> 2026-02-24T22:21:39.879+0000 7f803667da80 4 set_mon_vals no callback set 2026-02-24T22:21:40.171 INFO:tasks.workunit.client.0.trial046.stderr: -465> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient: shutdown 2026-02-24T22:21:40.171 INFO:tasks.workunit.client.0.trial046.stderr: -464> 2026-02-24T22:21:39.879+0000 7f803667da80 5 asok(0x555cf93acf00) unregister_commands rotate-key 2026-02-24T22:21:40.171 INFO:tasks.workunit.client.0.trial046.stderr: -463> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient: build_initial_monmap 2026-02-24T22:21:40.171 INFO:tasks.workunit.client.0.trial046.stderr: -462> 2026-02-24T22:21:39.879+0000 7f803667da80 1 build_initial for_mkfs: 0 2026-02-24T22:21:40.171 INFO:tasks.workunit.client.0.trial046.stderr: -461> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient: monmap: 2026-02-24T22:21:40.172 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:21:40.172 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:21:40.172 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:21:40.172 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:21:40.172 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:21:40.172 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.172 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:21:40.172 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:21:40.172 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:21:40.172 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.172 INFO:tasks.workunit.client.0.trial046.stderr: -460> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding auth protocol: cephx 2026-02-24T22:21:40.172 INFO:tasks.workunit.client.0.trial046.stderr: -459> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding auth protocol: cephx 2026-02-24T22:21:40.172 INFO:tasks.workunit.client.0.trial046.stderr: -458> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding auth protocol: cephx 2026-02-24T22:21:40.172 INFO:tasks.workunit.client.0.trial046.stderr: -457> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding con mode: secure 2026-02-24T22:21:40.173 INFO:tasks.workunit.client.0.trial046.stderr: -456> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding con mode: crc 2026-02-24T22:21:40.173 INFO:tasks.workunit.client.0.trial046.stderr: -455> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding con mode: secure 2026-02-24T22:21:40.173 INFO:tasks.workunit.client.0.trial046.stderr: -454> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding con mode: crc 2026-02-24T22:21:40.173 INFO:tasks.workunit.client.0.trial046.stderr: -453> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding con mode: secure 2026-02-24T22:21:40.173 INFO:tasks.workunit.client.0.trial046.stderr: -452> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding con mode: crc 2026-02-24T22:21:40.173 INFO:tasks.workunit.client.0.trial046.stderr: -451> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding con mode: crc 2026-02-24T22:21:40.173 INFO:tasks.workunit.client.0.trial046.stderr: -450> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding con mode: secure 2026-02-24T22:21:40.173 INFO:tasks.workunit.client.0.trial046.stderr: -449> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding con mode: crc 2026-02-24T22:21:40.173 INFO:tasks.workunit.client.0.trial046.stderr: -448> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding con mode: secure 2026-02-24T22:21:40.173 INFO:tasks.workunit.client.0.trial046.stderr: -447> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding con mode: crc 2026-02-24T22:21:40.173 INFO:tasks.workunit.client.0.trial046.stderr: -446> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding con mode: secure 2026-02-24T22:21:40.173 INFO:tasks.workunit.client.0.trial046.stderr: -445> 2026-02-24T22:21:39.879+0000 7f803667da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:40.173 INFO:tasks.workunit.client.0.trial046.stderr: -444> 2026-02-24T22:21:39.879+0000 7f803667da80 1 librados: starting msgr at 2026-02-24T22:21:40.173 INFO:tasks.workunit.client.0.trial046.stderr: -443> 2026-02-24T22:21:39.879+0000 7f803667da80 1 librados: starting objecter 2026-02-24T22:21:40.174 INFO:tasks.workunit.client.0.trial046.stderr: -442> 2026-02-24T22:21:39.879+0000 7f803667da80 5 asok(0x555cf93acf00) register_command objecter_requests hook 0x555cf940de40 2026-02-24T22:21:40.174 INFO:tasks.workunit.client.0.trial046.stderr: -441> 2026-02-24T22:21:39.879+0000 7f803667da80 1 librados: setting wanted keys 2026-02-24T22:21:40.174 INFO:tasks.workunit.client.0.trial046.stderr: -440> 2026-02-24T22:21:39.879+0000 7f803667da80 1 librados: calling monclient init 2026-02-24T22:21:40.174 INFO:tasks.workunit.client.0.trial046.stderr: -439> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient: init 2026-02-24T22:21:40.174 INFO:tasks.workunit.client.0.trial046.stderr: -438> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding auth protocol: cephx 2026-02-24T22:21:40.174 INFO:tasks.workunit.client.0.trial046.stderr: -437> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding auth protocol: cephx 2026-02-24T22:21:40.174 INFO:tasks.workunit.client.0.trial046.stderr: -436> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding auth protocol: cephx 2026-02-24T22:21:40.174 INFO:tasks.workunit.client.0.trial046.stderr: -435> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding con mode: secure 2026-02-24T22:21:40.174 INFO:tasks.workunit.client.0.trial046.stderr: -434> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding con mode: crc 2026-02-24T22:21:40.174 INFO:tasks.workunit.client.0.trial046.stderr: -433> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding con mode: secure 2026-02-24T22:21:40.174 INFO:tasks.workunit.client.0.trial046.stderr: -432> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding con mode: crc 2026-02-24T22:21:40.174 INFO:tasks.workunit.client.0.trial046.stderr: -431> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding con mode: secure 2026-02-24T22:21:40.174 INFO:tasks.workunit.client.0.trial046.stderr: -430> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding con mode: crc 2026-02-24T22:21:40.174 INFO:tasks.workunit.client.0.trial046.stderr: -429> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding con mode: crc 2026-02-24T22:21:40.174 INFO:tasks.workunit.client.0.trial046.stderr: -428> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding con mode: secure 2026-02-24T22:21:40.175 INFO:tasks.workunit.client.0.trial046.stderr: -427> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding con mode: crc 2026-02-24T22:21:40.175 INFO:tasks.workunit.client.0.trial046.stderr: -426> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding con mode: secure 2026-02-24T22:21:40.175 INFO:tasks.workunit.client.0.trial046.stderr: -425> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding con mode: crc 2026-02-24T22:21:40.175 INFO:tasks.workunit.client.0.trial046.stderr: -424> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding con mode: secure 2026-02-24T22:21:40.175 INFO:tasks.workunit.client.0.trial046.stderr: -423> 2026-02-24T22:21:39.879+0000 7f803667da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:40.175 INFO:tasks.workunit.client.0.trial046.stderr: -422> 2026-02-24T22:21:39.879+0000 7f803667da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:40.175 INFO:tasks.workunit.client.0.trial046.stderr: -421> 2026-02-24T22:21:39.879+0000 7f803667da80 5 asok(0x555cf93acf00) register_command rotate-key hook 0x555cf94ad158 2026-02-24T22:21:40.175 INFO:tasks.workunit.client.0.trial046.stderr: -420> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient: _reopen_session rank -1 2026-02-24T22:21:40.175 INFO:tasks.workunit.client.0.trial046.stderr: -419> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T22:21:40.175 INFO:tasks.workunit.client.0.trial046.stderr: -418> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient(hunting): picked mon.noname-c con 0x555cf943afe0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:21:40.175 INFO:tasks.workunit.client.0.trial046.stderr: -417> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient(hunting): picked mon.noname-b con 0x555cf94e3b00 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:21:40.175 INFO:tasks.workunit.client.0.trial046.stderr: -416> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient(hunting): picked mon.noname-a con 0x555cf94e6cf0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:21:40.175 INFO:tasks.workunit.client.0.trial046.stderr: -415> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.175 INFO:tasks.workunit.client.0.trial046.stderr: -414> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.176 INFO:tasks.workunit.client.0.trial046.stderr: -413> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.176 INFO:tasks.workunit.client.0.trial046.stderr: -412> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient(hunting): _renew_subs 2026-02-24T22:21:40.176 INFO:tasks.workunit.client.0.trial046.stderr: -411> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient(hunting): authenticate will time out at 2069.987415s 2026-02-24T22:21:40.176 INFO:tasks.workunit.client.0.trial046.stderr: -410> 2026-02-24T22:21:39.879+0000 7f802ffff640 10 monclient(hunting): get_auth_request con 0x555cf943afe0 auth_method 0 2026-02-24T22:21:40.176 INFO:tasks.workunit.client.0.trial046.stderr: -409> 2026-02-24T22:21:39.879+0000 7f802ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:40.176 INFO:tasks.workunit.client.0.trial046.stderr: -408> 2026-02-24T22:21:39.879+0000 7f802ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:40.176 INFO:tasks.workunit.client.0.trial046.stderr: -407> 2026-02-24T22:21:39.879+0000 7f802ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:40.176 INFO:tasks.workunit.client.0.trial046.stderr: -406> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): get_auth_request con 0x555cf94e6cf0 auth_method 0 2026-02-24T22:21:40.176 INFO:tasks.workunit.client.0.trial046.stderr: -405> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:40.176 INFO:tasks.workunit.client.0.trial046.stderr: -404> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:40.176 INFO:tasks.workunit.client.0.trial046.stderr: -403> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:40.176 INFO:tasks.workunit.client.0.trial046.stderr: -402> 2026-02-24T22:21:39.879+0000 7f802ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:40.176 INFO:tasks.workunit.client.0.trial046.stderr: -401> 2026-02-24T22:21:39.879+0000 7f802ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:40.176 INFO:tasks.workunit.client.0.trial046.stderr: -400> 2026-02-24T22:21:39.879+0000 7f802ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:40.176 INFO:tasks.workunit.client.0.trial046.stderr: -399> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient(hunting): get_auth_request con 0x555cf94e3b00 auth_method 0 2026-02-24T22:21:40.176 INFO:tasks.workunit.client.0.trial046.stderr: -398> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:40.177 INFO:tasks.workunit.client.0.trial046.stderr: -397> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:40.177 INFO:tasks.workunit.client.0.trial046.stderr: -396> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:40.177 INFO:tasks.workunit.client.0.trial046.stderr: -395> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:40.177 INFO:tasks.workunit.client.0.trial046.stderr: -394> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:40.177 INFO:tasks.workunit.client.0.trial046.stderr: -393> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:40.177 INFO:tasks.workunit.client.0.trial046.stderr: -392> 2026-02-24T22:21:39.879+0000 7f802ffff640 10 monclient(hunting): handle_auth_done global_id 7934 payload 824 2026-02-24T22:21:40.177 INFO:tasks.workunit.client.0.trial046.stderr: -391> 2026-02-24T22:21:39.879+0000 7f802ffff640 10 monclient: _finish_hunting 0 2026-02-24T22:21:40.177 INFO:tasks.workunit.client.0.trial046.stderr: -390> 2026-02-24T22:21:39.879+0000 7f802ffff640 1 monclient: found mon.noname-c 2026-02-24T22:21:40.177 INFO:tasks.workunit.client.0.trial046.stderr: -389> 2026-02-24T22:21:39.879+0000 7f802ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:21:40.177 INFO:tasks.workunit.client.0.trial046.stderr: -388> 2026-02-24T22:21:39.879+0000 7f8024ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:40.177 INFO:tasks.workunit.client.0.trial046.stderr: -387> 2026-02-24T22:21:39.879+0000 7f8024ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:21:40.177 INFO:tasks.workunit.client.0.trial046.stderr: -386> 2026-02-24T22:21:39.879+0000 7f8024ff9640 10 monclient: dump: 2026-02-24T22:21:40.177 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:40.177 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:40.177 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.178 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.178 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:40.178 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.178 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:40.178 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:40.178 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:40.178 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.178 INFO:tasks.workunit.client.0.trial046.stderr: -385> 2026-02-24T22:21:39.879+0000 7f8024ff9640 10 monclient: _finish_auth 0 2026-02-24T22:21:40.178 INFO:tasks.workunit.client.0.trial046.stderr: -384> 2026-02-24T22:21:39.879+0000 7f8024ff9640 10 monclient: _check_auth_tickets 2026-02-24T22:21:40.178 INFO:tasks.workunit.client.0.trial046.stderr: -383> 2026-02-24T22:21:39.879+0000 7f8024ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T22:21:40.178 INFO:tasks.workunit.client.0.trial046.stderr: -382> 2026-02-24T22:21:39.879+0000 7f8024ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:40.178 INFO:tasks.workunit.client.0.trial046.stderr: -381> 2026-02-24T22:21:39.879+0000 7f8024ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:21:40.178 INFO:tasks.workunit.client.0.trial046.stderr: -380> 2026-02-24T22:21:39.879+0000 7f8024ff9640 10 monclient: dump: 2026-02-24T22:21:40.178 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:40.179 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:40.179 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.179 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.179 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:40.179 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.179 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:40.179 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:40.179 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:40.179 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.179 INFO:tasks.workunit.client.0.trial046.stderr: -379> 2026-02-24T22:21:39.879+0000 7f80267fc640 4 set_mon_vals no callback set 2026-02-24T22:21:40.179 INFO:tasks.workunit.client.0.trial046.stderr: -378> 2026-02-24T22:21:39.879+0000 7f803667da80 5 monclient: authenticate success, global_id 7934 2026-02-24T22:21:40.179 INFO:tasks.workunit.client.0.trial046.stderr: -377> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient: _renew_subs 2026-02-24T22:21:40.179 INFO:tasks.workunit.client.0.trial046.stderr: -376> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:21:40.179 INFO:tasks.workunit.client.0.trial046.stderr: -375> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient: _renew_subs 2026-02-24T22:21:40.179 INFO:tasks.workunit.client.0.trial046.stderr: -374> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:21:40.179 INFO:tasks.workunit.client.0.trial046.stderr: -373> 2026-02-24T22:21:39.879+0000 7f803667da80 1 librados: init done 2026-02-24T22:21:40.180 INFO:tasks.workunit.client.0.trial046.stderr: -372> 2026-02-24T22:21:39.883+0000 7f8024ff9640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:21:40.180 INFO:tasks.workunit.client.0.trial046.stderr: -371> 2026-02-24T22:21:39.883+0000 7f8024ff9640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:21:40.180 INFO:tasks.workunit.client.0.trial046.stderr: -370> 2026-02-24T22:21:39.883+0000 7f8024ff9640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:21:40.180 INFO:tasks.workunit.client.0.trial046.stderr: -369> 2026-02-24T22:21:39.883+0000 7f8027fff640 10 monclient: get_auth_request con 0x7f80100707b0 auth_method 0 2026-02-24T22:21:40.180 INFO:tasks.workunit.client.0.trial046.stderr: -368> 2026-02-24T22:21:39.883+0000 7f8034952640 10 monclient: get_auth_request con 0x555cf953a7a0 auth_method 0 2026-02-24T22:21:40.180 INFO:tasks.workunit.client.0.trial046.stderr: -367> 2026-02-24T22:21:39.883+0000 7f802ffff640 10 monclient: get_auth_request con 0x555cf953b990 auth_method 0 2026-02-24T22:21:40.180 INFO:tasks.workunit.client.0.trial046.stderr: -366> 2026-02-24T22:21:39.883+0000 7f8027fff640 10 monclient: get_auth_request con 0x555cf953d070 auth_method 0 2026-02-24T22:21:40.180 INFO:tasks.workunit.client.0.trial046.stderr: -365> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: build_initial_monmap 2026-02-24T22:21:40.180 INFO:tasks.workunit.client.0.trial046.stderr: -364> 2026-02-24T22:21:39.887+0000 7f803667da80 1 build_initial for_mkfs: 0 2026-02-24T22:21:40.180 INFO:tasks.workunit.client.0.trial046.stderr: -363> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: monmap: 2026-02-24T22:21:40.180 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:21:40.180 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:21:40.180 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:21:40.180 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:21:40.180 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:21:40.181 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.181 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:21:40.181 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:21:40.181 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:21:40.181 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.181 INFO:tasks.workunit.client.0.trial046.stderr: -362> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding auth protocol: cephx 2026-02-24T22:21:40.181 INFO:tasks.workunit.client.0.trial046.stderr: -361> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding auth protocol: cephx 2026-02-24T22:21:40.181 INFO:tasks.workunit.client.0.trial046.stderr: -360> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding auth protocol: cephx 2026-02-24T22:21:40.181 INFO:tasks.workunit.client.0.trial046.stderr: -359> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding con mode: secure 2026-02-24T22:21:40.181 INFO:tasks.workunit.client.0.trial046.stderr: -358> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding con mode: crc 2026-02-24T22:21:40.181 INFO:tasks.workunit.client.0.trial046.stderr: -357> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding con mode: secure 2026-02-24T22:21:40.181 INFO:tasks.workunit.client.0.trial046.stderr: -356> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding con mode: crc 2026-02-24T22:21:40.181 INFO:tasks.workunit.client.0.trial046.stderr: -355> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding con mode: secure 2026-02-24T22:21:40.181 INFO:tasks.workunit.client.0.trial046.stderr: -354> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding con mode: crc 2026-02-24T22:21:40.181 INFO:tasks.workunit.client.0.trial046.stderr: -353> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding con mode: crc 2026-02-24T22:21:40.181 INFO:tasks.workunit.client.0.trial046.stderr: -352> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding con mode: secure 2026-02-24T22:21:40.182 INFO:tasks.workunit.client.0.trial046.stderr: -351> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding con mode: crc 2026-02-24T22:21:40.182 INFO:tasks.workunit.client.0.trial046.stderr: -350> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding con mode: secure 2026-02-24T22:21:40.182 INFO:tasks.workunit.client.0.trial046.stderr: -349> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding con mode: crc 2026-02-24T22:21:40.182 INFO:tasks.workunit.client.0.trial046.stderr: -348> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding con mode: secure 2026-02-24T22:21:40.182 INFO:tasks.workunit.client.0.trial046.stderr: -347> 2026-02-24T22:21:39.887+0000 7f803667da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:40.182 INFO:tasks.workunit.client.0.trial046.stderr: -346> 2026-02-24T22:21:39.887+0000 7f803667da80 5 asok(0x555cf93acf00) register_command objecter_requests cmddesc objecter_requests hook 0x555cf954b2d0 EEXIST 2026-02-24T22:21:40.182 INFO:tasks.workunit.client.0.trial046.stderr: -345> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: init 2026-02-24T22:21:40.182 INFO:tasks.workunit.client.0.trial046.stderr: -344> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding auth protocol: cephx 2026-02-24T22:21:40.182 INFO:tasks.workunit.client.0.trial046.stderr: -343> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding auth protocol: cephx 2026-02-24T22:21:40.182 INFO:tasks.workunit.client.0.trial046.stderr: -342> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding auth protocol: cephx 2026-02-24T22:21:40.182 INFO:tasks.workunit.client.0.trial046.stderr: -341> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding con mode: secure 2026-02-24T22:21:40.182 INFO:tasks.workunit.client.0.trial046.stderr: -340> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding con mode: crc 2026-02-24T22:21:40.182 INFO:tasks.workunit.client.0.trial046.stderr: -339> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding con mode: secure 2026-02-24T22:21:40.182 INFO:tasks.workunit.client.0.trial046.stderr: -338> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding con mode: crc 2026-02-24T22:21:40.182 INFO:tasks.workunit.client.0.trial046.stderr: -337> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding con mode: secure 2026-02-24T22:21:40.182 INFO:tasks.workunit.client.0.trial046.stderr: -336> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding con mode: crc 2026-02-24T22:21:40.182 INFO:tasks.workunit.client.0.trial046.stderr: -335> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding con mode: crc 2026-02-24T22:21:40.182 INFO:tasks.workunit.client.0.trial046.stderr: -334> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding con mode: secure 2026-02-24T22:21:40.182 INFO:tasks.workunit.client.0.trial046.stderr: -333> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding con mode: crc 2026-02-24T22:21:40.183 INFO:tasks.workunit.client.0.trial046.stderr: -332> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding con mode: secure 2026-02-24T22:21:40.183 INFO:tasks.workunit.client.0.trial046.stderr: -331> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding con mode: crc 2026-02-24T22:21:40.183 INFO:tasks.workunit.client.0.trial046.stderr: -330> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding con mode: secure 2026-02-24T22:21:40.183 INFO:tasks.workunit.client.0.trial046.stderr: -329> 2026-02-24T22:21:39.887+0000 7f803667da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:40.183 INFO:tasks.workunit.client.0.trial046.stderr: -328> 2026-02-24T22:21:39.887+0000 7f803667da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:40.183 INFO:tasks.workunit.client.0.trial046.stderr: -327> 2026-02-24T22:21:39.887+0000 7f803667da80 5 asok(0x555cf93acf00) register_command rotate-key cmddesc rotate-key hook 0x555cf953dd60 EEXIST 2026-02-24T22:21:40.183 INFO:tasks.workunit.client.0.trial046.stderr: -326> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: _reopen_session rank -1 2026-02-24T22:21:40.183 INFO:tasks.workunit.client.0.trial046.stderr: -325> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:21:40.183 INFO:tasks.workunit.client.0.trial046.stderr: -324> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): picked mon.noname-c con 0x555cf96976e0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:21:40.183 INFO:tasks.workunit.client.0.trial046.stderr: -323> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): picked mon.noname-a con 0x555cf9690430 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:21:40.183 INFO:tasks.workunit.client.0.trial046.stderr: -322> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): picked mon.noname-b con 0x555cf9693110 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:21:40.183 INFO:tasks.workunit.client.0.trial046.stderr: -321> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.183 INFO:tasks.workunit.client.0.trial046.stderr: -320> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.183 INFO:tasks.workunit.client.0.trial046.stderr: -319> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.183 INFO:tasks.workunit.client.0.trial046.stderr: -318> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): _renew_subs 2026-02-24T22:21:40.183 INFO:tasks.workunit.client.0.trial046.stderr: -317> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): authenticate will time out at 2069.992809s 2026-02-24T22:21:40.183 INFO:tasks.workunit.client.0.trial046.stderr: -316> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): get_auth_request con 0x555cf9690430 auth_method 0 2026-02-24T22:21:40.183 INFO:tasks.workunit.client.0.trial046.stderr: -315> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:40.184 INFO:tasks.workunit.client.0.trial046.stderr: -314> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:40.184 INFO:tasks.workunit.client.0.trial046.stderr: -313> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:40.184 INFO:tasks.workunit.client.0.trial046.stderr: -312> 2026-02-24T22:21:39.887+0000 7f802ffff640 10 monclient(hunting): get_auth_request con 0x555cf96976e0 auth_method 0 2026-02-24T22:21:40.184 INFO:tasks.workunit.client.0.trial046.stderr: -311> 2026-02-24T22:21:39.887+0000 7f802ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:40.184 INFO:tasks.workunit.client.0.trial046.stderr: -310> 2026-02-24T22:21:39.887+0000 7f802ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:40.184 INFO:tasks.workunit.client.0.trial046.stderr: -309> 2026-02-24T22:21:39.887+0000 7f802ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:40.184 INFO:tasks.workunit.client.0.trial046.stderr: -308> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:40.184 INFO:tasks.workunit.client.0.trial046.stderr: -307> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:40.184 INFO:tasks.workunit.client.0.trial046.stderr: -306> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:40.184 INFO:tasks.workunit.client.0.trial046.stderr: -305> 2026-02-24T22:21:39.887+0000 7f802ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:40.184 INFO:tasks.workunit.client.0.trial046.stderr: -304> 2026-02-24T22:21:39.887+0000 7f802ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:40.184 INFO:tasks.workunit.client.0.trial046.stderr: -303> 2026-02-24T22:21:39.887+0000 7f802ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:40.184 INFO:tasks.workunit.client.0.trial046.stderr: -302> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): handle_auth_done global_id 8511 payload 824 2026-02-24T22:21:40.184 INFO:tasks.workunit.client.0.trial046.stderr: -301> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient: _finish_hunting 0 2026-02-24T22:21:40.184 INFO:tasks.workunit.client.0.trial046.stderr: -300> 2026-02-24T22:21:39.887+0000 7f8027fff640 1 monclient: found mon.noname-a 2026-02-24T22:21:40.184 INFO:tasks.workunit.client.0.trial046.stderr: -299> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:21:40.184 INFO:tasks.workunit.client.0.trial046.stderr: -298> 2026-02-24T22:21:39.887+0000 7f8015ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:40.185 INFO:tasks.workunit.client.0.trial046.stderr: -297> 2026-02-24T22:21:39.887+0000 7f8015ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:21:40.185 INFO:tasks.workunit.client.0.trial046.stderr: -296> 2026-02-24T22:21:39.887+0000 7f8015ffb640 10 monclient: dump: 2026-02-24T22:21:40.185 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:40.185 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:40.185 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.185 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.185 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:40.185 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.185 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:40.185 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:40.185 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:40.185 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.185 INFO:tasks.workunit.client.0.trial046.stderr: -295> 2026-02-24T22:21:39.887+0000 7f8015ffb640 10 monclient: _finish_auth 0 2026-02-24T22:21:40.185 INFO:tasks.workunit.client.0.trial046.stderr: -294> 2026-02-24T22:21:39.887+0000 7f8015ffb640 10 monclient: _check_auth_tickets 2026-02-24T22:21:40.185 INFO:tasks.workunit.client.0.trial046.stderr: -293> 2026-02-24T22:21:39.887+0000 7f8015ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T22:21:40.185 INFO:tasks.workunit.client.0.trial046.stderr: -292> 2026-02-24T22:21:39.887+0000 7f8015ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:40.185 INFO:tasks.workunit.client.0.trial046.stderr: -291> 2026-02-24T22:21:39.887+0000 7f8015ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:21:40.185 INFO:tasks.workunit.client.0.trial046.stderr: -290> 2026-02-24T22:21:39.887+0000 7f8015ffb640 10 monclient: dump: 2026-02-24T22:21:40.186 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:40.186 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:40.186 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.186 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.186 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:40.186 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.186 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:40.186 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:40.186 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:40.186 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.186 INFO:tasks.workunit.client.0.trial046.stderr: -289> 2026-02-24T22:21:39.887+0000 7f803667da80 5 monclient: authenticate success, global_id 8511 2026-02-24T22:21:40.186 INFO:tasks.workunit.client.0.trial046.stderr: -288> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: _renew_subs 2026-02-24T22:21:40.186 INFO:tasks.workunit.client.0.trial046.stderr: -287> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:21:40.186 INFO:tasks.workunit.client.0.trial046.stderr: -286> 2026-02-24T22:21:39.887+0000 7f8035153640 4 set_mon_vals no callback set 2026-02-24T22:21:40.186 INFO:tasks.workunit.client.0.trial046.stderr: -285> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: _renew_subs 2026-02-24T22:21:40.186 INFO:tasks.workunit.client.0.trial046.stderr: -284> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:21:40.186 INFO:tasks.workunit.client.0.trial046.stderr: -283> 2026-02-24T22:21:39.887+0000 7f8015ffb640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:21:40.186 INFO:tasks.workunit.client.0.trial046.stderr: -282> 2026-02-24T22:21:39.887+0000 7f8015ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:21:40.186 INFO:tasks.workunit.client.0.trial046.stderr: -281> 2026-02-24T22:21:39.887+0000 7f8015ffb640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:21:40.186 INFO:tasks.workunit.client.0.trial046.stderr: -280> 2026-02-24T22:21:39.887+0000 7f802ffff640 10 monclient: get_auth_request con 0x7f80040706b0 auth_method 0 2026-02-24T22:21:40.187 INFO:tasks.workunit.client.0.trial046.stderr: -279> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: get_monmap_and_config 2026-02-24T22:21:40.187 INFO:tasks.workunit.client.0.trial046.stderr: -278> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: build_initial_monmap 2026-02-24T22:21:40.187 INFO:tasks.workunit.client.0.trial046.stderr: -277> 2026-02-24T22:21:39.887+0000 7f803667da80 1 build_initial for_mkfs: 0 2026-02-24T22:21:40.187 INFO:tasks.workunit.client.0.trial046.stderr: -276> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: monmap: 2026-02-24T22:21:40.187 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:21:40.187 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:21:40.187 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:21:40.187 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:21:40.187 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:21:40.187 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.187 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:21:40.187 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:21:40.187 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:21:40.187 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.187 INFO:tasks.workunit.client.0.trial046.stderr: -275> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding auth protocol: cephx 2026-02-24T22:21:40.187 INFO:tasks.workunit.client.0.trial046.stderr: -274> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding auth protocol: cephx 2026-02-24T22:21:40.187 INFO:tasks.workunit.client.0.trial046.stderr: -273> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding auth protocol: cephx 2026-02-24T22:21:40.187 INFO:tasks.workunit.client.0.trial046.stderr: -272> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding con mode: secure 2026-02-24T22:21:40.187 INFO:tasks.workunit.client.0.trial046.stderr: -271> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding con mode: crc 2026-02-24T22:21:40.188 INFO:tasks.workunit.client.0.trial046.stderr: -270> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding con mode: secure 2026-02-24T22:21:40.188 INFO:tasks.workunit.client.0.trial046.stderr: -269> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding con mode: crc 2026-02-24T22:21:40.188 INFO:tasks.workunit.client.0.trial046.stderr: -268> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding con mode: secure 2026-02-24T22:21:40.188 INFO:tasks.workunit.client.0.trial046.stderr: -267> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding con mode: crc 2026-02-24T22:21:40.188 INFO:tasks.workunit.client.0.trial046.stderr: -266> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding con mode: crc 2026-02-24T22:21:40.188 INFO:tasks.workunit.client.0.trial046.stderr: -265> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding con mode: secure 2026-02-24T22:21:40.188 INFO:tasks.workunit.client.0.trial046.stderr: -264> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding con mode: crc 2026-02-24T22:21:40.188 INFO:tasks.workunit.client.0.trial046.stderr: -263> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding con mode: secure 2026-02-24T22:21:40.188 INFO:tasks.workunit.client.0.trial046.stderr: -262> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding con mode: crc 2026-02-24T22:21:40.188 INFO:tasks.workunit.client.0.trial046.stderr: -261> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding con mode: secure 2026-02-24T22:21:40.188 INFO:tasks.workunit.client.0.trial046.stderr: -260> 2026-02-24T22:21:39.887+0000 7f803667da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:40.188 INFO:tasks.workunit.client.0.trial046.stderr: -259> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: init 2026-02-24T22:21:40.188 INFO:tasks.workunit.client.0.trial046.stderr: -258> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding auth protocol: cephx 2026-02-24T22:21:40.188 INFO:tasks.workunit.client.0.trial046.stderr: -257> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding auth protocol: cephx 2026-02-24T22:21:40.188 INFO:tasks.workunit.client.0.trial046.stderr: -256> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding auth protocol: cephx 2026-02-24T22:21:40.188 INFO:tasks.workunit.client.0.trial046.stderr: -255> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding con mode: secure 2026-02-24T22:21:40.188 INFO:tasks.workunit.client.0.trial046.stderr: -254> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding con mode: crc 2026-02-24T22:21:40.188 INFO:tasks.workunit.client.0.trial046.stderr: -253> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding con mode: secure 2026-02-24T22:21:40.188 INFO:tasks.workunit.client.0.trial046.stderr: -252> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding con mode: crc 2026-02-24T22:21:40.188 INFO:tasks.workunit.client.0.trial046.stderr: -251> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding con mode: secure 2026-02-24T22:21:40.188 INFO:tasks.workunit.client.0.trial046.stderr: -250> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding con mode: crc 2026-02-24T22:21:40.188 INFO:tasks.workunit.client.0.trial046.stderr: -249> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding con mode: crc 2026-02-24T22:21:40.189 INFO:tasks.workunit.client.0.trial046.stderr: -248> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding con mode: secure 2026-02-24T22:21:40.189 INFO:tasks.workunit.client.0.trial046.stderr: -247> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding con mode: crc 2026-02-24T22:21:40.189 INFO:tasks.workunit.client.0.trial046.stderr: -246> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding con mode: secure 2026-02-24T22:21:40.189 INFO:tasks.workunit.client.0.trial046.stderr: -245> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding con mode: crc 2026-02-24T22:21:40.189 INFO:tasks.workunit.client.0.trial046.stderr: -244> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding con mode: secure 2026-02-24T22:21:40.189 INFO:tasks.workunit.client.0.trial046.stderr: -243> 2026-02-24T22:21:39.887+0000 7f803667da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:40.189 INFO:tasks.workunit.client.0.trial046.stderr: -242> 2026-02-24T22:21:39.887+0000 7f803667da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:40.189 INFO:tasks.workunit.client.0.trial046.stderr: -241> 2026-02-24T22:21:39.887+0000 7f803667da80 5 asok(0x555cf93acf00) register_command rotate-key cmddesc rotate-key hook 0x7ffdfb2beb78 EEXIST 2026-02-24T22:21:40.189 INFO:tasks.workunit.client.0.trial046.stderr: -240> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: _reopen_session rank -1 2026-02-24T22:21:40.189 INFO:tasks.workunit.client.0.trial046.stderr: -239> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T22:21:40.189 INFO:tasks.workunit.client.0.trial046.stderr: -238> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): picked mon.noname-b con 0x555cf97541b0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:21:40.189 INFO:tasks.workunit.client.0.trial046.stderr: -237> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): picked mon.noname-a con 0x555cf97326a0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:21:40.189 INFO:tasks.workunit.client.0.trial046.stderr: -236> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): picked mon.noname-c con 0x555cf975b690 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:21:40.189 INFO:tasks.workunit.client.0.trial046.stderr: -235> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.189 INFO:tasks.workunit.client.0.trial046.stderr: -234> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.189 INFO:tasks.workunit.client.0.trial046.stderr: -233> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.189 INFO:tasks.workunit.client.0.trial046.stderr: -232> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): _renew_subs 2026-02-24T22:21:40.189 INFO:tasks.workunit.client.0.trial046.stderr: -231> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): authenticate will time out at 2069.995335s 2026-02-24T22:21:40.189 INFO:tasks.workunit.client.0.trial046.stderr: -230> 2026-02-24T22:21:39.887+0000 7f8034952640 10 monclient(hunting): get_auth_request con 0x555cf975b690 auth_method 0 2026-02-24T22:21:40.189 INFO:tasks.workunit.client.0.trial046.stderr: -229> 2026-02-24T22:21:39.887+0000 7f8034952640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:40.189 INFO:tasks.workunit.client.0.trial046.stderr: -228> 2026-02-24T22:21:39.887+0000 7f8034952640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:40.190 INFO:tasks.workunit.client.0.trial046.stderr: -227> 2026-02-24T22:21:39.887+0000 7f8034952640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:40.190 INFO:tasks.workunit.client.0.trial046.stderr: -226> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): get_auth_request con 0x555cf97326a0 auth_method 0 2026-02-24T22:21:40.190 INFO:tasks.workunit.client.0.trial046.stderr: -225> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:40.190 INFO:tasks.workunit.client.0.trial046.stderr: -224> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:40.190 INFO:tasks.workunit.client.0.trial046.stderr: -223> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:40.190 INFO:tasks.workunit.client.0.trial046.stderr: -222> 2026-02-24T22:21:39.887+0000 7f8034952640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:40.190 INFO:tasks.workunit.client.0.trial046.stderr: -221> 2026-02-24T22:21:39.887+0000 7f8034952640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:40.190 INFO:tasks.workunit.client.0.trial046.stderr: -220> 2026-02-24T22:21:39.887+0000 7f8034952640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:40.190 INFO:tasks.workunit.client.0.trial046.stderr: -219> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:40.190 INFO:tasks.workunit.client.0.trial046.stderr: -218> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:40.190 INFO:tasks.workunit.client.0.trial046.stderr: -217> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:40.190 INFO:tasks.workunit.client.0.trial046.stderr: -216> 2026-02-24T22:21:39.887+0000 7f802ffff640 10 monclient(hunting): get_auth_request con 0x555cf97541b0 auth_method 0 2026-02-24T22:21:40.190 INFO:tasks.workunit.client.0.trial046.stderr: -215> 2026-02-24T22:21:39.887+0000 7f802ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:40.190 INFO:tasks.workunit.client.0.trial046.stderr: -214> 2026-02-24T22:21:39.887+0000 7f802ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:40.190 INFO:tasks.workunit.client.0.trial046.stderr: -213> 2026-02-24T22:21:39.887+0000 7f802ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:40.190 INFO:tasks.workunit.client.0.trial046.stderr: -212> 2026-02-24T22:21:39.887+0000 7f8034952640 10 monclient(hunting): handle_auth_done global_id 7940 payload 274 2026-02-24T22:21:40.190 INFO:tasks.workunit.client.0.trial046.stderr: -211> 2026-02-24T22:21:39.887+0000 7f8034952640 10 monclient: _finish_hunting 0 2026-02-24T22:21:40.190 INFO:tasks.workunit.client.0.trial046.stderr: -210> 2026-02-24T22:21:39.887+0000 7f8034952640 1 monclient: found mon.noname-c 2026-02-24T22:21:40.190 INFO:tasks.workunit.client.0.trial046.stderr: -209> 2026-02-24T22:21:39.887+0000 7f8034952640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:21:40.190 INFO:tasks.workunit.client.0.trial046.stderr: -208> 2026-02-24T22:21:39.887+0000 7f80037fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:40.190 INFO:tasks.workunit.client.0.trial046.stderr: -207> 2026-02-24T22:21:39.887+0000 7f80037fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:21:40.191 INFO:tasks.workunit.client.0.trial046.stderr: -206> 2026-02-24T22:21:39.887+0000 7f80037fe640 10 monclient: dump: 2026-02-24T22:21:40.191 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:40.191 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:40.191 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.191 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.191 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:40.191 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.191 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:40.191 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:40.191 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:40.191 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.191 INFO:tasks.workunit.client.0.trial046.stderr: -205> 2026-02-24T22:21:39.887+0000 7f80037fe640 10 monclient: _finish_auth 0 2026-02-24T22:21:40.191 INFO:tasks.workunit.client.0.trial046.stderr: -204> 2026-02-24T22:21:39.887+0000 7f80037fe640 10 monclient: _check_auth_tickets 2026-02-24T22:21:40.191 INFO:tasks.workunit.client.0.trial046.stderr: -203> 2026-02-24T22:21:39.887+0000 7f80037fe640 10 monclient: handle_config config(0 keys) 2026-02-24T22:21:40.191 INFO:tasks.workunit.client.0.trial046.stderr: -202> 2026-02-24T22:21:39.887+0000 7f80037fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:40.191 INFO:tasks.workunit.client.0.trial046.stderr: -201> 2026-02-24T22:21:39.887+0000 7f80037fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:21:40.191 INFO:tasks.workunit.client.0.trial046.stderr: -200> 2026-02-24T22:21:39.887+0000 7f80037fe640 10 monclient: dump: 2026-02-24T22:21:40.191 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:40.191 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:40.191 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.191 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.191 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:40.191 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.192 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:40.192 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:40.192 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:40.192 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.192 INFO:tasks.workunit.client.0.trial046.stderr: -199> 2026-02-24T22:21:39.887+0000 7f803667da80 5 monclient: authenticate success, global_id 7940 2026-02-24T22:21:40.192 INFO:tasks.workunit.client.0.trial046.stderr: -198> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: get_monmap_and_config success 2026-02-24T22:21:40.192 INFO:tasks.workunit.client.0.trial046.stderr: -197> 2026-02-24T22:21:39.887+0000 7f803667da80 4 set_mon_vals no callback set 2026-02-24T22:21:40.192 INFO:tasks.workunit.client.0.trial046.stderr: -196> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: shutdown 2026-02-24T22:21:40.192 INFO:tasks.workunit.client.0.trial046.stderr: -195> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient: build_initial_monmap 2026-02-24T22:21:40.192 INFO:tasks.workunit.client.0.trial046.stderr: -194> 2026-02-24T22:21:39.891+0000 7f803667da80 1 build_initial for_mkfs: 0 2026-02-24T22:21:40.192 INFO:tasks.workunit.client.0.trial046.stderr: -193> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient: monmap: 2026-02-24T22:21:40.192 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:21:40.192 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:21:40.192 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:21:40.192 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:21:40.192 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:21:40.192 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.192 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:21:40.192 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:21:40.192 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:21:40.192 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.192 INFO:tasks.workunit.client.0.trial046.stderr: -192> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding auth protocol: cephx 2026-02-24T22:21:40.192 INFO:tasks.workunit.client.0.trial046.stderr: -191> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding auth protocol: cephx 2026-02-24T22:21:40.193 INFO:tasks.workunit.client.0.trial046.stderr: -190> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding auth protocol: cephx 2026-02-24T22:21:40.193 INFO:tasks.workunit.client.0.trial046.stderr: -189> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding con mode: secure 2026-02-24T22:21:40.193 INFO:tasks.workunit.client.0.trial046.stderr: -188> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding con mode: crc 2026-02-24T22:21:40.193 INFO:tasks.workunit.client.0.trial046.stderr: -187> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding con mode: secure 2026-02-24T22:21:40.193 INFO:tasks.workunit.client.0.trial046.stderr: -186> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding con mode: crc 2026-02-24T22:21:40.193 INFO:tasks.workunit.client.0.trial046.stderr: -185> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding con mode: secure 2026-02-24T22:21:40.193 INFO:tasks.workunit.client.0.trial046.stderr: -184> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding con mode: crc 2026-02-24T22:21:40.193 INFO:tasks.workunit.client.0.trial046.stderr: -183> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding con mode: crc 2026-02-24T22:21:40.193 INFO:tasks.workunit.client.0.trial046.stderr: -182> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding con mode: secure 2026-02-24T22:21:40.193 INFO:tasks.workunit.client.0.trial046.stderr: -181> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding con mode: crc 2026-02-24T22:21:40.193 INFO:tasks.workunit.client.0.trial046.stderr: -180> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding con mode: secure 2026-02-24T22:21:40.193 INFO:tasks.workunit.client.0.trial046.stderr: -179> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding con mode: crc 2026-02-24T22:21:40.193 INFO:tasks.workunit.client.0.trial046.stderr: -178> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding con mode: secure 2026-02-24T22:21:40.193 INFO:tasks.workunit.client.0.trial046.stderr: -177> 2026-02-24T22:21:39.891+0000 7f803667da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:40.193 INFO:tasks.workunit.client.0.trial046.stderr: -176> 2026-02-24T22:21:39.891+0000 7f803667da80 1 librados: starting msgr at 2026-02-24T22:21:40.193 INFO:tasks.workunit.client.0.trial046.stderr: -175> 2026-02-24T22:21:39.891+0000 7f803667da80 1 librados: starting objecter 2026-02-24T22:21:40.193 INFO:tasks.workunit.client.0.trial046.stderr: -174> 2026-02-24T22:21:39.891+0000 7f803667da80 5 asok(0x555cf93acf00) register_command objecter_requests cmddesc objecter_requests hook 0x555cf9698710 EEXIST 2026-02-24T22:21:40.193 INFO:tasks.workunit.client.0.trial046.stderr: -173> 2026-02-24T22:21:39.891+0000 7f803667da80 1 librados: setting wanted keys 2026-02-24T22:21:40.193 INFO:tasks.workunit.client.0.trial046.stderr: -172> 2026-02-24T22:21:39.891+0000 7f803667da80 1 librados: calling monclient init 2026-02-24T22:21:40.193 INFO:tasks.workunit.client.0.trial046.stderr: -171> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient: init 2026-02-24T22:21:40.193 INFO:tasks.workunit.client.0.trial046.stderr: -170> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding auth protocol: cephx 2026-02-24T22:21:40.193 INFO:tasks.workunit.client.0.trial046.stderr: -169> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding auth protocol: cephx 2026-02-24T22:21:40.193 INFO:tasks.workunit.client.0.trial046.stderr: -168> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding auth protocol: cephx 2026-02-24T22:21:40.194 INFO:tasks.workunit.client.0.trial046.stderr: -167> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding con mode: secure 2026-02-24T22:21:40.194 INFO:tasks.workunit.client.0.trial046.stderr: -166> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding con mode: crc 2026-02-24T22:21:40.194 INFO:tasks.workunit.client.0.trial046.stderr: -165> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding con mode: secure 2026-02-24T22:21:40.194 INFO:tasks.workunit.client.0.trial046.stderr: -164> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding con mode: crc 2026-02-24T22:21:40.194 INFO:tasks.workunit.client.0.trial046.stderr: -163> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding con mode: secure 2026-02-24T22:21:40.194 INFO:tasks.workunit.client.0.trial046.stderr: -162> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding con mode: crc 2026-02-24T22:21:40.194 INFO:tasks.workunit.client.0.trial046.stderr: -161> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding con mode: crc 2026-02-24T22:21:40.194 INFO:tasks.workunit.client.0.trial046.stderr: -160> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding con mode: secure 2026-02-24T22:21:40.194 INFO:tasks.workunit.client.0.trial046.stderr: -159> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding con mode: crc 2026-02-24T22:21:40.194 INFO:tasks.workunit.client.0.trial046.stderr: -158> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding con mode: secure 2026-02-24T22:21:40.194 INFO:tasks.workunit.client.0.trial046.stderr: -157> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding con mode: crc 2026-02-24T22:21:40.194 INFO:tasks.workunit.client.0.trial046.stderr: -156> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding con mode: secure 2026-02-24T22:21:40.194 INFO:tasks.workunit.client.0.trial046.stderr: -155> 2026-02-24T22:21:39.891+0000 7f803667da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:40.194 INFO:tasks.workunit.client.0.trial046.stderr: -154> 2026-02-24T22:21:39.891+0000 7f803667da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:40.194 INFO:tasks.workunit.client.0.trial046.stderr: -153> 2026-02-24T22:21:39.891+0000 7f803667da80 5 asok(0x555cf93acf00) register_command rotate-key cmddesc rotate-key hook 0x555cf9727a78 EEXIST 2026-02-24T22:21:40.194 INFO:tasks.workunit.client.0.trial046.stderr: -152> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient: _reopen_session rank -1 2026-02-24T22:21:40.194 INFO:tasks.workunit.client.0.trial046.stderr: -151> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:21:40.194 INFO:tasks.workunit.client.0.trial046.stderr: -150> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient(hunting): picked mon.noname-c con 0x555cf97541b0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:21:40.194 INFO:tasks.workunit.client.0.trial046.stderr: -149> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient(hunting): picked mon.noname-a con 0x555cf975b690 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:21:40.194 INFO:tasks.workunit.client.0.trial046.stderr: -148> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient(hunting): picked mon.noname-b con 0x555cf9759e10 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:21:40.194 INFO:tasks.workunit.client.0.trial046.stderr: -147> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.194 INFO:tasks.workunit.client.0.trial046.stderr: -146> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.194 INFO:tasks.workunit.client.0.trial046.stderr: -145> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.195 INFO:tasks.workunit.client.0.trial046.stderr: -144> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient(hunting): _renew_subs 2026-02-24T22:21:40.195 INFO:tasks.workunit.client.0.trial046.stderr: -143> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient(hunting): authenticate will time out at 2069.996645s 2026-02-24T22:21:40.195 INFO:tasks.workunit.client.0.trial046.stderr: -142> 2026-02-24T22:21:39.891+0000 7f802ffff640 10 monclient(hunting): get_auth_request con 0x555cf97541b0 auth_method 0 2026-02-24T22:21:40.195 INFO:tasks.workunit.client.0.trial046.stderr: -141> 2026-02-24T22:21:39.891+0000 7f802ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:40.195 INFO:tasks.workunit.client.0.trial046.stderr: -140> 2026-02-24T22:21:39.891+0000 7f802ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:40.195 INFO:tasks.workunit.client.0.trial046.stderr: -139> 2026-02-24T22:21:39.891+0000 7f802ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:40.195 INFO:tasks.workunit.client.0.trial046.stderr: -138> 2026-02-24T22:21:39.891+0000 7f8027fff640 10 monclient(hunting): get_auth_request con 0x555cf975b690 auth_method 0 2026-02-24T22:21:40.195 INFO:tasks.workunit.client.0.trial046.stderr: -137> 2026-02-24T22:21:39.891+0000 7f8027fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:40.195 INFO:tasks.workunit.client.0.trial046.stderr: -136> 2026-02-24T22:21:39.891+0000 7f8027fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:40.195 INFO:tasks.workunit.client.0.trial046.stderr: -135> 2026-02-24T22:21:39.891+0000 7f8027fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:40.195 INFO:tasks.workunit.client.0.trial046.stderr: -134> 2026-02-24T22:21:39.891+0000 7f802ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:40.195 INFO:tasks.workunit.client.0.trial046.stderr: -133> 2026-02-24T22:21:39.891+0000 7f802ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:40.195 INFO:tasks.workunit.client.0.trial046.stderr: -132> 2026-02-24T22:21:39.891+0000 7f802ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:40.195 INFO:tasks.workunit.client.0.trial046.stderr: -131> 2026-02-24T22:21:39.891+0000 7f8027fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:40.195 INFO:tasks.workunit.client.0.trial046.stderr: -130> 2026-02-24T22:21:39.891+0000 7f8027fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:40.195 INFO:tasks.workunit.client.0.trial046.stderr: -129> 2026-02-24T22:21:39.891+0000 7f8027fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:40.195 INFO:tasks.workunit.client.0.trial046.stderr: -128> 2026-02-24T22:21:39.891+0000 7f8034952640 10 monclient(hunting): get_auth_request con 0x555cf9759e10 auth_method 0 2026-02-24T22:21:40.195 INFO:tasks.workunit.client.0.trial046.stderr: -127> 2026-02-24T22:21:39.891+0000 7f8034952640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:40.195 INFO:tasks.workunit.client.0.trial046.stderr: -126> 2026-02-24T22:21:39.891+0000 7f8034952640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:40.195 INFO:tasks.workunit.client.0.trial046.stderr: -125> 2026-02-24T22:21:39.891+0000 7f8034952640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:40.195 INFO:tasks.workunit.client.0.trial046.stderr: -124> 2026-02-24T22:21:39.891+0000 7f802ffff640 10 monclient(hunting): handle_auth_done global_id 7943 payload 824 2026-02-24T22:21:40.195 INFO:tasks.workunit.client.0.trial046.stderr: -123> 2026-02-24T22:21:39.891+0000 7f802ffff640 10 monclient: _finish_hunting 0 2026-02-24T22:21:40.195 INFO:tasks.workunit.client.0.trial046.stderr: -122> 2026-02-24T22:21:39.891+0000 7f802ffff640 1 monclient: found mon.noname-c 2026-02-24T22:21:40.195 INFO:tasks.workunit.client.0.trial046.stderr: -121> 2026-02-24T22:21:39.891+0000 7f802ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:21:40.195 INFO:tasks.workunit.client.0.trial046.stderr: -120> 2026-02-24T22:21:39.891+0000 7f8001ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:40.196 INFO:tasks.workunit.client.0.trial046.stderr: -119> 2026-02-24T22:21:39.891+0000 7f8001ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:21:40.196 INFO:tasks.workunit.client.0.trial046.stderr: -118> 2026-02-24T22:21:39.891+0000 7f8001ffb640 10 monclient: dump: 2026-02-24T22:21:40.196 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:40.196 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:40.196 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.196 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.196 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:40.196 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.196 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:40.196 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:40.196 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:40.196 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.196 INFO:tasks.workunit.client.0.trial046.stderr: -117> 2026-02-24T22:21:39.891+0000 7f8001ffb640 10 monclient: _finish_auth 0 2026-02-24T22:21:40.196 INFO:tasks.workunit.client.0.trial046.stderr: -116> 2026-02-24T22:21:39.891+0000 7f8001ffb640 10 monclient: _check_auth_tickets 2026-02-24T22:21:40.196 INFO:tasks.workunit.client.0.trial046.stderr: -115> 2026-02-24T22:21:39.891+0000 7f8001ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T22:21:40.196 INFO:tasks.workunit.client.0.trial046.stderr: -114> 2026-02-24T22:21:39.891+0000 7f8001ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:40.196 INFO:tasks.workunit.client.0.trial046.stderr: -113> 2026-02-24T22:21:39.891+0000 7f8001ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:21:40.196 INFO:tasks.workunit.client.0.trial046.stderr: -112> 2026-02-24T22:21:39.891+0000 7f8001ffb640 10 monclient: dump: 2026-02-24T22:21:40.196 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:40.196 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:40.196 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.196 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.196 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:40.196 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.197 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:40.197 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:40.197 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:40.197 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.197 INFO:tasks.workunit.client.0.trial046.stderr: -111> 2026-02-24T22:21:39.891+0000 7f803667da80 5 monclient: authenticate success, global_id 7943 2026-02-24T22:21:40.197 INFO:tasks.workunit.client.0.trial046.stderr: -110> 2026-02-24T22:21:39.891+0000 7f80037fe640 4 set_mon_vals no callback set 2026-02-24T22:21:40.197 INFO:tasks.workunit.client.0.trial046.stderr: -109> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient: _renew_subs 2026-02-24T22:21:40.197 INFO:tasks.workunit.client.0.trial046.stderr: -108> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:21:40.197 INFO:tasks.workunit.client.0.trial046.stderr: -107> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient: _renew_subs 2026-02-24T22:21:40.197 INFO:tasks.workunit.client.0.trial046.stderr: -106> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:21:40.197 INFO:tasks.workunit.client.0.trial046.stderr: -105> 2026-02-24T22:21:39.891+0000 7f803667da80 1 librados: init done 2026-02-24T22:21:40.197 INFO:tasks.workunit.client.0.trial046.stderr: -104> 2026-02-24T22:21:39.891+0000 7f803667da80 5 asok(0x555cf93acf00) register_command cr dump hook 0x555cf9784058 2026-02-24T22:21:40.197 INFO:tasks.workunit.client.0.trial046.stderr: -103> 2026-02-24T22:21:39.891+0000 7f8001ffb640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:21:40.197 INFO:tasks.workunit.client.0.trial046.stderr: -102> 2026-02-24T22:21:39.891+0000 7f8001ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:21:40.197 INFO:tasks.workunit.client.0.trial046.stderr: -101> 2026-02-24T22:21:39.891+0000 7f8001ffb640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:21:40.197 INFO:tasks.workunit.client.0.trial046.stderr: -100> 2026-02-24T22:21:39.891+0000 7f8027fff640 10 monclient: get_auth_request con 0x7f7ff40707b0 auth_method 0 2026-02-24T22:21:40.197 INFO:tasks.workunit.client.0.trial046.stderr: -99> 2026-02-24T22:21:39.891+0000 7f803667da80 4 Realm: () 2026-02-24T22:21:40.197 INFO:tasks.workunit.client.0.trial046.stderr: -98> 2026-02-24T22:21:39.891+0000 7f803667da80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:21:40.197 INFO:tasks.workunit.client.0.trial046.stderr: -97> 2026-02-24T22:21:39.891+0000 7f803667da80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:21:40.197 INFO:tasks.workunit.client.0.trial046.stderr: -96> 2026-02-24T22:21:39.891+0000 7f803667da80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:21:40.197 INFO:tasks.workunit.client.0.trial046.stderr: -95> 2026-02-24T22:21:39.891+0000 7f803667da80 20 zonegroup default 2026-02-24T22:21:40.197 INFO:tasks.workunit.client.0.trial046.stderr: -94> 2026-02-24T22:21:39.891+0000 7f8034952640 10 monclient: get_auth_request con 0x555cf97f5dc0 auth_method 0 2026-02-24T22:21:40.197 INFO:tasks.workunit.client.0.trial046.stderr: -93> 2026-02-24T22:21:39.891+0000 7f803667da80 20 rados->read ofs=0 len=0 2026-02-24T22:21:40.197 INFO:tasks.workunit.client.0.trial046.stderr: -92> 2026-02-24T22:21:39.891+0000 7f802ffff640 10 monclient: get_auth_request con 0x555cf9819250 auth_method 0 2026-02-24T22:21:40.197 INFO:tasks.workunit.client.0.trial046.stderr: -91> 2026-02-24T22:21:39.891+0000 7f803667da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:40.198 INFO:tasks.workunit.client.0.trial046.stderr: -90> 2026-02-24T22:21:39.891+0000 7f803667da80 20 rados->read ofs=0 len=0 2026-02-24T22:21:40.198 INFO:tasks.workunit.client.0.trial046.stderr: -89> 2026-02-24T22:21:39.891+0000 7f8027fff640 10 monclient: get_auth_request con 0x555cf983a8b0 auth_method 0 2026-02-24T22:21:40.198 INFO:tasks.workunit.client.0.trial046.stderr: -88> 2026-02-24T22:21:39.891+0000 7f803667da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:40.198 INFO:tasks.workunit.client.0.trial046.stderr: -87> 2026-02-24T22:21:39.891+0000 7f803667da80 20 started sync module instance, tier type = 2026-02-24T22:21:40.198 INFO:tasks.workunit.client.0.trial046.stderr: -86> 2026-02-24T22:21:39.891+0000 7f803667da80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:21:40.198 INFO:tasks.workunit.client.0.trial046.stderr: -85> 2026-02-24T22:21:39.895+0000 7f8034952640 10 monclient: get_auth_request con 0x555cf989bd60 auth_method 0 2026-02-24T22:21:40.198 INFO:tasks.workunit.client.0.trial046.stderr: -84> 2026-02-24T22:21:39.895+0000 7f8034952640 10 monclient: get_auth_request con 0x555cf99a35c0 auth_method 0 2026-02-24T22:21:40.198 INFO:tasks.workunit.client.0.trial046.stderr: -83> 2026-02-24T22:21:39.895+0000 7f802ffff640 10 monclient: get_auth_request con 0x555cf98fe610 auth_method 0 2026-02-24T22:21:40.198 INFO:tasks.workunit.client.0.trial046.stderr: -82> 2026-02-24T22:21:39.895+0000 7f8027fff640 10 monclient: get_auth_request con 0x555cf9961480 auth_method 0 2026-02-24T22:21:40.198 INFO:tasks.workunit.client.0.trial046.stderr: -81> 2026-02-24T22:21:39.895+0000 7f802ffff640 10 monclient: get_auth_request con 0x555cf9a065c0 auth_method 0 2026-02-24T22:21:40.198 INFO:tasks.workunit.client.0.trial046.stderr: -80> 2026-02-24T22:21:39.895+0000 7f8034952640 0 -- 10.20.193.46:0/1916772995 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x555cf989bd60 msgr2=0x555cf98bc140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:40.198 INFO:tasks.workunit.client.0.trial046.stderr: -79> 2026-02-24T22:21:39.895+0000 7f8001ffb640 1 client.7943.objecter ms_handle_reset 0x555cf989bd60 session 0x555cf98186c0 osd.0 2026-02-24T22:21:40.198 INFO:tasks.workunit.client.0.trial046.stderr: -78> 2026-02-24T22:21:39.895+0000 7f8001ffb640 10 monclient: _renew_subs 2026-02-24T22:21:40.198 INFO:tasks.workunit.client.0.trial046.stderr: -77> 2026-02-24T22:21:39.895+0000 7f8001ffb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:21:40.198 INFO:tasks.workunit.client.0.trial046.stderr: -76> 2026-02-24T22:21:39.895+0000 7f8034952640 10 monclient: get_auth_request con 0x7f7ff409d4e0 auth_method 0 2026-02-24T22:21:40.198 INFO:tasks.workunit.client.0.trial046.stderr: -75> 2026-02-24T22:21:39.895+0000 7f803667da80 20 add_watcher() i=1 2026-02-24T22:21:40.198 INFO:tasks.workunit.client.0.trial046.stderr: -74> 2026-02-24T22:21:39.895+0000 7f803667da80 20 add_watcher() i=6 2026-02-24T22:21:40.198 INFO:tasks.workunit.client.0.trial046.stderr: -73> 2026-02-24T22:21:39.895+0000 7f803667da80 20 add_watcher() i=5 2026-02-24T22:21:40.198 INFO:tasks.workunit.client.0.trial046.stderr: -72> 2026-02-24T22:21:39.895+0000 7f803667da80 20 add_watcher() i=2 2026-02-24T22:21:40.198 INFO:tasks.workunit.client.0.trial046.stderr: -71> 2026-02-24T22:21:39.895+0000 7f803667da80 20 add_watcher() i=0 2026-02-24T22:21:40.198 INFO:tasks.workunit.client.0.trial046.stderr: -70> 2026-02-24T22:21:39.895+0000 7f803667da80 20 add_watcher() i=3 2026-02-24T22:21:40.198 INFO:tasks.workunit.client.0.trial046.stderr: -69> 2026-02-24T22:21:39.895+0000 7f803667da80 20 add_watcher() i=4 2026-02-24T22:21:40.198 INFO:tasks.workunit.client.0.trial046.stderr: -68> 2026-02-24T22:21:39.895+0000 7f803667da80 20 add_watcher() i=7 2026-02-24T22:21:40.198 INFO:tasks.workunit.client.0.trial046.stderr: -67> 2026-02-24T22:21:39.895+0000 7f803667da80 2 all 8 watchers are set, enabling cache 2026-02-24T22:21:40.198 INFO:tasks.workunit.client.0.trial046.stderr: -66> 2026-02-24T22:21:39.895+0000 7f8027fff640 10 monclient: get_auth_request con 0x7f7f54008af0 auth_method 0 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -65> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding auth protocol: cephx 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -64> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding auth protocol: cephx 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -63> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding auth protocol: cephx 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -62> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding con mode: secure 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -61> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding con mode: crc 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -60> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding con mode: secure 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -59> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding con mode: crc 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -58> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding con mode: secure 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -57> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding con mode: crc 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -56> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding con mode: crc 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -55> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding con mode: secure 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -54> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding con mode: crc 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -53> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding con mode: secure 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -52> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding con mode: crc 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -51> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding con mode: secure 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -50> 2026-02-24T22:21:39.899+0000 7f803667da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -49> 2026-02-24T22:21:39.899+0000 7f803667da80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -48> 2026-02-24T22:21:39.899+0000 7f803667da80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -47> 2026-02-24T22:21:39.899+0000 7f803667da80 5 asok(0x555cf93acf00) register_command cache list hook 0x555cf9759bb0 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -46> 2026-02-24T22:21:39.899+0000 7f803667da80 5 asok(0x555cf93acf00) register_command cache inspect hook 0x555cf9759bb0 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -45> 2026-02-24T22:21:39.899+0000 7f803667da80 5 asok(0x555cf93acf00) register_command cache erase hook 0x555cf9759bb0 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -44> 2026-02-24T22:21:39.899+0000 7f803667da80 5 asok(0x555cf93acf00) register_command cache zap hook 0x555cf9759bb0 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -43> 2026-02-24T22:21:39.899+0000 7f803667da80 5 note: GC not initialized 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -42> 2026-02-24T22:21:39.899+0000 7f7fe1fdb640 20 reqs_thread_entry: start 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -41> 2026-02-24T22:21:39.899+0000 7f803667da80 5 asok(0x555cf93acf00) register_command sync trace show hook 0x7f80280a5630 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -40> 2026-02-24T22:21:39.899+0000 7f803667da80 5 asok(0x555cf93acf00) register_command sync trace history hook 0x7f80280a5630 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -39> 2026-02-24T22:21:39.899+0000 7f803667da80 5 asok(0x555cf93acf00) register_command sync trace active hook 0x7f80280a5630 2026-02-24T22:21:40.199 INFO:tasks.workunit.client.0.trial046.stderr: -38> 2026-02-24T22:21:39.899+0000 7f803667da80 5 asok(0x555cf93acf00) register_command sync trace active_short hook 0x7f80280a5630 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -37> 2026-02-24T22:21:39.899+0000 7f8034952640 0 -- 10.20.193.46:0/2371875498 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x555cf99918b0 msgr2=0x555cf998c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -36> 2026-02-24T22:21:40.103+0000 7f8034952640 10 monclient: get_auth_request con 0x555cf99918b0 auth_method 0 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -35> 2026-02-24T22:21:40.103+0000 7f802ffff640 10 monclient: get_auth_request con 0x555cf998d8f0 auth_method 0 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -34> 2026-02-24T22:21:40.107+0000 7f8027fff640 10 monclient: get_auth_request con 0x555cf998ee50 auth_method 0 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -33> 2026-02-24T22:21:40.107+0000 7f8034952640 10 monclient: get_auth_request con 0x555cf9989bd0 auth_method 0 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -32> 2026-02-24T22:21:40.111+0000 7f802ffff640 10 monclient: get_auth_request con 0x555cf998c080 auth_method 0 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -31> 2026-02-24T22:21:40.115+0000 7f8027fff640 10 monclient: get_auth_request con 0x555cf9985380 auth_method 0 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -30> 2026-02-24T22:21:40.139+0000 7f803667da80 20 init_complete bucket index max shards: 1 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -29> 2026-02-24T22:21:40.139+0000 7f803667da80 20 Filter name: none 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -28> 2026-02-24T22:21:40.139+0000 7f7f3bfff640 20 reqs_thread_entry: start 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -27> 2026-02-24T22:21:40.139+0000 7f803667da80 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -26> 2026-02-24T22:21:40.139+0000 7f803667da80 20 rados->read ofs=0 len=0 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -25> 2026-02-24T22:21:40.139+0000 7f8034952640 10 monclient: get_auth_request con 0x555cf94ab430 auth_method 0 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -24> 2026-02-24T22:21:40.139+0000 7f803667da80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -23> 2026-02-24T22:21:40.139+0000 7f803667da80 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -22> 2026-02-24T22:21:40.139+0000 7f803667da80 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -21> 2026-02-24T22:21:40.139+0000 7f803667da80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : miss 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -20> 2026-02-24T22:21:40.139+0000 7f803667da80 20 rados->read ofs=0 len=0 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -19> 2026-02-24T22:21:40.139+0000 7f803667da80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -18> 2026-02-24T22:21:40.139+0000 7f803667da80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -17> 2026-02-24T22:21:40.139+0000 7f803667da80 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -16> 2026-02-24T22:21:40.139+0000 7f803667da80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -15> 2026-02-24T22:21:40.139+0000 7f803667da80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -14> 2026-02-24T22:21:40.139+0000 7f803667da80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -13> 2026-02-24T22:21:40.143+0000 7f803667da80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -12> 2026-02-24T22:21:40.143+0000 7f803667da80 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:21:40.200 INFO:tasks.workunit.client.0.trial046.stderr: -11> 2026-02-24T22:21:40.143+0000 7f803667da80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr: -10> 2026-02-24T22:21:40.143+0000 7f803667da80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518, ofs0, ns] 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr: -9> 2026-02-24T22:21:40.143+0000 7f8002ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959945 cookie 93857813528032 notifier 7943 bl.length()=929 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr: -8> 2026-02-24T22:21:40.143+0000 7f8002ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr: -7> 2026-02-24T22:21:40.143+0000 7f8002ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr: -6> 2026-02-24T22:21:40.143+0000 7f8002ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr: -5> 2026-02-24T22:21:40.147+0000 7f803667da80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_sources= new_sources= 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr: -4> 2026-02-24T22:21:40.147+0000 7f803667da80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential sources added=[] removed=[] 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr: -3> 2026-02-24T22:21:40.147+0000 7f803667da80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_dests= new_dests= 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr: -2> 2026-02-24T22:21:40.147+0000 7f803667da80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential dests added=[] removed=[] 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr: -1> 2026-02-24T22:21:40.147+0000 7f803667da80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f803667da80 time 2026-02-24T22:21:40.149765+0000 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f8039abfeb1] 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x555ceb63eb9f] 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x16a) [0x555cebdaa26a] 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f8038dbfd90] 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr: 0> 2026-02-24T22:21:40.151+0000 7f803667da80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f803667da80 thread_name:radosgw-admin 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f8038dd8520] 2026-02-24T22:21:40.201 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f8039abff10] 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x555ceb63eb9f] 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x16a) [0x555cebdaa26a] 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f8038dbfd90] 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr:--- logging levels --- 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 none 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 lockdep 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 context 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 crush 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_balancer 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_locker 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log_expire 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_migrator 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr: 3/ 5 mds_quiesce 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 buffer 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 timer 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 filer 2026-02-24T22:21:40.202 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 striper 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 objecter 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rados 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_mirror 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_replay 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_pwl 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 journaler 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objectcacher 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 immutable_obj_cache 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 client 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 osd 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 optracker 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objclass 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 filestore 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 journal 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 ms 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mon 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 0/10 monc 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 paxos 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 tp 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 auth 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 crypto 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 finisher 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 reserver 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 heartbeatmap 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 perfcounter 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_sync 2026-02-24T22:21:40.203 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_datacache 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_access 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dbstore 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_flight 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_lifecycle 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_restore 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw_notification 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_bucket_logging 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 javaclient 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 asok 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 throttle 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 refs 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 compressor 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore_compression 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluefs 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 bdev 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 kstore 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 4/ 5 rocksdb 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 fuse 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 2/ 5 mgr 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mgrc 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 dpdk 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 eventtrace 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 prioritycache 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 test 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephfs_mirror 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephsqlite 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 crimson_interrupt 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore 2026-02-24T22:21:40.204 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_onode 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_odata 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_omap 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_tm 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_t 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cleaner 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_epm 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_lba 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_fixedkv_tree 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cache 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_journal 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_device 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_backref 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 alienstore 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mclock 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dedup 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cyanstore 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 ceph_exporter 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 memstore 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 trace 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 ceph_dedup 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: -2/-2 (syslog threshold) 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 99/99 (stderr threshold) 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr:--- pthread ID / name mapping for recent threads --- 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 7f7f3bfff640 / http_manager 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 7f7fe1fdb640 / http_manager 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 7f8001ffb640 / ms_dispatch 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 7f8002ffd640 / io_context_pool 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 7f80037fe640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 7f8015ffb640 / ms_dispatch 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 7f8024ff9640 / ms_dispatch 2026-02-24T22:21:40.205 INFO:tasks.workunit.client.0.trial046.stderr: 7f80267fc640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: 7f8026ffd640 / admin_socket 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: 7f8027fff640 / msgr-worker-2 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: 7f802f7fe640 / ms_dispatch 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: 7f802ffff640 / msgr-worker-1 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: 7f8034952640 / msgr-worker-0 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: 7f8035153640 / io_context_pool 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: 7f803667da80 / radosgw-admin 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: max_recent 500 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: max_new 1000 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: log_file /var/log/ceph/ceph-client.admin.55191.log 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr:--- end dump of recent events --- 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr:--- begin dump of recent events --- 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: -499> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: -498> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: -497> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient(hunting): _renew_subs 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: -496> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient(hunting): authenticate will time out at 2069.986078s 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: -495> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient(hunting): get_auth_request con 0x555cf94e3b00 auth_method 0 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: -494> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: -493> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: -492> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: -491> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): get_auth_request con 0x555cf94e6cf0 auth_method 0 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: -490> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: -489> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: -488> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: -487> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: -486> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: -485> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: -484> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: -483> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: -482> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:40.206 INFO:tasks.workunit.client.0.trial046.stderr: -481> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient(hunting): handle_auth_done global_id 8505 payload 274 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr: -480> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient: _finish_hunting 0 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr: -479> 2026-02-24T22:21:39.879+0000 7f8027fff640 1 monclient: found mon.noname-a 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr: -478> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr: -477> 2026-02-24T22:21:39.879+0000 7f80267fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr: -476> 2026-02-24T22:21:39.879+0000 7f80267fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr: -475> 2026-02-24T22:21:39.879+0000 7f80267fc640 10 monclient: dump: 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr: -474> 2026-02-24T22:21:39.879+0000 7f80267fc640 10 monclient: _finish_auth 0 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr: -473> 2026-02-24T22:21:39.879+0000 7f80267fc640 10 monclient: _check_auth_tickets 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr: -472> 2026-02-24T22:21:39.879+0000 7f80267fc640 10 monclient: handle_config config(0 keys) 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr: -471> 2026-02-24T22:21:39.879+0000 7f80267fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr: -470> 2026-02-24T22:21:39.879+0000 7f80267fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr: -469> 2026-02-24T22:21:39.879+0000 7f80267fc640 10 monclient: dump: 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:40.207 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr: -468> 2026-02-24T22:21:39.879+0000 7f803667da80 5 monclient: authenticate success, global_id 8505 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr: -467> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient: get_monmap_and_config success 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr: -466> 2026-02-24T22:21:39.879+0000 7f803667da80 4 set_mon_vals no callback set 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr: -465> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient: shutdown 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr: -464> 2026-02-24T22:21:39.879+0000 7f803667da80 5 asok(0x555cf93acf00) unregister_commands rotate-key 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr: -463> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient: build_initial_monmap 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr: -462> 2026-02-24T22:21:39.879+0000 7f803667da80 1 build_initial for_mkfs: 0 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr: -461> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient: monmap: 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr: -460> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding auth protocol: cephx 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr: -459> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding auth protocol: cephx 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr: -458> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding auth protocol: cephx 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr: -457> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding con mode: secure 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr: -456> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding con mode: crc 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr: -455> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding con mode: secure 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr: -454> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding con mode: crc 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr: -453> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding con mode: secure 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr: -452> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding con mode: crc 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr: -451> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding con mode: crc 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr: -450> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding con mode: secure 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr: -449> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding con mode: crc 2026-02-24T22:21:40.208 INFO:tasks.workunit.client.0.trial046.stderr: -448> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding con mode: secure 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -447> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding con mode: crc 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -446> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94b47a0) adding con mode: secure 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -445> 2026-02-24T22:21:39.879+0000 7f803667da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -444> 2026-02-24T22:21:39.879+0000 7f803667da80 1 librados: starting msgr at 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -443> 2026-02-24T22:21:39.879+0000 7f803667da80 1 librados: starting objecter 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -442> 2026-02-24T22:21:39.879+0000 7f803667da80 5 asok(0x555cf93acf00) register_command objecter_requests hook 0x555cf940de40 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -441> 2026-02-24T22:21:39.879+0000 7f803667da80 1 librados: setting wanted keys 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -440> 2026-02-24T22:21:39.879+0000 7f803667da80 1 librados: calling monclient init 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -439> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient: init 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -438> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding auth protocol: cephx 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -437> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding auth protocol: cephx 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -436> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding auth protocol: cephx 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -435> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding con mode: secure 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -434> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding con mode: crc 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -433> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding con mode: secure 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -432> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding con mode: crc 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -431> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding con mode: secure 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -430> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding con mode: crc 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -429> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding con mode: crc 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -428> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding con mode: secure 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -427> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding con mode: crc 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -426> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding con mode: secure 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -425> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding con mode: crc 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -424> 2026-02-24T22:21:39.879+0000 7f803667da80 5 AuthRegistry(0x555cf94ad010) adding con mode: secure 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -423> 2026-02-24T22:21:39.879+0000 7f803667da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -422> 2026-02-24T22:21:39.879+0000 7f803667da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -421> 2026-02-24T22:21:39.879+0000 7f803667da80 5 asok(0x555cf93acf00) register_command rotate-key hook 0x555cf94ad158 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -420> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient: _reopen_session rank -1 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -419> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -418> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient(hunting): picked mon.noname-c con 0x555cf943afe0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -417> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient(hunting): picked mon.noname-b con 0x555cf94e3b00 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:21:40.209 INFO:tasks.workunit.client.0.trial046.stderr: -416> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient(hunting): picked mon.noname-a con 0x555cf94e6cf0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -415> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -414> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -413> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -412> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient(hunting): _renew_subs 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -411> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient(hunting): authenticate will time out at 2069.987415s 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -410> 2026-02-24T22:21:39.879+0000 7f802ffff640 10 monclient(hunting): get_auth_request con 0x555cf943afe0 auth_method 0 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -409> 2026-02-24T22:21:39.879+0000 7f802ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -408> 2026-02-24T22:21:39.879+0000 7f802ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -407> 2026-02-24T22:21:39.879+0000 7f802ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -406> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): get_auth_request con 0x555cf94e6cf0 auth_method 0 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -405> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -404> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -403> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -402> 2026-02-24T22:21:39.879+0000 7f802ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -401> 2026-02-24T22:21:39.879+0000 7f802ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -400> 2026-02-24T22:21:39.879+0000 7f802ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -399> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient(hunting): get_auth_request con 0x555cf94e3b00 auth_method 0 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -398> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -397> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -396> 2026-02-24T22:21:39.879+0000 7f8027fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -395> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -394> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -393> 2026-02-24T22:21:39.879+0000 7f8034952640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -392> 2026-02-24T22:21:39.879+0000 7f802ffff640 10 monclient(hunting): handle_auth_done global_id 7934 payload 824 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -391> 2026-02-24T22:21:39.879+0000 7f802ffff640 10 monclient: _finish_hunting 0 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -390> 2026-02-24T22:21:39.879+0000 7f802ffff640 1 monclient: found mon.noname-c 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -389> 2026-02-24T22:21:39.879+0000 7f802ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -388> 2026-02-24T22:21:39.879+0000 7f8024ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -387> 2026-02-24T22:21:39.879+0000 7f8024ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr: -386> 2026-02-24T22:21:39.879+0000 7f8024ff9640 10 monclient: dump: 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:40.210 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr: -385> 2026-02-24T22:21:39.879+0000 7f8024ff9640 10 monclient: _finish_auth 0 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr: -384> 2026-02-24T22:21:39.879+0000 7f8024ff9640 10 monclient: _check_auth_tickets 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr: -383> 2026-02-24T22:21:39.879+0000 7f8024ff9640 10 monclient: handle_config config(0 keys) 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr: -382> 2026-02-24T22:21:39.879+0000 7f8024ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr: -381> 2026-02-24T22:21:39.879+0000 7f8024ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr: -380> 2026-02-24T22:21:39.879+0000 7f8024ff9640 10 monclient: dump: 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr: -379> 2026-02-24T22:21:39.879+0000 7f80267fc640 4 set_mon_vals no callback set 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr: -378> 2026-02-24T22:21:39.879+0000 7f803667da80 5 monclient: authenticate success, global_id 7934 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr: -377> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient: _renew_subs 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr: -376> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr: -375> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient: _renew_subs 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr: -374> 2026-02-24T22:21:39.879+0000 7f803667da80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr: -373> 2026-02-24T22:21:39.879+0000 7f803667da80 1 librados: init done 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr: -372> 2026-02-24T22:21:39.883+0000 7f8024ff9640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr: -371> 2026-02-24T22:21:39.883+0000 7f8024ff9640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr: -370> 2026-02-24T22:21:39.883+0000 7f8024ff9640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:21:40.211 INFO:tasks.workunit.client.0.trial046.stderr: -369> 2026-02-24T22:21:39.883+0000 7f8027fff640 10 monclient: get_auth_request con 0x7f80100707b0 auth_method 0 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr: -368> 2026-02-24T22:21:39.883+0000 7f8034952640 10 monclient: get_auth_request con 0x555cf953a7a0 auth_method 0 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr: -367> 2026-02-24T22:21:39.883+0000 7f802ffff640 10 monclient: get_auth_request con 0x555cf953b990 auth_method 0 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr: -366> 2026-02-24T22:21:39.883+0000 7f8027fff640 10 monclient: get_auth_request con 0x555cf953d070 auth_method 0 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr: -365> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: build_initial_monmap 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr: -364> 2026-02-24T22:21:39.887+0000 7f803667da80 1 build_initial for_mkfs: 0 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr: -363> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: monmap: 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr: -362> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding auth protocol: cephx 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr: -361> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding auth protocol: cephx 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr: -360> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding auth protocol: cephx 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr: -359> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding con mode: secure 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr: -358> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding con mode: crc 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr: -357> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding con mode: secure 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr: -356> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding con mode: crc 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr: -355> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding con mode: secure 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr: -354> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding con mode: crc 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr: -353> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding con mode: crc 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr: -352> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding con mode: secure 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr: -351> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding con mode: crc 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr: -350> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding con mode: secure 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr: -349> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding con mode: crc 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr: -348> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953fc40) adding con mode: secure 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr: -347> 2026-02-24T22:21:39.887+0000 7f803667da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr: -346> 2026-02-24T22:21:39.887+0000 7f803667da80 5 asok(0x555cf93acf00) register_command objecter_requests cmddesc objecter_requests hook 0x555cf954b2d0 EEXIST 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr: -345> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: init 2026-02-24T22:21:40.212 INFO:tasks.workunit.client.0.trial046.stderr: -344> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding auth protocol: cephx 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -343> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding auth protocol: cephx 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -342> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding auth protocol: cephx 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -341> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding con mode: secure 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -340> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding con mode: crc 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -339> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding con mode: secure 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -338> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding con mode: crc 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -337> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding con mode: secure 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -336> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding con mode: crc 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -335> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding con mode: crc 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -334> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding con mode: secure 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -333> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding con mode: crc 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -332> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding con mode: secure 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -331> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding con mode: crc 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -330> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf953dc18) adding con mode: secure 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -329> 2026-02-24T22:21:39.887+0000 7f803667da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -328> 2026-02-24T22:21:39.887+0000 7f803667da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -327> 2026-02-24T22:21:39.887+0000 7f803667da80 5 asok(0x555cf93acf00) register_command rotate-key cmddesc rotate-key hook 0x555cf953dd60 EEXIST 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -326> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: _reopen_session rank -1 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -325> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -324> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): picked mon.noname-c con 0x555cf96976e0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -323> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): picked mon.noname-a con 0x555cf9690430 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -322> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): picked mon.noname-b con 0x555cf9693110 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -321> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -320> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -319> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -318> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): _renew_subs 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -317> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): authenticate will time out at 2069.992809s 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -316> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): get_auth_request con 0x555cf9690430 auth_method 0 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -315> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -314> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -313> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -312> 2026-02-24T22:21:39.887+0000 7f802ffff640 10 monclient(hunting): get_auth_request con 0x555cf96976e0 auth_method 0 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -311> 2026-02-24T22:21:39.887+0000 7f802ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:40.213 INFO:tasks.workunit.client.0.trial046.stderr: -310> 2026-02-24T22:21:39.887+0000 7f802ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr: -309> 2026-02-24T22:21:39.887+0000 7f802ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr: -308> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr: -307> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr: -306> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr: -305> 2026-02-24T22:21:39.887+0000 7f802ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr: -304> 2026-02-24T22:21:39.887+0000 7f802ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr: -303> 2026-02-24T22:21:39.887+0000 7f802ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr: -302> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): handle_auth_done global_id 8511 payload 824 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr: -301> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient: _finish_hunting 0 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr: -300> 2026-02-24T22:21:39.887+0000 7f8027fff640 1 monclient: found mon.noname-a 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr: -299> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr: -298> 2026-02-24T22:21:39.887+0000 7f8015ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr: -297> 2026-02-24T22:21:39.887+0000 7f8015ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr: -296> 2026-02-24T22:21:39.887+0000 7f8015ffb640 10 monclient: dump: 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr: -295> 2026-02-24T22:21:39.887+0000 7f8015ffb640 10 monclient: _finish_auth 0 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr: -294> 2026-02-24T22:21:39.887+0000 7f8015ffb640 10 monclient: _check_auth_tickets 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr: -293> 2026-02-24T22:21:39.887+0000 7f8015ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr: -292> 2026-02-24T22:21:39.887+0000 7f8015ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr: -291> 2026-02-24T22:21:39.887+0000 7f8015ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr: -290> 2026-02-24T22:21:39.887+0000 7f8015ffb640 10 monclient: dump: 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.214 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr: -289> 2026-02-24T22:21:39.887+0000 7f803667da80 5 monclient: authenticate success, global_id 8511 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr: -288> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: _renew_subs 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr: -287> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr: -286> 2026-02-24T22:21:39.887+0000 7f8035153640 4 set_mon_vals no callback set 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr: -285> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: _renew_subs 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr: -284> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr: -283> 2026-02-24T22:21:39.887+0000 7f8015ffb640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr: -282> 2026-02-24T22:21:39.887+0000 7f8015ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr: -281> 2026-02-24T22:21:39.887+0000 7f8015ffb640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr: -280> 2026-02-24T22:21:39.887+0000 7f802ffff640 10 monclient: get_auth_request con 0x7f80040706b0 auth_method 0 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr: -279> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: get_monmap_and_config 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr: -278> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: build_initial_monmap 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr: -277> 2026-02-24T22:21:39.887+0000 7f803667da80 1 build_initial for_mkfs: 0 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr: -276> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: monmap: 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr: -275> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding auth protocol: cephx 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr: -274> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding auth protocol: cephx 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr: -273> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding auth protocol: cephx 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr: -272> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding con mode: secure 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr: -271> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding con mode: crc 2026-02-24T22:21:40.215 INFO:tasks.workunit.client.0.trial046.stderr: -270> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding con mode: secure 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -269> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding con mode: crc 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -268> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding con mode: secure 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -267> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding con mode: crc 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -266> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding con mode: crc 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -265> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding con mode: secure 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -264> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding con mode: crc 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -263> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding con mode: secure 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -262> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding con mode: crc 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -261> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x555cf9728bf0) adding con mode: secure 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -260> 2026-02-24T22:21:39.887+0000 7f803667da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -259> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: init 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -258> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding auth protocol: cephx 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -257> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding auth protocol: cephx 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -256> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding auth protocol: cephx 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -255> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding con mode: secure 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -254> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding con mode: crc 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -253> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding con mode: secure 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -252> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding con mode: crc 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -251> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding con mode: secure 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -250> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding con mode: crc 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -249> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding con mode: crc 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -248> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding con mode: secure 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -247> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding con mode: crc 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -246> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding con mode: secure 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -245> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding con mode: crc 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -244> 2026-02-24T22:21:39.887+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2bea30) adding con mode: secure 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -243> 2026-02-24T22:21:39.887+0000 7f803667da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -242> 2026-02-24T22:21:39.887+0000 7f803667da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -241> 2026-02-24T22:21:39.887+0000 7f803667da80 5 asok(0x555cf93acf00) register_command rotate-key cmddesc rotate-key hook 0x7ffdfb2beb78 EEXIST 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -240> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: _reopen_session rank -1 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -239> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -238> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): picked mon.noname-b con 0x555cf97541b0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -237> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): picked mon.noname-a con 0x555cf97326a0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -236> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): picked mon.noname-c con 0x555cf975b690 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -235> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -234> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.216 INFO:tasks.workunit.client.0.trial046.stderr: -233> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -232> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): _renew_subs 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -231> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient(hunting): authenticate will time out at 2069.995335s 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -230> 2026-02-24T22:21:39.887+0000 7f8034952640 10 monclient(hunting): get_auth_request con 0x555cf975b690 auth_method 0 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -229> 2026-02-24T22:21:39.887+0000 7f8034952640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -228> 2026-02-24T22:21:39.887+0000 7f8034952640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -227> 2026-02-24T22:21:39.887+0000 7f8034952640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -226> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): get_auth_request con 0x555cf97326a0 auth_method 0 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -225> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -224> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -223> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -222> 2026-02-24T22:21:39.887+0000 7f8034952640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -221> 2026-02-24T22:21:39.887+0000 7f8034952640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -220> 2026-02-24T22:21:39.887+0000 7f8034952640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -219> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -218> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -217> 2026-02-24T22:21:39.887+0000 7f8027fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -216> 2026-02-24T22:21:39.887+0000 7f802ffff640 10 monclient(hunting): get_auth_request con 0x555cf97541b0 auth_method 0 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -215> 2026-02-24T22:21:39.887+0000 7f802ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -214> 2026-02-24T22:21:39.887+0000 7f802ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -213> 2026-02-24T22:21:39.887+0000 7f802ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -212> 2026-02-24T22:21:39.887+0000 7f8034952640 10 monclient(hunting): handle_auth_done global_id 7940 payload 274 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -211> 2026-02-24T22:21:39.887+0000 7f8034952640 10 monclient: _finish_hunting 0 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -210> 2026-02-24T22:21:39.887+0000 7f8034952640 1 monclient: found mon.noname-c 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -209> 2026-02-24T22:21:39.887+0000 7f8034952640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -208> 2026-02-24T22:21:39.887+0000 7f80037fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -207> 2026-02-24T22:21:39.887+0000 7f80037fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: -206> 2026-02-24T22:21:39.887+0000 7f80037fe640 10 monclient: dump: 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:40.217 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr: -205> 2026-02-24T22:21:39.887+0000 7f80037fe640 10 monclient: _finish_auth 0 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr: -204> 2026-02-24T22:21:39.887+0000 7f80037fe640 10 monclient: _check_auth_tickets 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr: -203> 2026-02-24T22:21:39.887+0000 7f80037fe640 10 monclient: handle_config config(0 keys) 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr: -202> 2026-02-24T22:21:39.887+0000 7f80037fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr: -201> 2026-02-24T22:21:39.887+0000 7f80037fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr: -200> 2026-02-24T22:21:39.887+0000 7f80037fe640 10 monclient: dump: 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr: -199> 2026-02-24T22:21:39.887+0000 7f803667da80 5 monclient: authenticate success, global_id 7940 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr: -198> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: get_monmap_and_config success 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr: -197> 2026-02-24T22:21:39.887+0000 7f803667da80 4 set_mon_vals no callback set 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr: -196> 2026-02-24T22:21:39.887+0000 7f803667da80 10 monclient: shutdown 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr: -195> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient: build_initial_monmap 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr: -194> 2026-02-24T22:21:39.891+0000 7f803667da80 1 build_initial for_mkfs: 0 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr: -193> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient: monmap: 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr: -192> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding auth protocol: cephx 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr: -191> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding auth protocol: cephx 2026-02-24T22:21:40.218 INFO:tasks.workunit.client.0.trial046.stderr: -190> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding auth protocol: cephx 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -189> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding con mode: secure 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -188> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding con mode: crc 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -187> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding con mode: secure 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -186> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding con mode: crc 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -185> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding con mode: secure 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -184> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding con mode: crc 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -183> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding con mode: crc 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -182> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding con mode: secure 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -181> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding con mode: crc 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -180> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding con mode: secure 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -179> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding con mode: crc 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -178> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9728b30) adding con mode: secure 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -177> 2026-02-24T22:21:39.891+0000 7f803667da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -176> 2026-02-24T22:21:39.891+0000 7f803667da80 1 librados: starting msgr at 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -175> 2026-02-24T22:21:39.891+0000 7f803667da80 1 librados: starting objecter 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -174> 2026-02-24T22:21:39.891+0000 7f803667da80 5 asok(0x555cf93acf00) register_command objecter_requests cmddesc objecter_requests hook 0x555cf9698710 EEXIST 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -173> 2026-02-24T22:21:39.891+0000 7f803667da80 1 librados: setting wanted keys 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -172> 2026-02-24T22:21:39.891+0000 7f803667da80 1 librados: calling monclient init 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -171> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient: init 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -170> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding auth protocol: cephx 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -169> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding auth protocol: cephx 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -168> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding auth protocol: cephx 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -167> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding con mode: secure 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -166> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding con mode: crc 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -165> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding con mode: secure 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -164> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding con mode: crc 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -163> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding con mode: secure 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -162> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding con mode: crc 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -161> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding con mode: crc 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -160> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding con mode: secure 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -159> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding con mode: crc 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -158> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding con mode: secure 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -157> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding con mode: crc 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -156> 2026-02-24T22:21:39.891+0000 7f803667da80 5 AuthRegistry(0x555cf9727930) adding con mode: secure 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -155> 2026-02-24T22:21:39.891+0000 7f803667da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -154> 2026-02-24T22:21:39.891+0000 7f803667da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -153> 2026-02-24T22:21:39.891+0000 7f803667da80 5 asok(0x555cf93acf00) register_command rotate-key cmddesc rotate-key hook 0x555cf9727a78 EEXIST 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -152> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient: _reopen_session rank -1 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -151> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:21:40.219 INFO:tasks.workunit.client.0.trial046.stderr: -150> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient(hunting): picked mon.noname-c con 0x555cf97541b0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -149> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient(hunting): picked mon.noname-a con 0x555cf975b690 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -148> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient(hunting): picked mon.noname-b con 0x555cf9759e10 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -147> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -146> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -145> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient(hunting): start opening mon connection 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -144> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient(hunting): _renew_subs 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -143> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient(hunting): authenticate will time out at 2069.996645s 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -142> 2026-02-24T22:21:39.891+0000 7f802ffff640 10 monclient(hunting): get_auth_request con 0x555cf97541b0 auth_method 0 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -141> 2026-02-24T22:21:39.891+0000 7f802ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -140> 2026-02-24T22:21:39.891+0000 7f802ffff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -139> 2026-02-24T22:21:39.891+0000 7f802ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -138> 2026-02-24T22:21:39.891+0000 7f8027fff640 10 monclient(hunting): get_auth_request con 0x555cf975b690 auth_method 0 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -137> 2026-02-24T22:21:39.891+0000 7f8027fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -136> 2026-02-24T22:21:39.891+0000 7f8027fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -135> 2026-02-24T22:21:39.891+0000 7f8027fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -134> 2026-02-24T22:21:39.891+0000 7f802ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -133> 2026-02-24T22:21:39.891+0000 7f802ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -132> 2026-02-24T22:21:39.891+0000 7f802ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -131> 2026-02-24T22:21:39.891+0000 7f8027fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -130> 2026-02-24T22:21:39.891+0000 7f8027fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -129> 2026-02-24T22:21:39.891+0000 7f8027fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -128> 2026-02-24T22:21:39.891+0000 7f8034952640 10 monclient(hunting): get_auth_request con 0x555cf9759e10 auth_method 0 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -127> 2026-02-24T22:21:39.891+0000 7f8034952640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -126> 2026-02-24T22:21:39.891+0000 7f8034952640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -125> 2026-02-24T22:21:39.891+0000 7f8034952640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -124> 2026-02-24T22:21:39.891+0000 7f802ffff640 10 monclient(hunting): handle_auth_done global_id 7943 payload 824 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -123> 2026-02-24T22:21:39.891+0000 7f802ffff640 10 monclient: _finish_hunting 0 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -122> 2026-02-24T22:21:39.891+0000 7f802ffff640 1 monclient: found mon.noname-c 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -121> 2026-02-24T22:21:39.891+0000 7f802ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -120> 2026-02-24T22:21:39.891+0000 7f8001ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -119> 2026-02-24T22:21:39.891+0000 7f8001ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr: -118> 2026-02-24T22:21:39.891+0000 7f8001ffb640 10 monclient: dump: 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.220 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -117> 2026-02-24T22:21:39.891+0000 7f8001ffb640 10 monclient: _finish_auth 0 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -116> 2026-02-24T22:21:39.891+0000 7f8001ffb640 10 monclient: _check_auth_tickets 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -115> 2026-02-24T22:21:39.891+0000 7f8001ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -114> 2026-02-24T22:21:39.891+0000 7f8001ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -113> 2026-02-24T22:21:39.891+0000 7f8001ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -112> 2026-02-24T22:21:39.891+0000 7f8001ffb640 10 monclient: dump: 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -111> 2026-02-24T22:21:39.891+0000 7f803667da80 5 monclient: authenticate success, global_id 7943 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -110> 2026-02-24T22:21:39.891+0000 7f80037fe640 4 set_mon_vals no callback set 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -109> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient: _renew_subs 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -108> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -107> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient: _renew_subs 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -106> 2026-02-24T22:21:39.891+0000 7f803667da80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -105> 2026-02-24T22:21:39.891+0000 7f803667da80 1 librados: init done 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -104> 2026-02-24T22:21:39.891+0000 7f803667da80 5 asok(0x555cf93acf00) register_command cr dump hook 0x555cf9784058 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -103> 2026-02-24T22:21:39.891+0000 7f8001ffb640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -102> 2026-02-24T22:21:39.891+0000 7f8001ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -101> 2026-02-24T22:21:39.891+0000 7f8001ffb640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -100> 2026-02-24T22:21:39.891+0000 7f8027fff640 10 monclient: get_auth_request con 0x7f7ff40707b0 auth_method 0 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -99> 2026-02-24T22:21:39.891+0000 7f803667da80 4 Realm: () 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -98> 2026-02-24T22:21:39.891+0000 7f803667da80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -97> 2026-02-24T22:21:39.891+0000 7f803667da80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -96> 2026-02-24T22:21:39.891+0000 7f803667da80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -95> 2026-02-24T22:21:39.891+0000 7f803667da80 20 zonegroup default 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -94> 2026-02-24T22:21:39.891+0000 7f8034952640 10 monclient: get_auth_request con 0x555cf97f5dc0 auth_method 0 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -93> 2026-02-24T22:21:39.891+0000 7f803667da80 20 rados->read ofs=0 len=0 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -92> 2026-02-24T22:21:39.891+0000 7f802ffff640 10 monclient: get_auth_request con 0x555cf9819250 auth_method 0 2026-02-24T22:21:40.221 INFO:tasks.workunit.client.0.trial046.stderr: -91> 2026-02-24T22:21:39.891+0000 7f803667da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -90> 2026-02-24T22:21:39.891+0000 7f803667da80 20 rados->read ofs=0 len=0 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -89> 2026-02-24T22:21:39.891+0000 7f8027fff640 10 monclient: get_auth_request con 0x555cf983a8b0 auth_method 0 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -88> 2026-02-24T22:21:39.891+0000 7f803667da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -87> 2026-02-24T22:21:39.891+0000 7f803667da80 20 started sync module instance, tier type = 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -86> 2026-02-24T22:21:39.891+0000 7f803667da80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -85> 2026-02-24T22:21:39.895+0000 7f8034952640 10 monclient: get_auth_request con 0x555cf989bd60 auth_method 0 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -84> 2026-02-24T22:21:39.895+0000 7f8034952640 10 monclient: get_auth_request con 0x555cf99a35c0 auth_method 0 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -83> 2026-02-24T22:21:39.895+0000 7f802ffff640 10 monclient: get_auth_request con 0x555cf98fe610 auth_method 0 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -82> 2026-02-24T22:21:39.895+0000 7f8027fff640 10 monclient: get_auth_request con 0x555cf9961480 auth_method 0 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -81> 2026-02-24T22:21:39.895+0000 7f802ffff640 10 monclient: get_auth_request con 0x555cf9a065c0 auth_method 0 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -80> 2026-02-24T22:21:39.895+0000 7f8034952640 0 -- 10.20.193.46:0/1916772995 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x555cf989bd60 msgr2=0x555cf98bc140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -79> 2026-02-24T22:21:39.895+0000 7f8001ffb640 1 client.7943.objecter ms_handle_reset 0x555cf989bd60 session 0x555cf98186c0 osd.0 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -78> 2026-02-24T22:21:39.895+0000 7f8001ffb640 10 monclient: _renew_subs 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -77> 2026-02-24T22:21:39.895+0000 7f8001ffb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -76> 2026-02-24T22:21:39.895+0000 7f8034952640 10 monclient: get_auth_request con 0x7f7ff409d4e0 auth_method 0 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -75> 2026-02-24T22:21:39.895+0000 7f803667da80 20 add_watcher() i=1 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -74> 2026-02-24T22:21:39.895+0000 7f803667da80 20 add_watcher() i=6 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -73> 2026-02-24T22:21:39.895+0000 7f803667da80 20 add_watcher() i=5 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -72> 2026-02-24T22:21:39.895+0000 7f803667da80 20 add_watcher() i=2 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -71> 2026-02-24T22:21:39.895+0000 7f803667da80 20 add_watcher() i=0 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -70> 2026-02-24T22:21:39.895+0000 7f803667da80 20 add_watcher() i=3 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -69> 2026-02-24T22:21:39.895+0000 7f803667da80 20 add_watcher() i=4 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -68> 2026-02-24T22:21:39.895+0000 7f803667da80 20 add_watcher() i=7 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -67> 2026-02-24T22:21:39.895+0000 7f803667da80 2 all 8 watchers are set, enabling cache 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -66> 2026-02-24T22:21:39.895+0000 7f8027fff640 10 monclient: get_auth_request con 0x7f7f54008af0 auth_method 0 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -65> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding auth protocol: cephx 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -64> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding auth protocol: cephx 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -63> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding auth protocol: cephx 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -62> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding con mode: secure 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -61> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding con mode: crc 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -60> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding con mode: secure 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -59> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding con mode: crc 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -58> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding con mode: secure 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -57> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding con mode: crc 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -56> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding con mode: crc 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -55> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding con mode: secure 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -54> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding con mode: crc 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -53> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding con mode: secure 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -52> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding con mode: crc 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -51> 2026-02-24T22:21:39.895+0000 7f803667da80 5 AuthRegistry(0x7ffdfb2beea0) adding con mode: secure 2026-02-24T22:21:40.222 INFO:tasks.workunit.client.0.trial046.stderr: -50> 2026-02-24T22:21:39.899+0000 7f803667da80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -49> 2026-02-24T22:21:39.899+0000 7f803667da80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -48> 2026-02-24T22:21:39.899+0000 7f803667da80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -47> 2026-02-24T22:21:39.899+0000 7f803667da80 5 asok(0x555cf93acf00) register_command cache list hook 0x555cf9759bb0 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -46> 2026-02-24T22:21:39.899+0000 7f803667da80 5 asok(0x555cf93acf00) register_command cache inspect hook 0x555cf9759bb0 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -45> 2026-02-24T22:21:39.899+0000 7f803667da80 5 asok(0x555cf93acf00) register_command cache erase hook 0x555cf9759bb0 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -44> 2026-02-24T22:21:39.899+0000 7f803667da80 5 asok(0x555cf93acf00) register_command cache zap hook 0x555cf9759bb0 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -43> 2026-02-24T22:21:39.899+0000 7f803667da80 5 note: GC not initialized 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -42> 2026-02-24T22:21:39.899+0000 7f7fe1fdb640 20 reqs_thread_entry: start 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -41> 2026-02-24T22:21:39.899+0000 7f803667da80 5 asok(0x555cf93acf00) register_command sync trace show hook 0x7f80280a5630 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -40> 2026-02-24T22:21:39.899+0000 7f803667da80 5 asok(0x555cf93acf00) register_command sync trace history hook 0x7f80280a5630 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -39> 2026-02-24T22:21:39.899+0000 7f803667da80 5 asok(0x555cf93acf00) register_command sync trace active hook 0x7f80280a5630 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -38> 2026-02-24T22:21:39.899+0000 7f803667da80 5 asok(0x555cf93acf00) register_command sync trace active_short hook 0x7f80280a5630 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -37> 2026-02-24T22:21:39.899+0000 7f8034952640 0 -- 10.20.193.46:0/2371875498 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x555cf99918b0 msgr2=0x555cf998c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -36> 2026-02-24T22:21:40.103+0000 7f8034952640 10 monclient: get_auth_request con 0x555cf99918b0 auth_method 0 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -35> 2026-02-24T22:21:40.103+0000 7f802ffff640 10 monclient: get_auth_request con 0x555cf998d8f0 auth_method 0 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -34> 2026-02-24T22:21:40.107+0000 7f8027fff640 10 monclient: get_auth_request con 0x555cf998ee50 auth_method 0 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -33> 2026-02-24T22:21:40.107+0000 7f8034952640 10 monclient: get_auth_request con 0x555cf9989bd0 auth_method 0 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -32> 2026-02-24T22:21:40.111+0000 7f802ffff640 10 monclient: get_auth_request con 0x555cf998c080 auth_method 0 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -31> 2026-02-24T22:21:40.115+0000 7f8027fff640 10 monclient: get_auth_request con 0x555cf9985380 auth_method 0 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -30> 2026-02-24T22:21:40.139+0000 7f803667da80 20 init_complete bucket index max shards: 1 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -29> 2026-02-24T22:21:40.139+0000 7f803667da80 20 Filter name: none 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -28> 2026-02-24T22:21:40.139+0000 7f7f3bfff640 20 reqs_thread_entry: start 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -27> 2026-02-24T22:21:40.139+0000 7f803667da80 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -26> 2026-02-24T22:21:40.139+0000 7f803667da80 20 rados->read ofs=0 len=0 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -25> 2026-02-24T22:21:40.139+0000 7f8034952640 10 monclient: get_auth_request con 0x555cf94ab430 auth_method 0 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -24> 2026-02-24T22:21:40.139+0000 7f803667da80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -23> 2026-02-24T22:21:40.139+0000 7f803667da80 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -22> 2026-02-24T22:21:40.139+0000 7f803667da80 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -21> 2026-02-24T22:21:40.139+0000 7f803667da80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : miss 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -20> 2026-02-24T22:21:40.139+0000 7f803667da80 20 rados->read ofs=0 len=0 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -19> 2026-02-24T22:21:40.139+0000 7f803667da80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -18> 2026-02-24T22:21:40.139+0000 7f803667da80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -17> 2026-02-24T22:21:40.139+0000 7f803667da80 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -16> 2026-02-24T22:21:40.139+0000 7f803667da80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -15> 2026-02-24T22:21:40.139+0000 7f803667da80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -14> 2026-02-24T22:21:40.139+0000 7f803667da80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -13> 2026-02-24T22:21:40.143+0000 7f803667da80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -12> 2026-02-24T22:21:40.143+0000 7f803667da80 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -11> 2026-02-24T22:21:40.143+0000 7f803667da80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -10> 2026-02-24T22:21:40.143+0000 7f803667da80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518, ofs0, ns] 2026-02-24T22:21:40.223 INFO:tasks.workunit.client.0.trial046.stderr: -9> 2026-02-24T22:21:40.143+0000 7f8002ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959945 cookie 93857813528032 notifier 7943 bl.length()=929 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: -8> 2026-02-24T22:21:40.143+0000 7f8002ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: -7> 2026-02-24T22:21:40.143+0000 7f8002ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: -6> 2026-02-24T22:21:40.143+0000 7f8002ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: -5> 2026-02-24T22:21:40.147+0000 7f803667da80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_sources= new_sources= 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: -4> 2026-02-24T22:21:40.147+0000 7f803667da80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential sources added=[] removed=[] 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: -3> 2026-02-24T22:21:40.147+0000 7f803667da80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_dests= new_dests= 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: -2> 2026-02-24T22:21:40.147+0000 7f803667da80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential dests added=[] removed=[] 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: -1> 2026-02-24T22:21:40.147+0000 7f803667da80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f803667da80 time 2026-02-24T22:21:40.149765+0000 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f8039abfeb1] 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x555ceb63eb9f] 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x16a) [0x555cebdaa26a] 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f8038dbfd90] 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: 0> 2026-02-24T22:21:40.151+0000 7f803667da80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f803667da80 thread_name:radosgw-admin 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f8038dd8520] 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f8039abff10] 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x555ceb63eb9f] 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x16a) [0x555cebdaa26a] 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f8038dbfd90] 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr:--- logging levels --- 2026-02-24T22:21:40.224 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 none 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 lockdep 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 context 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 crush 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_balancer 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_locker 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log_expire 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_migrator 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 3/ 5 mds_quiesce 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 buffer 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 timer 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 filer 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 striper 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 objecter 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rados 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_mirror 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_replay 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_pwl 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 journaler 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objectcacher 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 immutable_obj_cache 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 client 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 osd 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 optracker 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objclass 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 filestore 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 journal 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 ms 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mon 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 0/10 monc 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 paxos 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 tp 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 auth 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 crypto 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 finisher 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 reserver 2026-02-24T22:21:40.225 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 heartbeatmap 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 perfcounter 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_sync 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_datacache 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_access 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dbstore 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_flight 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_lifecycle 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_restore 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw_notification 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_bucket_logging 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 javaclient 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 asok 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 throttle 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 refs 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 compressor 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore_compression 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluefs 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 bdev 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 kstore 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 4/ 5 rocksdb 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 fuse 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 2/ 5 mgr 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mgrc 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 dpdk 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 eventtrace 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 prioritycache 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 test 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephfs_mirror 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephsqlite 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 crimson_interrupt 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_onode 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_odata 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_omap 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_tm 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_t 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cleaner 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_epm 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_lba 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_fixedkv_tree 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cache 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_journal 2026-02-24T22:21:40.226 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_device 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_backref 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 alienstore 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mclock 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dedup 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cyanstore 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 ceph_exporter 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 memstore 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 trace 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 ceph_dedup 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: -2/-2 (syslog threshold) 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: 99/99 (stderr threshold) 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr:--- pthread ID / name mapping for recent threads --- 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: 7f7f3bfff640 / http_manager 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: 7f7fe1fdb640 / http_manager 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: 7f8001ffb640 / ms_dispatch 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: 7f8002ffd640 / io_context_pool 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: 7f80037fe640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: 7f8015ffb640 / ms_dispatch 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: 7f8024ff9640 / ms_dispatch 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: 7f80267fc640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: 7f8026ffd640 / admin_socket 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: 7f8027fff640 / msgr-worker-2 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: 7f802f7fe640 / ms_dispatch 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: 7f802ffff640 / msgr-worker-1 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: 7f8034952640 / msgr-worker-0 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: 7f8035153640 / io_context_pool 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: 7f803667da80 / radosgw-admin 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: max_recent 500 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: max_new 1000 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr: log_file /var/lib/ceph/crash/2026-02-24T22:21:40.151767Z_bfa2aeea-4679-4e2c-b289-30215048bd06/log 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr:--- end dump of recent events --- 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr:Aborted 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket a-bucket 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:21:40.227 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:21:40.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.187+0000 7fcb96331a80 4 Realm: () 2026-02-24T22:21:40.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.187+0000 7fcb96331a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:21:40.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.187+0000 7fcb96331a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:21:40.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.187+0000 7fcb96331a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:21:40.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.187+0000 7fcb96331a80 20 zonegroup default 2026-02-24T22:21:40.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.187+0000 7fcb96331a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:40.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.191+0000 7fcb96331a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:40.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.191+0000 7fcb96331a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:40.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.191+0000 7fcb96331a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:40.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.191+0000 7fcb96331a80 20 started sync module instance, tier type = 2026-02-24T22:21:40.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.191+0000 7fcb96331a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:21:40.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.191+0000 7fcb96331a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:21:40.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.191+0000 7fcb96331a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:21:40.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.191+0000 7fcb96331a80 5 note: GC not initialized 2026-02-24T22:21:40.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.191+0000 7fcb417e2640 20 reqs_thread_entry: start 2026-02-24T22:21:40.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.195+0000 7fcb8effd640 0 -- 10.20.193.46:0/2756787706 >> [v2:10.20.193.46:6826/3065971976,v1:10.20.193.46:6827/3065971976] conn(0x55a66068c1d0 msgr2=0x55a66068c5d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:40.228 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.199+0000 7fcb8effd640 0 -- 10.20.193.46:0/2756787706 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x55a66068a930 msgr2=0x55a660677320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:40.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.431+0000 7fcb96331a80 20 init_complete bucket index max shards: 1 2026-02-24T22:21:40.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.431+0000 7fcb96331a80 20 Filter name: none 2026-02-24T22:21:40.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.431+0000 7fcaab7fe640 20 reqs_thread_entry: start 2026-02-24T22:21:40.426 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.431+0000 7fcb96331a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:40.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.435+0000 7fcb96331a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:21:40.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.435+0000 7fcb96331a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:40.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.435+0000 7fcb96331a80 20 rados_obj.operate() r=0 bl.length=470 2026-02-24T22:21:40.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.435+0000 7fcb96331a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:21:40.427 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.435+0000 7fcb96331a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:40.428 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.435+0000 7fcb96331a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:21:40.428 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.435+0000 7fcb96331a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:40.428 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.435+0000 7fcb96331a80 20 rados_obj.operate() r=0 bl.length=470 2026-02-24T22:21:40.428 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.435+0000 7fcb96331a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:21:40.440 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:21:40.440 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:21:40.447 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin reshardlog list --bucket a-bucket 2026-02-24T22:21:40.462 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:21:40.463 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:21:40.480 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.487+0000 7ff341b08a80 4 Realm: () 2026-02-24T22:21:40.480 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.487+0000 7ff341b08a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:21:40.480 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.487+0000 7ff341b08a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:21:40.480 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.487+0000 7ff341b08a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:21:40.480 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.487+0000 7ff341b08a80 20 zonegroup default 2026-02-24T22:21:40.480 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.487+0000 7ff341b08a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:40.481 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.487+0000 7ff341b08a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:40.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.487+0000 7ff341b08a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:40.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.487+0000 7ff341b08a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:21:40.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.487+0000 7ff341b08a80 20 started sync module instance, tier type = 2026-02-24T22:21:40.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.487+0000 7ff341b08a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:21:40.483 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.491+0000 7ff33b7fe640 0 -- 10.20.193.46:0/618422400 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x560ea9971b00 msgr2=0x560ea9991f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:40.484 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.491+0000 7ff341b08a80 20 add_watcher() i=1 2026-02-24T22:21:40.484 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.491+0000 7ff341b08a80 20 add_watcher() i=3 2026-02-24T22:21:40.484 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.491+0000 7ff341b08a80 20 add_watcher() i=0 2026-02-24T22:21:40.484 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.491+0000 7ff341b08a80 20 add_watcher() i=6 2026-02-24T22:21:40.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.491+0000 7ff341b08a80 20 add_watcher() i=7 2026-02-24T22:21:40.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.491+0000 7ff341b08a80 20 add_watcher() i=5 2026-02-24T22:21:40.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.491+0000 7ff341b08a80 20 add_watcher() i=2 2026-02-24T22:21:40.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.491+0000 7ff341b08a80 20 add_watcher() i=4 2026-02-24T22:21:40.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.491+0000 7ff341b08a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:21:40.487 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.491+0000 7ff341b08a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:21:40.487 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.491+0000 7ff341b08a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:21:40.487 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.491+0000 7ff341b08a80 5 note: GC not initialized 2026-02-24T22:21:40.487 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.495+0000 7ff257fff640 20 reqs_thread_entry: start 2026-02-24T22:21:40.487 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.495+0000 7ff33b7fe640 0 -- 10.20.193.46:0/797992860 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x560ea9974d90 msgr2=0x560ea98ff2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:21:40.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.735+0000 7ff341b08a80 20 init_complete bucket index max shards: 1 2026-02-24T22:21:40.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.735+0000 7ff341b08a80 20 Filter name: none 2026-02-24T22:21:40.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.735+0000 7ff2567fc640 20 reqs_thread_entry: start 2026-02-24T22:21:40.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.735+0000 7ff341b08a80 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2026-02-24T22:21:40.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.735+0000 7ff341b08a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:40.730 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.735+0000 7ff341b08a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:21:40.730 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.735+0000 7ff341b08a80 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2026-02-24T22:21:40.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.735+0000 7ff341b08a80 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2026-02-24T22:21:40.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.735+0000 7ff341b08a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : miss 2026-02-24T22:21:40.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.735+0000 7ff341b08a80 20 rados->read ofs=0 len=0 2026-02-24T22:21:40.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.735+0000 7ff341b08a80 20 rados_obj.operate() r=0 bl.length=470 2026-02-24T22:21:40.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.735+0000 7ff341b08a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:21:40.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.735+0000 7ff341b08a80 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:21:40.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.735+0000 7ff341b08a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:21:40.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.735+0000 7ff341b08a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:21:40.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.735+0000 7ff341b08a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:21:40.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.735+0000 7ff341b08a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 2026-02-24T22:21:40.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.735+0000 7ff341b08a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 2026-02-24T22:21:40.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.735+0000 7ff341b08a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1 2026-02-24T22:21:40.732 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.735+0000 7ff341b08a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1 2026-02-24T22:21:40.732 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.735+0000 7ff341b08a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.2 2026-02-24T22:21:40.732 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.735+0000 7ff341b08a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.2 2026-02-24T22:21:40.734 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.739+0000 7ff341b08a80 20 remove_watcher() i=3 2026-02-24T22:21:40.734 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.739+0000 7ff341b08a80 2 removed watcher, disabling cache 2026-02-24T22:21:40.734 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.739+0000 7ff341b08a80 20 remove_watcher() i=1 2026-02-24T22:21:40.734 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.739+0000 7ff341b08a80 20 remove_watcher() i=2 2026-02-24T22:21:40.734 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.739+0000 7ff341b08a80 20 remove_watcher() i=0 2026-02-24T22:21:40.734 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.739+0000 7ff341b08a80 20 remove_watcher() i=6 2026-02-24T22:21:40.734 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.739+0000 7ff341b08a80 20 remove_watcher() i=4 2026-02-24T22:21:40.734 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.739+0000 7ff341b08a80 20 remove_watcher() i=7 2026-02-24T22:21:40.735 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:21:40.739+0000 7ff341b08a80 20 remove_watcher() i=5 2026-02-24T22:22:10.770 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket a-bucket --num-shards 4 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at logrecord_writes 2026-02-24T22:22:10.786 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:22:10.786 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:22:10.803 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:10.810+0000 7f5b01f84a80 4 Realm: () 2026-02-24T22:22:10.803 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:10.810+0000 7f5b01f84a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:22:10.803 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:10.810+0000 7f5b01f84a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:22:10.803 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:10.810+0000 7f5b01f84a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:22:10.803 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:10.810+0000 7f5b01f84a80 20 zonegroup default 2026-02-24T22:22:10.803 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:10.810+0000 7f5b01f84a80 20 rados->read ofs=0 len=0 2026-02-24T22:22:10.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:10.810+0000 7f5b01f84a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:22:10.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:10.810+0000 7f5b01f84a80 20 rados->read ofs=0 len=0 2026-02-24T22:22:10.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:10.810+0000 7f5b01f84a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:22:10.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:10.810+0000 7f5b01f84a80 20 started sync module instance, tier type = 2026-02-24T22:22:10.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:10.810+0000 7f5b01f84a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:22:10.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:10.814+0000 7f5afb7fe640 0 -- 10.20.193.46:0/1549073879 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x55b13851e0a0 msgr2=0x55b13853e520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:22:10.807 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:10.814+0000 7f5b01f84a80 20 add_watcher() i=1 2026-02-24T22:22:10.807 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:10.814+0000 7f5b01f84a80 20 add_watcher() i=5 2026-02-24T22:22:10.807 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:10.814+0000 7f5b01f84a80 20 add_watcher() i=0 2026-02-24T22:22:10.808 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:10.814+0000 7f5b01f84a80 20 add_watcher() i=6 2026-02-24T22:22:10.808 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:10.814+0000 7f5b01f84a80 20 add_watcher() i=2 2026-02-24T22:22:10.808 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:10.814+0000 7f5b01f84a80 20 add_watcher() i=4 2026-02-24T22:22:10.808 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:10.814+0000 7f5b01f84a80 20 add_watcher() i=3 2026-02-24T22:22:10.809 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:10.814+0000 7f5b01f84a80 20 add_watcher() i=7 2026-02-24T22:22:10.809 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:10.814+0000 7f5b01f84a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:22:10.810 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:10.814+0000 7f5af3fff640 0 -- 10.20.193.46:0/3650014496 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f5ad009d3f0 msgr2=0x7f5ad00bd7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:22:11.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.018+0000 7f5b01f84a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:22:11.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.018+0000 7f5b01f84a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:22:11.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 note: GC not initialized 2026-02-24T22:22:11.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.018+0000 7f5aacfe1640 20 reqs_thread_entry: start 2026-02-24T22:22:11.049 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.054+0000 7f5b01f84a80 20 init_complete bucket index max shards: 1 2026-02-24T22:22:11.049 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.054+0000 7f5b01f84a80 20 Filter name: none 2026-02-24T22:22:11.049 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.054+0000 7f5a16ffd640 20 reqs_thread_entry: start 2026-02-24T22:22:11.049 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2026-02-24T22:22:11.049 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.054+0000 7f5b01f84a80 20 rados->read ofs=0 len=0 2026-02-24T22:22:11.050 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.054+0000 7f5b01f84a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:22:11.050 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2026-02-24T22:22:11.050 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2026-02-24T22:22:11.050 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : miss 2026-02-24T22:22:11.050 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.054+0000 7f5b01f84a80 20 rados->read ofs=0 len=0 2026-02-24T22:22:11.050 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.054+0000 7f5b01f84a80 20 rados_obj.operate() r=0 bl.length=470 2026-02-24T22:22:11.050 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:22:11.050 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:22:11.050 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:22:11.050 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:22:11.050 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:22:11.051 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.058+0000 7f5b01f84a80 10 init_target_layout removing existing target index objects from a previous reshard attempt 2026-02-24T22:22:11.056 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.062+0000 7f5b01f84a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:22:11.056 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.062+0000 7f5b01f84a80 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:22:11.056 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.062+0000 7f5b01f84a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:22:11.056 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.062+0000 7f5b01f84a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518, ofs0, ns] 2026-02-24T22:22:11.056 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.062+0000 7f5ad6ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959946 cookie 94219642602960 notifier 8571 bl.length()=929 2026-02-24T22:22:11.056 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.062+0000 7f5ad6ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:22:11.056 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.062+0000 7f5ad6ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:22:11.056 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.062+0000 7f5ad6ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:22:11.056 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.062+0000 7f5b01f84a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_sources= new_sources= 2026-02-24T22:22:11.057 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.062+0000 7f5b01f84a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential sources added=[] removed=[] 2026-02-24T22:22:11.057 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.062+0000 7f5b01f84a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_dests= new_dests= 2026-02-24T22:22:11.057 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.062+0000 7f5b01f84a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential dests added=[] removed=[] 2026-02-24T22:22:11.057 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.062+0000 7f5ad6ffd640 20 trim reshard logs: write on '.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1' needs retry: Success 2026-02-24T22:22:11.058 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f5b01f84a80 time 2026-02-24T22:22:11.067093+0000 2026-02-24T22:22:11.058 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:22:11.059 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:22:11.059 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f5b053c6eb1] 2026-02-24T22:22:11.059 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x55b12bb99b9f] 2026-02-24T22:22:11.059 INFO:tasks.workunit.client.0.trial046.stderr: 3: radosgw-admin(+0x1714451) [0x55b12cda8451] 2026-02-24T22:22:11.059 INFO:tasks.workunit.client.0.trial046.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x55b12c305202] 2026-02-24T22:22:11.059 INFO:tasks.workunit.client.0.trial046.stderr: 5: main() 2026-02-24T22:22:11.059 INFO:tasks.workunit.client.0.trial046.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f5b046c6d90] 2026-02-24T22:22:11.059 INFO:tasks.workunit.client.0.trial046.stderr: 7: __libc_start_main() 2026-02-24T22:22:11.059 INFO:tasks.workunit.client.0.trial046.stderr: 8: _start() 2026-02-24T22:22:11.059 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.066+0000 7f5b01f84a80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f5b01f84a80 time 2026-02-24T22:22:11.067093+0000 2026-02-24T22:22:11.060 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:22:11.060 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.060 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:22:11.060 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f5b053c6eb1] 2026-02-24T22:22:11.060 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x55b12bb99b9f] 2026-02-24T22:22:11.060 INFO:tasks.workunit.client.0.trial046.stderr: 3: radosgw-admin(+0x1714451) [0x55b12cda8451] 2026-02-24T22:22:11.060 INFO:tasks.workunit.client.0.trial046.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x55b12c305202] 2026-02-24T22:22:11.060 INFO:tasks.workunit.client.0.trial046.stderr: 5: main() 2026-02-24T22:22:11.060 INFO:tasks.workunit.client.0.trial046.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f5b046c6d90] 2026-02-24T22:22:11.060 INFO:tasks.workunit.client.0.trial046.stderr: 7: __libc_start_main() 2026-02-24T22:22:11.060 INFO:tasks.workunit.client.0.trial046.stderr: 8: _start() 2026-02-24T22:22:11.060 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.060 INFO:tasks.workunit.client.0.trial046.stderr:*** Caught signal (Aborted) ** 2026-02-24T22:22:11.060 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f5b01f84a80 thread_name:radosgw-admin 2026-02-24T22:22:11.076 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:22:11.076 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f5b046df520] 2026-02-24T22:22:11.076 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:22:11.076 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:22:11.076 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:22:11.077 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f5b053c6f10] 2026-02-24T22:22:11.077 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x55b12bb99b9f] 2026-02-24T22:22:11.077 INFO:tasks.workunit.client.0.trial046.stderr: 7: radosgw-admin(+0x1714451) [0x55b12cda8451] 2026-02-24T22:22:11.077 INFO:tasks.workunit.client.0.trial046.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x55b12c305202] 2026-02-24T22:22:11.077 INFO:tasks.workunit.client.0.trial046.stderr: 9: main() 2026-02-24T22:22:11.077 INFO:tasks.workunit.client.0.trial046.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f5b046c6d90] 2026-02-24T22:22:11.077 INFO:tasks.workunit.client.0.trial046.stderr: 11: __libc_start_main() 2026-02-24T22:22:11.077 INFO:tasks.workunit.client.0.trial046.stderr: 12: _start() 2026-02-24T22:22:11.077 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.066+0000 7f5b01f84a80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:22:11.077 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f5b01f84a80 thread_name:radosgw-admin 2026-02-24T22:22:11.077 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.077 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:22:11.077 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f5b046df520] 2026-02-24T22:22:11.077 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:22:11.077 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:22:11.078 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:22:11.078 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f5b053c6f10] 2026-02-24T22:22:11.078 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x55b12bb99b9f] 2026-02-24T22:22:11.078 INFO:tasks.workunit.client.0.trial046.stderr: 7: radosgw-admin(+0x1714451) [0x55b12cda8451] 2026-02-24T22:22:11.078 INFO:tasks.workunit.client.0.trial046.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x55b12c305202] 2026-02-24T22:22:11.078 INFO:tasks.workunit.client.0.trial046.stderr: 9: main() 2026-02-24T22:22:11.078 INFO:tasks.workunit.client.0.trial046.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f5b046c6d90] 2026-02-24T22:22:11.078 INFO:tasks.workunit.client.0.trial046.stderr: 11: __libc_start_main() 2026-02-24T22:22:11.078 INFO:tasks.workunit.client.0.trial046.stderr: 12: _start() 2026-02-24T22:22:11.078 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:22:11.078 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.078 INFO:tasks.workunit.client.0.trial046.stderr:--- begin dump of recent events --- 2026-02-24T22:22:11.078 INFO:tasks.workunit.client.0.trial046.stderr: -499> 2026-02-24T22:22:10.794+0000 7f5afb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:11.078 INFO:tasks.workunit.client.0.trial046.stderr: -498> 2026-02-24T22:22:10.794+0000 7f5afb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:11.078 INFO:tasks.workunit.client.0.trial046.stderr: -497> 2026-02-24T22:22:10.794+0000 7f5afb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:11.078 INFO:tasks.workunit.client.0.trial046.stderr: -496> 2026-02-24T22:22:10.794+0000 7f5af3fff640 10 monclient(hunting): get_auth_request con 0x55b137f57fe0 auth_method 0 2026-02-24T22:22:11.079 INFO:tasks.workunit.client.0.trial046.stderr: -495> 2026-02-24T22:22:10.794+0000 7f5af3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:11.079 INFO:tasks.workunit.client.0.trial046.stderr: -494> 2026-02-24T22:22:10.794+0000 7f5af3fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:11.079 INFO:tasks.workunit.client.0.trial046.stderr: -493> 2026-02-24T22:22:10.794+0000 7f5af3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:11.079 INFO:tasks.workunit.client.0.trial046.stderr: -492> 2026-02-24T22:22:10.794+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:11.079 INFO:tasks.workunit.client.0.trial046.stderr: -491> 2026-02-24T22:22:10.794+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:11.079 INFO:tasks.workunit.client.0.trial046.stderr: -490> 2026-02-24T22:22:10.794+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:11.079 INFO:tasks.workunit.client.0.trial046.stderr: -489> 2026-02-24T22:22:10.798+0000 7f5af3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:11.079 INFO:tasks.workunit.client.0.trial046.stderr: -488> 2026-02-24T22:22:10.798+0000 7f5af3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:11.079 INFO:tasks.workunit.client.0.trial046.stderr: -487> 2026-02-24T22:22:10.798+0000 7f5af3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:11.079 INFO:tasks.workunit.client.0.trial046.stderr: -486> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_done global_id 8559 payload 274 2026-02-24T22:22:11.079 INFO:tasks.workunit.client.0.trial046.stderr: -485> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient: _finish_hunting 0 2026-02-24T22:22:11.079 INFO:tasks.workunit.client.0.trial046.stderr: -484> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 1 monclient: found mon.noname-a 2026-02-24T22:22:11.079 INFO:tasks.workunit.client.0.trial046.stderr: -483> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:22:11.079 INFO:tasks.workunit.client.0.trial046.stderr: -482> 2026-02-24T22:22:10.798+0000 7f5af1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:11.079 INFO:tasks.workunit.client.0.trial046.stderr: -481> 2026-02-24T22:22:10.798+0000 7f5af1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:22:11.079 INFO:tasks.workunit.client.0.trial046.stderr: -480> 2026-02-24T22:22:10.798+0000 7f5af1ffb640 10 monclient: dump: 2026-02-24T22:22:11.079 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:11.079 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:11.080 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.080 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.080 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:11.080 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.080 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:11.080 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:11.080 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:11.080 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.080 INFO:tasks.workunit.client.0.trial046.stderr: -479> 2026-02-24T22:22:10.798+0000 7f5af1ffb640 10 monclient: _finish_auth 0 2026-02-24T22:22:11.080 INFO:tasks.workunit.client.0.trial046.stderr: -478> 2026-02-24T22:22:10.798+0000 7f5af1ffb640 10 monclient: _check_auth_tickets 2026-02-24T22:22:11.080 INFO:tasks.workunit.client.0.trial046.stderr: -477> 2026-02-24T22:22:10.798+0000 7f5af1ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T22:22:11.080 INFO:tasks.workunit.client.0.trial046.stderr: -476> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 monclient: authenticate success, global_id 8559 2026-02-24T22:22:11.080 INFO:tasks.workunit.client.0.trial046.stderr: -475> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient: get_monmap_and_config success 2026-02-24T22:22:11.080 INFO:tasks.workunit.client.0.trial046.stderr: -474> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 4 set_mon_vals no callback set 2026-02-24T22:22:11.080 INFO:tasks.workunit.client.0.trial046.stderr: -473> 2026-02-24T22:22:10.798+0000 7f5af1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:11.080 INFO:tasks.workunit.client.0.trial046.stderr: -472> 2026-02-24T22:22:10.798+0000 7f5af1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:22:11.081 INFO:tasks.workunit.client.0.trial046.stderr: -471> 2026-02-24T22:22:10.798+0000 7f5af1ffb640 10 monclient: dump: 2026-02-24T22:22:11.081 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:11.081 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:11.081 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.081 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.081 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:11.081 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.081 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:11.081 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:11.081 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:11.081 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.081 INFO:tasks.workunit.client.0.trial046.stderr: -470> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient: shutdown 2026-02-24T22:22:11.081 INFO:tasks.workunit.client.0.trial046.stderr: -469> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) unregister_commands rotate-key 2026-02-24T22:22:11.081 INFO:tasks.workunit.client.0.trial046.stderr: -468> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient: build_initial_monmap 2026-02-24T22:22:11.081 INFO:tasks.workunit.client.0.trial046.stderr: -467> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 1 build_initial for_mkfs: 0 2026-02-24T22:22:11.081 INFO:tasks.workunit.client.0.trial046.stderr: -466> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient: monmap: 2026-02-24T22:22:11.082 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:22:11.082 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:22:11.082 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:22:11.082 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:22:11.082 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:22:11.082 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.082 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:22:11.082 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:22:11.082 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:22:11.082 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.082 INFO:tasks.workunit.client.0.trial046.stderr: -465> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding auth protocol: cephx 2026-02-24T22:22:11.082 INFO:tasks.workunit.client.0.trial046.stderr: -464> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding auth protocol: cephx 2026-02-24T22:22:11.082 INFO:tasks.workunit.client.0.trial046.stderr: -463> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding auth protocol: cephx 2026-02-24T22:22:11.082 INFO:tasks.workunit.client.0.trial046.stderr: -462> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding con mode: secure 2026-02-24T22:22:11.082 INFO:tasks.workunit.client.0.trial046.stderr: -461> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding con mode: crc 2026-02-24T22:22:11.082 INFO:tasks.workunit.client.0.trial046.stderr: -460> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding con mode: secure 2026-02-24T22:22:11.082 INFO:tasks.workunit.client.0.trial046.stderr: -459> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding con mode: crc 2026-02-24T22:22:11.083 INFO:tasks.workunit.client.0.trial046.stderr: -458> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding con mode: secure 2026-02-24T22:22:11.083 INFO:tasks.workunit.client.0.trial046.stderr: -457> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding con mode: crc 2026-02-24T22:22:11.083 INFO:tasks.workunit.client.0.trial046.stderr: -456> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding con mode: crc 2026-02-24T22:22:11.083 INFO:tasks.workunit.client.0.trial046.stderr: -455> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding con mode: secure 2026-02-24T22:22:11.083 INFO:tasks.workunit.client.0.trial046.stderr: -454> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding con mode: crc 2026-02-24T22:22:11.083 INFO:tasks.workunit.client.0.trial046.stderr: -453> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding con mode: secure 2026-02-24T22:22:11.083 INFO:tasks.workunit.client.0.trial046.stderr: -452> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding con mode: crc 2026-02-24T22:22:11.083 INFO:tasks.workunit.client.0.trial046.stderr: -451> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding con mode: secure 2026-02-24T22:22:11.083 INFO:tasks.workunit.client.0.trial046.stderr: -450> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:11.083 INFO:tasks.workunit.client.0.trial046.stderr: -449> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 1 librados: starting msgr at 2026-02-24T22:22:11.083 INFO:tasks.workunit.client.0.trial046.stderr: -448> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 1 librados: starting objecter 2026-02-24T22:22:11.083 INFO:tasks.workunit.client.0.trial046.stderr: -447> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command objecter_requests hook 0x55b138009280 2026-02-24T22:22:11.083 INFO:tasks.workunit.client.0.trial046.stderr: -446> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 1 librados: setting wanted keys 2026-02-24T22:22:11.083 INFO:tasks.workunit.client.0.trial046.stderr: -445> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 1 librados: calling monclient init 2026-02-24T22:22:11.083 INFO:tasks.workunit.client.0.trial046.stderr: -444> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient: init 2026-02-24T22:22:11.083 INFO:tasks.workunit.client.0.trial046.stderr: -443> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding auth protocol: cephx 2026-02-24T22:22:11.083 INFO:tasks.workunit.client.0.trial046.stderr: -442> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding auth protocol: cephx 2026-02-24T22:22:11.084 INFO:tasks.workunit.client.0.trial046.stderr: -441> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding auth protocol: cephx 2026-02-24T22:22:11.084 INFO:tasks.workunit.client.0.trial046.stderr: -440> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding con mode: secure 2026-02-24T22:22:11.084 INFO:tasks.workunit.client.0.trial046.stderr: -439> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding con mode: crc 2026-02-24T22:22:11.084 INFO:tasks.workunit.client.0.trial046.stderr: -438> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding con mode: secure 2026-02-24T22:22:11.084 INFO:tasks.workunit.client.0.trial046.stderr: -437> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding con mode: crc 2026-02-24T22:22:11.084 INFO:tasks.workunit.client.0.trial046.stderr: -436> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding con mode: secure 2026-02-24T22:22:11.084 INFO:tasks.workunit.client.0.trial046.stderr: -435> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding con mode: crc 2026-02-24T22:22:11.084 INFO:tasks.workunit.client.0.trial046.stderr: -434> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding con mode: crc 2026-02-24T22:22:11.084 INFO:tasks.workunit.client.0.trial046.stderr: -433> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding con mode: secure 2026-02-24T22:22:11.084 INFO:tasks.workunit.client.0.trial046.stderr: -432> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding con mode: crc 2026-02-24T22:22:11.084 INFO:tasks.workunit.client.0.trial046.stderr: -431> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding con mode: secure 2026-02-24T22:22:11.084 INFO:tasks.workunit.client.0.trial046.stderr: -430> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding con mode: crc 2026-02-24T22:22:11.084 INFO:tasks.workunit.client.0.trial046.stderr: -429> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding con mode: secure 2026-02-24T22:22:11.084 INFO:tasks.workunit.client.0.trial046.stderr: -428> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:11.084 INFO:tasks.workunit.client.0.trial046.stderr: -427> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:11.084 INFO:tasks.workunit.client.0.trial046.stderr: -426> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command rotate-key hook 0x55b137fc9e08 2026-02-24T22:22:11.084 INFO:tasks.workunit.client.0.trial046.stderr: -425> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient: _reopen_session rank -1 2026-02-24T22:22:11.085 INFO:tasks.workunit.client.0.trial046.stderr: -424> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:22:11.085 INFO:tasks.workunit.client.0.trial046.stderr: -423> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient(hunting): picked mon.noname-c con 0x55b137f11c30 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:22:11.085 INFO:tasks.workunit.client.0.trial046.stderr: -422> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient(hunting): picked mon.noname-a con 0x55b137f57fe0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:22:11.085 INFO:tasks.workunit.client.0.trial046.stderr: -421> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient(hunting): picked mon.noname-b con 0x55b138000b00 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:22:11.085 INFO:tasks.workunit.client.0.trial046.stderr: -420> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:11.085 INFO:tasks.workunit.client.0.trial046.stderr: -419> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:11.085 INFO:tasks.workunit.client.0.trial046.stderr: -418> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:11.085 INFO:tasks.workunit.client.0.trial046.stderr: -417> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient(hunting): _renew_subs 2026-02-24T22:22:11.085 INFO:tasks.workunit.client.0.trial046.stderr: -416> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient(hunting): authenticate will time out at 2100.905167s 2026-02-24T22:22:11.085 INFO:tasks.workunit.client.0.trial046.stderr: -415> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient(hunting): get_auth_request con 0x55b137f11c30 auth_method 0 2026-02-24T22:22:11.085 INFO:tasks.workunit.client.0.trial046.stderr: -414> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:11.085 INFO:tasks.workunit.client.0.trial046.stderr: -413> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:11.085 INFO:tasks.workunit.client.0.trial046.stderr: -412> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:11.085 INFO:tasks.workunit.client.0.trial046.stderr: -411> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:11.085 INFO:tasks.workunit.client.0.trial046.stderr: -410> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:11.085 INFO:tasks.workunit.client.0.trial046.stderr: -409> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:11.086 INFO:tasks.workunit.client.0.trial046.stderr: -408> 2026-02-24T22:22:10.798+0000 7f5af3fff640 10 monclient(hunting): get_auth_request con 0x55b137f57fe0 auth_method 0 2026-02-24T22:22:11.086 INFO:tasks.workunit.client.0.trial046.stderr: -407> 2026-02-24T22:22:10.798+0000 7f5af3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:11.086 INFO:tasks.workunit.client.0.trial046.stderr: -406> 2026-02-24T22:22:10.798+0000 7f5af3fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:11.086 INFO:tasks.workunit.client.0.trial046.stderr: -405> 2026-02-24T22:22:10.798+0000 7f5af3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:11.086 INFO:tasks.workunit.client.0.trial046.stderr: -404> 2026-02-24T22:22:10.798+0000 7f5af3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:11.086 INFO:tasks.workunit.client.0.trial046.stderr: -403> 2026-02-24T22:22:10.798+0000 7f5af3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:11.086 INFO:tasks.workunit.client.0.trial046.stderr: -402> 2026-02-24T22:22:10.798+0000 7f5af3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:11.086 INFO:tasks.workunit.client.0.trial046.stderr: -401> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_done global_id 7985 payload 824 2026-02-24T22:22:11.086 INFO:tasks.workunit.client.0.trial046.stderr: -400> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient: _finish_hunting 0 2026-02-24T22:22:11.086 INFO:tasks.workunit.client.0.trial046.stderr: -399> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 1 monclient: found mon.noname-c 2026-02-24T22:22:11.086 INFO:tasks.workunit.client.0.trial046.stderr: -398> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:22:11.086 INFO:tasks.workunit.client.0.trial046.stderr: -397> 2026-02-24T22:22:10.798+0000 7f5ae3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:11.086 INFO:tasks.workunit.client.0.trial046.stderr: -396> 2026-02-24T22:22:10.798+0000 7f5ae3fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:22:11.086 INFO:tasks.workunit.client.0.trial046.stderr: -395> 2026-02-24T22:22:10.798+0000 7f5ae3fff640 10 monclient: dump: 2026-02-24T22:22:11.086 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:11.086 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:11.086 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.087 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.087 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:11.087 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.087 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:11.087 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:11.087 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:11.087 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.087 INFO:tasks.workunit.client.0.trial046.stderr: -394> 2026-02-24T22:22:10.798+0000 7f5ae3fff640 10 monclient: _finish_auth 0 2026-02-24T22:22:11.087 INFO:tasks.workunit.client.0.trial046.stderr: -393> 2026-02-24T22:22:10.798+0000 7f5ae3fff640 10 monclient: _check_auth_tickets 2026-02-24T22:22:11.087 INFO:tasks.workunit.client.0.trial046.stderr: -392> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 monclient: authenticate success, global_id 7985 2026-02-24T22:22:11.087 INFO:tasks.workunit.client.0.trial046.stderr: -391> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient: _renew_subs 2026-02-24T22:22:11.087 INFO:tasks.workunit.client.0.trial046.stderr: -390> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:22:11.087 INFO:tasks.workunit.client.0.trial046.stderr: -389> 2026-02-24T22:22:10.798+0000 7f5ae3fff640 10 monclient: handle_config config(0 keys) 2026-02-24T22:22:11.087 INFO:tasks.workunit.client.0.trial046.stderr: -388> 2026-02-24T22:22:10.798+0000 7f5ae3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:11.087 INFO:tasks.workunit.client.0.trial046.stderr: -387> 2026-02-24T22:22:10.798+0000 7f5ae3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:22:11.087 INFO:tasks.workunit.client.0.trial046.stderr: -386> 2026-02-24T22:22:10.798+0000 7f5ae3fff640 10 monclient: dump: 2026-02-24T22:22:11.087 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:11.088 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:11.088 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.088 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.088 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:11.088 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.088 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:11.088 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:11.088 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:11.088 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.088 INFO:tasks.workunit.client.0.trial046.stderr: -385> 2026-02-24T22:22:10.798+0000 7f5af1ffb640 4 set_mon_vals no callback set 2026-02-24T22:22:11.088 INFO:tasks.workunit.client.0.trial046.stderr: -384> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient: _renew_subs 2026-02-24T22:22:11.088 INFO:tasks.workunit.client.0.trial046.stderr: -383> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:22:11.088 INFO:tasks.workunit.client.0.trial046.stderr: -382> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 1 librados: init done 2026-02-24T22:22:11.088 INFO:tasks.workunit.client.0.trial046.stderr: -381> 2026-02-24T22:22:10.798+0000 7f5ae3fff640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:22:11.088 INFO:tasks.workunit.client.0.trial046.stderr: -380> 2026-02-24T22:22:10.798+0000 7f5ae3fff640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:22:11.088 INFO:tasks.workunit.client.0.trial046.stderr: -379> 2026-02-24T22:22:10.798+0000 7f5ae3fff640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:22:11.088 INFO:tasks.workunit.client.0.trial046.stderr: -378> 2026-02-24T22:22:10.798+0000 7f5af3fff640 10 monclient: get_auth_request con 0x7f5adc070730 auth_method 0 2026-02-24T22:22:11.089 INFO:tasks.workunit.client.0.trial046.stderr: -377> 2026-02-24T22:22:10.798+0000 7f5afbfff640 10 monclient: get_auth_request con 0x55b138001410 auth_method 0 2026-02-24T22:22:11.089 INFO:tasks.workunit.client.0.trial046.stderr: -376> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient: get_auth_request con 0x55b1380579a0 auth_method 0 2026-02-24T22:22:11.089 INFO:tasks.workunit.client.0.trial046.stderr: -375> 2026-02-24T22:22:10.802+0000 7f5af3fff640 10 monclient: get_auth_request con 0x55b1380591a0 auth_method 0 2026-02-24T22:22:11.089 INFO:tasks.workunit.client.0.trial046.stderr: -374> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient: build_initial_monmap 2026-02-24T22:22:11.089 INFO:tasks.workunit.client.0.trial046.stderr: -373> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 1 build_initial for_mkfs: 0 2026-02-24T22:22:11.089 INFO:tasks.workunit.client.0.trial046.stderr: -372> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient: monmap: 2026-02-24T22:22:11.089 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:22:11.089 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:22:11.089 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:22:11.089 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:22:11.089 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:22:11.089 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.089 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:22:11.089 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:22:11.089 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:22:11.090 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.090 INFO:tasks.workunit.client.0.trial046.stderr: -371> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding auth protocol: cephx 2026-02-24T22:22:11.090 INFO:tasks.workunit.client.0.trial046.stderr: -370> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding auth protocol: cephx 2026-02-24T22:22:11.090 INFO:tasks.workunit.client.0.trial046.stderr: -369> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding auth protocol: cephx 2026-02-24T22:22:11.090 INFO:tasks.workunit.client.0.trial046.stderr: -368> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding con mode: secure 2026-02-24T22:22:11.090 INFO:tasks.workunit.client.0.trial046.stderr: -367> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding con mode: crc 2026-02-24T22:22:11.090 INFO:tasks.workunit.client.0.trial046.stderr: -366> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding con mode: secure 2026-02-24T22:22:11.090 INFO:tasks.workunit.client.0.trial046.stderr: -365> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding con mode: crc 2026-02-24T22:22:11.090 INFO:tasks.workunit.client.0.trial046.stderr: -364> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding con mode: secure 2026-02-24T22:22:11.090 INFO:tasks.workunit.client.0.trial046.stderr: -363> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding con mode: crc 2026-02-24T22:22:11.090 INFO:tasks.workunit.client.0.trial046.stderr: -362> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding con mode: crc 2026-02-24T22:22:11.090 INFO:tasks.workunit.client.0.trial046.stderr: -361> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding con mode: secure 2026-02-24T22:22:11.090 INFO:tasks.workunit.client.0.trial046.stderr: -360> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding con mode: crc 2026-02-24T22:22:11.090 INFO:tasks.workunit.client.0.trial046.stderr: -359> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding con mode: secure 2026-02-24T22:22:11.090 INFO:tasks.workunit.client.0.trial046.stderr: -358> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding con mode: crc 2026-02-24T22:22:11.090 INFO:tasks.workunit.client.0.trial046.stderr: -357> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding con mode: secure 2026-02-24T22:22:11.091 INFO:tasks.workunit.client.0.trial046.stderr: -356> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:11.091 INFO:tasks.workunit.client.0.trial046.stderr: -355> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command objecter_requests cmddesc objecter_requests hook 0x55b138060b60 EEXIST 2026-02-24T22:22:11.091 INFO:tasks.workunit.client.0.trial046.stderr: -354> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient: init 2026-02-24T22:22:11.091 INFO:tasks.workunit.client.0.trial046.stderr: -353> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding auth protocol: cephx 2026-02-24T22:22:11.091 INFO:tasks.workunit.client.0.trial046.stderr: -352> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding auth protocol: cephx 2026-02-24T22:22:11.091 INFO:tasks.workunit.client.0.trial046.stderr: -351> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding auth protocol: cephx 2026-02-24T22:22:11.091 INFO:tasks.workunit.client.0.trial046.stderr: -350> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding con mode: secure 2026-02-24T22:22:11.091 INFO:tasks.workunit.client.0.trial046.stderr: -349> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding con mode: crc 2026-02-24T22:22:11.091 INFO:tasks.workunit.client.0.trial046.stderr: -348> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding con mode: secure 2026-02-24T22:22:11.091 INFO:tasks.workunit.client.0.trial046.stderr: -347> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding con mode: crc 2026-02-24T22:22:11.091 INFO:tasks.workunit.client.0.trial046.stderr: -346> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding con mode: secure 2026-02-24T22:22:11.091 INFO:tasks.workunit.client.0.trial046.stderr: -345> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding con mode: crc 2026-02-24T22:22:11.091 INFO:tasks.workunit.client.0.trial046.stderr: -344> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding con mode: crc 2026-02-24T22:22:11.091 INFO:tasks.workunit.client.0.trial046.stderr: -343> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding con mode: secure 2026-02-24T22:22:11.091 INFO:tasks.workunit.client.0.trial046.stderr: -342> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding con mode: crc 2026-02-24T22:22:11.091 INFO:tasks.workunit.client.0.trial046.stderr: -341> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding con mode: secure 2026-02-24T22:22:11.091 INFO:tasks.workunit.client.0.trial046.stderr: -340> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding con mode: crc 2026-02-24T22:22:11.091 INFO:tasks.workunit.client.0.trial046.stderr: -339> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding con mode: secure 2026-02-24T22:22:11.092 INFO:tasks.workunit.client.0.trial046.stderr: -338> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:11.092 INFO:tasks.workunit.client.0.trial046.stderr: -337> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:11.092 INFO:tasks.workunit.client.0.trial046.stderr: -336> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command rotate-key cmddesc rotate-key hook 0x55b138059d50 EEXIST 2026-02-24T22:22:11.092 INFO:tasks.workunit.client.0.trial046.stderr: -335> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient: _reopen_session rank -1 2026-02-24T22:22:11.092 INFO:tasks.workunit.client.0.trial046.stderr: -334> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T22:22:11.092 INFO:tasks.workunit.client.0.trial046.stderr: -333> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient(hunting): picked mon.noname-b con 0x55b1381b3fc0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:22:11.092 INFO:tasks.workunit.client.0.trial046.stderr: -332> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient(hunting): picked mon.noname-a con 0x55b1381b0c50 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:22:11.092 INFO:tasks.workunit.client.0.trial046.stderr: -331> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient(hunting): picked mon.noname-c con 0x55b1381ad1a0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:22:11.092 INFO:tasks.workunit.client.0.trial046.stderr: -330> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:11.092 INFO:tasks.workunit.client.0.trial046.stderr: -329> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:11.092 INFO:tasks.workunit.client.0.trial046.stderr: -328> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:11.092 INFO:tasks.workunit.client.0.trial046.stderr: -327> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient(hunting): _renew_subs 2026-02-24T22:22:11.092 INFO:tasks.workunit.client.0.trial046.stderr: -326> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient(hunting): authenticate will time out at 2100.910986s 2026-02-24T22:22:11.092 INFO:tasks.workunit.client.0.trial046.stderr: -325> 2026-02-24T22:22:10.802+0000 7f5af3fff640 10 monclient(hunting): get_auth_request con 0x55b1381b0c50 auth_method 0 2026-02-24T22:22:11.092 INFO:tasks.workunit.client.0.trial046.stderr: -324> 2026-02-24T22:22:10.802+0000 7f5af3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:11.092 INFO:tasks.workunit.client.0.trial046.stderr: -323> 2026-02-24T22:22:10.802+0000 7f5af3fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:11.092 INFO:tasks.workunit.client.0.trial046.stderr: -322> 2026-02-24T22:22:10.802+0000 7f5af3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:11.092 INFO:tasks.workunit.client.0.trial046.stderr: -321> 2026-02-24T22:22:10.802+0000 7f5afbfff640 10 monclient(hunting): get_auth_request con 0x55b1381ad1a0 auth_method 0 2026-02-24T22:22:11.092 INFO:tasks.workunit.client.0.trial046.stderr: -320> 2026-02-24T22:22:10.802+0000 7f5afbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:11.093 INFO:tasks.workunit.client.0.trial046.stderr: -319> 2026-02-24T22:22:10.802+0000 7f5afbfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:11.093 INFO:tasks.workunit.client.0.trial046.stderr: -318> 2026-02-24T22:22:10.802+0000 7f5afbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:11.093 INFO:tasks.workunit.client.0.trial046.stderr: -317> 2026-02-24T22:22:10.802+0000 7f5af3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:11.093 INFO:tasks.workunit.client.0.trial046.stderr: -316> 2026-02-24T22:22:10.802+0000 7f5af3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:11.093 INFO:tasks.workunit.client.0.trial046.stderr: -315> 2026-02-24T22:22:10.802+0000 7f5af3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:11.093 INFO:tasks.workunit.client.0.trial046.stderr: -314> 2026-02-24T22:22:10.802+0000 7f5afbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:11.093 INFO:tasks.workunit.client.0.trial046.stderr: -313> 2026-02-24T22:22:10.802+0000 7f5afbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:11.093 INFO:tasks.workunit.client.0.trial046.stderr: -312> 2026-02-24T22:22:10.802+0000 7f5afbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:11.093 INFO:tasks.workunit.client.0.trial046.stderr: -311> 2026-02-24T22:22:10.802+0000 7f5afb7fe640 10 monclient(hunting): get_auth_request con 0x55b1381b3fc0 auth_method 0 2026-02-24T22:22:11.093 INFO:tasks.workunit.client.0.trial046.stderr: -310> 2026-02-24T22:22:10.802+0000 7f5afb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:11.093 INFO:tasks.workunit.client.0.trial046.stderr: -309> 2026-02-24T22:22:10.802+0000 7f5afb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:11.093 INFO:tasks.workunit.client.0.trial046.stderr: -308> 2026-02-24T22:22:10.802+0000 7f5afb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:11.093 INFO:tasks.workunit.client.0.trial046.stderr: -307> 2026-02-24T22:22:10.802+0000 7f5af3fff640 10 monclient(hunting): handle_auth_done global_id 8565 payload 824 2026-02-24T22:22:11.093 INFO:tasks.workunit.client.0.trial046.stderr: -306> 2026-02-24T22:22:10.802+0000 7f5af3fff640 10 monclient: _finish_hunting 0 2026-02-24T22:22:11.093 INFO:tasks.workunit.client.0.trial046.stderr: -305> 2026-02-24T22:22:10.802+0000 7f5af3fff640 1 monclient: found mon.noname-a 2026-02-24T22:22:11.093 INFO:tasks.workunit.client.0.trial046.stderr: -304> 2026-02-24T22:22:10.802+0000 7f5af3fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:22:11.093 INFO:tasks.workunit.client.0.trial046.stderr: -303> 2026-02-24T22:22:10.802+0000 7f5ae17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:11.093 INFO:tasks.workunit.client.0.trial046.stderr: -302> 2026-02-24T22:22:10.802+0000 7f5ae17fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:22:11.094 INFO:tasks.workunit.client.0.trial046.stderr: -301> 2026-02-24T22:22:10.802+0000 7f5ae17fa640 10 monclient: dump: 2026-02-24T22:22:11.094 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:11.094 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:11.094 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.094 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.094 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:11.094 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.094 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:11.094 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:11.094 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:11.094 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.094 INFO:tasks.workunit.client.0.trial046.stderr: -300> 2026-02-24T22:22:10.802+0000 7f5ae17fa640 10 monclient: _finish_auth 0 2026-02-24T22:22:11.094 INFO:tasks.workunit.client.0.trial046.stderr: -299> 2026-02-24T22:22:10.802+0000 7f5ae17fa640 10 monclient: _check_auth_tickets 2026-02-24T22:22:11.094 INFO:tasks.workunit.client.0.trial046.stderr: -298> 2026-02-24T22:22:10.802+0000 7f5ae17fa640 10 monclient: handle_config config(0 keys) 2026-02-24T22:22:11.094 INFO:tasks.workunit.client.0.trial046.stderr: -297> 2026-02-24T22:22:10.802+0000 7f5ae17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:11.094 INFO:tasks.workunit.client.0.trial046.stderr: -296> 2026-02-24T22:22:10.802+0000 7f5ae17fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:22:11.094 INFO:tasks.workunit.client.0.trial046.stderr: -295> 2026-02-24T22:22:10.802+0000 7f5ae17fa640 10 monclient: dump: 2026-02-24T22:22:11.094 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:11.094 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:11.095 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.095 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.095 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:11.095 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.095 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:11.095 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:11.095 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:11.095 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.095 INFO:tasks.workunit.client.0.trial046.stderr: -294> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 monclient: authenticate success, global_id 8565 2026-02-24T22:22:11.095 INFO:tasks.workunit.client.0.trial046.stderr: -293> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient: _renew_subs 2026-02-24T22:22:11.095 INFO:tasks.workunit.client.0.trial046.stderr: -292> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:22:11.095 INFO:tasks.workunit.client.0.trial046.stderr: -291> 2026-02-24T22:22:10.802+0000 7f5b00a5a640 4 set_mon_vals no callback set 2026-02-24T22:22:11.095 INFO:tasks.workunit.client.0.trial046.stderr: -290> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient: _renew_subs 2026-02-24T22:22:11.095 INFO:tasks.workunit.client.0.trial046.stderr: -289> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:22:11.095 INFO:tasks.workunit.client.0.trial046.stderr: -288> 2026-02-24T22:22:10.806+0000 7f5ae17fa640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:22:11.095 INFO:tasks.workunit.client.0.trial046.stderr: -287> 2026-02-24T22:22:10.806+0000 7f5ae17fa640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:22:11.095 INFO:tasks.workunit.client.0.trial046.stderr: -286> 2026-02-24T22:22:10.806+0000 7f5ae17fa640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:22:11.095 INFO:tasks.workunit.client.0.trial046.stderr: -285> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient: get_auth_request con 0x7f5ad00706c0 auth_method 0 2026-02-24T22:22:11.095 INFO:tasks.workunit.client.0.trial046.stderr: -284> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: get_monmap_and_config 2026-02-24T22:22:11.095 INFO:tasks.workunit.client.0.trial046.stderr: -283> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: build_initial_monmap 2026-02-24T22:22:11.095 INFO:tasks.workunit.client.0.trial046.stderr: -282> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 1 build_initial for_mkfs: 0 2026-02-24T22:22:11.096 INFO:tasks.workunit.client.0.trial046.stderr: -281> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: monmap: 2026-02-24T22:22:11.096 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:22:11.096 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:22:11.096 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:22:11.096 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:22:11.096 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:22:11.096 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.096 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:22:11.096 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:22:11.096 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:22:11.096 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.096 INFO:tasks.workunit.client.0.trial046.stderr: -280> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding auth protocol: cephx 2026-02-24T22:22:11.096 INFO:tasks.workunit.client.0.trial046.stderr: -279> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding auth protocol: cephx 2026-02-24T22:22:11.096 INFO:tasks.workunit.client.0.trial046.stderr: -278> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding auth protocol: cephx 2026-02-24T22:22:11.096 INFO:tasks.workunit.client.0.trial046.stderr: -277> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding con mode: secure 2026-02-24T22:22:11.096 INFO:tasks.workunit.client.0.trial046.stderr: -276> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding con mode: crc 2026-02-24T22:22:11.096 INFO:tasks.workunit.client.0.trial046.stderr: -275> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding con mode: secure 2026-02-24T22:22:11.096 INFO:tasks.workunit.client.0.trial046.stderr: -274> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding con mode: crc 2026-02-24T22:22:11.096 INFO:tasks.workunit.client.0.trial046.stderr: -273> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding con mode: secure 2026-02-24T22:22:11.096 INFO:tasks.workunit.client.0.trial046.stderr: -272> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding con mode: crc 2026-02-24T22:22:11.097 INFO:tasks.workunit.client.0.trial046.stderr: -271> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding con mode: crc 2026-02-24T22:22:11.097 INFO:tasks.workunit.client.0.trial046.stderr: -270> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding con mode: secure 2026-02-24T22:22:11.097 INFO:tasks.workunit.client.0.trial046.stderr: -269> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding con mode: crc 2026-02-24T22:22:11.097 INFO:tasks.workunit.client.0.trial046.stderr: -268> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding con mode: secure 2026-02-24T22:22:11.097 INFO:tasks.workunit.client.0.trial046.stderr: -267> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding con mode: crc 2026-02-24T22:22:11.097 INFO:tasks.workunit.client.0.trial046.stderr: -266> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding con mode: secure 2026-02-24T22:22:11.097 INFO:tasks.workunit.client.0.trial046.stderr: -265> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:11.097 INFO:tasks.workunit.client.0.trial046.stderr: -264> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: init 2026-02-24T22:22:11.097 INFO:tasks.workunit.client.0.trial046.stderr: -263> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding auth protocol: cephx 2026-02-24T22:22:11.097 INFO:tasks.workunit.client.0.trial046.stderr: -262> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding auth protocol: cephx 2026-02-24T22:22:11.097 INFO:tasks.workunit.client.0.trial046.stderr: -261> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding auth protocol: cephx 2026-02-24T22:22:11.097 INFO:tasks.workunit.client.0.trial046.stderr: -260> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding con mode: secure 2026-02-24T22:22:11.097 INFO:tasks.workunit.client.0.trial046.stderr: -259> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding con mode: crc 2026-02-24T22:22:11.097 INFO:tasks.workunit.client.0.trial046.stderr: -258> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding con mode: secure 2026-02-24T22:22:11.097 INFO:tasks.workunit.client.0.trial046.stderr: -257> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding con mode: crc 2026-02-24T22:22:11.097 INFO:tasks.workunit.client.0.trial046.stderr: -256> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding con mode: secure 2026-02-24T22:22:11.097 INFO:tasks.workunit.client.0.trial046.stderr: -255> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding con mode: crc 2026-02-24T22:22:11.097 INFO:tasks.workunit.client.0.trial046.stderr: -254> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding con mode: crc 2026-02-24T22:22:11.097 INFO:tasks.workunit.client.0.trial046.stderr: -253> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding con mode: secure 2026-02-24T22:22:11.097 INFO:tasks.workunit.client.0.trial046.stderr: -252> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding con mode: crc 2026-02-24T22:22:11.097 INFO:tasks.workunit.client.0.trial046.stderr: -251> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding con mode: secure 2026-02-24T22:22:11.097 INFO:tasks.workunit.client.0.trial046.stderr: -250> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding con mode: crc 2026-02-24T22:22:11.098 INFO:tasks.workunit.client.0.trial046.stderr: -249> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding con mode: secure 2026-02-24T22:22:11.098 INFO:tasks.workunit.client.0.trial046.stderr: -248> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:11.098 INFO:tasks.workunit.client.0.trial046.stderr: -247> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:11.098 INFO:tasks.workunit.client.0.trial046.stderr: -246> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command rotate-key cmddesc rotate-key hook 0x7fff59fa58c8 EEXIST 2026-02-24T22:22:11.098 INFO:tasks.workunit.client.0.trial046.stderr: -245> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: _reopen_session rank -1 2026-02-24T22:22:11.098 INFO:tasks.workunit.client.0.trial046.stderr: -244> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T22:22:11.098 INFO:tasks.workunit.client.0.trial046.stderr: -243> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): picked mon.noname-a con 0x55b13826c180 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:22:11.098 INFO:tasks.workunit.client.0.trial046.stderr: -242> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): picked mon.noname-b con 0x55b138246d70 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:22:11.098 INFO:tasks.workunit.client.0.trial046.stderr: -241> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): picked mon.noname-c con 0x55b138273660 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:22:11.098 INFO:tasks.workunit.client.0.trial046.stderr: -240> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:11.098 INFO:tasks.workunit.client.0.trial046.stderr: -239> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:11.098 INFO:tasks.workunit.client.0.trial046.stderr: -238> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:11.098 INFO:tasks.workunit.client.0.trial046.stderr: -237> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): _renew_subs 2026-02-24T22:22:11.098 INFO:tasks.workunit.client.0.trial046.stderr: -236> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): authenticate will time out at 2100.913713s 2026-02-24T22:22:11.098 INFO:tasks.workunit.client.0.trial046.stderr: -235> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient(hunting): get_auth_request con 0x55b138273660 auth_method 0 2026-02-24T22:22:11.098 INFO:tasks.workunit.client.0.trial046.stderr: -234> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:11.098 INFO:tasks.workunit.client.0.trial046.stderr: -233> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:11.098 INFO:tasks.workunit.client.0.trial046.stderr: -232> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:11.098 INFO:tasks.workunit.client.0.trial046.stderr: -231> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): get_auth_request con 0x55b13826c180 auth_method 0 2026-02-24T22:22:11.098 INFO:tasks.workunit.client.0.trial046.stderr: -230> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:11.099 INFO:tasks.workunit.client.0.trial046.stderr: -229> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:11.099 INFO:tasks.workunit.client.0.trial046.stderr: -228> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:11.099 INFO:tasks.workunit.client.0.trial046.stderr: -227> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:11.099 INFO:tasks.workunit.client.0.trial046.stderr: -226> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:11.099 INFO:tasks.workunit.client.0.trial046.stderr: -225> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:11.099 INFO:tasks.workunit.client.0.trial046.stderr: -224> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:11.099 INFO:tasks.workunit.client.0.trial046.stderr: -223> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:11.099 INFO:tasks.workunit.client.0.trial046.stderr: -222> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:11.099 INFO:tasks.workunit.client.0.trial046.stderr: -221> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient(hunting): handle_auth_done global_id 7991 payload 274 2026-02-24T22:22:11.099 INFO:tasks.workunit.client.0.trial046.stderr: -220> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient: _finish_hunting 0 2026-02-24T22:22:11.099 INFO:tasks.workunit.client.0.trial046.stderr: -219> 2026-02-24T22:22:10.806+0000 7f5afbfff640 1 monclient: found mon.noname-c 2026-02-24T22:22:11.099 INFO:tasks.workunit.client.0.trial046.stderr: -218> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:22:11.099 INFO:tasks.workunit.client.0.trial046.stderr: -217> 2026-02-24T22:22:10.806+0000 7f5ad6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:11.099 INFO:tasks.workunit.client.0.trial046.stderr: -216> 2026-02-24T22:22:10.806+0000 7f5ad6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:22:11.099 INFO:tasks.workunit.client.0.trial046.stderr: -215> 2026-02-24T22:22:10.806+0000 7f5ad6ffd640 10 monclient: dump: 2026-02-24T22:22:11.099 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:11.099 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:11.099 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.099 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.099 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:11.099 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.099 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:11.100 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:11.100 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:11.100 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.100 INFO:tasks.workunit.client.0.trial046.stderr: -214> 2026-02-24T22:22:10.806+0000 7f5ad6ffd640 10 monclient: _finish_auth 0 2026-02-24T22:22:11.100 INFO:tasks.workunit.client.0.trial046.stderr: -213> 2026-02-24T22:22:10.806+0000 7f5ad6ffd640 10 monclient: _check_auth_tickets 2026-02-24T22:22:11.100 INFO:tasks.workunit.client.0.trial046.stderr: -212> 2026-02-24T22:22:10.806+0000 7f5ad6ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T22:22:11.100 INFO:tasks.workunit.client.0.trial046.stderr: -211> 2026-02-24T22:22:10.806+0000 7f5ad6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:11.100 INFO:tasks.workunit.client.0.trial046.stderr: -210> 2026-02-24T22:22:10.806+0000 7f5ad6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:22:11.100 INFO:tasks.workunit.client.0.trial046.stderr: -209> 2026-02-24T22:22:10.806+0000 7f5ad6ffd640 10 monclient: dump: 2026-02-24T22:22:11.100 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:11.100 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:11.100 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.100 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.100 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:11.100 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.100 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:11.100 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:11.100 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:11.100 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.100 INFO:tasks.workunit.client.0.trial046.stderr: -208> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 monclient: authenticate success, global_id 7991 2026-02-24T22:22:11.100 INFO:tasks.workunit.client.0.trial046.stderr: -207> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: get_monmap_and_config success 2026-02-24T22:22:11.100 INFO:tasks.workunit.client.0.trial046.stderr: -206> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 4 set_mon_vals no callback set 2026-02-24T22:22:11.101 INFO:tasks.workunit.client.0.trial046.stderr: -205> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: shutdown 2026-02-24T22:22:11.101 INFO:tasks.workunit.client.0.trial046.stderr: -204> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: build_initial_monmap 2026-02-24T22:22:11.101 INFO:tasks.workunit.client.0.trial046.stderr: -203> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 1 build_initial for_mkfs: 0 2026-02-24T22:22:11.101 INFO:tasks.workunit.client.0.trial046.stderr: -202> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: monmap: 2026-02-24T22:22:11.101 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:22:11.101 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:22:11.101 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:22:11.101 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:22:11.101 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:22:11.101 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.101 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:22:11.101 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:22:11.101 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:22:11.101 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.101 INFO:tasks.workunit.client.0.trial046.stderr: -201> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding auth protocol: cephx 2026-02-24T22:22:11.101 INFO:tasks.workunit.client.0.trial046.stderr: -200> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding auth protocol: cephx 2026-02-24T22:22:11.101 INFO:tasks.workunit.client.0.trial046.stderr: -199> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding auth protocol: cephx 2026-02-24T22:22:11.101 INFO:tasks.workunit.client.0.trial046.stderr: -198> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding con mode: secure 2026-02-24T22:22:11.101 INFO:tasks.workunit.client.0.trial046.stderr: -197> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding con mode: crc 2026-02-24T22:22:11.101 INFO:tasks.workunit.client.0.trial046.stderr: -196> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding con mode: secure 2026-02-24T22:22:11.101 INFO:tasks.workunit.client.0.trial046.stderr: -195> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding con mode: crc 2026-02-24T22:22:11.101 INFO:tasks.workunit.client.0.trial046.stderr: -194> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding con mode: secure 2026-02-24T22:22:11.101 INFO:tasks.workunit.client.0.trial046.stderr: -193> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding con mode: crc 2026-02-24T22:22:11.102 INFO:tasks.workunit.client.0.trial046.stderr: -192> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding con mode: crc 2026-02-24T22:22:11.102 INFO:tasks.workunit.client.0.trial046.stderr: -191> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding con mode: secure 2026-02-24T22:22:11.102 INFO:tasks.workunit.client.0.trial046.stderr: -190> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding con mode: crc 2026-02-24T22:22:11.102 INFO:tasks.workunit.client.0.trial046.stderr: -189> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding con mode: secure 2026-02-24T22:22:11.102 INFO:tasks.workunit.client.0.trial046.stderr: -188> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding con mode: crc 2026-02-24T22:22:11.102 INFO:tasks.workunit.client.0.trial046.stderr: -187> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding con mode: secure 2026-02-24T22:22:11.102 INFO:tasks.workunit.client.0.trial046.stderr: -186> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:11.102 INFO:tasks.workunit.client.0.trial046.stderr: -185> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 1 librados: starting msgr at 2026-02-24T22:22:11.102 INFO:tasks.workunit.client.0.trial046.stderr: -184> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 1 librados: starting objecter 2026-02-24T22:22:11.102 INFO:tasks.workunit.client.0.trial046.stderr: -183> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command objecter_requests cmddesc objecter_requests hook 0x55b138273540 EEXIST 2026-02-24T22:22:11.102 INFO:tasks.workunit.client.0.trial046.stderr: -182> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 1 librados: setting wanted keys 2026-02-24T22:22:11.102 INFO:tasks.workunit.client.0.trial046.stderr: -181> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 1 librados: calling monclient init 2026-02-24T22:22:11.102 INFO:tasks.workunit.client.0.trial046.stderr: -180> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: init 2026-02-24T22:22:11.102 INFO:tasks.workunit.client.0.trial046.stderr: -179> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding auth protocol: cephx 2026-02-24T22:22:11.102 INFO:tasks.workunit.client.0.trial046.stderr: -178> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding auth protocol: cephx 2026-02-24T22:22:11.102 INFO:tasks.workunit.client.0.trial046.stderr: -177> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding auth protocol: cephx 2026-02-24T22:22:11.102 INFO:tasks.workunit.client.0.trial046.stderr: -176> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding con mode: secure 2026-02-24T22:22:11.102 INFO:tasks.workunit.client.0.trial046.stderr: -175> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding con mode: crc 2026-02-24T22:22:11.102 INFO:tasks.workunit.client.0.trial046.stderr: -174> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding con mode: secure 2026-02-24T22:22:11.102 INFO:tasks.workunit.client.0.trial046.stderr: -173> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding con mode: crc 2026-02-24T22:22:11.102 INFO:tasks.workunit.client.0.trial046.stderr: -172> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding con mode: secure 2026-02-24T22:22:11.102 INFO:tasks.workunit.client.0.trial046.stderr: -171> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding con mode: crc 2026-02-24T22:22:11.102 INFO:tasks.workunit.client.0.trial046.stderr: -170> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding con mode: crc 2026-02-24T22:22:11.102 INFO:tasks.workunit.client.0.trial046.stderr: -169> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding con mode: secure 2026-02-24T22:22:11.102 INFO:tasks.workunit.client.0.trial046.stderr: -168> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding con mode: crc 2026-02-24T22:22:11.102 INFO:tasks.workunit.client.0.trial046.stderr: -167> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding con mode: secure 2026-02-24T22:22:11.103 INFO:tasks.workunit.client.0.trial046.stderr: -166> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding con mode: crc 2026-02-24T22:22:11.103 INFO:tasks.workunit.client.0.trial046.stderr: -165> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding con mode: secure 2026-02-24T22:22:11.103 INFO:tasks.workunit.client.0.trial046.stderr: -164> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:11.103 INFO:tasks.workunit.client.0.trial046.stderr: -163> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:11.103 INFO:tasks.workunit.client.0.trial046.stderr: -162> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command rotate-key cmddesc rotate-key hook 0x55b138243e28 EEXIST 2026-02-24T22:22:11.103 INFO:tasks.workunit.client.0.trial046.stderr: -161> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: _reopen_session rank -1 2026-02-24T22:22:11.103 INFO:tasks.workunit.client.0.trial046.stderr: -160> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:22:11.103 INFO:tasks.workunit.client.0.trial046.stderr: -159> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): picked mon.noname-c con 0x55b13826c180 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:22:11.103 INFO:tasks.workunit.client.0.trial046.stderr: -158> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): picked mon.noname-a con 0x55b138273660 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:22:11.103 INFO:tasks.workunit.client.0.trial046.stderr: -157> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): picked mon.noname-b con 0x55b138271fa0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:22:11.103 INFO:tasks.workunit.client.0.trial046.stderr: -156> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:11.103 INFO:tasks.workunit.client.0.trial046.stderr: -155> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:11.103 INFO:tasks.workunit.client.0.trial046.stderr: -154> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:11.103 INFO:tasks.workunit.client.0.trial046.stderr: -153> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): _renew_subs 2026-02-24T22:22:11.103 INFO:tasks.workunit.client.0.trial046.stderr: -152> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): authenticate will time out at 2100.914961s 2026-02-24T22:22:11.103 INFO:tasks.workunit.client.0.trial046.stderr: -151> 2026-02-24T22:22:10.806+0000 7f5af3fff640 10 monclient(hunting): get_auth_request con 0x55b138273660 auth_method 0 2026-02-24T22:22:11.103 INFO:tasks.workunit.client.0.trial046.stderr: -150> 2026-02-24T22:22:10.806+0000 7f5af3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:11.103 INFO:tasks.workunit.client.0.trial046.stderr: -149> 2026-02-24T22:22:10.806+0000 7f5af3fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:11.103 INFO:tasks.workunit.client.0.trial046.stderr: -148> 2026-02-24T22:22:10.806+0000 7f5af3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:11.103 INFO:tasks.workunit.client.0.trial046.stderr: -147> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): get_auth_request con 0x55b13826c180 auth_method 0 2026-02-24T22:22:11.103 INFO:tasks.workunit.client.0.trial046.stderr: -146> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:11.103 INFO:tasks.workunit.client.0.trial046.stderr: -145> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:11.103 INFO:tasks.workunit.client.0.trial046.stderr: -144> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:11.104 INFO:tasks.workunit.client.0.trial046.stderr: -143> 2026-02-24T22:22:10.806+0000 7f5af3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:11.104 INFO:tasks.workunit.client.0.trial046.stderr: -142> 2026-02-24T22:22:10.806+0000 7f5af3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:11.104 INFO:tasks.workunit.client.0.trial046.stderr: -141> 2026-02-24T22:22:10.806+0000 7f5af3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:11.104 INFO:tasks.workunit.client.0.trial046.stderr: -140> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:11.104 INFO:tasks.workunit.client.0.trial046.stderr: -139> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:11.104 INFO:tasks.workunit.client.0.trial046.stderr: -138> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:11.104 INFO:tasks.workunit.client.0.trial046.stderr: -137> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient(hunting): get_auth_request con 0x55b138271fa0 auth_method 0 2026-02-24T22:22:11.104 INFO:tasks.workunit.client.0.trial046.stderr: -136> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:11.104 INFO:tasks.workunit.client.0.trial046.stderr: -135> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:11.104 INFO:tasks.workunit.client.0.trial046.stderr: -134> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:11.104 INFO:tasks.workunit.client.0.trial046.stderr: -133> 2026-02-24T22:22:10.806+0000 7f5af3fff640 10 monclient(hunting): handle_auth_done global_id 8571 payload 824 2026-02-24T22:22:11.104 INFO:tasks.workunit.client.0.trial046.stderr: -132> 2026-02-24T22:22:10.806+0000 7f5af3fff640 10 monclient: _finish_hunting 0 2026-02-24T22:22:11.104 INFO:tasks.workunit.client.0.trial046.stderr: -131> 2026-02-24T22:22:10.806+0000 7f5af3fff640 1 monclient: found mon.noname-a 2026-02-24T22:22:11.104 INFO:tasks.workunit.client.0.trial046.stderr: -130> 2026-02-24T22:22:10.806+0000 7f5af3fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:22:11.104 INFO:tasks.workunit.client.0.trial046.stderr: -129> 2026-02-24T22:22:10.806+0000 7f5ad57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:11.104 INFO:tasks.workunit.client.0.trial046.stderr: -128> 2026-02-24T22:22:10.806+0000 7f5ad57fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:22:11.104 INFO:tasks.workunit.client.0.trial046.stderr: -127> 2026-02-24T22:22:10.806+0000 7f5ad57fa640 10 monclient: dump: 2026-02-24T22:22:11.104 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:11.104 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:11.104 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.104 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.104 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:11.104 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.105 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:11.105 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:11.105 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:11.105 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.105 INFO:tasks.workunit.client.0.trial046.stderr: -126> 2026-02-24T22:22:10.806+0000 7f5ad57fa640 10 monclient: _finish_auth 0 2026-02-24T22:22:11.105 INFO:tasks.workunit.client.0.trial046.stderr: -125> 2026-02-24T22:22:10.806+0000 7f5ad57fa640 10 monclient: _check_auth_tickets 2026-02-24T22:22:11.105 INFO:tasks.workunit.client.0.trial046.stderr: -124> 2026-02-24T22:22:10.806+0000 7f5ad57fa640 10 monclient: handle_config config(0 keys) 2026-02-24T22:22:11.105 INFO:tasks.workunit.client.0.trial046.stderr: -123> 2026-02-24T22:22:10.806+0000 7f5ad57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:11.105 INFO:tasks.workunit.client.0.trial046.stderr: -122> 2026-02-24T22:22:10.806+0000 7f5ad57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:22:11.105 INFO:tasks.workunit.client.0.trial046.stderr: -121> 2026-02-24T22:22:10.806+0000 7f5ad57fa640 10 monclient: dump: 2026-02-24T22:22:11.105 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:11.105 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:11.105 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.105 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.105 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:11.105 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.105 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:11.105 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:11.105 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:11.105 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.105 INFO:tasks.workunit.client.0.trial046.stderr: -120> 2026-02-24T22:22:10.806+0000 7f5ad6ffd640 4 set_mon_vals no callback set 2026-02-24T22:22:11.105 INFO:tasks.workunit.client.0.trial046.stderr: -119> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 monclient: authenticate success, global_id 8571 2026-02-24T22:22:11.105 INFO:tasks.workunit.client.0.trial046.stderr: -118> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: _renew_subs 2026-02-24T22:22:11.106 INFO:tasks.workunit.client.0.trial046.stderr: -117> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:22:11.106 INFO:tasks.workunit.client.0.trial046.stderr: -116> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: _renew_subs 2026-02-24T22:22:11.106 INFO:tasks.workunit.client.0.trial046.stderr: -115> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:22:11.106 INFO:tasks.workunit.client.0.trial046.stderr: -114> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 1 librados: init done 2026-02-24T22:22:11.106 INFO:tasks.workunit.client.0.trial046.stderr: -113> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command cr dump hook 0x55b138271888 2026-02-24T22:22:11.106 INFO:tasks.workunit.client.0.trial046.stderr: -112> 2026-02-24T22:22:10.810+0000 7f5b01f84a80 4 Realm: () 2026-02-24T22:22:11.106 INFO:tasks.workunit.client.0.trial046.stderr: -111> 2026-02-24T22:22:10.810+0000 7f5b01f84a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:22:11.106 INFO:tasks.workunit.client.0.trial046.stderr: -110> 2026-02-24T22:22:10.810+0000 7f5b01f84a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:22:11.106 INFO:tasks.workunit.client.0.trial046.stderr: -109> 2026-02-24T22:22:10.810+0000 7f5b01f84a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:22:11.106 INFO:tasks.workunit.client.0.trial046.stderr: -108> 2026-02-24T22:22:10.810+0000 7f5b01f84a80 20 zonegroup default 2026-02-24T22:22:11.106 INFO:tasks.workunit.client.0.trial046.stderr: -107> 2026-02-24T22:22:10.810+0000 7f5afb7fe640 10 monclient: get_auth_request con 0x55b13830d8e0 auth_method 0 2026-02-24T22:22:11.106 INFO:tasks.workunit.client.0.trial046.stderr: -106> 2026-02-24T22:22:10.810+0000 7f5ad57fa640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:22:11.106 INFO:tasks.workunit.client.0.trial046.stderr: -105> 2026-02-24T22:22:10.810+0000 7f5ad57fa640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:22:11.106 INFO:tasks.workunit.client.0.trial046.stderr: -104> 2026-02-24T22:22:10.810+0000 7f5ad57fa640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:22:11.106 INFO:tasks.workunit.client.0.trial046.stderr: -103> 2026-02-24T22:22:10.810+0000 7f5b01f84a80 20 rados->read ofs=0 len=0 2026-02-24T22:22:11.106 INFO:tasks.workunit.client.0.trial046.stderr: -102> 2026-02-24T22:22:10.810+0000 7f5afbfff640 10 monclient: get_auth_request con 0x7f5ac00707b0 auth_method 0 2026-02-24T22:22:11.106 INFO:tasks.workunit.client.0.trial046.stderr: -101> 2026-02-24T22:22:10.810+0000 7f5afb7fe640 10 monclient: get_auth_request con 0x55b138330c00 auth_method 0 2026-02-24T22:22:11.106 INFO:tasks.workunit.client.0.trial046.stderr: -100> 2026-02-24T22:22:10.810+0000 7f5b01f84a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:22:11.106 INFO:tasks.workunit.client.0.trial046.stderr: -99> 2026-02-24T22:22:10.810+0000 7f5b01f84a80 20 rados->read ofs=0 len=0 2026-02-24T22:22:11.106 INFO:tasks.workunit.client.0.trial046.stderr: -98> 2026-02-24T22:22:10.810+0000 7f5af3fff640 10 monclient: get_auth_request con 0x55b138352230 auth_method 0 2026-02-24T22:22:11.106 INFO:tasks.workunit.client.0.trial046.stderr: -97> 2026-02-24T22:22:10.810+0000 7f5b01f84a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:22:11.106 INFO:tasks.workunit.client.0.trial046.stderr: -96> 2026-02-24T22:22:10.810+0000 7f5b01f84a80 20 started sync module instance, tier type = 2026-02-24T22:22:11.106 INFO:tasks.workunit.client.0.trial046.stderr: -95> 2026-02-24T22:22:10.810+0000 7f5b01f84a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:22:11.106 INFO:tasks.workunit.client.0.trial046.stderr: -94> 2026-02-24T22:22:10.810+0000 7f5afbfff640 10 monclient: get_auth_request con 0x55b1383b36b0 auth_method 0 2026-02-24T22:22:11.106 INFO:tasks.workunit.client.0.trial046.stderr: -93> 2026-02-24T22:22:10.810+0000 7f5afbfff640 10 monclient: get_auth_request con 0x55b1384bb0a0 auth_method 0 2026-02-24T22:22:11.107 INFO:tasks.workunit.client.0.trial046.stderr: -92> 2026-02-24T22:22:10.810+0000 7f5afb7fe640 10 monclient: get_auth_request con 0x55b138415fe0 auth_method 0 2026-02-24T22:22:11.107 INFO:tasks.workunit.client.0.trial046.stderr: -91> 2026-02-24T22:22:10.810+0000 7f5af3fff640 10 monclient: get_auth_request con 0x55b138478f60 auth_method 0 2026-02-24T22:22:11.107 INFO:tasks.workunit.client.0.trial046.stderr: -90> 2026-02-24T22:22:10.810+0000 7f5afb7fe640 10 monclient: get_auth_request con 0x55b13851e0a0 auth_method 0 2026-02-24T22:22:11.107 INFO:tasks.workunit.client.0.trial046.stderr: -89> 2026-02-24T22:22:10.814+0000 7f5afb7fe640 0 -- 10.20.193.46:0/1549073879 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x55b13851e0a0 msgr2=0x55b13853e520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:22:11.107 INFO:tasks.workunit.client.0.trial046.stderr: -88> 2026-02-24T22:22:10.814+0000 7f5ad57fa640 1 client.8571.objecter ms_handle_reset 0x55b13851e0a0 session 0x55b13851d990 osd.6 2026-02-24T22:22:11.107 INFO:tasks.workunit.client.0.trial046.stderr: -87> 2026-02-24T22:22:10.814+0000 7f5ad57fa640 10 monclient: _renew_subs 2026-02-24T22:22:11.107 INFO:tasks.workunit.client.0.trial046.stderr: -86> 2026-02-24T22:22:10.814+0000 7f5ad57fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:22:11.107 INFO:tasks.workunit.client.0.trial046.stderr: -85> 2026-02-24T22:22:10.814+0000 7f5afb7fe640 10 monclient: get_auth_request con 0x7f5ac009d540 auth_method 0 2026-02-24T22:22:11.107 INFO:tasks.workunit.client.0.trial046.stderr: -84> 2026-02-24T22:22:10.814+0000 7f5b01f84a80 20 add_watcher() i=1 2026-02-24T22:22:11.107 INFO:tasks.workunit.client.0.trial046.stderr: -83> 2026-02-24T22:22:10.814+0000 7f5b01f84a80 20 add_watcher() i=5 2026-02-24T22:22:11.107 INFO:tasks.workunit.client.0.trial046.stderr: -82> 2026-02-24T22:22:10.814+0000 7f5b01f84a80 20 add_watcher() i=0 2026-02-24T22:22:11.107 INFO:tasks.workunit.client.0.trial046.stderr: -81> 2026-02-24T22:22:10.814+0000 7f5b01f84a80 20 add_watcher() i=6 2026-02-24T22:22:11.107 INFO:tasks.workunit.client.0.trial046.stderr: -80> 2026-02-24T22:22:10.814+0000 7f5b01f84a80 20 add_watcher() i=2 2026-02-24T22:22:11.107 INFO:tasks.workunit.client.0.trial046.stderr: -79> 2026-02-24T22:22:10.814+0000 7f5b01f84a80 20 add_watcher() i=4 2026-02-24T22:22:11.107 INFO:tasks.workunit.client.0.trial046.stderr: -78> 2026-02-24T22:22:10.814+0000 7f5b01f84a80 20 add_watcher() i=3 2026-02-24T22:22:11.107 INFO:tasks.workunit.client.0.trial046.stderr: -77> 2026-02-24T22:22:10.814+0000 7f5b01f84a80 20 add_watcher() i=7 2026-02-24T22:22:11.107 INFO:tasks.workunit.client.0.trial046.stderr: -76> 2026-02-24T22:22:10.814+0000 7f5b01f84a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:22:11.107 INFO:tasks.workunit.client.0.trial046.stderr: -75> 2026-02-24T22:22:10.814+0000 7f5af3fff640 10 monclient: get_auth_request con 0x7f5a20008af0 auth_method 0 2026-02-24T22:22:11.107 INFO:tasks.workunit.client.0.trial046.stderr: -74> 2026-02-24T22:22:10.814+0000 7f5ae17fa640 1 client.8565.objecter ms_handle_reset 0x7f5a20008af0 session 0x7f5a20008470 osd.1 2026-02-24T22:22:11.107 INFO:tasks.workunit.client.0.trial046.stderr: -73> 2026-02-24T22:22:10.814+0000 7f5ae17fa640 10 monclient: _renew_subs 2026-02-24T22:22:11.107 INFO:tasks.workunit.client.0.trial046.stderr: -72> 2026-02-24T22:22:10.814+0000 7f5ae17fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:22:11.107 INFO:tasks.workunit.client.0.trial046.stderr: -71> 2026-02-24T22:22:10.814+0000 7f5af3fff640 10 monclient: get_auth_request con 0x7f5ad009d3f0 auth_method 0 2026-02-24T22:22:11.107 INFO:tasks.workunit.client.0.trial046.stderr: -70> 2026-02-24T22:22:10.814+0000 7f5af3fff640 0 -- 10.20.193.46:0/3650014496 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f5ad009d3f0 msgr2=0x7f5ad00bd7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:22:11.107 INFO:tasks.workunit.client.0.trial046.stderr: -69> 2026-02-24T22:22:11.018+0000 7f5af3fff640 10 monclient: get_auth_request con 0x7f5ad009d3f0 auth_method 0 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -68> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding auth protocol: cephx 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -67> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding auth protocol: cephx 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -66> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding auth protocol: cephx 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -65> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding con mode: secure 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -64> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding con mode: crc 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -63> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding con mode: secure 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -62> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding con mode: crc 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -61> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding con mode: secure 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -60> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding con mode: crc 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -59> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding con mode: crc 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -58> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding con mode: secure 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -57> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding con mode: crc 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -56> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding con mode: secure 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -55> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding con mode: crc 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -54> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding con mode: secure 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -53> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -52> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -51> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -50> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command cache list hook 0x55b1382474e0 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -49> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command cache inspect hook 0x55b1382474e0 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -48> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command cache erase hook 0x55b1382474e0 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -47> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command cache zap hook 0x55b1382474e0 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -46> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 note: GC not initialized 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -45> 2026-02-24T22:22:11.018+0000 7f5aacfe1640 20 reqs_thread_entry: start 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -44> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command sync trace show hook 0x7f5aec0043a0 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -43> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command sync trace history hook 0x7f5aec0043a0 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -42> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command sync trace active hook 0x7f5aec0043a0 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -41> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command sync trace active_short hook 0x7f5aec0043a0 2026-02-24T22:22:11.108 INFO:tasks.workunit.client.0.trial046.stderr: -40> 2026-02-24T22:22:11.018+0000 7f5afbfff640 10 monclient: get_auth_request con 0x55b138403c20 auth_method 0 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -39> 2026-02-24T22:22:11.022+0000 7f5afb7fe640 10 monclient: get_auth_request con 0x55b1383f5fc0 auth_method 0 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -38> 2026-02-24T22:22:11.022+0000 7f5af3fff640 10 monclient: get_auth_request con 0x55b1383f7670 auth_method 0 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -37> 2026-02-24T22:22:11.022+0000 7f5afbfff640 10 monclient: get_auth_request con 0x55b1383f8ea0 auth_method 0 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -36> 2026-02-24T22:22:11.026+0000 7f5afb7fe640 10 monclient: get_auth_request con 0x55b138408010 auth_method 0 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -35> 2026-02-24T22:22:11.030+0000 7f5af3fff640 10 monclient: get_auth_request con 0x55b1383fd010 auth_method 0 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -34> 2026-02-24T22:22:11.034+0000 7f5ae17fa640 1 client.8565.objecter ms_handle_reset 0x55b1383f8ea0 session 0x55b1383f8820 osd.0 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -33> 2026-02-24T22:22:11.034+0000 7f5afbfff640 10 monclient: get_auth_request con 0x7f5a20008af0 auth_method 0 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -32> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 20 init_complete bucket index max shards: 1 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -31> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 20 Filter name: none 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -30> 2026-02-24T22:22:11.054+0000 7f5a16ffd640 20 reqs_thread_entry: start 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -29> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -28> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 20 rados->read ofs=0 len=0 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -27> 2026-02-24T22:22:11.054+0000 7f5afbfff640 10 monclient: get_auth_request con 0x55b137fc80e0 auth_method 0 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -26> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -25> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -24> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -23> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : miss 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -22> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 20 rados->read ofs=0 len=0 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -21> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 20 rados_obj.operate() r=0 bl.length=470 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -20> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -19> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -18> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -17> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -16> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -15> 2026-02-24T22:22:11.058+0000 7f5b01f84a80 10 init_target_layout removing existing target index objects from a previous reshard attempt 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -14> 2026-02-24T22:22:11.062+0000 7f5b01f84a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -13> 2026-02-24T22:22:11.062+0000 7f5b01f84a80 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -12> 2026-02-24T22:22:11.062+0000 7f5b01f84a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:22:11.109 INFO:tasks.workunit.client.0.trial046.stderr: -11> 2026-02-24T22:22:11.062+0000 7f5b01f84a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518, ofs0, ns] 2026-02-24T22:22:11.110 INFO:tasks.workunit.client.0.trial046.stderr: -10> 2026-02-24T22:22:11.062+0000 7f5ad6ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959946 cookie 94219642602960 notifier 8571 bl.length()=929 2026-02-24T22:22:11.110 INFO:tasks.workunit.client.0.trial046.stderr: -9> 2026-02-24T22:22:11.062+0000 7f5ad6ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:22:11.110 INFO:tasks.workunit.client.0.trial046.stderr: -8> 2026-02-24T22:22:11.062+0000 7f5ad6ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:22:11.110 INFO:tasks.workunit.client.0.trial046.stderr: -7> 2026-02-24T22:22:11.062+0000 7f5ad6ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:22:11.110 INFO:tasks.workunit.client.0.trial046.stderr: -6> 2026-02-24T22:22:11.062+0000 7f5b01f84a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_sources= new_sources= 2026-02-24T22:22:11.110 INFO:tasks.workunit.client.0.trial046.stderr: -5> 2026-02-24T22:22:11.062+0000 7f5b01f84a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential sources added=[] removed=[] 2026-02-24T22:22:11.110 INFO:tasks.workunit.client.0.trial046.stderr: -4> 2026-02-24T22:22:11.062+0000 7f5b01f84a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_dests= new_dests= 2026-02-24T22:22:11.110 INFO:tasks.workunit.client.0.trial046.stderr: -3> 2026-02-24T22:22:11.062+0000 7f5b01f84a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential dests added=[] removed=[] 2026-02-24T22:22:11.110 INFO:tasks.workunit.client.0.trial046.stderr: -2> 2026-02-24T22:22:11.062+0000 7f5ad6ffd640 20 trim reshard logs: write on '.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1' needs retry: Success 2026-02-24T22:22:11.110 INFO:tasks.workunit.client.0.trial046.stderr: -1> 2026-02-24T22:22:11.066+0000 7f5b01f84a80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f5b01f84a80 time 2026-02-24T22:22:11.067093+0000 2026-02-24T22:22:11.110 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:22:11.110 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.110 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:22:11.110 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f5b053c6eb1] 2026-02-24T22:22:11.110 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x55b12bb99b9f] 2026-02-24T22:22:11.110 INFO:tasks.workunit.client.0.trial046.stderr: 3: radosgw-admin(+0x1714451) [0x55b12cda8451] 2026-02-24T22:22:11.110 INFO:tasks.workunit.client.0.trial046.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x55b12c305202] 2026-02-24T22:22:11.110 INFO:tasks.workunit.client.0.trial046.stderr: 5: main() 2026-02-24T22:22:11.110 INFO:tasks.workunit.client.0.trial046.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f5b046c6d90] 2026-02-24T22:22:11.110 INFO:tasks.workunit.client.0.trial046.stderr: 7: __libc_start_main() 2026-02-24T22:22:11.110 INFO:tasks.workunit.client.0.trial046.stderr: 8: _start() 2026-02-24T22:22:11.110 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.110 INFO:tasks.workunit.client.0.trial046.stderr: 0> 2026-02-24T22:22:11.066+0000 7f5b01f84a80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:22:11.110 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f5b01f84a80 thread_name:radosgw-admin 2026-02-24T22:22:11.110 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.110 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f5b046df520] 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f5b053c6f10] 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x55b12bb99b9f] 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr: 7: radosgw-admin(+0x1714451) [0x55b12cda8451] 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x55b12c305202] 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr: 9: main() 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f5b046c6d90] 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr: 11: __libc_start_main() 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr: 12: _start() 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr:--- logging levels --- 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 none 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 lockdep 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 context 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 crush 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_balancer 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_locker 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log_expire 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_migrator 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr: 3/ 5 mds_quiesce 2026-02-24T22:22:11.111 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 buffer 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 timer 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 filer 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 striper 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 objecter 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rados 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_mirror 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_replay 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_pwl 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 journaler 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objectcacher 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 immutable_obj_cache 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 client 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 osd 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 optracker 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objclass 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 filestore 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 journal 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 ms 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mon 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 0/10 monc 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 paxos 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 tp 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 auth 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 crypto 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 finisher 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 reserver 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 heartbeatmap 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 perfcounter 2026-02-24T22:22:11.112 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_sync 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_datacache 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_access 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dbstore 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_flight 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_lifecycle 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_restore 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw_notification 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_bucket_logging 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 javaclient 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 asok 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 throttle 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 refs 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 compressor 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore_compression 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluefs 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 bdev 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 kstore 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 4/ 5 rocksdb 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 fuse 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 2/ 5 mgr 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mgrc 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 dpdk 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 eventtrace 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 prioritycache 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 test 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephfs_mirror 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephsqlite 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 crimson_interrupt 2026-02-24T22:22:11.113 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_onode 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_odata 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_omap 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_tm 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_t 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cleaner 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_epm 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_lba 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_fixedkv_tree 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cache 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_journal 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_device 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_backref 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 alienstore 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mclock 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dedup 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cyanstore 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 ceph_exporter 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 memstore 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 trace 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 ceph_dedup 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: -2/-2 (syslog threshold) 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 99/99 (stderr threshold) 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr:--- pthread ID / name mapping for recent threads --- 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 7f5a16ffd640 / http_manager 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 7f5aacfe1640 / http_manager 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 7f5ad57fa640 / ms_dispatch 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 7f5ad6ffd640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 7f5ae17fa640 / ms_dispatch 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 7f5ae3fff640 / ms_dispatch 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 7f5af1ffb640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 7f5af27fc640 / admin_socket 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 7f5af3fff640 / msgr-worker-2 2026-02-24T22:22:11.114 INFO:tasks.workunit.client.0.trial046.stderr: 7f5afaffd640 / ms_dispatch 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: 7f5afb7fe640 / msgr-worker-1 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: 7f5afbfff640 / msgr-worker-0 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: 7f5b00a5a640 / io_context_pool 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: 7f5b01f84a80 / radosgw-admin 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: max_recent 500 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: max_new 1000 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: log_file /var/log/ceph/ceph-client.admin.55541.log 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr:--- end dump of recent events --- 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr:--- begin dump of recent events --- 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: -499> 2026-02-24T22:22:10.794+0000 7f5afb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: -498> 2026-02-24T22:22:10.794+0000 7f5afb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: -497> 2026-02-24T22:22:10.794+0000 7f5afb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: -496> 2026-02-24T22:22:10.794+0000 7f5af3fff640 10 monclient(hunting): get_auth_request con 0x55b137f57fe0 auth_method 0 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: -495> 2026-02-24T22:22:10.794+0000 7f5af3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: -494> 2026-02-24T22:22:10.794+0000 7f5af3fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: -493> 2026-02-24T22:22:10.794+0000 7f5af3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: -492> 2026-02-24T22:22:10.794+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: -491> 2026-02-24T22:22:10.794+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: -490> 2026-02-24T22:22:10.794+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: -489> 2026-02-24T22:22:10.798+0000 7f5af3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: -488> 2026-02-24T22:22:10.798+0000 7f5af3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: -487> 2026-02-24T22:22:10.798+0000 7f5af3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: -486> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_done global_id 8559 payload 274 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: -485> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient: _finish_hunting 0 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: -484> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 1 monclient: found mon.noname-a 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: -483> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: -482> 2026-02-24T22:22:10.798+0000 7f5af1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: -481> 2026-02-24T22:22:10.798+0000 7f5af1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:22:11.115 INFO:tasks.workunit.client.0.trial046.stderr: -480> 2026-02-24T22:22:10.798+0000 7f5af1ffb640 10 monclient: dump: 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr: -479> 2026-02-24T22:22:10.798+0000 7f5af1ffb640 10 monclient: _finish_auth 0 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr: -478> 2026-02-24T22:22:10.798+0000 7f5af1ffb640 10 monclient: _check_auth_tickets 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr: -477> 2026-02-24T22:22:10.798+0000 7f5af1ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr: -476> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 monclient: authenticate success, global_id 8559 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr: -475> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient: get_monmap_and_config success 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr: -474> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 4 set_mon_vals no callback set 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr: -473> 2026-02-24T22:22:10.798+0000 7f5af1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr: -472> 2026-02-24T22:22:10.798+0000 7f5af1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr: -471> 2026-02-24T22:22:10.798+0000 7f5af1ffb640 10 monclient: dump: 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr: -470> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient: shutdown 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr: -469> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) unregister_commands rotate-key 2026-02-24T22:22:11.116 INFO:tasks.workunit.client.0.trial046.stderr: -468> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient: build_initial_monmap 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr: -467> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 1 build_initial for_mkfs: 0 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr: -466> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient: monmap: 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr: -465> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding auth protocol: cephx 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr: -464> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding auth protocol: cephx 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr: -463> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding auth protocol: cephx 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr: -462> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding con mode: secure 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr: -461> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding con mode: crc 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr: -460> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding con mode: secure 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr: -459> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding con mode: crc 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr: -458> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding con mode: secure 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr: -457> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding con mode: crc 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr: -456> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding con mode: crc 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr: -455> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding con mode: secure 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr: -454> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding con mode: crc 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr: -453> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding con mode: secure 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr: -452> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding con mode: crc 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr: -451> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fcbbf0) adding con mode: secure 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr: -450> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr: -449> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 1 librados: starting msgr at 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr: -448> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 1 librados: starting objecter 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr: -447> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command objecter_requests hook 0x55b138009280 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr: -446> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 1 librados: setting wanted keys 2026-02-24T22:22:11.117 INFO:tasks.workunit.client.0.trial046.stderr: -445> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 1 librados: calling monclient init 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -444> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient: init 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -443> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding auth protocol: cephx 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -442> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding auth protocol: cephx 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -441> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding auth protocol: cephx 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -440> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding con mode: secure 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -439> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding con mode: crc 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -438> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding con mode: secure 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -437> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding con mode: crc 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -436> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding con mode: secure 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -435> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding con mode: crc 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -434> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding con mode: crc 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -433> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding con mode: secure 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -432> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding con mode: crc 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -431> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding con mode: secure 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -430> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding con mode: crc 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -429> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 AuthRegistry(0x55b137fc9cc0) adding con mode: secure 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -428> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -427> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -426> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command rotate-key hook 0x55b137fc9e08 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -425> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient: _reopen_session rank -1 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -424> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -423> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient(hunting): picked mon.noname-c con 0x55b137f11c30 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -422> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient(hunting): picked mon.noname-a con 0x55b137f57fe0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -421> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient(hunting): picked mon.noname-b con 0x55b138000b00 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -420> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -419> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -418> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -417> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient(hunting): _renew_subs 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -416> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient(hunting): authenticate will time out at 2100.905167s 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -415> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient(hunting): get_auth_request con 0x55b137f11c30 auth_method 0 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -414> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -413> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -412> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:11.118 INFO:tasks.workunit.client.0.trial046.stderr: -411> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr: -410> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr: -409> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr: -408> 2026-02-24T22:22:10.798+0000 7f5af3fff640 10 monclient(hunting): get_auth_request con 0x55b137f57fe0 auth_method 0 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr: -407> 2026-02-24T22:22:10.798+0000 7f5af3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr: -406> 2026-02-24T22:22:10.798+0000 7f5af3fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr: -405> 2026-02-24T22:22:10.798+0000 7f5af3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr: -404> 2026-02-24T22:22:10.798+0000 7f5af3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr: -403> 2026-02-24T22:22:10.798+0000 7f5af3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr: -402> 2026-02-24T22:22:10.798+0000 7f5af3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr: -401> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_done global_id 7985 payload 824 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr: -400> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient: _finish_hunting 0 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr: -399> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 1 monclient: found mon.noname-c 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr: -398> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr: -397> 2026-02-24T22:22:10.798+0000 7f5ae3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr: -396> 2026-02-24T22:22:10.798+0000 7f5ae3fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr: -395> 2026-02-24T22:22:10.798+0000 7f5ae3fff640 10 monclient: dump: 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr: -394> 2026-02-24T22:22:10.798+0000 7f5ae3fff640 10 monclient: _finish_auth 0 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr: -393> 2026-02-24T22:22:10.798+0000 7f5ae3fff640 10 monclient: _check_auth_tickets 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr: -392> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 5 monclient: authenticate success, global_id 7985 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr: -391> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient: _renew_subs 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr: -390> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr: -389> 2026-02-24T22:22:10.798+0000 7f5ae3fff640 10 monclient: handle_config config(0 keys) 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr: -388> 2026-02-24T22:22:10.798+0000 7f5ae3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr: -387> 2026-02-24T22:22:10.798+0000 7f5ae3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr: -386> 2026-02-24T22:22:10.798+0000 7f5ae3fff640 10 monclient: dump: 2026-02-24T22:22:11.119 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr: -385> 2026-02-24T22:22:10.798+0000 7f5af1ffb640 4 set_mon_vals no callback set 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr: -384> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient: _renew_subs 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr: -383> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr: -382> 2026-02-24T22:22:10.798+0000 7f5b01f84a80 1 librados: init done 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr: -381> 2026-02-24T22:22:10.798+0000 7f5ae3fff640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr: -380> 2026-02-24T22:22:10.798+0000 7f5ae3fff640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr: -379> 2026-02-24T22:22:10.798+0000 7f5ae3fff640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr: -378> 2026-02-24T22:22:10.798+0000 7f5af3fff640 10 monclient: get_auth_request con 0x7f5adc070730 auth_method 0 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr: -377> 2026-02-24T22:22:10.798+0000 7f5afbfff640 10 monclient: get_auth_request con 0x55b138001410 auth_method 0 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr: -376> 2026-02-24T22:22:10.798+0000 7f5afb7fe640 10 monclient: get_auth_request con 0x55b1380579a0 auth_method 0 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr: -375> 2026-02-24T22:22:10.802+0000 7f5af3fff640 10 monclient: get_auth_request con 0x55b1380591a0 auth_method 0 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr: -374> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient: build_initial_monmap 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr: -373> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 1 build_initial for_mkfs: 0 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr: -372> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient: monmap: 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr: -371> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding auth protocol: cephx 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr: -370> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding auth protocol: cephx 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr: -369> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding auth protocol: cephx 2026-02-24T22:22:11.120 INFO:tasks.workunit.client.0.trial046.stderr: -368> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding con mode: secure 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -367> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding con mode: crc 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -366> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding con mode: secure 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -365> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding con mode: crc 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -364> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding con mode: secure 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -363> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding con mode: crc 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -362> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding con mode: crc 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -361> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding con mode: secure 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -360> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding con mode: crc 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -359> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding con mode: secure 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -358> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding con mode: crc 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -357> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b13805c020) adding con mode: secure 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -356> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -355> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command objecter_requests cmddesc objecter_requests hook 0x55b138060b60 EEXIST 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -354> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient: init 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -353> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding auth protocol: cephx 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -352> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding auth protocol: cephx 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -351> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding auth protocol: cephx 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -350> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding con mode: secure 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -349> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding con mode: crc 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -348> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding con mode: secure 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -347> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding con mode: crc 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -346> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding con mode: secure 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -345> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding con mode: crc 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -344> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding con mode: crc 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -343> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding con mode: secure 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -342> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding con mode: crc 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -341> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding con mode: secure 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -340> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding con mode: crc 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -339> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 AuthRegistry(0x55b138059c08) adding con mode: secure 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -338> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -337> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -336> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command rotate-key cmddesc rotate-key hook 0x55b138059d50 EEXIST 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -335> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient: _reopen_session rank -1 2026-02-24T22:22:11.121 INFO:tasks.workunit.client.0.trial046.stderr: -334> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -333> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient(hunting): picked mon.noname-b con 0x55b1381b3fc0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -332> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient(hunting): picked mon.noname-a con 0x55b1381b0c50 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -331> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient(hunting): picked mon.noname-c con 0x55b1381ad1a0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -330> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -329> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -328> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -327> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient(hunting): _renew_subs 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -326> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient(hunting): authenticate will time out at 2100.910986s 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -325> 2026-02-24T22:22:10.802+0000 7f5af3fff640 10 monclient(hunting): get_auth_request con 0x55b1381b0c50 auth_method 0 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -324> 2026-02-24T22:22:10.802+0000 7f5af3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -323> 2026-02-24T22:22:10.802+0000 7f5af3fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -322> 2026-02-24T22:22:10.802+0000 7f5af3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -321> 2026-02-24T22:22:10.802+0000 7f5afbfff640 10 monclient(hunting): get_auth_request con 0x55b1381ad1a0 auth_method 0 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -320> 2026-02-24T22:22:10.802+0000 7f5afbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -319> 2026-02-24T22:22:10.802+0000 7f5afbfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -318> 2026-02-24T22:22:10.802+0000 7f5afbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -317> 2026-02-24T22:22:10.802+0000 7f5af3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -316> 2026-02-24T22:22:10.802+0000 7f5af3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -315> 2026-02-24T22:22:10.802+0000 7f5af3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -314> 2026-02-24T22:22:10.802+0000 7f5afbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -313> 2026-02-24T22:22:10.802+0000 7f5afbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -312> 2026-02-24T22:22:10.802+0000 7f5afbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -311> 2026-02-24T22:22:10.802+0000 7f5afb7fe640 10 monclient(hunting): get_auth_request con 0x55b1381b3fc0 auth_method 0 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -310> 2026-02-24T22:22:10.802+0000 7f5afb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -309> 2026-02-24T22:22:10.802+0000 7f5afb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -308> 2026-02-24T22:22:10.802+0000 7f5afb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -307> 2026-02-24T22:22:10.802+0000 7f5af3fff640 10 monclient(hunting): handle_auth_done global_id 8565 payload 824 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -306> 2026-02-24T22:22:10.802+0000 7f5af3fff640 10 monclient: _finish_hunting 0 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -305> 2026-02-24T22:22:10.802+0000 7f5af3fff640 1 monclient: found mon.noname-a 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -304> 2026-02-24T22:22:10.802+0000 7f5af3fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -303> 2026-02-24T22:22:10.802+0000 7f5ae17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -302> 2026-02-24T22:22:10.802+0000 7f5ae17fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr: -301> 2026-02-24T22:22:10.802+0000 7f5ae17fa640 10 monclient: dump: 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.122 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr: -300> 2026-02-24T22:22:10.802+0000 7f5ae17fa640 10 monclient: _finish_auth 0 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr: -299> 2026-02-24T22:22:10.802+0000 7f5ae17fa640 10 monclient: _check_auth_tickets 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr: -298> 2026-02-24T22:22:10.802+0000 7f5ae17fa640 10 monclient: handle_config config(0 keys) 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr: -297> 2026-02-24T22:22:10.802+0000 7f5ae17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr: -296> 2026-02-24T22:22:10.802+0000 7f5ae17fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr: -295> 2026-02-24T22:22:10.802+0000 7f5ae17fa640 10 monclient: dump: 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr: -294> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 5 monclient: authenticate success, global_id 8565 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr: -293> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient: _renew_subs 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr: -292> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr: -291> 2026-02-24T22:22:10.802+0000 7f5b00a5a640 4 set_mon_vals no callback set 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr: -290> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient: _renew_subs 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr: -289> 2026-02-24T22:22:10.802+0000 7f5b01f84a80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr: -288> 2026-02-24T22:22:10.806+0000 7f5ae17fa640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr: -287> 2026-02-24T22:22:10.806+0000 7f5ae17fa640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr: -286> 2026-02-24T22:22:10.806+0000 7f5ae17fa640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr: -285> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient: get_auth_request con 0x7f5ad00706c0 auth_method 0 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr: -284> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: get_monmap_and_config 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr: -283> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: build_initial_monmap 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr: -282> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 1 build_initial for_mkfs: 0 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr: -281> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: monmap: 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:22:11.123 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -280> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding auth protocol: cephx 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -279> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding auth protocol: cephx 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -278> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding auth protocol: cephx 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -277> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding con mode: secure 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -276> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding con mode: crc 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -275> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding con mode: secure 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -274> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding con mode: crc 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -273> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding con mode: secure 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -272> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding con mode: crc 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -271> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding con mode: crc 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -270> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding con mode: secure 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -269> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding con mode: crc 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -268> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding con mode: secure 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -267> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding con mode: crc 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -266> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245150) adding con mode: secure 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -265> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -264> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: init 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -263> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding auth protocol: cephx 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -262> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding auth protocol: cephx 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -261> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding auth protocol: cephx 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -260> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding con mode: secure 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -259> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding con mode: crc 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -258> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding con mode: secure 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -257> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding con mode: crc 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -256> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding con mode: secure 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -255> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding con mode: crc 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -254> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding con mode: crc 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -253> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding con mode: secure 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -252> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding con mode: crc 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -251> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding con mode: secure 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -250> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding con mode: crc 2026-02-24T22:22:11.124 INFO:tasks.workunit.client.0.trial046.stderr: -249> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5780) adding con mode: secure 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -248> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -247> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -246> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command rotate-key cmddesc rotate-key hook 0x7fff59fa58c8 EEXIST 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -245> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: _reopen_session rank -1 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -244> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: _add_conns ranks=[0,1,2] 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -243> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): picked mon.noname-a con 0x55b13826c180 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -242> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): picked mon.noname-b con 0x55b138246d70 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -241> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): picked mon.noname-c con 0x55b138273660 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -240> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -239> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -238> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -237> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): _renew_subs 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -236> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): authenticate will time out at 2100.913713s 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -235> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient(hunting): get_auth_request con 0x55b138273660 auth_method 0 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -234> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -233> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -232> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -231> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): get_auth_request con 0x55b13826c180 auth_method 0 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -230> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -229> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -228> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -227> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -226> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -225> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -224> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -223> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -222> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -221> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient(hunting): handle_auth_done global_id 7991 payload 274 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -220> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient: _finish_hunting 0 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -219> 2026-02-24T22:22:10.806+0000 7f5afbfff640 1 monclient: found mon.noname-c 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -218> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -217> 2026-02-24T22:22:10.806+0000 7f5ad6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -216> 2026-02-24T22:22:10.806+0000 7f5ad6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr: -215> 2026-02-24T22:22:10.806+0000 7f5ad6ffd640 10 monclient: dump: 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:11.125 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr: -214> 2026-02-24T22:22:10.806+0000 7f5ad6ffd640 10 monclient: _finish_auth 0 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr: -213> 2026-02-24T22:22:10.806+0000 7f5ad6ffd640 10 monclient: _check_auth_tickets 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr: -212> 2026-02-24T22:22:10.806+0000 7f5ad6ffd640 10 monclient: handle_config config(0 keys) 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr: -211> 2026-02-24T22:22:10.806+0000 7f5ad6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr: -210> 2026-02-24T22:22:10.806+0000 7f5ad6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr: -209> 2026-02-24T22:22:10.806+0000 7f5ad6ffd640 10 monclient: dump: 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr: -208> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 monclient: authenticate success, global_id 7991 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr: -207> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: get_monmap_and_config success 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr: -206> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 4 set_mon_vals no callback set 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr: -205> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: shutdown 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr: -204> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: build_initial_monmap 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr: -203> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 1 build_initial for_mkfs: 0 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr: -202> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: monmap: 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:22:11.126 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -201> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding auth protocol: cephx 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -200> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding auth protocol: cephx 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -199> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding auth protocol: cephx 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -198> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding con mode: secure 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -197> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding con mode: crc 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -196> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding con mode: secure 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -195> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding con mode: crc 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -194> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding con mode: secure 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -193> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding con mode: crc 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -192> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding con mode: crc 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -191> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding con mode: secure 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -190> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding con mode: crc 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -189> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding con mode: secure 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -188> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding con mode: crc 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -187> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138245020) adding con mode: secure 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -186> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -185> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 1 librados: starting msgr at 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -184> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 1 librados: starting objecter 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -183> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command objecter_requests cmddesc objecter_requests hook 0x55b138273540 EEXIST 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -182> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 1 librados: setting wanted keys 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -181> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 1 librados: calling monclient init 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -180> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: init 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -179> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding auth protocol: cephx 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -178> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding auth protocol: cephx 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -177> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding auth protocol: cephx 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -176> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding con mode: secure 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -175> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding con mode: crc 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -174> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding con mode: secure 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -173> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding con mode: crc 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -172> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding con mode: secure 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -171> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding con mode: crc 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -170> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding con mode: crc 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -169> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding con mode: secure 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -168> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding con mode: crc 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -167> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding con mode: secure 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -166> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding con mode: crc 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -165> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 AuthRegistry(0x55b138243ce0) adding con mode: secure 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -164> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -163> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -162> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command rotate-key cmddesc rotate-key hook 0x55b138243e28 EEXIST 2026-02-24T22:22:11.127 INFO:tasks.workunit.client.0.trial046.stderr: -161> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: _reopen_session rank -1 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -160> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -159> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): picked mon.noname-c con 0x55b13826c180 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -158> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): picked mon.noname-a con 0x55b138273660 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -157> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): picked mon.noname-b con 0x55b138271fa0 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -156> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -155> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -154> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -153> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): _renew_subs 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -152> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient(hunting): authenticate will time out at 2100.914961s 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -151> 2026-02-24T22:22:10.806+0000 7f5af3fff640 10 monclient(hunting): get_auth_request con 0x55b138273660 auth_method 0 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -150> 2026-02-24T22:22:10.806+0000 7f5af3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -149> 2026-02-24T22:22:10.806+0000 7f5af3fff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -148> 2026-02-24T22:22:10.806+0000 7f5af3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -147> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): get_auth_request con 0x55b13826c180 auth_method 0 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -146> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -145> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -144> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -143> 2026-02-24T22:22:10.806+0000 7f5af3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -142> 2026-02-24T22:22:10.806+0000 7f5af3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -141> 2026-02-24T22:22:10.806+0000 7f5af3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -140> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -139> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -138> 2026-02-24T22:22:10.806+0000 7f5afb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -137> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient(hunting): get_auth_request con 0x55b138271fa0 auth_method 0 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -136> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -135> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -134> 2026-02-24T22:22:10.806+0000 7f5afbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -133> 2026-02-24T22:22:10.806+0000 7f5af3fff640 10 monclient(hunting): handle_auth_done global_id 8571 payload 824 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -132> 2026-02-24T22:22:10.806+0000 7f5af3fff640 10 monclient: _finish_hunting 0 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -131> 2026-02-24T22:22:10.806+0000 7f5af3fff640 1 monclient: found mon.noname-a 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -130> 2026-02-24T22:22:10.806+0000 7f5af3fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -129> 2026-02-24T22:22:10.806+0000 7f5ad57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -128> 2026-02-24T22:22:10.806+0000 7f5ad57fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr: -127> 2026-02-24T22:22:10.806+0000 7f5ad57fa640 10 monclient: dump: 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:11.128 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: -126> 2026-02-24T22:22:10.806+0000 7f5ad57fa640 10 monclient: _finish_auth 0 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: -125> 2026-02-24T22:22:10.806+0000 7f5ad57fa640 10 monclient: _check_auth_tickets 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: -124> 2026-02-24T22:22:10.806+0000 7f5ad57fa640 10 monclient: handle_config config(0 keys) 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: -123> 2026-02-24T22:22:10.806+0000 7f5ad57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: -122> 2026-02-24T22:22:10.806+0000 7f5ad57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: -121> 2026-02-24T22:22:10.806+0000 7f5ad57fa640 10 monclient: dump: 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: -120> 2026-02-24T22:22:10.806+0000 7f5ad6ffd640 4 set_mon_vals no callback set 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: -119> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 monclient: authenticate success, global_id 8571 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: -118> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: _renew_subs 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: -117> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: -116> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: _renew_subs 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: -115> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: -114> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 1 librados: init done 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: -113> 2026-02-24T22:22:10.806+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command cr dump hook 0x55b138271888 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: -112> 2026-02-24T22:22:10.810+0000 7f5b01f84a80 4 Realm: () 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: -111> 2026-02-24T22:22:10.810+0000 7f5b01f84a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: -110> 2026-02-24T22:22:10.810+0000 7f5b01f84a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: -109> 2026-02-24T22:22:10.810+0000 7f5b01f84a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: -108> 2026-02-24T22:22:10.810+0000 7f5b01f84a80 20 zonegroup default 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: -107> 2026-02-24T22:22:10.810+0000 7f5afb7fe640 10 monclient: get_auth_request con 0x55b13830d8e0 auth_method 0 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: -106> 2026-02-24T22:22:10.810+0000 7f5ad57fa640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: -105> 2026-02-24T22:22:10.810+0000 7f5ad57fa640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: -104> 2026-02-24T22:22:10.810+0000 7f5ad57fa640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: -103> 2026-02-24T22:22:10.810+0000 7f5b01f84a80 20 rados->read ofs=0 len=0 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: -102> 2026-02-24T22:22:10.810+0000 7f5afbfff640 10 monclient: get_auth_request con 0x7f5ac00707b0 auth_method 0 2026-02-24T22:22:11.129 INFO:tasks.workunit.client.0.trial046.stderr: -101> 2026-02-24T22:22:10.810+0000 7f5afb7fe640 10 monclient: get_auth_request con 0x55b138330c00 auth_method 0 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -100> 2026-02-24T22:22:10.810+0000 7f5b01f84a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -99> 2026-02-24T22:22:10.810+0000 7f5b01f84a80 20 rados->read ofs=0 len=0 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -98> 2026-02-24T22:22:10.810+0000 7f5af3fff640 10 monclient: get_auth_request con 0x55b138352230 auth_method 0 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -97> 2026-02-24T22:22:10.810+0000 7f5b01f84a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -96> 2026-02-24T22:22:10.810+0000 7f5b01f84a80 20 started sync module instance, tier type = 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -95> 2026-02-24T22:22:10.810+0000 7f5b01f84a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -94> 2026-02-24T22:22:10.810+0000 7f5afbfff640 10 monclient: get_auth_request con 0x55b1383b36b0 auth_method 0 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -93> 2026-02-24T22:22:10.810+0000 7f5afbfff640 10 monclient: get_auth_request con 0x55b1384bb0a0 auth_method 0 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -92> 2026-02-24T22:22:10.810+0000 7f5afb7fe640 10 monclient: get_auth_request con 0x55b138415fe0 auth_method 0 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -91> 2026-02-24T22:22:10.810+0000 7f5af3fff640 10 monclient: get_auth_request con 0x55b138478f60 auth_method 0 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -90> 2026-02-24T22:22:10.810+0000 7f5afb7fe640 10 monclient: get_auth_request con 0x55b13851e0a0 auth_method 0 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -89> 2026-02-24T22:22:10.814+0000 7f5afb7fe640 0 -- 10.20.193.46:0/1549073879 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x55b13851e0a0 msgr2=0x55b13853e520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -88> 2026-02-24T22:22:10.814+0000 7f5ad57fa640 1 client.8571.objecter ms_handle_reset 0x55b13851e0a0 session 0x55b13851d990 osd.6 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -87> 2026-02-24T22:22:10.814+0000 7f5ad57fa640 10 monclient: _renew_subs 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -86> 2026-02-24T22:22:10.814+0000 7f5ad57fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -85> 2026-02-24T22:22:10.814+0000 7f5afb7fe640 10 monclient: get_auth_request con 0x7f5ac009d540 auth_method 0 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -84> 2026-02-24T22:22:10.814+0000 7f5b01f84a80 20 add_watcher() i=1 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -83> 2026-02-24T22:22:10.814+0000 7f5b01f84a80 20 add_watcher() i=5 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -82> 2026-02-24T22:22:10.814+0000 7f5b01f84a80 20 add_watcher() i=0 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -81> 2026-02-24T22:22:10.814+0000 7f5b01f84a80 20 add_watcher() i=6 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -80> 2026-02-24T22:22:10.814+0000 7f5b01f84a80 20 add_watcher() i=2 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -79> 2026-02-24T22:22:10.814+0000 7f5b01f84a80 20 add_watcher() i=4 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -78> 2026-02-24T22:22:10.814+0000 7f5b01f84a80 20 add_watcher() i=3 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -77> 2026-02-24T22:22:10.814+0000 7f5b01f84a80 20 add_watcher() i=7 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -76> 2026-02-24T22:22:10.814+0000 7f5b01f84a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -75> 2026-02-24T22:22:10.814+0000 7f5af3fff640 10 monclient: get_auth_request con 0x7f5a20008af0 auth_method 0 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -74> 2026-02-24T22:22:10.814+0000 7f5ae17fa640 1 client.8565.objecter ms_handle_reset 0x7f5a20008af0 session 0x7f5a20008470 osd.1 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -73> 2026-02-24T22:22:10.814+0000 7f5ae17fa640 10 monclient: _renew_subs 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -72> 2026-02-24T22:22:10.814+0000 7f5ae17fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -71> 2026-02-24T22:22:10.814+0000 7f5af3fff640 10 monclient: get_auth_request con 0x7f5ad009d3f0 auth_method 0 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -70> 2026-02-24T22:22:10.814+0000 7f5af3fff640 0 -- 10.20.193.46:0/3650014496 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f5ad009d3f0 msgr2=0x7f5ad00bd7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -69> 2026-02-24T22:22:11.018+0000 7f5af3fff640 10 monclient: get_auth_request con 0x7f5ad009d3f0 auth_method 0 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -68> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding auth protocol: cephx 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -67> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding auth protocol: cephx 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -66> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding auth protocol: cephx 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -65> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding con mode: secure 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -64> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding con mode: crc 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -63> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding con mode: secure 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -62> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding con mode: crc 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -61> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding con mode: secure 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -60> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding con mode: crc 2026-02-24T22:22:11.130 INFO:tasks.workunit.client.0.trial046.stderr: -59> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding con mode: crc 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -58> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding con mode: secure 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -57> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding con mode: crc 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -56> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding con mode: secure 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -55> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding con mode: crc 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -54> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 AuthRegistry(0x7fff59fa5bf0) adding con mode: secure 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -53> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -52> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -51> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -50> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command cache list hook 0x55b1382474e0 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -49> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command cache inspect hook 0x55b1382474e0 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -48> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command cache erase hook 0x55b1382474e0 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -47> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command cache zap hook 0x55b1382474e0 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -46> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 note: GC not initialized 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -45> 2026-02-24T22:22:11.018+0000 7f5aacfe1640 20 reqs_thread_entry: start 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -44> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command sync trace show hook 0x7f5aec0043a0 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -43> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command sync trace history hook 0x7f5aec0043a0 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -42> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command sync trace active hook 0x7f5aec0043a0 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -41> 2026-02-24T22:22:11.018+0000 7f5b01f84a80 5 asok(0x55b137ec9f00) register_command sync trace active_short hook 0x7f5aec0043a0 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -40> 2026-02-24T22:22:11.018+0000 7f5afbfff640 10 monclient: get_auth_request con 0x55b138403c20 auth_method 0 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -39> 2026-02-24T22:22:11.022+0000 7f5afb7fe640 10 monclient: get_auth_request con 0x55b1383f5fc0 auth_method 0 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -38> 2026-02-24T22:22:11.022+0000 7f5af3fff640 10 monclient: get_auth_request con 0x55b1383f7670 auth_method 0 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -37> 2026-02-24T22:22:11.022+0000 7f5afbfff640 10 monclient: get_auth_request con 0x55b1383f8ea0 auth_method 0 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -36> 2026-02-24T22:22:11.026+0000 7f5afb7fe640 10 monclient: get_auth_request con 0x55b138408010 auth_method 0 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -35> 2026-02-24T22:22:11.030+0000 7f5af3fff640 10 monclient: get_auth_request con 0x55b1383fd010 auth_method 0 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -34> 2026-02-24T22:22:11.034+0000 7f5ae17fa640 1 client.8565.objecter ms_handle_reset 0x55b1383f8ea0 session 0x55b1383f8820 osd.0 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -33> 2026-02-24T22:22:11.034+0000 7f5afbfff640 10 monclient: get_auth_request con 0x7f5a20008af0 auth_method 0 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -32> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 20 init_complete bucket index max shards: 1 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -31> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 20 Filter name: none 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -30> 2026-02-24T22:22:11.054+0000 7f5a16ffd640 20 reqs_thread_entry: start 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -29> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -28> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 20 rados->read ofs=0 len=0 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -27> 2026-02-24T22:22:11.054+0000 7f5afbfff640 10 monclient: get_auth_request con 0x55b137fc80e0 auth_method 0 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -26> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -25> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -24> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -23> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : miss 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -22> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 20 rados->read ofs=0 len=0 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -21> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 20 rados_obj.operate() r=0 bl.length=470 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -20> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -19> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -18> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -17> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:22:11.131 INFO:tasks.workunit.client.0.trial046.stderr: -16> 2026-02-24T22:22:11.054+0000 7f5b01f84a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: -15> 2026-02-24T22:22:11.058+0000 7f5b01f84a80 10 init_target_layout removing existing target index objects from a previous reshard attempt 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: -14> 2026-02-24T22:22:11.062+0000 7f5b01f84a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: -13> 2026-02-24T22:22:11.062+0000 7f5b01f84a80 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: -12> 2026-02-24T22:22:11.062+0000 7f5b01f84a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: -11> 2026-02-24T22:22:11.062+0000 7f5b01f84a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518, ofs0, ns] 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: -10> 2026-02-24T22:22:11.062+0000 7f5ad6ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959946 cookie 94219642602960 notifier 8571 bl.length()=929 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: -9> 2026-02-24T22:22:11.062+0000 7f5ad6ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: -8> 2026-02-24T22:22:11.062+0000 7f5ad6ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: -7> 2026-02-24T22:22:11.062+0000 7f5ad6ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: -6> 2026-02-24T22:22:11.062+0000 7f5b01f84a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_sources= new_sources= 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: -5> 2026-02-24T22:22:11.062+0000 7f5b01f84a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential sources added=[] removed=[] 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: -4> 2026-02-24T22:22:11.062+0000 7f5b01f84a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_dests= new_dests= 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: -3> 2026-02-24T22:22:11.062+0000 7f5b01f84a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential dests added=[] removed=[] 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: -2> 2026-02-24T22:22:11.062+0000 7f5ad6ffd640 20 trim reshard logs: write on '.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1' needs retry: Success 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: -1> 2026-02-24T22:22:11.066+0000 7f5b01f84a80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f5b01f84a80 time 2026-02-24T22:22:11.067093+0000 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f5b053c6eb1] 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x55b12bb99b9f] 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: 3: radosgw-admin(+0x1714451) [0x55b12cda8451] 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x55b12c305202] 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: 5: main() 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f5b046c6d90] 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: 7: __libc_start_main() 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: 8: _start() 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: 0> 2026-02-24T22:22:11.066+0000 7f5b01f84a80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f5b01f84a80 thread_name:radosgw-admin 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f5b046df520] 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f5b053c6f10] 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x55b12bb99b9f] 2026-02-24T22:22:11.132 INFO:tasks.workunit.client.0.trial046.stderr: 7: radosgw-admin(+0x1714451) [0x55b12cda8451] 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x102) [0x55b12c305202] 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 9: main() 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f5b046c6d90] 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 11: __libc_start_main() 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 12: _start() 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr:--- logging levels --- 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 none 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 lockdep 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 context 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 crush 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_balancer 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_locker 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log_expire 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_migrator 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 3/ 5 mds_quiesce 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 buffer 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 timer 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 filer 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 striper 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 objecter 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rados 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_mirror 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_replay 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_pwl 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 journaler 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objectcacher 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 immutable_obj_cache 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 client 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 osd 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 optracker 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objclass 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 filestore 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 journal 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 ms 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mon 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 0/10 monc 2026-02-24T22:22:11.133 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 paxos 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 tp 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 auth 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 crypto 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 finisher 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 reserver 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 heartbeatmap 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 perfcounter 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_sync 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_datacache 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_access 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dbstore 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_flight 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_lifecycle 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_restore 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw_notification 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_bucket_logging 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 javaclient 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 asok 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 throttle 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 refs 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 compressor 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore_compression 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluefs 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 bdev 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 kstore 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 4/ 5 rocksdb 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 fuse 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 2/ 5 mgr 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mgrc 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 dpdk 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 eventtrace 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 prioritycache 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 test 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephfs_mirror 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephsqlite 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 crimson_interrupt 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_onode 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_odata 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_omap 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_tm 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_t 2026-02-24T22:22:11.134 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cleaner 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_epm 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_lba 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_fixedkv_tree 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cache 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_journal 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_device 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_backref 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 alienstore 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mclock 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dedup 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cyanstore 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 ceph_exporter 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 memstore 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 trace 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 ceph_dedup 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: -2/-2 (syslog threshold) 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 99/99 (stderr threshold) 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr:--- pthread ID / name mapping for recent threads --- 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 7f5a16ffd640 / http_manager 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 7f5aacfe1640 / http_manager 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 7f5ad57fa640 / ms_dispatch 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 7f5ad6ffd640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 7f5ae17fa640 / ms_dispatch 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 7f5ae3fff640 / ms_dispatch 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 7f5af1ffb640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 7f5af27fc640 / admin_socket 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 7f5af3fff640 / msgr-worker-2 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 7f5afaffd640 / ms_dispatch 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 7f5afb7fe640 / msgr-worker-1 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 7f5afbfff640 / msgr-worker-0 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 7f5b00a5a640 / io_context_pool 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: 7f5b01f84a80 / radosgw-admin 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: max_recent 500 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: max_new 1000 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr: log_file /var/lib/ceph/crash/2026-02-24T22:22:11.069590Z_554d0aed-f7c1-4558-a06f-6f78016a8343/log 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr:--- end dump of recent events --- 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr:Aborted 2026-02-24T22:22:11.135 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin reshardlog list --bucket a-bucket 2026-02-24T22:22:11.136 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:22:11.136 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:22:11.136 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.106+0000 7fca0ee12a80 4 Realm: () 2026-02-24T22:22:11.136 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.106+0000 7fca0ee12a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:22:11.136 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.106+0000 7fca0ee12a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:22:11.136 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.106+0000 7fca0ee12a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:22:11.136 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.106+0000 7fca0ee12a80 20 zonegroup default 2026-02-24T22:22:11.136 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.106+0000 7fca0ee12a80 20 rados->read ofs=0 len=0 2026-02-24T22:22:11.136 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.106+0000 7fca0ee12a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:22:11.136 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.106+0000 7fca0ee12a80 20 rados->read ofs=0 len=0 2026-02-24T22:22:11.136 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.106+0000 7fca0ee12a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:22:11.136 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.106+0000 7fca0ee12a80 20 started sync module instance, tier type = 2026-02-24T22:22:11.136 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.106+0000 7fca0ee12a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:22:11.136 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.110+0000 7fca0d0e7640 0 -- 10.20.193.46:0/4041342326 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x565069f75160 msgr2=0x565069f95540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:22:11.136 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.110+0000 7fca0ee12a80 20 add_watcher() i=1 2026-02-24T22:22:11.137 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.110+0000 7fca0ee12a80 20 add_watcher() i=6 2026-02-24T22:22:11.137 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.110+0000 7fca0ee12a80 20 add_watcher() i=5 2026-02-24T22:22:11.137 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.110+0000 7fca0ee12a80 20 add_watcher() i=3 2026-02-24T22:22:11.137 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.110+0000 7fca0ee12a80 20 add_watcher() i=2 2026-02-24T22:22:11.137 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.110+0000 7fca0ee12a80 20 add_watcher() i=7 2026-02-24T22:22:11.137 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.110+0000 7fca0ee12a80 20 add_watcher() i=4 2026-02-24T22:22:11.137 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.110+0000 7fca0ee12a80 20 add_watcher() i=0 2026-02-24T22:22:11.137 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.110+0000 7fca0ee12a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:22:11.137 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.110+0000 7fca0ee12a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:22:11.137 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.110+0000 7fca0ee12a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:22:11.137 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.110+0000 7fca0ee12a80 5 note: GC not initialized 2026-02-24T22:22:11.137 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.110+0000 7fc9b9fe3640 20 reqs_thread_entry: start 2026-02-24T22:22:11.137 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.110+0000 7fca0d0e7640 0 -- 10.20.193.46:0/3627028729 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x56506a06b800 msgr2=0x56506a066570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:22:11.343 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.350+0000 7fca0ee12a80 20 init_complete bucket index max shards: 1 2026-02-24T22:22:11.343 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.350+0000 7fca0ee12a80 20 Filter name: none 2026-02-24T22:22:11.343 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.350+0000 7fc913fff640 20 reqs_thread_entry: start 2026-02-24T22:22:11.343 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.350+0000 7fca0ee12a80 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2026-02-24T22:22:11.343 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.350+0000 7fca0ee12a80 20 rados->read ofs=0 len=0 2026-02-24T22:22:11.343 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.350+0000 7fca0ee12a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:22:11.343 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.350+0000 7fca0ee12a80 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2026-02-24T22:22:11.343 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.350+0000 7fca0ee12a80 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2026-02-24T22:22:11.344 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.350+0000 7fca0ee12a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : miss 2026-02-24T22:22:11.344 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.350+0000 7fca0ee12a80 20 rados->read ofs=0 len=0 2026-02-24T22:22:11.344 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.350+0000 7fca0ee12a80 20 rados_obj.operate() r=0 bl.length=470 2026-02-24T22:22:11.344 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.350+0000 7fca0ee12a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:22:11.344 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.350+0000 7fca0ee12a80 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:22:11.344 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.350+0000 7fca0ee12a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:22:11.344 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.350+0000 7fca0ee12a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:22:11.344 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.350+0000 7fca0ee12a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:22:11.344 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.350+0000 7fca0ee12a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 2026-02-24T22:22:11.344 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.350+0000 7fca0ee12a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 2026-02-24T22:22:11.344 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.350+0000 7fca0ee12a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1 2026-02-24T22:22:11.344 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.350+0000 7fca0ee12a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1 2026-02-24T22:22:11.344 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.350+0000 7fca0ee12a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.2 2026-02-24T22:22:11.345 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.350+0000 7fca0ee12a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.2 2026-02-24T22:22:11.348 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.354+0000 7fca0ee12a80 20 remove_watcher() i=3 2026-02-24T22:22:11.348 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.354+0000 7fca0ee12a80 2 removed watcher, disabling cache 2026-02-24T22:22:11.348 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.354+0000 7fca0ee12a80 20 remove_watcher() i=2 2026-02-24T22:22:11.348 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.354+0000 7fca0ee12a80 20 remove_watcher() i=5 2026-02-24T22:22:11.348 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.354+0000 7fca0ee12a80 20 remove_watcher() i=6 2026-02-24T22:22:11.348 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.354+0000 7fca0ee12a80 20 remove_watcher() i=1 2026-02-24T22:22:11.348 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.354+0000 7fca0ee12a80 20 remove_watcher() i=4 2026-02-24T22:22:11.348 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.354+0000 7fca0ee12a80 20 remove_watcher() i=0 2026-02-24T22:22:11.348 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:11.354+0000 7fca0ee12a80 20 remove_watcher() i=7 2026-02-24T22:22:11.354 INFO:tasks.workunit.client.0.trial046.stderr: test: duplicated entries can be purged manually 2026-02-24T22:22:41.361 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket a-bucket 2026-02-24T22:22:41.377 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:22:41.377 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:22:41.395 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.402+0000 7f925ef87a80 4 Realm: () 2026-02-24T22:22:41.395 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.402+0000 7f925ef87a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:22:41.395 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.402+0000 7f925ef87a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:22:41.395 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.402+0000 7f925ef87a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:22:41.395 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.402+0000 7f925ef87a80 20 zonegroup default 2026-02-24T22:22:41.396 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.402+0000 7f925ef87a80 20 rados->read ofs=0 len=0 2026-02-24T22:22:41.397 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.402+0000 7f925ef87a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:22:41.397 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.402+0000 7f925ef87a80 20 rados->read ofs=0 len=0 2026-02-24T22:22:41.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.402+0000 7f925ef87a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:22:41.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.402+0000 7f925ef87a80 20 started sync module instance, tier type = 2026-02-24T22:22:41.398 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.402+0000 7f925ef87a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:22:41.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.406+0000 7f925ef87a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:22:41.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.406+0000 7f925ef87a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:22:41.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.406+0000 7f925ef87a80 5 note: GC not initialized 2026-02-24T22:22:41.399 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.406+0000 7f920a7e4640 20 reqs_thread_entry: start 2026-02-24T22:22:41.403 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.410+0000 7f925ca5b640 0 -- 10.20.193.46:0/1468453597 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x5602078ad2f0 msgr2=0x5602078aa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:22:41.616 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.622+0000 7f925d25c640 0 -- 10.20.193.46:0/1468453597 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x5602078993a0 msgr2=0x56020789b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:22:41.842 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.850+0000 7f925ef87a80 20 init_complete bucket index max shards: 1 2026-02-24T22:22:41.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.850+0000 7f925ef87a80 20 Filter name: none 2026-02-24T22:22:41.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.850+0000 7f9208fe1640 20 reqs_thread_entry: start 2026-02-24T22:22:41.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.850+0000 7f925ef87a80 20 rados->read ofs=0 len=0 2026-02-24T22:22:41.844 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.850+0000 7f925ef87a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:22:41.844 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.850+0000 7f925ef87a80 20 rados->read ofs=0 len=0 2026-02-24T22:22:41.844 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.850+0000 7f925ef87a80 20 rados_obj.operate() r=0 bl.length=470 2026-02-24T22:22:41.844 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.850+0000 7f925ef87a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:22:41.844 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.850+0000 7f925ef87a80 20 rados->read ofs=0 len=0 2026-02-24T22:22:41.844 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.850+0000 7f925ef87a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:22:41.844 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.850+0000 7f925ef87a80 20 rados->read ofs=0 len=0 2026-02-24T22:22:41.844 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.850+0000 7f925ef87a80 20 rados_obj.operate() r=0 bl.length=470 2026-02-24T22:22:41.845 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.850+0000 7f925ef87a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:22:41.855 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:41.855 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:22:41.855 INFO:tasks.workunit.client.0.trial046.stderr:bucket: a-bucket id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 num_objects: 11 size_kb: 1 num_shards: 3 2026-02-24T22:22:41.855 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:41.856 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket a-bucket --num-shards 4 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at do_reshard 2026-02-24T22:22:41.872 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:22:41.872 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:22:41.892 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.898+0000 7f9c5382ca80 4 Realm: () 2026-02-24T22:22:41.892 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.898+0000 7f9c5382ca80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:22:41.892 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.898+0000 7f9c5382ca80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:22:41.892 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:22:41.892 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.898+0000 7f9c5382ca80 20 zonegroup default 2026-02-24T22:22:41.893 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.898+0000 7f9c5382ca80 20 rados->read ofs=0 len=0 2026-02-24T22:22:41.893 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.898+0000 7f9c5382ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:22:41.893 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.898+0000 7f9c5382ca80 20 rados->read ofs=0 len=0 2026-02-24T22:22:41.894 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.898+0000 7f9c5382ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:22:41.894 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.898+0000 7f9c5382ca80 20 started sync module instance, tier type = 2026-02-24T22:22:41.894 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.898+0000 7f9c5382ca80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:22:41.894 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.902+0000 7f9c51300640 0 -- 10.20.193.46:0/303166036 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x5609886995e0 msgr2=0x5609886b9a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:22:41.896 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.902+0000 7f9c5382ca80 20 add_watcher() i=1 2026-02-24T22:22:41.896 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.902+0000 7f9c5382ca80 20 add_watcher() i=6 2026-02-24T22:22:41.896 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.902+0000 7f9c5382ca80 20 add_watcher() i=3 2026-02-24T22:22:41.896 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.902+0000 7f9c5382ca80 20 add_watcher() i=0 2026-02-24T22:22:41.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.902+0000 7f9c5382ca80 20 add_watcher() i=4 2026-02-24T22:22:41.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.902+0000 7f9c5382ca80 20 add_watcher() i=2 2026-02-24T22:22:41.897 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:41.902+0000 7f9c5382ca80 20 add_watcher() i=5 2026-02-24T22:22:42.101 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.106+0000 7f9c5382ca80 20 add_watcher() i=7 2026-02-24T22:22:42.101 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.106+0000 7f9c5382ca80 2 all 8 watchers are set, enabling cache 2026-02-24T22:22:42.102 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.106+0000 7f9c50aff640 0 -- 10.20.193.46:0/65096060 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f9b70008af0 msgr2=0x7f9b70028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:22:42.103 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.110+0000 7f9c5382ca80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:22:42.103 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.110+0000 7f9c5382ca80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:22:42.104 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 note: GC not initialized 2026-02-24T22:22:42.104 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.110+0000 7f9bfe7e4640 20 reqs_thread_entry: start 2026-02-24T22:22:42.139 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.146+0000 7f9c5382ca80 20 init_complete bucket index max shards: 1 2026-02-24T22:22:42.139 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.146+0000 7f9c5382ca80 20 Filter name: none 2026-02-24T22:22:42.139 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.146+0000 7f9bfcfe1640 20 reqs_thread_entry: start 2026-02-24T22:22:42.139 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2026-02-24T22:22:42.139 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.146+0000 7f9c5382ca80 20 rados->read ofs=0 len=0 2026-02-24T22:22:42.140 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.146+0000 7f9c5382ca80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:22:42.140 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2026-02-24T22:22:42.141 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2026-02-24T22:22:42.141 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : miss 2026-02-24T22:22:42.141 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.146+0000 7f9c5382ca80 20 rados->read ofs=0 len=0 2026-02-24T22:22:42.141 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.146+0000 7f9c5382ca80 20 rados_obj.operate() r=0 bl.length=470 2026-02-24T22:22:42.141 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:22:42.141 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:22:42.141 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:22:42.141 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:22:42.141 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:22:42.142 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 init_target_layout removing existing target index objects from a previous reshard attempt 2026-02-24T22:22:42.146 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.154+0000 7f9c5382ca80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:22:42.146 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.154+0000 7f9c5382ca80 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:22:42.146 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.154+0000 7f9c5382ca80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:22:42.146 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.154+0000 7f9c5382ca80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518, ofs0, ns] 2026-02-24T22:22:42.147 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.154+0000 7f9c2cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959947 cookie 94598943450400 notifier 8625 bl.length()=929 2026-02-24T22:22:42.147 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.154+0000 7f9c2cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:22:42.147 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.154+0000 7f9c2cff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:22:42.147 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.154+0000 7f9c2cff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:22:42.147 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.154+0000 7f9c5382ca80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_sources= new_sources= 2026-02-24T22:22:42.147 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.154+0000 7f9c5382ca80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential sources added=[] removed=[] 2026-02-24T22:22:42.147 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.154+0000 7f9c5382ca80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_dests= new_dests= 2026-02-24T22:22:42.148 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.154+0000 7f9c5382ca80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential dests added=[] removed=[] 2026-02-24T22:22:42.149 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f9c5382ca80 time 2026-02-24T22:22:42.158406+0000 2026-02-24T22:22:42.149 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:22:42.150 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:22:42.150 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f9c56c6eeb1] 2026-02-24T22:22:42.150 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x5609532fcb9f] 2026-02-24T22:22:42.150 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x16a) [0x560953a6826a] 2026-02-24T22:22:42.150 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:22:42.150 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f9c55f6ed90] 2026-02-24T22:22:42.150 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:22:42.150 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:22:42.150 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.154+0000 7f9c5382ca80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f9c5382ca80 time 2026-02-24T22:22:42.158406+0000 2026-02-24T22:22:42.151 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:22:42.151 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.151 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:22:42.151 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f9c56c6eeb1] 2026-02-24T22:22:42.151 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x5609532fcb9f] 2026-02-24T22:22:42.151 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x16a) [0x560953a6826a] 2026-02-24T22:22:42.151 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:22:42.151 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f9c55f6ed90] 2026-02-24T22:22:42.151 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:22:42.151 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:22:42.151 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.152 INFO:tasks.workunit.client.0.trial046.stderr:*** Caught signal (Aborted) ** 2026-02-24T22:22:42.152 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f9c5382ca80 thread_name:radosgw-admin 2026-02-24T22:22:42.173 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:22:42.174 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f9c55f87520] 2026-02-24T22:22:42.174 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:22:42.174 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:22:42.174 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:22:42.174 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f9c56c6ef10] 2026-02-24T22:22:42.174 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x5609532fcb9f] 2026-02-24T22:22:42.174 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x16a) [0x560953a6826a] 2026-02-24T22:22:42.174 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:22:42.174 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f9c55f6ed90] 2026-02-24T22:22:42.174 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:22:42.174 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:22:42.175 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.158+0000 7f9c5382ca80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:22:42.175 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f9c5382ca80 thread_name:radosgw-admin 2026-02-24T22:22:42.175 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.175 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:22:42.175 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f9c55f87520] 2026-02-24T22:22:42.175 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:22:42.175 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:22:42.175 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:22:42.175 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f9c56c6ef10] 2026-02-24T22:22:42.175 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x5609532fcb9f] 2026-02-24T22:22:42.175 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x16a) [0x560953a6826a] 2026-02-24T22:22:42.175 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:22:42.176 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f9c55f6ed90] 2026-02-24T22:22:42.176 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:22:42.176 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:22:42.176 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:22:42.176 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.176 INFO:tasks.workunit.client.0.trial046.stderr:--- begin dump of recent events --- 2026-02-24T22:22:42.176 INFO:tasks.workunit.client.0.trial046.stderr: -499> 2026-02-24T22:22:41.882+0000 7f9c377fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:22:42.176 INFO:tasks.workunit.client.0.trial046.stderr: -498> 2026-02-24T22:22:41.882+0000 7f9c377fe640 10 monclient: dump: 2026-02-24T22:22:42.176 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:42.176 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:42.176 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.177 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.177 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:42.177 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:42.177 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:42.177 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:42.177 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:42.177 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.177 INFO:tasks.workunit.client.0.trial046.stderr: -497> 2026-02-24T22:22:41.882+0000 7f9c5382ca80 5 monclient: authenticate success, global_id 8033 2026-02-24T22:22:42.177 INFO:tasks.workunit.client.0.trial046.stderr: -496> 2026-02-24T22:22:41.882+0000 7f9c5382ca80 10 monclient: get_monmap_and_config success 2026-02-24T22:22:42.177 INFO:tasks.workunit.client.0.trial046.stderr: -495> 2026-02-24T22:22:41.882+0000 7f9c5382ca80 4 set_mon_vals no callback set 2026-02-24T22:22:42.177 INFO:tasks.workunit.client.0.trial046.stderr: -494> 2026-02-24T22:22:41.882+0000 7f9c5382ca80 10 monclient: shutdown 2026-02-24T22:22:42.177 INFO:tasks.workunit.client.0.trial046.stderr: -493> 2026-02-24T22:22:41.882+0000 7f9c5382ca80 5 asok(0x560988045f00) unregister_commands rotate-key 2026-02-24T22:22:42.177 INFO:tasks.workunit.client.0.trial046.stderr: -492> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient: build_initial_monmap 2026-02-24T22:22:42.178 INFO:tasks.workunit.client.0.trial046.stderr: -491> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 1 build_initial for_mkfs: 0 2026-02-24T22:22:42.178 INFO:tasks.workunit.client.0.trial046.stderr: -490> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient: monmap: 2026-02-24T22:22:42.178 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:22:42.178 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:22:42.178 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:22:42.178 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:22:42.178 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:22:42.178 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:42.178 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:22:42.178 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:22:42.178 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:22:42.178 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.178 INFO:tasks.workunit.client.0.trial046.stderr: -489> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding auth protocol: cephx 2026-02-24T22:22:42.179 INFO:tasks.workunit.client.0.trial046.stderr: -488> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding auth protocol: cephx 2026-02-24T22:22:42.179 INFO:tasks.workunit.client.0.trial046.stderr: -487> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding auth protocol: cephx 2026-02-24T22:22:42.179 INFO:tasks.workunit.client.0.trial046.stderr: -486> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding con mode: secure 2026-02-24T22:22:42.179 INFO:tasks.workunit.client.0.trial046.stderr: -485> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding con mode: crc 2026-02-24T22:22:42.179 INFO:tasks.workunit.client.0.trial046.stderr: -484> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding con mode: secure 2026-02-24T22:22:42.179 INFO:tasks.workunit.client.0.trial046.stderr: -483> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding con mode: crc 2026-02-24T22:22:42.179 INFO:tasks.workunit.client.0.trial046.stderr: -482> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding con mode: secure 2026-02-24T22:22:42.179 INFO:tasks.workunit.client.0.trial046.stderr: -481> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding con mode: crc 2026-02-24T22:22:42.179 INFO:tasks.workunit.client.0.trial046.stderr: -480> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding con mode: crc 2026-02-24T22:22:42.179 INFO:tasks.workunit.client.0.trial046.stderr: -479> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding con mode: secure 2026-02-24T22:22:42.179 INFO:tasks.workunit.client.0.trial046.stderr: -478> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding con mode: crc 2026-02-24T22:22:42.179 INFO:tasks.workunit.client.0.trial046.stderr: -477> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding con mode: secure 2026-02-24T22:22:42.179 INFO:tasks.workunit.client.0.trial046.stderr: -476> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding con mode: crc 2026-02-24T22:22:42.180 INFO:tasks.workunit.client.0.trial046.stderr: -475> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding con mode: secure 2026-02-24T22:22:42.180 INFO:tasks.workunit.client.0.trial046.stderr: -474> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:42.180 INFO:tasks.workunit.client.0.trial046.stderr: -473> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 1 librados: starting msgr at 2026-02-24T22:22:42.180 INFO:tasks.workunit.client.0.trial046.stderr: -472> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 1 librados: starting objecter 2026-02-24T22:22:42.180 INFO:tasks.workunit.client.0.trial046.stderr: -471> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command objecter_requests hook 0x560988150200 2026-02-24T22:22:42.180 INFO:tasks.workunit.client.0.trial046.stderr: -470> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 1 librados: setting wanted keys 2026-02-24T22:22:42.180 INFO:tasks.workunit.client.0.trial046.stderr: -469> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 1 librados: calling monclient init 2026-02-24T22:22:42.180 INFO:tasks.workunit.client.0.trial046.stderr: -468> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient: init 2026-02-24T22:22:42.180 INFO:tasks.workunit.client.0.trial046.stderr: -467> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding auth protocol: cephx 2026-02-24T22:22:42.180 INFO:tasks.workunit.client.0.trial046.stderr: -466> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding auth protocol: cephx 2026-02-24T22:22:42.180 INFO:tasks.workunit.client.0.trial046.stderr: -465> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding auth protocol: cephx 2026-02-24T22:22:42.180 INFO:tasks.workunit.client.0.trial046.stderr: -464> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding con mode: secure 2026-02-24T22:22:42.180 INFO:tasks.workunit.client.0.trial046.stderr: -463> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding con mode: crc 2026-02-24T22:22:42.180 INFO:tasks.workunit.client.0.trial046.stderr: -462> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding con mode: secure 2026-02-24T22:22:42.181 INFO:tasks.workunit.client.0.trial046.stderr: -461> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding con mode: crc 2026-02-24T22:22:42.181 INFO:tasks.workunit.client.0.trial046.stderr: -460> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding con mode: secure 2026-02-24T22:22:42.181 INFO:tasks.workunit.client.0.trial046.stderr: -459> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding con mode: crc 2026-02-24T22:22:42.181 INFO:tasks.workunit.client.0.trial046.stderr: -458> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding con mode: crc 2026-02-24T22:22:42.181 INFO:tasks.workunit.client.0.trial046.stderr: -457> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding con mode: secure 2026-02-24T22:22:42.181 INFO:tasks.workunit.client.0.trial046.stderr: -456> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding con mode: crc 2026-02-24T22:22:42.181 INFO:tasks.workunit.client.0.trial046.stderr: -455> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding con mode: secure 2026-02-24T22:22:42.181 INFO:tasks.workunit.client.0.trial046.stderr: -454> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding con mode: crc 2026-02-24T22:22:42.181 INFO:tasks.workunit.client.0.trial046.stderr: -453> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding con mode: secure 2026-02-24T22:22:42.181 INFO:tasks.workunit.client.0.trial046.stderr: -452> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:42.181 INFO:tasks.workunit.client.0.trial046.stderr: -451> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:42.181 INFO:tasks.workunit.client.0.trial046.stderr: -450> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command rotate-key hook 0x560988148f78 2026-02-24T22:22:42.181 INFO:tasks.workunit.client.0.trial046.stderr: -449> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient: _reopen_session rank -1 2026-02-24T22:22:42.181 INFO:tasks.workunit.client.0.trial046.stderr: -448> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T22:22:42.181 INFO:tasks.workunit.client.0.trial046.stderr: -447> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient(hunting): picked mon.noname-b con 0x5609880d3c90 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:22:42.181 INFO:tasks.workunit.client.0.trial046.stderr: -446> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient(hunting): picked mon.noname-a con 0x5609880d9f10 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:22:42.182 INFO:tasks.workunit.client.0.trial046.stderr: -445> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient(hunting): picked mon.noname-c con 0x56098817ebf0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:22:42.182 INFO:tasks.workunit.client.0.trial046.stderr: -444> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.182 INFO:tasks.workunit.client.0.trial046.stderr: -443> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.182 INFO:tasks.workunit.client.0.trial046.stderr: -442> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.182 INFO:tasks.workunit.client.0.trial046.stderr: -441> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient(hunting): _renew_subs 2026-02-24T22:22:42.182 INFO:tasks.workunit.client.0.trial046.stderr: -440> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient(hunting): authenticate will time out at 2131.992710s 2026-02-24T22:22:42.182 INFO:tasks.workunit.client.0.trial046.stderr: -439> 2026-02-24T22:22:41.886+0000 7f9c51b01640 10 monclient(hunting): get_auth_request con 0x56098817ebf0 auth_method 0 2026-02-24T22:22:42.182 INFO:tasks.workunit.client.0.trial046.stderr: -438> 2026-02-24T22:22:41.886+0000 7f9c51b01640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:42.182 INFO:tasks.workunit.client.0.trial046.stderr: -437> 2026-02-24T22:22:41.886+0000 7f9c51b01640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:42.182 INFO:tasks.workunit.client.0.trial046.stderr: -436> 2026-02-24T22:22:41.886+0000 7f9c51b01640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:42.182 INFO:tasks.workunit.client.0.trial046.stderr: -435> 2026-02-24T22:22:41.886+0000 7f9c50aff640 10 monclient(hunting): get_auth_request con 0x5609880d9f10 auth_method 0 2026-02-24T22:22:42.182 INFO:tasks.workunit.client.0.trial046.stderr: -434> 2026-02-24T22:22:41.886+0000 7f9c50aff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:42.182 INFO:tasks.workunit.client.0.trial046.stderr: -433> 2026-02-24T22:22:41.886+0000 7f9c50aff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:42.182 INFO:tasks.workunit.client.0.trial046.stderr: -432> 2026-02-24T22:22:41.886+0000 7f9c50aff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:42.183 INFO:tasks.workunit.client.0.trial046.stderr: -431> 2026-02-24T22:22:41.886+0000 7f9c51b01640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:42.183 INFO:tasks.workunit.client.0.trial046.stderr: -430> 2026-02-24T22:22:41.886+0000 7f9c51b01640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:42.183 INFO:tasks.workunit.client.0.trial046.stderr: -429> 2026-02-24T22:22:41.886+0000 7f9c51b01640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:42.183 INFO:tasks.workunit.client.0.trial046.stderr: -428> 2026-02-24T22:22:41.886+0000 7f9c50aff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:42.183 INFO:tasks.workunit.client.0.trial046.stderr: -427> 2026-02-24T22:22:41.886+0000 7f9c50aff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:42.183 INFO:tasks.workunit.client.0.trial046.stderr: -426> 2026-02-24T22:22:41.886+0000 7f9c50aff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:42.183 INFO:tasks.workunit.client.0.trial046.stderr: -425> 2026-02-24T22:22:41.886+0000 7f9c51300640 10 monclient(hunting): get_auth_request con 0x5609880d3c90 auth_method 0 2026-02-24T22:22:42.183 INFO:tasks.workunit.client.0.trial046.stderr: -424> 2026-02-24T22:22:41.886+0000 7f9c51300640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:42.183 INFO:tasks.workunit.client.0.trial046.stderr: -423> 2026-02-24T22:22:41.886+0000 7f9c51300640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:42.183 INFO:tasks.workunit.client.0.trial046.stderr: -422> 2026-02-24T22:22:41.886+0000 7f9c51300640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:42.183 INFO:tasks.workunit.client.0.trial046.stderr: -421> 2026-02-24T22:22:41.886+0000 7f9c51b01640 10 monclient(hunting): handle_auth_done global_id 8036 payload 824 2026-02-24T22:22:42.183 INFO:tasks.workunit.client.0.trial046.stderr: -420> 2026-02-24T22:22:41.886+0000 7f9c51b01640 10 monclient: _finish_hunting 0 2026-02-24T22:22:42.183 INFO:tasks.workunit.client.0.trial046.stderr: -419> 2026-02-24T22:22:41.886+0000 7f9c51b01640 1 monclient: found mon.noname-c 2026-02-24T22:22:42.183 INFO:tasks.workunit.client.0.trial046.stderr: -418> 2026-02-24T22:22:41.886+0000 7f9c51b01640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:22:42.183 INFO:tasks.workunit.client.0.trial046.stderr: -417> 2026-02-24T22:22:41.886+0000 7f9c35ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:42.183 INFO:tasks.workunit.client.0.trial046.stderr: -416> 2026-02-24T22:22:41.886+0000 7f9c35ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:22:42.183 INFO:tasks.workunit.client.0.trial046.stderr: -415> 2026-02-24T22:22:41.886+0000 7f9c35ffb640 10 monclient: dump: 2026-02-24T22:22:42.184 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:42.184 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:42.184 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.184 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.184 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:42.184 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:42.184 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:42.184 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:42.184 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:42.184 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.184 INFO:tasks.workunit.client.0.trial046.stderr: -414> 2026-02-24T22:22:41.886+0000 7f9c35ffb640 10 monclient: _finish_auth 0 2026-02-24T22:22:42.184 INFO:tasks.workunit.client.0.trial046.stderr: -413> 2026-02-24T22:22:41.886+0000 7f9c35ffb640 10 monclient: _check_auth_tickets 2026-02-24T22:22:42.184 INFO:tasks.workunit.client.0.trial046.stderr: -412> 2026-02-24T22:22:41.886+0000 7f9c35ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T22:22:42.184 INFO:tasks.workunit.client.0.trial046.stderr: -411> 2026-02-24T22:22:41.886+0000 7f9c35ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:42.184 INFO:tasks.workunit.client.0.trial046.stderr: -410> 2026-02-24T22:22:41.886+0000 7f9c35ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:22:42.184 INFO:tasks.workunit.client.0.trial046.stderr: -409> 2026-02-24T22:22:41.886+0000 7f9c35ffb640 10 monclient: dump: 2026-02-24T22:22:42.185 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:42.185 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:42.185 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.185 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.185 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:42.185 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:42.185 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:42.185 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:42.185 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:42.185 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.185 INFO:tasks.workunit.client.0.trial046.stderr: -408> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 monclient: authenticate success, global_id 8036 2026-02-24T22:22:42.185 INFO:tasks.workunit.client.0.trial046.stderr: -407> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient: _renew_subs 2026-02-24T22:22:42.185 INFO:tasks.workunit.client.0.trial046.stderr: -406> 2026-02-24T22:22:41.886+0000 7f9c377fe640 4 set_mon_vals no callback set 2026-02-24T22:22:42.185 INFO:tasks.workunit.client.0.trial046.stderr: -405> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:22:42.185 INFO:tasks.workunit.client.0.trial046.stderr: -404> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient: _renew_subs 2026-02-24T22:22:42.185 INFO:tasks.workunit.client.0.trial046.stderr: -403> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:22:42.186 INFO:tasks.workunit.client.0.trial046.stderr: -402> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 1 librados: init done 2026-02-24T22:22:42.186 INFO:tasks.workunit.client.0.trial046.stderr: -401> 2026-02-24T22:22:41.886+0000 7f9c35ffb640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:22:42.186 INFO:tasks.workunit.client.0.trial046.stderr: -400> 2026-02-24T22:22:41.886+0000 7f9c35ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:22:42.186 INFO:tasks.workunit.client.0.trial046.stderr: -399> 2026-02-24T22:22:41.886+0000 7f9c35ffb640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:22:42.186 INFO:tasks.workunit.client.0.trial046.stderr: -398> 2026-02-24T22:22:41.886+0000 7f9c51300640 10 monclient: get_auth_request con 0x7f9c300707b0 auth_method 0 2026-02-24T22:22:42.186 INFO:tasks.workunit.client.0.trial046.stderr: -397> 2026-02-24T22:22:41.886+0000 7f9c50aff640 10 monclient: get_auth_request con 0x5609881a9ef0 auth_method 0 2026-02-24T22:22:42.186 INFO:tasks.workunit.client.0.trial046.stderr: -396> 2026-02-24T22:22:41.890+0000 7f9c51300640 10 monclient: get_auth_request con 0x5609881d35f0 auth_method 0 2026-02-24T22:22:42.186 INFO:tasks.workunit.client.0.trial046.stderr: -395> 2026-02-24T22:22:41.890+0000 7f9c50aff640 10 monclient: get_auth_request con 0x5609881d4e10 auth_method 0 2026-02-24T22:22:42.186 INFO:tasks.workunit.client.0.trial046.stderr: -394> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient: build_initial_monmap 2026-02-24T22:22:42.186 INFO:tasks.workunit.client.0.trial046.stderr: -393> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 1 build_initial for_mkfs: 0 2026-02-24T22:22:42.186 INFO:tasks.workunit.client.0.trial046.stderr: -392> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient: monmap: 2026-02-24T22:22:42.186 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:22:42.186 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:22:42.186 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:22:42.186 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:22:42.186 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:22:42.187 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:42.187 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:22:42.187 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:22:42.187 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:22:42.187 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.187 INFO:tasks.workunit.client.0.trial046.stderr: -391> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding auth protocol: cephx 2026-02-24T22:22:42.187 INFO:tasks.workunit.client.0.trial046.stderr: -390> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding auth protocol: cephx 2026-02-24T22:22:42.187 INFO:tasks.workunit.client.0.trial046.stderr: -389> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding auth protocol: cephx 2026-02-24T22:22:42.187 INFO:tasks.workunit.client.0.trial046.stderr: -388> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding con mode: secure 2026-02-24T22:22:42.187 INFO:tasks.workunit.client.0.trial046.stderr: -387> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding con mode: crc 2026-02-24T22:22:42.187 INFO:tasks.workunit.client.0.trial046.stderr: -386> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding con mode: secure 2026-02-24T22:22:42.187 INFO:tasks.workunit.client.0.trial046.stderr: -385> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding con mode: crc 2026-02-24T22:22:42.187 INFO:tasks.workunit.client.0.trial046.stderr: -384> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding con mode: secure 2026-02-24T22:22:42.187 INFO:tasks.workunit.client.0.trial046.stderr: -383> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding con mode: crc 2026-02-24T22:22:42.187 INFO:tasks.workunit.client.0.trial046.stderr: -382> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding con mode: crc 2026-02-24T22:22:42.187 INFO:tasks.workunit.client.0.trial046.stderr: -381> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding con mode: secure 2026-02-24T22:22:42.187 INFO:tasks.workunit.client.0.trial046.stderr: -380> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding con mode: crc 2026-02-24T22:22:42.187 INFO:tasks.workunit.client.0.trial046.stderr: -379> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding con mode: secure 2026-02-24T22:22:42.188 INFO:tasks.workunit.client.0.trial046.stderr: -378> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding con mode: crc 2026-02-24T22:22:42.188 INFO:tasks.workunit.client.0.trial046.stderr: -377> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding con mode: secure 2026-02-24T22:22:42.188 INFO:tasks.workunit.client.0.trial046.stderr: -376> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:42.188 INFO:tasks.workunit.client.0.trial046.stderr: -375> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command objecter_requests cmddesc objecter_requests hook 0x5609881dc650 EEXIST 2026-02-24T22:22:42.188 INFO:tasks.workunit.client.0.trial046.stderr: -374> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient: init 2026-02-24T22:22:42.188 INFO:tasks.workunit.client.0.trial046.stderr: -373> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding auth protocol: cephx 2026-02-24T22:22:42.188 INFO:tasks.workunit.client.0.trial046.stderr: -372> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding auth protocol: cephx 2026-02-24T22:22:42.188 INFO:tasks.workunit.client.0.trial046.stderr: -371> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding auth protocol: cephx 2026-02-24T22:22:42.188 INFO:tasks.workunit.client.0.trial046.stderr: -370> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding con mode: secure 2026-02-24T22:22:42.188 INFO:tasks.workunit.client.0.trial046.stderr: -369> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding con mode: crc 2026-02-24T22:22:42.188 INFO:tasks.workunit.client.0.trial046.stderr: -368> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding con mode: secure 2026-02-24T22:22:42.188 INFO:tasks.workunit.client.0.trial046.stderr: -367> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding con mode: crc 2026-02-24T22:22:42.188 INFO:tasks.workunit.client.0.trial046.stderr: -366> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding con mode: secure 2026-02-24T22:22:42.188 INFO:tasks.workunit.client.0.trial046.stderr: -365> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding con mode: crc 2026-02-24T22:22:42.188 INFO:tasks.workunit.client.0.trial046.stderr: -364> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding con mode: crc 2026-02-24T22:22:42.188 INFO:tasks.workunit.client.0.trial046.stderr: -363> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding con mode: secure 2026-02-24T22:22:42.188 INFO:tasks.workunit.client.0.trial046.stderr: -362> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding con mode: crc 2026-02-24T22:22:42.188 INFO:tasks.workunit.client.0.trial046.stderr: -361> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding con mode: secure 2026-02-24T22:22:42.189 INFO:tasks.workunit.client.0.trial046.stderr: -360> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding con mode: crc 2026-02-24T22:22:42.189 INFO:tasks.workunit.client.0.trial046.stderr: -359> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding con mode: secure 2026-02-24T22:22:42.189 INFO:tasks.workunit.client.0.trial046.stderr: -358> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:42.189 INFO:tasks.workunit.client.0.trial046.stderr: -357> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:42.189 INFO:tasks.workunit.client.0.trial046.stderr: -356> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command rotate-key cmddesc rotate-key hook 0x5609881d5970 EEXIST 2026-02-24T22:22:42.189 INFO:tasks.workunit.client.0.trial046.stderr: -355> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient: _reopen_session rank -1 2026-02-24T22:22:42.189 INFO:tasks.workunit.client.0.trial046.stderr: -354> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T22:22:42.189 INFO:tasks.workunit.client.0.trial046.stderr: -353> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient(hunting): picked mon.noname-b con 0x560988330720 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:22:42.189 INFO:tasks.workunit.client.0.trial046.stderr: -352> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient(hunting): picked mon.noname-c con 0x56098832d7d0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:22:42.189 INFO:tasks.workunit.client.0.trial046.stderr: -351> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient(hunting): picked mon.noname-a con 0x56098832e3a0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:22:42.189 INFO:tasks.workunit.client.0.trial046.stderr: -350> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.189 INFO:tasks.workunit.client.0.trial046.stderr: -349> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.189 INFO:tasks.workunit.client.0.trial046.stderr: -348> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.189 INFO:tasks.workunit.client.0.trial046.stderr: -347> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient(hunting): _renew_subs 2026-02-24T22:22:42.189 INFO:tasks.workunit.client.0.trial046.stderr: -346> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient(hunting): authenticate will time out at 2131.999176s 2026-02-24T22:22:42.189 INFO:tasks.workunit.client.0.trial046.stderr: -345> 2026-02-24T22:22:41.890+0000 7f9c50aff640 10 monclient(hunting): get_auth_request con 0x56098832d7d0 auth_method 0 2026-02-24T22:22:42.189 INFO:tasks.workunit.client.0.trial046.stderr: -344> 2026-02-24T22:22:41.890+0000 7f9c50aff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:42.189 INFO:tasks.workunit.client.0.trial046.stderr: -343> 2026-02-24T22:22:41.890+0000 7f9c50aff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:42.189 INFO:tasks.workunit.client.0.trial046.stderr: -342> 2026-02-24T22:22:41.890+0000 7f9c50aff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:42.190 INFO:tasks.workunit.client.0.trial046.stderr: -341> 2026-02-24T22:22:41.890+0000 7f9c51b01640 10 monclient(hunting): get_auth_request con 0x56098832e3a0 auth_method 0 2026-02-24T22:22:42.190 INFO:tasks.workunit.client.0.trial046.stderr: -340> 2026-02-24T22:22:41.890+0000 7f9c51b01640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:42.190 INFO:tasks.workunit.client.0.trial046.stderr: -339> 2026-02-24T22:22:41.890+0000 7f9c51b01640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:42.190 INFO:tasks.workunit.client.0.trial046.stderr: -338> 2026-02-24T22:22:41.890+0000 7f9c51b01640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:42.190 INFO:tasks.workunit.client.0.trial046.stderr: -337> 2026-02-24T22:22:41.890+0000 7f9c50aff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:42.190 INFO:tasks.workunit.client.0.trial046.stderr: -336> 2026-02-24T22:22:41.890+0000 7f9c50aff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:42.190 INFO:tasks.workunit.client.0.trial046.stderr: -335> 2026-02-24T22:22:41.890+0000 7f9c50aff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:42.190 INFO:tasks.workunit.client.0.trial046.stderr: -334> 2026-02-24T22:22:41.890+0000 7f9c51b01640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:42.190 INFO:tasks.workunit.client.0.trial046.stderr: -333> 2026-02-24T22:22:41.890+0000 7f9c51b01640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:42.190 INFO:tasks.workunit.client.0.trial046.stderr: -332> 2026-02-24T22:22:41.890+0000 7f9c51b01640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:42.190 INFO:tasks.workunit.client.0.trial046.stderr: -331> 2026-02-24T22:22:41.890+0000 7f9c51300640 10 monclient(hunting): get_auth_request con 0x560988330720 auth_method 0 2026-02-24T22:22:42.190 INFO:tasks.workunit.client.0.trial046.stderr: -330> 2026-02-24T22:22:41.890+0000 7f9c51300640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:42.190 INFO:tasks.workunit.client.0.trial046.stderr: -329> 2026-02-24T22:22:41.890+0000 7f9c51300640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:42.190 INFO:tasks.workunit.client.0.trial046.stderr: -328> 2026-02-24T22:22:41.890+0000 7f9c51300640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:42.190 INFO:tasks.workunit.client.0.trial046.stderr: -327> 2026-02-24T22:22:41.890+0000 7f9c51b01640 10 monclient(hunting): handle_auth_done global_id 8619 payload 824 2026-02-24T22:22:42.190 INFO:tasks.workunit.client.0.trial046.stderr: -326> 2026-02-24T22:22:41.890+0000 7f9c51b01640 10 monclient: _finish_hunting 0 2026-02-24T22:22:42.190 INFO:tasks.workunit.client.0.trial046.stderr: -325> 2026-02-24T22:22:41.890+0000 7f9c51b01640 1 monclient: found mon.noname-a 2026-02-24T22:22:42.190 INFO:tasks.workunit.client.0.trial046.stderr: -324> 2026-02-24T22:22:41.890+0000 7f9c51b01640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:22:42.190 INFO:tasks.workunit.client.0.trial046.stderr: -323> 2026-02-24T22:22:41.890+0000 7f9c2effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:42.190 INFO:tasks.workunit.client.0.trial046.stderr: -322> 2026-02-24T22:22:41.890+0000 7f9c2effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:22:42.191 INFO:tasks.workunit.client.0.trial046.stderr: -321> 2026-02-24T22:22:41.890+0000 7f9c2effd640 10 monclient: dump: 2026-02-24T22:22:42.191 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:42.191 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:42.191 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.191 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.191 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:42.191 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:42.191 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:42.191 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:42.191 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:42.191 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.191 INFO:tasks.workunit.client.0.trial046.stderr: -320> 2026-02-24T22:22:41.890+0000 7f9c2effd640 10 monclient: _finish_auth 0 2026-02-24T22:22:42.191 INFO:tasks.workunit.client.0.trial046.stderr: -319> 2026-02-24T22:22:41.890+0000 7f9c2effd640 10 monclient: _check_auth_tickets 2026-02-24T22:22:42.191 INFO:tasks.workunit.client.0.trial046.stderr: -318> 2026-02-24T22:22:41.890+0000 7f9c2effd640 10 monclient: handle_config config(0 keys) 2026-02-24T22:22:42.191 INFO:tasks.workunit.client.0.trial046.stderr: -317> 2026-02-24T22:22:41.890+0000 7f9c2effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:42.191 INFO:tasks.workunit.client.0.trial046.stderr: -316> 2026-02-24T22:22:41.890+0000 7f9c2effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:22:42.191 INFO:tasks.workunit.client.0.trial046.stderr: -315> 2026-02-24T22:22:41.890+0000 7f9c2effd640 10 monclient: dump: 2026-02-24T22:22:42.191 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:42.191 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:42.192 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.192 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.192 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:42.192 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:42.192 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:42.192 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:42.192 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:42.192 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.192 INFO:tasks.workunit.client.0.trial046.stderr: -314> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 monclient: authenticate success, global_id 8619 2026-02-24T22:22:42.192 INFO:tasks.workunit.client.0.trial046.stderr: -313> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient: _renew_subs 2026-02-24T22:22:42.192 INFO:tasks.workunit.client.0.trial046.stderr: -312> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:22:42.192 INFO:tasks.workunit.client.0.trial046.stderr: -311> 2026-02-24T22:22:41.890+0000 7f9c52302640 4 set_mon_vals no callback set 2026-02-24T22:22:42.192 INFO:tasks.workunit.client.0.trial046.stderr: -310> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient: _renew_subs 2026-02-24T22:22:42.192 INFO:tasks.workunit.client.0.trial046.stderr: -309> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:22:42.192 INFO:tasks.workunit.client.0.trial046.stderr: -308> 2026-02-24T22:22:41.894+0000 7f9c2effd640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:22:42.192 INFO:tasks.workunit.client.0.trial046.stderr: -307> 2026-02-24T22:22:41.894+0000 7f9c2effd640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:22:42.192 INFO:tasks.workunit.client.0.trial046.stderr: -306> 2026-02-24T22:22:41.894+0000 7f9c2effd640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:22:42.192 INFO:tasks.workunit.client.0.trial046.stderr: -305> 2026-02-24T22:22:41.894+0000 7f9c51300640 10 monclient: get_auth_request con 0x7f9c240706c0 auth_method 0 2026-02-24T22:22:42.192 INFO:tasks.workunit.client.0.trial046.stderr: -304> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient: get_monmap_and_config 2026-02-24T22:22:42.193 INFO:tasks.workunit.client.0.trial046.stderr: -303> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient: build_initial_monmap 2026-02-24T22:22:42.193 INFO:tasks.workunit.client.0.trial046.stderr: -302> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 1 build_initial for_mkfs: 0 2026-02-24T22:22:42.193 INFO:tasks.workunit.client.0.trial046.stderr: -301> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient: monmap: 2026-02-24T22:22:42.193 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:22:42.193 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:22:42.193 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:22:42.193 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:22:42.193 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:22:42.193 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:42.193 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:22:42.193 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:22:42.193 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:22:42.193 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.193 INFO:tasks.workunit.client.0.trial046.stderr: -300> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding auth protocol: cephx 2026-02-24T22:22:42.193 INFO:tasks.workunit.client.0.trial046.stderr: -299> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding auth protocol: cephx 2026-02-24T22:22:42.193 INFO:tasks.workunit.client.0.trial046.stderr: -298> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding auth protocol: cephx 2026-02-24T22:22:42.193 INFO:tasks.workunit.client.0.trial046.stderr: -297> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: secure 2026-02-24T22:22:42.193 INFO:tasks.workunit.client.0.trial046.stderr: -296> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: crc 2026-02-24T22:22:42.193 INFO:tasks.workunit.client.0.trial046.stderr: -295> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: secure 2026-02-24T22:22:42.194 INFO:tasks.workunit.client.0.trial046.stderr: -294> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: crc 2026-02-24T22:22:42.194 INFO:tasks.workunit.client.0.trial046.stderr: -293> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: secure 2026-02-24T22:22:42.194 INFO:tasks.workunit.client.0.trial046.stderr: -292> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: crc 2026-02-24T22:22:42.194 INFO:tasks.workunit.client.0.trial046.stderr: -291> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: crc 2026-02-24T22:22:42.194 INFO:tasks.workunit.client.0.trial046.stderr: -290> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: secure 2026-02-24T22:22:42.194 INFO:tasks.workunit.client.0.trial046.stderr: -289> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: crc 2026-02-24T22:22:42.194 INFO:tasks.workunit.client.0.trial046.stderr: -288> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: secure 2026-02-24T22:22:42.194 INFO:tasks.workunit.client.0.trial046.stderr: -287> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: crc 2026-02-24T22:22:42.194 INFO:tasks.workunit.client.0.trial046.stderr: -286> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: secure 2026-02-24T22:22:42.194 INFO:tasks.workunit.client.0.trial046.stderr: -285> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:42.194 INFO:tasks.workunit.client.0.trial046.stderr: -284> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient: init 2026-02-24T22:22:42.194 INFO:tasks.workunit.client.0.trial046.stderr: -283> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding auth protocol: cephx 2026-02-24T22:22:42.194 INFO:tasks.workunit.client.0.trial046.stderr: -282> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding auth protocol: cephx 2026-02-24T22:22:42.194 INFO:tasks.workunit.client.0.trial046.stderr: -281> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding auth protocol: cephx 2026-02-24T22:22:42.194 INFO:tasks.workunit.client.0.trial046.stderr: -280> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding con mode: secure 2026-02-24T22:22:42.194 INFO:tasks.workunit.client.0.trial046.stderr: -279> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding con mode: crc 2026-02-24T22:22:42.194 INFO:tasks.workunit.client.0.trial046.stderr: -278> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding con mode: secure 2026-02-24T22:22:42.194 INFO:tasks.workunit.client.0.trial046.stderr: -277> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding con mode: crc 2026-02-24T22:22:42.194 INFO:tasks.workunit.client.0.trial046.stderr: -276> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding con mode: secure 2026-02-24T22:22:42.194 INFO:tasks.workunit.client.0.trial046.stderr: -275> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding con mode: crc 2026-02-24T22:22:42.194 INFO:tasks.workunit.client.0.trial046.stderr: -274> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding con mode: crc 2026-02-24T22:22:42.195 INFO:tasks.workunit.client.0.trial046.stderr: -273> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding con mode: secure 2026-02-24T22:22:42.195 INFO:tasks.workunit.client.0.trial046.stderr: -272> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding con mode: crc 2026-02-24T22:22:42.195 INFO:tasks.workunit.client.0.trial046.stderr: -271> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding con mode: secure 2026-02-24T22:22:42.195 INFO:tasks.workunit.client.0.trial046.stderr: -270> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding con mode: crc 2026-02-24T22:22:42.195 INFO:tasks.workunit.client.0.trial046.stderr: -269> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding con mode: secure 2026-02-24T22:22:42.195 INFO:tasks.workunit.client.0.trial046.stderr: -268> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:42.195 INFO:tasks.workunit.client.0.trial046.stderr: -267> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:42.195 INFO:tasks.workunit.client.0.trial046.stderr: -266> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command rotate-key cmddesc rotate-key hook 0x7fff0e4f72b8 EEXIST 2026-02-24T22:22:42.195 INFO:tasks.workunit.client.0.trial046.stderr: -265> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient: _reopen_session rank -1 2026-02-24T22:22:42.195 INFO:tasks.workunit.client.0.trial046.stderr: -264> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:22:42.195 INFO:tasks.workunit.client.0.trial046.stderr: -263> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient(hunting): picked mon.noname-c con 0x5609883e8110 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:22:42.195 INFO:tasks.workunit.client.0.trial046.stderr: -262> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient(hunting): picked mon.noname-a con 0x5609883eb880 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:22:42.195 INFO:tasks.workunit.client.0.trial046.stderr: -261> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient(hunting): picked mon.noname-b con 0x5609883c6640 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:22:42.195 INFO:tasks.workunit.client.0.trial046.stderr: -260> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.195 INFO:tasks.workunit.client.0.trial046.stderr: -259> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.195 INFO:tasks.workunit.client.0.trial046.stderr: -258> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.195 INFO:tasks.workunit.client.0.trial046.stderr: -257> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient(hunting): _renew_subs 2026-02-24T22:22:42.195 INFO:tasks.workunit.client.0.trial046.stderr: -256> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient(hunting): authenticate will time out at 2132.002373s 2026-02-24T22:22:42.195 INFO:tasks.workunit.client.0.trial046.stderr: -255> 2026-02-24T22:22:41.894+0000 7f9c51300640 10 monclient(hunting): get_auth_request con 0x5609883e8110 auth_method 0 2026-02-24T22:22:42.195 INFO:tasks.workunit.client.0.trial046.stderr: -254> 2026-02-24T22:22:41.894+0000 7f9c51300640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:42.195 INFO:tasks.workunit.client.0.trial046.stderr: -253> 2026-02-24T22:22:41.894+0000 7f9c51300640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:42.196 INFO:tasks.workunit.client.0.trial046.stderr: -252> 2026-02-24T22:22:41.894+0000 7f9c51300640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:42.196 INFO:tasks.workunit.client.0.trial046.stderr: -251> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): get_auth_request con 0x5609883eb880 auth_method 0 2026-02-24T22:22:42.196 INFO:tasks.workunit.client.0.trial046.stderr: -250> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:42.196 INFO:tasks.workunit.client.0.trial046.stderr: -249> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:42.196 INFO:tasks.workunit.client.0.trial046.stderr: -248> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:42.196 INFO:tasks.workunit.client.0.trial046.stderr: -247> 2026-02-24T22:22:41.894+0000 7f9c51300640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:42.196 INFO:tasks.workunit.client.0.trial046.stderr: -246> 2026-02-24T22:22:41.894+0000 7f9c51300640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:42.196 INFO:tasks.workunit.client.0.trial046.stderr: -245> 2026-02-24T22:22:41.894+0000 7f9c51300640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:42.196 INFO:tasks.workunit.client.0.trial046.stderr: -244> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:42.196 INFO:tasks.workunit.client.0.trial046.stderr: -243> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:42.196 INFO:tasks.workunit.client.0.trial046.stderr: -242> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:42.196 INFO:tasks.workunit.client.0.trial046.stderr: -241> 2026-02-24T22:22:41.894+0000 7f9c51300640 10 monclient(hunting): handle_auth_done global_id 8042 payload 274 2026-02-24T22:22:42.196 INFO:tasks.workunit.client.0.trial046.stderr: -240> 2026-02-24T22:22:41.894+0000 7f9c51300640 10 monclient: _finish_hunting 0 2026-02-24T22:22:42.196 INFO:tasks.workunit.client.0.trial046.stderr: -239> 2026-02-24T22:22:41.894+0000 7f9c51300640 1 monclient: found mon.noname-c 2026-02-24T22:22:42.196 INFO:tasks.workunit.client.0.trial046.stderr: -238> 2026-02-24T22:22:41.894+0000 7f9c51300640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:22:42.196 INFO:tasks.workunit.client.0.trial046.stderr: -237> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:42.196 INFO:tasks.workunit.client.0.trial046.stderr: -236> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:22:42.196 INFO:tasks.workunit.client.0.trial046.stderr: -235> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient: dump: 2026-02-24T22:22:42.196 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:42.196 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:42.196 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.196 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.197 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:42.197 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:42.197 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:42.197 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:42.197 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:42.197 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.197 INFO:tasks.workunit.client.0.trial046.stderr: -234> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient: _finish_auth 0 2026-02-24T22:22:42.197 INFO:tasks.workunit.client.0.trial046.stderr: -233> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient: _check_auth_tickets 2026-02-24T22:22:42.197 INFO:tasks.workunit.client.0.trial046.stderr: -232> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient: ms_handle_reset current mon [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:22:42.197 INFO:tasks.workunit.client.0.trial046.stderr: -231> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient: _reopen_session rank -1 2026-02-24T22:22:42.197 INFO:tasks.workunit.client.0.trial046.stderr: -230> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T22:22:42.197 INFO:tasks.workunit.client.0.trial046.stderr: -229> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient(hunting): picked mon.b con 0x5609883c6640 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:22:42.197 INFO:tasks.workunit.client.0.trial046.stderr: -228> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient(hunting): picked mon.a con 0x5609883eb880 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:22:42.197 INFO:tasks.workunit.client.0.trial046.stderr: -227> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient(hunting): picked mon.c con 0x7f9c14042f30 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:22:42.197 INFO:tasks.workunit.client.0.trial046.stderr: -226> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.197 INFO:tasks.workunit.client.0.trial046.stderr: -225> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.197 INFO:tasks.workunit.client.0.trial046.stderr: -224> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.197 INFO:tasks.workunit.client.0.trial046.stderr: -223> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient(hunting): _renew_subs 2026-02-24T22:22:42.197 INFO:tasks.workunit.client.0.trial046.stderr: -222> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient(hunting): ms_handle_reset hunted mon [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:22:42.197 INFO:tasks.workunit.client.0.trial046.stderr: -221> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): get_auth_request con 0x5609883eb880 auth_method 0 2026-02-24T22:22:42.198 INFO:tasks.workunit.client.0.trial046.stderr: -220> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:42.198 INFO:tasks.workunit.client.0.trial046.stderr: -219> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:42.198 INFO:tasks.workunit.client.0.trial046.stderr: -218> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T22:22:42.198 INFO:tasks.workunit.client.0.trial046.stderr: -217> 2026-02-24T22:22:41.894+0000 7f9c51b01640 10 monclient(hunting): get_auth_request con 0x7f9c14042f30 auth_method 0 2026-02-24T22:22:42.198 INFO:tasks.workunit.client.0.trial046.stderr: -216> 2026-02-24T22:22:41.894+0000 7f9c51b01640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:42.198 INFO:tasks.workunit.client.0.trial046.stderr: -215> 2026-02-24T22:22:41.894+0000 7f9c51b01640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:42.198 INFO:tasks.workunit.client.0.trial046.stderr: -214> 2026-02-24T22:22:41.894+0000 7f9c51b01640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T22:22:42.198 INFO:tasks.workunit.client.0.trial046.stderr: -213> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:42.198 INFO:tasks.workunit.client.0.trial046.stderr: -212> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:42.198 INFO:tasks.workunit.client.0.trial046.stderr: -211> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T22:22:42.198 INFO:tasks.workunit.client.0.trial046.stderr: -210> 2026-02-24T22:22:41.894+0000 7f9c51b01640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:42.198 INFO:tasks.workunit.client.0.trial046.stderr: -209> 2026-02-24T22:22:41.894+0000 7f9c51b01640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:42.198 INFO:tasks.workunit.client.0.trial046.stderr: -208> 2026-02-24T22:22:41.894+0000 7f9c51b01640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T22:22:42.198 INFO:tasks.workunit.client.0.trial046.stderr: -207> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): handle_auth_done global_id 8042 payload 293 2026-02-24T22:22:42.198 INFO:tasks.workunit.client.0.trial046.stderr: -206> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient: _finish_hunting 0 2026-02-24T22:22:42.198 INFO:tasks.workunit.client.0.trial046.stderr: -205> 2026-02-24T22:22:41.894+0000 7f9c50aff640 1 monclient: found mon.a 2026-02-24T22:22:42.198 INFO:tasks.workunit.client.0.trial046.stderr: -204> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:22:42.198 INFO:tasks.workunit.client.0.trial046.stderr: -203> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient: _finish_auth 0 2026-02-24T22:22:42.198 INFO:tasks.workunit.client.0.trial046.stderr: -202> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient: _check_auth_tickets 2026-02-24T22:22:42.198 INFO:tasks.workunit.client.0.trial046.stderr: -201> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 monclient: authenticate success, global_id 8042 2026-02-24T22:22:42.198 INFO:tasks.workunit.client.0.trial046.stderr: -200> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:42.198 INFO:tasks.workunit.client.0.trial046.stderr: -199> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:22:42.198 INFO:tasks.workunit.client.0.trial046.stderr: -198> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient: dump: 2026-02-24T22:22:42.199 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:42.199 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:42.199 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.199 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.199 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:42.199 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:42.199 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:42.199 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:42.199 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:42.199 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.199 INFO:tasks.workunit.client.0.trial046.stderr: -197> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient: handle_config config(0 keys) 2026-02-24T22:22:42.199 INFO:tasks.workunit.client.0.trial046.stderr: -196> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient: get_monmap_and_config success 2026-02-24T22:22:42.199 INFO:tasks.workunit.client.0.trial046.stderr: -195> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 4 set_mon_vals no callback set 2026-02-24T22:22:42.199 INFO:tasks.workunit.client.0.trial046.stderr: -194> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient: shutdown 2026-02-24T22:22:42.199 INFO:tasks.workunit.client.0.trial046.stderr: -193> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient: build_initial_monmap 2026-02-24T22:22:42.199 INFO:tasks.workunit.client.0.trial046.stderr: -192> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 1 build_initial for_mkfs: 0 2026-02-24T22:22:42.199 INFO:tasks.workunit.client.0.trial046.stderr: -191> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient: monmap: 2026-02-24T22:22:42.199 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:22:42.199 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:22:42.199 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:22:42.199 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:22:42.199 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:22:42.200 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:42.200 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:22:42.200 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:22:42.200 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:22:42.200 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.200 INFO:tasks.workunit.client.0.trial046.stderr: -190> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding auth protocol: cephx 2026-02-24T22:22:42.200 INFO:tasks.workunit.client.0.trial046.stderr: -189> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding auth protocol: cephx 2026-02-24T22:22:42.200 INFO:tasks.workunit.client.0.trial046.stderr: -188> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding auth protocol: cephx 2026-02-24T22:22:42.200 INFO:tasks.workunit.client.0.trial046.stderr: -187> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: secure 2026-02-24T22:22:42.200 INFO:tasks.workunit.client.0.trial046.stderr: -186> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: crc 2026-02-24T22:22:42.200 INFO:tasks.workunit.client.0.trial046.stderr: -185> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: secure 2026-02-24T22:22:42.200 INFO:tasks.workunit.client.0.trial046.stderr: -184> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: crc 2026-02-24T22:22:42.200 INFO:tasks.workunit.client.0.trial046.stderr: -183> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: secure 2026-02-24T22:22:42.200 INFO:tasks.workunit.client.0.trial046.stderr: -182> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: crc 2026-02-24T22:22:42.200 INFO:tasks.workunit.client.0.trial046.stderr: -181> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: crc 2026-02-24T22:22:42.200 INFO:tasks.workunit.client.0.trial046.stderr: -180> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: secure 2026-02-24T22:22:42.200 INFO:tasks.workunit.client.0.trial046.stderr: -179> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: crc 2026-02-24T22:22:42.200 INFO:tasks.workunit.client.0.trial046.stderr: -178> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: secure 2026-02-24T22:22:42.200 INFO:tasks.workunit.client.0.trial046.stderr: -177> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: crc 2026-02-24T22:22:42.200 INFO:tasks.workunit.client.0.trial046.stderr: -176> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: secure 2026-02-24T22:22:42.200 INFO:tasks.workunit.client.0.trial046.stderr: -175> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:42.200 INFO:tasks.workunit.client.0.trial046.stderr: -174> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 1 librados: starting msgr at 2026-02-24T22:22:42.200 INFO:tasks.workunit.client.0.trial046.stderr: -173> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 1 librados: starting objecter 2026-02-24T22:22:42.200 INFO:tasks.workunit.client.0.trial046.stderr: -172> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command objecter_requests cmddesc objecter_requests hook 0x5609883c6ca0 EEXIST 2026-02-24T22:22:42.200 INFO:tasks.workunit.client.0.trial046.stderr: -171> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 1 librados: setting wanted keys 2026-02-24T22:22:42.201 INFO:tasks.workunit.client.0.trial046.stderr: -170> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 1 librados: calling monclient init 2026-02-24T22:22:42.201 INFO:tasks.workunit.client.0.trial046.stderr: -169> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient: init 2026-02-24T22:22:42.201 INFO:tasks.workunit.client.0.trial046.stderr: -168> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding auth protocol: cephx 2026-02-24T22:22:42.201 INFO:tasks.workunit.client.0.trial046.stderr: -167> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding auth protocol: cephx 2026-02-24T22:22:42.201 INFO:tasks.workunit.client.0.trial046.stderr: -166> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding auth protocol: cephx 2026-02-24T22:22:42.201 INFO:tasks.workunit.client.0.trial046.stderr: -165> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding con mode: secure 2026-02-24T22:22:42.201 INFO:tasks.workunit.client.0.trial046.stderr: -164> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding con mode: crc 2026-02-24T22:22:42.201 INFO:tasks.workunit.client.0.trial046.stderr: -163> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding con mode: secure 2026-02-24T22:22:42.201 INFO:tasks.workunit.client.0.trial046.stderr: -162> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding con mode: crc 2026-02-24T22:22:42.201 INFO:tasks.workunit.client.0.trial046.stderr: -161> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding con mode: secure 2026-02-24T22:22:42.201 INFO:tasks.workunit.client.0.trial046.stderr: -160> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding con mode: crc 2026-02-24T22:22:42.201 INFO:tasks.workunit.client.0.trial046.stderr: -159> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding con mode: crc 2026-02-24T22:22:42.201 INFO:tasks.workunit.client.0.trial046.stderr: -158> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding con mode: secure 2026-02-24T22:22:42.201 INFO:tasks.workunit.client.0.trial046.stderr: -157> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding con mode: crc 2026-02-24T22:22:42.201 INFO:tasks.workunit.client.0.trial046.stderr: -156> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding con mode: secure 2026-02-24T22:22:42.201 INFO:tasks.workunit.client.0.trial046.stderr: -155> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding con mode: crc 2026-02-24T22:22:42.201 INFO:tasks.workunit.client.0.trial046.stderr: -154> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding con mode: secure 2026-02-24T22:22:42.201 INFO:tasks.workunit.client.0.trial046.stderr: -153> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:42.201 INFO:tasks.workunit.client.0.trial046.stderr: -152> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:42.201 INFO:tasks.workunit.client.0.trial046.stderr: -151> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command rotate-key cmddesc rotate-key hook 0x5609883c0548 EEXIST 2026-02-24T22:22:42.201 INFO:tasks.workunit.client.0.trial046.stderr: -150> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient: _reopen_session rank -1 2026-02-24T22:22:42.201 INFO:tasks.workunit.client.0.trial046.stderr: -149> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T22:22:42.201 INFO:tasks.workunit.client.0.trial046.stderr: -148> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient(hunting): picked mon.noname-c con 0x5609883eb880 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:22:42.201 INFO:tasks.workunit.client.0.trial046.stderr: -147> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient(hunting): picked mon.noname-b con 0x7f9c14042f30 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:22:42.202 INFO:tasks.workunit.client.0.trial046.stderr: -146> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient(hunting): picked mon.noname-a con 0x560988411900 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:22:42.202 INFO:tasks.workunit.client.0.trial046.stderr: -145> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.202 INFO:tasks.workunit.client.0.trial046.stderr: -144> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.202 INFO:tasks.workunit.client.0.trial046.stderr: -143> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.202 INFO:tasks.workunit.client.0.trial046.stderr: -142> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient(hunting): _renew_subs 2026-02-24T22:22:42.202 INFO:tasks.workunit.client.0.trial046.stderr: -141> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient(hunting): authenticate will time out at 2132.004178s 2026-02-24T22:22:42.202 INFO:tasks.workunit.client.0.trial046.stderr: -140> 2026-02-24T22:22:41.898+0000 7f9c51b01640 10 monclient(hunting): get_auth_request con 0x560988411900 auth_method 0 2026-02-24T22:22:42.202 INFO:tasks.workunit.client.0.trial046.stderr: -139> 2026-02-24T22:22:41.898+0000 7f9c51b01640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:42.202 INFO:tasks.workunit.client.0.trial046.stderr: -138> 2026-02-24T22:22:41.898+0000 7f9c51b01640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:42.202 INFO:tasks.workunit.client.0.trial046.stderr: -137> 2026-02-24T22:22:41.898+0000 7f9c51b01640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:42.202 INFO:tasks.workunit.client.0.trial046.stderr: -136> 2026-02-24T22:22:41.898+0000 7f9c51300640 10 monclient(hunting): get_auth_request con 0x5609883eb880 auth_method 0 2026-02-24T22:22:42.202 INFO:tasks.workunit.client.0.trial046.stderr: -135> 2026-02-24T22:22:41.898+0000 7f9c51300640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:42.202 INFO:tasks.workunit.client.0.trial046.stderr: -134> 2026-02-24T22:22:41.898+0000 7f9c51300640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:42.202 INFO:tasks.workunit.client.0.trial046.stderr: -133> 2026-02-24T22:22:41.898+0000 7f9c51300640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:42.202 INFO:tasks.workunit.client.0.trial046.stderr: -132> 2026-02-24T22:22:41.898+0000 7f9c51b01640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:42.202 INFO:tasks.workunit.client.0.trial046.stderr: -131> 2026-02-24T22:22:41.898+0000 7f9c51b01640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:42.202 INFO:tasks.workunit.client.0.trial046.stderr: -130> 2026-02-24T22:22:41.898+0000 7f9c51b01640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:42.202 INFO:tasks.workunit.client.0.trial046.stderr: -129> 2026-02-24T22:22:41.898+0000 7f9c51300640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:42.202 INFO:tasks.workunit.client.0.trial046.stderr: -128> 2026-02-24T22:22:41.898+0000 7f9c51300640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:42.202 INFO:tasks.workunit.client.0.trial046.stderr: -127> 2026-02-24T22:22:41.898+0000 7f9c51300640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:42.202 INFO:tasks.workunit.client.0.trial046.stderr: -126> 2026-02-24T22:22:41.898+0000 7f9c51b01640 10 monclient(hunting): handle_auth_done global_id 8625 payload 824 2026-02-24T22:22:42.202 INFO:tasks.workunit.client.0.trial046.stderr: -125> 2026-02-24T22:22:41.898+0000 7f9c51b01640 10 monclient: _finish_hunting 0 2026-02-24T22:22:42.202 INFO:tasks.workunit.client.0.trial046.stderr: -124> 2026-02-24T22:22:41.898+0000 7f9c51b01640 1 monclient: found mon.noname-a 2026-02-24T22:22:42.202 INFO:tasks.workunit.client.0.trial046.stderr: -123> 2026-02-24T22:22:41.898+0000 7f9c51b01640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:22:42.202 INFO:tasks.workunit.client.0.trial046.stderr: -122> 2026-02-24T22:22:41.898+0000 7f9c1affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:42.203 INFO:tasks.workunit.client.0.trial046.stderr: -121> 2026-02-24T22:22:41.898+0000 7f9c1affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:22:42.203 INFO:tasks.workunit.client.0.trial046.stderr: -120> 2026-02-24T22:22:41.898+0000 7f9c1affd640 10 monclient: dump: 2026-02-24T22:22:42.203 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:42.203 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:42.203 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.203 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.203 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:42.203 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:42.203 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:42.203 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:42.203 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:42.203 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.203 INFO:tasks.workunit.client.0.trial046.stderr: -119> 2026-02-24T22:22:41.898+0000 7f9c1affd640 10 monclient: _finish_auth 0 2026-02-24T22:22:42.203 INFO:tasks.workunit.client.0.trial046.stderr: -118> 2026-02-24T22:22:41.898+0000 7f9c1affd640 10 monclient: _check_auth_tickets 2026-02-24T22:22:42.203 INFO:tasks.workunit.client.0.trial046.stderr: -117> 2026-02-24T22:22:41.898+0000 7f9c1affd640 10 monclient: handle_config config(0 keys) 2026-02-24T22:22:42.203 INFO:tasks.workunit.client.0.trial046.stderr: -116> 2026-02-24T22:22:41.898+0000 7f9c1affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:42.203 INFO:tasks.workunit.client.0.trial046.stderr: -115> 2026-02-24T22:22:41.898+0000 7f9c1affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:22:42.203 INFO:tasks.workunit.client.0.trial046.stderr: -114> 2026-02-24T22:22:41.898+0000 7f9c1affd640 10 monclient: dump: 2026-02-24T22:22:42.203 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:42.203 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:42.203 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.203 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.203 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:42.203 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:42.203 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:42.204 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:42.204 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:42.204 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.204 INFO:tasks.workunit.client.0.trial046.stderr: -113> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 5 monclient: authenticate success, global_id 8625 2026-02-24T22:22:42.204 INFO:tasks.workunit.client.0.trial046.stderr: -112> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient: _renew_subs 2026-02-24T22:22:42.204 INFO:tasks.workunit.client.0.trial046.stderr: -111> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:22:42.204 INFO:tasks.workunit.client.0.trial046.stderr: -110> 2026-02-24T22:22:41.898+0000 7f9c2cff9640 4 set_mon_vals no callback set 2026-02-24T22:22:42.204 INFO:tasks.workunit.client.0.trial046.stderr: -109> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient: _renew_subs 2026-02-24T22:22:42.204 INFO:tasks.workunit.client.0.trial046.stderr: -108> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:22:42.204 INFO:tasks.workunit.client.0.trial046.stderr: -107> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 1 librados: init done 2026-02-24T22:22:42.204 INFO:tasks.workunit.client.0.trial046.stderr: -106> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command cr dump hook 0x560988416198 2026-02-24T22:22:42.204 INFO:tasks.workunit.client.0.trial046.stderr: -105> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 4 Realm: () 2026-02-24T22:22:42.204 INFO:tasks.workunit.client.0.trial046.stderr: -104> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:22:42.204 INFO:tasks.workunit.client.0.trial046.stderr: -103> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:22:42.204 INFO:tasks.workunit.client.0.trial046.stderr: -102> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:22:42.204 INFO:tasks.workunit.client.0.trial046.stderr: -101> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 20 zonegroup default 2026-02-24T22:22:42.204 INFO:tasks.workunit.client.0.trial046.stderr: -100> 2026-02-24T22:22:41.898+0000 7f9c51300640 10 monclient: get_auth_request con 0x560988488ef0 auth_method 0 2026-02-24T22:22:42.204 INFO:tasks.workunit.client.0.trial046.stderr: -99> 2026-02-24T22:22:41.898+0000 7f9c1affd640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:22:42.204 INFO:tasks.workunit.client.0.trial046.stderr: -98> 2026-02-24T22:22:41.898+0000 7f9c1affd640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:22:42.204 INFO:tasks.workunit.client.0.trial046.stderr: -97> 2026-02-24T22:22:41.898+0000 7f9c1affd640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:22:42.204 INFO:tasks.workunit.client.0.trial046.stderr: -96> 2026-02-24T22:22:41.898+0000 7f9c50aff640 10 monclient: get_auth_request con 0x7f9c14071960 auth_method 0 2026-02-24T22:22:42.204 INFO:tasks.workunit.client.0.trial046.stderr: -95> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 20 rados->read ofs=0 len=0 2026-02-24T22:22:42.204 INFO:tasks.workunit.client.0.trial046.stderr: -94> 2026-02-24T22:22:41.898+0000 7f9c51300640 10 monclient: get_auth_request con 0x5609884ac2a0 auth_method 0 2026-02-24T22:22:42.204 INFO:tasks.workunit.client.0.trial046.stderr: -93> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:22:42.205 INFO:tasks.workunit.client.0.trial046.stderr: -92> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 20 rados->read ofs=0 len=0 2026-02-24T22:22:42.205 INFO:tasks.workunit.client.0.trial046.stderr: -91> 2026-02-24T22:22:41.898+0000 7f9c50aff640 10 monclient: get_auth_request con 0x5609884cd900 auth_method 0 2026-02-24T22:22:42.205 INFO:tasks.workunit.client.0.trial046.stderr: -90> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:22:42.205 INFO:tasks.workunit.client.0.trial046.stderr: -89> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 20 started sync module instance, tier type = 2026-02-24T22:22:42.205 INFO:tasks.workunit.client.0.trial046.stderr: -88> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:22:42.205 INFO:tasks.workunit.client.0.trial046.stderr: -87> 2026-02-24T22:22:41.902+0000 7f9c51b01640 10 monclient: get_auth_request con 0x56098852edb0 auth_method 0 2026-02-24T22:22:42.205 INFO:tasks.workunit.client.0.trial046.stderr: -86> 2026-02-24T22:22:41.902+0000 7f9c51300640 10 monclient: get_auth_request con 0x560988591520 auth_method 0 2026-02-24T22:22:42.205 INFO:tasks.workunit.client.0.trial046.stderr: -85> 2026-02-24T22:22:41.902+0000 7f9c50aff640 10 monclient: get_auth_request con 0x5609885f44a0 auth_method 0 2026-02-24T22:22:42.205 INFO:tasks.workunit.client.0.trial046.stderr: -84> 2026-02-24T22:22:41.902+0000 7f9c51b01640 10 monclient: get_auth_request con 0x5609886365e0 auth_method 0 2026-02-24T22:22:42.205 INFO:tasks.workunit.client.0.trial046.stderr: -83> 2026-02-24T22:22:41.902+0000 7f9c51300640 10 monclient: get_auth_request con 0x5609886995e0 auth_method 0 2026-02-24T22:22:42.205 INFO:tasks.workunit.client.0.trial046.stderr: -82> 2026-02-24T22:22:41.902+0000 7f9c51300640 0 -- 10.20.193.46:0/303166036 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x5609886995e0 msgr2=0x5609886b9a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:22:42.205 INFO:tasks.workunit.client.0.trial046.stderr: -81> 2026-02-24T22:22:41.902+0000 7f9c5382ca80 20 add_watcher() i=1 2026-02-24T22:22:42.205 INFO:tasks.workunit.client.0.trial046.stderr: -80> 2026-02-24T22:22:41.902+0000 7f9c5382ca80 20 add_watcher() i=6 2026-02-24T22:22:42.205 INFO:tasks.workunit.client.0.trial046.stderr: -79> 2026-02-24T22:22:41.902+0000 7f9c5382ca80 20 add_watcher() i=3 2026-02-24T22:22:42.205 INFO:tasks.workunit.client.0.trial046.stderr: -78> 2026-02-24T22:22:41.902+0000 7f9c5382ca80 20 add_watcher() i=0 2026-02-24T22:22:42.205 INFO:tasks.workunit.client.0.trial046.stderr: -77> 2026-02-24T22:22:41.902+0000 7f9c5382ca80 20 add_watcher() i=4 2026-02-24T22:22:42.205 INFO:tasks.workunit.client.0.trial046.stderr: -76> 2026-02-24T22:22:41.902+0000 7f9c5382ca80 20 add_watcher() i=2 2026-02-24T22:22:42.205 INFO:tasks.workunit.client.0.trial046.stderr: -75> 2026-02-24T22:22:41.902+0000 7f9c5382ca80 20 add_watcher() i=5 2026-02-24T22:22:42.205 INFO:tasks.workunit.client.0.trial046.stderr: -74> 2026-02-24T22:22:42.106+0000 7f9c51300640 10 monclient: get_auth_request con 0x5609886995e0 auth_method 0 2026-02-24T22:22:42.205 INFO:tasks.workunit.client.0.trial046.stderr: -73> 2026-02-24T22:22:42.106+0000 7f9c5382ca80 20 add_watcher() i=7 2026-02-24T22:22:42.205 INFO:tasks.workunit.client.0.trial046.stderr: -72> 2026-02-24T22:22:42.106+0000 7f9c5382ca80 2 all 8 watchers are set, enabling cache 2026-02-24T22:22:42.205 INFO:tasks.workunit.client.0.trial046.stderr: -71> 2026-02-24T22:22:42.106+0000 7f9c50aff640 10 monclient: get_auth_request con 0x7f9b70008af0 auth_method 0 2026-02-24T22:22:42.205 INFO:tasks.workunit.client.0.trial046.stderr: -70> 2026-02-24T22:22:42.106+0000 7f9c50aff640 0 -- 10.20.193.46:0/65096060 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f9b70008af0 msgr2=0x7f9b70028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:22:42.205 INFO:tasks.workunit.client.0.trial046.stderr: -69> 2026-02-24T22:22:42.106+0000 7f9c2effd640 1 client.8619.objecter ms_handle_reset 0x7f9b70008af0 session 0x7f9b70008470 osd.1 2026-02-24T22:22:42.205 INFO:tasks.workunit.client.0.trial046.stderr: -68> 2026-02-24T22:22:42.106+0000 7f9c2effd640 10 monclient: _renew_subs 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -67> 2026-02-24T22:22:42.106+0000 7f9c2effd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -66> 2026-02-24T22:22:42.106+0000 7f9c50aff640 10 monclient: get_auth_request con 0x7f9c2409d3f0 auth_method 0 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -65> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding auth protocol: cephx 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -64> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding auth protocol: cephx 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -63> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding auth protocol: cephx 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -62> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding con mode: secure 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -61> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding con mode: crc 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -60> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding con mode: secure 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -59> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding con mode: crc 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -58> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding con mode: secure 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -57> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding con mode: crc 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -56> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding con mode: crc 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -55> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding con mode: secure 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -54> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding con mode: crc 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -53> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding con mode: secure 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -52> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding con mode: crc 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -51> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding con mode: secure 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -50> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -49> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -48> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -47> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command cache list hook 0x560988416b90 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -46> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command cache inspect hook 0x560988416b90 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -45> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command cache erase hook 0x560988416b90 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -44> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command cache zap hook 0x560988416b90 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -43> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 note: GC not initialized 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -42> 2026-02-24T22:22:42.110+0000 7f9bfe7e4640 20 reqs_thread_entry: start 2026-02-24T22:22:42.206 INFO:tasks.workunit.client.0.trial046.stderr: -41> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command sync trace show hook 0x7f9c3c00e6c0 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -40> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command sync trace history hook 0x7f9c3c00e6c0 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -39> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command sync trace active hook 0x7f9c3c00e6c0 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -38> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command sync trace active_short hook 0x7f9c3c00e6c0 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -37> 2026-02-24T22:22:42.110+0000 7f9c51b01640 10 monclient: get_auth_request con 0x56098857cba0 auth_method 0 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -36> 2026-02-24T22:22:42.110+0000 7f9c51300640 10 monclient: get_auth_request con 0x560988571c60 auth_method 0 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -35> 2026-02-24T22:22:42.114+0000 7f9c50aff640 10 monclient: get_auth_request con 0x5609885734f0 auth_method 0 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -34> 2026-02-24T22:22:42.114+0000 7f9c51b01640 10 monclient: get_auth_request con 0x560988574d40 auth_method 0 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -33> 2026-02-24T22:22:42.118+0000 7f9c51300640 10 monclient: get_auth_request con 0x560988577280 auth_method 0 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -32> 2026-02-24T22:22:42.122+0000 7f9c50aff640 10 monclient: get_auth_request con 0x56098857a140 auth_method 0 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -31> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 20 init_complete bucket index max shards: 1 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -30> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 20 Filter name: none 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -29> 2026-02-24T22:22:42.146+0000 7f9bfcfe1640 20 reqs_thread_entry: start 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -28> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -27> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 20 rados->read ofs=0 len=0 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -26> 2026-02-24T22:22:42.146+0000 7f9c51b01640 10 monclient: get_auth_request con 0x56098857c2b0 auth_method 0 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -25> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -24> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -23> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -22> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : miss 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -21> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 20 rados->read ofs=0 len=0 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -20> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 20 rados_obj.operate() r=0 bl.length=470 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -19> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -18> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -17> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -16> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -15> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -14> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 init_target_layout removing existing target index objects from a previous reshard attempt 2026-02-24T22:22:42.207 INFO:tasks.workunit.client.0.trial046.stderr: -13> 2026-02-24T22:22:42.154+0000 7f9c5382ca80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:22:42.208 INFO:tasks.workunit.client.0.trial046.stderr: -12> 2026-02-24T22:22:42.154+0000 7f9c5382ca80 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:22:42.208 INFO:tasks.workunit.client.0.trial046.stderr: -11> 2026-02-24T22:22:42.154+0000 7f9c5382ca80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:22:42.208 INFO:tasks.workunit.client.0.trial046.stderr: -10> 2026-02-24T22:22:42.154+0000 7f9c5382ca80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518, ofs0, ns] 2026-02-24T22:22:42.208 INFO:tasks.workunit.client.0.trial046.stderr: -9> 2026-02-24T22:22:42.154+0000 7f9c2cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959947 cookie 94598943450400 notifier 8625 bl.length()=929 2026-02-24T22:22:42.208 INFO:tasks.workunit.client.0.trial046.stderr: -8> 2026-02-24T22:22:42.154+0000 7f9c2cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:22:42.208 INFO:tasks.workunit.client.0.trial046.stderr: -7> 2026-02-24T22:22:42.154+0000 7f9c2cff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:22:42.208 INFO:tasks.workunit.client.0.trial046.stderr: -6> 2026-02-24T22:22:42.154+0000 7f9c2cff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:22:42.208 INFO:tasks.workunit.client.0.trial046.stderr: -5> 2026-02-24T22:22:42.154+0000 7f9c5382ca80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_sources= new_sources= 2026-02-24T22:22:42.208 INFO:tasks.workunit.client.0.trial046.stderr: -4> 2026-02-24T22:22:42.154+0000 7f9c5382ca80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential sources added=[] removed=[] 2026-02-24T22:22:42.208 INFO:tasks.workunit.client.0.trial046.stderr: -3> 2026-02-24T22:22:42.154+0000 7f9c5382ca80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_dests= new_dests= 2026-02-24T22:22:42.208 INFO:tasks.workunit.client.0.trial046.stderr: -2> 2026-02-24T22:22:42.154+0000 7f9c5382ca80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential dests added=[] removed=[] 2026-02-24T22:22:42.208 INFO:tasks.workunit.client.0.trial046.stderr: -1> 2026-02-24T22:22:42.154+0000 7f9c5382ca80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f9c5382ca80 time 2026-02-24T22:22:42.158406+0000 2026-02-24T22:22:42.208 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:22:42.208 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.208 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:22:42.208 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f9c56c6eeb1] 2026-02-24T22:22:42.208 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x5609532fcb9f] 2026-02-24T22:22:42.208 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x16a) [0x560953a6826a] 2026-02-24T22:22:42.208 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:22:42.208 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f9c55f6ed90] 2026-02-24T22:22:42.208 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:22:42.208 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:22:42.208 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.208 INFO:tasks.workunit.client.0.trial046.stderr: 0> 2026-02-24T22:22:42.158+0000 7f9c5382ca80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:22:42.208 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f9c5382ca80 thread_name:radosgw-admin 2026-02-24T22:22:42.208 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f9c55f87520] 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f9c56c6ef10] 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x5609532fcb9f] 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x16a) [0x560953a6826a] 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f9c55f6ed90] 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr:--- logging levels --- 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 none 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 lockdep 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 context 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 crush 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_balancer 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_locker 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log_expire 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_migrator 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: 3/ 5 mds_quiesce 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 buffer 2026-02-24T22:22:42.209 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 timer 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 filer 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 striper 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 objecter 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rados 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_mirror 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_replay 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_pwl 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 journaler 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objectcacher 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 immutable_obj_cache 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 client 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 osd 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 optracker 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objclass 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 filestore 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 journal 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 ms 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mon 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 0/10 monc 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 paxos 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 tp 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 auth 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 crypto 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 finisher 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 reserver 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 heartbeatmap 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 perfcounter 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_sync 2026-02-24T22:22:42.210 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_datacache 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_access 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dbstore 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_flight 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_lifecycle 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_restore 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw_notification 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_bucket_logging 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 javaclient 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 asok 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 throttle 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 refs 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 compressor 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore_compression 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluefs 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 bdev 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 kstore 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 4/ 5 rocksdb 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 fuse 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 2/ 5 mgr 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mgrc 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 dpdk 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 eventtrace 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 prioritycache 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 test 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephfs_mirror 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephsqlite 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 crimson_interrupt 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_onode 2026-02-24T22:22:42.211 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_odata 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_omap 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_tm 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_t 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cleaner 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_epm 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_lba 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_fixedkv_tree 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cache 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_journal 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_device 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_backref 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 alienstore 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mclock 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dedup 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cyanstore 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 ceph_exporter 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 memstore 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 trace 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 ceph_dedup 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: -2/-2 (syslog threshold) 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 99/99 (stderr threshold) 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr:--- pthread ID / name mapping for recent threads --- 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 7f9bfcfe1640 / http_manager 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 7f9bfe7e4640 / http_manager 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 7f9c1affd640 / ms_dispatch 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 7f9c2cff9640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 7f9c2effd640 / ms_dispatch 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 7f9c35ffb640 / ms_dispatch 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 7f9c377fe640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 7f9c37fff640 / admin_socket 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 7f9c43fff640 / ms_dispatch 2026-02-24T22:22:42.212 INFO:tasks.workunit.client.0.trial046.stderr: 7f9c50aff640 / msgr-worker-2 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr: 7f9c51300640 / msgr-worker-1 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr: 7f9c51b01640 / msgr-worker-0 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr: 7f9c52302640 / io_context_pool 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr: 7f9c5382ca80 / radosgw-admin 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr: max_recent 500 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr: max_new 1000 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr: log_file /var/log/ceph/ceph-client.admin.55892.log 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr:--- end dump of recent events --- 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr:--- begin dump of recent events --- 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr: -499> 2026-02-24T22:22:41.882+0000 7f9c377fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr: -498> 2026-02-24T22:22:41.882+0000 7f9c377fe640 10 monclient: dump: 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr: -497> 2026-02-24T22:22:41.882+0000 7f9c5382ca80 5 monclient: authenticate success, global_id 8033 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr: -496> 2026-02-24T22:22:41.882+0000 7f9c5382ca80 10 monclient: get_monmap_and_config success 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr: -495> 2026-02-24T22:22:41.882+0000 7f9c5382ca80 4 set_mon_vals no callback set 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr: -494> 2026-02-24T22:22:41.882+0000 7f9c5382ca80 10 monclient: shutdown 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr: -493> 2026-02-24T22:22:41.882+0000 7f9c5382ca80 5 asok(0x560988045f00) unregister_commands rotate-key 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr: -492> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient: build_initial_monmap 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr: -491> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 1 build_initial for_mkfs: 0 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr: -490> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient: monmap: 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:22:42.213 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr: -489> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding auth protocol: cephx 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr: -488> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding auth protocol: cephx 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr: -487> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding auth protocol: cephx 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr: -486> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding con mode: secure 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr: -485> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding con mode: crc 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr: -484> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding con mode: secure 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr: -483> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding con mode: crc 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr: -482> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding con mode: secure 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr: -481> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding con mode: crc 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr: -480> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding con mode: crc 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr: -479> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding con mode: secure 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr: -478> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding con mode: crc 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr: -477> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding con mode: secure 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr: -476> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding con mode: crc 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr: -475> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x56098814e4a0) adding con mode: secure 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr: -474> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr: -473> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 1 librados: starting msgr at 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr: -472> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 1 librados: starting objecter 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr: -471> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command objecter_requests hook 0x560988150200 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr: -470> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 1 librados: setting wanted keys 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr: -469> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 1 librados: calling monclient init 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr: -468> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient: init 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr: -467> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding auth protocol: cephx 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr: -466> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding auth protocol: cephx 2026-02-24T22:22:42.214 INFO:tasks.workunit.client.0.trial046.stderr: -465> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding auth protocol: cephx 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -464> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding con mode: secure 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -463> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding con mode: crc 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -462> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding con mode: secure 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -461> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding con mode: crc 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -460> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding con mode: secure 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -459> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding con mode: crc 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -458> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding con mode: crc 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -457> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding con mode: secure 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -456> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding con mode: crc 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -455> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding con mode: secure 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -454> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding con mode: crc 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -453> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 AuthRegistry(0x560988148e30) adding con mode: secure 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -452> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -451> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -450> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command rotate-key hook 0x560988148f78 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -449> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient: _reopen_session rank -1 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -448> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -447> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient(hunting): picked mon.noname-b con 0x5609880d3c90 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -446> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient(hunting): picked mon.noname-a con 0x5609880d9f10 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -445> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient(hunting): picked mon.noname-c con 0x56098817ebf0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -444> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -443> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -442> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -441> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient(hunting): _renew_subs 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -440> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient(hunting): authenticate will time out at 2131.992710s 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -439> 2026-02-24T22:22:41.886+0000 7f9c51b01640 10 monclient(hunting): get_auth_request con 0x56098817ebf0 auth_method 0 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -438> 2026-02-24T22:22:41.886+0000 7f9c51b01640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -437> 2026-02-24T22:22:41.886+0000 7f9c51b01640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -436> 2026-02-24T22:22:41.886+0000 7f9c51b01640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -435> 2026-02-24T22:22:41.886+0000 7f9c50aff640 10 monclient(hunting): get_auth_request con 0x5609880d9f10 auth_method 0 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -434> 2026-02-24T22:22:41.886+0000 7f9c50aff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -433> 2026-02-24T22:22:41.886+0000 7f9c50aff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -432> 2026-02-24T22:22:41.886+0000 7f9c50aff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:42.215 INFO:tasks.workunit.client.0.trial046.stderr: -431> 2026-02-24T22:22:41.886+0000 7f9c51b01640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr: -430> 2026-02-24T22:22:41.886+0000 7f9c51b01640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr: -429> 2026-02-24T22:22:41.886+0000 7f9c51b01640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr: -428> 2026-02-24T22:22:41.886+0000 7f9c50aff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr: -427> 2026-02-24T22:22:41.886+0000 7f9c50aff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr: -426> 2026-02-24T22:22:41.886+0000 7f9c50aff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr: -425> 2026-02-24T22:22:41.886+0000 7f9c51300640 10 monclient(hunting): get_auth_request con 0x5609880d3c90 auth_method 0 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr: -424> 2026-02-24T22:22:41.886+0000 7f9c51300640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr: -423> 2026-02-24T22:22:41.886+0000 7f9c51300640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr: -422> 2026-02-24T22:22:41.886+0000 7f9c51300640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr: -421> 2026-02-24T22:22:41.886+0000 7f9c51b01640 10 monclient(hunting): handle_auth_done global_id 8036 payload 824 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr: -420> 2026-02-24T22:22:41.886+0000 7f9c51b01640 10 monclient: _finish_hunting 0 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr: -419> 2026-02-24T22:22:41.886+0000 7f9c51b01640 1 monclient: found mon.noname-c 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr: -418> 2026-02-24T22:22:41.886+0000 7f9c51b01640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr: -417> 2026-02-24T22:22:41.886+0000 7f9c35ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr: -416> 2026-02-24T22:22:41.886+0000 7f9c35ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr: -415> 2026-02-24T22:22:41.886+0000 7f9c35ffb640 10 monclient: dump: 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr: -414> 2026-02-24T22:22:41.886+0000 7f9c35ffb640 10 monclient: _finish_auth 0 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr: -413> 2026-02-24T22:22:41.886+0000 7f9c35ffb640 10 monclient: _check_auth_tickets 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr: -412> 2026-02-24T22:22:41.886+0000 7f9c35ffb640 10 monclient: handle_config config(0 keys) 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr: -411> 2026-02-24T22:22:41.886+0000 7f9c35ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr: -410> 2026-02-24T22:22:41.886+0000 7f9c35ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-24T22:22:42.216 INFO:tasks.workunit.client.0.trial046.stderr: -409> 2026-02-24T22:22:41.886+0000 7f9c35ffb640 10 monclient: dump: 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr: -408> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 5 monclient: authenticate success, global_id 8036 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr: -407> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient: _renew_subs 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr: -406> 2026-02-24T22:22:41.886+0000 7f9c377fe640 4 set_mon_vals no callback set 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr: -405> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr: -404> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient: _renew_subs 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr: -403> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 10 monclient: _send_mon_message to mon.c at v2:10.20.193.46:3301/0 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr: -402> 2026-02-24T22:22:41.886+0000 7f9c5382ca80 1 librados: init done 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr: -401> 2026-02-24T22:22:41.886+0000 7f9c35ffb640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr: -400> 2026-02-24T22:22:41.886+0000 7f9c35ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr: -399> 2026-02-24T22:22:41.886+0000 7f9c35ffb640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr: -398> 2026-02-24T22:22:41.886+0000 7f9c51300640 10 monclient: get_auth_request con 0x7f9c300707b0 auth_method 0 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr: -397> 2026-02-24T22:22:41.886+0000 7f9c50aff640 10 monclient: get_auth_request con 0x5609881a9ef0 auth_method 0 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr: -396> 2026-02-24T22:22:41.890+0000 7f9c51300640 10 monclient: get_auth_request con 0x5609881d35f0 auth_method 0 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr: -395> 2026-02-24T22:22:41.890+0000 7f9c50aff640 10 monclient: get_auth_request con 0x5609881d4e10 auth_method 0 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr: -394> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient: build_initial_monmap 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr: -393> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 1 build_initial for_mkfs: 0 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr: -392> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient: monmap: 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:42.217 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -391> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding auth protocol: cephx 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -390> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding auth protocol: cephx 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -389> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding auth protocol: cephx 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -388> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding con mode: secure 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -387> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding con mode: crc 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -386> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding con mode: secure 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -385> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding con mode: crc 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -384> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding con mode: secure 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -383> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding con mode: crc 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -382> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding con mode: crc 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -381> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding con mode: secure 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -380> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding con mode: crc 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -379> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding con mode: secure 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -378> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding con mode: crc 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -377> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d7d90) adding con mode: secure 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -376> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -375> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command objecter_requests cmddesc objecter_requests hook 0x5609881dc650 EEXIST 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -374> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient: init 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -373> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding auth protocol: cephx 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -372> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding auth protocol: cephx 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -371> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding auth protocol: cephx 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -370> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding con mode: secure 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -369> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding con mode: crc 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -368> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding con mode: secure 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -367> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding con mode: crc 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -366> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding con mode: secure 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -365> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding con mode: crc 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -364> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding con mode: crc 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -363> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding con mode: secure 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -362> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding con mode: crc 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -361> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding con mode: secure 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -360> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding con mode: crc 2026-02-24T22:22:42.218 INFO:tasks.workunit.client.0.trial046.stderr: -359> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 AuthRegistry(0x5609881d5828) adding con mode: secure 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -358> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -357> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -356> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command rotate-key cmddesc rotate-key hook 0x5609881d5970 EEXIST 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -355> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient: _reopen_session rank -1 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -354> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient: _add_conns ranks=[1,2,0] 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -353> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient(hunting): picked mon.noname-b con 0x560988330720 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -352> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient(hunting): picked mon.noname-c con 0x56098832d7d0 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -351> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient(hunting): picked mon.noname-a con 0x56098832e3a0 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -350> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -349> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -348> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -347> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient(hunting): _renew_subs 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -346> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient(hunting): authenticate will time out at 2131.999176s 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -345> 2026-02-24T22:22:41.890+0000 7f9c50aff640 10 monclient(hunting): get_auth_request con 0x56098832d7d0 auth_method 0 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -344> 2026-02-24T22:22:41.890+0000 7f9c50aff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -343> 2026-02-24T22:22:41.890+0000 7f9c50aff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -342> 2026-02-24T22:22:41.890+0000 7f9c50aff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -341> 2026-02-24T22:22:41.890+0000 7f9c51b01640 10 monclient(hunting): get_auth_request con 0x56098832e3a0 auth_method 0 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -340> 2026-02-24T22:22:41.890+0000 7f9c51b01640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -339> 2026-02-24T22:22:41.890+0000 7f9c51b01640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -338> 2026-02-24T22:22:41.890+0000 7f9c51b01640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -337> 2026-02-24T22:22:41.890+0000 7f9c50aff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -336> 2026-02-24T22:22:41.890+0000 7f9c50aff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -335> 2026-02-24T22:22:41.890+0000 7f9c50aff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -334> 2026-02-24T22:22:41.890+0000 7f9c51b01640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -333> 2026-02-24T22:22:41.890+0000 7f9c51b01640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -332> 2026-02-24T22:22:41.890+0000 7f9c51b01640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -331> 2026-02-24T22:22:41.890+0000 7f9c51300640 10 monclient(hunting): get_auth_request con 0x560988330720 auth_method 0 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -330> 2026-02-24T22:22:41.890+0000 7f9c51300640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -329> 2026-02-24T22:22:41.890+0000 7f9c51300640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -328> 2026-02-24T22:22:41.890+0000 7f9c51300640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -327> 2026-02-24T22:22:41.890+0000 7f9c51b01640 10 monclient(hunting): handle_auth_done global_id 8619 payload 824 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -326> 2026-02-24T22:22:41.890+0000 7f9c51b01640 10 monclient: _finish_hunting 0 2026-02-24T22:22:42.219 INFO:tasks.workunit.client.0.trial046.stderr: -325> 2026-02-24T22:22:41.890+0000 7f9c51b01640 1 monclient: found mon.noname-a 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr: -324> 2026-02-24T22:22:41.890+0000 7f9c51b01640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr: -323> 2026-02-24T22:22:41.890+0000 7f9c2effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr: -322> 2026-02-24T22:22:41.890+0000 7f9c2effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr: -321> 2026-02-24T22:22:41.890+0000 7f9c2effd640 10 monclient: dump: 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr: -320> 2026-02-24T22:22:41.890+0000 7f9c2effd640 10 monclient: _finish_auth 0 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr: -319> 2026-02-24T22:22:41.890+0000 7f9c2effd640 10 monclient: _check_auth_tickets 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr: -318> 2026-02-24T22:22:41.890+0000 7f9c2effd640 10 monclient: handle_config config(0 keys) 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr: -317> 2026-02-24T22:22:41.890+0000 7f9c2effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr: -316> 2026-02-24T22:22:41.890+0000 7f9c2effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr: -315> 2026-02-24T22:22:41.890+0000 7f9c2effd640 10 monclient: dump: 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr: -314> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 5 monclient: authenticate success, global_id 8619 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr: -313> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient: _renew_subs 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr: -312> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr: -311> 2026-02-24T22:22:41.890+0000 7f9c52302640 4 set_mon_vals no callback set 2026-02-24T22:22:42.220 INFO:tasks.workunit.client.0.trial046.stderr: -310> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient: _renew_subs 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr: -309> 2026-02-24T22:22:41.890+0000 7f9c5382ca80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr: -308> 2026-02-24T22:22:41.894+0000 7f9c2effd640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr: -307> 2026-02-24T22:22:41.894+0000 7f9c2effd640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr: -306> 2026-02-24T22:22:41.894+0000 7f9c2effd640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr: -305> 2026-02-24T22:22:41.894+0000 7f9c51300640 10 monclient: get_auth_request con 0x7f9c240706c0 auth_method 0 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr: -304> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient: get_monmap_and_config 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr: -303> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient: build_initial_monmap 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr: -302> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 1 build_initial for_mkfs: 0 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr: -301> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient: monmap: 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr: -300> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding auth protocol: cephx 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr: -299> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding auth protocol: cephx 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr: -298> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding auth protocol: cephx 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr: -297> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: secure 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr: -296> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: crc 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr: -295> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: secure 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr: -294> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: crc 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr: -293> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: secure 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr: -292> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: crc 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr: -291> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: crc 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr: -290> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: secure 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr: -289> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: crc 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr: -288> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: secure 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr: -287> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: crc 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr: -286> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: secure 2026-02-24T22:22:42.221 INFO:tasks.workunit.client.0.trial046.stderr: -285> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -284> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient: init 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -283> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding auth protocol: cephx 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -282> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding auth protocol: cephx 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -281> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding auth protocol: cephx 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -280> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding con mode: secure 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -279> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding con mode: crc 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -278> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding con mode: secure 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -277> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding con mode: crc 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -276> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding con mode: secure 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -275> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding con mode: crc 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -274> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding con mode: crc 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -273> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding con mode: secure 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -272> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding con mode: crc 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -271> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding con mode: secure 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -270> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding con mode: crc 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -269> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f7170) adding con mode: secure 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -268> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -267> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -266> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command rotate-key cmddesc rotate-key hook 0x7fff0e4f72b8 EEXIST 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -265> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient: _reopen_session rank -1 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -264> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient: _add_conns ranks=[2,0,1] 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -263> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient(hunting): picked mon.noname-c con 0x5609883e8110 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -262> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient(hunting): picked mon.noname-a con 0x5609883eb880 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -261> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient(hunting): picked mon.noname-b con 0x5609883c6640 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -260> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -259> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -258> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -257> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient(hunting): _renew_subs 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -256> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient(hunting): authenticate will time out at 2132.002373s 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -255> 2026-02-24T22:22:41.894+0000 7f9c51300640 10 monclient(hunting): get_auth_request con 0x5609883e8110 auth_method 0 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -254> 2026-02-24T22:22:41.894+0000 7f9c51300640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -253> 2026-02-24T22:22:41.894+0000 7f9c51300640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -252> 2026-02-24T22:22:41.894+0000 7f9c51300640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -251> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): get_auth_request con 0x5609883eb880 auth_method 0 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -250> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:42.222 INFO:tasks.workunit.client.0.trial046.stderr: -249> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -248> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -247> 2026-02-24T22:22:41.894+0000 7f9c51300640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -246> 2026-02-24T22:22:41.894+0000 7f9c51300640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -245> 2026-02-24T22:22:41.894+0000 7f9c51300640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -244> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -243> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -242> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -241> 2026-02-24T22:22:41.894+0000 7f9c51300640 10 monclient(hunting): handle_auth_done global_id 8042 payload 274 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -240> 2026-02-24T22:22:41.894+0000 7f9c51300640 10 monclient: _finish_hunting 0 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -239> 2026-02-24T22:22:41.894+0000 7f9c51300640 1 monclient: found mon.noname-c 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -238> 2026-02-24T22:22:41.894+0000 7f9c51300640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.46:3301/0 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -237> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -236> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -235> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient: dump: 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -234> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient: _finish_auth 0 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -233> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient: _check_auth_tickets 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -232> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient: ms_handle_reset current mon [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -231> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient: _reopen_session rank -1 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -230> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -229> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient(hunting): picked mon.b con 0x5609883c6640 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -228> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient(hunting): picked mon.a con 0x5609883eb880 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -227> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient(hunting): picked mon.c con 0x7f9c14042f30 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -226> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -225> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -224> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -223> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient(hunting): _renew_subs 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -222> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient(hunting): ms_handle_reset hunted mon [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -221> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): get_auth_request con 0x5609883eb880 auth_method 0 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -220> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:42.223 INFO:tasks.workunit.client.0.trial046.stderr: -219> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -218> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -217> 2026-02-24T22:22:41.894+0000 7f9c51b01640 10 monclient(hunting): get_auth_request con 0x7f9c14042f30 auth_method 0 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -216> 2026-02-24T22:22:41.894+0000 7f9c51b01640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -215> 2026-02-24T22:22:41.894+0000 7f9c51b01640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -214> 2026-02-24T22:22:41.894+0000 7f9c51b01640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -213> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -212> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -211> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -210> 2026-02-24T22:22:41.894+0000 7f9c51b01640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -209> 2026-02-24T22:22:41.894+0000 7f9c51b01640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -208> 2026-02-24T22:22:41.894+0000 7f9c51b01640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -207> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient(hunting): handle_auth_done global_id 8042 payload 293 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -206> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient: _finish_hunting 0 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -205> 2026-02-24T22:22:41.894+0000 7f9c50aff640 1 monclient: found mon.a 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -204> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -203> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient: _finish_auth 0 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -202> 2026-02-24T22:22:41.894+0000 7f9c50aff640 10 monclient: _check_auth_tickets 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -201> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 monclient: authenticate success, global_id 8042 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -200> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -199> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -198> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient: dump: 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -197> 2026-02-24T22:22:41.894+0000 7f9c2cff9640 10 monclient: handle_config config(0 keys) 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -196> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient: get_monmap_and_config success 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -195> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 4 set_mon_vals no callback set 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -194> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient: shutdown 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -193> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient: build_initial_monmap 2026-02-24T22:22:42.224 INFO:tasks.workunit.client.0.trial046.stderr: -192> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 1 build_initial for_mkfs: 0 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -191> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient: monmap: 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr:epoch 0 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 0.000000 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr:created 0.000000 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 0 (unknown) 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.noname-a 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.noname-b 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.noname-c 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -190> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding auth protocol: cephx 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -189> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding auth protocol: cephx 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -188> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding auth protocol: cephx 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -187> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: secure 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -186> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: crc 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -185> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: secure 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -184> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: crc 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -183> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: secure 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -182> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: crc 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -181> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: crc 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -180> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: secure 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -179> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: crc 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -178> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: secure 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -177> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: crc 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -176> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c11f0) adding con mode: secure 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -175> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -174> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 1 librados: starting msgr at 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -173> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 1 librados: starting objecter 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -172> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command objecter_requests cmddesc objecter_requests hook 0x5609883c6ca0 EEXIST 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -171> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 1 librados: setting wanted keys 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -170> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 1 librados: calling monclient init 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -169> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 10 monclient: init 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -168> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding auth protocol: cephx 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -167> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding auth protocol: cephx 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -166> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding auth protocol: cephx 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -165> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding con mode: secure 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -164> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding con mode: crc 2026-02-24T22:22:42.225 INFO:tasks.workunit.client.0.trial046.stderr: -163> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding con mode: secure 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -162> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding con mode: crc 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -161> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding con mode: secure 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -160> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding con mode: crc 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -159> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding con mode: crc 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -158> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding con mode: secure 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -157> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding con mode: crc 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -156> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding con mode: secure 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -155> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding con mode: crc 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -154> 2026-02-24T22:22:41.894+0000 7f9c5382ca80 5 AuthRegistry(0x5609883c0400) adding con mode: secure 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -153> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -152> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -151> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command rotate-key cmddesc rotate-key hook 0x5609883c0548 EEXIST 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -150> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient: _reopen_session rank -1 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -149> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient: _add_conns ranks=[2,1,0] 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -148> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient(hunting): picked mon.noname-c con 0x5609883eb880 addr [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -147> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient(hunting): picked mon.noname-b con 0x7f9c14042f30 addr [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -146> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient(hunting): picked mon.noname-a con 0x560988411900 addr [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -145> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -144> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -143> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient(hunting): start opening mon connection 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -142> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient(hunting): _renew_subs 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -141> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient(hunting): authenticate will time out at 2132.004178s 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -140> 2026-02-24T22:22:41.898+0000 7f9c51b01640 10 monclient(hunting): get_auth_request con 0x560988411900 auth_method 0 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -139> 2026-02-24T22:22:41.898+0000 7f9c51b01640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -138> 2026-02-24T22:22:41.898+0000 7f9c51b01640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -137> 2026-02-24T22:22:41.898+0000 7f9c51b01640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -136> 2026-02-24T22:22:41.898+0000 7f9c51300640 10 monclient(hunting): get_auth_request con 0x5609883eb880 auth_method 0 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -135> 2026-02-24T22:22:41.898+0000 7f9c51300640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -134> 2026-02-24T22:22:41.898+0000 7f9c51300640 10 monclient(hunting): _init_auth method 2 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -133> 2026-02-24T22:22:41.898+0000 7f9c51300640 10 monclient(hunting): _init_auth creating new auth 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -132> 2026-02-24T22:22:41.898+0000 7f9c51b01640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -131> 2026-02-24T22:22:41.898+0000 7f9c51b01640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -130> 2026-02-24T22:22:41.898+0000 7f9c51b01640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -129> 2026-02-24T22:22:41.898+0000 7f9c51300640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -128> 2026-02-24T22:22:41.898+0000 7f9c51300640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -127> 2026-02-24T22:22:41.898+0000 7f9c51300640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -126> 2026-02-24T22:22:41.898+0000 7f9c51b01640 10 monclient(hunting): handle_auth_done global_id 8625 payload 824 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -125> 2026-02-24T22:22:41.898+0000 7f9c51b01640 10 monclient: _finish_hunting 0 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -124> 2026-02-24T22:22:41.898+0000 7f9c51b01640 1 monclient: found mon.noname-a 2026-02-24T22:22:42.226 INFO:tasks.workunit.client.0.trial046.stderr: -123> 2026-02-24T22:22:41.898+0000 7f9c51b01640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.46:3300/0 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr: -122> 2026-02-24T22:22:41.898+0000 7f9c1affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr: -121> 2026-02-24T22:22:41.898+0000 7f9c1affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr: -120> 2026-02-24T22:22:41.898+0000 7f9c1affd640 10 monclient: dump: 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr: -119> 2026-02-24T22:22:41.898+0000 7f9c1affd640 10 monclient: _finish_auth 0 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr: -118> 2026-02-24T22:22:41.898+0000 7f9c1affd640 10 monclient: _check_auth_tickets 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr: -117> 2026-02-24T22:22:41.898+0000 7f9c1affd640 10 monclient: handle_config config(0 keys) 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr: -116> 2026-02-24T22:22:41.898+0000 7f9c1affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr: -115> 2026-02-24T22:22:41.898+0000 7f9c1affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr: -114> 2026-02-24T22:22:41.898+0000 7f9c1affd640 10 monclient: dump: 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr:epoch 1 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr:fsid 262e7508-9310-4d2a-bc05-2c8861262eff 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr:last_changed 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr:created 2026-02-24T22:04:35.561560+0000 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr:min_mon_release 20 (tentacle) 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr:election_strategy: 1 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr:0: [v2:10.20.193.46:3300/0,v1:10.20.193.46:6789/0] mon.a 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr:1: [v2:10.20.193.110:3300/0,v1:10.20.193.110:6789/0] mon.b 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr:2: [v2:10.20.193.46:3301/0,v1:10.20.193.46:6790/0] mon.c 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr: -113> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 5 monclient: authenticate success, global_id 8625 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr: -112> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient: _renew_subs 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr: -111> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr: -110> 2026-02-24T22:22:41.898+0000 7f9c2cff9640 4 set_mon_vals no callback set 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr: -109> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient: _renew_subs 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr: -108> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr: -107> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 1 librados: init done 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr: -106> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command cr dump hook 0x560988416198 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr: -105> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 4 Realm: () 2026-02-24T22:22:42.227 INFO:tasks.workunit.client.0.trial046.stderr: -104> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -103> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -102> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -101> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 20 zonegroup default 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -100> 2026-02-24T22:22:41.898+0000 7f9c51300640 10 monclient: get_auth_request con 0x560988488ef0 auth_method 0 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -99> 2026-02-24T22:22:41.898+0000 7f9c1affd640 4 mgrc handle_mgr_map Got map version 5 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -98> 2026-02-24T22:22:41.898+0000 7f9c1affd640 4 mgrc handle_mgr_map Active mgr is now [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -97> 2026-02-24T22:22:41.898+0000 7f9c1affd640 4 mgrc reconnect Starting new session with [v2:10.20.193.46:6800/530373280,v1:10.20.193.46:6801/530373280] 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -96> 2026-02-24T22:22:41.898+0000 7f9c50aff640 10 monclient: get_auth_request con 0x7f9c14071960 auth_method 0 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -95> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 20 rados->read ofs=0 len=0 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -94> 2026-02-24T22:22:41.898+0000 7f9c51300640 10 monclient: get_auth_request con 0x5609884ac2a0 auth_method 0 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -93> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -92> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 20 rados->read ofs=0 len=0 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -91> 2026-02-24T22:22:41.898+0000 7f9c50aff640 10 monclient: get_auth_request con 0x5609884cd900 auth_method 0 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -90> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -89> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 20 started sync module instance, tier type = 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -88> 2026-02-24T22:22:41.898+0000 7f9c5382ca80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -87> 2026-02-24T22:22:41.902+0000 7f9c51b01640 10 monclient: get_auth_request con 0x56098852edb0 auth_method 0 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -86> 2026-02-24T22:22:41.902+0000 7f9c51300640 10 monclient: get_auth_request con 0x560988591520 auth_method 0 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -85> 2026-02-24T22:22:41.902+0000 7f9c50aff640 10 monclient: get_auth_request con 0x5609885f44a0 auth_method 0 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -84> 2026-02-24T22:22:41.902+0000 7f9c51b01640 10 monclient: get_auth_request con 0x5609886365e0 auth_method 0 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -83> 2026-02-24T22:22:41.902+0000 7f9c51300640 10 monclient: get_auth_request con 0x5609886995e0 auth_method 0 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -82> 2026-02-24T22:22:41.902+0000 7f9c51300640 0 -- 10.20.193.46:0/303166036 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x5609886995e0 msgr2=0x5609886b9a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -81> 2026-02-24T22:22:41.902+0000 7f9c5382ca80 20 add_watcher() i=1 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -80> 2026-02-24T22:22:41.902+0000 7f9c5382ca80 20 add_watcher() i=6 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -79> 2026-02-24T22:22:41.902+0000 7f9c5382ca80 20 add_watcher() i=3 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -78> 2026-02-24T22:22:41.902+0000 7f9c5382ca80 20 add_watcher() i=0 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -77> 2026-02-24T22:22:41.902+0000 7f9c5382ca80 20 add_watcher() i=4 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -76> 2026-02-24T22:22:41.902+0000 7f9c5382ca80 20 add_watcher() i=2 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -75> 2026-02-24T22:22:41.902+0000 7f9c5382ca80 20 add_watcher() i=5 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -74> 2026-02-24T22:22:42.106+0000 7f9c51300640 10 monclient: get_auth_request con 0x5609886995e0 auth_method 0 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -73> 2026-02-24T22:22:42.106+0000 7f9c5382ca80 20 add_watcher() i=7 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -72> 2026-02-24T22:22:42.106+0000 7f9c5382ca80 2 all 8 watchers are set, enabling cache 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -71> 2026-02-24T22:22:42.106+0000 7f9c50aff640 10 monclient: get_auth_request con 0x7f9b70008af0 auth_method 0 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -70> 2026-02-24T22:22:42.106+0000 7f9c50aff640 0 -- 10.20.193.46:0/65096060 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f9b70008af0 msgr2=0x7f9b70028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -69> 2026-02-24T22:22:42.106+0000 7f9c2effd640 1 client.8619.objecter ms_handle_reset 0x7f9b70008af0 session 0x7f9b70008470 osd.1 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -68> 2026-02-24T22:22:42.106+0000 7f9c2effd640 10 monclient: _renew_subs 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -67> 2026-02-24T22:22:42.106+0000 7f9c2effd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.46:3300/0 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -66> 2026-02-24T22:22:42.106+0000 7f9c50aff640 10 monclient: get_auth_request con 0x7f9c2409d3f0 auth_method 0 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -65> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding auth protocol: cephx 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -64> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding auth protocol: cephx 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -63> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding auth protocol: cephx 2026-02-24T22:22:42.228 INFO:tasks.workunit.client.0.trial046.stderr: -62> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding con mode: secure 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -61> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding con mode: crc 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -60> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding con mode: secure 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -59> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding con mode: crc 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -58> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding con mode: secure 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -57> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding con mode: crc 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -56> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding con mode: crc 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -55> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding con mode: secure 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -54> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding con mode: crc 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -53> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding con mode: secure 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -52> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding con mode: crc 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -51> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 AuthRegistry(0x7fff0e4f75e0) adding con mode: secure 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -50> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -49> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -48> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -47> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command cache list hook 0x560988416b90 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -46> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command cache inspect hook 0x560988416b90 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -45> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command cache erase hook 0x560988416b90 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -44> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command cache zap hook 0x560988416b90 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -43> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 note: GC not initialized 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -42> 2026-02-24T22:22:42.110+0000 7f9bfe7e4640 20 reqs_thread_entry: start 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -41> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command sync trace show hook 0x7f9c3c00e6c0 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -40> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command sync trace history hook 0x7f9c3c00e6c0 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -39> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command sync trace active hook 0x7f9c3c00e6c0 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -38> 2026-02-24T22:22:42.110+0000 7f9c5382ca80 5 asok(0x560988045f00) register_command sync trace active_short hook 0x7f9c3c00e6c0 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -37> 2026-02-24T22:22:42.110+0000 7f9c51b01640 10 monclient: get_auth_request con 0x56098857cba0 auth_method 0 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -36> 2026-02-24T22:22:42.110+0000 7f9c51300640 10 monclient: get_auth_request con 0x560988571c60 auth_method 0 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -35> 2026-02-24T22:22:42.114+0000 7f9c50aff640 10 monclient: get_auth_request con 0x5609885734f0 auth_method 0 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -34> 2026-02-24T22:22:42.114+0000 7f9c51b01640 10 monclient: get_auth_request con 0x560988574d40 auth_method 0 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -33> 2026-02-24T22:22:42.118+0000 7f9c51300640 10 monclient: get_auth_request con 0x560988577280 auth_method 0 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -32> 2026-02-24T22:22:42.122+0000 7f9c50aff640 10 monclient: get_auth_request con 0x56098857a140 auth_method 0 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -31> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 20 init_complete bucket index max shards: 1 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -30> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 20 Filter name: none 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -29> 2026-02-24T22:22:42.146+0000 7f9bfcfe1640 20 reqs_thread_entry: start 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -28> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -27> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 20 rados->read ofs=0 len=0 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -26> 2026-02-24T22:22:42.146+0000 7f9c51b01640 10 monclient: get_auth_request con 0x56098857c2b0 auth_method 0 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -25> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -24> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -23> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -22> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : miss 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -21> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 20 rados->read ofs=0 len=0 2026-02-24T22:22:42.229 INFO:tasks.workunit.client.0.trial046.stderr: -20> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 20 rados_obj.operate() r=0 bl.length=470 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: -19> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: -18> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: -17> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: -16> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: -15> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: -14> 2026-02-24T22:22:42.146+0000 7f9c5382ca80 10 init_target_layout removing existing target index objects from a previous reshard attempt 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: -13> 2026-02-24T22:22:42.154+0000 7f9c5382ca80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: -12> 2026-02-24T22:22:42.154+0000 7f9c5382ca80 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: -11> 2026-02-24T22:22:42.154+0000 7f9c5382ca80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: -10> 2026-02-24T22:22:42.154+0000 7f9c5382ca80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518, ofs0, ns] 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: -9> 2026-02-24T22:22:42.154+0000 7f9c2cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959947 cookie 94598943450400 notifier 8625 bl.length()=929 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: -8> 2026-02-24T22:22:42.154+0000 7f9c2cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: -7> 2026-02-24T22:22:42.154+0000 7f9c2cff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: -6> 2026-02-24T22:22:42.154+0000 7f9c2cff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: -5> 2026-02-24T22:22:42.154+0000 7f9c5382ca80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_sources= new_sources= 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: -4> 2026-02-24T22:22:42.154+0000 7f9c5382ca80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential sources added=[] removed=[] 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: -3> 2026-02-24T22:22:42.154+0000 7f9c5382ca80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_dests= new_dests= 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: -2> 2026-02-24T22:22:42.154+0000 7f9c5382ca80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential dests added=[] removed=[] 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: -1> 2026-02-24T22:22:42.154+0000 7f9c5382ca80 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f9c5382ca80 time 2026-02-24T22:22:42.158406+0000 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr:./src/common/fault_injector.h: 121: FAILED ceph_assert(!"FaultInjector") 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x137) [0x7f9c56c6eeb1] 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: 2: radosgw-admin(+0x505b9f) [0x5609532fcb9f] 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: 3: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x16a) [0x560953a6826a] 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: 4: main() 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: 5: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f9c55f6ed90] 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: 6: __libc_start_main() 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: 7: _start() 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: 0> 2026-02-24T22:22:42.158+0000 7f9c5382ca80 -1 *** Caught signal (Aborted) ** 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: in thread 7f9c5382ca80 thread_name:radosgw-admin 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: ceph version 20.3.0-4957-gf7e05f4f (f7e05f4fdea5af173284f247d6b0935b337d4ccf) tentacle (dev - None) 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f9c55f87520] 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: 2: pthread_kill() 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: 3: raise() 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: 4: abort() 2026-02-24T22:22:42.230 INFO:tasks.workunit.client.0.trial046.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x196) [0x7f9c56c6ef10] 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 6: radosgw-admin(+0x505b9f) [0x5609532fcb9f] 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 7: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x16a) [0x560953a6826a] 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 8: main() 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 9: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f9c55f6ed90] 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 10: __libc_start_main() 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 11: _start() 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr:--- logging levels --- 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 none 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 lockdep 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 context 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 crush 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_balancer 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_locker 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_log_expire 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mds_migrator 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 3/ 5 mds_quiesce 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 buffer 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 timer 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 filer 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 striper 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 1 objecter 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rados 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_mirror 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_replay 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 rbd_pwl 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 journaler 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objectcacher 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 immutable_obj_cache 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 client 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 osd 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 optracker 2026-02-24T22:22:42.231 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 objclass 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 filestore 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 journal 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 ms 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mon 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 0/10 monc 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 paxos 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 tp 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 auth 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 crypto 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 finisher 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 reserver 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 heartbeatmap 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 perfcounter 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_sync 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_datacache 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_access 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dbstore 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_flight 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_lifecycle 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_restore 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 20/20 rgw_notification 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_bucket_logging 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 javaclient 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 asok 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 1 throttle 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 0 refs 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 compressor 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluestore_compression 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 bluefs 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 3 bdev 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 kstore 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 4/ 5 rocksdb 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 fuse 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 2/ 5 mgr 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mgrc 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 dpdk 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 eventtrace 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 prioritycache 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 test 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephfs_mirror 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cephsqlite 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 crimson_interrupt 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore 2026-02-24T22:22:42.232 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_onode 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_odata 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_omap 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_tm 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_t 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cleaner 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_epm 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_lba 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_fixedkv_tree 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_cache 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_journal 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_device 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 seastore_backref 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 alienstore 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 mclock 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 rgw_dedup 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 cyanstore 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 ceph_exporter 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 memstore 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 1/ 5 trace 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 0/ 5 ceph_dedup 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: -2/-2 (syslog threshold) 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 99/99 (stderr threshold) 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr:--- pthread ID / name mapping for recent threads --- 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 7f9bfcfe1640 / http_manager 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 7f9bfe7e4640 / http_manager 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 7f9c1affd640 / ms_dispatch 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 7f9c2cff9640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 7f9c2effd640 / ms_dispatch 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 7f9c35ffb640 / ms_dispatch 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 7f9c377fe640 / io_context_pool, ms_dispatch, io_context_pool, ms_dispatch 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 7f9c37fff640 / admin_socket 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 7f9c43fff640 / ms_dispatch 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 7f9c50aff640 / msgr-worker-2 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 7f9c51300640 / msgr-worker-1 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 7f9c51b01640 / msgr-worker-0 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 7f9c52302640 / io_context_pool 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: 7f9c5382ca80 / radosgw-admin 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: max_recent 500 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: max_new 1000 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr: log_file /var/lib/ceph/crash/2026-02-24T22:22:42.160452Z_7a4802b3-d48b-49eb-9793-8890369b1fba/log 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr:--- end dump of recent events --- 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr:Aborted 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket a-bucket 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:22:42.233 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:22:42.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.194+0000 7f381e24ca80 4 Realm: () 2026-02-24T22:22:42.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.194+0000 7f381e24ca80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:22:42.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.194+0000 7f381e24ca80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:22:42.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.194+0000 7f381e24ca80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:22:42.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.194+0000 7f381e24ca80 20 zonegroup default 2026-02-24T22:22:42.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.194+0000 7f381e24ca80 20 rados->read ofs=0 len=0 2026-02-24T22:22:42.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.198+0000 7f381e24ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:22:42.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.198+0000 7f381e24ca80 20 rados->read ofs=0 len=0 2026-02-24T22:22:42.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.198+0000 7f381e24ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:22:42.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.198+0000 7f381e24ca80 20 started sync module instance, tier type = 2026-02-24T22:22:42.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.198+0000 7f381e24ca80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:22:42.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.198+0000 7f381e24ca80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:22:42.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.198+0000 7f381e24ca80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:22:42.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.198+0000 7f381e24ca80 5 note: GC not initialized 2026-02-24T22:22:42.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.198+0000 7f37c97e2640 20 reqs_thread_entry: start 2026-02-24T22:22:42.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.202+0000 7f3817fff640 0 -- 10.20.193.46:0/2716362656 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55b44ae89830 msgr2=0x55b44ae8ff90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:22:42.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.210+0000 7f38177fe640 0 -- 10.20.193.46:0/2716362656 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55b44ae8ba10 msgr2=0x55b44ae86780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:22:42.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.234+0000 7f381e24ca80 20 init_complete bucket index max shards: 1 2026-02-24T22:22:42.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.234+0000 7f381e24ca80 20 Filter name: none 2026-02-24T22:22:42.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.234+0000 7f37337fe640 20 reqs_thread_entry: start 2026-02-24T22:22:42.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.234+0000 7f381e24ca80 20 rados->read ofs=0 len=0 2026-02-24T22:22:42.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.234+0000 7f381e24ca80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:22:42.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.234+0000 7f381e24ca80 20 rados->read ofs=0 len=0 2026-02-24T22:22:42.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.234+0000 7f381e24ca80 20 rados_obj.operate() r=0 bl.length=470 2026-02-24T22:22:42.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.234+0000 7f381e24ca80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:22:42.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.234+0000 7f381e24ca80 20 rados->read ofs=0 len=0 2026-02-24T22:22:42.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.234+0000 7f381e24ca80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:22:42.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.234+0000 7f381e24ca80 20 rados->read ofs=0 len=0 2026-02-24T22:22:42.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.234+0000 7f381e24ca80 20 rados_obj.operate() r=0 bl.length=470 2026-02-24T22:22:42.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.234+0000 7f381e24ca80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:22:42.242 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:22:42.242 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:22:42.248 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin reshardlog list --bucket a-bucket 2026-02-24T22:22:42.264 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:22:42.264 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:22:42.280 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.286+0000 7f30f4d0da80 4 Realm: () 2026-02-24T22:22:42.280 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.286+0000 7f30f4d0da80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:22:42.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.286+0000 7f30f4d0da80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:22:42.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.286+0000 7f30f4d0da80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:22:42.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.286+0000 7f30f4d0da80 20 zonegroup default 2026-02-24T22:22:42.281 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.286+0000 7f30f4d0da80 20 rados->read ofs=0 len=0 2026-02-24T22:22:42.282 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.290+0000 7f30f4d0da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:22:42.282 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.290+0000 7f30f4d0da80 20 rados->read ofs=0 len=0 2026-02-24T22:22:42.282 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.290+0000 7f30f4d0da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:22:42.282 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.290+0000 7f30f4d0da80 20 started sync module instance, tier type = 2026-02-24T22:22:42.282 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.290+0000 7f30f4d0da80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:22:42.283 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.290+0000 7f30f2fe2640 0 -- 10.20.193.46:0/3360371271 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x55c45dc1c240 msgr2=0x55c45dc3c6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:22:42.284 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.290+0000 7f30f4d0da80 20 add_watcher() i=1 2026-02-24T22:22:42.284 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.290+0000 7f30f4d0da80 20 add_watcher() i=0 2026-02-24T22:22:42.284 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.290+0000 7f30f4d0da80 20 add_watcher() i=3 2026-02-24T22:22:42.285 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.290+0000 7f30f4d0da80 20 add_watcher() i=5 2026-02-24T22:22:42.285 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.290+0000 7f30f4d0da80 20 add_watcher() i=6 2026-02-24T22:22:42.285 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.290+0000 7f30f4d0da80 20 add_watcher() i=7 2026-02-24T22:22:42.285 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.290+0000 7f30f4d0da80 20 add_watcher() i=4 2026-02-24T22:22:42.286 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.294+0000 7f30f4d0da80 20 add_watcher() i=2 2026-02-24T22:22:42.286 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.294+0000 7f30f4d0da80 2 all 8 watchers are set, enabling cache 2026-02-24T22:22:42.288 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.294+0000 7f30f4d0da80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:22:42.288 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.294+0000 7f30f4d0da80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:22:42.288 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.294+0000 7f30f4d0da80 5 note: GC not initialized 2026-02-24T22:22:42.288 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.294+0000 7f300b7fe640 20 reqs_thread_entry: start 2026-02-24T22:22:42.288 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.294+0000 7f30f2fe2640 0 -- 10.20.193.46:0/4277468458 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55c45dba71f0 msgr2=0x55c45dba1f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:22:42.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.534+0000 7f30f4d0da80 20 init_complete bucket index max shards: 1 2026-02-24T22:22:42.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.534+0000 7f30f4d0da80 20 Filter name: none 2026-02-24T22:22:42.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.534+0000 7f3009ffb640 20 reqs_thread_entry: start 2026-02-24T22:22:42.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.534+0000 7f30f4d0da80 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2026-02-24T22:22:42.528 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.534+0000 7f30f4d0da80 20 rados->read ofs=0 len=0 2026-02-24T22:22:42.530 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.534+0000 7f30f4d0da80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:22:42.530 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.534+0000 7f30f4d0da80 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2026-02-24T22:22:42.530 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.534+0000 7f30f4d0da80 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2026-02-24T22:22:42.530 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.534+0000 7f30f4d0da80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : miss 2026-02-24T22:22:42.530 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.534+0000 7f30f4d0da80 20 rados->read ofs=0 len=0 2026-02-24T22:22:42.530 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.534+0000 7f30f4d0da80 20 rados_obj.operate() r=0 bl.length=470 2026-02-24T22:22:42.530 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.534+0000 7f30f4d0da80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:22:42.530 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.534+0000 7f30f4d0da80 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:22:42.530 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.534+0000 7f30f4d0da80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:22:42.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.534+0000 7f30f4d0da80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:22:42.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.534+0000 7f30f4d0da80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:22:42.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.534+0000 7f30f4d0da80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 2026-02-24T22:22:42.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.534+0000 7f30f4d0da80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 2026-02-24T22:22:42.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.534+0000 7f30f4d0da80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1 2026-02-24T22:22:42.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.534+0000 7f30f4d0da80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1 2026-02-24T22:22:42.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.534+0000 7f30f4d0da80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.2 2026-02-24T22:22:42.531 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.534+0000 7f30f4d0da80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.2 2026-02-24T22:22:42.533 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.538+0000 7f30f4d0da80 20 remove_watcher() i=3 2026-02-24T22:22:42.533 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.538+0000 7f30f4d0da80 2 removed watcher, disabling cache 2026-02-24T22:22:42.533 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.538+0000 7f30f4d0da80 20 remove_watcher() i=0 2026-02-24T22:22:42.534 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.538+0000 7f30f4d0da80 20 remove_watcher() i=1 2026-02-24T22:22:42.534 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.538+0000 7f30f4d0da80 20 remove_watcher() i=6 2026-02-24T22:22:42.534 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.538+0000 7f30f4d0da80 20 remove_watcher() i=4 2026-02-24T22:22:42.534 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.538+0000 7f30f4d0da80 20 remove_watcher() i=2 2026-02-24T22:22:42.534 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.538+0000 7f30f4d0da80 20 remove_watcher() i=5 2026-02-24T22:22:42.534 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:22:42.538+0000 7f30f4d0da80 20 remove_watcher() i=7 2026-02-24T22:23:12.570 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin reshardlog purge --bucket a-bucket 2026-02-24T22:23:12.586 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:12.586 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:12.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.609+0000 7fea05d15a80 4 Realm: () 2026-02-24T22:23:12.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.609+0000 7fea05d15a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:12.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.609+0000 7fea05d15a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:12.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.609+0000 7fea05d15a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:12.605 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.609+0000 7fea05d15a80 20 zonegroup default 2026-02-24T22:23:12.606 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.609+0000 7fea05d15a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:12.606 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.613+0000 7fea05d15a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:12.606 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.613+0000 7fea05d15a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:12.607 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.613+0000 7fea05d15a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:12.607 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.613+0000 7fea05d15a80 20 started sync module instance, tier type = 2026-02-24T22:23:12.607 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.613+0000 7fea05d15a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:12.608 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.613+0000 7fe9feffd640 0 -- 10.20.193.46:0/3874753895 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x556c96535a50 msgr2=0x556c96555ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:12.608 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.613+0000 7fea05d15a80 20 add_watcher() i=3 2026-02-24T22:23:12.609 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.613+0000 7fea05d15a80 20 add_watcher() i=0 2026-02-24T22:23:12.610 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.613+0000 7fea05d15a80 20 add_watcher() i=1 2026-02-24T22:23:12.610 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.613+0000 7fea05d15a80 20 add_watcher() i=2 2026-02-24T22:23:12.610 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.613+0000 7fea05d15a80 20 add_watcher() i=6 2026-02-24T22:23:12.610 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.617+0000 7fea05d15a80 20 add_watcher() i=4 2026-02-24T22:23:12.610 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.617+0000 7fea05d15a80 20 add_watcher() i=5 2026-02-24T22:23:12.610 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.617+0000 7fea05d15a80 20 add_watcher() i=7 2026-02-24T22:23:12.611 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.617+0000 7fea05d15a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:23:12.611 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.617+0000 7fea05d15a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:12.612 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.617+0000 7fea05d15a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:12.612 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.617+0000 7fea05d15a80 5 note: GC not initialized 2026-02-24T22:23:12.612 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.617+0000 7fe91bfff640 20 reqs_thread_entry: start 2026-02-24T22:23:12.612 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.617+0000 7fe9ff7fe640 0 -- 10.20.193.46:0/622109816 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x556c9663f510 msgr2=0x556c9660a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:12.853 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.857+0000 7fea05d15a80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:12.853 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.857+0000 7fea05d15a80 20 Filter name: none 2026-02-24T22:23:12.853 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.857+0000 7fe91a7fc640 20 reqs_thread_entry: start 2026-02-24T22:23:12.853 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.857+0000 7fea05d15a80 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2026-02-24T22:23:12.853 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.857+0000 7fea05d15a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:12.854 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.857+0000 7fea05d15a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:23:12.854 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.857+0000 7fea05d15a80 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2026-02-24T22:23:12.854 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.857+0000 7fea05d15a80 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2026-02-24T22:23:12.854 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.857+0000 7fea05d15a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : miss 2026-02-24T22:23:12.854 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.857+0000 7fea05d15a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:12.854 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.861+0000 7fea05d15a80 20 rados_obj.operate() r=0 bl.length=470 2026-02-24T22:23:12.854 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.861+0000 7fea05d15a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:23:12.854 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.861+0000 7fea05d15a80 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:23:12.854 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.861+0000 7fea05d15a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:23:12.855 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.861+0000 7fea05d15a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:12.855 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.861+0000 7fea05d15a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:23:12.856 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.861+0000 7fe9ddffb640 20 trim reshard logs: write on '.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1' needs retry: Success 2026-02-24T22:23:12.860 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.865+0000 7fea05d15a80 20 remove_watcher() i=3 2026-02-24T22:23:12.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.865+0000 7fea05d15a80 2 removed watcher, disabling cache 2026-02-24T22:23:12.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.865+0000 7fea05d15a80 20 remove_watcher() i=0 2026-02-24T22:23:12.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.865+0000 7fea05d15a80 20 remove_watcher() i=5 2026-02-24T22:23:12.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.865+0000 7fea05d15a80 20 remove_watcher() i=1 2026-02-24T22:23:12.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.865+0000 7fea05d15a80 20 remove_watcher() i=2 2026-02-24T22:23:12.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.865+0000 7fea05d15a80 20 remove_watcher() i=6 2026-02-24T22:23:12.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.865+0000 7fea05d15a80 20 remove_watcher() i=7 2026-02-24T22:23:12.861 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.865+0000 7fea05d15a80 20 remove_watcher() i=4 2026-02-24T22:23:12.867 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin reshardlog list --bucket a-bucket 2026-02-24T22:23:12.882 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:12.882 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:12.898 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.905+0000 7f9e9db68a80 4 Realm: () 2026-02-24T22:23:12.899 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.905+0000 7f9e9db68a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:12.899 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.905+0000 7f9e9db68a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:12.899 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.905+0000 7f9e9db68a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:12.899 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.905+0000 7f9e9db68a80 20 zonegroup default 2026-02-24T22:23:12.899 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.905+0000 7f9e9db68a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:12.900 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.905+0000 7f9e9db68a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:12.900 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.905+0000 7f9e9db68a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:12.901 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.905+0000 7f9e9db68a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:12.901 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.905+0000 7f9e9db68a80 20 started sync module instance, tier type = 2026-02-24T22:23:12.901 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.905+0000 7f9e9db68a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:12.901 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.909+0000 7f9e977fe640 0 -- 10.20.193.46:0/118048076 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x5621f38156f0 msgr2=0x5621f3835b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:12.902 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.909+0000 7f9e9db68a80 20 add_watcher() i=3 2026-02-24T22:23:12.902 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.909+0000 7f9e9db68a80 20 add_watcher() i=1 2026-02-24T22:23:12.902 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.909+0000 7f9e9db68a80 20 add_watcher() i=0 2026-02-24T22:23:12.903 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.909+0000 7f9e9db68a80 20 add_watcher() i=5 2026-02-24T22:23:12.903 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.909+0000 7f9e9db68a80 20 add_watcher() i=2 2026-02-24T22:23:12.903 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.909+0000 7f9e9db68a80 20 add_watcher() i=4 2026-02-24T22:23:12.904 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.909+0000 7f9e9db68a80 20 add_watcher() i=7 2026-02-24T22:23:12.904 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.909+0000 7f9e9db68a80 20 add_watcher() i=6 2026-02-24T22:23:12.904 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.909+0000 7f9e9db68a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:23:12.905 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.909+0000 7f9e9db68a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:12.905 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.909+0000 7f9e9db68a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:12.905 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.909+0000 7f9e9db68a80 5 note: GC not initialized 2026-02-24T22:23:12.905 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.909+0000 7f9db3fff640 20 reqs_thread_entry: start 2026-02-24T22:23:12.906 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:12.913+0000 7f9e977fe640 0 -- 10.20.193.46:0/971264009 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5621f373dd90 msgr2=0x5621f3738b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:13.148 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.153+0000 7f9e9db68a80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:13.148 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.153+0000 7f9e9db68a80 20 Filter name: none 2026-02-24T22:23:13.148 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.153+0000 7f9db27fc640 20 reqs_thread_entry: start 2026-02-24T22:23:13.148 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.153+0000 7f9e9db68a80 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2026-02-24T22:23:13.148 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.153+0000 7f9e9db68a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:13.149 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.153+0000 7f9e9db68a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:23:13.149 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.153+0000 7f9e9db68a80 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2026-02-24T22:23:13.149 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.153+0000 7f9e9db68a80 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2026-02-24T22:23:13.150 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.153+0000 7f9e9db68a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : miss 2026-02-24T22:23:13.150 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.153+0000 7f9e9db68a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:13.150 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.153+0000 7f9e9db68a80 20 rados_obj.operate() r=0 bl.length=470 2026-02-24T22:23:13.150 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.153+0000 7f9e9db68a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:23:13.150 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.153+0000 7f9e9db68a80 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:23:13.150 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.153+0000 7f9e9db68a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:23:13.150 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.153+0000 7f9e9db68a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:13.150 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.153+0000 7f9e9db68a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:23:13.150 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.153+0000 7f9e9db68a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 2026-02-24T22:23:13.150 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.153+0000 7f9e9db68a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 2026-02-24T22:23:13.150 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.153+0000 7f9e9db68a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1 2026-02-24T22:23:13.150 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.153+0000 7f9e9db68a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1 2026-02-24T22:23:13.151 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.153+0000 7f9e9db68a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.2 2026-02-24T22:23:13.151 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.153+0000 7f9e9db68a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.2 2026-02-24T22:23:13.153 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.157+0000 7f9e9db68a80 20 remove_watcher() i=3 2026-02-24T22:23:13.153 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.157+0000 7f9e9db68a80 2 removed watcher, disabling cache 2026-02-24T22:23:13.153 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.157+0000 7f9e9db68a80 20 remove_watcher() i=1 2026-02-24T22:23:13.153 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.157+0000 7f9e9db68a80 20 remove_watcher() i=4 2026-02-24T22:23:13.153 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.157+0000 7f9e9db68a80 20 remove_watcher() i=0 2026-02-24T22:23:13.153 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.157+0000 7f9e9db68a80 20 remove_watcher() i=5 2026-02-24T22:23:13.153 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.157+0000 7f9e9db68a80 20 remove_watcher() i=6 2026-02-24T22:23:13.154 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.157+0000 7f9e9db68a80 20 remove_watcher() i=2 2026-02-24T22:23:13.154 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.157+0000 7f9e9db68a80 20 remove_watcher() i=7 2026-02-24T22:23:13.160 INFO:tasks.workunit.client.0.trial046.stderr:check reshard logrecord successfully 2026-02-24T22:23:13.160 INFO:tasks.workunit.client.0.trial046.stderr: test: reshard versioned bucket 2026-02-24T22:23:13.160 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket myver 2026-02-24T22:23:13.175 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:13.175 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:13.193 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.197+0000 7f766fb51a80 4 Realm: () 2026-02-24T22:23:13.193 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.197+0000 7f766fb51a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:13.193 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.197+0000 7f766fb51a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:13.193 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.197+0000 7f766fb51a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:13.193 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.197+0000 7f766fb51a80 20 zonegroup default 2026-02-24T22:23:13.193 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.197+0000 7f766fb51a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:13.194 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.201+0000 7f766fb51a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:13.195 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.201+0000 7f766fb51a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:13.195 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.201+0000 7f766fb51a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:13.195 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.201+0000 7f766fb51a80 20 started sync module instance, tier type = 2026-02-24T22:23:13.195 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.201+0000 7f766fb51a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:13.197 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.201+0000 7f766fb51a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:13.197 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.201+0000 7f766fb51a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:13.197 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.201+0000 7f766fb51a80 5 note: GC not initialized 2026-02-24T22:23:13.197 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.201+0000 7f761afe5640 20 reqs_thread_entry: start 2026-02-24T22:23:13.200 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.205+0000 7f766de26640 0 -- 10.20.193.46:0/830731518 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x5633f57e7140 msgr2=0x5633f57da3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:13.409 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.413+0000 7f766d625640 0 -- 10.20.193.46:0/830731518 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5633f57e2bc0 msgr2=0x5633f57dd930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:23:13.437 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.441+0000 7f766fb51a80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:13.437 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.441+0000 7f766fb51a80 20 Filter name: none 2026-02-24T22:23:13.437 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.441+0000 7f76197e2640 20 reqs_thread_entry: start 2026-02-24T22:23:13.437 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.441+0000 7f766fb51a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:13.438 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.445+0000 7f766fb51a80 20 rados_obj.operate() r=0 bl.length=201 2026-02-24T22:23:13.438 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.445+0000 7f766fb51a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:13.439 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.445+0000 7f766fb51a80 20 rados_obj.operate() r=0 bl.length=431 2026-02-24T22:23:13.439 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.445+0000 7f766fb51a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:23:13.439 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.445+0000 7f766fb51a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:13.439 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.445+0000 7f766fb51a80 20 rados_obj.operate() r=0 bl.length=201 2026-02-24T22:23:13.439 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.445+0000 7f766fb51a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:13.439 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.445+0000 7f766fb51a80 20 rados_obj.operate() r=0 bl.length=431 2026-02-24T22:23:13.440 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.445+0000 7f766fb51a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:23:13.449 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:23:13.449 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:23:13.449 INFO:tasks.workunit.client.0.trial046.stderr:bucket: myver id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 num_objects: 0 size_kb: 0 num_shards: 1 2026-02-24T22:23:13.449 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:23:13.449 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket myver --num-shards 2 2026-02-24T22:23:13.463 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:13.463 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:13.481 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.485+0000 7f5ea4f57a80 4 Realm: () 2026-02-24T22:23:13.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.485+0000 7f5ea4f57a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:13.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.485+0000 7f5ea4f57a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:13.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.485+0000 7f5ea4f57a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:13.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.485+0000 7f5ea4f57a80 20 zonegroup default 2026-02-24T22:23:13.482 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.489+0000 7f5ea4f57a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:13.483 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.489+0000 7f5ea4f57a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:13.483 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.489+0000 7f5ea4f57a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:13.484 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.489+0000 7f5ea4f57a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:13.484 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.489+0000 7f5ea4f57a80 20 started sync module instance, tier type = 2026-02-24T22:23:13.484 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.489+0000 7f5ea4f57a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:13.485 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.489+0000 7f5ea222a640 0 -- 10.20.193.46:0/934047239 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x55bdbc4957b0 msgr2=0x55bdbc4b5c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:13.486 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.493+0000 7f5ea4f57a80 20 add_watcher() i=1 2026-02-24T22:23:13.486 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.493+0000 7f5ea4f57a80 20 add_watcher() i=3 2026-02-24T22:23:13.487 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.493+0000 7f5ea4f57a80 20 add_watcher() i=0 2026-02-24T22:23:13.487 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.493+0000 7f5ea4f57a80 20 add_watcher() i=6 2026-02-24T22:23:13.487 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.493+0000 7f5ea4f57a80 20 add_watcher() i=5 2026-02-24T22:23:13.487 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.493+0000 7f5ea4f57a80 20 add_watcher() i=7 2026-02-24T22:23:13.487 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.493+0000 7f5ea4f57a80 20 add_watcher() i=2 2026-02-24T22:23:13.487 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.493+0000 7f5ea4f57a80 20 add_watcher() i=4 2026-02-24T22:23:13.488 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.493+0000 7f5ea4f57a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:23:13.488 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.493+0000 7f5ea222a640 0 -- 10.20.193.46:0/2069209166 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f5dc4008af0 msgr2=0x7f5dc4028fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:23:13.692 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.697+0000 7f5ea4f57a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:13.692 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.697+0000 7f5ea4f57a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:13.692 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.697+0000 7f5ea4f57a80 5 note: GC not initialized 2026-02-24T22:23:13.692 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.697+0000 7f5e50fe1640 20 reqs_thread_entry: start 2026-02-24T22:23:13.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.733+0000 7f5ea4f57a80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:13.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.733+0000 7f5ea4f57a80 20 Filter name: none 2026-02-24T22:23:13.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.733+0000 7f5dbaffd640 20 reqs_thread_entry: start 2026-02-24T22:23:13.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.733+0000 7f5ea4f57a80 10 cache get: name=default.rgw.meta+root+myver : miss 2026-02-24T22:23:13.729 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.733+0000 7f5ea4f57a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:13.730 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.733+0000 7f5ea4f57a80 20 rados_obj.operate() r=0 bl.length=201 2026-02-24T22:23:13.730 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.733+0000 7f5ea4f57a80 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2026-02-24T22:23:13.730 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.733+0000 7f5ea4f57a80 10 adding default.rgw.meta+root+myver to cache LRU end 2026-02-24T22:23:13.730 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.733+0000 7f5ea4f57a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 : miss 2026-02-24T22:23:13.730 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.733+0000 7f5ea4f57a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:13.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.737+0000 7f5ea4f57a80 20 rados_obj.operate() r=0 bl.length=431 2026-02-24T22:23:13.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.737+0000 7f5ea4f57a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 info.flags=0x17 2026-02-24T22:23:13.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.737+0000 7f5ea4f57a80 10 adding default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 to cache LRU end 2026-02-24T22:23:13.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.737+0000 7f5ea4f57a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:23:13.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.737+0000 7f5ea4f57a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:13.731 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.737+0000 7f5ea4f57a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 2026-02-24T22:23:13.734 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.741+0000 7f5ea4f57a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 info.flags=0x17 2026-02-24T22:23:13.735 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.741+0000 7f5ea4f57a80 10 moving default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 to cache LRU end 2026-02-24T22:23:13.735 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.741+0000 7f5ea4f57a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:13.735 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.741+0000 7f5ea4f57a80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519, ofs0, ns] 2026-02-24T22:23:13.735 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.741+0000 7f5e7e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959914 cookie 94273396776112 notifier 8724 bl.length()=923 2026-02-24T22:23:13.735 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.741+0000 7f5e7e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 info.flags=0x17 2026-02-24T22:23:13.735 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.741+0000 7f5e7e7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 to cache LRU end 2026-02-24T22:23:13.735 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.741+0000 7f5e7e7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:13.735 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.741+0000 7f5ea4f57a80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): orig_sources= new_sources= 2026-02-24T22:23:13.736 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.741+0000 7f5ea4f57a80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): potential sources added=[] removed=[] 2026-02-24T22:23:13.736 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.741+0000 7f5ea4f57a80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): orig_dests= new_dests= 2026-02-24T22:23:13.736 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.741+0000 7f5ea4f57a80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): potential dests added=[] removed=[] 2026-02-24T22:23:13.736 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.741+0000 7f5ea4f57a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.1.0 2026-02-24T22:23:13.737 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.741+0000 7f5ea4f57a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.1.0 2026-02-24T22:23:13.737 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.741+0000 7f5ea4f57a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.1.1 2026-02-24T22:23:13.737 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.741+0000 7f5ea4f57a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.1.1 2026-02-24T22:23:13.737 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.741+0000 7f5ea4f57a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.0 2026-02-24T22:23:13.737 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.741+0000 7f5ea4f57a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 : hit (requested=0x17, cached=0x17) 2026-02-24T22:23:13.737 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.741+0000 7f5ea4f57a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 2026-02-24T22:23:13.738 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.741+0000 7f5ea4f57a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 info.flags=0x17 2026-02-24T22:23:13.738 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.741+0000 7f5ea4f57a80 10 moving default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 to cache LRU end 2026-02-24T22:23:13.738 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.741+0000 7f5ea4f57a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:13.738 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.741+0000 7f5ea4f57a80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519, ofs0, ns] 2026-02-24T22:23:13.738 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.745+0000 7f5e7effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959915 cookie 94273396776112 notifier 8724 bl.length()=923 2026-02-24T22:23:13.738 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.745+0000 7f5e7effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 info.flags=0x17 2026-02-24T22:23:13.738 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.745+0000 7f5e7effd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 to cache LRU end 2026-02-24T22:23:13.738 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.745+0000 7f5e7effd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:13.739 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.745+0000 7f5ea4f57a80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): orig_sources= new_sources= 2026-02-24T22:23:13.739 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.745+0000 7f5ea4f57a80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): potential sources added=[] removed=[] 2026-02-24T22:23:13.739 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.745+0000 7f5ea4f57a80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): orig_dests= new_dests= 2026-02-24T22:23:13.739 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.745+0000 7f5ea4f57a80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): potential dests added=[] removed=[] 2026-02-24T22:23:13.740 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.745+0000 7f5ea4f57a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.0 2026-02-24T22:23:13.740 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.745+0000 7f5ea4f57a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 : hit (requested=0x17, cached=0x17) 2026-02-24T22:23:13.740 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.745+0000 7f5ea4f57a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 2026-02-24T22:23:13.740 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.745+0000 7f5ea4f57a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 info.flags=0x17 2026-02-24T22:23:13.741 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.745+0000 7f5ea4f57a80 10 moving default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 to cache LRU end 2026-02-24T22:23:13.741 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.745+0000 7f5ea4f57a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:13.741 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.745+0000 7f5ea4f57a80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519, ofs0, ns] 2026-02-24T22:23:13.741 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.745+0000 7f5e7effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959916 cookie 94273396776112 notifier 8724 bl.length()=887 2026-02-24T22:23:13.741 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.745+0000 7f5e7effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 info.flags=0x17 2026-02-24T22:23:13.741 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.745+0000 7f5e7effd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 to cache LRU end 2026-02-24T22:23:13.741 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.745+0000 7f5e7effd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:13.742 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.745+0000 7f5ea4f57a80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): orig_sources= new_sources= 2026-02-24T22:23:13.742 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.745+0000 7f5ea4f57a80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): potential sources added=[] removed=[] 2026-02-24T22:23:13.742 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.745+0000 7f5ea4f57a80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): orig_dests= new_dests= 2026-02-24T22:23:13.742 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.745+0000 7f5ea4f57a80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): potential dests added=[] removed=[] 2026-02-24T22:23:13.742 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.749+0000 7f5ea4f57a80 1 execute INFO: reshard of bucket "myver" from 1 shards to 2 shards completed successfully 2026-02-24T22:23:13.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.749+0000 7f5ea4f57a80 20 remove_watcher() i=3 2026-02-24T22:23:13.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.749+0000 7f5ea4f57a80 2 removed watcher, disabling cache 2026-02-24T22:23:13.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.749+0000 7f5ea4f57a80 20 remove_watcher() i=2 2026-02-24T22:23:13.746 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.749+0000 7f5ea4f57a80 20 remove_watcher() i=1 2026-02-24T22:23:13.746 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.749+0000 7f5ea4f57a80 20 remove_watcher() i=6 2026-02-24T22:23:13.746 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.749+0000 7f5ea4f57a80 20 remove_watcher() i=0 2026-02-24T22:23:13.746 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.749+0000 7f5ea4f57a80 20 remove_watcher() i=5 2026-02-24T22:23:13.746 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.749+0000 7f5ea4f57a80 20 remove_watcher() i=4 2026-02-24T22:23:13.746 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.749+0000 7f5ea4f57a80 20 remove_watcher() i=7 2026-02-24T22:23:13.752 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket myver 2026-02-24T22:23:13.767 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:13.767 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:13.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.789+0000 7fbcd880ca80 4 Realm: () 2026-02-24T22:23:13.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.789+0000 7fbcd880ca80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:13.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.789+0000 7fbcd880ca80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:13.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.789+0000 7fbcd880ca80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:13.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.789+0000 7fbcd880ca80 20 zonegroup default 2026-02-24T22:23:13.786 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.789+0000 7fbcd880ca80 20 rados->read ofs=0 len=0 2026-02-24T22:23:13.787 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.793+0000 7fbcd880ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:13.787 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.793+0000 7fbcd880ca80 20 rados->read ofs=0 len=0 2026-02-24T22:23:13.788 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.793+0000 7fbcd880ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:13.788 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.793+0000 7fbcd880ca80 20 started sync module instance, tier type = 2026-02-24T22:23:13.788 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.793+0000 7fbcd880ca80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:13.789 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.793+0000 7fbcd880ca80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:13.789 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.793+0000 7fbcd880ca80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:13.790 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.793+0000 7fbcd880ca80 5 note: GC not initialized 2026-02-24T22:23:13.790 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.793+0000 7fbbf7fff640 20 reqs_thread_entry: start 2026-02-24T22:23:13.794 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:13.801+0000 7fbcd62e0640 0 -- 10.20.193.46:0/3477826776 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55c2cf4ec720 msgr2=0x55c2cf4ecbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:14.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.013+0000 7fbcd6ae1640 0 -- 10.20.193.46:0/3477826776 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x55c2cf4ddb60 msgr2=0x55c2cf4ddf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:23:14.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.241+0000 7fbcd880ca80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:14.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.241+0000 7fbcd880ca80 20 Filter name: none 2026-02-24T22:23:14.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.241+0000 7fbbf67fc640 20 reqs_thread_entry: start 2026-02-24T22:23:14.234 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.241+0000 7fbcd880ca80 20 rados->read ofs=0 len=0 2026-02-24T22:23:14.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.241+0000 7fbcd880ca80 20 rados_obj.operate() r=0 bl.length=201 2026-02-24T22:23:14.235 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.241+0000 7fbcd880ca80 20 rados->read ofs=0 len=0 2026-02-24T22:23:14.236 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.241+0000 7fbcd880ca80 20 rados_obj.operate() r=0 bl.length=431 2026-02-24T22:23:14.236 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.241+0000 7fbcd880ca80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:23:14.236 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.241+0000 7fbcd880ca80 20 rados->read ofs=0 len=0 2026-02-24T22:23:14.236 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.241+0000 7fbcd880ca80 20 rados_obj.operate() r=0 bl.length=201 2026-02-24T22:23:14.236 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.241+0000 7fbcd880ca80 20 rados->read ofs=0 len=0 2026-02-24T22:23:14.236 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.241+0000 7fbcd880ca80 20 rados_obj.operate() r=0 bl.length=431 2026-02-24T22:23:14.236 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.241+0000 7fbcd880ca80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:23:14.247 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:23:14.247 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:23:14.247 INFO:tasks.workunit.client.0.trial046.stderr:bucket: myver id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 num_objects: 0 size_kb: 0 num_shards: 2 2026-02-24T22:23:14.247 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:23:14.248 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:23:14.248 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:23:14.250 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:23:14.251 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:23:14.294 INFO:tasks.workunit.client.0.trial046.stderr: test: reshard removes olh entries with empty name 2026-02-24T22:23:14.295 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:23:14.296 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:23:14.341 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:23:14.341 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:23:14.382 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket a-bucket 2026-02-24T22:23:14.397 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:14.397 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:14.415 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.421+0000 7fcc1e3b8a80 4 Realm: () 2026-02-24T22:23:14.415 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.421+0000 7fcc1e3b8a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:14.415 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.421+0000 7fcc1e3b8a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:14.415 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.421+0000 7fcc1e3b8a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:14.415 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.421+0000 7fcc1e3b8a80 20 zonegroup default 2026-02-24T22:23:14.415 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.421+0000 7fcc1e3b8a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:14.417 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.421+0000 7fcc1e3b8a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:14.417 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.421+0000 7fcc1e3b8a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:14.417 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.421+0000 7fcc1e3b8a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:14.417 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.421+0000 7fcc1e3b8a80 20 started sync module instance, tier type = 2026-02-24T22:23:14.417 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.421+0000 7fcc1e3b8a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:14.419 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.425+0000 7fcc1e3b8a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:14.419 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.425+0000 7fcc1e3b8a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:14.419 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.425+0000 7fcc1e3b8a80 5 note: GC not initialized 2026-02-24T22:23:14.419 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.425+0000 7fcbc97e2640 20 reqs_thread_entry: start 2026-02-24T22:23:14.422 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.429+0000 7fcc17fff640 0 -- 10.20.193.46:0/164898207 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55ae9da82c20 msgr2=0x55ae9da7c5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:14.632 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.637+0000 7fcc177fe640 0 -- 10.20.193.46:0/164898207 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55ae9da57240 msgr2=0x55ae9da84dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:23:14.660 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.665+0000 7fcc1e3b8a80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:14.660 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.665+0000 7fcc1e3b8a80 20 Filter name: none 2026-02-24T22:23:14.660 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.665+0000 7fcb337fe640 20 reqs_thread_entry: start 2026-02-24T22:23:14.661 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.665+0000 7fcc1e3b8a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:14.661 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.665+0000 7fcc1e3b8a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:23:14.661 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.665+0000 7fcc1e3b8a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:14.661 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.665+0000 7fcc1e3b8a80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:23:14.662 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.665+0000 7fcc1e3b8a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:23:14.662 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.665+0000 7fcc1e3b8a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:14.662 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.669+0000 7fcc1e3b8a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:23:14.662 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.669+0000 7fcc1e3b8a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:14.662 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.669+0000 7fcc1e3b8a80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:23:14.662 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.669+0000 7fcc1e3b8a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:23:14.673 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:23:14.673 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:23:14.673 INFO:tasks.workunit.client.0.trial046.stderr:bucket: a-bucket id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 num_objects: 0 size_kb: 0 num_shards: 3 2026-02-24T22:23:14.673 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:23:14.673 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket layout --bucket a-bucket 2026-02-24T22:23:14.687 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:14.687 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:14.704 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.709+0000 7f5e525f3a80 4 Realm: () 2026-02-24T22:23:14.705 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.709+0000 7f5e525f3a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:14.705 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.709+0000 7f5e525f3a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:14.705 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.709+0000 7f5e525f3a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:14.705 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.709+0000 7f5e525f3a80 20 zonegroup default 2026-02-24T22:23:14.705 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.709+0000 7f5e525f3a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:14.706 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.713+0000 7f5e525f3a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:14.706 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.713+0000 7f5e525f3a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:14.706 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.713+0000 7f5e525f3a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:14.707 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.713+0000 7f5e525f3a80 20 started sync module instance, tier type = 2026-02-24T22:23:14.707 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.713+0000 7f5e525f3a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:14.708 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.713+0000 7f5e525f3a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:14.708 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.713+0000 7f5e525f3a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:14.708 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.713+0000 7f5e525f3a80 5 note: GC not initialized 2026-02-24T22:23:14.708 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.713+0000 7f5dfdfe3640 20 reqs_thread_entry: start 2026-02-24T22:23:14.710 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.717+0000 7f5e4b7fe640 0 -- 10.20.193.46:0/551079400 >> [v2:10.20.193.46:6826/3065971976,v1:10.20.193.46:6827/3065971976] conn(0x55a86683f560 msgr2=0x55a86683f950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:14.715 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.721+0000 7f5e4bfff640 0 -- 10.20.193.46:0/551079400 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55a86683bd20 msgr2=0x55a86683c190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:14.740 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.745+0000 7f5e525f3a80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:14.740 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.745+0000 7f5e525f3a80 20 Filter name: none 2026-02-24T22:23:14.740 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.745+0000 7f5d5ffff640 20 reqs_thread_entry: start 2026-02-24T22:23:14.740 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.745+0000 7f5e525f3a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:14.741 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.745+0000 7f5e525f3a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:23:14.741 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.745+0000 7f5e525f3a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:14.741 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.745+0000 7f5e525f3a80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:23:14.741 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.745+0000 7f5e525f3a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:23:14.748 INFO:tasks.workunit.client.0.trial046.stderr:rados -p default.rgw.buckets.index setomapval .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 --omap-key-file /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/olh_noname_key < /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/olh_noname_val 2026-02-24T22:23:14.761 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:14.762 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:14.773 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bi list --bucket a-bucket 2026-02-24T22:23:14.787 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:14.787 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:14.803 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.809+0000 7fad4d27fa80 4 Realm: () 2026-02-24T22:23:14.803 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.809+0000 7fad4d27fa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:14.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.809+0000 7fad4d27fa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:14.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.809+0000 7fad4d27fa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:14.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.809+0000 7fad4d27fa80 20 zonegroup default 2026-02-24T22:23:14.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.809+0000 7fad4d27fa80 20 rados->read ofs=0 len=0 2026-02-24T22:23:14.805 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.813+0000 7fad4d27fa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:14.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.813+0000 7fad4d27fa80 20 rados->read ofs=0 len=0 2026-02-24T22:23:14.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.813+0000 7fad4d27fa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:14.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.813+0000 7fad4d27fa80 20 started sync module instance, tier type = 2026-02-24T22:23:14.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.813+0000 7fad4d27fa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:14.808 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.813+0000 7fad4d27fa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:14.808 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.813+0000 7fad4d27fa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:14.808 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.813+0000 7fad4d27fa80 5 note: GC not initialized 2026-02-24T22:23:14.808 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.813+0000 7facf97e2640 20 reqs_thread_entry: start 2026-02-24T22:23:14.812 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:14.817+0000 7fad4ad53640 0 -- 10.20.193.46:0/2975909879 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x555bca4cef50 msgr2=0x555bca4cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:23:15.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.029+0000 7fad4b554640 0 -- 10.20.193.46:0/2975909879 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x555bca4bf0b0 msgr2=0x555bca4c08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:15.250 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.257+0000 7fad4d27fa80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:15.251 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.257+0000 7fad4d27fa80 20 Filter name: none 2026-02-24T22:23:15.251 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.257+0000 7fac637fe640 20 reqs_thread_entry: start 2026-02-24T22:23:15.251 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.257+0000 7fad4d27fa80 20 rados->read ofs=0 len=0 2026-02-24T22:23:15.251 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.257+0000 7fad4d27fa80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:23:15.252 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.257+0000 7fad4d27fa80 20 rados->read ofs=0 len=0 2026-02-24T22:23:15.252 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.257+0000 7fad4d27fa80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:23:15.252 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.257+0000 7fad4d27fa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:23:15.252 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.257+0000 7fad4d27fa80 20 INFO: main: max_entries=1000, index=gen=2, max_shards=3 2026-02-24T22:23:15.252 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.257+0000 7fad4d27fa80 20 INFO: main: starting shard=0 2026-02-24T22:23:15.252 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.257+0000 7fad4d27fa80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 2026-02-24T22:23:15.252 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.257+0000 7fad4d27fa80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 2026-02-24T22:23:15.253 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.257+0000 7fad4d27fa80 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, next_marker=�1001_04/57/0457f727ec113e418d5b16d206b200ed068c0533554883ce811df7c932a3df68/2018_12_11/2889999/3386469/metadata.gz 2026-02-24T22:23:15.253 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.257+0000 7fad4d27fa80 20 INFO: main: starting shard=1 2026-02-24T22:23:15.253 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.257+0000 7fad4d27fa80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1 2026-02-24T22:23:15.253 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.257+0000 7fad4d27fa80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1 2026-02-24T22:23:15.253 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.257+0000 7fad4d27fa80 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, next_marker= 2026-02-24T22:23:15.253 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.257+0000 7fad4d27fa80 20 INFO: main: starting shard=2 2026-02-24T22:23:15.253 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.257+0000 7fad4d27fa80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.2 2026-02-24T22:23:15.253 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.257+0000 7fad4d27fa80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.2 2026-02-24T22:23:15.254 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.257+0000 7fad4d27fa80 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, next_marker= 2026-02-24T22:23:15.254 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.257+0000 7fad4d27fa80 20 INFO: main: done 2026-02-24T22:23:15.261 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket a-bucket --num-shards 1 --yes-i-really-mean-it 2026-02-24T22:23:15.276 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:15.276 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:15.293 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.297+0000 7f09a2480a80 4 Realm: () 2026-02-24T22:23:15.293 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.297+0000 7f09a2480a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:15.293 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.297+0000 7f09a2480a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:15.294 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.297+0000 7f09a2480a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:15.294 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.297+0000 7f09a2480a80 20 zonegroup default 2026-02-24T22:23:15.294 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.301+0000 7f09a2480a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:15.295 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.301+0000 7f09a2480a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:15.295 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.301+0000 7f09a2480a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:15.295 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.301+0000 7f09a2480a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:15.295 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.301+0000 7f09a2480a80 20 started sync module instance, tier type = 2026-02-24T22:23:15.295 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.301+0000 7f09a2480a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:15.296 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.301+0000 7f099b7fe640 0 -- 10.20.193.46:0/3830423994 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x55f745ac7310 msgr2=0x55f745ae7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:15.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.301+0000 7f09a2480a80 20 add_watcher() i=1 2026-02-24T22:23:15.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.301+0000 7f09a2480a80 20 add_watcher() i=3 2026-02-24T22:23:15.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.301+0000 7f09a2480a80 20 add_watcher() i=6 2026-02-24T22:23:15.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.301+0000 7f09a2480a80 20 add_watcher() i=0 2026-02-24T22:23:15.297 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.301+0000 7f09a2480a80 20 add_watcher() i=5 2026-02-24T22:23:15.298 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.305+0000 7f09a2480a80 20 add_watcher() i=4 2026-02-24T22:23:15.298 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.305+0000 7f09a2480a80 20 add_watcher() i=7 2026-02-24T22:23:15.501 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.509+0000 7f09a2480a80 20 add_watcher() i=2 2026-02-24T22:23:15.502 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.509+0000 7f09a2480a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:23:15.502 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.509+0000 7f099affd640 0 -- 10.20.193.46:0/3338627205 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f08c0008ac0 msgr2=0x7f08c0028f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:23:15.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.713+0000 7f09a2480a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:15.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.713+0000 7f09a2480a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:15.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.713+0000 7f09a2480a80 5 note: GC not initialized 2026-02-24T22:23:15.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.713+0000 7f094cfe1640 20 reqs_thread_entry: start 2026-02-24T22:23:15.744 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.749+0000 7f09a2480a80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:15.744 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.749+0000 7f09a2480a80 20 Filter name: none 2026-02-24T22:23:15.744 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.749+0000 7f08b6ffd640 20 reqs_thread_entry: start 2026-02-24T22:23:15.744 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.749+0000 7f09a2480a80 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2026-02-24T22:23:15.744 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.749+0000 7f09a2480a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:15.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.749+0000 7f09a2480a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:23:15.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.749+0000 7f09a2480a80 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2026-02-24T22:23:15.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.749+0000 7f09a2480a80 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2026-02-24T22:23:15.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.749+0000 7f09a2480a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : miss 2026-02-24T22:23:15.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.749+0000 7f09a2480a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:15.745 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.749+0000 7f09a2480a80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:23:15.746 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.749+0000 7f09a2480a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:23:15.746 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.749+0000 7f09a2480a80 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:23:15.746 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.749+0000 7f09a2480a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:23:15.746 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.749+0000 7f09a2480a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:15.746 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.749+0000 7f09a2480a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:23:15.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.753+0000 7f09a2480a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:23:15.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.753+0000 7f09a2480a80 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:23:15.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.753+0000 7f09a2480a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:15.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.753+0000 7f09a2480a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518, ofs0, ns] 2026-02-24T22:23:15.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.753+0000 7f0976ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959950 cookie 94520515432576 notifier 8237 bl.length()=929 2026-02-24T22:23:15.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.753+0000 7f0976ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:23:15.749 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.753+0000 7f0976ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:23:15.750 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.753+0000 7f0976ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:15.750 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.753+0000 7f09a2480a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_sources= new_sources= 2026-02-24T22:23:15.750 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.753+0000 7f09a2480a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential sources added=[] removed=[] 2026-02-24T22:23:15.750 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.753+0000 7f09a2480a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_dests= new_dests= 2026-02-24T22:23:15.750 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.753+0000 7f09a2480a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential dests added=[] removed=[] 2026-02-24T22:23:15.751 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.757+0000 7f09a2480a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.0 2026-02-24T22:23:15.751 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.757+0000 7f09a2480a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.0 2026-02-24T22:23:15.751 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.757+0000 7f09a2480a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 2026-02-24T22:23:15.751 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.757+0000 7f09a2480a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 2026-02-24T22:23:15.752 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.757+0000 7f09a2480a80 10 Dropping entry with empty name, idx=�1001_04/57/0457f727ec113e418d5b16d206b200ed068c0533554883ce811df7c932a3df68/2018_12_11/2889999/3386469/metadata.gz 2026-02-24T22:23:15.752 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.757+0000 7f09a2480a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1 2026-02-24T22:23:15.752 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.757+0000 7f09a2480a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1 2026-02-24T22:23:15.752 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.757+0000 7f09a2480a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.2 2026-02-24T22:23:15.752 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.757+0000 7f09a2480a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.2 2026-02-24T22:23:15.752 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.757+0000 7f09a2480a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : hit (requested=0x17, cached=0x17) 2026-02-24T22:23:15.753 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.757+0000 7f09a2480a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:23:15.753 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.757+0000 7f09a2480a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:23:15.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.757+0000 7f09a2480a80 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:23:15.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.757+0000 7f09a2480a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:15.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.757+0000 7f09a2480a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518, ofs0, ns] 2026-02-24T22:23:15.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.757+0000 7f0976ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959951 cookie 94520515432576 notifier 8237 bl.length()=929 2026-02-24T22:23:15.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.757+0000 7f0976ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:23:15.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.757+0000 7f0976ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:23:15.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.757+0000 7f0976ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:15.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.757+0000 7f09a2480a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_sources= new_sources= 2026-02-24T22:23:15.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.757+0000 7f09a2480a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential sources added=[] removed=[] 2026-02-24T22:23:15.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.757+0000 7f09a2480a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_dests= new_dests= 2026-02-24T22:23:15.754 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.757+0000 7f09a2480a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential dests added=[] removed=[] 2026-02-24T22:23:15.755 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.761+0000 7f09a2480a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 2026-02-24T22:23:15.755 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.761+0000 7f09a2480a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.0 2026-02-24T22:23:15.755 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.761+0000 7f09a2480a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1 2026-02-24T22:23:15.755 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.761+0000 7f09a2480a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.1 2026-02-24T22:23:15.756 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.761+0000 7f09a2480a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.2 2026-02-24T22:23:15.756 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.761+0000 7f09a2480a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.2.2 2026-02-24T22:23:15.756 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.761+0000 7f09a2480a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 : hit (requested=0x17, cached=0x17) 2026-02-24T22:23:15.756 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.761+0000 7f09a2480a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 2026-02-24T22:23:15.757 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.761+0000 7f09a2480a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:23:15.757 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.761+0000 7f09a2480a80 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:23:15.757 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.761+0000 7f09a2480a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:15.757 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.761+0000 7f09a2480a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518, ofs0, ns] 2026-02-24T22:23:15.757 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.761+0000 7f0976ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959952 cookie 94520515432576 notifier 8237 bl.length()=893 2026-02-24T22:23:15.757 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.761+0000 7f0976ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 info.flags=0x17 2026-02-24T22:23:15.757 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.761+0000 7f0976ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518 to cache LRU end 2026-02-24T22:23:15.757 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.761+0000 7f0976ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:15.758 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.761+0000 7f09a2480a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_sources= new_sources= 2026-02-24T22:23:15.758 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.761+0000 7f09a2480a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential sources added=[] removed=[] 2026-02-24T22:23:15.758 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.761+0000 7f09a2480a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): orig_dests= new_dests= 2026-02-24T22:23:15.758 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.761+0000 7f09a2480a80 20 handle_bi_update(): bucket=:a-bucket[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518]): potential dests added=[] removed=[] 2026-02-24T22:23:15.758 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.761+0000 7f09a2480a80 1 execute INFO: reshard of bucket "a-bucket" from 3 shards to 1 shards completed successfully 2026-02-24T22:23:15.761 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.765+0000 7f09a2480a80 20 remove_watcher() i=2 2026-02-24T22:23:15.761 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.765+0000 7f09a2480a80 2 removed watcher, disabling cache 2026-02-24T22:23:15.761 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.765+0000 7f09a2480a80 20 remove_watcher() i=1 2026-02-24T22:23:15.761 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.765+0000 7f09a2480a80 20 remove_watcher() i=3 2026-02-24T22:23:15.761 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.765+0000 7f09a2480a80 20 remove_watcher() i=0 2026-02-24T22:23:15.761 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.765+0000 7f09a2480a80 20 remove_watcher() i=4 2026-02-24T22:23:15.761 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.765+0000 7f09a2480a80 20 remove_watcher() i=5 2026-02-24T22:23:15.761 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.765+0000 7f09a2480a80 20 remove_watcher() i=6 2026-02-24T22:23:15.761 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.765+0000 7f09a2480a80 20 remove_watcher() i=7 2026-02-24T22:23:15.767 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bi list --bucket a-bucket 2026-02-24T22:23:15.782 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:15.782 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:15.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.805+0000 7fdb67b72a80 4 Realm: () 2026-02-24T22:23:15.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.805+0000 7fdb67b72a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:15.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.805+0000 7fdb67b72a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:15.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.805+0000 7fdb67b72a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:15.800 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.805+0000 7fdb67b72a80 20 zonegroup default 2026-02-24T22:23:15.801 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.805+0000 7fdb67b72a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:15.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.809+0000 7fdb67b72a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:15.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.809+0000 7fdb67b72a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:15.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.809+0000 7fdb67b72a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:15.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.809+0000 7fdb67b72a80 20 started sync module instance, tier type = 2026-02-24T22:23:15.803 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.809+0000 7fdb67b72a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:15.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.809+0000 7fdb67b72a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:15.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.809+0000 7fdb67b72a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:15.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.809+0000 7fdb67b72a80 5 note: GC not initialized 2026-02-24T22:23:15.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.809+0000 7fdb12fe5640 20 reqs_thread_entry: start 2026-02-24T22:23:15.808 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.813+0000 7fdb65e47640 0 -- 10.20.193.46:0/3501555237 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55eedeeb8f70 msgr2=0x55eedeeb9370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:23:15.814 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.821+0000 7fdb65646640 0 -- 10.20.193.46:0/3501555237 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x7fdb3809d450 msgr2=0x7fdb380bd850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:15.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.841+0000 7fdb67b72a80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:15.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.841+0000 7fdb67b72a80 20 Filter name: none 2026-02-24T22:23:15.837 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.841+0000 7fdb117e2640 20 reqs_thread_entry: start 2026-02-24T22:23:15.838 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.841+0000 7fdb67b72a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:15.839 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.845+0000 7fdb67b72a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:23:15.839 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.845+0000 7fdb67b72a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:15.839 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.845+0000 7fdb67b72a80 20 rados_obj.operate() r=0 bl.length=434 2026-02-24T22:23:15.839 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.845+0000 7fdb67b72a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:23:15.839 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.845+0000 7fdb67b72a80 20 INFO: main: max_entries=1000, index=gen=3, max_shards=1 2026-02-24T22:23:15.839 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.845+0000 7fdb67b72a80 20 INFO: main: starting shard=0 2026-02-24T22:23:15.839 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.845+0000 7fdb67b72a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.0 2026-02-24T22:23:15.839 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.845+0000 7fdb67b72a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518.3.0 2026-02-24T22:23:15.839 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.845+0000 7fdb67b72a80 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, next_marker= 2026-02-24T22:23:15.839 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.845+0000 7fdb67b72a80 20 INFO: main: done 2026-02-24T22:23:15.846 INFO:tasks.workunit.client.0.trial046.stderr: test: PUT succeeds during reshard 2026-02-24T22:23:15.846 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket myver 2026-02-24T22:23:15.861 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:15.861 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:15.878 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.885+0000 7feaff2daa80 4 Realm: () 2026-02-24T22:23:15.878 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.885+0000 7feaff2daa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:15.878 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.885+0000 7feaff2daa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:15.878 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.885+0000 7feaff2daa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:15.878 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.885+0000 7feaff2daa80 20 zonegroup default 2026-02-24T22:23:15.878 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.885+0000 7feaff2daa80 20 rados->read ofs=0 len=0 2026-02-24T22:23:15.879 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.885+0000 7feaff2daa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:15.879 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.885+0000 7feaff2daa80 20 rados->read ofs=0 len=0 2026-02-24T22:23:15.880 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.885+0000 7feaff2daa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:15.880 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.885+0000 7feaff2daa80 20 started sync module instance, tier type = 2026-02-24T22:23:15.880 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.885+0000 7feaff2daa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:15.881 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.885+0000 7feaff2daa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:15.882 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.885+0000 7feaff2daa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:15.882 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.885+0000 7feaff2daa80 5 note: GC not initialized 2026-02-24T22:23:15.882 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.889+0000 7feaaa7e4640 20 reqs_thread_entry: start 2026-02-24T22:23:15.885 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.889+0000 7feafd5af640 0 -- 10.20.193.46:0/2295733588 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55bb23a7f360 msgr2=0x55bb23a85ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:15.891 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.897+0000 7feafcdae640 0 -- 10.20.193.46:0/2295733588 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55bb23a53960 msgr2=0x55bb23a81530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:15.916 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.921+0000 7feaff2daa80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:15.916 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.921+0000 7feaff2daa80 20 Filter name: none 2026-02-24T22:23:15.916 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.921+0000 7feaa8fe1640 20 reqs_thread_entry: start 2026-02-24T22:23:15.916 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.921+0000 7feaff2daa80 20 rados->read ofs=0 len=0 2026-02-24T22:23:15.917 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.921+0000 7feaff2daa80 20 rados_obj.operate() r=0 bl.length=201 2026-02-24T22:23:15.917 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.921+0000 7feaff2daa80 20 rados->read ofs=0 len=0 2026-02-24T22:23:15.918 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.921+0000 7feaff2daa80 20 rados_obj.operate() r=0 bl.length=431 2026-02-24T22:23:15.918 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.921+0000 7feaff2daa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:23:15.918 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.921+0000 7feaff2daa80 20 rados->read ofs=0 len=0 2026-02-24T22:23:15.918 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.925+0000 7feaff2daa80 20 rados_obj.operate() r=0 bl.length=201 2026-02-24T22:23:15.918 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.925+0000 7feaff2daa80 20 rados->read ofs=0 len=0 2026-02-24T22:23:15.918 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.925+0000 7feaff2daa80 20 rados_obj.operate() r=0 bl.length=431 2026-02-24T22:23:15.918 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.925+0000 7feaff2daa80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:23:15.928 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:23:15.928 INFO:tasks.workunit.client.0.trial046.stderr:BUCKET_STATS: 2026-02-24T22:23:15.929 INFO:tasks.workunit.client.0.trial046.stderr:bucket: myver id: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 num_objects: 0 size_kb: 0 num_shards: 2 2026-02-24T22:23:15.929 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:23:15.929 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin --inject-delay-at=do_reshard --inject-delay-ms=5000 bucket reshard --bucket myver --num-shards 3 2026-02-24T22:23:15.944 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:15.944 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:15.961 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.965+0000 7fbeca4dba80 4 Realm: () 2026-02-24T22:23:15.961 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.965+0000 7fbeca4dba80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:15.961 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.965+0000 7fbeca4dba80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:15.961 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.965+0000 7fbeca4dba80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:15.962 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.965+0000 7fbeca4dba80 20 zonegroup default 2026-02-24T22:23:15.962 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.969+0000 7fbeca4dba80 20 rados->read ofs=0 len=0 2026-02-24T22:23:15.963 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.969+0000 7fbeca4dba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:15.963 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.969+0000 7fbeca4dba80 20 rados->read ofs=0 len=0 2026-02-24T22:23:15.963 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.969+0000 7fbeca4dba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:15.963 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.969+0000 7fbeca4dba80 20 started sync module instance, tier type = 2026-02-24T22:23:15.963 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.969+0000 7fbeca4dba80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:15.964 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.969+0000 7fbeca4dba80 20 add_watcher() i=1 2026-02-24T22:23:15.965 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.969+0000 7fbec37fe640 0 -- 10.20.193.46:0/4200849716 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x557948580210 msgr2=0x5579485a0690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:15.965 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.969+0000 7fbeca4dba80 20 add_watcher() i=3 2026-02-24T22:23:15.965 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.969+0000 7fbeca4dba80 20 add_watcher() i=0 2026-02-24T22:23:15.965 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.969+0000 7fbeca4dba80 20 add_watcher() i=5 2026-02-24T22:23:15.966 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.969+0000 7fbeca4dba80 20 add_watcher() i=6 2026-02-24T22:23:15.966 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.973+0000 7fbeca4dba80 20 add_watcher() i=7 2026-02-24T22:23:15.966 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.973+0000 7fbeca4dba80 20 add_watcher() i=2 2026-02-24T22:23:15.967 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.973+0000 7fbeca4dba80 20 add_watcher() i=4 2026-02-24T22:23:15.967 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.973+0000 7fbeca4dba80 2 all 8 watchers are set, enabling cache 2026-02-24T22:23:15.968 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.973+0000 7fbeca4dba80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:15.968 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.973+0000 7fbeca4dba80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:15.968 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.973+0000 7fbeca4dba80 5 note: GC not initialized 2026-02-24T22:23:15.968 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.973+0000 7fbe74fe1640 20 reqs_thread_entry: start 2026-02-24T22:23:15.970 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:15.977+0000 7fbec3fff640 0 -- 10.20.193.46:0/2794952865 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x557948655050 msgr2=0x557948657270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:16.002 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.005+0000 7fbeca4dba80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:16.002 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.005+0000 7fbeca4dba80 20 Filter name: none 2026-02-24T22:23:16.002 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.009+0000 7fbddeffd640 20 reqs_thread_entry: start 2026-02-24T22:23:16.002 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.009+0000 7fbeca4dba80 10 cache get: name=default.rgw.meta+root+myver : miss 2026-02-24T22:23:16.002 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.009+0000 7fbeca4dba80 20 rados->read ofs=0 len=0 2026-02-24T22:23:16.003 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.009+0000 7fbeca4dba80 20 rados_obj.operate() r=0 bl.length=201 2026-02-24T22:23:16.003 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.009+0000 7fbeca4dba80 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2026-02-24T22:23:16.003 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.009+0000 7fbeca4dba80 10 adding default.rgw.meta+root+myver to cache LRU end 2026-02-24T22:23:16.003 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.009+0000 7fbeca4dba80 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 : miss 2026-02-24T22:23:16.003 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.009+0000 7fbeca4dba80 20 rados->read ofs=0 len=0 2026-02-24T22:23:16.003 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.009+0000 7fbeca4dba80 20 rados_obj.operate() r=0 bl.length=431 2026-02-24T22:23:16.003 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.009+0000 7fbeca4dba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 info.flags=0x17 2026-02-24T22:23:16.003 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.009+0000 7fbeca4dba80 10 adding default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 to cache LRU end 2026-02-24T22:23:16.004 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.009+0000 7fbeca4dba80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:23:16.004 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.009+0000 7fbeca4dba80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:16.004 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.009+0000 7fbeca4dba80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 2026-02-24T22:23:16.007 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.013+0000 7fbeca4dba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 info.flags=0x17 2026-02-24T22:23:16.007 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.013+0000 7fbeca4dba80 10 moving default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 to cache LRU end 2026-02-24T22:23:16.007 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.013+0000 7fbeca4dba80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:16.007 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.013+0000 7fbeca4dba80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519, ofs0, ns] 2026-02-24T22:23:16.007 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.013+0000 7fbe9effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959917 cookie 93979394208880 notifier 8880 bl.length()=923 2026-02-24T22:23:16.007 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.013+0000 7fbe9effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 info.flags=0x17 2026-02-24T22:23:16.007 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.013+0000 7fbe9effd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 to cache LRU end 2026-02-24T22:23:16.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.013+0000 7fbe9effd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:16.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.013+0000 7fbeca4dba80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): orig_sources= new_sources= 2026-02-24T22:23:16.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.013+0000 7fbeca4dba80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): potential sources added=[] removed=[] 2026-02-24T22:23:16.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.013+0000 7fbeca4dba80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): orig_dests= new_dests= 2026-02-24T22:23:16.008 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.013+0000 7fbeca4dba80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): potential dests added=[] removed=[] 2026-02-24T22:23:16.009 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:16.013+0000 7fbeca4dba80 -1 Injecting delay=5.000000000s at location=do_reshard 2026-02-24T22:23:16.931 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:23:16.931 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:23:16.942 INFO:tasks.workunit.client.0.trial046.stderr:put object successful 2026-02-24T22:23:21.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.013+0000 7fbeca4dba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.0 2026-02-24T22:23:21.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.013+0000 7fbeca4dba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.0 2026-02-24T22:23:21.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.013+0000 7fbeca4dba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.1 2026-02-24T22:23:21.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.013+0000 7fbeca4dba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.1 2026-02-24T22:23:21.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.013+0000 7fbeca4dba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.2 2026-02-24T22:23:21.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.013+0000 7fbeca4dba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.2 2026-02-24T22:23:21.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.013+0000 7fbeca4dba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.1.0 2026-02-24T22:23:21.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.013+0000 7fbeca4dba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.1.0 2026-02-24T22:23:21.010 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.017+0000 7fbeca4dba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.1.1 2026-02-24T22:23:21.011 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.017+0000 7fbeca4dba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.1.1 2026-02-24T22:23:21.011 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.017+0000 7fbeca4dba80 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 : hit (requested=0x17, cached=0x17) 2026-02-24T22:23:21.011 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.017+0000 7fbeca4dba80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 2026-02-24T22:23:21.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.017+0000 7fbeca4dba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 info.flags=0x17 2026-02-24T22:23:21.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.017+0000 7fbeca4dba80 10 moving default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 to cache LRU end 2026-02-24T22:23:21.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.017+0000 7fbeca4dba80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:21.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.017+0000 7fbeca4dba80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519, ofs0, ns] 2026-02-24T22:23:21.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.017+0000 7fbe9e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959918 cookie 93979394208880 notifier 8880 bl.length()=923 2026-02-24T22:23:21.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.017+0000 7fbe9e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 info.flags=0x17 2026-02-24T22:23:21.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.017+0000 7fbe9e7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 to cache LRU end 2026-02-24T22:23:21.013 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.017+0000 7fbe9e7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:21.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.017+0000 7fbeca4dba80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): orig_sources= new_sources= 2026-02-24T22:23:21.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.017+0000 7fbeca4dba80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): potential sources added=[] removed=[] 2026-02-24T22:23:21.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.017+0000 7fbeca4dba80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): orig_dests= new_dests= 2026-02-24T22:23:21.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.017+0000 7fbeca4dba80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): potential dests added=[] removed=[] 2026-02-24T22:23:21.014 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.021+0000 7fbeca4dba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.1.0 2026-02-24T22:23:21.015 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.021+0000 7fbeca4dba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.1.0 2026-02-24T22:23:21.015 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.021+0000 7fbeca4dba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.1.1 2026-02-24T22:23:21.015 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.021+0000 7fbeca4dba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.1.1 2026-02-24T22:23:21.015 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.021+0000 7fbeca4dba80 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 : hit (requested=0x17, cached=0x17) 2026-02-24T22:23:21.016 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.021+0000 7fbeca4dba80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 2026-02-24T22:23:21.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.021+0000 7fbeca4dba80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 info.flags=0x17 2026-02-24T22:23:21.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.021+0000 7fbeca4dba80 10 moving default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 to cache LRU end 2026-02-24T22:23:21.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.021+0000 7fbeca4dba80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:21.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.021+0000 7fbeca4dba80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519, ofs0, ns] 2026-02-24T22:23:21.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.021+0000 7fbe9effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959919 cookie 93979394208880 notifier 8880 bl.length()=887 2026-02-24T22:23:21.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.021+0000 7fbe9effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 info.flags=0x17 2026-02-24T22:23:21.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.021+0000 7fbe9effd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 to cache LRU end 2026-02-24T22:23:21.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.021+0000 7fbe9effd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:21.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.021+0000 7fbeca4dba80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): orig_sources= new_sources= 2026-02-24T22:23:21.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.021+0000 7fbeca4dba80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): potential sources added=[] removed=[] 2026-02-24T22:23:21.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.021+0000 7fbeca4dba80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): orig_dests= new_dests= 2026-02-24T22:23:21.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.021+0000 7fbeca4dba80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): potential dests added=[] removed=[] 2026-02-24T22:23:21.019 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.025+0000 7fbeca4dba80 1 execute INFO: reshard of bucket "myver" from 2 shards to 3 shards completed successfully 2026-02-24T22:23:21.022 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.029+0000 7fbeca4dba80 20 remove_watcher() i=3 2026-02-24T22:23:21.022 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.029+0000 7fbeca4dba80 2 removed watcher, disabling cache 2026-02-24T22:23:21.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.029+0000 7fbeca4dba80 20 remove_watcher() i=1 2026-02-24T22:23:21.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.029+0000 7fbeca4dba80 20 remove_watcher() i=4 2026-02-24T22:23:21.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.029+0000 7fbeca4dba80 20 remove_watcher() i=0 2026-02-24T22:23:21.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.029+0000 7fbeca4dba80 20 remove_watcher() i=6 2026-02-24T22:23:21.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.029+0000 7fbeca4dba80 20 remove_watcher() i=5 2026-02-24T22:23:21.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.029+0000 7fbeca4dba80 20 remove_watcher() i=7 2026-02-24T22:23:21.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:21.029+0000 7fbeca4dba80 20 remove_watcher() i=2 2026-02-24T22:23:21.947 INFO:tasks.workunit.client.0.trial046.stderr:TEST: check that bucket stats are correct after reshard with unlinked entries 2026-02-24T22:23:21.947 INFO:tasks.workunit.client.0.trial046.stderr: 2026-02-24T22:23:21.949 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:23:21.949 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:23:21.953 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:23:21.953 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:23:21.999 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:23:21.999 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:23:22.010 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:23:22.010 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:23:22.013 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:23:22.013 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:23:22.024 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:23:22.024 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:23:22.026 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:23:22.026 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:23:22.035 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:23:22.035 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:23:22.038 INFO:tasks.workunit.client.0.trial046.stderr:ceph config set client rgw_debug_inject_set_olh_err 2 2026-02-24T22:23:22.246 INFO:tasks.workunit.client.0.trial046.stderr:ceph config set client rgw_debug_inject_olh_cancel_modification_err true 2026-02-24T22:23:23.452 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:23:23.452 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:23:23.503 INFO:tasks.workunit.client.0.trial046.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2026-02-24T22:23:23.504 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bi list --bucket myver --object x 2026-02-24T22:23:23.521 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:23.521 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:23.538 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.545+0000 7f0c8896ba80 4 Realm: () 2026-02-24T22:23:23.538 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.545+0000 7f0c8896ba80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:23.539 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.545+0000 7f0c8896ba80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:23.539 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.545+0000 7f0c8896ba80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:23.539 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.545+0000 7f0c8896ba80 20 zonegroup default 2026-02-24T22:23:23.539 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.545+0000 7f0c8896ba80 20 rados->read ofs=0 len=0 2026-02-24T22:23:23.540 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.545+0000 7f0c8896ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:23.540 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.545+0000 7f0c8896ba80 20 rados->read ofs=0 len=0 2026-02-24T22:23:23.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.545+0000 7f0c8896ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:23.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.545+0000 7f0c8896ba80 20 started sync module instance, tier type = 2026-02-24T22:23:23.541 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.545+0000 7f0c8896ba80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:23.542 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.549+0000 7f0c8896ba80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:23.542 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.549+0000 7f0c8896ba80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:23.542 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.549+0000 7f0c8896ba80 5 note: GC not initialized 2026-02-24T22:23:23.543 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.549+0000 7f0ba7fff640 20 reqs_thread_entry: start 2026-02-24T22:23:23.546 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.553+0000 7f0c86c40640 0 -- 10.20.193.46:0/3875663105 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55c364286580 msgr2=0x55c36427fef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:23:23.552 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.557+0000 7f0c8643f640 0 -- 10.20.193.46:0/3875663105 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55c36425ab90 msgr2=0x55c364288700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:23.781 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.785+0000 7f0c8896ba80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:23.781 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.785+0000 7f0c8896ba80 20 Filter name: none 2026-02-24T22:23:23.781 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.785+0000 7f0ba67fc640 20 reqs_thread_entry: start 2026-02-24T22:23:23.781 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.785+0000 7f0c8896ba80 20 rados->read ofs=0 len=0 2026-02-24T22:23:23.782 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.789+0000 7f0c8896ba80 20 rados_obj.operate() r=0 bl.length=201 2026-02-24T22:23:23.782 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.789+0000 7f0c8896ba80 20 rados->read ofs=0 len=0 2026-02-24T22:23:23.783 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.789+0000 7f0c8896ba80 20 rados_obj.operate() r=0 bl.length=431 2026-02-24T22:23:23.783 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.789+0000 7f0c8896ba80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:23:23.783 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.789+0000 7f0c8896ba80 20 INFO: main: max_entries=1000, index=gen=2, max_shards=3 2026-02-24T22:23:23.783 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.789+0000 7f0c8896ba80 20 INFO: main: starting shard=0 2026-02-24T22:23:23.783 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.789+0000 7f0c8896ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.0 2026-02-24T22:23:23.783 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.789+0000 7f0c8896ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.0 2026-02-24T22:23:23.783 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.789+0000 7f0c8896ba80 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, next_marker=�1000_xi.RIq1Sk0TODS6TPiGYLQkX4Y7iYsHH1 2026-02-24T22:23:23.783 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.789+0000 7f0c8896ba80 20 INFO: main: starting shard=1 2026-02-24T22:23:23.783 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.789+0000 7f0c8896ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.1 2026-02-24T22:23:23.784 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.789+0000 7f0c8896ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.1 2026-02-24T22:23:23.784 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.789+0000 7f0c8896ba80 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, next_marker= 2026-02-24T22:23:23.784 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.789+0000 7f0c8896ba80 20 INFO: main: starting shard=2 2026-02-24T22:23:23.784 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.789+0000 7f0c8896ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.2 2026-02-24T22:23:23.784 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.789+0000 7f0c8896ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.2 2026-02-24T22:23:23.784 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.789+0000 7f0c8896ba80 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, next_marker= 2026-02-24T22:23:23.785 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.789+0000 7f0c8896ba80 20 INFO: main: done 2026-02-24T22:23:23.793 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:23:23.793 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:23:23.796 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:23:23.796 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:23:23.846 INFO:tasks.workunit.client.0.trial046.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2026-02-24T22:23:23.846 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bi list --bucket myver --object y 2026-02-24T22:23:23.862 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:23.863 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:23.880 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.885+0000 7f86ea4c6a80 4 Realm: () 2026-02-24T22:23:23.880 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.885+0000 7f86ea4c6a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:23.880 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.885+0000 7f86ea4c6a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:23.881 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.885+0000 7f86ea4c6a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:23.881 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.885+0000 7f86ea4c6a80 20 zonegroup default 2026-02-24T22:23:23.881 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.885+0000 7f86ea4c6a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:23.882 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.889+0000 7f86ea4c6a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:23.882 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.889+0000 7f86ea4c6a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:23.882 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.889+0000 7f86ea4c6a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:23.882 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.889+0000 7f86ea4c6a80 20 started sync module instance, tier type = 2026-02-24T22:23:23.882 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.889+0000 7f86ea4c6a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:23.884 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.889+0000 7f86ea4c6a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:23.884 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.889+0000 7f86ea4c6a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:23.884 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.889+0000 7f86ea4c6a80 5 note: GC not initialized 2026-02-24T22:23:23.884 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.889+0000 7f86957e2640 20 reqs_thread_entry: start 2026-02-24T22:23:23.888 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.893+0000 7f86e3fff640 0 -- 10.20.193.46:0/420138570 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x556b0a9c2ca0 msgr2=0x556b0a9c93f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:23.893 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.897+0000 7f86e37fe640 0 -- 10.20.193.46:0/420138570 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x556b0a9c4e70 msgr2=0x556b0a9bfbe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:23.917 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.921+0000 7f86ea4c6a80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:23.917 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.921+0000 7f86ea4c6a80 20 Filter name: none 2026-02-24T22:23:23.917 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.921+0000 7f85ff7fe640 20 reqs_thread_entry: start 2026-02-24T22:23:23.917 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.921+0000 7f86ea4c6a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:23.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.925+0000 7f86ea4c6a80 20 rados_obj.operate() r=0 bl.length=201 2026-02-24T22:23:23.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.925+0000 7f86ea4c6a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:23.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.925+0000 7f86ea4c6a80 20 rados_obj.operate() r=0 bl.length=431 2026-02-24T22:23:23.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.925+0000 7f86ea4c6a80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:23:23.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.925+0000 7f86ea4c6a80 20 INFO: main: max_entries=1000, index=gen=2, max_shards=3 2026-02-24T22:23:23.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.925+0000 7f86ea4c6a80 20 INFO: main: starting shard=0 2026-02-24T22:23:23.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.925+0000 7f86ea4c6a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.0 2026-02-24T22:23:23.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.925+0000 7f86ea4c6a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.0 2026-02-24T22:23:23.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.925+0000 7f86ea4c6a80 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, next_marker= 2026-02-24T22:23:23.919 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.925+0000 7f86ea4c6a80 20 INFO: main: starting shard=1 2026-02-24T22:23:23.920 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.925+0000 7f86ea4c6a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.1 2026-02-24T22:23:23.920 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.925+0000 7f86ea4c6a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.1 2026-02-24T22:23:23.920 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.925+0000 7f86ea4c6a80 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, next_marker= 2026-02-24T22:23:23.920 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.925+0000 7f86ea4c6a80 20 INFO: main: starting shard=2 2026-02-24T22:23:23.920 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.925+0000 7f86ea4c6a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.2 2026-02-24T22:23:23.920 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.925+0000 7f86ea4c6a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.2 2026-02-24T22:23:23.921 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.925+0000 7f86ea4c6a80 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, next_marker=�1000_yidR4dv0kksDH8G1ZbEvAcH-MeYZBVFPa 2026-02-24T22:23:23.921 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:23.925+0000 7f86ea4c6a80 20 INFO: main: done 2026-02-24T22:23:23.930 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:23:23.930 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:23:23.933 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:23:23.933 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:23:23.982 INFO:tasks.workunit.client.0.trial046.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2026-02-24T22:23:23.982 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bi list --bucket myver --object z 2026-02-24T22:23:23.998 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:23.998 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:24.016 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.021+0000 7fcc5235ba80 4 Realm: () 2026-02-24T22:23:24.016 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.021+0000 7fcc5235ba80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:24.016 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.021+0000 7fcc5235ba80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:24.016 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.021+0000 7fcc5235ba80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:24.016 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.021+0000 7fcc5235ba80 20 zonegroup default 2026-02-24T22:23:24.016 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.021+0000 7fcc5235ba80 20 rados->read ofs=0 len=0 2026-02-24T22:23:24.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.021+0000 7fcc5235ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:24.017 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.021+0000 7fcc5235ba80 20 rados->read ofs=0 len=0 2026-02-24T22:23:24.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.025+0000 7fcc5235ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:24.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.025+0000 7fcc5235ba80 20 started sync module instance, tier type = 2026-02-24T22:23:24.018 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.025+0000 7fcc5235ba80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:24.019 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.025+0000 7fcc5235ba80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:24.019 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.025+0000 7fcc5235ba80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:24.019 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.025+0000 7fcc5235ba80 5 note: GC not initialized 2026-02-24T22:23:24.020 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.025+0000 7fcbfd7e2640 20 reqs_thread_entry: start 2026-02-24T22:23:24.023 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.029+0000 7fcc4bfff640 0 -- 10.20.193.46:0/2005644680 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x55a90b9578e0 msgr2=0x55a90b94ab50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:24.233 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.237+0000 7fcc4b7fe640 0 -- 10.20.193.46:0/2005644680 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55a90b953360 msgr2=0x55a90b94e0d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:23:24.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.265+0000 7fcc5235ba80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:24.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.265+0000 7fcc5235ba80 20 Filter name: none 2026-02-24T22:23:24.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.265+0000 7fcb677fe640 20 reqs_thread_entry: start 2026-02-24T22:23:24.260 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.265+0000 7fcc5235ba80 20 rados->read ofs=0 len=0 2026-02-24T22:23:24.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.265+0000 7fcc5235ba80 20 rados_obj.operate() r=0 bl.length=201 2026-02-24T22:23:24.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.265+0000 7fcc5235ba80 20 rados->read ofs=0 len=0 2026-02-24T22:23:24.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.269+0000 7fcc5235ba80 20 rados_obj.operate() r=0 bl.length=431 2026-02-24T22:23:24.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.269+0000 7fcc5235ba80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:23:24.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.269+0000 7fcc5235ba80 20 INFO: main: max_entries=1000, index=gen=2, max_shards=3 2026-02-24T22:23:24.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.269+0000 7fcc5235ba80 20 INFO: main: starting shard=0 2026-02-24T22:23:24.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.269+0000 7fcc5235ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.0 2026-02-24T22:23:24.262 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.269+0000 7fcc5235ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.0 2026-02-24T22:23:24.263 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.269+0000 7fcc5235ba80 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, next_marker= 2026-02-24T22:23:24.263 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.269+0000 7fcc5235ba80 20 INFO: main: starting shard=1 2026-02-24T22:23:24.263 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.269+0000 7fcc5235ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.1 2026-02-24T22:23:24.263 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.269+0000 7fcc5235ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.1 2026-02-24T22:23:24.263 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.269+0000 7fcc5235ba80 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, next_marker=�1000_ziNIDQal-7oGB0iH13F80LQvr-y58xIOe 2026-02-24T22:23:24.263 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.269+0000 7fcc5235ba80 20 INFO: main: starting shard=2 2026-02-24T22:23:24.263 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.269+0000 7fcc5235ba80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.2 2026-02-24T22:23:24.263 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.269+0000 7fcc5235ba80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.2 2026-02-24T22:23:24.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.269+0000 7fcc5235ba80 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, next_marker= 2026-02-24T22:23:24.264 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.269+0000 7fcc5235ba80 20 INFO: main: done 2026-02-24T22:23:24.273 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:23:24.273 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:23:24.275 INFO:tasks.workunit.client.0.trial046.stderr:ceph config rm client rgw_debug_inject_set_olh_err 2026-02-24T22:23:24.479 INFO:tasks.workunit.client.0.trial046.stderr:ceph config rm client rgw_debug_inject_olh_cancel_modification_err 2026-02-24T22:23:24.686 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket reshard --bucket myver --num-shards 17 --yes-i-really-mean-it 2026-02-24T22:23:24.702 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:24.702 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:24.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.929+0000 7f1dc0808a80 4 Realm: () 2026-02-24T22:23:24.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.929+0000 7f1dc0808a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:24.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.929+0000 7f1dc0808a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:24.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.929+0000 7f1dc0808a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:24.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.929+0000 7f1dc0808a80 20 zonegroup default 2026-02-24T22:23:24.923 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.929+0000 7f1dc0808a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:24.925 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.929+0000 7f1dc0808a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:24.925 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.929+0000 7f1dc0808a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:24.925 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.929+0000 7f1dc0808a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:24.925 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.929+0000 7f1dc0808a80 20 started sync module instance, tier type = 2026-02-24T22:23:24.925 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.929+0000 7f1dc0808a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:24.926 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.933+0000 7f1dbe2dc640 0 -- 10.20.193.46:0/4038187274 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x5653c5dbb100 msgr2=0x5653c5ddb580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:23:24.927 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.933+0000 7f1dc0808a80 20 add_watcher() i=1 2026-02-24T22:23:24.927 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.933+0000 7f1dc0808a80 20 add_watcher() i=6 2026-02-24T22:23:24.927 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.933+0000 7f1dc0808a80 20 add_watcher() i=5 2026-02-24T22:23:24.928 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.933+0000 7f1dc0808a80 20 add_watcher() i=4 2026-02-24T22:23:24.928 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:24.933+0000 7f1dc0808a80 20 add_watcher() i=7 2026-02-24T22:23:25.131 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.137+0000 7f1dc0808a80 20 add_watcher() i=0 2026-02-24T22:23:25.132 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.137+0000 7f1dc0808a80 20 add_watcher() i=3 2026-02-24T22:23:25.132 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.137+0000 7f1dc0808a80 20 add_watcher() i=2 2026-02-24T22:23:25.132 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.137+0000 7f1dc0808a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:23:25.133 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.137+0000 7f1dbdadb640 0 -- 10.20.193.46:0/1340019498 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f1ce0008af0 msgr2=0x7f1ce0028fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:25.336 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.341+0000 7f1dc0808a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:25.336 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.341+0000 7f1dc0808a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:25.336 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.341+0000 7f1dc0808a80 5 note: GC not initialized 2026-02-24T22:23:25.336 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.341+0000 7f1cd77fe640 20 reqs_thread_entry: start 2026-02-24T22:23:25.373 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.377+0000 7f1dc0808a80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:25.373 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.377+0000 7f1dc0808a80 20 Filter name: none 2026-02-24T22:23:25.373 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.377+0000 7f1cd5ffb640 20 reqs_thread_entry: start 2026-02-24T22:23:25.373 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.377+0000 7f1dc0808a80 10 cache get: name=default.rgw.meta+root+myver : miss 2026-02-24T22:23:25.373 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.377+0000 7f1dc0808a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:25.374 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.381+0000 7f1dc0808a80 20 rados_obj.operate() r=0 bl.length=201 2026-02-24T22:23:25.375 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.381+0000 7f1dc0808a80 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2026-02-24T22:23:25.375 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.381+0000 7f1dc0808a80 10 adding default.rgw.meta+root+myver to cache LRU end 2026-02-24T22:23:25.375 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.381+0000 7f1dc0808a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 : miss 2026-02-24T22:23:25.375 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.381+0000 7f1dc0808a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:25.375 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.381+0000 7f1dc0808a80 20 rados_obj.operate() r=0 bl.length=431 2026-02-24T22:23:25.375 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.381+0000 7f1dc0808a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 info.flags=0x17 2026-02-24T22:23:25.375 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.381+0000 7f1dc0808a80 10 adding default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 to cache LRU end 2026-02-24T22:23:25.375 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.381+0000 7f1dc0808a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:23:25.375 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.381+0000 7f1dc0808a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:25.375 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.381+0000 7f1dc0808a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 2026-02-24T22:23:25.379 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.385+0000 7f1dc0808a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 info.flags=0x17 2026-02-24T22:23:25.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.385+0000 7f1dc0808a80 10 moving default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 to cache LRU end 2026-02-24T22:23:25.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.385+0000 7f1dc0808a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:25.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.385+0000 7f1dc0808a80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519, ofs0, ns] 2026-02-24T22:23:25.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.385+0000 7f1d997fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959920 cookie 94917802861680 notifier 8366 bl.length()=923 2026-02-24T22:23:25.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.385+0000 7f1d997fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 info.flags=0x17 2026-02-24T22:23:25.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.385+0000 7f1d997fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 to cache LRU end 2026-02-24T22:23:25.380 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.385+0000 7f1d997fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:25.381 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.385+0000 7f1dc0808a80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): orig_sources= new_sources= 2026-02-24T22:23:25.381 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.385+0000 7f1dc0808a80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): potential sources added=[] removed=[] 2026-02-24T22:23:25.381 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.385+0000 7f1dc0808a80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): orig_dests= new_dests= 2026-02-24T22:23:25.381 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.385+0000 7f1dc0808a80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): potential dests added=[] removed=[] 2026-02-24T22:23:25.382 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.0 2026-02-24T22:23:25.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.0 2026-02-24T22:23:25.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.1 2026-02-24T22:23:25.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.1 2026-02-24T22:23:25.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.2 2026-02-24T22:23:25.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.2 2026-02-24T22:23:25.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.3 2026-02-24T22:23:25.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.3 2026-02-24T22:23:25.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.4 2026-02-24T22:23:25.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.4 2026-02-24T22:23:25.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.5 2026-02-24T22:23:25.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.5 2026-02-24T22:23:25.383 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.6 2026-02-24T22:23:25.384 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.6 2026-02-24T22:23:25.384 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.7 2026-02-24T22:23:25.384 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.7 2026-02-24T22:23:25.384 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.8 2026-02-24T22:23:25.384 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.8 2026-02-24T22:23:25.384 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.9 2026-02-24T22:23:25.384 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.9 2026-02-24T22:23:25.384 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.10 2026-02-24T22:23:25.384 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.10 2026-02-24T22:23:25.384 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.11 2026-02-24T22:23:25.384 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.11 2026-02-24T22:23:25.384 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.12 2026-02-24T22:23:25.384 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.12 2026-02-24T22:23:25.385 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.13 2026-02-24T22:23:25.385 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.13 2026-02-24T22:23:25.385 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.14 2026-02-24T22:23:25.385 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.14 2026-02-24T22:23:25.385 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.15 2026-02-24T22:23:25.385 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.15 2026-02-24T22:23:25.385 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.16 2026-02-24T22:23:25.385 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.16 2026-02-24T22:23:25.385 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.0 2026-02-24T22:23:25.385 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.0 2026-02-24T22:23:25.385 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.1 2026-02-24T22:23:25.385 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.1 2026-02-24T22:23:25.385 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.2 2026-02-24T22:23:25.386 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.2 2026-02-24T22:23:25.387 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 : hit (requested=0x17, cached=0x17) 2026-02-24T22:23:25.387 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 2026-02-24T22:23:25.387 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 info.flags=0x17 2026-02-24T22:23:25.387 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 10 moving default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 to cache LRU end 2026-02-24T22:23:25.387 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:25.387 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.389+0000 7f1dc0808a80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519, ofs0, ns] 2026-02-24T22:23:25.387 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1d99ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959921 cookie 94917802861680 notifier 8366 bl.length()=923 2026-02-24T22:23:25.387 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1d99ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 info.flags=0x17 2026-02-24T22:23:25.387 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1d99ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 to cache LRU end 2026-02-24T22:23:25.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1d99ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:25.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1dc0808a80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): orig_sources= new_sources= 2026-02-24T22:23:25.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1dc0808a80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): potential sources added=[] removed=[] 2026-02-24T22:23:25.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1dc0808a80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): orig_dests= new_dests= 2026-02-24T22:23:25.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1dc0808a80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): potential dests added=[] removed=[] 2026-02-24T22:23:25.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1dc0808a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.0 2026-02-24T22:23:25.388 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1dc0808a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.0 2026-02-24T22:23:25.389 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1dc0808a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.1 2026-02-24T22:23:25.389 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1dc0808a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.1 2026-02-24T22:23:25.389 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1dc0808a80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.2 2026-02-24T22:23:25.390 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1dc0808a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.2.2 2026-02-24T22:23:25.390 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1dc0808a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 : hit (requested=0x17, cached=0x17) 2026-02-24T22:23:25.390 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1dc0808a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 2026-02-24T22:23:25.390 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1dc0808a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 info.flags=0x17 2026-02-24T22:23:25.390 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1dc0808a80 10 moving default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 to cache LRU end 2026-02-24T22:23:25.390 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1dc0808a80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:25.390 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1dc0808a80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519, ofs0, ns] 2026-02-24T22:23:25.390 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1d99ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554959922 cookie 94917802861680 notifier 8366 bl.length()=887 2026-02-24T22:23:25.390 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1d99ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 info.flags=0x17 2026-02-24T22:23:25.390 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1d99ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 to cache LRU end 2026-02-24T22:23:25.390 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1d99ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:25.390 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1dc0808a80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): orig_sources= new_sources= 2026-02-24T22:23:25.390 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1dc0808a80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): potential sources added=[] removed=[] 2026-02-24T22:23:25.391 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1dc0808a80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): orig_dests= new_dests= 2026-02-24T22:23:25.391 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.393+0000 7f1dc0808a80 20 handle_bi_update(): bucket=:myver[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519]): potential dests added=[] removed=[] 2026-02-24T22:23:25.391 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.397+0000 7f1dc0808a80 1 execute INFO: reshard of bucket "myver" from 3 shards to 17 shards completed successfully 2026-02-24T22:23:25.394 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.401+0000 7f1dc0808a80 20 remove_watcher() i=3 2026-02-24T22:23:25.394 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.401+0000 7f1dc0808a80 2 removed watcher, disabling cache 2026-02-24T22:23:25.394 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.401+0000 7f1dc0808a80 20 remove_watcher() i=5 2026-02-24T22:23:25.394 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.401+0000 7f1dc0808a80 20 remove_watcher() i=4 2026-02-24T22:23:25.395 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.401+0000 7f1dc0808a80 20 remove_watcher() i=1 2026-02-24T22:23:25.395 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.401+0000 7f1dc0808a80 20 remove_watcher() i=7 2026-02-24T22:23:25.395 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.401+0000 7f1dc0808a80 20 remove_watcher() i=0 2026-02-24T22:23:25.395 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.401+0000 7f1dc0808a80 20 remove_watcher() i=2 2026-02-24T22:23:25.395 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.401+0000 7f1dc0808a80 20 remove_watcher() i=6 2026-02-24T22:23:25.401 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket check unlinked --bucket myver --fix --min-age-hours 0 --rgw-olh-pending-timeout-sec 0 --dump-keys 2026-02-24T22:23:25.419 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:25.419 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:25.435 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.441+0000 7f04e606ca80 4 Realm: () 2026-02-24T22:23:25.435 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.441+0000 7f04e606ca80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:25.435 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.441+0000 7f04e606ca80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:25.435 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.441+0000 7f04e606ca80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:25.435 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.441+0000 7f04e606ca80 20 zonegroup default 2026-02-24T22:23:25.435 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.441+0000 7f04e606ca80 20 rados->read ofs=0 len=0 2026-02-24T22:23:25.436 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.441+0000 7f04e606ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:25.436 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.441+0000 7f04e606ca80 20 rados->read ofs=0 len=0 2026-02-24T22:23:25.437 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.441+0000 7f04e606ca80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:25.437 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.441+0000 7f04e606ca80 20 started sync module instance, tier type = 2026-02-24T22:23:25.437 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.441+0000 7f04e606ca80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:25.437 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.441+0000 7f04df7fe640 0 -- 10.20.193.46:0/1789607547 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x56451b8171d0 msgr2=0x56451b837650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:25.438 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.445+0000 7f04e606ca80 20 add_watcher() i=1 2026-02-24T22:23:25.438 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.445+0000 7f04e606ca80 20 add_watcher() i=6 2026-02-24T22:23:25.439 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.445+0000 7f04e606ca80 20 add_watcher() i=3 2026-02-24T22:23:25.439 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.445+0000 7f04e606ca80 20 add_watcher() i=5 2026-02-24T22:23:25.439 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.445+0000 7f04e606ca80 20 add_watcher() i=0 2026-02-24T22:23:25.439 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.445+0000 7f04e606ca80 20 add_watcher() i=4 2026-02-24T22:23:25.440 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.445+0000 7f04e606ca80 20 add_watcher() i=2 2026-02-24T22:23:25.440 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.445+0000 7f04e606ca80 20 add_watcher() i=7 2026-02-24T22:23:25.440 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.445+0000 7f04e606ca80 2 all 8 watchers are set, enabling cache 2026-02-24T22:23:25.440 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.445+0000 7f04d7fff640 0 -- 10.20.193.46:0/1837521584 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f0404008af0 msgr2=0x7f0404028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:25.644 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.649+0000 7f04e606ca80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:25.644 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.649+0000 7f04e606ca80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:25.644 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.649+0000 7f04e606ca80 5 note: GC not initialized 2026-02-24T22:23:25.645 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.649+0000 7f0490fe1640 20 reqs_thread_entry: start 2026-02-24T22:23:25.680 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f04e606ca80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:25.680 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f04e606ca80 20 Filter name: none 2026-02-24T22:23:25.680 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f03faffd640 20 reqs_thread_entry: start 2026-02-24T22:23:25.680 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f04e606ca80 10 cache get: name=default.rgw.meta+root+myver : miss 2026-02-24T22:23:25.680 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f04e606ca80 20 rados->read ofs=0 len=0 2026-02-24T22:23:25.680 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f04e606ca80 20 rados_obj.operate() r=0 bl.length=201 2026-02-24T22:23:25.680 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f04e606ca80 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2026-02-24T22:23:25.680 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f04e606ca80 10 adding default.rgw.meta+root+myver to cache LRU end 2026-02-24T22:23:25.680 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f04e606ca80 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 : miss 2026-02-24T22:23:25.681 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f04e606ca80 20 rados->read ofs=0 len=0 2026-02-24T22:23:25.682 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f04e606ca80 20 rados_obj.operate() r=0 bl.length=431 2026-02-24T22:23:25.682 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f04e606ca80 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 info.flags=0x17 2026-02-24T22:23:25.682 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f04e606ca80 10 adding default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 to cache LRU end 2026-02-24T22:23:25.682 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f04e606ca80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:23:25.682 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f04e606ca80 10 updating xattr: name=user.rgw.acl bl.length()=203 2026-02-24T22:23:25.682 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f04e606ca80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519 2026-02-24T22:23:25.682 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f04e606ca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.0 2026-02-24T22:23:25.682 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.0 2026-02-24T22:23:25.682 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f04e606ca80 1 NOTICE: finished shard 0 (0 entries removed) 2026-02-24T22:23:25.683 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f04e606ca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.1 2026-02-24T22:23:25.683 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.1 2026-02-24T22:23:25.683 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f04e606ca80 1 NOTICE: finished shard 1 (0 entries removed) 2026-02-24T22:23:25.683 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f04e606ca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.2 2026-02-24T22:23:25.683 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.2 2026-02-24T22:23:25.683 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f04e606ca80 1 NOTICE: finished shard 2 (0 entries removed) 2026-02-24T22:23:25.683 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f04e606ca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.3 2026-02-24T22:23:25.683 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.685+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.3 2026-02-24T22:23:25.683 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 1 NOTICE: finished shard 3 (0 entries removed) 2026-02-24T22:23:25.683 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.4 2026-02-24T22:23:25.683 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.4 2026-02-24T22:23:25.685 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 1 NOTICE: finished shard 4 (0 entries removed) 2026-02-24T22:23:25.685 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.5 2026-02-24T22:23:25.685 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.5 2026-02-24T22:23:25.685 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 1 NOTICE: finished shard 5 (0 entries removed) 2026-02-24T22:23:25.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.6 2026-02-24T22:23:25.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.6 2026-02-24T22:23:25.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 1 NOTICE: finished shard 6 (0 entries removed) 2026-02-24T22:23:25.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.7 2026-02-24T22:23:25.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.7 2026-02-24T22:23:25.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 1 NOTICE: finished shard 7 (0 entries removed) 2026-02-24T22:23:25.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.8 2026-02-24T22:23:25.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.8 2026-02-24T22:23:25.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 1 NOTICE: finished shard 8 (0 entries removed) 2026-02-24T22:23:25.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.9 2026-02-24T22:23:25.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.9 2026-02-24T22:23:25.686 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.10 2026-02-24T22:23:25.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.10 2026-02-24T22:23:25.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 1 NOTICE: finished shard 10 (0 entries removed) 2026-02-24T22:23:25.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.11 2026-02-24T22:23:25.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.11 2026-02-24T22:23:25.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 1 NOTICE: finished shard 11 (0 entries removed) 2026-02-24T22:23:25.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.12 2026-02-24T22:23:25.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.12 2026-02-24T22:23:25.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 1 NOTICE: finished shard 12 (0 entries removed) 2026-02-24T22:23:25.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.13 2026-02-24T22:23:25.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.13 2026-02-24T22:23:25.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 1 NOTICE: finished shard 13 (0 entries removed) 2026-02-24T22:23:25.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.14 2026-02-24T22:23:25.687 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.14 2026-02-24T22:23:25.688 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 1 NOTICE: finished shard 14 (0 entries removed) 2026-02-24T22:23:25.688 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.15 2026-02-24T22:23:25.688 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.689+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.15 2026-02-24T22:23:25.689 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.693+0000 7f04e606ca80 1 NOTICE: finished shard 15 (0 entries removed) 2026-02-24T22:23:25.689 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.693+0000 7f04e606ca80 10 bucket_obj is .dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.16 2026-02-24T22:23:25.689 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.693+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.16 2026-02-24T22:23:25.689 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.693+0000 7f04e606ca80 1 NOTICE: finished shard 16 (0 entries removed) 2026-02-24T22:23:25.689 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.693+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.9 2026-02-24T22:23:25.690 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.693+0000 7f04e606ca80 20 get_obj_state: octx=0x56451b74aa90 obj=myver:x state=0x56451b74ee18 s->prefetch_data=0 2026-02-24T22:23:25.690 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.693+0000 7f04e606ca80 20 get_obj_state: setting s->obj_tag to lb5huxg0ij6gsgfakvp69qy8wzguu9jo 2026-02-24T22:23:25.690 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.693+0000 7f04e606ca80 20 get_obj_state_impl: setting s->olh_tag to 2m1rq2j3zmrdar2jz14po5aairzxitej 2026-02-24T22:23:25.690 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.693+0000 7f04e606ca80 20 bucket_index_guard_olh_op(): olh_state.olh_tag=2m1rq2j3zmrdar2jz14po5aairzxitej 2026-02-24T22:23:25.690 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.693+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.9 2026-02-24T22:23:25.690 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.693+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.9 2026-02-24T22:23:25.690 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.693+0000 7f04e606ca80 20 olh_log_entry: epoch=1771971805697868512 op=2 key=x[] 2026-02-24T22:23:25.690 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.693+0000 7f04e606ca80 20 olh_log_entry: epoch=1771971805697868512 op=3 key=x[.RIq1Sk0TODS6TPiGYLQkX4Y7iYsHH1] 2026-02-24T22:23:25.690 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.693+0000 7f04e606ca80 20 get_obj_state: octx=0x56451b74aa90 obj=myver:_:.RIq1Sk0TODS6TPiGYLQkX4Y7iYsHH1_x state=0x56451b68d668 s->prefetch_data=0 2026-02-24T22:23:25.691 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.697+0000 7f04e606ca80 10 manifest: total_size = 12 2026-02-24T22:23:25.691 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.697+0000 7f04e606ca80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2646905385798058454 2026-02-24T22:23:25.691 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.697+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.9 2026-02-24T22:23:25.691 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.697+0000 7f04e606ca80 20 get_obj_state: octx=0x56451b74aa90 obj=myver:_:.RIq1Sk0TODS6TPiGYLQkX4Y7iYsHH1_x state=0x56451b68d668 s->prefetch_data=0 2026-02-24T22:23:25.691 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.697+0000 7f04e606ca80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:23:25.691 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.697+0000 7f04e606ca80 20 RGWObjManifest::operator++(): result: ofs=12 stripe_ofs=12 part_ofs=0 rule->part_size=0 2026-02-24T22:23:25.692 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.697+0000 7f04e606ca80 0 deleting objects inline since gc isn't initialized 2026-02-24T22:23:25.692 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.697+0000 7f04ba7fc640 20 handle_completion(): completion ok for obj=x[.RIq1Sk0TODS6TPiGYLQkX4Y7iYsHH1] 2026-02-24T22:23:25.692 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.697+0000 7f04e606ca80 20 get_obj_state: octx=0x56451b74aa90 obj=myver:x state=0x56451b74ee18 s->prefetch_data=0 2026-02-24T22:23:25.693 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.697+0000 7f04e606ca80 20 bucket_index_guard_olh_op(): olh_state.olh_tag=2m1rq2j3zmrdar2jz14po5aairzxitej 2026-02-24T22:23:25.694 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.701+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.9 2026-02-24T22:23:25.694 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.701+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.9 2026-02-24T22:23:25.695 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.701+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.9 2026-02-24T22:23:25.695 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.701+0000 7f04e606ca80 20 get_obj_state: octx=0x56451b74aa90 obj=myver:y state=0x56451b74ee18 s->prefetch_data=0 2026-02-24T22:23:25.695 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.701+0000 7f04e606ca80 20 get_obj_state: setting s->obj_tag to hwoeo5u73bskytuetrx7raky4pizjenb 2026-02-24T22:23:25.695 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.701+0000 7f04e606ca80 20 get_obj_state_impl: setting s->olh_tag to zlr0e3b88b7fzyugi2deijlidt99qyl2 2026-02-24T22:23:25.696 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.701+0000 7f04e606ca80 20 bucket_index_guard_olh_op(): olh_state.olh_tag=zlr0e3b88b7fzyugi2deijlidt99qyl2 2026-02-24T22:23:25.696 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.701+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.9 2026-02-24T22:23:25.697 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.701+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.9 2026-02-24T22:23:25.697 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.701+0000 7f04e606ca80 20 olh_log_entry: epoch=1771971805706527218 op=2 key=y[] 2026-02-24T22:23:25.698 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.701+0000 7f04e606ca80 20 olh_log_entry: epoch=1771971805706527218 op=3 key=y[dR4dv0kksDH8G1ZbEvAcH-MeYZBVFPa] 2026-02-24T22:23:25.698 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.701+0000 7f04e606ca80 20 get_obj_state: octx=0x56451b74aa90 obj=myver:_:dR4dv0kksDH8G1ZbEvAcH-MeYZBVFPa_y state=0x56451b68d668 s->prefetch_data=0 2026-02-24T22:23:25.698 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.701+0000 7f04e606ca80 10 manifest: total_size = 22 2026-02-24T22:23:25.698 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.701+0000 7f04e606ca80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2819607810328981840 2026-02-24T22:23:25.698 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.701+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.9 2026-02-24T22:23:25.699 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.705+0000 7f04e606ca80 20 get_obj_state: octx=0x56451b74aa90 obj=myver:_:dR4dv0kksDH8G1ZbEvAcH-MeYZBVFPa_y state=0x56451b68d668 s->prefetch_data=0 2026-02-24T22:23:25.699 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.705+0000 7f04e606ca80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:23:25.699 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.705+0000 7f04e606ca80 20 RGWObjManifest::operator++(): result: ofs=22 stripe_ofs=22 part_ofs=0 rule->part_size=0 2026-02-24T22:23:25.699 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.705+0000 7f04e606ca80 0 deleting objects inline since gc isn't initialized 2026-02-24T22:23:25.700 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.705+0000 7f04ba7fc640 20 handle_completion(): completion ok for obj=y[dR4dv0kksDH8G1ZbEvAcH-MeYZBVFPa] 2026-02-24T22:23:25.701 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.705+0000 7f04e606ca80 20 get_obj_state: octx=0x56451b74aa90 obj=myver:y state=0x56451b74ee18 s->prefetch_data=0 2026-02-24T22:23:25.701 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.705+0000 7f04e606ca80 20 bucket_index_guard_olh_op(): olh_state.olh_tag=zlr0e3b88b7fzyugi2deijlidt99qyl2 2026-02-24T22:23:25.702 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.709+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.9 2026-02-24T22:23:25.703 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.709+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.9 2026-02-24T22:23:25.703 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.709+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.9 2026-02-24T22:23:25.703 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.709+0000 7f04e606ca80 20 get_obj_state: octx=0x56451b733e20 obj=myver:z state=0x56451b74ee18 s->prefetch_data=0 2026-02-24T22:23:25.704 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.709+0000 7f04e606ca80 20 get_obj_state: setting s->obj_tag to 9h1z5lg3kw4bi3yy92b8j9l1oe201jwf 2026-02-24T22:23:25.704 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.709+0000 7f04e606ca80 20 get_obj_state_impl: setting s->olh_tag to oiprjhbppw5tlm21ztux0qbykad0t0tl 2026-02-24T22:23:25.704 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.709+0000 7f04e606ca80 20 bucket_index_guard_olh_op(): olh_state.olh_tag=oiprjhbppw5tlm21ztux0qbykad0t0tl 2026-02-24T22:23:25.704 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.709+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.9 2026-02-24T22:23:25.705 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.709+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.9 2026-02-24T22:23:25.705 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.709+0000 7f04e606ca80 20 olh_log_entry: epoch=1771971805714781952 op=2 key=z[] 2026-02-24T22:23:25.705 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.709+0000 7f04e606ca80 20 olh_log_entry: epoch=1771971805714781952 op=3 key=z[NIDQal-7oGB0iH13F80LQvr-y58xIOe] 2026-02-24T22:23:25.706 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.709+0000 7f04e606ca80 20 get_obj_state: octx=0x56451b733e20 obj=myver:_:NIDQal-7oGB0iH13F80LQvr-y58xIOe_z state=0x56451b68d668 s->prefetch_data=0 2026-02-24T22:23:25.706 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.713+0000 7f04e606ca80 10 manifest: total_size = 12 2026-02-24T22:23:25.706 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.713+0000 7f04e606ca80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6280349093484818115 2026-02-24T22:23:25.706 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.713+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.9 2026-02-24T22:23:25.707 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.713+0000 7f04e606ca80 20 get_obj_state: octx=0x56451b733e20 obj=myver:_:NIDQal-7oGB0iH13F80LQvr-y58xIOe_z state=0x56451b68d668 s->prefetch_data=0 2026-02-24T22:23:25.707 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.713+0000 7f04e606ca80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:23:25.708 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.713+0000 7f04e606ca80 20 RGWObjManifest::operator++(): result: ofs=12 stripe_ofs=12 part_ofs=0 rule->part_size=0 2026-02-24T22:23:25.708 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.713+0000 7f04e606ca80 0 deleting objects inline since gc isn't initialized 2026-02-24T22:23:25.708 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.713+0000 7f04ba7fc640 20 handle_completion(): completion ok for obj=z[NIDQal-7oGB0iH13F80LQvr-y58xIOe] 2026-02-24T22:23:25.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.713+0000 7f04e606ca80 20 get_obj_state: octx=0x56451b733e20 obj=myver:z state=0x56451b74ee18 s->prefetch_data=0 2026-02-24T22:23:25.709 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.713+0000 7f04e606ca80 20 bucket_index_guard_olh_op(): olh_state.olh_tag=oiprjhbppw5tlm21ztux0qbykad0t0tl 2026-02-24T22:23:25.710 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.717+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.9 2026-02-24T22:23:25.711 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.717+0000 7f04e606ca80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519.3.9 2026-02-24T22:23:25.711 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.717+0000 7f04e606ca80 1 NOTICE: finished shard 9 (3 entries removed) 2026-02-24T22:23:25.711 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.717+0000 7f04e606ca80 1 NOTICE: finished all shards (3 entries removed) 2026-02-24T22:23:25.715 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.721+0000 7f04e606ca80 20 remove_watcher() i=5 2026-02-24T22:23:25.715 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.721+0000 7f04e606ca80 2 removed watcher, disabling cache 2026-02-24T22:23:25.715 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.721+0000 7f04e606ca80 20 remove_watcher() i=1 2026-02-24T22:23:25.715 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.721+0000 7f04e606ca80 20 remove_watcher() i=3 2026-02-24T22:23:25.715 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.721+0000 7f04e606ca80 20 remove_watcher() i=0 2026-02-24T22:23:25.715 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.721+0000 7f04e606ca80 20 remove_watcher() i=2 2026-02-24T22:23:25.715 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.721+0000 7f04e606ca80 20 remove_watcher() i=7 2026-02-24T22:23:25.715 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.721+0000 7f04e606ca80 20 remove_watcher() i=4 2026-02-24T22:23:25.715 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.721+0000 7f04e606ca80 20 remove_watcher() i=6 2026-02-24T22:23:25.722 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:23:25.722 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:23:25.725 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:23:25.725 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:23:25.770 INFO:tasks.workunit.client.0.trial046.stderr:radosgw-admin bucket stats --bucket myver 2026-02-24T22:23:25.785 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:25.785 INFO:tasks.workunit.client.0.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:25.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.809+0000 7f3f7d68da80 4 Realm: () 2026-02-24T22:23:25.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.809+0000 7f3f7d68da80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:25.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.809+0000 7f3f7d68da80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:25.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.809+0000 7f3f7d68da80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:25.802 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.809+0000 7f3f7d68da80 20 zonegroup default 2026-02-24T22:23:25.803 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.809+0000 7f3f7d68da80 20 rados->read ofs=0 len=0 2026-02-24T22:23:25.803 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.809+0000 7f3f7d68da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:25.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.809+0000 7f3f7d68da80 20 rados->read ofs=0 len=0 2026-02-24T22:23:25.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.809+0000 7f3f7d68da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:25.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.809+0000 7f3f7d68da80 20 started sync module instance, tier type = 2026-02-24T22:23:25.804 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.809+0000 7f3f7d68da80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:25.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.813+0000 7f3f7d68da80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:25.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.813+0000 7f3f7d68da80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:25.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.813+0000 7f3f7d68da80 5 note: GC not initialized 2026-02-24T22:23:25.806 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.813+0000 7f3f28fe1640 20 reqs_thread_entry: start 2026-02-24T22:23:25.810 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.817+0000 7f3f777fe640 0 -- 10.20.193.46:0/1363314015 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x5581e65bf6a0 msgr2=0x5581e65c5e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:25.815 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.821+0000 7f3f76ffd640 0 -- 10.20.193.46:0/1363314015 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5581e65c1890 msgr2=0x5581e65bc600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:23:25.841 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.845+0000 7f3f7d68da80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:25.841 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.845+0000 7f3f7d68da80 20 Filter name: none 2026-02-24T22:23:25.842 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.845+0000 7f3e9affd640 20 reqs_thread_entry: start 2026-02-24T22:23:25.842 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.845+0000 7f3f7d68da80 20 rados->read ofs=0 len=0 2026-02-24T22:23:25.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.849+0000 7f3f7d68da80 20 rados_obj.operate() r=0 bl.length=201 2026-02-24T22:23:25.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.849+0000 7f3f7d68da80 20 rados->read ofs=0 len=0 2026-02-24T22:23:25.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.849+0000 7f3f7d68da80 20 rados_obj.operate() r=0 bl.length=431 2026-02-24T22:23:25.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.849+0000 7f3f7d68da80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:23:25.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.849+0000 7f3f7d68da80 20 rados->read ofs=0 len=0 2026-02-24T22:23:25.843 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.849+0000 7f3f7d68da80 20 rados_obj.operate() r=0 bl.length=201 2026-02-24T22:23:25.844 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.849+0000 7f3f7d68da80 20 rados->read ofs=0 len=0 2026-02-24T22:23:25.844 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.849+0000 7f3f7d68da80 20 rados_obj.operate() r=0 bl.length=431 2026-02-24T22:23:25.844 INFO:tasks.workunit.client.0.trial046.stderr:2026-02-24T22:23:25.849+0000 7f3f7d68da80 20 couldn't put binfo cache entry, might have raced with data changes 2026-02-24T22:23:25.855 INFO:tasks.workunit.client.0.trial046.stderr:{'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}} 2026-02-24T22:23:25.855 INFO:tasks.workunit.client.0.trial046.stderr:Deleting bucket a-bucket 2026-02-24T22:23:25.856 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:23:25.856 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:23:25.859 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:23:25.860 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:23:25.870 INFO:tasks.workunit.client.0.trial046.stderr:Deleting bucket myver 2026-02-24T22:23:25.871 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:23:25.871 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:23:25.915 INFO:tasks.workunit.client.0.trial046.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:23:25.915 INFO:tasks.workunit.client.0.trial046.stderr: warnings.warn( 2026-02-24T22:23:25.939 INFO:tasks.workunit.client.0.trial046.stderr:Completed resharding tests 2026-02-24T22:23:25.939 INFO:tasks.workunit.client.0.trial046.stdout:Could not connect to the endpoint URL: "http://localhost:80/" 2026-02-24T22:23:25.939 INFO:tasks.workunit.client.0.trial046.stdout:Could not connect to the endpoint URL: "http://localhost:8000/" 2026-02-24T22:23:25.939 INFO:tasks.workunit.client.0.trial046.stdout:connected to https://localhost:443 2026-02-24T22:23:25.983 INFO:tasks.workunit.client.0.trial046.stderr:+ deactivate 2026-02-24T22:23:25.983 INFO:tasks.workunit.client.0.trial046.stderr:+ '[' -n /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin ']' 2026-02-24T22:23:25.984 INFO:tasks.workunit.client.0.trial046.stderr:+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T22:23:25.984 INFO:tasks.workunit.client.0.trial046.stderr:+ export PATH 2026-02-24T22:23:25.984 INFO:tasks.workunit.client.0.trial046.stderr:+ unset _OLD_VIRTUAL_PATH 2026-02-24T22:23:25.984 INFO:tasks.workunit.client.0.trial046.stderr:+ '[' -n '' ']' 2026-02-24T22:23:25.984 INFO:tasks.workunit.client.0.trial046.stderr:+ '[' -n /usr/bin/bash -o -n '' ']' 2026-02-24T22:23:25.984 INFO:tasks.workunit.client.0.trial046.stderr:+ hash -r 2026-02-24T22:23:25.984 INFO:tasks.workunit.client.0.trial046.stderr:+ '[' -n '' ']' 2026-02-24T22:23:25.984 INFO:tasks.workunit.client.0.trial046.stderr:+ unset VIRTUAL_ENV 2026-02-24T22:23:25.984 INFO:tasks.workunit.client.0.trial046.stderr:+ unset VIRTUAL_ENV_PROMPT 2026-02-24T22:23:25.984 INFO:tasks.workunit.client.0.trial046.stderr:+ '[' '!' '' = nondestructive ']' 2026-02-24T22:23:25.984 INFO:tasks.workunit.client.0.trial046.stderr:+ unset -f deactivate 2026-02-24T22:23:25.985 INFO:tasks.workunit.client.0.trial046.stderr:+ echo OK. 2026-02-24T22:23:25.985 INFO:tasks.workunit.client.0.trial046.stdout:OK. 2026-02-24T22:23:25.986 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T22:23:25.986 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T22:23:26.042 INFO:tasks.workunit:Stopping ['rgw/run-reshard.sh'] on client.0... 2026-02-24T22:23:26.042 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-24T22:23:26.504 DEBUG:teuthology.parallel:result is None 2026-02-24T22:23:26.505 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-24T22:23:26.513 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-24T22:23:26.514 DEBUG:teuthology.orchestra.run.trial046:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-24T22:23:26.561 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-24T22:23:26.561 INFO:teuthology.run_tasks:Running task s3tests... 2026-02-24T22:23:26.593 DEBUG:tasks.s3tests:s3tests config is {'client.0': {'rgw_server': 'client.0', 'accounts': {'iam alt root': 'RGW99999999999999999', 'iam root': 'RGW88888888888888888'}, 'force-branch': 'ceph-master', 'storage classes': 'LUKEWARM, FROZEN'}} 2026-02-24T22:23:26.594 INFO:tasks.s3tests:Downloading s3-tests... 2026-02-24T22:23:26.594 INFO:tasks.s3tests:Using branch 'ceph-master' for s3tests 2026-02-24T22:23:26.594 DEBUG:teuthology.orchestra.run.trial046:> git clone -b ceph-master https://git.ceph.com/s3-tests.git /home/ubuntu/cephtest/s3-tests-client.0 2026-02-24T22:23:26.608 INFO:teuthology.orchestra.run.trial046.stderr:Cloning into '/home/ubuntu/cephtest/s3-tests-client.0'... 2026-02-24T22:23:26.954 INFO:tasks.s3tests:Creating rgw users... 2026-02-24T22:23:26.955 DEBUG:tasks.s3tests:Creating user foo.client.0 on client.0 2026-02-24T22:23:26.956 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid foo.client.0 --display-name Mr.foo.client.0 --email foo+test@test.test --access-key MFYBMLOBGXKEVCZKEMRM --secret 8BNPkHCS4SHcYHEsxseJAvF2SMzi8FiBfgIeDXOeJcOmUpfB2Vh3jA== --cluster ceph 2026-02-24T22:23:27.017 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:27.017 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:27.032 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.037+0000 7f57fe057a80 4 Realm: () 2026-02-24T22:23:27.033 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.037+0000 7f57fe057a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:27.033 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.037+0000 7f57fe057a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:27.033 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.037+0000 7f57fe057a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:27.033 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.037+0000 7f57fe057a80 20 zonegroup default 2026-02-24T22:23:27.033 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.037+0000 7f57fe057a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:27.034 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.041+0000 7f57fe057a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:27.034 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.041+0000 7f57fe057a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:27.034 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.041+0000 7f57fe057a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:27.034 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.041+0000 7f57fe057a80 20 started sync module instance, tier type = 2026-02-24T22:23:27.035 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.041+0000 7f57fe057a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:27.035 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.041+0000 7f57f7fff640 0 -- 10.20.193.46:0/479681406 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x5620b4205c70 msgr2=0x5620b42060e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:27.036 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.041+0000 7f57fe057a80 20 add_watcher() i=1 2026-02-24T22:23:27.036 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.041+0000 7f57fe057a80 20 add_watcher() i=6 2026-02-24T22:23:27.036 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.041+0000 7f57fe057a80 20 add_watcher() i=3 2026-02-24T22:23:27.037 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.041+0000 7f57fe057a80 20 add_watcher() i=5 2026-02-24T22:23:27.037 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.041+0000 7f57fe057a80 20 add_watcher() i=0 2026-02-24T22:23:27.037 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.041+0000 7f57fe057a80 20 add_watcher() i=7 2026-02-24T22:23:27.037 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.041+0000 7f57fe057a80 20 add_watcher() i=2 2026-02-24T22:23:27.037 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.041+0000 7f57fe057a80 20 add_watcher() i=4 2026-02-24T22:23:27.037 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.041+0000 7f57fe057a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:23:27.038 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.045+0000 7f57fe057a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:27.039 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.045+0000 7f57fe057a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:27.039 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.045+0000 7f57fe057a80 5 note: GC not initialized 2026-02-24T22:23:27.039 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.045+0000 7f57a8fe1640 20 reqs_thread_entry: start 2026-02-24T22:23:27.039 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.045+0000 7f57f7fff640 0 -- 10.20.193.46:0/1326566924 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5620b43c7eb0 msgr2=0x5620b43c82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:27.279 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.285+0000 7f57fe057a80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:27.280 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.285+0000 7f57fe057a80 20 Filter name: none 2026-02-24T22:23:27.280 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.285+0000 7f5712ffd640 20 reqs_thread_entry: start 2026-02-24T22:23:27.280 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.285+0000 7f57fe057a80 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2026-02-24T22:23:27.280 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.285+0000 7f57fe057a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:27.281 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.285+0000 7f57fe057a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:27.281 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.285+0000 7f57fe057a80 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x0 2026-02-24T22:23:27.281 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.285+0000 7f57fe057a80 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2026-02-24T22:23:27.281 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.285+0000 7f57fe057a80 10 cache get: name=default.rgw.meta+users.email+foo+test@test.test : miss 2026-02-24T22:23:27.281 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.285+0000 7f57fe057a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:27.281 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.285+0000 7f57fe057a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:27.281 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.285+0000 7f57fe057a80 10 cache put: name=default.rgw.meta+users.email+foo+test@test.test info.flags=0x0 2026-02-24T22:23:27.281 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.285+0000 7f57fe057a80 10 adding default.rgw.meta+users.email+foo+test@test.test to cache LRU end 2026-02-24T22:23:27.282 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.285+0000 7f57fe057a80 10 cache get: name=default.rgw.meta+users.keys+MFYBMLOBGXKEVCZKEMRM : miss 2026-02-24T22:23:27.282 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.285+0000 7f57fe057a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:27.282 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.285+0000 7f57fe057a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:27.282 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.285+0000 7f57fe057a80 10 cache put: name=default.rgw.meta+users.keys+MFYBMLOBGXKEVCZKEMRM info.flags=0x0 2026-02-24T22:23:27.282 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.285+0000 7f57fe057a80 10 adding default.rgw.meta+users.keys+MFYBMLOBGXKEVCZKEMRM to cache LRU end 2026-02-24T22:23:27.282 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.285+0000 7f57fe057a80 10 cache get: name=default.rgw.meta+users.keys+MFYBMLOBGXKEVCZKEMRM : hit (negative entry) 2026-02-24T22:23:27.282 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.285+0000 7f57fe057a80 10 cache get: name=default.rgw.meta+users.keys+MFYBMLOBGXKEVCZKEMRM : hit (negative entry) 2026-02-24T22:23:27.283 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.289+0000 7f57fe057a80 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2026-02-24T22:23:27.283 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.289+0000 7f57fe057a80 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2026-02-24T22:23:27.283 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.289+0000 7f57fe057a80 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.uid:foo.client.0, ofs0, ns] 2026-02-24T22:23:27.283 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.289+0000 7f57d2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554961373 cookie 94698460978288 notifier 8417 bl.length()=645 2026-02-24T22:23:27.283 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.289+0000 7f57d2ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2026-02-24T22:23:27.283 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.289+0000 7f57d2ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2026-02-24T22:23:27.284 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.289+0000 7f57fe057a80 10 cache put: name=default.rgw.meta+users.email+foo+test@test.test info.flags=0x7 2026-02-24T22:23:27.284 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.289+0000 7f57fe057a80 10 moving default.rgw.meta+users.email+foo+test@test.test to cache LRU end 2026-02-24T22:23:27.284 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.289+0000 7f57fe057a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.email:foo+test@test.test, ofs0, ns] 2026-02-24T22:23:27.284 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.289+0000 7f57d27fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554960031 cookie 94698461798096 notifier 8417 bl.length()=185 2026-02-24T22:23:27.284 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.289+0000 7f57d27fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+foo+test@test.test info.flags=0x7 2026-02-24T22:23:27.284 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.289+0000 7f57d27fc640 10 rgw watcher librados: moving default.rgw.meta+users.email+foo+test@test.test to cache LRU end 2026-02-24T22:23:27.285 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.289+0000 7f57fe057a80 10 cache put: name=default.rgw.meta+users.keys+MFYBMLOBGXKEVCZKEMRM info.flags=0x7 2026-02-24T22:23:27.285 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.289+0000 7f57fe057a80 10 moving default.rgw.meta+users.keys+MFYBMLOBGXKEVCZKEMRM to cache LRU end 2026-02-24T22:23:27.285 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.289+0000 7f57fe057a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.keys:MFYBMLOBGXKEVCZKEMRM, ofs0, ns] 2026-02-24T22:23:27.285 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.289+0000 7f57d27fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1318554960032 cookie 94698461798096 notifier 8417 bl.length()=186 2026-02-24T22:23:27.285 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.289+0000 7f57d27fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+MFYBMLOBGXKEVCZKEMRM info.flags=0x7 2026-02-24T22:23:27.285 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.289+0000 7f57d27fc640 10 rgw watcher librados: moving default.rgw.meta+users.keys+MFYBMLOBGXKEVCZKEMRM to cache LRU end 2026-02-24T22:23:27.286 INFO:teuthology.orchestra.run.trial046.stdout:{ 2026-02-24T22:23:27.286 INFO:teuthology.orchestra.run.trial046.stdout: "user_id": "foo.client.0", 2026-02-24T22:23:27.286 INFO:teuthology.orchestra.run.trial046.stdout: "display_name": "Mr.foo.client.0", 2026-02-24T22:23:27.286 INFO:teuthology.orchestra.run.trial046.stdout: "email": "foo+test@test.test", 2026-02-24T22:23:27.286 INFO:teuthology.orchestra.run.trial046.stdout: "suspended": 0, 2026-02-24T22:23:27.286 INFO:teuthology.orchestra.run.trial046.stdout: "max_buckets": 1000, 2026-02-24T22:23:27.286 INFO:teuthology.orchestra.run.trial046.stdout: "subusers": [], 2026-02-24T22:23:27.286 INFO:teuthology.orchestra.run.trial046.stdout: "keys": [ 2026-02-24T22:23:27.286 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:23:27.287 INFO:teuthology.orchestra.run.trial046.stdout: "user": "foo.client.0", 2026-02-24T22:23:27.287 INFO:teuthology.orchestra.run.trial046.stdout: "access_key": "MFYBMLOBGXKEVCZKEMRM", 2026-02-24T22:23:27.287 INFO:teuthology.orchestra.run.trial046.stdout: "secret_key": "8BNPkHCS4SHcYHEsxseJAvF2SMzi8FiBfgIeDXOeJcOmUpfB2Vh3jA==", 2026-02-24T22:23:27.287 INFO:teuthology.orchestra.run.trial046.stdout: "active": true, 2026-02-24T22:23:27.287 INFO:teuthology.orchestra.run.trial046.stdout: "create_date": "2026-02-24T22:23:27.290458Z" 2026-02-24T22:23:27.287 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:23:27.287 INFO:teuthology.orchestra.run.trial046.stdout: ], 2026-02-24T22:23:27.287 INFO:teuthology.orchestra.run.trial046.stdout: "swift_keys": [], 2026-02-24T22:23:27.287 INFO:teuthology.orchestra.run.trial046.stdout: "caps": [], 2026-02-24T22:23:27.287 INFO:teuthology.orchestra.run.trial046.stdout: "op_mask": "read, write, delete", 2026-02-24T22:23:27.287 INFO:teuthology.orchestra.run.trial046.stdout: "default_placement": "", 2026-02-24T22:23:27.287 INFO:teuthology.orchestra.run.trial046.stdout: "default_storage_class": "", 2026-02-24T22:23:27.287 INFO:teuthology.orchestra.run.trial046.stdout: "placement_tags": [], 2026-02-24T22:23:27.288 INFO:teuthology.orchestra.run.trial046.stdout: "bucket_quota": { 2026-02-24T22:23:27.288 INFO:teuthology.orchestra.run.trial046.stdout: "enabled": false, 2026-02-24T22:23:27.288 INFO:teuthology.orchestra.run.trial046.stdout: "check_on_raw": false, 2026-02-24T22:23:27.288 INFO:teuthology.orchestra.run.trial046.stdout: "max_size": -1, 2026-02-24T22:23:27.288 INFO:teuthology.orchestra.run.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:23:27.288 INFO:teuthology.orchestra.run.trial046.stdout: "max_objects": -1 2026-02-24T22:23:27.288 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:23:27.288 INFO:teuthology.orchestra.run.trial046.stdout: "user_quota": { 2026-02-24T22:23:27.288 INFO:teuthology.orchestra.run.trial046.stdout: "enabled": false, 2026-02-24T22:23:27.288 INFO:teuthology.orchestra.run.trial046.stdout: "check_on_raw": false, 2026-02-24T22:23:27.288 INFO:teuthology.orchestra.run.trial046.stdout: "max_size": -1, 2026-02-24T22:23:27.288 INFO:teuthology.orchestra.run.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:23:27.288 INFO:teuthology.orchestra.run.trial046.stdout: "max_objects": -1 2026-02-24T22:23:27.289 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:23:27.289 INFO:teuthology.orchestra.run.trial046.stdout: "temp_url_keys": [], 2026-02-24T22:23:27.289 INFO:teuthology.orchestra.run.trial046.stdout: "type": "rgw", 2026-02-24T22:23:27.289 INFO:teuthology.orchestra.run.trial046.stdout: "mfa_ids": [], 2026-02-24T22:23:27.289 INFO:teuthology.orchestra.run.trial046.stdout: "account_id": "", 2026-02-24T22:23:27.289 INFO:teuthology.orchestra.run.trial046.stdout: "path": "/", 2026-02-24T22:23:27.289 INFO:teuthology.orchestra.run.trial046.stdout: "create_date": "2026-02-24T22:23:27.290447Z", 2026-02-24T22:23:27.289 INFO:teuthology.orchestra.run.trial046.stdout: "tags": [], 2026-02-24T22:23:27.289 INFO:teuthology.orchestra.run.trial046.stdout: "group_ids": [] 2026-02-24T22:23:27.289 INFO:teuthology.orchestra.run.trial046.stdout:} 2026-02-24T22:23:27.289 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:23:27.290 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.293+0000 7f57fe057a80 20 remove_watcher() i=3 2026-02-24T22:23:27.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.293+0000 7f57fe057a80 2 removed watcher, disabling cache 2026-02-24T22:23:27.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.293+0000 7f57fe057a80 20 remove_watcher() i=2 2026-02-24T22:23:27.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.293+0000 7f57fe057a80 20 remove_watcher() i=0 2026-02-24T22:23:27.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.293+0000 7f57fe057a80 20 remove_watcher() i=6 2026-02-24T22:23:27.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.293+0000 7f57fe057a80 20 remove_watcher() i=5 2026-02-24T22:23:27.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.293+0000 7f57fe057a80 20 remove_watcher() i=4 2026-02-24T22:23:27.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.293+0000 7f57fe057a80 20 remove_watcher() i=1 2026-02-24T22:23:27.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.293+0000 7f57fe057a80 20 remove_watcher() i=7 2026-02-24T22:23:27.295 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid foo.client.0 --totp-serial 7624616313 --totp-seed M7RDRI6M5ELLDYZOTPLFUQYRGJVPNKOQASRR2DKK5FXTTYG7FWQAFMUICNZERVEN --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2026-02-24T22:23:27.358 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:27.358 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:27.374 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.381+0000 7f9bfd86da80 4 Realm: () 2026-02-24T22:23:27.374 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.381+0000 7f9bfd86da80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:27.374 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.381+0000 7f9bfd86da80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:27.374 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.381+0000 7f9bfd86da80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:27.374 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.381+0000 7f9bfd86da80 20 zonegroup default 2026-02-24T22:23:27.375 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.381+0000 7f9bfd86da80 20 rados->read ofs=0 len=0 2026-02-24T22:23:27.375 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.381+0000 7f9bfd86da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:27.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.381+0000 7f9bfd86da80 20 rados->read ofs=0 len=0 2026-02-24T22:23:27.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.381+0000 7f9bfd86da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:27.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.381+0000 7f9bfd86da80 20 started sync module instance, tier type = 2026-02-24T22:23:27.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.381+0000 7f9bfd86da80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:27.377 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.381+0000 7f9bf77fe640 0 -- 10.20.193.46:0/2676989943 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x56071c8361c0 msgr2=0x56071c836630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:27.378 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.385+0000 7f9bfd86da80 20 add_watcher() i=1 2026-02-24T22:23:27.378 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.385+0000 7f9bfd86da80 20 add_watcher() i=6 2026-02-24T22:23:27.379 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.385+0000 7f9bfd86da80 20 add_watcher() i=5 2026-02-24T22:23:27.379 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.385+0000 7f9bf77fe640 0 -- 10.20.193.46:0/2676989943 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x7f9bbc09d4e0 msgr2=0x7f9bbc0bd8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:27.379 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.385+0000 7f9bfd86da80 20 add_watcher() i=0 2026-02-24T22:23:27.379 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.385+0000 7f9bfd86da80 20 add_watcher() i=4 2026-02-24T22:23:27.379 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.385+0000 7f9bfd86da80 20 add_watcher() i=2 2026-02-24T22:23:27.379 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.385+0000 7f9bfd86da80 20 add_watcher() i=7 2026-02-24T22:23:27.380 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.385+0000 7f9bfd86da80 20 add_watcher() i=3 2026-02-24T22:23:27.380 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.385+0000 7f9bfd86da80 2 all 8 watchers are set, enabling cache 2026-02-24T22:23:27.381 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.385+0000 7f9bfd86da80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:27.381 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.385+0000 7f9bfd86da80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:27.381 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.385+0000 7f9bfd86da80 5 note: GC not initialized 2026-02-24T22:23:27.381 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.385+0000 7f9b13fff640 20 reqs_thread_entry: start 2026-02-24T22:23:27.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.421+0000 7f9bfd86da80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:27.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.421+0000 7f9bfd86da80 20 Filter name: none 2026-02-24T22:23:27.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.421+0000 7f9b127fc640 20 reqs_thread_entry: start 2026-02-24T22:23:27.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.421+0000 7f9bfd86da80 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2026-02-24T22:23:27.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.421+0000 7f9bfd86da80 20 rados->read ofs=0 len=0 2026-02-24T22:23:27.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.421+0000 7f9bfd86da80 20 rados_obj.operate() r=0 bl.length=460 2026-02-24T22:23:27.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.421+0000 7f9bfd86da80 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x13 2026-02-24T22:23:27.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.421+0000 7f9bfd86da80 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2026-02-24T22:23:27.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:27.421+0000 7f9bfd86da80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:23:29.207 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.213+0000 7f9bfd86da80 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2026-02-24T22:23:29.208 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.213+0000 7f9bfd86da80 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2026-02-24T22:23:29.208 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.213+0000 7f9bfd86da80 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.uid:foo.client.0, ofs0, ns] 2026-02-24T22:23:29.208 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.213+0000 7f9bd5ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1327144895966 cookie 94588543116000 notifier 8432 bl.length()=659 2026-02-24T22:23:29.208 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.213+0000 7f9bd5ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2026-02-24T22:23:29.208 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.213+0000 7f9bd5ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2026-02-24T22:23:29.211 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.217+0000 7f9bfd86da80 20 remove_watcher() i=2 2026-02-24T22:23:29.211 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.217+0000 7f9bfd86da80 2 removed watcher, disabling cache 2026-02-24T22:23:29.211 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.217+0000 7f9bfd86da80 20 remove_watcher() i=3 2026-02-24T22:23:29.211 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.217+0000 7f9bfd86da80 20 remove_watcher() i=0 2026-02-24T22:23:29.211 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.217+0000 7f9bfd86da80 20 remove_watcher() i=7 2026-02-24T22:23:29.212 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.217+0000 7f9bfd86da80 20 remove_watcher() i=4 2026-02-24T22:23:29.212 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.217+0000 7f9bfd86da80 20 remove_watcher() i=1 2026-02-24T22:23:29.212 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.217+0000 7f9bfd86da80 20 remove_watcher() i=5 2026-02-24T22:23:29.212 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.217+0000 7f9bfd86da80 20 remove_watcher() i=6 2026-02-24T22:23:29.218 DEBUG:tasks.s3tests:Creating user bar.client.0 on client.0 2026-02-24T22:23:29.218 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid bar.client.0 --display-name Mr.bar.client.0 --email bar+test@test.test --access-key NJNOTGSXWYZXXIZADMEE --secret 9m/Hj+eImiNvucM2Bbp8DF8eF0fAm2lIa56VbfvQnED1+1nUob48uQ== --cluster ceph 2026-02-24T22:23:29.282 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:29.282 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:29.296 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.301+0000 7fb863f35a80 4 Realm: () 2026-02-24T22:23:29.296 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.301+0000 7fb863f35a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:29.297 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.301+0000 7fb863f35a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:29.297 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.301+0000 7fb863f35a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:29.297 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.301+0000 7fb863f35a80 20 zonegroup default 2026-02-24T22:23:29.297 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.301+0000 7fb863f35a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:29.298 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.305+0000 7fb863f35a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:29.298 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.305+0000 7fb863f35a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:29.298 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.305+0000 7fb863f35a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:29.298 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.305+0000 7fb863f35a80 20 started sync module instance, tier type = 2026-02-24T22:23:29.298 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.305+0000 7fb863f35a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:29.299 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.305+0000 7fb86220a640 0 -- 10.20.193.46:0/2612715466 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x5570722e22a0 msgr2=0x5570722e2710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:29.300 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.305+0000 7fb863f35a80 20 add_watcher() i=1 2026-02-24T22:23:29.300 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.305+0000 7fb863f35a80 20 add_watcher() i=6 2026-02-24T22:23:29.300 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.305+0000 7fb863f35a80 20 add_watcher() i=5 2026-02-24T22:23:29.301 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.305+0000 7fb863f35a80 20 add_watcher() i=2 2026-02-24T22:23:29.301 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.305+0000 7fb863f35a80 20 add_watcher() i=3 2026-02-24T22:23:29.301 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.305+0000 7fb863f35a80 20 add_watcher() i=0 2026-02-24T22:23:29.301 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.305+0000 7fb863f35a80 20 add_watcher() i=7 2026-02-24T22:23:29.301 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.305+0000 7fb863f35a80 20 add_watcher() i=4 2026-02-24T22:23:29.301 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.305+0000 7fb863f35a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:23:29.302 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.309+0000 7fb863f35a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:29.302 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.309+0000 7fb863f35a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:29.302 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.309+0000 7fb863f35a80 5 note: GC not initialized 2026-02-24T22:23:29.302 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.309+0000 7fb7927fc640 20 reqs_thread_entry: start 2026-02-24T22:23:29.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.309+0000 7fb86220a640 0 -- 10.20.193.46:0/1501718042 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5570724a5e20 msgr2=0x5570724a6290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:29.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.549+0000 7fb863f35a80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:29.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.549+0000 7fb863f35a80 20 Filter name: none 2026-02-24T22:23:29.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.549+0000 7fb790ff9640 20 reqs_thread_entry: start 2026-02-24T22:23:29.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.549+0000 7fb863f35a80 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2026-02-24T22:23:29.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.549+0000 7fb863f35a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:29.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.549+0000 7fb863f35a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:29.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.549+0000 7fb863f35a80 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x0 2026-02-24T22:23:29.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.549+0000 7fb863f35a80 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2026-02-24T22:23:29.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.549+0000 7fb863f35a80 10 cache get: name=default.rgw.meta+users.email+bar+test@test.test : miss 2026-02-24T22:23:29.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.549+0000 7fb863f35a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:29.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.549+0000 7fb863f35a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:29.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.549+0000 7fb863f35a80 10 cache put: name=default.rgw.meta+users.email+bar+test@test.test info.flags=0x0 2026-02-24T22:23:29.546 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.549+0000 7fb863f35a80 10 adding default.rgw.meta+users.email+bar+test@test.test to cache LRU end 2026-02-24T22:23:29.546 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.549+0000 7fb863f35a80 10 cache get: name=default.rgw.meta+users.keys+NJNOTGSXWYZXXIZADMEE : miss 2026-02-24T22:23:29.546 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.549+0000 7fb863f35a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:29.546 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.549+0000 7fb863f35a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:29.546 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.549+0000 7fb863f35a80 10 cache put: name=default.rgw.meta+users.keys+NJNOTGSXWYZXXIZADMEE info.flags=0x0 2026-02-24T22:23:29.546 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.549+0000 7fb863f35a80 10 adding default.rgw.meta+users.keys+NJNOTGSXWYZXXIZADMEE to cache LRU end 2026-02-24T22:23:29.546 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.549+0000 7fb863f35a80 10 cache get: name=default.rgw.meta+users.keys+NJNOTGSXWYZXXIZADMEE : hit (negative entry) 2026-02-24T22:23:29.546 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.549+0000 7fb863f35a80 10 cache get: name=default.rgw.meta+users.keys+NJNOTGSXWYZXXIZADMEE : hit (negative entry) 2026-02-24T22:23:29.547 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.553+0000 7fb863f35a80 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2026-02-24T22:23:29.547 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.553+0000 7fb863f35a80 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2026-02-24T22:23:29.547 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.553+0000 7fb863f35a80 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:bar.client.0, ofs0, ns] 2026-02-24T22:23:29.547 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.553+0000 7fb83d7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1327144894594 cookie 93941440328416 notifier 8450 bl.length()=645 2026-02-24T22:23:29.547 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.553+0000 7fb83d7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2026-02-24T22:23:29.547 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.553+0000 7fb83d7fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2026-02-24T22:23:29.548 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.553+0000 7fb863f35a80 10 cache put: name=default.rgw.meta+users.email+bar+test@test.test info.flags=0x7 2026-02-24T22:23:29.548 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.553+0000 7fb863f35a80 10 moving default.rgw.meta+users.email+bar+test@test.test to cache LRU end 2026-02-24T22:23:29.548 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.553+0000 7fb863f35a80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.email:bar+test@test.test, ofs0, ns] 2026-02-24T22:23:29.548 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.553+0000 7fb83d7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1327144895991 cookie 93941440340608 notifier 8450 bl.length()=185 2026-02-24T22:23:29.548 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.553+0000 7fb83d7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+bar+test@test.test info.flags=0x7 2026-02-24T22:23:29.548 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.553+0000 7fb83d7fa640 10 rgw watcher librados: moving default.rgw.meta+users.email+bar+test@test.test to cache LRU end 2026-02-24T22:23:29.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.553+0000 7fb863f35a80 10 cache put: name=default.rgw.meta+users.keys+NJNOTGSXWYZXXIZADMEE info.flags=0x7 2026-02-24T22:23:29.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.553+0000 7fb863f35a80 10 moving default.rgw.meta+users.keys+NJNOTGSXWYZXXIZADMEE to cache LRU end 2026-02-24T22:23:29.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.553+0000 7fb863f35a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.keys:NJNOTGSXWYZXXIZADMEE, ofs0, ns] 2026-02-24T22:23:29.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.553+0000 7fb83cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1327144894625 cookie 93941440344288 notifier 8450 bl.length()=186 2026-02-24T22:23:29.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.553+0000 7fb83cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+NJNOTGSXWYZXXIZADMEE info.flags=0x7 2026-02-24T22:23:29.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.553+0000 7fb83cff9640 10 rgw watcher librados: moving default.rgw.meta+users.keys+NJNOTGSXWYZXXIZADMEE to cache LRU end 2026-02-24T22:23:29.550 INFO:teuthology.orchestra.run.trial046.stdout:{ 2026-02-24T22:23:29.550 INFO:teuthology.orchestra.run.trial046.stdout: "user_id": "bar.client.0", 2026-02-24T22:23:29.550 INFO:teuthology.orchestra.run.trial046.stdout: "display_name": "Mr.bar.client.0", 2026-02-24T22:23:29.550 INFO:teuthology.orchestra.run.trial046.stdout: "email": "bar+test@test.test", 2026-02-24T22:23:29.550 INFO:teuthology.orchestra.run.trial046.stdout: "suspended": 0, 2026-02-24T22:23:29.550 INFO:teuthology.orchestra.run.trial046.stdout: "max_buckets": 1000, 2026-02-24T22:23:29.550 INFO:teuthology.orchestra.run.trial046.stdout: "subusers": [], 2026-02-24T22:23:29.550 INFO:teuthology.orchestra.run.trial046.stdout: "keys": [ 2026-02-24T22:23:29.550 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:23:29.551 INFO:teuthology.orchestra.run.trial046.stdout: "user": "bar.client.0", 2026-02-24T22:23:29.551 INFO:teuthology.orchestra.run.trial046.stdout: "access_key": "NJNOTGSXWYZXXIZADMEE", 2026-02-24T22:23:29.551 INFO:teuthology.orchestra.run.trial046.stdout: "secret_key": "9m/Hj+eImiNvucM2Bbp8DF8eF0fAm2lIa56VbfvQnED1+1nUob48uQ==", 2026-02-24T22:23:29.551 INFO:teuthology.orchestra.run.trial046.stdout: "active": true, 2026-02-24T22:23:29.551 INFO:teuthology.orchestra.run.trial046.stdout: "create_date": "2026-02-24T22:23:29.554730Z" 2026-02-24T22:23:29.551 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:23:29.551 INFO:teuthology.orchestra.run.trial046.stdout: ], 2026-02-24T22:23:29.551 INFO:teuthology.orchestra.run.trial046.stdout: "swift_keys": [], 2026-02-24T22:23:29.551 INFO:teuthology.orchestra.run.trial046.stdout: "caps": [], 2026-02-24T22:23:29.551 INFO:teuthology.orchestra.run.trial046.stdout: "op_mask": "read, write, delete", 2026-02-24T22:23:29.551 INFO:teuthology.orchestra.run.trial046.stdout: "default_placement": "", 2026-02-24T22:23:29.551 INFO:teuthology.orchestra.run.trial046.stdout: "default_storage_class": "", 2026-02-24T22:23:29.552 INFO:teuthology.orchestra.run.trial046.stdout: "placement_tags": [], 2026-02-24T22:23:29.552 INFO:teuthology.orchestra.run.trial046.stdout: "bucket_quota": { 2026-02-24T22:23:29.552 INFO:teuthology.orchestra.run.trial046.stdout: "enabled": false, 2026-02-24T22:23:29.552 INFO:teuthology.orchestra.run.trial046.stdout: "check_on_raw": false, 2026-02-24T22:23:29.552 INFO:teuthology.orchestra.run.trial046.stdout: "max_size": -1, 2026-02-24T22:23:29.552 INFO:teuthology.orchestra.run.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:23:29.552 INFO:teuthology.orchestra.run.trial046.stdout: "max_objects": -1 2026-02-24T22:23:29.552 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:23:29.552 INFO:teuthology.orchestra.run.trial046.stdout: "user_quota": { 2026-02-24T22:23:29.552 INFO:teuthology.orchestra.run.trial046.stdout: "enabled": false, 2026-02-24T22:23:29.552 INFO:teuthology.orchestra.run.trial046.stdout: "check_on_raw": false, 2026-02-24T22:23:29.552 INFO:teuthology.orchestra.run.trial046.stdout: "max_size": -1, 2026-02-24T22:23:29.552 INFO:teuthology.orchestra.run.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:23:29.552 INFO:teuthology.orchestra.run.trial046.stdout: "max_objects": -1 2026-02-24T22:23:29.553 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:23:29.553 INFO:teuthology.orchestra.run.trial046.stdout: "temp_url_keys": [], 2026-02-24T22:23:29.553 INFO:teuthology.orchestra.run.trial046.stdout: "type": "rgw", 2026-02-24T22:23:29.553 INFO:teuthology.orchestra.run.trial046.stdout: "mfa_ids": [], 2026-02-24T22:23:29.553 INFO:teuthology.orchestra.run.trial046.stdout: "account_id": "", 2026-02-24T22:23:29.553 INFO:teuthology.orchestra.run.trial046.stdout: "path": "/", 2026-02-24T22:23:29.553 INFO:teuthology.orchestra.run.trial046.stdout: "create_date": "2026-02-24T22:23:29.554720Z", 2026-02-24T22:23:29.553 INFO:teuthology.orchestra.run.trial046.stdout: "tags": [], 2026-02-24T22:23:29.553 INFO:teuthology.orchestra.run.trial046.stdout: "group_ids": [] 2026-02-24T22:23:29.553 INFO:teuthology.orchestra.run.trial046.stdout:} 2026-02-24T22:23:29.553 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:23:29.554 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.557+0000 7fb863f35a80 20 remove_watcher() i=5 2026-02-24T22:23:29.555 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.557+0000 7fb863f35a80 2 removed watcher, disabling cache 2026-02-24T22:23:29.555 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.557+0000 7fb863f35a80 20 remove_watcher() i=2 2026-02-24T22:23:29.555 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.557+0000 7fb863f35a80 20 remove_watcher() i=3 2026-02-24T22:23:29.555 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.557+0000 7fb863f35a80 20 remove_watcher() i=0 2026-02-24T22:23:29.555 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.557+0000 7fb863f35a80 20 remove_watcher() i=1 2026-02-24T22:23:29.555 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.557+0000 7fb863f35a80 20 remove_watcher() i=7 2026-02-24T22:23:29.555 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.557+0000 7fb863f35a80 20 remove_watcher() i=6 2026-02-24T22:23:29.555 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.557+0000 7fb863f35a80 20 remove_watcher() i=4 2026-02-24T22:23:29.560 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid bar.client.0 --totp-serial 1556570504 --totp-seed EZKAKKPIKR2KFTLA2L3LM4ZZF5KHJ5256NHMNHEAHHCISMSR3JC2ZPHWPDIZXZB6 --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2026-02-24T22:23:29.623 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:29.623 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:29.638 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.645+0000 7f01ba3efa80 4 Realm: () 2026-02-24T22:23:29.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.645+0000 7f01ba3efa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:29.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.645+0000 7f01ba3efa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:29.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.645+0000 7f01ba3efa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:29.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.645+0000 7f01ba3efa80 20 zonegroup default 2026-02-24T22:23:29.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.645+0000 7f01ba3efa80 20 rados->read ofs=0 len=0 2026-02-24T22:23:29.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.645+0000 7f01ba3efa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:29.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.645+0000 7f01ba3efa80 20 rados->read ofs=0 len=0 2026-02-24T22:23:29.641 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.645+0000 7f01ba3efa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:29.641 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.645+0000 7f01ba3efa80 20 started sync module instance, tier type = 2026-02-24T22:23:29.641 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.645+0000 7f01ba3efa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:29.641 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.649+0000 7f01b2ffd640 0 -- 10.20.193.46:0/3243064687 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x56199ea8f080 msgr2=0x56199ea8f4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:29.642 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.649+0000 7f01ba3efa80 20 add_watcher() i=1 2026-02-24T22:23:29.642 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.649+0000 7f01ba3efa80 20 add_watcher() i=6 2026-02-24T22:23:29.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.649+0000 7f01ba3efa80 20 add_watcher() i=3 2026-02-24T22:23:29.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.649+0000 7f01ba3efa80 20 add_watcher() i=0 2026-02-24T22:23:29.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.649+0000 7f01ba3efa80 20 add_watcher() i=5 2026-02-24T22:23:29.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.649+0000 7f01ba3efa80 20 add_watcher() i=7 2026-02-24T22:23:29.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.649+0000 7f01ba3efa80 20 add_watcher() i=2 2026-02-24T22:23:29.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.649+0000 7f01ba3efa80 20 add_watcher() i=4 2026-02-24T22:23:29.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.649+0000 7f01ba3efa80 2 all 8 watchers are set, enabling cache 2026-02-24T22:23:29.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.649+0000 7f01b2ffd640 0 -- 10.20.193.46:0/833704222 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f00d8008af0 msgr2=0x7f00d8028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:23:29.848 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.853+0000 7f01ba3efa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:29.848 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.853+0000 7f01ba3efa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:29.848 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.853+0000 7f01ba3efa80 5 note: GC not initialized 2026-02-24T22:23:29.848 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.853+0000 7f0164fe1640 20 reqs_thread_entry: start 2026-02-24T22:23:29.885 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.893+0000 7f01ba3efa80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:29.886 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.893+0000 7f01ba3efa80 20 Filter name: none 2026-02-24T22:23:29.886 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.893+0000 7f00ceffd640 20 reqs_thread_entry: start 2026-02-24T22:23:29.886 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.893+0000 7f01ba3efa80 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2026-02-24T22:23:29.886 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.893+0000 7f01ba3efa80 20 rados->read ofs=0 len=0 2026-02-24T22:23:29.886 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.893+0000 7f01ba3efa80 20 rados_obj.operate() r=0 bl.length=460 2026-02-24T22:23:29.887 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.893+0000 7f01ba3efa80 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x13 2026-02-24T22:23:29.887 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.893+0000 7f01ba3efa80 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2026-02-24T22:23:29.887 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.893+0000 7f01ba3efa80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:23:29.888 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.893+0000 7f01ba3efa80 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2026-02-24T22:23:29.888 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.893+0000 7f01ba3efa80 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2026-02-24T22:23:29.888 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.893+0000 7f01ba3efa80 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:bar.client.0, ofs0, ns] 2026-02-24T22:23:29.888 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.893+0000 7f018e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1327144894595 cookie 94668036024096 notifier 8465 bl.length()=659 2026-02-24T22:23:29.888 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.893+0000 7f018e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2026-02-24T22:23:29.889 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.893+0000 7f018e7fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2026-02-24T22:23:29.892 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.897+0000 7f01ba3efa80 20 remove_watcher() i=1 2026-02-24T22:23:29.892 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.897+0000 7f01ba3efa80 2 removed watcher, disabling cache 2026-02-24T22:23:29.892 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.897+0000 7f01ba3efa80 20 remove_watcher() i=2 2026-02-24T22:23:29.892 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.897+0000 7f01ba3efa80 20 remove_watcher() i=3 2026-02-24T22:23:29.892 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.897+0000 7f01ba3efa80 20 remove_watcher() i=4 2026-02-24T22:23:29.892 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.897+0000 7f01ba3efa80 20 remove_watcher() i=7 2026-02-24T22:23:29.892 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.897+0000 7f01ba3efa80 20 remove_watcher() i=0 2026-02-24T22:23:29.892 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.897+0000 7f01ba3efa80 20 remove_watcher() i=6 2026-02-24T22:23:29.892 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.897+0000 7f01ba3efa80 20 remove_watcher() i=5 2026-02-24T22:23:29.899 DEBUG:tasks.s3tests:Creating user testx$tenanteduser.client.0 on client.0 2026-02-24T22:23:29.899 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid 'testx$tenanteduser.client.0' --display-name 'Mr.testx$tenanteduser.client.0' --email 'testx$tenanteduser+test@test.test' --access-key UQSXPKRUJTAZNKGXUOWD --secret cP6u+YC7eldI12PXQLNFqgi/IJwRFm479kquXLm+eqEl6SKOgerREA== --cluster ceph 2026-02-24T22:23:29.966 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:29.967 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:29.983 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.989+0000 7ffb84564a80 4 Realm: () 2026-02-24T22:23:29.983 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.989+0000 7ffb84564a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:29.983 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.989+0000 7ffb84564a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:29.983 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.989+0000 7ffb84564a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:29.983 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.989+0000 7ffb84564a80 20 zonegroup default 2026-02-24T22:23:29.984 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.989+0000 7ffb84564a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:29.984 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.989+0000 7ffb84564a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:29.984 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.989+0000 7ffb84564a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:29.985 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.989+0000 7ffb84564a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:29.985 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.989+0000 7ffb84564a80 20 started sync module instance, tier type = 2026-02-24T22:23:29.985 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.989+0000 7ffb84564a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:29.986 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.993+0000 7ffb82839640 0 -- 10.20.193.46:0/1187873317 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55b197cf1650 msgr2=0x55b197cf1ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:29.986 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.993+0000 7ffb84564a80 20 add_watcher() i=1 2026-02-24T22:23:29.987 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.993+0000 7ffb84564a80 20 add_watcher() i=6 2026-02-24T22:23:29.987 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.993+0000 7ffb84564a80 20 add_watcher() i=3 2026-02-24T22:23:29.987 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.993+0000 7ffb84564a80 20 add_watcher() i=5 2026-02-24T22:23:29.987 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.993+0000 7ffb84564a80 20 add_watcher() i=0 2026-02-24T22:23:29.988 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.993+0000 7ffb84564a80 20 add_watcher() i=2 2026-02-24T22:23:29.988 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.993+0000 7ffb84564a80 20 add_watcher() i=7 2026-02-24T22:23:29.988 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.993+0000 7ffb84564a80 20 add_watcher() i=4 2026-02-24T22:23:29.988 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.993+0000 7ffb84564a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:23:29.989 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.993+0000 7ffb84564a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:29.989 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.993+0000 7ffb84564a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:29.989 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.993+0000 7ffb84564a80 5 note: GC not initialized 2026-02-24T22:23:29.989 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.993+0000 7ffa9b7fe640 20 reqs_thread_entry: start 2026-02-24T22:23:29.990 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:29.997+0000 7ffb82839640 0 -- 10.20.193.46:0/598226643 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55b197ef5120 msgr2=0x55b197ef5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:30.237 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.241+0000 7ffb84564a80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:30.237 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.241+0000 7ffb84564a80 20 Filter name: none 2026-02-24T22:23:30.237 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.241+0000 7ffa99ffb640 20 reqs_thread_entry: start 2026-02-24T22:23:30.237 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.241+0000 7ffb84564a80 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2026-02-24T22:23:30.237 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.241+0000 7ffb84564a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:30.238 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.241+0000 7ffb84564a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:30.238 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.241+0000 7ffb84564a80 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x0 2026-02-24T22:23:30.238 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.241+0000 7ffb84564a80 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2026-02-24T22:23:30.238 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.241+0000 7ffb84564a80 10 cache get: name=default.rgw.meta+users.email+testx$tenanteduser+test@test.test : miss 2026-02-24T22:23:30.239 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.241+0000 7ffb84564a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:30.239 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.245+0000 7ffb84564a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:30.239 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.245+0000 7ffb84564a80 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser+test@test.test info.flags=0x0 2026-02-24T22:23:30.239 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.245+0000 7ffb84564a80 10 adding default.rgw.meta+users.email+testx$tenanteduser+test@test.test to cache LRU end 2026-02-24T22:23:30.239 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.245+0000 7ffb84564a80 10 cache get: name=default.rgw.meta+users.keys+UQSXPKRUJTAZNKGXUOWD : miss 2026-02-24T22:23:30.239 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.245+0000 7ffb84564a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:30.239 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.245+0000 7ffb84564a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:30.239 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.245+0000 7ffb84564a80 10 cache put: name=default.rgw.meta+users.keys+UQSXPKRUJTAZNKGXUOWD info.flags=0x0 2026-02-24T22:23:30.239 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.245+0000 7ffb84564a80 10 adding default.rgw.meta+users.keys+UQSXPKRUJTAZNKGXUOWD to cache LRU end 2026-02-24T22:23:30.239 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.245+0000 7ffb84564a80 10 cache get: name=default.rgw.meta+users.keys+UQSXPKRUJTAZNKGXUOWD : hit (negative entry) 2026-02-24T22:23:30.239 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.245+0000 7ffb84564a80 10 cache get: name=default.rgw.meta+users.keys+UQSXPKRUJTAZNKGXUOWD : hit (negative entry) 2026-02-24T22:23:30.241 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.245+0000 7ffb84564a80 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2026-02-24T22:23:30.241 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.245+0000 7ffb84564a80 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2026-02-24T22:23:30.241 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.245+0000 7ffb84564a80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:testx$tenanteduser.client.0, ofs0, ns] 2026-02-24T22:23:30.241 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.245+0000 7ffb5d7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1331439863288 cookie 94221245334352 notifier 8483 bl.length()=719 2026-02-24T22:23:30.241 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.245+0000 7ffb5d7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2026-02-24T22:23:30.241 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.245+0000 7ffb5d7fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2026-02-24T22:23:30.241 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.245+0000 7ffb84564a80 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser+test@test.test info.flags=0x7 2026-02-24T22:23:30.241 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.245+0000 7ffb84564a80 10 moving default.rgw.meta+users.email+testx$tenanteduser+test@test.test to cache LRU end 2026-02-24T22:23:30.241 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.245+0000 7ffb84564a80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.email:testx$tenanteduser+test@test.test, ofs0, ns] 2026-02-24T22:23:30.241 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.245+0000 7ffb5dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1331439861812 cookie 94221246776704 notifier 8483 bl.length()=215 2026-02-24T22:23:30.241 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.245+0000 7ffb5dffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+testx$tenanteduser+test@test.test info.flags=0x7 2026-02-24T22:23:30.242 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.245+0000 7ffb5dffb640 10 rgw watcher librados: moving default.rgw.meta+users.email+testx$tenanteduser+test@test.test to cache LRU end 2026-02-24T22:23:30.242 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.249+0000 7ffb84564a80 10 cache put: name=default.rgw.meta+users.keys+UQSXPKRUJTAZNKGXUOWD info.flags=0x7 2026-02-24T22:23:30.242 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.249+0000 7ffb84564a80 10 moving default.rgw.meta+users.keys+UQSXPKRUJTAZNKGXUOWD to cache LRU end 2026-02-24T22:23:30.243 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.249+0000 7ffb84564a80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:UQSXPKRUJTAZNKGXUOWD, ofs0, ns] 2026-02-24T22:23:30.243 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.249+0000 7ffb5dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1331439863289 cookie 94221245334352 notifier 8483 bl.length()=201 2026-02-24T22:23:30.243 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.249+0000 7ffb5dffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+UQSXPKRUJTAZNKGXUOWD info.flags=0x7 2026-02-24T22:23:30.243 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.249+0000 7ffb5dffb640 10 rgw watcher librados: moving default.rgw.meta+users.keys+UQSXPKRUJTAZNKGXUOWD to cache LRU end 2026-02-24T22:23:30.244 INFO:teuthology.orchestra.run.trial046.stdout:{ 2026-02-24T22:23:30.244 INFO:teuthology.orchestra.run.trial046.stdout: "user_id": "testx$tenanteduser.client.0", 2026-02-24T22:23:30.244 INFO:teuthology.orchestra.run.trial046.stdout: "display_name": "Mr.testx$tenanteduser.client.0", 2026-02-24T22:23:30.244 INFO:teuthology.orchestra.run.trial046.stdout: "email": "testx$tenanteduser+test@test.test", 2026-02-24T22:23:30.244 INFO:teuthology.orchestra.run.trial046.stdout: "suspended": 0, 2026-02-24T22:23:30.244 INFO:teuthology.orchestra.run.trial046.stdout: "max_buckets": 1000, 2026-02-24T22:23:30.244 INFO:teuthology.orchestra.run.trial046.stdout: "subusers": [], 2026-02-24T22:23:30.244 INFO:teuthology.orchestra.run.trial046.stdout: "keys": [ 2026-02-24T22:23:30.244 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:23:30.244 INFO:teuthology.orchestra.run.trial046.stdout: "user": "testx$tenanteduser.client.0", 2026-02-24T22:23:30.244 INFO:teuthology.orchestra.run.trial046.stdout: "access_key": "UQSXPKRUJTAZNKGXUOWD", 2026-02-24T22:23:30.245 INFO:teuthology.orchestra.run.trial046.stdout: "secret_key": "cP6u+YC7eldI12PXQLNFqgi/IJwRFm479kquXLm+eqEl6SKOgerREA==", 2026-02-24T22:23:30.245 INFO:teuthology.orchestra.run.trial046.stdout: "active": true, 2026-02-24T22:23:30.245 INFO:teuthology.orchestra.run.trial046.stdout: "create_date": "2026-02-24T22:23:30.247980Z" 2026-02-24T22:23:30.245 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:23:30.245 INFO:teuthology.orchestra.run.trial046.stdout: ], 2026-02-24T22:23:30.245 INFO:teuthology.orchestra.run.trial046.stdout: "swift_keys": [], 2026-02-24T22:23:30.245 INFO:teuthology.orchestra.run.trial046.stdout: "caps": [], 2026-02-24T22:23:30.245 INFO:teuthology.orchestra.run.trial046.stdout: "op_mask": "read, write, delete", 2026-02-24T22:23:30.245 INFO:teuthology.orchestra.run.trial046.stdout: "default_placement": "", 2026-02-24T22:23:30.245 INFO:teuthology.orchestra.run.trial046.stdout: "default_storage_class": "", 2026-02-24T22:23:30.245 INFO:teuthology.orchestra.run.trial046.stdout: "placement_tags": [], 2026-02-24T22:23:30.245 INFO:teuthology.orchestra.run.trial046.stdout: "bucket_quota": { 2026-02-24T22:23:30.245 INFO:teuthology.orchestra.run.trial046.stdout: "enabled": false, 2026-02-24T22:23:30.245 INFO:teuthology.orchestra.run.trial046.stdout: "check_on_raw": false, 2026-02-24T22:23:30.246 INFO:teuthology.orchestra.run.trial046.stdout: "max_size": -1, 2026-02-24T22:23:30.246 INFO:teuthology.orchestra.run.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:23:30.246 INFO:teuthology.orchestra.run.trial046.stdout: "max_objects": -1 2026-02-24T22:23:30.246 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:23:30.246 INFO:teuthology.orchestra.run.trial046.stdout: "user_quota": { 2026-02-24T22:23:30.246 INFO:teuthology.orchestra.run.trial046.stdout: "enabled": false, 2026-02-24T22:23:30.246 INFO:teuthology.orchestra.run.trial046.stdout: "check_on_raw": false, 2026-02-24T22:23:30.246 INFO:teuthology.orchestra.run.trial046.stdout: "max_size": -1, 2026-02-24T22:23:30.246 INFO:teuthology.orchestra.run.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:23:30.246 INFO:teuthology.orchestra.run.trial046.stdout: "max_objects": -1 2026-02-24T22:23:30.246 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:23:30.246 INFO:teuthology.orchestra.run.trial046.stdout: "temp_url_keys": [], 2026-02-24T22:23:30.246 INFO:teuthology.orchestra.run.trial046.stdout: "type": "rgw", 2026-02-24T22:23:30.247 INFO:teuthology.orchestra.run.trial046.stdout: "mfa_ids": [], 2026-02-24T22:23:30.247 INFO:teuthology.orchestra.run.trial046.stdout: "account_id": "", 2026-02-24T22:23:30.247 INFO:teuthology.orchestra.run.trial046.stdout: "path": "/", 2026-02-24T22:23:30.247 INFO:teuthology.orchestra.run.trial046.stdout: "create_date": "2026-02-24T22:23:30.247970Z", 2026-02-24T22:23:30.247 INFO:teuthology.orchestra.run.trial046.stdout: "tags": [], 2026-02-24T22:23:30.247 INFO:teuthology.orchestra.run.trial046.stdout: "group_ids": [] 2026-02-24T22:23:30.247 INFO:teuthology.orchestra.run.trial046.stdout:} 2026-02-24T22:23:30.247 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:23:30.248 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.253+0000 7ffb84564a80 20 remove_watcher() i=2 2026-02-24T22:23:30.248 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.253+0000 7ffb84564a80 2 removed watcher, disabling cache 2026-02-24T22:23:30.248 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.253+0000 7ffb84564a80 20 remove_watcher() i=0 2026-02-24T22:23:30.249 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.253+0000 7ffb84564a80 20 remove_watcher() i=3 2026-02-24T22:23:30.249 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.253+0000 7ffb84564a80 20 remove_watcher() i=1 2026-02-24T22:23:30.249 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.253+0000 7ffb84564a80 20 remove_watcher() i=4 2026-02-24T22:23:30.249 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.253+0000 7ffb84564a80 20 remove_watcher() i=5 2026-02-24T22:23:30.249 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.253+0000 7ffb84564a80 20 remove_watcher() i=6 2026-02-24T22:23:30.249 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.253+0000 7ffb84564a80 20 remove_watcher() i=7 2026-02-24T22:23:30.253 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid 'testx$tenanteduser.client.0' --totp-serial 9974667717 --totp-seed KALPFCWWCGUAUXA5L3KXJPV4VMUUB5UIKWFSSEBJOYFCW65UD4CFHULGIDG23RC7 --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2026-02-24T22:23:30.314 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:30.314 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:30.329 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.333+0000 7f85ba666a80 4 Realm: () 2026-02-24T22:23:30.329 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.333+0000 7f85ba666a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:30.329 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.333+0000 7f85ba666a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:30.330 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.333+0000 7f85ba666a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:30.330 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.333+0000 7f85ba666a80 20 zonegroup default 2026-02-24T22:23:30.330 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.337+0000 7f85ba666a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:30.330 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.337+0000 7f85ba666a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:30.331 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.337+0000 7f85ba666a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:30.331 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.337+0000 7f85ba666a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:30.331 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.337+0000 7f85ba666a80 20 started sync module instance, tier type = 2026-02-24T22:23:30.331 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.337+0000 7f85ba666a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:30.332 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.337+0000 7f85b3fff640 0 -- 10.20.193.46:0/3648484762 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5607948096c0 msgr2=0x560794809b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:30.332 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.337+0000 7f85ba666a80 20 add_watcher() i=3 2026-02-24T22:23:30.333 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.337+0000 7f85ba666a80 20 add_watcher() i=1 2026-02-24T22:23:30.333 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.337+0000 7f85ba666a80 20 add_watcher() i=0 2026-02-24T22:23:30.333 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.337+0000 7f85ba666a80 20 add_watcher() i=5 2026-02-24T22:23:30.334 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.337+0000 7f85ba666a80 20 add_watcher() i=2 2026-02-24T22:23:30.334 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.337+0000 7f85ba666a80 20 add_watcher() i=7 2026-02-24T22:23:30.334 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.341+0000 7f85ba666a80 20 add_watcher() i=4 2026-02-24T22:23:30.334 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.341+0000 7f85ba666a80 20 add_watcher() i=6 2026-02-24T22:23:30.334 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.341+0000 7f85ba666a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:23:30.335 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.341+0000 7f85ba666a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:30.335 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.341+0000 7f85ba666a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:30.335 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.341+0000 7f85ba666a80 5 note: GC not initialized 2026-02-24T22:23:30.335 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.341+0000 7f8565fdb640 20 reqs_thread_entry: start 2026-02-24T22:23:30.336 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.341+0000 7f85b893b640 0 -- 10.20.193.46:0/2155509295 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x560794a10340 msgr2=0x560794a107b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:23:30.575 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.581+0000 7f85ba666a80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:30.575 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.581+0000 7f85ba666a80 20 Filter name: none 2026-02-24T22:23:30.575 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.581+0000 7f84bffff640 20 reqs_thread_entry: start 2026-02-24T22:23:30.575 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.581+0000 7f85ba666a80 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2026-02-24T22:23:30.576 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.581+0000 7f85ba666a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:30.576 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.581+0000 7f85ba666a80 20 rados_obj.operate() r=0 bl.length=519 2026-02-24T22:23:30.576 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.581+0000 7f85ba666a80 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x13 2026-02-24T22:23:30.576 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.581+0000 7f85ba666a80 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2026-02-24T22:23:30.576 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.581+0000 7f85ba666a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:23:30.577 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.585+0000 7f85ba666a80 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2026-02-24T22:23:30.577 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.585+0000 7f85ba666a80 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2026-02-24T22:23:30.578 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.585+0000 7f85ba666a80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:testx$tenanteduser.client.0, ofs0, ns] 2026-02-24T22:23:30.578 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.585+0000 7f8586ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1331439863290 cookie 94590558648800 notifier 8501 bl.length()=733 2026-02-24T22:23:30.578 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.585+0000 7f8586ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2026-02-24T22:23:30.578 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.585+0000 7f8586ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2026-02-24T22:23:30.581 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.585+0000 7f85ba666a80 20 remove_watcher() i=3 2026-02-24T22:23:30.581 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.585+0000 7f85ba666a80 2 removed watcher, disabling cache 2026-02-24T22:23:30.581 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.585+0000 7f85ba666a80 20 remove_watcher() i=4 2026-02-24T22:23:30.582 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.585+0000 7f85ba666a80 20 remove_watcher() i=1 2026-02-24T22:23:30.582 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.585+0000 7f85ba666a80 20 remove_watcher() i=2 2026-02-24T22:23:30.582 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.585+0000 7f85ba666a80 20 remove_watcher() i=6 2026-02-24T22:23:30.582 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.585+0000 7f85ba666a80 20 remove_watcher() i=0 2026-02-24T22:23:30.582 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.585+0000 7f85ba666a80 20 remove_watcher() i=7 2026-02-24T22:23:30.582 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.585+0000 7f85ba666a80 20 remove_watcher() i=5 2026-02-24T22:23:30.588 DEBUG:tasks.s3tests:Creating user foobar.client.0 on client.0 2026-02-24T22:23:30.588 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid foobar.client.0 --display-name Mr.foobar.client.0 --email foobar+test@test.test --access-key TUMJWNTVMIGYQUKJBAVA --secret 9SkbCwdKxUCN63Te59wuq6iB+tpZS45KsPpZjNI9ET/pmgkJWg+Xcw== --cluster ceph 2026-02-24T22:23:30.649 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:30.650 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:30.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.669+0000 7fbcffb6ba80 4 Realm: () 2026-02-24T22:23:30.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.669+0000 7fbcffb6ba80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:30.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.669+0000 7fbcffb6ba80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:30.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.669+0000 7fbcffb6ba80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:30.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.669+0000 7fbcffb6ba80 20 zonegroup default 2026-02-24T22:23:30.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.669+0000 7fbcffb6ba80 20 rados->read ofs=0 len=0 2026-02-24T22:23:30.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.673+0000 7fbcffb6ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:30.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.673+0000 7fbcffb6ba80 20 rados->read ofs=0 len=0 2026-02-24T22:23:30.666 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.673+0000 7fbcffb6ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:30.666 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.673+0000 7fbcffb6ba80 20 started sync module instance, tier type = 2026-02-24T22:23:30.666 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.673+0000 7fbcffb6ba80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:30.667 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.673+0000 7fbcfde40640 0 -- 10.20.193.46:0/2708139671 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x5596a21d6440 msgr2=0x5596a21d68b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:23:30.667 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.673+0000 7fbcffb6ba80 20 add_watcher() i=1 2026-02-24T22:23:30.668 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.673+0000 7fbcffb6ba80 20 add_watcher() i=6 2026-02-24T22:23:30.668 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.673+0000 7fbcffb6ba80 20 add_watcher() i=5 2026-02-24T22:23:30.668 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.673+0000 7fbcffb6ba80 20 add_watcher() i=3 2026-02-24T22:23:30.668 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.673+0000 7fbcffb6ba80 20 add_watcher() i=2 2026-02-24T22:23:30.668 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.673+0000 7fbcffb6ba80 20 add_watcher() i=0 2026-02-24T22:23:30.669 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.673+0000 7fbcffb6ba80 20 add_watcher() i=4 2026-02-24T22:23:30.669 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.673+0000 7fbcffb6ba80 20 add_watcher() i=7 2026-02-24T22:23:30.669 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.673+0000 7fbcffb6ba80 2 all 8 watchers are set, enabling cache 2026-02-24T22:23:30.670 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.677+0000 7fbcffb6ba80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:30.670 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.677+0000 7fbcffb6ba80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:30.670 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.677+0000 7fbcffb6ba80 5 note: GC not initialized 2026-02-24T22:23:30.670 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.677+0000 7fbcaa7e4640 20 reqs_thread_entry: start 2026-02-24T22:23:30.670 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.677+0000 7fbcfde40640 0 -- 10.20.193.46:0/2656067079 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x5596a2398dc0 msgr2=0x5596a2399230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:23:30.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.917+0000 7fbcffb6ba80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:30.913 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.917+0000 7fbcffb6ba80 20 Filter name: none 2026-02-24T22:23:30.913 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.917+0000 7fbca8fe1640 20 reqs_thread_entry: start 2026-02-24T22:23:30.913 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.917+0000 7fbcffb6ba80 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2026-02-24T22:23:30.913 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.917+0000 7fbcffb6ba80 20 rados->read ofs=0 len=0 2026-02-24T22:23:30.913 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.917+0000 7fbcffb6ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:30.913 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.917+0000 7fbcffb6ba80 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x0 2026-02-24T22:23:30.913 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.917+0000 7fbcffb6ba80 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2026-02-24T22:23:30.913 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.917+0000 7fbcffb6ba80 10 cache get: name=default.rgw.meta+users.email+foobar+test@test.test : miss 2026-02-24T22:23:30.913 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.917+0000 7fbcffb6ba80 20 rados->read ofs=0 len=0 2026-02-24T22:23:30.913 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.917+0000 7fbcffb6ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:30.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.917+0000 7fbcffb6ba80 10 cache put: name=default.rgw.meta+users.email+foobar+test@test.test info.flags=0x0 2026-02-24T22:23:30.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.917+0000 7fbcffb6ba80 10 adding default.rgw.meta+users.email+foobar+test@test.test to cache LRU end 2026-02-24T22:23:30.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.917+0000 7fbcffb6ba80 10 cache get: name=default.rgw.meta+users.keys+TUMJWNTVMIGYQUKJBAVA : miss 2026-02-24T22:23:30.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.917+0000 7fbcffb6ba80 20 rados->read ofs=0 len=0 2026-02-24T22:23:30.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.917+0000 7fbcffb6ba80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:30.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.917+0000 7fbcffb6ba80 10 cache put: name=default.rgw.meta+users.keys+TUMJWNTVMIGYQUKJBAVA info.flags=0x0 2026-02-24T22:23:30.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.917+0000 7fbcffb6ba80 10 adding default.rgw.meta+users.keys+TUMJWNTVMIGYQUKJBAVA to cache LRU end 2026-02-24T22:23:30.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.917+0000 7fbcffb6ba80 10 cache get: name=default.rgw.meta+users.keys+TUMJWNTVMIGYQUKJBAVA : hit (negative entry) 2026-02-24T22:23:30.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.917+0000 7fbcffb6ba80 10 cache get: name=default.rgw.meta+users.keys+TUMJWNTVMIGYQUKJBAVA : hit (negative entry) 2026-02-24T22:23:30.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.921+0000 7fbcffb6ba80 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2026-02-24T22:23:30.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.921+0000 7fbcffb6ba80 10 moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2026-02-24T22:23:30.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.921+0000 7fbcffb6ba80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2026-02-24T22:23:30.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.921+0000 7fbcd8ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1331439861813 cookie 94105455564368 notifier 8519 bl.length()=660 2026-02-24T22:23:30.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.921+0000 7fbcd8ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2026-02-24T22:23:30.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.921+0000 7fbcd8ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2026-02-24T22:23:30.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.921+0000 7fbcffb6ba80 10 cache put: name=default.rgw.meta+users.email+foobar+test@test.test info.flags=0x7 2026-02-24T22:23:30.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.921+0000 7fbcffb6ba80 10 moving default.rgw.meta+users.email+foobar+test@test.test to cache LRU end 2026-02-24T22:23:30.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.921+0000 7fbcffb6ba80 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.email:foobar+test@test.test, ofs0, ns] 2026-02-24T22:23:30.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.921+0000 7fbcc7fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1331439861892 cookie 94105455702496 notifier 8519 bl.length()=191 2026-02-24T22:23:30.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.921+0000 7fbcc7fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+foobar+test@test.test info.flags=0x7 2026-02-24T22:23:30.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.921+0000 7fbcc7fff640 10 rgw watcher librados: moving default.rgw.meta+users.email+foobar+test@test.test to cache LRU end 2026-02-24T22:23:30.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.925+0000 7fbcffb6ba80 10 cache put: name=default.rgw.meta+users.keys+TUMJWNTVMIGYQUKJBAVA info.flags=0x7 2026-02-24T22:23:30.918 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.925+0000 7fbcffb6ba80 10 moving default.rgw.meta+users.keys+TUMJWNTVMIGYQUKJBAVA to cache LRU end 2026-02-24T22:23:30.918 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.925+0000 7fbcffb6ba80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:users.keys:TUMJWNTVMIGYQUKJBAVA, ofs0, ns] 2026-02-24T22:23:30.918 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.925+0000 7fbcc7fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1331439861842 cookie 94105453327600 notifier 8519 bl.length()=189 2026-02-24T22:23:30.918 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.925+0000 7fbcc7fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+TUMJWNTVMIGYQUKJBAVA info.flags=0x7 2026-02-24T22:23:30.918 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.925+0000 7fbcc7fff640 10 rgw watcher librados: moving default.rgw.meta+users.keys+TUMJWNTVMIGYQUKJBAVA to cache LRU end 2026-02-24T22:23:30.919 INFO:teuthology.orchestra.run.trial046.stdout:{ 2026-02-24T22:23:30.919 INFO:teuthology.orchestra.run.trial046.stdout: "user_id": "foobar.client.0", 2026-02-24T22:23:30.919 INFO:teuthology.orchestra.run.trial046.stdout: "display_name": "Mr.foobar.client.0", 2026-02-24T22:23:30.919 INFO:teuthology.orchestra.run.trial046.stdout: "email": "foobar+test@test.test", 2026-02-24T22:23:30.919 INFO:teuthology.orchestra.run.trial046.stdout: "suspended": 0, 2026-02-24T22:23:30.919 INFO:teuthology.orchestra.run.trial046.stdout: "max_buckets": 1000, 2026-02-24T22:23:30.919 INFO:teuthology.orchestra.run.trial046.stdout: "subusers": [], 2026-02-24T22:23:30.919 INFO:teuthology.orchestra.run.trial046.stdout: "keys": [ 2026-02-24T22:23:30.919 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:23:30.919 INFO:teuthology.orchestra.run.trial046.stdout: "user": "foobar.client.0", 2026-02-24T22:23:30.919 INFO:teuthology.orchestra.run.trial046.stdout: "access_key": "TUMJWNTVMIGYQUKJBAVA", 2026-02-24T22:23:30.920 INFO:teuthology.orchestra.run.trial046.stdout: "secret_key": "9SkbCwdKxUCN63Te59wuq6iB+tpZS45KsPpZjNI9ET/pmgkJWg+Xcw==", 2026-02-24T22:23:30.920 INFO:teuthology.orchestra.run.trial046.stdout: "active": true, 2026-02-24T22:23:30.920 INFO:teuthology.orchestra.run.trial046.stdout: "create_date": "2026-02-24T22:23:30.922759Z" 2026-02-24T22:23:30.920 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:23:30.920 INFO:teuthology.orchestra.run.trial046.stdout: ], 2026-02-24T22:23:30.920 INFO:teuthology.orchestra.run.trial046.stdout: "swift_keys": [], 2026-02-24T22:23:30.920 INFO:teuthology.orchestra.run.trial046.stdout: "caps": [], 2026-02-24T22:23:30.920 INFO:teuthology.orchestra.run.trial046.stdout: "op_mask": "read, write, delete", 2026-02-24T22:23:30.920 INFO:teuthology.orchestra.run.trial046.stdout: "default_placement": "", 2026-02-24T22:23:30.920 INFO:teuthology.orchestra.run.trial046.stdout: "default_storage_class": "", 2026-02-24T22:23:30.920 INFO:teuthology.orchestra.run.trial046.stdout: "placement_tags": [], 2026-02-24T22:23:30.920 INFO:teuthology.orchestra.run.trial046.stdout: "bucket_quota": { 2026-02-24T22:23:30.921 INFO:teuthology.orchestra.run.trial046.stdout: "enabled": false, 2026-02-24T22:23:30.921 INFO:teuthology.orchestra.run.trial046.stdout: "check_on_raw": false, 2026-02-24T22:23:30.921 INFO:teuthology.orchestra.run.trial046.stdout: "max_size": -1, 2026-02-24T22:23:30.921 INFO:teuthology.orchestra.run.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:23:30.921 INFO:teuthology.orchestra.run.trial046.stdout: "max_objects": -1 2026-02-24T22:23:30.921 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:23:30.921 INFO:teuthology.orchestra.run.trial046.stdout: "user_quota": { 2026-02-24T22:23:30.921 INFO:teuthology.orchestra.run.trial046.stdout: "enabled": false, 2026-02-24T22:23:30.921 INFO:teuthology.orchestra.run.trial046.stdout: "check_on_raw": false, 2026-02-24T22:23:30.921 INFO:teuthology.orchestra.run.trial046.stdout: "max_size": -1, 2026-02-24T22:23:30.921 INFO:teuthology.orchestra.run.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:23:30.921 INFO:teuthology.orchestra.run.trial046.stdout: "max_objects": -1 2026-02-24T22:23:30.921 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:23:30.921 INFO:teuthology.orchestra.run.trial046.stdout: "temp_url_keys": [], 2026-02-24T22:23:30.922 INFO:teuthology.orchestra.run.trial046.stdout: "type": "rgw", 2026-02-24T22:23:30.922 INFO:teuthology.orchestra.run.trial046.stdout: "mfa_ids": [], 2026-02-24T22:23:30.922 INFO:teuthology.orchestra.run.trial046.stdout: "account_id": "", 2026-02-24T22:23:30.922 INFO:teuthology.orchestra.run.trial046.stdout: "path": "/", 2026-02-24T22:23:30.922 INFO:teuthology.orchestra.run.trial046.stdout: "create_date": "2026-02-24T22:23:30.922748Z", 2026-02-24T22:23:30.922 INFO:teuthology.orchestra.run.trial046.stdout: "tags": [], 2026-02-24T22:23:30.922 INFO:teuthology.orchestra.run.trial046.stdout: "group_ids": [] 2026-02-24T22:23:30.922 INFO:teuthology.orchestra.run.trial046.stdout:} 2026-02-24T22:23:30.922 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:23:30.923 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.925+0000 7fbcffb6ba80 20 remove_watcher() i=3 2026-02-24T22:23:30.923 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.925+0000 7fbcffb6ba80 2 removed watcher, disabling cache 2026-02-24T22:23:30.923 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.925+0000 7fbcffb6ba80 20 remove_watcher() i=2 2026-02-24T22:23:30.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.925+0000 7fbcffb6ba80 20 remove_watcher() i=6 2026-02-24T22:23:30.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.925+0000 7fbcffb6ba80 20 remove_watcher() i=1 2026-02-24T22:23:30.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.925+0000 7fbcffb6ba80 20 remove_watcher() i=4 2026-02-24T22:23:30.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.925+0000 7fbcffb6ba80 20 remove_watcher() i=0 2026-02-24T22:23:30.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.925+0000 7fbcffb6ba80 20 remove_watcher() i=7 2026-02-24T22:23:30.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:30.925+0000 7fbcffb6ba80 20 remove_watcher() i=5 2026-02-24T22:23:30.928 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid foobar.client.0 --totp-serial 4155491448 --totp-seed OGVDCHJ4XDRBERMK3ZRL6ZHMQWTNKP3J2SEXEAYE2S2RT5XXHRJ7PTUZ26GB5HHI --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2026-02-24T22:23:30.991 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:30.991 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:31.007 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.013+0000 7f1cf2114a80 4 Realm: () 2026-02-24T22:23:31.007 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.013+0000 7f1cf2114a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:31.007 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.013+0000 7f1cf2114a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:31.007 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.013+0000 7f1cf2114a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:31.007 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.013+0000 7f1cf2114a80 20 zonegroup default 2026-02-24T22:23:31.007 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.013+0000 7f1cf2114a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:31.008 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.013+0000 7f1cf2114a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:31.008 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.013+0000 7f1cf2114a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:31.009 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.013+0000 7f1cf2114a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:31.009 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.013+0000 7f1cf2114a80 20 started sync module instance, tier type = 2026-02-24T22:23:31.009 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.013+0000 7f1cf2114a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:31.009 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.013+0000 7f1ceb7fe640 0 -- 10.20.193.46:0/2380359614 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x563f41c0cdb0 msgr2=0x563f41c2d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:31.011 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.017+0000 7f1cf2114a80 20 add_watcher() i=1 2026-02-24T22:23:31.011 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.017+0000 7f1cf2114a80 20 add_watcher() i=3 2026-02-24T22:23:31.011 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.017+0000 7f1cf2114a80 20 add_watcher() i=5 2026-02-24T22:23:31.011 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.017+0000 7f1cf2114a80 20 add_watcher() i=6 2026-02-24T22:23:31.011 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.017+0000 7f1cf2114a80 20 add_watcher() i=0 2026-02-24T22:23:31.011 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.017+0000 7f1cf2114a80 20 add_watcher() i=2 2026-02-24T22:23:31.011 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.017+0000 7f1cf2114a80 20 add_watcher() i=4 2026-02-24T22:23:31.212 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.217+0000 7f1cf2114a80 20 add_watcher() i=7 2026-02-24T22:23:31.212 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.217+0000 7f1cf2114a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:23:31.213 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.217+0000 7f1ceaffd640 0 -- 10.20.193.46:0/2439231828 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f1c10008af0 msgr2=0x7f1c10028fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:31.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.421+0000 7f1cf2114a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:31.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.421+0000 7f1cf2114a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:31.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.421+0000 7f1cf2114a80 5 note: GC not initialized 2026-02-24T22:23:31.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.421+0000 7f1c9cfe1640 20 reqs_thread_entry: start 2026-02-24T22:23:31.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.457+0000 7f1cf2114a80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:31.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.457+0000 7f1cf2114a80 20 Filter name: none 2026-02-24T22:23:31.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.457+0000 7f1c06ffd640 20 reqs_thread_entry: start 2026-02-24T22:23:31.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.457+0000 7f1cf2114a80 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2026-02-24T22:23:31.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.457+0000 7f1cf2114a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:31.453 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.457+0000 7f1cf2114a80 20 rados_obj.operate() r=0 bl.length=472 2026-02-24T22:23:31.453 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.457+0000 7f1cf2114a80 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x13 2026-02-24T22:23:31.453 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.457+0000 7f1cf2114a80 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2026-02-24T22:23:31.453 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.457+0000 7f1cf2114a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:23:31.454 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.461+0000 7f1cf2114a80 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2026-02-24T22:23:31.454 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.461+0000 7f1cf2114a80 10 moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2026-02-24T22:23:31.454 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.461+0000 7f1cf2114a80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2026-02-24T22:23:31.455 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.461+0000 7f1cc6ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1331439861814 cookie 94829686222320 notifier 8534 bl.length()=674 2026-02-24T22:23:31.455 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.461+0000 7f1cc6ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2026-02-24T22:23:31.455 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.461+0000 7f1cc6ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2026-02-24T22:23:31.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.465+0000 7f1cf2114a80 20 remove_watcher() i=3 2026-02-24T22:23:31.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.465+0000 7f1cf2114a80 2 removed watcher, disabling cache 2026-02-24T22:23:31.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.465+0000 7f1cf2114a80 20 remove_watcher() i=1 2026-02-24T22:23:31.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.465+0000 7f1cf2114a80 20 remove_watcher() i=2 2026-02-24T22:23:31.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.465+0000 7f1cf2114a80 20 remove_watcher() i=7 2026-02-24T22:23:31.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.465+0000 7f1cf2114a80 20 remove_watcher() i=4 2026-02-24T22:23:31.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.465+0000 7f1cf2114a80 20 remove_watcher() i=0 2026-02-24T22:23:31.459 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.465+0000 7f1cf2114a80 20 remove_watcher() i=5 2026-02-24T22:23:31.459 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.465+0000 7f1cf2114a80 20 remove_watcher() i=6 2026-02-24T22:23:31.465 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 caps add --uid foobar.client.0 --caps 'oidc-provider=*;roles=*;user-policy=*' --cluster ceph 2026-02-24T22:23:31.524 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:31.524 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:31.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.545+0000 7f58e87c0a80 4 Realm: () 2026-02-24T22:23:31.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.545+0000 7f58e87c0a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:31.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.545+0000 7f58e87c0a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:31.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.545+0000 7f58e87c0a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:31.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.545+0000 7f58e87c0a80 20 zonegroup default 2026-02-24T22:23:31.541 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.545+0000 7f58e87c0a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:31.541 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.549+0000 7f58e87c0a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:31.541 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.549+0000 7f58e87c0a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:31.542 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.549+0000 7f58e87c0a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:31.542 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.549+0000 7f58e87c0a80 20 started sync module instance, tier type = 2026-02-24T22:23:31.542 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.549+0000 7f58e87c0a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:31.542 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.549+0000 7f58e5a93640 0 -- 10.20.193.46:0/3877797602 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x5577de273990 msgr2=0x5577de273e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:23:31.543 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.549+0000 7f58e87c0a80 20 add_watcher() i=1 2026-02-24T22:23:31.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.549+0000 7f58e87c0a80 20 add_watcher() i=6 2026-02-24T22:23:31.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.549+0000 7f58e87c0a80 20 add_watcher() i=3 2026-02-24T22:23:31.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.549+0000 7f58e87c0a80 20 add_watcher() i=0 2026-02-24T22:23:31.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.549+0000 7f58e87c0a80 20 add_watcher() i=5 2026-02-24T22:23:31.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.549+0000 7f58e87c0a80 20 add_watcher() i=2 2026-02-24T22:23:31.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.549+0000 7f58e87c0a80 20 add_watcher() i=7 2026-02-24T22:23:31.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.753+0000 7f58e5a93640 0 -- 10.20.193.46:0/3877797602 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x5577de273990 msgr2=0x5577de273e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:31.749 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.753+0000 7f58e87c0a80 20 add_watcher() i=4 2026-02-24T22:23:31.749 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.753+0000 7f58e87c0a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:23:31.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.757+0000 7f58e87c0a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:31.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.757+0000 7f58e87c0a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:31.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.757+0000 7f58e87c0a80 5 note: GC not initialized 2026-02-24T22:23:31.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.757+0000 7f57ff7fe640 20 reqs_thread_entry: start 2026-02-24T22:23:31.786 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.793+0000 7f58e87c0a80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:31.786 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.793+0000 7f58e87c0a80 20 Filter name: none 2026-02-24T22:23:31.786 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.793+0000 7f57fdffb640 20 reqs_thread_entry: start 2026-02-24T22:23:31.786 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.793+0000 7f58e87c0a80 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2026-02-24T22:23:31.786 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.793+0000 7f58e87c0a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:31.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.793+0000 7f58e87c0a80 20 rados_obj.operate() r=0 bl.length=486 2026-02-24T22:23:31.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.793+0000 7f58e87c0a80 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x13 2026-02-24T22:23:31.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.793+0000 7f58e87c0a80 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2026-02-24T22:23:31.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.793+0000 7f58e87c0a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:23:31.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.793+0000 7f58e87c0a80 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2026-02-24T22:23:31.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.793+0000 7f58e87c0a80 10 moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2026-02-24T22:23:31.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.793+0000 7f58e87c0a80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2026-02-24T22:23:31.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.793+0000 7f58c1ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1331439861815 cookie 93973318830704 notifier 9192 bl.length()=727 2026-02-24T22:23:31.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.793+0000 7f58c1ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2026-02-24T22:23:31.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.793+0000 7f58c1ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2026-02-24T22:23:31.789 INFO:teuthology.orchestra.run.trial046.stdout:{ 2026-02-24T22:23:31.789 INFO:teuthology.orchestra.run.trial046.stdout: "user_id": "foobar.client.0", 2026-02-24T22:23:31.789 INFO:teuthology.orchestra.run.trial046.stdout: "display_name": "Mr.foobar.client.0", 2026-02-24T22:23:31.789 INFO:teuthology.orchestra.run.trial046.stdout: "email": "foobar+test@test.test", 2026-02-24T22:23:31.789 INFO:teuthology.orchestra.run.trial046.stdout: "suspended": 0, 2026-02-24T22:23:31.790 INFO:teuthology.orchestra.run.trial046.stdout: "max_buckets": 1000, 2026-02-24T22:23:31.790 INFO:teuthology.orchestra.run.trial046.stdout: "subusers": [], 2026-02-24T22:23:31.790 INFO:teuthology.orchestra.run.trial046.stdout: "keys": [ 2026-02-24T22:23:31.790 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:23:31.790 INFO:teuthology.orchestra.run.trial046.stdout: "user": "foobar.client.0", 2026-02-24T22:23:31.790 INFO:teuthology.orchestra.run.trial046.stdout: "access_key": "TUMJWNTVMIGYQUKJBAVA", 2026-02-24T22:23:31.790 INFO:teuthology.orchestra.run.trial046.stdout: "secret_key": "9SkbCwdKxUCN63Te59wuq6iB+tpZS45KsPpZjNI9ET/pmgkJWg+Xcw==", 2026-02-24T22:23:31.790 INFO:teuthology.orchestra.run.trial046.stdout: "active": true, 2026-02-24T22:23:31.790 INFO:teuthology.orchestra.run.trial046.stdout: "create_date": "2026-02-24T22:23:30.922759Z" 2026-02-24T22:23:31.790 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:23:31.790 INFO:teuthology.orchestra.run.trial046.stdout: ], 2026-02-24T22:23:31.790 INFO:teuthology.orchestra.run.trial046.stdout: "swift_keys": [], 2026-02-24T22:23:31.791 INFO:teuthology.orchestra.run.trial046.stdout: "caps": [ 2026-02-24T22:23:31.791 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:23:31.791 INFO:teuthology.orchestra.run.trial046.stdout: "type": "oidc-provider", 2026-02-24T22:23:31.791 INFO:teuthology.orchestra.run.trial046.stdout: "perm": "*" 2026-02-24T22:23:31.791 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:23:31.791 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:23:31.791 INFO:teuthology.orchestra.run.trial046.stdout: "type": "roles", 2026-02-24T22:23:31.791 INFO:teuthology.orchestra.run.trial046.stdout: "perm": "*" 2026-02-24T22:23:31.791 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:23:31.791 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:23:31.791 INFO:teuthology.orchestra.run.trial046.stdout: "type": "user-policy", 2026-02-24T22:23:31.791 INFO:teuthology.orchestra.run.trial046.stdout: "perm": "*" 2026-02-24T22:23:31.791 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:23:31.792 INFO:teuthology.orchestra.run.trial046.stdout: ], 2026-02-24T22:23:31.792 INFO:teuthology.orchestra.run.trial046.stdout: "op_mask": "read, write, delete", 2026-02-24T22:23:31.792 INFO:teuthology.orchestra.run.trial046.stdout: "default_placement": "", 2026-02-24T22:23:31.792 INFO:teuthology.orchestra.run.trial046.stdout: "default_storage_class": "", 2026-02-24T22:23:31.792 INFO:teuthology.orchestra.run.trial046.stdout: "placement_tags": [], 2026-02-24T22:23:31.792 INFO:teuthology.orchestra.run.trial046.stdout: "bucket_quota": { 2026-02-24T22:23:31.792 INFO:teuthology.orchestra.run.trial046.stdout: "enabled": false, 2026-02-24T22:23:31.792 INFO:teuthology.orchestra.run.trial046.stdout: "check_on_raw": false, 2026-02-24T22:23:31.792 INFO:teuthology.orchestra.run.trial046.stdout: "max_size": -1, 2026-02-24T22:23:31.792 INFO:teuthology.orchestra.run.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:23:31.792 INFO:teuthology.orchestra.run.trial046.stdout: "max_objects": -1 2026-02-24T22:23:31.792 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:23:31.792 INFO:teuthology.orchestra.run.trial046.stdout: "user_quota": { 2026-02-24T22:23:31.793 INFO:teuthology.orchestra.run.trial046.stdout: "enabled": false, 2026-02-24T22:23:31.793 INFO:teuthology.orchestra.run.trial046.stdout: "check_on_raw": false, 2026-02-24T22:23:31.793 INFO:teuthology.orchestra.run.trial046.stdout: "max_size": -1, 2026-02-24T22:23:31.793 INFO:teuthology.orchestra.run.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:23:31.793 INFO:teuthology.orchestra.run.trial046.stdout: "max_objects": -1 2026-02-24T22:23:31.793 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:23:31.793 INFO:teuthology.orchestra.run.trial046.stdout: "temp_url_keys": [], 2026-02-24T22:23:31.793 INFO:teuthology.orchestra.run.trial046.stdout: "type": "rgw", 2026-02-24T22:23:31.793 INFO:teuthology.orchestra.run.trial046.stdout: "mfa_ids": [ 2026-02-24T22:23:31.793 INFO:teuthology.orchestra.run.trial046.stdout: "4155491448" 2026-02-24T22:23:31.793 INFO:teuthology.orchestra.run.trial046.stdout: ], 2026-02-24T22:23:31.793 INFO:teuthology.orchestra.run.trial046.stdout: "account_id": "", 2026-02-24T22:23:31.793 INFO:teuthology.orchestra.run.trial046.stdout: "path": "/", 2026-02-24T22:23:31.793 INFO:teuthology.orchestra.run.trial046.stdout: "create_date": "2026-02-24T22:23:30.922748Z", 2026-02-24T22:23:31.794 INFO:teuthology.orchestra.run.trial046.stdout: "tags": [], 2026-02-24T22:23:31.794 INFO:teuthology.orchestra.run.trial046.stdout: "group_ids": [] 2026-02-24T22:23:31.794 INFO:teuthology.orchestra.run.trial046.stdout:} 2026-02-24T22:23:31.794 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:23:31.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.797+0000 7f58e87c0a80 20 remove_watcher() i=2 2026-02-24T22:23:31.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.797+0000 7f58e87c0a80 2 removed watcher, disabling cache 2026-02-24T22:23:31.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.797+0000 7f58e87c0a80 20 remove_watcher() i=1 2026-02-24T22:23:31.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.797+0000 7f58e87c0a80 20 remove_watcher() i=4 2026-02-24T22:23:31.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.797+0000 7f58e87c0a80 20 remove_watcher() i=6 2026-02-24T22:23:31.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.797+0000 7f58e87c0a80 20 remove_watcher() i=3 2026-02-24T22:23:31.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.797+0000 7f58e87c0a80 20 remove_watcher() i=7 2026-02-24T22:23:31.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.797+0000 7f58e87c0a80 20 remove_watcher() i=0 2026-02-24T22:23:31.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.797+0000 7f58e87c0a80 20 remove_watcher() i=5 2026-02-24T22:23:31.799 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph account create --account-id RGW88888888888888888 --account-name Mr.RGW88888888888888888 --email RGW88888888888888888+test@test.test 2026-02-24T22:23:31.860 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:31.861 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:31.875 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.881+0000 7fb4845c4a80 4 Realm: () 2026-02-24T22:23:31.875 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.881+0000 7fb4845c4a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:31.876 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.881+0000 7fb4845c4a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:31.876 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.881+0000 7fb4845c4a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:31.876 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.881+0000 7fb4845c4a80 20 zonegroup default 2026-02-24T22:23:31.876 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.881+0000 7fb4845c4a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:31.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.885+0000 7fb4845c4a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:31.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.885+0000 7fb4845c4a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:31.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.885+0000 7fb4845c4a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:31.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.885+0000 7fb4845c4a80 20 started sync module instance, tier type = 2026-02-24T22:23:31.879 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.885+0000 7fb4845c4a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:31.879 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.885+0000 7fb482098640 0 -- 10.20.193.46:0/3867213225 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x560ac1e278e0 msgr2=0x560ac1e27d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:31.880 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.885+0000 7fb4845c4a80 20 add_watcher() i=1 2026-02-24T22:23:31.880 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.885+0000 7fb4845c4a80 20 add_watcher() i=3 2026-02-24T22:23:31.880 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.885+0000 7fb4845c4a80 20 add_watcher() i=0 2026-02-24T22:23:31.880 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.885+0000 7fb4845c4a80 20 add_watcher() i=6 2026-02-24T22:23:31.880 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.885+0000 7fb4845c4a80 20 add_watcher() i=5 2026-02-24T22:23:31.881 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.885+0000 7fb4845c4a80 20 add_watcher() i=7 2026-02-24T22:23:31.881 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:31.885+0000 7fb4845c4a80 20 add_watcher() i=4 2026-02-24T22:23:32.084 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.089+0000 7fb4845c4a80 20 add_watcher() i=2 2026-02-24T22:23:32.084 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.089+0000 7fb4845c4a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:23:32.084 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.089+0000 7fb481897640 0 -- 10.20.193.46:0/3139575505 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7fb3a4008af0 msgr2=0x7fb3a4028fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:32.287 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.293+0000 7fb4845c4a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:32.287 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.293+0000 7fb4845c4a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:32.288 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.293+0000 7fb4845c4a80 5 note: GC not initialized 2026-02-24T22:23:32.288 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.293+0000 7fb39b7fe640 20 reqs_thread_entry: start 2026-02-24T22:23:32.527 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.533+0000 7fb4845c4a80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:32.527 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.533+0000 7fb4845c4a80 20 Filter name: none 2026-02-24T22:23:32.527 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.533+0000 7fb399ffb640 20 reqs_thread_entry: start 2026-02-24T22:23:32.528 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.533+0000 7fb4845c4a80 10 cache get: name=default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 : miss 2026-02-24T22:23:32.528 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.533+0000 7fb4845c4a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:32.528 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.533+0000 7fb4845c4a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:32.528 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.533+0000 7fb4845c4a80 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 info.flags=0x0 2026-02-24T22:23:32.528 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.533+0000 7fb4845c4a80 10 adding default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 to cache LRU end 2026-02-24T22:23:32.528 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.533+0000 7fb4845c4a80 20 failed to read name.$Mr.RGW88888888888888888 with: (2) No such file or directory 2026-02-24T22:23:32.528 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.533+0000 7fb4845c4a80 10 cache get: name=default.rgw.meta+users.email+rgw88888888888888888+test@test.test : miss 2026-02-24T22:23:32.528 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.533+0000 7fb4845c4a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:32.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.533+0000 7fb4845c4a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:32.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.533+0000 7fb4845c4a80 10 cache put: name=default.rgw.meta+users.email+rgw88888888888888888+test@test.test info.flags=0x0 2026-02-24T22:23:32.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.533+0000 7fb4845c4a80 10 adding default.rgw.meta+users.email+rgw88888888888888888+test@test.test to cache LRU end 2026-02-24T22:23:32.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.533+0000 7fb4845c4a80 20 failed to read rgw88888888888888888+test@test.test with: (2) No such file or directory 2026-02-24T22:23:32.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.533+0000 7fb4845c4a80 10 cache put: name=default.rgw.meta+accounts+account.RGW88888888888888888 info.flags=0x17 2026-02-24T22:23:32.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.533+0000 7fb4845c4a80 10 adding default.rgw.meta+accounts+account.RGW88888888888888888 to cache LRU end 2026-02-24T22:23:32.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.533+0000 7fb4845c4a80 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:accounts:account.RGW88888888888888888, ofs0, ns] 2026-02-24T22:23:32.530 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.533+0000 7fb45dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1331439861893 cookie 94604204906992 notifier 9210 bl.length()=384 2026-02-24T22:23:32.530 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.533+0000 7fb45dffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+account.RGW88888888888888888 info.flags=0x17 2026-02-24T22:23:32.530 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.533+0000 7fb45dffb640 10 rgw watcher librados: moving default.rgw.meta+accounts+account.RGW88888888888888888 to cache LRU end 2026-02-24T22:23:32.530 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.537+0000 7fb4845c4a80 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 info.flags=0x17 2026-02-24T22:23:32.530 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.537+0000 7fb4845c4a80 10 moving default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 to cache LRU end 2026-02-24T22:23:32.530 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.537+0000 7fb4845c4a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:accounts:name.$Mr.RGW88888888888888888, ofs0, ns] 2026-02-24T22:23:32.531 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.537+0000 7fb45dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1331439861922 cookie 94604203849520 notifier 9210 bl.length()=225 2026-02-24T22:23:32.531 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.537+0000 7fb45dffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 info.flags=0x17 2026-02-24T22:23:32.531 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.537+0000 7fb45dffb640 10 rgw watcher librados: moving default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 to cache LRU end 2026-02-24T22:23:32.531 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.537+0000 7fb4845c4a80 10 cache put: name=default.rgw.meta+users.email+rgw88888888888888888+test@test.test info.flags=0x17 2026-02-24T22:23:32.532 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.537+0000 7fb4845c4a80 10 moving default.rgw.meta+users.email+rgw88888888888888888+test@test.test to cache LRU end 2026-02-24T22:23:32.532 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.537+0000 7fb4845c4a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:rgw88888888888888888+test@test.test, ofs0, ns] 2026-02-24T22:23:32.532 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.537+0000 7fb45d7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1331439863263 cookie 94604204910192 notifier 9210 bl.length()=234 2026-02-24T22:23:32.532 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.537+0000 7fb45d7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+rgw88888888888888888+test@test.test info.flags=0x17 2026-02-24T22:23:32.532 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.537+0000 7fb45d7fa640 10 rgw watcher librados: moving default.rgw.meta+users.email+rgw88888888888888888+test@test.test to cache LRU end 2026-02-24T22:23:32.535 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.541+0000 7fb4845c4a80 20 remove_watcher() i=3 2026-02-24T22:23:32.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.541+0000 7fb4845c4a80 2 removed watcher, disabling cache 2026-02-24T22:23:32.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.541+0000 7fb4845c4a80 20 remove_watcher() i=0 2026-02-24T22:23:32.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.541+0000 7fb4845c4a80 20 remove_watcher() i=5 2026-02-24T22:23:32.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.541+0000 7fb4845c4a80 20 remove_watcher() i=4 2026-02-24T22:23:32.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.541+0000 7fb4845c4a80 20 remove_watcher() i=1 2026-02-24T22:23:32.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.541+0000 7fb4845c4a80 20 remove_watcher() i=2 2026-02-24T22:23:32.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.541+0000 7fb4845c4a80 20 remove_watcher() i=6 2026-02-24T22:23:32.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.541+0000 7fb4845c4a80 20 remove_watcher() i=7 2026-02-24T22:23:32.538 INFO:teuthology.orchestra.run.trial046.stdout:{ 2026-02-24T22:23:32.538 INFO:teuthology.orchestra.run.trial046.stdout: "id": "RGW88888888888888888", 2026-02-24T22:23:32.539 INFO:teuthology.orchestra.run.trial046.stdout: "tenant": "", 2026-02-24T22:23:32.539 INFO:teuthology.orchestra.run.trial046.stdout: "name": "Mr.RGW88888888888888888", 2026-02-24T22:23:32.539 INFO:teuthology.orchestra.run.trial046.stdout: "email": "RGW88888888888888888+test@test.test", 2026-02-24T22:23:32.539 INFO:teuthology.orchestra.run.trial046.stdout: "quota": { 2026-02-24T22:23:32.539 INFO:teuthology.orchestra.run.trial046.stdout: "enabled": false, 2026-02-24T22:23:32.539 INFO:teuthology.orchestra.run.trial046.stdout: "check_on_raw": false, 2026-02-24T22:23:32.539 INFO:teuthology.orchestra.run.trial046.stdout: "max_size": -1, 2026-02-24T22:23:32.539 INFO:teuthology.orchestra.run.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:23:32.539 INFO:teuthology.orchestra.run.trial046.stdout: "max_objects": -1 2026-02-24T22:23:32.539 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:23:32.539 INFO:teuthology.orchestra.run.trial046.stdout: "bucket_quota": { 2026-02-24T22:23:32.540 INFO:teuthology.orchestra.run.trial046.stdout: "enabled": false, 2026-02-24T22:23:32.540 INFO:teuthology.orchestra.run.trial046.stdout: "check_on_raw": false, 2026-02-24T22:23:32.540 INFO:teuthology.orchestra.run.trial046.stdout: "max_size": -1, 2026-02-24T22:23:32.540 INFO:teuthology.orchestra.run.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:23:32.540 INFO:teuthology.orchestra.run.trial046.stdout: "max_objects": -1 2026-02-24T22:23:32.540 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:23:32.540 INFO:teuthology.orchestra.run.trial046.stdout: "max_users": 1000, 2026-02-24T22:23:32.540 INFO:teuthology.orchestra.run.trial046.stdout: "max_roles": 1000, 2026-02-24T22:23:32.540 INFO:teuthology.orchestra.run.trial046.stdout: "max_groups": 1000, 2026-02-24T22:23:32.540 INFO:teuthology.orchestra.run.trial046.stdout: "max_buckets": 1000, 2026-02-24T22:23:32.540 INFO:teuthology.orchestra.run.trial046.stdout: "max_access_keys": 4 2026-02-24T22:23:32.540 INFO:teuthology.orchestra.run.trial046.stdout:} 2026-02-24T22:23:32.542 DEBUG:tasks.s3tests:Creating user RGW88888888888888888 on client.0 2026-02-24T22:23:32.542 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid root1.client.0 --display-name Mr.RGW88888888888888888 --email root1+test@test.test --access-key YFRXIPWFCTLLKUCOAATR --secret qpQPOUKEhiv5xYwfttkS5caDASAjBeuDiXZ6FQf2gIeEGpK+2P6FSQ== --cluster ceph --account-id RGW88888888888888888 --account-root 2026-02-24T22:23:32.607 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:32.607 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:32.623 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.629+0000 7fdccbb13a80 4 Realm: () 2026-02-24T22:23:32.623 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.629+0000 7fdccbb13a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:32.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.629+0000 7fdccbb13a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:32.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.629+0000 7fdccbb13a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:32.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.629+0000 7fdccbb13a80 20 zonegroup default 2026-02-24T22:23:32.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.629+0000 7fdccbb13a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:32.625 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.629+0000 7fdccbb13a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:32.625 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.629+0000 7fdccbb13a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:32.625 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.629+0000 7fdccbb13a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:32.626 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.629+0000 7fdccbb13a80 20 started sync module instance, tier type = 2026-02-24T22:23:32.626 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.629+0000 7fdccbb13a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:32.626 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.633+0000 7fdcc95e7640 0 -- 10.20.193.46:0/2465066772 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x555b09705db0 msgr2=0x555b09706250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:32.627 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.633+0000 7fdccbb13a80 20 add_watcher() i=3 2026-02-24T22:23:32.627 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.633+0000 7fdccbb13a80 20 add_watcher() i=1 2026-02-24T22:23:32.628 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.633+0000 7fdccbb13a80 20 add_watcher() i=5 2026-02-24T22:23:32.628 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.633+0000 7fdccbb13a80 20 add_watcher() i=2 2026-02-24T22:23:32.628 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.633+0000 7fdccbb13a80 20 add_watcher() i=0 2026-02-24T22:23:32.628 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.633+0000 7fdccbb13a80 20 add_watcher() i=7 2026-02-24T22:23:32.628 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.633+0000 7fdccbb13a80 20 add_watcher() i=4 2026-02-24T22:23:32.629 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.633+0000 7fdccbb13a80 20 add_watcher() i=6 2026-02-24T22:23:32.629 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.633+0000 7fdccbb13a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:23:32.629 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.637+0000 7fdccbb13a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:32.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.637+0000 7fdccbb13a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:32.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.637+0000 7fdccbb13a80 5 note: GC not initialized 2026-02-24T22:23:32.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.637+0000 7fdc767e4640 20 reqs_thread_entry: start 2026-02-24T22:23:32.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.637+0000 7fdcc9de8640 0 -- 10.20.193.46:0/3232333739 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x555b0976f4e0 msgr2=0x555b0976f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:32.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:32.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 20 Filter name: none 2026-02-24T22:23:32.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdc74fe1640 20 reqs_thread_entry: start 2026-02-24T22:23:32.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 10 cache get: name=default.rgw.meta+users.uid+root1.client.0 : miss 2026-02-24T22:23:32.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:32.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:32.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 10 cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x0 2026-02-24T22:23:32.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 10 adding default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2026-02-24T22:23:32.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 10 cache get: name=default.rgw.meta+users.email+root1+test@test.test : miss 2026-02-24T22:23:32.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:32.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:32.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 10 cache put: name=default.rgw.meta+users.email+root1+test@test.test info.flags=0x0 2026-02-24T22:23:32.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 10 adding default.rgw.meta+users.email+root1+test@test.test to cache LRU end 2026-02-24T22:23:32.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 10 cache get: name=default.rgw.meta+users.keys+YFRXIPWFCTLLKUCOAATR : miss 2026-02-24T22:23:32.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:32.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:32.873 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 10 cache put: name=default.rgw.meta+users.keys+YFRXIPWFCTLLKUCOAATR info.flags=0x0 2026-02-24T22:23:32.873 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 10 adding default.rgw.meta+users.keys+YFRXIPWFCTLLKUCOAATR to cache LRU end 2026-02-24T22:23:32.873 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 10 cache get: name=default.rgw.meta+accounts+account.RGW88888888888888888 : miss 2026-02-24T22:23:32.873 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:32.873 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 20 rados_obj.operate() r=0 bl.length=184 2026-02-24T22:23:32.873 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 10 cache put: name=default.rgw.meta+accounts+account.RGW88888888888888888 info.flags=0x17 2026-02-24T22:23:32.873 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 10 adding default.rgw.meta+accounts+account.RGW88888888888888888 to cache LRU end 2026-02-24T22:23:32.873 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:23:32.873 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 10 cache get: name=default.rgw.meta+users.keys+YFRXIPWFCTLLKUCOAATR : hit (negative entry) 2026-02-24T22:23:32.873 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 10 cache get: name=default.rgw.meta+users.keys+YFRXIPWFCTLLKUCOAATR : hit (negative entry) 2026-02-24T22:23:32.874 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 10 cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x17 2026-02-24T22:23:32.874 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 10 moving default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2026-02-24T22:23:32.874 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdccbb13a80 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:root1.client.0, ofs0, ns] 2026-02-24T22:23:32.874 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdc93fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1331439861894 cookie 93849488759216 notifier 8579 bl.length()=681 2026-02-24T22:23:32.874 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdc93fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x17 2026-02-24T22:23:32.874 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.877+0000 7fdc93fff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2026-02-24T22:23:32.875 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.881+0000 7fdccbb13a80 10 cache put: name=default.rgw.meta+users.email+root1+test@test.test info.flags=0x7 2026-02-24T22:23:32.875 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.881+0000 7fdccbb13a80 10 moving default.rgw.meta+users.email+root1+test@test.test to cache LRU end 2026-02-24T22:23:32.875 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.881+0000 7fdccbb13a80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.email:root1+test@test.test, ofs0, ns] 2026-02-24T22:23:32.875 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.881+0000 7fdc93fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1331439863291 cookie 93849489068224 notifier 8579 bl.length()=189 2026-02-24T22:23:32.875 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.881+0000 7fdc93fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+root1+test@test.test info.flags=0x7 2026-02-24T22:23:32.875 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.881+0000 7fdc93fff640 10 rgw watcher librados: moving default.rgw.meta+users.email+root1+test@test.test to cache LRU end 2026-02-24T22:23:32.876 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.881+0000 7fdccbb13a80 10 cache put: name=default.rgw.meta+users.keys+YFRXIPWFCTLLKUCOAATR info.flags=0x7 2026-02-24T22:23:32.876 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.881+0000 7fdccbb13a80 10 moving default.rgw.meta+users.keys+YFRXIPWFCTLLKUCOAATR to cache LRU end 2026-02-24T22:23:32.876 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.881+0000 7fdccbb13a80 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.keys:YFRXIPWFCTLLKUCOAATR, ofs0, ns] 2026-02-24T22:23:32.876 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.881+0000 7fdca4ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1331439863264 cookie 93849489064880 notifier 8579 bl.length()=188 2026-02-24T22:23:32.876 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.881+0000 7fdca4ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+YFRXIPWFCTLLKUCOAATR info.flags=0x7 2026-02-24T22:23:32.876 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.881+0000 7fdca4ff9640 10 rgw watcher librados: moving default.rgw.meta+users.keys+YFRXIPWFCTLLKUCOAATR to cache LRU end 2026-02-24T22:23:32.877 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.881+0000 7fdccbb13a80 20 linked user root1.client.0 to account RGW88888888888888888 2026-02-24T22:23:32.877 INFO:teuthology.orchestra.run.trial046.stdout:{ 2026-02-24T22:23:32.878 INFO:teuthology.orchestra.run.trial046.stdout: "user_id": "root1.client.0", 2026-02-24T22:23:32.878 INFO:teuthology.orchestra.run.trial046.stdout: "display_name": "Mr.RGW88888888888888888", 2026-02-24T22:23:32.878 INFO:teuthology.orchestra.run.trial046.stdout: "email": "root1+test@test.test", 2026-02-24T22:23:32.878 INFO:teuthology.orchestra.run.trial046.stdout: "suspended": 0, 2026-02-24T22:23:32.878 INFO:teuthology.orchestra.run.trial046.stdout: "max_buckets": 1000, 2026-02-24T22:23:32.878 INFO:teuthology.orchestra.run.trial046.stdout: "subusers": [], 2026-02-24T22:23:32.878 INFO:teuthology.orchestra.run.trial046.stdout: "keys": [ 2026-02-24T22:23:32.878 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:23:32.878 INFO:teuthology.orchestra.run.trial046.stdout: "user": "root1.client.0", 2026-02-24T22:23:32.878 INFO:teuthology.orchestra.run.trial046.stdout: "access_key": "YFRXIPWFCTLLKUCOAATR", 2026-02-24T22:23:32.878 INFO:teuthology.orchestra.run.trial046.stdout: "secret_key": "qpQPOUKEhiv5xYwfttkS5caDASAjBeuDiXZ6FQf2gIeEGpK+2P6FSQ==", 2026-02-24T22:23:32.879 INFO:teuthology.orchestra.run.trial046.stdout: "active": true, 2026-02-24T22:23:32.879 INFO:teuthology.orchestra.run.trial046.stdout: "create_date": "2026-02-24T22:23:32.881967Z" 2026-02-24T22:23:32.879 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:23:32.879 INFO:teuthology.orchestra.run.trial046.stdout: ], 2026-02-24T22:23:32.879 INFO:teuthology.orchestra.run.trial046.stdout: "swift_keys": [], 2026-02-24T22:23:32.879 INFO:teuthology.orchestra.run.trial046.stdout: "caps": [], 2026-02-24T22:23:32.879 INFO:teuthology.orchestra.run.trial046.stdout: "op_mask": "read, write, delete", 2026-02-24T22:23:32.879 INFO:teuthology.orchestra.run.trial046.stdout: "default_placement": "", 2026-02-24T22:23:32.879 INFO:teuthology.orchestra.run.trial046.stdout: "default_storage_class": "", 2026-02-24T22:23:32.879 INFO:teuthology.orchestra.run.trial046.stdout: "placement_tags": [], 2026-02-24T22:23:32.879 INFO:teuthology.orchestra.run.trial046.stdout: "bucket_quota": { 2026-02-24T22:23:32.879 INFO:teuthology.orchestra.run.trial046.stdout: "enabled": false, 2026-02-24T22:23:32.880 INFO:teuthology.orchestra.run.trial046.stdout: "check_on_raw": false, 2026-02-24T22:23:32.880 INFO:teuthology.orchestra.run.trial046.stdout: "max_size": -1, 2026-02-24T22:23:32.880 INFO:teuthology.orchestra.run.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:23:32.880 INFO:teuthology.orchestra.run.trial046.stdout: "max_objects": -1 2026-02-24T22:23:32.880 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:23:32.880 INFO:teuthology.orchestra.run.trial046.stdout: "user_quota": { 2026-02-24T22:23:32.880 INFO:teuthology.orchestra.run.trial046.stdout: "enabled": false, 2026-02-24T22:23:32.880 INFO:teuthology.orchestra.run.trial046.stdout: "check_on_raw": false, 2026-02-24T22:23:32.880 INFO:teuthology.orchestra.run.trial046.stdout: "max_size": -1, 2026-02-24T22:23:32.880 INFO:teuthology.orchestra.run.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:23:32.880 INFO:teuthology.orchestra.run.trial046.stdout: "max_objects": -1 2026-02-24T22:23:32.880 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:23:32.880 INFO:teuthology.orchestra.run.trial046.stdout: "temp_url_keys": [], 2026-02-24T22:23:32.881 INFO:teuthology.orchestra.run.trial046.stdout: "type": "root", 2026-02-24T22:23:32.881 INFO:teuthology.orchestra.run.trial046.stdout: "mfa_ids": [], 2026-02-24T22:23:32.881 INFO:teuthology.orchestra.run.trial046.stdout: "account_id": "RGW88888888888888888", 2026-02-24T22:23:32.881 INFO:teuthology.orchestra.run.trial046.stdout: "path": "/", 2026-02-24T22:23:32.881 INFO:teuthology.orchestra.run.trial046.stdout: "create_date": "2026-02-24T22:23:32.881956Z", 2026-02-24T22:23:32.881 INFO:teuthology.orchestra.run.trial046.stdout: "tags": [], 2026-02-24T22:23:32.881 INFO:teuthology.orchestra.run.trial046.stdout: "group_ids": [] 2026-02-24T22:23:32.881 INFO:teuthology.orchestra.run.trial046.stdout:} 2026-02-24T22:23:32.881 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:23:32.882 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.885+0000 7fdccbb13a80 20 remove_watcher() i=3 2026-02-24T22:23:32.882 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.885+0000 7fdccbb13a80 2 removed watcher, disabling cache 2026-02-24T22:23:32.882 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.885+0000 7fdccbb13a80 20 remove_watcher() i=2 2026-02-24T22:23:32.882 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.885+0000 7fdccbb13a80 20 remove_watcher() i=4 2026-02-24T22:23:32.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.885+0000 7fdccbb13a80 20 remove_watcher() i=1 2026-02-24T22:23:32.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.885+0000 7fdccbb13a80 20 remove_watcher() i=5 2026-02-24T22:23:32.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.885+0000 7fdccbb13a80 20 remove_watcher() i=6 2026-02-24T22:23:32.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.885+0000 7fdccbb13a80 20 remove_watcher() i=0 2026-02-24T22:23:32.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.885+0000 7fdccbb13a80 20 remove_watcher() i=7 2026-02-24T22:23:32.887 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid root1.client.0 --totp-serial 3236524159 --totp-seed IJO73N5G5ZPJDHHWNREX6R753HCN2YYJUQOXS6NT3BRMR3WZHZVXTTNPNRGGEMOJ --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2026-02-24T22:23:32.950 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:32.950 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:32.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.973+0000 7f58a629fa80 4 Realm: () 2026-02-24T22:23:32.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.973+0000 7f58a629fa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:32.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.973+0000 7f58a629fa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:32.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.973+0000 7f58a629fa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:32.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.973+0000 7f58a629fa80 20 zonegroup default 2026-02-24T22:23:32.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.973+0000 7f58a629fa80 20 rados->read ofs=0 len=0 2026-02-24T22:23:32.967 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.973+0000 7f58a629fa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:32.967 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.973+0000 7f58a629fa80 20 rados->read ofs=0 len=0 2026-02-24T22:23:32.967 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.973+0000 7f58a629fa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:32.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.973+0000 7f58a629fa80 20 started sync module instance, tier type = 2026-02-24T22:23:32.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.973+0000 7f58a629fa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:32.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.973+0000 7f589f7fe640 0 -- 10.20.193.46:0/3813183640 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x55cfb282f350 msgr2=0x55cfb284f7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:32.969 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.973+0000 7f58a629fa80 20 add_watcher() i=1 2026-02-24T22:23:32.970 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.973+0000 7f58a629fa80 20 add_watcher() i=6 2026-02-24T22:23:32.970 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.977+0000 7f58a629fa80 20 add_watcher() i=3 2026-02-24T22:23:32.970 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.977+0000 7f58a629fa80 20 add_watcher() i=0 2026-02-24T22:23:32.970 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.977+0000 7f58a629fa80 20 add_watcher() i=5 2026-02-24T22:23:32.970 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.977+0000 7f58a629fa80 20 add_watcher() i=2 2026-02-24T22:23:32.970 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.977+0000 7f58a629fa80 20 add_watcher() i=4 2026-02-24T22:23:32.971 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.977+0000 7f58a629fa80 20 add_watcher() i=7 2026-02-24T22:23:32.971 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.977+0000 7f58a629fa80 2 all 8 watchers are set, enabling cache 2026-02-24T22:23:32.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.977+0000 7f58a629fa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:32.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.977+0000 7f58a629fa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:32.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.977+0000 7f58a629fa80 5 note: GC not initialized 2026-02-24T22:23:32.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.977+0000 7f5850fe1640 20 reqs_thread_entry: start 2026-02-24T22:23:32.973 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:32.977+0000 7f589ffff640 0 -- 10.20.193.46:0/711673521 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55cfb284b230 msgr2=0x55cfb284b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:23:33.213 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.217+0000 7f58a629fa80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:33.213 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.217+0000 7f58a629fa80 20 Filter name: none 2026-02-24T22:23:33.213 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.217+0000 7f57baffd640 20 reqs_thread_entry: start 2026-02-24T22:23:33.213 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.217+0000 7f58a629fa80 10 cache get: name=default.rgw.meta+users.uid+root1.client.0 : miss 2026-02-24T22:23:33.213 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.217+0000 7f58a629fa80 20 rados->read ofs=0 len=0 2026-02-24T22:23:33.214 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.221+0000 7f58a629fa80 20 rados_obj.operate() r=0 bl.length=494 2026-02-24T22:23:33.214 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.221+0000 7f58a629fa80 10 cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x13 2026-02-24T22:23:33.214 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.221+0000 7f58a629fa80 10 adding default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2026-02-24T22:23:33.214 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.221+0000 7f58a629fa80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:23:33.215 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.221+0000 7f58a629fa80 10 cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x17 2026-02-24T22:23:33.215 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.221+0000 7f58a629fa80 10 moving default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2026-02-24T22:23:33.216 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.221+0000 7f58a629fa80 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:root1.client.0, ofs0, ns] 2026-02-24T22:23:33.216 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.221+0000 7f587a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1331439861895 cookie 94350541652624 notifier 9249 bl.length()=695 2026-02-24T22:23:33.216 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.221+0000 7f587a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x17 2026-02-24T22:23:33.216 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.221+0000 7f587a7fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2026-02-24T22:23:33.219 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.225+0000 7f58a629fa80 20 remove_watcher() i=3 2026-02-24T22:23:33.219 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.225+0000 7f58a629fa80 2 removed watcher, disabling cache 2026-02-24T22:23:33.219 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.225+0000 7f58a629fa80 20 remove_watcher() i=2 2026-02-24T22:23:33.219 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.225+0000 7f58a629fa80 20 remove_watcher() i=6 2026-02-24T22:23:33.219 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.225+0000 7f58a629fa80 20 remove_watcher() i=4 2026-02-24T22:23:33.220 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.225+0000 7f58a629fa80 20 remove_watcher() i=0 2026-02-24T22:23:33.220 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.225+0000 7f58a629fa80 20 remove_watcher() i=1 2026-02-24T22:23:33.220 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.225+0000 7f58a629fa80 20 remove_watcher() i=7 2026-02-24T22:23:33.220 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.225+0000 7f58a629fa80 20 remove_watcher() i=5 2026-02-24T22:23:33.225 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph account create --account-id RGW99999999999999999 --account-name Mr.RGW99999999999999999 --email RGW99999999999999999+test@test.test 2026-02-24T22:23:33.288 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:33.288 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:33.302 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.309+0000 7f4dbb4d9a80 4 Realm: () 2026-02-24T22:23:33.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.309+0000 7f4dbb4d9a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:33.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.309+0000 7f4dbb4d9a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:33.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.309+0000 7f4dbb4d9a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:33.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.309+0000 7f4dbb4d9a80 20 zonegroup default 2026-02-24T22:23:33.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.309+0000 7f4dbb4d9a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:33.304 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.309+0000 7f4dbb4d9a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:33.304 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.309+0000 7f4dbb4d9a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:33.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.309+0000 7f4dbb4d9a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:33.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.309+0000 7f4dbb4d9a80 20 started sync module instance, tier type = 2026-02-24T22:23:33.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.309+0000 7f4dbb4d9a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:33.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.313+0000 7f4db97ae640 0 -- 10.20.193.46:0/933756981 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55b55d7838b0 msgr2=0x55b55d783d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:33.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.313+0000 7f4dbb4d9a80 20 add_watcher() i=1 2026-02-24T22:23:33.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.313+0000 7f4dbb4d9a80 20 add_watcher() i=5 2026-02-24T22:23:33.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.313+0000 7f4dbb4d9a80 20 add_watcher() i=6 2026-02-24T22:23:33.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.313+0000 7f4dbb4d9a80 20 add_watcher() i=3 2026-02-24T22:23:33.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.313+0000 7f4dbb4d9a80 20 add_watcher() i=2 2026-02-24T22:23:33.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.313+0000 7f4dbb4d9a80 20 add_watcher() i=7 2026-02-24T22:23:33.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.313+0000 7f4dbb4d9a80 20 add_watcher() i=4 2026-02-24T22:23:33.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.313+0000 7f4dbb4d9a80 20 add_watcher() i=0 2026-02-24T22:23:33.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.313+0000 7f4dbb4d9a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:23:33.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.313+0000 7f4dbb4d9a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:33.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.313+0000 7f4dbb4d9a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:33.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.313+0000 7f4dbb4d9a80 5 note: GC not initialized 2026-02-24T22:23:33.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.313+0000 7f4d65fe3640 20 reqs_thread_entry: start 2026-02-24T22:23:33.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.317+0000 7f4db97ae640 0 -- 10.20.193.46:0/1567967156 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55b55d985d40 msgr2=0x55b55d9861b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:33.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.557+0000 7f4dbb4d9a80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:33.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.557+0000 7f4dbb4d9a80 20 Filter name: none 2026-02-24T22:23:33.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.557+0000 7f4cbffff640 20 reqs_thread_entry: start 2026-02-24T22:23:33.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.557+0000 7f4dbb4d9a80 10 cache get: name=default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 : miss 2026-02-24T22:23:33.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.557+0000 7f4dbb4d9a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:33.551 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.557+0000 7f4dbb4d9a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:33.551 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.557+0000 7f4dbb4d9a80 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 info.flags=0x0 2026-02-24T22:23:33.551 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.557+0000 7f4dbb4d9a80 10 adding default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 to cache LRU end 2026-02-24T22:23:33.551 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.557+0000 7f4dbb4d9a80 20 failed to read name.$Mr.RGW99999999999999999 with: (2) No such file or directory 2026-02-24T22:23:33.551 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.557+0000 7f4dbb4d9a80 10 cache get: name=default.rgw.meta+users.email+rgw99999999999999999+test@test.test : miss 2026-02-24T22:23:33.552 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.557+0000 7f4dbb4d9a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:33.552 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.557+0000 7f4dbb4d9a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:33.552 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.557+0000 7f4dbb4d9a80 10 cache put: name=default.rgw.meta+users.email+rgw99999999999999999+test@test.test info.flags=0x0 2026-02-24T22:23:33.552 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.557+0000 7f4dbb4d9a80 10 adding default.rgw.meta+users.email+rgw99999999999999999+test@test.test to cache LRU end 2026-02-24T22:23:33.552 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.557+0000 7f4dbb4d9a80 20 failed to read rgw99999999999999999+test@test.test with: (2) No such file or directory 2026-02-24T22:23:33.552 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.557+0000 7f4dbb4d9a80 10 cache put: name=default.rgw.meta+accounts+account.RGW99999999999999999 info.flags=0x17 2026-02-24T22:23:33.553 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.557+0000 7f4dbb4d9a80 10 adding default.rgw.meta+accounts+account.RGW99999999999999999 to cache LRU end 2026-02-24T22:23:33.553 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.557+0000 7f4dbb4d9a80 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:accounts:account.RGW99999999999999999, ofs0, ns] 2026-02-24T22:23:33.553 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.557+0000 7f4d87fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1331439861896 cookie 94237448011568 notifier 9267 bl.length()=384 2026-02-24T22:23:33.553 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.557+0000 7f4d87fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+account.RGW99999999999999999 info.flags=0x17 2026-02-24T22:23:33.553 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.557+0000 7f4d87fff640 10 rgw watcher librados: moving default.rgw.meta+accounts+account.RGW99999999999999999 to cache LRU end 2026-02-24T22:23:33.554 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.561+0000 7f4dbb4d9a80 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 info.flags=0x17 2026-02-24T22:23:33.554 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.561+0000 7f4dbb4d9a80 10 moving default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 to cache LRU end 2026-02-24T22:23:33.554 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.561+0000 7f4dbb4d9a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:accounts:name.$Mr.RGW99999999999999999, ofs0, ns] 2026-02-24T22:23:33.554 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.561+0000 7f4d87fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1331439861923 cookie 94237446431440 notifier 9267 bl.length()=225 2026-02-24T22:23:33.554 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.561+0000 7f4d87fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 info.flags=0x17 2026-02-24T22:23:33.554 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.561+0000 7f4d87fff640 10 rgw watcher librados: moving default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 to cache LRU end 2026-02-24T22:23:33.555 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.561+0000 7f4dbb4d9a80 10 cache put: name=default.rgw.meta+users.email+rgw99999999999999999+test@test.test info.flags=0x17 2026-02-24T22:23:33.555 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.561+0000 7f4dbb4d9a80 10 moving default.rgw.meta+users.email+rgw99999999999999999+test@test.test to cache LRU end 2026-02-24T22:23:33.555 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.561+0000 7f4dbb4d9a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:rgw99999999999999999+test@test.test, ofs0, ns] 2026-02-24T22:23:33.555 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.561+0000 7f4d877fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1331439863265 cookie 94237448018096 notifier 9267 bl.length()=234 2026-02-24T22:23:33.556 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.561+0000 7f4d877fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+rgw99999999999999999+test@test.test info.flags=0x17 2026-02-24T22:23:33.556 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.561+0000 7f4d877fe640 10 rgw watcher librados: moving default.rgw.meta+users.email+rgw99999999999999999+test@test.test to cache LRU end 2026-02-24T22:23:33.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.565+0000 7f4dbb4d9a80 20 remove_watcher() i=3 2026-02-24T22:23:33.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.565+0000 7f4dbb4d9a80 2 removed watcher, disabling cache 2026-02-24T22:23:33.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.565+0000 7f4dbb4d9a80 20 remove_watcher() i=5 2026-02-24T22:23:33.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.565+0000 7f4dbb4d9a80 20 remove_watcher() i=2 2026-02-24T22:23:33.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.565+0000 7f4dbb4d9a80 20 remove_watcher() i=6 2026-02-24T22:23:33.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.565+0000 7f4dbb4d9a80 20 remove_watcher() i=7 2026-02-24T22:23:33.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.565+0000 7f4dbb4d9a80 20 remove_watcher() i=1 2026-02-24T22:23:33.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.565+0000 7f4dbb4d9a80 20 remove_watcher() i=4 2026-02-24T22:23:33.560 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.565+0000 7f4dbb4d9a80 20 remove_watcher() i=0 2026-02-24T22:23:33.562 INFO:teuthology.orchestra.run.trial046.stdout:{ 2026-02-24T22:23:33.562 INFO:teuthology.orchestra.run.trial046.stdout: "id": "RGW99999999999999999", 2026-02-24T22:23:33.562 INFO:teuthology.orchestra.run.trial046.stdout: "tenant": "", 2026-02-24T22:23:33.562 INFO:teuthology.orchestra.run.trial046.stdout: "name": "Mr.RGW99999999999999999", 2026-02-24T22:23:33.562 INFO:teuthology.orchestra.run.trial046.stdout: "email": "RGW99999999999999999+test@test.test", 2026-02-24T22:23:33.562 INFO:teuthology.orchestra.run.trial046.stdout: "quota": { 2026-02-24T22:23:33.563 INFO:teuthology.orchestra.run.trial046.stdout: "enabled": false, 2026-02-24T22:23:33.563 INFO:teuthology.orchestra.run.trial046.stdout: "check_on_raw": false, 2026-02-24T22:23:33.563 INFO:teuthology.orchestra.run.trial046.stdout: "max_size": -1, 2026-02-24T22:23:33.563 INFO:teuthology.orchestra.run.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:23:33.563 INFO:teuthology.orchestra.run.trial046.stdout: "max_objects": -1 2026-02-24T22:23:33.563 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:23:33.563 INFO:teuthology.orchestra.run.trial046.stdout: "bucket_quota": { 2026-02-24T22:23:33.563 INFO:teuthology.orchestra.run.trial046.stdout: "enabled": false, 2026-02-24T22:23:33.563 INFO:teuthology.orchestra.run.trial046.stdout: "check_on_raw": false, 2026-02-24T22:23:33.563 INFO:teuthology.orchestra.run.trial046.stdout: "max_size": -1, 2026-02-24T22:23:33.563 INFO:teuthology.orchestra.run.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:23:33.563 INFO:teuthology.orchestra.run.trial046.stdout: "max_objects": -1 2026-02-24T22:23:33.564 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:23:33.564 INFO:teuthology.orchestra.run.trial046.stdout: "max_users": 1000, 2026-02-24T22:23:33.564 INFO:teuthology.orchestra.run.trial046.stdout: "max_roles": 1000, 2026-02-24T22:23:33.564 INFO:teuthology.orchestra.run.trial046.stdout: "max_groups": 1000, 2026-02-24T22:23:33.564 INFO:teuthology.orchestra.run.trial046.stdout: "max_buckets": 1000, 2026-02-24T22:23:33.564 INFO:teuthology.orchestra.run.trial046.stdout: "max_access_keys": 4 2026-02-24T22:23:33.564 INFO:teuthology.orchestra.run.trial046.stdout:} 2026-02-24T22:23:33.566 DEBUG:tasks.s3tests:Creating user RGW99999999999999999 on client.0 2026-02-24T22:23:33.566 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid root2.client.0 --display-name Mr.RGW99999999999999999 --email root2+test@test.test --access-key FYRXAKZJTVNMCJKDMCVA --secret oStNY5GISRNnSpPIvQBwEqjoFkvG27KteBIN67EmL/pAe9o1zKD89Q== --cluster ceph --account-id RGW99999999999999999 --account-root 2026-02-24T22:23:33.630 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:33.631 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:33.646 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.653+0000 7fe373b9ea80 4 Realm: () 2026-02-24T22:23:33.646 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.653+0000 7fe373b9ea80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:33.647 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.653+0000 7fe373b9ea80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:33.647 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.653+0000 7fe373b9ea80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:33.647 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.653+0000 7fe373b9ea80 20 zonegroup default 2026-02-24T22:23:33.647 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.653+0000 7fe373b9ea80 20 rados->read ofs=0 len=0 2026-02-24T22:23:33.648 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.653+0000 7fe373b9ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:33.648 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.653+0000 7fe373b9ea80 20 rados->read ofs=0 len=0 2026-02-24T22:23:33.649 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.653+0000 7fe373b9ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:33.649 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.653+0000 7fe373b9ea80 20 started sync module instance, tier type = 2026-02-24T22:23:33.649 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.653+0000 7fe373b9ea80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:33.650 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.657+0000 7fe371672640 0 -- 10.20.193.46:0/2892243904 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x5622800de8e0 msgr2=0x5622800fecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:33.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.657+0000 7fe373b9ea80 20 add_watcher() i=3 2026-02-24T22:23:33.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.657+0000 7fe373b9ea80 20 add_watcher() i=1 2026-02-24T22:23:33.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.657+0000 7fe373b9ea80 20 add_watcher() i=5 2026-02-24T22:23:33.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.657+0000 7fe373b9ea80 20 add_watcher() i=6 2026-02-24T22:23:33.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.657+0000 7fe373b9ea80 20 add_watcher() i=0 2026-02-24T22:23:33.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.657+0000 7fe373b9ea80 20 add_watcher() i=2 2026-02-24T22:23:33.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.657+0000 7fe373b9ea80 20 add_watcher() i=4 2026-02-24T22:23:33.856 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.861+0000 7fe373b9ea80 20 add_watcher() i=7 2026-02-24T22:23:33.856 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.861+0000 7fe373b9ea80 2 all 8 watchers are set, enabling cache 2026-02-24T22:23:33.856 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:33.861+0000 7fe370e71640 0 -- 10.20.193.46:0/669525590 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7fe290008af0 msgr2=0x7fe290028fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:34.059 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.065+0000 7fe373b9ea80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:34.060 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.065+0000 7fe373b9ea80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:34.060 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.065+0000 7fe373b9ea80 5 note: GC not initialized 2026-02-24T22:23:34.060 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.065+0000 7fe31e7e4640 20 reqs_thread_entry: start 2026-02-24T22:23:34.094 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:34.094 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 20 Filter name: none 2026-02-24T22:23:34.094 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe31cfe1640 20 reqs_thread_entry: start 2026-02-24T22:23:34.095 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 10 cache get: name=default.rgw.meta+users.uid+root2.client.0 : miss 2026-02-24T22:23:34.095 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 20 rados->read ofs=0 len=0 2026-02-24T22:23:34.096 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:34.096 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 10 cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x0 2026-02-24T22:23:34.096 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 10 adding default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2026-02-24T22:23:34.096 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 10 cache get: name=default.rgw.meta+users.email+root2+test@test.test : miss 2026-02-24T22:23:34.096 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 20 rados->read ofs=0 len=0 2026-02-24T22:23:34.096 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:34.096 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 10 cache put: name=default.rgw.meta+users.email+root2+test@test.test info.flags=0x0 2026-02-24T22:23:34.097 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 10 adding default.rgw.meta+users.email+root2+test@test.test to cache LRU end 2026-02-24T22:23:34.097 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 10 cache get: name=default.rgw.meta+users.keys+FYRXAKZJTVNMCJKDMCVA : miss 2026-02-24T22:23:34.097 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 20 rados->read ofs=0 len=0 2026-02-24T22:23:34.097 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:34.097 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 10 cache put: name=default.rgw.meta+users.keys+FYRXAKZJTVNMCJKDMCVA info.flags=0x0 2026-02-24T22:23:34.097 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 10 adding default.rgw.meta+users.keys+FYRXAKZJTVNMCJKDMCVA to cache LRU end 2026-02-24T22:23:34.097 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 10 cache get: name=default.rgw.meta+accounts+account.RGW99999999999999999 : miss 2026-02-24T22:23:34.097 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 20 rados->read ofs=0 len=0 2026-02-24T22:23:34.097 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 20 rados_obj.operate() r=0 bl.length=184 2026-02-24T22:23:34.097 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 10 cache put: name=default.rgw.meta+accounts+account.RGW99999999999999999 info.flags=0x17 2026-02-24T22:23:34.097 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 10 adding default.rgw.meta+accounts+account.RGW99999999999999999 to cache LRU end 2026-02-24T22:23:34.097 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:23:34.098 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 10 cache get: name=default.rgw.meta+users.keys+FYRXAKZJTVNMCJKDMCVA : hit (negative entry) 2026-02-24T22:23:34.098 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 10 cache get: name=default.rgw.meta+users.keys+FYRXAKZJTVNMCJKDMCVA : hit (negative entry) 2026-02-24T22:23:34.098 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 10 cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x17 2026-02-24T22:23:34.099 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 10 moving default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2026-02-24T22:23:34.099 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe373b9ea80 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:root2.client.0, ofs0, ns] 2026-02-24T22:23:34.099 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe33bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1331439861897 cookie 94706177406784 notifier 8627 bl.length()=681 2026-02-24T22:23:34.099 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe33bfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x17 2026-02-24T22:23:34.099 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.101+0000 7fe33bfff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2026-02-24T22:23:34.099 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.105+0000 7fe373b9ea80 10 cache put: name=default.rgw.meta+users.email+root2+test@test.test info.flags=0x7 2026-02-24T22:23:34.099 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.105+0000 7fe373b9ea80 10 moving default.rgw.meta+users.email+root2+test@test.test to cache LRU end 2026-02-24T22:23:34.099 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.105+0000 7fe373b9ea80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.email:root2+test@test.test, ofs0, ns] 2026-02-24T22:23:34.099 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.105+0000 7fe33bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1331439863292 cookie 94706175024304 notifier 8627 bl.length()=189 2026-02-24T22:23:34.099 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.105+0000 7fe33bfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+root2+test@test.test info.flags=0x7 2026-02-24T22:23:34.099 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.105+0000 7fe33bfff640 10 rgw watcher librados: moving default.rgw.meta+users.email+root2+test@test.test to cache LRU end 2026-02-24T22:23:34.100 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.105+0000 7fe373b9ea80 10 cache put: name=default.rgw.meta+users.keys+FYRXAKZJTVNMCJKDMCVA info.flags=0x7 2026-02-24T22:23:34.100 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.105+0000 7fe373b9ea80 10 moving default.rgw.meta+users.keys+FYRXAKZJTVNMCJKDMCVA to cache LRU end 2026-02-24T22:23:34.100 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.105+0000 7fe373b9ea80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:FYRXAKZJTVNMCJKDMCVA, ofs0, ns] 2026-02-24T22:23:34.100 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.105+0000 7fe34cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1331439861796 cookie 94706177411200 notifier 8627 bl.length()=188 2026-02-24T22:23:34.100 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.105+0000 7fe34cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+FYRXAKZJTVNMCJKDMCVA info.flags=0x7 2026-02-24T22:23:34.101 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.105+0000 7fe34cff9640 10 rgw watcher librados: moving default.rgw.meta+users.keys+FYRXAKZJTVNMCJKDMCVA to cache LRU end 2026-02-24T22:23:34.101 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.105+0000 7fe373b9ea80 20 linked user root2.client.0 to account RGW99999999999999999 2026-02-24T22:23:34.101 INFO:teuthology.orchestra.run.trial046.stdout:{ 2026-02-24T22:23:34.101 INFO:teuthology.orchestra.run.trial046.stdout: "user_id": "root2.client.0", 2026-02-24T22:23:34.102 INFO:teuthology.orchestra.run.trial046.stdout: "display_name": "Mr.RGW99999999999999999", 2026-02-24T22:23:34.102 INFO:teuthology.orchestra.run.trial046.stdout: "email": "root2+test@test.test", 2026-02-24T22:23:34.102 INFO:teuthology.orchestra.run.trial046.stdout: "suspended": 0, 2026-02-24T22:23:34.102 INFO:teuthology.orchestra.run.trial046.stdout: "max_buckets": 1000, 2026-02-24T22:23:34.102 INFO:teuthology.orchestra.run.trial046.stdout: "subusers": [], 2026-02-24T22:23:34.102 INFO:teuthology.orchestra.run.trial046.stdout: "keys": [ 2026-02-24T22:23:34.102 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:23:34.102 INFO:teuthology.orchestra.run.trial046.stdout: "user": "root2.client.0", 2026-02-24T22:23:34.102 INFO:teuthology.orchestra.run.trial046.stdout: "access_key": "FYRXAKZJTVNMCJKDMCVA", 2026-02-24T22:23:34.102 INFO:teuthology.orchestra.run.trial046.stdout: "secret_key": "oStNY5GISRNnSpPIvQBwEqjoFkvG27KteBIN67EmL/pAe9o1zKD89Q==", 2026-02-24T22:23:34.102 INFO:teuthology.orchestra.run.trial046.stdout: "active": true, 2026-02-24T22:23:34.102 INFO:teuthology.orchestra.run.trial046.stdout: "create_date": "2026-02-24T22:23:34.105962Z" 2026-02-24T22:23:34.103 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:23:34.103 INFO:teuthology.orchestra.run.trial046.stdout: ], 2026-02-24T22:23:34.103 INFO:teuthology.orchestra.run.trial046.stdout: "swift_keys": [], 2026-02-24T22:23:34.103 INFO:teuthology.orchestra.run.trial046.stdout: "caps": [], 2026-02-24T22:23:34.103 INFO:teuthology.orchestra.run.trial046.stdout: "op_mask": "read, write, delete", 2026-02-24T22:23:34.103 INFO:teuthology.orchestra.run.trial046.stdout: "default_placement": "", 2026-02-24T22:23:34.103 INFO:teuthology.orchestra.run.trial046.stdout: "default_storage_class": "", 2026-02-24T22:23:34.103 INFO:teuthology.orchestra.run.trial046.stdout: "placement_tags": [], 2026-02-24T22:23:34.103 INFO:teuthology.orchestra.run.trial046.stdout: "bucket_quota": { 2026-02-24T22:23:34.103 INFO:teuthology.orchestra.run.trial046.stdout: "enabled": false, 2026-02-24T22:23:34.103 INFO:teuthology.orchestra.run.trial046.stdout: "check_on_raw": false, 2026-02-24T22:23:34.103 INFO:teuthology.orchestra.run.trial046.stdout: "max_size": -1, 2026-02-24T22:23:34.103 INFO:teuthology.orchestra.run.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:23:34.104 INFO:teuthology.orchestra.run.trial046.stdout: "max_objects": -1 2026-02-24T22:23:34.104 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:23:34.104 INFO:teuthology.orchestra.run.trial046.stdout: "user_quota": { 2026-02-24T22:23:34.104 INFO:teuthology.orchestra.run.trial046.stdout: "enabled": false, 2026-02-24T22:23:34.104 INFO:teuthology.orchestra.run.trial046.stdout: "check_on_raw": false, 2026-02-24T22:23:34.104 INFO:teuthology.orchestra.run.trial046.stdout: "max_size": -1, 2026-02-24T22:23:34.104 INFO:teuthology.orchestra.run.trial046.stdout: "max_size_kb": 0, 2026-02-24T22:23:34.104 INFO:teuthology.orchestra.run.trial046.stdout: "max_objects": -1 2026-02-24T22:23:34.104 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:23:34.104 INFO:teuthology.orchestra.run.trial046.stdout: "temp_url_keys": [], 2026-02-24T22:23:34.104 INFO:teuthology.orchestra.run.trial046.stdout: "type": "root", 2026-02-24T22:23:34.104 INFO:teuthology.orchestra.run.trial046.stdout: "mfa_ids": [], 2026-02-24T22:23:34.104 INFO:teuthology.orchestra.run.trial046.stdout: "account_id": "RGW99999999999999999", 2026-02-24T22:23:34.105 INFO:teuthology.orchestra.run.trial046.stdout: "path": "/", 2026-02-24T22:23:34.105 INFO:teuthology.orchestra.run.trial046.stdout: "create_date": "2026-02-24T22:23:34.105951Z", 2026-02-24T22:23:34.105 INFO:teuthology.orchestra.run.trial046.stdout: "tags": [], 2026-02-24T22:23:34.105 INFO:teuthology.orchestra.run.trial046.stdout: "group_ids": [] 2026-02-24T22:23:34.105 INFO:teuthology.orchestra.run.trial046.stdout:} 2026-02-24T22:23:34.105 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:23:34.106 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.109+0000 7fe373b9ea80 20 remove_watcher() i=3 2026-02-24T22:23:34.106 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.109+0000 7fe373b9ea80 2 removed watcher, disabling cache 2026-02-24T22:23:34.106 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.109+0000 7fe373b9ea80 20 remove_watcher() i=2 2026-02-24T22:23:34.106 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.109+0000 7fe373b9ea80 20 remove_watcher() i=4 2026-02-24T22:23:34.106 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.109+0000 7fe373b9ea80 20 remove_watcher() i=6 2026-02-24T22:23:34.106 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.109+0000 7fe373b9ea80 20 remove_watcher() i=0 2026-02-24T22:23:34.106 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.109+0000 7fe373b9ea80 20 remove_watcher() i=1 2026-02-24T22:23:34.107 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.109+0000 7fe373b9ea80 20 remove_watcher() i=5 2026-02-24T22:23:34.107 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.109+0000 7fe373b9ea80 20 remove_watcher() i=7 2026-02-24T22:23:34.111 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid root2.client.0 --totp-serial 8395217603 --totp-seed 7ISA7GF4UNIZJFH72HFC6S6WQTVIZDIEAANKLQLLSTKJ2QOI22WF5HDNTOP45RJV --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2026-02-24T22:23:34.175 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:23:34.175 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:23:34.191 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.197+0000 7fca4c187a80 4 Realm: () 2026-02-24T22:23:34.191 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.197+0000 7fca4c187a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:23:34.191 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.197+0000 7fca4c187a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:23:34.192 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.197+0000 7fca4c187a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:23:34.192 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.197+0000 7fca4c187a80 20 zonegroup default 2026-02-24T22:23:34.192 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.197+0000 7fca4c187a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:34.193 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.197+0000 7fca4c187a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:34.193 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.197+0000 7fca4c187a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:34.193 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.201+0000 7fca4c187a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:23:34.194 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.201+0000 7fca4c187a80 20 started sync module instance, tier type = 2026-02-24T22:23:34.194 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.201+0000 7fca4c187a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:23:34.194 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.201+0000 7fca4945a640 0 -- 10.20.193.46:0/2630291924 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x557a69129ce0 msgr2=0x557a6912a180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:23:34.195 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.201+0000 7fca4c187a80 20 add_watcher() i=3 2026-02-24T22:23:34.195 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.201+0000 7fca4c187a80 20 add_watcher() i=6 2026-02-24T22:23:34.195 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.201+0000 7fca4c187a80 20 add_watcher() i=5 2026-02-24T22:23:34.196 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.201+0000 7fca4c187a80 20 add_watcher() i=1 2026-02-24T22:23:34.196 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.201+0000 7fca4c187a80 20 add_watcher() i=0 2026-02-24T22:23:34.196 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.201+0000 7fca4c187a80 20 add_watcher() i=7 2026-02-24T22:23:34.196 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.201+0000 7fca4c187a80 20 add_watcher() i=2 2026-02-24T22:23:34.196 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.201+0000 7fca4c187a80 20 add_watcher() i=4 2026-02-24T22:23:34.197 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.201+0000 7fca4c187a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:23:34.197 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.201+0000 7fca4945a640 0 -- 10.20.193.46:0/2234343599 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7fc968008af0 msgr2=0x7fc968028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:23:34.198 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.205+0000 7fca4c187a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:23:34.198 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.205+0000 7fca4c187a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:23:34.198 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.205+0000 7fca4c187a80 5 note: GC not initialized 2026-02-24T22:23:34.198 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.205+0000 7fc97a7fc640 20 reqs_thread_entry: start 2026-02-24T22:23:34.230 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.237+0000 7fca4c187a80 20 init_complete bucket index max shards: 1 2026-02-24T22:23:34.230 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.237+0000 7fca4c187a80 20 Filter name: none 2026-02-24T22:23:34.230 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.237+0000 7fc978ff9640 20 reqs_thread_entry: start 2026-02-24T22:23:34.231 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.237+0000 7fca4c187a80 10 cache get: name=default.rgw.meta+users.uid+root2.client.0 : miss 2026-02-24T22:23:34.231 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.237+0000 7fca4c187a80 20 rados->read ofs=0 len=0 2026-02-24T22:23:34.231 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.237+0000 7fca4c187a80 20 rados_obj.operate() r=0 bl.length=494 2026-02-24T22:23:34.231 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.237+0000 7fca4c187a80 10 cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x13 2026-02-24T22:23:34.231 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.237+0000 7fca4c187a80 10 adding default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2026-02-24T22:23:34.231 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.237+0000 7fca4c187a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:23:34.232 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.237+0000 7fca4c187a80 10 cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x17 2026-02-24T22:23:34.233 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.237+0000 7fca4c187a80 10 moving default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2026-02-24T22:23:34.233 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.237+0000 7fca4c187a80 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:root2.client.0, ofs0, ns] 2026-02-24T22:23:34.233 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.237+0000 7fca24ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1331439861898 cookie 93984239614256 notifier 9306 bl.length()=695 2026-02-24T22:23:34.233 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.237+0000 7fca24ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x17 2026-02-24T22:23:34.233 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.237+0000 7fca24ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2026-02-24T22:23:34.236 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.241+0000 7fca4c187a80 20 remove_watcher() i=1 2026-02-24T22:23:34.236 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.241+0000 7fca4c187a80 2 removed watcher, disabling cache 2026-02-24T22:23:34.236 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.241+0000 7fca4c187a80 20 remove_watcher() i=3 2026-02-24T22:23:34.236 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.241+0000 7fca4c187a80 20 remove_watcher() i=2 2026-02-24T22:23:34.236 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.241+0000 7fca4c187a80 20 remove_watcher() i=4 2026-02-24T22:23:34.237 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.241+0000 7fca4c187a80 20 remove_watcher() i=6 2026-02-24T22:23:34.237 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.241+0000 7fca4c187a80 20 remove_watcher() i=7 2026-02-24T22:23:34.237 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.241+0000 7fca4c187a80 20 remove_watcher() i=5 2026-02-24T22:23:34.237 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:23:34.241+0000 7fca4c187a80 20 remove_watcher() i=0 2026-02-24T22:23:34.243 INFO:tasks.s3tests:Configuring s3-tests... 2026-02-24T22:23:34.244 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:23:34.244 DEBUG:teuthology.orchestra.run.trial046:> dd of=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf 2026-02-24T22:23:34.293 INFO:tasks.s3tests:Configuring boto... 2026-02-24T22:23:34.294 DEBUG:teuthology.orchestra.run.trial046:> set -ex 2026-02-24T22:23:34.294 DEBUG:teuthology.orchestra.run.trial046:> dd of=/home/ubuntu/cephtest/boto-client.0.cfg 2026-02-24T22:23:34.341 DEBUG:teuthology.orchestra.run.trial046:s3 tests against rgw> source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3 and not bucket_logging' 2026-02-24T22:23:34.546 INFO:teuthology.orchestra.run.trial046.stdout:py: install_deps> python -I -m pip install -r requirements.txt 2026-02-24T22:23:38.205 INFO:teuthology.orchestra.run.trial046.stdout:.pkg: install_requires> python -I -m pip install 'setuptools>=40.8.0' 2026-02-24T22:23:38.575 INFO:teuthology.orchestra.run.trial046.stdout:.pkg: _optional_hooks> python /home/ubuntu/cephtest/tox-venv/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__ 2026-02-24T22:23:38.800 INFO:teuthology.orchestra.run.trial046.stdout:.pkg: get_requires_for_build_sdist> python /home/ubuntu/cephtest/tox-venv/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__ 2026-02-24T22:23:38.983 INFO:teuthology.orchestra.run.trial046.stdout:.pkg: get_requires_for_build_wheel> python /home/ubuntu/cephtest/tox-venv/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__ 2026-02-24T22:23:39.157 INFO:teuthology.orchestra.run.trial046.stdout:.pkg: prepare_metadata_for_build_wheel> python /home/ubuntu/cephtest/tox-venv/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__ 2026-02-24T22:23:39.337 INFO:teuthology.orchestra.run.trial046.stdout:.pkg: build_sdist> python /home/ubuntu/cephtest/tox-venv/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__ 2026-02-24T22:23:39.551 INFO:teuthology.orchestra.run.trial046.stdout:py: install_package_deps> python -I -m pip install PyYAML 'boto3>=1.0.0' 'gevent>=1.0' 'isodate>=0.4.4' 'munch>=2.0.0' 2026-02-24T22:23:39.753 INFO:teuthology.orchestra.run.trial046.stdout:py: install_package> python -I -m pip install --force-reinstall --no-deps /home/ubuntu/cephtest/s3-tests-client.0/.tox/.tmp/package/1/s3tests-0.0.1.tar.gz 2026-02-24T22:23:40.736 INFO:teuthology.orchestra.run.trial046.stdout:py: commands[0]> pytest -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3 and not bucket_logging' 2026-02-24T22:23:40.806 INFO:teuthology.orchestra.run.trial046.stdout:============================= test session starts ============================== 2026-02-24T22:23:40.806 INFO:teuthology.orchestra.run.trial046.stdout:platform linux -- Python 3.10.12, pytest-9.0.2, pluggy-1.6.0 -- /home/ubuntu/cephtest/s3-tests-client.0/.tox/py/bin/python 2026-02-24T22:23:40.806 INFO:teuthology.orchestra.run.trial046.stdout:cachedir: .tox/py/.pytest_cache 2026-02-24T22:23:40.806 INFO:teuthology.orchestra.run.trial046.stdout:rootdir: /home/ubuntu/cephtest/s3-tests-client.0 2026-02-24T22:23:40.806 INFO:teuthology.orchestra.run.trial046.stdout:configfile: pytest.ini 2026-02-24T22:23:41.936 INFO:teuthology.orchestra.run.trial046.stdout:collecting ... collected 1700 items / 572 deselected / 1128 selected 2026-02-24T22:23:41.936 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:23:42.508 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_object_create_bad_md5_invalid_short PASSED [ 0%] 2026-02-24T22:23:52.268 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_object_create_bad_md5_bad PASSED [ 0%] 2026-02-24T22:23:52.824 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_object_create_bad_md5_empty PASSED [ 0%] 2026-02-24T22:23:53.296 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_object_create_bad_md5_none PASSED [ 0%] 2026-02-24T22:23:53.828 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_object_create_bad_expect_mismatch PASSED [ 0%] 2026-02-24T22:23:54.388 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_object_create_bad_expect_empty PASSED [ 0%] 2026-02-24T22:23:54.896 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_object_create_bad_expect_none PASSED [ 0%] 2026-02-24T22:23:55.356 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_object_create_bad_contentlength_negative PASSED [ 0%] 2026-02-24T22:23:55.960 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_object_create_bad_contenttype_invalid PASSED [ 0%] 2026-02-24T22:23:56.460 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_object_create_bad_contenttype_empty PASSED [ 0%] 2026-02-24T22:23:56.972 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_object_create_bad_contenttype_none PASSED [ 0%] 2026-02-24T22:23:57.480 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_bucket_put_bad_canned_acl PASSED [ 1%] 2026-02-24T22:23:57.956 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_expect_mismatch PASSED [ 1%] 2026-02-24T22:23:58.408 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_expect_empty PASSED [ 1%] 2026-02-24T22:23:58.800 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_contentlength_negative PASSED [ 1%] 2026-02-24T22:23:59.296 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_object_create_bad_md5_invalid_garbage_aws2 PASSED [ 1%] 2026-02-24T22:23:59.856 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_object_create_bad_ua_empty_aws2 PASSED [ 1%] 2026-02-24T22:24:00.408 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_object_create_bad_ua_none_aws2 PASSED [ 1%] 2026-02-24T22:24:00.904 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_invalid_aws2 PASSED [ 1%] 2026-02-24T22:24:01.471 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_empty_aws2 PASSED [ 1%] 2026-02-24T22:24:01.952 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_before_today_aws2 PASSED [ 1%] 2026-02-24T22:24:02.436 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_before_epoch_aws2 PASSED [ 1%] 2026-02-24T22:24:02.936 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_after_end_aws2 PASSED [ 2%] 2026-02-24T22:24:03.371 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_invalid_aws2 PASSED [ 2%] 2026-02-24T22:24:03.784 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_empty_aws2 PASSED [ 2%] 2026-02-24T22:24:04.200 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_before_today_aws2 PASSED [ 2%] 2026-02-24T22:24:04.607 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_after_today_aws2 PASSED [ 2%] 2026-02-24T22:24:05.031 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_before_epoch_aws2 PASSED [ 2%] 2026-02-24T22:24:05.496 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_put_user_policy PASSED [ 2%] 2026-02-24T22:24:05.908 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_put_user_policy_invalid_user PASSED [ 2%] 2026-02-24T22:24:06.323 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_put_user_policy_parameter_limit PASSED [ 2%] 2026-02-24T22:24:06.868 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_put_existing_user_policy PASSED [ 2%] 2026-02-24T22:24:07.384 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_list_user_policy PASSED [ 2%] 2026-02-24T22:24:07.787 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_list_user_policy_invalid_user PASSED [ 3%] 2026-02-24T22:24:08.280 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_get_user_policy PASSED [ 3%] 2026-02-24T22:24:08.823 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_get_user_policy_invalid_user PASSED [ 3%] 2026-02-24T22:24:09.423 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_get_user_policy_from_multiple_policies PASSED [ 3%] 2026-02-24T22:24:09.860 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_delete_user_policy PASSED [ 3%] 2026-02-24T22:24:10.352 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_delete_user_policy_invalid_user PASSED [ 3%] 2026-02-24T22:24:10.871 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_delete_user_policy_invalid_policy_name PASSED [ 3%] 2026-02-24T22:24:11.563 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_delete_user_policy_from_multiple_policies PASSED [ 3%] 2026-02-24T22:24:12.143 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_allow_bucket_actions_in_user_policy PASSED [ 3%] 2026-02-24T22:24:12.683 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_deny_bucket_actions_in_user_policy PASSED [ 3%] 2026-02-24T22:24:13.207 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_allow_object_actions_in_user_policy PASSED [ 3%] 2026-02-24T22:24:13.860 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_deny_object_actions_in_user_policy PASSED [ 3%] 2026-02-24T22:24:14.523 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_allow_multipart_actions_in_user_policy PASSED [ 4%] 2026-02-24T22:24:15.163 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_deny_multipart_actions_in_user_policy PASSED [ 4%] 2026-02-24T22:24:15.760 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_allow_tagging_actions_in_user_policy PASSED [ 4%] 2026-02-24T22:24:16.387 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_deny_tagging_actions_in_user_policy PASSED [ 4%] 2026-02-24T22:24:16.967 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_verify_conflicting_user_policy_statements PASSED [ 4%] 2026-02-24T22:24:17.639 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_verify_conflicting_user_policies PASSED [ 4%] 2026-02-24T22:24:18.176 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_verify_allow_iam_actions PASSED [ 4%] 2026-02-24T22:24:19.328 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_user_create PASSED [ 4%] 2026-02-24T22:24:20.132 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_user_case_insensitive_name PASSED [ 4%] 2026-02-24T22:24:21.080 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_user_delete PASSED [ 4%] 2026-02-24T22:24:22.911 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_user_list PASSED [ 4%] 2026-02-24T22:24:25.324 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_user_list_path_prefix PASSED [ 5%] 2026-02-24T22:24:26.695 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_user_update_name PASSED [ 5%] 2026-02-24T22:24:27.960 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_user_update_path PASSED [ 5%] 2026-02-24T22:24:28.927 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_user_access_key_create PASSED [ 5%] 2026-02-24T22:24:29.663 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_current_user_access_key_create PASSED [ 5%] 2026-02-24T22:24:30.815 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_user_access_key_update PASSED [ 5%] 2026-02-24T22:24:31.695 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_current_user_access_key_update PASSED [ 5%] 2026-02-24T22:24:32.799 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_user_access_key_delete PASSED [ 5%] 2026-02-24T22:24:33.671 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_current_user_access_key_delete PASSED [ 5%] 2026-02-24T22:24:35.048 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_user_access_key_list PASSED [ 5%] 2026-02-24T22:24:36.075 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_user_bucket_policy_allow PASSED [ 5%] 2026-02-24T22:24:37.615 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_user_policy PASSED [ 6%] 2026-02-24T22:24:39.059 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_user_policy_managed PASSED [ 6%] 2026-02-24T22:24:40.107 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_user_policy_allow PASSED [ 6%] 2026-02-24T22:24:41.071 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_group_create PASSED [ 6%] 2026-02-24T22:24:41.915 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_group_case_insensitive_name PASSED [ 6%] 2026-02-24T22:24:43.695 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_group_list PASSED [ 6%] 2026-02-24T22:24:44.987 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_group_update PASSED [ 6%] 2026-02-24T22:24:46.519 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_inline_group_policy PASSED [ 6%] 2026-02-24T22:24:47.987 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_managed_group_policy PASSED [ 6%] 2026-02-24T22:24:49.275 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_inline_group_policy_allow PASSED [ 6%] 2026-02-24T22:24:50.559 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_managed_group_policy_allow PASSED [ 6%] 2026-02-24T22:24:51.635 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_role_create PASSED [ 7%] 2026-02-24T22:24:52.459 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_role_case_insensitive_name PASSED [ 7%] 2026-02-24T22:24:53.367 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_role_delete PASSED [ 7%] 2026-02-24T22:24:54.955 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_role_list PASSED [ 7%] 2026-02-24T22:24:57.091 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_role_list_path_prefix PASSED [ 7%] 2026-02-24T22:24:58.267 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_role_update PASSED [ 7%] 2026-02-24T22:24:59.647 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_role_policy PASSED [ 7%] 2026-02-24T22:25:01.039 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_role_policy_managed PASSED [ 7%] 2026-02-24T22:25:02.323 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_role_policy_allow PASSED [ 7%] 2026-02-24T22:25:03.683 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_same_account_role_policy_allow PASSED [ 7%] 2026-02-24T22:25:05.075 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_cross_account_role_policy_allow PASSED [ 7%] 2026-02-24T22:25:06.387 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_role_policy_allow_create_bucket PASSED [ 7%] 2026-02-24T22:25:07.703 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_role_policy_allow_get_role PASSED [ 8%] 2026-02-24T22:25:08.659 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_account_oidc_provider PASSED [ 8%] 2026-02-24T22:25:09.503 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_verify_add_new_client_id_to_oidc PASSED [ 8%] 2026-02-24T22:25:10.335 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_verify_add_existing_client_id_to_oidc PASSED [ 8%] 2026-02-24T22:25:11.167 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_verify_remove_client_id_from_oidc PASSED [ 8%] 2026-02-24T22:25:12.015 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_verify_update_thumbprintlist_of_oidc PASSED [ 8%] 2026-02-24T22:25:13.195 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_cross_account_bucket_user_policy_allow_user_arn PASSED [ 8%] 2026-02-24T22:25:14.411 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_cross_account_user_bucket_policy_allow_user_arn PASSED [ 8%] 2026-02-24T22:25:15.659 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_cross_account_user_bucket_policy_allow_account_arn PASSED [ 8%] 2026-02-24T22:25:16.803 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_cross_account_bucket_user_policy_allow_account_arn PASSED [ 8%] 2026-02-24T22:25:18.003 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_cross_account_user_bucket_policy_allow_account_id PASSED [ 8%] 2026-02-24T22:25:19.167 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_cross_account_bucket_user_policy_allow_account_id PASSED [ 9%] 2026-02-24T22:25:20.355 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_cross_account_bucket_acl_user_policy_grant_user_id PASSED [ 9%] 2026-02-24T22:25:21.599 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_cross_account_user_policy_bucket_acl_grant_user_id PASSED [ 9%] 2026-02-24T22:25:22.799 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_cross_account_bucket_acl_user_policy_grant_canonical_id PASSED [ 9%] 2026-02-24T22:25:24.019 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_cross_account_user_policy_bucket_acl_grant_canonical_id PASSED [ 9%] 2026-02-24T22:25:25.239 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_cross_account_bucket_acl_user_policy_grant_account_email PASSED [ 9%] 2026-02-24T22:25:26.483 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_cross_account_user_policy_bucket_acl_grant_account_email PASSED [ 9%] 2026-02-24T22:25:27.331 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_cross_account_root_bucket_policy_allow_account_arn PASSED [ 9%] 2026-02-24T22:25:28.203 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_cross_account_root_bucket_policy_allow_account_id PASSED [ 9%] 2026-02-24T22:25:29.011 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_cross_account_root_bucket_acl_grant_canonical_id PASSED [ 9%] 2026-02-24T22:25:29.835 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_cross_account_root_bucket_acl_grant_account_email PASSED [ 9%] 2026-02-24T22:25:30.531 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_get_account_summary_root PASSED [ 10%] 2026-02-24T22:25:31.555 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py::test_get_account_summary_policy PASSED [ 10%] 2026-02-24T22:25:32.026 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_empty PASSED [ 10%] 2026-02-24T22:25:32.643 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_distinct PASSED [ 10%] 2026-02-24T22:25:33.282 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_many PASSED [ 10%] 2026-02-24T22:25:33.894 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_many PASSED [ 10%] 2026-02-24T22:25:34.459 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_basic_key_count PASSED [ 10%] 2026-02-24T22:25:35.090 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_delimiter_basic PASSED [ 10%] 2026-02-24T22:25:35.619 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_delimiter_basic PASSED [ 10%] 2026-02-24T22:25:36.186 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_encoding_basic PASSED [ 10%] 2026-02-24T22:25:36.795 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_encoding_basic PASSED [ 10%] 2026-02-24T22:25:37.719 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_delimiter_prefix PASSED [ 10%] 2026-02-24T22:25:38.682 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_delimiter_prefix PASSED [ 11%] 2026-02-24T22:25:39.231 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_delimiter_prefix_ends_with_delimiter PASSED [ 11%] 2026-02-24T22:25:39.782 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_delimiter_prefix_ends_with_delimiter PASSED [ 11%] 2026-02-24T22:25:40.370 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_delimiter_alt PASSED [ 11%] 2026-02-24T22:25:40.994 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_delimiter_alt PASSED [ 11%] 2026-02-24T22:25:41.902 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_delimiter_prefix_underscore PASSED [ 11%] 2026-02-24T22:25:42.862 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_delimiter_prefix_underscore PASSED [ 11%] 2026-02-24T22:25:43.455 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_delimiter_percentage PASSED [ 11%] 2026-02-24T22:25:44.087 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_delimiter_percentage PASSED [ 11%] 2026-02-24T22:25:44.618 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_delimiter_whitespace PASSED [ 11%] 2026-02-24T22:25:45.219 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_delimiter_whitespace PASSED [ 11%] 2026-02-24T22:25:45.758 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_delimiter_dot PASSED [ 12%] 2026-02-24T22:25:46.338 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_delimiter_dot PASSED [ 12%] 2026-02-24T22:25:46.930 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_delimiter_unreadable PASSED [ 12%] 2026-02-24T22:25:47.530 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_delimiter_unreadable PASSED [ 12%] 2026-02-24T22:25:48.134 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_delimiter_empty PASSED [ 12%] 2026-02-24T22:25:48.703 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_delimiter_empty PASSED [ 12%] 2026-02-24T22:25:49.290 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_delimiter_none PASSED [ 12%] 2026-02-24T22:25:49.890 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_delimiter_none PASSED [ 12%] 2026-02-24T22:25:50.451 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_fetchowner_notempty PASSED [ 12%] 2026-02-24T22:25:51.026 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_fetchowner_defaultempty PASSED [ 12%] 2026-02-24T22:25:51.606 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_fetchowner_empty PASSED [ 12%] 2026-02-24T22:25:52.142 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_delimiter_not_exist PASSED [ 13%] 2026-02-24T22:25:52.718 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_delimiter_not_exist PASSED [ 13%] 2026-02-24T22:26:00.438 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_delimiter_not_skip_special PASSED [ 13%] 2026-02-24T22:26:01.006 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_prefix_basic PASSED [ 13%] 2026-02-24T22:26:01.562 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_prefix_basic PASSED [ 13%] 2026-02-24T22:26:02.126 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_prefix_alt PASSED [ 13%] 2026-02-24T22:26:02.726 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_prefix_alt PASSED [ 13%] 2026-02-24T22:26:03.342 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_prefix_empty PASSED [ 13%] 2026-02-24T22:26:03.906 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_prefix_empty PASSED [ 13%] 2026-02-24T22:26:04.474 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_prefix_none PASSED [ 13%] 2026-02-24T22:26:05.090 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_prefix_none PASSED [ 13%] 2026-02-24T22:26:05.614 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_prefix_not_exist PASSED [ 14%] 2026-02-24T22:26:06.138 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_prefix_not_exist PASSED [ 14%] 2026-02-24T22:26:06.734 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_prefix_unreadable PASSED [ 14%] 2026-02-24T22:26:07.290 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_prefix_unreadable PASSED [ 14%] 2026-02-24T22:26:07.870 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_prefix_delimiter_basic PASSED [ 14%] 2026-02-24T22:26:08.462 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_basic PASSED [ 14%] 2026-02-24T22:26:09.030 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_prefix_delimiter_alt PASSED [ 14%] 2026-02-24T22:26:09.614 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_alt PASSED [ 14%] 2026-02-24T22:26:10.234 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_prefix_delimiter_prefix_not_exist PASSED [ 14%] 2026-02-24T22:26:10.806 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_prefix_not_exist PASSED [ 14%] 2026-02-24T22:26:11.378 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_prefix_delimiter_delimiter_not_exist PASSED [ 14%] 2026-02-24T22:26:11.998 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_delimiter_not_exist PASSED [ 14%] 2026-02-24T22:26:12.558 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_prefix_delimiter_prefix_delimiter_not_exist PASSED [ 15%] 2026-02-24T22:26:13.150 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_prefix_delimiter_not_exist PASSED [ 15%] 2026-02-24T22:26:13.794 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_maxkeys_one PASSED [ 15%] 2026-02-24T22:26:14.426 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_maxkeys_one PASSED [ 15%] 2026-02-24T22:26:15.022 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_maxkeys_zero PASSED [ 15%] 2026-02-24T22:26:15.622 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_maxkeys_zero PASSED [ 15%] 2026-02-24T22:26:16.202 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_maxkeys_none PASSED [ 15%] 2026-02-24T22:26:16.790 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_maxkeys_none PASSED [ 15%] 2026-02-24T22:26:17.201 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_account_usage PASSED [ 15%] 2026-02-24T22:26:17.794 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_head_bucket_usage PASSED [ 15%] 2026-02-24T22:26:18.714 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_unordered PASSED [ 15%] 2026-02-24T22:26:19.658 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_unordered PASSED [ 16%] 2026-02-24T22:26:20.222 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_maxkeys_invalid PASSED [ 16%] 2026-02-24T22:26:20.754 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_marker_none PASSED [ 16%] 2026-02-24T22:26:21.314 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_marker_empty PASSED [ 16%] 2026-02-24T22:26:21.870 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_continuationtoken_empty PASSED [ 16%] 2026-02-24T22:26:22.482 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_continuationtoken PASSED [ 16%] 2026-02-24T22:26:23.098 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_both_continuationtoken_startafter PASSED [ 16%] 2026-02-24T22:26:23.666 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_marker_unreadable PASSED [ 16%] 2026-02-24T22:26:24.261 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_startafter_unreadable PASSED [ 16%] 2026-02-24T22:26:24.854 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_marker_not_in_list PASSED [ 16%] 2026-02-24T22:26:25.462 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_startafter_not_in_list PASSED [ 16%] 2026-02-24T22:26:26.034 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_marker_after_list PASSED [ 17%] 2026-02-24T22:26:26.626 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_startafter_after_list PASSED [ 17%] 2026-02-24T22:26:27.370 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_return_data PASSED [ 17%] 2026-02-24T22:26:28.282 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_return_data_versioning PASSED [ 17%] 2026-02-24T22:26:28.822 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_objects_anonymous PASSED [ 17%] 2026-02-24T22:26:29.406 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_objects_anonymous PASSED [ 17%] 2026-02-24T22:26:29.898 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_objects_anonymous_fail PASSED [ 17%] 2026-02-24T22:26:30.358 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_listv2_objects_anonymous_fail PASSED [ 17%] 2026-02-24T22:26:30.778 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_notexist PASSED [ 17%] 2026-02-24T22:26:31.186 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucketv2_notexist PASSED [ 17%] 2026-02-24T22:26:31.573 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_delete_notexist PASSED [ 17%] 2026-02-24T22:26:32.122 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_delete_nonempty PASSED [ 17%] 2026-02-24T22:26:32.670 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_concurrent_set_canned_acl PASSED [ 18%] 2026-02-24T22:26:33.074 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_write_to_nonexist_bucket PASSED [ 18%] 2026-02-24T22:26:33.598 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_write_with_chunked_transfer_encoding PASSED [ 18%] 2026-02-24T22:26:34.142 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_create_delete PASSED [ 18%] 2026-02-24T22:26:34.634 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_read_not_exist PASSED [ 18%] 2026-02-24T22:26:35.102 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_requestid_matches_header_on_error PASSED [ 18%] 2026-02-24T22:26:36.234 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_versioning_concurrent_multi_object_delete PASSED [ 18%] 2026-02-24T22:26:36.930 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multi_object_delete PASSED [ 18%] 2026-02-24T22:26:37.569 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_expected_bucket_owner PASSED [ 18%] 2026-02-24T22:26:38.321 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multi_objectv2_delete PASSED [ 18%] 2026-02-24T22:26:46.306 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multi_object_delete_key_limit PASSED [ 18%] 2026-02-24T22:26:54.702 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multi_objectv2_delete_key_limit PASSED [ 19%] 2026-02-24T22:26:55.254 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_head_zero_bytes PASSED [ 19%] 2026-02-24T22:26:55.813 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_write_check_etag PASSED [ 19%] 2026-02-24T22:26:56.334 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_write_cache_control PASSED [ 19%] 2026-02-24T22:26:56.885 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_write_expires PASSED [ 19%] 2026-02-24T22:26:57.505 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_write_read_update_read_delete PASSED [ 19%] 2026-02-24T22:26:58.073 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_set_get_metadata_none_to_good PASSED [ 19%] 2026-02-24T22:26:58.677 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_set_get_metadata_none_to_empty PASSED [ 19%] 2026-02-24T22:26:59.265 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_set_get_metadata_overwrite_to_empty PASSED [ 19%] 2026-02-24T22:26:59.817 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_metadata_replaced_on_put PASSED [ 19%] 2026-02-24T22:27:00.370 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_write_file PASSED [ 19%] 2026-02-24T22:27:00.965 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_anonymous_request PASSED [ 20%] 2026-02-24T22:27:01.565 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_authenticated_request PASSED [ 20%] 2026-02-24T22:27:02.105 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_authenticated_no_content_type PASSED [ 20%] 2026-02-24T22:27:02.637 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_authenticated_request_bad_access_key PASSED [ 20%] 2026-02-24T22:27:03.185 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_set_success_code PASSED [ 20%] 2026-02-24T22:27:03.717 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_set_invalid_success_code PASSED [ 20%] 2026-02-24T22:27:04.297 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_upload_larger_than_chunk PASSED [ 20%] 2026-02-24T22:27:04.941 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_set_key_from_filename PASSED [ 20%] 2026-02-24T22:27:05.477 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_ignored_header PASSED [ 20%] 2026-02-24T22:27:06.041 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_case_insensitive_condition_fields PASSED [ 20%] 2026-02-24T22:27:06.629 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_escaped_field_values PASSED [ 20%] 2026-02-24T22:27:07.209 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_success_redirect_action PASSED [ 21%] 2026-02-24T22:27:07.718 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_invalid_signature PASSED [ 21%] 2026-02-24T22:27:08.221 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_invalid_access_key PASSED [ 21%] 2026-02-24T22:27:08.781 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_invalid_date_format PASSED [ 21%] 2026-02-24T22:27:09.305 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_no_key_specified PASSED [ 21%] 2026-02-24T22:27:09.813 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_missing_signature PASSED [ 21%] 2026-02-24T22:27:10.337 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_missing_policy_condition PASSED [ 21%] 2026-02-24T22:27:10.981 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_user_specified_header PASSED [ 21%] 2026-02-24T22:27:11.477 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_request_missing_policy_specified_field PASSED [ 21%] 2026-02-24T22:27:11.973 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_condition_is_case_sensitive PASSED [ 21%] 2026-02-24T22:27:12.489 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_expires_is_case_sensitive PASSED [ 21%] 2026-02-24T22:27:13.021 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_expired_policy PASSED [ 21%] 2026-02-24T22:27:13.609 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_wrong_bucket PASSED [ 22%] 2026-02-24T22:27:14.129 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_invalid_request_field_value PASSED [ 22%] 2026-02-24T22:27:14.653 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_missing_expires_condition PASSED [ 22%] 2026-02-24T22:27:15.137 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_missing_conditions_list PASSED [ 22%] 2026-02-24T22:27:15.661 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_upload_size_limit_exceeded PASSED [ 22%] 2026-02-24T22:27:16.153 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_missing_content_length_argument PASSED [ 22%] 2026-02-24T22:27:16.681 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_invalid_content_length_argument PASSED [ 22%] 2026-02-24T22:27:17.201 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_upload_size_below_minimum PASSED [ 22%] 2026-02-24T22:27:17.737 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_upload_size_rgw_chunk_size_bug PASSED [ 22%] 2026-02-24T22:27:18.265 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_empty_conditions PASSED [ 22%] 2026-02-24T22:27:18.885 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_object_ifmatch_good PASSED [ 22%] 2026-02-24T22:27:19.493 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_object_ifmatch_failed PASSED [ 23%] 2026-02-24T22:27:19.993 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_object_ifnonematch_good PASSED [ 23%] 2026-02-24T22:27:20.573 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_object_ifnonematch_failed PASSED [ 23%] 2026-02-24T22:27:21.121 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_object_ifmodifiedsince_good PASSED [ 23%] 2026-02-24T22:27:22.725 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_object_ifmodifiedsince_failed PASSED [ 23%] 2026-02-24T22:27:23.317 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_object_ifunmodifiedsince_good PASSED [ 23%] 2026-02-24T22:27:23.929 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_object_ifunmodifiedsince_failed PASSED [ 23%] 2026-02-24T22:27:24.529 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_object_ifmatch_good PASSED [ 23%] 2026-02-24T22:27:25.169 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_object_ifmatch_failed PASSED [ 23%] 2026-02-24T22:27:25.845 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_object_ifmatch_overwrite_existed_good PASSED [ 23%] 2026-02-24T22:27:26.389 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_object_ifmatch_nonexisted_failed PASSED [ 23%] 2026-02-24T22:27:26.993 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_object_ifnonmatch_good PASSED [ 24%] 2026-02-24T22:27:27.689 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_object_ifnonmatch_failed PASSED [ 24%] 2026-02-24T22:27:28.297 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_object_ifnonmatch_nonexisted_good PASSED [ 24%] 2026-02-24T22:27:28.929 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_object_ifnonmatch_overwrite_existed_failed PASSED [ 24%] 2026-02-24T22:27:29.473 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_raw_get PASSED [ 24%] 2026-02-24T22:27:30.025 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_raw_get_bucket_gone PASSED [ 24%] 2026-02-24T22:27:30.537 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_delete_key_bucket_gone PASSED [ 24%] 2026-02-24T22:27:31.125 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_raw_get_object_gone PASSED [ 24%] 2026-02-24T22:27:31.649 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_head PASSED [ 24%] 2026-02-24T22:27:32.056 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_head_notexist PASSED [ 24%] 2026-02-24T22:27:32.669 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_head_extended PASSED [ 24%] 2026-02-24T22:27:33.249 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_raw_get_bucket_acl PASSED [ 25%] 2026-02-24T22:27:33.801 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_raw_get_object_acl PASSED [ 25%] 2026-02-24T22:27:36.629 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_put_acl_mtime PASSED [ 25%] 2026-02-24T22:27:37.169 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_raw_authenticated PASSED [ 25%] 2026-02-24T22:27:37.737 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_raw_response_headers PASSED [ 25%] 2026-02-24T22:27:38.289 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_raw_authenticated_bucket_acl PASSED [ 25%] 2026-02-24T22:27:38.801 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_raw_authenticated_object_acl PASSED [ 25%] 2026-02-24T22:27:39.368 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_raw_authenticated_bucket_gone PASSED [ 25%] 2026-02-24T22:27:39.933 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_raw_authenticated_object_gone PASSED [ 25%] 2026-02-24T22:27:40.537 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired PASSED [ 25%] 2026-02-24T22:27:41.121 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired_tenant PASSED [ 25%] 2026-02-24T22:27:41.701 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_range_zero PASSED [ 25%] 2026-02-24T22:27:42.265 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_max_range PASSED [ 26%] 2026-02-24T22:27:42.813 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_positive_range PASSED [ 26%] 2026-02-24T22:27:43.401 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_content_encoding_aws_chunked PASSED [ 26%] 2026-02-24T22:27:43.969 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_anon_put PASSED [ 26%] 2026-02-24T22:27:44.549 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_anon_put_write_access PASSED [ 26%] 2026-02-24T22:27:45.133 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_put_authenticated PASSED [ 26%] 2026-02-24T22:27:45.717 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_presigned_put_object_with_acl PASSED [ 26%] 2026-02-24T22:27:46.284 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_presigned_put_object_with_acl_tenant PASSED [ 26%] 2026-02-24T22:27:46.809 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_raw_put_authenticated_expired PASSED [ 26%] 2026-02-24T22:27:47.261 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_create_naming_bad_starts_nonalpha PASSED [ 26%] 2026-02-24T22:27:47.660 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_create_naming_bad_short_one PASSED [ 26%] 2026-02-24T22:27:48.064 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_create_naming_bad_short_two PASSED [ 27%] 2026-02-24T22:27:48.525 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_create_naming_good_long_60 PASSED [ 27%] 2026-02-24T22:27:49.020 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_create_naming_good_long_61 PASSED [ 27%] 2026-02-24T22:27:49.468 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_create_naming_good_long_62 PASSED [ 27%] 2026-02-24T22:27:49.913 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_create_naming_good_long_63 PASSED [ 27%] 2026-02-24T22:27:50.420 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_long_name PASSED [ 27%] 2026-02-24T22:27:50.844 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_create_naming_bad_ip PASSED [ 27%] 2026-02-24T22:27:51.240 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_create_naming_dns_underscore PASSED [ 27%] 2026-02-24T22:27:51.685 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_create_naming_dns_long PASSED [ 27%] 2026-02-24T22:27:52.092 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_create_naming_dns_dash_at_end PASSED [ 27%] 2026-02-24T22:27:52.528 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_create_naming_dns_dot_dot PASSED [ 27%] 2026-02-24T22:27:52.928 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_create_naming_dns_dot_dash PASSED [ 28%] 2026-02-24T22:27:53.320 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_create_naming_dns_dash_dot PASSED [ 28%] 2026-02-24T22:27:53.772 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_create_exists PASSED [ 28%] 2026-02-24T22:27:54.293 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_get_location PASSED [ 28%] 2026-02-24T22:27:54.788 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_create_exists_nonowner PASSED [ 28%] 2026-02-24T22:27:55.304 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_recreate_overwrite_acl PASSED [ 28%] 2026-02-24T22:27:55.797 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_recreate_new_acl PASSED [ 28%] 2026-02-24T22:27:56.324 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_acl_default PASSED [ 28%] 2026-02-24T22:27:56.824 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_acl_canned_during_create PASSED [ 28%] 2026-02-24T22:27:57.360 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_acl_canned PASSED [ 28%] 2026-02-24T22:27:57.812 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_acl_canned_publicreadwrite PASSED [ 28%] 2026-02-24T22:27:58.341 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_acl_canned_authenticatedread PASSED [ 28%] 2026-02-24T22:27:58.892 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_bucket_acl_grant_group_read PASSED [ 29%] 2026-02-24T22:27:59.505 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_acl_default PASSED [ 29%] 2026-02-24T22:28:00.088 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_acl_canned_during_create PASSED [ 29%] 2026-02-24T22:28:00.700 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_acl_canned PASSED [ 29%] 2026-02-24T22:28:01.292 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_acl_canned_publicreadwrite PASSED [ 29%] 2026-02-24T22:28:01.896 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_acl_canned_authenticatedread PASSED [ 29%] 2026-02-24T22:28:02.552 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_acl_canned_bucketownerread PASSED [ 29%] 2026-02-24T22:28:03.176 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_acl_canned_bucketownerfullcontrol PASSED [ 29%] 2026-02-24T22:28:03.796 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_acl_full_control_verify_owner PASSED [ 29%] 2026-02-24T22:28:04.400 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_acl_full_control_verify_attributes PASSED [ 29%] 2026-02-24T22:28:04.820 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_acl_canned_private_to_private PASSED [ 29%] 2026-02-24T22:28:05.412 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_acl PASSED [ 30%] 2026-02-24T22:28:06.068 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_acl_write PASSED [ 30%] 2026-02-24T22:28:06.701 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_acl_writeacp PASSED [ 30%] 2026-02-24T22:28:07.336 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_acl_read PASSED [ 30%] 2026-02-24T22:28:07.972 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_acl_readacp PASSED [ 30%] 2026-02-24T22:28:08.828 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_acl_grant_userid_fullcontrol PASSED [ 30%] 2026-02-24T22:28:09.544 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_acl_grant_userid_read PASSED [ 30%] 2026-02-24T22:28:10.388 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_acl_grant_userid_readacp PASSED [ 30%] 2026-02-24T22:28:11.224 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_acl_grant_userid_write PASSED [ 30%] 2026-02-24T22:28:11.972 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_acl_grant_userid_writeacp PASSED [ 30%] 2026-02-24T22:28:12.536 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_acl_grant_nonexist_user PASSED [ 30%] 2026-02-24T22:28:13.096 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_header_acl_grants PASSED [ 31%] 2026-02-24T22:28:13.648 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_header_acl_grants PASSED [ 31%] 2026-02-24T22:28:14.276 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_acl_grant_email PASSED [ 31%] 2026-02-24T22:28:14.848 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_acl_grant_email_not_exist PASSED [ 31%] 2026-02-24T22:28:15.484 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_acl_revoke_all PASSED [ 31%] 2026-02-24T22:28:16.380 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_access_bucket_private_object_private PASSED [ 31%] 2026-02-24T22:28:17.205 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_access_bucket_private_objectv2_private PASSED [ 31%] 2026-02-24T22:28:18.084 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_access_bucket_private_object_publicread PASSED [ 31%] 2026-02-24T22:28:18.944 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_access_bucket_private_objectv2_publicread PASSED [ 31%] 2026-02-24T22:28:19.744 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_access_bucket_private_object_publicreadwrite PASSED [ 31%] 2026-02-24T22:28:20.624 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_access_bucket_private_objectv2_publicreadwrite PASSED [ 31%] 2026-02-24T22:28:21.484 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_access_bucket_publicread_object_private PASSED [ 32%] 2026-02-24T22:28:22.364 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_access_bucket_publicread_object_publicread PASSED [ 32%] 2026-02-24T22:28:23.160 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_access_bucket_publicread_object_publicreadwrite PASSED [ 32%] 2026-02-24T22:28:23.912 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_access_bucket_publicreadwrite_object_private PASSED [ 32%] 2026-02-24T22:28:24.608 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicread PASSED [ 32%] 2026-02-24T22:28:25.328 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicreadwrite PASSED [ 32%] 2026-02-24T22:28:26.120 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_buckets_create_then_list PASSED [ 32%] 2026-02-24T22:28:26.864 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_buckets_list_ctime PASSED [ 32%] 2026-02-24T22:28:27.276 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_list_buckets_anonymous PASSED [ 32%] 2026-02-24T22:28:27.692 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_list_buckets_invalid_auth PASSED [ 32%] 2026-02-24T22:28:28.088 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_list_buckets_bad_auth PASSED [ 32%] 2026-02-24T22:28:28.796 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_list_buckets_paginated PASSED [ 32%] 2026-02-24T22:28:29.340 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_create_naming_good_starts_alpha PASSED [ 33%] 2026-02-24T22:28:29.888 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_create_naming_good_starts_digit PASSED [ 33%] 2026-02-24T22:28:30.400 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_create_naming_good_contains_period PASSED [ 33%] 2026-02-24T22:28:30.848 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_create_naming_good_contains_hyphen PASSED [ 33%] 2026-02-24T22:28:31.488 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_recreate_not_overriding PASSED [ 33%] 2026-02-24T22:28:32.728 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_create_special_key_names PASSED [ 33%] 2026-02-24T22:28:33.376 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_list_special_prefix PASSED [ 33%] 2026-02-24T22:28:33.912 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_copy_zero_size PASSED [ 33%] 2026-02-24T22:28:34.660 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_copy_16m PASSED [ 33%] 2026-02-24T22:28:35.304 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_copy_same_bucket PASSED [ 33%] 2026-02-24T22:28:35.940 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_copy_verify_contenttype PASSED [ 33%] 2026-02-24T22:28:36.464 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_copy_to_itself PASSED [ 34%] 2026-02-24T22:28:37.052 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_copy_to_itself_with_metadata PASSED [ 34%] 2026-02-24T22:28:37.864 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_copy_diff_bucket PASSED [ 34%] 2026-02-24T22:28:38.476 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_copy_not_owned_bucket PASSED [ 34%] 2026-02-24T22:28:39.272 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_copy_not_owned_object_bucket PASSED [ 34%] 2026-02-24T22:28:39.940 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_copy_canned_acl PASSED [ 34%] 2026-02-24T22:28:40.832 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_copy_retaining_metadata PASSED [ 34%] 2026-02-24T22:28:41.652 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_copy_replacing_metadata PASSED [ 34%] 2026-02-24T22:28:42.152 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_copy_bucket_not_found PASSED [ 34%] 2026-02-24T22:28:42.612 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_copy_key_not_found PASSED [ 34%] 2026-02-24T22:28:44.068 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_copy_versioned_bucket PASSED [ 34%] 2026-02-24T22:28:44.696 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_copy_versioned_url_encoding PASSED [ 35%] 2026-02-24T22:28:46.732 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_copy_versioning_multipart_upload PASSED [ 35%] 2026-02-24T22:28:47.248 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_upload_empty PASSED [ 35%] 2026-02-24T22:28:47.900 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_upload_small PASSED [ 35%] 2026-02-24T22:28:48.976 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_copy_small PASSED [ 35%] 2026-02-24T22:28:49.572 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_copy_invalid_range PASSED [ 35%] 2026-02-24T22:28:50.512 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_copy_without_range PASSED [ 35%] 2026-02-24T22:28:52.684 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_copy_special_names PASSED [ 35%] 2026-02-24T22:28:55.232 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_upload PASSED [ 35%] 2026-02-24T22:28:58.144 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_copy_versioned PASSED [ 35%] 2026-02-24T22:29:09.852 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_upload_resend_part PASSED [ 35%] 2026-02-24T22:29:11.431 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_upload_multiple_sizes PASSED [ 35%] 2026-02-24T22:29:14.232 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_copy_multiple_sizes PASSED [ 36%] 2026-02-24T22:29:14.919 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_upload_size_too_small PASSED [ 36%] 2026-02-24T22:29:15.803 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_upload_contents PASSED [ 36%] 2026-02-24T22:29:16.623 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_upload_overwrite_existing_object PASSED [ 36%] 2026-02-24T22:29:17.299 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_abort_multipart_upload PASSED [ 36%] 2026-02-24T22:29:17.903 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_abort_multipart_upload_not_found PASSED [ 36%] 2026-02-24T22:29:18.815 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_list_multipart_upload PASSED [ 36%] 2026-02-24T22:29:19.483 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_list_multipart_upload_owner PASSED [ 36%] 2026-02-24T22:29:20.071 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_upload_missing_part PASSED [ 36%] 2026-02-24T22:29:20.640 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_upload_incorrect_etag PASSED [ 36%] 2026-02-24T22:29:30.915 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_get_part PASSED [ 36%] 2026-02-24T22:29:41.379 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_sse_c_get_part PASSED [ 37%] 2026-02-24T22:29:43.115 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_single_get_part PASSED [ 37%] 2026-02-24T22:29:43.727 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_non_multipart_get_part PASSED [ 37%] 2026-02-24T22:29:44.423 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_non_multipart_sse_c_get_part PASSED [ 37%] 2026-02-24T22:29:44.888 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_100_continue PASSED [ 37%] 2026-02-24T22:29:45.479 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_set_cors PASSED [ 37%] 2026-02-24T22:29:50.471 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_cors_origin_response PASSED [ 37%] 2026-02-24T22:29:54.107 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_cors_origin_wildcard PASSED [ 37%] 2026-02-24T22:29:57.655 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_cors_header_option PASSED [ 37%] 2026-02-24T22:29:58.247 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_cors_presigned_get_object PASSED [ 37%] 2026-02-24T22:29:58.831 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_cors_presigned_get_object_tenant PASSED [ 37%] 2026-02-24T22:29:59.423 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_cors_presigned_put_object PASSED [ 38%] 2026-02-24T22:29:59.990 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_cors_presigned_put_object_with_acl PASSED [ 38%] 2026-02-24T22:30:00.559 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_cors_presigned_put_object_tenant PASSED [ 38%] 2026-02-24T22:30:01.115 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_cors_presigned_put_object_tenant_with_acl PASSED [ 38%] 2026-02-24T22:30:01.758 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_set_bucket_tagging PASSED [ 38%] 2026-02-24T22:30:02.415 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_atomic_read_1mb PASSED [ 38%] 2026-02-24T22:30:03.199 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_atomic_read_4mb PASSED [ 38%] 2026-02-24T22:30:04.050 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_atomic_read_8mb PASSED [ 38%] 2026-02-24T22:30:04.799 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_atomic_write_1mb PASSED [ 38%] 2026-02-24T22:30:05.607 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_atomic_write_4mb PASSED [ 38%] 2026-02-24T22:30:06.691 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_atomic_write_8mb PASSED [ 38%] 2026-02-24T22:30:07.291 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_atomic_dual_write_1mb PASSED [ 39%] 2026-02-24T22:30:08.023 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_atomic_dual_write_4mb PASSED [ 39%] 2026-02-24T22:30:08.827 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_atomic_dual_write_8mb PASSED [ 39%] 2026-02-24T22:30:09.519 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_atomic_conditional_write_1mb PASSED [ 39%] 2026-02-24T22:30:10.183 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_atomic_multipart_upload_write PASSED [ 39%] 2026-02-24T22:30:11.031 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_resend_first_finishes_last PASSED [ 39%] 2026-02-24T22:30:11.582 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_ranged_request_response_code PASSED [ 39%] 2026-02-24T22:30:14.231 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_ranged_big_request_response_code PASSED [ 39%] 2026-02-24T22:30:14.838 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_ranged_request_skip_leading_bytes_response_code PASSED [ 39%] 2026-02-24T22:30:15.454 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_ranged_request_return_trailing_bytes_response_code PASSED [ 39%] 2026-02-24T22:30:16.087 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_ranged_request_invalid_range PASSED [ 39%] 2026-02-24T22:30:16.799 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_ranged_request_empty_object PASSED [ 39%] 2026-02-24T22:30:17.587 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_versioning_bucket_create_suspend PASSED [ 40%] 2026-02-24T22:30:23.899 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_versioning_obj_create_read_remove PASSED [ 40%] 2026-02-24T22:30:24.546 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_versioning_obj_create_read_remove_head PASSED [ 40%] 2026-02-24T22:30:25.166 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_versioning_stack_delete_merkers PASSED [ 40%] 2026-02-24T22:30:25.891 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_versioning_obj_plain_null_version_removal PASSED [ 40%] 2026-02-24T22:30:26.730 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite PASSED [ 40%] 2026-02-24T22:30:27.586 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite_suspended PASSED [ 40%] 2026-02-24T22:30:30.546 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_versioning_obj_suspend_versions PASSED [ 40%] 2026-02-24T22:30:31.378 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_versioning_obj_suspended_copy PASSED [ 40%] 2026-02-24T22:30:34.946 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_versioning_obj_create_versions_remove_all PASSED [ 40%] 2026-02-24T22:30:47.638 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_versioning_obj_create_versions_remove_special_names PASSED [ 40%] 2026-02-24T22:30:49.994 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_versioning_obj_create_overwrite_multipart PASSED [ 41%] 2026-02-24T22:30:51.194 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_versioning_obj_list_marker PASSED [ 41%] 2026-02-24T22:30:52.670 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_versioning_copy_obj_version PASSED [ 41%] 2026-02-24T22:30:53.450 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_versioning_multi_object_delete PASSED [ 41%] 2026-02-24T22:30:54.234 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_versioning_multi_object_delete_with_marker PASSED [ 41%] 2026-02-24T22:30:54.966 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_versioning_multi_object_delete_with_marker_create PASSED [ 41%] 2026-02-24T22:30:55.774 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_versioned_object_acl PASSED [ 41%] 2026-02-24T22:30:56.546 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_versioned_object_acl_no_version_specified PASSED [ 41%] 2026-02-24T22:30:58.070 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_versioned_concurrent_object_create_concurrent_remove PASSED [ 41%] 2026-02-24T22:30:59.106 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_versioned_concurrent_object_create_and_remove FAILED [ 41%] 2026-02-24T22:30:59.106 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_versioned_concurrent_object_create_and_remove ERROR [ 41%] 2026-02-24T22:30:59.335 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_lifecycle_set ERROR [ 41%] 2026-02-24T22:30:59.598 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_lifecycle_get ERROR [ 42%] 2026-02-24T22:30:59.828 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_lifecycle_delete ERROR [ 42%] 2026-02-24T22:31:00.057 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_lifecycle_get_no_id ERROR [ 42%] 2026-02-24T22:31:00.321 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_lifecycle_id_too_long ERROR [ 42%] 2026-02-24T22:31:00.548 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_lifecycle_same_id ERROR [ 42%] 2026-02-24T22:31:00.777 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_lifecycle_invalid_status ERROR [ 42%] 2026-02-24T22:31:01.003 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_lifecycle_set_date ERROR [ 42%] 2026-02-24T22:31:01.256 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_lifecycle_set_invalid_date ERROR [ 42%] 2026-02-24T22:31:01.489 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_lifecycle_set_noncurrent ERROR [ 42%] 2026-02-24T22:31:01.716 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_lifecycle_set_deletemarker ERROR [ 42%] 2026-02-24T22:31:01.943 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_lifecycle_set_filter ERROR [ 42%] 2026-02-24T22:31:02.210 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_lifecycle_set_empty_filter ERROR [ 42%] 2026-02-24T22:31:02.444 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_lifecycle_set_multipart ERROR [ 43%] 2026-02-24T22:31:02.672 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_lifecycle_transition_set_invalid_date ERROR [ 43%] 2026-02-24T22:31:02.933 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_restore_object_temporary ERROR [ 43%] 2026-02-24T22:31:03.159 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_restore_object_permanent ERROR [ 43%] 2026-02-24T22:31:03.389 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_read_through ERROR [ 43%] 2026-02-24T22:31:03.616 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_restore_noncur_obj ERROR [ 43%] 2026-02-24T22:31:03.881 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_encrypted_transfer_1b ERROR [ 43%] 2026-02-24T22:31:04.108 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_encrypted_transfer_1kb ERROR [ 43%] 2026-02-24T22:31:04.337 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_encrypted_transfer_1MB ERROR [ 43%] 2026-02-24T22:31:04.599 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_encrypted_transfer_13b ERROR [ 43%] 2026-02-24T22:31:04.833 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_encryption_sse_c_method_head ERROR [ 43%] 2026-02-24T22:31:05.060 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_encryption_sse_c_present ERROR [ 44%] 2026-02-24T22:31:05.289 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_encryption_sse_c_other_key ERROR [ 44%] 2026-02-24T22:31:05.546 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_encryption_sse_c_invalid_md5 ERROR [ 44%] 2026-02-24T22:31:05.776 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_encryption_sse_c_no_md5 ERROR [ 44%] 2026-02-24T22:31:06.004 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_encryption_sse_c_no_key ERROR [ 44%] 2026-02-24T22:31:06.232 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_encryption_key_no_sse_c ERROR [ 44%] 2026-02-24T22:31:06.498 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_encryption_sse_c_multipart_upload ERROR [ 44%] 2026-02-24T22:31:06.724 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_encryption_sse_c_unaligned_multipart_upload ERROR [ 44%] 2026-02-24T22:31:06.953 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_encryption_sse_c_multipart_bad_download ERROR [ 44%] 2026-02-24T22:31:07.215 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_encryption_sse_c_post_object_authenticated_request ERROR [ 44%] 2026-02-24T22:31:07.448 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_encryption_sse_c_enforced_with_bucket_policy ERROR [ 44%] 2026-02-24T22:31:07.677 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_encryption_sse_c_deny_algo_with_bucket_policy ERROR [ 45%] 2026-02-24T22:31:07.905 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_sse_kms_method_head ERROR [ 45%] 2026-02-24T22:31:08.173 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_sse_kms_present ERROR [ 45%] 2026-02-24T22:31:08.400 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_sse_kms_no_key ERROR [ 45%] 2026-02-24T22:31:08.629 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_sse_kms_not_declared ERROR [ 45%] 2026-02-24T22:31:08.890 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_sse_kms_multipart_upload ERROR [ 45%] 2026-02-24T22:31:09.121 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_1 ERROR [ 45%] 2026-02-24T22:31:09.347 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_2 ERROR [ 45%] 2026-02-24T22:31:09.581 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_sse_kms_post_object_authenticated_request ERROR [ 45%] 2026-02-24T22:31:09.838 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_sse_kms_transfer_1b ERROR [ 45%] 2026-02-24T22:31:10.073 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_sse_kms_transfer_1kb ERROR [ 45%] 2026-02-24T22:31:10.301 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_sse_kms_transfer_1MB ERROR [ 46%] 2026-02-24T22:31:10.529 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_sse_kms_transfer_13b ERROR [ 46%] 2026-02-24T22:31:10.796 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_sse_kms_read_declare ERROR [ 46%] 2026-02-24T22:31:11.025 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_policy ERROR [ 46%] 2026-02-24T22:31:11.252 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucketv2_policy ERROR [ 46%] 2026-02-24T22:31:11.521 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_policy_deny_self_denied_policy ERROR [ 46%] 2026-02-24T22:31:11.748 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_policy_deny_self_denied_policy_confirm_header ERROR [ 46%] 2026-02-24T22:31:11.975 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_policy_acl ERROR [ 46%] 2026-02-24T22:31:12.205 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucketv2_policy_acl ERROR [ 46%] 2026-02-24T22:31:12.473 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_policy_different_tenant ERROR [ 46%] 2026-02-24T22:31:12.701 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_policy_multipart ERROR [ 46%] 2026-02-24T22:31:12.928 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_policy_tenanted_bucket ERROR [ 46%] 2026-02-24T22:31:13.197 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_policy_another_bucket ERROR [ 47%] 2026-02-24T22:31:13.425 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucketv2_policy_another_bucket ERROR [ 47%] 2026-02-24T22:31:13.653 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_set_get_del_bucket_policy ERROR [ 47%] 2026-02-24T22:31:13.879 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_set_multipart_tagging ERROR [ 47%] 2026-02-24T22:31:14.139 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_obj_tagging ERROR [ 47%] 2026-02-24T22:31:14.368 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_obj_head_tagging ERROR [ 47%] 2026-02-24T22:31:14.598 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_max_tags ERROR [ 47%] 2026-02-24T22:31:14.823 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_excess_tags ERROR [ 47%] 2026-02-24T22:31:15.092 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_max_kvsize_tags ERROR [ 47%] 2026-02-24T22:31:15.321 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_excess_key_tags ERROR [ 47%] 2026-02-24T22:31:15.549 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_excess_val_tags ERROR [ 47%] 2026-02-24T22:31:15.808 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_modify_tags ERROR [ 48%] 2026-02-24T22:31:16.036 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_delete_tags ERROR [ 48%] 2026-02-24T22:31:16.264 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_tags_anonymous_request ERROR [ 48%] 2026-02-24T22:31:16.491 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_tags_authenticated_request ERROR [ 48%] 2026-02-24T22:31:16.757 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_obj_with_tags ERROR [ 48%] 2026-02-24T22:31:16.985 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_tags_acl_public ERROR [ 48%] 2026-02-24T22:31:17.214 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_tags_acl_public ERROR [ 48%] 2026-02-24T22:31:17.477 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_delete_tags_obj_public ERROR [ 48%] 2026-02-24T22:31:17.706 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_versioning_bucket_atomic_upload_return_version_id ERROR [ 48%] 2026-02-24T22:31:17.932 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_versioning_bucket_multipart_upload_return_version_id ERROR [ 48%] 2026-02-24T22:31:18.160 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_policy_get_obj_existing_tag ERROR [ 48%] 2026-02-24T22:31:18.419 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_policy_get_obj_tagging_existing_tag ERROR [ 49%] 2026-02-24T22:31:18.650 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_policy_put_obj_tagging_existing_tag ERROR [ 49%] 2026-02-24T22:31:18.875 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_policy_upload_part_copy ERROR [ 49%] 2026-02-24T22:31:19.103 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_policy_put_obj_copy_source ERROR [ 49%] 2026-02-24T22:31:19.369 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_policy_put_obj_copy_source_meta ERROR [ 49%] 2026-02-24T22:31:19.596 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_policy_put_obj_acl ERROR [ 49%] 2026-02-24T22:31:19.828 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_policy_put_obj_grant ERROR [ 49%] 2026-02-24T22:31:20.087 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_obj_enc_conflict_c_s3 ERROR [ 49%] 2026-02-24T22:31:20.318 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_obj_enc_conflict_c_kms ERROR [ 49%] 2026-02-24T22:31:20.544 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_obj_enc_conflict_s3_kms ERROR [ 49%] 2026-02-24T22:31:20.772 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_obj_enc_conflict_bad_enc_kms ERROR [ 49%] 2026-02-24T22:31:21.035 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_policy_put_obj_kms_noenc ERROR [ 50%] 2026-02-24T22:31:21.261 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_policy_put_obj_kms_s3 ERROR [ 50%] 2026-02-24T22:31:21.488 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_policy_get_obj_acl_existing_tag ERROR [ 50%] 2026-02-24T22:31:21.753 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_put_obj_lock ERROR [ 50%] 2026-02-24T22:31:21.982 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_bucket ERROR [ 50%] 2026-02-24T22:31:22.210 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_put_obj_lock_enable_after_create ERROR [ 50%] 2026-02-24T22:31:22.439 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_put_obj_lock_with_days_and_years ERROR [ 50%] 2026-02-24T22:31:22.699 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_days ERROR [ 50%] 2026-02-24T22:31:22.923 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_years ERROR [ 50%] 2026-02-24T22:31:23.153 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_mode ERROR [ 50%] 2026-02-24T22:31:23.425 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_status ERROR [ 50%] 2026-02-24T22:31:23.652 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_suspend_versioning ERROR [ 50%] 2026-02-24T22:31:23.880 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_get_obj_lock ERROR [ 51%] 2026-02-24T22:31:24.105 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_get_obj_lock_invalid_bucket ERROR [ 51%] 2026-02-24T22:31:24.365 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_put_obj_retention ERROR [ 51%] 2026-02-24T22:31:24.596 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_bucket ERROR [ 51%] 2026-02-24T22:31:24.824 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_mode ERROR [ 51%] 2026-02-24T22:31:25.051 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_get_obj_retention ERROR [ 51%] 2026-02-24T22:31:25.322 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_get_obj_retention_iso8601 ERROR [ 51%] 2026-02-24T22:31:25.550 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_get_obj_retention_invalid_bucket ERROR [ 51%] 2026-02-24T22:31:25.778 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_put_obj_retention_versionid ERROR [ 51%] 2026-02-24T22:31:26.045 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_put_obj_retention_override_default_retention ERROR [ 51%] 2026-02-24T22:31:26.271 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_put_obj_retention_increase_period ERROR [ 51%] 2026-02-24T22:31:26.495 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period ERROR [ 52%] 2026-02-24T22:31:26.718 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period_bypass ERROR [ 52%] 2026-02-24T22:31:26.925 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_delete_object_with_retention ERROR [ 52%] 2026-02-24T22:31:27.154 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_delete_multipart_object_with_retention ERROR [ 52%] 2026-02-24T22:31:27.380 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_delete_object_with_retention_and_marker ERROR [ 52%] 2026-02-24T22:31:27.650 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_multi_delete_object_with_retention ERROR [ 52%] 2026-02-24T22:31:27.876 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_put_legal_hold ERROR [ 52%] 2026-02-24T22:31:28.105 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_bucket ERROR [ 52%] 2026-02-24T22:31:28.335 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_status ERROR [ 52%] 2026-02-24T22:31:28.592 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_get_legal_hold ERROR [ 52%] 2026-02-24T22:31:28.822 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_get_legal_hold_invalid_bucket ERROR [ 52%] 2026-02-24T22:31:29.048 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_on ERROR [ 53%] 2026-02-24T22:31:29.277 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_delete_multipart_object_with_legal_hold_on ERROR [ 53%] 2026-02-24T22:31:29.546 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_off ERROR [ 53%] 2026-02-24T22:31:29.773 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_get_obj_metadata ERROR [ 53%] 2026-02-24T22:31:30.002 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_uploading_obj ERROR [ 53%] 2026-02-24T22:31:30.272 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_changing_mode_from_governance_with_bypass ERROR [ 53%] 2026-02-24T22:31:30.501 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_changing_mode_from_governance_without_bypass ERROR [ 53%] 2026-02-24T22:31:30.733 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_lock_changing_mode_from_compliance ERROR [ 53%] 2026-02-24T22:31:30.959 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_object_ifmatch_good ERROR [ 53%] 2026-02-24T22:31:31.229 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_object_ifnonematch_failed ERROR [ 53%] 2026-02-24T22:31:31.456 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_bucket_policy_status ERROR [ 53%] 2026-02-24T22:31:31.686 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_public_acl_bucket_policy_status ERROR [ 53%] 2026-02-24T22:31:31.953 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_authpublic_acl_bucket_policy_status ERROR [ 54%] 2026-02-24T22:31:32.180 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_publicpolicy_acl_bucket_policy_status ERROR [ 54%] 2026-02-24T22:31:32.410 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_nonpublicpolicy_acl_bucket_policy_status ERROR [ 54%] 2026-02-24T22:31:32.635 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_nonpublicpolicy_principal_bucket_policy_status ERROR [ 54%] 2026-02-24T22:31:32.895 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_policy_allow_notprincipal ERROR [ 54%] 2026-02-24T22:31:33.122 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_undefined_public_block ERROR [ 54%] 2026-02-24T22:31:33.349 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_public_block_deny_bucket_policy ERROR [ 54%] 2026-02-24T22:31:33.577 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_public_block ERROR [ 54%] 2026-02-24T22:31:33.849 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_block_public_put_bucket_acls ERROR [ 54%] 2026-02-24T22:31:34.079 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_block_public_object_canned_acls ERROR [ 54%] 2026-02-24T22:31:34.306 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_block_public_policy ERROR [ 54%] 2026-02-24T22:31:34.573 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_block_public_policy_with_principal ERROR [ 55%] 2026-02-24T22:31:34.801 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_block_public_restrict_public_buckets ERROR [ 55%] 2026-02-24T22:31:35.030 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_ignore_public_acls ERROR [ 55%] 2026-02-24T22:31:35.255 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_get_delete_public_block ERROR [ 55%] 2026-02-24T22:31:35.526 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_upload_on_a_bucket_with_policy ERROR [ 55%] 2026-02-24T22:31:35.761 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_bucket_encryption_kms ERROR [ 55%] 2026-02-24T22:31:35.988 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_bucket_encryption_kms ERROR [ 55%] 2026-02-24T22:31:36.257 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_delete_bucket_encryption_kms ERROR [ 55%] 2026-02-24T22:31:36.484 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_sse_kms_default_post_object_authenticated_request ERROR [ 55%] 2026-02-24T22:31:36.714 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_object_torrent ERROR [ 55%] 2026-02-24T22:31:36.940 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_checksum_sha256 ERROR [ 55%] 2026-02-24T22:31:37.201 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_object_checksum_crc64nvme ERROR [ 56%] 2026-02-24T22:31:37.428 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_checksum_sha256 ERROR [ 56%] 2026-02-24T22:31:37.658 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_use_cksum_helper_sha256 ERROR [ 56%] 2026-02-24T22:31:37.885 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_use_cksum_helper_crc64nvme ERROR [ 56%] 2026-02-24T22:31:38.158 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_use_cksum_helper_crc32 ERROR [ 56%] 2026-02-24T22:31:38.384 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_use_cksum_helper_crc32c ERROR [ 56%] 2026-02-24T22:31:38.614 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_use_cksum_helper_sha1 ERROR [ 56%] 2026-02-24T22:31:38.881 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_post_object_upload_checksum ERROR [ 56%] 2026-02-24T22:31:39.109 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_multipart_checksum_object_attributes ERROR [ 56%] 2026-02-24T22:31:39.337 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_multipart_object_attributes ERROR [ 56%] 2026-02-24T22:31:39.565 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_paginated_multipart_object_attributes ERROR [ 56%] 2026-02-24T22:31:39.837 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_single_multipart_object_attributes ERROR [ 57%] 2026-02-24T22:31:40.064 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_checksum_object_attributes ERROR [ 57%] 2026-02-24T22:31:40.294 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_versioned_object_attributes ERROR [ 57%] 2026-02-24T22:31:40.565 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_sse_c_encrypted_object_attributes ERROR [ 57%] 2026-02-24T22:31:40.793 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_get_object_attributes ERROR [ 57%] 2026-02-24T22:31:41.020 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_upload_part_copy_percent_encoded_key ERROR [ 57%] 2026-02-24T22:31:41.455 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_delete_marker_nonversioned ERROR [ 57%] 2026-02-24T22:31:41.715 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_delete_marker_versioned ERROR [ 57%] 2026-02-24T22:31:41.944 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_delete_marker_suspended ERROR [ 57%] 2026-02-24T22:31:42.172 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_object_if_match ERROR [ 57%] 2026-02-24T22:31:42.400 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_put_object_if_match ERROR [ 57%] 2026-02-24T22:31:42.668 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_current_object_if_none_match ERROR [ 57%] 2026-02-24T22:31:43.009 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_put_current_object_if_none_match ERROR [ 58%] 2026-02-24T22:31:43.589 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_current_object_if_match ERROR [ 58%] 2026-02-24T22:31:43.862 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_multipart_put_current_object_if_match ERROR [ 58%] 2026-02-24T22:31:44.089 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_object_current_if_match ERROR [ 58%] 2026-02-24T22:31:44.321 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_delete_object_if_match ERROR [ 58%] 2026-02-24T22:31:44.550 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_delete_object_current_if_match ERROR [ 58%] 2026-02-24T22:31:44.826 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_delete_object_version_if_match ERROR [ 58%] 2026-02-24T22:31:45.054 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_delete_object_if_match_last_modified_time ERROR [ 58%] 2026-02-24T22:31:45.282 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_delete_object_current_if_match_last_modified_time ERROR [ 58%] 2026-02-24T22:31:45.553 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_delete_object_version_if_match_last_modified_time ERROR [ 58%] 2026-02-24T22:31:45.781 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_delete_object_if_match_size ERROR [ 58%] 2026-02-24T22:31:46.011 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_delete_object_current_if_match_size ERROR [ 59%] 2026-02-24T22:31:46.236 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_delete_object_version_if_match_size ERROR [ 59%] 2026-02-24T22:31:46.495 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_delete_objects_if_match ERROR [ 59%] 2026-02-24T22:31:46.722 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_delete_objects_current_if_match ERROR [ 59%] 2026-02-24T22:31:46.952 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_delete_objects_version_if_match ERROR [ 59%] 2026-02-24T22:31:47.182 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_delete_objects_if_match_last_modified_time ERROR [ 59%] 2026-02-24T22:31:47.451 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_delete_objects_current_if_match_last_modified_time ERROR [ 59%] 2026-02-24T22:31:47.685 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_delete_objects_version_if_match_last_modified_time ERROR [ 59%] 2026-02-24T22:31:47.915 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_delete_objects_if_match_size ERROR [ 59%] 2026-02-24T22:31:48.172 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_delete_objects_current_if_match_size ERROR [ 59%] 2026-02-24T22:31:48.402 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_delete_objects_version_if_match_size ERROR [ 59%] 2026-02-24T22:31:48.630 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_create_bucket_no_ownership_controls ERROR [ 60%] 2026-02-24T22:31:48.855 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_create_bucket_bucket_owner_enforced ERROR [ 60%] 2026-02-24T22:31:49.124 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_create_bucket_bucket_owner_preferred ERROR [ 60%] 2026-02-24T22:31:49.352 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_create_bucket_object_writer ERROR [ 60%] 2026-02-24T22:31:49.582 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_bucket_ownership_bucket_owner_enforced ERROR [ 60%] 2026-02-24T22:31:49.853 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_bucket_ownership_bucket_owner_preferred ERROR [ 60%] 2026-02-24T22:31:50.081 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_put_bucket_ownership_object_writer ERROR [ 60%] 2026-02-24T22:31:50.311 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_bucket_create_delete_bucket_ownership ERROR [ 60%] 2026-02-24T22:31:50.537 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-unencrypted-STANDARD-STANDARD-8388608] ERROR [ 60%] 2026-02-24T22:31:50.801 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-unencrypted-STANDARD-LUKEWARM-8388608] ERROR [ 60%] 2026-02-24T22:31:51.030 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-unencrypted-STANDARD-FROZEN-8388608] ERROR [ 60%] 2026-02-24T22:31:51.256 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-unencrypted-LUKEWARM-STANDARD-8388608] ERROR [ 60%] 2026-02-24T22:31:51.489 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-unencrypted-LUKEWARM-LUKEWARM-8388608] ERROR [ 61%] 2026-02-24T22:31:51.758 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-unencrypted-LUKEWARM-FROZEN-8388608] ERROR [ 61%] 2026-02-24T22:31:51.986 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-unencrypted-FROZEN-STANDARD-8388608] ERROR [ 61%] 2026-02-24T22:31:52.212 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-unencrypted-FROZEN-LUKEWARM-8388608] ERROR [ 61%] 2026-02-24T22:31:52.477 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-unencrypted-FROZEN-FROZEN-8388608] ERROR [ 61%] 2026-02-24T22:31:52.706 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-c-STANDARD-STANDARD-8388608] ERROR [ 61%] 2026-02-24T22:31:52.934 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-c-STANDARD-LUKEWARM-8388608] ERROR [ 61%] 2026-02-24T22:31:53.161 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-c-STANDARD-FROZEN-8388608] ERROR [ 61%] 2026-02-24T22:31:53.434 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-c-LUKEWARM-STANDARD-8388608] ERROR [ 61%] 2026-02-24T22:31:53.661 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-c-LUKEWARM-LUKEWARM-8388608] ERROR [ 61%] 2026-02-24T22:31:53.890 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-c-LUKEWARM-FROZEN-8388608] ERROR [ 61%] 2026-02-24T22:31:54.161 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-c-FROZEN-STANDARD-8388608] ERROR [ 62%] 2026-02-24T22:31:54.391 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-c-FROZEN-LUKEWARM-8388608] ERROR [ 62%] 2026-02-24T22:31:54.622 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-c-FROZEN-FROZEN-8388608] ERROR [ 62%] 2026-02-24T22:31:54.848 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-kms-STANDARD-STANDARD-8388608] ERROR [ 62%] 2026-02-24T22:31:55.116 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-kms-STANDARD-LUKEWARM-8388608] ERROR [ 62%] 2026-02-24T22:31:55.347 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-kms-STANDARD-FROZEN-8388608] ERROR [ 62%] 2026-02-24T22:31:55.573 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-kms-LUKEWARM-STANDARD-8388608] ERROR [ 62%] 2026-02-24T22:31:55.848 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-kms-LUKEWARM-LUKEWARM-8388608] ERROR [ 62%] 2026-02-24T22:31:56.078 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-kms-LUKEWARM-FROZEN-8388608] ERROR [ 62%] 2026-02-24T22:31:56.310 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-kms-FROZEN-STANDARD-8388608] ERROR [ 62%] 2026-02-24T22:31:56.542 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-kms-FROZEN-LUKEWARM-8388608] ERROR [ 62%] 2026-02-24T22:31:56.806 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-kms-FROZEN-FROZEN-8388608] ERROR [ 63%] 2026-02-24T22:31:57.035 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-unencrypted-STANDARD-STANDARD-8388608] ERROR [ 63%] 2026-02-24T22:31:57.261 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-unencrypted-STANDARD-LUKEWARM-8388608] ERROR [ 63%] 2026-02-24T22:31:57.492 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-unencrypted-STANDARD-FROZEN-8388608] ERROR [ 63%] 2026-02-24T22:31:57.771 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-unencrypted-LUKEWARM-STANDARD-8388608] ERROR [ 63%] 2026-02-24T22:31:57.998 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-unencrypted-LUKEWARM-LUKEWARM-8388608] ERROR [ 63%] 2026-02-24T22:31:58.226 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-unencrypted-LUKEWARM-FROZEN-8388608] ERROR [ 63%] 2026-02-24T22:31:58.494 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-unencrypted-FROZEN-STANDARD-8388608] ERROR [ 63%] 2026-02-24T22:31:58.722 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-unencrypted-FROZEN-LUKEWARM-8388608] ERROR [ 63%] 2026-02-24T22:31:58.949 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-unencrypted-FROZEN-FROZEN-8388608] ERROR [ 63%] 2026-02-24T22:31:59.177 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-c-STANDARD-STANDARD-8388608] ERROR [ 63%] 2026-02-24T22:31:59.396 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-c-STANDARD-LUKEWARM-8388608] ERROR [ 64%] 2026-02-24T22:31:59.625 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-c-STANDARD-FROZEN-8388608] ERROR [ 64%] 2026-02-24T22:31:59.858 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-c-LUKEWARM-STANDARD-8388608] ERROR [ 64%] 2026-02-24T22:32:00.134 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-c-LUKEWARM-LUKEWARM-8388608] ERROR [ 64%] 2026-02-24T22:32:00.361 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-c-LUKEWARM-FROZEN-8388608] ERROR [ 64%] 2026-02-24T22:32:00.590 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-c-FROZEN-STANDARD-8388608] ERROR [ 64%] 2026-02-24T22:32:00.818 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-c-FROZEN-LUKEWARM-8388608] ERROR [ 64%] 2026-02-24T22:32:01.081 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-c-FROZEN-FROZEN-8388608] ERROR [ 64%] 2026-02-24T22:32:01.309 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-kms-STANDARD-STANDARD-8388608] ERROR [ 64%] 2026-02-24T22:32:01.539 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-kms-STANDARD-LUKEWARM-8388608] ERROR [ 64%] 2026-02-24T22:32:01.764 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-kms-STANDARD-FROZEN-8388608] ERROR [ 64%] 2026-02-24T22:32:02.032 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-kms-LUKEWARM-STANDARD-8388608] ERROR [ 64%] 2026-02-24T22:32:02.257 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-kms-LUKEWARM-LUKEWARM-8388608] ERROR [ 65%] 2026-02-24T22:32:02.486 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-kms-LUKEWARM-FROZEN-8388608] ERROR [ 65%] 2026-02-24T22:32:02.755 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-kms-FROZEN-STANDARD-8388608] ERROR [ 65%] 2026-02-24T22:32:02.981 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-kms-FROZEN-LUKEWARM-8388608] ERROR [ 65%] 2026-02-24T22:32:03.210 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-kms-FROZEN-FROZEN-8388608] ERROR [ 65%] 2026-02-24T22:32:03.436 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-unencrypted-STANDARD-STANDARD-8388608] ERROR [ 65%] 2026-02-24T22:32:03.711 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-unencrypted-STANDARD-LUKEWARM-8388608] ERROR [ 65%] 2026-02-24T22:32:03.937 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-unencrypted-STANDARD-FROZEN-8388608] ERROR [ 65%] 2026-02-24T22:32:04.167 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-unencrypted-LUKEWARM-STANDARD-8388608] ERROR [ 65%] 2026-02-24T22:32:04.444 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-unencrypted-LUKEWARM-LUKEWARM-8388608] ERROR [ 65%] 2026-02-24T22:32:04.674 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-unencrypted-LUKEWARM-FROZEN-8388608] ERROR [ 65%] 2026-02-24T22:32:04.901 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-unencrypted-FROZEN-STANDARD-8388608] ERROR [ 66%] 2026-02-24T22:32:05.135 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-unencrypted-FROZEN-LUKEWARM-8388608] ERROR [ 66%] 2026-02-24T22:32:05.403 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-unencrypted-FROZEN-FROZEN-8388608] ERROR [ 66%] 2026-02-24T22:32:05.631 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-c-STANDARD-STANDARD-8388608] ERROR [ 66%] 2026-02-24T22:32:05.859 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-c-STANDARD-LUKEWARM-8388608] ERROR [ 66%] 2026-02-24T22:32:06.087 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-c-STANDARD-FROZEN-8388608] ERROR [ 66%] 2026-02-24T22:32:06.357 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-c-LUKEWARM-STANDARD-8388608] ERROR [ 66%] 2026-02-24T22:32:06.585 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-c-LUKEWARM-LUKEWARM-8388608] ERROR [ 66%] 2026-02-24T22:32:06.815 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-c-LUKEWARM-FROZEN-8388608] ERROR [ 66%] 2026-02-24T22:32:07.083 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-c-FROZEN-STANDARD-8388608] ERROR [ 66%] 2026-02-24T22:32:07.307 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-c-FROZEN-LUKEWARM-8388608] ERROR [ 66%] 2026-02-24T22:32:07.533 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-c-FROZEN-FROZEN-8388608] ERROR [ 67%] 2026-02-24T22:32:07.760 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-kms-STANDARD-STANDARD-8388608] ERROR [ 67%] 2026-02-24T22:32:08.034 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-kms-STANDARD-LUKEWARM-8388608] ERROR [ 67%] 2026-02-24T22:32:08.271 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-kms-STANDARD-FROZEN-8388608] ERROR [ 67%] 2026-02-24T22:32:08.497 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-kms-LUKEWARM-STANDARD-8388608] ERROR [ 67%] 2026-02-24T22:32:08.768 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-kms-LUKEWARM-LUKEWARM-8388608] ERROR [ 67%] 2026-02-24T22:32:08.999 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-kms-LUKEWARM-FROZEN-8388608] ERROR [ 67%] 2026-02-24T22:32:09.225 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-kms-FROZEN-STANDARD-8388608] ERROR [ 67%] 2026-02-24T22:32:09.453 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-kms-FROZEN-LUKEWARM-8388608] ERROR [ 67%] 2026-02-24T22:32:09.718 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-kms-FROZEN-FROZEN-8388608] ERROR [ 67%] 2026-02-24T22:32:09.951 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-STANDARD-STANDARD-1] ERROR [ 67%] 2026-02-24T22:32:10.179 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-STANDARD-STANDARD-1024] ERROR [ 67%] 2026-02-24T22:32:10.406 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-STANDARD-STANDARD-1048576] ERROR [ 68%] 2026-02-24T22:32:10.679 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-STANDARD-STANDARD-8388608] ERROR [ 68%] 2026-02-24T22:32:10.906 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-STANDARD-LUKEWARM-1] ERROR [ 68%] 2026-02-24T22:32:11.132 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-STANDARD-LUKEWARM-1024] ERROR [ 68%] 2026-02-24T22:32:11.404 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-STANDARD-LUKEWARM-1048576] ERROR [ 68%] 2026-02-24T22:32:11.633 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-STANDARD-LUKEWARM-8388608] ERROR [ 68%] 2026-02-24T22:32:11.860 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-STANDARD-FROZEN-1] ERROR [ 68%] 2026-02-24T22:32:12.090 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-STANDARD-FROZEN-1024] ERROR [ 68%] 2026-02-24T22:32:12.370 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-STANDARD-FROZEN-1048576] ERROR [ 68%] 2026-02-24T22:32:12.597 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-STANDARD-FROZEN-8388608] ERROR [ 68%] 2026-02-24T22:32:12.827 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-LUKEWARM-STANDARD-1] ERROR [ 68%] 2026-02-24T22:32:13.100 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-LUKEWARM-STANDARD-1024] ERROR [ 69%] 2026-02-24T22:32:13.331 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-LUKEWARM-STANDARD-1048576] ERROR [ 69%] 2026-02-24T22:32:13.558 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-LUKEWARM-STANDARD-8388608] ERROR [ 69%] 2026-02-24T22:32:13.787 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-LUKEWARM-LUKEWARM-1] ERROR [ 69%] 2026-02-24T22:32:14.050 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-LUKEWARM-LUKEWARM-1024] ERROR [ 69%] 2026-02-24T22:32:14.280 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-LUKEWARM-LUKEWARM-1048576] ERROR [ 69%] 2026-02-24T22:32:14.507 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-LUKEWARM-LUKEWARM-8388608] ERROR [ 69%] 2026-02-24T22:32:14.734 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-LUKEWARM-FROZEN-1] ERROR [ 69%] 2026-02-24T22:32:15.010 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-LUKEWARM-FROZEN-1024] ERROR [ 69%] 2026-02-24T22:32:15.240 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-LUKEWARM-FROZEN-1048576] ERROR [ 69%] 2026-02-24T22:32:15.469 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-LUKEWARM-FROZEN-8388608] ERROR [ 69%] 2026-02-24T22:32:15.739 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-FROZEN-STANDARD-1] ERROR [ 70%] 2026-02-24T22:32:15.965 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-FROZEN-STANDARD-1024] ERROR [ 70%] 2026-02-24T22:32:16.194 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-FROZEN-STANDARD-1048576] ERROR [ 70%] 2026-02-24T22:32:16.422 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-FROZEN-STANDARD-8388608] ERROR [ 70%] 2026-02-24T22:32:16.695 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-FROZEN-LUKEWARM-1] ERROR [ 70%] 2026-02-24T22:32:16.921 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-FROZEN-LUKEWARM-1024] ERROR [ 70%] 2026-02-24T22:32:17.149 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-FROZEN-LUKEWARM-1048576] ERROR [ 70%] 2026-02-24T22:32:17.421 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-FROZEN-LUKEWARM-8388608] ERROR [ 70%] 2026-02-24T22:32:17.649 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-FROZEN-FROZEN-1] ERROR [ 70%] 2026-02-24T22:32:17.879 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-FROZEN-FROZEN-1024] ERROR [ 70%] 2026-02-24T22:32:18.105 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-FROZEN-FROZEN-1048576] ERROR [ 70%] 2026-02-24T22:32:18.374 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-FROZEN-FROZEN-8388608] ERROR [ 71%] 2026-02-24T22:32:18.599 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-STANDARD-STANDARD-1] ERROR [ 71%] 2026-02-24T22:32:18.827 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-STANDARD-STANDARD-1024] ERROR [ 71%] 2026-02-24T22:32:19.052 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-STANDARD-STANDARD-1048576] ERROR [ 71%] 2026-02-24T22:32:19.331 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-STANDARD-STANDARD-8388608] ERROR [ 71%] 2026-02-24T22:32:19.558 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-STANDARD-LUKEWARM-1] ERROR [ 71%] 2026-02-24T22:32:19.785 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-STANDARD-LUKEWARM-1024] ERROR [ 71%] 2026-02-24T22:32:20.051 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-STANDARD-LUKEWARM-1048576] ERROR [ 71%] 2026-02-24T22:32:20.277 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-STANDARD-LUKEWARM-8388608] ERROR [ 71%] 2026-02-24T22:32:20.506 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-STANDARD-FROZEN-1] ERROR [ 71%] 2026-02-24T22:32:20.733 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-STANDARD-FROZEN-1024] ERROR [ 71%] 2026-02-24T22:32:21.011 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-STANDARD-FROZEN-1048576] ERROR [ 71%] 2026-02-24T22:32:21.239 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-STANDARD-FROZEN-8388608] ERROR [ 72%] 2026-02-24T22:32:21.465 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-LUKEWARM-STANDARD-1] ERROR [ 72%] 2026-02-24T22:32:21.737 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-LUKEWARM-STANDARD-1024] ERROR [ 72%] 2026-02-24T22:32:21.964 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-LUKEWARM-STANDARD-1048576] ERROR [ 72%] 2026-02-24T22:32:22.193 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-LUKEWARM-STANDARD-8388608] ERROR [ 72%] 2026-02-24T22:32:22.427 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-LUKEWARM-LUKEWARM-1] ERROR [ 72%] 2026-02-24T22:32:22.690 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-LUKEWARM-LUKEWARM-1024] ERROR [ 72%] 2026-02-24T22:32:22.917 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-LUKEWARM-LUKEWARM-1048576] ERROR [ 72%] 2026-02-24T22:32:23.146 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-LUKEWARM-LUKEWARM-8388608] ERROR [ 72%] 2026-02-24T22:32:23.374 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-LUKEWARM-FROZEN-1] ERROR [ 72%] 2026-02-24T22:32:23.646 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-LUKEWARM-FROZEN-1024] ERROR [ 72%] 2026-02-24T22:32:23.875 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-LUKEWARM-FROZEN-1048576] ERROR [ 73%] 2026-02-24T22:32:24.103 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-LUKEWARM-FROZEN-8388608] ERROR [ 73%] 2026-02-24T22:32:24.367 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-FROZEN-STANDARD-1] ERROR [ 73%] 2026-02-24T22:32:24.592 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-FROZEN-STANDARD-1024] ERROR [ 73%] 2026-02-24T22:32:24.821 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-FROZEN-STANDARD-1048576] ERROR [ 73%] 2026-02-24T22:32:25.048 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-FROZEN-STANDARD-8388608] ERROR [ 73%] 2026-02-24T22:32:25.324 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-FROZEN-LUKEWARM-1] ERROR [ 73%] 2026-02-24T22:32:25.551 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-FROZEN-LUKEWARM-1024] ERROR [ 73%] 2026-02-24T22:32:25.779 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-FROZEN-LUKEWARM-1048576] ERROR [ 73%] 2026-02-24T22:32:26.052 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-FROZEN-LUKEWARM-8388608] ERROR [ 73%] 2026-02-24T22:32:26.277 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-FROZEN-FROZEN-1] ERROR [ 73%] 2026-02-24T22:32:26.505 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-FROZEN-FROZEN-1024] ERROR [ 74%] 2026-02-24T22:32:26.739 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-FROZEN-FROZEN-1048576] ERROR [ 74%] 2026-02-24T22:32:27.004 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-FROZEN-FROZEN-8388608] ERROR [ 74%] 2026-02-24T22:32:27.235 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-STANDARD-STANDARD-1] ERROR [ 74%] 2026-02-24T22:32:27.462 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-STANDARD-STANDARD-1024] ERROR [ 74%] 2026-02-24T22:32:27.739 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-STANDARD-STANDARD-1048576] ERROR [ 74%] 2026-02-24T22:32:27.964 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-STANDARD-STANDARD-8388608] ERROR [ 74%] 2026-02-24T22:32:28.190 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-STANDARD-LUKEWARM-1] ERROR [ 74%] 2026-02-24T22:32:28.418 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-STANDARD-LUKEWARM-1024] ERROR [ 74%] 2026-02-24T22:32:28.687 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-STANDARD-LUKEWARM-1048576] ERROR [ 74%] 2026-02-24T22:32:28.912 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-STANDARD-LUKEWARM-8388608] ERROR [ 74%] 2026-02-24T22:32:29.137 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-STANDARD-FROZEN-1] ERROR [ 75%] 2026-02-24T22:32:29.365 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-STANDARD-FROZEN-1024] ERROR [ 75%] 2026-02-24T22:32:29.640 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-STANDARD-FROZEN-1048576] ERROR [ 75%] 2026-02-24T22:32:29.866 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-STANDARD-FROZEN-8388608] ERROR [ 75%] 2026-02-24T22:32:30.099 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-LUKEWARM-STANDARD-1] ERROR [ 75%] 2026-02-24T22:32:30.373 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-LUKEWARM-STANDARD-1024] ERROR [ 75%] 2026-02-24T22:32:30.602 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-LUKEWARM-STANDARD-1048576] ERROR [ 75%] 2026-02-24T22:32:30.829 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-LUKEWARM-STANDARD-8388608] ERROR [ 75%] 2026-02-24T22:32:31.056 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-LUKEWARM-LUKEWARM-1] ERROR [ 75%] 2026-02-24T22:32:31.272 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-LUKEWARM-LUKEWARM-1024] ERROR [ 75%] 2026-02-24T22:32:31.500 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-LUKEWARM-LUKEWARM-1048576] ERROR [ 75%] 2026-02-24T22:32:31.727 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-LUKEWARM-LUKEWARM-8388608] ERROR [ 75%] 2026-02-24T22:32:32.004 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-LUKEWARM-FROZEN-1] ERROR [ 76%] 2026-02-24T22:32:32.236 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-LUKEWARM-FROZEN-1024] ERROR [ 76%] 2026-02-24T22:32:32.463 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-LUKEWARM-FROZEN-1048576] ERROR [ 76%] 2026-02-24T22:32:32.689 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-LUKEWARM-FROZEN-8388608] ERROR [ 76%] 2026-02-24T22:32:32.954 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-FROZEN-STANDARD-1] ERROR [ 76%] 2026-02-24T22:32:33.183 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-FROZEN-STANDARD-1024] ERROR [ 76%] 2026-02-24T22:32:33.409 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-FROZEN-STANDARD-1048576] ERROR [ 76%] 2026-02-24T22:32:33.635 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-FROZEN-STANDARD-8388608] ERROR [ 76%] 2026-02-24T22:32:33.915 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-FROZEN-LUKEWARM-1] ERROR [ 76%] 2026-02-24T22:32:34.146 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-FROZEN-LUKEWARM-1024] ERROR [ 76%] 2026-02-24T22:32:34.375 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-FROZEN-LUKEWARM-1048576] ERROR [ 76%] 2026-02-24T22:32:34.650 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-FROZEN-LUKEWARM-8388608] ERROR [ 77%] 2026-02-24T22:32:34.876 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-FROZEN-FROZEN-1] ERROR [ 77%] 2026-02-24T22:32:35.103 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-FROZEN-FROZEN-1024] ERROR [ 77%] 2026-02-24T22:32:35.330 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-FROZEN-FROZEN-1048576] ERROR [ 77%] 2026-02-24T22:32:35.611 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-FROZEN-FROZEN-8388608] ERROR [ 77%] 2026-02-24T22:32:35.839 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-STANDARD-STANDARD-1] ERROR [ 77%] 2026-02-24T22:32:36.067 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-STANDARD-STANDARD-1024] ERROR [ 77%] 2026-02-24T22:32:36.339 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-STANDARD-STANDARD-1048576] ERROR [ 77%] 2026-02-24T22:32:36.567 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-STANDARD-STANDARD-8388608] ERROR [ 77%] 2026-02-24T22:32:36.794 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-STANDARD-LUKEWARM-1] ERROR [ 77%] 2026-02-24T22:32:37.021 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-STANDARD-LUKEWARM-1024] ERROR [ 77%] 2026-02-24T22:32:37.290 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-STANDARD-LUKEWARM-1048576] ERROR [ 78%] 2026-02-24T22:32:37.516 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-STANDARD-LUKEWARM-8388608] ERROR [ 78%] 2026-02-24T22:32:37.743 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-STANDARD-FROZEN-1] ERROR [ 78%] 2026-02-24T22:32:37.971 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-STANDARD-FROZEN-1024] ERROR [ 78%] 2026-02-24T22:32:38.245 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-STANDARD-FROZEN-1048576] ERROR [ 78%] 2026-02-24T22:32:38.474 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-STANDARD-FROZEN-8388608] ERROR [ 78%] 2026-02-24T22:32:38.703 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-LUKEWARM-STANDARD-1] ERROR [ 78%] 2026-02-24T22:32:38.976 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-LUKEWARM-STANDARD-1024] ERROR [ 78%] 2026-02-24T22:32:39.203 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-LUKEWARM-STANDARD-1048576] ERROR [ 78%] 2026-02-24T22:32:39.428 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-LUKEWARM-STANDARD-8388608] ERROR [ 78%] 2026-02-24T22:32:39.653 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-LUKEWARM-LUKEWARM-1] ERROR [ 78%] 2026-02-24T22:32:39.929 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-LUKEWARM-LUKEWARM-1024] ERROR [ 78%] 2026-02-24T22:32:40.158 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-LUKEWARM-LUKEWARM-1048576] ERROR [ 79%] 2026-02-24T22:32:40.386 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-LUKEWARM-LUKEWARM-8388608] ERROR [ 79%] 2026-02-24T22:32:40.663 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-LUKEWARM-FROZEN-1] ERROR [ 79%] 2026-02-24T22:32:40.891 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-LUKEWARM-FROZEN-1024] ERROR [ 79%] 2026-02-24T22:32:41.118 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-LUKEWARM-FROZEN-1048576] ERROR [ 79%] 2026-02-24T22:32:41.347 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-LUKEWARM-FROZEN-8388608] ERROR [ 79%] 2026-02-24T22:32:41.617 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-FROZEN-STANDARD-1] ERROR [ 79%] 2026-02-24T22:32:41.848 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-FROZEN-STANDARD-1024] ERROR [ 79%] 2026-02-24T22:32:42.074 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-FROZEN-STANDARD-1048576] ERROR [ 79%] 2026-02-24T22:32:42.303 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-FROZEN-STANDARD-8388608] ERROR [ 79%] 2026-02-24T22:32:42.579 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-FROZEN-LUKEWARM-1] ERROR [ 79%] 2026-02-24T22:32:42.806 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-FROZEN-LUKEWARM-1024] ERROR [ 80%] 2026-02-24T22:32:43.033 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-FROZEN-LUKEWARM-1048576] ERROR [ 80%] 2026-02-24T22:32:43.309 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-FROZEN-LUKEWARM-8388608] ERROR [ 80%] 2026-02-24T22:32:43.540 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-FROZEN-FROZEN-1] ERROR [ 80%] 2026-02-24T22:32:43.767 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-FROZEN-FROZEN-1024] ERROR [ 80%] 2026-02-24T22:32:43.995 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-FROZEN-FROZEN-1048576] ERROR [ 80%] 2026-02-24T22:32:44.273 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-FROZEN-FROZEN-8388608] ERROR [ 80%] 2026-02-24T22:32:44.502 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-STANDARD-STANDARD-1] ERROR [ 80%] 2026-02-24T22:32:44.732 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-STANDARD-STANDARD-1024] ERROR [ 80%] 2026-02-24T22:32:45.006 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-STANDARD-STANDARD-1048576] ERROR [ 80%] 2026-02-24T22:32:45.234 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-STANDARD-STANDARD-8388608] ERROR [ 80%] 2026-02-24T22:32:45.467 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-STANDARD-LUKEWARM-1] ERROR [ 81%] 2026-02-24T22:32:45.693 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-STANDARD-LUKEWARM-1024] ERROR [ 81%] 2026-02-24T22:32:45.958 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-STANDARD-LUKEWARM-1048576] ERROR [ 81%] 2026-02-24T22:32:46.188 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-STANDARD-LUKEWARM-8388608] ERROR [ 81%] 2026-02-24T22:32:46.415 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-STANDARD-FROZEN-1] ERROR [ 81%] 2026-02-24T22:32:46.643 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-STANDARD-FROZEN-1024] ERROR [ 81%] 2026-02-24T22:32:46.918 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-STANDARD-FROZEN-1048576] ERROR [ 81%] 2026-02-24T22:32:47.147 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-STANDARD-FROZEN-8388608] ERROR [ 81%] 2026-02-24T22:32:47.375 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-LUKEWARM-STANDARD-1] ERROR [ 81%] 2026-02-24T22:32:47.647 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-LUKEWARM-STANDARD-1024] ERROR [ 81%] 2026-02-24T22:32:47.872 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-LUKEWARM-STANDARD-1048576] ERROR [ 81%] 2026-02-24T22:32:48.100 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-LUKEWARM-STANDARD-8388608] ERROR [ 82%] 2026-02-24T22:32:48.330 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-LUKEWARM-LUKEWARM-1] ERROR [ 82%] 2026-02-24T22:32:48.614 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-LUKEWARM-LUKEWARM-1024] ERROR [ 82%] 2026-02-24T22:32:48.844 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-LUKEWARM-LUKEWARM-1048576] ERROR [ 82%] 2026-02-24T22:32:49.075 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-LUKEWARM-LUKEWARM-8388608] ERROR [ 82%] 2026-02-24T22:32:49.353 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-LUKEWARM-FROZEN-1] ERROR [ 82%] 2026-02-24T22:32:49.584 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-LUKEWARM-FROZEN-1024] ERROR [ 82%] 2026-02-24T22:32:49.810 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-LUKEWARM-FROZEN-1048576] ERROR [ 82%] 2026-02-24T22:32:50.037 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-LUKEWARM-FROZEN-8388608] ERROR [ 82%] 2026-02-24T22:32:50.308 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-FROZEN-STANDARD-1] ERROR [ 82%] 2026-02-24T22:32:50.535 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-FROZEN-STANDARD-1024] ERROR [ 82%] 2026-02-24T22:32:50.762 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-FROZEN-STANDARD-1048576] ERROR [ 82%] 2026-02-24T22:32:50.989 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-FROZEN-STANDARD-8388608] ERROR [ 83%] 2026-02-24T22:32:51.267 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-FROZEN-LUKEWARM-1] ERROR [ 83%] 2026-02-24T22:32:51.495 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-FROZEN-LUKEWARM-1024] ERROR [ 83%] 2026-02-24T22:32:51.722 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-FROZEN-LUKEWARM-1048576] ERROR [ 83%] 2026-02-24T22:32:51.990 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-FROZEN-LUKEWARM-8388608] ERROR [ 83%] 2026-02-24T22:32:52.218 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-FROZEN-FROZEN-1] ERROR [ 83%] 2026-02-24T22:32:52.446 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-FROZEN-FROZEN-1024] ERROR [ 83%] 2026-02-24T22:32:52.673 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-FROZEN-FROZEN-1048576] ERROR [ 83%] 2026-02-24T22:32:52.951 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-FROZEN-FROZEN-8388608] ERROR [ 83%] 2026-02-24T22:32:53.180 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-STANDARD-STANDARD-1] ERROR [ 83%] 2026-02-24T22:32:53.408 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-STANDARD-STANDARD-1024] ERROR [ 83%] 2026-02-24T22:32:53.683 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-STANDARD-STANDARD-1048576] ERROR [ 84%] 2026-02-24T22:32:53.911 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-STANDARD-STANDARD-8388608] ERROR [ 84%] 2026-02-24T22:32:54.139 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-STANDARD-LUKEWARM-1] ERROR [ 84%] 2026-02-24T22:32:54.367 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-STANDARD-LUKEWARM-1024] ERROR [ 84%] 2026-02-24T22:32:54.636 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-STANDARD-LUKEWARM-1048576] ERROR [ 84%] 2026-02-24T22:32:54.862 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-STANDARD-LUKEWARM-8388608] ERROR [ 84%] 2026-02-24T22:32:55.089 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-STANDARD-FROZEN-1] ERROR [ 84%] 2026-02-24T22:32:55.315 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-STANDARD-FROZEN-1024] ERROR [ 84%] 2026-02-24T22:32:55.594 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-STANDARD-FROZEN-1048576] ERROR [ 84%] 2026-02-24T22:32:55.823 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-STANDARD-FROZEN-8388608] ERROR [ 84%] 2026-02-24T22:32:56.053 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-LUKEWARM-STANDARD-1] ERROR [ 84%] 2026-02-24T22:32:56.320 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-LUKEWARM-STANDARD-1024] ERROR [ 85%] 2026-02-24T22:32:56.548 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-LUKEWARM-STANDARD-1048576] ERROR [ 85%] 2026-02-24T22:32:56.775 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-LUKEWARM-STANDARD-8388608] ERROR [ 85%] 2026-02-24T22:32:57.001 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-LUKEWARM-LUKEWARM-1] ERROR [ 85%] 2026-02-24T22:32:57.276 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-LUKEWARM-LUKEWARM-1024] ERROR [ 85%] 2026-02-24T22:32:57.502 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-LUKEWARM-LUKEWARM-1048576] ERROR [ 85%] 2026-02-24T22:32:57.730 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-LUKEWARM-LUKEWARM-8388608] ERROR [ 85%] 2026-02-24T22:32:58.006 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-LUKEWARM-FROZEN-1] ERROR [ 85%] 2026-02-24T22:32:58.236 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-LUKEWARM-FROZEN-1024] ERROR [ 85%] 2026-02-24T22:32:58.466 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-LUKEWARM-FROZEN-1048576] ERROR [ 85%] 2026-02-24T22:32:58.693 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-LUKEWARM-FROZEN-8388608] ERROR [ 85%] 2026-02-24T22:32:58.963 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-FROZEN-STANDARD-1] ERROR [ 85%] 2026-02-24T22:32:59.190 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-FROZEN-STANDARD-1024] ERROR [ 86%] 2026-02-24T22:32:59.419 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-FROZEN-STANDARD-1048576] ERROR [ 86%] 2026-02-24T22:32:59.701 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-FROZEN-STANDARD-8388608] ERROR [ 86%] 2026-02-24T22:32:59.930 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-FROZEN-LUKEWARM-1] ERROR [ 86%] 2026-02-24T22:33:00.158 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-FROZEN-LUKEWARM-1024] ERROR [ 86%] 2026-02-24T22:33:00.386 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-FROZEN-LUKEWARM-1048576] ERROR [ 86%] 2026-02-24T22:33:00.659 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-FROZEN-LUKEWARM-8388608] ERROR [ 86%] 2026-02-24T22:33:00.892 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-FROZEN-FROZEN-1] ERROR [ 86%] 2026-02-24T22:33:01.118 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-FROZEN-FROZEN-1024] ERROR [ 86%] 2026-02-24T22:33:01.345 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-FROZEN-FROZEN-1048576] ERROR [ 86%] 2026-02-24T22:33:01.620 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-FROZEN-FROZEN-8388608] ERROR [ 86%] 2026-02-24T22:33:01.846 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-STANDARD-STANDARD-1] ERROR [ 87%] 2026-02-24T22:33:02.074 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-STANDARD-STANDARD-1024] ERROR [ 87%] 2026-02-24T22:33:02.350 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-STANDARD-STANDARD-1048576] ERROR [ 87%] 2026-02-24T22:33:02.578 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-STANDARD-STANDARD-8388608] ERROR [ 87%] 2026-02-24T22:33:02.806 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-STANDARD-LUKEWARM-1] ERROR [ 87%] 2026-02-24T22:33:03.035 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-STANDARD-LUKEWARM-1024] ERROR [ 87%] 2026-02-24T22:33:03.257 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-STANDARD-LUKEWARM-1048576] ERROR [ 87%] 2026-02-24T22:33:03.486 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-STANDARD-LUKEWARM-8388608] ERROR [ 87%] 2026-02-24T22:33:03.714 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-STANDARD-FROZEN-1] ERROR [ 87%] 2026-02-24T22:33:03.990 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-STANDARD-FROZEN-1024] ERROR [ 87%] 2026-02-24T22:33:04.224 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-STANDARD-FROZEN-1048576] ERROR [ 87%] 2026-02-24T22:33:04.450 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-STANDARD-FROZEN-8388608] ERROR [ 88%] 2026-02-24T22:33:04.679 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-LUKEWARM-STANDARD-1] ERROR [ 88%] 2026-02-24T22:33:04.950 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-LUKEWARM-STANDARD-1024] ERROR [ 88%] 2026-02-24T22:33:05.180 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-LUKEWARM-STANDARD-1048576] ERROR [ 88%] 2026-02-24T22:33:05.407 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-LUKEWARM-STANDARD-8388608] ERROR [ 88%] 2026-02-24T22:33:05.635 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-LUKEWARM-LUKEWARM-1] ERROR [ 88%] 2026-02-24T22:33:05.916 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-LUKEWARM-LUKEWARM-1024] ERROR [ 88%] 2026-02-24T22:33:06.144 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-LUKEWARM-LUKEWARM-1048576] ERROR [ 88%] 2026-02-24T22:33:06.372 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-LUKEWARM-LUKEWARM-8388608] ERROR [ 88%] 2026-02-24T22:33:06.648 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-LUKEWARM-FROZEN-1] ERROR [ 88%] 2026-02-24T22:33:06.874 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-LUKEWARM-FROZEN-1024] ERROR [ 88%] 2026-02-24T22:33:07.103 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-LUKEWARM-FROZEN-1048576] ERROR [ 89%] 2026-02-24T22:33:07.331 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-LUKEWARM-FROZEN-8388608] ERROR [ 89%] 2026-02-24T22:33:07.611 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-FROZEN-STANDARD-1] ERROR [ 89%] 2026-02-24T22:33:07.840 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-FROZEN-STANDARD-1024] ERROR [ 89%] 2026-02-24T22:33:08.068 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-FROZEN-STANDARD-1048576] ERROR [ 89%] 2026-02-24T22:33:08.350 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-FROZEN-STANDARD-8388608] ERROR [ 89%] 2026-02-24T22:33:08.579 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-FROZEN-LUKEWARM-1] ERROR [ 89%] 2026-02-24T22:33:08.806 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-FROZEN-LUKEWARM-1024] ERROR [ 89%] 2026-02-24T22:33:09.034 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-FROZEN-LUKEWARM-1048576] ERROR [ 89%] 2026-02-24T22:33:09.308 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-FROZEN-LUKEWARM-8388608] ERROR [ 89%] 2026-02-24T22:33:09.532 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-FROZEN-FROZEN-1] ERROR [ 89%] 2026-02-24T22:33:09.760 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-FROZEN-FROZEN-1024] ERROR [ 89%] 2026-02-24T22:33:09.985 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-FROZEN-FROZEN-1048576] ERROR [ 90%] 2026-02-24T22:33:10.267 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-FROZEN-FROZEN-8388608] ERROR [ 90%] 2026-02-24T22:33:10.494 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-STANDARD-STANDARD-1] ERROR [ 90%] 2026-02-24T22:33:10.722 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-STANDARD-STANDARD-1024] ERROR [ 90%] 2026-02-24T22:33:11.001 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-STANDARD-STANDARD-1048576] ERROR [ 90%] 2026-02-24T22:33:11.226 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-STANDARD-STANDARD-8388608] ERROR [ 90%] 2026-02-24T22:33:11.454 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-STANDARD-LUKEWARM-1] ERROR [ 90%] 2026-02-24T22:33:11.683 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-STANDARD-LUKEWARM-1024] ERROR [ 90%] 2026-02-24T22:33:11.967 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-STANDARD-LUKEWARM-1048576] ERROR [ 90%] 2026-02-24T22:33:12.194 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-STANDARD-LUKEWARM-8388608] ERROR [ 90%] 2026-02-24T22:33:12.422 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-STANDARD-FROZEN-1] ERROR [ 90%] 2026-02-24T22:33:12.704 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-STANDARD-FROZEN-1024] ERROR [ 91%] 2026-02-24T22:33:12.930 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-STANDARD-FROZEN-1048576] ERROR [ 91%] 2026-02-24T22:33:13.158 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-STANDARD-FROZEN-8388608] ERROR [ 91%] 2026-02-24T22:33:13.387 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-LUKEWARM-STANDARD-1] ERROR [ 91%] 2026-02-24T22:33:13.659 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-LUKEWARM-STANDARD-1024] ERROR [ 91%] 2026-02-24T22:33:13.888 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-LUKEWARM-STANDARD-1048576] ERROR [ 91%] 2026-02-24T22:33:14.114 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-LUKEWARM-STANDARD-8388608] ERROR [ 91%] 2026-02-24T22:33:14.342 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-LUKEWARM-LUKEWARM-1] ERROR [ 91%] 2026-02-24T22:33:14.623 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-LUKEWARM-LUKEWARM-1024] ERROR [ 91%] 2026-02-24T22:33:14.856 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-LUKEWARM-LUKEWARM-1048576] ERROR [ 91%] 2026-02-24T22:33:15.081 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-LUKEWARM-LUKEWARM-8388608] ERROR [ 91%] 2026-02-24T22:33:15.361 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-LUKEWARM-FROZEN-1] ERROR [ 92%] 2026-02-24T22:33:15.590 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-LUKEWARM-FROZEN-1024] ERROR [ 92%] 2026-02-24T22:33:15.818 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-LUKEWARM-FROZEN-1048576] ERROR [ 92%] 2026-02-24T22:33:16.047 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-LUKEWARM-FROZEN-8388608] ERROR [ 92%] 2026-02-24T22:33:16.330 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-FROZEN-STANDARD-1] ERROR [ 92%] 2026-02-24T22:33:16.558 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-FROZEN-STANDARD-1024] ERROR [ 92%] 2026-02-24T22:33:16.788 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-FROZEN-STANDARD-1048576] ERROR [ 92%] 2026-02-24T22:33:17.064 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-FROZEN-STANDARD-8388608] ERROR [ 92%] 2026-02-24T22:33:17.290 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-FROZEN-LUKEWARM-1] ERROR [ 92%] 2026-02-24T22:33:17.518 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-FROZEN-LUKEWARM-1024] ERROR [ 92%] 2026-02-24T22:33:17.748 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-FROZEN-LUKEWARM-1048576] ERROR [ 92%] 2026-02-24T22:33:18.019 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-FROZEN-LUKEWARM-8388608] ERROR [ 92%] 2026-02-24T22:33:18.247 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-FROZEN-FROZEN-1] ERROR [ 93%] 2026-02-24T22:33:18.476 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-FROZEN-FROZEN-1024] ERROR [ 93%] 2026-02-24T22:33:18.701 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-FROZEN-FROZEN-1048576] ERROR [ 93%] 2026-02-24T22:33:18.980 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-FROZEN-FROZEN-8388608] ERROR [ 93%] 2026-02-24T22:33:19.210 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-STANDARD-STANDARD-1] ERROR [ 93%] 2026-02-24T22:33:19.440 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-STANDARD-STANDARD-1024] ERROR [ 93%] 2026-02-24T22:33:19.716 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-STANDARD-STANDARD-1048576] ERROR [ 93%] 2026-02-24T22:33:19.944 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-STANDARD-STANDARD-8388608] ERROR [ 93%] 2026-02-24T22:33:20.175 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-STANDARD-LUKEWARM-1] ERROR [ 93%] 2026-02-24T22:33:20.403 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-STANDARD-LUKEWARM-1024] ERROR [ 93%] 2026-02-24T22:33:20.686 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-STANDARD-LUKEWARM-1048576] ERROR [ 93%] 2026-02-24T22:33:20.919 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-STANDARD-LUKEWARM-8388608] ERROR [ 94%] 2026-02-24T22:33:21.148 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-STANDARD-FROZEN-1] ERROR [ 94%] 2026-02-24T22:33:21.426 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-STANDARD-FROZEN-1024] ERROR [ 94%] 2026-02-24T22:33:21.654 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-STANDARD-FROZEN-1048576] ERROR [ 94%] 2026-02-24T22:33:21.884 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-STANDARD-FROZEN-8388608] ERROR [ 94%] 2026-02-24T22:33:22.110 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-LUKEWARM-STANDARD-1] ERROR [ 94%] 2026-02-24T22:33:22.383 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-LUKEWARM-STANDARD-1024] ERROR [ 94%] 2026-02-24T22:33:22.616 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-LUKEWARM-STANDARD-1048576] ERROR [ 94%] 2026-02-24T22:33:22.842 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-LUKEWARM-STANDARD-8388608] ERROR [ 94%] 2026-02-24T22:33:23.069 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-LUKEWARM-LUKEWARM-1] ERROR [ 94%] 2026-02-24T22:33:23.350 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-LUKEWARM-LUKEWARM-1024] ERROR [ 94%] 2026-02-24T22:33:23.579 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-LUKEWARM-LUKEWARM-1048576] ERROR [ 95%] 2026-02-24T22:33:23.806 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-LUKEWARM-LUKEWARM-8388608] ERROR [ 95%] 2026-02-24T22:33:24.078 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-LUKEWARM-FROZEN-1] ERROR [ 95%] 2026-02-24T22:33:24.305 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-LUKEWARM-FROZEN-1024] ERROR [ 95%] 2026-02-24T22:33:24.530 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-LUKEWARM-FROZEN-1048576] ERROR [ 95%] 2026-02-24T22:33:24.759 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-LUKEWARM-FROZEN-8388608] ERROR [ 95%] 2026-02-24T22:33:25.044 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-FROZEN-STANDARD-1] ERROR [ 95%] 2026-02-24T22:33:25.272 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-FROZEN-STANDARD-1024] ERROR [ 95%] 2026-02-24T22:33:25.503 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-FROZEN-STANDARD-1048576] ERROR [ 95%] 2026-02-24T22:33:25.780 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-FROZEN-STANDARD-8388608] ERROR [ 95%] 2026-02-24T22:33:26.005 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-FROZEN-LUKEWARM-1] ERROR [ 95%] 2026-02-24T22:33:26.230 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-FROZEN-LUKEWARM-1024] ERROR [ 96%] 2026-02-24T22:33:26.458 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-FROZEN-LUKEWARM-1048576] ERROR [ 96%] 2026-02-24T22:33:26.731 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-FROZEN-LUKEWARM-8388608] ERROR [ 96%] 2026-02-24T22:33:26.959 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-FROZEN-FROZEN-1] ERROR [ 96%] 2026-02-24T22:33:27.186 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-FROZEN-FROZEN-1024] ERROR [ 96%] 2026-02-24T22:33:27.414 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-FROZEN-FROZEN-1048576] ERROR [ 96%] 2026-02-24T22:33:27.696 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-FROZEN-FROZEN-8388608] ERROR [ 96%] 2026-02-24T22:33:33.329 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_generate_where_clause PASSED [ 96%] 2026-02-24T22:33:38.897 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_generate_projection PASSED [ 96%] 2026-02-24T22:33:39.585 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_count_operation PASSED [ 96%] 2026-02-24T22:33:40.389 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_count_json_operation PASSED [ 96%] 2026-02-24T22:33:41.800 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_json_column_sum_min_max PASSED [ 96%] 2026-02-24T22:33:43.061 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_json_nullif_expressions PASSED [ 97%] 2026-02-24T22:33:44.257 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_column_sum_min_max PASSED [ 97%] 2026-02-24T22:33:45.221 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_nullif_expressions PASSED [ 97%] 2026-02-24T22:33:46.101 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_nulliftrue_expressions PASSED [ 97%] 2026-02-24T22:33:46.941 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_is_not_null_expressions PASSED [ 97%] 2026-02-24T22:33:47.585 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_lowerupper_expressions PASSED [ 97%] 2026-02-24T22:33:48.721 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_in_expressions PASSED [ 97%] 2026-02-24T22:33:50.005 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_true_false_in_expressions PASSED [ 97%] 2026-02-24T22:33:51.737 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_like_expressions PASSED [ 97%] 2026-02-24T22:33:52.936 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_truefalselike_expressions PASSED [ 97%] 2026-02-24T22:33:53.845 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_complex_expressions PASSED [ 97%] 2026-02-24T22:33:54.565 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_alias PASSED [ 98%] 2026-02-24T22:33:55.233 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_alias_cyclic_refernce PASSED [ 98%] 2026-02-24T22:33:56.157 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_datetime PASSED [ 98%] 2026-02-24T22:33:57.025 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_true_false_datetime PASSED [ 98%] 2026-02-24T22:33:57.941 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_csv_parser PASSED [ 98%] 2026-02-24T22:33:58.729 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_csv_definition PASSED [ 98%] 2026-02-24T22:33:59.817 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_schema_definition PASSED [ 98%] 2026-02-24T22:34:00.685 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_when_then_else_expressions PASSED [ 98%] 2026-02-24T22:34:01.489 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_coalesce_expressions PASSED [ 98%] 2026-02-24T22:34:02.401 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_cast_expressions PASSED [ 98%] 2026-02-24T22:34:02.829 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_version PASSED [ 98%] 2026-02-24T22:34:03.933 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_trim_expressions PASSED [ 99%] 2026-02-24T22:34:05.049 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_truefalse_trim_expressions PASSED [ 99%] 2026-02-24T22:34:05.873 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_escape_expressions PASSED [ 99%] 2026-02-24T22:34:06.613 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_case_value_expressions PASSED [ 99%] 2026-02-24T22:34:07.297 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_bool_cast_expressions PASSED [ 99%] 2026-02-24T22:34:12.841 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_progress_expressions PASSED [ 99%] 2026-02-24T22:34:13.185 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py::test_output_serial_expressions PASSED [ 99%] 2026-02-24T22:34:13.807 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_sns.py::test_account_topic PASSED [ 99%] 2026-02-24T22:34:14.391 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_sns.py::test_cross_account_topic PASSED [ 99%] 2026-02-24T22:34:14.820 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_sns.py::test_account_topic_publish PASSED [ 99%] 2026-02-24T22:34:15.423 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_sns.py::test_cross_account_topic_publish PASSED [ 99%] 2026-02-24T22:34:15.440 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_utils.py::test_generate PASSED [100%] 2026-02-24T22:34:15.440 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.440 INFO:teuthology.orchestra.run.trial046.stdout:==================================== ERRORS ==================================== 2026-02-24T22:34:15.441 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at teardown of test_versioned_concurrent_object_create_and_remove ____ 2026-02-24T22:34:15.441 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.441 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.441 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.441 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.441 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.441 INFO:teuthology.orchestra.run.trial046.stdout: setup() 2026-02-24T22:34:15.441 INFO:teuthology.orchestra.run.trial046.stdout: yield 2026-02-24T22:34:15.441 INFO:teuthology.orchestra.run.trial046.stdout:> teardown() 2026-02-24T22:34:15.441 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.441 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:344: 2026-02-24T22:34:15.441 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.442 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:313: in teardown 2026-02-24T22:34:15.442 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.442 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.442 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.442 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.442 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.442 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.442 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.442 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.442 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.442 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.442 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.443 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.443 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.443 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.443 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.443 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.443 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.443 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.443 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.443 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.443 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.443 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.443 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.444 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.444 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.444 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.444 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.444 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.444 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.444 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.444 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.444 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.444 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.444 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.444 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.444 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.445 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.445 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.445 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.445 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.445 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.445 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.445 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.445 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.445 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.445 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.445 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.445 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.446 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.446 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.446 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.446 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.446 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.446 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.446 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.446 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.446 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.446 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.446 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.446 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.446 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.447 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.447 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.447 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.447 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.447 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.447 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.447 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.447 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.447 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.447 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.447 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.447 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.447 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.448 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.448 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.448 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.448 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.448 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.448 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.448 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.448 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.448 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.448 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.448 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.448 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.448 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.449 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.449 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.449 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.449 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.449 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.449 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.449 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.449 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.449 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.449 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.449 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.449 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.449 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.450 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.450 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.450 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.450 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.450 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.450 INFO:teuthology.orchestra.run.trial046.stdout:---------------------------- Captured stdout setup ----------------------------- 2026-02-24T22:34:15.450 INFO:teuthology.orchestra.run.trial046.stdout:Done with cleanup of buckets in tests. 2026-02-24T22:34:15.450 INFO:teuthology.orchestra.run.trial046.stdout:Done with cleanup of buckets in tests. 2026-02-24T22:34:15.450 INFO:teuthology.orchestra.run.trial046.stdout:Done with cleanup of buckets in tests. 2026-02-24T22:34:15.450 INFO:teuthology.orchestra.run.trial046.stdout:_____________________ ERROR at setup of test_lifecycle_set _____________________ 2026-02-24T22:34:15.450 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.450 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.450 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.451 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.451 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.451 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.451 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.451 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.451 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.451 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.451 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.451 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.451 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.451 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.451 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.452 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.452 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.452 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.452 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.452 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.452 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.452 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.452 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.452 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.452 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.452 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.452 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.452 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.453 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.453 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.453 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.453 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.453 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.453 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.453 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.453 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.453 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.453 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.453 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.453 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.453 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.454 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.454 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.454 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.454 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.454 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.454 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.454 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.454 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.454 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.454 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.454 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.454 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.454 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.455 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.455 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.455 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.455 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.455 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.455 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.455 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.455 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.455 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.455 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.455 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.455 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.455 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.456 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.456 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.456 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.456 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.456 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.456 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.456 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.456 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.456 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.456 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.456 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.456 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.457 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.463 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.463 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.464 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.464 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.464 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.464 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.464 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.464 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.464 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.464 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.464 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.464 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.464 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.465 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.465 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.465 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.465 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.465 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.465 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.465 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.465 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.465 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.465 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.465 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.465 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.466 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.466 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.466 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.466 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.466 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.466 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.466 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.466 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.466 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.466 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.466 INFO:teuthology.orchestra.run.trial046.stdout:_____________________ ERROR at setup of test_lifecycle_get _____________________ 2026-02-24T22:34:15.466 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.467 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.467 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.467 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.467 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.467 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.467 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.467 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.467 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.467 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.467 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.467 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.467 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.468 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.468 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.468 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.468 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.468 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.468 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.468 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.468 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.468 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.468 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.468 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.468 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.469 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.469 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.469 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.469 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.469 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.469 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.469 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.469 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.469 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.469 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.469 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.469 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.469 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.470 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.470 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.470 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.470 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.470 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.470 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.470 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.470 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.470 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.470 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.470 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.470 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.471 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.471 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.471 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.471 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.471 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.471 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.471 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.471 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.471 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.471 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.471 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.471 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.471 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.472 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.472 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.472 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.472 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.472 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.472 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.472 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.472 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.472 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.472 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.472 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.472 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.472 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.473 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.473 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.473 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.473 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.473 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.473 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.473 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.473 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.473 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.473 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.473 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.473 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.473 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.474 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.474 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.474 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.474 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.474 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.474 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.474 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.474 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.474 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.474 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.474 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.474 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.475 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.475 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.475 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.475 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.475 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.475 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.475 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.475 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.475 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.475 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.475 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.475 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.475 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.475 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.476 INFO:teuthology.orchestra.run.trial046.stdout:___________________ ERROR at setup of test_lifecycle_delete ____________________ 2026-02-24T22:34:15.476 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.476 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.476 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.476 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.476 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.476 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.476 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.476 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.476 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.476 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.476 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.476 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.476 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.477 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.477 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.477 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.477 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.477 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.477 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.477 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.477 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.477 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.477 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.477 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.477 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.477 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.477 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.478 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.478 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.478 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.478 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.478 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.478 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.478 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.478 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.478 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.478 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.478 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.478 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.478 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.478 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.479 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.479 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.479 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.479 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.479 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.479 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.479 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.479 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.479 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.479 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.479 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.479 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.479 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.479 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.480 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.480 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.480 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.480 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.480 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.480 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.480 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.480 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.480 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.480 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.480 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.480 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.480 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.480 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.480 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.480 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.480 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.481 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.481 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.481 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.481 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.481 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.481 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.481 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.481 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.481 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.481 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.481 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.481 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.481 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.481 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.481 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.482 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.482 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.482 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.482 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.482 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.482 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.482 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.482 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.482 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.482 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.482 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.482 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.482 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.482 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.482 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.482 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.483 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.483 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.483 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.483 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.483 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.483 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.483 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.483 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.483 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.483 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.483 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.483 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.483 INFO:teuthology.orchestra.run.trial046.stdout:__________________ ERROR at setup of test_lifecycle_get_no_id __________________ 2026-02-24T22:34:15.483 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.483 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.484 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.484 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.484 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.484 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.484 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.484 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.484 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.484 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.484 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.484 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.484 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.484 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.484 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.484 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.484 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.484 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.485 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.485 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.485 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.485 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.485 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.485 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.485 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.485 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.485 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.485 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.485 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.485 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.485 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.485 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.485 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.486 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.486 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.486 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.486 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.486 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.486 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.486 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.486 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.486 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.486 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.486 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.486 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.486 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.486 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.486 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.486 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.486 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.486 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.487 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.487 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.487 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.487 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.487 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.487 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.487 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.487 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.487 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.487 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.487 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.487 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.487 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.487 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.487 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.487 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.487 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.487 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.488 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.488 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.488 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.488 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.488 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.488 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.488 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.488 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.488 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.488 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.488 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.488 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.488 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.488 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.488 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.488 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.488 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.489 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.489 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.489 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.489 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.489 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.489 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.489 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.489 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.489 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.489 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.489 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.489 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.489 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.489 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.489 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.489 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.489 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.489 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.490 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.490 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.490 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.490 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.490 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.490 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.490 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.490 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.490 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.490 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.490 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.490 INFO:teuthology.orchestra.run.trial046.stdout:_________________ ERROR at setup of test_lifecycle_id_too_long _________________ 2026-02-24T22:34:15.490 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.490 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.490 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.490 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.490 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.490 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.490 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.491 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.491 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.491 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.491 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.491 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.491 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.491 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.491 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.491 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.491 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.491 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.491 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.491 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.491 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.491 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.491 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.491 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.491 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.491 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.491 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.491 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.492 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.492 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.492 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.492 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.492 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.492 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.492 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.492 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.492 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.492 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.492 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.492 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.492 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.492 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.492 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.492 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.492 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.492 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.492 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.492 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.493 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.493 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.493 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.493 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.493 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.493 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.493 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.493 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.493 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.493 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.493 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.493 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.493 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.493 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.493 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.493 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.493 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.493 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.493 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.493 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.494 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.494 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.494 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.494 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.494 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.494 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.494 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.494 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.494 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.494 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.494 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.494 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.494 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.494 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.494 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.494 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.494 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.494 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.494 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.494 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.494 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.495 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.495 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.495 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.495 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.495 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.495 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.495 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.495 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.495 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.495 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.495 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.495 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.495 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.495 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.495 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.495 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.495 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.495 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.495 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.495 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.495 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.495 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.495 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.496 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.496 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.496 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.496 INFO:teuthology.orchestra.run.trial046.stdout:___________________ ERROR at setup of test_lifecycle_same_id ___________________ 2026-02-24T22:34:15.496 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.496 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.496 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.496 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.496 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.496 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.496 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.496 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.496 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.496 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.496 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.496 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.496 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.496 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.496 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.496 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.496 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.497 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.497 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.497 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.497 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.497 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.497 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.497 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.497 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.497 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.497 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.497 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.497 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.497 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.497 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.497 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.497 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.497 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.497 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.497 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.497 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.497 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.497 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.497 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.498 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.498 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.498 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.498 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.498 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.498 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.498 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.498 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.498 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.498 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.498 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.498 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.498 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.498 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.498 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.498 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.498 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.498 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.498 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.498 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.498 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.498 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.499 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.499 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.499 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.499 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.499 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.499 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.499 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.499 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.499 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.499 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.499 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.499 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.499 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.499 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.499 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.499 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.499 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.499 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.499 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.499 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.499 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.499 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.499 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.499 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.500 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.500 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.500 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.500 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.500 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.500 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.500 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.500 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.500 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.500 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.500 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.500 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.500 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.500 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.500 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.500 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.500 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.500 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.500 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.500 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.500 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.500 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.500 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.500 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.501 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.501 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.501 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.501 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.501 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.501 INFO:teuthology.orchestra.run.trial046.stdout:_______________ ERROR at setup of test_lifecycle_invalid_status ________________ 2026-02-24T22:34:15.501 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.501 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.501 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.501 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.501 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.501 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.501 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.501 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.501 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.501 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.501 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.501 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.501 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.501 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.501 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.501 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.501 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.501 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.502 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.502 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.502 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.502 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.502 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.502 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.502 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.502 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.502 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.502 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.502 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.502 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.502 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.502 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.502 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.502 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.502 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.502 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.502 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.502 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.502 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.502 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.502 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.502 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.502 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.503 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.504 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.504 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.504 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.504 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.504 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.504 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.504 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.504 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.504 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.504 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.504 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.504 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.504 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.504 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.504 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.504 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.504 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.504 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.504 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.504 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.504 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.504 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.504 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.504 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.504 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.504 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout:__________________ ERROR at setup of test_lifecycle_set_date ___________________ 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.505 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.506 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.507 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.508 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout:______________ ERROR at setup of test_lifecycle_set_invalid_date _______________ 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.509 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.510 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.511 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.512 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout:_______________ ERROR at setup of test_lifecycle_set_noncurrent ________________ 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.513 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.514 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.515 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.516 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout:______________ ERROR at setup of test_lifecycle_set_deletemarker _______________ 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.517 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.518 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.519 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout:_________________ ERROR at setup of test_lifecycle_set_filter __________________ 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.520 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.521 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.522 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout:______________ ERROR at setup of test_lifecycle_set_empty_filter _______________ 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.523 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.524 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.525 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout:________________ ERROR at setup of test_lifecycle_set_multipart ________________ 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.526 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.527 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.528 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout:_________ ERROR at setup of test_lifecycle_transition_set_invalid_date _________ 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.529 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.530 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.531 INFO:teuthology.orchestra.run.trial046.stdout:_______________ ERROR at setup of test_restore_object_temporary ________________ 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.532 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.533 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout:_______________ ERROR at setup of test_restore_object_permanent ________________ 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.534 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.535 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.536 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout:_____________________ ERROR at setup of test_read_through ______________________ 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.537 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.538 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout:__________________ ERROR at setup of test_restore_noncur_obj ___________________ 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.539 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.540 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.541 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout:_________________ ERROR at setup of test_encrypted_transfer_1b _________________ 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.542 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.543 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout:________________ ERROR at setup of test_encrypted_transfer_1kb _________________ 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.544 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.545 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.546 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout:________________ ERROR at setup of test_encrypted_transfer_1MB _________________ 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.547 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.548 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout:________________ ERROR at setup of test_encrypted_transfer_13b _________________ 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.549 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.550 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.551 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout:_____________ ERROR at setup of test_encryption_sse_c_method_head ______________ 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.552 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.553 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout:_______________ ERROR at setup of test_encryption_sse_c_present ________________ 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.554 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.555 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.556 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout:______________ ERROR at setup of test_encryption_sse_c_other_key _______________ 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.557 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.558 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout:_____________ ERROR at setup of test_encryption_sse_c_invalid_md5 ______________ 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.559 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.560 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.561 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout:________________ ERROR at setup of test_encryption_sse_c_no_md5 ________________ 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.562 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.563 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout:________________ ERROR at setup of test_encryption_sse_c_no_key ________________ 2026-02-24T22:34:15.564 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.565 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.566 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout:________________ ERROR at setup of test_encryption_key_no_sse_c ________________ 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.567 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.568 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout:___________ ERROR at setup of test_encryption_sse_c_multipart_upload ___________ 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.569 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.570 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.571 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_encryption_sse_c_unaligned_multipart_upload ______ 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.572 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.573 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.574 INFO:teuthology.orchestra.run.trial046.stdout:________ ERROR at setup of test_encryption_sse_c_multipart_bad_download ________ 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.575 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.576 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_encryption_sse_c_post_object_authenticated_request ___ 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.577 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.578 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.579 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_encryption_sse_c_enforced_with_bucket_policy ______ 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.580 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.581 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_encryption_sse_c_deny_algo_with_bucket_policy _____ 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.582 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.583 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.584 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout:__________________ ERROR at setup of test_sse_kms_method_head __________________ 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.585 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.586 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout:____________________ ERROR at setup of test_sse_kms_present ____________________ 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.587 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.588 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.589 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout:____________________ ERROR at setup of test_sse_kms_no_key _____________________ 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.590 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.591 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout:_________________ ERROR at setup of test_sse_kms_not_declared __________________ 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.592 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.593 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.594 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout:_______________ ERROR at setup of test_sse_kms_multipart_upload ________________ 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.595 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.596 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout:__________ ERROR at setup of test_sse_kms_multipart_invalid_chunks_1 ___________ 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.597 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.598 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.599 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout:__________ ERROR at setup of test_sse_kms_multipart_invalid_chunks_2 ___________ 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.600 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.601 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout:_______ ERROR at setup of test_sse_kms_post_object_authenticated_request _______ 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.602 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.603 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.604 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout:__________________ ERROR at setup of test_sse_kms_transfer_1b __________________ 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.605 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.606 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout:_________________ ERROR at setup of test_sse_kms_transfer_1kb __________________ 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.607 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.608 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.609 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout:_________________ ERROR at setup of test_sse_kms_transfer_1MB __________________ 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.610 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.611 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout:_________________ ERROR at setup of test_sse_kms_transfer_13b __________________ 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.612 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.613 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.614 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout:_________________ ERROR at setup of test_sse_kms_read_declare __________________ 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.615 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.616 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout:_____________________ ERROR at setup of test_bucket_policy _____________________ 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.617 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.618 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.619 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout:____________________ ERROR at setup of test_bucketv2_policy ____________________ 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.620 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.621 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout:_________ ERROR at setup of test_bucket_policy_deny_self_denied_policy _________ 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.622 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.623 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.624 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_bucket_policy_deny_self_denied_policy_confirm_header __ 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.625 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.626 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout:___________________ ERROR at setup of test_bucket_policy_acl ___________________ 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.627 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.628 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.629 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout:__________________ ERROR at setup of test_bucketv2_policy_acl __________________ 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.630 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.631 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout:____________ ERROR at setup of test_bucket_policy_different_tenant _____________ 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.632 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.633 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.634 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout:________________ ERROR at setup of test_bucket_policy_multipart ________________ 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.635 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.636 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout:_____________ ERROR at setup of test_bucket_policy_tenanted_bucket _____________ 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.637 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.638 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.639 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout:_____________ ERROR at setup of test_bucket_policy_another_bucket ______________ 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.640 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.641 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout:____________ ERROR at setup of test_bucketv2_policy_another_bucket _____________ 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.642 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.643 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.644 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout:_______________ ERROR at setup of test_set_get_del_bucket_policy _______________ 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.645 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.646 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout:_________________ ERROR at setup of test_set_multipart_tagging _________________ 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.647 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.648 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.649 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout:____________________ ERROR at setup of test_get_obj_tagging ____________________ 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.650 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.651 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout:_________________ ERROR at setup of test_get_obj_head_tagging __________________ 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.652 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.653 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.654 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout:_____________________ ERROR at setup of test_put_max_tags ______________________ 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.655 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.656 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout:____________________ ERROR at setup of test_put_excess_tags ____________________ 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.657 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.658 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.659 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout:__________________ ERROR at setup of test_put_max_kvsize_tags __________________ 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.660 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.661 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout:__________________ ERROR at setup of test_put_excess_key_tags __________________ 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.662 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.663 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.664 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout:__________________ ERROR at setup of test_put_excess_val_tags __________________ 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.665 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.666 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout:____________________ ERROR at setup of test_put_modify_tags ____________________ 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.667 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.668 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.669 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout:____________________ ERROR at setup of test_put_delete_tags ____________________ 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.670 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.671 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout:__________ ERROR at setup of test_post_object_tags_anonymous_request ___________ 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.672 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.673 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.674 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout:________ ERROR at setup of test_post_object_tags_authenticated_request _________ 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.675 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.676 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout:___________________ ERROR at setup of test_put_obj_with_tags ___________________ 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.677 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.678 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.679 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout:__________________ ERROR at setup of test_get_tags_acl_public __________________ 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.680 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.681 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout:__________________ ERROR at setup of test_put_tags_acl_public __________________ 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.682 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.683 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.684 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout:________________ ERROR at setup of test_delete_tags_obj_public _________________ 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.685 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.686 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_versioning_bucket_atomic_upload_return_version_id ___ 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.687 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.688 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.689 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_versioning_bucket_multipart_upload_return_version_id __ 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.690 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.691 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout:__________ ERROR at setup of test_bucket_policy_get_obj_existing_tag ___________ 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.692 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.693 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.694 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_bucket_policy_get_obj_tagging_existing_tag _______ 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.695 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.696 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_bucket_policy_put_obj_tagging_existing_tag _______ 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.697 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.698 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.699 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout:____________ ERROR at setup of test_bucket_policy_upload_part_copy _____________ 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.700 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.701 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.701 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.701 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.701 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.707 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.708 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout:___________ ERROR at setup of test_bucket_policy_put_obj_copy_source ___________ 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.709 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.710 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout:________ ERROR at setup of test_bucket_policy_put_obj_copy_source_meta _________ 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.711 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.712 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.713 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout:_______________ ERROR at setup of test_bucket_policy_put_obj_acl _______________ 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.714 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.715 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout:______________ ERROR at setup of test_bucket_policy_put_obj_grant ______________ 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.716 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.717 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.718 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout:_______________ ERROR at setup of test_put_obj_enc_conflict_c_s3 _______________ 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.719 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.720 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout:______________ ERROR at setup of test_put_obj_enc_conflict_c_kms _______________ 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.721 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.722 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.723 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout:______________ ERROR at setup of test_put_obj_enc_conflict_s3_kms ______________ 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.724 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.725 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.726 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout:___________ ERROR at setup of test_put_obj_enc_conflict_bad_enc_kms ____________ 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.727 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.728 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout:____________ ERROR at setup of test_bucket_policy_put_obj_kms_noenc ____________ 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.729 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.730 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.731 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout:_____________ ERROR at setup of test_bucket_policy_put_obj_kms_s3 ______________ 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.732 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.733 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout:________ ERROR at setup of test_bucket_policy_get_obj_acl_existing_tag _________ 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.734 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.735 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.736 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout:_______________ ERROR at setup of test_object_lock_put_obj_lock ________________ 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.737 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.738 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout:________ ERROR at setup of test_object_lock_put_obj_lock_invalid_bucket ________ 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.739 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.740 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.741 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_object_lock_put_obj_lock_enable_after_create ______ 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.742 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.743 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.744 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_object_lock_put_obj_lock_with_days_and_years ______ 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.745 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.746 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout:_________ ERROR at setup of test_object_lock_put_obj_lock_invalid_days _________ 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.747 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.748 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.749 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout:________ ERROR at setup of test_object_lock_put_obj_lock_invalid_years _________ 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.750 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.751 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout:_________ ERROR at setup of test_object_lock_put_obj_lock_invalid_mode _________ 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.752 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.753 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.754 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout:________ ERROR at setup of test_object_lock_put_obj_lock_invalid_status ________ 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.755 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.756 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.757 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout:____________ ERROR at setup of test_object_lock_suspend_versioning _____________ 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.758 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.759 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout:_______________ ERROR at setup of test_object_lock_get_obj_lock ________________ 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.760 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.761 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.762 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout:________ ERROR at setup of test_object_lock_get_obj_lock_invalid_bucket ________ 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.763 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.764 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout:_____________ ERROR at setup of test_object_lock_put_obj_retention _____________ 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.765 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.766 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.767 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_object_lock_put_obj_retention_invalid_bucket ______ 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.768 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.769 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_object_lock_put_obj_retention_invalid_mode _______ 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.770 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.771 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.772 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout:_____________ ERROR at setup of test_object_lock_get_obj_retention _____________ 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.773 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.774 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.775 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout:_________ ERROR at setup of test_object_lock_get_obj_retention_iso8601 _________ 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.776 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.777 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_object_lock_get_obj_retention_invalid_bucket ______ 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.778 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.779 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.780 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout:________ ERROR at setup of test_object_lock_put_obj_retention_versionid ________ 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.781 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.782 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_object_lock_put_obj_retention_override_default_retention _ 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.783 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.784 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.785 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_object_lock_put_obj_retention_increase_period _____ 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.786 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.787 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_object_lock_put_obj_retention_shorten_period ______ 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.788 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.789 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.790 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_object_lock_put_obj_retention_shorten_period_bypass __ 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.791 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.792 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.793 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout:_______ ERROR at setup of test_object_lock_delete_object_with_retention ________ 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.794 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.795 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_object_lock_delete_multipart_object_with_retention ___ 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.796 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.797 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.798 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_object_lock_delete_object_with_retention_and_marker __ 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.799 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.800 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_object_lock_multi_delete_object_with_retention _____ 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.801 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.802 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.803 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout:______________ ERROR at setup of test_object_lock_put_legal_hold _______________ 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.804 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.805 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout:_______ ERROR at setup of test_object_lock_put_legal_hold_invalid_bucket _______ 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.806 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.807 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.808 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout:_______ ERROR at setup of test_object_lock_put_legal_hold_invalid_status _______ 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.809 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.810 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.811 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout:______________ ERROR at setup of test_object_lock_get_legal_hold _______________ 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.812 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.813 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout:_______ ERROR at setup of test_object_lock_get_legal_hold_invalid_bucket _______ 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.814 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.815 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.816 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_object_lock_delete_object_with_legal_hold_on ______ 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.817 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.818 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_object_lock_delete_multipart_object_with_legal_hold_on _ 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.819 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.820 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.821 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_object_lock_delete_object_with_legal_hold_off _____ 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.822 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.823 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout:_____________ ERROR at setup of test_object_lock_get_obj_metadata ______________ 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.824 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.825 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.826 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout:_______________ ERROR at setup of test_object_lock_uploading_obj _______________ 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.827 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.828 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_object_lock_changing_mode_from_governance_with_bypass _ 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.829 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.830 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.831 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_object_lock_changing_mode_from_governance_without_bypass _ 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.832 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.833 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.834 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout:_______ ERROR at setup of test_object_lock_changing_mode_from_compliance _______ 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.835 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.836 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout:_______________ ERROR at setup of test_copy_object_ifmatch_good ________________ 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.837 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.838 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.839 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout:____________ ERROR at setup of test_copy_object_ifnonematch_failed _____________ 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.840 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.841 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout:_______________ ERROR at setup of test_get_bucket_policy_status ________________ 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.842 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.843 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.844 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout:__________ ERROR at setup of test_get_public_acl_bucket_policy_status __________ 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.845 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.846 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout:________ ERROR at setup of test_get_authpublic_acl_bucket_policy_status ________ 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.847 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.848 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.849 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout:_______ ERROR at setup of test_get_publicpolicy_acl_bucket_policy_status _______ 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.850 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.851 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_get_nonpublicpolicy_acl_bucket_policy_status ______ 2026-02-24T22:34:15.852 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.853 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.854 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_get_nonpublicpolicy_principal_bucket_policy_status ___ 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.855 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.856 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.857 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout:___________ ERROR at setup of test_bucket_policy_allow_notprincipal ____________ 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.858 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.859 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout:______________ ERROR at setup of test_get_undefined_public_block _______________ 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.860 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.861 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.862 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout:__________ ERROR at setup of test_get_public_block_deny_bucket_policy __________ 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.863 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.864 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout:___________________ ERROR at setup of test_put_public_block ____________________ 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.865 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.866 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.867 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout:_____________ ERROR at setup of test_block_public_put_bucket_acls ______________ 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.868 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.869 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout:____________ ERROR at setup of test_block_public_object_canned_acls ____________ 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.870 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.871 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.872 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout:__________________ ERROR at setup of test_block_public_policy __________________ 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.873 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.874 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout:__________ ERROR at setup of test_block_public_policy_with_principal ___________ 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.875 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.876 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.877 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout:_________ ERROR at setup of test_block_public_restrict_public_buckets __________ 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.878 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.879 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.880 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout:__________________ ERROR at setup of test_ignore_public_acls ___________________ 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.881 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.882 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout:______________ ERROR at setup of test_put_get_delete_public_block ______________ 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.883 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.884 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.885 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout:_______ ERROR at setup of test_multipart_upload_on_a_bucket_with_policy ________ 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.886 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.887 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout:_______________ ERROR at setup of test_put_bucket_encryption_kms _______________ 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.888 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.889 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.890 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout:_______________ ERROR at setup of test_get_bucket_encryption_kms _______________ 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.891 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.892 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout:_____________ ERROR at setup of test_delete_bucket_encryption_kms ______________ 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.893 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.894 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.895 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_sse_kms_default_post_object_authenticated_request ___ 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.896 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.897 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.898 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout:__________________ ERROR at setup of test_get_object_torrent ___________________ 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.899 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.900 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout:________________ ERROR at setup of test_object_checksum_sha256 _________________ 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.901 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.902 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.903 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout:_______________ ERROR at setup of test_object_checksum_crc64nvme _______________ 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.904 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.905 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout:_______________ ERROR at setup of test_multipart_checksum_sha256 _______________ 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.906 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.907 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.908 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout:___________ ERROR at setup of test_multipart_use_cksum_helper_sha256 ___________ 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.909 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.910 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout:_________ ERROR at setup of test_multipart_use_cksum_helper_crc64nvme __________ 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.911 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.912 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.913 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout:___________ ERROR at setup of test_multipart_use_cksum_helper_crc32 ____________ 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.914 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.915 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.916 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout:___________ ERROR at setup of test_multipart_use_cksum_helper_crc32c ___________ 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.917 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.918 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout:____________ ERROR at setup of test_multipart_use_cksum_helper_sha1 ____________ 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.919 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.920 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.921 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout:______________ ERROR at setup of test_post_object_upload_checksum ______________ 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.922 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.923 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout:_______ ERROR at setup of test_get_multipart_checksum_object_attributes ________ 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.924 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.925 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.926 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout:____________ ERROR at setup of test_get_multipart_object_attributes ____________ 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.927 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.928 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout:_______ ERROR at setup of test_get_paginated_multipart_object_attributes _______ 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.929 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.930 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.931 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout:________ ERROR at setup of test_get_single_multipart_object_attributes _________ 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.932 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.933 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout:____________ ERROR at setup of test_get_checksum_object_attributes _____________ 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.934 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.935 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.936 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout:____________ ERROR at setup of test_get_versioned_object_attributes ____________ 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.937 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.938 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.939 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout:_________ ERROR at setup of test_get_sse_c_encrypted_object_attributes _________ 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.940 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.941 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout:_________________ ERROR at setup of test_get_object_attributes _________________ 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.942 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.943 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.944 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout:_________ ERROR at setup of test_upload_part_copy_percent_encoded_key __________ 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.945 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.946 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout:______________ ERROR at setup of test_delete_marker_nonversioned _______________ 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.947 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.948 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.949 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout:________________ ERROR at setup of test_delete_marker_versioned ________________ 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.950 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.951 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout:________________ ERROR at setup of test_delete_marker_suspended ________________ 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.952 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.953 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.954 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout:__________________ ERROR at setup of test_put_object_if_match __________________ 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.955 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.956 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout:_____________ ERROR at setup of test_multipart_put_object_if_match _____________ 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.957 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.958 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.959 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout:___________ ERROR at setup of test_put_current_object_if_none_match ____________ 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.960 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.961 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.962 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_multipart_put_current_object_if_none_match _______ 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.963 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.964 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout:______________ ERROR at setup of test_put_current_object_if_match ______________ 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.965 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.966 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.967 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout:_________ ERROR at setup of test_multipart_put_current_object_if_match _________ 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.968 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.969 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout:______________ ERROR at setup of test_put_object_current_if_match ______________ 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.970 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.971 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.972 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout:________________ ERROR at setup of test_delete_object_if_match _________________ 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.973 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.974 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout:____________ ERROR at setup of test_delete_object_current_if_match _____________ 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.975 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.976 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.977 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout:____________ ERROR at setup of test_delete_object_version_if_match _____________ 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.978 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.979 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout:_______ ERROR at setup of test_delete_object_if_match_last_modified_time _______ 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.980 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.981 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.982 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_delete_object_current_if_match_last_modified_time ___ 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.983 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.984 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.985 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_delete_object_version_if_match_last_modified_time ___ 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.986 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.987 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout:______________ ERROR at setup of test_delete_object_if_match_size ______________ 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.988 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.989 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.990 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout:__________ ERROR at setup of test_delete_object_current_if_match_size __________ 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.991 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.992 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout:__________ ERROR at setup of test_delete_object_version_if_match_size __________ 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.993 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.994 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.995 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout:________________ ERROR at setup of test_delete_objects_if_match ________________ 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.996 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.997 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout:____________ ERROR at setup of test_delete_objects_current_if_match ____________ 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:15.998 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:15.999 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.000 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout:____________ ERROR at setup of test_delete_objects_version_if_match ____________ 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.001 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.002 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_delete_objects_if_match_last_modified_time _______ 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.003 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.004 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.005 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_delete_objects_current_if_match_last_modified_time ___ 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.006 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.007 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_delete_objects_version_if_match_last_modified_time ___ 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.008 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.009 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.010 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout:_____________ ERROR at setup of test_delete_objects_if_match_size ______________ 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.011 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.012 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout:_________ ERROR at setup of test_delete_objects_current_if_match_size __________ 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.013 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.014 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.015 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout:_________ ERROR at setup of test_delete_objects_version_if_match_size __________ 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.016 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.017 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.018 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout:__________ ERROR at setup of test_create_bucket_no_ownership_controls __________ 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.019 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.020 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout:__________ ERROR at setup of test_create_bucket_bucket_owner_enforced __________ 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.021 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.022 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.023 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout:_________ ERROR at setup of test_create_bucket_bucket_owner_preferred __________ 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.024 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.025 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout:______________ ERROR at setup of test_create_bucket_object_writer ______________ 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.026 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.027 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.028 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_put_bucket_ownership_bucket_owner_enforced _______ 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.029 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.030 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_put_bucket_ownership_bucket_owner_preferred ______ 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.031 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.032 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.033 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout:__________ ERROR at setup of test_put_bucket_ownership_object_writer ___________ 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.034 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.035 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout:_________ ERROR at setup of test_bucket_create_delete_bucket_ownership _________ 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.036 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.037 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.038 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-unencrypted-STANDARD-STANDARD-8388608] _ 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.039 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.040 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-unencrypted-STANDARD-LUKEWARM-8388608] _ 2026-02-24T22:34:16.041 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.042 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.043 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-unencrypted-STANDARD-FROZEN-8388608] _ 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.044 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.045 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.046 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-unencrypted-LUKEWARM-STANDARD-8388608] _ 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.047 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.048 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-unencrypted-LUKEWARM-LUKEWARM-8388608] _ 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.049 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.050 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.051 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-unencrypted-LUKEWARM-FROZEN-8388608] _ 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.052 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.053 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-unencrypted-FROZEN-STANDARD-8388608] _ 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.054 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.055 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.056 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-unencrypted-FROZEN-LUKEWARM-8388608] _ 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.057 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.058 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-unencrypted-FROZEN-FROZEN-8388608] _ 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.059 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.060 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.061 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-sse-c-STANDARD-STANDARD-8388608] _ 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.062 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.063 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-sse-c-STANDARD-LUKEWARM-8388608] _ 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.064 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.065 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.066 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-sse-c-STANDARD-FROZEN-8388608] _ 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.067 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.068 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-sse-c-LUKEWARM-STANDARD-8388608] _ 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.069 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.070 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.071 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-sse-c-LUKEWARM-LUKEWARM-8388608] _ 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.072 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.073 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-sse-c-LUKEWARM-FROZEN-8388608] _ 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.074 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.075 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.076 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-sse-c-FROZEN-STANDARD-8388608] _ 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.077 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.078 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.079 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-sse-c-FROZEN-LUKEWARM-8388608] _ 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.080 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.081 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-sse-c-FROZEN-FROZEN-8388608] _ 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.082 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.083 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.084 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-sse-kms-STANDARD-STANDARD-8388608] _ 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.085 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.086 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-sse-kms-STANDARD-LUKEWARM-8388608] _ 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.087 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.088 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.089 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-sse-kms-STANDARD-FROZEN-8388608] _ 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.090 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.091 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-sse-kms-LUKEWARM-STANDARD-8388608] _ 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.092 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.093 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.094 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-sse-kms-LUKEWARM-LUKEWARM-8388608] _ 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.095 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.096 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-sse-kms-LUKEWARM-FROZEN-8388608] _ 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.097 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.098 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.099 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-sse-kms-FROZEN-STANDARD-8388608] _ 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.100 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.101 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-sse-kms-FROZEN-LUKEWARM-8388608] _ 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.102 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.103 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.104 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[unencrypted-sse-kms-FROZEN-FROZEN-8388608] _ 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.105 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.106 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.107 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-c-unencrypted-STANDARD-STANDARD-8388608] _ 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.108 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.109 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-c-unencrypted-STANDARD-LUKEWARM-8388608] _ 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.110 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.111 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.112 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-c-unencrypted-STANDARD-FROZEN-8388608] _ 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.113 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.114 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-c-unencrypted-LUKEWARM-STANDARD-8388608] _ 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.115 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.116 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.117 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-c-unencrypted-LUKEWARM-LUKEWARM-8388608] _ 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.118 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.119 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-c-unencrypted-LUKEWARM-FROZEN-8388608] _ 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.120 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.121 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.122 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-c-unencrypted-FROZEN-STANDARD-8388608] _ 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.123 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.124 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-c-unencrypted-FROZEN-LUKEWARM-8388608] _ 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.125 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.126 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.127 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-c-unencrypted-FROZEN-FROZEN-8388608] _ 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.128 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.129 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-c-sse-c-STANDARD-STANDARD-8388608] __ 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.130 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.131 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.132 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-c-sse-c-STANDARD-LUKEWARM-8388608] __ 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.133 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.134 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.135 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_part_enc[sse-c-sse-c-STANDARD-FROZEN-8388608] ___ 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.136 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.137 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-c-sse-c-LUKEWARM-STANDARD-8388608] __ 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.138 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.139 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.140 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-c-sse-c-LUKEWARM-LUKEWARM-8388608] __ 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.141 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.142 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_part_enc[sse-c-sse-c-LUKEWARM-FROZEN-8388608] ___ 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.143 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.144 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.145 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_part_enc[sse-c-sse-c-FROZEN-STANDARD-8388608] ___ 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.146 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.147 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_part_enc[sse-c-sse-c-FROZEN-LUKEWARM-8388608] ___ 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.148 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.149 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.150 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_part_enc[sse-c-sse-c-FROZEN-FROZEN-8388608] ____ 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.151 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.152 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-c-sse-kms-STANDARD-STANDARD-8388608] _ 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.153 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.154 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.155 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-c-sse-kms-STANDARD-LUKEWARM-8388608] _ 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.156 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.157 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-c-sse-kms-STANDARD-FROZEN-8388608] __ 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.158 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.159 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.160 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-c-sse-kms-LUKEWARM-STANDARD-8388608] _ 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.161 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.162 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-c-sse-kms-LUKEWARM-LUKEWARM-8388608] _ 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.163 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.164 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.165 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-c-sse-kms-LUKEWARM-FROZEN-8388608] __ 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.166 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.167 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.168 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-c-sse-kms-FROZEN-STANDARD-8388608] __ 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.169 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.170 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-c-sse-kms-FROZEN-LUKEWARM-8388608] __ 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.171 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.172 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.173 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_part_enc[sse-c-sse-kms-FROZEN-FROZEN-8388608] ___ 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.174 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.175 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-kms-unencrypted-STANDARD-STANDARD-8388608] _ 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.176 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.177 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.178 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-kms-unencrypted-STANDARD-LUKEWARM-8388608] _ 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.179 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.180 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-kms-unencrypted-STANDARD-FROZEN-8388608] _ 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.181 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.182 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.183 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-kms-unencrypted-LUKEWARM-STANDARD-8388608] _ 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.184 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.185 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-kms-unencrypted-LUKEWARM-LUKEWARM-8388608] _ 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.186 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.187 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.188 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-kms-unencrypted-LUKEWARM-FROZEN-8388608] _ 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.189 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.190 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-kms-unencrypted-FROZEN-STANDARD-8388608] _ 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.191 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.192 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.193 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-kms-unencrypted-FROZEN-LUKEWARM-8388608] _ 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.194 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.195 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-kms-unencrypted-FROZEN-FROZEN-8388608] _ 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.196 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.197 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.198 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-kms-sse-c-STANDARD-STANDARD-8388608] _ 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.199 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.200 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.201 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-kms-sse-c-STANDARD-LUKEWARM-8388608] _ 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.202 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.203 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-kms-sse-c-STANDARD-FROZEN-8388608] __ 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.204 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.205 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.206 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-kms-sse-c-LUKEWARM-STANDARD-8388608] _ 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.207 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.208 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-kms-sse-c-LUKEWARM-LUKEWARM-8388608] _ 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.209 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.210 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.211 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-kms-sse-c-LUKEWARM-FROZEN-8388608] __ 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.212 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.213 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-kms-sse-c-FROZEN-STANDARD-8388608] __ 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.214 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.215 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.216 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-kms-sse-c-FROZEN-LUKEWARM-8388608] __ 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.217 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.218 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_part_enc[sse-kms-sse-c-FROZEN-FROZEN-8388608] ___ 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.219 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.220 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.221 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-kms-sse-kms-STANDARD-STANDARD-8388608] _ 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.222 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.223 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-kms-sse-kms-STANDARD-LUKEWARM-8388608] _ 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.224 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.225 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.226 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-kms-sse-kms-STANDARD-FROZEN-8388608] _ 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.227 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.228 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-kms-sse-kms-LUKEWARM-STANDARD-8388608] _ 2026-02-24T22:34:16.229 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.230 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.231 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-kms-sse-kms-LUKEWARM-LUKEWARM-8388608] _ 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.232 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.233 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.234 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-kms-sse-kms-LUKEWARM-FROZEN-8388608] _ 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.235 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.236 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-kms-sse-kms-FROZEN-STANDARD-8388608] _ 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.237 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.238 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.239 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-kms-sse-kms-FROZEN-LUKEWARM-8388608] _ 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.240 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.241 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_part_enc[sse-kms-sse-kms-FROZEN-FROZEN-8388608] __ 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.242 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.243 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.244 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-STANDARD-STANDARD-1] _ 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.245 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.246 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-STANDARD-STANDARD-1024] _ 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.247 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.248 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.249 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-STANDARD-STANDARD-1048576] _ 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.250 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.251 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-STANDARD-STANDARD-8388608] _ 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.252 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.253 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.254 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-STANDARD-LUKEWARM-1] _ 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.255 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.256 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.257 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-STANDARD-LUKEWARM-1024] _ 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.258 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.259 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-STANDARD-LUKEWARM-1048576] _ 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.260 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.261 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.262 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-STANDARD-LUKEWARM-8388608] _ 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.263 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.264 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[unencrypted-unencrypted-STANDARD-FROZEN-1] __ 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.265 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.266 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.267 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-STANDARD-FROZEN-1024] _ 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.268 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.269 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-STANDARD-FROZEN-1048576] _ 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.270 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.271 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.272 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-STANDARD-FROZEN-8388608] _ 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.273 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.274 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-LUKEWARM-STANDARD-1] _ 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.275 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.276 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.277 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-LUKEWARM-STANDARD-1024] _ 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.278 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.279 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-LUKEWARM-STANDARD-1048576] _ 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.280 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.281 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.282 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-LUKEWARM-STANDARD-8388608] _ 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.283 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.284 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-LUKEWARM-LUKEWARM-1] _ 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.285 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.286 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.287 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-LUKEWARM-LUKEWARM-1024] _ 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.288 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.289 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.290 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-LUKEWARM-LUKEWARM-1048576] _ 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.291 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.292 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-LUKEWARM-LUKEWARM-8388608] _ 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.293 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.294 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.295 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[unencrypted-unencrypted-LUKEWARM-FROZEN-1] __ 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.296 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.297 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-LUKEWARM-FROZEN-1024] _ 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.298 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.299 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.300 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-LUKEWARM-FROZEN-1048576] _ 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.301 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.302 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-LUKEWARM-FROZEN-8388608] _ 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.303 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.304 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.305 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[unencrypted-unencrypted-FROZEN-STANDARD-1] __ 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.306 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.307 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-FROZEN-STANDARD-1024] _ 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.308 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.309 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.310 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-FROZEN-STANDARD-1048576] _ 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.311 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.312 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-FROZEN-STANDARD-8388608] _ 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.313 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.314 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.315 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[unencrypted-unencrypted-FROZEN-LUKEWARM-1] __ 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.316 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.317 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-FROZEN-LUKEWARM-1024] _ 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.318 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.319 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.320 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-FROZEN-LUKEWARM-1048576] _ 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.321 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.322 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.323 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-FROZEN-LUKEWARM-8388608] _ 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.324 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.325 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[unencrypted-unencrypted-FROZEN-FROZEN-1] ___ 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.326 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.327 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.328 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-FROZEN-FROZEN-1024] __ 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.329 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.330 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-FROZEN-FROZEN-1048576] _ 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.331 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.332 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.333 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-unencrypted-FROZEN-FROZEN-8388608] _ 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.334 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.335 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[unencrypted-sse-c-STANDARD-STANDARD-1] ____ 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.336 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.337 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.338 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[unencrypted-sse-c-STANDARD-STANDARD-1024] ___ 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.339 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.340 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-c-STANDARD-STANDARD-1048576] _ 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.341 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.342 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.343 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-c-STANDARD-STANDARD-8388608] _ 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.344 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.345 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[unencrypted-sse-c-STANDARD-LUKEWARM-1] ____ 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.346 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.347 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.348 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[unencrypted-sse-c-STANDARD-LUKEWARM-1024] ___ 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.349 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.350 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-c-STANDARD-LUKEWARM-1048576] _ 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.351 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.352 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.353 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-c-STANDARD-LUKEWARM-8388608] _ 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.354 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.355 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[unencrypted-sse-c-STANDARD-FROZEN-1] _____ 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.356 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.357 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.358 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[unencrypted-sse-c-STANDARD-FROZEN-1024] ____ 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.359 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.360 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.361 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[unencrypted-sse-c-STANDARD-FROZEN-1048576] __ 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.362 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.363 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[unencrypted-sse-c-STANDARD-FROZEN-8388608] __ 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.364 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.365 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.366 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[unencrypted-sse-c-LUKEWARM-STANDARD-1] ____ 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.367 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.368 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[unencrypted-sse-c-LUKEWARM-STANDARD-1024] ___ 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.369 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.370 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.371 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-c-LUKEWARM-STANDARD-1048576] _ 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.372 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.373 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-c-LUKEWARM-STANDARD-8388608] _ 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.374 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.375 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.376 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[unencrypted-sse-c-LUKEWARM-LUKEWARM-1] ____ 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.377 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.378 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[unencrypted-sse-c-LUKEWARM-LUKEWARM-1024] ___ 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.379 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.380 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.381 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-c-LUKEWARM-LUKEWARM-1048576] _ 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.382 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.383 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-c-LUKEWARM-LUKEWARM-8388608] _ 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.384 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.385 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.386 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[unencrypted-sse-c-LUKEWARM-FROZEN-1] _____ 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.387 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.388 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[unencrypted-sse-c-LUKEWARM-FROZEN-1024] ____ 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.389 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.390 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.391 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[unencrypted-sse-c-LUKEWARM-FROZEN-1048576] __ 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.392 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.393 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[unencrypted-sse-c-LUKEWARM-FROZEN-8388608] __ 2026-02-24T22:34:16.394 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.395 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.396 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[unencrypted-sse-c-FROZEN-STANDARD-1] _____ 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.397 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.398 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.399 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[unencrypted-sse-c-FROZEN-STANDARD-1024] ____ 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.400 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.401 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[unencrypted-sse-c-FROZEN-STANDARD-1048576] __ 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.402 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.403 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.404 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[unencrypted-sse-c-FROZEN-STANDARD-8388608] __ 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.405 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.406 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[unencrypted-sse-c-FROZEN-LUKEWARM-1] _____ 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.407 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.408 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.409 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[unencrypted-sse-c-FROZEN-LUKEWARM-1024] ____ 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.410 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.411 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[unencrypted-sse-c-FROZEN-LUKEWARM-1048576] __ 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.412 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.413 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.414 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[unencrypted-sse-c-FROZEN-LUKEWARM-8388608] __ 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.415 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.416 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_copy_enc[unencrypted-sse-c-FROZEN-FROZEN-1] ______ 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.417 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.418 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.419 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[unencrypted-sse-c-FROZEN-FROZEN-1024] _____ 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.420 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.421 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[unencrypted-sse-c-FROZEN-FROZEN-1048576] ___ 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.422 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.423 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.424 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[unencrypted-sse-c-FROZEN-FROZEN-8388608] ___ 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.425 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.426 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[unencrypted-sse-kms-STANDARD-STANDARD-1] ___ 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.427 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.428 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.429 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-kms-STANDARD-STANDARD-1024] __ 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.430 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.431 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.432 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-kms-STANDARD-STANDARD-1048576] _ 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.433 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.434 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-kms-STANDARD-STANDARD-8388608] _ 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.435 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.436 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.437 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[unencrypted-sse-kms-STANDARD-LUKEWARM-1] ___ 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.438 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.439 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-kms-STANDARD-LUKEWARM-1024] __ 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.440 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.441 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.442 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-kms-STANDARD-LUKEWARM-1048576] _ 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.443 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.444 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-kms-STANDARD-LUKEWARM-8388608] _ 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.445 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.446 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.447 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[unencrypted-sse-kms-STANDARD-FROZEN-1] ____ 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.448 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.449 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[unencrypted-sse-kms-STANDARD-FROZEN-1024] ___ 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.450 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.451 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.452 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-kms-STANDARD-FROZEN-1048576] _ 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.453 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.454 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-kms-STANDARD-FROZEN-8388608] _ 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.455 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.456 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.457 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[unencrypted-sse-kms-LUKEWARM-STANDARD-1] ___ 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.458 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.459 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-kms-LUKEWARM-STANDARD-1024] __ 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.460 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.461 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.462 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-kms-LUKEWARM-STANDARD-1048576] _ 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.463 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.464 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-kms-LUKEWARM-STANDARD-8388608] _ 2026-02-24T22:34:16.465 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.466 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.467 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[unencrypted-sse-kms-LUKEWARM-LUKEWARM-1] ___ 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.468 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.469 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.470 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-kms-LUKEWARM-LUKEWARM-1024] __ 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.471 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.472 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-kms-LUKEWARM-LUKEWARM-1048576] _ 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.473 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.474 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.475 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-kms-LUKEWARM-LUKEWARM-8388608] _ 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.476 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.477 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[unencrypted-sse-kms-LUKEWARM-FROZEN-1] ____ 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.478 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.479 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.480 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[unencrypted-sse-kms-LUKEWARM-FROZEN-1024] ___ 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.481 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.482 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-kms-LUKEWARM-FROZEN-1048576] _ 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.483 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.484 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.485 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-kms-LUKEWARM-FROZEN-8388608] _ 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.486 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.487 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[unencrypted-sse-kms-FROZEN-STANDARD-1] ____ 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.488 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.489 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.490 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[unencrypted-sse-kms-FROZEN-STANDARD-1024] ___ 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.491 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.492 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-kms-FROZEN-STANDARD-1048576] _ 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.493 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.494 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.495 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-kms-FROZEN-STANDARD-8388608] _ 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.496 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.497 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.498 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[unencrypted-sse-kms-FROZEN-LUKEWARM-1] ____ 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.499 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.500 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[unencrypted-sse-kms-FROZEN-LUKEWARM-1024] ___ 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.501 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.502 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.503 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-kms-FROZEN-LUKEWARM-1048576] _ 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.504 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.505 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[unencrypted-sse-kms-FROZEN-LUKEWARM-8388608] _ 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.506 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.507 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.508 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[unencrypted-sse-kms-FROZEN-FROZEN-1] _____ 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.509 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.510 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[unencrypted-sse-kms-FROZEN-FROZEN-1024] ____ 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.511 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.512 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.513 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[unencrypted-sse-kms-FROZEN-FROZEN-1048576] __ 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.514 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.515 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[unencrypted-sse-kms-FROZEN-FROZEN-8388608] __ 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.516 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.517 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.518 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-c-unencrypted-STANDARD-STANDARD-1] ____ 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.519 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.520 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[sse-c-unencrypted-STANDARD-STANDARD-1024] ___ 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.521 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.522 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.523 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-c-unencrypted-STANDARD-STANDARD-1048576] _ 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.524 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.525 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-c-unencrypted-STANDARD-STANDARD-8388608] _ 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.526 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.527 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.528 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-c-unencrypted-STANDARD-LUKEWARM-1] ____ 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.529 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.530 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[sse-c-unencrypted-STANDARD-LUKEWARM-1024] ___ 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.531 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.532 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.533 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-c-unencrypted-STANDARD-LUKEWARM-1048576] _ 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.534 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.535 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-c-unencrypted-STANDARD-LUKEWARM-8388608] _ 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.536 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.537 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.538 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-c-unencrypted-STANDARD-FROZEN-1] _____ 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.539 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.540 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.541 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-c-unencrypted-STANDARD-FROZEN-1024] ____ 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.542 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.543 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[sse-c-unencrypted-STANDARD-FROZEN-1048576] __ 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.544 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.545 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.546 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[sse-c-unencrypted-STANDARD-FROZEN-8388608] __ 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.547 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.548 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-c-unencrypted-LUKEWARM-STANDARD-1] ____ 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.549 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.550 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.551 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[sse-c-unencrypted-LUKEWARM-STANDARD-1024] ___ 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.552 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.553 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-c-unencrypted-LUKEWARM-STANDARD-1048576] _ 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.554 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.555 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.556 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-c-unencrypted-LUKEWARM-STANDARD-8388608] _ 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.557 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.558 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-c-unencrypted-LUKEWARM-LUKEWARM-1] ____ 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.559 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.560 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.561 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[sse-c-unencrypted-LUKEWARM-LUKEWARM-1024] ___ 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.562 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.563 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-c-unencrypted-LUKEWARM-LUKEWARM-1048576] _ 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.564 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.565 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.566 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-c-unencrypted-LUKEWARM-LUKEWARM-8388608] _ 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.567 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.568 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-c-unencrypted-LUKEWARM-FROZEN-1] _____ 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.569 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.570 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.571 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-c-unencrypted-LUKEWARM-FROZEN-1024] ____ 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.572 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.573 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[sse-c-unencrypted-LUKEWARM-FROZEN-1048576] __ 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.574 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.575 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.576 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[sse-c-unencrypted-LUKEWARM-FROZEN-8388608] __ 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.577 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.578 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-c-unencrypted-FROZEN-STANDARD-1] _____ 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.579 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.580 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.581 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-c-unencrypted-FROZEN-STANDARD-1024] ____ 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.582 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.583 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[sse-c-unencrypted-FROZEN-STANDARD-1048576] __ 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.584 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.585 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.586 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[sse-c-unencrypted-FROZEN-STANDARD-8388608] __ 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.587 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.588 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-c-unencrypted-FROZEN-LUKEWARM-1] _____ 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.589 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.590 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.591 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-c-unencrypted-FROZEN-LUKEWARM-1024] ____ 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.592 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.593 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.594 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[sse-c-unencrypted-FROZEN-LUKEWARM-1048576] __ 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.595 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.596 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[sse-c-unencrypted-FROZEN-LUKEWARM-8388608] __ 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.597 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.598 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.599 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_copy_enc[sse-c-unencrypted-FROZEN-FROZEN-1] ______ 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.600 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.601 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.602 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-c-unencrypted-FROZEN-FROZEN-1024] _____ 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.603 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.604 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-c-unencrypted-FROZEN-FROZEN-1048576] ___ 2026-02-24T22:34:16.605 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.606 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.607 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-c-unencrypted-FROZEN-FROZEN-8388608] ___ 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.608 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.609 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.610 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout:_______ ERROR at setup of test_copy_enc[sse-c-sse-c-STANDARD-STANDARD-1] _______ 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.611 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.612 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-c-sse-c-STANDARD-STANDARD-1024] ______ 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.613 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.614 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.615 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-c-sse-c-STANDARD-STANDARD-1048576] ____ 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.616 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.617 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-c-sse-c-STANDARD-STANDARD-8388608] ____ 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.618 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.619 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.620 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout:_______ ERROR at setup of test_copy_enc[sse-c-sse-c-STANDARD-LUKEWARM-1] _______ 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.621 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.622 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-c-sse-c-STANDARD-LUKEWARM-1024] ______ 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.623 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.624 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.625 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-c-sse-c-STANDARD-LUKEWARM-1048576] ____ 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.626 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.627 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-c-sse-c-STANDARD-LUKEWARM-8388608] ____ 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.628 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.629 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.630 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout:________ ERROR at setup of test_copy_enc[sse-c-sse-c-STANDARD-FROZEN-1] ________ 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.631 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.632 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_copy_enc[sse-c-sse-c-STANDARD-FROZEN-1024] _______ 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.633 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.634 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.635 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-c-sse-c-STANDARD-FROZEN-1048576] _____ 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.636 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.637 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-c-sse-c-STANDARD-FROZEN-8388608] _____ 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.638 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.639 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.640 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout:_______ ERROR at setup of test_copy_enc[sse-c-sse-c-LUKEWARM-STANDARD-1] _______ 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.641 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.642 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-c-sse-c-LUKEWARM-STANDARD-1024] ______ 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.643 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.644 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.645 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-c-sse-c-LUKEWARM-STANDARD-1048576] ____ 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.646 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.647 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-c-sse-c-LUKEWARM-STANDARD-8388608] ____ 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.648 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.649 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.650 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout:_______ ERROR at setup of test_copy_enc[sse-c-sse-c-LUKEWARM-LUKEWARM-1] _______ 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.651 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.652 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-c-sse-c-LUKEWARM-LUKEWARM-1024] ______ 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.653 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.654 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.655 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-c-sse-c-LUKEWARM-LUKEWARM-1048576] ____ 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.656 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.657 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.658 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-c-sse-c-LUKEWARM-LUKEWARM-8388608] ____ 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.659 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.660 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout:________ ERROR at setup of test_copy_enc[sse-c-sse-c-LUKEWARM-FROZEN-1] ________ 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.661 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.662 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.663 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_copy_enc[sse-c-sse-c-LUKEWARM-FROZEN-1024] _______ 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.664 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.665 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-c-sse-c-LUKEWARM-FROZEN-1048576] _____ 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.666 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.667 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.668 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-c-sse-c-LUKEWARM-FROZEN-8388608] _____ 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.669 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.670 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout:________ ERROR at setup of test_copy_enc[sse-c-sse-c-FROZEN-STANDARD-1] ________ 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.671 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.672 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.673 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_copy_enc[sse-c-sse-c-FROZEN-STANDARD-1024] _______ 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.674 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.675 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-c-sse-c-FROZEN-STANDARD-1048576] _____ 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.676 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.677 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.678 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-c-sse-c-FROZEN-STANDARD-8388608] _____ 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.679 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.680 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout:________ ERROR at setup of test_copy_enc[sse-c-sse-c-FROZEN-LUKEWARM-1] ________ 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.681 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.682 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.683 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_copy_enc[sse-c-sse-c-FROZEN-LUKEWARM-1024] _______ 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.684 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.685 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-c-sse-c-FROZEN-LUKEWARM-1048576] _____ 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.686 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.687 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.688 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-c-sse-c-FROZEN-LUKEWARM-8388608] _____ 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.689 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.690 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout:_________ ERROR at setup of test_copy_enc[sse-c-sse-c-FROZEN-FROZEN-1] _________ 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.691 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.692 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.693 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout:_______ ERROR at setup of test_copy_enc[sse-c-sse-c-FROZEN-FROZEN-1024] ________ 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.694 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.695 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_copy_enc[sse-c-sse-c-FROZEN-FROZEN-1048576] ______ 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.696 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.697 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.698 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_copy_enc[sse-c-sse-c-FROZEN-FROZEN-8388608] ______ 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.699 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.700 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_copy_enc[sse-c-sse-kms-STANDARD-STANDARD-1] ______ 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.701 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.702 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.703 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-c-sse-kms-STANDARD-STANDARD-1024] _____ 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.704 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.705 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.706 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-c-sse-kms-STANDARD-STANDARD-1048576] ___ 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.707 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.708 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-c-sse-kms-STANDARD-STANDARD-8388608] ___ 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.709 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.710 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.711 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_copy_enc[sse-c-sse-kms-STANDARD-LUKEWARM-1] ______ 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.712 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.713 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-c-sse-kms-STANDARD-LUKEWARM-1024] _____ 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.714 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.715 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.716 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-c-sse-kms-STANDARD-LUKEWARM-1048576] ___ 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.717 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.718 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-c-sse-kms-STANDARD-LUKEWARM-8388608] ___ 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.719 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.720 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.721 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout:_______ ERROR at setup of test_copy_enc[sse-c-sse-kms-STANDARD-FROZEN-1] _______ 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.722 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.723 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-c-sse-kms-STANDARD-FROZEN-1024] ______ 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.724 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.725 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.726 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-c-sse-kms-STANDARD-FROZEN-1048576] ____ 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.727 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.728 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-c-sse-kms-STANDARD-FROZEN-8388608] ____ 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.729 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.730 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.731 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_copy_enc[sse-c-sse-kms-LUKEWARM-STANDARD-1] ______ 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.732 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.733 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-c-sse-kms-LUKEWARM-STANDARD-1024] _____ 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.734 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.735 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.736 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-c-sse-kms-LUKEWARM-STANDARD-1048576] ___ 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.737 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.738 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-c-sse-kms-LUKEWARM-STANDARD-8388608] ___ 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.739 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.740 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.741 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_copy_enc[sse-c-sse-kms-LUKEWARM-LUKEWARM-1] ______ 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.742 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.743 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-c-sse-kms-LUKEWARM-LUKEWARM-1024] _____ 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.744 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.745 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.746 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-c-sse-kms-LUKEWARM-LUKEWARM-1048576] ___ 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.747 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.748 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-c-sse-kms-LUKEWARM-LUKEWARM-8388608] ___ 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.749 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.750 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.751 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout:_______ ERROR at setup of test_copy_enc[sse-c-sse-kms-LUKEWARM-FROZEN-1] _______ 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.752 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.753 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-c-sse-kms-LUKEWARM-FROZEN-1024] ______ 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.754 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.755 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.756 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-c-sse-kms-LUKEWARM-FROZEN-1048576] ____ 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.757 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.758 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-c-sse-kms-LUKEWARM-FROZEN-8388608] ____ 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.759 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.760 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.761 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout:_______ ERROR at setup of test_copy_enc[sse-c-sse-kms-FROZEN-STANDARD-1] _______ 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.762 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.763 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.764 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-c-sse-kms-FROZEN-STANDARD-1024] ______ 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.765 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.766 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-c-sse-kms-FROZEN-STANDARD-1048576] ____ 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.767 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.768 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.769 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-c-sse-kms-FROZEN-STANDARD-8388608] ____ 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.770 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.771 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout:_______ ERROR at setup of test_copy_enc[sse-c-sse-kms-FROZEN-LUKEWARM-1] _______ 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.772 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.773 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.774 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-c-sse-kms-FROZEN-LUKEWARM-1024] ______ 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.775 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.776 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-c-sse-kms-FROZEN-LUKEWARM-1048576] ____ 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.777 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.778 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.779 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-c-sse-kms-FROZEN-LUKEWARM-8388608] ____ 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.780 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.781 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout:________ ERROR at setup of test_copy_enc[sse-c-sse-kms-FROZEN-FROZEN-1] ________ 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.782 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.783 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.784 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_copy_enc[sse-c-sse-kms-FROZEN-FROZEN-1024] _______ 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.785 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.786 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-c-sse-kms-FROZEN-FROZEN-1048576] _____ 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.787 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.788 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.789 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-c-sse-kms-FROZEN-FROZEN-8388608] _____ 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.790 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.791 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-kms-unencrypted-STANDARD-STANDARD-1] ___ 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.792 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.793 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.794 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-kms-unencrypted-STANDARD-STANDARD-1024] __ 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.795 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.796 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-kms-unencrypted-STANDARD-STANDARD-1048576] _ 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.797 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.798 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.799 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-kms-unencrypted-STANDARD-STANDARD-8388608] _ 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.800 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.801 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-kms-unencrypted-STANDARD-LUKEWARM-1] ___ 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.802 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.803 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.804 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-kms-unencrypted-STANDARD-LUKEWARM-1024] __ 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.805 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.806 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-kms-unencrypted-STANDARD-LUKEWARM-1048576] _ 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.807 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.808 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.809 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-kms-unencrypted-STANDARD-LUKEWARM-8388608] _ 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.810 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.811 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-kms-unencrypted-STANDARD-FROZEN-1] ____ 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.812 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.813 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.814 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[sse-kms-unencrypted-STANDARD-FROZEN-1024] ___ 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.815 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.816 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-kms-unencrypted-STANDARD-FROZEN-1048576] _ 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.817 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.818 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.819 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-kms-unencrypted-STANDARD-FROZEN-8388608] _ 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.820 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.821 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-kms-unencrypted-LUKEWARM-STANDARD-1] ___ 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.822 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.823 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.824 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-kms-unencrypted-LUKEWARM-STANDARD-1024] __ 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.825 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.826 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-kms-unencrypted-LUKEWARM-STANDARD-1048576] _ 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.827 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.828 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T22:34:16.830 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.830 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.830 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.830 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.830 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.830 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.831 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-kms-unencrypted-LUKEWARM-STANDARD-8388608] _ 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.832 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.833 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.834 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-kms-unencrypted-LUKEWARM-LUKEWARM-1] ___ 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.835 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.836 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-kms-unencrypted-LUKEWARM-LUKEWARM-1024] __ 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.837 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.838 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.839 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-kms-unencrypted-LUKEWARM-LUKEWARM-1048576] _ 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.840 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.841 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-kms-unencrypted-LUKEWARM-LUKEWARM-8388608] _ 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.842 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.843 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.844 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-kms-unencrypted-LUKEWARM-FROZEN-1] ____ 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.845 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.846 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[sse-kms-unencrypted-LUKEWARM-FROZEN-1024] ___ 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.847 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.848 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.849 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-kms-unencrypted-LUKEWARM-FROZEN-1048576] _ 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.850 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.851 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.852 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-kms-unencrypted-LUKEWARM-FROZEN-8388608] _ 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.853 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.854 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-kms-unencrypted-FROZEN-STANDARD-1] ____ 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.855 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.856 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.857 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[sse-kms-unencrypted-FROZEN-STANDARD-1024] ___ 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.858 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.859 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-kms-unencrypted-FROZEN-STANDARD-1048576] _ 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.860 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.861 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.862 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-kms-unencrypted-FROZEN-STANDARD-8388608] _ 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.863 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.864 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-kms-unencrypted-FROZEN-LUKEWARM-1] ____ 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.865 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.866 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.867 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[sse-kms-unencrypted-FROZEN-LUKEWARM-1024] ___ 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.868 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.869 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-kms-unencrypted-FROZEN-LUKEWARM-1048576] _ 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.870 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.871 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.872 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout:_ ERROR at setup of test_copy_enc[sse-kms-unencrypted-FROZEN-LUKEWARM-8388608] _ 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.873 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.874 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-kms-unencrypted-FROZEN-FROZEN-1] _____ 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.875 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.876 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.877 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-kms-unencrypted-FROZEN-FROZEN-1024] ____ 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.878 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.879 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[sse-kms-unencrypted-FROZEN-FROZEN-1048576] __ 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.880 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.881 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.882 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[sse-kms-unencrypted-FROZEN-FROZEN-8388608] __ 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.883 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.884 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_copy_enc[sse-kms-sse-c-STANDARD-STANDARD-1] ______ 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.885 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.886 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.887 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-kms-sse-c-STANDARD-STANDARD-1024] _____ 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.888 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.889 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-kms-sse-c-STANDARD-STANDARD-1048576] ___ 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.890 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.891 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.892 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-kms-sse-c-STANDARD-STANDARD-8388608] ___ 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.893 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.894 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_copy_enc[sse-kms-sse-c-STANDARD-LUKEWARM-1] ______ 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.895 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.896 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.897 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-kms-sse-c-STANDARD-LUKEWARM-1024] _____ 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.898 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.899 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-kms-sse-c-STANDARD-LUKEWARM-1048576] ___ 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.900 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.901 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.902 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-kms-sse-c-STANDARD-LUKEWARM-8388608] ___ 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.903 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.904 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout:_______ ERROR at setup of test_copy_enc[sse-kms-sse-c-STANDARD-FROZEN-1] _______ 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.905 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.906 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.907 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-kms-sse-c-STANDARD-FROZEN-1024] ______ 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.908 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.909 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-kms-sse-c-STANDARD-FROZEN-1048576] ____ 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.910 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.911 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.912 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-kms-sse-c-STANDARD-FROZEN-8388608] ____ 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.913 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.914 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_copy_enc[sse-kms-sse-c-LUKEWARM-STANDARD-1] ______ 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.915 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.916 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.917 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-kms-sse-c-LUKEWARM-STANDARD-1024] _____ 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.918 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.919 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-kms-sse-c-LUKEWARM-STANDARD-1048576] ___ 2026-02-24T22:34:16.920 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.921 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.922 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-kms-sse-c-LUKEWARM-STANDARD-8388608] ___ 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.923 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.924 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.925 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_copy_enc[sse-kms-sse-c-LUKEWARM-LUKEWARM-1] ______ 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.926 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.927 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-kms-sse-c-LUKEWARM-LUKEWARM-1024] _____ 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.928 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.929 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.930 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-kms-sse-c-LUKEWARM-LUKEWARM-1048576] ___ 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.931 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.932 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-kms-sse-c-LUKEWARM-LUKEWARM-8388608] ___ 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.933 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.934 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.935 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout:_______ ERROR at setup of test_copy_enc[sse-kms-sse-c-LUKEWARM-FROZEN-1] _______ 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.936 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.937 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-kms-sse-c-LUKEWARM-FROZEN-1024] ______ 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.938 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.939 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.940 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-kms-sse-c-LUKEWARM-FROZEN-1048576] ____ 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.941 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.942 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-kms-sse-c-LUKEWARM-FROZEN-8388608] ____ 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.943 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.944 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.945 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout:_______ ERROR at setup of test_copy_enc[sse-kms-sse-c-FROZEN-STANDARD-1] _______ 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.946 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.947 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-kms-sse-c-FROZEN-STANDARD-1024] ______ 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.948 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.949 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.950 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-kms-sse-c-FROZEN-STANDARD-1048576] ____ 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.951 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.952 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-kms-sse-c-FROZEN-STANDARD-8388608] ____ 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.953 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.954 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.955 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout:_______ ERROR at setup of test_copy_enc[sse-kms-sse-c-FROZEN-LUKEWARM-1] _______ 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.956 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.957 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-kms-sse-c-FROZEN-LUKEWARM-1024] ______ 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.958 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.959 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.960 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-kms-sse-c-FROZEN-LUKEWARM-1048576] ____ 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.961 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.962 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-kms-sse-c-FROZEN-LUKEWARM-8388608] ____ 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.963 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.964 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.965 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout:________ ERROR at setup of test_copy_enc[sse-kms-sse-c-FROZEN-FROZEN-1] ________ 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.966 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.967 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_copy_enc[sse-kms-sse-c-FROZEN-FROZEN-1024] _______ 2026-02-24T22:34:16.968 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.969 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.970 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-kms-sse-c-FROZEN-FROZEN-1048576] _____ 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.971 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.972 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.973 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-kms-sse-c-FROZEN-FROZEN-8388608] _____ 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.974 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.975 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-kms-sse-kms-STANDARD-STANDARD-1] _____ 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.976 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.977 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.978 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-kms-sse-kms-STANDARD-STANDARD-1024] ____ 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.979 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.980 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[sse-kms-sse-kms-STANDARD-STANDARD-1048576] __ 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.981 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.982 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.983 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[sse-kms-sse-kms-STANDARD-STANDARD-8388608] __ 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.984 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.985 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-kms-sse-kms-STANDARD-LUKEWARM-1] _____ 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.986 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.987 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.988 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-kms-sse-kms-STANDARD-LUKEWARM-1024] ____ 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.989 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.990 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[sse-kms-sse-kms-STANDARD-LUKEWARM-1048576] __ 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.991 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.992 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.993 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[sse-kms-sse-kms-STANDARD-LUKEWARM-8388608] __ 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.994 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.995 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_copy_enc[sse-kms-sse-kms-STANDARD-FROZEN-1] ______ 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.996 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:16.997 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.998 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-kms-sse-kms-STANDARD-FROZEN-1024] _____ 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:16.999 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.000 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-kms-sse-kms-STANDARD-FROZEN-1048576] ___ 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:17.001 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.002 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.003 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-kms-sse-kms-STANDARD-FROZEN-8388608] ___ 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:17.004 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.005 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-kms-sse-kms-LUKEWARM-STANDARD-1] _____ 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.006 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.007 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.008 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-kms-sse-kms-LUKEWARM-STANDARD-1024] ____ 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:17.009 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.010 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[sse-kms-sse-kms-LUKEWARM-STANDARD-1048576] __ 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.011 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.012 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.013 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[sse-kms-sse-kms-LUKEWARM-STANDARD-8388608] __ 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:17.014 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.015 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-kms-sse-kms-LUKEWARM-LUKEWARM-1] _____ 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:17.016 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.017 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.018 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-kms-sse-kms-LUKEWARM-LUKEWARM-1024] ____ 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.019 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.020 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[sse-kms-sse-kms-LUKEWARM-LUKEWARM-1048576] __ 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.021 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.022 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.023 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout:__ ERROR at setup of test_copy_enc[sse-kms-sse-kms-LUKEWARM-LUKEWARM-8388608] __ 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:17.024 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.025 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_copy_enc[sse-kms-sse-kms-LUKEWARM-FROZEN-1] ______ 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:17.026 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.027 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.028 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-kms-sse-kms-LUKEWARM-FROZEN-1024] _____ 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.029 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.030 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-kms-sse-kms-LUKEWARM-FROZEN-1048576] ___ 2026-02-24T22:34:17.031 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.032 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.033 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-kms-sse-kms-LUKEWARM-FROZEN-8388608] ___ 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.034 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.035 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.036 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_copy_enc[sse-kms-sse-kms-FROZEN-STANDARD-1] ______ 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.037 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.038 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-kms-sse-kms-FROZEN-STANDARD-1024] _____ 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:17.039 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.040 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.041 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-kms-sse-kms-FROZEN-STANDARD-1048576] ___ 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.042 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.043 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-kms-sse-kms-FROZEN-STANDARD-8388608] ___ 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:17.044 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.045 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.046 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout:______ ERROR at setup of test_copy_enc[sse-kms-sse-kms-FROZEN-LUKEWARM-1] ______ 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.047 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.048 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-kms-sse-kms-FROZEN-LUKEWARM-1024] _____ 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:17.049 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.050 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.051 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-kms-sse-kms-FROZEN-LUKEWARM-1048576] ___ 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.052 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.053 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout:___ ERROR at setup of test_copy_enc[sse-kms-sse-kms-FROZEN-LUKEWARM-8388608] ___ 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:17.054 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.055 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.056 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout:_______ ERROR at setup of test_copy_enc[sse-kms-sse-kms-FROZEN-FROZEN-1] _______ 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.057 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.058 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout:_____ ERROR at setup of test_copy_enc[sse-kms-sse-kms-FROZEN-FROZEN-1024] ______ 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:17.059 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.060 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.061 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-kms-sse-kms-FROZEN-FROZEN-1048576] ____ 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.062 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.063 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout:____ ERROR at setup of test_copy_enc[sse-kms-sse-kms-FROZEN-FROZEN-8388608] ____ 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout:configfile = 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: @pytest.fixture(autouse=True) 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: def setup_teardown(configfile): 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout:> setup() 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:342: 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:306: in setup 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: nuke_prefixed_buckets(prefix=prefix) 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:159: in nuke_prefixed_buckets 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: raise err 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:150: in nuke_prefixed_buckets 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout: nuke_bucket(client, bucket_name) 2026-02-24T22:34:17.064 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/__init__.py:139: in nuke_bucket 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: client.delete_bucket(Bucket=bucket) 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:602: in _api_call 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: return self._make_api_call(operation_name, kwargs) 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/context.py:123: in wrapper 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: return func(*args, **kwargs) 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout:self = 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout:operation_name = 'DeleteBucket' 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout:api_params = {'Bucket': 'test-e7n3yd9d2xql4uuphll37swt-435'} 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: @with_current_context() 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: def _make_api_call(self, operation_name, api_params): 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: operation_model = self._service_model.operation_model(operation_name) 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: service_name = self._service_model.service_name 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: history_recorder.record( 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: 'API_CALL', 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: { 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: 'service': service_name, 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: 'operation': operation_name, 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: 'params': api_params, 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: }, 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: if operation_model.deprecated: 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: logger.debug( 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: request_context = { 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: 'client_region': self.meta.region_name, 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: 'client_config': self.meta.config, 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_type': operation_model.resolved_auth_type, 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: 'unsigned_payload': operation_model.unsigned_payload, 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: 'auth_options': self._service_model.metadata.get('auth'), 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: } 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: api_params = self._emit_api_params( 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: ( 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url, 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: additional_headers, 2026-02-24T22:34:17.065 INFO:teuthology.orchestra.run.trial046.stdout: properties, 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: ) = self._resolve_endpoint_ruleset( 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, api_params, request_context 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: if properties: 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: # Pass arbitrary endpoint info with the Request 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: # for use during construction. 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: request_context['endpoint_properties'] = properties 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: request_dict = self._convert_to_request_dict( 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: api_params=api_params, 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: operation_model=operation_model, 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: endpoint_url=endpoint_url, 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: headers=additional_headers, 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: service_id = self._service_model.service_id.hyphenize() 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: handler, event_response = self.meta.events.emit_until_response( 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: f'before-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: params=request_dict, 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: request_signer=self._request_signer, 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: if event_response is not None: 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = event_response 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: else: 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: maybe_compress_request( 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.config, request_dict, operation_model 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: apply_request_checksum(request_dict) 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: http, parsed_response = self._make_request( 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: operation_model, request_dict, request_context 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: self.meta.events.emit( 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: f'after-call.{service_id}.{operation_name}', 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: http_response=http, 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: parsed=parsed_response, 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: model=operation_model, 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: context=request_context, 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: ) 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: if http.status_code >= 300: 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: error_info = parsed_response.get("Error", {}) 2026-02-24T22:34:17.066 INFO:teuthology.orchestra.run.trial046.stdout: error_code = request_context.get( 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: 'error_code_override' 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: ) or error_info.get("Code") 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: error_class = self.exceptions.from_code(error_code) 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout:> raise error_class(parsed_response, operation_name) 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout:E botocore.exceptions.ClientError: An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: None 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:1078: ClientError 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout:=================================== FAILURES =================================== 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout:______________ test_versioned_concurrent_object_create_and_remove ______________ 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: def test_versioned_concurrent_object_create_and_remove(): 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: bucket_name = get_new_bucket() 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: client = get_client() 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: check_configure_versioning_retry(bucket_name, "Enabled", "Enabled") 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: key = 'myobj' 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: num_versions = 3 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: all_threads = [] 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: for i in range(3): 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: t = _do_create_versioned_obj_concurrent(client, bucket_name, key, num_versions) 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: all_threads.append(t) 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: t = _do_clear_versioned_bucket_concurrent(client, bucket_name) 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: all_threads.append(t) 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: for t in all_threads: 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: _do_wait_completion(t) 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: t = _do_clear_versioned_bucket_concurrent(client, bucket_name) 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: _do_wait_completion(t) 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: response = client.list_object_versions(Bucket=bucket_name) 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout:> assert not 'Versions' in response 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout:E AssertionError: assert not 'Versions' in {'EncodingType': 'url', 'IsTruncated': False, 'KeyMarker': '', 'MaxKeys': 1000, ...} 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py:8373: AssertionError 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout:---------------------------- Captured stdout setup ----------------------------- 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout:Done with cleanup of buckets in tests. 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout:Done with cleanup of buckets in tests. 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout:Done with cleanup of buckets in tests. 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout:=============================== warnings summary =============================== 2026-02-24T22:34:17.067 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_headers.py: 27 warnings 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_iam.py: 85 warnings 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py: 975 warnings 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3select.py: 33 warnings 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_sns.py: 3 warnings 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout: /home/ubuntu/cephtest/s3-tests-client.0/.tox/py/lib/python3.10/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'trial046.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout: warnings.warn( 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_100_continue 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_100_continue 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests/functional/test_s3.py:6779: DeprecationWarning: ssl.wrap_socket() is deprecated, use SSLContext.wrap_socket() 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout: s = ssl.wrap_socket(s); 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_100_continue 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:s3tests/functional/test_s3.py::test_100_continue 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout: /usr/lib/python3.10/ssl.py:1463: DeprecationWarning: ssl.PROTOCOL_TLS is deprecated 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout: context = SSLContext(ssl_version) 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:=========================== short test summary info ============================ 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:FAILED s3tests/functional/test_s3.py::test_versioned_concurrent_object_create_and_remove 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_versioned_concurrent_object_create_and_remove 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_lifecycle_set - botocore.exceptions... 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_lifecycle_get - botocore.exceptions... 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_lifecycle_delete - botocore.excepti... 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_lifecycle_get_no_id - botocore.exce... 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_lifecycle_id_too_long - botocore.ex... 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_lifecycle_same_id - botocore.except... 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_lifecycle_invalid_status - botocore... 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_lifecycle_set_date - botocore.excep... 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_lifecycle_set_invalid_date - botoco... 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_lifecycle_set_noncurrent - botocore... 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_lifecycle_set_deletemarker - botoco... 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_lifecycle_set_filter - botocore.exc... 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_lifecycle_set_empty_filter - botoco... 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_lifecycle_set_multipart - botocore.... 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_lifecycle_transition_set_invalid_date 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_restore_object_temporary - botocore... 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_restore_object_permanent - botocore... 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_read_through - botocore.exceptions.... 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_restore_noncur_obj - botocore.excep... 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_encrypted_transfer_1b - botocore.ex... 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_encrypted_transfer_1kb - botocore.e... 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_encrypted_transfer_1MB - botocore.e... 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_encrypted_transfer_13b - botocore.e... 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_encryption_sse_c_method_head - boto... 2026-02-24T22:34:17.068 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_encryption_sse_c_present - botocore... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_encryption_sse_c_other_key - botoco... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_encryption_sse_c_invalid_md5 - boto... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_encryption_sse_c_no_md5 - botocore.... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_encryption_sse_c_no_key - botocore.... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_encryption_key_no_sse_c - botocore.... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_encryption_sse_c_multipart_upload 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_encryption_sse_c_unaligned_multipart_upload 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_encryption_sse_c_multipart_bad_download 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_encryption_sse_c_post_object_authenticated_request 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_encryption_sse_c_enforced_with_bucket_policy 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_encryption_sse_c_deny_algo_with_bucket_policy 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_sse_kms_method_head - botocore.exce... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_sse_kms_present - botocore.exceptio... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_sse_kms_no_key - botocore.exception... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_sse_kms_not_declared - botocore.exc... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_sse_kms_multipart_upload - botocore... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_1 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_2 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_sse_kms_post_object_authenticated_request 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_sse_kms_transfer_1b - botocore.exce... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_sse_kms_transfer_1kb - botocore.exc... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_sse_kms_transfer_1MB - botocore.exc... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_sse_kms_transfer_13b - botocore.exc... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_sse_kms_read_declare - botocore.exc... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_bucket_policy - botocore.exceptions... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_bucketv2_policy - botocore.exceptio... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_bucket_policy_deny_self_denied_policy 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_bucket_policy_deny_self_denied_policy_confirm_header 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_bucket_policy_acl - botocore.except... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_bucketv2_policy_acl - botocore.exce... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_bucket_policy_different_tenant - bo... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_bucket_policy_multipart - botocore.... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_bucket_policy_tenanted_bucket - bot... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_bucket_policy_another_bucket - boto... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_bucketv2_policy_another_bucket - bo... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_set_get_del_bucket_policy - botocor... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_set_multipart_tagging - botocore.ex... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_get_obj_tagging - botocore.exceptio... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_get_obj_head_tagging - botocore.exc... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_put_max_tags - botocore.exceptions.... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_put_excess_tags - botocore.exceptio... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_put_max_kvsize_tags - botocore.exce... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_put_excess_key_tags - botocore.exce... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_put_excess_val_tags - botocore.exce... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_put_modify_tags - botocore.exceptio... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_put_delete_tags - botocore.exceptio... 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_post_object_tags_anonymous_request 2026-02-24T22:34:17.069 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_post_object_tags_authenticated_request 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_put_obj_with_tags - botocore.except... 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_get_tags_acl_public - botocore.exce... 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_put_tags_acl_public - botocore.exce... 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_delete_tags_obj_public - botocore.e... 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_versioning_bucket_atomic_upload_return_version_id 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_versioning_bucket_multipart_upload_return_version_id 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_bucket_policy_get_obj_existing_tag 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_bucket_policy_get_obj_tagging_existing_tag 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_bucket_policy_put_obj_tagging_existing_tag 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_bucket_policy_upload_part_copy - bo... 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_bucket_policy_put_obj_copy_source 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_bucket_policy_put_obj_copy_source_meta 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_bucket_policy_put_obj_acl - botocor... 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_bucket_policy_put_obj_grant - botoc... 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_put_obj_enc_conflict_c_s3 - botocor... 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_put_obj_enc_conflict_c_kms - botoco... 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_put_obj_enc_conflict_s3_kms - botoc... 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_put_obj_enc_conflict_bad_enc_kms - ... 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_bucket_policy_put_obj_kms_noenc - b... 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_bucket_policy_put_obj_kms_s3 - boto... 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_bucket_policy_get_obj_acl_existing_tag 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_put_obj_lock - botocore... 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_bucket 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_put_obj_lock_enable_after_create 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_put_obj_lock_with_days_and_years 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_days 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_years 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_mode 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_status 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_suspend_versioning - bo... 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_get_obj_lock - botocore... 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_get_obj_lock_invalid_bucket 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_put_obj_retention - bot... 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_bucket 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_mode 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_get_obj_retention - bot... 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_get_obj_retention_iso8601 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_get_obj_retention_invalid_bucket 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_put_obj_retention_versionid 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_put_obj_retention_override_default_retention 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_put_obj_retention_increase_period 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period_bypass 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_delete_object_with_retention 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_delete_multipart_object_with_retention 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_delete_object_with_retention_and_marker 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_multi_delete_object_with_retention 2026-02-24T22:34:17.070 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_put_legal_hold - botoco... 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_bucket 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_status 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_get_legal_hold - botoco... 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_get_legal_hold_invalid_bucket 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_on 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_delete_multipart_object_with_legal_hold_on 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_off 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_get_obj_metadata - boto... 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_uploading_obj - botocor... 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_changing_mode_from_governance_with_bypass 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_changing_mode_from_governance_without_bypass 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_lock_changing_mode_from_compliance 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_object_ifmatch_good - botocore... 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_object_ifnonematch_failed - bo... 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_get_bucket_policy_status - botocore... 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_get_public_acl_bucket_policy_status 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_get_authpublic_acl_bucket_policy_status 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_get_publicpolicy_acl_bucket_policy_status 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_get_nonpublicpolicy_acl_bucket_policy_status 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_get_nonpublicpolicy_principal_bucket_policy_status 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_bucket_policy_allow_notprincipal - ... 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_get_undefined_public_block - botoco... 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_get_public_block_deny_bucket_policy 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_put_public_block - botocore.excepti... 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_block_public_put_bucket_acls - boto... 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_block_public_object_canned_acls - b... 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_block_public_policy - botocore.exce... 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_block_public_policy_with_principal 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_block_public_restrict_public_buckets 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_ignore_public_acls - botocore.excep... 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_put_get_delete_public_block - botoc... 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_multipart_upload_on_a_bucket_with_policy 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_put_bucket_encryption_kms - botocor... 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_get_bucket_encryption_kms - botocor... 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_delete_bucket_encryption_kms - boto... 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_sse_kms_default_post_object_authenticated_request 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_get_object_torrent - botocore.excep... 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_checksum_sha256 - botocore.e... 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_object_checksum_crc64nvme - botocor... 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_multipart_checksum_sha256 - botocor... 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_multipart_use_cksum_helper_sha256 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_multipart_use_cksum_helper_crc64nvme 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_multipart_use_cksum_helper_crc32 - ... 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_multipart_use_cksum_helper_crc32c 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_multipart_use_cksum_helper_sha1 - b... 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_post_object_upload_checksum - botoc... 2026-02-24T22:34:17.071 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_get_multipart_checksum_object_attributes 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_get_multipart_object_attributes - b... 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_get_paginated_multipart_object_attributes 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_get_single_multipart_object_attributes 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_get_checksum_object_attributes - bo... 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_get_versioned_object_attributes - b... 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_get_sse_c_encrypted_object_attributes 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_get_object_attributes - botocore.ex... 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_upload_part_copy_percent_encoded_key 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_delete_marker_nonversioned - botoco... 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_delete_marker_versioned - botocore.... 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_delete_marker_suspended - botocore.... 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_put_object_if_match - botocore.exce... 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_multipart_put_object_if_match - bot... 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_put_current_object_if_none_match - ... 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_multipart_put_current_object_if_none_match 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_put_current_object_if_match - botoc... 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_multipart_put_current_object_if_match 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_put_object_current_if_match - botoc... 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_delete_object_if_match - botocore.e... 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_delete_object_current_if_match - bo... 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_delete_object_version_if_match - bo... 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_delete_object_if_match_last_modified_time 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_delete_object_current_if_match_last_modified_time 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_delete_object_version_if_match_last_modified_time 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_delete_object_if_match_size - botoc... 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_delete_object_current_if_match_size 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_delete_object_version_if_match_size 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_delete_objects_if_match - botocore.... 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_delete_objects_current_if_match - b... 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_delete_objects_version_if_match - b... 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_delete_objects_if_match_last_modified_time 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_delete_objects_current_if_match_last_modified_time 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_delete_objects_version_if_match_last_modified_time 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_delete_objects_if_match_size - boto... 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_delete_objects_current_if_match_size 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_delete_objects_version_if_match_size 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_create_bucket_no_ownership_controls 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_create_bucket_bucket_owner_enforced 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_create_bucket_bucket_owner_preferred 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_create_bucket_object_writer - botoc... 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_put_bucket_ownership_bucket_owner_enforced 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_put_bucket_ownership_bucket_owner_preferred 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_put_bucket_ownership_object_writer 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_bucket_create_delete_bucket_ownership 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-unencrypted-STANDARD-STANDARD-8388608] 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-unencrypted-STANDARD-LUKEWARM-8388608] 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-unencrypted-STANDARD-FROZEN-8388608] 2026-02-24T22:34:17.072 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-unencrypted-LUKEWARM-STANDARD-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-unencrypted-LUKEWARM-LUKEWARM-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-unencrypted-LUKEWARM-FROZEN-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-unencrypted-FROZEN-STANDARD-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-unencrypted-FROZEN-LUKEWARM-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-unencrypted-FROZEN-FROZEN-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-c-STANDARD-STANDARD-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-c-STANDARD-LUKEWARM-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-c-STANDARD-FROZEN-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-c-LUKEWARM-STANDARD-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-c-LUKEWARM-LUKEWARM-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-c-LUKEWARM-FROZEN-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-c-FROZEN-STANDARD-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-c-FROZEN-LUKEWARM-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-c-FROZEN-FROZEN-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-kms-STANDARD-STANDARD-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-kms-STANDARD-LUKEWARM-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-kms-STANDARD-FROZEN-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-kms-LUKEWARM-STANDARD-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-kms-LUKEWARM-LUKEWARM-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-kms-LUKEWARM-FROZEN-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-kms-FROZEN-STANDARD-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-kms-FROZEN-LUKEWARM-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[unencrypted-sse-kms-FROZEN-FROZEN-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-unencrypted-STANDARD-STANDARD-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-unencrypted-STANDARD-LUKEWARM-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-unencrypted-STANDARD-FROZEN-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-unencrypted-LUKEWARM-STANDARD-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-unencrypted-LUKEWARM-LUKEWARM-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-unencrypted-LUKEWARM-FROZEN-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-unencrypted-FROZEN-STANDARD-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-unencrypted-FROZEN-LUKEWARM-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-unencrypted-FROZEN-FROZEN-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-c-STANDARD-STANDARD-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-c-STANDARD-LUKEWARM-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-c-STANDARD-FROZEN-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-c-LUKEWARM-STANDARD-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-c-LUKEWARM-LUKEWARM-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-c-LUKEWARM-FROZEN-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-c-FROZEN-STANDARD-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-c-FROZEN-LUKEWARM-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-c-FROZEN-FROZEN-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-kms-STANDARD-STANDARD-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-kms-STANDARD-LUKEWARM-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-kms-STANDARD-FROZEN-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-kms-LUKEWARM-STANDARD-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-kms-LUKEWARM-LUKEWARM-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-kms-LUKEWARM-FROZEN-8388608] 2026-02-24T22:34:17.073 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-kms-FROZEN-STANDARD-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-kms-FROZEN-LUKEWARM-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-c-sse-kms-FROZEN-FROZEN-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-unencrypted-STANDARD-STANDARD-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-unencrypted-STANDARD-LUKEWARM-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-unencrypted-STANDARD-FROZEN-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-unencrypted-LUKEWARM-STANDARD-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-unencrypted-LUKEWARM-LUKEWARM-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-unencrypted-LUKEWARM-FROZEN-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-unencrypted-FROZEN-STANDARD-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-unencrypted-FROZEN-LUKEWARM-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-unencrypted-FROZEN-FROZEN-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-c-STANDARD-STANDARD-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-c-STANDARD-LUKEWARM-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-c-STANDARD-FROZEN-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-c-LUKEWARM-STANDARD-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-c-LUKEWARM-LUKEWARM-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-c-LUKEWARM-FROZEN-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-c-FROZEN-STANDARD-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-c-FROZEN-LUKEWARM-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-c-FROZEN-FROZEN-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-kms-STANDARD-STANDARD-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-kms-STANDARD-LUKEWARM-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-kms-STANDARD-FROZEN-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-kms-LUKEWARM-STANDARD-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-kms-LUKEWARM-LUKEWARM-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-kms-LUKEWARM-FROZEN-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-kms-FROZEN-STANDARD-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-kms-FROZEN-LUKEWARM-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_part_enc[sse-kms-sse-kms-FROZEN-FROZEN-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-STANDARD-STANDARD-1] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-STANDARD-STANDARD-1024] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-STANDARD-STANDARD-1048576] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-STANDARD-STANDARD-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-STANDARD-LUKEWARM-1] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-STANDARD-LUKEWARM-1024] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-STANDARD-LUKEWARM-1048576] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-STANDARD-LUKEWARM-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-STANDARD-FROZEN-1] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-STANDARD-FROZEN-1024] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-STANDARD-FROZEN-1048576] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-STANDARD-FROZEN-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-LUKEWARM-STANDARD-1] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-LUKEWARM-STANDARD-1024] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-LUKEWARM-STANDARD-1048576] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-LUKEWARM-STANDARD-8388608] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-LUKEWARM-LUKEWARM-1] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-LUKEWARM-LUKEWARM-1024] 2026-02-24T22:34:17.074 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-LUKEWARM-LUKEWARM-1048576] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-LUKEWARM-LUKEWARM-8388608] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-LUKEWARM-FROZEN-1] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-LUKEWARM-FROZEN-1024] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-LUKEWARM-FROZEN-1048576] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-LUKEWARM-FROZEN-8388608] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-FROZEN-STANDARD-1] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-FROZEN-STANDARD-1024] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-FROZEN-STANDARD-1048576] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-FROZEN-STANDARD-8388608] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-FROZEN-LUKEWARM-1] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-FROZEN-LUKEWARM-1024] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-FROZEN-LUKEWARM-1048576] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-FROZEN-LUKEWARM-8388608] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-FROZEN-FROZEN-1] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-FROZEN-FROZEN-1024] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-FROZEN-FROZEN-1048576] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-unencrypted-FROZEN-FROZEN-8388608] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-STANDARD-STANDARD-1] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-STANDARD-STANDARD-1024] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-STANDARD-STANDARD-1048576] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-STANDARD-STANDARD-8388608] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-STANDARD-LUKEWARM-1] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-STANDARD-LUKEWARM-1024] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-STANDARD-LUKEWARM-1048576] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-STANDARD-LUKEWARM-8388608] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-STANDARD-FROZEN-1] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-STANDARD-FROZEN-1024] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-STANDARD-FROZEN-1048576] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-STANDARD-FROZEN-8388608] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-LUKEWARM-STANDARD-1] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-LUKEWARM-STANDARD-1024] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-LUKEWARM-STANDARD-1048576] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-LUKEWARM-STANDARD-8388608] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-LUKEWARM-LUKEWARM-1] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-LUKEWARM-LUKEWARM-1024] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-LUKEWARM-LUKEWARM-1048576] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-LUKEWARM-LUKEWARM-8388608] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-LUKEWARM-FROZEN-1] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-LUKEWARM-FROZEN-1024] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-LUKEWARM-FROZEN-1048576] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-LUKEWARM-FROZEN-8388608] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-FROZEN-STANDARD-1] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-FROZEN-STANDARD-1024] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-FROZEN-STANDARD-1048576] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-FROZEN-STANDARD-8388608] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-FROZEN-LUKEWARM-1] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-FROZEN-LUKEWARM-1024] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-FROZEN-LUKEWARM-1048576] 2026-02-24T22:34:17.075 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-FROZEN-LUKEWARM-8388608] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-FROZEN-FROZEN-1] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-FROZEN-FROZEN-1024] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-FROZEN-FROZEN-1048576] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-c-FROZEN-FROZEN-8388608] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-STANDARD-STANDARD-1] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-STANDARD-STANDARD-1024] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-STANDARD-STANDARD-1048576] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-STANDARD-STANDARD-8388608] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-STANDARD-LUKEWARM-1] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-STANDARD-LUKEWARM-1024] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-STANDARD-LUKEWARM-1048576] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-STANDARD-LUKEWARM-8388608] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-STANDARD-FROZEN-1] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-STANDARD-FROZEN-1024] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-STANDARD-FROZEN-1048576] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-STANDARD-FROZEN-8388608] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-LUKEWARM-STANDARD-1] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-LUKEWARM-STANDARD-1024] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-LUKEWARM-STANDARD-1048576] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-LUKEWARM-STANDARD-8388608] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-LUKEWARM-LUKEWARM-1] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-LUKEWARM-LUKEWARM-1024] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-LUKEWARM-LUKEWARM-1048576] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-LUKEWARM-LUKEWARM-8388608] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-LUKEWARM-FROZEN-1] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-LUKEWARM-FROZEN-1024] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-LUKEWARM-FROZEN-1048576] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-LUKEWARM-FROZEN-8388608] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-FROZEN-STANDARD-1] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-FROZEN-STANDARD-1024] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-FROZEN-STANDARD-1048576] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-FROZEN-STANDARD-8388608] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-FROZEN-LUKEWARM-1] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-FROZEN-LUKEWARM-1024] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-FROZEN-LUKEWARM-1048576] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-FROZEN-LUKEWARM-8388608] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-FROZEN-FROZEN-1] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-FROZEN-FROZEN-1024] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-FROZEN-FROZEN-1048576] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[unencrypted-sse-kms-FROZEN-FROZEN-8388608] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-STANDARD-STANDARD-1] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-STANDARD-STANDARD-1024] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-STANDARD-STANDARD-1048576] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-STANDARD-STANDARD-8388608] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-STANDARD-LUKEWARM-1] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-STANDARD-LUKEWARM-1024] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-STANDARD-LUKEWARM-1048576] 2026-02-24T22:34:17.076 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-STANDARD-LUKEWARM-8388608] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-STANDARD-FROZEN-1] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-STANDARD-FROZEN-1024] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-STANDARD-FROZEN-1048576] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-STANDARD-FROZEN-8388608] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-LUKEWARM-STANDARD-1] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-LUKEWARM-STANDARD-1024] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-LUKEWARM-STANDARD-1048576] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-LUKEWARM-STANDARD-8388608] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-LUKEWARM-LUKEWARM-1] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-LUKEWARM-LUKEWARM-1024] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-LUKEWARM-LUKEWARM-1048576] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-LUKEWARM-LUKEWARM-8388608] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-LUKEWARM-FROZEN-1] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-LUKEWARM-FROZEN-1024] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-LUKEWARM-FROZEN-1048576] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-LUKEWARM-FROZEN-8388608] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-FROZEN-STANDARD-1] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-FROZEN-STANDARD-1024] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-FROZEN-STANDARD-1048576] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-FROZEN-STANDARD-8388608] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-FROZEN-LUKEWARM-1] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-FROZEN-LUKEWARM-1024] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-FROZEN-LUKEWARM-1048576] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-FROZEN-LUKEWARM-8388608] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-FROZEN-FROZEN-1] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-FROZEN-FROZEN-1024] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-FROZEN-FROZEN-1048576] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-unencrypted-FROZEN-FROZEN-8388608] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-STANDARD-STANDARD-1] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-STANDARD-STANDARD-1024] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-STANDARD-STANDARD-1048576] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-STANDARD-STANDARD-8388608] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-STANDARD-LUKEWARM-1] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-STANDARD-LUKEWARM-1024] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-STANDARD-LUKEWARM-1048576] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-STANDARD-LUKEWARM-8388608] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-STANDARD-FROZEN-1] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-STANDARD-FROZEN-1024] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-STANDARD-FROZEN-1048576] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-STANDARD-FROZEN-8388608] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-LUKEWARM-STANDARD-1] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-LUKEWARM-STANDARD-1024] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-LUKEWARM-STANDARD-1048576] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-LUKEWARM-STANDARD-8388608] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-LUKEWARM-LUKEWARM-1] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-LUKEWARM-LUKEWARM-1024] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-LUKEWARM-LUKEWARM-1048576] 2026-02-24T22:34:17.077 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-LUKEWARM-LUKEWARM-8388608] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-LUKEWARM-FROZEN-1] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-LUKEWARM-FROZEN-1024] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-LUKEWARM-FROZEN-1048576] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-LUKEWARM-FROZEN-8388608] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-FROZEN-STANDARD-1] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-FROZEN-STANDARD-1024] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-FROZEN-STANDARD-1048576] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-FROZEN-STANDARD-8388608] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-FROZEN-LUKEWARM-1] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-FROZEN-LUKEWARM-1024] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-FROZEN-LUKEWARM-1048576] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-FROZEN-LUKEWARM-8388608] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-FROZEN-FROZEN-1] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-FROZEN-FROZEN-1024] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-FROZEN-FROZEN-1048576] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-c-FROZEN-FROZEN-8388608] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-STANDARD-STANDARD-1] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-STANDARD-STANDARD-1024] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-STANDARD-STANDARD-1048576] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-STANDARD-STANDARD-8388608] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-STANDARD-LUKEWARM-1] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-STANDARD-LUKEWARM-1024] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-STANDARD-LUKEWARM-1048576] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-STANDARD-LUKEWARM-8388608] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-STANDARD-FROZEN-1] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-STANDARD-FROZEN-1024] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-STANDARD-FROZEN-1048576] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-STANDARD-FROZEN-8388608] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-LUKEWARM-STANDARD-1] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-LUKEWARM-STANDARD-1024] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-LUKEWARM-STANDARD-1048576] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-LUKEWARM-STANDARD-8388608] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-LUKEWARM-LUKEWARM-1] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-LUKEWARM-LUKEWARM-1024] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-LUKEWARM-LUKEWARM-1048576] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-LUKEWARM-LUKEWARM-8388608] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-LUKEWARM-FROZEN-1] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-LUKEWARM-FROZEN-1024] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-LUKEWARM-FROZEN-1048576] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-LUKEWARM-FROZEN-8388608] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-FROZEN-STANDARD-1] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-FROZEN-STANDARD-1024] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-FROZEN-STANDARD-1048576] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-FROZEN-STANDARD-8388608] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-FROZEN-LUKEWARM-1] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-FROZEN-LUKEWARM-1024] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-FROZEN-LUKEWARM-1048576] 2026-02-24T22:34:17.078 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-FROZEN-LUKEWARM-8388608] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-FROZEN-FROZEN-1] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-FROZEN-FROZEN-1024] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-FROZEN-FROZEN-1048576] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-c-sse-kms-FROZEN-FROZEN-8388608] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-STANDARD-STANDARD-1] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-STANDARD-STANDARD-1024] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-STANDARD-STANDARD-1048576] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-STANDARD-STANDARD-8388608] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-STANDARD-LUKEWARM-1] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-STANDARD-LUKEWARM-1024] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-STANDARD-LUKEWARM-1048576] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-STANDARD-LUKEWARM-8388608] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-STANDARD-FROZEN-1] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-STANDARD-FROZEN-1024] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-STANDARD-FROZEN-1048576] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-STANDARD-FROZEN-8388608] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-LUKEWARM-STANDARD-1] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-LUKEWARM-STANDARD-1024] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-LUKEWARM-STANDARD-1048576] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-LUKEWARM-STANDARD-8388608] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-LUKEWARM-LUKEWARM-1] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-LUKEWARM-LUKEWARM-1024] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-LUKEWARM-LUKEWARM-1048576] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-LUKEWARM-LUKEWARM-8388608] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-LUKEWARM-FROZEN-1] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-LUKEWARM-FROZEN-1024] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-LUKEWARM-FROZEN-1048576] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-LUKEWARM-FROZEN-8388608] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-FROZEN-STANDARD-1] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-FROZEN-STANDARD-1024] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-FROZEN-STANDARD-1048576] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-FROZEN-STANDARD-8388608] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-FROZEN-LUKEWARM-1] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-FROZEN-LUKEWARM-1024] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-FROZEN-LUKEWARM-1048576] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-FROZEN-LUKEWARM-8388608] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-FROZEN-FROZEN-1] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-FROZEN-FROZEN-1024] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-FROZEN-FROZEN-1048576] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-unencrypted-FROZEN-FROZEN-8388608] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-STANDARD-STANDARD-1] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-STANDARD-STANDARD-1024] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-STANDARD-STANDARD-1048576] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-STANDARD-STANDARD-8388608] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-STANDARD-LUKEWARM-1] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-STANDARD-LUKEWARM-1024] 2026-02-24T22:34:17.079 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-STANDARD-LUKEWARM-1048576] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-STANDARD-LUKEWARM-8388608] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-STANDARD-FROZEN-1] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-STANDARD-FROZEN-1024] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-STANDARD-FROZEN-1048576] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-STANDARD-FROZEN-8388608] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-LUKEWARM-STANDARD-1] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-LUKEWARM-STANDARD-1024] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-LUKEWARM-STANDARD-1048576] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-LUKEWARM-STANDARD-8388608] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-LUKEWARM-LUKEWARM-1] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-LUKEWARM-LUKEWARM-1024] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-LUKEWARM-LUKEWARM-1048576] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-LUKEWARM-LUKEWARM-8388608] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-LUKEWARM-FROZEN-1] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-LUKEWARM-FROZEN-1024] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-LUKEWARM-FROZEN-1048576] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-LUKEWARM-FROZEN-8388608] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-FROZEN-STANDARD-1] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-FROZEN-STANDARD-1024] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-FROZEN-STANDARD-1048576] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-FROZEN-STANDARD-8388608] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-FROZEN-LUKEWARM-1] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-FROZEN-LUKEWARM-1024] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-FROZEN-LUKEWARM-1048576] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-FROZEN-LUKEWARM-8388608] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-FROZEN-FROZEN-1] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-FROZEN-FROZEN-1024] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-FROZEN-FROZEN-1048576] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-c-FROZEN-FROZEN-8388608] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-STANDARD-STANDARD-1] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-STANDARD-STANDARD-1024] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-STANDARD-STANDARD-1048576] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-STANDARD-STANDARD-8388608] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-STANDARD-LUKEWARM-1] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-STANDARD-LUKEWARM-1024] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-STANDARD-LUKEWARM-1048576] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-STANDARD-LUKEWARM-8388608] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-STANDARD-FROZEN-1] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-STANDARD-FROZEN-1024] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-STANDARD-FROZEN-1048576] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-STANDARD-FROZEN-8388608] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-LUKEWARM-STANDARD-1] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-LUKEWARM-STANDARD-1024] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-LUKEWARM-STANDARD-1048576] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-LUKEWARM-STANDARD-8388608] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-LUKEWARM-LUKEWARM-1] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-LUKEWARM-LUKEWARM-1024] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-LUKEWARM-LUKEWARM-1048576] 2026-02-24T22:34:17.080 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-LUKEWARM-LUKEWARM-8388608] 2026-02-24T22:34:17.081 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-LUKEWARM-FROZEN-1] 2026-02-24T22:34:17.081 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-LUKEWARM-FROZEN-1024] 2026-02-24T22:34:17.081 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-LUKEWARM-FROZEN-1048576] 2026-02-24T22:34:17.081 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-LUKEWARM-FROZEN-8388608] 2026-02-24T22:34:17.081 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-FROZEN-STANDARD-1] 2026-02-24T22:34:17.081 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-FROZEN-STANDARD-1024] 2026-02-24T22:34:17.081 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-FROZEN-STANDARD-1048576] 2026-02-24T22:34:17.081 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-FROZEN-STANDARD-8388608] 2026-02-24T22:34:17.081 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-FROZEN-LUKEWARM-1] 2026-02-24T22:34:17.081 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-FROZEN-LUKEWARM-1024] 2026-02-24T22:34:17.081 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-FROZEN-LUKEWARM-1048576] 2026-02-24T22:34:17.081 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-FROZEN-LUKEWARM-8388608] 2026-02-24T22:34:17.081 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-FROZEN-FROZEN-1] 2026-02-24T22:34:17.081 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-FROZEN-FROZEN-1024] 2026-02-24T22:34:17.081 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-FROZEN-FROZEN-1048576] 2026-02-24T22:34:17.081 INFO:teuthology.orchestra.run.trial046.stdout:ERROR s3tests/functional/test_s3.py::test_copy_enc[sse-kms-sse-kms-FROZEN-FROZEN-8388608] 2026-02-24T22:34:17.081 INFO:teuthology.orchestra.run.trial046.stdout:= 1 failed, 510 passed, 572 deselected, 1127 warnings, 618 errors in 635.11s (0:10:35) = 2026-02-24T22:34:17.081 INFO:teuthology.orchestra.run.trial046.stdout:py: exit 1 (635.43 seconds) /home/ubuntu/cephtest/s3-tests-client.0> pytest -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3 and not bucket_logging' pid=59737 2026-02-24T22:34:17.081 INFO:teuthology.orchestra.run.trial046.stdout: py: FAIL code 1 (641.70=setup[6.27]+cmd[635.43] seconds) 2026-02-24T22:34:17.081 INFO:teuthology.orchestra.run.trial046.stdout: evaluation failed :( (641.71 seconds) 2026-02-24T22:34:17.081 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/contextutil.py", line 30, in nested vars.append(enter()) ^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_samarahu_ceph_f7e05f4fdea5af173284f247d6b0935b337d4ccf/qa/tasks/s3tests.py", line 503, in run_tests toxvenv_sh(ctx, remote, args, label="s3 tests against rgw") File "/home/teuthworker/src/github.com_samarahu_ceph_f7e05f4fdea5af173284f247d6b0935b337d4ccf/qa/tasks/s3tests.py", line 444, in toxvenv_sh return remote.sh(['source', activate, run.Raw('&&')] + args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) ^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (s3 tests against rgw) on trial046 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3 and not bucket_logging'" 2026-02-24T22:34:17.082 INFO:tasks.s3tests:Cleaning up boto... 2026-02-24T22:34:17.082 DEBUG:teuthology.orchestra.run.trial046:> rm /home/ubuntu/cephtest/boto-client.0.cfg 2026-02-24T22:34:17.086 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid foo.client.0 --purge-data --cluster ceph 2026-02-24T22:34:17.143 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:34:17.143 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:34:17.158 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.171+0000 7ff078da8a80 4 Realm: () 2026-02-24T22:34:17.158 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.171+0000 7ff078da8a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:34:17.158 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.171+0000 7ff078da8a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:34:17.158 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.171+0000 7ff078da8a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:34:17.158 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.171+0000 7ff078da8a80 20 zonegroup default 2026-02-24T22:34:17.159 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.171+0000 7ff078da8a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:17.160 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.171+0000 7ff078da8a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:34:17.160 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.171+0000 7ff078da8a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:17.161 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.171+0000 7ff078da8a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:34:17.161 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.171+0000 7ff078da8a80 20 started sync module instance, tier type = 2026-02-24T22:34:17.161 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.171+0000 7ff078da8a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:34:17.162 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.171+0000 7ff07687c640 0 -- 10.20.193.46:0/3246684274 >> [v2:10.20.193.110:6808/884480935,v1:10.20.193.110:6809/884480935] conn(0x55ac633d76e0 msgr2=0x55ac633f7b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:34:17.163 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.175+0000 7ff078da8a80 20 add_watcher() i=3 2026-02-24T22:34:17.163 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.175+0000 7ff078da8a80 20 add_watcher() i=0 2026-02-24T22:34:17.163 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.175+0000 7ff078da8a80 20 add_watcher() i=6 2026-02-24T22:34:17.163 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.175+0000 7ff078da8a80 20 add_watcher() i=5 2026-02-24T22:34:17.163 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.175+0000 7ff078da8a80 20 add_watcher() i=2 2026-02-24T22:34:17.163 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.175+0000 7ff078da8a80 20 add_watcher() i=1 2026-02-24T22:34:17.164 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.175+0000 7ff078da8a80 20 add_watcher() i=4 2026-02-24T22:34:17.164 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.175+0000 7ff078da8a80 20 add_watcher() i=7 2026-02-24T22:34:17.164 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.175+0000 7ff078da8a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:34:17.164 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.175+0000 7ff07607b640 0 -- 10.20.193.46:0/4033274809 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7fef98008af0 msgr2=0x7fef98028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:34:17.165 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.175+0000 7ff078da8a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:34:17.166 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.175+0000 7ff078da8a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:34:17.189 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.199+0000 7ff024fe1640 20 reqs_thread_entry: start 2026-02-24T22:34:17.224 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.235+0000 7ff078da8a80 20 init_complete bucket index max shards: 1 2026-02-24T22:34:17.224 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.235+0000 7ff078da8a80 20 Filter name: none 2026-02-24T22:34:17.224 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.235+0000 7fef8effd640 20 reqs_thread_entry: start 2026-02-24T22:34:17.224 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.235+0000 7ff078da8a80 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2026-02-24T22:34:17.224 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.235+0000 7ff078da8a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:17.225 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.235+0000 7ff078da8a80 20 rados_obj.operate() r=0 bl.length=474 2026-02-24T22:34:17.225 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.235+0000 7ff078da8a80 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x13 2026-02-24T22:34:17.225 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.235+0000 7ff078da8a80 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2026-02-24T22:34:17.225 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.235+0000 7ff078da8a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:34:17.225 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.235+0000 7ff078da8a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-e7n3yd9d2xql4uuphll37swt-435:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958 : miss 2026-02-24T22:34:17.225 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.235+0000 7ff078da8a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:17.226 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.235+0000 7ff078da8a80 20 rados_obj.operate() r=0 bl.length=447 2026-02-24T22:34:17.226 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.235+0000 7ff078da8a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-e7n3yd9d2xql4uuphll37swt-435:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958 info.flags=0x17 2026-02-24T22:34:17.226 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.235+0000 7ff078da8a80 10 adding default.rgw.meta+root+.bucket.meta.test-e7n3yd9d2xql4uuphll37swt-435:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958 to cache LRU end 2026-02-24T22:34:17.226 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.235+0000 7ff078da8a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:34:17.226 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.235+0000 7ff078da8a80 10 updating xattr: name=user.rgw.acl bl.length()=183 2026-02-24T22:34:17.226 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.235+0000 7ff078da8a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-e7n3yd9d2xql4uuphll37swt-435:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958 2026-02-24T22:34:17.227 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.239+0000 7ff078da8a80 10 cls_bucket_list_unordered: got myobj[w02dSU3ENzxHmZfDddltBf8ffRtQHQh] 2026-02-24T22:34:17.227 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.239+0000 7ff078da8a80 10 cls_bucket_list_unordered: got myobj[cQa-Olu8or8z1ueBPUqzLTsQgZ0e.2.] 2026-02-24T22:34:17.227 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.239+0000 7ff078da8a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958.0 2026-02-24T22:34:17.227 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.239+0000 7ff078da8a80 20 get_obj_state: octx=0x55ac633c6190 obj=test-e7n3yd9d2xql4uuphll37swt-435:myobj state=0x55ac633cc958 s->prefetch_data=0 2026-02-24T22:34:17.227 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.239+0000 7ff078da8a80 20 get_obj_state: setting s->obj_tag to rcnw9qgizqnbhdbxib1njrsg868aximh 2026-02-24T22:34:17.227 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.239+0000 7ff078da8a80 20 get_obj_state_impl: setting s->olh_tag to yne0c966g9chwzex805aadbeq0l6iydf 2026-02-24T22:34:17.227 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.239+0000 7ff078da8a80 20 bucket_index_guard_olh_op(): olh_state.olh_tag=yne0c966g9chwzex805aadbeq0l6iydf 2026-02-24T22:34:17.228 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.239+0000 7ff078da8a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958.0 2026-02-24T22:34:17.229 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.239+0000 7ff078da8a80 0 ERROR: bucket shard callback failed. obj=myobj[w02dSU3ENzxHmZfDddltBf8ffRtQHQh]. ret=(2) No such file or directory 2026-02-24T22:34:17.229 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.239+0000 7ff078da8a80 20 rgw_rados_operate() after cls_rgw_bucket_link_instance() returned r=-2 2026-02-24T22:34:17.229 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.239+0000 7ff078da8a80 20 bucket_index_guard_olh_op(): olh_state.olh_tag=yne0c966g9chwzex805aadbeq0l6iydf 2026-02-24T22:34:17.230 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.239+0000 7ff078da8a80 20 bucket_index_unlink_instance() target_obj=test-e7n3yd9d2xql4uuphll37swt-435:_:w02dSU3ENzxHmZfDddltBf8ffRtQHQh_myobj returned -2 2026-02-24T22:34:17.230 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.239+0000 7ff078da8a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958.0 2026-02-24T22:34:17.230 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.239+0000 7ff078da8a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958.0 2026-02-24T22:34:17.230 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.243+0000 7ff078da8a80 20 list_objects_ordered: starting attempt 1 2026-02-24T22:34:17.231 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.243+0000 7ff078da8a80 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2026-02-24T22:34:17.232 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.243+0000 7ff078da8a80 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2026-02-24T22:34:17.232 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.243+0000 7ff078da8a80 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2026-02-24T22:34:17.232 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.243+0000 7ff078da8a80 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2026-02-24T22:34:17.232 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.243+0000 7ff078da8a80 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2026-02-24T22:34:17.233 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.243+0000 7ff078da8a80 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-e7n3yd9d2xql4uuphll37swt-435[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958])"; uploads.size()=0; is_truncated=0 2026-02-24T22:34:17.233 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.243+0000 7ff078da8a80 10 cache get: name=default.rgw.meta+root+test-e7n3yd9d2xql4uuphll37swt-435 : miss 2026-02-24T22:34:17.233 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.243+0000 7ff078da8a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:17.234 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.243+0000 7ff078da8a80 20 rados_obj.operate() r=0 bl.length=217 2026-02-24T22:34:17.234 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.243+0000 7ff078da8a80 10 cache put: name=default.rgw.meta+root+test-e7n3yd9d2xql4uuphll37swt-435 info.flags=0x11 2026-02-24T22:34:17.234 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.243+0000 7ff078da8a80 10 adding default.rgw.meta+root+test-e7n3yd9d2xql4uuphll37swt-435 to cache LRU end 2026-02-24T22:34:17.234 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.247+0000 7ff078da8a80 10 removing default.rgw.meta+root+test-e7n3yd9d2xql4uuphll37swt-435 from cache 2026-02-24T22:34:17.235 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.247+0000 7ff078da8a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:test-e7n3yd9d2xql4uuphll37swt-435, ofs0, ns] 2026-02-24T22:34:17.235 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.247+0000 7ff052ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029798645 cookie 94198885447952 notifier 9324 bl.length()=177 2026-02-24T22:34:17.236 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.247+0000 7ff078da8a80 10 removing default.rgw.meta+root+.bucket.meta.test-e7n3yd9d2xql4uuphll37swt-435:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958 from cache 2026-02-24T22:34:17.236 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.247+0000 7ff078da8a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-e7n3yd9d2xql4uuphll37swt-435:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958, ofs0, ns] 2026-02-24T22:34:17.236 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.247+0000 7ff0527fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029797455 cookie 94198885451632 notifier 9324 bl.length()=237 2026-02-24T22:34:17.237 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.247+0000 7ff078da8a80 10 cache get: name=default.rgw.log++pubsub..bucket.test-e7n3yd9d2xql4uuphll37swt-435/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958 : miss 2026-02-24T22:34:17.238 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.247+0000 7ff078da8a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:17.238 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.247+0000 7ff078da8a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:34:17.238 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.247+0000 7ff078da8a80 10 cache put: name=default.rgw.log++pubsub..bucket.test-e7n3yd9d2xql4uuphll37swt-435/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958 info.flags=0x0 2026-02-24T22:34:17.238 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.247+0000 7ff078da8a80 10 adding default.rgw.log++pubsub..bucket.test-e7n3yd9d2xql4uuphll37swt-435/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958 to cache LRU end 2026-02-24T22:34:17.239 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.251+0000 7ff078da8a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.testbuck2:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1964 : miss 2026-02-24T22:34:17.239 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.251+0000 7ff078da8a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:17.240 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.251+0000 7ff078da8a80 20 rados_obj.operate() r=0 bl.length=423 2026-02-24T22:34:17.240 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.251+0000 7ff078da8a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.testbuck2:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1964 info.flags=0x17 2026-02-24T22:34:17.240 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.251+0000 7ff078da8a80 10 adding default.rgw.meta+root+.bucket.meta.testbuck2:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1964 to cache LRU end 2026-02-24T22:34:17.240 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.251+0000 7ff078da8a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:34:17.240 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.251+0000 7ff078da8a80 10 updating xattr: name=user.rgw.acl bl.length()=183 2026-02-24T22:34:17.241 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.251+0000 7ff078da8a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.testbuck2:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1964 2026-02-24T22:34:17.241 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.251+0000 7ff078da8a80 10 cls_bucket_list_unordered: got AED138 2026-02-24T22:34:17.241 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.251+0000 7ff078da8a80 10 cls_bucket_list_unordered: got F6A897 2026-02-24T22:34:17.241 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.251+0000 7ff078da8a80 20 get_obj_state: octx=0x55ac633c6190 obj=testbuck2:AED138 state=0x55ac633cb368 s->prefetch_data=0 2026-02-24T22:34:17.241 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.251+0000 7ff078da8a80 10 manifest: total_size = 174681 2026-02-24T22:34:17.241 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.251+0000 7ff078da8a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15267829849310681377 2026-02-24T22:34:17.242 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.251+0000 7ff078da8a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1964.0 2026-02-24T22:34:17.242 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.251+0000 7ff078da8a80 20 get_obj_state: octx=0x55ac633c6190 obj=testbuck2:AED138 state=0x55ac633cb368 s->prefetch_data=0 2026-02-24T22:34:17.242 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.251+0000 7ff078da8a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:34:17.243 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.251+0000 7ff078da8a80 20 RGWObjManifest::operator++(): result: ofs=174681 stripe_ofs=174681 part_ofs=0 rule->part_size=0 2026-02-24T22:34:17.243 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.251+0000 7ff078da8a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15267829849310681377 2026-02-24T22:34:17.243 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.251+0000 7ff078da8a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:34:17.243 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.255+0000 7ff078da8a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1964__shadow_.EgZHLlGqNWiHRvYELTP37SP-5THTj9F_0 2026-02-24T22:34:17.243 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.255+0000 7ff078da8a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 335 2026-02-24T22:34:17.243 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.255+0000 7ff078da8a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:34:17.243 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.255+0000 7ff078da8a80 20 garbage collection: RGWGC::send_chain - on object name: gc.23tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15267829849310681377 2026-02-24T22:34:17.244 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.255+0000 7ff052ffd640 20 handle_completion(): completion ok for obj=AED138 2026-02-24T22:34:17.244 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.255+0000 7ff078da8a80 20 get_obj_state: octx=0x55ac633c6190 obj=testbuck2:F6A897 state=0x55ac632b98c8 s->prefetch_data=0 2026-02-24T22:34:17.244 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.255+0000 7ff078da8a80 10 manifest: total_size = 493167 2026-02-24T22:34:17.244 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.255+0000 7ff078da8a80 20 get_obj_state: setting s->obj_tag to 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17066240684055347104 2026-02-24T22:34:17.244 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.255+0000 7ff078da8a80 20 bucket index object: :.dir.301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1964.0 2026-02-24T22:34:17.245 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.255+0000 7ff078da8a80 20 get_obj_state: octx=0x55ac633c6190 obj=testbuck2:F6A897 state=0x55ac632b98c8 s->prefetch_data=0 2026-02-24T22:34:17.246 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.255+0000 7ff078da8a80 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2026-02-24T22:34:17.246 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.255+0000 7ff078da8a80 20 RGWObjManifest::operator++(): result: ofs=493167 stripe_ofs=493167 part_ofs=0 rule->part_size=0 2026-02-24T22:34:17.246 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.255+0000 7ff078da8a80 20 garbage collection: RGWGC::send_split_chain - tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17066240684055347104 2026-02-24T22:34:17.246 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.255+0000 7ff078da8a80 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2026-02-24T22:34:17.246 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.255+0000 7ff078da8a80 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1964__shadow_.0zw8njJRZg9oS7Yphhj1liLdE2GBs6p_0 2026-02-24T22:34:17.246 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.255+0000 7ff078da8a80 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 335 2026-02-24T22:34:17.246 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.255+0000 7ff078da8a80 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2026-02-24T22:34:17.246 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.255+0000 7ff078da8a80 20 garbage collection: RGWGC::send_chain - on object name: gc.27tag is: 301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17066240684055347104 2026-02-24T22:34:17.247 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.259+0000 7ff0527fc640 20 handle_completion(): completion ok for obj=F6A897 2026-02-24T22:34:17.247 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.259+0000 7ff078da8a80 20 list_objects_ordered: starting attempt 1 2026-02-24T22:34:17.247 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.259+0000 7ff078da8a80 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2026-02-24T22:34:17.247 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.259+0000 7ff078da8a80 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2026-02-24T22:34:17.247 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.259+0000 7ff078da8a80 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2026-02-24T22:34:17.247 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.259+0000 7ff078da8a80 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2026-02-24T22:34:17.247 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.259+0000 7ff078da8a80 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2026-02-24T22:34:17.247 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.259+0000 7ff078da8a80 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":testbuck2[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1964])"; uploads.size()=0; is_truncated=0 2026-02-24T22:34:17.248 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.259+0000 7ff078da8a80 10 cache get: name=default.rgw.meta+root+testbuck2 : miss 2026-02-24T22:34:17.248 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.259+0000 7ff078da8a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:17.248 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.259+0000 7ff078da8a80 20 rados_obj.operate() r=0 bl.length=193 2026-02-24T22:34:17.248 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.259+0000 7ff078da8a80 10 cache put: name=default.rgw.meta+root+testbuck2 info.flags=0x11 2026-02-24T22:34:17.249 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.259+0000 7ff078da8a80 10 adding default.rgw.meta+root+testbuck2 to cache LRU end 2026-02-24T22:34:17.249 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.259+0000 7ff078da8a80 10 removing default.rgw.meta+root+testbuck2 from cache 2026-02-24T22:34:17.249 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.259+0000 7ff078da8a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:testbuck2, ofs0, ns] 2026-02-24T22:34:17.249 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.259+0000 7ff052ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029797456 cookie 94198885451632 notifier 9324 bl.length()=153 2026-02-24T22:34:17.250 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.263+0000 7ff078da8a80 10 removing default.rgw.meta+root+.bucket.meta.testbuck2:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1964 from cache 2026-02-24T22:34:17.250 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.263+0000 7ff078da8a80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.testbuck2:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1964, ofs0, ns] 2026-02-24T22:34:17.251 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.263+0000 7ff0527fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029796594 cookie 94198887717296 notifier 9324 bl.length()=213 2026-02-24T22:34:17.252 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.263+0000 7ff078da8a80 10 cache get: name=default.rgw.log++pubsub..bucket.testbuck2/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1964 : miss 2026-02-24T22:34:17.252 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.263+0000 7ff078da8a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:17.252 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.263+0000 7ff078da8a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:34:17.252 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.263+0000 7ff078da8a80 10 cache put: name=default.rgw.log++pubsub..bucket.testbuck2/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1964 info.flags=0x0 2026-02-24T22:34:17.252 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.263+0000 7ff078da8a80 10 adding default.rgw.log++pubsub..bucket.testbuck2/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1964 to cache LRU end 2026-02-24T22:34:17.253 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.263+0000 7ff078da8a80 10 removing key index: MFYBMLOBGXKEVCZKEMRM 2026-02-24T22:34:17.254 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.263+0000 7ff078da8a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.keys:MFYBMLOBGXKEVCZKEMRM, ofs0, ns] 2026-02-24T22:34:17.254 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.267+0000 7ff0527fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029797457 cookie 94198885451632 notifier 9324 bl.length()=170 2026-02-24T22:34:17.255 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.267+0000 7ff078da8a80 10 removing email index: foo+test@test.test 2026-02-24T22:34:17.255 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.267+0000 7ff078da8a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.email:foo+test@test.test, ofs0, ns] 2026-02-24T22:34:17.256 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.267+0000 7ff052ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029797458 cookie 94198885451632 notifier 9324 bl.length()=169 2026-02-24T22:34:17.256 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.267+0000 7ff078da8a80 10 removing user buckets index 2026-02-24T22:34:17.257 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.267+0000 7ff078da8a80 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:users.uid:foo.client.0.buckets, ofs0, ns] 2026-02-24T22:34:17.257 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.267+0000 7ff0527fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029796704 cookie 94198887723920 notifier 9324 bl.length()=169 2026-02-24T22:34:17.258 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.267+0000 7ff078da8a80 10 removing user index: foo.client.0 2026-02-24T22:34:17.258 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.271+0000 7ff078da8a80 10 removing default.rgw.meta+users.uid+foo.client.0 from cache 2026-02-24T22:34:17.258 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.271+0000 7ff078da8a80 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:users.uid:foo.client.0, ofs0, ns] 2026-02-24T22:34:17.259 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.271+0000 7ff052ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029798646 cookie 94198887712944 notifier 9324 bl.length()=161 2026-02-24T22:34:17.262 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.271+0000 7ff078da8a80 20 remove_watcher() i=3 2026-02-24T22:34:17.262 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.271+0000 7ff078da8a80 2 removed watcher, disabling cache 2026-02-24T22:34:17.262 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.271+0000 7ff078da8a80 20 remove_watcher() i=0 2026-02-24T22:34:17.262 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.271+0000 7ff078da8a80 20 remove_watcher() i=1 2026-02-24T22:34:17.262 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.271+0000 7ff078da8a80 20 remove_watcher() i=6 2026-02-24T22:34:17.263 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.271+0000 7ff078da8a80 20 remove_watcher() i=2 2026-02-24T22:34:17.263 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.271+0000 7ff078da8a80 20 remove_watcher() i=4 2026-02-24T22:34:17.263 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.271+0000 7ff078da8a80 20 remove_watcher() i=7 2026-02-24T22:34:17.263 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.271+0000 7ff078da8a80 20 remove_watcher() i=5 2026-02-24T22:34:17.270 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid bar.client.0 --purge-data --cluster ceph 2026-02-24T22:34:17.328 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:34:17.328 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:34:17.344 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.355+0000 7fe4f237ea80 4 Realm: () 2026-02-24T22:34:17.344 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.355+0000 7fe4f237ea80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:34:17.344 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.355+0000 7fe4f237ea80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:34:17.344 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.355+0000 7fe4f237ea80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:34:17.344 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.355+0000 7fe4f237ea80 20 zonegroup default 2026-02-24T22:34:17.345 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.355+0000 7fe4f237ea80 20 rados->read ofs=0 len=0 2026-02-24T22:34:17.346 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.355+0000 7fe4f237ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:34:17.346 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.355+0000 7fe4f237ea80 20 rados->read ofs=0 len=0 2026-02-24T22:34:17.347 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.359+0000 7fe4f237ea80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:34:17.347 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.359+0000 7fe4f237ea80 20 started sync module instance, tier type = 2026-02-24T22:34:17.347 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.359+0000 7fe4f237ea80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:34:17.348 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.359+0000 7fe4eb7fe640 0 -- 10.20.193.46:0/3979219541 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x559bb0fe6250 msgr2=0x559bb0fe66f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:34:17.348 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.359+0000 7fe4f237ea80 20 add_watcher() i=1 2026-02-24T22:34:17.349 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.359+0000 7fe4f237ea80 20 add_watcher() i=3 2026-02-24T22:34:17.349 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.359+0000 7fe4f237ea80 20 add_watcher() i=0 2026-02-24T22:34:17.349 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.359+0000 7fe4f237ea80 20 add_watcher() i=2 2026-02-24T22:34:17.350 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.359+0000 7fe4f237ea80 20 add_watcher() i=7 2026-02-24T22:34:17.350 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.359+0000 7fe4f237ea80 20 add_watcher() i=6 2026-02-24T22:34:17.350 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.359+0000 7fe4f237ea80 20 add_watcher() i=4 2026-02-24T22:34:17.350 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.359+0000 7fe4f237ea80 20 add_watcher() i=5 2026-02-24T22:34:17.350 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.359+0000 7fe4f237ea80 2 all 8 watchers are set, enabling cache 2026-02-24T22:34:17.351 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.363+0000 7fe4f237ea80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:34:17.351 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.363+0000 7fe4f237ea80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:34:17.351 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.363+0000 7fe4e3fff640 0 -- 10.20.193.46:0/3979219541 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x559bb0fee830 msgr2=0x559bb0feeca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:34:17.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.383+0000 7fe49cfe1640 20 reqs_thread_entry: start 2026-02-24T22:34:17.406 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.415+0000 7fe4f237ea80 20 init_complete bucket index max shards: 1 2026-02-24T22:34:17.406 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.415+0000 7fe4f237ea80 20 Filter name: none 2026-02-24T22:34:17.406 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.419+0000 7fe406ffd640 20 reqs_thread_entry: start 2026-02-24T22:34:17.407 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.419+0000 7fe4f237ea80 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2026-02-24T22:34:17.407 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.419+0000 7fe4f237ea80 20 rados->read ofs=0 len=0 2026-02-24T22:34:17.407 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.419+0000 7fe4f237ea80 20 rados_obj.operate() r=0 bl.length=474 2026-02-24T22:34:17.407 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.419+0000 7fe4f237ea80 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x13 2026-02-24T22:34:17.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.419+0000 7fe4f237ea80 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2026-02-24T22:34:17.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.419+0000 7fe4f237ea80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:34:17.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.419+0000 7fe4f237ea80 10 updating xattr: name=user.rgw.user-policy bl.length()=4 2026-02-24T22:34:17.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.419+0000 7fe4f237ea80 10 removing key index: NJNOTGSXWYZXXIZADMEE 2026-02-24T22:34:17.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.419+0000 7fe4f237ea80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.keys:NJNOTGSXWYZXXIZADMEE, ofs0, ns] 2026-02-24T22:34:17.409 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.419+0000 7fe4c67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029797459 cookie 94127177784304 notifier 8678 bl.length()=170 2026-02-24T22:34:17.409 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.419+0000 7fe4f237ea80 10 removing email index: bar+test@test.test 2026-02-24T22:34:17.409 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.419+0000 7fe4f237ea80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.email:bar+test@test.test, ofs0, ns] 2026-02-24T22:34:17.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.419+0000 7fe4c67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029798051 cookie 94127178072160 notifier 8678 bl.length()=169 2026-02-24T22:34:17.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.419+0000 7fe4f237ea80 10 removing user buckets index 2026-02-24T22:34:17.411 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.423+0000 7fe4f237ea80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.uid:bar.client.0.buckets, ofs0, ns] 2026-02-24T22:34:17.411 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.423+0000 7fe4c67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029796463 cookie 94127178075440 notifier 8678 bl.length()=169 2026-02-24T22:34:17.411 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.423+0000 7fe4f237ea80 10 removing user index: bar.client.0 2026-02-24T22:34:17.412 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.423+0000 7fe4f237ea80 10 removing default.rgw.meta+users.uid+bar.client.0 from cache 2026-02-24T22:34:17.412 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.423+0000 7fe4f237ea80 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:users.uid:bar.client.0, ofs0, ns] 2026-02-24T22:34:17.412 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.423+0000 7fe4c6ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029796705 cookie 94127180176096 notifier 8678 bl.length()=161 2026-02-24T22:34:17.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.427+0000 7fe4f237ea80 20 remove_watcher() i=3 2026-02-24T22:34:17.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.427+0000 7fe4f237ea80 2 removed watcher, disabling cache 2026-02-24T22:34:17.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.427+0000 7fe4f237ea80 20 remove_watcher() i=4 2026-02-24T22:34:17.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.427+0000 7fe4f237ea80 20 remove_watcher() i=2 2026-02-24T22:34:17.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.427+0000 7fe4f237ea80 20 remove_watcher() i=6 2026-02-24T22:34:17.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.427+0000 7fe4f237ea80 20 remove_watcher() i=0 2026-02-24T22:34:17.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.427+0000 7fe4f237ea80 20 remove_watcher() i=5 2026-02-24T22:34:17.417 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.427+0000 7fe4f237ea80 20 remove_watcher() i=1 2026-02-24T22:34:17.417 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.427+0000 7fe4f237ea80 20 remove_watcher() i=7 2026-02-24T22:34:17.423 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid 'testx$tenanteduser.client.0' --purge-data --cluster ceph 2026-02-24T22:34:17.484 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:34:17.484 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:34:17.499 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.511+0000 7f3bbdda0a80 4 Realm: () 2026-02-24T22:34:17.499 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.511+0000 7f3bbdda0a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:34:17.499 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.511+0000 7f3bbdda0a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:34:17.499 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.511+0000 7f3bbdda0a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:34:17.499 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.511+0000 7f3bbdda0a80 20 zonegroup default 2026-02-24T22:34:17.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.511+0000 7f3bbdda0a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:17.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.511+0000 7f3bbdda0a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:34:17.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.511+0000 7f3bbdda0a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:17.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.511+0000 7f3bbdda0a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:34:17.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.511+0000 7f3bbdda0a80 20 started sync module instance, tier type = 2026-02-24T22:34:17.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.511+0000 7f3bbdda0a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:34:17.502 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.511+0000 7f3bb7fff640 0 -- 10.20.193.46:0/3418644357 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55798d6f7220 msgr2=0x55798d6f7690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:34:17.502 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.511+0000 7f3bbdda0a80 20 add_watcher() i=1 2026-02-24T22:34:17.503 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.515+0000 7f3bbdda0a80 20 add_watcher() i=5 2026-02-24T22:34:17.503 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.515+0000 7f3bbdda0a80 20 add_watcher() i=3 2026-02-24T22:34:17.503 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.515+0000 7f3bbdda0a80 20 add_watcher() i=0 2026-02-24T22:34:17.503 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.515+0000 7f3bbdda0a80 20 add_watcher() i=6 2026-02-24T22:34:17.503 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.515+0000 7f3bbdda0a80 20 add_watcher() i=4 2026-02-24T22:34:17.504 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.515+0000 7f3bbdda0a80 20 add_watcher() i=7 2026-02-24T22:34:17.504 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.515+0000 7f3bbdda0a80 20 add_watcher() i=2 2026-02-24T22:34:17.504 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.515+0000 7f3bbdda0a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:34:17.504 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.515+0000 7f3bbdda0a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:34:17.504 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.515+0000 7f3bbdda0a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:34:17.505 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.515+0000 7f3bb6ffd640 0 -- 10.20.193.46:0/3418644357 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x55798d6fccf0 msgr2=0x55798d6fd160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:34:17.527 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.539+0000 7f3b68fe1640 20 reqs_thread_entry: start 2026-02-24T22:34:17.561 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.571+0000 7f3bbdda0a80 20 init_complete bucket index max shards: 1 2026-02-24T22:34:17.561 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.571+0000 7f3bbdda0a80 20 Filter name: none 2026-02-24T22:34:17.561 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.571+0000 7f3ad2ffd640 20 reqs_thread_entry: start 2026-02-24T22:34:17.561 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.571+0000 7f3bbdda0a80 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2026-02-24T22:34:17.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.571+0000 7f3bbdda0a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:17.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.571+0000 7f3bbdda0a80 20 rados_obj.operate() r=0 bl.length=533 2026-02-24T22:34:17.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.571+0000 7f3bbdda0a80 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x13 2026-02-24T22:34:17.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.571+0000 7f3bbdda0a80 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2026-02-24T22:34:17.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.571+0000 7f3bbdda0a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:34:17.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.571+0000 7f3bbdda0a80 10 removing key index: UQSXPKRUJTAZNKGXUOWD 2026-02-24T22:34:17.563 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.575+0000 7f3bbdda0a80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.keys:UQSXPKRUJTAZNKGXUOWD, ofs0, ns] 2026-02-24T22:34:17.563 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.575+0000 7f3b927fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029798052 cookie 93980553121584 notifier 8696 bl.length()=170 2026-02-24T22:34:17.563 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.575+0000 7f3bbdda0a80 10 removing email index: testx$tenanteduser+test@test.test 2026-02-24T22:34:17.564 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.575+0000 7f3bbdda0a80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:users.email:testx$tenanteduser+test@test.test, ofs0, ns] 2026-02-24T22:34:17.564 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.575+0000 7f3b92ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029796595 cookie 93980554566624 notifier 8696 bl.length()=184 2026-02-24T22:34:17.565 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.575+0000 7f3bbdda0a80 10 removing user buckets index 2026-02-24T22:34:17.565 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.575+0000 7f3bbdda0a80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:testx$tenanteduser.client.0.buckets, ofs0, ns] 2026-02-24T22:34:17.566 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.575+0000 7f3b927fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029796623 cookie 93980554432464 notifier 8696 bl.length()=184 2026-02-24T22:34:17.566 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.575+0000 7f3bbdda0a80 10 removing user index: testx$tenanteduser.client.0 2026-02-24T22:34:17.567 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.579+0000 7f3bbdda0a80 10 removing default.rgw.meta+users.uid+testx$tenanteduser.client.0 from cache 2026-02-24T22:34:17.567 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.579+0000 7f3bbdda0a80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:testx$tenanteduser.client.0, ofs0, ns] 2026-02-24T22:34:17.567 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.579+0000 7f3b92ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029798053 cookie 93980553121584 notifier 8696 bl.length()=176 2026-02-24T22:34:17.570 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.579+0000 7f3bbdda0a80 20 remove_watcher() i=3 2026-02-24T22:34:17.571 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.579+0000 7f3bbdda0a80 2 removed watcher, disabling cache 2026-02-24T22:34:17.571 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.579+0000 7f3bbdda0a80 20 remove_watcher() i=2 2026-02-24T22:34:17.571 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.579+0000 7f3bbdda0a80 20 remove_watcher() i=6 2026-02-24T22:34:17.571 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.579+0000 7f3bbdda0a80 20 remove_watcher() i=4 2026-02-24T22:34:17.571 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.583+0000 7f3bbdda0a80 20 remove_watcher() i=0 2026-02-24T22:34:17.571 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.583+0000 7f3bbdda0a80 20 remove_watcher() i=1 2026-02-24T22:34:17.571 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.583+0000 7f3bbdda0a80 20 remove_watcher() i=5 2026-02-24T22:34:17.571 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.583+0000 7f3bbdda0a80 20 remove_watcher() i=7 2026-02-24T22:34:17.577 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid foobar.client.0 --purge-data --cluster ceph 2026-02-24T22:34:17.636 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:34:17.636 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:34:17.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.663+0000 7f65f3f1fa80 4 Realm: () 2026-02-24T22:34:17.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.663+0000 7f65f3f1fa80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:34:17.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.663+0000 7f65f3f1fa80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:34:17.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.663+0000 7f65f3f1fa80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:34:17.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.663+0000 7f65f3f1fa80 20 zonegroup default 2026-02-24T22:34:17.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.663+0000 7f65f3f1fa80 20 rados->read ofs=0 len=0 2026-02-24T22:34:17.653 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.663+0000 7f65f3f1fa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:34:17.653 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.663+0000 7f65f3f1fa80 20 rados->read ofs=0 len=0 2026-02-24T22:34:17.653 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.663+0000 7f65f3f1fa80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:34:17.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.663+0000 7f65f3f1fa80 20 started sync module instance, tier type = 2026-02-24T22:34:17.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.663+0000 7f65f3f1fa80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:34:17.655 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.667+0000 7f65f19f3640 0 -- 10.20.193.46:0/2543855257 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x563c1a024ed0 msgr2=0x563c1a025340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:34:17.655 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.667+0000 7f65f3f1fa80 20 add_watcher() i=1 2026-02-24T22:34:17.656 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.667+0000 7f65f3f1fa80 20 add_watcher() i=6 2026-02-24T22:34:17.656 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.667+0000 7f65f3f1fa80 20 add_watcher() i=3 2026-02-24T22:34:17.656 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.667+0000 7f65f3f1fa80 20 add_watcher() i=0 2026-02-24T22:34:17.656 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.667+0000 7f65f3f1fa80 20 add_watcher() i=5 2026-02-24T22:34:17.656 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.667+0000 7f65f3f1fa80 20 add_watcher() i=7 2026-02-24T22:34:17.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.667+0000 7f65f3f1fa80 20 add_watcher() i=2 2026-02-24T22:34:17.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.667+0000 7f65f3f1fa80 20 add_watcher() i=4 2026-02-24T22:34:17.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.667+0000 7f65f3f1fa80 2 all 8 watchers are set, enabling cache 2026-02-24T22:34:17.658 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.667+0000 7f65f11f2640 0 -- 10.20.193.46:0/2911812542 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7f6510008af0 msgr2=0x7f6510028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:34:17.658 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.671+0000 7f65f3f1fa80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:34:17.659 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.671+0000 7f65f3f1fa80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:34:17.680 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.691+0000 7f65227fc640 20 reqs_thread_entry: start 2026-02-24T22:34:17.715 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.727+0000 7f65f3f1fa80 20 init_complete bucket index max shards: 1 2026-02-24T22:34:17.715 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.727+0000 7f65f3f1fa80 20 Filter name: none 2026-02-24T22:34:17.715 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.727+0000 7f6520ff9640 20 reqs_thread_entry: start 2026-02-24T22:34:17.715 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.727+0000 7f65f3f1fa80 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2026-02-24T22:34:17.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.727+0000 7f65f3f1fa80 20 rados->read ofs=0 len=0 2026-02-24T22:34:17.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.727+0000 7f65f3f1fa80 20 rados_obj.operate() r=0 bl.length=539 2026-02-24T22:34:17.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.727+0000 7f65f3f1fa80 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x13 2026-02-24T22:34:17.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.727+0000 7f65f3f1fa80 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2026-02-24T22:34:17.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.727+0000 7f65f3f1fa80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:34:17.717 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.727+0000 7f65f3f1fa80 10 removing key index: TUMJWNTVMIGYQUKJBAVA 2026-02-24T22:34:17.717 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.727+0000 7f65f3f1fa80 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.keys:TUMJWNTVMIGYQUKJBAVA, ofs0, ns] 2026-02-24T22:34:17.717 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.727+0000 7f65ccff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029796624 cookie 94816136788960 notifier 9381 bl.length()=170 2026-02-24T22:34:17.717 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.727+0000 7f65f3f1fa80 10 removing email index: foobar+test@test.test 2026-02-24T22:34:17.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.727+0000 7f65f3f1fa80 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:users.email:foobar+test@test.test, ofs0, ns] 2026-02-24T22:34:17.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.727+0000 7f65cd7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029796706 cookie 94816136785760 notifier 9381 bl.length()=172 2026-02-24T22:34:17.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.731+0000 7f65f3f1fa80 10 removing user buckets index 2026-02-24T22:34:17.719 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.731+0000 7f65f3f1fa80 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:users.uid:foobar.client.0.buckets, ofs0, ns] 2026-02-24T22:34:17.720 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.731+0000 7f65cd7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029798647 cookie 94816134359008 notifier 9381 bl.length()=172 2026-02-24T22:34:17.720 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.731+0000 7f65f3f1fa80 10 removing user index: foobar.client.0 2026-02-24T22:34:17.720 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.731+0000 7f65f3f1fa80 10 removing default.rgw.meta+users.uid+foobar.client.0 from cache 2026-02-24T22:34:17.720 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.731+0000 7f65f3f1fa80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2026-02-24T22:34:17.721 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.731+0000 7f65ccff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029796596 cookie 94816136790032 notifier 9381 bl.length()=164 2026-02-24T22:34:17.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.735+0000 7f65f3f1fa80 20 remove_watcher() i=3 2026-02-24T22:34:17.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.735+0000 7f65f3f1fa80 2 removed watcher, disabling cache 2026-02-24T22:34:17.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.735+0000 7f65f3f1fa80 20 remove_watcher() i=0 2026-02-24T22:34:17.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.735+0000 7f65f3f1fa80 20 remove_watcher() i=2 2026-02-24T22:34:17.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.735+0000 7f65f3f1fa80 20 remove_watcher() i=4 2026-02-24T22:34:17.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.735+0000 7f65f3f1fa80 20 remove_watcher() i=6 2026-02-24T22:34:17.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.735+0000 7f65f3f1fa80 20 remove_watcher() i=5 2026-02-24T22:34:17.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.735+0000 7f65f3f1fa80 20 remove_watcher() i=1 2026-02-24T22:34:17.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.735+0000 7f65f3f1fa80 20 remove_watcher() i=7 2026-02-24T22:34:17.731 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid root1.client.0 --purge-data --cluster ceph 2026-02-24T22:34:17.793 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:34:17.793 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:34:17.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.819+0000 7f701b065a80 4 Realm: () 2026-02-24T22:34:17.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.819+0000 7f701b065a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:34:17.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.819+0000 7f701b065a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:34:17.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.819+0000 7f701b065a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:34:17.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.819+0000 7f701b065a80 20 zonegroup default 2026-02-24T22:34:17.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.819+0000 7f701b065a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:17.810 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.819+0000 7f701b065a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:34:17.810 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.819+0000 7f701b065a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:17.810 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.819+0000 7f701b065a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:34:17.810 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.819+0000 7f701b065a80 20 started sync module instance, tier type = 2026-02-24T22:34:17.810 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.819+0000 7f701b065a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:34:17.811 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.823+0000 7f7013fff640 0 -- 10.20.193.46:0/2409839282 >> [v2:10.20.193.46:6802/2359184024,v1:10.20.193.46:6803/2359184024] conn(0x561ff8ee2e00 msgr2=0x561ff8ee32a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:34:17.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.823+0000 7f701b065a80 20 add_watcher() i=3 2026-02-24T22:34:17.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.823+0000 7f701b065a80 20 add_watcher() i=6 2026-02-24T22:34:17.813 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.823+0000 7f701b065a80 20 add_watcher() i=1 2026-02-24T22:34:17.813 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.823+0000 7f701b065a80 20 add_watcher() i=5 2026-02-24T22:34:17.813 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.823+0000 7f701b065a80 20 add_watcher() i=0 2026-02-24T22:34:17.813 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.823+0000 7f701b065a80 20 add_watcher() i=2 2026-02-24T22:34:17.813 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.823+0000 7f701b065a80 20 add_watcher() i=7 2026-02-24T22:34:17.813 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.823+0000 7f701b065a80 20 add_watcher() i=4 2026-02-24T22:34:17.813 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.823+0000 7f701b065a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:34:17.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.823+0000 7f701b065a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:34:17.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.823+0000 7f701b065a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:34:17.815 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.827+0000 7f7013fff640 0 -- 10.20.193.46:0/2409839282 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x561ff8ee9d60 msgr2=0x561ff8eea1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:34:17.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.847+0000 7f6fc5fe3640 20 reqs_thread_entry: start 2026-02-24T22:34:17.869 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.879+0000 7f701b065a80 20 init_complete bucket index max shards: 1 2026-02-24T22:34:17.869 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.879+0000 7f701b065a80 20 Filter name: none 2026-02-24T22:34:17.869 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.879+0000 7f6f1ffff640 20 reqs_thread_entry: start 2026-02-24T22:34:17.869 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.879+0000 7f701b065a80 10 cache get: name=default.rgw.meta+users.uid+root1.client.0 : miss 2026-02-24T22:34:17.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.879+0000 7f701b065a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:17.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.879+0000 7f701b065a80 20 rados_obj.operate() r=0 bl.length=508 2026-02-24T22:34:17.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.879+0000 7f701b065a80 10 cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x13 2026-02-24T22:34:17.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.879+0000 7f701b065a80 10 adding default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2026-02-24T22:34:17.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.879+0000 7f701b065a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:34:17.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.879+0000 7f701b065a80 10 removing key index: YFRXIPWFCTLLKUCOAATR 2026-02-24T22:34:17.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.883+0000 7f701b065a80 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:users.keys:YFRXIPWFCTLLKUCOAATR, ofs0, ns] 2026-02-24T22:34:17.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.883+0000 7f6fe77fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029798648 cookie 94695320324064 notifier 9399 bl.length()=170 2026-02-24T22:34:17.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.883+0000 7f701b065a80 10 removing email index: root1+test@test.test 2026-02-24T22:34:17.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.883+0000 7f701b065a80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.email:root1+test@test.test, ofs0, ns] 2026-02-24T22:34:17.873 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.883+0000 7f6fe7fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029798054 cookie 94695320775664 notifier 9399 bl.length()=171 2026-02-24T22:34:17.873 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.883+0000 7f701b065a80 10 removing user index: root1.client.0 2026-02-24T22:34:17.873 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.883+0000 7f701b065a80 10 removing default.rgw.meta+users.uid+root1.client.0 from cache 2026-02-24T22:34:17.873 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.883+0000 7f701b065a80 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:users.uid:root1.client.0, ofs0, ns] 2026-02-24T22:34:17.874 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.883+0000 7f6fe7fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029796707 cookie 94695320343328 notifier 9399 bl.length()=163 2026-02-24T22:34:17.877 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.887+0000 7f701b065a80 20 remove_watcher() i=2 2026-02-24T22:34:17.877 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.887+0000 7f701b065a80 2 removed watcher, disabling cache 2026-02-24T22:34:17.877 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.887+0000 7f701b065a80 20 remove_watcher() i=5 2026-02-24T22:34:17.877 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.887+0000 7f701b065a80 20 remove_watcher() i=3 2026-02-24T22:34:17.877 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.887+0000 7f701b065a80 20 remove_watcher() i=4 2026-02-24T22:34:17.877 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.887+0000 7f701b065a80 20 remove_watcher() i=1 2026-02-24T22:34:17.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.887+0000 7f701b065a80 20 remove_watcher() i=6 2026-02-24T22:34:17.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.887+0000 7f701b065a80 20 remove_watcher() i=7 2026-02-24T22:34:17.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.887+0000 7f701b065a80 20 remove_watcher() i=0 2026-02-24T22:34:17.884 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph account rm --account-id RGW88888888888888888 --purge-data 2026-02-24T22:34:17.949 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:34:17.949 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:34:17.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.975+0000 7f4d5072da80 4 Realm: () 2026-02-24T22:34:17.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.975+0000 7f4d5072da80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:34:17.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.975+0000 7f4d5072da80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:34:17.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.975+0000 7f4d5072da80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:34:17.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.975+0000 7f4d5072da80 20 zonegroup default 2026-02-24T22:34:17.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.975+0000 7f4d5072da80 20 rados->read ofs=0 len=0 2026-02-24T22:34:17.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.979+0000 7f4d5072da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:34:17.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.979+0000 7f4d5072da80 20 rados->read ofs=0 len=0 2026-02-24T22:34:17.967 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.979+0000 7f4d5072da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:34:17.967 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.979+0000 7f4d5072da80 20 started sync module instance, tier type = 2026-02-24T22:34:17.967 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.979+0000 7f4d5072da80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:34:17.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.979+0000 7f4d4ea02640 0 -- 10.20.193.46:0/507070281 >> [v2:10.20.193.46:6810/3488974090,v1:10.20.193.46:6811/3488974090] conn(0x55f63cacd0d0 msgr2=0x55f63cacd540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:34:17.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.979+0000 7f4d5072da80 20 add_watcher() i=1 2026-02-24T22:34:17.969 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.979+0000 7f4d5072da80 20 add_watcher() i=6 2026-02-24T22:34:17.969 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.979+0000 7f4d5072da80 20 add_watcher() i=3 2026-02-24T22:34:17.970 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.979+0000 7f4d5072da80 20 add_watcher() i=4 2026-02-24T22:34:17.970 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.979+0000 7f4d5072da80 20 add_watcher() i=5 2026-02-24T22:34:17.970 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.979+0000 7f4d5072da80 20 add_watcher() i=7 2026-02-24T22:34:17.970 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.979+0000 7f4d5072da80 20 add_watcher() i=2 2026-02-24T22:34:17.970 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.979+0000 7f4d5072da80 20 add_watcher() i=0 2026-02-24T22:34:17.970 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.979+0000 7f4d5072da80 2 all 8 watchers are set, enabling cache 2026-02-24T22:34:17.971 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.983+0000 7f4d5072da80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:34:17.971 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.983+0000 7f4d5072da80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:34:17.971 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.983+0000 7f4d5072da80 5 note: GC not initialized 2026-02-24T22:34:17.971 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.983+0000 7f4c677fe640 20 reqs_thread_entry: start 2026-02-24T22:34:17.971 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:17.983+0000 7f4d4ea02640 0 -- 10.20.193.46:0/2315638855 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55f63cc900a0 msgr2=0x55f63cc90510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:34:18.214 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.223+0000 7f4d5072da80 20 init_complete bucket index max shards: 1 2026-02-24T22:34:18.214 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.223+0000 7f4d5072da80 20 Filter name: none 2026-02-24T22:34:18.214 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.223+0000 7f4c65ffb640 20 reqs_thread_entry: start 2026-02-24T22:34:18.215 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.227+0000 7f4d5072da80 10 cache get: name=default.rgw.meta+accounts+account.RGW88888888888888888 : miss 2026-02-24T22:34:18.215 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.227+0000 7f4d5072da80 20 rados->read ofs=0 len=0 2026-02-24T22:34:18.215 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.227+0000 7f4d5072da80 20 rados_obj.operate() r=0 bl.length=184 2026-02-24T22:34:18.216 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.227+0000 7f4d5072da80 10 cache put: name=default.rgw.meta+accounts+account.RGW88888888888888888 info.flags=0x17 2026-02-24T22:34:18.216 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.227+0000 7f4d5072da80 10 adding default.rgw.meta+accounts+account.RGW88888888888888888 to cache LRU end 2026-02-24T22:34:18.216 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.227+0000 7f4d5072da80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:34:18.216 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.227+0000 7f4d5072da80 10 cache get: name=default.rgw.meta+users.uid+root1.client.0 : miss 2026-02-24T22:34:18.216 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.227+0000 7f4d5072da80 20 rados->read ofs=0 len=0 2026-02-24T22:34:18.216 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.227+0000 7f4d5072da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:34:18.216 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.227+0000 7f4d5072da80 10 cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x0 2026-02-24T22:34:18.216 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.227+0000 7f4d5072da80 10 adding default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2026-02-24T22:34:18.217 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.227+0000 7f4d5072da80 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-emu8v8fyvjgc4k2ywc0nl3jx-45:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1574 : miss 2026-02-24T22:34:18.217 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.227+0000 7f4d5072da80 20 rados->read ofs=0 len=0 2026-02-24T22:34:18.217 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.227+0000 7f4d5072da80 20 rados_obj.operate() r=0 bl.length=428 2026-02-24T22:34:18.217 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.227+0000 7f4d5072da80 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-emu8v8fyvjgc4k2ywc0nl3jx-45:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1574 info.flags=0x17 2026-02-24T22:34:18.217 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.227+0000 7f4d5072da80 10 adding default.rgw.meta+root+.bucket.meta.test-emu8v8fyvjgc4k2ywc0nl3jx-45:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1574 to cache LRU end 2026-02-24T22:34:18.218 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.227+0000 7f4d5072da80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:34:18.218 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.227+0000 7f4d5072da80 10 updating xattr: name=user.rgw.acl bl.length()=231 2026-02-24T22:34:18.218 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.227+0000 7f4d5072da80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-emu8v8fyvjgc4k2ywc0nl3jx-45:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1574 2026-02-24T22:34:18.218 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.227+0000 7f4d5072da80 20 list_objects_ordered: starting attempt 1 2026-02-24T22:34:18.218 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.227+0000 7f4d5072da80 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2026-02-24T22:34:18.219 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.231+0000 7f4d5072da80 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2026-02-24T22:34:18.219 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.231+0000 7f4d5072da80 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2026-02-24T22:34:18.219 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.231+0000 7f4d5072da80 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2026-02-24T22:34:18.219 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.231+0000 7f4d5072da80 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2026-02-24T22:34:18.219 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.231+0000 7f4d5072da80 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-emu8v8fyvjgc4k2ywc0nl3jx-45[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1574])"; uploads.size()=0; is_truncated=0 2026-02-24T22:34:18.219 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.231+0000 7f4d5072da80 10 cache get: name=default.rgw.meta+root+test-emu8v8fyvjgc4k2ywc0nl3jx-45 : miss 2026-02-24T22:34:18.220 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.231+0000 7f4d5072da80 20 rados->read ofs=0 len=0 2026-02-24T22:34:18.220 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.231+0000 7f4d5072da80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:34:18.220 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.231+0000 7f4d5072da80 10 cache put: name=default.rgw.meta+root+test-emu8v8fyvjgc4k2ywc0nl3jx-45 info.flags=0x11 2026-02-24T22:34:18.220 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.231+0000 7f4d5072da80 10 adding default.rgw.meta+root+test-emu8v8fyvjgc4k2ywc0nl3jx-45 to cache LRU end 2026-02-24T22:34:18.221 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.231+0000 7f4d5072da80 10 removing default.rgw.meta+root+test-emu8v8fyvjgc4k2ywc0nl3jx-45 from cache 2026-02-24T22:34:18.221 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.231+0000 7f4d5072da80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:test-emu8v8fyvjgc4k2ywc0nl3jx-45, ofs0, ns] 2026-02-24T22:34:18.221 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.231+0000 7f4d29ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029798055 cookie 94516068302656 notifier 9417 bl.length()=176 2026-02-24T22:34:18.222 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.231+0000 7f4d5072da80 10 removing default.rgw.meta+root+.bucket.meta.test-emu8v8fyvjgc4k2ywc0nl3jx-45:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1574 from cache 2026-02-24T22:34:18.222 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.231+0000 7f4d5072da80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-emu8v8fyvjgc4k2ywc0nl3jx-45:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1574, ofs0, ns] 2026-02-24T22:34:18.222 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.235+0000 7f4d297fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029797460 cookie 94516069105664 notifier 9417 bl.length()=236 2026-02-24T22:34:18.224 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.235+0000 7f4d5072da80 10 cache get: name=default.rgw.log++pubsub..bucket.test-emu8v8fyvjgc4k2ywc0nl3jx-45/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1574 : miss 2026-02-24T22:34:18.224 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.235+0000 7f4d5072da80 20 rados->read ofs=0 len=0 2026-02-24T22:34:18.225 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.235+0000 7f4d5072da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:34:18.225 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.235+0000 7f4d5072da80 10 cache put: name=default.rgw.log++pubsub..bucket.test-emu8v8fyvjgc4k2ywc0nl3jx-45/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1574 info.flags=0x0 2026-02-24T22:34:18.225 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.235+0000 7f4d5072da80 10 adding default.rgw.log++pubsub..bucket.test-emu8v8fyvjgc4k2ywc0nl3jx-45/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1574 to cache LRU end 2026-02-24T22:34:18.226 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.239+0000 7f4d5072da80 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-emu8v8fyvjgc4k2ywc0nl3jx-47:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1576 : miss 2026-02-24T22:34:18.226 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.239+0000 7f4d5072da80 20 rados->read ofs=0 len=0 2026-02-24T22:34:18.227 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.239+0000 7f4d5072da80 20 rados_obj.operate() r=0 bl.length=428 2026-02-24T22:34:18.227 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.239+0000 7f4d5072da80 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-emu8v8fyvjgc4k2ywc0nl3jx-47:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1576 info.flags=0x17 2026-02-24T22:34:18.227 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.239+0000 7f4d5072da80 10 adding default.rgw.meta+root+.bucket.meta.test-emu8v8fyvjgc4k2ywc0nl3jx-47:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1576 to cache LRU end 2026-02-24T22:34:18.227 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.239+0000 7f4d5072da80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:34:18.227 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.239+0000 7f4d5072da80 10 updating xattr: name=user.rgw.acl bl.length()=231 2026-02-24T22:34:18.228 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.239+0000 7f4d5072da80 10 updating xattr: name=user.rgw.ownership-controls bl.length()=7 2026-02-24T22:34:18.228 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.239+0000 7f4d5072da80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-emu8v8fyvjgc4k2ywc0nl3jx-47:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1576 2026-02-24T22:34:18.228 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.239+0000 7f4d5072da80 20 list_objects_ordered: starting attempt 1 2026-02-24T22:34:18.228 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.239+0000 7f4d5072da80 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2026-02-24T22:34:18.229 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.239+0000 7f4d5072da80 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2026-02-24T22:34:18.229 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.239+0000 7f4d5072da80 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2026-02-24T22:34:18.229 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.239+0000 7f4d5072da80 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2026-02-24T22:34:18.229 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.239+0000 7f4d5072da80 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2026-02-24T22:34:18.229 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.239+0000 7f4d5072da80 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-emu8v8fyvjgc4k2ywc0nl3jx-47[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1576])"; uploads.size()=0; is_truncated=0 2026-02-24T22:34:18.229 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.239+0000 7f4d5072da80 10 cache get: name=default.rgw.meta+root+test-emu8v8fyvjgc4k2ywc0nl3jx-47 : miss 2026-02-24T22:34:18.229 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.239+0000 7f4d5072da80 20 rados->read ofs=0 len=0 2026-02-24T22:34:18.229 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.239+0000 7f4d5072da80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:34:18.229 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.239+0000 7f4d5072da80 10 cache put: name=default.rgw.meta+root+test-emu8v8fyvjgc4k2ywc0nl3jx-47 info.flags=0x11 2026-02-24T22:34:18.229 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.239+0000 7f4d5072da80 10 adding default.rgw.meta+root+test-emu8v8fyvjgc4k2ywc0nl3jx-47 to cache LRU end 2026-02-24T22:34:18.230 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.239+0000 7f4d5072da80 10 removing default.rgw.meta+root+test-emu8v8fyvjgc4k2ywc0nl3jx-47 from cache 2026-02-24T22:34:18.230 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.239+0000 7f4d5072da80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:test-emu8v8fyvjgc4k2ywc0nl3jx-47, ofs0, ns] 2026-02-24T22:34:18.230 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.239+0000 7f4d29ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029798056 cookie 94516068302656 notifier 9417 bl.length()=176 2026-02-24T22:34:18.231 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.243+0000 7f4d5072da80 10 removing default.rgw.meta+root+.bucket.meta.test-emu8v8fyvjgc4k2ywc0nl3jx-47:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1576 from cache 2026-02-24T22:34:18.231 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.243+0000 7f4d5072da80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-emu8v8fyvjgc4k2ywc0nl3jx-47:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1576, ofs0, ns] 2026-02-24T22:34:18.231 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.243+0000 7f4d297fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029797461 cookie 94516069105664 notifier 9417 bl.length()=236 2026-02-24T22:34:18.232 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.243+0000 7f4d5072da80 10 cache get: name=default.rgw.log++pubsub..bucket.test-emu8v8fyvjgc4k2ywc0nl3jx-47/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1576 : miss 2026-02-24T22:34:18.232 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.243+0000 7f4d5072da80 20 rados->read ofs=0 len=0 2026-02-24T22:34:18.232 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.243+0000 7f4d5072da80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:34:18.232 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.243+0000 7f4d5072da80 10 cache put: name=default.rgw.log++pubsub..bucket.test-emu8v8fyvjgc4k2ywc0nl3jx-47/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1576 info.flags=0x0 2026-02-24T22:34:18.232 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.243+0000 7f4d5072da80 10 adding default.rgw.log++pubsub..bucket.test-emu8v8fyvjgc4k2ywc0nl3jx-47/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1576 to cache LRU end 2026-02-24T22:34:18.255 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.267+0000 7f4d5072da80 10 removing default.rgw.meta+accounts+account.RGW88888888888888888 from cache 2026-02-24T22:34:18.255 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.267+0000 7f4d5072da80 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:accounts:account.RGW88888888888888888, ofs0, ns] 2026-02-24T22:34:18.255 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.267+0000 7f4d297fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029796708 cookie 94516070685728 notifier 9417 bl.length()=176 2026-02-24T22:34:18.256 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.267+0000 7f4d5072da80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:accounts:name.$Mr.RGW88888888888888888, ofs0, ns] 2026-02-24T22:34:18.256 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.267+0000 7f4d29ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029797462 cookie 94516069105664 notifier 9417 bl.length()=177 2026-02-24T22:34:18.258 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.267+0000 7f4d5072da80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.email:rgw88888888888888888+test@test.test, ofs0, ns] 2026-02-24T22:34:18.258 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.267+0000 7f4d297fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029798649 cookie 94516068290496 notifier 9417 bl.length()=186 2026-02-24T22:34:18.259 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.271+0000 7f4d5072da80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:accounts:users.RGW88888888888888888, ofs0, ns] 2026-02-24T22:34:18.259 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.271+0000 7f4d29ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029796597 cookie 94516070547568 notifier 9417 bl.length()=174 2026-02-24T22:34:18.263 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.271+0000 7f4d5072da80 20 remove_watcher() i=3 2026-02-24T22:34:18.263 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.271+0000 7f4d5072da80 2 removed watcher, disabling cache 2026-02-24T22:34:18.263 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.271+0000 7f4d5072da80 20 remove_watcher() i=2 2026-02-24T22:34:18.263 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.271+0000 7f4d5072da80 20 remove_watcher() i=4 2026-02-24T22:34:18.263 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.275+0000 7f4d5072da80 20 remove_watcher() i=0 2026-02-24T22:34:18.263 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.275+0000 7f4d5072da80 20 remove_watcher() i=7 2026-02-24T22:34:18.263 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.275+0000 7f4d5072da80 20 remove_watcher() i=6 2026-02-24T22:34:18.263 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.275+0000 7f4d5072da80 20 remove_watcher() i=1 2026-02-24T22:34:18.263 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.275+0000 7f4d5072da80 20 remove_watcher() i=5 2026-02-24T22:34:18.270 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid root2.client.0 --purge-data --cluster ceph 2026-02-24T22:34:18.328 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:34:18.328 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:34:18.343 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.355+0000 7fa255a36a80 4 Realm: () 2026-02-24T22:34:18.343 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.355+0000 7fa255a36a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:34:18.343 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.355+0000 7fa255a36a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:34:18.343 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.355+0000 7fa255a36a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:34:18.343 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.355+0000 7fa255a36a80 20 zonegroup default 2026-02-24T22:34:18.344 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.355+0000 7fa255a36a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:18.345 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.355+0000 7fa255a36a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:34:18.345 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.355+0000 7fa255a36a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:18.345 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.355+0000 7fa255a36a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:34:18.345 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.355+0000 7fa255a36a80 20 started sync module instance, tier type = 2026-02-24T22:34:18.346 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.355+0000 7fa255a36a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:34:18.346 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.359+0000 7fa24effd640 0 -- 10.20.193.46:0/150550506 >> [v2:10.20.193.110:6800/341553926,v1:10.20.193.110:6801/341553926] conn(0x55b10ee0d500 msgr2=0x55b10ee0d970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:34:18.347 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.359+0000 7fa255a36a80 20 add_watcher() i=1 2026-02-24T22:34:18.347 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.359+0000 7fa255a36a80 20 add_watcher() i=3 2026-02-24T22:34:18.347 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.359+0000 7fa255a36a80 20 add_watcher() i=5 2026-02-24T22:34:18.348 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.359+0000 7fa255a36a80 20 add_watcher() i=0 2026-02-24T22:34:18.348 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.359+0000 7fa255a36a80 20 add_watcher() i=2 2026-02-24T22:34:18.348 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.359+0000 7fa255a36a80 20 add_watcher() i=7 2026-02-24T22:34:18.348 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.359+0000 7fa255a36a80 20 add_watcher() i=4 2026-02-24T22:34:18.348 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.359+0000 7fa255a36a80 20 add_watcher() i=6 2026-02-24T22:34:18.348 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.359+0000 7fa255a36a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:34:18.349 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.359+0000 7fa247fff640 0 -- 10.20.193.46:0/4100052442 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7fa174008af0 msgr2=0x7fa174028fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:34:18.349 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.359+0000 7fa255a36a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:34:18.350 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.359+0000 7fa255a36a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:34:18.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.383+0000 7fa16bfff640 20 reqs_thread_entry: start 2026-02-24T22:34:18.407 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.419+0000 7fa255a36a80 20 init_complete bucket index max shards: 1 2026-02-24T22:34:18.407 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.419+0000 7fa255a36a80 20 Filter name: none 2026-02-24T22:34:18.407 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.419+0000 7fa16a7fc640 20 reqs_thread_entry: start 2026-02-24T22:34:18.407 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.419+0000 7fa255a36a80 10 cache get: name=default.rgw.meta+users.uid+root2.client.0 : miss 2026-02-24T22:34:18.407 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.419+0000 7fa255a36a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:18.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.419+0000 7fa255a36a80 20 rados_obj.operate() r=0 bl.length=508 2026-02-24T22:34:18.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.419+0000 7fa255a36a80 10 cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x13 2026-02-24T22:34:18.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.419+0000 7fa255a36a80 10 adding default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2026-02-24T22:34:18.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.419+0000 7fa255a36a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:34:18.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.419+0000 7fa255a36a80 10 removing key index: FYRXAKZJTVNMCJKDMCVA 2026-02-24T22:34:18.409 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.419+0000 7fa255a36a80 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:FYRXAKZJTVNMCJKDMCVA, ofs0, ns] 2026-02-24T22:34:18.409 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.419+0000 7fa22e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029796464 cookie 94218947207184 notifier 8765 bl.length()=170 2026-02-24T22:34:18.409 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.419+0000 7fa255a36a80 10 removing email index: root2+test@test.test 2026-02-24T22:34:18.409 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.419+0000 7fa255a36a80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.email:root2+test@test.test, ofs0, ns] 2026-02-24T22:34:18.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.419+0000 7fa22e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029798057 cookie 94218947227776 notifier 8765 bl.length()=171 2026-02-24T22:34:18.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.419+0000 7fa255a36a80 10 removing user index: root2.client.0 2026-02-24T22:34:18.411 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.423+0000 7fa255a36a80 10 removing default.rgw.meta+users.uid+root2.client.0 from cache 2026-02-24T22:34:18.411 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.423+0000 7fa255a36a80 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:users.uid:root2.client.0, ofs0, ns] 2026-02-24T22:34:18.411 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.423+0000 7fa22dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029796709 cookie 94218947203424 notifier 8765 bl.length()=163 2026-02-24T22:34:18.414 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.423+0000 7fa255a36a80 20 remove_watcher() i=1 2026-02-24T22:34:18.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.423+0000 7fa255a36a80 2 removed watcher, disabling cache 2026-02-24T22:34:18.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.423+0000 7fa255a36a80 20 remove_watcher() i=3 2026-02-24T22:34:18.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.423+0000 7fa255a36a80 20 remove_watcher() i=7 2026-02-24T22:34:18.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.423+0000 7fa255a36a80 20 remove_watcher() i=2 2026-02-24T22:34:18.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.427+0000 7fa255a36a80 20 remove_watcher() i=6 2026-02-24T22:34:18.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.427+0000 7fa255a36a80 20 remove_watcher() i=4 2026-02-24T22:34:18.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.427+0000 7fa255a36a80 20 remove_watcher() i=0 2026-02-24T22:34:18.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.427+0000 7fa255a36a80 20 remove_watcher() i=5 2026-02-24T22:34:18.421 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph account rm --account-id RGW99999999999999999 --purge-data 2026-02-24T22:34:18.480 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:34:18.480 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:34:18.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.507+0000 7f785d301a80 4 Realm: () 2026-02-24T22:34:18.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.507+0000 7f785d301a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:34:18.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.507+0000 7f785d301a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:34:18.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.507+0000 7f785d301a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:34:18.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.507+0000 7f785d301a80 20 zonegroup default 2026-02-24T22:34:18.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.507+0000 7f785d301a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:18.497 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.507+0000 7f785d301a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:34:18.498 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.507+0000 7f785d301a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:18.498 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.507+0000 7f785d301a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:34:18.498 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.507+0000 7f785d301a80 20 started sync module instance, tier type = 2026-02-24T22:34:18.498 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.507+0000 7f785d301a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:34:18.498 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.511+0000 7f785b5d6640 0 -- 10.20.193.46:0/3983751307 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x5631769cc0b0 msgr2=0x5631769cc520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2026-02-24T22:34:18.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.511+0000 7f785d301a80 20 add_watcher() i=1 2026-02-24T22:34:18.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.511+0000 7f785d301a80 20 add_watcher() i=3 2026-02-24T22:34:18.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.511+0000 7f785d301a80 20 add_watcher() i=6 2026-02-24T22:34:18.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.511+0000 7f785d301a80 20 add_watcher() i=0 2026-02-24T22:34:18.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.511+0000 7f785d301a80 20 add_watcher() i=5 2026-02-24T22:34:18.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.511+0000 7f785d301a80 20 add_watcher() i=2 2026-02-24T22:34:18.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.511+0000 7f785a5d4640 0 -- 10.20.193.46:0/3983751307 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x5631769c9f90 msgr2=0x5631769ca400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:34:18.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.511+0000 7f785d301a80 20 add_watcher() i=4 2026-02-24T22:34:18.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.511+0000 7f785d301a80 20 add_watcher() i=7 2026-02-24T22:34:18.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.511+0000 7f785d301a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:34:18.502 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.511+0000 7f785d301a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:34:18.502 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.511+0000 7f785d301a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:34:18.502 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.511+0000 7f785d301a80 5 note: GC not initialized 2026-02-24T22:34:18.502 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.511+0000 7f7808fe1640 20 reqs_thread_entry: start 2026-02-24T22:34:18.533 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.543+0000 7f785d301a80 20 init_complete bucket index max shards: 1 2026-02-24T22:34:18.533 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.543+0000 7f785d301a80 20 Filter name: none 2026-02-24T22:34:18.533 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.543+0000 7f7772ffd640 20 reqs_thread_entry: start 2026-02-24T22:34:18.533 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.543+0000 7f785d301a80 10 cache get: name=default.rgw.meta+accounts+account.RGW99999999999999999 : miss 2026-02-24T22:34:18.533 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.543+0000 7f785d301a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:18.534 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.543+0000 7f785d301a80 20 rados_obj.operate() r=0 bl.length=184 2026-02-24T22:34:18.534 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.543+0000 7f785d301a80 10 cache put: name=default.rgw.meta+accounts+account.RGW99999999999999999 info.flags=0x17 2026-02-24T22:34:18.534 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.543+0000 7f785d301a80 10 adding default.rgw.meta+accounts+account.RGW99999999999999999 to cache LRU end 2026-02-24T22:34:18.535 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.543+0000 7f785d301a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:34:18.535 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.543+0000 7f785d301a80 10 cache get: name=default.rgw.meta+users.uid+root2.client.0 : miss 2026-02-24T22:34:18.535 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.543+0000 7f785d301a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:18.535 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.543+0000 7f785d301a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:34:18.535 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.543+0000 7f785d301a80 10 cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x0 2026-02-24T22:34:18.535 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.543+0000 7f785d301a80 10 adding default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2026-02-24T22:34:18.535 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.543+0000 7f785d301a80 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-emu8v8fyvjgc4k2ywc0nl3jx-46:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1575 : miss 2026-02-24T22:34:18.535 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.543+0000 7f785d301a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:18.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.547+0000 7f785d301a80 20 rados_obj.operate() r=0 bl.length=428 2026-02-24T22:34:18.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.547+0000 7f785d301a80 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-emu8v8fyvjgc4k2ywc0nl3jx-46:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1575 info.flags=0x17 2026-02-24T22:34:18.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.547+0000 7f785d301a80 10 adding default.rgw.meta+root+.bucket.meta.test-emu8v8fyvjgc4k2ywc0nl3jx-46:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1575 to cache LRU end 2026-02-24T22:34:18.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.547+0000 7f785d301a80 10 updating xattr: name=ceph.objclass.version bl.length()=42 2026-02-24T22:34:18.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.547+0000 7f785d301a80 10 updating xattr: name=user.rgw.acl bl.length()=231 2026-02-24T22:34:18.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.547+0000 7f785d301a80 10 updating xattr: name=user.rgw.iam-policy bl.length()=213 2026-02-24T22:34:18.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.547+0000 7f785d301a80 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-emu8v8fyvjgc4k2ywc0nl3jx-46:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1575 2026-02-24T22:34:18.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.547+0000 7f785d301a80 20 list_objects_ordered: starting attempt 1 2026-02-24T22:34:18.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.547+0000 7f785d301a80 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2026-02-24T22:34:18.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.547+0000 7f785d301a80 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2026-02-24T22:34:18.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.547+0000 7f785d301a80 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2026-02-24T22:34:18.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.547+0000 7f785d301a80 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2026-02-24T22:34:18.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.547+0000 7f785d301a80 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2026-02-24T22:34:18.538 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.547+0000 7f785d301a80 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-emu8v8fyvjgc4k2ywc0nl3jx-46[301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1575])"; uploads.size()=0; is_truncated=0 2026-02-24T22:34:18.538 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.547+0000 7f785d301a80 10 cache get: name=default.rgw.meta+root+test-emu8v8fyvjgc4k2ywc0nl3jx-46 : miss 2026-02-24T22:34:18.538 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.547+0000 7f785d301a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:18.538 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.547+0000 7f785d301a80 20 rados_obj.operate() r=0 bl.length=204 2026-02-24T22:34:18.538 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.547+0000 7f785d301a80 10 cache put: name=default.rgw.meta+root+test-emu8v8fyvjgc4k2ywc0nl3jx-46 info.flags=0x11 2026-02-24T22:34:18.538 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.547+0000 7f785d301a80 10 adding default.rgw.meta+root+test-emu8v8fyvjgc4k2ywc0nl3jx-46 to cache LRU end 2026-02-24T22:34:18.539 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.551+0000 7f785d301a80 10 removing default.rgw.meta+root+test-emu8v8fyvjgc4k2ywc0nl3jx-46 from cache 2026-02-24T22:34:18.539 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.551+0000 7f785d301a80 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:test-emu8v8fyvjgc4k2ywc0nl3jx-46, ofs0, ns] 2026-02-24T22:34:18.539 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.551+0000 7f7836ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029798058 cookie 94770445232368 notifier 8783 bl.length()=176 2026-02-24T22:34:18.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.551+0000 7f785d301a80 10 removing default.rgw.meta+root+.bucket.meta.test-emu8v8fyvjgc4k2ywc0nl3jx-46:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1575 from cache 2026-02-24T22:34:18.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.551+0000 7f785d301a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-emu8v8fyvjgc4k2ywc0nl3jx-46:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1575, ofs0, ns] 2026-02-24T22:34:18.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.551+0000 7f78367fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029797463 cookie 94770445227376 notifier 8783 bl.length()=236 2026-02-24T22:34:18.541 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.551+0000 7f785d301a80 10 cache get: name=default.rgw.log++pubsub..bucket.test-emu8v8fyvjgc4k2ywc0nl3jx-46/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1575 : miss 2026-02-24T22:34:18.542 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.551+0000 7f785d301a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:18.542 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.551+0000 7f785d301a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:34:18.542 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.551+0000 7f785d301a80 10 cache put: name=default.rgw.log++pubsub..bucket.test-emu8v8fyvjgc4k2ywc0nl3jx-46/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1575 info.flags=0x0 2026-02-24T22:34:18.542 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.551+0000 7f785d301a80 10 adding default.rgw.log++pubsub..bucket.test-emu8v8fyvjgc4k2ywc0nl3jx-46/301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1575 to cache LRU end 2026-02-24T22:34:18.565 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.575+0000 7f785d301a80 10 removing default.rgw.meta+accounts+account.RGW99999999999999999 from cache 2026-02-24T22:34:18.565 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.575+0000 7f785d301a80 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:accounts:account.RGW99999999999999999, ofs0, ns] 2026-02-24T22:34:18.566 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.575+0000 7f78367fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029796710 cookie 94770443343184 notifier 8783 bl.length()=176 2026-02-24T22:34:18.567 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.579+0000 7f785d301a80 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:accounts:name.$Mr.RGW99999999999999999, ofs0, ns] 2026-02-24T22:34:18.567 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.579+0000 7f7836ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029797464 cookie 94770445227376 notifier 8783 bl.length()=177 2026-02-24T22:34:18.568 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.579+0000 7f785d301a80 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.email:rgw99999999999999999+test@test.test, ofs0, ns] 2026-02-24T22:34:18.568 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.579+0000 7f7836ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029798650 cookie 94770443355376 notifier 8783 bl.length()=186 2026-02-24T22:34:18.569 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.579+0000 7f785d301a80 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:accounts:users.RGW99999999999999999, ofs0, ns] 2026-02-24T22:34:18.570 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.579+0000 7f78367fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1340029796598 cookie 94770445612752 notifier 8783 bl.length()=174 2026-02-24T22:34:18.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.583+0000 7f785d301a80 20 remove_watcher() i=3 2026-02-24T22:34:18.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.583+0000 7f785d301a80 2 removed watcher, disabling cache 2026-02-24T22:34:18.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.583+0000 7f785d301a80 20 remove_watcher() i=2 2026-02-24T22:34:18.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.583+0000 7f785d301a80 20 remove_watcher() i=6 2026-02-24T22:34:18.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.583+0000 7f785d301a80 20 remove_watcher() i=4 2026-02-24T22:34:18.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.583+0000 7f785d301a80 20 remove_watcher() i=5 2026-02-24T22:34:18.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.583+0000 7f785d301a80 20 remove_watcher() i=0 2026-02-24T22:34:18.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.583+0000 7f785d301a80 20 remove_watcher() i=1 2026-02-24T22:34:18.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:18.583+0000 7f785d301a80 20 remove_watcher() i=7 2026-02-24T22:34:18.581 INFO:tasks.s3tests:Removing s3-tests... 2026-02-24T22:34:18.581 DEBUG:teuthology.orchestra.run.trial046:> rm -rf /home/ubuntu/cephtest/s3-tests-client.0 2026-02-24T22:34:18.788 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 112, in run_tasks manager.__enter__() File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_samarahu_ceph_f7e05f4fdea5af173284f247d6b0935b337d4ccf/qa/tasks/s3tests.py", line 777, in task with contextutil.nested( File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/contextutil.py", line 30, in nested vars.append(enter()) ^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_samarahu_ceph_f7e05f4fdea5af173284f247d6b0935b337d4ccf/qa/tasks/s3tests.py", line 503, in run_tests toxvenv_sh(ctx, remote, args, label="s3 tests against rgw") File "/home/teuthworker/src/github.com_samarahu_ceph_f7e05f4fdea5af173284f247d6b0935b337d4ccf/qa/tasks/s3tests.py", line 444, in toxvenv_sh return remote.sh(['source', activate, run.Raw('&&')] + args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) ^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (s3 tests against rgw) on trial046 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3 and not bucket_logging'" 2026-02-24T22:34:18.791 DEBUG:teuthology.run_tasks:Unwinding manager s3tests 2026-02-24T22:34:18.808 DEBUG:teuthology.run_tasks:Unwinding manager ragweed 2026-02-24T22:34:18.824 DEBUG:teuthology.run_tasks:Unwinding manager tox 2026-02-24T22:34:18.841 DEBUG:teuthology.orchestra.run.trial046:> rm -rf /home/ubuntu/cephtest/tox-venv 2026-02-24T22:34:18.881 DEBUG:teuthology.run_tasks:Unwinding manager rgw 2026-02-24T22:34:18.898 DEBUG:tasks.rgw.client.0:waiting for process to exit 2026-02-24T22:34:18.899 INFO:teuthology.orchestra.run:waiting for 300 2026-02-24T22:34:18.923 INFO:tasks.rgw.client.0.trial046.stdout:2026-02-24T22:34:18.935+0000 7f25b1153640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term radosgw --rgw-frontends beast ssl_certificate=/home/ubuntu/cephtest/ca/rgw.client.0.crt ssl_port=443 -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground (PID: 23893) UID: 0 2026-02-24T22:34:18.923 INFO:tasks.rgw.client.0.trial046.stdout:2026-02-24T22:34:18.935+0000 7f25b499bb40 -1 shutting down 2026-02-24T22:34:21.026 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~0s 2026-02-24T22:34:25.000 INFO:tasks.rgw.client.0:Stopped 2026-02-24T22:34:25.000 DEBUG:teuthology.orchestra.run.trial046:> rm -f /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock 2026-02-24T22:34:25.006 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -f /etc/ceph/vault-root-token 2026-02-24T22:34:25.054 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -f /home/ubuntu/cephtest/url_file 2026-02-24T22:34:25.106 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'process', '--include-all'] 2026-02-24T22:34:25.106 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'process', '--include-all'] 2026-02-24T22:34:25.106 DEBUG:teuthology.orchestra.run.trial046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc process --include-all 2026-02-24T22:34:25.164 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setuser ceph since I am not root 2026-02-24T22:34:25.164 INFO:teuthology.orchestra.run.trial046.stderr:ignoring --setgroup ceph since I am not root 2026-02-24T22:34:25.181 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.191+0000 7fa0ccd13a80 4 Realm: () 2026-02-24T22:34:25.182 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.191+0000 7fa0ccd13a80 4 ZoneGroup: default (169e03d0-6e95-434d-a7b6-e57bd0ab1737) 2026-02-24T22:34:25.182 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.191+0000 7fa0ccd13a80 4 Zone: default (301937db-3a1f-489c-b387-e7c21a76f9b9) 2026-02-24T22:34:25.182 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.191+0000 7fa0ccd13a80 10 cannot find current period zonegroup using local zonegroup configuration 2026-02-24T22:34:25.182 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.191+0000 7fa0ccd13a80 20 zonegroup default 2026-02-24T22:34:25.182 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.191+0000 7fa0ccd13a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:25.183 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.195+0000 7fa0ccd13a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:34:25.183 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.195+0000 7fa0ccd13a80 20 rados->read ofs=0 len=0 2026-02-24T22:34:25.184 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.195+0000 7fa0ccd13a80 20 rados_obj.operate() r=-2 bl.length=0 2026-02-24T22:34:25.184 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.195+0000 7fa0ccd13a80 20 started sync module instance, tier type = 2026-02-24T22:34:25.184 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.195+0000 7fa0ccd13a80 20 started zone id=301937db-3a1f-489c-b387-e7c21a76f9b9 (name=default) with tier type = 2026-02-24T22:34:25.185 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.195+0000 7fa0cafe8640 0 -- 10.20.193.46:0/1777517895 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x55661f0341f0 msgr2=0x55661f034660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:34:25.186 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.195+0000 7fa0ccd13a80 20 add_watcher() i=3 2026-02-24T22:34:25.186 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.195+0000 7fa0ccd13a80 20 add_watcher() i=1 2026-02-24T22:34:25.186 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.195+0000 7fa0ccd13a80 20 add_watcher() i=6 2026-02-24T22:34:25.186 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.195+0000 7fa0ccd13a80 20 add_watcher() i=0 2026-02-24T22:34:25.187 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.195+0000 7fa0cafe8640 0 -- 10.20.193.46:0/1777517895 >> [v2:10.20.193.46:6818/2329512561,v1:10.20.193.46:6819/2329512561] conn(0x7fa08c09e2e0 msgr2=0x7fa08c0be6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:34:25.187 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.195+0000 7fa0ccd13a80 20 add_watcher() i=5 2026-02-24T22:34:25.187 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.195+0000 7fa0ccd13a80 20 add_watcher() i=2 2026-02-24T22:34:25.187 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.195+0000 7fa0ccd13a80 20 add_watcher() i=4 2026-02-24T22:34:25.187 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.199+0000 7fa0ccd13a80 20 add_watcher() i=7 2026-02-24T22:34:25.187 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.199+0000 7fa0ccd13a80 2 all 8 watchers are set, enabling cache 2026-02-24T22:34:25.188 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.199+0000 7fa0ccd13a80 20 rgw_check_secure_mon_conn(): auth registry supported: methods=[2] modes=[2,1] 2026-02-24T22:34:25.188 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.199+0000 7fa0ccd13a80 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2026-02-24T22:34:25.211 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.223+0000 7f9fe37fe640 20 reqs_thread_entry: start 2026-02-24T22:34:25.244 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 20 init_complete bucket index max shards: 1 2026-02-24T22:34:25.244 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 20 Filter name: none 2026-02-24T22:34:25.244 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7f9fe1ffb640 20 reqs_thread_entry: start 2026-02-24T22:34:25.244 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2026-02-24T22:34:25.245 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.245 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.247 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/37935' 2026-02-24T22:34:25.248 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15979522926328386554', time=2026-02-25T00:15:09.938089+0000, chain.objs.size()=1 2026-02-24T22:34:25.248 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1505__shadow_.WZRsRCwWV49XN7yG2KItOCCYo00YoYu_0 2026-02-24T22:34:25.248 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3710598691646779170', time=2026-02-25T00:16:32.608494+0000, chain.objs.size()=1 2026-02-24T22:34:25.248 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520__shadow_.CdnR3_wONZ8mcqMqUOHy0b-yowiTmLF_0 2026-02-24T22:34:25.248 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7156339815134996942', time=2026-02-25T00:16:33.512549+0000, chain.objs.size()=1 2026-02-24T22:34:25.248 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521__shadow_.6AQJHDHqWnXgjxUyuoXJvLcDt4v7aib_0 2026-02-24T22:34:25.248 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15405536990859963488', time=2026-02-25T00:17:05.169446+0000, chain.objs.size()=1 2026-02-24T22:34:25.248 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522__shadow_.BjYHG5FB5Qc1KGgV9IoQVaC7h81kpTu_0 2026-02-24T22:34:25.248 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4104172572877499732', time=2026-02-25T00:17:07.115679+0000, chain.objs.size()=1 2026-02-24T22:34:25.248 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523__shadow_.3iXfkLdWngEMdkH1mXYtIyz5c3ZewHB_0 2026-02-24T22:34:25.249 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2045264886541903755', time=2026-02-25T00:17:39.874041+0000, chain.objs.size()=1 2026-02-24T22:34:25.249 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525__shadow_.0Ed-5ix5Mwmfq685O8dTcJcKahWXnS__0 2026-02-24T22:34:25.249 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2519164252681640203', time=2026-02-25T00:17:41.423405+0000, chain.objs.size()=1 2026-02-24T22:34:25.249 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526__shadow_.lzC5SNLPti6GJ0SLLTt1XkBVGL5g-NE_0 2026-02-24T22:34:25.249 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6895698795396155443', time=2026-02-25T00:17:42.532903+0000, chain.objs.size()=1 2026-02-24T22:34:25.249 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527__shadow_.uyz1x0J94YWfZxG9hAlvAyMPG_WKNzd_0 2026-02-24T22:34:25.249 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15151056243675974361', time=2026-02-25T00:18:48.183427+0000, chain.objs.size()=1 2026-02-24T22:34:25.249 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530__shadow_.nzh8N_AuAka8MvFhp7ZCVc4iq8duvV9_0 2026-02-24T22:34:25.249 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6626833407271954350', time=2026-02-25T00:19:23.059346+0000, chain.objs.size()=1 2026-02-24T22:34:25.249 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533__shadow_.0PsaxY7KDQdyDT6R7QtbLyiwSa0XtkZ_0 2026-02-24T22:34:25.249 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1269164113373918966', time=2026-02-25T00:19:25.019780+0000, chain.objs.size()=1 2026-02-24T22:34:25.249 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534__shadow_.-pVcR00z2b_40I3iPRUQkj6dnskS7Cn_0 2026-02-24T22:34:25.250 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12005314684263890511', time=2026-02-25T00:19:57.109488+0000, chain.objs.size()=1 2026-02-24T22:34:25.250 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.255+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535__shadow_.fpjH9vmIurNOFR9RDX-4WV7pF9HmpN__0 2026-02-24T22:34:25.250 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17838682629703809682', time=2026-02-25T00:23:25.729124+0000, chain.objs.size()=1 2026-02-24T22:34:25.250 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519__shadow:SA99TqC8hxWnqSUUXyJ6LebUJ9mG-Fw_.jsIyAlVNs_v5_PPxeS0YPV70c5oJgF1_0 2026-02-24T22:34:25.250 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13344353866731728781', time=2026-02-25T00:24:00.206320+0000, chain.objs.size()=1 2026-02-24T22:34:25.250 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1557__shadow_.s8a-RJwIKS3PmEA9CVL-FRWrfYJVHRV_0 2026-02-24T22:34:25.250 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3477682439632978327', time=2026-02-25T00:24:12.985393+0000, chain.objs.size()=1 2026-02-24T22:34:25.250 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1565__shadow_.ptOjZvrQwOrygO3zVIfWlS8DbyaCPi1_0 2026-02-24T22:34:25.250 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15434160549802148199', time=2026-02-25T00:25:40.782462+0000, chain.objs.size()=1 2026-02-24T22:34:25.250 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1608__shadow_.zBC3DLMg47w-C95Mp9t08rZNN7D724l_0 2026-02-24T22:34:25.250 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8579561470635690690', time=2026-02-25T00:25:42.667917+0000, chain.objs.size()=1 2026-02-24T22:34:25.250 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1610__shadow_.UN_m0fMN1ldU39Uuzi4Ebev9cRNbm4S_0 2026-02-24T22:34:25.250 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5155393004966899862', time=2026-02-25T00:25:51.942782+0000, chain.objs.size()=1 2026-02-24T22:34:25.251 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1626__shadow_.mkyjV27g4ZeV8OvGbPsZNMe3cEL0tg1_0 2026-02-24T22:34:25.251 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13640946414896491978', time=2026-02-25T00:25:59.100520+0000, chain.objs.size()=1 2026-02-24T22:34:25.251 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.DjLj0UdrTi6pJwFiTWTcjd2wKAfrZEv_0 2026-02-24T22:34:25.251 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7022076712404928888', time=2026-02-25T00:25:59.115105+0000, chain.objs.size()=1 2026-02-24T22:34:25.251 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.BduVor3yHThpaE78h-hUhwf22X1d3em_0 2026-02-24T22:34:25.251 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14509387916533779394', time=2026-02-25T00:25:59.115356+0000, chain.objs.size()=1 2026-02-24T22:34:25.251 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.3cjueF1IuqpkNcYybmEaA6AUZN8wh2B_0 2026-02-24T22:34:25.251 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4649399983295267944', time=2026-02-25T00:25:59.128756+0000, chain.objs.size()=1 2026-02-24T22:34:25.251 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.0lKyRJbLCrFVQbfQgZpxcQmENaSWhds_0 2026-02-24T22:34:25.251 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4564586432894240749', time=2026-02-25T00:25:59.242286+0000, chain.objs.size()=1 2026-02-24T22:34:25.251 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.OSHmg8csfWKLhFGYKVf1eJvbGXNsqfx_0 2026-02-24T22:34:25.258 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14745213718126513965', time=2026-02-25T00:25:59.267670+0000, chain.objs.size()=1 2026-02-24T22:34:25.258 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.IP2VIR2MQxlg4d_gK2qyFytfkRv8zQD_0 2026-02-24T22:34:25.258 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5563312846359689760', time=2026-02-25T00:25:59.279312+0000, chain.objs.size()=1 2026-02-24T22:34:25.258 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.jxLM1owBcVEehoBmXWeMA_f_frxOwXa_0 2026-02-24T22:34:25.258 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5156737357430565516', time=2026-02-25T00:25:59.314384+0000, chain.objs.size()=1 2026-02-24T22:34:25.258 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.b_fYxgrIh3enOn8v2lNDnQI1JzzCHBY_0 2026-02-24T22:34:25.258 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4648073341267838181', time=2026-02-25T00:25:59.332823+0000, chain.objs.size()=1 2026-02-24T22:34:25.258 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.B5gMRoxpAPXtjfDaluX-VPnhvdB8RSK_0 2026-02-24T22:34:25.258 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14318008287569389301', time=2026-02-25T00:25:59.415007+0000, chain.objs.size()=1 2026-02-24T22:34:25.259 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.blA5GA_HuIOKV5VrANYO63CUYmfeqGY_0 2026-02-24T22:34:25.259 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14738935734629331304', time=2026-02-25T00:25:59.431616+0000, chain.objs.size()=1 2026-02-24T22:34:25.259 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.7Q8lG1QX2LTB5XjWNgnSl7RMD8EA7Gx_0 2026-02-24T22:34:25.259 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5380049519262863395', time=2026-02-25T00:25:59.438151+0000, chain.objs.size()=1 2026-02-24T22:34:25.259 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.k59XXB0bnxlC7Ma9kwdvXOjayI6VthU_0 2026-02-24T22:34:25.259 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10770187422271904855', time=2026-02-25T00:25:59.442591+0000, chain.objs.size()=1 2026-02-24T22:34:25.259 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.AFE22P1FXZp-rd7Qt-qeTSG874Gcfcz_0 2026-02-24T22:34:25.259 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8137164093588008258', time=2026-02-25T00:25:59.464851+0000, chain.objs.size()=1 2026-02-24T22:34:25.259 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.8j9wgIS5ixLo2a7WQ8u0kpDzvJVBLEo_0 2026-02-24T22:34:25.259 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8866933603197401162', time=2026-02-25T00:25:59.490435+0000, chain.objs.size()=1 2026-02-24T22:34:25.259 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.P3ly9kqji-vmLCm4I4WTugSQSlWmugd_0 2026-02-24T22:34:25.259 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3822723519298073717', time=2026-02-25T00:25:59.567182+0000, chain.objs.size()=1 2026-02-24T22:34:25.259 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.rjdEPDP7sXDDRlmXVKO_2Ayict_A0RK_0 2026-02-24T22:34:25.260 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7825827692082226173', time=2026-02-25T00:25:59.567451+0000, chain.objs.size()=1 2026-02-24T22:34:25.260 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.7i-q3KqD1xHcouPqh-Br7Kgui0EaTbt_0 2026-02-24T22:34:25.260 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17958382026463393271', time=2026-02-25T00:25:59.590622+0000, chain.objs.size()=1 2026-02-24T22:34:25.260 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.8gfYBkHgeee1dsF5_bvl1VP_-1Na-eo_0 2026-02-24T22:34:25.260 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14382903434110026730', time=2026-02-25T00:25:59.628377+0000, chain.objs.size()=1 2026-02-24T22:34:25.260 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.iEgWkzrz4_vXofd_BO-AvHHXKjxXO9O_0 2026-02-24T22:34:25.260 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2311257793092523965', time=2026-02-25T00:25:59.631376+0000, chain.objs.size()=1 2026-02-24T22:34:25.260 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.n2e9SCzMgiy-pEn4HuqqNGGhMaRRRpG_0 2026-02-24T22:34:25.260 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10908079661180231942', time=2026-02-25T00:25:59.640515+0000, chain.objs.size()=1 2026-02-24T22:34:25.260 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.a24Jr5C48HoYdeAnEKLZXzDfXCJb4Wr_0 2026-02-24T22:34:25.260 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2231566482247316215', time=2026-02-25T00:25:59.649118+0000, chain.objs.size()=1 2026-02-24T22:34:25.260 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.h3shFXMI_vr7ZjzYTyCxWKIisH3kcYr_0 2026-02-24T22:34:25.260 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10612084923454123038', time=2026-02-25T00:25:59.726786+0000, chain.objs.size()=1 2026-02-24T22:34:25.261 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ItLhPNk_sm2JCvOn_71gRIrg0mZSjg9_0 2026-02-24T22:34:25.261 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1378299006209339895', time=2026-02-25T00:25:59.730134+0000, chain.objs.size()=1 2026-02-24T22:34:25.261 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.48lQCkXK1vdtWqaGAUgZLZy84UL8uDt_0 2026-02-24T22:34:25.261 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15906053345205995130', time=2026-02-25T00:25:59.738923+0000, chain.objs.size()=1 2026-02-24T22:34:25.261 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.4spZlSB5KHePPsbwIccSVQ5PExtBtPx_0 2026-02-24T22:34:25.261 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17319530687335590020', time=2026-02-25T00:25:59.786414+0000, chain.objs.size()=1 2026-02-24T22:34:25.261 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.2bkBLVHuc6jf_84G4EupaIrb8dgN2Zd_0 2026-02-24T22:34:25.261 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13843315746432287791', time=2026-02-25T00:25:59.806564+0000, chain.objs.size()=1 2026-02-24T22:34:25.261 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.o9kgFUW3v7iRHYLfsD45JU6Anmk-qcx_0 2026-02-24T22:34:25.261 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14535350284324453439', time=2026-02-25T00:25:59.824281+0000, chain.objs.size()=1 2026-02-24T22:34:25.261 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.259+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.HKkFZKvBM7e5CHxeLGCQlbClzGDazRd_0 2026-02-24T22:34:25.261 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4199466934815001658', time=2026-02-25T00:25:59.902180+0000, chain.objs.size()=1 2026-02-24T22:34:25.262 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.16RxEiD8zcY4XmMUvBxxrWlv5h1ieEr_0 2026-02-24T22:34:25.262 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3864227790253143216', time=2026-02-25T00:25:59.926411+0000, chain.objs.size()=1 2026-02-24T22:34:25.262 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.BeA0e1ojvQs7WpFh96EbupkS2mypKgC_0 2026-02-24T22:34:25.262 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6375634072992840884', time=2026-02-25T00:25:59.957465+0000, chain.objs.size()=1 2026-02-24T22:34:25.262 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.i8LTz42NjzHhFZnrilJxK8XcFILRfhX_0 2026-02-24T22:34:25.262 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10062123550152473207', time=2026-02-25T00:26:00.047792+0000, chain.objs.size()=1 2026-02-24T22:34:25.262 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.JSCHu2qvtNRp6hvg3YBzqVwuS36AJc5_0 2026-02-24T22:34:25.262 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1973456448560036345', time=2026-02-25T00:26:00.087024+0000, chain.objs.size()=1 2026-02-24T22:34:25.262 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.V_J8WpZLwGBFmyYNgHOzkWA8QqtYksE_0 2026-02-24T22:34:25.262 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15113242067847022753', time=2026-02-25T00:26:00.106970+0000, chain.objs.size()=1 2026-02-24T22:34:25.262 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.QsOZ-v9M2OWHrqsP-6-ONz5Zla_oZxn_0 2026-02-24T22:34:25.262 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.664526038495262069', time=2026-02-25T00:26:00.224159+0000, chain.objs.size()=1 2026-02-24T22:34:25.263 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.HZhA0Rnzn9DhuNnO1uiM74koMHP59fz_0 2026-02-24T22:34:25.263 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16612153218638500727', time=2026-02-25T00:26:00.250368+0000, chain.objs.size()=1 2026-02-24T22:34:25.263 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.PgtIAhb16ngcy6RWAP6YtQJl70-JtNs_0 2026-02-24T22:34:25.263 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16040465971950121021', time=2026-02-25T00:26:00.253161+0000, chain.objs.size()=1 2026-02-24T22:34:25.263 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.SHSsKZwr2SRTwDIKOfbWdM_8K0dlohp_0 2026-02-24T22:34:25.263 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7331005265535141170', time=2026-02-25T00:26:00.256394+0000, chain.objs.size()=1 2026-02-24T22:34:25.263 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.0Sj8ecnN6e9qWnaPQ6XmWFHrIbsd1kx_0 2026-02-24T22:34:25.263 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13246785336174372655', time=2026-02-25T00:26:00.263055+0000, chain.objs.size()=1 2026-02-24T22:34:25.263 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.T4C932AMFy-MR3_MElFnsAGt-Hi9hyW_0 2026-02-24T22:34:25.263 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6663450252813253590', time=2026-02-25T00:26:01.369853+0000, chain.objs.size()=1 2026-02-24T22:34:25.263 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1630__shadow_.MYxLDLS3jtdNWYQeVjzL2PqG7aiNdnl_0 2026-02-24T22:34:25.263 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3417565756480609584', time=2026-02-25T00:26:05.422092+0000, chain.objs.size()=1 2026-02-24T22:34:25.263 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1637__shadow_.p1zj1HY0Ha2brOiU4dD-cHnVnHZW0UL_0 2026-02-24T22:34:25.264 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16680060543804737560', time=2026-02-25T00:26:10.022650+0000, chain.objs.size()=1 2026-02-24T22:34:25.264 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1645__shadow_.cfx14G3Vg9M8adCbJp2OD902_SbXb2j_0 2026-02-24T22:34:25.264 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15592213215871918151', time=2026-02-25T00:26:11.187989+0000, chain.objs.size()=1 2026-02-24T22:34:25.264 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1647__shadow_.s2YhNoqeYBiYbwawUbH6liDz_s-mH0Z_0 2026-02-24T22:34:25.264 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15644865262611720987', time=2026-02-25T00:26:12.938484+0000, chain.objs.size()=1 2026-02-24T22:34:25.264 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1650__shadow_.m8IO0X2JcRWQEEfhidWDQuF5XsJVESs_0 2026-02-24T22:34:25.264 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9968565163156263727', time=2026-02-25T00:26:12.938920+0000, chain.objs.size()=1 2026-02-24T22:34:25.264 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1650__shadow_.8NrZuh76rnNTiMGO9Ug0kNQE_2W-nv3_0 2026-02-24T22:34:25.264 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6271785677727905269', time=2026-02-25T00:26:18.500760+0000, chain.objs.size()=1 2026-02-24T22:34:25.264 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1658__shadow_.h-MWeUQxEFEm6TZcP2BWDh_dkf1Tusr_0 2026-02-24T22:34:25.264 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14920227544640145665', time=2026-02-25T00:26:19.447187+0000, chain.objs.size()=1 2026-02-24T22:34:25.264 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1659__shadow_.0I8YQh8u3ZM3WLQ4zvkJODcPyN-BXqG_0 2026-02-24T22:34:25.265 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9111980404513311674', time=2026-02-25T00:26:19.451603+0000, chain.objs.size()=1 2026-02-24T22:34:25.265 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1659__shadow_.UCHeyQyificLKUy3kDl6YVg-UpvlSJM_0 2026-02-24T22:34:25.265 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18021213895371179265', time=2026-02-25T00:26:21.658349+0000, chain.objs.size()=1 2026-02-24T22:34:25.265 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1663__shadow_.O5rUDfclBwD_iFEWpbizSCvs83qfBsE_0 2026-02-24T22:34:25.265 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10664181916489290811', time=2026-02-25T00:26:22.866697+0000, chain.objs.size()=1 2026-02-24T22:34:25.265 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1665__shadow_.ETmypHQiOFdI54i8Fdpe5NECKvZKicJ_0 2026-02-24T22:34:25.265 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5494544687046815647', time=2026-02-25T00:26:24.638157+0000, chain.objs.size()=1 2026-02-24T22:34:25.265 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1668__shadow_.4cVMJ-HurVNsn-xAswB6x3-c8WV_h1g_0 2026-02-24T22:34:25.265 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7909605595357223722', time=2026-02-25T00:26:27.177538+0000, chain.objs.size()=1 2026-02-24T22:34:25.265 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1672__shadow_.ndD_C4LsR5LhEbmKJ811jNfurNcRqxu_0 2026-02-24T22:34:25.265 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12328432938980693684', time=2026-02-25T00:26:44.781188+0000, chain.objs.size()=1 2026-02-24T22:34:25.265 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.nLH9weIGZ8KgyJ0BfHopMbjDbJ-j_kE_0 2026-02-24T22:34:25.265 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1956591339368919122', time=2026-02-25T00:26:44.819363+0000, chain.objs.size()=1 2026-02-24T22:34:25.266 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.LwZfZLYVJShXz87JSU92X7Xy-R9WUpx_0 2026-02-24T22:34:25.266 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15643103193392424138', time=2026-02-25T00:26:44.826583+0000, chain.objs.size()=1 2026-02-24T22:34:25.266 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Ve0b9A0DKQvi1M7BuoSKBMb7h7yImPE_0 2026-02-24T22:34:25.266 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1417774155526246118', time=2026-02-25T00:26:44.844355+0000, chain.objs.size()=1 2026-02-24T22:34:25.266 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.nTiy3OojrL-f3o7Frj4pSgtllB_2HO1_0 2026-02-24T22:34:25.266 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2269418552869261461', time=2026-02-25T00:26:44.934591+0000, chain.objs.size()=1 2026-02-24T22:34:25.266 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.UG5mGBQnLCmE4Mnz1PZzDn0eYTSD-NB_0 2026-02-24T22:34:25.266 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3580045408834489812', time=2026-02-25T00:26:44.989812+0000, chain.objs.size()=1 2026-02-24T22:34:25.266 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Dk-KJ_s5fY7Eu7r9MoB9C2eIK97_0Px_0 2026-02-24T22:34:25.266 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14313968572926988039', time=2026-02-25T00:26:45.001280+0000, chain.objs.size()=1 2026-02-24T22:34:25.266 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.tpXQpxN1IEHlPvHUQCTzdVWGbPoXVDh_0 2026-02-24T22:34:25.266 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1027498527898970449', time=2026-02-25T00:26:45.114963+0000, chain.objs.size()=1 2026-02-24T22:34:25.266 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ynChwWCtUsHcoXAyeZgAbLAGKflh7dH_0 2026-02-24T22:34:25.267 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8118986795047526768', time=2026-02-25T00:26:45.142999+0000, chain.objs.size()=1 2026-02-24T22:34:25.267 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.YqpnNurKs5tdSV-qDz59zLBReH4Agv__0 2026-02-24T22:34:25.267 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2508344273513586851', time=2026-02-25T00:26:45.149900+0000, chain.objs.size()=1 2026-02-24T22:34:25.267 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.i87VuRUoq8YbKR2vVyP1m9h-XmZ_4zi_0 2026-02-24T22:34:25.267 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2829822750571889713', time=2026-02-25T00:26:45.159631+0000, chain.objs.size()=1 2026-02-24T22:34:25.267 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.263+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.KOvvX5tt13QvZacOreS9r2m3kbFb55y_0 2026-02-24T22:34:25.267 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11804138707992425370', time=2026-02-25T00:26:45.175020+0000, chain.objs.size()=1 2026-02-24T22:34:25.267 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.0UmbTI69fAt3j-rB6OS02CmtLomtTbv_0 2026-02-24T22:34:25.267 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8757051813999394127', time=2026-02-25T00:26:45.279612+0000, chain.objs.size()=1 2026-02-24T22:34:25.267 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.5mvFJyiOYzevLtz4bvmTp-YhsfsLXhT_0 2026-02-24T22:34:25.267 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6000828128089726279', time=2026-02-25T00:26:45.427418+0000, chain.objs.size()=1 2026-02-24T22:34:25.267 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.DGQSd0SXZ8hIhOuW_RmaCjaFKx3p-F8_0 2026-02-24T22:34:25.268 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9891274285367578377', time=2026-02-25T00:26:45.482221+0000, chain.objs.size()=1 2026-02-24T22:34:25.268 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ys73obu7v8E9m82amxJbqJj9zytDgEo_0 2026-02-24T22:34:25.268 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2628463214551687138', time=2026-02-25T00:26:45.585141+0000, chain.objs.size()=1 2026-02-24T22:34:25.268 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.1gZ8T92mDia07ZahuNWNaAfKuTFfQgB_0 2026-02-24T22:34:25.268 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.893140664563896511', time=2026-02-25T00:26:45.585607+0000, chain.objs.size()=1 2026-02-24T22:34:25.268 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.LjK5lsM-sy-cs-S_N4CW4vKHgatNByV_0 2026-02-24T22:34:25.268 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18313365713248916180', time=2026-02-25T00:26:45.591812+0000, chain.objs.size()=1 2026-02-24T22:34:25.268 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.60UuSdvONiOgRvqW6BTG41IN8CijtUp_0 2026-02-24T22:34:25.268 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15430120465789685827', time=2026-02-25T00:26:45.598990+0000, chain.objs.size()=1 2026-02-24T22:34:25.268 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.2RzPBcayoY7IqzPvXcuXDSXIkDjDeuo_0 2026-02-24T22:34:25.268 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16228669299566807991', time=2026-02-25T00:26:45.818395+0000, chain.objs.size()=1 2026-02-24T22:34:25.268 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.u82pyuVmz6jgSQhBDUfugherkRAmQSh_0 2026-02-24T22:34:25.269 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11197947837816776', time=2026-02-25T00:26:45.911346+0000, chain.objs.size()=1 2026-02-24T22:34:25.269 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.CkRMUc7Z3hK_oDQME4ZXepuiNmAR6Hj_0 2026-02-24T22:34:25.269 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13100586267010757615', time=2026-02-25T00:26:45.936205+0000, chain.objs.size()=1 2026-02-24T22:34:25.269 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.CDreifVS4cIOFryEVg6NF1GWRPoAXyO_0 2026-02-24T22:34:25.269 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15504635149482584649', time=2026-02-25T00:26:45.936612+0000, chain.objs.size()=1 2026-02-24T22:34:25.269 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.No7odnVMFvzyJvzXfCz1rQm6Jf9O-Mx_0 2026-02-24T22:34:25.269 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12268668868960222988', time=2026-02-25T00:26:45.941355+0000, chain.objs.size()=1 2026-02-24T22:34:25.269 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ew9GEaIf_DLQ37JolwmVohS8jrvmGEr_0 2026-02-24T22:34:25.269 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3720460468997639416', time=2026-02-25T00:26:45.945619+0000, chain.objs.size()=1 2026-02-24T22:34:25.269 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.0XRnaUO1TwUNQ-THZox0N4FTcsDoss1_0 2026-02-24T22:34:25.269 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11184898227037603689', time=2026-02-25T00:26:46.079572+0000, chain.objs.size()=1 2026-02-24T22:34:25.269 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ZP-nJJtZBeHAerisN528lhVwzaG9SvV_0 2026-02-24T22:34:25.269 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8402013103957247646', time=2026-02-25T00:26:46.092949+0000, chain.objs.size()=1 2026-02-24T22:34:25.269 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.3WIigwA7fN4b2MetySBjtOBcZ6YH-31_0 2026-02-24T22:34:25.270 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3739885229076348106', time=2026-02-25T00:26:46.094289+0000, chain.objs.size()=1 2026-02-24T22:34:25.270 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.RytixLty-E_F3S6b153kYlnS_NwrUS__0 2026-02-24T22:34:25.270 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12340891542599504086', time=2026-02-25T00:26:53.172280+0000, chain.objs.size()=1 2026-02-24T22:34:25.270 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ac-5tyCupHkVAcCvurEKq78LusgRyyM_0 2026-02-24T22:34:25.270 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11928224337920005848', time=2026-02-25T00:26:53.186318+0000, chain.objs.size()=1 2026-02-24T22:34:25.270 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.-SmFteA34wPb5440rDj3M91PdfsaN6v_0 2026-02-24T22:34:25.282 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.267+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/37935 2026-02-24T22:34:25.282 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=49, truncated=0, next_marker='' 2026-02-24T22:34:25.282 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16086900626918022591', time=2026-02-25T00:26:53.233389+0000, chain.objs.size()=1 2026-02-24T22:34:25.282 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.WDSynnndQq8_d7uSiFOei58VnqzpV4J_0 2026-02-24T22:34:25.282 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2543000994458794058', time=2026-02-25T00:26:53.331325+0000, chain.objs.size()=1 2026-02-24T22:34:25.282 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.W-oPyBqI70q35XDbz5Ajh67hOGUdxN7_0 2026-02-24T22:34:25.282 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15548524254050869150', time=2026-02-25T00:26:53.341918+0000, chain.objs.size()=1 2026-02-24T22:34:25.283 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.-UhQVwlomElpwoe7kcpxkfIXe3AMDMa_0 2026-02-24T22:34:25.283 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12626942154922382006', time=2026-02-25T00:26:53.356634+0000, chain.objs.size()=1 2026-02-24T22:34:25.283 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.aaiQpgCqjrVsEUy4LpxORkxh-u-usMf_0 2026-02-24T22:34:25.283 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15587357429582338747', time=2026-02-25T00:26:53.362453+0000, chain.objs.size()=1 2026-02-24T22:34:25.283 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.IHx9zJCXfnLqauxibIbKopdWMVx-wce_0 2026-02-24T22:34:25.283 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8473487685076190317', time=2026-02-25T00:26:53.521951+0000, chain.objs.size()=1 2026-02-24T22:34:25.283 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.fWc27UtaHbGjm96mKldq13OOtJsufxz_0 2026-02-24T22:34:25.283 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3781838602506186201', time=2026-02-25T00:26:53.531829+0000, chain.objs.size()=1 2026-02-24T22:34:25.283 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.SUvKhmL4TPubBpPnac-_X_bquN9dOdF_0 2026-02-24T22:34:25.283 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3677947998119846871', time=2026-02-25T00:26:53.549954+0000, chain.objs.size()=1 2026-02-24T22:34:25.283 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.MTDGLOE66LYB3A5QY611R6azy_CoBo5_0 2026-02-24T22:34:25.283 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4031819436438181625', time=2026-02-25T00:26:53.559709+0000, chain.objs.size()=1 2026-02-24T22:34:25.283 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.x-v3t1dzfxnJbrLSUdWQlclgWJ_FWM4_0 2026-02-24T22:34:25.283 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9395419557500169503', time=2026-02-25T00:26:53.741107+0000, chain.objs.size()=1 2026-02-24T22:34:25.283 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.bSI7rqn3st-Se2f7QyvpfNPz5OAU6VS_0 2026-02-24T22:34:25.284 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15512336047452505063', time=2026-02-25T00:26:53.833968+0000, chain.objs.size()=1 2026-02-24T22:34:25.284 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.OtmQrR0wj7fFB1YlAhNzHYbn8fagiAg_0 2026-02-24T22:34:25.284 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2054537520765971628', time=2026-02-25T00:26:53.935987+0000, chain.objs.size()=1 2026-02-24T22:34:25.284 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.M4A8J4AdQtAy7k9Z9YkgLSBi2RA_r-4_0 2026-02-24T22:34:25.284 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13291448099255888548', time=2026-02-25T00:26:53.967924+0000, chain.objs.size()=1 2026-02-24T22:34:25.284 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.OdFBd9Jus1-HuyALvlE7xo8hBHyZU6H_0 2026-02-24T22:34:25.284 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17183353534631649048', time=2026-02-25T00:26:54.017406+0000, chain.objs.size()=1 2026-02-24T22:34:25.284 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.NQW9QB9fBhqatUQYBqGSagOyFW17Vew_0 2026-02-24T22:34:25.284 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8388123876339056415', time=2026-02-25T00:26:54.024396+0000, chain.objs.size()=1 2026-02-24T22:34:25.284 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.eh5tfNNRL57H3NCxKpVukaSlMH6PPs3_0 2026-02-24T22:34:25.284 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2499756352629004822', time=2026-02-25T00:26:54.097174+0000, chain.objs.size()=1 2026-02-24T22:34:25.284 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.VQEmi4xzOsOQHVWfluTrLhp7eajT9nW_0 2026-02-24T22:34:25.284 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6276581196564771681', time=2026-02-25T00:26:54.128269+0000, chain.objs.size()=1 2026-02-24T22:34:25.284 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.q65mtwnXrsn5Qfp8AP9cybzvVuQP0I6_0 2026-02-24T22:34:25.284 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6392795311758941348', time=2026-02-25T00:26:54.148405+0000, chain.objs.size()=1 2026-02-24T22:34:25.285 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.BVe4LNfP0j9SntNYWIzZbQgNpKS6cea_0 2026-02-24T22:34:25.285 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5947840749154170687', time=2026-02-25T00:26:54.197449+0000, chain.objs.size()=1 2026-02-24T22:34:25.285 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.YokazBu_AjZxwVmaoQXGlgkyCjwgxGr_0 2026-02-24T22:34:25.285 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11471325856687527548', time=2026-02-25T00:26:54.371999+0000, chain.objs.size()=1 2026-02-24T22:34:25.285 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.csfEX7xkZzVMD02er1hIIRtY1Y6KPxQ_0 2026-02-24T22:34:25.285 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2200671640925886136', time=2026-02-25T00:26:54.381268+0000, chain.objs.size()=1 2026-02-24T22:34:25.285 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.FEoX_Ztukb4E-R-u1iJBHjbrUW26W4E_0 2026-02-24T22:34:25.285 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9851827751494191573', time=2026-02-25T00:26:54.387094+0000, chain.objs.size()=1 2026-02-24T22:34:25.285 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.oiU6JmRB2s7CaX4olj5_PT85V9Lr8QI_0 2026-02-24T22:34:25.285 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2093743474458592942', time=2026-02-25T00:26:54.451644+0000, chain.objs.size()=1 2026-02-24T22:34:25.285 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.wCVQ2VEApZN81lYw4g8sQnavHuZoHeN_0 2026-02-24T22:34:25.285 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9600882412831173488', time=2026-02-25T00:26:54.469537+0000, chain.objs.size()=1 2026-02-24T22:34:25.285 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.08WuQiiWxLpb7M0J_pTuYrufu5Y4TMn_0 2026-02-24T22:34:25.285 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.99345262909779671', time=2026-02-25T00:26:54.498098+0000, chain.objs.size()=1 2026-02-24T22:34:25.285 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.akwYagVYvmq8E7BN0yVltk-0P6GsAOa_0 2026-02-24T22:34:25.285 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14609772460140338380', time=2026-02-25T00:26:54.505050+0000, chain.objs.size()=1 2026-02-24T22:34:25.285 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.i_ceRsckUbq3Y_cON1x4l289AqLm8fM_0 2026-02-24T22:34:25.286 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9293101776003190044', time=2026-02-25T00:26:54.525106+0000, chain.objs.size()=1 2026-02-24T22:34:25.286 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.NC5cuJ2FdCogTEJ1AkZnCdmAK3qiZ2g_0 2026-02-24T22:34:25.286 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8046306622206705844', time=2026-02-25T00:27:05.280944+0000, chain.objs.size()=1 2026-02-24T22:34:25.286 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1708__shadow_.LjHxpktHgFURY47b4fzOHxmUIDPLPja_0 2026-02-24T22:34:25.286 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9547556291469145776', time=2026-02-25T00:27:27.472426+0000, chain.objs.size()=1 2026-02-24T22:34:25.286 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1747__shadow_.n7Gxc_OxM6xoWXkJLlE_XzzWMeyF1Be_0 2026-02-24T22:34:25.286 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17159665320428342838', time=2026-02-25T00:28:07.780082+0000, chain.objs.size()=1 2026-02-24T22:34:25.286 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1808__shadow_.nOXR7yqEwP1BRdoHV875RG7V6QVje2E_0 2026-02-24T22:34:25.286 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17206965821808938990', time=2026-02-25T00:28:46.323901+0000, chain.objs.size()=6 2026-02-24T22:34:25.286 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.271+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.1 2026-02-24T22:34:25.286 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.2 2026-02-24T22:34:25.286 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.3 2026-02-24T22:34:25.286 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.4 2026-02-24T22:34:25.286 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.5 2026-02-24T22:34:25.286 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.6 2026-02-24T22:34:25.286 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15769087550878145379', time=2026-02-25T00:28:52.393358+0000, chain.objs.size()=1 2026-02-24T22:34:25.286 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1883__shadow_.7-QVsJJ_a26tM_Yy9lHpv3LRHdcRvBq_0 2026-02-24T22:34:25.286 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13764212706828688495', time=2026-02-25T00:28:55.039586+0000, chain.objs.size()=6 2026-02-24T22:34:25.287 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1885__multipart_mymultipart.2~YRBCUGfoYzCdgTemIZziGRHlSsRoikI.1 2026-02-24T22:34:25.287 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1885__multipart_mymultipart.2~YRBCUGfoYzCdgTemIZziGRHlSsRoikI.2 2026-02-24T22:34:25.287 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1885__multipart_mymultipart.2~YRBCUGfoYzCdgTemIZziGRHlSsRoikI.3 2026-02-24T22:34:25.287 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1885__multipart_mymultipart.2~YRBCUGfoYzCdgTemIZziGRHlSsRoikI.4 2026-02-24T22:34:25.287 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1885__multipart_mymultipart.2~YRBCUGfoYzCdgTemIZziGRHlSsRoikI.5 2026-02-24T22:34:25.287 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1885__multipart_mymultipart.2~YRBCUGfoYzCdgTemIZziGRHlSsRoikI.6 2026-02-24T22:34:25.287 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17824052400611775830', time=2026-02-25T00:29:09.659170+0000, chain.objs.size()=6 2026-02-24T22:34:25.287 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.g_jusjS-F1e4pFW4pZVxCtpkyV-Bmdei.1 2026-02-24T22:34:25.287 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.MCZueAHxPqr-yHGzbCrcyr-x_qFt8Yxe.2 2026-02-24T22:34:25.287 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.QyKI_WOr-q-Xdj1WHo9i0UAt2FlAHpCp.3 2026-02-24T22:34:25.287 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.3m6lJCR650DycFP38YjIi9eGJ2UMr2Nl.4 2026-02-24T22:34:25.287 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.yHDYgh__38PPirKO809VOOujidPBn5NC.5 2026-02-24T22:34:25.287 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.MJp3O1Vbqqa9TzRgj5tBr6532GF2S-o2.6 2026-02-24T22:34:25.287 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8071369932490162856', time=2026-02-25T00:29:30.714931+0000, chain.objs.size()=4 2026-02-24T22:34:25.287 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1901__multipart_mymultipart.2~jRnSmZ1txKp6p3-SQ7aUPqPRjO15SLU.1 2026-02-24T22:34:25.287 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1901__multipart_mymultipart.2~jRnSmZ1txKp6p3-SQ7aUPqPRjO15SLU.2 2026-02-24T22:34:25.288 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1901__multipart_mymultipart.WBtHT3_yFZEtKR2kHf4akdiwEuCeo57Y.3 2026-02-24T22:34:25.288 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1901__multipart_mymultipart.2~jRnSmZ1txKp6p3-SQ7aUPqPRjO15SLU.4 2026-02-24T22:34:25.288 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.406174939292114331', time=2026-02-25T00:30:06.905262+0000, chain.objs.size()=1 2026-02-24T22:34:25.288 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1924__shadow_.91QpDn_UbYtzIJNKJRn5ZTLw-hs34eI_0 2026-02-24T22:34:25.288 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13778463530573172672', time=2026-02-25T00:30:34.522676+0000, chain.objs.size()=1 2026-02-24T22:34:25.288 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1946__shadow:6Yly2FkOc1C596.UrqLJTXEOYf77Bw9_.QYhP9I_o6XX4THI0UYHfZMNpKLC63d5_0 2026-02-24T22:34:25.288 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11407488871667652210', time=2026-02-25T00:30:40.746874+0000, chain.objs.size()=1 2026-02-24T22:34:25.288 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:29beBKiWZp5r2XYmDvLg-ZyJvsgn0DR_._Wo8S049oa6Rh-ybEGGy8ICwhvnhf-B_0 2026-02-24T22:34:25.288 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7932496981664894317', time=2026-02-25T00:30:47.334788+0000, chain.objs.size()=1 2026-02-24T22:34:25.288 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.275+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:xgDAmP5-DWl9MEn.18m-ZRveygDq7P-_.KhBEUDZaoeeP1d7lkYrUM3bm0Lr-4OE_0 2026-02-24T22:34:25.288 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.279+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1199801132943037942', time=2026-02-25T00:30:52.389568+0000, chain.objs.size()=1 2026-02-24T22:34:25.288 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.279+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1950__shadow:3qosJlIk2xJ72kWDV6fsMvIm8nsCIqd_.aEMiPEY5YHs3lv1MLBaT1QummQJqwDE_0 2026-02-24T22:34:25.288 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.279+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13068593423740127777', time=2026-02-25T00:30:57.543248+0000, chain.objs.size()=1 2026-02-24T22:34:25.288 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.279+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:0OD36C1EcxNxL5pLir6vRbETVFyTiNS_.xnTpSD2MKyh6NxzTADd7DDAiQ1KRCrl_0 2026-02-24T22:34:25.288 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.279+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13068593423740127777', time=2026-02-25T00:30:57.544315+0000, chain.objs.size()=1 2026-02-24T22:34:25.288 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.279+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:0OD36C1EcxNxL5pLir6vRbETVFyTiNS_.xnTpSD2MKyh6NxzTADd7DDAiQ1KRCrl_0 2026-02-24T22:34:25.288 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.279+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7069841419630979585', time=2026-02-25T00:30:57.545675+0000, chain.objs.size()=1 2026-02-24T22:34:25.288 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.279+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:bN-R4YSJxm4V-txutkHHQkmE8InD1x9_.0I3NAIfB3BYV9U3NkP5Sy0FO9vqXeII_0 2026-02-24T22:34:25.289 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.279+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7069841419630979585', time=2026-02-25T00:30:57.546583+0000, chain.objs.size()=1 2026-02-24T22:34:25.289 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.279+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:bN-R4YSJxm4V-txutkHHQkmE8InD1x9_.0I3NAIfB3BYV9U3NkP5Sy0FO9vqXeII_0 2026-02-24T22:34:25.289 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.279+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6970051933367477340', time=2026-02-25T00:30:57.717906+0000, chain.objs.size()=1 2026-02-24T22:34:25.289 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.279+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:xv7uSxZQtDoIQZISCPtTcPNP.aeJDTK_.gm_6AZPsC82P6oBhHKpcw-bhaqNq0Fm_0 2026-02-24T22:34:25.289 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.279+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6970051933367477340', time=2026-02-25T00:30:57.718399+0000, chain.objs.size()=1 2026-02-24T22:34:25.289 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.279+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:xv7uSxZQtDoIQZISCPtTcPNP.aeJDTK_.gm_6AZPsC82P6oBhHKpcw-bhaqNq0Fm_0 2026-02-24T22:34:25.289 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.279+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3410291384313178571', time=2026-02-25T00:30:58.532481+0000, chain.objs.size()=1 2026-02-24T22:34:25.289 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.279+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958__shadow:UoTSyiRk4xuGxvxtP-pjCTE3hqbTU8g_.SzqTYKru7BvK5jCfg-A8HzQ1r7fU4IB_0 2026-02-24T22:34:25.289 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.279+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3410291384313178571', time=2026-02-25T00:30:58.532937+0000, chain.objs.size()=1 2026-02-24T22:34:25.289 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.279+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958__shadow:UoTSyiRk4xuGxvxtP-pjCTE3hqbTU8g_.SzqTYKru7BvK5jCfg-A8HzQ1r7fU4IB_0 2026-02-24T22:34:25.289 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.279+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17066240684055347104', time=2026-02-25T00:34:17.244131+0000, chain.objs.size()=1 2026-02-24T22:34:25.289 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.279+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1964__shadow_.0zw8njJRZg9oS7Yphhj1liLdE2GBs6p_0 2026-02-24T22:34:25.289 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.279+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.289 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2026-02-24T22:34:25.289 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.289 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.289 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/39199' 2026-02-24T22:34:25.290 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16726018150874475645', time=2026-02-25T00:16:33.518979+0000, chain.objs.size()=1 2026-02-24T22:34:25.290 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521__shadow_.sxaMDDjxwChak3zWu4GqnfbY2FA4qmd_0 2026-02-24T22:34:25.290 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12481400847967550854', time=2026-02-25T00:17:42.540427+0000, chain.objs.size()=1 2026-02-24T22:34:25.290 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527__shadow_.C5g5DDNTi_ZTPyhhqSqap9B9ufI2QfW_0 2026-02-24T22:34:25.290 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9962421514402434834', time=2026-02-25T00:18:14.582933+0000, chain.objs.size()=1 2026-02-24T22:34:25.290 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528__shadow_.xGykQs1KnYTbxyi7TQKsz6DK525V5Cl_0 2026-02-24T22:34:25.290 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1921196200392897447', time=2026-02-25T00:18:14.583658+0000, chain.objs.size()=1 2026-02-24T22:34:25.290 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528__shadow_.le4wXgEawvSrh_fV4wDW4HAw2HwfQIo_0 2026-02-24T22:34:25.290 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9187985563043216435', time=2026-02-25T00:18:16.131179+0000, chain.objs.size()=1 2026-02-24T22:34:25.290 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529__shadow_.Jep6xTLbBMb0MFPw96d_rNTmW_QKaPH_0 2026-02-24T22:34:25.290 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12880227614091666374', time=2026-02-25T00:18:16.133811+0000, chain.objs.size()=1 2026-02-24T22:34:25.290 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529__shadow_.jKUEnJIL8V0AkxPSEHB7b__SRsaM7sT_0 2026-02-24T22:34:25.290 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.883448174673746255', time=2026-02-25T00:18:48.193042+0000, chain.objs.size()=1 2026-02-24T22:34:25.290 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530__shadow_.28IxsZ3aG53WVoc99Gv0pRx00Y9IBjj_0 2026-02-24T22:34:25.290 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9741622361895389715', time=2026-02-25T00:19:23.071626+0000, chain.objs.size()=1 2026-02-24T22:34:25.290 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533__shadow_.8QeTanRHaSam3m6Mur7HzUkagM0entE_0 2026-02-24T22:34:25.290 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16261901585608956721', time=2026-02-25T00:19:58.464692+0000, chain.objs.size()=1 2026-02-24T22:34:25.290 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536__shadow_.Z7LH3DAFeRTBeJO0Y_ynZ7QU6UDlAA9_0 2026-02-24T22:34:25.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4211684762830522153', time=2026-02-25T00:20:34.039216+0000, chain.objs.size()=1 2026-02-24T22:34:25.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539__shadow_.yUP2gq8TjE9vRJnR91tD0pt8N9I1DUZ_0 2026-02-24T22:34:25.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1016688141615440648', time=2026-02-25T00:21:06.321011+0000, chain.objs.size()=1 2026-02-24T22:34:25.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540__shadow_.2aqt2117-PwI5q-zQqFU0uNzhQ628KV_0 2026-02-24T22:34:25.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5685869552602153787', time=2026-02-25T00:23:14.364095+0000, chain.objs.size()=1 2026-02-24T22:34:25.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518__shadow_.i-fatkgstnsD5ok7BdLa4GXr331O_ZY_0 2026-02-24T22:34:25.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3950675936394060826', time=2026-02-25T00:25:51.958420+0000, chain.objs.size()=1 2026-02-24T22:34:25.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1626__shadow_.g7ofT0-cyy1JnBEHb5XHG9yngtiwIsR_0 2026-02-24T22:34:25.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7512969908912935379', time=2026-02-25T00:25:59.108074+0000, chain.objs.size()=1 2026-02-24T22:34:25.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.8B3YG4JVuSyZxYB71WBRjIP0C_sTA8g_0 2026-02-24T22:34:25.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6443431158370806609', time=2026-02-25T00:25:59.109003+0000, chain.objs.size()=1 2026-02-24T22:34:25.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.YQa_05DxHnLjqi-g068J4hrGKq6bvRh_0 2026-02-24T22:34:25.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.857549482419020919', time=2026-02-25T00:25:59.123602+0000, chain.objs.size()=1 2026-02-24T22:34:25.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.YIgG7eaCngTzHrp0A81_I63rsHepHgC_0 2026-02-24T22:34:25.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5076188384296889567', time=2026-02-25T00:25:59.144093+0000, chain.objs.size()=1 2026-02-24T22:34:25.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.tlDuvVNg3T6j1aAmUgUcMt0mxPV8syb_0 2026-02-24T22:34:25.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10896042047107743107', time=2026-02-25T00:25:59.170691+0000, chain.objs.size()=1 2026-02-24T22:34:25.291 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.EnUOD3xk3wo3SCoJBoPkC233DWQqrpn_0 2026-02-24T22:34:25.292 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.106594790644453121', time=2026-02-25T00:25:59.258828+0000, chain.objs.size()=1 2026-02-24T22:34:25.292 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.MoyXvG8P2DhY4Knp403iaIsFaciZ13m_0 2026-02-24T22:34:25.292 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3382110060139296956', time=2026-02-25T00:25:59.273556+0000, chain.objs.size()=1 2026-02-24T22:34:25.292 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ZW6gZbvMp840N7iRXcckdcpP_KJNXY4_0 2026-02-24T22:34:25.292 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17244353126087811458', time=2026-02-25T00:25:59.285270+0000, chain.objs.size()=1 2026-02-24T22:34:25.292 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.WPPLbC5_9H0Rx4KmE97SyZl85VQNVbN_0 2026-02-24T22:34:25.292 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17924867382766666389', time=2026-02-25T00:25:59.286299+0000, chain.objs.size()=1 2026-02-24T22:34:25.292 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.CK1VH_iPq7jHu8TUxjXz50_aGni7HI-_0 2026-02-24T22:34:25.292 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4861568760981258871', time=2026-02-25T00:25:59.291458+0000, chain.objs.size()=1 2026-02-24T22:34:25.292 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.XwFT-ar__ZChM-wMm9AvCUlwFPkeYNN_0 2026-02-24T22:34:25.292 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10460131167261244630', time=2026-02-25T00:25:59.295401+0000, chain.objs.size()=1 2026-02-24T22:34:25.292 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.283+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.vK2jqcTmmaRzzjdw7C9Cibelfnv6yax_0 2026-02-24T22:34:25.292 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12123431267961990067', time=2026-02-25T00:25:59.297282+0000, chain.objs.size()=1 2026-02-24T22:34:25.292 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.nzAC5i5nYug4wQ5LY_jJhSyxYlCwEO4_0 2026-02-24T22:34:25.292 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11246089220914157889', time=2026-02-25T00:25:59.331397+0000, chain.objs.size()=1 2026-02-24T22:34:25.292 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ijVv3njb9cyQ8w0bszyaCCM04xozOET_0 2026-02-24T22:34:25.292 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5641588376728647817', time=2026-02-25T00:25:59.339794+0000, chain.objs.size()=1 2026-02-24T22:34:25.292 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.CXWuLsV4_H--_42qiAVxPE9QDvHTWRd_0 2026-02-24T22:34:25.292 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8986998809479166483', time=2026-02-25T00:25:59.462095+0000, chain.objs.size()=1 2026-02-24T22:34:25.293 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.T7PiUX6aqzySLgOCOeyhG_0jaTePSaD_0 2026-02-24T22:34:25.293 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13490729260715055098', time=2026-02-25T00:25:59.491746+0000, chain.objs.size()=1 2026-02-24T22:34:25.293 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.4GGTuKH4-ABE4LLs0YijDI3XJyCoeSC_0 2026-02-24T22:34:25.293 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6023113947187123151', time=2026-02-25T00:25:59.502526+0000, chain.objs.size()=1 2026-02-24T22:34:25.293 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.IsAYrcKukWSzugRTSjZz3a5-1YX03Cx_0 2026-02-24T22:34:25.293 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2687741429774636596', time=2026-02-25T00:25:59.506006+0000, chain.objs.size()=1 2026-02-24T22:34:25.293 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.YReFhxhQNRAq3SDEm_bFpVp93Ypt7QP_0 2026-02-24T22:34:25.293 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12205582948601329835', time=2026-02-25T00:25:59.581201+0000, chain.objs.size()=1 2026-02-24T22:34:25.293 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.AhIcbIwfXDyA4GaPHhaQdCMFPcPASex_0 2026-02-24T22:34:25.293 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12139076163100627754', time=2026-02-25T00:25:59.592194+0000, chain.objs.size()=1 2026-02-24T22:34:25.293 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.CddlcwkYbnOirNuXrHihIpHZJSLuD4w_0 2026-02-24T22:34:25.293 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11031461568094333426', time=2026-02-25T00:25:59.628412+0000, chain.objs.size()=1 2026-02-24T22:34:25.293 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.mGRq6jVE0895UCv5c78ipLVrJCvyc5__0 2026-02-24T22:34:25.293 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11358735427315484006', time=2026-02-25T00:25:59.662638+0000, chain.objs.size()=1 2026-02-24T22:34:25.293 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.2LTdB71Bk2DzyLMha_7ROpJu-PnKNAD_0 2026-02-24T22:34:25.293 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9657411644936576381', time=2026-02-25T00:25:59.808891+0000, chain.objs.size()=1 2026-02-24T22:34:25.293 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.fjgf96d-hxrt1sFJ5zH6OhFe90x2OOp_0 2026-02-24T22:34:25.293 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1191288434704323597', time=2026-02-25T00:25:59.813260+0000, chain.objs.size()=1 2026-02-24T22:34:25.293 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.6k1cSBmOMeB5i5lfpCN3Owj8NXkCETj_0 2026-02-24T22:34:25.294 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13724720149873093448', time=2026-02-25T00:25:59.907183+0000, chain.objs.size()=1 2026-02-24T22:34:25.294 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.CWELig3cVzpRatQKgsTQSlSRp41QEqw_0 2026-02-24T22:34:25.294 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1603857197963056315', time=2026-02-25T00:25:59.920552+0000, chain.objs.size()=1 2026-02-24T22:34:25.294 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.O1uAd_Dkgk50a2AWlcQfD3mrWNV93bB_0 2026-02-24T22:34:25.294 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14482117181944547422', time=2026-02-25T00:25:59.959705+0000, chain.objs.size()=1 2026-02-24T22:34:25.294 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.L2mUVaDuuVZ5R4m-poQxPIuuc_6Mdqp_0 2026-02-24T22:34:25.294 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6944019902809625234', time=2026-02-25T00:25:59.964377+0000, chain.objs.size()=1 2026-02-24T22:34:25.294 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.mydwQqRxr_5sSQVmfnNXsuDb6Gsc2zT_0 2026-02-24T22:34:25.294 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11771396049906093799', time=2026-02-25T00:25:59.972677+0000, chain.objs.size()=1 2026-02-24T22:34:25.294 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.dGnGGuTaIBPrUznwQVEaNif7t-aHyCG_0 2026-02-24T22:34:25.294 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.382880853699463564', time=2026-02-25T00:26:00.145443+0000, chain.objs.size()=1 2026-02-24T22:34:25.294 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ziz2Qd-DV4UA55ayun8eguddBAnHSIW_0 2026-02-24T22:34:25.294 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11685776287032468186', time=2026-02-25T00:26:00.248345+0000, chain.objs.size()=1 2026-02-24T22:34:25.294 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ghPiCYnSdOd2aVO5fAleuLKBLFeaTzP_0 2026-02-24T22:34:25.294 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12387789001095064392', time=2026-02-25T00:26:00.262139+0000, chain.objs.size()=1 2026-02-24T22:34:25.294 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.NytL5Bpfu8slwcrCFmpVBgMbwLDQZ7Q_0 2026-02-24T22:34:25.294 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.805872871187529708', time=2026-02-25T00:26:00.267074+0000, chain.objs.size()=1 2026-02-24T22:34:25.294 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.waiFdc1qZvuWH5JVLR80Wn7JWzZ2UQi_0 2026-02-24T22:34:25.294 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10140027043247935832', time=2026-02-25T00:26:16.590442+0000, chain.objs.size()=1 2026-02-24T22:34:25.294 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1656__shadow_.D95sxNfjhEk2rXwlCz4yhZz58-N5ESV_0 2026-02-24T22:34:25.294 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17635006054512209668', time=2026-02-25T00:26:19.464085+0000, chain.objs.size()=1 2026-02-24T22:34:25.295 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1659__shadow_.FT6X2YzJk7hXmLw4HJiHwH7O6Mhprwb_0 2026-02-24T22:34:25.295 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.274637596609677294', time=2026-02-25T00:26:20.046009+0000, chain.objs.size()=1 2026-02-24T22:34:25.295 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1660__shadow_.d0AccXSM-EmcF3ZgWygRifISoY_KHdC_0 2026-02-24T22:34:25.295 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18384009194512493809', time=2026-02-25T00:26:21.673923+0000, chain.objs.size()=1 2026-02-24T22:34:25.295 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1663__shadow_.irGy5-n7H8FQZTXHzm-CFGtc1MahQ-9_0 2026-02-24T22:34:25.295 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2702247556962992031', time=2026-02-25T00:26:25.857704+0000, chain.objs.size()=1 2026-02-24T22:34:25.295 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1670__shadow_.3msmYBvRNiRK50LWNUUOZExm4inStGB_0 2026-02-24T22:34:25.295 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2555886613510213951', time=2026-02-25T00:26:35.804093+0000, chain.objs.size()=1 2026-02-24T22:34:25.295 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:cPQAMdlKODQDEvBVQibdtaSelUQLYGc_.ErxFESPPBO7ycRod-oaMNdZsFp0hGKQ_0 2026-02-24T22:34:25.295 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2555886613510213951', time=2026-02-25T00:26:35.811790+0000, chain.objs.size()=1 2026-02-24T22:34:25.295 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:cPQAMdlKODQDEvBVQibdtaSelUQLYGc_.ErxFESPPBO7ycRod-oaMNdZsFp0hGKQ_0 2026-02-24T22:34:25.295 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2555886613510213951', time=2026-02-25T00:26:35.812414+0000, chain.objs.size()=1 2026-02-24T22:34:25.295 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:cPQAMdlKODQDEvBVQibdtaSelUQLYGc_.ErxFESPPBO7ycRod-oaMNdZsFp0hGKQ_0 2026-02-24T22:34:25.295 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2555886613510213951', time=2026-02-25T00:26:35.815580+0000, chain.objs.size()=1 2026-02-24T22:34:25.295 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:cPQAMdlKODQDEvBVQibdtaSelUQLYGc_.ErxFESPPBO7ycRod-oaMNdZsFp0hGKQ_0 2026-02-24T22:34:25.295 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2555886613510213951', time=2026-02-25T00:26:35.817994+0000, chain.objs.size()=1 2026-02-24T22:34:25.295 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:cPQAMdlKODQDEvBVQibdtaSelUQLYGc_.ErxFESPPBO7ycRod-oaMNdZsFp0hGKQ_0 2026-02-24T22:34:25.295 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2555886613510213951', time=2026-02-25T00:26:35.831732+0000, chain.objs.size()=1 2026-02-24T22:34:25.295 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:cPQAMdlKODQDEvBVQibdtaSelUQLYGc_.ErxFESPPBO7ycRod-oaMNdZsFp0hGKQ_0 2026-02-24T22:34:25.295 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2555886613510213951', time=2026-02-25T00:26:35.832123+0000, chain.objs.size()=1 2026-02-24T22:34:25.296 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:cPQAMdlKODQDEvBVQibdtaSelUQLYGc_.ErxFESPPBO7ycRod-oaMNdZsFp0hGKQ_0 2026-02-24T22:34:25.296 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2555886613510213951', time=2026-02-25T00:26:35.837077+0000, chain.objs.size()=1 2026-02-24T22:34:25.296 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:cPQAMdlKODQDEvBVQibdtaSelUQLYGc_.ErxFESPPBO7ycRod-oaMNdZsFp0hGKQ_0 2026-02-24T22:34:25.296 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2555886613510213951', time=2026-02-25T00:26:35.838276+0000, chain.objs.size()=1 2026-02-24T22:34:25.296 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:cPQAMdlKODQDEvBVQibdtaSelUQLYGc_.ErxFESPPBO7ycRod-oaMNdZsFp0hGKQ_0 2026-02-24T22:34:25.296 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2555886613510213951', time=2026-02-25T00:26:35.847764+0000, chain.objs.size()=1 2026-02-24T22:34:25.296 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:cPQAMdlKODQDEvBVQibdtaSelUQLYGc_.ErxFESPPBO7ycRod-oaMNdZsFp0hGKQ_0 2026-02-24T22:34:25.296 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2555886613510213951', time=2026-02-25T00:26:35.848162+0000, chain.objs.size()=1 2026-02-24T22:34:25.296 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:cPQAMdlKODQDEvBVQibdtaSelUQLYGc_.ErxFESPPBO7ycRod-oaMNdZsFp0hGKQ_0 2026-02-24T22:34:25.296 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2555886613510213951', time=2026-02-25T00:26:35.850488+0000, chain.objs.size()=1 2026-02-24T22:34:25.296 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:cPQAMdlKODQDEvBVQibdtaSelUQLYGc_.ErxFESPPBO7ycRod-oaMNdZsFp0hGKQ_0 2026-02-24T22:34:25.296 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2555886613510213951', time=2026-02-25T00:26:35.851867+0000, chain.objs.size()=1 2026-02-24T22:34:25.296 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.287+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:cPQAMdlKODQDEvBVQibdtaSelUQLYGc_.ErxFESPPBO7ycRod-oaMNdZsFp0hGKQ_0 2026-02-24T22:34:25.296 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2555886613510213951', time=2026-02-25T00:26:35.864144+0000, chain.objs.size()=1 2026-02-24T22:34:25.296 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:cPQAMdlKODQDEvBVQibdtaSelUQLYGc_.ErxFESPPBO7ycRod-oaMNdZsFp0hGKQ_0 2026-02-24T22:34:25.296 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2555886613510213951', time=2026-02-25T00:26:35.864528+0000, chain.objs.size()=1 2026-02-24T22:34:25.296 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:cPQAMdlKODQDEvBVQibdtaSelUQLYGc_.ErxFESPPBO7ycRod-oaMNdZsFp0hGKQ_0 2026-02-24T22:34:25.296 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2555886613510213951', time=2026-02-25T00:26:35.865934+0000, chain.objs.size()=1 2026-02-24T22:34:25.296 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:cPQAMdlKODQDEvBVQibdtaSelUQLYGc_.ErxFESPPBO7ycRod-oaMNdZsFp0hGKQ_0 2026-02-24T22:34:25.296 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2555886613510213951', time=2026-02-25T00:26:35.868300+0000, chain.objs.size()=1 2026-02-24T22:34:25.296 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:cPQAMdlKODQDEvBVQibdtaSelUQLYGc_.ErxFESPPBO7ycRod-oaMNdZsFp0hGKQ_0 2026-02-24T22:34:25.297 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2555886613510213951', time=2026-02-25T00:26:35.877912+0000, chain.objs.size()=1 2026-02-24T22:34:25.297 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:cPQAMdlKODQDEvBVQibdtaSelUQLYGc_.ErxFESPPBO7ycRod-oaMNdZsFp0hGKQ_0 2026-02-24T22:34:25.297 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2555886613510213951', time=2026-02-25T00:26:35.878197+0000, chain.objs.size()=1 2026-02-24T22:34:25.297 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:cPQAMdlKODQDEvBVQibdtaSelUQLYGc_.ErxFESPPBO7ycRod-oaMNdZsFp0hGKQ_0 2026-02-24T22:34:25.297 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2555886613510213951', time=2026-02-25T00:26:35.879922+0000, chain.objs.size()=1 2026-02-24T22:34:25.297 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:cPQAMdlKODQDEvBVQibdtaSelUQLYGc_.ErxFESPPBO7ycRod-oaMNdZsFp0hGKQ_0 2026-02-24T22:34:25.297 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2555886613510213951', time=2026-02-25T00:26:35.894941+0000, chain.objs.size()=1 2026-02-24T22:34:25.297 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:cPQAMdlKODQDEvBVQibdtaSelUQLYGc_.ErxFESPPBO7ycRod-oaMNdZsFp0hGKQ_0 2026-02-24T22:34:25.297 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11741060830022728702', time=2026-02-25T00:26:44.828928+0000, chain.objs.size()=1 2026-02-24T22:34:25.297 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.M72WRGU96RrkLi68XGMLxZdIvXgUnmD_0 2026-02-24T22:34:25.297 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16915321204883209327', time=2026-02-25T00:26:44.842559+0000, chain.objs.size()=1 2026-02-24T22:34:25.297 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.4sQ_120_YsYFEXKjbsZ8weBprY0XTaG_0 2026-02-24T22:34:25.297 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7209988823927669081', time=2026-02-25T00:26:44.863795+0000, chain.objs.size()=1 2026-02-24T22:34:25.297 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.e7zh2TNtHIv7nPBW1Ae6Xk3hFsCwFsu_0 2026-02-24T22:34:25.297 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4566166353272614591', time=2026-02-25T00:26:44.981140+0000, chain.objs.size()=1 2026-02-24T22:34:25.297 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.osjzcqMEMm5ftdJL8xTLfspj6Q4hbUQ_0 2026-02-24T22:34:25.297 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6082762009233122787', time=2026-02-25T00:26:45.007767+0000, chain.objs.size()=1 2026-02-24T22:34:25.297 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.vICfZznjfwFmr2VK2Ps1Nu4qcIsiz41_0 2026-02-24T22:34:25.297 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1471190027374722865', time=2026-02-25T00:26:45.120974+0000, chain.objs.size()=1 2026-02-24T22:34:25.297 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.291+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.SQMdUMwk4WRto-olnvgN5DjwXxUzgXZ_0 2026-02-24T22:34:25.302 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9767643021438359992', time=2026-02-25T00:26:45.134372+0000, chain.objs.size()=1 2026-02-24T22:34:25.302 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.3qAOUhrqFXQWivTxxiX-9FiIyXGh45K_0 2026-02-24T22:34:25.302 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15859980571981946755', time=2026-02-25T00:26:45.145655+0000, chain.objs.size()=1 2026-02-24T22:34:25.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Z-ssZIVb5piWoFRUiVr3MwY91aBmgPC_0 2026-02-24T22:34:25.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8966012951863571302', time=2026-02-25T00:26:45.156120+0000, chain.objs.size()=1 2026-02-24T22:34:25.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Fq017zAtnfmh6r-Z6ON8fe20vNBjHKk_0 2026-02-24T22:34:25.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17000048292620981029', time=2026-02-25T00:26:45.312378+0000, chain.objs.size()=1 2026-02-24T22:34:25.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.7LH1s9e2gvxjaQgkk2GCcawF3SWUxgX_0 2026-02-24T22:34:25.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5075629321208925559', time=2026-02-25T00:26:45.319322+0000, chain.objs.size()=1 2026-02-24T22:34:25.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.KKo5kloG8bG5UWrbMDCpd5ZY5tn9sgt_0 2026-02-24T22:34:25.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4420332775801114239', time=2026-02-25T00:26:45.417332+0000, chain.objs.size()=1 2026-02-24T22:34:25.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.QT0OmOFPX_UEcSKxC3jM6S3b_aINFv__0 2026-02-24T22:34:25.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7070622332400758017', time=2026-02-25T00:26:45.429734+0000, chain.objs.size()=1 2026-02-24T22:34:25.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Ktu9FZNPDVt_gh1aZQtaCTqMGU2jHHL_0 2026-02-24T22:34:25.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15980010166263699977', time=2026-02-25T00:26:45.461122+0000, chain.objs.size()=1 2026-02-24T22:34:25.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.spAJ1xfwShCXR594w2jVSJllHff87hy_0 2026-02-24T22:34:25.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3292652963065977607', time=2026-02-25T00:26:45.481341+0000, chain.objs.size()=1 2026-02-24T22:34:25.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_._2ZCMsjsKp7wknbskcqaTg5A73a8cWS_0 2026-02-24T22:34:25.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16974999331935550963', time=2026-02-25T00:26:45.497703+0000, chain.objs.size()=1 2026-02-24T22:34:25.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.tEFOlCllIOV7bXsxRnBpLBrPwDL1LBp_0 2026-02-24T22:34:25.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7645395120625212513', time=2026-02-25T00:26:45.579005+0000, chain.objs.size()=1 2026-02-24T22:34:25.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.iptsULSXpwrvT38Z2278uwBITZ93mem_0 2026-02-24T22:34:25.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.818133816717145537', time=2026-02-25T00:26:45.582589+0000, chain.objs.size()=1 2026-02-24T22:34:25.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.STqMPDnYi0DoKhlartmmv7-lNoKNxvx_0 2026-02-24T22:34:25.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9742773297583238665', time=2026-02-25T00:26:45.589159+0000, chain.objs.size()=1 2026-02-24T22:34:25.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.uDO3MPyfsZZi-DEZkr-oOpVeZWPhA8R_0 2026-02-24T22:34:25.303 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.997117815366009000', time=2026-02-25T00:26:45.830361+0000, chain.objs.size()=1 2026-02-24T22:34:25.304 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.gySHgDhZhPcQKhp15CGa84W9EUgqn_F_0 2026-02-24T22:34:25.304 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11539699341924614350', time=2026-02-25T00:26:45.835049+0000, chain.objs.size()=1 2026-02-24T22:34:25.304 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.nWz7FQ325TrIShiPICIvkmgEbCwQVpp_0 2026-02-24T22:34:25.304 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13621136090438934409', time=2026-02-25T00:26:45.843290+0000, chain.objs.size()=1 2026-02-24T22:34:25.304 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.IoAijw0uneV74jdBvtaBSFzV5jGoH9u_0 2026-02-24T22:34:25.304 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10183681146841631737', time=2026-02-25T00:26:45.850058+0000, chain.objs.size()=1 2026-02-24T22:34:25.304 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.yocevJSYvy14Lz-hu3Q4HxyAk_t4C2-_0 2026-02-24T22:34:25.304 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8059886775117166344', time=2026-02-25T00:26:45.925628+0000, chain.objs.size()=1 2026-02-24T22:34:25.304 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.-9oOIhUE3VCaijvqEPXUh2o4F4QVDgb_0 2026-02-24T22:34:25.304 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15220414164595771311', time=2026-02-25T00:26:45.933089+0000, chain.objs.size()=1 2026-02-24T22:34:25.304 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.V56IAi6dXTDgRKqRHRKd0KR4ls6CUgy_0 2026-02-24T22:34:25.304 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8330582132130020852', time=2026-02-25T00:26:45.972568+0000, chain.objs.size()=1 2026-02-24T22:34:25.304 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.CszsOq3kUbtfZ0c3cmxOtRpFAY-wP4g_0 2026-02-24T22:34:25.304 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3183601936565764493', time=2026-02-25T00:26:45.975823+0000, chain.objs.size()=1 2026-02-24T22:34:25.304 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.IpvKM5i9rjam3AsUyIqqPFwP_C3QAiy_0 2026-02-24T22:34:25.304 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4372220686754269003', time=2026-02-25T00:26:45.977438+0000, chain.objs.size()=1 2026-02-24T22:34:25.304 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.295+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.VlKE2Q393U4Xw1SNVCpWH3TAI-_NQgc_0 2026-02-24T22:34:25.304 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/39199 2026-02-24T22:34:25.304 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=60, truncated=0, next_marker='' 2026-02-24T22:34:25.304 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13599183546231856319', time=2026-02-25T00:26:45.998658+0000, chain.objs.size()=1 2026-02-24T22:34:25.304 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.XEfU3_UKJ4qw9uimLJMbPyn6z2CiQyp_0 2026-02-24T22:34:25.304 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2500347806466301429', time=2026-02-25T00:26:46.005818+0000, chain.objs.size()=1 2026-02-24T22:34:25.304 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.TPaAP-XixSAQOIzbILEr4No3rta3Ikl_0 2026-02-24T22:34:25.304 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.549249953646530479', time=2026-02-25T00:26:46.015432+0000, chain.objs.size()=1 2026-02-24T22:34:25.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.CwffdEq7VNgZRfnz-iPrc25nN_dZzwL_0 2026-02-24T22:34:25.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11180943033718048407', time=2026-02-25T00:26:46.092329+0000, chain.objs.size()=1 2026-02-24T22:34:25.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.J4UcIMUYHLjMnJDP0Pgbi1PluB71CaX_0 2026-02-24T22:34:25.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17106008606404045335', time=2026-02-25T00:26:46.094358+0000, chain.objs.size()=1 2026-02-24T22:34:25.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.GnNOAhdcMv_sCx7MJDqlqlneISykPkr_0 2026-02-24T22:34:25.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13680784734457540002', time=2026-02-25T00:26:46.103589+0000, chain.objs.size()=1 2026-02-24T22:34:25.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.HOSJszpqfirc7dSSImEoX_OluOUYJbM_0 2026-02-24T22:34:25.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4810689372785503433', time=2026-02-25T00:26:46.115381+0000, chain.objs.size()=1 2026-02-24T22:34:25.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Ovne7RX9fMnK2trab8gw1nx05LP7thZ_0 2026-02-24T22:34:25.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12450975744452725442', time=2026-02-25T00:26:46.150564+0000, chain.objs.size()=1 2026-02-24T22:34:25.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.d0lvffM1YwVdwjBmCu1W8WLeDqNM23n_0 2026-02-24T22:34:25.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1090361919965906030', time=2026-02-25T00:26:53.170687+0000, chain.objs.size()=1 2026-02-24T22:34:25.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.NgWAPfek6BTLoHMT_LwLZ306_t5qDWn_0 2026-02-24T22:34:25.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8047654664076149752', time=2026-02-25T00:26:53.360508+0000, chain.objs.size()=1 2026-02-24T22:34:25.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.EVdmBj7AUaR9cQ1IqTUuYgxg0UhskfI_0 2026-02-24T22:34:25.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8533708726013072595', time=2026-02-25T00:26:53.436614+0000, chain.objs.size()=1 2026-02-24T22:34:25.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.tzZoNjsR0FV4dKGiv5ByN3i1cFL8-rS_0 2026-02-24T22:34:25.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12921265417883304319', time=2026-02-25T00:26:53.523990+0000, chain.objs.size()=1 2026-02-24T22:34:25.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.iqzSgNL1zvpHkqounovbvL7PiqvXnl9_0 2026-02-24T22:34:25.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1902675297484918245', time=2026-02-25T00:26:53.553182+0000, chain.objs.size()=1 2026-02-24T22:34:25.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.B1UVt7PTuZ3NU1jP-Y7EdeF8wZn01d1_0 2026-02-24T22:34:25.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2818843983072824282', time=2026-02-25T00:26:53.601427+0000, chain.objs.size()=1 2026-02-24T22:34:25.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.eZJRU0pd7DR5VyqdVvyNzM8iQ1HrQ-9_0 2026-02-24T22:34:25.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2734422886232115997', time=2026-02-25T00:26:53.743988+0000, chain.objs.size()=1 2026-02-24T22:34:25.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.mJnhIOfzxOYsV48A4HDyi4ThRow5GiZ_0 2026-02-24T22:34:25.305 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17409367529061547424', time=2026-02-25T00:26:53.748437+0000, chain.objs.size()=1 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.WLyOOCyKpk7_YEM5rjLh3tCRTepPiAR_0 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2563584045414403223', time=2026-02-25T00:26:53.759616+0000, chain.objs.size()=1 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.SFirWt18CzoPrGR8-4UmxYyGauaAetr_0 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17932197804695903896', time=2026-02-25T00:26:53.779010+0000, chain.objs.size()=1 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.FwLGYO7XIgAgHnQYrFZDuXbDfRal4Dj_0 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10071812116559312638', time=2026-02-25T00:26:53.789587+0000, chain.objs.size()=1 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.4m0-Mlwht7M_L9lvC50z1xUUAfGoREt_0 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17350537233040600153', time=2026-02-25T00:26:53.816806+0000, chain.objs.size()=1 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.8b2lOnco1z58QgsOKjQX-kW7HY1k5jt_0 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1730076251478301826', time=2026-02-25T00:26:53.821619+0000, chain.objs.size()=1 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ykJb9YIDIOz3NS21W1F6T8W9Nw00P8L_0 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.861890673912533392', time=2026-02-25T00:26:53.832636+0000, chain.objs.size()=1 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.F9bN_1FuB0olYX22Y1_tCoLJthwQ9xk_0 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3379968523294937295', time=2026-02-25T00:26:53.930985+0000, chain.objs.size()=1 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.iHmA_fIBF7eSE9rZzCQEiy6Jck74GRB_0 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3893733014096376113', time=2026-02-25T00:26:53.946464+0000, chain.objs.size()=1 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.kIxJJsV0Q3SlYGvXbCHmdGLzoYvQf9c_0 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18434045407457836986', time=2026-02-25T00:26:53.962683+0000, chain.objs.size()=1 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.spxCaAmTC8YUbWj89o02kDmxBBd0eiz_0 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7074220312642444926', time=2026-02-25T00:26:53.985665+0000, chain.objs.size()=1 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.aNahvMJHXsX6Ro4GxeRHM_BzgPxNTOg_0 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3084527552920658495', time=2026-02-25T00:26:54.007909+0000, chain.objs.size()=1 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.299+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Z_5HyDsT3pt5zmzsPqdPk5MvZnRp5mt_0 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5418774548819003559', time=2026-02-25T00:26:54.018437+0000, chain.objs.size()=1 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.bkz1DDFpzmT3qdpYAOHPujAwBs6CD6N_0 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16585879839094720957', time=2026-02-25T00:26:54.125330+0000, chain.objs.size()=1 2026-02-24T22:34:25.306 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.JyKeIJRaUV7pvgkSHMEcUWBa-5jcvZU_0 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9729544821494478704', time=2026-02-25T00:26:54.171337+0000, chain.objs.size()=1 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.gwIC8zk7XVH_q64kd-je8RlAlaUeL61_0 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7795984318522646184', time=2026-02-25T00:26:54.180939+0000, chain.objs.size()=1 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.M1j_keXAAvflk4NreMyNR3ymfB1c057_0 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4958881940849750579', time=2026-02-25T00:26:54.211125+0000, chain.objs.size()=1 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.bxfPl0SW58t16DuzH_ku8ikbGJl4jzS_0 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3883454862177650991', time=2026-02-25T00:26:54.223148+0000, chain.objs.size()=1 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.mOAe9EEZXatknp1zsJ5cPmzKHbItrup_0 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15169963801893032566', time=2026-02-25T00:26:54.298956+0000, chain.objs.size()=1 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.j-mbvA83wifF23vXbGGGUQBNI5Elexw_0 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2989394637919712656', time=2026-02-25T00:26:54.331370+0000, chain.objs.size()=1 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.2H16SazmZ7829LNq96HIdnylEYjdXsZ_0 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1078988873531153446', time=2026-02-25T00:26:54.484150+0000, chain.objs.size()=1 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.46RMo_l48P1CXzIxHwX1ooD-CZw_8Q__0 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1818089302674409096', time=2026-02-25T00:26:54.492514+0000, chain.objs.size()=1 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.bkN2PM7fWDMEn-40Pn0Bn-pdfQ2EEhL_0 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11782658480740012806', time=2026-02-25T00:26:54.511428+0000, chain.objs.size()=1 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.WiHdT16BjHYj3jHy3D_-6T8BkJSSP_E_0 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14947505651598449592', time=2026-02-25T00:26:54.518907+0000, chain.objs.size()=1 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.zUfTpNWf4w3JLqDK_ttX0n7X84h2wZU_0 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17009893087538275757', time=2026-02-25T00:26:54.546019+0000, chain.objs.size()=1 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.OalXpW12FMCU5-XCAQtlf1CjjL3GcMh_0 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11387887499004188347', time=2026-02-25T00:28:11.048849+0000, chain.objs.size()=1 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1812__shadow_.Mz5vwHR3rp8yQxDw6SA7CG9kACJvVPE_0 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.301258211944237605', time=2026-02-25T00:28:18.741838+0000, chain.objs.size()=1 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1823__shadow_.GZDc72ICdAWdXtulUFwpcqAVrYRqkk4_0 2026-02-24T22:34:25.307 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4059941210424501550', time=2026-02-25T00:28:25.145006+0000, chain.objs.size()=1 2026-02-24T22:34:25.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1831__shadow_._mfXQoaAAXrPLAIYkuPmunD-yURaWuU_0 2026-02-24T22:34:25.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7093441906293280292', time=2026-02-25T00:28:33.181471+0000, chain.objs.size()=1 2026-02-24T22:34:25.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1850__shadow_.s4NdN9p9wdZf_hkzpGFlj0iGwxnFVLp_0 2026-02-24T22:34:25.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9169775489350804453', time=2026-02-25T00:28:39.745455+0000, chain.objs.size()=1 2026-02-24T22:34:25.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1862__shadow_.VP2z0ytCbKy_u_pXkaJCy3qveya8nE2_0 2026-02-24T22:34:25.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2822109933974455149', time=2026-02-25T00:28:46.341682+0000, chain.objs.size()=6 2026-02-24T22:34:25.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.1 2026-02-24T22:34:25.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.2 2026-02-24T22:34:25.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.3 2026-02-24T22:34:25.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.4 2026-02-24T22:34:25.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.5 2026-02-24T22:34:25.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.6 2026-02-24T22:34:25.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2307914100526994045', time=2026-02-25T00:28:46.536388+0000, chain.objs.size()=6 2026-02-24T22:34:25.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.1 2026-02-24T22:34:25.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.2 2026-02-24T22:34:25.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.3 2026-02-24T22:34:25.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.4 2026-02-24T22:34:25.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.5 2026-02-24T22:34:25.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.6 2026-02-24T22:34:25.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='2~sJYAJxfnctZsGBcD-kKtQjlHjtrN__F', time=2026-02-25T00:29:03.393974+0000, chain.objs.size()=1 2026-02-24T22:34:25.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~sJYAJxfnctZsGBcD-kKtQjlHjtrN__F.3 2026-02-24T22:34:25.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='2~pj_3WTBboDa_0XCojflIVBc95wtcFhk', time=2026-02-25T00:29:05.431016+0000, chain.objs.size()=1 2026-02-24T22:34:25.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~pj_3WTBboDa_0XCojflIVBc95wtcFhk.2 2026-02-24T22:34:25.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='2~pj_3WTBboDa_0XCojflIVBc95wtcFhk', time=2026-02-25T00:29:05.432324+0000, chain.objs.size()=1 2026-02-24T22:34:25.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~pj_3WTBboDa_0XCojflIVBc95wtcFhk.3 2026-02-24T22:34:25.308 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11552640802430313602', time=2026-02-25T00:30:02.799989+0000, chain.objs.size()=1 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1919__shadow_.SxGANEpZ6V2yLOF32YcL7w0BEE9VP0T_0 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8294484090485676423', time=2026-02-25T00:30:07.647852+0000, chain.objs.size()=1 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.303+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1925__shadow_.2C6VKszzS6-Hw2ZyE2U84tOxlBpU52L_0 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.307+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13067105079223588020', time=2026-02-25T00:30:10.833579+0000, chain.objs.size()=1 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.307+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1929__multipart_mymultipart.rP-Wyf-QUpkr9RAAsLMWNpCVZC0rtHCe.1 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.307+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18379316225264759506', time=2026-02-25T00:30:18.186138+0000, chain.objs.size()=1 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.307+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:fwzd3sZhIvGF06bOciGHQl4583EzBBA_.dtuIIWFfGNxVAhCpWVu5MDy77wWyumn_0 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.307+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12206271798026671351', time=2026-02-25T00:30:20.134759+0000, chain.objs.size()=1 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.307+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:PrM3GvopJRBFartU.uOMhLj.90PktVh_.Yq78C9gPYNbDoSpWR65vyXB3QX25lwI_0 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.307+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5086394691587976380', time=2026-02-25T00:30:22.306295+0000, chain.objs.size()=1 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.307+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:JZkoSH1BNOis2QVE6J15vID-Bs5psuO_.GoHvq-h5y6MBBJZOIs1Gdh5QAv77iwA_0 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.307+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17154969447759996079', time=2026-02-25T00:30:34.130339+0000, chain.objs.size()=1 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.307+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1946__shadow:7DfKal.pvZnUNsO.OyOSZwcAzAOJ4mR_.QiSpKXBvaZRNhotIgRanFQfAA1wr6fL_0 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.307+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15427755724190709093', time=2026-02-25T00:30:39.406854+0000, chain.objs.size()=1 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.307+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:wcEUyAVyaXBBwtLTpfoBtEdK1PW2uni_.Uho34RpabWuezal_CtIAkdV5t5QbOfe_0 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.307+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2106713971317917207', time=2026-02-25T00:30:51.018110+0000, chain.objs.size()=1 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.307+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1949__shadow:-v13-8Z018YJfjM-PdMOLx6287eeszj_.G05eN9GH-LqUCNvVCaVIMzdhfGRQ7EM_0 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.307+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14275300233087806169', time=2026-02-25T00:33:52.740772+0000, chain.objs.size()=1 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.307+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1974__shadow_.gkzwql9fYGYG-XG5Tr7rOFdccpHvYA7_0 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.307+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.307+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/38436' 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4740371126170548554', time=2026-02-25T00:15:34.797873+0000, chain.objs.size()=2 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1514__shadow_.N46S_0BhEFOUya4Zwxt-QeVsF39DQtA_0 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1514__shadow_.N46S_0BhEFOUya4Zwxt-QeVsF39DQtA_1 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16210188404592268179', time=2026-02-25T00:17:08.009159+0000, chain.objs.size()=1 2026-02-24T22:34:25.309 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524__shadow_.lGf90yk87QtzXyDM_LOuanaCS0oS9mO_0 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.95034722690232867', time=2026-02-25T00:17:41.421012+0000, chain.objs.size()=1 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526__shadow_.yrbSXLKOu-5JLBU26wyoOMnzpS5f_St_0 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15595319648292399725', time=2026-02-25T00:17:41.423912+0000, chain.objs.size()=1 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526__shadow_.4zrrQ5kBcvPXO1-D8kS2loL1sHTU1VG_0 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4637881138759743478', time=2026-02-25T00:17:42.533934+0000, chain.objs.size()=1 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527__shadow_.C6ShxJQSp88q_-2fudV1NwWbOyI7gl1_0 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7886096879979281517', time=2026-02-25T00:18:16.124081+0000, chain.objs.size()=1 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529__shadow_.w6FoxNxa1FFM559Ksd2yhvrJW6MXuhK_0 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1886841342848882991', time=2026-02-25T00:18:48.182485+0000, chain.objs.size()=1 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530__shadow_.J9R6jrnhDAMSwrKxFUgEG14-Yq0GZS-_0 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7498209996563528957', time=2026-02-25T00:18:48.188329+0000, chain.objs.size()=1 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530__shadow_.iWZMZRuPN6f8gozMcEqFJmhOYmws6Zc_0 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17775272241353955500', time=2026-02-25T00:18:49.771057+0000, chain.objs.size()=1 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531__shadow_.8-W7IMTdG6DjAvVK8pp83Ls0m8waeDZ_0 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11847610484502643863', time=2026-02-25T00:18:51.087461+0000, chain.objs.size()=1 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532__shadow_.ZqMSaO0kHjCawG_cjrc4R8Lt-8utrwd_0 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1914738262800254745', time=2026-02-25T00:20:32.509204+0000, chain.objs.size()=1 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538__shadow_.bq4AJFr7qiIuxGBV0Mzhsx9RbXdh-fC_0 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5415652388105221019', time=2026-02-25T00:20:34.030918+0000, chain.objs.size()=1 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539__shadow_.P4-m77L4m1vsnxZMX_IWqjNh92cGkjO_0 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14979397412022739914', time=2026-02-25T00:21:06.309009+0000, chain.objs.size()=1 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540__shadow_.Txq1i6O-kiUS4CauO5Hj1WRiUZVDJZm_0 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.369349099006712325', time=2026-02-25T00:21:06.311314+0000, chain.objs.size()=1 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540__shadow_.bJyAKbIjXrrU1ujMh0PkVFcaClneler_0 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2021409611835840510', time=2026-02-25T00:23:14.349667+0000, chain.objs.size()=1 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518__shadow_.xe6ijdWw9-2EQqqTelRAIwLTpBm77_Y_0 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4337558562697905242', time=2026-02-25T00:25:34.874904+0000, chain.objs.size()=1 2026-02-24T22:34:25.310 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1599__shadow_.D7tOaM3xosXox8dvvznM_nD7we8I0AK_0 2026-02-24T22:34:25.311 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13564615398942833097', time=2026-02-25T00:25:45.006208+0000, chain.objs.size()=1 2026-02-24T22:34:25.311 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1614__shadow_.MN4BGKbteidIuFnHntzpC8J0FLr6Tqz_0 2026-02-24T22:34:25.311 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6207036934883150662', time=2026-02-25T00:25:46.124268+0000, chain.objs.size()=1 2026-02-24T22:34:25.311 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1616__shadow_.i2LDQpkIOCPA4vuALDOmE806eDH6H2c_0 2026-02-24T22:34:25.311 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3817681304199151671', time=2026-02-25T00:25:59.072650+0000, chain.objs.size()=1 2026-02-24T22:34:25.311 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.MEUmPJASiKlJfUK7AOmTTsOkZ-mj-Es_0 2026-02-24T22:34:25.311 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13689288406751850388', time=2026-02-25T00:25:59.077463+0000, chain.objs.size()=1 2026-02-24T22:34:25.311 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.y3NJzBdedbOoL0CLEq0r9QqUPUK46Ho_0 2026-02-24T22:34:25.311 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4589049113070965485', time=2026-02-25T00:25:59.115525+0000, chain.objs.size()=1 2026-02-24T22:34:25.311 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.lFrHHvldy3tZ9eMyy1r8DjpLMthU5ky_0 2026-02-24T22:34:25.311 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3740020697699941058', time=2026-02-25T00:25:59.126997+0000, chain.objs.size()=1 2026-02-24T22:34:25.311 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.311+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.9irVuNxaQWz5F1FnEQ_YPm6pxNtHsZV_0 2026-02-24T22:34:25.313 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10132403743059384120', time=2026-02-25T00:25:59.155057+0000, chain.objs.size()=1 2026-02-24T22:34:25.313 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.-44Y8rR15vCPCmkq_1YvMkI1Zgeifqt_0 2026-02-24T22:34:25.313 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4887926646815381450', time=2026-02-25T00:25:59.169496+0000, chain.objs.size()=1 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.fYg7Woxj_Z4x-pgZlNW2exEN6Pl4T3c_0 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8475931738161897247', time=2026-02-25T00:25:59.328370+0000, chain.objs.size()=1 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.SLrX2KuKWd0BO5X_WPMT3hnTpsgIY6n_0 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6327043634662557645', time=2026-02-25T00:25:59.407012+0000, chain.objs.size()=1 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.c8RjHT8BnnmCbgzJ1DiTCqfLCYP5VS9_0 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13552787035167000935', time=2026-02-25T00:25:59.424861+0000, chain.objs.size()=1 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.vTtx0G4ek8rBzFDtFSoPktCeG0jD0ME_0 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8423694803259676117', time=2026-02-25T00:25:59.431860+0000, chain.objs.size()=1 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.b2xytxF_2c4sX3mec49tJ4P6rctVQGE_0 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1578701031907495230', time=2026-02-25T00:25:59.442360+0000, chain.objs.size()=1 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.0Jz9_Jc5d61o0sIsvEiA-4jKYDzU2bF_0 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6069768729541475839', time=2026-02-25T00:25:59.463058+0000, chain.objs.size()=1 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.xKKWn7NzHcYPQvj0m0UrvLdwK3R_EWE_0 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.106210592075892350', time=2026-02-25T00:25:59.475478+0000, chain.objs.size()=1 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.nzu1tcLPEB-O6UBKo3s1BzmgUQXFSBV_0 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13758201006945193281', time=2026-02-25T00:25:59.485091+0000, chain.objs.size()=1 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.2THcAvdzXfkaH2FaIm1c6qgVUT1wp-Q_0 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5720233315830563640', time=2026-02-25T00:25:59.586463+0000, chain.objs.size()=1 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.8wd2lxL1T5ukuVCaMdA6GX_fnxjDGoj_0 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.941145159543045541', time=2026-02-25T00:25:59.610742+0000, chain.objs.size()=1 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.9UG29DuvXL4qI74FQVhez5k4rcp-1KZ_0 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15765788541354541030', time=2026-02-25T00:25:59.623265+0000, chain.objs.size()=1 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.tFJ1tGptwrNVuTLzDIbuoIDB9TUlpJw_0 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6110272432790432254', time=2026-02-25T00:25:59.734684+0000, chain.objs.size()=1 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.fkNc4B9oRkiEfshTX72DxxYIYUasKCj_0 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1399189007994714968', time=2026-02-25T00:25:59.771244+0000, chain.objs.size()=1 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.NKwcqI2k7R4ppYKQyu_KgVRbbjGYT2H_0 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4486120771425159078', time=2026-02-25T00:25:59.781065+0000, chain.objs.size()=1 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Kfc3c1bR1p3YNeUvJEyJ4th-H-TAGxS_0 2026-02-24T22:34:25.314 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6537410401768326376', time=2026-02-25T00:25:59.803342+0000, chain.objs.size()=1 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.8hGTnrTakxRboLEGW_loXM0b-LVzFVK_0 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14518003370521669336', time=2026-02-25T00:25:59.895585+0000, chain.objs.size()=1 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.dppTCI_WzIl-R2aqPgW-QagPY7NyXSy_0 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11041490236295431726', time=2026-02-25T00:25:59.916497+0000, chain.objs.size()=1 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.jVYmOZ9UFLDJynQjvr3u2YhvMaviUI5_0 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7503664241836899460', time=2026-02-25T00:25:59.930682+0000, chain.objs.size()=1 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.cRCWtODdwLjLEhZIG7MnGWVpVGi1Xol_0 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9990575155147976826', time=2026-02-25T00:26:00.127578+0000, chain.objs.size()=1 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.kRDpqkSss-ijUcVjyQNeFaLZg8Xdn1W_0 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3082166582332477229', time=2026-02-25T00:26:00.206769+0000, chain.objs.size()=1 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.9-RA2-oX1fso7Shj0jjZad5MMRR1Wyj_0 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.375724806659509204', time=2026-02-25T00:26:00.264931+0000, chain.objs.size()=1 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.P5Hi3WyPIr3HaGGneI6kL1Q8M1j29P3_0 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13726446360129842909', time=2026-02-25T00:26:00.266568+0000, chain.objs.size()=1 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.F4HBJ2LZ7BzCSsrhO2jFN2C1J32nX44_0 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4496629317930001111', time=2026-02-25T00:26:00.277760+0000, chain.objs.size()=1 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.PsK6N88g1Rb0_XCz8v6DJ7VatcOPH9m_0 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14182749038836624084', time=2026-02-25T00:26:09.397953+0000, chain.objs.size()=1 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1644__shadow_.quQ5S1iLRUS3ybNAZ2qqfq0M9nL-jUT_0 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.781591083563979780', time=2026-02-25T00:26:14.802787+0000, chain.objs.size()=1 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1653__shadow_.xojUV_noThLnlGD_rI-EGkm5hgae6VK_0 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4068018427783820678', time=2026-02-25T00:26:16.010169+0000, chain.objs.size()=1 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1655__shadow_.tvsZ7GrveP4WSMifkdv37bmCENMtHB__0 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15652455051055703634', time=2026-02-25T00:26:19.445545+0000, chain.objs.size()=1 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1659__shadow_.LTcWQVUl7WDCUkNAZb3Jcl_-2XSCIjF_0 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17550694843122197746', time=2026-02-25T00:26:19.446245+0000, chain.objs.size()=1 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1659__shadow_.ga7HOIGmtu7-MXVc_YL2oWZIyw4DpWe_0 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.979426889170657921', time=2026-02-25T00:26:19.451538+0000, chain.objs.size()=1 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1659__shadow_.5xA-9CCsg86p0sFp-JfHurBqdOxyCqm_0 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5476891780199484480', time=2026-02-25T00:26:44.776289+0000, chain.objs.size()=1 2026-02-24T22:34:25.315 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.55H050qrZS1prLBPUnWM2lVeO8SBats_0 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11137296927416039474', time=2026-02-25T00:26:44.781223+0000, chain.objs.size()=1 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.zjKxPrAKCHuzVxV3Ph3BtlauBQCtCSw_0 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9229129053922165129', time=2026-02-25T00:26:44.815886+0000, chain.objs.size()=1 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.fIoBwxC3Hcbzopt3AS3TDfAqMboRAV4_0 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18087550334693431221', time=2026-02-25T00:26:44.959476+0000, chain.objs.size()=1 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.315+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.RmmPgEDWwyt7KqDJ2G1DS8jkK_JUbzU_0 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.667593373522496325', time=2026-02-25T00:26:45.010276+0000, chain.objs.size()=1 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.R5K9LDqQIsxc8R9iuy9jl55x-5SsyqT_0 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8105855688776806280', time=2026-02-25T00:26:45.097279+0000, chain.objs.size()=1 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.2pH9CcLIBh84OiHj30xBeuy2KMWK021_0 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13188799735834158369', time=2026-02-25T00:26:45.138396+0000, chain.objs.size()=1 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.vKILBwoZLxEGZa3Pw8wBBDnVROCMUT2_0 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3776806517166900956', time=2026-02-25T00:26:45.159886+0000, chain.objs.size()=1 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.T1Y1sZNUN_Sjg26s-A2vyGPWVF0Tkzm_0 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9902759152150918671', time=2026-02-25T00:26:45.171500+0000, chain.objs.size()=1 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.IufAvSARszCVqYJ_sBmh0WsMdWwoRNo_0 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2871897569447345864', time=2026-02-25T00:26:45.260895+0000, chain.objs.size()=1 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.1zAVUOHHzOQ0Z0FrQLeMSfSKrAnNsq__0 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2791196463086578489', time=2026-02-25T00:26:45.288887+0000, chain.objs.size()=1 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.j7U1PBtbhLHPSGGYS75idLmPUNiPXtR_0 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7009878718703609107', time=2026-02-25T00:26:45.296860+0000, chain.objs.size()=1 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.rTHjrVJ9Yd5LsRlmd4TYuS2SD3moJzM_0 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5396662780338321431', time=2026-02-25T00:26:45.298555+0000, chain.objs.size()=1 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.aypw8bCWHrRnti8ZeU1N9dSiB7Yuo2T_0 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7752282433043871029', time=2026-02-25T00:26:45.306764+0000, chain.objs.size()=1 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.X8UpufJVexAGQ42HLd_0s_RcbXtre1N_0 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18129456768058367642', time=2026-02-25T00:26:45.316378+0000, chain.objs.size()=1 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.4N-UQOWL3W1-D-RCtCPJhKWafQ5yzKF_0 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12350060847691775411', time=2026-02-25T00:26:45.407578+0000, chain.objs.size()=1 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.c-8U5K2ulZDDmV39Bkib-sd5sIqWMxM_0 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2182718137702612682', time=2026-02-25T00:26:45.408506+0000, chain.objs.size()=1 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.6Tt7GcZSMtIzheYvxkTbxbLiAj5MtjE_0 2026-02-24T22:34:25.316 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.345046306681214798', time=2026-02-25T00:26:45.438576+0000, chain.objs.size()=1 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.iPuWDc4NmeyMeIxngpcSk2x4q_zD0rx_0 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1797909810003961303', time=2026-02-25T00:26:45.445108+0000, chain.objs.size()=1 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.mBMfM_tWhqGrwV6G7vQ3S-yj6U2xsej_0 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14829575116602112497', time=2026-02-25T00:26:45.453354+0000, chain.objs.size()=1 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.uOBqDZEbwkGXkko2AXRd2CQq7UVr2F8_0 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5355089265378645400', time=2026-02-25T00:26:45.463685+0000, chain.objs.size()=1 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.FU5bjmiTvb3xa4mfxsp1DXXGHqKuc-H_0 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10259282070623749237', time=2026-02-25T00:26:45.837881+0000, chain.objs.size()=1 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.mwwVP4TYG6TLBjNGpMJ2Z6v7NoOB496_0 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16836657157322279516', time=2026-02-25T00:26:45.842602+0000, chain.objs.size()=1 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.TiNpukqVuPgV7EL0zTDz-y3DP_1lqrr_0 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10835473149752466177', time=2026-02-25T00:26:45.908887+0000, chain.objs.size()=1 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.bFb0Hx-5DlBe4q9oAwVyqz4YKggZcQC_0 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14353684096264633070', time=2026-02-25T00:26:45.931426+0000, chain.objs.size()=1 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.6pKuEFJQy-zvbNr8am_2SBENbhPCRI6_0 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9080646532494220763', time=2026-02-25T00:26:45.942121+0000, chain.objs.size()=1 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.siHS0qb69unBqv6neET7iXaHArOe7yn_0 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15934216978271144931', time=2026-02-25T00:26:45.946875+0000, chain.objs.size()=1 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.wGDkthTW1Il_kZTzT5vRpDhiP1xyzki_0 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14352134470675149699', time=2026-02-25T00:26:45.952263+0000, chain.objs.size()=1 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.UUUa8pTq6MVRPoLrbs3IvcNct2EtOb6_0 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1363902403691587913', time=2026-02-25T00:26:45.970310+0000, chain.objs.size()=1 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.vwiKxtTFJzBtoMNiXoJqawCOsAqJn51_0 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16143700877875611796', time=2026-02-25T00:26:45.979448+0000, chain.objs.size()=1 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.fSFgPaHHdI6UbzMqZLmhLAim5oNp8TH_0 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18196185433305890489', time=2026-02-25T00:26:46.118735+0000, chain.objs.size()=1 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.hfnxnvD0XrobckVuGEoFcpIXjiuVLVS_0 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4498219475443034404', time=2026-02-25T00:26:46.132057+0000, chain.objs.size()=1 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_._Xo5Jx1sKlk6D7KWv4veciwDaxu1ftx_0 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11898570184920357401', time=2026-02-25T00:26:46.133270+0000, chain.objs.size()=1 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.6yXzOV4jM8FPxiuhe_mTWBf1Vs4_U9E_0 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1066910167037496456', time=2026-02-25T00:26:46.145535+0000, chain.objs.size()=1 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.orgoHuDBZzcNf9yIlqxgiDGGIaZVMMS_0 2026-02-24T22:34:25.317 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9019265940097672099', time=2026-02-25T00:26:53.135263+0000, chain.objs.size()=1 2026-02-24T22:34:25.318 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.07YeWLL5E8sinLd28-qwCnNsazGHlgZ_0 2026-02-24T22:34:25.318 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16815572828220895667', time=2026-02-25T00:26:53.136296+0000, chain.objs.size()=1 2026-02-24T22:34:25.318 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.rUbRJWzXQb-m_lPqAR7m6o7m5lvIx7I_0 2026-02-24T22:34:25.318 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1139704198204834987', time=2026-02-25T00:26:53.227672+0000, chain.objs.size()=1 2026-02-24T22:34:25.318 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.PdFMxcc8li2KV6JrJm2szuaZyVsQf6q_0 2026-02-24T22:34:25.318 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10678663957312565398', time=2026-02-25T00:26:53.234865+0000, chain.objs.size()=1 2026-02-24T22:34:25.318 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ugweb56lcrMY2VT5R-6fXpt0gEXL9WP_0 2026-02-24T22:34:25.318 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.319+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14342252766667569260', time=2026-02-25T00:26:53.339027+0000, chain.objs.size()=1 2026-02-24T22:34:25.318 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ULlPV5R_qggNdL8WXgffORPK-l2tcrY_0 2026-02-24T22:34:25.318 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5136427520072605629', time=2026-02-25T00:26:53.365339+0000, chain.objs.size()=1 2026-02-24T22:34:25.318 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.qBW6vKHBI-Jfixwowx-EvjIsyQMI36M_0 2026-02-24T22:34:25.318 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6184465629222998811', time=2026-02-25T00:26:53.376992+0000, chain.objs.size()=1 2026-02-24T22:34:25.318 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.O3U34gf8cmSN-AUTO_fFlW7smRZXhEC_0 2026-02-24T22:34:25.318 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10655614081176445691', time=2026-02-25T00:26:53.427485+0000, chain.objs.size()=1 2026-02-24T22:34:25.318 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.xaAJ9I9DCgahhz1lxoeLupLveyROUzN_0 2026-02-24T22:34:25.318 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14351767776382781331', time=2026-02-25T00:26:53.507912+0000, chain.objs.size()=1 2026-02-24T22:34:25.318 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.jSoVqtdxmbsRcAYIU2j0k0icOWlMDCb_0 2026-02-24T22:34:25.318 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1999790428419116128', time=2026-02-25T00:26:53.530927+0000, chain.objs.size()=1 2026-02-24T22:34:25.318 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.LY0YVKcsM9DbdDoL0RMrhHM36ZOIiET_0 2026-02-24T22:34:25.318 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16310085147765895509', time=2026-02-25T00:26:53.535021+0000, chain.objs.size()=1 2026-02-24T22:34:25.318 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.PQGbpXD4wCn2Mzi4Ybd4GYCKvnPyRiZ_0 2026-02-24T22:34:25.318 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1466869138350493054', time=2026-02-25T00:26:53.551600+0000, chain.objs.size()=1 2026-02-24T22:34:25.318 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.lyXU5EGDsSv05xoblOFqHUFIT8xWBj2_0 2026-02-24T22:34:25.318 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6963481400178566589', time=2026-02-25T00:26:53.613884+0000, chain.objs.size()=1 2026-02-24T22:34:25.318 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.yCtMOpM_cvc_PC547-vdrYpHb-C0vzx_0 2026-02-24T22:34:25.318 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/38436 2026-02-24T22:34:25.319 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=36, truncated=0, next_marker='' 2026-02-24T22:34:25.319 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5662822687261669186', time=2026-02-25T00:26:53.746591+0000, chain.objs.size()=1 2026-02-24T22:34:25.319 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.LliXC_R7SVjrjOl1BSrr81zi6y_LBRC_0 2026-02-24T22:34:25.319 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9069311672912523425', time=2026-02-25T00:26:53.747452+0000, chain.objs.size()=1 2026-02-24T22:34:25.319 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.igKP-kzXIa4G7v1cj99P7TI9PqI9bI5_0 2026-02-24T22:34:25.319 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6364062341963660387', time=2026-02-25T00:26:53.793898+0000, chain.objs.size()=1 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ArUH4vP0qVQL3fqW6ueX3jN1JNLkY8h_0 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14749149480861970503', time=2026-02-25T00:26:53.922778+0000, chain.objs.size()=1 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Rpa6d2NvgydUsDP_83FUuqd82b0R-PG_0 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3228268605950621199', time=2026-02-25T00:26:53.989935+0000, chain.objs.size()=1 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ELlTbQ8qrCPWmR2enhraam9G0moWCSJ_0 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.936587622411396489', time=2026-02-25T00:26:54.101336+0000, chain.objs.size()=1 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.d19dixiXhQMu3p-Tsssm-z6zr5vAoqy_0 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1259076418975369508', time=2026-02-25T00:26:54.111381+0000, chain.objs.size()=1 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.1K-DKKFm6_WYIfc92TpyDKm2dwaR6mJ_0 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14845231232430617109', time=2026-02-25T00:26:54.119345+0000, chain.objs.size()=1 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.nWiSUn4qjjLZCkCWEPZKRmD-OgrRkq1_0 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13271737307149343429', time=2026-02-25T00:26:54.141190+0000, chain.objs.size()=1 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.GEO8jIMHQSq-TK1-jwif-7kyggOloOY_0 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.788269709040691833', time=2026-02-25T00:26:54.150449+0000, chain.objs.size()=1 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.4rMWv_k_OKSXKLDdZdZelv-9GnBAPoG_0 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10037182825637374794', time=2026-02-25T00:26:54.158072+0000, chain.objs.size()=1 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.bavVkL1emEBCoPV1Sis37PeBBG6U0KD_0 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4589734745540628407', time=2026-02-25T00:26:54.162678+0000, chain.objs.size()=1 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.323+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.8aG1bDjQwpqEpZ7nAnufVJLiT62KoKp_0 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6141546127551812393', time=2026-02-25T00:26:54.170966+0000, chain.objs.size()=1 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.1lcZcuGeERWbpGmsJW-9gydmAPmfXK2_0 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12635125226874030410', time=2026-02-25T00:26:54.173813+0000, chain.objs.size()=1 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.KN4A-i7C0jNRhuc-yDJkLGCy0Y00_g7_0 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8283483603167029350', time=2026-02-25T00:26:54.283054+0000, chain.objs.size()=1 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.BohL4HMEbSaSIkAko-Jhj2HDmVonMCB_0 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17803470751973962474', time=2026-02-25T00:26:54.326291+0000, chain.objs.size()=1 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.1vV5LJ1vm7jPpxy0FqXdv3wCVRuOlO4_0 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4254968459103549647', time=2026-02-25T00:26:54.340717+0000, chain.objs.size()=1 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.pLqicWXQm1QA_cKxA03PmRTy3JjxzVi_0 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2911555088797738559', time=2026-02-25T00:26:54.369224+0000, chain.objs.size()=1 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.paXuoudeM2KVs6xLAU1YZSTkqphVhQF_0 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2145323871227271588', time=2026-02-25T00:26:54.453528+0000, chain.objs.size()=1 2026-02-24T22:34:25.320 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.NEjftkijAjQ3cpz2osyEkSfYKoCPrqr_0 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2121818959072631646', time=2026-02-25T00:26:54.453941+0000, chain.objs.size()=1 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.1TALDYyR7ujfCDU47x6EpR0PsXBh9dY_0 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2913512773833332790', time=2026-02-25T00:26:54.458491+0000, chain.objs.size()=1 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.SSTEf5-CuuaRzcyxxHFXLF0uW-K5KmP_0 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7459896816791799198', time=2026-02-25T00:26:54.462114+0000, chain.objs.size()=1 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.FE1h5CQkJkdJFX1E5u8yEN_y4G8te7G_0 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12190047105683195941', time=2026-02-25T00:26:54.486321+0000, chain.objs.size()=1 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.DBZDidIATrHPkEZJZcBB7jEdMx0a37U_0 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4517678700756513291', time=2026-02-25T00:26:54.511756+0000, chain.objs.size()=1 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.e4TVajj_pAKADfKiV4mhujlTJRULDxM_0 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13418446338950548279', time=2026-02-25T00:26:54.523673+0000, chain.objs.size()=1 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.uUvv9MTfVJQDw0vpi8H8p1KzrKj5f1t_0 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12545119638857477172', time=2026-02-25T00:28:20.428845+0000, chain.objs.size()=1 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1825__shadow_.-d3DEZ_O0N2qxs0kgztCzV1p9dSaqLN_0 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10688245468674305520', time=2026-02-25T00:28:48.684250+0000, chain.objs.size()=1 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1878__shadow_.zNPc_DWHa2-wi6O27gh6_K71DOyvwBz_0 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1256360915389885213', time=2026-02-25T00:28:52.392919+0000, chain.objs.size()=1 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1883__shadow_.OU0Ii-PR9z2KmpIwWLe6szz594UJF3J_0 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11348412719057343234', time=2026-02-25T00:30:08.452494+0000, chain.objs.size()=1 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1926__shadow_.bU7dv_8j119NwUyZZSNxbvKES_I5wIV_0 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7844328034820885159', time=2026-02-25T00:30:14.038938+0000, chain.objs.size()=2 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1931__shadow_.gd-RXD_RY6l8_kwmzXurH3pZK0uqs4x_0 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1931__shadow_.gd-RXD_RY6l8_kwmzXurH3pZK0uqs4x_1 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7274681407488077285', time=2026-02-25T00:30:40.159139+0000, chain.objs.size()=1 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:kBfZ23xtLMSQlqD9meO8qxeazdEPpYz_.cwWzfOkNHd-7-uQ1PPWMlFmwt5DF8Ar_0 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13774209072327972769', time=2026-02-25T00:30:58.457418+0000, chain.objs.size()=1 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958__shadow:T0WnCUeBpQgsoYsnGH-bW4vktFE.U7A_.jvKSvMJ5j0jzSE5uodSvc_JKFwJXHqg_0 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13774209072327972769', time=2026-02-25T00:30:58.458074+0000, chain.objs.size()=1 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958__shadow:T0WnCUeBpQgsoYsnGH-bW4vktFE.U7A_.jvKSvMJ5j0jzSE5uodSvc_JKFwJXHqg_0 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13774209072327972769', time=2026-02-25T00:30:58.458521+0000, chain.objs.size()=1 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958__shadow:T0WnCUeBpQgsoYsnGH-bW4vktFE.U7A_.jvKSvMJ5j0jzSE5uodSvc_JKFwJXHqg_0 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13774209072327972769', time=2026-02-25T00:30:58.460071+0000, chain.objs.size()=1 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958__shadow:T0WnCUeBpQgsoYsnGH-bW4vktFE.U7A_.jvKSvMJ5j0jzSE5uodSvc_JKFwJXHqg_0 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11897352206415866244', time=2026-02-25T00:33:55.937951+0000, chain.objs.size()=1 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.327+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1978__shadow_.tqvR7aUr1XzEeRtI9sRCUWUGi-XfVUP_0 2026-02-24T22:34:25.321 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.322 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2026-02-24T22:34:25.322 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.322 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.322 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/37939' 2026-02-24T22:34:25.322 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.385483785311184793', time=2026-02-25T00:16:33.515478+0000, chain.objs.size()=1 2026-02-24T22:34:25.322 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521__shadow_.Hvx3PRKdxp_t10P3eB0RDO1qM59cplU_0 2026-02-24T22:34:25.322 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9097326640963796589', time=2026-02-25T00:17:39.881596+0000, chain.objs.size()=1 2026-02-24T22:34:25.322 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525__shadow_.mVbh3DSRg_2l2Oa6W7amhmfslSDpF5-_0 2026-02-24T22:34:25.322 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17016903808927574701', time=2026-02-25T00:17:39.882277+0000, chain.objs.size()=1 2026-02-24T22:34:25.322 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525__shadow_.kETN52X5XPHstm0KHZENK8pr4rWCfgd_0 2026-02-24T22:34:25.322 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6839369289422900168', time=2026-02-25T00:17:42.542308+0000, chain.objs.size()=1 2026-02-24T22:34:25.322 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527__shadow_.mEv7E0Jy12OSLwSpN0MX-MTe7MbYcZn_0 2026-02-24T22:34:25.323 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15621907941597641933', time=2026-02-25T00:18:14.580524+0000, chain.objs.size()=1 2026-02-24T22:34:25.323 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528__shadow_.qnvfHpQlH2ZrXqKJw2LL4oEzhSsUXUn_0 2026-02-24T22:34:25.323 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.279537415248561816', time=2026-02-25T00:18:14.582834+0000, chain.objs.size()=1 2026-02-24T22:34:25.323 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528__shadow_.mHcm49Nv3TUVpsf10F68Jbmkha-WJHg_0 2026-02-24T22:34:25.323 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7317240659515754788', time=2026-02-25T00:18:16.131790+0000, chain.objs.size()=1 2026-02-24T22:34:25.323 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529__shadow_.fWRj8P4_XbguSpGEwQII3fnem93H2mA_0 2026-02-24T22:34:25.323 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9453260920972424432', time=2026-02-25T00:18:48.919573+0000, chain.objs.size()=1 2026-02-24T22:34:25.323 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531__shadow_.62FKDf5eM1-IyokkrTFkl84Hrj-wjKR_0 2026-02-24T22:34:25.323 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1992947297965186110', time=2026-02-25T00:18:49.777661+0000, chain.objs.size()=1 2026-02-24T22:34:25.323 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531__shadow_.QeQxORpWYykaLwm0okFJyjnpovbVn-R_0 2026-02-24T22:34:25.323 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9234043329534099247', time=2026-02-25T00:18:49.783775+0000, chain.objs.size()=1 2026-02-24T22:34:25.323 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531__shadow_.OBTXZ6nh1ca9YdqiTXrChdbZMmbLcq6_0 2026-02-24T22:34:25.323 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6500605407269821641', time=2026-02-25T00:19:23.069694+0000, chain.objs.size()=1 2026-02-24T22:34:25.323 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533__shadow_.SEJDL8vh5U_cUczwnFv-ReJA-10-SSb_0 2026-02-24T22:34:25.323 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16368586852594995185', time=2026-02-25T00:19:23.070924+0000, chain.objs.size()=1 2026-02-24T22:34:25.323 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.331+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533__shadow_.is4nkh_ZCVGdUQ4TkYQCbYI-AOVAs5U_0 2026-02-24T22:34:25.323 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4868263896441470394', time=2026-02-25T00:19:23.072763+0000, chain.objs.size()=1 2026-02-24T22:34:25.323 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533__shadow_.jJjd3CkHHqbszeMQ9adjKTgmrg1YEyT_0 2026-02-24T22:34:25.323 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7820525148388372849', time=2026-02-25T00:19:57.120039+0000, chain.objs.size()=1 2026-02-24T22:34:25.323 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535__shadow_.HbKLg39vQ1QcSm7Sv-p1EPX7NB67OJu_0 2026-02-24T22:34:25.324 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16627333830621684367', time=2026-02-25T00:19:57.814030+0000, chain.objs.size()=1 2026-02-24T22:34:25.324 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536__shadow_.m7tAuvwifunO-NHSKFDVtYKzJA46bPI_0 2026-02-24T22:34:25.324 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9687730071366399018', time=2026-02-25T00:19:59.353154+0000, chain.objs.size()=1 2026-02-24T22:34:25.324 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537__shadow_.0nqw1YlCSylldxaJ30yeKac7iCF8Rhk_0 2026-02-24T22:34:25.324 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12632587257151012431', time=2026-02-25T00:25:33.718003+0000, chain.objs.size()=1 2026-02-24T22:34:25.324 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1597__shadow_.pKs05EjBlm7p93Vri3MK1xpCzBiuBtM_0 2026-02-24T22:34:25.324 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11898312424735967911', time=2026-02-25T00:25:39.604462+0000, chain.objs.size()=1 2026-02-24T22:34:25.324 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1606__shadow_.LJFnH1d02IJ0cVIs1frR17-RAx4manI_0 2026-02-24T22:34:25.324 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12256513750105447352', time=2026-02-25T00:25:49.714328+0000, chain.objs.size()=1 2026-02-24T22:34:25.324 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1622__shadow_.PHieJpr0I4L9wHhvcErnSECajXQkFkd_0 2026-02-24T22:34:25.324 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11756329674209524300', time=2026-02-25T00:25:59.091291+0000, chain.objs.size()=1 2026-02-24T22:34:25.324 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.QEkeszOJmMk6YV7aMdBRpPW22SkT3GZ_0 2026-02-24T22:34:25.324 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6341151920211957594', time=2026-02-25T00:25:59.175018+0000, chain.objs.size()=1 2026-02-24T22:34:25.324 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.pdokkjmGgdJfa6yOAAocEmWysIq7vK7_0 2026-02-24T22:34:25.324 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6518601083106974303', time=2026-02-25T00:25:59.260979+0000, chain.objs.size()=1 2026-02-24T22:34:25.324 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.7TnuXKxbOE6p47pHNPTbinyTKuv-_LN_0 2026-02-24T22:34:25.324 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8764936379093074625', time=2026-02-25T00:25:59.272450+0000, chain.objs.size()=1 2026-02-24T22:34:25.324 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.er-CLYzinrjuHnwHqrc1M71HGG7fby9_0 2026-02-24T22:34:25.324 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9803579414281088946', time=2026-02-25T00:25:59.276111+0000, chain.objs.size()=1 2026-02-24T22:34:25.324 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.a5x1PV18T9LYlU92RMUVMmVqH2n8D1O_0 2026-02-24T22:34:25.324 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1102237215216654860', time=2026-02-25T00:25:59.299550+0000, chain.objs.size()=1 2026-02-24T22:34:25.324 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.jYvUKLiEP_BWYeymaFc7v1tqWxk1Y3d_0 2026-02-24T22:34:25.325 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11143636125413634900', time=2026-02-25T00:25:59.320676+0000, chain.objs.size()=1 2026-02-24T22:34:25.325 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.DYUNEXGbdGuIbP4cAHavSVoVAQPgnTt_0 2026-02-24T22:34:25.325 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9785480130137064040', time=2026-02-25T00:25:59.457341+0000, chain.objs.size()=1 2026-02-24T22:34:25.325 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.WSsiLBE-fFgxsfzMTkpIu7Ie92snl7f_0 2026-02-24T22:34:25.325 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3608490798254074119', time=2026-02-25T00:25:59.758142+0000, chain.objs.size()=1 2026-02-24T22:34:25.325 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.5eD6SJVHbmA46g54hhVHt6zOG5F8zqU_0 2026-02-24T22:34:25.325 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9304826419802126406', time=2026-02-25T00:25:59.761900+0000, chain.objs.size()=1 2026-02-24T22:34:25.325 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.e8lMeXtGL1bDQPWvv8ECM5FFx23Csf5_0 2026-02-24T22:34:25.325 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6384907218322016571', time=2026-02-25T00:25:59.770731+0000, chain.objs.size()=1 2026-02-24T22:34:25.325 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ToRizieomD_KQv93_E2jQqZLciUHekm_0 2026-02-24T22:34:25.325 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10708309470917891426', time=2026-02-25T00:25:59.781317+0000, chain.objs.size()=1 2026-02-24T22:34:25.325 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ZNm1lL1Jg_XfTXDbhCpim1-A1A7sBJL_0 2026-02-24T22:34:25.325 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17104566566292273164', time=2026-02-25T00:25:59.810403+0000, chain.objs.size()=1 2026-02-24T22:34:25.325 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.QQ1upQQxxWOm95lj__hpsfBVT3M0Tnp_0 2026-02-24T22:34:25.325 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13825790802694432728', time=2026-02-25T00:25:59.834823+0000, chain.objs.size()=1 2026-02-24T22:34:25.325 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.vQ0RbOcxUMmw0C34mCvJQPF9ZLTeEpX_0 2026-02-24T22:34:25.325 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18090331201750220283', time=2026-02-25T00:25:59.837523+0000, chain.objs.size()=1 2026-02-24T22:34:25.325 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.e8unQOYQyRfGsOgvv_APoksuJriooE2_0 2026-02-24T22:34:25.325 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5341688654243557965', time=2026-02-25T00:25:59.922683+0000, chain.objs.size()=1 2026-02-24T22:34:25.325 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.IIOJe1LZrQoEQOzdJoooYr5Exw-FhW9_0 2026-02-24T22:34:25.325 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5691611766948786131', time=2026-02-25T00:25:59.924678+0000, chain.objs.size()=1 2026-02-24T22:34:25.325 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.crRtTglh-F0n9f0WzcdSP4aM1fD5OTW_0 2026-02-24T22:34:25.326 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2488863384215538100', time=2026-02-25T00:25:59.945343+0000, chain.objs.size()=1 2026-02-24T22:34:25.326 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.FXMzkKV1kz51X_cQn_8FSdk94H5hlXU_0 2026-02-24T22:34:25.326 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15708118148983351243', time=2026-02-25T00:25:59.977948+0000, chain.objs.size()=1 2026-02-24T22:34:25.326 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.PlyBNQ-93f80cjjMmBP0bpC0TCU_Ena_0 2026-02-24T22:34:25.326 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12074335207092515725', time=2026-02-25T00:26:00.094354+0000, chain.objs.size()=1 2026-02-24T22:34:25.326 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.QxCvrxwoHGcq0KIMZ_ABMuBm3HckTUs_0 2026-02-24T22:34:25.326 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.974873318472328754', time=2026-02-25T00:26:00.095628+0000, chain.objs.size()=1 2026-02-24T22:34:25.326 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.C-Nd9O8i-Mss7jqICXsdu4LuhEIJ2I6_0 2026-02-24T22:34:25.326 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2372116970231566970', time=2026-02-25T00:26:00.110335+0000, chain.objs.size()=1 2026-02-24T22:34:25.326 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.vuGci3BIM7Xx4aIqPXUS3hxTWC_6_Rj_0 2026-02-24T22:34:25.326 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2021347293520194903', time=2026-02-25T00:26:03.145690+0000, chain.objs.size()=1 2026-02-24T22:34:25.326 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1633__shadow_.RQyZBs9FVJWr96VL_41qLhGUWJZizUK_0 2026-02-24T22:34:25.326 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7626013965543201988', time=2026-02-25T00:26:07.693790+0000, chain.objs.size()=1 2026-02-24T22:34:25.326 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1641__shadow_.JcJerxufOToZFFdBhOTlbewxK4k6rIN_0 2026-02-24T22:34:25.326 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11525533088895486572', time=2026-02-25T00:26:09.413710+0000, chain.objs.size()=1 2026-02-24T22:34:25.326 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1644__shadow_.E-V1zM2PDebn4JvUoO9cv-c058aumjH_0 2026-02-24T22:34:25.326 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13277716663528499204', time=2026-02-25T00:26:11.201804+0000, chain.objs.size()=1 2026-02-24T22:34:25.327 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.335+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1647__shadow_.pWXD5jzB5NkuLxnrkA55MVSflhr2x0H_0 2026-02-24T22:34:25.327 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16310193489163749359', time=2026-02-25T00:26:15.438849+0000, chain.objs.size()=1 2026-02-24T22:34:25.327 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1654__shadow_.Nb_sQThrIbZgpRSoZ9KwdOfaiNLEFWV_0 2026-02-24T22:34:25.327 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8306516816847134230', time=2026-02-25T00:26:22.306326+0000, chain.objs.size()=1 2026-02-24T22:34:25.327 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1664__shadow_.3TxgPPFH5G9LnsSBdoHxY67-4ORIB6I_0 2026-02-24T22:34:25.327 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6256721657839332711', time=2026-02-25T00:26:22.882279+0000, chain.objs.size()=1 2026-02-24T22:34:25.327 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1665__shadow_.XpzOxDnc8yzDiiJeioMXfUMHaU3q08C_0 2026-02-24T22:34:25.327 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1405983491120883177', time=2026-02-25T00:26:44.819774+0000, chain.objs.size()=1 2026-02-24T22:34:25.327 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.e8ouMjgGnNmSp1pa6V3A8gNk4N5BXlW_0 2026-02-24T22:34:25.327 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11393596942493907911', time=2026-02-25T00:26:44.839267+0000, chain.objs.size()=1 2026-02-24T22:34:25.327 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.hlCT7amgogkdES3ZWYfbTUnY_fDuVju_0 2026-02-24T22:34:25.327 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15871165473754511556', time=2026-02-25T00:26:44.870824+0000, chain.objs.size()=1 2026-02-24T22:34:25.327 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.5SLlI09PYQiUMA4f-CB9kJeXWCaU5cZ_0 2026-02-24T22:34:25.327 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5570218675236845566', time=2026-02-25T00:26:44.960110+0000, chain.objs.size()=1 2026-02-24T22:34:25.328 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.UD1nTnapzhYav4nwON3WUwJOFA2wsjr_0 2026-02-24T22:34:25.328 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15603436018879977949', time=2026-02-25T00:26:44.984341+0000, chain.objs.size()=1 2026-02-24T22:34:25.328 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.cjm6lD5mGRoc8upHWfAURdWS6nEll7b_0 2026-02-24T22:34:25.328 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17396044515707828691', time=2026-02-25T00:26:45.036039+0000, chain.objs.size()=1 2026-02-24T22:34:25.328 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_._0JzNgfEvuKNg4yFOc6rTGJ3Jpkp-zC_0 2026-02-24T22:34:25.328 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12030559728920699907', time=2026-02-25T00:26:45.112172+0000, chain.objs.size()=1 2026-02-24T22:34:25.328 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.pz6j--pWdRMGnthz7JV5bOMbIsqkRX8_0 2026-02-24T22:34:25.328 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13028318057771854299', time=2026-02-25T00:26:45.115657+0000, chain.objs.size()=1 2026-02-24T22:34:25.328 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.GXcO5G3Ad5Yq0bhaTDDnd3j-bG7uHwc_0 2026-02-24T22:34:25.328 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7084831763074010619', time=2026-02-25T00:26:45.118333+0000, chain.objs.size()=1 2026-02-24T22:34:25.328 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.jRrvths8kF_LlyRO-8h0UbkJ4WcP1Yy_0 2026-02-24T22:34:25.328 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2312386149960610073', time=2026-02-25T00:26:45.124959+0000, chain.objs.size()=1 2026-02-24T22:34:25.328 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Xiehd4v_hDPRJTxVRTIgkJnW0ACcqP6_0 2026-02-24T22:34:25.328 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3804023592162313817', time=2026-02-25T00:26:45.126963+0000, chain.objs.size()=1 2026-02-24T22:34:25.329 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.O4lyN2Z_hGxZNFnXMAIdOhFHTsL8HGg_0 2026-02-24T22:34:25.329 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.308344310599436849', time=2026-02-25T00:26:45.139538+0000, chain.objs.size()=1 2026-02-24T22:34:25.329 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.fy9DvaPjv0Kzap6uKErLaP2uCPx56AR_0 2026-02-24T22:34:25.329 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14259296155613384708', time=2026-02-25T00:26:45.199565+0000, chain.objs.size()=1 2026-02-24T22:34:25.329 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.StJCFwWt-wtGs5pVnvdf5XHU9N9ubt6_0 2026-02-24T22:34:25.329 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4770746783487443641', time=2026-02-25T00:26:45.281701+0000, chain.objs.size()=1 2026-02-24T22:34:25.329 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.-ELAk-EbajfctMnbzGBYRiO9vY0QOd-_0 2026-02-24T22:34:25.329 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18366173144531513090', time=2026-02-25T00:26:45.344749+0000, chain.objs.size()=1 2026-02-24T22:34:25.329 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.rf1RPdTSJux2EzkWlqdjNgeVEKzVVtP_0 2026-02-24T22:34:25.329 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14409064626082641327', time=2026-02-25T00:26:45.347540+0000, chain.objs.size()=1 2026-02-24T22:34:25.329 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.TOOSYWrt5jPXbcoWUpgnkLR3rIJeJII_0 2026-02-24T22:34:25.329 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15526083080475684546', time=2026-02-25T00:26:45.435272+0000, chain.objs.size()=1 2026-02-24T22:34:25.329 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.RRZE7amQL64_4tsXw6sCM95qRHXjntq_0 2026-02-24T22:34:25.329 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11175599754088085320', time=2026-02-25T00:26:45.445379+0000, chain.objs.size()=1 2026-02-24T22:34:25.329 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.5wntBYCB_HbPBDnsUa4MSNn_498nnf6_0 2026-02-24T22:34:25.329 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11476941211739883017', time=2026-02-25T00:26:45.465755+0000, chain.objs.size()=1 2026-02-24T22:34:25.329 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.8TzeGrIGh3nCV6muMYzwu72PScfszPK_0 2026-02-24T22:34:25.330 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12454272640835884039', time=2026-02-25T00:26:45.598382+0000, chain.objs.size()=1 2026-02-24T22:34:25.330 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.4l9k4S7LFQu_0fjmIZSGXEGarbNcaAs_0 2026-02-24T22:34:25.330 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16575682587813061251', time=2026-02-25T00:26:45.840571+0000, chain.objs.size()=1 2026-02-24T22:34:25.330 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.fNUFOcptajFmEEULPDdwjxcAz0rcy7r_0 2026-02-24T22:34:25.330 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13215971567290027426', time=2026-02-25T00:26:45.846264+0000, chain.objs.size()=1 2026-02-24T22:34:25.330 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.rfssWiLVz9cqCiE6q5fBhwX2aL8Gbm0_0 2026-02-24T22:34:25.330 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15423841092859710075', time=2026-02-25T00:26:45.851985+0000, chain.objs.size()=1 2026-02-24T22:34:25.330 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.meLO95mha-JAX2eBXmhL60kL-aLpWJb_0 2026-02-24T22:34:25.330 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18312698600823159930', time=2026-02-25T00:26:45.930272+0000, chain.objs.size()=1 2026-02-24T22:34:25.330 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.IZsDQQkRpDP7NCNNoV9_FbUDmtYiain_0 2026-02-24T22:34:25.330 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4347430585773443557', time=2026-02-25T00:26:45.946682+0000, chain.objs.size()=1 2026-02-24T22:34:25.330 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.6cGvTwY0bPzGphsZy5ygBxs_fIY35V-_0 2026-02-24T22:34:25.330 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1615297699534538082', time=2026-02-25T00:26:45.960756+0000, chain.objs.size()=1 2026-02-24T22:34:25.330 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.yJ7097Zp-mAc90fttVkADrRzFFB7vOp_0 2026-02-24T22:34:25.330 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7941501444622376556', time=2026-02-25T00:26:46.012557+0000, chain.objs.size()=1 2026-02-24T22:34:25.330 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.WfTp35Md-6tLr2ltX8EEt0VkQZ1nhP1_0 2026-02-24T22:34:25.330 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16264067275390928079', time=2026-02-25T00:26:46.014111+0000, chain.objs.size()=1 2026-02-24T22:34:25.331 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.BEz9-_4nn8K_y_hOEAtFTTGaIsgwUGz_0 2026-02-24T22:34:25.331 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11334804384498421209', time=2026-02-25T00:26:46.015027+0000, chain.objs.size()=1 2026-02-24T22:34:25.331 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.KSeIUFN8HoVFsyHCz5kBh-MMerKajPD_0 2026-02-24T22:34:25.331 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16909026266869950231', time=2026-02-25T00:26:46.024270+0000, chain.objs.size()=1 2026-02-24T22:34:25.331 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.QJSV_qLFDTeM9OGf-BGYrjxm4nCEGWK_0 2026-02-24T22:34:25.331 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15366290131578713430', time=2026-02-25T00:26:46.096966+0000, chain.objs.size()=1 2026-02-24T22:34:25.331 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.tajQ1IUf27qnZQSCF8nAHCR_qD8jN5Z_0 2026-02-24T22:34:25.331 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4195218343541039351', time=2026-02-25T00:26:46.110367+0000, chain.objs.size()=1 2026-02-24T22:34:25.331 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Q9niiDBvdTfJhhttGMgM9eiC9qXQFPw_0 2026-02-24T22:34:25.331 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9099938508324919557', time=2026-02-25T00:26:46.144643+0000, chain.objs.size()=1 2026-02-24T22:34:25.331 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.qHzacpArlojUvCyZte_BqiSJJDREIll_0 2026-02-24T22:34:25.331 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1377981515447333857', time=2026-02-25T00:26:53.154080+0000, chain.objs.size()=1 2026-02-24T22:34:25.331 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.339+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.BHdOYD3T-15QztOZP0l30107OgdU-zP_0 2026-02-24T22:34:25.331 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15406619442758366268', time=2026-02-25T00:26:53.164304+0000, chain.objs.size()=1 2026-02-24T22:34:25.332 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.3pfYY4KzwiZE0vlx9DYMe0p2Roou65w_0 2026-02-24T22:34:25.332 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15073516764792314760', time=2026-02-25T00:26:53.181915+0000, chain.objs.size()=1 2026-02-24T22:34:25.332 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.JUg7iR0nz3uPwHPwH5pKrq-LmDf1JCr_0 2026-02-24T22:34:25.332 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14029879354372986681', time=2026-02-25T00:26:53.223188+0000, chain.objs.size()=1 2026-02-24T22:34:25.332 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.hSH3fvjTSAJFf82eB_lGVUI8ceWnRvu_0 2026-02-24T22:34:25.332 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6880236276733505587', time=2026-02-25T00:26:53.260501+0000, chain.objs.size()=1 2026-02-24T22:34:25.332 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ogLyHWXjF77AdEPHxktNp_a4mcQGOj5_0 2026-02-24T22:34:25.332 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4369643907403047602', time=2026-02-25T00:26:53.343701+0000, chain.objs.size()=1 2026-02-24T22:34:25.332 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.wCcGa05HQMnLmDzavdr8ToLmddLTzen_0 2026-02-24T22:34:25.332 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5163759507817258520', time=2026-02-25T00:26:53.404666+0000, chain.objs.size()=1 2026-02-24T22:34:25.332 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.SxnMFOW0GFVyngf3PM3sjiTaaPIW2W2_0 2026-02-24T22:34:25.332 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5158375687673760505', time=2026-02-25T00:26:53.414164+0000, chain.objs.size()=1 2026-02-24T22:34:25.332 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Uoxp1m-WSqynHOkKh-ZbbUjxGHLVkJd_0 2026-02-24T22:34:25.332 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16852381587287694511', time=2026-02-25T00:26:53.422045+0000, chain.objs.size()=1 2026-02-24T22:34:25.332 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.PwC91G6LwNvSNmoRN5D1EK_z1x_JroT_0 2026-02-24T22:34:25.332 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11130040046639798146', time=2026-02-25T00:26:53.626473+0000, chain.objs.size()=1 2026-02-24T22:34:25.332 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ZbiOtHxEMAdgrYJR3yeSJcz6REe5sFp_0 2026-02-24T22:34:25.332 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10098947047514880990', time=2026-02-25T00:26:53.628663+0000, chain.objs.size()=1 2026-02-24T22:34:25.332 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.V6MDJSeD5_zJg0MsP7uxhEvVt95QctT_0 2026-02-24T22:34:25.332 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13847270447974239336', time=2026-02-25T00:26:53.737961+0000, chain.objs.size()=1 2026-02-24T22:34:25.332 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.UvqiQpWtTpvHmN4cSmqJ4Q5DCOvEeOO_0 2026-02-24T22:34:25.332 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16113172425901485759', time=2026-02-25T00:26:53.740511+0000, chain.objs.size()=1 2026-02-24T22:34:25.332 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.0HMbmJtBHPD8Fvt7VFlTDPVgzjafUQn_0 2026-02-24T22:34:25.333 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4024013600858625366', time=2026-02-25T00:26:53.743710+0000, chain.objs.size()=1 2026-02-24T22:34:25.333 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.HxA7DRyuDooDZpGuXEFhEChFuNWiULO_0 2026-02-24T22:34:25.333 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2196565967954571432', time=2026-02-25T00:26:54.007858+0000, chain.objs.size()=1 2026-02-24T22:34:25.333 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.jekeXf-AMc0rP2U-BOGtWSoUbK0MoZ0_0 2026-02-24T22:34:25.333 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7610282240518748152', time=2026-02-25T00:26:54.195700+0000, chain.objs.size()=1 2026-02-24T22:34:25.333 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Cc8j9nNRAhacafGp2x-dB2taMSdB3_y_0 2026-02-24T22:34:25.333 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4094836361902520794', time=2026-02-25T00:26:54.298567+0000, chain.objs.size()=1 2026-02-24T22:34:25.333 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.GbPze1_dbjNF3SdQTCDZg7w-pZXpKOs_0 2026-02-24T22:34:25.333 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15264215131450463786', time=2026-02-25T00:26:54.347511+0000, chain.objs.size()=1 2026-02-24T22:34:25.333 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.cll7N37FJKisfnnndsBvJr07e96fRRm_0 2026-02-24T22:34:25.333 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10224905613520178507', time=2026-02-25T00:26:54.350365+0000, chain.objs.size()=1 2026-02-24T22:34:25.333 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.8bOTsixyK7J0Leh5Iq_aban_xL2-cjp_0 2026-02-24T22:34:25.334 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.343+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/37939 2026-02-24T22:34:25.336 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=17, truncated=0, next_marker='' 2026-02-24T22:34:25.336 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1969497385444647951', time=2026-02-25T00:26:54.364860+0000, chain.objs.size()=1 2026-02-24T22:34:25.336 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.gT-4wYakj2yAPW9ExMFpmzHaPIcxvj8_0 2026-02-24T22:34:25.336 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14477163939936893619', time=2026-02-25T00:26:54.493721+0000, chain.objs.size()=1 2026-02-24T22:34:25.336 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.JqPCDqb5_ajD4Djq0y-hhRGTQgkU10a_0 2026-02-24T22:34:25.336 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16927143209910200156', time=2026-02-25T00:26:54.514528+0000, chain.objs.size()=1 2026-02-24T22:34:25.336 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.GB9NW0cuoDOlX7xPJ8V8mDstbXodGRm_0 2026-02-24T22:34:25.336 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3464117673563246766', time=2026-02-25T00:26:54.528536+0000, chain.objs.size()=1 2026-02-24T22:34:25.336 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.RNrXcAYzfaTXwYIzrnvrDERpzm5m3CR_0 2026-02-24T22:34:25.336 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14607851078599918920', time=2026-02-25T00:26:54.531190+0000, chain.objs.size()=1 2026-02-24T22:34:25.336 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.6fLFV4B_r72nNHysmkaTBZvgMlise2P_0 2026-02-24T22:34:25.336 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8483345794496293645', time=2026-02-25T00:28:22.181707+0000, chain.objs.size()=1 2026-02-24T22:34:25.336 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1827__shadow_.6xYhav2fYoJBsRNt-oIwnK2BglFypyD_0 2026-02-24T22:34:25.336 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13443718669748113493', time=2026-02-25T00:28:57.859094+0000, chain.objs.size()=1 2026-02-24T22:34:25.336 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1886__shadow:qIW99n4oN65yr8XfPIK9tE4Lqu8nC-5_.xlERclzNDNl8pcOXOxhFYE5FgIKZCfG_0 2026-02-24T22:34:25.336 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18080080653802301217', time=2026-02-25T00:29:03.400000+0000, chain.objs.size()=6 2026-02-24T22:34:25.337 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~FPBPqLlYNU4LR56rnpatuhFZ5_ekRiy.1 2026-02-24T22:34:25.337 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.i_-W9AdgFJYQ7x5TSB4NyTZU5tFKQDNx.2 2026-02-24T22:34:25.337 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~FPBPqLlYNU4LR56rnpatuhFZ5_ekRiy.3 2026-02-24T22:34:25.337 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~FPBPqLlYNU4LR56rnpatuhFZ5_ekRiy.4 2026-02-24T22:34:25.337 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~FPBPqLlYNU4LR56rnpatuhFZ5_ekRiy.5 2026-02-24T22:34:25.337 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~FPBPqLlYNU4LR56rnpatuhFZ5_ekRiy.6 2026-02-24T22:34:25.337 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3002093703606763633', time=2026-02-25T00:29:10.486105+0000, chain.objs.size()=2 2026-02-24T22:34:25.337 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1889__multipart_mymultipart.2~2t-PF_5IfqBbrQJ5pe9N-uitfJl3k3a.1 2026-02-24T22:34:25.337 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1889__multipart_mymultipart.2~2t-PF_5IfqBbrQJ5pe9N-uitfJl3k3a.2 2026-02-24T22:34:25.337 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7339637800741634274', time=2026-02-25T00:30:02.074947+0000, chain.objs.size()=1 2026-02-24T22:34:25.337 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1918__shadow_.6cZz2EnB19heB5947n6Yeh-MXRNvx4s_0 2026-02-24T22:34:25.337 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17695888128301082559', time=2026-02-25T00:30:03.856853+0000, chain.objs.size()=1 2026-02-24T22:34:25.338 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1920__shadow_.Dkld3cWvjR3NwELTA1ZFBpzWHKNiHlz_0 2026-02-24T22:34:25.338 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8778147219908114550', time=2026-02-25T00:30:07.088918+0000, chain.objs.size()=1 2026-02-24T22:34:25.338 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1924__shadow_.FoIXQfUeaMGpb4K9xDOBPpWV-nlouwo_0 2026-02-24T22:34:25.338 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13574555489192729700', time=2026-02-25T00:30:24.327724+0000, chain.objs.size()=1 2026-02-24T22:34:25.338 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.347+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1938__shadow:yilDhxAwWZ.HZ5hZD4.YdGYlMscD5QI_.MYCGQEMOd_tza2MYV2H6wJICjfJSyok_0 2026-02-24T22:34:25.338 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.351+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16832776239600875339', time=2026-02-25T00:30:38.957946+0000, chain.objs.size()=1 2026-02-24T22:34:25.338 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.351+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:s70pL99AxeYGJimwyjlAzHk8A6AcSQF_.bTezVMWMV4yuTGBwNemsm7nqo33AIDK_0 2026-02-24T22:34:25.339 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.351+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3912604745563393725', time=2026-02-25T00:30:57.380970+0000, chain.objs.size()=1 2026-02-24T22:34:25.339 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.351+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:1ytO7oi3pMLFPKVS7SNeEIZWqwlpndn_.REXaF7DSHeSxItpcr1ZNjmf89hZGacx_0 2026-02-24T22:34:25.339 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.351+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3912604745563393725', time=2026-02-25T00:30:57.381443+0000, chain.objs.size()=1 2026-02-24T22:34:25.339 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.351+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:1ytO7oi3pMLFPKVS7SNeEIZWqwlpndn_.REXaF7DSHeSxItpcr1ZNjmf89hZGacx_0 2026-02-24T22:34:25.339 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.351+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.95932659202196628', time=2026-02-25T00:34:00.485690+0000, chain.objs.size()=1 2026-02-24T22:34:25.339 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.351+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1983__shadow_.rItnoXvGSOCQWx0dDSz9eDWAyPgjlAJ_0 2026-02-24T22:34:25.340 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.351+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.342 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.351+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2026-02-24T22:34:25.343 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.343 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.344 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/38159' 2026-02-24T22:34:25.344 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10304081461979102702', time=2026-02-25T00:15:34.719086+0000, chain.objs.size()=1 2026-02-24T22:34:25.344 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516__shadow_.CdgduzJ1eKGKrr2KZaGcK9i4YvqxGVS_0 2026-02-24T22:34:25.344 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='_B2FfFxI9qhwogYvcRSKsvoB7bA6j04b', time=2026-02-25T00:15:34.766230+0000, chain.objs.size()=2 2026-02-24T22:34:25.345 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__shadow_.gZ3g481IEgRHeOYtYzixIcbqGp57sGE_0 2026-02-24T22:34:25.345 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__shadow_.gZ3g481IEgRHeOYtYzixIcbqGp57sGE_1 2026-02-24T22:34:25.345 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11280477865082799394', time=2026-02-25T00:16:32.616419+0000, chain.objs.size()=1 2026-02-24T22:34:25.345 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520__shadow_.t8ErI29lchqUb3TWPWz_aMI0LCj5wDI_0 2026-02-24T22:34:25.345 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8003811483805052656', time=2026-02-25T00:16:32.617977+0000, chain.objs.size()=1 2026-02-24T22:34:25.345 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520__shadow_.cMq0V9Uvt-A37pXs4F8YqF5NbHXZjIB_0 2026-02-24T22:34:25.345 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4712831011271975909', time=2026-02-25T00:16:32.618603+0000, chain.objs.size()=1 2026-02-24T22:34:25.345 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520__shadow_._D_GcXIEDGFFvnv8gf9iI2zd1OaNb_6_0 2026-02-24T22:34:25.345 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13715510785307388667', time=2026-02-25T00:17:39.881960+0000, chain.objs.size()=1 2026-02-24T22:34:25.345 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525__shadow_.wyPvwd4sTPP8BvYNPloBuiqTeXFXn52_0 2026-02-24T22:34:25.345 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11896502773349133941', time=2026-02-25T00:17:39.884301+0000, chain.objs.size()=1 2026-02-24T22:34:25.345 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525__shadow_.CYSS7GPnrVc_RkgU6_dwRqeTE1ozR4u_0 2026-02-24T22:34:25.345 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13793452527533419675', time=2026-02-25T00:17:41.434337+0000, chain.objs.size()=1 2026-02-24T22:34:25.345 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526__shadow_.LiPTVKtS36dJ67Tfi2kbCNUtpW1VsZZ_0 2026-02-24T22:34:25.345 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13431817540066931879', time=2026-02-25T00:17:41.434982+0000, chain.objs.size()=1 2026-02-24T22:34:25.345 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526__shadow_.99ekLJopctJvoh900GHyi9-vWFDj7Z5_0 2026-02-24T22:34:25.345 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11680370994031923818', time=2026-02-25T00:17:42.538324+0000, chain.objs.size()=1 2026-02-24T22:34:25.345 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527__shadow_.La6NgddCtaVOjMj7tMKZQR19Df8YH7T_0 2026-02-24T22:34:25.345 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1664270371470484403', time=2026-02-25T00:17:42.540447+0000, chain.objs.size()=1 2026-02-24T22:34:25.346 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527__shadow_.z7CdmH5qVExmf_Fo8GN6ye_7U7-rnop_0 2026-02-24T22:34:25.346 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14103930418488054627', time=2026-02-25T00:18:14.580935+0000, chain.objs.size()=1 2026-02-24T22:34:25.346 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.355+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528__shadow_.G_Eaz_Q_W7m5NcNqq1U53vzDklfA5af_0 2026-02-24T22:34:25.346 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7194180894849623734', time=2026-02-25T00:18:16.135759+0000, chain.objs.size()=1 2026-02-24T22:34:25.346 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529__shadow_.RYWa73syB5avRhB1q-E-4S7kU93qEc7_0 2026-02-24T22:34:25.346 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17836867710498364503', time=2026-02-25T00:19:23.074187+0000, chain.objs.size()=1 2026-02-24T22:34:25.347 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533__shadow_.EziEW3rI_yTwa0Xw3HKCFJEJ4Az5vs2_0 2026-02-24T22:34:25.347 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15411920715131951416', time=2026-02-25T00:19:23.074790+0000, chain.objs.size()=1 2026-02-24T22:34:25.347 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533__shadow_.LR61w3CiqcfreGg4YuagLsgEY_6QBvQ_0 2026-02-24T22:34:25.347 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7779658914850426433', time=2026-02-25T00:19:23.075988+0000, chain.objs.size()=1 2026-02-24T22:34:25.347 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533__shadow_.bfIIayHMZoCLGbyR4C9aDccK_31vRqq_0 2026-02-24T22:34:25.347 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12091601072080686349', time=2026-02-25T00:19:57.122362+0000, chain.objs.size()=1 2026-02-24T22:34:25.347 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535__shadow_.v2bNvoq8FY1z3FCPYycKAL0Qy1rdXco_0 2026-02-24T22:34:25.347 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5495269601594781651', time=2026-02-25T00:19:58.461886+0000, chain.objs.size()=1 2026-02-24T22:34:25.347 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536__shadow_.FtubAUUJsDKcr-3NYtW_BsishEGQ9Be_0 2026-02-24T22:34:25.347 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10789581185329058137', time=2026-02-25T00:23:14.364001+0000, chain.objs.size()=1 2026-02-24T22:34:25.347 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518__shadow_.VDJ6hVjq91NCFESmfyBnTrxjmiJjJgw_0 2026-02-24T22:34:25.347 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17247981998123531578', time=2026-02-25T00:25:37.543332+0000, chain.objs.size()=1 2026-02-24T22:34:25.347 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1603__shadow_.hhqiC-jj7QSx0Ergva6urYJt8XXoLEZ_0 2026-02-24T22:34:25.347 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12189686613372135837', time=2026-02-25T00:25:38.506442+0000, chain.objs.size()=1 2026-02-24T22:34:25.347 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1604__shadow_.8BAcQTsjXBDHi53XZxBR8BnaCko0VfD_0 2026-02-24T22:34:25.347 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8563774142529507778', time=2026-02-25T00:25:40.170584+0000, chain.objs.size()=1 2026-02-24T22:34:25.347 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1607__shadow_.ggZpuqnfpkM95n78B9XY02aIIKG-3QD_0 2026-02-24T22:34:25.347 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8691895408996129271', time=2026-02-25T00:25:43.254094+0000, chain.objs.size()=1 2026-02-24T22:34:25.347 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1611__shadow_.B_rpUhcVLQMV3VgePIGd01q8Bj_LHsj_0 2026-02-24T22:34:25.348 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3906157974973089817', time=2026-02-25T00:25:44.441558+0000, chain.objs.size()=1 2026-02-24T22:34:25.348 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1613__shadow_.Sn3VAPh72baTQfSpvXLKOgvTodQaVo-_0 2026-02-24T22:34:25.348 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16243098619253944622', time=2026-02-25T00:25:46.733772+0000, chain.objs.size()=1 2026-02-24T22:34:25.349 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1617__shadow_.10iS2vYc_KFDlVPiaFikaQw0rQIVD43_0 2026-02-24T22:34:25.349 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.239219262481262766', time=2026-02-25T00:25:50.841887+0000, chain.objs.size()=1 2026-02-24T22:34:25.349 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1624__shadow_.TKdgLOL7dO6ETG9SIUOG5UUiinRPbwz_0 2026-02-24T22:34:25.349 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10965476129131122508', time=2026-02-25T00:25:59.118915+0000, chain.objs.size()=1 2026-02-24T22:34:25.349 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.W674ng3904f5e8OSTVRFM6LvY7NdiOY_0 2026-02-24T22:34:25.349 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5405496474059764304', time=2026-02-25T00:25:59.121653+0000, chain.objs.size()=1 2026-02-24T22:34:25.349 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.yfUckJ39F5IiHWjVaLtZEDA4hJbcaYQ_0 2026-02-24T22:34:25.349 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14766237526370114562', time=2026-02-25T00:25:59.134285+0000, chain.objs.size()=1 2026-02-24T22:34:25.349 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ikRqkVs8qODWVtGP_ERrmOCobICEixu_0 2026-02-24T22:34:25.349 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.20045353349668860', time=2026-02-25T00:25:59.143413+0000, chain.objs.size()=1 2026-02-24T22:34:25.349 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.v3f4b6jy_HTl5rzLHwNyTHq4wREulZy_0 2026-02-24T22:34:25.349 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6010592258595005421', time=2026-02-25T00:25:59.175078+0000, chain.objs.size()=1 2026-02-24T22:34:25.349 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.88yglo6RthFZmBl42CMvbn0ZiwW-sCJ_0 2026-02-24T22:34:25.349 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3003132956386053865', time=2026-02-25T00:25:59.278366+0000, chain.objs.size()=1 2026-02-24T22:34:25.349 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.rh6vpWKMhoCgwE5WI3f_wxLA9W9f7Oz_0 2026-02-24T22:34:25.349 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4852391830114895749', time=2026-02-25T00:25:59.278794+0000, chain.objs.size()=1 2026-02-24T22:34:25.349 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.e0MGHecV6Vb7GrJjTBHxen94mpZf1oM_0 2026-02-24T22:34:25.349 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4058724449552750438', time=2026-02-25T00:25:59.280766+0000, chain.objs.size()=1 2026-02-24T22:34:25.349 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.9FWXSU5tH5SUnFRYg5UHNwPJXUPpIjV_0 2026-02-24T22:34:25.349 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5203510056887291981', time=2026-02-25T00:25:59.293262+0000, chain.objs.size()=1 2026-02-24T22:34:25.350 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.OG5Pvh-iPpP6GXAg42XnQu_AfW75Hel_0 2026-02-24T22:34:25.350 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17996357056518775938', time=2026-02-25T00:25:59.300629+0000, chain.objs.size()=1 2026-02-24T22:34:25.350 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.lrQdU7n3Q4GoSe8kDK3bC7G9to_EOlE_0 2026-02-24T22:34:25.350 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9505619455940296234', time=2026-02-25T00:25:59.306649+0000, chain.objs.size()=1 2026-02-24T22:34:25.350 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Mc_70PdeojqnYAVJL0mU_RbDCaNljI9_0 2026-02-24T22:34:25.350 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15103662351795661339', time=2026-02-25T00:25:59.317045+0000, chain.objs.size()=1 2026-02-24T22:34:25.350 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.vKNVuJm1fpmgM-pn1O7d0trqnYHjRbA_0 2026-02-24T22:34:25.350 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14287139942108531530', time=2026-02-25T00:25:59.627978+0000, chain.objs.size()=1 2026-02-24T22:34:25.350 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.gJ1JJO6IEeqN023m7JZ8hQYMkvGcRRN_0 2026-02-24T22:34:25.350 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15684710507861729936', time=2026-02-25T00:25:59.631942+0000, chain.objs.size()=1 2026-02-24T22:34:25.350 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ANnp70-D1_-LNqE7TFV8RKPZNNPil-1_0 2026-02-24T22:34:25.351 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2896623526737123847', time=2026-02-25T00:25:59.638129+0000, chain.objs.size()=1 2026-02-24T22:34:25.351 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.sHIeN8ARyFhFdOYpnYTlkDIo3l1apPw_0 2026-02-24T22:34:25.351 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18037830216634782826', time=2026-02-25T00:25:59.792650+0000, chain.objs.size()=1 2026-02-24T22:34:25.351 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.G3cdhbuh538rXDhdUfEna0XUI1MUE2L_0 2026-02-24T22:34:25.351 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6172293454919625449', time=2026-02-25T00:25:59.803764+0000, chain.objs.size()=1 2026-02-24T22:34:25.352 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.weGLRsRAzBt8sUXLJXf8t3YqLMFliI3_0 2026-02-24T22:34:25.352 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10314887933369306684', time=2026-02-25T00:25:59.835696+0000, chain.objs.size()=1 2026-02-24T22:34:25.352 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.trkGKDktoFuhZJPBBpN_8DCzEm-tjZJ_0 2026-02-24T22:34:25.352 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14017612387252100640', time=2026-02-25T00:25:59.965190+0000, chain.objs.size()=1 2026-02-24T22:34:25.352 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.kK69CZRecpdSVn3rC1C8frAtEnIB8UN_0 2026-02-24T22:34:25.352 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11630265620834474970', time=2026-02-25T00:26:00.064507+0000, chain.objs.size()=1 2026-02-24T22:34:25.352 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.dpBOoxv4TU710ZkalPLe4Eor56CccNR_0 2026-02-24T22:34:25.352 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17411465866786794537', time=2026-02-25T00:26:00.096455+0000, chain.objs.size()=1 2026-02-24T22:34:25.352 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.57ZqagD3D2oi348IbrnmE39V4nz0J9M_0 2026-02-24T22:34:25.352 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16279174687673653955', time=2026-02-25T00:26:11.202272+0000, chain.objs.size()=1 2026-02-24T22:34:25.352 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1647__shadow_.qX4JN201bSqKOfFy6yLP81gz0Kzpj9G_0 2026-02-24T22:34:25.352 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9516201938738251410', time=2026-02-25T00:26:14.230305+0000, chain.objs.size()=1 2026-02-24T22:34:25.352 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1652__shadow_.SMIqjhx8mc2rBFmcvtkTOrJWoBspxiv_0 2026-02-24T22:34:25.352 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7647789280832878088', time=2026-02-25T00:26:15.438134+0000, chain.objs.size()=1 2026-02-24T22:34:25.352 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1654__shadow_.X2A1d-sjiLYSZvvZ-bYMYLG-fmTn1xO_0 2026-02-24T22:34:25.352 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4521211689044490608', time=2026-02-25T00:26:20.573638+0000, chain.objs.size()=1 2026-02-24T22:34:25.352 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1661__shadow_.OzorbPDsd7MdYiLLGw9uVFnimIqpiPp_0 2026-02-24T22:34:25.352 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13063422796704101337', time=2026-02-25T00:26:27.193331+0000, chain.objs.size()=1 2026-02-24T22:34:25.352 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1672__shadow_.AklHMRirTRjIHPrDIjgpPXkHrk-Dc5P_0 2026-02-24T22:34:25.352 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13252451477464941121', time=2026-02-25T00:26:44.803837+0000, chain.objs.size()=1 2026-02-24T22:34:25.352 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Nn2dDcIQ87gRayhq_1D99eN3tBQlTht_0 2026-02-24T22:34:25.352 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9983940373040496415', time=2026-02-25T00:26:44.809385+0000, chain.objs.size()=1 2026-02-24T22:34:25.353 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.NK8z9QhDoqB27F7u3rUDb055hbI7cEj_0 2026-02-24T22:34:25.353 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8994868058599616185', time=2026-02-25T00:26:44.822785+0000, chain.objs.size()=1 2026-02-24T22:34:25.353 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.NiA0gKkilhrvdh_bZuOXpUguhuV7T2d_0 2026-02-24T22:34:25.353 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15474811436289651875', time=2026-02-25T00:26:44.860166+0000, chain.objs.size()=1 2026-02-24T22:34:25.353 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.359+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.aU5xLFp_zyQuijfYiogO4E-mr_wNHgP_0 2026-02-24T22:34:25.353 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16207609600148119599', time=2026-02-25T00:26:44.972536+0000, chain.objs.size()=1 2026-02-24T22:34:25.353 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.v7OJqsU8fJT-M97FyL7hPpsZMIvT1X5_0 2026-02-24T22:34:25.353 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14937351238655826330', time=2026-02-25T00:26:44.980925+0000, chain.objs.size()=1 2026-02-24T22:34:25.353 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.G91X-tk9gtpqJd3ODZmR-e0lq8katYG_0 2026-02-24T22:34:25.353 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.761135080882670548', time=2026-02-25T00:26:44.984018+0000, chain.objs.size()=1 2026-02-24T22:34:25.353 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.s5-nq3VgdDfjhZqhzIKxtPCWKuHn5jh_0 2026-02-24T22:34:25.353 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2393971685055491839', time=2026-02-25T00:26:45.002378+0000, chain.objs.size()=1 2026-02-24T22:34:25.353 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.oZPQgAw9i2vXT-FD8B9q5gFn1exfTOS_0 2026-02-24T22:34:25.353 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14562453958410928376', time=2026-02-25T00:26:45.007112+0000, chain.objs.size()=1 2026-02-24T22:34:25.353 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.baB-a-zORjarXZ89KipHP0xsvQohnAh_0 2026-02-24T22:34:25.353 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11544107919834841510', time=2026-02-25T00:26:45.114636+0000, chain.objs.size()=1 2026-02-24T22:34:25.353 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.NQIBuVyUpn_I_G_YO24V2Gg0SVAzCT0_0 2026-02-24T22:34:25.353 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2990413867404779116', time=2026-02-25T00:26:45.128773+0000, chain.objs.size()=1 2026-02-24T22:34:25.353 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.r44SO8M2-1JeLUwEcwSYmaC7urZK3wX_0 2026-02-24T22:34:25.353 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11036042012613331435', time=2026-02-25T00:26:45.162755+0000, chain.objs.size()=1 2026-02-24T22:34:25.353 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Z9kfXGCU_EamJnFBC6CFMm688-ZSK9X_0 2026-02-24T22:34:25.353 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15937698347090716681', time=2026-02-25T00:26:45.185090+0000, chain.objs.size()=1 2026-02-24T22:34:25.353 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.106Y-ordHOAB8BYJvCo8QhpHVxxfUhd_0 2026-02-24T22:34:25.354 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10732682404903184839', time=2026-02-25T00:26:45.187341+0000, chain.objs.size()=1 2026-02-24T22:34:25.354 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.-IzvKz91WEvVHrKJ8C1PcVJ5Xq6AW-a_0 2026-02-24T22:34:25.354 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15916689861262848239', time=2026-02-25T00:26:45.187953+0000, chain.objs.size()=1 2026-02-24T22:34:25.354 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.6jYbe7u-PmPO6QDeRJ2a8w-FkYscH9A_0 2026-02-24T22:34:25.355 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5461565056580479626', time=2026-02-25T00:26:45.190266+0000, chain.objs.size()=1 2026-02-24T22:34:25.355 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.LjP__7p1Bh15KdhUsegIPrK329-00y4_0 2026-02-24T22:34:25.355 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2707831189138655784', time=2026-02-25T00:26:45.284606+0000, chain.objs.size()=1 2026-02-24T22:34:25.355 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.FBn3UmNym4cHAK6BX0xd23Jpy8bzbL8_0 2026-02-24T22:34:25.355 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17046377417269121726', time=2026-02-25T00:26:45.305143+0000, chain.objs.size()=1 2026-02-24T22:34:25.355 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.x3PJFH0ZGOXGeJSwpJnsYJFiXfbCULJ_0 2026-02-24T22:34:25.355 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11938207680976543130', time=2026-02-25T00:26:45.315941+0000, chain.objs.size()=1 2026-02-24T22:34:25.355 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.8go3WqjW1FYf5YJ4zXARsAw8mzmV-IV_0 2026-02-24T22:34:25.355 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16769225183528848298', time=2026-02-25T00:26:45.316504+0000, chain.objs.size()=1 2026-02-24T22:34:25.355 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.QT9Olj7AhWjafre3QahqJwT3r-sLMYa_0 2026-02-24T22:34:25.355 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.753298188151563003', time=2026-02-25T00:26:45.325201+0000, chain.objs.size()=1 2026-02-24T22:34:25.355 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.V4jebOmL5w8trYrzYNyKdgWiZsP31d1_0 2026-02-24T22:34:25.355 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14345475187735017817', time=2026-02-25T00:26:45.344687+0000, chain.objs.size()=1 2026-02-24T22:34:25.355 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.nBTdDLGREL4oJpSpDooitOBHFbkuw3V_0 2026-02-24T22:34:25.355 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14380659736374100147', time=2026-02-25T00:26:45.420426+0000, chain.objs.size()=1 2026-02-24T22:34:25.355 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.crDU-8rhK3zvtGergBWu4YnO3PWUvWF_0 2026-02-24T22:34:25.355 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17883393202587927520', time=2026-02-25T00:26:45.459714+0000, chain.objs.size()=1 2026-02-24T22:34:25.356 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.XNh4bpPDj1b562Cie2zYgyPFH6G-jmC_0 2026-02-24T22:34:25.356 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7071504504111896305', time=2026-02-25T00:26:45.479937+0000, chain.objs.size()=1 2026-02-24T22:34:25.356 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.y6-6w-wIEu4_id0_l3vT049MdPUD7YM_0 2026-02-24T22:34:25.356 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13152447277663646577', time=2026-02-25T00:26:45.491266+0000, chain.objs.size()=1 2026-02-24T22:34:25.356 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.0h_-LoB8MsSwQUl7B7ZLg1CShfjY2Pt_0 2026-02-24T22:34:25.356 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7541680162448424506', time=2026-02-25T00:26:45.602025+0000, chain.objs.size()=1 2026-02-24T22:34:25.356 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.jfIN7muU9twH_BOFUPAYGppX8PIsoE-_0 2026-02-24T22:34:25.356 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10761845901294299140', time=2026-02-25T00:26:45.928165+0000, chain.objs.size()=1 2026-02-24T22:34:25.356 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.3xljig0ZVajNEr6n1-8Bo4nLvsLNM4C_0 2026-02-24T22:34:25.356 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.988643557385420161', time=2026-02-25T00:26:45.930181+0000, chain.objs.size()=1 2026-02-24T22:34:25.356 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.iiqjhupTEWhXGeQk1aJdAISN2ofeNrW_0 2026-02-24T22:34:25.356 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11408767937970046211', time=2026-02-25T00:26:45.998113+0000, chain.objs.size()=1 2026-02-24T22:34:25.356 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.zBU0cin8W2rsD-rTcPqLX0n98jX-Ygx_0 2026-02-24T22:34:25.356 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12695562785249119119', time=2026-02-25T00:26:46.025963+0000, chain.objs.size()=1 2026-02-24T22:34:25.356 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.mBXdlapJQ7St4jC9gVkcsdn-kQvzzyD_0 2026-02-24T22:34:25.356 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9089472699226592677', time=2026-02-25T00:26:46.104411+0000, chain.objs.size()=1 2026-02-24T22:34:25.356 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.OAs5EOzA38gNvp5BD0VU2IOriupNxhw_0 2026-02-24T22:34:25.356 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10533567612451629458', time=2026-02-25T00:26:46.110316+0000, chain.objs.size()=1 2026-02-24T22:34:25.356 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.s22CSAvP3pFOkk9TXQ5f0UjGvttY2EK_0 2026-02-24T22:34:25.356 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10998388222855712132', time=2026-02-25T00:26:46.137561+0000, chain.objs.size()=1 2026-02-24T22:34:25.356 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.4Nw1MMc-aKZNCqSglE_RaWwkD5rYjby_0 2026-02-24T22:34:25.356 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14806348155431121151', time=2026-02-25T00:26:53.152760+0000, chain.objs.size()=1 2026-02-24T22:34:25.356 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.fS9QNOQqB7BnmHkl9QGpIzJvlYUYxEK_0 2026-02-24T22:34:25.356 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9792025243363112630', time=2026-02-25T00:26:53.155141+0000, chain.objs.size()=1 2026-02-24T22:34:25.356 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.PCriH2N7SWVniM2zQSFQU41TTQhBcII_0 2026-02-24T22:34:25.357 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4968040892842321185', time=2026-02-25T00:26:53.178282+0000, chain.objs.size()=1 2026-02-24T22:34:25.357 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.7ZUNcTI_K-2rvbr-gFDip6y9NbXKyuY_0 2026-02-24T22:34:25.357 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.314956279809588235', time=2026-02-25T00:26:53.186407+0000, chain.objs.size()=1 2026-02-24T22:34:25.357 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.363+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.1TGouJk5pk9jgWld9NeRelp_7jS4uVR_0 2026-02-24T22:34:25.357 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14318215364595570732', time=2026-02-25T00:26:53.233199+0000, chain.objs.size()=1 2026-02-24T22:34:25.357 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.5OXgcECWmFyz0k7OJOBACRQ3aQBJGds_0 2026-02-24T22:34:25.357 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13025626097233853049', time=2026-02-25T00:26:53.263996+0000, chain.objs.size()=1 2026-02-24T22:34:25.357 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.909Hx12aA_NucZEA_URelEPYVBO16WC_0 2026-02-24T22:34:25.357 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14254054682537413798', time=2026-02-25T00:26:53.388060+0000, chain.objs.size()=1 2026-02-24T22:34:25.357 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.VAzHB29F5Qjes9KsY7hNugv4Q6wxR5a_0 2026-02-24T22:34:25.357 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2071406053072810933', time=2026-02-25T00:26:53.566877+0000, chain.objs.size()=1 2026-02-24T22:34:25.357 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.6PMLZczdShVM6kd3KAgWNKy8ZtjA0Zi_0 2026-02-24T22:34:25.357 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5045774125659368611', time=2026-02-25T00:26:53.579290+0000, chain.objs.size()=1 2026-02-24T22:34:25.358 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.qMSC2ddvdvKXGiNi4LdjI-chyAHAd2v_0 2026-02-24T22:34:25.358 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15819489384365647230', time=2026-02-25T00:26:53.593239+0000, chain.objs.size()=1 2026-02-24T22:34:25.358 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.3ZArC2VeBK4m5Nc7OBBTdeUtmzFeiPl_0 2026-02-24T22:34:25.358 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12706044290428256976', time=2026-02-25T00:26:53.614676+0000, chain.objs.size()=1 2026-02-24T22:34:25.358 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.jDJCveXJ3q3xOifN8e4c8V_4ptK8JAc_0 2026-02-24T22:34:25.358 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11285999681457940348', time=2026-02-25T00:26:53.799414+0000, chain.objs.size()=1 2026-02-24T22:34:25.358 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.RC0vW6rgwtVGLFvhs0mRPUNJpxwiCBL_0 2026-02-24T22:34:25.358 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.351586083368637683', time=2026-02-25T00:26:53.928316+0000, chain.objs.size()=1 2026-02-24T22:34:25.358 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.0DTaqg3u7MJN90suUkCG9Jn6B14aaeT_0 2026-02-24T22:34:25.358 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15203745261783576045', time=2026-02-25T00:26:53.973818+0000, chain.objs.size()=1 2026-02-24T22:34:25.358 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.KXZpDfGDVEA3RsRpTd8ornUJWtKPHg5_0 2026-02-24T22:34:25.358 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/38159 2026-02-24T22:34:25.358 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=37, truncated=0, next_marker='' 2026-02-24T22:34:25.359 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13192278435310683846', time=2026-02-25T00:26:53.975244+0000, chain.objs.size()=1 2026-02-24T22:34:25.359 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.URGdkbaEHY8qVapr1ngQfD0Euap8znj_0 2026-02-24T22:34:25.359 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6884820717750846629', time=2026-02-25T00:26:53.991280+0000, chain.objs.size()=1 2026-02-24T22:34:25.359 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.eL_z631xMeKkzhBHwIqk5ar9CaRlCp0_0 2026-02-24T22:34:25.359 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8489433870233399804', time=2026-02-25T00:26:53.997091+0000, chain.objs.size()=1 2026-02-24T22:34:25.359 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.tnNbEed4xax405WjYuQY9qIw7yOWD08_0 2026-02-24T22:34:25.359 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8506770201153934717', time=2026-02-25T00:26:54.126537+0000, chain.objs.size()=1 2026-02-24T22:34:25.359 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Ebtp4t8YGqGULsZuwsRQIzJV97731L7_0 2026-02-24T22:34:25.359 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12455448433841493270', time=2026-02-25T00:26:54.148284+0000, chain.objs.size()=1 2026-02-24T22:34:25.359 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.0o0T27rgWU8BQ0UMTknz-BSzYUZlqbq_0 2026-02-24T22:34:25.359 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7345189269803030252', time=2026-02-25T00:26:54.298837+0000, chain.objs.size()=1 2026-02-24T22:34:25.359 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.gSlzgJvvB6NDn1Hsaf70FZT7tu50-uc_0 2026-02-24T22:34:25.359 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4750204631677961861', time=2026-02-25T00:26:54.309959+0000, chain.objs.size()=1 2026-02-24T22:34:25.359 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.3niXATyW5uIgxh9CDMzBN9XGnpEsjsJ_0 2026-02-24T22:34:25.359 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5584466687903236766', time=2026-02-25T00:26:54.336654+0000, chain.objs.size()=1 2026-02-24T22:34:25.359 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.pY_GDFMF0UtFY1eekn4q4nVuMFHHgTZ_0 2026-02-24T22:34:25.359 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9137410808524221312', time=2026-02-25T00:26:54.359681+0000, chain.objs.size()=1 2026-02-24T22:34:25.359 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.NtYbPcWcwix-S-eL95iCUITeL5XWvSf_0 2026-02-24T22:34:25.359 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1155891382305398299', time=2026-02-25T00:26:54.378892+0000, chain.objs.size()=1 2026-02-24T22:34:25.359 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.j9br6XxlfKiv2IIXkYt_UhWXbJ8RlfQ_0 2026-02-24T22:34:25.359 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16685570997754964885', time=2026-02-25T00:26:54.394066+0000, chain.objs.size()=1 2026-02-24T22:34:25.359 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.exvasHKw2H8RTTpDrpikQ3lKVAv2VP5_0 2026-02-24T22:34:25.359 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15252459333862438947', time=2026-02-25T00:26:54.472011+0000, chain.objs.size()=1 2026-02-24T22:34:25.359 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.367+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.SYc4YLt45qfLoK5IB0EzhYjmlst25xr_0 2026-02-24T22:34:25.360 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13749831662951735304', time=2026-02-25T00:26:54.486722+0000, chain.objs.size()=1 2026-02-24T22:34:25.360 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.RFLMyrkN5ERPf35Ptz71BIYKL0kjpm7_0 2026-02-24T22:34:25.360 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13357234711889641655', time=2026-02-25T00:26:54.497245+0000, chain.objs.size()=1 2026-02-24T22:34:25.360 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.k1P1P7TNgSurA0_TfqpLWZcIxCPnyRH_0 2026-02-24T22:34:25.360 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7116255166534470747', time=2026-02-25T00:26:54.532412+0000, chain.objs.size()=1 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.wApb3DdYz6Ae2dgUbjsPMI_U8CFTE5k_0 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17571047132269112537', time=2026-02-25T00:26:54.537126+0000, chain.objs.size()=1 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.mzJINL_RZkCXUafdKL_eS0dgwuF2KNH_0 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10211116972675705343', time=2026-02-25T00:27:19.296772+0000, chain.objs.size()=1 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1735__shadow_.GiUmEeBbM67ThOyqRJwYRGbPdU1sx9O_0 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15524086570779417334', time=2026-02-25T00:28:06.524980+0000, chain.objs.size()=1 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1806__shadow_.OsMKMkUh-oU4rtJAL0M95fKa9_KSZNk_0 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7379714144957908718', time=2026-02-25T00:28:22.985751+0000, chain.objs.size()=1 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1828__shadow_.AknppR0m1rSYcy6bblB0tv2TpNkfR3W_0 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11223612851954049068', time=2026-02-25T00:28:24.215190+0000, chain.objs.size()=1 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1830__shadow_.9PG8z6QddpOaE3HyhMebrFFl4IrhpSJ_0 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7756291676802344674', time=2026-02-25T00:28:24.433962+0000, chain.objs.size()=1 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1830__shadow_.xoKOvLbcSA0FwBEVQGDN3errcpaW-Y8_0 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12822065270169169282', time=2026-02-25T00:28:33.182516+0000, chain.objs.size()=1 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1850__shadow_.7rJyMLwm_slvTDH8mPhjc2yicsju2Vu_0 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18073547852196349676', time=2026-02-25T00:28:36.873856+0000, chain.objs.size()=1 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1856__shadow_.qpC4h95cGoYw2Wiwo3vaR_5V59TJCle_0 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='2~2U75DU5jxYT9C6w0U_bbZ5zy1gpqaKh', time=2026-02-25T00:29:18.551905+0000, chain.objs.size()=1 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1897__multipart_mymultipart2.2~2U75DU5jxYT9C6w0U_bbZ5zy1gpqaKh.1 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9126553052675699237', time=2026-02-25T00:29:44.228849+0000, chain.objs.size()=1 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1905__shadow_.zQOkyzYX9fC-nMKKLHjcPU_DxVx5666_0 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14040622354239801795', time=2026-02-25T00:30:22.489995+0000, chain.objs.size()=1 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:C9EhAjzslf7iE87.oF9rI1jdrr3jrRM_.Mz10IIp5g0E54XdSj50gFmjag7s_jZU_0 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14849707744370174471', time=2026-02-25T00:30:29.650710+0000, chain.objs.size()=1 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1943__shadow:uxCMRU-O0pJJJ9tJyotHHWvfqJIrKib_.-HjlgaBKAcjWX2te8qCzpujIgYRF1B2_0 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10400500691096454611', time=2026-02-25T00:30:31.830961+0000, chain.objs.size()=1 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1946__shadow:SObCo2EqiNA6GOc9WlutbDY5iKN70bE_.7tUkC1O53plJNvTTgE_C1jYifeEFk9s_0 2026-02-24T22:34:25.361 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6001641897616435611', time=2026-02-25T00:30:33.858488+0000, chain.objs.size()=1 2026-02-24T22:34:25.362 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1946__shadow:ruZSaXcmuMEzOzUA7VC6H0X5urwe36G_.c5tAuoVWI0be_1ZeTu5BzHyetMLhs6i_0 2026-02-24T22:34:25.362 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5956665185001160677', time=2026-02-25T00:30:38.170071+0000, chain.objs.size()=1 2026-02-24T22:34:25.362 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:vc2jrAJvHLyhovaz8HkR5FKe1LYEZ4q_.BZqf3CufrQB37MEyPJ7kS2S6cDXRxeY_0 2026-02-24T22:34:25.362 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2657714026163719966', time=2026-02-25T00:30:57.556372+0000, chain.objs.size()=1 2026-02-24T22:34:25.362 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:oHVUxXZXJQpU2o9E6pIvIM5Nswg2OYi_.uD4YBCFNd2uDOxRKn--85jJ2yRecfam_0 2026-02-24T22:34:25.362 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2657714026163719966', time=2026-02-25T00:30:57.557477+0000, chain.objs.size()=1 2026-02-24T22:34:25.362 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:oHVUxXZXJQpU2o9E6pIvIM5Nswg2OYi_.uD4YBCFNd2uDOxRKn--85jJ2yRecfam_0 2026-02-24T22:34:25.362 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17985714463035640839', time=2026-02-25T00:30:57.559949+0000, chain.objs.size()=1 2026-02-24T22:34:25.362 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:vyfr15Ft49sX2rjpRPHnNvsz29S8Ssc_.B4awwdGWN51HVR6pt2upNt-DDRimyeL_0 2026-02-24T22:34:25.362 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17985714463035640839', time=2026-02-25T00:30:57.560253+0000, chain.objs.size()=1 2026-02-24T22:34:25.362 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:vyfr15Ft49sX2rjpRPHnNvsz29S8Ssc_.B4awwdGWN51HVR6pt2upNt-DDRimyeL_0 2026-02-24T22:34:25.362 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17985714463035640839', time=2026-02-25T00:30:57.560843+0000, chain.objs.size()=1 2026-02-24T22:34:25.362 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:vyfr15Ft49sX2rjpRPHnNvsz29S8Ssc_.B4awwdGWN51HVR6pt2upNt-DDRimyeL_0 2026-02-24T22:34:25.362 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17985714463035640839', time=2026-02-25T00:30:57.561573+0000, chain.objs.size()=1 2026-02-24T22:34:25.362 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:vyfr15Ft49sX2rjpRPHnNvsz29S8Ssc_.B4awwdGWN51HVR6pt2upNt-DDRimyeL_0 2026-02-24T22:34:25.362 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2028763246097491300', time=2026-02-25T00:33:38.721002+0000, chain.objs.size()=1 2026-02-24T22:34:25.362 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1960__shadow_.4Vxoisa7u_wL12YPt1yKMSo4nBojngh_0 2026-02-24T22:34:25.362 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.371+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.363 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2026-02-24T22:34:25.364 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.365 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.366 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/39060' 2026-02-24T22:34:25.366 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6423560274400474440', time=2026-02-25T00:17:07.119817+0000, chain.objs.size()=1 2026-02-24T22:34:25.366 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523__shadow_.l7fquJ8nz3Xy-D0J9pAzCjaF0dJx793_0 2026-02-24T22:34:25.366 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3307483745077833482', time=2026-02-25T00:17:41.421960+0000, chain.objs.size()=1 2026-02-24T22:34:25.366 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526__shadow_.fbhbLGV4NDt29aPGmS0mzzWAazoazOn_0 2026-02-24T22:34:25.366 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8028934324948567281', time=2026-02-25T00:17:42.529069+0000, chain.objs.size()=1 2026-02-24T22:34:25.366 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527__shadow_.7_V6IvJajBUrLxlgJimIUKryEv7Kvo3_0 2026-02-24T22:34:25.366 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11638507972388644792', time=2026-02-25T00:18:48.183778+0000, chain.objs.size()=1 2026-02-24T22:34:25.366 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530__shadow_.AH7SrtN2hRf271u318V1OcP9uRq11ge_0 2026-02-24T22:34:25.366 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6733362752840085490', time=2026-02-25T00:18:49.766326+0000, chain.objs.size()=1 2026-02-24T22:34:25.366 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531__shadow_.ie5Tq55LeSf8nh5Lo0o6rIEoUdeH9hZ_0 2026-02-24T22:34:25.366 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8246346149828377973', time=2026-02-25T00:19:23.060628+0000, chain.objs.size()=1 2026-02-24T22:34:25.366 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533__shadow_.5zXkoVuWCsPvnQS-a16Us1x2OgwKXF__0 2026-02-24T22:34:25.366 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14997058971149396218', time=2026-02-25T00:19:23.061324+0000, chain.objs.size()=1 2026-02-24T22:34:25.366 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533__shadow_.qUb6xxzsorZ_DzNwRmM8VKaulXqt69C_0 2026-02-24T22:34:25.366 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13197491950078257149', time=2026-02-25T00:19:25.017661+0000, chain.objs.size()=1 2026-02-24T22:34:25.366 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534__shadow_.94T6HED1baOcPLz0OyRQGmSei63eauQ_0 2026-02-24T22:34:25.366 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14370396458043512503', time=2026-02-25T00:20:30.726500+0000, chain.objs.size()=1 2026-02-24T22:34:25.366 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537__shadow_.-Ws35h2W7ev5oF-P9fS9ehiHxel4UQh_0 2026-02-24T22:34:25.366 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15867329125257525747', time=2026-02-25T00:20:30.726992+0000, chain.objs.size()=1 2026-02-24T22:34:25.367 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537__shadow_.wGIjvRqvQFzPE8eeSEy3ZbcdzFUyNko_0 2026-02-24T22:34:25.367 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14560964286739108665', time=2026-02-25T00:20:32.508435+0000, chain.objs.size()=1 2026-02-24T22:34:25.367 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538__shadow_.uCgIgCsI2aN9BjgZ6H08Z1mOioQrQdj_0 2026-02-24T22:34:25.367 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.250040860864674486', time=2026-02-25T00:20:32.509963+0000, chain.objs.size()=1 2026-02-24T22:34:25.367 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.375+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538__shadow_.mbjlC6_qxhHX9ymBnPZdTJivVYRSQNH_0 2026-02-24T22:34:25.367 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4738607124200844005', time=2026-02-25T00:20:32.510480+0000, chain.objs.size()=1 2026-02-24T22:34:25.367 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538__shadow_.GdKw2zlX09Cwv9RcWIkt67QZjIDzOpg_0 2026-02-24T22:34:25.367 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11393784588935990598', time=2026-02-25T00:20:34.034546+0000, chain.objs.size()=1 2026-02-24T22:34:25.367 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539__shadow_.WPoDoARTxqEaDX-25CZJFQJQwfNgJMm_0 2026-02-24T22:34:25.367 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9372481757265191413', time=2026-02-25T00:21:06.309503+0000, chain.objs.size()=1 2026-02-24T22:34:25.368 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540__shadow_.JqQOEw8KuCRtwkXHeHeorqnDliQAbVf_0 2026-02-24T22:34:25.368 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1984240560175523691', time=2026-02-25T00:21:06.311199+0000, chain.objs.size()=1 2026-02-24T22:34:25.368 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540__shadow_.5WqeXsoiZeNxUeYppmGVqdgCTPLatPf_0 2026-02-24T22:34:25.368 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13148756598417388664', time=2026-02-25T00:23:21.956279+0000, chain.objs.size()=1 2026-02-24T22:34:25.368 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519__shadow:xmteDA9MA-ac6-PWfpypImi9pxCn9Ac_.7R-21A5XmWUMuvNzxSCmxxL4ZD5qTvm_0 2026-02-24T22:34:25.368 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3540340421134067199', time=2026-02-25T00:25:40.155143+0000, chain.objs.size()=1 2026-02-24T22:34:25.368 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1607__shadow_.bF--r-P-acSAsI25F6mmYFZTNmE0q6O_0 2026-02-24T22:34:25.368 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8240854284642071816', time=2026-02-25T00:25:45.567021+0000, chain.objs.size()=1 2026-02-24T22:34:25.368 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1615__shadow_.rQUAWWMiNtmHrt0oeDCWyUjDK1hk7DE_0 2026-02-24T22:34:25.368 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7113460888337369506', time=2026-02-25T00:25:52.522456+0000, chain.objs.size()=1 2026-02-24T22:34:25.368 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1627__shadow_.-Ux2Uxb75kHedx0A1y6CnPc1tlNSusj_0 2026-02-24T22:34:25.368 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.241342195529214894', time=2026-02-25T00:25:59.079866+0000, chain.objs.size()=1 2026-02-24T22:34:25.368 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.MekwehxeKcJr1Aek9o3etGePl8C3LFH_0 2026-02-24T22:34:25.368 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17388313153296482019', time=2026-02-25T00:25:59.118963+0000, chain.objs.size()=1 2026-02-24T22:34:25.368 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.cgTTywb2erNCYwqw1LRtheFAASONfzw_0 2026-02-24T22:34:25.368 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15153805982713680382', time=2026-02-25T00:25:59.141198+0000, chain.objs.size()=1 2026-02-24T22:34:25.368 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.U6PskKxQ4oWUe81bXAMCju-wiWpXd5H_0 2026-02-24T22:34:25.369 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11645869878822040538', time=2026-02-25T00:25:59.151356+0000, chain.objs.size()=1 2026-02-24T22:34:25.369 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ygHiky49r6FFP8eQPOZpxhDlStMH1UB_0 2026-02-24T22:34:25.369 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16778265932801484878', time=2026-02-25T00:25:59.160980+0000, chain.objs.size()=1 2026-02-24T22:34:25.369 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.YDBtd9v4w0_sQpJqKDxPiZ8efaOsVll_0 2026-02-24T22:34:25.369 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15561797393854050605', time=2026-02-25T00:25:59.244686+0000, chain.objs.size()=1 2026-02-24T22:34:25.369 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.oJj9SCNWPp4Iw-cFI99e2xIsF7T8N07_0 2026-02-24T22:34:25.369 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13826581340561241506', time=2026-02-25T00:25:59.406855+0000, chain.objs.size()=1 2026-02-24T22:34:25.369 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.lwbjkI2yrbEToZkGs-M2KXVhlPm_FWs_0 2026-02-24T22:34:25.369 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9902167979636181407', time=2026-02-25T00:25:59.410448+0000, chain.objs.size()=1 2026-02-24T22:34:25.369 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.c0dCgWQxQMkg4-lRD_ihJNfLn_bRxAW_0 2026-02-24T22:34:25.369 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1441280532276195006', time=2026-02-25T00:25:59.431723+0000, chain.objs.size()=1 2026-02-24T22:34:25.369 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.af99KmBWXtY4xt_eLGrA_Zrb7F7aDp4_0 2026-02-24T22:34:25.369 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13353668136694579418', time=2026-02-25T00:25:59.434947+0000, chain.objs.size()=1 2026-02-24T22:34:25.369 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.gM24KgJtqUun5WoCzRwqr7EY6xr1fDD_0 2026-02-24T22:34:25.369 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17570204818166381929', time=2026-02-25T00:25:59.446364+0000, chain.objs.size()=1 2026-02-24T22:34:25.369 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.3tSKAAehnVuW38hFv14Z9ddptU0YOE-_0 2026-02-24T22:34:25.369 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10182137622300952145', time=2026-02-25T00:25:59.466047+0000, chain.objs.size()=1 2026-02-24T22:34:25.369 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.EFs6sokMIqrxgTtNp0r7OUFwkXXTJny_0 2026-02-24T22:34:25.369 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2662665099195916239', time=2026-02-25T00:25:59.469664+0000, chain.objs.size()=1 2026-02-24T22:34:25.369 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.3nxHR0kLlPi5dbUrNUcil4NfMFEB0c0_0 2026-02-24T22:34:25.370 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1964254323218341821', time=2026-02-25T00:25:59.470924+0000, chain.objs.size()=1 2026-02-24T22:34:25.370 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.PIrqzWEWoKA5hMsol5r_Uct6eO-Mmxm_0 2026-02-24T22:34:25.370 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1300095649035606262', time=2026-02-25T00:25:59.559872+0000, chain.objs.size()=1 2026-02-24T22:34:25.370 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.DW-vh7ezV78lUjmMACT-FHteZRKymoV_0 2026-02-24T22:34:25.370 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14048013297026822554', time=2026-02-25T00:25:59.579318+0000, chain.objs.size()=1 2026-02-24T22:34:25.370 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.--uCSyZ06jta3S4O3F7DEmYeNa7TFHD_0 2026-02-24T22:34:25.370 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7016232633521808942', time=2026-02-25T00:25:59.584187+0000, chain.objs.size()=1 2026-02-24T22:34:25.370 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ByNw2s0bbTwh_K51xzYf0f4TXp6zQgy_0 2026-02-24T22:34:25.370 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17421679024604326420', time=2026-02-25T00:25:59.586811+0000, chain.objs.size()=1 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.lul2Rfj7bNo6VTnmLG3OIuqVvMTZj9R_0 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7364991895471463816', time=2026-02-25T00:25:59.633546+0000, chain.objs.size()=1 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.VhJNb6D6SMy3YohEGOvWRybyRdVJYSd_0 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16164653028324113495', time=2026-02-25T00:25:59.810222+0000, chain.objs.size()=1 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.j--QFgLUjNGDc3Jf_3I4ut6VkOP-6su_0 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15192876087675128563', time=2026-02-25T00:25:59.825266+0000, chain.objs.size()=1 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.NiP40_YLk6CY2ByJQm16xSk8JHT4zB0_0 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13696699956953218135', time=2026-02-25T00:26:00.064364+0000, chain.objs.size()=1 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ug5CWjCJBpXzdUIrk90cXndKxR3ttbN_0 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3240968270790298563', time=2026-02-25T00:26:00.074189+0000, chain.objs.size()=1 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.-kjZ7D-6LEIXaNRLNhphrVr506oeyZs_0 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6597853934489658525', time=2026-02-25T00:26:00.087099+0000, chain.objs.size()=1 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.EKlC-o-027vU_YUfyrXd8uMMWUO5KYw_0 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17616412443681376439', time=2026-02-25T00:26:00.208133+0000, chain.objs.size()=1 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.sBVxO9np3ks_e9Q6YI5ezWrDWP6AAUh_0 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17309513991449534001', time=2026-02-25T00:26:00.229582+0000, chain.objs.size()=1 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.KlJBzNXbGyFtENxSBuwbz6SdzhVJptR_0 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10580094891261586177', time=2026-02-25T00:26:00.234586+0000, chain.objs.size()=1 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.QOw4eNVcPZ8wEXq9NL7qElJaf_9kfTb_0 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1069622253949760028', time=2026-02-25T00:26:00.250205+0000, chain.objs.size()=1 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.3cytjAFrqwmuFuPEmneHVMqDK8h0FHU_0 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10435634176469782313', time=2026-02-25T00:26:00.265261+0000, chain.objs.size()=1 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.qRXrWYaMnvr7TG76GhQrtyx-khS3B8d_0 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12208160421595180749', time=2026-02-25T00:26:00.268733+0000, chain.objs.size()=1 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.379+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.QLNBbfmoFuQOFwbFJb9yj_7m-lTXrKK_0 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.88243362029723320', time=2026-02-25T00:26:04.278330+0000, chain.objs.size()=1 2026-02-24T22:34:25.371 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1635__shadow_.UaK94_12sFCrQyUIjD_4JqYnn6DNu3O_0 2026-02-24T22:34:25.372 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3862765054679505484', time=2026-02-25T00:26:10.022867+0000, chain.objs.size()=1 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1645__shadow_.OuL4Qd39HHiowPJKc9nWf1mggD-PTgG_0 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6533018174926548358', time=2026-02-25T00:26:11.786130+0000, chain.objs.size()=1 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1648__shadow_.7Ho92ludAeZj4_iG_UPGf9aEcH9MT4a_0 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2188680524533205657', time=2026-02-25T00:26:20.029714+0000, chain.objs.size()=1 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1660__shadow_.huBWaD3DaWNf2E-_ZgH3FtWJOdH6N-w_0 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14736791526294391305', time=2026-02-25T00:26:25.269490+0000, chain.objs.size()=1 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1669__shadow_.sopHqaILdJJrLAqcDPssnlVXuH1Cg26_0 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11886496428415244576', time=2026-02-25T00:26:25.270016+0000, chain.objs.size()=1 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1669__shadow_.zHuZ8bLBgB7XZRiu8EOBubldegyErMD_0 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6915862658558323108', time=2026-02-25T00:26:44.792990+0000, chain.objs.size()=1 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.7WBiQfD1qi6N5wiPXHrg2xKIopcWg5z_0 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4590621341102716681', time=2026-02-25T00:26:44.796880+0000, chain.objs.size()=1 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.xfYR1iu6G_dk9Id1DVHbetFB3uTUzHZ_0 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14244334361041375856', time=2026-02-25T00:26:44.807586+0000, chain.objs.size()=1 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ENzIzUPMyuRuM6Z0pVHwrQLMhUZZugi_0 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10602833010461926162', time=2026-02-25T00:26:44.862042+0000, chain.objs.size()=1 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.W0ossZcfmMeVaar9DpYKdXuYqMKcQXu_0 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14891275117478571728', time=2026-02-25T00:26:44.980146+0000, chain.objs.size()=1 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.oqUiw5sKOFQcmylGWup-C2dHtwT_tP3_0 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14341456512154554596', time=2026-02-25T00:26:45.014339+0000, chain.objs.size()=1 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.fkPk7Tpd449-0tS3soVugeVY5mQ23NA_0 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13548718833931090628', time=2026-02-25T00:26:45.022416+0000, chain.objs.size()=1 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.xA6dETdNrlsx0UYm4hzJVtmO3TtQyoi_0 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17068675610264128784', time=2026-02-25T00:26:45.106298+0000, chain.objs.size()=1 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.mIdKahQegXV5sNMkr_MxTnC0Qkvehx7_0 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14420316914271920815', time=2026-02-25T00:26:45.106680+0000, chain.objs.size()=1 2026-02-24T22:34:25.373 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.5TSbC5WAKxY79-A1_dpRBq106u4oUF9_0 2026-02-24T22:34:25.374 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13649023692385840596', time=2026-02-25T00:26:45.164167+0000, chain.objs.size()=1 2026-02-24T22:34:25.374 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.pfjDC2_uelfTE6fbIZFUG7KE5Bz-KAH_0 2026-02-24T22:34:25.374 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7348626174625722637', time=2026-02-25T00:26:45.258646+0000, chain.objs.size()=1 2026-02-24T22:34:25.374 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.HUI2xgVg5eqJQVjdgBHr-ywBo2GXrlx_0 2026-02-24T22:34:25.374 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6539137979982847592', time=2026-02-25T00:26:45.262840+0000, chain.objs.size()=1 2026-02-24T22:34:25.374 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.sCQJMeGZc1cRTSi4R_VhKw6mikXrYjD_0 2026-02-24T22:34:25.374 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5034360143570964135', time=2026-02-25T00:26:45.268213+0000, chain.objs.size()=1 2026-02-24T22:34:25.374 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.z8Lpu2XsYNbhW6IdTTMrjBbxX-XOOdf_0 2026-02-24T22:34:25.374 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11986382867881698412', time=2026-02-25T00:26:45.415338+0000, chain.objs.size()=1 2026-02-24T22:34:25.374 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.3O5XTYCv9_Ww-Hjs_GrOUfCiCh6Bv0-_0 2026-02-24T22:34:25.374 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1358626308613313737', time=2026-02-25T00:26:45.424774+0000, chain.objs.size()=1 2026-02-24T22:34:25.374 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.fpmXGBVI1lszwFeWfD7j8CXIgJAKqOh_0 2026-02-24T22:34:25.374 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18232065516539241330', time=2026-02-25T00:26:45.429017+0000, chain.objs.size()=1 2026-02-24T22:34:25.374 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.SHYg-PNLtyMygL0T4jg8y9gvACvOva4_0 2026-02-24T22:34:25.374 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9671700649091614445', time=2026-02-25T00:26:45.587652+0000, chain.objs.size()=1 2026-02-24T22:34:25.374 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.egRcT0f5CnQaJ40vGSJnZwC9B1Ux4tH_0 2026-02-24T22:34:25.375 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16124316441412925893', time=2026-02-25T00:26:45.834479+0000, chain.objs.size()=1 2026-02-24T22:34:25.375 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.8agdCYA-LxzXHFTgFbkIz1uqYwD118S_0 2026-02-24T22:34:25.375 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4931001133594277684', time=2026-02-25T00:26:45.840450+0000, chain.objs.size()=1 2026-02-24T22:34:25.375 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.9Hc8cqOlEc-swWLml_SJnm1XXUcws9O_0 2026-02-24T22:34:25.375 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9856344327202200103', time=2026-02-25T00:26:45.841395+0000, chain.objs.size()=1 2026-02-24T22:34:25.375 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.DRooxD49t5tna7HmIjMCMXQNpcLfewI_0 2026-02-24T22:34:25.375 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2107091572765103168', time=2026-02-25T00:26:45.938230+0000, chain.objs.size()=1 2026-02-24T22:34:25.375 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.O3oQ0lT4lsEw0wuS6RICsxKZPrtTizU_0 2026-02-24T22:34:25.375 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16480037685081581617', time=2026-02-25T00:26:45.975790+0000, chain.objs.size()=1 2026-02-24T22:34:25.375 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Vpg96LS3MjucqRKBKY3STCSeBGssq9K_0 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16665049754009893513', time=2026-02-25T00:26:45.977852+0000, chain.objs.size()=1 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.3bUm3WD007fX9tUQuLl8nTUIC6D87dk_0 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.889748567338155971', time=2026-02-25T00:26:46.012046+0000, chain.objs.size()=1 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.RzS__3DxPJNM6Q_UzFacd6tDyLvGK65_0 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.255838255235446665', time=2026-02-25T00:26:46.087167+0000, chain.objs.size()=1 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.YuVmguCcHMyS2aXH320dVyM8z7BewQU_0 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.428969827666929371', time=2026-02-25T00:26:46.100943+0000, chain.objs.size()=1 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.dJMj1N8F4AF86Qi03pUyXDfn5o1Abyw_0 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10313350174867343397', time=2026-02-25T00:26:46.101509+0000, chain.objs.size()=1 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.HULgBvN6RHBS3oPxYEJqv9gbtkzzzpl_0 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6319463127715532864', time=2026-02-25T00:26:46.134931+0000, chain.objs.size()=1 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.3vy1e0lDgoEYepqhaJbRwSuF-aYZ0LI_0 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2164395644509003409', time=2026-02-25T00:26:46.140099+0000, chain.objs.size()=1 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Rfs9aRpiP2_seY-ozSksK96ZvN-fW_R_0 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8881873841159862800', time=2026-02-25T00:26:46.145661+0000, chain.objs.size()=1 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.5ypHstUwb7aTaieopx1eemY1gTaJPPT_0 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9377690060049165285', time=2026-02-25T00:26:53.139816+0000, chain.objs.size()=1 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.383+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.rPT51VQFeSFFS8cTAJik0o7g2V7uDUW_0 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18124597655290569735', time=2026-02-25T00:26:53.240917+0000, chain.objs.size()=1 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.BWTiHaFtbNFnTxxWqOUDNeYKxaNA3lW_0 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12560433228363711478', time=2026-02-25T00:26:53.351614+0000, chain.objs.size()=1 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.bpzP2U_g1k9al_gb9HJhXiQOytV4udC_0 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17615517230199987082', time=2026-02-25T00:26:53.376495+0000, chain.objs.size()=1 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.PIth1vP7E3OnZ2B5DXZxDN-hNniAiCS_0 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11968919714174394325', time=2026-02-25T00:26:53.390773+0000, chain.objs.size()=1 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.5i8ay5-M3j09YISUgNo5NYBXsqxGtw9_0 2026-02-24T22:34:25.376 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5695807763789170821', time=2026-02-25T00:26:53.392083+0000, chain.objs.size()=1 2026-02-24T22:34:25.377 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.GXBlOdncPetKHc2UtEplOr75J_EL-NX_0 2026-02-24T22:34:25.377 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12946701491295426683', time=2026-02-25T00:26:53.523457+0000, chain.objs.size()=1 2026-02-24T22:34:25.377 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.tJyrb7YmXf_g8PTySq962mnxFoOZlSa_0 2026-02-24T22:34:25.377 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3135505281614288', time=2026-02-25T00:26:53.543549+0000, chain.objs.size()=1 2026-02-24T22:34:25.377 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Zf1NOZ1FuC8oETuWDogJY1auDMKR3qJ_0 2026-02-24T22:34:25.377 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8794494721359995671', time=2026-02-25T00:26:53.561811+0000, chain.objs.size()=1 2026-02-24T22:34:25.377 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.XNiQj5NaCVjmJ4pLeMI51hwuWhzAjSF_0 2026-02-24T22:34:25.377 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5963636815439713662', time=2026-02-25T00:26:53.606246+0000, chain.objs.size()=1 2026-02-24T22:34:25.377 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.n9pjzj_JLCcSYjKbhtE_MfRM-aZDQT1_0 2026-02-24T22:34:25.377 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3244039271600789284', time=2026-02-25T00:26:53.606742+0000, chain.objs.size()=1 2026-02-24T22:34:25.377 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.lJAPLXG2va2UW9JZ5Bn0UygIwKal7ix_0 2026-02-24T22:34:25.377 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14651859135365795544', time=2026-02-25T00:26:53.618122+0000, chain.objs.size()=1 2026-02-24T22:34:25.377 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.8D5-PKRkLIrkDfZ-W8fOKRQvlwZJyAX_0 2026-02-24T22:34:25.377 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1069137202317408212', time=2026-02-25T00:26:53.741068+0000, chain.objs.size()=1 2026-02-24T22:34:25.377 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.OX-fU40XA_zTZ_aQbIRRCR6BnFzZpnf_0 2026-02-24T22:34:25.377 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16150163007389155560', time=2026-02-25T00:26:53.749976+0000, chain.objs.size()=1 2026-02-24T22:34:25.377 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Al5yAnw0sP_dk_AUy8y8-gUp4QoJvF0_0 2026-02-24T22:34:25.377 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.387+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/39060 2026-02-24T22:34:25.379 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=40, truncated=0, next_marker='' 2026-02-24T22:34:25.379 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4780489202763142953', time=2026-02-25T00:26:53.750424+0000, chain.objs.size()=1 2026-02-24T22:34:25.379 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_._DQYLn1XDLMqOdMDHFd-P4g1PAFgI0q_0 2026-02-24T22:34:25.379 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8457593450102439353', time=2026-02-25T00:26:53.815688+0000, chain.objs.size()=1 2026-02-24T22:34:25.379 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.jkNalRRKWEXDFB2ajjBfSymazSMlOXM_0 2026-02-24T22:34:25.380 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9817999411668775910', time=2026-02-25T00:26:53.821918+0000, chain.objs.size()=1 2026-02-24T22:34:25.380 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.YBvoKc-w4YMohNJiAjI6f40QlU4dnQW_0 2026-02-24T22:34:25.380 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15782711555747872945', time=2026-02-25T00:26:53.831159+0000, chain.objs.size()=1 2026-02-24T22:34:25.380 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.k9u11b9mT-QOWLmOfX-7GjwHHIug_Hr_0 2026-02-24T22:34:25.380 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13072560658094739836', time=2026-02-25T00:26:53.831562+0000, chain.objs.size()=1 2026-02-24T22:34:25.380 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.x7QuBVLinnUr6kEw-v1hU6qXyCcigUm_0 2026-02-24T22:34:25.380 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.603994262832815697', time=2026-02-25T00:26:53.915420+0000, chain.objs.size()=1 2026-02-24T22:34:25.380 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.RKmnHUDxbB6rS6DTkQU6Cz7Hc7fiB___0 2026-02-24T22:34:25.380 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18363656394260471204', time=2026-02-25T00:26:53.924900+0000, chain.objs.size()=1 2026-02-24T22:34:25.380 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.RK6-62KxzkzK3nBZpnUCgppNRHvy8WB_0 2026-02-24T22:34:25.380 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16032985489131712427', time=2026-02-25T00:26:53.939397+0000, chain.objs.size()=1 2026-02-24T22:34:25.380 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.L2EqhLkZOwu_ZuJNqkmuZxKi5ZWQPBq_0 2026-02-24T22:34:25.380 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7327634868801478195', time=2026-02-25T00:26:53.965998+0000, chain.objs.size()=1 2026-02-24T22:34:25.380 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.o4nhZieAS4q_lon6waEt93KBkYiHll4_0 2026-02-24T22:34:25.380 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17772303614037576548', time=2026-02-25T00:26:54.007449+0000, chain.objs.size()=1 2026-02-24T22:34:25.380 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Y9o_wo5wxaLpFeKkMO3cyL9wZLthMI2_0 2026-02-24T22:34:25.380 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10001507435704719360', time=2026-02-25T00:26:54.118667+0000, chain.objs.size()=1 2026-02-24T22:34:25.380 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.E_cWKXfWw2g2XmcDQE0wuYVpN0tDQVt_0 2026-02-24T22:34:25.380 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14684425747807491534', time=2026-02-25T00:26:54.282586+0000, chain.objs.size()=1 2026-02-24T22:34:25.380 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.bQSIyAFYdOuVjenaEcQjob7eqAb0cDO_0 2026-02-24T22:34:25.381 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16548937599989748639', time=2026-02-25T00:26:54.290542+0000, chain.objs.size()=1 2026-02-24T22:34:25.381 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.-iZWRh0D_pcY6iP96cpOMwxnKAp92uE_0 2026-02-24T22:34:25.381 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16806299152459108089', time=2026-02-25T00:26:54.306299+0000, chain.objs.size()=1 2026-02-24T22:34:25.381 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.PLtJDLfUFOO_rfGmcE9pthfImAukSQN_0 2026-02-24T22:34:25.381 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7836187528475698862', time=2026-02-25T00:26:54.313638+0000, chain.objs.size()=1 2026-02-24T22:34:25.381 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.SJAKck5bNlYhiJQKnVwn--8BC6M7TIT_0 2026-02-24T22:34:25.381 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3128952426639493313', time=2026-02-25T00:26:54.318099+0000, chain.objs.size()=1 2026-02-24T22:34:25.381 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.WgEGaDn9eKRchvNdXbgZGUF1jLnM79s_0 2026-02-24T22:34:25.381 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18393778893486430774', time=2026-02-25T00:26:54.323988+0000, chain.objs.size()=1 2026-02-24T22:34:25.381 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.80aRcVjHgy1cRFwPXm-SH__2K7QatPf_0 2026-02-24T22:34:25.381 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13896025230219206565', time=2026-02-25T00:26:54.340795+0000, chain.objs.size()=1 2026-02-24T22:34:25.381 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.4S1dRKbA2CpGlv3U3Rj7iu2zP5EdBUb_0 2026-02-24T22:34:25.381 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10351194042037558183', time=2026-02-25T00:26:54.382774+0000, chain.objs.size()=1 2026-02-24T22:34:25.381 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.70VkhFsio0JapAoLZ_vOUJ-XsqV5ulz_0 2026-02-24T22:34:25.381 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11314948691946293562', time=2026-02-25T00:26:54.469590+0000, chain.objs.size()=1 2026-02-24T22:34:25.381 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.XPBUzzjSW3un4BRbjPFnI3p7FBo7-TC_0 2026-02-24T22:34:25.381 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13869004715940256794', time=2026-02-25T00:26:54.526378+0000, chain.objs.size()=1 2026-02-24T22:34:25.381 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.xnWXyC4c8BmI62iGZKaL5IKIMgUc-jp_0 2026-02-24T22:34:25.381 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6339510111018028745', time=2026-02-25T00:27:20.376560+0000, chain.objs.size()=1 2026-02-24T22:34:25.381 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1737__shadow_.JDfDujMy1MrIiOCgoq2xq4Ak-HY7CET_0 2026-02-24T22:34:25.381 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7579126643249779842', time=2026-02-25T00:27:25.649006+0000, chain.objs.size()=1 2026-02-24T22:34:25.381 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1744__shadow_.v1-DZ-cAjs8McATMyLfaXHkv4X-zXFu_0 2026-02-24T22:34:25.382 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17653510541926742944', time=2026-02-25T00:27:44.932206+0000, chain.objs.size()=1 2026-02-24T22:34:25.382 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1773__shadow_.Q-QgoGYEmnv59Wk6539Bnl1lCQ8BoJq_0 2026-02-24T22:34:25.382 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8314928934754631867', time=2026-02-25T00:28:02.806471+0000, chain.objs.size()=1 2026-02-24T22:34:25.382 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1800__shadow_.nK_SwnSql6hms-tDQnyi0D8YtQL_JDH_0 2026-02-24T22:34:25.382 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4047904405959683496', time=2026-02-25T00:28:17.888301+0000, chain.objs.size()=1 2026-02-24T22:34:25.382 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1822__shadow_.WBtugYTLjHXhrZnYOjIJyUxnBhyx5oX_0 2026-02-24T22:34:25.382 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9741446722128339751', time=2026-02-25T00:28:32.541928+0000, chain.objs.size()=1 2026-02-24T22:34:25.382 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1849__shadow_.OLIkKPVRNkPy2RFcS1_31oGXam6DpoR_0 2026-02-24T22:34:25.382 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8145777372157939311', time=2026-02-25T00:28:39.079533+0000, chain.objs.size()=1 2026-02-24T22:34:25.382 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1861__shadow_.drMWqH-tPXns6x4CaZzP8HfQIavyVlP_0 2026-02-24T22:34:25.382 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18291952889593221927', time=2026-02-25T00:28:46.323853+0000, chain.objs.size()=6 2026-02-24T22:34:25.382 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.1 2026-02-24T22:34:25.382 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.2 2026-02-24T22:34:25.382 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.3 2026-02-24T22:34:25.382 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.4 2026-02-24T22:34:25.382 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.5 2026-02-24T22:34:25.382 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.6 2026-02-24T22:34:25.383 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2940588053668818161', time=2026-02-25T00:28:46.421935+0000, chain.objs.size()=6 2026-02-24T22:34:25.383 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.391+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.1 2026-02-24T22:34:25.383 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.395+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.2 2026-02-24T22:34:25.383 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.395+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.3 2026-02-24T22:34:25.383 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.395+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.4 2026-02-24T22:34:25.383 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.395+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.5 2026-02-24T22:34:25.383 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.395+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.6 2026-02-24T22:34:25.383 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.395+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4141866441436056568', time=2026-02-25T00:30:21.782954+0000, chain.objs.size()=1 2026-02-24T22:34:25.383 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.395+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:F34TyL19j4cMV6sghJdJ-2.80Wol5pr_.ZX_hVl4EmXjddnCNhylznBytJXN-zXW_0 2026-02-24T22:34:25.383 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.395+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7912568718629941635', time=2026-02-25T00:30:44.171096+0000, chain.objs.size()=1 2026-02-24T22:34:25.383 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.395+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:aKxyt-XwkweAqyXXt6UIPF.-gsxrnRq_.-ncLQtdWl_OosccQGZLH2XCv01Vz9Ij_0 2026-02-24T22:34:25.383 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.395+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18231408153765541700', time=2026-02-25T00:30:50.998991+0000, chain.objs.size()=1 2026-02-24T22:34:25.383 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.395+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1949__shadow:67E2aE7ousDwtZEVAhUIZ-cTcBBKDb7_.c_-sznSRqsnvBn3_Z276lac0eY9s0L1_0 2026-02-24T22:34:25.383 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.395+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8801268253471744412', time=2026-02-25T00:30:51.001927+0000, chain.objs.size()=1 2026-02-24T22:34:25.383 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.395+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1949__shadow:VaHdGE4Fqd4OfHCgL216lWVD7.sbcx4_.8aUF-kCTeWGnKg6d0A56n73KxAoDCNk_0 2026-02-24T22:34:25.383 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.395+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6850748817014036712', time=2026-02-25T00:30:57.363698+0000, chain.objs.size()=1 2026-02-24T22:34:25.383 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.395+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:H9PMIu-CkofLVSr47w0nsThrrRmsPbK_.91n91McLfj7csNkHVj8r-Ru-Vhg-Zlk_0 2026-02-24T22:34:25.384 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.395+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6850748817014036712', time=2026-02-25T00:30:57.364456+0000, chain.objs.size()=1 2026-02-24T22:34:25.384 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.395+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:H9PMIu-CkofLVSr47w0nsThrrRmsPbK_.91n91McLfj7csNkHVj8r-Ru-Vhg-Zlk_0 2026-02-24T22:34:25.384 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.395+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6850748817014036712', time=2026-02-25T00:30:57.365729+0000, chain.objs.size()=1 2026-02-24T22:34:25.384 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.395+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:H9PMIu-CkofLVSr47w0nsThrrRmsPbK_.91n91McLfj7csNkHVj8r-Ru-Vhg-Zlk_0 2026-02-24T22:34:25.384 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.395+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6850748817014036712', time=2026-02-25T00:30:57.366158+0000, chain.objs.size()=1 2026-02-24T22:34:25.384 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.395+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:H9PMIu-CkofLVSr47w0nsThrrRmsPbK_.91n91McLfj7csNkHVj8r-Ru-Vhg-Zlk_0 2026-02-24T22:34:25.384 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.395+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4603059517697218147', time=2026-02-25T00:30:58.452531+0000, chain.objs.size()=1 2026-02-24T22:34:25.384 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.395+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958__shadow:Drdd4nDpB6dw4e62zpWz1o2dXveg9Lr_.vuQ1CK_ZG2WJRFCpQZn3p0VI0bCMVPT_0 2026-02-24T22:34:25.384 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.395+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4603059517697218147', time=2026-02-25T00:30:58.453569+0000, chain.objs.size()=1 2026-02-24T22:34:25.384 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.395+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958__shadow:Drdd4nDpB6dw4e62zpWz1o2dXveg9Lr_.vuQ1CK_ZG2WJRFCpQZn3p0VI0bCMVPT_0 2026-02-24T22:34:25.385 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.395+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.386 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2026-02-24T22:34:25.387 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.387 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.388 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/38095' 2026-02-24T22:34:25.388 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2086538704928360918', time=2026-02-25T00:15:09.928707+0000, chain.objs.size()=1 2026-02-24T22:34:25.388 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1505__shadow_.SJ3vq5X9CX-y9h8fW259fanrgvN5rJi_0 2026-02-24T22:34:25.388 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='_Ifx-HNEmzwwadjGmce14gvFB1L2Q25d', time=2026-02-25T00:15:34.774960+0000, chain.objs.size()=2 2026-02-24T22:34:25.388 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__shadow_.KnK3Gb3aUR1Rg2m1qiZ5r5bTDgHzkTU_0 2026-02-24T22:34:25.388 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__shadow_.KnK3Gb3aUR1Rg2m1qiZ5r5bTDgHzkTU_1 2026-02-24T22:34:25.388 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.514453387471807106', time=2026-02-25T00:16:32.618975+0000, chain.objs.size()=1 2026-02-24T22:34:25.388 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520__shadow_.eKtz9W66JCt9cYITk3wxTF4MC-4l55A_0 2026-02-24T22:34:25.389 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9335269800693307301', time=2026-02-25T00:17:05.172379+0000, chain.objs.size()=1 2026-02-24T22:34:25.389 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522__shadow_.mI0dUBhyQuX62RPj9A4HPHND6xBqY5Z_0 2026-02-24T22:34:25.389 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11427274566442514322', time=2026-02-25T00:17:05.173343+0000, chain.objs.size()=1 2026-02-24T22:34:25.389 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522__shadow_.Sv3L9y3akF4GgD59VgZGynY1uOM329N_0 2026-02-24T22:34:25.389 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13910315400466990608', time=2026-02-25T00:17:08.014562+0000, chain.objs.size()=1 2026-02-24T22:34:25.389 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524__shadow_.gldy1znqQu-JGvfu8fDwDhj0GkzMjvm_0 2026-02-24T22:34:25.389 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5129174603582200539', time=2026-02-25T00:17:08.019047+0000, chain.objs.size()=1 2026-02-24T22:34:25.389 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524__shadow_.4I_ijeCUBedVLYj8kaioHMhfnk39jS9_0 2026-02-24T22:34:25.389 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3377288745526989108', time=2026-02-25T00:17:42.543077+0000, chain.objs.size()=1 2026-02-24T22:34:25.389 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527__shadow_.m13f-7-eabQVfJtAS56KgQ9_8UP2GBA_0 2026-02-24T22:34:25.389 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7704321324016373958', time=2026-02-25T00:18:14.581988+0000, chain.objs.size()=1 2026-02-24T22:34:25.389 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528__shadow_.DNdoYNICiCeqFot7FN9B9PUXzkUKIPK_0 2026-02-24T22:34:25.389 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14790526417925179820', time=2026-02-25T00:18:48.195190+0000, chain.objs.size()=1 2026-02-24T22:34:25.389 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530__shadow_.VylRrAMaIq6kfw0Yh_rpNM4E1Ti7Dws_0 2026-02-24T22:34:25.389 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16962262455208007209', time=2026-02-25T00:18:49.776610+0000, chain.objs.size()=1 2026-02-24T22:34:25.389 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531__shadow_.ReIBeEu_hArP94bw9kyppNBq-m8ct2V_0 2026-02-24T22:34:25.389 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.867065329659392854', time=2026-02-25T00:19:23.071473+0000, chain.objs.size()=1 2026-02-24T22:34:25.389 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533__shadow_.gWaWxGtgq6gHHEO-nfjAGBUw1yBhHVk_0 2026-02-24T22:34:25.390 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15524573778502223962', time=2026-02-25T00:19:25.033182+0000, chain.objs.size()=1 2026-02-24T22:34:25.390 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.399+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534__shadow_.KSMDgpkpXdYcxKQ48g6lihAHXQXvhz1_0 2026-02-24T22:34:25.390 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7401013123172088082', time=2026-02-25T00:19:57.118165+0000, chain.objs.size()=1 2026-02-24T22:34:25.390 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535__shadow_.oqwltAQVR8l8qijt5ufJsdwx9zBgpBI_0 2026-02-24T22:34:25.390 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12618967096307112541', time=2026-02-25T00:19:58.467788+0000, chain.objs.size()=1 2026-02-24T22:34:25.390 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536__shadow_.MATR7qkRC1KaAd5mFaVhAu40aAJI6Ry_0 2026-02-24T22:34:25.390 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3436242643084009247', time=2026-02-25T00:21:06.327551+0000, chain.objs.size()=1 2026-02-24T22:34:25.390 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540__shadow_.dVNRyAT8XzVavWQqtUjgTCZuGxqA6-K_0 2026-02-24T22:34:25.390 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2798795359079982309', time=2026-02-25T00:25:41.727555+0000, chain.objs.size()=1 2026-02-24T22:34:25.391 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1609__shadow_.Re2_PRSRRuziTbp2Ino7XodXcjBFol9_0 2026-02-24T22:34:25.391 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8213884891611405710', time=2026-02-25T00:25:45.021502+0000, chain.objs.size()=1 2026-02-24T22:34:25.391 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1614__shadow_.4vuc4w1M_hnoBSHzloP6xfooYeFyEHc_0 2026-02-24T22:34:25.391 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9842734787895540751', time=2026-02-25T00:25:51.409415+0000, chain.objs.size()=1 2026-02-24T22:34:25.391 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1625__shadow_.SA77se1y2AcfWXIrItHgPkWzzHJc-2w_0 2026-02-24T22:34:25.391 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1182040790943027285', time=2026-02-25T00:25:59.103590+0000, chain.objs.size()=1 2026-02-24T22:34:25.391 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.MDeXDYDUIC1PmIpjxMqIRchRcek4g1d_0 2026-02-24T22:34:25.391 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9179294246454307009', time=2026-02-25T00:25:59.145703+0000, chain.objs.size()=1 2026-02-24T22:34:25.391 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.YX8fshFIT9NUCU3i4RJAEuxeGjMz-eF_0 2026-02-24T22:34:25.391 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13106461010052845948', time=2026-02-25T00:25:59.274914+0000, chain.objs.size()=1 2026-02-24T22:34:25.391 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.3tGo5tT4XRQiodr3r3LZYs4K7klaz-l_0 2026-02-24T22:34:25.391 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8439249421774270590', time=2026-02-25T00:25:59.286677+0000, chain.objs.size()=1 2026-02-24T22:34:25.391 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.8xQRQbCF8RPx6cYMA6CLhcr9i612X7I_0 2026-02-24T22:34:25.392 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14819670384947174539', time=2026-02-25T00:25:59.287668+0000, chain.objs.size()=1 2026-02-24T22:34:25.392 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.1Q_rFRL5M2v4KrckesBjUDVp4Sau4iE_0 2026-02-24T22:34:25.392 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8862030490292838544', time=2026-02-25T00:25:59.329768+0000, chain.objs.size()=1 2026-02-24T22:34:25.392 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.uew0uqB6Ube9Bcymj9SAaaC-uPlxYiY_0 2026-02-24T22:34:25.392 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7304223411571551907', time=2026-02-25T00:25:59.335948+0000, chain.objs.size()=1 2026-02-24T22:34:25.392 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.FkV7g12werHTu2o5GuQuZZbvmukMz99_0 2026-02-24T22:34:25.392 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11075479840519864971', time=2026-02-25T00:25:59.340294+0000, chain.objs.size()=1 2026-02-24T22:34:25.392 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Wa9k4wJwDmP7U_rB62hwhYkFn7ogCve_0 2026-02-24T22:34:25.392 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15001253028796871849', time=2026-02-25T00:25:59.426928+0000, chain.objs.size()=1 2026-02-24T22:34:25.392 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Zf-pgEYnXHYAU_W71vqgwDn7wnQ_v18_0 2026-02-24T22:34:25.392 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3078533219873326692', time=2026-02-25T00:25:59.441630+0000, chain.objs.size()=1 2026-02-24T22:34:25.392 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.fTfyOiZXwPUYe2xKpWKhXp4cY0CjL2y_0 2026-02-24T22:34:25.392 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13854347071857439944', time=2026-02-25T00:25:59.457167+0000, chain.objs.size()=1 2026-02-24T22:34:25.392 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.9YOLFdLxSflTF8TkVciDjEFzCyt8kQM_0 2026-02-24T22:34:25.392 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4844365237931238837', time=2026-02-25T00:25:59.495328+0000, chain.objs.size()=1 2026-02-24T22:34:25.392 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.C7oJBTmiU4y3EB41CFfLB1fPsu-Rtbz_0 2026-02-24T22:34:25.392 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4628703522840735849', time=2026-02-25T00:25:59.763492+0000, chain.objs.size()=1 2026-02-24T22:34:25.392 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.o3s6TtsWK5qa-UPRqbM0HMm0lkRU5Qb_0 2026-02-24T22:34:25.392 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16663309582852715905', time=2026-02-25T00:25:59.768234+0000, chain.objs.size()=1 2026-02-24T22:34:25.392 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Vqdhad_FhiQDGlOsWkzZdJJ9EZx8QJR_0 2026-02-24T22:34:25.392 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8166737588618142478', time=2026-02-25T00:25:59.828221+0000, chain.objs.size()=1 2026-02-24T22:34:25.392 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.6phPIaD9-1AUjk22VlfJzXfmTzfBk9z_0 2026-02-24T22:34:25.393 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10106583956413803363', time=2026-02-25T00:25:59.918212+0000, chain.objs.size()=1 2026-02-24T22:34:25.393 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.cf3Cx_pSkbAvclIgAbtl9cuJ2mhSD4A_0 2026-02-24T22:34:25.393 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9057086165435383002', time=2026-02-25T00:25:59.921345+0000, chain.objs.size()=1 2026-02-24T22:34:25.393 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.vpzFREeWHnnF0-3Y_Jiz7qTBTGszvFH_0 2026-02-24T22:34:25.393 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5817816339868477063', time=2026-02-25T00:25:59.934174+0000, chain.objs.size()=1 2026-02-24T22:34:25.393 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.mbTlEEpJtKAYdZ3zttNHBA3xaCbFhyX_0 2026-02-24T22:34:25.393 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17866218391618980902', time=2026-02-25T00:25:59.936683+0000, chain.objs.size()=1 2026-02-24T22:34:25.393 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.kp016FWeX_GdFD4LleeP2DAL3pD51NX_0 2026-02-24T22:34:25.393 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1349127705801562586', time=2026-02-25T00:26:00.072656+0000, chain.objs.size()=1 2026-02-24T22:34:25.393 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.o6ZuGTNc1vKGMvfWbph-vZKy-vR01P6_0 2026-02-24T22:34:25.393 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14022329767137484194', time=2026-02-25T00:26:00.087021+0000, chain.objs.size()=1 2026-02-24T22:34:25.393 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Ul-HF2sW792u7WqOtgl-HkM6V-08brm_0 2026-02-24T22:34:25.393 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6537720363393727390', time=2026-02-25T00:26:00.111702+0000, chain.objs.size()=1 2026-02-24T22:34:25.393 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.KXXi5myqreqBI7t4TiLbrcm4MWxUlnN_0 2026-02-24T22:34:25.393 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11454662417116467908', time=2026-02-25T00:26:00.138546+0000, chain.objs.size()=1 2026-02-24T22:34:25.393 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.C7vmEhi0AwFc_AXUUf4z2W0uA1ww7FR_0 2026-02-24T22:34:25.393 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16891358686152023715', time=2026-02-25T00:26:00.233630+0000, chain.objs.size()=1 2026-02-24T22:34:25.394 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.WFwFjZJ_st34LoBG6IwTdnFhYIlBUZH_0 2026-02-24T22:34:25.394 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10524486157307906263', time=2026-02-25T00:26:00.250796+0000, chain.objs.size()=1 2026-02-24T22:34:25.394 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.uqmU7d7K-DsY2439Dsl9Eo-1OqPSoV5_0 2026-02-24T22:34:25.394 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10044968025885094982', time=2026-02-25T00:26:00.285060+0000, chain.objs.size()=1 2026-02-24T22:34:25.394 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.YopcoGy4hFoutGpAz1TlEC4Jpt-lJR1_0 2026-02-24T22:34:25.394 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7725951339425400078', time=2026-02-25T00:26:01.385545+0000, chain.objs.size()=1 2026-02-24T22:34:25.394 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1630__shadow_.ggbU7e_bXMJgt2BjdbJXvXMfFFExhnu_0 2026-02-24T22:34:25.394 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.799816518327115265', time=2026-02-25T00:26:07.113206+0000, chain.objs.size()=1 2026-02-24T22:34:25.394 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1640__shadow_.Jezp6gMonjZ9AIkqVGqc_jcfgubrhIm_0 2026-02-24T22:34:25.394 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5912305507248595292', time=2026-02-25T00:26:12.953926+0000, chain.objs.size()=1 2026-02-24T22:34:25.394 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1650__shadow_.C1MinjIpwKydp5t0Nk57CyPLtVXyIaj_0 2026-02-24T22:34:25.394 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11570255019909500327', time=2026-02-25T00:26:17.612398+0000, chain.objs.size()=1 2026-02-24T22:34:25.394 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1657__shadow_.0e-L49Tmr-g8_uiQHzfSm029moFbzne_0 2026-02-24T22:34:25.394 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13713586448806345879', time=2026-02-25T00:26:19.470143+0000, chain.objs.size()=1 2026-02-24T22:34:25.394 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1659__shadow_.LTSbP6T07vJyeWCe_lGDHM_0jAG6djy_0 2026-02-24T22:34:25.394 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13423390847073859731', time=2026-02-25T00:26:24.066788+0000, chain.objs.size()=1 2026-02-24T22:34:25.395 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1667__shadow_.pqth4pCSor9JAzx5HbSll1HDX0txvFz_0 2026-02-24T22:34:25.395 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13029183951429575653', time=2026-02-25T00:26:26.429856+0000, chain.objs.size()=1 2026-02-24T22:34:25.395 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1671__shadow_.SNqFxnzNgTFY_5-RUf5dGduDaC6xAIr_0 2026-02-24T22:34:25.395 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5395564232924023416', time=2026-02-25T00:26:44.793583+0000, chain.objs.size()=1 2026-02-24T22:34:25.395 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.LV4WJAk7EhnKX5SntnSypxR5r6Op8oI_0 2026-02-24T22:34:25.395 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16876102366187870013', time=2026-02-25T00:26:44.833044+0000, chain.objs.size()=1 2026-02-24T22:34:25.395 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.FWpyjPjyE2p2_EZYVtQ0IJHCybSlHKD_0 2026-02-24T22:34:25.395 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17310694474596115778', time=2026-02-25T00:26:44.864240+0000, chain.objs.size()=1 2026-02-24T22:34:25.395 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.XtWXmKMrJaAo0dQn8rQKA2dBqCFtFKe_0 2026-02-24T22:34:25.395 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9570924070687341691', time=2026-02-25T00:26:44.963867+0000, chain.objs.size()=1 2026-02-24T22:34:25.395 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.2uQsZdNF--SzlLDrT_AuQQdXeTPb6dN_0 2026-02-24T22:34:25.395 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1946647091558145511', time=2026-02-25T00:26:44.975839+0000, chain.objs.size()=1 2026-02-24T22:34:25.395 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Zgxst58IZTDzBlRJvKB98fsqXRQzROa_0 2026-02-24T22:34:25.395 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13261552992410395789', time=2026-02-25T00:26:44.977023+0000, chain.objs.size()=1 2026-02-24T22:34:25.395 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.403+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.gEX1daFRDAUlt_O1Fgf2LtyO7GAYBaf_0 2026-02-24T22:34:25.395 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12833267165112818672', time=2026-02-25T00:26:44.978353+0000, chain.objs.size()=1 2026-02-24T22:34:25.395 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.8WUSgRGOnfd4DHXNcqS_cyf_ODf_wvm_0 2026-02-24T22:34:25.396 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10280740717644109604', time=2026-02-25T00:26:44.978741+0000, chain.objs.size()=1 2026-02-24T22:34:25.396 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.T9pbHoEzxjxAQRHPnLmv4lIa1iIKir7_0 2026-02-24T22:34:25.396 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8053257071848274084', time=2026-02-25T00:26:45.012578+0000, chain.objs.size()=1 2026-02-24T22:34:25.396 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.hI7mHIRreKNadS35uag4sfdjG6G7fg6_0 2026-02-24T22:34:25.396 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14317200576141457882', time=2026-02-25T00:26:45.190006+0000, chain.objs.size()=1 2026-02-24T22:34:25.396 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.sviC-8LO_oOq32drW2lVHV-WWJZT2QM_0 2026-02-24T22:34:25.396 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17876795172803216557', time=2026-02-25T00:26:45.194180+0000, chain.objs.size()=1 2026-02-24T22:34:25.396 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.PuCVi6yPCVvSASJM7n7Je3Ax3B9N8aw_0 2026-02-24T22:34:25.396 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15198113882082823484', time=2026-02-25T00:26:45.265259+0000, chain.objs.size()=1 2026-02-24T22:34:25.396 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.RN6O9pU_kNEZf-RWn3uu-5zt9wFaVq5_0 2026-02-24T22:34:25.396 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11006191638202274666', time=2026-02-25T00:26:45.266891+0000, chain.objs.size()=1 2026-02-24T22:34:25.396 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.oqEoihvMlJcqOOsOOuIv1EcoR5BsgcW_0 2026-02-24T22:34:25.396 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5559352698756572601', time=2026-02-25T00:26:45.267651+0000, chain.objs.size()=1 2026-02-24T22:34:25.396 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.8zi6hfXKyqI3Nea5_6m4iiFJUWus1Cv_0 2026-02-24T22:34:25.396 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9798214142098986686', time=2026-02-25T00:26:45.276914+0000, chain.objs.size()=1 2026-02-24T22:34:25.396 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.0Jn5k_rFz2K1VLfDiePAdGx5b1AF1yF_0 2026-02-24T22:34:25.396 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3333171707408106834', time=2026-02-25T00:26:45.305593+0000, chain.objs.size()=1 2026-02-24T22:34:25.396 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.QIareydNv_0ZVvBQQgzfowy2xZq2ZCG_0 2026-02-24T22:34:25.396 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9403131676159956842', time=2026-02-25T00:26:45.306186+0000, chain.objs.size()=1 2026-02-24T22:34:25.396 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.aeqxzgpcDJl0fSCROHnb4OeBqXYYX6I_0 2026-02-24T22:34:25.396 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12508399008914285348', time=2026-02-25T00:26:45.312692+0000, chain.objs.size()=1 2026-02-24T22:34:25.396 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.9pZCYEz1Ftm3JA0RZVn5x5qyoD2d0O7_0 2026-02-24T22:34:25.396 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12819284400964896573', time=2026-02-25T00:26:45.339901+0000, chain.objs.size()=1 2026-02-24T22:34:25.397 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.svNAIIFzCi-kyXaAc9DqFqrD_c5uo9q_0 2026-02-24T22:34:25.397 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2154118392392951288', time=2026-02-25T00:26:45.345457+0000, chain.objs.size()=1 2026-02-24T22:34:25.397 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.eM5-aM0NofJiTZaVyDU2vHBDaW6FC4R_0 2026-02-24T22:34:25.397 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5966931484864902498', time=2026-02-25T00:26:45.418724+0000, chain.objs.size()=1 2026-02-24T22:34:25.397 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.iA4tTy8yWbYkzIT0QvTsDUM2j1Yrhmx_0 2026-02-24T22:34:25.397 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12316892164170543182', time=2026-02-25T00:26:45.436042+0000, chain.objs.size()=1 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ilgILFyVMgtixmEj9vvTfmlPOOZRysn_0 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16801840905010903492', time=2026-02-25T00:26:45.485550+0000, chain.objs.size()=1 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.2zUiSHvdC8bUoeAw6QaD7HKy6G5kY4R_0 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2358999678831260631', time=2026-02-25T00:26:45.500300+0000, chain.objs.size()=1 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.2hpHK_qp2ok5L-2k1-eNGJRBSnejrSS_0 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17940905695693309576', time=2026-02-25T00:26:45.501467+0000, chain.objs.size()=1 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.C10TmcHikxFGEaCe12Jcb9LZmCCxDIQ_0 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9585969171096923319', time=2026-02-25T00:26:45.577539+0000, chain.objs.size()=1 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.NheBfeEa9xfGlEQ8W1nuazigVvhnf3j_0 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.872594907101170950', time=2026-02-25T00:26:45.578958+0000, chain.objs.size()=1 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.SUXzO4ZfUS8xZw77VSz2KtM4ldt8gLS_0 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12540202446939927880', time=2026-02-25T00:26:45.819837+0000, chain.objs.size()=1 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.9Cj5IQ9q7GH5wqvPUcMvsp-P1zU6Mav_0 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4660548025370057453', time=2026-02-25T00:26:45.823497+0000, chain.objs.size()=1 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.mBN7J0NZgf8i6r5M5C-YTIo-rcA7RiQ_0 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15749876398374512499', time=2026-02-25T00:26:45.824078+0000, chain.objs.size()=1 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.1vSyI96VV-9qzEIczG8Pi-4Q8OeQGFl_0 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9500580346127029226', time=2026-02-25T00:26:45.840277+0000, chain.objs.size()=1 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.nJN48LFI1sfVTzF1ao_PJq0xH-ZcToV_0 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10180038676833060027', time=2026-02-25T00:26:45.857924+0000, chain.objs.size()=1 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.-44TSYy0MkoGCytNKYTg6S98v49N5vG_0 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8814470111292389438', time=2026-02-25T00:26:46.025994+0000, chain.objs.size()=1 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.AUkKzRDicxgxR3KpRBMvZQt10vz8c0b_0 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8849935084835659026', time=2026-02-25T00:26:46.090972+0000, chain.objs.size()=1 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.AmNhvagvmiBKicLSe_nU6fzZtYoM6St_0 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8053791599743562567', time=2026-02-25T00:26:46.102416+0000, chain.objs.size()=1 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.b04X2dd7v3WQ998WbB1DFWXDbofL0cK_0 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2477901253402482060', time=2026-02-25T00:26:46.121200+0000, chain.objs.size()=1 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ysR7acshw6yXnhYH4zm1SBlVWpsZN6Y_0 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17220865696734376195', time=2026-02-25T00:26:46.162709+0000, chain.objs.size()=1 2026-02-24T22:34:25.398 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.JfjqCgtlpnXb0WjuvjVv-SnnIDN-40S_0 2026-02-24T22:34:25.399 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10958960973536410074', time=2026-02-25T00:26:53.157244+0000, chain.objs.size()=1 2026-02-24T22:34:25.399 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.YAAWxXSs9SlH-dAmHaMbV_Dv8SPZ4YG_0 2026-02-24T22:34:25.399 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16263200007332933611', time=2026-02-25T00:26:53.169290+0000, chain.objs.size()=1 2026-02-24T22:34:25.399 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.P3OMNcILYWJBoZ-mdktSMJgaO09_js-_0 2026-02-24T22:34:25.399 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4186121422099053650', time=2026-02-25T00:26:53.200742+0000, chain.objs.size()=1 2026-02-24T22:34:25.399 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.IkV_Hzhb5VemOYpFwsy_T2_tj71jchr_0 2026-02-24T22:34:25.399 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7056424208317390460', time=2026-02-25T00:26:53.230718+0000, chain.objs.size()=1 2026-02-24T22:34:25.399 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.wul84pw7MHHJTc2v4usuIvPcJ3Ya7oj_0 2026-02-24T22:34:25.399 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17453653150260991104', time=2026-02-25T00:26:53.374255+0000, chain.objs.size()=1 2026-02-24T22:34:25.399 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.2nEvXFhcGPhlB6o9hZKX7ygKPQ7cJQf_0 2026-02-24T22:34:25.399 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5455641157253005317', time=2026-02-25T00:26:53.379572+0000, chain.objs.size()=1 2026-02-24T22:34:25.399 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.KvJsHB1GNzgbbNYbuFXPeESk9-j2Ioj_0 2026-02-24T22:34:25.399 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3365339599883162061', time=2026-02-25T00:26:53.387526+0000, chain.objs.size()=1 2026-02-24T22:34:25.399 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.NPrOy9OIE869fcTRjZV2Qz9lwEUh8aH_0 2026-02-24T22:34:25.399 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12332611203909821147', time=2026-02-25T00:26:53.395041+0000, chain.objs.size()=1 2026-02-24T22:34:25.399 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Yr3gmRN9Z0I-JmJ5MtBAdGE2nmk_qUe_0 2026-02-24T22:34:25.399 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9127195273423704764', time=2026-02-25T00:26:53.428055+0000, chain.objs.size()=1 2026-02-24T22:34:25.399 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.407+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.3e8IGAxdJYnJC89C6Y-eH9CW6Ym4AO7_0 2026-02-24T22:34:25.399 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5477066872353918929', time=2026-02-25T00:26:53.539838+0000, chain.objs.size()=1 2026-02-24T22:34:25.399 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.vn31UFfQhFT8goDdhP2-Ai-LS4JpgA5_0 2026-02-24T22:34:25.400 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7804329473071018375', time=2026-02-25T00:26:53.546140+0000, chain.objs.size()=1 2026-02-24T22:34:25.400 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.dKLtnM3qaQ2FGRulZGYJgv2elIGIbPr_0 2026-02-24T22:34:25.400 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/38095 2026-02-24T22:34:25.401 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=36, truncated=0, next_marker='' 2026-02-24T22:34:25.401 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11195103744048506992', time=2026-02-25T00:26:53.552097+0000, chain.objs.size()=1 2026-02-24T22:34:25.401 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.e5tN46ZKhjFY-Veui1aRkgMTKupLdc0_0 2026-02-24T22:34:25.401 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5985631055515587200', time=2026-02-25T00:26:53.585704+0000, chain.objs.size()=1 2026-02-24T22:34:25.401 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.GWfzjc8DUhEpKtTxD8GmNs68tKkf0hU_0 2026-02-24T22:34:25.401 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15166750634405052792', time=2026-02-25T00:26:53.588729+0000, chain.objs.size()=1 2026-02-24T22:34:25.401 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.NhWx6kkEYRqG_VOXV4vKKzlovHQOhA3_0 2026-02-24T22:34:25.401 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17636588674298652931', time=2026-02-25T00:26:53.595868+0000, chain.objs.size()=1 2026-02-24T22:34:25.401 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.tK_ZSdD2bEs4bw_jWXO_PpVbM24IXxe_0 2026-02-24T22:34:25.401 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14874155252456198762', time=2026-02-25T00:26:53.626719+0000, chain.objs.size()=1 2026-02-24T22:34:25.401 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.T4xEBh0HddPDF6v_V6DR5IgXzhbQPze_0 2026-02-24T22:34:25.401 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6078333158918619307', time=2026-02-25T00:26:53.739190+0000, chain.objs.size()=1 2026-02-24T22:34:25.401 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Z443PtHkxxuekwCFeJfQnmlMidgSpfU_0 2026-02-24T22:34:25.401 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6299701227245689992', time=2026-02-25T00:26:53.745181+0000, chain.objs.size()=1 2026-02-24T22:34:25.401 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.JGDzThCwKLZCj7M0FTJAlHxYkGtNQFn_0 2026-02-24T22:34:25.401 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12335170229874780791', time=2026-02-25T00:26:53.758393+0000, chain.objs.size()=1 2026-02-24T22:34:25.401 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.luXSdTE7FA5iiHDv_F9LVkShMi2DOsU_0 2026-02-24T22:34:25.401 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15640210162386974886', time=2026-02-25T00:26:53.760090+0000, chain.objs.size()=1 2026-02-24T22:34:25.402 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Zljja-r36tw12jcU7LhHHqcjcAR6Lq4_0 2026-02-24T22:34:25.402 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1192152878799222370', time=2026-02-25T00:26:53.800199+0000, chain.objs.size()=1 2026-02-24T22:34:25.402 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.9E5WQLFhwcVKLCz8eXefVv1EdhX9cZR_0 2026-02-24T22:34:25.402 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2944948027201292572', time=2026-02-25T00:26:53.826375+0000, chain.objs.size()=1 2026-02-24T22:34:25.402 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.dvyJIa0lx6Mal9n4M-ffLDsm-JNM1hu_0 2026-02-24T22:34:25.402 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3618099831981619889', time=2026-02-25T00:26:53.829047+0000, chain.objs.size()=1 2026-02-24T22:34:25.402 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.RLfv1l469NQDJVozHL0cDj6L6pt5iUa_0 2026-02-24T22:34:25.402 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9804499182282657905', time=2026-02-25T00:26:53.946816+0000, chain.objs.size()=1 2026-02-24T22:34:25.402 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.HygeITiT8D1t6Jzdw7XD_c1WYKzj_sM_0 2026-02-24T22:34:25.402 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8879471011492851658', time=2026-02-25T00:26:53.967468+0000, chain.objs.size()=1 2026-02-24T22:34:25.402 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.2AW2y1gr_hqov1yjVJOjXh1byWZl5-S_0 2026-02-24T22:34:25.402 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9086204042474391030', time=2026-02-25T00:26:53.986453+0000, chain.objs.size()=1 2026-02-24T22:34:25.402 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.UY3VbaJ7z1HJSPFcnm02K56sM1yuhQn_0 2026-02-24T22:34:25.402 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9373381115422455060', time=2026-02-25T00:26:54.129199+0000, chain.objs.size()=1 2026-02-24T22:34:25.402 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.9T4LiwReZWVXJgIE_dHNq0Bs1h4JySo_0 2026-02-24T22:34:25.402 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13875644929076366036', time=2026-02-25T00:26:54.136986+0000, chain.objs.size()=1 2026-02-24T22:34:25.403 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.XgEQHBAMYY-EnoN1JUhw1gtCBb3Q8Oo_0 2026-02-24T22:34:25.403 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13107537738862541483', time=2026-02-25T00:26:54.150547+0000, chain.objs.size()=1 2026-02-24T22:34:25.403 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.mkAGMDA5RdD3W237T0G5PxbW_zaGCoU_0 2026-02-24T22:34:25.403 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8521569895949232964', time=2026-02-25T00:26:54.180038+0000, chain.objs.size()=1 2026-02-24T22:34:25.403 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.j2PikH69F37iMcKUl3Nlb22wE5TjMWp_0 2026-02-24T22:34:25.403 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5649265529786059410', time=2026-02-25T00:26:54.190975+0000, chain.objs.size()=1 2026-02-24T22:34:25.403 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.t3L4DuISEJIjq1W74DCmeYruDMCHw0T_0 2026-02-24T22:34:25.403 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5668459085670980408', time=2026-02-25T00:26:54.334572+0000, chain.objs.size()=1 2026-02-24T22:34:25.403 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.AfFo3Jai92Q5L_7cG8FpVqXsbGJZddq_0 2026-02-24T22:34:25.403 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17572632563133648777', time=2026-02-25T00:26:54.350943+0000, chain.objs.size()=1 2026-02-24T22:34:25.403 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.VWjLEP1u3HpOWSnEMmorHqyYtdnWDoI_0 2026-02-24T22:34:25.403 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16249628072906689936', time=2026-02-25T00:26:54.354654+0000, chain.objs.size()=1 2026-02-24T22:34:25.403 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.411+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.icOoORkvqRI0trTgjohNiH82URA7XaK_0 2026-02-24T22:34:25.403 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13768625081519716559', time=2026-02-25T00:26:54.362553+0000, chain.objs.size()=1 2026-02-24T22:34:25.403 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.mHUIoWwnstodSCD5IJ9ssu5z5lBXcpG_0 2026-02-24T22:34:25.403 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8272157009314575609', time=2026-02-25T00:26:56.148507+0000, chain.objs.size()=1 2026-02-24T22:34:25.403 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1692__shadow_.OPhC-zWLG8eWMuRHjTmX2cXp2FHfjQB_0 2026-02-24T22:34:25.403 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1356874437873668185', time=2026-02-25T00:26:56.708610+0000, chain.objs.size()=1 2026-02-24T22:34:25.403 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1693__shadow_.7Cq9S7Qmtf_c0M7aoYBkhQ27k6GEOvy_0 2026-02-24T22:34:25.403 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15271586300053313321', time=2026-02-25T00:27:18.685340+0000, chain.objs.size()=1 2026-02-24T22:34:25.403 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1734__shadow_.g5vo1vK-eiW6FIkV4gEXG1Uub5hPc4D_0 2026-02-24T22:34:25.403 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14034640637757631367', time=2026-02-25T00:28:04.224925+0000, chain.objs.size()=1 2026-02-24T22:34:25.403 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1802__shadow_.yyXDZW4hp5E1tXVs__xKI3wC-_W3-Xi_0 2026-02-24T22:34:25.404 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3825721318260623027', time=2026-02-25T00:28:17.009155+0000, chain.objs.size()=1 2026-02-24T22:34:25.404 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1821__shadow_.k2i1X1SYiBq2ssvnVRLoas_2W-42kZA_0 2026-02-24T22:34:25.404 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12412216174499299226', time=2026-02-25T00:28:23.713938+0000, chain.objs.size()=1 2026-02-24T22:34:25.404 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1829__shadow_.BQN5SbEn5j0N3G3NvIstmpxkWQs-nzg_0 2026-02-24T22:34:25.404 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.528966597116656535', time=2026-02-25T00:28:24.434351+0000, chain.objs.size()=1 2026-02-24T22:34:25.404 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1830__shadow_.mKGt19zIQYu6JiMpSkr68K-in3Du2fv_0 2026-02-24T22:34:25.404 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9330695038460524736', time=2026-02-25T00:28:25.145186+0000, chain.objs.size()=1 2026-02-24T22:34:25.404 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1831__shadow_.mMPBtcrDdTjFqGk7idvX5Iw-D0_EadO_0 2026-02-24T22:34:25.404 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5743694519908466867', time=2026-02-25T00:30:06.279614+0000, chain.objs.size()=1 2026-02-24T22:34:25.404 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1923__shadow_.mFKkLbn3_ohJUS77m-u6KDKIvskLUbq_0 2026-02-24T22:34:25.404 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11408553881567375724', time=2026-02-25T00:30:11.404992+0000, chain.objs.size()=1 2026-02-24T22:34:25.404 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1930__shadow_.XBUVQ0ZcqBGeXqMWSMH0EDYci3QBwGC_0 2026-02-24T22:34:25.404 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12886170926823218997', time=2026-02-25T00:30:18.742269+0000, chain.objs.size()=1 2026-02-24T22:34:25.404 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:O9yLkMWb4CeE4b.iyBJtNw0159cZD4R_.wXHjkizzP4Iba0VB4JPpmGlk6JifmLu_0 2026-02-24T22:34:25.404 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12548664421217942365', time=2026-02-25T00:30:37.162078+0000, chain.objs.size()=1 2026-02-24T22:34:25.404 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:9KKX2cYrHjGEYkMpiNYOtbTuKiJl.IK_.tME3X-dpiDRyXVKcQL68hiWrmVZE4bj_0 2026-02-24T22:34:25.404 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.406 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.415+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2026-02-24T22:34:25.407 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.407 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/39811' 2026-02-24T22:34:25.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.852628313330632671', time=2026-02-25T00:15:34.625689+0000, chain.objs.size()=1 2026-02-24T22:34:25.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506__shadow_.kRtf1G5_yWIFNoAWmj_mAn1ou7HB-yn_0 2026-02-24T22:34:25.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='_nCCGDm0rkN93CCQ-QNE6qTxPVqof9O0', time=2026-02-25T00:15:34.706431+0000, chain.objs.size()=1 2026-02-24T22:34:25.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516__shadow_.uhuY4WSjo13TdnwcizzSKCVTzCu2KNn_0 2026-02-24T22:34:25.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='_tOBSvPkQmPQ23t_Bs9nDZLtArbY2MN0', time=2026-02-25T00:15:34.709316+0000, chain.objs.size()=1 2026-02-24T22:34:25.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516__shadow_.F-HaF-nU-REigcj3r1Z_eoaX9ZwtJ3K_0 2026-02-24T22:34:25.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12454372969909564552', time=2026-02-25T00:16:33.516312+0000, chain.objs.size()=1 2026-02-24T22:34:25.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521__shadow_.96DGUO_PK48XgUuc0_03lf6kfV1me-i_0 2026-02-24T22:34:25.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16132523294784590819', time=2026-02-25T00:17:05.172887+0000, chain.objs.size()=1 2026-02-24T22:34:25.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522__shadow_.qvNcV1gLgwH_umvqbcy7CeIyemFCSt0_0 2026-02-24T22:34:25.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14381715720329419702', time=2026-02-25T00:17:08.017536+0000, chain.objs.size()=1 2026-02-24T22:34:25.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524__shadow_.is7ND5wZerPjuC0K_vfoYdidQPGfQvJ_0 2026-02-24T22:34:25.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17323469776549265061', time=2026-02-25T00:17:39.885571+0000, chain.objs.size()=1 2026-02-24T22:34:25.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525__shadow_.K5csHGW_2pKA-jX9LP37x_e-NFn3s7x_0 2026-02-24T22:34:25.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7266113275820152845', time=2026-02-25T00:17:42.541841+0000, chain.objs.size()=1 2026-02-24T22:34:25.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527__shadow_.BEcsVQFPo3ADFn63PI7s9aIMTrNrT0A_0 2026-02-24T22:34:25.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16613108852045152045', time=2026-02-25T00:18:14.580264+0000, chain.objs.size()=1 2026-02-24T22:34:25.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528__shadow_.EWRglw91rBi252ttzGdV9HqDGq2SWEe_0 2026-02-24T22:34:25.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13184367644941471312', time=2026-02-25T00:18:16.132075+0000, chain.objs.size()=1 2026-02-24T22:34:25.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529__shadow_.RmJScEzJNFa4oWSOwdUNKmuxhJsLKrK_0 2026-02-24T22:34:25.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13764189979831394396', time=2026-02-25T00:18:16.134313+0000, chain.objs.size()=1 2026-02-24T22:34:25.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529__shadow_.9MoZTLRQHAegAxD1MwQXnQ25fdgfWPa_0 2026-02-24T22:34:25.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13013642997460363619', time=2026-02-25T00:18:16.135805+0000, chain.objs.size()=1 2026-02-24T22:34:25.408 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529__shadow_.p6C4vW1zwXO8yhWBUfFtq5gyZMDyyUa_0 2026-02-24T22:34:25.409 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3901244498231324215', time=2026-02-25T00:18:49.785030+0000, chain.objs.size()=1 2026-02-24T22:34:25.409 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531__shadow_.PDJq0Qa9YZ15ee3JSBjvZm4DrznldxI_0 2026-02-24T22:34:25.409 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5139306115579993009', time=2026-02-25T00:18:51.097533+0000, chain.objs.size()=1 2026-02-24T22:34:25.409 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532__shadow_.a2365KjiPZthpS05eAzMIeJRITSoz8W_0 2026-02-24T22:34:25.409 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1454637158307004095', time=2026-02-25T00:18:51.100423+0000, chain.objs.size()=1 2026-02-24T22:34:25.409 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532__shadow_.L75-e87A4KxQoRwG9ZfSCO6nFBBfogZ_0 2026-02-24T22:34:25.409 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11285073818850068969', time=2026-02-25T00:18:51.104132+0000, chain.objs.size()=1 2026-02-24T22:34:25.409 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532__shadow_.l-tbyGORhrIOhSKu2cs6j9Wwv7CrJC__0 2026-02-24T22:34:25.409 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15814320365807460087', time=2026-02-25T00:19:24.182676+0000, chain.objs.size()=1 2026-02-24T22:34:25.409 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534__shadow_.t9PGUuwjG-WwII3V2Tb3Y8VLM6W4i40_0 2026-02-24T22:34:25.409 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17368794352025436006', time=2026-02-25T00:19:25.029191+0000, chain.objs.size()=1 2026-02-24T22:34:25.409 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534__shadow_.QO3Xq3WpdL7zRCX3WERX9BT1YJ0gSAA_0 2026-02-24T22:34:25.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1652694401275409684', time=2026-02-25T00:19:25.029728+0000, chain.objs.size()=1 2026-02-24T22:34:25.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534__shadow_.hNi9saGvLwVQ7FUruHh8P8iEufxULSf_0 2026-02-24T22:34:25.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.197830400348037940', time=2026-02-25T00:19:25.035409+0000, chain.objs.size()=1 2026-02-24T22:34:25.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534__shadow_.HFh3JXcLqoVg7ZxTTtPQxpkDCHMA1Am_0 2026-02-24T22:34:25.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14500228751938663941', time=2026-02-25T00:19:57.118678+0000, chain.objs.size()=1 2026-02-24T22:34:25.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535__shadow_.nXq8CGMwRq03o_GLs7_3uqGfqFnp_Ps_0 2026-02-24T22:34:25.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5719619946751127362', time=2026-02-25T00:19:57.119637+0000, chain.objs.size()=1 2026-02-24T22:34:25.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535__shadow_.4Z7GmJfCP1veOOUXc1qwfvU_jUsxJpz_0 2026-02-24T22:34:25.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3001649306305504366', time=2026-02-25T00:19:58.467830+0000, chain.objs.size()=1 2026-02-24T22:34:25.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536__shadow_.2kXtN15epKHUmIoCO9qqPukQIDgEw9m_0 2026-02-24T22:34:25.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10693073444833035380', time=2026-02-25T00:20:34.041669+0000, chain.objs.size()=1 2026-02-24T22:34:25.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539__shadow_.xGcPqj2XEsODVUJE3A4G2Afcx7oUAoi_0 2026-02-24T22:34:25.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10809630292509518841', time=2026-02-25T00:25:37.543843+0000, chain.objs.size()=1 2026-02-24T22:34:25.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1603__shadow_.cszsJ5xLa9YWS_eS5GYPDsympwngQLw_0 2026-02-24T22:34:25.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9639316730955973330', time=2026-02-25T00:25:48.521909+0000, chain.objs.size()=1 2026-02-24T22:34:25.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1620__shadow_.0xLGrOa2SaEKMgQJ1W7VOiylRApUL--_0 2026-02-24T22:34:25.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13709696840150625217', time=2026-02-25T00:25:50.841771+0000, chain.objs.size()=1 2026-02-24T22:34:25.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.419+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1624__shadow_.T7f5WvtzghGzU40P_4hbAsfzGO91oB3_0 2026-02-24T22:34:25.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3193952809292346360', time=2026-02-25T00:25:59.088302+0000, chain.objs.size()=1 2026-02-24T22:34:25.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.pjR7XfaiGcdQLcidiBcjdFTurxPajYh_0 2026-02-24T22:34:25.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18402871511337525469', time=2026-02-25T00:25:59.111377+0000, chain.objs.size()=1 2026-02-24T22:34:25.410 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.P_fo598IN8qH-7UwOCPGu72k0Q0uf_6_0 2026-02-24T22:34:25.411 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4143227686138285520', time=2026-02-25T00:25:59.115591+0000, chain.objs.size()=1 2026-02-24T22:34:25.411 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.DKWrC8rk7ifxsFBoZr2zIWh85GsZf5t_0 2026-02-24T22:34:25.411 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9977111143356998666', time=2026-02-25T00:25:59.177059+0000, chain.objs.size()=1 2026-02-24T22:34:25.411 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.KfZQtxDZya3L1BeHuqI0bqA-JLKEzCz_0 2026-02-24T22:34:25.411 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4207592548244546400', time=2026-02-25T00:25:59.283431+0000, chain.objs.size()=1 2026-02-24T22:34:25.411 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.fB8Rx9q8-UIbmypR9X5awB4HR7g9J_J_0 2026-02-24T22:34:25.411 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1057810359354901923', time=2026-02-25T00:25:59.293423+0000, chain.objs.size()=1 2026-02-24T22:34:25.411 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ReDjTYS8FLto7uBTfU1dZHQLXWbTNjj_0 2026-02-24T22:34:25.411 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11825496925006600739', time=2026-02-25T00:25:59.306633+0000, chain.objs.size()=1 2026-02-24T22:34:25.411 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.z5acyjZZU2SaaYd7WtsSJ37UlzElAZ6_0 2026-02-24T22:34:25.411 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10285872938782034592', time=2026-02-25T00:25:59.320195+0000, chain.objs.size()=1 2026-02-24T22:34:25.411 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.RfaPeDHIF4CMrFkE3igATClaa__dLNh_0 2026-02-24T22:34:25.412 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2095223415805860110', time=2026-02-25T00:25:59.337315+0000, chain.objs.size()=1 2026-02-24T22:34:25.412 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.d6QCgRPVfxfRGahS9ZzB0gTXE-EY_kE_0 2026-02-24T22:34:25.412 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15688817245993269328', time=2026-02-25T00:25:59.437569+0000, chain.objs.size()=1 2026-02-24T22:34:25.412 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.IaKIDPSrJdK5U1dXg6_s3oaSwJTZh-e_0 2026-02-24T22:34:25.412 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4384436633559375864', time=2026-02-25T00:25:59.437966+0000, chain.objs.size()=1 2026-02-24T22:34:25.412 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.EQCkS25h8ziIE9xe1dahflHedt6xSLv_0 2026-02-24T22:34:25.412 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1475578339323175064', time=2026-02-25T00:25:59.477718+0000, chain.objs.size()=1 2026-02-24T22:34:25.412 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.z_knaXTmTzLwnqG3UpNsXcZlGKKSnmc_0 2026-02-24T22:34:25.412 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5548746320721093700', time=2026-02-25T00:25:59.484946+0000, chain.objs.size()=1 2026-02-24T22:34:25.412 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.obv0T8kfUAbwP_dpdw_AkI4ukawLETl_0 2026-02-24T22:34:25.412 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10204784388979505157', time=2026-02-25T00:25:59.499748+0000, chain.objs.size()=1 2026-02-24T22:34:25.412 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.k31iwSNE5fpQiXLZpeo8bsSbqia-E2y_0 2026-02-24T22:34:25.412 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11852818121561477305', time=2026-02-25T00:25:59.595449+0000, chain.objs.size()=1 2026-02-24T22:34:25.412 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_._kKLwn9n88CO23BMcE6F1Nzoob4NNkd_0 2026-02-24T22:34:25.412 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4437812310641873810', time=2026-02-25T00:25:59.607871+0000, chain.objs.size()=1 2026-02-24T22:34:25.412 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.yrbGqkeychdVVy30A-Zci8RuBM1G4K5_0 2026-02-24T22:34:25.412 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2752921989862996796', time=2026-02-25T00:25:59.625245+0000, chain.objs.size()=1 2026-02-24T22:34:25.412 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Cp7VEycgAKsSwa07Fa5tauxvU_X5YU6_0 2026-02-24T22:34:25.412 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7633788084688567890', time=2026-02-25T00:25:59.738313+0000, chain.objs.size()=1 2026-02-24T22:34:25.412 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ECorTqbIXHA7gz-YDmLCM2urLPz58uO_0 2026-02-24T22:34:25.412 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2650880147141741472', time=2026-02-25T00:25:59.790807+0000, chain.objs.size()=1 2026-02-24T22:34:25.412 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.qVpwPYH1Pd5EXyHJ2X37q9vvBcS_vAq_0 2026-02-24T22:34:25.413 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17977469592463439648', time=2026-02-25T00:25:59.810053+0000, chain.objs.size()=1 2026-02-24T22:34:25.413 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.FBQ2MEztHP2X-cF3OApLj18zF2_lRR1_0 2026-02-24T22:34:25.413 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3476045724310123994', time=2026-02-25T00:25:59.959998+0000, chain.objs.size()=1 2026-02-24T22:34:25.413 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.OSo9lvceQRzWxsTrO4mBgxhQmchWMSq_0 2026-02-24T22:34:25.413 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13752204333122384514', time=2026-02-25T00:26:00.061550+0000, chain.objs.size()=1 2026-02-24T22:34:25.413 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.B_esiQPm5ivwKrnrDgnX5P-YOQZZISq_0 2026-02-24T22:34:25.413 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9588797641796237226', time=2026-02-25T00:26:00.062175+0000, chain.objs.size()=1 2026-02-24T22:34:25.413 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.LqoG0XJEAnVCzTmVrgS6hw6SYxhP_-v_0 2026-02-24T22:34:25.413 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6920053671050139579', time=2026-02-25T00:26:00.063226+0000, chain.objs.size()=1 2026-02-24T22:34:25.413 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.baD50mASX3jFrdT26Nb9Kg74W-w2PJo_0 2026-02-24T22:34:25.413 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11862936304023958004', time=2026-02-25T00:26:00.077699+0000, chain.objs.size()=1 2026-02-24T22:34:25.413 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.R8iajoM8aNIz3pwwggePRenMqxeBFgq_0 2026-02-24T22:34:25.413 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9293121910967437372', time=2026-02-25T00:26:00.101593+0000, chain.objs.size()=1 2026-02-24T22:34:25.413 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.k7OoYHrdPgVN0CRU4gydG2L3S65aNGM_0 2026-02-24T22:34:25.413 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2945346891050463686', time=2026-02-25T00:26:00.110035+0000, chain.objs.size()=1 2026-02-24T22:34:25.413 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.8dHE4OSjvtvcdkq5dWozVeArPrRGxRt_0 2026-02-24T22:34:25.413 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8087361281729399386', time=2026-02-25T00:26:00.294698+0000, chain.objs.size()=1 2026-02-24T22:34:25.413 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.AdgEMaiTjud07rDY0yw29GsiLkP2owY_0 2026-02-24T22:34:25.413 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17045836294399942080', time=2026-02-25T00:26:01.949604+0000, chain.objs.size()=1 2026-02-24T22:34:25.413 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1631__shadow_.fIohHUMIVYASPrhpPCEHPABLkayd7m2_0 2026-02-24T22:34:25.413 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.956133422058137618', time=2026-02-25T00:26:03.145224+0000, chain.objs.size()=1 2026-02-24T22:34:25.413 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1633__shadow_.nIAIh5ft9jBtq6ia7m3l5k0cwzqX1wj_0 2026-02-24T22:34:25.414 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9893109071099714259', time=2026-02-25T00:26:05.437501+0000, chain.objs.size()=1 2026-02-24T22:34:25.414 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1637__shadow_.aHVHT59rteWhJv9i5JCyF6Ov1X0fFJE_0 2026-02-24T22:34:25.414 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.408265421460704370', time=2026-02-25T00:26:12.381939+0000, chain.objs.size()=1 2026-02-24T22:34:25.414 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1649__shadow_.14po6OgyFhLd17zgf2VFhRzSVohEZoo_0 2026-02-24T22:34:25.414 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8893581650702764901', time=2026-02-25T00:26:14.818525+0000, chain.objs.size()=1 2026-02-24T22:34:25.414 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1653__shadow_.VieDXKhyW4mnk_GiHbG5h2ufV3FShrU_0 2026-02-24T22:34:25.414 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.71666385134322', time=2026-02-25T00:26:19.468982+0000, chain.objs.size()=1 2026-02-24T22:34:25.414 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1659__shadow_.yAYQNyCwju29y03gxRTWJFR7ZreGMQP_0 2026-02-24T22:34:25.414 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10178957421092409568', time=2026-02-25T00:26:35.769621+0000, chain.objs.size()=1 2026-02-24T22:34:25.414 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:vCaNlQ97.OdWtOT4T3UFMlGdcij2Ar-_.NgVjT-AwigoKZyvIHI3Kj2bK-JDh2Mn_0 2026-02-24T22:34:25.414 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7412866159580624761', time=2026-02-25T00:26:35.772112+0000, chain.objs.size()=1 2026-02-24T22:34:25.414 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:WUwpVPGaNc4BwGq33BfjlNxlimBknjW_.dw0iAP0aosrkUkxHQDEB5ME6-fAQ6C-_0 2026-02-24T22:34:25.414 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10178957421092409568', time=2026-02-25T00:26:35.783331+0000, chain.objs.size()=1 2026-02-24T22:34:25.414 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:vCaNlQ97.OdWtOT4T3UFMlGdcij2Ar-_.NgVjT-AwigoKZyvIHI3Kj2bK-JDh2Mn_0 2026-02-24T22:34:25.414 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7412866159580624761', time=2026-02-25T00:26:35.804004+0000, chain.objs.size()=1 2026-02-24T22:34:25.414 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:WUwpVPGaNc4BwGq33BfjlNxlimBknjW_.dw0iAP0aosrkUkxHQDEB5ME6-fAQ6C-_0 2026-02-24T22:34:25.414 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14494819458144775634', time=2026-02-25T00:26:35.806704+0000, chain.objs.size()=1 2026-02-24T22:34:25.414 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:Zx-.rsDIFxGSLJjrAvy-yzyltRKEU6s_.kI4GO9C4pgE22YkdxjWzFQO0xLskMkA_0 2026-02-24T22:34:25.414 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14494819458144775634', time=2026-02-25T00:26:35.809133+0000, chain.objs.size()=1 2026-02-24T22:34:25.414 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:Zx-.rsDIFxGSLJjrAvy-yzyltRKEU6s_.kI4GO9C4pgE22YkdxjWzFQO0xLskMkA_0 2026-02-24T22:34:25.414 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14494819458144775634', time=2026-02-25T00:26:35.811932+0000, chain.objs.size()=1 2026-02-24T22:34:25.414 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:Zx-.rsDIFxGSLJjrAvy-yzyltRKEU6s_.kI4GO9C4pgE22YkdxjWzFQO0xLskMkA_0 2026-02-24T22:34:25.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14494819458144775634', time=2026-02-25T00:26:35.813436+0000, chain.objs.size()=1 2026-02-24T22:34:25.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:Zx-.rsDIFxGSLJjrAvy-yzyltRKEU6s_.kI4GO9C4pgE22YkdxjWzFQO0xLskMkA_0 2026-02-24T22:34:25.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14494819458144775634', time=2026-02-25T00:26:35.823433+0000, chain.objs.size()=1 2026-02-24T22:34:25.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.423+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:Zx-.rsDIFxGSLJjrAvy-yzyltRKEU6s_.kI4GO9C4pgE22YkdxjWzFQO0xLskMkA_0 2026-02-24T22:34:25.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14494819458144775634', time=2026-02-25T00:26:35.826280+0000, chain.objs.size()=1 2026-02-24T22:34:25.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:Zx-.rsDIFxGSLJjrAvy-yzyltRKEU6s_.kI4GO9C4pgE22YkdxjWzFQO0xLskMkA_0 2026-02-24T22:34:25.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14494819458144775634', time=2026-02-25T00:26:35.831751+0000, chain.objs.size()=1 2026-02-24T22:34:25.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:Zx-.rsDIFxGSLJjrAvy-yzyltRKEU6s_.kI4GO9C4pgE22YkdxjWzFQO0xLskMkA_0 2026-02-24T22:34:25.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14494819458144775634', time=2026-02-25T00:26:35.833643+0000, chain.objs.size()=1 2026-02-24T22:34:25.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:Zx-.rsDIFxGSLJjrAvy-yzyltRKEU6s_.kI4GO9C4pgE22YkdxjWzFQO0xLskMkA_0 2026-02-24T22:34:25.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14494819458144775634', time=2026-02-25T00:26:35.841791+0000, chain.objs.size()=1 2026-02-24T22:34:25.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:Zx-.rsDIFxGSLJjrAvy-yzyltRKEU6s_.kI4GO9C4pgE22YkdxjWzFQO0xLskMkA_0 2026-02-24T22:34:25.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14494819458144775634', time=2026-02-25T00:26:35.844015+0000, chain.objs.size()=1 2026-02-24T22:34:25.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:Zx-.rsDIFxGSLJjrAvy-yzyltRKEU6s_.kI4GO9C4pgE22YkdxjWzFQO0xLskMkA_0 2026-02-24T22:34:25.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10178957421092409568', time=2026-02-25T00:26:35.844665+0000, chain.objs.size()=1 2026-02-24T22:34:25.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:vCaNlQ97.OdWtOT4T3UFMlGdcij2Ar-_.NgVjT-AwigoKZyvIHI3Kj2bK-JDh2Mn_0 2026-02-24T22:34:25.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14494819458144775634', time=2026-02-25T00:26:35.847833+0000, chain.objs.size()=1 2026-02-24T22:34:25.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:Zx-.rsDIFxGSLJjrAvy-yzyltRKEU6s_.kI4GO9C4pgE22YkdxjWzFQO0xLskMkA_0 2026-02-24T22:34:25.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14494819458144775634', time=2026-02-25T00:26:35.848664+0000, chain.objs.size()=1 2026-02-24T22:34:25.415 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:Zx-.rsDIFxGSLJjrAvy-yzyltRKEU6s_.kI4GO9C4pgE22YkdxjWzFQO0xLskMkA_0 2026-02-24T22:34:25.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7412866159580624761', time=2026-02-25T00:26:35.853156+0000, chain.objs.size()=1 2026-02-24T22:34:25.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:WUwpVPGaNc4BwGq33BfjlNxlimBknjW_.dw0iAP0aosrkUkxHQDEB5ME6-fAQ6C-_0 2026-02-24T22:34:25.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14494819458144775634', time=2026-02-25T00:26:35.857534+0000, chain.objs.size()=1 2026-02-24T22:34:25.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:Zx-.rsDIFxGSLJjrAvy-yzyltRKEU6s_.kI4GO9C4pgE22YkdxjWzFQO0xLskMkA_0 2026-02-24T22:34:25.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14494819458144775634', time=2026-02-25T00:26:35.859891+0000, chain.objs.size()=1 2026-02-24T22:34:25.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:Zx-.rsDIFxGSLJjrAvy-yzyltRKEU6s_.kI4GO9C4pgE22YkdxjWzFQO0xLskMkA_0 2026-02-24T22:34:25.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10178957421092409568', time=2026-02-25T00:26:35.860484+0000, chain.objs.size()=1 2026-02-24T22:34:25.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:vCaNlQ97.OdWtOT4T3UFMlGdcij2Ar-_.NgVjT-AwigoKZyvIHI3Kj2bK-JDh2Mn_0 2026-02-24T22:34:25.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14494819458144775634', time=2026-02-25T00:26:35.864271+0000, chain.objs.size()=1 2026-02-24T22:34:25.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:Zx-.rsDIFxGSLJjrAvy-yzyltRKEU6s_.kI4GO9C4pgE22YkdxjWzFQO0xLskMkA_0 2026-02-24T22:34:25.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14494819458144775634', time=2026-02-25T00:26:35.864738+0000, chain.objs.size()=1 2026-02-24T22:34:25.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:Zx-.rsDIFxGSLJjrAvy-yzyltRKEU6s_.kI4GO9C4pgE22YkdxjWzFQO0xLskMkA_0 2026-02-24T22:34:25.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7412866159580624761', time=2026-02-25T00:26:35.868970+0000, chain.objs.size()=1 2026-02-24T22:34:25.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:WUwpVPGaNc4BwGq33BfjlNxlimBknjW_.dw0iAP0aosrkUkxHQDEB5ME6-fAQ6C-_0 2026-02-24T22:34:25.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14494819458144775634', time=2026-02-25T00:26:35.871792+0000, chain.objs.size()=1 2026-02-24T22:34:25.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:Zx-.rsDIFxGSLJjrAvy-yzyltRKEU6s_.kI4GO9C4pgE22YkdxjWzFQO0xLskMkA_0 2026-02-24T22:34:25.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14494819458144775634', time=2026-02-25T00:26:35.875216+0000, chain.objs.size()=1 2026-02-24T22:34:25.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:Zx-.rsDIFxGSLJjrAvy-yzyltRKEU6s_.kI4GO9C4pgE22YkdxjWzFQO0xLskMkA_0 2026-02-24T22:34:25.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14494819458144775634', time=2026-02-25T00:26:35.878381+0000, chain.objs.size()=1 2026-02-24T22:34:25.416 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:Zx-.rsDIFxGSLJjrAvy-yzyltRKEU6s_.kI4GO9C4pgE22YkdxjWzFQO0xLskMkA_0 2026-02-24T22:34:25.417 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14494819458144775634', time=2026-02-25T00:26:35.878878+0000, chain.objs.size()=1 2026-02-24T22:34:25.417 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:Zx-.rsDIFxGSLJjrAvy-yzyltRKEU6s_.kI4GO9C4pgE22YkdxjWzFQO0xLskMkA_0 2026-02-24T22:34:25.417 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10178957421092409568', time=2026-02-25T00:26:35.888197+0000, chain.objs.size()=1 2026-02-24T22:34:25.417 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:vCaNlQ97.OdWtOT4T3UFMlGdcij2Ar-_.NgVjT-AwigoKZyvIHI3Kj2bK-JDh2Mn_0 2026-02-24T22:34:25.417 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7412866159580624761', time=2026-02-25T00:26:35.891735+0000, chain.objs.size()=1 2026-02-24T22:34:25.417 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:WUwpVPGaNc4BwGq33BfjlNxlimBknjW_.dw0iAP0aosrkUkxHQDEB5ME6-fAQ6C-_0 2026-02-24T22:34:25.417 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14494819458144775634', time=2026-02-25T00:26:35.895839+0000, chain.objs.size()=1 2026-02-24T22:34:25.417 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:Zx-.rsDIFxGSLJjrAvy-yzyltRKEU6s_.kI4GO9C4pgE22YkdxjWzFQO0xLskMkA_0 2026-02-24T22:34:25.417 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18407165161465992622', time=2026-02-25T00:26:37.853547+0000, chain.objs.size()=1 2026-02-24T22:34:25.417 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1687__shadow_._goyYYP7acB_k63lorgH_dghU9isBUP_0 2026-02-24T22:34:25.417 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17079444605623927463', time=2026-02-25T00:26:44.845117+0000, chain.objs.size()=1 2026-02-24T22:34:25.417 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.3-lLH5saD8y3Qb_3Jk9R6FvFVGyYIga_0 2026-02-24T22:34:25.417 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3835879203711478790', time=2026-02-25T00:26:44.845984+0000, chain.objs.size()=1 2026-02-24T22:34:25.417 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.DcU7JZE7sJGAmLgo25PLjoksRh_0t9G_0 2026-02-24T22:34:25.417 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12202397486016677408', time=2026-02-25T00:26:44.854020+0000, chain.objs.size()=1 2026-02-24T22:34:25.417 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.68mSQkpiSGpIOMZkuqFg2GaDtEiAHlB_0 2026-02-24T22:34:25.417 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2553215387089086416', time=2026-02-25T00:26:44.870996+0000, chain.objs.size()=1 2026-02-24T22:34:25.417 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.VgWWNK3SBxiPJ5ZD_4CWkozEuJ4lV2K_0 2026-02-24T22:34:25.417 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10476785432710352248', time=2026-02-25T00:26:44.874555+0000, chain.objs.size()=1 2026-02-24T22:34:25.417 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.e-wmBGHTNWubz8AVOQ0bLni98wkKAZA_0 2026-02-24T22:34:25.418 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5489721697384137653', time=2026-02-25T00:26:44.969567+0000, chain.objs.size()=1 2026-02-24T22:34:25.418 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.3b7oviXpKcwHn_Qvl1TOwqVQjd1bVi6_0 2026-02-24T22:34:25.418 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8735072870569247308', time=2026-02-25T00:26:45.008707+0000, chain.objs.size()=1 2026-02-24T22:34:25.418 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.427+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.mDV8f_p66vsq02MtuXjzzuu5iqtVH6-_0 2026-02-24T22:34:25.419 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/39811 2026-02-24T22:34:25.421 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=61, truncated=0, next_marker='' 2026-02-24T22:34:25.421 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11899615622500952273', time=2026-02-25T00:26:45.014153+0000, chain.objs.size()=1 2026-02-24T22:34:25.421 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.qMJFoGIfJrDg99T0xsrI_GKv9HPAJCc_0 2026-02-24T22:34:25.421 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17308270363199615406', time=2026-02-25T00:26:45.113736+0000, chain.objs.size()=1 2026-02-24T22:34:25.421 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.GsGtOKd32hf3Kx6sew_tNJIuVlNU3ki_0 2026-02-24T22:34:25.421 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13792996793577252215', time=2026-02-25T00:26:45.123485+0000, chain.objs.size()=1 2026-02-24T22:34:25.421 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_._IJ34grDj8g6_RbulWndc27-I8cbcyA_0 2026-02-24T22:34:25.421 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18227391086662917776', time=2026-02-25T00:26:45.126239+0000, chain.objs.size()=1 2026-02-24T22:34:25.421 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.vd05TS2guxC-5SNowQwatmqPRyMf_RN_0 2026-02-24T22:34:25.421 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14043411265833270989', time=2026-02-25T00:26:45.137838+0000, chain.objs.size()=1 2026-02-24T22:34:25.421 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.66mk_F_djaGh0Fkf7TlN5ShVMvSRlhY_0 2026-02-24T22:34:25.421 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15710005632229120098', time=2026-02-25T00:26:45.157256+0000, chain.objs.size()=1 2026-02-24T22:34:25.421 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.6w8eKOhW8ny1WB7i1q0Xbf73l76jqCo_0 2026-02-24T22:34:25.421 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15639374082478059337', time=2026-02-25T00:26:45.280872+0000, chain.objs.size()=1 2026-02-24T22:34:25.421 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.R6bXin_U3CA1cZxX5EmcDOLXSwLpVB-_0 2026-02-24T22:34:25.421 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15941156836440478975', time=2026-02-25T00:26:45.291760+0000, chain.objs.size()=1 2026-02-24T22:34:25.421 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.d8z2qeVKVbfnBnaV_OAfuZxhNCt9-Nd_0 2026-02-24T22:34:25.421 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3078159032951877807', time=2026-02-25T00:26:45.447939+0000, chain.objs.size()=1 2026-02-24T22:34:25.421 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.iLHcTkVzYqylXAPv5DoC1BOph_iK2_G_0 2026-02-24T22:34:25.421 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5373640787348083476', time=2026-02-25T00:26:45.467418+0000, chain.objs.size()=1 2026-02-24T22:34:25.421 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.99C6mKcvoipwoVa4qjxmtdXoYEdOCY5_0 2026-02-24T22:34:25.421 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1598787219978261079', time=2026-02-25T00:26:45.475580+0000, chain.objs.size()=1 2026-02-24T22:34:25.421 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.95Z3A0chhCtEMRYyjKd-dHJdcCd5j-8_0 2026-02-24T22:34:25.421 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14946335424914014537', time=2026-02-25T00:26:45.606779+0000, chain.objs.size()=1 2026-02-24T22:34:25.421 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.zqFL0tQLGCInQiCz4a4xN3E-ExNQYyh_0 2026-02-24T22:34:25.422 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8242388652240819816', time=2026-02-25T00:26:45.615786+0000, chain.objs.size()=1 2026-02-24T22:34:25.422 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.sFem523iT3pvqBXZjx-BGuHECVMaRGn_0 2026-02-24T22:34:25.422 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11134695582807136206', time=2026-02-25T00:26:45.829975+0000, chain.objs.size()=1 2026-02-24T22:34:25.422 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.F3peeuk4BkXXHdP9HStSR81zZkKznO3_0 2026-02-24T22:34:25.422 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1588242367830760628', time=2026-02-25T00:26:45.989271+0000, chain.objs.size()=1 2026-02-24T22:34:25.422 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.KK5rrifCARTqfx-cpY9z_KmM-p1UnLL_0 2026-02-24T22:34:25.422 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2289510861316318360', time=2026-02-25T00:26:45.999800+0000, chain.objs.size()=1 2026-02-24T22:34:25.422 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.4Q_LZ15tTSijTayTMNexIxnJ7MLEp1E_0 2026-02-24T22:34:25.422 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16233557437950728646', time=2026-02-25T00:26:46.021138+0000, chain.objs.size()=1 2026-02-24T22:34:25.422 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.qhP8HfEAV1tIufkQ1UyfJ-1CUtSL9Xa_0 2026-02-24T22:34:25.422 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.604382631847256194', time=2026-02-25T00:26:46.111594+0000, chain.objs.size()=1 2026-02-24T22:34:25.422 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.TRRhgeNPo5EPQXMwi4x1sSI5DqUGT7V_0 2026-02-24T22:34:25.422 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7363077832612588180', time=2026-02-25T00:26:46.116122+0000, chain.objs.size()=1 2026-02-24T22:34:25.422 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.a8SLT-fpufsNQDLBcA2P68EPE6rzqzO_0 2026-02-24T22:34:25.422 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5518678803949215483', time=2026-02-25T00:26:46.134754+0000, chain.objs.size()=1 2026-02-24T22:34:25.422 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.6DPh3v7nz6B76_mC70ryYmNR1zOBNtd_0 2026-02-24T22:34:25.422 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8822417331859286664', time=2026-02-25T00:26:46.139635+0000, chain.objs.size()=1 2026-02-24T22:34:25.422 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.uq0wg4nwQxQJpt2pYJ0QO0hmcQ836yd_0 2026-02-24T22:34:25.422 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16187056371531301245', time=2026-02-25T00:26:53.165878+0000, chain.objs.size()=1 2026-02-24T22:34:25.422 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.qiRkpctCA3PFk7aIAyBS_d_EzPIl_z0_0 2026-02-24T22:34:25.422 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.434687294218341343', time=2026-02-25T00:26:53.169714+0000, chain.objs.size()=1 2026-02-24T22:34:25.422 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.431+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.bguVfgqFZPeUtbNIZ0z7qb1uj0XbgPY_0 2026-02-24T22:34:25.423 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17179295967817981997', time=2026-02-25T00:26:53.214375+0000, chain.objs.size()=1 2026-02-24T22:34:25.423 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.DEmWLOUXA93DdweyIVN1Yw4MtJd2L80_0 2026-02-24T22:34:25.423 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6129833892010476659', time=2026-02-25T00:26:53.337251+0000, chain.objs.size()=1 2026-02-24T22:34:25.423 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.0YaKY7zoDJrQh6mx4DtilCSyhR9dYzf_0 2026-02-24T22:34:25.423 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5487189974739937451', time=2026-02-25T00:26:53.443043+0000, chain.objs.size()=1 2026-02-24T22:34:25.423 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.rUEE2eeQsBR60-9EC1lUNJFxf2t7Cgt_0 2026-02-24T22:34:25.423 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5935449057354474859', time=2026-02-25T00:26:53.539978+0000, chain.objs.size()=1 2026-02-24T22:34:25.423 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.yAcL7RR_2HkQpLB6KLg02pmWcyQXaY9_0 2026-02-24T22:34:25.423 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14045520590170852456', time=2026-02-25T00:26:53.542372+0000, chain.objs.size()=1 2026-02-24T22:34:25.423 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.jBFcWRax-mjKNKRfQRrDimO3SdJEKdF_0 2026-02-24T22:34:25.423 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4940692094464122210', time=2026-02-25T00:26:53.613691+0000, chain.objs.size()=1 2026-02-24T22:34:25.423 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.RdBy4kVZDynwC0ehFVUSmUR6FQbunXs_0 2026-02-24T22:34:25.423 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13502137502534731899', time=2026-02-25T00:26:53.824965+0000, chain.objs.size()=1 2026-02-24T22:34:25.423 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.gqczbv51vkiNd81sVpuNGq1gd-KcbBu_0 2026-02-24T22:34:25.423 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4638603074724581171', time=2026-02-25T00:26:53.942311+0000, chain.objs.size()=1 2026-02-24T22:34:25.423 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.WNfPaOeYS42dOP1waaBUDrzSzub763__0 2026-02-24T22:34:25.423 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3254702687248316671', time=2026-02-25T00:26:53.948089+0000, chain.objs.size()=1 2026-02-24T22:34:25.423 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.nu7VxG5hBm-n0mSmCp1ACtg-IyCP0p3_0 2026-02-24T22:34:25.423 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11898866963040477448', time=2026-02-25T00:26:53.949719+0000, chain.objs.size()=1 2026-02-24T22:34:25.423 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.53mpF1YgZvmQ4QN4qjxbmT9jEyv8gTs_0 2026-02-24T22:34:25.423 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10675117441919234634', time=2026-02-25T00:26:53.987159+0000, chain.objs.size()=1 2026-02-24T22:34:25.423 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.eZtuE0zVGo2s3YxYJNf5QCn2pzwxqj__0 2026-02-24T22:34:25.424 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14880534368192231358', time=2026-02-25T00:26:54.002977+0000, chain.objs.size()=1 2026-02-24T22:34:25.424 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.8a1iRibMV-2mlSs1ro3dIPrOwN82QU6_0 2026-02-24T22:34:25.424 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6720908957474772549', time=2026-02-25T00:26:54.163226+0000, chain.objs.size()=1 2026-02-24T22:34:25.424 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.EG4m7JsGZ7bqeQN2GbzNgZUxSCjHFcv_0 2026-02-24T22:34:25.424 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3711343943024018278', time=2026-02-25T00:26:54.306988+0000, chain.objs.size()=1 2026-02-24T22:34:25.424 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.zUmZwlLwFmvtjWkTBQ2Idp5D4JGzJJG_0 2026-02-24T22:34:25.424 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3330042228012780470', time=2026-02-25T00:26:54.365702+0000, chain.objs.size()=1 2026-02-24T22:34:25.424 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.4U2xoSo3aNLaEkdn3d0hHHlsTuBszUQ_0 2026-02-24T22:34:25.424 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4467626394442758526', time=2026-02-25T00:26:54.385041+0000, chain.objs.size()=1 2026-02-24T22:34:25.424 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.7XPyB4FXBrmhnUEm0ovx5FkCUUE5kmP_0 2026-02-24T22:34:25.424 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12135536717278788266', time=2026-02-25T00:26:54.393573+0000, chain.objs.size()=1 2026-02-24T22:34:25.424 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.B_Q_65J0_YT6lYbIISBw9d74VdHhBHU_0 2026-02-24T22:34:25.424 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9904347301751028409', time=2026-02-25T00:26:54.478706+0000, chain.objs.size()=1 2026-02-24T22:34:25.424 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.4FQjCUNYMZ6AWan35GTkszKCs_NcSSt_0 2026-02-24T22:34:25.424 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13980883281124604019', time=2026-02-25T00:26:54.534137+0000, chain.objs.size()=1 2026-02-24T22:34:25.424 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.jc5h3c0F5xXfwE09UM2wDyJFI5AzkX5_0 2026-02-24T22:34:25.424 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5167715285256953824', time=2026-02-25T00:26:57.197784+0000, chain.objs.size()=1 2026-02-24T22:34:25.425 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1694__shadow_.jbMAlKPFNSmiG-1AACZIT6r2Sc_nlCd_0 2026-02-24T22:34:25.425 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5307320587424257106', time=2026-02-25T00:28:21.289550+0000, chain.objs.size()=1 2026-02-24T22:34:25.425 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1826__shadow_.2x4S2BF7wiy8SmESENfcKgrqXQU9ruB_0 2026-02-24T22:34:25.425 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15207672627792261353', time=2026-02-25T00:28:24.935098+0000, chain.objs.size()=1 2026-02-24T22:34:25.425 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1831__shadow_.Suvc9AHtld0ExzTjVlDX6tgWVy3X7Fg_0 2026-02-24T22:34:25.425 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16636883013511605027', time=2026-02-25T00:28:32.554443+0000, chain.objs.size()=1 2026-02-24T22:34:25.425 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1849__shadow_.Rgb2Rc1VGMuIgCCjpjF5CC6BAgN7_zK_0 2026-02-24T22:34:25.425 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3187132081563142038', time=2026-02-25T00:28:32.559142+0000, chain.objs.size()=1 2026-02-24T22:34:25.425 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1849__shadow_.i9lZSj4RQh-Vxm-yYLob-G6eXMdWoWp_0 2026-02-24T22:34:25.425 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10692530631880999223', time=2026-02-25T00:28:50.316177+0000, chain.objs.size()=1 2026-02-24T22:34:25.425 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1882__multipart_mymultipartcopy.2~ufnHZUxRg4SNQNtHWUItSiv3hv-Lfvn.1 2026-02-24T22:34:25.425 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1790589880761920348', time=2026-02-25T00:29:14.031925+0000, chain.objs.size()=2 2026-02-24T22:34:25.425 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1891__multipart_mymultipart.2~PB0n5STwQYSQiyLosqqGO-FmHiyoGAH.1 2026-02-24T22:34:25.425 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1891__multipart_mymultipart.2~PB0n5STwQYSQiyLosqqGO-FmHiyoGAH.2 2026-02-24T22:34:25.425 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3441202224513000306', time=2026-02-25T00:30:08.645259+0000, chain.objs.size()=1 2026-02-24T22:34:25.425 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1926__shadow_.SocZn6ygbP0sxVesfnMUW8rWPv7NZcH_0 2026-02-24T22:34:25.426 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12297555747965470006', time=2026-02-25T00:30:24.337077+0000, chain.objs.size()=1 2026-02-24T22:34:25.426 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1938__shadow:SPIOFQAV44m4X3WBSjS41cx9aUpin4c_.pHlQUKrx7pBejBN5K1yshe8yYfuh0bQ_0 2026-02-24T22:34:25.426 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3835444300509722236', time=2026-02-25T00:30:40.494455+0000, chain.objs.size()=1 2026-02-24T22:34:25.426 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:R.lCauJpgH0s1ZT1re17-YRgd6MbEq2_.UXDkXnO40mOvP0-BHyrQZc90ITrRpWz_0 2026-02-24T22:34:25.426 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1922793002272332322', time=2026-02-25T00:30:43.502489+0000, chain.objs.size()=1 2026-02-24T22:34:25.426 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:om6owiLaxCn02f9hR8vURTvl5P6QOy1_.EtwL22ou1EkMg2DTfN6cJxPrv3yiYJS_0 2026-02-24T22:34:25.426 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9330055506383069559', time=2026-02-25T00:30:44.986258+0000, chain.objs.size()=1 2026-02-24T22:34:25.426 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:hbIOeu8BDPePsUPCMtjy4Pra04468zS_.ENI9up0dzwvY5OH5UmZoVBTs-t3JFie_0 2026-02-24T22:34:25.426 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12673213844670890412', time=2026-02-25T00:30:52.402553+0000, chain.objs.size()=1 2026-02-24T22:34:25.426 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1950__shadow:Nykho.DgQ0mMgqLIfrSu1R5pC50D1bs_.2AcveQPCg_R1aoE4AlaLfrhlsOD0tB7_0 2026-02-24T22:34:25.426 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11407934014151585474', time=2026-02-25T00:30:53.049154+0000, chain.objs.size()=1 2026-02-24T22:34:25.426 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1952__shadow:UtkHLk8ufqKIwICXJCCzkGdaTnlqJPz_.CMAbJ96W63up7OSjLzxV1mFuFOcbqJq_0 2026-02-24T22:34:25.426 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15091580703226383739', time=2026-02-25T00:30:56.354933+0000, chain.objs.size()=1 2026-02-24T22:34:25.426 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1956__shadow:YGmm7V8I543DYHre-8ihj-4HmjfGhuE_.eYf6jZl_uuYBN6uuJw-D5TPX2pLXQ9m_0 2026-02-24T22:34:25.426 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3395648014665872812', time=2026-02-25T00:30:57.027426+0000, chain.objs.size()=1 2026-02-24T22:34:25.426 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:YNnxcLpeTc7S4jSDIiSCRacWgzquXbW_.R8F2yPEN99qBVr4BwK24RJAlHnDJAzN_0 2026-02-24T22:34:25.426 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3395648014665872812', time=2026-02-25T00:30:57.028162+0000, chain.objs.size()=1 2026-02-24T22:34:25.426 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:YNnxcLpeTc7S4jSDIiSCRacWgzquXbW_.R8F2yPEN99qBVr4BwK24RJAlHnDJAzN_0 2026-02-24T22:34:25.426 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3395648014665872812', time=2026-02-25T00:30:57.028521+0000, chain.objs.size()=1 2026-02-24T22:34:25.426 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:YNnxcLpeTc7S4jSDIiSCRacWgzquXbW_.R8F2yPEN99qBVr4BwK24RJAlHnDJAzN_0 2026-02-24T22:34:25.426 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11281462062392976065', time=2026-02-25T00:33:49.829378+0000, chain.objs.size()=1 2026-02-24T22:34:25.426 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.435+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1972__shadow_.OFL-RAG9IzQze7A2KUaR5_4NQ8nIS6-_0 2026-02-24T22:34:25.427 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.439+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.428 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.439+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2026-02-24T22:34:25.429 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.439+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.429 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.439+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.430 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.439+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/38168' 2026-02-24T22:34:25.430 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.439+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8640604134508455727', time=2026-02-25T00:15:34.841285+0000, chain.objs.size()=1 2026-02-24T22:34:25.430 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.439+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1508__shadow_.b6CswsPtCBa2VmIODmGbByFBjOUtXWy_0 2026-02-24T22:34:25.430 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.439+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2476230221022388787', time=2026-02-25T00:16:32.618145+0000, chain.objs.size()=1 2026-02-24T22:34:25.430 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.439+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520__shadow_.YTIbeuZEHphpsquZbyLjojdmbyjIK0z_0 2026-02-24T22:34:25.430 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.439+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16669553449010201755', time=2026-02-25T00:16:33.516102+0000, chain.objs.size()=1 2026-02-24T22:34:25.430 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.439+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521__shadow_.nn-gXS_8HZZN-_osJo69nLILEUXb03M_0 2026-02-24T22:34:25.430 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.439+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7397452743515832750', time=2026-02-25T00:16:33.517374+0000, chain.objs.size()=1 2026-02-24T22:34:25.430 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.439+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521__shadow_.ILLHjLYeibDVQvLmQXrc9leRo_i_IPI_0 2026-02-24T22:34:25.431 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7670061652282455080', time=2026-02-25T00:17:05.174407+0000, chain.objs.size()=1 2026-02-24T22:34:25.431 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522__shadow_.XX8bkAzOhvbS_CxzxT9CeKQj8xFXaQe_0 2026-02-24T22:34:25.431 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1481505691831377797', time=2026-02-25T00:17:05.176345+0000, chain.objs.size()=1 2026-02-24T22:34:25.431 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522__shadow_.1c76gIltMG0QmqpIn88yJd_Bho8EQ0b_0 2026-02-24T22:34:25.431 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1844515433925444368', time=2026-02-25T00:17:08.012974+0000, chain.objs.size()=1 2026-02-24T22:34:25.431 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524__shadow_.eFGErr48Nm7eDzQgJZoE7uScsGV0kuq_0 2026-02-24T22:34:25.431 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14340547299117712132', time=2026-02-25T00:18:15.482835+0000, chain.objs.size()=1 2026-02-24T22:34:25.431 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529__shadow_.TDI4F_gUMPIVC1I_LteLhoSiNvJ-z3o_0 2026-02-24T22:34:25.431 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8023033919276065598', time=2026-02-25T00:19:23.069794+0000, chain.objs.size()=1 2026-02-24T22:34:25.431 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533__shadow_.6JMHydvAXzR5cS8Xq4xgXQfn2mg2vUL_0 2026-02-24T22:34:25.431 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.844317097879436498', time=2026-02-25T00:19:25.029053+0000, chain.objs.size()=1 2026-02-24T22:34:25.431 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534__shadow_.kjA1c6S_lnCWOqtqrR-6ck5e41dBdRA_0 2026-02-24T22:34:25.431 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13696558215480818776', time=2026-02-25T00:19:57.117767+0000, chain.objs.size()=1 2026-02-24T22:34:25.431 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535__shadow_.wS_19bJfVrRSfbF9uKJJp9POHmufn_f_0 2026-02-24T22:34:25.431 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2299109851184245449', time=2026-02-25T00:19:58.461344+0000, chain.objs.size()=1 2026-02-24T22:34:25.431 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536__shadow_.h3K1TIauQhgKIXgdCYq7m_66E__1H93_0 2026-02-24T22:34:25.431 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16711732176842576764', time=2026-02-25T00:20:30.739761+0000, chain.objs.size()=1 2026-02-24T22:34:25.431 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537__shadow_.EFCc6twEXDj3pYhcmYBzEXIqj5FQ0Mo_0 2026-02-24T22:34:25.431 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16603628464332243240', time=2026-02-25T00:20:30.740275+0000, chain.objs.size()=1 2026-02-24T22:34:25.431 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537__shadow_.dwXMwv-J05ZuSEmohisAWr1Rh4ibst-_0 2026-02-24T22:34:25.431 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18237412530672492184', time=2026-02-25T00:21:06.323582+0000, chain.objs.size()=1 2026-02-24T22:34:25.431 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540__shadow_.KF8oCBv4ACcMrZ87k-CpfXgsr1F_dhE_0 2026-02-24T22:34:25.432 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9892588263584819993', time=2026-02-25T00:21:06.325202+0000, chain.objs.size()=1 2026-02-24T22:34:25.432 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540__shadow_.oDJytQk6dpm1ZQ8Y-kvoyYPwoCqoHJR_0 2026-02-24T22:34:25.432 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4746023139055653539', time=2026-02-25T00:24:13.724836+0000, chain.objs.size()=1 2026-02-24T22:34:25.432 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1566__shadow_.1ZWNGRT5elZ8xutUr-1B1vmRyvRaGoT_0 2026-02-24T22:34:25.432 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13654400161635287992', time=2026-02-25T00:25:46.733984+0000, chain.objs.size()=1 2026-02-24T22:34:25.432 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1617__shadow_.wZ--cw0295sZVYHkXMm2NYsvn1_HUgB_0 2026-02-24T22:34:25.432 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9181543542723914563', time=2026-02-25T00:25:59.103093+0000, chain.objs.size()=1 2026-02-24T22:34:25.432 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.txRmtu3WeY1BZcAWjiElB2XJD4v8XBE_0 2026-02-24T22:34:25.432 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5904438347646086359', time=2026-02-25T00:25:59.134465+0000, chain.objs.size()=1 2026-02-24T22:34:25.432 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.PRGOLVN0mrP7SA7K2O13H9kxsis28C-_0 2026-02-24T22:34:25.432 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2968507365607413346', time=2026-02-25T00:25:59.141437+0000, chain.objs.size()=1 2026-02-24T22:34:25.432 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.U3riE8nIAZrlFO14V8ibqMZNZ0brp97_0 2026-02-24T22:34:25.432 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9429625026322654661', time=2026-02-25T00:25:59.148119+0000, chain.objs.size()=1 2026-02-24T22:34:25.432 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.qkON5ic1n94T5VNfSiNML9YXTeLq-fz_0 2026-02-24T22:34:25.432 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16896102166757913031', time=2026-02-25T00:25:59.157251+0000, chain.objs.size()=1 2026-02-24T22:34:25.432 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Q8KOW0BZ-6UY6mPQWytQgtocHgYb-NQ_0 2026-02-24T22:34:25.432 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6774615285202035711', time=2026-02-25T00:25:59.161401+0000, chain.objs.size()=1 2026-02-24T22:34:25.432 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.4M_n4PgcNMxCQeWJauqPfFrkSKCo3Td_0 2026-02-24T22:34:25.432 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11251163706502403972', time=2026-02-25T00:25:59.167025+0000, chain.objs.size()=1 2026-02-24T22:34:25.432 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_._mjIDyw4ALp2XxAuuIUbjxNne93K-rF_0 2026-02-24T22:34:25.432 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10935332696070747094', time=2026-02-25T00:25:59.258873+0000, chain.objs.size()=1 2026-02-24T22:34:25.432 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.wEkxrKFaPEht0Sq3q2UCsM2j_Ry2eju_0 2026-02-24T22:34:25.433 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14084339785474356217', time=2026-02-25T00:25:59.302337+0000, chain.objs.size()=1 2026-02-24T22:34:25.433 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.CNpyZJ5SgOop57RWmkLVPhhMjxTkBLV_0 2026-02-24T22:34:25.433 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7776803327662892495', time=2026-02-25T00:25:59.319587+0000, chain.objs.size()=1 2026-02-24T22:34:25.433 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.gSowlEqn-pfYwfg1wak8rB-6A8xRXPA_0 2026-02-24T22:34:25.433 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2445319711537636814', time=2026-02-25T00:25:59.425467+0000, chain.objs.size()=1 2026-02-24T22:34:25.433 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.tXGktifLdWzUCQGRFwR_1Vlz2XXazSq_0 2026-02-24T22:34:25.433 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9372850944805585714', time=2026-02-25T00:25:59.438588+0000, chain.objs.size()=1 2026-02-24T22:34:25.433 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.HfiT6aFEjYyfKyo4TecQjerLuGvWLvN_0 2026-02-24T22:34:25.433 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15292757072490431779', time=2026-02-25T00:25:59.449041+0000, chain.objs.size()=1 2026-02-24T22:34:25.433 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Y2h894EVxkveTf_2cTMxI-VvceYnupK_0 2026-02-24T22:34:25.433 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1601467384296290806', time=2026-02-25T00:25:59.457040+0000, chain.objs.size()=1 2026-02-24T22:34:25.433 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.-8YDhTngk2NbTHCt8iJnZAmfyAgCNfO_0 2026-02-24T22:34:25.433 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12543385826449140897', time=2026-02-25T00:25:59.457556+0000, chain.objs.size()=1 2026-02-24T22:34:25.433 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.vFio-ZH4c-35KFZrapvoeojGeAj9K_s_0 2026-02-24T22:34:25.433 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10151911701600659742', time=2026-02-25T00:25:59.458090+0000, chain.objs.size()=1 2026-02-24T22:34:25.433 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Ztx9vCvqh0N9cz94vTBRFtAFfWWEzJh_0 2026-02-24T22:34:25.433 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13638477732128089591', time=2026-02-25T00:25:59.462939+0000, chain.objs.size()=1 2026-02-24T22:34:25.433 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.d11RyyXfvZflU0Nj4f9HkpRuJftzeg2_0 2026-02-24T22:34:25.433 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2450208253651642633', time=2026-02-25T00:25:59.468096+0000, chain.objs.size()=1 2026-02-24T22:34:25.433 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.tZpBmrV5v_HBavjNEfRon1ALjdcUxjk_0 2026-02-24T22:34:25.433 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17517391343817527083', time=2026-02-25T00:25:59.472892+0000, chain.objs.size()=1 2026-02-24T22:34:25.433 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.SX6Han6wQtfSuilf2QkW0KvWmA12eKD_0 2026-02-24T22:34:25.434 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2866118351520407877', time=2026-02-25T00:25:59.494735+0000, chain.objs.size()=1 2026-02-24T22:34:25.434 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.njFar_NQ73j-CE6sX_dnOMZpOf8Gjxq_0 2026-02-24T22:34:25.434 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1887859556566235182', time=2026-02-25T00:25:59.507237+0000, chain.objs.size()=1 2026-02-24T22:34:25.434 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.WaqxhKUinzfwfu1LKXM8uwPko9S53sS_0 2026-02-24T22:34:25.434 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1242984953264997312', time=2026-02-25T00:25:59.595871+0000, chain.objs.size()=1 2026-02-24T22:34:25.434 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.d1kEuS9CiCneb7fZKF2LMFFIoZOXgNd_0 2026-02-24T22:34:25.434 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15018310940578659412', time=2026-02-25T00:25:59.650839+0000, chain.objs.size()=1 2026-02-24T22:34:25.434 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.tHYAgerr9eSATFTobIfrnAFTx48Q-lS_0 2026-02-24T22:34:25.434 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.158764085724889304', time=2026-02-25T00:25:59.656109+0000, chain.objs.size()=1 2026-02-24T22:34:25.434 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.04-rwLNeDDMIycIrkzHYuRT3kA9EMTa_0 2026-02-24T22:34:25.434 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15117854329732732410', time=2026-02-25T00:25:59.660586+0000, chain.objs.size()=1 2026-02-24T22:34:25.434 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.r9WTDCx_Vxr2YDYcAsI6n0LyTiNqvC0_0 2026-02-24T22:34:25.434 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13011896997986956569', time=2026-02-25T00:25:59.666339+0000, chain.objs.size()=1 2026-02-24T22:34:25.434 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.0MonUO6Xr7bY5dDUIh06waO77gE2Jb5_0 2026-02-24T22:34:25.434 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7438860580963060284', time=2026-02-25T00:25:59.782263+0000, chain.objs.size()=1 2026-02-24T22:34:25.434 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.443+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ogCHKytAiaKDp4ZLaQms3hhiLyg3Ge0_0 2026-02-24T22:34:25.434 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14526357939712027479', time=2026-02-25T00:25:59.784087+0000, chain.objs.size()=1 2026-02-24T22:34:25.434 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.oP4G1JqGRJdWKT6rAAyKCqVuKP4EgJH_0 2026-02-24T22:34:25.435 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6753805142516006349', time=2026-02-25T00:25:59.825881+0000, chain.objs.size()=1 2026-02-24T22:34:25.435 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.B06L-YWVzOlqyebxMxwsNds1Wx4vi7v_0 2026-02-24T22:34:25.435 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17839536280461448483', time=2026-02-25T00:25:59.830788+0000, chain.objs.size()=1 2026-02-24T22:34:25.435 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.v3TEbwFGBIwlpdyitrzXFSUgXtGaNts_0 2026-02-24T22:34:25.435 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17890199142629012484', time=2026-02-25T00:25:59.835742+0000, chain.objs.size()=1 2026-02-24T22:34:25.435 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.UZn-6rAUlu0zQ23PnoGmeo__8_8hf_m_0 2026-02-24T22:34:25.435 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15678527592062117938', time=2026-02-25T00:25:59.907186+0000, chain.objs.size()=1 2026-02-24T22:34:25.435 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.maWxymh9sbhvxzd4SRQyOtlA--g70dD_0 2026-02-24T22:34:25.435 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16606140767129863769', time=2026-02-25T00:25:59.909996+0000, chain.objs.size()=1 2026-02-24T22:34:25.435 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ZGeEEIY4YczZcyGtR1tWUzvbFsnQF9w_0 2026-02-24T22:34:25.435 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4784725982784613760', time=2026-02-25T00:25:59.938304+0000, chain.objs.size()=1 2026-02-24T22:34:25.435 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.4mjCJTFV0I-OlXD27h9s56eyCfmHnu0_0 2026-02-24T22:34:25.435 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13458053956832931419', time=2026-02-25T00:25:59.949456+0000, chain.objs.size()=1 2026-02-24T22:34:25.435 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.-bggpSjOxmC01MDfVKTGuYLGv3Jy2ue_0 2026-02-24T22:34:25.435 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15610808808845669499', time=2026-02-25T00:25:59.950559+0000, chain.objs.size()=1 2026-02-24T22:34:25.435 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.9MYbqiD5P4zbahsmbHqP3UAFtwmnycf_0 2026-02-24T22:34:25.435 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6195381923786386019', time=2026-02-25T00:25:59.967454+0000, chain.objs.size()=1 2026-02-24T22:34:25.435 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.a8eCR06zL3JkCUaWcWLKnWlWQ_L2Buk_0 2026-02-24T22:34:25.435 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8744320501445540902', time=2026-02-25T00:26:00.071647+0000, chain.objs.size()=1 2026-02-24T22:34:25.435 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.zqxU1o8Oksobr8cRKLgDpDYXInj-W3R_0 2026-02-24T22:34:25.435 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.333754921098687992', time=2026-02-25T00:26:00.124618+0000, chain.objs.size()=1 2026-02-24T22:34:25.435 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.xEtOSl2geyvC2Fp18_KkREOneCr2s_K_0 2026-02-24T22:34:25.435 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16791838455642835869', time=2026-02-25T00:26:00.129316+0000, chain.objs.size()=1 2026-02-24T22:34:25.436 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.V0_ghMtu4Nl1eTfKhcpPf3hyKaMWgNm_0 2026-02-24T22:34:25.436 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15001969561650552988', time=2026-02-25T00:26:00.220360+0000, chain.objs.size()=1 2026-02-24T22:34:25.436 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.aU1sbO6aLGlO2g5TYsH7LYHylTog1oj_0 2026-02-24T22:34:25.436 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.867391852808304210', time=2026-02-25T00:26:00.232769+0000, chain.objs.size()=1 2026-02-24T22:34:25.436 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.5YmlYIRKKwc1NvTVMJz-5fUqT3AiEYL_0 2026-02-24T22:34:25.436 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5720370962019599128', time=2026-02-25T00:26:00.265863+0000, chain.objs.size()=1 2026-02-24T22:34:25.436 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.PziaA6L7RdR2hJGy9suyUaxvpMJ9EOs_0 2026-02-24T22:34:25.436 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5756253564270981338', time=2026-02-25T00:26:00.269169+0000, chain.objs.size()=1 2026-02-24T22:34:25.436 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ZZV34vu4zfnjld-RVoJX35AdlbfUDL8_0 2026-02-24T22:34:25.436 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3978405544585139350', time=2026-02-25T00:26:03.145345+0000, chain.objs.size()=1 2026-02-24T22:34:25.436 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1633__shadow_.p2ilGMKVooRNDvBnT9olfpuc4YCURI2_0 2026-02-24T22:34:25.436 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12347843096056305168', time=2026-02-25T00:26:08.286015+0000, chain.objs.size()=1 2026-02-24T22:34:25.436 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1642__shadow_.zumGVXIJaGAoW-CcURDHG90RyH0-1qQ_0 2026-02-24T22:34:25.436 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15497124363285283772', time=2026-02-25T00:26:13.618756+0000, chain.objs.size()=1 2026-02-24T22:34:25.436 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1651__shadow_.k3ba0NhH9kYuMfXEkbKkAbIuC2-Fo_o_0 2026-02-24T22:34:25.436 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14195056795581097601', time=2026-02-25T00:26:15.438536+0000, chain.objs.size()=1 2026-02-24T22:34:25.436 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1654__shadow_.7lhEMXkyf2XeJw-Mvwza2FrzIszfgBf_0 2026-02-24T22:34:25.436 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16445789828895138339', time=2026-02-25T00:26:16.590216+0000, chain.objs.size()=1 2026-02-24T22:34:25.436 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1656__shadow_.04W9lRIr7ZxnehIxUzMP4kr02gZKlvC_0 2026-02-24T22:34:25.437 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6562567248581514256', time=2026-02-25T00:26:44.802818+0000, chain.objs.size()=1 2026-02-24T22:34:25.437 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.VnTvI_Y6tKjhS-sItOoVT1v3zRiULPM_0 2026-02-24T22:34:25.437 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9857352368209906433', time=2026-02-25T00:26:44.803642+0000, chain.objs.size()=1 2026-02-24T22:34:25.437 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.g3tdEZ3F7FO4Ff1Lx5ZQdCzDHghg4Dd_0 2026-02-24T22:34:25.437 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.785098351239031516', time=2026-02-25T00:26:44.830421+0000, chain.objs.size()=1 2026-02-24T22:34:25.437 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.jhD9rI7LHUahx_sIE_0a_9EKcjIToMb_0 2026-02-24T22:34:25.437 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3353343033134348674', time=2026-02-25T00:26:44.954962+0000, chain.objs.size()=1 2026-02-24T22:34:25.437 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.356k-FvHc0ajpNCtBplzVLuLDvg5u08_0 2026-02-24T22:34:25.437 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13841499315386506894', time=2026-02-25T00:26:45.031409+0000, chain.objs.size()=1 2026-02-24T22:34:25.437 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ElcZo3hXGvAbT9OgtfOo2mEBJuJ5qwC_0 2026-02-24T22:34:25.438 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4584695528352338749', time=2026-02-25T00:26:45.147159+0000, chain.objs.size()=1 2026-02-24T22:34:25.438 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.6_jp0aCxAp3EaKmBDDc3miRyvrYiRX9_0 2026-02-24T22:34:25.438 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8949839432308390429', time=2026-02-25T00:26:45.157230+0000, chain.objs.size()=1 2026-02-24T22:34:25.438 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.QraIcyn8XWtQaukgQFAi97mE0dIrghO_0 2026-02-24T22:34:25.438 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.612638130399199282', time=2026-02-25T00:26:45.159828+0000, chain.objs.size()=1 2026-02-24T22:34:25.438 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.hfVhEr-cUwCB515-8OoJ8CIdH3379n8_0 2026-02-24T22:34:25.438 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13103958035997210057', time=2026-02-25T00:26:45.160354+0000, chain.objs.size()=1 2026-02-24T22:34:25.438 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.rl4Brvu7ycsx8RAwWZpyJtLbhVquXC7_0 2026-02-24T22:34:25.438 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2772701388166530712', time=2026-02-25T00:26:45.176211+0000, chain.objs.size()=1 2026-02-24T22:34:25.438 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.dnYQ4f3SQnOcsTb0_JOZlkTitkscGOb_0 2026-02-24T22:34:25.438 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14197435414328001621', time=2026-02-25T00:26:45.287754+0000, chain.objs.size()=1 2026-02-24T22:34:25.438 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.8R-InRDdC1XTenKHzjGBadkZ90kgEyZ_0 2026-02-24T22:34:25.439 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7465736463233984152', time=2026-02-25T00:26:45.304767+0000, chain.objs.size()=1 2026-02-24T22:34:25.439 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.EbHJy-ZWTG09KlH4Eq5c4Cl0SHh2bKr_0 2026-02-24T22:34:25.439 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4436664828134753284', time=2026-02-25T00:26:45.336174+0000, chain.objs.size()=1 2026-02-24T22:34:25.439 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.mhj7Egb1TogIL3lwdcOZgEZf0VhPhKq_0 2026-02-24T22:34:25.439 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9132932862054361346', time=2026-02-25T00:26:45.341087+0000, chain.objs.size()=1 2026-02-24T22:34:25.439 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.awLg9dZMmO06_UHsC7YGxT11Cw8-c7b_0 2026-02-24T22:34:25.439 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10067389521913611941', time=2026-02-25T00:26:45.423059+0000, chain.objs.size()=1 2026-02-24T22:34:25.439 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.447+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.bCoOdoqS4nrnNswdEtbs1ZbaIf0VnCX_0 2026-02-24T22:34:25.439 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13920842542178255168', time=2026-02-25T00:26:45.429394+0000, chain.objs.size()=1 2026-02-24T22:34:25.439 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.GRy31ri2bqJix1WbiX3d9_33W1l4AHT_0 2026-02-24T22:34:25.439 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1679395674579484222', time=2026-02-25T00:26:45.430000+0000, chain.objs.size()=1 2026-02-24T22:34:25.439 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.DlvpUKw0KDXfG1ofHJI0JRZfbYUtDzQ_0 2026-02-24T22:34:25.439 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6912661686725399115', time=2026-02-25T00:26:45.431623+0000, chain.objs.size()=1 2026-02-24T22:34:25.439 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_._mKANFbVB7j4ClETMAYTwfkHbKFeuiL_0 2026-02-24T22:34:25.439 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3811186577776231820', time=2026-02-25T00:26:45.437872+0000, chain.objs.size()=1 2026-02-24T22:34:25.439 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.mEVCS4SjshQDNfyleIBwm5duN1n-wn4_0 2026-02-24T22:34:25.439 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.635238203975419521', time=2026-02-25T00:26:45.444519+0000, chain.objs.size()=1 2026-02-24T22:34:25.440 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.T1DxjcWC9YN6BqD8yV_3qB1FqwkvVIa_0 2026-02-24T22:34:25.440 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3807802703329725817', time=2026-02-25T00:26:45.467363+0000, chain.objs.size()=1 2026-02-24T22:34:25.440 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.CzAQEtpJ7KoQN4PPUa6LhhuChU1ZkYO_0 2026-02-24T22:34:25.440 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12063618088901160167', time=2026-02-25T00:26:45.581778+0000, chain.objs.size()=1 2026-02-24T22:34:25.440 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.bpI4lxZozTkj942_rhbEHx4LAvd98Tk_0 2026-02-24T22:34:25.440 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15718060015216122477', time=2026-02-25T00:26:45.596450+0000, chain.objs.size()=1 2026-02-24T22:34:25.440 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.UOpRxRLVqxDNO9txTjRrb-YunnXX3Go_0 2026-02-24T22:34:25.440 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2194072831598510734', time=2026-02-25T00:26:45.605634+0000, chain.objs.size()=1 2026-02-24T22:34:25.440 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.YHUHA5W8d7M6EacJFQJYzWvDD_0oMMj_0 2026-02-24T22:34:25.440 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7762643560085115971', time=2026-02-25T00:26:45.928578+0000, chain.objs.size()=1 2026-02-24T22:34:25.440 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.x_ewune3BhnjZSy4yqngUk84o6qj-w8_0 2026-02-24T22:34:25.440 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3825198889382886746', time=2026-02-25T00:26:46.146491+0000, chain.objs.size()=1 2026-02-24T22:34:25.440 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.iyenoB2vt3N9lii3Pfoq6jUlm07Snic_0 2026-02-24T22:34:25.440 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12133696442535477710', time=2026-02-25T00:26:53.171745+0000, chain.objs.size()=1 2026-02-24T22:34:25.440 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.6V_ZFsCB73OZdhRkhPswzJUagZreZkv_0 2026-02-24T22:34:25.440 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14753159643750621226', time=2026-02-25T00:26:53.217586+0000, chain.objs.size()=1 2026-02-24T22:34:25.440 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.UDb7xnhlIJ1zp-Nh-Zc3AvESzmWU7_0_0 2026-02-24T22:34:25.440 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16750957305566742915', time=2026-02-25T00:26:53.262841+0000, chain.objs.size()=1 2026-02-24T22:34:25.440 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.P6yM-ET3ewIdKhTFBeyklJCEvA3QjQD_0 2026-02-24T22:34:25.440 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.506553641697441157', time=2026-02-25T00:26:53.342452+0000, chain.objs.size()=1 2026-02-24T22:34:25.440 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.gHexQgV92QoywNFAYndMbiiBbfE3bl0_0 2026-02-24T22:34:25.440 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4102107557916765014', time=2026-02-25T00:26:53.346130+0000, chain.objs.size()=1 2026-02-24T22:34:25.441 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.YRmqTafXopVlCyHDSRZVBPvsWO4RQuI_0 2026-02-24T22:34:25.441 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15920968706523192410', time=2026-02-25T00:26:53.358262+0000, chain.objs.size()=1 2026-02-24T22:34:25.441 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.EnK3IP6eBEq7PdCdSGBLmZWyx5IuhWw_0 2026-02-24T22:34:25.441 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4649883321222258266', time=2026-02-25T00:26:53.364238+0000, chain.objs.size()=1 2026-02-24T22:34:25.441 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.tK3d7lgF6q3Lrc5oUFCMmbtlJelSMb3_0 2026-02-24T22:34:25.441 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.451+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/38168 2026-02-24T22:34:25.443 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=34, truncated=0, next_marker='' 2026-02-24T22:34:25.443 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13616644492308694722', time=2026-02-25T00:26:53.411605+0000, chain.objs.size()=1 2026-02-24T22:34:25.443 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.6qlFdmTGC3zxlG_vUc1ulONch9dy8Qi_0 2026-02-24T22:34:25.443 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10103842626987871290', time=2026-02-25T00:26:53.441775+0000, chain.objs.size()=1 2026-02-24T22:34:25.443 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.J6c-0Ty-9MNWoiGWlzcx-YeCtoL97lX_0 2026-02-24T22:34:25.443 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1232297547889926268', time=2026-02-25T00:26:53.446783+0000, chain.objs.size()=1 2026-02-24T22:34:25.443 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.QtWS0csOfARnrCQP23LoF7bG-ckWBJ-_0 2026-02-24T22:34:25.443 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8570500386681264653', time=2026-02-25T00:26:53.521076+0000, chain.objs.size()=1 2026-02-24T22:34:25.443 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.FjvJY8azhh0OU7YoQDWmH4bxKAhAjfX_0 2026-02-24T22:34:25.443 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7119180841694306052', time=2026-02-25T00:26:53.524866+0000, chain.objs.size()=1 2026-02-24T22:34:25.443 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.q5u-2HujfnUQJlutbBz7BJEEnRmPg7P_0 2026-02-24T22:34:25.443 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7933130189353498603', time=2026-02-25T00:26:53.533261+0000, chain.objs.size()=1 2026-02-24T22:34:25.443 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.zuCnkb_kZ5Bher_foIoP7JFuNVZGnud_0 2026-02-24T22:34:25.444 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15346782898892607728', time=2026-02-25T00:26:53.582377+0000, chain.objs.size()=1 2026-02-24T22:34:25.444 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.94mwMkpNm5IkQR_Qlf74VDOAZWiPog5_0 2026-02-24T22:34:25.444 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18213625963095922221', time=2026-02-25T00:26:53.748345+0000, chain.objs.size()=1 2026-02-24T22:34:25.444 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.36d8zYvuuQ4Ub4c8yxbyN1Vv0u_e89X_0 2026-02-24T22:34:25.444 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16194212130806194903', time=2026-02-25T00:26:53.767263+0000, chain.objs.size()=1 2026-02-24T22:34:25.444 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.6lzB_WD4PKzpD9cLOU9rll7VA_V0YUM_0 2026-02-24T22:34:25.444 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.149094997147175389', time=2026-02-25T00:26:53.773682+0000, chain.objs.size()=1 2026-02-24T22:34:25.444 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.LFqQstiElgMixEwZfrRK4uF4PzXk1Je_0 2026-02-24T22:34:25.444 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17346100696821975386', time=2026-02-25T00:26:53.809406+0000, chain.objs.size()=1 2026-02-24T22:34:25.444 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.6GlSvyciBAlqtVs2JWBYdg4FnBjWqy-_0 2026-02-24T22:34:25.444 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10758754834239575587', time=2026-02-25T00:26:53.969332+0000, chain.objs.size()=1 2026-02-24T22:34:25.444 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.8J9lT_U2TLSF1tGdRL4KKFGpoDWhNVA_0 2026-02-24T22:34:25.444 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.355636980045897716', time=2026-02-25T00:26:53.972343+0000, chain.objs.size()=1 2026-02-24T22:34:25.444 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.nXUW6-XQHIQivYnlOKW7YNeY2Semm8b_0 2026-02-24T22:34:25.444 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6304389869280011500', time=2026-02-25T00:26:53.989392+0000, chain.objs.size()=1 2026-02-24T22:34:25.444 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.1M0OftjQNkg49D71sybwyfWyVssS14Y_0 2026-02-24T22:34:25.444 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14984684354125802193', time=2026-02-25T00:26:54.011117+0000, chain.objs.size()=1 2026-02-24T22:34:25.444 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.V6BoGRLAhHFdm2RwDjW9gh_bnBsACwh_0 2026-02-24T22:34:25.444 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10320819387603374647', time=2026-02-25T00:26:54.128823+0000, chain.objs.size()=1 2026-02-24T22:34:25.444 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.xgBggwvoecSBrDMbgzgdYt-ZyPl1TvI_0 2026-02-24T22:34:25.445 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16794073707259480803', time=2026-02-25T00:26:54.144285+0000, chain.objs.size()=1 2026-02-24T22:34:25.445 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.LgLso5vfOmmZNG0gxl0he6SMlbTB0BU_0 2026-02-24T22:34:25.445 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7003343767449704113', time=2026-02-25T00:26:54.159370+0000, chain.objs.size()=1 2026-02-24T22:34:25.445 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.BogFWe0A_4WmWQH4cg-APr3T2GrrcHb_0 2026-02-24T22:34:25.445 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4554578410762497591', time=2026-02-25T00:26:54.495518+0000, chain.objs.size()=1 2026-02-24T22:34:25.445 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.WgC_7qbPQmbSAcDZZ-RhQ0Es0aoSEUk_0 2026-02-24T22:34:25.445 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16173515732629175945', time=2026-02-25T00:26:54.505089+0000, chain.objs.size()=1 2026-02-24T22:34:25.445 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.4QmnQa9gAl446j_KSXkIo53EFNW2SIB_0 2026-02-24T22:34:25.445 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15517562365617779152', time=2026-02-25T00:27:03.540244+0000, chain.objs.size()=1 2026-02-24T22:34:25.445 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1705__shadow_.qZ5jqaNNNpvDss8rGx6H2wJDJnqngcV_0 2026-02-24T22:34:25.445 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3627590817609264249', time=2026-02-25T00:27:04.744883+0000, chain.objs.size()=1 2026-02-24T22:34:25.445 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1707__shadow_.2VTdeWY2KgLSMaCYE7S_YexRdNqVM91_0 2026-02-24T22:34:25.445 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7365335836996532649', time=2026-02-25T00:28:16.177649+0000, chain.objs.size()=1 2026-02-24T22:34:25.445 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1820__shadow_.NFCdDq5fJJdaxTwV0jTyxfPU8gZXE9C_0 2026-02-24T22:34:25.445 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12759412159199091008', time=2026-02-25T00:28:57.561957+0000, chain.objs.size()=3 2026-02-24T22:34:25.445 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1887__multipart_mymultipart.2~J04sNbjLXk2OLK9oHL0zjFd55cp8fzJ.1 2026-02-24T22:34:25.445 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1887__multipart_mymultipart.2~J04sNbjLXk2OLK9oHL0zjFd55cp8fzJ.2 2026-02-24T22:34:25.445 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1887__multipart_mymultipart.2~J04sNbjLXk2OLK9oHL0zjFd55cp8fzJ.3 2026-02-24T22:34:25.445 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14851536237925999869', time=2026-02-25T00:29:13.692043+0000, chain.objs.size()=3 2026-02-24T22:34:25.445 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1891__multipart_mymultipart.2~LwyTBARoIqVxuvj5KYHsf39PbErpHDZ.1 2026-02-24T22:34:25.445 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1891__multipart_mymultipart.2~LwyTBARoIqVxuvj5KYHsf39PbErpHDZ.2 2026-02-24T22:34:25.446 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1891__multipart_mymultipart.2~LwyTBARoIqVxuvj5KYHsf39PbErpHDZ.3 2026-02-24T22:34:25.446 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16196888854828498924', time=2026-02-25T00:29:16.269106+0000, chain.objs.size()=1 2026-02-24T22:34:25.446 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1894__shadow_.Gu81XhzHlRIk0mqfH_Sy8yc8fyYplTl_0 2026-02-24T22:34:25.446 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17009609613183462045', time=2026-02-25T00:30:19.389546+0000, chain.objs.size()=1 2026-02-24T22:34:25.446 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:B-7nWvsbBgVz2K71SulIDevAqo3-u.-_.wK5rVOg8c-MQ6V047vE0Bcd5eM3LKYB_0 2026-02-24T22:34:25.446 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3142435166781905280', time=2026-02-25T00:30:30.066531+0000, chain.objs.size()=1 2026-02-24T22:34:25.446 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1943__shadow:qNppct469.1sA-3ejbM0-odE82is-9k_.xC5QErZB3XU_hVGTK1BI5TMcreh1SKn_0 2026-02-24T22:34:25.446 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6126049193046902759', time=2026-02-25T00:30:33.177997+0000, chain.objs.size()=1 2026-02-24T22:34:25.446 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1946__shadow:DfJiDm0ABbb5-pO-fBPULFBwPkZiV5T_.r8SIe9tTzG4y-DkrPJLyuCMGZbdaCjd_0 2026-02-24T22:34:25.446 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2414770937766724038', time=2026-02-25T00:30:52.402522+0000, chain.objs.size()=1 2026-02-24T22:34:25.446 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1950__shadow:3qosJlIk2xJ72kWDV6fsMvIm8nsCIqd_.aEMiPEY5YHs3lv1MLBaT1QummQJqwDE_0 2026-02-24T22:34:25.446 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.207193695598746920', time=2026-02-25T00:30:57.742679+0000, chain.objs.size()=1 2026-02-24T22:34:25.446 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.455+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:U7wdG5KylcaDlVVt9GIuAECujHsmZJf_.fili_nkQhuYzd5u3Gg2aMKZHPNd2I8x_0 2026-02-24T22:34:25.446 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.459+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.207193695598746920', time=2026-02-25T00:30:57.743135+0000, chain.objs.size()=1 2026-02-24T22:34:25.446 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.459+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:U7wdG5KylcaDlVVt9GIuAECujHsmZJf_.fili_nkQhuYzd5u3Gg2aMKZHPNd2I8x_0 2026-02-24T22:34:25.446 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.459+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.207193695598746920', time=2026-02-25T00:30:57.743573+0000, chain.objs.size()=1 2026-02-24T22:34:25.446 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.459+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:U7wdG5KylcaDlVVt9GIuAECujHsmZJf_.fili_nkQhuYzd5u3Gg2aMKZHPNd2I8x_0 2026-02-24T22:34:25.446 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.459+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11675160502944988339', time=2026-02-25T00:33:45.045512+0000, chain.objs.size()=1 2026-02-24T22:34:25.446 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.459+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1967__shadow_.NN1-Sk_x6OYwfAHdCQSjPQkSsNpzxC1_0 2026-02-24T22:34:25.447 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.459+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.449 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.459+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2026-02-24T22:34:25.450 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.450 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.451 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/37536' 2026-02-24T22:34:25.451 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3559773479154526281', time=2026-02-25T00:16:33.510954+0000, chain.objs.size()=1 2026-02-24T22:34:25.451 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521__shadow_.RvExglVmahUCYEnOY_6cQIIFgx-OU9L_0 2026-02-24T22:34:25.451 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11835980979268776307', time=2026-02-25T00:17:05.166063+0000, chain.objs.size()=1 2026-02-24T22:34:25.451 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522__shadow_.PY0yyIBfx6-Kxnd0Qe89GCxxjNQQPY9_0 2026-02-24T22:34:25.451 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16847975852668067123', time=2026-02-25T00:17:07.113855+0000, chain.objs.size()=1 2026-02-24T22:34:25.451 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523__shadow_.v6x1o1sa6OHXWUVQwGgcd6ZFier7FPC_0 2026-02-24T22:34:25.451 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10635130635586864378', time=2026-02-25T00:17:08.006100+0000, chain.objs.size()=1 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524__shadow_.kgyKpuDA1-pE_-2D5jVoK3MReOBBovd_0 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16958514379717158771', time=2026-02-25T00:17:08.006552+0000, chain.objs.size()=1 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524__shadow_.jC4tm-T4rt3Y9GmiizR-1lstg3KJP_2_0 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11749845844203902971', time=2026-02-25T00:17:08.008499+0000, chain.objs.size()=1 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524__shadow_.xDn5u6WFic0BRvGC8o03LmhHhhVYpNG_0 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8967914138309713944', time=2026-02-25T00:17:39.876138+0000, chain.objs.size()=1 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525__shadow_.VUjPtPNYyizYGYZc76l9i-HybCxO6rT_0 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12203682835905396887', time=2026-02-25T00:17:42.529954+0000, chain.objs.size()=1 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527__shadow_.jEoFew5Di9cxn0LoJPJiYFrzEUBXtYu_0 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5543586403269826371', time=2026-02-25T00:17:42.533572+0000, chain.objs.size()=1 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527__shadow_.SNUjXNomqCG7jlnMwZQSfFS0ZVRKIgh_0 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17637855703861007867', time=2026-02-25T00:18:14.572991+0000, chain.objs.size()=1 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528__shadow_.6Jq0sk26sna5pHrPCI4YMlnMVyT3tRJ_0 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6936250536082745015', time=2026-02-25T00:18:49.769073+0000, chain.objs.size()=1 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531__shadow_.Ul589lKijU9aztpzJFW7339op2oS8ue_0 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10697453877602325522', time=2026-02-25T00:19:23.059460+0000, chain.objs.size()=1 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533__shadow_.8yipVyZgjQc2pKXQVtosJ2M6mqbkpB6_0 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10060824752615933848', time=2026-02-25T00:20:30.730469+0000, chain.objs.size()=1 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537__shadow_.CSxRZn6Fqsk4GjxJ7UGiGRVOjd6UBbh_0 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17214434641552387341', time=2026-02-25T00:20:32.508128+0000, chain.objs.size()=1 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538__shadow_.229_oBfEzhMo_7cTq0LxzN120PyTm2Y_0 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16203108091303289317', time=2026-02-25T00:20:32.514950+0000, chain.objs.size()=1 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538__shadow_.Z2jCVVnEW_l063pWo2g_Mj0OxeCPmf2_0 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8180885102558317378', time=2026-02-25T00:21:06.311341+0000, chain.objs.size()=1 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540__shadow_.csLDeBj6ilZ9fkeT1qAuTAEG5jeNmMS_0 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2860773220745054304', time=2026-02-25T00:23:14.350244+0000, chain.objs.size()=1 2026-02-24T22:34:25.452 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518__shadow_.n8hLDdEp472Xby8AS_GEWJBh4-yG7Bu_0 2026-02-24T22:34:25.453 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16511430673493125906', time=2026-02-25T00:23:14.351310+0000, chain.objs.size()=1 2026-02-24T22:34:25.453 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518__shadow_.AIJOoQbkIvgHR8Myfhr41JjsHdm2e_Q_0 2026-02-24T22:34:25.453 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5483293015190166170', time=2026-02-25T00:23:56.774924+0000, chain.objs.size()=1 2026-02-24T22:34:25.453 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1551__shadow_.bKhAdVZeQIAEH4CFzaK9n2u_tzqSpqI_0 2026-02-24T22:34:25.453 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7076191058775777497', time=2026-02-25T00:25:33.086933+0000, chain.objs.size()=1 2026-02-24T22:34:25.453 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1596__shadow_.Q05hmOaF6o_-Ap-N7QWaH8ZVc9UPImk_0 2026-02-24T22:34:25.453 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5104168176290742591', time=2026-02-25T00:25:39.037589+0000, chain.objs.size()=1 2026-02-24T22:34:25.453 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1605__shadow_.htSBLUfAFPCH6eMFyhQDXMqUEfTFFql_0 2026-02-24T22:34:25.453 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12734982292829288112', time=2026-02-25T00:25:48.507617+0000, chain.objs.size()=1 2026-02-24T22:34:25.453 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1620__shadow_.OF23AtjO-QnbAFnOluAZXzLKaI6rL04_0 2026-02-24T22:34:25.453 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1556037985027991219', time=2026-02-25T00:25:49.078392+0000, chain.objs.size()=1 2026-02-24T22:34:25.453 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1621__shadow_.Rflug6x5rchTW-T-RMMWL7p00KN2dDE_0 2026-02-24T22:34:25.453 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9987517153157711308', time=2026-02-25T00:25:59.075935+0000, chain.objs.size()=1 2026-02-24T22:34:25.453 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.nn64PTnDYLc5ODhOo8lLhk6BvQok5RK_0 2026-02-24T22:34:25.453 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17862172573780292214', time=2026-02-25T00:25:59.146116+0000, chain.objs.size()=1 2026-02-24T22:34:25.453 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.P1CMqW4vNMcvaizSf0kXlA1WE5-u-Hd_0 2026-02-24T22:34:25.454 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14459296824214342996', time=2026-02-25T00:25:59.246381+0000, chain.objs.size()=1 2026-02-24T22:34:25.454 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.BGdVvzfCnAMAmsqFxvXKzSh4PkT8P4a_0 2026-02-24T22:34:25.454 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11521169911292336285', time=2026-02-25T00:25:59.251740+0000, chain.objs.size()=1 2026-02-24T22:34:25.454 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.R6-gh9doH6XuQ9XotD8A08y4AHdxASI_0 2026-02-24T22:34:25.454 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1082605979411874773', time=2026-02-25T00:25:59.255782+0000, chain.objs.size()=1 2026-02-24T22:34:25.454 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.4sm71-k4Vu-sLkYhY9NzpMguoowrUxX_0 2026-02-24T22:34:25.454 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6433630017842720465', time=2026-02-25T00:25:59.262013+0000, chain.objs.size()=1 2026-02-24T22:34:25.454 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.okiiAD3LqoBecClbcyYWXmjmxXhN2yN_0 2026-02-24T22:34:25.454 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6450567968101702995', time=2026-02-25T00:25:59.327558+0000, chain.objs.size()=1 2026-02-24T22:34:25.454 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.pl0QtLscvCtvFKvfB8Ri1Whovx1J30R_0 2026-02-24T22:34:25.454 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.539596947699709081', time=2026-02-25T00:25:59.327969+0000, chain.objs.size()=1 2026-02-24T22:34:25.454 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.zpEyRVQN2Vt-8iZ6DsuCtLwdpHJvMxM_0 2026-02-24T22:34:25.454 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.27493958671353122', time=2026-02-25T00:25:59.446775+0000, chain.objs.size()=1 2026-02-24T22:34:25.454 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.g6sNMBPk6gygH_MmzIlEb1tVIq41pRx_0 2026-02-24T22:34:25.454 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1634629640348667206', time=2026-02-25T00:25:59.453742+0000, chain.objs.size()=1 2026-02-24T22:34:25.454 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.463+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.H2EYHd3bibbJkGOJj8t29rqYCj5RWb-_0 2026-02-24T22:34:25.454 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3116974575357111015', time=2026-02-25T00:25:59.456692+0000, chain.objs.size()=1 2026-02-24T22:34:25.454 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.F6IXVreBXo5TVW6aye5JTd0KsJ8fr0g_0 2026-02-24T22:34:25.454 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1271757668998662397', time=2026-02-25T00:25:59.472490+0000, chain.objs.size()=1 2026-02-24T22:34:25.454 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.u8EXI0ETCgbJ87mX4jtnU6YM1CZLP1O_0 2026-02-24T22:34:25.454 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13646284675817896404', time=2026-02-25T00:25:59.481875+0000, chain.objs.size()=1 2026-02-24T22:34:25.455 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Cif7oLDaXhIgUjfZzoGYwaAAv5ERdER_0 2026-02-24T22:34:25.455 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10497946902175525116', time=2026-02-25T00:25:59.558237+0000, chain.objs.size()=1 2026-02-24T22:34:25.455 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.D3fAwoRL0t8_Buuz-1insGXxaJE_QY8_0 2026-02-24T22:34:25.455 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.243579753561346350', time=2026-02-25T00:25:59.565111+0000, chain.objs.size()=1 2026-02-24T22:34:25.455 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.RM6SMkB7Ng6ap0zVvnG3G5MLWcinFlh_0 2026-02-24T22:34:25.455 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7681207147421690525', time=2026-02-25T00:25:59.634153+0000, chain.objs.size()=1 2026-02-24T22:34:25.455 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.7tjuehm-8-V5jnSF3HfgK-2lyrgOnTJ_0 2026-02-24T22:34:25.455 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13271478318594694823', time=2026-02-25T00:25:59.726767+0000, chain.objs.size()=1 2026-02-24T22:34:25.455 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.3BSx0pr22dNiqBH890mi0F3htmyh5ig_0 2026-02-24T22:34:25.455 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12607809622410333456', time=2026-02-25T00:25:59.759919+0000, chain.objs.size()=1 2026-02-24T22:34:25.455 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.8IRPAY4GmkEy8EXfwDHNRd6mwFVcUqd_0 2026-02-24T22:34:25.455 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10233260227808029357', time=2026-02-25T00:25:59.800755+0000, chain.objs.size()=1 2026-02-24T22:34:25.455 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.8wWMd8v7x4PzDh15WY19elwPG950zUl_0 2026-02-24T22:34:25.455 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6308849143233449437', time=2026-02-25T00:25:59.812562+0000, chain.objs.size()=1 2026-02-24T22:34:25.455 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ygIDX31lmL7rWCIUPv-cDqyhcCf54_-_0 2026-02-24T22:34:25.455 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11399362807263029144', time=2026-02-25T00:25:59.894026+0000, chain.objs.size()=1 2026-02-24T22:34:25.455 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.1GB0VIfIARKd39gK7WUpKWlgcBgE1PU_0 2026-02-24T22:34:25.455 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11569518360681556866', time=2026-02-25T00:25:59.894612+0000, chain.objs.size()=1 2026-02-24T22:34:25.456 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Ig-TP-qBzCwZ5xN8A5V0FiND49M3xTo_0 2026-02-24T22:34:25.456 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13675851739084157083', time=2026-02-25T00:25:59.905276+0000, chain.objs.size()=1 2026-02-24T22:34:25.456 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.0QgTk5cfVmvuhiTGNk3kGI0HwTcFQP0_0 2026-02-24T22:34:25.456 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12881704987028391754', time=2026-02-25T00:25:59.913563+0000, chain.objs.size()=1 2026-02-24T22:34:25.456 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.hC8Wck-3SVVukUJWWFL44oHzQcm848O_0 2026-02-24T22:34:25.456 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18431167149108020727', time=2026-02-25T00:25:59.936814+0000, chain.objs.size()=1 2026-02-24T22:34:25.456 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ohhb_i0LNJVQKJrt3E5zevgoZ_vukHh_0 2026-02-24T22:34:25.456 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16299471259921315661', time=2026-02-25T00:25:59.959850+0000, chain.objs.size()=1 2026-02-24T22:34:25.456 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.5dPEUKIVOlGSmFL3vgqJ1lyRlRoEtiR_0 2026-02-24T22:34:25.456 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1226012103976169885', time=2026-02-25T00:25:59.968726+0000, chain.objs.size()=1 2026-02-24T22:34:25.456 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Rmg58as8siZ_iSL-aU2ny6lsuM1bTHx_0 2026-02-24T22:34:25.456 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3444756812360533897', time=2026-02-25T00:26:00.049930+0000, chain.objs.size()=1 2026-02-24T22:34:25.456 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.mSC_V6o1CMN2uyPZOdkE64V2_JZ74kF_0 2026-02-24T22:34:25.457 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15892515336991924755', time=2026-02-25T00:26:00.067192+0000, chain.objs.size()=1 2026-02-24T22:34:25.457 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.7CYz3KkAL9ehRoEa1yDTdqpqzu6YMue_0 2026-02-24T22:34:25.457 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3554482622158138183', time=2026-02-25T00:26:00.085478+0000, chain.objs.size()=1 2026-02-24T22:34:25.457 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.2OgeeJzCU6xaf36nknMEz74gTYSM8lP_0 2026-02-24T22:34:25.457 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8579791186611664453', time=2026-02-25T00:26:00.089850+0000, chain.objs.size()=1 2026-02-24T22:34:25.457 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.9duQ0POOjLevpHIq-v0fzdEnCGtqDGH_0 2026-02-24T22:34:25.457 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8282580890614058117', time=2026-02-25T00:26:00.093240+0000, chain.objs.size()=1 2026-02-24T22:34:25.457 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.6Lkcr0Hp_Ww-hS29JPZDtEtuo-qVM8F_0 2026-02-24T22:34:25.457 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11971770467338425366', time=2026-02-25T00:26:00.132526+0000, chain.objs.size()=1 2026-02-24T22:34:25.457 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.jbehzM3ZqWEuEKHXo5Xcheqtcy-Fj3C_0 2026-02-24T22:34:25.457 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11820835167530450338', time=2026-02-25T00:26:00.219961+0000, chain.objs.size()=1 2026-02-24T22:34:25.457 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.OJQrCc90NXGkX85G6slPxaolYZcrQn3_0 2026-02-24T22:34:25.457 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16047733633178619603', time=2026-02-25T00:26:00.223601+0000, chain.objs.size()=1 2026-02-24T22:34:25.457 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.lXEoNv_nVODr1nqGV5jrliUR4quSl3t_0 2026-02-24T22:34:25.457 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3655033107419701385', time=2026-02-25T00:26:00.240630+0000, chain.objs.size()=1 2026-02-24T22:34:25.457 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.D4Pp8G1QgLUzDmfzy9AlZ_XOJOqKbXS_0 2026-02-24T22:34:25.457 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9941525533975798609', time=2026-02-25T00:26:00.278741+0000, chain.objs.size()=1 2026-02-24T22:34:25.457 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.N1AmCMJjOnQRONElHN5uOdYLyCGYg68_0 2026-02-24T22:34:25.457 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3377462933583732844', time=2026-02-25T00:26:01.934334+0000, chain.objs.size()=1 2026-02-24T22:34:25.457 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1631__shadow_.dmnhHnIFNf88in-99ZRIw_0zOl5i7SG_0 2026-02-24T22:34:25.457 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6604894757392807170', time=2026-02-25T00:26:10.610732+0000, chain.objs.size()=1 2026-02-24T22:34:25.457 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1646__shadow_.8peWnrhza3WGArIa-lctkZ0yH9pn55U_0 2026-02-24T22:34:25.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14775161026413018248', time=2026-02-25T00:26:12.367491+0000, chain.objs.size()=1 2026-02-24T22:34:25.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1649__shadow_.ZCMdF5BR4hyt7CJTEWfzSXiXlVb8DHC_0 2026-02-24T22:34:25.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3840624361515221899', time=2026-02-25T00:26:16.011495+0000, chain.objs.size()=1 2026-02-24T22:34:25.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1655__shadow_.burocVfRGavcdXDIUVCiKTMsNW7uzJj_0 2026-02-24T22:34:25.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9638585255794140217', time=2026-02-25T00:26:18.495157+0000, chain.objs.size()=1 2026-02-24T22:34:25.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1658__shadow_.HshwN451Ejt-PpwyWJgEPQGlJlg7-M7_0 2026-02-24T22:34:25.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15883407905682722072', time=2026-02-25T00:26:19.455746+0000, chain.objs.size()=1 2026-02-24T22:34:25.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1659__shadow_.8pTtdvHXn_DQfEVMkbh0WSc8KoS1PN8_0 2026-02-24T22:34:25.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3095289540986479035', time=2026-02-25T00:26:21.658565+0000, chain.objs.size()=1 2026-02-24T22:34:25.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1663__shadow_.UOGC9qM1WpKQG4LQirYZFf7VFj1NZaj_0 2026-02-24T22:34:25.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5559603909064580441', time=2026-02-25T00:26:44.778048+0000, chain.objs.size()=1 2026-02-24T22:34:25.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.y3tFk24ibPc1RJ98V7tIY4gHcUVdX5B_0 2026-02-24T22:34:25.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12279403727652820779', time=2026-02-25T00:26:44.785640+0000, chain.objs.size()=1 2026-02-24T22:34:25.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.7OkdUSzQkaiUPyDRfYAVa25Y6QygFBY_0 2026-02-24T22:34:25.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15141618861807137334', time=2026-02-25T00:26:44.814053+0000, chain.objs.size()=1 2026-02-24T22:34:25.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.hij1kEUxulojYQmEPlAWMq-rFYh59U0_0 2026-02-24T22:34:25.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8281846039581496082', time=2026-02-25T00:26:44.833652+0000, chain.objs.size()=1 2026-02-24T22:34:25.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.DJw46g2znWEFw68Ph4gn5jOzRpf-1sJ_0 2026-02-24T22:34:25.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5070807164776440135', time=2026-02-25T00:26:44.845521+0000, chain.objs.size()=1 2026-02-24T22:34:25.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.NcrGgp3o7H6Q5idBdeW67EWbu9RUTBr_0 2026-02-24T22:34:25.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4126958635025452942', time=2026-02-25T00:26:44.939051+0000, chain.objs.size()=1 2026-02-24T22:34:25.458 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.467+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.aBJ12pV04dwzx1Y90FrMYdJjHXOyCDk_0 2026-02-24T22:34:25.459 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8544452462472543088', time=2026-02-25T00:26:44.943719+0000, chain.objs.size()=1 2026-02-24T22:34:25.459 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.g1YOJUOv4tnCrj0sxFjQ0LsIQ3evz2g_0 2026-02-24T22:34:25.459 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1661301097756452292', time=2026-02-25T00:26:44.969339+0000, chain.objs.size()=1 2026-02-24T22:34:25.459 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.zgqPlOLuPMiVwAXvm14fvDWAt5aE7Au_0 2026-02-24T22:34:25.459 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8969297543069134209', time=2026-02-25T00:26:44.972167+0000, chain.objs.size()=1 2026-02-24T22:34:25.459 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_._3x4awKA-Ib4Vo21dE35JjBnB_RsRdU_0 2026-02-24T22:34:25.459 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8971248783455852181', time=2026-02-25T00:26:44.972694+0000, chain.objs.size()=1 2026-02-24T22:34:25.459 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.hCNgJ6QgqGtZ9MbMH3mBjoHamz2F_d0_0 2026-02-24T22:34:25.459 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5455775927938951044', time=2026-02-25T00:26:45.019352+0000, chain.objs.size()=1 2026-02-24T22:34:25.459 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.IdpKxVl1KGCSd1CcOSBOTCLhLOBgkhP_0 2026-02-24T22:34:25.459 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.451795156860665686', time=2026-02-25T00:26:45.134019+0000, chain.objs.size()=1 2026-02-24T22:34:25.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.nYuKqiq9dClVOLHczfkmwzYJX0rkVhs_0 2026-02-24T22:34:25.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16431949452617268758', time=2026-02-25T00:26:45.250221+0000, chain.objs.size()=1 2026-02-24T22:34:25.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.pIR4IS6EwiM_apCgqzF7q03Di8A4dcC_0 2026-02-24T22:34:25.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14584443808995892788', time=2026-02-25T00:26:45.255425+0000, chain.objs.size()=1 2026-02-24T22:34:25.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.bj8LPmFwE2b1FKfda_g1hIWXuA6I4fB_0 2026-02-24T22:34:25.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10345877511673237664', time=2026-02-25T00:26:45.329411+0000, chain.objs.size()=1 2026-02-24T22:34:25.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.fIfd5njdWmzfr_cXCFfEoKjYFtgmMW-_0 2026-02-24T22:34:25.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10440311243476662151', time=2026-02-25T00:26:45.397931+0000, chain.objs.size()=1 2026-02-24T22:34:25.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ivMhBG2-O_LhSr4JObWWWMn2iIvKYHH_0 2026-02-24T22:34:25.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9735805530498017433', time=2026-02-25T00:26:45.398395+0000, chain.objs.size()=1 2026-02-24T22:34:25.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.MViO3EYm_gZxGkahBpyHkoy49lkvxbp_0 2026-02-24T22:34:25.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7514474963130516735', time=2026-02-25T00:26:45.448798+0000, chain.objs.size()=1 2026-02-24T22:34:25.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.uORDiX3b6TvUAaLbZAA8qdkiBu0m54i_0 2026-02-24T22:34:25.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18439258647345352305', time=2026-02-25T00:26:45.453028+0000, chain.objs.size()=1 2026-02-24T22:34:25.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Bdy5YYHLrM1rWr2Xp6-yBOnONucI9Ko_0 2026-02-24T22:34:25.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11903933779904605868', time=2026-02-25T00:26:45.471789+0000, chain.objs.size()=1 2026-02-24T22:34:25.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.MmKrTylZFIE8q8MhTtAKyy1UUdaDeRd_0 2026-02-24T22:34:25.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10138406747713545137', time=2026-02-25T00:26:45.477532+0000, chain.objs.size()=1 2026-02-24T22:34:25.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Z0y8gvMpCmDsgNMPhZT6faP74NYZrDi_0 2026-02-24T22:34:25.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3177353955797410695', time=2026-02-25T00:26:45.477959+0000, chain.objs.size()=1 2026-02-24T22:34:25.460 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.FuCEXh1wf6rpQdANhARq4WswOflehir_0 2026-02-24T22:34:25.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17854873758762705155', time=2026-02-25T00:26:45.575426+0000, chain.objs.size()=1 2026-02-24T22:34:25.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.n-1pp84ynmxJd9ho5isCxvUoxiM8LNi_0 2026-02-24T22:34:25.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15202868652098594221', time=2026-02-25T00:26:45.581954+0000, chain.objs.size()=1 2026-02-24T22:34:25.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.aED0IAXyiy1qnVHdcjDSY5SqYp3pmKS_0 2026-02-24T22:34:25.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5236505188011914562', time=2026-02-25T00:26:45.817582+0000, chain.objs.size()=1 2026-02-24T22:34:25.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ywK4-tmW6jNv73rFDbeGf-QYmS2pxI5_0 2026-02-24T22:34:25.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8566932926716015780', time=2026-02-25T00:26:45.927565+0000, chain.objs.size()=1 2026-02-24T22:34:25.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.8Z5N7spa7QsepR3RPtqhmOVrNA4X6TD_0 2026-02-24T22:34:25.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7372284635210252532', time=2026-02-25T00:26:45.929897+0000, chain.objs.size()=1 2026-02-24T22:34:25.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.1s_bDwTjwaf_1ISmZ7EKWeNagMhbTcp_0 2026-02-24T22:34:25.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1910230154686515588', time=2026-02-25T00:26:45.949917+0000, chain.objs.size()=1 2026-02-24T22:34:25.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.nI5fSv5DjDseL4kONpwBs-Uy8e9fEKR_0 2026-02-24T22:34:25.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9951372966023649621', time=2026-02-25T00:26:45.950907+0000, chain.objs.size()=1 2026-02-24T22:34:25.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.JJJcIItsUk63cjPrdvccMOxFefuBNg-_0 2026-02-24T22:34:25.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2970625361551824027', time=2026-02-25T00:26:45.965075+0000, chain.objs.size()=1 2026-02-24T22:34:25.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Oo2YflvPD8Aci6kVvio5r8kuCTwv_z3_0 2026-02-24T22:34:25.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9424145754079004896', time=2026-02-25T00:26:45.972134+0000, chain.objs.size()=1 2026-02-24T22:34:25.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.NVL3kAORA4EMb07rUkxtVdqo_HZd5WN_0 2026-02-24T22:34:25.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15731052055799991238', time=2026-02-25T00:26:45.987993+0000, chain.objs.size()=1 2026-02-24T22:34:25.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.uzZ0y7c5DIZ8PTjn06T5HrplJmfKjYf_0 2026-02-24T22:34:25.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9825520095695664664', time=2026-02-25T00:26:46.125729+0000, chain.objs.size()=1 2026-02-24T22:34:25.461 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.471+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.gBddlQtsVEct7zaJepUuRDst6PIxvLz_0 2026-02-24T22:34:25.462 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/37536 2026-02-24T22:34:25.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=40, truncated=0, next_marker='' 2026-02-24T22:34:25.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1800576863831725465', time=2026-02-25T00:26:53.138843+0000, chain.objs.size()=1 2026-02-24T22:34:25.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.O-26avQ1CGNvDJt8mxd4YTD9Zip99_A_0 2026-02-24T22:34:25.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17095434206797812057', time=2026-02-25T00:26:53.177267+0000, chain.objs.size()=1 2026-02-24T22:34:25.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.jefzeHhjXFm2Xre15fZIeLcOg-JqmK8_0 2026-02-24T22:34:25.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11584701792332818489', time=2026-02-25T00:26:53.397124+0000, chain.objs.size()=1 2026-02-24T22:34:25.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.4DTDbTXds4XUjoutIf0h11IC8slxSE4_0 2026-02-24T22:34:25.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9863111101130023253', time=2026-02-25T00:26:53.414642+0000, chain.objs.size()=1 2026-02-24T22:34:25.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.XreOmPPR9tnkWkKqJjZnLwonoLOAoiz_0 2026-02-24T22:34:25.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2341737398021346064', time=2026-02-25T00:26:53.434330+0000, chain.objs.size()=1 2026-02-24T22:34:25.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.zTDd848xfQIEuJn0oA8stn6-XytQDIh_0 2026-02-24T22:34:25.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6399484485757878537', time=2026-02-25T00:26:53.540096+0000, chain.objs.size()=1 2026-02-24T22:34:25.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.uS-qYgWiL-DGuZ-iu3nA803mNlHz3d5_0 2026-02-24T22:34:25.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12815934757675594584', time=2026-02-25T00:26:53.562196+0000, chain.objs.size()=1 2026-02-24T22:34:25.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.MSGq4-wKnl5ilyR57bnHL7rrfsiBRvO_0 2026-02-24T22:34:25.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8262650196608592639', time=2026-02-25T00:26:53.570728+0000, chain.objs.size()=1 2026-02-24T22:34:25.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.l0By18dNPjeVxcn4TlY-O6HpgOnH0OC_0 2026-02-24T22:34:25.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2587089805831722100', time=2026-02-25T00:26:53.601574+0000, chain.objs.size()=1 2026-02-24T22:34:25.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.qCsNXngLz1JU79-MQUOc4G49o0Vp2Zc_0 2026-02-24T22:34:25.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9819100564001123414', time=2026-02-25T00:26:53.918519+0000, chain.objs.size()=1 2026-02-24T22:34:25.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.O00zcgdAMySoER_SN-yiWTTWXz_Ey0E_0 2026-02-24T22:34:25.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2060451870158931446', time=2026-02-25T00:26:53.937091+0000, chain.objs.size()=1 2026-02-24T22:34:25.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.9W2MoOwZHQ7v8ZWhgmQifzczpdjUzru_0 2026-02-24T22:34:25.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1237014062332627263', time=2026-02-25T00:26:53.979169+0000, chain.objs.size()=1 2026-02-24T22:34:25.464 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.0i8Q_s7dGQj33MwXX90WxeGam3M8gSf_0 2026-02-24T22:34:25.465 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15085036326592990528', time=2026-02-25T00:26:54.006177+0000, chain.objs.size()=1 2026-02-24T22:34:25.465 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.RdmI7P5ehVsGRbciT8frez1N8s-eNUh_0 2026-02-24T22:34:25.465 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1171700395871172411', time=2026-02-25T00:26:54.171288+0000, chain.objs.size()=1 2026-02-24T22:34:25.465 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.evcXmnfg-bKFF7qecY5eH7H4vq3PbNE_0 2026-02-24T22:34:25.465 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11464922439836385610', time=2026-02-25T00:26:54.187681+0000, chain.objs.size()=1 2026-02-24T22:34:25.465 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.adeVp-m_Twp3Dh5VV9opYXn1oVnkVFG_0 2026-02-24T22:34:25.465 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13775572455940652909', time=2026-02-25T00:26:54.211016+0000, chain.objs.size()=1 2026-02-24T22:34:25.465 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.hfK57uftkqPAw62ReouHZvJ2wTrEbp3_0 2026-02-24T22:34:25.465 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1544267064741776354', time=2026-02-25T00:26:54.312261+0000, chain.objs.size()=1 2026-02-24T22:34:25.465 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.qEuesYGKghg_XaoI3Ydi2S0sjYX-LDJ_0 2026-02-24T22:34:25.465 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4534152477382548071', time=2026-02-25T00:26:54.321684+0000, chain.objs.size()=1 2026-02-24T22:34:25.465 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.OwoqWPRpBDNqholv25F-jF_1I91HJnu_0 2026-02-24T22:34:25.465 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12298742963525971938', time=2026-02-25T00:26:54.326721+0000, chain.objs.size()=1 2026-02-24T22:34:25.465 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.qEO1EYdZgj24Xg38C3g_UmvqtPrB-og_0 2026-02-24T22:34:25.465 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5566235145648796143', time=2026-02-25T00:26:54.452574+0000, chain.objs.size()=1 2026-02-24T22:34:25.465 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.LHdpqb8AUTuDYDlrPGPF-jndFLwEEQV_0 2026-02-24T22:34:25.465 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14606596989875293762', time=2026-02-25T00:26:54.490886+0000, chain.objs.size()=1 2026-02-24T22:34:25.465 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.nMWFPgIY3Xvmam19QwWWHKL9Zd8570M_0 2026-02-24T22:34:25.465 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9608062116736118096', time=2026-02-25T00:26:54.494790+0000, chain.objs.size()=1 2026-02-24T22:34:25.465 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.IRagfOvG-HZm9BiHtoQaW0m1py3zaza_0 2026-02-24T22:34:25.466 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16147376758696248086', time=2026-02-25T00:26:54.498339+0000, chain.objs.size()=1 2026-02-24T22:34:25.466 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.mlGw5MXFWKZE6zMMZnN91CUcYENagOP_0 2026-02-24T22:34:25.466 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5870156554509984322', time=2026-02-25T00:26:54.508908+0000, chain.objs.size()=1 2026-02-24T22:34:25.466 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.t1TY0q7WVgb248qm2d_rGBi8-tUeryS_0 2026-02-24T22:34:25.466 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17402670559635949039', time=2026-02-25T00:27:00.752591+0000, chain.objs.size()=1 2026-02-24T22:34:25.466 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1700__shadow_.Xf0dlszqV_EljsB_nm_Ez8Qaydfufzk_0 2026-02-24T22:34:25.466 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3602606986317572767', time=2026-02-25T00:28:32.540481+0000, chain.objs.size()=1 2026-02-24T22:34:25.466 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1849__shadow_.tFPNgnvgdX0b8d_grQzKwMWe7W5_zjG_0 2026-02-24T22:34:25.466 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12314890725960719529', time=2026-02-25T00:28:41.359452+0000, chain.objs.size()=1 2026-02-24T22:34:25.466 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1865__shadow_.FUxahofO3CGKH8-QgsARNb4V9uhf5VO_0 2026-02-24T22:34:25.466 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11893444883028891979', time=2026-02-25T00:28:43.753465+0000, chain.objs.size()=1 2026-02-24T22:34:25.466 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1869__shadow:ba1nU1fNR3N1sfUQ2kL8K8Wm.5Ibl31_.IVNpljy9O7AjWtEyJ7RLhDmshakOZnh_0 2026-02-24T22:34:25.466 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4465693929652501133', time=2026-02-25T00:30:15.850535+0000, chain.objs.size()=1 2026-02-24T22:34:25.466 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1934__shadow_.P4ND69j_raBuC7NPEesozw4YddYOnNS_0 2026-02-24T22:34:25.466 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15943166581182038026', time=2026-02-25T00:30:19.143384+0000, chain.objs.size()=1 2026-02-24T22:34:25.466 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:bbfnDgG1vU91jvIozUBvRH13if8aKvj_.mn0IphyVVqTesr85FYteqWUwcdCsr0G_0 2026-02-24T22:34:25.466 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14948275461790886014', time=2026-02-25T00:30:19.555004+0000, chain.objs.size()=1 2026-02-24T22:34:25.466 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:TXMZtLm-EGfC.kqJAL5fiOT1WHmtSe1_.ZTrkFkmvGuFf3K3rmZgc4Rcc9VxYnZT_0 2026-02-24T22:34:25.467 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15858332036816599110', time=2026-02-25T00:30:26.331314+0000, chain.objs.size()=1 2026-02-24T22:34:25.467 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.475+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1941__shadow:OQXs4iNL9Cl6CElSvxB5AfNnL71NP0W_.shq4Ma8PiBelF0YTBi_gGKHKXrHJXFt_0 2026-02-24T22:34:25.467 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.479+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1674753311179924864', time=2026-02-25T00:30:26.382871+0000, chain.objs.size()=1 2026-02-24T22:34:25.467 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.479+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1941__shadow_.5W5WNpTuOSiCfnjk6VwwsrSB8K9Nxha_0 2026-02-24T22:34:25.467 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.479+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8623441910039647336', time=2026-02-25T00:30:37.735767+0000, chain.objs.size()=1 2026-02-24T22:34:25.467 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.479+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:ySI9iJIDeY6AsBA5-UxRCkdzmgm0Ygh_.7-7j7Z-cPyhf0oU-jWTAS-BGTgxe3Ki_0 2026-02-24T22:34:25.467 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.479+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8590620921874275784', time=2026-02-25T00:30:43.983307+0000, chain.objs.size()=1 2026-02-24T22:34:25.467 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.479+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:UxTLB947yQ3mrPZtkdwRvBLANcDGIo7_.2G7ZydeOlwrD7JX0wvOsDKsT6-8uMgj_0 2026-02-24T22:34:25.467 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.479+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15981760896680604986', time=2026-02-25T00:30:52.383095+0000, chain.objs.size()=1 2026-02-24T22:34:25.467 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.479+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1950__shadow:1J0Mgj4PHIQwvlYxfMGpbYQEahXIYwp_.zUTs7Goq8b7TeuXXHhAVDabGV0-iy_X_0 2026-02-24T22:34:25.467 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.479+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8954446510127989738', time=2026-02-25T00:30:58.535862+0000, chain.objs.size()=1 2026-02-24T22:34:25.467 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.479+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958__shadow:eBVm-OA4wrJgQWyOaB1q0pjrddDnsp7_.ft-AJ9wBMC2E9n4kMmg8hhPL0PMu-cH_0 2026-02-24T22:34:25.467 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.479+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8954446510127989738', time=2026-02-25T00:30:58.536301+0000, chain.objs.size()=1 2026-02-24T22:34:25.467 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.479+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958__shadow:eBVm-OA4wrJgQWyOaB1q0pjrddDnsp7_.ft-AJ9wBMC2E9n4kMmg8hhPL0PMu-cH_0 2026-02-24T22:34:25.467 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.479+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8954446510127989738', time=2026-02-25T00:30:58.536752+0000, chain.objs.size()=1 2026-02-24T22:34:25.467 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.479+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958__shadow:eBVm-OA4wrJgQWyOaB1q0pjrddDnsp7_.ft-AJ9wBMC2E9n4kMmg8hhPL0PMu-cH_0 2026-02-24T22:34:25.467 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.479+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16697271003281220784', time=2026-02-25T00:33:53.650065+0000, chain.objs.size()=1 2026-02-24T22:34:25.467 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.479+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1975__shadow_.dClV_o3GO_IeNQ64fA6D53tWY-PmOxx_0 2026-02-24T22:34:25.468 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.479+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.469 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.479+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2026-02-24T22:34:25.470 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.471 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.471 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/42897' 2026-02-24T22:34:25.472 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14943672638169667292', time=2026-02-25T00:15:34.689231+0000, chain.objs.size()=4 2026-02-24T22:34:25.472 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1510__multipart_foo.2~C_s_8O1FVHE1j3RXfNFuXkknTNKZslW.1 2026-02-24T22:34:25.472 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1510__multipart_foo.2~C_s_8O1FVHE1j3RXfNFuXkknTNKZslW.2 2026-02-24T22:34:25.472 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1510__multipart_foo.2~C_s_8O1FVHE1j3RXfNFuXkknTNKZslW.3 2026-02-24T22:34:25.472 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1510__multipart_foo.2~C_s_8O1FVHE1j3RXfNFuXkknTNKZslW.4 2026-02-24T22:34:25.472 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9762428589688093258', time=2026-02-25T00:15:34.763242+0000, chain.objs.size()=3 2026-02-24T22:34:25.472 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-LUKEWARM.2~eOtMH0vmpN_usdE1y1Q_d9UHSttRW0T.1 2026-02-24T22:34:25.472 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-LUKEWARM.2~eOtMH0vmpN_usdE1y1Q_d9UHSttRW0T.2 2026-02-24T22:34:25.472 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-LUKEWARM.2~eOtMH0vmpN_usdE1y1Q_d9UHSttRW0T.3 2026-02-24T22:34:25.472 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6188250287384080910', time=2026-02-25T00:16:32.617368+0000, chain.objs.size()=1 2026-02-24T22:34:25.472 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520__shadow_.-a5UfmxIlfrghSqHT7cNMGkgU7B_hIN_0 2026-02-24T22:34:25.472 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.718925658826320810', time=2026-02-25T00:17:07.124570+0000, chain.objs.size()=1 2026-02-24T22:34:25.472 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523__shadow_.wfqbW1wbihaCijBcahambpCpVYru22U_0 2026-02-24T22:34:25.472 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18281584929205552345', time=2026-02-25T00:17:08.013150+0000, chain.objs.size()=1 2026-02-24T22:34:25.472 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524__shadow_.UTxQJRyohwBYmy-BdOEppj20GG0CQzV_0 2026-02-24T22:34:25.472 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15141568013291411200', time=2026-02-25T00:18:51.099173+0000, chain.objs.size()=1 2026-02-24T22:34:25.472 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532__shadow_.0fTo7xu0nSN8yka8SP3VYCCwXE4NWuV_0 2026-02-24T22:34:25.472 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6601310123866435656', time=2026-02-25T00:20:30.736431+0000, chain.objs.size()=1 2026-02-24T22:34:25.472 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537__shadow_.ojTS8_8OZoDCjjeZXM_bCD_-pKrUQ8D_0 2026-02-24T22:34:25.472 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13779353787623516505', time=2026-02-25T00:20:34.039512+0000, chain.objs.size()=1 2026-02-24T22:34:25.472 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539__shadow_.E3BqdmYEnT0fCtkvGqZhFnyARZCGRn7_0 2026-02-24T22:34:25.472 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6406974207123235937', time=2026-02-25T00:20:34.044272+0000, chain.objs.size()=1 2026-02-24T22:34:25.472 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539__shadow_.4TLa-H9dhVCduJsrl7qVZIWm2TWQENK_0 2026-02-24T22:34:25.473 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17540545846175716950', time=2026-02-25T00:21:06.321897+0000, chain.objs.size()=1 2026-02-24T22:34:25.473 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540__shadow_._qxwBDbQG63x1SRdWwpY5jQDT3VuYxG_0 2026-02-24T22:34:25.473 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2707756022327799878', time=2026-02-25T00:23:25.743465+0000, chain.objs.size()=1 2026-02-24T22:34:25.473 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519__shadow:nAzJ9H0aQB5SqgvkGh379ny-Ftet3lh_.kmbrHg69AGSMY-VhimhZySoGSQQO6dJ_0 2026-02-24T22:34:25.473 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='2~GyftM8VVbGi524EtSIwOmUOO4faqlVp', time=2026-02-25T00:24:14.937300+0000, chain.objs.size()=1 2026-02-24T22:34:25.473 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1568__multipart_mymultipart.2~GyftM8VVbGi524EtSIwOmUOO4faqlVp.1 2026-02-24T22:34:25.473 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14541125817140614078', time=2026-02-25T00:25:35.441697+0000, chain.objs.size()=1 2026-02-24T22:34:25.473 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1600__shadow_.4elNBwJUGoPCtkcfkEDPJsV1TGOYd5P_0 2026-02-24T22:34:25.473 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18304360606041314847', time=2026-02-25T00:25:37.543002+0000, chain.objs.size()=1 2026-02-24T22:34:25.473 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1603__shadow_.Ku8PSZ-K6ACRGAZ95YXIUApCjIyAEky_0 2026-02-24T22:34:25.473 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7945950610291165416', time=2026-02-25T00:25:37.543624+0000, chain.objs.size()=1 2026-02-24T22:34:25.473 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1603__shadow_.buSH4sRqrBgCDO5WOROcQcia9KyGV4o_0 2026-02-24T22:34:25.473 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.936294192244482669', time=2026-02-25T00:25:40.170347+0000, chain.objs.size()=1 2026-02-24T22:34:25.473 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1607__shadow_.nziIxZV0gk5ZwBvN43NdaZiPudsadTS_0 2026-02-24T22:34:25.473 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3684866344777143214', time=2026-02-25T00:25:52.537693+0000, chain.objs.size()=1 2026-02-24T22:34:25.473 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1627__shadow_.0UGef4RyVDZvk4FoWk3BHEphJ6Qizr7_0 2026-02-24T22:34:25.473 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15365274778331964635', time=2026-02-25T00:25:59.086967+0000, chain.objs.size()=1 2026-02-24T22:34:25.473 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Yz-CLxCjmI4k-bkDZWFZM8UGGNhtrfr_0 2026-02-24T22:34:25.473 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12419799803911982669', time=2026-02-25T00:25:59.092076+0000, chain.objs.size()=1 2026-02-24T22:34:25.473 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Qg_zfAkOY1v7b6xU3IokqJPKZ4GuvJd_0 2026-02-24T22:34:25.474 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3148709733605867218', time=2026-02-25T00:25:59.114484+0000, chain.objs.size()=1 2026-02-24T22:34:25.474 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.JrDe10Jbw5RSFdnUi3-7XlD_oyjpqB__0 2026-02-24T22:34:25.474 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14702366686005597679', time=2026-02-25T00:25:59.156708+0000, chain.objs.size()=1 2026-02-24T22:34:25.474 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.o3HqecOB_0gPC_yyl22QmMgAIJqeSd3_0 2026-02-24T22:34:25.474 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8403918995576899617', time=2026-02-25T00:25:59.175900+0000, chain.objs.size()=1 2026-02-24T22:34:25.474 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.rXrvw1Fi1FcqRrr09nKRbq82IaxWMbI_0 2026-02-24T22:34:25.474 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3105895693534427177', time=2026-02-25T00:25:59.272897+0000, chain.objs.size()=1 2026-02-24T22:34:25.474 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.k51sX72t_l-5NT2aAoMyCB9UNxmZqKt_0 2026-02-24T22:34:25.474 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3270318133713242369', time=2026-02-25T00:25:59.347521+0000, chain.objs.size()=1 2026-02-24T22:34:25.474 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.svZHQBnA9JHQcWrqkroYeiWkOH9gA64_0 2026-02-24T22:34:25.474 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9013989997491913280', time=2026-02-25T00:25:59.431440+0000, chain.objs.size()=1 2026-02-24T22:34:25.474 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.lMiU_Ey2lvbjLkdhdnfJN1q_sssgCdp_0 2026-02-24T22:34:25.474 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6482724259779311116', time=2026-02-25T00:25:59.431804+0000, chain.objs.size()=1 2026-02-24T22:34:25.474 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.483+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.zRaIA3PvYahSL5V5-hvViU6z4eckMdW_0 2026-02-24T22:34:25.475 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1626422838516996346', time=2026-02-25T00:25:59.470403+0000, chain.objs.size()=1 2026-02-24T22:34:25.475 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.r33wNFkXwQ8cteYlJvuWiq56Pn00_3n_0 2026-02-24T22:34:25.475 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9253468338722860485', time=2026-02-25T00:25:59.577075+0000, chain.objs.size()=1 2026-02-24T22:34:25.475 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.uIXSznonYazL7PohJej15ElEytqyaZT_0 2026-02-24T22:34:25.475 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10996294573123953918', time=2026-02-25T00:25:59.584919+0000, chain.objs.size()=1 2026-02-24T22:34:25.475 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.yi7L65tNZGaSqD1-fgvuS7-lx5OC3hF_0 2026-02-24T22:34:25.475 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16676742814519684410', time=2026-02-25T00:25:59.635085+0000, chain.objs.size()=1 2026-02-24T22:34:25.475 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.LTj4v8Qp7oyb-GR3-vNT98z8IL3YrKH_0 2026-02-24T22:34:25.475 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16622905654622323129', time=2026-02-25T00:25:59.666681+0000, chain.objs.size()=1 2026-02-24T22:34:25.475 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.jt0RnUQSI01iYu79H4iMzyznWVaDuE3_0 2026-02-24T22:34:25.475 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8550723992730864484', time=2026-02-25T00:25:59.750234+0000, chain.objs.size()=1 2026-02-24T22:34:25.475 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Uy0VOKkA41zqzddwpEb_99TYcsolT0h_0 2026-02-24T22:34:25.475 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13783800058461711527', time=2026-02-25T00:25:59.754719+0000, chain.objs.size()=1 2026-02-24T22:34:25.475 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.85kZ6QjAcvoVWpE4KBew0CSVntakW04_0 2026-02-24T22:34:25.475 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14663968972026019222', time=2026-02-25T00:25:59.759688+0000, chain.objs.size()=1 2026-02-24T22:34:25.475 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.CUWNe-uVDXsCZhC0A_1IdhAN6h4l2SG_0 2026-02-24T22:34:25.475 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12918087626170286814', time=2026-02-25T00:25:59.787515+0000, chain.objs.size()=1 2026-02-24T22:34:25.475 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.-naiA9vN1dbWBfu0z4iF-69fs47ZVvL_0 2026-02-24T22:34:25.475 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13764436974596012997', time=2026-02-25T00:25:59.907130+0000, chain.objs.size()=1 2026-02-24T22:34:25.475 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.YkDNKwfOEL04akzndgqST26aJGENszm_0 2026-02-24T22:34:25.475 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7088160975043985960', time=2026-02-25T00:25:59.908522+0000, chain.objs.size()=1 2026-02-24T22:34:25.475 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.dBmfbO7tYsTvVC-UDJiNyCq26MB25vi_0 2026-02-24T22:34:25.476 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3668797837994114041', time=2026-02-25T00:25:59.938236+0000, chain.objs.size()=1 2026-02-24T22:34:25.476 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Rv49RNOGRjE4hheFfpyOvjlccD7JsPD_0 2026-02-24T22:34:25.476 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2794321915918555478', time=2026-02-25T00:25:59.943714+0000, chain.objs.size()=1 2026-02-24T22:34:25.476 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.jkSnYP-f3lmcy8V6g6-YjULBxhrOXIn_0 2026-02-24T22:34:25.476 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11650141813425943761', time=2026-02-25T00:25:59.959746+0000, chain.objs.size()=1 2026-02-24T22:34:25.476 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.rAU-JfhxFydyUbk9PwaNueb03pI1e3r_0 2026-02-24T22:34:25.476 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12278486079234198931', time=2026-02-25T00:25:59.979623+0000, chain.objs.size()=1 2026-02-24T22:34:25.476 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.g1vns7ORMzV4O2QFUJjRJNUBIa-_mLP_0 2026-02-24T22:34:25.477 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11316980912427610517', time=2026-02-25T00:25:59.980229+0000, chain.objs.size()=1 2026-02-24T22:34:25.477 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.VBhxDOBVQ4hrinKYf607Wy82dSzHVFs_0 2026-02-24T22:34:25.477 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9361730646289165318', time=2026-02-25T00:25:59.984018+0000, chain.objs.size()=1 2026-02-24T22:34:25.477 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.xgXxD9JzuegaXi1SYG4TO148HdbVyox_0 2026-02-24T22:34:25.477 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8004204914478060874', time=2026-02-25T00:26:00.085324+0000, chain.objs.size()=1 2026-02-24T22:34:25.477 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.DO8Rmen0XfO2bvzey6gZyJtQ7cS88bB_0 2026-02-24T22:34:25.477 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8119144756590847636', time=2026-02-25T00:26:00.100627+0000, chain.objs.size()=1 2026-02-24T22:34:25.477 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Ex3GKEDeMu15sHvqXOLs_nLOw6T8XGA_0 2026-02-24T22:34:25.477 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14739001912003290952', time=2026-02-25T00:26:00.131218+0000, chain.objs.size()=1 2026-02-24T22:34:25.477 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.gp1ZCq5fomK-5Nh2Ud_34BVVvYnhiVE_0 2026-02-24T22:34:25.477 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6413929074173252804', time=2026-02-25T00:26:00.148487+0000, chain.objs.size()=1 2026-02-24T22:34:25.477 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.rJ82ED_w3eGeOOSJzf4i8kCrTGgPYNP_0 2026-02-24T22:34:25.477 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6314059398095335086', time=2026-02-25T00:26:00.230026+0000, chain.objs.size()=1 2026-02-24T22:34:25.477 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.1_ElVcOSrrRZ8nt3iUb_-d1DXXT-ymL_0 2026-02-24T22:34:25.478 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13595572824869981761', time=2026-02-25T00:26:00.243864+0000, chain.objs.size()=1 2026-02-24T22:34:25.478 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Y5QPq_e-9XqATlcE5jOKvHAgimyExdO_0 2026-02-24T22:34:25.478 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8569046034949395433', time=2026-02-25T00:26:00.261763+0000, chain.objs.size()=1 2026-02-24T22:34:25.478 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.mMXL46M3smOScyY39nvJrKh-5tC-bRP_0 2026-02-24T22:34:25.478 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2903217044623383057', time=2026-02-25T00:26:00.265955+0000, chain.objs.size()=1 2026-02-24T22:34:25.478 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.4yyfyRk19OJPWHbSzFKv278n9gI5KEz_0 2026-02-24T22:34:25.478 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10226193048631821043', time=2026-02-25T00:26:10.038480+0000, chain.objs.size()=1 2026-02-24T22:34:25.478 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1645__shadow_.GfNhXt2qL9TqZSoLjDD8nBIgjotnkBR_0 2026-02-24T22:34:25.478 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2413731478827390318', time=2026-02-25T00:26:13.618813+0000, chain.objs.size()=1 2026-02-24T22:34:25.478 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1651__shadow_.5phGUtAfMJvPghxhBcAbtraVa-CxRxg_0 2026-02-24T22:34:25.478 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16968020228195421538', time=2026-02-25T00:26:18.524309+0000, chain.objs.size()=1 2026-02-24T22:34:25.478 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1658__shadow_.qJ-qzJxkEiNUKpS0JNnDTBrNBkbzvN0_0 2026-02-24T22:34:25.478 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18274999696376040809', time=2026-02-25T00:26:20.573579+0000, chain.objs.size()=1 2026-02-24T22:34:25.478 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1661__shadow_.ADz0JNPmHZ-hgpQOaX7k9KAQHfBvNKI_0 2026-02-24T22:34:25.478 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17290105264609207409', time=2026-02-25T00:26:21.137867+0000, chain.objs.size()=1 2026-02-24T22:34:25.478 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.487+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1662__shadow_.Jk5CN_d98HaSrNcC-R7ngb40Vrx3xxE_0 2026-02-24T22:34:25.478 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9976727848886128608', time=2026-02-25T00:26:44.806365+0000, chain.objs.size()=1 2026-02-24T22:34:25.478 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.6dYa9a0ZIfrRbwll5uz2-u2ia2i9gky_0 2026-02-24T22:34:25.478 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5687449311917993654', time=2026-02-25T00:26:44.807075+0000, chain.objs.size()=1 2026-02-24T22:34:25.478 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.dBQJ7V6G44NIAI3FKxOTCGeSBKDkGf-_0 2026-02-24T22:34:25.479 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7967399127551888982', time=2026-02-25T00:26:44.813573+0000, chain.objs.size()=1 2026-02-24T22:34:25.479 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.n5WSJRwt_OVFdZzYvzm6hxfmgDaMvHk_0 2026-02-24T22:34:25.479 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3629698608631522274', time=2026-02-25T00:26:44.827019+0000, chain.objs.size()=1 2026-02-24T22:34:25.479 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.aj95FkvVtO-Bkdq-1_ka_58GHmQx9Ds_0 2026-02-24T22:34:25.479 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2564946413259369094', time=2026-02-25T00:26:44.830577+0000, chain.objs.size()=1 2026-02-24T22:34:25.479 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Ksy5YRClybKaVBmqNQrjwixL1GTVy2w_0 2026-02-24T22:34:25.479 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1606426647895214821', time=2026-02-25T00:26:44.867443+0000, chain.objs.size()=1 2026-02-24T22:34:25.479 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.XQ6UVGtQdqumEsDCG6Ee6FJWZUFNSrQ_0 2026-02-24T22:34:25.479 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10339451391447306610', time=2026-02-25T00:26:44.950829+0000, chain.objs.size()=1 2026-02-24T22:34:25.479 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.VsPzhv4ktTCbG2mvwR8WOfN1ttvK_mK_0 2026-02-24T22:34:25.479 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6851471764451570962', time=2026-02-25T00:26:44.954239+0000, chain.objs.size()=1 2026-02-24T22:34:25.479 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.StcLkc8L5wPDMsale7QmV-Ex6YMiHw6_0 2026-02-24T22:34:25.479 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11302972580139278194', time=2026-02-25T00:26:45.111865+0000, chain.objs.size()=1 2026-02-24T22:34:25.479 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.xxY6DxCQmV3EVkVUEl_kuqIYH2QJ2BM_0 2026-02-24T22:34:25.479 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8797327154197662169', time=2026-02-25T00:26:45.140584+0000, chain.objs.size()=1 2026-02-24T22:34:25.479 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.bQSu_SuxruA3ViyVWnY6qCs8-r1hVZJ_0 2026-02-24T22:34:25.479 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6071029625842537007', time=2026-02-25T00:26:45.144280+0000, chain.objs.size()=1 2026-02-24T22:34:25.479 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ij15reyF6Rd4Xgx5bYpnckGWUELNIjf_0 2026-02-24T22:34:25.479 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12268703110854946073', time=2026-02-25T00:26:45.198733+0000, chain.objs.size()=1 2026-02-24T22:34:25.479 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.IaDcllzg76XM5gAlZTNDTQYTzHosMos_0 2026-02-24T22:34:25.480 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14759095373658091112', time=2026-02-25T00:26:45.273295+0000, chain.objs.size()=1 2026-02-24T22:34:25.480 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.NDYnGDmvZbj_kKnhU3vhak3cxIYBHq5_0 2026-02-24T22:34:25.480 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.820410465700022952', time=2026-02-25T00:26:45.292424+0000, chain.objs.size()=1 2026-02-24T22:34:25.480 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.bbh4-5jS9P3UtnW2bvgY0J-lS2uDX0p_0 2026-02-24T22:34:25.480 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15686213573493891814', time=2026-02-25T00:26:45.293202+0000, chain.objs.size()=1 2026-02-24T22:34:25.480 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.xoiOXSumlly3i5QbBPYqG9JsJsqRzCP_0 2026-02-24T22:34:25.480 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7889848032838295910', time=2026-02-25T00:26:45.296051+0000, chain.objs.size()=1 2026-02-24T22:34:25.480 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.FopxVqpBY1ZjHAVd9R1u3_IXTUB0Xgw_0 2026-02-24T22:34:25.480 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12130487247923319180', time=2026-02-25T00:26:45.317642+0000, chain.objs.size()=1 2026-02-24T22:34:25.480 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.vS9lUD1aCcTse6H0-Uu4MpXI3bF-Kvq_0 2026-02-24T22:34:25.480 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.166213013171641516', time=2026-02-25T00:26:45.338617+0000, chain.objs.size()=1 2026-02-24T22:34:25.480 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.xys0N2pToH_8XpP6Zudhu5yI46yJqGO_0 2026-02-24T22:34:25.480 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12557849104560957623', time=2026-02-25T00:26:45.340504+0000, chain.objs.size()=1 2026-02-24T22:34:25.480 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.wA8XjTASlQ1GR7KexEm2M4Uz1PYkts2_0 2026-02-24T22:34:25.480 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3887758285580679747', time=2026-02-25T00:26:45.418493+0000, chain.objs.size()=1 2026-02-24T22:34:25.480 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.XoWmKLpAk2HtKBWyOHSqTsQRYhBiJYO_0 2026-02-24T22:34:25.480 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12044412620831226378', time=2026-02-25T00:26:45.455956+0000, chain.objs.size()=1 2026-02-24T22:34:25.480 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.T2eM4u_zY4rHsc_vmsbV40dv8LpVhJo_0 2026-02-24T22:34:25.480 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11205460146588755334', time=2026-02-25T00:26:45.456491+0000, chain.objs.size()=1 2026-02-24T22:34:25.480 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.EqcXeNcbKvps60B_ahDfsX9a3uBxOMp_0 2026-02-24T22:34:25.480 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16631294989697498083', time=2026-02-25T00:26:45.458351+0000, chain.objs.size()=1 2026-02-24T22:34:25.480 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.wRYpp-4k0XV3MuFfVyR9_qhm_AsMUL0_0 2026-02-24T22:34:25.481 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12899655917546105708', time=2026-02-25T00:26:45.491978+0000, chain.objs.size()=1 2026-02-24T22:34:25.481 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.73XMSoSdhT6Y7Fh11z7gKICcJYTvfPl_0 2026-02-24T22:34:25.481 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8610279237659768201', time=2026-02-25T00:26:45.586155+0000, chain.objs.size()=1 2026-02-24T22:34:25.481 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ZYmnGrSGWZ4RKxZL_IibFITdTMXQHjv_0 2026-02-24T22:34:25.481 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13890800176931699293', time=2026-02-25T00:26:45.590673+0000, chain.objs.size()=1 2026-02-24T22:34:25.481 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.XS22AeuWtljafm7YcwOcqZN9HHSAK1g_0 2026-02-24T22:34:25.481 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4679566048862934331', time=2026-02-25T00:26:45.820621+0000, chain.objs.size()=1 2026-02-24T22:34:25.481 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.tIg5QiCqWPPCBs7CVTutdyvy4mdsyjf_0 2026-02-24T22:34:25.481 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14027607618871128924', time=2026-02-25T00:26:45.837009+0000, chain.objs.size()=1 2026-02-24T22:34:25.481 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.hwOCkpp67mE2NIjCmczpCtPuJybLpzg_0 2026-02-24T22:34:25.481 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14277128389486060446', time=2026-02-25T00:26:45.925711+0000, chain.objs.size()=1 2026-02-24T22:34:25.481 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.2UF3u3A3244W_NIZvIkWGoXpnFTxc2J_0 2026-02-24T22:34:25.481 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11519030214855848039', time=2026-02-25T00:26:45.968720+0000, chain.objs.size()=1 2026-02-24T22:34:25.481 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.DRrP4hN7Akw7eeIGdh-PzUrjzEVLI4__0 2026-02-24T22:34:25.481 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7564007168457009465', time=2026-02-25T00:26:45.977296+0000, chain.objs.size()=1 2026-02-24T22:34:25.481 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.67M-rUWDvUVRSyBFMEATs25EynZRa30_0 2026-02-24T22:34:25.482 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6575234146859296552', time=2026-02-25T00:26:45.988917+0000, chain.objs.size()=1 2026-02-24T22:34:25.482 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Q3w078Rd8ijJcnrO2nyTD4zhT5LbPSL_0 2026-02-24T22:34:25.482 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17771922683964688437', time=2026-02-25T00:26:46.126154+0000, chain.objs.size()=1 2026-02-24T22:34:25.482 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.pqfL26uYulrQstkCl-vY6dSktXSgvGP_0 2026-02-24T22:34:25.482 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8833443969290949918', time=2026-02-25T00:26:46.161191+0000, chain.objs.size()=1 2026-02-24T22:34:25.482 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.qs7n-6RtgNdG-cRXzXGqXo8mpRNTOeg_0 2026-02-24T22:34:25.482 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14712525069676558538', time=2026-02-25T00:26:53.171145+0000, chain.objs.size()=1 2026-02-24T22:34:25.482 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.f4cfTFdbA-U_5dDKg46nb5IatVjDG_u_0 2026-02-24T22:34:25.482 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17541061929189614616', time=2026-02-25T00:26:53.175794+0000, chain.objs.size()=1 2026-02-24T22:34:25.482 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.1rQXwl-XCNTXlqk4Vf95Cb7_-60c6nL_0 2026-02-24T22:34:25.482 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16753936889430449879', time=2026-02-25T00:26:53.176251+0000, chain.objs.size()=1 2026-02-24T22:34:25.482 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.hPI9Q0r27eprvQlje-mjqvFvbOgTYah_0 2026-02-24T22:34:25.482 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15707116921994582282', time=2026-02-25T00:26:53.177911+0000, chain.objs.size()=1 2026-02-24T22:34:25.482 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.__356qFBLugktT23-ZvDra0Cr28-7K1_0 2026-02-24T22:34:25.482 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11717733163965494374', time=2026-02-25T00:26:53.181020+0000, chain.objs.size()=1 2026-02-24T22:34:25.482 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.svKG7a6sFULGtKsAjdp6GjKUfZfSLBL_0 2026-02-24T22:34:25.482 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7141883902949750812', time=2026-02-25T00:26:53.191709+0000, chain.objs.size()=1 2026-02-24T22:34:25.482 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.cgrk5QNlwiHhjrnRJMQdjHPuUeQxTuR_0 2026-02-24T22:34:25.482 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15766224477504032283', time=2026-02-25T00:26:53.255983+0000, chain.objs.size()=1 2026-02-24T22:34:25.482 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.491+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.z8f_EdMCBiRizSOnZkERuGk5XfEiLhL_0 2026-02-24T22:34:25.482 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11062032950750015386', time=2026-02-25T00:26:53.257984+0000, chain.objs.size()=1 2026-02-24T22:34:25.482 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.JUZPb6AkjT74kHlQcC60ZTDiT6R5SWL_0 2026-02-24T22:34:25.482 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13578264108657771975', time=2026-02-25T00:26:53.350105+0000, chain.objs.size()=1 2026-02-24T22:34:25.483 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.00X4IYCKCz6fn2fwvhjSfi94GG9fyW__0 2026-02-24T22:34:25.483 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14833379903284940565', time=2026-02-25T00:26:53.382169+0000, chain.objs.size()=1 2026-02-24T22:34:25.483 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.NELdS3tU_wT-k7AcU72NwBXv-1zBREr_0 2026-02-24T22:34:25.483 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/42897 2026-02-24T22:34:25.485 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=40, truncated=0, next_marker='' 2026-02-24T22:34:25.485 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10569308486115615027', time=2026-02-25T00:26:53.418415+0000, chain.objs.size()=1 2026-02-24T22:34:25.485 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.PGjfQ40MeWOxzmpQdL5YwtAWTSLbxBn_0 2026-02-24T22:34:25.485 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6532834084745199033', time=2026-02-25T00:26:53.439745+0000, chain.objs.size()=1 2026-02-24T22:34:25.485 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.jDzbmSgD7TAefl3eAVckd_TLa7LYqcw_0 2026-02-24T22:34:25.485 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11299223849660791591', time=2026-02-25T00:26:53.527454+0000, chain.objs.size()=1 2026-02-24T22:34:25.485 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.tshvx1KVePZK6pwZZJO7zmPDC9dl0jo_0 2026-02-24T22:34:25.485 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1941733215381295651', time=2026-02-25T00:26:53.551864+0000, chain.objs.size()=1 2026-02-24T22:34:25.485 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.rlh7-e9mGAuTfk5BoJP7tTcWvDTkeeQ_0 2026-02-24T22:34:25.485 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16800919026016372428', time=2026-02-25T00:26:53.558839+0000, chain.objs.size()=1 2026-02-24T22:34:25.485 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.0dDcNEgsPwvPgWRRrBYq_X8rn4xqxrc_0 2026-02-24T22:34:25.485 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16433693444177752253', time=2026-02-25T00:26:53.571898+0000, chain.objs.size()=1 2026-02-24T22:34:25.485 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.GAPr-jGR5XwuSRKnhG7f_yHjL2EMRCz_0 2026-02-24T22:34:25.485 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8076319433552027533', time=2026-02-25T00:26:53.603237+0000, chain.objs.size()=1 2026-02-24T22:34:25.485 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.UNPu21dEsM_7EmvCeARpyVehXPTVAfq_0 2026-02-24T22:34:25.485 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15485202609009734740', time=2026-02-25T00:26:53.744054+0000, chain.objs.size()=1 2026-02-24T22:34:25.485 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.uYOzvSZfH_HWhYn2eLBcpxhcVB4qSau_0 2026-02-24T22:34:25.485 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.783082239158612514', time=2026-02-25T00:26:53.778049+0000, chain.objs.size()=1 2026-02-24T22:34:25.485 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.NbNg1XtaOlR12Lh5Mq0uE-JcKcbxwgB_0 2026-02-24T22:34:25.485 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.886659675701391307', time=2026-02-25T00:26:53.941089+0000, chain.objs.size()=1 2026-02-24T22:34:25.485 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.j3r9a5BKnjelJtqJtOuYeXQeKsUe6Uj_0 2026-02-24T22:34:25.485 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16679107639871162005', time=2026-02-25T00:26:53.964883+0000, chain.objs.size()=1 2026-02-24T22:34:25.485 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_._bxQhxIrd-hms_4VHXnpE_lW_HEaFqp_0 2026-02-24T22:34:25.485 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15939193048208614124', time=2026-02-25T00:26:53.980423+0000, chain.objs.size()=1 2026-02-24T22:34:25.485 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.X9iGJhXo7R8JlXloXM2ab86Y3Wx7Xuw_0 2026-02-24T22:34:25.486 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5790800056165436508', time=2026-02-25T00:26:54.130768+0000, chain.objs.size()=1 2026-02-24T22:34:25.486 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.MAFbIP9IfgrlokjOXtDCwS-9ZkxjhiB_0 2026-02-24T22:34:25.486 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11918893100556076058', time=2026-02-25T00:26:54.139140+0000, chain.objs.size()=1 2026-02-24T22:34:25.486 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.DkFxFVyw4VesxJEePvpGp0t0bC0ed_9_0 2026-02-24T22:34:25.486 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2324179855226631268', time=2026-02-25T00:26:54.141969+0000, chain.objs.size()=1 2026-02-24T22:34:25.486 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.KVcVF0eEu1pPFXmNo8334rtq3BhaTQG_0 2026-02-24T22:34:25.486 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14789869499684057155', time=2026-02-25T00:26:54.146166+0000, chain.objs.size()=1 2026-02-24T22:34:25.486 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.3tygtrvSBMlqKNl_P0vIIPqm2TKLHkh_0 2026-02-24T22:34:25.486 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10983922419283379484', time=2026-02-25T00:26:54.191781+0000, chain.objs.size()=1 2026-02-24T22:34:25.486 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.kb5Qvg6ml4LWY7E2F9Gwl0O1ZUPghuy_0 2026-02-24T22:34:25.486 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14102507609242839071', time=2026-02-25T00:26:54.229105+0000, chain.objs.size()=1 2026-02-24T22:34:25.486 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.sBsZYd8phQSXXrSH-DmQ518dFe8GSAt_0 2026-02-24T22:34:25.486 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10749281911238258216', time=2026-02-25T00:26:54.316064+0000, chain.objs.size()=1 2026-02-24T22:34:25.486 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.I88s9LS1narN6hONR3Jpql8nd5J_Mz8_0 2026-02-24T22:34:25.486 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11690030083847528565', time=2026-02-25T00:26:54.361088+0000, chain.objs.size()=1 2026-02-24T22:34:25.486 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.D65b2aM_G9yNkHcv8IWjdyBKoR3FC8m_0 2026-02-24T22:34:25.486 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2372677362622311385', time=2026-02-25T00:26:54.367687+0000, chain.objs.size()=1 2026-02-24T22:34:25.486 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.G2XWMEtt74UgI1z3RJ9uBH9j3AlqZ-B_0 2026-02-24T22:34:25.486 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13424760527157868067', time=2026-02-25T00:26:54.372892+0000, chain.objs.size()=1 2026-02-24T22:34:25.486 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.EJCxTmkya9g3qkEfx6U4PMlOuhpuBS6_0 2026-02-24T22:34:25.486 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15428792333396941682', time=2026-02-25T00:26:54.468663+0000, chain.objs.size()=1 2026-02-24T22:34:25.486 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.495+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.td68HvhXRo3LzZIICZRCXmmjPQui7fJ_0 2026-02-24T22:34:25.487 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14570101252622041457', time=2026-02-25T00:26:54.538781+0000, chain.objs.size()=1 2026-02-24T22:34:25.487 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.HuhUefUFJNpXoD9TIDAko2fJf-B_Ogg_0 2026-02-24T22:34:25.487 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10359732834554276931', time=2026-02-25T00:27:01.368978+0000, chain.objs.size()=1 2026-02-24T22:34:25.487 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1701__shadow_.R1uYjGIf6SkV5wdwYmwfpq67InLBRJH_0 2026-02-24T22:34:25.487 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13336875235058710673', time=2026-02-25T00:27:07.032500+0000, chain.objs.size()=1 2026-02-24T22:34:25.487 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1711__shadow_.QYwqsADUSsTlsA-xmjW38tkmSxc1FMP_0 2026-02-24T22:34:25.487 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13504875572054976318', time=2026-02-25T00:27:17.560931+0000, chain.objs.size()=1 2026-02-24T22:34:25.487 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1732__shadow_.WZrEpcECVTCt_lw3AEB_BUcoPyOzG5G_0 2026-02-24T22:34:25.487 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13295248105602708238', time=2026-02-25T00:27:28.752914+0000, chain.objs.size()=1 2026-02-24T22:34:25.487 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1749__shadow_.Wt4WeP9kJPtBci98xVNNrf-PiujFU1D_0 2026-02-24T22:34:25.487 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12488193604503354851', time=2026-02-25T00:28:07.156672+0000, chain.objs.size()=1 2026-02-24T22:34:25.487 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1807__shadow_.XTNBnUrwjkGBx9s5LjdaKtfj2bchqPm_0 2026-02-24T22:34:25.487 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13372269145664473232', time=2026-02-25T00:28:20.445644+0000, chain.objs.size()=1 2026-02-24T22:34:25.487 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1825__shadow_.Khwbkrv4p_Sy6byRPHM6wlGo0WwSXPZ_0 2026-02-24T22:34:25.487 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2421170944387146612', time=2026-02-25T00:28:52.153519+0000, chain.objs.size()=1 2026-02-24T22:34:25.487 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1884__multipart_mymultipart.2~hhXBR1qhG6_sQyt1QXc8sHjy8VIH4bH.1 2026-02-24T22:34:25.488 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2266989616039243198', time=2026-02-25T00:29:10.350987+0000, chain.objs.size()=1 2026-02-24T22:34:25.488 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1889__multipart_mymultipart.2~pIGW5rtP3SPpUVj9P3K5sMPYn2N8Nj0.1 2026-02-24T22:34:25.488 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.136743139333382258', time=2026-02-25T00:29:15.608934+0000, chain.objs.size()=4 2026-02-24T22:34:25.488 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1893__multipart_mymultipart.2~PekxUt4ki71LIndTi0tcPO0wdPTg1Qk.1 2026-02-24T22:34:25.488 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1893__multipart_mymultipart.2~PekxUt4ki71LIndTi0tcPO0wdPTg1Qk.2 2026-02-24T22:34:25.488 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1893__multipart_mymultipart.2~PekxUt4ki71LIndTi0tcPO0wdPTg1Qk.3 2026-02-24T22:34:25.488 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1893__multipart_mymultipart.2~PekxUt4ki71LIndTi0tcPO0wdPTg1Qk.4 2026-02-24T22:34:25.489 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2724853748767986061', time=2026-02-25T00:30:04.459798+0000, chain.objs.size()=1 2026-02-24T22:34:25.489 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1921__shadow_.uxmQHZkrVxRZk5M6G_Jje-SHYjhSfRD_0 2026-02-24T22:34:25.489 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18132609865663692906', time=2026-02-25T00:30:18.602371+0000, chain.objs.size()=1 2026-02-24T22:34:25.489 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:vmsy-uybNA2rusdW3H03XN4q8Yy943L_.maOvoZKHyM-T7hSBLz1UvGwIKtHliBb_0 2026-02-24T22:34:25.489 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15364653384043352302', time=2026-02-25T00:30:20.685839+0000, chain.objs.size()=1 2026-02-24T22:34:25.489 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:B.wX6nc7K0gpmaOumbyV.5ShhOXwJWK_.T6zwNcH8yiHpPYKlCI370fZ1bboIqcS_0 2026-02-24T22:34:25.489 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.434977084371987328', time=2026-02-25T00:30:20.829961+0000, chain.objs.size()=1 2026-02-24T22:34:25.489 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:YmfDKM6SZxXs4TKinvS2sswu41z05Cq_.no2ZZWxexnNtPvuhkfjl5eHZj-nemSX_0 2026-02-24T22:34:25.489 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15240796310441500967', time=2026-02-25T00:30:52.494282+0000, chain.objs.size()=1 2026-02-24T22:34:25.489 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1950__shadow:Nykho.DgQ0mMgqLIfrSu1R5pC50D1bs_.2AcveQPCg_R1aoE4AlaLfrhlsOD0tB7_0 2026-02-24T22:34:25.489 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8822173079452746544', time=2026-02-25T00:30:55.612908+0000, chain.objs.size()=1 2026-02-24T22:34:25.489 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1955__shadow:avLJ9ZVo8HAKv8oRRlLk-9dTuoswMgF_.nIv5qo_R5iHV9ALQUMGO7uRr2K3yQZs_0 2026-02-24T22:34:25.489 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13810109123020535604', time=2026-02-25T00:30:56.361125+0000, chain.objs.size()=1 2026-02-24T22:34:25.489 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1956__shadow:ZOFZuYEaNh5j08MXWr0rFUFjJU8oE4M_.2AePyOc8ZyjPjk47HJGbCm2AVrBOTk0_0 2026-02-24T22:34:25.490 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.499+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.491 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2026-02-24T22:34:25.492 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.493 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.493 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/37520' 2026-02-24T22:34:25.493 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9413642548407789474', time=2026-02-25T00:15:34.630164+0000, chain.objs.size()=1 2026-02-24T22:34:25.493 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506__shadow_.ub6QlRpEa6_Iap2pmdfBUlE6y26uQVV_0 2026-02-24T22:34:25.493 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9494459432960479438', time=2026-02-25T00:16:33.508212+0000, chain.objs.size()=1 2026-02-24T22:34:25.493 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521__shadow_.qxOKfaMGZSNE1PN3D9fHilbwfP7Gz50_0 2026-02-24T22:34:25.493 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.830548241956636823', time=2026-02-25T00:16:33.509986+0000, chain.objs.size()=1 2026-02-24T22:34:25.493 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521__shadow_.zkmmzAf7CQ5tPfeHUY09W-bd_Vk9zSc_0 2026-02-24T22:34:25.493 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15075745850935159131', time=2026-02-25T00:16:33.513720+0000, chain.objs.size()=1 2026-02-24T22:34:25.493 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521__shadow_.QqL6ZIe00LS05HX4nMRUsXsqyaaTNCO_0 2026-02-24T22:34:25.493 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1665712168914705385', time=2026-02-25T00:17:39.874335+0000, chain.objs.size()=1 2026-02-24T22:34:25.493 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525__shadow_.BMseItkGZKi-mm28nVn9KfeR0qGULzU_0 2026-02-24T22:34:25.494 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12354992367732330313', time=2026-02-25T00:17:39.874986+0000, chain.objs.size()=1 2026-02-24T22:34:25.494 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525__shadow_.Kp-hWztK6IQhsRtNL1ljfpbDF9haNrH_0 2026-02-24T22:34:25.494 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3127713479940562258', time=2026-02-25T00:18:14.571159+0000, chain.objs.size()=1 2026-02-24T22:34:25.494 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528__shadow_.5VMoO_glW4mmRErg0qeLRC8-YkVvBGb_0 2026-02-24T22:34:25.494 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14909905214342101427', time=2026-02-25T00:18:48.185727+0000, chain.objs.size()=1 2026-02-24T22:34:25.494 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530__shadow_.nTIylaIByy44mZh8X_MS0clIlz4udkp_0 2026-02-24T22:34:25.494 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16345061013048440758', time=2026-02-25T00:19:57.106771+0000, chain.objs.size()=1 2026-02-24T22:34:25.494 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535__shadow_.XOXuQy2QYJHrCVFV3sKZZeVD-s8nwK7_0 2026-02-24T22:34:25.494 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9451227931685962109', time=2026-02-25T00:19:58.449922+0000, chain.objs.size()=1 2026-02-24T22:34:25.494 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536__shadow_.ryDb9t5atoCGsO55Q0YhnPtev7hRIIe_0 2026-02-24T22:34:25.494 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5243740982481191077', time=2026-02-25T00:20:32.064757+0000, chain.objs.size()=1 2026-02-24T22:34:25.494 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538__shadow_.5sfe7kbHc0JwSXRCUSGF1TdezTmuavl_0 2026-02-24T22:34:25.494 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11176993926414760121', time=2026-02-25T00:20:35.135817+0000, chain.objs.size()=1 2026-02-24T22:34:25.494 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.503+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540__shadow_.DMaggzpL3nwq8X1f-UGUGz-vBSWSMxo_0 2026-02-24T22:34:25.494 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5842934037748423559', time=2026-02-25T00:23:54.194719+0000, chain.objs.size()=1 2026-02-24T22:34:25.494 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1546__shadow_.p0fFtA5G5WD_a92WGXSPJIRmue2vl4q_0 2026-02-24T22:34:25.494 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8722649747952921372', time=2026-02-25T00:24:16.153643+0000, chain.objs.size()=1 2026-02-24T22:34:25.494 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1570__shadow_.wLpeN0mBCuRlLwlaTU2Uixux528AMLX_0 2026-02-24T22:34:25.494 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7279230373945418414', time=2026-02-25T00:25:35.426487+0000, chain.objs.size()=1 2026-02-24T22:34:25.494 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1600__shadow_.6v8ZoHDmZ23TnM-_qKqDC-EstOANOpI_0 2026-02-24T22:34:25.494 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7439874823952074878', time=2026-02-25T00:25:43.239074+0000, chain.objs.size()=1 2026-02-24T22:34:25.494 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1611__shadow_.CH_Tw02o4b64ge1e-aQZpG7KSq2j8re_0 2026-02-24T22:34:25.494 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4282128663991966162', time=2026-02-25T00:25:44.427134+0000, chain.objs.size()=1 2026-02-24T22:34:25.494 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1613__shadow_.McpS-gxvWIhHLTDTJxxxhdMGi8BGuHx_0 2026-02-24T22:34:25.494 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.362555625205757026', time=2026-02-25T00:25:49.698435+0000, chain.objs.size()=1 2026-02-24T22:34:25.494 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1622__shadow_.oyaxpA2XQ2u7WWQ3fV8-xYEC2Kf3EMG_0 2026-02-24T22:34:25.495 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13491702911650490280', time=2026-02-25T00:25:59.073698+0000, chain.objs.size()=1 2026-02-24T22:34:25.495 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.j-VeTfgbvcL1obSxds7weN5tuq5N2qQ_0 2026-02-24T22:34:25.495 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8224686193968036127', time=2026-02-25T00:25:59.131950+0000, chain.objs.size()=1 2026-02-24T22:34:25.495 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.VjjBq_IBhNNPQdNLCmnxVfjK4KQFPMk_0 2026-02-24T22:34:25.495 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3272540818157236347', time=2026-02-25T00:25:59.155392+0000, chain.objs.size()=1 2026-02-24T22:34:25.495 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.35nKpW02UZJkVuWrcfqHBNY4KSKP1vA_0 2026-02-24T22:34:25.495 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.782556425820162305', time=2026-02-25T00:25:59.244496+0000, chain.objs.size()=1 2026-02-24T22:34:25.495 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ffMdTdfil0VTZR5wrnBrnz3a4sEDADS_0 2026-02-24T22:34:25.495 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3516838264222508456', time=2026-02-25T00:25:59.258039+0000, chain.objs.size()=1 2026-02-24T22:34:25.495 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.bAwSmZFzLVlJZFDNY9UnfYqa89lK2gJ_0 2026-02-24T22:34:25.495 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.384569645908303147', time=2026-02-25T00:25:59.282593+0000, chain.objs.size()=1 2026-02-24T22:34:25.495 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.CSPc1nnqr-CKTj3VIvUtGM7Zs8fSWVU_0 2026-02-24T22:34:25.495 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13987955983671977449', time=2026-02-25T00:25:59.285671+0000, chain.objs.size()=1 2026-02-24T22:34:25.495 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.wWbO-SVgPuG34OblClE9XLlakQXDwP5_0 2026-02-24T22:34:25.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1770389797857129191', time=2026-02-25T00:25:59.310502+0000, chain.objs.size()=1 2026-02-24T22:34:25.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.w5dkn5UAJysDjULpXsvz3zv29o4SG67_0 2026-02-24T22:34:25.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15264404682725465090', time=2026-02-25T00:25:59.435809+0000, chain.objs.size()=1 2026-02-24T22:34:25.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.5nN3KvsGnpEwzfd3yDUYx_SlXZ-w7r4_0 2026-02-24T22:34:25.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16461035589599564640', time=2026-02-25T00:25:59.571356+0000, chain.objs.size()=1 2026-02-24T22:34:25.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.n7dY3cVrPGxHlgGPgc13XKadgQTaTMH_0 2026-02-24T22:34:25.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4483391940451863465', time=2026-02-25T00:25:59.572674+0000, chain.objs.size()=1 2026-02-24T22:34:25.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.lGmwlUlyG-CvBROC4GWt4027GnwxjB2_0 2026-02-24T22:34:25.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7283837561287557632', time=2026-02-25T00:25:59.584614+0000, chain.objs.size()=1 2026-02-24T22:34:25.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.G4Nfy6wOrVQ1jxTGNn9pgh3ngqNFODR_0 2026-02-24T22:34:25.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9047138651276420378', time=2026-02-25T00:25:59.599728+0000, chain.objs.size()=1 2026-02-24T22:34:25.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.qWxrxRl42kDB4g98NflvkilWLIlZoxz_0 2026-02-24T22:34:25.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9713488604856558856', time=2026-02-25T00:25:59.600191+0000, chain.objs.size()=1 2026-02-24T22:34:25.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.AfWjOyiRcZDsjrhb8ANkWgxLLT9Px0G_0 2026-02-24T22:34:25.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5411702868481845757', time=2026-02-25T00:25:59.600710+0000, chain.objs.size()=1 2026-02-24T22:34:25.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.NJlBPhIW-_MrOYvSq1KlWCl5fsxwSis_0 2026-02-24T22:34:25.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6942805877853451244', time=2026-02-25T00:25:59.735971+0000, chain.objs.size()=1 2026-02-24T22:34:25.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Rw9H4Bwru9GyC0c6tuB0DeaSGiiW11q_0 2026-02-24T22:34:25.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17993893854171529607', time=2026-02-25T00:25:59.750951+0000, chain.objs.size()=1 2026-02-24T22:34:25.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.jNcAmB2SsMZMoKczToWC-8v9NOENBuT_0 2026-02-24T22:34:25.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3237490668686504569', time=2026-02-25T00:25:59.804631+0000, chain.objs.size()=1 2026-02-24T22:34:25.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.kOl4bmTZu2oc3uf2H8VRcdj0mveXK2l_0 2026-02-24T22:34:25.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12004579614186282270', time=2026-02-25T00:25:59.815608+0000, chain.objs.size()=1 2026-02-24T22:34:25.496 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.z0S3PshDcYXLBe1XhA9GIW6WC_N1Zjh_0 2026-02-24T22:34:25.497 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4953638728298412739', time=2026-02-25T00:25:59.903122+0000, chain.objs.size()=1 2026-02-24T22:34:25.497 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.tjIxpks4hyRiN-rIued3q9qBsebL68G_0 2026-02-24T22:34:25.497 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10821448831470778528', time=2026-02-25T00:25:59.927618+0000, chain.objs.size()=1 2026-02-24T22:34:25.497 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.dhcqeYXVZKHXvuAT1It70O9XRPiurpx_0 2026-02-24T22:34:25.497 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11682710809593125775', time=2026-02-25T00:25:59.978956+0000, chain.objs.size()=1 2026-02-24T22:34:25.497 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.78QgfhrTL8umC-7nEJDLA0lVcjZq8r-_0 2026-02-24T22:34:25.497 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10803789432729561136', time=2026-02-25T00:26:00.062980+0000, chain.objs.size()=1 2026-02-24T22:34:25.497 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.6-lACvTfrHgn2eQZS3AmtLSEPlUC8tZ_0 2026-02-24T22:34:25.497 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13402018312674164211', time=2026-02-25T00:26:00.073446+0000, chain.objs.size()=1 2026-02-24T22:34:25.497 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.KqUtmQtuQFkbvVLdkTGDLj37PYMJ2Zj_0 2026-02-24T22:34:25.497 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2228805017857136814', time=2026-02-25T00:26:00.083475+0000, chain.objs.size()=1 2026-02-24T22:34:25.497 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_._7Jm8TgExT3mJikpvPQ_tenrTmLe9o4_0 2026-02-24T22:34:25.497 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14195854806749482819', time=2026-02-25T00:26:00.095958+0000, chain.objs.size()=1 2026-02-24T22:34:25.497 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.SRRuDvxvyLNFSbVScyy37mw1lftbKBr_0 2026-02-24T22:34:25.497 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9343309239224674059', time=2026-02-25T00:26:00.118351+0000, chain.objs.size()=1 2026-02-24T22:34:25.497 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ADXe2A6bzTiSCzCZ9dvoADRp1eW9Yqi_0 2026-02-24T22:34:25.497 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3091542371740804957', time=2026-02-25T00:26:00.209376+0000, chain.objs.size()=1 2026-02-24T22:34:25.497 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.B4q97VZCDoO7ajWZhVGdm2nEsC1aAOM_0 2026-02-24T22:34:25.497 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14942915801027467443', time=2026-02-25T00:26:00.214581+0000, chain.objs.size()=1 2026-02-24T22:34:25.497 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.-Bms6t0bc-c1CuwFfqDnxeHhCupd5hK_0 2026-02-24T22:34:25.497 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7925052579001596576', time=2026-02-25T00:26:00.246331+0000, chain.objs.size()=1 2026-02-24T22:34:25.497 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.LFiBYRB9ZFICANTqJ2RC0AWCHMTHbbF_0 2026-02-24T22:34:25.498 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2962782792977002397', time=2026-02-25T00:26:00.271595+0000, chain.objs.size()=1 2026-02-24T22:34:25.498 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.O_E-hhnhKyuf27tdFwB4K5IHt1H_QWY_0 2026-02-24T22:34:25.498 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3062354170504479344', time=2026-02-25T00:26:00.276558+0000, chain.objs.size()=1 2026-02-24T22:34:25.498 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.89Nb2X16AQbDFLvsKiLWwrNefnqKVFk_0 2026-02-24T22:34:25.498 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9425667533030114216', time=2026-02-25T00:26:00.280061+0000, chain.objs.size()=1 2026-02-24T22:34:25.498 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.HhXrZi12jTXetHV-L5KSoNzBsbmWkXV_0 2026-02-24T22:34:25.498 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11981188957645760546', time=2026-02-25T00:26:03.713860+0000, chain.objs.size()=1 2026-02-24T22:34:25.498 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1634__shadow_.0iohNkWd4BMPDCy0kKEwTesfVxtA0jU_0 2026-02-24T22:34:25.498 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17892798992577600464', time=2026-02-25T00:26:03.714308+0000, chain.objs.size()=1 2026-02-24T22:34:25.498 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1634__shadow_.jxfKyMs4kwBuDbc1WOv9xNm41EBekQc_0 2026-02-24T22:34:25.498 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4081754122004608321', time=2026-02-25T00:26:04.277937+0000, chain.objs.size()=1 2026-02-24T22:34:25.498 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1635__shadow_.pwebo3JFD22ipB0BQ0SkR7dW7D92Nfo_0 2026-02-24T22:34:25.498 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8691391457212290602', time=2026-02-25T00:26:07.678222+0000, chain.objs.size()=1 2026-02-24T22:34:25.498 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1641__shadow_.QKqeJvHyf3r6_vBp9WZBUpWavTyf1Pj_0 2026-02-24T22:34:25.498 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3895843510969204918', time=2026-02-25T00:26:08.835514+0000, chain.objs.size()=1 2026-02-24T22:34:25.498 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.507+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1643__shadow_.5SievjTdb5hQLIShOF7IudnhOL6alUw_0 2026-02-24T22:34:25.498 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.223820664169596455', time=2026-02-25T00:26:18.503910+0000, chain.objs.size()=1 2026-02-24T22:34:25.498 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1658__shadow_.UTiobuzZyunkvFIZsxY2NPTPsOJIgIJ_0 2026-02-24T22:34:25.498 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1809726216497271161', time=2026-02-25T00:26:19.455874+0000, chain.objs.size()=1 2026-02-24T22:34:25.499 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1659__shadow_.Z2CK1HwGkgq06JuhkOJAyRbmN659hPs_0 2026-02-24T22:34:25.499 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18215232377187384724', time=2026-02-25T00:26:20.030310+0000, chain.objs.size()=1 2026-02-24T22:34:25.499 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1660__shadow_.TyG45bNQq7l_yXENAnKC5Uhlcbh29T2_0 2026-02-24T22:34:25.499 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11597448949145030754', time=2026-02-25T00:26:44.815833+0000, chain.objs.size()=1 2026-02-24T22:34:25.499 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.jVqxDd0OOQic8dsqgRsPKizwbkwPHua_0 2026-02-24T22:34:25.499 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9779224744787148298', time=2026-02-25T00:26:44.842594+0000, chain.objs.size()=1 2026-02-24T22:34:25.499 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.TWbxTOKWydVcjVzrQIuq8eoiduCl6ys_0 2026-02-24T22:34:25.499 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10404373173436079404', time=2026-02-25T00:26:44.865070+0000, chain.objs.size()=1 2026-02-24T22:34:25.499 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Jw7JKcarElnX0D65gItCzYtx5GDrN34_0 2026-02-24T22:34:25.499 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8488713617908917263', time=2026-02-25T00:26:44.936383+0000, chain.objs.size()=1 2026-02-24T22:34:25.499 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_._oWpeD4U0DhVTFwVXCS2GH3ESQnJhxy_0 2026-02-24T22:34:25.499 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4520089989427808800', time=2026-02-25T00:26:44.946082+0000, chain.objs.size()=1 2026-02-24T22:34:25.499 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Q7ZGCq-jDiIX4Taoz-GXPv_DYph_cQk_0 2026-02-24T22:34:25.499 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1939929544783206480', time=2026-02-25T00:26:44.948824+0000, chain.objs.size()=1 2026-02-24T22:34:25.499 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.eSmZYapi_hYbkvSr6Bndi4OY1ibfQA__0 2026-02-24T22:34:25.499 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7333534726861364363', time=2026-02-25T00:26:44.955029+0000, chain.objs.size()=1 2026-02-24T22:34:25.499 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.deXFeWn9qhB7kbyiOKOMjX7JorP_IIb_0 2026-02-24T22:34:25.499 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5848659327643985810', time=2026-02-25T00:26:44.969582+0000, chain.objs.size()=1 2026-02-24T22:34:25.499 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.7O01ji7-RbjDn6GE-Jc9eDNrLCvuZLC_0 2026-02-24T22:34:25.499 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9770161105307183191', time=2026-02-25T00:26:44.982423+0000, chain.objs.size()=1 2026-02-24T22:34:25.499 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.A3KuN3xyBNVI1EZvjV5TpEzypYGd0ro_0 2026-02-24T22:34:25.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16386999800655202077', time=2026-02-25T00:26:44.987991+0000, chain.objs.size()=1 2026-02-24T22:34:25.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.f-YNknGNRupCXPVSxJ9fnL2s3DzCVmn_0 2026-02-24T22:34:25.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13468366855308986081', time=2026-02-25T00:26:45.252644+0000, chain.objs.size()=1 2026-02-24T22:34:25.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.kznERlxZ1hJprpK1t6qpEfQVVu9aPh5_0 2026-02-24T22:34:25.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2380398114233919784', time=2026-02-25T00:26:45.263482+0000, chain.objs.size()=1 2026-02-24T22:34:25.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.pktGq6kt9BnTSvBhhuDVnXzuYY4uKkA_0 2026-02-24T22:34:25.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16561800603713897989', time=2026-02-25T00:26:45.283012+0000, chain.objs.size()=1 2026-02-24T22:34:25.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.wTgzUlbDJBdtfzcs2IP5yDKc8W-wYNP_0 2026-02-24T22:34:25.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7070106897931997157', time=2026-02-25T00:26:45.290280+0000, chain.objs.size()=1 2026-02-24T22:34:25.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.RY569bCQo4-eTSt3F4A5RrfshrXTXxW_0 2026-02-24T22:34:25.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2502736411090311899', time=2026-02-25T00:26:45.290778+0000, chain.objs.size()=1 2026-02-24T22:34:25.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.uct74Rbpr4HL0-8GHoK3EWxAkFQb7v0_0 2026-02-24T22:34:25.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10545576321168020913', time=2026-02-25T00:26:45.297925+0000, chain.objs.size()=1 2026-02-24T22:34:25.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.IbA-3pklZMljMQN40_P4QwgR87Fp04W_0 2026-02-24T22:34:25.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1073597470563539082', time=2026-02-25T00:26:45.312348+0000, chain.objs.size()=1 2026-02-24T22:34:25.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.2nynQZKCtHxT89MHP7LFj_hpPvRGQ-4_0 2026-02-24T22:34:25.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4749063245924627402', time=2026-02-25T00:26:45.430798+0000, chain.objs.size()=1 2026-02-24T22:34:25.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.iZ2MRuKPGN_ODYdpDAekBOdhDsmiwXb_0 2026-02-24T22:34:25.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9263406234227294122', time=2026-02-25T00:26:45.460739+0000, chain.objs.size()=1 2026-02-24T22:34:25.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.4AhucVDWAqlMBU-iuw7QtAmXWrKcLqA_0 2026-02-24T22:34:25.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7322078443309036693', time=2026-02-25T00:26:45.477777+0000, chain.objs.size()=1 2026-02-24T22:34:25.500 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.1d7qUnSBoa501IrOTiiSbXuHEtymQjB_0 2026-02-24T22:34:25.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6452293426733277100', time=2026-02-25T00:26:45.559318+0000, chain.objs.size()=1 2026-02-24T22:34:25.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.rs9PMpIhnya_U5Yi5mmRAUeRr37AyYx_0 2026-02-24T22:34:25.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17850590332883894735', time=2026-02-25T00:26:45.585305+0000, chain.objs.size()=1 2026-02-24T22:34:25.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.kyAzu3hknmTxU1cvQ8CbnN_oTgR1sxv_0 2026-02-24T22:34:25.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5963230046641375055', time=2026-02-25T00:26:45.595524+0000, chain.objs.size()=1 2026-02-24T22:34:25.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.iCGH39Bv8AVoosHbFiqarAlsyRCrpr4_0 2026-02-24T22:34:25.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17318888054267497517', time=2026-02-25T00:26:45.600041+0000, chain.objs.size()=1 2026-02-24T22:34:25.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.v5E2qslgNSo5_GNasxVIZ_t-XelJ5Yv_0 2026-02-24T22:34:25.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3880117536950947410', time=2026-02-25T00:26:45.835240+0000, chain.objs.size()=1 2026-02-24T22:34:25.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.QTHZj1C8C4qPTNN84STMZynFTWfK1DR_0 2026-02-24T22:34:25.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.863224967484060388', time=2026-02-25T00:26:45.908821+0000, chain.objs.size()=1 2026-02-24T22:34:25.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.-hkoCRaVtRiiCJs5su-eviV39plM915_0 2026-02-24T22:34:25.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12995989972500507537', time=2026-02-25T00:26:45.927684+0000, chain.objs.size()=1 2026-02-24T22:34:25.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Kwb_Tfp4MKTXY-X0VvNjHN1LpCHBNmf_0 2026-02-24T22:34:25.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.661814338133080037', time=2026-02-25T00:26:45.936034+0000, chain.objs.size()=1 2026-02-24T22:34:25.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.1lJ-83-mdwmyvYLqwWkvYauwpf8Ospx_0 2026-02-24T22:34:25.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10697626064188647062', time=2026-02-25T00:26:45.976498+0000, chain.objs.size()=1 2026-02-24T22:34:25.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.EW5YweQbnP5_VOm1M4f_dxY0jaxwi_V_0 2026-02-24T22:34:25.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9936975111513133540', time=2026-02-25T00:26:45.995817+0000, chain.objs.size()=1 2026-02-24T22:34:25.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Byob-o0EeuCdfjsRzMbGs0tgn46Nx6p_0 2026-02-24T22:34:25.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9109190423464600080', time=2026-02-25T00:26:46.077567+0000, chain.objs.size()=1 2026-02-24T22:34:25.501 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.GTNnivVttQG8jBK7iOocijXZmcmHvsY_0 2026-02-24T22:34:25.502 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11710599259802710897', time=2026-02-25T00:26:46.106184+0000, chain.objs.size()=1 2026-02-24T22:34:25.502 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.aUCNz_aKik4-EkekK68b5W0QrSVixpP_0 2026-02-24T22:34:25.502 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8689780279964823345', time=2026-02-25T00:26:46.113231+0000, chain.objs.size()=1 2026-02-24T22:34:25.502 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.KQwFSLGJaD8MV7x40-QchOTpKvmLuZT_0 2026-02-24T22:34:25.502 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16438696611275937400', time=2026-02-25T00:26:46.132990+0000, chain.objs.size()=1 2026-02-24T22:34:25.502 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Q5_4-msUBMnaYkpGynqZaX8POslM1zA_0 2026-02-24T22:34:25.502 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.727934556372221688', time=2026-02-25T00:26:53.166778+0000, chain.objs.size()=1 2026-02-24T22:34:25.502 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.fFQDOXkdKL0O6QJgIS-wZi5brpmN74W_0 2026-02-24T22:34:25.502 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5900308444427160456', time=2026-02-25T00:26:53.195263+0000, chain.objs.size()=1 2026-02-24T22:34:25.502 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.sKDm78k93OSxOntF-yWJw2iWcHbaB0G_0 2026-02-24T22:34:25.502 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8510430430991252737', time=2026-02-25T00:26:53.205063+0000, chain.objs.size()=1 2026-02-24T22:34:25.502 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.eDdCShfnPj9PhPztidrZ18crU4qKun8_0 2026-02-24T22:34:25.502 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2372077439043211340', time=2026-02-25T00:26:53.380315+0000, chain.objs.size()=1 2026-02-24T22:34:25.502 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.NPKUyWarPcGiZz7r6Z1vfjukvhnhKei_0 2026-02-24T22:34:25.502 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7097536800959821389', time=2026-02-25T00:26:53.385256+0000, chain.objs.size()=1 2026-02-24T22:34:25.502 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.RQK7m5FTxoQg0IXh59eO3U34xggaPSe_0 2026-02-24T22:34:25.502 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4702443976785265975', time=2026-02-25T00:26:53.417252+0000, chain.objs.size()=1 2026-02-24T22:34:25.502 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.InCYm2y4Zf8f60VU2KXrYOK5UVpFDhh_0 2026-02-24T22:34:25.502 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9835375256597646023', time=2026-02-25T00:26:53.514137+0000, chain.objs.size()=1 2026-02-24T22:34:25.502 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.511+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.kgV2LlRVVmJMP6B4uklsLLGtvN9TgPf_0 2026-02-24T22:34:25.503 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.515+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/37520 2026-02-24T22:34:25.505 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.515+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=47, truncated=0, next_marker='' 2026-02-24T22:34:25.505 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.515+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4096452506318535840', time=2026-02-25T00:26:53.519860+0000, chain.objs.size()=1 2026-02-24T22:34:25.505 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.515+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.bnoldITp7pZqekU-FmECPKw9s8_KVsL_0 2026-02-24T22:34:25.505 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.515+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13690817111391920633', time=2026-02-25T00:26:53.542299+0000, chain.objs.size()=1 2026-02-24T22:34:25.505 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.515+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.91Tfwkdh9fJaVkPtPJlnQiyq-D7BjkV_0 2026-02-24T22:34:25.505 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.515+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4219345485361967843', time=2026-02-25T00:26:53.591165+0000, chain.objs.size()=1 2026-02-24T22:34:25.505 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.515+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.gu3huqBnP8waRfoKhqu6NqR4g-soNEp_0 2026-02-24T22:34:25.505 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.515+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14628335338092188726', time=2026-02-25T00:26:53.599770+0000, chain.objs.size()=1 2026-02-24T22:34:25.505 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.515+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ZWqL0C_oT0dSYyhJCUBWDoZGr056rUZ_0 2026-02-24T22:34:25.505 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.515+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1467166077533147076', time=2026-02-25T00:26:53.724435+0000, chain.objs.size()=1 2026-02-24T22:34:25.505 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.515+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.EghYscp0xP5UPrutOB6z5jCCxMQc5i1_0 2026-02-24T22:34:25.505 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.515+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11611522881129684062', time=2026-02-25T00:26:53.726302+0000, chain.objs.size()=1 2026-02-24T22:34:25.505 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.515+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.iMXjtxiyuoZXHLVD5cGosZqPlj7GLg3_0 2026-02-24T22:34:25.506 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.515+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11218126634693934301', time=2026-02-25T00:26:53.742875+0000, chain.objs.size()=1 2026-02-24T22:34:25.506 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.515+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.L7vWyy18lx_b1eFWdjgmYBAgvjIgyQe_0 2026-02-24T22:34:25.506 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.515+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9595606032135339063', time=2026-02-25T00:26:53.758739+0000, chain.objs.size()=1 2026-02-24T22:34:25.506 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.515+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.glzvW5B53WYU3Tq2gJE3ptGv9sT_7jR_0 2026-02-24T22:34:25.506 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.515+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12593531052959996519', time=2026-02-25T00:26:53.825616+0000, chain.objs.size()=1 2026-02-24T22:34:25.506 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.515+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.OMrdY7_beoZPMUOywfuDhyUtO7KSUFq_0 2026-02-24T22:34:25.506 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.515+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12405709230946302258', time=2026-02-25T00:26:53.829185+0000, chain.objs.size()=1 2026-02-24T22:34:25.506 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.515+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.4FqmY24HGcFb5cqq7CpX0QFv2sRx2q3_0 2026-02-24T22:34:25.506 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.515+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4463315328085617543', time=2026-02-25T00:26:53.838481+0000, chain.objs.size()=1 2026-02-24T22:34:25.506 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.515+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.UZdnIuoZ-FPakGYnUev4CvP-wQvv3_K_0 2026-02-24T22:34:25.506 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.515+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13738231312184017676', time=2026-02-25T00:26:53.911453+0000, chain.objs.size()=1 2026-02-24T22:34:25.506 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.515+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.BExbqDLl7sTtBqpMAngbGOo-3xaaT6__0 2026-02-24T22:34:25.506 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17595094818720780482', time=2026-02-25T00:26:53.912966+0000, chain.objs.size()=1 2026-02-24T22:34:25.506 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.i15N5dGzGuwKn3436wy-7XBi2_HAftJ_0 2026-02-24T22:34:25.506 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.391218081823646926', time=2026-02-25T00:26:53.956402+0000, chain.objs.size()=1 2026-02-24T22:34:25.506 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.AAvesgFtV2wu7XgkqO9YPtdUn9LkKIt_0 2026-02-24T22:34:25.506 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11241597654996424997', time=2026-02-25T00:26:53.999948+0000, chain.objs.size()=1 2026-02-24T22:34:25.506 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.6tkt6a7kcWwA9XhVY2G_RT9sJb9Tw1h_0 2026-02-24T22:34:25.506 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17779109100956049123', time=2026-02-25T00:26:54.098173+0000, chain.objs.size()=1 2026-02-24T22:34:25.506 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.9yxYGXIfPwhffqqFXrHeQIYR-bFPIEU_0 2026-02-24T22:34:25.506 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7257874177729169318', time=2026-02-25T00:26:54.132568+0000, chain.objs.size()=1 2026-02-24T22:34:25.507 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.DfQsdv0PBIkFHodmftTHn7rBJHw1Nbx_0 2026-02-24T22:34:25.507 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2911056995395152480', time=2026-02-25T00:26:54.133453+0000, chain.objs.size()=1 2026-02-24T22:34:25.507 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.pzKDCJfaIQ-gFE4SauGXuSVd3QKmpg7_0 2026-02-24T22:34:25.507 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17961373318499569991', time=2026-02-25T00:26:54.176386+0000, chain.objs.size()=1 2026-02-24T22:34:25.507 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.XJbYNYOgCk4eJIytYWbpTCFPQabEnkx_0 2026-02-24T22:34:25.507 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13766735869536099845', time=2026-02-25T00:26:54.183285+0000, chain.objs.size()=1 2026-02-24T22:34:25.507 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.4UbefJl6VqtxAJpBPRF1EZFvQCq16oh_0 2026-02-24T22:34:25.507 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4264195050917905200', time=2026-02-25T00:26:54.189481+0000, chain.objs.size()=1 2026-02-24T22:34:25.507 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.t5Mj1D9IygWQkhlcBnv9UpCUnO8DIRP_0 2026-02-24T22:34:25.507 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9649466281957571000', time=2026-02-25T00:26:54.192648+0000, chain.objs.size()=1 2026-02-24T22:34:25.507 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.vjpRLmYzLFa08AcuachmdTX2atLUhty_0 2026-02-24T22:34:25.507 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14398101833839445763', time=2026-02-25T00:26:54.196058+0000, chain.objs.size()=1 2026-02-24T22:34:25.507 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.uOLwGJ8hZcevUFRZdUfm2rtbaiI1IRV_0 2026-02-24T22:34:25.507 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15848587128587891577', time=2026-02-25T00:26:54.200376+0000, chain.objs.size()=1 2026-02-24T22:34:25.507 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_._D6e5U1fAf0E-YS8iF8o0IZDrSJz21s_0 2026-02-24T22:34:25.507 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15140485816945562327', time=2026-02-25T00:26:54.284960+0000, chain.objs.size()=1 2026-02-24T22:34:25.507 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.AaVuEXJSYYNerFog5E_04urq52lmPBJ_0 2026-02-24T22:34:25.507 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5137930171237163272', time=2026-02-25T00:26:54.291896+0000, chain.objs.size()=1 2026-02-24T22:34:25.507 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.-KI892K8EA_WmcYcseMg5TCKpLfZQPQ_0 2026-02-24T22:34:25.508 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.901524293730384650', time=2026-02-25T00:26:54.307877+0000, chain.objs.size()=1 2026-02-24T22:34:25.508 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.-jNV2O_14mCyK7R6pbygmRZ65VQ2g6q_0 2026-02-24T22:34:25.508 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13122114170920727232', time=2026-02-25T00:26:54.333277+0000, chain.objs.size()=1 2026-02-24T22:34:25.508 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.AT5Rw3LUyb6O_erZc1Sg4clCYNnp0Qk_0 2026-02-24T22:34:25.508 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14485325923434093563', time=2026-02-25T00:26:54.487045+0000, chain.objs.size()=1 2026-02-24T22:34:25.508 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.-S6txR31s2CBlzWXa76LHdzEFjjwmL3_0 2026-02-24T22:34:25.508 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10053428237065237554', time=2026-02-25T00:26:54.508891+0000, chain.objs.size()=1 2026-02-24T22:34:25.508 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.TgydJQrz5iK8Y6B-Ggd8NM2KFRGc1ep_0 2026-02-24T22:34:25.508 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7926028562122523716', time=2026-02-25T00:26:54.527923+0000, chain.objs.size()=1 2026-02-24T22:34:25.508 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.DCh7nyV5YtIl5yAQPCOkqXCShupdCRQ_0 2026-02-24T22:34:25.508 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8868879661665542851', time=2026-02-25T00:27:36.435921+0000, chain.objs.size()=1 2026-02-24T22:34:25.508 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1758__shadow:BqAkqQ5J9oe8jVS1iS0vSJMaaLaDbDT_.Now862i1IrWNqo3Y2h31RgqObk5pY18_0 2026-02-24T22:34:25.508 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5567765471651708403', time=2026-02-25T00:28:01.100142+0000, chain.objs.size()=1 2026-02-24T22:34:25.508 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1797__shadow_.C98l-oyWvjFMDogAXia-_yeZn7blg3r_0 2026-02-24T22:34:25.508 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9207052637593940286', time=2026-02-25T00:28:23.483222+0000, chain.objs.size()=1 2026-02-24T22:34:25.508 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1829__shadow_.nln_GQeX-2H5U_XXOSAeasJCw82AC_a_0 2026-02-24T22:34:25.508 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8846797689753646098', time=2026-02-25T00:28:23.696702+0000, chain.objs.size()=1 2026-02-24T22:34:25.508 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1829__shadow_.gXi0t1n5_CI7eGxMkagxoiqnyNHwr01_0 2026-02-24T22:34:25.508 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7248855663676223100', time=2026-02-25T00:28:34.463814+0000, chain.objs.size()=1 2026-02-24T22:34:25.508 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1852__shadow_.D00HjHC7n0q-JlhV2oWJrcHKSeTvCzF_0 2026-02-24T22:34:25.509 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10561929992388720309', time=2026-02-25T00:28:40.619202+0000, chain.objs.size()=1 2026-02-24T22:34:25.509 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1864__shadow_.TxpoJDketjixn2OScofAr67xBC4ATX1_0 2026-02-24T22:34:25.509 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18405892039444869331', time=2026-02-25T00:28:52.392536+0000, chain.objs.size()=2 2026-02-24T22:34:25.509 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1883__shadow_.4Mgy8gvdOvL8FTmyi-MT1EBpJaDEQcc_0 2026-02-24T22:34:25.509 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1883__shadow_.4Mgy8gvdOvL8FTmyi-MT1EBpJaDEQcc_1 2026-02-24T22:34:25.509 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4456907605496050534', time=2026-02-25T00:28:57.834689+0000, chain.objs.size()=1 2026-02-24T22:34:25.509 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1886__shadow:ZNPZ.K4wJY0FO4G8T2ubz7hMJwtXXpS_.jW2zH0PWQEmE4yJ3fCzl8jwmlBAJkb5_0 2026-02-24T22:34:25.509 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2381234718132945865', time=2026-02-25T00:29:43.510355+0000, chain.objs.size()=1 2026-02-24T22:34:25.509 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1904__shadow_.wqoOjs5f2zJQDJenTDJunoFkjkL96QZ_0 2026-02-24T22:34:25.509 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='2~W1hyNU6oaJ62vYHj9ie3HojYyacIxTN', time=2026-02-25T00:30:10.564444+0000, chain.objs.size()=1 2026-02-24T22:34:25.509 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1929__multipart_mymultipart.2~W1hyNU6oaJ62vYHj9ie3HojYyacIxTN.1 2026-02-24T22:34:25.509 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17682339547942282065', time=2026-02-25T00:30:20.342938+0000, chain.objs.size()=1 2026-02-24T22:34:25.509 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:Ac10QHRU0vs4E2qXUFq2sLsZH2ACBR3_.fb4Za6vMWrsvmNps-PKY-YrBC_toD-g_0 2026-02-24T22:34:25.509 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5206096160685480308', time=2026-02-25T00:30:23.587929+0000, chain.objs.size()=1 2026-02-24T22:34:25.509 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:bBnRFW-96I3C6DYmPMpc9CycytH6PEz_.xQsn2NS4FI7zK-erAyPzrW1jXFV1Rt__0 2026-02-24T22:34:25.509 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7571356700992795025', time=2026-02-25T00:30:42.395350+0000, chain.objs.size()=1 2026-02-24T22:34:25.509 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:filtpz96nIiX6kOZjBYEIo9xRfWdg2s_.xsb3_YU56W5VgV2471QDqLGv5BoCyXR_0 2026-02-24T22:34:25.509 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.287481156864725182', time=2026-02-25T00:30:43.167266+0000, chain.objs.size()=1 2026-02-24T22:34:25.509 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:33DXeBKlwFVPaJTX.rVBvVGCmkGIF6z_.4Y5eyXTaqVtaB8yi8_mYo-NocG8H5Iw_0 2026-02-24T22:34:25.509 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17791324416396039608', time=2026-02-25T00:30:50.992446+0000, chain.objs.size()=1 2026-02-24T22:34:25.509 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1949__shadow:40vP-xbOVYQhM5z7wO89kG2JpO-KYZx_.ui5S0P2qBSTAbM6UJt4O8mLTkzTBh5Y_0 2026-02-24T22:34:25.510 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6096520393943696828', time=2026-02-25T00:34:06.421621+0000, chain.objs.size()=1 2026-02-24T22:34:25.510 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.519+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1989__shadow_.mRIqXFLxkwSaiJzAWcfo1BjaSh0Da8o_0 2026-02-24T22:34:25.511 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.523+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.513 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.523+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2026-02-24T22:34:25.514 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.523+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.514 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.523+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/38184' 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14814647138719753842', time=2026-02-25T00:16:32.616391+0000, chain.objs.size()=1 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520__shadow_.-_11lI6JCWXFoM9bCLy7foMpADUXQ1E_0 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12069920064273403744', time=2026-02-25T00:17:05.174281+0000, chain.objs.size()=1 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522__shadow_.w_xsKBX9Zi_F3am1jFciJK74q7xT0zk_0 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17885479875090730743', time=2026-02-25T00:17:41.431239+0000, chain.objs.size()=1 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526__shadow_.VS2SE8T15agcL3y_g3Eows8dsCSsvw1_0 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6855739442709576339', time=2026-02-25T00:18:14.580818+0000, chain.objs.size()=1 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528__shadow_.hELMploay-YfyFL6gTUqaxZQTBs8RnP_0 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.497714108922249771', time=2026-02-25T00:18:14.582117+0000, chain.objs.size()=1 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528__shadow_.Ywo-CelhnHEsfA2qVu7MMgPvf3UOr_R_0 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9243883380622893298', time=2026-02-25T00:18:16.131705+0000, chain.objs.size()=1 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529__shadow_.pckhlZnpW7yR0GLOE80vutayeEAKU-__0 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2880736898209977409', time=2026-02-25T00:18:16.132934+0000, chain.objs.size()=1 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529__shadow_.6zLMELb9zom_uhJBfbOHc6JasB5r3uT_0 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2721750999310798188', time=2026-02-25T00:18:16.138110+0000, chain.objs.size()=1 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529__shadow_.nTzMTavpWbUcetH5P-NE5XUd9brUHS5_0 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7566057086489559593', time=2026-02-25T00:18:48.194104+0000, chain.objs.size()=1 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530__shadow_.f3dZsLH2GlYhZBHKOxvrxDaDc7aqG6c_0 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15923209908863609413', time=2026-02-25T00:18:48.199298+0000, chain.objs.size()=1 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530__shadow_.1LLrOEUCJISgJMDZnt6__zx7xis_2-v_0 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9693356552618601480', time=2026-02-25T00:18:49.777746+0000, chain.objs.size()=1 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531__shadow_.bQR1lBBF6P2DvO1G9O4R50vdVEcgnTk_0 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14385206196698412625', time=2026-02-25T00:18:51.103324+0000, chain.objs.size()=1 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532__shadow_.DSCbLu2k6HMO5CKJnahxIO2mDV2WdVz_0 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.425364722018739408', time=2026-02-25T00:19:58.463894+0000, chain.objs.size()=1 2026-02-24T22:34:25.515 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536__shadow_.lbVXgBbiXZ1kQfAMHePtU33S_eHg2yA_0 2026-02-24T22:34:25.516 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4870522712580586097', time=2026-02-25T00:19:58.465328+0000, chain.objs.size()=1 2026-02-24T22:34:25.516 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536__shadow_.-bGR0KI4aQt63pTsFM9y96Z0MzyRBzw_0 2026-02-24T22:34:25.516 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1639399520177625251', time=2026-02-25T00:20:30.736369+0000, chain.objs.size()=1 2026-02-24T22:34:25.516 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537__shadow_.kxeOoPm-tEKRoHLaVslQl4CxnZ0ounH_0 2026-02-24T22:34:25.516 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11485627696452773193', time=2026-02-25T00:21:06.327579+0000, chain.objs.size()=1 2026-02-24T22:34:25.516 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540__shadow_.8AbPAWbMQ1m0hVB0MzzxZrRiBY2k2Pl_0 2026-02-24T22:34:25.516 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='2~vdOhM6bqr81kYSVvoESfiQGSxT7CQSd', time=2026-02-25T00:24:14.252538+0000, chain.objs.size()=1 2026-02-24T22:34:25.516 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1567__multipart_mymultipart.2~vdOhM6bqr81kYSVvoESfiQGSxT7CQSd.1 2026-02-24T22:34:25.516 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.731231669652926038', time=2026-02-25T00:25:40.797752+0000, chain.objs.size()=1 2026-02-24T22:34:25.516 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1608__shadow_.ZZyy9Cr1LVlsvrMBa_3ir4clJS3llyr_0 2026-02-24T22:34:25.516 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5655346636884195584', time=2026-02-25T00:25:41.727200+0000, chain.objs.size()=1 2026-02-24T22:34:25.516 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1609__shadow_.vyVarpxgist-of72FNFZDCd5RhnDxXV_0 2026-02-24T22:34:25.516 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2776551968149018552', time=2026-02-25T00:25:50.273142+0000, chain.objs.size()=1 2026-02-24T22:34:25.516 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1623__shadow_.2ORhtS8UXXbSphUWRDNS0PvKt7Cw9RX_0 2026-02-24T22:34:25.516 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9946178003512489785', time=2026-02-25T00:25:59.282393+0000, chain.objs.size()=1 2026-02-24T22:34:25.516 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.xv8o25zbXwrSz9QvmqXnf-b6Hw7Uzm4_0 2026-02-24T22:34:25.516 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2187925886431469680', time=2026-02-25T00:25:59.290508+0000, chain.objs.size()=1 2026-02-24T22:34:25.516 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.pH333owggTZrmRxR35tdDTFvzpUWZvL_0 2026-02-24T22:34:25.516 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.618575619109540175', time=2026-02-25T00:25:59.425025+0000, chain.objs.size()=1 2026-02-24T22:34:25.516 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.UDBtWfvvPqDqyMNP5fEO_dJizKxDPYm_0 2026-02-24T22:34:25.517 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11372886314978870569', time=2026-02-25T00:25:59.428482+0000, chain.objs.size()=1 2026-02-24T22:34:25.517 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.9FMKByJAzed8dSvlcT8pTazWirty0u7_0 2026-02-24T22:34:25.517 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13949358470545235972', time=2026-02-25T00:25:59.472060+0000, chain.objs.size()=1 2026-02-24T22:34:25.517 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.-jcyAKpT96mK-elEHVXuqwQwUOfRpk7_0 2026-02-24T22:34:25.517 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16832521232512083401', time=2026-02-25T00:25:59.496048+0000, chain.objs.size()=1 2026-02-24T22:34:25.517 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.GMiHjOsSwrdDORWDArqw0PYmkERKDon_0 2026-02-24T22:34:25.517 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12775748326143598001', time=2026-02-25T00:25:59.573708+0000, chain.objs.size()=1 2026-02-24T22:34:25.517 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.M60RR3T4NflZlRljpnrp06I9qaFzPXN_0 2026-02-24T22:34:25.517 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16222347195672818445', time=2026-02-25T00:25:59.574185+0000, chain.objs.size()=1 2026-02-24T22:34:25.517 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.kWFUQXV6Ei1CTPgD2CNELXM_dIDrzj0_0 2026-02-24T22:34:25.518 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4411014953872514340', time=2026-02-25T00:25:59.575684+0000, chain.objs.size()=1 2026-02-24T22:34:25.518 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.d4hnT-blGVAOBfUbexmr-ihawR876H3_0 2026-02-24T22:34:25.518 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17855928084847674994', time=2026-02-25T00:25:59.616574+0000, chain.objs.size()=1 2026-02-24T22:34:25.518 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.F907IcFhjI_BqnJNDcjJg5SawKER2ji_0 2026-02-24T22:34:25.518 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.103400332180403077', time=2026-02-25T00:25:59.636798+0000, chain.objs.size()=1 2026-02-24T22:34:25.518 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.bwBeO5_zR9AAH23VbJXEAsgfvk10JBa_0 2026-02-24T22:34:25.518 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1278174542087659639', time=2026-02-25T00:25:59.654298+0000, chain.objs.size()=1 2026-02-24T22:34:25.518 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.UhAfZynB0lnhw1Z15_uKLioX8JCYo7N_0 2026-02-24T22:34:25.518 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9458015476238166675', time=2026-02-25T00:25:59.663704+0000, chain.objs.size()=1 2026-02-24T22:34:25.518 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.jmVA9zBSW5TDBF4Ve6rDq-Bn3SrwCrX_0 2026-02-24T22:34:25.518 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7825875211972250157', time=2026-02-25T00:25:59.664542+0000, chain.objs.size()=1 2026-02-24T22:34:25.518 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.1lHlKqthT1_HSGK8qGb30efCT4IDUSH_0 2026-02-24T22:34:25.518 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.187800484449193253', time=2026-02-25T00:25:59.670232+0000, chain.objs.size()=1 2026-02-24T22:34:25.518 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.DVQYKUgf_qzhSI7u6zEC4LzGavm0NjB_0 2026-02-24T22:34:25.518 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7384994595167118384', time=2026-02-25T00:25:59.741278+0000, chain.objs.size()=1 2026-02-24T22:34:25.518 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.b6UgVTPchJJrSRc9g7PxmrKwFV86j3j_0 2026-02-24T22:34:25.518 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13917887555784736866', time=2026-02-25T00:25:59.742540+0000, chain.objs.size()=1 2026-02-24T22:34:25.518 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.YDWTuN6E8xMpPuCrYa719n9XaMizkvE_0 2026-02-24T22:34:25.518 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3201038330269906941', time=2026-02-25T00:25:59.762795+0000, chain.objs.size()=1 2026-02-24T22:34:25.518 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Bk-gn9SAr5entK8dgT_MXbZhhmIfd2e_0 2026-02-24T22:34:25.518 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9273621180912703354', time=2026-02-25T00:25:59.775648+0000, chain.objs.size()=1 2026-02-24T22:34:25.518 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.527+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.KaRKowifuwiZtmlmGhSAl5WiY9TWduw_0 2026-02-24T22:34:25.519 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11396802155640209773', time=2026-02-25T00:25:59.791531+0000, chain.objs.size()=1 2026-02-24T22:34:25.519 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.y9_nfqxKg_hcm5eYn059zsWcAnYfiBR_0 2026-02-24T22:34:25.519 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2061225625309700029', time=2026-02-25T00:25:59.797857+0000, chain.objs.size()=1 2026-02-24T22:34:25.519 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.wlNWdmGpvZdivc4gCEvxb1m3AW6Iriq_0 2026-02-24T22:34:25.519 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7835104539702182832', time=2026-02-25T00:25:59.807835+0000, chain.objs.size()=1 2026-02-24T22:34:25.519 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.9yUkvr-1aD8ULmSWtClGdzNwmsCTjRo_0 2026-02-24T22:34:25.519 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8662902389780975922', time=2026-02-25T00:25:59.908639+0000, chain.objs.size()=1 2026-02-24T22:34:25.519 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ACgA6b2GWhL-T8fSSp4xNn7Dfx2q3EQ_0 2026-02-24T22:34:25.519 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4849915156497307749', time=2026-02-25T00:25:59.910477+0000, chain.objs.size()=1 2026-02-24T22:34:25.519 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.y-FsoXJAcfJFVCGaGswJ9JnItJMQV5A_0 2026-02-24T22:34:25.519 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5194591049415352777', time=2026-02-25T00:25:59.911628+0000, chain.objs.size()=1 2026-02-24T22:34:25.519 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.fQreZ_Xz3m_mG5tNKRHiga8hyqFaaFV_0 2026-02-24T22:34:25.519 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13936503365327752756', time=2026-02-25T00:25:59.912010+0000, chain.objs.size()=1 2026-02-24T22:34:25.519 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.A2iw9lY12aVf1tAz5uNdcZmr4IwEw6t_0 2026-02-24T22:34:25.519 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11994991714010694473', time=2026-02-25T00:25:59.914772+0000, chain.objs.size()=1 2026-02-24T22:34:25.519 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.zUyjCSOBRoPoz596TqNWLjYYrguFymm_0 2026-02-24T22:34:25.519 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7637043195007912013', time=2026-02-25T00:25:59.922118+0000, chain.objs.size()=1 2026-02-24T22:34:25.520 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.HvRVt0FNZ_k7z-9KHae6Fcouq-M0hog_0 2026-02-24T22:34:25.520 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17631918041203130281', time=2026-02-25T00:25:59.932988+0000, chain.objs.size()=1 2026-02-24T22:34:25.520 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.qf6Y_NuSr5JbFNiWt4Q9KY02GEeCsxR_0 2026-02-24T22:34:25.520 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.533237247515288864', time=2026-02-25T00:25:59.974148+0000, chain.objs.size()=1 2026-02-24T22:34:25.520 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.P7dEXTtKFW31QIZ_lSDKRH_mAH2VSuL_0 2026-02-24T22:34:25.520 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7043056168321673676', time=2026-02-25T00:26:00.063141+0000, chain.objs.size()=1 2026-02-24T22:34:25.520 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.SZBr5PWFESP4Kk5HyKAICRm5jE7yce3_0 2026-02-24T22:34:25.520 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17435172379841477566', time=2026-02-25T00:26:00.124851+0000, chain.objs.size()=1 2026-02-24T22:34:25.520 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.bqF2z_P1W3_8R3whlSW7Jo9A7itxgKB_0 2026-02-24T22:34:25.520 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.821845535025341491', time=2026-02-25T00:26:00.148009+0000, chain.objs.size()=1 2026-02-24T22:34:25.520 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.0Mh14bVaaNa0iC7N64qgz9-uguLCghU_0 2026-02-24T22:34:25.520 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15134180356180083531', time=2026-02-25T00:26:00.158965+0000, chain.objs.size()=1 2026-02-24T22:34:25.520 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.FGnwuExEIT1AFhHFnEl4IkgvuQOlSuc_0 2026-02-24T22:34:25.520 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1812526804050703316', time=2026-02-25T00:26:00.259810+0000, chain.objs.size()=1 2026-02-24T22:34:25.520 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.DHTgBLzVRFGp_Nc3Jo5g--V7SDi1CHi_0 2026-02-24T22:34:25.520 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1754878247464119633', time=2026-02-25T00:26:00.291306+0000, chain.objs.size()=1 2026-02-24T22:34:25.520 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.I4cTeqvzAMf_lX3AR6mfGtmUIBNY2a-_0 2026-02-24T22:34:25.520 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16655825122863468355', time=2026-02-25T00:26:07.113173+0000, chain.objs.size()=1 2026-02-24T22:34:25.520 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1640__shadow_.uwTwmSQvJ2QD69WaZjeMAxlR55Gw68h_0 2026-02-24T22:34:25.520 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5904167146256361542', time=2026-02-25T00:26:08.849484+0000, chain.objs.size()=1 2026-02-24T22:34:25.520 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1643__shadow_.XsFP9svtk33Fflsi6cjD6FnCoHIEWhY_0 2026-02-24T22:34:25.520 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17499350924808419360', time=2026-02-25T00:26:14.230162+0000, chain.objs.size()=1 2026-02-24T22:34:25.520 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1652__shadow_.TD-4TkNgLCWYrzujbHnmIdywREuu7_C_0 2026-02-24T22:34:25.521 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2162824065022389019', time=2026-02-25T00:26:16.025769+0000, chain.objs.size()=1 2026-02-24T22:34:25.521 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1655__shadow_.fGzDBTnQt7BITHwATXk2MZhrLe_EaVR_0 2026-02-24T22:34:25.521 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7535045262228419180', time=2026-02-25T00:26:18.511748+0000, chain.objs.size()=1 2026-02-24T22:34:25.521 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1658__shadow_.7TaJ59xhnHCidajjvAbcxl30reYfrgP_0 2026-02-24T22:34:25.521 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1164543582173556330', time=2026-02-25T00:26:18.520140+0000, chain.objs.size()=1 2026-02-24T22:34:25.521 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1658__shadow_.Nwq_jECQPnuIMEJYMo3-gxm5-ysaH-H_0 2026-02-24T22:34:25.521 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10225487474472951563', time=2026-02-25T00:26:24.065644+0000, chain.objs.size()=1 2026-02-24T22:34:25.521 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1667__shadow_.N2VIbWgH18JyJHKXN5IhLkK3SQ2YqQw_0 2026-02-24T22:34:25.521 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10479237876080329049', time=2026-02-25T00:26:35.780551+0000, chain.objs.size()=1 2026-02-24T22:34:25.521 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:VJiryDR3PPu1l7fz6QezP88q8L0.bFa_.57skJ1COQTx4ZD85xVA8w4IJnppiNGC_0 2026-02-24T22:34:25.521 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10479237876080329049', time=2026-02-25T00:26:35.792857+0000, chain.objs.size()=1 2026-02-24T22:34:25.521 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:VJiryDR3PPu1l7fz6QezP88q8L0.bFa_.57skJ1COQTx4ZD85xVA8w4IJnppiNGC_0 2026-02-24T22:34:25.521 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10479237876080329049', time=2026-02-25T00:26:35.818323+0000, chain.objs.size()=1 2026-02-24T22:34:25.521 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:VJiryDR3PPu1l7fz6QezP88q8L0.bFa_.57skJ1COQTx4ZD85xVA8w4IJnppiNGC_0 2026-02-24T22:34:25.521 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10479237876080329049', time=2026-02-25T00:26:35.838766+0000, chain.objs.size()=1 2026-02-24T22:34:25.521 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:VJiryDR3PPu1l7fz6QezP88q8L0.bFa_.57skJ1COQTx4ZD85xVA8w4IJnppiNGC_0 2026-02-24T22:34:25.521 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10479237876080329049', time=2026-02-25T00:26:35.882554+0000, chain.objs.size()=1 2026-02-24T22:34:25.521 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:VJiryDR3PPu1l7fz6QezP88q8L0.bFa_.57skJ1COQTx4ZD85xVA8w4IJnppiNGC_0 2026-02-24T22:34:25.521 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8439314439497985618', time=2026-02-25T00:26:44.829333+0000, chain.objs.size()=1 2026-02-24T22:34:25.522 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.tWRDh21qZHym6fvI1USuJ7yu2bE_ftq_0 2026-02-24T22:34:25.522 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7597559919549274832', time=2026-02-25T00:26:44.835207+0000, chain.objs.size()=1 2026-02-24T22:34:25.522 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.zB0nPK1AfWMI-iRelnJBuWg9bFsp8Wa_0 2026-02-24T22:34:25.522 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1211357466393970429', time=2026-02-25T00:26:44.846939+0000, chain.objs.size()=1 2026-02-24T22:34:25.522 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.4g2otn9nPzsYs0o5MCItAq9ZrLNhv7b_0 2026-02-24T22:34:25.522 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3120284212860457261', time=2026-02-25T00:26:44.851180+0000, chain.objs.size()=1 2026-02-24T22:34:25.522 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.S-ufjL0E2QuVlcBf_4ZMnRRP67HWndp_0 2026-02-24T22:34:25.522 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.830825183615072509', time=2026-02-25T00:26:44.852070+0000, chain.objs.size()=1 2026-02-24T22:34:25.522 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.xlhL057iCapc4gr-dY7imStwSUFE3kw_0 2026-02-24T22:34:25.522 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2587165533140196239', time=2026-02-25T00:26:44.961242+0000, chain.objs.size()=1 2026-02-24T22:34:25.522 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.WhloAHQVlvFzw56tGwzZpnQPXg7WrnM_0 2026-02-24T22:34:25.522 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9740518110719888532', time=2026-02-25T00:26:44.969775+0000, chain.objs.size()=1 2026-02-24T22:34:25.522 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.531+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.mjLW2Hpce9Be_QQQXo-lxvFFk96nUYc_0 2026-02-24T22:34:25.522 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11115293113783129656', time=2026-02-25T00:26:44.986478+0000, chain.objs.size()=1 2026-02-24T22:34:25.522 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.HgibkGuapavV8NlQDFGW30gRvJEo2FQ_0 2026-02-24T22:34:25.523 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12787005738268958347', time=2026-02-25T00:26:45.004974+0000, chain.objs.size()=1 2026-02-24T22:34:25.523 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Ga2Qe06C5wmdiXm-NqIFHp54YKoJ9Hp_0 2026-02-24T22:34:25.523 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9135597117689035249', time=2026-02-25T00:26:45.120496+0000, chain.objs.size()=1 2026-02-24T22:34:25.523 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.hl0QNBu8HWD6iLtCjeqzLc0SOH1jHGb_0 2026-02-24T22:34:25.523 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2955771296340056726', time=2026-02-25T00:26:45.299034+0000, chain.objs.size()=1 2026-02-24T22:34:25.523 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.j-D-PZX_D4INkWwr_v6S1FxN0GFTdtp_0 2026-02-24T22:34:25.523 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5048218286120081385', time=2026-02-25T00:26:45.330096+0000, chain.objs.size()=1 2026-02-24T22:34:25.523 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.nBbuLyTdgrz2yRM-m3mVgdMHvieNPiI_0 2026-02-24T22:34:25.523 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15875930042814849332', time=2026-02-25T00:26:45.415053+0000, chain.objs.size()=1 2026-02-24T22:34:25.523 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_._GrwFyVwV_s1haBaPca9Ly52FoLCqSU_0 2026-02-24T22:34:25.523 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13359711323229716518', time=2026-02-25T00:26:45.486472+0000, chain.objs.size()=1 2026-02-24T22:34:25.523 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.pRBZv6w6P7GAZ-aoLVMhV39XU58q-0E_0 2026-02-24T22:34:25.523 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12841927819690243358', time=2026-02-25T00:26:45.577298+0000, chain.objs.size()=1 2026-02-24T22:34:25.523 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.iKwGPwoCLSFZ9FFDAceVd_bDPxOY5pp_0 2026-02-24T22:34:25.523 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8101767483574176278', time=2026-02-25T00:26:45.579388+0000, chain.objs.size()=1 2026-02-24T22:34:25.523 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ZjjHk2LWW_9XsPapec1Nh4bkCjd2KFZ_0 2026-02-24T22:34:25.524 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7636197745554520025', time=2026-02-25T00:26:45.599878+0000, chain.objs.size()=1 2026-02-24T22:34:25.524 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.g4PcSEuK_bGqRmAEZ-gEHbUSDNXWOHU_0 2026-02-24T22:34:25.524 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5505885948327363378', time=2026-02-25T00:26:45.823683+0000, chain.objs.size()=1 2026-02-24T22:34:25.524 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_._yJCyGz8Dc6_dHHwmdmmFbzlHxgVxTw_0 2026-02-24T22:34:25.524 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18398290998098512964', time=2026-02-25T00:26:45.937723+0000, chain.objs.size()=1 2026-02-24T22:34:25.524 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.gR_7AazcWxhKKs09zQOjdfJsUbzZ38o_0 2026-02-24T22:34:25.524 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8378118607302910809', time=2026-02-25T00:26:45.938251+0000, chain.objs.size()=1 2026-02-24T22:34:25.524 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.KNF6Y0zFzPplGzPJIm1PqAf35YehQ3Q_0 2026-02-24T22:34:25.524 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5406574637399762443', time=2026-02-25T00:26:46.095027+0000, chain.objs.size()=1 2026-02-24T22:34:25.524 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.WPxruAbmyJRPiYIiM5-ZZNtFtaUwgga_0 2026-02-24T22:34:25.524 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9915143018458980573', time=2026-02-25T00:26:46.104502+0000, chain.objs.size()=1 2026-02-24T22:34:25.524 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Ww873yyDo9YUKcQzs9v-AZuqBdV_HID_0 2026-02-24T22:34:25.524 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15955176623496210794', time=2026-02-25T00:26:46.160832+0000, chain.objs.size()=1 2026-02-24T22:34:25.524 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.0r_Dpb6ZoV5LvwNAyjs50HTIYqKTY7u_0 2026-02-24T22:34:25.524 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6232647969452179842', time=2026-02-25T00:26:53.166873+0000, chain.objs.size()=1 2026-02-24T22:34:25.524 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.gajgRIcUNyKv9YfnQ5J87yFHiCzfO1e_0 2026-02-24T22:34:25.524 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5487816995479753450', time=2026-02-25T00:26:53.175312+0000, chain.objs.size()=1 2026-02-24T22:34:25.524 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.j-Ricztvoyd90_HRYa0pvdlAi8IQZYn_0 2026-02-24T22:34:25.524 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12833122614709809983', time=2026-02-25T00:26:53.339915+0000, chain.objs.size()=1 2026-02-24T22:34:25.525 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.hAFa7ylMxtvocaooQbBesbH4hjeG371_0 2026-02-24T22:34:25.525 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16369163819394235854', time=2026-02-25T00:26:53.343921+0000, chain.objs.size()=1 2026-02-24T22:34:25.525 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.K5Hz_a69aTpcXdoMnMTh5myNuTI0O-1_0 2026-02-24T22:34:25.525 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17015854980443764542', time=2026-02-25T00:26:53.385240+0000, chain.objs.size()=1 2026-02-24T22:34:25.525 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.sZ4eh7552_Y0cp8p_ln2lKRcmspGlO3_0 2026-02-24T22:34:25.525 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15026160872932517573', time=2026-02-25T00:26:53.389066+0000, chain.objs.size()=1 2026-02-24T22:34:25.525 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.W-772SQbxCuZ1MdguRC7fqq0ct2A6be_0 2026-02-24T22:34:25.525 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10836126853185393895', time=2026-02-25T00:26:53.390073+0000, chain.objs.size()=1 2026-02-24T22:34:25.525 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.d5yqL4LCdxDg-CLiWPbg0c5QHPzDeia_0 2026-02-24T22:34:25.525 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15804404905240258941', time=2026-02-25T00:26:53.400226+0000, chain.objs.size()=1 2026-02-24T22:34:25.525 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.8vMMNQ-m48TrQvNBb0vYrFqtsqHmMID_0 2026-02-24T22:34:25.525 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3006376996340562773', time=2026-02-25T00:26:53.420879+0000, chain.objs.size()=1 2026-02-24T22:34:25.525 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ayY-gsp0UYXK9dyztYLC3Zu0QZajpCb_0 2026-02-24T22:34:25.525 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.535+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/38184 2026-02-24T22:34:25.527 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=35, truncated=0, next_marker='' 2026-02-24T22:34:25.527 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8862868571672176289', time=2026-02-25T00:26:53.427191+0000, chain.objs.size()=1 2026-02-24T22:34:25.527 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.kenJ5Kcr_j8mMabsCrvCcN_yh1qFgxy_0 2026-02-24T22:34:25.527 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16507811591911987819', time=2026-02-25T00:26:53.534466+0000, chain.objs.size()=1 2026-02-24T22:34:25.527 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ttLhnnUO07LR7bn4bRDNWH2aVuOh_cn_0 2026-02-24T22:34:25.527 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10453552018300516443', time=2026-02-25T00:26:53.537066+0000, chain.objs.size()=1 2026-02-24T22:34:25.527 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Q22j8NDqpIE7FxIw6camUrTKNXNMHdO_0 2026-02-24T22:34:25.527 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4308565373830232912', time=2026-02-25T00:26:53.554956+0000, chain.objs.size()=1 2026-02-24T22:34:25.527 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.CIs8r1r2jyC5vPWYYhlq4eZK0LAtQ66_0 2026-02-24T22:34:25.527 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16146271878303114174', time=2026-02-25T00:26:53.585743+0000, chain.objs.size()=1 2026-02-24T22:34:25.527 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.5YD-wC-cLD9smN2htWSnRJJ_nkfa0gB_0 2026-02-24T22:34:25.527 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2667321500805936858', time=2026-02-25T00:26:53.605387+0000, chain.objs.size()=1 2026-02-24T22:34:25.527 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.DZoGZRe6zKYTk-dC0zf6vdd9AaQQbct_0 2026-02-24T22:34:25.527 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2091653879207673186', time=2026-02-25T00:26:53.635656+0000, chain.objs.size()=1 2026-02-24T22:34:25.527 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.YQY6RpMYR4QEg6K8bJ34mbn78mezWQl_0 2026-02-24T22:34:25.527 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12494459443885475797', time=2026-02-25T00:26:53.807124+0000, chain.objs.size()=1 2026-02-24T22:34:25.527 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.nvpLXpp0o4FNNR0cPo1qILGW1oiiG4l_0 2026-02-24T22:34:25.527 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18404908228684774200', time=2026-02-25T00:26:53.971324+0000, chain.objs.size()=1 2026-02-24T22:34:25.527 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Bkt6wRS7B6th-V0uSeUKu66QLSADIv2_0 2026-02-24T22:34:25.527 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4157236164456631841', time=2026-02-25T00:26:53.996906+0000, chain.objs.size()=1 2026-02-24T22:34:25.527 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.jj5cwaxT7Ctvb2nZohl6BFRTXPqd34u_0 2026-02-24T22:34:25.527 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14601066718790886153', time=2026-02-25T00:26:54.030329+0000, chain.objs.size()=1 2026-02-24T22:34:25.528 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.mk6qJZTol-HGp8y8NyYjQBbVe8N-5V__0 2026-02-24T22:34:25.528 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14088779676088688570', time=2026-02-25T00:26:54.138194+0000, chain.objs.size()=1 2026-02-24T22:34:25.528 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.3bAW_TUge8AkUgfgEi1QhBuCFuOFp7K_0 2026-02-24T22:34:25.528 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1405327021707355189', time=2026-02-25T00:26:54.227524+0000, chain.objs.size()=1 2026-02-24T22:34:25.528 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.6MFjvxOAy3nXw75M_pv9es6GKrmKXT2_0 2026-02-24T22:34:25.528 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6342302316877209647', time=2026-02-25T00:26:54.314247+0000, chain.objs.size()=1 2026-02-24T22:34:25.528 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_._FavVTdUkIpQT33YZDhFOVxuXlCtxP1_0 2026-02-24T22:34:25.528 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.492996454156808233', time=2026-02-25T00:26:54.331854+0000, chain.objs.size()=1 2026-02-24T22:34:25.528 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.hocyhdbr2kHXwAKDOJvDLiNaL4ybn_S_0 2026-02-24T22:34:25.528 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18280050452084264436', time=2026-02-25T00:26:54.344737+0000, chain.objs.size()=1 2026-02-24T22:34:25.528 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.n354VWe4No4e77viaMGxrMF7H3CwwZ-_0 2026-02-24T22:34:25.528 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17843541785421616649', time=2026-02-25T00:26:54.393698+0000, chain.objs.size()=1 2026-02-24T22:34:25.528 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.mWA_O5PjrIowmknaPluUAxbkKWjI8pg_0 2026-02-24T22:34:25.528 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11099091268696973713', time=2026-02-25T00:26:54.394089+0000, chain.objs.size()=1 2026-02-24T22:34:25.528 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.HXdRzX1WjU_n5aB6FmWV3xsIVwahyVY_0 2026-02-24T22:34:25.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4705757840569503725', time=2026-02-25T00:26:54.467170+0000, chain.objs.size()=1 2026-02-24T22:34:25.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.8pkxGAJRT_S94I7_jmYaA5jnXFnNeE5_0 2026-02-24T22:34:25.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8070552608850203809', time=2026-02-25T00:26:54.500461+0000, chain.objs.size()=1 2026-02-24T22:34:25.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.1r942XiRG8DRfe8E7qjy8l00kDX8zpc_0 2026-02-24T22:34:25.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11432870351323258408', time=2026-02-25T00:27:24.992874+0000, chain.objs.size()=1 2026-02-24T22:34:25.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1743__shadow_.-8evQ1Vdh2R56iejGd90IKrDGmZqD1m_0 2026-02-24T22:34:25.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5633844820809729011', time=2026-02-25T00:27:59.304753+0000, chain.objs.size()=1 2026-02-24T22:34:25.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1794__shadow_.1DeIxP07Xwi3pAnB3bUE7XcQo__iJcG_0 2026-02-24T22:34:25.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6426551671664173357', time=2026-02-25T00:28:02.207174+0000, chain.objs.size()=1 2026-02-24T22:34:25.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1799__shadow_.YxxbPJSF_ujHPLvJKMeAS7Jeus3p5fx_0 2026-02-24T22:34:25.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12375905375690075181', time=2026-02-25T00:28:33.576579+0000, chain.objs.size()=1 2026-02-24T22:34:25.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1851__shadow_.YbmqDSMAerH4XPrSGP_iKn_nc1QJH9D_0 2026-02-24T22:34:25.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1120200064361882964', time=2026-02-25T00:28:43.675435+0000, chain.objs.size()=1 2026-02-24T22:34:25.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1869__shadow:ba1nU1fNR3N1sfUQ2kL8K8Wm.5Ibl31_.IVNpljy9O7AjWtEyJ7RLhDmshakOZnh_0 2026-02-24T22:34:25.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2206466168383553560', time=2026-02-25T00:28:52.508563+0000, chain.objs.size()=1 2026-02-24T22:34:25.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1884__multipart_mymultipart.2~Lm909dbKJ77WJQT9guVS4tfgUzebywk.1 2026-02-24T22:34:25.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13544653050780690399', time=2026-02-25T00:28:57.855959+0000, chain.objs.size()=1 2026-02-24T22:34:25.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1886__shadow:GK665f717TjFM2yW3JN8NUin5lIdEQV_.LrWz9SiY_cpHNbBu_gHxJq-b_vHSu8a_0 2026-02-24T22:34:25.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10692025352941880600', time=2026-02-25T00:29:13.327413+0000, chain.objs.size()=3 2026-02-24T22:34:25.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1891__multipart_mymultipart.2~GXgzmANFeGoM3IdQbe3aiI_O260sS3Q.1 2026-02-24T22:34:25.529 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1891__multipart_mymultipart.2~GXgzmANFeGoM3IdQbe3aiI_O260sS3Q.2 2026-02-24T22:34:25.530 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1891__multipart_mymultipart.2~GXgzmANFeGoM3IdQbe3aiI_O260sS3Q.3 2026-02-24T22:34:25.530 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='2~p4QXSbjJYagQ2QikQ0tB6YJv6tLI1Au', time=2026-02-25T00:29:14.777265+0000, chain.objs.size()=10 2026-02-24T22:34:25.530 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1892__multipart_mymultipart.2~p4QXSbjJYagQ2QikQ0tB6YJv6tLI1Au.1 2026-02-24T22:34:25.530 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1892__multipart_mymultipart.2~p4QXSbjJYagQ2QikQ0tB6YJv6tLI1Au.2 2026-02-24T22:34:25.530 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1892__multipart_mymultipart.2~p4QXSbjJYagQ2QikQ0tB6YJv6tLI1Au.3 2026-02-24T22:34:25.530 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1892__multipart_mymultipart.2~p4QXSbjJYagQ2QikQ0tB6YJv6tLI1Au.4 2026-02-24T22:34:25.530 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1892__multipart_mymultipart.2~p4QXSbjJYagQ2QikQ0tB6YJv6tLI1Au.5 2026-02-24T22:34:25.530 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1892__multipart_mymultipart.2~p4QXSbjJYagQ2QikQ0tB6YJv6tLI1Au.6 2026-02-24T22:34:25.530 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.539+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1892__multipart_mymultipart.2~p4QXSbjJYagQ2QikQ0tB6YJv6tLI1Au.7 2026-02-24T22:34:25.530 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.543+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1892__multipart_mymultipart.2~p4QXSbjJYagQ2QikQ0tB6YJv6tLI1Au.8 2026-02-24T22:34:25.530 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.543+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1892__multipart_mymultipart.2~p4QXSbjJYagQ2QikQ0tB6YJv6tLI1Au.9 2026-02-24T22:34:25.530 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.543+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1892__multipart_mymultipart.2~p4QXSbjJYagQ2QikQ0tB6YJv6tLI1Au.10 2026-02-24T22:34:25.531 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.543+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9282736759838624719', time=2026-02-25T00:30:10.515964+0000, chain.objs.size()=1 2026-02-24T22:34:25.531 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.543+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1929__multipart_mymultipart.2~OFITD6l2luaY4w1lZcQOdgVbT1pWLFy.1 2026-02-24T22:34:25.531 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.543+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10370460488382854907', time=2026-02-25T00:30:14.644769+0000, chain.objs.size()=1 2026-02-24T22:34:25.531 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.543+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1932__shadow_.Z_R7vw7WkPvSuMpn_ipcK54U-2xYsjl_0 2026-02-24T22:34:25.531 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.543+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16067231425452750723', time=2026-02-25T00:30:24.211254+0000, chain.objs.size()=1 2026-02-24T22:34:25.531 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.543+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1938__shadow:HIm9eDKPV9xwv7EMPMm7ovkxmcStwx4_.CWF6jxte_R0mWJHOL-nf_Ihj60ceNc-_0 2026-02-24T22:34:25.531 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.543+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14258534535987676028', time=2026-02-25T00:30:24.319005+0000, chain.objs.size()=1 2026-02-24T22:34:25.531 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.543+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1938__shadow:ZcLlNQbrTNLHM-xaBPSPoZSmKsVvS12_.0bBwkZYsMIeZG5bwThM5hwOMDxxVhvr_0 2026-02-24T22:34:25.531 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.543+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7800050750218183219', time=2026-02-25T00:30:52.406109+0000, chain.objs.size()=1 2026-02-24T22:34:25.531 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.543+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1950__shadow:1J0Mgj4PHIQwvlYxfMGpbYQEahXIYwp_.zUTs7Goq8b7TeuXXHhAVDabGV0-iy_X_0 2026-02-24T22:34:25.531 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.543+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18205229268536678030', time=2026-02-25T00:33:47.410457+0000, chain.objs.size()=1 2026-02-24T22:34:25.531 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.543+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1970__shadow_.7bTT0fBW52vxC43G0CzxmGNCqAbGTI3_0 2026-02-24T22:34:25.532 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.543+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.533 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.543+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2026-02-24T22:34:25.534 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.535 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/38732' 2026-02-24T22:34:25.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17135096283144166018', time=2026-02-25T00:15:34.632919+0000, chain.objs.size()=1 2026-02-24T22:34:25.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506__shadow_.pDOXUX8a_f8l6cGjUX2faN9weGnPaQZ_0 2026-02-24T22:34:25.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.376449562292424880', time=2026-02-25T00:15:34.761815+0000, chain.objs.size()=3 2026-02-24T22:34:25.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-STANDARD.2~jB8VjG_eXaI_0rIVLXOTPSldrI2t5vd.1 2026-02-24T22:34:25.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-STANDARD.2~jB8VjG_eXaI_0rIVLXOTPSldrI2t5vd.2 2026-02-24T22:34:25.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-STANDARD.2~jB8VjG_eXaI_0rIVLXOTPSldrI2t5vd.3 2026-02-24T22:34:25.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18117921011956775801', time=2026-02-25T00:15:34.786830+0000, chain.objs.size()=3 2026-02-24T22:34:25.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515__multipart_foo-STANDARD.2~FbIvQzmk1UO-cPN9QEsQGk3xJ3CxYDu.1 2026-02-24T22:34:25.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515__multipart_foo-STANDARD.2~FbIvQzmk1UO-cPN9QEsQGk3xJ3CxYDu.2 2026-02-24T22:34:25.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515__multipart_foo-STANDARD.2~FbIvQzmk1UO-cPN9QEsQGk3xJ3CxYDu.3 2026-02-24T22:34:25.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4485438413893670175', time=2026-02-25T00:16:31.960445+0000, chain.objs.size()=1 2026-02-24T22:34:25.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520__shadow_.UqBMPJTsGdTXyUJrQ855_veVy5Sp0Cr_0 2026-02-24T22:34:25.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17787149830135188134', time=2026-02-25T00:16:33.510277+0000, chain.objs.size()=1 2026-02-24T22:34:25.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521__shadow_.mBBrnrdyQJB0tzVc98MCut5Lisdn4uY_0 2026-02-24T22:34:25.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4653077730293781085', time=2026-02-25T00:17:05.167116+0000, chain.objs.size()=1 2026-02-24T22:34:25.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522__shadow_.7kgLPCITIqbGEzQoQwSVXg0DP_xjSFs_0 2026-02-24T22:34:25.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14945957902924362143', time=2026-02-25T00:17:07.114355+0000, chain.objs.size()=1 2026-02-24T22:34:25.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523__shadow_.U4rIYkYtlGBBnnDfZc2kV-CXeCoWaLW_0 2026-02-24T22:34:25.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9573315879382028037', time=2026-02-25T00:17:08.003988+0000, chain.objs.size()=1 2026-02-24T22:34:25.536 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524__shadow_.hrG80Psdvj3FlRT181oD-3nDF-exSBF_0 2026-02-24T22:34:25.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9262772859033654502', time=2026-02-25T00:17:08.005581+0000, chain.objs.size()=1 2026-02-24T22:34:25.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524__shadow_.VbJZe0rTB0fajsj9RDbfJDb2T3HY8HZ_0 2026-02-24T22:34:25.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14209475266095962482', time=2026-02-25T00:17:39.879962+0000, chain.objs.size()=1 2026-02-24T22:34:25.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525__shadow_.QG_AXBxZy4pwwHe_EDc1DliGGhAhuY7_0 2026-02-24T22:34:25.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7319658961923529795', time=2026-02-25T00:18:14.570563+0000, chain.objs.size()=1 2026-02-24T22:34:25.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528__shadow_.6dyv3w10rXTUEsvqO42Ad483UcSj6AE_0 2026-02-24T22:34:25.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.891574574088456795', time=2026-02-25T00:18:48.183672+0000, chain.objs.size()=1 2026-02-24T22:34:25.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530__shadow_.UhMRz4UzQHcXnjfsX70SxsY_CIH3gUu_0 2026-02-24T22:34:25.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15691931108765318282', time=2026-02-25T00:19:23.058580+0000, chain.objs.size()=1 2026-02-24T22:34:25.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533__shadow_.PQydp4PPdz2dYgbst1edWqwtR8C92Gw_0 2026-02-24T22:34:25.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13555785770439747753', time=2026-02-25T00:20:30.730529+0000, chain.objs.size()=1 2026-02-24T22:34:25.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537__shadow_.uGavSZynqSADE4pgSuFw4mzAgyOSj73_0 2026-02-24T22:34:25.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3789790423019053753', time=2026-02-25T00:20:32.512635+0000, chain.objs.size()=1 2026-02-24T22:34:25.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538__shadow_.9N3VMz-Qp37AbnB8ozjfFmDz8cMSXe4_0 2026-02-24T22:34:25.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12213573486399864524', time=2026-02-25T00:20:32.515176+0000, chain.objs.size()=1 2026-02-24T22:34:25.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538__shadow_.5CC4qYWGTYwmgrjPUSqAKlhQCEF_MHB_0 2026-02-24T22:34:25.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5526251225557034719', time=2026-02-25T00:20:34.029758+0000, chain.objs.size()=1 2026-02-24T22:34:25.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539__shadow_.hiTKSygSJP1Ndvj6LnUxLichpcSri1k_0 2026-02-24T22:34:25.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16191699127923443502', time=2026-02-25T00:21:06.309113+0000, chain.objs.size()=1 2026-02-24T22:34:25.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540__shadow_.VsJH_DT9SuTTs5oqxvB0o8CH_GZJy___0 2026-02-24T22:34:25.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15002360316538013574', time=2026-02-25T00:23:14.349807+0000, chain.objs.size()=1 2026-02-24T22:34:25.537 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518__shadow_.Inshac19kJG2zew9mv-c54ma1VYHW-i_0 2026-02-24T22:34:25.538 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11296083269835606864', time=2026-02-25T00:23:14.350917+0000, chain.objs.size()=1 2026-02-24T22:34:25.538 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518__shadow_.JYL2yTqC3JbuxwcUAt25Kek0qY_rtJm_0 2026-02-24T22:34:25.538 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6992868706919425709', time=2026-02-25T00:25:34.875035+0000, chain.objs.size()=1 2026-02-24T22:34:25.538 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1599__shadow_.j8QSmdofKC2jCYj7fKSkRPon6Fi5Dvu_0 2026-02-24T22:34:25.538 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.648596889459146231', time=2026-02-25T00:25:38.491930+0000, chain.objs.size()=1 2026-02-24T22:34:25.538 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.547+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1604__shadow_.2K48pT3hnAK24_vcxG7opYSMdmEO-M7_0 2026-02-24T22:34:25.538 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4050987071992634913', time=2026-02-25T00:25:48.506756+0000, chain.objs.size()=1 2026-02-24T22:34:25.538 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1620__shadow_.aRCQnOzrNURoSsBPvBc3HdXvBKxUf3D_0 2026-02-24T22:34:25.538 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12279309929128034645', time=2026-02-25T00:25:49.698476+0000, chain.objs.size()=1 2026-02-24T22:34:25.538 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1622__shadow_.lqbYvhqFZNXvJ18aIQZynsdm0voNWSN_0 2026-02-24T22:34:25.538 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5064076359333704282', time=2026-02-25T00:25:59.074028+0000, chain.objs.size()=1 2026-02-24T22:34:25.538 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.d5W-OUMYvJDG8uLlJUaS3MQZuj2CBql_0 2026-02-24T22:34:25.539 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5606506845833796350', time=2026-02-25T00:25:59.074947+0000, chain.objs.size()=1 2026-02-24T22:34:25.539 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.lCLstVLMz_yo3XXEKp97WwamdttzEb-_0 2026-02-24T22:34:25.539 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10934066901321152633', time=2026-02-25T00:25:59.082974+0000, chain.objs.size()=1 2026-02-24T22:34:25.539 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.OoVhVW-zr6AaBiBwhJMFTzZVvbLR1sv_0 2026-02-24T22:34:25.539 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8692849186962372391', time=2026-02-25T00:25:59.139187+0000, chain.objs.size()=1 2026-02-24T22:34:25.539 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.R1LWa7INvARXhq4f6mfszJwmvAtseUX_0 2026-02-24T22:34:25.539 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1592242312912255977', time=2026-02-25T00:25:59.249439+0000, chain.objs.size()=1 2026-02-24T22:34:25.539 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.NorW75dm2SGi0_KMlgHbjRo4MSEfQWX_0 2026-02-24T22:34:25.539 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2698581170241858454', time=2026-02-25T00:25:59.306600+0000, chain.objs.size()=1 2026-02-24T22:34:25.539 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Pt8KQA9JsH4a_fLbZYYyf6M6Krb50_h_0 2026-02-24T22:34:25.539 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16806876469300922597', time=2026-02-25T00:25:59.331006+0000, chain.objs.size()=1 2026-02-24T22:34:25.539 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_._iw6x5b41q_ouEV6K_DukqxTV8njLAs_0 2026-02-24T22:34:25.539 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5906549523129058312', time=2026-02-25T00:25:59.419734+0000, chain.objs.size()=1 2026-02-24T22:34:25.539 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.tzh6qpzU_ZhxX8U1a55HRgKtS6i401T_0 2026-02-24T22:34:25.539 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16591056232169463702', time=2026-02-25T00:25:59.455284+0000, chain.objs.size()=1 2026-02-24T22:34:25.539 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.1Pi4ZpcAC1VK6YdGJ6c6hFjlV_T6u8T_0 2026-02-24T22:34:25.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12596324177949354499', time=2026-02-25T00:25:59.468749+0000, chain.objs.size()=1 2026-02-24T22:34:25.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Fzs1zYkSwS1Am034UpOUz20z9HpwJ-D_0 2026-02-24T22:34:25.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9313129852711697713', time=2026-02-25T00:25:59.564613+0000, chain.objs.size()=1 2026-02-24T22:34:25.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.zgaeUp4Z0-sdozvEv5Jyv0j7i9K43c3_0 2026-02-24T22:34:25.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1469158197738505162', time=2026-02-25T00:25:59.587434+0000, chain.objs.size()=1 2026-02-24T22:34:25.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.EZ5Hpym7hnphETqFGUgHYeL0jXQSHwK_0 2026-02-24T22:34:25.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14616418402807656226', time=2026-02-25T00:25:59.594780+0000, chain.objs.size()=1 2026-02-24T22:34:25.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.SYw_fnyipFEglpJNjNC5aYcgSneUDtZ_0 2026-02-24T22:34:25.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2444168058584636562', time=2026-02-25T00:25:59.643260+0000, chain.objs.size()=1 2026-02-24T22:34:25.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.s_tI3t004f2WI50Vn3oQXQ9spUCkp8F_0 2026-02-24T22:34:25.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1460128948330794397', time=2026-02-25T00:25:59.731603+0000, chain.objs.size()=1 2026-02-24T22:34:25.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Y5zEAOzpbnClt5edV_gPqjYGo6RLCoD_0 2026-02-24T22:34:25.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6389275277193696527', time=2026-02-25T00:25:59.800370+0000, chain.objs.size()=1 2026-02-24T22:34:25.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.73I4vdBoKehuZB0PqX_IRlaAQ76zkXf_0 2026-02-24T22:34:25.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7668709233807087958', time=2026-02-25T00:25:59.922665+0000, chain.objs.size()=1 2026-02-24T22:34:25.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.OR5yt6WMBwQAcVIAmAiBMMltf4IV4Wh_0 2026-02-24T22:34:25.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13731909526120095174', time=2026-02-25T00:25:59.936161+0000, chain.objs.size()=1 2026-02-24T22:34:25.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.FTYLs1LNh_IUoo6kHiOZni2fXD6nUaG_0 2026-02-24T22:34:25.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4235377514702285552', time=2026-02-25T00:25:59.940749+0000, chain.objs.size()=1 2026-02-24T22:34:25.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.apC1iZAjvYmkM3e5VvEczJ8NuGuKagB_0 2026-02-24T22:34:25.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11761999760458424927', time=2026-02-25T00:25:59.980477+0000, chain.objs.size()=1 2026-02-24T22:34:25.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.8S9inF4pX949AY9ZvLi9kBNW53QNswZ_0 2026-02-24T22:34:25.540 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16508264134682482121', time=2026-02-25T00:26:00.097862+0000, chain.objs.size()=1 2026-02-24T22:34:25.541 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.0_MCMN-P63xSAISV01VMDy-3SmRHQ91_0 2026-02-24T22:34:25.541 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13473868594702375132', time=2026-02-25T00:26:00.108271+0000, chain.objs.size()=1 2026-02-24T22:34:25.541 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.n_KRA7uPWuYEZwDX7W07BKWPFplNuDx_0 2026-02-24T22:34:25.541 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.711504844307998821', time=2026-02-25T00:26:00.111509+0000, chain.objs.size()=1 2026-02-24T22:34:25.541 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.er9-IC3TmWzWPY2MNjfaxtnClmHf3c4_0 2026-02-24T22:34:25.541 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7110861704113938762', time=2026-02-25T00:26:00.123503+0000, chain.objs.size()=1 2026-02-24T22:34:25.541 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Eh4SutsfowZc6TagiBWS7mTQIRIdEbF_0 2026-02-24T22:34:25.541 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17841107874980549238', time=2026-02-25T00:26:00.131715+0000, chain.objs.size()=1 2026-02-24T22:34:25.541 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.KmatFc3kLKWFDYSEc7E_jyK9CfPSe4s_0 2026-02-24T22:34:25.541 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8110605813749097178', time=2026-02-25T00:26:00.232089+0000, chain.objs.size()=1 2026-02-24T22:34:25.541 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.7IZ2-9kiBQYo_PXC6pKMeC9M5pxU3um_0 2026-02-24T22:34:25.541 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12995429599437267199', time=2026-02-25T00:26:05.945644+0000, chain.objs.size()=1 2026-02-24T22:34:25.541 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1638__shadow_.NTeMYWRSWRUq-8LeL9lc_cQ2zWye-Th_0 2026-02-24T22:34:25.541 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9603660395794837365', time=2026-02-25T00:26:06.541959+0000, chain.objs.size()=1 2026-02-24T22:34:25.541 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1639__shadow_.w1doN0f4CGEXGK3pDiVgMDgPZMPTIvu_0 2026-02-24T22:34:25.541 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11551855425007110189', time=2026-02-25T00:26:08.270317+0000, chain.objs.size()=1 2026-02-24T22:34:25.541 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1642__shadow_.noOQe8mim3vCaPj7NKrww88QLielvm2_0 2026-02-24T22:34:25.541 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5716679812470758814', time=2026-02-25T00:26:10.610133+0000, chain.objs.size()=1 2026-02-24T22:34:25.541 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1646__shadow_.kpkIdr2zoawKEbJtz4qGCjuTbtpOhBh_0 2026-02-24T22:34:25.541 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8934376617290661253', time=2026-02-25T00:26:10.610718+0000, chain.objs.size()=1 2026-02-24T22:34:25.541 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1646__shadow_.yOK9hAPlqnx3C_gkj0h41w62nEcSYWH_0 2026-02-24T22:34:25.542 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7857760093952606326', time=2026-02-25T00:26:18.507645+0000, chain.objs.size()=1 2026-02-24T22:34:25.542 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1658__shadow_.XYlVZRHxPU8wV_mqWAvg54S8pQgdbsU_0 2026-02-24T22:34:25.542 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14110787917867515668', time=2026-02-25T00:26:19.451664+0000, chain.objs.size()=1 2026-02-24T22:34:25.542 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1659__shadow_.pogxCrlSQjP1ulIH_zWoYrg27PEnENZ_0 2026-02-24T22:34:25.542 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14552719497771924688', time=2026-02-25T00:26:19.454105+0000, chain.objs.size()=1 2026-02-24T22:34:25.542 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1659__shadow_.PZhqOwyN7nU1BNHXvZrm_E22Ru0cP3R_0 2026-02-24T22:34:25.542 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1598882218955817246', time=2026-02-25T00:26:19.457338+0000, chain.objs.size()=1 2026-02-24T22:34:25.542 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1659__shadow_.LIUJy7gbo1FlIOBeap4Ow7YmkEQB7TE_0 2026-02-24T22:34:25.542 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5682820711848837573', time=2026-02-25T00:26:23.470670+0000, chain.objs.size()=1 2026-02-24T22:34:25.542 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1666__shadow_.gE9un3U2ooaesyH_LdN4sSa6hezeshh_0 2026-02-24T22:34:25.542 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17235414151156660260', time=2026-02-25T00:26:24.639056+0000, chain.objs.size()=1 2026-02-24T22:34:25.542 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1668__shadow_.D5YL6a-N5NeLkLZj4xsVFqgpW5GgjNr_0 2026-02-24T22:34:25.542 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10762700533459628156', time=2026-02-25T00:26:28.072519+0000, chain.objs.size()=1 2026-02-24T22:34:25.542 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1673__shadow:6T8XMXXw-tMUw.pTczH-b.mHpWq1Qzv_.J5RcRDus0MqUboRUioXKZW_osoE_U_n_0 2026-02-24T22:34:25.542 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9308419544545289082', time=2026-02-25T00:26:36.501668+0000, chain.objs.size()=1 2026-02-24T22:34:25.542 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.551+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1685__shadow_.ACxKuQiCbsMGMYSrGp1g94_AHs9jO6Z_0 2026-02-24T22:34:25.542 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7156104650950373918', time=2026-02-25T00:26:44.784624+0000, chain.objs.size()=1 2026-02-24T22:34:25.542 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.SiFzj5viRxgr-kP3_QxJFrzXmFTV15W_0 2026-02-24T22:34:25.543 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2361959209335737481', time=2026-02-25T00:26:44.840643+0000, chain.objs.size()=1 2026-02-24T22:34:25.543 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.M7mA4yYnB-dC7IwGYoQ6IeMeX7O96Nh_0 2026-02-24T22:34:25.543 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.450361382478137112', time=2026-02-25T00:26:44.850010+0000, chain.objs.size()=1 2026-02-24T22:34:25.543 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ULENfb_J2SrOo0hPQesWWBg2TF5qG0s_0 2026-02-24T22:34:25.543 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17590566781802837485', time=2026-02-25T00:26:44.979567+0000, chain.objs.size()=1 2026-02-24T22:34:25.543 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.k7IfO4pKRUoicdyA3i6UQvbObMVW3vb_0 2026-02-24T22:34:25.543 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14568094675443650581', time=2026-02-25T00:26:44.980875+0000, chain.objs.size()=1 2026-02-24T22:34:25.543 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.VclSnNAzHXjT2Tttaci4-oMT7zvG870_0 2026-02-24T22:34:25.543 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9463461844479858768', time=2026-02-25T00:26:44.981302+0000, chain.objs.size()=1 2026-02-24T22:34:25.543 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.UFc9q8MGNkhR-CAZc_rxZlhxuMLJDe3_0 2026-02-24T22:34:25.543 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1624295236803059396', time=2026-02-25T00:26:45.016494+0000, chain.objs.size()=1 2026-02-24T22:34:25.543 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.YCWnG5oLSkyPDc50trpJRxjVvDHeGJj_0 2026-02-24T22:34:25.543 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7265107457475432176', time=2026-02-25T00:26:45.142123+0000, chain.objs.size()=1 2026-02-24T22:34:25.543 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.kvLqSZoZ8-igk8oORfDiKrqgSxqNR8s_0 2026-02-24T22:34:25.543 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10927476069957960126', time=2026-02-25T00:26:45.155088+0000, chain.objs.size()=1 2026-02-24T22:34:25.543 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.brGmiqhFVGxmV70IFTzaa2zrbL84wnR_0 2026-02-24T22:34:25.543 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8601063527230224805', time=2026-02-25T00:26:45.159239+0000, chain.objs.size()=1 2026-02-24T22:34:25.543 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.fAEHRBm4n1nuSoW-qGa3BBDz1fMNWTZ_0 2026-02-24T22:34:25.543 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7688694388424024710', time=2026-02-25T00:26:45.176329+0000, chain.objs.size()=1 2026-02-24T22:34:25.543 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.luzE01hinNMWA4BElCIHevyCpLL_LWb_0 2026-02-24T22:34:25.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6209772603497888627', time=2026-02-25T00:26:45.280714+0000, chain.objs.size()=1 2026-02-24T22:34:25.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.p6zlE0q5MnDyaIQ26aUxPq2vVUKt_7i_0 2026-02-24T22:34:25.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12857923189050327904', time=2026-02-25T00:26:45.289913+0000, chain.objs.size()=1 2026-02-24T22:34:25.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.toZ0Inl_ba1H39Ak_8ybuj5uRqwXDeD_0 2026-02-24T22:34:25.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.33137869037232132', time=2026-02-25T00:26:45.310484+0000, chain.objs.size()=1 2026-02-24T22:34:25.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Rm437RMo7xyj521Ui_8haiwOBum_dpC_0 2026-02-24T22:34:25.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7209700917131920377', time=2026-02-25T00:26:45.317828+0000, chain.objs.size()=1 2026-02-24T22:34:25.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ML3OMP6ZWKHxnga5tbd8EAfUcI0ivt7_0 2026-02-24T22:34:25.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17071256366786189141', time=2026-02-25T00:26:45.439166+0000, chain.objs.size()=1 2026-02-24T22:34:25.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.D852E3YADVSVEUgkBAAGs9RH1ecv_6B_0 2026-02-24T22:34:25.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3136970673636373061', time=2026-02-25T00:26:45.467635+0000, chain.objs.size()=1 2026-02-24T22:34:25.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.rVhKSZQ3SGS65l5LLPNrSUszcx_lrau_0 2026-02-24T22:34:25.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.829864978179973560', time=2026-02-25T00:26:45.474579+0000, chain.objs.size()=1 2026-02-24T22:34:25.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.oHEI9o_Xscdtz-eOtjqIFWQbIlYd39o_0 2026-02-24T22:34:25.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10169341045060814991', time=2026-02-25T00:26:45.579467+0000, chain.objs.size()=1 2026-02-24T22:34:25.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.2-n-sR-rPBjptUWLnELtuCHNO9HYiM6_0 2026-02-24T22:34:25.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8555687826210752483', time=2026-02-25T00:26:45.804248+0000, chain.objs.size()=1 2026-02-24T22:34:25.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.y6HRSWwl5FbVmHKh8EbAFOF6lGD1bP3_0 2026-02-24T22:34:25.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1155017918708596926', time=2026-02-25T00:26:45.924687+0000, chain.objs.size()=1 2026-02-24T22:34:25.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.PLQg9kuVjh65vHmhKNqhAA00ycHEi91_0 2026-02-24T22:34:25.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14863160606338783849', time=2026-02-25T00:26:45.941200+0000, chain.objs.size()=1 2026-02-24T22:34:25.544 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.pGzGeysTH_zz2VI3HCALzOwZ82DUJYZ_0 2026-02-24T22:34:25.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1847638118629337440', time=2026-02-25T00:26:45.968029+0000, chain.objs.size()=1 2026-02-24T22:34:25.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.M9O8-o_i7MzWHdofymbzsGUpjppKea0_0 2026-02-24T22:34:25.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11590637049790188715', time=2026-02-25T00:26:45.969253+0000, chain.objs.size()=1 2026-02-24T22:34:25.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.-ElH-19DObQGRZDI1rkWcXf9b8rhiRo_0 2026-02-24T22:34:25.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12736449527416391685', time=2026-02-25T00:26:45.992920+0000, chain.objs.size()=1 2026-02-24T22:34:25.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.tVl_H3mX_-y5-TyJ_OQG5MdzDfb6P9v_0 2026-02-24T22:34:25.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14570094670533153012', time=2026-02-25T00:26:45.994051+0000, chain.objs.size()=1 2026-02-24T22:34:25.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.5COQqHSchjPoArfp7npeBjU3RJiSbzI_0 2026-02-24T22:34:25.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8696509573567223480', time=2026-02-25T00:26:46.078512+0000, chain.objs.size()=1 2026-02-24T22:34:25.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.clQUtLVMfKqXOCJiTsiIrMpSIddt08P_0 2026-02-24T22:34:25.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9146770813804219490', time=2026-02-25T00:26:46.090081+0000, chain.objs.size()=1 2026-02-24T22:34:25.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.XgcGz6DY9HZiwhZbn_WA2LQZOOniz5A_0 2026-02-24T22:34:25.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8943716494048913462', time=2026-02-25T00:26:46.120698+0000, chain.objs.size()=1 2026-02-24T22:34:25.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.gnJw8OtzgIWsWw5NKYfu7Yft8jrAZBB_0 2026-02-24T22:34:25.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8820858422625361780', time=2026-02-25T00:26:46.122933+0000, chain.objs.size()=1 2026-02-24T22:34:25.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.WNKruPEfz5YC16iGO9mbATbNvxcppBC_0 2026-02-24T22:34:25.545 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.635828212544911259', time=2026-02-25T00:26:53.157258+0000, chain.objs.size()=1 2026-02-24T22:34:25.546 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.apdfXU2MauJDkSdhR9NCyBa3XTLmRSC_0 2026-02-24T22:34:25.546 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16601985033775472376', time=2026-02-25T00:26:53.190449+0000, chain.objs.size()=1 2026-02-24T22:34:25.546 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.XP28-XIQ48UVIJ5eYWmwxnyfpmrSNu0_0 2026-02-24T22:34:25.546 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9055859053611465236', time=2026-02-25T00:26:53.206860+0000, chain.objs.size()=1 2026-02-24T22:34:25.546 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.xJlSAjvWKTwvdAUi17Xad7nUmqiOdaL_0 2026-02-24T22:34:25.546 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14486084998271765438', time=2026-02-25T00:26:53.245172+0000, chain.objs.size()=1 2026-02-24T22:34:25.546 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Hia88GNHMThCPpECQXuXPNrTxafEyPN_0 2026-02-24T22:34:25.546 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1242710629409613833', time=2026-02-25T00:26:53.322986+0000, chain.objs.size()=1 2026-02-24T22:34:25.546 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.GGgdiqN_Va46iDZvEErDTrhwXM3_Zn__0 2026-02-24T22:34:25.546 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15326741370241262467', time=2026-02-25T00:26:53.323991+0000, chain.objs.size()=1 2026-02-24T22:34:25.546 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.gWRrpsVRehwxlA1X71ES3rkvrt8NoiG_0 2026-02-24T22:34:25.546 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2551142142906615802', time=2026-02-25T00:26:53.399660+0000, chain.objs.size()=1 2026-02-24T22:34:25.546 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.555+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.8_VKWQF2RiXh9txb9LtedRPyAPTd1JE_0 2026-02-24T22:34:25.547 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/38732 2026-02-24T22:34:25.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=38, truncated=0, next_marker='' 2026-02-24T22:34:25.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3667475652030124253', time=2026-02-25T00:26:53.518882+0000, chain.objs.size()=1 2026-02-24T22:34:25.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.4epOobLZmvfNeWu2JXhfWgHJ8Fa3Ruv_0 2026-02-24T22:34:25.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16889173778388498460', time=2026-02-25T00:26:53.548123+0000, chain.objs.size()=1 2026-02-24T22:34:25.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.H8RvtBbS11Ug6iehoJyMFTWBxjOSKT3_0 2026-02-24T22:34:25.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.894250056855498982', time=2026-02-25T00:26:53.560074+0000, chain.objs.size()=1 2026-02-24T22:34:25.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.N595B9I5LexdPJn92GRVse2Ss8NfUKN_0 2026-02-24T22:34:25.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5898434799674310733', time=2026-02-25T00:26:53.597316+0000, chain.objs.size()=1 2026-02-24T22:34:25.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.IUem7lrBScpIqIEVXdffLqUKVCvkMJF_0 2026-02-24T22:34:25.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15505336538163088038', time=2026-02-25T00:26:53.603357+0000, chain.objs.size()=1 2026-02-24T22:34:25.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ylUn_PepcZaW1412DS1IUiSzd1poV1T_0 2026-02-24T22:34:25.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4379360837995870257', time=2026-02-25T00:26:53.604602+0000, chain.objs.size()=1 2026-02-24T22:34:25.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.vAqNCNUrVtORdj0cskFABpQPSCuvDnb_0 2026-02-24T22:34:25.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.937509048283315164', time=2026-02-25T00:26:53.608622+0000, chain.objs.size()=1 2026-02-24T22:34:25.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.1eK_icS-d5creB1WyTAhK07mHGnzYEr_0 2026-02-24T22:34:25.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14180788250662875268', time=2026-02-25T00:26:53.754740+0000, chain.objs.size()=1 2026-02-24T22:34:25.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ecwbPIgHStwPIY64iYqweWlWbLb4m3P_0 2026-02-24T22:34:25.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6962866450538813709', time=2026-02-25T00:26:53.776874+0000, chain.objs.size()=1 2026-02-24T22:34:25.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.UbHoixHf_8YhHfqPdxDVvzDGSoxrBne_0 2026-02-24T22:34:25.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15730464497528516373', time=2026-02-25T00:26:53.952476+0000, chain.objs.size()=1 2026-02-24T22:34:25.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.vSHv9w77wP_Va1yHAaMN2xUDX1mNr5n_0 2026-02-24T22:34:25.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6338967510668537435', time=2026-02-25T00:26:53.959843+0000, chain.objs.size()=1 2026-02-24T22:34:25.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.isVvamdL-vtraURX7N1MiRUc9WcM0Eg_0 2026-02-24T22:34:25.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17423642134226048123', time=2026-02-25T00:26:53.981656+0000, chain.objs.size()=1 2026-02-24T22:34:25.549 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.BwV8GNDobBILJ_bO7PVv6Va1APkMHNV_0 2026-02-24T22:34:25.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7588220173788550256', time=2026-02-25T00:26:53.994832+0000, chain.objs.size()=1 2026-02-24T22:34:25.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.SKXAzmQ07UennWaowi8jGhzEw5cgrFM_0 2026-02-24T22:34:25.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3975269633425939445', time=2026-02-25T00:26:54.017716+0000, chain.objs.size()=1 2026-02-24T22:34:25.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.MbgEqXVIUY5PWQ3Jvhl2MvTd4iDiezv_0 2026-02-24T22:34:25.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4310782341628021004', time=2026-02-25T00:26:54.117437+0000, chain.objs.size()=1 2026-02-24T22:34:25.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.1SMac08yhxwulj5Isb91xokt9kUys6F_0 2026-02-24T22:34:25.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18145052698039176476', time=2026-02-25T00:26:54.121161+0000, chain.objs.size()=1 2026-02-24T22:34:25.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.HSL52HCZoxc3_OQ7FzHmrMJLWplifKV_0 2026-02-24T22:34:25.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.499676456181327465', time=2026-02-25T00:26:54.282836+0000, chain.objs.size()=1 2026-02-24T22:34:25.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.wiuaQyfPToCLfO7zqsJK_N6HiX-jmo2_0 2026-02-24T22:34:25.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17257957376100871877', time=2026-02-25T00:26:54.335378+0000, chain.objs.size()=1 2026-02-24T22:34:25.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Dt5KQM94Byikw5x90TgMtflNbwfCllq_0 2026-02-24T22:34:25.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17593122863544692872', time=2026-02-25T00:26:54.340777+0000, chain.objs.size()=1 2026-02-24T22:34:25.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_._Dbe-mLtsr1vhktO_8v617A3dLyYPuR_0 2026-02-24T22:34:25.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16234971146089024388', time=2026-02-25T00:26:54.387555+0000, chain.objs.size()=1 2026-02-24T22:34:25.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.P5qPt8gChy_Afnjdpx4DOBQZmhYLl-V_0 2026-02-24T22:34:25.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1916896908780740263', time=2026-02-25T00:26:54.451179+0000, chain.objs.size()=1 2026-02-24T22:34:25.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.XG8EJoUqTMvnOY8z90Q02AnNEcY668P_0 2026-02-24T22:34:25.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7151673457712876215', time=2026-02-25T00:26:54.489122+0000, chain.objs.size()=1 2026-02-24T22:34:25.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.bG5fkkAb4qon17GpzHcQKvgbrAprJOh_0 2026-02-24T22:34:25.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4300892683278762653', time=2026-02-25T00:26:54.509267+0000, chain.objs.size()=1 2026-02-24T22:34:25.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.559+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.INLeDtT4wZgec2q9Vet7U-MJASR_ZtO_0 2026-02-24T22:34:25.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17870387310626136909', time=2026-02-25T00:26:54.526510+0000, chain.objs.size()=1 2026-02-24T22:34:25.550 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.nnxy_3TyZ3leBeZQIXj743sbJw1ipWH_0 2026-02-24T22:34:25.551 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2002461637660240039', time=2026-02-25T00:26:54.530369+0000, chain.objs.size()=1 2026-02-24T22:34:25.551 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.LXW0sXSLLli8t8xOrYOHZL7Av_8Ts6O_0 2026-02-24T22:34:25.551 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4835611433374164823', time=2026-02-25T00:26:54.533015+0000, chain.objs.size()=1 2026-02-24T22:34:25.551 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.vbtvCq07Ots62eTqTkRldMp0pQNjL2m_0 2026-02-24T22:34:25.551 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2705967251087853621', time=2026-02-25T00:27:32.477023+0000, chain.objs.size()=1 2026-02-24T22:34:25.551 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1755__shadow_.geUHdv0gGAH7Eet6fwHCbyK6lZR9No0_0 2026-02-24T22:34:25.551 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13635686490496628892', time=2026-02-25T00:28:25.127590+0000, chain.objs.size()=1 2026-02-24T22:34:25.551 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1831__shadow_.wo0ZYrXwcPV2ECdP6DIx9vDGl52szWf_0 2026-02-24T22:34:25.551 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17820693916063125646', time=2026-02-25T00:28:32.540669+0000, chain.objs.size()=1 2026-02-24T22:34:25.551 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1849__shadow_.0R-1WvQOclKTYyLEcEEDi495VqPwoR7_0 2026-02-24T22:34:25.551 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7671758536273001567', time=2026-02-25T00:28:32.541553+0000, chain.objs.size()=1 2026-02-24T22:34:25.551 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1849__shadow_.XXDkscuSmeBMKfWmk8XIrCuNSSxZgPl_0 2026-02-24T22:34:25.551 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13942912277632570727', time=2026-02-25T00:28:41.459190+0000, chain.objs.size()=1 2026-02-24T22:34:25.551 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1866__shadow_.g-aEcsLY9bhugudSKbljczizHarh7IO_0 2026-02-24T22:34:25.551 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1132959260061617941', time=2026-02-25T00:29:13.915585+0000, chain.objs.size()=1 2026-02-24T22:34:25.551 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1890__shadow_.pLku4X47dY3B-HCgQGLn5-79yURPQjx_0 2026-02-24T22:34:25.551 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11094451314792939427', time=2026-02-25T00:30:22.762684+0000, chain.objs.size()=1 2026-02-24T22:34:25.551 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:A9p96a1bPhRBRRa7pPPyi8uo1aFut1A_.h67YjDSfSjHWD2gnfk89eqOth7Heyn3_0 2026-02-24T22:34:25.551 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3865355715473655077', time=2026-02-25T00:30:27.143450+0000, chain.objs.size()=1 2026-02-24T22:34:25.551 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1942__shadow_.YOO30epkmr5HScWGjjOSZIx98s4-kTY_0 2026-02-24T22:34:25.552 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8270368487184681900', time=2026-02-25T00:30:28.073544+0000, chain.objs.size()=1 2026-02-24T22:34:25.552 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1943__shadow_.HmrbHQAR97gS6cl_X-B21OuaHZjzucw_0 2026-02-24T22:34:25.552 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2859577097352673328', time=2026-02-25T00:30:37.460246+0000, chain.objs.size()=1 2026-02-24T22:34:25.552 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:Z64DHASHH.rEQSOC9rTOgXQEzU2sqz1_.LF7Xb6cx3rk9tE1bmmkAKzLpwjQ2c5f_0 2026-02-24T22:34:25.552 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12323517928923604640', time=2026-02-25T00:30:37.967049+0000, chain.objs.size()=1 2026-02-24T22:34:25.552 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:Yuz5933jTVQ5VxO4pscsMbA1hZwoApn_.SXGIKcB3bQervYsoEZwP6E9aH3Qo4BN_0 2026-02-24T22:34:25.552 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17762832887929205790', time=2026-02-25T00:33:57.693617+0000, chain.objs.size()=1 2026-02-24T22:34:25.552 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1980__shadow_.ras6Q7-B2-op2BTLkzCJ3i_xPV02p_M_0 2026-02-24T22:34:25.552 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.563+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.554 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2026-02-24T22:34:25.556 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.556 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.557 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/39588' 2026-02-24T22:34:25.557 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='_Wt1Ru69B-lzXu5V57DiCvWsM7R-nYyD', time=2026-02-25T00:15:34.709165+0000, chain.objs.size()=1 2026-02-24T22:34:25.557 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516__shadow_.D8W143s3I_9bUC-zDmO5dBf-ExUdgqz_0 2026-02-24T22:34:25.557 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3297900582288245003', time=2026-02-25T00:16:33.514235+0000, chain.objs.size()=1 2026-02-24T22:34:25.557 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521__shadow_.dWsI1sIvFNjWVYiIbnaA_ILpL0Fizu6_0 2026-02-24T22:34:25.557 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1387109872000967378', time=2026-02-25T00:17:41.423657+0000, chain.objs.size()=1 2026-02-24T22:34:25.557 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526__shadow_.xe7XfskchX7mlGxyJ68PVSKM-4YwV-J_0 2026-02-24T22:34:25.557 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10494030720349182226', time=2026-02-25T00:17:42.529493+0000, chain.objs.size()=1 2026-02-24T22:34:25.557 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527__shadow_.Ni0vNeuuIJ2_k4iTfDG_nDCe2Wh9RfR_0 2026-02-24T22:34:25.557 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4287600581046564402', time=2026-02-25T00:17:42.530960+0000, chain.objs.size()=1 2026-02-24T22:34:25.557 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527__shadow_.TViFO_gbkMapH5EIg8-Dlk9Epm42xNs_0 2026-02-24T22:34:25.557 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7262191912075233672', time=2026-02-25T00:17:42.534597+0000, chain.objs.size()=1 2026-02-24T22:34:25.557 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527__shadow_.G0XWCHRzx_fn_lM4q0yLAnbbI4kmgos_0 2026-02-24T22:34:25.557 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2472649489504340374', time=2026-02-25T00:18:16.121598+0000, chain.objs.size()=1 2026-02-24T22:34:25.557 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529__shadow_.izc8lgGPXupVhK6u4cAfNJXYhd_lu___0 2026-02-24T22:34:25.557 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2780610887887970993', time=2026-02-25T00:18:16.123502+0000, chain.objs.size()=1 2026-02-24T22:34:25.557 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529__shadow_.xjB3hj50v2B5LDhS6lyU9RQ5ng-zjyv_0 2026-02-24T22:34:25.557 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7849204890237408946', time=2026-02-25T00:18:49.768912+0000, chain.objs.size()=1 2026-02-24T22:34:25.558 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531__shadow_.j4pleO3jZaoCK9VDyUfFuPWASS_dGUV_0 2026-02-24T22:34:25.558 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18172590487194758522', time=2026-02-25T00:18:49.772318+0000, chain.objs.size()=1 2026-02-24T22:34:25.558 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531__shadow_.kV710WOt88CEGC-af0OoF89NWWh5evY_0 2026-02-24T22:34:25.558 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8353076479963922189', time=2026-02-25T00:18:51.086864+0000, chain.objs.size()=1 2026-02-24T22:34:25.558 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532__shadow_.WtRtBE_WPGtdMTylQEu1dqR_K9XJTuF_0 2026-02-24T22:34:25.558 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9869250648157986916', time=2026-02-25T00:19:25.017618+0000, chain.objs.size()=1 2026-02-24T22:34:25.558 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.567+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534__shadow_.F9WmGTxa8rRspOh83l1Uwh9GF8Vq86U_0 2026-02-24T22:34:25.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1913041331564681486', time=2026-02-25T00:19:26.131259+0000, chain.objs.size()=1 2026-02-24T22:34:25.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535__shadow_.w13q1ZvebCMP8GL4A31-akd-XP5t1mL_0 2026-02-24T22:34:25.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17250618048487238973', time=2026-02-25T00:20:30.724253+0000, chain.objs.size()=1 2026-02-24T22:34:25.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537__shadow_.uX_VEMxs6_UrV7IV1_6HcrazoWdYt7X_0 2026-02-24T22:34:25.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1420609878334606612', time=2026-02-25T00:20:30.726000+0000, chain.objs.size()=1 2026-02-24T22:34:25.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537__shadow_.fKLTf_9GmX4ThurNZp2tCFAtc4_oQbS_0 2026-02-24T22:34:25.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4119682231322593391', time=2026-02-25T00:20:34.033233+0000, chain.objs.size()=1 2026-02-24T22:34:25.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539__shadow_.C6XLFpZfzNp5pD58pjJvM4ijk2lj9Dy_0 2026-02-24T22:34:25.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.754760182837544450', time=2026-02-25T00:21:06.309855+0000, chain.objs.size()=1 2026-02-24T22:34:25.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540__shadow_.AhEJtrdmWggCH7pIISOxxFXPj2560MK_0 2026-02-24T22:34:25.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11214852068020485998', time=2026-02-25T00:23:14.350027+0000, chain.objs.size()=1 2026-02-24T22:34:25.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518__shadow_.EAY0KsZuh-qy4fgiNXukKfg-ULpibZx_0 2026-02-24T22:34:25.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14077982741747996361', time=2026-02-25T00:23:53.102151+0000, chain.objs.size()=1 2026-02-24T22:34:25.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1544__shadow_.mGuRSUzLTbxY7HHnry36nRcklOgfPSI_0 2026-02-24T22:34:25.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10264867160952220577', time=2026-02-25T00:23:55.742875+0000, chain.objs.size()=1 2026-02-24T22:34:25.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1549__shadow_.yCMfjxE_QVyscbuK3PsHgOVlAS4u6bS_0 2026-02-24T22:34:25.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3046154949663876967', time=2026-02-25T00:25:36.599373+0000, chain.objs.size()=1 2026-02-24T22:34:25.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1602__shadow_.Z8bPqCZvRf3CP3r_l1qrz-P3yeH4nMJ_0 2026-02-24T22:34:25.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15508290747161425548', time=2026-02-25T00:25:42.667459+0000, chain.objs.size()=1 2026-02-24T22:34:25.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1610__shadow_.y7kdM1wmlvfesXHv5fmpY18HQQN7x8x_0 2026-02-24T22:34:25.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8217328389260505611', time=2026-02-25T00:25:45.566605+0000, chain.objs.size()=1 2026-02-24T22:34:25.559 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1615__shadow_.VqaEH8QRy8gyoFhoW-8j0scZFO75lBj_0 2026-02-24T22:34:25.560 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14582661020692208589', time=2026-02-25T00:25:46.124230+0000, chain.objs.size()=1 2026-02-24T22:34:25.560 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1616__shadow_.v-IbbqT5vxtq0mt8oy1SwlM_vTMaGuQ_0 2026-02-24T22:34:25.560 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4584716099259178036', time=2026-02-25T00:25:47.942607+0000, chain.objs.size()=1 2026-02-24T22:34:25.560 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1619__shadow_.uDX39OccVYWZToqVgJZpBTbgcXVpm8F_0 2026-02-24T22:34:25.560 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5733941017382929622', time=2026-02-25T00:25:49.078737+0000, chain.objs.size()=1 2026-02-24T22:34:25.560 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1621__shadow_.RQTiPwLQv3tXMn6USiqPHPgl7TZb0Un_0 2026-02-24T22:34:25.560 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10899763983223094435', time=2026-02-25T00:25:51.942556+0000, chain.objs.size()=1 2026-02-24T22:34:25.560 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1626__shadow_.O7y2fJCqTfDibM7t9YvvS4tkFxghao7_0 2026-02-24T22:34:25.560 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12951207726788754018', time=2026-02-25T00:25:59.106836+0000, chain.objs.size()=1 2026-02-24T22:34:25.560 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.TzA3RioudfzwG7Ha6kpWSFbZyY1ifxt_0 2026-02-24T22:34:25.560 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3866723892307346313', time=2026-02-25T00:25:59.115553+0000, chain.objs.size()=1 2026-02-24T22:34:25.560 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.t-bry6bOrfyfcqs3R9yuEIH23a_uyiO_0 2026-02-24T22:34:25.560 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4251743671490618690', time=2026-02-25T00:25:59.118191+0000, chain.objs.size()=1 2026-02-24T22:34:25.560 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.wzI4LKRItMYLAUq-hAfXDppJXKMA36E_0 2026-02-24T22:34:25.560 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.782171404758928035', time=2026-02-25T00:25:59.137143+0000, chain.objs.size()=1 2026-02-24T22:34:25.561 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ddnQDzKQyn-c9lqGvXSx962x13DKQzW_0 2026-02-24T22:34:25.561 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13582680945854741373', time=2026-02-25T00:25:59.162412+0000, chain.objs.size()=1 2026-02-24T22:34:25.561 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.CUhi9vLXHR1Y9SYM8DUACWYgpLDYo4G_0 2026-02-24T22:34:25.561 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17910918149690817432', time=2026-02-25T00:25:59.246087+0000, chain.objs.size()=1 2026-02-24T22:34:25.561 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.z0b_gi-qaOctFOxnlUqSOTH1hjCJnlh_0 2026-02-24T22:34:25.561 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15943597367625442985', time=2026-02-25T00:25:59.293576+0000, chain.objs.size()=1 2026-02-24T22:34:25.561 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Mqo115YZLAfXzKEhJbMFDA_keYqGlaK_0 2026-02-24T22:34:25.561 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4318317856293038176', time=2026-02-25T00:25:59.437545+0000, chain.objs.size()=1 2026-02-24T22:34:25.561 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.IQptIAoDtvpNRDQDvbcFW3c6a1OPq-F_0 2026-02-24T22:34:25.561 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8505343912957985301', time=2026-02-25T00:25:59.443859+0000, chain.objs.size()=1 2026-02-24T22:34:25.561 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.51y3h8mXpWkk5Io8MA5Qf2ErmJ1TPm9_0 2026-02-24T22:34:25.561 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4753420215103968066', time=2026-02-25T00:25:59.478021+0000, chain.objs.size()=1 2026-02-24T22:34:25.561 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.eg2bLPWdXb34BPRrLNnXHuxUXi_YEeM_0 2026-02-24T22:34:25.561 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7296527918860271694', time=2026-02-25T00:25:59.485300+0000, chain.objs.size()=1 2026-02-24T22:34:25.561 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.6tO-6bQ2UxHrBqNIRaYj-M-wt7Zx1PS_0 2026-02-24T22:34:25.561 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9753223140015714296', time=2026-02-25T00:25:59.558687+0000, chain.objs.size()=1 2026-02-24T22:34:25.561 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.f3RA2_1drBOhHjWlPj_iJwnnpX4rcWj_0 2026-02-24T22:34:25.561 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4842524587375365812', time=2026-02-25T00:25:59.559791+0000, chain.objs.size()=1 2026-02-24T22:34:25.561 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.zTkqch2yrImEaIfhyVr7QZ0ZGgXmhqQ_0 2026-02-24T22:34:25.561 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15086230959620057122', time=2026-02-25T00:25:59.562366+0000, chain.objs.size()=1 2026-02-24T22:34:25.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.q5ITGJ4zY8YQh0ntPzTDiQeOs8tgK4f_0 2026-02-24T22:34:25.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15255013349708841742', time=2026-02-25T00:25:59.567486+0000, chain.objs.size()=1 2026-02-24T22:34:25.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.0TbGAKUvNwTs2Q4tMUSVQjarTS4qlHx_0 2026-02-24T22:34:25.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14542810919943675407', time=2026-02-25T00:25:59.601472+0000, chain.objs.size()=1 2026-02-24T22:34:25.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.AtJfBwyIqur5rJ3RYnDNh3J62gwbxO1_0 2026-02-24T22:34:25.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14940390067762030658', time=2026-02-25T00:25:59.635604+0000, chain.objs.size()=1 2026-02-24T22:34:25.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.yENFiuYudSgbTt-AwL8Y2rrh_DZhclA_0 2026-02-24T22:34:25.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15027861993941474635', time=2026-02-25T00:25:59.651704+0000, chain.objs.size()=1 2026-02-24T22:34:25.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.P2-kFM122juQp1mr4ZQFUh69KpzJ-E__0 2026-02-24T22:34:25.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1789717832367917502', time=2026-02-25T00:25:59.753405+0000, chain.objs.size()=1 2026-02-24T22:34:25.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.M1hhwxwEuFhOOL0wuAd78O1D3eZ0VGd_0 2026-02-24T22:34:25.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9955294847316177967', time=2026-02-25T00:25:59.761490+0000, chain.objs.size()=1 2026-02-24T22:34:25.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.43aFs0gbe-EfYFwUg-Xs73GRprbJvve_0 2026-02-24T22:34:25.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17200410024768136725', time=2026-02-25T00:25:59.913263+0000, chain.objs.size()=1 2026-02-24T22:34:25.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.4y8UrGGUMv_ew5dFLcrvzG3imq_spfv_0 2026-02-24T22:34:25.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.479910516263132500', time=2026-02-25T00:25:59.929875+0000, chain.objs.size()=1 2026-02-24T22:34:25.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.tU0Mob328qkyECLADWQNIvRHTB3tpAk_0 2026-02-24T22:34:25.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14984413477008645779', time=2026-02-25T00:25:59.931060+0000, chain.objs.size()=1 2026-02-24T22:34:25.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.571+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.XQiI6rCt-0GZoVm9FhuEQpJf8wi0VDg_0 2026-02-24T22:34:25.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8308407540671086662', time=2026-02-25T00:25:59.935618+0000, chain.objs.size()=1 2026-02-24T22:34:25.562 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.JgSvu_d8maNJVBcBUjaljoX3xmdQ_ZR_0 2026-02-24T22:34:25.563 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12312920035662398736', time=2026-02-25T00:25:59.939260+0000, chain.objs.size()=1 2026-02-24T22:34:25.563 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.uXHktpLR8-uYtqYMnIOqeXxZSNywnSh_0 2026-02-24T22:34:25.563 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16811734242028437832', time=2026-02-25T00:25:59.952481+0000, chain.objs.size()=1 2026-02-24T22:34:25.563 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.zziUDQmai_SS51q808gmpa4apwrQFDS_0 2026-02-24T22:34:25.563 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.171973013117025606', time=2026-02-25T00:25:59.960727+0000, chain.objs.size()=1 2026-02-24T22:34:25.563 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.zW43VDON6ij2WBUpdwBRnPzDC0B_36M_0 2026-02-24T22:34:25.563 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12458356626200123896', time=2026-02-25T00:25:59.969082+0000, chain.objs.size()=1 2026-02-24T22:34:25.563 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Q-2fFiqEIzE1ULJuc7F8KSMKOn6jT2X_0 2026-02-24T22:34:25.563 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17115652485473941483', time=2026-02-25T00:25:59.969818+0000, chain.objs.size()=1 2026-02-24T22:34:25.563 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.tcPfgAEFuyooD6LvNDXC0tfWLPhfXlF_0 2026-02-24T22:34:25.563 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3955193918714882877', time=2026-02-25T00:25:59.973153+0000, chain.objs.size()=1 2026-02-24T22:34:25.563 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.XOZOYs_wBe_ZWEyqrIbqawrKVf3yAOH_0 2026-02-24T22:34:25.563 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1511874906646897078', time=2026-02-25T00:26:00.053806+0000, chain.objs.size()=1 2026-02-24T22:34:25.563 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.dpXoQtUjtAjy9EdvsF8SiMF4mQR63RA_0 2026-02-24T22:34:25.563 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5524491209348154132', time=2026-02-25T00:26:00.062616+0000, chain.objs.size()=1 2026-02-24T22:34:25.563 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.sH5hI9efCTgS6gqAvWI38aZHDqMhs08_0 2026-02-24T22:34:25.563 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9385211280951167289', time=2026-02-25T00:26:00.098454+0000, chain.objs.size()=1 2026-02-24T22:34:25.563 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.kKoun9roRWltqbOg2UoG5P9AClRaWnz_0 2026-02-24T22:34:25.563 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14093226909007069730', time=2026-02-25T00:26:00.100694+0000, chain.objs.size()=1 2026-02-24T22:34:25.563 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.n1qp7oTp4FW_CDU94Fre_kEbDm_jb6M_0 2026-02-24T22:34:25.564 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13920695552214748677', time=2026-02-25T00:26:00.108163+0000, chain.objs.size()=1 2026-02-24T22:34:25.564 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.X0I9UYA8zRvAeXjlaQMMPxREuDwS3gm_0 2026-02-24T22:34:25.564 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7235794971797872835', time=2026-02-25T00:26:00.206468+0000, chain.objs.size()=1 2026-02-24T22:34:25.564 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.nVfA38AMYzAIhCL4k_k-jJvnjtrnHQ-_0 2026-02-24T22:34:25.564 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3070682422076769166', time=2026-02-25T00:26:00.257025+0000, chain.objs.size()=1 2026-02-24T22:34:25.564 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Mb2KXKQ_6NHyFNf9k_1cWuYc3JtjkSN_0 2026-02-24T22:34:25.564 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7934679621667941204', time=2026-02-25T00:26:00.268175+0000, chain.objs.size()=1 2026-02-24T22:34:25.564 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.WgF9I6X5NpQCxYm5zxMfFhPa8n5m4mG_0 2026-02-24T22:34:25.564 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6788508807821132942', time=2026-02-25T00:26:09.398281+0000, chain.objs.size()=1 2026-02-24T22:34:25.564 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1644__shadow_.H5a1byLPKMAsBfD9GhnzFg6sZcrsjs__0 2026-02-24T22:34:25.564 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10368175616620742955', time=2026-02-25T00:26:24.638572+0000, chain.objs.size()=1 2026-02-24T22:34:25.564 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1668__shadow_.ZUWc3niyRAjLb4dY2jlCvi_MZiLv23B_0 2026-02-24T22:34:25.564 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.493258603832123655', time=2026-02-25T00:26:28.072812+0000, chain.objs.size()=1 2026-02-24T22:34:25.564 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1673__shadow:Zt9rfsX0UxgyNLTkGlCta6eagl1PqG9_.be4FPgWecjQnxCoLhyy-b-SDQtthD5j_0 2026-02-24T22:34:25.564 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18431522042885399860', time=2026-02-25T00:26:35.755095+0000, chain.objs.size()=1 2026-02-24T22:34:25.564 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:LYW0dfFC-oCKZ5b-PO31m7FYbqvrGDd_.m_UFue4M58Fqt8uMXKwA2PBOb1mLJ4g_0 2026-02-24T22:34:25.564 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18431522042885399860', time=2026-02-25T00:26:35.773625+0000, chain.objs.size()=1 2026-02-24T22:34:25.564 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:LYW0dfFC-oCKZ5b-PO31m7FYbqvrGDd_.m_UFue4M58Fqt8uMXKwA2PBOb1mLJ4g_0 2026-02-24T22:34:25.564 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14409070877456460010', time=2026-02-25T00:26:35.790063+0000, chain.objs.size()=1 2026-02-24T22:34:25.564 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:f9qDcJ1aQlqgN3lJekSN4jyh0oau5j8_.dGu7xsNU5YvSgptw60C3rqc9N9OERP0_0 2026-02-24T22:34:25.564 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14409070877456460010', time=2026-02-25T00:26:35.791570+0000, chain.objs.size()=1 2026-02-24T22:34:25.564 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:f9qDcJ1aQlqgN3lJekSN4jyh0oau5j8_.dGu7xsNU5YvSgptw60C3rqc9N9OERP0_0 2026-02-24T22:34:25.564 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14409070877456460010', time=2026-02-25T00:26:35.795798+0000, chain.objs.size()=1 2026-02-24T22:34:25.565 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:f9qDcJ1aQlqgN3lJekSN4jyh0oau5j8_.dGu7xsNU5YvSgptw60C3rqc9N9OERP0_0 2026-02-24T22:34:25.565 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14409070877456460010', time=2026-02-25T00:26:35.801590+0000, chain.objs.size()=1 2026-02-24T22:34:25.565 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:f9qDcJ1aQlqgN3lJekSN4jyh0oau5j8_.dGu7xsNU5YvSgptw60C3rqc9N9OERP0_0 2026-02-24T22:34:25.565 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18431522042885399860', time=2026-02-25T00:26:35.802960+0000, chain.objs.size()=1 2026-02-24T22:34:25.565 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:LYW0dfFC-oCKZ5b-PO31m7FYbqvrGDd_.m_UFue4M58Fqt8uMXKwA2PBOb1mLJ4g_0 2026-02-24T22:34:25.565 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14409070877456460010', time=2026-02-25T00:26:35.805688+0000, chain.objs.size()=1 2026-02-24T22:34:25.565 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:f9qDcJ1aQlqgN3lJekSN4jyh0oau5j8_.dGu7xsNU5YvSgptw60C3rqc9N9OERP0_0 2026-02-24T22:34:25.565 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14409070877456460010', time=2026-02-25T00:26:35.807617+0000, chain.objs.size()=1 2026-02-24T22:34:25.565 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:f9qDcJ1aQlqgN3lJekSN4jyh0oau5j8_.dGu7xsNU5YvSgptw60C3rqc9N9OERP0_0 2026-02-24T22:34:25.565 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14409070877456460010', time=2026-02-25T00:26:35.814824+0000, chain.objs.size()=1 2026-02-24T22:34:25.565 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:f9qDcJ1aQlqgN3lJekSN4jyh0oau5j8_.dGu7xsNU5YvSgptw60C3rqc9N9OERP0_0 2026-02-24T22:34:25.565 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14409070877456460010', time=2026-02-25T00:26:35.822627+0000, chain.objs.size()=1 2026-02-24T22:34:25.565 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:f9qDcJ1aQlqgN3lJekSN4jyh0oau5j8_.dGu7xsNU5YvSgptw60C3rqc9N9OERP0_0 2026-02-24T22:34:25.565 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14409070877456460010', time=2026-02-25T00:26:35.824410+0000, chain.objs.size()=1 2026-02-24T22:34:25.565 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:f9qDcJ1aQlqgN3lJekSN4jyh0oau5j8_.dGu7xsNU5YvSgptw60C3rqc9N9OERP0_0 2026-02-24T22:34:25.565 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14409070877456460010', time=2026-02-25T00:26:35.825718+0000, chain.objs.size()=1 2026-02-24T22:34:25.565 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:f9qDcJ1aQlqgN3lJekSN4jyh0oau5j8_.dGu7xsNU5YvSgptw60C3rqc9N9OERP0_0 2026-02-24T22:34:25.565 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14409070877456460010', time=2026-02-25T00:26:35.830616+0000, chain.objs.size()=1 2026-02-24T22:34:25.565 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:f9qDcJ1aQlqgN3lJekSN4jyh0oau5j8_.dGu7xsNU5YvSgptw60C3rqc9N9OERP0_0 2026-02-24T22:34:25.566 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14409070877456460010', time=2026-02-25T00:26:35.835946+0000, chain.objs.size()=1 2026-02-24T22:34:25.566 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:f9qDcJ1aQlqgN3lJekSN4jyh0oau5j8_.dGu7xsNU5YvSgptw60C3rqc9N9OERP0_0 2026-02-24T22:34:25.566 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14409070877456460010', time=2026-02-25T00:26:35.840269+0000, chain.objs.size()=1 2026-02-24T22:34:25.566 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:f9qDcJ1aQlqgN3lJekSN4jyh0oau5j8_.dGu7xsNU5YvSgptw60C3rqc9N9OERP0_0 2026-02-24T22:34:25.566 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18431522042885399860', time=2026-02-25T00:26:35.840789+0000, chain.objs.size()=1 2026-02-24T22:34:25.566 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.575+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:LYW0dfFC-oCKZ5b-PO31m7FYbqvrGDd_.m_UFue4M58Fqt8uMXKwA2PBOb1mLJ4g_0 2026-02-24T22:34:25.566 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14409070877456460010', time=2026-02-25T00:26:35.841859+0000, chain.objs.size()=1 2026-02-24T22:34:25.566 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:f9qDcJ1aQlqgN3lJekSN4jyh0oau5j8_.dGu7xsNU5YvSgptw60C3rqc9N9OERP0_0 2026-02-24T22:34:25.566 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14409070877456460010', time=2026-02-25T00:26:35.848337+0000, chain.objs.size()=1 2026-02-24T22:34:25.566 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:f9qDcJ1aQlqgN3lJekSN4jyh0oau5j8_.dGu7xsNU5YvSgptw60C3rqc9N9OERP0_0 2026-02-24T22:34:25.567 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14409070877456460010', time=2026-02-25T00:26:35.851975+0000, chain.objs.size()=1 2026-02-24T22:34:25.567 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:f9qDcJ1aQlqgN3lJekSN4jyh0oau5j8_.dGu7xsNU5YvSgptw60C3rqc9N9OERP0_0 2026-02-24T22:34:25.567 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14409070877456460010', time=2026-02-25T00:26:35.854533+0000, chain.objs.size()=1 2026-02-24T22:34:25.567 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:f9qDcJ1aQlqgN3lJekSN4jyh0oau5j8_.dGu7xsNU5YvSgptw60C3rqc9N9OERP0_0 2026-02-24T22:34:25.567 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14409070877456460010', time=2026-02-25T00:26:35.856412+0000, chain.objs.size()=1 2026-02-24T22:34:25.567 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:f9qDcJ1aQlqgN3lJekSN4jyh0oau5j8_.dGu7xsNU5YvSgptw60C3rqc9N9OERP0_0 2026-02-24T22:34:25.567 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14409070877456460010', time=2026-02-25T00:26:35.862291+0000, chain.objs.size()=1 2026-02-24T22:34:25.567 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:f9qDcJ1aQlqgN3lJekSN4jyh0oau5j8_.dGu7xsNU5YvSgptw60C3rqc9N9OERP0_0 2026-02-24T22:34:25.568 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14409070877456460010', time=2026-02-25T00:26:35.866588+0000, chain.objs.size()=1 2026-02-24T22:34:25.568 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:f9qDcJ1aQlqgN3lJekSN4jyh0oau5j8_.dGu7xsNU5YvSgptw60C3rqc9N9OERP0_0 2026-02-24T22:34:25.568 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18431522042885399860', time=2026-02-25T00:26:35.869290+0000, chain.objs.size()=1 2026-02-24T22:34:25.568 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:LYW0dfFC-oCKZ5b-PO31m7FYbqvrGDd_.m_UFue4M58Fqt8uMXKwA2PBOb1mLJ4g_0 2026-02-24T22:34:25.568 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14409070877456460010', time=2026-02-25T00:26:35.876420+0000, chain.objs.size()=1 2026-02-24T22:34:25.568 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:f9qDcJ1aQlqgN3lJekSN4jyh0oau5j8_.dGu7xsNU5YvSgptw60C3rqc9N9OERP0_0 2026-02-24T22:34:25.568 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3065424761142383542', time=2026-02-25T00:26:44.777950+0000, chain.objs.size()=1 2026-02-24T22:34:25.568 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.JYUbwth6VKk9-7ZyBGvVbqnqrndgx_b_0 2026-02-24T22:34:25.569 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3128149907867657248', time=2026-02-25T00:26:44.780593+0000, chain.objs.size()=1 2026-02-24T22:34:25.569 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.qtIoXWQeG-Kna97izLANMlgmid6-5C-_0 2026-02-24T22:34:25.569 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10432328724444802158', time=2026-02-25T00:26:44.794265+0000, chain.objs.size()=1 2026-02-24T22:34:25.569 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Kz6VRMzYvnCjR8LDvd_yY3OzngXG1ZL_0 2026-02-24T22:34:25.569 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17771355179052443686', time=2026-02-25T00:26:44.796896+0000, chain.objs.size()=1 2026-02-24T22:34:25.569 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.lT0LUSn35oAvnbS0aQXIUqilygOvfCM_0 2026-02-24T22:34:25.569 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13526019900171516534', time=2026-02-25T00:26:45.021261+0000, chain.objs.size()=1 2026-02-24T22:34:25.569 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.-NF-iroP4rO8XIh6C88v2NbCV9_q2xi_0 2026-02-24T22:34:25.569 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2261236679222849025', time=2026-02-25T00:26:45.093315+0000, chain.objs.size()=1 2026-02-24T22:34:25.569 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.579+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.MDcICWxcLyzls3hWPlq_zLlxNrmo02S_0 2026-02-24T22:34:25.570 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/39588 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=68, truncated=0, next_marker='' 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4848917238806736861', time=2026-02-25T00:26:45.131321+0000, chain.objs.size()=1 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.EaNBcsjC5kFjNWo3nAyFYqOuyLYcWc__0 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18084809503487604244', time=2026-02-25T00:26:45.134950+0000, chain.objs.size()=1 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Prbi8W7IerC_KxEolYRHJKaPLf4lkte_0 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15459839548465530729', time=2026-02-25T00:26:45.138988+0000, chain.objs.size()=1 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_._9QodsfxCnYbbj5sgaQw6nrcCxcL1ng_0 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2608375037895501811', time=2026-02-25T00:26:45.155502+0000, chain.objs.size()=1 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.A7ysDMXBM_Lh1_pwcCbOxBZ_1OQiRZL_0 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10009166723705862446', time=2026-02-25T00:26:45.166463+0000, chain.objs.size()=1 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.oVy-me5ACkgGI1vkaPAlsK0dEk6sX8m_0 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10473743854803568080', time=2026-02-25T00:26:45.172969+0000, chain.objs.size()=1 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.fKng517rdF73lBMz9HXSwlKCeVK-oOK_0 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12089569143659431002', time=2026-02-25T00:26:45.252300+0000, chain.objs.size()=1 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.KbUY4FPj8Cfp4Tqs02aydfpBk84_jk2_0 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1673818414037664520', time=2026-02-25T00:26:45.260249+0000, chain.objs.size()=1 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.C3kMaiXFe8KgO5-VZNvCoARzIDhYY_V_0 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12647344083542267892', time=2026-02-25T00:26:45.283671+0000, chain.objs.size()=1 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.AOxLvV89Jvf_C0Ex0KpTM3kk47XM_5h_0 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5716583555057937189', time=2026-02-25T00:26:45.289095+0000, chain.objs.size()=1 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.PQHAsNIDoSdWlvE0OJ8rBXMYaBi0H4A_0 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12370428502204431417', time=2026-02-25T00:26:45.291465+0000, chain.objs.size()=1 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.tJnknpQk956NY-J8zB0wAPmil-hFA7J_0 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6536779141862616155', time=2026-02-25T00:26:45.304806+0000, chain.objs.size()=1 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.veOlB5JEkDz__JE5jx-BSig5Hn3GRzi_0 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7440949789663898500', time=2026-02-25T00:26:45.318245+0000, chain.objs.size()=1 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.PDyn2SdokOUlJSq240cxlAimxXN-tUB_0 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15282488077513638249', time=2026-02-25T00:26:45.320987+0000, chain.objs.size()=1 2026-02-24T22:34:25.573 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_._0MPxv8TpNSElYBHFpp4GQdRfswZTHJ_0 2026-02-24T22:34:25.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12672455772715930088', time=2026-02-25T00:26:45.592206+0000, chain.objs.size()=1 2026-02-24T22:34:25.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.6nUprYrfWlNfnM4KfDoyhvw9pgxvwby_0 2026-02-24T22:34:25.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3669416657429683227', time=2026-02-25T00:26:45.817499+0000, chain.objs.size()=1 2026-02-24T22:34:25.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.3YB1d5yw4SSI3Y7sHIN9Jno7poVh0_F_0 2026-02-24T22:34:25.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10494092699359773065', time=2026-02-25T00:26:45.824825+0000, chain.objs.size()=1 2026-02-24T22:34:25.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.mZR0NEO-YV5E5lJE_3p6XDJOnENXaTG_0 2026-02-24T22:34:25.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4940534171575792690', time=2026-02-25T00:26:45.825261+0000, chain.objs.size()=1 2026-02-24T22:34:25.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.oiMs3MIYCDL4lWLzb6yp86pVfBV9isC_0 2026-02-24T22:34:25.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11702663397365709588', time=2026-02-25T00:26:45.826764+0000, chain.objs.size()=1 2026-02-24T22:34:25.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.kYHgHCm6CMLpQ-_DkL3RMRW5ydVIpIf_0 2026-02-24T22:34:25.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18024294982991297727', time=2026-02-25T00:26:45.841333+0000, chain.objs.size()=1 2026-02-24T22:34:25.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.bf1zhou98PEme76v5w2Uk_7X7yIXam3_0 2026-02-24T22:34:25.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11497851352784523130', time=2026-02-25T00:26:45.921320+0000, chain.objs.size()=1 2026-02-24T22:34:25.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.yWYCXUZqqRPTWFtG9UeF0wzqfycHxdh_0 2026-02-24T22:34:25.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9287164630892921631', time=2026-02-25T00:26:45.931295+0000, chain.objs.size()=1 2026-02-24T22:34:25.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.jaEi809geoZbuARY1zLGL1NDO3tTQFW_0 2026-02-24T22:34:25.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16766674509193150371', time=2026-02-25T00:26:45.981735+0000, chain.objs.size()=1 2026-02-24T22:34:25.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.583+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.TqvqM5SxTgP_Kt4vCkMbLbAoqyV3kYE_0 2026-02-24T22:34:25.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17464172056308035215', time=2026-02-25T00:26:45.987832+0000, chain.objs.size()=1 2026-02-24T22:34:25.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.VfowDCHWGserIEIi8qhL7f97aoJOkEw_0 2026-02-24T22:34:25.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16655434879081115583', time=2026-02-25T00:26:45.989814+0000, chain.objs.size()=1 2026-02-24T22:34:25.574 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.sfjejDRIEcRL-5pE8KCk7olzDnCgHvv_0 2026-02-24T22:34:25.575 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.144712582234967128', time=2026-02-25T00:26:45.991303+0000, chain.objs.size()=1 2026-02-24T22:34:25.575 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.xitQkToAdqWy74wJuBklN0fEluA2rQs_0 2026-02-24T22:34:25.575 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7111394305701320904', time=2026-02-25T00:26:45.992668+0000, chain.objs.size()=1 2026-02-24T22:34:25.575 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.l7h0cY3iwhRqqHaic3pWB_6rehOEh4r_0 2026-02-24T22:34:25.575 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.250107929451322129', time=2026-02-25T00:26:46.097722+0000, chain.objs.size()=1 2026-02-24T22:34:25.575 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Etr7whAJoWSX0DEY80-WHqPocklUl-i_0 2026-02-24T22:34:25.575 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6026551416802451481', time=2026-02-25T00:26:46.131385+0000, chain.objs.size()=1 2026-02-24T22:34:25.575 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Nvu_WXuen4roRQE-N_PKulxXhqFvtH0_0 2026-02-24T22:34:25.575 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6231939704475225947', time=2026-02-25T00:26:53.188483+0000, chain.objs.size()=1 2026-02-24T22:34:25.575 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.8-5EdY_zmoEFQQ9halADuuKt8rmwULg_0 2026-02-24T22:34:25.575 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.69001521744617070', time=2026-02-25T00:26:53.218279+0000, chain.objs.size()=1 2026-02-24T22:34:25.575 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.dMDM2CWAfW_9BTJMX-PVuRJLkEiun2I_0 2026-02-24T22:34:25.575 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18302553026992401216', time=2026-02-25T00:26:53.424927+0000, chain.objs.size()=1 2026-02-24T22:34:25.575 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.25ZoXnVN1-f1LP3nEZ8CMYe6eSgNLCf_0 2026-02-24T22:34:25.575 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14530034857775365762', time=2026-02-25T00:26:53.434585+0000, chain.objs.size()=1 2026-02-24T22:34:25.575 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.OLB5Lzra-QGZEl_XuDPbxsbhI8VzeUc_0 2026-02-24T22:34:25.576 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4328327012305020518', time=2026-02-25T00:26:53.528297+0000, chain.objs.size()=1 2026-02-24T22:34:25.576 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Xs6uWe922EzSEcZlbuLGvPlPSPcyyv1_0 2026-02-24T22:34:25.576 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16060214433362129019', time=2026-02-25T00:26:53.552559+0000, chain.objs.size()=1 2026-02-24T22:34:25.576 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.bFsdzZgikCNxKl40OFvFSrKeOCXc3wA_0 2026-02-24T22:34:25.576 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12815479283566998743', time=2026-02-25T00:26:53.580844+0000, chain.objs.size()=1 2026-02-24T22:34:25.576 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.xjbzG9qN2CeIibXv-kDAdCE9Jle92Vp_0 2026-02-24T22:34:25.576 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16345113326159475838', time=2026-02-25T00:26:53.583168+0000, chain.objs.size()=1 2026-02-24T22:34:25.576 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Uv4v0kCxbdYnEpLxio1aZr0tvpnK0rN_0 2026-02-24T22:34:25.576 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6310178242001397432', time=2026-02-25T00:26:53.590763+0000, chain.objs.size()=1 2026-02-24T22:34:25.576 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.FWr6O9UKVvYTBYvub0ElJgNxUZDByYL_0 2026-02-24T22:34:25.576 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4238048365310850618', time=2026-02-25T00:26:53.609712+0000, chain.objs.size()=1 2026-02-24T22:34:25.576 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Cudl6ugE7Fn2t-lXAfEY_0eGmvzopzj_0 2026-02-24T22:34:25.576 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10384748070892593085', time=2026-02-25T00:26:53.614357+0000, chain.objs.size()=1 2026-02-24T22:34:25.576 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Ulyu9e2C2sLWJAeJ61nTwkhIyoXply4_0 2026-02-24T22:34:25.576 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7544084055128143077', time=2026-02-25T00:26:53.735972+0000, chain.objs.size()=1 2026-02-24T22:34:25.576 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.OSgJ4A4DB25ESX-SJsTyNtjMt4eb6um_0 2026-02-24T22:34:25.576 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7087198819678458881', time=2026-02-25T00:26:53.788588+0000, chain.objs.size()=1 2026-02-24T22:34:25.577 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.fZL7bgGE8Ij1NMKI-9wUa85unVyBc-k_0 2026-02-24T22:34:25.577 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6967331889966052209', time=2026-02-25T00:26:53.909581+0000, chain.objs.size()=1 2026-02-24T22:34:25.577 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.g1RoUUtKZdP9aKYBCw1ZBhfc6fUF3cw_0 2026-02-24T22:34:25.577 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15627754831558854856', time=2026-02-25T00:26:53.942738+0000, chain.objs.size()=1 2026-02-24T22:34:25.577 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.PpGvoRCULShSmIpPZF6ySiNRVikcYd3_0 2026-02-24T22:34:25.577 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8005607968866609283', time=2026-02-25T00:26:53.965401+0000, chain.objs.size()=1 2026-02-24T22:34:25.577 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.gHHiiaOzBxAn757yWDfqgE7kN4-aQvh_0 2026-02-24T22:34:25.577 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9621903912511803916', time=2026-02-25T00:26:53.996262+0000, chain.objs.size()=1 2026-02-24T22:34:25.577 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.TPdXRDUQehg-SXS5Eyv9BcRNNtHUC-C_0 2026-02-24T22:34:25.577 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2223706768028894639', time=2026-02-25T00:26:54.013663+0000, chain.objs.size()=1 2026-02-24T22:34:25.577 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.x_f5ptDlLzZl8NLiH3RhsLPx_CHGKmx_0 2026-02-24T22:34:25.577 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17308455707521229283', time=2026-02-25T00:26:54.022254+0000, chain.objs.size()=1 2026-02-24T22:34:25.577 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.jjKg8l05awFOMS8b6YE1CHRoTIJrhzX_0 2026-02-24T22:34:25.577 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18077232591456354591', time=2026-02-25T00:26:54.105811+0000, chain.objs.size()=1 2026-02-24T22:34:25.577 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.mLGaqXh3O4ik4gnTmfbP-FhQDFjSN4U_0 2026-02-24T22:34:25.577 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14515592236470423249', time=2026-02-25T00:26:54.142184+0000, chain.objs.size()=1 2026-02-24T22:34:25.577 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.CttqZCzNj2o0QmZfmyIUDA-iCRhQ_dy_0 2026-02-24T22:34:25.577 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16423099921079968204', time=2026-02-25T00:26:54.204092+0000, chain.objs.size()=1 2026-02-24T22:34:25.577 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_._O1mAyDXGtwTmTKfTJJFEqYRm4RRyaT_0 2026-02-24T22:34:25.577 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6666958948576106571', time=2026-02-25T00:26:54.297488+0000, chain.objs.size()=1 2026-02-24T22:34:25.577 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.MhD8aQpZbbR6O82wF6mYSXpGJXxP2ai_0 2026-02-24T22:34:25.577 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4393514766519898268', time=2026-02-25T00:26:54.297839+0000, chain.objs.size()=1 2026-02-24T22:34:25.577 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.yF_mk9sUN6eqa5CUqYdpq9RRUBF8p7O_0 2026-02-24T22:34:25.578 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10931855381732013802', time=2026-02-25T00:26:54.317502+0000, chain.objs.size()=1 2026-02-24T22:34:25.578 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Fh4eDM6LK4cu6Dcw2RgvTh8g4chmx2O_0 2026-02-24T22:34:25.578 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18338168100079833239', time=2026-02-25T00:26:54.488783+0000, chain.objs.size()=1 2026-02-24T22:34:25.578 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.XhYWksj0U6s8ePOeairRqivreRTp461_0 2026-02-24T22:34:25.578 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6310299779789534967', time=2026-02-25T00:26:54.535725+0000, chain.objs.size()=1 2026-02-24T22:34:25.578 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.2mm1Ox0szKzXcG-eS0TxpdWMmELHjI1_0 2026-02-24T22:34:25.578 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.422167291414062412', time=2026-02-25T00:26:57.231679+0000, chain.objs.size()=1 2026-02-24T22:34:25.578 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1694__shadow_.ijP_wcY2NAvgBEob67IIge87zO5dUHH_0 2026-02-24T22:34:25.578 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16126769529033021788', time=2026-02-25T00:27:46.160223+0000, chain.objs.size()=1 2026-02-24T22:34:25.578 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1775__shadow_.a1OTTLUztvl6jgBhd93v3McdwxW-TPW_0 2026-02-24T22:34:25.578 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7925251394593552122', time=2026-02-25T00:28:16.160427+0000, chain.objs.size()=1 2026-02-24T22:34:25.578 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1820__shadow_.AW3c4z6XdpoquChpqUXNsZz5tXEB_-f_0 2026-02-24T22:34:25.578 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13247875712349447005', time=2026-02-25T00:28:22.968864+0000, chain.objs.size()=1 2026-02-24T22:34:25.578 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1828__shadow_.-5ncmJxw-szTI9YG7YOg0exeOzOLy4d_0 2026-02-24T22:34:25.578 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8924659081102727658', time=2026-02-25T00:28:32.539948+0000, chain.objs.size()=1 2026-02-24T22:34:25.578 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.587+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1849__shadow_.nYcF6r5u6_45_ATUjr5F67mUlzv5xNn_0 2026-02-24T22:34:25.578 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.591+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15457429668554532388', time=2026-02-25T00:28:35.724142+0000, chain.objs.size()=1 2026-02-24T22:34:25.578 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.591+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1854__shadow_._1EenPX2OyudDXhzqMFjrRAmfiWPrLs_0 2026-02-24T22:34:25.578 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.591+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9036786380527060414', time=2026-02-25T00:29:12.893335+0000, chain.objs.size()=2 2026-02-24T22:34:25.578 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.591+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1891__multipart_mymultipart.2~M4QZyvYdst3oU3s4sKbFVAfNFRsb8uk.1 2026-02-24T22:34:25.579 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.591+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1891__multipart_mymultipart.2~M4QZyvYdst3oU3s4sKbFVAfNFRsb8uk.2 2026-02-24T22:34:25.579 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.591+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4412457380534368051', time=2026-02-25T00:30:42.803287+0000, chain.objs.size()=1 2026-02-24T22:34:25.579 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.591+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:XqBbu7ZwbUCcoythN3vGMh5ldylB5Lv_.Bn4yJ1fNMpbV7KVgfCmECOppfSXgPHm_0 2026-02-24T22:34:25.579 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.591+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14728762919755326285', time=2026-02-25T00:30:46.779601+0000, chain.objs.size()=1 2026-02-24T22:34:25.579 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.591+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:cd3SChT95BUZx-WBpIT2gWJDS6Pq7Ds_.KHYVsqK8nt72XKSO8ie7T-zg2A3pmQN_0 2026-02-24T22:34:25.579 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.591+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.942518829682895551', time=2026-02-25T00:33:42.865614+0000, chain.objs.size()=1 2026-02-24T22:34:25.579 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.591+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1965__shadow_.CsU317oXMAhIRZOQeCCwe6p4_mpVSJo_0 2026-02-24T22:34:25.579 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.591+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17023599689570426609', time=2026-02-25T00:34:07.105703+0000, chain.objs.size()=1 2026-02-24T22:34:25.579 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.591+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1990__shadow_.H7gWxT2JdH82alL_2_XrZdSOvtESfkj_0 2026-02-24T22:34:25.579 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.591+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12508733469224854186', time=2026-02-25T00:34:12.641932+0000, chain.objs.size()=5 2026-02-24T22:34:25.579 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.591+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1991__shadow_.TxnN_zpaEfP7FEZqaaqgLIErPyj0ykg_0 2026-02-24T22:34:25.579 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.591+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1991__shadow_.TxnN_zpaEfP7FEZqaaqgLIErPyj0ykg_1 2026-02-24T22:34:25.580 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.591+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1991__shadow_.TxnN_zpaEfP7FEZqaaqgLIErPyj0ykg_2 2026-02-24T22:34:25.580 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.591+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1991__shadow_.TxnN_zpaEfP7FEZqaaqgLIErPyj0ykg_3 2026-02-24T22:34:25.580 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.591+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1991__shadow_.TxnN_zpaEfP7FEZqaaqgLIErPyj0ykg_4 2026-02-24T22:34:25.582 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.595+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.584 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.595+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2026-02-24T22:34:25.585 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.595+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.586 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.595+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.586 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/38264' 2026-02-24T22:34:25.587 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2510832466323604743', time=2026-02-25T00:17:06.058852+0000, chain.objs.size()=1 2026-02-24T22:34:25.587 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523__shadow_.SJIoqP4hz5c8k3zlgBXShCJgr3YgO90_0 2026-02-24T22:34:25.587 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3639195480557891501', time=2026-02-25T00:17:07.116198+0000, chain.objs.size()=1 2026-02-24T22:34:25.587 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523__shadow_.b_Zhjb4jgciAanUdMllVgh4SaAdY-KB_0 2026-02-24T22:34:25.587 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3541637859302836615', time=2026-02-25T00:17:07.116615+0000, chain.objs.size()=1 2026-02-24T22:34:25.587 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523__shadow_.rqdCceBCVWiiE8ADwUtyWYkKdSCiC9C_0 2026-02-24T22:34:25.587 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3638762553056551707', time=2026-02-25T00:17:08.010353+0000, chain.objs.size()=1 2026-02-24T22:34:25.587 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524__shadow_.GdN9SYjral-Pd9cSjyR0Sup049dWu0H_0 2026-02-24T22:34:25.587 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16811624300742729691', time=2026-02-25T00:17:41.424883+0000, chain.objs.size()=1 2026-02-24T22:34:25.587 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526__shadow_.OrLpHIKy31rELi0CRL3sElFZRZNYrfz_0 2026-02-24T22:34:25.587 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12856342274379261129', time=2026-02-25T00:17:42.534133+0000, chain.objs.size()=1 2026-02-24T22:34:25.587 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527__shadow_.PzEMfEUCkYKR6hySBjw5dM9vNVnuN_3_0 2026-02-24T22:34:25.587 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1255039534738565300', time=2026-02-25T00:18:14.572428+0000, chain.objs.size()=1 2026-02-24T22:34:25.587 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528__shadow_.JmBCEXMztlJd_mOMfDcV0sLQ9sVTcZm_0 2026-02-24T22:34:25.587 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3981559791103190163', time=2026-02-25T00:18:48.186821+0000, chain.objs.size()=1 2026-02-24T22:34:25.587 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530__shadow_.cMnUosXMqJZbBmjSaA-hdID2k8G4CAp_0 2026-02-24T22:34:25.587 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6496923062472821821', time=2026-02-25T00:18:49.769689+0000, chain.objs.size()=1 2026-02-24T22:34:25.587 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531__shadow_.oVDK_lEtHywFw9dCJsLJXlWY0IJAZGi_0 2026-02-24T22:34:25.587 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15021662817268415715', time=2026-02-25T00:18:51.089893+0000, chain.objs.size()=1 2026-02-24T22:34:25.587 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532__shadow_.rE0DxKCrim-vCxR8Qn7mDZ2PbZuYh9w_0 2026-02-24T22:34:25.587 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12742054278111510240', time=2026-02-25T00:19:25.020073+0000, chain.objs.size()=1 2026-02-24T22:34:25.587 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534__shadow_.luEZy4PN6FC78p2m347M2pUGj37JDLd_0 2026-02-24T22:34:25.587 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15580199829899556868', time=2026-02-25T00:20:30.724536+0000, chain.objs.size()=1 2026-02-24T22:34:25.587 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537__shadow_.GWMfd9RWTyokz-wYM_dCHY90Sr3I56j_0 2026-02-24T22:34:25.587 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16985706045487211736', time=2026-02-25T00:20:32.510373+0000, chain.objs.size()=1 2026-02-24T22:34:25.587 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538__shadow_.hGQWw8q8W6KjUk0zpQIekQbkqorVoSO_0 2026-02-24T22:34:25.588 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11818152834236963081', time=2026-02-25T00:20:32.510879+0000, chain.objs.size()=1 2026-02-24T22:34:25.588 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538__shadow_.TNcwatE06BHdJSHxb9GIlGYtBuwIChy_0 2026-02-24T22:34:25.588 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10247107542659862581', time=2026-02-25T00:20:32.511448+0000, chain.objs.size()=1 2026-02-24T22:34:25.588 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538__shadow_.2eS3yIru4WGZ3N5lFe6NZn1ZxMXQBY8_0 2026-02-24T22:34:25.588 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.318716402777994809', time=2026-02-25T00:23:54.702274+0000, chain.objs.size()=1 2026-02-24T22:34:25.588 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1547__shadow_.H4E7jA13KZY0AmaCcIRDJQaiU364YuB_0 2026-02-24T22:34:25.588 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12745735688745813005', time=2026-02-25T00:25:35.994527+0000, chain.objs.size()=1 2026-02-24T22:34:25.588 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1601__shadow_.Zy9bY39Kg1Kpop7N0fH_UEgMeu8sMid_0 2026-02-24T22:34:25.588 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15776252561584990561', time=2026-02-25T00:25:42.667792+0000, chain.objs.size()=1 2026-02-24T22:34:25.588 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1610__shadow_.DngLE5zwF1xnwX6qTABNfKrNUF70hS5_0 2026-02-24T22:34:25.588 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6281093052066801323', time=2026-02-25T00:25:43.871623+0000, chain.objs.size()=1 2026-02-24T22:34:25.588 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1612__shadow_.hWEyhA-fKxsCm7dBIGqS-8yQhusNGT3_0 2026-02-24T22:34:25.588 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2014448832717093340', time=2026-02-25T00:25:47.314514+0000, chain.objs.size()=1 2026-02-24T22:34:25.588 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1618__shadow_.UbYJcyJFvAGbMxuGcpChtAEkzpSUPWr_0 2026-02-24T22:34:25.588 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10288535240191279459', time=2026-02-25T00:25:47.942610+0000, chain.objs.size()=1 2026-02-24T22:34:25.588 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1619__shadow_.BIzWwQ9-JfD1s7Z4VcZJSOa5Gy2KFng_0 2026-02-24T22:34:25.588 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2104686954230214677', time=2026-02-25T00:25:59.084672+0000, chain.objs.size()=1 2026-02-24T22:34:25.588 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ITaqi_anYTRdGWzuCnL94wkiYvHvMMz_0 2026-02-24T22:34:25.588 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3249875713972298362', time=2026-02-25T00:25:59.114312+0000, chain.objs.size()=1 2026-02-24T22:34:25.588 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.l5Pe3fJuSBGN2-nrkY5Xw6Xrlp7Mn31_0 2026-02-24T22:34:25.588 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9527241847722714204', time=2026-02-25T00:25:59.260756+0000, chain.objs.size()=1 2026-02-24T22:34:25.588 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.FEc4mdxJ7EuVQv4qdfvTWclNwdq7SlF_0 2026-02-24T22:34:25.589 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6065569732642975114', time=2026-02-25T00:25:59.269428+0000, chain.objs.size()=1 2026-02-24T22:34:25.589 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.IRv450ch9MULOdvsGRE7ijOMoZcY4rV_0 2026-02-24T22:34:25.589 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13229029652102844555', time=2026-02-25T00:25:59.409319+0000, chain.objs.size()=1 2026-02-24T22:34:25.589 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.a2zXA85aq1NKCMYCTUtzutGHh5tgaWA_0 2026-02-24T22:34:25.589 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3765107265413304060', time=2026-02-25T00:25:59.459680+0000, chain.objs.size()=1 2026-02-24T22:34:25.589 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.-uyvV3wBlgck_oqDuFw3sFMLO_fwyaF_0 2026-02-24T22:34:25.589 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17327614098086232406', time=2026-02-25T00:25:59.562367+0000, chain.objs.size()=1 2026-02-24T22:34:25.589 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.hn_to8VEhHDPq0mwuswLx0zOPt98dQq_0 2026-02-24T22:34:25.589 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12040130674580211269', time=2026-02-25T00:25:59.564715+0000, chain.objs.size()=1 2026-02-24T22:34:25.589 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.iITnJbMa4YrclN1O2fB_zeQbPy79yVc_0 2026-02-24T22:34:25.590 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7876080549168285702', time=2026-02-25T00:25:59.576574+0000, chain.objs.size()=1 2026-02-24T22:34:25.590 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.U4vVaEpcmTdr261eGBn8SHA7AXFz0GZ_0 2026-02-24T22:34:25.590 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10344210331908957073', time=2026-02-25T00:25:59.594992+0000, chain.objs.size()=1 2026-02-24T22:34:25.590 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.VDGsG9e7baPgPCXqGbLsqaFS3_C1Hx0_0 2026-02-24T22:34:25.590 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3565096662475233388', time=2026-02-25T00:25:59.628869+0000, chain.objs.size()=1 2026-02-24T22:34:25.590 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.0V2L7wwrnJ5RGxi0rZ6BrgZTd_LTHhf_0 2026-02-24T22:34:25.590 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.216291445975907479', time=2026-02-25T00:25:59.644999+0000, chain.objs.size()=1 2026-02-24T22:34:25.590 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.6CvX5ThKB_BEgW2sKr4CFTgbYTacw7E_0 2026-02-24T22:34:25.590 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7750753234478713400', time=2026-02-25T00:25:59.791459+0000, chain.objs.size()=1 2026-02-24T22:34:25.590 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_._OVxsYKmMpo2A9XeWy5w1rJ6LM-7J39_0 2026-02-24T22:34:25.590 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13498820192012267084', time=2026-02-25T00:25:59.797871+0000, chain.objs.size()=1 2026-02-24T22:34:25.590 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.5f5mG8nW3TtJiffrllOWwP_LnJ4JpIm_0 2026-02-24T22:34:25.590 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16931285229326698691', time=2026-02-25T00:25:59.916475+0000, chain.objs.size()=1 2026-02-24T22:34:25.590 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.599+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.7wzDDRp2_-giS62Z9m36lMyoGFKQvaM_0 2026-02-24T22:34:25.590 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3163272393743647878', time=2026-02-25T00:25:59.946376+0000, chain.objs.size()=1 2026-02-24T22:34:25.590 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.wLC9TvjqkwjkfUxHBpKL3b-OLDR0yxY_0 2026-02-24T22:34:25.590 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7348515657809909898', time=2026-02-25T00:26:00.045840+0000, chain.objs.size()=1 2026-02-24T22:34:25.590 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ap2zsDGmZmnVxXMtnys94HsFdj4-HWY_0 2026-02-24T22:34:25.590 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3619503264330264832', time=2026-02-25T00:26:00.073215+0000, chain.objs.size()=1 2026-02-24T22:34:25.590 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.cNUJt33Py3E_oy-FxF6AUHIu1BenOqu_0 2026-02-24T22:34:25.590 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7384566495246780128', time=2026-02-25T00:26:00.120222+0000, chain.objs.size()=1 2026-02-24T22:34:25.591 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.e2xfMwxCbIlZ3Nq40IsLcmi1etKBhCa_0 2026-02-24T22:34:25.591 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17032610504834248822', time=2026-02-25T00:26:00.126285+0000, chain.objs.size()=1 2026-02-24T22:34:25.591 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.VitOgyRWvoh_P7gBvQQRfKNsq9qsfUh_0 2026-02-24T22:34:25.591 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5228716010593355418', time=2026-02-25T00:26:00.139328+0000, chain.objs.size()=1 2026-02-24T22:34:25.591 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.JmvCtn0JZ4illfqh8TV5vcan1nkUmc__0 2026-02-24T22:34:25.591 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4710966942223646113', time=2026-02-25T00:26:00.219543+0000, chain.objs.size()=1 2026-02-24T22:34:25.591 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.q6lmvbmLBv_9Ed1_K3Uxa_h0y-CjOZ2_0 2026-02-24T22:34:25.591 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.334985017948235428', time=2026-02-25T00:26:00.248991+0000, chain.objs.size()=1 2026-02-24T22:34:25.591 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.FPtKik_ODD_TlC61x4e2vkaEQmlr5tk_0 2026-02-24T22:34:25.591 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4657087229604808637', time=2026-02-25T00:26:00.775068+0000, chain.objs.size()=1 2026-02-24T22:34:25.591 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1629__shadow_.ZoWTmwKSoC7_YyQ3R0YH-RtpRxFGVem_0 2026-02-24T22:34:25.591 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6075167376028118172', time=2026-02-25T00:26:04.278102+0000, chain.objs.size()=1 2026-02-24T22:34:25.591 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1635__shadow_.7EQ8CN2x2FarF8C3N-WRsFbSIG4Hqog_0 2026-02-24T22:34:25.591 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3372478423969483361', time=2026-02-25T00:26:10.610057+0000, chain.objs.size()=1 2026-02-24T22:34:25.591 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1646__shadow_.G9i5LO45c-VbkT3TkGWHJuNRzvfgCoQ_0 2026-02-24T22:34:25.591 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1039067791450130449', time=2026-02-25T00:26:11.787394+0000, chain.objs.size()=1 2026-02-24T22:34:25.591 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1648__shadow_.Yu2VLAbkcPAQBNv8RJmuJtB5eQw6KUF_0 2026-02-24T22:34:25.591 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4717272222599086857', time=2026-02-25T00:26:18.498023+0000, chain.objs.size()=1 2026-02-24T22:34:25.591 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1658__shadow_.OhHo67rlTAvoDukmleNx1BGgEh1nLmd_0 2026-02-24T22:34:25.591 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15394038730116281027', time=2026-02-25T00:26:20.030226+0000, chain.objs.size()=1 2026-02-24T22:34:25.591 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1660__shadow_.Lpvshw3mINHZbXGl_HsiLtvhpcqOErv_0 2026-02-24T22:34:25.591 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3780047958645861734', time=2026-02-25T00:26:20.558343+0000, chain.objs.size()=1 2026-02-24T22:34:25.592 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1661__shadow_.pv8ezhpWz-het1JXWN9XEfPBcIBaoKB_0 2026-02-24T22:34:25.592 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1763047085782782359', time=2026-02-25T00:26:21.122063+0000, chain.objs.size()=1 2026-02-24T22:34:25.592 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1662__shadow_.Te5U8eDlOEycHCrDcHeWgdt4VsoZuKf_0 2026-02-24T22:34:25.592 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17982211662161548149', time=2026-02-25T00:26:25.270186+0000, chain.objs.size()=1 2026-02-24T22:34:25.592 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1669__shadow_.SlKjDJm5LocNUBHpxmv_7vky-l8-99s_0 2026-02-24T22:34:25.592 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15084239717742401119', time=2026-02-25T00:26:44.862564+0000, chain.objs.size()=1 2026-02-24T22:34:25.592 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.mLknvw_IE6Q7LdVbDBMo5fJWVdNzsnX_0 2026-02-24T22:34:25.592 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6072949179860070964', time=2026-02-25T00:26:44.952754+0000, chain.objs.size()=1 2026-02-24T22:34:25.592 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.GQTocdhqOQ_RYaVEaW_Jctm5xOXc38l_0 2026-02-24T22:34:25.592 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5502674732917295381', time=2026-02-25T00:26:44.983696+0000, chain.objs.size()=1 2026-02-24T22:34:25.592 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.fxF0Y1yu4NOp-07l2YACgy2wpPj7NyB_0 2026-02-24T22:34:25.592 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8398693087584212051', time=2026-02-25T00:26:44.992902+0000, chain.objs.size()=1 2026-02-24T22:34:25.592 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.l1Sqk_BuaXhguKrcCZWoUlF64UnS34-_0 2026-02-24T22:34:25.592 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12942677455553832774', time=2026-02-25T00:26:45.094390+0000, chain.objs.size()=1 2026-02-24T22:34:25.592 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.gHIo7bT3zQQ9txdAHBsXZtadQf6Bh97_0 2026-02-24T22:34:25.592 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9230904811309366206', time=2026-02-25T00:26:45.128441+0000, chain.objs.size()=1 2026-02-24T22:34:25.592 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.jIOqQ2GIlVUv4qBg6HW_PWVAU8jzLpW_0 2026-02-24T22:34:25.592 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6144189513777084042', time=2026-02-25T00:26:45.153613+0000, chain.objs.size()=1 2026-02-24T22:34:25.592 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.qVC9zDWIHOcZGGXFhl46MoXC_wg21GX_0 2026-02-24T22:34:25.593 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4633283733282167675', time=2026-02-25T00:26:45.165909+0000, chain.objs.size()=1 2026-02-24T22:34:25.593 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.vRcsDCj3w8o9SMe2h-leGGTAmJWfZGz_0 2026-02-24T22:34:25.593 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2542527760521451907', time=2026-02-25T00:26:45.272491+0000, chain.objs.size()=1 2026-02-24T22:34:25.593 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.CyeusnBXNQrukFycVBfAB13pKAFvhkR_0 2026-02-24T22:34:25.593 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1902827355549493816', time=2026-02-25T00:26:45.298563+0000, chain.objs.size()=1 2026-02-24T22:34:25.593 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.SZ96FT5tguEyanNoWIAoWDaviVxdx6I_0 2026-02-24T22:34:25.593 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13128976731958755994', time=2026-02-25T00:26:45.303008+0000, chain.objs.size()=1 2026-02-24T22:34:25.593 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.BZx_z1eNWVAIGsoJlf3KVptdnwHmMs8_0 2026-02-24T22:34:25.593 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12760092017040052619', time=2026-02-25T00:26:45.317191+0000, chain.objs.size()=1 2026-02-24T22:34:25.593 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.b8oYjaORArPLePRBc1G6qEx_2CcxNLz_0 2026-02-24T22:34:25.593 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12124558409344819777', time=2026-02-25T00:26:45.402654+0000, chain.objs.size()=1 2026-02-24T22:34:25.593 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.nAzfj4xO2Ua9ufQr9kcdzo0RXyevjkN_0 2026-02-24T22:34:25.593 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7523380574494630125', time=2026-02-25T00:26:45.434012+0000, chain.objs.size()=1 2026-02-24T22:34:25.593 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ArKMQz1gyaIL8sq_aP2RCs2wNVD1xiG_0 2026-02-24T22:34:25.593 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1483888401566627976', time=2026-02-25T00:26:45.441465+0000, chain.objs.size()=1 2026-02-24T22:34:25.593 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ypcGwqac5fm39HddAnsr8aG6EEd9k6V_0 2026-02-24T22:34:25.593 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17349515594060042233', time=2026-02-25T00:26:45.487343+0000, chain.objs.size()=1 2026-02-24T22:34:25.593 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.XDtE40ApYW8WOx0mQ8_Z3Q5WP0yDoN4_0 2026-02-24T22:34:25.593 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7078636370296749885', time=2026-02-25T00:26:45.587808+0000, chain.objs.size()=1 2026-02-24T22:34:25.593 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Iwuqhe-dCs0n_8PVU2Z2Gylg_hOE3Ff_0 2026-02-24T22:34:25.594 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14397695393608693271', time=2026-02-25T00:26:45.589756+0000, chain.objs.size()=1 2026-02-24T22:34:25.594 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.q0oliCPvsOl7vTAQGwjt3xhVteoZwJu_0 2026-02-24T22:34:25.594 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5322763138627694968', time=2026-02-25T00:26:45.599411+0000, chain.objs.size()=1 2026-02-24T22:34:25.594 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.A9KPZRG09LFoM6XaIlCbAKcDbJkBqab_0 2026-02-24T22:34:25.594 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12343909053468711483', time=2026-02-25T00:26:45.838785+0000, chain.objs.size()=1 2026-02-24T22:34:25.594 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.603+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.bKuXU6ECCx1yzHMn-JS_OOhvXxFXOWX_0 2026-02-24T22:34:25.594 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17431740351992297138', time=2026-02-25T00:26:45.946489+0000, chain.objs.size()=1 2026-02-24T22:34:25.594 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_._C5g90FlIStAdf3EDyZGAlp5xUDvAlE_0 2026-02-24T22:34:25.594 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6886665817927675076', time=2026-02-25T00:26:45.975964+0000, chain.objs.size()=1 2026-02-24T22:34:25.594 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.bw1q4NBnUO5hmA5mzhrbLOeDOR9rVYP_0 2026-02-24T22:34:25.595 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5035746473225984580', time=2026-02-25T00:26:46.083789+0000, chain.objs.size()=1 2026-02-24T22:34:25.595 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ULmBIWocdPYvFQvqkt0oQWOnSO02-nZ_0 2026-02-24T22:34:25.595 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7519407726573850730', time=2026-02-25T00:26:46.111727+0000, chain.objs.size()=1 2026-02-24T22:34:25.595 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.x5L1SA5-fYOcDZIwHmwLSVoAe2ioIFU_0 2026-02-24T22:34:25.595 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10655911239404145919', time=2026-02-25T00:26:46.114232+0000, chain.objs.size()=1 2026-02-24T22:34:25.595 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.NXmSnV80oggFa6aAmpHVQdyfUBXnuR9_0 2026-02-24T22:34:25.595 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10532683645430112462', time=2026-02-25T00:26:46.117732+0000, chain.objs.size()=1 2026-02-24T22:34:25.595 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.u9fxvdeU4rs9j21Shin_jh5w1NQ___z_0 2026-02-24T22:34:25.595 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4648394068313010084', time=2026-02-25T00:26:46.139697+0000, chain.objs.size()=1 2026-02-24T22:34:25.595 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ufLW7R40vFPi_Tb2RY3cxex1kroBu7t_0 2026-02-24T22:34:25.595 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4588197547536283353', time=2026-02-25T00:26:46.142493+0000, chain.objs.size()=1 2026-02-24T22:34:25.595 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.jEmbmIR2Ip_i-DsssEOWRk4ygM2-nn-_0 2026-02-24T22:34:25.595 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4872005714115998790', time=2026-02-25T00:26:53.146209+0000, chain.objs.size()=1 2026-02-24T22:34:25.595 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.m0ZUAh_x5sp0z-uQiX88ISwgfVQaIL__0 2026-02-24T22:34:25.595 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1328959365100967222', time=2026-02-25T00:26:53.166914+0000, chain.objs.size()=1 2026-02-24T22:34:25.595 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.hoKaSuiRmQY2zpGmgQPPJ-Jnete_4cq_0 2026-02-24T22:34:25.595 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2518422342815105963', time=2026-02-25T00:26:53.207902+0000, chain.objs.size()=1 2026-02-24T22:34:25.595 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ILrAknGD0pEbkwxBQId196RCk5o0XJ0_0 2026-02-24T22:34:25.595 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5063392191459350306', time=2026-02-25T00:26:53.221656+0000, chain.objs.size()=1 2026-02-24T22:34:25.595 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.DgVW3c6T60kYlfP6H25yLdoSa4Rbw6y_0 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11904532103595851622', time=2026-02-25T00:26:53.339724+0000, chain.objs.size()=1 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Hct6S4WkmsqNe_DQNWugt17cXGi5yT0_0 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7769830569786601685', time=2026-02-25T00:26:53.347081+0000, chain.objs.size()=1 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ukfJc6S4ZOziLO3ybah-mShQTKs5qTW_0 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6237135672016597787', time=2026-02-25T00:26:53.393758+0000, chain.objs.size()=1 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.VBp9HSx8iCsYiWTLNjBGQXiBTTWcB5e_0 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1951986832107345248', time=2026-02-25T00:26:53.579728+0000, chain.objs.size()=1 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.D0Ge-hFmYaF456bG2EG5RqgWGVAH38X_0 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16155164772169544898', time=2026-02-25T00:26:53.584164+0000, chain.objs.size()=1 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.6D3bEDQ-H4kXPlha_994JMUYLejSy8G_0 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9294984262015387962', time=2026-02-25T00:26:53.587091+0000, chain.objs.size()=1 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.PVmVcey-Aa7Qgv_ohu12VGw9AVSYHTF_0 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10674574150074597458', time=2026-02-25T00:26:53.615682+0000, chain.objs.size()=1 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.0IhlMJll07LYZfs4krzyNkI58k-gK3n_0 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9070440652555740903', time=2026-02-25T00:26:53.732506+0000, chain.objs.size()=1 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Bk-Eu85Erqd68oYq7JxmIeOi5OztJ-5_0 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15398316306203874281', time=2026-02-25T00:26:53.767934+0000, chain.objs.size()=1 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.NuPwcnZuPvnSkG9vSqIbcRi0ndcKMwb_0 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11353076301908878685', time=2026-02-25T00:26:53.777672+0000, chain.objs.size()=1 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.K9B7Fm-6tXfku9bnngF8wABYBnz_Z35_0 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12771375517000916009', time=2026-02-25T00:26:53.788136+0000, chain.objs.size()=1 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.v3lROtB2cHE1h0WHbeQ1jwLO9n4dfLL_0 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15844963449208266811', time=2026-02-25T00:26:53.836814+0000, chain.objs.size()=1 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.XPt8NBhNmmsoy1YPfASxbt6fIUWyrrB_0 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3037600235578799677', time=2026-02-25T00:26:53.838064+0000, chain.objs.size()=1 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.jFlfkMKKwK8VsL-Z3MQsO9XrY1RdAiD_0 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9796590041193810081', time=2026-02-25T00:26:53.916087+0000, chain.objs.size()=1 2026-02-24T22:34:25.596 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.mWUXnMIO2JBj_gBxWQH1_UTSU-GgtZF_0 2026-02-24T22:34:25.597 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14777618705389099463', time=2026-02-25T00:26:53.917420+0000, chain.objs.size()=1 2026-02-24T22:34:25.597 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.xNPt7fCI6Mnz1XwdEu1ARdkV4RhU5K__0 2026-02-24T22:34:25.597 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.607+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/38264 2026-02-24T22:34:25.599 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=33, truncated=0, next_marker='' 2026-02-24T22:34:25.600 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2250955709783529567', time=2026-02-25T00:26:54.097755+0000, chain.objs.size()=1 2026-02-24T22:34:25.600 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.CbI9CH1-b7Jbxx2v9q5e0eiGyTivImo_0 2026-02-24T22:34:25.600 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7620507898936400142', time=2026-02-25T00:26:54.114313+0000, chain.objs.size()=1 2026-02-24T22:34:25.600 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.XoDBTlWgT2StenVwVNPyqqSA5uWxzE__0 2026-02-24T22:34:25.600 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9416286619685705630', time=2026-02-25T00:26:54.170790+0000, chain.objs.size()=1 2026-02-24T22:34:25.600 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.6g-yIFdbKfufE1ZdhNlO5ElIhEF7lgF_0 2026-02-24T22:34:25.600 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.374246993485174153', time=2026-02-25T00:26:54.197541+0000, chain.objs.size()=1 2026-02-24T22:34:25.600 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.bCN06oFzS6m0THGcHez5c8QDQkHlucs_0 2026-02-24T22:34:25.600 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1646202947643180205', time=2026-02-25T00:26:54.200718+0000, chain.objs.size()=1 2026-02-24T22:34:25.600 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.xhv5DVBTwd7IsTA6LGqG2UZ-qB38KOY_0 2026-02-24T22:34:25.600 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12956029642701205512', time=2026-02-25T00:26:54.204967+0000, chain.objs.size()=1 2026-02-24T22:34:25.600 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.WCRhfBeQe8mp8Ai-kS6F_4uUUK2bI1U_0 2026-02-24T22:34:25.600 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2798598040295842504', time=2026-02-25T00:26:54.283045+0000, chain.objs.size()=1 2026-02-24T22:34:25.600 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.nqewzTiioGDGEtydGbmZGjDb34BLAhM_0 2026-02-24T22:34:25.600 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1944622168283384656', time=2026-02-25T00:26:54.307376+0000, chain.objs.size()=1 2026-02-24T22:34:25.600 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.tAWXcM9NT4ASXth7J51lIcMXwMNQVfT_0 2026-02-24T22:34:25.600 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.398917540968263830', time=2026-02-25T00:26:54.311860+0000, chain.objs.size()=1 2026-02-24T22:34:25.600 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.xo-fEUH7wRrCwNYTmmV-YZVRY1IPOjb_0 2026-02-24T22:34:25.600 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5154646054916785625', time=2026-02-25T00:26:54.465081+0000, chain.objs.size()=1 2026-02-24T22:34:25.600 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Z2IQkZbXOyCRVbiSp_nd5heP49QagBU_0 2026-02-24T22:34:25.600 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16516746849519871680', time=2026-02-25T00:26:54.488953+0000, chain.objs.size()=1 2026-02-24T22:34:25.600 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.c7cZovkehdKGcyafj20iKbOqIlq086t_0 2026-02-24T22:34:25.600 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15475612091031094736', time=2026-02-25T00:26:54.504585+0000, chain.objs.size()=1 2026-02-24T22:34:25.600 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.hhhlxvnqAo-O0atVPkjgmBkgmOBRLLE_0 2026-02-24T22:34:25.601 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5012813410377005808', time=2026-02-25T00:26:54.517474+0000, chain.objs.size()=1 2026-02-24T22:34:25.601 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ODnKfFOUfeybNfEoPxWxOG0NkrW_lGi_0 2026-02-24T22:34:25.601 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15564824154007725879', time=2026-02-25T00:26:58.464333+0000, chain.objs.size()=1 2026-02-24T22:34:25.601 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1696__shadow_.h_WJcslxnpjH7dokSgek2P1Lj22C00g_0 2026-02-24T22:34:25.601 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6287272725140729291', time=2026-02-25T00:27:01.908952+0000, chain.objs.size()=1 2026-02-24T22:34:25.601 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1702__shadow_.w7MuVLX3QTpxoZdWtjRd1NyfVAMGCwY_0 2026-02-24T22:34:25.601 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1673347712003339650', time=2026-02-25T00:27:06.436507+0000, chain.objs.size()=1 2026-02-24T22:34:25.601 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1710__shadow_.YTw9B58--LhEdb_AbiJRa0nwo2A-StK_0 2026-02-24T22:34:25.601 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7039875516138052775', time=2026-02-25T00:27:26.649030+0000, chain.objs.size()=1 2026-02-24T22:34:25.601 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1746__shadow_.inSa_aqPgVKYsTZbLUqp-DNn-GiBJ4r_0 2026-02-24T22:34:25.601 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18175497524198162240', time=2026-02-25T00:27:59.892318+0000, chain.objs.size()=1 2026-02-24T22:34:25.601 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1795__shadow_.kqubPJAhNze4E2K2aKp1kKtiIBUTDW1_0 2026-02-24T22:34:25.601 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4947329627519666108', time=2026-02-25T00:28:05.852007+0000, chain.objs.size()=1 2026-02-24T22:34:25.601 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1805__shadow_.wzruTx-BrtW1AatmQZ00OFTGfKmpDJu_0 2026-02-24T22:34:25.601 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12860518351005426985', time=2026-02-25T00:28:16.991693+0000, chain.objs.size()=1 2026-02-24T22:34:25.601 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1821__shadow_.o7LE3YjUb9x6XdfXyrTA0MP2U7BOSu7_0 2026-02-24T22:34:25.601 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16351295460268939750', time=2026-02-25T00:28:19.548534+0000, chain.objs.size()=1 2026-02-24T22:34:25.601 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1824__shadow_.71UHv0oUtiEC39TXlyKUk80blySzsBT_0 2026-02-24T22:34:25.601 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11549027916387982420', time=2026-02-25T00:28:35.723699+0000, chain.objs.size()=1 2026-02-24T22:34:25.601 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1854__shadow_._1EenPX2OyudDXhzqMFjrRAmfiWPrLs_0 2026-02-24T22:34:25.601 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17507111087348341215', time=2026-02-25T00:30:22.059887+0000, chain.objs.size()=1 2026-02-24T22:34:25.601 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:xJpcX9yQxcpeNLemsyKIL4TETn75dXN_.VDhafxCL717IDRj6GS5GRM3_Dwo7Gi7_0 2026-02-24T22:34:25.602 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12087577639786719733', time=2026-02-25T00:30:29.043518+0000, chain.objs.size()=1 2026-02-24T22:34:25.602 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1943__shadow:6xkDhmZuSTbwOV5pLanrpGWYqB1Qe28_.SFpd2lKMoIazWNpRSDNNRmhCNEiQaLM_0 2026-02-24T22:34:25.602 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3499006341737398403', time=2026-02-25T00:30:31.034373+0000, chain.objs.size()=1 2026-02-24T22:34:25.602 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1944__shadow_.YC5sSPHuu3Hzx29zZwsyGB201uava8Z_0 2026-02-24T22:34:25.602 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17365624503916577342', time=2026-02-25T00:30:31.130622+0000, chain.objs.size()=1 2026-02-24T22:34:25.602 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1944__shadow_.YC5sSPHuu3Hzx29zZwsyGB201uava8Z_0 2026-02-24T22:34:25.602 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6241103368083807241', time=2026-02-25T00:30:41.946970+0000, chain.objs.size()=1 2026-02-24T22:34:25.602 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:-Zy6KM.6z3cOuYYFrKmW5OoRWY6SMYD_._rgmaQ8s2HoAcO8Lx70-ghGenNkweEG_0 2026-02-24T22:34:25.602 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12830618274843885984', time=2026-02-25T00:30:45.414849+0000, chain.objs.size()=1 2026-02-24T22:34:25.602 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:MXyMGcl5Dy8zUM2TQXE6UmIufpEgr8e_.uPfYQKCHFmB6oUpRIOL2MHagXNZdsRn_0 2026-02-24T22:34:25.602 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12424871280852077293', time=2026-02-25T00:30:50.989097+0000, chain.objs.size()=1 2026-02-24T22:34:25.602 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1949__shadow:xqb5I3i9WXjDOIMOTFpwwDi5a3zfwOE_.txmCdIW1yr7a0iQwyrtEILiN_lOdTTT_0 2026-02-24T22:34:25.602 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18250398468060073519', time=2026-02-25T00:30:53.033712+0000, chain.objs.size()=1 2026-02-24T22:34:25.602 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1952__shadow:nNrX53TBBnqKhS3CHA.9PLP3NAn3W1A_.nx_crXmYEXNrIwuckymKUuqzfhDIiOC_0 2026-02-24T22:34:25.602 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1023848614653828641', time=2026-02-25T00:30:57.190868+0000, chain.objs.size()=1 2026-02-24T22:34:25.602 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:WIeboASLfpdSdDTtL2T4HuXFoUnburp_.WOkQzDuwvX6MmojFePZ76yx9aZvYidM_0 2026-02-24T22:34:25.602 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1023848614653828641', time=2026-02-25T00:30:57.191280+0000, chain.objs.size()=1 2026-02-24T22:34:25.602 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:WIeboASLfpdSdDTtL2T4HuXFoUnburp_.WOkQzDuwvX6MmojFePZ76yx9aZvYidM_0 2026-02-24T22:34:25.602 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10028021305234262400', time=2026-02-25T00:33:39.963183+0000, chain.objs.size()=1 2026-02-24T22:34:25.602 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.611+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1962__shadow_.DrqDnR9yWmUkuWnKv8rxtZTsbrL9UtZ_0 2026-02-24T22:34:25.603 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.615+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.605 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.615+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2026-02-24T22:34:25.606 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.615+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.606 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.607 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/38987' 2026-02-24T22:34:25.607 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='_ts56h4efjrgjs2u92KKZkl03X4Y-q0M', time=2026-02-25T00:15:34.760265+0000, chain.objs.size()=2 2026-02-24T22:34:25.607 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__shadow_.0YMmA6C8ADUnevp6GXGSxQcYwmX1W7K_0 2026-02-24T22:34:25.607 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__shadow_.0YMmA6C8ADUnevp6GXGSxQcYwmX1W7K_1 2026-02-24T22:34:25.607 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17574378362889513698', time=2026-02-25T00:15:34.777774+0000, chain.objs.size()=3 2026-02-24T22:34:25.607 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-STANDARD.2~jB8VjG_eXaI_0rIVLXOTPSldrI2t5vd.1 2026-02-24T22:34:25.607 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-STANDARD.2~jB8VjG_eXaI_0rIVLXOTPSldrI2t5vd.2 2026-02-24T22:34:25.607 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-STANDARD.2~jB8VjG_eXaI_0rIVLXOTPSldrI2t5vd.3 2026-02-24T22:34:25.607 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15417456784467582125', time=2026-02-25T00:15:34.847330+0000, chain.objs.size()=1 2026-02-24T22:34:25.607 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1508__shadow_.euiyy4tPIEvRNg9E7wHR8g4NX9Qp2EJ_0 2026-02-24T22:34:25.607 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14003482079085538370', time=2026-02-25T00:16:32.620835+0000, chain.objs.size()=1 2026-02-24T22:34:25.607 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520__shadow_.CRL3xb-6jbi5zJqttWOr3OVwv3heRVn_0 2026-02-24T22:34:25.607 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.685613790712368000', time=2026-02-25T00:17:05.177528+0000, chain.objs.size()=1 2026-02-24T22:34:25.607 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522__shadow_.fCPa8NHyKcYfnrJ_tjlzRbAvBqa387e_0 2026-02-24T22:34:25.607 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1694032610315432793', time=2026-02-25T00:17:40.988128+0000, chain.objs.size()=1 2026-02-24T22:34:25.607 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526__shadow_.oVqr2MyfrpfQ3WddUnfJ0wdwzDW3guW_0 2026-02-24T22:34:25.607 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.104561608082950028', time=2026-02-25T00:17:41.431813+0000, chain.objs.size()=1 2026-02-24T22:34:25.608 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526__shadow_.Lj81P7tbavcNIvQSYZhqcia9QcM5jeX_0 2026-02-24T22:34:25.608 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14652824338034465430', time=2026-02-25T00:18:14.580868+0000, chain.objs.size()=1 2026-02-24T22:34:25.608 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528__shadow_.lpMHlNsao1GQ32puE4aB65DzhoGLWck_0 2026-02-24T22:34:25.608 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10212756095346567489', time=2026-02-25T00:18:51.096500+0000, chain.objs.size()=1 2026-02-24T22:34:25.608 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532__shadow_.Dkx7mqGOIm_CytL0FdFPcGr1jjWd9qd_0 2026-02-24T22:34:25.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7075396812175686388', time=2026-02-25T00:18:51.099156+0000, chain.objs.size()=1 2026-02-24T22:34:25.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532__shadow_.TkcwfqMVuTXWANL349R9MR_npiSbe3j_0 2026-02-24T22:34:25.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13985041441154490496', time=2026-02-25T00:19:58.463342+0000, chain.objs.size()=1 2026-02-24T22:34:25.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536__shadow_.M2bgCepucPnDYCXmEPSnCYwLmYVQRSa_0 2026-02-24T22:34:25.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9185254107824367256', time=2026-02-25T00:20:32.520137+0000, chain.objs.size()=1 2026-02-24T22:34:25.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538__shadow_.1L9ot63xztL6dS8JfHtg7bdUcN3natf_0 2026-02-24T22:34:25.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8430326170841419452', time=2026-02-25T00:20:32.525733+0000, chain.objs.size()=1 2026-02-24T22:34:25.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538__shadow_.tKGKx92DpmHQfV750vLGStcAdilttmZ_0 2026-02-24T22:34:25.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12026951494364349923', time=2026-02-25T00:21:06.321466+0000, chain.objs.size()=1 2026-02-24T22:34:25.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540__shadow_.DAWkLyv9lt9bSiOiVJxxTgR4dib2UTZ_0 2026-02-24T22:34:25.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9086512638289503936', time=2026-02-25T00:25:45.021790+0000, chain.objs.size()=1 2026-02-24T22:34:25.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1614__shadow_.exJbSNKrsp94TueXe4DCeQ9VwqQUmfT_0 2026-02-24T22:34:25.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2309844739941866388', time=2026-02-25T00:25:59.102310+0000, chain.objs.size()=1 2026-02-24T22:34:25.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.bBdBJUvQh8OX3h3BpLNknosDY1rH3Cj_0 2026-02-24T22:34:25.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6393998891272895140', time=2026-02-25T00:25:59.107505+0000, chain.objs.size()=1 2026-02-24T22:34:25.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.AoL7ID5RJkQyRTw6A83289sPsZpMNoj_0 2026-02-24T22:34:25.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5117813213779906839', time=2026-02-25T00:25:59.110157+0000, chain.objs.size()=1 2026-02-24T22:34:25.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.PYdu5BujmzV4RZ6eg8xK-RlpuUBvQqN_0 2026-02-24T22:34:25.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12488511065200337741', time=2026-02-25T00:25:59.114517+0000, chain.objs.size()=1 2026-02-24T22:34:25.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.UeJMlqA7wZojbhKEdqbRF27cUysMuiz_0 2026-02-24T22:34:25.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4851381944404300292', time=2026-02-25T00:25:59.124814+0000, chain.objs.size()=1 2026-02-24T22:34:25.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.619+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.TprRbweKffwwC6I5HNufxIJ4IJOmXsZ_0 2026-02-24T22:34:25.610 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1272961295757642961', time=2026-02-25T00:25:59.154470+0000, chain.objs.size()=1 2026-02-24T22:34:25.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.TjvHitnk0-I1mnA_xw9XgaRDhZIL6Md_0 2026-02-24T22:34:25.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2998357995571024022', time=2026-02-25T00:25:59.164963+0000, chain.objs.size()=1 2026-02-24T22:34:25.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.cUbl132qZq07_B9Xpv1SOvNK2SYJXL-_0 2026-02-24T22:34:25.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10181415338441041066', time=2026-02-25T00:25:59.289339+0000, chain.objs.size()=1 2026-02-24T22:34:25.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_._AjKe80qfNDgBvOk08Jb0VCzbaShZpY_0 2026-02-24T22:34:25.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5345461505447149470', time=2026-02-25T00:25:59.293889+0000, chain.objs.size()=1 2026-02-24T22:34:25.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.TOWDEmVRaOysBdyAoWTyjGfTYK6taEZ_0 2026-02-24T22:34:25.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10635272651939321721', time=2026-02-25T00:25:59.301674+0000, chain.objs.size()=1 2026-02-24T22:34:25.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.4LTEq4MqN1bbhoa-IOfmiMqJoAt_ycH_0 2026-02-24T22:34:25.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3582507614741525951', time=2026-02-25T00:25:59.303411+0000, chain.objs.size()=1 2026-02-24T22:34:25.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.QChWvE38FZRC3631bSL7uVVGYcKHvWS_0 2026-02-24T22:34:25.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13220804379619777442', time=2026-02-25T00:25:59.325117+0000, chain.objs.size()=1 2026-02-24T22:34:25.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.1OGjSdMJ-Zd8xN0-m_2aTRK1zBWne3J_0 2026-02-24T22:34:25.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15766586945515509147', time=2026-02-25T00:25:59.335089+0000, chain.objs.size()=1 2026-02-24T22:34:25.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.L3uphs7FJlOPn_TsALE-4mMVY3_rJ6M_0 2026-02-24T22:34:25.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7780994788982303643', time=2026-02-25T00:25:59.437806+0000, chain.objs.size()=1 2026-02-24T22:34:25.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.2M52u4uyb8mkgzWbGVOWobmBhWpjERu_0 2026-02-24T22:34:25.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8484277474682875395', time=2026-02-25T00:25:59.747502+0000, chain.objs.size()=1 2026-02-24T22:34:25.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.fInrWtIqU86kboRS_AWJjV4mbMxOPL6_0 2026-02-24T22:34:25.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15295720511445733089', time=2026-02-25T00:25:59.785299+0000, chain.objs.size()=1 2026-02-24T22:34:25.611 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.hi27xx4CX0Qe6HU8KZQs6njmzK4t1WI_0 2026-02-24T22:34:25.612 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12585140354222017104', time=2026-02-25T00:25:59.802683+0000, chain.objs.size()=1 2026-02-24T22:34:25.612 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.VHduT7hfk2utSly-FtFt9f86xhxSEHn_0 2026-02-24T22:34:25.612 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5123487917154798222', time=2026-02-25T00:25:59.833220+0000, chain.objs.size()=1 2026-02-24T22:34:25.612 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Z_j9Je44l93oJyBka7fGPHjzHh-MLsL_0 2026-02-24T22:34:25.612 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16789610149042655937', time=2026-02-25T00:25:59.914443+0000, chain.objs.size()=1 2026-02-24T22:34:25.612 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.JDknzJVQdh_AK0Hs-S6abY8FWxLqegx_0 2026-02-24T22:34:25.612 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13770641412645181767', time=2026-02-25T00:25:59.914926+0000, chain.objs.size()=1 2026-02-24T22:34:25.612 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.6Qo7bdIZgbZwdOYDJgZLPaELLe4UgdS_0 2026-02-24T22:34:25.612 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17531864360843881358', time=2026-02-25T00:25:59.931255+0000, chain.objs.size()=1 2026-02-24T22:34:25.612 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Ay4uhygMXKDNlrz0hBS5ngWv5oP-ocZ_0 2026-02-24T22:34:25.612 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3395162028870823941', time=2026-02-25T00:26:00.062244+0000, chain.objs.size()=1 2026-02-24T22:34:25.612 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Z3df6wf-HExYWGTjvLpdeDzP5WjFaFr_0 2026-02-24T22:34:25.612 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12683215701328833007', time=2026-02-25T00:26:00.085317+0000, chain.objs.size()=1 2026-02-24T22:34:25.612 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Be-uth8JCGf0qzb6kIeSJchBhZbSgWt_0 2026-02-24T22:34:25.612 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4624182579116789111', time=2026-02-25T00:26:00.220710+0000, chain.objs.size()=1 2026-02-24T22:34:25.612 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ZxiP2gan1cbWVVZm-DFNMEeC1fwnB14_0 2026-02-24T22:34:25.612 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2097947558785666824', time=2026-02-25T00:26:00.222942+0000, chain.objs.size()=1 2026-02-24T22:34:25.612 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.EOcFCF4U5keeHzLl9oLsKR9pZtgceoE_0 2026-02-24T22:34:25.612 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2833473250084655190', time=2026-02-25T00:26:00.236696+0000, chain.objs.size()=1 2026-02-24T22:34:25.612 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.eeGU-wt6eEo63Ct7LWhZ_7iGuCHmTvT_0 2026-02-24T22:34:25.612 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17547256115071603958', time=2026-02-25T00:26:00.285781+0000, chain.objs.size()=1 2026-02-24T22:34:25.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.qsh-_MNdog12aknPAwIkxNlQX1L73zQ_0 2026-02-24T22:34:25.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6194825023231378182', time=2026-02-25T00:26:00.286902+0000, chain.objs.size()=1 2026-02-24T22:34:25.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.wPEVpNRkcsA5_NfEw-FXReTPSoBhL7d_0 2026-02-24T22:34:25.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11547568964710142874', time=2026-02-25T00:26:00.790156+0000, chain.objs.size()=1 2026-02-24T22:34:25.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1629__shadow_.TbAQef3qKBtKiOeX-DzlX1fg6P88TVM_0 2026-02-24T22:34:25.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.251902161784920365', time=2026-02-25T00:26:02.513187+0000, chain.objs.size()=1 2026-02-24T22:34:25.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1632__shadow_.YGzAnGaPryVmVBEUEDIhwt3uAjmG1t8_0 2026-02-24T22:34:25.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8048143420146766187', time=2026-02-25T00:26:04.893428+0000, chain.objs.size()=1 2026-02-24T22:34:25.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1636__shadow_.tHEcqk8_8LoiILbkzHsaaO3iRRIuR7h_0 2026-02-24T22:34:25.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11158033732961582080', time=2026-02-25T00:26:08.849703+0000, chain.objs.size()=1 2026-02-24T22:34:25.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1643__shadow_.W0vCHGKV7NklDMFO4YQWLHcwUtek-RU_0 2026-02-24T22:34:25.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14900013810074974787', time=2026-02-25T00:26:11.801733+0000, chain.objs.size()=1 2026-02-24T22:34:25.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1648__shadow_.aEhAl1QQU6IB6yJ1a1Uvap5mIhF6BKI_0 2026-02-24T22:34:25.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13288761606254040636', time=2026-02-25T00:26:14.818601+0000, chain.objs.size()=1 2026-02-24T22:34:25.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1653__shadow_.VktrVEOKGCsh9AzBxDrNCNT4hcWhWvi_0 2026-02-24T22:34:25.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10660603940214323597', time=2026-02-25T00:26:18.511117+0000, chain.objs.size()=1 2026-02-24T22:34:25.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1658__shadow_.OlZIBEuhWrr4ds8OPPXKtRRRT1zQyuM_0 2026-02-24T22:34:25.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16708183715813835582', time=2026-02-25T00:26:19.462774+0000, chain.objs.size()=1 2026-02-24T22:34:25.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1659__shadow_.CKhpOERb9aMdn4cAzY7DydlLPuxmQEZ_0 2026-02-24T22:34:25.613 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13266893808857339285', time=2026-02-25T00:26:35.769519+0000, chain.objs.size()=1 2026-02-24T22:34:25.614 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:FaTO98Y5u9hZ33qDBkv7Mwd0mJDX9yQ_.xVEknzvxGsSpHXykZ1zz3bGwymID4mG_0 2026-02-24T22:34:25.614 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13266893808857339285', time=2026-02-25T00:26:35.796272+0000, chain.objs.size()=1 2026-02-24T22:34:25.614 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:FaTO98Y5u9hZ33qDBkv7Mwd0mJDX9yQ_.xVEknzvxGsSpHXykZ1zz3bGwymID4mG_0 2026-02-24T22:34:25.614 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13266893808857339285', time=2026-02-25T00:26:35.843102+0000, chain.objs.size()=1 2026-02-24T22:34:25.614 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:FaTO98Y5u9hZ33qDBkv7Mwd0mJDX9yQ_.xVEknzvxGsSpHXykZ1zz3bGwymID4mG_0 2026-02-24T22:34:25.614 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13266893808857339285', time=2026-02-25T00:26:35.873496+0000, chain.objs.size()=1 2026-02-24T22:34:25.614 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:FaTO98Y5u9hZ33qDBkv7Mwd0mJDX9yQ_.xVEknzvxGsSpHXykZ1zz3bGwymID4mG_0 2026-02-24T22:34:25.614 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13266893808857339285', time=2026-02-25T00:26:35.887331+0000, chain.objs.size()=1 2026-02-24T22:34:25.614 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:FaTO98Y5u9hZ33qDBkv7Mwd0mJDX9yQ_.xVEknzvxGsSpHXykZ1zz3bGwymID4mG_0 2026-02-24T22:34:25.614 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8799363940264925396', time=2026-02-25T00:26:37.388475+0000, chain.objs.size()=1 2026-02-24T22:34:25.614 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1686__shadow_.e7wyFsqr82EZaqSe64WgF86J8FtSirV_0 2026-02-24T22:34:25.614 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16204260422866432717', time=2026-02-25T00:26:44.831496+0000, chain.objs.size()=1 2026-02-24T22:34:25.614 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.8Ua5JFUmMP07vzeeQkYyvOP9SjUbeBv_0 2026-02-24T22:34:25.614 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12765324749789179819', time=2026-02-25T00:26:44.861342+0000, chain.objs.size()=1 2026-02-24T22:34:25.614 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.oGCKovO49SG2PIYQGfnrJ-YhdcvdIoh_0 2026-02-24T22:34:25.614 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.451972832458161900', time=2026-02-25T00:26:44.865546+0000, chain.objs.size()=1 2026-02-24T22:34:25.614 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.623+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.G9dBhen_larWvy2L_rOG3JnnO9fDAjU_0 2026-02-24T22:34:25.614 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10239694439815667315', time=2026-02-25T00:26:44.875028+0000, chain.objs.size()=1 2026-02-24T22:34:25.614 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.p8qLGfu06o0bHVaWw0gInYraE86Cu9v_0 2026-02-24T22:34:25.614 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7482232144364639532', time=2026-02-25T00:26:44.879893+0000, chain.objs.size()=1 2026-02-24T22:34:25.614 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.FoVccGNs25dD_5cbI61bZ_B-AhxVpb6_0 2026-02-24T22:34:25.615 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17781318932609831896', time=2026-02-25T00:26:44.963766+0000, chain.objs.size()=1 2026-02-24T22:34:25.615 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.fkKR7YdhbxCEl_OqQxnWKhRJrsGi7RA_0 2026-02-24T22:34:25.615 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16095668280705534808', time=2026-02-25T00:26:44.991098+0000, chain.objs.size()=1 2026-02-24T22:34:25.615 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.BSHj9_TLpg5hfseLgGM69PYtqR5XNnZ_0 2026-02-24T22:34:25.615 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13252186559661541782', time=2026-02-25T00:26:45.150349+0000, chain.objs.size()=1 2026-02-24T22:34:25.615 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.gFL8aLyvb3TUux5AzUGS8DuOApXm4m3_0 2026-02-24T22:34:25.615 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7610915179769795813', time=2026-02-25T00:26:45.328298+0000, chain.objs.size()=1 2026-02-24T22:34:25.615 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ljIzE7cXHeix-27POaOhVgHgGUGmvyG_0 2026-02-24T22:34:25.615 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4881862247270769616', time=2026-02-25T00:26:45.329780+0000, chain.objs.size()=1 2026-02-24T22:34:25.615 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Knu4I0z5rS8-LxWtXMug8IMjmoQkLr__0 2026-02-24T22:34:25.616 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3350051211294760516', time=2026-02-25T00:26:45.333630+0000, chain.objs.size()=1 2026-02-24T22:34:25.616 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.8AiFabRzds2-J1aW9Na4SPiiJMAAsHd_0 2026-02-24T22:34:25.616 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13400827893042986403', time=2026-02-25T00:26:45.335947+0000, chain.objs.size()=1 2026-02-24T22:34:25.616 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.aT-EHS1he_cM5v_yQFiDWJ2Bhcm4n-1_0 2026-02-24T22:34:25.616 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4778070047797688105', time=2026-02-25T00:26:45.418755+0000, chain.objs.size()=1 2026-02-24T22:34:25.616 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.woy4Ju50IUbdWhBFnqU2vDtwS9yDbJG_0 2026-02-24T22:34:25.616 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7629446085734901895', time=2026-02-25T00:26:45.419372+0000, chain.objs.size()=1 2026-02-24T22:34:25.616 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.AQblwxd-8sqf07z2lHZoMfhoku1_lBG_0 2026-02-24T22:34:25.616 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11214232986589861546', time=2026-02-25T00:26:45.453280+0000, chain.objs.size()=1 2026-02-24T22:34:25.616 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.G5IyN18WLXi3n_zc8ARpfsqWsYeQ6JU_0 2026-02-24T22:34:25.616 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10970898557037781363', time=2026-02-25T00:26:45.487956+0000, chain.objs.size()=1 2026-02-24T22:34:25.616 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ifLX7NndV1C0Byqebv7ACTGre4sZtEf_0 2026-02-24T22:34:25.616 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11840255763605780661', time=2026-02-25T00:26:45.596968+0000, chain.objs.size()=1 2026-02-24T22:34:25.616 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.RiqFIz2L_lmLfhYGC4Yzi1NYQ9BcinQ_0 2026-02-24T22:34:25.616 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3921572705308159064', time=2026-02-25T00:26:45.608051+0000, chain.objs.size()=1 2026-02-24T22:34:25.616 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.6lxDCUdbxiMxWhIwHFJA8KqpDN051sD_0 2026-02-24T22:34:25.616 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4244604873174302826', time=2026-02-25T00:26:45.835024+0000, chain.objs.size()=1 2026-02-24T22:34:25.616 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.PhSCv-3zq0m9aXxX_BQMqpUCvwv9WH1_0 2026-02-24T22:34:25.616 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15432325508517458410', time=2026-02-25T00:26:45.840672+0000, chain.objs.size()=1 2026-02-24T22:34:25.617 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Rb_P5DUQJoLgqUHFcYC9XvxRB2rMDQD_0 2026-02-24T22:34:25.617 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14539751406113107783', time=2026-02-25T00:26:45.852880+0000, chain.objs.size()=1 2026-02-24T22:34:25.617 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Z5VJ2IfVA_iR3636m1yGuByC09WWTJY_0 2026-02-24T22:34:25.617 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11832001937956673654', time=2026-02-25T00:26:45.943685+0000, chain.objs.size()=1 2026-02-24T22:34:25.617 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.1cZn3oFtiRhQtvWqw6sb4kP_KPx29vj_0 2026-02-24T22:34:25.617 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14979307622881299531', time=2026-02-25T00:26:46.107963+0000, chain.objs.size()=1 2026-02-24T22:34:25.617 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.qi9Lyfl4TWGGdjrhGaX-y-0FdYcYtEg_0 2026-02-24T22:34:25.617 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17569593907974981142', time=2026-02-25T00:26:46.113405+0000, chain.objs.size()=1 2026-02-24T22:34:25.617 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.RH5FWvRbHoSlJxYoIKRw6vCz-MkWzlv_0 2026-02-24T22:34:25.617 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2660520414115892480', time=2026-02-25T00:26:46.122313+0000, chain.objs.size()=1 2026-02-24T22:34:25.617 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.r8j-_olsVAudMuga8SN5eZT5N3G7xZC_0 2026-02-24T22:34:25.617 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5349945280233238878', time=2026-02-25T00:26:53.156028+0000, chain.objs.size()=1 2026-02-24T22:34:25.617 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.YzZAF9BE_dAuu9p8NJkIV7dT32PulRE_0 2026-02-24T22:34:25.617 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8159381703776016944', time=2026-02-25T00:26:53.161741+0000, chain.objs.size()=1 2026-02-24T22:34:25.617 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.9kDfbPjSgiCBp8RMuIy8dLqZ6scSbW-_0 2026-02-24T22:34:25.617 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17485918092576535113', time=2026-02-25T00:26:53.243789+0000, chain.objs.size()=1 2026-02-24T22:34:25.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.sadoink5NixKRtjqX-PwDQO5qtOF1gV_0 2026-02-24T22:34:25.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1143375537061996376', time=2026-02-25T00:26:53.247853+0000, chain.objs.size()=1 2026-02-24T22:34:25.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.yplzG-LBpevL5y8Wuau0KZWJ-hOe-oK_0 2026-02-24T22:34:25.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14192309539053915248', time=2026-02-25T00:26:53.357357+0000, chain.objs.size()=1 2026-02-24T22:34:25.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.LCAvUZH7-lEVQg2h5_giMjdqaFsq3ZR_0 2026-02-24T22:34:25.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.992097043059110159', time=2026-02-25T00:26:53.369783+0000, chain.objs.size()=1 2026-02-24T22:34:25.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.yNMmJY5hd9-LBfcisacjZtu8HWOerb4_0 2026-02-24T22:34:25.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12251678910535269704', time=2026-02-25T00:26:53.370768+0000, chain.objs.size()=1 2026-02-24T22:34:25.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.-uj5zGkdsZJhX8s-HLMlagsHsZH5rfH_0 2026-02-24T22:34:25.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9176479708737932166', time=2026-02-25T00:26:53.378419+0000, chain.objs.size()=1 2026-02-24T22:34:25.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.5FDs6pgB5a9e4LyU4YXeNaEzNHjlwQD_0 2026-02-24T22:34:25.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1228682782784681905', time=2026-02-25T00:26:53.424190+0000, chain.objs.size()=1 2026-02-24T22:34:25.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.9DDXBaYcL8NQitjejSjUznw7ne83lYt_0 2026-02-24T22:34:25.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2295817269129496929', time=2026-02-25T00:26:53.527311+0000, chain.objs.size()=1 2026-02-24T22:34:25.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.udqkzd2Kx9SquQAbpzsut2r3xiUnZO-_0 2026-02-24T22:34:25.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4648462605789984385', time=2026-02-25T00:26:53.549937+0000, chain.objs.size()=1 2026-02-24T22:34:25.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ExNm5nl7fdBEo5kmoJtl9QI3iiqWmwa_0 2026-02-24T22:34:25.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13140197838441457922', time=2026-02-25T00:26:53.571947+0000, chain.objs.size()=1 2026-02-24T22:34:25.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.627+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.SC_gbvO_Xrx7fXxouG1tJm7jtwoeCqH_0 2026-02-24T22:34:25.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1232439426802354619', time=2026-02-25T00:26:53.628739+0000, chain.objs.size()=1 2026-02-24T22:34:25.618 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.qVB6nNBIryvYyhCyupQve6cwfxJIk20_0 2026-02-24T22:34:25.619 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9340640950584149422', time=2026-02-25T00:26:53.758845+0000, chain.objs.size()=1 2026-02-24T22:34:25.619 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.MqEvY0R7xo4hq7pI9JdE54akwWpCpIO_0 2026-02-24T22:34:25.619 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15487015062884676681', time=2026-02-25T00:26:53.805855+0000, chain.objs.size()=1 2026-02-24T22:34:25.619 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.u35CA-_cPPue-kcNV5zz2-M_wqJR1UK_0 2026-02-24T22:34:25.619 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15673223558002950232', time=2026-02-25T00:26:53.830267+0000, chain.objs.size()=1 2026-02-24T22:34:25.619 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.hY4hhjNDWdSr3ivLpk-hGXJWdDDoBkb_0 2026-02-24T22:34:25.619 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2822493297165339703', time=2026-02-25T00:26:53.841513+0000, chain.objs.size()=1 2026-02-24T22:34:25.619 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_._SLAaxRAHUNCi3WtK9SV4B2dGH0Eyn7_0 2026-02-24T22:34:25.619 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17499244842581025260', time=2026-02-25T00:26:53.957852+0000, chain.objs.size()=1 2026-02-24T22:34:25.619 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.VvUb8zj6Ztd_Cb7KdEdnSVRbh-JGF5w_0 2026-02-24T22:34:25.620 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/38987 2026-02-24T22:34:25.621 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=36, truncated=0, next_marker='' 2026-02-24T22:34:25.621 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.702639803522426701', time=2026-02-25T00:26:53.999877+0000, chain.objs.size()=1 2026-02-24T22:34:25.621 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.taFgxnsLEEYKDg3tHwqZyCgI1lHRno1_0 2026-02-24T22:34:25.621 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3068666301406365158', time=2026-02-25T00:26:54.012161+0000, chain.objs.size()=1 2026-02-24T22:34:25.621 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.jBM-bQWYY2SNEa7YEMycPRE_0NHZjIn_0 2026-02-24T22:34:25.621 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14182143879651419625', time=2026-02-25T00:26:54.038632+0000, chain.objs.size()=1 2026-02-24T22:34:25.621 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.rfWdYJEHq2ILXCw53DAYr4Eh8NEhAmT_0 2026-02-24T22:34:25.621 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2340817838327304240', time=2026-02-25T00:26:54.041480+0000, chain.objs.size()=1 2026-02-24T22:34:25.621 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.rsuYzy4wKsZCPp6lfzMb-azNqQ1xqnS_0 2026-02-24T22:34:25.621 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4627547203369234422', time=2026-02-25T00:26:54.116433+0000, chain.objs.size()=1 2026-02-24T22:34:25.621 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_._LgDdStoWYXykqEPZrA_wlcOIuwKfyp_0 2026-02-24T22:34:25.621 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16620957972643855778', time=2026-02-25T00:26:54.120353+0000, chain.objs.size()=1 2026-02-24T22:34:25.621 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.uzYQKXjGUrt0nMZPlZsVNin1cJefWHV_0 2026-02-24T22:34:25.621 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5443847509969496353', time=2026-02-25T00:26:54.144245+0000, chain.objs.size()=1 2026-02-24T22:34:25.622 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.LHjcAznkaENz_rDXxaDQPZGnYQv2HKz_0 2026-02-24T22:34:25.622 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3123261778073738405', time=2026-02-25T00:26:54.152858+0000, chain.objs.size()=1 2026-02-24T22:34:25.622 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.F7e7PC9vXh1hFejmdl5pz6Xmzd7MPYz_0 2026-02-24T22:34:25.622 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11706305555986715613', time=2026-02-25T00:26:54.165236+0000, chain.objs.size()=1 2026-02-24T22:34:25.622 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.mStXcXKzuftrgFOcgnbsdHJ8tPutMuE_0 2026-02-24T22:34:25.622 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17950933771184282848', time=2026-02-25T00:26:54.301305+0000, chain.objs.size()=1 2026-02-24T22:34:25.622 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.SOmTjrSNCd82EJoPLGp7oOigonG3bIz_0 2026-02-24T22:34:25.622 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8600593997731147273', time=2026-02-25T00:26:54.316529+0000, chain.objs.size()=1 2026-02-24T22:34:25.622 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.QSz-3S3eqclTMSGhvnjuzhl0dlGO0JF_0 2026-02-24T22:34:25.622 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1197540192925974222', time=2026-02-25T00:26:54.320221+0000, chain.objs.size()=1 2026-02-24T22:34:25.622 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.631+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.l8Ik_NjNIn6-_GMESTFgyM0IMj7jcqQ_0 2026-02-24T22:34:25.622 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2988026304318488351', time=2026-02-25T00:26:54.329518+0000, chain.objs.size()=1 2026-02-24T22:34:25.622 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.3WJF3mlmoi2P2fL20pNXVqd7zOOXC3F_0 2026-02-24T22:34:25.622 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7361931247760657008', time=2026-02-25T00:26:54.342577+0000, chain.objs.size()=1 2026-02-24T22:34:25.622 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.P_xec2dp4pJViKMTM9Ogk9Bqxr2pMET_0 2026-02-24T22:34:25.622 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16641805872682849786', time=2026-02-25T00:26:54.376927+0000, chain.objs.size()=1 2026-02-24T22:34:25.623 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.0lAKOgdAvlh0BILZmJckCHZFJXD96Ee_0 2026-02-24T22:34:25.623 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8354362287270221351', time=2026-02-25T00:26:54.399073+0000, chain.objs.size()=1 2026-02-24T22:34:25.623 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Pj8MjDMi8ozud5mxzg_6GunHrHI0qAF_0 2026-02-24T22:34:25.623 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5684806387405319400', time=2026-02-25T00:26:54.481765+0000, chain.objs.size()=1 2026-02-24T22:34:25.623 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.i6xN3k9OtNw5m5xhh5qoKDVCzgk-Dn__0 2026-02-24T22:34:25.623 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4155821335323099633', time=2026-02-25T00:26:54.487533+0000, chain.objs.size()=1 2026-02-24T22:34:25.623 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.kgAP8tX993sO2BW0SL7kHQ80kKRimVL_0 2026-02-24T22:34:25.623 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16780675344557384465', time=2026-02-25T00:26:54.489640+0000, chain.objs.size()=1 2026-02-24T22:34:25.623 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.imy5eInKoFOhMKvAA19UZVStLkqdWAu_0 2026-02-24T22:34:25.623 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17651903429897633216', time=2026-02-25T00:26:54.497847+0000, chain.objs.size()=1 2026-02-24T22:34:25.623 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.aH9hD9vsPg5q6KH-3r1z2q_cWYQGCzF_0 2026-02-24T22:34:25.623 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11465135532011335894', time=2026-02-25T00:26:54.499297+0000, chain.objs.size()=1 2026-02-24T22:34:25.623 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.OpOFfMXeznifPTbzYk4bqFgmX-ZRuL9_0 2026-02-24T22:34:25.623 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11274042993792522365', time=2026-02-25T00:26:54.510783+0000, chain.objs.size()=1 2026-02-24T22:34:25.623 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.0JJwzxZ5HX1U04Tav8Er185YZ6cxzB3_0 2026-02-24T22:34:25.623 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11455838871552216663', time=2026-02-25T00:27:32.493328+0000, chain.objs.size()=1 2026-02-24T22:34:25.623 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1755__shadow_.U6ri-pHnbs350A_-BlHgZWr41e33YBj_0 2026-02-24T22:34:25.623 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18247305960032546003', time=2026-02-25T00:28:03.597031+0000, chain.objs.size()=1 2026-02-24T22:34:25.623 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1801__shadow_.5r88Rdx_56jxKLc7Uz-jEECCXTfTjxg_0 2026-02-24T22:34:25.623 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4721670402312120591', time=2026-02-25T00:28:21.289498+0000, chain.objs.size()=1 2026-02-24T22:34:25.623 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1826__shadow_.4HTgSHpd3BmYzFf99LbL1AJiFeEStm7_0 2026-02-24T22:34:25.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6614973634154479804', time=2026-02-25T00:28:37.569013+0000, chain.objs.size()=1 2026-02-24T22:34:25.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1857__shadow_.cPXipHhHnh1JBJq4IDc7MCM1gfiBBdW_0 2026-02-24T22:34:25.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12592958698991728894', time=2026-02-25T00:28:50.217572+0000, chain.objs.size()=1 2026-02-24T22:34:25.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1881__shadow_._sWc7rfDXscAaUrN706HcOMfYb3juZ-_0 2026-02-24T22:34:25.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='2~Ar_zwvzAj5m_J2ri0hcQ1t2g2U1FbFa', time=2026-02-25T00:28:58.801889+0000, chain.objs.size()=1 2026-02-24T22:34:25.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~Ar_zwvzAj5m_J2ri0hcQ1t2g2U1FbFa.1 2026-02-24T22:34:25.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16876295845241199854', time=2026-02-25T00:29:01.164827+0000, chain.objs.size()=6 2026-02-24T22:34:25.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.oY9mejaV72AbcAnOJaNnDPiK0ttWuDMP.1 2026-02-24T22:34:25.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~Ar_zwvzAj5m_J2ri0hcQ1t2g2U1FbFa.2 2026-02-24T22:34:25.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~Ar_zwvzAj5m_J2ri0hcQ1t2g2U1FbFa.3 2026-02-24T22:34:25.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~Ar_zwvzAj5m_J2ri0hcQ1t2g2U1FbFa.4 2026-02-24T22:34:25.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~Ar_zwvzAj5m_J2ri0hcQ1t2g2U1FbFa.5 2026-02-24T22:34:25.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~Ar_zwvzAj5m_J2ri0hcQ1t2g2U1FbFa.6 2026-02-24T22:34:25.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1745070973674739721', time=2026-02-25T00:29:11.121321+0000, chain.objs.size()=3 2026-02-24T22:34:25.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1889__multipart_mymultipart.2~COjaHKe7ca8G-bAW2c0v0YbSc1UVk-2.1 2026-02-24T22:34:25.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1889__multipart_mymultipart.2~COjaHKe7ca8G-bAW2c0v0YbSc1UVk-2.2 2026-02-24T22:34:25.624 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1889__multipart_mymultipart.2~COjaHKe7ca8G-bAW2c0v0YbSc1UVk-2.3 2026-02-24T22:34:25.625 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9837651826123006677', time=2026-02-25T00:30:09.140109+0000, chain.objs.size()=1 2026-02-24T22:34:25.625 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1927__shadow_.lNRohG0jIlZMA_rXEpTDkuRVoMIRcNl_0 2026-02-24T22:34:25.625 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11898876575252505076', time=2026-02-25T00:30:21.102196+0000, chain.objs.size()=1 2026-02-24T22:34:25.625 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:NBByLTSljbiTBJvN00VPbo3c7veJItI_.4fWZsUl4b2hfZ5WDe3N2MbJz7uMrOD1_0 2026-02-24T22:34:25.625 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13487377491400304639', time=2026-02-25T00:30:44.334728+0000, chain.objs.size()=1 2026-02-24T22:34:25.625 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:wNrJlpYBbxX-9bPEQZrJ7qgPOBL6S0L_.v4OoglWWY5cWSm-7uwpSeLK56pXa8Vh_0 2026-02-24T22:34:25.625 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13375772287991154755', time=2026-02-25T00:30:57.204334+0000, chain.objs.size()=1 2026-02-24T22:34:25.625 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:hoX5.tpDIkf-QMpvxJQsYn5KL0E9nIy_.nXZe9meBg115tEtuEefuOZZOwiS8VYS_0 2026-02-24T22:34:25.625 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13375772287991154755', time=2026-02-25T00:30:57.204945+0000, chain.objs.size()=1 2026-02-24T22:34:25.625 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:hoX5.tpDIkf-QMpvxJQsYn5KL0E9nIy_.nXZe9meBg115tEtuEefuOZZOwiS8VYS_0 2026-02-24T22:34:25.625 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5804941479794326700', time=2026-02-25T00:33:33.154635+0000, chain.objs.size()=1 2026-02-24T22:34:25.625 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1959__shadow_.JW3LeIcIBnHIuYHP6ZqQ6r7wmWMfaeI_0 2026-02-24T22:34:25.626 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.635+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.628 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2026-02-24T22:34:25.629 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.629 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/37493' 2026-02-24T22:34:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3246403053267063527', time=2026-02-25T00:15:34.615303+0000, chain.objs.size()=1 2026-02-24T22:34:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506__shadow_.-n722HI8LwjS-atk1OepQ7HDrxQWHgl_0 2026-02-24T22:34:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='_RhxOYsClOISq4Qa0MqZpqN0V19U2jYQ', time=2026-02-25T00:15:34.721491+0000, chain.objs.size()=1 2026-02-24T22:34:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516__shadow_.-XjTT5X1d7UuY3Uf7qRWM3a15gOiNVj_0 2026-02-24T22:34:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='_wRdzuuKc7-rNqVuziPyNzmkO3KRpsXv', time=2026-02-25T00:15:34.724329+0000, chain.objs.size()=1 2026-02-24T22:34:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516__shadow_.qgVdRUKpPtJp93cwY89LYuQAR3QEpgp_0 2026-02-24T22:34:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15953924534094524185', time=2026-02-25T00:15:34.857451+0000, chain.objs.size()=1 2026-02-24T22:34:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1509__shadow_.YoLUFnLK5ZRrNRyZOMBdYRic6VB-wPW_0 2026-02-24T22:34:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4385877069488326450', time=2026-02-25T00:16:33.508541+0000, chain.objs.size()=1 2026-02-24T22:34:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521__shadow_.EgayUZRQ1nPhzDtmAX6qX3oF4V2s_27_0 2026-02-24T22:34:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14129752243111565281', time=2026-02-25T00:16:33.510762+0000, chain.objs.size()=1 2026-02-24T22:34:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521__shadow_.-zljqiqVuf4Kpn3ICNG6Dvq7d1fnQ5r_0 2026-02-24T22:34:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4394780904869565209', time=2026-02-25T00:17:09.124043+0000, chain.objs.size()=1 2026-02-24T22:34:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525__shadow_.OqZTi12wm1giq_CQ4W46vqKxTzEr3sz_0 2026-02-24T22:34:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10616542865957263309', time=2026-02-25T00:17:41.421148+0000, chain.objs.size()=1 2026-02-24T22:34:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526__shadow_.OgLeSW-ZxuYm8MFGsmIUyoGrCXsF8ko_0 2026-02-24T22:34:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6292327844569873414', time=2026-02-25T00:18:14.572251+0000, chain.objs.size()=1 2026-02-24T22:34:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528__shadow_.LEk02YYiRtJgIe21hbFH-OwCg_tOU9L_0 2026-02-24T22:34:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11901194632339134043', time=2026-02-25T00:18:48.186097+0000, chain.objs.size()=1 2026-02-24T22:34:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530__shadow_.o2HFXprlWUicIO5AWnLiuHwpZMHuL2b_0 2026-02-24T22:34:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4848279605119698942', time=2026-02-25T00:19:57.108254+0000, chain.objs.size()=1 2026-02-24T22:34:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535__shadow_.rXqKOe7cz-ipb-uG3CtBTmrW3V8sHdR_0 2026-02-24T22:34:25.630 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14892874376098524835', time=2026-02-25T00:19:57.109095+0000, chain.objs.size()=1 2026-02-24T22:34:25.631 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.639+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535__shadow_.uByU0U1ypekNiahxkngbuyGoPBop_nr_0 2026-02-24T22:34:25.631 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11525317928677583243', time=2026-02-25T00:19:58.455389+0000, chain.objs.size()=1 2026-02-24T22:34:25.631 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536__shadow_.4wuy3Y6luVkogjp3xAw1vM82Sq7eGKI_0 2026-02-24T22:34:25.631 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16020288955357964', time=2026-02-25T00:20:30.731434+0000, chain.objs.size()=1 2026-02-24T22:34:25.631 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537__shadow_.AioNP5xYXVaPodHxOEtYeCjB64mR0av_0 2026-02-24T22:34:25.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11958674519945121394', time=2026-02-25T00:20:32.508012+0000, chain.objs.size()=1 2026-02-24T22:34:25.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538__shadow_.vWl7r9EJwD8YNDDiJutW3dOO3VOPmx4_0 2026-02-24T22:34:25.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1714995831128769707', time=2026-02-25T00:20:34.029666+0000, chain.objs.size()=1 2026-02-24T22:34:25.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539__shadow_.X-dDF9CS5ShX2YUQoSF8iuqW89UaHgA_0 2026-02-24T22:34:25.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9381365818462258948', time=2026-02-25T00:25:36.599226+0000, chain.objs.size()=1 2026-02-24T22:34:25.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1602__shadow_.XYfJvlfTPTBdaQY10GPO3zjQgupY-pk_0 2026-02-24T22:34:25.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13870723497992922522', time=2026-02-25T00:25:40.155238+0000, chain.objs.size()=1 2026-02-24T22:34:25.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1607__shadow_._vZRuKcGG_J7cF6clfwCdF_0iTf5ZQC_0 2026-02-24T22:34:25.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8715713793939394076', time=2026-02-25T00:25:59.093505+0000, chain.objs.size()=1 2026-02-24T22:34:25.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.c1j9-DXEwgG1J1hKsg8VTorp4Kk5cCf_0 2026-02-24T22:34:25.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.77853660347450760', time=2026-02-25T00:25:59.094239+0000, chain.objs.size()=1 2026-02-24T22:34:25.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_._rYPZZmRMdA-OOx2c_Zt4HeV9X45ExT_0 2026-02-24T22:34:25.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6790019655994919686', time=2026-02-25T00:25:59.137964+0000, chain.objs.size()=1 2026-02-24T22:34:25.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.R9IYvHAzX3-OvHid3T-samj62bP280G_0 2026-02-24T22:34:25.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5974830302919072751', time=2026-02-25T00:25:59.165401+0000, chain.objs.size()=1 2026-02-24T22:34:25.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.avH4UV6DOKZpyJXO46bB1x-osHzoxrT_0 2026-02-24T22:34:25.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10895558621133251018', time=2026-02-25T00:25:59.242218+0000, chain.objs.size()=1 2026-02-24T22:34:25.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ebfzvb1vGzvXOb8uMpGkeaNFr8njYNT_0 2026-02-24T22:34:25.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8288485941330039639', time=2026-02-25T00:25:59.242777+0000, chain.objs.size()=1 2026-02-24T22:34:25.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ktGIHfSDjlwVOX9rKWCw52RT1_2-4C1_0 2026-02-24T22:34:25.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12770860469082236689', time=2026-02-25T00:25:59.244192+0000, chain.objs.size()=1 2026-02-24T22:34:25.632 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.uTvUI1x2oPsEOxqhorfg8r8Li5q5hXT_0 2026-02-24T22:34:25.633 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8348745212070003029', time=2026-02-25T00:25:59.244751+0000, chain.objs.size()=1 2026-02-24T22:34:25.633 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.A8iz83jbOtAFYDmsIpeF7l8GsB-hcRL_0 2026-02-24T22:34:25.633 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5614411470729208321', time=2026-02-25T00:25:59.258349+0000, chain.objs.size()=1 2026-02-24T22:34:25.633 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.wX_h3zTLNDHKgYCOMpqEbIlFcs3HT4E_0 2026-02-24T22:34:25.633 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9974678072588625184', time=2026-02-25T00:25:59.311613+0000, chain.objs.size()=1 2026-02-24T22:34:25.633 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.swMwCI0cfX7M7r8HcoD0l5pgFLVntMr_0 2026-02-24T22:34:25.633 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14063836475505567848', time=2026-02-25T00:25:59.411792+0000, chain.objs.size()=1 2026-02-24T22:34:25.633 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.F7wS6XrlvmqtHQHLChVB3-PzZJGG9mT_0 2026-02-24T22:34:25.633 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3115385416431788367', time=2026-02-25T00:25:59.412260+0000, chain.objs.size()=1 2026-02-24T22:34:25.634 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.T6LXv7RqjC4ddqf2F0xb3DLsXrr6Bph_0 2026-02-24T22:34:25.634 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6278314884418081682', time=2026-02-25T00:25:59.441780+0000, chain.objs.size()=1 2026-02-24T22:34:25.634 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.OeOu0e0f-P1FmOvm9iIS8T27jWObsdn_0 2026-02-24T22:34:25.634 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7183589907949891484', time=2026-02-25T00:25:59.486990+0000, chain.objs.size()=1 2026-02-24T22:34:25.634 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.x7kbsmTKYPlxpT5jEitCwnYNZ-lVk9z_0 2026-02-24T22:34:25.634 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13336677604887005183', time=2026-02-25T00:25:59.630856+0000, chain.objs.size()=1 2026-02-24T22:34:25.634 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.qwoWZqFGlkURcV4j93ybgHPpJkxypML_0 2026-02-24T22:34:25.634 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17282847975807896817', time=2026-02-25T00:25:59.636604+0000, chain.objs.size()=1 2026-02-24T22:34:25.634 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.TotB3gC0_L9GnmTcMQzbGU_ePUJTcXA_0 2026-02-24T22:34:25.634 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14134267633399340561', time=2026-02-25T00:25:59.726302+0000, chain.objs.size()=1 2026-02-24T22:34:25.634 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.rkXeQu5mGs497ASs1_yQ18CIK9fgNQs_0 2026-02-24T22:34:25.634 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9695708402772358855', time=2026-02-25T00:25:59.755329+0000, chain.objs.size()=1 2026-02-24T22:34:25.634 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.643+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.BOUUe91cq6PlKCYjFN2QBIntIGDij1s_0 2026-02-24T22:34:25.634 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15902181483494018060', time=2026-02-25T00:25:59.756822+0000, chain.objs.size()=1 2026-02-24T22:34:25.634 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.rvkziMtlnmXi6pbXnD-ccc60blEs9pa_0 2026-02-24T22:34:25.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16795795766397854615', time=2026-02-25T00:25:59.757115+0000, chain.objs.size()=1 2026-02-24T22:34:25.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.7KaEqjHHbP9sv-EiHtdZsyoyKM39IyD_0 2026-02-24T22:34:25.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13203062471763655337', time=2026-02-25T00:25:59.812567+0000, chain.objs.size()=1 2026-02-24T22:34:25.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.NdM9aNkEnCj_LRh9Y5kiKVGo8Gd6Pnp_0 2026-02-24T22:34:25.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6478426464974969673', time=2026-02-25T00:25:59.891448+0000, chain.objs.size()=1 2026-02-24T22:34:25.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.c8IYqz3Ph34vlr4FNEZ4quGD4z9SC-c_0 2026-02-24T22:34:25.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.28282793570302671', time=2026-02-25T00:25:59.912256+0000, chain.objs.size()=1 2026-02-24T22:34:25.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.P2nZeSoqNCsBd3QWZ4eZciXO_5-uFlt_0 2026-02-24T22:34:25.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12558237198695340136', time=2026-02-25T00:25:59.929089+0000, chain.objs.size()=1 2026-02-24T22:34:25.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.z3IuT0zxvd1b2Xwdhc_1nHBPZW7k1dy_0 2026-02-24T22:34:25.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3328540770858457889', time=2026-02-25T00:25:59.964074+0000, chain.objs.size()=1 2026-02-24T22:34:25.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ED2L5AErF0ux_2bGkxQIIceI3g2Y1GE_0 2026-02-24T22:34:25.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7545077596212790751', time=2026-02-25T00:25:59.964738+0000, chain.objs.size()=1 2026-02-24T22:34:25.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ohTF8TVcwbI_p-KOKlA9WfpdZs_G_Wa_0 2026-02-24T22:34:25.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9716366227546241027', time=2026-02-25T00:25:59.965191+0000, chain.objs.size()=1 2026-02-24T22:34:25.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.sd3jbH4ZTeHgI3YmYP4wQT_ILdcyNVC_0 2026-02-24T22:34:25.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11662841029438026448', time=2026-02-25T00:26:00.057186+0000, chain.objs.size()=1 2026-02-24T22:34:25.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.i1D5VAWxT2MFWmaVZCmyJaSOFprO3yl_0 2026-02-24T22:34:25.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17462117823947075010', time=2026-02-25T00:26:00.061686+0000, chain.objs.size()=1 2026-02-24T22:34:25.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.cQ5lEw6R4fCptKJ90Fz0jwyRdJCKthn_0 2026-02-24T22:34:25.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9176356702059467973', time=2026-02-25T00:26:00.072421+0000, chain.objs.size()=1 2026-02-24T22:34:25.635 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.BKsPeUN36gj3c4R4tjfbYxbmWIAHhJE_0 2026-02-24T22:34:25.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11237135411049877367', time=2026-02-25T00:26:00.130028+0000, chain.objs.size()=1 2026-02-24T22:34:25.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.X9tpiFwY3SOKR0_OBbJwzBw_fyLJqHv_0 2026-02-24T22:34:25.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3449328669025203553', time=2026-02-25T00:26:00.135390+0000, chain.objs.size()=1 2026-02-24T22:34:25.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.tq-a9axpMo2xnKiNQvuR_OJ-wwzVOVq_0 2026-02-24T22:34:25.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1446543592026719472', time=2026-02-25T00:26:00.236642+0000, chain.objs.size()=1 2026-02-24T22:34:25.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.m1SnAAeyFVPvfePgT-JwFqW0WPXvDVv_0 2026-02-24T22:34:25.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12323218483607921967', time=2026-02-25T00:26:00.237699+0000, chain.objs.size()=1 2026-02-24T22:34:25.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ydtGVo4_JQWlzdqAYvo9p0jViX9E9CB_0 2026-02-24T22:34:25.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17883631040410614529', time=2026-02-25T00:26:00.267556+0000, chain.objs.size()=1 2026-02-24T22:34:25.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ffaKe5D6_mA_wXneGWL_W4uOFZ1QAFl_0 2026-02-24T22:34:25.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.522175876411298788', time=2026-02-25T00:26:04.877917+0000, chain.objs.size()=1 2026-02-24T22:34:25.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1636__shadow_.zgpKCuGlT3B_JvT0l0ERyoFETl0iF2X_0 2026-02-24T22:34:25.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8878599467761455267', time=2026-02-25T00:26:09.397960+0000, chain.objs.size()=1 2026-02-24T22:34:25.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1644__shadow_.Ht6P1C-niLgq4NcebgdX7Q2lRhAHS9B_0 2026-02-24T22:34:25.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8725284327278818462', time=2026-02-25T00:26:11.186533+0000, chain.objs.size()=1 2026-02-24T22:34:25.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1647__shadow_.RKirI9sRuQn_xKKKn8B-c_WWoIn-6uU_0 2026-02-24T22:34:25.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16203382415158672796', time=2026-02-25T00:26:11.786759+0000, chain.objs.size()=1 2026-02-24T22:34:25.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1648__shadow_.RtNiMvslbchN0CcgxUpu3cwAupxCZa1_0 2026-02-24T22:34:25.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3176218741960378158', time=2026-02-25T00:26:13.602958+0000, chain.objs.size()=1 2026-02-24T22:34:25.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1651__shadow_.O_C1HbTkw3zfMK7SwbREat6lHBaw-gx_0 2026-02-24T22:34:25.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9515199315874187999', time=2026-02-25T00:26:18.494425+0000, chain.objs.size()=1 2026-02-24T22:34:25.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1658__shadow_.aaIHfULplFPPg5l8hf1v9B_K8xV0wTg_0 2026-02-24T22:34:25.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16602290698252483743', time=2026-02-25T00:26:18.496995+0000, chain.objs.size()=1 2026-02-24T22:34:25.636 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1658__shadow_.WAahjVB07AmPGbCay5wzXGuHTNaJgr5_0 2026-02-24T22:34:25.637 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7113378821607558111', time=2026-02-25T00:26:44.978628+0000, chain.objs.size()=1 2026-02-24T22:34:25.637 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.itbeJ1i88meQMlHyYmFnHVg2xXVkhWi_0 2026-02-24T22:34:25.637 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8287926029979396696', time=2026-02-25T00:26:45.004698+0000, chain.objs.size()=1 2026-02-24T22:34:25.637 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.BOfVz4XteJoPvr9EFb0qiCQ_TgiESUt_0 2026-02-24T22:34:25.637 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9118109461001562432', time=2026-02-25T00:26:45.007394+0000, chain.objs.size()=1 2026-02-24T22:34:25.637 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.to3rpzfG43MM0apYhaZOPXSJnOPQJ2m_0 2026-02-24T22:34:25.637 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6465963169067652660', time=2026-02-25T00:26:45.120163+0000, chain.objs.size()=1 2026-02-24T22:34:25.637 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.i9IzJ2URvpNPkmm4vZHCoWXoIvoAG9V_0 2026-02-24T22:34:25.637 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3049796522017259323', time=2026-02-25T00:26:45.129644+0000, chain.objs.size()=1 2026-02-24T22:34:25.637 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.-WHQAkZscBAtpagX64X-07jRmM7iqmK_0 2026-02-24T22:34:25.637 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14478829393110809998', time=2026-02-25T00:26:45.138355+0000, chain.objs.size()=1 2026-02-24T22:34:25.637 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.8_qamp8kl5CEzjedWOdDV3Eq2Ky03Ku_0 2026-02-24T22:34:25.637 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11496694204377846845', time=2026-02-25T00:26:45.169235+0000, chain.objs.size()=1 2026-02-24T22:34:25.637 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.XLW6bA9zXRo_aQo4NxGklWr7kJf2LND_0 2026-02-24T22:34:25.637 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13933125063644922288', time=2026-02-25T00:26:45.178341+0000, chain.objs.size()=1 2026-02-24T22:34:25.638 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.NguOUQ-w4ZXiNjD5w1RQhSepVZvUE6h_0 2026-02-24T22:34:25.638 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16885409154259142398', time=2026-02-25T00:26:45.181018+0000, chain.objs.size()=1 2026-02-24T22:34:25.638 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.KL4wTpA8vewyXL0clzQW6EDqAM2om2j_0 2026-02-24T22:34:25.638 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7086853563883112812', time=2026-02-25T00:26:45.250869+0000, chain.objs.size()=1 2026-02-24T22:34:25.638 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.uzzwMiuaslbnauLwsCdv5bKLkLwyv3B_0 2026-02-24T22:34:25.638 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18395515029490274001', time=2026-02-25T00:26:45.254932+0000, chain.objs.size()=1 2026-02-24T22:34:25.638 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.lrtouQkALr5Qu4arqWwG9MJkDmnXXh8_0 2026-02-24T22:34:25.638 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17355902829495270816', time=2026-02-25T00:26:45.258772+0000, chain.objs.size()=1 2026-02-24T22:34:25.638 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.kNoPZLRbXrvOsydcs6rAD0Kb_bAFyJG_0 2026-02-24T22:34:25.638 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13641480874071765965', time=2026-02-25T00:26:45.264088+0000, chain.objs.size()=1 2026-02-24T22:34:25.638 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Deg7MAHZTo4VPn0AnMdq2BvBlviBvBA_0 2026-02-24T22:34:25.638 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15128634818514955545', time=2026-02-25T00:26:45.267447+0000, chain.objs.size()=1 2026-02-24T22:34:25.638 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.skx1FXd074jXtluK3dHuXmKbxqg54pq_0 2026-02-24T22:34:25.638 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2369276939537297758', time=2026-02-25T00:26:45.416671+0000, chain.objs.size()=1 2026-02-24T22:34:25.638 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.647+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.3k7HxbywpuaZ8anpseAoSsnq9DaOHbu_0 2026-02-24T22:34:25.638 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17453687834540807507', time=2026-02-25T00:26:45.420011+0000, chain.objs.size()=1 2026-02-24T22:34:25.638 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.x7Zcci-rWsbXU1_Za3DQeDs_hmjqwbS_0 2026-02-24T22:34:25.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17846322624560883016', time=2026-02-25T00:26:45.442553+0000, chain.objs.size()=1 2026-02-24T22:34:25.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.XQkTKUaC2RKme17TobaN8a0bTb9TqOz_0 2026-02-24T22:34:25.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11421725650759173767', time=2026-02-25T00:26:45.472101+0000, chain.objs.size()=1 2026-02-24T22:34:25.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.sBuiNAoFVVk5OLBoJDHhzRBj4Xkfxvv_0 2026-02-24T22:34:25.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13286117977361217798', time=2026-02-25T00:26:45.578698+0000, chain.objs.size()=1 2026-02-24T22:34:25.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.wSXMQ6u-f4a2y78OvALxLnQR3PvBvDT_0 2026-02-24T22:34:25.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13319475915802372011', time=2026-02-25T00:26:45.803838+0000, chain.objs.size()=1 2026-02-24T22:34:25.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.6PP9lRKke9fVTuTW89qj_9ITMupm_o7_0 2026-02-24T22:34:25.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6903332968379500694', time=2026-02-25T00:26:45.806333+0000, chain.objs.size()=1 2026-02-24T22:34:25.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.h9qWmoUlWPgs3BYa1FqUsXJS7UPPWSN_0 2026-02-24T22:34:25.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15046466627892116307', time=2026-02-25T00:26:45.811202+0000, chain.objs.size()=1 2026-02-24T22:34:25.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.jq9CRWLtlAgGKQnKY6ME1CjoCPw_3hk_0 2026-02-24T22:34:25.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14234655119081520157', time=2026-02-25T00:26:45.820666+0000, chain.objs.size()=1 2026-02-24T22:34:25.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.bNFs8QQBQCw5bMW8NV-UkU8LggJ5bMv_0 2026-02-24T22:34:25.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11738623908786026185', time=2026-02-25T00:26:45.824558+0000, chain.objs.size()=1 2026-02-24T22:34:25.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.lV9KWGQHiiTxjy2h5J9fyjOr-pVZJ6X_0 2026-02-24T22:34:25.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12583571901496079088', time=2026-02-25T00:26:45.912167+0000, chain.objs.size()=1 2026-02-24T22:34:25.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ozF3G6awNkRRyeNKXCMp8sB8zUs-SEH_0 2026-02-24T22:34:25.639 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8252511784691844680', time=2026-02-25T00:26:45.936306+0000, chain.objs.size()=1 2026-02-24T22:34:25.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.WOXXN4qrg7-YLggQEGsp7ltb301OpP-_0 2026-02-24T22:34:25.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4224727127940006516', time=2026-02-25T00:26:45.967446+0000, chain.objs.size()=1 2026-02-24T22:34:25.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ejHIqaSiL38MtQB59eT0dihg4qo1u5R_0 2026-02-24T22:34:25.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7830928065788433171', time=2026-02-25T00:26:45.991406+0000, chain.objs.size()=1 2026-02-24T22:34:25.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.xVLl5dGTmcvGrVnmizKm71mnEoa8tO6_0 2026-02-24T22:34:25.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4931925656161866391', time=2026-02-25T00:26:46.013419+0000, chain.objs.size()=1 2026-02-24T22:34:25.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.pB7BlfKD-0zajI_4zE05zxIUXRfZb1X_0 2026-02-24T22:34:25.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18209741366706258861', time=2026-02-25T00:26:46.089944+0000, chain.objs.size()=1 2026-02-24T22:34:25.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.MgWkWjHMegMCLFo2zskplWfUGFzAU3u_0 2026-02-24T22:34:25.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.657338766798721089', time=2026-02-25T00:26:46.145498+0000, chain.objs.size()=1 2026-02-24T22:34:25.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_._hvVeJ4WCUuGA3MQRQ4ZVbmNIvlo1w5_0 2026-02-24T22:34:25.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14938110457376028556', time=2026-02-25T00:26:53.144273+0000, chain.objs.size()=1 2026-02-24T22:34:25.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.a4VP_ny_Q-uFXmrqxwDDXOAQ1j5c1T-_0 2026-02-24T22:34:25.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1663415378525079283', time=2026-02-25T00:26:53.199796+0000, chain.objs.size()=1 2026-02-24T22:34:25.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.DDlWM9MVJxXUrW-UPr__HYwWQPLkycw_0 2026-02-24T22:34:25.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14918881491356121235', time=2026-02-25T00:26:53.215847+0000, chain.objs.size()=1 2026-02-24T22:34:25.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.8uhOZ8EBIK2NfGEAlESv-Ueqm6JYzhP_0 2026-02-24T22:34:25.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11495910796212558833', time=2026-02-25T00:26:53.223776+0000, chain.objs.size()=1 2026-02-24T22:34:25.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.FYkBF72llhrM6L5thtL1SNfARzPC7pU_0 2026-02-24T22:34:25.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2191933341322065976', time=2026-02-25T00:26:53.245661+0000, chain.objs.size()=1 2026-02-24T22:34:25.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.FZxVAkiRtqPKM3KsPpbtoOUE87zq2Ob_0 2026-02-24T22:34:25.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1721054651467204647', time=2026-02-25T00:26:53.360536+0000, chain.objs.size()=1 2026-02-24T22:34:25.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.v9ViIAsKnsMvbmdxMBy2B1lOqYa4S5T_0 2026-02-24T22:34:25.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8936000479823304381', time=2026-02-25T00:26:53.406458+0000, chain.objs.size()=1 2026-02-24T22:34:25.640 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.4M2URrEu593qFFxtaFS8Fzxd3J7xqsC_0 2026-02-24T22:34:25.641 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13606347672168514865', time=2026-02-25T00:26:53.511686+0000, chain.objs.size()=1 2026-02-24T22:34:25.641 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.42JhUlsMkMrahTv0k1h_0rAuPOQkmW1_0 2026-02-24T22:34:25.641 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5115625744048341155', time=2026-02-25T00:26:53.721371+0000, chain.objs.size()=1 2026-02-24T22:34:25.641 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Tk6Jl0Mwijqpqnhp-uZRt6U7MNhAtWY_0 2026-02-24T22:34:25.641 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.651+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/37493 2026-02-24T22:34:25.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=27, truncated=0, next_marker='' 2026-02-24T22:34:25.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1676744933165161048', time=2026-02-25T00:26:53.728220+0000, chain.objs.size()=1 2026-02-24T22:34:25.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_._JkuPNou48DUC-pSLjYBGLu7Hfuy1_u_0 2026-02-24T22:34:25.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7606645545138877753', time=2026-02-25T00:26:53.735935+0000, chain.objs.size()=1 2026-02-24T22:34:25.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.6OtN3c1GazgeWyuzPQATLEOABAAmBx__0 2026-02-24T22:34:25.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7622278888872578753', time=2026-02-25T00:26:53.827684+0000, chain.objs.size()=1 2026-02-24T22:34:25.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.G24A3hnZycbAeCCbl1ViFiVgkno4DKw_0 2026-02-24T22:34:25.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13439990853776538657', time=2026-02-25T00:26:53.928968+0000, chain.objs.size()=1 2026-02-24T22:34:25.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.XgKDn5fr4-jfyItWC_ZVzIc9BdNf1E-_0 2026-02-24T22:34:25.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5398556328125369558', time=2026-02-25T00:26:53.958182+0000, chain.objs.size()=1 2026-02-24T22:34:25.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.xB6hIbBgHiWKK277BXT9BVWAkWNtmX5_0 2026-02-24T22:34:25.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1887532404199691097', time=2026-02-25T00:26:53.972573+0000, chain.objs.size()=1 2026-02-24T22:34:25.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.UWN8W09OxIIeUWOmxWxgKEiDmvgny5k_0 2026-02-24T22:34:25.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17169119697615748101', time=2026-02-25T00:26:53.978460+0000, chain.objs.size()=1 2026-02-24T22:34:25.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.779ftmCYTn3G8vhQWz0FLlEWsTE-D1i_0 2026-02-24T22:34:25.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13248869809636695200', time=2026-02-25T00:26:54.015862+0000, chain.objs.size()=1 2026-02-24T22:34:25.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.-JVvNldiucYeiaaX9hXdUb2bTSngTrJ_0 2026-02-24T22:34:25.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10121157594573899759', time=2026-02-25T00:26:54.104595+0000, chain.objs.size()=1 2026-02-24T22:34:25.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.9TEvs4Biur1qW6rMhp2u018AJdRKSfA_0 2026-02-24T22:34:25.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18122067682859238728', time=2026-02-25T00:26:54.138131+0000, chain.objs.size()=1 2026-02-24T22:34:25.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.GzaigQTgnaomU8RE_zffW1nrC-RkFyo_0 2026-02-24T22:34:25.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5473382075111418589', time=2026-02-25T00:26:54.191562+0000, chain.objs.size()=1 2026-02-24T22:34:25.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.PfcWtoooSHPAxcIKkTDl0wY2AsKMU1u_0 2026-02-24T22:34:25.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17000087246131697996', time=2026-02-25T00:26:54.286172+0000, chain.objs.size()=1 2026-02-24T22:34:25.643 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Vx8IiBpy63zZErvraMKCBBupfxLsubu_0 2026-02-24T22:34:25.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17732902291491226753', time=2026-02-25T00:26:54.331206+0000, chain.objs.size()=1 2026-02-24T22:34:25.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.jxYWXYd2-sb6PR1gk9MRMM0JvS4Qkna_0 2026-02-24T22:34:25.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.601662322155758085', time=2026-02-25T00:26:54.377730+0000, chain.objs.size()=1 2026-02-24T22:34:25.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.DplQKiz4iD9hTnyZ280dcFiEAqiKRk9_0 2026-02-24T22:34:25.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14216736042438025552', time=2026-02-25T00:26:54.457678+0000, chain.objs.size()=1 2026-02-24T22:34:25.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.byEdb62l2Kp6IMyo2ug23nYQqGPirMp_0 2026-02-24T22:34:25.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8127715016431704193', time=2026-02-25T00:26:54.464028+0000, chain.objs.size()=1 2026-02-24T22:34:25.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.X7p8MKEW4idmJhKZxhuyUSPkgDoO33V_0 2026-02-24T22:34:25.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6143940180384472124', time=2026-02-25T00:26:54.511684+0000, chain.objs.size()=1 2026-02-24T22:34:25.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.teqO2aG0kLRZvHN9kFqXXurvX9d9LAL_0 2026-02-24T22:34:25.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16305159602071584047', time=2026-02-25T00:27:22.532780+0000, chain.objs.size()=1 2026-02-24T22:34:25.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1739__shadow_.aSD5YuJ-6pkMc3NQQhDO2trLDTxTdwy_0 2026-02-24T22:34:25.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15417314332172532823', time=2026-02-25T00:27:23.120297+0000, chain.objs.size()=1 2026-02-24T22:34:25.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1740__shadow_.XIvsrHKXSkUNKnS71p0A6dvnqXOwNn0_0 2026-02-24T22:34:25.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2674233112901829399', time=2026-02-25T00:28:40.619133+0000, chain.objs.size()=1 2026-02-24T22:34:25.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1864__shadow_.TxpoJDketjixn2OScofAr67xBC4ATX1_0 2026-02-24T22:34:25.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6980323163892259249', time=2026-02-25T00:28:57.926766+0000, chain.objs.size()=3 2026-02-24T22:34:25.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1887__multipart_mymultipart.2~AkCjoZvKPaUy_B8uKFvBBcHG-bQCkh2.1 2026-02-24T22:34:25.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1887__multipart_mymultipart.2~AkCjoZvKPaUy_B8uKFvBBcHG-bQCkh2.2 2026-02-24T22:34:25.644 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1887__multipart_mymultipart.2~AkCjoZvKPaUy_B8uKFvBBcHG-bQCkh2.3 2026-02-24T22:34:25.645 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='2~R8Y0puRshK7Gw6gPvfcVZWeySh08jGk', time=2026-02-25T00:29:19.890613+0000, chain.objs.size()=1 2026-02-24T22:34:25.645 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1899__multipart_mymultipart.2~R8Y0puRshK7Gw6gPvfcVZWeySh08jGk.1 2026-02-24T22:34:25.645 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14288531221752096140', time=2026-02-25T00:30:23.035231+0000, chain.objs.size()=1 2026-02-24T22:34:25.645 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:p8Hv2UHulGrYE936rjEMM0Gvuyob.K5_.-gfp11DPpJ8q_G4Pl5R4cT9OgSn8c2M_0 2026-02-24T22:34:25.645 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15667832230197851723', time=2026-02-25T00:30:30.187481+0000, chain.objs.size()=1 2026-02-24T22:34:25.645 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1943__shadow:K-Sr--ZdmSz.9LE0D6-RU0vfkI5tYNr_.WCmfmNtRlLDe3UGs5EHWsaXZwjftVVS_0 2026-02-24T22:34:25.645 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2271507168859974649', time=2026-02-25T00:30:31.143944+0000, chain.objs.size()=1 2026-02-24T22:34:25.645 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1944__shadow:-0n5w6PFB22jTO-PNK8wyqssCG1.fiN_.Y7C_k1kFuiAw8QC3F1iIWRRKGS4YTLQ_0 2026-02-24T22:34:25.646 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1799607172887108809', time=2026-02-25T00:30:32.755291+0000, chain.objs.size()=1 2026-02-24T22:34:25.646 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1946__shadow:v7DBAROCs28NPEFhChwwM3LHjnDfda9_.3rbTGmdoMHnIfC8cuO5pklI_mAxS1Xa_0 2026-02-24T22:34:25.646 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5671788845253571264', time=2026-02-25T00:30:55.590647+0000, chain.objs.size()=1 2026-02-24T22:34:25.646 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.655+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1955__shadow:ljyVLAh-YUrOTxnxsYEM09ccmt-O-XU_.Jx_EimNPlyHQqY5c-SAYL1Lnpvp8SLm_0 2026-02-24T22:34:25.647 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.659+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.648 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.659+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2026-02-24T22:34:25.649 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.659+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.650 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.659+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.650 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.659+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/38334' 2026-02-24T22:34:25.650 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.659+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10995261666208502301', time=2026-02-25T00:15:34.619540+0000, chain.objs.size()=1 2026-02-24T22:34:25.650 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.659+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506__shadow_.qE75D0gZJzHACOSf5Kud3X72AsgZOrF_0 2026-02-24T22:34:25.650 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.659+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1044560560734127319', time=2026-02-25T00:15:34.634747+0000, chain.objs.size()=1 2026-02-24T22:34:25.650 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.659+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506__shadow_.t8VES_JBHmVWgluUpXu3wfjP4xUnpqp_0 2026-02-24T22:34:25.650 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.659+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9358090815558633539', time=2026-02-25T00:15:34.725820+0000, chain.objs.size()=1 2026-02-24T22:34:25.650 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.659+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516__shadow_.xxehxqX99AfpMEclRDOFmiTORUyCPIY_0 2026-02-24T22:34:25.650 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7566596603028383589', time=2026-02-25T00:15:34.811086+0000, chain.objs.size()=1 2026-02-24T22:34:25.650 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1514__shadow_.MfeKSHY3n6XK6cjaWNSd-2QVQOvM8H9_0 2026-02-24T22:34:25.650 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2403732768257628174', time=2026-02-25T00:16:32.619458+0000, chain.objs.size()=1 2026-02-24T22:34:25.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520__shadow_.Q4z24VEDDLtjBSI0HcPGUKEGe82ffF5_0 2026-02-24T22:34:25.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2444826778578639294', time=2026-02-25T00:16:33.518223+0000, chain.objs.size()=1 2026-02-24T22:34:25.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521__shadow_.4wAPcOvBgJN6Rj4bP2wptludg7GBJR-_0 2026-02-24T22:34:25.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1895426774780439993', time=2026-02-25T00:16:33.993543+0000, chain.objs.size()=1 2026-02-24T22:34:25.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522__shadow_.r9otJmNZHaBr7EpjiMffmwTCoNN4-ro_0 2026-02-24T22:34:25.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11605364545620257102', time=2026-02-25T00:17:07.124673+0000, chain.objs.size()=1 2026-02-24T22:34:25.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523__shadow_.oatCjAz1bwt-mFPTy-dHDXlsxAn-_iw_0 2026-02-24T22:34:25.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8091715295948116722', time=2026-02-25T00:17:39.882890+0000, chain.objs.size()=1 2026-02-24T22:34:25.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525__shadow_.VMh1TNzLxtYE96z6RxAKnq9lE5Mriuj_0 2026-02-24T22:34:25.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.86560872734713292', time=2026-02-25T00:17:41.428887+0000, chain.objs.size()=1 2026-02-24T22:34:25.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526__shadow_.vsiy1630wunNpB8ugLIzZS7ivFw3JZt_0 2026-02-24T22:34:25.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6072429650107946233', time=2026-02-25T00:18:16.134747+0000, chain.objs.size()=1 2026-02-24T22:34:25.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529__shadow_.lSIpti6Thq3WbLYvOVwDf40t0HX9eRn_0 2026-02-24T22:34:25.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12851844981077803280', time=2026-02-25T00:19:22.417009+0000, chain.objs.size()=1 2026-02-24T22:34:25.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533__shadow_.5gUK-xxcPIlOzgQ1mi51r4CrDIUdHtA_0 2026-02-24T22:34:25.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10136229643966666084', time=2026-02-25T00:19:23.075713+0000, chain.objs.size()=1 2026-02-24T22:34:25.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533__shadow_.0f5QxZu2QZyXmXc0gU7MMF4faBiqlrb_0 2026-02-24T22:34:25.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3611563493442439063', time=2026-02-25T00:19:25.034054+0000, chain.objs.size()=1 2026-02-24T22:34:25.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534__shadow_.12h9HvA8hrCYf6HRtddcYIVIwg8m6U1_0 2026-02-24T22:34:25.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4687920401155729915', time=2026-02-25T00:19:57.119506+0000, chain.objs.size()=1 2026-02-24T22:34:25.651 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535__shadow_.ROsz7bEtvWX2Gp-Fuvzg-YhI7qbg0X__0 2026-02-24T22:34:25.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16960869947194183483', time=2026-02-25T00:19:58.464465+0000, chain.objs.size()=1 2026-02-24T22:34:25.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536__shadow_.swcCJDTYYeHblHX1wL-yUZ2l7YcASX__0 2026-02-24T22:34:25.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11906223382610215779', time=2026-02-25T00:20:30.737925+0000, chain.objs.size()=1 2026-02-24T22:34:25.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537__shadow_.X6lD-Vg0BdwU9vjRlmYoU_4IZR4QWnt_0 2026-02-24T22:34:25.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16042979780145542297', time=2026-02-25T00:21:06.321589+0000, chain.objs.size()=1 2026-02-24T22:34:25.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540__shadow_.vHbvVVwyHLF8tcFM6K4_GP7dtS2UK-k_0 2026-02-24T22:34:25.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1415962921431578287', time=2026-02-25T00:23:56.280843+0000, chain.objs.size()=1 2026-02-24T22:34:25.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1550__shadow_.PUaf9QuVmuzk2QAZ2K9jCfHgjLdpPr0_0 2026-02-24T22:34:25.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13317181949375841604', time=2026-02-25T00:25:32.408443+0000, chain.objs.size()=1 2026-02-24T22:34:25.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1594__shadow_.5N5pui-C9MkasUEprdngYA1Qx0j9wRn_0 2026-02-24T22:34:25.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14621303249073382432', time=2026-02-25T00:25:37.542740+0000, chain.objs.size()=1 2026-02-24T22:34:25.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1603__shadow_.sf6pcaDG7_p6-kWouHXNSEW4pRHFOsI_0 2026-02-24T22:34:25.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15366155180502729651', time=2026-02-25T00:25:38.507360+0000, chain.objs.size()=1 2026-02-24T22:34:25.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1604__shadow_.eTYgsvzv2fIIvVPkmCqEcwKdzNdItop_0 2026-02-24T22:34:25.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14740777364445410897', time=2026-02-25T00:25:46.139674+0000, chain.objs.size()=1 2026-02-24T22:34:25.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1616__shadow_.Kvqg-n1VOqiYBAUm3zwXGZkuLEr0HX4_0 2026-02-24T22:34:25.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1574381004030947072', time=2026-02-25T00:25:50.841718+0000, chain.objs.size()=1 2026-02-24T22:34:25.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1624__shadow_.qWpVJrDZhzvmB7EnrvTCqonADYXQUua_0 2026-02-24T22:34:25.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15985960971624333117', time=2026-02-25T00:25:59.157342+0000, chain.objs.size()=1 2026-02-24T22:34:25.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.KbRKWyWJCU6nJaJUPaHY6vTqB0Rob3w_0 2026-02-24T22:34:25.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14887529887425806709', time=2026-02-25T00:25:59.262721+0000, chain.objs.size()=1 2026-02-24T22:34:25.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.FhZ8w9G3Y82UApdZWZh58u83bOkz3fx_0 2026-02-24T22:34:25.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3491277229906526475', time=2026-02-25T00:25:59.271765+0000, chain.objs.size()=1 2026-02-24T22:34:25.652 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.1kVSesD17XE7ZjewDY2mtNE5hhSddCD_0 2026-02-24T22:34:25.653 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4716209673734882481', time=2026-02-25T00:25:59.284227+0000, chain.objs.size()=1 2026-02-24T22:34:25.653 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.jk1mVo1SZnQN9ZhsNf-Ko068xLIWzFg_0 2026-02-24T22:34:25.653 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12255465651771791213', time=2026-02-25T00:25:59.423655+0000, chain.objs.size()=1 2026-02-24T22:34:25.653 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.t5q2somH3-Cwfn0Go0ytZPS0FnlPbNG_0 2026-02-24T22:34:25.653 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12026809155886807824', time=2026-02-25T00:25:59.435299+0000, chain.objs.size()=1 2026-02-24T22:34:25.653 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.SfJD3XwCKH2hvnaRammP3M78hYbB5V1_0 2026-02-24T22:34:25.653 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5462604814022376313', time=2026-02-25T00:25:59.442590+0000, chain.objs.size()=1 2026-02-24T22:34:25.653 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.qFtz8PXzH-lyZ3We8HbSRJYFsJ9ABV4_0 2026-02-24T22:34:25.653 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5772511164911398520', time=2026-02-25T00:25:59.452143+0000, chain.objs.size()=1 2026-02-24T22:34:25.653 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.UCVYZc7piiEwb-Dr1rJwiFHVQoNPQrW_0 2026-02-24T22:34:25.653 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7154365035995813637', time=2026-02-25T00:25:59.475453+0000, chain.objs.size()=1 2026-02-24T22:34:25.653 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Exa9tFUS5vAomFv2Q0-yng1b7kD3c5U_0 2026-02-24T22:34:25.653 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12081430288742109868', time=2026-02-25T00:25:59.483657+0000, chain.objs.size()=1 2026-02-24T22:34:25.653 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.VZYySrCo6annDxz7OqJhLB21VM7TPzn_0 2026-02-24T22:34:25.653 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10319559676722563535', time=2026-02-25T00:25:59.577653+0000, chain.objs.size()=1 2026-02-24T22:34:25.653 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.3_c-I1jqPjQ7iCUAovIrNvG8yHEIGkM_0 2026-02-24T22:34:25.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11334624374567949390', time=2026-02-25T00:25:59.578190+0000, chain.objs.size()=1 2026-02-24T22:34:25.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.b93JRr_IlwBwu9GZkPjSxHAtvyi8FYn_0 2026-02-24T22:34:25.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17962967990058538684', time=2026-02-25T00:25:59.588105+0000, chain.objs.size()=1 2026-02-24T22:34:25.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.1pdBW8pSwejcSwOrIfA6TG5k9qczJw4_0 2026-02-24T22:34:25.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8353614510432449515', time=2026-02-25T00:25:59.604022+0000, chain.objs.size()=1 2026-02-24T22:34:25.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.zc_JVECYVYqgQF5Z_X2_1L1xdBmv61F_0 2026-02-24T22:34:25.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.553720772562403744', time=2026-02-25T00:25:59.669283+0000, chain.objs.size()=1 2026-02-24T22:34:25.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.RL0-rxMZNexHYca2f93RUIquC7gHV-B_0 2026-02-24T22:34:25.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10869984235806935930', time=2026-02-25T00:25:59.752519+0000, chain.objs.size()=1 2026-02-24T22:34:25.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.jCSVE-voZ0ee2JumgfirvevDQfeWwml_0 2026-02-24T22:34:25.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3395949779282044725', time=2026-02-25T00:25:59.815036+0000, chain.objs.size()=1 2026-02-24T22:34:25.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.UpGzulNimtTD0rGUupHu4AMzzNmuUcg_0 2026-02-24T22:34:25.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2851543828590124845', time=2026-02-25T00:25:59.932267+0000, chain.objs.size()=1 2026-02-24T22:34:25.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.4982NHcKCe3lPVebf0woKtnI05EanVm_0 2026-02-24T22:34:25.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10320361491114251016', time=2026-02-25T00:25:59.943101+0000, chain.objs.size()=1 2026-02-24T22:34:25.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.cxQQMkCSHiBII4B8d3SPrKQSMTUsE9A_0 2026-02-24T22:34:25.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12510393225080068756', time=2026-02-25T00:26:00.097758+0000, chain.objs.size()=1 2026-02-24T22:34:25.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.myq3oCiNlwB2-max4j5OcL8WgbBbOdP_0 2026-02-24T22:34:25.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10367482667617019856', time=2026-02-25T00:26:00.135458+0000, chain.objs.size()=1 2026-02-24T22:34:25.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.INf6NQbARWG7GodvIob6RkNVzd7LcMq_0 2026-02-24T22:34:25.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4486355909839776666', time=2026-02-25T00:26:00.135970+0000, chain.objs.size()=1 2026-02-24T22:34:25.654 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.663+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.EG_6RqiFan4tlKPcdia_70_mfEWzSof_0 2026-02-24T22:34:25.655 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6593867686166035604', time=2026-02-25T00:26:00.241644+0000, chain.objs.size()=1 2026-02-24T22:34:25.655 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.rskP0l2f3-Nik7ImN5uboKusrImrHYv_0 2026-02-24T22:34:25.655 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10502784522758772593', time=2026-02-25T00:26:00.261985+0000, chain.objs.size()=1 2026-02-24T22:34:25.655 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.AahXEZkCBp2daEhvmy80PJBGytVu6qF_0 2026-02-24T22:34:25.655 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11997087766795476400', time=2026-02-25T00:26:00.286534+0000, chain.objs.size()=1 2026-02-24T22:34:25.655 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.DMxAVxTlinBmdOLNIxKKRUADHFeHNaT_0 2026-02-24T22:34:25.655 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7538730070208438591', time=2026-02-25T00:26:08.285888+0000, chain.objs.size()=1 2026-02-24T22:34:25.655 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1642__shadow_.VPvSEHcG6TzLxH4dcKMLjhEucqHKNGc_0 2026-02-24T22:34:25.655 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6875457482630167139', time=2026-02-25T00:26:12.382065+0000, chain.objs.size()=1 2026-02-24T22:34:25.655 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1649__shadow_.PUtP4gteQs061I3J0AZECuWKx3cCdKn_0 2026-02-24T22:34:25.655 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3966319470017428117', time=2026-02-25T00:26:14.230350+0000, chain.objs.size()=1 2026-02-24T22:34:25.655 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1652__shadow_.eH4PG49RZPgRxwve4zhPdMGX0S4wyl5_0 2026-02-24T22:34:25.655 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15635393698978190365', time=2026-02-25T00:26:15.438316+0000, chain.objs.size()=1 2026-02-24T22:34:25.655 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1654__shadow_.5j7dEY4bkFhr0TTgSz_Lp1IYSM17NjN_0 2026-02-24T22:34:25.655 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8395593555082702772', time=2026-02-25T00:26:19.465916+0000, chain.objs.size()=1 2026-02-24T22:34:25.655 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1659__shadow_.fbUm0b7pC8Qvv8doChjdQZ_MO6XAwSp_0 2026-02-24T22:34:25.655 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14179504045391914044', time=2026-02-25T00:26:19.474383+0000, chain.objs.size()=1 2026-02-24T22:34:25.655 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1659__shadow_.GTCEXvM6yYp9zdBRBYA3w2ewG5X3Lh__0 2026-02-24T22:34:25.655 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6290626783690945842', time=2026-02-25T00:26:23.485889+0000, chain.objs.size()=1 2026-02-24T22:34:25.655 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1666__shadow_.gFTnA92fhUoV2isRGvVAklosHLhc8Nq_0 2026-02-24T22:34:25.655 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6619429261652148547', time=2026-02-25T00:26:24.065614+0000, chain.objs.size()=1 2026-02-24T22:34:25.655 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1667__shadow_.36nYVf5Il876091UItgvNxj8hixa_Ux_0 2026-02-24T22:34:25.656 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3806420393795553388', time=2026-02-25T00:26:44.796869+0000, chain.objs.size()=1 2026-02-24T22:34:25.656 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.0G4boNM-xcdBeG2FygcANPSIxr6Q_wO_0 2026-02-24T22:34:25.656 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2588301107757304880', time=2026-02-25T00:26:44.797378+0000, chain.objs.size()=1 2026-02-24T22:34:25.656 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.2H5A0L2EU8YTBgNmr2yeIV0_-JeJStN_0 2026-02-24T22:34:25.656 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.953775088600512636', time=2026-02-25T00:26:44.813485+0000, chain.objs.size()=1 2026-02-24T22:34:25.656 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.FwxX05pd8R0-_MGcaBPmcSnRig-CQM4_0 2026-02-24T22:34:25.656 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12515029416563326192', time=2026-02-25T00:26:44.849341+0000, chain.objs.size()=1 2026-02-24T22:34:25.656 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.7dUf7wah9bUhDiL1z4xIKX4XaSwbWKK_0 2026-02-24T22:34:25.656 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15318163549051310897', time=2026-02-25T00:26:44.950218+0000, chain.objs.size()=1 2026-02-24T22:34:25.656 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.RE_pmkjtDoIo-PtQTeISkONdEd0uE90_0 2026-02-24T22:34:25.656 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12475932308931165183', time=2026-02-25T00:26:44.950793+0000, chain.objs.size()=1 2026-02-24T22:34:25.656 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.3mRqgfc4CF3XPIuSuk-kihVX79FHXGE_0 2026-02-24T22:34:25.656 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14194475851611017976', time=2026-02-25T00:26:44.973126+0000, chain.objs.size()=1 2026-02-24T22:34:25.656 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.3sLNwL75KMgsSLLrgwE8cdUuQ-z6V97_0 2026-02-24T22:34:25.656 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9340604133836383190', time=2026-02-25T00:26:44.998989+0000, chain.objs.size()=1 2026-02-24T22:34:25.656 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.lbBMEe67-bQMQIA1xHQBPAELnhin1Dm_0 2026-02-24T22:34:25.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3392880741286968046', time=2026-02-25T00:26:45.011297+0000, chain.objs.size()=1 2026-02-24T22:34:25.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.6hGFpmmbdh6Rq9X1gQZkDfRiPB32cyQ_0 2026-02-24T22:34:25.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.99768172321506220', time=2026-02-25T00:26:45.130903+0000, chain.objs.size()=1 2026-02-24T22:34:25.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.nhlBp3uEwX3mnG9i5D80SoNMVU5lY59_0 2026-02-24T22:34:25.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12581686731108298114', time=2026-02-25T00:26:45.142000+0000, chain.objs.size()=1 2026-02-24T22:34:25.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.sjZTYxdjk6puh9npu2hr5bvqstVHGRt_0 2026-02-24T22:34:25.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14970187433271888769', time=2026-02-25T00:26:45.144297+0000, chain.objs.size()=1 2026-02-24T22:34:25.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.M-mtcp_sx-G1211EIw52zYThl_AJHv9_0 2026-02-24T22:34:25.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9936020521791719186', time=2026-02-25T00:26:45.166208+0000, chain.objs.size()=1 2026-02-24T22:34:25.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.L1oSIUn5EHO8qTuMI2DGJt7BHz0ruGu_0 2026-02-24T22:34:25.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2228331633594762953', time=2026-02-25T00:26:45.276832+0000, chain.objs.size()=1 2026-02-24T22:34:25.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ItiVc8_vinXIpNnS4vqMaUQVUM85957_0 2026-02-24T22:34:25.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7168479347431501510', time=2026-02-25T00:26:45.279865+0000, chain.objs.size()=1 2026-02-24T22:34:25.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.zmG-_j-XlHTS7YkoTWG3mTy25NsR34c_0 2026-02-24T22:34:25.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4043526124985847260', time=2026-02-25T00:26:45.321888+0000, chain.objs.size()=1 2026-02-24T22:34:25.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.CjM76Ci0GC3aXFJ1vAXRE_FK4hce6FK_0 2026-02-24T22:34:25.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13989100704493515417', time=2026-02-25T00:26:45.324457+0000, chain.objs.size()=1 2026-02-24T22:34:25.657 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.4W-k1XjOsGMKwujpfuT33kXaWouWN-3_0 2026-02-24T22:34:25.658 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12596115163877747656', time=2026-02-25T00:26:45.475920+0000, chain.objs.size()=1 2026-02-24T22:34:25.658 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.-aGa5OpzTM1rqhZwDxDoz3-RCsHW2RF_0 2026-02-24T22:34:25.658 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.28381385316099901', time=2026-02-25T00:26:45.577541+0000, chain.objs.size()=1 2026-02-24T22:34:25.658 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.tlBzjO3F1fzAypLLmgkvokZph8eEvzQ_0 2026-02-24T22:34:25.658 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14307737319039662658', time=2026-02-25T00:26:45.582653+0000, chain.objs.size()=1 2026-02-24T22:34:25.658 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.clfEgOv0tuasPnpbAmB2iJaDUfuLqV9_0 2026-02-24T22:34:25.658 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12066952186434666083', time=2026-02-25T00:26:45.614946+0000, chain.objs.size()=1 2026-02-24T22:34:25.658 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.dCn_eLPPaktwRBOQbACtxMa3dnsem1X_0 2026-02-24T22:34:25.658 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15416520272404981125', time=2026-02-25T00:26:46.015520+0000, chain.objs.size()=1 2026-02-24T22:34:25.658 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.TVDO374PhH6a92Z5iRpmDDvM7JuQjkz_0 2026-02-24T22:34:25.658 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13133139186989548471', time=2026-02-25T00:26:46.015988+0000, chain.objs.size()=1 2026-02-24T22:34:25.658 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.2m2ZfPwXseELgOepipqZ6HZ3c-i6zbh_0 2026-02-24T22:34:25.658 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14156667383838287248', time=2026-02-25T00:26:46.026808+0000, chain.objs.size()=1 2026-02-24T22:34:25.658 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ybnvpApuFXm0zsSJ1xqZuqM1gxlL_a0_0 2026-02-24T22:34:25.658 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13173593768313142964', time=2026-02-25T00:26:46.095743+0000, chain.objs.size()=1 2026-02-24T22:34:25.658 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.syAeG9xMip3lX_9FVGBc3sYyPI2YmYo_0 2026-02-24T22:34:25.658 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2705734960258836711', time=2026-02-25T00:26:46.125413+0000, chain.objs.size()=1 2026-02-24T22:34:25.658 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.667+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.k0ZxH3cmCJeCcvY-OeUevZMnsNo-skZ_0 2026-02-24T22:34:25.659 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6270715642677121735', time=2026-02-25T00:26:46.129802+0000, chain.objs.size()=1 2026-02-24T22:34:25.659 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.scdrOYJfbhoo0_6vdllhJqzdnlBdiV3_0 2026-02-24T22:34:25.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12204420094285966735', time=2026-02-25T00:26:46.155010+0000, chain.objs.size()=1 2026-02-24T22:34:25.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.JrQOZqVfSIbWR0QkiDZxjFreVujFTOs_0 2026-02-24T22:34:25.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.408773556575425483', time=2026-02-25T00:26:53.151358+0000, chain.objs.size()=1 2026-02-24T22:34:25.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.8R0zcbRHEo8nExij5iafWHDrdW3gZWA_0 2026-02-24T22:34:25.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12257258758815135271', time=2026-02-25T00:26:53.153089+0000, chain.objs.size()=1 2026-02-24T22:34:25.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.4fqzLVGhJo6g5Jp3G3vNXByfj660YHl_0 2026-02-24T22:34:25.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10203419472051501575', time=2026-02-25T00:26:53.169895+0000, chain.objs.size()=1 2026-02-24T22:34:25.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.vyelI0JZFDi98_ueKFJp1Ou8YIEIDOK_0 2026-02-24T22:34:25.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17124318811600079348', time=2026-02-25T00:26:53.197899+0000, chain.objs.size()=1 2026-02-24T22:34:25.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.vAc2Qd8idgTLT2Z_IbiVlkoRMxFFwp8_0 2026-02-24T22:34:25.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13308359912286857615', time=2026-02-25T00:26:53.227400+0000, chain.objs.size()=1 2026-02-24T22:34:25.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.YiiK1D_4OtPYVu5FxMjYONQezwOYvqL_0 2026-02-24T22:34:25.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3849018195603862694', time=2026-02-25T00:26:53.235071+0000, chain.objs.size()=1 2026-02-24T22:34:25.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Fw5slkGc0cLFPy1JVv0-rtayj68B6l8_0 2026-02-24T22:34:25.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11732583078058886026', time=2026-02-25T00:26:53.350406+0000, chain.objs.size()=1 2026-02-24T22:34:25.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.pPzE5qxovYp15dlbzgH9tIK1bhbfMb7_0 2026-02-24T22:34:25.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14661546136244804738', time=2026-02-25T00:26:53.364708+0000, chain.objs.size()=1 2026-02-24T22:34:25.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.3L8LSfYFQVhKgRdta8jrMpllKULwxPP_0 2026-02-24T22:34:25.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14036638912699270945', time=2026-02-25T00:26:53.369895+0000, chain.objs.size()=1 2026-02-24T22:34:25.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Ha28WkZMesyXhVBovUN_p1O3QarThR6_0 2026-02-24T22:34:25.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12084776180348103660', time=2026-02-25T00:26:53.409668+0000, chain.objs.size()=1 2026-02-24T22:34:25.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.bU4VILq-FNJyqOo5ybMWMtu_Rr8fOn7_0 2026-02-24T22:34:25.660 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5005792252135522090', time=2026-02-25T00:26:53.448196+0000, chain.objs.size()=1 2026-02-24T22:34:25.661 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Rl0oV0DEuqlUjC-dIIgzNlr14qRjDyc_0 2026-02-24T22:34:25.661 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2655173887465244457', time=2026-02-25T00:26:53.532753+0000, chain.objs.size()=1 2026-02-24T22:34:25.661 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Aoxbmfz5PrY5_9YHM-2-lnwblzm-VOk_0 2026-02-24T22:34:25.661 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13830229845046086146', time=2026-02-25T00:26:53.600023+0000, chain.objs.size()=1 2026-02-24T22:34:25.661 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.3B2muKXaKVbDv3b7NLk0XIfzy0N7sAY_0 2026-02-24T22:34:25.661 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2477997122117704886', time=2026-02-25T00:26:53.740951+0000, chain.objs.size()=1 2026-02-24T22:34:25.661 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ojmWqWdyob3OcL1Mt_dWHc-ZuJltiJ7_0 2026-02-24T22:34:25.661 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9050706461284243090', time=2026-02-25T00:26:53.745545+0000, chain.objs.size()=1 2026-02-24T22:34:25.661 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.DBUqWR3dmF66lXB0uzY3ASMiMP_D21O_0 2026-02-24T22:34:25.662 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.671+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/38334 2026-02-24T22:34:25.663 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=36, truncated=0, next_marker='' 2026-02-24T22:34:25.663 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5247925037071539828', time=2026-02-25T00:26:53.805394+0000, chain.objs.size()=1 2026-02-24T22:34:25.663 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.FSp8yPA8cylvmTPg_ifJ_klzW39dimG_0 2026-02-24T22:34:25.663 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15284501636216001828', time=2026-02-25T00:26:53.828065+0000, chain.objs.size()=1 2026-02-24T22:34:25.663 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.hejCpzBFjQZoeHn0si5cYJVhRt3Pg4w_0 2026-02-24T22:34:25.663 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.584145478789628545', time=2026-02-25T00:26:53.834278+0000, chain.objs.size()=1 2026-02-24T22:34:25.663 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.uPdRQFaZDs81KC0a5p35Fcc3VV7VO9Y_0 2026-02-24T22:34:25.663 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14127467109354398063', time=2026-02-25T00:26:53.944993+0000, chain.objs.size()=1 2026-02-24T22:34:25.663 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.7VIIQp3aanprAss0Fh6ibFswUB05qi0_0 2026-02-24T22:34:25.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.912621328374397697', time=2026-02-25T00:26:54.008750+0000, chain.objs.size()=1 2026-02-24T22:34:25.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.YaszgyLP3RGztpXkGCh8oOWEbC15RFT_0 2026-02-24T22:34:25.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16618681300038456574', time=2026-02-25T00:26:54.032531+0000, chain.objs.size()=1 2026-02-24T22:34:25.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.aok1oathkprpjgkLgefquQovLKj9o3S_0 2026-02-24T22:34:25.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11793957689666689891', time=2026-02-25T00:26:54.119502+0000, chain.objs.size()=1 2026-02-24T22:34:25.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.029MMmvAWTuCGuPZybtnsvwtzSgcUFE_0 2026-02-24T22:34:25.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11674818569332843061', time=2026-02-25T00:26:54.134908+0000, chain.objs.size()=1 2026-02-24T22:34:25.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.mQ79Vn1pJQaejhuFgY41qGwSVmvgElG_0 2026-02-24T22:34:25.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1935333837978505806', time=2026-02-25T00:26:54.135452+0000, chain.objs.size()=1 2026-02-24T22:34:25.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Lce66jeWfXTOfcgxhQDjvKOaG05z8aw_0 2026-02-24T22:34:25.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7544436951730117726', time=2026-02-25T00:26:54.154954+0000, chain.objs.size()=1 2026-02-24T22:34:25.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.7osIkiTaKsLpyhH9cXObnHu_t_SScRX_0 2026-02-24T22:34:25.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7047045657223480174', time=2026-02-25T00:26:54.157787+0000, chain.objs.size()=1 2026-02-24T22:34:25.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.XgxC0h6oYE4Cdy0yBxIju56Im6243zf_0 2026-02-24T22:34:25.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10075117078954172703', time=2026-02-25T00:26:54.174290+0000, chain.objs.size()=1 2026-02-24T22:34:25.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.EVTulDKFbrm6RCmzW04Km7MnCAiIFhh_0 2026-02-24T22:34:25.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2444926411862156229', time=2026-02-25T00:26:54.201489+0000, chain.objs.size()=1 2026-02-24T22:34:25.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_._CxQI_EUNY2qmGy63GHnQBk387TSrnd_0 2026-02-24T22:34:25.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12735432123306710275', time=2026-02-25T00:26:54.207225+0000, chain.objs.size()=1 2026-02-24T22:34:25.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.QWKZjnyM23zHtmZxd47N2AszU-ky-2k_0 2026-02-24T22:34:25.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17181773027985651537', time=2026-02-25T00:26:54.335897+0000, chain.objs.size()=1 2026-02-24T22:34:25.664 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.YXryRg3hrvS1bptn8YOfFinup4wl-54_0 2026-02-24T22:34:25.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13936036576039270058', time=2026-02-25T00:26:54.337782+0000, chain.objs.size()=1 2026-02-24T22:34:25.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.QO2fPEMCx_yJi3HB73Ho9_IuAzPRZUi_0 2026-02-24T22:34:25.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16008331760872170646', time=2026-02-25T00:26:54.381330+0000, chain.objs.size()=1 2026-02-24T22:34:25.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.KMTDEQf4US20sH4noyW47XOio4BHiGp_0 2026-02-24T22:34:25.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9265537613394527899', time=2026-02-25T00:26:54.537800+0000, chain.objs.size()=1 2026-02-24T22:34:25.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.UdIPecCarKvicS3WfHNyt7ORd5_CJ4W_0 2026-02-24T22:34:25.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2168044181206354844', time=2026-02-25T00:26:54.544687+0000, chain.objs.size()=1 2026-02-24T22:34:25.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.vmowfQAOXjfpyM4Pj2VRJ7mRqfqnsK3_0 2026-02-24T22:34:25.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8953875040929908749', time=2026-02-25T00:27:26.817135+0000, chain.objs.size()=1 2026-02-24T22:34:25.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1746__shadow_._J50eBgf7zkNXh5Y8eTVXotRuopweyL_0 2026-02-24T22:34:25.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9558774226093678623', time=2026-02-25T00:28:17.905374+0000, chain.objs.size()=1 2026-02-24T22:34:25.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1822__shadow_.lD2rW--lEavVVsxShimaIpanlkebfjl_0 2026-02-24T22:34:25.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11486318414203525625', time=2026-02-25T00:28:24.434172+0000, chain.objs.size()=1 2026-02-24T22:34:25.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1830__shadow_.BTILplAEmIcW9J2OW16zGDphMwqvSPG_0 2026-02-24T22:34:25.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11953387285781997507', time=2026-02-25T00:28:32.554567+0000, chain.objs.size()=1 2026-02-24T22:34:25.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1849__shadow_.14Jb8KwWq8L8dg9Vf1J89nEwmZu1BtJ_0 2026-02-24T22:34:25.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9959137348401961621', time=2026-02-25T00:28:32.557197+0000, chain.objs.size()=1 2026-02-24T22:34:25.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1849__shadow_.KD9Lj6RMOWAzoa6boLi7K3IG6cjI178_0 2026-02-24T22:34:25.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11815452217124618982', time=2026-02-25T00:28:34.481068+0000, chain.objs.size()=1 2026-02-24T22:34:25.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1852__shadow_.D00HjHC7n0q-JlhV2oWJrcHKSeTvCzF_0 2026-02-24T22:34:25.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7709934301785744762', time=2026-02-25T00:28:37.668256+0000, chain.objs.size()=1 2026-02-24T22:34:25.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1857__shadow_.cPXipHhHnh1JBJq4IDc7MCM1gfiBBdW_0 2026-02-24T22:34:25.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7389561950670838552', time=2026-02-25T00:28:43.675662+0000, chain.objs.size()=1 2026-02-24T22:34:25.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1869__shadow:ba1nU1fNR3N1sfUQ2kL8K8Wm.5Ibl31_.IVNpljy9O7AjWtEyJ7RLhDmshakOZnh_0 2026-02-24T22:34:25.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1454598483971346018', time=2026-02-25T00:28:43.868054+0000, chain.objs.size()=1 2026-02-24T22:34:25.665 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1869__shadow:ba1nU1fNR3N1sfUQ2kL8K8Wm.5Ibl31_.IVNpljy9O7AjWtEyJ7RLhDmshakOZnh_0 2026-02-24T22:34:25.666 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6258940309277475955', time=2026-02-25T00:28:49.369100+0000, chain.objs.size()=1 2026-02-24T22:34:25.666 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1880__shadow_.9i16ynfX5dcaAwrGGQ3W9x3y-mgfz_N_0 2026-02-24T22:34:25.666 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18080550776011503777', time=2026-02-25T00:28:52.410133+0000, chain.objs.size()=1 2026-02-24T22:34:25.666 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1883__shadow_.3llLvLQSacbiXGiEUA3LWc7IsDca3jp_0 2026-02-24T22:34:25.666 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1314260942132031549', time=2026-02-25T00:30:38.310023+0000, chain.objs.size()=1 2026-02-24T22:34:25.666 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:lV1uW5JosyfFX7NJfnPhUfXjN0PQtDO_.7QclBygSg5ijdp2Bf3RX1_4Zebaqy84_0 2026-02-24T22:34:25.666 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4262290550878546773', time=2026-02-25T00:30:41.322077+0000, chain.objs.size()=1 2026-02-24T22:34:25.666 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:8hstZygwa6aA6mZ3qu2AYWCj0UOssD8_.1_gMOoYwPh_SK3bhVmHGb3S23G9rIlw_0 2026-02-24T22:34:25.666 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10811781815230984952', time=2026-02-25T00:30:41.473589+0000, chain.objs.size()=1 2026-02-24T22:34:25.666 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:A.hGLHkYz-Y.Gx4xMmCt36uOePG1LOI_.0ntUWnuKh9OlQdUXFDnrpjqz1egGBcr_0 2026-02-24T22:34:25.666 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18072268279083865573', time=2026-02-25T00:30:57.394187+0000, chain.objs.size()=1 2026-02-24T22:34:25.666 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.675+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:UiYHky8eUQ254RMRyk71eomULJbyp2V_.MSMXzO5-RzznYBzSgYWzreZoqY0aCoq_0 2026-02-24T22:34:25.666 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.679+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9045940670296147557', time=2026-02-25T00:33:54.384909+0000, chain.objs.size()=1 2026-02-24T22:34:25.666 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.679+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1976__shadow_.uPnY46Q2M8wdAKgGWm63UAWCS03BGfO_0 2026-02-24T22:34:25.666 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.679+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3976616852802709854', time=2026-02-25T00:33:58.548912+0000, chain.objs.size()=1 2026-02-24T22:34:25.666 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.679+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1981__shadow_.L3CVJ0mBLe499pYusJK_w6ijrUeEWcZ_0 2026-02-24T22:34:25.667 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.679+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.669 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.679+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2026-02-24T22:34:25.670 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.679+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.670 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.679+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.670 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/38277' 2026-02-24T22:34:25.670 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11343189274247675259', time=2026-02-25T00:15:34.838265+0000, chain.objs.size()=1 2026-02-24T22:34:25.670 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1508__shadow_.kBc0WU5eDwqW85uZimjNJv1eLL83L_j_0 2026-02-24T22:34:25.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12536409681964652512', time=2026-02-25T00:17:05.174919+0000, chain.objs.size()=1 2026-02-24T22:34:25.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522__shadow_.NZNVmPTuGW2J9QUMq1rWt2xiC3Rox-8_0 2026-02-24T22:34:25.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5546503746127887230', time=2026-02-25T00:17:39.883277+0000, chain.objs.size()=1 2026-02-24T22:34:25.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525__shadow_.lJ19dz5Sw09KD0WHPPn0cHQyaSTrMVF_0 2026-02-24T22:34:25.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7692704441790169136', time=2026-02-25T00:17:41.431327+0000, chain.objs.size()=1 2026-02-24T22:34:25.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526__shadow_.YhfEmfvM8DbYSx_RbuurYes2C6J7Ugx_0 2026-02-24T22:34:25.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1222943909948754429', time=2026-02-25T00:18:14.585693+0000, chain.objs.size()=1 2026-02-24T22:34:25.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528__shadow_.ZPBoLsQLR0yzKd1j_ZC1dILMGG758nN_0 2026-02-24T22:34:25.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8057753226974475037', time=2026-02-25T00:18:49.777223+0000, chain.objs.size()=1 2026-02-24T22:34:25.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531__shadow_.CmfgxH9L4QW_ZpFYrJ2CgSJy3o8o0BK_0 2026-02-24T22:34:25.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14365257498049166001', time=2026-02-25T00:18:51.095495+0000, chain.objs.size()=1 2026-02-24T22:34:25.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532__shadow_.kF1TmOqMOBFlfbYP9USGfLjsO3EAftL_0 2026-02-24T22:34:25.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2870868764257759477', time=2026-02-25T00:19:57.119766+0000, chain.objs.size()=1 2026-02-24T22:34:25.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535__shadow_.8sGsI_6HvBfjZ_sTLhbbuGWGYOHb2ZI_0 2026-02-24T22:34:25.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12390997852107150442', time=2026-02-25T00:19:57.124090+0000, chain.objs.size()=1 2026-02-24T22:34:25.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535__shadow_.bImEuOyg7Hwr19cv8DuLa9OXlIvICNH_0 2026-02-24T22:34:25.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12169428715460132208', time=2026-02-25T00:19:58.462904+0000, chain.objs.size()=1 2026-02-24T22:34:25.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536__shadow_.ETUEp_umtdEMLr78lD69OtGcy9ya-9F_0 2026-02-24T22:34:25.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16058853654667498368', time=2026-02-25T00:20:30.737734+0000, chain.objs.size()=1 2026-02-24T22:34:25.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537__shadow_.OUqEvaYUcwS9DAhO8YmI776wN8smlYx_0 2026-02-24T22:34:25.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2879925147333693897', time=2026-02-25T00:20:30.739219+0000, chain.objs.size()=1 2026-02-24T22:34:25.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537__shadow_.laiUgWO-m9r-7dVvUg4Buq7Y7ivnWxZ_0 2026-02-24T22:34:25.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15331330273764544843', time=2026-02-25T00:20:34.039704+0000, chain.objs.size()=1 2026-02-24T22:34:25.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539__shadow_.uL5p26oyJtEM7DzoqzQslShoNTg40M7_0 2026-02-24T22:34:25.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10881843222939244678', time=2026-02-25T00:21:06.324370+0000, chain.objs.size()=1 2026-02-24T22:34:25.671 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540__shadow_.mi4j0DoXYpkUAaIGY6wvffqIvf3Q0x6_0 2026-02-24T22:34:25.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5141102516889048775', time=2026-02-25T00:23:59.657008+0000, chain.objs.size()=1 2026-02-24T22:34:25.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1556__shadow_.DqmvdEmZUSNBwQmdpdY-wWL-LhHhUKK_0 2026-02-24T22:34:25.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12109261196677040049', time=2026-02-25T00:25:36.614016+0000, chain.objs.size()=1 2026-02-24T22:34:25.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1602__shadow_.IWwxd8tUzJLTgde7o_lL2b9mtQTMWkt_0 2026-02-24T22:34:25.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4474359575020995397', time=2026-02-25T00:25:41.726621+0000, chain.objs.size()=1 2026-02-24T22:34:25.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1609__shadow_.ZP8c6jW0CawgluYLRexqwZzsBX65CEw_0 2026-02-24T22:34:25.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11417940779995546821', time=2026-02-25T00:25:43.886437+0000, chain.objs.size()=1 2026-02-24T22:34:25.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1612__shadow_.PC1lLNryw1jZ8Uz_CRd48rY4ad641fj_0 2026-02-24T22:34:25.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3796788403288165048', time=2026-02-25T00:25:47.958610+0000, chain.objs.size()=1 2026-02-24T22:34:25.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1619__shadow_.JVFlwg9U3mIkiQpp_voy8IcY2U3AQOk_0 2026-02-24T22:34:25.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17144971031397779190', time=2026-02-25T00:25:59.102211+0000, chain.objs.size()=1 2026-02-24T22:34:25.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.GV_Rn8zcFrfe2reQNAb6o20skZVW44h_0 2026-02-24T22:34:25.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13827329534014101224', time=2026-02-25T00:25:59.122222+0000, chain.objs.size()=1 2026-02-24T22:34:25.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Y9d4eR1KZ6_5J1EhOrkVUNJmo9voVEb_0 2026-02-24T22:34:25.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12812154235600401769', time=2026-02-25T00:25:59.131146+0000, chain.objs.size()=1 2026-02-24T22:34:25.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.RvDq_te5YNjgwgdiXMFPlRIy5bmLgS6_0 2026-02-24T22:34:25.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1858539202447016049', time=2026-02-25T00:25:59.295915+0000, chain.objs.size()=1 2026-02-24T22:34:25.672 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.mdmJ9Uo01OcH_zdd8eqAzzyyVzoIxqH_0 2026-02-24T22:34:25.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2969624308510591759', time=2026-02-25T00:25:59.309699+0000, chain.objs.size()=1 2026-02-24T22:34:25.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.u0VPTgRbbMVR6gO0Dt-Ar3O1KQvppfr_0 2026-02-24T22:34:25.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10678548005173106116', time=2026-02-25T00:25:59.315537+0000, chain.objs.size()=1 2026-02-24T22:34:25.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.T-wyGGG0Puqy8lNa_UMpPBdWP0IFYQ7_0 2026-02-24T22:34:25.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5438941113536724292', time=2026-02-25T00:25:59.326170+0000, chain.objs.size()=1 2026-02-24T22:34:25.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.H5zzJmbBIcTLpGT_UjjFlkEOTSjVCp-_0 2026-02-24T22:34:25.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11920373694743497744', time=2026-02-25T00:25:59.350742+0000, chain.objs.size()=1 2026-02-24T22:34:25.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.u72hLlUsq4mxg0JQF4Ns6TxN5x5zWhl_0 2026-02-24T22:34:25.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9042714508282909874', time=2026-02-25T00:25:59.422418+0000, chain.objs.size()=1 2026-02-24T22:34:25.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.3JZ9VjZ1stGyiKlgO2oRwOlCarhzyiQ_0 2026-02-24T22:34:25.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4830915038863191179', time=2026-02-25T00:25:59.437663+0000, chain.objs.size()=1 2026-02-24T22:34:25.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Nzm08_w3IoHVeuS8AhBZDcll4RyO_Fg_0 2026-02-24T22:34:25.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4781872534507531010', time=2026-02-25T00:25:59.496618+0000, chain.objs.size()=1 2026-02-24T22:34:25.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ND7hWJr7Iuvg6b0koo9HeJ1lhMcNfkD_0 2026-02-24T22:34:25.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.770651817188246831', time=2026-02-25T00:25:59.605925+0000, chain.objs.size()=1 2026-02-24T22:34:25.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.GDlFvMOip3ZFgEHM-j6QJlZYlPkQgst_0 2026-02-24T22:34:25.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3709908759881192536', time=2026-02-25T00:25:59.619553+0000, chain.objs.size()=1 2026-02-24T22:34:25.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.xGEFqoIBu8_4CD23Z15wmHzl7UClFVK_0 2026-02-24T22:34:25.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16501014510310271480', time=2026-02-25T00:25:59.623611+0000, chain.objs.size()=1 2026-02-24T22:34:25.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.GMp4XTuPLQfW48hkkdGCeydCGL5SyXY_0 2026-02-24T22:34:25.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17001873811888679022', time=2026-02-25T00:25:59.644962+0000, chain.objs.size()=1 2026-02-24T22:34:25.673 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.9ihVhYM6UEV85-3IV5wieWjHdcyhSv8_0 2026-02-24T22:34:25.674 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16000012198086713662', time=2026-02-25T00:25:59.755862+0000, chain.objs.size()=1 2026-02-24T22:34:25.674 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.JfEJUZ9bkbgoKWNHX7oLrNO6mZC_yGU_0 2026-02-24T22:34:25.674 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14334034825751048689', time=2026-02-25T00:25:59.785121+0000, chain.objs.size()=1 2026-02-24T22:34:25.674 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.C6Kmli_MF0IE04W28jXVUJIKqHAV6HB_0 2026-02-24T22:34:25.674 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6242605413653237138', time=2026-02-25T00:25:59.819150+0000, chain.objs.size()=1 2026-02-24T22:34:25.674 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.qQcNpn_Ivn44v3405YwfN9PlkRbuIKO_0 2026-02-24T22:34:25.674 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11370678884230594250', time=2026-02-25T00:25:59.822529+0000, chain.objs.size()=1 2026-02-24T22:34:25.674 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ySWQVescf_xaTlVpaCf82sscBm7v7HM_0 2026-02-24T22:34:25.674 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17848101976863842415', time=2026-02-25T00:25:59.829643+0000, chain.objs.size()=1 2026-02-24T22:34:25.674 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.MZV_eM21IW6SqunNpQwFoW6oZSqeWnD_0 2026-02-24T22:34:25.674 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6325906357077589775', time=2026-02-25T00:25:59.927912+0000, chain.objs.size()=1 2026-02-24T22:34:25.674 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.nytFTeSyFSJqSQw02bbgxM-3BhOFSWR_0 2026-02-24T22:34:25.674 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18323188640169098009', time=2026-02-25T00:25:59.933081+0000, chain.objs.size()=1 2026-02-24T22:34:25.674 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.683+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.1z09NbbieUGNPMMl1kSXvjShLndrKv3_0 2026-02-24T22:34:25.675 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14225760213007148166', time=2026-02-25T00:25:59.985316+0000, chain.objs.size()=1 2026-02-24T22:34:25.675 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.dYJNXvsZjnYu1Eh2qdQYIgzGBXafM3Y_0 2026-02-24T22:34:25.675 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9772319233740059536', time=2026-02-25T00:25:59.986496+0000, chain.objs.size()=1 2026-02-24T22:34:25.675 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.uCzxVh_UZDjFOwuiL3GT3BuFnKzWgmM_0 2026-02-24T22:34:25.675 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16076582909450951397', time=2026-02-25T00:26:00.066814+0000, chain.objs.size()=1 2026-02-24T22:34:25.675 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.3cjXNMCSss57ndrHRmXiZPqaf2WAXEu_0 2026-02-24T22:34:25.675 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18258004951172577788', time=2026-02-25T00:26:00.081206+0000, chain.objs.size()=1 2026-02-24T22:34:25.675 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.kKJ2b-p8ofCcLKnvuWhUtRkOyDDN4s1_0 2026-02-24T22:34:25.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15593239197098408843', time=2026-02-25T00:26:00.094413+0000, chain.objs.size()=1 2026-02-24T22:34:25.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.XcSc5nY8SVsnJ_pkEIdNf8y9iIYRPuO_0 2026-02-24T22:34:25.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6472539874172865333', time=2026-02-25T00:26:00.119001+0000, chain.objs.size()=1 2026-02-24T22:34:25.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Xlm8yo9y5cq70GxFUsiMjFqXdz_4pJu_0 2026-02-24T22:34:25.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9883374747900739563', time=2026-02-25T00:26:00.127520+0000, chain.objs.size()=1 2026-02-24T22:34:25.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.CdMyXUojYJbtp8YepCdTw89Rld0eSVX_0 2026-02-24T22:34:25.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14311503817936914189', time=2026-02-25T00:26:00.237284+0000, chain.objs.size()=1 2026-02-24T22:34:25.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.KAYZc94CmgXs862KKUPtBbbAKgG7dPl_0 2026-02-24T22:34:25.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7918774558996906214', time=2026-02-25T00:26:00.264453+0000, chain.objs.size()=1 2026-02-24T22:34:25.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.WWHVx032Ymr9RjBw65vtXZa3FLleizO_0 2026-02-24T22:34:25.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17648154534061594926', time=2026-02-25T00:26:00.268100+0000, chain.objs.size()=1 2026-02-24T22:34:25.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.gw569ouhPGi_K2O2yuKDJjWksd2BsTw_0 2026-02-24T22:34:25.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15497505439388185881', time=2026-02-25T00:26:00.274192+0000, chain.objs.size()=1 2026-02-24T22:34:25.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.4mFuj2P9Q8SNXb6FNjcSdtbDLIjImqB_0 2026-02-24T22:34:25.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13053618748460418637', time=2026-02-25T00:26:00.291270+0000, chain.objs.size()=1 2026-02-24T22:34:25.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.stxKlK9_BFouA57XDh5i10YPn_0iQeS_0 2026-02-24T22:34:25.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9251230449975659267', time=2026-02-25T00:26:25.857807+0000, chain.objs.size()=1 2026-02-24T22:34:25.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1670__shadow_.mZbGhQGzuBtRKLSzTbfDLQxm8VZaRE__0 2026-02-24T22:34:25.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12858246626241088299', time=2026-02-25T00:26:37.853289+0000, chain.objs.size()=1 2026-02-24T22:34:25.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1687__shadow_.fNx1MhO-oLHGImhKr8FYseMpLqFJ8Q8_0 2026-02-24T22:34:25.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10929522337239723348', time=2026-02-25T00:26:44.801585+0000, chain.objs.size()=1 2026-02-24T22:34:25.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.QXcoGMczmZsOWRrOWPW63YNIHGrYgNl_0 2026-02-24T22:34:25.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.817432101616590098', time=2026-02-25T00:26:44.807180+0000, chain.objs.size()=1 2026-02-24T22:34:25.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.MeOR46jHBxQjoHqHPtllm11xVs_d2az_0 2026-02-24T22:34:25.676 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2003604472976397079', time=2026-02-25T00:26:44.807752+0000, chain.objs.size()=1 2026-02-24T22:34:25.677 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.bsCZlJOhefKyeFppIEJgKQk1hlYnsHY_0 2026-02-24T22:34:25.677 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18290796200751659390', time=2026-02-25T00:26:44.816836+0000, chain.objs.size()=1 2026-02-24T22:34:25.677 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.BRVVk3SqMJajDFqsreT25HubITacDNy_0 2026-02-24T22:34:25.677 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4932836241451498322', time=2026-02-25T00:26:44.853188+0000, chain.objs.size()=1 2026-02-24T22:34:25.677 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.4tPUZVTyRz1zMX7-eMYc-zizOT2f-N6_0 2026-02-24T22:34:25.677 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14145229951044936012', time=2026-02-25T00:26:44.855921+0000, chain.objs.size()=1 2026-02-24T22:34:25.677 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.r4YTPJ6VE6XRxLMFPzp0ItoYOYAzNch_0 2026-02-24T22:34:25.677 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7796084303996254217', time=2026-02-25T00:26:44.856414+0000, chain.objs.size()=1 2026-02-24T22:34:25.677 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.XRPF6x65Y4DhR2CzSzNZUST_jYx_268_0 2026-02-24T22:34:25.677 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14102230641814397949', time=2026-02-25T00:26:44.857986+0000, chain.objs.size()=1 2026-02-24T22:34:25.677 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.14JXYIpXw5gtR-kPLgN2XlFvSPMVilx_0 2026-02-24T22:34:25.677 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11016286900392420496', time=2026-02-25T00:26:44.950391+0000, chain.objs.size()=1 2026-02-24T22:34:25.677 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.iK1F-iVqI3hxtmUuvNUzLNo4kFM2SVl_0 2026-02-24T22:34:25.677 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13800895341046382196', time=2026-02-25T00:26:44.960009+0000, chain.objs.size()=1 2026-02-24T22:34:25.677 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.MRrPFBivnJT17VY46ERiPLdcUOdlecO_0 2026-02-24T22:34:25.677 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5860317669513936250', time=2026-02-25T00:26:44.965606+0000, chain.objs.size()=1 2026-02-24T22:34:25.677 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.C3X5hSbxf8d7R1h_xI5rpHRPW5lHAAt_0 2026-02-24T22:34:25.677 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2991468884984894821', time=2026-02-25T00:26:45.003883+0000, chain.objs.size()=1 2026-02-24T22:34:25.677 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.tNP7YpckG4X5HECGXepWsykcL1PexMv_0 2026-02-24T22:34:25.677 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16827188898691929255', time=2026-02-25T00:26:45.113535+0000, chain.objs.size()=1 2026-02-24T22:34:25.677 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.neQ36a9lHCXqwSYeu6AZcqInM2wXRpI_0 2026-02-24T22:34:25.678 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13765748923425343334', time=2026-02-25T00:26:45.121712+0000, chain.objs.size()=1 2026-02-24T22:34:25.678 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ha4PyOvF1OkxrE_ErxQ9oH_-fLsl6XO_0 2026-02-24T22:34:25.678 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5315728019485749491', time=2026-02-25T00:26:45.126403+0000, chain.objs.size()=1 2026-02-24T22:34:25.678 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.5dyD360TVtQDdnIcXcivpFrR7zXG2sH_0 2026-02-24T22:34:25.678 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14564708874689238237', time=2026-02-25T00:26:45.128555+0000, chain.objs.size()=1 2026-02-24T22:34:25.678 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.KNGqiKD-B_juIi0LTUPByPa_oxSFtQ2_0 2026-02-24T22:34:25.678 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3153429102076670250', time=2026-02-25T00:26:45.180884+0000, chain.objs.size()=1 2026-02-24T22:34:25.678 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.qaBIkWNR3_m8rpBUj-TCS0pMy9IyDwp_0 2026-02-24T22:34:25.678 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9544030746417738557', time=2026-02-25T00:26:45.270413+0000, chain.objs.size()=1 2026-02-24T22:34:25.678 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.LHU7BqaLmRWHzgpmICM48Z5Gzf873Fl_0 2026-02-24T22:34:25.678 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537916248766730198', time=2026-02-25T00:26:45.284589+0000, chain.objs.size()=1 2026-02-24T22:34:25.678 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.687+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.zzHcaJ6UuEpzXeUFfev2p5VUyqCSK_q_0 2026-02-24T22:34:25.678 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8182569436559208', time=2026-02-25T00:26:45.288412+0000, chain.objs.size()=1 2026-02-24T22:34:25.678 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Yle8XPNOkt-qJ4_XpPDHOZUUTOOHzDL_0 2026-02-24T22:34:25.678 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11915486810499841641', time=2026-02-25T00:26:45.444107+0000, chain.objs.size()=1 2026-02-24T22:34:25.678 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.pIDH_WGbb90sbaft2ahce_AYqtIoPzw_0 2026-02-24T22:34:25.678 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7874020948056209803', time=2026-02-25T00:26:45.455155+0000, chain.objs.size()=1 2026-02-24T22:34:25.678 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.NYHKp9d3jB_J2Hkx3YA859XicT7aZT1_0 2026-02-24T22:34:25.679 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5173432021658211776', time=2026-02-25T00:26:45.466411+0000, chain.objs.size()=1 2026-02-24T22:34:25.679 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.r-HMjz9yOrDaHm9RP7csRIS9vmGiDcY_0 2026-02-24T22:34:25.679 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2803437812674543887', time=2026-02-25T00:26:45.595996+0000, chain.objs.size()=1 2026-02-24T22:34:25.679 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.vJVnekA8pm5S9FfYM_PxBcW5mFQggWq_0 2026-02-24T22:34:25.679 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14092868510508951301', time=2026-02-25T00:26:45.938582+0000, chain.objs.size()=1 2026-02-24T22:34:25.679 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.fc8DdhhLEuXMhCLbXlnLryn84jfpF-q_0 2026-02-24T22:34:25.679 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9596792137438106794', time=2026-02-25T00:26:45.939364+0000, chain.objs.size()=1 2026-02-24T22:34:25.679 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.RfRYbHGLkNJMxhaOL_Lfa6t1zh7VUrk_0 2026-02-24T22:34:25.679 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7911394400957160421', time=2026-02-25T00:26:46.001748+0000, chain.objs.size()=1 2026-02-24T22:34:25.679 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.kp-jpe5c_JbWk-d59LkXADR3src8pWl_0 2026-02-24T22:34:25.679 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3499401572015803041', time=2026-02-25T00:26:46.137798+0000, chain.objs.size()=1 2026-02-24T22:34:25.679 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.VRNucc3nwxkBixXd0O4KzHD9eM5Myrs_0 2026-02-24T22:34:25.679 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.853409567552983130', time=2026-02-25T00:26:46.141448+0000, chain.objs.size()=1 2026-02-24T22:34:25.679 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.cNpIFDyT6ntDr9M1J9-0Zigym5z1fNb_0 2026-02-24T22:34:25.679 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2765805689049011707', time=2026-02-25T00:26:46.150955+0000, chain.objs.size()=1 2026-02-24T22:34:25.679 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.qBaGPVUAvYwt3PXxW9tanV48ALAfeFc_0 2026-02-24T22:34:25.679 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10370091032404903981', time=2026-02-25T00:26:46.163431+0000, chain.objs.size()=1 2026-02-24T22:34:25.679 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.YQO22tGaxFlfUZVxDVv5idkBtdGMP7e_0 2026-02-24T22:34:25.680 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11140287963899662287', time=2026-02-25T00:26:53.179892+0000, chain.objs.size()=1 2026-02-24T22:34:25.680 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.QnU3j5MKGjiVEgT7gmg-AuepFwKtETP_0 2026-02-24T22:34:25.680 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11289500674136651612', time=2026-02-25T00:26:53.192330+0000, chain.objs.size()=1 2026-02-24T22:34:25.680 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.45mkBE4g5Nv71L6cfF-bNnKyR24tB6v_0 2026-02-24T22:34:25.680 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11531719687658375465', time=2026-02-25T00:26:53.193216+0000, chain.objs.size()=1 2026-02-24T22:34:25.680 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.tO5xZUlNHdCVNV3lIpZHfVJh7kOxnp6_0 2026-02-24T22:34:25.680 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16201405958821635698', time=2026-02-25T00:26:53.207909+0000, chain.objs.size()=1 2026-02-24T22:34:25.680 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.IKHYtZ2I5A0VJBFnPnF-I9Bf0wHyOmp_0 2026-02-24T22:34:25.680 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14910178522429770637', time=2026-02-25T00:26:53.210168+0000, chain.objs.size()=1 2026-02-24T22:34:25.680 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.v39PFbQGaBzSbz4793N2IF4aZmYDsFc_0 2026-02-24T22:34:25.680 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18423184303656108684', time=2026-02-25T00:26:53.225683+0000, chain.objs.size()=1 2026-02-24T22:34:25.680 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.XKK5pzURr1s4JImInWpAYEQ3ny8wPHU_0 2026-02-24T22:34:25.680 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ca7e7640 0 -- 10.20.193.46:0/1777517895 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x55661f02f180 msgr2=0x55661f02f5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:34:25.680 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17685294044705680277', time=2026-02-25T00:26:53.338101+0000, chain.objs.size()=1 2026-02-24T22:34:25.680 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Xae1uI4B0ijWnDhr_qbUaxhwKsQYz4g_0 2026-02-24T22:34:25.680 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3823175325475912640', time=2026-02-25T00:26:53.386088+0000, chain.objs.size()=1 2026-02-24T22:34:25.680 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.T59v6m9H6bI2JsWRY8b6mOF3pWwtvtZ_0 2026-02-24T22:34:25.680 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3449471767717906014', time=2026-02-25T00:26:53.390906+0000, chain.objs.size()=1 2026-02-24T22:34:25.680 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.QZoFomOL5cyzpDySQNF_kqJYQ7-TVQf_0 2026-02-24T22:34:25.680 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14707917083464267610', time=2026-02-25T00:26:53.526278+0000, chain.objs.size()=1 2026-02-24T22:34:25.680 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.IFw0RYw-uoYz8sTiGzZoQzBG-8LeRVa_0 2026-02-24T22:34:25.680 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18004731498678326143', time=2026-02-25T00:26:53.592639+0000, chain.objs.size()=1 2026-02-24T22:34:25.681 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.j1N1LUpN7sf8YGed4ygFwKdbGUG9-xs_0 2026-02-24T22:34:25.681 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11102743273607573549', time=2026-02-25T00:26:53.599045+0000, chain.objs.size()=1 2026-02-24T22:34:25.681 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.U3Kq0HVdGVn-54lsPVL3CDsfKIvY5Sy_0 2026-02-24T22:34:25.681 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3348417337115005502', time=2026-02-25T00:26:53.764394+0000, chain.objs.size()=1 2026-02-24T22:34:25.681 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.D6zsugFNuwNJDViu_tDyrETqUgNpMWg_0 2026-02-24T22:34:25.681 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16038502164390752093', time=2026-02-25T00:26:53.783307+0000, chain.objs.size()=1 2026-02-24T22:34:25.681 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.uBzNDHDbt2JJuCJb6T_DYP9mR7sIOiA_0 2026-02-24T22:34:25.682 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.691+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/38277 2026-02-24T22:34:25.683 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=32, truncated=0, next_marker='' 2026-02-24T22:34:25.683 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12843752765575320400', time=2026-02-25T00:26:53.796662+0000, chain.objs.size()=1 2026-02-24T22:34:25.683 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.PiTR80keKzk1ZlKgfoYla3YKizWRaJC_0 2026-02-24T22:34:25.683 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2841188579257619764', time=2026-02-25T00:26:53.836959+0000, chain.objs.size()=1 2026-02-24T22:34:25.683 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.5UZeF77gc5mV3LYVTIYFXE83iqVQYIs_0 2026-02-24T22:34:25.683 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3604198962763489751', time=2026-02-25T00:26:53.840129+0000, chain.objs.size()=1 2026-02-24T22:34:25.683 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ICSWEr3V4wjQ-tAE930TdprRGs0pznq_0 2026-02-24T22:34:25.683 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6386372404193671176', time=2026-02-25T00:26:53.937174+0000, chain.objs.size()=1 2026-02-24T22:34:25.683 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.SsI9VLDd2YblAT0P69TSb4kUXmcQ-wO_0 2026-02-24T22:34:25.683 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13174750495809713655', time=2026-02-25T00:26:53.939934+0000, chain.objs.size()=1 2026-02-24T22:34:25.683 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.kgdfmP4Ztx4fut3KEVWg2Uysjrv1H9C_0 2026-02-24T22:34:25.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11363531289752422898', time=2026-02-25T00:26:53.977238+0000, chain.objs.size()=1 2026-02-24T22:34:25.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.DlTQ8AUCiBC5Qi7-cvWpVGk2HusxABg_0 2026-02-24T22:34:25.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10574541356304771219', time=2026-02-25T00:26:53.982752+0000, chain.objs.size()=1 2026-02-24T22:34:25.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.pJXgIjCBktQcyss7Js9HwuRN8bCQSdc_0 2026-02-24T22:34:25.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6053214651431002030', time=2026-02-25T00:26:53.991909+0000, chain.objs.size()=1 2026-02-24T22:34:25.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.fL7Rg5zbV14pjk_I625hjVqFlMWiX_I_0 2026-02-24T22:34:25.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17143487692704795376', time=2026-02-25T00:26:54.000777+0000, chain.objs.size()=1 2026-02-24T22:34:25.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.OrF3h6ep9X0gTKmg-Lod0Sl7V7uxBUp_0 2026-02-24T22:34:25.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.914123352468839870', time=2026-02-25T00:26:54.018469+0000, chain.objs.size()=1 2026-02-24T22:34:25.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.pVke08pgjnpbfftPSoVsTaYrj6P0bus_0 2026-02-24T22:34:25.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17578572446306994765', time=2026-02-25T00:26:54.183953+0000, chain.objs.size()=1 2026-02-24T22:34:25.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.1JoNVHDbn-lIIaWWKtrPH33iz44D9xM_0 2026-02-24T22:34:25.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13508815060703735617', time=2026-02-25T00:26:54.194180+0000, chain.objs.size()=1 2026-02-24T22:34:25.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.qweCCVpUWy5ZuKWyt093EVQMHHbc8zk_0 2026-02-24T22:34:25.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0c9fe6640 0 -- 10.20.193.46:0/1777517895 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x55661f02c690 msgr2=0x7fa08c1010e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:34:25.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2279935225515834452', time=2026-02-25T00:26:54.222173+0000, chain.objs.size()=1 2026-02-24T22:34:25.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.xvd_ERfoiBqVcGMN0oqRSxiVKht3oaF_0 2026-02-24T22:34:25.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17943609123883746652', time=2026-02-25T00:26:54.302268+0000, chain.objs.size()=1 2026-02-24T22:34:25.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.o43b1VHSHbgUk4ZmnqE9p6Qcd-d_pra_0 2026-02-24T22:34:25.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.31927058889253089', time=2026-02-25T00:26:54.364747+0000, chain.objs.size()=1 2026-02-24T22:34:25.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.B0uuo8HHaOMbIcZTDvCfkiKuGyyugMD_0 2026-02-24T22:34:25.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8832241910207316850', time=2026-02-25T00:26:54.393255+0000, chain.objs.size()=1 2026-02-24T22:34:25.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.P9Y_2rKHrucgTSmrvQ76knKP6kCEnAq_0 2026-02-24T22:34:25.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16690962033500001786', time=2026-02-25T00:26:54.473734+0000, chain.objs.size()=1 2026-02-24T22:34:25.684 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.3PtlfpAa8khOHON3jnq3WAGVwk4Y7eS_0 2026-02-24T22:34:25.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6516984345980834229', time=2026-02-25T00:26:54.493507+0000, chain.objs.size()=1 2026-02-24T22:34:25.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.KjNfFKBHbxioiH-R_AYAhwiW0QEO7HL_0 2026-02-24T22:34:25.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14564745622716989973', time=2026-02-25T00:26:54.529666+0000, chain.objs.size()=1 2026-02-24T22:34:25.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Pee1tHDpNXw5ApdUWq8Mb3lIaU4HlnD_0 2026-02-24T22:34:25.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18051705605760054925', time=2026-02-25T00:28:18.741691+0000, chain.objs.size()=1 2026-02-24T22:34:25.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1823__shadow_.cvkbN8QzZykaUuhOkhFKitGa8qb3Qgr_0 2026-02-24T22:34:25.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14363570288548160279', time=2026-02-25T00:28:32.556080+0000, chain.objs.size()=1 2026-02-24T22:34:25.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1849__shadow_.ep7H1aqTHWV1yWW6zz5dOSHNUNRf6Sw_0 2026-02-24T22:34:25.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4086090805894669947', time=2026-02-25T00:29:12.235542+0000, chain.objs.size()=1 2026-02-24T22:34:25.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1891__multipart_mymultipart.2~yeNtJBDUIm3lkwCKZhPI9RDqlipc3Tm.1 2026-02-24T22:34:25.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18367979304082610482', time=2026-02-25T00:30:09.304930+0000, chain.objs.size()=1 2026-02-24T22:34:25.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1927__shadow_.0Ou3EjudlH9UL40h3UB7LhtTYTW_02s_0 2026-02-24T22:34:25.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10853158863964217825', time=2026-02-25T00:30:21.654119+0000, chain.objs.size()=1 2026-02-24T22:34:25.685 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:M-ymY2FefzOX8NRmM7GsN4nCF0D88KH_.4Gqn8Q_gK_MDbpW6s2O9EHVtoLjku2K_0 2026-02-24T22:34:25.686 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1103630726113190137', time=2026-02-25T00:30:24.345389+0000, chain.objs.size()=1 2026-02-24T22:34:25.686 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1938__shadow:NY2y1zVJPyouYDDKeqnnRWPqVvfG43o_.sxe3BwnCV2bh5bOmc7eizzkwXRVO7pU_0 2026-02-24T22:34:25.686 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5309681989660012619', time=2026-02-25T00:30:34.357787+0000, chain.objs.size()=1 2026-02-24T22:34:25.686 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1946__shadow:z7BTcfKFtYqIFWIsRF1SykplHk7CwLm_.ybnlT49GCpYPvt1JbWA9IXlz7L-PC8F_0 2026-02-24T22:34:25.686 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13018893421318222633', time=2026-02-25T00:30:58.468961+0000, chain.objs.size()=1 2026-02-24T22:34:25.686 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958__shadow:B1m9ouZShE2eqRps-DliLgBqmeqi2Gn_.f8pbNIO68GeLiewf1uzj5aKq9ToorOF_0 2026-02-24T22:34:25.686 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13018893421318222633', time=2026-02-25T00:30:58.469774+0000, chain.objs.size()=1 2026-02-24T22:34:25.686 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958__shadow:B1m9ouZShE2eqRps-DliLgBqmeqi2Gn_.f8pbNIO68GeLiewf1uzj5aKq9ToorOF_0 2026-02-24T22:34:25.686 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ca7e7640 0 -- 10.20.193.46:0/1777517895 >> [v2:10.20.193.110:6817/3330554869,v1:10.20.193.110:6819/3330554869] conn(0x55661f0320d0 msgr2=0x7fa08c1419c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:34:25.686 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13018893421318222633', time=2026-02-25T00:30:58.471157+0000, chain.objs.size()=1 2026-02-24T22:34:25.686 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958__shadow:B1m9ouZShE2eqRps-DliLgBqmeqi2Gn_.f8pbNIO68GeLiewf1uzj5aKq9ToorOF_0 2026-02-24T22:34:25.686 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13018893421318222633', time=2026-02-25T00:30:58.471734+0000, chain.objs.size()=1 2026-02-24T22:34:25.686 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.695+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958__shadow:B1m9ouZShE2eqRps-DliLgBqmeqi2Gn_.f8pbNIO68GeLiewf1uzj5aKq9ToorOF_0 2026-02-24T22:34:25.686 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.699+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13018893421318222633', time=2026-02-25T00:30:58.472191+0000, chain.objs.size()=1 2026-02-24T22:34:25.686 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.699+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958__shadow:B1m9ouZShE2eqRps-DliLgBqmeqi2Gn_.f8pbNIO68GeLiewf1uzj5aKq9ToorOF_0 2026-02-24T22:34:25.687 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.699+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7020806865321979616', time=2026-02-25T00:33:48.525219+0000, chain.objs.size()=1 2026-02-24T22:34:25.687 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.699+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1971__shadow_.0XIwQpPCbrkz-5pZKD4xx0lifJ9hG_G_0 2026-02-24T22:34:25.688 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.699+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.689 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.699+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2026-02-24T22:34:25.691 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.691 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/37540' 2026-02-24T22:34:25.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13109709138848217368', time=2026-02-25T00:15:34.706021+0000, chain.objs.size()=1 2026-02-24T22:34:25.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516__shadow_.CdgduzJ1eKGKrr2KZaGcK9i4YvqxGVS_0 2026-02-24T22:34:25.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17966126535872513209', time=2026-02-25T00:15:34.837006+0000, chain.objs.size()=1 2026-02-24T22:34:25.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1508__shadow_.lYElLIF4Mtlfsvm-eHTqZl6S-7ZT7RQ_0 2026-02-24T22:34:25.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17046905199865487336', time=2026-02-25T00:15:34.860079+0000, chain.objs.size()=1 2026-02-24T22:34:25.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1509__shadow_.p4oMCcql17T6nBiRW_VJfDwhMxy7Jxf_0 2026-02-24T22:34:25.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17555658981547150071', time=2026-02-25T00:16:33.509894+0000, chain.objs.size()=1 2026-02-24T22:34:25.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521__shadow_.bddQTDiiftjEQPBlTjyiNKvq7sFqdEy_0 2026-02-24T22:34:25.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4814789971213247698', time=2026-02-25T00:17:05.164040+0000, chain.objs.size()=1 2026-02-24T22:34:25.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522__shadow_.tLrMt9ZWOjBx8YCJP4C9agE2T3SCxFD_0 2026-02-24T22:34:25.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7318901437581184986', time=2026-02-25T00:17:05.170158+0000, chain.objs.size()=1 2026-02-24T22:34:25.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522__shadow_.oMNBNHoiYQ4jkjaw003JiSqoO0fz6Ve_0 2026-02-24T22:34:25.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3598923486165346039', time=2026-02-25T00:17:07.116521+0000, chain.objs.size()=1 2026-02-24T22:34:25.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523__shadow_.IeiKa8AjUHoQpvGm5GXyTRflp3KnrO5_0 2026-02-24T22:34:25.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4685132337824939656', time=2026-02-25T00:17:07.118640+0000, chain.objs.size()=1 2026-02-24T22:34:25.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523__shadow_.wMlzrqXg7THBBT_kfGEk5PgKmqE4Mcl_0 2026-02-24T22:34:25.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5343876529852709266', time=2026-02-25T00:17:08.003954+0000, chain.objs.size()=1 2026-02-24T22:34:25.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524__shadow_.5xnj_JjEtlTLN8LmBiuLDYOcGKv86Pc_0 2026-02-24T22:34:25.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6175632646324205881', time=2026-02-25T00:17:08.006026+0000, chain.objs.size()=1 2026-02-24T22:34:25.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524__shadow_.Wfj8vfMdUF5yJ8fpd0d1DeAqfatCK5d_0 2026-02-24T22:34:25.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4522163635192847319', time=2026-02-25T00:17:08.006796+0000, chain.objs.size()=1 2026-02-24T22:34:25.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524__shadow_.EKcN7jhFlHTAGxPUZTxTniQopZlooJY_0 2026-02-24T22:34:25.692 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8906592505538102395', time=2026-02-25T00:17:08.009838+0000, chain.objs.size()=1 2026-02-24T22:34:25.693 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524__shadow_.n3jOtyK7fLtEOTbzT8LSoz4hKc2VzcM_0 2026-02-24T22:34:25.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10012268766635418088', time=2026-02-25T00:17:39.874144+0000, chain.objs.size()=1 2026-02-24T22:34:25.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525__shadow_.Y_ObTMAlr-aKHOcCBVjYokUPvgtvk-7_0 2026-02-24T22:34:25.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17275836071145848270', time=2026-02-25T00:17:43.619840+0000, chain.objs.size()=1 2026-02-24T22:34:25.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528__shadow_.YTCdCK11P-guFLOADMnZvueQnGpDEZL_0 2026-02-24T22:34:25.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527662425880637113', time=2026-02-25T00:18:14.576203+0000, chain.objs.size()=1 2026-02-24T22:34:25.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528__shadow_.5-3H-z8R974DPGEV1A-_P4zpoApM0fB_0 2026-02-24T22:34:25.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.596557028526030070', time=2026-02-25T00:18:16.122340+0000, chain.objs.size()=1 2026-02-24T22:34:25.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529__shadow_.4qIY1S_2gMk9aYURVRA9IFP7koZMUKZ_0 2026-02-24T22:34:25.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11565503426046882771', time=2026-02-25T00:18:49.768232+0000, chain.objs.size()=1 2026-02-24T22:34:25.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531__shadow_.g2_pCF6d-wOYYBQseN-Qf_XQf3J0DOV_0 2026-02-24T22:34:25.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6574801502827999748', time=2026-02-25T00:18:51.089854+0000, chain.objs.size()=1 2026-02-24T22:34:25.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532__shadow_.WBx6njXPteV_nXKjBRcbP6MJN7iZEsW_0 2026-02-24T22:34:25.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1675540855431102160', time=2026-02-25T00:19:23.064035+0000, chain.objs.size()=1 2026-02-24T22:34:25.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533__shadow_.M5gZDWrxSa654hg8BilBg0iUQpfYOmN_0 2026-02-24T22:34:25.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8300887174474264952', time=2026-02-25T00:19:57.106856+0000, chain.objs.size()=1 2026-02-24T22:34:25.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535__shadow_.OPlFHBfJkPkxJnbJKVYuhqQxzmhwYTm_0 2026-02-24T22:34:25.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5823986667020450381', time=2026-02-25T00:19:58.450027+0000, chain.objs.size()=1 2026-02-24T22:34:25.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536__shadow_.h2lnKNaM6t9zTMNVyw_TN56lG9IpZSP_0 2026-02-24T22:34:25.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9532744602883817524', time=2026-02-25T00:19:58.450519+0000, chain.objs.size()=1 2026-02-24T22:34:25.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536__shadow_.IF8z15kuHDEcJSYv-pWKgVf1YVKvClw_0 2026-02-24T22:34:25.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12104035675234434023', time=2026-02-25T00:20:34.029552+0000, chain.objs.size()=1 2026-02-24T22:34:25.694 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.703+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539__shadow_.O3SyUS0zMw7-PvuNWDuT4ztajA3Mapa_0 2026-02-24T22:34:25.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3589136708120289559', time=2026-02-25T00:20:34.032592+0000, chain.objs.size()=1 2026-02-24T22:34:25.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539__shadow_.p58lB_mC2-50i4KOwy7saCtfFV4MRb7_0 2026-02-24T22:34:25.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3063216475688926929', time=2026-02-25T00:20:34.033033+0000, chain.objs.size()=1 2026-02-24T22:34:25.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539__shadow_.a5-SDqVtmOTHg-pv-3k68oBNb8ZYHS-_0 2026-02-24T22:34:25.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12497554589043559597', time=2026-02-25T00:20:34.033961+0000, chain.objs.size()=1 2026-02-24T22:34:25.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539__shadow_.45QrLdUS26BLBkFyC3midEGbsY2NL5k_0 2026-02-24T22:34:25.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16283737191973867602', time=2026-02-25T00:21:06.310120+0000, chain.objs.size()=1 2026-02-24T22:34:25.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540__shadow_.0W0odYfbfBzxgPHkWi5gJS4l_fEBMUQ_0 2026-02-24T22:34:25.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16304172756893348827', time=2026-02-25T00:23:14.350241+0000, chain.objs.size()=1 2026-02-24T22:34:25.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518__shadow_.uQ33zGG4Bc93JEaKjIbSM2OFZ62_Kiy_0 2026-02-24T22:34:25.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2635778799400294868', time=2026-02-25T00:25:33.087578+0000, chain.objs.size()=1 2026-02-24T22:34:25.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1596__shadow_.eXpPJsrM8IXVRXdWZS32goPg7049Yzk_0 2026-02-24T22:34:25.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3804448451504468496', time=2026-02-25T00:25:35.426652+0000, chain.objs.size()=1 2026-02-24T22:34:25.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1600__shadow_.8PaHyTDWEsFIAb9YRCcZg9Qwpakrfnb_0 2026-02-24T22:34:25.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16725080242936580195', time=2026-02-25T00:25:35.994899+0000, chain.objs.size()=1 2026-02-24T22:34:25.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1601__shadow_.AyxJOLCz_auhZBXg8Yglu3TQH-5FHlr_0 2026-02-24T22:34:25.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11778253723233091533', time=2026-02-25T00:25:42.668404+0000, chain.objs.size()=1 2026-02-24T22:34:25.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1610__shadow_.t8Ur1KELfFnJsL7DTvWqvouo3leioAF_0 2026-02-24T22:34:25.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17818464917967799848', time=2026-02-25T00:25:45.566937+0000, chain.objs.size()=1 2026-02-24T22:34:25.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1615__shadow_.6dJRrZSBXJGdJeivZccq4eF3avxshfh_0 2026-02-24T22:34:25.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13674759232588633357', time=2026-02-25T00:25:59.116146+0000, chain.objs.size()=1 2026-02-24T22:34:25.695 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.seJiPqwJ7OosYy912BFPtvJxsY4JsmD_0 2026-02-24T22:34:25.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1585724071359392582', time=2026-02-25T00:25:59.131709+0000, chain.objs.size()=1 2026-02-24T22:34:25.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Cqh_JIElCeImO3IO6-Y4bFdJlozhF7Z_0 2026-02-24T22:34:25.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6507365863251295259', time=2026-02-25T00:25:59.146778+0000, chain.objs.size()=1 2026-02-24T22:34:25.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.XEvxYNPzxxRU6twK-IWu0dKP9iqutGY_0 2026-02-24T22:34:25.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4100265129230516905', time=2026-02-25T00:25:59.147370+0000, chain.objs.size()=1 2026-02-24T22:34:25.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.U46YxY5anJBaAlL43LgiMyTF8U-0Iyh_0 2026-02-24T22:34:25.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.500638843476886662', time=2026-02-25T00:25:59.289140+0000, chain.objs.size()=1 2026-02-24T22:34:25.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.FRi4vviJNVGbd_Irx9J6Iz6zM1fkTS__0 2026-02-24T22:34:25.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15304332787132952591', time=2026-02-25T00:25:59.292052+0000, chain.objs.size()=1 2026-02-24T22:34:25.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.bGPCVyhdpuZiGN4jnHAKbYTf-Jobzzr_0 2026-02-24T22:34:25.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10696790594169860165', time=2026-02-25T00:25:59.308027+0000, chain.objs.size()=1 2026-02-24T22:34:25.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.aDyfQncY0PHjJ_2wWKQJHoHkL0s2Jqw_0 2026-02-24T22:34:25.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1215541609031837753', time=2026-02-25T00:25:59.414470+0000, chain.objs.size()=1 2026-02-24T22:34:25.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.oYX5puNa3M-EM19wtX2my8xsoquTwkh_0 2026-02-24T22:34:25.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.959983153777946523', time=2026-02-25T00:25:59.424586+0000, chain.objs.size()=1 2026-02-24T22:34:25.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.H3o2sn9kF6EukRAoyteszoJfgXAhVLG_0 2026-02-24T22:34:25.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10872539589546137750', time=2026-02-25T00:25:59.426369+0000, chain.objs.size()=1 2026-02-24T22:34:25.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.4fdJV0yG_RG4sGOOd3HhUfkxn3GhH0o_0 2026-02-24T22:34:25.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6468977020596088145', time=2026-02-25T00:25:59.466401+0000, chain.objs.size()=1 2026-02-24T22:34:25.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.jf0XyikOKFOL8i3peIYtKzmYKFELYMz_0 2026-02-24T22:34:25.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11261422178726977802', time=2026-02-25T00:25:59.603917+0000, chain.objs.size()=1 2026-02-24T22:34:25.696 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.kfZF0Hg1n1NArKFXwUus_UCcv3oYV54_0 2026-02-24T22:34:25.697 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9080854623140632816', time=2026-02-25T00:25:59.608342+0000, chain.objs.size()=1 2026-02-24T22:34:25.697 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.NP1Z6gPDHyp-yEzVkj8FW1xqftvS3kr_0 2026-02-24T22:34:25.697 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9417515170334926906', time=2026-02-25T00:25:59.614282+0000, chain.objs.size()=1 2026-02-24T22:34:25.697 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.qnrVdjblbeGn-5DIjJdd5oBvl82xlVQ_0 2026-02-24T22:34:25.697 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7225857849393602278', time=2026-02-25T00:25:59.616499+0000, chain.objs.size()=1 2026-02-24T22:34:25.697 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.koG5fc0nQIoRB_ZE5s1HT8y5pdqvD1X_0 2026-02-24T22:34:25.697 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15677065685423655064', time=2026-02-25T00:25:59.638897+0000, chain.objs.size()=1 2026-02-24T22:34:25.697 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.oddPKWOSZpKs8HOGB-K0yKSZdrGx7JI_0 2026-02-24T22:34:25.697 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12036383443057024054', time=2026-02-25T00:25:59.725309+0000, chain.objs.size()=1 2026-02-24T22:34:25.697 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.zjJNMNKKjsoLmxcEfXfvT788evFn_uO_0 2026-02-24T22:34:25.697 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.469086972244310362', time=2026-02-25T00:25:59.756061+0000, chain.objs.size()=1 2026-02-24T22:34:25.697 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.E3sUtB2K_1g_-FU8GE-Dtky0cZfW7G5_0 2026-02-24T22:34:25.697 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3559201479523833401', time=2026-02-25T00:25:59.759705+0000, chain.objs.size()=1 2026-02-24T22:34:25.697 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.mNvRk2xXbSPRYLrKnjwheqbfflxKNZo_0 2026-02-24T22:34:25.697 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14301645639796796365', time=2026-02-25T00:25:59.783785+0000, chain.objs.size()=1 2026-02-24T22:34:25.697 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.S6TwbLErreyjg7mPxNXSY1vXWItv91j_0 2026-02-24T22:34:25.697 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3450500560022489683', time=2026-02-25T00:25:59.821368+0000, chain.objs.size()=1 2026-02-24T22:34:25.697 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.oQQUBS-4MXAKLzTFakJnoKGFquVCipl_0 2026-02-24T22:34:25.697 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16628502762363329457', time=2026-02-25T00:25:59.904062+0000, chain.objs.size()=1 2026-02-24T22:34:25.697 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.MP7KFnAywI4xy0bi92FK5khIetNyijR_0 2026-02-24T22:34:25.698 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13960448229457714255', time=2026-02-25T00:25:59.940428+0000, chain.objs.size()=1 2026-02-24T22:34:25.698 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.4LkvbrhxhEGoE2-a8JwTNSoSqCanYFX_0 2026-02-24T22:34:25.698 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4822443018578074366', time=2026-02-25T00:25:59.975566+0000, chain.objs.size()=1 2026-02-24T22:34:25.698 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ZOrsNuoWrC5PfmiM1NeTct0upc2L7cI_0 2026-02-24T22:34:25.698 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3340548094437057159', time=2026-02-25T00:26:00.061495+0000, chain.objs.size()=1 2026-02-24T22:34:25.698 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.rJXYPnRyzwzSdy57hYYJIKiwGnbdlC8_0 2026-02-24T22:34:25.698 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8903358183023143312', time=2026-02-25T00:26:00.092638+0000, chain.objs.size()=1 2026-02-24T22:34:25.698 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.rbjLOQKSvUVd2I2fAP_qit70pp40WH5_0 2026-02-24T22:34:25.698 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1387580281975213834', time=2026-02-25T00:26:00.104792+0000, chain.objs.size()=1 2026-02-24T22:34:25.698 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.QHFaz_RJ1inW8Ybd-drBTBQ8JU-RIVd_0 2026-02-24T22:34:25.698 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11945769650699640809', time=2026-02-25T00:26:00.128711+0000, chain.objs.size()=1 2026-02-24T22:34:25.698 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.1weLzRTfJ9VM4YlsH3svnUkyuKFEIFk_0 2026-02-24T22:34:25.698 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13248218530731778919', time=2026-02-25T00:26:00.215325+0000, chain.objs.size()=1 2026-02-24T22:34:25.698 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Wh3R6UDoPRFIK0EuUWpIemr29TjncWt_0 2026-02-24T22:34:25.698 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11753443856658096675', time=2026-02-25T00:26:00.233821+0000, chain.objs.size()=1 2026-02-24T22:34:25.698 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.D8Ld7B4Eds8RJtK-XbjXTgSLkFlUjB-_0 2026-02-24T22:34:25.698 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14971230731942032453', time=2026-02-25T00:26:00.234529+0000, chain.objs.size()=1 2026-02-24T22:34:25.698 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.J6ILmoqRyHoPGTtnSyzd23Y5vaYOoOU_0 2026-02-24T22:34:25.698 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16425707008105082794', time=2026-02-25T00:26:00.252566+0000, chain.objs.size()=1 2026-02-24T22:34:25.698 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.707+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.TbshXc4Irn417IeVpQxSRnTIpGTqxme_0 2026-02-24T22:34:25.699 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12831060565447200858', time=2026-02-25T00:26:00.266294+0000, chain.objs.size()=1 2026-02-24T22:34:25.699 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.laALVbNKQDK73ulgC0goxPdwrzqHiCA_0 2026-02-24T22:34:25.699 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6047564148238715504', time=2026-02-25T00:26:04.878135+0000, chain.objs.size()=1 2026-02-24T22:34:25.699 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1636__shadow_.d9oKEUwRIit3KdYNQeCLbnetgt75WH4_0 2026-02-24T22:34:25.699 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2314690727186878753', time=2026-02-25T00:26:05.422120+0000, chain.objs.size()=1 2026-02-24T22:34:25.699 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1637__shadow_.7fBCkumV7bXwsHn1Bi27_lZgldfh4JX_0 2026-02-24T22:34:25.699 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.65599419314266003', time=2026-02-25T00:26:06.541881+0000, chain.objs.size()=1 2026-02-24T22:34:25.699 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1639__shadow_.iSDF79ItuMwJE5w3MnSG8tMuBkh1Rgz_0 2026-02-24T22:34:25.699 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4132260817939230162', time=2026-02-25T00:26:13.602773+0000, chain.objs.size()=1 2026-02-24T22:34:25.699 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1651__shadow_.5h7AcdbQV7wukqif00Gut-5gS5Bm5pU_0 2026-02-24T22:34:25.699 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9746596885632532748', time=2026-02-25T00:26:14.214998+0000, chain.objs.size()=1 2026-02-24T22:34:25.699 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1652__shadow_.-F_lP6iVZODWUKDsBvRz-2XFlmr3kvR_0 2026-02-24T22:34:25.699 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3789191394947659415', time=2026-02-25T00:26:18.496708+0000, chain.objs.size()=1 2026-02-24T22:34:25.699 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1658__shadow_.bdERXpVAKdRkbNUBdBKHKTTLlJRqirR_0 2026-02-24T22:34:25.699 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2234921809970317295', time=2026-02-25T00:26:19.456868+0000, chain.objs.size()=1 2026-02-24T22:34:25.699 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1659__shadow_.Bd0QI_iY3oeVosOknNrmgSyhIs7bPpS_0 2026-02-24T22:34:25.699 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13412992188167197708', time=2026-02-25T00:26:44.787414+0000, chain.objs.size()=1 2026-02-24T22:34:25.699 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.NEGvSSmDTlKl3x-KCz7F9kwv5OrfOLR_0 2026-02-24T22:34:25.699 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.79727721715388365', time=2026-02-25T00:26:44.800957+0000, chain.objs.size()=1 2026-02-24T22:34:25.699 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.PA2Rp57VIgpv_gaC2pcY48Bdcl23Ktu_0 2026-02-24T22:34:25.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3404371722004114837', time=2026-02-25T00:26:44.802592+0000, chain.objs.size()=1 2026-02-24T22:34:25.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.AR5ux-a_c4asZmzrlj-zYH9VTwHhLNv_0 2026-02-24T22:34:25.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4000326086740450028', time=2026-02-25T00:26:44.808438+0000, chain.objs.size()=1 2026-02-24T22:34:25.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.VtqA2X9ijqXceuW3Ne9yRmMJaCyGUbd_0 2026-02-24T22:34:25.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17762402200270821669', time=2026-02-25T00:26:44.852951+0000, chain.objs.size()=1 2026-02-24T22:34:25.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.jDjcjk5ZEx9nxy9iZkpvE7_39JIi1K-_0 2026-02-24T22:34:25.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11478912529108352892', time=2026-02-25T00:26:44.857722+0000, chain.objs.size()=1 2026-02-24T22:34:25.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.XjTEoIrZbd_wWQgoKHgekDfc1fMP5RH_0 2026-02-24T22:34:25.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12465738772825289167', time=2026-02-25T00:26:44.943462+0000, chain.objs.size()=1 2026-02-24T22:34:25.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.EzkRX9BSG2C_nan-Hbg4imZrRKGY0E7_0 2026-02-24T22:34:25.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13981725513622317413', time=2026-02-25T00:26:44.974356+0000, chain.objs.size()=1 2026-02-24T22:34:25.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.xoOQ28eRI96MOtkC_nFV07APJbNfkVo_0 2026-02-24T22:34:25.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9758279144215045055', time=2026-02-25T00:26:44.977863+0000, chain.objs.size()=1 2026-02-24T22:34:25.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.BIY7pBUZ6TeMURBcVoD6gHNcALRTh5B_0 2026-02-24T22:34:25.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16416406913792722410', time=2026-02-25T00:26:45.008066+0000, chain.objs.size()=1 2026-02-24T22:34:25.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.oJ65ixA-sl2fK12Abd84nkpuimP6cAG_0 2026-02-24T22:34:25.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2655142682136161914', time=2026-02-25T00:26:45.011886+0000, chain.objs.size()=1 2026-02-24T22:34:25.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_._hW-hJskfm7_bDN2LNHHPKmdpqOXQiV_0 2026-02-24T22:34:25.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13968264903472073611', time=2026-02-25T00:26:45.095287+0000, chain.objs.size()=1 2026-02-24T22:34:25.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.XT-0GRomfqcTtrtE-bKjqFr3V9KSwdy_0 2026-02-24T22:34:25.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7920579245362709654', time=2026-02-25T00:26:45.103233+0000, chain.objs.size()=1 2026-02-24T22:34:25.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.j6e2GJykzHf2jZDte9bbe2ExPtg-b0V_0 2026-02-24T22:34:25.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15442662905678680896', time=2026-02-25T00:26:45.131022+0000, chain.objs.size()=1 2026-02-24T22:34:25.700 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.hK6SufyeaivM0ac8hvJE26xiIFFWvUC_0 2026-02-24T22:34:25.701 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9945052245119951807', time=2026-02-25T00:26:45.153678+0000, chain.objs.size()=1 2026-02-24T22:34:25.701 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.IMWTnEZPaR-32e36I2UU-Fn4L4E8_e8_0 2026-02-24T22:34:25.701 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9500595443055213816', time=2026-02-25T00:26:45.156387+0000, chain.objs.size()=1 2026-02-24T22:34:25.701 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.0Ge1xswCqCCJe6Kjc2RJEZgnJVSO4HU_0 2026-02-24T22:34:25.701 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8963798732866363050', time=2026-02-25T00:26:45.262721+0000, chain.objs.size()=1 2026-02-24T22:34:25.701 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.VEcHhX028UE8OJW_ROQL14FZcTaLVX6_0 2026-02-24T22:34:25.701 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2340933620814244541', time=2026-02-25T00:26:45.471896+0000, chain.objs.size()=1 2026-02-24T22:34:25.701 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.GDQYgFYCeiJAuZkrzrGgl_EW3expikj_0 2026-02-24T22:34:25.701 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12841416106839767547', time=2026-02-25T00:26:45.559273+0000, chain.objs.size()=1 2026-02-24T22:34:25.701 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.zPvnvB-3UkUaKtstXgOaIltHUNwNmLN_0 2026-02-24T22:34:25.702 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7701274583518500778', time=2026-02-25T00:26:45.573256+0000, chain.objs.size()=1 2026-02-24T22:34:25.702 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.b3Wvo7Y6o_4ROUoXWqWYIYxbgWMgDUU_0 2026-02-24T22:34:25.702 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3851882347023203196', time=2026-02-25T00:26:45.815202+0000, chain.objs.size()=1 2026-02-24T22:34:25.702 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.W0atior_bDSK3AoEaei6UNdEhAMDQuE_0 2026-02-24T22:34:25.702 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2708408896825552738', time=2026-02-25T00:26:45.824745+0000, chain.objs.size()=1 2026-02-24T22:34:25.702 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.8mJXjuzjxXMYX6sXrDJtu7X2cCVnuK5_0 2026-02-24T22:34:25.702 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6087149436411388350', time=2026-02-25T00:26:45.825950+0000, chain.objs.size()=1 2026-02-24T22:34:25.702 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.9Wl4GZ9lSRUgsqmfLN1Vl_2H-Vzh7aV_0 2026-02-24T22:34:25.702 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14735606694459375397', time=2026-02-25T00:26:45.941640+0000, chain.objs.size()=1 2026-02-24T22:34:25.702 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.XB12_TkRcIPEQ4mfIesv05iBDe_NQQW_0 2026-02-24T22:34:25.702 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13158483040289717656', time=2026-02-25T00:26:45.958618+0000, chain.objs.size()=1 2026-02-24T22:34:25.702 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.711+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.yUy6DmDik7rrr-bmJyiRsAskO99fdae_0 2026-02-24T22:34:25.702 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11564269787969088338', time=2026-02-25T00:26:45.990946+0000, chain.objs.size()=1 2026-02-24T22:34:25.702 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.OCfXtOzfzn72fYqWW1_ero7AN6MLsW1_0 2026-02-24T22:34:25.702 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16228827748592731690', time=2026-02-25T00:26:46.114130+0000, chain.objs.size()=1 2026-02-24T22:34:25.702 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.CtoVWhzL6Qm8QFUpAZvUlewwSb7tsyM_0 2026-02-24T22:34:25.703 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/37540 2026-02-24T22:34:25.705 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=49, truncated=0, next_marker='' 2026-02-24T22:34:25.705 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.844669292248746210', time=2026-02-25T00:26:46.125723+0000, chain.objs.size()=1 2026-02-24T22:34:25.705 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ZMzUJSQpIguF8hgeheZGb43XOGMu9dO_0 2026-02-24T22:34:25.705 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5893516143161452495', time=2026-02-25T00:26:46.138073+0000, chain.objs.size()=1 2026-02-24T22:34:25.705 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.c0iRFUmsw5lEbj7v2MceEtX4TLuRJYk_0 2026-02-24T22:34:25.705 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12230474976396156020', time=2026-02-25T00:26:46.144268+0000, chain.objs.size()=1 2026-02-24T22:34:25.705 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.nIx_rviH06GQ7ZP09XzWeksO1UnVc4i_0 2026-02-24T22:34:25.705 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5758951301585217964', time=2026-02-25T00:26:53.135771+0000, chain.objs.size()=1 2026-02-24T22:34:25.705 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.6c4Lr5yuAUGgS2ENxD2jaIbPvdGSpfU_0 2026-02-24T22:34:25.705 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3092900320029838554', time=2026-02-25T00:26:53.182724+0000, chain.objs.size()=1 2026-02-24T22:34:25.705 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.syvMifQ3weODjaCxNjMe5M7OwEvcmE4_0 2026-02-24T22:34:25.705 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2442605043329828736', time=2026-02-25T00:26:53.190559+0000, chain.objs.size()=1 2026-02-24T22:34:25.705 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.PZ56mDS9E_4MuVIIkvOk8Np-azNUgf7_0 2026-02-24T22:34:25.705 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15847653150575808311', time=2026-02-25T00:26:53.211854+0000, chain.objs.size()=1 2026-02-24T22:34:25.705 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.CB4JBH95-GKqWVtfI8GOTdXIs8isjkI_0 2026-02-24T22:34:25.705 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8607631257931027585', time=2026-02-25T00:26:53.215196+0000, chain.objs.size()=1 2026-02-24T22:34:25.705 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ClbWKrvW7x0NC7Z8SbYzsD9GE38mWSV_0 2026-02-24T22:34:25.705 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6968579690809334177', time=2026-02-25T00:26:53.225910+0000, chain.objs.size()=1 2026-02-24T22:34:25.705 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.W5OgKlcrBLP-jtpb4GNd3V1gleUAMVI_0 2026-02-24T22:34:25.705 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1585858562870262613', time=2026-02-25T00:26:53.403652+0000, chain.objs.size()=1 2026-02-24T22:34:25.705 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ndZB6RAXn4FAeitMggBQlYchiSv9UyQ_0 2026-02-24T22:34:25.706 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.310621893745028125', time=2026-02-25T00:26:53.505556+0000, chain.objs.size()=1 2026-02-24T22:34:25.706 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Ij8HRMYOxqjEOyD8PcerPYqiPpAM7L0_0 2026-02-24T22:34:25.706 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5922500746322330537', time=2026-02-25T00:26:53.506670+0000, chain.objs.size()=1 2026-02-24T22:34:25.706 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.OnBuC-mPgjdOUC-yRX-MeBGU77DOUd3_0 2026-02-24T22:34:25.706 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11450925712640518105', time=2026-02-25T00:26:53.537887+0000, chain.objs.size()=1 2026-02-24T22:34:25.706 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.SU332S-m6jnMlUQJ-rt_ovW3zkJgIoo_0 2026-02-24T22:34:25.706 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9625121061465640358', time=2026-02-25T00:26:53.567998+0000, chain.objs.size()=1 2026-02-24T22:34:25.706 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.s0GaIDSIXDQKLJ0TnStyXkcl3Dn9OZa_0 2026-02-24T22:34:25.706 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14355301982303498693', time=2026-02-25T00:26:53.617239+0000, chain.objs.size()=1 2026-02-24T22:34:25.706 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.QMzKcGSWbJj2O9QKpvZyz-cSuT52vHq_0 2026-02-24T22:34:25.706 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4954759910367485374', time=2026-02-25T00:26:53.755556+0000, chain.objs.size()=1 2026-02-24T22:34:25.706 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.VI_5ZkWMv877IvJXsFIWwqQOfoURGNu_0 2026-02-24T22:34:25.706 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17158549343900368275', time=2026-02-25T00:26:53.766131+0000, chain.objs.size()=1 2026-02-24T22:34:25.706 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.v17RYB4g1vkgnI8jl5h4ygOVWA6Z7n7_0 2026-02-24T22:34:25.706 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5064992135131947920', time=2026-02-25T00:26:53.773977+0000, chain.objs.size()=1 2026-02-24T22:34:25.706 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.V4LmeChaWAjd4Q4kcU0vVN6RVQtJvd2_0 2026-02-24T22:34:25.706 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5151267658743935225', time=2026-02-25T00:26:53.774522+0000, chain.objs.size()=1 2026-02-24T22:34:25.706 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.RDU76RniIb2J6VT3SsCAXPn8mOWxuk3_0 2026-02-24T22:34:25.706 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11333249141047769596', time=2026-02-25T00:26:53.795391+0000, chain.objs.size()=1 2026-02-24T22:34:25.706 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.715+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.VxZCbZEsruQdeV0hb2qCmpO9Gv_8xOt_0 2026-02-24T22:34:25.707 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17326599163602071946', time=2026-02-25T00:26:53.806145+0000, chain.objs.size()=1 2026-02-24T22:34:25.707 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.rDBSZmOyYMxJNwgaJ_De1hZ18Tfh_zk_0 2026-02-24T22:34:25.707 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10154619688934541731', time=2026-02-25T00:26:53.815799+0000, chain.objs.size()=1 2026-02-24T22:34:25.707 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.PmCTst3yqll9g__tVbVgXUM0fMfujJ-_0 2026-02-24T22:34:25.707 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16746953550819443944', time=2026-02-25T00:26:53.820039+0000, chain.objs.size()=1 2026-02-24T22:34:25.707 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Pgdr5CCImg6D2g1PNiTua0vekvc9-t6_0 2026-02-24T22:34:25.707 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1797201394701975698', time=2026-02-25T00:26:54.001916+0000, chain.objs.size()=1 2026-02-24T22:34:25.707 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.E9RI7PfWCUBI6Z-AT1bHdr4qThHV13n_0 2026-02-24T22:34:25.707 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8166615977814972735', time=2026-02-25T00:26:54.009924+0000, chain.objs.size()=1 2026-02-24T22:34:25.707 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ixXF7zxgoBdswxZLOXUIE4-ezOjXXmb_0 2026-02-24T22:34:25.707 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7725557790639866945', time=2026-02-25T00:26:54.107069+0000, chain.objs.size()=1 2026-02-24T22:34:25.707 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.1za8X-sJKfZQalGFy7GFd3ea5N0WVH2_0 2026-02-24T22:34:25.707 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11592477206420974356', time=2026-02-25T00:26:54.112394+0000, chain.objs.size()=1 2026-02-24T22:34:25.707 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.VqUfX7HwFU87ZIDS08GB8p98eAb197l_0 2026-02-24T22:34:25.707 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5418004246848963400', time=2026-02-25T00:26:54.139392+0000, chain.objs.size()=1 2026-02-24T22:34:25.707 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.yWbOtkV8AB5bO50gXA0gHwZZMjHc-91_0 2026-02-24T22:34:25.707 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18230718228442456425', time=2026-02-25T00:26:54.147079+0000, chain.objs.size()=1 2026-02-24T22:34:25.707 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Qa0T8XEeVUCs63W1Tq-484zDqcH7SkC_0 2026-02-24T22:34:25.708 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10611577960874649263', time=2026-02-25T00:26:54.285064+0000, chain.objs.size()=1 2026-02-24T22:34:25.708 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.plp_rVm81hQg3unuF87ouA6AJ59Eedg_0 2026-02-24T22:34:25.708 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12088554638675361142', time=2026-02-25T00:26:54.286529+0000, chain.objs.size()=1 2026-02-24T22:34:25.708 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ZdxBP5RanPwOdCtHEQfhYRgXaqsi14N_0 2026-02-24T22:34:25.708 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15528130230380150757', time=2026-02-25T00:26:54.306426+0000, chain.objs.size()=1 2026-02-24T22:34:25.708 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Pxgo3UdlmDL7qz3RxKb2s6tKFVXr6yk_0 2026-02-24T22:34:25.708 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14073264500714931154', time=2026-02-25T00:26:54.372111+0000, chain.objs.size()=1 2026-02-24T22:34:25.708 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.gOBi2Ok0sruC6KBqb_Gd_u4jFwFF9YW_0 2026-02-24T22:34:25.708 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13146392395841042439', time=2026-02-25T00:26:54.372851+0000, chain.objs.size()=1 2026-02-24T22:34:25.708 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.fFD4mzKrw59Vo7XINQJ8hiRtF-f4o8v_0 2026-02-24T22:34:25.708 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4703320382457687958', time=2026-02-25T00:26:54.381290+0000, chain.objs.size()=1 2026-02-24T22:34:25.708 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.zMIuFckTDaB-sF3Qn9rQxq-Y2q3s61b_0 2026-02-24T22:34:25.708 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12866572552070387522', time=2026-02-25T00:26:54.470021+0000, chain.objs.size()=1 2026-02-24T22:34:25.708 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.wBSJqy-AW-PcLD0IVoSjuEt3kR4Z_SG_0 2026-02-24T22:34:25.708 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10414771430760139910', time=2026-02-25T00:26:54.473967+0000, chain.objs.size()=1 2026-02-24T22:34:25.708 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Ia3dnF_0EFyreKJMVAIbHhE-IkC_fnn_0 2026-02-24T22:34:25.710 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14442931936598794387', time=2026-02-25T00:26:54.492076+0000, chain.objs.size()=1 2026-02-24T22:34:25.710 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Y2xcGenbKs8vghuTCeSrE7x3qBYXZrX_0 2026-02-24T22:34:25.710 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5587614244957376484', time=2026-02-25T00:26:54.504282+0000, chain.objs.size()=1 2026-02-24T22:34:25.710 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.LgNkgJu-WGAKqwZMxgJJKkzmedGy7jt_0 2026-02-24T22:34:25.710 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17612210367267226112', time=2026-02-25T00:26:54.519769+0000, chain.objs.size()=1 2026-02-24T22:34:25.710 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Izquiu3tsmdRy88eLmBjSFk2pTHTyya_0 2026-02-24T22:34:25.710 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11501035900533965773', time=2026-02-25T00:26:59.624521+0000, chain.objs.size()=1 2026-02-24T22:34:25.710 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1698__shadow_.g9yDWltsUoFf95F1o0wf4eS7dsD6mAp_0 2026-02-24T22:34:25.710 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13532167311346155889', time=2026-02-25T00:27:00.177100+0000, chain.objs.size()=1 2026-02-24T22:34:25.710 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1699__shadow_.DCVcGHjfYdXm4Lss9BmDgnuUstKEORI_0 2026-02-24T22:34:25.710 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10640643284282554669', time=2026-02-25T00:28:12.897888+0000, chain.objs.size()=1 2026-02-24T22:34:25.710 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1815__shadow_.A3n68RjDbUsWdadxaU4IxVAV1k5J7Ed_0 2026-02-24T22:34:25.710 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10322953953661280676', time=2026-02-25T00:28:24.250305+0000, chain.objs.size()=1 2026-02-24T22:34:25.710 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1830__shadow_.82IJW5WyGjURuECcZKJfHVLC9J7KKOZ_0 2026-02-24T22:34:25.710 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.295996161518926049', time=2026-02-25T00:28:33.164922+0000, chain.objs.size()=1 2026-02-24T22:34:25.710 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1850__shadow_.C_jx-IGA6h7vLZgyFCC_kWmP_qy81UF_0 2026-02-24T22:34:25.710 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11910164101776717841', time=2026-02-25T00:30:04.606341+0000, chain.objs.size()=1 2026-02-24T22:34:25.710 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1921__shadow_.I2vXgP1KUmNkLTFpJb4rD7wM2JBpKea_0 2026-02-24T22:34:25.710 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7696564460037147911', time=2026-02-25T00:30:05.410183+0000, chain.objs.size()=1 2026-02-24T22:34:25.710 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1922__shadow_.kHcYjNQCZIwj7-cFJz0D3mRlgvLO1_y_0 2026-02-24T22:34:25.710 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2807895664982681938', time=2026-02-25T00:30:24.981095+0000, chain.objs.size()=1 2026-02-24T22:34:25.710 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.719+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1939__shadow:usyJPKbLXbvc2vL0NugpteuVPc6HsHy_.50--LKYonxsQrsVXtUP-RAzv4TarWjP_0 2026-02-24T22:34:25.710 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.723+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9941988913033561943', time=2026-02-25T00:30:51.001725+0000, chain.objs.size()=1 2026-02-24T22:34:25.710 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.723+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1949__shadow:hLxxXFS2WBel-TEWiPASgexsAXQbDxr_.3gc71flDIvpSv1DIuP3t9ZF5Jei5049_0 2026-02-24T22:34:25.711 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.723+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.713 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.723+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2026-02-24T22:34:25.714 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.715 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/39463' 2026-02-24T22:34:25.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14210244994683381690', time=2026-02-25T00:15:34.624344+0000, chain.objs.size()=1 2026-02-24T22:34:25.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506__shadow_.JixsIATasPZSVbPoHz513AK0FF1w3lG_0 2026-02-24T22:34:25.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7590825751120846387', time=2026-02-25T00:15:34.656258+0000, chain.objs.size()=4 2026-02-24T22:34:25.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1513__multipart_foo.2~ATtL2jemudp08pGeq99v6rRNAWXMFEL.1 2026-02-24T22:34:25.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1513__multipart_foo.2~ATtL2jemudp08pGeq99v6rRNAWXMFEL.2 2026-02-24T22:34:25.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1513__multipart_foo.2~ATtL2jemudp08pGeq99v6rRNAWXMFEL.3 2026-02-24T22:34:25.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1513__multipart_foo.2~ATtL2jemudp08pGeq99v6rRNAWXMFEL.4 2026-02-24T22:34:25.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.498845480880988697', time=2026-02-25T00:16:32.609461+0000, chain.objs.size()=1 2026-02-24T22:34:25.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520__shadow_.EhnttfZaSX9sedib3-_8AOxRcX5Ih9c_0 2026-02-24T22:34:25.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12965956373350784002', time=2026-02-25T00:16:32.613061+0000, chain.objs.size()=1 2026-02-24T22:34:25.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520__shadow_.dU7o-pcxiOe_gPKpFhuwJZImtVC-i9O_0 2026-02-24T22:34:25.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6194995723747403052', time=2026-02-25T00:16:32.614666+0000, chain.objs.size()=1 2026-02-24T22:34:25.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520__shadow_.aSqLXvZ5mt9P4mf5E3WarD8qGpRnM8n_0 2026-02-24T22:34:25.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8889067259002700266', time=2026-02-25T00:17:05.168424+0000, chain.objs.size()=1 2026-02-24T22:34:25.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522__shadow_.tKYtGvde3MaX39aCyFaZGjCWDzuYarc_0 2026-02-24T22:34:25.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9582974595532553962', time=2026-02-25T00:17:07.117097+0000, chain.objs.size()=1 2026-02-24T22:34:25.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523__shadow_.1__Im1Yl8FIeIQE7BQqkB0NwU44o2Nz_0 2026-02-24T22:34:25.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13741835595339534140', time=2026-02-25T00:17:08.006869+0000, chain.objs.size()=1 2026-02-24T22:34:25.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524__shadow_.P3w764XVP8edj7NcEpasEvj-sRIuMzM_0 2026-02-24T22:34:25.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17944395029592078309', time=2026-02-25T00:17:39.873516+0000, chain.objs.size()=1 2026-02-24T22:34:25.716 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525__shadow_.kSeIRnKMTybvG8u-gCwnDfvWQDVss27_0 2026-02-24T22:34:25.717 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7892111809703458043', time=2026-02-25T00:18:51.091376+0000, chain.objs.size()=1 2026-02-24T22:34:25.717 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532__shadow_.PhPXXdQE7r9TjTwwOdoE0FWLK4UYbZ6_0 2026-02-24T22:34:25.717 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8840860594035888104', time=2026-02-25T00:18:51.093268+0000, chain.objs.size()=1 2026-02-24T22:34:25.717 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532__shadow_.bj3GqS2wsNbySQ-Jy4GJEY-HbqIwr22_0 2026-02-24T22:34:25.717 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5173230594705705382', time=2026-02-25T00:19:23.061120+0000, chain.objs.size()=1 2026-02-24T22:34:25.717 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533__shadow_.VIxrZuOdfymD9E1pQPvvyZoYAJvvbzz_0 2026-02-24T22:34:25.717 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13213407385176480129', time=2026-02-25T00:19:25.017642+0000, chain.objs.size()=1 2026-02-24T22:34:25.717 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534__shadow_.XsMGvHZKP2r9-eh1S5k77ZXqKDJ3Nyt_0 2026-02-24T22:34:25.717 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5276014003693439272', time=2026-02-25T00:19:25.021134+0000, chain.objs.size()=1 2026-02-24T22:34:25.717 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534__shadow_.QLE5hH6uwoONmG1KpbezKl74IAshUhu_0 2026-02-24T22:34:25.717 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7236180641404243911', time=2026-02-25T00:23:14.349734+0000, chain.objs.size()=1 2026-02-24T22:34:25.717 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518__shadow_.DJQGrtlQ0OhuEss7M7y_lzQra-B7cP5_0 2026-02-24T22:34:25.717 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18285592373358629570', time=2026-02-25T00:23:14.350668+0000, chain.objs.size()=1 2026-02-24T22:34:25.717 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518__shadow_.dC6UacgrECAN5efRQbooL9cSaJ-mDaS_0 2026-02-24T22:34:25.717 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9599833436169832072', time=2026-02-25T00:23:25.729232+0000, chain.objs.size()=1 2026-02-24T22:34:25.717 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1519__shadow:iLZDaUYPEobIBPz.JUqE1EDNhsOeB3V_.igd98596FNfUTGkWzLTih4NrPhzZfCw_0 2026-02-24T22:34:25.717 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4555633802007692118', time=2026-02-25T00:25:40.782603+0000, chain.objs.size()=1 2026-02-24T22:34:25.717 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1608__shadow_.5UyZD2hQGeNDwlesAkRi5CHOZ5BB7iS_0 2026-02-24T22:34:25.717 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16770325395255997388', time=2026-02-25T00:25:43.238994+0000, chain.objs.size()=1 2026-02-24T22:34:25.717 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1611__shadow_.11vrCSpAPVJQH1lqJnbczd-n1WsX2x3_0 2026-02-24T22:34:25.717 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5753944496621248629', time=2026-02-25T00:25:43.871161+0000, chain.objs.size()=1 2026-02-24T22:34:25.717 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1612__shadow_.MoSJFY_mTOa4xv7EMAbZroPuT4vdU9G_0 2026-02-24T22:34:25.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6165006814082106447', time=2026-02-25T00:25:48.506705+0000, chain.objs.size()=1 2026-02-24T22:34:25.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1620__shadow_.uQ56d32rzlrw18LWAc7lRPxKCjEDvve_0 2026-02-24T22:34:25.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7508499923227254684', time=2026-02-25T00:25:50.257900+0000, chain.objs.size()=1 2026-02-24T22:34:25.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1623__shadow_.3Ya_RSX3UKHI-NGk1MCA4AbsbtEleF6_0 2026-02-24T22:34:25.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6902910101917783205', time=2026-02-25T00:25:59.072498+0000, chain.objs.size()=1 2026-02-24T22:34:25.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.F99iup-sTEGPVgU0GwHS5DxS-9VHRLZ_0 2026-02-24T22:34:25.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9110847505681750665', time=2026-02-25T00:25:59.083006+0000, chain.objs.size()=1 2026-02-24T22:34:25.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.nDVAsZwIDrVolJzl-saceynrpbHogeP_0 2026-02-24T22:34:25.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8790477278444809278', time=2026-02-25T00:25:59.087844+0000, chain.objs.size()=1 2026-02-24T22:34:25.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.BteZfMFsuhnTFmEjJuxxeTERn1Vn_PZ_0 2026-02-24T22:34:25.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10052881882884954400', time=2026-02-25T00:25:59.102322+0000, chain.objs.size()=1 2026-02-24T22:34:25.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.1YAEa7Edz0xz2qUhGfS4hBZv-EBWdWL_0 2026-02-24T22:34:25.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.170780248415353122', time=2026-02-25T00:25:59.140551+0000, chain.objs.size()=1 2026-02-24T22:34:25.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.x7sLzpFiw-urbqcd4omoIuFP8gLefjD_0 2026-02-24T22:34:25.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3348623535813501656', time=2026-02-25T00:25:59.155054+0000, chain.objs.size()=1 2026-02-24T22:34:25.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.727+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.xGHu6pkQRfiGL9zstEQvhfL9aD5qgEH_0 2026-02-24T22:34:25.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8803731537424007957', time=2026-02-25T00:25:59.249240+0000, chain.objs.size()=1 2026-02-24T22:34:25.718 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.SNhuAoMhZTy2B4ZI2SKEWNBWw4ZyctA_0 2026-02-24T22:34:25.719 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10035660707381210706', time=2026-02-25T00:25:59.254027+0000, chain.objs.size()=1 2026-02-24T22:34:25.719 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.T-AT9fD4X5YQUJxtJ3VmQE_I4MWSiac_0 2026-02-24T22:34:25.719 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4106337813831876413', time=2026-02-25T00:25:59.259581+0000, chain.objs.size()=1 2026-02-24T22:34:25.719 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.IfLFjyyWmxLj5y0LoVAk89Uw2CnJ5Tm_0 2026-02-24T22:34:25.719 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5948790294397020944', time=2026-02-25T00:25:59.408640+0000, chain.objs.size()=1 2026-02-24T22:34:25.719 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.0S_q3qaSXIdEkR_K1Q_039fJ02pna4Z_0 2026-02-24T22:34:25.719 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9184025988822720227', time=2026-02-25T00:25:59.452053+0000, chain.objs.size()=1 2026-02-24T22:34:25.719 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.HZOKc8jXCJWbkUvFY66kY1PdcmRzrqW_0 2026-02-24T22:34:25.719 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7868608525354161475', time=2026-02-25T00:25:59.568323+0000, chain.objs.size()=1 2026-02-24T22:34:25.719 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.QrXLyUjdMSxrb-IjmGWgfaCz5znG6_7_0 2026-02-24T22:34:25.719 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3058804713035527970', time=2026-02-25T00:25:59.582583+0000, chain.objs.size()=1 2026-02-24T22:34:25.719 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.GNxUVLWDHIRYA3_EGYcFvmImSHbz8FV_0 2026-02-24T22:34:25.719 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2230282169601332341', time=2026-02-25T00:25:59.592025+0000, chain.objs.size()=1 2026-02-24T22:34:25.719 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.jM3j4cYMQhK1rV1ufSgb_2sSBaocGEw_0 2026-02-24T22:34:25.719 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.353890572547387790', time=2026-02-25T00:25:59.594634+0000, chain.objs.size()=1 2026-02-24T22:34:25.719 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.e94-a92T34xXFv6egLHPHK6vwWLpDDk_0 2026-02-24T22:34:25.719 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4435112984177301964', time=2026-02-25T00:25:59.753510+0000, chain.objs.size()=1 2026-02-24T22:34:25.719 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.2sOruK_wnKn70Kboyu5YUJCZOCyLeUb_0 2026-02-24T22:34:25.719 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16675840763396852701', time=2026-02-25T00:25:59.779793+0000, chain.objs.size()=1 2026-02-24T22:34:25.719 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.wJKFGUPRewdSwWHyobakUgHNLBIzCpr_0 2026-02-24T22:34:25.720 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1779108469306792125', time=2026-02-25T00:25:59.785565+0000, chain.objs.size()=1 2026-02-24T22:34:25.720 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.5dULl2ZQ4PAmylRVAaBbK8wzHSsiVbH_0 2026-02-24T22:34:25.720 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1976592797809311856', time=2026-02-25T00:25:59.797106+0000, chain.objs.size()=1 2026-02-24T22:34:25.720 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.KXmCn4oLk7Puw3kpEcDZ4uObTlEHC0q_0 2026-02-24T22:34:25.720 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1180145896673195127', time=2026-02-25T00:25:59.817854+0000, chain.objs.size()=1 2026-02-24T22:34:25.720 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.18Str2RqUC7HWc0x32Jqr7wVhuRBYkR_0 2026-02-24T22:34:25.720 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9672147045715812261', time=2026-02-25T00:25:59.902185+0000, chain.objs.size()=1 2026-02-24T22:34:25.720 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.FAEO8jGcxoGGar4WQfQ4NSa8B6D6wnb_0 2026-02-24T22:34:25.720 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14588101676582712508', time=2026-02-25T00:25:59.950105+0000, chain.objs.size()=1 2026-02-24T22:34:25.720 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.6L3nfFuyFD9g5eVjP0RnG5eHtKHWvbd_0 2026-02-24T22:34:25.720 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8085533533356247985', time=2026-02-25T00:26:00.053929+0000, chain.objs.size()=1 2026-02-24T22:34:25.720 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Ogh4eA5GQXvXt-Oh6NvrZJyqmUv3YKL_0 2026-02-24T22:34:25.720 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14535498322268180994', time=2026-02-25T00:26:00.088743+0000, chain.objs.size()=1 2026-02-24T22:34:25.720 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.xO14KSDh-SnehfdZP-i90-PQKDvZ71l_0 2026-02-24T22:34:25.720 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17117600850357569694', time=2026-02-25T00:26:00.090132+0000, chain.objs.size()=1 2026-02-24T22:34:25.720 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.izdfPycnjEoIjYcngRaR2uoeqkwB8gO_0 2026-02-24T22:34:25.720 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.410843901632284547', time=2026-02-25T00:26:00.102074+0000, chain.objs.size()=1 2026-02-24T22:34:25.720 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.bf8-XTC-221MrDfTZJgywkO4upK3tb6_0 2026-02-24T22:34:25.720 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6906744867706935153', time=2026-02-25T00:26:00.103424+0000, chain.objs.size()=1 2026-02-24T22:34:25.720 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.W0bpbZEYoxMpzsK2UAwd_NvNp5eQoki_0 2026-02-24T22:34:25.720 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9338352232648726147', time=2026-02-25T00:26:00.135193+0000, chain.objs.size()=1 2026-02-24T22:34:25.720 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.D6iWdiGfzXT_J_QTRb2vXw6vXGhenXE_0 2026-02-24T22:34:25.721 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11511584940307366380', time=2026-02-25T00:26:00.207946+0000, chain.objs.size()=1 2026-02-24T22:34:25.721 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.atoguwy1Uu7DK_xqewaz2BKt488PkN8_0 2026-02-24T22:34:25.721 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6223742632968124436', time=2026-02-25T00:26:00.216715+0000, chain.objs.size()=1 2026-02-24T22:34:25.721 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.3Nm2AKMbX8s6SuWBTUsVPqQL99DY9mh_0 2026-02-24T22:34:25.721 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13015627257814046919', time=2026-02-25T00:26:00.217900+0000, chain.objs.size()=1 2026-02-24T22:34:25.721 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.YPLRF-L_ZIC5Lhkym2rkrJDadaVO97j_0 2026-02-24T22:34:25.721 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17163530344498877556', time=2026-02-25T00:26:00.243018+0000, chain.objs.size()=1 2026-02-24T22:34:25.721 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.eKUxNcXJTA8agBfqXTxMAR-PfBSIvpM_0 2026-02-24T22:34:25.721 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11497266038651021650', time=2026-02-25T00:26:00.273068+0000, chain.objs.size()=1 2026-02-24T22:34:25.721 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.VaVnJSx7hjhBxTnZvYFjRI6SDKXPfnN_0 2026-02-24T22:34:25.721 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12183701443709236985', time=2026-02-25T00:26:18.498577+0000, chain.objs.size()=1 2026-02-24T22:34:25.721 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1658__shadow_.xc-2ZU-RV7FTLpaPSw1vIrX6CRoJxgm_0 2026-02-24T22:34:25.722 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9175141796628125059', time=2026-02-25T00:26:26.414425+0000, chain.objs.size()=1 2026-02-24T22:34:25.722 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1671__shadow_.YLnPYZdWm5BRA_6GF0BaUkkM-zyl7_x_0 2026-02-24T22:34:25.722 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2248686727377269362', time=2026-02-25T00:26:35.760017+0000, chain.objs.size()=1 2026-02-24T22:34:25.722 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:zFUvgf6oNcCIOOC6HEG3twyc8tHuYJG_.odBRLQF6Jq2HiPU0HxV9582TchzkdqB_0 2026-02-24T22:34:25.722 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2248686727377269362', time=2026-02-25T00:26:35.783322+0000, chain.objs.size()=1 2026-02-24T22:34:25.722 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:zFUvgf6oNcCIOOC6HEG3twyc8tHuYJG_.odBRLQF6Jq2HiPU0HxV9582TchzkdqB_0 2026-02-24T22:34:25.722 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2248686727377269362', time=2026-02-25T00:26:35.830608+0000, chain.objs.size()=1 2026-02-24T22:34:25.722 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:zFUvgf6oNcCIOOC6HEG3twyc8tHuYJG_.odBRLQF6Jq2HiPU0HxV9582TchzkdqB_0 2026-02-24T22:34:25.722 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2248686727377269362', time=2026-02-25T00:26:35.846478+0000, chain.objs.size()=1 2026-02-24T22:34:25.722 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:zFUvgf6oNcCIOOC6HEG3twyc8tHuYJG_.odBRLQF6Jq2HiPU0HxV9582TchzkdqB_0 2026-02-24T22:34:25.722 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2248686727377269362', time=2026-02-25T00:26:35.872708+0000, chain.objs.size()=1 2026-02-24T22:34:25.722 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:zFUvgf6oNcCIOOC6HEG3twyc8tHuYJG_.odBRLQF6Jq2HiPU0HxV9582TchzkdqB_0 2026-02-24T22:34:25.722 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17874111091444377364', time=2026-02-25T00:26:44.777572+0000, chain.objs.size()=1 2026-02-24T22:34:25.722 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.731+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.bvOcvVwaMaCpfE-WCM9oS_Y5uOay5Wb_0 2026-02-24T22:34:25.722 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4595387181483322633', time=2026-02-25T00:26:44.802422+0000, chain.objs.size()=1 2026-02-24T22:34:25.722 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.hWkqy1s6ToQQGH9rkzULvoj9MeeqJY-_0 2026-02-24T22:34:25.722 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6902084007885974106', time=2026-02-25T00:26:44.849679+0000, chain.objs.size()=1 2026-02-24T22:34:25.722 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.lxCYV_NYRb_dgfS-IFy1FfKz-ogck8h_0 2026-02-24T22:34:25.722 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4631367026850234396', time=2026-02-25T00:26:44.943973+0000, chain.objs.size()=1 2026-02-24T22:34:25.723 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.iaI2JPG2EPWEaj0iIqIJkoB0kKStoz5_0 2026-02-24T22:34:25.723 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16766469790122449528', time=2026-02-25T00:26:44.957078+0000, chain.objs.size()=1 2026-02-24T22:34:25.723 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.662_o8G3llCyx-mYkhG8H3xWjdH7FVC_0 2026-02-24T22:34:25.723 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17100269353460537678', time=2026-02-25T00:26:44.968789+0000, chain.objs.size()=1 2026-02-24T22:34:25.723 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.M4T_Q-fqdkvJrMEG0ATUqBGABoYKPxY_0 2026-02-24T22:34:25.723 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.37079876704022800', time=2026-02-25T00:26:44.976451+0000, chain.objs.size()=1 2026-02-24T22:34:25.723 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.E75ZAzsh5djhd5OD58fkaYYX3mdIeeM_0 2026-02-24T22:34:25.723 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12671290425152541055', time=2026-02-25T00:26:45.110906+0000, chain.objs.size()=1 2026-02-24T22:34:25.723 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.-8Z0IsRbOW81f1qgoeM1yAtzV1NZ4xF_0 2026-02-24T22:34:25.723 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2610667913391802094', time=2026-02-25T00:26:45.117431+0000, chain.objs.size()=1 2026-02-24T22:34:25.723 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Px1e0ZCdAzyxzWyVKxHI5gsyTyEKC4c_0 2026-02-24T22:34:25.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1472913795923440459', time=2026-02-25T00:26:45.129852+0000, chain.objs.size()=1 2026-02-24T22:34:25.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.NYD9J0x-AznDnvZn5ZsZ7AD1fU1tdwU_0 2026-02-24T22:34:25.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13381572817712588314', time=2026-02-25T00:26:45.153117+0000, chain.objs.size()=1 2026-02-24T22:34:25.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.8qdQYHLPmMxKLX0TnZPaRYIJ9GgGEh0_0 2026-02-24T22:34:25.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1337201716655137044', time=2026-02-25T00:26:45.159504+0000, chain.objs.size()=1 2026-02-24T22:34:25.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.THEFVW--kZn86_7R-haLl06TJ2gh-bu_0 2026-02-24T22:34:25.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7996123369905328951', time=2026-02-25T00:26:45.164863+0000, chain.objs.size()=1 2026-02-24T22:34:25.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.uGpoxED5OdzmaBdNYZruBOUWyWtPRSY_0 2026-02-24T22:34:25.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6945885376323234486', time=2026-02-25T00:26:45.249515+0000, chain.objs.size()=1 2026-02-24T22:34:25.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.W5ujFUxYOuCpA1a25iNAOlgIK2M3ezs_0 2026-02-24T22:34:25.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4621744534737349675', time=2026-02-25T00:26:45.274867+0000, chain.objs.size()=1 2026-02-24T22:34:25.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.6TL4h9CRMlYkChan8Pghr9TZMHDwZKd_0 2026-02-24T22:34:25.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5365734115884756602', time=2026-02-25T00:26:45.275270+0000, chain.objs.size()=1 2026-02-24T22:34:25.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.mrmBJlSKYLfNxFUKXctqxu_N2o03JbH_0 2026-02-24T22:34:25.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12332380315398600046', time=2026-02-25T00:26:45.282834+0000, chain.objs.size()=1 2026-02-24T22:34:25.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.pVGKnvAp3E47v4Tzy_URs8hnS5A4Gat_0 2026-02-24T22:34:25.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4389469932236433860', time=2026-02-25T00:26:45.407080+0000, chain.objs.size()=1 2026-02-24T22:34:25.724 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.2kdequOgM3FOOxkyTGJ_x1Zj5PTwew8_0 2026-02-24T22:34:25.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14336204161988789241', time=2026-02-25T00:26:45.407649+0000, chain.objs.size()=1 2026-02-24T22:34:25.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.6J7gg5qYmaqZZO8OBsSSNJ6f1L1vUCL_0 2026-02-24T22:34:25.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18120809558981279189', time=2026-02-25T00:26:45.427149+0000, chain.objs.size()=1 2026-02-24T22:34:25.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Yt08QAD24WfEZNuVtSBJGxiGT3ZZyr0_0 2026-02-24T22:34:25.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13768387565822114306', time=2026-02-25T00:26:45.456713+0000, chain.objs.size()=1 2026-02-24T22:34:25.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.n4jKAsqoXugmuaQWKZJZAqQM8AO3vJ2_0 2026-02-24T22:34:25.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5049977673111329243', time=2026-02-25T00:26:45.482833+0000, chain.objs.size()=1 2026-02-24T22:34:25.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.oJvBaPN2hQQyfdlxJ6pc0UbFOR25kBD_0 2026-02-24T22:34:25.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2568854365863976088', time=2026-02-25T00:26:45.562878+0000, chain.objs.size()=1 2026-02-24T22:34:25.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.WqV4VuVy8ZlbYJDa5po1k9wYJHbE-f5_0 2026-02-24T22:34:25.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16368894862142484422', time=2026-02-25T00:26:45.567905+0000, chain.objs.size()=1 2026-02-24T22:34:25.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Yr0lEUtdwm4GCJwqm11x5ZRxs3UBdjH_0 2026-02-24T22:34:25.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17503341221170379380', time=2026-02-25T00:26:45.596286+0000, chain.objs.size()=1 2026-02-24T22:34:25.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.RsnPy5OzD-4Qo4jzisxEM3yuyWuV6Is_0 2026-02-24T22:34:25.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9074062564542476044', time=2026-02-25T00:26:45.807821+0000, chain.objs.size()=1 2026-02-24T22:34:25.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.gJx5YtoQq72iE4YVFTyPuTYCcC-QgDs_0 2026-02-24T22:34:25.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18098637772157936617', time=2026-02-25T00:26:45.912510+0000, chain.objs.size()=1 2026-02-24T22:34:25.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.JBogFqrsYWIytbjDLJYmSHPhGpvtV6A_0 2026-02-24T22:34:25.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4245310715770728931', time=2026-02-25T00:26:45.944150+0000, chain.objs.size()=1 2026-02-24T22:34:25.725 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.735+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.2QeyTPqkoFjY6t1WQ38PBNj6ba9c1eR_0 2026-02-24T22:34:25.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13639848421537310837', time=2026-02-25T00:26:45.971242+0000, chain.objs.size()=1 2026-02-24T22:34:25.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.oAFP3QqZ-ieYC4hHtktnOe2g_6gzh0r_0 2026-02-24T22:34:25.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18028321026645815191', time=2026-02-25T00:26:45.975616+0000, chain.objs.size()=1 2026-02-24T22:34:25.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.TpYuHUf7to0RfJq_7kzUOWc11aEG2Rq_0 2026-02-24T22:34:25.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10068551027598707363', time=2026-02-25T00:26:46.076024+0000, chain.objs.size()=1 2026-02-24T22:34:25.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.GCDTH7SpdWFMYbpTxrzTt7kYePStH5B_0 2026-02-24T22:34:25.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12505311384302771148', time=2026-02-25T00:26:46.080936+0000, chain.objs.size()=1 2026-02-24T22:34:25.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.P9F6t6SUfoM0GF38GDP6Qm2WhQbQ093_0 2026-02-24T22:34:25.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1906267511687888849', time=2026-02-25T00:26:46.106480+0000, chain.objs.size()=1 2026-02-24T22:34:25.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.WNz6PxfXsijH9D5XtXPPonL05GQVb8H_0 2026-02-24T22:34:25.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4530411909553266515', time=2026-02-25T00:26:46.133252+0000, chain.objs.size()=1 2026-02-24T22:34:25.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.aP_fVgFhFRjGA5ubaeVmFjWOXK29fif_0 2026-02-24T22:34:25.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13584573353319048957', time=2026-02-25T00:26:53.169199+0000, chain.objs.size()=1 2026-02-24T22:34:25.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.9MyhC6d1kjlYuFZKIF5rQQ--pvFfD8P_0 2026-02-24T22:34:25.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17424872941898245502', time=2026-02-25T00:26:53.191315+0000, chain.objs.size()=1 2026-02-24T22:34:25.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.WuOxQSSg1Wsrp2BmXv_PlcS6HNHvXs__0 2026-02-24T22:34:25.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16397192563660915013', time=2026-02-25T00:26:53.200340+0000, chain.objs.size()=1 2026-02-24T22:34:25.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.CXioGsgIM5fz3PSW1Xeuq9toPhtTnJK_0 2026-02-24T22:34:25.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10409212602214708307', time=2026-02-25T00:26:53.206075+0000, chain.objs.size()=1 2026-02-24T22:34:25.727 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.kkUIl89As5ldXmH8hDQfNpMBI8agtiG_0 2026-02-24T22:34:25.728 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/39463 2026-02-24T22:34:25.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=59, truncated=0, next_marker='' 2026-02-24T22:34:25.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9783703848759769237', time=2026-02-25T00:26:53.213716+0000, chain.objs.size()=1 2026-02-24T22:34:25.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.icDX4mdgHB1yotEpeUqCZgUSX1iRlBs_0 2026-02-24T22:34:25.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5466408943570048957', time=2026-02-25T00:26:53.230837+0000, chain.objs.size()=1 2026-02-24T22:34:25.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.EzZE-DP74LrhuzRmIX5NyqiaxbfNjQ5_0 2026-02-24T22:34:25.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1005222121296059076', time=2026-02-25T00:26:53.242231+0000, chain.objs.size()=1 2026-02-24T22:34:25.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.6bs16ii1WO3glx7DIOZK3ES_zDEJow8_0 2026-02-24T22:34:25.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5482871798008779779', time=2026-02-25T00:26:53.246636+0000, chain.objs.size()=1 2026-02-24T22:34:25.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.DLPzcwmN3DNK87t0mvbFnapKmQGbaME_0 2026-02-24T22:34:25.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.63909214050111540', time=2026-02-25T00:26:53.320694+0000, chain.objs.size()=1 2026-02-24T22:34:25.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.2anrnduUs70JnyDjtkxJMlPGUPqKWYL_0 2026-02-24T22:34:25.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15004653267196829712', time=2026-02-25T00:26:53.322967+0000, chain.objs.size()=1 2026-02-24T22:34:25.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.G7KAMnFRbp0xzNg2WF_mBpue05MIqJx_0 2026-02-24T22:34:25.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521258796426950328', time=2026-02-25T00:26:53.324618+0000, chain.objs.size()=1 2026-02-24T22:34:25.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.SR2yG3PhTNe6T_ufk6egCE6uQMtKzRr_0 2026-02-24T22:34:25.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15132733441242283050', time=2026-02-25T00:26:53.344766+0000, chain.objs.size()=1 2026-02-24T22:34:25.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_._iOjgjnN3uY8_AajvRKgNQw6qFovbBM_0 2026-02-24T22:34:25.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7944719733898953541', time=2026-02-25T00:26:53.345263+0000, chain.objs.size()=1 2026-02-24T22:34:25.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.kHaWc3bgNZ1zPjojLCX8BaPJRsG5HcQ_0 2026-02-24T22:34:25.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3265628119426131394', time=2026-02-25T00:26:53.367504+0000, chain.objs.size()=1 2026-02-24T22:34:25.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Upj-zyQqzo0L3q8p_sSyH-UaeYDL0Sq_0 2026-02-24T22:34:25.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3215693184749652150', time=2026-02-25T00:26:53.372693+0000, chain.objs.size()=1 2026-02-24T22:34:25.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.WAFsQ6sdSU0XD096m6VvjvwLslc5gAs_0 2026-02-24T22:34:25.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7245134648838582915', time=2026-02-25T00:26:53.428715+0000, chain.objs.size()=1 2026-02-24T22:34:25.730 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.739+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.iGLPp7kXxgoZifY8LsnyG-xyzTYUtrq_0 2026-02-24T22:34:25.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17774565576264855896', time=2026-02-25T00:26:53.533397+0000, chain.objs.size()=1 2026-02-24T22:34:25.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.cs2QUU0CihXj9mMcPEWW7GcyTWkIAWq_0 2026-02-24T22:34:25.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9598717944976959887', time=2026-02-25T00:26:53.540645+0000, chain.objs.size()=1 2026-02-24T22:34:25.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.4VOyqMaQXpnU6oImkJ5SHoAuxoTYyx1_0 2026-02-24T22:34:25.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9338213995897049278', time=2026-02-25T00:26:53.556343+0000, chain.objs.size()=1 2026-02-24T22:34:25.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.xaLUoZSb8a4gwkX1RV8ZrmPxAu4HJfr_0 2026-02-24T22:34:25.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9191137870368358189', time=2026-02-25T00:26:53.559089+0000, chain.objs.size()=1 2026-02-24T22:34:25.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.EUuf1UPcbitPl1ygLuEbwXKujI0erMo_0 2026-02-24T22:34:25.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10376461036296497692', time=2026-02-25T00:26:53.588679+0000, chain.objs.size()=1 2026-02-24T22:34:25.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.XKc_PWXOFxfkxC8fGjnueFZ2xNnP2Ze_0 2026-02-24T22:34:25.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16970096738726081165', time=2026-02-25T00:26:53.722110+0000, chain.objs.size()=1 2026-02-24T22:34:25.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Q0Vrp6eat_mymk8auzP33vniR87vrhh_0 2026-02-24T22:34:25.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12886896017466222147', time=2026-02-25T00:26:53.750151+0000, chain.objs.size()=1 2026-02-24T22:34:25.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.FqUrjtIm6o-ts79GopJREYE0vm8omhF_0 2026-02-24T22:34:25.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6684785416410066004', time=2026-02-25T00:26:53.808531+0000, chain.objs.size()=1 2026-02-24T22:34:25.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.n1hGhyTsvQl5rWAwjuh2yog13fiFC25_0 2026-02-24T22:34:25.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2722540270152339696', time=2026-02-25T00:26:53.909649+0000, chain.objs.size()=1 2026-02-24T22:34:25.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.9kGsEr-cV72hKGkK2c1l3GlsDhRcYDm_0 2026-02-24T22:34:25.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15730852890165478659', time=2026-02-25T00:26:53.913911+0000, chain.objs.size()=1 2026-02-24T22:34:25.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.LiPeuarjYWyCPx2JA258HR6Ufxxl3U6_0 2026-02-24T22:34:25.731 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1356607258752291837', time=2026-02-25T00:26:53.914409+0000, chain.objs.size()=1 2026-02-24T22:34:25.732 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.fp9_C8VuNI0v6c_Es3NiONvJpk6_-7w_0 2026-02-24T22:34:25.732 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13286828299643605667', time=2026-02-25T00:26:53.935153+0000, chain.objs.size()=1 2026-02-24T22:34:25.732 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.mT90Qvmd2mfzuTfUk6X4TtktnPZg2x4_0 2026-02-24T22:34:25.732 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17780298839829600136', time=2026-02-25T00:26:53.940453+0000, chain.objs.size()=1 2026-02-24T22:34:25.732 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.J1yZBToq5i9fO7O_uSfPjrovng2IjSC_0 2026-02-24T22:34:25.732 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4312054875234036979', time=2026-02-25T00:26:54.005218+0000, chain.objs.size()=1 2026-02-24T22:34:25.732 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.4CB8y19rsSkc5TAiFCDChpuUpGJVZWB_0 2026-02-24T22:34:25.732 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10184330677681196082', time=2026-02-25T00:26:54.021255+0000, chain.objs.size()=1 2026-02-24T22:34:25.732 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.m99HfKUNfP7BXMYt7l4yxYezLLtkEIx_0 2026-02-24T22:34:25.732 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17991984983968749325', time=2026-02-25T00:26:54.151441+0000, chain.objs.size()=1 2026-02-24T22:34:25.732 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.YsqZ5fkCHRTdPqNUeRNscT5g2msQsu-_0 2026-02-24T22:34:25.732 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6828031232213028313', time=2026-02-25T00:26:54.152047+0000, chain.objs.size()=1 2026-02-24T22:34:25.732 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.k3i7GJhst4uF8h1bttiKb9VBXRC-EXS_0 2026-02-24T22:34:25.732 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6381422716299267088', time=2026-02-25T00:26:54.159761+0000, chain.objs.size()=1 2026-02-24T22:34:25.732 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.spSqxpk1CQZxXLEbT00DqnkZO-LoU9Q_0 2026-02-24T22:34:25.732 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17502820531768010029', time=2026-02-25T00:26:54.167039+0000, chain.objs.size()=1 2026-02-24T22:34:25.732 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.xrLv_ijDB0-SGTmJBFwhGbq5JAMNR9C_0 2026-02-24T22:34:25.733 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14982696277663813912', time=2026-02-25T00:26:54.282585+0000, chain.objs.size()=1 2026-02-24T22:34:25.733 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.bFlgT9zCnm-IRCZOFZRRdaevdH0NSui_0 2026-02-24T22:34:25.733 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12365016695835013100', time=2026-02-25T00:26:54.340923+0000, chain.objs.size()=1 2026-02-24T22:34:25.733 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.VBlRCqDUxDQ9QVwy5zOrftdZ8_p-5yB_0 2026-02-24T22:34:25.733 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15426730316346839352', time=2026-02-25T00:26:54.359417+0000, chain.objs.size()=1 2026-02-24T22:34:25.733 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.jdXKC41XnQtO-0cDCL-XqfOkddLYQ_g_0 2026-02-24T22:34:25.733 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10836271603305944713', time=2026-02-25T00:26:54.377675+0000, chain.objs.size()=1 2026-02-24T22:34:25.733 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.2E23Pgoe3ZVN6VNq3VFZEftENWnbmwp_0 2026-02-24T22:34:25.733 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8143095131206971533', time=2026-02-25T00:26:54.483402+0000, chain.objs.size()=1 2026-02-24T22:34:25.733 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.z6iZo9UFKhhJsqEwrLk3Nl-7AcJLTWQ_0 2026-02-24T22:34:25.733 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7890431800771471230', time=2026-02-25T00:26:54.491539+0000, chain.objs.size()=1 2026-02-24T22:34:25.733 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ZXo60hQwDZw4cn8wZADVqwJ8upSkshW_0 2026-02-24T22:34:25.733 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6925174078788072614', time=2026-02-25T00:26:54.507609+0000, chain.objs.size()=1 2026-02-24T22:34:25.733 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.4ps9QN-Iq7SvP5pFB6i8-P2OdxkkmR3_0 2026-02-24T22:34:25.733 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14917861503803411672', time=2026-02-25T00:27:24.180758+0000, chain.objs.size()=1 2026-02-24T22:34:25.733 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1742__shadow_.gilxOPzEeipUKXpS7wkCjR0Mjq6vaux_0 2026-02-24T22:34:25.733 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8437123746368239981', time=2026-02-25T00:28:31.276115+0000, chain.objs.size()=1 2026-02-24T22:34:25.733 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1848__shadow_.ttYiIsFP42bY7Vcj1abPzvMevTTwYfc_0 2026-02-24T22:34:25.734 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7722556300124038928', time=2026-02-25T00:28:35.113061+0000, chain.objs.size()=1 2026-02-24T22:34:25.734 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1853__shadow_.Y0h9SDZ-1Cc8jpeThxa2PRTp9RUo8Au_0 2026-02-24T22:34:25.734 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1236581467693840878', time=2026-02-25T00:28:39.079178+0000, chain.objs.size()=1 2026-02-24T22:34:25.734 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1861__shadow_.drMWqH-tPXns6x4CaZzP8HfQIavyVlP_0 2026-02-24T22:34:25.734 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11731058659829736775', time=2026-02-25T00:28:40.520017+0000, chain.objs.size()=1 2026-02-24T22:34:25.734 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1863__shadow_.lRJujtd9Ru3sUbNJNOL6hQNd7RO314M_0 2026-02-24T22:34:25.734 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7617982178555966075', time=2026-02-25T00:28:40.520706+0000, chain.objs.size()=1 2026-02-24T22:34:25.734 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1863__shadow_.lRJujtd9Ru3sUbNJNOL6hQNd7RO314M_0 2026-02-24T22:34:25.734 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11934593787756482685', time=2026-02-25T00:28:51.379583+0000, chain.objs.size()=1 2026-02-24T22:34:25.734 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1884__multipart_mymultipart.2~z0lYEp_I3gCM3cXOtIAQqkzruZD_j02.1 2026-02-24T22:34:25.734 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10196895080057323982', time=2026-02-25T00:30:19.694690+0000, chain.objs.size()=1 2026-02-24T22:34:25.734 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.743+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:uJOYYvOyGPwCSuh50G.43o0yh6012v2_.vtfMIxd28r9JozCJ15_9lgvuKQWHfg6_0 2026-02-24T22:34:25.734 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6727417833778177217', time=2026-02-25T00:30:19.843157+0000, chain.objs.size()=1 2026-02-24T22:34:25.734 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:grtrhEG0dv-M57l1drfZaoa-JX8SC5K_.bEehBxIez_Ib6C7mZnZht_QJCEb6kyH_0 2026-02-24T22:34:25.734 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6224110461140731988', time=2026-02-25T00:30:21.311110+0000, chain.objs.size()=1 2026-02-24T22:34:25.734 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:IO5FzoCpZpgyOsZyp0MFdUFo2CvzpPs_.KtF8n0NJ-zWmzEqqWVmMOzB9N53USgH_0 2026-02-24T22:34:25.734 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0c9fe6640 0 -- 10.20.193.46:0/1777517895 >> [v2:10.20.193.110:6816/2680757198,v1:10.20.193.110:6818/2680757198] conn(0x55661f02f180 msgr2=0x7fa08c161f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2026-02-24T22:34:25.734 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16250841698427779668', time=2026-02-25T00:30:21.495056+0000, chain.objs.size()=1 2026-02-24T22:34:25.735 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:8GA2GQfyl7u8hj-Qp6pfWua4SbGLhz4_.aUEztSQlchIUoe-8WBCG8dJOBGOLq0y_0 2026-02-24T22:34:25.735 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14081882686323397312', time=2026-02-25T00:30:28.271356+0000, chain.objs.size()=1 2026-02-24T22:34:25.735 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1943__shadow:hC--eS1i3fsLw1RcqmMUChAPKlqxeDE_.Z_EEfvgjVHMAtjeVTSh3-d2HMHDF2YX_0 2026-02-24T22:34:25.735 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16037395063482149779', time=2026-02-25T00:30:31.167916+0000, chain.objs.size()=1 2026-02-24T22:34:25.735 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1944__shadow_.YC5sSPHuu3Hzx29zZwsyGB201uava8Z_0 2026-02-24T22:34:25.735 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14684766397214365918', time=2026-02-25T00:30:33.522922+0000, chain.objs.size()=1 2026-02-24T22:34:25.735 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1946__shadow:sCYD50VQJotiAVOztnyhq-dszbizf.f_.vlytpALxnAh8LGP6tMJccW4X5HPdbo6_0 2026-02-24T22:34:25.735 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16125487486295016735', time=2026-02-25T00:30:35.431500+0000, chain.objs.size()=1 2026-02-24T22:34:25.735 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:w76zX9TWMnuRsbakyTiVpOO9BQSCNo-_.feWaaiUxgQesVXTsG9G5R93HEmA_Nni_0 2026-02-24T22:34:25.735 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.767377987107453907', time=2026-02-25T00:30:50.992174+0000, chain.objs.size()=1 2026-02-24T22:34:25.735 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1949__shadow:lfN3iL2zi-noV1qoQcYarDDKlDANJhv_.i86aHzFnZ0gU8a0HZHakxZtI7J9FkXi_0 2026-02-24T22:34:25.735 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4024564067502276015', time=2026-02-25T00:30:57.367183+0000, chain.objs.size()=1 2026-02-24T22:34:25.736 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:uhWP8NhLhYnaVhzBqEuvCU4LfCfPtq7_.-YAI69TYiiuVuAX0cWukXR5tH4s3O4I_0 2026-02-24T22:34:25.736 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4024564067502276015', time=2026-02-25T00:30:57.367697+0000, chain.objs.size()=1 2026-02-24T22:34:25.736 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:uhWP8NhLhYnaVhzBqEuvCU4LfCfPtq7_.-YAI69TYiiuVuAX0cWukXR5tH4s3O4I_0 2026-02-24T22:34:25.736 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4024564067502276015', time=2026-02-25T00:30:57.368996+0000, chain.objs.size()=1 2026-02-24T22:34:25.736 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:uhWP8NhLhYnaVhzBqEuvCU4LfCfPtq7_.-YAI69TYiiuVuAX0cWukXR5tH4s3O4I_0 2026-02-24T22:34:25.736 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6975524567687226882', time=2026-02-25T00:33:45.909452+0000, chain.objs.size()=1 2026-02-24T22:34:25.736 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1968__shadow_.yHt2biVgCftQ_iulNCUXXNhUpzD3Aqj_0 2026-02-24T22:34:25.736 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9919793879517832452', time=2026-02-25T00:33:55.041553+0000, chain.objs.size()=1 2026-02-24T22:34:25.736 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1977__shadow_.W80HSqc-d62_dsJDBrzCA3_H8K57Les_0 2026-02-24T22:34:25.737 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.747+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.739 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2026-02-24T22:34:25.740 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.740 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/44919' 2026-02-24T22:34:25.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12808193196784596615', time=2026-02-25T00:15:09.900914+0000, chain.objs.size()=1 2026-02-24T22:34:25.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1505__shadow_.UeJuRGD0SCGAqvGz6RsU6gGOvXPhaFI_0 2026-02-24T22:34:25.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1759400737388392531', time=2026-02-25T00:15:09.911680+0000, chain.objs.size()=1 2026-02-24T22:34:25.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1505__shadow_.rT9_5iD6CR_fp-tX0UGCNeC2QcIKMos_0 2026-02-24T22:34:25.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6435273285189456901', time=2026-02-25T00:15:09.950449+0000, chain.objs.size()=1 2026-02-24T22:34:25.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1505__shadow_.9PC_6tVFv66UGz2h-Q2AyzZYRLbZLMw_0 2026-02-24T22:34:25.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1281654688979473632', time=2026-02-25T00:15:34.757353+0000, chain.objs.size()=3 2026-02-24T22:34:25.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-LUKEWARM.2~eOtMH0vmpN_usdE1y1Q_d9UHSttRW0T.1 2026-02-24T22:34:25.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-LUKEWARM.2~eOtMH0vmpN_usdE1y1Q_d9UHSttRW0T.2 2026-02-24T22:34:25.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-LUKEWARM.2~eOtMH0vmpN_usdE1y1Q_d9UHSttRW0T.3 2026-02-24T22:34:25.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='_pkoGCCK1k5qeAAjR3H0FufucfZZXwL5', time=2026-02-25T00:15:34.772077+0000, chain.objs.size()=2 2026-02-24T22:34:25.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__shadow_.4c_YiLaIb-mvcfuglGE64eqdsF08fxH_0 2026-02-24T22:34:25.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__shadow_.4c_YiLaIb-mvcfuglGE64eqdsF08fxH_1 2026-02-24T22:34:25.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14401360298554497472', time=2026-02-25T00:15:34.858791+0000, chain.objs.size()=1 2026-02-24T22:34:25.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1509__shadow_.e7u9i7S9dKl2B_a6pDZisvf8QwJe33f_0 2026-02-24T22:34:25.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15183047725431300191', time=2026-02-25T00:17:07.124066+0000, chain.objs.size()=1 2026-02-24T22:34:25.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523__shadow_.RoJm8U_PjL9pt2m-zNY_vP5TRBdbqLJ_0 2026-02-24T22:34:25.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16128280368616823526', time=2026-02-25T00:17:07.124865+0000, chain.objs.size()=1 2026-02-24T22:34:25.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523__shadow_.rYyYcb4oYaCycLfy0AOL8qX5-xTvrYu_0 2026-02-24T22:34:25.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.660422772871955178', time=2026-02-25T00:17:41.429448+0000, chain.objs.size()=1 2026-02-24T22:34:25.741 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526__shadow_.UPiUeQ6qSK9EsDGXFhK4-Mg3ZV4K9fq_0 2026-02-24T22:34:25.742 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17488520369200618177', time=2026-02-25T00:17:41.431355+0000, chain.objs.size()=1 2026-02-24T22:34:25.742 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.751+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526__shadow_.DBbUjyB3tWjm4GbV7j582OBln6ZdJTa_0 2026-02-24T22:34:25.742 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16343818224471216092', time=2026-02-25T00:17:42.543742+0000, chain.objs.size()=1 2026-02-24T22:34:25.742 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527__shadow_.gJOAJQB0J7N1woiFfjTXUJtWsUS0Sjy_0 2026-02-24T22:34:25.742 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5040641305368817236', time=2026-02-25T00:18:16.132004+0000, chain.objs.size()=1 2026-02-24T22:34:25.742 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529__shadow_.inVbatqGpcxNtCsERCTHop2kwzoixSL_0 2026-02-24T22:34:25.742 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10932286672838025216', time=2026-02-25T00:18:51.097308+0000, chain.objs.size()=1 2026-02-24T22:34:25.742 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532__shadow_.hCOwJ-VoNi8plj-5gOHlAEDpo5VzB0t_0 2026-02-24T22:34:25.743 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3752765088531553053', time=2026-02-25T00:18:51.098887+0000, chain.objs.size()=1 2026-02-24T22:34:25.743 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532__shadow_.vhO3UDlyIV6OgOO6uVnGbPJ_atznxfr_0 2026-02-24T22:34:25.743 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13090773800565282349', time=2026-02-25T00:19:58.465235+0000, chain.objs.size()=1 2026-02-24T22:34:25.743 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536__shadow_.a7ZHc971A4XqgtbvwPFYE-4Ks6_nA_n_0 2026-02-24T22:34:25.743 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4807058284529332623', time=2026-02-25T00:20:30.736927+0000, chain.objs.size()=1 2026-02-24T22:34:25.743 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537__shadow_.FRMcXeThIuvA3GqJ393dSboo4CJI5d__0 2026-02-24T22:34:25.743 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11726757892884751574', time=2026-02-25T00:20:32.525042+0000, chain.objs.size()=1 2026-02-24T22:34:25.743 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538__shadow_.77InMWBhjtMz8UvAF1Adbaodfluj4JU_0 2026-02-24T22:34:25.743 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6627814239448904364', time=2026-02-25T00:20:34.041869+0000, chain.objs.size()=1 2026-02-24T22:34:25.743 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539__shadow_.8zdLZ0wm6iB7zWGbeX67cmV6SX6RJcW_0 2026-02-24T22:34:25.743 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8545278730451888880', time=2026-02-25T00:21:06.321942+0000, chain.objs.size()=1 2026-02-24T22:34:25.743 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540__shadow_.ejtItVmNmdRX07bbmOwrxJQ-_q2QNtD_0 2026-02-24T22:34:25.743 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.946124942508814777', time=2026-02-25T00:25:36.010771+0000, chain.objs.size()=1 2026-02-24T22:34:25.743 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1601__shadow_.4o30u7hJgLXjvOJHe-kTtCXHkiwPZ6M_0 2026-02-24T22:34:25.744 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6196996141894494724', time=2026-02-25T00:25:38.506823+0000, chain.objs.size()=1 2026-02-24T22:34:25.744 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1604__shadow_.QYm0yCl0KVzmbOyVXaaFxuw1fUdr-7P_0 2026-02-24T22:34:25.744 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3282565309469953562', time=2026-02-25T00:25:40.797922+0000, chain.objs.size()=1 2026-02-24T22:34:25.744 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1608__shadow_.wMYqLoS1NyZvYCCxi-3R1q2BJAkGh6X_0 2026-02-24T22:34:25.744 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17700908593515416621', time=2026-02-25T00:25:44.441528+0000, chain.objs.size()=1 2026-02-24T22:34:25.744 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1613__shadow_.TILErVjwJoqyPaEiHAl8fNhV_5ZSk7w_0 2026-02-24T22:34:25.744 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5468759509518535438', time=2026-02-25T00:25:49.093683+0000, chain.objs.size()=1 2026-02-24T22:34:25.744 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1621__shadow_.c_2BdMp_lg9znIlKjBx1GVIXDYWBMx3_0 2026-02-24T22:34:25.744 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15402621544244785822', time=2026-02-25T00:25:59.112408+0000, chain.objs.size()=1 2026-02-24T22:34:25.744 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.lqCfO6F1pFRGdf7vzx_a6zDcopvgHS0_0 2026-02-24T22:34:25.744 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12336736612410298921', time=2026-02-25T00:25:59.122178+0000, chain.objs.size()=1 2026-02-24T22:34:25.744 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.khh3kyXxKktIbIF1hKVIvQMNf1xfVN9_0 2026-02-24T22:34:25.744 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16048543195751941508', time=2026-02-25T00:25:59.166817+0000, chain.objs.size()=1 2026-02-24T22:34:25.744 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.oS7bz5lmR3o2Hf2qe6cvAemZCdxeMqR_0 2026-02-24T22:34:25.744 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10836515544324919148', time=2026-02-25T00:25:59.184112+0000, chain.objs.size()=1 2026-02-24T22:34:25.744 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Vd4GV908rDOUHCbbUtjq4NQax5yzGOF_0 2026-02-24T22:34:25.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4583949899015003107', time=2026-02-25T00:25:59.271040+0000, chain.objs.size()=1 2026-02-24T22:34:25.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.CmnTBLNvan0TClT12aw3LTcNu8TcPgX_0 2026-02-24T22:34:25.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8259514090026868317', time=2026-02-25T00:25:59.331917+0000, chain.objs.size()=1 2026-02-24T22:34:25.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_._EbPePSt5NGfrGSqxG4pU24-l2t85at_0 2026-02-24T22:34:25.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17582580401565179804', time=2026-02-25T00:25:59.344951+0000, chain.objs.size()=1 2026-02-24T22:34:25.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.vojsjo908ijszQNrKBrcu0McIQwxc05_0 2026-02-24T22:34:25.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4042486708996522600', time=2026-02-25T00:25:59.612903+0000, chain.objs.size()=1 2026-02-24T22:34:25.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.4km2OXn4bVD8iRFYyPW1Io5Vr9Jz_4C_0 2026-02-24T22:34:25.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18230490919814490475', time=2026-02-25T00:25:59.628994+0000, chain.objs.size()=1 2026-02-24T22:34:25.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Co1geCxHA2JqhLbwxkw1cn-Z-0HarAw_0 2026-02-24T22:34:25.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4671153563732859231', time=2026-02-25T00:25:59.668146+0000, chain.objs.size()=1 2026-02-24T22:34:25.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.U6xmmS489YcrqueD4qOuhOueoCNijOH_0 2026-02-24T22:34:25.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9407314921142634812', time=2026-02-25T00:25:59.756375+0000, chain.objs.size()=1 2026-02-24T22:34:25.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.iuD9ef0WnpIJvpsS1wAP3gTHVXt8fT3_0 2026-02-24T22:34:25.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4945296640380687432', time=2026-02-25T00:25:59.766324+0000, chain.objs.size()=1 2026-02-24T22:34:25.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.1XVU699TEcToWCKfkcfEMKl5-94WlVS_0 2026-02-24T22:34:25.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3544064126789791799', time=2026-02-25T00:25:59.791677+0000, chain.objs.size()=1 2026-02-24T22:34:25.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.LhKF3bWMBBF__lBlCRmrNr7_Ue6yRuA_0 2026-02-24T22:34:25.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9261193810540464578', time=2026-02-25T00:25:59.807477+0000, chain.objs.size()=1 2026-02-24T22:34:25.745 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.YIjFE0__l3Jyfh31a1194OSVtIw6fVU_0 2026-02-24T22:34:25.746 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2368929869104068904', time=2026-02-25T00:25:59.824029+0000, chain.objs.size()=1 2026-02-24T22:34:25.746 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.n6Gr0DGPjJwSigynf9OjwwfasmAaIgX_0 2026-02-24T22:34:25.746 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13167887286226548087', time=2026-02-25T00:25:59.971324+0000, chain.objs.size()=1 2026-02-24T22:34:25.746 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.PPTKwZi9gSTlY9ECBxY_TLoHRSz8kZ7_0 2026-02-24T22:34:25.746 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8187518216057929002', time=2026-02-25T00:25:59.989349+0000, chain.objs.size()=1 2026-02-24T22:34:25.746 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.YftGAvCA9fX5i9w5iJDJP2rJplIVlDM_0 2026-02-24T22:34:25.746 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5614937621225229651', time=2026-02-25T00:26:00.065882+0000, chain.objs.size()=1 2026-02-24T22:34:25.746 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.wBugbOFbPMKyUsaMUqETaIUlT3P0-qu_0 2026-02-24T22:34:25.746 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4555708382550924355', time=2026-02-25T00:26:00.080323+0000, chain.objs.size()=1 2026-02-24T22:34:25.746 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.GKv93REEYEbUpQEfHL8gr32atRvLX0g_0 2026-02-24T22:34:25.746 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16234237291496672545', time=2026-02-25T00:26:00.135410+0000, chain.objs.size()=1 2026-02-24T22:34:25.746 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.mAVZ4JjjOprbxTPYy--MuYwa9VrZBoS_0 2026-02-24T22:34:25.746 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16431167904187356238', time=2026-02-25T00:26:00.139740+0000, chain.objs.size()=1 2026-02-24T22:34:25.746 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.qunfq67ICiKYP_FvVAF-llOi5loLgRf_0 2026-02-24T22:34:25.746 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10862966668186209628', time=2026-02-25T00:26:00.153991+0000, chain.objs.size()=1 2026-02-24T22:34:25.746 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ZKTZC_Q3eOA1vDA5nLypS8H1wIWToym_0 2026-02-24T22:34:25.746 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4905794569998085595', time=2026-02-25T00:26:00.221138+0000, chain.objs.size()=1 2026-02-24T22:34:25.746 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.VOx3x5Pz_2mUJQTF32907G3LQjoJXs4_0 2026-02-24T22:34:25.746 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3680527479026621340', time=2026-02-25T00:26:00.274241+0000, chain.objs.size()=1 2026-02-24T22:34:25.746 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.755+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.kLIUDVZoE7NrwnDouSdkFHuNRg9UwUS_0 2026-02-24T22:34:25.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2350470541476578571', time=2026-02-25T00:26:01.385285+0000, chain.objs.size()=1 2026-02-24T22:34:25.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1630__shadow_.72Ztkg_iQnHIAtyxrjkzv20shPA8rYT_0 2026-02-24T22:34:25.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15086725823181978695', time=2026-02-25T00:26:03.729720+0000, chain.objs.size()=1 2026-02-24T22:34:25.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1634__shadow_.VPJUXDDhIjrIMhPn2kKzVX3qBjjBqEy_0 2026-02-24T22:34:25.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5258147635301046293', time=2026-02-25T00:26:05.961156+0000, chain.objs.size()=1 2026-02-24T22:34:25.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1638__shadow_.vaxGNsAJ21wzbVN7bQdewd8posowpV__0 2026-02-24T22:34:25.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3987738066154340034', time=2026-02-25T00:26:08.285939+0000, chain.objs.size()=1 2026-02-24T22:34:25.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1642__shadow_.0oQ25ZCfVUQqL-HUdrBXl7rjNupWqFf_0 2026-02-24T22:34:25.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16545764144080128935', time=2026-02-25T00:26:12.382068+0000, chain.objs.size()=1 2026-02-24T22:34:25.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1649__shadow_.oeyt-g3PZXWc2qEBpdi5Qmiv8Dg0Pye_0 2026-02-24T22:34:25.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.809393996683004653', time=2026-02-25T00:26:18.524289+0000, chain.objs.size()=1 2026-02-24T22:34:25.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1658__shadow_.u7Iysm8BWwiEJLm9tXoI0kkYrtDMjmO_0 2026-02-24T22:34:25.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10124004439287106340', time=2026-02-25T00:26:20.573757+0000, chain.objs.size()=1 2026-02-24T22:34:25.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1661__shadow_.wLq9BZHem7-rR4YVYO2SwIF7E3TGGtj_0 2026-02-24T22:34:25.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14885669621600779176', time=2026-02-25T00:26:21.673854+0000, chain.objs.size()=1 2026-02-24T22:34:25.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1663__shadow_.30NpMPf2O-Cgs_ygrXtES7GDt09_h3G_0 2026-02-24T22:34:25.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6474803406222992630', time=2026-02-25T00:26:22.882415+0000, chain.objs.size()=1 2026-02-24T22:34:25.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1665__shadow_.4-LMhI82nAvVDOdbBD2Y3M35i5kBJ8y_0 2026-02-24T22:34:25.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16651491361721523262', time=2026-02-25T00:26:23.486181+0000, chain.objs.size()=1 2026-02-24T22:34:25.747 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1666__shadow_.y89_EFY4M74vgAWQd7EMq2gTa_QdMDG_0 2026-02-24T22:34:25.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17781385580286407557', time=2026-02-25T00:26:25.857722+0000, chain.objs.size()=1 2026-02-24T22:34:25.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1670__shadow_.LBVdPafiJ50ZyKCHJa_RCUrXBQbgjRb_0 2026-02-24T22:34:25.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10938622417654892218', time=2026-02-25T00:26:27.193578+0000, chain.objs.size()=1 2026-02-24T22:34:25.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1672__shadow_.uDHafweWJa1nPUe-q6JDw_djdcr-q9X_0 2026-02-24T22:34:25.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8878646139388800316', time=2026-02-25T00:26:35.769756+0000, chain.objs.size()=1 2026-02-24T22:34:25.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:5pLU1eXJ5vzKUwEiQWHIJObMdsoDKDA_.Grcd_RkPi1tIcMlivWW3ftMQEKe7Vkk_0 2026-02-24T22:34:25.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8878646139388800316', time=2026-02-25T00:26:35.785122+0000, chain.objs.size()=1 2026-02-24T22:34:25.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:5pLU1eXJ5vzKUwEiQWHIJObMdsoDKDA_.Grcd_RkPi1tIcMlivWW3ftMQEKe7Vkk_0 2026-02-24T22:34:25.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8878646139388800316', time=2026-02-25T00:26:35.810856+0000, chain.objs.size()=1 2026-02-24T22:34:25.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:5pLU1eXJ5vzKUwEiQWHIJObMdsoDKDA_.Grcd_RkPi1tIcMlivWW3ftMQEKe7Vkk_0 2026-02-24T22:34:25.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8878646139388800316', time=2026-02-25T00:26:35.829414+0000, chain.objs.size()=1 2026-02-24T22:34:25.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:5pLU1eXJ5vzKUwEiQWHIJObMdsoDKDA_.Grcd_RkPi1tIcMlivWW3ftMQEKe7Vkk_0 2026-02-24T22:34:25.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8878646139388800316', time=2026-02-25T00:26:35.876830+0000, chain.objs.size()=1 2026-02-24T22:34:25.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:5pLU1eXJ5vzKUwEiQWHIJObMdsoDKDA_.Grcd_RkPi1tIcMlivWW3ftMQEKe7Vkk_0 2026-02-24T22:34:25.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10801211261798062997', time=2026-02-25T00:26:44.817010+0000, chain.objs.size()=1 2026-02-24T22:34:25.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.LRPP2BfzS4FhuBT8z1iV0GlvojIRJHV_0 2026-02-24T22:34:25.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3242662049495190012', time=2026-02-25T00:26:44.987906+0000, chain.objs.size()=1 2026-02-24T22:34:25.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.wCuiidhgXCYUglH3SPfHwLlhS6LnhzX_0 2026-02-24T22:34:25.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7529692505678973810', time=2026-02-25T00:26:45.024963+0000, chain.objs.size()=1 2026-02-24T22:34:25.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.3rTVXvldOnFgdVRzdcGSDQb25GxAs6d_0 2026-02-24T22:34:25.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5606392076967086671', time=2026-02-25T00:26:45.038321+0000, chain.objs.size()=1 2026-02-24T22:34:25.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.nCyR69suREpiLRQYsngtwLAVYrL1o6w_0 2026-02-24T22:34:25.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6026864520221015234', time=2026-02-25T00:26:45.109505+0000, chain.objs.size()=1 2026-02-24T22:34:25.748 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.uALkSd0ODM23x03QYfKARucyDu8P7bF_0 2026-02-24T22:34:25.749 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17674663659762637289', time=2026-02-25T00:26:45.135299+0000, chain.objs.size()=1 2026-02-24T22:34:25.749 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.wmMznK25fFolSGuTbte1xQFzm0TFLEr_0 2026-02-24T22:34:25.749 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17977935219954194079', time=2026-02-25T00:26:45.171486+0000, chain.objs.size()=1 2026-02-24T22:34:25.749 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.0n8vO8sPbV_ZVRmTvrgZ_BALLcRB3NO_0 2026-02-24T22:34:25.749 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.169010645764450343', time=2026-02-25T00:26:45.183746+0000, chain.objs.size()=1 2026-02-24T22:34:25.749 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.psU52QWnxIUKiOLquJU6azbXLEXaxHj_0 2026-02-24T22:34:25.749 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1504776288064073650', time=2026-02-25T00:26:45.194092+0000, chain.objs.size()=1 2026-02-24T22:34:25.749 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.3EOXZl3QyX0lvoRq2kBtiMpPN5E8jRw_0 2026-02-24T22:34:25.749 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9388721297426798865', time=2026-02-25T00:26:45.296072+0000, chain.objs.size()=1 2026-02-24T22:34:25.749 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.zDaLTuyfgv3XgPkkVVCi1wKWqMK8Ymu_0 2026-02-24T22:34:25.749 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2156321334865280837', time=2026-02-25T00:26:45.306147+0000, chain.objs.size()=1 2026-02-24T22:34:25.749 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.AosRWOCH7hJaBZohVdebCO2nrY5Rit-_0 2026-02-24T22:34:25.750 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11305057805226776095', time=2026-02-25T00:26:45.418938+0000, chain.objs.size()=1 2026-02-24T22:34:25.750 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.-J8Oabt9xpJiWZFvBkK1xUR5tUuahcX_0 2026-02-24T22:34:25.750 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13152441698457469683', time=2026-02-25T00:26:45.438047+0000, chain.objs.size()=1 2026-02-24T22:34:25.750 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ww60Z59G3efqbXPo4-80vKvMvNkNX-w_0 2026-02-24T22:34:25.750 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11900972241487205463', time=2026-02-25T00:26:45.450002+0000, chain.objs.size()=1 2026-02-24T22:34:25.750 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.vN3A85HpTxfhsQ-I0uEE-bZl1UY21kP_0 2026-02-24T22:34:25.750 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18420599222068135985', time=2026-02-25T00:26:45.499115+0000, chain.objs.size()=1 2026-02-24T22:34:25.750 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.TIu7EVeT_RD2MurvgVFxO0Y2NnIamIp_0 2026-02-24T22:34:25.750 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10729740621342451581', time=2026-02-25T00:26:45.577472+0000, chain.objs.size()=1 2026-02-24T22:34:25.750 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.w7CD9_6S9N999PR2yagmk6yMaB6JtiY_0 2026-02-24T22:34:25.750 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8512523554585416772', time=2026-02-25T00:26:45.821914+0000, chain.objs.size()=1 2026-02-24T22:34:25.750 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.YdgdRF2i2X738sDWZ4nhtNstkZokY6p_0 2026-02-24T22:34:25.750 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.759+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2380627486609821067', time=2026-02-25T00:26:45.834928+0000, chain.objs.size()=1 2026-02-24T22:34:25.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.S06WQY36_KYBXyBnreiacQjWygAilx7_0 2026-02-24T22:34:25.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6016469917942708928', time=2026-02-25T00:26:46.018953+0000, chain.objs.size()=1 2026-02-24T22:34:25.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.hF5Twn80bwp1hWeKopPfBO0zEhNLm54_0 2026-02-24T22:34:25.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11064393085960520435', time=2026-02-25T00:26:46.091564+0000, chain.objs.size()=1 2026-02-24T22:34:25.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.DYW-YZGlsItHsNjLx5069Aw-65IPzNp_0 2026-02-24T22:34:25.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15956382019795564630', time=2026-02-25T00:26:46.113642+0000, chain.objs.size()=1 2026-02-24T22:34:25.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Ozr-5lqFE7w4Sz79oBd9AHNknul-3hd_0 2026-02-24T22:34:25.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18005208703344005891', time=2026-02-25T00:26:46.122314+0000, chain.objs.size()=1 2026-02-24T22:34:25.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.fqg4mrrDE-VIs76jpNQYtEzc_VmJnEA_0 2026-02-24T22:34:25.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9123382382074167074', time=2026-02-25T00:26:46.127890+0000, chain.objs.size()=1 2026-02-24T22:34:25.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Qd1YV7DndFsJaD93ArGjubLuqgEtA_v_0 2026-02-24T22:34:25.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8566867017779612757', time=2026-02-25T00:26:46.161049+0000, chain.objs.size()=1 2026-02-24T22:34:25.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.IeiTbFjn7MHQ1yeoI4yvH4pLJe-v5RO_0 2026-02-24T22:34:25.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18370576654340671170', time=2026-02-25T00:26:53.201902+0000, chain.objs.size()=1 2026-02-24T22:34:25.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.De7FSPJFLayFCE9u4X3Ma5XFeJqgo1P_0 2026-02-24T22:34:25.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14781116886989717750', time=2026-02-25T00:26:53.246394+0000, chain.objs.size()=1 2026-02-24T22:34:25.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.rBYKcELHKdKzAjswPgi-F6EovX_p-Lz_0 2026-02-24T22:34:25.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11113276138407545354', time=2026-02-25T00:26:53.354086+0000, chain.objs.size()=1 2026-02-24T22:34:25.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.y93M6jxd-QuCCNsdU7EbANy1NFJ_cZv_0 2026-02-24T22:34:25.751 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12342933587895393516', time=2026-02-25T00:26:53.406956+0000, chain.objs.size()=1 2026-02-24T22:34:25.752 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.NEhwDkn7zW9cOpwyi26Lz8U-zjtJpLs_0 2026-02-24T22:34:25.752 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18415442633242570806', time=2026-02-25T00:26:53.445700+0000, chain.objs.size()=1 2026-02-24T22:34:25.752 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.HVHEkNcttnAZmyofAeWEcn9evEOtXQD_0 2026-02-24T22:34:25.752 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7769586605145122267', time=2026-02-25T00:26:53.561746+0000, chain.objs.size()=1 2026-02-24T22:34:25.752 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.-nPAo3OZ__fQkt8lB-I9syJFpYw4pHI_0 2026-02-24T22:34:25.752 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3856412810616658083', time=2026-02-25T00:26:53.775036+0000, chain.objs.size()=1 2026-02-24T22:34:25.752 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.mj1ujfilnh1trwadyG33sCMT1lwHiyn_0 2026-02-24T22:34:25.752 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5335943409773786438', time=2026-02-25T00:26:53.782821+0000, chain.objs.size()=1 2026-02-24T22:34:25.752 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.MfTdbv-TYbTi5DFMw4RELnqnDi_9GB9_0 2026-02-24T22:34:25.752 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4226235418440770543', time=2026-02-25T00:26:53.786193+0000, chain.objs.size()=1 2026-02-24T22:34:25.752 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.UiGrLKNJ2C-dtdvUc5v8a5jm3MoHo4a_0 2026-02-24T22:34:25.752 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.271049864240152101', time=2026-02-25T00:26:53.813983+0000, chain.objs.size()=1 2026-02-24T22:34:25.752 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.DnntPMNBHsKKAPkYvKatLnHrbmwI0VM_0 2026-02-24T22:34:25.752 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8211660094026965980', time=2026-02-25T00:26:53.820051+0000, chain.objs.size()=1 2026-02-24T22:34:25.752 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Iou1muzgtF1XjU2pyMcMPNLAT-UX_ts_0 2026-02-24T22:34:25.753 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.763+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/44919 2026-02-24T22:34:25.755 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=25, truncated=0, next_marker='' 2026-02-24T22:34:25.755 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12706007480122992103', time=2026-02-25T00:26:54.003958+0000, chain.objs.size()=1 2026-02-24T22:34:25.755 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.K0nsojfBu0XaGDkgRv9JaKS1hf3BUmz_0 2026-02-24T22:34:25.755 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17830422168278319711', time=2026-02-25T00:26:54.023064+0000, chain.objs.size()=1 2026-02-24T22:34:25.755 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.4irvga_QymwyBi6iZJZqAwMmICZ_vup_0 2026-02-24T22:34:25.755 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12830006885759884178', time=2026-02-25T00:26:54.115333+0000, chain.objs.size()=1 2026-02-24T22:34:25.755 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ekZ2Q_SOLSlHKarEJZtGE9vCQ4Mup2X_0 2026-02-24T22:34:25.755 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3012879595500143322', time=2026-02-25T00:26:54.177170+0000, chain.objs.size()=1 2026-02-24T22:34:25.755 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.IK9pKVEF4MEExzdw2CzIZTpGAatWPmm_0 2026-02-24T22:34:25.755 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11567061822028216194', time=2026-02-25T00:26:54.210078+0000, chain.objs.size()=1 2026-02-24T22:34:25.755 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.XRG9fMPacDY6HXsQiISBxVAgVN38PRk_0 2026-02-24T22:34:25.755 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3047523843762719505', time=2026-02-25T00:26:54.224688+0000, chain.objs.size()=1 2026-02-24T22:34:25.755 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.BjVgN_R03_tsijDaTLTX5D5xGCxncPg_0 2026-02-24T22:34:25.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.164326665953612298', time=2026-02-25T00:26:54.331332+0000, chain.objs.size()=1 2026-02-24T22:34:25.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Or6XSrA1zlb9E14RrhYVc4xv3ODWERK_0 2026-02-24T22:34:25.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9191112205590444386', time=2026-02-25T00:26:54.365471+0000, chain.objs.size()=1 2026-02-24T22:34:25.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.9EC0tBhYI0frS3eYwPxeKVbL9n_Yser_0 2026-02-24T22:34:25.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3567826240778686645', time=2026-02-25T00:26:54.373819+0000, chain.objs.size()=1 2026-02-24T22:34:25.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.COHIMWIz6x4KhFAxHp23p0IoM25JqGi_0 2026-02-24T22:34:25.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11086549582580637647', time=2026-02-25T00:26:54.374235+0000, chain.objs.size()=1 2026-02-24T22:34:25.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.U4cWMTEHXgcE5e3nN8gWULo4M0WKbqM_0 2026-02-24T22:34:25.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5307578360493649314', time=2026-02-25T00:26:54.375299+0000, chain.objs.size()=1 2026-02-24T22:34:25.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ScHLXD8EuDPD49jiKvaY6zJA1B7Qci2_0 2026-02-24T22:34:25.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.986763392821195130', time=2026-02-25T00:26:58.959415+0000, chain.objs.size()=1 2026-02-24T22:34:25.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1697__shadow_.BNcCapzzoupUNGfSV6IqF0PEbXtlKLP_0 2026-02-24T22:34:25.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6225326732382363010', time=2026-02-25T00:28:22.181892+0000, chain.objs.size()=1 2026-02-24T22:34:25.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1827__shadow_.2qJn_4X6mzrcKGpkWPn3xsT_b2mZsuw_0 2026-02-24T22:34:25.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12221084315547174461', time=2026-02-25T00:28:38.245014+0000, chain.objs.size()=1 2026-02-24T22:34:25.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1859__shadow_.YsTUE3bjRH0THvYa2hgIP5qAKorOAKh_0 2026-02-24T22:34:25.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15736517079378135348', time=2026-02-25T00:28:39.552082+0000, chain.objs.size()=1 2026-02-24T22:34:25.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1862__shadow_.VP2z0ytCbKy_u_pXkaJCy3qveya8nE2_0 2026-02-24T22:34:25.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8195634328945469672', time=2026-02-25T00:28:39.745363+0000, chain.objs.size()=1 2026-02-24T22:34:25.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1862__shadow_.VP2z0ytCbKy_u_pXkaJCy3qveya8nE2_0 2026-02-24T22:34:25.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='2~FPBPqLlYNU4LR56rnpatuhFZ5_ekRiy', time=2026-02-25T00:29:01.160600+0000, chain.objs.size()=1 2026-02-24T22:34:25.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~FPBPqLlYNU4LR56rnpatuhFZ5_ekRiy.2 2026-02-24T22:34:25.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3856007257303575874', time=2026-02-25T00:29:12.547375+0000, chain.objs.size()=2 2026-02-24T22:34:25.756 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1891__multipart_mymultipart.2~GeINBgMtJYsFrKIWwZv4WFU4Rqc-Ht5.1 2026-02-24T22:34:25.757 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1891__multipart_mymultipart.2~GeINBgMtJYsFrKIWwZv4WFU4Rqc-Ht5.2 2026-02-24T22:34:25.757 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.218826398460788993', time=2026-02-25T00:30:35.945943+0000, chain.objs.size()=1 2026-02-24T22:34:25.757 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:T-ZseHiXvv.c5q9-1YhzIG8f13.mgsT_.9ArYCbgT7wrvM9CA_NHfxD5S53Q8Hy2_0 2026-02-24T22:34:25.757 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15026276917414513140', time=2026-02-25T00:30:47.030593+0000, chain.objs.size()=1 2026-02-24T22:34:25.757 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:FKyyQobmH028PargVy7uOxl9h-lms09_.2E5A5YeRjF-Ib8vZ0GvhsAN5oPcKF0__0 2026-02-24T22:34:25.757 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4008398882396353235', time=2026-02-25T00:30:57.021879+0000, chain.objs.size()=1 2026-02-24T22:34:25.757 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:kwKVPJkPKg6ZQp.TmatcGifj6UF6PaM_.T9I9dVJjmJ5OKSIH9qAEzXxxeCIcbLH_0 2026-02-24T22:34:25.757 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4008398882396353235', time=2026-02-25T00:30:57.023305+0000, chain.objs.size()=1 2026-02-24T22:34:25.757 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:kwKVPJkPKg6ZQp.TmatcGifj6UF6PaM_.T9I9dVJjmJ5OKSIH9qAEzXxxeCIcbLH_0 2026-02-24T22:34:25.757 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4008398882396353235', time=2026-02-25T00:30:57.023741+0000, chain.objs.size()=1 2026-02-24T22:34:25.757 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:kwKVPJkPKg6ZQp.TmatcGifj6UF6PaM_.T9I9dVJjmJ5OKSIH9qAEzXxxeCIcbLH_0 2026-02-24T22:34:25.757 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4008398882396353235', time=2026-02-25T00:30:57.024220+0000, chain.objs.size()=1 2026-02-24T22:34:25.757 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:kwKVPJkPKg6ZQp.TmatcGifj6UF6PaM_.T9I9dVJjmJ5OKSIH9qAEzXxxeCIcbLH_0 2026-02-24T22:34:25.757 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12496004626296505105', time=2026-02-25T00:30:58.455502+0000, chain.objs.size()=1 2026-02-24T22:34:25.757 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.767+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958__shadow:AJKDQnu3VfWQnexon8TNCQ7KN6i9I-o_.gzKmg0c0zNYEcfQ8ytzYZjIGzb35Qhv_0 2026-02-24T22:34:25.758 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.760 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2026-02-24T22:34:25.761 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.761 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/37516' 2026-02-24T22:34:25.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18405830696094412580', time=2026-02-25T00:15:34.696013+0000, chain.objs.size()=1 2026-02-24T22:34:25.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516__shadow_.sWqJtHRKb8-jjjuiy67dcn_vIAyyWJ5_0 2026-02-24T22:34:25.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='_W5viz0oj3FO2M-LaumlMS9NAsjzcb7c', time=2026-02-25T00:15:34.715720+0000, chain.objs.size()=1 2026-02-24T22:34:25.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516__shadow_.15O0tu_ni1pbUws8wBwFTuroGmO0gDt_0 2026-02-24T22:34:25.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18364486315138087549', time=2026-02-25T00:16:32.610178+0000, chain.objs.size()=1 2026-02-24T22:34:25.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520__shadow_.9wiGFGIFK7STJUg8HP5qGXXCvybJBb2_0 2026-02-24T22:34:25.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10405772581368858619', time=2026-02-25T00:16:33.509462+0000, chain.objs.size()=1 2026-02-24T22:34:25.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521__shadow_.b_ZC2dsGQJ7OVCetn4u4i5WlOwfOGC6_0 2026-02-24T22:34:25.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17605736609038746159', time=2026-02-25T00:16:33.510624+0000, chain.objs.size()=1 2026-02-24T22:34:25.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521__shadow_.caeIeJpZSBKlhm1yWisv45i_TGvgdGt_0 2026-02-24T22:34:25.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2251171112668984085', time=2026-02-25T00:17:39.875859+0000, chain.objs.size()=1 2026-02-24T22:34:25.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525__shadow_.uC6-OyQ7NSkVD--JvTwT2YZbGSMLQ1z_0 2026-02-24T22:34:25.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.34541453858392432', time=2026-02-25T00:17:41.422506+0000, chain.objs.size()=1 2026-02-24T22:34:25.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526__shadow_.yt2WsRThjxSnIIc8lxTwC5Vs7QSUy4X_0 2026-02-24T22:34:25.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18003176749152761914', time=2026-02-25T00:18:48.182550+0000, chain.objs.size()=1 2026-02-24T22:34:25.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530__shadow_.02SBUFNfv--bdWHU-iZJXmZMa46yYW2_0 2026-02-24T22:34:25.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.883446130253314999', time=2026-02-25T00:19:25.017697+0000, chain.objs.size()=1 2026-02-24T22:34:25.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534__shadow_.__c78lJwIOUW5iTS7xEAq76WvLtiWPz_0 2026-02-24T22:34:25.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5188624193636317109', time=2026-02-25T00:19:57.106442+0000, chain.objs.size()=1 2026-02-24T22:34:25.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535__shadow_.FAtExBC3-zh5pffjdNur3fMnMTTBkMV_0 2026-02-24T22:34:25.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3095239422620906921', time=2026-02-25T00:20:30.725575+0000, chain.objs.size()=1 2026-02-24T22:34:25.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537__shadow_.Nl52njuqATvKgw5oDulksBWiP8qr0Ym_0 2026-02-24T22:34:25.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14902649218336046735', time=2026-02-25T00:20:34.028991+0000, chain.objs.size()=1 2026-02-24T22:34:25.762 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.771+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539__shadow_.Fk4MfoYFAQKoo694CG5wEf9ByvCW8Jy_0 2026-02-24T22:34:25.763 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4358788470855014767', time=2026-02-25T00:21:06.316167+0000, chain.objs.size()=1 2026-02-24T22:34:25.763 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1540__shadow_.1Qfg9qrB6kAtn4uOjo7jn_TcegtthZm_0 2026-02-24T22:34:25.763 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10140478121423121188', time=2026-02-25T00:25:35.994540+0000, chain.objs.size()=1 2026-02-24T22:34:25.763 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1601__shadow_.TPyB0IRJIgvWpA1p4J4SSm6jzTgtXJK_0 2026-02-24T22:34:25.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13327724011920270414', time=2026-02-25T00:25:36.598743+0000, chain.objs.size()=1 2026-02-24T22:34:25.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1602__shadow_.xWVNcdEd2YeScOMpZyBAyexbDw7fkdc_0 2026-02-24T22:34:25.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10725507874326831508', time=2026-02-25T00:25:41.711866+0000, chain.objs.size()=1 2026-02-24T22:34:25.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1609__shadow_.jaZhjxKjy54YSskaj9yOCOkVosjNLlc_0 2026-02-24T22:34:25.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7058468478423749198', time=2026-02-25T00:25:42.668357+0000, chain.objs.size()=1 2026-02-24T22:34:25.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1610__shadow_.Yzdcxo7I_ioJegNeSvdffGgquNYOoWi_0 2026-02-24T22:34:25.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15038519364375210021', time=2026-02-25T00:25:43.239141+0000, chain.objs.size()=1 2026-02-24T22:34:25.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1611__shadow_.FE9GrJUXIQWBXRMj3nwcNCyInyJ7TwW_0 2026-02-24T22:34:25.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17124508294155066156', time=2026-02-25T00:25:46.124154+0000, chain.objs.size()=1 2026-02-24T22:34:25.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1616__shadow_.guSIdPiyTGFM9USr4rewAO2j5D0AymP_0 2026-02-24T22:34:25.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11964430890614739914', time=2026-02-25T00:25:51.394279+0000, chain.objs.size()=1 2026-02-24T22:34:25.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1625__shadow_.1lB0VW6kRMrR6hmOyBXZZ5zn5XR6QKK_0 2026-02-24T22:34:25.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9295746442138714875', time=2026-02-25T00:25:51.942802+0000, chain.objs.size()=1 2026-02-24T22:34:25.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1626__shadow_.wRTV3YI3a-90mCVtG7CyuqACRZuiWHf_0 2026-02-24T22:34:25.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13077680077982447685', time=2026-02-25T00:25:59.094766+0000, chain.objs.size()=1 2026-02-24T22:34:25.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.qiy2j0Igc23f_ekktRcYDVFzrhlXDEY_0 2026-02-24T22:34:25.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5238354772169672067', time=2026-02-25T00:25:59.100965+0000, chain.objs.size()=1 2026-02-24T22:34:25.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.T27b-0HEUBn3hC54s1Wt2SUzcvFo-ee_0 2026-02-24T22:34:25.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8535409885677021005', time=2026-02-25T00:25:59.107092+0000, chain.objs.size()=1 2026-02-24T22:34:25.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.OKc4xObDJ0DrZ4IpMjiwYGXjCZUeKkJ_0 2026-02-24T22:34:25.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18050720467844679840', time=2026-02-25T00:25:59.148884+0000, chain.objs.size()=1 2026-02-24T22:34:25.764 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_._ohl0EiZ9KiSCGAr1k--pdDeTCJe_R9_0 2026-02-24T22:34:25.765 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15261791767384121058', time=2026-02-25T00:25:59.244118+0000, chain.objs.size()=1 2026-02-24T22:34:25.765 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.5HN_9ttB-cmJDJhQ1mmVk8Ih0loDI8t_0 2026-02-24T22:34:25.765 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9623425276995309001', time=2026-02-25T00:25:59.245948+0000, chain.objs.size()=1 2026-02-24T22:34:25.765 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.sghr-P6i8l8NZE76kkOZvM3grrkm39w_0 2026-02-24T22:34:25.765 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11316421202941993350', time=2026-02-25T00:25:59.303552+0000, chain.objs.size()=1 2026-02-24T22:34:25.765 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.XotTLbNvmtZ1185m5xAQ7LjW1KN_zhx_0 2026-02-24T22:34:25.765 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3485663483332010303', time=2026-02-25T00:25:59.325546+0000, chain.objs.size()=1 2026-02-24T22:34:25.765 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.FndhyVMK1TkhbghXRxQ7XrIv6CtHF4I_0 2026-02-24T22:34:25.765 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9062545951259922245', time=2026-02-25T00:25:59.417154+0000, chain.objs.size()=1 2026-02-24T22:34:25.765 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Cy0rRjF_IJyjXrPfhHOrKwH7RZgRyiY_0 2026-02-24T22:34:25.765 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9428796015985230388', time=2026-02-25T00:25:59.461386+0000, chain.objs.size()=1 2026-02-24T22:34:25.765 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.2HakMbcjU2TOBmVDoUfCZT1O08LEMhQ_0 2026-02-24T22:34:25.765 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11825579663904053923', time=2026-02-25T00:25:59.581804+0000, chain.objs.size()=1 2026-02-24T22:34:25.765 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.LUdDM6SwTACAX3IDIcFRLR0AkFlmXsG_0 2026-02-24T22:34:25.766 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1542127997522937643', time=2026-02-25T00:25:59.594434+0000, chain.objs.size()=1 2026-02-24T22:34:25.766 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.eEkcyikrh0bmk6ttzzXZeWZkD8S2yH4_0 2026-02-24T22:34:25.766 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9196098162619999191', time=2026-02-25T00:25:59.607430+0000, chain.objs.size()=1 2026-02-24T22:34:25.766 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.zZPN9IfHHRDPvoTMqMs8H-NIuozzQns_0 2026-02-24T22:34:25.766 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6968059656394602991', time=2026-02-25T00:25:59.614290+0000, chain.objs.size()=1 2026-02-24T22:34:25.766 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.jWLopTlsfvT1yRAd1IckiWBE3S46N0G_0 2026-02-24T22:34:25.766 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15498815936507209990', time=2026-02-25T00:25:59.642873+0000, chain.objs.size()=1 2026-02-24T22:34:25.766 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.QwpP63Cri-nQyPdU2qn2ghRoqqjAUcs_0 2026-02-24T22:34:25.766 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15341775816901096762', time=2026-02-25T00:25:59.724055+0000, chain.objs.size()=1 2026-02-24T22:34:25.766 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.gx_jGbiPmY766PlUOF-q3oUwx6MePtH_0 2026-02-24T22:34:25.766 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9132770620542620843', time=2026-02-25T00:25:59.727065+0000, chain.objs.size()=1 2026-02-24T22:34:25.766 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Q6Yza3GZECovlRJOC36OTLVWtk-Eyr6_0 2026-02-24T22:34:25.766 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6826343380609387144', time=2026-02-25T00:25:59.771564+0000, chain.objs.size()=1 2026-02-24T22:34:25.766 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.4BO3-mtqN68fTCbYzhBYHKLMtYi7nDj_0 2026-02-24T22:34:25.766 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14053368490989282079', time=2026-02-25T00:25:59.801440+0000, chain.objs.size()=1 2026-02-24T22:34:25.766 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.smjqIrlar70jmA0caUO9ctiJX3DtkZn_0 2026-02-24T22:34:25.766 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7278883290627186842', time=2026-02-25T00:25:59.823018+0000, chain.objs.size()=1 2026-02-24T22:34:25.766 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.JLVM-WjnAZ3u_xgXJNcYwzRaedW4KCf_0 2026-02-24T22:34:25.766 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1772365212656446584', time=2026-02-25T00:25:59.891275+0000, chain.objs.size()=1 2026-02-24T22:34:25.766 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.775+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.m7-SqYAXuSrrNsPeGAsKxtDZ9ISr9tR_0 2026-02-24T22:34:25.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11520208230524409319', time=2026-02-25T00:25:59.899067+0000, chain.objs.size()=1 2026-02-24T22:34:25.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.sOwUFYQNr5RwwuP8IUQmOvL8Z3cNHrd_0 2026-02-24T22:34:25.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12777697784104939983', time=2026-02-25T00:25:59.899556+0000, chain.objs.size()=1 2026-02-24T22:34:25.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.oVtglh3jOU4DcBYHUhzj5II-5eZxfBf_0 2026-02-24T22:34:25.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15269798394256049623', time=2026-02-25T00:25:59.907472+0000, chain.objs.size()=1 2026-02-24T22:34:25.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.RJI0ZxXYfbQm2_Sd33WwbasIudXl-bN_0 2026-02-24T22:34:25.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3565870125565940363', time=2026-02-25T00:25:59.916049+0000, chain.objs.size()=1 2026-02-24T22:34:25.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.M8yOrkAd-tA5I-bh2p-LLPfOgUZ8fpC_0 2026-02-24T22:34:25.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12623126727374809323', time=2026-02-25T00:25:59.931591+0000, chain.objs.size()=1 2026-02-24T22:34:25.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.HAk1VXPaotI9e3ORt8XOIPEa_2O61RD_0 2026-02-24T22:34:25.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11034217945055939670', time=2026-02-25T00:25:59.968994+0000, chain.objs.size()=1 2026-02-24T22:34:25.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.yxUZF7vGdMV9gZcuaGeC1A0ekq5sZNN_0 2026-02-24T22:34:25.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11368526240513874867', time=2026-02-25T00:25:59.969531+0000, chain.objs.size()=1 2026-02-24T22:34:25.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.qKiv4g6tnrJqvjWH_Uq382tMsMPhZ-y_0 2026-02-24T22:34:25.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5942414524466710928', time=2026-02-25T00:26:00.051954+0000, chain.objs.size()=1 2026-02-24T22:34:25.767 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.j3D3NbgxDKolsEw6JZOc8q6Uw55jW87_0 2026-02-24T22:34:25.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14571020133705940133', time=2026-02-25T00:26:00.077663+0000, chain.objs.size()=1 2026-02-24T22:34:25.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.s0DGm-on3AnzcunQ7LC67TbOGpkvtrL_0 2026-02-24T22:34:25.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13095151203211433308', time=2026-02-25T00:26:00.204476+0000, chain.objs.size()=1 2026-02-24T22:34:25.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.acXMWwx98a0TgpmymSuXGY6SwSAWuhE_0 2026-02-24T22:34:25.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16103245508419367858', time=2026-02-25T00:26:01.934267+0000, chain.objs.size()=1 2026-02-24T22:34:25.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1631__shadow_.QKZL34jMS1nb_8YOvc7qHsW3I9CRFdJ_0 2026-02-24T22:34:25.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14508011854078424742', time=2026-02-25T00:26:06.541948+0000, chain.objs.size()=1 2026-02-24T22:34:25.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1639__shadow_.afkfPmALYUrDxPqZDwUiSFqRyBh27I1_0 2026-02-24T22:34:25.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15483557851297683922', time=2026-02-25T00:26:18.498605+0000, chain.objs.size()=1 2026-02-24T22:34:25.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1658__shadow_.03FBHdIc7-6b3hf7sDLtvET0TlPqt5t_0 2026-02-24T22:34:25.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12894927765798449906', time=2026-02-25T00:26:25.270137+0000, chain.objs.size()=1 2026-02-24T22:34:25.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1669__shadow_.KSWfZJDoj5TFAJMNl7NA7iP4URScYj__0 2026-02-24T22:34:25.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2274644628366984361', time=2026-02-25T00:26:44.820940+0000, chain.objs.size()=1 2026-02-24T22:34:25.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.84vB8yy8vqzvaNkid6fFi32hWZsrbs5_0 2026-02-24T22:34:25.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11444227321569619834', time=2026-02-25T00:26:44.832134+0000, chain.objs.size()=1 2026-02-24T22:34:25.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.UwREgy7VivregLDATsx7hmTCw59jpox_0 2026-02-24T22:34:25.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3496324717119077442', time=2026-02-25T00:26:44.832479+0000, chain.objs.size()=1 2026-02-24T22:34:25.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.yW34qrphMoNDMxXwrA1SYa1BS5-YLvb_0 2026-02-24T22:34:25.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6013046677665398982', time=2026-02-25T00:26:44.853703+0000, chain.objs.size()=1 2026-02-24T22:34:25.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.sSYQYTQtLJcvCZScFGOsQ6DWR816lt9_0 2026-02-24T22:34:25.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6291591861804589454', time=2026-02-25T00:26:44.955089+0000, chain.objs.size()=1 2026-02-24T22:34:25.768 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.PcDCv7BEOqjXSJ7qXj5VhFvw72Q7nkL_0 2026-02-24T22:34:25.769 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16583183736028329064', time=2026-02-25T00:26:44.972683+0000, chain.objs.size()=1 2026-02-24T22:34:25.769 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.2Cqdt1PGH62khLUl7a48Bv9jvNNPTyE_0 2026-02-24T22:34:25.769 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13568269882068938889', time=2026-02-25T00:26:45.003360+0000, chain.objs.size()=1 2026-02-24T22:34:25.769 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.sNH94NEDFf2g5gH1KIGL_YXCT9TNpxg_0 2026-02-24T22:34:25.769 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12644020450778469445', time=2026-02-25T00:26:45.095193+0000, chain.objs.size()=1 2026-02-24T22:34:25.769 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.eqIPgW5kPcwwi8cKNmM0rsu3ybyNmz1_0 2026-02-24T22:34:25.769 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18032912301009580873', time=2026-02-25T00:26:45.128550+0000, chain.objs.size()=1 2026-02-24T22:34:25.769 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.EOQrvUcGZxUGWZAoph0hYjZwgi995U8_0 2026-02-24T22:34:25.769 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12731634262312843132', time=2026-02-25T00:26:45.143355+0000, chain.objs.size()=1 2026-02-24T22:34:25.769 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.B3Hv17s_ZgPFRS7cAE-Knsj1llw5R3J_0 2026-02-24T22:34:25.769 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.655182253286185400', time=2026-02-25T00:26:45.159548+0000, chain.objs.size()=1 2026-02-24T22:34:25.769 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.yFfLpzWT5Syd3dFxssTODZURlcHsUgF_0 2026-02-24T22:34:25.769 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8159245419102251852', time=2026-02-25T00:26:45.182898+0000, chain.objs.size()=1 2026-02-24T22:34:25.770 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ysYbJRS7uvxfPHox2jGx5kKX0meNr4M_0 2026-02-24T22:34:25.770 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11775492084895375439', time=2026-02-25T00:26:45.255060+0000, chain.objs.size()=1 2026-02-24T22:34:25.770 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.qFqCcnJOHuzHYDwHHZvEtGTbJre8tlJ_0 2026-02-24T22:34:25.770 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12920258989648454998', time=2026-02-25T00:26:45.299503+0000, chain.objs.size()=1 2026-02-24T22:34:25.770 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.vmf3VdwmQ3ouY38YpUt9lyhUAqSGJ1t_0 2026-02-24T22:34:25.770 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6096298196366809018', time=2026-02-25T00:26:45.309498+0000, chain.objs.size()=1 2026-02-24T22:34:25.770 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.HiMoFlpk1m-DPPuEcX-oswgfEFeJ4j9_0 2026-02-24T22:34:25.770 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.499938058244010566', time=2026-02-25T00:26:45.314101+0000, chain.objs.size()=1 2026-02-24T22:34:25.770 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.NpO3oaizsFAP_Wt1JaENgD0i78Raxjz_0 2026-02-24T22:34:25.770 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17394740205537626481', time=2026-02-25T00:26:45.320704+0000, chain.objs.size()=1 2026-02-24T22:34:25.770 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.AmH21Xalb1x7Oz4hY28_1JrksJImzn5_0 2026-02-24T22:34:25.770 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1173632617993878615', time=2026-02-25T00:26:45.420688+0000, chain.objs.size()=1 2026-02-24T22:34:25.770 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.p944jpviKLvQuM1gIv0HO2sG5pFygfc_0 2026-02-24T22:34:25.770 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6945056638718367744', time=2026-02-25T00:26:45.462340+0000, chain.objs.size()=1 2026-02-24T22:34:25.770 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.779+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.PBNWCgrQzU030Qbt477Fz8jBEvpb674_0 2026-02-24T22:34:25.770 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12415955002853213852', time=2026-02-25T00:26:45.488702+0000, chain.objs.size()=1 2026-02-24T22:34:25.770 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.y0BbmAZmZHLqyWhpchxDgjVJmbjIzZu_0 2026-02-24T22:34:25.770 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18077042909183825654', time=2026-02-25T00:26:45.572742+0000, chain.objs.size()=1 2026-02-24T22:34:25.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.l4RIQeK4YnWMN-FY0aHRjoIytTgU1jF_0 2026-02-24T22:34:25.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11518507811748117194', time=2026-02-25T00:26:45.596699+0000, chain.objs.size()=1 2026-02-24T22:34:25.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.YtGKTBwolSh2G9Vy_tthEU0D_ajnATJ_0 2026-02-24T22:34:25.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.215901657463650743', time=2026-02-25T00:26:45.804326+0000, chain.objs.size()=1 2026-02-24T22:34:25.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.KmjHJlrCS83YlWQmmc-XR04NPHot3Wl_0 2026-02-24T22:34:25.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8639675480149995448', time=2026-02-25T00:26:45.942154+0000, chain.objs.size()=1 2026-02-24T22:34:25.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.kTGhQOYYxA9kmja0jnJodddryKUStwb_0 2026-02-24T22:34:25.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.411382874853510740', time=2026-02-25T00:26:45.956669+0000, chain.objs.size()=1 2026-02-24T22:34:25.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Pwl0vzWFozjyMxUjpkmx54oU9VECgSc_0 2026-02-24T22:34:25.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4839301449042765953', time=2026-02-25T00:26:45.964346+0000, chain.objs.size()=1 2026-02-24T22:34:25.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Q90V6lt5DifFG62GHel4XZ4rkFznotT_0 2026-02-24T22:34:25.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6276657664760902105', time=2026-02-25T00:26:46.004698+0000, chain.objs.size()=1 2026-02-24T22:34:25.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.rQXxa2inMVGA3EKKW9lLqbB1U1GZAGg_0 2026-02-24T22:34:25.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3005995241627517516', time=2026-02-25T00:26:46.076694+0000, chain.objs.size()=1 2026-02-24T22:34:25.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Vt6-DoYblWdBAUKDj2Z_Mfa2MI3Suu0_0 2026-02-24T22:34:25.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12671207099987464202', time=2026-02-25T00:26:46.117646+0000, chain.objs.size()=1 2026-02-24T22:34:25.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.RNMMfWGojwH9JnTeJ8CKddFZsk8wstw_0 2026-02-24T22:34:25.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2764946456937712365', time=2026-02-25T00:26:53.140899+0000, chain.objs.size()=1 2026-02-24T22:34:25.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.vRa5nFdZe8upW1OuwVSYcjCiwvb3-QI_0 2026-02-24T22:34:25.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13164326270910158667', time=2026-02-25T00:26:53.146764+0000, chain.objs.size()=1 2026-02-24T22:34:25.771 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ifH1y9GOM9KJVHu2d-sfagFYUYjgEcw_0 2026-02-24T22:34:25.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10107847083967835198', time=2026-02-25T00:26:53.151378+0000, chain.objs.size()=1 2026-02-24T22:34:25.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_._MGvA6mkY-zkuBeqjPVPbcJjzwpRuUd_0 2026-02-24T22:34:25.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4466039614947954608', time=2026-02-25T00:26:53.181937+0000, chain.objs.size()=1 2026-02-24T22:34:25.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.5cNRTvX3PWtI1GBS8wgbkfIhRoG7Zd5_0 2026-02-24T22:34:25.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14159631329603678660', time=2026-02-25T00:26:53.199322+0000, chain.objs.size()=1 2026-02-24T22:34:25.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.2xO82fdmImRVuwWx-ye5q_o-AJUsssT_0 2026-02-24T22:34:25.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.352565077285199929', time=2026-02-25T00:26:53.235140+0000, chain.objs.size()=1 2026-02-24T22:34:25.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.QSvgODPal7Q9wXjMR-fIRg-cJ1r-eoX_0 2026-02-24T22:34:25.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6368594222907397448', time=2026-02-25T00:26:53.331751+0000, chain.objs.size()=1 2026-02-24T22:34:25.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.GYFR4aAP-6KQlRnk5DcQQtkDf4x5RDC_0 2026-02-24T22:34:25.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8126129233377110467', time=2026-02-25T00:26:53.334986+0000, chain.objs.size()=1 2026-02-24T22:34:25.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.bYQYNRjUKcb1yA-BZRd4yzms1olfzn3_0 2026-02-24T22:34:25.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5185921544988544472', time=2026-02-25T00:26:53.389528+0000, chain.objs.size()=1 2026-02-24T22:34:25.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.dyXltGggU9KAAfAHmyxw83VQAwQ2QZq_0 2026-02-24T22:34:25.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11572752028208877784', time=2026-02-25T00:26:53.418604+0000, chain.objs.size()=1 2026-02-24T22:34:25.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.FL9LZQC1Ku5-4fMmgxzMAlRcC6mujNc_0 2026-02-24T22:34:25.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8011547420392364218', time=2026-02-25T00:26:53.726860+0000, chain.objs.size()=1 2026-02-24T22:34:25.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.je2PSvXxHRgcr2fWdS0yP-y8xEfLAw-_0 2026-02-24T22:34:25.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8081659371671285752', time=2026-02-25T00:26:53.768909+0000, chain.objs.size()=1 2026-02-24T22:34:25.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.fT-xFmofgAssGDpTyyWmCk-n4mXT_XJ_0 2026-02-24T22:34:25.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7135914111537595893', time=2026-02-25T00:26:53.773426+0000, chain.objs.size()=1 2026-02-24T22:34:25.772 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.PviONyMr9G29XVbXR42_cScmhHqjLba_0 2026-02-24T22:34:25.773 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12349472675235154182', time=2026-02-25T00:26:53.778401+0000, chain.objs.size()=1 2026-02-24T22:34:25.773 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.2vgKr5zkPb9PRzOozotrnBOoscPLVVY_0 2026-02-24T22:34:25.773 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8481034394661068978', time=2026-02-25T00:26:53.804539+0000, chain.objs.size()=1 2026-02-24T22:34:25.773 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.HxlJkuSd4xDsfm_0F4Px2Xp7GafIn4y_0 2026-02-24T22:34:25.774 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.783+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/37516 2026-02-24T22:34:25.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=40, truncated=0, next_marker='' 2026-02-24T22:34:25.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7103520050757315259', time=2026-02-25T00:26:53.807111+0000, chain.objs.size()=1 2026-02-24T22:34:25.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.SBQOCYP73h5utkE88qPJDvm_BmDhkI7_0 2026-02-24T22:34:25.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7771430732504245505', time=2026-02-25T00:26:53.917085+0000, chain.objs.size()=1 2026-02-24T22:34:25.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.jcM-xMhL_xrbrPMFHI5UbHWuvwdln82_0 2026-02-24T22:34:25.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11559295657224528204', time=2026-02-25T00:26:53.940303+0000, chain.objs.size()=1 2026-02-24T22:34:25.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.6xSMJ7NPhJ4bzc5I6Jsfn1u_GJYqSZB_0 2026-02-24T22:34:25.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7571018637176466053', time=2026-02-25T00:26:54.019848+0000, chain.objs.size()=1 2026-02-24T22:34:25.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.V5RI6M4vCCu6nImiIHf12uZwtmy5Giw_0 2026-02-24T22:34:25.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9810996444903612749', time=2026-02-25T00:26:54.104624+0000, chain.objs.size()=1 2026-02-24T22:34:25.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.gZ0HklKVDAHC-eqVQeDU7U4NqKNvF5q_0 2026-02-24T22:34:25.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4042054187372768423', time=2026-02-25T00:26:54.167796+0000, chain.objs.size()=1 2026-02-24T22:34:25.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.jN-uMi1V9dy16lfqvi_HAp2jIszXGX0_0 2026-02-24T22:34:25.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3744574332823267009', time=2026-02-25T00:26:54.186140+0000, chain.objs.size()=1 2026-02-24T22:34:25.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.p7UXYRP5IIfNmzMqLGukblf9a1Zxjew_0 2026-02-24T22:34:25.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7097212378066369873', time=2026-02-25T00:26:54.299475+0000, chain.objs.size()=1 2026-02-24T22:34:25.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ogd8g1yqugGa-8FtJVZv4UzVBHGsmkk_0 2026-02-24T22:34:25.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4156338034669627310', time=2026-02-25T00:26:54.303959+0000, chain.objs.size()=1 2026-02-24T22:34:25.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.VQsRxgrJRb0SVGIoOC9AeMtPppLdz9D_0 2026-02-24T22:34:25.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9983697480001743044', time=2026-02-25T00:26:54.311129+0000, chain.objs.size()=1 2026-02-24T22:34:25.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.zTmO-epefynP3bpBnGNOFhw_pOzXjec_0 2026-02-24T22:34:25.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16006858878358875683', time=2026-02-25T00:26:54.329213+0000, chain.objs.size()=1 2026-02-24T22:34:25.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.W5_POR3BAL-BxgAq1IuIIhUMNQKLcdQ_0 2026-02-24T22:34:25.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9530168620255280992', time=2026-02-25T00:26:54.343643+0000, chain.objs.size()=1 2026-02-24T22:34:25.776 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.NPhmoYc7mscP0kyMj4LtjZA8yX8ZqvD_0 2026-02-24T22:34:25.777 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9349019644794579761', time=2026-02-25T00:26:54.351720+0000, chain.objs.size()=1 2026-02-24T22:34:25.777 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.BC-jFimaMtIHaOvoEAG7E7X8h6ZCGCy_0 2026-02-24T22:34:25.777 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18191851457413916902', time=2026-02-25T00:26:54.383017+0000, chain.objs.size()=1 2026-02-24T22:34:25.777 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ua9KlAbumpkapHEcuVoDvvoUdVIFFve_0 2026-02-24T22:34:25.777 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7941066088298007613', time=2026-02-25T00:26:54.464252+0000, chain.objs.size()=1 2026-02-24T22:34:25.777 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.rSFSBuoWo_CFhsATnW_Z7GtUQ4Jb5Sf_0 2026-02-24T22:34:25.777 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10031515756626610632', time=2026-02-25T00:26:54.476891+0000, chain.objs.size()=1 2026-02-24T22:34:25.777 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.uErhAyILjC5bIgsRdQQDzSDfoVZDP3W_0 2026-02-24T22:34:25.777 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6900790585543051800', time=2026-02-25T00:26:57.856887+0000, chain.objs.size()=1 2026-02-24T22:34:25.777 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1695__shadow_.AdyRQiF6hb6Rb7rnKrcQuvbuBFNx64U_0 2026-02-24T22:34:25.777 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17997923862111883309', time=2026-02-25T00:27:02.984658+0000, chain.objs.size()=1 2026-02-24T22:34:25.777 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1704__shadow_.wiS3IX1riL82OfpX2Zvkv2_Dy8M1DlA_0 2026-02-24T22:34:25.778 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14359700091149034284', time=2026-02-25T00:27:19.800457+0000, chain.objs.size()=1 2026-02-24T22:34:25.778 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1736__shadow_.3UAVaKIxYiXgbq1aiqHZ556gS2_ShlD_0 2026-02-24T22:34:25.778 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9680339266019407251', time=2026-02-25T00:28:41.459487+0000, chain.objs.size()=1 2026-02-24T22:34:25.778 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1866__shadow_.g-aEcsLY9bhugudSKbljczizHarh7IO_0 2026-02-24T22:34:25.778 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17831174880072149347', time=2026-02-25T00:28:43.658600+0000, chain.objs.size()=1 2026-02-24T22:34:25.778 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1869__shadow:ba1nU1fNR3N1sfUQ2kL8K8Wm.5Ibl31_.IVNpljy9O7AjWtEyJ7RLhDmshakOZnh_0 2026-02-24T22:34:25.778 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6736037331509436964', time=2026-02-25T00:28:48.783010+0000, chain.objs.size()=1 2026-02-24T22:34:25.778 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1879__multipart_mymultipart.2~tS2la-8l_5yo_nD9oNgpi6697-eIX-d.1 2026-02-24T22:34:25.778 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='2~bVYs_cE9St3LXYLywunoGXw0aR_6UFr', time=2026-02-25T00:29:07.805336+0000, chain.objs.size()=1 2026-02-24T22:34:25.778 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~bVYs_cE9St3LXYLywunoGXw0aR_6UFr.1 2026-02-24T22:34:25.778 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='2~bVYs_cE9St3LXYLywunoGXw0aR_6UFr', time=2026-02-25T00:29:07.806770+0000, chain.objs.size()=1 2026-02-24T22:34:25.778 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~bVYs_cE9St3LXYLywunoGXw0aR_6UFr.2 2026-02-24T22:34:25.778 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='2~bVYs_cE9St3LXYLywunoGXw0aR_6UFr', time=2026-02-25T00:29:07.807746+0000, chain.objs.size()=1 2026-02-24T22:34:25.778 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.787+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~bVYs_cE9St3LXYLywunoGXw0aR_6UFr.3 2026-02-24T22:34:25.778 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='2~bVYs_cE9St3LXYLywunoGXw0aR_6UFr', time=2026-02-25T00:29:07.808618+0000, chain.objs.size()=1 2026-02-24T22:34:25.778 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~bVYs_cE9St3LXYLywunoGXw0aR_6UFr.4 2026-02-24T22:34:25.778 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='2~bVYs_cE9St3LXYLywunoGXw0aR_6UFr', time=2026-02-25T00:29:07.809533+0000, chain.objs.size()=1 2026-02-24T22:34:25.778 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~bVYs_cE9St3LXYLywunoGXw0aR_6UFr.5 2026-02-24T22:34:25.778 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='2~bVYs_cE9St3LXYLywunoGXw0aR_6UFr', time=2026-02-25T00:29:07.810404+0000, chain.objs.size()=1 2026-02-24T22:34:25.778 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~bVYs_cE9St3LXYLywunoGXw0aR_6UFr.6 2026-02-24T22:34:25.779 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11643659467343135455', time=2026-02-25T00:29:42.894608+0000, chain.objs.size()=1 2026-02-24T22:34:25.779 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1903__multipart_mymultipart.2~Ci95I4pN56E2CZDoSmtjHeSU0vFNhVJ.1 2026-02-24T22:34:25.779 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7031318121972410141', time=2026-02-25T00:30:22.611502+0000, chain.objs.size()=1 2026-02-24T22:34:25.779 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:9dnFf6pNM2ES7Hatua3qWw0ltvpbWFz_.pLXvjoQx7pG33cAMlX8QYp5T7gjQx9F_0 2026-02-24T22:34:25.779 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7803060392645730824', time=2026-02-25T00:30:27.240403+0000, chain.objs.size()=1 2026-02-24T22:34:25.779 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1942__shadow_.BMTYFjgH0D9jw6ePQsEaGYSnFdRFgWn_0 2026-02-24T22:34:25.779 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14162363377532652867', time=2026-02-25T00:30:39.798840+0000, chain.objs.size()=1 2026-02-24T22:34:25.779 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:Iuolkm7nbTDntQtKmoQtzhOOZctMHSH_.14qy-Gyx5L68HT0cUHa8UWDMyxhEJuQ_0 2026-02-24T22:34:25.779 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15628199061355078789', time=2026-02-25T00:30:43.755133+0000, chain.objs.size()=1 2026-02-24T22:34:25.779 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:2ZT30fhikhjpGD4ettZFxWyWMtxb66v_.Oe8Bp61XwPqSOVcBv7LdCB5JdEBKW7R_0 2026-02-24T22:34:25.779 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3384215463141051245', time=2026-02-25T00:30:49.386325+0000, chain.objs.size()=4 2026-02-24T22:34:25.779 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1948__multipart_testobj.2~ExYgJbahXfjW8KKqNW7peKOqasq6cWR.1 2026-02-24T22:34:25.780 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1948__multipart_testobj.2~ExYgJbahXfjW8KKqNW7peKOqasq6cWR.2 2026-02-24T22:34:25.780 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1948__multipart_testobj.2~ExYgJbahXfjW8KKqNW7peKOqasq6cWR.3 2026-02-24T22:34:25.780 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1948__multipart_testobj.2~ExYgJbahXfjW8KKqNW7peKOqasq6cWR.4 2026-02-24T22:34:25.780 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7855421253356593225', time=2026-02-25T00:30:50.995532+0000, chain.objs.size()=1 2026-02-24T22:34:25.780 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1949__shadow:QtHZqx6fFOBICu04hWH5cmg7lpM3-YA_.xuxPAdjfbkNPGBNP32eXeyHRl6E4soV_0 2026-02-24T22:34:25.781 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11559602601611792753', time=2026-02-25T00:30:52.383310+0000, chain.objs.size()=1 2026-02-24T22:34:25.781 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1950__shadow:Nykho.DgQ0mMgqLIfrSu1R5pC50D1bs_.2AcveQPCg_R1aoE4AlaLfrhlsOD0tB7_0 2026-02-24T22:34:25.781 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4174842399676617959', time=2026-02-25T00:30:52.475342+0000, chain.objs.size()=1 2026-02-24T22:34:25.781 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1950__shadow:1J0Mgj4PHIQwvlYxfMGpbYQEahXIYwp_.zUTs7Goq8b7TeuXXHhAVDabGV0-iy_X_0 2026-02-24T22:34:25.781 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.489026780241698372', time=2026-02-25T00:30:57.187401+0000, chain.objs.size()=1 2026-02-24T22:34:25.781 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:RsWjaW18r9i307J66mSeu02jisCJXnK_.nDhEXz2tsNgtO2Igou5yGNmU9Qy-Ush_0 2026-02-24T22:34:25.781 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.489026780241698372', time=2026-02-25T00:30:57.188070+0000, chain.objs.size()=1 2026-02-24T22:34:25.781 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:RsWjaW18r9i307J66mSeu02jisCJXnK_.nDhEXz2tsNgtO2Igou5yGNmU9Qy-Ush_0 2026-02-24T22:34:25.781 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6339440948618439427', time=2026-02-25T00:33:46.685544+0000, chain.objs.size()=1 2026-02-24T22:34:25.781 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1969__shadow_.D79ZOoBySz0ZgQWD1bIIy3mqDVyCtZg_0 2026-02-24T22:34:25.782 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.791+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.783 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2026-02-24T22:34:25.784 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.784 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/38182' 2026-02-24T22:34:25.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4180207770643741057', time=2026-02-25T00:15:34.861933+0000, chain.objs.size()=1 2026-02-24T22:34:25.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1509__shadow_.TtaC2q81tR68GthRWZMpiyQYGJZVmAg_0 2026-02-24T22:34:25.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9185279684554762486', time=2026-02-25T00:17:05.175426+0000, chain.objs.size()=1 2026-02-24T22:34:25.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522__shadow_.9k3FKk9R5DRqmK8nO7wRHeGkfGHPiY0_0 2026-02-24T22:34:25.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11679373108553859725', time=2026-02-25T00:17:07.123137+0000, chain.objs.size()=1 2026-02-24T22:34:25.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523__shadow_.cOdYNSIYwKxlBqzxvPeyYszSLS4QQpF_0 2026-02-24T22:34:25.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1486964655163243541', time=2026-02-25T00:17:39.885207+0000, chain.objs.size()=1 2026-02-24T22:34:25.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525__shadow_.Aygn2JSDIAo4F9QmxDFaSu-8h9lCf32_0 2026-02-24T22:34:25.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12200405352926333410', time=2026-02-25T00:17:42.538857+0000, chain.objs.size()=1 2026-02-24T22:34:25.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527__shadow_.3pobs_-GXFHhGN3E-up2liZOKwQaROk_0 2026-02-24T22:34:25.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6531929432999318741', time=2026-02-25T00:18:14.580912+0000, chain.objs.size()=1 2026-02-24T22:34:25.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528__shadow_.UAsi-cb31jVkQHXg3JyELlXjKVnQEHO_0 2026-02-24T22:34:25.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9433745703879634476', time=2026-02-25T00:18:17.231972+0000, chain.objs.size()=1 2026-02-24T22:34:25.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530__shadow_.VFxiJVOGpVsp6uh8cTG_WZ25MuN-r5E_0 2026-02-24T22:34:25.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13333271463008302023', time=2026-02-25T00:18:49.778729+0000, chain.objs.size()=1 2026-02-24T22:34:25.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531__shadow_.Ysqzj-bQdSyOim4OJqKb9gOX2slvzUP_0 2026-02-24T22:34:25.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4666577281069755942', time=2026-02-25T00:18:51.097344+0000, chain.objs.size()=1 2026-02-24T22:34:25.785 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532__shadow_.2ocDz1h6yMZVxfS0y1wJRzvmy_n3sQk_0 2026-02-24T22:34:25.786 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13099904502189393740', time=2026-02-25T00:19:57.119140+0000, chain.objs.size()=1 2026-02-24T22:34:25.786 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535__shadow_.6Lda1igPMWX1DifnUNEfg9EwGQeWP_I_0 2026-02-24T22:34:25.786 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13618040565173270428', time=2026-02-25T00:20:32.522182+0000, chain.objs.size()=1 2026-02-24T22:34:25.786 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538__shadow_.SUicheN-amHSopc99u9z6iKcu1SNN5N_0 2026-02-24T22:34:25.786 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15913231116360532038', time=2026-02-25T00:20:34.040051+0000, chain.objs.size()=1 2026-02-24T22:34:25.786 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539__shadow_.z6SXQMM8kiCxuk4QHjD839jCprbQjx8_0 2026-02-24T22:34:25.786 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14415806632887740028', time=2026-02-25T00:25:47.329861+0000, chain.objs.size()=1 2026-02-24T22:34:25.786 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.795+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1618__shadow_.bPmLPtCbIvXLDlxQm_6U7UaPuB27LNW_0 2026-02-24T22:34:25.786 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15662311708765500734', time=2026-02-25T00:25:59.097988+0000, chain.objs.size()=1 2026-02-24T22:34:25.786 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.jJucbF9pLiUo7PmAKvB552zvU3_k1cg_0 2026-02-24T22:34:25.786 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2353748861980818718', time=2026-02-25T00:25:59.130375+0000, chain.objs.size()=1 2026-02-24T22:34:25.786 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.NqCaFqdaN7esIYiW3Vzsl_6ygAXwtfR_0 2026-02-24T22:34:25.786 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11950805706767263651', time=2026-02-25T00:25:59.132317+0000, chain.objs.size()=1 2026-02-24T22:34:25.786 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.DZ1hQjH46Q0_YdBNAYvk-xlUAuDh4W4_0 2026-02-24T22:34:25.786 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18122390984757453778', time=2026-02-25T00:25:59.143955+0000, chain.objs.size()=1 2026-02-24T22:34:25.786 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.p226eVsA-Q2FbHF2iV3ReAV8VEBYW23_0 2026-02-24T22:34:25.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.543781807602727707', time=2026-02-25T00:25:59.154637+0000, chain.objs.size()=1 2026-02-24T22:34:25.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.MXgkBf3Wluja0QGDDQ016WKrTazzQyH_0 2026-02-24T22:34:25.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2659074716810832044', time=2026-02-25T00:25:59.161821+0000, chain.objs.size()=1 2026-02-24T22:34:25.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.mKOpjvosD_ef8cXh_huAeJmteX1WIQb_0 2026-02-24T22:34:25.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18390076362185189560', time=2026-02-25T00:25:59.178120+0000, chain.objs.size()=1 2026-02-24T22:34:25.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ItW76zGpzQg5sPgfsE0RzANciqYy4RV_0 2026-02-24T22:34:25.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8000824119547204473', time=2026-02-25T00:25:59.266221+0000, chain.objs.size()=1 2026-02-24T22:34:25.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.18VAOeWu4W2202uVV9xz3PbQx7gXtQw_0 2026-02-24T22:34:25.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2502547302200403913', time=2026-02-25T00:25:59.312960+0000, chain.objs.size()=1 2026-02-24T22:34:25.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.GX9k8dxTLzB2CSEvWYfyYRHnsGpD8YC_0 2026-02-24T22:34:25.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14821923452227577561', time=2026-02-25T00:25:59.322889+0000, chain.objs.size()=1 2026-02-24T22:34:25.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.i1osM3yqh6mi6sDc5n1xEFSu4j68ApA_0 2026-02-24T22:34:25.787 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8287489112422215228', time=2026-02-25T00:25:59.438077+0000, chain.objs.size()=1 2026-02-24T22:34:25.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ZTbgv32ZoU6LrepYpakJBVyTXRAezaC_0 2026-02-24T22:34:25.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17481728259062467339', time=2026-02-25T00:25:59.448050+0000, chain.objs.size()=1 2026-02-24T22:34:25.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.oedtF-Ir7kSpNTtr4lll1izRZKVgpq0_0 2026-02-24T22:34:25.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17782879884116704389', time=2026-02-25T00:25:59.470620+0000, chain.objs.size()=1 2026-02-24T22:34:25.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.LI06vQkbpEZwDPlF4Z5EsqXJKvhr8aU_0 2026-02-24T22:34:25.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2946443662481887070', time=2026-02-25T00:25:59.474756+0000, chain.objs.size()=1 2026-02-24T22:34:25.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.pRhmWxvdJIa4M3LN1OO7g23gfrq3-XU_0 2026-02-24T22:34:25.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12843118694591517202', time=2026-02-25T00:25:59.475163+0000, chain.objs.size()=1 2026-02-24T22:34:25.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.T6Z-hJ6Tok8R5zw0OMBe-KjmYnDsJoZ_0 2026-02-24T22:34:25.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8133168295329870846', time=2026-02-25T00:25:59.488107+0000, chain.objs.size()=1 2026-02-24T22:34:25.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.k7cVUnNTAAN25POO4IJLUQhBV72cyy3_0 2026-02-24T22:34:25.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5710970680368604935', time=2026-02-25T00:25:59.490495+0000, chain.objs.size()=1 2026-02-24T22:34:25.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.rXe9lmdkNescaAzJspf2At3QpBq5hcy_0 2026-02-24T22:34:25.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12534591953828598003', time=2026-02-25T00:25:59.576824+0000, chain.objs.size()=1 2026-02-24T22:34:25.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ce11WUkHVmc7E9Vg3pwVdXsRFbwWn5f_0 2026-02-24T22:34:25.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12058169373441456139', time=2026-02-25T00:25:59.587721+0000, chain.objs.size()=1 2026-02-24T22:34:25.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.pJXmiDQw-d_yrCVjmdmXNwJKrxwgkUM_0 2026-02-24T22:34:25.788 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15117924956727630674', time=2026-02-25T00:25:59.590764+0000, chain.objs.size()=1 2026-02-24T22:34:25.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.8xmFzoZIpZ7XUQXmkVYWzFT1cPIzN5X_0 2026-02-24T22:34:25.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1090943312763580332', time=2026-02-25T00:25:59.597330+0000, chain.objs.size()=1 2026-02-24T22:34:25.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Ji3gZTSO4MJv3lQcLhl0C0qfdlOMclN_0 2026-02-24T22:34:25.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2035137551432492370', time=2026-02-25T00:25:59.602568+0000, chain.objs.size()=1 2026-02-24T22:34:25.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.twBpoKkdEojQF9BzKZk9lKH8SzlwU37_0 2026-02-24T22:34:25.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4978914306116662889', time=2026-02-25T00:25:59.607892+0000, chain.objs.size()=1 2026-02-24T22:34:25.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.NHd_MkH3N-zdiadLYmDNqBc-khiRUOV_0 2026-02-24T22:34:25.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8641546337447257177', time=2026-02-25T00:25:59.631886+0000, chain.objs.size()=1 2026-02-24T22:34:25.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.NIqT7Gl_s7mdpvDWiz7emP6nZ1N9l7h_0 2026-02-24T22:34:25.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6918099531537810057', time=2026-02-25T00:25:59.657402+0000, chain.objs.size()=1 2026-02-24T22:34:25.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.XxTWskEcnbfi2HsIhS8s4QDNdacX-X3_0 2026-02-24T22:34:25.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9443549992643384718', time=2026-02-25T00:25:59.759333+0000, chain.objs.size()=1 2026-02-24T22:34:25.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.gmhvNsdPqDngNFsXCK5cugqU9jPiCyz_0 2026-02-24T22:34:25.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1837919042683579044', time=2026-02-25T00:25:59.938859+0000, chain.objs.size()=1 2026-02-24T22:34:25.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.UBc8StAm2kfz48HZDdur4DjitVof5q1_0 2026-02-24T22:34:25.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4039912472285854357', time=2026-02-25T00:26:00.072132+0000, chain.objs.size()=1 2026-02-24T22:34:25.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.uxVxenTZJOaORf_5NGL8sFCN4UzQ4y6_0 2026-02-24T22:34:25.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.686750393982193628', time=2026-02-25T00:26:00.074765+0000, chain.objs.size()=1 2026-02-24T22:34:25.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.HQfXaATeys_vRKeyqScnhNivIddxmwS_0 2026-02-24T22:34:25.789 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10974867170087684248', time=2026-02-25T00:26:00.077475+0000, chain.objs.size()=1 2026-02-24T22:34:25.790 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.uEfC_UVlCUdo7FuPhu6xu3Yp3EH4jxu_0 2026-02-24T22:34:25.790 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12369251674103603840', time=2026-02-25T00:26:00.127060+0000, chain.objs.size()=1 2026-02-24T22:34:25.790 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.KvkPQYLWkSi6DFBQFDApjHEfLNeT1ZM_0 2026-02-24T22:34:25.790 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12742431576761115522', time=2026-02-25T00:26:00.139966+0000, chain.objs.size()=1 2026-02-24T22:34:25.790 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.gec0TwqzxLcF2YRtt1fwFfPDPjKSukE_0 2026-02-24T22:34:25.790 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1086874274002510161', time=2026-02-25T00:26:00.148446+0000, chain.objs.size()=1 2026-02-24T22:34:25.790 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.9NfFXPSTuJXEsRRhPSN97IG0uqtEqbQ_0 2026-02-24T22:34:25.790 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17999235119446067780', time=2026-02-25T00:26:00.269711+0000, chain.objs.size()=1 2026-02-24T22:34:25.790 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.hIsUPEXdfTDa9dyUHZMFkSYST6iqgjA_0 2026-02-24T22:34:25.790 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8804628483938262885', time=2026-02-25T00:26:00.273226+0000, chain.objs.size()=1 2026-02-24T22:34:25.790 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.heAou8__74M7tn8U-bNup6Sa_hqF0r4_0 2026-02-24T22:34:25.790 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6918677058834668959', time=2026-02-25T00:26:18.510004+0000, chain.objs.size()=1 2026-02-24T22:34:25.790 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1658__shadow_.GJ8ALY8gYZDBL8WZO8hOAENIOqKusIL_0 2026-02-24T22:34:25.790 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7582088679887285343', time=2026-02-25T00:26:18.521645+0000, chain.objs.size()=1 2026-02-24T22:34:25.790 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.799+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1658__shadow_.URGCHxVM6nTtDEB7jco78t-as50Mf_j_0 2026-02-24T22:34:25.790 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14875919488503469494', time=2026-02-25T00:26:18.522060+0000, chain.objs.size()=1 2026-02-24T22:34:25.790 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1658__shadow_.a63iOSvayXz5yF1Gs9O9hDmFoK9lgZk_0 2026-02-24T22:34:25.790 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7723224840637596423', time=2026-02-25T00:26:19.461000+0000, chain.objs.size()=1 2026-02-24T22:34:25.790 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1659__shadow_.bKFFhfIdVOFc9v0Kegy8tg-fj5zs2zj_0 2026-02-24T22:34:25.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11447359753778085879', time=2026-02-25T00:26:35.775436+0000, chain.objs.size()=1 2026-02-24T22:34:25.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:34RHHAVkJFg8E-nqXlsbIkncMLeoIxJ_.o27bp9R4J1L3K5SvK_EwP0tRTUOvMAH_0 2026-02-24T22:34:25.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11447359753778085879', time=2026-02-25T00:26:35.783029+0000, chain.objs.size()=1 2026-02-24T22:34:25.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:34RHHAVkJFg8E-nqXlsbIkncMLeoIxJ_.o27bp9R4J1L3K5SvK_EwP0tRTUOvMAH_0 2026-02-24T22:34:25.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11447359753778085879', time=2026-02-25T00:26:35.808325+0000, chain.objs.size()=1 2026-02-24T22:34:25.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:34RHHAVkJFg8E-nqXlsbIkncMLeoIxJ_.o27bp9R4J1L3K5SvK_EwP0tRTUOvMAH_0 2026-02-24T22:34:25.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11447359753778085879', time=2026-02-25T00:26:35.825955+0000, chain.objs.size()=1 2026-02-24T22:34:25.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:34RHHAVkJFg8E-nqXlsbIkncMLeoIxJ_.o27bp9R4J1L3K5SvK_EwP0tRTUOvMAH_0 2026-02-24T22:34:25.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11447359753778085879', time=2026-02-25T00:26:35.858950+0000, chain.objs.size()=1 2026-02-24T22:34:25.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:34RHHAVkJFg8E-nqXlsbIkncMLeoIxJ_.o27bp9R4J1L3K5SvK_EwP0tRTUOvMAH_0 2026-02-24T22:34:25.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18099155581762198495', time=2026-02-25T00:26:44.792197+0000, chain.objs.size()=1 2026-02-24T22:34:25.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ErGLzPEtOZLf4mbUQC2dryMvsiGo7DC_0 2026-02-24T22:34:25.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14000827620347947859', time=2026-02-25T00:26:44.951366+0000, chain.objs.size()=1 2026-02-24T22:34:25.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.FzX-4fZyaSf3M1rzPyKo_6Pf25jciQ4_0 2026-02-24T22:34:25.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13972163825837258934', time=2026-02-25T00:26:44.953340+0000, chain.objs.size()=1 2026-02-24T22:34:25.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.tq8v-4jITQbMf4Pk08GTCZrVpPjITag_0 2026-02-24T22:34:25.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7315782968420664347', time=2026-02-25T00:26:44.959475+0000, chain.objs.size()=1 2026-02-24T22:34:25.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.qMgNKpXML961Ebt6vYF7hK2lCPDiUeV_0 2026-02-24T22:34:25.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17819810840254296645', time=2026-02-25T00:26:45.000804+0000, chain.objs.size()=1 2026-02-24T22:34:25.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.BfcAewWA-y_BxKk_QK7brusdRPBM3Hv_0 2026-02-24T22:34:25.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17800726680189951920', time=2026-02-25T00:26:45.114060+0000, chain.objs.size()=1 2026-02-24T22:34:25.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Ed8GM_cT6tub29fi9AvHspoQcrImVZb_0 2026-02-24T22:34:25.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5290066716226116487', time=2026-02-25T00:26:45.133745+0000, chain.objs.size()=1 2026-02-24T22:34:25.791 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.qiHuMK44KgA2wOt_vuRG1nGrDI_HfDm_0 2026-02-24T22:34:25.792 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2305596747820513662', time=2026-02-25T00:26:45.164777+0000, chain.objs.size()=1 2026-02-24T22:34:25.792 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.kQNGA1e1_0fsp3WI6fXFmnq-wEf_sD-_0 2026-02-24T22:34:25.792 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8738812839245902477', time=2026-02-25T00:26:45.273847+0000, chain.objs.size()=1 2026-02-24T22:34:25.792 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.lidlh2GPOKyovNdWCnj14k-S3t9AOaR_0 2026-02-24T22:34:25.792 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9263793392051495933', time=2026-02-25T00:26:45.304428+0000, chain.objs.size()=1 2026-02-24T22:34:25.792 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Gz8VeryhIPF6YB9n2hN-O46kiYo4KrA_0 2026-02-24T22:34:25.793 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5385340225934219783', time=2026-02-25T00:26:45.341393+0000, chain.objs.size()=1 2026-02-24T22:34:25.793 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.tr90Qf80b4WGY-EWPAUzAE2yGicqPP8_0 2026-02-24T22:34:25.793 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.723366339720819867', time=2026-02-25T00:26:45.430987+0000, chain.objs.size()=1 2026-02-24T22:34:25.793 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.gm6jd5Th8OTisQznGoudr_cfV01_2qT_0 2026-02-24T22:34:25.793 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2741847222651180906', time=2026-02-25T00:26:45.443169+0000, chain.objs.size()=1 2026-02-24T22:34:25.793 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.826HY5yXXLYvv_l7h1SMhL3cm44LX_C_0 2026-02-24T22:34:25.793 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17203053073835837253', time=2026-02-25T00:26:45.445507+0000, chain.objs.size()=1 2026-02-24T22:34:25.793 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.H2G2PSsO4Be2KV2EFRGh3XHiURml1Kz_0 2026-02-24T22:34:25.793 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11624926047790310567', time=2026-02-25T00:26:45.446839+0000, chain.objs.size()=1 2026-02-24T22:34:25.793 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.kT7cMJMhoXzN7-c9Z0AJ4r1XVmCbtb4_0 2026-02-24T22:34:25.793 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6653496193072910222', time=2026-02-25T00:26:45.449981+0000, chain.objs.size()=1 2026-02-24T22:34:25.793 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.FnhrAn-8GNuwKPjUaEp8l6PCi3F9maB_0 2026-02-24T22:34:25.793 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.291773819143000761', time=2026-02-25T00:26:45.462618+0000, chain.objs.size()=1 2026-02-24T22:34:25.793 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Ioz4ieuSe6Jf1sq7URi26pQgZwwlbhU_0 2026-02-24T22:34:25.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4025864454189167400', time=2026-02-25T00:26:45.472770+0000, chain.objs.size()=1 2026-02-24T22:34:25.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.hDwPLvLN_Ab9V9O6GxwhSuINZg9alpg_0 2026-02-24T22:34:25.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14480696569592278479', time=2026-02-25T00:26:45.473825+0000, chain.objs.size()=1 2026-02-24T22:34:25.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.jJLn18s2znJHodS9w3RHBJ-wnKNmQv3_0 2026-02-24T22:34:25.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6524748408381595538', time=2026-02-25T00:26:45.499619+0000, chain.objs.size()=1 2026-02-24T22:34:25.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.0fSD8N94vKmDgbgWs_PNT565yzwYJWF_0 2026-02-24T22:34:25.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13038397761321218144', time=2026-02-25T00:26:45.503529+0000, chain.objs.size()=1 2026-02-24T22:34:25.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.SLk7mKpF8BVVXzmOSoVKuPlHj1VZT_f_0 2026-02-24T22:34:25.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6124948042862038902', time=2026-02-25T00:26:45.579499+0000, chain.objs.size()=1 2026-02-24T22:34:25.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.PbYELVPsQYAMSVU6U1mggGwuNBSwcJ1_0 2026-02-24T22:34:25.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13501901853015006497', time=2026-02-25T00:26:45.587970+0000, chain.objs.size()=1 2026-02-24T22:34:25.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.2LPFjIBNg6c4s89c9pK9iuSxCRYgp2Y_0 2026-02-24T22:34:25.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15793321537289111980', time=2026-02-25T00:26:45.610379+0000, chain.objs.size()=1 2026-02-24T22:34:25.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.3-eexwvP0TZdDpN_fsdf3lB93CEyPX7_0 2026-02-24T22:34:25.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.581521029223368256', time=2026-02-25T00:26:45.836847+0000, chain.objs.size()=1 2026-02-24T22:34:25.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.WQuNudcitdT6LPyD9VxLUZ-eGyngpkp_0 2026-02-24T22:34:25.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.593511716115958135', time=2026-02-25T00:26:45.849903+0000, chain.objs.size()=1 2026-02-24T22:34:25.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.maX5r4jWA8ltUW660NAHn4jrW1FDGg__0 2026-02-24T22:34:25.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4100870241170694592', time=2026-02-25T00:26:45.928043+0000, chain.objs.size()=1 2026-02-24T22:34:25.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.4xA3tNj4Y_qM-9QmNF1NJWv0qdBkEml_0 2026-02-24T22:34:25.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.267050997203884158', time=2026-02-25T00:26:45.930331+0000, chain.objs.size()=1 2026-02-24T22:34:25.794 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.803+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Dn15kkvYBJxi3SsB_S2Sa_qvpmKSK-H_0 2026-02-24T22:34:25.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2818478031815595846', time=2026-02-25T00:26:45.953199+0000, chain.objs.size()=1 2026-02-24T22:34:25.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.vnd-HVr8RLu-qkNjNpKWI7poj1yli25_0 2026-02-24T22:34:25.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4722894262820369321', time=2026-02-25T00:26:46.022133+0000, chain.objs.size()=1 2026-02-24T22:34:25.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ilnZR-kIcXm5ylNIkXth-Adz7VCjpO9_0 2026-02-24T22:34:25.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2857726044304589165', time=2026-02-25T00:26:46.022913+0000, chain.objs.size()=1 2026-02-24T22:34:25.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.IRouB8Txd3Nw93xeoh5vaduNCTLMzDX_0 2026-02-24T22:34:25.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1936882224460210990', time=2026-02-25T00:26:46.101286+0000, chain.objs.size()=1 2026-02-24T22:34:25.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.nZHfmsBSTLHS2USI324Lhcj3vETE4N1_0 2026-02-24T22:34:25.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7610752607542985548', time=2026-02-25T00:26:46.122436+0000, chain.objs.size()=1 2026-02-24T22:34:25.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.htz87WzOvJ2tN_7JIj27KJ4iQAq_pmX_0 2026-02-24T22:34:25.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13123895842524175871', time=2026-02-25T00:26:53.190590+0000, chain.objs.size()=1 2026-02-24T22:34:25.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.TgFrB7C98O0A005dWin8DTsoaACD7W5_0 2026-02-24T22:34:25.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7868834942769439430', time=2026-02-25T00:26:53.208427+0000, chain.objs.size()=1 2026-02-24T22:34:25.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.X6Zuq8nAH22nCwdcnLGEMcvGbkl80wp_0 2026-02-24T22:34:25.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15629258953761222804', time=2026-02-25T00:26:53.212392+0000, chain.objs.size()=1 2026-02-24T22:34:25.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Gtxjrr29MXjgvstJSIAULnyMogxO9oW_0 2026-02-24T22:34:25.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14922900478324150141', time=2026-02-25T00:26:53.237453+0000, chain.objs.size()=1 2026-02-24T22:34:25.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.n1htMc5Hbm_sMgpIDq5rc8Y6kgD3rRO_0 2026-02-24T22:34:25.795 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4838995576258227451', time=2026-02-25T00:26:53.362190+0000, chain.objs.size()=1 2026-02-24T22:34:25.796 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.0t43r-1JSZM9tgEbF3cCaI8UIBrjIqx_0 2026-02-24T22:34:25.796 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3645728026611289135', time=2026-02-25T00:26:53.383581+0000, chain.objs.size()=1 2026-02-24T22:34:25.796 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.sVunJtulk0Kvh8qlrsJodJfD8nJ2mCG_0 2026-02-24T22:34:25.796 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5581808745919894683', time=2026-02-25T00:26:53.393385+0000, chain.objs.size()=1 2026-02-24T22:34:25.796 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.6cO1f_2oV621o7YhAZr4JyJfjMi4ptZ_0 2026-02-24T22:34:25.796 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18331144525900323090', time=2026-02-25T00:26:53.414114+0000, chain.objs.size()=1 2026-02-24T22:34:25.796 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.qddIVi_XcbJa1DxgLFjp552r7ovEX59_0 2026-02-24T22:34:25.796 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13630733877706607818', time=2026-02-25T00:26:53.416193+0000, chain.objs.size()=1 2026-02-24T22:34:25.796 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ZFXy8rUeQE9fIloPfKstNMKvFFFnbxI_0 2026-02-24T22:34:25.796 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10511973189715128166', time=2026-02-25T00:26:53.431977+0000, chain.objs.size()=1 2026-02-24T22:34:25.796 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.WwBhgZFPE81HhdsPuIYpTfK7jeQ1KYz_0 2026-02-24T22:34:25.797 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.807+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/38182 2026-02-24T22:34:25.799 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=40, truncated=0, next_marker='' 2026-02-24T22:34:25.799 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10983523405109176660', time=2026-02-25T00:26:53.525814+0000, chain.objs.size()=1 2026-02-24T22:34:25.799 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.RNuiSk5rPVZgV13Npx-ZT0rTK7as0Bd_0 2026-02-24T22:34:25.799 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1854927832540004685', time=2026-02-25T00:26:53.569544+0000, chain.objs.size()=1 2026-02-24T22:34:25.799 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.jvVzQcrG-M5IZ9hbe46I2eeWBfs8_CI_0 2026-02-24T22:34:25.799 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11118870353465458297', time=2026-02-25T00:26:53.593107+0000, chain.objs.size()=1 2026-02-24T22:34:25.799 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.NXFWFXk1aH7gbD3ScBL2LHz3Er9Z4qV_0 2026-02-24T22:34:25.799 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10483979870615921695', time=2026-02-25T00:26:53.760630+0000, chain.objs.size()=1 2026-02-24T22:34:25.799 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.WcsN4SBGwAOfZLh4ZYnCJENhKck6byd_0 2026-02-24T22:34:25.799 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15780572795621582207', time=2026-02-25T00:26:53.774204+0000, chain.objs.size()=1 2026-02-24T22:34:25.799 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.__NHfu5vSmC6yE7Ywm5fLkYAq6YH6WN_0 2026-02-24T22:34:25.799 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4627016190932187500', time=2026-02-25T00:26:53.835169+0000, chain.objs.size()=1 2026-02-24T22:34:25.799 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.VVScpckVIRRETDUvRID0WCanJJ68BWl_0 2026-02-24T22:34:25.799 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6754021475269157016', time=2026-02-25T00:26:53.852033+0000, chain.objs.size()=1 2026-02-24T22:34:25.799 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.WFoTvXe4hKc2AIrXTHkbi-ewtWZ3lSE_0 2026-02-24T22:34:25.799 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14597206975423965362', time=2026-02-25T00:26:53.926751+0000, chain.objs.size()=1 2026-02-24T22:34:25.799 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_._WbNvMtAdHm0VtRPJPjmXv7euVtJl3X_0 2026-02-24T22:34:25.799 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16577554033525430296', time=2026-02-25T00:26:53.943454+0000, chain.objs.size()=1 2026-02-24T22:34:25.799 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.LbEqCvl9igM5F7812UPb-d6JkFGlmS0_0 2026-02-24T22:34:25.799 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1571613711324538229', time=2026-02-25T00:26:54.040253+0000, chain.objs.size()=1 2026-02-24T22:34:25.799 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.iF8J9eQwHNy199tRoXv38wh8JfBeiwx_0 2026-02-24T22:34:25.799 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8716637075251720949', time=2026-02-25T00:26:54.131510+0000, chain.objs.size()=1 2026-02-24T22:34:25.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.JoO3oFdMGq30uTdJjjtq835WfkvCI6M_0 2026-02-24T22:34:25.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4681409851949489166', time=2026-02-25T00:26:54.161300+0000, chain.objs.size()=1 2026-02-24T22:34:25.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.03wIanOORdnYSBIw-QJT1E21F2kjWTR_0 2026-02-24T22:34:25.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17953380663265571838', time=2026-02-25T00:26:54.204530+0000, chain.objs.size()=1 2026-02-24T22:34:25.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.0fMLrGW_hqnC6A88sRiOYHc55IXGnzX_0 2026-02-24T22:34:25.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4752355248154253612', time=2026-02-25T00:26:54.210811+0000, chain.objs.size()=1 2026-02-24T22:34:25.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.WZVIfveVpNA3ADwPX2_1ZLDgnYsaKag_0 2026-02-24T22:34:25.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14264659739860114104', time=2026-02-25T00:26:54.214511+0000, chain.objs.size()=1 2026-02-24T22:34:25.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.0LWg4SsBKqY0w4Qjzq9bgpdty9WEddf_0 2026-02-24T22:34:25.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12335314812490296275', time=2026-02-25T00:26:54.302798+0000, chain.objs.size()=1 2026-02-24T22:34:25.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.d_letqegVMmlRaYwdhXe7JFYbzAPdJ9_0 2026-02-24T22:34:25.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2119860848831964527', time=2026-02-25T00:26:54.314822+0000, chain.objs.size()=1 2026-02-24T22:34:25.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.4wdgQ9IYgCLq899MTNaPz5NADV7m5jc_0 2026-02-24T22:34:25.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14143513836187832910', time=2026-02-25T00:26:54.383178+0000, chain.objs.size()=1 2026-02-24T22:34:25.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.N-yc-RbsDdhv8NCSgRhxuUtTo1KG5rH_0 2026-02-24T22:34:25.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12743913347778785452', time=2026-02-25T00:26:54.486901+0000, chain.objs.size()=1 2026-02-24T22:34:25.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.DrAJDryBFQVNUy748LivUWUZeXnlQ5F_0 2026-02-24T22:34:25.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14549956001846316749', time=2026-02-25T00:26:54.499130+0000, chain.objs.size()=1 2026-02-24T22:34:25.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.8SopRm9kd8onYBMpuuHTKGnXxCQMacC_0 2026-02-24T22:34:25.800 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11031279876670990102', time=2026-02-25T00:26:54.502120+0000, chain.objs.size()=1 2026-02-24T22:34:25.801 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.lWG0SOjOHI7YEdaCetTC3B9OzH3jwX1_0 2026-02-24T22:34:25.801 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13352419867696198696', time=2026-02-25T00:26:54.505839+0000, chain.objs.size()=1 2026-02-24T22:34:25.801 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.NFvYGMdsbiDIiB_65Ubi9modI3cLSRy_0 2026-02-24T22:34:25.801 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15784479927880999349', time=2026-02-25T00:26:54.510905+0000, chain.objs.size()=1 2026-02-24T22:34:25.801 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Vj-OnlBWidRkfBG0XahP1xASvYh_XLg_0 2026-02-24T22:34:25.801 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12421529196764386670', time=2026-02-25T00:28:19.565709+0000, chain.objs.size()=1 2026-02-24T22:34:25.801 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1824__shadow_.u-QaBNhZBusZkoVXmS41UNplkKeVQcZ_0 2026-02-24T22:34:25.801 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8767092525383032851', time=2026-02-25T00:28:33.182189+0000, chain.objs.size()=1 2026-02-24T22:34:25.801 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1850__shadow_.R1VN5Kbvy5ZVAHv5JQ3ac_3nORpP5oT_0 2026-02-24T22:34:25.801 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11553951660329413815', time=2026-02-25T00:28:36.288694+0000, chain.objs.size()=1 2026-02-24T22:34:25.801 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1855__shadow_.TuR1MjolvwPukdf34UrDz1M-mYTQk0o_0 2026-02-24T22:34:25.801 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1837092549216685687', time=2026-02-25T00:29:11.232740+0000, chain.objs.size()=2 2026-02-24T22:34:25.801 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1889__multipart_mymultipart.2~iYxZ_gI4zAMPen7oCUmTqZ1_PSmZkjK.1 2026-02-24T22:34:25.801 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1889__multipart_mymultipart.2~iYxZ_gI4zAMPen7oCUmTqZ1_PSmZkjK.2 2026-02-24T22:34:25.801 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12041243846472594214', time=2026-02-25T00:29:16.449108+0000, chain.objs.size()=2 2026-02-24T22:34:25.801 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1894__multipart_mymultipart.2~wYekf27L8JQ4qqq-4ebsFvlB1TgeFWl.1 2026-02-24T22:34:25.801 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1894__multipart_mymultipart.2~wYekf27L8JQ4qqq-4ebsFvlB1TgeFWl.2 2026-02-24T22:34:25.801 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='2~jRnSmZ1txKp6p3-SQ7aUPqPRjO15SLU', time=2026-02-25T00:29:21.214466+0000, chain.objs.size()=1 2026-02-24T22:34:25.801 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1901__multipart_mymultipart.2~jRnSmZ1txKp6p3-SQ7aUPqPRjO15SLU.3 2026-02-24T22:34:25.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17180035874066610394', time=2026-02-25T00:30:02.982483+0000, chain.objs.size()=1 2026-02-24T22:34:25.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1919__shadow_.1X5HrMR7tF3lNyKK0qbv134TAT-5DEF_0 2026-02-24T22:34:25.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13528802738633704644', time=2026-02-25T00:30:07.821623+0000, chain.objs.size()=1 2026-02-24T22:34:25.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1925__shadow_.2BOgDBoKaOf0sqhz2vHJUl0LloJ4NLC_0 2026-02-24T22:34:25.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2272870490195141652', time=2026-02-25T00:30:23.281851+0000, chain.objs.size()=1 2026-02-24T22:34:25.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:cuYM9H0Fo5lIcTEDOHY5OgctQuW4A.T_.dPzqSAk7-G66LHcoe-PNNIx328rYZlK_0 2026-02-24T22:34:25.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16616099482651189241', time=2026-02-25T00:30:38.461963+0000, chain.objs.size()=1 2026-02-24T22:34:25.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:wox2OEIm7n-9xvKGniET-k9Q5HoxqFq_.bYsWMZXTySj-jGSPGiLd4z9eVsWNwi8_0 2026-02-24T22:34:25.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.49963782050081199', time=2026-02-25T00:30:45.839079+0000, chain.objs.size()=1 2026-02-24T22:34:25.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:N6KARhU9fX3T59Wlk2kGr1X3JV64vSH_.6xZLcAraYw6pbBuPS3D42ekT1ng9i7A_0 2026-02-24T22:34:25.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17366115832692753277', time=2026-02-25T00:30:47.214131+0000, chain.objs.size()=1 2026-02-24T22:34:25.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.811+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:H0w.ggW0Ij52TX.PkV2BmL7suAkDC8X_.q_jIqAZH7pwoVmuAGMl_qJNNL5i4diO_0 2026-02-24T22:34:25.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.815+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13629681054871955079', time=2026-02-25T00:30:51.008716+0000, chain.objs.size()=1 2026-02-24T22:34:25.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.815+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1949__shadow:ygyJYQ0hbImQZ4ReYFE8xfBAxsGQjRH_.cquycsGpkri_TyMUyv3_Ya99-WmUcMW_0 2026-02-24T22:34:25.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.815+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12914647096366812959', time=2026-02-25T00:30:57.390525+0000, chain.objs.size()=1 2026-02-24T22:34:25.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.815+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:ZJyB8mNy5BT4illhDLl1hIsoLywoGWc_.XTJKCNHLKIyk0QC49p74uFbC-1XFd0T_0 2026-02-24T22:34:25.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.815+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12914647096366812959', time=2026-02-25T00:30:57.391036+0000, chain.objs.size()=1 2026-02-24T22:34:25.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.815+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:ZJyB8mNy5BT4illhDLl1hIsoLywoGWc_.XTJKCNHLKIyk0QC49p74uFbC-1XFd0T_0 2026-02-24T22:34:25.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.815+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12914647096366812959', time=2026-02-25T00:30:57.391506+0000, chain.objs.size()=1 2026-02-24T22:34:25.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.815+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:ZJyB8mNy5BT4illhDLl1hIsoLywoGWc_.XTJKCNHLKIyk0QC49p74uFbC-1XFd0T_0 2026-02-24T22:34:25.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.815+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3506452393681060854', time=2026-02-25T00:34:04.873211+0000, chain.objs.size()=1 2026-02-24T22:34:25.802 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.815+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1987__shadow_.IWLRL3oGlzRt2dsgV9NBAJnU4pmM-ZA_0 2026-02-24T22:34:25.803 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.815+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.805 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.815+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2026-02-24T22:34:25.806 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.807 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/39746' 2026-02-24T22:34:25.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3919362993914401659', time=2026-02-25T00:15:09.910849+0000, chain.objs.size()=1 2026-02-24T22:34:25.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1505__shadow_.gNDkQ2SfpIBJ92q_LVeek51zpie-f-j_0 2026-02-24T22:34:25.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11810741133710563624', time=2026-02-25T00:15:09.916427+0000, chain.objs.size()=1 2026-02-24T22:34:25.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1505__shadow_.ppVAAgNOI_xecoTNQibUXpnjsfnECD9_0 2026-02-24T22:34:25.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14580771443633498059', time=2026-02-25T00:15:34.609407+0000, chain.objs.size()=1 2026-02-24T22:34:25.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506__shadow_.XKxo8uUMDUo-InHXqbai9FtIq0UvkWQ_0 2026-02-24T22:34:25.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15636809563206940919', time=2026-02-25T00:15:34.741305+0000, chain.objs.size()=3 2026-02-24T22:34:25.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-FROZEN.2~5YnNWY2srlSZqsTYGvvi4fm2BZ8_YNh.1 2026-02-24T22:34:25.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-FROZEN.2~5YnNWY2srlSZqsTYGvvi4fm2BZ8_YNh.2 2026-02-24T22:34:25.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-FROZEN.2~5YnNWY2srlSZqsTYGvvi4fm2BZ8_YNh.3 2026-02-24T22:34:25.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10654159368664500367', time=2026-02-25T00:16:32.609985+0000, chain.objs.size()=1 2026-02-24T22:34:25.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520__shadow_.3KkwxZ5WbaBW-yWn_ojOFzQxzEEWzXt_0 2026-02-24T22:34:25.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5513023717990962255', time=2026-02-25T00:16:32.611715+0000, chain.objs.size()=1 2026-02-24T22:34:25.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520__shadow_.Mdu_vgPj-tCTM0M5fXh7mGBUKu4kuh6_0 2026-02-24T22:34:25.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12879982683074983106', time=2026-02-25T00:16:33.507703+0000, chain.objs.size()=1 2026-02-24T22:34:25.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521__shadow_.youBjLXbka3UoXlcJkTZF3HFo2dwPM5_0 2026-02-24T22:34:25.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16586363496999261970', time=2026-02-25T00:17:07.114291+0000, chain.objs.size()=1 2026-02-24T22:34:25.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523__shadow_.J3MsJObo6yZ4CZe4Kt5CEPkT7RZztrP_0 2026-02-24T22:34:25.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5326138491792403467', time=2026-02-25T00:17:08.004417+0000, chain.objs.size()=1 2026-02-24T22:34:25.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524__shadow_.g9aY8ctqC8Ke6SzycIZ-iu-g1Nd4akD_0 2026-02-24T22:34:25.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12420793920526734941', time=2026-02-25T00:17:39.879721+0000, chain.objs.size()=1 2026-02-24T22:34:25.808 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525__shadow_.tKmeucwQuoCFXgjRsfq_Sd0b2zuTm-u_0 2026-02-24T22:34:25.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16265269115429514941', time=2026-02-25T00:17:41.419866+0000, chain.objs.size()=1 2026-02-24T22:34:25.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526__shadow_.sHYvzRGEYW58yKYaQOqErPT0yRH7dLq_0 2026-02-24T22:34:25.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16869744018928735186', time=2026-02-25T00:18:16.123507+0000, chain.objs.size()=1 2026-02-24T22:34:25.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529__shadow_.axzH3VwVef4OVUSxTcIsaQZHceCj0E2_0 2026-02-24T22:34:25.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6020739266956066739', time=2026-02-25T00:18:48.184667+0000, chain.objs.size()=1 2026-02-24T22:34:25.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530__shadow_.0x4L-Y_dQi9XKvRZK7IxeDtQ9-vEE3P_0 2026-02-24T22:34:25.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2127077947162314030', time=2026-02-25T00:18:48.185851+0000, chain.objs.size()=1 2026-02-24T22:34:25.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530__shadow_.9yDveyYWzXwny2rL_Ncbr7WRAFO9Rvz_0 2026-02-24T22:34:25.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15367898514928091334', time=2026-02-25T00:18:49.766380+0000, chain.objs.size()=1 2026-02-24T22:34:25.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531__shadow_.jBs7Y6xiX-cYlmm7JfUKihn5wJ1FMEA_0 2026-02-24T22:34:25.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6977442614159442833', time=2026-02-25T00:24:15.521415+0000, chain.objs.size()=1 2026-02-24T22:34:25.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1569__shadow_.SVDtPBUTOpavpfQsi6fBPYoh6cnTMDp_0 2026-02-24T22:34:25.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13373072443978973736', time=2026-02-25T00:25:59.251895+0000, chain.objs.size()=1 2026-02-24T22:34:25.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_._o1eAGD7-ju1nTBpJAU7FW0k_OJod7K_0 2026-02-24T22:34:25.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13055444171090579439', time=2026-02-25T00:25:59.318888+0000, chain.objs.size()=1 2026-02-24T22:34:25.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.womN1EvRqAXkVcuZV20ksAeF3MlOSh__0 2026-02-24T22:34:25.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7133926659998239435', time=2026-02-25T00:25:59.419469+0000, chain.objs.size()=1 2026-02-24T22:34:25.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.4-fftBP32Sl5dcknshuBaEA7SFB633__0 2026-02-24T22:34:25.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7292894842018681406', time=2026-02-25T00:25:59.442671+0000, chain.objs.size()=1 2026-02-24T22:34:25.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.9ffMv4o6Zpdt0iLSCoLFiBduzzFFfGg_0 2026-02-24T22:34:25.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2865994234584209483', time=2026-02-25T00:25:59.443620+0000, chain.objs.size()=1 2026-02-24T22:34:25.809 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.AKaxwWcImjkF_gWhhIvtiYVoAeCJUlg_0 2026-02-24T22:34:25.810 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17354423006260528563', time=2026-02-25T00:25:59.486420+0000, chain.objs.size()=1 2026-02-24T22:34:25.810 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_._THiuvFcMz97fg72ZCeSVzk3wJi7Kld_0 2026-02-24T22:34:25.810 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11649078860831654221', time=2026-02-25T00:25:59.583777+0000, chain.objs.size()=1 2026-02-24T22:34:25.810 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.RpVZppCrpea3tPAYeBzRpJaFSflXeZM_0 2026-02-24T22:34:25.810 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8901770413530613755', time=2026-02-25T00:25:59.585957+0000, chain.objs.size()=1 2026-02-24T22:34:25.810 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.819+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.TDYejpVuJx7sq0mmHIwiwzRaKbOEqdi_0 2026-02-24T22:34:25.810 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8524522307489708119', time=2026-02-25T00:25:59.610348+0000, chain.objs.size()=1 2026-02-24T22:34:25.810 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.bK7qzV_4OSrmgfXKdIPVOnhef1a2cZd_0 2026-02-24T22:34:25.810 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11404240141572170756', time=2026-02-25T00:25:59.619443+0000, chain.objs.size()=1 2026-02-24T22:34:25.810 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.gp1mB9vPxUq0N-WF_nPEWPvU0C91c7r_0 2026-02-24T22:34:25.810 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1214892187714813317', time=2026-02-25T00:25:59.631819+0000, chain.objs.size()=1 2026-02-24T22:34:25.810 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Z0dWxnWl-D8oC4Rjcsv12ME0Yf9eZbY_0 2026-02-24T22:34:25.810 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2605767527812797980', time=2026-02-25T00:25:59.637620+0000, chain.objs.size()=1 2026-02-24T22:34:25.810 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Fyfhq4bz_ZsVG2KcCy1wfovUuj9sNSc_0 2026-02-24T22:34:25.810 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17975622458603699016', time=2026-02-25T00:25:59.721911+0000, chain.objs.size()=1 2026-02-24T22:34:25.810 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.CXi-VpohTR1O1295wO5SKi8X0e6zMk3_0 2026-02-24T22:34:25.810 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13255512928340917351', time=2026-02-25T00:25:59.727836+0000, chain.objs.size()=1 2026-02-24T22:34:25.810 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Nw8o0S8AAw7L982R9_JfDUilp2zpSy1_0 2026-02-24T22:34:25.811 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2889755642045472098', time=2026-02-25T00:25:59.740250+0000, chain.objs.size()=1 2026-02-24T22:34:25.811 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.9nE2wyZbiKZmGi4xwi1tJbW80nU3qJC_0 2026-02-24T22:34:25.811 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11265649284258907844', time=2026-02-25T00:25:59.764131+0000, chain.objs.size()=1 2026-02-24T22:34:25.811 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.fUhzTHb86i_BTLnqu4gVyQ72QiZHlED_0 2026-02-24T22:34:25.811 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.695716139654008730', time=2026-02-25T00:25:59.766961+0000, chain.objs.size()=1 2026-02-24T22:34:25.811 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.RBWFsmsKvntLST3sgTzSZFbxDESYrN7_0 2026-02-24T22:34:25.811 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6983563757108541092', time=2026-02-25T00:25:59.781423+0000, chain.objs.size()=1 2026-02-24T22:34:25.811 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.fcCC2agKY1-X3NEdNEFEfdtAejeci1Y_0 2026-02-24T22:34:25.811 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4904507776989078506', time=2026-02-25T00:25:59.795716+0000, chain.objs.size()=1 2026-02-24T22:34:25.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.1fTB8TyMs8EvE-767WaSqqDWeU11Jr8_0 2026-02-24T22:34:25.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.489077665264932207', time=2026-02-25T00:25:59.891897+0000, chain.objs.size()=1 2026-02-24T22:34:25.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ANMytlyVoqWDj8ZN67KZ7V2bmQ4he-x_0 2026-02-24T22:34:25.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18066940633257200326', time=2026-02-25T00:25:59.899062+0000, chain.objs.size()=1 2026-02-24T22:34:25.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.6LYNj64Vc6I6Y77N6anTohoAvqRr6CQ_0 2026-02-24T22:34:25.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8759104923217617413', time=2026-02-25T00:25:59.909405+0000, chain.objs.size()=1 2026-02-24T22:34:25.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.O-o3udzHR9zcLyXNhRycy4e4Rf_Gq-l_0 2026-02-24T22:34:25.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7575506788568478675', time=2026-02-25T00:25:59.915989+0000, chain.objs.size()=1 2026-02-24T22:34:25.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.EPlD8ESy2YpSJHSuKSSMBwMBet__7J2_0 2026-02-24T22:34:25.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18287648106822406422', time=2026-02-25T00:26:00.067764+0000, chain.objs.size()=1 2026-02-24T22:34:25.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Z8jh9H7Xqw9cc1uqZAqoHfJRx4CdXIG_0 2026-02-24T22:34:25.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17476544416308653137', time=2026-02-25T00:26:00.068110+0000, chain.objs.size()=1 2026-02-24T22:34:25.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.09IZJNGvKDNw1xMQpj0vt_DXgH3vm8P_0 2026-02-24T22:34:25.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14156072659357783916', time=2026-02-25T00:26:00.117228+0000, chain.objs.size()=1 2026-02-24T22:34:25.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.gSznWOecoTV2u4xU8-XW1THlYiA2M2X_0 2026-02-24T22:34:25.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12791208662945787609', time=2026-02-25T00:26:00.207180+0000, chain.objs.size()=1 2026-02-24T22:34:25.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.0rAFKaZpCOsm1qotoGiEw0y3Bk07JE2_0 2026-02-24T22:34:25.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12019689723729593920', time=2026-02-25T00:26:00.247229+0000, chain.objs.size()=1 2026-02-24T22:34:25.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.nI2KPziUbXzUGHtpb3IpsZDtBOAkT77_0 2026-02-24T22:34:25.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6989963533946415195', time=2026-02-25T00:26:00.265923+0000, chain.objs.size()=1 2026-02-24T22:34:25.812 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ox1k5HTzur_d8GeLxkZY7Hn2ik8ZxMB_0 2026-02-24T22:34:25.813 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.277376157907525450', time=2026-02-25T00:26:00.774678+0000, chain.objs.size()=1 2026-02-24T22:34:25.813 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1629__shadow_.1r-PTm_IL57th5ixtm-9awOr9sEsl_p_0 2026-02-24T22:34:25.813 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6250416970174016995', time=2026-02-25T00:26:12.938469+0000, chain.objs.size()=1 2026-02-24T22:34:25.813 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1650__shadow_.kiVqq-9T-ePK_OABoL8opsMOi3GUZPm_0 2026-02-24T22:34:25.813 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9006494228765280232', time=2026-02-25T00:26:16.574585+0000, chain.objs.size()=1 2026-02-24T22:34:25.813 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1656__shadow_.EHPl27V97Jx7Nfh43naiOMmTDAT-8az_0 2026-02-24T22:34:25.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4044373812134711274', time=2026-02-25T00:26:18.502461+0000, chain.objs.size()=1 2026-02-24T22:34:25.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1658__shadow_.PuaH1PIQqZjg4hAnWkYN6xCDE5Ws4C4_0 2026-02-24T22:34:25.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11994034182213659509', time=2026-02-25T00:26:19.461295+0000, chain.objs.size()=1 2026-02-24T22:34:25.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1659__shadow_._vk93O2t0tEULKr3DlvzCJD8olAgSRn_0 2026-02-24T22:34:25.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8669719176846848989', time=2026-02-25T00:26:22.290574+0000, chain.objs.size()=1 2026-02-24T22:34:25.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1664__shadow_.SFWnqmzk8F87dwFAIA-6JuWD-r3F5JO_0 2026-02-24T22:34:25.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11291684130189870639', time=2026-02-25T00:26:35.759536+0000, chain.objs.size()=1 2026-02-24T22:34:25.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:M.iEN9g1d.fbJ-c1BaiHR0DXrj..mdk_.nhO_hb9QQmPUI8ykOGaJiW-xKlM2Xuq_0 2026-02-24T22:34:25.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11291684130189870639', time=2026-02-25T00:26:35.781510+0000, chain.objs.size()=1 2026-02-24T22:34:25.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:M.iEN9g1d.fbJ-c1BaiHR0DXrj..mdk_.nhO_hb9QQmPUI8ykOGaJiW-xKlM2Xuq_0 2026-02-24T22:34:25.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11291684130189870639', time=2026-02-25T00:26:35.794122+0000, chain.objs.size()=1 2026-02-24T22:34:25.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:M.iEN9g1d.fbJ-c1BaiHR0DXrj..mdk_.nhO_hb9QQmPUI8ykOGaJiW-xKlM2Xuq_0 2026-02-24T22:34:25.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11291684130189870639', time=2026-02-25T00:26:35.811862+0000, chain.objs.size()=1 2026-02-24T22:34:25.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:M.iEN9g1d.fbJ-c1BaiHR0DXrj..mdk_.nhO_hb9QQmPUI8ykOGaJiW-xKlM2Xuq_0 2026-02-24T22:34:25.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11291684130189870639', time=2026-02-25T00:26:35.859245+0000, chain.objs.size()=1 2026-02-24T22:34:25.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:M.iEN9g1d.fbJ-c1BaiHR0DXrj..mdk_.nhO_hb9QQmPUI8ykOGaJiW-xKlM2Xuq_0 2026-02-24T22:34:25.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9430981434853863457', time=2026-02-25T00:26:44.789735+0000, chain.objs.size()=1 2026-02-24T22:34:25.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.SDHUTHkqK3tLOanC2wbsUbdPgDgnMi8_0 2026-02-24T22:34:25.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13653917478695935835', time=2026-02-25T00:26:44.795260+0000, chain.objs.size()=1 2026-02-24T22:34:25.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.823+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.kccInFtek3zsuPdKHg1zw-AsTPkYay7_0 2026-02-24T22:34:25.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3716064408867552793', time=2026-02-25T00:26:44.808890+0000, chain.objs.size()=1 2026-02-24T22:34:25.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.LQe_kbry-4tUM3ES_v33iJEBtkl5pqN_0 2026-02-24T22:34:25.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11150549210694821393', time=2026-02-25T00:26:44.817983+0000, chain.objs.size()=1 2026-02-24T22:34:25.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.__BN3cb1GjjEAyg3zb5n_7uMwp8zyuq_0 2026-02-24T22:34:25.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17462145320542875222', time=2026-02-25T00:26:44.827856+0000, chain.objs.size()=1 2026-02-24T22:34:25.814 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.CyTKnKD9r2sIFjXWgfUjB_E3zEL66s9_0 2026-02-24T22:34:25.815 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.238730499968144972', time=2026-02-25T00:26:44.936094+0000, chain.objs.size()=1 2026-02-24T22:34:25.815 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.7a0i5TP8ZMNU1Y8IlTwKMlCVkcfsVYa_0 2026-02-24T22:34:25.815 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3771403532729343563', time=2026-02-25T00:26:44.947944+0000, chain.objs.size()=1 2026-02-24T22:34:25.815 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.sf6w_4TMCQVaufnwXRrWCRPVPEKzhM7_0 2026-02-24T22:34:25.815 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.264893392761118345', time=2026-02-25T00:26:44.965997+0000, chain.objs.size()=1 2026-02-24T22:34:25.815 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.CkaisJIBxwLRfo6EvCdR80TiDgVa0me_0 2026-02-24T22:34:25.815 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12979107097038142645', time=2026-02-25T00:26:44.966976+0000, chain.objs.size()=1 2026-02-24T22:34:25.815 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Lk-GvTtK8wa--ZzL5r8TYrt7fjdRumS_0 2026-02-24T22:34:25.815 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1705552346063645254', time=2026-02-25T00:26:44.979426+0000, chain.objs.size()=1 2026-02-24T22:34:25.815 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.RvxMJDbBkhY8WvUre2WvqqiMT7mrQHm_0 2026-02-24T22:34:25.815 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7934144687895067104', time=2026-02-25T00:26:44.996091+0000, chain.objs.size()=1 2026-02-24T22:34:25.815 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.60lH6k_gDoV9H88H-ruo_UEiM8WwRUB_0 2026-02-24T22:34:25.815 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9470794819787151698', time=2026-02-25T00:26:45.024651+0000, chain.objs.size()=1 2026-02-24T22:34:25.815 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.wxpfwFhQUxPyw-ro_PWEMA8eG76AhKZ_0 2026-02-24T22:34:25.815 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13179552377690439112', time=2026-02-25T00:26:45.025532+0000, chain.objs.size()=1 2026-02-24T22:34:25.815 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.zIpeXaUs-7IRR3bFitLy_oDWkLEFC70_0 2026-02-24T22:34:25.815 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.322548400439062840', time=2026-02-25T00:26:45.098576+0000, chain.objs.size()=1 2026-02-24T22:34:25.815 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.NMTlma8fEOmVLP-NpyuV__ko0w0EXqC_0 2026-02-24T22:34:25.816 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8921265334596622408', time=2026-02-25T00:26:45.164585+0000, chain.objs.size()=1 2026-02-24T22:34:25.816 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.KIYETgGi48IvAdYCtqhyktFPFJpN4G0_0 2026-02-24T22:34:25.816 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2407149344841032461', time=2026-02-25T00:26:45.180714+0000, chain.objs.size()=1 2026-02-24T22:34:25.816 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.9fZpC7HXd2bjukp35DufOyEWlgGkzmK_0 2026-02-24T22:34:25.816 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14448321353128472920', time=2026-02-25T00:26:45.181750+0000, chain.objs.size()=1 2026-02-24T22:34:25.816 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.MtP-fvNdUDbUIfZ_I6UIcEDl6maFSN2_0 2026-02-24T22:34:25.816 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11650902468823932238', time=2026-02-25T00:26:45.257523+0000, chain.objs.size()=1 2026-02-24T22:34:25.816 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.paQEN8Im857lE6yriNF2SmMPWiGFxMI_0 2026-02-24T22:34:25.816 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12925431955312457949', time=2026-02-25T00:26:45.301343+0000, chain.objs.size()=1 2026-02-24T22:34:25.816 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.98rguLAL7I2FNEXUR1fnUDj-R9xkM3W_0 2026-02-24T22:34:25.816 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4529898906055018451', time=2026-02-25T00:26:45.400811+0000, chain.objs.size()=1 2026-02-24T22:34:25.816 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.bBohiwssv9mCnyu1GgCRp8HEqSHn1XV_0 2026-02-24T22:34:25.816 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17740585773232473337', time=2026-02-25T00:26:45.416731+0000, chain.objs.size()=1 2026-02-24T22:34:25.816 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.R14fD1bkdW4A_c440ARyNCR3OCCXyRE_0 2026-02-24T22:34:25.816 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12582601342961279420', time=2026-02-25T00:26:45.419354+0000, chain.objs.size()=1 2026-02-24T22:34:25.816 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.hYgSGIudWyKEehZMOA_4-JxoZObIuE4_0 2026-02-24T22:34:25.816 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10992904829558693060', time=2026-02-25T00:26:45.446714+0000, chain.objs.size()=1 2026-02-24T22:34:25.816 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.LQlKJ3IC91PhOZ_bjii5wJaIkaJYiIi_0 2026-02-24T22:34:25.816 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6307822202074256254', time=2026-02-25T00:26:45.472347+0000, chain.objs.size()=1 2026-02-24T22:34:25.816 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.PdJWR7chS__3gO1adKut-ewfrPIMUv9_0 2026-02-24T22:34:25.816 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15544372171447264663', time=2026-02-25T00:26:45.828908+0000, chain.objs.size()=1 2026-02-24T22:34:25.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_._UxyR6l74So9TPZKNdjesi-6brkR_qP_0 2026-02-24T22:34:25.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11314389484975799203', time=2026-02-25T00:26:45.917515+0000, chain.objs.size()=1 2026-02-24T22:34:25.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.U1ABWC91XJZTWXNxNt1gHTkiH5OBPjt_0 2026-02-24T22:34:25.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6947844190944050189', time=2026-02-25T00:26:45.954959+0000, chain.objs.size()=1 2026-02-24T22:34:25.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ZzZVHUlJfBKRsXCHvB2krkMNlNBCeLr_0 2026-02-24T22:34:25.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1973750563779594672', time=2026-02-25T00:26:46.000800+0000, chain.objs.size()=1 2026-02-24T22:34:25.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.2JGIT8MsYVVsX48-ALmH9adVmjyShLX_0 2026-02-24T22:34:25.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11135753168797793699', time=2026-02-25T00:26:46.012092+0000, chain.objs.size()=1 2026-02-24T22:34:25.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.tVKCK7lqfdGcYIhjJOvD5dLf9Og18_s_0 2026-02-24T22:34:25.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13369122806243073559', time=2026-02-25T00:26:46.102469+0000, chain.objs.size()=1 2026-02-24T22:34:25.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.euq0y_W9s-y7-YBgOTK0og7bvT51kVM_0 2026-02-24T22:34:25.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13438100310342624277', time=2026-02-25T00:26:46.130062+0000, chain.objs.size()=1 2026-02-24T22:34:25.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Yz3Bq2F2E42qvw58GVzR-ZkClxfRRAS_0 2026-02-24T22:34:25.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17348171909469017462', time=2026-02-25T00:26:46.138485+0000, chain.objs.size()=1 2026-02-24T22:34:25.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.H180Sfbil2wTsCE9fEYgRZ0iCZXBR4H_0 2026-02-24T22:34:25.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11305978264723490920', time=2026-02-25T00:26:46.141106+0000, chain.objs.size()=1 2026-02-24T22:34:25.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.I-eFc0bypsVvozLRPTLUqnbmckWvgrD_0 2026-02-24T22:34:25.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.898872474270834070', time=2026-02-25T00:26:46.141924+0000, chain.objs.size()=1 2026-02-24T22:34:25.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.b_sihDX6a0_scfgxD4z8iI2ZVaF9eJk_0 2026-02-24T22:34:25.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17339632655239701319', time=2026-02-25T00:26:53.137563+0000, chain.objs.size()=1 2026-02-24T22:34:25.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.-6urw59KtQ3RNHzIlAgAdMhbUOi9nae_0 2026-02-24T22:34:25.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18014497115253901955', time=2026-02-25T00:26:53.195294+0000, chain.objs.size()=1 2026-02-24T22:34:25.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.40FLOY40T-NyCyPQhd_UFBwH5G2_b5c_0 2026-02-24T22:34:25.817 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1307411802971834037', time=2026-02-25T00:26:53.211411+0000, chain.objs.size()=1 2026-02-24T22:34:25.818 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.dJpb3NBsg0MeK5MCOtu9XMTCjJuYbHS_0 2026-02-24T22:34:25.818 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2130085167209333845', time=2026-02-25T00:26:53.238745+0000, chain.objs.size()=1 2026-02-24T22:34:25.818 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.GguZp1cYbtSjlAM_rb_K7XPRevVVNQh_0 2026-02-24T22:34:25.818 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3400477676612441471', time=2026-02-25T00:26:53.326038+0000, chain.objs.size()=1 2026-02-24T22:34:25.818 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.as_WkHMTubtkIirvBwBLFyIfTsIgIjF_0 2026-02-24T22:34:25.818 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8608496640483641125', time=2026-02-25T00:26:53.336761+0000, chain.objs.size()=1 2026-02-24T22:34:25.818 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.nIe50aN3N1KRl3w6TAtZi3ZUtjuNUt6_0 2026-02-24T22:34:25.818 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2956164160072945352', time=2026-02-25T00:26:53.357437+0000, chain.objs.size()=1 2026-02-24T22:34:25.818 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.-4DdfiuyP1YjG11u15wfPHyPbE4TT1Z_0 2026-02-24T22:34:25.818 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2349873441860574221', time=2026-02-25T00:26:53.506068+0000, chain.objs.size()=1 2026-02-24T22:34:25.818 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.827+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.23PFuNHQGUghQNrD8k5GmDdC9Bx4NZr_0 2026-02-24T22:34:25.818 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15423025308977575829', time=2026-02-25T00:26:53.507262+0000, chain.objs.size()=1 2026-02-24T22:34:25.818 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.QBsT4OGQtO9sXy3H2kbBtpSSv1SliY8_0 2026-02-24T22:34:25.818 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9738142657388669040', time=2026-02-25T00:26:53.527479+0000, chain.objs.size()=1 2026-02-24T22:34:25.818 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.32v0XZANlRoIFIhmA_VmUEcR0oLZHGr_0 2026-02-24T22:34:25.820 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/39746 2026-02-24T22:34:25.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=44, truncated=0, next_marker='' 2026-02-24T22:34:25.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15278796799114753949', time=2026-02-25T00:26:53.572396+0000, chain.objs.size()=1 2026-02-24T22:34:25.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.XUVVgZHVbTKbbeUBJSMs9X82eNqzyz3_0 2026-02-24T22:34:25.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17507839287572000309', time=2026-02-25T00:26:53.574637+0000, chain.objs.size()=1 2026-02-24T22:34:25.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.xjFeB2uqYgfaV9xHR7gnYluvsxA40PY_0 2026-02-24T22:34:25.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8993610834524329647', time=2026-02-25T00:26:53.590876+0000, chain.objs.size()=1 2026-02-24T22:34:25.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.gZ9mMNERmr6UhrYt47_n0Pty4LTw01d_0 2026-02-24T22:34:25.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16783393322063390616', time=2026-02-25T00:26:53.594963+0000, chain.objs.size()=1 2026-02-24T22:34:25.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.S-eEIhrcWcm9VPs39EwanUOX7u5dquL_0 2026-02-24T22:34:25.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10088422734061713031', time=2026-02-25T00:26:53.753635+0000, chain.objs.size()=1 2026-02-24T22:34:25.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.0PCKTtjXxen_IiriNA72xcpldOFHU9m_0 2026-02-24T22:34:25.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17606352331778662119', time=2026-02-25T00:26:53.778830+0000, chain.objs.size()=1 2026-02-24T22:34:25.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.R2or84Kl_cOpebMu_8gUDNOiS3fHQFW_0 2026-02-24T22:34:25.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13146514214951739317', time=2026-02-25T00:26:53.792631+0000, chain.objs.size()=1 2026-02-24T22:34:25.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.vvodcML1lH07moP-WHDKPaI0lMKDlq9_0 2026-02-24T22:34:25.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.260470059375278622', time=2026-02-25T00:26:53.798814+0000, chain.objs.size()=1 2026-02-24T22:34:25.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.hdSABzhY1CXQoIBSWAVMeIqw1-aXQDw_0 2026-02-24T22:34:25.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4588142373954644877', time=2026-02-25T00:26:53.926353+0000, chain.objs.size()=1 2026-02-24T22:34:25.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.0yXaymxwyfLiIxSxolWEutgYL4BRAFu_0 2026-02-24T22:34:25.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14249536929999140218', time=2026-02-25T00:26:53.946265+0000, chain.objs.size()=1 2026-02-24T22:34:25.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.YvUh8kdp49AwJoyLvYE15woUax3n84i_0 2026-02-24T22:34:25.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14035880415539220927', time=2026-02-25T00:26:53.952006+0000, chain.objs.size()=1 2026-02-24T22:34:25.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ZNSagzPZsU67OmSnlYSAUa5V56p0ES3_0 2026-02-24T22:34:25.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9670683111836921755', time=2026-02-25T00:26:53.961820+0000, chain.objs.size()=1 2026-02-24T22:34:25.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.831+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.yA6iNek_F8G6hOvp38jB2IM7clJNG94_0 2026-02-24T22:34:25.822 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15325209392810501850', time=2026-02-25T00:26:54.026535+0000, chain.objs.size()=1 2026-02-24T22:34:25.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.clv0JJqifBgK5eHWZGv8F21_S2StwnD_0 2026-02-24T22:34:25.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8297336806465121506', time=2026-02-25T00:26:54.166227+0000, chain.objs.size()=1 2026-02-24T22:34:25.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.4gZvuJCF7dqv-8bJWuZ1kRLdNFDIbsz_0 2026-02-24T22:34:25.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1837192824602838069', time=2026-02-25T00:26:54.182660+0000, chain.objs.size()=1 2026-02-24T22:34:25.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.uSYtUz7hsceLZMr0uzz0EU3BPzGeVQb_0 2026-02-24T22:34:25.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4093421114006400251', time=2026-02-25T00:26:54.209597+0000, chain.objs.size()=1 2026-02-24T22:34:25.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.E8gO4Vn1kDC4TgX_CGdntZwOXOyoAGk_0 2026-02-24T22:34:25.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3168909236685536065', time=2026-02-25T00:26:54.326705+0000, chain.objs.size()=1 2026-02-24T22:34:25.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.PNUe1CvnxlT5KdrlX20q1EJKjV9whL-_0 2026-02-24T22:34:25.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7372872161269220602', time=2026-02-25T00:26:54.380161+0000, chain.objs.size()=1 2026-02-24T22:34:25.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.XHYZ3bR-FEIwCSszZTA5RdiKN5qhOr3_0 2026-02-24T22:34:25.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1750197534001470601', time=2026-02-25T00:26:54.457529+0000, chain.objs.size()=1 2026-02-24T22:34:25.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.YM6WeNCr2W-OgFLE8SGOojmugQ-RAyT_0 2026-02-24T22:34:25.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15668490290909198578', time=2026-02-25T00:26:54.502162+0000, chain.objs.size()=1 2026-02-24T22:34:25.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.C9MgvQ7Od6QpDOi6GwpXlAu-1nqGfGx_0 2026-02-24T22:34:25.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10286071556803931656', time=2026-02-25T00:26:54.507761+0000, chain.objs.size()=1 2026-02-24T22:34:25.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.AUbggtJ_mFRTus7VBFcgunVQ1jOvzsD_0 2026-02-24T22:34:25.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14343206407193783388', time=2026-02-25T00:26:54.529910+0000, chain.objs.size()=1 2026-02-24T22:34:25.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.OcaLI9vcjzersq0ZNbvv_MGi9Pxt-kS_0 2026-02-24T22:34:25.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3616609193934596328', time=2026-02-25T00:26:59.052820+0000, chain.objs.size()=1 2026-02-24T22:34:25.823 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1697__shadow_.4tWVu24QivRlFDM-VIjq0PbWZ6yGcNH_0 2026-02-24T22:34:25.824 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.267629123653369447', time=2026-02-25T00:27:04.104559+0000, chain.objs.size()=1 2026-02-24T22:34:25.824 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1706__shadow_.c4--G0drvKEzsOx8JTbCoIUxfkEddWS_0 2026-02-24T22:34:25.824 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16270197328319389029', time=2026-02-25T00:27:23.712903+0000, chain.objs.size()=1 2026-02-24T22:34:25.824 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1741__shadow_.E5lysbYmFegx-SzaLNCWKgE0as9av2t_0 2026-02-24T22:34:25.824 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16967321062177903810', time=2026-02-25T00:28:23.536944+0000, chain.objs.size()=1 2026-02-24T22:34:25.824 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1829__shadow_.xMwrsVvy3sHM82VCl9izm8N8Yqo9H_m_0 2026-02-24T22:34:25.824 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10408549037459700413', time=2026-02-25T00:28:23.696485+0000, chain.objs.size()=1 2026-02-24T22:34:25.824 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1829__shadow_.EswKvTVLlPIO9oDNY_-PoeI-Ccl1Tjl_0 2026-02-24T22:34:25.824 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6263449031576327864', time=2026-02-25T00:28:46.518873+0000, chain.objs.size()=6 2026-02-24T22:34:25.824 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.1 2026-02-24T22:34:25.824 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.2 2026-02-24T22:34:25.824 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.3 2026-02-24T22:34:25.824 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.4 2026-02-24T22:34:25.824 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.5 2026-02-24T22:34:25.824 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1873__multipart_srcmultipart.2~iZiSJYfVjsTBMqjNuZWcmyaKHolBiW1.6 2026-02-24T22:34:25.825 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1102657706542871331', time=2026-02-25T00:29:10.681586+0000, chain.objs.size()=2 2026-02-24T22:34:25.825 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1889__multipart_mymultipart.2~D4Twap_CUYs4PwmJtXY3JwGIWUygmeA.1 2026-02-24T22:34:25.825 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1889__multipart_mymultipart.2~D4Twap_CUYs4PwmJtXY3JwGIWUygmeA.2 2026-02-24T22:34:25.825 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.511476206661123855', time=2026-02-25T00:29:10.921575+0000, chain.objs.size()=3 2026-02-24T22:34:25.825 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1889__multipart_mymultipart.2~3-MajPamBRoNA_vAgSLNoiT9Kc5qYmq.1 2026-02-24T22:34:25.825 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1889__multipart_mymultipart.2~3-MajPamBRoNA_vAgSLNoiT9Kc5qYmq.2 2026-02-24T22:34:25.825 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1889__multipart_mymultipart.2~3-MajPamBRoNA_vAgSLNoiT9Kc5qYmq.3 2026-02-24T22:34:25.825 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='2~iW4jVxhjcUtGCGvLD8sBdTBn8VH7QWj', time=2026-02-25T00:29:16.960841+0000, chain.objs.size()=2 2026-02-24T22:34:25.825 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1895__multipart_mymultipart.2~iW4jVxhjcUtGCGvLD8sBdTBn8VH7QWj.1 2026-02-24T22:34:25.825 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1895__multipart_mymultipart.2~iW4jVxhjcUtGCGvLD8sBdTBn8VH7QWj.2 2026-02-24T22:34:25.825 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='2~B0zJ7cfdV9nHP3nWB3MAyKQdMdY0AHX', time=2026-02-25T00:29:31.568379+0000, chain.objs.size()=1 2026-02-24T22:34:25.825 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1902__multipart_mymultipart.2~B0zJ7cfdV9nHP3nWB3MAyKQdMdY0AHX.3 2026-02-24T22:34:25.825 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11406789705650134482', time=2026-02-25T00:30:06.478140+0000, chain.objs.size()=2 2026-02-24T22:34:25.825 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1923__shadow_.sEV8h7SgwnvTOkDGpBdvESKa43lc8Yj_0 2026-02-24T22:34:25.825 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1923__shadow_.sEV8h7SgwnvTOkDGpBdvESKa43lc8Yj_1 2026-02-24T22:34:25.826 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13506306191759339435', time=2026-02-25T00:30:57.011412+0000, chain.objs.size()=1 2026-02-24T22:34:25.826 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:nEQNIqhCuslnkEQ.QjxGdIZI9E51ESz_.bzR_VQJ3_eRtJYsRjqES02Q5OVZGh4L_0 2026-02-24T22:34:25.826 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13506306191759339435', time=2026-02-25T00:30:57.011763+0000, chain.objs.size()=1 2026-02-24T22:34:25.826 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:nEQNIqhCuslnkEQ.QjxGdIZI9E51ESz_.bzR_VQJ3_eRtJYsRjqES02Q5OVZGh4L_0 2026-02-24T22:34:25.826 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12324822522563883452', time=2026-02-25T00:30:57.177138+0000, chain.objs.size()=1 2026-02-24T22:34:25.826 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:UYxO0wWr.DZjHAWmuNnMK.YM-kuCT0K_.REyUYow5k1m1DVVWBBLNdisdoOJeMlY_0 2026-02-24T22:34:25.826 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12324822522563883452', time=2026-02-25T00:30:57.178102+0000, chain.objs.size()=1 2026-02-24T22:34:25.826 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.835+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:UYxO0wWr.DZjHAWmuNnMK.YM-kuCT0K_.REyUYow5k1m1DVVWBBLNdisdoOJeMlY_0 2026-02-24T22:34:25.827 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.839+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9464434808215165796', time=2026-02-25T00:30:58.433027+0000, chain.objs.size()=1 2026-02-24T22:34:25.827 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.839+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958__shadow:YCQHI4sXv6sCgezNAWlkvupK1uW6b0S_.GF4pfKKhW0g6_k-We95poYQJ7fWIxoJ_0 2026-02-24T22:34:25.828 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.839+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15409798434924153806', time=2026-02-25T00:30:58.445667+0000, chain.objs.size()=1 2026-02-24T22:34:25.828 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.839+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958__shadow:cQa-Olu8or8z1ueBPUqzLTsQgZ0e.2._._Q0hgiwcmDB3B2oDwvcWnvJI2fzJRwy_0 2026-02-24T22:34:25.828 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.839+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15409798434924153806', time=2026-02-25T00:30:58.446970+0000, chain.objs.size()=1 2026-02-24T22:34:25.828 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.839+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958__shadow:cQa-Olu8or8z1ueBPUqzLTsQgZ0e.2._._Q0hgiwcmDB3B2oDwvcWnvJI2fzJRwy_0 2026-02-24T22:34:25.828 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.839+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15409798434924153806', time=2026-02-25T00:30:58.447496+0000, chain.objs.size()=1 2026-02-24T22:34:25.828 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.839+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958__shadow:cQa-Olu8or8z1ueBPUqzLTsQgZ0e.2._._Q0hgiwcmDB3B2oDwvcWnvJI2fzJRwy_0 2026-02-24T22:34:25.828 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.839+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15409798434924153806', time=2026-02-25T00:30:58.448328+0000, chain.objs.size()=1 2026-02-24T22:34:25.828 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.839+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958__shadow:cQa-Olu8or8z1ueBPUqzLTsQgZ0e.2._._Q0hgiwcmDB3B2oDwvcWnvJI2fzJRwy_0 2026-02-24T22:34:25.828 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.839+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15409798434924153806', time=2026-02-25T00:30:58.451534+0000, chain.objs.size()=1 2026-02-24T22:34:25.828 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.839+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1958__shadow:cQa-Olu8or8z1ueBPUqzLTsQgZ0e.2._._Q0hgiwcmDB3B2oDwvcWnvJI2fzJRwy_0 2026-02-24T22:34:25.828 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.839+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1329941818640759251', time=2026-02-25T00:33:41.582059+0000, chain.objs.size()=1 2026-02-24T22:34:25.828 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.839+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1963__shadow_.shchfQMXYyGBy9w5gmgF00OC8jjVdqj_0 2026-02-24T22:34:25.830 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.832 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2026-02-24T22:34:25.833 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.833 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/38223' 2026-02-24T22:34:25.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15541942487287088756', time=2026-02-25T00:15:34.800746+0000, chain.objs.size()=2 2026-02-24T22:34:25.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1514__shadow_.PvSGPD7Ox7xQx-3vb1DSf09f7jCO-zh_0 2026-02-24T22:34:25.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1514__shadow_.PvSGPD7Ox7xQx-3vb1DSf09f7jCO-zh_1 2026-02-24T22:34:25.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5322043341573308463', time=2026-02-25T00:17:05.165099+0000, chain.objs.size()=1 2026-02-24T22:34:25.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522__shadow_.5f8afJ3zAxFaJTQ169W7brltGtRoFZ5_0 2026-02-24T22:34:25.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6412854495011510722', time=2026-02-25T00:17:07.121050+0000, chain.objs.size()=1 2026-02-24T22:34:25.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523__shadow_.9HdaQ2oApb8BoavRN0ZhA0NobtZos-m_0 2026-02-24T22:34:25.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18401940724511115361', time=2026-02-25T00:18:49.770601+0000, chain.objs.size()=1 2026-02-24T22:34:25.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531__shadow_.9tf4qm8yg4KU8juLBtNn6dwwAUfXzpP_0 2026-02-24T22:34:25.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3366773056435623591', time=2026-02-25T00:18:51.085846+0000, chain.objs.size()=1 2026-02-24T22:34:25.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532__shadow_.R5_kv_9aanXdhhjvcAbLJ5yQDCIAvIy_0 2026-02-24T22:34:25.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7147732109328741424', time=2026-02-25T00:18:51.086310+0000, chain.objs.size()=1 2026-02-24T22:34:25.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532__shadow_.TI88qK1qf0eUjsAvyn3WqJ-gkniFS38_0 2026-02-24T22:34:25.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5442408716878409118', time=2026-02-25T00:19:25.020218+0000, chain.objs.size()=1 2026-02-24T22:34:25.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534__shadow_.pnBAXV4AIvTM18tJJgioPmQ2Kg5VKWw_0 2026-02-24T22:34:25.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8552656096741045773', time=2026-02-25T00:19:58.453140+0000, chain.objs.size()=1 2026-02-24T22:34:25.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536__shadow_.OVHt-yXxsXVbzBHzJy71s6Hy6SSzOLz_0 2026-02-24T22:34:25.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5003986543559503022', time=2026-02-25T00:20:32.508919+0000, chain.objs.size()=1 2026-02-24T22:34:25.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538__shadow_.vwpH1xSI3voWAY0wg9dRFoH-p_YDQRf_0 2026-02-24T22:34:25.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17141699999510076525', time=2026-02-25T00:20:34.027808+0000, chain.objs.size()=1 2026-02-24T22:34:25.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539__shadow_.mWCRlstxNUwrHMBrmUJBnkh-_hLipPd_0 2026-02-24T22:34:25.834 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13202536733587720983', time=2026-02-25T00:25:34.875101+0000, chain.objs.size()=1 2026-02-24T22:34:25.835 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.843+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1599__shadow_.i5qtrfRbQ-pDGPOmpqLIXWZeqhWiDIX_0 2026-02-24T22:34:25.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10418849519088135829', time=2026-02-25T00:25:47.314525+0000, chain.objs.size()=1 2026-02-24T22:34:25.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1618__shadow_.likCvxOMjrL-PpcgaCeRrqnq3I7quqm_0 2026-02-24T22:34:25.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8485161549079415041', time=2026-02-25T00:25:59.071538+0000, chain.objs.size()=1 2026-02-24T22:34:25.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.c5VFJd5wjXGt374Y7JH645cCr6JzbEg_0 2026-02-24T22:34:25.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16839841891904261570', time=2026-02-25T00:25:59.078243+0000, chain.objs.size()=1 2026-02-24T22:34:25.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.JMDn4-mN4RutTavv_HBAuexBIaqfXxF_0 2026-02-24T22:34:25.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13928894526407251546', time=2026-02-25T00:25:59.083222+0000, chain.objs.size()=1 2026-02-24T22:34:25.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.QKdBXx1E_57As0nTTfYj7VTaNyAGaES_0 2026-02-24T22:34:25.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13235674304122390148', time=2026-02-25T00:25:59.085541+0000, chain.objs.size()=1 2026-02-24T22:34:25.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.LNBNqaPBODTUXHRei1q3tl2aWNdyCED_0 2026-02-24T22:34:25.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4959744694861792841', time=2026-02-25T00:25:59.105204+0000, chain.objs.size()=1 2026-02-24T22:34:25.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.TpQgVOzTCWf1i1IF-HJsP2cuCmGW6-4_0 2026-02-24T22:34:25.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17259994623880732499', time=2026-02-25T00:25:59.119911+0000, chain.objs.size()=1 2026-02-24T22:34:25.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.D_J7UmI1TJxcsPQo5F9gxFaevk867nu_0 2026-02-24T22:34:25.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12672649002659207381', time=2026-02-25T00:25:59.133412+0000, chain.objs.size()=1 2026-02-24T22:34:25.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.muKK2bkq75TaViABB5nehStQREm1jQF_0 2026-02-24T22:34:25.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12876047070664541472', time=2026-02-25T00:25:59.134224+0000, chain.objs.size()=1 2026-02-24T22:34:25.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.PK4yYhkYNPLRckPbmcP1X7KhYMnqhpG_0 2026-02-24T22:34:25.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16337331098052399408', time=2026-02-25T00:25:59.146109+0000, chain.objs.size()=1 2026-02-24T22:34:25.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.rlyN6TSNJpycOJzAWl94eVoEeqUlgkL_0 2026-02-24T22:34:25.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2692995811475830489', time=2026-02-25T00:25:59.174095+0000, chain.objs.size()=1 2026-02-24T22:34:25.836 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.E3yZb2f4fGPGbwUXNe8m-2iohn6BHUa_0 2026-02-24T22:34:25.837 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12139578278771562302', time=2026-02-25T00:25:59.245836+0000, chain.objs.size()=1 2026-02-24T22:34:25.837 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.4TnzNUv6E0ZLhse8JBdeU-LUUo3FPr5_0 2026-02-24T22:34:25.837 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4886280680060575593', time=2026-02-25T00:25:59.253372+0000, chain.objs.size()=1 2026-02-24T22:34:25.837 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.p-AhExVUVllZQgf2U8hBj6NLCG9otGV_0 2026-02-24T22:34:25.837 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17447150355678886138', time=2026-02-25T00:25:59.296700+0000, chain.objs.size()=1 2026-02-24T22:34:25.837 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.rMkfhNFT9cn4InAYtgUVgA1h1ck5WHy_0 2026-02-24T22:34:25.837 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.138137236497541204', time=2026-02-25T00:25:59.298748+0000, chain.objs.size()=1 2026-02-24T22:34:25.837 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Rjg6rRBDvBvU1yuilrc92n7Dl1t52ta_0 2026-02-24T22:34:25.837 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6152227788363188468', time=2026-02-25T00:25:59.314188+0000, chain.objs.size()=1 2026-02-24T22:34:25.837 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.kIMmpQeMQtQmDxavhFPr-pEYHZmxVYb_0 2026-02-24T22:34:25.837 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15851706503993586576', time=2026-02-25T00:25:59.318490+0000, chain.objs.size()=1 2026-02-24T22:34:25.837 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.zr8BiXr5Myq5xQo8VE-CnNjnae1OzZ8_0 2026-02-24T22:34:25.837 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2198416381047503795', time=2026-02-25T00:25:59.322382+0000, chain.objs.size()=1 2026-02-24T22:34:25.837 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.TMEWxAf2lIfZKvqBvEakzFsj6Y5lDYf_0 2026-02-24T22:34:25.837 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7621779927879809662', time=2026-02-25T00:25:59.323654+0000, chain.objs.size()=1 2026-02-24T22:34:25.837 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.3Jr_BZDyze-oRlevuN6CD0F03NxXaOP_0 2026-02-24T22:34:25.837 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13331499485221898226', time=2026-02-25T00:25:59.331431+0000, chain.objs.size()=1 2026-02-24T22:34:25.838 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.VUpw10jAEKMrS5jQyI_ZL4N2INjQSiz_0 2026-02-24T22:34:25.838 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6726482805638389240', time=2026-02-25T00:25:59.407120+0000, chain.objs.size()=1 2026-02-24T22:34:25.838 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.02ax8qCa8lATJ6a2aVnvb8OdxxBCUNu_0 2026-02-24T22:34:25.838 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7487217888791649490', time=2026-02-25T00:25:59.422520+0000, chain.objs.size()=1 2026-02-24T22:34:25.838 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.HGQgoVhcdmxbb6DJqmBINasihP5fk2E_0 2026-02-24T22:34:25.838 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14992165689501907434', time=2026-02-25T00:25:59.430172+0000, chain.objs.size()=1 2026-02-24T22:34:25.838 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.bqPEcP6WvJz--L1tFiKZiKp_QBsDG5C_0 2026-02-24T22:34:25.838 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12739743960616530691', time=2026-02-25T00:25:59.445046+0000, chain.objs.size()=1 2026-02-24T22:34:25.838 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.4fIyfM9e_V4itF76TTOL9xJcrM6nsBz_0 2026-02-24T22:34:25.838 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3030726073743323809', time=2026-02-25T00:25:59.466164+0000, chain.objs.size()=1 2026-02-24T22:34:25.838 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.847+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.cYDuR0ciaEddekIUKfsmPbErBPyOujm_0 2026-02-24T22:34:25.838 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7269018105193090879', time=2026-02-25T00:25:59.491843+0000, chain.objs.size()=1 2026-02-24T22:34:25.838 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.-nhlbnL6sY9H5d8y89P3g1Vsru6_y9s_0 2026-02-24T22:34:25.838 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14915848484123644990', time=2026-02-25T00:25:59.603963+0000, chain.objs.size()=1 2026-02-24T22:34:25.838 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.cBCu7jQu9GX6IMDpkeMHiJaLBarURSW_0 2026-02-24T22:34:25.838 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8202541399077887312', time=2026-02-25T00:25:59.620031+0000, chain.objs.size()=1 2026-02-24T22:34:25.838 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Eha8-TpeVGJrTgdZnQFhP0iqEZ9WTuW_0 2026-02-24T22:34:25.839 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15951542698391941571', time=2026-02-25T00:25:59.625200+0000, chain.objs.size()=1 2026-02-24T22:34:25.839 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.zVGUScT67-EUC33fr-RxGpC1HKhG91m_0 2026-02-24T22:34:25.839 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17198936124339643846', time=2026-02-25T00:25:59.740319+0000, chain.objs.size()=1 2026-02-24T22:34:25.839 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.pz39OtDyoNATiLQ-Bn-9iAT4lnLFNlv_0 2026-02-24T22:34:25.839 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15715574564764183066', time=2026-02-25T00:25:59.809871+0000, chain.objs.size()=1 2026-02-24T22:34:25.839 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.MomCku3hyg4HkLBGXhc55z1aFRkHFnA_0 2026-02-24T22:34:25.839 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4598331490504588619', time=2026-02-25T00:25:59.916378+0000, chain.objs.size()=1 2026-02-24T22:34:25.839 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.WsqGQNEd_7WArBF86DdWHwccIC6fiuZ_0 2026-02-24T22:34:25.839 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15115309410698231717', time=2026-02-25T00:25:59.943628+0000, chain.objs.size()=1 2026-02-24T22:34:25.839 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Y-NEyqRAPeMUIX-7WQ4NMwpC5Qv-UME_0 2026-02-24T22:34:25.839 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10344342053144259540', time=2026-02-25T00:25:59.944071+0000, chain.objs.size()=1 2026-02-24T22:34:25.839 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.odVPJny6oRpoAB7hTqVQvCyCQemqu_-_0 2026-02-24T22:34:25.839 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15297062682048938495', time=2026-02-25T00:26:00.045714+0000, chain.objs.size()=1 2026-02-24T22:34:25.839 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.3dDnn29BRn01CzVtzH46yf-crEMZG3a_0 2026-02-24T22:34:25.839 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16295753762689400684', time=2026-02-25T00:26:00.055477+0000, chain.objs.size()=1 2026-02-24T22:34:25.839 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.nBdbsoLtR8hNHPXrbFASbVI-2wEXvmY_0 2026-02-24T22:34:25.840 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7149089396187635497', time=2026-02-25T00:26:00.117216+0000, chain.objs.size()=1 2026-02-24T22:34:25.840 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.BxhBtLpKX5lD_Bn8j9K2EQXZLFk1lQy_0 2026-02-24T22:34:25.840 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15345314148170201143', time=2026-02-25T00:26:00.228909+0000, chain.objs.size()=1 2026-02-24T22:34:25.840 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Nk76j3ANuIsj5imAujlGciT46Z-1uUU_0 2026-02-24T22:34:25.840 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14690730809137886072', time=2026-02-25T00:26:00.273397+0000, chain.objs.size()=1 2026-02-24T22:34:25.840 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.wNTeTTQ-Nu97UqNrMceUFh_JyhAdmRr_0 2026-02-24T22:34:25.840 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9963841400899533857', time=2026-02-25T00:26:00.278724+0000, chain.objs.size()=1 2026-02-24T22:34:25.840 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.0zMRChjCZTIqUjstnKrA5nsXXWwwhh1_0 2026-02-24T22:34:25.840 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5969626387478524358', time=2026-02-25T00:26:00.280998+0000, chain.objs.size()=1 2026-02-24T22:34:25.840 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.YSD3_HMbMSwAIP6N-mV5NJln9Ghbdsr_0 2026-02-24T22:34:25.840 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1880114856029558121', time=2026-02-25T00:26:07.678363+0000, chain.objs.size()=1 2026-02-24T22:34:25.840 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1641__shadow_.OKDb675fh4M5bfgnrxt7UBS8Z4qM5dw_0 2026-02-24T22:34:25.840 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6856075056227888438', time=2026-02-25T00:26:08.833989+0000, chain.objs.size()=1 2026-02-24T22:34:25.840 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1643__shadow_.2DMTaVddrbdtBFIvP8t3MHY5J7aPBcs_0 2026-02-24T22:34:25.840 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17481167456499136401', time=2026-02-25T00:26:10.022823+0000, chain.objs.size()=1 2026-02-24T22:34:25.840 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1645__shadow_.ldFJZ3LVOXIzcKqYFJRfz82M3m4r5oK_0 2026-02-24T22:34:25.840 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2314101934102643413', time=2026-02-25T00:26:28.072593+0000, chain.objs.size()=1 2026-02-24T22:34:25.840 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1673__shadow:0XTTEXotzjsaSKmoK8q5YBWOtG2L8YN_.arGShsn_8jqQsJr-550sia-8nnM7rq1_0 2026-02-24T22:34:25.840 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7442380396483258899', time=2026-02-25T00:26:36.501473+0000, chain.objs.size()=1 2026-02-24T22:34:25.840 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1685__shadow_.24CqaY8cqkoXEyUjPw3ROkxtxgR_qZB_0 2026-02-24T22:34:25.840 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4592569789738845753', time=2026-02-25T00:26:44.776325+0000, chain.objs.size()=1 2026-02-24T22:34:25.841 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.lBAwdwXjAVn0Qp43qYH6ZPIe341z-p6_0 2026-02-24T22:34:25.841 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1096208047564474944', time=2026-02-25T00:26:44.777033+0000, chain.objs.size()=1 2026-02-24T22:34:25.841 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.dDkmFIAeueTcnGx_73BQpqoc0XIVAWO_0 2026-02-24T22:34:25.841 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12294464025923108288', time=2026-02-25T00:26:44.788541+0000, chain.objs.size()=1 2026-02-24T22:34:25.841 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.pNVXX2SMyL5nt6ZSrgG8Z5E7foUWcZ7_0 2026-02-24T22:34:25.841 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14769950528883581410', time=2026-02-25T00:26:44.797671+0000, chain.objs.size()=1 2026-02-24T22:34:25.841 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Ms6URNDeOeqJALBJNSc3VB2UiI2gQ4J_0 2026-02-24T22:34:25.841 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12416529270244385100', time=2026-02-25T00:26:44.814259+0000, chain.objs.size()=1 2026-02-24T22:34:25.841 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.A1NlkIT8dBw7XkuiHjZJZrLlULB8hGx_0 2026-02-24T22:34:25.841 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4095408204765680605', time=2026-02-25T00:26:44.822591+0000, chain.objs.size()=1 2026-02-24T22:34:25.841 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.MSrsQoJ3Jqk2gm-xmQJtSRG4KaX_s2o_0 2026-02-24T22:34:25.841 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3953602577929099981', time=2026-02-25T00:26:44.938084+0000, chain.objs.size()=1 2026-02-24T22:34:25.841 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.9vGush9NYYNmPSWCAvXevNPABoHC5-x_0 2026-02-24T22:34:25.841 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5070860126432612021', time=2026-02-25T00:26:44.939297+0000, chain.objs.size()=1 2026-02-24T22:34:25.841 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.iAZI1lt4Nr7Ky7yprIPY8tR_Kgnf89D_0 2026-02-24T22:34:25.841 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2806423873399617099', time=2026-02-25T00:26:44.984350+0000, chain.objs.size()=1 2026-02-24T22:34:25.841 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.JD2m2jiqdIOrD5bc_lU4yOoUzJFe5Oi_0 2026-02-24T22:34:25.841 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3044831348497088652', time=2026-02-25T00:26:44.993792+0000, chain.objs.size()=1 2026-02-24T22:34:25.841 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.6iO7OjBdmXDtHiRFZX5Vgkw2QYVClB9_0 2026-02-24T22:34:25.841 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4673693515598795329', time=2026-02-25T00:26:45.003042+0000, chain.objs.size()=1 2026-02-24T22:34:25.841 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.BAb-s7DxrHSNK18V2zZyhkgqd0BTZ8e_0 2026-02-24T22:34:25.841 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7720411705239017389', time=2026-02-25T00:26:45.119254+0000, chain.objs.size()=1 2026-02-24T22:34:25.842 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.XqJXiocTOB8t62mc6BhuLv7C2ejT8f7_0 2026-02-24T22:34:25.842 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3008392867900914233', time=2026-02-25T00:26:45.180958+0000, chain.objs.size()=1 2026-02-24T22:34:25.842 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.YMusm2aytfzH_BCWXmJr6ld1dUMIGKx_0 2026-02-24T22:34:25.842 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13274432098092941733', time=2026-02-25T00:26:45.249609+0000, chain.objs.size()=1 2026-02-24T22:34:25.842 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_._UCKPzloU8bEhHJs1GCHLy2PwvTUR_K_0 2026-02-24T22:34:25.842 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12922964769351871303', time=2026-02-25T00:26:45.289434+0000, chain.objs.size()=1 2026-02-24T22:34:25.842 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.KcQGgoSBFMXt4C3Q1kA1FyRrmvLeQfN_0 2026-02-24T22:34:25.842 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.734426632779582537', time=2026-02-25T00:26:45.300970+0000, chain.objs.size()=1 2026-02-24T22:34:25.842 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.WGb9cY0V5Ks9n5fp-TyCOcO0xJwfkx7_0 2026-02-24T22:34:25.842 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.851+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7532643864944226995', time=2026-02-25T00:26:45.315191+0000, chain.objs.size()=1 2026-02-24T22:34:25.842 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.eQQaLXJ99Eh4ITeCUHHwXRrotrdFRVs_0 2026-02-24T22:34:25.842 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.453611057473834692', time=2026-02-25T00:26:45.326238+0000, chain.objs.size()=1 2026-02-24T22:34:25.842 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.FSA2j7Ltp3K4ZeRiMFf_U109WCt7gcM_0 2026-02-24T22:34:25.842 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13529970996060224298', time=2026-02-25T00:26:45.490222+0000, chain.objs.size()=1 2026-02-24T22:34:25.842 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.rwHO6yYFZpl5-4whOJhI_1LfQJWpqxX_0 2026-02-24T22:34:25.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1679614848475734208', time=2026-02-25T00:26:45.570802+0000, chain.objs.size()=1 2026-02-24T22:34:25.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.oDol9sXsKBHQ0EeXNwz_ecQPJLO1EMK_0 2026-02-24T22:34:25.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7497055627022455660', time=2026-02-25T00:26:45.839567+0000, chain.objs.size()=1 2026-02-24T22:34:25.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.OQRBCzDDipGeo3SQ2w6H3lgjQlVEd1u_0 2026-02-24T22:34:25.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6885920630975432752', time=2026-02-25T00:26:45.927961+0000, chain.objs.size()=1 2026-02-24T22:34:25.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.tkCHgVmvcLzQeerrdHm8k8J4GGZ2A3W_0 2026-02-24T22:34:25.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14304550183552174150', time=2026-02-25T00:26:46.102406+0000, chain.objs.size()=1 2026-02-24T22:34:25.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.2qOd5_lwrN_pqQiDpUdJ3hTzsSSS2uL_0 2026-02-24T22:34:25.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5448366196670466988', time=2026-02-25T00:26:46.117649+0000, chain.objs.size()=1 2026-02-24T22:34:25.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.NxzUtaHrQvbX7CXSkGduLJ_EKpY7wZ9_0 2026-02-24T22:34:25.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4382523698138238053', time=2026-02-25T00:26:53.182997+0000, chain.objs.size()=1 2026-02-24T22:34:25.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.NxWOvkJShJOLp3UYdJQ5XNRuKJGERjb_0 2026-02-24T22:34:25.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14141721405607114741', time=2026-02-25T00:26:53.204002+0000, chain.objs.size()=1 2026-02-24T22:34:25.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.OLKNI2_U33kUNVeZBpyFgtN--KJSU7q_0 2026-02-24T22:34:25.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2142506378550273417', time=2026-02-25T00:26:53.237879+0000, chain.objs.size()=1 2026-02-24T22:34:25.843 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.iScQHak8MQJpMhenqygtawgCk4vUcd7_0 2026-02-24T22:34:25.844 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11182840496398422353', time=2026-02-25T00:26:53.324911+0000, chain.objs.size()=1 2026-02-24T22:34:25.844 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Ts79dVZ-d-mXCFx7vCD6wXWxp3rQ-Dt_0 2026-02-24T22:34:25.844 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12603400932830082740', time=2026-02-25T00:26:53.360047+0000, chain.objs.size()=1 2026-02-24T22:34:25.844 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.TyXC5Ic5S388wfJc_HEyRUrWE-n0upP_0 2026-02-24T22:34:25.844 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2911244517282208893', time=2026-02-25T00:26:53.409136+0000, chain.objs.size()=1 2026-02-24T22:34:25.844 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.YJiLZplxXcyV-XczYZo_YMZ9md3EIGz_0 2026-02-24T22:34:25.844 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3999395179654031358', time=2026-02-25T00:26:53.434370+0000, chain.objs.size()=1 2026-02-24T22:34:25.844 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.zScQ6cmo52-LxPT2njWqV9_BCyqLWFK_0 2026-02-24T22:34:25.844 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7518729919595821734', time=2026-02-25T00:26:53.515342+0000, chain.objs.size()=1 2026-02-24T22:34:25.844 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.aniHvRNpYmosgFp6wSqi7h6T1VLZOpE_0 2026-02-24T22:34:25.844 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7176437556758216737', time=2026-02-25T00:26:53.540414+0000, chain.objs.size()=1 2026-02-24T22:34:25.844 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.DJDKb180m26SiiUSJ9yYurUfEcpo4Jq_0 2026-02-24T22:34:25.844 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15611229609411968008', time=2026-02-25T00:26:53.592984+0000, chain.objs.size()=1 2026-02-24T22:34:25.844 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.r_OsMj0awUh84JGRteZts7Mxm3nvxV1_0 2026-02-24T22:34:25.844 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9842327121442012790', time=2026-02-25T00:26:53.593700+0000, chain.objs.size()=1 2026-02-24T22:34:25.844 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.6nyBQ7WRaAIrgEmb6lLrltUM0i2EFHd_0 2026-02-24T22:34:25.844 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.414323238916427759', time=2026-02-25T00:26:53.733219+0000, chain.objs.size()=1 2026-02-24T22:34:25.844 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.N9JDFYRa_JAj0zf3ngcGiHQ6KI_7CHx_0 2026-02-24T22:34:25.844 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16419522823132686267', time=2026-02-25T00:26:53.735679+0000, chain.objs.size()=1 2026-02-24T22:34:25.844 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.USRGDHXeI0IkmODqnQUlxcZOwr2mgxk_0 2026-02-24T22:34:25.844 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17797847075357351630', time=2026-02-25T00:26:53.738510+0000, chain.objs.size()=1 2026-02-24T22:34:25.844 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.jEIngZ9WGqoNXilLUQllc-vqwjdCL-Y_0 2026-02-24T22:34:25.844 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17406534202863709284', time=2026-02-25T00:26:53.763101+0000, chain.objs.size()=1 2026-02-24T22:34:25.845 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.HvMqErCv96eEGqsV8R7RqKV3a375k4I_0 2026-02-24T22:34:25.845 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1405088342064490604', time=2026-02-25T00:26:53.783010+0000, chain.objs.size()=1 2026-02-24T22:34:25.845 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.aQmHk0UTCT6f3k1gJc7NMBjIFu8_zD0_0 2026-02-24T22:34:25.845 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13487003553588908302', time=2026-02-25T00:26:53.785649+0000, chain.objs.size()=1 2026-02-24T22:34:25.845 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.p8ZHc5KynsbVpoRm2FRB2kmi--clHjg_0 2026-02-24T22:34:25.845 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10029571740431416552', time=2026-02-25T00:26:53.790387+0000, chain.objs.size()=1 2026-02-24T22:34:25.845 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.gvag-BpzTuFWxDfkRTJSDwJp6JYsEzl_0 2026-02-24T22:34:25.845 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16099287699444244016', time=2026-02-25T00:26:53.822866+0000, chain.objs.size()=1 2026-02-24T22:34:25.845 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.e9KPCqGfPzPBtDGevDFIjSR8rzN4Rh7_0 2026-02-24T22:34:25.845 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.855+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/38223 2026-02-24T22:34:25.847 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=31, truncated=0, next_marker='' 2026-02-24T22:34:25.847 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7084526794387162830', time=2026-02-25T00:26:53.913860+0000, chain.objs.size()=1 2026-02-24T22:34:25.847 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.UPNhxLZqDVR29JIWA1SXUwVzA4_pfm9_0 2026-02-24T22:34:25.847 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3763773062036779284', time=2026-02-25T00:26:53.963878+0000, chain.objs.size()=1 2026-02-24T22:34:25.847 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ldS518tNB2e8cJSNh-9aE_8O72KwzZc_0 2026-02-24T22:34:25.847 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12331511228407721887', time=2026-02-25T00:26:53.990474+0000, chain.objs.size()=1 2026-02-24T22:34:25.847 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ZUrBWfbJ65VQOu9paquwdzLJUA8Za7I_0 2026-02-24T22:34:25.847 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17772254500702199632', time=2026-02-25T00:26:54.021270+0000, chain.objs.size()=1 2026-02-24T22:34:25.847 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.4rcygo4w98du1LuWyOD6HccQF_htm_w_0 2026-02-24T22:34:25.847 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9172113015676540318', time=2026-02-25T00:26:54.124742+0000, chain.objs.size()=1 2026-02-24T22:34:25.847 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.xb7CqYKh9CAffO6j86TktYrF5TBnGJF_0 2026-02-24T22:34:25.847 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5329931765725303002', time=2026-02-25T00:26:54.154944+0000, chain.objs.size()=1 2026-02-24T22:34:25.847 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.nPSukaezU2yFGhdhfQ1Ht6yUPGPQZXH_0 2026-02-24T22:34:25.848 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5637371613558899423', time=2026-02-25T00:26:54.203193+0000, chain.objs.size()=1 2026-02-24T22:34:25.848 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.0umZr1Ymlo8IZF9lUvkR-Szf8csgSOt_0 2026-02-24T22:34:25.848 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10766556166510698711', time=2026-02-25T00:26:54.282377+0000, chain.objs.size()=1 2026-02-24T22:34:25.848 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ghWz_62WAsnQuLTXXXSUY6mngarRLrS_0 2026-02-24T22:34:25.848 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11281510735490743310', time=2026-02-25T00:26:54.325602+0000, chain.objs.size()=1 2026-02-24T22:34:25.848 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.rZ4cBDpKrQnFtRegxuUbN1nwa9oROHi_0 2026-02-24T22:34:25.848 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9393898083494124509', time=2026-02-25T00:26:54.343862+0000, chain.objs.size()=1 2026-02-24T22:34:25.848 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ED_hHoiD3pmMMjfkjwQjSPOoRCiWk3__0 2026-02-24T22:34:25.848 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9016887160288234104', time=2026-02-25T00:26:54.353866+0000, chain.objs.size()=1 2026-02-24T22:34:25.848 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.nOx1-Dq2DAssu7mot_0EC_7zxSsDDbR_0 2026-02-24T22:34:25.848 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3883939565914912615', time=2026-02-25T00:26:54.364158+0000, chain.objs.size()=1 2026-02-24T22:34:25.848 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.guN5EqUL9u9pNXt6LG7leifdLNFJuQs_0 2026-02-24T22:34:25.848 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11991861324075176037', time=2026-02-25T00:27:05.824347+0000, chain.objs.size()=1 2026-02-24T22:34:25.848 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1709__shadow_._ryXSjnbfZ0ZAVGdqJmWwPKgscDjVEv_0 2026-02-24T22:34:25.848 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7002125715930180748', time=2026-02-25T00:27:20.924336+0000, chain.objs.size()=1 2026-02-24T22:34:25.848 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1738__shadow_.CMri_rDe4I1Mm8Xt3UwOK6Rn7GMPGKA_0 2026-02-24T22:34:25.849 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1721058385970492414', time=2026-02-25T00:28:15.292680+0000, chain.objs.size()=1 2026-02-24T22:34:25.849 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1819__shadow_.RkajS78g1S5yyEJ6ExWwtFssuwb41FS_0 2026-02-24T22:34:25.849 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13449384370801105900', time=2026-02-25T00:28:32.542045+0000, chain.objs.size()=1 2026-02-24T22:34:25.849 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1849__shadow_.MNmpkGYViGIUj-CV8XgSk1BVPZKgj_O_0 2026-02-24T22:34:25.849 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10568423080268777715', time=2026-02-25T00:29:05.418968+0000, chain.objs.size()=6 2026-02-24T22:34:25.849 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~sJYAJxfnctZsGBcD-kKtQjlHjtrN__F.1 2026-02-24T22:34:25.849 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~sJYAJxfnctZsGBcD-kKtQjlHjtrN__F.2 2026-02-24T22:34:25.849 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.OcI8ePLHSFPFzTpiEdN-6UeRAsgo-1_U.3 2026-02-24T22:34:25.849 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~sJYAJxfnctZsGBcD-kKtQjlHjtrN__F.4 2026-02-24T22:34:25.849 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~sJYAJxfnctZsGBcD-kKtQjlHjtrN__F.5 2026-02-24T22:34:25.849 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~sJYAJxfnctZsGBcD-kKtQjlHjtrN__F.6 2026-02-24T22:34:25.849 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4624760518733734989', time=2026-02-25T00:29:07.814485+0000, chain.objs.size()=6 2026-02-24T22:34:25.849 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~pj_3WTBboDa_0XCojflIVBc95wtcFhk.1 2026-02-24T22:34:25.849 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.kKt2O4Jp70PyJD0-4ySVjSeO_DGiETOy.2 2026-02-24T22:34:25.849 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.UHFIz5ynaCDNyqqmP7TxNdGAjrD-4Y0J.3 2026-02-24T22:34:25.850 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~pj_3WTBboDa_0XCojflIVBc95wtcFhk.4 2026-02-24T22:34:25.850 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~pj_3WTBboDa_0XCojflIVBc95wtcFhk.5 2026-02-24T22:34:25.850 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.859+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1888__multipart_mymultipart.2~pj_3WTBboDa_0XCojflIVBc95wtcFhk.6 2026-02-24T22:34:25.850 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='2~Nyl4dcXMulS4USw5WgpjdbL1JlxrPVA', time=2026-02-25T00:29:18.526659+0000, chain.objs.size()=2 2026-02-24T22:34:25.850 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1897__multipart_mymultipart.2~Nyl4dcXMulS4USw5WgpjdbL1JlxrPVA.1 2026-02-24T22:34:25.850 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1897__multipart_mymultipart.2~Nyl4dcXMulS4USw5WgpjdbL1JlxrPVA.2 2026-02-24T22:34:25.850 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6160003634098918582', time=2026-02-25T00:30:18.395312+0000, chain.objs.size()=1 2026-02-24T22:34:25.850 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:S32.umgwsxTOyy419wBEbis0qcdrr-g_.SZc6caa0uJcwOGp3UmAhUbflNzyLYUf_0 2026-02-24T22:34:25.850 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9267929303976689330', time=2026-02-25T00:30:18.870791+0000, chain.objs.size()=1 2026-02-24T22:34:25.850 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:hybR55.2FDhnetDnTllxU-B7Jb4758J_.IS-ZYqEfGOTCklaUmxJ9BdUGXI8Kogh_0 2026-02-24T22:34:25.850 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15365547195206945850', time=2026-02-25T00:30:49.582122+0000, chain.objs.size()=4 2026-02-24T22:34:25.850 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1948__multipart_testobj.2~dGtIclLmI9UuSHKAhbOlvfXbrD-2XGb.1 2026-02-24T22:34:25.850 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1948__multipart_testobj.2~dGtIclLmI9UuSHKAhbOlvfXbrD-2XGb.2 2026-02-24T22:34:25.850 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1948__multipart_testobj.2~dGtIclLmI9UuSHKAhbOlvfXbrD-2XGb.3 2026-02-24T22:34:25.850 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1948__multipart_testobj.2~dGtIclLmI9UuSHKAhbOlvfXbrD-2XGb.4 2026-02-24T22:34:25.852 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8724934121494928228', time=2026-02-25T00:30:50.995840+0000, chain.objs.size()=1 2026-02-24T22:34:25.852 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1949__shadow:PrVzKDgMGiOI.egQnKByrgH4eCUoeE._.I0wZh4ECUSOVy6hDoQzxYDVzGkqEe-q_0 2026-02-24T22:34:25.852 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1708457319319113639', time=2026-02-25T00:30:52.473761+0000, chain.objs.size()=1 2026-02-24T22:34:25.852 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1950__shadow:3qosJlIk2xJ72kWDV6fsMvIm8nsCIqd_.aEMiPEY5YHs3lv1MLBaT1QummQJqwDE_0 2026-02-24T22:34:25.852 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11063557205769147015', time=2026-02-25T00:30:55.587470+0000, chain.objs.size()=1 2026-02-24T22:34:25.852 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1955__shadow:6njJdD805nSfVBO62CCQcMRIk7DadLQ_.cybJrf1IPHD7JZINeMTeH7-mZp1sGuF_0 2026-02-24T22:34:25.852 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3597324925483040157', time=2026-02-25T00:30:57.005090+0000, chain.objs.size()=1 2026-02-24T22:34:25.852 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:vKM5oE9iH9Tfrm9mS1mY52qOdQAgxU4_.Pm5HiVobm6dgvRWR0rvaOmdKrIOnWQd_0 2026-02-24T22:34:25.852 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3597324925483040157', time=2026-02-25T00:30:57.005738+0000, chain.objs.size()=1 2026-02-24T22:34:25.852 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:vKM5oE9iH9Tfrm9mS1mY52qOdQAgxU4_.Pm5HiVobm6dgvRWR0rvaOmdKrIOnWQd_0 2026-02-24T22:34:25.852 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3459680576847003766', time=2026-02-25T00:30:57.720431+0000, chain.objs.size()=1 2026-02-24T22:34:25.852 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:GgqeW5JWsDSE6m2HBeoSN8MlXPWtfBL_.FJ31DpADCIilzAnaSt-OtfMW8KkraCo_0 2026-02-24T22:34:25.852 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3459680576847003766', time=2026-02-25T00:30:57.721600+0000, chain.objs.size()=1 2026-02-24T22:34:25.852 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:GgqeW5JWsDSE6m2HBeoSN8MlXPWtfBL_.FJ31DpADCIilzAnaSt-OtfMW8KkraCo_0 2026-02-24T22:34:25.852 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3459680576847003766', time=2026-02-25T00:30:57.721970+0000, chain.objs.size()=1 2026-02-24T22:34:25.852 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:GgqeW5JWsDSE6m2HBeoSN8MlXPWtfBL_.FJ31DpADCIilzAnaSt-OtfMW8KkraCo_0 2026-02-24T22:34:25.852 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3615002331015539148', time=2026-02-25T00:34:02.190578+0000, chain.objs.size()=1 2026-02-24T22:34:25.852 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1985__shadow_.ZZcpaAz73pnTpm2Fh7QWOrJWETszhnF_0 2026-02-24T22:34:25.853 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.863+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.855 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2026-02-24T22:34:25.855 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.856 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/38761' 2026-02-24T22:34:25.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6571258558727922323', time=2026-02-25T00:15:34.734251+0000, chain.objs.size()=1 2026-02-24T22:34:25.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516__shadow_.xxehxqX99AfpMEclRDOFmiTORUyCPIY_0 2026-02-24T22:34:25.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='_eMxk_RffcSV4X2zgMvC2RgaeyyGFAMg', time=2026-02-25T00:15:34.754537+0000, chain.objs.size()=1 2026-02-24T22:34:25.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__shadow_.QKLJaehx_iyyTi0XhEEtK594VLJ3McC_0 2026-02-24T22:34:25.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2409765189372994385', time=2026-02-25T00:15:34.791332+0000, chain.objs.size()=3 2026-02-24T22:34:25.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515__multipart_foo-LUKEWARM.2~UQF9y4W3iByiBZqrzUJprUfj83wj_ro.1 2026-02-24T22:34:25.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515__multipart_foo-LUKEWARM.2~UQF9y4W3iByiBZqrzUJprUfj83wj_ro.2 2026-02-24T22:34:25.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515__multipart_foo-LUKEWARM.2~UQF9y4W3iByiBZqrzUJprUfj83wj_ro.3 2026-02-24T22:34:25.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17383254574244108113', time=2026-02-25T00:17:05.175291+0000, chain.objs.size()=1 2026-02-24T22:34:25.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522__shadow_.0OBTTHefHskSKiAYfei4YHdDT8N6iWy_0 2026-02-24T22:34:25.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6627817961067189413', time=2026-02-25T00:17:05.175813+0000, chain.objs.size()=1 2026-02-24T22:34:25.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522__shadow_.zSss7FQOOkS19LVatEOdnoJNd0W8yt1_0 2026-02-24T22:34:25.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12030908096444202373', time=2026-02-25T00:17:05.176343+0000, chain.objs.size()=1 2026-02-24T22:34:25.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1522__shadow_.yFoPC17IRDEVmH_j3W2EFHpqOrB2_pB_0 2026-02-24T22:34:25.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12570194753950941131', time=2026-02-25T00:17:08.013127+0000, chain.objs.size()=1 2026-02-24T22:34:25.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524__shadow_.wxO7E_XvaqxJt7uB64BXxljfIKtfLU7_0 2026-02-24T22:34:25.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.218363270660561854', time=2026-02-25T00:17:41.429063+0000, chain.objs.size()=1 2026-02-24T22:34:25.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526__shadow_.mprxpmiZ0rpMgkW_Z8H8J7f-3XJ-M0n_0 2026-02-24T22:34:25.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17059561397148495527', time=2026-02-25T00:17:42.538143+0000, chain.objs.size()=1 2026-02-24T22:34:25.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527__shadow_.t3qqyBIjlVgevwJo1vvGQ1sIoWIXpNT_0 2026-02-24T22:34:25.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13202215662942476600', time=2026-02-25T00:18:48.193119+0000, chain.objs.size()=1 2026-02-24T22:34:25.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530__shadow_.uol6WI2PgNr-qLBX6ayu5QWobgogZPh_0 2026-02-24T22:34:25.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14440804233952336146', time=2026-02-25T00:18:49.780335+0000, chain.objs.size()=1 2026-02-24T22:34:25.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531__shadow_.7yaRxf_FUil_k9E1n1Q2Ljn0UAMKuvJ_0 2026-02-24T22:34:25.857 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5946815452980324502', time=2026-02-25T00:18:51.100292+0000, chain.objs.size()=1 2026-02-24T22:34:25.858 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.867+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1532__shadow_.y9MHgvaw6YU5sFSKRUqZW3draLdo9ad_0 2026-02-24T22:34:25.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6917255553122228374', time=2026-02-25T00:19:23.069482+0000, chain.objs.size()=1 2026-02-24T22:34:25.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533__shadow_.mYuJ_vsIbT2dKAw0XpiM4toD9f7dtnQ_0 2026-02-24T22:34:25.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6917376206140179273', time=2026-02-25T00:19:23.074113+0000, chain.objs.size()=1 2026-02-24T22:34:25.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1533__shadow_.Tb4jDTRVKw9Ee0tkARsn8TDBXnA5cSa_0 2026-02-24T22:34:25.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11952843961724853536', time=2026-02-25T00:19:57.122297+0000, chain.objs.size()=1 2026-02-24T22:34:25.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535__shadow_.6aub_OJBII-dyt_0LwpyvlsuDT6Cqu2_0 2026-02-24T22:34:25.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12115975335368743363', time=2026-02-25T00:19:57.124217+0000, chain.objs.size()=1 2026-02-24T22:34:25.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535__shadow_.ZPCmogfq6DyeTAJ4YHdW4ARda54xmDM_0 2026-02-24T22:34:25.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3692841229444875986', time=2026-02-25T00:20:32.522319+0000, chain.objs.size()=1 2026-02-24T22:34:25.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538__shadow_.hnnnNc8I28raPzqnD2ZfCf0iPSaq3PA_0 2026-02-24T22:34:25.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9290406102436888751', time=2026-02-25T00:25:38.506506+0000, chain.objs.size()=1 2026-02-24T22:34:25.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1604__shadow_._uVRCRkUxWaiZWaL_evP1z5XNzjmCdT_0 2026-02-24T22:34:25.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13453650517891271482', time=2026-02-25T00:25:43.886213+0000, chain.objs.size()=1 2026-02-24T22:34:25.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1612__shadow_.ups9X5dFz-yXMWdyr_obFWUATYosEZA_0 2026-02-24T22:34:25.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10536393604935437409', time=2026-02-25T00:25:45.021355+0000, chain.objs.size()=1 2026-02-24T22:34:25.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1614__shadow_.QZz86As07SLVYCDwx9TO36WS_Rs7rH6_0 2026-02-24T22:34:25.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7960766331882344402', time=2026-02-25T00:25:46.733686+0000, chain.objs.size()=1 2026-02-24T22:34:25.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1617__shadow_.FVmfT7HLUZ896D4amV-MupwT8_PiLcj_0 2026-02-24T22:34:25.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2780482755125942295', time=2026-02-25T00:25:50.273379+0000, chain.objs.size()=1 2026-02-24T22:34:25.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1623__shadow_.i-Sf8o4jWkFhuSf0fxrIZ0awgAIaRJ__0 2026-02-24T22:34:25.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3133685908483273461', time=2026-02-25T00:25:52.538026+0000, chain.objs.size()=1 2026-02-24T22:34:25.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1627__shadow_.f7aB57d7_x3xqvBX6t0IZkPU856TXTh_0 2026-02-24T22:34:25.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13313507698508065947', time=2026-02-25T00:25:59.098634+0000, chain.objs.size()=1 2026-02-24T22:34:25.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.hriIta2mdrc__609eZl1nrAdy-byDhx_0 2026-02-24T22:34:25.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13235900796700183958', time=2026-02-25T00:25:59.099068+0000, chain.objs.size()=1 2026-02-24T22:34:25.859 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.AYJSe7XQtkONQFhl2pntN4lI1FxXQe5_0 2026-02-24T22:34:25.860 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7033020379009090034', time=2026-02-25T00:25:59.144061+0000, chain.objs.size()=1 2026-02-24T22:34:25.860 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.WwlMcLpElMB5yKdfjZ5bxgTw16sYT-7_0 2026-02-24T22:34:25.860 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12345226139119125396', time=2026-02-25T00:25:59.279622+0000, chain.objs.size()=1 2026-02-24T22:34:25.860 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.l8LOQIb0L3fj588Fz65DDhpeYNhbJqd_0 2026-02-24T22:34:25.860 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5105878291031596817', time=2026-02-25T00:25:59.290028+0000, chain.objs.size()=1 2026-02-24T22:34:25.860 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.8APiFJvU-pplpX9gZXAVkUIJ_-xC61h_0 2026-02-24T22:34:25.860 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.285414144728585980', time=2026-02-25T00:25:59.310816+0000, chain.objs.size()=1 2026-02-24T22:34:25.860 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.4zgVb60SNQLk6LhOCxluS1g8Qyj1tyv_0 2026-02-24T22:34:25.860 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17098479063937787956', time=2026-02-25T00:25:59.427833+0000, chain.objs.size()=1 2026-02-24T22:34:25.860 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.HiC9a33c4WaEd7m6FlUxUBX9iLwLAou_0 2026-02-24T22:34:25.860 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7311836764538005323', time=2026-02-25T00:25:59.437735+0000, chain.objs.size()=1 2026-02-24T22:34:25.860 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.i1ko1u9CYLW52FBpqqBfTCg8CVo6VmY_0 2026-02-24T22:34:25.860 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14097202110588362419', time=2026-02-25T00:25:59.439190+0000, chain.objs.size()=1 2026-02-24T22:34:25.860 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Qjm0r92kOyjAb9Ij8dzrIIAseFvqFMz_0 2026-02-24T22:34:25.860 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6003634187474312678', time=2026-02-25T00:25:59.447413+0000, chain.objs.size()=1 2026-02-24T22:34:25.860 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.UySZOcbab0p5cRbgoB1kprAcDlihl4A_0 2026-02-24T22:34:25.861 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1693030545801503968', time=2026-02-25T00:25:59.489493+0000, chain.objs.size()=1 2026-02-24T22:34:25.861 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.GE6eoDgt4hyBKQtk2__Sx-UJblGUhIC_0 2026-02-24T22:34:25.861 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12307438388283247953', time=2026-02-25T00:25:59.500315+0000, chain.objs.size()=1 2026-02-24T22:34:25.861 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.chdpeBbv3BOcqVzo9PA-t27aFKHij18_0 2026-02-24T22:34:25.861 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8978732309282402676', time=2026-02-25T00:25:59.589145+0000, chain.objs.size()=1 2026-02-24T22:34:25.861 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.HI104Wk9gjMxwsVid6kxvT4HF6dxJHH_0 2026-02-24T22:34:25.861 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6201683469658538996', time=2026-02-25T00:25:59.642658+0000, chain.objs.size()=1 2026-02-24T22:34:25.861 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.yoCFJzFOhq04Th5UPLCGwnJiCd6Xnd6_0 2026-02-24T22:34:25.861 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3528300850688665681', time=2026-02-25T00:25:59.769700+0000, chain.objs.size()=1 2026-02-24T22:34:25.861 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.G9PrUKivdHEB_gICnJiOsiAyo4Gl5Yq_0 2026-02-24T22:34:25.861 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2364638808861131263', time=2026-02-25T00:25:59.820597+0000, chain.objs.size()=1 2026-02-24T22:34:25.861 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.JfxLJUIihSDto9e_zSVdrz7UnD7dUw1_0 2026-02-24T22:34:25.861 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18037189372434567672', time=2026-02-25T00:25:59.959958+0000, chain.objs.size()=1 2026-02-24T22:34:25.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.J5mkcP7awyCDFtAMm76nKhkon-oA29J_0 2026-02-24T22:34:25.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18126324851250027123', time=2026-02-25T00:26:00.083479+0000, chain.objs.size()=1 2026-02-24T22:34:25.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ItoBV3UBIAC1Le072WM6zuIlLi64r97_0 2026-02-24T22:34:25.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13285282015564827164', time=2026-02-25T00:26:00.091268+0000, chain.objs.size()=1 2026-02-24T22:34:25.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.qfG1Q5aAHBfu07T-UnS4BXczWE0TZoq_0 2026-02-24T22:34:25.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9922216502696502089', time=2026-02-25T00:26:00.109867+0000, chain.objs.size()=1 2026-02-24T22:34:25.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.kryt2OAZLEyTuV2pf264-H7Ve2jUJpf_0 2026-02-24T22:34:25.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12693446110017487033', time=2026-02-25T00:26:00.115278+0000, chain.objs.size()=1 2026-02-24T22:34:25.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.5uMtvbitB48CvrbER-33tj_lQ3OdEBZ_0 2026-02-24T22:34:25.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12559267051865147587', time=2026-02-25T00:26:00.116986+0000, chain.objs.size()=1 2026-02-24T22:34:25.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.dshjhEtSSubprS_QJ8xFCamqpzqIEDe_0 2026-02-24T22:34:25.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3175770045363195179', time=2026-02-25T00:26:00.144097+0000, chain.objs.size()=1 2026-02-24T22:34:25.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.R_c1J4HxKfwVS7ODRtkbeudp1HQ9xXL_0 2026-02-24T22:34:25.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6231770810007098856', time=2026-02-25T00:26:00.151792+0000, chain.objs.size()=1 2026-02-24T22:34:25.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.g60zDaQOMbKhF0THA8kV0oSPI6W594A_0 2026-02-24T22:34:25.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10197446407848656284', time=2026-02-25T00:26:00.222679+0000, chain.objs.size()=1 2026-02-24T22:34:25.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.871+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.H1zV5hVyRoTzz46yVS-WMZMWWEqnTAl_0 2026-02-24T22:34:25.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5096769229769533501', time=2026-02-25T00:26:00.224469+0000, chain.objs.size()=1 2026-02-24T22:34:25.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.fbfu2My1zoSuUh_ulaawBIFPV2pMUrc_0 2026-02-24T22:34:25.862 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5632698972550453218', time=2026-02-25T00:26:00.253639+0000, chain.objs.size()=1 2026-02-24T22:34:25.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.jNFmEciTqzK_h6RRSRIyIfQf5h7_DUb_0 2026-02-24T22:34:25.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14614685619052036938', time=2026-02-25T00:26:00.266049+0000, chain.objs.size()=1 2026-02-24T22:34:25.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.du8wA5vYbSIg3x6IzUREBanz_kulPEa_0 2026-02-24T22:34:25.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1377398795494379601', time=2026-02-25T00:26:00.269120+0000, chain.objs.size()=1 2026-02-24T22:34:25.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.7TGl3F65z9j14wDC4mo154UlBgIZygw_0 2026-02-24T22:34:25.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8111541481844036955', time=2026-02-25T00:26:00.274269+0000, chain.objs.size()=1 2026-02-24T22:34:25.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.G4LJoDHugaT1kjcJ_dRchF3HHss24VD_0 2026-02-24T22:34:25.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1168680597913982569', time=2026-02-25T00:26:21.137583+0000, chain.objs.size()=1 2026-02-24T22:34:25.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1662__shadow_.x0fRtv-nKLKwfoAWCJPxtRZRudzFVAj_0 2026-02-24T22:34:25.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2489626904379147685', time=2026-02-25T00:26:25.857620+0000, chain.objs.size()=1 2026-02-24T22:34:25.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1670__shadow_.2l9A8JTw2Idor7BIcy7fJvj9Kgwo0xk_0 2026-02-24T22:34:25.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1228951508106384413', time=2026-02-25T00:26:35.777563+0000, chain.objs.size()=1 2026-02-24T22:34:25.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:11x9T0qsrT8.C46imDPIJXhei75sRWJ_.J_0fMdYDO6-lEBzZuwQhMPQxUnLTwfa_0 2026-02-24T22:34:25.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1228951508106384413', time=2026-02-25T00:26:35.803640+0000, chain.objs.size()=1 2026-02-24T22:34:25.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:11x9T0qsrT8.C46imDPIJXhei75sRWJ_.J_0fMdYDO6-lEBzZuwQhMPQxUnLTwfa_0 2026-02-24T22:34:25.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1228951508106384413', time=2026-02-25T00:26:35.839784+0000, chain.objs.size()=1 2026-02-24T22:34:25.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:11x9T0qsrT8.C46imDPIJXhei75sRWJ_.J_0fMdYDO6-lEBzZuwQhMPQxUnLTwfa_0 2026-02-24T22:34:25.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1228951508106384413', time=2026-02-25T00:26:35.871309+0000, chain.objs.size()=1 2026-02-24T22:34:25.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:11x9T0qsrT8.C46imDPIJXhei75sRWJ_.J_0fMdYDO6-lEBzZuwQhMPQxUnLTwfa_0 2026-02-24T22:34:25.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1228951508106384413', time=2026-02-25T00:26:35.891907+0000, chain.objs.size()=1 2026-02-24T22:34:25.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:11x9T0qsrT8.C46imDPIJXhei75sRWJ_.J_0fMdYDO6-lEBzZuwQhMPQxUnLTwfa_0 2026-02-24T22:34:25.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15794481125256286067', time=2026-02-25T00:26:44.792090+0000, chain.objs.size()=1 2026-02-24T22:34:25.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Zw6iiO9aUIFk9IIxhZnzRpAvadAXXjd_0 2026-02-24T22:34:25.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8982067261521088886', time=2026-02-25T00:26:44.800576+0000, chain.objs.size()=1 2026-02-24T22:34:25.863 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.p9qOroCpZUWF0uS8HzkHePs1C6cnGtD_0 2026-02-24T22:34:25.864 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14288888228669921025', time=2026-02-25T00:26:44.808852+0000, chain.objs.size()=1 2026-02-24T22:34:25.864 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.k9Z5VLhzPHsHBPcTn70cogT283vFM1o_0 2026-02-24T22:34:25.864 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17767809773916847123', time=2026-02-25T00:26:44.820094+0000, chain.objs.size()=1 2026-02-24T22:34:25.864 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.83y2SjaoIbtx5TrO68AR7D1QFY7poL7_0 2026-02-24T22:34:25.864 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10015944518606010392', time=2026-02-25T00:26:44.880044+0000, chain.objs.size()=1 2026-02-24T22:34:25.864 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.x1yqHmgvTmx-O64ChpqGjkXq0i1LFWs_0 2026-02-24T22:34:25.864 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10704558414696414224', time=2026-02-25T00:26:44.959427+0000, chain.objs.size()=1 2026-02-24T22:34:25.864 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.whiUDjeVcg0EXbH1vZur1FOpbSbwVzQ_0 2026-02-24T22:34:25.864 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3617975893287975900', time=2026-02-25T00:26:44.959997+0000, chain.objs.size()=1 2026-02-24T22:34:25.864 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.pzxhKQjWu-_pVo5jBsO-5gWvpp74Z20_0 2026-02-24T22:34:25.864 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16278390789878245853', time=2026-02-25T00:26:44.971710+0000, chain.objs.size()=1 2026-02-24T22:34:25.864 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.rrIP36cY6WZC7fQn8YQVlFvZh2k8TNy_0 2026-02-24T22:34:25.864 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10604108125363486643', time=2026-02-25T00:26:44.984505+0000, chain.objs.size()=1 2026-02-24T22:34:25.864 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.h-5Zgye1jvTJYiAj5gZBbFtKWBLbNyg_0 2026-02-24T22:34:25.864 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3549875666746047889', time=2026-02-25T00:26:45.026128+0000, chain.objs.size()=1 2026-02-24T22:34:25.864 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.bl7jT34oX7xOvfOaoQra_9-6VGwQKIy_0 2026-02-24T22:34:25.864 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3656931461312763326', time=2026-02-25T00:26:45.032328+0000, chain.objs.size()=1 2026-02-24T22:34:25.864 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.B3CVokZRHZUUAsPpFh9fnL2JEQqbG6Q_0 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9393685747239316875', time=2026-02-25T00:26:45.033998+0000, chain.objs.size()=1 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.JyOaYqzvPjqPSilAFiqyadnCq1Yl5CM_0 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13098997569798032362', time=2026-02-25T00:26:45.144397+0000, chain.objs.size()=1 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.fu3pBbMxI3V9P9VyKA70tH8Ugvts06G_0 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14556256562015761799', time=2026-02-25T00:26:45.154241+0000, chain.objs.size()=1 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.t26KF9iPtqelQz25Npga2uWz1fezqO__0 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13020104356021227950', time=2026-02-25T00:26:45.169268+0000, chain.objs.size()=1 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.zkCzc04IfNnUZhjTf1iyobS7xCBNTwQ_0 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12505979497463755302', time=2026-02-25T00:26:45.184102+0000, chain.objs.size()=1 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.wm1CHdPrWe7w82kC_lkjV6mf4HiI__f_0 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12332463498828647001', time=2026-02-25T00:26:45.187460+0000, chain.objs.size()=1 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.bm23PedBQcK_cOFbyiry1HJwcO9t7g9_0 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5699980105912048910', time=2026-02-25T00:26:45.192994+0000, chain.objs.size()=1 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.2f36V5z55doW_HfEqQRxyGR49AFF0IS_0 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6533310058893136018', time=2026-02-25T00:26:45.285346+0000, chain.objs.size()=1 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.em3PttLdcGE1g1tl3u5iiQ8SfIzuuz4_0 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.498993237502156909', time=2026-02-25T00:26:45.305572+0000, chain.objs.size()=1 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.VSxNItMB38I9r2H0POlK-YV6ocLD594_0 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18145741719264571038', time=2026-02-25T00:26:45.310742+0000, chain.objs.size()=1 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.yBSUsES4b1rzWWiUDMfSJdLl7wdDV13_0 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12346269618179307132', time=2026-02-25T00:26:45.343846+0000, chain.objs.size()=1 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.011eIRjtMTV-UYYMt_yNJwnfcjeh2t0_0 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13297816387294637118', time=2026-02-25T00:26:45.348314+0000, chain.objs.size()=1 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.DnkIb7J7Va_eCzgv8gP57tb4nI-4g6g_0 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9847656113751462068', time=2026-02-25T00:26:45.417887+0000, chain.objs.size()=1 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Co9bDdv_FaOfaCKL4lG2fohJbFNHbLm_0 2026-02-24T22:34:25.865 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5981169550309178596', time=2026-02-25T00:26:45.472530+0000, chain.objs.size()=1 2026-02-24T22:34:25.866 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.aYNI8adYHfzbC_MriUNjI0y5sqorhJq_0 2026-02-24T22:34:25.866 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9729242309548149147', time=2026-02-25T00:26:45.579594+0000, chain.objs.size()=1 2026-02-24T22:34:25.866 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.GYoDW8dHyty9-HYuRzH996OgTznCQhB_0 2026-02-24T22:34:25.866 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13923450726576101241', time=2026-02-25T00:26:45.581439+0000, chain.objs.size()=1 2026-02-24T22:34:25.866 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.dk9Ztw5qT2DUWsgvsNZwBB5F3VSQYQf_0 2026-02-24T22:34:25.866 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13321653081826764201', time=2026-02-25T00:26:45.590302+0000, chain.objs.size()=1 2026-02-24T22:34:25.866 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.v-7DcXX0P9O3QWzBKesE8BIvpJK-vGl_0 2026-02-24T22:34:25.866 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18213702463504559440', time=2026-02-25T00:26:45.594432+0000, chain.objs.size()=1 2026-02-24T22:34:25.866 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.xpLsXxiQfIGcwhDt2A5Ow_XXcs4XYII_0 2026-02-24T22:34:25.866 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14120110938894712280', time=2026-02-25T00:26:45.822295+0000, chain.objs.size()=1 2026-02-24T22:34:25.866 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.3OGk1yP3iH-opdf1vbpgDuLWB3Zye5A_0 2026-02-24T22:34:25.866 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5744793329317820148', time=2026-02-25T00:26:45.845173+0000, chain.objs.size()=1 2026-02-24T22:34:25.866 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.4gm1jDDwuh5Mv8OS4efKfIf28Pl3r03_0 2026-02-24T22:34:25.866 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18371798865568555190', time=2026-02-25T00:26:45.930107+0000, chain.objs.size()=1 2026-02-24T22:34:25.866 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.875+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.0ABPwbCvC5y6kpcY0xu3uHI-NlHYh97_0 2026-02-24T22:34:25.866 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17952114011663313436', time=2026-02-25T00:26:45.950281+0000, chain.objs.size()=1 2026-02-24T22:34:25.866 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.eDe2czNlddWsNTjZbpRbCEWps0NYzhJ_0 2026-02-24T22:34:25.866 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1712741488233080797', time=2026-02-25T00:26:45.967370+0000, chain.objs.size()=1 2026-02-24T22:34:25.866 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.3KLnRLXlaWT4kwuUBtOEUuUvG4qc6TE_0 2026-02-24T22:34:25.866 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5058455319847649722', time=2026-02-25T00:26:45.977203+0000, chain.objs.size()=1 2026-02-24T22:34:25.866 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.OQgGXKbUIY4Kynd2KUaQxYJJ3bd6BJT_0 2026-02-24T22:34:25.866 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2988081671175167597', time=2026-02-25T00:26:45.982007+0000, chain.objs.size()=1 2026-02-24T22:34:25.866 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_._jqvDoGlLNHhFFv_ib4X4zsbDGIBY9q_0 2026-02-24T22:34:25.867 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10731734092325594604', time=2026-02-25T00:26:45.996226+0000, chain.objs.size()=1 2026-02-24T22:34:25.867 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.5Tx_CbCO1Qr05VCR8zglZUoSututJ7K_0 2026-02-24T22:34:25.867 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9393542934164191469', time=2026-02-25T00:26:46.110451+0000, chain.objs.size()=1 2026-02-24T22:34:25.867 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.XpSGx8S-oazKnj2MLd5aq91UrFRxaDz_0 2026-02-24T22:34:25.867 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4538268310178505918', time=2026-02-25T00:26:46.115487+0000, chain.objs.size()=1 2026-02-24T22:34:25.867 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.gk1iBFUiNvY5Ur5XBEivFXOViKfhPK8_0 2026-02-24T22:34:25.867 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7970949476233528351', time=2026-02-25T00:26:46.124708+0000, chain.objs.size()=1 2026-02-24T22:34:25.867 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.4lLsN7JTDEVWjqUR5QlvtzQnamFsK_2_0 2026-02-24T22:34:25.868 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/38761 2026-02-24T22:34:25.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=35, truncated=0, next_marker='' 2026-02-24T22:34:25.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3401602518022286018', time=2026-02-25T00:26:46.139643+0000, chain.objs.size()=1 2026-02-24T22:34:25.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.wvC47VzDmUSLCF7zzyjzppuzNRoW7X2_0 2026-02-24T22:34:25.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12973470088738702771', time=2026-02-25T00:26:46.153945+0000, chain.objs.size()=1 2026-02-24T22:34:25.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.siVfknvwfQySn6H9BQ0341xdIS_wC4W_0 2026-02-24T22:34:25.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1391849031835945258', time=2026-02-25T00:26:53.176837+0000, chain.objs.size()=1 2026-02-24T22:34:25.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.IIvec0m9ax0UyA1TuOdLnjx64md4xs9_0 2026-02-24T22:34:25.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2036867135412453494', time=2026-02-25T00:26:53.189848+0000, chain.objs.size()=1 2026-02-24T22:34:25.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.XzFxqeu4Iglr0sGs0OJ0dW_iu5RxUkL_0 2026-02-24T22:34:25.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9485736384904893290', time=2026-02-25T00:26:53.348693+0000, chain.objs.size()=1 2026-02-24T22:34:25.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.zSSB1d-wbKgiKClXzWTrr9Do62aKwiP_0 2026-02-24T22:34:25.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16004537831951632946', time=2026-02-25T00:26:53.358075+0000, chain.objs.size()=1 2026-02-24T22:34:25.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.XJqhL-2AnK0e8jBob8UXlrSzQOhSw6e_0 2026-02-24T22:34:25.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13608088734538380468', time=2026-02-25T00:26:53.365636+0000, chain.objs.size()=1 2026-02-24T22:34:25.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ITAEBmdSQDI_2Uu2bbj3L8zTqWaHED-_0 2026-02-24T22:34:25.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11273959505417411084', time=2026-02-25T00:26:53.404742+0000, chain.objs.size()=1 2026-02-24T22:34:25.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.kqTPdrrs-ldeCnatwiJUf_YYxtgA4X4_0 2026-02-24T22:34:25.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13949937114542330434', time=2026-02-25T00:26:53.410519+0000, chain.objs.size()=1 2026-02-24T22:34:25.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.gRrnaMcElGk4EDjqZTpinpKzi5uKcsm_0 2026-02-24T22:34:25.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18073353679500390322', time=2026-02-25T00:26:53.434277+0000, chain.objs.size()=1 2026-02-24T22:34:25.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.25icZh8UqwumCU8ah8HggfpDQaCp1CF_0 2026-02-24T22:34:25.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2939760918099289357', time=2026-02-25T00:26:53.437541+0000, chain.objs.size()=1 2026-02-24T22:34:25.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.x86dz_3JyM8Aw9iuDWMRRtxWkGsG-sb_0 2026-02-24T22:34:25.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9915936292241366473', time=2026-02-25T00:26:53.630650+0000, chain.objs.size()=1 2026-02-24T22:34:25.870 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.879+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.3FDRAPfDbg5zSHEqQDwQEg9c4o8d1XI_0 2026-02-24T22:34:25.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4333129650105888625', time=2026-02-25T00:26:53.775805+0000, chain.objs.size()=1 2026-02-24T22:34:25.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.WAXPk34EFngKCZXdByclOY3r_S1PMzz_0 2026-02-24T22:34:25.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4400387014932470513', time=2026-02-25T00:26:53.798578+0000, chain.objs.size()=1 2026-02-24T22:34:25.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.swwxSYTmea9RSD8VG2Pzkf-xKbvn-Um_0 2026-02-24T22:34:25.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6535909193724056343', time=2026-02-25T00:26:53.812982+0000, chain.objs.size()=1 2026-02-24T22:34:25.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.7V1RrYlFcv1nOCohvwVEWqjkctuQs1P_0 2026-02-24T22:34:25.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12433212870549697504', time=2026-02-25T00:26:53.819579+0000, chain.objs.size()=1 2026-02-24T22:34:25.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.8ZI_0TsYnhJBj1Kod54zrbih2tZJUCX_0 2026-02-24T22:34:25.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11777874807515628336', time=2026-02-25T00:26:53.824966+0000, chain.objs.size()=1 2026-02-24T22:34:25.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ZUn-TQ5AbMd-vrmgdHMhnPJX6UdF-Fa_0 2026-02-24T22:34:25.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12239080637781980282', time=2026-02-25T00:26:53.990407+0000, chain.objs.size()=1 2026-02-24T22:34:25.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.B2UVo9aXGXyrD4CjIHMK03xfY7R2gVh_0 2026-02-24T22:34:25.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16843392846181028685', time=2026-02-25T00:26:54.020765+0000, chain.objs.size()=1 2026-02-24T22:34:25.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.rB_vKIHKFigTYfzamwkJtKv1-GEriHN_0 2026-02-24T22:34:25.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7751166787230966228', time=2026-02-25T00:26:54.026869+0000, chain.objs.size()=1 2026-02-24T22:34:25.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.YJbuTq1Zki8CpaPLA7YwDxrobGzREWi_0 2026-02-24T22:34:25.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17270690559127619665', time=2026-02-25T00:26:54.034998+0000, chain.objs.size()=1 2026-02-24T22:34:25.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.cs_j9SNQ4kRlt76Ix9DBfV6PLLpmCoN_0 2026-02-24T22:34:25.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17903415077315126925', time=2026-02-25T00:26:54.044846+0000, chain.objs.size()=1 2026-02-24T22:34:25.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.X9sINm_ovkn0GD_-KbQ85WYy4QmbGdP_0 2026-02-24T22:34:25.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8120232414030348488', time=2026-02-25T00:26:54.142811+0000, chain.objs.size()=1 2026-02-24T22:34:25.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.DmjtZoZXJzcuZWv-VxuQSBKzB977Tcf_0 2026-02-24T22:34:25.871 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13879959210241701966', time=2026-02-25T00:26:54.147316+0000, chain.objs.size()=1 2026-02-24T22:34:25.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.CI3BySZzsPBr6DN8GluVK2NVbDEu_zh_0 2026-02-24T22:34:25.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8979846185554934205', time=2026-02-25T00:26:54.171938+0000, chain.objs.size()=1 2026-02-24T22:34:25.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.AHTZNlEwnFtiOZwXcTz4MZlb0iOr2TL_0 2026-02-24T22:34:25.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4557151449167929866', time=2026-02-25T00:26:54.201310+0000, chain.objs.size()=1 2026-02-24T22:34:25.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.j6_SfBHXTzI4oEZPL_2kuJ874v7jBSY_0 2026-02-24T22:34:25.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9031970161933310729', time=2026-02-25T00:26:54.367625+0000, chain.objs.size()=1 2026-02-24T22:34:25.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.d0ViosmQfr0h7rcFddzL1nfrIdB6-FY_0 2026-02-24T22:34:25.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13083159457601120100', time=2026-02-25T00:26:54.376881+0000, chain.objs.size()=1 2026-02-24T22:34:25.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.SiCL6MgQsH9-BAHdMlIkm_9S2VVs2TN_0 2026-02-24T22:34:25.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12934000449907958946', time=2026-02-25T00:26:54.536826+0000, chain.objs.size()=1 2026-02-24T22:34:25.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.RyNBf3-k1ulNKC-0uZwGDenjzDnLXUV_0 2026-02-24T22:34:25.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7219595332194682123', time=2026-02-25T00:27:25.513512+0000, chain.objs.size()=1 2026-02-24T22:34:25.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1744__shadow_.UNlCKSCxT3M1lNhbwcUA_HG-mQRIEB1_0 2026-02-24T22:34:25.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4535851137355969189', time=2026-02-25T00:27:32.493639+0000, chain.objs.size()=1 2026-02-24T22:34:25.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1755__shadow_.XkZ9Z1DWJ122PJ5fnQ8eoM1q0FSsJUJ_0 2026-02-24T22:34:25.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16686922013912536198', time=2026-02-25T00:28:00.504740+0000, chain.objs.size()=1 2026-02-24T22:34:25.872 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1796__shadow_.zdqSJ-jaHgLRyxZ3KIjEVRsU_4-2mvI_0 2026-02-24T22:34:25.873 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17317837269410774675', time=2026-02-25T00:28:13.469190+0000, chain.objs.size()=1 2026-02-24T22:34:25.873 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1816__shadow_.4z8jTYxygN6hKzPxCk3RTnZacpZWyFR_0 2026-02-24T22:34:25.873 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10402256717249493524', time=2026-02-25T00:30:46.522893+0000, chain.objs.size()=1 2026-02-24T22:34:25.873 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:gex4KorMc5g71RvIaZdDX0Qn4pv9pRU_.7kfR7Olf5pU39XW6Ve2iP-y5YLV-c7F_0 2026-02-24T22:34:25.873 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2739547808016001238', time=2026-02-25T00:33:56.849147+0000, chain.objs.size()=1 2026-02-24T22:34:25.873 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1979__shadow_.kCaL0aeANUvl0WaGiuPyG7HhTfcmZ-Q_0 2026-02-24T22:34:25.874 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.883+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.875 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2026-02-24T22:34:25.876 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.877 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/39435' 2026-02-24T22:34:25.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.271960254107259615', time=2026-02-25T00:15:09.927072+0000, chain.objs.size()=1 2026-02-24T22:34:25.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1505__shadow_.8Va6e_1531As9J9QD7ePKNSCrJLUdov_0 2026-02-24T22:34:25.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6535627183964551577', time=2026-02-25T00:15:34.692871+0000, chain.objs.size()=1 2026-02-24T22:34:25.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1516__shadow_.sWqJtHRKb8-jjjuiy67dcn_vIAyyWJ5_0 2026-02-24T22:34:25.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='_fsAoz_6UOtDohCTsbCNwbDGWOgo4Lig', time=2026-02-25T00:15:34.744356+0000, chain.objs.size()=2 2026-02-24T22:34:25.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__shadow_.VKLnlZFWfqVGhJjB9E_MozyOXaKiE-l_0 2026-02-24T22:34:25.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__shadow_.VKLnlZFWfqVGhJjB9E_MozyOXaKiE-l_1 2026-02-24T22:34:25.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6444175142722795253', time=2026-02-25T00:16:33.514821+0000, chain.objs.size()=1 2026-02-24T22:34:25.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1521__shadow_.r3ofPKJ6iKselqzQ9R02X_zsdQs-i2O_0 2026-02-24T22:34:25.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12036611151120074262', time=2026-02-25T00:17:08.005472+0000, chain.objs.size()=1 2026-02-24T22:34:25.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1524__shadow_.VJdNyWX2xRAY-jy43oRNo6UW_DyPZ3q_0 2026-02-24T22:34:25.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3177622782051634494', time=2026-02-25T00:17:42.530676+0000, chain.objs.size()=1 2026-02-24T22:34:25.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527__shadow_.4f9Dym25BSeFuJzh24wTph5pOMzAaFw_0 2026-02-24T22:34:25.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4973485345513626008', time=2026-02-25T00:17:42.532802+0000, chain.objs.size()=1 2026-02-24T22:34:25.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1527__shadow_.dAPAIllZNg0PowjFq_WtKgZN4Fsb_mL_0 2026-02-24T22:34:25.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16531126959234127816', time=2026-02-25T00:18:14.576269+0000, chain.objs.size()=1 2026-02-24T22:34:25.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1528__shadow_.CO1jJ6BzjzQ1PMjEfJvjrnHEJcEp-T2_0 2026-02-24T22:34:25.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12499852912748509141', time=2026-02-25T00:18:16.127341+0000, chain.objs.size()=1 2026-02-24T22:34:25.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529__shadow_.mRP0_Nr974mgxXh644oU7j4tyW9AnZd_0 2026-02-24T22:34:25.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4005761399649826883', time=2026-02-25T00:18:48.185165+0000, chain.objs.size()=1 2026-02-24T22:34:25.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530__shadow_.3w6V9KoI496jgRF5zURAXl1CIwBDPAq_0 2026-02-24T22:34:25.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14938946880885016585', time=2026-02-25T00:18:49.769359+0000, chain.objs.size()=1 2026-02-24T22:34:25.878 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.887+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531__shadow_.KV8ZBK073M4Gs47BVPB95TsFVKVGgGY_0 2026-02-24T22:34:25.879 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1924055474838725992', time=2026-02-25T00:19:58.449960+0000, chain.objs.size()=1 2026-02-24T22:34:25.879 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536__shadow_._n_BgvPKsH2uQH64pf3FyeFGPwIUz1d_0 2026-02-24T22:34:25.879 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4175620108375720515', time=2026-02-25T00:19:58.450517+0000, chain.objs.size()=1 2026-02-24T22:34:25.880 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536__shadow_.x4-V5HNpRJQJ6miCQtCHuibAqezi-wu_0 2026-02-24T22:34:25.880 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2174837603467299271', time=2026-02-25T00:19:58.452759+0000, chain.objs.size()=1 2026-02-24T22:34:25.880 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1536__shadow_.NSVAtVmqWXC1V2hAHRv-UQQA-XL4am7_0 2026-02-24T22:34:25.880 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3962203362341412286', time=2026-02-25T00:25:33.087331+0000, chain.objs.size()=1 2026-02-24T22:34:25.880 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1596__shadow_.r-ucYMqDLWWfdCeZFrN4aQIhkjuct-n_0 2026-02-24T22:34:25.880 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3307657288884458124', time=2026-02-25T00:25:35.426444+0000, chain.objs.size()=1 2026-02-24T22:34:25.880 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1600__shadow_.IqAAJWtqhOTXeMWUy6TqKKhjtpGK5f0_0 2026-02-24T22:34:25.880 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14727002710271404928', time=2026-02-25T00:25:45.566807+0000, chain.objs.size()=1 2026-02-24T22:34:25.880 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1615__shadow_.4vqAPK3-mn9SevKf-Ad3eXRMpLx_3ju_0 2026-02-24T22:34:25.880 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17016268876796089850', time=2026-02-25T00:25:46.718512+0000, chain.objs.size()=1 2026-02-24T22:34:25.880 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1617__shadow_.yAQwtvMvTHgV5LBa0yJt8yDBeqBNvZg_0 2026-02-24T22:34:25.880 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14409732545968584711', time=2026-02-25T00:25:59.155592+0000, chain.objs.size()=1 2026-02-24T22:34:25.880 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.oJjnspnJ_sAEL7_Yzfsiys3GA8v-MwF_0 2026-02-24T22:34:25.880 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8834490564374288957', time=2026-02-25T00:25:59.254322+0000, chain.objs.size()=1 2026-02-24T22:34:25.880 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.BlEyv4uyN5HDIlvAMtInNMC1pV5Soof_0 2026-02-24T22:34:25.880 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16946547212249530148', time=2026-02-25T00:25:59.282468+0000, chain.objs.size()=1 2026-02-24T22:34:25.880 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.jRgE-gDQobHm82D0kF_xT77DKQ16jGc_0 2026-02-24T22:34:25.880 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3288571077957180784', time=2026-02-25T00:25:59.295705+0000, chain.objs.size()=1 2026-02-24T22:34:25.880 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.xeYadbNDmLq-mW7VezLI_V9oDOJkcQM_0 2026-02-24T22:34:25.880 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1312106685642471963', time=2026-02-25T00:25:59.299257+0000, chain.objs.size()=1 2026-02-24T22:34:25.880 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.WYvTu8RdJ0oLtNEkZ6p3G4L9GHWc5rC_0 2026-02-24T22:34:25.881 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18275190907668875476', time=2026-02-25T00:25:59.302057+0000, chain.objs.size()=1 2026-02-24T22:34:25.881 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.MZNUCE5ITG7AwK7eeFQWe4epboDtDGO_0 2026-02-24T22:34:25.881 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1785575548808880054', time=2026-02-25T00:25:59.333810+0000, chain.objs.size()=1 2026-02-24T22:34:25.881 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.AOloLNMm3cI5ZeO00MeYL26NX_sIpWw_0 2026-02-24T22:34:25.881 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15023038899051274444', time=2026-02-25T00:25:59.420141+0000, chain.objs.size()=1 2026-02-24T22:34:25.881 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.11cTfBscMB_2J-IZmtc_5fq3po-MJxe_0 2026-02-24T22:34:25.881 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2683807294844506626', time=2026-02-25T00:25:59.433283+0000, chain.objs.size()=1 2026-02-24T22:34:25.881 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.nka0TsT9EYe2KZlxqVSEvUm2r3pUi6K_0 2026-02-24T22:34:25.881 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16993608589454684750', time=2026-02-25T00:25:59.626410+0000, chain.objs.size()=1 2026-02-24T22:34:25.881 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Qi-1f4_jnrKChD6KV9aTpzum-IQhdSm_0 2026-02-24T22:34:25.881 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1279586311033046322', time=2026-02-25T00:25:59.723668+0000, chain.objs.size()=1 2026-02-24T22:34:25.881 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.iTSddi6G80ZRgk8664tT3PLI1K0lGY4_0 2026-02-24T22:34:25.881 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13548985570667373658', time=2026-02-25T00:25:59.724561+0000, chain.objs.size()=1 2026-02-24T22:34:25.881 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.qBfrbFP-vj4ojd-F6ntR-0UXR5t2jw1_0 2026-02-24T22:34:25.881 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2317399030203344139', time=2026-02-25T00:25:59.735193+0000, chain.objs.size()=1 2026-02-24T22:34:25.881 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.VRgdarkCKqqyLDl1_ft-WZp_8Z0s8ed_0 2026-02-24T22:34:25.881 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14216192911655922351', time=2026-02-25T00:25:59.778674+0000, chain.objs.size()=1 2026-02-24T22:34:25.881 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_._yQPD3lOgKscDceGabHZMXL5BtAaPWb_0 2026-02-24T22:34:25.882 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12538827112487226795', time=2026-02-25T00:25:59.784684+0000, chain.objs.size()=1 2026-02-24T22:34:25.882 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.dw67iri8tcYoW36_wHCsYZSfH2Y4l_d_0 2026-02-24T22:34:25.882 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.880523375694112580', time=2026-02-25T00:25:59.786440+0000, chain.objs.size()=1 2026-02-24T22:34:25.882 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.xwjvSiMrxpYahGf-thm8V5gAfo2zlms_0 2026-02-24T22:34:25.882 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5206015979461049841', time=2026-02-25T00:25:59.823664+0000, chain.objs.size()=1 2026-02-24T22:34:25.882 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.o1NFCirb3eGPKuAfQiRSZGGu79icuX2_0 2026-02-24T22:34:25.882 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5744999381986302304', time=2026-02-25T00:25:59.901723+0000, chain.objs.size()=1 2026-02-24T22:34:25.882 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.891+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.56siALKEy-EonZUlRbC_qP9EdoG_1Ni_0 2026-02-24T22:34:25.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3258803846173679169', time=2026-02-25T00:25:59.902358+0000, chain.objs.size()=1 2026-02-24T22:34:25.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.kQVCRXmzOlQdiwrlH0NVd6XGKtyCr_B_0 2026-02-24T22:34:25.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5676972911347432578', time=2026-02-25T00:25:59.911793+0000, chain.objs.size()=1 2026-02-24T22:34:25.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.2B9hjltSfBu3o-0E4uMymf9GVAph5Ua_0 2026-02-24T22:34:25.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11307949519814447658', time=2026-02-25T00:25:59.936596+0000, chain.objs.size()=1 2026-02-24T22:34:25.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.yFfclfuICJMROgC1XgMBzV8p75Ulao8_0 2026-02-24T22:34:25.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7519227463238308651', time=2026-02-25T00:25:59.974362+0000, chain.objs.size()=1 2026-02-24T22:34:25.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.QqQwNahLpyKtGrA23dN5rv85VqYqAf2_0 2026-02-24T22:34:25.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9081870212612431441', time=2026-02-25T00:26:00.072157+0000, chain.objs.size()=1 2026-02-24T22:34:25.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.P3M09hswN9XbVIlpKPGod83jVJdpXbn_0 2026-02-24T22:34:25.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1837451259102369801', time=2026-02-25T00:26:00.075042+0000, chain.objs.size()=1 2026-02-24T22:34:25.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.eeuTgbBQQl_qn-XCcBJmmguJz7mBNXp_0 2026-02-24T22:34:25.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7922201969720095886', time=2026-02-25T00:26:00.094309+0000, chain.objs.size()=1 2026-02-24T22:34:25.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.M8gEy0kA7T3bDp41jHlKEthPCW_DWgU_0 2026-02-24T22:34:25.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5138845092330226616', time=2026-02-25T00:26:00.099161+0000, chain.objs.size()=1 2026-02-24T22:34:25.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.e4R-eGup6hxx99Pk4SwEckuATv2WFh7_0 2026-02-24T22:34:25.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14826398132349248514', time=2026-02-25T00:26:00.109023+0000, chain.objs.size()=1 2026-02-24T22:34:25.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.G7UAAeSIeXk5ArlyRaZc50MAPCoy5Hq_0 2026-02-24T22:34:25.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16943706255982582499', time=2026-02-25T00:26:00.111677+0000, chain.objs.size()=1 2026-02-24T22:34:25.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.oCOgNo0Gf0zkr7vlPyUH9jAPUjylP9X_0 2026-02-24T22:34:25.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9997766322410339548', time=2026-02-25T00:26:00.136288+0000, chain.objs.size()=1 2026-02-24T22:34:25.883 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.n0TsGtEUlZ8ijwV15bhxC81Gv8KzVTb_0 2026-02-24T22:34:25.884 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17739822240183861034', time=2026-02-25T00:26:00.235354+0000, chain.objs.size()=1 2026-02-24T22:34:25.884 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.tXwwuemptgvl_-L2KqVDinzp_H_aTFF_0 2026-02-24T22:34:25.884 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6478412800429587234', time=2026-02-25T00:26:00.266018+0000, chain.objs.size()=1 2026-02-24T22:34:25.884 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.QwM6cX_FmlEfosx6PMATrXlmfKu5YKP_0 2026-02-24T22:34:25.884 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5846280378182012028', time=2026-02-25T00:26:00.278568+0000, chain.objs.size()=1 2026-02-24T22:34:25.884 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.N3mFwzluSEOnjZJXkDeXu-2e7ofS4Se_0 2026-02-24T22:34:25.884 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8288435603919421319', time=2026-02-25T00:26:07.097802+0000, chain.objs.size()=1 2026-02-24T22:34:25.884 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1640__shadow_.DOtiSCIdJSGgNAFmYPorZ5nfDKsokoC_0 2026-02-24T22:34:25.884 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6300015898339601241', time=2026-02-25T00:26:14.802862+0000, chain.objs.size()=1 2026-02-24T22:34:25.884 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1653__shadow_.PIYYuxe9VkdLB7TQD8fR0E_bVuNNQ6i_0 2026-02-24T22:34:25.884 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1512362058534691928', time=2026-02-25T00:26:18.493157+0000, chain.objs.size()=1 2026-02-24T22:34:25.884 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1658__shadow_.4_pGsoVq6aT6OFgcWSjGqbbp6LtSut6_0 2026-02-24T22:34:25.884 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12264021723077154799', time=2026-02-25T00:26:18.493648+0000, chain.objs.size()=1 2026-02-24T22:34:25.885 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1658__shadow_.LWSJiHjThcR2WHt-kp0yQ4w3HyoYlpq_0 2026-02-24T22:34:25.885 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7961364855291132285', time=2026-02-25T00:26:22.290629+0000, chain.objs.size()=1 2026-02-24T22:34:25.885 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1664__shadow_.Wgbx804nV--Rrwwq-8Bn7Pd7kjjYwcf_0 2026-02-24T22:34:25.885 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4489580540685174554', time=2026-02-25T00:26:22.866325+0000, chain.objs.size()=1 2026-02-24T22:34:25.885 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1665__shadow_.J5iLBls-ABkO8I0LdohItWRRWVmBSIf_0 2026-02-24T22:34:25.885 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9663502067891694475', time=2026-02-25T00:26:33.380769+0000, chain.objs.size()=1 2026-02-24T22:34:25.885 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1680__shadow_.I8xnEMdooMw49Smx68nfvs4DJbGAQQw_0 2026-02-24T22:34:25.885 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5858843419042869532', time=2026-02-25T00:26:35.788299+0000, chain.objs.size()=1 2026-02-24T22:34:25.885 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:pA3EcyPInXTkW0DgsVnP3oKX6peKe.E_.6dYClBjU_Zv7xFzvDR6LIy6FjkiaZE0_0 2026-02-24T22:34:25.886 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5858843419042869532', time=2026-02-25T00:26:35.791848+0000, chain.objs.size()=1 2026-02-24T22:34:25.886 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:pA3EcyPInXTkW0DgsVnP3oKX6peKe.E_.6dYClBjU_Zv7xFzvDR6LIy6FjkiaZE0_0 2026-02-24T22:34:25.886 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5858843419042869532', time=2026-02-25T00:26:35.795079+0000, chain.objs.size()=1 2026-02-24T22:34:25.886 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:pA3EcyPInXTkW0DgsVnP3oKX6peKe.E_.6dYClBjU_Zv7xFzvDR6LIy6FjkiaZE0_0 2026-02-24T22:34:25.886 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5858843419042869532', time=2026-02-25T00:26:35.799223+0000, chain.objs.size()=1 2026-02-24T22:34:25.886 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:pA3EcyPInXTkW0DgsVnP3oKX6peKe.E_.6dYClBjU_Zv7xFzvDR6LIy6FjkiaZE0_0 2026-02-24T22:34:25.886 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5858843419042869532', time=2026-02-25T00:26:35.804874+0000, chain.objs.size()=1 2026-02-24T22:34:25.886 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:pA3EcyPInXTkW0DgsVnP3oKX6peKe.E_.6dYClBjU_Zv7xFzvDR6LIy6FjkiaZE0_0 2026-02-24T22:34:25.886 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5858843419042869532', time=2026-02-25T00:26:35.808472+0000, chain.objs.size()=1 2026-02-24T22:34:25.886 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:pA3EcyPInXTkW0DgsVnP3oKX6peKe.E_.6dYClBjU_Zv7xFzvDR6LIy6FjkiaZE0_0 2026-02-24T22:34:25.886 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5858843419042869532', time=2026-02-25T00:26:35.814413+0000, chain.objs.size()=1 2026-02-24T22:34:25.886 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:pA3EcyPInXTkW0DgsVnP3oKX6peKe.E_.6dYClBjU_Zv7xFzvDR6LIy6FjkiaZE0_0 2026-02-24T22:34:25.886 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5858843419042869532', time=2026-02-25T00:26:35.821294+0000, chain.objs.size()=1 2026-02-24T22:34:25.886 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:pA3EcyPInXTkW0DgsVnP3oKX6peKe.E_.6dYClBjU_Zv7xFzvDR6LIy6FjkiaZE0_0 2026-02-24T22:34:25.886 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5858843419042869532', time=2026-02-25T00:26:35.826496+0000, chain.objs.size()=1 2026-02-24T22:34:25.886 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:pA3EcyPInXTkW0DgsVnP3oKX6peKe.E_.6dYClBjU_Zv7xFzvDR6LIy6FjkiaZE0_0 2026-02-24T22:34:25.887 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5858843419042869532', time=2026-02-25T00:26:35.826966+0000, chain.objs.size()=1 2026-02-24T22:34:25.887 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:pA3EcyPInXTkW0DgsVnP3oKX6peKe.E_.6dYClBjU_Zv7xFzvDR6LIy6FjkiaZE0_0 2026-02-24T22:34:25.887 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5858843419042869532', time=2026-02-25T00:26:35.830685+0000, chain.objs.size()=1 2026-02-24T22:34:25.887 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:pA3EcyPInXTkW0DgsVnP3oKX6peKe.E_.6dYClBjU_Zv7xFzvDR6LIy6FjkiaZE0_0 2026-02-24T22:34:25.887 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5858843419042869532', time=2026-02-25T00:26:35.835163+0000, chain.objs.size()=1 2026-02-24T22:34:25.887 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.895+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:pA3EcyPInXTkW0DgsVnP3oKX6peKe.E_.6dYClBjU_Zv7xFzvDR6LIy6FjkiaZE0_0 2026-02-24T22:34:25.887 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.899+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5858843419042869532', time=2026-02-25T00:26:35.841869+0000, chain.objs.size()=1 2026-02-24T22:34:25.887 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.899+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:pA3EcyPInXTkW0DgsVnP3oKX6peKe.E_.6dYClBjU_Zv7xFzvDR6LIy6FjkiaZE0_0 2026-02-24T22:34:25.887 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.899+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5858843419042869532', time=2026-02-25T00:26:35.842619+0000, chain.objs.size()=1 2026-02-24T22:34:25.887 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.899+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:pA3EcyPInXTkW0DgsVnP3oKX6peKe.E_.6dYClBjU_Zv7xFzvDR6LIy6FjkiaZE0_0 2026-02-24T22:34:25.887 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.899+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5858843419042869532', time=2026-02-25T00:26:35.846335+0000, chain.objs.size()=1 2026-02-24T22:34:25.887 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.899+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:pA3EcyPInXTkW0DgsVnP3oKX6peKe.E_.6dYClBjU_Zv7xFzvDR6LIy6FjkiaZE0_0 2026-02-24T22:34:25.887 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.899+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5858843419042869532', time=2026-02-25T00:26:35.851347+0000, chain.objs.size()=1 2026-02-24T22:34:25.887 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.899+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:pA3EcyPInXTkW0DgsVnP3oKX6peKe.E_.6dYClBjU_Zv7xFzvDR6LIy6FjkiaZE0_0 2026-02-24T22:34:25.888 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.899+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5858843419042869532', time=2026-02-25T00:26:35.857533+0000, chain.objs.size()=1 2026-02-24T22:34:25.888 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.899+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:pA3EcyPInXTkW0DgsVnP3oKX6peKe.E_.6dYClBjU_Zv7xFzvDR6LIy6FjkiaZE0_0 2026-02-24T22:34:25.888 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.899+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5858843419042869532', time=2026-02-25T00:26:35.858080+0000, chain.objs.size()=1 2026-02-24T22:34:25.888 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.899+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:pA3EcyPInXTkW0DgsVnP3oKX6peKe.E_.6dYClBjU_Zv7xFzvDR6LIy6FjkiaZE0_0 2026-02-24T22:34:25.888 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.899+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5858843419042869532', time=2026-02-25T00:26:35.860783+0000, chain.objs.size()=1 2026-02-24T22:34:25.888 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.899+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:pA3EcyPInXTkW0DgsVnP3oKX6peKe.E_.6dYClBjU_Zv7xFzvDR6LIy6FjkiaZE0_0 2026-02-24T22:34:25.889 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.899+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5858843419042869532', time=2026-02-25T00:26:35.865971+0000, chain.objs.size()=1 2026-02-24T22:34:25.889 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.899+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:pA3EcyPInXTkW0DgsVnP3oKX6peKe.E_.6dYClBjU_Zv7xFzvDR6LIy6FjkiaZE0_0 2026-02-24T22:34:25.889 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.899+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5858843419042869532', time=2026-02-25T00:26:35.876451+0000, chain.objs.size()=1 2026-02-24T22:34:25.889 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.899+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:pA3EcyPInXTkW0DgsVnP3oKX6peKe.E_.6dYClBjU_Zv7xFzvDR6LIy6FjkiaZE0_0 2026-02-24T22:34:25.889 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.899+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13347675579630122246', time=2026-02-25T00:26:37.837553+0000, chain.objs.size()=1 2026-02-24T22:34:25.889 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.899+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1687__shadow_.oEnjZ237v-x3_W4H1zXB4TCLJ1_IK04_0 2026-02-24T22:34:25.889 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.899+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4223894756764373298', time=2026-02-25T00:26:44.776550+0000, chain.objs.size()=1 2026-02-24T22:34:25.889 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.899+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.vjAGKL68fDsu_SXdRuNX_1CEB2-LBym_0 2026-02-24T22:34:25.890 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.899+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18370880119777201859', time=2026-02-25T00:26:44.814336+0000, chain.objs.size()=1 2026-02-24T22:34:25.890 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.899+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.XTpCdLC4olKGMAhuj_871D0K-5g5KnX_0 2026-02-24T22:34:25.890 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11281384028669015404', time=2026-02-25T00:26:44.825720+0000, chain.objs.size()=1 2026-02-24T22:34:25.891 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.irqfB_192-CEZ6jZZmb766RHpYtn7tM_0 2026-02-24T22:34:25.891 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12492503882727846389', time=2026-02-25T00:26:44.855648+0000, chain.objs.size()=1 2026-02-24T22:34:25.891 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.6h92y_i580z7b4uN4OYtRenOuSos_P4_0 2026-02-24T22:34:25.891 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14005058385220649813', time=2026-02-25T00:26:44.858751+0000, chain.objs.size()=1 2026-02-24T22:34:25.891 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.nEIj0KDrr5s-t7zA0yUzWxeyjsPM4rb_0 2026-02-24T22:34:25.891 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13604665936646163840', time=2026-02-25T00:26:44.936124+0000, chain.objs.size()=1 2026-02-24T22:34:25.891 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.hAtPvVQVGJPpHZyuBf3TmkNBw5-txfu_0 2026-02-24T22:34:25.892 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17947540499275777711', time=2026-02-25T00:26:44.955485+0000, chain.objs.size()=1 2026-02-24T22:34:25.892 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.iV4G0pO8Bk8mVcxadPJDi--Kj-7FQ2K_0 2026-02-24T22:34:25.892 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12191995902214398399', time=2026-02-25T00:26:44.970144+0000, chain.objs.size()=1 2026-02-24T22:34:25.892 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.SPMpC4_gmt4Y8FXPmq0sj0RhXY5Mn_w_0 2026-02-24T22:34:25.892 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12349097771048553995', time=2026-02-25T00:26:45.022975+0000, chain.objs.size()=1 2026-02-24T22:34:25.892 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.JA1mnJj6ycCUT3PYsBheHwTW0u6wZrB_0 2026-02-24T22:34:25.892 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12556172404927168883', time=2026-02-25T00:26:45.133426+0000, chain.objs.size()=1 2026-02-24T22:34:25.893 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.px3531tD8LJwrcu1BeWUkbkh6UuKFha_0 2026-02-24T22:34:25.893 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5820566852006346164', time=2026-02-25T00:26:45.147120+0000, chain.objs.size()=1 2026-02-24T22:34:25.893 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.dYlK336tgNafPJzUXqL0lkh5FfBISir_0 2026-02-24T22:34:25.893 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15355921761018440519', time=2026-02-25T00:26:45.254579+0000, chain.objs.size()=1 2026-02-24T22:34:25.893 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.vIAKPxz7eXlgDAkXnyvX88cqnRR0ndi_0 2026-02-24T22:34:25.893 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1168578709754973178', time=2026-02-25T00:26:45.271885+0000, chain.objs.size()=1 2026-02-24T22:34:25.893 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.2_S2AFKlu5cMShTeNGIFSvRLv0vCoSf_0 2026-02-24T22:34:25.893 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17767333138338770614', time=2026-02-25T00:26:45.280569+0000, chain.objs.size()=1 2026-02-24T22:34:25.893 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.NobRtcpkrmruFq-efq1gFl_BqflLDnn_0 2026-02-24T22:34:25.893 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3388038677525266506', time=2026-02-25T00:26:45.398356+0000, chain.objs.size()=1 2026-02-24T22:34:25.893 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.d5iLrJ9q57f-H3AVtn3MI27DMcb67Ts_0 2026-02-24T22:34:25.893 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5799355773133225349', time=2026-02-25T00:26:45.409894+0000, chain.objs.size()=1 2026-02-24T22:34:25.893 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Zwb6pJANs7t527uernL3Txpu-vdgtj9_0 2026-02-24T22:34:25.893 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16290048150340506078', time=2026-02-25T00:26:45.411244+0000, chain.objs.size()=1 2026-02-24T22:34:25.893 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.MMivXTdEdKvS107KiHnn7izOij56zCQ_0 2026-02-24T22:34:25.893 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3420615065389024552', time=2026-02-25T00:26:45.427357+0000, chain.objs.size()=1 2026-02-24T22:34:25.893 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.IjXZx31xKHMr-oWN0FBp1_mb3km7yRW_0 2026-02-24T22:34:25.893 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2659383014123381983', time=2026-02-25T00:26:45.453250+0000, chain.objs.size()=1 2026-02-24T22:34:25.893 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.VecA_73JMVMTEAnXRUkDTMYSUUfsSGu_0 2026-02-24T22:34:25.894 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4772298852932442526', time=2026-02-25T00:26:45.464034+0000, chain.objs.size()=1 2026-02-24T22:34:25.894 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.istpoUPOMdcQPpqfwTofoNtTdWIE3_O_0 2026-02-24T22:34:25.894 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12330067304938052435', time=2026-02-25T00:26:45.466760+0000, chain.objs.size()=1 2026-02-24T22:34:25.894 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.903+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.5xPHowMHkz1wcj1pKqFZ03atOvszsb3_0 2026-02-24T22:34:25.894 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/39435 2026-02-24T22:34:25.896 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=52, truncated=0, next_marker='' 2026-02-24T22:34:25.896 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12185742653627450198', time=2026-02-25T00:26:45.479492+0000, chain.objs.size()=1 2026-02-24T22:34:25.896 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.7MrjmR-ZxdCYrcq-MY0I5vxXx2j7GWw_0 2026-02-24T22:34:25.896 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.321989927913723407', time=2026-02-25T00:26:45.596357+0000, chain.objs.size()=1 2026-02-24T22:34:25.896 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ZZUCtYkFqzpTu0NC0RR_5Vigc8_zXjx_0 2026-02-24T22:34:25.896 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3721084373714298112', time=2026-02-25T00:26:45.915368+0000, chain.objs.size()=1 2026-02-24T22:34:25.896 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.rILUQ2XxrnPq8uFOcL4lTn2AoXGxtBh_0 2026-02-24T22:34:25.897 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.354088526991661683', time=2026-02-25T00:26:45.922824+0000, chain.objs.size()=1 2026-02-24T22:34:25.897 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.tqNG2w6sEF7dgtxhLSRM07lGNhwD3ae_0 2026-02-24T22:34:25.897 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4349330996971939472', time=2026-02-25T00:26:46.077002+0000, chain.objs.size()=1 2026-02-24T22:34:25.897 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.31CbeEr5wIhU8CC1vOovTFGP85DOAGo_0 2026-02-24T22:34:25.897 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10030856675852531796', time=2026-02-25T00:26:46.119738+0000, chain.objs.size()=1 2026-02-24T22:34:25.897 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.BF5tTQQ2cnJoDZ_VboL6qmNc0YFeHAe_0 2026-02-24T22:34:25.897 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.606533883983682033', time=2026-02-25T00:26:46.125686+0000, chain.objs.size()=1 2026-02-24T22:34:25.897 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.-pfV-bzeyBBhssRO6KBr3v0WzdDTpJf_0 2026-02-24T22:34:25.897 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12872316974047485550', time=2026-02-25T00:26:53.145057+0000, chain.objs.size()=1 2026-02-24T22:34:25.897 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.vvP1O2m1h55sQU_J46PrHfdqnxUjqWa_0 2026-02-24T22:34:25.897 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17447353910804324649', time=2026-02-25T00:26:53.170712+0000, chain.objs.size()=1 2026-02-24T22:34:25.897 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.DpbJ8UIPXJJpkysGbd0-upB5XURWi83_0 2026-02-24T22:34:25.897 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1120052739388288999', time=2026-02-25T00:26:53.350791+0000, chain.objs.size()=1 2026-02-24T22:34:25.897 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.SkGJl5nHr8_BCKKT06H13m1DlkRioYh_0 2026-02-24T22:34:25.897 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4740208083641155888', time=2026-02-25T00:26:53.433234+0000, chain.objs.size()=1 2026-02-24T22:34:25.897 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.QNF5ktYx3FA6CG2EKlhMncgPEp8mGKe_0 2026-02-24T22:34:25.897 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1790416167302776725', time=2026-02-25T00:26:53.506956+0000, chain.objs.size()=1 2026-02-24T22:34:25.897 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.11aVo1Qt3uBODB1yKRK0w8u-yxRtRsC_0 2026-02-24T22:34:25.897 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10018228754707988256', time=2026-02-25T00:26:53.602471+0000, chain.objs.size()=1 2026-02-24T22:34:25.897 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.42w-ZDJG8UMgpgRufGOmyfCnapBjH0g_0 2026-02-24T22:34:25.898 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13535453317754517323', time=2026-02-25T00:26:53.617553+0000, chain.objs.size()=1 2026-02-24T22:34:25.898 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Yzu5n2t25uMRFr65eTtjTtSC3bkOt4d_0 2026-02-24T22:34:25.898 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16422186760937293126', time=2026-02-25T00:26:53.762189+0000, chain.objs.size()=1 2026-02-24T22:34:25.898 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.BYv8cy2XbhzO2syTNkiANem66bd3cr9_0 2026-02-24T22:34:25.898 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1121637734310233280', time=2026-02-25T00:26:53.828641+0000, chain.objs.size()=1 2026-02-24T22:34:25.898 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.yMgo6F1wwsewvoZJbQwJxgngC-102ba_0 2026-02-24T22:34:25.898 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4597258189105616072', time=2026-02-25T00:26:53.844658+0000, chain.objs.size()=1 2026-02-24T22:34:25.898 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.JJ2adEGFX6QUfsS_o0mPEFkpvbgm8kz_0 2026-02-24T22:34:25.898 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.647752786488538274', time=2026-02-25T00:26:53.919992+0000, chain.objs.size()=1 2026-02-24T22:34:25.898 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.hJ-RVCRgP0HO6CNdL7gu-wByUg3Qmfj_0 2026-02-24T22:34:25.898 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4339471914734130935', time=2026-02-25T00:26:53.949696+0000, chain.objs.size()=1 2026-02-24T22:34:25.898 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.n5UhAPKwa9gC9mrEbMKVuWA5Y3cOPWr_0 2026-02-24T22:34:25.898 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4500407993154821152', time=2026-02-25T00:26:53.970740+0000, chain.objs.size()=1 2026-02-24T22:34:25.898 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.5ykqBbZdNrRChM9dQKWCxCCVlG7chm1_0 2026-02-24T22:34:25.898 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16137987519492566413', time=2026-02-25T00:26:53.999958+0000, chain.objs.size()=1 2026-02-24T22:34:25.898 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Fr40OuAgttCLUIrLcnoPNPsxqtW7Hpl_0 2026-02-24T22:34:25.898 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17248624418589079234', time=2026-02-25T00:26:54.145697+0000, chain.objs.size()=1 2026-02-24T22:34:25.898 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.I-yt9h7QkNvGe194RTGjeqhh_wYv2Sn_0 2026-02-24T22:34:25.898 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14367907848618389923', time=2026-02-25T00:26:54.163878+0000, chain.objs.size()=1 2026-02-24T22:34:25.898 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.907+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.z7Ed7VSvAjFfWwFOtPd46b-obrrEEpi_0 2026-02-24T22:34:25.898 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.911+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6251868167227720758', time=2026-02-25T00:26:54.293133+0000, chain.objs.size()=1 2026-02-24T22:34:25.898 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.911+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.m1jdKlyPRZql2XvIV2_G2C7LO-R4kuV_0 2026-02-24T22:34:25.898 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.911+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10173888538332862168', time=2026-02-25T00:26:54.295344+0000, chain.objs.size()=1 2026-02-24T22:34:25.898 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.911+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.PFM1gozLwq72yZCFde9TgB4wBQzo12x_0 2026-02-24T22:34:25.898 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.911+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7155957779099392342', time=2026-02-25T00:26:54.345031+0000, chain.objs.size()=1 2026-02-24T22:34:25.899 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.911+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.QXW776OjDwIrJSbD0kBHS8NpTn5rWdw_0 2026-02-24T22:34:25.899 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.911+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10904527192915645462', time=2026-02-25T00:26:54.364280+0000, chain.objs.size()=1 2026-02-24T22:34:25.899 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.911+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.BJmrsBnoupFsRL4UxUT0WWboUGaU_Gi_0 2026-02-24T22:34:25.903 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6407791655318221258', time=2026-02-25T00:26:54.384813+0000, chain.objs.size()=1 2026-02-24T22:34:25.903 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.vOVn_qRGgYDlLE9eD2FoIwPZdPf6Dls_0 2026-02-24T22:34:25.903 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17762644277259402900', time=2026-02-25T00:26:54.450618+0000, chain.objs.size()=1 2026-02-24T22:34:25.903 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.CxO1ZW4VD5P85BmhX7puCVRbfMlfASc_0 2026-02-24T22:34:25.903 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10137662168503947818', time=2026-02-25T00:26:54.457604+0000, chain.objs.size()=1 2026-02-24T22:34:25.903 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.TnMUImrBMBsRyExUPPMy3dHJT6Rnj_D_0 2026-02-24T22:34:25.903 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5496589644353302539', time=2026-02-25T00:26:54.475233+0000, chain.objs.size()=1 2026-02-24T22:34:25.903 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.KqBy5kKvYHYUoCGLEVtWjxPKB-cF9FA_0 2026-02-24T22:34:25.903 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10577749717686737179', time=2026-02-25T00:26:54.501367+0000, chain.objs.size()=1 2026-02-24T22:34:25.903 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.V192EhZ70Bi3ogU6RTFuqazf0mTNP4F_0 2026-02-24T22:34:25.903 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14766051041725760142', time=2026-02-25T00:27:45.525505+0000, chain.objs.size()=1 2026-02-24T22:34:25.903 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1774__shadow_.dQC7MFZMhO01lxRF5pS78o3Ew6InPmX_0 2026-02-24T22:34:25.903 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1550179513596835267', time=2026-02-25T00:28:01.684691+0000, chain.objs.size()=1 2026-02-24T22:34:25.903 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1798__shadow_.3q43a6yqX5sXa-B7roZO6-WzZy5P539_0 2026-02-24T22:34:25.903 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14431788294752433075', time=2026-02-25T00:28:24.969979+0000, chain.objs.size()=1 2026-02-24T22:34:25.903 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1831__shadow_.byFewDkJB2WMd_bQlf5h3RXhwaT7QNN_0 2026-02-24T22:34:25.903 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10557307794018414621', time=2026-02-25T00:28:33.164530+0000, chain.objs.size()=1 2026-02-24T22:34:25.903 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1850__shadow_.ljOcGNf0WQqPzAJmgU46Jc3FZqpgRR7_0 2026-02-24T22:34:25.903 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2308262244710449399', time=2026-02-25T00:28:41.359986+0000, chain.objs.size()=1 2026-02-24T22:34:25.903 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1865__shadow_.FUxahofO3CGKH8-QgsARNb4V9uhf5VO_0 2026-02-24T22:34:25.903 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6284711062045213363', time=2026-02-25T00:28:47.684457+0000, chain.objs.size()=1 2026-02-24T22:34:25.903 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1877__multipart_mymultipart.2~aN1H-jXk5iUqJ04Z-3nkOgyQazGVMgp.1 2026-02-24T22:34:25.904 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='2~oD2SxznN4wv-EXhzkNJlEspeV1HcpXs', time=2026-02-25T00:29:18.518767+0000, chain.objs.size()=1 2026-02-24T22:34:25.904 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1897__multipart_mymultipart.2~oD2SxznN4wv-EXhzkNJlEspeV1HcpXs.1 2026-02-24T22:34:25.904 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.281134528515562376', time=2026-02-25T00:30:34.664004+0000, chain.objs.size()=1 2026-02-24T22:34:25.904 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1946__shadow:AgFXCXL3MgQKm8.l6vMzUZSmfPZBoJV_.yvgcfjdo_WNAagmJ_xr-LySCGFSnMEB_0 2026-02-24T22:34:25.904 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1542732909745573201', time=2026-02-25T00:30:56.999986+0000, chain.objs.size()=1 2026-02-24T22:34:25.904 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:OpupQh2yRRIgzEx5VKyXk45KT76HHn3_.JqnNbbIf35ecKPgu_84dwH9zSiuVHqT_0 2026-02-24T22:34:25.904 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1542732909745573201', time=2026-02-25T00:30:57.001122+0000, chain.objs.size()=1 2026-02-24T22:34:25.904 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:OpupQh2yRRIgzEx5VKyXk45KT76HHn3_.JqnNbbIf35ecKPgu_84dwH9zSiuVHqT_0 2026-02-24T22:34:25.904 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12816447853516362127', time=2026-02-25T00:30:57.181721+0000, chain.objs.size()=1 2026-02-24T22:34:25.904 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:wEZQ8F3qnRlZBVkXH2U7Y.gTB-NgYEc_.MgDyJ8pDsHvdn0Dmu3TMb8_Z-SM1Wox_0 2026-02-24T22:34:25.904 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12816447853516362127', time=2026-02-25T00:30:57.182192+0000, chain.objs.size()=1 2026-02-24T22:34:25.904 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:wEZQ8F3qnRlZBVkXH2U7Y.gTB-NgYEc_.MgDyJ8pDsHvdn0Dmu3TMb8_Z-SM1Wox_0 2026-02-24T22:34:25.904 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12816447853516362127', time=2026-02-25T00:30:57.182612+0000, chain.objs.size()=1 2026-02-24T22:34:25.904 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:wEZQ8F3qnRlZBVkXH2U7Y.gTB-NgYEc_.MgDyJ8pDsHvdn0Dmu3TMb8_Z-SM1Wox_0 2026-02-24T22:34:25.904 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12816447853516362127', time=2026-02-25T00:30:57.183532+0000, chain.objs.size()=1 2026-02-24T22:34:25.904 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:wEZQ8F3qnRlZBVkXH2U7Y.gTB-NgYEc_.MgDyJ8pDsHvdn0Dmu3TMb8_Z-SM1Wox_0 2026-02-24T22:34:25.904 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9769927805152953371', time=2026-02-25T00:30:57.549267+0000, chain.objs.size()=1 2026-02-24T22:34:25.904 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:A2BIBb3RocRAqGyMpXTUorRNIMfWAXB_.IFb3yqvhrQ6We8OopDAUusMWF-el6xM_0 2026-02-24T22:34:25.904 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9769927805152953371', time=2026-02-25T00:30:57.549666+0000, chain.objs.size()=1 2026-02-24T22:34:25.904 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:A2BIBb3RocRAqGyMpXTUorRNIMfWAXB_.IFb3yqvhrQ6We8OopDAUusMWF-el6xM_0 2026-02-24T22:34:25.904 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10272283873744677336', time=2026-02-25T00:33:59.605585+0000, chain.objs.size()=1 2026-02-24T22:34:25.904 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1982__shadow_.1gEMk3WHJxEOMdbCv7RVHoFnzSGpEAE_0 2026-02-24T22:34:25.905 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4666501487188750954', time=2026-02-25T00:34:01.298875+0000, chain.objs.size()=1 2026-02-24T22:34:25.905 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1984__shadow_.WxzePnL5sXxyb5J7rpIMgl_9_Yvn81S_0 2026-02-24T22:34:25.905 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17776931387319041418', time=2026-02-25T00:34:05.698241+0000, chain.objs.size()=1 2026-02-24T22:34:25.905 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1988__shadow_.z36bNfh4YetRlAp_GmybJsbPwI9kqWD_0 2026-02-24T22:34:25.905 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15267829849310681377', time=2026-02-25T00:34:17.240995+0000, chain.objs.size()=1 2026-02-24T22:34:25.905 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1964__shadow_.EgZHLlGqNWiHRvYELTP37SP-5THTj9F_0 2026-02-24T22:34:25.905 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.915+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.907 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.919+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2026-02-24T22:34:25.908 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.919+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.908 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.919+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.909 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.919+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/41024' 2026-02-24T22:34:25.909 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.919+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7954056344309016737', time=2026-02-25T00:15:34.652882+0000, chain.objs.size()=4 2026-02-24T22:34:25.909 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.919+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1512__multipart_foo.2~GzM7_FGELiRE827ZarTvcbTNPjj0TKX.1 2026-02-24T22:34:25.909 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.919+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1512__multipart_foo.2~GzM7_FGELiRE827ZarTvcbTNPjj0TKX.2 2026-02-24T22:34:25.909 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.919+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1512__multipart_foo.2~GzM7_FGELiRE827ZarTvcbTNPjj0TKX.3 2026-02-24T22:34:25.909 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.919+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1512__multipart_foo.2~GzM7_FGELiRE827ZarTvcbTNPjj0TKX.4 2026-02-24T22:34:25.909 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.919+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18294110152583415713', time=2026-02-25T00:15:34.745541+0000, chain.objs.size()=3 2026-02-24T22:34:25.909 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.919+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-FROZEN.2~5YnNWY2srlSZqsTYGvvi4fm2BZ8_YNh.1 2026-02-24T22:34:25.909 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.919+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-FROZEN.2~5YnNWY2srlSZqsTYGvvi4fm2BZ8_YNh.2 2026-02-24T22:34:25.909 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.919+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1517__multipart_foo-FROZEN.2~5YnNWY2srlSZqsTYGvvi4fm2BZ8_YNh.3 2026-02-24T22:34:25.909 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.919+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13500111645097273989', time=2026-02-25T00:16:32.616404+0000, chain.objs.size()=1 2026-02-24T22:34:25.909 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.919+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520__shadow_.e9sKNX2NbT4oFEqf0VCsTnKoHkYvSgM_0 2026-02-24T22:34:25.910 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.919+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4137491699542966876', time=2026-02-25T00:16:32.617403+0000, chain.objs.size()=1 2026-02-24T22:34:25.910 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.919+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520__shadow_.TPYtfASYAMqV49vH_q4e2Vi3SKvUAan_0 2026-02-24T22:34:25.910 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.919+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8381418688429241205', time=2026-02-25T00:17:41.431833+0000, chain.objs.size()=1 2026-02-24T22:34:25.910 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.919+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1526__shadow_.6LGf927z7N43_RD8h0bALuP1nv7quPC_0 2026-02-24T22:34:25.910 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.919+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10929298741640057441', time=2026-02-25T00:20:32.521477+0000, chain.objs.size()=1 2026-02-24T22:34:25.910 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.919+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1538__shadow_.HmAlB44Y_viY7aQda__HGW49gWSv2EW_0 2026-02-24T22:34:25.910 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.919+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3357650495985729523', time=2026-02-25T00:23:14.372176+0000, chain.objs.size()=1 2026-02-24T22:34:25.910 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.919+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1518__shadow_.-rTGucEqt8IeKoH0FLKKl0qBZesxTya_0 2026-02-24T22:34:25.910 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7763659742267039624', time=2026-02-25T00:23:53.629198+0000, chain.objs.size()=1 2026-02-24T22:34:25.910 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1545__shadow_.4NTjaqt7kr2ofVYmHe3-QwICBcTE5jo_0 2026-02-24T22:34:25.910 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7887801793474021771', time=2026-02-25T00:25:33.717516+0000, chain.objs.size()=1 2026-02-24T22:34:25.910 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1597__shadow_.xeO0zLOYM_6nOMfU0g8AjQODXNtZ76p_0 2026-02-24T22:34:25.911 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3395524234887245539', time=2026-02-25T00:25:34.889957+0000, chain.objs.size()=1 2026-02-24T22:34:25.911 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1599__shadow_.M7Qjl-8Y75JaHoOCPxML0PU5O1vNJgY_0 2026-02-24T22:34:25.911 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2531208723723075467', time=2026-02-25T00:25:44.442038+0000, chain.objs.size()=1 2026-02-24T22:34:25.911 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1613__shadow_.cm6YFE40J8aJB5iWK_5fetMuGuTxFcc_0 2026-02-24T22:34:25.911 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7656177935150426247', time=2026-02-25T00:25:47.958440+0000, chain.objs.size()=1 2026-02-24T22:34:25.911 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1619__shadow_.shg7ip8inUnHUTMImfXefd87SrJBEj__0 2026-02-24T22:34:25.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4542529825110194379', time=2026-02-25T00:25:51.409670+0000, chain.objs.size()=1 2026-02-24T22:34:25.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1625__shadow_.jltVjpqpca_NydK4fCyZgMb90fRTLQ7_0 2026-02-24T22:34:25.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11828427614579976993', time=2026-02-25T00:25:59.092841+0000, chain.objs.size()=1 2026-02-24T22:34:25.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.GafPHVv3oYTWxduKabnbeXjPq3FPfqn_0 2026-02-24T22:34:25.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1611429035450735529', time=2026-02-25T00:25:59.180090+0000, chain.objs.size()=1 2026-02-24T22:34:25.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_._08ahV6pOyx4DyZdy_wDTH8mvfVzkBu_0 2026-02-24T22:34:25.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17681284019257058546', time=2026-02-25T00:25:59.282513+0000, chain.objs.size()=1 2026-02-24T22:34:25.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.uvT12vTatfNOzPXoxHw6qBRoVdX84u9_0 2026-02-24T22:34:25.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9035257682329491528', time=2026-02-25T00:25:59.292058+0000, chain.objs.size()=1 2026-02-24T22:34:25.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.7BQJPiZ_lWNbF_dcdplwItcOwn6OkHC_0 2026-02-24T22:34:25.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2704598088551454458', time=2026-02-25T00:25:59.304447+0000, chain.objs.size()=1 2026-02-24T22:34:25.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.jAVoVaJfgWlWf352HL0p5Gt50fPUaV4_0 2026-02-24T22:34:25.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12777085097930664565', time=2026-02-25T00:25:59.341838+0000, chain.objs.size()=1 2026-02-24T22:34:25.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.8v-70jTzkSVCzBT1yGWwhvCCkRU8XO0_0 2026-02-24T22:34:25.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14820865784631902854', time=2026-02-25T00:25:59.440455+0000, chain.objs.size()=1 2026-02-24T22:34:25.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.KaXZ9tLxZfllab-t8Oti93ehTBD0UHD_0 2026-02-24T22:34:25.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6104607659457635543', time=2026-02-25T00:25:59.442339+0000, chain.objs.size()=1 2026-02-24T22:34:25.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.nltHVT4CV8nJ7mCbJbw8Bk702lPgrJo_0 2026-02-24T22:34:25.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1585854018759304065', time=2026-02-25T00:25:59.446469+0000, chain.objs.size()=1 2026-02-24T22:34:25.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.nJC2wJ70NQa0EsFl8DnOLj_LB_iKL0m_0 2026-02-24T22:34:25.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8895966941733559586', time=2026-02-25T00:25:59.449710+0000, chain.objs.size()=1 2026-02-24T22:34:25.912 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.tqMto88cndIncteAEOzDgnK2ECo0I5y_0 2026-02-24T22:34:25.913 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1495850966940785820', time=2026-02-25T00:25:59.452327+0000, chain.objs.size()=1 2026-02-24T22:34:25.913 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.viFYuM2aTlAkjL0YFAAtxuy--biVkYX_0 2026-02-24T22:34:25.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4611244150881712688', time=2026-02-25T00:25:59.503605+0000, chain.objs.size()=1 2026-02-24T22:34:25.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.2goKsDTc_KkLWbOFCNPMXbc8NSSa530_0 2026-02-24T22:34:25.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15588112611001672527', time=2026-02-25T00:25:59.582986+0000, chain.objs.size()=1 2026-02-24T22:34:25.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.GZh_Rvl8KA8vY36INO2YfD0VeKUK2Ja_0 2026-02-24T22:34:25.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10786966210534796484', time=2026-02-25T00:25:59.599855+0000, chain.objs.size()=1 2026-02-24T22:34:25.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ZsGTN1LUrzcGkcvnbH0tZAXq3ZaipKH_0 2026-02-24T22:34:25.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6501185148639647586', time=2026-02-25T00:25:59.639057+0000, chain.objs.size()=1 2026-02-24T22:34:25.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.0DWtsIUlGflCHndJZskEHj6mjYjvG7m_0 2026-02-24T22:34:25.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13870600489166974682', time=2026-02-25T00:25:59.665145+0000, chain.objs.size()=1 2026-02-24T22:34:25.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.0G0Ayb8CMRYl2BokXrAj35OGHKSzF7p_0 2026-02-24T22:34:25.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.60674839995158391', time=2026-02-25T00:25:59.757116+0000, chain.objs.size()=1 2026-02-24T22:34:25.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.TwyFQ2VreozFApG5nRdgT3-oRysFRla_0 2026-02-24T22:34:25.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16213933800002551273', time=2026-02-25T00:25:59.761195+0000, chain.objs.size()=1 2026-02-24T22:34:25.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.wfwMBm9ChSs4pFad23Gwsq5CAG-trpd_0 2026-02-24T22:34:25.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6423932666011544617', time=2026-02-25T00:25:59.774526+0000, chain.objs.size()=1 2026-02-24T22:34:25.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.k1P2_uws-muPgJdlZdcJKmjF_NrdbQb_0 2026-02-24T22:34:25.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4315496212497664356', time=2026-02-25T00:25:59.777918+0000, chain.objs.size()=1 2026-02-24T22:34:25.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.s9b2juK3vPgwcQc8QY0-JObFhWdEIUS_0 2026-02-24T22:34:25.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4992465697553556360', time=2026-02-25T00:25:59.793809+0000, chain.objs.size()=1 2026-02-24T22:34:25.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.923+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.E1gIOixxJ0RkomFzX6NEYWQ4cMVlKEb_0 2026-02-24T22:34:25.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11059725339105606617', time=2026-02-25T00:25:59.817591+0000, chain.objs.size()=1 2026-02-24T22:34:25.914 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.bCFX5r0T1hKQ8uAcVE6hT3DMsMdRslo_0 2026-02-24T22:34:25.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15228575871065429624', time=2026-02-25T00:25:59.908199+0000, chain.objs.size()=1 2026-02-24T22:34:25.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.isAZjs710qKMbF1-TuWNZLDDiXEj4eI_0 2026-02-24T22:34:25.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16322923281287281860', time=2026-02-25T00:25:59.984473+0000, chain.objs.size()=1 2026-02-24T22:34:25.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.4qOmlA7Nu6w32KfQhq9nUnKDLt8hbnf_0 2026-02-24T22:34:25.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8023850257213316718', time=2026-02-25T00:25:59.990324+0000, chain.objs.size()=1 2026-02-24T22:34:25.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.eqnjAtn9aHG29zNKHbk8CXraGqxOV3k_0 2026-02-24T22:34:25.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15413975626103054769', time=2026-02-25T00:26:00.246125+0000, chain.objs.size()=1 2026-02-24T22:34:25.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.CGlYy-EPHBaDvW34x77C2temNf7mp7q_0 2026-02-24T22:34:25.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15544452505144210487', time=2026-02-25T00:26:02.513485+0000, chain.objs.size()=1 2026-02-24T22:34:25.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1632__shadow_.vtSPLtQdQbgdmeORKWTDeyL6d00LM6o_0 2026-02-24T22:34:25.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3180676091753393401', time=2026-02-25T00:26:16.025665+0000, chain.objs.size()=1 2026-02-24T22:34:25.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1655__shadow_.QV1K-grz29hm_x6AlBQMCBunFvXAzFw_0 2026-02-24T22:34:25.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13737200020188906715', time=2026-02-25T00:26:18.512275+0000, chain.objs.size()=1 2026-02-24T22:34:25.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1658__shadow_.LQJdvXDWVZT1wH3VcrBWR0uR0UpksbW_0 2026-02-24T22:34:25.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15995847949893026124', time=2026-02-25T00:26:19.461135+0000, chain.objs.size()=1 2026-02-24T22:34:25.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1659__shadow_.CSdbnF5iyeTikUQPVU9b3i4Xj8SR4wJ_0 2026-02-24T22:34:25.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16417610031594078807', time=2026-02-25T00:26:19.463806+0000, chain.objs.size()=1 2026-02-24T22:34:25.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1659__shadow_.wcUU9XqoqK786Fk9vb7V8Z36bImkF9w_0 2026-02-24T22:34:25.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14215008771316052309', time=2026-02-25T00:26:19.464296+0000, chain.objs.size()=1 2026-02-24T22:34:25.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1659__shadow_.Gbdc62lZMMi-rhmwUIYQ8DdCeNhp6oe_0 2026-02-24T22:34:25.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3726846716582676425', time=2026-02-25T00:26:31.945217+0000, chain.objs.size()=1 2026-02-24T22:34:25.915 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1678__shadow_.8nuezlNBGy4zmdAKBHvx2acAMiLyRPU_0 2026-02-24T22:34:25.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17369277458022377404', time=2026-02-25T00:26:35.805248+0000, chain.objs.size()=1 2026-02-24T22:34:25.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:aK2LFLumB248mWhTiJBvIpIF3NH8VeH_.R99vwjocYE4ZyEPpJc_f7ez_KjxUUdF_0 2026-02-24T22:34:25.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17369277458022377404', time=2026-02-25T00:26:35.810065+0000, chain.objs.size()=1 2026-02-24T22:34:25.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:aK2LFLumB248mWhTiJBvIpIF3NH8VeH_.R99vwjocYE4ZyEPpJc_f7ez_KjxUUdF_0 2026-02-24T22:34:25.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17369277458022377404', time=2026-02-25T00:26:35.813985+0000, chain.objs.size()=1 2026-02-24T22:34:25.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:aK2LFLumB248mWhTiJBvIpIF3NH8VeH_.R99vwjocYE4ZyEPpJc_f7ez_KjxUUdF_0 2026-02-24T22:34:25.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17369277458022377404', time=2026-02-25T00:26:35.815495+0000, chain.objs.size()=1 2026-02-24T22:34:25.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:aK2LFLumB248mWhTiJBvIpIF3NH8VeH_.R99vwjocYE4ZyEPpJc_f7ez_KjxUUdF_0 2026-02-24T22:34:25.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17369277458022377404', time=2026-02-25T00:26:35.821246+0000, chain.objs.size()=1 2026-02-24T22:34:25.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:aK2LFLumB248mWhTiJBvIpIF3NH8VeH_.R99vwjocYE4ZyEPpJc_f7ez_KjxUUdF_0 2026-02-24T22:34:25.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17369277458022377404', time=2026-02-25T00:26:35.827573+0000, chain.objs.size()=1 2026-02-24T22:34:25.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:aK2LFLumB248mWhTiJBvIpIF3NH8VeH_.R99vwjocYE4ZyEPpJc_f7ez_KjxUUdF_0 2026-02-24T22:34:25.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17369277458022377404', time=2026-02-25T00:26:35.834167+0000, chain.objs.size()=1 2026-02-24T22:34:25.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:aK2LFLumB248mWhTiJBvIpIF3NH8VeH_.R99vwjocYE4ZyEPpJc_f7ez_KjxUUdF_0 2026-02-24T22:34:25.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17369277458022377404', time=2026-02-25T00:26:35.838034+0000, chain.objs.size()=1 2026-02-24T22:34:25.916 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:aK2LFLumB248mWhTiJBvIpIF3NH8VeH_.R99vwjocYE4ZyEPpJc_f7ez_KjxUUdF_0 2026-02-24T22:34:25.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17369277458022377404', time=2026-02-25T00:26:35.840252+0000, chain.objs.size()=1 2026-02-24T22:34:25.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:aK2LFLumB248mWhTiJBvIpIF3NH8VeH_.R99vwjocYE4ZyEPpJc_f7ez_KjxUUdF_0 2026-02-24T22:34:25.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17369277458022377404', time=2026-02-25T00:26:35.845057+0000, chain.objs.size()=1 2026-02-24T22:34:25.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:aK2LFLumB248mWhTiJBvIpIF3NH8VeH_.R99vwjocYE4ZyEPpJc_f7ez_KjxUUdF_0 2026-02-24T22:34:25.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17369277458022377404', time=2026-02-25T00:26:35.849328+0000, chain.objs.size()=1 2026-02-24T22:34:25.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:aK2LFLumB248mWhTiJBvIpIF3NH8VeH_.R99vwjocYE4ZyEPpJc_f7ez_KjxUUdF_0 2026-02-24T22:34:25.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17369277458022377404', time=2026-02-25T00:26:35.851630+0000, chain.objs.size()=1 2026-02-24T22:34:25.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:aK2LFLumB248mWhTiJBvIpIF3NH8VeH_.R99vwjocYE4ZyEPpJc_f7ez_KjxUUdF_0 2026-02-24T22:34:25.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17369277458022377404', time=2026-02-25T00:26:35.856249+0000, chain.objs.size()=1 2026-02-24T22:34:25.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:aK2LFLumB248mWhTiJBvIpIF3NH8VeH_.R99vwjocYE4ZyEPpJc_f7ez_KjxUUdF_0 2026-02-24T22:34:25.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17369277458022377404', time=2026-02-25T00:26:35.860555+0000, chain.objs.size()=1 2026-02-24T22:34:25.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:aK2LFLumB248mWhTiJBvIpIF3NH8VeH_.R99vwjocYE4ZyEPpJc_f7ez_KjxUUdF_0 2026-02-24T22:34:25.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17369277458022377404', time=2026-02-25T00:26:35.865426+0000, chain.objs.size()=1 2026-02-24T22:34:25.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:aK2LFLumB248mWhTiJBvIpIF3NH8VeH_.R99vwjocYE4ZyEPpJc_f7ez_KjxUUdF_0 2026-02-24T22:34:25.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17369277458022377404', time=2026-02-25T00:26:35.867250+0000, chain.objs.size()=1 2026-02-24T22:34:25.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:aK2LFLumB248mWhTiJBvIpIF3NH8VeH_.R99vwjocYE4ZyEPpJc_f7ez_KjxUUdF_0 2026-02-24T22:34:25.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17369277458022377404', time=2026-02-25T00:26:35.871064+0000, chain.objs.size()=1 2026-02-24T22:34:25.917 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:aK2LFLumB248mWhTiJBvIpIF3NH8VeH_.R99vwjocYE4ZyEPpJc_f7ez_KjxUUdF_0 2026-02-24T22:34:25.918 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17369277458022377404', time=2026-02-25T00:26:35.875085+0000, chain.objs.size()=1 2026-02-24T22:34:25.918 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:aK2LFLumB248mWhTiJBvIpIF3NH8VeH_.R99vwjocYE4ZyEPpJc_f7ez_KjxUUdF_0 2026-02-24T22:34:25.918 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17369277458022377404', time=2026-02-25T00:26:35.880065+0000, chain.objs.size()=1 2026-02-24T22:34:25.918 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.927+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:aK2LFLumB248mWhTiJBvIpIF3NH8VeH_.R99vwjocYE4ZyEPpJc_f7ez_KjxUUdF_0 2026-02-24T22:34:25.918 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17369277458022377404', time=2026-02-25T00:26:35.882611+0000, chain.objs.size()=1 2026-02-24T22:34:25.918 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:aK2LFLumB248mWhTiJBvIpIF3NH8VeH_.R99vwjocYE4ZyEPpJc_f7ez_KjxUUdF_0 2026-02-24T22:34:25.918 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17369277458022377404', time=2026-02-25T00:26:35.892063+0000, chain.objs.size()=1 2026-02-24T22:34:25.918 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1684__shadow:aK2LFLumB248mWhTiJBvIpIF3NH8VeH_.R99vwjocYE4ZyEPpJc_f7ez_KjxUUdF_0 2026-02-24T22:34:25.918 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11431380537916391993', time=2026-02-25T00:26:44.800509+0000, chain.objs.size()=1 2026-02-24T22:34:25.919 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Q5KZKlz53_Zb8qWUvMPpjqfp1HgBOs0_0 2026-02-24T22:34:25.919 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11829070724393186868', time=2026-02-25T00:26:44.817876+0000, chain.objs.size()=1 2026-02-24T22:34:25.919 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ljor1d29rveV6AUGshhH1bnCMs8D-J6_0 2026-02-24T22:34:25.919 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17069483645964628606', time=2026-02-25T00:26:44.818677+0000, chain.objs.size()=1 2026-02-24T22:34:25.919 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.N2SnQoweBSWbQ0PhuQK5b9oGkVOGKLb_0 2026-02-24T22:34:25.920 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17811987502546031630', time=2026-02-25T00:26:44.835395+0000, chain.objs.size()=1 2026-02-24T22:34:25.920 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.oQbcLiUdi7JQuDryLx0Nab-j5MpOPWm_0 2026-02-24T22:34:25.920 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.249004474998489009', time=2026-02-25T00:26:44.838656+0000, chain.objs.size()=1 2026-02-24T22:34:25.920 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.mHPGdwEfmbPZCiL2qCrghG_SAsZ5u3C_0 2026-02-24T22:34:25.920 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10231271230068346739', time=2026-02-25T00:26:44.854866+0000, chain.objs.size()=1 2026-02-24T22:34:25.920 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.tEEHr0daPsWNL5_oVecvpd_l0yNk_su_0 2026-02-24T22:34:25.920 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13717907511778112383', time=2026-02-25T00:26:44.954033+0000, chain.objs.size()=1 2026-02-24T22:34:25.920 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.kTGNZUfygYmRk2gXdR8jKRDZ8cPa4KO_0 2026-02-24T22:34:25.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16682222842683493631', time=2026-02-25T00:26:45.002429+0000, chain.objs.size()=1 2026-02-24T22:34:25.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.nwerDYsJbn3xQxoHdCl_oqDX1Sf7JwE_0 2026-02-24T22:34:25.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2130105156143567175', time=2026-02-25T00:26:45.016552+0000, chain.objs.size()=1 2026-02-24T22:34:25.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.oB6zqTMCwjw0dbSdEIJhHNzI-bTQcEe_0 2026-02-24T22:34:25.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14802970845893499969', time=2026-02-25T00:26:45.029723+0000, chain.objs.size()=1 2026-02-24T22:34:25.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.lQQQVFbiDKyAAvUaGkNzoyuCchB7gsZ_0 2026-02-24T22:34:25.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1698811451663798013', time=2026-02-25T00:26:45.031132+0000, chain.objs.size()=1 2026-02-24T22:34:25.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.HJT-MTf-wyECJjN8j_ZVd9EIeeFVJqk_0 2026-02-24T22:34:25.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16874283875689171495', time=2026-02-25T00:26:45.169469+0000, chain.objs.size()=1 2026-02-24T22:34:25.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.cu8zLYkioODjfZczwoMlZa0AUVX-mav_0 2026-02-24T22:34:25.921 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16056365760360043443', time=2026-02-25T00:26:45.195557+0000, chain.objs.size()=1 2026-02-24T22:34:25.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Q50xp6X-UTjGaUVMOhMP47bnUg3nDlu_0 2026-02-24T22:34:25.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17715433623002759362', time=2026-02-25T00:26:45.270354+0000, chain.objs.size()=1 2026-02-24T22:34:25.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.plrpFHydV27ntNTafKeEYjrCrDP6ZLt_0 2026-02-24T22:34:25.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11251559299708368361', time=2026-02-25T00:26:45.270754+0000, chain.objs.size()=1 2026-02-24T22:34:25.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.YpR5zT1nTjMeTNPzEuwRjA9IIr2MaWp_0 2026-02-24T22:34:25.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7627873333713040167', time=2026-02-25T00:26:45.292419+0000, chain.objs.size()=1 2026-02-24T22:34:25.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.HV-vTMn6HGxEzFSQs4icor33LqeIzR3_0 2026-02-24T22:34:25.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15214196334733794439', time=2026-02-25T00:26:45.293760+0000, chain.objs.size()=1 2026-02-24T22:34:25.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.p6m-4rpYyGPo9Ce-a85e2QGjuvvG_sD_0 2026-02-24T22:34:25.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4917472439089671970', time=2026-02-25T00:26:45.304982+0000, chain.objs.size()=1 2026-02-24T22:34:25.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.hoEaj4Ur3vJFuffTxVmZsH5B28bfkIi_0 2026-02-24T22:34:25.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4937339053829103575', time=2026-02-25T00:26:45.413939+0000, chain.objs.size()=1 2026-02-24T22:34:25.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.mnw1BXfRgoR-sHZuccPpsFmZ72Wl6oe_0 2026-02-24T22:34:25.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16709294705845758279', time=2026-02-25T00:26:45.414334+0000, chain.objs.size()=1 2026-02-24T22:34:25.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.a9_zO5T8JkH05XKlZZxmaSFmn8Gske8_0 2026-02-24T22:34:25.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5275468871440913875', time=2026-02-25T00:26:45.432690+0000, chain.objs.size()=1 2026-02-24T22:34:25.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.931+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.zUP8B0ERNm6C4Y2-qKwDMt5Z9mbeRUj_0 2026-02-24T22:34:25.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.935+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11538609820454720294', time=2026-02-25T00:26:45.460928+0000, chain.objs.size()=1 2026-02-24T22:34:25.922 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.935+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Q2SkLTWXXc_cW-suvCINcXcgenMgubc_0 2026-02-24T22:34:25.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.935+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3012031708633737179', time=2026-02-25T00:26:45.575014+0000, chain.objs.size()=1 2026-02-24T22:34:25.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.935+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.JjhK-JBVRCfw53DQ3Z6RCXw5yorcJaC_0 2026-02-24T22:34:25.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.935+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5175878325324393450', time=2026-02-25T00:26:45.590936+0000, chain.objs.size()=1 2026-02-24T22:34:25.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.935+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Ju6q2dqZUhKsj_ufT79kECn4_GPLtdM_0 2026-02-24T22:34:25.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.935+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14173814835651462646', time=2026-02-25T00:26:45.820386+0000, chain.objs.size()=1 2026-02-24T22:34:25.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.935+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.tgbDeGMhhCzDrINGJVhKwAuiUVB-kfp_0 2026-02-24T22:34:25.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.935+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1469729641166580715', time=2026-02-25T00:26:45.823613+0000, chain.objs.size()=1 2026-02-24T22:34:25.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.935+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.sXswkezm_s2nlzu4riWL5eVshS3Py2N_0 2026-02-24T22:34:25.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.935+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5311861363314870493', time=2026-02-25T00:26:45.830288+0000, chain.objs.size()=1 2026-02-24T22:34:25.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.935+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.P5NDnzvq3uMQCZK9VdVaP0xmUNKunIO_0 2026-02-24T22:34:25.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.935+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10956670187558033754', time=2026-02-25T00:26:45.835845+0000, chain.objs.size()=1 2026-02-24T22:34:25.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.935+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.HWou-K9dXr1oPFisop2OXPAYBlwthx__0 2026-02-24T22:34:25.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.935+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7370085545353735665', time=2026-02-25T00:26:45.844061+0000, chain.objs.size()=1 2026-02-24T22:34:25.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.935+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.M1k4CH8cxqHTWWeWh9_Et1iNFTMomPq_0 2026-02-24T22:34:25.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.935+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15931856181630500358', time=2026-02-25T00:26:45.844584+0000, chain.objs.size()=1 2026-02-24T22:34:25.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.935+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.UwGiq1TSJKko3K7PQIzVGSjZMkobiJD_0 2026-02-24T22:34:25.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.935+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6271682505363624984', time=2026-02-25T00:26:45.845926+0000, chain.objs.size()=1 2026-02-24T22:34:25.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.935+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.mFMX0B7L0xR6Xk3qxn1ULVoGPOTflaz_0 2026-02-24T22:34:25.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.935+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5054962002335540674', time=2026-02-25T00:26:45.850696+0000, chain.objs.size()=1 2026-02-24T22:34:25.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.935+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.DG8gYmZgxPKtXM8_BJlYD1R81rKvPYH_0 2026-02-24T22:34:25.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.935+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.950177643086438524', time=2026-02-25T00:26:45.926931+0000, chain.objs.size()=1 2026-02-24T22:34:25.924 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.935+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.2uwnpBKXX5qamzB2DOJbcm01_C_W89g_0 2026-02-24T22:34:25.925 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.935+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/41024 2026-02-24T22:34:25.927 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=55, truncated=0, next_marker='' 2026-02-24T22:34:25.927 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7110978124408496798', time=2026-02-25T00:26:45.934752+0000, chain.objs.size()=1 2026-02-24T22:34:25.927 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.7hiSZMkGrFgHz8JjJ-YiI1DmdFM8aYH_0 2026-02-24T22:34:25.927 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11444064921142862615', time=2026-02-25T00:26:45.935198+0000, chain.objs.size()=1 2026-02-24T22:34:25.927 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.QdrjpPV8k6QkwHsos4cTTpOH-eGZdcE_0 2026-02-24T22:34:25.927 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18248370783099730106', time=2026-02-25T00:26:45.971714+0000, chain.objs.size()=1 2026-02-24T22:34:25.927 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.hrdBZRUdVZk2RxWWvvhJvRmj81Jafzn_0 2026-02-24T22:34:25.927 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6910426896821351576', time=2026-02-25T00:26:46.001192+0000, chain.objs.size()=1 2026-02-24T22:34:25.927 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.fft66ZQ7xdyhBKALh3zrFcsVE9wqGtp_0 2026-02-24T22:34:25.927 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.827792968810263421', time=2026-02-25T00:26:46.002832+0000, chain.objs.size()=1 2026-02-24T22:34:25.927 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.5J2KDbmdaeNa_9BXpLqdM23_vunSRKF_0 2026-02-24T22:34:25.927 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3085891267248408397', time=2026-02-25T00:26:46.018926+0000, chain.objs.size()=1 2026-02-24T22:34:25.927 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.aRHERR7b90psuCz6No24UJGk-jw22Nh_0 2026-02-24T22:34:25.928 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13971412217858662140', time=2026-02-25T00:26:46.097010+0000, chain.objs.size()=1 2026-02-24T22:34:25.928 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.kJ6cXA0bzg4e0hQb6DIX_VBYzWBFBN3_0 2026-02-24T22:34:25.928 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4470972266721265047', time=2026-02-25T00:26:46.146727+0000, chain.objs.size()=1 2026-02-24T22:34:25.928 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.VjTE2lNwNt77cAYY6ZacyQMwMQPnGyt_0 2026-02-24T22:34:25.928 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13109420029452214984', time=2026-02-25T00:26:53.174465+0000, chain.objs.size()=1 2026-02-24T22:34:25.928 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.XcLxgXVZuGVu39b7XArd_VjVXt8_MPM_0 2026-02-24T22:34:25.928 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12768514369934709254', time=2026-02-25T00:26:53.194662+0000, chain.objs.size()=1 2026-02-24T22:34:25.928 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ru6YrgyRIZ8NR0ypBReidnpAc7gnx5O_0 2026-02-24T22:34:25.928 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1015536894540026013', time=2026-02-25T00:26:53.208254+0000, chain.objs.size()=1 2026-02-24T22:34:25.928 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Met55U89XD7Q2d_rDLhBOvxMwFLZF6C_0 2026-02-24T22:34:25.928 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15644684541858599225', time=2026-02-25T00:26:53.226537+0000, chain.objs.size()=1 2026-02-24T22:34:25.928 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.2IsiEVY8VM77rRY2TT_apayhCc8W9QK_0 2026-02-24T22:34:25.928 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5494540074215842603', time=2026-02-25T00:26:53.236372+0000, chain.objs.size()=1 2026-02-24T22:34:25.928 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.mqlXfRTBc2nw2dj7csNAfultYRUrWnS_0 2026-02-24T22:34:25.928 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5504276641552790684', time=2026-02-25T00:26:53.241724+0000, chain.objs.size()=1 2026-02-24T22:34:25.928 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.vl5OpZVSr8ygBOUrrMw0fA_V3qWMiDN_0 2026-02-24T22:34:25.928 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10691644787944086305', time=2026-02-25T00:26:53.253807+0000, chain.objs.size()=1 2026-02-24T22:34:25.928 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.cTkvkkcpGOXkviy95fAq5PnrII-EQdb_0 2026-02-24T22:34:25.928 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12023879363968620924', time=2026-02-25T00:26:53.337642+0000, chain.objs.size()=1 2026-02-24T22:34:25.928 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.twe7c-avyYoqXYAj8VvYX-TT6QtfWwz_0 2026-02-24T22:34:25.928 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8742375525317644331', time=2026-02-25T00:26:53.342518+0000, chain.objs.size()=1 2026-02-24T22:34:25.928 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.B6SYytPZRWrO_ajzNDiOinp89XFN_Ym_0 2026-02-24T22:34:25.929 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18386579454052323096', time=2026-02-25T00:26:53.351754+0000, chain.objs.size()=1 2026-02-24T22:34:25.929 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.VXbs6gkw83wD0k0khdzoEQam84oyUO__0 2026-02-24T22:34:25.929 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17663601330873428369', time=2026-02-25T00:26:53.371324+0000, chain.objs.size()=1 2026-02-24T22:34:25.929 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.-7ZjNJLPxsxrs4djK2hFiqGbnSE9M2E_0 2026-02-24T22:34:25.929 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9036027466751922514', time=2026-02-25T00:26:53.435398+0000, chain.objs.size()=1 2026-02-24T22:34:25.929 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Tl9pJI93_ddkN0KkfuuI0SV3QnK1CDn_0 2026-02-24T22:34:25.929 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.359704766814203788', time=2026-02-25T00:26:53.524922+0000, chain.objs.size()=1 2026-02-24T22:34:25.929 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.MGph000KmwbTAQOPY8gUypMxaEAvQHl_0 2026-02-24T22:34:25.929 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3582089225818236161', time=2026-02-25T00:26:53.526603+0000, chain.objs.size()=1 2026-02-24T22:34:25.929 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.X7S_Co9lcRLTq3H8gn15KFV1sFetawN_0 2026-02-24T22:34:25.929 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15363180961095171087', time=2026-02-25T00:26:53.540327+0000, chain.objs.size()=1 2026-02-24T22:34:25.929 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.xEL4BA4FRTwopFV1l8e52ee80VqcOX4_0 2026-02-24T22:34:25.929 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.946822974646093321', time=2026-02-25T00:26:53.545247+0000, chain.objs.size()=1 2026-02-24T22:34:25.929 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.LDoFOTU_cweCTQh9SUc2oLW-JsCF5a1_0 2026-02-24T22:34:25.929 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17148815770179034870', time=2026-02-25T00:26:53.562813+0000, chain.objs.size()=1 2026-02-24T22:34:25.929 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.A1G7QvIOGeyeCKMuXTFFhXeJcajXZZw_0 2026-02-24T22:34:25.929 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1122541466925936494', time=2026-02-25T00:26:53.787860+0000, chain.objs.size()=1 2026-02-24T22:34:25.929 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.1JMXB8iWfyQ-1jhCYRN7qUUozfpQ62x_0 2026-02-24T22:34:25.929 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.87908983324171696', time=2026-02-25T00:26:53.953845+0000, chain.objs.size()=1 2026-02-24T22:34:25.929 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.92vX4sXOctxq8w3-vhj9NZI_W6myuju_0 2026-02-24T22:34:25.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2987020623413796674', time=2026-02-25T00:26:53.996994+0000, chain.objs.size()=1 2026-02-24T22:34:25.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.V84s1bOJ630aUABMBCIw9coj6R3opoW_0 2026-02-24T22:34:25.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506029779783556068', time=2026-02-25T00:26:54.001684+0000, chain.objs.size()=1 2026-02-24T22:34:25.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Y6GMEIFYjdZ5CJu4ev_3ueR7bh0soPL_0 2026-02-24T22:34:25.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10192304185801138481', time=2026-02-25T00:26:54.026395+0000, chain.objs.size()=1 2026-02-24T22:34:25.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.BQnFECtc5bqIHw8W8c-zEaRQ0fVZTbX_0 2026-02-24T22:34:25.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5529674809434417145', time=2026-02-25T00:26:54.116492+0000, chain.objs.size()=1 2026-02-24T22:34:25.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.nSqc6EYk8fuxa5AsqNR9jl1TZv8vzkQ_0 2026-02-24T22:34:25.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15046790874690547483', time=2026-02-25T00:26:54.120620+0000, chain.objs.size()=1 2026-02-24T22:34:25.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ffjVALQrLZscFb_ausne81YIJnJtWei_0 2026-02-24T22:34:25.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3984914727981091991', time=2026-02-25T00:26:54.141244+0000, chain.objs.size()=1 2026-02-24T22:34:25.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.s1eQTv2fwERS1kjhP6b9LjSb1EmE2Er_0 2026-02-24T22:34:25.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9349258406042461560', time=2026-02-25T00:26:54.174824+0000, chain.objs.size()=1 2026-02-24T22:34:25.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.939+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.agXGsHsXAE3qLTDnsmDkGx3QErcpcyV_0 2026-02-24T22:34:25.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12384131775610880119', time=2026-02-25T00:26:54.197115+0000, chain.objs.size()=1 2026-02-24T22:34:25.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.DGPrOhgOF_tzDVnAr8c5VT28d5EA_Sc_0 2026-02-24T22:34:25.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6134312645940523759', time=2026-02-25T00:26:54.363984+0000, chain.objs.size()=1 2026-02-24T22:34:25.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.cve5yIBgJgOul_NBiV_ngaXoHpzRi2i_0 2026-02-24T22:34:25.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17886536088828598769', time=2026-02-25T00:26:54.383391+0000, chain.objs.size()=1 2026-02-24T22:34:25.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.SI2bh11MgLpBjvUOaTz5fCXD5H2rHGk_0 2026-02-24T22:34:25.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16034760381076214139', time=2026-02-25T00:26:54.519812+0000, chain.objs.size()=1 2026-02-24T22:34:25.930 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.AxPkyo8Y8Po5JoEW6mkGQkZ4d069-uK_0 2026-02-24T22:34:25.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1079682123184923159', time=2026-02-25T00:26:54.530059+0000, chain.objs.size()=1 2026-02-24T22:34:25.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.UkWsRcFIVSyYjP2kGq1BtNSnzAoYOjA_0 2026-02-24T22:34:25.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16895219975719270666', time=2026-02-25T00:26:54.532366+0000, chain.objs.size()=1 2026-02-24T22:34:25.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Xl2G_rdTMLLBCJqzMvnlfJj8e7ZQrU8_0 2026-02-24T22:34:25.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15448846747051004943', time=2026-02-25T00:26:54.537617+0000, chain.objs.size()=1 2026-02-24T22:34:25.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.HQFxBdlmtosua8_VUoNPIXrZbJgaXnv_0 2026-02-24T22:34:25.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16992252043483446962', time=2026-02-25T00:26:54.539662+0000, chain.objs.size()=1 2026-02-24T22:34:25.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.MZDwz3Q3_D5c98YoFqrnMijsd38czDh_0 2026-02-24T22:34:25.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3759562956118435190', time=2026-02-25T00:27:10.784893+0000, chain.objs.size()=1 2026-02-24T22:34:25.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1718__shadow_.LSOhKrZFCZi2kbx8QT6FbUrY9QA_y21_0 2026-02-24T22:34:25.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5924145248481827448', time=2026-02-25T00:28:51.769164+0000, chain.objs.size()=1 2026-02-24T22:34:25.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1884__multipart_mymultipart.2~mjQpsXfiwZ1hCKofZAjXbxhFKw19lUH.1 2026-02-24T22:34:25.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10100062304630220604', time=2026-02-25T00:30:03.641788+0000, chain.objs.size()=1 2026-02-24T22:34:25.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1920__shadow_.7x4D8myCVgTZgyzer7EiaZuN3gVG8po_0 2026-02-24T22:34:25.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8816302981454138050', time=2026-02-25T00:30:09.984583+0000, chain.objs.size()=1 2026-02-24T22:34:25.931 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1928__shadow_.ahgAgVcX4nVIFvNNSVE1O8an-Eg04Yx_0 2026-02-24T22:34:25.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8999984854241770468', time=2026-02-25T00:30:15.260768+0000, chain.objs.size()=1 2026-02-24T22:34:25.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1933__shadow_.Uk3ccUtPc353M3tcBWp8WFi5fBjrAZT_0 2026-02-24T22:34:25.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13131556814165674109', time=2026-02-25T00:30:25.538834+0000, chain.objs.size()=1 2026-02-24T22:34:25.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1940__shadow_.N1AnihGJ6QLACuBSql2w1raQdu8IVJj_0 2026-02-24T22:34:25.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16491793675944647226', time=2026-02-25T00:30:28.698821+0000, chain.objs.size()=1 2026-02-24T22:34:25.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1943__shadow:bizpW96b00U4JmdWlXVvUsdAPscfKw0_.hgx3Pu4voiISPDAfhKBOoyBFD8NLK0l_0 2026-02-24T22:34:25.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6223695707935709127', time=2026-02-25T00:30:29.378509+0000, chain.objs.size()=1 2026-02-24T22:34:25.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1943__shadow:PRJRzlw4CDTHBHjDzR3GGMT-yUfoSTh_.5XkPOjlBm1um1llGNlDtj6m1DlS13-B_0 2026-02-24T22:34:25.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15598905061361435678', time=2026-02-25T00:30:29.879019+0000, chain.objs.size()=1 2026-02-24T22:34:25.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1943__shadow:8ewOA.jKL9qBKxIAmtwzEUesKEEH3rJ_.Gybln8KIj8gDdPN00ubAiRJWoSbCnpi_0 2026-02-24T22:34:25.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10604475075208408918', time=2026-02-25T00:30:41.178446+0000, chain.objs.size()=1 2026-02-24T22:34:25.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:HAMaNqhl18dPHOJiOpfOoYm0Ey.K.2K_.gVmDvUIcH-aRLHKWE7eTZp1rttK4CyH_0 2026-02-24T22:34:25.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13885060960495699337', time=2026-02-25T00:30:52.494029+0000, chain.objs.size()=1 2026-02-24T22:34:25.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1950__shadow:Nykho.DgQ0mMgqLIfrSu1R5pC50D1bs_.2AcveQPCg_R1aoE4AlaLfrhlsOD0tB7_0 2026-02-24T22:34:25.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9876920596351378694', time=2026-02-25T00:30:53.833768+0000, chain.objs.size()=1 2026-02-24T22:34:25.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1953__shadow:PL22fKp.zNC3znH.whVaSxUGhxHPAAv_.7CRVHjierH0oULIZhnCJ7LazjhGSoZu_0 2026-02-24T22:34:25.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9924913466910575635', time=2026-02-25T00:33:39.389176+0000, chain.objs.size()=1 2026-02-24T22:34:25.932 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1961__shadow_.sxjsOZHZ9AACp4CKp4fzIPAxPR9pPNf_0 2026-02-24T22:34:25.933 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.943+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.935 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2026-02-24T22:34:25.935 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.936 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/38970' 2026-02-24T22:34:25.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11492922603019535792', time=2026-02-25T00:15:34.628612+0000, chain.objs.size()=1 2026-02-24T22:34:25.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1506__shadow_.oKPbSpKRnPnnn0nvf11HifYFmNwFH_N_0 2026-02-24T22:34:25.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7117079776643466823', time=2026-02-25T00:15:34.676687+0000, chain.objs.size()=4 2026-02-24T22:34:25.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1511__multipart_foo.2~NF5_B9ScYFVByojVZs4d1K2b1N7WHkE.1 2026-02-24T22:34:25.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1511__multipart_foo.2~NF5_B9ScYFVByojVZs4d1K2b1N7WHkE.2 2026-02-24T22:34:25.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1511__multipart_foo.2~NF5_B9ScYFVByojVZs4d1K2b1N7WHkE.3 2026-02-24T22:34:25.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1511__multipart_foo.2~NF5_B9ScYFVByojVZs4d1K2b1N7WHkE.4 2026-02-24T22:34:25.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11137753281999821045', time=2026-02-25T00:17:07.121898+0000, chain.objs.size()=1 2026-02-24T22:34:25.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523__shadow_.xpEuwdxPYn9t_SR2qGHrV5LOQNTh8og_0 2026-02-24T22:34:25.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1982692942498963875', time=2026-02-25T00:17:39.890317+0000, chain.objs.size()=1 2026-02-24T22:34:25.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525__shadow_.jpW1EI_2L7dshASnbR551tgAtqZsCF8_0 2026-02-24T22:34:25.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8160098498624753349', time=2026-02-25T00:18:16.134058+0000, chain.objs.size()=1 2026-02-24T22:34:25.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1529__shadow_.nzWTqeEzc5FVa6evM3u_zgcbL3Jf-_s_0 2026-02-24T22:34:25.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18049118254264103405', time=2026-02-25T00:18:48.200729+0000, chain.objs.size()=1 2026-02-24T22:34:25.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530__shadow_.XmIjOuMxCgtPYdrdzKo50PgGy_6yYZe_0 2026-02-24T22:34:25.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10261044146285547445', time=2026-02-25T00:18:49.779009+0000, chain.objs.size()=1 2026-02-24T22:34:25.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1531__shadow_.hWnW2JNoeLd0TAJd5BjCxL5uoxAvXXC_0 2026-02-24T22:34:25.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10317544110107276401', time=2026-02-25T00:19:25.028996+0000, chain.objs.size()=1 2026-02-24T22:34:25.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534__shadow_.VUTgAB49HcQQCTRepbJ9Vjcy-k_0nnH_0 2026-02-24T22:34:25.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11065884005330103339', time=2026-02-25T00:19:25.030379+0000, chain.objs.size()=1 2026-02-24T22:34:25.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534__shadow_.BylK2S0llMR5wExm8-wJlv0BTt-YDsY_0 2026-02-24T22:34:25.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5295186602121422637', time=2026-02-25T00:19:25.033786+0000, chain.objs.size()=1 2026-02-24T22:34:25.937 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534__shadow_.U10Gx6p6JhcCLp51Ia_ukigySELOOJW_0 2026-02-24T22:34:25.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6158912772588741854', time=2026-02-25T00:19:57.123913+0000, chain.objs.size()=1 2026-02-24T22:34:25.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1535__shadow_.1mgI-Xni1_035XXmI0r97Z_x0CHkEFT_0 2026-02-24T22:34:25.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.613651233065473333', time=2026-02-25T00:20:30.738410+0000, chain.objs.size()=1 2026-02-24T22:34:25.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537__shadow_.JcKr_lsmn7-0sqq8nVCDdixRneQ-BwX_0 2026-02-24T22:34:25.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16504827300727287879', time=2026-02-25T00:24:11.876431+0000, chain.objs.size()=1 2026-02-24T22:34:25.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1563__shadow_.XzAt17zuytn54nYjHG5s0mWV9DZ1F4W_0 2026-02-24T22:34:25.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8320103326358952416', time=2026-02-25T00:25:41.726870+0000, chain.objs.size()=1 2026-02-24T22:34:25.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1609__shadow_.TookPZxOFn5U3UYAJBJhwSFlDNI1zIz_0 2026-02-24T22:34:25.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9581485016203253730', time=2026-02-25T00:25:47.329708+0000, chain.objs.size()=1 2026-02-24T22:34:25.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1618__shadow_.2o8qtMuTUu4fu_ZysP0LDSkOrXbidLI_0 2026-02-24T22:34:25.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16463664838829337384', time=2026-02-25T00:25:49.713933+0000, chain.objs.size()=1 2026-02-24T22:34:25.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1622__shadow_.HHSRpsNW92wkZ1UacamcNZ-ZoJQk3Df_0 2026-02-24T22:34:25.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1641156781143424639', time=2026-02-25T00:25:59.089747+0000, chain.objs.size()=1 2026-02-24T22:34:25.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.2FkOoAfFzfZ4EmEqY0Lfr8TmWZN29uy_0 2026-02-24T22:34:25.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14686844292882817730', time=2026-02-25T00:25:59.098359+0000, chain.objs.size()=1 2026-02-24T22:34:25.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.72HULwDodYxHjzAq9EcpBq1-UQthKW1_0 2026-02-24T22:34:25.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9364183564256742232', time=2026-02-25T00:25:59.104472+0000, chain.objs.size()=1 2026-02-24T22:34:25.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.jKs5mhtCOzQxNwuP6wSTJgCLP-4R6Cn_0 2026-02-24T22:34:25.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8057925529525837845', time=2026-02-25T00:25:59.121299+0000, chain.objs.size()=1 2026-02-24T22:34:25.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.947+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.hpnSSqKb9rB81oOGub-vHnf2rlBYh6c_0 2026-02-24T22:34:25.938 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18145460484752712999', time=2026-02-25T00:25:59.135549+0000, chain.objs.size()=1 2026-02-24T22:34:25.939 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.7zguaOEBkjXo-B7lTPcA2kAaZiC4oxt_0 2026-02-24T22:34:25.939 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17001823437970553001', time=2026-02-25T00:25:59.141422+0000, chain.objs.size()=1 2026-02-24T22:34:25.939 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.RhiaAN2E4MykBwtq7eZyn-ro02t848E_0 2026-02-24T22:34:25.939 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6774815689944326477', time=2026-02-25T00:25:59.181121+0000, chain.objs.size()=1 2026-02-24T22:34:25.939 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.I1eT2TTj_fh7cab2WjjuAQ9gp_zk70m_0 2026-02-24T22:34:25.939 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3899986997128496046', time=2026-02-25T00:25:59.298909+0000, chain.objs.size()=1 2026-02-24T22:34:25.939 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.fiIWceN-bkbe6BAVIc3cZH1JT2o85aF_0 2026-02-24T22:34:25.939 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13163044629754721519', time=2026-02-25T00:25:59.305495+0000, chain.objs.size()=1 2026-02-24T22:34:25.939 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.PNXSGal-wX7zapO6NaZ0QeLWcVgwxS6_0 2026-02-24T22:34:25.939 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4029973264658627131', time=2026-02-25T00:25:59.426459+0000, chain.objs.size()=1 2026-02-24T22:34:25.939 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.CoBfL1rbCzg0I5JQqjEC5b4LbSdtsMW_0 2026-02-24T22:34:25.939 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10072002468851966105', time=2026-02-25T00:25:59.469970+0000, chain.objs.size()=1 2026-02-24T22:34:25.939 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ODPBJcAsQ514_sSl8_n-7rwY_8PfFgw_0 2026-02-24T22:34:25.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7499053689361550391', time=2026-02-25T00:25:59.488142+0000, chain.objs.size()=1 2026-02-24T22:34:25.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.2UcDf-QR7zFAtSpKtExg5OX9bMMo0Gb_0 2026-02-24T22:34:25.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7850326476036583551', time=2026-02-25T00:25:59.589329+0000, chain.objs.size()=1 2026-02-24T22:34:25.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.zhwU94zenJa64sGVX-ESBh1wGCP5P2t_0 2026-02-24T22:34:25.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13396038438747399263', time=2026-02-25T00:25:59.590855+0000, chain.objs.size()=1 2026-02-24T22:34:25.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.fpOPCILBdGcjhntIglKWfZGZdtGUDxQ_0 2026-02-24T22:34:25.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14288087273278431550', time=2026-02-25T00:25:59.612847+0000, chain.objs.size()=1 2026-02-24T22:34:25.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.KFVXSyLH70v6_bUlWHWU9VNvrFA3dbe_0 2026-02-24T22:34:25.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.638068457100109418', time=2026-02-25T00:25:59.613327+0000, chain.objs.size()=1 2026-02-24T22:34:25.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.qX12EbSWViXFIJIGIlnoLMls3GGpxX8_0 2026-02-24T22:34:25.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11880674004926692682', time=2026-02-25T00:25:59.622889+0000, chain.objs.size()=1 2026-02-24T22:34:25.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.YFNXrefxAf4KPpjgEQDMaW3lwSJAMvH_0 2026-02-24T22:34:25.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8101645149063353874', time=2026-02-25T00:25:59.737985+0000, chain.objs.size()=1 2026-02-24T22:34:25.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.K6RmIUuno0fejG4QJd-IxFjrPtar7ZT_0 2026-02-24T22:34:25.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9312887801493121918', time=2026-02-25T00:25:59.739090+0000, chain.objs.size()=1 2026-02-24T22:34:25.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.gnIbxWHBH69_h4aun9-viFCqF0EUOPJ_0 2026-02-24T22:34:25.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14951153166085585399', time=2026-02-25T00:25:59.740373+0000, chain.objs.size()=1 2026-02-24T22:34:25.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.IdOGAkq9dFEXAN4YkCuOxg_PgDHiyco_0 2026-02-24T22:34:25.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16472372238049139327', time=2026-02-25T00:25:59.761598+0000, chain.objs.size()=1 2026-02-24T22:34:25.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.iHeYK8x0glcUhS5fEDfVk9oMXcFhk45_0 2026-02-24T22:34:25.940 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10335487103463332873', time=2026-02-25T00:25:59.805252+0000, chain.objs.size()=1 2026-02-24T22:34:25.941 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.08eHDgSkKrPRmYStSGw0-NnBn5X2n61_0 2026-02-24T22:34:25.941 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5503742878167680275', time=2026-02-25T00:25:59.805655+0000, chain.objs.size()=1 2026-02-24T22:34:25.941 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.zO7Ez2sxU2YThzGppOUEtA1O4dc3yLl_0 2026-02-24T22:34:25.941 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6069117607404092627', time=2026-02-25T00:25:59.823226+0000, chain.objs.size()=1 2026-02-24T22:34:25.941 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.q7ZXB9P2DZGJH1Dp31ENqIiXBUR8UEH_0 2026-02-24T22:34:25.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7089904884999452241', time=2026-02-25T00:25:59.936752+0000, chain.objs.size()=1 2026-02-24T22:34:25.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.m13SD-joQprInD4qPyiSGb_PU7mOUW5_0 2026-02-24T22:34:25.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15784365047344266114', time=2026-02-25T00:25:59.959189+0000, chain.objs.size()=1 2026-02-24T22:34:25.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.1OSnt4rIP9fvGZlaW4Z0wBr7IHWrcF4_0 2026-02-24T22:34:25.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7902313051040959423', time=2026-02-25T00:25:59.972711+0000, chain.objs.size()=1 2026-02-24T22:34:25.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.RL16HkF3mwFmrLBuOAMyoFpkXWGfo37_0 2026-02-24T22:34:25.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15313229905132791172', time=2026-02-25T00:25:59.973918+0000, chain.objs.size()=1 2026-02-24T22:34:25.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.J0jzwMEunWGAxOh_yCUFGtWZEmmDh9c_0 2026-02-24T22:34:25.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13348958298830786800', time=2026-02-25T00:25:59.982663+0000, chain.objs.size()=1 2026-02-24T22:34:25.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.BviwTKvCEp61BrcW3IJKGpN1LUdRNx7_0 2026-02-24T22:34:25.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17327041862230489696', time=2026-02-25T00:26:00.065209+0000, chain.objs.size()=1 2026-02-24T22:34:25.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Q5H1sA1pa8ZGiz1fykDo_rgCTkmv0xl_0 2026-02-24T22:34:25.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2494863010851486548', time=2026-02-25T00:26:00.066012+0000, chain.objs.size()=1 2026-02-24T22:34:25.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.wlyOz7ctrLN2DxO3pzYy_56G8azBB24_0 2026-02-24T22:34:25.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11378364889284873109', time=2026-02-25T00:26:00.066590+0000, chain.objs.size()=1 2026-02-24T22:34:25.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.951+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.A2lzz_ixrnCTj9PpJk6rx84Hku9vmXx_0 2026-02-24T22:34:25.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11012231708714254463', time=2026-02-25T00:26:00.088319+0000, chain.objs.size()=1 2026-02-24T22:34:25.942 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.wgdlpsq9oBMwULBGXxeDLWLR6HngQly_0 2026-02-24T22:34:25.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1073847875558877047', time=2026-02-25T00:26:00.094893+0000, chain.objs.size()=1 2026-02-24T22:34:25.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.SsnWIIyBHfwbsWORrAHnwAU_PjgqPO7_0 2026-02-24T22:34:25.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1229670020955500024', time=2026-02-25T00:26:00.219826+0000, chain.objs.size()=1 2026-02-24T22:34:25.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.dbYKx83bj34MKV3lqrtHuePI9e5yWsU_0 2026-02-24T22:34:25.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15516435423952491571', time=2026-02-25T00:26:00.221807+0000, chain.objs.size()=1 2026-02-24T22:34:25.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.JkVBKBGYfpLe3Wso2TW4w0i_AEgoNoX_0 2026-02-24T22:34:25.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13469606375390538087', time=2026-02-25T00:26:00.232718+0000, chain.objs.size()=1 2026-02-24T22:34:25.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.YvK-zBCG816HaeLDJIJXUnznYzsyNN4_0 2026-02-24T22:34:25.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10669258371335452295', time=2026-02-25T00:26:00.234999+0000, chain.objs.size()=1 2026-02-24T22:34:25.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Ocy70gVVnchd1w_dZgZ8v9TLSSK1sql_0 2026-02-24T22:34:25.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15307014222771250494', time=2026-02-25T00:26:00.236213+0000, chain.objs.size()=1 2026-02-24T22:34:25.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.EzSYDHWKdhqr_R17cYFsoihPlYf9xVd_0 2026-02-24T22:34:25.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18254264233018264522', time=2026-02-25T00:26:00.265493+0000, chain.objs.size()=1 2026-02-24T22:34:25.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.INVZ5vhV79xcj89PAA7h-b-UahxInVL_0 2026-02-24T22:34:25.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9767314284216498399', time=2026-02-25T00:26:00.275611+0000, chain.objs.size()=1 2026-02-24T22:34:25.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.9O8byvvXa4EttkPbSN-lXrhGzBrFTh2_0 2026-02-24T22:34:25.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13338841106366014934', time=2026-02-25T00:26:00.278629+0000, chain.objs.size()=1 2026-02-24T22:34:25.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_._7Kz2dZrS-FuptUdZedEPufZ2xfGSLO_0 2026-02-24T22:34:25.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5542792624824834016', time=2026-02-25T00:26:02.513094+0000, chain.objs.size()=1 2026-02-24T22:34:25.943 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1632__shadow_.DijF3Wr0hUpmvY7ZtTRVSFWQ8cBPOfE_0 2026-02-24T22:34:25.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17061936910609744509', time=2026-02-25T00:26:07.694026+0000, chain.objs.size()=1 2026-02-24T22:34:25.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1641__shadow_.6pGTfqO8Rd7zNmyFU60rQVm5tYaGr4__0 2026-02-24T22:34:25.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1791261865866221261', time=2026-02-25T00:26:18.508067+0000, chain.objs.size()=1 2026-02-24T22:34:25.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1658__shadow_.jfnpKh9lGL26p2TPQdk0sABdb3ceX5j_0 2026-02-24T22:34:25.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8243707333484070308', time=2026-02-25T00:26:18.509301+0000, chain.objs.size()=1 2026-02-24T22:34:25.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1658__shadow_.N4eodDjN9mFuwoJN23KPQuI9U3SeoRM_0 2026-02-24T22:34:25.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9060788595804119731', time=2026-02-25T00:26:18.521606+0000, chain.objs.size()=1 2026-02-24T22:34:25.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1658__shadow_.mAGawdYUSE7zeR85zlEgb82VNPLAAvR_0 2026-02-24T22:34:25.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3829686347132269855', time=2026-02-25T00:26:19.462809+0000, chain.objs.size()=1 2026-02-24T22:34:25.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1659__shadow_.npBWL9Uf_9th8pzxnbjHlyLDYmC-EMm_0 2026-02-24T22:34:25.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17936798813679239833', time=2026-02-25T00:26:22.306291+0000, chain.objs.size()=1 2026-02-24T22:34:25.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1664__shadow_.hln5DceX8_ty-MQZq-6AMHosmBSvePR_0 2026-02-24T22:34:25.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14615427084385994213', time=2026-02-25T00:26:23.485911+0000, chain.objs.size()=1 2026-02-24T22:34:25.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1666__shadow_.1DKF3R9C8XBxYkRGfAK_sIqr_fnizHQ_0 2026-02-24T22:34:25.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7788682458349433577', time=2026-02-25T00:26:24.066453+0000, chain.objs.size()=1 2026-02-24T22:34:25.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1667__shadow_.ibGZ0WWNe_VnbIzs62OnK8r93P9_Ycr_0 2026-02-24T22:34:25.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15468661117399415964', time=2026-02-25T00:26:24.654459+0000, chain.objs.size()=1 2026-02-24T22:34:25.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1668__shadow_.fIDhAx6TNCH617A5-PB4eQuDOBbL5MI_0 2026-02-24T22:34:25.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16146534706588572744', time=2026-02-25T00:26:26.429288+0000, chain.objs.size()=1 2026-02-24T22:34:25.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1671__shadow_.GFoOA7k9fBciS2BZqzlPS3lhnf0EAqp_0 2026-02-24T22:34:25.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10406327263555898507', time=2026-02-25T00:26:36.517752+0000, chain.objs.size()=1 2026-02-24T22:34:25.944 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1685__shadow_.uB0hEo4HBOoqgLIVtbH-RbBjaMlUyDW_0 2026-02-24T22:34:25.945 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15695109366485934088', time=2026-02-25T00:26:44.820471+0000, chain.objs.size()=1 2026-02-24T22:34:25.945 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.boL2vgDaSalOflWU4VcDPZhC4cxH3-N_0 2026-02-24T22:34:25.945 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1393858351372414615', time=2026-02-25T00:26:44.843583+0000, chain.objs.size()=1 2026-02-24T22:34:25.945 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.qRvOcacSEd-wu5XDsPKhNAX7UM4BVoQ_0 2026-02-24T22:34:25.945 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15132445976831627664', time=2026-02-25T00:26:44.845039+0000, chain.objs.size()=1 2026-02-24T22:34:25.945 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.8Khu6_nqtYxLs-gHfvlBDYUAldLPQf4_0 2026-02-24T22:34:25.945 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10494027142613427528', time=2026-02-25T00:26:44.868385+0000, chain.objs.size()=1 2026-02-24T22:34:25.945 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.8-Pwskf1IvTK9h7-bH-_gPwMvtD7Cs0_0 2026-02-24T22:34:25.945 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16419856750863159567', time=2026-02-25T00:26:44.965377+0000, chain.objs.size()=1 2026-02-24T22:34:25.945 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.363ISl9ERshNuwvDsQ72t7oqe75ow_U_0 2026-02-24T22:34:25.945 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13788390765885357617', time=2026-02-25T00:26:44.990502+0000, chain.objs.size()=1 2026-02-24T22:34:25.945 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.j6rE_cZ_gcX7UntL8TNVlsDMyk1IuB2_0 2026-02-24T22:34:25.945 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7369093148886846246', time=2026-02-25T00:26:45.027901+0000, chain.objs.size()=1 2026-02-24T22:34:25.945 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.TRMkqqQQIuoBCMWD8N8tPpZRJU3mXFb_0 2026-02-24T22:34:25.945 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8797786025651974656', time=2026-02-25T00:26:45.111114+0000, chain.objs.size()=1 2026-02-24T22:34:25.946 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.mz1ThtC1gjd1lSHn5ZFvirlYqtek593_0 2026-02-24T22:34:25.946 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8634163260954395757', time=2026-02-25T00:26:45.140092+0000, chain.objs.size()=1 2026-02-24T22:34:25.946 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.DAqAOJ4ZoP01Q1XMyxc1c5XesgeaqAM_0 2026-02-24T22:34:25.946 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8759163955857569930', time=2026-02-25T00:26:45.147648+0000, chain.objs.size()=1 2026-02-24T22:34:25.946 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.VW3AKdDj0CScXU7QcjEazD_Wys0BCZE_0 2026-02-24T22:34:25.946 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5435293781587126990', time=2026-02-25T00:26:45.163589+0000, chain.objs.size()=1 2026-02-24T22:34:25.946 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.qV0EcSW-eGxxGh6WdQNLbPr8TI2Vmq1_0 2026-02-24T22:34:25.946 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1441186415921341828', time=2026-02-25T00:26:45.169291+0000, chain.objs.size()=1 2026-02-24T22:34:25.946 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_._SNjKpChRsGzkfWqACYF7MnHKf9-14__0 2026-02-24T22:34:25.946 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9976585458868592443', time=2026-02-25T00:26:45.188753+0000, chain.objs.size()=1 2026-02-24T22:34:25.946 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.h-TOzwHjdlw9Dzyuob-a4O76iuX4e9n_0 2026-02-24T22:34:25.946 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1075776144133249278', time=2026-02-25T00:26:45.317273+0000, chain.objs.size()=1 2026-02-24T22:34:25.946 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.1nawJQ4rbeU17QEbu3xsBGQfd42g34n_0 2026-02-24T22:34:25.946 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5943370157873362246', time=2026-02-25T00:26:45.322894+0000, chain.objs.size()=1 2026-02-24T22:34:25.947 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_._gHgzIUdKx3TdtPnhiXA7zovS3DWGo6_0 2026-02-24T22:34:25.947 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6989069978335375554', time=2026-02-25T00:26:45.336502+0000, chain.objs.size()=1 2026-02-24T22:34:25.947 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.fNJ8rimwa2w-NJ9ZIRgG-8SWK9qweX9_0 2026-02-24T22:34:25.947 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16878155986511201113', time=2026-02-25T00:26:45.440740+0000, chain.objs.size()=1 2026-02-24T22:34:25.947 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.955+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.d0dTcIl7ij9_8J0uNGe8C4AKL3HFyi8_0 2026-02-24T22:34:25.947 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15957836384412436568', time=2026-02-25T00:26:45.478317+0000, chain.objs.size()=1 2026-02-24T22:34:25.947 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.EDTWozAJG1q4fBijn12WQsA6A6g1My4_0 2026-02-24T22:34:25.947 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7630955898813470293', time=2026-02-25T00:26:45.491446+0000, chain.objs.size()=1 2026-02-24T22:34:25.947 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.AhKRaQharPUUYfrkwFoK2ZWB1mrt6qG_0 2026-02-24T22:34:25.947 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9658142918233383718', time=2026-02-25T00:26:45.502691+0000, chain.objs.size()=1 2026-02-24T22:34:25.947 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.7gZF7Pnag_170wNefq_E8sq_MYa8VMU_0 2026-02-24T22:34:25.947 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2970849223441562912', time=2026-02-25T00:26:45.575715+0000, chain.objs.size()=1 2026-02-24T22:34:25.947 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.O3yOjAshLOO0uMQSLxIXb0G-1IrcZ13_0 2026-02-24T22:34:25.947 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8525354442440291507', time=2026-02-25T00:26:45.603231+0000, chain.objs.size()=1 2026-02-24T22:34:25.947 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.hDzMLFQLWXPlVzffmuq-miYr4l8P-9H_0 2026-02-24T22:34:25.947 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11460077247166524049', time=2026-02-25T00:26:45.603697+0000, chain.objs.size()=1 2026-02-24T22:34:25.947 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.VDVt7f8ehXTRFSrmCmh7n8b2luJC6_H_0 2026-02-24T22:34:25.947 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.609647231355605461', time=2026-02-25T00:26:45.823303+0000, chain.objs.size()=1 2026-02-24T22:34:25.947 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.m7tKDow4S9jVrhnUC5tGDFawbnGwGdM_0 2026-02-24T22:34:25.947 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7907179439540292094', time=2026-02-25T00:26:45.851263+0000, chain.objs.size()=1 2026-02-24T22:34:25.947 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_._p9JFIj0BHwhBuksWpZKQJa_z0wbe8x_0 2026-02-24T22:34:25.947 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17705695090702896786', time=2026-02-25T00:26:45.967015+0000, chain.objs.size()=1 2026-02-24T22:34:25.947 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.iqut4tfvUOm6C9Ioj-Sus4UwGdrs-X__0 2026-02-24T22:34:25.948 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3557222405383744096', time=2026-02-25T00:26:45.973435+0000, chain.objs.size()=1 2026-02-24T22:34:25.948 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ehpvpp2DmSXlBOSoruofj8BCIoUXL08_0 2026-02-24T22:34:25.948 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12526539490707000606', time=2026-02-25T00:26:45.982040+0000, chain.objs.size()=1 2026-02-24T22:34:25.948 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.64gXPFNXFfHvx6GwfXTi9d1y14vpXMN_0 2026-02-24T22:34:25.948 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11099447612037175427', time=2026-02-25T00:26:46.093624+0000, chain.objs.size()=1 2026-02-24T22:34:25.948 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.9SbLsxVn3uieW8eRtopG_W3jGmzt_Cg_0 2026-02-24T22:34:25.948 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1601460140777262837', time=2026-02-25T00:26:46.100507+0000, chain.objs.size()=1 2026-02-24T22:34:25.948 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.pZOynW8fhAHxO7jpgGIuLkefYAz-rbP_0 2026-02-24T22:34:25.948 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/38970 2026-02-24T22:34:25.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=61, truncated=0, next_marker='' 2026-02-24T22:34:25.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17518194693672021653', time=2026-02-25T00:26:46.128162+0000, chain.objs.size()=1 2026-02-24T22:34:25.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Y1a_qfvNespbZ3pR88y-Kaz5SAOW7Be_0 2026-02-24T22:34:25.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4410026023471356063', time=2026-02-25T00:26:53.154739+0000, chain.objs.size()=1 2026-02-24T22:34:25.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.b0DhNrxi_tXwmWjBv43TTUarKHOVjbz_0 2026-02-24T22:34:25.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7593909525264031552', time=2026-02-25T00:26:53.157090+0000, chain.objs.size()=1 2026-02-24T22:34:25.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.0_lpfjC0Mdk1NibFksVjA2PmnT1Yz2__0 2026-02-24T22:34:25.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8954176337390981908', time=2026-02-25T00:26:53.200620+0000, chain.objs.size()=1 2026-02-24T22:34:25.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.cwtqlAvcRx2a8uwKW4uUaclEBjF2DA2_0 2026-02-24T22:34:25.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12148621321574682686', time=2026-02-25T00:26:53.246793+0000, chain.objs.size()=1 2026-02-24T22:34:25.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.BDKJmUzAc_G3Hb2b058LxJ7Rq_Xh_d4_0 2026-02-24T22:34:25.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11180131302169268972', time=2026-02-25T00:26:53.252552+0000, chain.objs.size()=1 2026-02-24T22:34:25.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.IEt9qvVLci1dS55CUVFk1xX11w2aOG5_0 2026-02-24T22:34:25.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5176581861505333033', time=2026-02-25T00:26:53.367680+0000, chain.objs.size()=1 2026-02-24T22:34:25.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.wHypUTB5rGmO85fsm5OZs3zedywoLcE_0 2026-02-24T22:34:25.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4523862376771285651', time=2026-02-25T00:26:53.397328+0000, chain.objs.size()=1 2026-02-24T22:34:25.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.kMJvWdaRplj7rmeQuRPj5OR8rzs8kvW_0 2026-02-24T22:34:25.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8445671453159339132', time=2026-02-25T00:26:53.402101+0000, chain.objs.size()=1 2026-02-24T22:34:25.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.2-mxKsJFtY_UR--UUDhbDEt_JwAAWP-_0 2026-02-24T22:34:25.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3660763377368814143', time=2026-02-25T00:26:53.417733+0000, chain.objs.size()=1 2026-02-24T22:34:25.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.6W88VLV8YytfbpBe6mxmsoIljqCRhx5_0 2026-02-24T22:34:25.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15214507754512003040', time=2026-02-25T00:26:53.419744+0000, chain.objs.size()=1 2026-02-24T22:34:25.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.sZnFZKi3rZScu2ipwK1_DdZpC08jwqY_0 2026-02-24T22:34:25.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14627835881026798186', time=2026-02-25T00:26:53.425249+0000, chain.objs.size()=1 2026-02-24T22:34:25.950 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.959+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ygDnnPDW4bIy9noA1LvBym20DJiKReH_0 2026-02-24T22:34:25.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2387849691696230998', time=2026-02-25T00:26:53.428922+0000, chain.objs.size()=1 2026-02-24T22:34:25.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.e3yaei93r-FPoa0kdYWvljFHOZnd_NV_0 2026-02-24T22:34:25.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4997915310668449742', time=2026-02-25T00:26:53.451858+0000, chain.objs.size()=1 2026-02-24T22:34:25.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.qylXfsfNNwEQT55VyTMFUOB0zcfFs6d_0 2026-02-24T22:34:25.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4357448275952226121', time=2026-02-25T00:26:53.536086+0000, chain.objs.size()=1 2026-02-24T22:34:25.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.m8wevg5V6Ibssr42yZ1OMntcq_f19TQ_0 2026-02-24T22:34:25.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17927827039973653160', time=2026-02-25T00:26:53.572694+0000, chain.objs.size()=1 2026-02-24T22:34:25.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.xOHOCc17Ni7u97wJysDvqQ_RDo2wD-V_0 2026-02-24T22:34:25.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15951794164252503318', time=2026-02-25T00:26:53.580527+0000, chain.objs.size()=1 2026-02-24T22:34:25.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.KrLF9qBcKEUMCkOY6c6pfadA4Lr5DDv_0 2026-02-24T22:34:25.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1835205927118598485', time=2026-02-25T00:26:53.583795+0000, chain.objs.size()=1 2026-02-24T22:34:25.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.oqpJYTGqOc5Dp8VN2Ts4ke43gUkV_8V_0 2026-02-24T22:34:25.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5118330456444049125', time=2026-02-25T00:26:53.589252+0000, chain.objs.size()=1 2026-02-24T22:34:25.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.-4MGxf9Gy17Frj8n97_e9lr5rRdGFs1_0 2026-02-24T22:34:25.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8538328673802903539', time=2026-02-25T00:26:53.614688+0000, chain.objs.size()=1 2026-02-24T22:34:25.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.wQDKCiACSrEtVevnFXnsFBnhHIy7T8N_0 2026-02-24T22:34:25.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6118081691485710595', time=2026-02-25T00:26:53.762629+0000, chain.objs.size()=1 2026-02-24T22:34:25.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.tDeJ2h0tpqYcS-2FRmMN1TaK5uUiTzM_0 2026-02-24T22:34:25.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15539536908819497774', time=2026-02-25T00:26:53.797530+0000, chain.objs.size()=1 2026-02-24T22:34:25.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.eB37NzTWvIssbvTHYbDwcYFQPUed_jd_0 2026-02-24T22:34:25.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2598062852389580739', time=2026-02-25T00:26:53.814791+0000, chain.objs.size()=1 2026-02-24T22:34:25.951 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.c74Uv8vlv1HBnS069SGiFTowDE1kqDA_0 2026-02-24T22:34:25.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1437621773476687291', time=2026-02-25T00:26:53.816221+0000, chain.objs.size()=1 2026-02-24T22:34:25.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.KbPa0jvDr9c0NLvIewiO6i3fIhyW0gQ_0 2026-02-24T22:34:25.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3132175010916976298', time=2026-02-25T00:26:53.947675+0000, chain.objs.size()=1 2026-02-24T22:34:25.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.SEpY_Cb50QUF1I9eUr3JyjVtwyfwW4j_0 2026-02-24T22:34:25.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1346075261542486238', time=2026-02-25T00:26:54.000808+0000, chain.objs.size()=1 2026-02-24T22:34:25.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.jxyXu5N1Wdib9FN37PGNKuIETCCC_-S_0 2026-02-24T22:34:25.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7996554882203870218', time=2026-02-25T00:26:54.016321+0000, chain.objs.size()=1 2026-02-24T22:34:25.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.67fPx5du9wNL07lFm8s6ep0mA5TRL-d_0 2026-02-24T22:34:25.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11261156913028505077', time=2026-02-25T00:26:54.043926+0000, chain.objs.size()=1 2026-02-24T22:34:25.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Jb7AScnUtUUmT_2ILVk0BTar9QrwxmD_0 2026-02-24T22:34:25.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17679630566315673531', time=2026-02-25T00:26:54.114758+0000, chain.objs.size()=1 2026-02-24T22:34:25.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.i7pu_r85vFR59BPt3TMqoJjBpmToWUu_0 2026-02-24T22:34:25.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9369717013641820281', time=2026-02-25T00:26:54.194850+0000, chain.objs.size()=1 2026-02-24T22:34:25.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.jWta7LRqvbA6RM68uGtQsRVXOhQJ6Hl_0 2026-02-24T22:34:25.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.793199612013422432', time=2026-02-25T00:26:54.307974+0000, chain.objs.size()=1 2026-02-24T22:34:25.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.BybuWWmY40f5ZkCmHPcsXbwgH7RsumV_0 2026-02-24T22:34:25.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17120613039720637751', time=2026-02-25T00:26:54.343950+0000, chain.objs.size()=1 2026-02-24T22:34:25.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ywz8Y1OEUdiZJowWkE1-XWtQuOZdaKb_0 2026-02-24T22:34:25.952 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14487765750238908989', time=2026-02-25T00:26:54.345773+0000, chain.objs.size()=1 2026-02-24T22:34:25.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.0sbZ_G_F5KAVG1au8nQvpV8r9iduuNc_0 2026-02-24T22:34:25.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15305377013523291971', time=2026-02-25T00:26:54.352660+0000, chain.objs.size()=1 2026-02-24T22:34:25.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.prlL69zD5RMi9dxbS7ApVrKilRbetD7_0 2026-02-24T22:34:25.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7748311599478516548', time=2026-02-25T00:26:54.386554+0000, chain.objs.size()=1 2026-02-24T22:34:25.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.DBOHKtzEtJA6Vj5nxACUtqy5xLJPQl__0 2026-02-24T22:34:25.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16860584910168631511', time=2026-02-25T00:26:54.403303+0000, chain.objs.size()=1 2026-02-24T22:34:25.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.HtwUQ-sIX2izhNjYBNiWPiOkGzzr2Oe_0 2026-02-24T22:34:25.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10380992423315130563', time=2026-02-25T00:26:54.476697+0000, chain.objs.size()=1 2026-02-24T22:34:25.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ch8G_hr7RspBkmCD0WeXGzVu9IEvzk-_0 2026-02-24T22:34:25.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12927320523045955569', time=2026-02-25T00:26:54.498810+0000, chain.objs.size()=1 2026-02-24T22:34:25.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.yo4TX5fkhKaoDHOiXWi7pe0fbJT60cm_0 2026-02-24T22:34:25.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4936668262735071336', time=2026-02-25T00:26:54.511202+0000, chain.objs.size()=1 2026-02-24T22:34:25.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.ryKY6pBOPC34tO-Ts-d1ufRPKe6J99__0 2026-02-24T22:34:25.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9648920290704864403', time=2026-02-25T00:26:55.616626+0000, chain.objs.size()=1 2026-02-24T22:34:25.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1691__shadow_.oVWiDJUcy4qTeIF0xkx9s-3i6mu1Wu-_0 2026-02-24T22:34:25.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9520964338823131656', time=2026-02-25T00:26:59.529884+0000, chain.objs.size()=1 2026-02-24T22:34:25.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1698__shadow_.vjEY_h91WkhGX_g0fomOvZBOcNkKsTy_0 2026-02-24T22:34:25.953 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.884569537132856942', time=2026-02-25T00:27:28.100337+0000, chain.objs.size()=1 2026-02-24T22:34:25.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1748__shadow_.-nvcjlWoYm2jd3yxpesOv622v7w0a8a_0 2026-02-24T22:34:25.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14129276739266244309', time=2026-02-25T00:28:08.633333+0000, chain.objs.size()=1 2026-02-24T22:34:25.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1809__shadow_.HpxdndjzaeZBVWUubiyhD6Gm_UE-EvM_0 2026-02-24T22:34:25.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4747676847549592461', time=2026-02-25T00:28:32.555619+0000, chain.objs.size()=1 2026-02-24T22:34:25.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1849__shadow_.fAcwt-BizR_NX2aW5YHj20QiazYzhHy_0 2026-02-24T22:34:25.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.963+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15166698789449116466', time=2026-02-25T00:28:43.675893+0000, chain.objs.size()=1 2026-02-24T22:34:25.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1869__shadow:ba1nU1fNR3N1sfUQ2kL8K8Wm.5Ibl31_.IVNpljy9O7AjWtEyJ7RLhDmshakOZnh_0 2026-02-24T22:34:25.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='2~iU8Y5XWOPZyi697HnOwAKeovAh-HzJA', time=2026-02-25T00:29:20.501063+0000, chain.objs.size()=1 2026-02-24T22:34:25.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1900__multipart_mymultipart.2~iU8Y5XWOPZyi697HnOwAKeovAh-HzJA.1 2026-02-24T22:34:25.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6269685415842235462', time=2026-02-25T00:30:02.241078+0000, chain.objs.size()=1 2026-02-24T22:34:25.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1918__shadow_.1qRWXIohh2B0Vrh7ZGfWe994fN9AHxT_0 2026-02-24T22:34:25.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9297774079854065128', time=2026-02-25T00:30:05.249058+0000, chain.objs.size()=1 2026-02-24T22:34:25.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1922__shadow_.UXEyYfUNLc9O1-PKkfJ2tF9o-ufhHGc_0 2026-02-24T22:34:25.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18225457681267461969', time=2026-02-25T00:30:17.910610+0000, chain.objs.size()=1 2026-02-24T22:34:25.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:oa2FO8aCkNuQ4.Xd7N6z4NcPgqntQfb_.Kfhn3e7Bd4Q7Nmt4DSlvn3baULJI0Yl_0 2026-02-24T22:34:25.954 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16204896179638074714', time=2026-02-25T00:30:23.465920+0000, chain.objs.size()=1 2026-02-24T22:34:25.955 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:s-PDmEfLIZb1BvfoOVOQ5Io299LyMtZ_.hNQPSSM6iZrpFwA2i0uECRLTX_r41xv_0 2026-02-24T22:34:25.955 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17389650455516160474', time=2026-02-25T00:30:28.080966+0000, chain.objs.size()=1 2026-02-24T22:34:25.955 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1943__shadow_.EaRiggLumAmVcjEExPfkz-2ooUDsLPA_0 2026-02-24T22:34:25.955 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7056002501505917591', time=2026-02-25T00:30:28.111270+0000, chain.objs.size()=1 2026-02-24T22:34:25.955 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1943__shadow_.8SjLfQ2lgpoY1Pa3QfXdLq6oWf3BqAd_0 2026-02-24T22:34:25.955 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7182323314330128507', time=2026-02-25T00:30:36.394083+0000, chain.objs.size()=1 2026-02-24T22:34:25.955 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:gP8KD2OCQ1yfP9YypQX2EHGlqudzT8M_.cvvTFKSCkMb1qo9W7uLp8Dcwk8f7YD7_0 2026-02-24T22:34:25.955 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6358004749995484563', time=2026-02-25T00:30:36.798379+0000, chain.objs.size()=1 2026-02-24T22:34:25.955 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:rdXMNXADMjxswt0zbf3x3ODSh8B.gDr_.Uy6f6PzmOf8-y-j904A5YSPTHMfUzs2_0 2026-02-24T22:34:25.955 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10714468371219954331', time=2026-02-25T00:30:44.489970+0000, chain.objs.size()=1 2026-02-24T22:34:25.955 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:QfYwo-AhAf9j2WmWwvuj6soHoClg2dP_.sK7B1F3N1Btrr57DJ4TTz3ywE2e_Jaf_0 2026-02-24T22:34:25.955 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8149012703998111304', time=2026-02-25T00:30:49.741713+0000, chain.objs.size()=4 2026-02-24T22:34:25.955 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1948__multipart_testobj.2~zbmejsdBg5XQxfpH9uaKUb73bhO2OAq.1 2026-02-24T22:34:25.955 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1948__multipart_testobj.2~zbmejsdBg5XQxfpH9uaKUb73bhO2OAq.2 2026-02-24T22:34:25.956 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1948__multipart_testobj.2~zbmejsdBg5XQxfpH9uaKUb73bhO2OAq.3 2026-02-24T22:34:25.956 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1948__multipart_testobj.2~zbmejsdBg5XQxfpH9uaKUb73bhO2OAq.4 2026-02-24T22:34:25.956 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14308349330898795834', time=2026-02-25T00:30:57.749731+0000, chain.objs.size()=1 2026-02-24T22:34:25.956 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:x.qx1WkvT4TJXMyKRWl48I2W78cLrjp_.JtJdNnMPVzD9evQzGtf25qNwd_w-IJq_0 2026-02-24T22:34:25.956 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14308349330898795834', time=2026-02-25T00:30:57.750150+0000, chain.objs.size()=1 2026-02-24T22:34:25.956 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:x.qx1WkvT4TJXMyKRWl48I2W78cLrjp_.JtJdNnMPVzD9evQzGtf25qNwd_w-IJq_0 2026-02-24T22:34:25.956 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10003132232400059543', time=2026-02-25T00:33:40.188462+0000, chain.objs.size()=1 2026-02-24T22:34:25.956 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1962__shadow_.7WK0KF-trCOy08CrLrvUj8HEXkJA-8C_0 2026-02-24T22:34:25.956 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5161653808460458523', time=2026-02-25T00:33:44.077166+0000, chain.objs.size()=1 2026-02-24T22:34:25.956 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1966__shadow_.qI0TTx_2KGTkXk07nL443f64tONLX0d_0 2026-02-24T22:34:25.956 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15317282107490309823', time=2026-02-25T00:33:51.533571+0000, chain.objs.size()=1 2026-02-24T22:34:25.956 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1973__shadow_.rtRu81pHAuyInvRGpLbMDpc2UNO-g1m_0 2026-02-24T22:34:25.957 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.967+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.959 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.971+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2026-02-24T22:34:25.960 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.971+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2026-02-24T22:34:25.961 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.971+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2026-02-24T22:34:25.961 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.971+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/38786' 2026-02-24T22:34:25.961 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.971+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1233588049632215688', time=2026-02-25T00:15:09.906513+0000, chain.objs.size()=1 2026-02-24T22:34:25.961 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.971+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1505__shadow_.v9lKFT-9tLw4jZJbsXUoPicxtTA_Kh9_0 2026-02-24T22:34:25.961 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.971+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11050123119396998216', time=2026-02-25T00:15:09.939705+0000, chain.objs.size()=1 2026-02-24T22:34:25.961 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.971+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1505__shadow_.vakzp46gtORFgHBDmUryALhhpWwThyS_0 2026-02-24T22:34:25.961 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.971+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15874434673824327812', time=2026-02-25T00:15:34.788567+0000, chain.objs.size()=3 2026-02-24T22:34:25.962 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.971+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515__multipart_foo-FROZEN.2~v_f2anZgPC1pddb-uul3GH8WZsvkJ8F.1 2026-02-24T22:34:25.962 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.971+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515__multipart_foo-FROZEN.2~v_f2anZgPC1pddb-uul3GH8WZsvkJ8F.2 2026-02-24T22:34:25.962 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.971+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1515__multipart_foo-FROZEN.2~v_f2anZgPC1pddb-uul3GH8WZsvkJ8F.3 2026-02-24T22:34:25.962 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.971+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17368290082411530293', time=2026-02-25T00:16:32.621236+0000, chain.objs.size()=1 2026-02-24T22:34:25.962 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.971+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1520__shadow_.KeqV2LezhTLQFZjr0m0-XQQoCGSPqaE_0 2026-02-24T22:34:25.962 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.971+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9116431103497423976', time=2026-02-25T00:17:07.122396+0000, chain.objs.size()=1 2026-02-24T22:34:25.962 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.971+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523__shadow_.65GbnghtLjtiqROKdP0AugssU8zKMdf_0 2026-02-24T22:34:25.962 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.971+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3679942476280685545', time=2026-02-25T00:17:07.126580+0000, chain.objs.size()=1 2026-02-24T22:34:25.962 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.971+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1523__shadow_.wI2pBENpNZefU9RfASq62Up7XBrNeRZ_0 2026-02-24T22:34:25.962 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.971+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16654371009504132612', time=2026-02-25T00:17:39.882120+0000, chain.objs.size()=1 2026-02-24T22:34:25.962 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.971+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1525__shadow_.RDMZtvTAJTbEoQUu2Mf4r9xRkJUxcor_0 2026-02-24T22:34:25.962 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.971+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16341839650698715530', time=2026-02-25T00:18:48.194999+0000, chain.objs.size()=1 2026-02-24T22:34:25.962 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.971+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1530__shadow_.-ArRvOJtu7WbBl1S_Vt-C0NYkfvSq46_0 2026-02-24T22:34:25.962 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.971+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14997772484617457245', time=2026-02-25T00:19:25.027893+0000, chain.objs.size()=1 2026-02-24T22:34:25.962 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.971+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534__shadow_.tUIN-iYHK-H_jIXyedBTnfjS1EvbZl5_0 2026-02-24T22:34:25.962 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.971+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2302514942935243898', time=2026-02-25T00:19:25.032203+0000, chain.objs.size()=1 2026-02-24T22:34:25.962 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.971+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1534__shadow_.GLH6ud660k0jxMpox2T1CdTiDHo1nRV_0 2026-02-24T22:34:25.962 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2455067684249507584', time=2026-02-25T00:20:30.738779+0000, chain.objs.size()=1 2026-02-24T22:34:25.962 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1537__shadow_.BM_lX-TQAXX9af1uGy0eS3J0Lc6N8Q1_0 2026-02-24T22:34:25.962 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9500281978856573959', time=2026-02-25T00:20:34.042917+0000, chain.objs.size()=1 2026-02-24T22:34:25.963 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539__shadow_.zTX3lUPCiI8DzQbIkUJW3snHBplQ4fe_0 2026-02-24T22:34:25.963 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7209486424786657931', time=2026-02-25T00:20:34.043472+0000, chain.objs.size()=1 2026-02-24T22:34:25.963 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539__shadow_.N0F4fnFJ7u-AIEeixdJV7HYFV575kgY_0 2026-02-24T22:34:25.963 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10717253129547547423', time=2026-02-25T00:25:33.717491+0000, chain.objs.size()=1 2026-02-24T22:34:25.963 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1597__shadow_.TJMnV4iQGI9Di-0eQ5pJbnkFao7bjSv_0 2026-02-24T22:34:25.963 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3258746913789169686', time=2026-02-25T00:25:49.093208+0000, chain.objs.size()=1 2026-02-24T22:34:25.963 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1621__shadow_.MYI5SY4xYrywtqRjUfyHJFUuz8eKm_x_0 2026-02-24T22:34:25.964 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8629721358132138943', time=2026-02-25T00:25:52.537839+0000, chain.objs.size()=1 2026-02-24T22:34:25.964 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1627__shadow_.JGHnUVqXDK-Tlw45z4fSAWiKfTRvCUS_0 2026-02-24T22:34:25.964 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8210462817426841743', time=2026-02-25T00:25:59.091416+0000, chain.objs.size()=1 2026-02-24T22:34:25.964 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.6WaoBPN9CzxG05ZXe_GHMh4BtubCjls_0 2026-02-24T22:34:25.964 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9036865430018735344', time=2026-02-25T00:25:59.099063+0000, chain.objs.size()=1 2026-02-24T22:34:25.964 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.FCR9S3MaBb-SOzyEzAaMJfcySWt4nZo_0 2026-02-24T22:34:25.964 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13761175623640352456', time=2026-02-25T00:25:59.143617+0000, chain.objs.size()=1 2026-02-24T22:34:25.964 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.4rQ5-PHMoUwbiz1u84md4O7YJnrhYbK_0 2026-02-24T22:34:25.964 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6559035862328874877', time=2026-02-25T00:25:59.184026+0000, chain.objs.size()=1 2026-02-24T22:34:25.964 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.jyxWnb6At1DMLNlBKR4PjdXyIxOhGwp_0 2026-02-24T22:34:25.964 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1539360118548905131', time=2026-02-25T00:25:59.288281+0000, chain.objs.size()=1 2026-02-24T22:34:25.964 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.K1C1vad-MJfG20BHRX9C3C_rvr6Hx_l_0 2026-02-24T22:34:25.964 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4500428806729211628', time=2026-02-25T00:25:59.349602+0000, chain.objs.size()=1 2026-02-24T22:34:25.964 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.M1NUR5PEhETjCC5k87NCgosryDigCU0_0 2026-02-24T22:34:25.964 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6289517906799468897', time=2026-02-25T00:25:59.468322+0000, chain.objs.size()=1 2026-02-24T22:34:25.964 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.XztEMXzyYOrnV4KcPQGI6mkfgFCuFqa_0 2026-02-24T22:34:25.964 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.56970390975995842', time=2026-02-25T00:25:59.469481+0000, chain.objs.size()=1 2026-02-24T22:34:25.964 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.DllSFJIhlWt2v_kUCTWzFe06gIIgNBQ_0 2026-02-24T22:34:25.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8427344591949791651', time=2026-02-25T00:25:59.500214+0000, chain.objs.size()=1 2026-02-24T22:34:25.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.yxWIpB2AXkOIDEyOiPAoTVEeeS_swT3_0 2026-02-24T22:34:25.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11080625021823023814', time=2026-02-25T00:25:59.500805+0000, chain.objs.size()=1 2026-02-24T22:34:25.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.xXghi4qlGAjLCAd49i4MRLTwFGJ8bhe_0 2026-02-24T22:34:25.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.535003942450561475', time=2026-02-25T00:25:59.610217+0000, chain.objs.size()=1 2026-02-24T22:34:25.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.j8PF1tZjAiIOGltpSExVMvod6ovQId5_0 2026-02-24T22:34:25.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17632535189883347033', time=2026-02-25T00:25:59.661379+0000, chain.objs.size()=1 2026-02-24T22:34:25.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.YL48R9eaMH5g_5OrBB9F2E1_QrspZqv_0 2026-02-24T22:34:25.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11909073845861668859', time=2026-02-25T00:25:59.924849+0000, chain.objs.size()=1 2026-02-24T22:34:25.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.Sa_1_lMIBjP2BO7LTFK0eFd4cF8r5Gg_0 2026-02-24T22:34:25.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15003154979753192565', time=2026-02-25T00:25:59.927830+0000, chain.objs.size()=1 2026-02-24T22:34:25.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.uS23opUN7oqqwma4Nko48Ob8658U-t8_0 2026-02-24T22:34:25.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10289513714412748885', time=2026-02-25T00:25:59.962148+0000, chain.objs.size()=1 2026-02-24T22:34:25.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.nLU_99McLKQ0XVRxVV3IlA7crUJrrAS_0 2026-02-24T22:34:25.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4084472535453206431', time=2026-02-25T00:25:59.965503+0000, chain.objs.size()=1 2026-02-24T22:34:25.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.nP437dtVERwf5EqlkUQPMTu_r9kXSqh_0 2026-02-24T22:34:25.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5953992021534607730', time=2026-02-25T00:25:59.968348+0000, chain.objs.size()=1 2026-02-24T22:34:25.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.GxawSt9vpDbTasJGD4-d9G6qHaSozQf_0 2026-02-24T22:34:25.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18243267561807922430', time=2026-02-25T00:25:59.971594+0000, chain.objs.size()=1 2026-02-24T22:34:25.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.wNTL9mVbdnoSIDvUfpipu9CyTy9_cDx_0 2026-02-24T22:34:25.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11243638517019548018', time=2026-02-25T00:25:59.984340+0000, chain.objs.size()=1 2026-02-24T22:34:25.965 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.i-yeN-MEQgKdeZG78RdJXR12-E5L3tn_0 2026-02-24T22:34:25.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12965244865648162883', time=2026-02-25T00:26:00.072703+0000, chain.objs.size()=1 2026-02-24T22:34:25.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.JEj9vGvcUjR-pxEdJtjO2a9GMAaAIAD_0 2026-02-24T22:34:25.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6126641127924300838', time=2026-02-25T00:26:00.077125+0000, chain.objs.size()=1 2026-02-24T22:34:25.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.lbs9cJsc8taC2Q5QY0pt8AU4ErE6qDd_0 2026-02-24T22:34:25.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12697885221089154947', time=2026-02-25T00:26:00.091929+0000, chain.objs.size()=1 2026-02-24T22:34:25.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.PtrWcqMgXD5xl58ipigMSsXF9D0ffoZ_0 2026-02-24T22:34:25.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13756734936303352040', time=2026-02-25T00:26:00.130230+0000, chain.objs.size()=1 2026-02-24T22:34:25.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.pJ1dFHtkdZITqHMLjBKyr1DjjJhuBHV_0 2026-02-24T22:34:25.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15608025970383481389', time=2026-02-25T00:26:00.151663+0000, chain.objs.size()=1 2026-02-24T22:34:25.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.P_-VcNIak5ZxeDiUIxUXZ7dn6m_ZrmO_0 2026-02-24T22:34:25.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15672767325826677676', time=2026-02-25T00:26:00.227080+0000, chain.objs.size()=1 2026-02-24T22:34:25.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.7K4loXMoKj2AVy57FfeyNRQJ15Zx1SR_0 2026-02-24T22:34:25.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9914024585190487778', time=2026-02-25T00:26:00.255349+0000, chain.objs.size()=1 2026-02-24T22:34:25.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.ADYzCA9D1cyl41BvE7uUIhcxCNDuUSm_0 2026-02-24T22:34:25.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15961085836068087036', time=2026-02-25T00:26:00.261432+0000, chain.objs.size()=1 2026-02-24T22:34:25.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.DIMi88nF-Cn--wxqx7tnrXX60NF0dYQ_0 2026-02-24T22:34:25.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3089428949972375972', time=2026-02-25T00:26:00.261967+0000, chain.objs.size()=1 2026-02-24T22:34:25.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1628__shadow_.haIqLmYoLqhvySkOGNPTQYBd7bbwCdP_0 2026-02-24T22:34:25.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2279294228843070329', time=2026-02-25T00:26:05.961253+0000, chain.objs.size()=1 2026-02-24T22:34:25.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.975+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1638__shadow_.UuUdXTk3pOhNO7UvR3uv4Iii_V-uPlW_0 2026-02-24T22:34:25.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17915031875788830886', time=2026-02-25T00:26:16.590123+0000, chain.objs.size()=1 2026-02-24T22:34:25.966 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1656__shadow_.qbzieGnPBMB-rh9C_LbDXTvTxquACG7_0 2026-02-24T22:34:25.967 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14440116812614795875', time=2026-02-25T00:26:18.509998+0000, chain.objs.size()=1 2026-02-24T22:34:25.967 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1658__shadow_.t4IhEMKRlHKTsx3_0-zKAlwhfABjcWR_0 2026-02-24T22:34:25.967 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11218049792832725055', time=2026-02-25T00:26:19.465524+0000, chain.objs.size()=1 2026-02-24T22:34:25.967 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1659__shadow_.lAXLrLjtpL1-g1BOwonXlZgb6cihAqp_0 2026-02-24T22:34:25.967 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4607209710500423994', time=2026-02-25T00:26:19.465900+0000, chain.objs.size()=1 2026-02-24T22:34:25.967 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1659__shadow_.zEm87OThNfd-tH2zDmPIV7_m_nyZU8T_0 2026-02-24T22:34:25.967 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3947807663921351017', time=2026-02-25T00:26:19.469778+0000, chain.objs.size()=1 2026-02-24T22:34:25.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1659__shadow_.6gJJ7WcEUUx_WzvN6KKMfzV11STe_Mo_0 2026-02-24T22:34:25.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3744204971335336015', time=2026-02-25T00:26:21.137731+0000, chain.objs.size()=1 2026-02-24T22:34:25.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1662__shadow_.R3vOMMFj7CvXRek-BxLhdU-4OEWm5tL_0 2026-02-24T22:34:25.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17170976168807475183', time=2026-02-25T00:26:26.429486+0000, chain.objs.size()=1 2026-02-24T22:34:25.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1671__shadow_.hgv-bgQtII99A0_wsvR4xQjZIlR_HqH_0 2026-02-24T22:34:25.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8180450841459196271', time=2026-02-25T00:26:44.794687+0000, chain.objs.size()=1 2026-02-24T22:34:25.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.KLo5FvfhYhhyGEalxEAJQHT0BJzeXzS_0 2026-02-24T22:34:25.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5370999937320863626', time=2026-02-25T00:26:44.803176+0000, chain.objs.size()=1 2026-02-24T22:34:25.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.EFxUDEuiatNAfLdyugs9Iqv2W0QaWKS_0 2026-02-24T22:34:25.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15319585179318777756', time=2026-02-25T00:26:44.814458+0000, chain.objs.size()=1 2026-02-24T22:34:25.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Gk-BtbyPEGLMeBqqCS_fAfv_7TB-A1c_0 2026-02-24T22:34:25.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4896035610673914083', time=2026-02-25T00:26:44.824310+0000, chain.objs.size()=1 2026-02-24T22:34:25.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.KkwVQpc1gEDW7NFeIDT7f_w0_Ha3Qyi_0 2026-02-24T22:34:25.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1756728072141944680', time=2026-02-25T00:26:44.851656+0000, chain.objs.size()=1 2026-02-24T22:34:25.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.XrL-6iQ6T90R79MaG0s8XVGwnmAjZRy_0 2026-02-24T22:34:25.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17135807397854221044', time=2026-02-25T00:26:44.857227+0000, chain.objs.size()=1 2026-02-24T22:34:25.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.g_Kjf4muMS6Bco9cXArAfa5uD2xyHU7_0 2026-02-24T22:34:25.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8852628509968969222', time=2026-02-25T00:26:44.977633+0000, chain.objs.size()=1 2026-02-24T22:34:25.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.sctGXfiN4yuj0iNu2KISEOMUUh1LtZx_0 2026-02-24T22:34:25.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14380036078620010352', time=2026-02-25T00:26:45.014564+0000, chain.objs.size()=1 2026-02-24T22:34:25.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.RNtGmlTU1hQnOLeHPLSSzPYwtqf3WHT_0 2026-02-24T22:34:25.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2144478166221325693', time=2026-02-25T00:26:45.017814+0000, chain.objs.size()=1 2026-02-24T22:34:25.968 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.5kOvENrAkFqYPJ5nP47jNg9Urg0OGUG_0 2026-02-24T22:34:25.969 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.699453771850171035', time=2026-02-25T00:26:45.018156+0000, chain.objs.size()=1 2026-02-24T22:34:25.969 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.w-eHixxB83cE-6aqj1U0-86KX6yotLh_0 2026-02-24T22:34:25.969 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2782704735189787689', time=2026-02-25T00:26:45.021691+0000, chain.objs.size()=1 2026-02-24T22:34:25.969 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.ovl0lyjYtX1Tu1n7_qQZEPY7rpTTolc_0 2026-02-24T22:34:25.969 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3645671680494403321', time=2026-02-25T00:26:45.022551+0000, chain.objs.size()=1 2026-02-24T22:34:25.969 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.EuR84cjXRHbWaIfSxcaCr4XX_1L-c7g_0 2026-02-24T22:34:25.969 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.5946465234311431550', time=2026-02-25T00:26:45.034415+0000, chain.objs.size()=1 2026-02-24T22:34:25.969 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.gABt4heouHZCdPIkFOVJ-zMwYjHCNB1_0 2026-02-24T22:34:25.969 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12427689720576775237', time=2026-02-25T00:26:45.110412+0000, chain.objs.size()=1 2026-02-24T22:34:25.969 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.BET1XkAfQ-tvNEsRbtkMzg-d0zkVbs3_0 2026-02-24T22:34:25.969 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15699032890616325785', time=2026-02-25T00:26:45.127806+0000, chain.objs.size()=1 2026-02-24T22:34:25.969 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.VUFmWM3ntX6_TSM6QWauDoWbAu5TWHS_0 2026-02-24T22:34:25.969 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16860686450432550710', time=2026-02-25T00:26:45.190263+0000, chain.objs.size()=1 2026-02-24T22:34:25.969 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.2RYMYWsmdzoACC-2YuE2GnMXeZYOGtF_0 2026-02-24T22:34:25.969 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.288794484389346345', time=2026-02-25T00:26:45.266920+0000, chain.objs.size()=1 2026-02-24T22:34:25.969 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.yP9VLhT5F4Lyu4cmqiP-dDt8zo7sPS2_0 2026-02-24T22:34:25.969 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1894291069165803719', time=2026-02-25T00:26:45.305461+0000, chain.objs.size()=1 2026-02-24T22:34:25.969 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Cz9bzXLA_roFsR09PJR_7837FkMUW-G_0 2026-02-24T22:34:25.969 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2745793763691411051', time=2026-02-25T00:26:45.330155+0000, chain.objs.size()=1 2026-02-24T22:34:25.969 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.6nQ4keE32ffj2RFbae_m02h3HDXZBej_0 2026-02-24T22:34:25.969 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4943126649155308715', time=2026-02-25T00:26:45.440799+0000, chain.objs.size()=1 2026-02-24T22:34:25.969 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.VJAYLVNxpSqw0O4mVP5rwVaZ9y5WqSF_0 2026-02-24T22:34:25.970 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3958310132999911060', time=2026-02-25T00:26:45.447650+0000, chain.objs.size()=1 2026-02-24T22:34:25.970 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.zq61NkogPGDDNH7Zs_m4lnqH-2elzpD_0 2026-02-24T22:34:25.970 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16073268321959060986', time=2026-02-25T00:26:45.463291+0000, chain.objs.size()=1 2026-02-24T22:34:25.970 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.979+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.DNPjoGUT6Qu3yxf5_ZreQXkpu6p5yOz_0 2026-02-24T22:34:25.970 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3250868838441858154', time=2026-02-25T00:26:45.466279+0000, chain.objs.size()=1 2026-02-24T22:34:25.970 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.92h5K21afOOnhjXZQoTU4DVzxNpQ60p_0 2026-02-24T22:34:25.970 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.9338113048343255107', time=2026-02-25T00:26:45.483102+0000, chain.objs.size()=1 2026-02-24T22:34:25.970 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.GXZcZ3HmIkL0zyfOKkuBpiDrNvqKD2l_0 2026-02-24T22:34:25.970 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11305212043004328267', time=2026-02-25T00:26:45.489932+0000, chain.objs.size()=1 2026-02-24T22:34:25.970 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.MdrnVHGinDLsQWFBk0iEwAkyQ2swuZm_0 2026-02-24T22:34:25.970 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12365544005981279050', time=2026-02-25T00:26:45.493923+0000, chain.objs.size()=1 2026-02-24T22:34:25.970 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.dHx7soE_LczsZN3yV6zWzLThIl5xPLp_0 2026-02-24T22:34:25.971 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6393559564220188600', time=2026-02-25T00:26:45.577456+0000, chain.objs.size()=1 2026-02-24T22:34:25.971 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.YSbG2gWa7ieKul3am4rB6R4T2dxxJ2E_0 2026-02-24T22:34:25.971 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17513151056927799721', time=2026-02-25T00:26:45.601282+0000, chain.objs.size()=1 2026-02-24T22:34:25.971 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.DqUzCEDBqZTC4Wc6KVmmlvS3ABH-4OX_0 2026-02-24T22:34:25.971 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3768768566853364933', time=2026-02-25T00:26:45.822048+0000, chain.objs.size()=1 2026-02-24T22:34:25.971 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.2RaOMTNI9Za3LGUB_qIVNzt944LKic2_0 2026-02-24T22:34:25.971 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13155245857304155177', time=2026-02-25T00:26:45.827832+0000, chain.objs.size()=1 2026-02-24T22:34:25.971 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.DfVHSqw5AMjabW9Ni-nMyYejnPtIZrL_0 2026-02-24T22:34:25.971 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12269871960991573949', time=2026-02-25T00:26:45.943548+0000, chain.objs.size()=1 2026-02-24T22:34:25.971 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.UcdHO4qAwY_jAOdDnzaU17-CcL1mK9e_0 2026-02-24T22:34:25.971 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15467547103686855443', time=2026-02-25T00:26:46.011148+0000, chain.objs.size()=1 2026-02-24T22:34:25.971 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Mv48jcODHKRdA8R3WxMe4n5kHvhRjts_0 2026-02-24T22:34:25.971 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.801780130463959776', time=2026-02-25T00:26:46.020009+0000, chain.objs.size()=1 2026-02-24T22:34:25.971 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1688__shadow_.Ji9iwzYTNvJS2njw3f_Krur2SeRdsw0_0 2026-02-24T22:34:25.971 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.18028786325568983507', time=2026-02-25T00:26:53.168632+0000, chain.objs.size()=1 2026-02-24T22:34:25.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.UzfPCO7rqvtvkBosCHGANt0cWgUTJ9O_0 2026-02-24T22:34:25.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13837651179259534568', time=2026-02-25T00:26:53.193385+0000, chain.objs.size()=1 2026-02-24T22:34:25.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.2xdYjL7Sl0pxZS-90Ocj9JRe0RVNkGX_0 2026-02-24T22:34:25.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8700228915502027584', time=2026-02-25T00:26:53.343521+0000, chain.objs.size()=1 2026-02-24T22:34:25.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.I_brnnA0Ba8yntPazhfCsPVUWP26s5a_0 2026-02-24T22:34:25.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14567745216989917991', time=2026-02-25T00:26:53.386640+0000, chain.objs.size()=1 2026-02-24T22:34:25.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.TYPOxw2eAhuk0HJ7i7xYPPlhpWgJVUE_0 2026-02-24T22:34:25.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15390022098956249062', time=2026-02-25T00:26:53.396592+0000, chain.objs.size()=1 2026-02-24T22:34:25.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.qB-WowhP1RXAKjDaSoS0mVHRWL--OLm_0 2026-02-24T22:34:25.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.16869570472974626897', time=2026-02-25T00:26:53.581411+0000, chain.objs.size()=1 2026-02-24T22:34:25.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.1g31agkyuFm6oJHLJCn6MVJiOSKg3Tv_0 2026-02-24T22:34:25.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1593413948356988200', time=2026-02-25T00:26:53.623223+0000, chain.objs.size()=1 2026-02-24T22:34:25.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.C4_o-h-3pyPIzv0ypABRz6KE-hhkrHy_0 2026-02-24T22:34:25.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12388575231075964452', time=2026-02-25T00:26:53.738240+0000, chain.objs.size()=1 2026-02-24T22:34:25.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.jjhbR_2Hlthf67_dg_WWuQW4_gZFyBh_0 2026-02-24T22:34:25.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8927130218464147824', time=2026-02-25T00:26:53.842245+0000, chain.objs.size()=1 2026-02-24T22:34:25.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.NSiOA3oAqzrXupjCOACflOp8lNd80UR_0 2026-02-24T22:34:25.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14281641662338372262', time=2026-02-25T00:26:53.848185+0000, chain.objs.size()=1 2026-02-24T22:34:25.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.xfvb3BDs7Wr0uGp9jzQhWiq3TEgbBaC_0 2026-02-24T22:34:25.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4581267101861434808', time=2026-02-25T00:26:53.937941+0000, chain.objs.size()=1 2026-02-24T22:34:25.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.vPcmm2RVCNad5nCiAWcMZ6qvQvyR2dI_0 2026-02-24T22:34:25.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14852444536127649355', time=2026-02-25T00:26:53.959692+0000, chain.objs.size()=1 2026-02-24T22:34:25.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.omntAX8yZeAMtZiEEu8XKjPrfRtHMXU_0 2026-02-24T22:34:25.972 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12359587679396964441', time=2026-02-25T00:26:54.028257+0000, chain.objs.size()=1 2026-02-24T22:34:25.973 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.XxjCTnU5N-cQrhA8t11HQGSC1Jcd3ns_0 2026-02-24T22:34:25.973 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17215181898695066357', time=2026-02-25T00:26:54.115904+0000, chain.objs.size()=1 2026-02-24T22:34:25.973 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.XKan_E-VIL_b9n8yJBPES4ZmKwK58j__0 2026-02-24T22:34:25.973 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2880807843034942717', time=2026-02-25T00:26:54.118105+0000, chain.objs.size()=1 2026-02-24T22:34:25.973 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.5N6UxquSmSUBQok4E2m_nIrxBIX6gbU_0 2026-02-24T22:34:25.974 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.983+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 0/38786 2026-02-24T22:34:25.975 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=29, truncated=0, next_marker='' 2026-02-24T22:34:25.975 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4901874683733478086', time=2026-02-25T00:26:54.151393+0000, chain.objs.size()=1 2026-02-24T22:34:25.975 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.4hJ7soQSBnPke7rx86fBwEXEudP1tGc_0 2026-02-24T22:34:25.975 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6042994653824544148', time=2026-02-25T00:26:54.152415+0000, chain.objs.size()=1 2026-02-24T22:34:25.975 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.Jlnjytb5L0FHgAP_Fxdb6Rl_WcwwvRl_0 2026-02-24T22:34:25.975 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14907573093275005763', time=2026-02-25T00:26:54.190633+0000, chain.objs.size()=1 2026-02-24T22:34:25.975 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.7ujHH9TE35hhi8SDVJYr8gW8hkotJ7E_0 2026-02-24T22:34:25.975 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.12675907037874168490', time=2026-02-25T00:26:54.195967+0000, chain.objs.size()=1 2026-02-24T22:34:25.975 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.-mkfj_ArND83CfKmSN6gYRhs7UuJXEf_0 2026-02-24T22:34:25.975 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11856747205570541819', time=2026-02-25T00:26:54.198521+0000, chain.objs.size()=1 2026-02-24T22:34:25.975 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.IaMjOH3t6_Uh9Kdov1xeEd0hHzw6pHf_0 2026-02-24T22:34:25.975 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17892755469185919513', time=2026-02-25T00:26:54.224008+0000, chain.objs.size()=1 2026-02-24T22:34:25.975 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.fNQjgQTYPOS73OPI7VS1hsHrLnhMI7g_0 2026-02-24T22:34:25.975 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11060618233286506734', time=2026-02-25T00:26:54.320654+0000, chain.objs.size()=1 2026-02-24T22:34:25.976 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.M1EVkI4xeA7TbGMEG9YRzpOC0PnHNVi_0 2026-02-24T22:34:25.976 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1396993689148993569', time=2026-02-25T00:26:54.328763+0000, chain.objs.size()=1 2026-02-24T22:34:25.976 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.BE9mjSmsDJi4RdaTqMoUCBgqsbcUr9v_0 2026-02-24T22:34:25.976 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.6641043675470366816', time=2026-02-25T00:26:54.361165+0000, chain.objs.size()=1 2026-02-24T22:34:25.976 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.rLmh2KiVjru9qS9D_ARYHIkj9WxU7bI_0 2026-02-24T22:34:25.976 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.11231212858205187089', time=2026-02-25T00:26:54.399111+0000, chain.objs.size()=1 2026-02-24T22:34:25.976 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.LEhwAEEuom4eNy4JDbSMWYxuxYXpmjT_0 2026-02-24T22:34:25.976 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.4793608652401404557', time=2026-02-25T00:26:54.472196+0000, chain.objs.size()=1 2026-02-24T22:34:25.976 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.pf_3dsQWiNV3Ctt8NRmf1YxiTI5huFF_0 2026-02-24T22:34:25.976 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.2544163223851542141', time=2026-02-25T00:26:54.474460+0000, chain.objs.size()=1 2026-02-24T22:34:25.976 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.E19Ptd6xixDbGV5l3FfxI4IorRi_1SG_0 2026-02-24T22:34:25.976 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1771986981991627894', time=2026-02-25T00:26:54.478439+0000, chain.objs.size()=1 2026-02-24T22:34:25.976 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1689__shadow_.H7qGoBmXuufWhqV80GOQswZyCY29pZj_0 2026-02-24T22:34:25.976 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15878423653768510143', time=2026-02-25T00:27:24.350249+0000, chain.objs.size()=1 2026-02-24T22:34:25.976 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1742__shadow_.jsMNSwyE0MvW7z1EdX75FiyjyLex9WZ_0 2026-02-24T22:34:25.976 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1257766121573887528', time=2026-02-25T00:27:44.349939+0000, chain.objs.size()=1 2026-02-24T22:34:25.976 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1772__shadow_.GKslaTLZf0FGYFiWi-302gpruxHQTbr_0 2026-02-24T22:34:25.976 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8085755157550514844', time=2026-02-25T00:28:05.234341+0000, chain.objs.size()=1 2026-02-24T22:34:25.976 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1804__shadow_.vURzxjQ5AlSGG170cNKN8LHmtcjvA6z_0 2026-02-24T22:34:25.976 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7760788804746250836', time=2026-02-25T00:28:31.293219+0000, chain.objs.size()=1 2026-02-24T22:34:25.976 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1848__shadow_.GjfU-CWvjc6WP6H5J94CjphPqOdNoKU_0 2026-02-24T22:34:25.977 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7909515451878191788', time=2026-02-25T00:28:35.128985+0000, chain.objs.size()=1 2026-02-24T22:34:25.977 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1853__shadow_.Y0h9SDZ-1Cc8jpeThxa2PRTp9RUo8Au_0 2026-02-24T22:34:25.977 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.3473366638037321673', time=2026-02-25T00:28:57.069591+0000, chain.objs.size()=3 2026-02-24T22:34:25.977 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1887__multipart_mymultipart.2~OOC-jbL73u0ktIbS67kedeIt0shEm7n.1 2026-02-24T22:34:25.977 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1887__multipart_mymultipart.2~OOC-jbL73u0ktIbS67kedeIt0shEm7n.2 2026-02-24T22:34:25.977 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1887__multipart_mymultipart.2~OOC-jbL73u0ktIbS67kedeIt0shEm7n.3 2026-02-24T22:34:25.977 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.14207077566807533557', time=2026-02-25T00:29:41.177005+0000, chain.objs.size()=4 2026-02-24T22:34:25.977 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1902__multipart_mymultipart.2~B0zJ7cfdV9nHP3nWB3MAyKQdMdY0AHX.1 2026-02-24T22:34:25.977 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1902__multipart_mymultipart.2~B0zJ7cfdV9nHP3nWB3MAyKQdMdY0AHX.2 2026-02-24T22:34:25.977 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.987+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1902__multipart_mymultipart.evThDCyiv6eEwOOeveP9ctDKaOMHt4Ef.3 2026-02-24T22:34:25.978 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.991+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1902__multipart_mymultipart.2~B0zJ7cfdV9nHP3nWB3MAyKQdMdY0AHX.4 2026-02-24T22:34:25.978 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.991+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8999359781284882146', time=2026-02-25T00:30:20.545789+0000, chain.objs.size()=1 2026-02-24T22:34:25.978 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.991+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1937__shadow:DLm63taBAB49-s4Kl7EZTeduwQs3.Jl_.Zv6g2uuALYgDHjwNek7_k1QO9TFsnWf_0 2026-02-24T22:34:25.978 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.991+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.13016583539765835043', time=2026-02-25T00:30:32.326051+0000, chain.objs.size()=1 2026-02-24T22:34:25.978 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.991+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1946__shadow:GKqog0EiYW.udMlOlRFbxNYS-K2rd27_.7OrXLpjD_ST-nJv85sQOPRSVDxKs0kl_0 2026-02-24T22:34:25.978 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.991+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.10137599665503246447', time=2026-02-25T00:30:40.994654+0000, chain.objs.size()=1 2026-02-24T22:34:25.978 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.991+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:0zT6B1ZYSUjCUceV8UqIsjWR7nMRbA5_.P23MD5Qp9NR9KMhZ_sfNaHeiyFVnxG8_0 2026-02-24T22:34:25.978 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.991+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.73599257937262635', time=2026-02-25T00:30:46.206165+0000, chain.objs.size()=1 2026-02-24T22:34:25.978 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.991+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1947__shadow:z4.LxWWsJdUO5uVF4qVcs-kE1klcCTM_.8t40yRqFr3xcuegDT13ZnvTydUHbnvj_0 2026-02-24T22:34:25.979 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.991+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.17949354112800964272', time=2026-02-25T00:30:53.836813+0000, chain.objs.size()=1 2026-02-24T22:34:25.979 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.991+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1953__shadow:rh47Y.wphEylY1EmiEY4lWh-fX6whUR_.0iCPMR1eX_Xel1Pwad9l5IFg44FFJU__0 2026-02-24T22:34:25.979 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.991+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.8231146337043644060', time=2026-02-25T00:30:56.358104+0000, chain.objs.size()=1 2026-02-24T22:34:25.979 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.991+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1956__shadow:TBohRu7oTR0iKCq1pY18qPAuY3qkeni_.GMGRT3HctO6Dch87u0AtXgnuWO4PX9z_0 2026-02-24T22:34:25.979 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.991+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7057996953896750829', time=2026-02-25T00:30:57.746267+0000, chain.objs.size()=1 2026-02-24T22:34:25.979 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.991+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:Nfbm6x.zs54IE0.iocNOCcKOsQ-C6wb_.Fpu2rq-RfC7cH2TDPdmgAtDc6AMrYNp_0 2026-02-24T22:34:25.979 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.991+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.7057996953896750829', time=2026-02-25T00:30:57.746811+0000, chain.objs.size()=1 2026-02-24T22:34:25.979 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.991+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1957__shadow:Nfbm6x.zs54IE0.iocNOCcKOsQ-C6wb_.Fpu2rq-RfC7cH2TDPdmgAtDc6AMrYNp_0 2026-02-24T22:34:25.979 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.991+0000 7fa0ccd13a80 20 garbage collection: RGWGC::process iterating over entry tag='301937db-3a1f-489c-b387-e7c21a76f9b9.4644.15434292545917508650', time=2026-02-25T00:34:03.749327+0000, chain.objs.size()=1 2026-02-24T22:34:25.979 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.991+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:301937db-3a1f-489c-b387-e7c21a76f9b9.4644.1986__shadow_.HLCtWu5Nse7f2VSy3-DckkxZoZRxpfL_0 2026-02-24T22:34:25.980 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.991+0000 7fa0ccd13a80 5 garbage collection: RGWGC::process removing entries, marker: 2026-02-24T22:34:25.985 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.995+0000 7fa0ccd13a80 20 remove_watcher() i=1 2026-02-24T22:34:25.985 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.995+0000 7fa0ccd13a80 2 removed watcher, disabling cache 2026-02-24T22:34:25.985 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.995+0000 7fa0ccd13a80 20 remove_watcher() i=4 2026-02-24T22:34:25.985 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.995+0000 7fa0ccd13a80 20 remove_watcher() i=3 2026-02-24T22:34:25.985 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.995+0000 7fa0ccd13a80 20 remove_watcher() i=2 2026-02-24T22:34:25.985 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.995+0000 7fa0ccd13a80 20 remove_watcher() i=6 2026-02-24T22:34:25.985 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.995+0000 7fa0ccd13a80 20 remove_watcher() i=0 2026-02-24T22:34:25.985 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.995+0000 7fa0ccd13a80 20 remove_watcher() i=7 2026-02-24T22:34:25.985 INFO:teuthology.orchestra.run.trial046.stderr:2026-02-24T22:34:25.995+0000 7fa0ccd13a80 20 remove_watcher() i=5 2026-02-24T22:34:25.991 DEBUG:teuthology.run_tasks:Unwinding manager openssl_keys 2026-02-24T22:34:26.003 DEBUG:teuthology.orchestra.run.trial046:> rm -f /home/ubuntu/cephtest/ca/root.crt /home/ubuntu/cephtest/ca/root.key 2026-02-24T22:34:26.038 DEBUG:teuthology.orchestra.run.trial046:> rmdir --ignore-fail-on-non-empty /home/ubuntu/cephtest/ca 2026-02-24T22:34:26.085 DEBUG:teuthology.orchestra.run.trial046:> rm -f /home/ubuntu/cephtest/ca/rgw.client.0.crt /home/ubuntu/cephtest/ca/rgw.client.0.key 2026-02-24T22:34:26.129 DEBUG:teuthology.orchestra.run.trial046:> rmdir --ignore-fail-on-non-empty /home/ubuntu/cephtest/ca 2026-02-24T22:34:26.174 DEBUG:teuthology.orchestra.run.trial046:> sudo rm /usr/local/share/ca-certificates/root.crt 2026-02-24T22:34:26.222 DEBUG:teuthology.orchestra.run.trial046:> sudo update-ca-certificates 2026-02-24T22:34:26.274 INFO:teuthology.orchestra.run.trial046.stdout:Updating certificates in /etc/ssl/certs... 2026-02-24T22:34:26.723 INFO:teuthology.orchestra.run.trial046.stdout:0 added, 0 removed; done. 2026-02-24T22:34:26.723 INFO:teuthology.orchestra.run.trial046.stdout:Running hooks in /etc/ca-certificates/update.d... 2026-02-24T22:34:26.725 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:26.824 INFO:teuthology.orchestra.run.trial046.stdout:done. 2026-02-24T22:34:26.824 INFO:teuthology.orchestra.run.trial046.stdout:done. 2026-02-24T22:34:26.827 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-24T22:34:26.845 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-24T22:34:26.845 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-24T22:34:27.043 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:27.056 INFO:teuthology.orchestra.run.trial046.stderr:dumped all 2026-02-24T22:34:27.063 INFO:teuthology.orchestra.run.trial046.stdout:{"pg_ready":true,"pg_map":{"version":1186,"stamp":"2026-02-24T22:34:26.023106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":523579384,"num_objects":9510,"num_object_clones":0,"num_object_copies":19020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9510,"num_whiteouts":0,"num_read":202525,"num_read_kb":1382960,"num_write":156802,"num_write_kb":706681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":1543,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11819,"ondisk_log_size":11819,"up":786,"acting":786,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":786,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2339352,"kb_used_data":1113112,"kb_used_omap":37260,"kb_used_meta":1188851,"kb_avail":752635368,"statfs":{"total":773094113280,"available":770698616832,"internally_reserved":0,"allocated":1139826688,"data_stored":1067288018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38154327,"internal_metadata":1217384361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30542903,"num_objects":-41,"num_object_clones":0,"num_object_copies":-82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-41,"num_whiteouts":0,"num_read":1479,"num_read_kb":60619,"num_write":280,"num_write_kb":29926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003202"},"pg_stats":[{"pgid":"82.2","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744748+0000","last_change":"2026-02-24T22:23:29.198729+0000","last_active":"2026-02-24T22:31:49.744748+0000","last_peered":"2026-02-24T22:31:49.744748+0000","last_clean":"2026-02-24T22:31:49.744748+0000","last_became_active":"2026-02-24T22:23:29.198476+0000","last_became_peered":"2026-02-24T22:23:29.198476+0000","last_unstale":"2026-02-24T22:31:49.744748+0000","last_undegraded":"2026-02-24T22:31:49.744748+0000","last_fullsized":"2026-02-24T22:31:49.744748+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:12:35.165001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.1","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.730924+0000","last_change":"2026-02-24T22:15:25.830397+0000","last_active":"2026-02-24T22:32:09.730924+0000","last_peered":"2026-02-24T22:32:09.730924+0000","last_clean":"2026-02-24T22:32:09.730924+0000","last_became_active":"2026-02-24T22:15:25.830000+0000","last_became_peered":"2026-02-24T22:15:25.830000+0000","last_unstale":"2026-02-24T22:32:09.730924+0000","last_undegraded":"2026-02-24T22:32:09.730924+0000","last_fullsized":"2026-02-24T22:32:09.730924+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:41:30.400463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.0","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:29.723258+0000","last_change":"2026-02-24T22:15:22.817921+0000","last_active":"2026-02-24T22:31:29.723258+0000","last_peered":"2026-02-24T22:31:29.723258+0000","last_clean":"2026-02-24T22:31:29.723258+0000","last_became_active":"2026-02-24T22:15:22.817446+0000","last_became_peered":"2026-02-24T22:15:22.817446+0000","last_unstale":"2026-02-24T22:31:29.723258+0000","last_undegraded":"2026-02-24T22:31:29.723258+0000","last_fullsized":"2026-02-24T22:31:29.723258+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:25:30.206672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.1f","version":"312'20","reported_seq":120,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.733669+0000","last_change":"2026-02-24T22:13:43.761604+0000","last_active":"2026-02-24T22:30:24.733669+0000","last_peered":"2026-02-24T22:30:24.733669+0000","last_clean":"2026-02-24T22:30:24.733669+0000","last_became_active":"2026-02-24T22:13:43.761120+0000","last_became_peered":"2026-02-24T22:13:43.761120+0000","last_unstale":"2026-02-24T22:30:24.733669+0000","last_undegraded":"2026-02-24T22:30:24.733669+0000","last_fullsized":"2026-02-24T22:30:24.733669+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:48:55.248638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":168,"num_read_kb":114,"num_write":47,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.3","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742865+0000","last_change":"2026-02-24T22:23:29.213779+0000","last_active":"2026-02-24T22:31:49.742865+0000","last_peered":"2026-02-24T22:31:49.742865+0000","last_clean":"2026-02-24T22:31:49.742865+0000","last_became_active":"2026-02-24T22:23:29.213534+0000","last_became_peered":"2026-02-24T22:23:29.213534+0000","last_unstale":"2026-02-24T22:31:49.742865+0000","last_undegraded":"2026-02-24T22:31:49.742865+0000","last_fullsized":"2026-02-24T22:31:49.742865+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:38:56.711711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.0","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.743144+0000","last_change":"2026-02-24T22:15:25.827702+0000","last_active":"2026-02-24T22:31:34.743144+0000","last_peered":"2026-02-24T22:31:34.743144+0000","last_clean":"2026-02-24T22:31:34.743144+0000","last_became_active":"2026-02-24T22:15:25.827355+0000","last_became_peered":"2026-02-24T22:15:25.827355+0000","last_unstale":"2026-02-24T22:31:34.743144+0000","last_undegraded":"2026-02-24T22:31:34.743144+0000","last_fullsized":"2026-02-24T22:31:34.743144+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:22:12.090995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.1","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:04.744507+0000","last_change":"2026-02-24T22:15:22.818757+0000","last_active":"2026-02-24T22:32:04.744507+0000","last_peered":"2026-02-24T22:32:04.744507+0000","last_clean":"2026-02-24T22:32:04.744507+0000","last_became_active":"2026-02-24T22:15:22.818519+0000","last_became_peered":"2026-02-24T22:15:22.818519+0000","last_unstale":"2026-02-24T22:32:04.744507+0000","last_undegraded":"2026-02-24T22:32:04.744507+0000","last_fullsized":"2026-02-24T22:32:04.744507+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:17:56.378157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.1e","version":"312'9","reported_seq":88,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.745172+0000","last_change":"2026-02-24T22:13:43.757760+0000","last_active":"2026-02-24T22:31:49.745172+0000","last_peered":"2026-02-24T22:31:49.745172+0000","last_clean":"2026-02-24T22:31:49.745172+0000","last_became_active":"2026-02-24T22:13:43.757543+0000","last_became_peered":"2026-02-24T22:13:43.757543+0000","last_unstale":"2026-02-24T22:31:49.745172+0000","last_undegraded":"2026-02-24T22:31:49.745172+0000","last_fullsized":"2026-02-24T22:31:49.745172+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:44:23.087405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":38,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.0","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.723743+0000","last_change":"2026-02-24T22:23:29.212801+0000","last_active":"2026-02-24T22:31:34.723743+0000","last_peered":"2026-02-24T22:31:34.723743+0000","last_clean":"2026-02-24T22:31:34.723743+0000","last_became_active":"2026-02-24T22:23:29.212043+0000","last_became_peered":"2026-02-24T22:23:29.212043+0000","last_unstale":"2026-02-24T22:31:34.723743+0000","last_undegraded":"2026-02-24T22:31:34.723743+0000","last_fullsized":"2026-02-24T22:31:34.723743+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:45:48.953242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.3","version":"307'5","reported_seq":50,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.747921+0000","last_change":"2026-02-24T22:15:25.827590+0000","last_active":"2026-02-24T22:32:09.747921+0000","last_peered":"2026-02-24T22:32:09.747921+0000","last_clean":"2026-02-24T22:32:09.747921+0000","last_became_active":"2026-02-24T22:15:25.827216+0000","last_became_peered":"2026-02-24T22:15:25.827216+0000","last_unstale":"2026-02-24T22:32:09.747921+0000","last_undegraded":"2026-02-24T22:32:09.747921+0000","last_fullsized":"2026-02-24T22:32:09.747921+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:08:31.068519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485760,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":25600,"num_write":10,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.2","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:04.726927+0000","last_change":"2026-02-24T22:15:22.818494+0000","last_active":"2026-02-24T22:32:04.726927+0000","last_peered":"2026-02-24T22:32:04.726927+0000","last_clean":"2026-02-24T22:32:04.726927+0000","last_became_active":"2026-02-24T22:15:22.818194+0000","last_became_peered":"2026-02-24T22:15:22.818194+0000","last_unstale":"2026-02-24T22:32:04.726927+0000","last_undegraded":"2026-02-24T22:32:04.726927+0000","last_fullsized":"2026-02-24T22:32:04.726927+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:30:45.313454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.1d","version":"312'22","reported_seq":121,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.737198+0000","last_change":"2026-02-24T22:13:43.757707+0000","last_active":"2026-02-24T22:31:14.737198+0000","last_peered":"2026-02-24T22:31:14.737198+0000","last_clean":"2026-02-24T22:31:14.737198+0000","last_became_active":"2026-02-24T22:13:43.757278+0000","last_became_peered":"2026-02-24T22:13:43.757278+0000","last_unstale":"2026-02-24T22:31:14.737198+0000","last_undegraded":"2026-02-24T22:31:14.737198+0000","last_fullsized":"2026-02-24T22:31:14.737198+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:52:33.539660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":103,"num_write":54,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.1","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742841+0000","last_change":"2026-02-24T22:23:29.214295+0000","last_active":"2026-02-24T22:31:49.742841+0000","last_peered":"2026-02-24T22:31:49.742841+0000","last_clean":"2026-02-24T22:31:49.742841+0000","last_became_active":"2026-02-24T22:23:29.214064+0000","last_became_peered":"2026-02-24T22:23:29.214064+0000","last_unstale":"2026-02-24T22:31:49.742841+0000","last_undegraded":"2026-02-24T22:31:49.742841+0000","last_fullsized":"2026-02-24T22:31:49.742841+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:03:04.365969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.2","version":"307'1","reported_seq":35,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.735734+0000","last_change":"2026-02-24T22:15:25.833367+0000","last_active":"2026-02-24T22:32:09.735734+0000","last_peered":"2026-02-24T22:32:09.735734+0000","last_clean":"2026-02-24T22:32:09.735734+0000","last_became_active":"2026-02-24T22:15:25.832809+0000","last_became_peered":"2026-02-24T22:15:25.832809+0000","last_unstale":"2026-02-24T22:32:09.735734+0000","last_undegraded":"2026-02-24T22:32:09.735734+0000","last_fullsized":"2026-02-24T22:32:09.735734+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:28:13.551199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":4096,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"80.3","version":"307'2","reported_seq":43,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:04.726940+0000","last_change":"2026-02-24T22:15:22.818401+0000","last_active":"2026-02-24T22:32:04.726940+0000","last_peered":"2026-02-24T22:32:04.726940+0000","last_clean":"2026-02-24T22:32:04.726940+0000","last_became_active":"2026-02-24T22:15:22.818089+0000","last_became_peered":"2026-02-24T22:15:22.818089+0000","last_unstale":"2026-02-24T22:32:04.726940+0000","last_undegraded":"2026-02-24T22:32:04.726940+0000","last_fullsized":"2026-02-24T22:32:04.726940+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:40:25.179819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.1c","version":"312'14","reported_seq":101,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.739030+0000","last_change":"2026-02-24T22:13:43.760682+0000","last_active":"2026-02-24T22:31:14.739030+0000","last_peered":"2026-02-24T22:31:14.739030+0000","last_clean":"2026-02-24T22:31:14.739030+0000","last_became_active":"2026-02-24T22:13:43.759958+0000","last_became_peered":"2026-02-24T22:13:43.759958+0000","last_unstale":"2026-02-24T22:31:14.739030+0000","last_undegraded":"2026-02-24T22:31:14.739030+0000","last_fullsized":"2026-02-24T22:31:14.739030+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:24:39.289428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":75,"num_write":36,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.6","version":"309'1","reported_seq":23,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.730973+0000","last_change":"2026-02-24T22:23:29.215576+0000","last_active":"2026-02-24T22:32:09.730973+0000","last_peered":"2026-02-24T22:32:09.730973+0000","last_clean":"2026-02-24T22:32:09.730973+0000","last_became_active":"2026-02-24T22:23:29.215327+0000","last_became_peered":"2026-02-24T22:23:29.215327+0000","last_unstale":"2026-02-24T22:32:09.730973+0000","last_undegraded":"2026-02-24T22:32:09.730973+0000","last_fullsized":"2026-02-24T22:32:09.730973+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:51:01.281339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.5","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:14.745596+0000","last_change":"2026-02-24T22:15:25.832444+0000","last_active":"2026-02-24T22:32:14.745596+0000","last_peered":"2026-02-24T22:32:14.745596+0000","last_clean":"2026-02-24T22:32:14.745596+0000","last_became_active":"2026-02-24T22:15:25.831975+0000","last_became_peered":"2026-02-24T22:15:25.831975+0000","last_unstale":"2026-02-24T22:32:14.745596+0000","last_undegraded":"2026-02-24T22:32:14.745596+0000","last_fullsized":"2026-02-24T22:32:14.745596+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:04:17.334836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"80.4","version":"307'2","reported_seq":43,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:04.747123+0000","last_change":"2026-02-24T22:15:22.812647+0000","last_active":"2026-02-24T22:32:04.747123+0000","last_peered":"2026-02-24T22:32:04.747123+0000","last_clean":"2026-02-24T22:32:04.747123+0000","last_became_active":"2026-02-24T22:15:22.812270+0000","last_became_peered":"2026-02-24T22:15:22.812270+0000","last_unstale":"2026-02-24T22:32:04.747123+0000","last_undegraded":"2026-02-24T22:32:04.747123+0000","last_fullsized":"2026-02-24T22:32:04.747123+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:42:27.774508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.1b","version":"312'18","reported_seq":112,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.741475+0000","last_change":"2026-02-24T22:13:43.755424+0000","last_active":"2026-02-24T22:31:14.741475+0000","last_peered":"2026-02-24T22:31:14.741475+0000","last_clean":"2026-02-24T22:31:14.741475+0000","last_became_active":"2026-02-24T22:13:43.755170+0000","last_became_peered":"2026-02-24T22:13:43.755170+0000","last_unstale":"2026-02-24T22:31:14.741475+0000","last_undegraded":"2026-02-24T22:31:14.741475+0000","last_fullsized":"2026-02-24T22:31:14.741475+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:29:17.090743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":101,"num_write":42,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.7","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.735768+0000","last_change":"2026-02-24T22:23:29.214459+0000","last_active":"2026-02-24T22:32:09.735768+0000","last_peered":"2026-02-24T22:32:09.735768+0000","last_clean":"2026-02-24T22:32:09.735768+0000","last_became_active":"2026-02-24T22:23:29.214105+0000","last_became_peered":"2026-02-24T22:23:29.214105+0000","last_unstale":"2026-02-24T22:32:09.735768+0000","last_undegraded":"2026-02-24T22:32:09.735768+0000","last_fullsized":"2026-02-24T22:32:09.735768+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:00:10.302637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.4","version":"307'2","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.747275+0000","last_change":"2026-02-24T22:15:25.827716+0000","last_active":"2026-02-24T22:32:09.747275+0000","last_peered":"2026-02-24T22:32:09.747275+0000","last_clean":"2026-02-24T22:32:09.747275+0000","last_became_active":"2026-02-24T22:15:25.826663+0000","last_became_peered":"2026-02-24T22:15:25.826663+0000","last_unstale":"2026-02-24T22:32:09.747275+0000","last_undegraded":"2026-02-24T22:32:09.747275+0000","last_fullsized":"2026-02-24T22:32:09.747275+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:08:34.544962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.5","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.730960+0000","last_change":"2026-02-24T22:15:22.816880+0000","last_active":"2026-02-24T22:32:09.730960+0000","last_peered":"2026-02-24T22:32:09.730960+0000","last_clean":"2026-02-24T22:32:09.730960+0000","last_became_active":"2026-02-24T22:15:22.816697+0000","last_became_peered":"2026-02-24T22:15:22.816697+0000","last_unstale":"2026-02-24T22:32:09.730960+0000","last_undegraded":"2026-02-24T22:32:09.730960+0000","last_fullsized":"2026-02-24T22:32:09.730960+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:56:01.670361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"79.1a","version":"312'16","reported_seq":102,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.737175+0000","last_change":"2026-02-24T22:13:43.757739+0000","last_active":"2026-02-24T22:31:14.737175+0000","last_peered":"2026-02-24T22:31:14.737175+0000","last_clean":"2026-02-24T22:31:14.737175+0000","last_became_active":"2026-02-24T22:13:43.757353+0000","last_became_peered":"2026-02-24T22:13:43.757353+0000","last_unstale":"2026-02-24T22:31:14.737175+0000","last_undegraded":"2026-02-24T22:31:14.737175+0000","last_fullsized":"2026-02-24T22:31:14.737175+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:47:30.725736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":65,"num_write":38,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.4","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742881+0000","last_change":"2026-02-24T22:23:29.215206+0000","last_active":"2026-02-24T22:31:49.742881+0000","last_peered":"2026-02-24T22:31:49.742881+0000","last_clean":"2026-02-24T22:31:49.742881+0000","last_became_active":"2026-02-24T22:23:29.215014+0000","last_became_peered":"2026-02-24T22:23:29.215014+0000","last_unstale":"2026-02-24T22:31:49.742881+0000","last_undegraded":"2026-02-24T22:31:49.742881+0000","last_fullsized":"2026-02-24T22:31:49.742881+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:43:42.586588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.7","version":"307'2","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.718321+0000","last_change":"2026-02-24T22:15:25.832214+0000","last_active":"2026-02-24T22:30:39.718321+0000","last_peered":"2026-02-24T22:30:39.718321+0000","last_clean":"2026-02-24T22:30:39.718321+0000","last_became_active":"2026-02-24T22:15:25.831806+0000","last_became_peered":"2026-02-24T22:15:25.831806+0000","last_unstale":"2026-02-24T22:30:39.718321+0000","last_undegraded":"2026-02-24T22:30:39.718321+0000","last_fullsized":"2026-02-24T22:30:39.718321+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:36:56.341773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"80.6","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:34.736157+0000","last_change":"2026-02-24T22:15:22.811674+0000","last_active":"2026-02-24T22:30:34.736157+0000","last_peered":"2026-02-24T22:30:34.736157+0000","last_clean":"2026-02-24T22:30:34.736157+0000","last_became_active":"2026-02-24T22:15:22.811413+0000","last_became_peered":"2026-02-24T22:15:22.811413+0000","last_unstale":"2026-02-24T22:30:34.736157+0000","last_undegraded":"2026-02-24T22:30:34.736157+0000","last_fullsized":"2026-02-24T22:30:34.736157+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:14:20.030861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"79.19","version":"312'30","reported_seq":140,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.738215+0000","last_change":"2026-02-24T22:13:43.756439+0000","last_active":"2026-02-24T22:30:44.738215+0000","last_peered":"2026-02-24T22:30:44.738215+0000","last_clean":"2026-02-24T22:30:44.738215+0000","last_became_active":"2026-02-24T22:13:43.756114+0000","last_became_peered":"2026-02-24T22:13:43.756114+0000","last_unstale":"2026-02-24T22:30:44.738215+0000","last_undegraded":"2026-02-24T22:30:44.738215+0000","last_fullsized":"2026-02-24T22:30:44.738215+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:45:46.139575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":216,"num_read_kb":143,"num_write":68,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.5","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742892+0000","last_change":"2026-02-24T22:23:29.213537+0000","last_active":"2026-02-24T22:31:49.742892+0000","last_peered":"2026-02-24T22:31:49.742892+0000","last_clean":"2026-02-24T22:31:49.742892+0000","last_became_active":"2026-02-24T22:23:29.213256+0000","last_became_peered":"2026-02-24T22:23:29.213256+0000","last_unstale":"2026-02-24T22:31:49.742892+0000","last_undegraded":"2026-02-24T22:31:49.742892+0000","last_fullsized":"2026-02-24T22:31:49.742892+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:48:26.724830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.6","version":"307'2","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.721669+0000","last_change":"2026-02-24T22:15:25.833922+0000","last_active":"2026-02-24T22:30:39.721669+0000","last_peered":"2026-02-24T22:30:39.721669+0000","last_clean":"2026-02-24T22:30:39.721669+0000","last_became_active":"2026-02-24T22:15:25.833680+0000","last_became_peered":"2026-02-24T22:15:25.833680+0000","last_unstale":"2026-02-24T22:30:39.721669+0000","last_undegraded":"2026-02-24T22:30:39.721669+0000","last_fullsized":"2026-02-24T22:30:39.721669+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:26:51.677018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.7","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:34.736181+0000","last_change":"2026-02-24T22:15:22.812054+0000","last_active":"2026-02-24T22:30:34.736181+0000","last_peered":"2026-02-24T22:30:34.736181+0000","last_clean":"2026-02-24T22:30:34.736181+0000","last_became_active":"2026-02-24T22:15:22.811618+0000","last_became_peered":"2026-02-24T22:15:22.811618+0000","last_unstale":"2026-02-24T22:30:34.736181+0000","last_undegraded":"2026-02-24T22:30:34.736181+0000","last_fullsized":"2026-02-24T22:30:34.736181+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:37:58.903441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"79.18","version":"312'12","reported_seq":95,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.737670+0000","last_change":"2026-02-24T22:13:43.755501+0000","last_active":"2026-02-24T22:30:39.737670+0000","last_peered":"2026-02-24T22:30:39.737670+0000","last_clean":"2026-02-24T22:30:39.737670+0000","last_became_active":"2026-02-24T22:13:43.754981+0000","last_became_peered":"2026-02-24T22:13:43.754981+0000","last_unstale":"2026-02-24T22:30:39.737670+0000","last_undegraded":"2026-02-24T22:30:39.737670+0000","last_fullsized":"2026-02-24T22:30:39.737670+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:41:38.765379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":86,"num_read_kb":57,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.a","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.745052+0000","last_change":"2026-02-24T22:23:29.212877+0000","last_active":"2026-02-24T22:32:09.745052+0000","last_peered":"2026-02-24T22:32:09.745052+0000","last_clean":"2026-02-24T22:32:09.745052+0000","last_became_active":"2026-02-24T22:23:29.212213+0000","last_became_peered":"2026-02-24T22:23:29.212213+0000","last_unstale":"2026-02-24T22:32:09.745052+0000","last_undegraded":"2026-02-24T22:32:09.745052+0000","last_fullsized":"2026-02-24T22:32:09.745052+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:29:28.605583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.9","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.736816+0000","last_change":"2026-02-24T22:15:25.826412+0000","last_active":"2026-02-24T22:30:39.736816+0000","last_peered":"2026-02-24T22:30:39.736816+0000","last_clean":"2026-02-24T22:30:39.736816+0000","last_became_active":"2026-02-24T22:15:25.826076+0000","last_became_peered":"2026-02-24T22:15:25.826076+0000","last_unstale":"2026-02-24T22:30:39.736816+0000","last_undegraded":"2026-02-24T22:30:39.736816+0000","last_fullsized":"2026-02-24T22:30:39.736816+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:00:25.763716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.8","version":"307'4","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:34.717818+0000","last_change":"2026-02-24T22:15:22.816346+0000","last_active":"2026-02-24T22:30:34.717818+0000","last_peered":"2026-02-24T22:30:34.717818+0000","last_clean":"2026-02-24T22:30:34.717818+0000","last_became_active":"2026-02-24T22:15:22.815960+0000","last_became_peered":"2026-02-24T22:15:22.815960+0000","last_unstale":"2026-02-24T22:30:34.717818+0000","last_undegraded":"2026-02-24T22:30:34.717818+0000","last_fullsized":"2026-02-24T22:30:34.717818+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:55:34.277066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":6144,"num_write":9,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.17","version":"312'34","reported_seq":169,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.721687+0000","last_change":"2026-02-24T22:14:00.749701+0000","last_active":"2026-02-24T22:30:39.721687+0000","last_peered":"2026-02-24T22:30:39.721687+0000","last_clean":"2026-02-24T22:30:39.721687+0000","last_became_active":"2026-02-24T22:14:00.748438+0000","last_became_peered":"2026-02-24T22:14:00.748438+0000","last_unstale":"2026-02-24T22:30:39.721687+0000","last_undegraded":"2026-02-24T22:30:39.721687+0000","last_fullsized":"2026-02-24T22:30:39.721687+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":293,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:21:49.815767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":298,"num_read_kb":211,"num_write":78,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.b","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.730451+0000","last_change":"2026-02-24T22:23:29.197335+0000","last_active":"2026-02-24T22:30:14.730451+0000","last_peered":"2026-02-24T22:30:14.730451+0000","last_clean":"2026-02-24T22:30:14.730451+0000","last_became_active":"2026-02-24T22:23:29.196790+0000","last_became_peered":"2026-02-24T22:23:29.196790+0000","last_unstale":"2026-02-24T22:30:14.730451+0000","last_undegraded":"2026-02-24T22:30:14.730451+0000","last_fullsized":"2026-02-24T22:30:14.730451+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:55:13.008271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.8","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.734884+0000","last_change":"2026-02-24T22:15:25.825567+0000","last_active":"2026-02-24T22:30:39.734884+0000","last_peered":"2026-02-24T22:30:39.734884+0000","last_clean":"2026-02-24T22:30:39.734884+0000","last_became_active":"2026-02-24T22:15:25.825315+0000","last_became_peered":"2026-02-24T22:15:25.825315+0000","last_unstale":"2026-02-24T22:30:39.734884+0000","last_undegraded":"2026-02-24T22:30:39.734884+0000","last_fullsized":"2026-02-24T22:30:39.734884+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:24:39.313916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.9","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:34.732583+0000","last_change":"2026-02-24T22:15:22.811365+0000","last_active":"2026-02-24T22:30:34.732583+0000","last_peered":"2026-02-24T22:30:34.732583+0000","last_clean":"2026-02-24T22:30:34.732583+0000","last_became_active":"2026-02-24T22:15:22.811141+0000","last_became_peered":"2026-02-24T22:15:22.811141+0000","last_unstale":"2026-02-24T22:30:34.732583+0000","last_undegraded":"2026-02-24T22:30:34.732583+0000","last_fullsized":"2026-02-24T22:30:34.732583+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:45:06.384015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"79.16","version":"0'0","reported_seq":63,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.733272+0000","last_change":"2026-02-24T22:13:43.756602+0000","last_active":"2026-02-24T22:30:39.733272+0000","last_peered":"2026-02-24T22:30:39.733272+0000","last_clean":"2026-02-24T22:30:39.733272+0000","last_became_active":"2026-02-24T22:13:43.755885+0000","last_became_peered":"2026-02-24T22:13:43.755885+0000","last_unstale":"2026-02-24T22:30:39.733272+0000","last_undegraded":"2026-02-24T22:30:39.733272+0000","last_fullsized":"2026-02-24T22:30:39.733272+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:40:07.171226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.8","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.744882+0000","last_change":"2026-02-24T22:23:29.199299+0000","last_active":"2026-02-24T22:32:09.744882+0000","last_peered":"2026-02-24T22:32:09.744882+0000","last_clean":"2026-02-24T22:32:09.744882+0000","last_became_active":"2026-02-24T22:23:29.198882+0000","last_became_peered":"2026-02-24T22:23:29.198882+0000","last_unstale":"2026-02-24T22:32:09.744882+0000","last_undegraded":"2026-02-24T22:32:09.744882+0000","last_fullsized":"2026-02-24T22:32:09.744882+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:59:03.245529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"81.b","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.741464+0000","last_change":"2026-02-24T22:15:25.827664+0000","last_active":"2026-02-24T22:31:14.741464+0000","last_peered":"2026-02-24T22:31:14.741464+0000","last_clean":"2026-02-24T22:31:14.741464+0000","last_became_active":"2026-02-24T22:15:25.827265+0000","last_became_peered":"2026-02-24T22:15:25.827265+0000","last_unstale":"2026-02-24T22:31:14.741464+0000","last_undegraded":"2026-02-24T22:31:14.741464+0000","last_fullsized":"2026-02-24T22:31:14.741464+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:17:29.198425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.a","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.735221+0000","last_change":"2026-02-24T22:15:22.817325+0000","last_active":"2026-02-24T22:30:39.735221+0000","last_peered":"2026-02-24T22:30:39.735221+0000","last_clean":"2026-02-24T22:30:39.735221+0000","last_became_active":"2026-02-24T22:15:22.816702+0000","last_became_peered":"2026-02-24T22:15:22.816702+0000","last_unstale":"2026-02-24T22:30:39.735221+0000","last_undegraded":"2026-02-24T22:30:39.735221+0000","last_fullsized":"2026-02-24T22:30:39.735221+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:48:16.071491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.15","version":"307'6","reported_seq":82,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.735250+0000","last_change":"2026-02-24T22:13:43.761758+0000","last_active":"2026-02-24T22:30:39.735250+0000","last_peered":"2026-02-24T22:30:39.735250+0000","last_clean":"2026-02-24T22:30:39.735250+0000","last_became_active":"2026-02-24T22:13:43.760892+0000","last_became_peered":"2026-02-24T22:13:43.760892+0000","last_unstale":"2026-02-24T22:30:39.735250+0000","last_undegraded":"2026-02-24T22:30:39.735250+0000","last_fullsized":"2026-02-24T22:30:39.735250+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:07:44.028035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":37,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.9","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.743246+0000","last_change":"2026-02-24T22:23:29.198300+0000","last_active":"2026-02-24T22:32:09.743246+0000","last_peered":"2026-02-24T22:32:09.743246+0000","last_clean":"2026-02-24T22:32:09.743246+0000","last_became_active":"2026-02-24T22:23:29.197357+0000","last_became_peered":"2026-02-24T22:23:29.197357+0000","last_unstale":"2026-02-24T22:32:09.743246+0000","last_undegraded":"2026-02-24T22:32:09.743246+0000","last_fullsized":"2026-02-24T22:32:09.743246+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:58:07.606981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.a","version":"307'2","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.738233+0000","last_change":"2026-02-24T22:15:25.827469+0000","last_active":"2026-02-24T22:30:44.738233+0000","last_peered":"2026-02-24T22:30:44.738233+0000","last_clean":"2026-02-24T22:30:44.738233+0000","last_became_active":"2026-02-24T22:15:25.827055+0000","last_became_peered":"2026-02-24T22:15:25.827055+0000","last_unstale":"2026-02-24T22:30:44.738233+0000","last_undegraded":"2026-02-24T22:30:44.738233+0000","last_fullsized":"2026-02-24T22:30:44.738233+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:35:15.420239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.b","version":"307'2","reported_seq":42,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.738494+0000","last_change":"2026-02-24T22:15:22.817894+0000","last_active":"2026-02-24T22:31:09.738494+0000","last_peered":"2026-02-24T22:31:09.738494+0000","last_clean":"2026-02-24T22:31:09.738494+0000","last_became_active":"2026-02-24T22:15:22.817498+0000","last_became_peered":"2026-02-24T22:15:22.817498+0000","last_unstale":"2026-02-24T22:31:09.738494+0000","last_undegraded":"2026-02-24T22:31:09.738494+0000","last_fullsized":"2026-02-24T22:31:09.738494+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:08:45.124818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.14","version":"312'10","reported_seq":89,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:09.718735+0000","last_change":"2026-02-24T22:13:43.762482+0000","last_active":"2026-02-24T22:30:09.718735+0000","last_peered":"2026-02-24T22:30:09.718735+0000","last_clean":"2026-02-24T22:30:09.718735+0000","last_became_active":"2026-02-24T22:13:43.762341+0000","last_became_peered":"2026-02-24T22:13:43.762341+0000","last_unstale":"2026-02-24T22:30:09.718735+0000","last_undegraded":"2026-02-24T22:30:09.718735+0000","last_fullsized":"2026-02-24T22:30:09.718735+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:44:09.549825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":46,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.e","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.724040+0000","last_change":"2026-02-24T22:23:29.215664+0000","last_active":"2026-02-24T22:30:14.724040+0000","last_peered":"2026-02-24T22:30:14.724040+0000","last_clean":"2026-02-24T22:30:14.724040+0000","last_became_active":"2026-02-24T22:23:29.215425+0000","last_became_peered":"2026-02-24T22:23:29.215425+0000","last_unstale":"2026-02-24T22:30:14.724040+0000","last_undegraded":"2026-02-24T22:30:14.724040+0000","last_fullsized":"2026-02-24T22:30:14.724040+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:44:31.538355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.d","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.741498+0000","last_change":"2026-02-24T22:15:25.829241+0000","last_active":"2026-02-24T22:31:14.741498+0000","last_peered":"2026-02-24T22:31:14.741498+0000","last_clean":"2026-02-24T22:31:14.741498+0000","last_became_active":"2026-02-24T22:15:25.829109+0000","last_became_peered":"2026-02-24T22:15:25.829109+0000","last_unstale":"2026-02-24T22:31:14.741498+0000","last_undegraded":"2026-02-24T22:31:14.741498+0000","last_fullsized":"2026-02-24T22:31:14.741498+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:38:08.330741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.c","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.721310+0000","last_change":"2026-02-24T22:15:22.815624+0000","last_active":"2026-02-24T22:31:09.721310+0000","last_peered":"2026-02-24T22:31:09.721310+0000","last_clean":"2026-02-24T22:31:09.721310+0000","last_became_active":"2026-02-24T22:15:22.815364+0000","last_became_peered":"2026-02-24T22:15:22.815364+0000","last_unstale":"2026-02-24T22:31:09.721310+0000","last_undegraded":"2026-02-24T22:31:09.721310+0000","last_fullsized":"2026-02-24T22:31:09.721310+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:21:36.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.13","version":"312'11","reported_seq":95,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.714998+0000","last_change":"2026-02-24T22:13:43.760988+0000","last_active":"2026-02-24T22:30:04.714998+0000","last_peered":"2026-02-24T22:30:04.714998+0000","last_clean":"2026-02-24T22:30:04.714998+0000","last_became_active":"2026-02-24T22:13:43.760714+0000","last_became_peered":"2026-02-24T22:13:43.760714+0000","last_unstale":"2026-02-24T22:30:04.714998+0000","last_undegraded":"2026-02-24T22:30:04.714998+0000","last_fullsized":"2026-02-24T22:30:04.714998+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:55:23.837996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":61,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"82.f","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.730477+0000","last_change":"2026-02-24T22:23:29.199754+0000","last_active":"2026-02-24T22:30:14.730477+0000","last_peered":"2026-02-24T22:30:14.730477+0000","last_clean":"2026-02-24T22:30:14.730477+0000","last_became_active":"2026-02-24T22:23:29.199538+0000","last_became_peered":"2026-02-24T22:23:29.199538+0000","last_unstale":"2026-02-24T22:30:14.730477+0000","last_undegraded":"2026-02-24T22:30:14.730477+0000","last_fullsized":"2026-02-24T22:30:14.730477+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:15:25.304156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.c","version":"307'7","reported_seq":54,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.740681+0000","last_change":"2026-02-24T22:15:25.827267+0000","last_active":"2026-02-24T22:31:14.740681+0000","last_peered":"2026-02-24T22:31:14.740681+0000","last_clean":"2026-02-24T22:31:14.740681+0000","last_became_active":"2026-02-24T22:15:25.826430+0000","last_became_peered":"2026-02-24T22:15:25.826430+0000","last_unstale":"2026-02-24T22:31:14.740681+0000","last_undegraded":"2026-02-24T22:31:14.740681+0000","last_fullsized":"2026-02-24T22:31:14.740681+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:49:19.806352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":31744,"num_write":12,"num_write_kb":16384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.d","version":"307'3","reported_seq":50,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.740980+0000","last_change":"2026-02-24T22:15:22.813197+0000","last_active":"2026-02-24T22:31:09.740980+0000","last_peered":"2026-02-24T22:31:09.740980+0000","last_clean":"2026-02-24T22:31:09.740980+0000","last_became_active":"2026-02-24T22:15:22.813062+0000","last_became_peered":"2026-02-24T22:15:22.813062+0000","last_unstale":"2026-02-24T22:31:09.740980+0000","last_undegraded":"2026-02-24T22:31:09.740980+0000","last_fullsized":"2026-02-24T22:31:09.740980+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:08:36.468537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":20480,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.12","version":"312'20","reported_seq":117,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:48.279108+0000","last_change":"2026-02-24T22:13:43.760420+0000","last_active":"2026-02-24T22:30:48.279108+0000","last_peered":"2026-02-24T22:30:48.279108+0000","last_clean":"2026-02-24T22:30:48.279108+0000","last_became_active":"2026-02-24T22:13:43.760196+0000","last_became_peered":"2026-02-24T22:13:43.760196+0000","last_unstale":"2026-02-24T22:30:48.279108+0000","last_undegraded":"2026-02-24T22:30:48.279108+0000","last_fullsized":"2026-02-24T22:30:48.279108+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:14:15.856109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":106,"num_write":52,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"82.c","version":"310'1","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.734866+0000","last_change":"2026-02-24T22:23:29.201743+0000","last_active":"2026-02-24T22:30:14.734866+0000","last_peered":"2026-02-24T22:30:14.734866+0000","last_clean":"2026-02-24T22:30:14.734866+0000","last_became_active":"2026-02-24T22:23:29.201545+0000","last_became_peered":"2026-02-24T22:23:29.201545+0000","last_unstale":"2026-02-24T22:30:14.734866+0000","last_undegraded":"2026-02-24T22:30:14.734866+0000","last_fullsized":"2026-02-24T22:30:14.734866+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:07:57.527033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"81.f","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.741445+0000","last_change":"2026-02-24T22:15:25.825325+0000","last_active":"2026-02-24T22:31:19.741445+0000","last_peered":"2026-02-24T22:31:19.741445+0000","last_clean":"2026-02-24T22:31:19.741445+0000","last_became_active":"2026-02-24T22:15:25.824862+0000","last_became_peered":"2026-02-24T22:15:25.824862+0000","last_unstale":"2026-02-24T22:31:19.741445+0000","last_undegraded":"2026-02-24T22:31:19.741445+0000","last_fullsized":"2026-02-24T22:31:19.741445+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:16:51.320757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.e","version":"307'3","reported_seq":52,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.721326+0000","last_change":"2026-02-24T22:15:22.817941+0000","last_active":"2026-02-24T22:31:09.721326+0000","last_peered":"2026-02-24T22:31:09.721326+0000","last_clean":"2026-02-24T22:31:09.721326+0000","last_became_active":"2026-02-24T22:15:22.817295+0000","last_became_peered":"2026-02-24T22:15:22.817295+0000","last_unstale":"2026-02-24T22:31:09.721326+0000","last_undegraded":"2026-02-24T22:31:09.721326+0000","last_fullsized":"2026-02-24T22:31:09.721326+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:37:42.904886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":25600,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.11","version":"312'11","reported_seq":92,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.731364+0000","last_change":"2026-02-24T22:13:43.760497+0000","last_active":"2026-02-24T22:30:04.731364+0000","last_peered":"2026-02-24T22:30:04.731364+0000","last_clean":"2026-02-24T22:30:04.731364+0000","last_became_active":"2026-02-24T22:13:43.759864+0000","last_became_peered":"2026-02-24T22:13:43.759864+0000","last_unstale":"2026-02-24T22:30:04.731364+0000","last_undegraded":"2026-02-24T22:30:04.731364+0000","last_fullsized":"2026-02-24T22:30:04.731364+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:50:51.585161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":53,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.d","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.724028+0000","last_change":"2026-02-24T22:23:29.214699+0000","last_active":"2026-02-24T22:30:14.724028+0000","last_peered":"2026-02-24T22:30:14.724028+0000","last_clean":"2026-02-24T22:30:14.724028+0000","last_became_active":"2026-02-24T22:23:29.214432+0000","last_became_peered":"2026-02-24T22:23:29.214432+0000","last_unstale":"2026-02-24T22:30:14.724028+0000","last_undegraded":"2026-02-24T22:30:14.724028+0000","last_fullsized":"2026-02-24T22:30:14.724028+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:09:09.120137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.e","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.738719+0000","last_change":"2026-02-24T22:15:25.827425+0000","last_active":"2026-02-24T22:31:14.738719+0000","last_peered":"2026-02-24T22:31:14.738719+0000","last_clean":"2026-02-24T22:31:14.738719+0000","last_became_active":"2026-02-24T22:15:25.827110+0000","last_became_peered":"2026-02-24T22:15:25.827110+0000","last_unstale":"2026-02-24T22:31:14.738719+0000","last_undegraded":"2026-02-24T22:31:14.738719+0000","last_fullsized":"2026-02-24T22:31:14.738719+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:25:06.342633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.f","version":"307'5","reported_seq":55,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.739061+0000","last_change":"2026-02-24T22:15:22.816168+0000","last_active":"2026-02-24T22:31:14.739061+0000","last_peered":"2026-02-24T22:31:14.739061+0000","last_clean":"2026-02-24T22:31:14.739061+0000","last_became_active":"2026-02-24T22:15:22.815819+0000","last_became_peered":"2026-02-24T22:15:22.815819+0000","last_unstale":"2026-02-24T22:31:14.739061+0000","last_undegraded":"2026-02-24T22:31:14.739061+0000","last_fullsized":"2026-02-24T22:31:14.739061+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:39:39.503227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11534336,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":26624,"num_write":10,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.10","version":"312'31","reported_seq":152,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.731343+0000","last_change":"2026-02-24T22:13:43.760545+0000","last_active":"2026-02-24T22:30:04.731343+0000","last_peered":"2026-02-24T22:30:04.731343+0000","last_clean":"2026-02-24T22:30:04.731343+0000","last_became_active":"2026-02-24T22:13:43.759855+0000","last_became_peered":"2026-02-24T22:13:43.759855+0000","last_unstale":"2026-02-24T22:30:04.731343+0000","last_undegraded":"2026-02-24T22:30:04.731343+0000","last_fullsized":"2026-02-24T22:30:04.731343+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:57:17.855796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":272,"num_read_kb":204,"num_write":72,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.12","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.725516+0000","last_change":"2026-02-24T22:23:29.212745+0000","last_active":"2026-02-24T22:30:29.725516+0000","last_peered":"2026-02-24T22:30:29.725516+0000","last_clean":"2026-02-24T22:30:29.725516+0000","last_became_active":"2026-02-24T22:23:29.212534+0000","last_became_peered":"2026-02-24T22:23:29.212534+0000","last_unstale":"2026-02-24T22:30:29.725516+0000","last_undegraded":"2026-02-24T22:30:29.725516+0000","last_fullsized":"2026-02-24T22:30:29.725516+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:50:08.072181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.11","version":"307'3","reported_seq":44,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744735+0000","last_change":"2026-02-24T22:15:25.827242+0000","last_active":"2026-02-24T22:31:49.744735+0000","last_peered":"2026-02-24T22:31:49.744735+0000","last_clean":"2026-02-24T22:31:49.744735+0000","last_became_active":"2026-02-24T22:15:25.826517+0000","last_became_peered":"2026-02-24T22:15:25.826517+0000","last_unstale":"2026-02-24T22:31:49.744735+0000","last_undegraded":"2026-02-24T22:31:49.744735+0000","last_fullsized":"2026-02-24T22:31:49.744735+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:24:40.732721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":20480,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.10","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.741867+0000","last_change":"2026-02-24T22:15:22.810414+0000","last_active":"2026-02-24T22:31:44.741867+0000","last_peered":"2026-02-24T22:31:44.741867+0000","last_clean":"2026-02-24T22:31:44.741867+0000","last_became_active":"2026-02-24T22:15:22.810000+0000","last_became_peered":"2026-02-24T22:15:22.810000+0000","last_unstale":"2026-02-24T22:31:44.741867+0000","last_undegraded":"2026-02-24T22:31:44.741867+0000","last_fullsized":"2026-02-24T22:31:44.741867+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:21:42.629513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"79.f","version":"312'11","reported_seq":95,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742413+0000","last_change":"2026-02-24T22:13:43.755156+0000","last_active":"2026-02-24T22:31:49.742413+0000","last_peered":"2026-02-24T22:31:49.742413+0000","last_clean":"2026-02-24T22:31:49.742413+0000","last_became_active":"2026-02-24T22:13:43.754602+0000","last_became_peered":"2026-02-24T22:13:43.754602+0000","last_unstale":"2026-02-24T22:31:49.742413+0000","last_undegraded":"2026-02-24T22:31:49.742413+0000","last_fullsized":"2026-02-24T22:31:49.742413+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:13:53.006921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":67,"num_write":27,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.13","version":"310'1","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.735479+0000","last_change":"2026-02-24T22:23:29.198937+0000","last_active":"2026-02-24T22:30:29.735479+0000","last_peered":"2026-02-24T22:30:29.735479+0000","last_clean":"2026-02-24T22:30:29.735479+0000","last_became_active":"2026-02-24T22:23:29.198627+0000","last_became_peered":"2026-02-24T22:23:29.198627+0000","last_unstale":"2026-02-24T22:30:29.735479+0000","last_undegraded":"2026-02-24T22:30:29.735479+0000","last_fullsized":"2026-02-24T22:30:29.735479+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:28:29.061001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.10","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744722+0000","last_change":"2026-02-24T22:15:25.827252+0000","last_active":"2026-02-24T22:31:49.744722+0000","last_peered":"2026-02-24T22:31:49.744722+0000","last_clean":"2026-02-24T22:31:49.744722+0000","last_became_active":"2026-02-24T22:15:25.826606+0000","last_became_peered":"2026-02-24T22:15:25.826606+0000","last_unstale":"2026-02-24T22:31:49.744722+0000","last_undegraded":"2026-02-24T22:31:49.744722+0000","last_fullsized":"2026-02-24T22:31:49.744722+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:30:33.376941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.11","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.741885+0000","last_change":"2026-02-24T22:15:22.810459+0000","last_active":"2026-02-24T22:31:44.741885+0000","last_peered":"2026-02-24T22:31:44.741885+0000","last_clean":"2026-02-24T22:31:44.741885+0000","last_became_active":"2026-02-24T22:15:22.809791+0000","last_became_peered":"2026-02-24T22:15:22.809791+0000","last_unstale":"2026-02-24T22:31:44.741885+0000","last_undegraded":"2026-02-24T22:31:44.741885+0000","last_fullsized":"2026-02-24T22:31:44.741885+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:37:39.655379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"79.e","version":"312'23","reported_seq":126,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.741917+0000","last_change":"2026-02-24T22:13:43.756185+0000","last_active":"2026-02-24T22:31:44.741917+0000","last_peered":"2026-02-24T22:31:44.741917+0000","last_clean":"2026-02-24T22:31:44.741917+0000","last_became_active":"2026-02-24T22:13:43.755868+0000","last_became_peered":"2026-02-24T22:13:43.755868+0000","last_unstale":"2026-02-24T22:31:44.741917+0000","last_undegraded":"2026-02-24T22:31:44.741917+0000","last_fullsized":"2026-02-24T22:31:44.741917+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:56:04.041994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":169,"num_read_kb":112,"num_write":56,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.10","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.734148+0000","last_change":"2026-02-24T22:23:29.212895+0000","last_active":"2026-02-24T22:30:29.734148+0000","last_peered":"2026-02-24T22:30:29.734148+0000","last_clean":"2026-02-24T22:30:29.734148+0000","last_became_active":"2026-02-24T22:23:29.212665+0000","last_became_peered":"2026-02-24T22:23:29.212665+0000","last_unstale":"2026-02-24T22:30:29.734148+0000","last_undegraded":"2026-02-24T22:30:29.734148+0000","last_fullsized":"2026-02-24T22:30:29.734148+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:30:27.871463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.13","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.740862+0000","last_change":"2026-02-24T22:15:25.826171+0000","last_active":"2026-02-24T22:31:49.740862+0000","last_peered":"2026-02-24T22:31:49.740862+0000","last_clean":"2026-02-24T22:31:49.740862+0000","last_became_active":"2026-02-24T22:15:25.825862+0000","last_became_peered":"2026-02-24T22:15:25.825862+0000","last_unstale":"2026-02-24T22:31:49.740862+0000","last_undegraded":"2026-02-24T22:31:49.740862+0000","last_fullsized":"2026-02-24T22:31:49.740862+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:02:37.431216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"80.12","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.744628+0000","last_change":"2026-02-24T22:15:22.812712+0000","last_active":"2026-02-24T22:31:44.744628+0000","last_peered":"2026-02-24T22:31:44.744628+0000","last_clean":"2026-02-24T22:31:44.744628+0000","last_became_active":"2026-02-24T22:15:22.812405+0000","last_became_peered":"2026-02-24T22:15:22.812405+0000","last_unstale":"2026-02-24T22:31:44.744628+0000","last_undegraded":"2026-02-24T22:31:44.744628+0000","last_fullsized":"2026-02-24T22:31:44.744628+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:36:00.971366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.d","version":"312'20","reported_seq":113,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.733217+0000","last_change":"2026-02-24T22:13:43.761593+0000","last_active":"2026-02-24T22:31:44.733217+0000","last_peered":"2026-02-24T22:31:44.733217+0000","last_clean":"2026-02-24T22:31:44.733217+0000","last_became_active":"2026-02-24T22:13:43.761339+0000","last_became_peered":"2026-02-24T22:13:43.761339+0000","last_unstale":"2026-02-24T22:31:44.733217+0000","last_undegraded":"2026-02-24T22:31:44.733217+0000","last_fullsized":"2026-02-24T22:31:44.733217+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:02:40.622911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":126,"num_read_kb":76,"num_write":49,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"82.11","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.720712+0000","last_change":"2026-02-24T22:23:29.214793+0000","last_active":"2026-02-24T22:30:29.720712+0000","last_peered":"2026-02-24T22:30:29.720712+0000","last_clean":"2026-02-24T22:30:29.720712+0000","last_became_active":"2026-02-24T22:23:29.214167+0000","last_became_peered":"2026-02-24T22:23:29.214167+0000","last_unstale":"2026-02-24T22:30:29.720712+0000","last_undegraded":"2026-02-24T22:30:29.720712+0000","last_fullsized":"2026-02-24T22:30:29.720712+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:21:51.443877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.12","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744756+0000","last_change":"2026-02-24T22:15:25.826465+0000","last_active":"2026-02-24T22:31:49.744756+0000","last_peered":"2026-02-24T22:31:49.744756+0000","last_clean":"2026-02-24T22:31:49.744756+0000","last_became_active":"2026-02-24T22:15:25.826102+0000","last_became_peered":"2026-02-24T22:15:25.826102+0000","last_unstale":"2026-02-24T22:31:49.744756+0000","last_undegraded":"2026-02-24T22:31:49.744756+0000","last_fullsized":"2026-02-24T22:31:49.744756+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:22:01.000780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.13","version":"307'2","reported_seq":42,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.744644+0000","last_change":"2026-02-24T22:15:22.812393+0000","last_active":"2026-02-24T22:31:44.744644+0000","last_peered":"2026-02-24T22:31:44.744644+0000","last_clean":"2026-02-24T22:31:44.744644+0000","last_became_active":"2026-02-24T22:15:22.811927+0000","last_became_peered":"2026-02-24T22:15:22.811927+0000","last_unstale":"2026-02-24T22:31:44.744644+0000","last_undegraded":"2026-02-24T22:31:44.744644+0000","last_fullsized":"2026-02-24T22:31:44.744644+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:54:16.404989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.c","version":"312'4","reported_seq":74,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.724938+0000","last_change":"2026-02-24T22:13:43.762114+0000","last_active":"2026-02-24T22:31:44.724938+0000","last_peered":"2026-02-24T22:31:44.724938+0000","last_clean":"2026-02-24T22:31:44.724938+0000","last_became_active":"2026-02-24T22:13:43.761910+0000","last_became_peered":"2026-02-24T22:13:43.761910+0000","last_unstale":"2026-02-24T22:31:44.724938+0000","last_undegraded":"2026-02-24T22:31:44.724938+0000","last_fullsized":"2026-02-24T22:31:44.724938+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:51:11.166418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":15,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"82.16","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.718767+0000","last_change":"2026-02-24T22:23:29.213342+0000","last_active":"2026-02-24T22:30:44.718767+0000","last_peered":"2026-02-24T22:30:44.718767+0000","last_clean":"2026-02-24T22:30:44.718767+0000","last_became_active":"2026-02-24T22:23:29.213080+0000","last_became_peered":"2026-02-24T22:23:29.213080+0000","last_unstale":"2026-02-24T22:30:44.718767+0000","last_undegraded":"2026-02-24T22:30:44.718767+0000","last_fullsized":"2026-02-24T22:30:44.718767+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:33:29.031496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.15","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:59.714511+0000","last_change":"2026-02-24T22:15:25.831361+0000","last_active":"2026-02-24T22:29:59.714511+0000","last_peered":"2026-02-24T22:29:59.714511+0000","last_clean":"2026-02-24T22:29:59.714511+0000","last_became_active":"2026-02-24T22:15:25.830922+0000","last_became_peered":"2026-02-24T22:15:25.830922+0000","last_unstale":"2026-02-24T22:29:59.714511+0000","last_undegraded":"2026-02-24T22:29:59.714511+0000","last_fullsized":"2026-02-24T22:29:59.714511+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:08:07.538696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"80.14","version":"307'2","reported_seq":42,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.745149+0000","last_change":"2026-02-24T22:15:22.812446+0000","last_active":"2026-02-24T22:31:49.745149+0000","last_peered":"2026-02-24T22:31:49.745149+0000","last_clean":"2026-02-24T22:31:49.745149+0000","last_became_active":"2026-02-24T22:15:22.811924+0000","last_became_peered":"2026-02-24T22:15:22.811924+0000","last_unstale":"2026-02-24T22:31:49.745149+0000","last_undegraded":"2026-02-24T22:31:49.745149+0000","last_fullsized":"2026-02-24T22:31:49.745149+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:18:30.397097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.b","version":"312'13","reported_seq":98,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.740368+0000","last_change":"2026-02-24T22:13:43.757072+0000","last_active":"2026-02-24T22:31:44.740368+0000","last_peered":"2026-02-24T22:31:44.740368+0000","last_clean":"2026-02-24T22:31:44.740368+0000","last_became_active":"2026-02-24T22:13:43.756596+0000","last_became_peered":"2026-02-24T22:13:43.756596+0000","last_unstale":"2026-02-24T22:31:44.740368+0000","last_undegraded":"2026-02-24T22:31:44.740368+0000","last_fullsized":"2026-02-24T22:31:44.740368+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:30:51.803180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":56,"num_write":37,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.17","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.718784+0000","last_change":"2026-02-24T22:23:29.214456+0000","last_active":"2026-02-24T22:30:44.718784+0000","last_peered":"2026-02-24T22:30:44.718784+0000","last_clean":"2026-02-24T22:30:44.718784+0000","last_became_active":"2026-02-24T22:23:29.214217+0000","last_became_peered":"2026-02-24T22:23:29.214217+0000","last_unstale":"2026-02-24T22:30:44.718784+0000","last_undegraded":"2026-02-24T22:30:44.718784+0000","last_fullsized":"2026-02-24T22:30:44.718784+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:48:46.666871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.14","version":"307'3","reported_seq":40,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.745241+0000","last_change":"2026-02-24T22:15:25.827827+0000","last_active":"2026-02-24T22:31:54.745241+0000","last_peered":"2026-02-24T22:31:54.745241+0000","last_clean":"2026-02-24T22:31:54.745241+0000","last_became_active":"2026-02-24T22:15:25.827557+0000","last_became_peered":"2026-02-24T22:15:25.827557+0000","last_unstale":"2026-02-24T22:31:54.745241+0000","last_undegraded":"2026-02-24T22:31:54.745241+0000","last_fullsized":"2026-02-24T22:31:54.745241+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:13:31.380536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":4096,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.15","version":"307'3","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:54.732905+0000","last_change":"2026-02-24T22:15:22.812814+0000","last_active":"2026-02-24T22:29:54.732905+0000","last_peered":"2026-02-24T22:29:54.732905+0000","last_clean":"2026-02-24T22:29:54.732905+0000","last_became_active":"2026-02-24T22:15:22.812450+0000","last_became_peered":"2026-02-24T22:15:22.812450+0000","last_unstale":"2026-02-24T22:29:54.732905+0000","last_undegraded":"2026-02-24T22:29:54.732905+0000","last_fullsized":"2026-02-24T22:29:54.732905+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:22:29.140595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":4096,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.a","version":"312'10","reported_seq":90,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.724683+0000","last_change":"2026-02-24T22:13:43.762418+0000","last_active":"2026-02-24T22:31:09.724683+0000","last_peered":"2026-02-24T22:31:09.724683+0000","last_clean":"2026-02-24T22:31:09.724683+0000","last_became_active":"2026-02-24T22:13:43.762226+0000","last_became_peered":"2026-02-24T22:13:43.762226+0000","last_unstale":"2026-02-24T22:31:09.724683+0000","last_undegraded":"2026-02-24T22:31:09.724683+0000","last_fullsized":"2026-02-24T22:31:09.724683+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:41:14.108185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":42,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.14","version":"309'1","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.735499+0000","last_change":"2026-02-24T22:24:01.299030+0000","last_active":"2026-02-24T22:30:29.735499+0000","last_peered":"2026-02-24T22:30:29.735499+0000","last_clean":"2026-02-24T22:30:29.735499+0000","last_became_active":"2026-02-24T22:24:01.297193+0000","last_became_peered":"2026-02-24T22:24:01.297193+0000","last_unstale":"2026-02-24T22:30:29.735499+0000","last_undegraded":"2026-02-24T22:30:29.735499+0000","last_fullsized":"2026-02-24T22:30:29.735499+0000","mapping_epoch":311,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":312,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:15:53.769966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.17","version":"307'2","reported_seq":34,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:59.730863+0000","last_change":"2026-02-24T22:15:25.833669+0000","last_active":"2026-02-24T22:29:59.730863+0000","last_peered":"2026-02-24T22:29:59.730863+0000","last_clean":"2026-02-24T22:29:59.730863+0000","last_became_active":"2026-02-24T22:15:25.833430+0000","last_became_peered":"2026-02-24T22:15:25.833430+0000","last_unstale":"2026-02-24T22:29:59.730863+0000","last_undegraded":"2026-02-24T22:29:59.730863+0000","last_fullsized":"2026-02-24T22:29:59.730863+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:41:23.281878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1024,"num_write":7,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"80.16","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:54.714013+0000","last_change":"2026-02-24T22:15:22.817544+0000","last_active":"2026-02-24T22:29:54.714013+0000","last_peered":"2026-02-24T22:29:54.714013+0000","last_clean":"2026-02-24T22:29:54.714013+0000","last_became_active":"2026-02-24T22:15:22.817150+0000","last_became_peered":"2026-02-24T22:15:22.817150+0000","last_unstale":"2026-02-24T22:29:54.714013+0000","last_undegraded":"2026-02-24T22:29:54.714013+0000","last_fullsized":"2026-02-24T22:29:54.714013+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:30:27.018037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.9","version":"312'24","reported_seq":125,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.737699+0000","last_change":"2026-02-24T22:13:43.757041+0000","last_active":"2026-02-24T22:31:04.737699+0000","last_peered":"2026-02-24T22:31:04.737699+0000","last_clean":"2026-02-24T22:31:04.737699+0000","last_became_active":"2026-02-24T22:13:43.756777+0000","last_became_peered":"2026-02-24T22:13:43.756777+0000","last_unstale":"2026-02-24T22:31:04.737699+0000","last_undegraded":"2026-02-24T22:31:04.737699+0000","last_fullsized":"2026-02-24T22:31:04.737699+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:54:52.853203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":107,"num_write":60,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.15","version":"310'1","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.733776+0000","last_change":"2026-02-24T22:23:29.198331+0000","last_active":"2026-02-24T22:30:44.733776+0000","last_peered":"2026-02-24T22:30:44.733776+0000","last_clean":"2026-02-24T22:30:44.733776+0000","last_became_active":"2026-02-24T22:23:29.198000+0000","last_became_peered":"2026-02-24T22:23:29.198000+0000","last_unstale":"2026-02-24T22:30:44.733776+0000","last_undegraded":"2026-02-24T22:30:44.733776+0000","last_fullsized":"2026-02-24T22:30:44.733776+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:34:21.768724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.16","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:59.732575+0000","last_change":"2026-02-24T22:15:25.828675+0000","last_active":"2026-02-24T22:29:59.732575+0000","last_peered":"2026-02-24T22:29:59.732575+0000","last_clean":"2026-02-24T22:29:59.732575+0000","last_became_active":"2026-02-24T22:15:25.828066+0000","last_became_peered":"2026-02-24T22:15:25.828066+0000","last_unstale":"2026-02-24T22:29:59.732575+0000","last_undegraded":"2026-02-24T22:29:59.732575+0000","last_fullsized":"2026-02-24T22:29:59.732575+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:13:41.162339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.17","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:54.730332+0000","last_change":"2026-02-24T22:15:22.818517+0000","last_active":"2026-02-24T22:29:54.730332+0000","last_peered":"2026-02-24T22:29:54.730332+0000","last_clean":"2026-02-24T22:29:54.730332+0000","last_became_active":"2026-02-24T22:15:22.817950+0000","last_became_peered":"2026-02-24T22:15:22.817950+0000","last_unstale":"2026-02-24T22:29:54.730332+0000","last_undegraded":"2026-02-24T22:29:54.730332+0000","last_fullsized":"2026-02-24T22:29:54.730332+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:47:07.389545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.8","version":"312'26","reported_seq":136,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.737895+0000","last_change":"2026-02-24T22:13:43.761888+0000","last_active":"2026-02-24T22:31:04.737895+0000","last_peered":"2026-02-24T22:31:04.737895+0000","last_clean":"2026-02-24T22:31:04.737895+0000","last_became_active":"2026-02-24T22:13:43.761558+0000","last_became_peered":"2026-02-24T22:13:43.761558+0000","last_unstale":"2026-02-24T22:31:04.737895+0000","last_undegraded":"2026-02-24T22:31:04.737895+0000","last_fullsized":"2026-02-24T22:31:04.737895+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:22:52.093177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":152,"num_write":62,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.1a","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.723700+0000","last_change":"2026-02-24T22:23:29.213344+0000","last_active":"2026-02-24T22:30:59.723700+0000","last_peered":"2026-02-24T22:30:59.723700+0000","last_clean":"2026-02-24T22:30:59.723700+0000","last_became_active":"2026-02-24T22:23:29.212709+0000","last_became_peered":"2026-02-24T22:23:29.212709+0000","last_unstale":"2026-02-24T22:30:59.723700+0000","last_undegraded":"2026-02-24T22:30:59.723700+0000","last_fullsized":"2026-02-24T22:30:59.723700+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:26:30.998666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.19","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.718214+0000","last_change":"2026-02-24T22:15:25.832162+0000","last_active":"2026-02-24T22:30:04.718214+0000","last_peered":"2026-02-24T22:30:04.718214+0000","last_clean":"2026-02-24T22:30:04.718214+0000","last_became_active":"2026-02-24T22:15:25.831885+0000","last_became_peered":"2026-02-24T22:15:25.831885+0000","last_unstale":"2026-02-24T22:30:04.718214+0000","last_undegraded":"2026-02-24T22:30:04.718214+0000","last_fullsized":"2026-02-24T22:30:04.718214+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:54:09.173036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.18","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:54.730283+0000","last_change":"2026-02-24T22:15:22.810683+0000","last_active":"2026-02-24T22:29:54.730283+0000","last_peered":"2026-02-24T22:29:54.730283+0000","last_clean":"2026-02-24T22:29:54.730283+0000","last_became_active":"2026-02-24T22:15:22.810156+0000","last_became_peered":"2026-02-24T22:15:22.810156+0000","last_unstale":"2026-02-24T22:29:54.730283+0000","last_undegraded":"2026-02-24T22:29:54.730283+0000","last_fullsized":"2026-02-24T22:29:54.730283+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:29:33.293117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"79.7","version":"0'0","reported_seq":64,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.740460+0000","last_change":"2026-02-24T22:13:43.756250+0000","last_active":"2026-02-24T22:31:04.740460+0000","last_peered":"2026-02-24T22:31:04.740460+0000","last_clean":"2026-02-24T22:31:04.740460+0000","last_became_active":"2026-02-24T22:13:43.755957+0000","last_became_peered":"2026-02-24T22:13:43.755957+0000","last_unstale":"2026-02-24T22:31:04.740460+0000","last_undegraded":"2026-02-24T22:31:04.740460+0000","last_fullsized":"2026-02-24T22:31:04.740460+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:16:21.505648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.1b","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.735465+0000","last_change":"2026-02-24T22:23:29.200410+0000","last_active":"2026-02-24T22:30:59.735465+0000","last_peered":"2026-02-24T22:30:59.735465+0000","last_clean":"2026-02-24T22:30:59.735465+0000","last_became_active":"2026-02-24T22:23:29.200152+0000","last_became_peered":"2026-02-24T22:23:29.200152+0000","last_unstale":"2026-02-24T22:30:59.735465+0000","last_undegraded":"2026-02-24T22:30:59.735465+0000","last_fullsized":"2026-02-24T22:30:59.735465+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:24:51.029708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.18","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:59.730819+0000","last_change":"2026-02-24T22:15:25.824057+0000","last_active":"2026-02-24T22:29:59.730819+0000","last_peered":"2026-02-24T22:29:59.730819+0000","last_clean":"2026-02-24T22:29:59.730819+0000","last_became_active":"2026-02-24T22:15:25.823415+0000","last_became_peered":"2026-02-24T22:15:25.823415+0000","last_unstale":"2026-02-24T22:29:59.730819+0000","last_undegraded":"2026-02-24T22:29:59.730819+0000","last_fullsized":"2026-02-24T22:29:59.730819+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:00:21.357856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.19","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:59.714470+0000","last_change":"2026-02-24T22:15:22.817214+0000","last_active":"2026-02-24T22:29:59.714470+0000","last_peered":"2026-02-24T22:29:59.714470+0000","last_clean":"2026-02-24T22:29:59.714470+0000","last_became_active":"2026-02-24T22:15:22.816826+0000","last_became_peered":"2026-02-24T22:15:22.816826+0000","last_unstale":"2026-02-24T22:29:59.714470+0000","last_undegraded":"2026-02-24T22:29:59.714470+0000","last_fullsized":"2026-02-24T22:29:59.714470+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:13:51.793483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.6","version":"301'14","reported_seq":108,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.739656+0000","last_change":"2026-02-24T22:13:43.755144+0000","last_active":"2026-02-24T22:31:04.739656+0000","last_peered":"2026-02-24T22:31:04.739656+0000","last_clean":"2026-02-24T22:31:04.739656+0000","last_became_active":"2026-02-24T22:13:43.754753+0000","last_became_peered":"2026-02-24T22:13:43.754753+0000","last_unstale":"2026-02-24T22:31:04.739656+0000","last_undegraded":"2026-02-24T22:31:04.739656+0000","last_fullsized":"2026-02-24T22:31:04.739656+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:57:00.870335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":92,"num_write":34,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"82.18","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.726953+0000","last_change":"2026-02-24T22:23:29.213704+0000","last_active":"2026-02-24T22:30:44.726953+0000","last_peered":"2026-02-24T22:30:44.726953+0000","last_clean":"2026-02-24T22:30:44.726953+0000","last_became_active":"2026-02-24T22:23:29.213387+0000","last_became_peered":"2026-02-24T22:23:29.213387+0000","last_unstale":"2026-02-24T22:30:44.726953+0000","last_undegraded":"2026-02-24T22:30:44.726953+0000","last_fullsized":"2026-02-24T22:30:44.726953+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:46:11.186061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.1b","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.720738+0000","last_change":"2026-02-24T22:15:25.832293+0000","last_active":"2026-02-24T22:30:29.720738+0000","last_peered":"2026-02-24T22:30:29.720738+0000","last_clean":"2026-02-24T22:30:29.720738+0000","last_became_active":"2026-02-24T22:15:25.832097+0000","last_became_peered":"2026-02-24T22:15:25.832097+0000","last_unstale":"2026-02-24T22:30:29.720738+0000","last_undegraded":"2026-02-24T22:30:29.720738+0000","last_fullsized":"2026-02-24T22:30:29.720738+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:19:51.400688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.1a","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.720229+0000","last_change":"2026-02-24T22:15:22.819032+0000","last_active":"2026-02-24T22:30:24.720229+0000","last_peered":"2026-02-24T22:30:24.720229+0000","last_clean":"2026-02-24T22:30:24.720229+0000","last_became_active":"2026-02-24T22:15:22.818885+0000","last_became_peered":"2026-02-24T22:15:22.818885+0000","last_unstale":"2026-02-24T22:30:24.720229+0000","last_undegraded":"2026-02-24T22:30:24.720229+0000","last_fullsized":"2026-02-24T22:30:24.720229+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:15:52.421379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"79.5","version":"312'19","reported_seq":121,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:49.005257+0000","last_change":"2026-02-24T22:13:43.756043+0000","last_active":"2026-02-24T22:30:49.005257+0000","last_peered":"2026-02-24T22:30:49.005257+0000","last_clean":"2026-02-24T22:30:49.005257+0000","last_became_active":"2026-02-24T22:13:43.755495+0000","last_became_peered":"2026-02-24T22:13:43.755495+0000","last_unstale":"2026-02-24T22:30:49.005257+0000","last_undegraded":"2026-02-24T22:30:49.005257+0000","last_fullsized":"2026-02-24T22:30:49.005257+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:47:41.399446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":164,"num_read_kb":112,"num_write":47,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.19","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:49.719308+0000","last_change":"2026-02-24T22:23:29.212738+0000","last_active":"2026-02-24T22:30:49.719308+0000","last_peered":"2026-02-24T22:30:49.719308+0000","last_clean":"2026-02-24T22:30:49.719308+0000","last_became_active":"2026-02-24T22:23:29.212268+0000","last_became_peered":"2026-02-24T22:23:29.212268+0000","last_unstale":"2026-02-24T22:30:49.719308+0000","last_undegraded":"2026-02-24T22:30:49.719308+0000","last_fullsized":"2026-02-24T22:30:49.719308+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:32:25.776768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.1a","version":"307'2","reported_seq":35,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.731903+0000","last_change":"2026-02-24T22:15:25.826222+0000","last_active":"2026-02-24T22:30:29.731903+0000","last_peered":"2026-02-24T22:30:29.731903+0000","last_clean":"2026-02-24T22:30:29.731903+0000","last_became_active":"2026-02-24T22:15:25.825969+0000","last_became_peered":"2026-02-24T22:15:25.825969+0000","last_unstale":"2026-02-24T22:30:29.731903+0000","last_undegraded":"2026-02-24T22:30:29.731903+0000","last_fullsized":"2026-02-24T22:30:29.731903+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:29:05.133831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"80.1b","version":"307'2","reported_seq":41,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.733578+0000","last_change":"2026-02-24T22:15:22.816734+0000","last_active":"2026-02-24T22:30:24.733578+0000","last_peered":"2026-02-24T22:30:24.733578+0000","last_clean":"2026-02-24T22:30:24.733578+0000","last_became_active":"2026-02-24T22:15:22.816479+0000","last_became_peered":"2026-02-24T22:15:22.816479+0000","last_unstale":"2026-02-24T22:30:24.733578+0000","last_undegraded":"2026-02-24T22:30:24.733578+0000","last_fullsized":"2026-02-24T22:30:24.733578+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:57:39.996802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.4","version":"0'0","reported_seq":70,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.735005+0000","last_change":"2026-02-24T22:14:00.742004+0000","last_active":"2026-02-24T22:30:24.735005+0000","last_peered":"2026-02-24T22:30:24.735005+0000","last_clean":"2026-02-24T22:30:24.735005+0000","last_became_active":"2026-02-24T22:14:00.741839+0000","last_became_peered":"2026-02-24T22:14:00.741839+0000","last_unstale":"2026-02-24T22:30:24.735005+0000","last_undegraded":"2026-02-24T22:30:24.735005+0000","last_fullsized":"2026-02-24T22:30:24.735005+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":293,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:38:11.152954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"82.1e","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.727290+0000","last_change":"2026-02-24T22:23:29.215779+0000","last_active":"2026-02-24T22:31:34.727290+0000","last_peered":"2026-02-24T22:31:34.727290+0000","last_clean":"2026-02-24T22:31:34.727290+0000","last_became_active":"2026-02-24T22:23:29.215604+0000","last_became_peered":"2026-02-24T22:23:29.215604+0000","last_unstale":"2026-02-24T22:31:34.727290+0000","last_undegraded":"2026-02-24T22:31:34.727290+0000","last_fullsized":"2026-02-24T22:31:34.727290+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:58:56.390417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.1d","version":"307'2","reported_seq":35,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.733670+0000","last_change":"2026-02-24T22:15:25.828517+0000","last_active":"2026-02-24T22:30:29.733670+0000","last_peered":"2026-02-24T22:30:29.733670+0000","last_clean":"2026-02-24T22:30:29.733670+0000","last_became_active":"2026-02-24T22:15:25.828335+0000","last_became_peered":"2026-02-24T22:15:25.828335+0000","last_unstale":"2026-02-24T22:30:29.733670+0000","last_undegraded":"2026-02-24T22:30:29.733670+0000","last_fullsized":"2026-02-24T22:30:29.733670+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:18:15.606086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.1c","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.733619+0000","last_change":"2026-02-24T22:15:22.816853+0000","last_active":"2026-02-24T22:30:24.733619+0000","last_peered":"2026-02-24T22:30:24.733619+0000","last_clean":"2026-02-24T22:30:24.733619+0000","last_became_active":"2026-02-24T22:15:22.816548+0000","last_became_peered":"2026-02-24T22:15:22.816548+0000","last_unstale":"2026-02-24T22:30:24.733619+0000","last_undegraded":"2026-02-24T22:30:24.733619+0000","last_fullsized":"2026-02-24T22:30:24.733619+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:00:55.883988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.3","version":"312'6","reported_seq":77,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.731442+0000","last_change":"2026-02-24T22:13:43.756810+0000","last_active":"2026-02-24T22:30:24.731442+0000","last_peered":"2026-02-24T22:30:24.731442+0000","last_clean":"2026-02-24T22:30:24.731442+0000","last_became_active":"2026-02-24T22:13:43.756408+0000","last_became_peered":"2026-02-24T22:13:43.756408+0000","last_unstale":"2026-02-24T22:30:24.731442+0000","last_undegraded":"2026-02-24T22:30:24.731442+0000","last_fullsized":"2026-02-24T22:30:24.731442+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:24:33.412598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":20,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.1f","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744790+0000","last_change":"2026-02-24T22:23:29.199410+0000","last_active":"2026-02-24T22:31:49.744790+0000","last_peered":"2026-02-24T22:31:49.744790+0000","last_clean":"2026-02-24T22:31:49.744790+0000","last_became_active":"2026-02-24T22:23:29.199179+0000","last_became_peered":"2026-02-24T22:23:29.199179+0000","last_unstale":"2026-02-24T22:31:49.744790+0000","last_undegraded":"2026-02-24T22:31:49.744790+0000","last_fullsized":"2026-02-24T22:31:49.744790+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:21:23.823724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.1c","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.725548+0000","last_change":"2026-02-24T22:15:25.834316+0000","last_active":"2026-02-24T22:30:29.725548+0000","last_peered":"2026-02-24T22:30:29.725548+0000","last_clean":"2026-02-24T22:30:29.725548+0000","last_became_active":"2026-02-24T22:15:25.834072+0000","last_became_peered":"2026-02-24T22:15:25.834072+0000","last_unstale":"2026-02-24T22:30:29.725548+0000","last_undegraded":"2026-02-24T22:30:29.725548+0000","last_fullsized":"2026-02-24T22:30:29.725548+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:45:39.925744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"80.1d","version":"304'1","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.733639+0000","last_change":"2026-02-24T22:15:22.817173+0000","last_active":"2026-02-24T22:30:24.733639+0000","last_peered":"2026-02-24T22:30:24.733639+0000","last_clean":"2026-02-24T22:30:24.733639+0000","last_became_active":"2026-02-24T22:15:22.816615+0000","last_became_peered":"2026-02-24T22:15:22.816615+0000","last_unstale":"2026-02-24T22:30:24.733639+0000","last_undegraded":"2026-02-24T22:30:24.733639+0000","last_fullsized":"2026-02-24T22:30:24.733639+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:25:34.323299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":4096,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.2","version":"312'5","reported_seq":77,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.733168+0000","last_change":"2026-02-24T22:13:43.755888+0000","last_active":"2026-02-24T22:30:24.733168+0000","last_peered":"2026-02-24T22:30:24.733168+0000","last_clean":"2026-02-24T22:30:24.733168+0000","last_became_active":"2026-02-24T22:13:43.755241+0000","last_became_peered":"2026-02-24T22:13:43.755241+0000","last_unstale":"2026-02-24T22:30:24.733168+0000","last_undegraded":"2026-02-24T22:30:24.733168+0000","last_fullsized":"2026-02-24T22:30:24.733168+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:13:28.809470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":23,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.1c","version":"310'1","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.720361+0000","last_change":"2026-02-24T22:23:29.213715+0000","last_active":"2026-02-24T22:30:59.720361+0000","last_peered":"2026-02-24T22:30:59.720361+0000","last_clean":"2026-02-24T22:30:59.720361+0000","last_became_active":"2026-02-24T22:23:29.213490+0000","last_became_peered":"2026-02-24T22:23:29.213490+0000","last_unstale":"2026-02-24T22:30:59.720361+0000","last_undegraded":"2026-02-24T22:30:59.720361+0000","last_fullsized":"2026-02-24T22:30:59.720361+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:21:40.032070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.1f","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.747978+0000","last_change":"2026-02-24T22:15:25.828181+0000","last_active":"2026-02-24T22:32:09.747978+0000","last_peered":"2026-02-24T22:32:09.747978+0000","last_clean":"2026-02-24T22:32:09.747978+0000","last_became_active":"2026-02-24T22:15:25.827873+0000","last_became_peered":"2026-02-24T22:15:25.827873+0000","last_unstale":"2026-02-24T22:32:09.747978+0000","last_undegraded":"2026-02-24T22:32:09.747978+0000","last_fullsized":"2026-02-24T22:32:09.747978+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:19:57.290828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.1e","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:29.726743+0000","last_change":"2026-02-24T22:15:22.817535+0000","last_active":"2026-02-24T22:31:29.726743+0000","last_peered":"2026-02-24T22:31:29.726743+0000","last_clean":"2026-02-24T22:31:29.726743+0000","last_became_active":"2026-02-24T22:15:22.817337+0000","last_became_peered":"2026-02-24T22:15:22.817337+0000","last_unstale":"2026-02-24T22:31:29.726743+0000","last_undegraded":"2026-02-24T22:31:29.726743+0000","last_fullsized":"2026-02-24T22:31:29.726743+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:02:48.141978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"79.1","version":"312'23","reported_seq":125,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.720202+0000","last_change":"2026-02-24T22:13:43.761201+0000","last_active":"2026-02-24T22:30:24.720202+0000","last_peered":"2026-02-24T22:30:24.720202+0000","last_clean":"2026-02-24T22:30:24.720202+0000","last_became_active":"2026-02-24T22:13:43.760933+0000","last_became_peered":"2026-02-24T22:13:43.760933+0000","last_unstale":"2026-02-24T22:30:24.720202+0000","last_undegraded":"2026-02-24T22:30:24.720202+0000","last_fullsized":"2026-02-24T22:30:24.720202+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:18:32.108308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":174,"num_read_kb":113,"num_write":59,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.1d","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.723721+0000","last_change":"2026-02-24T22:23:29.215553+0000","last_active":"2026-02-24T22:30:59.723721+0000","last_peered":"2026-02-24T22:30:59.723721+0000","last_clean":"2026-02-24T22:30:59.723721+0000","last_became_active":"2026-02-24T22:23:29.215334+0000","last_became_peered":"2026-02-24T22:23:29.215334+0000","last_unstale":"2026-02-24T22:30:59.723721+0000","last_undegraded":"2026-02-24T22:30:59.723721+0000","last_fullsized":"2026-02-24T22:30:59.723721+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:01:42.978764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.1e","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.739370+0000","last_change":"2026-02-24T22:15:25.827409+0000","last_active":"2026-02-24T22:31:34.739370+0000","last_peered":"2026-02-24T22:31:34.739370+0000","last_clean":"2026-02-24T22:31:34.739370+0000","last_became_active":"2026-02-24T22:15:25.827124+0000","last_became_peered":"2026-02-24T22:15:25.827124+0000","last_unstale":"2026-02-24T22:31:34.739370+0000","last_undegraded":"2026-02-24T22:31:34.739370+0000","last_fullsized":"2026-02-24T22:31:34.739370+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:44:21.789639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"80.1f","version":"304'2","reported_seq":43,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:04.726966+0000","last_change":"2026-02-24T22:15:22.816227+0000","last_active":"2026-02-24T22:32:04.726966+0000","last_peered":"2026-02-24T22:32:04.726966+0000","last_clean":"2026-02-24T22:32:04.726966+0000","last_became_active":"2026-02-24T22:15:22.815938+0000","last_became_peered":"2026-02-24T22:15:22.815938+0000","last_unstale":"2026-02-24T22:32:04.726966+0000","last_undegraded":"2026-02-24T22:32:04.726966+0000","last_fullsized":"2026-02-24T22:32:04.726966+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:40:15.210510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.0","version":"0'0","reported_seq":65,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.745118+0000","last_change":"2026-02-24T22:13:43.755907+0000","last_active":"2026-02-24T22:31:49.745118+0000","last_peered":"2026-02-24T22:31:49.745118+0000","last_clean":"2026-02-24T22:31:49.745118+0000","last_became_active":"2026-02-24T22:13:43.755266+0000","last_became_peered":"2026-02-24T22:13:43.755266+0000","last_unstale":"2026-02-24T22:31:49.745118+0000","last_undegraded":"2026-02-24T22:31:49.745118+0000","last_fullsized":"2026-02-24T22:31:49.745118+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:42:48.314773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"312'215","reported_seq":920,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:47.193151+0000","last_change":"2026-02-24T22:05:16.193051+0000","last_active":"2026-02-24T22:33:47.193151+0000","last_peered":"2026-02-24T22:33:47.193151+0000","last_clean":"2026-02-24T22:33:47.193151+0000","last_became_active":"2026-02-24T22:05:16.192589+0000","last_became_peered":"2026-02-24T22:05:16.192589+0000","last_unstale":"2026-02-24T22:33:47.193151+0000","last_undegraded":"2026-02-24T22:33:47.193151+0000","last_fullsized":"2026-02-24T22:33:47.193151+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:20:06.945662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8436,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":377,"num_read_kb":342,"num_write":1213,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"312'197","reported_seq":898,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:46.730087+0000","last_change":"2026-02-24T22:05:18.192403+0000","last_active":"2026-02-24T22:33:46.730087+0000","last_peered":"2026-02-24T22:33:46.730087+0000","last_clean":"2026-02-24T22:33:46.730087+0000","last_became_active":"2026-02-24T22:05:18.192203+0000","last_became_peered":"2026-02-24T22:05:18.192203+0000","last_unstale":"2026-02-24T22:33:46.730087+0000","last_undegraded":"2026-02-24T22:33:46.730087+0000","last_fullsized":"2026-02-24T22:33:46.730087+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":100,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:33:32.754536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":780,"num_read_kb":643,"num_write":282,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"312'304","reported_seq":1112,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:51.011231+0000","last_change":"2026-02-24T22:05:16.181637+0000","last_active":"2026-02-24T22:30:51.011231+0000","last_peered":"2026-02-24T22:30:51.011231+0000","last_clean":"2026-02-24T22:30:51.011231+0000","last_became_active":"2026-02-24T22:05:16.181379+0000","last_became_peered":"2026-02-24T22:05:16.181379+0000","last_unstale":"2026-02-24T22:30:51.011231+0000","last_undegraded":"2026-02-24T22:30:51.011231+0000","last_fullsized":"2026-02-24T22:30:51.011231+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":200,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:35:21.970288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21628,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":848,"num_read_kb":611,"num_write":1466,"num_write_kb":5240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"312'459","reported_seq":1257,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:57.922033+0000","last_change":"2026-02-24T22:05:18.190910+0000","last_active":"2026-02-24T22:30:57.922033+0000","last_peered":"2026-02-24T22:30:57.922033+0000","last_clean":"2026-02-24T22:30:57.922033+0000","last_became_active":"2026-02-24T22:05:18.190689+0000","last_became_peered":"2026-02-24T22:05:18.190689+0000","last_unstale":"2026-02-24T22:30:57.922033+0000","last_undegraded":"2026-02-24T22:30:57.922033+0000","last_fullsized":"2026-02-24T22:30:57.922033+0000","mapping_epoch":24,"log_start":"312'400","ondisk_log_start":"312'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":400,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:35:35.142199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":2017,"num_read_kb":1670,"num_write":788,"num_write_kb":502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"312'368","reported_seq":1154,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:58.398455+0000","last_change":"2026-02-24T22:05:16.180825+0000","last_active":"2026-02-24T22:30:58.398455+0000","last_peered":"2026-02-24T22:30:58.398455+0000","last_clean":"2026-02-24T22:30:58.398455+0000","last_became_active":"2026-02-24T22:05:16.180681+0000","last_became_peered":"2026-02-24T22:05:16.180681+0000","last_unstale":"2026-02-24T22:30:58.398455+0000","last_undegraded":"2026-02-24T22:30:58.398455+0000","last_fullsized":"2026-02-24T22:30:58.398455+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":300,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:38:22.293881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5517838,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":825,"num_read_kb":6166,"num_write":1529,"num_write_kb":5459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"312'80","reported_seq":727,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.732683+0000","last_change":"2026-02-24T22:05:18.192661+0000","last_active":"2026-02-24T22:30:19.732683+0000","last_peered":"2026-02-24T22:30:19.732683+0000","last_clean":"2026-02-24T22:30:19.732683+0000","last_became_active":"2026-02-24T22:05:18.190727+0000","last_became_peered":"2026-02-24T22:05:18.190727+0000","last_unstale":"2026-02-24T22:30:19.732683+0000","last_undegraded":"2026-02-24T22:30:19.732683+0000","last_fullsized":"2026-02-24T22:30:19.732683+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:10:39.624197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":268,"num_read_kb":220,"num_write":112,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"312'218","reported_seq":902,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:01.970038+0000","last_change":"2026-02-24T22:05:16.193170+0000","last_active":"2026-02-24T22:34:01.970038+0000","last_peered":"2026-02-24T22:34:01.970038+0000","last_clean":"2026-02-24T22:34:01.970038+0000","last_became_active":"2026-02-24T22:05:16.192922+0000","last_became_peered":"2026-02-24T22:05:16.192922+0000","last_unstale":"2026-02-24T22:34:01.970038+0000","last_undegraded":"2026-02-24T22:34:01.970038+0000","last_fullsized":"2026-02-24T22:34:01.970038+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":200,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:51:56.894641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":549324,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":307,"num_read_kb":3190,"num_write":1131,"num_write_kb":12582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"312'135","reported_seq":815,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.733034+0000","last_change":"2026-02-24T22:05:18.184446+0000","last_active":"2026-02-24T22:30:19.733034+0000","last_peered":"2026-02-24T22:30:19.733034+0000","last_clean":"2026-02-24T22:30:19.733034+0000","last_became_active":"2026-02-24T22:05:18.184014+0000","last_became_peered":"2026-02-24T22:05:18.184014+0000","last_unstale":"2026-02-24T22:30:19.733034+0000","last_undegraded":"2026-02-24T22:30:19.733034+0000","last_fullsized":"2026-02-24T22:30:19.733034+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:12:45.735332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":489,"num_read_kb":401,"num_write":208,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"312'208","reported_seq":919,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:12.258587+0000","last_change":"2026-02-24T22:05:16.191359+0000","last_active":"2026-02-24T22:34:12.258587+0000","last_peered":"2026-02-24T22:34:12.258587+0000","last_clean":"2026-02-24T22:34:12.258587+0000","last_became_active":"2026-02-24T22:05:16.191055+0000","last_became_peered":"2026-02-24T22:05:16.191055+0000","last_unstale":"2026-02-24T22:34:12.258587+0000","last_undegraded":"2026-02-24T22:34:12.258587+0000","last_fullsized":"2026-02-24T22:34:12.258587+0000","mapping_epoch":22,"log_start":"312'100","ondisk_log_start":"312'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":100,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:40:09.331901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6857028,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":360,"num_read_kb":15420,"num_write":1138,"num_write_kb":8454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"312'470","reported_seq":1186,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.738353+0000","last_change":"2026-02-24T22:05:18.190983+0000","last_active":"2026-02-24T22:31:24.738353+0000","last_peered":"2026-02-24T22:31:24.738353+0000","last_clean":"2026-02-24T22:31:24.738353+0000","last_became_active":"2026-02-24T22:05:18.190598+0000","last_became_peered":"2026-02-24T22:05:18.190598+0000","last_unstale":"2026-02-24T22:31:24.738353+0000","last_undegraded":"2026-02-24T22:31:24.738353+0000","last_fullsized":"2026-02-24T22:31:24.738353+0000","mapping_epoch":24,"log_start":"312'400","ondisk_log_start":"312'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":400,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:52:56.661245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1905,"num_read_kb":1522,"num_write":772,"num_write_kb":389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"312'233","reported_seq":943,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:50.432924+0000","last_change":"2026-02-24T22:05:16.191753+0000","last_active":"2026-02-24T22:30:50.432924+0000","last_peered":"2026-02-24T22:30:50.432924+0000","last_clean":"2026-02-24T22:30:50.432924+0000","last_became_active":"2026-02-24T22:05:16.191164+0000","last_became_peered":"2026-02-24T22:05:16.191164+0000","last_unstale":"2026-02-24T22:30:50.432924+0000","last_undegraded":"2026-02-24T22:30:50.432924+0000","last_fullsized":"2026-02-24T22:30:50.432924+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":200,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:23:55.135404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5295803,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":424,"num_read_kb":5551,"num_write":1270,"num_write_kb":5240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"312'56","reported_seq":709,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.732718+0000","last_change":"2026-02-24T22:05:18.198708+0000","last_active":"2026-02-24T22:30:19.732718+0000","last_peered":"2026-02-24T22:30:19.732718+0000","last_clean":"2026-02-24T22:30:19.732718+0000","last_became_active":"2026-02-24T22:05:18.198559+0000","last_became_peered":"2026-02-24T22:05:18.198559+0000","last_unstale":"2026-02-24T22:30:19.732718+0000","last_undegraded":"2026-02-24T22:30:19.732718+0000","last_fullsized":"2026-02-24T22:30:19.732718+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:01:34.891706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":195,"num_read_kb":167,"num_write":70,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"312'219","reported_seq":919,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:57.717745+0000","last_change":"2026-02-24T22:05:16.190594+0000","last_active":"2026-02-24T22:30:57.717745+0000","last_peered":"2026-02-24T22:30:57.717745+0000","last_clean":"2026-02-24T22:30:57.717745+0000","last_became_active":"2026-02-24T22:05:16.190367+0000","last_became_peered":"2026-02-24T22:05:16.190367+0000","last_unstale":"2026-02-24T22:30:57.717745+0000","last_undegraded":"2026-02-24T22:30:57.717745+0000","last_fullsized":"2026-02-24T22:30:57.717745+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":200,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:34:28.350502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1051952,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":361,"num_read_kb":1335,"num_write":1155,"num_write_kb":1104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"312'201","reported_seq":941,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:49.857860+0000","last_change":"2026-02-24T22:05:18.192094+0000","last_active":"2026-02-24T22:33:49.857860+0000","last_peered":"2026-02-24T22:33:49.857860+0000","last_clean":"2026-02-24T22:33:49.857860+0000","last_became_active":"2026-02-24T22:05:18.191767+0000","last_became_peered":"2026-02-24T22:05:18.191767+0000","last_unstale":"2026-02-24T22:33:49.857860+0000","last_undegraded":"2026-02-24T22:33:49.857860+0000","last_fullsized":"2026-02-24T22:33:49.857860+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":100,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:15:25.401981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":876,"num_read_kb":754,"num_write":323,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"312'272","reported_seq":1042,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:40.186391+0000","last_change":"2026-02-24T22:05:16.186732+0000","last_active":"2026-02-24T22:33:40.186391+0000","last_peered":"2026-02-24T22:33:40.186391+0000","last_clean":"2026-02-24T22:33:40.186391+0000","last_became_active":"2026-02-24T22:05:16.185779+0000","last_became_peered":"2026-02-24T22:05:16.185779+0000","last_unstale":"2026-02-24T22:33:40.186391+0000","last_undegraded":"2026-02-24T22:33:40.186391+0000","last_fullsized":"2026-02-24T22:33:40.186391+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":200,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:01:24.063571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50495,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":533,"num_read_kb":877,"num_write":1531,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"312'83","reported_seq":729,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.716462+0000","last_change":"2026-02-24T22:05:18.188067+0000","last_active":"2026-02-24T22:30:19.716462+0000","last_peered":"2026-02-24T22:30:19.716462+0000","last_clean":"2026-02-24T22:30:19.716462+0000","last_became_active":"2026-02-24T22:05:18.187838+0000","last_became_peered":"2026-02-24T22:05:18.187838+0000","last_unstale":"2026-02-24T22:30:19.716462+0000","last_undegraded":"2026-02-24T22:30:19.716462+0000","last_fullsized":"2026-02-24T22:30:19.716462+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:05:06.602774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":263,"num_read_kb":217,"num_write":106,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"312'135","reported_seq":808,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.736992+0000","last_change":"2026-02-24T22:05:18.192713+0000","last_active":"2026-02-24T22:30:59.736992+0000","last_peered":"2026-02-24T22:30:59.736992+0000","last_clean":"2026-02-24T22:30:59.736992+0000","last_became_active":"2026-02-24T22:05:18.191625+0000","last_became_peered":"2026-02-24T22:05:18.191625+0000","last_unstale":"2026-02-24T22:30:59.736992+0000","last_undegraded":"2026-02-24T22:30:59.736992+0000","last_fullsized":"2026-02-24T22:30:59.736992+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:24:04.071735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":477,"num_read_kb":396,"num_write":183,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"312'237","reported_seq":974,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:53.497360+0000","last_change":"2026-02-24T22:06:01.025897+0000","last_active":"2026-02-24T22:33:53.497360+0000","last_peered":"2026-02-24T22:33:53.497360+0000","last_clean":"2026-02-24T22:33:53.497360+0000","last_became_active":"2026-02-24T22:06:01.025659+0000","last_became_peered":"2026-02-24T22:06:01.025659+0000","last_unstale":"2026-02-24T22:33:53.497360+0000","last_undegraded":"2026-02-24T22:33:53.497360+0000","last_fullsized":"2026-02-24T22:33:53.497360+0000","mapping_epoch":36,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":200,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:15:11.362680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":623937,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":456,"num_read_kb":16219,"num_write":1259,"num_write_kb":8500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"312'4249","reported_seq":5059,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.162950+0000","last_change":"2026-02-24T22:05:18.193454+0000","last_active":"2026-02-24T22:34:15.162950+0000","last_peered":"2026-02-24T22:34:15.162950+0000","last_clean":"2026-02-24T22:34:15.162950+0000","last_became_active":"2026-02-24T22:05:18.193257+0000","last_became_peered":"2026-02-24T22:05:18.193257+0000","last_unstale":"2026-02-24T22:34:15.162950+0000","last_undegraded":"2026-02-24T22:34:15.162950+0000","last_fullsized":"2026-02-24T22:34:15.162950+0000","mapping_epoch":24,"log_start":"312'4200","ondisk_log_start":"312'4200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":3000,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:25:08.863245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":17332,"num_read_kb":14553,"num_write":6398,"num_write_kb":3198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"312'213","reported_seq":925,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.741319+0000","last_change":"2026-02-24T22:05:16.191122+0000","last_active":"2026-02-24T22:31:54.741319+0000","last_peered":"2026-02-24T22:31:54.741319+0000","last_clean":"2026-02-24T22:31:54.741319+0000","last_became_active":"2026-02-24T22:05:16.190819+0000","last_became_peered":"2026-02-24T22:05:16.190819+0000","last_unstale":"2026-02-24T22:31:54.741319+0000","last_undegraded":"2026-02-24T22:31:54.741319+0000","last_fullsized":"2026-02-24T22:31:54.741319+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":200,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:31:00.005386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5568323,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":396,"num_read_kb":8113,"num_write":1091,"num_write_kb":5510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"312'862","reported_seq":4024,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.230995+0000","last_change":"2026-02-24T22:05:18.190719+0000","last_active":"2026-02-24T22:34:17.230995+0000","last_peered":"2026-02-24T22:34:17.230995+0000","last_clean":"2026-02-24T22:34:17.230995+0000","last_became_active":"2026-02-24T22:05:18.190363+0000","last_became_peered":"2026-02-24T22:05:18.190363+0000","last_unstale":"2026-02-24T22:34:17.230995+0000","last_undegraded":"2026-02-24T22:34:17.230995+0000","last_fullsized":"2026-02-24T22:34:17.230995+0000","mapping_epoch":24,"log_start":"312'800","ondisk_log_start":"312'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":800,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:46:25.367911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4747,"num_read_kb":5785,"num_write":398,"num_write_kb":236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"312'229","reported_seq":960,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:03.733775+0000","last_change":"2026-02-24T22:05:16.191246+0000","last_active":"2026-02-24T22:34:03.733775+0000","last_peered":"2026-02-24T22:34:03.733775+0000","last_clean":"2026-02-24T22:34:03.733775+0000","last_became_active":"2026-02-24T22:05:16.190816+0000","last_became_peered":"2026-02-24T22:05:16.190816+0000","last_unstale":"2026-02-24T22:34:03.733775+0000","last_undegraded":"2026-02-24T22:34:03.733775+0000","last_fullsized":"2026-02-24T22:34:03.733775+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":200,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:47:51.922901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10732886,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":476,"num_read_kb":2322,"num_write":1236,"num_write_kb":12235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"312'93","reported_seq":796,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.725906+0000","last_change":"2026-02-24T22:05:18.187293+0000","last_active":"2026-02-24T22:31:54.725906+0000","last_peered":"2026-02-24T22:31:54.725906+0000","last_clean":"2026-02-24T22:31:54.725906+0000","last_became_active":"2026-02-24T22:05:18.186994+0000","last_became_peered":"2026-02-24T22:05:18.186994+0000","last_unstale":"2026-02-24T22:31:54.725906+0000","last_undegraded":"2026-02-24T22:31:54.725906+0000","last_fullsized":"2026-02-24T22:31:54.725906+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:00:02.034352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":352,"num_read_kb":306,"num_write":116,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"312'315","reported_seq":1107,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:54.369202+0000","last_change":"2026-02-24T22:05:16.189106+0000","last_active":"2026-02-24T22:33:54.369202+0000","last_peered":"2026-02-24T22:33:54.369202+0000","last_clean":"2026-02-24T22:33:54.369202+0000","last_became_active":"2026-02-24T22:05:16.188658+0000","last_became_peered":"2026-02-24T22:05:16.188658+0000","last_unstale":"2026-02-24T22:33:54.369202+0000","last_undegraded":"2026-02-24T22:33:54.369202+0000","last_fullsized":"2026-02-24T22:33:54.369202+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:33:27.530851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52140,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":775,"num_read_kb":685,"num_write":1754,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"312'169","reported_seq":891,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.245262+0000","last_change":"2026-02-24T22:05:18.187949+0000","last_active":"2026-02-24T22:34:18.245262+0000","last_peered":"2026-02-24T22:34:18.245262+0000","last_clean":"2026-02-24T22:34:18.245262+0000","last_became_active":"2026-02-24T22:05:18.187682+0000","last_became_peered":"2026-02-24T22:05:18.187682+0000","last_unstale":"2026-02-24T22:34:18.245262+0000","last_undegraded":"2026-02-24T22:34:18.245262+0000","last_fullsized":"2026-02-24T22:34:18.245262+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:07:24.443062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":639,"num_read_kb":531,"num_write":250,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"312'224","reported_seq":924,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:45.924671+0000","last_change":"2026-02-24T22:05:16.185278+0000","last_active":"2026-02-24T22:33:45.924671+0000","last_peered":"2026-02-24T22:33:45.924671+0000","last_clean":"2026-02-24T22:33:45.924671+0000","last_became_active":"2026-02-24T22:05:16.184209+0000","last_became_peered":"2026-02-24T22:05:16.184209+0000","last_unstale":"2026-02-24T22:33:45.924671+0000","last_undegraded":"2026-02-24T22:33:45.924671+0000","last_fullsized":"2026-02-24T22:33:45.924671+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":200,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:50:09.257349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297746,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":353,"num_read_kb":601,"num_write":1169,"num_write_kb":10636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"312'111","reported_seq":789,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.222919+0000","last_change":"2026-02-24T22:05:18.194092+0000","last_active":"2026-02-24T22:34:18.222919+0000","last_peered":"2026-02-24T22:34:18.222919+0000","last_clean":"2026-02-24T22:34:18.222919+0000","last_became_active":"2026-02-24T22:05:18.193320+0000","last_became_peered":"2026-02-24T22:05:18.193320+0000","last_unstale":"2026-02-24T22:34:18.222919+0000","last_undegraded":"2026-02-24T22:34:18.222919+0000","last_fullsized":"2026-02-24T22:34:18.222919+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":100,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:27:50.398388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":425,"num_read_kb":355,"num_write":146,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"312'212","reported_seq":925,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:46.698672+0000","last_change":"2026-02-24T22:05:16.181803+0000","last_active":"2026-02-24T22:33:46.698672+0000","last_peered":"2026-02-24T22:33:46.698672+0000","last_clean":"2026-02-24T22:33:46.698672+0000","last_became_active":"2026-02-24T22:05:16.181527+0000","last_became_peered":"2026-02-24T22:05:16.181527+0000","last_unstale":"2026-02-24T22:33:46.698672+0000","last_undegraded":"2026-02-24T22:33:46.698672+0000","last_fullsized":"2026-02-24T22:33:46.698672+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:53:05.002989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5243778,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":406,"num_read_kb":5463,"num_write":1149,"num_write_kb":5189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:39.728581+0000","last_change":"2026-02-24T22:05:29.658481+0000","last_active":"2026-02-24T22:29:39.728581+0000","last_peered":"2026-02-24T22:29:39.728581+0000","last_clean":"2026-02-24T22:29:39.728581+0000","last_became_active":"2026-02-24T22:05:29.657764+0000","last_became_peered":"2026-02-24T22:05:29.657764+0000","last_unstale":"2026-02-24T22:29:39.728581+0000","last_undegraded":"2026-02-24T22:29:39.728581+0000","last_fullsized":"2026-02-24T22:29:39.728581+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:34:57.407311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"312'80","reported_seq":734,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.729056+0000","last_change":"2026-02-24T22:05:18.185097+0000","last_active":"2026-02-24T22:31:04.729056+0000","last_peered":"2026-02-24T22:31:04.729056+0000","last_clean":"2026-02-24T22:31:04.729056+0000","last_became_active":"2026-02-24T22:05:18.184771+0000","last_became_peered":"2026-02-24T22:05:18.184771+0000","last_unstale":"2026-02-24T22:31:04.729056+0000","last_undegraded":"2026-02-24T22:31:04.729056+0000","last_fullsized":"2026-02-24T22:31:04.729056+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:34:23.195340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":246,"num_read_kb":208,"num_write":102,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"312'259","reported_seq":1040,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:03.524714+0000","last_change":"2026-02-24T22:05:16.192243+0000","last_active":"2026-02-24T22:34:03.524714+0000","last_peered":"2026-02-24T22:34:03.524714+0000","last_clean":"2026-02-24T22:34:03.524714+0000","last_became_active":"2026-02-24T22:05:16.191841+0000","last_became_peered":"2026-02-24T22:05:16.191841+0000","last_unstale":"2026-02-24T22:34:03.524714+0000","last_undegraded":"2026-02-24T22:34:03.524714+0000","last_fullsized":"2026-02-24T22:34:03.524714+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":200,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:59:17.072003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19280971,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":534,"num_read_kb":81366,"num_write":1249,"num_write_kb":20589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":579,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.745270+0000","last_change":"2026-02-24T22:05:20.194969+0000","last_active":"2026-02-24T22:31:54.745270+0000","last_peered":"2026-02-24T22:31:54.745270+0000","last_clean":"2026-02-24T22:31:54.745270+0000","last_became_active":"2026-02-24T22:05:20.194031+0000","last_became_peered":"2026-02-24T22:05:20.194031+0000","last_unstale":"2026-02-24T22:31:54.745270+0000","last_undegraded":"2026-02-24T22:31:54.745270+0000","last_fullsized":"2026-02-24T22:31:54.745270+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:17:15.557806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"312'254","reported_seq":889,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.375668+0000","last_change":"2026-02-24T22:05:22.192071+0000","last_active":"2026-02-24T22:34:18.375668+0000","last_peered":"2026-02-24T22:34:18.375668+0000","last_clean":"2026-02-24T22:34:18.375668+0000","last_became_active":"2026-02-24T22:05:22.191928+0000","last_became_peered":"2026-02-24T22:05:22.191928+0000","last_unstale":"2026-02-24T22:34:18.375668+0000","last_undegraded":"2026-02-24T22:34:18.375668+0000","last_fullsized":"2026-02-24T22:34:18.375668+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":200,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:56:18.209592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54024,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":400,"num_read_kb":789,"num_write":352,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"312'158","reported_seq":747,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.253470+0000","last_change":"2026-02-24T22:05:24.205235+0000","last_active":"2026-02-24T22:34:18.253470+0000","last_peered":"2026-02-24T22:34:18.253470+0000","last_clean":"2026-02-24T22:34:18.253470+0000","last_became_active":"2026-02-24T22:05:24.204990+0000","last_became_peered":"2026-02-24T22:05:24.204990+0000","last_unstale":"2026-02-24T22:34:18.253470+0000","last_undegraded":"2026-02-24T22:34:18.253470+0000","last_fullsized":"2026-02-24T22:34:18.253470+0000","mapping_epoch":30,"log_start":"37'100","ondisk_log_start":"37'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":100,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:14:42.637540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32373,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":92,"num_read_kb":87,"num_write":391,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":564,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:44.730561+0000","last_change":"2026-02-24T22:05:29.662185+0000","last_active":"2026-02-24T22:29:44.730561+0000","last_peered":"2026-02-24T22:29:44.730561+0000","last_clean":"2026-02-24T22:29:44.730561+0000","last_became_active":"2026-02-24T22:05:29.661747+0000","last_became_peered":"2026-02-24T22:05:29.661747+0000","last_unstale":"2026-02-24T22:29:44.730561+0000","last_undegraded":"2026-02-24T22:29:44.730561+0000","last_fullsized":"2026-02-24T22:29:44.730561+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:08:40.631093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"312'73","reported_seq":743,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:07.153602+0000","last_change":"2026-02-24T22:05:18.191685+0000","last_active":"2026-02-24T22:34:07.153602+0000","last_peered":"2026-02-24T22:34:07.153602+0000","last_clean":"2026-02-24T22:34:07.153602+0000","last_became_active":"2026-02-24T22:05:18.191346+0000","last_became_peered":"2026-02-24T22:05:18.191346+0000","last_unstale":"2026-02-24T22:34:07.153602+0000","last_undegraded":"2026-02-24T22:34:07.153602+0000","last_fullsized":"2026-02-24T22:34:07.153602+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:48:20.436580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":237,"num_read_kb":209,"num_write":87,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"312'213","reported_seq":933,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:41.594986+0000","last_change":"2026-02-24T22:05:16.184337+0000","last_active":"2026-02-24T22:33:41.594986+0000","last_peered":"2026-02-24T22:33:41.594986+0000","last_clean":"2026-02-24T22:33:41.594986+0000","last_became_active":"2026-02-24T22:05:16.183990+0000","last_became_peered":"2026-02-24T22:05:16.183990+0000","last_unstale":"2026-02-24T22:33:41.594986+0000","last_undegraded":"2026-02-24T22:33:41.594986+0000","last_fullsized":"2026-02-24T22:33:41.594986+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":200,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:27:53.239432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22095805,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":400,"num_read_kb":23691,"num_write":1212,"num_write_kb":23324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.729065+0000","last_change":"2026-02-24T22:05:20.186544+0000","last_active":"2026-02-24T22:31:04.729065+0000","last_peered":"2026-02-24T22:31:04.729065+0000","last_clean":"2026-02-24T22:31:04.729065+0000","last_became_active":"2026-02-24T22:05:20.186217+0000","last_became_peered":"2026-02-24T22:05:20.186217+0000","last_unstale":"2026-02-24T22:31:04.729065+0000","last_undegraded":"2026-02-24T22:31:04.729065+0000","last_fullsized":"2026-02-24T22:31:04.729065+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:41:55.793283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"312'304","reported_seq":1150,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.527143+0000","last_change":"2026-02-24T22:05:22.197384+0000","last_active":"2026-02-24T22:34:18.527143+0000","last_peered":"2026-02-24T22:34:18.527143+0000","last_clean":"2026-02-24T22:34:18.527143+0000","last_became_active":"2026-02-24T22:05:22.196911+0000","last_became_peered":"2026-02-24T22:05:22.196911+0000","last_unstale":"2026-02-24T22:34:18.527143+0000","last_undegraded":"2026-02-24T22:34:18.527143+0000","last_fullsized":"2026-02-24T22:34:18.527143+0000","mapping_epoch":28,"log_start":"307'200","ondisk_log_start":"307'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":200,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:19:12.370112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1012,"num_read_kb":538,"num_write":51,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"312'187","reported_seq":800,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.220647+0000","last_change":"2026-02-24T22:05:24.207903+0000","last_active":"2026-02-24T22:34:18.220647+0000","last_peered":"2026-02-24T22:34:18.220647+0000","last_clean":"2026-02-24T22:34:18.220647+0000","last_became_active":"2026-02-24T22:05:24.207692+0000","last_became_peered":"2026-02-24T22:05:24.207692+0000","last_unstale":"2026-02-24T22:34:18.220647+0000","last_undegraded":"2026-02-24T22:34:18.220647+0000","last_fullsized":"2026-02-24T22:34:18.220647+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:14:09.532597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29978,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":173,"num_read_kb":171,"num_write":478,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":573,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.734173+0000","last_change":"2026-02-24T22:05:29.654457+0000","last_active":"2026-02-24T22:30:29.734173+0000","last_peered":"2026-02-24T22:30:29.734173+0000","last_clean":"2026-02-24T22:30:29.734173+0000","last_became_active":"2026-02-24T22:05:29.654288+0000","last_became_peered":"2026-02-24T22:05:29.654288+0000","last_unstale":"2026-02-24T22:30:29.734173+0000","last_undegraded":"2026-02-24T22:30:29.734173+0000","last_fullsized":"2026-02-24T22:30:29.734173+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:05:32.007637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"312'274","reported_seq":998,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.743416+0000","last_change":"2026-02-24T22:05:18.184707+0000","last_active":"2026-02-24T22:31:54.743416+0000","last_peered":"2026-02-24T22:31:54.743416+0000","last_clean":"2026-02-24T22:31:54.743416+0000","last_became_active":"2026-02-24T22:05:18.184542+0000","last_became_peered":"2026-02-24T22:05:18.184542+0000","last_unstale":"2026-02-24T22:31:54.743416+0000","last_undegraded":"2026-02-24T22:31:54.743416+0000","last_fullsized":"2026-02-24T22:31:54.743416+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":200,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:55:45.622376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1179,"num_read_kb":976,"num_write":460,"num_write_kb":243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"312'223","reported_seq":937,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.739663+0000","last_change":"2026-02-24T22:05:16.190582+0000","last_active":"2026-02-24T22:30:59.739663+0000","last_peered":"2026-02-24T22:30:59.739663+0000","last_clean":"2026-02-24T22:30:59.739663+0000","last_became_active":"2026-02-24T22:05:16.190372+0000","last_became_peered":"2026-02-24T22:05:16.190372+0000","last_unstale":"2026-02-24T22:30:59.739663+0000","last_undegraded":"2026-02-24T22:30:59.739663+0000","last_fullsized":"2026-02-24T22:30:59.739663+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":200,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:46:33.357028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":364264,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":363,"num_read_kb":3398,"num_write":1108,"num_write_kb":524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.734254+0000","last_change":"2026-02-24T22:05:20.187522+0000","last_active":"2026-02-24T22:31:54.734254+0000","last_peered":"2026-02-24T22:31:54.734254+0000","last_clean":"2026-02-24T22:31:54.734254+0000","last_became_active":"2026-02-24T22:05:20.187327+0000","last_became_peered":"2026-02-24T22:05:20.187327+0000","last_unstale":"2026-02-24T22:31:54.734254+0000","last_undegraded":"2026-02-24T22:31:54.734254+0000","last_fullsized":"2026-02-24T22:31:54.734254+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:09:52.559206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"312'791","reported_seq":2117,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.545233+0000","last_change":"2026-02-24T22:05:22.191261+0000","last_active":"2026-02-24T22:34:18.545233+0000","last_peered":"2026-02-24T22:34:18.545233+0000","last_clean":"2026-02-24T22:34:18.545233+0000","last_became_active":"2026-02-24T22:05:22.191073+0000","last_became_peered":"2026-02-24T22:05:22.191073+0000","last_unstale":"2026-02-24T22:34:18.545233+0000","last_undegraded":"2026-02-24T22:34:18.545233+0000","last_fullsized":"2026-02-24T22:34:18.545233+0000","mapping_epoch":28,"log_start":"307'700","ondisk_log_start":"307'700","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":700,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:26:27.583739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":460,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2922,"num_read_kb":1499,"num_write":81,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"312'204","reported_seq":1716,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.409302+0000","last_change":"2026-02-24T22:05:24.206373+0000","last_active":"2026-02-24T22:34:17.409302+0000","last_peered":"2026-02-24T22:34:17.409302+0000","last_clean":"2026-02-24T22:34:17.409302+0000","last_became_active":"2026-02-24T22:05:24.206134+0000","last_became_peered":"2026-02-24T22:05:24.206134+0000","last_unstale":"2026-02-24T22:34:17.409302+0000","last_undegraded":"2026-02-24T22:34:17.409302+0000","last_fullsized":"2026-02-24T22:34:17.409302+0000","mapping_epoch":30,"log_start":"37'100","ondisk_log_start":"37'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:32:52.956429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33889,"num_objects":107,"num_object_clones":0,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":0,"num_read":1064,"num_read_kb":1063,"num_write":484,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.733657+0000","last_change":"2026-02-24T22:05:29.658922+0000","last_active":"2026-02-24T22:30:29.733657+0000","last_peered":"2026-02-24T22:30:29.733657+0000","last_clean":"2026-02-24T22:30:29.733657+0000","last_became_active":"2026-02-24T22:05:29.658119+0000","last_became_peered":"2026-02-24T22:05:29.658119+0000","last_unstale":"2026-02-24T22:30:29.733657+0000","last_undegraded":"2026-02-24T22:30:29.733657+0000","last_fullsized":"2026-02-24T22:30:29.733657+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:10:58.821048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"312'668","reported_seq":1465,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:09.715537+0000","last_change":"2026-02-24T22:06:00.662810+0000","last_active":"2026-02-24T22:30:09.715537+0000","last_peered":"2026-02-24T22:30:09.715537+0000","last_clean":"2026-02-24T22:30:09.715537+0000","last_became_active":"2026-02-24T22:06:00.662609+0000","last_became_peered":"2026-02-24T22:06:00.662609+0000","last_unstale":"2026-02-24T22:30:09.715537+0000","last_undegraded":"2026-02-24T22:30:09.715537+0000","last_fullsized":"2026-02-24T22:30:09.715537+0000","mapping_epoch":36,"log_start":"312'600","ondisk_log_start":"312'600","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":600,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:42:35.250489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":2886,"num_read_kb":2388,"num_write":1101,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"312'241","reported_seq":967,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:45.705330+0000","last_change":"2026-02-24T22:05:16.190904+0000","last_active":"2026-02-24T22:33:45.705330+0000","last_peered":"2026-02-24T22:33:45.705330+0000","last_clean":"2026-02-24T22:33:45.705330+0000","last_became_active":"2026-02-24T22:05:16.189739+0000","last_became_peered":"2026-02-24T22:05:16.189739+0000","last_unstale":"2026-02-24T22:33:45.705330+0000","last_undegraded":"2026-02-24T22:33:45.705330+0000","last_fullsized":"2026-02-24T22:33:45.705330+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":200,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:04:06.385930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11122410,"num_objects":84,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":391,"num_read_kb":16925,"num_write":1300,"num_write_kb":17766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":579,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.745280+0000","last_change":"2026-02-24T22:05:20.193438+0000","last_active":"2026-02-24T22:31:54.745280+0000","last_peered":"2026-02-24T22:31:54.745280+0000","last_clean":"2026-02-24T22:31:54.745280+0000","last_became_active":"2026-02-24T22:05:20.193200+0000","last_became_peered":"2026-02-24T22:05:20.193200+0000","last_unstale":"2026-02-24T22:31:54.745280+0000","last_undegraded":"2026-02-24T22:31:54.745280+0000","last_fullsized":"2026-02-24T22:31:54.745280+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:58:09.212117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"312'187","reported_seq":785,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.357268+0000","last_change":"2026-02-24T22:05:22.913003+0000","last_active":"2026-02-24T22:34:18.357268+0000","last_peered":"2026-02-24T22:34:18.357268+0000","last_clean":"2026-02-24T22:34:18.357268+0000","last_became_active":"2026-02-24T22:05:22.912826+0000","last_became_peered":"2026-02-24T22:05:22.912826+0000","last_unstale":"2026-02-24T22:34:18.357268+0000","last_undegraded":"2026-02-24T22:34:18.357268+0000","last_fullsized":"2026-02-24T22:34:18.357268+0000","mapping_epoch":28,"log_start":"312'100","ondisk_log_start":"312'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:49:38.039509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47409,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":295,"num_read_kb":667,"num_write":278,"num_write_kb":255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"312'257","reported_seq":903,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:13.928723+0000","last_change":"2026-02-24T22:05:24.199267+0000","last_active":"2026-02-24T22:34:13.928723+0000","last_peered":"2026-02-24T22:34:13.928723+0000","last_clean":"2026-02-24T22:34:13.928723+0000","last_became_active":"2026-02-24T22:05:24.198964+0000","last_became_peered":"2026-02-24T22:05:24.198964+0000","last_unstale":"2026-02-24T22:34:13.928723+0000","last_undegraded":"2026-02-24T22:34:13.928723+0000","last_fullsized":"2026-02-24T22:34:13.928723+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":200,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:47:04.450411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29433,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":428,"num_read_kb":365,"num_write":612,"num_write_kb":187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"312'113","reported_seq":839,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:55.982068+0000","last_change":"2026-02-24T22:05:18.190057+0000","last_active":"2026-02-24T22:33:55.982068+0000","last_peered":"2026-02-24T22:33:55.982068+0000","last_clean":"2026-02-24T22:33:55.982068+0000","last_became_active":"2026-02-24T22:05:18.189796+0000","last_became_peered":"2026-02-24T22:05:18.189796+0000","last_unstale":"2026-02-24T22:33:55.982068+0000","last_undegraded":"2026-02-24T22:33:55.982068+0000","last_fullsized":"2026-02-24T22:33:55.982068+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":100,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:22:22.190649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":455,"num_read_kb":399,"num_write":145,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"312'189","reported_seq":868,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:51.001577+0000","last_change":"2026-02-24T22:05:16.192443+0000","last_active":"2026-02-24T22:30:51.001577+0000","last_peered":"2026-02-24T22:30:51.001577+0000","last_clean":"2026-02-24T22:30:51.001577+0000","last_became_active":"2026-02-24T22:05:16.191985+0000","last_became_peered":"2026-02-24T22:05:16.191985+0000","last_unstale":"2026-02-24T22:30:51.001577+0000","last_undegraded":"2026-02-24T22:30:51.001577+0000","last_fullsized":"2026-02-24T22:30:51.001577+0000","mapping_epoch":22,"log_start":"312'100","ondisk_log_start":"312'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:48:03.816342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4344205,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":288,"num_read_kb":4498,"num_write":974,"num_write_kb":4311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":586,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.742035+0000","last_change":"2026-02-24T22:05:20.194496+0000","last_active":"2026-02-24T22:31:19.742035+0000","last_peered":"2026-02-24T22:31:19.742035+0000","last_clean":"2026-02-24T22:31:19.742035+0000","last_became_active":"2026-02-24T22:05:20.194288+0000","last_became_peered":"2026-02-24T22:05:20.194288+0000","last_unstale":"2026-02-24T22:31:19.742035+0000","last_undegraded":"2026-02-24T22:31:19.742035+0000","last_fullsized":"2026-02-24T22:31:19.742035+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:55:32.351699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"312'255","reported_seq":879,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.223994+0000","last_change":"2026-02-24T22:05:22.913184+0000","last_active":"2026-02-24T22:34:18.223994+0000","last_peered":"2026-02-24T22:34:18.223994+0000","last_clean":"2026-02-24T22:34:18.223994+0000","last_became_active":"2026-02-24T22:05:22.912859+0000","last_became_peered":"2026-02-24T22:05:22.912859+0000","last_unstale":"2026-02-24T22:34:18.223994+0000","last_undegraded":"2026-02-24T22:34:18.223994+0000","last_fullsized":"2026-02-24T22:34:18.223994+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:37:47.377695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53011,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":405,"num_read_kb":789,"num_write":356,"num_write_kb":269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"312'315","reported_seq":1010,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.329842+0000","last_change":"2026-02-24T22:05:24.200453+0000","last_active":"2026-02-24T22:34:15.329842+0000","last_peered":"2026-02-24T22:34:15.329842+0000","last_clean":"2026-02-24T22:34:15.329842+0000","last_became_active":"2026-02-24T22:05:24.199772+0000","last_became_peered":"2026-02-24T22:05:24.199772+0000","last_unstale":"2026-02-24T22:34:15.329842+0000","last_undegraded":"2026-02-24T22:34:15.329842+0000","last_fullsized":"2026-02-24T22:34:15.329842+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:08:33.217544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30266,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":473,"num_read_kb":498,"num_write":701,"num_write_kb":232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":566,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.735506+0000","last_change":"2026-02-24T22:05:29.661914+0000","last_active":"2026-02-24T22:30:29.735506+0000","last_peered":"2026-02-24T22:30:29.735506+0000","last_clean":"2026-02-24T22:30:29.735506+0000","last_became_active":"2026-02-24T22:05:29.661401+0000","last_became_peered":"2026-02-24T22:05:29.661401+0000","last_unstale":"2026-02-24T22:30:29.735506+0000","last_undegraded":"2026-02-24T22:30:29.735506+0000","last_fullsized":"2026-02-24T22:30:29.735506+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:09:29.373749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"312'84","reported_seq":745,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.739343+0000","last_change":"2026-02-24T22:05:18.193728+0000","last_active":"2026-02-24T22:31:19.739343+0000","last_peered":"2026-02-24T22:31:19.739343+0000","last_clean":"2026-02-24T22:31:19.739343+0000","last_became_active":"2026-02-24T22:05:18.193188+0000","last_became_peered":"2026-02-24T22:05:18.193188+0000","last_unstale":"2026-02-24T22:31:19.739343+0000","last_undegraded":"2026-02-24T22:31:19.739343+0000","last_fullsized":"2026-02-24T22:31:19.739343+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:53:00.913559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":312,"num_read_kb":260,"num_write":126,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"312'245","reported_seq":976,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:58.531217+0000","last_change":"2026-02-24T22:05:16.192300+0000","last_active":"2026-02-24T22:33:58.531217+0000","last_peered":"2026-02-24T22:33:58.531217+0000","last_clean":"2026-02-24T22:33:58.531217+0000","last_became_active":"2026-02-24T22:05:16.191956+0000","last_became_peered":"2026-02-24T22:05:16.191956+0000","last_unstale":"2026-02-24T22:33:58.531217+0000","last_undegraded":"2026-02-24T22:33:58.531217+0000","last_fullsized":"2026-02-24T22:33:58.531217+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":200,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:08:01.293337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10490308,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":447,"num_read_kb":10655,"num_write":1344,"num_write_kb":15460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":594,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.722249+0000","last_change":"2026-02-24T22:05:20.186383+0000","last_active":"2026-02-24T22:31:19.722249+0000","last_peered":"2026-02-24T22:31:19.722249+0000","last_clean":"2026-02-24T22:31:19.722249+0000","last_became_active":"2026-02-24T22:05:20.186055+0000","last_became_peered":"2026-02-24T22:05:20.186055+0000","last_unstale":"2026-02-24T22:31:19.722249+0000","last_undegraded":"2026-02-24T22:31:19.722249+0000","last_fullsized":"2026-02-24T22:31:19.722249+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:31:36.594293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"312'224","reported_seq":827,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.246141+0000","last_change":"2026-02-24T22:05:22.194034+0000","last_active":"2026-02-24T22:34:18.246141+0000","last_peered":"2026-02-24T22:34:18.246141+0000","last_clean":"2026-02-24T22:34:18.246141+0000","last_became_active":"2026-02-24T22:05:22.193741+0000","last_became_peered":"2026-02-24T22:05:22.193741+0000","last_unstale":"2026-02-24T22:34:18.246141+0000","last_undegraded":"2026-02-24T22:34:18.246141+0000","last_fullsized":"2026-02-24T22:34:18.246141+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":200,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:35:27.895920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61189,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":372,"num_read_kb":853,"num_write":354,"num_write_kb":326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"312'185","reported_seq":809,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.176596+0000","last_change":"2026-02-24T22:05:24.199051+0000","last_active":"2026-02-24T22:34:15.176596+0000","last_peered":"2026-02-24T22:34:15.176596+0000","last_clean":"2026-02-24T22:34:15.176596+0000","last_became_active":"2026-02-24T22:05:24.198731+0000","last_became_peered":"2026-02-24T22:05:24.198731+0000","last_unstale":"2026-02-24T22:34:15.176596+0000","last_undegraded":"2026-02-24T22:34:15.176596+0000","last_fullsized":"2026-02-24T22:34:15.176596+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:42:29.899927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30034,"num_objects":91,"num_object_clones":0,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":241,"num_read_kb":207,"num_write":470,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":570,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:34.732629+0000","last_change":"2026-02-24T22:05:29.661462+0000","last_active":"2026-02-24T22:30:34.732629+0000","last_peered":"2026-02-24T22:30:34.732629+0000","last_clean":"2026-02-24T22:30:34.732629+0000","last_became_active":"2026-02-24T22:05:29.661161+0000","last_became_peered":"2026-02-24T22:05:29.661161+0000","last_unstale":"2026-02-24T22:30:34.732629+0000","last_undegraded":"2026-02-24T22:30:34.732629+0000","last_fullsized":"2026-02-24T22:30:34.732629+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:23:22.386231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"312'106","reported_seq":752,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.738326+0000","last_change":"2026-02-24T22:05:18.191843+0000","last_active":"2026-02-24T22:31:24.738326+0000","last_peered":"2026-02-24T22:31:24.738326+0000","last_clean":"2026-02-24T22:31:24.738326+0000","last_became_active":"2026-02-24T22:05:18.191555+0000","last_became_peered":"2026-02-24T22:05:18.191555+0000","last_unstale":"2026-02-24T22:31:24.738326+0000","last_undegraded":"2026-02-24T22:31:24.738326+0000","last_fullsized":"2026-02-24T22:31:24.738326+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:53:20.472053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":387,"num_read_kb":325,"num_write":162,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"312'211","reported_seq":924,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:57.361747+0000","last_change":"2026-02-24T22:05:16.192146+0000","last_active":"2026-02-24T22:30:57.361747+0000","last_peered":"2026-02-24T22:30:57.361747+0000","last_clean":"2026-02-24T22:30:57.361747+0000","last_became_active":"2026-02-24T22:05:16.191733+0000","last_became_peered":"2026-02-24T22:05:16.191733+0000","last_unstale":"2026-02-24T22:30:57.361747+0000","last_undegraded":"2026-02-24T22:30:57.361747+0000","last_fullsized":"2026-02-24T22:30:57.361747+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":200,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:27:59.980345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10494886,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":325,"num_read_kb":31022,"num_write":976,"num_write_kb":10319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":593,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.715007+0000","last_change":"2026-02-24T22:05:20.187327+0000","last_active":"2026-02-24T22:30:04.715007+0000","last_peered":"2026-02-24T22:30:04.715007+0000","last_clean":"2026-02-24T22:30:04.715007+0000","last_became_active":"2026-02-24T22:05:20.186892+0000","last_became_peered":"2026-02-24T22:05:20.186892+0000","last_unstale":"2026-02-24T22:30:04.715007+0000","last_undegraded":"2026-02-24T22:30:04.715007+0000","last_fullsized":"2026-02-24T22:30:04.715007+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:53:23.369187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"312'233","reported_seq":841,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.828563+0000","last_change":"2026-02-24T22:05:22.200501+0000","last_active":"2026-02-24T22:34:17.828563+0000","last_peered":"2026-02-24T22:34:17.828563+0000","last_clean":"2026-02-24T22:34:17.828563+0000","last_became_active":"2026-02-24T22:05:22.200208+0000","last_became_peered":"2026-02-24T22:05:22.200208+0000","last_unstale":"2026-02-24T22:34:17.828563+0000","last_undegraded":"2026-02-24T22:34:17.828563+0000","last_fullsized":"2026-02-24T22:34:17.828563+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":200,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:45:47.054214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57434,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":377,"num_read_kb":830,"num_write":350,"num_write_kb":308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"312'275","reported_seq":1084,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.547089+0000","last_change":"2026-02-24T22:05:24.200107+0000","last_active":"2026-02-24T22:34:18.547089+0000","last_peered":"2026-02-24T22:34:18.547089+0000","last_clean":"2026-02-24T22:34:18.547089+0000","last_became_active":"2026-02-24T22:05:24.199525+0000","last_became_peered":"2026-02-24T22:05:24.199525+0000","last_unstale":"2026-02-24T22:34:18.547089+0000","last_undegraded":"2026-02-24T22:34:18.547089+0000","last_fullsized":"2026-02-24T22:34:18.547089+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":200,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:06:50.680774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32009,"num_objects":103,"num_object_clones":0,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":506,"num_read_kb":490,"num_write":606,"num_write_kb":207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:29.740751+0000","last_change":"2026-02-24T22:05:29.652942+0000","last_active":"2026-02-24T22:31:29.740751+0000","last_peered":"2026-02-24T22:31:29.740751+0000","last_clean":"2026-02-24T22:31:29.740751+0000","last_became_active":"2026-02-24T22:05:29.652598+0000","last_became_peered":"2026-02-24T22:05:29.652598+0000","last_unstale":"2026-02-24T22:31:29.740751+0000","last_undegraded":"2026-02-24T22:31:29.740751+0000","last_fullsized":"2026-02-24T22:31:29.740751+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:38:19.564467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"312'169","reported_seq":870,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:07.869080+0000","last_change":"2026-02-24T22:05:18.185305+0000","last_active":"2026-02-24T22:30:07.869080+0000","last_peered":"2026-02-24T22:30:07.869080+0000","last_clean":"2026-02-24T22:30:07.869080+0000","last_became_active":"2026-02-24T22:05:18.185152+0000","last_became_peered":"2026-02-24T22:05:18.185152+0000","last_unstale":"2026-02-24T22:30:07.869080+0000","last_undegraded":"2026-02-24T22:30:07.869080+0000","last_fullsized":"2026-02-24T22:30:07.869080+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:54:16.288957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":651,"num_read_kb":540,"num_write":239,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"312'258","reported_seq":976,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:12.419421+0000","last_change":"2026-02-24T22:05:16.190979+0000","last_active":"2026-02-24T22:34:12.419421+0000","last_peered":"2026-02-24T22:34:12.419421+0000","last_clean":"2026-02-24T22:34:12.419421+0000","last_became_active":"2026-02-24T22:05:16.190205+0000","last_became_peered":"2026-02-24T22:05:16.190205+0000","last_unstale":"2026-02-24T22:34:12.419421+0000","last_undegraded":"2026-02-24T22:34:12.419421+0000","last_fullsized":"2026-02-24T22:34:12.419421+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:07:24.667384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7827201,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":516,"num_read_kb":20088,"num_write":1234,"num_write_kb":14513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":594,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.722800+0000","last_change":"2026-02-24T22:05:20.186884+0000","last_active":"2026-02-24T22:31:24.722800+0000","last_peered":"2026-02-24T22:31:24.722800+0000","last_clean":"2026-02-24T22:31:24.722800+0000","last_became_active":"2026-02-24T22:05:20.186781+0000","last_became_peered":"2026-02-24T22:05:20.186781+0000","last_unstale":"2026-02-24T22:31:24.722800+0000","last_undegraded":"2026-02-24T22:31:24.722800+0000","last_fullsized":"2026-02-24T22:31:24.722800+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:19:52.348059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"312'391","reported_seq":1001,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.367184+0000","last_change":"2026-02-24T22:05:22.192711+0000","last_active":"2026-02-24T22:34:18.367184+0000","last_peered":"2026-02-24T22:34:18.367184+0000","last_clean":"2026-02-24T22:34:18.367184+0000","last_became_active":"2026-02-24T22:05:22.192495+0000","last_became_peered":"2026-02-24T22:05:22.192495+0000","last_unstale":"2026-02-24T22:34:18.367184+0000","last_undegraded":"2026-02-24T22:34:18.367184+0000","last_fullsized":"2026-02-24T22:34:18.367184+0000","mapping_epoch":28,"log_start":"312'300","ondisk_log_start":"312'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":300,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:40:20.670168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118238,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":673,"num_read_kb":1594,"num_write":646,"num_write_kb":622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"312'173","reported_seq":825,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:03.064297+0000","last_change":"2026-02-24T22:05:24.198313+0000","last_active":"2026-02-24T22:34:03.064297+0000","last_peered":"2026-02-24T22:34:03.064297+0000","last_clean":"2026-02-24T22:34:03.064297+0000","last_became_active":"2026-02-24T22:05:24.198114+0000","last_became_peered":"2026-02-24T22:05:24.198114+0000","last_unstale":"2026-02-24T22:34:03.064297+0000","last_undegraded":"2026-02-24T22:34:03.064297+0000","last_fullsized":"2026-02-24T22:34:03.064297+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":100,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:33:14.283058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29501,"num_objects":91,"num_object_clones":0,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":235,"num_read_kb":226,"num_write":429,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:29.726754+0000","last_change":"2026-02-24T22:05:29.652942+0000","last_active":"2026-02-24T22:31:29.726754+0000","last_peered":"2026-02-24T22:31:29.726754+0000","last_clean":"2026-02-24T22:31:29.726754+0000","last_became_active":"2026-02-24T22:05:29.652715+0000","last_became_peered":"2026-02-24T22:05:29.652715+0000","last_unstale":"2026-02-24T22:31:29.726754+0000","last_undegraded":"2026-02-24T22:31:29.726754+0000","last_fullsized":"2026-02-24T22:31:29.726754+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:13:37.213497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"312'571","reported_seq":1430,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:47.451269+0000","last_change":"2026-02-24T22:05:18.193095+0000","last_active":"2026-02-24T22:30:47.451269+0000","last_peered":"2026-02-24T22:30:47.451269+0000","last_clean":"2026-02-24T22:30:47.451269+0000","last_became_active":"2026-02-24T22:05:18.192543+0000","last_became_peered":"2026-02-24T22:05:18.192543+0000","last_unstale":"2026-02-24T22:30:47.451269+0000","last_undegraded":"2026-02-24T22:30:47.451269+0000","last_fullsized":"2026-02-24T22:30:47.451269+0000","mapping_epoch":24,"log_start":"312'500","ondisk_log_start":"312'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":500,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:42:22.975827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":2576,"num_read_kb":2171,"num_write":966,"num_write_kb":601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"312'189","reported_seq":863,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:55.054639+0000","last_change":"2026-02-24T22:05:16.184025+0000","last_active":"2026-02-24T22:33:55.054639+0000","last_peered":"2026-02-24T22:33:55.054639+0000","last_clean":"2026-02-24T22:33:55.054639+0000","last_became_active":"2026-02-24T22:05:16.182928+0000","last_became_peered":"2026-02-24T22:05:16.182928+0000","last_unstale":"2026-02-24T22:33:55.054639+0000","last_undegraded":"2026-02-24T22:33:55.054639+0000","last_fullsized":"2026-02-24T22:33:55.054639+0000","mapping_epoch":22,"log_start":"312'100","ondisk_log_start":"312'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:19:49.977609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":119466,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":254,"num_read_kb":262,"num_write":1021,"num_write_kb":181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":593,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.715020+0000","last_change":"2026-02-24T22:05:20.186597+0000","last_active":"2026-02-24T22:30:04.715020+0000","last_peered":"2026-02-24T22:30:04.715020+0000","last_clean":"2026-02-24T22:30:04.715020+0000","last_became_active":"2026-02-24T22:05:20.186209+0000","last_became_peered":"2026-02-24T22:05:20.186209+0000","last_unstale":"2026-02-24T22:30:04.715020+0000","last_undegraded":"2026-02-24T22:30:04.715020+0000","last_fullsized":"2026-02-24T22:30:04.715020+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:26:06.692781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"312'940","reported_seq":2252,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.520918+0000","last_change":"2026-02-24T22:05:22.200373+0000","last_active":"2026-02-24T22:34:18.520918+0000","last_peered":"2026-02-24T22:34:18.520918+0000","last_clean":"2026-02-24T22:34:18.520918+0000","last_became_active":"2026-02-24T22:05:22.200111+0000","last_became_peered":"2026-02-24T22:05:22.200111+0000","last_unstale":"2026-02-24T22:34:18.520918+0000","last_undegraded":"2026-02-24T22:34:18.520918+0000","last_fullsized":"2026-02-24T22:34:18.520918+0000","mapping_epoch":28,"log_start":"312'900","ondisk_log_start":"312'900","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":900,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:33:19.581577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54380,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":3207,"num_read_kb":2223,"num_write":413,"num_write_kb":370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"312'183","reported_seq":775,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.249482+0000","last_change":"2026-02-24T22:05:24.201417+0000","last_active":"2026-02-24T22:34:17.249482+0000","last_peered":"2026-02-24T22:34:17.249482+0000","last_clean":"2026-02-24T22:34:17.249482+0000","last_became_active":"2026-02-24T22:05:24.200878+0000","last_became_peered":"2026-02-24T22:05:24.200878+0000","last_unstale":"2026-02-24T22:34:17.249482+0000","last_undegraded":"2026-02-24T22:34:17.249482+0000","last_fullsized":"2026-02-24T22:34:17.249482+0000","mapping_epoch":30,"log_start":"37'100","ondisk_log_start":"37'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":100,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:01:48.708129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33021,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":110,"num_read_kb":109,"num_write":447,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"312'435","reported_seq":2196,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.586410+0000","last_change":"2026-02-24T22:05:29.654545+0000","last_active":"2026-02-24T22:34:18.586410+0000","last_peered":"2026-02-24T22:34:18.586410+0000","last_clean":"2026-02-24T22:34:18.586410+0000","last_became_active":"2026-02-24T22:05:29.654064+0000","last_became_peered":"2026-02-24T22:05:29.654064+0000","last_unstale":"2026-02-24T22:34:18.586410+0000","last_undegraded":"2026-02-24T22:34:18.586410+0000","last_fullsized":"2026-02-24T22:34:18.586410+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":400,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:01:24.704003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"312'128","reported_seq":810,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.731309+0000","last_change":"2026-02-24T22:05:18.192490+0000","last_active":"2026-02-24T22:30:04.731309+0000","last_peered":"2026-02-24T22:30:04.731309+0000","last_clean":"2026-02-24T22:30:04.731309+0000","last_became_active":"2026-02-24T22:05:18.191055+0000","last_became_peered":"2026-02-24T22:05:18.191055+0000","last_unstale":"2026-02-24T22:30:04.731309+0000","last_undegraded":"2026-02-24T22:30:04.731309+0000","last_fullsized":"2026-02-24T22:30:04.731309+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:47:55.429098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":486,"num_read_kb":398,"num_write":170,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"312'174","reported_seq":881,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:06.202574+0000","last_change":"2026-02-24T22:05:16.187505+0000","last_active":"2026-02-24T22:34:06.202574+0000","last_peered":"2026-02-24T22:34:06.202574+0000","last_clean":"2026-02-24T22:34:06.202574+0000","last_became_active":"2026-02-24T22:05:16.187261+0000","last_became_peered":"2026-02-24T22:05:16.187261+0000","last_unstale":"2026-02-24T22:34:06.202574+0000","last_undegraded":"2026-02-24T22:34:06.202574+0000","last_fullsized":"2026-02-24T22:34:06.202574+0000","mapping_epoch":22,"log_start":"312'100","ondisk_log_start":"312'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:13:37.546225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10744628,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":306,"num_read_kb":24764,"num_write":893,"num_write_kb":10562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":588,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.718234+0000","last_change":"2026-02-24T22:05:20.187956+0000","last_active":"2026-02-24T22:30:04.718234+0000","last_peered":"2026-02-24T22:30:04.718234+0000","last_clean":"2026-02-24T22:30:04.718234+0000","last_became_active":"2026-02-24T22:05:20.187488+0000","last_became_peered":"2026-02-24T22:05:20.187488+0000","last_unstale":"2026-02-24T22:30:04.718234+0000","last_undegraded":"2026-02-24T22:30:04.718234+0000","last_fullsized":"2026-02-24T22:30:04.718234+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:08:51.284932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"312'462","reported_seq":1312,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.531191+0000","last_change":"2026-02-24T22:05:22.193340+0000","last_active":"2026-02-24T22:34:18.531191+0000","last_peered":"2026-02-24T22:34:18.531191+0000","last_clean":"2026-02-24T22:34:18.531191+0000","last_became_active":"2026-02-24T22:05:22.193142+0000","last_became_peered":"2026-02-24T22:05:22.193142+0000","last_unstale":"2026-02-24T22:34:18.531191+0000","last_undegraded":"2026-02-24T22:34:18.531191+0000","last_fullsized":"2026-02-24T22:34:18.531191+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":400,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:37:15.676612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51399,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1291,"num_read_kb":1227,"num_write":329,"num_write_kb":279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"312'222","reported_seq":821,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.886791+0000","last_change":"2026-02-24T22:05:24.198994+0000","last_active":"2026-02-24T22:34:17.886791+0000","last_peered":"2026-02-24T22:34:17.886791+0000","last_clean":"2026-02-24T22:34:17.886791+0000","last_became_active":"2026-02-24T22:05:24.198669+0000","last_became_peered":"2026-02-24T22:05:24.198669+0000","last_unstale":"2026-02-24T22:34:17.886791+0000","last_undegraded":"2026-02-24T22:34:17.886791+0000","last_fullsized":"2026-02-24T22:34:17.886791+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":200,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:38:36.897067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29328,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":170,"num_read_kb":165,"num_write":551,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.720723+0000","last_change":"2026-02-24T22:05:29.653411+0000","last_active":"2026-02-24T22:30:29.720723+0000","last_peered":"2026-02-24T22:30:29.720723+0000","last_clean":"2026-02-24T22:30:29.720723+0000","last_became_active":"2026-02-24T22:05:29.653144+0000","last_became_peered":"2026-02-24T22:05:29.653144+0000","last_unstale":"2026-02-24T22:30:29.720723+0000","last_undegraded":"2026-02-24T22:30:29.720723+0000","last_fullsized":"2026-02-24T22:30:29.720723+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:09:20.796950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"312'171","reported_seq":875,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:14.602760+0000","last_change":"2026-02-24T22:05:18.191001+0000","last_active":"2026-02-24T22:34:14.602760+0000","last_peered":"2026-02-24T22:34:14.602760+0000","last_clean":"2026-02-24T22:34:14.602760+0000","last_became_active":"2026-02-24T22:05:18.190140+0000","last_became_peered":"2026-02-24T22:05:18.190140+0000","last_unstale":"2026-02-24T22:34:14.602760+0000","last_undegraded":"2026-02-24T22:34:14.602760+0000","last_fullsized":"2026-02-24T22:34:14.602760+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:34:38.707313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":675,"num_read_kb":556,"num_write":238,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"312'231","reported_seq":992,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:59.603149+0000","last_change":"2026-02-24T22:05:16.185212+0000","last_active":"2026-02-24T22:33:59.603149+0000","last_peered":"2026-02-24T22:33:59.603149+0000","last_clean":"2026-02-24T22:33:59.603149+0000","last_became_active":"2026-02-24T22:05:16.184944+0000","last_became_peered":"2026-02-24T22:05:16.184944+0000","last_unstale":"2026-02-24T22:33:59.603149+0000","last_undegraded":"2026-02-24T22:33:59.603149+0000","last_fullsized":"2026-02-24T22:33:59.603149+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":200,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:42:08.775595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17390927,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":470,"num_read_kb":45888,"num_write":1286,"num_write_kb":20430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"27'2","reported_seq":1055,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.487522+0000","last_change":"2026-02-24T22:05:20.194961+0000","last_active":"2026-02-24T22:34:18.487522+0000","last_peered":"2026-02-24T22:34:18.487522+0000","last_clean":"2026-02-24T22:34:18.487522+0000","last_became_active":"2026-02-24T22:05:20.194145+0000","last_became_peered":"2026-02-24T22:05:20.194145+0000","last_unstale":"2026-02-24T22:34:18.487522+0000","last_undegraded":"2026-02-24T22:34:18.487522+0000","last_fullsized":"2026-02-24T22:34:18.487522+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:18:36.330182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":474,"num_read_kb":474,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"312'56","reported_seq":657,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.737323+0000","last_change":"2026-02-24T22:05:22.905477+0000","last_active":"2026-02-24T22:32:24.737323+0000","last_peered":"2026-02-24T22:32:24.737323+0000","last_clean":"2026-02-24T22:32:24.737323+0000","last_became_active":"2026-02-24T22:05:22.905221+0000","last_became_peered":"2026-02-24T22:05:22.905221+0000","last_unstale":"2026-02-24T22:32:24.737323+0000","last_undegraded":"2026-02-24T22:32:24.737323+0000","last_fullsized":"2026-02-24T22:32:24.737323+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:59:47.641153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"312'159","reported_seq":744,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:14.502434+0000","last_change":"2026-02-24T22:05:24.208011+0000","last_active":"2026-02-24T22:34:14.502434+0000","last_peered":"2026-02-24T22:34:14.502434+0000","last_clean":"2026-02-24T22:34:14.502434+0000","last_became_active":"2026-02-24T22:05:24.207673+0000","last_became_peered":"2026-02-24T22:05:24.207673+0000","last_unstale":"2026-02-24T22:34:14.502434+0000","last_undegraded":"2026-02-24T22:34:14.502434+0000","last_fullsized":"2026-02-24T22:34:14.502434+0000","mapping_epoch":30,"log_start":"312'100","ondisk_log_start":"312'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:29:55.011561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25987,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":107,"num_read_kb":103,"num_write":388,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":577,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.733645+0000","last_change":"2026-02-24T22:05:29.659816+0000","last_active":"2026-02-24T22:30:29.733645+0000","last_peered":"2026-02-24T22:30:29.733645+0000","last_clean":"2026-02-24T22:30:29.733645+0000","last_became_active":"2026-02-24T22:05:29.658842+0000","last_became_peered":"2026-02-24T22:05:29.658842+0000","last_unstale":"2026-02-24T22:30:29.733645+0000","last_undegraded":"2026-02-24T22:30:29.733645+0000","last_fullsized":"2026-02-24T22:30:29.733645+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:55:04.860078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"312'98","reported_seq":778,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:54.820404+0000","last_change":"2026-02-24T22:05:18.188396+0000","last_active":"2026-02-24T22:30:54.820404+0000","last_peered":"2026-02-24T22:30:54.820404+0000","last_clean":"2026-02-24T22:30:54.820404+0000","last_became_active":"2026-02-24T22:05:18.188161+0000","last_became_peered":"2026-02-24T22:05:18.188161+0000","last_unstale":"2026-02-24T22:30:54.820404+0000","last_undegraded":"2026-02-24T22:30:54.820404+0000","last_fullsized":"2026-02-24T22:30:54.820404+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:15:32.648520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":368,"num_read_kb":314,"num_write":134,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"312'239","reported_seq":971,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:49.624638+0000","last_change":"2026-02-24T22:05:16.182699+0000","last_active":"2026-02-24T22:33:49.624638+0000","last_peered":"2026-02-24T22:33:49.624638+0000","last_clean":"2026-02-24T22:33:49.624638+0000","last_became_active":"2026-02-24T22:05:16.182194+0000","last_became_peered":"2026-02-24T22:05:16.182194+0000","last_unstale":"2026-02-24T22:33:49.624638+0000","last_undegraded":"2026-02-24T22:33:49.624638+0000","last_fullsized":"2026-02-24T22:33:49.624638+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":200,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:40:05.294459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5600581,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":390,"num_read_kb":7449,"num_write":1196,"num_write_kb":8933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.731323+0000","last_change":"2026-02-24T22:05:20.193494+0000","last_active":"2026-02-24T22:30:04.731323+0000","last_peered":"2026-02-24T22:30:04.731323+0000","last_clean":"2026-02-24T22:30:04.731323+0000","last_became_active":"2026-02-24T22:05:20.193180+0000","last_became_peered":"2026-02-24T22:05:20.193180+0000","last_unstale":"2026-02-24T22:30:04.731323+0000","last_undegraded":"2026-02-24T22:30:04.731323+0000","last_fullsized":"2026-02-24T22:30:04.731323+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:22:52.729272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"312'270","reported_seq":890,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.361543+0000","last_change":"2026-02-24T22:05:22.200410+0000","last_active":"2026-02-24T22:34:18.361543+0000","last_peered":"2026-02-24T22:34:18.361543+0000","last_clean":"2026-02-24T22:34:18.361543+0000","last_became_active":"2026-02-24T22:05:22.200138+0000","last_became_peered":"2026-02-24T22:05:22.200138+0000","last_unstale":"2026-02-24T22:34:18.361543+0000","last_undegraded":"2026-02-24T22:34:18.361543+0000","last_fullsized":"2026-02-24T22:34:18.361543+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:07:05.040956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52341,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":858,"num_read_kb":1265,"num_write":1270,"num_write_kb":1196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"312'327","reported_seq":951,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.407864+0000","last_change":"2026-02-24T22:05:24.207564+0000","last_active":"2026-02-24T22:34:17.407864+0000","last_peered":"2026-02-24T22:34:17.407864+0000","last_clean":"2026-02-24T22:34:17.407864+0000","last_became_active":"2026-02-24T22:05:24.206745+0000","last_became_peered":"2026-02-24T22:05:24.206745+0000","last_unstale":"2026-02-24T22:34:17.407864+0000","last_undegraded":"2026-02-24T22:34:17.407864+0000","last_fullsized":"2026-02-24T22:34:17.407864+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":300,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:53:45.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27098,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":391,"num_read_kb":376,"num_write":726,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"312'431","reported_seq":4795,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.586354+0000","last_change":"2026-02-24T22:05:29.654842+0000","last_active":"2026-02-24T22:34:18.586354+0000","last_peered":"2026-02-24T22:34:18.586354+0000","last_clean":"2026-02-24T22:34:18.586354+0000","last_became_active":"2026-02-24T22:05:29.654281+0000","last_became_peered":"2026-02-24T22:05:29.654281+0000","last_unstale":"2026-02-24T22:34:18.586354+0000","last_undegraded":"2026-02-24T22:34:18.586354+0000","last_fullsized":"2026-02-24T22:34:18.586354+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":400,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:30:38.038033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"312'114","reported_seq":799,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:05.704625+0000","last_change":"2026-02-24T22:05:18.193654+0000","last_active":"2026-02-24T22:34:05.704625+0000","last_peered":"2026-02-24T22:34:05.704625+0000","last_clean":"2026-02-24T22:34:05.704625+0000","last_became_active":"2026-02-24T22:05:18.193426+0000","last_became_peered":"2026-02-24T22:05:18.193426+0000","last_unstale":"2026-02-24T22:34:05.704625+0000","last_undegraded":"2026-02-24T22:34:05.704625+0000","last_fullsized":"2026-02-24T22:34:05.704625+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:36:20.176524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":443,"num_read_kb":376,"num_write":199,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'112","reported_seq":745,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.741431+0000","last_change":"2026-02-24T22:05:06.709275+0000","last_active":"2026-02-24T22:31:14.741431+0000","last_peered":"2026-02-24T22:31:14.741431+0000","last_clean":"2026-02-24T22:31:14.741431+0000","last_became_active":"2026-02-24T22:05:06.708577+0000","last_became_peered":"2026-02-24T22:05:06.708577+0000","last_unstale":"2026-02-24T22:31:14.741431+0000","last_undegraded":"2026-02-24T22:31:14.741431+0000","last_fullsized":"2026-02-24T22:31:14.741431+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:05.664397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:05.664397+0000","last_clean_scrub_stamp":"2026-02-24T22:05:05.664397+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:46:55.314440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":605,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:14.747819+0000","last_change":"2026-02-24T22:05:10.694707+0000","last_active":"2026-02-24T22:32:14.747819+0000","last_peered":"2026-02-24T22:32:14.747819+0000","last_clean":"2026-02-24T22:32:14.747819+0000","last_became_active":"2026-02-24T22:05:08.689983+0000","last_became_peered":"2026-02-24T22:05:08.689983+0000","last_unstale":"2026-02-24T22:32:14.747819+0000","last_undegraded":"2026-02-24T22:32:14.747819+0000","last_fullsized":"2026-02-24T22:32:14.747819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:42:42.134312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046251899999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"312'265","reported_seq":1043,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:12.240184+0000","last_change":"2026-02-24T22:05:16.186666+0000","last_active":"2026-02-24T22:34:12.240184+0000","last_peered":"2026-02-24T22:34:12.240184+0000","last_clean":"2026-02-24T22:34:12.240184+0000","last_became_active":"2026-02-24T22:05:16.185768+0000","last_became_peered":"2026-02-24T22:05:16.185768+0000","last_unstale":"2026-02-24T22:34:12.240184+0000","last_undegraded":"2026-02-24T22:34:12.240184+0000","last_fullsized":"2026-02-24T22:34:12.240184+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":200,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:33:06.518377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6322908,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":624,"num_read_kb":12841,"num_write":1442,"num_write_kb":6232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":586,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.735364+0000","last_change":"2026-02-24T22:05:20.194414+0000","last_active":"2026-02-24T22:30:19.735364+0000","last_peered":"2026-02-24T22:30:19.735364+0000","last_clean":"2026-02-24T22:30:19.735364+0000","last_became_active":"2026-02-24T22:05:20.194040+0000","last_became_peered":"2026-02-24T22:05:20.194040+0000","last_unstale":"2026-02-24T22:30:19.735364+0000","last_undegraded":"2026-02-24T22:30:19.735364+0000","last_fullsized":"2026-02-24T22:30:19.735364+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:14:22.868157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"312'1049","reported_seq":2141,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.521664+0000","last_change":"2026-02-24T22:05:22.199991+0000","last_active":"2026-02-24T22:34:18.521664+0000","last_peered":"2026-02-24T22:34:18.521664+0000","last_clean":"2026-02-24T22:34:18.521664+0000","last_became_active":"2026-02-24T22:05:22.199602+0000","last_became_peered":"2026-02-24T22:05:22.199602+0000","last_unstale":"2026-02-24T22:34:18.521664+0000","last_undegraded":"2026-02-24T22:34:18.521664+0000","last_fullsized":"2026-02-24T22:34:18.521664+0000","mapping_epoch":28,"log_start":"312'1000","ondisk_log_start":"312'1000","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":1000,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:00:35.201833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163236,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2894,"num_read_kb":3248,"num_write":1008,"num_write_kb":944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"312'202","reported_seq":795,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.242020+0000","last_change":"2026-02-24T22:05:24.201041+0000","last_active":"2026-02-24T22:34:18.242020+0000","last_peered":"2026-02-24T22:34:18.242020+0000","last_clean":"2026-02-24T22:34:18.242020+0000","last_became_active":"2026-02-24T22:05:24.200336+0000","last_became_peered":"2026-02-24T22:05:24.200336+0000","last_unstale":"2026-02-24T22:34:18.242020+0000","last_undegraded":"2026-02-24T22:34:18.242020+0000","last_fullsized":"2026-02-24T22:34:18.242020+0000","mapping_epoch":30,"log_start":"37'100","ondisk_log_start":"37'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:52:42.524705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32535,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":152,"num_read_kb":139,"num_write":501,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"312'424","reported_seq":1832,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.571551+0000","last_change":"2026-02-24T22:05:29.660713+0000","last_active":"2026-02-24T22:34:18.571551+0000","last_peered":"2026-02-24T22:34:18.571551+0000","last_clean":"2026-02-24T22:34:18.571551+0000","last_became_active":"2026-02-24T22:05:29.660370+0000","last_became_peered":"2026-02-24T22:05:29.660370+0000","last_unstale":"2026-02-24T22:34:18.571551+0000","last_undegraded":"2026-02-24T22:34:18.571551+0000","last_fullsized":"2026-02-24T22:34:18.571551+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":400,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:47:59.912901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"312'99","reported_seq":797,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.739591+0000","last_change":"2026-02-24T22:05:18.185501+0000","last_active":"2026-02-24T22:31:19.739591+0000","last_peered":"2026-02-24T22:31:19.739591+0000","last_clean":"2026-02-24T22:31:19.739591+0000","last_became_active":"2026-02-24T22:05:18.185205+0000","last_became_peered":"2026-02-24T22:05:18.185205+0000","last_unstale":"2026-02-24T22:31:19.739591+0000","last_undegraded":"2026-02-24T22:31:19.739591+0000","last_fullsized":"2026-02-24T22:31:19.739591+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:21:17.222637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":346,"num_read_kb":300,"num_write":140,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":610,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.741446+0000","last_change":"2026-02-24T22:05:10.808592+0000","last_active":"2026-02-24T22:31:14.741446+0000","last_peered":"2026-02-24T22:31:14.741446+0000","last_clean":"2026-02-24T22:31:14.741446+0000","last_became_active":"2026-02-24T22:05:08.689972+0000","last_became_peered":"2026-02-24T22:05:08.689972+0000","last_unstale":"2026-02-24T22:31:14.741446+0000","last_undegraded":"2026-02-24T22:31:14.741446+0000","last_fullsized":"2026-02-24T22:31:14.741446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:59:56.330904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048345700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"312'208","reported_seq":966,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:57.024826+0000","last_change":"2026-02-24T22:05:16.183783+0000","last_active":"2026-02-24T22:30:57.024826+0000","last_peered":"2026-02-24T22:30:57.024826+0000","last_clean":"2026-02-24T22:30:57.024826+0000","last_became_active":"2026-02-24T22:05:16.183484+0000","last_became_peered":"2026-02-24T22:05:16.183484+0000","last_unstale":"2026-02-24T22:30:57.024826+0000","last_undegraded":"2026-02-24T22:30:57.024826+0000","last_fullsized":"2026-02-24T22:30:57.024826+0000","mapping_epoch":22,"log_start":"312'100","ondisk_log_start":"312'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":100,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:41:07.836883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5664239,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":524,"num_read_kb":821,"num_write":1237,"num_write_kb":10723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":580,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.741456+0000","last_change":"2026-02-24T22:05:20.195460+0000","last_active":"2026-02-24T22:31:19.741456+0000","last_peered":"2026-02-24T22:31:19.741456+0000","last_clean":"2026-02-24T22:31:19.741456+0000","last_became_active":"2026-02-24T22:05:20.195236+0000","last_became_peered":"2026-02-24T22:05:20.195236+0000","last_unstale":"2026-02-24T22:31:19.741456+0000","last_undegraded":"2026-02-24T22:31:19.741456+0000","last_fullsized":"2026-02-24T22:31:19.741456+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:38:04.823028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"312'1045","reported_seq":2251,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.938361+0000","last_change":"2026-02-24T22:05:22.192588+0000","last_active":"2026-02-24T22:34:18.938361+0000","last_peered":"2026-02-24T22:34:18.938361+0000","last_clean":"2026-02-24T22:34:18.938361+0000","last_became_active":"2026-02-24T22:05:22.192295+0000","last_became_peered":"2026-02-24T22:05:22.192295+0000","last_unstale":"2026-02-24T22:34:18.938361+0000","last_undegraded":"2026-02-24T22:34:18.938361+0000","last_fullsized":"2026-02-24T22:34:18.938361+0000","mapping_epoch":28,"log_start":"312'1001","ondisk_log_start":"312'1001","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":1001,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:00:22.125417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":153342,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":3150,"num_read_kb":4374,"num_write":3940,"num_write_kb":3885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"312'182","reported_seq":763,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.067423+0000","last_change":"2026-02-24T22:05:24.205091+0000","last_active":"2026-02-24T22:34:15.067423+0000","last_peered":"2026-02-24T22:34:15.067423+0000","last_clean":"2026-02-24T22:34:15.067423+0000","last_became_active":"2026-02-24T22:05:24.204774+0000","last_became_peered":"2026-02-24T22:05:24.204774+0000","last_unstale":"2026-02-24T22:34:15.067423+0000","last_undegraded":"2026-02-24T22:34:15.067423+0000","last_fullsized":"2026-02-24T22:34:15.067423+0000","mapping_epoch":30,"log_start":"312'100","ondisk_log_start":"312'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":100,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:38:46.991451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29814,"num_objects":89,"num_object_clones":0,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":0,"num_read":112,"num_read_kb":108,"num_write":454,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.727262+0000","last_change":"2026-02-24T22:05:29.653742+0000","last_active":"2026-02-24T22:31:34.727262+0000","last_peered":"2026-02-24T22:31:34.727262+0000","last_clean":"2026-02-24T22:31:34.727262+0000","last_became_active":"2026-02-24T22:05:29.653391+0000","last_became_peered":"2026-02-24T22:05:29.653391+0000","last_unstale":"2026-02-24T22:31:34.727262+0000","last_undegraded":"2026-02-24T22:31:34.727262+0000","last_fullsized":"2026-02-24T22:31:34.727262+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:11:46.647600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"312'101","reported_seq":791,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.725723+0000","last_change":"2026-02-24T22:05:18.186987+0000","last_active":"2026-02-24T22:31:19.725723+0000","last_peered":"2026-02-24T22:31:19.725723+0000","last_clean":"2026-02-24T22:31:19.725723+0000","last_became_active":"2026-02-24T22:05:18.186126+0000","last_became_peered":"2026-02-24T22:05:18.186126+0000","last_unstale":"2026-02-24T22:31:19.725723+0000","last_undegraded":"2026-02-24T22:31:19.725723+0000","last_fullsized":"2026-02-24T22:31:19.725723+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:14:57.108610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":336,"num_read_kb":289,"num_write":128,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":618,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:14.727952+0000","last_change":"2026-02-24T22:05:10.684538+0000","last_active":"2026-02-24T22:32:14.727952+0000","last_peered":"2026-02-24T22:32:14.727952+0000","last_clean":"2026-02-24T22:32:14.727952+0000","last_became_active":"2026-02-24T22:05:08.682035+0000","last_became_peered":"2026-02-24T22:05:08.682035+0000","last_unstale":"2026-02-24T22:32:14.727952+0000","last_undegraded":"2026-02-24T22:32:14.727952+0000","last_fullsized":"2026-02-24T22:32:14.727952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:59:46.243250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047531599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"312'233","reported_seq":977,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.726180+0000","last_change":"2026-02-24T22:05:16.178900+0000","last_active":"2026-02-24T22:31:24.726180+0000","last_peered":"2026-02-24T22:31:24.726180+0000","last_clean":"2026-02-24T22:31:24.726180+0000","last_became_active":"2026-02-24T22:05:16.178331+0000","last_became_peered":"2026-02-24T22:05:16.178331+0000","last_unstale":"2026-02-24T22:31:24.726180+0000","last_undegraded":"2026-02-24T22:31:24.726180+0000","last_fullsized":"2026-02-24T22:31:24.726180+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":200,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:08:05.851722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11570089,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":524,"num_read_kb":6587,"num_write":1274,"num_write_kb":11354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"27'1","reported_seq":595,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.722286+0000","last_change":"2026-02-24T22:05:20.188183+0000","last_active":"2026-02-24T22:31:19.722286+0000","last_peered":"2026-02-24T22:31:19.722286+0000","last_clean":"2026-02-24T22:31:19.722286+0000","last_became_active":"2026-02-24T22:05:20.187908+0000","last_became_peered":"2026-02-24T22:05:20.187908+0000","last_unstale":"2026-02-24T22:31:19.722286+0000","last_undegraded":"2026-02-24T22:31:19.722286+0000","last_fullsized":"2026-02-24T22:31:19.722286+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:49:17.694606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"312'436","reported_seq":1644,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.921712+0000","last_change":"2026-02-24T22:05:22.912798+0000","last_active":"2026-02-24T22:34:18.921712+0000","last_peered":"2026-02-24T22:34:18.921712+0000","last_clean":"2026-02-24T22:34:18.921712+0000","last_became_active":"2026-02-24T22:05:22.912414+0000","last_became_peered":"2026-02-24T22:05:22.912414+0000","last_unstale":"2026-02-24T22:34:18.921712+0000","last_undegraded":"2026-02-24T22:34:18.921712+0000","last_fullsized":"2026-02-24T22:34:18.921712+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":400,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:05:47.634681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":308,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1076,"num_read_kb":597,"num_write":114,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"312'184","reported_seq":770,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.551095+0000","last_change":"2026-02-24T22:05:24.200671+0000","last_active":"2026-02-24T22:34:18.551095+0000","last_peered":"2026-02-24T22:34:18.551095+0000","last_clean":"2026-02-24T22:34:18.551095+0000","last_became_active":"2026-02-24T22:05:24.200029+0000","last_became_peered":"2026-02-24T22:05:24.200029+0000","last_unstale":"2026-02-24T22:34:18.551095+0000","last_undegraded":"2026-02-24T22:34:18.551095+0000","last_fullsized":"2026-02-24T22:34:18.551095+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":100,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:18:37.550289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29918,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":442,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":576,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.724006+0000","last_change":"2026-02-24T22:05:29.653992+0000","last_active":"2026-02-24T22:30:14.724006+0000","last_peered":"2026-02-24T22:30:14.724006+0000","last_clean":"2026-02-24T22:30:14.724006+0000","last_became_active":"2026-02-24T22:05:29.653806+0000","last_became_peered":"2026-02-24T22:05:29.653806+0000","last_unstale":"2026-02-24T22:30:14.724006+0000","last_undegraded":"2026-02-24T22:30:14.724006+0000","last_fullsized":"2026-02-24T22:30:14.724006+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:44:18.672652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"312'230","reported_seq":970,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:51.044848+0000","last_change":"2026-02-24T22:05:18.184825+0000","last_active":"2026-02-24T22:30:51.044848+0000","last_peered":"2026-02-24T22:30:51.044848+0000","last_clean":"2026-02-24T22:30:51.044848+0000","last_became_active":"2026-02-24T22:05:18.183952+0000","last_became_peered":"2026-02-24T22:05:18.183952+0000","last_unstale":"2026-02-24T22:30:51.044848+0000","last_undegraded":"2026-02-24T22:30:51.044848+0000","last_fullsized":"2026-02-24T22:30:51.044848+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:33:27.005363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":963,"num_read_kb":811,"num_write":374,"num_write_kb":210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":606,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:14.747800+0000","last_change":"2026-02-24T22:05:10.695226+0000","last_active":"2026-02-24T22:32:14.747800+0000","last_peered":"2026-02-24T22:32:14.747800+0000","last_clean":"2026-02-24T22:32:14.747800+0000","last_became_active":"2026-02-24T22:05:08.690042+0000","last_became_peered":"2026-02-24T22:05:08.690042+0000","last_unstale":"2026-02-24T22:32:14.747800+0000","last_undegraded":"2026-02-24T22:32:14.747800+0000","last_fullsized":"2026-02-24T22:32:14.747800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:27:03.349026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010128400000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"312'234","reported_seq":948,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:42.673103+0000","last_change":"2026-02-24T22:05:16.189390+0000","last_active":"2026-02-24T22:33:42.673103+0000","last_peered":"2026-02-24T22:33:42.673103+0000","last_clean":"2026-02-24T22:33:42.673103+0000","last_became_active":"2026-02-24T22:05:16.188900+0000","last_became_peered":"2026-02-24T22:05:16.188900+0000","last_unstale":"2026-02-24T22:33:42.673103+0000","last_undegraded":"2026-02-24T22:33:42.673103+0000","last_fullsized":"2026-02-24T22:33:42.673103+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:22:52.382242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11025931,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":322,"num_read_kb":10969,"num_write":1184,"num_write_kb":10859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":589,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.733650+0000","last_change":"2026-02-24T22:05:20.187605+0000","last_active":"2026-02-24T22:30:24.733650+0000","last_peered":"2026-02-24T22:30:24.733650+0000","last_clean":"2026-02-24T22:30:24.733650+0000","last_became_active":"2026-02-24T22:05:20.187251+0000","last_became_peered":"2026-02-24T22:05:20.187251+0000","last_unstale":"2026-02-24T22:30:24.733650+0000","last_undegraded":"2026-02-24T22:30:24.733650+0000","last_fullsized":"2026-02-24T22:30:24.733650+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:47:16.465774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"312'76","reported_seq":697,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.731188+0000","last_change":"2026-02-24T22:05:22.191860+0000","last_active":"2026-02-24T22:31:24.731188+0000","last_peered":"2026-02-24T22:31:24.731188+0000","last_clean":"2026-02-24T22:31:24.731188+0000","last_became_active":"2026-02-24T22:05:22.191394+0000","last_became_peered":"2026-02-24T22:05:22.191394+0000","last_unstale":"2026-02-24T22:31:24.731188+0000","last_undegraded":"2026-02-24T22:31:24.731188+0000","last_fullsized":"2026-02-24T22:31:24.731188+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:12:53.711063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":103,"num_read_kb":96,"num_write":72,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"312'229","reported_seq":843,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.223179+0000","last_change":"2026-02-24T22:05:24.208017+0000","last_active":"2026-02-24T22:34:17.223179+0000","last_peered":"2026-02-24T22:34:17.223179+0000","last_clean":"2026-02-24T22:34:17.223179+0000","last_became_active":"2026-02-24T22:05:24.207698+0000","last_became_peered":"2026-02-24T22:05:24.207698+0000","last_unstale":"2026-02-24T22:34:17.223179+0000","last_undegraded":"2026-02-24T22:34:17.223179+0000","last_fullsized":"2026-02-24T22:34:17.223179+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":200,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:35:36.488781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31273,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":246,"num_read_kb":224,"num_write":553,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":566,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:29.742534+0000","last_change":"2026-02-24T22:05:29.660487+0000","last_active":"2026-02-24T22:31:29.742534+0000","last_peered":"2026-02-24T22:31:29.742534+0000","last_clean":"2026-02-24T22:31:29.742534+0000","last_became_active":"2026-02-24T22:05:29.660065+0000","last_became_peered":"2026-02-24T22:05:29.660065+0000","last_unstale":"2026-02-24T22:31:29.742534+0000","last_undegraded":"2026-02-24T22:31:29.742534+0000","last_fullsized":"2026-02-24T22:31:29.742534+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:00:34.244351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"312'172","reported_seq":869,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:42.925419+0000","last_change":"2026-02-24T22:05:18.185556+0000","last_active":"2026-02-24T22:33:42.925419+0000","last_peered":"2026-02-24T22:33:42.925419+0000","last_clean":"2026-02-24T22:33:42.925419+0000","last_became_active":"2026-02-24T22:05:18.185112+0000","last_became_peered":"2026-02-24T22:05:18.185112+0000","last_unstale":"2026-02-24T22:33:42.925419+0000","last_undegraded":"2026-02-24T22:33:42.925419+0000","last_fullsized":"2026-02-24T22:33:42.925419+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":100,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:39:28.189482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":679,"num_read_kb":553,"num_write":246,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"312'218","reported_seq":939,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:12.225390+0000","last_change":"2026-02-24T22:05:16.191455+0000","last_active":"2026-02-24T22:34:12.225390+0000","last_peered":"2026-02-24T22:34:12.225390+0000","last_clean":"2026-02-24T22:34:12.225390+0000","last_became_active":"2026-02-24T22:05:16.191007+0000","last_became_peered":"2026-02-24T22:05:16.191007+0000","last_unstale":"2026-02-24T22:34:12.225390+0000","last_undegraded":"2026-02-24T22:34:12.225390+0000","last_fullsized":"2026-02-24T22:34:12.225390+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":200,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:46:55.713025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13669125,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":436,"num_read_kb":40330,"num_write":1279,"num_write_kb":18546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":613,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:14.731453+0000","last_change":"2026-02-24T22:05:10.689379+0000","last_active":"2026-02-24T22:32:14.731453+0000","last_peered":"2026-02-24T22:32:14.731453+0000","last_clean":"2026-02-24T22:32:14.731453+0000","last_became_active":"2026-02-24T22:05:08.758282+0000","last_became_peered":"2026-02-24T22:05:08.758282+0000","last_unstale":"2026-02-24T22:32:14.731453+0000","last_undegraded":"2026-02-24T22:32:14.731453+0000","last_fullsized":"2026-02-24T22:32:14.731453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:28:00.429651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033355899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":589,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.733043+0000","last_change":"2026-02-24T22:05:20.187544+0000","last_active":"2026-02-24T22:30:19.733043+0000","last_peered":"2026-02-24T22:30:19.733043+0000","last_clean":"2026-02-24T22:30:19.733043+0000","last_became_active":"2026-02-24T22:05:20.187208+0000","last_became_peered":"2026-02-24T22:05:20.187208+0000","last_unstale":"2026-02-24T22:30:19.733043+0000","last_undegraded":"2026-02-24T22:30:19.733043+0000","last_fullsized":"2026-02-24T22:30:19.733043+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:13:41.845882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"312'1120","reported_seq":2461,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.528903+0000","last_change":"2026-02-24T22:05:22.192049+0000","last_active":"2026-02-24T22:34:18.528903+0000","last_peered":"2026-02-24T22:34:18.528903+0000","last_clean":"2026-02-24T22:34:18.528903+0000","last_became_active":"2026-02-24T22:05:22.191763+0000","last_became_peered":"2026-02-24T22:05:22.191763+0000","last_unstale":"2026-02-24T22:34:18.528903+0000","last_undegraded":"2026-02-24T22:34:18.528903+0000","last_fullsized":"2026-02-24T22:34:18.528903+0000","mapping_epoch":28,"log_start":"312'1100","ondisk_log_start":"312'1100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":1100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:42:04.815445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96815,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":3479,"num_read_kb":2825,"num_write":625,"num_write_kb":530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"312'204","reported_seq":808,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.405693+0000","last_change":"2026-02-24T22:05:24.206550+0000","last_active":"2026-02-24T22:34:18.405693+0000","last_peered":"2026-02-24T22:34:18.405693+0000","last_clean":"2026-02-24T22:34:18.405693+0000","last_became_active":"2026-02-24T22:05:24.205923+0000","last_became_peered":"2026-02-24T22:05:24.205923+0000","last_unstale":"2026-02-24T22:34:18.405693+0000","last_undegraded":"2026-02-24T22:34:18.405693+0000","last_fullsized":"2026-02-24T22:34:18.405693+0000","mapping_epoch":30,"log_start":"37'100","ondisk_log_start":"37'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:42:44.001897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34551,"num_objects":102,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":190,"num_read_kb":174,"num_write":516,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"312'462","reported_seq":4824,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.586084+0000","last_change":"2026-02-24T22:05:29.652714+0000","last_active":"2026-02-24T22:34:18.586084+0000","last_peered":"2026-02-24T22:34:18.586084+0000","last_clean":"2026-02-24T22:34:18.586084+0000","last_became_active":"2026-02-24T22:05:29.652259+0000","last_became_peered":"2026-02-24T22:05:29.652259+0000","last_unstale":"2026-02-24T22:34:18.586084+0000","last_undegraded":"2026-02-24T22:34:18.586084+0000","last_fullsized":"2026-02-24T22:34:18.586084+0000","mapping_epoch":33,"log_start":"312'401","ondisk_log_start":"312'401","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":401,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:52:38.347176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"312'114","reported_seq":810,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.733052+0000","last_change":"2026-02-24T22:05:18.183793+0000","last_active":"2026-02-24T22:30:19.733052+0000","last_peered":"2026-02-24T22:30:19.733052+0000","last_clean":"2026-02-24T22:30:19.733052+0000","last_became_active":"2026-02-24T22:05:18.183369+0000","last_became_peered":"2026-02-24T22:05:18.183369+0000","last_unstale":"2026-02-24T22:30:19.733052+0000","last_undegraded":"2026-02-24T22:30:19.733052+0000","last_fullsized":"2026-02-24T22:30:19.733052+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:10:34.192055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":458,"num_read_kb":386,"num_write":168,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"312'435","reported_seq":1245,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:07.118566+0000","last_change":"2026-02-24T22:05:16.179055+0000","last_active":"2026-02-24T22:34:07.118566+0000","last_peered":"2026-02-24T22:34:07.118566+0000","last_clean":"2026-02-24T22:34:07.118566+0000","last_became_active":"2026-02-24T22:05:16.178353+0000","last_became_peered":"2026-02-24T22:05:16.178353+0000","last_unstale":"2026-02-24T22:34:07.118566+0000","last_undegraded":"2026-02-24T22:34:07.118566+0000","last_fullsized":"2026-02-24T22:34:07.118566+0000","mapping_epoch":22,"log_start":"312'400","ondisk_log_start":"312'400","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":400,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:22:52.210538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":621081,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":974,"num_read_kb":9549,"num_write":1984,"num_write_kb":2098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":610,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:19.749000+0000","last_change":"2026-02-24T22:05:10.808508+0000","last_active":"2026-02-24T22:32:19.749000+0000","last_peered":"2026-02-24T22:32:19.749000+0000","last_clean":"2026-02-24T22:32:19.749000+0000","last_became_active":"2026-02-24T22:05:08.765129+0000","last_became_peered":"2026-02-24T22:05:08.765129+0000","last_unstale":"2026-02-24T22:32:19.749000+0000","last_undegraded":"2026-02-24T22:32:19.749000+0000","last_fullsized":"2026-02-24T22:32:19.749000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:05:15.499818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439276,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":584,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.730929+0000","last_change":"2026-02-24T22:05:20.193089+0000","last_active":"2026-02-24T22:30:19.730929+0000","last_peered":"2026-02-24T22:30:19.730929+0000","last_clean":"2026-02-24T22:30:19.730929+0000","last_became_active":"2026-02-24T22:05:20.192811+0000","last_became_peered":"2026-02-24T22:05:20.192811+0000","last_unstale":"2026-02-24T22:30:19.730929+0000","last_undegraded":"2026-02-24T22:30:19.730929+0000","last_fullsized":"2026-02-24T22:30:19.730929+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:09:16.601460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"312'607","reported_seq":1715,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.543706+0000","last_change":"2026-02-24T22:05:22.905928+0000","last_active":"2026-02-24T22:34:18.543706+0000","last_peered":"2026-02-24T22:34:18.543706+0000","last_clean":"2026-02-24T22:34:18.543706+0000","last_became_active":"2026-02-24T22:05:22.905663+0000","last_became_peered":"2026-02-24T22:05:22.905663+0000","last_unstale":"2026-02-24T22:34:18.543706+0000","last_undegraded":"2026-02-24T22:34:18.543706+0000","last_fullsized":"2026-02-24T22:34:18.543706+0000","mapping_epoch":28,"log_start":"307'500","ondisk_log_start":"307'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":500,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:36:30.224989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":308,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":2056,"num_read_kb":1107,"num_write":154,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"312'171","reported_seq":753,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.533054+0000","last_change":"2026-02-24T22:05:24.206213+0000","last_active":"2026-02-24T22:34:18.533054+0000","last_peered":"2026-02-24T22:34:18.533054+0000","last_clean":"2026-02-24T22:34:18.533054+0000","last_became_active":"2026-02-24T22:05:24.205847+0000","last_became_peered":"2026-02-24T22:05:24.205847+0000","last_unstale":"2026-02-24T22:34:18.533054+0000","last_undegraded":"2026-02-24T22:34:18.533054+0000","last_fullsized":"2026-02-24T22:34:18.533054+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:10:54.760147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26185,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":121,"num_read_kb":112,"num_write":413,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":576,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.724548+0000","last_change":"2026-02-24T22:05:29.651555+0000","last_active":"2026-02-24T22:30:19.724548+0000","last_peered":"2026-02-24T22:30:19.724548+0000","last_clean":"2026-02-24T22:30:19.724548+0000","last_became_active":"2026-02-24T22:05:29.651401+0000","last_became_peered":"2026-02-24T22:05:29.651401+0000","last_unstale":"2026-02-24T22:30:19.724548+0000","last_undegraded":"2026-02-24T22:30:19.724548+0000","last_fullsized":"2026-02-24T22:30:19.724548+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:36:54.239488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"312'150","reported_seq":847,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:40.233137+0000","last_change":"2026-02-24T22:05:18.186337+0000","last_active":"2026-02-24T22:33:40.233137+0000","last_peered":"2026-02-24T22:33:40.233137+0000","last_clean":"2026-02-24T22:33:40.233137+0000","last_became_active":"2026-02-24T22:05:18.185592+0000","last_became_peered":"2026-02-24T22:05:18.185592+0000","last_unstale":"2026-02-24T22:33:40.233137+0000","last_undegraded":"2026-02-24T22:33:40.233137+0000","last_fullsized":"2026-02-24T22:33:40.233137+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:52:57.542872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":536,"num_read_kb":444,"num_write":207,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"312'232","reported_seq":958,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:01.140144+0000","last_change":"2026-02-24T22:05:16.185307+0000","last_active":"2026-02-24T22:34:01.140144+0000","last_peered":"2026-02-24T22:34:01.140144+0000","last_clean":"2026-02-24T22:34:01.140144+0000","last_became_active":"2026-02-24T22:05:16.184131+0000","last_became_peered":"2026-02-24T22:05:16.184131+0000","last_unstale":"2026-02-24T22:34:01.140144+0000","last_undegraded":"2026-02-24T22:34:01.140144+0000","last_fullsized":"2026-02-24T22:34:01.140144+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":200,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:09:05.496588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":173363,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":395,"num_read_kb":1193,"num_write":1232,"num_write_kb":244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":612,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.724713+0000","last_change":"2026-02-24T22:05:10.689444+0000","last_active":"2026-02-24T22:31:09.724713+0000","last_peered":"2026-02-24T22:31:09.724713+0000","last_clean":"2026-02-24T22:31:09.724713+0000","last_became_active":"2026-02-24T22:05:08.682913+0000","last_became_peered":"2026-02-24T22:05:08.682913+0000","last_unstale":"2026-02-24T22:31:09.724713+0000","last_undegraded":"2026-02-24T22:31:09.724713+0000","last_fullsized":"2026-02-24T22:31:09.724713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:20:01.744772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000359878,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":591,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.732657+0000","last_change":"2026-02-24T22:05:20.192401+0000","last_active":"2026-02-24T22:30:19.732657+0000","last_peered":"2026-02-24T22:30:19.732657+0000","last_clean":"2026-02-24T22:30:19.732657+0000","last_became_active":"2026-02-24T22:05:20.192162+0000","last_became_peered":"2026-02-24T22:05:20.192162+0000","last_unstale":"2026-02-24T22:30:19.732657+0000","last_undegraded":"2026-02-24T22:30:19.732657+0000","last_fullsized":"2026-02-24T22:30:19.732657+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:51:58.434704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"312'1027","reported_seq":2586,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.531120+0000","last_change":"2026-02-24T22:05:22.198775+0000","last_active":"2026-02-24T22:34:18.531120+0000","last_peered":"2026-02-24T22:34:18.531120+0000","last_clean":"2026-02-24T22:34:18.531120+0000","last_became_active":"2026-02-24T22:05:22.198252+0000","last_became_peered":"2026-02-24T22:05:22.198252+0000","last_unstale":"2026-02-24T22:34:18.531120+0000","last_undegraded":"2026-02-24T22:34:18.531120+0000","last_fullsized":"2026-02-24T22:34:18.531120+0000","mapping_epoch":28,"log_start":"312'1000","ondisk_log_start":"312'1000","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":1000,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:34:58.507593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":614,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":3859,"num_read_kb":1967,"num_write":82,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"312'1698","reported_seq":4314,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.551488+0000","last_change":"2026-02-24T22:05:24.198591+0000","last_active":"2026-02-24T22:34:18.551488+0000","last_peered":"2026-02-24T22:34:18.551488+0000","last_clean":"2026-02-24T22:34:18.551488+0000","last_became_active":"2026-02-24T22:05:24.198348+0000","last_became_peered":"2026-02-24T22:05:24.198348+0000","last_unstale":"2026-02-24T22:34:18.551488+0000","last_undegraded":"2026-02-24T22:34:18.551488+0000","last_fullsized":"2026-02-24T22:34:18.551488+0000","mapping_epoch":30,"log_start":"310'1600","ondisk_log_start":"310'1600","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":1600,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:04:55.163411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31448,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":5151,"num_read_kb":231050,"num_write":3475,"num_write_kb":1655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":574,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.732473+0000","last_change":"2026-02-24T22:05:29.652643+0000","last_active":"2026-02-24T22:30:14.732473+0000","last_peered":"2026-02-24T22:30:14.732473+0000","last_clean":"2026-02-24T22:30:14.732473+0000","last_became_active":"2026-02-24T22:05:29.652321+0000","last_became_peered":"2026-02-24T22:05:29.652321+0000","last_unstale":"2026-02-24T22:30:14.732473+0000","last_undegraded":"2026-02-24T22:30:14.732473+0000","last_fullsized":"2026-02-24T22:30:14.732473+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:23:33.530605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"312'55","reported_seq":718,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.732644+0000","last_change":"2026-02-24T22:05:18.191716+0000","last_active":"2026-02-24T22:30:19.732644+0000","last_peered":"2026-02-24T22:30:19.732644+0000","last_clean":"2026-02-24T22:30:19.732644+0000","last_became_active":"2026-02-24T22:05:18.191249+0000","last_became_peered":"2026-02-24T22:05:18.191249+0000","last_unstale":"2026-02-24T22:30:19.732644+0000","last_undegraded":"2026-02-24T22:30:19.732644+0000","last_fullsized":"2026-02-24T22:30:19.732644+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:48:35.012678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":226,"num_read_kb":230,"num_write":70,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"312'209","reported_seq":941,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:06.434490+0000","last_change":"2026-02-24T22:05:16.182104+0000","last_active":"2026-02-24T22:34:06.434490+0000","last_peered":"2026-02-24T22:34:06.434490+0000","last_clean":"2026-02-24T22:34:06.434490+0000","last_became_active":"2026-02-24T22:05:16.181840+0000","last_became_peered":"2026-02-24T22:05:16.181840+0000","last_unstale":"2026-02-24T22:34:06.434490+0000","last_undegraded":"2026-02-24T22:34:06.434490+0000","last_fullsized":"2026-02-24T22:34:06.434490+0000","mapping_epoch":22,"log_start":"312'100","ondisk_log_start":"312'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":100,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:50:28.643829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11590618,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":429,"num_read_kb":7386,"num_write":1193,"num_write_kb":13063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":607,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.736747+0000","last_change":"2026-02-24T22:05:10.808402+0000","last_active":"2026-02-24T22:31:09.736747+0000","last_peered":"2026-02-24T22:31:09.736747+0000","last_clean":"2026-02-24T22:31:09.736747+0000","last_became_active":"2026-02-24T22:05:08.689542+0000","last_became_peered":"2026-02-24T22:05:08.689542+0000","last_unstale":"2026-02-24T22:31:09.736747+0000","last_undegraded":"2026-02-24T22:31:09.736747+0000","last_fullsized":"2026-02-24T22:31:09.736747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:23:22.404541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038816099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":592,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.731136+0000","last_change":"2026-02-24T22:05:20.186366+0000","last_active":"2026-02-24T22:31:24.731136+0000","last_peered":"2026-02-24T22:31:24.731136+0000","last_clean":"2026-02-24T22:31:24.731136+0000","last_became_active":"2026-02-24T22:05:20.186108+0000","last_became_peered":"2026-02-24T22:05:20.186108+0000","last_unstale":"2026-02-24T22:31:24.731136+0000","last_undegraded":"2026-02-24T22:31:24.731136+0000","last_fullsized":"2026-02-24T22:31:24.731136+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:02:47.567361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"312'592","reported_seq":1229,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.355778+0000","last_change":"2026-02-24T22:05:22.911959+0000","last_active":"2026-02-24T22:34:18.355778+0000","last_peered":"2026-02-24T22:34:18.355778+0000","last_clean":"2026-02-24T22:34:18.355778+0000","last_became_active":"2026-02-24T22:05:22.911692+0000","last_became_peered":"2026-02-24T22:05:22.911692+0000","last_unstale":"2026-02-24T22:34:18.355778+0000","last_undegraded":"2026-02-24T22:34:18.355778+0000","last_fullsized":"2026-02-24T22:34:18.355778+0000","mapping_epoch":28,"log_start":"312'500","ondisk_log_start":"312'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":500,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:53:02.224623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":161291,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1010,"num_read_kb":2303,"num_write":954,"num_write_kb":876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"312'175","reported_seq":763,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:07.165906+0000","last_change":"2026-02-24T22:05:24.199612+0000","last_active":"2026-02-24T22:34:07.165906+0000","last_peered":"2026-02-24T22:34:07.165906+0000","last_clean":"2026-02-24T22:34:07.165906+0000","last_became_active":"2026-02-24T22:05:24.199398+0000","last_became_peered":"2026-02-24T22:05:24.199398+0000","last_unstale":"2026-02-24T22:34:07.165906+0000","last_undegraded":"2026-02-24T22:34:07.165906+0000","last_fullsized":"2026-02-24T22:34:07.165906+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":100,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:03:11.629672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30139,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":95,"num_read_kb":93,"num_write":436,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"312'428","reported_seq":3586,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.571386+0000","last_change":"2026-02-24T22:05:29.662212+0000","last_active":"2026-02-24T22:34:18.571386+0000","last_peered":"2026-02-24T22:34:18.571386+0000","last_clean":"2026-02-24T22:34:18.571386+0000","last_became_active":"2026-02-24T22:05:29.661694+0000","last_became_peered":"2026-02-24T22:05:29.661694+0000","last_unstale":"2026-02-24T22:34:18.571386+0000","last_undegraded":"2026-02-24T22:34:18.571386+0000","last_fullsized":"2026-02-24T22:34:18.571386+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":400,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:28:47.623604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"312'456","reported_seq":2590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.586383+0000","last_change":"2026-02-24T22:05:29.654721+0000","last_active":"2026-02-24T22:34:18.586383+0000","last_peered":"2026-02-24T22:34:18.586383+0000","last_clean":"2026-02-24T22:34:18.586383+0000","last_became_active":"2026-02-24T22:05:29.654058+0000","last_became_peered":"2026-02-24T22:05:29.654058+0000","last_unstale":"2026-02-24T22:34:18.586383+0000","last_undegraded":"2026-02-24T22:34:18.586383+0000","last_fullsized":"2026-02-24T22:34:18.586383+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":400,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:19:34.289230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"312'4194","reported_seq":4926,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:45.073603+0000","last_change":"2026-02-24T22:05:18.191351+0000","last_active":"2026-02-24T22:33:45.073603+0000","last_peered":"2026-02-24T22:33:45.073603+0000","last_clean":"2026-02-24T22:33:45.073603+0000","last_became_active":"2026-02-24T22:05:18.190114+0000","last_became_peered":"2026-02-24T22:05:18.190114+0000","last_unstale":"2026-02-24T22:33:45.073603+0000","last_undegraded":"2026-02-24T22:33:45.073603+0000","last_fullsized":"2026-02-24T22:33:45.073603+0000","mapping_epoch":24,"log_start":"312'4100","ondisk_log_start":"312'4100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":3000,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:58:28.766462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":16758,"num_read_kb":13963,"num_write":6320,"num_write_kb":3135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"312'194","reported_seq":915,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:57.691183+0000","last_change":"2026-02-24T22:05:16.189046+0000","last_active":"2026-02-24T22:33:57.691183+0000","last_peered":"2026-02-24T22:33:57.691183+0000","last_clean":"2026-02-24T22:33:57.691183+0000","last_became_active":"2026-02-24T22:05:16.188630+0000","last_became_peered":"2026-02-24T22:05:16.188630+0000","last_unstale":"2026-02-24T22:33:57.691183+0000","last_undegraded":"2026-02-24T22:33:57.691183+0000","last_fullsized":"2026-02-24T22:33:57.691183+0000","mapping_epoch":22,"log_start":"312'100","ondisk_log_start":"312'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":100,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:15:02.436674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11535007,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":414,"num_read_kb":33751,"num_write":1015,"num_write_kb":13009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":588,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.723682+0000","last_change":"2026-02-24T22:05:20.188706+0000","last_active":"2026-02-24T22:30:59.723682+0000","last_peered":"2026-02-24T22:30:59.723682+0000","last_clean":"2026-02-24T22:30:59.723682+0000","last_became_active":"2026-02-24T22:05:20.188330+0000","last_became_peered":"2026-02-24T22:05:20.188330+0000","last_unstale":"2026-02-24T22:30:59.723682+0000","last_undegraded":"2026-02-24T22:30:59.723682+0000","last_fullsized":"2026-02-24T22:30:59.723682+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:07:09.639326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"312'681","reported_seq":1757,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.538908+0000","last_change":"2026-02-24T22:05:22.192098+0000","last_active":"2026-02-24T22:34:18.538908+0000","last_peered":"2026-02-24T22:34:18.538908+0000","last_clean":"2026-02-24T22:34:18.538908+0000","last_became_active":"2026-02-24T22:05:22.191584+0000","last_became_peered":"2026-02-24T22:05:22.191584+0000","last_unstale":"2026-02-24T22:34:18.538908+0000","last_undegraded":"2026-02-24T22:34:18.538908+0000","last_fullsized":"2026-02-24T22:34:18.538908+0000","mapping_epoch":28,"log_start":"312'600","ondisk_log_start":"312'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":600,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:55:24.433104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50841,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2209,"num_read_kb":1670,"num_write":316,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"312'161","reported_seq":768,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:55.998434+0000","last_change":"2026-02-24T22:05:24.199098+0000","last_active":"2026-02-24T22:33:55.998434+0000","last_peered":"2026-02-24T22:33:55.998434+0000","last_clean":"2026-02-24T22:33:55.998434+0000","last_became_active":"2026-02-24T22:05:24.198821+0000","last_became_peered":"2026-02-24T22:05:24.198821+0000","last_unstale":"2026-02-24T22:33:55.998434+0000","last_undegraded":"2026-02-24T22:33:55.998434+0000","last_fullsized":"2026-02-24T22:33:55.998434+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":100,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:31:53.831339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28978,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":162,"num_read_kb":145,"num_write":411,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":578,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.732230+0000","last_change":"2026-02-24T22:05:29.653213+0000","last_active":"2026-02-24T22:31:34.732230+0000","last_peered":"2026-02-24T22:31:34.732230+0000","last_clean":"2026-02-24T22:31:34.732230+0000","last_became_active":"2026-02-24T22:05:29.652757+0000","last_became_peered":"2026-02-24T22:05:29.652757+0000","last_unstale":"2026-02-24T22:31:34.732230+0000","last_undegraded":"2026-02-24T22:31:34.732230+0000","last_fullsized":"2026-02-24T22:31:34.732230+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:57:03.928835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"312'4231","reported_seq":4978,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:52.784970+0000","last_change":"2026-02-24T22:05:18.184303+0000","last_active":"2026-02-24T22:33:52.784970+0000","last_peered":"2026-02-24T22:33:52.784970+0000","last_clean":"2026-02-24T22:33:52.784970+0000","last_became_active":"2026-02-24T22:05:18.183983+0000","last_became_peered":"2026-02-24T22:05:18.183983+0000","last_unstale":"2026-02-24T22:33:52.784970+0000","last_undegraded":"2026-02-24T22:33:52.784970+0000","last_fullsized":"2026-02-24T22:33:52.784970+0000","mapping_epoch":24,"log_start":"312'4200","ondisk_log_start":"312'4200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":3000,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:36:58.437134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":16912,"num_read_kb":14102,"num_write":6331,"num_write_kb":3151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"312'232","reported_seq":974,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:04.712850+0000","last_change":"2026-02-24T22:05:16.193213+0000","last_active":"2026-02-24T22:34:04.712850+0000","last_peered":"2026-02-24T22:34:04.712850+0000","last_clean":"2026-02-24T22:34:04.712850+0000","last_became_active":"2026-02-24T22:05:16.192845+0000","last_became_peered":"2026-02-24T22:05:16.192845+0000","last_unstale":"2026-02-24T22:34:04.712850+0000","last_undegraded":"2026-02-24T22:34:04.712850+0000","last_fullsized":"2026-02-24T22:34:04.712850+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":200,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:53:53.655783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933180,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":413,"num_read_kb":4336,"num_write":1200,"num_write_kb":2676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.728520+0000","last_change":"2026-02-24T22:05:20.189035+0000","last_active":"2026-02-24T22:30:59.728520+0000","last_peered":"2026-02-24T22:30:59.728520+0000","last_clean":"2026-02-24T22:30:59.728520+0000","last_became_active":"2026-02-24T22:05:20.188790+0000","last_became_peered":"2026-02-24T22:05:20.188790+0000","last_unstale":"2026-02-24T22:30:59.728520+0000","last_undegraded":"2026-02-24T22:30:59.728520+0000","last_fullsized":"2026-02-24T22:30:59.728520+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:29:04.813791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"312'25","reported_seq":615,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.737881+0000","last_change":"2026-02-24T22:05:22.192740+0000","last_active":"2026-02-24T22:31:04.737881+0000","last_peered":"2026-02-24T22:31:04.737881+0000","last_clean":"2026-02-24T22:31:04.737881+0000","last_became_active":"2026-02-24T22:05:22.192560+0000","last_became_peered":"2026-02-24T22:05:22.192560+0000","last_unstale":"2026-02-24T22:31:04.737881+0000","last_undegraded":"2026-02-24T22:31:04.737881+0000","last_fullsized":"2026-02-24T22:31:04.737881+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:40:42.414311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"312'163","reported_seq":746,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.531878+0000","last_change":"2026-02-24T22:05:24.205761+0000","last_active":"2026-02-24T22:34:18.531878+0000","last_peered":"2026-02-24T22:34:18.531878+0000","last_clean":"2026-02-24T22:34:18.531878+0000","last_became_active":"2026-02-24T22:05:24.205506+0000","last_became_peered":"2026-02-24T22:05:24.205506+0000","last_unstale":"2026-02-24T22:34:18.531878+0000","last_undegraded":"2026-02-24T22:34:18.531878+0000","last_fullsized":"2026-02-24T22:34:18.531878+0000","mapping_epoch":30,"log_start":"301'100","ondisk_log_start":"301'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":100,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:22:32.827602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30843,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":90,"num_read_kb":84,"num_write":389,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":573,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.734200+0000","last_change":"2026-02-24T22:05:29.653324+0000","last_active":"2026-02-24T22:30:29.734200+0000","last_peered":"2026-02-24T22:30:29.734200+0000","last_clean":"2026-02-24T22:30:29.734200+0000","last_became_active":"2026-02-24T22:05:29.652987+0000","last_became_peered":"2026-02-24T22:05:29.652987+0000","last_unstale":"2026-02-24T22:30:29.734200+0000","last_undegraded":"2026-02-24T22:30:29.734200+0000","last_fullsized":"2026-02-24T22:30:29.734200+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:58:50.918939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"312'187","reported_seq":884,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.550400+0000","last_change":"2026-02-24T22:05:18.186488+0000","last_active":"2026-02-24T22:34:18.550400+0000","last_peered":"2026-02-24T22:34:18.550400+0000","last_clean":"2026-02-24T22:34:18.550400+0000","last_became_active":"2026-02-24T22:05:18.186069+0000","last_became_peered":"2026-02-24T22:05:18.186069+0000","last_unstale":"2026-02-24T22:34:18.550400+0000","last_undegraded":"2026-02-24T22:34:18.550400+0000","last_fullsized":"2026-02-24T22:34:18.550400+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:26:51.294901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":691,"num_read_kb":561,"num_write":267,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"312'221","reported_seq":903,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:58.398587+0000","last_change":"2026-02-24T22:06:00.662484+0000","last_active":"2026-02-24T22:30:58.398587+0000","last_peered":"2026-02-24T22:30:58.398587+0000","last_clean":"2026-02-24T22:30:58.398587+0000","last_became_active":"2026-02-24T22:06:00.662291+0000","last_became_peered":"2026-02-24T22:06:00.662291+0000","last_unstale":"2026-02-24T22:30:58.398587+0000","last_undegraded":"2026-02-24T22:30:58.398587+0000","last_fullsized":"2026-02-24T22:30:58.398587+0000","mapping_epoch":36,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:44:04.240829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5088,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":359,"num_read_kb":357,"num_write":1312,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":585,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.739650+0000","last_change":"2026-02-24T22:05:20.194668+0000","last_active":"2026-02-24T22:30:59.739650+0000","last_peered":"2026-02-24T22:30:59.739650+0000","last_clean":"2026-02-24T22:30:59.739650+0000","last_became_active":"2026-02-24T22:05:20.194215+0000","last_became_peered":"2026-02-24T22:05:20.194215+0000","last_unstale":"2026-02-24T22:30:59.739650+0000","last_undegraded":"2026-02-24T22:30:59.739650+0000","last_fullsized":"2026-02-24T22:30:59.739650+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:55:09.941150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"312'470","reported_seq":1327,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.519070+0000","last_change":"2026-02-24T22:05:22.192102+0000","last_active":"2026-02-24T22:34:18.519070+0000","last_peered":"2026-02-24T22:34:18.519070+0000","last_clean":"2026-02-24T22:34:18.519070+0000","last_became_active":"2026-02-24T22:05:22.191675+0000","last_became_peered":"2026-02-24T22:05:22.191675+0000","last_unstale":"2026-02-24T22:34:18.519070+0000","last_undegraded":"2026-02-24T22:34:18.519070+0000","last_fullsized":"2026-02-24T22:34:18.519070+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":400,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:50:42.591507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53362,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1312,"num_read_kb":1248,"num_write":339,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"312'249","reported_seq":1278,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.564141+0000","last_change":"2026-02-24T22:05:24.207967+0000","last_active":"2026-02-24T22:34:17.564141+0000","last_peered":"2026-02-24T22:34:17.564141+0000","last_clean":"2026-02-24T22:34:17.564141+0000","last_became_active":"2026-02-24T22:05:24.207479+0000","last_became_peered":"2026-02-24T22:05:24.207479+0000","last_unstale":"2026-02-24T22:34:17.564141+0000","last_undegraded":"2026-02-24T22:34:17.564141+0000","last_fullsized":"2026-02-24T22:34:17.564141+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":200,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:21:33.217485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31962,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":675,"num_read_kb":669,"num_write":683,"num_write_kb":207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":564,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.735513+0000","last_change":"2026-02-24T22:05:29.661767+0000","last_active":"2026-02-24T22:30:29.735513+0000","last_peered":"2026-02-24T22:30:29.735513+0000","last_clean":"2026-02-24T22:30:29.735513+0000","last_became_active":"2026-02-24T22:05:29.661248+0000","last_became_peered":"2026-02-24T22:05:29.661248+0000","last_unstale":"2026-02-24T22:30:29.735513+0000","last_undegraded":"2026-02-24T22:30:29.735513+0000","last_fullsized":"2026-02-24T22:30:29.735513+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:30:06.329202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"312'116","reported_seq":824,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:06.481608+0000","last_change":"2026-02-24T22:05:18.187722+0000","last_active":"2026-02-24T22:34:06.481608+0000","last_peered":"2026-02-24T22:34:06.481608+0000","last_clean":"2026-02-24T22:34:06.481608+0000","last_became_active":"2026-02-24T22:05:18.184982+0000","last_became_peered":"2026-02-24T22:05:18.184982+0000","last_unstale":"2026-02-24T22:34:06.481608+0000","last_undegraded":"2026-02-24T22:34:06.481608+0000","last_fullsized":"2026-02-24T22:34:06.481608+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":100,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:08:28.087403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":444,"num_read_kb":381,"num_write":158,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"312'213","reported_seq":975,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:02.205401+0000","last_change":"2026-02-24T22:05:16.184926+0000","last_active":"2026-02-24T22:34:02.205401+0000","last_peered":"2026-02-24T22:34:02.205401+0000","last_clean":"2026-02-24T22:34:02.205401+0000","last_became_active":"2026-02-24T22:05:16.184594+0000","last_became_peered":"2026-02-24T22:05:16.184594+0000","last_unstale":"2026-02-24T22:34:02.205401+0000","last_undegraded":"2026-02-24T22:34:02.205401+0000","last_fullsized":"2026-02-24T22:34:02.205401+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":200,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:53:18.748033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6374454,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":456,"num_read_kb":28116,"num_write":1131,"num_write_kb":6289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":583,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.735449+0000","last_change":"2026-02-24T22:05:20.193777+0000","last_active":"2026-02-24T22:30:59.735449+0000","last_peered":"2026-02-24T22:30:59.735449+0000","last_clean":"2026-02-24T22:30:59.735449+0000","last_became_active":"2026-02-24T22:05:20.193522+0000","last_became_peered":"2026-02-24T22:05:20.193522+0000","last_unstale":"2026-02-24T22:30:59.735449+0000","last_undegraded":"2026-02-24T22:30:59.735449+0000","last_fullsized":"2026-02-24T22:30:59.735449+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:18:03.275177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"312'428","reported_seq":1261,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.513000+0000","last_change":"2026-02-24T22:05:22.200284+0000","last_active":"2026-02-24T22:34:18.513000+0000","last_peered":"2026-02-24T22:34:18.513000+0000","last_clean":"2026-02-24T22:34:18.513000+0000","last_became_active":"2026-02-24T22:05:22.199968+0000","last_became_peered":"2026-02-24T22:05:22.199968+0000","last_unstale":"2026-02-24T22:34:18.513000+0000","last_undegraded":"2026-02-24T22:34:18.513000+0000","last_fullsized":"2026-02-24T22:34:18.513000+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":400,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:25:54.291884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51783,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1258,"num_read_kb":1192,"num_write":305,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"312'212","reported_seq":808,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:04.178415+0000","last_change":"2026-02-24T22:05:24.200305+0000","last_active":"2026-02-24T22:34:04.178415+0000","last_peered":"2026-02-24T22:34:04.178415+0000","last_clean":"2026-02-24T22:34:04.178415+0000","last_became_active":"2026-02-24T22:05:24.199650+0000","last_became_peered":"2026-02-24T22:05:24.199650+0000","last_unstale":"2026-02-24T22:34:04.178415+0000","last_undegraded":"2026-02-24T22:34:04.178415+0000","last_fullsized":"2026-02-24T22:34:04.178415+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:33:18.336086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29523,"num_objects":91,"num_object_clones":0,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":182,"num_read_kb":193,"num_write":488,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"312'249","reported_seq":958,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:44.105284+0000","last_change":"2026-02-24T22:05:18.190553+0000","last_active":"2026-02-24T22:33:44.105284+0000","last_peered":"2026-02-24T22:33:44.105284+0000","last_clean":"2026-02-24T22:33:44.105284+0000","last_became_active":"2026-02-24T22:05:18.190238+0000","last_became_peered":"2026-02-24T22:05:18.190238+0000","last_unstale":"2026-02-24T22:33:44.105284+0000","last_undegraded":"2026-02-24T22:33:44.105284+0000","last_fullsized":"2026-02-24T22:33:44.105284+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":200,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:04:44.616609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":976,"num_read_kb":794,"num_write":353,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"312'218","reported_seq":1574,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:05.712936+0000","last_change":"2026-02-24T22:05:16.185696+0000","last_active":"2026-02-24T22:34:05.712936+0000","last_peered":"2026-02-24T22:34:05.712936+0000","last_clean":"2026-02-24T22:34:05.712936+0000","last_became_active":"2026-02-24T22:05:16.184415+0000","last_became_peered":"2026-02-24T22:05:16.184415+0000","last_unstale":"2026-02-24T22:34:05.712936+0000","last_undegraded":"2026-02-24T22:34:05.712936+0000","last_fullsized":"2026-02-24T22:34:05.712936+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":200,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:10:18.950972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5603567,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":2274,"num_read_kb":23832,"num_write":1037,"num_write_kb":5557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.740195+0000","last_change":"2026-02-24T22:05:20.186817+0000","last_active":"2026-02-24T22:31:24.740195+0000","last_peered":"2026-02-24T22:31:24.740195+0000","last_clean":"2026-02-24T22:31:24.740195+0000","last_became_active":"2026-02-24T22:05:20.186325+0000","last_became_peered":"2026-02-24T22:05:20.186325+0000","last_unstale":"2026-02-24T22:31:24.740195+0000","last_undegraded":"2026-02-24T22:31:24.740195+0000","last_fullsized":"2026-02-24T22:31:24.740195+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:45:07.507222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"312'399","reported_seq":1007,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.383952+0000","last_change":"2026-02-24T22:05:22.192972+0000","last_active":"2026-02-24T22:34:18.383952+0000","last_peered":"2026-02-24T22:34:18.383952+0000","last_clean":"2026-02-24T22:34:18.383952+0000","last_became_active":"2026-02-24T22:05:22.192756+0000","last_became_peered":"2026-02-24T22:05:22.192756+0000","last_unstale":"2026-02-24T22:34:18.383952+0000","last_undegraded":"2026-02-24T22:34:18.383952+0000","last_fullsized":"2026-02-24T22:34:18.383952+0000","mapping_epoch":28,"log_start":"312'300","ondisk_log_start":"312'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":300,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:43:14.513261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115669,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":689,"num_read_kb":1585,"num_write":692,"num_write_kb":688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"312'187","reported_seq":796,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.233131+0000","last_change":"2026-02-24T22:05:24.206653+0000","last_active":"2026-02-24T22:34:17.233131+0000","last_peered":"2026-02-24T22:34:17.233131+0000","last_clean":"2026-02-24T22:34:17.233131+0000","last_became_active":"2026-02-24T22:05:24.206407+0000","last_became_peered":"2026-02-24T22:05:24.206407+0000","last_unstale":"2026-02-24T22:34:17.233131+0000","last_undegraded":"2026-02-24T22:34:17.233131+0000","last_fullsized":"2026-02-24T22:34:17.233131+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:49:11.388315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30298,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":168,"num_read_kb":155,"num_write":455,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":576,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.729573+0000","last_change":"2026-02-24T22:05:29.650326+0000","last_active":"2026-02-24T22:31:09.729573+0000","last_peered":"2026-02-24T22:31:09.729573+0000","last_clean":"2026-02-24T22:31:09.729573+0000","last_became_active":"2026-02-24T22:05:29.650115+0000","last_became_peered":"2026-02-24T22:05:29.650115+0000","last_unstale":"2026-02-24T22:31:09.729573+0000","last_undegraded":"2026-02-24T22:31:09.729573+0000","last_fullsized":"2026-02-24T22:31:09.729573+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:38:20.581425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"312'162","reported_seq":873,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.740186+0000","last_change":"2026-02-24T22:05:18.183744+0000","last_active":"2026-02-24T22:31:24.740186+0000","last_peered":"2026-02-24T22:31:24.740186+0000","last_clean":"2026-02-24T22:31:24.740186+0000","last_became_active":"2026-02-24T22:05:18.183501+0000","last_became_peered":"2026-02-24T22:05:18.183501+0000","last_unstale":"2026-02-24T22:31:24.740186+0000","last_undegraded":"2026-02-24T22:31:24.740186+0000","last_fullsized":"2026-02-24T22:31:24.740186+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":100,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:10:06.918155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":578,"num_read_kb":488,"num_write":224,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"312'343","reported_seq":1285,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:38.575992+0000","last_change":"2026-02-24T22:05:16.183028+0000","last_active":"2026-02-24T22:33:38.575992+0000","last_peered":"2026-02-24T22:33:38.575992+0000","last_clean":"2026-02-24T22:33:38.575992+0000","last_became_active":"2026-02-24T22:05:16.182909+0000","last_became_peered":"2026-02-24T22:05:16.182909+0000","last_unstale":"2026-02-24T22:33:38.575992+0000","last_undegraded":"2026-02-24T22:33:38.575992+0000","last_fullsized":"2026-02-24T22:33:38.575992+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:01:15.081477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":445956,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":1143,"num_read_kb":9242,"num_write":1547,"num_write_kb":11784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":584,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.730971+0000","last_change":"2026-02-24T22:05:20.194023+0000","last_active":"2026-02-24T22:30:19.730971+0000","last_peered":"2026-02-24T22:30:19.730971+0000","last_clean":"2026-02-24T22:30:19.730971+0000","last_became_active":"2026-02-24T22:05:20.193751+0000","last_became_peered":"2026-02-24T22:05:20.193751+0000","last_unstale":"2026-02-24T22:30:19.730971+0000","last_undegraded":"2026-02-24T22:30:19.730971+0000","last_fullsized":"2026-02-24T22:30:19.730971+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:48:18.314998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"312'737","reported_seq":1813,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.525758+0000","last_change":"2026-02-24T22:05:22.200836+0000","last_active":"2026-02-24T22:34:18.525758+0000","last_peered":"2026-02-24T22:34:18.525758+0000","last_clean":"2026-02-24T22:34:18.525758+0000","last_became_active":"2026-02-24T22:05:22.200603+0000","last_became_peered":"2026-02-24T22:05:22.200603+0000","last_unstale":"2026-02-24T22:34:18.525758+0000","last_undegraded":"2026-02-24T22:34:18.525758+0000","last_fullsized":"2026-02-24T22:34:18.525758+0000","mapping_epoch":28,"log_start":"312'700","ondisk_log_start":"312'700","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":700,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:36:03.813153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63999,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":2312,"num_read_kb":1868,"num_write":428,"num_write_kb":371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"312'334","reported_seq":964,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.243815+0000","last_change":"2026-02-24T22:05:24.201087+0000","last_active":"2026-02-24T22:34:18.243815+0000","last_peered":"2026-02-24T22:34:18.243815+0000","last_clean":"2026-02-24T22:34:18.243815+0000","last_became_active":"2026-02-24T22:05:24.200391+0000","last_became_peered":"2026-02-24T22:05:24.200391+0000","last_unstale":"2026-02-24T22:34:18.243815+0000","last_undegraded":"2026-02-24T22:34:18.243815+0000","last_fullsized":"2026-02-24T22:34:18.243815+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:18:42.256342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31847,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":370,"num_read_kb":353,"num_write":777,"num_write_kb":271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"312'81","reported_seq":994,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:14.669283+0000","last_change":"2026-02-24T22:05:29.660890+0000","last_active":"2026-02-24T22:34:14.669283+0000","last_peered":"2026-02-24T22:34:14.669283+0000","last_clean":"2026-02-24T22:34:14.669283+0000","last_became_active":"2026-02-24T22:05:29.660289+0000","last_became_peered":"2026-02-24T22:05:29.660289+0000","last_unstale":"2026-02-24T22:34:14.669283+0000","last_undegraded":"2026-02-24T22:34:14.669283+0000","last_fullsized":"2026-02-24T22:34:14.669283+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:13:08.310592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"312'100","reported_seq":769,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:54.429078+0000","last_change":"2026-02-24T22:05:18.185312+0000","last_active":"2026-02-24T22:33:54.429078+0000","last_peered":"2026-02-24T22:33:54.429078+0000","last_clean":"2026-02-24T22:33:54.429078+0000","last_became_active":"2026-02-24T22:05:18.184745+0000","last_became_peered":"2026-02-24T22:05:18.184745+0000","last_unstale":"2026-02-24T22:33:54.429078+0000","last_undegraded":"2026-02-24T22:33:54.429078+0000","last_fullsized":"2026-02-24T22:33:54.429078+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:08:34.210048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":371,"num_read_kb":307,"num_write":132,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"312'221","reported_seq":941,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:55.590300+0000","last_change":"2026-02-24T22:05:16.185859+0000","last_active":"2026-02-24T22:30:55.590300+0000","last_peered":"2026-02-24T22:30:55.590300+0000","last_clean":"2026-02-24T22:30:55.590300+0000","last_became_active":"2026-02-24T22:05:16.185574+0000","last_became_peered":"2026-02-24T22:05:16.185574+0000","last_unstale":"2026-02-24T22:30:55.590300+0000","last_undegraded":"2026-02-24T22:30:55.590300+0000","last_fullsized":"2026-02-24T22:30:55.590300+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:09:04.869683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6081138,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":366,"num_read_kb":30007,"num_write":1195,"num_write_kb":21358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"32'3","reported_seq":830,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.335841+0000","last_change":"2026-02-24T22:05:20.192991+0000","last_active":"2026-02-24T22:34:18.335841+0000","last_peered":"2026-02-24T22:34:18.335841+0000","last_clean":"2026-02-24T22:34:18.335841+0000","last_became_active":"2026-02-24T22:05:20.192612+0000","last_became_peered":"2026-02-24T22:05:20.192612+0000","last_unstale":"2026-02-24T22:34:18.335841+0000","last_undegraded":"2026-02-24T22:34:18.335841+0000","last_fullsized":"2026-02-24T22:34:18.335841+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:32:11.097665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":482,"num_read_kb":480,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"312'199","reported_seq":806,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.845514+0000","last_change":"2026-02-24T22:05:22.906226+0000","last_active":"2026-02-24T22:34:17.845514+0000","last_peered":"2026-02-24T22:34:17.845514+0000","last_clean":"2026-02-24T22:34:17.845514+0000","last_became_active":"2026-02-24T22:05:22.906047+0000","last_became_peered":"2026-02-24T22:05:22.906047+0000","last_unstale":"2026-02-24T22:34:17.845514+0000","last_undegraded":"2026-02-24T22:34:17.845514+0000","last_fullsized":"2026-02-24T22:34:17.845514+0000","mapping_epoch":28,"log_start":"312'100","ondisk_log_start":"312'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:36:51.410405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50386,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":315,"num_read_kb":704,"num_write":294,"num_write_kb":265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"312'164","reported_seq":759,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:58.164024+0000","last_change":"2026-02-24T22:05:24.201114+0000","last_active":"2026-02-24T22:33:58.164024+0000","last_peered":"2026-02-24T22:33:58.164024+0000","last_clean":"2026-02-24T22:33:58.164024+0000","last_became_active":"2026-02-24T22:05:24.200747+0000","last_became_peered":"2026-02-24T22:05:24.200747+0000","last_unstale":"2026-02-24T22:33:58.164024+0000","last_undegraded":"2026-02-24T22:33:58.164024+0000","last_fullsized":"2026-02-24T22:33:58.164024+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:27:56.881747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30873,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":120,"num_read_kb":110,"num_write":405,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":565,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.740173+0000","last_change":"2026-02-24T22:05:29.660794+0000","last_active":"2026-02-24T22:31:09.740173+0000","last_peered":"2026-02-24T22:31:09.740173+0000","last_clean":"2026-02-24T22:31:09.740173+0000","last_became_active":"2026-02-24T22:05:29.660344+0000","last_became_peered":"2026-02-24T22:05:29.660344+0000","last_unstale":"2026-02-24T22:31:09.740173+0000","last_undegraded":"2026-02-24T22:31:09.740173+0000","last_fullsized":"2026-02-24T22:31:09.740173+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:36:46.842669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"312'157","reported_seq":828,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.716442+0000","last_change":"2026-02-24T22:06:00.759389+0000","last_active":"2026-02-24T22:30:19.716442+0000","last_peered":"2026-02-24T22:30:19.716442+0000","last_clean":"2026-02-24T22:30:19.716442+0000","last_became_active":"2026-02-24T22:06:00.759215+0000","last_became_peered":"2026-02-24T22:06:00.759215+0000","last_unstale":"2026-02-24T22:30:19.716442+0000","last_undegraded":"2026-02-24T22:30:19.716442+0000","last_fullsized":"2026-02-24T22:30:19.716442+0000","mapping_epoch":36,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:14:31.570934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":584,"num_read_kb":483,"num_write":222,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"312'212","reported_seq":963,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:59.428345+0000","last_change":"2026-02-24T22:05:16.184388+0000","last_active":"2026-02-24T22:33:59.428345+0000","last_peered":"2026-02-24T22:33:59.428345+0000","last_clean":"2026-02-24T22:33:59.428345+0000","last_became_active":"2026-02-24T22:05:16.183956+0000","last_became_peered":"2026-02-24T22:05:16.183956+0000","last_unstale":"2026-02-24T22:33:59.428345+0000","last_undegraded":"2026-02-24T22:33:59.428345+0000","last_fullsized":"2026-02-24T22:33:59.428345+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:01:46.056052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5448977,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":524,"num_read_kb":6430,"num_write":1216,"num_write_kb":5371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":587,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.719728+0000","last_change":"2026-02-24T22:05:20.187960+0000","last_active":"2026-02-24T22:30:19.719728+0000","last_peered":"2026-02-24T22:30:19.719728+0000","last_clean":"2026-02-24T22:30:19.719728+0000","last_became_active":"2026-02-24T22:05:20.187316+0000","last_became_peered":"2026-02-24T22:05:20.187316+0000","last_unstale":"2026-02-24T22:30:19.719728+0000","last_undegraded":"2026-02-24T22:30:19.719728+0000","last_fullsized":"2026-02-24T22:30:19.719728+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:07:22.260792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"312'287","reported_seq":1125,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.509524+0000","last_change":"2026-02-24T22:05:22.912321+0000","last_active":"2026-02-24T22:34:18.509524+0000","last_peered":"2026-02-24T22:34:18.509524+0000","last_clean":"2026-02-24T22:34:18.509524+0000","last_became_active":"2026-02-24T22:05:22.912087+0000","last_became_peered":"2026-02-24T22:05:22.912087+0000","last_unstale":"2026-02-24T22:34:18.509524+0000","last_undegraded":"2026-02-24T22:34:18.509524+0000","last_fullsized":"2026-02-24T22:34:18.509524+0000","mapping_epoch":28,"log_start":"307'200","ondisk_log_start":"307'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":200,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:16:48.283329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":152,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":996,"num_read_kb":518,"num_write":51,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"312'178","reported_seq":844,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.269129+0000","last_change":"2026-02-24T22:05:24.207156+0000","last_active":"2026-02-24T22:34:15.269129+0000","last_peered":"2026-02-24T22:34:15.269129+0000","last_clean":"2026-02-24T22:34:15.269129+0000","last_became_active":"2026-02-24T22:05:24.206383+0000","last_became_peered":"2026-02-24T22:05:24.206383+0000","last_unstale":"2026-02-24T22:34:15.269129+0000","last_undegraded":"2026-02-24T22:34:15.269129+0000","last_fullsized":"2026-02-24T22:34:15.269129+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":100,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:12:00.918186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29475,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":255,"num_read_kb":237,"num_write":434,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":571,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.740948+0000","last_change":"2026-02-24T22:05:29.661761+0000","last_active":"2026-02-24T22:31:09.740948+0000","last_peered":"2026-02-24T22:31:09.740948+0000","last_clean":"2026-02-24T22:31:09.740948+0000","last_became_active":"2026-02-24T22:05:29.661539+0000","last_became_peered":"2026-02-24T22:05:29.661539+0000","last_unstale":"2026-02-24T22:31:09.740948+0000","last_undegraded":"2026-02-24T22:31:09.740948+0000","last_fullsized":"2026-02-24T22:31:09.740948+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:02:56.515020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"312'178","reported_seq":849,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.732694+0000","last_change":"2026-02-24T22:05:18.193080+0000","last_active":"2026-02-24T22:30:19.732694+0000","last_peered":"2026-02-24T22:30:19.732694+0000","last_clean":"2026-02-24T22:30:19.732694+0000","last_became_active":"2026-02-24T22:05:18.192487+0000","last_became_peered":"2026-02-24T22:05:18.192487+0000","last_unstale":"2026-02-24T22:30:19.732694+0000","last_undegraded":"2026-02-24T22:30:19.732694+0000","last_fullsized":"2026-02-24T22:30:19.732694+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":100,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:11:58.202360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":681,"num_read_kb":551,"num_write":266,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"312'225","reported_seq":924,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.242241+0000","last_change":"2026-02-24T22:05:16.190846+0000","last_active":"2026-02-24T22:34:17.242241+0000","last_peered":"2026-02-24T22:34:17.242241+0000","last_clean":"2026-02-24T22:34:17.242241+0000","last_became_active":"2026-02-24T22:05:16.190108+0000","last_became_peered":"2026-02-24T22:05:16.190108+0000","last_unstale":"2026-02-24T22:34:17.242241+0000","last_undegraded":"2026-02-24T22:34:17.242241+0000","last_fullsized":"2026-02-24T22:34:17.242241+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":200,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:05:24.226362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10513634,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":400,"num_read_kb":15743,"num_write":1275,"num_write_kb":10328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":584,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.735341+0000","last_change":"2026-02-24T22:05:20.194434+0000","last_active":"2026-02-24T22:30:19.735341+0000","last_peered":"2026-02-24T22:30:19.735341+0000","last_clean":"2026-02-24T22:30:19.735341+0000","last_became_active":"2026-02-24T22:05:20.194183+0000","last_became_peered":"2026-02-24T22:05:20.194183+0000","last_unstale":"2026-02-24T22:30:19.735341+0000","last_undegraded":"2026-02-24T22:30:19.735341+0000","last_fullsized":"2026-02-24T22:30:19.735341+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:11:03.103844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"312'706","reported_seq":1776,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.522544+0000","last_change":"2026-02-24T22:05:22.199767+0000","last_active":"2026-02-24T22:34:18.522544+0000","last_peered":"2026-02-24T22:34:18.522544+0000","last_clean":"2026-02-24T22:34:18.522544+0000","last_became_active":"2026-02-24T22:05:22.199480+0000","last_became_peered":"2026-02-24T22:05:22.199480+0000","last_unstale":"2026-02-24T22:34:18.522544+0000","last_undegraded":"2026-02-24T22:34:18.522544+0000","last_fullsized":"2026-02-24T22:34:18.522544+0000","mapping_epoch":28,"log_start":"312'600","ondisk_log_start":"312'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":600,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:44:12.736876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56130,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":2245,"num_read_kb":1729,"num_write":346,"num_write_kb":294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"312'198","reported_seq":804,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.239318+0000","last_change":"2026-02-24T22:05:24.201028+0000","last_active":"2026-02-24T22:34:15.239318+0000","last_peered":"2026-02-24T22:34:15.239318+0000","last_clean":"2026-02-24T22:34:15.239318+0000","last_became_active":"2026-02-24T22:05:24.200434+0000","last_became_peered":"2026-02-24T22:05:24.200434+0000","last_unstale":"2026-02-24T22:34:15.239318+0000","last_undegraded":"2026-02-24T22:34:15.239318+0000","last_fullsized":"2026-02-24T22:34:15.239318+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":100,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:38:49.379078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27134,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":189,"num_read_kb":178,"num_write":476,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":570,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:39.731065+0000","last_change":"2026-02-24T22:05:29.662687+0000","last_active":"2026-02-24T22:29:39.731065+0000","last_peered":"2026-02-24T22:29:39.731065+0000","last_clean":"2026-02-24T22:29:39.731065+0000","last_became_active":"2026-02-24T22:05:29.662425+0000","last_became_peered":"2026-02-24T22:05:29.662425+0000","last_unstale":"2026-02-24T22:29:39.731065+0000","last_undegraded":"2026-02-24T22:29:39.731065+0000","last_fullsized":"2026-02-24T22:29:39.731065+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:37:43.066587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"312'132","reported_seq":810,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.732676+0000","last_change":"2026-02-24T22:05:18.193443+0000","last_active":"2026-02-24T22:30:19.732676+0000","last_peered":"2026-02-24T22:30:19.732676+0000","last_clean":"2026-02-24T22:30:19.732676+0000","last_became_active":"2026-02-24T22:05:18.191664+0000","last_became_peered":"2026-02-24T22:05:18.191664+0000","last_unstale":"2026-02-24T22:30:19.732676+0000","last_undegraded":"2026-02-24T22:30:19.732676+0000","last_fullsized":"2026-02-24T22:30:19.732676+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":100,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:59:22.441702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":463,"num_read_kb":379,"num_write":198,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"312'212","reported_seq":907,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:55.742771+0000","last_change":"2026-02-24T22:05:16.189190+0000","last_active":"2026-02-24T22:33:55.742771+0000","last_peered":"2026-02-24T22:33:55.742771+0000","last_clean":"2026-02-24T22:33:55.742771+0000","last_became_active":"2026-02-24T22:05:16.188154+0000","last_became_peered":"2026-02-24T22:05:16.188154+0000","last_unstale":"2026-02-24T22:33:55.742771+0000","last_undegraded":"2026-02-24T22:33:55.742771+0000","last_fullsized":"2026-02-24T22:33:55.742771+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:45:59.561385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65745,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":374,"num_read_kb":2459,"num_write":1169,"num_write_kb":1814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"32'5","reported_seq":820,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.334772+0000","last_change":"2026-02-24T22:05:20.195455+0000","last_active":"2026-02-24T22:34:18.334772+0000","last_peered":"2026-02-24T22:34:18.334772+0000","last_clean":"2026-02-24T22:34:18.334772+0000","last_became_active":"2026-02-24T22:05:20.195202+0000","last_became_peered":"2026-02-24T22:05:20.195202+0000","last_unstale":"2026-02-24T22:34:18.334772+0000","last_undegraded":"2026-02-24T22:34:18.334772+0000","last_fullsized":"2026-02-24T22:34:18.334772+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:34:18.002332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1242,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":486,"num_read_kb":719,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"312'680","reported_seq":1751,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.538422+0000","last_change":"2026-02-24T22:05:22.192219+0000","last_active":"2026-02-24T22:34:18.538422+0000","last_peered":"2026-02-24T22:34:18.538422+0000","last_clean":"2026-02-24T22:34:18.538422+0000","last_became_active":"2026-02-24T22:05:22.192035+0000","last_became_peered":"2026-02-24T22:05:22.192035+0000","last_unstale":"2026-02-24T22:34:18.538422+0000","last_undegraded":"2026-02-24T22:34:18.538422+0000","last_fullsized":"2026-02-24T22:34:18.538422+0000","mapping_epoch":28,"log_start":"312'600","ondisk_log_start":"312'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":600,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:05:53.268094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50571,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2200,"num_read_kb":1666,"num_write":310,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"312'2477","reported_seq":5136,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.218480+0000","last_change":"2026-02-24T22:05:24.206516+0000","last_active":"2026-02-24T22:34:18.218480+0000","last_peered":"2026-02-24T22:34:18.218480+0000","last_clean":"2026-02-24T22:34:18.218480+0000","last_became_active":"2026-02-24T22:05:24.205557+0000","last_became_peered":"2026-02-24T22:05:24.205557+0000","last_unstale":"2026-02-24T22:34:18.218480+0000","last_undegraded":"2026-02-24T22:34:18.218480+0000","last_fullsized":"2026-02-24T22:34:18.218480+0000","mapping_epoch":30,"log_start":"312'2400","ondisk_log_start":"312'2400","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":2400,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:33:57.249269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34698,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":6755,"num_read_kb":6753,"num_write":5061,"num_write_kb":2022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:39.720637+0000","last_change":"2026-02-24T22:05:29.651962+0000","last_active":"2026-02-24T22:29:39.720637+0000","last_peered":"2026-02-24T22:29:39.720637+0000","last_clean":"2026-02-24T22:29:39.720637+0000","last_became_active":"2026-02-24T22:05:29.651760+0000","last_became_peered":"2026-02-24T22:05:29.651760+0000","last_unstale":"2026-02-24T22:29:39.720637+0000","last_undegraded":"2026-02-24T22:29:39.720637+0000","last_fullsized":"2026-02-24T22:29:39.720637+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:48:41.143100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"312'128","reported_seq":805,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:59.734737+0000","last_change":"2026-02-24T22:05:18.185689+0000","last_active":"2026-02-24T22:31:59.734737+0000","last_peered":"2026-02-24T22:31:59.734737+0000","last_clean":"2026-02-24T22:31:59.734737+0000","last_became_active":"2026-02-24T22:05:18.184939+0000","last_became_peered":"2026-02-24T22:05:18.184939+0000","last_unstale":"2026-02-24T22:31:59.734737+0000","last_undegraded":"2026-02-24T22:31:59.734737+0000","last_fullsized":"2026-02-24T22:31:59.734737+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:18:46.745974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":436,"num_read_kb":360,"num_write":185,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"312'215","reported_seq":939,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.731199+0000","last_change":"2026-02-24T22:05:16.186833+0000","last_active":"2026-02-24T22:31:24.731199+0000","last_peered":"2026-02-24T22:31:24.731199+0000","last_clean":"2026-02-24T22:31:24.731199+0000","last_became_active":"2026-02-24T22:05:16.186043+0000","last_became_peered":"2026-02-24T22:05:16.186043+0000","last_unstale":"2026-02-24T22:31:24.731199+0000","last_undegraded":"2026-02-24T22:31:24.731199+0000","last_fullsized":"2026-02-24T22:31:24.731199+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:59:46.638347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5313539,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":397,"num_read_kb":15748,"num_write":1186,"num_write_kb":5255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.742847+0000","last_change":"2026-02-24T22:05:20.193254+0000","last_active":"2026-02-24T22:31:54.742847+0000","last_peered":"2026-02-24T22:31:54.742847+0000","last_clean":"2026-02-24T22:31:54.742847+0000","last_became_active":"2026-02-24T22:05:20.192962+0000","last_became_peered":"2026-02-24T22:05:20.192962+0000","last_unstale":"2026-02-24T22:31:54.742847+0000","last_undegraded":"2026-02-24T22:31:54.742847+0000","last_fullsized":"2026-02-24T22:31:54.742847+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:15:25.988404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"312'469","reported_seq":1300,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.539799+0000","last_change":"2026-02-24T22:05:22.194159+0000","last_active":"2026-02-24T22:34:18.539799+0000","last_peered":"2026-02-24T22:34:18.539799+0000","last_clean":"2026-02-24T22:34:18.539799+0000","last_became_active":"2026-02-24T22:05:22.193896+0000","last_became_peered":"2026-02-24T22:05:22.193896+0000","last_unstale":"2026-02-24T22:34:18.539799+0000","last_undegraded":"2026-02-24T22:34:18.539799+0000","last_fullsized":"2026-02-24T22:34:18.539799+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":400,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:30:50.341834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59985,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1293,"num_read_kb":1284,"num_write":349,"num_write_kb":316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"312'188","reported_seq":800,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.715171+0000","last_change":"2026-02-24T22:05:24.206176+0000","last_active":"2026-02-24T22:34:17.715171+0000","last_peered":"2026-02-24T22:34:17.715171+0000","last_clean":"2026-02-24T22:34:17.715171+0000","last_became_active":"2026-02-24T22:05:24.205855+0000","last_became_peered":"2026-02-24T22:05:24.205855+0000","last_unstale":"2026-02-24T22:34:17.715171+0000","last_undegraded":"2026-02-24T22:34:17.715171+0000","last_fullsized":"2026-02-24T22:34:17.715171+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":100,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:18:56.993738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27729,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":196,"num_read_kb":179,"num_write":471,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"312'443","reported_seq":1614,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.571252+0000","last_change":"2026-02-24T22:05:29.662310+0000","last_active":"2026-02-24T22:34:18.571252+0000","last_peered":"2026-02-24T22:34:18.571252+0000","last_clean":"2026-02-24T22:34:18.571252+0000","last_became_active":"2026-02-24T22:05:29.661983+0000","last_became_peered":"2026-02-24T22:05:29.661983+0000","last_unstale":"2026-02-24T22:34:18.571252+0000","last_undegraded":"2026-02-24T22:34:18.571252+0000","last_fullsized":"2026-02-24T22:34:18.571252+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":400,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:00:08.834953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"312'169","reported_seq":852,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.250160+0000","last_change":"2026-02-24T22:05:18.191009+0000","last_active":"2026-02-24T22:34:17.250160+0000","last_peered":"2026-02-24T22:34:17.250160+0000","last_clean":"2026-02-24T22:34:17.250160+0000","last_became_active":"2026-02-24T22:05:18.189877+0000","last_became_peered":"2026-02-24T22:05:18.189877+0000","last_unstale":"2026-02-24T22:34:17.250160+0000","last_undegraded":"2026-02-24T22:34:17.250160+0000","last_fullsized":"2026-02-24T22:34:17.250160+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:33:55.890050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":510,"num_write":249,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"312'253","reported_seq":1012,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:04.870640+0000","last_change":"2026-02-24T22:05:16.184964+0000","last_active":"2026-02-24T22:34:04.870640+0000","last_peered":"2026-02-24T22:34:04.870640+0000","last_clean":"2026-02-24T22:34:04.870640+0000","last_became_active":"2026-02-24T22:05:16.184525+0000","last_became_peered":"2026-02-24T22:05:16.184525+0000","last_unstale":"2026-02-24T22:34:04.870640+0000","last_undegraded":"2026-02-24T22:34:04.870640+0000","last_fullsized":"2026-02-24T22:34:04.870640+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:33:11.439600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10579665,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":508,"num_read_kb":16318,"num_write":1420,"num_write_kb":16752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"27'1","reported_seq":589,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:59.729910+0000","last_change":"2026-02-24T22:05:20.187071+0000","last_active":"2026-02-24T22:31:59.729910+0000","last_peered":"2026-02-24T22:31:59.729910+0000","last_clean":"2026-02-24T22:31:59.729910+0000","last_became_active":"2026-02-24T22:05:20.186850+0000","last_became_peered":"2026-02-24T22:05:20.186850+0000","last_unstale":"2026-02-24T22:31:59.729910+0000","last_undegraded":"2026-02-24T22:31:59.729910+0000","last_fullsized":"2026-02-24T22:31:59.729910+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:28:00.779043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"312'244","reported_seq":862,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.831208+0000","last_change":"2026-02-24T22:05:22.199568+0000","last_active":"2026-02-24T22:34:17.831208+0000","last_peered":"2026-02-24T22:34:17.831208+0000","last_clean":"2026-02-24T22:34:17.831208+0000","last_became_active":"2026-02-24T22:05:22.199358+0000","last_became_peered":"2026-02-24T22:05:22.199358+0000","last_unstale":"2026-02-24T22:34:17.831208+0000","last_undegraded":"2026-02-24T22:34:17.831208+0000","last_fullsized":"2026-02-24T22:34:17.831208+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":200,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:24:45.196918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59999,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":388,"num_read_kb":835,"num_write":354,"num_write_kb":306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"312'281","reported_seq":1222,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.228289+0000","last_change":"2026-02-24T22:05:24.200431+0000","last_active":"2026-02-24T22:34:18.228289+0000","last_peered":"2026-02-24T22:34:18.228289+0000","last_clean":"2026-02-24T22:34:18.228289+0000","last_became_active":"2026-02-24T22:05:24.199555+0000","last_became_peered":"2026-02-24T22:05:24.199555+0000","last_unstale":"2026-02-24T22:34:18.228289+0000","last_undegraded":"2026-02-24T22:34:18.228289+0000","last_fullsized":"2026-02-24T22:34:18.228289+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:29:23.063946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30969,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":655,"num_read_kb":647,"num_write":639,"num_write_kb":204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"312'429","reported_seq":1947,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.571358+0000","last_change":"2026-02-24T22:05:29.660609+0000","last_active":"2026-02-24T22:34:18.571358+0000","last_peered":"2026-02-24T22:34:18.571358+0000","last_clean":"2026-02-24T22:34:18.571358+0000","last_became_active":"2026-02-24T22:05:29.659532+0000","last_became_peered":"2026-02-24T22:05:29.659532+0000","last_unstale":"2026-02-24T22:34:18.571358+0000","last_undegraded":"2026-02-24T22:34:18.571358+0000","last_fullsized":"2026-02-24T22:34:18.571358+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":400,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:09:07.648679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"312'139","reported_seq":825,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.725732+0000","last_change":"2026-02-24T22:05:18.184639+0000","last_active":"2026-02-24T22:31:19.725732+0000","last_peered":"2026-02-24T22:31:19.725732+0000","last_clean":"2026-02-24T22:31:19.725732+0000","last_became_active":"2026-02-24T22:05:18.184250+0000","last_became_peered":"2026-02-24T22:05:18.184250+0000","last_unstale":"2026-02-24T22:31:19.725732+0000","last_undegraded":"2026-02-24T22:31:19.725732+0000","last_fullsized":"2026-02-24T22:31:19.725732+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:38:26.901167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":560,"num_read_kb":462,"num_write":204,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"312'1519","reported_seq":2874,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.226363+0000","last_change":"2026-02-24T22:05:16.190173+0000","last_active":"2026-02-24T22:34:17.226363+0000","last_peered":"2026-02-24T22:34:17.226363+0000","last_clean":"2026-02-24T22:34:17.226363+0000","last_became_active":"2026-02-24T22:05:16.189660+0000","last_became_peered":"2026-02-24T22:05:16.189660+0000","last_unstale":"2026-02-24T22:34:17.226363+0000","last_undegraded":"2026-02-24T22:34:17.226363+0000","last_fullsized":"2026-02-24T22:34:17.226363+0000","mapping_epoch":22,"log_start":"312'1500","ondisk_log_start":"312'1500","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":1500,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:28:22.592939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":80905,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":4244,"num_read_kb":2601,"num_write":2602,"num_write_kb":4261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"312'101","reported_seq":779,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:01.356810+0000","last_change":"2026-02-24T22:05:18.184329+0000","last_active":"2026-02-24T22:34:01.356810+0000","last_peered":"2026-02-24T22:34:01.356810+0000","last_clean":"2026-02-24T22:34:01.356810+0000","last_became_active":"2026-02-24T22:05:18.183822+0000","last_became_peered":"2026-02-24T22:05:18.183822+0000","last_unstale":"2026-02-24T22:34:01.356810+0000","last_undegraded":"2026-02-24T22:34:01.356810+0000","last_fullsized":"2026-02-24T22:34:01.356810+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:25:35.055559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":390,"num_read_kb":330,"num_write":143,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"312'231","reported_seq":979,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:53.833443+0000","last_change":"2026-02-24T22:05:16.183674+0000","last_active":"2026-02-24T22:30:53.833443+0000","last_peered":"2026-02-24T22:30:53.833443+0000","last_clean":"2026-02-24T22:30:53.833443+0000","last_became_active":"2026-02-24T22:05:16.183487+0000","last_became_peered":"2026-02-24T22:05:16.183487+0000","last_unstale":"2026-02-24T22:30:53.833443+0000","last_undegraded":"2026-02-24T22:30:53.833443+0000","last_fullsized":"2026-02-24T22:30:53.833443+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":200,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:25:02.663360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117418,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":445,"num_read_kb":706,"num_write":1288,"num_write_kb":183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"312'72","reported_seq":731,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.741422+0000","last_change":"2026-02-24T22:05:18.190755+0000","last_active":"2026-02-24T22:31:19.741422+0000","last_peered":"2026-02-24T22:31:19.741422+0000","last_clean":"2026-02-24T22:31:19.741422+0000","last_became_active":"2026-02-24T22:05:18.190352+0000","last_became_peered":"2026-02-24T22:05:18.190352+0000","last_unstale":"2026-02-24T22:31:19.741422+0000","last_undegraded":"2026-02-24T22:31:19.741422+0000","last_fullsized":"2026-02-24T22:31:19.741422+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:16:19.432104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":274,"num_read_kb":236,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"312'236","reported_seq":1158,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:12.225391+0000","last_change":"2026-02-24T22:05:16.191430+0000","last_active":"2026-02-24T22:34:12.225391+0000","last_peered":"2026-02-24T22:34:12.225391+0000","last_clean":"2026-02-24T22:34:12.225391+0000","last_became_active":"2026-02-24T22:05:16.190494+0000","last_became_peered":"2026-02-24T22:05:16.190494+0000","last_unstale":"2026-02-24T22:34:12.225391+0000","last_undegraded":"2026-02-24T22:34:12.225391+0000","last_fullsized":"2026-02-24T22:34:12.225391+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":200,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:14:56.208933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6381012,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":1034,"num_read_kb":14913,"num_write":1257,"num_write_kb":8068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"312'337","reported_seq":1150,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.725684+0000","last_change":"2026-02-24T22:05:18.186968+0000","last_active":"2026-02-24T22:31:19.725684+0000","last_peered":"2026-02-24T22:31:19.725684+0000","last_clean":"2026-02-24T22:31:19.725684+0000","last_became_active":"2026-02-24T22:05:18.186622+0000","last_became_peered":"2026-02-24T22:05:18.186622+0000","last_unstale":"2026-02-24T22:31:19.725684+0000","last_undegraded":"2026-02-24T22:31:19.725684+0000","last_fullsized":"2026-02-24T22:31:19.725684+0000","mapping_epoch":24,"log_start":"312'300","ondisk_log_start":"312'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":300,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:34:46.036169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1512,"num_read_kb":1316,"num_write":481,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"312'236","reported_seq":1075,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:06.918892+0000","last_change":"2026-02-24T22:05:16.186186+0000","last_active":"2026-02-24T22:34:06.918892+0000","last_peered":"2026-02-24T22:34:06.918892+0000","last_clean":"2026-02-24T22:34:06.918892+0000","last_became_active":"2026-02-24T22:05:16.185977+0000","last_became_peered":"2026-02-24T22:05:16.185977+0000","last_unstale":"2026-02-24T22:34:06.918892+0000","last_undegraded":"2026-02-24T22:34:06.918892+0000","last_fullsized":"2026-02-24T22:34:06.918892+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":200,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:53:50.492464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1207187,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":871,"num_read_kb":5969,"num_write":1291,"num_write_kb":2943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"312'89","reported_seq":764,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:00.529273+0000","last_change":"2026-02-24T22:05:18.186317+0000","last_active":"2026-02-24T22:34:00.529273+0000","last_peered":"2026-02-24T22:34:00.529273+0000","last_clean":"2026-02-24T22:34:00.529273+0000","last_became_active":"2026-02-24T22:05:18.185180+0000","last_became_peered":"2026-02-24T22:05:18.185180+0000","last_unstale":"2026-02-24T22:34:00.529273+0000","last_undegraded":"2026-02-24T22:34:00.529273+0000","last_fullsized":"2026-02-24T22:34:00.529273+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:17:22.057256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":311,"num_read_kb":264,"num_write":113,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"312'283","reported_seq":1035,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:59.726421+0000","last_change":"2026-02-24T22:06:00.660921+0000","last_active":"2026-02-24T22:31:59.726421+0000","last_peered":"2026-02-24T22:31:59.726421+0000","last_clean":"2026-02-24T22:31:59.726421+0000","last_became_active":"2026-02-24T22:06:00.660740+0000","last_became_peered":"2026-02-24T22:06:00.660740+0000","last_unstale":"2026-02-24T22:31:59.726421+0000","last_undegraded":"2026-02-24T22:31:59.726421+0000","last_fullsized":"2026-02-24T22:31:59.726421+0000","mapping_epoch":36,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":200,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:58:32.246490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12132727,"num_objects":86,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":579,"num_read_kb":9889,"num_write":1318,"num_write_kb":11975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"312'66","reported_seq":735,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.732670+0000","last_change":"2026-02-24T22:05:18.191552+0000","last_active":"2026-02-24T22:30:19.732670+0000","last_peered":"2026-02-24T22:30:19.732670+0000","last_clean":"2026-02-24T22:30:19.732670+0000","last_became_active":"2026-02-24T22:05:18.190390+0000","last_became_peered":"2026-02-24T22:05:18.190390+0000","last_unstale":"2026-02-24T22:30:19.732670+0000","last_undegraded":"2026-02-24T22:30:19.732670+0000","last_fullsized":"2026-02-24T22:30:19.732670+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:20:31.773222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":213,"num_read_kb":187,"num_write":90,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"312'255","reported_seq":978,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.725697+0000","last_change":"2026-02-24T22:05:16.184281+0000","last_active":"2026-02-24T22:31:19.725697+0000","last_peered":"2026-02-24T22:31:19.725697+0000","last_clean":"2026-02-24T22:31:19.725697+0000","last_became_active":"2026-02-24T22:05:16.184091+0000","last_became_peered":"2026-02-24T22:05:16.184091+0000","last_unstale":"2026-02-24T22:31:19.725697+0000","last_undegraded":"2026-02-24T22:31:19.725697+0000","last_fullsized":"2026-02-24T22:31:19.725697+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:42:08.074892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6296559,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":482,"num_read_kb":400,"num_write":1161,"num_write_kb":6221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"312'355","reported_seq":1157,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:45.969143+0000","last_change":"2026-02-24T22:05:18.187245+0000","last_active":"2026-02-24T22:33:45.969143+0000","last_peered":"2026-02-24T22:33:45.969143+0000","last_clean":"2026-02-24T22:33:45.969143+0000","last_became_active":"2026-02-24T22:05:18.186746+0000","last_became_peered":"2026-02-24T22:05:18.186746+0000","last_unstale":"2026-02-24T22:33:45.969143+0000","last_undegraded":"2026-02-24T22:33:45.969143+0000","last_fullsized":"2026-02-24T22:33:45.969143+0000","mapping_epoch":24,"log_start":"312'300","ondisk_log_start":"312'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":300,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:16:49.207184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":1676,"num_read_kb":1392,"num_write":638,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"312'250","reported_seq":954,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.737867+0000","last_change":"2026-02-24T22:05:16.190391+0000","last_active":"2026-02-24T22:31:19.737867+0000","last_peered":"2026-02-24T22:31:19.737867+0000","last_clean":"2026-02-24T22:31:19.737867+0000","last_became_active":"2026-02-24T22:05:16.189516+0000","last_became_peered":"2026-02-24T22:05:16.189516+0000","last_unstale":"2026-02-24T22:31:19.737867+0000","last_undegraded":"2026-02-24T22:31:19.737867+0000","last_fullsized":"2026-02-24T22:31:19.737867+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":200,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:11:15.635731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10540630,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":369,"num_read_kb":5491,"num_write":1149,"num_write_kb":20654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"312'164","reported_seq":854,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:03.781941+0000","last_change":"2026-02-24T22:05:18.190682+0000","last_active":"2026-02-24T22:34:03.781941+0000","last_peered":"2026-02-24T22:34:03.781941+0000","last_clean":"2026-02-24T22:34:03.781941+0000","last_became_active":"2026-02-24T22:05:18.190203+0000","last_became_peered":"2026-02-24T22:05:18.190203+0000","last_unstale":"2026-02-24T22:34:03.781941+0000","last_undegraded":"2026-02-24T22:34:03.781941+0000","last_fullsized":"2026-02-24T22:34:03.781941+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:20:56.349004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":631,"num_read_kb":516,"num_write":248,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"312'241","reported_seq":960,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:12.654487+0000","last_change":"2026-02-24T22:05:16.184697+0000","last_active":"2026-02-24T22:34:12.654487+0000","last_peered":"2026-02-24T22:34:12.654487+0000","last_clean":"2026-02-24T22:34:12.654487+0000","last_became_active":"2026-02-24T22:05:16.183772+0000","last_became_peered":"2026-02-24T22:05:16.183772+0000","last_unstale":"2026-02-24T22:34:12.654487+0000","last_undegraded":"2026-02-24T22:34:12.654487+0000","last_fullsized":"2026-02-24T22:34:12.654487+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":200,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:22:42.349222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6042885,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":404,"num_read_kb":12485,"num_write":1265,"num_write_kb":5970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"312'130","reported_seq":850,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.737414+0000","last_change":"2026-02-24T22:05:18.187529+0000","last_active":"2026-02-24T22:30:59.737414+0000","last_peered":"2026-02-24T22:30:59.737414+0000","last_clean":"2026-02-24T22:30:59.737414+0000","last_became_active":"2026-02-24T22:05:18.187321+0000","last_became_peered":"2026-02-24T22:05:18.187321+0000","last_unstale":"2026-02-24T22:30:59.737414+0000","last_undegraded":"2026-02-24T22:30:59.737414+0000","last_fullsized":"2026-02-24T22:30:59.737414+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:08:55.925019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":512,"num_read_kb":441,"num_write":201,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"312'238","reported_seq":946,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:58.282425+0000","last_change":"2026-02-24T22:05:16.193409+0000","last_active":"2026-02-24T22:33:58.282425+0000","last_peered":"2026-02-24T22:33:58.282425+0000","last_clean":"2026-02-24T22:33:58.282425+0000","last_became_active":"2026-02-24T22:05:16.193164+0000","last_became_peered":"2026-02-24T22:05:16.193164+0000","last_unstale":"2026-02-24T22:33:58.282425+0000","last_undegraded":"2026-02-24T22:33:58.282425+0000","last_fullsized":"2026-02-24T22:33:58.282425+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":200,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:41:44.916032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10795512,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":356,"num_read_kb":36420,"num_write":1300,"num_write_kb":10617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"312'148","reported_seq":825,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.737040+0000","last_change":"2026-02-24T22:05:18.193362+0000","last_active":"2026-02-24T22:30:59.737040+0000","last_peered":"2026-02-24T22:30:59.737040+0000","last_clean":"2026-02-24T22:30:59.737040+0000","last_became_active":"2026-02-24T22:05:18.191737+0000","last_became_peered":"2026-02-24T22:05:18.191737+0000","last_unstale":"2026-02-24T22:30:59.737040+0000","last_undegraded":"2026-02-24T22:30:59.737040+0000","last_fullsized":"2026-02-24T22:30:59.737040+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":100,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:51:20.868848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":539,"num_read_kb":448,"num_write":211,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"312'298","reported_seq":1049,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:54.775236+0000","last_change":"2026-02-24T22:05:16.184466+0000","last_active":"2026-02-24T22:30:54.775236+0000","last_peered":"2026-02-24T22:30:54.775236+0000","last_clean":"2026-02-24T22:30:54.775236+0000","last_became_active":"2026-02-24T22:05:16.183923+0000","last_became_peered":"2026-02-24T22:05:16.183923+0000","last_unstale":"2026-02-24T22:30:54.775236+0000","last_undegraded":"2026-02-24T22:30:54.775236+0000","last_fullsized":"2026-02-24T22:30:54.775236+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:15:14.475036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5523994,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":476,"num_read_kb":9842,"num_write":1525,"num_write_kb":9073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"312'146","reported_seq":817,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:09.715504+0000","last_change":"2026-02-24T22:05:18.184019+0000","last_active":"2026-02-24T22:30:09.715504+0000","last_peered":"2026-02-24T22:30:09.715504+0000","last_clean":"2026-02-24T22:30:09.715504+0000","last_became_active":"2026-02-24T22:05:18.183761+0000","last_became_peered":"2026-02-24T22:05:18.183761+0000","last_unstale":"2026-02-24T22:30:09.715504+0000","last_undegraded":"2026-02-24T22:30:09.715504+0000","last_fullsized":"2026-02-24T22:30:09.715504+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:07:03.909559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":561,"num_read_kb":451,"num_write":221,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"312'267","reported_seq":1036,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:44.059088+0000","last_change":"2026-02-24T22:05:16.191018+0000","last_active":"2026-02-24T22:33:44.059088+0000","last_peered":"2026-02-24T22:33:44.059088+0000","last_clean":"2026-02-24T22:33:44.059088+0000","last_became_active":"2026-02-24T22:05:16.189759+0000","last_became_peered":"2026-02-24T22:05:16.189759+0000","last_unstale":"2026-02-24T22:33:44.059088+0000","last_undegraded":"2026-02-24T22:33:44.059088+0000","last_fullsized":"2026-02-24T22:33:44.059088+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":200,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:34:51.111413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1371100,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":587,"num_read_kb":3529,"num_write":1434,"num_write_kb":3094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"312'140","reported_seq":832,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:31.161689+0000","last_change":"2026-02-24T22:05:18.194125+0000","last_active":"2026-02-24T22:30:31.161689+0000","last_peered":"2026-02-24T22:30:31.161689+0000","last_clean":"2026-02-24T22:30:31.161689+0000","last_became_active":"2026-02-24T22:05:18.192882+0000","last_became_peered":"2026-02-24T22:05:18.192882+0000","last_unstale":"2026-02-24T22:30:31.161689+0000","last_undegraded":"2026-02-24T22:30:31.161689+0000","last_fullsized":"2026-02-24T22:30:31.161689+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":100,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:13:16.621031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":593,"num_read_kb":494,"num_write":208,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"312'228","reported_seq":966,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.253876+0000","last_change":"2026-02-24T22:05:16.184964+0000","last_active":"2026-02-24T22:34:17.253876+0000","last_peered":"2026-02-24T22:34:17.253876+0000","last_clean":"2026-02-24T22:34:17.253876+0000","last_became_active":"2026-02-24T22:05:16.184557+0000","last_became_peered":"2026-02-24T22:05:16.184557+0000","last_unstale":"2026-02-24T22:34:17.253876+0000","last_undegraded":"2026-02-24T22:34:17.253876+0000","last_fullsized":"2026-02-24T22:34:17.253876+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":200,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:16:07.490514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5643921,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":420,"num_read_kb":1536,"num_write":1181,"num_write_kb":5590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"312'158","reported_seq":865,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:04.901988+0000","last_change":"2026-02-24T22:05:18.191676+0000","last_active":"2026-02-24T22:34:04.901988+0000","last_peered":"2026-02-24T22:34:04.901988+0000","last_clean":"2026-02-24T22:34:04.901988+0000","last_became_active":"2026-02-24T22:05:18.191432+0000","last_became_peered":"2026-02-24T22:05:18.191432+0000","last_unstale":"2026-02-24T22:34:04.901988+0000","last_undegraded":"2026-02-24T22:34:04.901988+0000","last_fullsized":"2026-02-24T22:34:04.901988+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":100,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:14:52.433749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":649,"num_read_kb":535,"num_write":233,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"312'222","reported_seq":943,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:00.269049+0000","last_change":"2026-02-24T22:05:16.190689+0000","last_active":"2026-02-24T22:34:00.269049+0000","last_peered":"2026-02-24T22:34:00.269049+0000","last_clean":"2026-02-24T22:34:00.269049+0000","last_became_active":"2026-02-24T22:05:16.190260+0000","last_became_peered":"2026-02-24T22:05:16.190260+0000","last_unstale":"2026-02-24T22:34:00.269049+0000","last_undegraded":"2026-02-24T22:34:00.269049+0000","last_fullsized":"2026-02-24T22:34:00.269049+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":200,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:55:30.357144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":254576,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":395,"num_read_kb":1504,"num_write":1294,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"312'143","reported_seq":831,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:15.308606+0000","last_change":"2026-02-24T22:05:18.185534+0000","last_active":"2026-02-24T22:30:15.308606+0000","last_peered":"2026-02-24T22:30:15.308606+0000","last_clean":"2026-02-24T22:30:15.308606+0000","last_became_active":"2026-02-24T22:05:18.185184+0000","last_became_peered":"2026-02-24T22:05:18.185184+0000","last_unstale":"2026-02-24T22:30:15.308606+0000","last_undegraded":"2026-02-24T22:30:15.308606+0000","last_fullsized":"2026-02-24T22:30:15.308606+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":100,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:50:27.214195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":536,"num_read_kb":445,"num_write":200,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"312'228","reported_seq":1041,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:32.940510+0000","last_change":"2026-02-24T22:05:16.191775+0000","last_active":"2026-02-24T22:33:32.940510+0000","last_peered":"2026-02-24T22:33:32.940510+0000","last_clean":"2026-02-24T22:33:32.940510+0000","last_became_active":"2026-02-24T22:05:16.191433+0000","last_became_peered":"2026-02-24T22:05:16.191433+0000","last_unstale":"2026-02-24T22:33:32.940510+0000","last_undegraded":"2026-02-24T22:33:32.940510+0000","last_fullsized":"2026-02-24T22:33:32.940510+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":200,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:08:07.147079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5270943,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":479,"num_read_kb":2264,"num_write":1260,"num_write_kb":7006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"312'62","reported_seq":721,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:08.697007+0000","last_change":"2026-02-24T22:05:18.192824+0000","last_active":"2026-02-24T22:30:08.697007+0000","last_peered":"2026-02-24T22:30:08.697007+0000","last_clean":"2026-02-24T22:30:08.697007+0000","last_became_active":"2026-02-24T22:05:18.192619+0000","last_became_peered":"2026-02-24T22:05:18.192619+0000","last_unstale":"2026-02-24T22:30:08.697007+0000","last_undegraded":"2026-02-24T22:30:08.697007+0000","last_fullsized":"2026-02-24T22:30:08.697007+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:21:57.498044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":204,"num_read_kb":176,"num_write":76,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"312'253","reported_seq":985,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:54.852196+0000","last_change":"2026-02-24T22:05:16.184618+0000","last_active":"2026-02-24T22:33:54.852196+0000","last_peered":"2026-02-24T22:33:54.852196+0000","last_clean":"2026-02-24T22:33:54.852196+0000","last_became_active":"2026-02-24T22:05:16.184441+0000","last_became_peered":"2026-02-24T22:05:16.184441+0000","last_unstale":"2026-02-24T22:33:54.852196+0000","last_undegraded":"2026-02-24T22:33:54.852196+0000","last_fullsized":"2026-02-24T22:33:54.852196+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:14:27.469981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1578451,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":396,"num_read_kb":3850,"num_write":1386,"num_write_kb":8445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"312'159","reported_seq":876,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:58.577766+0000","last_change":"2026-02-24T22:05:18.191572+0000","last_active":"2026-02-24T22:33:58.577766+0000","last_peered":"2026-02-24T22:33:58.577766+0000","last_clean":"2026-02-24T22:33:58.577766+0000","last_became_active":"2026-02-24T22:05:18.191350+0000","last_became_peered":"2026-02-24T22:05:18.191350+0000","last_unstale":"2026-02-24T22:33:58.577766+0000","last_undegraded":"2026-02-24T22:33:58.577766+0000","last_fullsized":"2026-02-24T22:33:58.577766+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:05:49.312781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":662,"num_read_kb":557,"num_write":245,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"312'219","reported_seq":987,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:43.905970+0000","last_change":"2026-02-24T22:06:00.662883+0000","last_active":"2026-02-24T22:33:43.905970+0000","last_peered":"2026-02-24T22:33:43.905970+0000","last_clean":"2026-02-24T22:33:43.905970+0000","last_became_active":"2026-02-24T22:06:00.662656+0000","last_became_peered":"2026-02-24T22:06:00.662656+0000","last_unstale":"2026-02-24T22:33:43.905970+0000","last_undegraded":"2026-02-24T22:33:43.905970+0000","last_fullsized":"2026-02-24T22:33:43.905970+0000","mapping_epoch":36,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":200,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:35:38.140607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5604982,"num_objects":86,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":721,"num_read_kb":6090,"num_write":1121,"num_write_kb":5556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.30","version":"312'268","reported_seq":1020,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:51.515479+0000","last_change":"2026-02-24T22:05:16.190494+0000","last_active":"2026-02-24T22:33:51.515479+0000","last_peered":"2026-02-24T22:33:51.515479+0000","last_clean":"2026-02-24T22:33:51.515479+0000","last_became_active":"2026-02-24T22:05:16.189713+0000","last_became_peered":"2026-02-24T22:05:16.189713+0000","last_unstale":"2026-02-24T22:33:51.515479+0000","last_undegraded":"2026-02-24T22:33:51.515479+0000","last_fullsized":"2026-02-24T22:33:51.515479+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":200,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:32:51.972858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10823593,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":564,"num_read_kb":31408,"num_write":1387,"num_write_kb":10638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"312'256","reported_seq":941,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:59.744014+0000","last_change":"2026-02-24T22:05:18.187784+0000","last_active":"2026-02-24T22:31:59.744014+0000","last_peered":"2026-02-24T22:31:59.744014+0000","last_clean":"2026-02-24T22:31:59.744014+0000","last_became_active":"2026-02-24T22:05:18.187565+0000","last_became_peered":"2026-02-24T22:05:18.187565+0000","last_unstale":"2026-02-24T22:31:59.744014+0000","last_undegraded":"2026-02-24T22:31:59.744014+0000","last_fullsized":"2026-02-24T22:31:59.744014+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":200,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:43:19.977135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":1060,"num_read_kb":853,"num_write":419,"num_write_kb":222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"312'225","reported_seq":931,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.723653+0000","last_change":"2026-02-24T22:05:16.178897+0000","last_active":"2026-02-24T22:30:59.723653+0000","last_peered":"2026-02-24T22:30:59.723653+0000","last_clean":"2026-02-24T22:30:59.723653+0000","last_became_active":"2026-02-24T22:05:16.178157+0000","last_became_peered":"2026-02-24T22:05:16.178157+0000","last_unstale":"2026-02-24T22:30:59.723653+0000","last_undegraded":"2026-02-24T22:30:59.723653+0000","last_fullsized":"2026-02-24T22:30:59.723653+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":200,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:50:48.164748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":621009,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":397,"num_read_kb":1640,"num_write":1065,"num_write_kb":1702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"312'78","reported_seq":752,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:51.565638+0000","last_change":"2026-02-24T22:05:18.191188+0000","last_active":"2026-02-24T22:33:51.565638+0000","last_peered":"2026-02-24T22:33:51.565638+0000","last_clean":"2026-02-24T22:33:51.565638+0000","last_became_active":"2026-02-24T22:05:18.190902+0000","last_became_peered":"2026-02-24T22:05:18.190902+0000","last_unstale":"2026-02-24T22:33:51.565638+0000","last_undegraded":"2026-02-24T22:33:51.565638+0000","last_fullsized":"2026-02-24T22:33:51.565638+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:18:59.644388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":302,"num_read_kb":262,"num_write":98,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":388,"ondisk_log_size":388,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":973109,"num_objects":3053,"num_object_clones":0,"num_object_copies":6106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3053,"num_whiteouts":0,"num_read":19978,"num_read_kb":245567,"num_write":23762,"num_write_kb":8619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24911872,"data_stored":1946218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5786661,"internal_metadata":0},"log_size":2182,"ondisk_log_size":2182,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1738830,"num_objects":223,"num_object_clones":0,"num_object_copies":446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":223,"num_whiteouts":0,"num_read":44721,"num_read_kb":43148,"num_write":15259,"num_write_kb":13598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3874816,"data_stored":3174742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2823344,"internal_metadata":0},"log_size":2043,"ondisk_log_size":2043,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1896,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1442,"num_read_kb":1673,"num_write":22,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":82,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11005,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":1511,"num_object_clones":0,"num_object_copies":3022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1511,"num_whiteouts":0,"num_read":95919,"num_read_kb":81897,"num_write":34645,"num_write_kb":17078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1511,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17301930,"internal_metadata":0},"log_size":3916,"ondisk_log_size":3916,"up":128,"acting":128,"num_store_stats":8},{"poolid":81,"num_pg":32,"stat_sum":{"num_bytes":72351744,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":71,"num_read_kb":119808,"num_write":63,"num_write_kb":70656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":144703488,"data_stored":144703488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":375571674,"num_objects":4668,"num_object_clones":0,"num_object_copies":9336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4668,"num_whiteouts":0,"num_read":36656,"num_read_kb":763453,"num_write":81696,"num_write_kb":523059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":786833408,"data_stored":751143348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2729,"ondisk_log_size":2729,"up":128,"acting":128,"num_store_stats":8},{"poolid":80,"num_pg":32,"stat_sum":{"num_bytes":72351744,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":73,"num_read_kb":124928,"num_write":63,"num_write_kb":70656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":144703488,"data_stored":144703488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":79,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3567,"num_read_kb":2401,"num_write":1127,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":317720,"internal_metadata":0},"log_size":462,"ondisk_log_size":462,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575202,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442900,"kb_used_data":199364,"kb_used_omap":3696,"kb_used_meta":239823,"kb_avail":93928940,"statfs":{"total":96636764160,"available":96183234560,"internally_reserved":0,"allocated":204148736,"data_stored":194101579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3785637,"internal_metadata":245578843},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575202,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":325324,"kb_used_data":81788,"kb_used_omap":3571,"kb_used_meta":239948,"kb_avail":94046516,"statfs":{"total":96636764160,"available":96303632384,"internally_reserved":0,"allocated":83750912,"data_stored":75161771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3657488,"internal_metadata":245706992},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575202,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":276592,"kb_used_data":177504,"kb_used_omap":2746,"kb_used_meta":96325,"kb_avail":94095248,"statfs":{"total":96636764160,"available":96353533952,"internally_reserved":0,"allocated":181764096,"data_stored":173324874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2812671,"internal_metadata":98637057},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575202,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":215740,"kb_used_data":152684,"kb_used_omap":3095,"kb_used_meta":59944,"kb_avail":94156100,"statfs":{"total":96636764160,"available":96415846400,"internally_reserved":0,"allocated":156348416,"data_stored":147731024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3169290,"internal_metadata":61383670},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575203,"num_pgs":107,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":329208,"kb_used_data":121768,"kb_used_omap":7232,"kb_used_meta":200191,"kb_avail":94042632,"statfs":{"total":96636764160,"available":96299655168,"internally_reserved":0,"allocated":124690432,"data_stored":113814900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7406186,"internal_metadata":204995990},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575203,"num_pgs":87,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":162392,"kb_used_data":117384,"kb_used_omap":1599,"kb_used_meta":43392,"kb_avail":94209448,"statfs":{"total":96636764160,"available":96470474752,"internally_reserved":0,"allocated":120201216,"data_stored":112600299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1638118,"internal_metadata":44433690},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575204,"num_pgs":102,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":178812,"kb_used_data":79660,"kb_used_omap":5094,"kb_used_meta":94041,"kb_avail":94193028,"statfs":{"total":96636764160,"available":96453660672,"internally_reserved":0,"allocated":81571840,"data_stored":73974457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5216565,"internal_metadata":96298699},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575204,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":408384,"kb_used_data":182960,"kb_used_omap":10223,"kb_used_meta":215184,"kb_avail":93963456,"statfs":{"total":96636764160,"available":96218578944,"internally_reserved":0,"allocated":187351040,"data_stored":176579114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10468372,"internal_metadata":220349420},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":129925120,"data_stored":124511737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":51691520,"data_stored":48716374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69758976,"data_stored":66666572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":88662016,"data_stored":83153600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":142684160,"data_stored":137577305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":122421248,"data_stored":117533084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":59645952,"data_stored":56294670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":122044416,"data_stored":116690006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1667351,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2633902,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1516609,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1844562,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2430346,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":723382,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3546560,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2939218,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":378928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1512,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":442706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1072852,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":148539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":667648,"data_stored":537960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18545,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":107453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5184,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":770048,"data_stored":659074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1705285,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":286394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11530,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":720896,"data_stored":613688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8436,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3928064,"data_stored":308322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51420,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3117056,"data_stored":241195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":152550,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3059712,"data_stored":239505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":825183,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3899392,"data_stored":305873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":336934,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1949696,"data_stored":151651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7338,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1925120,"data_stored":148450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":65762,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3837952,"data_stored":298046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2489145,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3194880,"data_stored":253176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1858329,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":79,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":133235,"internal_metadata":0},{"poolid":79,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14290,"internal_metadata":0},{"poolid":79,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23118,"internal_metadata":0},{"poolid":79,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15635,"internal_metadata":0},{"poolid":79,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49631,"internal_metadata":0},{"poolid":79,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49251,"internal_metadata":0},{"poolid":79,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16656,"internal_metadata":0},{"poolid":79,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15904,"internal_metadata":0},{"poolid":80,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26214400,"data_stored":26214400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16777216,"data_stored":16777216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":23068672,"data_stored":23068672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":23068672,"data_stored":23068672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":22020096,"data_stored":22020096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28311552,"data_stored":28311552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22020096,"data_stored":22020096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19922944,"data_stored":19922944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7340032,"data_stored":7340032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":30408704,"data_stored":30408704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45088768,"data_stored":45088768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":82,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":82,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1886,"internal_metadata":0},{"poolid":82,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6440,"internal_metadata":0},{"poolid":82,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":262,"internal_metadata":0},{"poolid":82,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1265,"internal_metadata":0},{"poolid":82,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":717,"internal_metadata":0},{"poolid":82,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":435,"internal_metadata":0},{"poolid":82,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T22:34:27.089 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-24T22:34:27.266 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:27.282 INFO:teuthology.orchestra.run.trial046.stderr:dumped all 2026-02-24T22:34:27.285 INFO:teuthology.orchestra.run.trial046.stdout:{"pg_ready":true,"pg_map":{"version":1186,"stamp":"2026-02-24T22:34:26.023106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":523579384,"num_objects":9510,"num_object_clones":0,"num_object_copies":19020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9510,"num_whiteouts":0,"num_read":202525,"num_read_kb":1382960,"num_write":156802,"num_write_kb":706681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":1543,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11819,"ondisk_log_size":11819,"up":786,"acting":786,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":786,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2339352,"kb_used_data":1113112,"kb_used_omap":37260,"kb_used_meta":1188851,"kb_avail":752635368,"statfs":{"total":773094113280,"available":770698616832,"internally_reserved":0,"allocated":1139826688,"data_stored":1067288018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38154327,"internal_metadata":1217384361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30542903,"num_objects":-41,"num_object_clones":0,"num_object_copies":-82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-41,"num_whiteouts":0,"num_read":1479,"num_read_kb":60619,"num_write":280,"num_write_kb":29926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003202"},"pg_stats":[{"pgid":"82.2","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744748+0000","last_change":"2026-02-24T22:23:29.198729+0000","last_active":"2026-02-24T22:31:49.744748+0000","last_peered":"2026-02-24T22:31:49.744748+0000","last_clean":"2026-02-24T22:31:49.744748+0000","last_became_active":"2026-02-24T22:23:29.198476+0000","last_became_peered":"2026-02-24T22:23:29.198476+0000","last_unstale":"2026-02-24T22:31:49.744748+0000","last_undegraded":"2026-02-24T22:31:49.744748+0000","last_fullsized":"2026-02-24T22:31:49.744748+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:12:35.165001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.1","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.730924+0000","last_change":"2026-02-24T22:15:25.830397+0000","last_active":"2026-02-24T22:32:09.730924+0000","last_peered":"2026-02-24T22:32:09.730924+0000","last_clean":"2026-02-24T22:32:09.730924+0000","last_became_active":"2026-02-24T22:15:25.830000+0000","last_became_peered":"2026-02-24T22:15:25.830000+0000","last_unstale":"2026-02-24T22:32:09.730924+0000","last_undegraded":"2026-02-24T22:32:09.730924+0000","last_fullsized":"2026-02-24T22:32:09.730924+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:41:30.400463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.0","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:29.723258+0000","last_change":"2026-02-24T22:15:22.817921+0000","last_active":"2026-02-24T22:31:29.723258+0000","last_peered":"2026-02-24T22:31:29.723258+0000","last_clean":"2026-02-24T22:31:29.723258+0000","last_became_active":"2026-02-24T22:15:22.817446+0000","last_became_peered":"2026-02-24T22:15:22.817446+0000","last_unstale":"2026-02-24T22:31:29.723258+0000","last_undegraded":"2026-02-24T22:31:29.723258+0000","last_fullsized":"2026-02-24T22:31:29.723258+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:25:30.206672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.1f","version":"312'20","reported_seq":120,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.733669+0000","last_change":"2026-02-24T22:13:43.761604+0000","last_active":"2026-02-24T22:30:24.733669+0000","last_peered":"2026-02-24T22:30:24.733669+0000","last_clean":"2026-02-24T22:30:24.733669+0000","last_became_active":"2026-02-24T22:13:43.761120+0000","last_became_peered":"2026-02-24T22:13:43.761120+0000","last_unstale":"2026-02-24T22:30:24.733669+0000","last_undegraded":"2026-02-24T22:30:24.733669+0000","last_fullsized":"2026-02-24T22:30:24.733669+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:48:55.248638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":168,"num_read_kb":114,"num_write":47,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.3","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742865+0000","last_change":"2026-02-24T22:23:29.213779+0000","last_active":"2026-02-24T22:31:49.742865+0000","last_peered":"2026-02-24T22:31:49.742865+0000","last_clean":"2026-02-24T22:31:49.742865+0000","last_became_active":"2026-02-24T22:23:29.213534+0000","last_became_peered":"2026-02-24T22:23:29.213534+0000","last_unstale":"2026-02-24T22:31:49.742865+0000","last_undegraded":"2026-02-24T22:31:49.742865+0000","last_fullsized":"2026-02-24T22:31:49.742865+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:38:56.711711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.0","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.743144+0000","last_change":"2026-02-24T22:15:25.827702+0000","last_active":"2026-02-24T22:31:34.743144+0000","last_peered":"2026-02-24T22:31:34.743144+0000","last_clean":"2026-02-24T22:31:34.743144+0000","last_became_active":"2026-02-24T22:15:25.827355+0000","last_became_peered":"2026-02-24T22:15:25.827355+0000","last_unstale":"2026-02-24T22:31:34.743144+0000","last_undegraded":"2026-02-24T22:31:34.743144+0000","last_fullsized":"2026-02-24T22:31:34.743144+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:22:12.090995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.1","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:04.744507+0000","last_change":"2026-02-24T22:15:22.818757+0000","last_active":"2026-02-24T22:32:04.744507+0000","last_peered":"2026-02-24T22:32:04.744507+0000","last_clean":"2026-02-24T22:32:04.744507+0000","last_became_active":"2026-02-24T22:15:22.818519+0000","last_became_peered":"2026-02-24T22:15:22.818519+0000","last_unstale":"2026-02-24T22:32:04.744507+0000","last_undegraded":"2026-02-24T22:32:04.744507+0000","last_fullsized":"2026-02-24T22:32:04.744507+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:17:56.378157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.1e","version":"312'9","reported_seq":88,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.745172+0000","last_change":"2026-02-24T22:13:43.757760+0000","last_active":"2026-02-24T22:31:49.745172+0000","last_peered":"2026-02-24T22:31:49.745172+0000","last_clean":"2026-02-24T22:31:49.745172+0000","last_became_active":"2026-02-24T22:13:43.757543+0000","last_became_peered":"2026-02-24T22:13:43.757543+0000","last_unstale":"2026-02-24T22:31:49.745172+0000","last_undegraded":"2026-02-24T22:31:49.745172+0000","last_fullsized":"2026-02-24T22:31:49.745172+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:44:23.087405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":38,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.0","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.723743+0000","last_change":"2026-02-24T22:23:29.212801+0000","last_active":"2026-02-24T22:31:34.723743+0000","last_peered":"2026-02-24T22:31:34.723743+0000","last_clean":"2026-02-24T22:31:34.723743+0000","last_became_active":"2026-02-24T22:23:29.212043+0000","last_became_peered":"2026-02-24T22:23:29.212043+0000","last_unstale":"2026-02-24T22:31:34.723743+0000","last_undegraded":"2026-02-24T22:31:34.723743+0000","last_fullsized":"2026-02-24T22:31:34.723743+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:45:48.953242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.3","version":"307'5","reported_seq":50,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.747921+0000","last_change":"2026-02-24T22:15:25.827590+0000","last_active":"2026-02-24T22:32:09.747921+0000","last_peered":"2026-02-24T22:32:09.747921+0000","last_clean":"2026-02-24T22:32:09.747921+0000","last_became_active":"2026-02-24T22:15:25.827216+0000","last_became_peered":"2026-02-24T22:15:25.827216+0000","last_unstale":"2026-02-24T22:32:09.747921+0000","last_undegraded":"2026-02-24T22:32:09.747921+0000","last_fullsized":"2026-02-24T22:32:09.747921+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:08:31.068519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485760,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":25600,"num_write":10,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.2","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:04.726927+0000","last_change":"2026-02-24T22:15:22.818494+0000","last_active":"2026-02-24T22:32:04.726927+0000","last_peered":"2026-02-24T22:32:04.726927+0000","last_clean":"2026-02-24T22:32:04.726927+0000","last_became_active":"2026-02-24T22:15:22.818194+0000","last_became_peered":"2026-02-24T22:15:22.818194+0000","last_unstale":"2026-02-24T22:32:04.726927+0000","last_undegraded":"2026-02-24T22:32:04.726927+0000","last_fullsized":"2026-02-24T22:32:04.726927+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:30:45.313454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.1d","version":"312'22","reported_seq":121,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.737198+0000","last_change":"2026-02-24T22:13:43.757707+0000","last_active":"2026-02-24T22:31:14.737198+0000","last_peered":"2026-02-24T22:31:14.737198+0000","last_clean":"2026-02-24T22:31:14.737198+0000","last_became_active":"2026-02-24T22:13:43.757278+0000","last_became_peered":"2026-02-24T22:13:43.757278+0000","last_unstale":"2026-02-24T22:31:14.737198+0000","last_undegraded":"2026-02-24T22:31:14.737198+0000","last_fullsized":"2026-02-24T22:31:14.737198+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:52:33.539660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":103,"num_write":54,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.1","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742841+0000","last_change":"2026-02-24T22:23:29.214295+0000","last_active":"2026-02-24T22:31:49.742841+0000","last_peered":"2026-02-24T22:31:49.742841+0000","last_clean":"2026-02-24T22:31:49.742841+0000","last_became_active":"2026-02-24T22:23:29.214064+0000","last_became_peered":"2026-02-24T22:23:29.214064+0000","last_unstale":"2026-02-24T22:31:49.742841+0000","last_undegraded":"2026-02-24T22:31:49.742841+0000","last_fullsized":"2026-02-24T22:31:49.742841+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:03:04.365969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.2","version":"307'1","reported_seq":35,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.735734+0000","last_change":"2026-02-24T22:15:25.833367+0000","last_active":"2026-02-24T22:32:09.735734+0000","last_peered":"2026-02-24T22:32:09.735734+0000","last_clean":"2026-02-24T22:32:09.735734+0000","last_became_active":"2026-02-24T22:15:25.832809+0000","last_became_peered":"2026-02-24T22:15:25.832809+0000","last_unstale":"2026-02-24T22:32:09.735734+0000","last_undegraded":"2026-02-24T22:32:09.735734+0000","last_fullsized":"2026-02-24T22:32:09.735734+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:28:13.551199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":4096,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"80.3","version":"307'2","reported_seq":43,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:04.726940+0000","last_change":"2026-02-24T22:15:22.818401+0000","last_active":"2026-02-24T22:32:04.726940+0000","last_peered":"2026-02-24T22:32:04.726940+0000","last_clean":"2026-02-24T22:32:04.726940+0000","last_became_active":"2026-02-24T22:15:22.818089+0000","last_became_peered":"2026-02-24T22:15:22.818089+0000","last_unstale":"2026-02-24T22:32:04.726940+0000","last_undegraded":"2026-02-24T22:32:04.726940+0000","last_fullsized":"2026-02-24T22:32:04.726940+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:40:25.179819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.1c","version":"312'14","reported_seq":101,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.739030+0000","last_change":"2026-02-24T22:13:43.760682+0000","last_active":"2026-02-24T22:31:14.739030+0000","last_peered":"2026-02-24T22:31:14.739030+0000","last_clean":"2026-02-24T22:31:14.739030+0000","last_became_active":"2026-02-24T22:13:43.759958+0000","last_became_peered":"2026-02-24T22:13:43.759958+0000","last_unstale":"2026-02-24T22:31:14.739030+0000","last_undegraded":"2026-02-24T22:31:14.739030+0000","last_fullsized":"2026-02-24T22:31:14.739030+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:24:39.289428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":75,"num_write":36,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.6","version":"309'1","reported_seq":23,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.730973+0000","last_change":"2026-02-24T22:23:29.215576+0000","last_active":"2026-02-24T22:32:09.730973+0000","last_peered":"2026-02-24T22:32:09.730973+0000","last_clean":"2026-02-24T22:32:09.730973+0000","last_became_active":"2026-02-24T22:23:29.215327+0000","last_became_peered":"2026-02-24T22:23:29.215327+0000","last_unstale":"2026-02-24T22:32:09.730973+0000","last_undegraded":"2026-02-24T22:32:09.730973+0000","last_fullsized":"2026-02-24T22:32:09.730973+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:51:01.281339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.5","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:14.745596+0000","last_change":"2026-02-24T22:15:25.832444+0000","last_active":"2026-02-24T22:32:14.745596+0000","last_peered":"2026-02-24T22:32:14.745596+0000","last_clean":"2026-02-24T22:32:14.745596+0000","last_became_active":"2026-02-24T22:15:25.831975+0000","last_became_peered":"2026-02-24T22:15:25.831975+0000","last_unstale":"2026-02-24T22:32:14.745596+0000","last_undegraded":"2026-02-24T22:32:14.745596+0000","last_fullsized":"2026-02-24T22:32:14.745596+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:04:17.334836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"80.4","version":"307'2","reported_seq":43,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:04.747123+0000","last_change":"2026-02-24T22:15:22.812647+0000","last_active":"2026-02-24T22:32:04.747123+0000","last_peered":"2026-02-24T22:32:04.747123+0000","last_clean":"2026-02-24T22:32:04.747123+0000","last_became_active":"2026-02-24T22:15:22.812270+0000","last_became_peered":"2026-02-24T22:15:22.812270+0000","last_unstale":"2026-02-24T22:32:04.747123+0000","last_undegraded":"2026-02-24T22:32:04.747123+0000","last_fullsized":"2026-02-24T22:32:04.747123+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:42:27.774508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.1b","version":"312'18","reported_seq":112,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.741475+0000","last_change":"2026-02-24T22:13:43.755424+0000","last_active":"2026-02-24T22:31:14.741475+0000","last_peered":"2026-02-24T22:31:14.741475+0000","last_clean":"2026-02-24T22:31:14.741475+0000","last_became_active":"2026-02-24T22:13:43.755170+0000","last_became_peered":"2026-02-24T22:13:43.755170+0000","last_unstale":"2026-02-24T22:31:14.741475+0000","last_undegraded":"2026-02-24T22:31:14.741475+0000","last_fullsized":"2026-02-24T22:31:14.741475+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:29:17.090743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":101,"num_write":42,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.7","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.735768+0000","last_change":"2026-02-24T22:23:29.214459+0000","last_active":"2026-02-24T22:32:09.735768+0000","last_peered":"2026-02-24T22:32:09.735768+0000","last_clean":"2026-02-24T22:32:09.735768+0000","last_became_active":"2026-02-24T22:23:29.214105+0000","last_became_peered":"2026-02-24T22:23:29.214105+0000","last_unstale":"2026-02-24T22:32:09.735768+0000","last_undegraded":"2026-02-24T22:32:09.735768+0000","last_fullsized":"2026-02-24T22:32:09.735768+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:00:10.302637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.4","version":"307'2","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.747275+0000","last_change":"2026-02-24T22:15:25.827716+0000","last_active":"2026-02-24T22:32:09.747275+0000","last_peered":"2026-02-24T22:32:09.747275+0000","last_clean":"2026-02-24T22:32:09.747275+0000","last_became_active":"2026-02-24T22:15:25.826663+0000","last_became_peered":"2026-02-24T22:15:25.826663+0000","last_unstale":"2026-02-24T22:32:09.747275+0000","last_undegraded":"2026-02-24T22:32:09.747275+0000","last_fullsized":"2026-02-24T22:32:09.747275+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:08:34.544962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.5","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.730960+0000","last_change":"2026-02-24T22:15:22.816880+0000","last_active":"2026-02-24T22:32:09.730960+0000","last_peered":"2026-02-24T22:32:09.730960+0000","last_clean":"2026-02-24T22:32:09.730960+0000","last_became_active":"2026-02-24T22:15:22.816697+0000","last_became_peered":"2026-02-24T22:15:22.816697+0000","last_unstale":"2026-02-24T22:32:09.730960+0000","last_undegraded":"2026-02-24T22:32:09.730960+0000","last_fullsized":"2026-02-24T22:32:09.730960+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:56:01.670361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"79.1a","version":"312'16","reported_seq":102,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.737175+0000","last_change":"2026-02-24T22:13:43.757739+0000","last_active":"2026-02-24T22:31:14.737175+0000","last_peered":"2026-02-24T22:31:14.737175+0000","last_clean":"2026-02-24T22:31:14.737175+0000","last_became_active":"2026-02-24T22:13:43.757353+0000","last_became_peered":"2026-02-24T22:13:43.757353+0000","last_unstale":"2026-02-24T22:31:14.737175+0000","last_undegraded":"2026-02-24T22:31:14.737175+0000","last_fullsized":"2026-02-24T22:31:14.737175+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:47:30.725736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":65,"num_write":38,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.4","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742881+0000","last_change":"2026-02-24T22:23:29.215206+0000","last_active":"2026-02-24T22:31:49.742881+0000","last_peered":"2026-02-24T22:31:49.742881+0000","last_clean":"2026-02-24T22:31:49.742881+0000","last_became_active":"2026-02-24T22:23:29.215014+0000","last_became_peered":"2026-02-24T22:23:29.215014+0000","last_unstale":"2026-02-24T22:31:49.742881+0000","last_undegraded":"2026-02-24T22:31:49.742881+0000","last_fullsized":"2026-02-24T22:31:49.742881+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:43:42.586588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.7","version":"307'2","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.718321+0000","last_change":"2026-02-24T22:15:25.832214+0000","last_active":"2026-02-24T22:30:39.718321+0000","last_peered":"2026-02-24T22:30:39.718321+0000","last_clean":"2026-02-24T22:30:39.718321+0000","last_became_active":"2026-02-24T22:15:25.831806+0000","last_became_peered":"2026-02-24T22:15:25.831806+0000","last_unstale":"2026-02-24T22:30:39.718321+0000","last_undegraded":"2026-02-24T22:30:39.718321+0000","last_fullsized":"2026-02-24T22:30:39.718321+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:36:56.341773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"80.6","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:34.736157+0000","last_change":"2026-02-24T22:15:22.811674+0000","last_active":"2026-02-24T22:30:34.736157+0000","last_peered":"2026-02-24T22:30:34.736157+0000","last_clean":"2026-02-24T22:30:34.736157+0000","last_became_active":"2026-02-24T22:15:22.811413+0000","last_became_peered":"2026-02-24T22:15:22.811413+0000","last_unstale":"2026-02-24T22:30:34.736157+0000","last_undegraded":"2026-02-24T22:30:34.736157+0000","last_fullsized":"2026-02-24T22:30:34.736157+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:14:20.030861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"79.19","version":"312'30","reported_seq":140,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.738215+0000","last_change":"2026-02-24T22:13:43.756439+0000","last_active":"2026-02-24T22:30:44.738215+0000","last_peered":"2026-02-24T22:30:44.738215+0000","last_clean":"2026-02-24T22:30:44.738215+0000","last_became_active":"2026-02-24T22:13:43.756114+0000","last_became_peered":"2026-02-24T22:13:43.756114+0000","last_unstale":"2026-02-24T22:30:44.738215+0000","last_undegraded":"2026-02-24T22:30:44.738215+0000","last_fullsized":"2026-02-24T22:30:44.738215+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:45:46.139575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":216,"num_read_kb":143,"num_write":68,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.5","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742892+0000","last_change":"2026-02-24T22:23:29.213537+0000","last_active":"2026-02-24T22:31:49.742892+0000","last_peered":"2026-02-24T22:31:49.742892+0000","last_clean":"2026-02-24T22:31:49.742892+0000","last_became_active":"2026-02-24T22:23:29.213256+0000","last_became_peered":"2026-02-24T22:23:29.213256+0000","last_unstale":"2026-02-24T22:31:49.742892+0000","last_undegraded":"2026-02-24T22:31:49.742892+0000","last_fullsized":"2026-02-24T22:31:49.742892+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:48:26.724830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.6","version":"307'2","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.721669+0000","last_change":"2026-02-24T22:15:25.833922+0000","last_active":"2026-02-24T22:30:39.721669+0000","last_peered":"2026-02-24T22:30:39.721669+0000","last_clean":"2026-02-24T22:30:39.721669+0000","last_became_active":"2026-02-24T22:15:25.833680+0000","last_became_peered":"2026-02-24T22:15:25.833680+0000","last_unstale":"2026-02-24T22:30:39.721669+0000","last_undegraded":"2026-02-24T22:30:39.721669+0000","last_fullsized":"2026-02-24T22:30:39.721669+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:26:51.677018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.7","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:34.736181+0000","last_change":"2026-02-24T22:15:22.812054+0000","last_active":"2026-02-24T22:30:34.736181+0000","last_peered":"2026-02-24T22:30:34.736181+0000","last_clean":"2026-02-24T22:30:34.736181+0000","last_became_active":"2026-02-24T22:15:22.811618+0000","last_became_peered":"2026-02-24T22:15:22.811618+0000","last_unstale":"2026-02-24T22:30:34.736181+0000","last_undegraded":"2026-02-24T22:30:34.736181+0000","last_fullsized":"2026-02-24T22:30:34.736181+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:37:58.903441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"79.18","version":"312'12","reported_seq":95,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.737670+0000","last_change":"2026-02-24T22:13:43.755501+0000","last_active":"2026-02-24T22:30:39.737670+0000","last_peered":"2026-02-24T22:30:39.737670+0000","last_clean":"2026-02-24T22:30:39.737670+0000","last_became_active":"2026-02-24T22:13:43.754981+0000","last_became_peered":"2026-02-24T22:13:43.754981+0000","last_unstale":"2026-02-24T22:30:39.737670+0000","last_undegraded":"2026-02-24T22:30:39.737670+0000","last_fullsized":"2026-02-24T22:30:39.737670+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:41:38.765379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":86,"num_read_kb":57,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.a","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.745052+0000","last_change":"2026-02-24T22:23:29.212877+0000","last_active":"2026-02-24T22:32:09.745052+0000","last_peered":"2026-02-24T22:32:09.745052+0000","last_clean":"2026-02-24T22:32:09.745052+0000","last_became_active":"2026-02-24T22:23:29.212213+0000","last_became_peered":"2026-02-24T22:23:29.212213+0000","last_unstale":"2026-02-24T22:32:09.745052+0000","last_undegraded":"2026-02-24T22:32:09.745052+0000","last_fullsized":"2026-02-24T22:32:09.745052+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:29:28.605583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.9","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.736816+0000","last_change":"2026-02-24T22:15:25.826412+0000","last_active":"2026-02-24T22:30:39.736816+0000","last_peered":"2026-02-24T22:30:39.736816+0000","last_clean":"2026-02-24T22:30:39.736816+0000","last_became_active":"2026-02-24T22:15:25.826076+0000","last_became_peered":"2026-02-24T22:15:25.826076+0000","last_unstale":"2026-02-24T22:30:39.736816+0000","last_undegraded":"2026-02-24T22:30:39.736816+0000","last_fullsized":"2026-02-24T22:30:39.736816+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:00:25.763716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.8","version":"307'4","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:34.717818+0000","last_change":"2026-02-24T22:15:22.816346+0000","last_active":"2026-02-24T22:30:34.717818+0000","last_peered":"2026-02-24T22:30:34.717818+0000","last_clean":"2026-02-24T22:30:34.717818+0000","last_became_active":"2026-02-24T22:15:22.815960+0000","last_became_peered":"2026-02-24T22:15:22.815960+0000","last_unstale":"2026-02-24T22:30:34.717818+0000","last_undegraded":"2026-02-24T22:30:34.717818+0000","last_fullsized":"2026-02-24T22:30:34.717818+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:55:34.277066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":6144,"num_write":9,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.17","version":"312'34","reported_seq":169,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.721687+0000","last_change":"2026-02-24T22:14:00.749701+0000","last_active":"2026-02-24T22:30:39.721687+0000","last_peered":"2026-02-24T22:30:39.721687+0000","last_clean":"2026-02-24T22:30:39.721687+0000","last_became_active":"2026-02-24T22:14:00.748438+0000","last_became_peered":"2026-02-24T22:14:00.748438+0000","last_unstale":"2026-02-24T22:30:39.721687+0000","last_undegraded":"2026-02-24T22:30:39.721687+0000","last_fullsized":"2026-02-24T22:30:39.721687+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":293,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:21:49.815767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":298,"num_read_kb":211,"num_write":78,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.b","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.730451+0000","last_change":"2026-02-24T22:23:29.197335+0000","last_active":"2026-02-24T22:30:14.730451+0000","last_peered":"2026-02-24T22:30:14.730451+0000","last_clean":"2026-02-24T22:30:14.730451+0000","last_became_active":"2026-02-24T22:23:29.196790+0000","last_became_peered":"2026-02-24T22:23:29.196790+0000","last_unstale":"2026-02-24T22:30:14.730451+0000","last_undegraded":"2026-02-24T22:30:14.730451+0000","last_fullsized":"2026-02-24T22:30:14.730451+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:55:13.008271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.8","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.734884+0000","last_change":"2026-02-24T22:15:25.825567+0000","last_active":"2026-02-24T22:30:39.734884+0000","last_peered":"2026-02-24T22:30:39.734884+0000","last_clean":"2026-02-24T22:30:39.734884+0000","last_became_active":"2026-02-24T22:15:25.825315+0000","last_became_peered":"2026-02-24T22:15:25.825315+0000","last_unstale":"2026-02-24T22:30:39.734884+0000","last_undegraded":"2026-02-24T22:30:39.734884+0000","last_fullsized":"2026-02-24T22:30:39.734884+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:24:39.313916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.9","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:34.732583+0000","last_change":"2026-02-24T22:15:22.811365+0000","last_active":"2026-02-24T22:30:34.732583+0000","last_peered":"2026-02-24T22:30:34.732583+0000","last_clean":"2026-02-24T22:30:34.732583+0000","last_became_active":"2026-02-24T22:15:22.811141+0000","last_became_peered":"2026-02-24T22:15:22.811141+0000","last_unstale":"2026-02-24T22:30:34.732583+0000","last_undegraded":"2026-02-24T22:30:34.732583+0000","last_fullsized":"2026-02-24T22:30:34.732583+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:45:06.384015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"79.16","version":"0'0","reported_seq":63,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.733272+0000","last_change":"2026-02-24T22:13:43.756602+0000","last_active":"2026-02-24T22:30:39.733272+0000","last_peered":"2026-02-24T22:30:39.733272+0000","last_clean":"2026-02-24T22:30:39.733272+0000","last_became_active":"2026-02-24T22:13:43.755885+0000","last_became_peered":"2026-02-24T22:13:43.755885+0000","last_unstale":"2026-02-24T22:30:39.733272+0000","last_undegraded":"2026-02-24T22:30:39.733272+0000","last_fullsized":"2026-02-24T22:30:39.733272+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:40:07.171226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.8","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.744882+0000","last_change":"2026-02-24T22:23:29.199299+0000","last_active":"2026-02-24T22:32:09.744882+0000","last_peered":"2026-02-24T22:32:09.744882+0000","last_clean":"2026-02-24T22:32:09.744882+0000","last_became_active":"2026-02-24T22:23:29.198882+0000","last_became_peered":"2026-02-24T22:23:29.198882+0000","last_unstale":"2026-02-24T22:32:09.744882+0000","last_undegraded":"2026-02-24T22:32:09.744882+0000","last_fullsized":"2026-02-24T22:32:09.744882+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:59:03.245529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"81.b","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.741464+0000","last_change":"2026-02-24T22:15:25.827664+0000","last_active":"2026-02-24T22:31:14.741464+0000","last_peered":"2026-02-24T22:31:14.741464+0000","last_clean":"2026-02-24T22:31:14.741464+0000","last_became_active":"2026-02-24T22:15:25.827265+0000","last_became_peered":"2026-02-24T22:15:25.827265+0000","last_unstale":"2026-02-24T22:31:14.741464+0000","last_undegraded":"2026-02-24T22:31:14.741464+0000","last_fullsized":"2026-02-24T22:31:14.741464+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:17:29.198425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.a","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.735221+0000","last_change":"2026-02-24T22:15:22.817325+0000","last_active":"2026-02-24T22:30:39.735221+0000","last_peered":"2026-02-24T22:30:39.735221+0000","last_clean":"2026-02-24T22:30:39.735221+0000","last_became_active":"2026-02-24T22:15:22.816702+0000","last_became_peered":"2026-02-24T22:15:22.816702+0000","last_unstale":"2026-02-24T22:30:39.735221+0000","last_undegraded":"2026-02-24T22:30:39.735221+0000","last_fullsized":"2026-02-24T22:30:39.735221+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:48:16.071491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.15","version":"307'6","reported_seq":82,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.735250+0000","last_change":"2026-02-24T22:13:43.761758+0000","last_active":"2026-02-24T22:30:39.735250+0000","last_peered":"2026-02-24T22:30:39.735250+0000","last_clean":"2026-02-24T22:30:39.735250+0000","last_became_active":"2026-02-24T22:13:43.760892+0000","last_became_peered":"2026-02-24T22:13:43.760892+0000","last_unstale":"2026-02-24T22:30:39.735250+0000","last_undegraded":"2026-02-24T22:30:39.735250+0000","last_fullsized":"2026-02-24T22:30:39.735250+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:07:44.028035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":37,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.9","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.743246+0000","last_change":"2026-02-24T22:23:29.198300+0000","last_active":"2026-02-24T22:32:09.743246+0000","last_peered":"2026-02-24T22:32:09.743246+0000","last_clean":"2026-02-24T22:32:09.743246+0000","last_became_active":"2026-02-24T22:23:29.197357+0000","last_became_peered":"2026-02-24T22:23:29.197357+0000","last_unstale":"2026-02-24T22:32:09.743246+0000","last_undegraded":"2026-02-24T22:32:09.743246+0000","last_fullsized":"2026-02-24T22:32:09.743246+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:58:07.606981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.a","version":"307'2","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.738233+0000","last_change":"2026-02-24T22:15:25.827469+0000","last_active":"2026-02-24T22:30:44.738233+0000","last_peered":"2026-02-24T22:30:44.738233+0000","last_clean":"2026-02-24T22:30:44.738233+0000","last_became_active":"2026-02-24T22:15:25.827055+0000","last_became_peered":"2026-02-24T22:15:25.827055+0000","last_unstale":"2026-02-24T22:30:44.738233+0000","last_undegraded":"2026-02-24T22:30:44.738233+0000","last_fullsized":"2026-02-24T22:30:44.738233+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:35:15.420239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.b","version":"307'2","reported_seq":42,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.738494+0000","last_change":"2026-02-24T22:15:22.817894+0000","last_active":"2026-02-24T22:31:09.738494+0000","last_peered":"2026-02-24T22:31:09.738494+0000","last_clean":"2026-02-24T22:31:09.738494+0000","last_became_active":"2026-02-24T22:15:22.817498+0000","last_became_peered":"2026-02-24T22:15:22.817498+0000","last_unstale":"2026-02-24T22:31:09.738494+0000","last_undegraded":"2026-02-24T22:31:09.738494+0000","last_fullsized":"2026-02-24T22:31:09.738494+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:08:45.124818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.14","version":"312'10","reported_seq":89,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:09.718735+0000","last_change":"2026-02-24T22:13:43.762482+0000","last_active":"2026-02-24T22:30:09.718735+0000","last_peered":"2026-02-24T22:30:09.718735+0000","last_clean":"2026-02-24T22:30:09.718735+0000","last_became_active":"2026-02-24T22:13:43.762341+0000","last_became_peered":"2026-02-24T22:13:43.762341+0000","last_unstale":"2026-02-24T22:30:09.718735+0000","last_undegraded":"2026-02-24T22:30:09.718735+0000","last_fullsized":"2026-02-24T22:30:09.718735+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:44:09.549825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":46,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.e","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.724040+0000","last_change":"2026-02-24T22:23:29.215664+0000","last_active":"2026-02-24T22:30:14.724040+0000","last_peered":"2026-02-24T22:30:14.724040+0000","last_clean":"2026-02-24T22:30:14.724040+0000","last_became_active":"2026-02-24T22:23:29.215425+0000","last_became_peered":"2026-02-24T22:23:29.215425+0000","last_unstale":"2026-02-24T22:30:14.724040+0000","last_undegraded":"2026-02-24T22:30:14.724040+0000","last_fullsized":"2026-02-24T22:30:14.724040+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:44:31.538355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.d","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.741498+0000","last_change":"2026-02-24T22:15:25.829241+0000","last_active":"2026-02-24T22:31:14.741498+0000","last_peered":"2026-02-24T22:31:14.741498+0000","last_clean":"2026-02-24T22:31:14.741498+0000","last_became_active":"2026-02-24T22:15:25.829109+0000","last_became_peered":"2026-02-24T22:15:25.829109+0000","last_unstale":"2026-02-24T22:31:14.741498+0000","last_undegraded":"2026-02-24T22:31:14.741498+0000","last_fullsized":"2026-02-24T22:31:14.741498+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:38:08.330741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.c","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.721310+0000","last_change":"2026-02-24T22:15:22.815624+0000","last_active":"2026-02-24T22:31:09.721310+0000","last_peered":"2026-02-24T22:31:09.721310+0000","last_clean":"2026-02-24T22:31:09.721310+0000","last_became_active":"2026-02-24T22:15:22.815364+0000","last_became_peered":"2026-02-24T22:15:22.815364+0000","last_unstale":"2026-02-24T22:31:09.721310+0000","last_undegraded":"2026-02-24T22:31:09.721310+0000","last_fullsized":"2026-02-24T22:31:09.721310+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:21:36.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.13","version":"312'11","reported_seq":95,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.714998+0000","last_change":"2026-02-24T22:13:43.760988+0000","last_active":"2026-02-24T22:30:04.714998+0000","last_peered":"2026-02-24T22:30:04.714998+0000","last_clean":"2026-02-24T22:30:04.714998+0000","last_became_active":"2026-02-24T22:13:43.760714+0000","last_became_peered":"2026-02-24T22:13:43.760714+0000","last_unstale":"2026-02-24T22:30:04.714998+0000","last_undegraded":"2026-02-24T22:30:04.714998+0000","last_fullsized":"2026-02-24T22:30:04.714998+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:55:23.837996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":61,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"82.f","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.730477+0000","last_change":"2026-02-24T22:23:29.199754+0000","last_active":"2026-02-24T22:30:14.730477+0000","last_peered":"2026-02-24T22:30:14.730477+0000","last_clean":"2026-02-24T22:30:14.730477+0000","last_became_active":"2026-02-24T22:23:29.199538+0000","last_became_peered":"2026-02-24T22:23:29.199538+0000","last_unstale":"2026-02-24T22:30:14.730477+0000","last_undegraded":"2026-02-24T22:30:14.730477+0000","last_fullsized":"2026-02-24T22:30:14.730477+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:15:25.304156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.c","version":"307'7","reported_seq":54,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.740681+0000","last_change":"2026-02-24T22:15:25.827267+0000","last_active":"2026-02-24T22:31:14.740681+0000","last_peered":"2026-02-24T22:31:14.740681+0000","last_clean":"2026-02-24T22:31:14.740681+0000","last_became_active":"2026-02-24T22:15:25.826430+0000","last_became_peered":"2026-02-24T22:15:25.826430+0000","last_unstale":"2026-02-24T22:31:14.740681+0000","last_undegraded":"2026-02-24T22:31:14.740681+0000","last_fullsized":"2026-02-24T22:31:14.740681+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:49:19.806352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":31744,"num_write":12,"num_write_kb":16384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.d","version":"307'3","reported_seq":50,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.740980+0000","last_change":"2026-02-24T22:15:22.813197+0000","last_active":"2026-02-24T22:31:09.740980+0000","last_peered":"2026-02-24T22:31:09.740980+0000","last_clean":"2026-02-24T22:31:09.740980+0000","last_became_active":"2026-02-24T22:15:22.813062+0000","last_became_peered":"2026-02-24T22:15:22.813062+0000","last_unstale":"2026-02-24T22:31:09.740980+0000","last_undegraded":"2026-02-24T22:31:09.740980+0000","last_fullsized":"2026-02-24T22:31:09.740980+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:08:36.468537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":20480,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.12","version":"312'20","reported_seq":117,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:48.279108+0000","last_change":"2026-02-24T22:13:43.760420+0000","last_active":"2026-02-24T22:30:48.279108+0000","last_peered":"2026-02-24T22:30:48.279108+0000","last_clean":"2026-02-24T22:30:48.279108+0000","last_became_active":"2026-02-24T22:13:43.760196+0000","last_became_peered":"2026-02-24T22:13:43.760196+0000","last_unstale":"2026-02-24T22:30:48.279108+0000","last_undegraded":"2026-02-24T22:30:48.279108+0000","last_fullsized":"2026-02-24T22:30:48.279108+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:14:15.856109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":106,"num_write":52,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"82.c","version":"310'1","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.734866+0000","last_change":"2026-02-24T22:23:29.201743+0000","last_active":"2026-02-24T22:30:14.734866+0000","last_peered":"2026-02-24T22:30:14.734866+0000","last_clean":"2026-02-24T22:30:14.734866+0000","last_became_active":"2026-02-24T22:23:29.201545+0000","last_became_peered":"2026-02-24T22:23:29.201545+0000","last_unstale":"2026-02-24T22:30:14.734866+0000","last_undegraded":"2026-02-24T22:30:14.734866+0000","last_fullsized":"2026-02-24T22:30:14.734866+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:07:57.527033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"81.f","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.741445+0000","last_change":"2026-02-24T22:15:25.825325+0000","last_active":"2026-02-24T22:31:19.741445+0000","last_peered":"2026-02-24T22:31:19.741445+0000","last_clean":"2026-02-24T22:31:19.741445+0000","last_became_active":"2026-02-24T22:15:25.824862+0000","last_became_peered":"2026-02-24T22:15:25.824862+0000","last_unstale":"2026-02-24T22:31:19.741445+0000","last_undegraded":"2026-02-24T22:31:19.741445+0000","last_fullsized":"2026-02-24T22:31:19.741445+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:16:51.320757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.e","version":"307'3","reported_seq":52,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.721326+0000","last_change":"2026-02-24T22:15:22.817941+0000","last_active":"2026-02-24T22:31:09.721326+0000","last_peered":"2026-02-24T22:31:09.721326+0000","last_clean":"2026-02-24T22:31:09.721326+0000","last_became_active":"2026-02-24T22:15:22.817295+0000","last_became_peered":"2026-02-24T22:15:22.817295+0000","last_unstale":"2026-02-24T22:31:09.721326+0000","last_undegraded":"2026-02-24T22:31:09.721326+0000","last_fullsized":"2026-02-24T22:31:09.721326+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:37:42.904886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":25600,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.11","version":"312'11","reported_seq":92,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.731364+0000","last_change":"2026-02-24T22:13:43.760497+0000","last_active":"2026-02-24T22:30:04.731364+0000","last_peered":"2026-02-24T22:30:04.731364+0000","last_clean":"2026-02-24T22:30:04.731364+0000","last_became_active":"2026-02-24T22:13:43.759864+0000","last_became_peered":"2026-02-24T22:13:43.759864+0000","last_unstale":"2026-02-24T22:30:04.731364+0000","last_undegraded":"2026-02-24T22:30:04.731364+0000","last_fullsized":"2026-02-24T22:30:04.731364+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:50:51.585161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":53,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.d","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.724028+0000","last_change":"2026-02-24T22:23:29.214699+0000","last_active":"2026-02-24T22:30:14.724028+0000","last_peered":"2026-02-24T22:30:14.724028+0000","last_clean":"2026-02-24T22:30:14.724028+0000","last_became_active":"2026-02-24T22:23:29.214432+0000","last_became_peered":"2026-02-24T22:23:29.214432+0000","last_unstale":"2026-02-24T22:30:14.724028+0000","last_undegraded":"2026-02-24T22:30:14.724028+0000","last_fullsized":"2026-02-24T22:30:14.724028+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:09:09.120137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.e","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.738719+0000","last_change":"2026-02-24T22:15:25.827425+0000","last_active":"2026-02-24T22:31:14.738719+0000","last_peered":"2026-02-24T22:31:14.738719+0000","last_clean":"2026-02-24T22:31:14.738719+0000","last_became_active":"2026-02-24T22:15:25.827110+0000","last_became_peered":"2026-02-24T22:15:25.827110+0000","last_unstale":"2026-02-24T22:31:14.738719+0000","last_undegraded":"2026-02-24T22:31:14.738719+0000","last_fullsized":"2026-02-24T22:31:14.738719+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:25:06.342633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.f","version":"307'5","reported_seq":55,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.739061+0000","last_change":"2026-02-24T22:15:22.816168+0000","last_active":"2026-02-24T22:31:14.739061+0000","last_peered":"2026-02-24T22:31:14.739061+0000","last_clean":"2026-02-24T22:31:14.739061+0000","last_became_active":"2026-02-24T22:15:22.815819+0000","last_became_peered":"2026-02-24T22:15:22.815819+0000","last_unstale":"2026-02-24T22:31:14.739061+0000","last_undegraded":"2026-02-24T22:31:14.739061+0000","last_fullsized":"2026-02-24T22:31:14.739061+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:39:39.503227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11534336,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":26624,"num_write":10,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.10","version":"312'31","reported_seq":152,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.731343+0000","last_change":"2026-02-24T22:13:43.760545+0000","last_active":"2026-02-24T22:30:04.731343+0000","last_peered":"2026-02-24T22:30:04.731343+0000","last_clean":"2026-02-24T22:30:04.731343+0000","last_became_active":"2026-02-24T22:13:43.759855+0000","last_became_peered":"2026-02-24T22:13:43.759855+0000","last_unstale":"2026-02-24T22:30:04.731343+0000","last_undegraded":"2026-02-24T22:30:04.731343+0000","last_fullsized":"2026-02-24T22:30:04.731343+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:57:17.855796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":272,"num_read_kb":204,"num_write":72,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.12","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.725516+0000","last_change":"2026-02-24T22:23:29.212745+0000","last_active":"2026-02-24T22:30:29.725516+0000","last_peered":"2026-02-24T22:30:29.725516+0000","last_clean":"2026-02-24T22:30:29.725516+0000","last_became_active":"2026-02-24T22:23:29.212534+0000","last_became_peered":"2026-02-24T22:23:29.212534+0000","last_unstale":"2026-02-24T22:30:29.725516+0000","last_undegraded":"2026-02-24T22:30:29.725516+0000","last_fullsized":"2026-02-24T22:30:29.725516+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:50:08.072181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.11","version":"307'3","reported_seq":44,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744735+0000","last_change":"2026-02-24T22:15:25.827242+0000","last_active":"2026-02-24T22:31:49.744735+0000","last_peered":"2026-02-24T22:31:49.744735+0000","last_clean":"2026-02-24T22:31:49.744735+0000","last_became_active":"2026-02-24T22:15:25.826517+0000","last_became_peered":"2026-02-24T22:15:25.826517+0000","last_unstale":"2026-02-24T22:31:49.744735+0000","last_undegraded":"2026-02-24T22:31:49.744735+0000","last_fullsized":"2026-02-24T22:31:49.744735+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:24:40.732721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":20480,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.10","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.741867+0000","last_change":"2026-02-24T22:15:22.810414+0000","last_active":"2026-02-24T22:31:44.741867+0000","last_peered":"2026-02-24T22:31:44.741867+0000","last_clean":"2026-02-24T22:31:44.741867+0000","last_became_active":"2026-02-24T22:15:22.810000+0000","last_became_peered":"2026-02-24T22:15:22.810000+0000","last_unstale":"2026-02-24T22:31:44.741867+0000","last_undegraded":"2026-02-24T22:31:44.741867+0000","last_fullsized":"2026-02-24T22:31:44.741867+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:21:42.629513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"79.f","version":"312'11","reported_seq":95,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742413+0000","last_change":"2026-02-24T22:13:43.755156+0000","last_active":"2026-02-24T22:31:49.742413+0000","last_peered":"2026-02-24T22:31:49.742413+0000","last_clean":"2026-02-24T22:31:49.742413+0000","last_became_active":"2026-02-24T22:13:43.754602+0000","last_became_peered":"2026-02-24T22:13:43.754602+0000","last_unstale":"2026-02-24T22:31:49.742413+0000","last_undegraded":"2026-02-24T22:31:49.742413+0000","last_fullsized":"2026-02-24T22:31:49.742413+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:13:53.006921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":67,"num_write":27,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.13","version":"310'1","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.735479+0000","last_change":"2026-02-24T22:23:29.198937+0000","last_active":"2026-02-24T22:30:29.735479+0000","last_peered":"2026-02-24T22:30:29.735479+0000","last_clean":"2026-02-24T22:30:29.735479+0000","last_became_active":"2026-02-24T22:23:29.198627+0000","last_became_peered":"2026-02-24T22:23:29.198627+0000","last_unstale":"2026-02-24T22:30:29.735479+0000","last_undegraded":"2026-02-24T22:30:29.735479+0000","last_fullsized":"2026-02-24T22:30:29.735479+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:28:29.061001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.10","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744722+0000","last_change":"2026-02-24T22:15:25.827252+0000","last_active":"2026-02-24T22:31:49.744722+0000","last_peered":"2026-02-24T22:31:49.744722+0000","last_clean":"2026-02-24T22:31:49.744722+0000","last_became_active":"2026-02-24T22:15:25.826606+0000","last_became_peered":"2026-02-24T22:15:25.826606+0000","last_unstale":"2026-02-24T22:31:49.744722+0000","last_undegraded":"2026-02-24T22:31:49.744722+0000","last_fullsized":"2026-02-24T22:31:49.744722+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:30:33.376941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.11","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.741885+0000","last_change":"2026-02-24T22:15:22.810459+0000","last_active":"2026-02-24T22:31:44.741885+0000","last_peered":"2026-02-24T22:31:44.741885+0000","last_clean":"2026-02-24T22:31:44.741885+0000","last_became_active":"2026-02-24T22:15:22.809791+0000","last_became_peered":"2026-02-24T22:15:22.809791+0000","last_unstale":"2026-02-24T22:31:44.741885+0000","last_undegraded":"2026-02-24T22:31:44.741885+0000","last_fullsized":"2026-02-24T22:31:44.741885+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:37:39.655379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"79.e","version":"312'23","reported_seq":126,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.741917+0000","last_change":"2026-02-24T22:13:43.756185+0000","last_active":"2026-02-24T22:31:44.741917+0000","last_peered":"2026-02-24T22:31:44.741917+0000","last_clean":"2026-02-24T22:31:44.741917+0000","last_became_active":"2026-02-24T22:13:43.755868+0000","last_became_peered":"2026-02-24T22:13:43.755868+0000","last_unstale":"2026-02-24T22:31:44.741917+0000","last_undegraded":"2026-02-24T22:31:44.741917+0000","last_fullsized":"2026-02-24T22:31:44.741917+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:56:04.041994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":169,"num_read_kb":112,"num_write":56,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.10","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.734148+0000","last_change":"2026-02-24T22:23:29.212895+0000","last_active":"2026-02-24T22:30:29.734148+0000","last_peered":"2026-02-24T22:30:29.734148+0000","last_clean":"2026-02-24T22:30:29.734148+0000","last_became_active":"2026-02-24T22:23:29.212665+0000","last_became_peered":"2026-02-24T22:23:29.212665+0000","last_unstale":"2026-02-24T22:30:29.734148+0000","last_undegraded":"2026-02-24T22:30:29.734148+0000","last_fullsized":"2026-02-24T22:30:29.734148+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:30:27.871463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.13","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.740862+0000","last_change":"2026-02-24T22:15:25.826171+0000","last_active":"2026-02-24T22:31:49.740862+0000","last_peered":"2026-02-24T22:31:49.740862+0000","last_clean":"2026-02-24T22:31:49.740862+0000","last_became_active":"2026-02-24T22:15:25.825862+0000","last_became_peered":"2026-02-24T22:15:25.825862+0000","last_unstale":"2026-02-24T22:31:49.740862+0000","last_undegraded":"2026-02-24T22:31:49.740862+0000","last_fullsized":"2026-02-24T22:31:49.740862+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:02:37.431216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"80.12","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.744628+0000","last_change":"2026-02-24T22:15:22.812712+0000","last_active":"2026-02-24T22:31:44.744628+0000","last_peered":"2026-02-24T22:31:44.744628+0000","last_clean":"2026-02-24T22:31:44.744628+0000","last_became_active":"2026-02-24T22:15:22.812405+0000","last_became_peered":"2026-02-24T22:15:22.812405+0000","last_unstale":"2026-02-24T22:31:44.744628+0000","last_undegraded":"2026-02-24T22:31:44.744628+0000","last_fullsized":"2026-02-24T22:31:44.744628+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:36:00.971366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.d","version":"312'20","reported_seq":113,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.733217+0000","last_change":"2026-02-24T22:13:43.761593+0000","last_active":"2026-02-24T22:31:44.733217+0000","last_peered":"2026-02-24T22:31:44.733217+0000","last_clean":"2026-02-24T22:31:44.733217+0000","last_became_active":"2026-02-24T22:13:43.761339+0000","last_became_peered":"2026-02-24T22:13:43.761339+0000","last_unstale":"2026-02-24T22:31:44.733217+0000","last_undegraded":"2026-02-24T22:31:44.733217+0000","last_fullsized":"2026-02-24T22:31:44.733217+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:02:40.622911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":126,"num_read_kb":76,"num_write":49,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"82.11","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.720712+0000","last_change":"2026-02-24T22:23:29.214793+0000","last_active":"2026-02-24T22:30:29.720712+0000","last_peered":"2026-02-24T22:30:29.720712+0000","last_clean":"2026-02-24T22:30:29.720712+0000","last_became_active":"2026-02-24T22:23:29.214167+0000","last_became_peered":"2026-02-24T22:23:29.214167+0000","last_unstale":"2026-02-24T22:30:29.720712+0000","last_undegraded":"2026-02-24T22:30:29.720712+0000","last_fullsized":"2026-02-24T22:30:29.720712+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:21:51.443877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.12","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744756+0000","last_change":"2026-02-24T22:15:25.826465+0000","last_active":"2026-02-24T22:31:49.744756+0000","last_peered":"2026-02-24T22:31:49.744756+0000","last_clean":"2026-02-24T22:31:49.744756+0000","last_became_active":"2026-02-24T22:15:25.826102+0000","last_became_peered":"2026-02-24T22:15:25.826102+0000","last_unstale":"2026-02-24T22:31:49.744756+0000","last_undegraded":"2026-02-24T22:31:49.744756+0000","last_fullsized":"2026-02-24T22:31:49.744756+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:22:01.000780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.13","version":"307'2","reported_seq":42,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.744644+0000","last_change":"2026-02-24T22:15:22.812393+0000","last_active":"2026-02-24T22:31:44.744644+0000","last_peered":"2026-02-24T22:31:44.744644+0000","last_clean":"2026-02-24T22:31:44.744644+0000","last_became_active":"2026-02-24T22:15:22.811927+0000","last_became_peered":"2026-02-24T22:15:22.811927+0000","last_unstale":"2026-02-24T22:31:44.744644+0000","last_undegraded":"2026-02-24T22:31:44.744644+0000","last_fullsized":"2026-02-24T22:31:44.744644+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:54:16.404989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.c","version":"312'4","reported_seq":74,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.724938+0000","last_change":"2026-02-24T22:13:43.762114+0000","last_active":"2026-02-24T22:31:44.724938+0000","last_peered":"2026-02-24T22:31:44.724938+0000","last_clean":"2026-02-24T22:31:44.724938+0000","last_became_active":"2026-02-24T22:13:43.761910+0000","last_became_peered":"2026-02-24T22:13:43.761910+0000","last_unstale":"2026-02-24T22:31:44.724938+0000","last_undegraded":"2026-02-24T22:31:44.724938+0000","last_fullsized":"2026-02-24T22:31:44.724938+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:51:11.166418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":15,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"82.16","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.718767+0000","last_change":"2026-02-24T22:23:29.213342+0000","last_active":"2026-02-24T22:30:44.718767+0000","last_peered":"2026-02-24T22:30:44.718767+0000","last_clean":"2026-02-24T22:30:44.718767+0000","last_became_active":"2026-02-24T22:23:29.213080+0000","last_became_peered":"2026-02-24T22:23:29.213080+0000","last_unstale":"2026-02-24T22:30:44.718767+0000","last_undegraded":"2026-02-24T22:30:44.718767+0000","last_fullsized":"2026-02-24T22:30:44.718767+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:33:29.031496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.15","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:59.714511+0000","last_change":"2026-02-24T22:15:25.831361+0000","last_active":"2026-02-24T22:29:59.714511+0000","last_peered":"2026-02-24T22:29:59.714511+0000","last_clean":"2026-02-24T22:29:59.714511+0000","last_became_active":"2026-02-24T22:15:25.830922+0000","last_became_peered":"2026-02-24T22:15:25.830922+0000","last_unstale":"2026-02-24T22:29:59.714511+0000","last_undegraded":"2026-02-24T22:29:59.714511+0000","last_fullsized":"2026-02-24T22:29:59.714511+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:08:07.538696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"80.14","version":"307'2","reported_seq":42,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.745149+0000","last_change":"2026-02-24T22:15:22.812446+0000","last_active":"2026-02-24T22:31:49.745149+0000","last_peered":"2026-02-24T22:31:49.745149+0000","last_clean":"2026-02-24T22:31:49.745149+0000","last_became_active":"2026-02-24T22:15:22.811924+0000","last_became_peered":"2026-02-24T22:15:22.811924+0000","last_unstale":"2026-02-24T22:31:49.745149+0000","last_undegraded":"2026-02-24T22:31:49.745149+0000","last_fullsized":"2026-02-24T22:31:49.745149+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:18:30.397097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.b","version":"312'13","reported_seq":98,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.740368+0000","last_change":"2026-02-24T22:13:43.757072+0000","last_active":"2026-02-24T22:31:44.740368+0000","last_peered":"2026-02-24T22:31:44.740368+0000","last_clean":"2026-02-24T22:31:44.740368+0000","last_became_active":"2026-02-24T22:13:43.756596+0000","last_became_peered":"2026-02-24T22:13:43.756596+0000","last_unstale":"2026-02-24T22:31:44.740368+0000","last_undegraded":"2026-02-24T22:31:44.740368+0000","last_fullsized":"2026-02-24T22:31:44.740368+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:30:51.803180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":56,"num_write":37,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.17","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.718784+0000","last_change":"2026-02-24T22:23:29.214456+0000","last_active":"2026-02-24T22:30:44.718784+0000","last_peered":"2026-02-24T22:30:44.718784+0000","last_clean":"2026-02-24T22:30:44.718784+0000","last_became_active":"2026-02-24T22:23:29.214217+0000","last_became_peered":"2026-02-24T22:23:29.214217+0000","last_unstale":"2026-02-24T22:30:44.718784+0000","last_undegraded":"2026-02-24T22:30:44.718784+0000","last_fullsized":"2026-02-24T22:30:44.718784+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:48:46.666871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.14","version":"307'3","reported_seq":40,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.745241+0000","last_change":"2026-02-24T22:15:25.827827+0000","last_active":"2026-02-24T22:31:54.745241+0000","last_peered":"2026-02-24T22:31:54.745241+0000","last_clean":"2026-02-24T22:31:54.745241+0000","last_became_active":"2026-02-24T22:15:25.827557+0000","last_became_peered":"2026-02-24T22:15:25.827557+0000","last_unstale":"2026-02-24T22:31:54.745241+0000","last_undegraded":"2026-02-24T22:31:54.745241+0000","last_fullsized":"2026-02-24T22:31:54.745241+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:13:31.380536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":4096,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.15","version":"307'3","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:54.732905+0000","last_change":"2026-02-24T22:15:22.812814+0000","last_active":"2026-02-24T22:29:54.732905+0000","last_peered":"2026-02-24T22:29:54.732905+0000","last_clean":"2026-02-24T22:29:54.732905+0000","last_became_active":"2026-02-24T22:15:22.812450+0000","last_became_peered":"2026-02-24T22:15:22.812450+0000","last_unstale":"2026-02-24T22:29:54.732905+0000","last_undegraded":"2026-02-24T22:29:54.732905+0000","last_fullsized":"2026-02-24T22:29:54.732905+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:22:29.140595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":4096,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.a","version":"312'10","reported_seq":90,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.724683+0000","last_change":"2026-02-24T22:13:43.762418+0000","last_active":"2026-02-24T22:31:09.724683+0000","last_peered":"2026-02-24T22:31:09.724683+0000","last_clean":"2026-02-24T22:31:09.724683+0000","last_became_active":"2026-02-24T22:13:43.762226+0000","last_became_peered":"2026-02-24T22:13:43.762226+0000","last_unstale":"2026-02-24T22:31:09.724683+0000","last_undegraded":"2026-02-24T22:31:09.724683+0000","last_fullsized":"2026-02-24T22:31:09.724683+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:41:14.108185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":42,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.14","version":"309'1","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.735499+0000","last_change":"2026-02-24T22:24:01.299030+0000","last_active":"2026-02-24T22:30:29.735499+0000","last_peered":"2026-02-24T22:30:29.735499+0000","last_clean":"2026-02-24T22:30:29.735499+0000","last_became_active":"2026-02-24T22:24:01.297193+0000","last_became_peered":"2026-02-24T22:24:01.297193+0000","last_unstale":"2026-02-24T22:30:29.735499+0000","last_undegraded":"2026-02-24T22:30:29.735499+0000","last_fullsized":"2026-02-24T22:30:29.735499+0000","mapping_epoch":311,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":312,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:15:53.769966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.17","version":"307'2","reported_seq":34,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:59.730863+0000","last_change":"2026-02-24T22:15:25.833669+0000","last_active":"2026-02-24T22:29:59.730863+0000","last_peered":"2026-02-24T22:29:59.730863+0000","last_clean":"2026-02-24T22:29:59.730863+0000","last_became_active":"2026-02-24T22:15:25.833430+0000","last_became_peered":"2026-02-24T22:15:25.833430+0000","last_unstale":"2026-02-24T22:29:59.730863+0000","last_undegraded":"2026-02-24T22:29:59.730863+0000","last_fullsized":"2026-02-24T22:29:59.730863+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:41:23.281878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1024,"num_write":7,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"80.16","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:54.714013+0000","last_change":"2026-02-24T22:15:22.817544+0000","last_active":"2026-02-24T22:29:54.714013+0000","last_peered":"2026-02-24T22:29:54.714013+0000","last_clean":"2026-02-24T22:29:54.714013+0000","last_became_active":"2026-02-24T22:15:22.817150+0000","last_became_peered":"2026-02-24T22:15:22.817150+0000","last_unstale":"2026-02-24T22:29:54.714013+0000","last_undegraded":"2026-02-24T22:29:54.714013+0000","last_fullsized":"2026-02-24T22:29:54.714013+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:30:27.018037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.9","version":"312'24","reported_seq":125,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.737699+0000","last_change":"2026-02-24T22:13:43.757041+0000","last_active":"2026-02-24T22:31:04.737699+0000","last_peered":"2026-02-24T22:31:04.737699+0000","last_clean":"2026-02-24T22:31:04.737699+0000","last_became_active":"2026-02-24T22:13:43.756777+0000","last_became_peered":"2026-02-24T22:13:43.756777+0000","last_unstale":"2026-02-24T22:31:04.737699+0000","last_undegraded":"2026-02-24T22:31:04.737699+0000","last_fullsized":"2026-02-24T22:31:04.737699+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:54:52.853203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":107,"num_write":60,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.15","version":"310'1","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.733776+0000","last_change":"2026-02-24T22:23:29.198331+0000","last_active":"2026-02-24T22:30:44.733776+0000","last_peered":"2026-02-24T22:30:44.733776+0000","last_clean":"2026-02-24T22:30:44.733776+0000","last_became_active":"2026-02-24T22:23:29.198000+0000","last_became_peered":"2026-02-24T22:23:29.198000+0000","last_unstale":"2026-02-24T22:30:44.733776+0000","last_undegraded":"2026-02-24T22:30:44.733776+0000","last_fullsized":"2026-02-24T22:30:44.733776+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:34:21.768724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.16","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:59.732575+0000","last_change":"2026-02-24T22:15:25.828675+0000","last_active":"2026-02-24T22:29:59.732575+0000","last_peered":"2026-02-24T22:29:59.732575+0000","last_clean":"2026-02-24T22:29:59.732575+0000","last_became_active":"2026-02-24T22:15:25.828066+0000","last_became_peered":"2026-02-24T22:15:25.828066+0000","last_unstale":"2026-02-24T22:29:59.732575+0000","last_undegraded":"2026-02-24T22:29:59.732575+0000","last_fullsized":"2026-02-24T22:29:59.732575+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:13:41.162339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.17","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:54.730332+0000","last_change":"2026-02-24T22:15:22.818517+0000","last_active":"2026-02-24T22:29:54.730332+0000","last_peered":"2026-02-24T22:29:54.730332+0000","last_clean":"2026-02-24T22:29:54.730332+0000","last_became_active":"2026-02-24T22:15:22.817950+0000","last_became_peered":"2026-02-24T22:15:22.817950+0000","last_unstale":"2026-02-24T22:29:54.730332+0000","last_undegraded":"2026-02-24T22:29:54.730332+0000","last_fullsized":"2026-02-24T22:29:54.730332+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:47:07.389545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.8","version":"312'26","reported_seq":136,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.737895+0000","last_change":"2026-02-24T22:13:43.761888+0000","last_active":"2026-02-24T22:31:04.737895+0000","last_peered":"2026-02-24T22:31:04.737895+0000","last_clean":"2026-02-24T22:31:04.737895+0000","last_became_active":"2026-02-24T22:13:43.761558+0000","last_became_peered":"2026-02-24T22:13:43.761558+0000","last_unstale":"2026-02-24T22:31:04.737895+0000","last_undegraded":"2026-02-24T22:31:04.737895+0000","last_fullsized":"2026-02-24T22:31:04.737895+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:22:52.093177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":152,"num_write":62,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.1a","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.723700+0000","last_change":"2026-02-24T22:23:29.213344+0000","last_active":"2026-02-24T22:30:59.723700+0000","last_peered":"2026-02-24T22:30:59.723700+0000","last_clean":"2026-02-24T22:30:59.723700+0000","last_became_active":"2026-02-24T22:23:29.212709+0000","last_became_peered":"2026-02-24T22:23:29.212709+0000","last_unstale":"2026-02-24T22:30:59.723700+0000","last_undegraded":"2026-02-24T22:30:59.723700+0000","last_fullsized":"2026-02-24T22:30:59.723700+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:26:30.998666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.19","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.718214+0000","last_change":"2026-02-24T22:15:25.832162+0000","last_active":"2026-02-24T22:30:04.718214+0000","last_peered":"2026-02-24T22:30:04.718214+0000","last_clean":"2026-02-24T22:30:04.718214+0000","last_became_active":"2026-02-24T22:15:25.831885+0000","last_became_peered":"2026-02-24T22:15:25.831885+0000","last_unstale":"2026-02-24T22:30:04.718214+0000","last_undegraded":"2026-02-24T22:30:04.718214+0000","last_fullsized":"2026-02-24T22:30:04.718214+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:54:09.173036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.18","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:54.730283+0000","last_change":"2026-02-24T22:15:22.810683+0000","last_active":"2026-02-24T22:29:54.730283+0000","last_peered":"2026-02-24T22:29:54.730283+0000","last_clean":"2026-02-24T22:29:54.730283+0000","last_became_active":"2026-02-24T22:15:22.810156+0000","last_became_peered":"2026-02-24T22:15:22.810156+0000","last_unstale":"2026-02-24T22:29:54.730283+0000","last_undegraded":"2026-02-24T22:29:54.730283+0000","last_fullsized":"2026-02-24T22:29:54.730283+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:29:33.293117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"79.7","version":"0'0","reported_seq":64,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.740460+0000","last_change":"2026-02-24T22:13:43.756250+0000","last_active":"2026-02-24T22:31:04.740460+0000","last_peered":"2026-02-24T22:31:04.740460+0000","last_clean":"2026-02-24T22:31:04.740460+0000","last_became_active":"2026-02-24T22:13:43.755957+0000","last_became_peered":"2026-02-24T22:13:43.755957+0000","last_unstale":"2026-02-24T22:31:04.740460+0000","last_undegraded":"2026-02-24T22:31:04.740460+0000","last_fullsized":"2026-02-24T22:31:04.740460+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:16:21.505648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.1b","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.735465+0000","last_change":"2026-02-24T22:23:29.200410+0000","last_active":"2026-02-24T22:30:59.735465+0000","last_peered":"2026-02-24T22:30:59.735465+0000","last_clean":"2026-02-24T22:30:59.735465+0000","last_became_active":"2026-02-24T22:23:29.200152+0000","last_became_peered":"2026-02-24T22:23:29.200152+0000","last_unstale":"2026-02-24T22:30:59.735465+0000","last_undegraded":"2026-02-24T22:30:59.735465+0000","last_fullsized":"2026-02-24T22:30:59.735465+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:24:51.029708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.18","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:59.730819+0000","last_change":"2026-02-24T22:15:25.824057+0000","last_active":"2026-02-24T22:29:59.730819+0000","last_peered":"2026-02-24T22:29:59.730819+0000","last_clean":"2026-02-24T22:29:59.730819+0000","last_became_active":"2026-02-24T22:15:25.823415+0000","last_became_peered":"2026-02-24T22:15:25.823415+0000","last_unstale":"2026-02-24T22:29:59.730819+0000","last_undegraded":"2026-02-24T22:29:59.730819+0000","last_fullsized":"2026-02-24T22:29:59.730819+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:00:21.357856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.19","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:59.714470+0000","last_change":"2026-02-24T22:15:22.817214+0000","last_active":"2026-02-24T22:29:59.714470+0000","last_peered":"2026-02-24T22:29:59.714470+0000","last_clean":"2026-02-24T22:29:59.714470+0000","last_became_active":"2026-02-24T22:15:22.816826+0000","last_became_peered":"2026-02-24T22:15:22.816826+0000","last_unstale":"2026-02-24T22:29:59.714470+0000","last_undegraded":"2026-02-24T22:29:59.714470+0000","last_fullsized":"2026-02-24T22:29:59.714470+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:13:51.793483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.6","version":"301'14","reported_seq":108,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.739656+0000","last_change":"2026-02-24T22:13:43.755144+0000","last_active":"2026-02-24T22:31:04.739656+0000","last_peered":"2026-02-24T22:31:04.739656+0000","last_clean":"2026-02-24T22:31:04.739656+0000","last_became_active":"2026-02-24T22:13:43.754753+0000","last_became_peered":"2026-02-24T22:13:43.754753+0000","last_unstale":"2026-02-24T22:31:04.739656+0000","last_undegraded":"2026-02-24T22:31:04.739656+0000","last_fullsized":"2026-02-24T22:31:04.739656+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:57:00.870335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":92,"num_write":34,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"82.18","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.726953+0000","last_change":"2026-02-24T22:23:29.213704+0000","last_active":"2026-02-24T22:30:44.726953+0000","last_peered":"2026-02-24T22:30:44.726953+0000","last_clean":"2026-02-24T22:30:44.726953+0000","last_became_active":"2026-02-24T22:23:29.213387+0000","last_became_peered":"2026-02-24T22:23:29.213387+0000","last_unstale":"2026-02-24T22:30:44.726953+0000","last_undegraded":"2026-02-24T22:30:44.726953+0000","last_fullsized":"2026-02-24T22:30:44.726953+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:46:11.186061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.1b","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.720738+0000","last_change":"2026-02-24T22:15:25.832293+0000","last_active":"2026-02-24T22:30:29.720738+0000","last_peered":"2026-02-24T22:30:29.720738+0000","last_clean":"2026-02-24T22:30:29.720738+0000","last_became_active":"2026-02-24T22:15:25.832097+0000","last_became_peered":"2026-02-24T22:15:25.832097+0000","last_unstale":"2026-02-24T22:30:29.720738+0000","last_undegraded":"2026-02-24T22:30:29.720738+0000","last_fullsized":"2026-02-24T22:30:29.720738+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:19:51.400688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.1a","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.720229+0000","last_change":"2026-02-24T22:15:22.819032+0000","last_active":"2026-02-24T22:30:24.720229+0000","last_peered":"2026-02-24T22:30:24.720229+0000","last_clean":"2026-02-24T22:30:24.720229+0000","last_became_active":"2026-02-24T22:15:22.818885+0000","last_became_peered":"2026-02-24T22:15:22.818885+0000","last_unstale":"2026-02-24T22:30:24.720229+0000","last_undegraded":"2026-02-24T22:30:24.720229+0000","last_fullsized":"2026-02-24T22:30:24.720229+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:15:52.421379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"79.5","version":"312'19","reported_seq":121,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:49.005257+0000","last_change":"2026-02-24T22:13:43.756043+0000","last_active":"2026-02-24T22:30:49.005257+0000","last_peered":"2026-02-24T22:30:49.005257+0000","last_clean":"2026-02-24T22:30:49.005257+0000","last_became_active":"2026-02-24T22:13:43.755495+0000","last_became_peered":"2026-02-24T22:13:43.755495+0000","last_unstale":"2026-02-24T22:30:49.005257+0000","last_undegraded":"2026-02-24T22:30:49.005257+0000","last_fullsized":"2026-02-24T22:30:49.005257+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:47:41.399446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":164,"num_read_kb":112,"num_write":47,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.19","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:49.719308+0000","last_change":"2026-02-24T22:23:29.212738+0000","last_active":"2026-02-24T22:30:49.719308+0000","last_peered":"2026-02-24T22:30:49.719308+0000","last_clean":"2026-02-24T22:30:49.719308+0000","last_became_active":"2026-02-24T22:23:29.212268+0000","last_became_peered":"2026-02-24T22:23:29.212268+0000","last_unstale":"2026-02-24T22:30:49.719308+0000","last_undegraded":"2026-02-24T22:30:49.719308+0000","last_fullsized":"2026-02-24T22:30:49.719308+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:32:25.776768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.1a","version":"307'2","reported_seq":35,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.731903+0000","last_change":"2026-02-24T22:15:25.826222+0000","last_active":"2026-02-24T22:30:29.731903+0000","last_peered":"2026-02-24T22:30:29.731903+0000","last_clean":"2026-02-24T22:30:29.731903+0000","last_became_active":"2026-02-24T22:15:25.825969+0000","last_became_peered":"2026-02-24T22:15:25.825969+0000","last_unstale":"2026-02-24T22:30:29.731903+0000","last_undegraded":"2026-02-24T22:30:29.731903+0000","last_fullsized":"2026-02-24T22:30:29.731903+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:29:05.133831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"80.1b","version":"307'2","reported_seq":41,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.733578+0000","last_change":"2026-02-24T22:15:22.816734+0000","last_active":"2026-02-24T22:30:24.733578+0000","last_peered":"2026-02-24T22:30:24.733578+0000","last_clean":"2026-02-24T22:30:24.733578+0000","last_became_active":"2026-02-24T22:15:22.816479+0000","last_became_peered":"2026-02-24T22:15:22.816479+0000","last_unstale":"2026-02-24T22:30:24.733578+0000","last_undegraded":"2026-02-24T22:30:24.733578+0000","last_fullsized":"2026-02-24T22:30:24.733578+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:57:39.996802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.4","version":"0'0","reported_seq":70,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.735005+0000","last_change":"2026-02-24T22:14:00.742004+0000","last_active":"2026-02-24T22:30:24.735005+0000","last_peered":"2026-02-24T22:30:24.735005+0000","last_clean":"2026-02-24T22:30:24.735005+0000","last_became_active":"2026-02-24T22:14:00.741839+0000","last_became_peered":"2026-02-24T22:14:00.741839+0000","last_unstale":"2026-02-24T22:30:24.735005+0000","last_undegraded":"2026-02-24T22:30:24.735005+0000","last_fullsized":"2026-02-24T22:30:24.735005+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":293,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:38:11.152954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"82.1e","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.727290+0000","last_change":"2026-02-24T22:23:29.215779+0000","last_active":"2026-02-24T22:31:34.727290+0000","last_peered":"2026-02-24T22:31:34.727290+0000","last_clean":"2026-02-24T22:31:34.727290+0000","last_became_active":"2026-02-24T22:23:29.215604+0000","last_became_peered":"2026-02-24T22:23:29.215604+0000","last_unstale":"2026-02-24T22:31:34.727290+0000","last_undegraded":"2026-02-24T22:31:34.727290+0000","last_fullsized":"2026-02-24T22:31:34.727290+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:58:56.390417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.1d","version":"307'2","reported_seq":35,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.733670+0000","last_change":"2026-02-24T22:15:25.828517+0000","last_active":"2026-02-24T22:30:29.733670+0000","last_peered":"2026-02-24T22:30:29.733670+0000","last_clean":"2026-02-24T22:30:29.733670+0000","last_became_active":"2026-02-24T22:15:25.828335+0000","last_became_peered":"2026-02-24T22:15:25.828335+0000","last_unstale":"2026-02-24T22:30:29.733670+0000","last_undegraded":"2026-02-24T22:30:29.733670+0000","last_fullsized":"2026-02-24T22:30:29.733670+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:18:15.606086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.1c","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.733619+0000","last_change":"2026-02-24T22:15:22.816853+0000","last_active":"2026-02-24T22:30:24.733619+0000","last_peered":"2026-02-24T22:30:24.733619+0000","last_clean":"2026-02-24T22:30:24.733619+0000","last_became_active":"2026-02-24T22:15:22.816548+0000","last_became_peered":"2026-02-24T22:15:22.816548+0000","last_unstale":"2026-02-24T22:30:24.733619+0000","last_undegraded":"2026-02-24T22:30:24.733619+0000","last_fullsized":"2026-02-24T22:30:24.733619+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:00:55.883988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.3","version":"312'6","reported_seq":77,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.731442+0000","last_change":"2026-02-24T22:13:43.756810+0000","last_active":"2026-02-24T22:30:24.731442+0000","last_peered":"2026-02-24T22:30:24.731442+0000","last_clean":"2026-02-24T22:30:24.731442+0000","last_became_active":"2026-02-24T22:13:43.756408+0000","last_became_peered":"2026-02-24T22:13:43.756408+0000","last_unstale":"2026-02-24T22:30:24.731442+0000","last_undegraded":"2026-02-24T22:30:24.731442+0000","last_fullsized":"2026-02-24T22:30:24.731442+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:24:33.412598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":20,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.1f","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744790+0000","last_change":"2026-02-24T22:23:29.199410+0000","last_active":"2026-02-24T22:31:49.744790+0000","last_peered":"2026-02-24T22:31:49.744790+0000","last_clean":"2026-02-24T22:31:49.744790+0000","last_became_active":"2026-02-24T22:23:29.199179+0000","last_became_peered":"2026-02-24T22:23:29.199179+0000","last_unstale":"2026-02-24T22:31:49.744790+0000","last_undegraded":"2026-02-24T22:31:49.744790+0000","last_fullsized":"2026-02-24T22:31:49.744790+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:21:23.823724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.1c","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.725548+0000","last_change":"2026-02-24T22:15:25.834316+0000","last_active":"2026-02-24T22:30:29.725548+0000","last_peered":"2026-02-24T22:30:29.725548+0000","last_clean":"2026-02-24T22:30:29.725548+0000","last_became_active":"2026-02-24T22:15:25.834072+0000","last_became_peered":"2026-02-24T22:15:25.834072+0000","last_unstale":"2026-02-24T22:30:29.725548+0000","last_undegraded":"2026-02-24T22:30:29.725548+0000","last_fullsized":"2026-02-24T22:30:29.725548+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:45:39.925744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"80.1d","version":"304'1","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.733639+0000","last_change":"2026-02-24T22:15:22.817173+0000","last_active":"2026-02-24T22:30:24.733639+0000","last_peered":"2026-02-24T22:30:24.733639+0000","last_clean":"2026-02-24T22:30:24.733639+0000","last_became_active":"2026-02-24T22:15:22.816615+0000","last_became_peered":"2026-02-24T22:15:22.816615+0000","last_unstale":"2026-02-24T22:30:24.733639+0000","last_undegraded":"2026-02-24T22:30:24.733639+0000","last_fullsized":"2026-02-24T22:30:24.733639+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:25:34.323299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":4096,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.2","version":"312'5","reported_seq":77,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.733168+0000","last_change":"2026-02-24T22:13:43.755888+0000","last_active":"2026-02-24T22:30:24.733168+0000","last_peered":"2026-02-24T22:30:24.733168+0000","last_clean":"2026-02-24T22:30:24.733168+0000","last_became_active":"2026-02-24T22:13:43.755241+0000","last_became_peered":"2026-02-24T22:13:43.755241+0000","last_unstale":"2026-02-24T22:30:24.733168+0000","last_undegraded":"2026-02-24T22:30:24.733168+0000","last_fullsized":"2026-02-24T22:30:24.733168+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:13:28.809470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":23,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.1c","version":"310'1","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.720361+0000","last_change":"2026-02-24T22:23:29.213715+0000","last_active":"2026-02-24T22:30:59.720361+0000","last_peered":"2026-02-24T22:30:59.720361+0000","last_clean":"2026-02-24T22:30:59.720361+0000","last_became_active":"2026-02-24T22:23:29.213490+0000","last_became_peered":"2026-02-24T22:23:29.213490+0000","last_unstale":"2026-02-24T22:30:59.720361+0000","last_undegraded":"2026-02-24T22:30:59.720361+0000","last_fullsized":"2026-02-24T22:30:59.720361+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:21:40.032070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.1f","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.747978+0000","last_change":"2026-02-24T22:15:25.828181+0000","last_active":"2026-02-24T22:32:09.747978+0000","last_peered":"2026-02-24T22:32:09.747978+0000","last_clean":"2026-02-24T22:32:09.747978+0000","last_became_active":"2026-02-24T22:15:25.827873+0000","last_became_peered":"2026-02-24T22:15:25.827873+0000","last_unstale":"2026-02-24T22:32:09.747978+0000","last_undegraded":"2026-02-24T22:32:09.747978+0000","last_fullsized":"2026-02-24T22:32:09.747978+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:19:57.290828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.1e","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:29.726743+0000","last_change":"2026-02-24T22:15:22.817535+0000","last_active":"2026-02-24T22:31:29.726743+0000","last_peered":"2026-02-24T22:31:29.726743+0000","last_clean":"2026-02-24T22:31:29.726743+0000","last_became_active":"2026-02-24T22:15:22.817337+0000","last_became_peered":"2026-02-24T22:15:22.817337+0000","last_unstale":"2026-02-24T22:31:29.726743+0000","last_undegraded":"2026-02-24T22:31:29.726743+0000","last_fullsized":"2026-02-24T22:31:29.726743+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:02:48.141978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"79.1","version":"312'23","reported_seq":125,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.720202+0000","last_change":"2026-02-24T22:13:43.761201+0000","last_active":"2026-02-24T22:30:24.720202+0000","last_peered":"2026-02-24T22:30:24.720202+0000","last_clean":"2026-02-24T22:30:24.720202+0000","last_became_active":"2026-02-24T22:13:43.760933+0000","last_became_peered":"2026-02-24T22:13:43.760933+0000","last_unstale":"2026-02-24T22:30:24.720202+0000","last_undegraded":"2026-02-24T22:30:24.720202+0000","last_fullsized":"2026-02-24T22:30:24.720202+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:18:32.108308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":174,"num_read_kb":113,"num_write":59,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.1d","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.723721+0000","last_change":"2026-02-24T22:23:29.215553+0000","last_active":"2026-02-24T22:30:59.723721+0000","last_peered":"2026-02-24T22:30:59.723721+0000","last_clean":"2026-02-24T22:30:59.723721+0000","last_became_active":"2026-02-24T22:23:29.215334+0000","last_became_peered":"2026-02-24T22:23:29.215334+0000","last_unstale":"2026-02-24T22:30:59.723721+0000","last_undegraded":"2026-02-24T22:30:59.723721+0000","last_fullsized":"2026-02-24T22:30:59.723721+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:01:42.978764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.1e","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.739370+0000","last_change":"2026-02-24T22:15:25.827409+0000","last_active":"2026-02-24T22:31:34.739370+0000","last_peered":"2026-02-24T22:31:34.739370+0000","last_clean":"2026-02-24T22:31:34.739370+0000","last_became_active":"2026-02-24T22:15:25.827124+0000","last_became_peered":"2026-02-24T22:15:25.827124+0000","last_unstale":"2026-02-24T22:31:34.739370+0000","last_undegraded":"2026-02-24T22:31:34.739370+0000","last_fullsized":"2026-02-24T22:31:34.739370+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:44:21.789639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"80.1f","version":"304'2","reported_seq":43,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:04.726966+0000","last_change":"2026-02-24T22:15:22.816227+0000","last_active":"2026-02-24T22:32:04.726966+0000","last_peered":"2026-02-24T22:32:04.726966+0000","last_clean":"2026-02-24T22:32:04.726966+0000","last_became_active":"2026-02-24T22:15:22.815938+0000","last_became_peered":"2026-02-24T22:15:22.815938+0000","last_unstale":"2026-02-24T22:32:04.726966+0000","last_undegraded":"2026-02-24T22:32:04.726966+0000","last_fullsized":"2026-02-24T22:32:04.726966+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:40:15.210510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.0","version":"0'0","reported_seq":65,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.745118+0000","last_change":"2026-02-24T22:13:43.755907+0000","last_active":"2026-02-24T22:31:49.745118+0000","last_peered":"2026-02-24T22:31:49.745118+0000","last_clean":"2026-02-24T22:31:49.745118+0000","last_became_active":"2026-02-24T22:13:43.755266+0000","last_became_peered":"2026-02-24T22:13:43.755266+0000","last_unstale":"2026-02-24T22:31:49.745118+0000","last_undegraded":"2026-02-24T22:31:49.745118+0000","last_fullsized":"2026-02-24T22:31:49.745118+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:42:48.314773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"312'215","reported_seq":920,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:47.193151+0000","last_change":"2026-02-24T22:05:16.193051+0000","last_active":"2026-02-24T22:33:47.193151+0000","last_peered":"2026-02-24T22:33:47.193151+0000","last_clean":"2026-02-24T22:33:47.193151+0000","last_became_active":"2026-02-24T22:05:16.192589+0000","last_became_peered":"2026-02-24T22:05:16.192589+0000","last_unstale":"2026-02-24T22:33:47.193151+0000","last_undegraded":"2026-02-24T22:33:47.193151+0000","last_fullsized":"2026-02-24T22:33:47.193151+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:20:06.945662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8436,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":377,"num_read_kb":342,"num_write":1213,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"312'197","reported_seq":898,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:46.730087+0000","last_change":"2026-02-24T22:05:18.192403+0000","last_active":"2026-02-24T22:33:46.730087+0000","last_peered":"2026-02-24T22:33:46.730087+0000","last_clean":"2026-02-24T22:33:46.730087+0000","last_became_active":"2026-02-24T22:05:18.192203+0000","last_became_peered":"2026-02-24T22:05:18.192203+0000","last_unstale":"2026-02-24T22:33:46.730087+0000","last_undegraded":"2026-02-24T22:33:46.730087+0000","last_fullsized":"2026-02-24T22:33:46.730087+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":100,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:33:32.754536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":780,"num_read_kb":643,"num_write":282,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"312'304","reported_seq":1112,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:51.011231+0000","last_change":"2026-02-24T22:05:16.181637+0000","last_active":"2026-02-24T22:30:51.011231+0000","last_peered":"2026-02-24T22:30:51.011231+0000","last_clean":"2026-02-24T22:30:51.011231+0000","last_became_active":"2026-02-24T22:05:16.181379+0000","last_became_peered":"2026-02-24T22:05:16.181379+0000","last_unstale":"2026-02-24T22:30:51.011231+0000","last_undegraded":"2026-02-24T22:30:51.011231+0000","last_fullsized":"2026-02-24T22:30:51.011231+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":200,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:35:21.970288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21628,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":848,"num_read_kb":611,"num_write":1466,"num_write_kb":5240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"312'459","reported_seq":1257,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:57.922033+0000","last_change":"2026-02-24T22:05:18.190910+0000","last_active":"2026-02-24T22:30:57.922033+0000","last_peered":"2026-02-24T22:30:57.922033+0000","last_clean":"2026-02-24T22:30:57.922033+0000","last_became_active":"2026-02-24T22:05:18.190689+0000","last_became_peered":"2026-02-24T22:05:18.190689+0000","last_unstale":"2026-02-24T22:30:57.922033+0000","last_undegraded":"2026-02-24T22:30:57.922033+0000","last_fullsized":"2026-02-24T22:30:57.922033+0000","mapping_epoch":24,"log_start":"312'400","ondisk_log_start":"312'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":400,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:35:35.142199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":2017,"num_read_kb":1670,"num_write":788,"num_write_kb":502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"312'368","reported_seq":1154,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:58.398455+0000","last_change":"2026-02-24T22:05:16.180825+0000","last_active":"2026-02-24T22:30:58.398455+0000","last_peered":"2026-02-24T22:30:58.398455+0000","last_clean":"2026-02-24T22:30:58.398455+0000","last_became_active":"2026-02-24T22:05:16.180681+0000","last_became_peered":"2026-02-24T22:05:16.180681+0000","last_unstale":"2026-02-24T22:30:58.398455+0000","last_undegraded":"2026-02-24T22:30:58.398455+0000","last_fullsized":"2026-02-24T22:30:58.398455+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":300,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:38:22.293881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5517838,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":825,"num_read_kb":6166,"num_write":1529,"num_write_kb":5459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"312'80","reported_seq":727,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.732683+0000","last_change":"2026-02-24T22:05:18.192661+0000","last_active":"2026-02-24T22:30:19.732683+0000","last_peered":"2026-02-24T22:30:19.732683+0000","last_clean":"2026-02-24T22:30:19.732683+0000","last_became_active":"2026-02-24T22:05:18.190727+0000","last_became_peered":"2026-02-24T22:05:18.190727+0000","last_unstale":"2026-02-24T22:30:19.732683+0000","last_undegraded":"2026-02-24T22:30:19.732683+0000","last_fullsized":"2026-02-24T22:30:19.732683+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:10:39.624197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":268,"num_read_kb":220,"num_write":112,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"312'218","reported_seq":902,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:01.970038+0000","last_change":"2026-02-24T22:05:16.193170+0000","last_active":"2026-02-24T22:34:01.970038+0000","last_peered":"2026-02-24T22:34:01.970038+0000","last_clean":"2026-02-24T22:34:01.970038+0000","last_became_active":"2026-02-24T22:05:16.192922+0000","last_became_peered":"2026-02-24T22:05:16.192922+0000","last_unstale":"2026-02-24T22:34:01.970038+0000","last_undegraded":"2026-02-24T22:34:01.970038+0000","last_fullsized":"2026-02-24T22:34:01.970038+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":200,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:51:56.894641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":549324,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":307,"num_read_kb":3190,"num_write":1131,"num_write_kb":12582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"312'135","reported_seq":815,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.733034+0000","last_change":"2026-02-24T22:05:18.184446+0000","last_active":"2026-02-24T22:30:19.733034+0000","last_peered":"2026-02-24T22:30:19.733034+0000","last_clean":"2026-02-24T22:30:19.733034+0000","last_became_active":"2026-02-24T22:05:18.184014+0000","last_became_peered":"2026-02-24T22:05:18.184014+0000","last_unstale":"2026-02-24T22:30:19.733034+0000","last_undegraded":"2026-02-24T22:30:19.733034+0000","last_fullsized":"2026-02-24T22:30:19.733034+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:12:45.735332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":489,"num_read_kb":401,"num_write":208,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"312'208","reported_seq":919,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:12.258587+0000","last_change":"2026-02-24T22:05:16.191359+0000","last_active":"2026-02-24T22:34:12.258587+0000","last_peered":"2026-02-24T22:34:12.258587+0000","last_clean":"2026-02-24T22:34:12.258587+0000","last_became_active":"2026-02-24T22:05:16.191055+0000","last_became_peered":"2026-02-24T22:05:16.191055+0000","last_unstale":"2026-02-24T22:34:12.258587+0000","last_undegraded":"2026-02-24T22:34:12.258587+0000","last_fullsized":"2026-02-24T22:34:12.258587+0000","mapping_epoch":22,"log_start":"312'100","ondisk_log_start":"312'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":100,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:40:09.331901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6857028,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":360,"num_read_kb":15420,"num_write":1138,"num_write_kb":8454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"312'470","reported_seq":1186,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.738353+0000","last_change":"2026-02-24T22:05:18.190983+0000","last_active":"2026-02-24T22:31:24.738353+0000","last_peered":"2026-02-24T22:31:24.738353+0000","last_clean":"2026-02-24T22:31:24.738353+0000","last_became_active":"2026-02-24T22:05:18.190598+0000","last_became_peered":"2026-02-24T22:05:18.190598+0000","last_unstale":"2026-02-24T22:31:24.738353+0000","last_undegraded":"2026-02-24T22:31:24.738353+0000","last_fullsized":"2026-02-24T22:31:24.738353+0000","mapping_epoch":24,"log_start":"312'400","ondisk_log_start":"312'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":400,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:52:56.661245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1905,"num_read_kb":1522,"num_write":772,"num_write_kb":389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"312'233","reported_seq":943,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:50.432924+0000","last_change":"2026-02-24T22:05:16.191753+0000","last_active":"2026-02-24T22:30:50.432924+0000","last_peered":"2026-02-24T22:30:50.432924+0000","last_clean":"2026-02-24T22:30:50.432924+0000","last_became_active":"2026-02-24T22:05:16.191164+0000","last_became_peered":"2026-02-24T22:05:16.191164+0000","last_unstale":"2026-02-24T22:30:50.432924+0000","last_undegraded":"2026-02-24T22:30:50.432924+0000","last_fullsized":"2026-02-24T22:30:50.432924+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":200,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:23:55.135404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5295803,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":424,"num_read_kb":5551,"num_write":1270,"num_write_kb":5240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"312'56","reported_seq":709,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.732718+0000","last_change":"2026-02-24T22:05:18.198708+0000","last_active":"2026-02-24T22:30:19.732718+0000","last_peered":"2026-02-24T22:30:19.732718+0000","last_clean":"2026-02-24T22:30:19.732718+0000","last_became_active":"2026-02-24T22:05:18.198559+0000","last_became_peered":"2026-02-24T22:05:18.198559+0000","last_unstale":"2026-02-24T22:30:19.732718+0000","last_undegraded":"2026-02-24T22:30:19.732718+0000","last_fullsized":"2026-02-24T22:30:19.732718+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:01:34.891706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":195,"num_read_kb":167,"num_write":70,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"312'219","reported_seq":919,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:57.717745+0000","last_change":"2026-02-24T22:05:16.190594+0000","last_active":"2026-02-24T22:30:57.717745+0000","last_peered":"2026-02-24T22:30:57.717745+0000","last_clean":"2026-02-24T22:30:57.717745+0000","last_became_active":"2026-02-24T22:05:16.190367+0000","last_became_peered":"2026-02-24T22:05:16.190367+0000","last_unstale":"2026-02-24T22:30:57.717745+0000","last_undegraded":"2026-02-24T22:30:57.717745+0000","last_fullsized":"2026-02-24T22:30:57.717745+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":200,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:34:28.350502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1051952,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":361,"num_read_kb":1335,"num_write":1155,"num_write_kb":1104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"312'201","reported_seq":941,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:49.857860+0000","last_change":"2026-02-24T22:05:18.192094+0000","last_active":"2026-02-24T22:33:49.857860+0000","last_peered":"2026-02-24T22:33:49.857860+0000","last_clean":"2026-02-24T22:33:49.857860+0000","last_became_active":"2026-02-24T22:05:18.191767+0000","last_became_peered":"2026-02-24T22:05:18.191767+0000","last_unstale":"2026-02-24T22:33:49.857860+0000","last_undegraded":"2026-02-24T22:33:49.857860+0000","last_fullsized":"2026-02-24T22:33:49.857860+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":100,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:15:25.401981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":876,"num_read_kb":754,"num_write":323,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"312'272","reported_seq":1042,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:40.186391+0000","last_change":"2026-02-24T22:05:16.186732+0000","last_active":"2026-02-24T22:33:40.186391+0000","last_peered":"2026-02-24T22:33:40.186391+0000","last_clean":"2026-02-24T22:33:40.186391+0000","last_became_active":"2026-02-24T22:05:16.185779+0000","last_became_peered":"2026-02-24T22:05:16.185779+0000","last_unstale":"2026-02-24T22:33:40.186391+0000","last_undegraded":"2026-02-24T22:33:40.186391+0000","last_fullsized":"2026-02-24T22:33:40.186391+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":200,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:01:24.063571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50495,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":533,"num_read_kb":877,"num_write":1531,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"312'83","reported_seq":729,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.716462+0000","last_change":"2026-02-24T22:05:18.188067+0000","last_active":"2026-02-24T22:30:19.716462+0000","last_peered":"2026-02-24T22:30:19.716462+0000","last_clean":"2026-02-24T22:30:19.716462+0000","last_became_active":"2026-02-24T22:05:18.187838+0000","last_became_peered":"2026-02-24T22:05:18.187838+0000","last_unstale":"2026-02-24T22:30:19.716462+0000","last_undegraded":"2026-02-24T22:30:19.716462+0000","last_fullsized":"2026-02-24T22:30:19.716462+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:05:06.602774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":263,"num_read_kb":217,"num_write":106,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"312'135","reported_seq":808,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.736992+0000","last_change":"2026-02-24T22:05:18.192713+0000","last_active":"2026-02-24T22:30:59.736992+0000","last_peered":"2026-02-24T22:30:59.736992+0000","last_clean":"2026-02-24T22:30:59.736992+0000","last_became_active":"2026-02-24T22:05:18.191625+0000","last_became_peered":"2026-02-24T22:05:18.191625+0000","last_unstale":"2026-02-24T22:30:59.736992+0000","last_undegraded":"2026-02-24T22:30:59.736992+0000","last_fullsized":"2026-02-24T22:30:59.736992+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:24:04.071735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":477,"num_read_kb":396,"num_write":183,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"312'237","reported_seq":974,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:53.497360+0000","last_change":"2026-02-24T22:06:01.025897+0000","last_active":"2026-02-24T22:33:53.497360+0000","last_peered":"2026-02-24T22:33:53.497360+0000","last_clean":"2026-02-24T22:33:53.497360+0000","last_became_active":"2026-02-24T22:06:01.025659+0000","last_became_peered":"2026-02-24T22:06:01.025659+0000","last_unstale":"2026-02-24T22:33:53.497360+0000","last_undegraded":"2026-02-24T22:33:53.497360+0000","last_fullsized":"2026-02-24T22:33:53.497360+0000","mapping_epoch":36,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":200,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:15:11.362680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":623937,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":456,"num_read_kb":16219,"num_write":1259,"num_write_kb":8500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"312'4249","reported_seq":5059,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.162950+0000","last_change":"2026-02-24T22:05:18.193454+0000","last_active":"2026-02-24T22:34:15.162950+0000","last_peered":"2026-02-24T22:34:15.162950+0000","last_clean":"2026-02-24T22:34:15.162950+0000","last_became_active":"2026-02-24T22:05:18.193257+0000","last_became_peered":"2026-02-24T22:05:18.193257+0000","last_unstale":"2026-02-24T22:34:15.162950+0000","last_undegraded":"2026-02-24T22:34:15.162950+0000","last_fullsized":"2026-02-24T22:34:15.162950+0000","mapping_epoch":24,"log_start":"312'4200","ondisk_log_start":"312'4200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":3000,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:25:08.863245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":17332,"num_read_kb":14553,"num_write":6398,"num_write_kb":3198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"312'213","reported_seq":925,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.741319+0000","last_change":"2026-02-24T22:05:16.191122+0000","last_active":"2026-02-24T22:31:54.741319+0000","last_peered":"2026-02-24T22:31:54.741319+0000","last_clean":"2026-02-24T22:31:54.741319+0000","last_became_active":"2026-02-24T22:05:16.190819+0000","last_became_peered":"2026-02-24T22:05:16.190819+0000","last_unstale":"2026-02-24T22:31:54.741319+0000","last_undegraded":"2026-02-24T22:31:54.741319+0000","last_fullsized":"2026-02-24T22:31:54.741319+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":200,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:31:00.005386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5568323,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":396,"num_read_kb":8113,"num_write":1091,"num_write_kb":5510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"312'862","reported_seq":4024,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.230995+0000","last_change":"2026-02-24T22:05:18.190719+0000","last_active":"2026-02-24T22:34:17.230995+0000","last_peered":"2026-02-24T22:34:17.230995+0000","last_clean":"2026-02-24T22:34:17.230995+0000","last_became_active":"2026-02-24T22:05:18.190363+0000","last_became_peered":"2026-02-24T22:05:18.190363+0000","last_unstale":"2026-02-24T22:34:17.230995+0000","last_undegraded":"2026-02-24T22:34:17.230995+0000","last_fullsized":"2026-02-24T22:34:17.230995+0000","mapping_epoch":24,"log_start":"312'800","ondisk_log_start":"312'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":800,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:46:25.367911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4747,"num_read_kb":5785,"num_write":398,"num_write_kb":236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"312'229","reported_seq":960,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:03.733775+0000","last_change":"2026-02-24T22:05:16.191246+0000","last_active":"2026-02-24T22:34:03.733775+0000","last_peered":"2026-02-24T22:34:03.733775+0000","last_clean":"2026-02-24T22:34:03.733775+0000","last_became_active":"2026-02-24T22:05:16.190816+0000","last_became_peered":"2026-02-24T22:05:16.190816+0000","last_unstale":"2026-02-24T22:34:03.733775+0000","last_undegraded":"2026-02-24T22:34:03.733775+0000","last_fullsized":"2026-02-24T22:34:03.733775+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":200,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:47:51.922901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10732886,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":476,"num_read_kb":2322,"num_write":1236,"num_write_kb":12235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"312'93","reported_seq":796,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.725906+0000","last_change":"2026-02-24T22:05:18.187293+0000","last_active":"2026-02-24T22:31:54.725906+0000","last_peered":"2026-02-24T22:31:54.725906+0000","last_clean":"2026-02-24T22:31:54.725906+0000","last_became_active":"2026-02-24T22:05:18.186994+0000","last_became_peered":"2026-02-24T22:05:18.186994+0000","last_unstale":"2026-02-24T22:31:54.725906+0000","last_undegraded":"2026-02-24T22:31:54.725906+0000","last_fullsized":"2026-02-24T22:31:54.725906+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:00:02.034352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":352,"num_read_kb":306,"num_write":116,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"312'315","reported_seq":1107,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:54.369202+0000","last_change":"2026-02-24T22:05:16.189106+0000","last_active":"2026-02-24T22:33:54.369202+0000","last_peered":"2026-02-24T22:33:54.369202+0000","last_clean":"2026-02-24T22:33:54.369202+0000","last_became_active":"2026-02-24T22:05:16.188658+0000","last_became_peered":"2026-02-24T22:05:16.188658+0000","last_unstale":"2026-02-24T22:33:54.369202+0000","last_undegraded":"2026-02-24T22:33:54.369202+0000","last_fullsized":"2026-02-24T22:33:54.369202+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:33:27.530851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52140,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":775,"num_read_kb":685,"num_write":1754,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"312'169","reported_seq":891,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.245262+0000","last_change":"2026-02-24T22:05:18.187949+0000","last_active":"2026-02-24T22:34:18.245262+0000","last_peered":"2026-02-24T22:34:18.245262+0000","last_clean":"2026-02-24T22:34:18.245262+0000","last_became_active":"2026-02-24T22:05:18.187682+0000","last_became_peered":"2026-02-24T22:05:18.187682+0000","last_unstale":"2026-02-24T22:34:18.245262+0000","last_undegraded":"2026-02-24T22:34:18.245262+0000","last_fullsized":"2026-02-24T22:34:18.245262+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:07:24.443062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":639,"num_read_kb":531,"num_write":250,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"312'224","reported_seq":924,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:45.924671+0000","last_change":"2026-02-24T22:05:16.185278+0000","last_active":"2026-02-24T22:33:45.924671+0000","last_peered":"2026-02-24T22:33:45.924671+0000","last_clean":"2026-02-24T22:33:45.924671+0000","last_became_active":"2026-02-24T22:05:16.184209+0000","last_became_peered":"2026-02-24T22:05:16.184209+0000","last_unstale":"2026-02-24T22:33:45.924671+0000","last_undegraded":"2026-02-24T22:33:45.924671+0000","last_fullsized":"2026-02-24T22:33:45.924671+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":200,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:50:09.257349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297746,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":353,"num_read_kb":601,"num_write":1169,"num_write_kb":10636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"312'111","reported_seq":789,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.222919+0000","last_change":"2026-02-24T22:05:18.194092+0000","last_active":"2026-02-24T22:34:18.222919+0000","last_peered":"2026-02-24T22:34:18.222919+0000","last_clean":"2026-02-24T22:34:18.222919+0000","last_became_active":"2026-02-24T22:05:18.193320+0000","last_became_peered":"2026-02-24T22:05:18.193320+0000","last_unstale":"2026-02-24T22:34:18.222919+0000","last_undegraded":"2026-02-24T22:34:18.222919+0000","last_fullsized":"2026-02-24T22:34:18.222919+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":100,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:27:50.398388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":425,"num_read_kb":355,"num_write":146,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"312'212","reported_seq":925,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:46.698672+0000","last_change":"2026-02-24T22:05:16.181803+0000","last_active":"2026-02-24T22:33:46.698672+0000","last_peered":"2026-02-24T22:33:46.698672+0000","last_clean":"2026-02-24T22:33:46.698672+0000","last_became_active":"2026-02-24T22:05:16.181527+0000","last_became_peered":"2026-02-24T22:05:16.181527+0000","last_unstale":"2026-02-24T22:33:46.698672+0000","last_undegraded":"2026-02-24T22:33:46.698672+0000","last_fullsized":"2026-02-24T22:33:46.698672+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:53:05.002989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5243778,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":406,"num_read_kb":5463,"num_write":1149,"num_write_kb":5189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:39.728581+0000","last_change":"2026-02-24T22:05:29.658481+0000","last_active":"2026-02-24T22:29:39.728581+0000","last_peered":"2026-02-24T22:29:39.728581+0000","last_clean":"2026-02-24T22:29:39.728581+0000","last_became_active":"2026-02-24T22:05:29.657764+0000","last_became_peered":"2026-02-24T22:05:29.657764+0000","last_unstale":"2026-02-24T22:29:39.728581+0000","last_undegraded":"2026-02-24T22:29:39.728581+0000","last_fullsized":"2026-02-24T22:29:39.728581+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:34:57.407311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"312'80","reported_seq":734,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.729056+0000","last_change":"2026-02-24T22:05:18.185097+0000","last_active":"2026-02-24T22:31:04.729056+0000","last_peered":"2026-02-24T22:31:04.729056+0000","last_clean":"2026-02-24T22:31:04.729056+0000","last_became_active":"2026-02-24T22:05:18.184771+0000","last_became_peered":"2026-02-24T22:05:18.184771+0000","last_unstale":"2026-02-24T22:31:04.729056+0000","last_undegraded":"2026-02-24T22:31:04.729056+0000","last_fullsized":"2026-02-24T22:31:04.729056+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:34:23.195340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":246,"num_read_kb":208,"num_write":102,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"312'259","reported_seq":1040,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:03.524714+0000","last_change":"2026-02-24T22:05:16.192243+0000","last_active":"2026-02-24T22:34:03.524714+0000","last_peered":"2026-02-24T22:34:03.524714+0000","last_clean":"2026-02-24T22:34:03.524714+0000","last_became_active":"2026-02-24T22:05:16.191841+0000","last_became_peered":"2026-02-24T22:05:16.191841+0000","last_unstale":"2026-02-24T22:34:03.524714+0000","last_undegraded":"2026-02-24T22:34:03.524714+0000","last_fullsized":"2026-02-24T22:34:03.524714+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":200,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:59:17.072003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19280971,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":534,"num_read_kb":81366,"num_write":1249,"num_write_kb":20589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":579,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.745270+0000","last_change":"2026-02-24T22:05:20.194969+0000","last_active":"2026-02-24T22:31:54.745270+0000","last_peered":"2026-02-24T22:31:54.745270+0000","last_clean":"2026-02-24T22:31:54.745270+0000","last_became_active":"2026-02-24T22:05:20.194031+0000","last_became_peered":"2026-02-24T22:05:20.194031+0000","last_unstale":"2026-02-24T22:31:54.745270+0000","last_undegraded":"2026-02-24T22:31:54.745270+0000","last_fullsized":"2026-02-24T22:31:54.745270+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:17:15.557806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"312'254","reported_seq":889,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.375668+0000","last_change":"2026-02-24T22:05:22.192071+0000","last_active":"2026-02-24T22:34:18.375668+0000","last_peered":"2026-02-24T22:34:18.375668+0000","last_clean":"2026-02-24T22:34:18.375668+0000","last_became_active":"2026-02-24T22:05:22.191928+0000","last_became_peered":"2026-02-24T22:05:22.191928+0000","last_unstale":"2026-02-24T22:34:18.375668+0000","last_undegraded":"2026-02-24T22:34:18.375668+0000","last_fullsized":"2026-02-24T22:34:18.375668+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":200,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:56:18.209592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54024,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":400,"num_read_kb":789,"num_write":352,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"312'158","reported_seq":747,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.253470+0000","last_change":"2026-02-24T22:05:24.205235+0000","last_active":"2026-02-24T22:34:18.253470+0000","last_peered":"2026-02-24T22:34:18.253470+0000","last_clean":"2026-02-24T22:34:18.253470+0000","last_became_active":"2026-02-24T22:05:24.204990+0000","last_became_peered":"2026-02-24T22:05:24.204990+0000","last_unstale":"2026-02-24T22:34:18.253470+0000","last_undegraded":"2026-02-24T22:34:18.253470+0000","last_fullsized":"2026-02-24T22:34:18.253470+0000","mapping_epoch":30,"log_start":"37'100","ondisk_log_start":"37'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":100,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:14:42.637540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32373,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":92,"num_read_kb":87,"num_write":391,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":564,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:44.730561+0000","last_change":"2026-02-24T22:05:29.662185+0000","last_active":"2026-02-24T22:29:44.730561+0000","last_peered":"2026-02-24T22:29:44.730561+0000","last_clean":"2026-02-24T22:29:44.730561+0000","last_became_active":"2026-02-24T22:05:29.661747+0000","last_became_peered":"2026-02-24T22:05:29.661747+0000","last_unstale":"2026-02-24T22:29:44.730561+0000","last_undegraded":"2026-02-24T22:29:44.730561+0000","last_fullsized":"2026-02-24T22:29:44.730561+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:08:40.631093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"312'73","reported_seq":743,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:07.153602+0000","last_change":"2026-02-24T22:05:18.191685+0000","last_active":"2026-02-24T22:34:07.153602+0000","last_peered":"2026-02-24T22:34:07.153602+0000","last_clean":"2026-02-24T22:34:07.153602+0000","last_became_active":"2026-02-24T22:05:18.191346+0000","last_became_peered":"2026-02-24T22:05:18.191346+0000","last_unstale":"2026-02-24T22:34:07.153602+0000","last_undegraded":"2026-02-24T22:34:07.153602+0000","last_fullsized":"2026-02-24T22:34:07.153602+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:48:20.436580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":237,"num_read_kb":209,"num_write":87,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"312'213","reported_seq":933,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:41.594986+0000","last_change":"2026-02-24T22:05:16.184337+0000","last_active":"2026-02-24T22:33:41.594986+0000","last_peered":"2026-02-24T22:33:41.594986+0000","last_clean":"2026-02-24T22:33:41.594986+0000","last_became_active":"2026-02-24T22:05:16.183990+0000","last_became_peered":"2026-02-24T22:05:16.183990+0000","last_unstale":"2026-02-24T22:33:41.594986+0000","last_undegraded":"2026-02-24T22:33:41.594986+0000","last_fullsized":"2026-02-24T22:33:41.594986+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":200,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:27:53.239432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22095805,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":400,"num_read_kb":23691,"num_write":1212,"num_write_kb":23324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.729065+0000","last_change":"2026-02-24T22:05:20.186544+0000","last_active":"2026-02-24T22:31:04.729065+0000","last_peered":"2026-02-24T22:31:04.729065+0000","last_clean":"2026-02-24T22:31:04.729065+0000","last_became_active":"2026-02-24T22:05:20.186217+0000","last_became_peered":"2026-02-24T22:05:20.186217+0000","last_unstale":"2026-02-24T22:31:04.729065+0000","last_undegraded":"2026-02-24T22:31:04.729065+0000","last_fullsized":"2026-02-24T22:31:04.729065+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:41:55.793283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"312'304","reported_seq":1150,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.527143+0000","last_change":"2026-02-24T22:05:22.197384+0000","last_active":"2026-02-24T22:34:18.527143+0000","last_peered":"2026-02-24T22:34:18.527143+0000","last_clean":"2026-02-24T22:34:18.527143+0000","last_became_active":"2026-02-24T22:05:22.196911+0000","last_became_peered":"2026-02-24T22:05:22.196911+0000","last_unstale":"2026-02-24T22:34:18.527143+0000","last_undegraded":"2026-02-24T22:34:18.527143+0000","last_fullsized":"2026-02-24T22:34:18.527143+0000","mapping_epoch":28,"log_start":"307'200","ondisk_log_start":"307'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":200,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:19:12.370112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1012,"num_read_kb":538,"num_write":51,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"312'187","reported_seq":800,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.220647+0000","last_change":"2026-02-24T22:05:24.207903+0000","last_active":"2026-02-24T22:34:18.220647+0000","last_peered":"2026-02-24T22:34:18.220647+0000","last_clean":"2026-02-24T22:34:18.220647+0000","last_became_active":"2026-02-24T22:05:24.207692+0000","last_became_peered":"2026-02-24T22:05:24.207692+0000","last_unstale":"2026-02-24T22:34:18.220647+0000","last_undegraded":"2026-02-24T22:34:18.220647+0000","last_fullsized":"2026-02-24T22:34:18.220647+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:14:09.532597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29978,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":173,"num_read_kb":171,"num_write":478,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":573,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.734173+0000","last_change":"2026-02-24T22:05:29.654457+0000","last_active":"2026-02-24T22:30:29.734173+0000","last_peered":"2026-02-24T22:30:29.734173+0000","last_clean":"2026-02-24T22:30:29.734173+0000","last_became_active":"2026-02-24T22:05:29.654288+0000","last_became_peered":"2026-02-24T22:05:29.654288+0000","last_unstale":"2026-02-24T22:30:29.734173+0000","last_undegraded":"2026-02-24T22:30:29.734173+0000","last_fullsized":"2026-02-24T22:30:29.734173+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:05:32.007637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"312'274","reported_seq":998,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.743416+0000","last_change":"2026-02-24T22:05:18.184707+0000","last_active":"2026-02-24T22:31:54.743416+0000","last_peered":"2026-02-24T22:31:54.743416+0000","last_clean":"2026-02-24T22:31:54.743416+0000","last_became_active":"2026-02-24T22:05:18.184542+0000","last_became_peered":"2026-02-24T22:05:18.184542+0000","last_unstale":"2026-02-24T22:31:54.743416+0000","last_undegraded":"2026-02-24T22:31:54.743416+0000","last_fullsized":"2026-02-24T22:31:54.743416+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":200,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:55:45.622376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1179,"num_read_kb":976,"num_write":460,"num_write_kb":243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"312'223","reported_seq":937,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.739663+0000","last_change":"2026-02-24T22:05:16.190582+0000","last_active":"2026-02-24T22:30:59.739663+0000","last_peered":"2026-02-24T22:30:59.739663+0000","last_clean":"2026-02-24T22:30:59.739663+0000","last_became_active":"2026-02-24T22:05:16.190372+0000","last_became_peered":"2026-02-24T22:05:16.190372+0000","last_unstale":"2026-02-24T22:30:59.739663+0000","last_undegraded":"2026-02-24T22:30:59.739663+0000","last_fullsized":"2026-02-24T22:30:59.739663+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":200,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:46:33.357028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":364264,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":363,"num_read_kb":3398,"num_write":1108,"num_write_kb":524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.734254+0000","last_change":"2026-02-24T22:05:20.187522+0000","last_active":"2026-02-24T22:31:54.734254+0000","last_peered":"2026-02-24T22:31:54.734254+0000","last_clean":"2026-02-24T22:31:54.734254+0000","last_became_active":"2026-02-24T22:05:20.187327+0000","last_became_peered":"2026-02-24T22:05:20.187327+0000","last_unstale":"2026-02-24T22:31:54.734254+0000","last_undegraded":"2026-02-24T22:31:54.734254+0000","last_fullsized":"2026-02-24T22:31:54.734254+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:09:52.559206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"312'791","reported_seq":2117,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.545233+0000","last_change":"2026-02-24T22:05:22.191261+0000","last_active":"2026-02-24T22:34:18.545233+0000","last_peered":"2026-02-24T22:34:18.545233+0000","last_clean":"2026-02-24T22:34:18.545233+0000","last_became_active":"2026-02-24T22:05:22.191073+0000","last_became_peered":"2026-02-24T22:05:22.191073+0000","last_unstale":"2026-02-24T22:34:18.545233+0000","last_undegraded":"2026-02-24T22:34:18.545233+0000","last_fullsized":"2026-02-24T22:34:18.545233+0000","mapping_epoch":28,"log_start":"307'700","ondisk_log_start":"307'700","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":700,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:26:27.583739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":460,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2922,"num_read_kb":1499,"num_write":81,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"312'204","reported_seq":1716,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.409302+0000","last_change":"2026-02-24T22:05:24.206373+0000","last_active":"2026-02-24T22:34:17.409302+0000","last_peered":"2026-02-24T22:34:17.409302+0000","last_clean":"2026-02-24T22:34:17.409302+0000","last_became_active":"2026-02-24T22:05:24.206134+0000","last_became_peered":"2026-02-24T22:05:24.206134+0000","last_unstale":"2026-02-24T22:34:17.409302+0000","last_undegraded":"2026-02-24T22:34:17.409302+0000","last_fullsized":"2026-02-24T22:34:17.409302+0000","mapping_epoch":30,"log_start":"37'100","ondisk_log_start":"37'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:32:52.956429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33889,"num_objects":107,"num_object_clones":0,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":0,"num_read":1064,"num_read_kb":1063,"num_write":484,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.733657+0000","last_change":"2026-02-24T22:05:29.658922+0000","last_active":"2026-02-24T22:30:29.733657+0000","last_peered":"2026-02-24T22:30:29.733657+0000","last_clean":"2026-02-24T22:30:29.733657+0000","last_became_active":"2026-02-24T22:05:29.658119+0000","last_became_peered":"2026-02-24T22:05:29.658119+0000","last_unstale":"2026-02-24T22:30:29.733657+0000","last_undegraded":"2026-02-24T22:30:29.733657+0000","last_fullsized":"2026-02-24T22:30:29.733657+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:10:58.821048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"312'668","reported_seq":1465,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:09.715537+0000","last_change":"2026-02-24T22:06:00.662810+0000","last_active":"2026-02-24T22:30:09.715537+0000","last_peered":"2026-02-24T22:30:09.715537+0000","last_clean":"2026-02-24T22:30:09.715537+0000","last_became_active":"2026-02-24T22:06:00.662609+0000","last_became_peered":"2026-02-24T22:06:00.662609+0000","last_unstale":"2026-02-24T22:30:09.715537+0000","last_undegraded":"2026-02-24T22:30:09.715537+0000","last_fullsized":"2026-02-24T22:30:09.715537+0000","mapping_epoch":36,"log_start":"312'600","ondisk_log_start":"312'600","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":600,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:42:35.250489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":2886,"num_read_kb":2388,"num_write":1101,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"312'241","reported_seq":967,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:45.705330+0000","last_change":"2026-02-24T22:05:16.190904+0000","last_active":"2026-02-24T22:33:45.705330+0000","last_peered":"2026-02-24T22:33:45.705330+0000","last_clean":"2026-02-24T22:33:45.705330+0000","last_became_active":"2026-02-24T22:05:16.189739+0000","last_became_peered":"2026-02-24T22:05:16.189739+0000","last_unstale":"2026-02-24T22:33:45.705330+0000","last_undegraded":"2026-02-24T22:33:45.705330+0000","last_fullsized":"2026-02-24T22:33:45.705330+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":200,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:04:06.385930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11122410,"num_objects":84,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":391,"num_read_kb":16925,"num_write":1300,"num_write_kb":17766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":579,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.745280+0000","last_change":"2026-02-24T22:05:20.193438+0000","last_active":"2026-02-24T22:31:54.745280+0000","last_peered":"2026-02-24T22:31:54.745280+0000","last_clean":"2026-02-24T22:31:54.745280+0000","last_became_active":"2026-02-24T22:05:20.193200+0000","last_became_peered":"2026-02-24T22:05:20.193200+0000","last_unstale":"2026-02-24T22:31:54.745280+0000","last_undegraded":"2026-02-24T22:31:54.745280+0000","last_fullsized":"2026-02-24T22:31:54.745280+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:58:09.212117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"312'187","reported_seq":785,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.357268+0000","last_change":"2026-02-24T22:05:22.913003+0000","last_active":"2026-02-24T22:34:18.357268+0000","last_peered":"2026-02-24T22:34:18.357268+0000","last_clean":"2026-02-24T22:34:18.357268+0000","last_became_active":"2026-02-24T22:05:22.912826+0000","last_became_peered":"2026-02-24T22:05:22.912826+0000","last_unstale":"2026-02-24T22:34:18.357268+0000","last_undegraded":"2026-02-24T22:34:18.357268+0000","last_fullsized":"2026-02-24T22:34:18.357268+0000","mapping_epoch":28,"log_start":"312'100","ondisk_log_start":"312'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:49:38.039509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47409,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":295,"num_read_kb":667,"num_write":278,"num_write_kb":255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"312'257","reported_seq":903,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:13.928723+0000","last_change":"2026-02-24T22:05:24.199267+0000","last_active":"2026-02-24T22:34:13.928723+0000","last_peered":"2026-02-24T22:34:13.928723+0000","last_clean":"2026-02-24T22:34:13.928723+0000","last_became_active":"2026-02-24T22:05:24.198964+0000","last_became_peered":"2026-02-24T22:05:24.198964+0000","last_unstale":"2026-02-24T22:34:13.928723+0000","last_undegraded":"2026-02-24T22:34:13.928723+0000","last_fullsized":"2026-02-24T22:34:13.928723+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":200,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:47:04.450411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29433,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":428,"num_read_kb":365,"num_write":612,"num_write_kb":187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"312'113","reported_seq":839,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:55.982068+0000","last_change":"2026-02-24T22:05:18.190057+0000","last_active":"2026-02-24T22:33:55.982068+0000","last_peered":"2026-02-24T22:33:55.982068+0000","last_clean":"2026-02-24T22:33:55.982068+0000","last_became_active":"2026-02-24T22:05:18.189796+0000","last_became_peered":"2026-02-24T22:05:18.189796+0000","last_unstale":"2026-02-24T22:33:55.982068+0000","last_undegraded":"2026-02-24T22:33:55.982068+0000","last_fullsized":"2026-02-24T22:33:55.982068+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":100,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:22:22.190649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":455,"num_read_kb":399,"num_write":145,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"312'189","reported_seq":868,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:51.001577+0000","last_change":"2026-02-24T22:05:16.192443+0000","last_active":"2026-02-24T22:30:51.001577+0000","last_peered":"2026-02-24T22:30:51.001577+0000","last_clean":"2026-02-24T22:30:51.001577+0000","last_became_active":"2026-02-24T22:05:16.191985+0000","last_became_peered":"2026-02-24T22:05:16.191985+0000","last_unstale":"2026-02-24T22:30:51.001577+0000","last_undegraded":"2026-02-24T22:30:51.001577+0000","last_fullsized":"2026-02-24T22:30:51.001577+0000","mapping_epoch":22,"log_start":"312'100","ondisk_log_start":"312'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:48:03.816342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4344205,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":288,"num_read_kb":4498,"num_write":974,"num_write_kb":4311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":586,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.742035+0000","last_change":"2026-02-24T22:05:20.194496+0000","last_active":"2026-02-24T22:31:19.742035+0000","last_peered":"2026-02-24T22:31:19.742035+0000","last_clean":"2026-02-24T22:31:19.742035+0000","last_became_active":"2026-02-24T22:05:20.194288+0000","last_became_peered":"2026-02-24T22:05:20.194288+0000","last_unstale":"2026-02-24T22:31:19.742035+0000","last_undegraded":"2026-02-24T22:31:19.742035+0000","last_fullsized":"2026-02-24T22:31:19.742035+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:55:32.351699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"312'255","reported_seq":879,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.223994+0000","last_change":"2026-02-24T22:05:22.913184+0000","last_active":"2026-02-24T22:34:18.223994+0000","last_peered":"2026-02-24T22:34:18.223994+0000","last_clean":"2026-02-24T22:34:18.223994+0000","last_became_active":"2026-02-24T22:05:22.912859+0000","last_became_peered":"2026-02-24T22:05:22.912859+0000","last_unstale":"2026-02-24T22:34:18.223994+0000","last_undegraded":"2026-02-24T22:34:18.223994+0000","last_fullsized":"2026-02-24T22:34:18.223994+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:37:47.377695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53011,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":405,"num_read_kb":789,"num_write":356,"num_write_kb":269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"312'315","reported_seq":1010,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.329842+0000","last_change":"2026-02-24T22:05:24.200453+0000","last_active":"2026-02-24T22:34:15.329842+0000","last_peered":"2026-02-24T22:34:15.329842+0000","last_clean":"2026-02-24T22:34:15.329842+0000","last_became_active":"2026-02-24T22:05:24.199772+0000","last_became_peered":"2026-02-24T22:05:24.199772+0000","last_unstale":"2026-02-24T22:34:15.329842+0000","last_undegraded":"2026-02-24T22:34:15.329842+0000","last_fullsized":"2026-02-24T22:34:15.329842+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:08:33.217544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30266,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":473,"num_read_kb":498,"num_write":701,"num_write_kb":232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":566,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.735506+0000","last_change":"2026-02-24T22:05:29.661914+0000","last_active":"2026-02-24T22:30:29.735506+0000","last_peered":"2026-02-24T22:30:29.735506+0000","last_clean":"2026-02-24T22:30:29.735506+0000","last_became_active":"2026-02-24T22:05:29.661401+0000","last_became_peered":"2026-02-24T22:05:29.661401+0000","last_unstale":"2026-02-24T22:30:29.735506+0000","last_undegraded":"2026-02-24T22:30:29.735506+0000","last_fullsized":"2026-02-24T22:30:29.735506+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:09:29.373749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"312'84","reported_seq":745,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.739343+0000","last_change":"2026-02-24T22:05:18.193728+0000","last_active":"2026-02-24T22:31:19.739343+0000","last_peered":"2026-02-24T22:31:19.739343+0000","last_clean":"2026-02-24T22:31:19.739343+0000","last_became_active":"2026-02-24T22:05:18.193188+0000","last_became_peered":"2026-02-24T22:05:18.193188+0000","last_unstale":"2026-02-24T22:31:19.739343+0000","last_undegraded":"2026-02-24T22:31:19.739343+0000","last_fullsized":"2026-02-24T22:31:19.739343+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:53:00.913559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":312,"num_read_kb":260,"num_write":126,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"312'245","reported_seq":976,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:58.531217+0000","last_change":"2026-02-24T22:05:16.192300+0000","last_active":"2026-02-24T22:33:58.531217+0000","last_peered":"2026-02-24T22:33:58.531217+0000","last_clean":"2026-02-24T22:33:58.531217+0000","last_became_active":"2026-02-24T22:05:16.191956+0000","last_became_peered":"2026-02-24T22:05:16.191956+0000","last_unstale":"2026-02-24T22:33:58.531217+0000","last_undegraded":"2026-02-24T22:33:58.531217+0000","last_fullsized":"2026-02-24T22:33:58.531217+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":200,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:08:01.293337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10490308,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":447,"num_read_kb":10655,"num_write":1344,"num_write_kb":15460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":594,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.722249+0000","last_change":"2026-02-24T22:05:20.186383+0000","last_active":"2026-02-24T22:31:19.722249+0000","last_peered":"2026-02-24T22:31:19.722249+0000","last_clean":"2026-02-24T22:31:19.722249+0000","last_became_active":"2026-02-24T22:05:20.186055+0000","last_became_peered":"2026-02-24T22:05:20.186055+0000","last_unstale":"2026-02-24T22:31:19.722249+0000","last_undegraded":"2026-02-24T22:31:19.722249+0000","last_fullsized":"2026-02-24T22:31:19.722249+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:31:36.594293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"312'224","reported_seq":827,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.246141+0000","last_change":"2026-02-24T22:05:22.194034+0000","last_active":"2026-02-24T22:34:18.246141+0000","last_peered":"2026-02-24T22:34:18.246141+0000","last_clean":"2026-02-24T22:34:18.246141+0000","last_became_active":"2026-02-24T22:05:22.193741+0000","last_became_peered":"2026-02-24T22:05:22.193741+0000","last_unstale":"2026-02-24T22:34:18.246141+0000","last_undegraded":"2026-02-24T22:34:18.246141+0000","last_fullsized":"2026-02-24T22:34:18.246141+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":200,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:35:27.895920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61189,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":372,"num_read_kb":853,"num_write":354,"num_write_kb":326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"312'185","reported_seq":809,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.176596+0000","last_change":"2026-02-24T22:05:24.199051+0000","last_active":"2026-02-24T22:34:15.176596+0000","last_peered":"2026-02-24T22:34:15.176596+0000","last_clean":"2026-02-24T22:34:15.176596+0000","last_became_active":"2026-02-24T22:05:24.198731+0000","last_became_peered":"2026-02-24T22:05:24.198731+0000","last_unstale":"2026-02-24T22:34:15.176596+0000","last_undegraded":"2026-02-24T22:34:15.176596+0000","last_fullsized":"2026-02-24T22:34:15.176596+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:42:29.899927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30034,"num_objects":91,"num_object_clones":0,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":241,"num_read_kb":207,"num_write":470,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":570,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:34.732629+0000","last_change":"2026-02-24T22:05:29.661462+0000","last_active":"2026-02-24T22:30:34.732629+0000","last_peered":"2026-02-24T22:30:34.732629+0000","last_clean":"2026-02-24T22:30:34.732629+0000","last_became_active":"2026-02-24T22:05:29.661161+0000","last_became_peered":"2026-02-24T22:05:29.661161+0000","last_unstale":"2026-02-24T22:30:34.732629+0000","last_undegraded":"2026-02-24T22:30:34.732629+0000","last_fullsized":"2026-02-24T22:30:34.732629+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:23:22.386231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"312'106","reported_seq":752,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.738326+0000","last_change":"2026-02-24T22:05:18.191843+0000","last_active":"2026-02-24T22:31:24.738326+0000","last_peered":"2026-02-24T22:31:24.738326+0000","last_clean":"2026-02-24T22:31:24.738326+0000","last_became_active":"2026-02-24T22:05:18.191555+0000","last_became_peered":"2026-02-24T22:05:18.191555+0000","last_unstale":"2026-02-24T22:31:24.738326+0000","last_undegraded":"2026-02-24T22:31:24.738326+0000","last_fullsized":"2026-02-24T22:31:24.738326+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:53:20.472053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":387,"num_read_kb":325,"num_write":162,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"312'211","reported_seq":924,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:57.361747+0000","last_change":"2026-02-24T22:05:16.192146+0000","last_active":"2026-02-24T22:30:57.361747+0000","last_peered":"2026-02-24T22:30:57.361747+0000","last_clean":"2026-02-24T22:30:57.361747+0000","last_became_active":"2026-02-24T22:05:16.191733+0000","last_became_peered":"2026-02-24T22:05:16.191733+0000","last_unstale":"2026-02-24T22:30:57.361747+0000","last_undegraded":"2026-02-24T22:30:57.361747+0000","last_fullsized":"2026-02-24T22:30:57.361747+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":200,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:27:59.980345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10494886,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":325,"num_read_kb":31022,"num_write":976,"num_write_kb":10319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":593,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.715007+0000","last_change":"2026-02-24T22:05:20.187327+0000","last_active":"2026-02-24T22:30:04.715007+0000","last_peered":"2026-02-24T22:30:04.715007+0000","last_clean":"2026-02-24T22:30:04.715007+0000","last_became_active":"2026-02-24T22:05:20.186892+0000","last_became_peered":"2026-02-24T22:05:20.186892+0000","last_unstale":"2026-02-24T22:30:04.715007+0000","last_undegraded":"2026-02-24T22:30:04.715007+0000","last_fullsized":"2026-02-24T22:30:04.715007+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:53:23.369187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"312'233","reported_seq":841,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.828563+0000","last_change":"2026-02-24T22:05:22.200501+0000","last_active":"2026-02-24T22:34:17.828563+0000","last_peered":"2026-02-24T22:34:17.828563+0000","last_clean":"2026-02-24T22:34:17.828563+0000","last_became_active":"2026-02-24T22:05:22.200208+0000","last_became_peered":"2026-02-24T22:05:22.200208+0000","last_unstale":"2026-02-24T22:34:17.828563+0000","last_undegraded":"2026-02-24T22:34:17.828563+0000","last_fullsized":"2026-02-24T22:34:17.828563+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":200,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:45:47.054214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57434,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":377,"num_read_kb":830,"num_write":350,"num_write_kb":308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"312'275","reported_seq":1084,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.547089+0000","last_change":"2026-02-24T22:05:24.200107+0000","last_active":"2026-02-24T22:34:18.547089+0000","last_peered":"2026-02-24T22:34:18.547089+0000","last_clean":"2026-02-24T22:34:18.547089+0000","last_became_active":"2026-02-24T22:05:24.199525+0000","last_became_peered":"2026-02-24T22:05:24.199525+0000","last_unstale":"2026-02-24T22:34:18.547089+0000","last_undegraded":"2026-02-24T22:34:18.547089+0000","last_fullsized":"2026-02-24T22:34:18.547089+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":200,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:06:50.680774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32009,"num_objects":103,"num_object_clones":0,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":506,"num_read_kb":490,"num_write":606,"num_write_kb":207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:29.740751+0000","last_change":"2026-02-24T22:05:29.652942+0000","last_active":"2026-02-24T22:31:29.740751+0000","last_peered":"2026-02-24T22:31:29.740751+0000","last_clean":"2026-02-24T22:31:29.740751+0000","last_became_active":"2026-02-24T22:05:29.652598+0000","last_became_peered":"2026-02-24T22:05:29.652598+0000","last_unstale":"2026-02-24T22:31:29.740751+0000","last_undegraded":"2026-02-24T22:31:29.740751+0000","last_fullsized":"2026-02-24T22:31:29.740751+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:38:19.564467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"312'169","reported_seq":870,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:07.869080+0000","last_change":"2026-02-24T22:05:18.185305+0000","last_active":"2026-02-24T22:30:07.869080+0000","last_peered":"2026-02-24T22:30:07.869080+0000","last_clean":"2026-02-24T22:30:07.869080+0000","last_became_active":"2026-02-24T22:05:18.185152+0000","last_became_peered":"2026-02-24T22:05:18.185152+0000","last_unstale":"2026-02-24T22:30:07.869080+0000","last_undegraded":"2026-02-24T22:30:07.869080+0000","last_fullsized":"2026-02-24T22:30:07.869080+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:54:16.288957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":651,"num_read_kb":540,"num_write":239,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"312'258","reported_seq":976,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:12.419421+0000","last_change":"2026-02-24T22:05:16.190979+0000","last_active":"2026-02-24T22:34:12.419421+0000","last_peered":"2026-02-24T22:34:12.419421+0000","last_clean":"2026-02-24T22:34:12.419421+0000","last_became_active":"2026-02-24T22:05:16.190205+0000","last_became_peered":"2026-02-24T22:05:16.190205+0000","last_unstale":"2026-02-24T22:34:12.419421+0000","last_undegraded":"2026-02-24T22:34:12.419421+0000","last_fullsized":"2026-02-24T22:34:12.419421+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:07:24.667384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7827201,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":516,"num_read_kb":20088,"num_write":1234,"num_write_kb":14513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":594,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.722800+0000","last_change":"2026-02-24T22:05:20.186884+0000","last_active":"2026-02-24T22:31:24.722800+0000","last_peered":"2026-02-24T22:31:24.722800+0000","last_clean":"2026-02-24T22:31:24.722800+0000","last_became_active":"2026-02-24T22:05:20.186781+0000","last_became_peered":"2026-02-24T22:05:20.186781+0000","last_unstale":"2026-02-24T22:31:24.722800+0000","last_undegraded":"2026-02-24T22:31:24.722800+0000","last_fullsized":"2026-02-24T22:31:24.722800+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:19:52.348059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"312'391","reported_seq":1001,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.367184+0000","last_change":"2026-02-24T22:05:22.192711+0000","last_active":"2026-02-24T22:34:18.367184+0000","last_peered":"2026-02-24T22:34:18.367184+0000","last_clean":"2026-02-24T22:34:18.367184+0000","last_became_active":"2026-02-24T22:05:22.192495+0000","last_became_peered":"2026-02-24T22:05:22.192495+0000","last_unstale":"2026-02-24T22:34:18.367184+0000","last_undegraded":"2026-02-24T22:34:18.367184+0000","last_fullsized":"2026-02-24T22:34:18.367184+0000","mapping_epoch":28,"log_start":"312'300","ondisk_log_start":"312'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":300,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:40:20.670168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118238,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":673,"num_read_kb":1594,"num_write":646,"num_write_kb":622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"312'173","reported_seq":825,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:03.064297+0000","last_change":"2026-02-24T22:05:24.198313+0000","last_active":"2026-02-24T22:34:03.064297+0000","last_peered":"2026-02-24T22:34:03.064297+0000","last_clean":"2026-02-24T22:34:03.064297+0000","last_became_active":"2026-02-24T22:05:24.198114+0000","last_became_peered":"2026-02-24T22:05:24.198114+0000","last_unstale":"2026-02-24T22:34:03.064297+0000","last_undegraded":"2026-02-24T22:34:03.064297+0000","last_fullsized":"2026-02-24T22:34:03.064297+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":100,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:33:14.283058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29501,"num_objects":91,"num_object_clones":0,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":235,"num_read_kb":226,"num_write":429,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:29.726754+0000","last_change":"2026-02-24T22:05:29.652942+0000","last_active":"2026-02-24T22:31:29.726754+0000","last_peered":"2026-02-24T22:31:29.726754+0000","last_clean":"2026-02-24T22:31:29.726754+0000","last_became_active":"2026-02-24T22:05:29.652715+0000","last_became_peered":"2026-02-24T22:05:29.652715+0000","last_unstale":"2026-02-24T22:31:29.726754+0000","last_undegraded":"2026-02-24T22:31:29.726754+0000","last_fullsized":"2026-02-24T22:31:29.726754+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:13:37.213497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"312'571","reported_seq":1430,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:47.451269+0000","last_change":"2026-02-24T22:05:18.193095+0000","last_active":"2026-02-24T22:30:47.451269+0000","last_peered":"2026-02-24T22:30:47.451269+0000","last_clean":"2026-02-24T22:30:47.451269+0000","last_became_active":"2026-02-24T22:05:18.192543+0000","last_became_peered":"2026-02-24T22:05:18.192543+0000","last_unstale":"2026-02-24T22:30:47.451269+0000","last_undegraded":"2026-02-24T22:30:47.451269+0000","last_fullsized":"2026-02-24T22:30:47.451269+0000","mapping_epoch":24,"log_start":"312'500","ondisk_log_start":"312'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":500,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:42:22.975827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":2576,"num_read_kb":2171,"num_write":966,"num_write_kb":601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"312'189","reported_seq":863,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:55.054639+0000","last_change":"2026-02-24T22:05:16.184025+0000","last_active":"2026-02-24T22:33:55.054639+0000","last_peered":"2026-02-24T22:33:55.054639+0000","last_clean":"2026-02-24T22:33:55.054639+0000","last_became_active":"2026-02-24T22:05:16.182928+0000","last_became_peered":"2026-02-24T22:05:16.182928+0000","last_unstale":"2026-02-24T22:33:55.054639+0000","last_undegraded":"2026-02-24T22:33:55.054639+0000","last_fullsized":"2026-02-24T22:33:55.054639+0000","mapping_epoch":22,"log_start":"312'100","ondisk_log_start":"312'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:19:49.977609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":119466,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":254,"num_read_kb":262,"num_write":1021,"num_write_kb":181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":593,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.715020+0000","last_change":"2026-02-24T22:05:20.186597+0000","last_active":"2026-02-24T22:30:04.715020+0000","last_peered":"2026-02-24T22:30:04.715020+0000","last_clean":"2026-02-24T22:30:04.715020+0000","last_became_active":"2026-02-24T22:05:20.186209+0000","last_became_peered":"2026-02-24T22:05:20.186209+0000","last_unstale":"2026-02-24T22:30:04.715020+0000","last_undegraded":"2026-02-24T22:30:04.715020+0000","last_fullsized":"2026-02-24T22:30:04.715020+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:26:06.692781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"312'940","reported_seq":2252,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.520918+0000","last_change":"2026-02-24T22:05:22.200373+0000","last_active":"2026-02-24T22:34:18.520918+0000","last_peered":"2026-02-24T22:34:18.520918+0000","last_clean":"2026-02-24T22:34:18.520918+0000","last_became_active":"2026-02-24T22:05:22.200111+0000","last_became_peered":"2026-02-24T22:05:22.200111+0000","last_unstale":"2026-02-24T22:34:18.520918+0000","last_undegraded":"2026-02-24T22:34:18.520918+0000","last_fullsized":"2026-02-24T22:34:18.520918+0000","mapping_epoch":28,"log_start":"312'900","ondisk_log_start":"312'900","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":900,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:33:19.581577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54380,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":3207,"num_read_kb":2223,"num_write":413,"num_write_kb":370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"312'183","reported_seq":775,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.249482+0000","last_change":"2026-02-24T22:05:24.201417+0000","last_active":"2026-02-24T22:34:17.249482+0000","last_peered":"2026-02-24T22:34:17.249482+0000","last_clean":"2026-02-24T22:34:17.249482+0000","last_became_active":"2026-02-24T22:05:24.200878+0000","last_became_peered":"2026-02-24T22:05:24.200878+0000","last_unstale":"2026-02-24T22:34:17.249482+0000","last_undegraded":"2026-02-24T22:34:17.249482+0000","last_fullsized":"2026-02-24T22:34:17.249482+0000","mapping_epoch":30,"log_start":"37'100","ondisk_log_start":"37'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":100,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:01:48.708129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33021,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":110,"num_read_kb":109,"num_write":447,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"312'435","reported_seq":2196,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.586410+0000","last_change":"2026-02-24T22:05:29.654545+0000","last_active":"2026-02-24T22:34:18.586410+0000","last_peered":"2026-02-24T22:34:18.586410+0000","last_clean":"2026-02-24T22:34:18.586410+0000","last_became_active":"2026-02-24T22:05:29.654064+0000","last_became_peered":"2026-02-24T22:05:29.654064+0000","last_unstale":"2026-02-24T22:34:18.586410+0000","last_undegraded":"2026-02-24T22:34:18.586410+0000","last_fullsized":"2026-02-24T22:34:18.586410+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":400,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:01:24.704003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"312'128","reported_seq":810,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.731309+0000","last_change":"2026-02-24T22:05:18.192490+0000","last_active":"2026-02-24T22:30:04.731309+0000","last_peered":"2026-02-24T22:30:04.731309+0000","last_clean":"2026-02-24T22:30:04.731309+0000","last_became_active":"2026-02-24T22:05:18.191055+0000","last_became_peered":"2026-02-24T22:05:18.191055+0000","last_unstale":"2026-02-24T22:30:04.731309+0000","last_undegraded":"2026-02-24T22:30:04.731309+0000","last_fullsized":"2026-02-24T22:30:04.731309+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:47:55.429098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":486,"num_read_kb":398,"num_write":170,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"312'174","reported_seq":881,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:06.202574+0000","last_change":"2026-02-24T22:05:16.187505+0000","last_active":"2026-02-24T22:34:06.202574+0000","last_peered":"2026-02-24T22:34:06.202574+0000","last_clean":"2026-02-24T22:34:06.202574+0000","last_became_active":"2026-02-24T22:05:16.187261+0000","last_became_peered":"2026-02-24T22:05:16.187261+0000","last_unstale":"2026-02-24T22:34:06.202574+0000","last_undegraded":"2026-02-24T22:34:06.202574+0000","last_fullsized":"2026-02-24T22:34:06.202574+0000","mapping_epoch":22,"log_start":"312'100","ondisk_log_start":"312'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:13:37.546225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10744628,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":306,"num_read_kb":24764,"num_write":893,"num_write_kb":10562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":588,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.718234+0000","last_change":"2026-02-24T22:05:20.187956+0000","last_active":"2026-02-24T22:30:04.718234+0000","last_peered":"2026-02-24T22:30:04.718234+0000","last_clean":"2026-02-24T22:30:04.718234+0000","last_became_active":"2026-02-24T22:05:20.187488+0000","last_became_peered":"2026-02-24T22:05:20.187488+0000","last_unstale":"2026-02-24T22:30:04.718234+0000","last_undegraded":"2026-02-24T22:30:04.718234+0000","last_fullsized":"2026-02-24T22:30:04.718234+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:08:51.284932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"312'462","reported_seq":1312,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.531191+0000","last_change":"2026-02-24T22:05:22.193340+0000","last_active":"2026-02-24T22:34:18.531191+0000","last_peered":"2026-02-24T22:34:18.531191+0000","last_clean":"2026-02-24T22:34:18.531191+0000","last_became_active":"2026-02-24T22:05:22.193142+0000","last_became_peered":"2026-02-24T22:05:22.193142+0000","last_unstale":"2026-02-24T22:34:18.531191+0000","last_undegraded":"2026-02-24T22:34:18.531191+0000","last_fullsized":"2026-02-24T22:34:18.531191+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":400,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:37:15.676612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51399,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1291,"num_read_kb":1227,"num_write":329,"num_write_kb":279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"312'222","reported_seq":821,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.886791+0000","last_change":"2026-02-24T22:05:24.198994+0000","last_active":"2026-02-24T22:34:17.886791+0000","last_peered":"2026-02-24T22:34:17.886791+0000","last_clean":"2026-02-24T22:34:17.886791+0000","last_became_active":"2026-02-24T22:05:24.198669+0000","last_became_peered":"2026-02-24T22:05:24.198669+0000","last_unstale":"2026-02-24T22:34:17.886791+0000","last_undegraded":"2026-02-24T22:34:17.886791+0000","last_fullsized":"2026-02-24T22:34:17.886791+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":200,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:38:36.897067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29328,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":170,"num_read_kb":165,"num_write":551,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.720723+0000","last_change":"2026-02-24T22:05:29.653411+0000","last_active":"2026-02-24T22:30:29.720723+0000","last_peered":"2026-02-24T22:30:29.720723+0000","last_clean":"2026-02-24T22:30:29.720723+0000","last_became_active":"2026-02-24T22:05:29.653144+0000","last_became_peered":"2026-02-24T22:05:29.653144+0000","last_unstale":"2026-02-24T22:30:29.720723+0000","last_undegraded":"2026-02-24T22:30:29.720723+0000","last_fullsized":"2026-02-24T22:30:29.720723+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:09:20.796950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"312'171","reported_seq":875,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:14.602760+0000","last_change":"2026-02-24T22:05:18.191001+0000","last_active":"2026-02-24T22:34:14.602760+0000","last_peered":"2026-02-24T22:34:14.602760+0000","last_clean":"2026-02-24T22:34:14.602760+0000","last_became_active":"2026-02-24T22:05:18.190140+0000","last_became_peered":"2026-02-24T22:05:18.190140+0000","last_unstale":"2026-02-24T22:34:14.602760+0000","last_undegraded":"2026-02-24T22:34:14.602760+0000","last_fullsized":"2026-02-24T22:34:14.602760+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:34:38.707313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":675,"num_read_kb":556,"num_write":238,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"312'231","reported_seq":992,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:59.603149+0000","last_change":"2026-02-24T22:05:16.185212+0000","last_active":"2026-02-24T22:33:59.603149+0000","last_peered":"2026-02-24T22:33:59.603149+0000","last_clean":"2026-02-24T22:33:59.603149+0000","last_became_active":"2026-02-24T22:05:16.184944+0000","last_became_peered":"2026-02-24T22:05:16.184944+0000","last_unstale":"2026-02-24T22:33:59.603149+0000","last_undegraded":"2026-02-24T22:33:59.603149+0000","last_fullsized":"2026-02-24T22:33:59.603149+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":200,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:42:08.775595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17390927,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":470,"num_read_kb":45888,"num_write":1286,"num_write_kb":20430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"27'2","reported_seq":1055,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.487522+0000","last_change":"2026-02-24T22:05:20.194961+0000","last_active":"2026-02-24T22:34:18.487522+0000","last_peered":"2026-02-24T22:34:18.487522+0000","last_clean":"2026-02-24T22:34:18.487522+0000","last_became_active":"2026-02-24T22:05:20.194145+0000","last_became_peered":"2026-02-24T22:05:20.194145+0000","last_unstale":"2026-02-24T22:34:18.487522+0000","last_undegraded":"2026-02-24T22:34:18.487522+0000","last_fullsized":"2026-02-24T22:34:18.487522+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:18:36.330182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":474,"num_read_kb":474,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"312'56","reported_seq":657,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.737323+0000","last_change":"2026-02-24T22:05:22.905477+0000","last_active":"2026-02-24T22:32:24.737323+0000","last_peered":"2026-02-24T22:32:24.737323+0000","last_clean":"2026-02-24T22:32:24.737323+0000","last_became_active":"2026-02-24T22:05:22.905221+0000","last_became_peered":"2026-02-24T22:05:22.905221+0000","last_unstale":"2026-02-24T22:32:24.737323+0000","last_undegraded":"2026-02-24T22:32:24.737323+0000","last_fullsized":"2026-02-24T22:32:24.737323+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:59:47.641153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"312'159","reported_seq":744,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:14.502434+0000","last_change":"2026-02-24T22:05:24.208011+0000","last_active":"2026-02-24T22:34:14.502434+0000","last_peered":"2026-02-24T22:34:14.502434+0000","last_clean":"2026-02-24T22:34:14.502434+0000","last_became_active":"2026-02-24T22:05:24.207673+0000","last_became_peered":"2026-02-24T22:05:24.207673+0000","last_unstale":"2026-02-24T22:34:14.502434+0000","last_undegraded":"2026-02-24T22:34:14.502434+0000","last_fullsized":"2026-02-24T22:34:14.502434+0000","mapping_epoch":30,"log_start":"312'100","ondisk_log_start":"312'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:29:55.011561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25987,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":107,"num_read_kb":103,"num_write":388,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":577,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.733645+0000","last_change":"2026-02-24T22:05:29.659816+0000","last_active":"2026-02-24T22:30:29.733645+0000","last_peered":"2026-02-24T22:30:29.733645+0000","last_clean":"2026-02-24T22:30:29.733645+0000","last_became_active":"2026-02-24T22:05:29.658842+0000","last_became_peered":"2026-02-24T22:05:29.658842+0000","last_unstale":"2026-02-24T22:30:29.733645+0000","last_undegraded":"2026-02-24T22:30:29.733645+0000","last_fullsized":"2026-02-24T22:30:29.733645+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:55:04.860078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"312'98","reported_seq":778,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:54.820404+0000","last_change":"2026-02-24T22:05:18.188396+0000","last_active":"2026-02-24T22:30:54.820404+0000","last_peered":"2026-02-24T22:30:54.820404+0000","last_clean":"2026-02-24T22:30:54.820404+0000","last_became_active":"2026-02-24T22:05:18.188161+0000","last_became_peered":"2026-02-24T22:05:18.188161+0000","last_unstale":"2026-02-24T22:30:54.820404+0000","last_undegraded":"2026-02-24T22:30:54.820404+0000","last_fullsized":"2026-02-24T22:30:54.820404+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:15:32.648520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":368,"num_read_kb":314,"num_write":134,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"312'239","reported_seq":971,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:49.624638+0000","last_change":"2026-02-24T22:05:16.182699+0000","last_active":"2026-02-24T22:33:49.624638+0000","last_peered":"2026-02-24T22:33:49.624638+0000","last_clean":"2026-02-24T22:33:49.624638+0000","last_became_active":"2026-02-24T22:05:16.182194+0000","last_became_peered":"2026-02-24T22:05:16.182194+0000","last_unstale":"2026-02-24T22:33:49.624638+0000","last_undegraded":"2026-02-24T22:33:49.624638+0000","last_fullsized":"2026-02-24T22:33:49.624638+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":200,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:40:05.294459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5600581,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":390,"num_read_kb":7449,"num_write":1196,"num_write_kb":8933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.731323+0000","last_change":"2026-02-24T22:05:20.193494+0000","last_active":"2026-02-24T22:30:04.731323+0000","last_peered":"2026-02-24T22:30:04.731323+0000","last_clean":"2026-02-24T22:30:04.731323+0000","last_became_active":"2026-02-24T22:05:20.193180+0000","last_became_peered":"2026-02-24T22:05:20.193180+0000","last_unstale":"2026-02-24T22:30:04.731323+0000","last_undegraded":"2026-02-24T22:30:04.731323+0000","last_fullsized":"2026-02-24T22:30:04.731323+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:22:52.729272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"312'270","reported_seq":890,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.361543+0000","last_change":"2026-02-24T22:05:22.200410+0000","last_active":"2026-02-24T22:34:18.361543+0000","last_peered":"2026-02-24T22:34:18.361543+0000","last_clean":"2026-02-24T22:34:18.361543+0000","last_became_active":"2026-02-24T22:05:22.200138+0000","last_became_peered":"2026-02-24T22:05:22.200138+0000","last_unstale":"2026-02-24T22:34:18.361543+0000","last_undegraded":"2026-02-24T22:34:18.361543+0000","last_fullsized":"2026-02-24T22:34:18.361543+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:07:05.040956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52341,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":858,"num_read_kb":1265,"num_write":1270,"num_write_kb":1196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"312'327","reported_seq":951,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.407864+0000","last_change":"2026-02-24T22:05:24.207564+0000","last_active":"2026-02-24T22:34:17.407864+0000","last_peered":"2026-02-24T22:34:17.407864+0000","last_clean":"2026-02-24T22:34:17.407864+0000","last_became_active":"2026-02-24T22:05:24.206745+0000","last_became_peered":"2026-02-24T22:05:24.206745+0000","last_unstale":"2026-02-24T22:34:17.407864+0000","last_undegraded":"2026-02-24T22:34:17.407864+0000","last_fullsized":"2026-02-24T22:34:17.407864+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":300,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:53:45.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27098,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":391,"num_read_kb":376,"num_write":726,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"312'431","reported_seq":4795,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.586354+0000","last_change":"2026-02-24T22:05:29.654842+0000","last_active":"2026-02-24T22:34:18.586354+0000","last_peered":"2026-02-24T22:34:18.586354+0000","last_clean":"2026-02-24T22:34:18.586354+0000","last_became_active":"2026-02-24T22:05:29.654281+0000","last_became_peered":"2026-02-24T22:05:29.654281+0000","last_unstale":"2026-02-24T22:34:18.586354+0000","last_undegraded":"2026-02-24T22:34:18.586354+0000","last_fullsized":"2026-02-24T22:34:18.586354+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":400,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:30:38.038033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"312'114","reported_seq":799,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:05.704625+0000","last_change":"2026-02-24T22:05:18.193654+0000","last_active":"2026-02-24T22:34:05.704625+0000","last_peered":"2026-02-24T22:34:05.704625+0000","last_clean":"2026-02-24T22:34:05.704625+0000","last_became_active":"2026-02-24T22:05:18.193426+0000","last_became_peered":"2026-02-24T22:05:18.193426+0000","last_unstale":"2026-02-24T22:34:05.704625+0000","last_undegraded":"2026-02-24T22:34:05.704625+0000","last_fullsized":"2026-02-24T22:34:05.704625+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:36:20.176524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":443,"num_read_kb":376,"num_write":199,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'112","reported_seq":745,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.741431+0000","last_change":"2026-02-24T22:05:06.709275+0000","last_active":"2026-02-24T22:31:14.741431+0000","last_peered":"2026-02-24T22:31:14.741431+0000","last_clean":"2026-02-24T22:31:14.741431+0000","last_became_active":"2026-02-24T22:05:06.708577+0000","last_became_peered":"2026-02-24T22:05:06.708577+0000","last_unstale":"2026-02-24T22:31:14.741431+0000","last_undegraded":"2026-02-24T22:31:14.741431+0000","last_fullsized":"2026-02-24T22:31:14.741431+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:05.664397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:05.664397+0000","last_clean_scrub_stamp":"2026-02-24T22:05:05.664397+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:46:55.314440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":605,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:14.747819+0000","last_change":"2026-02-24T22:05:10.694707+0000","last_active":"2026-02-24T22:32:14.747819+0000","last_peered":"2026-02-24T22:32:14.747819+0000","last_clean":"2026-02-24T22:32:14.747819+0000","last_became_active":"2026-02-24T22:05:08.689983+0000","last_became_peered":"2026-02-24T22:05:08.689983+0000","last_unstale":"2026-02-24T22:32:14.747819+0000","last_undegraded":"2026-02-24T22:32:14.747819+0000","last_fullsized":"2026-02-24T22:32:14.747819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:42:42.134312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046251899999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"312'265","reported_seq":1043,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:12.240184+0000","last_change":"2026-02-24T22:05:16.186666+0000","last_active":"2026-02-24T22:34:12.240184+0000","last_peered":"2026-02-24T22:34:12.240184+0000","last_clean":"2026-02-24T22:34:12.240184+0000","last_became_active":"2026-02-24T22:05:16.185768+0000","last_became_peered":"2026-02-24T22:05:16.185768+0000","last_unstale":"2026-02-24T22:34:12.240184+0000","last_undegraded":"2026-02-24T22:34:12.240184+0000","last_fullsized":"2026-02-24T22:34:12.240184+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":200,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:33:06.518377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6322908,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":624,"num_read_kb":12841,"num_write":1442,"num_write_kb":6232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":586,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.735364+0000","last_change":"2026-02-24T22:05:20.194414+0000","last_active":"2026-02-24T22:30:19.735364+0000","last_peered":"2026-02-24T22:30:19.735364+0000","last_clean":"2026-02-24T22:30:19.735364+0000","last_became_active":"2026-02-24T22:05:20.194040+0000","last_became_peered":"2026-02-24T22:05:20.194040+0000","last_unstale":"2026-02-24T22:30:19.735364+0000","last_undegraded":"2026-02-24T22:30:19.735364+0000","last_fullsized":"2026-02-24T22:30:19.735364+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:14:22.868157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"312'1049","reported_seq":2141,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.521664+0000","last_change":"2026-02-24T22:05:22.199991+0000","last_active":"2026-02-24T22:34:18.521664+0000","last_peered":"2026-02-24T22:34:18.521664+0000","last_clean":"2026-02-24T22:34:18.521664+0000","last_became_active":"2026-02-24T22:05:22.199602+0000","last_became_peered":"2026-02-24T22:05:22.199602+0000","last_unstale":"2026-02-24T22:34:18.521664+0000","last_undegraded":"2026-02-24T22:34:18.521664+0000","last_fullsized":"2026-02-24T22:34:18.521664+0000","mapping_epoch":28,"log_start":"312'1000","ondisk_log_start":"312'1000","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":1000,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:00:35.201833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163236,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2894,"num_read_kb":3248,"num_write":1008,"num_write_kb":944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"312'202","reported_seq":795,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.242020+0000","last_change":"2026-02-24T22:05:24.201041+0000","last_active":"2026-02-24T22:34:18.242020+0000","last_peered":"2026-02-24T22:34:18.242020+0000","last_clean":"2026-02-24T22:34:18.242020+0000","last_became_active":"2026-02-24T22:05:24.200336+0000","last_became_peered":"2026-02-24T22:05:24.200336+0000","last_unstale":"2026-02-24T22:34:18.242020+0000","last_undegraded":"2026-02-24T22:34:18.242020+0000","last_fullsized":"2026-02-24T22:34:18.242020+0000","mapping_epoch":30,"log_start":"37'100","ondisk_log_start":"37'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:52:42.524705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32535,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":152,"num_read_kb":139,"num_write":501,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"312'424","reported_seq":1832,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.571551+0000","last_change":"2026-02-24T22:05:29.660713+0000","last_active":"2026-02-24T22:34:18.571551+0000","last_peered":"2026-02-24T22:34:18.571551+0000","last_clean":"2026-02-24T22:34:18.571551+0000","last_became_active":"2026-02-24T22:05:29.660370+0000","last_became_peered":"2026-02-24T22:05:29.660370+0000","last_unstale":"2026-02-24T22:34:18.571551+0000","last_undegraded":"2026-02-24T22:34:18.571551+0000","last_fullsized":"2026-02-24T22:34:18.571551+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":400,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:47:59.912901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"312'99","reported_seq":797,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.739591+0000","last_change":"2026-02-24T22:05:18.185501+0000","last_active":"2026-02-24T22:31:19.739591+0000","last_peered":"2026-02-24T22:31:19.739591+0000","last_clean":"2026-02-24T22:31:19.739591+0000","last_became_active":"2026-02-24T22:05:18.185205+0000","last_became_peered":"2026-02-24T22:05:18.185205+0000","last_unstale":"2026-02-24T22:31:19.739591+0000","last_undegraded":"2026-02-24T22:31:19.739591+0000","last_fullsized":"2026-02-24T22:31:19.739591+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:21:17.222637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":346,"num_read_kb":300,"num_write":140,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":610,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.741446+0000","last_change":"2026-02-24T22:05:10.808592+0000","last_active":"2026-02-24T22:31:14.741446+0000","last_peered":"2026-02-24T22:31:14.741446+0000","last_clean":"2026-02-24T22:31:14.741446+0000","last_became_active":"2026-02-24T22:05:08.689972+0000","last_became_peered":"2026-02-24T22:05:08.689972+0000","last_unstale":"2026-02-24T22:31:14.741446+0000","last_undegraded":"2026-02-24T22:31:14.741446+0000","last_fullsized":"2026-02-24T22:31:14.741446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:59:56.330904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048345700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"312'208","reported_seq":966,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:57.024826+0000","last_change":"2026-02-24T22:05:16.183783+0000","last_active":"2026-02-24T22:30:57.024826+0000","last_peered":"2026-02-24T22:30:57.024826+0000","last_clean":"2026-02-24T22:30:57.024826+0000","last_became_active":"2026-02-24T22:05:16.183484+0000","last_became_peered":"2026-02-24T22:05:16.183484+0000","last_unstale":"2026-02-24T22:30:57.024826+0000","last_undegraded":"2026-02-24T22:30:57.024826+0000","last_fullsized":"2026-02-24T22:30:57.024826+0000","mapping_epoch":22,"log_start":"312'100","ondisk_log_start":"312'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":100,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:41:07.836883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5664239,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":524,"num_read_kb":821,"num_write":1237,"num_write_kb":10723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":580,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.741456+0000","last_change":"2026-02-24T22:05:20.195460+0000","last_active":"2026-02-24T22:31:19.741456+0000","last_peered":"2026-02-24T22:31:19.741456+0000","last_clean":"2026-02-24T22:31:19.741456+0000","last_became_active":"2026-02-24T22:05:20.195236+0000","last_became_peered":"2026-02-24T22:05:20.195236+0000","last_unstale":"2026-02-24T22:31:19.741456+0000","last_undegraded":"2026-02-24T22:31:19.741456+0000","last_fullsized":"2026-02-24T22:31:19.741456+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:38:04.823028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"312'1045","reported_seq":2251,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.938361+0000","last_change":"2026-02-24T22:05:22.192588+0000","last_active":"2026-02-24T22:34:18.938361+0000","last_peered":"2026-02-24T22:34:18.938361+0000","last_clean":"2026-02-24T22:34:18.938361+0000","last_became_active":"2026-02-24T22:05:22.192295+0000","last_became_peered":"2026-02-24T22:05:22.192295+0000","last_unstale":"2026-02-24T22:34:18.938361+0000","last_undegraded":"2026-02-24T22:34:18.938361+0000","last_fullsized":"2026-02-24T22:34:18.938361+0000","mapping_epoch":28,"log_start":"312'1001","ondisk_log_start":"312'1001","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":1001,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:00:22.125417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":153342,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":3150,"num_read_kb":4374,"num_write":3940,"num_write_kb":3885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"312'182","reported_seq":763,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.067423+0000","last_change":"2026-02-24T22:05:24.205091+0000","last_active":"2026-02-24T22:34:15.067423+0000","last_peered":"2026-02-24T22:34:15.067423+0000","last_clean":"2026-02-24T22:34:15.067423+0000","last_became_active":"2026-02-24T22:05:24.204774+0000","last_became_peered":"2026-02-24T22:05:24.204774+0000","last_unstale":"2026-02-24T22:34:15.067423+0000","last_undegraded":"2026-02-24T22:34:15.067423+0000","last_fullsized":"2026-02-24T22:34:15.067423+0000","mapping_epoch":30,"log_start":"312'100","ondisk_log_start":"312'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":100,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:38:46.991451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29814,"num_objects":89,"num_object_clones":0,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":0,"num_read":112,"num_read_kb":108,"num_write":454,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.727262+0000","last_change":"2026-02-24T22:05:29.653742+0000","last_active":"2026-02-24T22:31:34.727262+0000","last_peered":"2026-02-24T22:31:34.727262+0000","last_clean":"2026-02-24T22:31:34.727262+0000","last_became_active":"2026-02-24T22:05:29.653391+0000","last_became_peered":"2026-02-24T22:05:29.653391+0000","last_unstale":"2026-02-24T22:31:34.727262+0000","last_undegraded":"2026-02-24T22:31:34.727262+0000","last_fullsized":"2026-02-24T22:31:34.727262+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:11:46.647600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"312'101","reported_seq":791,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.725723+0000","last_change":"2026-02-24T22:05:18.186987+0000","last_active":"2026-02-24T22:31:19.725723+0000","last_peered":"2026-02-24T22:31:19.725723+0000","last_clean":"2026-02-24T22:31:19.725723+0000","last_became_active":"2026-02-24T22:05:18.186126+0000","last_became_peered":"2026-02-24T22:05:18.186126+0000","last_unstale":"2026-02-24T22:31:19.725723+0000","last_undegraded":"2026-02-24T22:31:19.725723+0000","last_fullsized":"2026-02-24T22:31:19.725723+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:14:57.108610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":336,"num_read_kb":289,"num_write":128,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":618,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:14.727952+0000","last_change":"2026-02-24T22:05:10.684538+0000","last_active":"2026-02-24T22:32:14.727952+0000","last_peered":"2026-02-24T22:32:14.727952+0000","last_clean":"2026-02-24T22:32:14.727952+0000","last_became_active":"2026-02-24T22:05:08.682035+0000","last_became_peered":"2026-02-24T22:05:08.682035+0000","last_unstale":"2026-02-24T22:32:14.727952+0000","last_undegraded":"2026-02-24T22:32:14.727952+0000","last_fullsized":"2026-02-24T22:32:14.727952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:59:46.243250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047531599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"312'233","reported_seq":977,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.726180+0000","last_change":"2026-02-24T22:05:16.178900+0000","last_active":"2026-02-24T22:31:24.726180+0000","last_peered":"2026-02-24T22:31:24.726180+0000","last_clean":"2026-02-24T22:31:24.726180+0000","last_became_active":"2026-02-24T22:05:16.178331+0000","last_became_peered":"2026-02-24T22:05:16.178331+0000","last_unstale":"2026-02-24T22:31:24.726180+0000","last_undegraded":"2026-02-24T22:31:24.726180+0000","last_fullsized":"2026-02-24T22:31:24.726180+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":200,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:08:05.851722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11570089,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":524,"num_read_kb":6587,"num_write":1274,"num_write_kb":11354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"27'1","reported_seq":595,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.722286+0000","last_change":"2026-02-24T22:05:20.188183+0000","last_active":"2026-02-24T22:31:19.722286+0000","last_peered":"2026-02-24T22:31:19.722286+0000","last_clean":"2026-02-24T22:31:19.722286+0000","last_became_active":"2026-02-24T22:05:20.187908+0000","last_became_peered":"2026-02-24T22:05:20.187908+0000","last_unstale":"2026-02-24T22:31:19.722286+0000","last_undegraded":"2026-02-24T22:31:19.722286+0000","last_fullsized":"2026-02-24T22:31:19.722286+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:49:17.694606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"312'436","reported_seq":1644,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.921712+0000","last_change":"2026-02-24T22:05:22.912798+0000","last_active":"2026-02-24T22:34:18.921712+0000","last_peered":"2026-02-24T22:34:18.921712+0000","last_clean":"2026-02-24T22:34:18.921712+0000","last_became_active":"2026-02-24T22:05:22.912414+0000","last_became_peered":"2026-02-24T22:05:22.912414+0000","last_unstale":"2026-02-24T22:34:18.921712+0000","last_undegraded":"2026-02-24T22:34:18.921712+0000","last_fullsized":"2026-02-24T22:34:18.921712+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":400,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:05:47.634681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":308,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1076,"num_read_kb":597,"num_write":114,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"312'184","reported_seq":770,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.551095+0000","last_change":"2026-02-24T22:05:24.200671+0000","last_active":"2026-02-24T22:34:18.551095+0000","last_peered":"2026-02-24T22:34:18.551095+0000","last_clean":"2026-02-24T22:34:18.551095+0000","last_became_active":"2026-02-24T22:05:24.200029+0000","last_became_peered":"2026-02-24T22:05:24.200029+0000","last_unstale":"2026-02-24T22:34:18.551095+0000","last_undegraded":"2026-02-24T22:34:18.551095+0000","last_fullsized":"2026-02-24T22:34:18.551095+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":100,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:18:37.550289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29918,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":442,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":576,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.724006+0000","last_change":"2026-02-24T22:05:29.653992+0000","last_active":"2026-02-24T22:30:14.724006+0000","last_peered":"2026-02-24T22:30:14.724006+0000","last_clean":"2026-02-24T22:30:14.724006+0000","last_became_active":"2026-02-24T22:05:29.653806+0000","last_became_peered":"2026-02-24T22:05:29.653806+0000","last_unstale":"2026-02-24T22:30:14.724006+0000","last_undegraded":"2026-02-24T22:30:14.724006+0000","last_fullsized":"2026-02-24T22:30:14.724006+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:44:18.672652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"312'230","reported_seq":970,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:51.044848+0000","last_change":"2026-02-24T22:05:18.184825+0000","last_active":"2026-02-24T22:30:51.044848+0000","last_peered":"2026-02-24T22:30:51.044848+0000","last_clean":"2026-02-24T22:30:51.044848+0000","last_became_active":"2026-02-24T22:05:18.183952+0000","last_became_peered":"2026-02-24T22:05:18.183952+0000","last_unstale":"2026-02-24T22:30:51.044848+0000","last_undegraded":"2026-02-24T22:30:51.044848+0000","last_fullsized":"2026-02-24T22:30:51.044848+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:33:27.005363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":963,"num_read_kb":811,"num_write":374,"num_write_kb":210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":606,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:14.747800+0000","last_change":"2026-02-24T22:05:10.695226+0000","last_active":"2026-02-24T22:32:14.747800+0000","last_peered":"2026-02-24T22:32:14.747800+0000","last_clean":"2026-02-24T22:32:14.747800+0000","last_became_active":"2026-02-24T22:05:08.690042+0000","last_became_peered":"2026-02-24T22:05:08.690042+0000","last_unstale":"2026-02-24T22:32:14.747800+0000","last_undegraded":"2026-02-24T22:32:14.747800+0000","last_fullsized":"2026-02-24T22:32:14.747800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:27:03.349026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010128400000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"312'234","reported_seq":948,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:42.673103+0000","last_change":"2026-02-24T22:05:16.189390+0000","last_active":"2026-02-24T22:33:42.673103+0000","last_peered":"2026-02-24T22:33:42.673103+0000","last_clean":"2026-02-24T22:33:42.673103+0000","last_became_active":"2026-02-24T22:05:16.188900+0000","last_became_peered":"2026-02-24T22:05:16.188900+0000","last_unstale":"2026-02-24T22:33:42.673103+0000","last_undegraded":"2026-02-24T22:33:42.673103+0000","last_fullsized":"2026-02-24T22:33:42.673103+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:22:52.382242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11025931,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":322,"num_read_kb":10969,"num_write":1184,"num_write_kb":10859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":589,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.733650+0000","last_change":"2026-02-24T22:05:20.187605+0000","last_active":"2026-02-24T22:30:24.733650+0000","last_peered":"2026-02-24T22:30:24.733650+0000","last_clean":"2026-02-24T22:30:24.733650+0000","last_became_active":"2026-02-24T22:05:20.187251+0000","last_became_peered":"2026-02-24T22:05:20.187251+0000","last_unstale":"2026-02-24T22:30:24.733650+0000","last_undegraded":"2026-02-24T22:30:24.733650+0000","last_fullsized":"2026-02-24T22:30:24.733650+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:47:16.465774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"312'76","reported_seq":697,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.731188+0000","last_change":"2026-02-24T22:05:22.191860+0000","last_active":"2026-02-24T22:31:24.731188+0000","last_peered":"2026-02-24T22:31:24.731188+0000","last_clean":"2026-02-24T22:31:24.731188+0000","last_became_active":"2026-02-24T22:05:22.191394+0000","last_became_peered":"2026-02-24T22:05:22.191394+0000","last_unstale":"2026-02-24T22:31:24.731188+0000","last_undegraded":"2026-02-24T22:31:24.731188+0000","last_fullsized":"2026-02-24T22:31:24.731188+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:12:53.711063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":103,"num_read_kb":96,"num_write":72,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"312'229","reported_seq":843,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.223179+0000","last_change":"2026-02-24T22:05:24.208017+0000","last_active":"2026-02-24T22:34:17.223179+0000","last_peered":"2026-02-24T22:34:17.223179+0000","last_clean":"2026-02-24T22:34:17.223179+0000","last_became_active":"2026-02-24T22:05:24.207698+0000","last_became_peered":"2026-02-24T22:05:24.207698+0000","last_unstale":"2026-02-24T22:34:17.223179+0000","last_undegraded":"2026-02-24T22:34:17.223179+0000","last_fullsized":"2026-02-24T22:34:17.223179+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":200,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:35:36.488781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31273,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":246,"num_read_kb":224,"num_write":553,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":566,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:29.742534+0000","last_change":"2026-02-24T22:05:29.660487+0000","last_active":"2026-02-24T22:31:29.742534+0000","last_peered":"2026-02-24T22:31:29.742534+0000","last_clean":"2026-02-24T22:31:29.742534+0000","last_became_active":"2026-02-24T22:05:29.660065+0000","last_became_peered":"2026-02-24T22:05:29.660065+0000","last_unstale":"2026-02-24T22:31:29.742534+0000","last_undegraded":"2026-02-24T22:31:29.742534+0000","last_fullsized":"2026-02-24T22:31:29.742534+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:00:34.244351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"312'172","reported_seq":869,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:42.925419+0000","last_change":"2026-02-24T22:05:18.185556+0000","last_active":"2026-02-24T22:33:42.925419+0000","last_peered":"2026-02-24T22:33:42.925419+0000","last_clean":"2026-02-24T22:33:42.925419+0000","last_became_active":"2026-02-24T22:05:18.185112+0000","last_became_peered":"2026-02-24T22:05:18.185112+0000","last_unstale":"2026-02-24T22:33:42.925419+0000","last_undegraded":"2026-02-24T22:33:42.925419+0000","last_fullsized":"2026-02-24T22:33:42.925419+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":100,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:39:28.189482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":679,"num_read_kb":553,"num_write":246,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"312'218","reported_seq":939,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:12.225390+0000","last_change":"2026-02-24T22:05:16.191455+0000","last_active":"2026-02-24T22:34:12.225390+0000","last_peered":"2026-02-24T22:34:12.225390+0000","last_clean":"2026-02-24T22:34:12.225390+0000","last_became_active":"2026-02-24T22:05:16.191007+0000","last_became_peered":"2026-02-24T22:05:16.191007+0000","last_unstale":"2026-02-24T22:34:12.225390+0000","last_undegraded":"2026-02-24T22:34:12.225390+0000","last_fullsized":"2026-02-24T22:34:12.225390+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":200,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:46:55.713025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13669125,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":436,"num_read_kb":40330,"num_write":1279,"num_write_kb":18546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":613,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:14.731453+0000","last_change":"2026-02-24T22:05:10.689379+0000","last_active":"2026-02-24T22:32:14.731453+0000","last_peered":"2026-02-24T22:32:14.731453+0000","last_clean":"2026-02-24T22:32:14.731453+0000","last_became_active":"2026-02-24T22:05:08.758282+0000","last_became_peered":"2026-02-24T22:05:08.758282+0000","last_unstale":"2026-02-24T22:32:14.731453+0000","last_undegraded":"2026-02-24T22:32:14.731453+0000","last_fullsized":"2026-02-24T22:32:14.731453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:28:00.429651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033355899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":589,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.733043+0000","last_change":"2026-02-24T22:05:20.187544+0000","last_active":"2026-02-24T22:30:19.733043+0000","last_peered":"2026-02-24T22:30:19.733043+0000","last_clean":"2026-02-24T22:30:19.733043+0000","last_became_active":"2026-02-24T22:05:20.187208+0000","last_became_peered":"2026-02-24T22:05:20.187208+0000","last_unstale":"2026-02-24T22:30:19.733043+0000","last_undegraded":"2026-02-24T22:30:19.733043+0000","last_fullsized":"2026-02-24T22:30:19.733043+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:13:41.845882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"312'1120","reported_seq":2461,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.528903+0000","last_change":"2026-02-24T22:05:22.192049+0000","last_active":"2026-02-24T22:34:18.528903+0000","last_peered":"2026-02-24T22:34:18.528903+0000","last_clean":"2026-02-24T22:34:18.528903+0000","last_became_active":"2026-02-24T22:05:22.191763+0000","last_became_peered":"2026-02-24T22:05:22.191763+0000","last_unstale":"2026-02-24T22:34:18.528903+0000","last_undegraded":"2026-02-24T22:34:18.528903+0000","last_fullsized":"2026-02-24T22:34:18.528903+0000","mapping_epoch":28,"log_start":"312'1100","ondisk_log_start":"312'1100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":1100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:42:04.815445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96815,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":3479,"num_read_kb":2825,"num_write":625,"num_write_kb":530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"312'204","reported_seq":808,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.405693+0000","last_change":"2026-02-24T22:05:24.206550+0000","last_active":"2026-02-24T22:34:18.405693+0000","last_peered":"2026-02-24T22:34:18.405693+0000","last_clean":"2026-02-24T22:34:18.405693+0000","last_became_active":"2026-02-24T22:05:24.205923+0000","last_became_peered":"2026-02-24T22:05:24.205923+0000","last_unstale":"2026-02-24T22:34:18.405693+0000","last_undegraded":"2026-02-24T22:34:18.405693+0000","last_fullsized":"2026-02-24T22:34:18.405693+0000","mapping_epoch":30,"log_start":"37'100","ondisk_log_start":"37'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:42:44.001897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34551,"num_objects":102,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":190,"num_read_kb":174,"num_write":516,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"312'462","reported_seq":4824,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.586084+0000","last_change":"2026-02-24T22:05:29.652714+0000","last_active":"2026-02-24T22:34:18.586084+0000","last_peered":"2026-02-24T22:34:18.586084+0000","last_clean":"2026-02-24T22:34:18.586084+0000","last_became_active":"2026-02-24T22:05:29.652259+0000","last_became_peered":"2026-02-24T22:05:29.652259+0000","last_unstale":"2026-02-24T22:34:18.586084+0000","last_undegraded":"2026-02-24T22:34:18.586084+0000","last_fullsized":"2026-02-24T22:34:18.586084+0000","mapping_epoch":33,"log_start":"312'401","ondisk_log_start":"312'401","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":401,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:52:38.347176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"312'114","reported_seq":810,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.733052+0000","last_change":"2026-02-24T22:05:18.183793+0000","last_active":"2026-02-24T22:30:19.733052+0000","last_peered":"2026-02-24T22:30:19.733052+0000","last_clean":"2026-02-24T22:30:19.733052+0000","last_became_active":"2026-02-24T22:05:18.183369+0000","last_became_peered":"2026-02-24T22:05:18.183369+0000","last_unstale":"2026-02-24T22:30:19.733052+0000","last_undegraded":"2026-02-24T22:30:19.733052+0000","last_fullsized":"2026-02-24T22:30:19.733052+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:10:34.192055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":458,"num_read_kb":386,"num_write":168,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"312'435","reported_seq":1245,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:07.118566+0000","last_change":"2026-02-24T22:05:16.179055+0000","last_active":"2026-02-24T22:34:07.118566+0000","last_peered":"2026-02-24T22:34:07.118566+0000","last_clean":"2026-02-24T22:34:07.118566+0000","last_became_active":"2026-02-24T22:05:16.178353+0000","last_became_peered":"2026-02-24T22:05:16.178353+0000","last_unstale":"2026-02-24T22:34:07.118566+0000","last_undegraded":"2026-02-24T22:34:07.118566+0000","last_fullsized":"2026-02-24T22:34:07.118566+0000","mapping_epoch":22,"log_start":"312'400","ondisk_log_start":"312'400","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":400,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:22:52.210538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":621081,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":974,"num_read_kb":9549,"num_write":1984,"num_write_kb":2098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":610,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:19.749000+0000","last_change":"2026-02-24T22:05:10.808508+0000","last_active":"2026-02-24T22:32:19.749000+0000","last_peered":"2026-02-24T22:32:19.749000+0000","last_clean":"2026-02-24T22:32:19.749000+0000","last_became_active":"2026-02-24T22:05:08.765129+0000","last_became_peered":"2026-02-24T22:05:08.765129+0000","last_unstale":"2026-02-24T22:32:19.749000+0000","last_undegraded":"2026-02-24T22:32:19.749000+0000","last_fullsized":"2026-02-24T22:32:19.749000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:05:15.499818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439276,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":584,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.730929+0000","last_change":"2026-02-24T22:05:20.193089+0000","last_active":"2026-02-24T22:30:19.730929+0000","last_peered":"2026-02-24T22:30:19.730929+0000","last_clean":"2026-02-24T22:30:19.730929+0000","last_became_active":"2026-02-24T22:05:20.192811+0000","last_became_peered":"2026-02-24T22:05:20.192811+0000","last_unstale":"2026-02-24T22:30:19.730929+0000","last_undegraded":"2026-02-24T22:30:19.730929+0000","last_fullsized":"2026-02-24T22:30:19.730929+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:09:16.601460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"312'607","reported_seq":1715,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.543706+0000","last_change":"2026-02-24T22:05:22.905928+0000","last_active":"2026-02-24T22:34:18.543706+0000","last_peered":"2026-02-24T22:34:18.543706+0000","last_clean":"2026-02-24T22:34:18.543706+0000","last_became_active":"2026-02-24T22:05:22.905663+0000","last_became_peered":"2026-02-24T22:05:22.905663+0000","last_unstale":"2026-02-24T22:34:18.543706+0000","last_undegraded":"2026-02-24T22:34:18.543706+0000","last_fullsized":"2026-02-24T22:34:18.543706+0000","mapping_epoch":28,"log_start":"307'500","ondisk_log_start":"307'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":500,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:36:30.224989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":308,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":2056,"num_read_kb":1107,"num_write":154,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"312'171","reported_seq":753,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.533054+0000","last_change":"2026-02-24T22:05:24.206213+0000","last_active":"2026-02-24T22:34:18.533054+0000","last_peered":"2026-02-24T22:34:18.533054+0000","last_clean":"2026-02-24T22:34:18.533054+0000","last_became_active":"2026-02-24T22:05:24.205847+0000","last_became_peered":"2026-02-24T22:05:24.205847+0000","last_unstale":"2026-02-24T22:34:18.533054+0000","last_undegraded":"2026-02-24T22:34:18.533054+0000","last_fullsized":"2026-02-24T22:34:18.533054+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:10:54.760147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26185,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":121,"num_read_kb":112,"num_write":413,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":576,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.724548+0000","last_change":"2026-02-24T22:05:29.651555+0000","last_active":"2026-02-24T22:30:19.724548+0000","last_peered":"2026-02-24T22:30:19.724548+0000","last_clean":"2026-02-24T22:30:19.724548+0000","last_became_active":"2026-02-24T22:05:29.651401+0000","last_became_peered":"2026-02-24T22:05:29.651401+0000","last_unstale":"2026-02-24T22:30:19.724548+0000","last_undegraded":"2026-02-24T22:30:19.724548+0000","last_fullsized":"2026-02-24T22:30:19.724548+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:36:54.239488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"312'150","reported_seq":847,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:40.233137+0000","last_change":"2026-02-24T22:05:18.186337+0000","last_active":"2026-02-24T22:33:40.233137+0000","last_peered":"2026-02-24T22:33:40.233137+0000","last_clean":"2026-02-24T22:33:40.233137+0000","last_became_active":"2026-02-24T22:05:18.185592+0000","last_became_peered":"2026-02-24T22:05:18.185592+0000","last_unstale":"2026-02-24T22:33:40.233137+0000","last_undegraded":"2026-02-24T22:33:40.233137+0000","last_fullsized":"2026-02-24T22:33:40.233137+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:52:57.542872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":536,"num_read_kb":444,"num_write":207,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"312'232","reported_seq":958,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:01.140144+0000","last_change":"2026-02-24T22:05:16.185307+0000","last_active":"2026-02-24T22:34:01.140144+0000","last_peered":"2026-02-24T22:34:01.140144+0000","last_clean":"2026-02-24T22:34:01.140144+0000","last_became_active":"2026-02-24T22:05:16.184131+0000","last_became_peered":"2026-02-24T22:05:16.184131+0000","last_unstale":"2026-02-24T22:34:01.140144+0000","last_undegraded":"2026-02-24T22:34:01.140144+0000","last_fullsized":"2026-02-24T22:34:01.140144+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":200,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:09:05.496588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":173363,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":395,"num_read_kb":1193,"num_write":1232,"num_write_kb":244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":612,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.724713+0000","last_change":"2026-02-24T22:05:10.689444+0000","last_active":"2026-02-24T22:31:09.724713+0000","last_peered":"2026-02-24T22:31:09.724713+0000","last_clean":"2026-02-24T22:31:09.724713+0000","last_became_active":"2026-02-24T22:05:08.682913+0000","last_became_peered":"2026-02-24T22:05:08.682913+0000","last_unstale":"2026-02-24T22:31:09.724713+0000","last_undegraded":"2026-02-24T22:31:09.724713+0000","last_fullsized":"2026-02-24T22:31:09.724713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:20:01.744772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000359878,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":591,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.732657+0000","last_change":"2026-02-24T22:05:20.192401+0000","last_active":"2026-02-24T22:30:19.732657+0000","last_peered":"2026-02-24T22:30:19.732657+0000","last_clean":"2026-02-24T22:30:19.732657+0000","last_became_active":"2026-02-24T22:05:20.192162+0000","last_became_peered":"2026-02-24T22:05:20.192162+0000","last_unstale":"2026-02-24T22:30:19.732657+0000","last_undegraded":"2026-02-24T22:30:19.732657+0000","last_fullsized":"2026-02-24T22:30:19.732657+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:51:58.434704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"312'1027","reported_seq":2586,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.531120+0000","last_change":"2026-02-24T22:05:22.198775+0000","last_active":"2026-02-24T22:34:18.531120+0000","last_peered":"2026-02-24T22:34:18.531120+0000","last_clean":"2026-02-24T22:34:18.531120+0000","last_became_active":"2026-02-24T22:05:22.198252+0000","last_became_peered":"2026-02-24T22:05:22.198252+0000","last_unstale":"2026-02-24T22:34:18.531120+0000","last_undegraded":"2026-02-24T22:34:18.531120+0000","last_fullsized":"2026-02-24T22:34:18.531120+0000","mapping_epoch":28,"log_start":"312'1000","ondisk_log_start":"312'1000","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":1000,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:34:58.507593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":614,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":3859,"num_read_kb":1967,"num_write":82,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"312'1698","reported_seq":4314,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.551488+0000","last_change":"2026-02-24T22:05:24.198591+0000","last_active":"2026-02-24T22:34:18.551488+0000","last_peered":"2026-02-24T22:34:18.551488+0000","last_clean":"2026-02-24T22:34:18.551488+0000","last_became_active":"2026-02-24T22:05:24.198348+0000","last_became_peered":"2026-02-24T22:05:24.198348+0000","last_unstale":"2026-02-24T22:34:18.551488+0000","last_undegraded":"2026-02-24T22:34:18.551488+0000","last_fullsized":"2026-02-24T22:34:18.551488+0000","mapping_epoch":30,"log_start":"310'1600","ondisk_log_start":"310'1600","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":1600,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:04:55.163411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31448,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":5151,"num_read_kb":231050,"num_write":3475,"num_write_kb":1655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":574,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.732473+0000","last_change":"2026-02-24T22:05:29.652643+0000","last_active":"2026-02-24T22:30:14.732473+0000","last_peered":"2026-02-24T22:30:14.732473+0000","last_clean":"2026-02-24T22:30:14.732473+0000","last_became_active":"2026-02-24T22:05:29.652321+0000","last_became_peered":"2026-02-24T22:05:29.652321+0000","last_unstale":"2026-02-24T22:30:14.732473+0000","last_undegraded":"2026-02-24T22:30:14.732473+0000","last_fullsized":"2026-02-24T22:30:14.732473+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:23:33.530605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"312'55","reported_seq":718,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.732644+0000","last_change":"2026-02-24T22:05:18.191716+0000","last_active":"2026-02-24T22:30:19.732644+0000","last_peered":"2026-02-24T22:30:19.732644+0000","last_clean":"2026-02-24T22:30:19.732644+0000","last_became_active":"2026-02-24T22:05:18.191249+0000","last_became_peered":"2026-02-24T22:05:18.191249+0000","last_unstale":"2026-02-24T22:30:19.732644+0000","last_undegraded":"2026-02-24T22:30:19.732644+0000","last_fullsized":"2026-02-24T22:30:19.732644+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:48:35.012678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":226,"num_read_kb":230,"num_write":70,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"312'209","reported_seq":941,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:06.434490+0000","last_change":"2026-02-24T22:05:16.182104+0000","last_active":"2026-02-24T22:34:06.434490+0000","last_peered":"2026-02-24T22:34:06.434490+0000","last_clean":"2026-02-24T22:34:06.434490+0000","last_became_active":"2026-02-24T22:05:16.181840+0000","last_became_peered":"2026-02-24T22:05:16.181840+0000","last_unstale":"2026-02-24T22:34:06.434490+0000","last_undegraded":"2026-02-24T22:34:06.434490+0000","last_fullsized":"2026-02-24T22:34:06.434490+0000","mapping_epoch":22,"log_start":"312'100","ondisk_log_start":"312'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":100,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:50:28.643829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11590618,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":429,"num_read_kb":7386,"num_write":1193,"num_write_kb":13063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":607,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.736747+0000","last_change":"2026-02-24T22:05:10.808402+0000","last_active":"2026-02-24T22:31:09.736747+0000","last_peered":"2026-02-24T22:31:09.736747+0000","last_clean":"2026-02-24T22:31:09.736747+0000","last_became_active":"2026-02-24T22:05:08.689542+0000","last_became_peered":"2026-02-24T22:05:08.689542+0000","last_unstale":"2026-02-24T22:31:09.736747+0000","last_undegraded":"2026-02-24T22:31:09.736747+0000","last_fullsized":"2026-02-24T22:31:09.736747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:23:22.404541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038816099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":592,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.731136+0000","last_change":"2026-02-24T22:05:20.186366+0000","last_active":"2026-02-24T22:31:24.731136+0000","last_peered":"2026-02-24T22:31:24.731136+0000","last_clean":"2026-02-24T22:31:24.731136+0000","last_became_active":"2026-02-24T22:05:20.186108+0000","last_became_peered":"2026-02-24T22:05:20.186108+0000","last_unstale":"2026-02-24T22:31:24.731136+0000","last_undegraded":"2026-02-24T22:31:24.731136+0000","last_fullsized":"2026-02-24T22:31:24.731136+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:02:47.567361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"312'592","reported_seq":1229,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.355778+0000","last_change":"2026-02-24T22:05:22.911959+0000","last_active":"2026-02-24T22:34:18.355778+0000","last_peered":"2026-02-24T22:34:18.355778+0000","last_clean":"2026-02-24T22:34:18.355778+0000","last_became_active":"2026-02-24T22:05:22.911692+0000","last_became_peered":"2026-02-24T22:05:22.911692+0000","last_unstale":"2026-02-24T22:34:18.355778+0000","last_undegraded":"2026-02-24T22:34:18.355778+0000","last_fullsized":"2026-02-24T22:34:18.355778+0000","mapping_epoch":28,"log_start":"312'500","ondisk_log_start":"312'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":500,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:53:02.224623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":161291,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1010,"num_read_kb":2303,"num_write":954,"num_write_kb":876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"312'175","reported_seq":763,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:07.165906+0000","last_change":"2026-02-24T22:05:24.199612+0000","last_active":"2026-02-24T22:34:07.165906+0000","last_peered":"2026-02-24T22:34:07.165906+0000","last_clean":"2026-02-24T22:34:07.165906+0000","last_became_active":"2026-02-24T22:05:24.199398+0000","last_became_peered":"2026-02-24T22:05:24.199398+0000","last_unstale":"2026-02-24T22:34:07.165906+0000","last_undegraded":"2026-02-24T22:34:07.165906+0000","last_fullsized":"2026-02-24T22:34:07.165906+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":100,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:03:11.629672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30139,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":95,"num_read_kb":93,"num_write":436,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"312'428","reported_seq":3586,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.571386+0000","last_change":"2026-02-24T22:05:29.662212+0000","last_active":"2026-02-24T22:34:18.571386+0000","last_peered":"2026-02-24T22:34:18.571386+0000","last_clean":"2026-02-24T22:34:18.571386+0000","last_became_active":"2026-02-24T22:05:29.661694+0000","last_became_peered":"2026-02-24T22:05:29.661694+0000","last_unstale":"2026-02-24T22:34:18.571386+0000","last_undegraded":"2026-02-24T22:34:18.571386+0000","last_fullsized":"2026-02-24T22:34:18.571386+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":400,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:28:47.623604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"312'456","reported_seq":2590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.586383+0000","last_change":"2026-02-24T22:05:29.654721+0000","last_active":"2026-02-24T22:34:18.586383+0000","last_peered":"2026-02-24T22:34:18.586383+0000","last_clean":"2026-02-24T22:34:18.586383+0000","last_became_active":"2026-02-24T22:05:29.654058+0000","last_became_peered":"2026-02-24T22:05:29.654058+0000","last_unstale":"2026-02-24T22:34:18.586383+0000","last_undegraded":"2026-02-24T22:34:18.586383+0000","last_fullsized":"2026-02-24T22:34:18.586383+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":400,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:19:34.289230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"312'4194","reported_seq":4926,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:45.073603+0000","last_change":"2026-02-24T22:05:18.191351+0000","last_active":"2026-02-24T22:33:45.073603+0000","last_peered":"2026-02-24T22:33:45.073603+0000","last_clean":"2026-02-24T22:33:45.073603+0000","last_became_active":"2026-02-24T22:05:18.190114+0000","last_became_peered":"2026-02-24T22:05:18.190114+0000","last_unstale":"2026-02-24T22:33:45.073603+0000","last_undegraded":"2026-02-24T22:33:45.073603+0000","last_fullsized":"2026-02-24T22:33:45.073603+0000","mapping_epoch":24,"log_start":"312'4100","ondisk_log_start":"312'4100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":3000,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:58:28.766462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":16758,"num_read_kb":13963,"num_write":6320,"num_write_kb":3135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"312'194","reported_seq":915,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:57.691183+0000","last_change":"2026-02-24T22:05:16.189046+0000","last_active":"2026-02-24T22:33:57.691183+0000","last_peered":"2026-02-24T22:33:57.691183+0000","last_clean":"2026-02-24T22:33:57.691183+0000","last_became_active":"2026-02-24T22:05:16.188630+0000","last_became_peered":"2026-02-24T22:05:16.188630+0000","last_unstale":"2026-02-24T22:33:57.691183+0000","last_undegraded":"2026-02-24T22:33:57.691183+0000","last_fullsized":"2026-02-24T22:33:57.691183+0000","mapping_epoch":22,"log_start":"312'100","ondisk_log_start":"312'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":100,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:15:02.436674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11535007,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":414,"num_read_kb":33751,"num_write":1015,"num_write_kb":13009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":588,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.723682+0000","last_change":"2026-02-24T22:05:20.188706+0000","last_active":"2026-02-24T22:30:59.723682+0000","last_peered":"2026-02-24T22:30:59.723682+0000","last_clean":"2026-02-24T22:30:59.723682+0000","last_became_active":"2026-02-24T22:05:20.188330+0000","last_became_peered":"2026-02-24T22:05:20.188330+0000","last_unstale":"2026-02-24T22:30:59.723682+0000","last_undegraded":"2026-02-24T22:30:59.723682+0000","last_fullsized":"2026-02-24T22:30:59.723682+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:07:09.639326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"312'681","reported_seq":1757,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.538908+0000","last_change":"2026-02-24T22:05:22.192098+0000","last_active":"2026-02-24T22:34:18.538908+0000","last_peered":"2026-02-24T22:34:18.538908+0000","last_clean":"2026-02-24T22:34:18.538908+0000","last_became_active":"2026-02-24T22:05:22.191584+0000","last_became_peered":"2026-02-24T22:05:22.191584+0000","last_unstale":"2026-02-24T22:34:18.538908+0000","last_undegraded":"2026-02-24T22:34:18.538908+0000","last_fullsized":"2026-02-24T22:34:18.538908+0000","mapping_epoch":28,"log_start":"312'600","ondisk_log_start":"312'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":600,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:55:24.433104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50841,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2209,"num_read_kb":1670,"num_write":316,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"312'161","reported_seq":768,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:55.998434+0000","last_change":"2026-02-24T22:05:24.199098+0000","last_active":"2026-02-24T22:33:55.998434+0000","last_peered":"2026-02-24T22:33:55.998434+0000","last_clean":"2026-02-24T22:33:55.998434+0000","last_became_active":"2026-02-24T22:05:24.198821+0000","last_became_peered":"2026-02-24T22:05:24.198821+0000","last_unstale":"2026-02-24T22:33:55.998434+0000","last_undegraded":"2026-02-24T22:33:55.998434+0000","last_fullsized":"2026-02-24T22:33:55.998434+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":100,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:31:53.831339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28978,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":162,"num_read_kb":145,"num_write":411,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":578,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.732230+0000","last_change":"2026-02-24T22:05:29.653213+0000","last_active":"2026-02-24T22:31:34.732230+0000","last_peered":"2026-02-24T22:31:34.732230+0000","last_clean":"2026-02-24T22:31:34.732230+0000","last_became_active":"2026-02-24T22:05:29.652757+0000","last_became_peered":"2026-02-24T22:05:29.652757+0000","last_unstale":"2026-02-24T22:31:34.732230+0000","last_undegraded":"2026-02-24T22:31:34.732230+0000","last_fullsized":"2026-02-24T22:31:34.732230+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:57:03.928835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"312'4231","reported_seq":4978,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:52.784970+0000","last_change":"2026-02-24T22:05:18.184303+0000","last_active":"2026-02-24T22:33:52.784970+0000","last_peered":"2026-02-24T22:33:52.784970+0000","last_clean":"2026-02-24T22:33:52.784970+0000","last_became_active":"2026-02-24T22:05:18.183983+0000","last_became_peered":"2026-02-24T22:05:18.183983+0000","last_unstale":"2026-02-24T22:33:52.784970+0000","last_undegraded":"2026-02-24T22:33:52.784970+0000","last_fullsized":"2026-02-24T22:33:52.784970+0000","mapping_epoch":24,"log_start":"312'4200","ondisk_log_start":"312'4200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":3000,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:36:58.437134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":16912,"num_read_kb":14102,"num_write":6331,"num_write_kb":3151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"312'232","reported_seq":974,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:04.712850+0000","last_change":"2026-02-24T22:05:16.193213+0000","last_active":"2026-02-24T22:34:04.712850+0000","last_peered":"2026-02-24T22:34:04.712850+0000","last_clean":"2026-02-24T22:34:04.712850+0000","last_became_active":"2026-02-24T22:05:16.192845+0000","last_became_peered":"2026-02-24T22:05:16.192845+0000","last_unstale":"2026-02-24T22:34:04.712850+0000","last_undegraded":"2026-02-24T22:34:04.712850+0000","last_fullsized":"2026-02-24T22:34:04.712850+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":200,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:53:53.655783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933180,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":413,"num_read_kb":4336,"num_write":1200,"num_write_kb":2676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.728520+0000","last_change":"2026-02-24T22:05:20.189035+0000","last_active":"2026-02-24T22:30:59.728520+0000","last_peered":"2026-02-24T22:30:59.728520+0000","last_clean":"2026-02-24T22:30:59.728520+0000","last_became_active":"2026-02-24T22:05:20.188790+0000","last_became_peered":"2026-02-24T22:05:20.188790+0000","last_unstale":"2026-02-24T22:30:59.728520+0000","last_undegraded":"2026-02-24T22:30:59.728520+0000","last_fullsized":"2026-02-24T22:30:59.728520+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:29:04.813791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"312'25","reported_seq":615,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.737881+0000","last_change":"2026-02-24T22:05:22.192740+0000","last_active":"2026-02-24T22:31:04.737881+0000","last_peered":"2026-02-24T22:31:04.737881+0000","last_clean":"2026-02-24T22:31:04.737881+0000","last_became_active":"2026-02-24T22:05:22.192560+0000","last_became_peered":"2026-02-24T22:05:22.192560+0000","last_unstale":"2026-02-24T22:31:04.737881+0000","last_undegraded":"2026-02-24T22:31:04.737881+0000","last_fullsized":"2026-02-24T22:31:04.737881+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:40:42.414311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"312'163","reported_seq":746,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.531878+0000","last_change":"2026-02-24T22:05:24.205761+0000","last_active":"2026-02-24T22:34:18.531878+0000","last_peered":"2026-02-24T22:34:18.531878+0000","last_clean":"2026-02-24T22:34:18.531878+0000","last_became_active":"2026-02-24T22:05:24.205506+0000","last_became_peered":"2026-02-24T22:05:24.205506+0000","last_unstale":"2026-02-24T22:34:18.531878+0000","last_undegraded":"2026-02-24T22:34:18.531878+0000","last_fullsized":"2026-02-24T22:34:18.531878+0000","mapping_epoch":30,"log_start":"301'100","ondisk_log_start":"301'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":100,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:22:32.827602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30843,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":90,"num_read_kb":84,"num_write":389,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":573,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.734200+0000","last_change":"2026-02-24T22:05:29.653324+0000","last_active":"2026-02-24T22:30:29.734200+0000","last_peered":"2026-02-24T22:30:29.734200+0000","last_clean":"2026-02-24T22:30:29.734200+0000","last_became_active":"2026-02-24T22:05:29.652987+0000","last_became_peered":"2026-02-24T22:05:29.652987+0000","last_unstale":"2026-02-24T22:30:29.734200+0000","last_undegraded":"2026-02-24T22:30:29.734200+0000","last_fullsized":"2026-02-24T22:30:29.734200+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:58:50.918939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"312'187","reported_seq":884,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.550400+0000","last_change":"2026-02-24T22:05:18.186488+0000","last_active":"2026-02-24T22:34:18.550400+0000","last_peered":"2026-02-24T22:34:18.550400+0000","last_clean":"2026-02-24T22:34:18.550400+0000","last_became_active":"2026-02-24T22:05:18.186069+0000","last_became_peered":"2026-02-24T22:05:18.186069+0000","last_unstale":"2026-02-24T22:34:18.550400+0000","last_undegraded":"2026-02-24T22:34:18.550400+0000","last_fullsized":"2026-02-24T22:34:18.550400+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:26:51.294901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":691,"num_read_kb":561,"num_write":267,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"312'221","reported_seq":903,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:58.398587+0000","last_change":"2026-02-24T22:06:00.662484+0000","last_active":"2026-02-24T22:30:58.398587+0000","last_peered":"2026-02-24T22:30:58.398587+0000","last_clean":"2026-02-24T22:30:58.398587+0000","last_became_active":"2026-02-24T22:06:00.662291+0000","last_became_peered":"2026-02-24T22:06:00.662291+0000","last_unstale":"2026-02-24T22:30:58.398587+0000","last_undegraded":"2026-02-24T22:30:58.398587+0000","last_fullsized":"2026-02-24T22:30:58.398587+0000","mapping_epoch":36,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:44:04.240829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5088,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":359,"num_read_kb":357,"num_write":1312,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":585,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.739650+0000","last_change":"2026-02-24T22:05:20.194668+0000","last_active":"2026-02-24T22:30:59.739650+0000","last_peered":"2026-02-24T22:30:59.739650+0000","last_clean":"2026-02-24T22:30:59.739650+0000","last_became_active":"2026-02-24T22:05:20.194215+0000","last_became_peered":"2026-02-24T22:05:20.194215+0000","last_unstale":"2026-02-24T22:30:59.739650+0000","last_undegraded":"2026-02-24T22:30:59.739650+0000","last_fullsized":"2026-02-24T22:30:59.739650+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:55:09.941150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"312'470","reported_seq":1327,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.519070+0000","last_change":"2026-02-24T22:05:22.192102+0000","last_active":"2026-02-24T22:34:18.519070+0000","last_peered":"2026-02-24T22:34:18.519070+0000","last_clean":"2026-02-24T22:34:18.519070+0000","last_became_active":"2026-02-24T22:05:22.191675+0000","last_became_peered":"2026-02-24T22:05:22.191675+0000","last_unstale":"2026-02-24T22:34:18.519070+0000","last_undegraded":"2026-02-24T22:34:18.519070+0000","last_fullsized":"2026-02-24T22:34:18.519070+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":400,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:50:42.591507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53362,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1312,"num_read_kb":1248,"num_write":339,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"312'249","reported_seq":1278,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.564141+0000","last_change":"2026-02-24T22:05:24.207967+0000","last_active":"2026-02-24T22:34:17.564141+0000","last_peered":"2026-02-24T22:34:17.564141+0000","last_clean":"2026-02-24T22:34:17.564141+0000","last_became_active":"2026-02-24T22:05:24.207479+0000","last_became_peered":"2026-02-24T22:05:24.207479+0000","last_unstale":"2026-02-24T22:34:17.564141+0000","last_undegraded":"2026-02-24T22:34:17.564141+0000","last_fullsized":"2026-02-24T22:34:17.564141+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":200,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:21:33.217485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31962,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":675,"num_read_kb":669,"num_write":683,"num_write_kb":207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":564,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.735513+0000","last_change":"2026-02-24T22:05:29.661767+0000","last_active":"2026-02-24T22:30:29.735513+0000","last_peered":"2026-02-24T22:30:29.735513+0000","last_clean":"2026-02-24T22:30:29.735513+0000","last_became_active":"2026-02-24T22:05:29.661248+0000","last_became_peered":"2026-02-24T22:05:29.661248+0000","last_unstale":"2026-02-24T22:30:29.735513+0000","last_undegraded":"2026-02-24T22:30:29.735513+0000","last_fullsized":"2026-02-24T22:30:29.735513+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:30:06.329202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"312'116","reported_seq":824,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:06.481608+0000","last_change":"2026-02-24T22:05:18.187722+0000","last_active":"2026-02-24T22:34:06.481608+0000","last_peered":"2026-02-24T22:34:06.481608+0000","last_clean":"2026-02-24T22:34:06.481608+0000","last_became_active":"2026-02-24T22:05:18.184982+0000","last_became_peered":"2026-02-24T22:05:18.184982+0000","last_unstale":"2026-02-24T22:34:06.481608+0000","last_undegraded":"2026-02-24T22:34:06.481608+0000","last_fullsized":"2026-02-24T22:34:06.481608+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":100,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:08:28.087403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":444,"num_read_kb":381,"num_write":158,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"312'213","reported_seq":975,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:02.205401+0000","last_change":"2026-02-24T22:05:16.184926+0000","last_active":"2026-02-24T22:34:02.205401+0000","last_peered":"2026-02-24T22:34:02.205401+0000","last_clean":"2026-02-24T22:34:02.205401+0000","last_became_active":"2026-02-24T22:05:16.184594+0000","last_became_peered":"2026-02-24T22:05:16.184594+0000","last_unstale":"2026-02-24T22:34:02.205401+0000","last_undegraded":"2026-02-24T22:34:02.205401+0000","last_fullsized":"2026-02-24T22:34:02.205401+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":200,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:53:18.748033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6374454,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":456,"num_read_kb":28116,"num_write":1131,"num_write_kb":6289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":583,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.735449+0000","last_change":"2026-02-24T22:05:20.193777+0000","last_active":"2026-02-24T22:30:59.735449+0000","last_peered":"2026-02-24T22:30:59.735449+0000","last_clean":"2026-02-24T22:30:59.735449+0000","last_became_active":"2026-02-24T22:05:20.193522+0000","last_became_peered":"2026-02-24T22:05:20.193522+0000","last_unstale":"2026-02-24T22:30:59.735449+0000","last_undegraded":"2026-02-24T22:30:59.735449+0000","last_fullsized":"2026-02-24T22:30:59.735449+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:18:03.275177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"312'428","reported_seq":1261,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.513000+0000","last_change":"2026-02-24T22:05:22.200284+0000","last_active":"2026-02-24T22:34:18.513000+0000","last_peered":"2026-02-24T22:34:18.513000+0000","last_clean":"2026-02-24T22:34:18.513000+0000","last_became_active":"2026-02-24T22:05:22.199968+0000","last_became_peered":"2026-02-24T22:05:22.199968+0000","last_unstale":"2026-02-24T22:34:18.513000+0000","last_undegraded":"2026-02-24T22:34:18.513000+0000","last_fullsized":"2026-02-24T22:34:18.513000+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":400,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:25:54.291884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51783,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1258,"num_read_kb":1192,"num_write":305,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"312'212","reported_seq":808,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:04.178415+0000","last_change":"2026-02-24T22:05:24.200305+0000","last_active":"2026-02-24T22:34:04.178415+0000","last_peered":"2026-02-24T22:34:04.178415+0000","last_clean":"2026-02-24T22:34:04.178415+0000","last_became_active":"2026-02-24T22:05:24.199650+0000","last_became_peered":"2026-02-24T22:05:24.199650+0000","last_unstale":"2026-02-24T22:34:04.178415+0000","last_undegraded":"2026-02-24T22:34:04.178415+0000","last_fullsized":"2026-02-24T22:34:04.178415+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:33:18.336086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29523,"num_objects":91,"num_object_clones":0,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":182,"num_read_kb":193,"num_write":488,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"312'249","reported_seq":958,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:44.105284+0000","last_change":"2026-02-24T22:05:18.190553+0000","last_active":"2026-02-24T22:33:44.105284+0000","last_peered":"2026-02-24T22:33:44.105284+0000","last_clean":"2026-02-24T22:33:44.105284+0000","last_became_active":"2026-02-24T22:05:18.190238+0000","last_became_peered":"2026-02-24T22:05:18.190238+0000","last_unstale":"2026-02-24T22:33:44.105284+0000","last_undegraded":"2026-02-24T22:33:44.105284+0000","last_fullsized":"2026-02-24T22:33:44.105284+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":200,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:04:44.616609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":976,"num_read_kb":794,"num_write":353,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"312'218","reported_seq":1574,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:05.712936+0000","last_change":"2026-02-24T22:05:16.185696+0000","last_active":"2026-02-24T22:34:05.712936+0000","last_peered":"2026-02-24T22:34:05.712936+0000","last_clean":"2026-02-24T22:34:05.712936+0000","last_became_active":"2026-02-24T22:05:16.184415+0000","last_became_peered":"2026-02-24T22:05:16.184415+0000","last_unstale":"2026-02-24T22:34:05.712936+0000","last_undegraded":"2026-02-24T22:34:05.712936+0000","last_fullsized":"2026-02-24T22:34:05.712936+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":200,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:10:18.950972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5603567,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":2274,"num_read_kb":23832,"num_write":1037,"num_write_kb":5557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.740195+0000","last_change":"2026-02-24T22:05:20.186817+0000","last_active":"2026-02-24T22:31:24.740195+0000","last_peered":"2026-02-24T22:31:24.740195+0000","last_clean":"2026-02-24T22:31:24.740195+0000","last_became_active":"2026-02-24T22:05:20.186325+0000","last_became_peered":"2026-02-24T22:05:20.186325+0000","last_unstale":"2026-02-24T22:31:24.740195+0000","last_undegraded":"2026-02-24T22:31:24.740195+0000","last_fullsized":"2026-02-24T22:31:24.740195+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:45:07.507222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"312'399","reported_seq":1007,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.383952+0000","last_change":"2026-02-24T22:05:22.192972+0000","last_active":"2026-02-24T22:34:18.383952+0000","last_peered":"2026-02-24T22:34:18.383952+0000","last_clean":"2026-02-24T22:34:18.383952+0000","last_became_active":"2026-02-24T22:05:22.192756+0000","last_became_peered":"2026-02-24T22:05:22.192756+0000","last_unstale":"2026-02-24T22:34:18.383952+0000","last_undegraded":"2026-02-24T22:34:18.383952+0000","last_fullsized":"2026-02-24T22:34:18.383952+0000","mapping_epoch":28,"log_start":"312'300","ondisk_log_start":"312'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":300,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:43:14.513261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115669,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":689,"num_read_kb":1585,"num_write":692,"num_write_kb":688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"312'187","reported_seq":796,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.233131+0000","last_change":"2026-02-24T22:05:24.206653+0000","last_active":"2026-02-24T22:34:17.233131+0000","last_peered":"2026-02-24T22:34:17.233131+0000","last_clean":"2026-02-24T22:34:17.233131+0000","last_became_active":"2026-02-24T22:05:24.206407+0000","last_became_peered":"2026-02-24T22:05:24.206407+0000","last_unstale":"2026-02-24T22:34:17.233131+0000","last_undegraded":"2026-02-24T22:34:17.233131+0000","last_fullsized":"2026-02-24T22:34:17.233131+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:49:11.388315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30298,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":168,"num_read_kb":155,"num_write":455,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":576,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.729573+0000","last_change":"2026-02-24T22:05:29.650326+0000","last_active":"2026-02-24T22:31:09.729573+0000","last_peered":"2026-02-24T22:31:09.729573+0000","last_clean":"2026-02-24T22:31:09.729573+0000","last_became_active":"2026-02-24T22:05:29.650115+0000","last_became_peered":"2026-02-24T22:05:29.650115+0000","last_unstale":"2026-02-24T22:31:09.729573+0000","last_undegraded":"2026-02-24T22:31:09.729573+0000","last_fullsized":"2026-02-24T22:31:09.729573+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:38:20.581425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"312'162","reported_seq":873,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.740186+0000","last_change":"2026-02-24T22:05:18.183744+0000","last_active":"2026-02-24T22:31:24.740186+0000","last_peered":"2026-02-24T22:31:24.740186+0000","last_clean":"2026-02-24T22:31:24.740186+0000","last_became_active":"2026-02-24T22:05:18.183501+0000","last_became_peered":"2026-02-24T22:05:18.183501+0000","last_unstale":"2026-02-24T22:31:24.740186+0000","last_undegraded":"2026-02-24T22:31:24.740186+0000","last_fullsized":"2026-02-24T22:31:24.740186+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":100,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:10:06.918155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":578,"num_read_kb":488,"num_write":224,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"312'343","reported_seq":1285,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:38.575992+0000","last_change":"2026-02-24T22:05:16.183028+0000","last_active":"2026-02-24T22:33:38.575992+0000","last_peered":"2026-02-24T22:33:38.575992+0000","last_clean":"2026-02-24T22:33:38.575992+0000","last_became_active":"2026-02-24T22:05:16.182909+0000","last_became_peered":"2026-02-24T22:05:16.182909+0000","last_unstale":"2026-02-24T22:33:38.575992+0000","last_undegraded":"2026-02-24T22:33:38.575992+0000","last_fullsized":"2026-02-24T22:33:38.575992+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:01:15.081477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":445956,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":1143,"num_read_kb":9242,"num_write":1547,"num_write_kb":11784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":584,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.730971+0000","last_change":"2026-02-24T22:05:20.194023+0000","last_active":"2026-02-24T22:30:19.730971+0000","last_peered":"2026-02-24T22:30:19.730971+0000","last_clean":"2026-02-24T22:30:19.730971+0000","last_became_active":"2026-02-24T22:05:20.193751+0000","last_became_peered":"2026-02-24T22:05:20.193751+0000","last_unstale":"2026-02-24T22:30:19.730971+0000","last_undegraded":"2026-02-24T22:30:19.730971+0000","last_fullsized":"2026-02-24T22:30:19.730971+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:48:18.314998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"312'737","reported_seq":1813,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.525758+0000","last_change":"2026-02-24T22:05:22.200836+0000","last_active":"2026-02-24T22:34:18.525758+0000","last_peered":"2026-02-24T22:34:18.525758+0000","last_clean":"2026-02-24T22:34:18.525758+0000","last_became_active":"2026-02-24T22:05:22.200603+0000","last_became_peered":"2026-02-24T22:05:22.200603+0000","last_unstale":"2026-02-24T22:34:18.525758+0000","last_undegraded":"2026-02-24T22:34:18.525758+0000","last_fullsized":"2026-02-24T22:34:18.525758+0000","mapping_epoch":28,"log_start":"312'700","ondisk_log_start":"312'700","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":700,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:36:03.813153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63999,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":2312,"num_read_kb":1868,"num_write":428,"num_write_kb":371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"312'334","reported_seq":964,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.243815+0000","last_change":"2026-02-24T22:05:24.201087+0000","last_active":"2026-02-24T22:34:18.243815+0000","last_peered":"2026-02-24T22:34:18.243815+0000","last_clean":"2026-02-24T22:34:18.243815+0000","last_became_active":"2026-02-24T22:05:24.200391+0000","last_became_peered":"2026-02-24T22:05:24.200391+0000","last_unstale":"2026-02-24T22:34:18.243815+0000","last_undegraded":"2026-02-24T22:34:18.243815+0000","last_fullsized":"2026-02-24T22:34:18.243815+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:18:42.256342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31847,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":370,"num_read_kb":353,"num_write":777,"num_write_kb":271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"312'81","reported_seq":994,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:14.669283+0000","last_change":"2026-02-24T22:05:29.660890+0000","last_active":"2026-02-24T22:34:14.669283+0000","last_peered":"2026-02-24T22:34:14.669283+0000","last_clean":"2026-02-24T22:34:14.669283+0000","last_became_active":"2026-02-24T22:05:29.660289+0000","last_became_peered":"2026-02-24T22:05:29.660289+0000","last_unstale":"2026-02-24T22:34:14.669283+0000","last_undegraded":"2026-02-24T22:34:14.669283+0000","last_fullsized":"2026-02-24T22:34:14.669283+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:13:08.310592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"312'100","reported_seq":769,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:54.429078+0000","last_change":"2026-02-24T22:05:18.185312+0000","last_active":"2026-02-24T22:33:54.429078+0000","last_peered":"2026-02-24T22:33:54.429078+0000","last_clean":"2026-02-24T22:33:54.429078+0000","last_became_active":"2026-02-24T22:05:18.184745+0000","last_became_peered":"2026-02-24T22:05:18.184745+0000","last_unstale":"2026-02-24T22:33:54.429078+0000","last_undegraded":"2026-02-24T22:33:54.429078+0000","last_fullsized":"2026-02-24T22:33:54.429078+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:08:34.210048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":371,"num_read_kb":307,"num_write":132,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"312'221","reported_seq":941,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:55.590300+0000","last_change":"2026-02-24T22:05:16.185859+0000","last_active":"2026-02-24T22:30:55.590300+0000","last_peered":"2026-02-24T22:30:55.590300+0000","last_clean":"2026-02-24T22:30:55.590300+0000","last_became_active":"2026-02-24T22:05:16.185574+0000","last_became_peered":"2026-02-24T22:05:16.185574+0000","last_unstale":"2026-02-24T22:30:55.590300+0000","last_undegraded":"2026-02-24T22:30:55.590300+0000","last_fullsized":"2026-02-24T22:30:55.590300+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:09:04.869683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6081138,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":366,"num_read_kb":30007,"num_write":1195,"num_write_kb":21358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"32'3","reported_seq":830,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.335841+0000","last_change":"2026-02-24T22:05:20.192991+0000","last_active":"2026-02-24T22:34:18.335841+0000","last_peered":"2026-02-24T22:34:18.335841+0000","last_clean":"2026-02-24T22:34:18.335841+0000","last_became_active":"2026-02-24T22:05:20.192612+0000","last_became_peered":"2026-02-24T22:05:20.192612+0000","last_unstale":"2026-02-24T22:34:18.335841+0000","last_undegraded":"2026-02-24T22:34:18.335841+0000","last_fullsized":"2026-02-24T22:34:18.335841+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:32:11.097665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":482,"num_read_kb":480,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"312'199","reported_seq":806,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.845514+0000","last_change":"2026-02-24T22:05:22.906226+0000","last_active":"2026-02-24T22:34:17.845514+0000","last_peered":"2026-02-24T22:34:17.845514+0000","last_clean":"2026-02-24T22:34:17.845514+0000","last_became_active":"2026-02-24T22:05:22.906047+0000","last_became_peered":"2026-02-24T22:05:22.906047+0000","last_unstale":"2026-02-24T22:34:17.845514+0000","last_undegraded":"2026-02-24T22:34:17.845514+0000","last_fullsized":"2026-02-24T22:34:17.845514+0000","mapping_epoch":28,"log_start":"312'100","ondisk_log_start":"312'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:36:51.410405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50386,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":315,"num_read_kb":704,"num_write":294,"num_write_kb":265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"312'164","reported_seq":759,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:58.164024+0000","last_change":"2026-02-24T22:05:24.201114+0000","last_active":"2026-02-24T22:33:58.164024+0000","last_peered":"2026-02-24T22:33:58.164024+0000","last_clean":"2026-02-24T22:33:58.164024+0000","last_became_active":"2026-02-24T22:05:24.200747+0000","last_became_peered":"2026-02-24T22:05:24.200747+0000","last_unstale":"2026-02-24T22:33:58.164024+0000","last_undegraded":"2026-02-24T22:33:58.164024+0000","last_fullsized":"2026-02-24T22:33:58.164024+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:27:56.881747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30873,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":120,"num_read_kb":110,"num_write":405,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":565,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.740173+0000","last_change":"2026-02-24T22:05:29.660794+0000","last_active":"2026-02-24T22:31:09.740173+0000","last_peered":"2026-02-24T22:31:09.740173+0000","last_clean":"2026-02-24T22:31:09.740173+0000","last_became_active":"2026-02-24T22:05:29.660344+0000","last_became_peered":"2026-02-24T22:05:29.660344+0000","last_unstale":"2026-02-24T22:31:09.740173+0000","last_undegraded":"2026-02-24T22:31:09.740173+0000","last_fullsized":"2026-02-24T22:31:09.740173+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:36:46.842669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"312'157","reported_seq":828,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.716442+0000","last_change":"2026-02-24T22:06:00.759389+0000","last_active":"2026-02-24T22:30:19.716442+0000","last_peered":"2026-02-24T22:30:19.716442+0000","last_clean":"2026-02-24T22:30:19.716442+0000","last_became_active":"2026-02-24T22:06:00.759215+0000","last_became_peered":"2026-02-24T22:06:00.759215+0000","last_unstale":"2026-02-24T22:30:19.716442+0000","last_undegraded":"2026-02-24T22:30:19.716442+0000","last_fullsized":"2026-02-24T22:30:19.716442+0000","mapping_epoch":36,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:14:31.570934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":584,"num_read_kb":483,"num_write":222,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"312'212","reported_seq":963,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:59.428345+0000","last_change":"2026-02-24T22:05:16.184388+0000","last_active":"2026-02-24T22:33:59.428345+0000","last_peered":"2026-02-24T22:33:59.428345+0000","last_clean":"2026-02-24T22:33:59.428345+0000","last_became_active":"2026-02-24T22:05:16.183956+0000","last_became_peered":"2026-02-24T22:05:16.183956+0000","last_unstale":"2026-02-24T22:33:59.428345+0000","last_undegraded":"2026-02-24T22:33:59.428345+0000","last_fullsized":"2026-02-24T22:33:59.428345+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:01:46.056052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5448977,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":524,"num_read_kb":6430,"num_write":1216,"num_write_kb":5371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":587,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.719728+0000","last_change":"2026-02-24T22:05:20.187960+0000","last_active":"2026-02-24T22:30:19.719728+0000","last_peered":"2026-02-24T22:30:19.719728+0000","last_clean":"2026-02-24T22:30:19.719728+0000","last_became_active":"2026-02-24T22:05:20.187316+0000","last_became_peered":"2026-02-24T22:05:20.187316+0000","last_unstale":"2026-02-24T22:30:19.719728+0000","last_undegraded":"2026-02-24T22:30:19.719728+0000","last_fullsized":"2026-02-24T22:30:19.719728+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:07:22.260792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"312'287","reported_seq":1125,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.509524+0000","last_change":"2026-02-24T22:05:22.912321+0000","last_active":"2026-02-24T22:34:18.509524+0000","last_peered":"2026-02-24T22:34:18.509524+0000","last_clean":"2026-02-24T22:34:18.509524+0000","last_became_active":"2026-02-24T22:05:22.912087+0000","last_became_peered":"2026-02-24T22:05:22.912087+0000","last_unstale":"2026-02-24T22:34:18.509524+0000","last_undegraded":"2026-02-24T22:34:18.509524+0000","last_fullsized":"2026-02-24T22:34:18.509524+0000","mapping_epoch":28,"log_start":"307'200","ondisk_log_start":"307'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":200,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:16:48.283329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":152,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":996,"num_read_kb":518,"num_write":51,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"312'178","reported_seq":844,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.269129+0000","last_change":"2026-02-24T22:05:24.207156+0000","last_active":"2026-02-24T22:34:15.269129+0000","last_peered":"2026-02-24T22:34:15.269129+0000","last_clean":"2026-02-24T22:34:15.269129+0000","last_became_active":"2026-02-24T22:05:24.206383+0000","last_became_peered":"2026-02-24T22:05:24.206383+0000","last_unstale":"2026-02-24T22:34:15.269129+0000","last_undegraded":"2026-02-24T22:34:15.269129+0000","last_fullsized":"2026-02-24T22:34:15.269129+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":100,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:12:00.918186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29475,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":255,"num_read_kb":237,"num_write":434,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":571,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.740948+0000","last_change":"2026-02-24T22:05:29.661761+0000","last_active":"2026-02-24T22:31:09.740948+0000","last_peered":"2026-02-24T22:31:09.740948+0000","last_clean":"2026-02-24T22:31:09.740948+0000","last_became_active":"2026-02-24T22:05:29.661539+0000","last_became_peered":"2026-02-24T22:05:29.661539+0000","last_unstale":"2026-02-24T22:31:09.740948+0000","last_undegraded":"2026-02-24T22:31:09.740948+0000","last_fullsized":"2026-02-24T22:31:09.740948+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:02:56.515020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"312'178","reported_seq":849,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.732694+0000","last_change":"2026-02-24T22:05:18.193080+0000","last_active":"2026-02-24T22:30:19.732694+0000","last_peered":"2026-02-24T22:30:19.732694+0000","last_clean":"2026-02-24T22:30:19.732694+0000","last_became_active":"2026-02-24T22:05:18.192487+0000","last_became_peered":"2026-02-24T22:05:18.192487+0000","last_unstale":"2026-02-24T22:30:19.732694+0000","last_undegraded":"2026-02-24T22:30:19.732694+0000","last_fullsized":"2026-02-24T22:30:19.732694+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":100,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:11:58.202360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":681,"num_read_kb":551,"num_write":266,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"312'225","reported_seq":924,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.242241+0000","last_change":"2026-02-24T22:05:16.190846+0000","last_active":"2026-02-24T22:34:17.242241+0000","last_peered":"2026-02-24T22:34:17.242241+0000","last_clean":"2026-02-24T22:34:17.242241+0000","last_became_active":"2026-02-24T22:05:16.190108+0000","last_became_peered":"2026-02-24T22:05:16.190108+0000","last_unstale":"2026-02-24T22:34:17.242241+0000","last_undegraded":"2026-02-24T22:34:17.242241+0000","last_fullsized":"2026-02-24T22:34:17.242241+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":200,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:05:24.226362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10513634,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":400,"num_read_kb":15743,"num_write":1275,"num_write_kb":10328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":584,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.735341+0000","last_change":"2026-02-24T22:05:20.194434+0000","last_active":"2026-02-24T22:30:19.735341+0000","last_peered":"2026-02-24T22:30:19.735341+0000","last_clean":"2026-02-24T22:30:19.735341+0000","last_became_active":"2026-02-24T22:05:20.194183+0000","last_became_peered":"2026-02-24T22:05:20.194183+0000","last_unstale":"2026-02-24T22:30:19.735341+0000","last_undegraded":"2026-02-24T22:30:19.735341+0000","last_fullsized":"2026-02-24T22:30:19.735341+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:11:03.103844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"312'706","reported_seq":1776,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.522544+0000","last_change":"2026-02-24T22:05:22.199767+0000","last_active":"2026-02-24T22:34:18.522544+0000","last_peered":"2026-02-24T22:34:18.522544+0000","last_clean":"2026-02-24T22:34:18.522544+0000","last_became_active":"2026-02-24T22:05:22.199480+0000","last_became_peered":"2026-02-24T22:05:22.199480+0000","last_unstale":"2026-02-24T22:34:18.522544+0000","last_undegraded":"2026-02-24T22:34:18.522544+0000","last_fullsized":"2026-02-24T22:34:18.522544+0000","mapping_epoch":28,"log_start":"312'600","ondisk_log_start":"312'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":600,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:44:12.736876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56130,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":2245,"num_read_kb":1729,"num_write":346,"num_write_kb":294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"312'198","reported_seq":804,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.239318+0000","last_change":"2026-02-24T22:05:24.201028+0000","last_active":"2026-02-24T22:34:15.239318+0000","last_peered":"2026-02-24T22:34:15.239318+0000","last_clean":"2026-02-24T22:34:15.239318+0000","last_became_active":"2026-02-24T22:05:24.200434+0000","last_became_peered":"2026-02-24T22:05:24.200434+0000","last_unstale":"2026-02-24T22:34:15.239318+0000","last_undegraded":"2026-02-24T22:34:15.239318+0000","last_fullsized":"2026-02-24T22:34:15.239318+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":100,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:38:49.379078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27134,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":189,"num_read_kb":178,"num_write":476,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":570,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:39.731065+0000","last_change":"2026-02-24T22:05:29.662687+0000","last_active":"2026-02-24T22:29:39.731065+0000","last_peered":"2026-02-24T22:29:39.731065+0000","last_clean":"2026-02-24T22:29:39.731065+0000","last_became_active":"2026-02-24T22:05:29.662425+0000","last_became_peered":"2026-02-24T22:05:29.662425+0000","last_unstale":"2026-02-24T22:29:39.731065+0000","last_undegraded":"2026-02-24T22:29:39.731065+0000","last_fullsized":"2026-02-24T22:29:39.731065+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:37:43.066587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"312'132","reported_seq":810,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.732676+0000","last_change":"2026-02-24T22:05:18.193443+0000","last_active":"2026-02-24T22:30:19.732676+0000","last_peered":"2026-02-24T22:30:19.732676+0000","last_clean":"2026-02-24T22:30:19.732676+0000","last_became_active":"2026-02-24T22:05:18.191664+0000","last_became_peered":"2026-02-24T22:05:18.191664+0000","last_unstale":"2026-02-24T22:30:19.732676+0000","last_undegraded":"2026-02-24T22:30:19.732676+0000","last_fullsized":"2026-02-24T22:30:19.732676+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":100,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:59:22.441702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":463,"num_read_kb":379,"num_write":198,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"312'212","reported_seq":907,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:55.742771+0000","last_change":"2026-02-24T22:05:16.189190+0000","last_active":"2026-02-24T22:33:55.742771+0000","last_peered":"2026-02-24T22:33:55.742771+0000","last_clean":"2026-02-24T22:33:55.742771+0000","last_became_active":"2026-02-24T22:05:16.188154+0000","last_became_peered":"2026-02-24T22:05:16.188154+0000","last_unstale":"2026-02-24T22:33:55.742771+0000","last_undegraded":"2026-02-24T22:33:55.742771+0000","last_fullsized":"2026-02-24T22:33:55.742771+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:45:59.561385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65745,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":374,"num_read_kb":2459,"num_write":1169,"num_write_kb":1814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"32'5","reported_seq":820,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.334772+0000","last_change":"2026-02-24T22:05:20.195455+0000","last_active":"2026-02-24T22:34:18.334772+0000","last_peered":"2026-02-24T22:34:18.334772+0000","last_clean":"2026-02-24T22:34:18.334772+0000","last_became_active":"2026-02-24T22:05:20.195202+0000","last_became_peered":"2026-02-24T22:05:20.195202+0000","last_unstale":"2026-02-24T22:34:18.334772+0000","last_undegraded":"2026-02-24T22:34:18.334772+0000","last_fullsized":"2026-02-24T22:34:18.334772+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:34:18.002332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1242,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":486,"num_read_kb":719,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"312'680","reported_seq":1751,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.538422+0000","last_change":"2026-02-24T22:05:22.192219+0000","last_active":"2026-02-24T22:34:18.538422+0000","last_peered":"2026-02-24T22:34:18.538422+0000","last_clean":"2026-02-24T22:34:18.538422+0000","last_became_active":"2026-02-24T22:05:22.192035+0000","last_became_peered":"2026-02-24T22:05:22.192035+0000","last_unstale":"2026-02-24T22:34:18.538422+0000","last_undegraded":"2026-02-24T22:34:18.538422+0000","last_fullsized":"2026-02-24T22:34:18.538422+0000","mapping_epoch":28,"log_start":"312'600","ondisk_log_start":"312'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":600,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:05:53.268094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50571,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2200,"num_read_kb":1666,"num_write":310,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"312'2477","reported_seq":5136,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.218480+0000","last_change":"2026-02-24T22:05:24.206516+0000","last_active":"2026-02-24T22:34:18.218480+0000","last_peered":"2026-02-24T22:34:18.218480+0000","last_clean":"2026-02-24T22:34:18.218480+0000","last_became_active":"2026-02-24T22:05:24.205557+0000","last_became_peered":"2026-02-24T22:05:24.205557+0000","last_unstale":"2026-02-24T22:34:18.218480+0000","last_undegraded":"2026-02-24T22:34:18.218480+0000","last_fullsized":"2026-02-24T22:34:18.218480+0000","mapping_epoch":30,"log_start":"312'2400","ondisk_log_start":"312'2400","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":2400,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:33:57.249269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34698,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":6755,"num_read_kb":6753,"num_write":5061,"num_write_kb":2022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:39.720637+0000","last_change":"2026-02-24T22:05:29.651962+0000","last_active":"2026-02-24T22:29:39.720637+0000","last_peered":"2026-02-24T22:29:39.720637+0000","last_clean":"2026-02-24T22:29:39.720637+0000","last_became_active":"2026-02-24T22:05:29.651760+0000","last_became_peered":"2026-02-24T22:05:29.651760+0000","last_unstale":"2026-02-24T22:29:39.720637+0000","last_undegraded":"2026-02-24T22:29:39.720637+0000","last_fullsized":"2026-02-24T22:29:39.720637+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:48:41.143100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"312'128","reported_seq":805,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:59.734737+0000","last_change":"2026-02-24T22:05:18.185689+0000","last_active":"2026-02-24T22:31:59.734737+0000","last_peered":"2026-02-24T22:31:59.734737+0000","last_clean":"2026-02-24T22:31:59.734737+0000","last_became_active":"2026-02-24T22:05:18.184939+0000","last_became_peered":"2026-02-24T22:05:18.184939+0000","last_unstale":"2026-02-24T22:31:59.734737+0000","last_undegraded":"2026-02-24T22:31:59.734737+0000","last_fullsized":"2026-02-24T22:31:59.734737+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:18:46.745974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":436,"num_read_kb":360,"num_write":185,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"312'215","reported_seq":939,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.731199+0000","last_change":"2026-02-24T22:05:16.186833+0000","last_active":"2026-02-24T22:31:24.731199+0000","last_peered":"2026-02-24T22:31:24.731199+0000","last_clean":"2026-02-24T22:31:24.731199+0000","last_became_active":"2026-02-24T22:05:16.186043+0000","last_became_peered":"2026-02-24T22:05:16.186043+0000","last_unstale":"2026-02-24T22:31:24.731199+0000","last_undegraded":"2026-02-24T22:31:24.731199+0000","last_fullsized":"2026-02-24T22:31:24.731199+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:59:46.638347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5313539,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":397,"num_read_kb":15748,"num_write":1186,"num_write_kb":5255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.742847+0000","last_change":"2026-02-24T22:05:20.193254+0000","last_active":"2026-02-24T22:31:54.742847+0000","last_peered":"2026-02-24T22:31:54.742847+0000","last_clean":"2026-02-24T22:31:54.742847+0000","last_became_active":"2026-02-24T22:05:20.192962+0000","last_became_peered":"2026-02-24T22:05:20.192962+0000","last_unstale":"2026-02-24T22:31:54.742847+0000","last_undegraded":"2026-02-24T22:31:54.742847+0000","last_fullsized":"2026-02-24T22:31:54.742847+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:15:25.988404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"312'469","reported_seq":1300,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.539799+0000","last_change":"2026-02-24T22:05:22.194159+0000","last_active":"2026-02-24T22:34:18.539799+0000","last_peered":"2026-02-24T22:34:18.539799+0000","last_clean":"2026-02-24T22:34:18.539799+0000","last_became_active":"2026-02-24T22:05:22.193896+0000","last_became_peered":"2026-02-24T22:05:22.193896+0000","last_unstale":"2026-02-24T22:34:18.539799+0000","last_undegraded":"2026-02-24T22:34:18.539799+0000","last_fullsized":"2026-02-24T22:34:18.539799+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":400,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:30:50.341834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59985,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1293,"num_read_kb":1284,"num_write":349,"num_write_kb":316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"312'188","reported_seq":800,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.715171+0000","last_change":"2026-02-24T22:05:24.206176+0000","last_active":"2026-02-24T22:34:17.715171+0000","last_peered":"2026-02-24T22:34:17.715171+0000","last_clean":"2026-02-24T22:34:17.715171+0000","last_became_active":"2026-02-24T22:05:24.205855+0000","last_became_peered":"2026-02-24T22:05:24.205855+0000","last_unstale":"2026-02-24T22:34:17.715171+0000","last_undegraded":"2026-02-24T22:34:17.715171+0000","last_fullsized":"2026-02-24T22:34:17.715171+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":100,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:18:56.993738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27729,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":196,"num_read_kb":179,"num_write":471,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"312'443","reported_seq":1614,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.571252+0000","last_change":"2026-02-24T22:05:29.662310+0000","last_active":"2026-02-24T22:34:18.571252+0000","last_peered":"2026-02-24T22:34:18.571252+0000","last_clean":"2026-02-24T22:34:18.571252+0000","last_became_active":"2026-02-24T22:05:29.661983+0000","last_became_peered":"2026-02-24T22:05:29.661983+0000","last_unstale":"2026-02-24T22:34:18.571252+0000","last_undegraded":"2026-02-24T22:34:18.571252+0000","last_fullsized":"2026-02-24T22:34:18.571252+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":400,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:00:08.834953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"312'169","reported_seq":852,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.250160+0000","last_change":"2026-02-24T22:05:18.191009+0000","last_active":"2026-02-24T22:34:17.250160+0000","last_peered":"2026-02-24T22:34:17.250160+0000","last_clean":"2026-02-24T22:34:17.250160+0000","last_became_active":"2026-02-24T22:05:18.189877+0000","last_became_peered":"2026-02-24T22:05:18.189877+0000","last_unstale":"2026-02-24T22:34:17.250160+0000","last_undegraded":"2026-02-24T22:34:17.250160+0000","last_fullsized":"2026-02-24T22:34:17.250160+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:33:55.890050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":510,"num_write":249,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"312'253","reported_seq":1012,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:04.870640+0000","last_change":"2026-02-24T22:05:16.184964+0000","last_active":"2026-02-24T22:34:04.870640+0000","last_peered":"2026-02-24T22:34:04.870640+0000","last_clean":"2026-02-24T22:34:04.870640+0000","last_became_active":"2026-02-24T22:05:16.184525+0000","last_became_peered":"2026-02-24T22:05:16.184525+0000","last_unstale":"2026-02-24T22:34:04.870640+0000","last_undegraded":"2026-02-24T22:34:04.870640+0000","last_fullsized":"2026-02-24T22:34:04.870640+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:33:11.439600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10579665,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":508,"num_read_kb":16318,"num_write":1420,"num_write_kb":16752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"27'1","reported_seq":589,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:59.729910+0000","last_change":"2026-02-24T22:05:20.187071+0000","last_active":"2026-02-24T22:31:59.729910+0000","last_peered":"2026-02-24T22:31:59.729910+0000","last_clean":"2026-02-24T22:31:59.729910+0000","last_became_active":"2026-02-24T22:05:20.186850+0000","last_became_peered":"2026-02-24T22:05:20.186850+0000","last_unstale":"2026-02-24T22:31:59.729910+0000","last_undegraded":"2026-02-24T22:31:59.729910+0000","last_fullsized":"2026-02-24T22:31:59.729910+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:28:00.779043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"312'244","reported_seq":862,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.831208+0000","last_change":"2026-02-24T22:05:22.199568+0000","last_active":"2026-02-24T22:34:17.831208+0000","last_peered":"2026-02-24T22:34:17.831208+0000","last_clean":"2026-02-24T22:34:17.831208+0000","last_became_active":"2026-02-24T22:05:22.199358+0000","last_became_peered":"2026-02-24T22:05:22.199358+0000","last_unstale":"2026-02-24T22:34:17.831208+0000","last_undegraded":"2026-02-24T22:34:17.831208+0000","last_fullsized":"2026-02-24T22:34:17.831208+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":200,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:24:45.196918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59999,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":388,"num_read_kb":835,"num_write":354,"num_write_kb":306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"312'281","reported_seq":1222,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.228289+0000","last_change":"2026-02-24T22:05:24.200431+0000","last_active":"2026-02-24T22:34:18.228289+0000","last_peered":"2026-02-24T22:34:18.228289+0000","last_clean":"2026-02-24T22:34:18.228289+0000","last_became_active":"2026-02-24T22:05:24.199555+0000","last_became_peered":"2026-02-24T22:05:24.199555+0000","last_unstale":"2026-02-24T22:34:18.228289+0000","last_undegraded":"2026-02-24T22:34:18.228289+0000","last_fullsized":"2026-02-24T22:34:18.228289+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:29:23.063946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30969,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":655,"num_read_kb":647,"num_write":639,"num_write_kb":204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"312'429","reported_seq":1947,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.571358+0000","last_change":"2026-02-24T22:05:29.660609+0000","last_active":"2026-02-24T22:34:18.571358+0000","last_peered":"2026-02-24T22:34:18.571358+0000","last_clean":"2026-02-24T22:34:18.571358+0000","last_became_active":"2026-02-24T22:05:29.659532+0000","last_became_peered":"2026-02-24T22:05:29.659532+0000","last_unstale":"2026-02-24T22:34:18.571358+0000","last_undegraded":"2026-02-24T22:34:18.571358+0000","last_fullsized":"2026-02-24T22:34:18.571358+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":400,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:09:07.648679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"312'139","reported_seq":825,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.725732+0000","last_change":"2026-02-24T22:05:18.184639+0000","last_active":"2026-02-24T22:31:19.725732+0000","last_peered":"2026-02-24T22:31:19.725732+0000","last_clean":"2026-02-24T22:31:19.725732+0000","last_became_active":"2026-02-24T22:05:18.184250+0000","last_became_peered":"2026-02-24T22:05:18.184250+0000","last_unstale":"2026-02-24T22:31:19.725732+0000","last_undegraded":"2026-02-24T22:31:19.725732+0000","last_fullsized":"2026-02-24T22:31:19.725732+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:38:26.901167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":560,"num_read_kb":462,"num_write":204,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"312'1519","reported_seq":2874,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.226363+0000","last_change":"2026-02-24T22:05:16.190173+0000","last_active":"2026-02-24T22:34:17.226363+0000","last_peered":"2026-02-24T22:34:17.226363+0000","last_clean":"2026-02-24T22:34:17.226363+0000","last_became_active":"2026-02-24T22:05:16.189660+0000","last_became_peered":"2026-02-24T22:05:16.189660+0000","last_unstale":"2026-02-24T22:34:17.226363+0000","last_undegraded":"2026-02-24T22:34:17.226363+0000","last_fullsized":"2026-02-24T22:34:17.226363+0000","mapping_epoch":22,"log_start":"312'1500","ondisk_log_start":"312'1500","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":1500,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:28:22.592939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":80905,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":4244,"num_read_kb":2601,"num_write":2602,"num_write_kb":4261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"312'101","reported_seq":779,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:01.356810+0000","last_change":"2026-02-24T22:05:18.184329+0000","last_active":"2026-02-24T22:34:01.356810+0000","last_peered":"2026-02-24T22:34:01.356810+0000","last_clean":"2026-02-24T22:34:01.356810+0000","last_became_active":"2026-02-24T22:05:18.183822+0000","last_became_peered":"2026-02-24T22:05:18.183822+0000","last_unstale":"2026-02-24T22:34:01.356810+0000","last_undegraded":"2026-02-24T22:34:01.356810+0000","last_fullsized":"2026-02-24T22:34:01.356810+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:25:35.055559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":390,"num_read_kb":330,"num_write":143,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"312'231","reported_seq":979,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:53.833443+0000","last_change":"2026-02-24T22:05:16.183674+0000","last_active":"2026-02-24T22:30:53.833443+0000","last_peered":"2026-02-24T22:30:53.833443+0000","last_clean":"2026-02-24T22:30:53.833443+0000","last_became_active":"2026-02-24T22:05:16.183487+0000","last_became_peered":"2026-02-24T22:05:16.183487+0000","last_unstale":"2026-02-24T22:30:53.833443+0000","last_undegraded":"2026-02-24T22:30:53.833443+0000","last_fullsized":"2026-02-24T22:30:53.833443+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":200,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:25:02.663360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117418,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":445,"num_read_kb":706,"num_write":1288,"num_write_kb":183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"312'72","reported_seq":731,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.741422+0000","last_change":"2026-02-24T22:05:18.190755+0000","last_active":"2026-02-24T22:31:19.741422+0000","last_peered":"2026-02-24T22:31:19.741422+0000","last_clean":"2026-02-24T22:31:19.741422+0000","last_became_active":"2026-02-24T22:05:18.190352+0000","last_became_peered":"2026-02-24T22:05:18.190352+0000","last_unstale":"2026-02-24T22:31:19.741422+0000","last_undegraded":"2026-02-24T22:31:19.741422+0000","last_fullsized":"2026-02-24T22:31:19.741422+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:16:19.432104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":274,"num_read_kb":236,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"312'236","reported_seq":1158,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:12.225391+0000","last_change":"2026-02-24T22:05:16.191430+0000","last_active":"2026-02-24T22:34:12.225391+0000","last_peered":"2026-02-24T22:34:12.225391+0000","last_clean":"2026-02-24T22:34:12.225391+0000","last_became_active":"2026-02-24T22:05:16.190494+0000","last_became_peered":"2026-02-24T22:05:16.190494+0000","last_unstale":"2026-02-24T22:34:12.225391+0000","last_undegraded":"2026-02-24T22:34:12.225391+0000","last_fullsized":"2026-02-24T22:34:12.225391+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":200,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:14:56.208933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6381012,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":1034,"num_read_kb":14913,"num_write":1257,"num_write_kb":8068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"312'337","reported_seq":1150,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.725684+0000","last_change":"2026-02-24T22:05:18.186968+0000","last_active":"2026-02-24T22:31:19.725684+0000","last_peered":"2026-02-24T22:31:19.725684+0000","last_clean":"2026-02-24T22:31:19.725684+0000","last_became_active":"2026-02-24T22:05:18.186622+0000","last_became_peered":"2026-02-24T22:05:18.186622+0000","last_unstale":"2026-02-24T22:31:19.725684+0000","last_undegraded":"2026-02-24T22:31:19.725684+0000","last_fullsized":"2026-02-24T22:31:19.725684+0000","mapping_epoch":24,"log_start":"312'300","ondisk_log_start":"312'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":300,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:34:46.036169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1512,"num_read_kb":1316,"num_write":481,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"312'236","reported_seq":1075,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:06.918892+0000","last_change":"2026-02-24T22:05:16.186186+0000","last_active":"2026-02-24T22:34:06.918892+0000","last_peered":"2026-02-24T22:34:06.918892+0000","last_clean":"2026-02-24T22:34:06.918892+0000","last_became_active":"2026-02-24T22:05:16.185977+0000","last_became_peered":"2026-02-24T22:05:16.185977+0000","last_unstale":"2026-02-24T22:34:06.918892+0000","last_undegraded":"2026-02-24T22:34:06.918892+0000","last_fullsized":"2026-02-24T22:34:06.918892+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":200,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:53:50.492464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1207187,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":871,"num_read_kb":5969,"num_write":1291,"num_write_kb":2943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"312'89","reported_seq":764,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:00.529273+0000","last_change":"2026-02-24T22:05:18.186317+0000","last_active":"2026-02-24T22:34:00.529273+0000","last_peered":"2026-02-24T22:34:00.529273+0000","last_clean":"2026-02-24T22:34:00.529273+0000","last_became_active":"2026-02-24T22:05:18.185180+0000","last_became_peered":"2026-02-24T22:05:18.185180+0000","last_unstale":"2026-02-24T22:34:00.529273+0000","last_undegraded":"2026-02-24T22:34:00.529273+0000","last_fullsized":"2026-02-24T22:34:00.529273+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:17:22.057256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":311,"num_read_kb":264,"num_write":113,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"312'283","reported_seq":1035,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:59.726421+0000","last_change":"2026-02-24T22:06:00.660921+0000","last_active":"2026-02-24T22:31:59.726421+0000","last_peered":"2026-02-24T22:31:59.726421+0000","last_clean":"2026-02-24T22:31:59.726421+0000","last_became_active":"2026-02-24T22:06:00.660740+0000","last_became_peered":"2026-02-24T22:06:00.660740+0000","last_unstale":"2026-02-24T22:31:59.726421+0000","last_undegraded":"2026-02-24T22:31:59.726421+0000","last_fullsized":"2026-02-24T22:31:59.726421+0000","mapping_epoch":36,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":200,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:58:32.246490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12132727,"num_objects":86,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":579,"num_read_kb":9889,"num_write":1318,"num_write_kb":11975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"312'66","reported_seq":735,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.732670+0000","last_change":"2026-02-24T22:05:18.191552+0000","last_active":"2026-02-24T22:30:19.732670+0000","last_peered":"2026-02-24T22:30:19.732670+0000","last_clean":"2026-02-24T22:30:19.732670+0000","last_became_active":"2026-02-24T22:05:18.190390+0000","last_became_peered":"2026-02-24T22:05:18.190390+0000","last_unstale":"2026-02-24T22:30:19.732670+0000","last_undegraded":"2026-02-24T22:30:19.732670+0000","last_fullsized":"2026-02-24T22:30:19.732670+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:20:31.773222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":213,"num_read_kb":187,"num_write":90,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"312'255","reported_seq":978,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.725697+0000","last_change":"2026-02-24T22:05:16.184281+0000","last_active":"2026-02-24T22:31:19.725697+0000","last_peered":"2026-02-24T22:31:19.725697+0000","last_clean":"2026-02-24T22:31:19.725697+0000","last_became_active":"2026-02-24T22:05:16.184091+0000","last_became_peered":"2026-02-24T22:05:16.184091+0000","last_unstale":"2026-02-24T22:31:19.725697+0000","last_undegraded":"2026-02-24T22:31:19.725697+0000","last_fullsized":"2026-02-24T22:31:19.725697+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:42:08.074892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6296559,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":482,"num_read_kb":400,"num_write":1161,"num_write_kb":6221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"312'355","reported_seq":1157,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:45.969143+0000","last_change":"2026-02-24T22:05:18.187245+0000","last_active":"2026-02-24T22:33:45.969143+0000","last_peered":"2026-02-24T22:33:45.969143+0000","last_clean":"2026-02-24T22:33:45.969143+0000","last_became_active":"2026-02-24T22:05:18.186746+0000","last_became_peered":"2026-02-24T22:05:18.186746+0000","last_unstale":"2026-02-24T22:33:45.969143+0000","last_undegraded":"2026-02-24T22:33:45.969143+0000","last_fullsized":"2026-02-24T22:33:45.969143+0000","mapping_epoch":24,"log_start":"312'300","ondisk_log_start":"312'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":300,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:16:49.207184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":1676,"num_read_kb":1392,"num_write":638,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"312'250","reported_seq":954,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.737867+0000","last_change":"2026-02-24T22:05:16.190391+0000","last_active":"2026-02-24T22:31:19.737867+0000","last_peered":"2026-02-24T22:31:19.737867+0000","last_clean":"2026-02-24T22:31:19.737867+0000","last_became_active":"2026-02-24T22:05:16.189516+0000","last_became_peered":"2026-02-24T22:05:16.189516+0000","last_unstale":"2026-02-24T22:31:19.737867+0000","last_undegraded":"2026-02-24T22:31:19.737867+0000","last_fullsized":"2026-02-24T22:31:19.737867+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":200,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:11:15.635731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10540630,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":369,"num_read_kb":5491,"num_write":1149,"num_write_kb":20654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"312'164","reported_seq":854,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:03.781941+0000","last_change":"2026-02-24T22:05:18.190682+0000","last_active":"2026-02-24T22:34:03.781941+0000","last_peered":"2026-02-24T22:34:03.781941+0000","last_clean":"2026-02-24T22:34:03.781941+0000","last_became_active":"2026-02-24T22:05:18.190203+0000","last_became_peered":"2026-02-24T22:05:18.190203+0000","last_unstale":"2026-02-24T22:34:03.781941+0000","last_undegraded":"2026-02-24T22:34:03.781941+0000","last_fullsized":"2026-02-24T22:34:03.781941+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:20:56.349004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":631,"num_read_kb":516,"num_write":248,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"312'241","reported_seq":960,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:12.654487+0000","last_change":"2026-02-24T22:05:16.184697+0000","last_active":"2026-02-24T22:34:12.654487+0000","last_peered":"2026-02-24T22:34:12.654487+0000","last_clean":"2026-02-24T22:34:12.654487+0000","last_became_active":"2026-02-24T22:05:16.183772+0000","last_became_peered":"2026-02-24T22:05:16.183772+0000","last_unstale":"2026-02-24T22:34:12.654487+0000","last_undegraded":"2026-02-24T22:34:12.654487+0000","last_fullsized":"2026-02-24T22:34:12.654487+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":200,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:22:42.349222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6042885,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":404,"num_read_kb":12485,"num_write":1265,"num_write_kb":5970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"312'130","reported_seq":850,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.737414+0000","last_change":"2026-02-24T22:05:18.187529+0000","last_active":"2026-02-24T22:30:59.737414+0000","last_peered":"2026-02-24T22:30:59.737414+0000","last_clean":"2026-02-24T22:30:59.737414+0000","last_became_active":"2026-02-24T22:05:18.187321+0000","last_became_peered":"2026-02-24T22:05:18.187321+0000","last_unstale":"2026-02-24T22:30:59.737414+0000","last_undegraded":"2026-02-24T22:30:59.737414+0000","last_fullsized":"2026-02-24T22:30:59.737414+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:08:55.925019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":512,"num_read_kb":441,"num_write":201,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"312'238","reported_seq":946,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:58.282425+0000","last_change":"2026-02-24T22:05:16.193409+0000","last_active":"2026-02-24T22:33:58.282425+0000","last_peered":"2026-02-24T22:33:58.282425+0000","last_clean":"2026-02-24T22:33:58.282425+0000","last_became_active":"2026-02-24T22:05:16.193164+0000","last_became_peered":"2026-02-24T22:05:16.193164+0000","last_unstale":"2026-02-24T22:33:58.282425+0000","last_undegraded":"2026-02-24T22:33:58.282425+0000","last_fullsized":"2026-02-24T22:33:58.282425+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":200,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:41:44.916032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10795512,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":356,"num_read_kb":36420,"num_write":1300,"num_write_kb":10617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"312'148","reported_seq":825,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.737040+0000","last_change":"2026-02-24T22:05:18.193362+0000","last_active":"2026-02-24T22:30:59.737040+0000","last_peered":"2026-02-24T22:30:59.737040+0000","last_clean":"2026-02-24T22:30:59.737040+0000","last_became_active":"2026-02-24T22:05:18.191737+0000","last_became_peered":"2026-02-24T22:05:18.191737+0000","last_unstale":"2026-02-24T22:30:59.737040+0000","last_undegraded":"2026-02-24T22:30:59.737040+0000","last_fullsized":"2026-02-24T22:30:59.737040+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":100,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:51:20.868848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":539,"num_read_kb":448,"num_write":211,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"312'298","reported_seq":1049,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:54.775236+0000","last_change":"2026-02-24T22:05:16.184466+0000","last_active":"2026-02-24T22:30:54.775236+0000","last_peered":"2026-02-24T22:30:54.775236+0000","last_clean":"2026-02-24T22:30:54.775236+0000","last_became_active":"2026-02-24T22:05:16.183923+0000","last_became_peered":"2026-02-24T22:05:16.183923+0000","last_unstale":"2026-02-24T22:30:54.775236+0000","last_undegraded":"2026-02-24T22:30:54.775236+0000","last_fullsized":"2026-02-24T22:30:54.775236+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:15:14.475036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5523994,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":476,"num_read_kb":9842,"num_write":1525,"num_write_kb":9073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"312'146","reported_seq":817,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:09.715504+0000","last_change":"2026-02-24T22:05:18.184019+0000","last_active":"2026-02-24T22:30:09.715504+0000","last_peered":"2026-02-24T22:30:09.715504+0000","last_clean":"2026-02-24T22:30:09.715504+0000","last_became_active":"2026-02-24T22:05:18.183761+0000","last_became_peered":"2026-02-24T22:05:18.183761+0000","last_unstale":"2026-02-24T22:30:09.715504+0000","last_undegraded":"2026-02-24T22:30:09.715504+0000","last_fullsized":"2026-02-24T22:30:09.715504+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:07:03.909559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":561,"num_read_kb":451,"num_write":221,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"312'267","reported_seq":1036,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:44.059088+0000","last_change":"2026-02-24T22:05:16.191018+0000","last_active":"2026-02-24T22:33:44.059088+0000","last_peered":"2026-02-24T22:33:44.059088+0000","last_clean":"2026-02-24T22:33:44.059088+0000","last_became_active":"2026-02-24T22:05:16.189759+0000","last_became_peered":"2026-02-24T22:05:16.189759+0000","last_unstale":"2026-02-24T22:33:44.059088+0000","last_undegraded":"2026-02-24T22:33:44.059088+0000","last_fullsized":"2026-02-24T22:33:44.059088+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":200,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:34:51.111413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1371100,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":587,"num_read_kb":3529,"num_write":1434,"num_write_kb":3094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"312'140","reported_seq":832,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:31.161689+0000","last_change":"2026-02-24T22:05:18.194125+0000","last_active":"2026-02-24T22:30:31.161689+0000","last_peered":"2026-02-24T22:30:31.161689+0000","last_clean":"2026-02-24T22:30:31.161689+0000","last_became_active":"2026-02-24T22:05:18.192882+0000","last_became_peered":"2026-02-24T22:05:18.192882+0000","last_unstale":"2026-02-24T22:30:31.161689+0000","last_undegraded":"2026-02-24T22:30:31.161689+0000","last_fullsized":"2026-02-24T22:30:31.161689+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":100,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:13:16.621031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":593,"num_read_kb":494,"num_write":208,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"312'228","reported_seq":966,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.253876+0000","last_change":"2026-02-24T22:05:16.184964+0000","last_active":"2026-02-24T22:34:17.253876+0000","last_peered":"2026-02-24T22:34:17.253876+0000","last_clean":"2026-02-24T22:34:17.253876+0000","last_became_active":"2026-02-24T22:05:16.184557+0000","last_became_peered":"2026-02-24T22:05:16.184557+0000","last_unstale":"2026-02-24T22:34:17.253876+0000","last_undegraded":"2026-02-24T22:34:17.253876+0000","last_fullsized":"2026-02-24T22:34:17.253876+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":200,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:16:07.490514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5643921,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":420,"num_read_kb":1536,"num_write":1181,"num_write_kb":5590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"312'158","reported_seq":865,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:04.901988+0000","last_change":"2026-02-24T22:05:18.191676+0000","last_active":"2026-02-24T22:34:04.901988+0000","last_peered":"2026-02-24T22:34:04.901988+0000","last_clean":"2026-02-24T22:34:04.901988+0000","last_became_active":"2026-02-24T22:05:18.191432+0000","last_became_peered":"2026-02-24T22:05:18.191432+0000","last_unstale":"2026-02-24T22:34:04.901988+0000","last_undegraded":"2026-02-24T22:34:04.901988+0000","last_fullsized":"2026-02-24T22:34:04.901988+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":100,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:14:52.433749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":649,"num_read_kb":535,"num_write":233,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"312'222","reported_seq":943,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:00.269049+0000","last_change":"2026-02-24T22:05:16.190689+0000","last_active":"2026-02-24T22:34:00.269049+0000","last_peered":"2026-02-24T22:34:00.269049+0000","last_clean":"2026-02-24T22:34:00.269049+0000","last_became_active":"2026-02-24T22:05:16.190260+0000","last_became_peered":"2026-02-24T22:05:16.190260+0000","last_unstale":"2026-02-24T22:34:00.269049+0000","last_undegraded":"2026-02-24T22:34:00.269049+0000","last_fullsized":"2026-02-24T22:34:00.269049+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":200,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:55:30.357144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":254576,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":395,"num_read_kb":1504,"num_write":1294,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"312'143","reported_seq":831,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:15.308606+0000","last_change":"2026-02-24T22:05:18.185534+0000","last_active":"2026-02-24T22:30:15.308606+0000","last_peered":"2026-02-24T22:30:15.308606+0000","last_clean":"2026-02-24T22:30:15.308606+0000","last_became_active":"2026-02-24T22:05:18.185184+0000","last_became_peered":"2026-02-24T22:05:18.185184+0000","last_unstale":"2026-02-24T22:30:15.308606+0000","last_undegraded":"2026-02-24T22:30:15.308606+0000","last_fullsized":"2026-02-24T22:30:15.308606+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":100,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:50:27.214195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":536,"num_read_kb":445,"num_write":200,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"312'228","reported_seq":1041,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:32.940510+0000","last_change":"2026-02-24T22:05:16.191775+0000","last_active":"2026-02-24T22:33:32.940510+0000","last_peered":"2026-02-24T22:33:32.940510+0000","last_clean":"2026-02-24T22:33:32.940510+0000","last_became_active":"2026-02-24T22:05:16.191433+0000","last_became_peered":"2026-02-24T22:05:16.191433+0000","last_unstale":"2026-02-24T22:33:32.940510+0000","last_undegraded":"2026-02-24T22:33:32.940510+0000","last_fullsized":"2026-02-24T22:33:32.940510+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":200,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:08:07.147079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5270943,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":479,"num_read_kb":2264,"num_write":1260,"num_write_kb":7006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"312'62","reported_seq":721,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:08.697007+0000","last_change":"2026-02-24T22:05:18.192824+0000","last_active":"2026-02-24T22:30:08.697007+0000","last_peered":"2026-02-24T22:30:08.697007+0000","last_clean":"2026-02-24T22:30:08.697007+0000","last_became_active":"2026-02-24T22:05:18.192619+0000","last_became_peered":"2026-02-24T22:05:18.192619+0000","last_unstale":"2026-02-24T22:30:08.697007+0000","last_undegraded":"2026-02-24T22:30:08.697007+0000","last_fullsized":"2026-02-24T22:30:08.697007+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:21:57.498044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":204,"num_read_kb":176,"num_write":76,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"312'253","reported_seq":985,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:54.852196+0000","last_change":"2026-02-24T22:05:16.184618+0000","last_active":"2026-02-24T22:33:54.852196+0000","last_peered":"2026-02-24T22:33:54.852196+0000","last_clean":"2026-02-24T22:33:54.852196+0000","last_became_active":"2026-02-24T22:05:16.184441+0000","last_became_peered":"2026-02-24T22:05:16.184441+0000","last_unstale":"2026-02-24T22:33:54.852196+0000","last_undegraded":"2026-02-24T22:33:54.852196+0000","last_fullsized":"2026-02-24T22:33:54.852196+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:14:27.469981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1578451,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":396,"num_read_kb":3850,"num_write":1386,"num_write_kb":8445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"312'159","reported_seq":876,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:58.577766+0000","last_change":"2026-02-24T22:05:18.191572+0000","last_active":"2026-02-24T22:33:58.577766+0000","last_peered":"2026-02-24T22:33:58.577766+0000","last_clean":"2026-02-24T22:33:58.577766+0000","last_became_active":"2026-02-24T22:05:18.191350+0000","last_became_peered":"2026-02-24T22:05:18.191350+0000","last_unstale":"2026-02-24T22:33:58.577766+0000","last_undegraded":"2026-02-24T22:33:58.577766+0000","last_fullsized":"2026-02-24T22:33:58.577766+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:05:49.312781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":662,"num_read_kb":557,"num_write":245,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"312'219","reported_seq":987,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:43.905970+0000","last_change":"2026-02-24T22:06:00.662883+0000","last_active":"2026-02-24T22:33:43.905970+0000","last_peered":"2026-02-24T22:33:43.905970+0000","last_clean":"2026-02-24T22:33:43.905970+0000","last_became_active":"2026-02-24T22:06:00.662656+0000","last_became_peered":"2026-02-24T22:06:00.662656+0000","last_unstale":"2026-02-24T22:33:43.905970+0000","last_undegraded":"2026-02-24T22:33:43.905970+0000","last_fullsized":"2026-02-24T22:33:43.905970+0000","mapping_epoch":36,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":200,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:35:38.140607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5604982,"num_objects":86,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":721,"num_read_kb":6090,"num_write":1121,"num_write_kb":5556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.30","version":"312'268","reported_seq":1020,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:51.515479+0000","last_change":"2026-02-24T22:05:16.190494+0000","last_active":"2026-02-24T22:33:51.515479+0000","last_peered":"2026-02-24T22:33:51.515479+0000","last_clean":"2026-02-24T22:33:51.515479+0000","last_became_active":"2026-02-24T22:05:16.189713+0000","last_became_peered":"2026-02-24T22:05:16.189713+0000","last_unstale":"2026-02-24T22:33:51.515479+0000","last_undegraded":"2026-02-24T22:33:51.515479+0000","last_fullsized":"2026-02-24T22:33:51.515479+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":200,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:32:51.972858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10823593,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":564,"num_read_kb":31408,"num_write":1387,"num_write_kb":10638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"312'256","reported_seq":941,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:59.744014+0000","last_change":"2026-02-24T22:05:18.187784+0000","last_active":"2026-02-24T22:31:59.744014+0000","last_peered":"2026-02-24T22:31:59.744014+0000","last_clean":"2026-02-24T22:31:59.744014+0000","last_became_active":"2026-02-24T22:05:18.187565+0000","last_became_peered":"2026-02-24T22:05:18.187565+0000","last_unstale":"2026-02-24T22:31:59.744014+0000","last_undegraded":"2026-02-24T22:31:59.744014+0000","last_fullsized":"2026-02-24T22:31:59.744014+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":200,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:43:19.977135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":1060,"num_read_kb":853,"num_write":419,"num_write_kb":222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"312'225","reported_seq":931,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.723653+0000","last_change":"2026-02-24T22:05:16.178897+0000","last_active":"2026-02-24T22:30:59.723653+0000","last_peered":"2026-02-24T22:30:59.723653+0000","last_clean":"2026-02-24T22:30:59.723653+0000","last_became_active":"2026-02-24T22:05:16.178157+0000","last_became_peered":"2026-02-24T22:05:16.178157+0000","last_unstale":"2026-02-24T22:30:59.723653+0000","last_undegraded":"2026-02-24T22:30:59.723653+0000","last_fullsized":"2026-02-24T22:30:59.723653+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":200,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:50:48.164748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":621009,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":397,"num_read_kb":1640,"num_write":1065,"num_write_kb":1702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"312'78","reported_seq":752,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:51.565638+0000","last_change":"2026-02-24T22:05:18.191188+0000","last_active":"2026-02-24T22:33:51.565638+0000","last_peered":"2026-02-24T22:33:51.565638+0000","last_clean":"2026-02-24T22:33:51.565638+0000","last_became_active":"2026-02-24T22:05:18.190902+0000","last_became_peered":"2026-02-24T22:05:18.190902+0000","last_unstale":"2026-02-24T22:33:51.565638+0000","last_undegraded":"2026-02-24T22:33:51.565638+0000","last_fullsized":"2026-02-24T22:33:51.565638+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:18:59.644388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":302,"num_read_kb":262,"num_write":98,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":388,"ondisk_log_size":388,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":973109,"num_objects":3053,"num_object_clones":0,"num_object_copies":6106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3053,"num_whiteouts":0,"num_read":19978,"num_read_kb":245567,"num_write":23762,"num_write_kb":8619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24911872,"data_stored":1946218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5786661,"internal_metadata":0},"log_size":2182,"ondisk_log_size":2182,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1738830,"num_objects":223,"num_object_clones":0,"num_object_copies":446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":223,"num_whiteouts":0,"num_read":44721,"num_read_kb":43148,"num_write":15259,"num_write_kb":13598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3874816,"data_stored":3174742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2823344,"internal_metadata":0},"log_size":2043,"ondisk_log_size":2043,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1896,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1442,"num_read_kb":1673,"num_write":22,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":82,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11005,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":1511,"num_object_clones":0,"num_object_copies":3022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1511,"num_whiteouts":0,"num_read":95919,"num_read_kb":81897,"num_write":34645,"num_write_kb":17078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1511,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17301930,"internal_metadata":0},"log_size":3916,"ondisk_log_size":3916,"up":128,"acting":128,"num_store_stats":8},{"poolid":81,"num_pg":32,"stat_sum":{"num_bytes":72351744,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":71,"num_read_kb":119808,"num_write":63,"num_write_kb":70656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":144703488,"data_stored":144703488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":375571674,"num_objects":4668,"num_object_clones":0,"num_object_copies":9336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4668,"num_whiteouts":0,"num_read":36656,"num_read_kb":763453,"num_write":81696,"num_write_kb":523059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":786833408,"data_stored":751143348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2729,"ondisk_log_size":2729,"up":128,"acting":128,"num_store_stats":8},{"poolid":80,"num_pg":32,"stat_sum":{"num_bytes":72351744,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":73,"num_read_kb":124928,"num_write":63,"num_write_kb":70656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":144703488,"data_stored":144703488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":79,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3567,"num_read_kb":2401,"num_write":1127,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":317720,"internal_metadata":0},"log_size":462,"ondisk_log_size":462,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575202,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442900,"kb_used_data":199364,"kb_used_omap":3696,"kb_used_meta":239823,"kb_avail":93928940,"statfs":{"total":96636764160,"available":96183234560,"internally_reserved":0,"allocated":204148736,"data_stored":194101579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3785637,"internal_metadata":245578843},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575202,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":325324,"kb_used_data":81788,"kb_used_omap":3571,"kb_used_meta":239948,"kb_avail":94046516,"statfs":{"total":96636764160,"available":96303632384,"internally_reserved":0,"allocated":83750912,"data_stored":75161771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3657488,"internal_metadata":245706992},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575202,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":276592,"kb_used_data":177504,"kb_used_omap":2746,"kb_used_meta":96325,"kb_avail":94095248,"statfs":{"total":96636764160,"available":96353533952,"internally_reserved":0,"allocated":181764096,"data_stored":173324874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2812671,"internal_metadata":98637057},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575202,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":215740,"kb_used_data":152684,"kb_used_omap":3095,"kb_used_meta":59944,"kb_avail":94156100,"statfs":{"total":96636764160,"available":96415846400,"internally_reserved":0,"allocated":156348416,"data_stored":147731024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3169290,"internal_metadata":61383670},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575203,"num_pgs":107,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":329208,"kb_used_data":121768,"kb_used_omap":7232,"kb_used_meta":200191,"kb_avail":94042632,"statfs":{"total":96636764160,"available":96299655168,"internally_reserved":0,"allocated":124690432,"data_stored":113814900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7406186,"internal_metadata":204995990},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575203,"num_pgs":87,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":162392,"kb_used_data":117384,"kb_used_omap":1599,"kb_used_meta":43392,"kb_avail":94209448,"statfs":{"total":96636764160,"available":96470474752,"internally_reserved":0,"allocated":120201216,"data_stored":112600299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1638118,"internal_metadata":44433690},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575204,"num_pgs":102,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":178812,"kb_used_data":79660,"kb_used_omap":5094,"kb_used_meta":94041,"kb_avail":94193028,"statfs":{"total":96636764160,"available":96453660672,"internally_reserved":0,"allocated":81571840,"data_stored":73974457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5216565,"internal_metadata":96298699},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575204,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":408384,"kb_used_data":182960,"kb_used_omap":10223,"kb_used_meta":215184,"kb_avail":93963456,"statfs":{"total":96636764160,"available":96218578944,"internally_reserved":0,"allocated":187351040,"data_stored":176579114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10468372,"internal_metadata":220349420},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":129925120,"data_stored":124511737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":51691520,"data_stored":48716374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69758976,"data_stored":66666572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":88662016,"data_stored":83153600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":142684160,"data_stored":137577305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":122421248,"data_stored":117533084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":59645952,"data_stored":56294670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":122044416,"data_stored":116690006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1667351,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2633902,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1516609,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1844562,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2430346,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":723382,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3546560,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2939218,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":378928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1512,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":442706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1072852,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":148539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":667648,"data_stored":537960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18545,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":107453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5184,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":770048,"data_stored":659074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1705285,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":286394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11530,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":720896,"data_stored":613688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8436,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3928064,"data_stored":308322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51420,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3117056,"data_stored":241195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":152550,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3059712,"data_stored":239505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":825183,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3899392,"data_stored":305873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":336934,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1949696,"data_stored":151651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7338,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1925120,"data_stored":148450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":65762,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3837952,"data_stored":298046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2489145,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3194880,"data_stored":253176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1858329,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":79,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":133235,"internal_metadata":0},{"poolid":79,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14290,"internal_metadata":0},{"poolid":79,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23118,"internal_metadata":0},{"poolid":79,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15635,"internal_metadata":0},{"poolid":79,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49631,"internal_metadata":0},{"poolid":79,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49251,"internal_metadata":0},{"poolid":79,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16656,"internal_metadata":0},{"poolid":79,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15904,"internal_metadata":0},{"poolid":80,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26214400,"data_stored":26214400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16777216,"data_stored":16777216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":23068672,"data_stored":23068672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":23068672,"data_stored":23068672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":22020096,"data_stored":22020096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28311552,"data_stored":28311552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22020096,"data_stored":22020096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19922944,"data_stored":19922944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7340032,"data_stored":7340032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":30408704,"data_stored":30408704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45088768,"data_stored":45088768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":82,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":82,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1886,"internal_metadata":0},{"poolid":82,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6440,"internal_metadata":0},{"poolid":82,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":262,"internal_metadata":0},{"poolid":82,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1265,"internal_metadata":0},{"poolid":82,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":717,"internal_metadata":0},{"poolid":82,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":435,"internal_metadata":0},{"poolid":82,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T22:34:27.307 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-24T22:34:27.307 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-24T22:34:27.408 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 has been restored 2026-02-24T22:34:27.484 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:27.501 INFO:teuthology.orchestra.run.trial046.stderr:dumped all 2026-02-24T22:34:27.503 INFO:teuthology.orchestra.run.trial046.stdout:{"pg_ready":true,"pg_map":{"version":1186,"stamp":"2026-02-24T22:34:26.023106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":523579384,"num_objects":9510,"num_object_clones":0,"num_object_copies":19020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9510,"num_whiteouts":0,"num_read":202525,"num_read_kb":1382960,"num_write":156802,"num_write_kb":706681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":1543,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11819,"ondisk_log_size":11819,"up":786,"acting":786,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":786,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2339352,"kb_used_data":1113112,"kb_used_omap":37260,"kb_used_meta":1188851,"kb_avail":752635368,"statfs":{"total":773094113280,"available":770698616832,"internally_reserved":0,"allocated":1139826688,"data_stored":1067288018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38154327,"internal_metadata":1217384361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30542903,"num_objects":-41,"num_object_clones":0,"num_object_copies":-82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-41,"num_whiteouts":0,"num_read":1479,"num_read_kb":60619,"num_write":280,"num_write_kb":29926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003202"},"pg_stats":[{"pgid":"82.2","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744748+0000","last_change":"2026-02-24T22:23:29.198729+0000","last_active":"2026-02-24T22:31:49.744748+0000","last_peered":"2026-02-24T22:31:49.744748+0000","last_clean":"2026-02-24T22:31:49.744748+0000","last_became_active":"2026-02-24T22:23:29.198476+0000","last_became_peered":"2026-02-24T22:23:29.198476+0000","last_unstale":"2026-02-24T22:31:49.744748+0000","last_undegraded":"2026-02-24T22:31:49.744748+0000","last_fullsized":"2026-02-24T22:31:49.744748+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:12:35.165001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.1","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.730924+0000","last_change":"2026-02-24T22:15:25.830397+0000","last_active":"2026-02-24T22:32:09.730924+0000","last_peered":"2026-02-24T22:32:09.730924+0000","last_clean":"2026-02-24T22:32:09.730924+0000","last_became_active":"2026-02-24T22:15:25.830000+0000","last_became_peered":"2026-02-24T22:15:25.830000+0000","last_unstale":"2026-02-24T22:32:09.730924+0000","last_undegraded":"2026-02-24T22:32:09.730924+0000","last_fullsized":"2026-02-24T22:32:09.730924+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:41:30.400463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.0","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:29.723258+0000","last_change":"2026-02-24T22:15:22.817921+0000","last_active":"2026-02-24T22:31:29.723258+0000","last_peered":"2026-02-24T22:31:29.723258+0000","last_clean":"2026-02-24T22:31:29.723258+0000","last_became_active":"2026-02-24T22:15:22.817446+0000","last_became_peered":"2026-02-24T22:15:22.817446+0000","last_unstale":"2026-02-24T22:31:29.723258+0000","last_undegraded":"2026-02-24T22:31:29.723258+0000","last_fullsized":"2026-02-24T22:31:29.723258+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:25:30.206672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.1f","version":"312'20","reported_seq":120,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.733669+0000","last_change":"2026-02-24T22:13:43.761604+0000","last_active":"2026-02-24T22:30:24.733669+0000","last_peered":"2026-02-24T22:30:24.733669+0000","last_clean":"2026-02-24T22:30:24.733669+0000","last_became_active":"2026-02-24T22:13:43.761120+0000","last_became_peered":"2026-02-24T22:13:43.761120+0000","last_unstale":"2026-02-24T22:30:24.733669+0000","last_undegraded":"2026-02-24T22:30:24.733669+0000","last_fullsized":"2026-02-24T22:30:24.733669+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:48:55.248638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":168,"num_read_kb":114,"num_write":47,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.3","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742865+0000","last_change":"2026-02-24T22:23:29.213779+0000","last_active":"2026-02-24T22:31:49.742865+0000","last_peered":"2026-02-24T22:31:49.742865+0000","last_clean":"2026-02-24T22:31:49.742865+0000","last_became_active":"2026-02-24T22:23:29.213534+0000","last_became_peered":"2026-02-24T22:23:29.213534+0000","last_unstale":"2026-02-24T22:31:49.742865+0000","last_undegraded":"2026-02-24T22:31:49.742865+0000","last_fullsized":"2026-02-24T22:31:49.742865+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:38:56.711711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.0","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.743144+0000","last_change":"2026-02-24T22:15:25.827702+0000","last_active":"2026-02-24T22:31:34.743144+0000","last_peered":"2026-02-24T22:31:34.743144+0000","last_clean":"2026-02-24T22:31:34.743144+0000","last_became_active":"2026-02-24T22:15:25.827355+0000","last_became_peered":"2026-02-24T22:15:25.827355+0000","last_unstale":"2026-02-24T22:31:34.743144+0000","last_undegraded":"2026-02-24T22:31:34.743144+0000","last_fullsized":"2026-02-24T22:31:34.743144+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:22:12.090995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.1","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:04.744507+0000","last_change":"2026-02-24T22:15:22.818757+0000","last_active":"2026-02-24T22:32:04.744507+0000","last_peered":"2026-02-24T22:32:04.744507+0000","last_clean":"2026-02-24T22:32:04.744507+0000","last_became_active":"2026-02-24T22:15:22.818519+0000","last_became_peered":"2026-02-24T22:15:22.818519+0000","last_unstale":"2026-02-24T22:32:04.744507+0000","last_undegraded":"2026-02-24T22:32:04.744507+0000","last_fullsized":"2026-02-24T22:32:04.744507+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:17:56.378157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.1e","version":"312'9","reported_seq":88,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.745172+0000","last_change":"2026-02-24T22:13:43.757760+0000","last_active":"2026-02-24T22:31:49.745172+0000","last_peered":"2026-02-24T22:31:49.745172+0000","last_clean":"2026-02-24T22:31:49.745172+0000","last_became_active":"2026-02-24T22:13:43.757543+0000","last_became_peered":"2026-02-24T22:13:43.757543+0000","last_unstale":"2026-02-24T22:31:49.745172+0000","last_undegraded":"2026-02-24T22:31:49.745172+0000","last_fullsized":"2026-02-24T22:31:49.745172+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:44:23.087405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":38,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.0","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.723743+0000","last_change":"2026-02-24T22:23:29.212801+0000","last_active":"2026-02-24T22:31:34.723743+0000","last_peered":"2026-02-24T22:31:34.723743+0000","last_clean":"2026-02-24T22:31:34.723743+0000","last_became_active":"2026-02-24T22:23:29.212043+0000","last_became_peered":"2026-02-24T22:23:29.212043+0000","last_unstale":"2026-02-24T22:31:34.723743+0000","last_undegraded":"2026-02-24T22:31:34.723743+0000","last_fullsized":"2026-02-24T22:31:34.723743+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:45:48.953242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.3","version":"307'5","reported_seq":50,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.747921+0000","last_change":"2026-02-24T22:15:25.827590+0000","last_active":"2026-02-24T22:32:09.747921+0000","last_peered":"2026-02-24T22:32:09.747921+0000","last_clean":"2026-02-24T22:32:09.747921+0000","last_became_active":"2026-02-24T22:15:25.827216+0000","last_became_peered":"2026-02-24T22:15:25.827216+0000","last_unstale":"2026-02-24T22:32:09.747921+0000","last_undegraded":"2026-02-24T22:32:09.747921+0000","last_fullsized":"2026-02-24T22:32:09.747921+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:08:31.068519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485760,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":25600,"num_write":10,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.2","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:04.726927+0000","last_change":"2026-02-24T22:15:22.818494+0000","last_active":"2026-02-24T22:32:04.726927+0000","last_peered":"2026-02-24T22:32:04.726927+0000","last_clean":"2026-02-24T22:32:04.726927+0000","last_became_active":"2026-02-24T22:15:22.818194+0000","last_became_peered":"2026-02-24T22:15:22.818194+0000","last_unstale":"2026-02-24T22:32:04.726927+0000","last_undegraded":"2026-02-24T22:32:04.726927+0000","last_fullsized":"2026-02-24T22:32:04.726927+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:30:45.313454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.1d","version":"312'22","reported_seq":121,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.737198+0000","last_change":"2026-02-24T22:13:43.757707+0000","last_active":"2026-02-24T22:31:14.737198+0000","last_peered":"2026-02-24T22:31:14.737198+0000","last_clean":"2026-02-24T22:31:14.737198+0000","last_became_active":"2026-02-24T22:13:43.757278+0000","last_became_peered":"2026-02-24T22:13:43.757278+0000","last_unstale":"2026-02-24T22:31:14.737198+0000","last_undegraded":"2026-02-24T22:31:14.737198+0000","last_fullsized":"2026-02-24T22:31:14.737198+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:52:33.539660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":103,"num_write":54,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.1","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742841+0000","last_change":"2026-02-24T22:23:29.214295+0000","last_active":"2026-02-24T22:31:49.742841+0000","last_peered":"2026-02-24T22:31:49.742841+0000","last_clean":"2026-02-24T22:31:49.742841+0000","last_became_active":"2026-02-24T22:23:29.214064+0000","last_became_peered":"2026-02-24T22:23:29.214064+0000","last_unstale":"2026-02-24T22:31:49.742841+0000","last_undegraded":"2026-02-24T22:31:49.742841+0000","last_fullsized":"2026-02-24T22:31:49.742841+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:03:04.365969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.2","version":"307'1","reported_seq":35,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.735734+0000","last_change":"2026-02-24T22:15:25.833367+0000","last_active":"2026-02-24T22:32:09.735734+0000","last_peered":"2026-02-24T22:32:09.735734+0000","last_clean":"2026-02-24T22:32:09.735734+0000","last_became_active":"2026-02-24T22:15:25.832809+0000","last_became_peered":"2026-02-24T22:15:25.832809+0000","last_unstale":"2026-02-24T22:32:09.735734+0000","last_undegraded":"2026-02-24T22:32:09.735734+0000","last_fullsized":"2026-02-24T22:32:09.735734+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:28:13.551199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":4096,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"80.3","version":"307'2","reported_seq":43,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:04.726940+0000","last_change":"2026-02-24T22:15:22.818401+0000","last_active":"2026-02-24T22:32:04.726940+0000","last_peered":"2026-02-24T22:32:04.726940+0000","last_clean":"2026-02-24T22:32:04.726940+0000","last_became_active":"2026-02-24T22:15:22.818089+0000","last_became_peered":"2026-02-24T22:15:22.818089+0000","last_unstale":"2026-02-24T22:32:04.726940+0000","last_undegraded":"2026-02-24T22:32:04.726940+0000","last_fullsized":"2026-02-24T22:32:04.726940+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:40:25.179819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.1c","version":"312'14","reported_seq":101,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.739030+0000","last_change":"2026-02-24T22:13:43.760682+0000","last_active":"2026-02-24T22:31:14.739030+0000","last_peered":"2026-02-24T22:31:14.739030+0000","last_clean":"2026-02-24T22:31:14.739030+0000","last_became_active":"2026-02-24T22:13:43.759958+0000","last_became_peered":"2026-02-24T22:13:43.759958+0000","last_unstale":"2026-02-24T22:31:14.739030+0000","last_undegraded":"2026-02-24T22:31:14.739030+0000","last_fullsized":"2026-02-24T22:31:14.739030+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:24:39.289428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":75,"num_write":36,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.6","version":"309'1","reported_seq":23,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.730973+0000","last_change":"2026-02-24T22:23:29.215576+0000","last_active":"2026-02-24T22:32:09.730973+0000","last_peered":"2026-02-24T22:32:09.730973+0000","last_clean":"2026-02-24T22:32:09.730973+0000","last_became_active":"2026-02-24T22:23:29.215327+0000","last_became_peered":"2026-02-24T22:23:29.215327+0000","last_unstale":"2026-02-24T22:32:09.730973+0000","last_undegraded":"2026-02-24T22:32:09.730973+0000","last_fullsized":"2026-02-24T22:32:09.730973+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:51:01.281339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.5","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:14.745596+0000","last_change":"2026-02-24T22:15:25.832444+0000","last_active":"2026-02-24T22:32:14.745596+0000","last_peered":"2026-02-24T22:32:14.745596+0000","last_clean":"2026-02-24T22:32:14.745596+0000","last_became_active":"2026-02-24T22:15:25.831975+0000","last_became_peered":"2026-02-24T22:15:25.831975+0000","last_unstale":"2026-02-24T22:32:14.745596+0000","last_undegraded":"2026-02-24T22:32:14.745596+0000","last_fullsized":"2026-02-24T22:32:14.745596+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:04:17.334836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"80.4","version":"307'2","reported_seq":43,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:04.747123+0000","last_change":"2026-02-24T22:15:22.812647+0000","last_active":"2026-02-24T22:32:04.747123+0000","last_peered":"2026-02-24T22:32:04.747123+0000","last_clean":"2026-02-24T22:32:04.747123+0000","last_became_active":"2026-02-24T22:15:22.812270+0000","last_became_peered":"2026-02-24T22:15:22.812270+0000","last_unstale":"2026-02-24T22:32:04.747123+0000","last_undegraded":"2026-02-24T22:32:04.747123+0000","last_fullsized":"2026-02-24T22:32:04.747123+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:42:27.774508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.1b","version":"312'18","reported_seq":112,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.741475+0000","last_change":"2026-02-24T22:13:43.755424+0000","last_active":"2026-02-24T22:31:14.741475+0000","last_peered":"2026-02-24T22:31:14.741475+0000","last_clean":"2026-02-24T22:31:14.741475+0000","last_became_active":"2026-02-24T22:13:43.755170+0000","last_became_peered":"2026-02-24T22:13:43.755170+0000","last_unstale":"2026-02-24T22:31:14.741475+0000","last_undegraded":"2026-02-24T22:31:14.741475+0000","last_fullsized":"2026-02-24T22:31:14.741475+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:29:17.090743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":101,"num_write":42,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.7","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.735768+0000","last_change":"2026-02-24T22:23:29.214459+0000","last_active":"2026-02-24T22:32:09.735768+0000","last_peered":"2026-02-24T22:32:09.735768+0000","last_clean":"2026-02-24T22:32:09.735768+0000","last_became_active":"2026-02-24T22:23:29.214105+0000","last_became_peered":"2026-02-24T22:23:29.214105+0000","last_unstale":"2026-02-24T22:32:09.735768+0000","last_undegraded":"2026-02-24T22:32:09.735768+0000","last_fullsized":"2026-02-24T22:32:09.735768+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:00:10.302637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.4","version":"307'2","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.747275+0000","last_change":"2026-02-24T22:15:25.827716+0000","last_active":"2026-02-24T22:32:09.747275+0000","last_peered":"2026-02-24T22:32:09.747275+0000","last_clean":"2026-02-24T22:32:09.747275+0000","last_became_active":"2026-02-24T22:15:25.826663+0000","last_became_peered":"2026-02-24T22:15:25.826663+0000","last_unstale":"2026-02-24T22:32:09.747275+0000","last_undegraded":"2026-02-24T22:32:09.747275+0000","last_fullsized":"2026-02-24T22:32:09.747275+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:08:34.544962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.5","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.730960+0000","last_change":"2026-02-24T22:15:22.816880+0000","last_active":"2026-02-24T22:32:09.730960+0000","last_peered":"2026-02-24T22:32:09.730960+0000","last_clean":"2026-02-24T22:32:09.730960+0000","last_became_active":"2026-02-24T22:15:22.816697+0000","last_became_peered":"2026-02-24T22:15:22.816697+0000","last_unstale":"2026-02-24T22:32:09.730960+0000","last_undegraded":"2026-02-24T22:32:09.730960+0000","last_fullsized":"2026-02-24T22:32:09.730960+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:56:01.670361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"79.1a","version":"312'16","reported_seq":102,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.737175+0000","last_change":"2026-02-24T22:13:43.757739+0000","last_active":"2026-02-24T22:31:14.737175+0000","last_peered":"2026-02-24T22:31:14.737175+0000","last_clean":"2026-02-24T22:31:14.737175+0000","last_became_active":"2026-02-24T22:13:43.757353+0000","last_became_peered":"2026-02-24T22:13:43.757353+0000","last_unstale":"2026-02-24T22:31:14.737175+0000","last_undegraded":"2026-02-24T22:31:14.737175+0000","last_fullsized":"2026-02-24T22:31:14.737175+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:47:30.725736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":65,"num_write":38,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.4","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742881+0000","last_change":"2026-02-24T22:23:29.215206+0000","last_active":"2026-02-24T22:31:49.742881+0000","last_peered":"2026-02-24T22:31:49.742881+0000","last_clean":"2026-02-24T22:31:49.742881+0000","last_became_active":"2026-02-24T22:23:29.215014+0000","last_became_peered":"2026-02-24T22:23:29.215014+0000","last_unstale":"2026-02-24T22:31:49.742881+0000","last_undegraded":"2026-02-24T22:31:49.742881+0000","last_fullsized":"2026-02-24T22:31:49.742881+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:43:42.586588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.7","version":"307'2","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.718321+0000","last_change":"2026-02-24T22:15:25.832214+0000","last_active":"2026-02-24T22:30:39.718321+0000","last_peered":"2026-02-24T22:30:39.718321+0000","last_clean":"2026-02-24T22:30:39.718321+0000","last_became_active":"2026-02-24T22:15:25.831806+0000","last_became_peered":"2026-02-24T22:15:25.831806+0000","last_unstale":"2026-02-24T22:30:39.718321+0000","last_undegraded":"2026-02-24T22:30:39.718321+0000","last_fullsized":"2026-02-24T22:30:39.718321+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:36:56.341773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"80.6","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:34.736157+0000","last_change":"2026-02-24T22:15:22.811674+0000","last_active":"2026-02-24T22:30:34.736157+0000","last_peered":"2026-02-24T22:30:34.736157+0000","last_clean":"2026-02-24T22:30:34.736157+0000","last_became_active":"2026-02-24T22:15:22.811413+0000","last_became_peered":"2026-02-24T22:15:22.811413+0000","last_unstale":"2026-02-24T22:30:34.736157+0000","last_undegraded":"2026-02-24T22:30:34.736157+0000","last_fullsized":"2026-02-24T22:30:34.736157+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:14:20.030861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"79.19","version":"312'30","reported_seq":140,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.738215+0000","last_change":"2026-02-24T22:13:43.756439+0000","last_active":"2026-02-24T22:30:44.738215+0000","last_peered":"2026-02-24T22:30:44.738215+0000","last_clean":"2026-02-24T22:30:44.738215+0000","last_became_active":"2026-02-24T22:13:43.756114+0000","last_became_peered":"2026-02-24T22:13:43.756114+0000","last_unstale":"2026-02-24T22:30:44.738215+0000","last_undegraded":"2026-02-24T22:30:44.738215+0000","last_fullsized":"2026-02-24T22:30:44.738215+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:45:46.139575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":216,"num_read_kb":143,"num_write":68,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.5","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742892+0000","last_change":"2026-02-24T22:23:29.213537+0000","last_active":"2026-02-24T22:31:49.742892+0000","last_peered":"2026-02-24T22:31:49.742892+0000","last_clean":"2026-02-24T22:31:49.742892+0000","last_became_active":"2026-02-24T22:23:29.213256+0000","last_became_peered":"2026-02-24T22:23:29.213256+0000","last_unstale":"2026-02-24T22:31:49.742892+0000","last_undegraded":"2026-02-24T22:31:49.742892+0000","last_fullsized":"2026-02-24T22:31:49.742892+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:48:26.724830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.6","version":"307'2","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.721669+0000","last_change":"2026-02-24T22:15:25.833922+0000","last_active":"2026-02-24T22:30:39.721669+0000","last_peered":"2026-02-24T22:30:39.721669+0000","last_clean":"2026-02-24T22:30:39.721669+0000","last_became_active":"2026-02-24T22:15:25.833680+0000","last_became_peered":"2026-02-24T22:15:25.833680+0000","last_unstale":"2026-02-24T22:30:39.721669+0000","last_undegraded":"2026-02-24T22:30:39.721669+0000","last_fullsized":"2026-02-24T22:30:39.721669+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:26:51.677018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.7","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:34.736181+0000","last_change":"2026-02-24T22:15:22.812054+0000","last_active":"2026-02-24T22:30:34.736181+0000","last_peered":"2026-02-24T22:30:34.736181+0000","last_clean":"2026-02-24T22:30:34.736181+0000","last_became_active":"2026-02-24T22:15:22.811618+0000","last_became_peered":"2026-02-24T22:15:22.811618+0000","last_unstale":"2026-02-24T22:30:34.736181+0000","last_undegraded":"2026-02-24T22:30:34.736181+0000","last_fullsized":"2026-02-24T22:30:34.736181+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:37:58.903441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"79.18","version":"312'12","reported_seq":95,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.737670+0000","last_change":"2026-02-24T22:13:43.755501+0000","last_active":"2026-02-24T22:30:39.737670+0000","last_peered":"2026-02-24T22:30:39.737670+0000","last_clean":"2026-02-24T22:30:39.737670+0000","last_became_active":"2026-02-24T22:13:43.754981+0000","last_became_peered":"2026-02-24T22:13:43.754981+0000","last_unstale":"2026-02-24T22:30:39.737670+0000","last_undegraded":"2026-02-24T22:30:39.737670+0000","last_fullsized":"2026-02-24T22:30:39.737670+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:41:38.765379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":86,"num_read_kb":57,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.a","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.745052+0000","last_change":"2026-02-24T22:23:29.212877+0000","last_active":"2026-02-24T22:32:09.745052+0000","last_peered":"2026-02-24T22:32:09.745052+0000","last_clean":"2026-02-24T22:32:09.745052+0000","last_became_active":"2026-02-24T22:23:29.212213+0000","last_became_peered":"2026-02-24T22:23:29.212213+0000","last_unstale":"2026-02-24T22:32:09.745052+0000","last_undegraded":"2026-02-24T22:32:09.745052+0000","last_fullsized":"2026-02-24T22:32:09.745052+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:29:28.605583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.9","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.736816+0000","last_change":"2026-02-24T22:15:25.826412+0000","last_active":"2026-02-24T22:30:39.736816+0000","last_peered":"2026-02-24T22:30:39.736816+0000","last_clean":"2026-02-24T22:30:39.736816+0000","last_became_active":"2026-02-24T22:15:25.826076+0000","last_became_peered":"2026-02-24T22:15:25.826076+0000","last_unstale":"2026-02-24T22:30:39.736816+0000","last_undegraded":"2026-02-24T22:30:39.736816+0000","last_fullsized":"2026-02-24T22:30:39.736816+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:00:25.763716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.8","version":"307'4","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:34.717818+0000","last_change":"2026-02-24T22:15:22.816346+0000","last_active":"2026-02-24T22:30:34.717818+0000","last_peered":"2026-02-24T22:30:34.717818+0000","last_clean":"2026-02-24T22:30:34.717818+0000","last_became_active":"2026-02-24T22:15:22.815960+0000","last_became_peered":"2026-02-24T22:15:22.815960+0000","last_unstale":"2026-02-24T22:30:34.717818+0000","last_undegraded":"2026-02-24T22:30:34.717818+0000","last_fullsized":"2026-02-24T22:30:34.717818+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:55:34.277066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":6144,"num_write":9,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.17","version":"312'34","reported_seq":169,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.721687+0000","last_change":"2026-02-24T22:14:00.749701+0000","last_active":"2026-02-24T22:30:39.721687+0000","last_peered":"2026-02-24T22:30:39.721687+0000","last_clean":"2026-02-24T22:30:39.721687+0000","last_became_active":"2026-02-24T22:14:00.748438+0000","last_became_peered":"2026-02-24T22:14:00.748438+0000","last_unstale":"2026-02-24T22:30:39.721687+0000","last_undegraded":"2026-02-24T22:30:39.721687+0000","last_fullsized":"2026-02-24T22:30:39.721687+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":293,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:21:49.815767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":298,"num_read_kb":211,"num_write":78,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.b","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.730451+0000","last_change":"2026-02-24T22:23:29.197335+0000","last_active":"2026-02-24T22:30:14.730451+0000","last_peered":"2026-02-24T22:30:14.730451+0000","last_clean":"2026-02-24T22:30:14.730451+0000","last_became_active":"2026-02-24T22:23:29.196790+0000","last_became_peered":"2026-02-24T22:23:29.196790+0000","last_unstale":"2026-02-24T22:30:14.730451+0000","last_undegraded":"2026-02-24T22:30:14.730451+0000","last_fullsized":"2026-02-24T22:30:14.730451+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:55:13.008271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.8","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.734884+0000","last_change":"2026-02-24T22:15:25.825567+0000","last_active":"2026-02-24T22:30:39.734884+0000","last_peered":"2026-02-24T22:30:39.734884+0000","last_clean":"2026-02-24T22:30:39.734884+0000","last_became_active":"2026-02-24T22:15:25.825315+0000","last_became_peered":"2026-02-24T22:15:25.825315+0000","last_unstale":"2026-02-24T22:30:39.734884+0000","last_undegraded":"2026-02-24T22:30:39.734884+0000","last_fullsized":"2026-02-24T22:30:39.734884+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:24:39.313916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.9","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:34.732583+0000","last_change":"2026-02-24T22:15:22.811365+0000","last_active":"2026-02-24T22:30:34.732583+0000","last_peered":"2026-02-24T22:30:34.732583+0000","last_clean":"2026-02-24T22:30:34.732583+0000","last_became_active":"2026-02-24T22:15:22.811141+0000","last_became_peered":"2026-02-24T22:15:22.811141+0000","last_unstale":"2026-02-24T22:30:34.732583+0000","last_undegraded":"2026-02-24T22:30:34.732583+0000","last_fullsized":"2026-02-24T22:30:34.732583+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:45:06.384015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"79.16","version":"0'0","reported_seq":63,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.733272+0000","last_change":"2026-02-24T22:13:43.756602+0000","last_active":"2026-02-24T22:30:39.733272+0000","last_peered":"2026-02-24T22:30:39.733272+0000","last_clean":"2026-02-24T22:30:39.733272+0000","last_became_active":"2026-02-24T22:13:43.755885+0000","last_became_peered":"2026-02-24T22:13:43.755885+0000","last_unstale":"2026-02-24T22:30:39.733272+0000","last_undegraded":"2026-02-24T22:30:39.733272+0000","last_fullsized":"2026-02-24T22:30:39.733272+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:40:07.171226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.8","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.744882+0000","last_change":"2026-02-24T22:23:29.199299+0000","last_active":"2026-02-24T22:32:09.744882+0000","last_peered":"2026-02-24T22:32:09.744882+0000","last_clean":"2026-02-24T22:32:09.744882+0000","last_became_active":"2026-02-24T22:23:29.198882+0000","last_became_peered":"2026-02-24T22:23:29.198882+0000","last_unstale":"2026-02-24T22:32:09.744882+0000","last_undegraded":"2026-02-24T22:32:09.744882+0000","last_fullsized":"2026-02-24T22:32:09.744882+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:59:03.245529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"81.b","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.741464+0000","last_change":"2026-02-24T22:15:25.827664+0000","last_active":"2026-02-24T22:31:14.741464+0000","last_peered":"2026-02-24T22:31:14.741464+0000","last_clean":"2026-02-24T22:31:14.741464+0000","last_became_active":"2026-02-24T22:15:25.827265+0000","last_became_peered":"2026-02-24T22:15:25.827265+0000","last_unstale":"2026-02-24T22:31:14.741464+0000","last_undegraded":"2026-02-24T22:31:14.741464+0000","last_fullsized":"2026-02-24T22:31:14.741464+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:17:29.198425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.a","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.735221+0000","last_change":"2026-02-24T22:15:22.817325+0000","last_active":"2026-02-24T22:30:39.735221+0000","last_peered":"2026-02-24T22:30:39.735221+0000","last_clean":"2026-02-24T22:30:39.735221+0000","last_became_active":"2026-02-24T22:15:22.816702+0000","last_became_peered":"2026-02-24T22:15:22.816702+0000","last_unstale":"2026-02-24T22:30:39.735221+0000","last_undegraded":"2026-02-24T22:30:39.735221+0000","last_fullsized":"2026-02-24T22:30:39.735221+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:48:16.071491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.15","version":"307'6","reported_seq":82,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.735250+0000","last_change":"2026-02-24T22:13:43.761758+0000","last_active":"2026-02-24T22:30:39.735250+0000","last_peered":"2026-02-24T22:30:39.735250+0000","last_clean":"2026-02-24T22:30:39.735250+0000","last_became_active":"2026-02-24T22:13:43.760892+0000","last_became_peered":"2026-02-24T22:13:43.760892+0000","last_unstale":"2026-02-24T22:30:39.735250+0000","last_undegraded":"2026-02-24T22:30:39.735250+0000","last_fullsized":"2026-02-24T22:30:39.735250+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:07:44.028035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":37,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.9","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.743246+0000","last_change":"2026-02-24T22:23:29.198300+0000","last_active":"2026-02-24T22:32:09.743246+0000","last_peered":"2026-02-24T22:32:09.743246+0000","last_clean":"2026-02-24T22:32:09.743246+0000","last_became_active":"2026-02-24T22:23:29.197357+0000","last_became_peered":"2026-02-24T22:23:29.197357+0000","last_unstale":"2026-02-24T22:32:09.743246+0000","last_undegraded":"2026-02-24T22:32:09.743246+0000","last_fullsized":"2026-02-24T22:32:09.743246+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:58:07.606981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.a","version":"307'2","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.738233+0000","last_change":"2026-02-24T22:15:25.827469+0000","last_active":"2026-02-24T22:30:44.738233+0000","last_peered":"2026-02-24T22:30:44.738233+0000","last_clean":"2026-02-24T22:30:44.738233+0000","last_became_active":"2026-02-24T22:15:25.827055+0000","last_became_peered":"2026-02-24T22:15:25.827055+0000","last_unstale":"2026-02-24T22:30:44.738233+0000","last_undegraded":"2026-02-24T22:30:44.738233+0000","last_fullsized":"2026-02-24T22:30:44.738233+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:35:15.420239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.b","version":"307'2","reported_seq":42,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.738494+0000","last_change":"2026-02-24T22:15:22.817894+0000","last_active":"2026-02-24T22:31:09.738494+0000","last_peered":"2026-02-24T22:31:09.738494+0000","last_clean":"2026-02-24T22:31:09.738494+0000","last_became_active":"2026-02-24T22:15:22.817498+0000","last_became_peered":"2026-02-24T22:15:22.817498+0000","last_unstale":"2026-02-24T22:31:09.738494+0000","last_undegraded":"2026-02-24T22:31:09.738494+0000","last_fullsized":"2026-02-24T22:31:09.738494+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:08:45.124818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.14","version":"312'10","reported_seq":89,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:09.718735+0000","last_change":"2026-02-24T22:13:43.762482+0000","last_active":"2026-02-24T22:30:09.718735+0000","last_peered":"2026-02-24T22:30:09.718735+0000","last_clean":"2026-02-24T22:30:09.718735+0000","last_became_active":"2026-02-24T22:13:43.762341+0000","last_became_peered":"2026-02-24T22:13:43.762341+0000","last_unstale":"2026-02-24T22:30:09.718735+0000","last_undegraded":"2026-02-24T22:30:09.718735+0000","last_fullsized":"2026-02-24T22:30:09.718735+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:44:09.549825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":46,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.e","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.724040+0000","last_change":"2026-02-24T22:23:29.215664+0000","last_active":"2026-02-24T22:30:14.724040+0000","last_peered":"2026-02-24T22:30:14.724040+0000","last_clean":"2026-02-24T22:30:14.724040+0000","last_became_active":"2026-02-24T22:23:29.215425+0000","last_became_peered":"2026-02-24T22:23:29.215425+0000","last_unstale":"2026-02-24T22:30:14.724040+0000","last_undegraded":"2026-02-24T22:30:14.724040+0000","last_fullsized":"2026-02-24T22:30:14.724040+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:44:31.538355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.d","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.741498+0000","last_change":"2026-02-24T22:15:25.829241+0000","last_active":"2026-02-24T22:31:14.741498+0000","last_peered":"2026-02-24T22:31:14.741498+0000","last_clean":"2026-02-24T22:31:14.741498+0000","last_became_active":"2026-02-24T22:15:25.829109+0000","last_became_peered":"2026-02-24T22:15:25.829109+0000","last_unstale":"2026-02-24T22:31:14.741498+0000","last_undegraded":"2026-02-24T22:31:14.741498+0000","last_fullsized":"2026-02-24T22:31:14.741498+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:38:08.330741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.c","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.721310+0000","last_change":"2026-02-24T22:15:22.815624+0000","last_active":"2026-02-24T22:31:09.721310+0000","last_peered":"2026-02-24T22:31:09.721310+0000","last_clean":"2026-02-24T22:31:09.721310+0000","last_became_active":"2026-02-24T22:15:22.815364+0000","last_became_peered":"2026-02-24T22:15:22.815364+0000","last_unstale":"2026-02-24T22:31:09.721310+0000","last_undegraded":"2026-02-24T22:31:09.721310+0000","last_fullsized":"2026-02-24T22:31:09.721310+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:21:36.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.13","version":"312'11","reported_seq":95,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.714998+0000","last_change":"2026-02-24T22:13:43.760988+0000","last_active":"2026-02-24T22:30:04.714998+0000","last_peered":"2026-02-24T22:30:04.714998+0000","last_clean":"2026-02-24T22:30:04.714998+0000","last_became_active":"2026-02-24T22:13:43.760714+0000","last_became_peered":"2026-02-24T22:13:43.760714+0000","last_unstale":"2026-02-24T22:30:04.714998+0000","last_undegraded":"2026-02-24T22:30:04.714998+0000","last_fullsized":"2026-02-24T22:30:04.714998+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:55:23.837996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":61,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"82.f","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.730477+0000","last_change":"2026-02-24T22:23:29.199754+0000","last_active":"2026-02-24T22:30:14.730477+0000","last_peered":"2026-02-24T22:30:14.730477+0000","last_clean":"2026-02-24T22:30:14.730477+0000","last_became_active":"2026-02-24T22:23:29.199538+0000","last_became_peered":"2026-02-24T22:23:29.199538+0000","last_unstale":"2026-02-24T22:30:14.730477+0000","last_undegraded":"2026-02-24T22:30:14.730477+0000","last_fullsized":"2026-02-24T22:30:14.730477+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:15:25.304156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.c","version":"307'7","reported_seq":54,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.740681+0000","last_change":"2026-02-24T22:15:25.827267+0000","last_active":"2026-02-24T22:31:14.740681+0000","last_peered":"2026-02-24T22:31:14.740681+0000","last_clean":"2026-02-24T22:31:14.740681+0000","last_became_active":"2026-02-24T22:15:25.826430+0000","last_became_peered":"2026-02-24T22:15:25.826430+0000","last_unstale":"2026-02-24T22:31:14.740681+0000","last_undegraded":"2026-02-24T22:31:14.740681+0000","last_fullsized":"2026-02-24T22:31:14.740681+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:49:19.806352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":31744,"num_write":12,"num_write_kb":16384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.d","version":"307'3","reported_seq":50,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.740980+0000","last_change":"2026-02-24T22:15:22.813197+0000","last_active":"2026-02-24T22:31:09.740980+0000","last_peered":"2026-02-24T22:31:09.740980+0000","last_clean":"2026-02-24T22:31:09.740980+0000","last_became_active":"2026-02-24T22:15:22.813062+0000","last_became_peered":"2026-02-24T22:15:22.813062+0000","last_unstale":"2026-02-24T22:31:09.740980+0000","last_undegraded":"2026-02-24T22:31:09.740980+0000","last_fullsized":"2026-02-24T22:31:09.740980+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:08:36.468537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":20480,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.12","version":"312'20","reported_seq":117,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:48.279108+0000","last_change":"2026-02-24T22:13:43.760420+0000","last_active":"2026-02-24T22:30:48.279108+0000","last_peered":"2026-02-24T22:30:48.279108+0000","last_clean":"2026-02-24T22:30:48.279108+0000","last_became_active":"2026-02-24T22:13:43.760196+0000","last_became_peered":"2026-02-24T22:13:43.760196+0000","last_unstale":"2026-02-24T22:30:48.279108+0000","last_undegraded":"2026-02-24T22:30:48.279108+0000","last_fullsized":"2026-02-24T22:30:48.279108+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:14:15.856109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":106,"num_write":52,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"82.c","version":"310'1","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.734866+0000","last_change":"2026-02-24T22:23:29.201743+0000","last_active":"2026-02-24T22:30:14.734866+0000","last_peered":"2026-02-24T22:30:14.734866+0000","last_clean":"2026-02-24T22:30:14.734866+0000","last_became_active":"2026-02-24T22:23:29.201545+0000","last_became_peered":"2026-02-24T22:23:29.201545+0000","last_unstale":"2026-02-24T22:30:14.734866+0000","last_undegraded":"2026-02-24T22:30:14.734866+0000","last_fullsized":"2026-02-24T22:30:14.734866+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:07:57.527033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"81.f","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.741445+0000","last_change":"2026-02-24T22:15:25.825325+0000","last_active":"2026-02-24T22:31:19.741445+0000","last_peered":"2026-02-24T22:31:19.741445+0000","last_clean":"2026-02-24T22:31:19.741445+0000","last_became_active":"2026-02-24T22:15:25.824862+0000","last_became_peered":"2026-02-24T22:15:25.824862+0000","last_unstale":"2026-02-24T22:31:19.741445+0000","last_undegraded":"2026-02-24T22:31:19.741445+0000","last_fullsized":"2026-02-24T22:31:19.741445+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:16:51.320757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.e","version":"307'3","reported_seq":52,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.721326+0000","last_change":"2026-02-24T22:15:22.817941+0000","last_active":"2026-02-24T22:31:09.721326+0000","last_peered":"2026-02-24T22:31:09.721326+0000","last_clean":"2026-02-24T22:31:09.721326+0000","last_became_active":"2026-02-24T22:15:22.817295+0000","last_became_peered":"2026-02-24T22:15:22.817295+0000","last_unstale":"2026-02-24T22:31:09.721326+0000","last_undegraded":"2026-02-24T22:31:09.721326+0000","last_fullsized":"2026-02-24T22:31:09.721326+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:37:42.904886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":25600,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.11","version":"312'11","reported_seq":92,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.731364+0000","last_change":"2026-02-24T22:13:43.760497+0000","last_active":"2026-02-24T22:30:04.731364+0000","last_peered":"2026-02-24T22:30:04.731364+0000","last_clean":"2026-02-24T22:30:04.731364+0000","last_became_active":"2026-02-24T22:13:43.759864+0000","last_became_peered":"2026-02-24T22:13:43.759864+0000","last_unstale":"2026-02-24T22:30:04.731364+0000","last_undegraded":"2026-02-24T22:30:04.731364+0000","last_fullsized":"2026-02-24T22:30:04.731364+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:50:51.585161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":53,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.d","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.724028+0000","last_change":"2026-02-24T22:23:29.214699+0000","last_active":"2026-02-24T22:30:14.724028+0000","last_peered":"2026-02-24T22:30:14.724028+0000","last_clean":"2026-02-24T22:30:14.724028+0000","last_became_active":"2026-02-24T22:23:29.214432+0000","last_became_peered":"2026-02-24T22:23:29.214432+0000","last_unstale":"2026-02-24T22:30:14.724028+0000","last_undegraded":"2026-02-24T22:30:14.724028+0000","last_fullsized":"2026-02-24T22:30:14.724028+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:09:09.120137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.e","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.738719+0000","last_change":"2026-02-24T22:15:25.827425+0000","last_active":"2026-02-24T22:31:14.738719+0000","last_peered":"2026-02-24T22:31:14.738719+0000","last_clean":"2026-02-24T22:31:14.738719+0000","last_became_active":"2026-02-24T22:15:25.827110+0000","last_became_peered":"2026-02-24T22:15:25.827110+0000","last_unstale":"2026-02-24T22:31:14.738719+0000","last_undegraded":"2026-02-24T22:31:14.738719+0000","last_fullsized":"2026-02-24T22:31:14.738719+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:25:06.342633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.f","version":"307'5","reported_seq":55,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.739061+0000","last_change":"2026-02-24T22:15:22.816168+0000","last_active":"2026-02-24T22:31:14.739061+0000","last_peered":"2026-02-24T22:31:14.739061+0000","last_clean":"2026-02-24T22:31:14.739061+0000","last_became_active":"2026-02-24T22:15:22.815819+0000","last_became_peered":"2026-02-24T22:15:22.815819+0000","last_unstale":"2026-02-24T22:31:14.739061+0000","last_undegraded":"2026-02-24T22:31:14.739061+0000","last_fullsized":"2026-02-24T22:31:14.739061+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:39:39.503227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11534336,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":26624,"num_write":10,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.10","version":"312'31","reported_seq":152,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.731343+0000","last_change":"2026-02-24T22:13:43.760545+0000","last_active":"2026-02-24T22:30:04.731343+0000","last_peered":"2026-02-24T22:30:04.731343+0000","last_clean":"2026-02-24T22:30:04.731343+0000","last_became_active":"2026-02-24T22:13:43.759855+0000","last_became_peered":"2026-02-24T22:13:43.759855+0000","last_unstale":"2026-02-24T22:30:04.731343+0000","last_undegraded":"2026-02-24T22:30:04.731343+0000","last_fullsized":"2026-02-24T22:30:04.731343+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:57:17.855796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":272,"num_read_kb":204,"num_write":72,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.12","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.725516+0000","last_change":"2026-02-24T22:23:29.212745+0000","last_active":"2026-02-24T22:30:29.725516+0000","last_peered":"2026-02-24T22:30:29.725516+0000","last_clean":"2026-02-24T22:30:29.725516+0000","last_became_active":"2026-02-24T22:23:29.212534+0000","last_became_peered":"2026-02-24T22:23:29.212534+0000","last_unstale":"2026-02-24T22:30:29.725516+0000","last_undegraded":"2026-02-24T22:30:29.725516+0000","last_fullsized":"2026-02-24T22:30:29.725516+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:50:08.072181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.11","version":"307'3","reported_seq":44,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744735+0000","last_change":"2026-02-24T22:15:25.827242+0000","last_active":"2026-02-24T22:31:49.744735+0000","last_peered":"2026-02-24T22:31:49.744735+0000","last_clean":"2026-02-24T22:31:49.744735+0000","last_became_active":"2026-02-24T22:15:25.826517+0000","last_became_peered":"2026-02-24T22:15:25.826517+0000","last_unstale":"2026-02-24T22:31:49.744735+0000","last_undegraded":"2026-02-24T22:31:49.744735+0000","last_fullsized":"2026-02-24T22:31:49.744735+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:24:40.732721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":20480,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.10","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.741867+0000","last_change":"2026-02-24T22:15:22.810414+0000","last_active":"2026-02-24T22:31:44.741867+0000","last_peered":"2026-02-24T22:31:44.741867+0000","last_clean":"2026-02-24T22:31:44.741867+0000","last_became_active":"2026-02-24T22:15:22.810000+0000","last_became_peered":"2026-02-24T22:15:22.810000+0000","last_unstale":"2026-02-24T22:31:44.741867+0000","last_undegraded":"2026-02-24T22:31:44.741867+0000","last_fullsized":"2026-02-24T22:31:44.741867+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:21:42.629513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"79.f","version":"312'11","reported_seq":95,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742413+0000","last_change":"2026-02-24T22:13:43.755156+0000","last_active":"2026-02-24T22:31:49.742413+0000","last_peered":"2026-02-24T22:31:49.742413+0000","last_clean":"2026-02-24T22:31:49.742413+0000","last_became_active":"2026-02-24T22:13:43.754602+0000","last_became_peered":"2026-02-24T22:13:43.754602+0000","last_unstale":"2026-02-24T22:31:49.742413+0000","last_undegraded":"2026-02-24T22:31:49.742413+0000","last_fullsized":"2026-02-24T22:31:49.742413+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:13:53.006921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":67,"num_write":27,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.13","version":"310'1","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.735479+0000","last_change":"2026-02-24T22:23:29.198937+0000","last_active":"2026-02-24T22:30:29.735479+0000","last_peered":"2026-02-24T22:30:29.735479+0000","last_clean":"2026-02-24T22:30:29.735479+0000","last_became_active":"2026-02-24T22:23:29.198627+0000","last_became_peered":"2026-02-24T22:23:29.198627+0000","last_unstale":"2026-02-24T22:30:29.735479+0000","last_undegraded":"2026-02-24T22:30:29.735479+0000","last_fullsized":"2026-02-24T22:30:29.735479+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:28:29.061001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.10","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744722+0000","last_change":"2026-02-24T22:15:25.827252+0000","last_active":"2026-02-24T22:31:49.744722+0000","last_peered":"2026-02-24T22:31:49.744722+0000","last_clean":"2026-02-24T22:31:49.744722+0000","last_became_active":"2026-02-24T22:15:25.826606+0000","last_became_peered":"2026-02-24T22:15:25.826606+0000","last_unstale":"2026-02-24T22:31:49.744722+0000","last_undegraded":"2026-02-24T22:31:49.744722+0000","last_fullsized":"2026-02-24T22:31:49.744722+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:30:33.376941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.11","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.741885+0000","last_change":"2026-02-24T22:15:22.810459+0000","last_active":"2026-02-24T22:31:44.741885+0000","last_peered":"2026-02-24T22:31:44.741885+0000","last_clean":"2026-02-24T22:31:44.741885+0000","last_became_active":"2026-02-24T22:15:22.809791+0000","last_became_peered":"2026-02-24T22:15:22.809791+0000","last_unstale":"2026-02-24T22:31:44.741885+0000","last_undegraded":"2026-02-24T22:31:44.741885+0000","last_fullsized":"2026-02-24T22:31:44.741885+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:37:39.655379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"79.e","version":"312'23","reported_seq":126,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.741917+0000","last_change":"2026-02-24T22:13:43.756185+0000","last_active":"2026-02-24T22:31:44.741917+0000","last_peered":"2026-02-24T22:31:44.741917+0000","last_clean":"2026-02-24T22:31:44.741917+0000","last_became_active":"2026-02-24T22:13:43.755868+0000","last_became_peered":"2026-02-24T22:13:43.755868+0000","last_unstale":"2026-02-24T22:31:44.741917+0000","last_undegraded":"2026-02-24T22:31:44.741917+0000","last_fullsized":"2026-02-24T22:31:44.741917+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:56:04.041994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":169,"num_read_kb":112,"num_write":56,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.10","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.734148+0000","last_change":"2026-02-24T22:23:29.212895+0000","last_active":"2026-02-24T22:30:29.734148+0000","last_peered":"2026-02-24T22:30:29.734148+0000","last_clean":"2026-02-24T22:30:29.734148+0000","last_became_active":"2026-02-24T22:23:29.212665+0000","last_became_peered":"2026-02-24T22:23:29.212665+0000","last_unstale":"2026-02-24T22:30:29.734148+0000","last_undegraded":"2026-02-24T22:30:29.734148+0000","last_fullsized":"2026-02-24T22:30:29.734148+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:30:27.871463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.13","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.740862+0000","last_change":"2026-02-24T22:15:25.826171+0000","last_active":"2026-02-24T22:31:49.740862+0000","last_peered":"2026-02-24T22:31:49.740862+0000","last_clean":"2026-02-24T22:31:49.740862+0000","last_became_active":"2026-02-24T22:15:25.825862+0000","last_became_peered":"2026-02-24T22:15:25.825862+0000","last_unstale":"2026-02-24T22:31:49.740862+0000","last_undegraded":"2026-02-24T22:31:49.740862+0000","last_fullsized":"2026-02-24T22:31:49.740862+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:02:37.431216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"80.12","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.744628+0000","last_change":"2026-02-24T22:15:22.812712+0000","last_active":"2026-02-24T22:31:44.744628+0000","last_peered":"2026-02-24T22:31:44.744628+0000","last_clean":"2026-02-24T22:31:44.744628+0000","last_became_active":"2026-02-24T22:15:22.812405+0000","last_became_peered":"2026-02-24T22:15:22.812405+0000","last_unstale":"2026-02-24T22:31:44.744628+0000","last_undegraded":"2026-02-24T22:31:44.744628+0000","last_fullsized":"2026-02-24T22:31:44.744628+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:36:00.971366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.d","version":"312'20","reported_seq":113,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.733217+0000","last_change":"2026-02-24T22:13:43.761593+0000","last_active":"2026-02-24T22:31:44.733217+0000","last_peered":"2026-02-24T22:31:44.733217+0000","last_clean":"2026-02-24T22:31:44.733217+0000","last_became_active":"2026-02-24T22:13:43.761339+0000","last_became_peered":"2026-02-24T22:13:43.761339+0000","last_unstale":"2026-02-24T22:31:44.733217+0000","last_undegraded":"2026-02-24T22:31:44.733217+0000","last_fullsized":"2026-02-24T22:31:44.733217+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:02:40.622911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":126,"num_read_kb":76,"num_write":49,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"82.11","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.720712+0000","last_change":"2026-02-24T22:23:29.214793+0000","last_active":"2026-02-24T22:30:29.720712+0000","last_peered":"2026-02-24T22:30:29.720712+0000","last_clean":"2026-02-24T22:30:29.720712+0000","last_became_active":"2026-02-24T22:23:29.214167+0000","last_became_peered":"2026-02-24T22:23:29.214167+0000","last_unstale":"2026-02-24T22:30:29.720712+0000","last_undegraded":"2026-02-24T22:30:29.720712+0000","last_fullsized":"2026-02-24T22:30:29.720712+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:21:51.443877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.12","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744756+0000","last_change":"2026-02-24T22:15:25.826465+0000","last_active":"2026-02-24T22:31:49.744756+0000","last_peered":"2026-02-24T22:31:49.744756+0000","last_clean":"2026-02-24T22:31:49.744756+0000","last_became_active":"2026-02-24T22:15:25.826102+0000","last_became_peered":"2026-02-24T22:15:25.826102+0000","last_unstale":"2026-02-24T22:31:49.744756+0000","last_undegraded":"2026-02-24T22:31:49.744756+0000","last_fullsized":"2026-02-24T22:31:49.744756+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:22:01.000780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.13","version":"307'2","reported_seq":42,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.744644+0000","last_change":"2026-02-24T22:15:22.812393+0000","last_active":"2026-02-24T22:31:44.744644+0000","last_peered":"2026-02-24T22:31:44.744644+0000","last_clean":"2026-02-24T22:31:44.744644+0000","last_became_active":"2026-02-24T22:15:22.811927+0000","last_became_peered":"2026-02-24T22:15:22.811927+0000","last_unstale":"2026-02-24T22:31:44.744644+0000","last_undegraded":"2026-02-24T22:31:44.744644+0000","last_fullsized":"2026-02-24T22:31:44.744644+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:54:16.404989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.c","version":"312'4","reported_seq":74,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.724938+0000","last_change":"2026-02-24T22:13:43.762114+0000","last_active":"2026-02-24T22:31:44.724938+0000","last_peered":"2026-02-24T22:31:44.724938+0000","last_clean":"2026-02-24T22:31:44.724938+0000","last_became_active":"2026-02-24T22:13:43.761910+0000","last_became_peered":"2026-02-24T22:13:43.761910+0000","last_unstale":"2026-02-24T22:31:44.724938+0000","last_undegraded":"2026-02-24T22:31:44.724938+0000","last_fullsized":"2026-02-24T22:31:44.724938+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:51:11.166418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":15,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"82.16","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.718767+0000","last_change":"2026-02-24T22:23:29.213342+0000","last_active":"2026-02-24T22:30:44.718767+0000","last_peered":"2026-02-24T22:30:44.718767+0000","last_clean":"2026-02-24T22:30:44.718767+0000","last_became_active":"2026-02-24T22:23:29.213080+0000","last_became_peered":"2026-02-24T22:23:29.213080+0000","last_unstale":"2026-02-24T22:30:44.718767+0000","last_undegraded":"2026-02-24T22:30:44.718767+0000","last_fullsized":"2026-02-24T22:30:44.718767+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:33:29.031496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.15","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:59.714511+0000","last_change":"2026-02-24T22:15:25.831361+0000","last_active":"2026-02-24T22:29:59.714511+0000","last_peered":"2026-02-24T22:29:59.714511+0000","last_clean":"2026-02-24T22:29:59.714511+0000","last_became_active":"2026-02-24T22:15:25.830922+0000","last_became_peered":"2026-02-24T22:15:25.830922+0000","last_unstale":"2026-02-24T22:29:59.714511+0000","last_undegraded":"2026-02-24T22:29:59.714511+0000","last_fullsized":"2026-02-24T22:29:59.714511+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:08:07.538696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"80.14","version":"307'2","reported_seq":42,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.745149+0000","last_change":"2026-02-24T22:15:22.812446+0000","last_active":"2026-02-24T22:31:49.745149+0000","last_peered":"2026-02-24T22:31:49.745149+0000","last_clean":"2026-02-24T22:31:49.745149+0000","last_became_active":"2026-02-24T22:15:22.811924+0000","last_became_peered":"2026-02-24T22:15:22.811924+0000","last_unstale":"2026-02-24T22:31:49.745149+0000","last_undegraded":"2026-02-24T22:31:49.745149+0000","last_fullsized":"2026-02-24T22:31:49.745149+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:18:30.397097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.b","version":"312'13","reported_seq":98,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.740368+0000","last_change":"2026-02-24T22:13:43.757072+0000","last_active":"2026-02-24T22:31:44.740368+0000","last_peered":"2026-02-24T22:31:44.740368+0000","last_clean":"2026-02-24T22:31:44.740368+0000","last_became_active":"2026-02-24T22:13:43.756596+0000","last_became_peered":"2026-02-24T22:13:43.756596+0000","last_unstale":"2026-02-24T22:31:44.740368+0000","last_undegraded":"2026-02-24T22:31:44.740368+0000","last_fullsized":"2026-02-24T22:31:44.740368+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:30:51.803180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":56,"num_write":37,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.17","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.718784+0000","last_change":"2026-02-24T22:23:29.214456+0000","last_active":"2026-02-24T22:30:44.718784+0000","last_peered":"2026-02-24T22:30:44.718784+0000","last_clean":"2026-02-24T22:30:44.718784+0000","last_became_active":"2026-02-24T22:23:29.214217+0000","last_became_peered":"2026-02-24T22:23:29.214217+0000","last_unstale":"2026-02-24T22:30:44.718784+0000","last_undegraded":"2026-02-24T22:30:44.718784+0000","last_fullsized":"2026-02-24T22:30:44.718784+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:48:46.666871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.14","version":"307'3","reported_seq":40,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.745241+0000","last_change":"2026-02-24T22:15:25.827827+0000","last_active":"2026-02-24T22:31:54.745241+0000","last_peered":"2026-02-24T22:31:54.745241+0000","last_clean":"2026-02-24T22:31:54.745241+0000","last_became_active":"2026-02-24T22:15:25.827557+0000","last_became_peered":"2026-02-24T22:15:25.827557+0000","last_unstale":"2026-02-24T22:31:54.745241+0000","last_undegraded":"2026-02-24T22:31:54.745241+0000","last_fullsized":"2026-02-24T22:31:54.745241+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:13:31.380536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":4096,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.15","version":"307'3","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:54.732905+0000","last_change":"2026-02-24T22:15:22.812814+0000","last_active":"2026-02-24T22:29:54.732905+0000","last_peered":"2026-02-24T22:29:54.732905+0000","last_clean":"2026-02-24T22:29:54.732905+0000","last_became_active":"2026-02-24T22:15:22.812450+0000","last_became_peered":"2026-02-24T22:15:22.812450+0000","last_unstale":"2026-02-24T22:29:54.732905+0000","last_undegraded":"2026-02-24T22:29:54.732905+0000","last_fullsized":"2026-02-24T22:29:54.732905+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:22:29.140595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":4096,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.a","version":"312'10","reported_seq":90,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.724683+0000","last_change":"2026-02-24T22:13:43.762418+0000","last_active":"2026-02-24T22:31:09.724683+0000","last_peered":"2026-02-24T22:31:09.724683+0000","last_clean":"2026-02-24T22:31:09.724683+0000","last_became_active":"2026-02-24T22:13:43.762226+0000","last_became_peered":"2026-02-24T22:13:43.762226+0000","last_unstale":"2026-02-24T22:31:09.724683+0000","last_undegraded":"2026-02-24T22:31:09.724683+0000","last_fullsized":"2026-02-24T22:31:09.724683+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:41:14.108185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":42,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.14","version":"309'1","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.735499+0000","last_change":"2026-02-24T22:24:01.299030+0000","last_active":"2026-02-24T22:30:29.735499+0000","last_peered":"2026-02-24T22:30:29.735499+0000","last_clean":"2026-02-24T22:30:29.735499+0000","last_became_active":"2026-02-24T22:24:01.297193+0000","last_became_peered":"2026-02-24T22:24:01.297193+0000","last_unstale":"2026-02-24T22:30:29.735499+0000","last_undegraded":"2026-02-24T22:30:29.735499+0000","last_fullsized":"2026-02-24T22:30:29.735499+0000","mapping_epoch":311,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":312,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:15:53.769966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.17","version":"307'2","reported_seq":34,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:59.730863+0000","last_change":"2026-02-24T22:15:25.833669+0000","last_active":"2026-02-24T22:29:59.730863+0000","last_peered":"2026-02-24T22:29:59.730863+0000","last_clean":"2026-02-24T22:29:59.730863+0000","last_became_active":"2026-02-24T22:15:25.833430+0000","last_became_peered":"2026-02-24T22:15:25.833430+0000","last_unstale":"2026-02-24T22:29:59.730863+0000","last_undegraded":"2026-02-24T22:29:59.730863+0000","last_fullsized":"2026-02-24T22:29:59.730863+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:41:23.281878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1024,"num_write":7,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"80.16","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:54.714013+0000","last_change":"2026-02-24T22:15:22.817544+0000","last_active":"2026-02-24T22:29:54.714013+0000","last_peered":"2026-02-24T22:29:54.714013+0000","last_clean":"2026-02-24T22:29:54.714013+0000","last_became_active":"2026-02-24T22:15:22.817150+0000","last_became_peered":"2026-02-24T22:15:22.817150+0000","last_unstale":"2026-02-24T22:29:54.714013+0000","last_undegraded":"2026-02-24T22:29:54.714013+0000","last_fullsized":"2026-02-24T22:29:54.714013+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:30:27.018037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.9","version":"312'24","reported_seq":125,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.737699+0000","last_change":"2026-02-24T22:13:43.757041+0000","last_active":"2026-02-24T22:31:04.737699+0000","last_peered":"2026-02-24T22:31:04.737699+0000","last_clean":"2026-02-24T22:31:04.737699+0000","last_became_active":"2026-02-24T22:13:43.756777+0000","last_became_peered":"2026-02-24T22:13:43.756777+0000","last_unstale":"2026-02-24T22:31:04.737699+0000","last_undegraded":"2026-02-24T22:31:04.737699+0000","last_fullsized":"2026-02-24T22:31:04.737699+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:54:52.853203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":107,"num_write":60,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.15","version":"310'1","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.733776+0000","last_change":"2026-02-24T22:23:29.198331+0000","last_active":"2026-02-24T22:30:44.733776+0000","last_peered":"2026-02-24T22:30:44.733776+0000","last_clean":"2026-02-24T22:30:44.733776+0000","last_became_active":"2026-02-24T22:23:29.198000+0000","last_became_peered":"2026-02-24T22:23:29.198000+0000","last_unstale":"2026-02-24T22:30:44.733776+0000","last_undegraded":"2026-02-24T22:30:44.733776+0000","last_fullsized":"2026-02-24T22:30:44.733776+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:34:21.768724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.16","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:59.732575+0000","last_change":"2026-02-24T22:15:25.828675+0000","last_active":"2026-02-24T22:29:59.732575+0000","last_peered":"2026-02-24T22:29:59.732575+0000","last_clean":"2026-02-24T22:29:59.732575+0000","last_became_active":"2026-02-24T22:15:25.828066+0000","last_became_peered":"2026-02-24T22:15:25.828066+0000","last_unstale":"2026-02-24T22:29:59.732575+0000","last_undegraded":"2026-02-24T22:29:59.732575+0000","last_fullsized":"2026-02-24T22:29:59.732575+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:13:41.162339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.17","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:54.730332+0000","last_change":"2026-02-24T22:15:22.818517+0000","last_active":"2026-02-24T22:29:54.730332+0000","last_peered":"2026-02-24T22:29:54.730332+0000","last_clean":"2026-02-24T22:29:54.730332+0000","last_became_active":"2026-02-24T22:15:22.817950+0000","last_became_peered":"2026-02-24T22:15:22.817950+0000","last_unstale":"2026-02-24T22:29:54.730332+0000","last_undegraded":"2026-02-24T22:29:54.730332+0000","last_fullsized":"2026-02-24T22:29:54.730332+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:47:07.389545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.8","version":"312'26","reported_seq":136,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.737895+0000","last_change":"2026-02-24T22:13:43.761888+0000","last_active":"2026-02-24T22:31:04.737895+0000","last_peered":"2026-02-24T22:31:04.737895+0000","last_clean":"2026-02-24T22:31:04.737895+0000","last_became_active":"2026-02-24T22:13:43.761558+0000","last_became_peered":"2026-02-24T22:13:43.761558+0000","last_unstale":"2026-02-24T22:31:04.737895+0000","last_undegraded":"2026-02-24T22:31:04.737895+0000","last_fullsized":"2026-02-24T22:31:04.737895+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:22:52.093177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":152,"num_write":62,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.1a","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.723700+0000","last_change":"2026-02-24T22:23:29.213344+0000","last_active":"2026-02-24T22:30:59.723700+0000","last_peered":"2026-02-24T22:30:59.723700+0000","last_clean":"2026-02-24T22:30:59.723700+0000","last_became_active":"2026-02-24T22:23:29.212709+0000","last_became_peered":"2026-02-24T22:23:29.212709+0000","last_unstale":"2026-02-24T22:30:59.723700+0000","last_undegraded":"2026-02-24T22:30:59.723700+0000","last_fullsized":"2026-02-24T22:30:59.723700+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:26:30.998666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.19","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.718214+0000","last_change":"2026-02-24T22:15:25.832162+0000","last_active":"2026-02-24T22:30:04.718214+0000","last_peered":"2026-02-24T22:30:04.718214+0000","last_clean":"2026-02-24T22:30:04.718214+0000","last_became_active":"2026-02-24T22:15:25.831885+0000","last_became_peered":"2026-02-24T22:15:25.831885+0000","last_unstale":"2026-02-24T22:30:04.718214+0000","last_undegraded":"2026-02-24T22:30:04.718214+0000","last_fullsized":"2026-02-24T22:30:04.718214+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:54:09.173036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.18","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:54.730283+0000","last_change":"2026-02-24T22:15:22.810683+0000","last_active":"2026-02-24T22:29:54.730283+0000","last_peered":"2026-02-24T22:29:54.730283+0000","last_clean":"2026-02-24T22:29:54.730283+0000","last_became_active":"2026-02-24T22:15:22.810156+0000","last_became_peered":"2026-02-24T22:15:22.810156+0000","last_unstale":"2026-02-24T22:29:54.730283+0000","last_undegraded":"2026-02-24T22:29:54.730283+0000","last_fullsized":"2026-02-24T22:29:54.730283+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:29:33.293117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"79.7","version":"0'0","reported_seq":64,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.740460+0000","last_change":"2026-02-24T22:13:43.756250+0000","last_active":"2026-02-24T22:31:04.740460+0000","last_peered":"2026-02-24T22:31:04.740460+0000","last_clean":"2026-02-24T22:31:04.740460+0000","last_became_active":"2026-02-24T22:13:43.755957+0000","last_became_peered":"2026-02-24T22:13:43.755957+0000","last_unstale":"2026-02-24T22:31:04.740460+0000","last_undegraded":"2026-02-24T22:31:04.740460+0000","last_fullsized":"2026-02-24T22:31:04.740460+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:16:21.505648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.1b","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.735465+0000","last_change":"2026-02-24T22:23:29.200410+0000","last_active":"2026-02-24T22:30:59.735465+0000","last_peered":"2026-02-24T22:30:59.735465+0000","last_clean":"2026-02-24T22:30:59.735465+0000","last_became_active":"2026-02-24T22:23:29.200152+0000","last_became_peered":"2026-02-24T22:23:29.200152+0000","last_unstale":"2026-02-24T22:30:59.735465+0000","last_undegraded":"2026-02-24T22:30:59.735465+0000","last_fullsized":"2026-02-24T22:30:59.735465+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:24:51.029708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.18","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:59.730819+0000","last_change":"2026-02-24T22:15:25.824057+0000","last_active":"2026-02-24T22:29:59.730819+0000","last_peered":"2026-02-24T22:29:59.730819+0000","last_clean":"2026-02-24T22:29:59.730819+0000","last_became_active":"2026-02-24T22:15:25.823415+0000","last_became_peered":"2026-02-24T22:15:25.823415+0000","last_unstale":"2026-02-24T22:29:59.730819+0000","last_undegraded":"2026-02-24T22:29:59.730819+0000","last_fullsized":"2026-02-24T22:29:59.730819+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:00:21.357856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.19","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:59.714470+0000","last_change":"2026-02-24T22:15:22.817214+0000","last_active":"2026-02-24T22:29:59.714470+0000","last_peered":"2026-02-24T22:29:59.714470+0000","last_clean":"2026-02-24T22:29:59.714470+0000","last_became_active":"2026-02-24T22:15:22.816826+0000","last_became_peered":"2026-02-24T22:15:22.816826+0000","last_unstale":"2026-02-24T22:29:59.714470+0000","last_undegraded":"2026-02-24T22:29:59.714470+0000","last_fullsized":"2026-02-24T22:29:59.714470+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:13:51.793483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.6","version":"301'14","reported_seq":108,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.739656+0000","last_change":"2026-02-24T22:13:43.755144+0000","last_active":"2026-02-24T22:31:04.739656+0000","last_peered":"2026-02-24T22:31:04.739656+0000","last_clean":"2026-02-24T22:31:04.739656+0000","last_became_active":"2026-02-24T22:13:43.754753+0000","last_became_peered":"2026-02-24T22:13:43.754753+0000","last_unstale":"2026-02-24T22:31:04.739656+0000","last_undegraded":"2026-02-24T22:31:04.739656+0000","last_fullsized":"2026-02-24T22:31:04.739656+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:57:00.870335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":92,"num_write":34,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"82.18","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.726953+0000","last_change":"2026-02-24T22:23:29.213704+0000","last_active":"2026-02-24T22:30:44.726953+0000","last_peered":"2026-02-24T22:30:44.726953+0000","last_clean":"2026-02-24T22:30:44.726953+0000","last_became_active":"2026-02-24T22:23:29.213387+0000","last_became_peered":"2026-02-24T22:23:29.213387+0000","last_unstale":"2026-02-24T22:30:44.726953+0000","last_undegraded":"2026-02-24T22:30:44.726953+0000","last_fullsized":"2026-02-24T22:30:44.726953+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:46:11.186061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.1b","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.720738+0000","last_change":"2026-02-24T22:15:25.832293+0000","last_active":"2026-02-24T22:30:29.720738+0000","last_peered":"2026-02-24T22:30:29.720738+0000","last_clean":"2026-02-24T22:30:29.720738+0000","last_became_active":"2026-02-24T22:15:25.832097+0000","last_became_peered":"2026-02-24T22:15:25.832097+0000","last_unstale":"2026-02-24T22:30:29.720738+0000","last_undegraded":"2026-02-24T22:30:29.720738+0000","last_fullsized":"2026-02-24T22:30:29.720738+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:19:51.400688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.1a","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.720229+0000","last_change":"2026-02-24T22:15:22.819032+0000","last_active":"2026-02-24T22:30:24.720229+0000","last_peered":"2026-02-24T22:30:24.720229+0000","last_clean":"2026-02-24T22:30:24.720229+0000","last_became_active":"2026-02-24T22:15:22.818885+0000","last_became_peered":"2026-02-24T22:15:22.818885+0000","last_unstale":"2026-02-24T22:30:24.720229+0000","last_undegraded":"2026-02-24T22:30:24.720229+0000","last_fullsized":"2026-02-24T22:30:24.720229+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:15:52.421379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"79.5","version":"312'19","reported_seq":121,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:49.005257+0000","last_change":"2026-02-24T22:13:43.756043+0000","last_active":"2026-02-24T22:30:49.005257+0000","last_peered":"2026-02-24T22:30:49.005257+0000","last_clean":"2026-02-24T22:30:49.005257+0000","last_became_active":"2026-02-24T22:13:43.755495+0000","last_became_peered":"2026-02-24T22:13:43.755495+0000","last_unstale":"2026-02-24T22:30:49.005257+0000","last_undegraded":"2026-02-24T22:30:49.005257+0000","last_fullsized":"2026-02-24T22:30:49.005257+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:47:41.399446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":164,"num_read_kb":112,"num_write":47,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.19","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:49.719308+0000","last_change":"2026-02-24T22:23:29.212738+0000","last_active":"2026-02-24T22:30:49.719308+0000","last_peered":"2026-02-24T22:30:49.719308+0000","last_clean":"2026-02-24T22:30:49.719308+0000","last_became_active":"2026-02-24T22:23:29.212268+0000","last_became_peered":"2026-02-24T22:23:29.212268+0000","last_unstale":"2026-02-24T22:30:49.719308+0000","last_undegraded":"2026-02-24T22:30:49.719308+0000","last_fullsized":"2026-02-24T22:30:49.719308+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:32:25.776768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.1a","version":"307'2","reported_seq":35,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.731903+0000","last_change":"2026-02-24T22:15:25.826222+0000","last_active":"2026-02-24T22:30:29.731903+0000","last_peered":"2026-02-24T22:30:29.731903+0000","last_clean":"2026-02-24T22:30:29.731903+0000","last_became_active":"2026-02-24T22:15:25.825969+0000","last_became_peered":"2026-02-24T22:15:25.825969+0000","last_unstale":"2026-02-24T22:30:29.731903+0000","last_undegraded":"2026-02-24T22:30:29.731903+0000","last_fullsized":"2026-02-24T22:30:29.731903+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:29:05.133831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":7,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"80.1b","version":"307'2","reported_seq":41,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.733578+0000","last_change":"2026-02-24T22:15:22.816734+0000","last_active":"2026-02-24T22:30:24.733578+0000","last_peered":"2026-02-24T22:30:24.733578+0000","last_clean":"2026-02-24T22:30:24.733578+0000","last_became_active":"2026-02-24T22:15:22.816479+0000","last_became_peered":"2026-02-24T22:15:22.816479+0000","last_unstale":"2026-02-24T22:30:24.733578+0000","last_undegraded":"2026-02-24T22:30:24.733578+0000","last_fullsized":"2026-02-24T22:30:24.733578+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:57:39.996802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.4","version":"0'0","reported_seq":70,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.735005+0000","last_change":"2026-02-24T22:14:00.742004+0000","last_active":"2026-02-24T22:30:24.735005+0000","last_peered":"2026-02-24T22:30:24.735005+0000","last_clean":"2026-02-24T22:30:24.735005+0000","last_became_active":"2026-02-24T22:14:00.741839+0000","last_became_peered":"2026-02-24T22:14:00.741839+0000","last_unstale":"2026-02-24T22:30:24.735005+0000","last_undegraded":"2026-02-24T22:30:24.735005+0000","last_fullsized":"2026-02-24T22:30:24.735005+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":293,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:38:11.152954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"82.1e","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.727290+0000","last_change":"2026-02-24T22:23:29.215779+0000","last_active":"2026-02-24T22:31:34.727290+0000","last_peered":"2026-02-24T22:31:34.727290+0000","last_clean":"2026-02-24T22:31:34.727290+0000","last_became_active":"2026-02-24T22:23:29.215604+0000","last_became_peered":"2026-02-24T22:23:29.215604+0000","last_unstale":"2026-02-24T22:31:34.727290+0000","last_undegraded":"2026-02-24T22:31:34.727290+0000","last_fullsized":"2026-02-24T22:31:34.727290+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:58:56.390417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.1d","version":"307'2","reported_seq":35,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.733670+0000","last_change":"2026-02-24T22:15:25.828517+0000","last_active":"2026-02-24T22:30:29.733670+0000","last_peered":"2026-02-24T22:30:29.733670+0000","last_clean":"2026-02-24T22:30:29.733670+0000","last_became_active":"2026-02-24T22:15:25.828335+0000","last_became_peered":"2026-02-24T22:15:25.828335+0000","last_unstale":"2026-02-24T22:30:29.733670+0000","last_undegraded":"2026-02-24T22:30:29.733670+0000","last_fullsized":"2026-02-24T22:30:29.733670+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:18:15.606086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.1c","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.733619+0000","last_change":"2026-02-24T22:15:22.816853+0000","last_active":"2026-02-24T22:30:24.733619+0000","last_peered":"2026-02-24T22:30:24.733619+0000","last_clean":"2026-02-24T22:30:24.733619+0000","last_became_active":"2026-02-24T22:15:22.816548+0000","last_became_peered":"2026-02-24T22:15:22.816548+0000","last_unstale":"2026-02-24T22:30:24.733619+0000","last_undegraded":"2026-02-24T22:30:24.733619+0000","last_fullsized":"2026-02-24T22:30:24.733619+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:00:55.883988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.3","version":"312'6","reported_seq":77,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.731442+0000","last_change":"2026-02-24T22:13:43.756810+0000","last_active":"2026-02-24T22:30:24.731442+0000","last_peered":"2026-02-24T22:30:24.731442+0000","last_clean":"2026-02-24T22:30:24.731442+0000","last_became_active":"2026-02-24T22:13:43.756408+0000","last_became_peered":"2026-02-24T22:13:43.756408+0000","last_unstale":"2026-02-24T22:30:24.731442+0000","last_undegraded":"2026-02-24T22:30:24.731442+0000","last_fullsized":"2026-02-24T22:30:24.731442+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:24:33.412598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":20,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.1f","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744790+0000","last_change":"2026-02-24T22:23:29.199410+0000","last_active":"2026-02-24T22:31:49.744790+0000","last_peered":"2026-02-24T22:31:49.744790+0000","last_clean":"2026-02-24T22:31:49.744790+0000","last_became_active":"2026-02-24T22:23:29.199179+0000","last_became_peered":"2026-02-24T22:23:29.199179+0000","last_unstale":"2026-02-24T22:31:49.744790+0000","last_undegraded":"2026-02-24T22:31:49.744790+0000","last_fullsized":"2026-02-24T22:31:49.744790+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:21:23.823724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.1c","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.725548+0000","last_change":"2026-02-24T22:15:25.834316+0000","last_active":"2026-02-24T22:30:29.725548+0000","last_peered":"2026-02-24T22:30:29.725548+0000","last_clean":"2026-02-24T22:30:29.725548+0000","last_became_active":"2026-02-24T22:15:25.834072+0000","last_became_peered":"2026-02-24T22:15:25.834072+0000","last_unstale":"2026-02-24T22:30:29.725548+0000","last_undegraded":"2026-02-24T22:30:29.725548+0000","last_fullsized":"2026-02-24T22:30:29.725548+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:45:39.925744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"80.1d","version":"304'1","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.733639+0000","last_change":"2026-02-24T22:15:22.817173+0000","last_active":"2026-02-24T22:30:24.733639+0000","last_peered":"2026-02-24T22:30:24.733639+0000","last_clean":"2026-02-24T22:30:24.733639+0000","last_became_active":"2026-02-24T22:15:22.816615+0000","last_became_peered":"2026-02-24T22:15:22.816615+0000","last_unstale":"2026-02-24T22:30:24.733639+0000","last_undegraded":"2026-02-24T22:30:24.733639+0000","last_fullsized":"2026-02-24T22:30:24.733639+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:25:34.323299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":4096,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.2","version":"312'5","reported_seq":77,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.733168+0000","last_change":"2026-02-24T22:13:43.755888+0000","last_active":"2026-02-24T22:30:24.733168+0000","last_peered":"2026-02-24T22:30:24.733168+0000","last_clean":"2026-02-24T22:30:24.733168+0000","last_became_active":"2026-02-24T22:13:43.755241+0000","last_became_peered":"2026-02-24T22:13:43.755241+0000","last_unstale":"2026-02-24T22:30:24.733168+0000","last_undegraded":"2026-02-24T22:30:24.733168+0000","last_fullsized":"2026-02-24T22:30:24.733168+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:13:28.809470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":23,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.1c","version":"310'1","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.720361+0000","last_change":"2026-02-24T22:23:29.213715+0000","last_active":"2026-02-24T22:30:59.720361+0000","last_peered":"2026-02-24T22:30:59.720361+0000","last_clean":"2026-02-24T22:30:59.720361+0000","last_became_active":"2026-02-24T22:23:29.213490+0000","last_became_peered":"2026-02-24T22:23:29.213490+0000","last_unstale":"2026-02-24T22:30:59.720361+0000","last_undegraded":"2026-02-24T22:30:59.720361+0000","last_fullsized":"2026-02-24T22:30:59.720361+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:21:40.032070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.1f","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.747978+0000","last_change":"2026-02-24T22:15:25.828181+0000","last_active":"2026-02-24T22:32:09.747978+0000","last_peered":"2026-02-24T22:32:09.747978+0000","last_clean":"2026-02-24T22:32:09.747978+0000","last_became_active":"2026-02-24T22:15:25.827873+0000","last_became_peered":"2026-02-24T22:15:25.827873+0000","last_unstale":"2026-02-24T22:32:09.747978+0000","last_undegraded":"2026-02-24T22:32:09.747978+0000","last_fullsized":"2026-02-24T22:32:09.747978+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:19:57.290828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.1e","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:29.726743+0000","last_change":"2026-02-24T22:15:22.817535+0000","last_active":"2026-02-24T22:31:29.726743+0000","last_peered":"2026-02-24T22:31:29.726743+0000","last_clean":"2026-02-24T22:31:29.726743+0000","last_became_active":"2026-02-24T22:15:22.817337+0000","last_became_peered":"2026-02-24T22:15:22.817337+0000","last_unstale":"2026-02-24T22:31:29.726743+0000","last_undegraded":"2026-02-24T22:31:29.726743+0000","last_fullsized":"2026-02-24T22:31:29.726743+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:02:48.141978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"79.1","version":"312'23","reported_seq":125,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.720202+0000","last_change":"2026-02-24T22:13:43.761201+0000","last_active":"2026-02-24T22:30:24.720202+0000","last_peered":"2026-02-24T22:30:24.720202+0000","last_clean":"2026-02-24T22:30:24.720202+0000","last_became_active":"2026-02-24T22:13:43.760933+0000","last_became_peered":"2026-02-24T22:13:43.760933+0000","last_unstale":"2026-02-24T22:30:24.720202+0000","last_undegraded":"2026-02-24T22:30:24.720202+0000","last_fullsized":"2026-02-24T22:30:24.720202+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:18:32.108308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":174,"num_read_kb":113,"num_write":59,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.1d","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.723721+0000","last_change":"2026-02-24T22:23:29.215553+0000","last_active":"2026-02-24T22:30:59.723721+0000","last_peered":"2026-02-24T22:30:59.723721+0000","last_clean":"2026-02-24T22:30:59.723721+0000","last_became_active":"2026-02-24T22:23:29.215334+0000","last_became_peered":"2026-02-24T22:23:29.215334+0000","last_unstale":"2026-02-24T22:30:59.723721+0000","last_undegraded":"2026-02-24T22:30:59.723721+0000","last_fullsized":"2026-02-24T22:30:59.723721+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:01:42.978764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.1e","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.739370+0000","last_change":"2026-02-24T22:15:25.827409+0000","last_active":"2026-02-24T22:31:34.739370+0000","last_peered":"2026-02-24T22:31:34.739370+0000","last_clean":"2026-02-24T22:31:34.739370+0000","last_became_active":"2026-02-24T22:15:25.827124+0000","last_became_peered":"2026-02-24T22:15:25.827124+0000","last_unstale":"2026-02-24T22:31:34.739370+0000","last_undegraded":"2026-02-24T22:31:34.739370+0000","last_fullsized":"2026-02-24T22:31:34.739370+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:44:21.789639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"80.1f","version":"304'2","reported_seq":43,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:04.726966+0000","last_change":"2026-02-24T22:15:22.816227+0000","last_active":"2026-02-24T22:32:04.726966+0000","last_peered":"2026-02-24T22:32:04.726966+0000","last_clean":"2026-02-24T22:32:04.726966+0000","last_became_active":"2026-02-24T22:15:22.815938+0000","last_became_peered":"2026-02-24T22:15:22.815938+0000","last_unstale":"2026-02-24T22:32:04.726966+0000","last_undegraded":"2026-02-24T22:32:04.726966+0000","last_fullsized":"2026-02-24T22:32:04.726966+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:40:15.210510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":2,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.0","version":"0'0","reported_seq":65,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.745118+0000","last_change":"2026-02-24T22:13:43.755907+0000","last_active":"2026-02-24T22:31:49.745118+0000","last_peered":"2026-02-24T22:31:49.745118+0000","last_clean":"2026-02-24T22:31:49.745118+0000","last_became_active":"2026-02-24T22:13:43.755266+0000","last_became_peered":"2026-02-24T22:13:43.755266+0000","last_unstale":"2026-02-24T22:31:49.745118+0000","last_undegraded":"2026-02-24T22:31:49.745118+0000","last_fullsized":"2026-02-24T22:31:49.745118+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:42:48.314773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"312'215","reported_seq":920,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:47.193151+0000","last_change":"2026-02-24T22:05:16.193051+0000","last_active":"2026-02-24T22:33:47.193151+0000","last_peered":"2026-02-24T22:33:47.193151+0000","last_clean":"2026-02-24T22:33:47.193151+0000","last_became_active":"2026-02-24T22:05:16.192589+0000","last_became_peered":"2026-02-24T22:05:16.192589+0000","last_unstale":"2026-02-24T22:33:47.193151+0000","last_undegraded":"2026-02-24T22:33:47.193151+0000","last_fullsized":"2026-02-24T22:33:47.193151+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:20:06.945662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8436,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":377,"num_read_kb":342,"num_write":1213,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"312'197","reported_seq":898,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:46.730087+0000","last_change":"2026-02-24T22:05:18.192403+0000","last_active":"2026-02-24T22:33:46.730087+0000","last_peered":"2026-02-24T22:33:46.730087+0000","last_clean":"2026-02-24T22:33:46.730087+0000","last_became_active":"2026-02-24T22:05:18.192203+0000","last_became_peered":"2026-02-24T22:05:18.192203+0000","last_unstale":"2026-02-24T22:33:46.730087+0000","last_undegraded":"2026-02-24T22:33:46.730087+0000","last_fullsized":"2026-02-24T22:33:46.730087+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":100,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:33:32.754536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":780,"num_read_kb":643,"num_write":282,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"312'304","reported_seq":1112,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:51.011231+0000","last_change":"2026-02-24T22:05:16.181637+0000","last_active":"2026-02-24T22:30:51.011231+0000","last_peered":"2026-02-24T22:30:51.011231+0000","last_clean":"2026-02-24T22:30:51.011231+0000","last_became_active":"2026-02-24T22:05:16.181379+0000","last_became_peered":"2026-02-24T22:05:16.181379+0000","last_unstale":"2026-02-24T22:30:51.011231+0000","last_undegraded":"2026-02-24T22:30:51.011231+0000","last_fullsized":"2026-02-24T22:30:51.011231+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":200,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:35:21.970288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21628,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":848,"num_read_kb":611,"num_write":1466,"num_write_kb":5240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"312'459","reported_seq":1257,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:57.922033+0000","last_change":"2026-02-24T22:05:18.190910+0000","last_active":"2026-02-24T22:30:57.922033+0000","last_peered":"2026-02-24T22:30:57.922033+0000","last_clean":"2026-02-24T22:30:57.922033+0000","last_became_active":"2026-02-24T22:05:18.190689+0000","last_became_peered":"2026-02-24T22:05:18.190689+0000","last_unstale":"2026-02-24T22:30:57.922033+0000","last_undegraded":"2026-02-24T22:30:57.922033+0000","last_fullsized":"2026-02-24T22:30:57.922033+0000","mapping_epoch":24,"log_start":"312'400","ondisk_log_start":"312'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":400,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:35:35.142199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":2017,"num_read_kb":1670,"num_write":788,"num_write_kb":502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"312'368","reported_seq":1154,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:58.398455+0000","last_change":"2026-02-24T22:05:16.180825+0000","last_active":"2026-02-24T22:30:58.398455+0000","last_peered":"2026-02-24T22:30:58.398455+0000","last_clean":"2026-02-24T22:30:58.398455+0000","last_became_active":"2026-02-24T22:05:16.180681+0000","last_became_peered":"2026-02-24T22:05:16.180681+0000","last_unstale":"2026-02-24T22:30:58.398455+0000","last_undegraded":"2026-02-24T22:30:58.398455+0000","last_fullsized":"2026-02-24T22:30:58.398455+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":300,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:38:22.293881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5517838,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":825,"num_read_kb":6166,"num_write":1529,"num_write_kb":5459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"312'80","reported_seq":727,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.732683+0000","last_change":"2026-02-24T22:05:18.192661+0000","last_active":"2026-02-24T22:30:19.732683+0000","last_peered":"2026-02-24T22:30:19.732683+0000","last_clean":"2026-02-24T22:30:19.732683+0000","last_became_active":"2026-02-24T22:05:18.190727+0000","last_became_peered":"2026-02-24T22:05:18.190727+0000","last_unstale":"2026-02-24T22:30:19.732683+0000","last_undegraded":"2026-02-24T22:30:19.732683+0000","last_fullsized":"2026-02-24T22:30:19.732683+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:10:39.624197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":268,"num_read_kb":220,"num_write":112,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"312'218","reported_seq":902,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:01.970038+0000","last_change":"2026-02-24T22:05:16.193170+0000","last_active":"2026-02-24T22:34:01.970038+0000","last_peered":"2026-02-24T22:34:01.970038+0000","last_clean":"2026-02-24T22:34:01.970038+0000","last_became_active":"2026-02-24T22:05:16.192922+0000","last_became_peered":"2026-02-24T22:05:16.192922+0000","last_unstale":"2026-02-24T22:34:01.970038+0000","last_undegraded":"2026-02-24T22:34:01.970038+0000","last_fullsized":"2026-02-24T22:34:01.970038+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":200,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:51:56.894641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":549324,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":307,"num_read_kb":3190,"num_write":1131,"num_write_kb":12582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"312'135","reported_seq":815,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.733034+0000","last_change":"2026-02-24T22:05:18.184446+0000","last_active":"2026-02-24T22:30:19.733034+0000","last_peered":"2026-02-24T22:30:19.733034+0000","last_clean":"2026-02-24T22:30:19.733034+0000","last_became_active":"2026-02-24T22:05:18.184014+0000","last_became_peered":"2026-02-24T22:05:18.184014+0000","last_unstale":"2026-02-24T22:30:19.733034+0000","last_undegraded":"2026-02-24T22:30:19.733034+0000","last_fullsized":"2026-02-24T22:30:19.733034+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:12:45.735332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":489,"num_read_kb":401,"num_write":208,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"312'208","reported_seq":919,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:12.258587+0000","last_change":"2026-02-24T22:05:16.191359+0000","last_active":"2026-02-24T22:34:12.258587+0000","last_peered":"2026-02-24T22:34:12.258587+0000","last_clean":"2026-02-24T22:34:12.258587+0000","last_became_active":"2026-02-24T22:05:16.191055+0000","last_became_peered":"2026-02-24T22:05:16.191055+0000","last_unstale":"2026-02-24T22:34:12.258587+0000","last_undegraded":"2026-02-24T22:34:12.258587+0000","last_fullsized":"2026-02-24T22:34:12.258587+0000","mapping_epoch":22,"log_start":"312'100","ondisk_log_start":"312'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":100,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:40:09.331901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6857028,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":360,"num_read_kb":15420,"num_write":1138,"num_write_kb":8454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"312'470","reported_seq":1186,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.738353+0000","last_change":"2026-02-24T22:05:18.190983+0000","last_active":"2026-02-24T22:31:24.738353+0000","last_peered":"2026-02-24T22:31:24.738353+0000","last_clean":"2026-02-24T22:31:24.738353+0000","last_became_active":"2026-02-24T22:05:18.190598+0000","last_became_peered":"2026-02-24T22:05:18.190598+0000","last_unstale":"2026-02-24T22:31:24.738353+0000","last_undegraded":"2026-02-24T22:31:24.738353+0000","last_fullsized":"2026-02-24T22:31:24.738353+0000","mapping_epoch":24,"log_start":"312'400","ondisk_log_start":"312'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":400,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:52:56.661245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1905,"num_read_kb":1522,"num_write":772,"num_write_kb":389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"312'233","reported_seq":943,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:50.432924+0000","last_change":"2026-02-24T22:05:16.191753+0000","last_active":"2026-02-24T22:30:50.432924+0000","last_peered":"2026-02-24T22:30:50.432924+0000","last_clean":"2026-02-24T22:30:50.432924+0000","last_became_active":"2026-02-24T22:05:16.191164+0000","last_became_peered":"2026-02-24T22:05:16.191164+0000","last_unstale":"2026-02-24T22:30:50.432924+0000","last_undegraded":"2026-02-24T22:30:50.432924+0000","last_fullsized":"2026-02-24T22:30:50.432924+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":200,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:23:55.135404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5295803,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":424,"num_read_kb":5551,"num_write":1270,"num_write_kb":5240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"312'56","reported_seq":709,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.732718+0000","last_change":"2026-02-24T22:05:18.198708+0000","last_active":"2026-02-24T22:30:19.732718+0000","last_peered":"2026-02-24T22:30:19.732718+0000","last_clean":"2026-02-24T22:30:19.732718+0000","last_became_active":"2026-02-24T22:05:18.198559+0000","last_became_peered":"2026-02-24T22:05:18.198559+0000","last_unstale":"2026-02-24T22:30:19.732718+0000","last_undegraded":"2026-02-24T22:30:19.732718+0000","last_fullsized":"2026-02-24T22:30:19.732718+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:01:34.891706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":195,"num_read_kb":167,"num_write":70,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"312'219","reported_seq":919,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:57.717745+0000","last_change":"2026-02-24T22:05:16.190594+0000","last_active":"2026-02-24T22:30:57.717745+0000","last_peered":"2026-02-24T22:30:57.717745+0000","last_clean":"2026-02-24T22:30:57.717745+0000","last_became_active":"2026-02-24T22:05:16.190367+0000","last_became_peered":"2026-02-24T22:05:16.190367+0000","last_unstale":"2026-02-24T22:30:57.717745+0000","last_undegraded":"2026-02-24T22:30:57.717745+0000","last_fullsized":"2026-02-24T22:30:57.717745+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":200,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:34:28.350502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1051952,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":361,"num_read_kb":1335,"num_write":1155,"num_write_kb":1104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"312'201","reported_seq":941,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:49.857860+0000","last_change":"2026-02-24T22:05:18.192094+0000","last_active":"2026-02-24T22:33:49.857860+0000","last_peered":"2026-02-24T22:33:49.857860+0000","last_clean":"2026-02-24T22:33:49.857860+0000","last_became_active":"2026-02-24T22:05:18.191767+0000","last_became_peered":"2026-02-24T22:05:18.191767+0000","last_unstale":"2026-02-24T22:33:49.857860+0000","last_undegraded":"2026-02-24T22:33:49.857860+0000","last_fullsized":"2026-02-24T22:33:49.857860+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":100,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:15:25.401981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":876,"num_read_kb":754,"num_write":323,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"312'272","reported_seq":1042,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:40.186391+0000","last_change":"2026-02-24T22:05:16.186732+0000","last_active":"2026-02-24T22:33:40.186391+0000","last_peered":"2026-02-24T22:33:40.186391+0000","last_clean":"2026-02-24T22:33:40.186391+0000","last_became_active":"2026-02-24T22:05:16.185779+0000","last_became_peered":"2026-02-24T22:05:16.185779+0000","last_unstale":"2026-02-24T22:33:40.186391+0000","last_undegraded":"2026-02-24T22:33:40.186391+0000","last_fullsized":"2026-02-24T22:33:40.186391+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":200,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:01:24.063571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50495,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":533,"num_read_kb":877,"num_write":1531,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"312'83","reported_seq":729,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.716462+0000","last_change":"2026-02-24T22:05:18.188067+0000","last_active":"2026-02-24T22:30:19.716462+0000","last_peered":"2026-02-24T22:30:19.716462+0000","last_clean":"2026-02-24T22:30:19.716462+0000","last_became_active":"2026-02-24T22:05:18.187838+0000","last_became_peered":"2026-02-24T22:05:18.187838+0000","last_unstale":"2026-02-24T22:30:19.716462+0000","last_undegraded":"2026-02-24T22:30:19.716462+0000","last_fullsized":"2026-02-24T22:30:19.716462+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:05:06.602774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":263,"num_read_kb":217,"num_write":106,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"312'135","reported_seq":808,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.736992+0000","last_change":"2026-02-24T22:05:18.192713+0000","last_active":"2026-02-24T22:30:59.736992+0000","last_peered":"2026-02-24T22:30:59.736992+0000","last_clean":"2026-02-24T22:30:59.736992+0000","last_became_active":"2026-02-24T22:05:18.191625+0000","last_became_peered":"2026-02-24T22:05:18.191625+0000","last_unstale":"2026-02-24T22:30:59.736992+0000","last_undegraded":"2026-02-24T22:30:59.736992+0000","last_fullsized":"2026-02-24T22:30:59.736992+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:24:04.071735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":477,"num_read_kb":396,"num_write":183,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"312'237","reported_seq":974,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:53.497360+0000","last_change":"2026-02-24T22:06:01.025897+0000","last_active":"2026-02-24T22:33:53.497360+0000","last_peered":"2026-02-24T22:33:53.497360+0000","last_clean":"2026-02-24T22:33:53.497360+0000","last_became_active":"2026-02-24T22:06:01.025659+0000","last_became_peered":"2026-02-24T22:06:01.025659+0000","last_unstale":"2026-02-24T22:33:53.497360+0000","last_undegraded":"2026-02-24T22:33:53.497360+0000","last_fullsized":"2026-02-24T22:33:53.497360+0000","mapping_epoch":36,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":200,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:15:11.362680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":623937,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":456,"num_read_kb":16219,"num_write":1259,"num_write_kb":8500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"312'4249","reported_seq":5059,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.162950+0000","last_change":"2026-02-24T22:05:18.193454+0000","last_active":"2026-02-24T22:34:15.162950+0000","last_peered":"2026-02-24T22:34:15.162950+0000","last_clean":"2026-02-24T22:34:15.162950+0000","last_became_active":"2026-02-24T22:05:18.193257+0000","last_became_peered":"2026-02-24T22:05:18.193257+0000","last_unstale":"2026-02-24T22:34:15.162950+0000","last_undegraded":"2026-02-24T22:34:15.162950+0000","last_fullsized":"2026-02-24T22:34:15.162950+0000","mapping_epoch":24,"log_start":"312'4200","ondisk_log_start":"312'4200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":3000,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:25:08.863245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":17332,"num_read_kb":14553,"num_write":6398,"num_write_kb":3198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"312'213","reported_seq":925,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.741319+0000","last_change":"2026-02-24T22:05:16.191122+0000","last_active":"2026-02-24T22:31:54.741319+0000","last_peered":"2026-02-24T22:31:54.741319+0000","last_clean":"2026-02-24T22:31:54.741319+0000","last_became_active":"2026-02-24T22:05:16.190819+0000","last_became_peered":"2026-02-24T22:05:16.190819+0000","last_unstale":"2026-02-24T22:31:54.741319+0000","last_undegraded":"2026-02-24T22:31:54.741319+0000","last_fullsized":"2026-02-24T22:31:54.741319+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":200,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:31:00.005386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5568323,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":396,"num_read_kb":8113,"num_write":1091,"num_write_kb":5510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"312'862","reported_seq":4024,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.230995+0000","last_change":"2026-02-24T22:05:18.190719+0000","last_active":"2026-02-24T22:34:17.230995+0000","last_peered":"2026-02-24T22:34:17.230995+0000","last_clean":"2026-02-24T22:34:17.230995+0000","last_became_active":"2026-02-24T22:05:18.190363+0000","last_became_peered":"2026-02-24T22:05:18.190363+0000","last_unstale":"2026-02-24T22:34:17.230995+0000","last_undegraded":"2026-02-24T22:34:17.230995+0000","last_fullsized":"2026-02-24T22:34:17.230995+0000","mapping_epoch":24,"log_start":"312'800","ondisk_log_start":"312'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":800,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:46:25.367911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4747,"num_read_kb":5785,"num_write":398,"num_write_kb":236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"312'229","reported_seq":960,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:03.733775+0000","last_change":"2026-02-24T22:05:16.191246+0000","last_active":"2026-02-24T22:34:03.733775+0000","last_peered":"2026-02-24T22:34:03.733775+0000","last_clean":"2026-02-24T22:34:03.733775+0000","last_became_active":"2026-02-24T22:05:16.190816+0000","last_became_peered":"2026-02-24T22:05:16.190816+0000","last_unstale":"2026-02-24T22:34:03.733775+0000","last_undegraded":"2026-02-24T22:34:03.733775+0000","last_fullsized":"2026-02-24T22:34:03.733775+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":200,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:47:51.922901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10732886,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":476,"num_read_kb":2322,"num_write":1236,"num_write_kb":12235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"312'93","reported_seq":796,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.725906+0000","last_change":"2026-02-24T22:05:18.187293+0000","last_active":"2026-02-24T22:31:54.725906+0000","last_peered":"2026-02-24T22:31:54.725906+0000","last_clean":"2026-02-24T22:31:54.725906+0000","last_became_active":"2026-02-24T22:05:18.186994+0000","last_became_peered":"2026-02-24T22:05:18.186994+0000","last_unstale":"2026-02-24T22:31:54.725906+0000","last_undegraded":"2026-02-24T22:31:54.725906+0000","last_fullsized":"2026-02-24T22:31:54.725906+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:00:02.034352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":352,"num_read_kb":306,"num_write":116,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"312'315","reported_seq":1107,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:54.369202+0000","last_change":"2026-02-24T22:05:16.189106+0000","last_active":"2026-02-24T22:33:54.369202+0000","last_peered":"2026-02-24T22:33:54.369202+0000","last_clean":"2026-02-24T22:33:54.369202+0000","last_became_active":"2026-02-24T22:05:16.188658+0000","last_became_peered":"2026-02-24T22:05:16.188658+0000","last_unstale":"2026-02-24T22:33:54.369202+0000","last_undegraded":"2026-02-24T22:33:54.369202+0000","last_fullsized":"2026-02-24T22:33:54.369202+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:33:27.530851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52140,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":775,"num_read_kb":685,"num_write":1754,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"312'169","reported_seq":891,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.245262+0000","last_change":"2026-02-24T22:05:18.187949+0000","last_active":"2026-02-24T22:34:18.245262+0000","last_peered":"2026-02-24T22:34:18.245262+0000","last_clean":"2026-02-24T22:34:18.245262+0000","last_became_active":"2026-02-24T22:05:18.187682+0000","last_became_peered":"2026-02-24T22:05:18.187682+0000","last_unstale":"2026-02-24T22:34:18.245262+0000","last_undegraded":"2026-02-24T22:34:18.245262+0000","last_fullsized":"2026-02-24T22:34:18.245262+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:07:24.443062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":639,"num_read_kb":531,"num_write":250,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"312'224","reported_seq":924,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:45.924671+0000","last_change":"2026-02-24T22:05:16.185278+0000","last_active":"2026-02-24T22:33:45.924671+0000","last_peered":"2026-02-24T22:33:45.924671+0000","last_clean":"2026-02-24T22:33:45.924671+0000","last_became_active":"2026-02-24T22:05:16.184209+0000","last_became_peered":"2026-02-24T22:05:16.184209+0000","last_unstale":"2026-02-24T22:33:45.924671+0000","last_undegraded":"2026-02-24T22:33:45.924671+0000","last_fullsized":"2026-02-24T22:33:45.924671+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":200,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:50:09.257349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":297746,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":353,"num_read_kb":601,"num_write":1169,"num_write_kb":10636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"312'111","reported_seq":789,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.222919+0000","last_change":"2026-02-24T22:05:18.194092+0000","last_active":"2026-02-24T22:34:18.222919+0000","last_peered":"2026-02-24T22:34:18.222919+0000","last_clean":"2026-02-24T22:34:18.222919+0000","last_became_active":"2026-02-24T22:05:18.193320+0000","last_became_peered":"2026-02-24T22:05:18.193320+0000","last_unstale":"2026-02-24T22:34:18.222919+0000","last_undegraded":"2026-02-24T22:34:18.222919+0000","last_fullsized":"2026-02-24T22:34:18.222919+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":100,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:27:50.398388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":425,"num_read_kb":355,"num_write":146,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"312'212","reported_seq":925,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:46.698672+0000","last_change":"2026-02-24T22:05:16.181803+0000","last_active":"2026-02-24T22:33:46.698672+0000","last_peered":"2026-02-24T22:33:46.698672+0000","last_clean":"2026-02-24T22:33:46.698672+0000","last_became_active":"2026-02-24T22:05:16.181527+0000","last_became_peered":"2026-02-24T22:05:16.181527+0000","last_unstale":"2026-02-24T22:33:46.698672+0000","last_undegraded":"2026-02-24T22:33:46.698672+0000","last_fullsized":"2026-02-24T22:33:46.698672+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:53:05.002989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5243778,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":406,"num_read_kb":5463,"num_write":1149,"num_write_kb":5189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:39.728581+0000","last_change":"2026-02-24T22:05:29.658481+0000","last_active":"2026-02-24T22:29:39.728581+0000","last_peered":"2026-02-24T22:29:39.728581+0000","last_clean":"2026-02-24T22:29:39.728581+0000","last_became_active":"2026-02-24T22:05:29.657764+0000","last_became_peered":"2026-02-24T22:05:29.657764+0000","last_unstale":"2026-02-24T22:29:39.728581+0000","last_undegraded":"2026-02-24T22:29:39.728581+0000","last_fullsized":"2026-02-24T22:29:39.728581+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:34:57.407311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"312'80","reported_seq":734,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.729056+0000","last_change":"2026-02-24T22:05:18.185097+0000","last_active":"2026-02-24T22:31:04.729056+0000","last_peered":"2026-02-24T22:31:04.729056+0000","last_clean":"2026-02-24T22:31:04.729056+0000","last_became_active":"2026-02-24T22:05:18.184771+0000","last_became_peered":"2026-02-24T22:05:18.184771+0000","last_unstale":"2026-02-24T22:31:04.729056+0000","last_undegraded":"2026-02-24T22:31:04.729056+0000","last_fullsized":"2026-02-24T22:31:04.729056+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:34:23.195340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":246,"num_read_kb":208,"num_write":102,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"312'259","reported_seq":1040,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:03.524714+0000","last_change":"2026-02-24T22:05:16.192243+0000","last_active":"2026-02-24T22:34:03.524714+0000","last_peered":"2026-02-24T22:34:03.524714+0000","last_clean":"2026-02-24T22:34:03.524714+0000","last_became_active":"2026-02-24T22:05:16.191841+0000","last_became_peered":"2026-02-24T22:05:16.191841+0000","last_unstale":"2026-02-24T22:34:03.524714+0000","last_undegraded":"2026-02-24T22:34:03.524714+0000","last_fullsized":"2026-02-24T22:34:03.524714+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":200,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:59:17.072003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19280971,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":534,"num_read_kb":81366,"num_write":1249,"num_write_kb":20589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":579,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.745270+0000","last_change":"2026-02-24T22:05:20.194969+0000","last_active":"2026-02-24T22:31:54.745270+0000","last_peered":"2026-02-24T22:31:54.745270+0000","last_clean":"2026-02-24T22:31:54.745270+0000","last_became_active":"2026-02-24T22:05:20.194031+0000","last_became_peered":"2026-02-24T22:05:20.194031+0000","last_unstale":"2026-02-24T22:31:54.745270+0000","last_undegraded":"2026-02-24T22:31:54.745270+0000","last_fullsized":"2026-02-24T22:31:54.745270+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:17:15.557806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"312'254","reported_seq":889,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.375668+0000","last_change":"2026-02-24T22:05:22.192071+0000","last_active":"2026-02-24T22:34:18.375668+0000","last_peered":"2026-02-24T22:34:18.375668+0000","last_clean":"2026-02-24T22:34:18.375668+0000","last_became_active":"2026-02-24T22:05:22.191928+0000","last_became_peered":"2026-02-24T22:05:22.191928+0000","last_unstale":"2026-02-24T22:34:18.375668+0000","last_undegraded":"2026-02-24T22:34:18.375668+0000","last_fullsized":"2026-02-24T22:34:18.375668+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":200,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:56:18.209592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54024,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":400,"num_read_kb":789,"num_write":352,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"312'158","reported_seq":747,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.253470+0000","last_change":"2026-02-24T22:05:24.205235+0000","last_active":"2026-02-24T22:34:18.253470+0000","last_peered":"2026-02-24T22:34:18.253470+0000","last_clean":"2026-02-24T22:34:18.253470+0000","last_became_active":"2026-02-24T22:05:24.204990+0000","last_became_peered":"2026-02-24T22:05:24.204990+0000","last_unstale":"2026-02-24T22:34:18.253470+0000","last_undegraded":"2026-02-24T22:34:18.253470+0000","last_fullsized":"2026-02-24T22:34:18.253470+0000","mapping_epoch":30,"log_start":"37'100","ondisk_log_start":"37'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":100,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:14:42.637540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32373,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":92,"num_read_kb":87,"num_write":391,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":564,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:44.730561+0000","last_change":"2026-02-24T22:05:29.662185+0000","last_active":"2026-02-24T22:29:44.730561+0000","last_peered":"2026-02-24T22:29:44.730561+0000","last_clean":"2026-02-24T22:29:44.730561+0000","last_became_active":"2026-02-24T22:05:29.661747+0000","last_became_peered":"2026-02-24T22:05:29.661747+0000","last_unstale":"2026-02-24T22:29:44.730561+0000","last_undegraded":"2026-02-24T22:29:44.730561+0000","last_fullsized":"2026-02-24T22:29:44.730561+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:08:40.631093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"312'73","reported_seq":743,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:07.153602+0000","last_change":"2026-02-24T22:05:18.191685+0000","last_active":"2026-02-24T22:34:07.153602+0000","last_peered":"2026-02-24T22:34:07.153602+0000","last_clean":"2026-02-24T22:34:07.153602+0000","last_became_active":"2026-02-24T22:05:18.191346+0000","last_became_peered":"2026-02-24T22:05:18.191346+0000","last_unstale":"2026-02-24T22:34:07.153602+0000","last_undegraded":"2026-02-24T22:34:07.153602+0000","last_fullsized":"2026-02-24T22:34:07.153602+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:48:20.436580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":237,"num_read_kb":209,"num_write":87,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"312'213","reported_seq":933,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:41.594986+0000","last_change":"2026-02-24T22:05:16.184337+0000","last_active":"2026-02-24T22:33:41.594986+0000","last_peered":"2026-02-24T22:33:41.594986+0000","last_clean":"2026-02-24T22:33:41.594986+0000","last_became_active":"2026-02-24T22:05:16.183990+0000","last_became_peered":"2026-02-24T22:05:16.183990+0000","last_unstale":"2026-02-24T22:33:41.594986+0000","last_undegraded":"2026-02-24T22:33:41.594986+0000","last_fullsized":"2026-02-24T22:33:41.594986+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":200,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:27:53.239432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22095805,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":400,"num_read_kb":23691,"num_write":1212,"num_write_kb":23324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.729065+0000","last_change":"2026-02-24T22:05:20.186544+0000","last_active":"2026-02-24T22:31:04.729065+0000","last_peered":"2026-02-24T22:31:04.729065+0000","last_clean":"2026-02-24T22:31:04.729065+0000","last_became_active":"2026-02-24T22:05:20.186217+0000","last_became_peered":"2026-02-24T22:05:20.186217+0000","last_unstale":"2026-02-24T22:31:04.729065+0000","last_undegraded":"2026-02-24T22:31:04.729065+0000","last_fullsized":"2026-02-24T22:31:04.729065+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:41:55.793283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"312'304","reported_seq":1150,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.527143+0000","last_change":"2026-02-24T22:05:22.197384+0000","last_active":"2026-02-24T22:34:18.527143+0000","last_peered":"2026-02-24T22:34:18.527143+0000","last_clean":"2026-02-24T22:34:18.527143+0000","last_became_active":"2026-02-24T22:05:22.196911+0000","last_became_peered":"2026-02-24T22:05:22.196911+0000","last_unstale":"2026-02-24T22:34:18.527143+0000","last_undegraded":"2026-02-24T22:34:18.527143+0000","last_fullsized":"2026-02-24T22:34:18.527143+0000","mapping_epoch":28,"log_start":"307'200","ondisk_log_start":"307'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":200,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:19:12.370112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1012,"num_read_kb":538,"num_write":51,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"312'187","reported_seq":800,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.220647+0000","last_change":"2026-02-24T22:05:24.207903+0000","last_active":"2026-02-24T22:34:18.220647+0000","last_peered":"2026-02-24T22:34:18.220647+0000","last_clean":"2026-02-24T22:34:18.220647+0000","last_became_active":"2026-02-24T22:05:24.207692+0000","last_became_peered":"2026-02-24T22:05:24.207692+0000","last_unstale":"2026-02-24T22:34:18.220647+0000","last_undegraded":"2026-02-24T22:34:18.220647+0000","last_fullsized":"2026-02-24T22:34:18.220647+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:14:09.532597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29978,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":173,"num_read_kb":171,"num_write":478,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":573,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.734173+0000","last_change":"2026-02-24T22:05:29.654457+0000","last_active":"2026-02-24T22:30:29.734173+0000","last_peered":"2026-02-24T22:30:29.734173+0000","last_clean":"2026-02-24T22:30:29.734173+0000","last_became_active":"2026-02-24T22:05:29.654288+0000","last_became_peered":"2026-02-24T22:05:29.654288+0000","last_unstale":"2026-02-24T22:30:29.734173+0000","last_undegraded":"2026-02-24T22:30:29.734173+0000","last_fullsized":"2026-02-24T22:30:29.734173+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:05:32.007637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"312'274","reported_seq":998,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.743416+0000","last_change":"2026-02-24T22:05:18.184707+0000","last_active":"2026-02-24T22:31:54.743416+0000","last_peered":"2026-02-24T22:31:54.743416+0000","last_clean":"2026-02-24T22:31:54.743416+0000","last_became_active":"2026-02-24T22:05:18.184542+0000","last_became_peered":"2026-02-24T22:05:18.184542+0000","last_unstale":"2026-02-24T22:31:54.743416+0000","last_undegraded":"2026-02-24T22:31:54.743416+0000","last_fullsized":"2026-02-24T22:31:54.743416+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":200,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:55:45.622376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1179,"num_read_kb":976,"num_write":460,"num_write_kb":243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"312'223","reported_seq":937,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.739663+0000","last_change":"2026-02-24T22:05:16.190582+0000","last_active":"2026-02-24T22:30:59.739663+0000","last_peered":"2026-02-24T22:30:59.739663+0000","last_clean":"2026-02-24T22:30:59.739663+0000","last_became_active":"2026-02-24T22:05:16.190372+0000","last_became_peered":"2026-02-24T22:05:16.190372+0000","last_unstale":"2026-02-24T22:30:59.739663+0000","last_undegraded":"2026-02-24T22:30:59.739663+0000","last_fullsized":"2026-02-24T22:30:59.739663+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":200,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:46:33.357028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":364264,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":363,"num_read_kb":3398,"num_write":1108,"num_write_kb":524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.734254+0000","last_change":"2026-02-24T22:05:20.187522+0000","last_active":"2026-02-24T22:31:54.734254+0000","last_peered":"2026-02-24T22:31:54.734254+0000","last_clean":"2026-02-24T22:31:54.734254+0000","last_became_active":"2026-02-24T22:05:20.187327+0000","last_became_peered":"2026-02-24T22:05:20.187327+0000","last_unstale":"2026-02-24T22:31:54.734254+0000","last_undegraded":"2026-02-24T22:31:54.734254+0000","last_fullsized":"2026-02-24T22:31:54.734254+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:09:52.559206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"312'791","reported_seq":2117,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.545233+0000","last_change":"2026-02-24T22:05:22.191261+0000","last_active":"2026-02-24T22:34:18.545233+0000","last_peered":"2026-02-24T22:34:18.545233+0000","last_clean":"2026-02-24T22:34:18.545233+0000","last_became_active":"2026-02-24T22:05:22.191073+0000","last_became_peered":"2026-02-24T22:05:22.191073+0000","last_unstale":"2026-02-24T22:34:18.545233+0000","last_undegraded":"2026-02-24T22:34:18.545233+0000","last_fullsized":"2026-02-24T22:34:18.545233+0000","mapping_epoch":28,"log_start":"307'700","ondisk_log_start":"307'700","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":700,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:26:27.583739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":460,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2922,"num_read_kb":1499,"num_write":81,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"312'204","reported_seq":1716,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.409302+0000","last_change":"2026-02-24T22:05:24.206373+0000","last_active":"2026-02-24T22:34:17.409302+0000","last_peered":"2026-02-24T22:34:17.409302+0000","last_clean":"2026-02-24T22:34:17.409302+0000","last_became_active":"2026-02-24T22:05:24.206134+0000","last_became_peered":"2026-02-24T22:05:24.206134+0000","last_unstale":"2026-02-24T22:34:17.409302+0000","last_undegraded":"2026-02-24T22:34:17.409302+0000","last_fullsized":"2026-02-24T22:34:17.409302+0000","mapping_epoch":30,"log_start":"37'100","ondisk_log_start":"37'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:32:52.956429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33889,"num_objects":107,"num_object_clones":0,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":0,"num_read":1064,"num_read_kb":1063,"num_write":484,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.733657+0000","last_change":"2026-02-24T22:05:29.658922+0000","last_active":"2026-02-24T22:30:29.733657+0000","last_peered":"2026-02-24T22:30:29.733657+0000","last_clean":"2026-02-24T22:30:29.733657+0000","last_became_active":"2026-02-24T22:05:29.658119+0000","last_became_peered":"2026-02-24T22:05:29.658119+0000","last_unstale":"2026-02-24T22:30:29.733657+0000","last_undegraded":"2026-02-24T22:30:29.733657+0000","last_fullsized":"2026-02-24T22:30:29.733657+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:10:58.821048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"312'668","reported_seq":1465,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:09.715537+0000","last_change":"2026-02-24T22:06:00.662810+0000","last_active":"2026-02-24T22:30:09.715537+0000","last_peered":"2026-02-24T22:30:09.715537+0000","last_clean":"2026-02-24T22:30:09.715537+0000","last_became_active":"2026-02-24T22:06:00.662609+0000","last_became_peered":"2026-02-24T22:06:00.662609+0000","last_unstale":"2026-02-24T22:30:09.715537+0000","last_undegraded":"2026-02-24T22:30:09.715537+0000","last_fullsized":"2026-02-24T22:30:09.715537+0000","mapping_epoch":36,"log_start":"312'600","ondisk_log_start":"312'600","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":600,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:42:35.250489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":2886,"num_read_kb":2388,"num_write":1101,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"312'241","reported_seq":967,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:45.705330+0000","last_change":"2026-02-24T22:05:16.190904+0000","last_active":"2026-02-24T22:33:45.705330+0000","last_peered":"2026-02-24T22:33:45.705330+0000","last_clean":"2026-02-24T22:33:45.705330+0000","last_became_active":"2026-02-24T22:05:16.189739+0000","last_became_peered":"2026-02-24T22:05:16.189739+0000","last_unstale":"2026-02-24T22:33:45.705330+0000","last_undegraded":"2026-02-24T22:33:45.705330+0000","last_fullsized":"2026-02-24T22:33:45.705330+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":200,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:04:06.385930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11122410,"num_objects":84,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":391,"num_read_kb":16925,"num_write":1300,"num_write_kb":17766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":579,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.745280+0000","last_change":"2026-02-24T22:05:20.193438+0000","last_active":"2026-02-24T22:31:54.745280+0000","last_peered":"2026-02-24T22:31:54.745280+0000","last_clean":"2026-02-24T22:31:54.745280+0000","last_became_active":"2026-02-24T22:05:20.193200+0000","last_became_peered":"2026-02-24T22:05:20.193200+0000","last_unstale":"2026-02-24T22:31:54.745280+0000","last_undegraded":"2026-02-24T22:31:54.745280+0000","last_fullsized":"2026-02-24T22:31:54.745280+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:58:09.212117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"312'187","reported_seq":785,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.357268+0000","last_change":"2026-02-24T22:05:22.913003+0000","last_active":"2026-02-24T22:34:18.357268+0000","last_peered":"2026-02-24T22:34:18.357268+0000","last_clean":"2026-02-24T22:34:18.357268+0000","last_became_active":"2026-02-24T22:05:22.912826+0000","last_became_peered":"2026-02-24T22:05:22.912826+0000","last_unstale":"2026-02-24T22:34:18.357268+0000","last_undegraded":"2026-02-24T22:34:18.357268+0000","last_fullsized":"2026-02-24T22:34:18.357268+0000","mapping_epoch":28,"log_start":"312'100","ondisk_log_start":"312'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:49:38.039509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47409,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":295,"num_read_kb":667,"num_write":278,"num_write_kb":255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"312'257","reported_seq":903,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:13.928723+0000","last_change":"2026-02-24T22:05:24.199267+0000","last_active":"2026-02-24T22:34:13.928723+0000","last_peered":"2026-02-24T22:34:13.928723+0000","last_clean":"2026-02-24T22:34:13.928723+0000","last_became_active":"2026-02-24T22:05:24.198964+0000","last_became_peered":"2026-02-24T22:05:24.198964+0000","last_unstale":"2026-02-24T22:34:13.928723+0000","last_undegraded":"2026-02-24T22:34:13.928723+0000","last_fullsized":"2026-02-24T22:34:13.928723+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":200,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:47:04.450411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29433,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":428,"num_read_kb":365,"num_write":612,"num_write_kb":187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"312'113","reported_seq":839,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:55.982068+0000","last_change":"2026-02-24T22:05:18.190057+0000","last_active":"2026-02-24T22:33:55.982068+0000","last_peered":"2026-02-24T22:33:55.982068+0000","last_clean":"2026-02-24T22:33:55.982068+0000","last_became_active":"2026-02-24T22:05:18.189796+0000","last_became_peered":"2026-02-24T22:05:18.189796+0000","last_unstale":"2026-02-24T22:33:55.982068+0000","last_undegraded":"2026-02-24T22:33:55.982068+0000","last_fullsized":"2026-02-24T22:33:55.982068+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":100,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:22:22.190649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":455,"num_read_kb":399,"num_write":145,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"312'189","reported_seq":868,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:51.001577+0000","last_change":"2026-02-24T22:05:16.192443+0000","last_active":"2026-02-24T22:30:51.001577+0000","last_peered":"2026-02-24T22:30:51.001577+0000","last_clean":"2026-02-24T22:30:51.001577+0000","last_became_active":"2026-02-24T22:05:16.191985+0000","last_became_peered":"2026-02-24T22:05:16.191985+0000","last_unstale":"2026-02-24T22:30:51.001577+0000","last_undegraded":"2026-02-24T22:30:51.001577+0000","last_fullsized":"2026-02-24T22:30:51.001577+0000","mapping_epoch":22,"log_start":"312'100","ondisk_log_start":"312'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:48:03.816342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4344205,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":288,"num_read_kb":4498,"num_write":974,"num_write_kb":4311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":586,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.742035+0000","last_change":"2026-02-24T22:05:20.194496+0000","last_active":"2026-02-24T22:31:19.742035+0000","last_peered":"2026-02-24T22:31:19.742035+0000","last_clean":"2026-02-24T22:31:19.742035+0000","last_became_active":"2026-02-24T22:05:20.194288+0000","last_became_peered":"2026-02-24T22:05:20.194288+0000","last_unstale":"2026-02-24T22:31:19.742035+0000","last_undegraded":"2026-02-24T22:31:19.742035+0000","last_fullsized":"2026-02-24T22:31:19.742035+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:55:32.351699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"312'255","reported_seq":879,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.223994+0000","last_change":"2026-02-24T22:05:22.913184+0000","last_active":"2026-02-24T22:34:18.223994+0000","last_peered":"2026-02-24T22:34:18.223994+0000","last_clean":"2026-02-24T22:34:18.223994+0000","last_became_active":"2026-02-24T22:05:22.912859+0000","last_became_peered":"2026-02-24T22:05:22.912859+0000","last_unstale":"2026-02-24T22:34:18.223994+0000","last_undegraded":"2026-02-24T22:34:18.223994+0000","last_fullsized":"2026-02-24T22:34:18.223994+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:37:47.377695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53011,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":405,"num_read_kb":789,"num_write":356,"num_write_kb":269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"312'315","reported_seq":1010,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.329842+0000","last_change":"2026-02-24T22:05:24.200453+0000","last_active":"2026-02-24T22:34:15.329842+0000","last_peered":"2026-02-24T22:34:15.329842+0000","last_clean":"2026-02-24T22:34:15.329842+0000","last_became_active":"2026-02-24T22:05:24.199772+0000","last_became_peered":"2026-02-24T22:05:24.199772+0000","last_unstale":"2026-02-24T22:34:15.329842+0000","last_undegraded":"2026-02-24T22:34:15.329842+0000","last_fullsized":"2026-02-24T22:34:15.329842+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:08:33.217544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30266,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":473,"num_read_kb":498,"num_write":701,"num_write_kb":232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":566,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.735506+0000","last_change":"2026-02-24T22:05:29.661914+0000","last_active":"2026-02-24T22:30:29.735506+0000","last_peered":"2026-02-24T22:30:29.735506+0000","last_clean":"2026-02-24T22:30:29.735506+0000","last_became_active":"2026-02-24T22:05:29.661401+0000","last_became_peered":"2026-02-24T22:05:29.661401+0000","last_unstale":"2026-02-24T22:30:29.735506+0000","last_undegraded":"2026-02-24T22:30:29.735506+0000","last_fullsized":"2026-02-24T22:30:29.735506+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:09:29.373749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"312'84","reported_seq":745,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.739343+0000","last_change":"2026-02-24T22:05:18.193728+0000","last_active":"2026-02-24T22:31:19.739343+0000","last_peered":"2026-02-24T22:31:19.739343+0000","last_clean":"2026-02-24T22:31:19.739343+0000","last_became_active":"2026-02-24T22:05:18.193188+0000","last_became_peered":"2026-02-24T22:05:18.193188+0000","last_unstale":"2026-02-24T22:31:19.739343+0000","last_undegraded":"2026-02-24T22:31:19.739343+0000","last_fullsized":"2026-02-24T22:31:19.739343+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:53:00.913559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":312,"num_read_kb":260,"num_write":126,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"312'245","reported_seq":976,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:58.531217+0000","last_change":"2026-02-24T22:05:16.192300+0000","last_active":"2026-02-24T22:33:58.531217+0000","last_peered":"2026-02-24T22:33:58.531217+0000","last_clean":"2026-02-24T22:33:58.531217+0000","last_became_active":"2026-02-24T22:05:16.191956+0000","last_became_peered":"2026-02-24T22:05:16.191956+0000","last_unstale":"2026-02-24T22:33:58.531217+0000","last_undegraded":"2026-02-24T22:33:58.531217+0000","last_fullsized":"2026-02-24T22:33:58.531217+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":200,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:08:01.293337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10490308,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":447,"num_read_kb":10655,"num_write":1344,"num_write_kb":15460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":594,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.722249+0000","last_change":"2026-02-24T22:05:20.186383+0000","last_active":"2026-02-24T22:31:19.722249+0000","last_peered":"2026-02-24T22:31:19.722249+0000","last_clean":"2026-02-24T22:31:19.722249+0000","last_became_active":"2026-02-24T22:05:20.186055+0000","last_became_peered":"2026-02-24T22:05:20.186055+0000","last_unstale":"2026-02-24T22:31:19.722249+0000","last_undegraded":"2026-02-24T22:31:19.722249+0000","last_fullsized":"2026-02-24T22:31:19.722249+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:31:36.594293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"312'224","reported_seq":827,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.246141+0000","last_change":"2026-02-24T22:05:22.194034+0000","last_active":"2026-02-24T22:34:18.246141+0000","last_peered":"2026-02-24T22:34:18.246141+0000","last_clean":"2026-02-24T22:34:18.246141+0000","last_became_active":"2026-02-24T22:05:22.193741+0000","last_became_peered":"2026-02-24T22:05:22.193741+0000","last_unstale":"2026-02-24T22:34:18.246141+0000","last_undegraded":"2026-02-24T22:34:18.246141+0000","last_fullsized":"2026-02-24T22:34:18.246141+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":200,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:35:27.895920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61189,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":372,"num_read_kb":853,"num_write":354,"num_write_kb":326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"312'185","reported_seq":809,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.176596+0000","last_change":"2026-02-24T22:05:24.199051+0000","last_active":"2026-02-24T22:34:15.176596+0000","last_peered":"2026-02-24T22:34:15.176596+0000","last_clean":"2026-02-24T22:34:15.176596+0000","last_became_active":"2026-02-24T22:05:24.198731+0000","last_became_peered":"2026-02-24T22:05:24.198731+0000","last_unstale":"2026-02-24T22:34:15.176596+0000","last_undegraded":"2026-02-24T22:34:15.176596+0000","last_fullsized":"2026-02-24T22:34:15.176596+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:42:29.899927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30034,"num_objects":91,"num_object_clones":0,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":241,"num_read_kb":207,"num_write":470,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":570,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:34.732629+0000","last_change":"2026-02-24T22:05:29.661462+0000","last_active":"2026-02-24T22:30:34.732629+0000","last_peered":"2026-02-24T22:30:34.732629+0000","last_clean":"2026-02-24T22:30:34.732629+0000","last_became_active":"2026-02-24T22:05:29.661161+0000","last_became_peered":"2026-02-24T22:05:29.661161+0000","last_unstale":"2026-02-24T22:30:34.732629+0000","last_undegraded":"2026-02-24T22:30:34.732629+0000","last_fullsized":"2026-02-24T22:30:34.732629+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:23:22.386231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"312'106","reported_seq":752,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.738326+0000","last_change":"2026-02-24T22:05:18.191843+0000","last_active":"2026-02-24T22:31:24.738326+0000","last_peered":"2026-02-24T22:31:24.738326+0000","last_clean":"2026-02-24T22:31:24.738326+0000","last_became_active":"2026-02-24T22:05:18.191555+0000","last_became_peered":"2026-02-24T22:05:18.191555+0000","last_unstale":"2026-02-24T22:31:24.738326+0000","last_undegraded":"2026-02-24T22:31:24.738326+0000","last_fullsized":"2026-02-24T22:31:24.738326+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:53:20.472053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":387,"num_read_kb":325,"num_write":162,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"312'211","reported_seq":924,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:57.361747+0000","last_change":"2026-02-24T22:05:16.192146+0000","last_active":"2026-02-24T22:30:57.361747+0000","last_peered":"2026-02-24T22:30:57.361747+0000","last_clean":"2026-02-24T22:30:57.361747+0000","last_became_active":"2026-02-24T22:05:16.191733+0000","last_became_peered":"2026-02-24T22:05:16.191733+0000","last_unstale":"2026-02-24T22:30:57.361747+0000","last_undegraded":"2026-02-24T22:30:57.361747+0000","last_fullsized":"2026-02-24T22:30:57.361747+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":200,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:27:59.980345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10494886,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":325,"num_read_kb":31022,"num_write":976,"num_write_kb":10319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":593,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.715007+0000","last_change":"2026-02-24T22:05:20.187327+0000","last_active":"2026-02-24T22:30:04.715007+0000","last_peered":"2026-02-24T22:30:04.715007+0000","last_clean":"2026-02-24T22:30:04.715007+0000","last_became_active":"2026-02-24T22:05:20.186892+0000","last_became_peered":"2026-02-24T22:05:20.186892+0000","last_unstale":"2026-02-24T22:30:04.715007+0000","last_undegraded":"2026-02-24T22:30:04.715007+0000","last_fullsized":"2026-02-24T22:30:04.715007+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:53:23.369187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"312'233","reported_seq":841,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.828563+0000","last_change":"2026-02-24T22:05:22.200501+0000","last_active":"2026-02-24T22:34:17.828563+0000","last_peered":"2026-02-24T22:34:17.828563+0000","last_clean":"2026-02-24T22:34:17.828563+0000","last_became_active":"2026-02-24T22:05:22.200208+0000","last_became_peered":"2026-02-24T22:05:22.200208+0000","last_unstale":"2026-02-24T22:34:17.828563+0000","last_undegraded":"2026-02-24T22:34:17.828563+0000","last_fullsized":"2026-02-24T22:34:17.828563+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":200,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:45:47.054214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57434,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":377,"num_read_kb":830,"num_write":350,"num_write_kb":308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"312'275","reported_seq":1084,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.547089+0000","last_change":"2026-02-24T22:05:24.200107+0000","last_active":"2026-02-24T22:34:18.547089+0000","last_peered":"2026-02-24T22:34:18.547089+0000","last_clean":"2026-02-24T22:34:18.547089+0000","last_became_active":"2026-02-24T22:05:24.199525+0000","last_became_peered":"2026-02-24T22:05:24.199525+0000","last_unstale":"2026-02-24T22:34:18.547089+0000","last_undegraded":"2026-02-24T22:34:18.547089+0000","last_fullsized":"2026-02-24T22:34:18.547089+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":200,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:06:50.680774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32009,"num_objects":103,"num_object_clones":0,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":506,"num_read_kb":490,"num_write":606,"num_write_kb":207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:29.740751+0000","last_change":"2026-02-24T22:05:29.652942+0000","last_active":"2026-02-24T22:31:29.740751+0000","last_peered":"2026-02-24T22:31:29.740751+0000","last_clean":"2026-02-24T22:31:29.740751+0000","last_became_active":"2026-02-24T22:05:29.652598+0000","last_became_peered":"2026-02-24T22:05:29.652598+0000","last_unstale":"2026-02-24T22:31:29.740751+0000","last_undegraded":"2026-02-24T22:31:29.740751+0000","last_fullsized":"2026-02-24T22:31:29.740751+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:38:19.564467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"312'169","reported_seq":870,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:07.869080+0000","last_change":"2026-02-24T22:05:18.185305+0000","last_active":"2026-02-24T22:30:07.869080+0000","last_peered":"2026-02-24T22:30:07.869080+0000","last_clean":"2026-02-24T22:30:07.869080+0000","last_became_active":"2026-02-24T22:05:18.185152+0000","last_became_peered":"2026-02-24T22:05:18.185152+0000","last_unstale":"2026-02-24T22:30:07.869080+0000","last_undegraded":"2026-02-24T22:30:07.869080+0000","last_fullsized":"2026-02-24T22:30:07.869080+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:54:16.288957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":651,"num_read_kb":540,"num_write":239,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"312'258","reported_seq":976,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:12.419421+0000","last_change":"2026-02-24T22:05:16.190979+0000","last_active":"2026-02-24T22:34:12.419421+0000","last_peered":"2026-02-24T22:34:12.419421+0000","last_clean":"2026-02-24T22:34:12.419421+0000","last_became_active":"2026-02-24T22:05:16.190205+0000","last_became_peered":"2026-02-24T22:05:16.190205+0000","last_unstale":"2026-02-24T22:34:12.419421+0000","last_undegraded":"2026-02-24T22:34:12.419421+0000","last_fullsized":"2026-02-24T22:34:12.419421+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:07:24.667384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7827201,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":516,"num_read_kb":20088,"num_write":1234,"num_write_kb":14513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":594,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.722800+0000","last_change":"2026-02-24T22:05:20.186884+0000","last_active":"2026-02-24T22:31:24.722800+0000","last_peered":"2026-02-24T22:31:24.722800+0000","last_clean":"2026-02-24T22:31:24.722800+0000","last_became_active":"2026-02-24T22:05:20.186781+0000","last_became_peered":"2026-02-24T22:05:20.186781+0000","last_unstale":"2026-02-24T22:31:24.722800+0000","last_undegraded":"2026-02-24T22:31:24.722800+0000","last_fullsized":"2026-02-24T22:31:24.722800+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:19:52.348059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"312'391","reported_seq":1001,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.367184+0000","last_change":"2026-02-24T22:05:22.192711+0000","last_active":"2026-02-24T22:34:18.367184+0000","last_peered":"2026-02-24T22:34:18.367184+0000","last_clean":"2026-02-24T22:34:18.367184+0000","last_became_active":"2026-02-24T22:05:22.192495+0000","last_became_peered":"2026-02-24T22:05:22.192495+0000","last_unstale":"2026-02-24T22:34:18.367184+0000","last_undegraded":"2026-02-24T22:34:18.367184+0000","last_fullsized":"2026-02-24T22:34:18.367184+0000","mapping_epoch":28,"log_start":"312'300","ondisk_log_start":"312'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":300,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:40:20.670168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118238,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":673,"num_read_kb":1594,"num_write":646,"num_write_kb":622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"312'173","reported_seq":825,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:03.064297+0000","last_change":"2026-02-24T22:05:24.198313+0000","last_active":"2026-02-24T22:34:03.064297+0000","last_peered":"2026-02-24T22:34:03.064297+0000","last_clean":"2026-02-24T22:34:03.064297+0000","last_became_active":"2026-02-24T22:05:24.198114+0000","last_became_peered":"2026-02-24T22:05:24.198114+0000","last_unstale":"2026-02-24T22:34:03.064297+0000","last_undegraded":"2026-02-24T22:34:03.064297+0000","last_fullsized":"2026-02-24T22:34:03.064297+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":100,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:33:14.283058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29501,"num_objects":91,"num_object_clones":0,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":235,"num_read_kb":226,"num_write":429,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:29.726754+0000","last_change":"2026-02-24T22:05:29.652942+0000","last_active":"2026-02-24T22:31:29.726754+0000","last_peered":"2026-02-24T22:31:29.726754+0000","last_clean":"2026-02-24T22:31:29.726754+0000","last_became_active":"2026-02-24T22:05:29.652715+0000","last_became_peered":"2026-02-24T22:05:29.652715+0000","last_unstale":"2026-02-24T22:31:29.726754+0000","last_undegraded":"2026-02-24T22:31:29.726754+0000","last_fullsized":"2026-02-24T22:31:29.726754+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:13:37.213497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"312'571","reported_seq":1430,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:47.451269+0000","last_change":"2026-02-24T22:05:18.193095+0000","last_active":"2026-02-24T22:30:47.451269+0000","last_peered":"2026-02-24T22:30:47.451269+0000","last_clean":"2026-02-24T22:30:47.451269+0000","last_became_active":"2026-02-24T22:05:18.192543+0000","last_became_peered":"2026-02-24T22:05:18.192543+0000","last_unstale":"2026-02-24T22:30:47.451269+0000","last_undegraded":"2026-02-24T22:30:47.451269+0000","last_fullsized":"2026-02-24T22:30:47.451269+0000","mapping_epoch":24,"log_start":"312'500","ondisk_log_start":"312'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":500,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:42:22.975827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":2576,"num_read_kb":2171,"num_write":966,"num_write_kb":601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"312'189","reported_seq":863,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:55.054639+0000","last_change":"2026-02-24T22:05:16.184025+0000","last_active":"2026-02-24T22:33:55.054639+0000","last_peered":"2026-02-24T22:33:55.054639+0000","last_clean":"2026-02-24T22:33:55.054639+0000","last_became_active":"2026-02-24T22:05:16.182928+0000","last_became_peered":"2026-02-24T22:05:16.182928+0000","last_unstale":"2026-02-24T22:33:55.054639+0000","last_undegraded":"2026-02-24T22:33:55.054639+0000","last_fullsized":"2026-02-24T22:33:55.054639+0000","mapping_epoch":22,"log_start":"312'100","ondisk_log_start":"312'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:19:49.977609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":119466,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":254,"num_read_kb":262,"num_write":1021,"num_write_kb":181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":593,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.715020+0000","last_change":"2026-02-24T22:05:20.186597+0000","last_active":"2026-02-24T22:30:04.715020+0000","last_peered":"2026-02-24T22:30:04.715020+0000","last_clean":"2026-02-24T22:30:04.715020+0000","last_became_active":"2026-02-24T22:05:20.186209+0000","last_became_peered":"2026-02-24T22:05:20.186209+0000","last_unstale":"2026-02-24T22:30:04.715020+0000","last_undegraded":"2026-02-24T22:30:04.715020+0000","last_fullsized":"2026-02-24T22:30:04.715020+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:26:06.692781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"312'940","reported_seq":2252,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.520918+0000","last_change":"2026-02-24T22:05:22.200373+0000","last_active":"2026-02-24T22:34:18.520918+0000","last_peered":"2026-02-24T22:34:18.520918+0000","last_clean":"2026-02-24T22:34:18.520918+0000","last_became_active":"2026-02-24T22:05:22.200111+0000","last_became_peered":"2026-02-24T22:05:22.200111+0000","last_unstale":"2026-02-24T22:34:18.520918+0000","last_undegraded":"2026-02-24T22:34:18.520918+0000","last_fullsized":"2026-02-24T22:34:18.520918+0000","mapping_epoch":28,"log_start":"312'900","ondisk_log_start":"312'900","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":900,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:33:19.581577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54380,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":3207,"num_read_kb":2223,"num_write":413,"num_write_kb":370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"312'183","reported_seq":775,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.249482+0000","last_change":"2026-02-24T22:05:24.201417+0000","last_active":"2026-02-24T22:34:17.249482+0000","last_peered":"2026-02-24T22:34:17.249482+0000","last_clean":"2026-02-24T22:34:17.249482+0000","last_became_active":"2026-02-24T22:05:24.200878+0000","last_became_peered":"2026-02-24T22:05:24.200878+0000","last_unstale":"2026-02-24T22:34:17.249482+0000","last_undegraded":"2026-02-24T22:34:17.249482+0000","last_fullsized":"2026-02-24T22:34:17.249482+0000","mapping_epoch":30,"log_start":"37'100","ondisk_log_start":"37'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":100,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:01:48.708129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33021,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":110,"num_read_kb":109,"num_write":447,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"312'435","reported_seq":2196,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.586410+0000","last_change":"2026-02-24T22:05:29.654545+0000","last_active":"2026-02-24T22:34:18.586410+0000","last_peered":"2026-02-24T22:34:18.586410+0000","last_clean":"2026-02-24T22:34:18.586410+0000","last_became_active":"2026-02-24T22:05:29.654064+0000","last_became_peered":"2026-02-24T22:05:29.654064+0000","last_unstale":"2026-02-24T22:34:18.586410+0000","last_undegraded":"2026-02-24T22:34:18.586410+0000","last_fullsized":"2026-02-24T22:34:18.586410+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":400,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:01:24.704003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"312'128","reported_seq":810,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.731309+0000","last_change":"2026-02-24T22:05:18.192490+0000","last_active":"2026-02-24T22:30:04.731309+0000","last_peered":"2026-02-24T22:30:04.731309+0000","last_clean":"2026-02-24T22:30:04.731309+0000","last_became_active":"2026-02-24T22:05:18.191055+0000","last_became_peered":"2026-02-24T22:05:18.191055+0000","last_unstale":"2026-02-24T22:30:04.731309+0000","last_undegraded":"2026-02-24T22:30:04.731309+0000","last_fullsized":"2026-02-24T22:30:04.731309+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:47:55.429098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":486,"num_read_kb":398,"num_write":170,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"312'174","reported_seq":881,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:06.202574+0000","last_change":"2026-02-24T22:05:16.187505+0000","last_active":"2026-02-24T22:34:06.202574+0000","last_peered":"2026-02-24T22:34:06.202574+0000","last_clean":"2026-02-24T22:34:06.202574+0000","last_became_active":"2026-02-24T22:05:16.187261+0000","last_became_peered":"2026-02-24T22:05:16.187261+0000","last_unstale":"2026-02-24T22:34:06.202574+0000","last_undegraded":"2026-02-24T22:34:06.202574+0000","last_fullsized":"2026-02-24T22:34:06.202574+0000","mapping_epoch":22,"log_start":"312'100","ondisk_log_start":"312'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:13:37.546225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10744628,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":306,"num_read_kb":24764,"num_write":893,"num_write_kb":10562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":588,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.718234+0000","last_change":"2026-02-24T22:05:20.187956+0000","last_active":"2026-02-24T22:30:04.718234+0000","last_peered":"2026-02-24T22:30:04.718234+0000","last_clean":"2026-02-24T22:30:04.718234+0000","last_became_active":"2026-02-24T22:05:20.187488+0000","last_became_peered":"2026-02-24T22:05:20.187488+0000","last_unstale":"2026-02-24T22:30:04.718234+0000","last_undegraded":"2026-02-24T22:30:04.718234+0000","last_fullsized":"2026-02-24T22:30:04.718234+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:08:51.284932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"312'462","reported_seq":1312,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.531191+0000","last_change":"2026-02-24T22:05:22.193340+0000","last_active":"2026-02-24T22:34:18.531191+0000","last_peered":"2026-02-24T22:34:18.531191+0000","last_clean":"2026-02-24T22:34:18.531191+0000","last_became_active":"2026-02-24T22:05:22.193142+0000","last_became_peered":"2026-02-24T22:05:22.193142+0000","last_unstale":"2026-02-24T22:34:18.531191+0000","last_undegraded":"2026-02-24T22:34:18.531191+0000","last_fullsized":"2026-02-24T22:34:18.531191+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":400,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:37:15.676612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51399,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1291,"num_read_kb":1227,"num_write":329,"num_write_kb":279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"312'222","reported_seq":821,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.886791+0000","last_change":"2026-02-24T22:05:24.198994+0000","last_active":"2026-02-24T22:34:17.886791+0000","last_peered":"2026-02-24T22:34:17.886791+0000","last_clean":"2026-02-24T22:34:17.886791+0000","last_became_active":"2026-02-24T22:05:24.198669+0000","last_became_peered":"2026-02-24T22:05:24.198669+0000","last_unstale":"2026-02-24T22:34:17.886791+0000","last_undegraded":"2026-02-24T22:34:17.886791+0000","last_fullsized":"2026-02-24T22:34:17.886791+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":200,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:38:36.897067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29328,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":170,"num_read_kb":165,"num_write":551,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.720723+0000","last_change":"2026-02-24T22:05:29.653411+0000","last_active":"2026-02-24T22:30:29.720723+0000","last_peered":"2026-02-24T22:30:29.720723+0000","last_clean":"2026-02-24T22:30:29.720723+0000","last_became_active":"2026-02-24T22:05:29.653144+0000","last_became_peered":"2026-02-24T22:05:29.653144+0000","last_unstale":"2026-02-24T22:30:29.720723+0000","last_undegraded":"2026-02-24T22:30:29.720723+0000","last_fullsized":"2026-02-24T22:30:29.720723+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:09:20.796950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"312'171","reported_seq":875,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:14.602760+0000","last_change":"2026-02-24T22:05:18.191001+0000","last_active":"2026-02-24T22:34:14.602760+0000","last_peered":"2026-02-24T22:34:14.602760+0000","last_clean":"2026-02-24T22:34:14.602760+0000","last_became_active":"2026-02-24T22:05:18.190140+0000","last_became_peered":"2026-02-24T22:05:18.190140+0000","last_unstale":"2026-02-24T22:34:14.602760+0000","last_undegraded":"2026-02-24T22:34:14.602760+0000","last_fullsized":"2026-02-24T22:34:14.602760+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:34:38.707313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":675,"num_read_kb":556,"num_write":238,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"312'231","reported_seq":992,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:59.603149+0000","last_change":"2026-02-24T22:05:16.185212+0000","last_active":"2026-02-24T22:33:59.603149+0000","last_peered":"2026-02-24T22:33:59.603149+0000","last_clean":"2026-02-24T22:33:59.603149+0000","last_became_active":"2026-02-24T22:05:16.184944+0000","last_became_peered":"2026-02-24T22:05:16.184944+0000","last_unstale":"2026-02-24T22:33:59.603149+0000","last_undegraded":"2026-02-24T22:33:59.603149+0000","last_fullsized":"2026-02-24T22:33:59.603149+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":200,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:42:08.775595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17390927,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":470,"num_read_kb":45888,"num_write":1286,"num_write_kb":20430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"27'2","reported_seq":1055,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.487522+0000","last_change":"2026-02-24T22:05:20.194961+0000","last_active":"2026-02-24T22:34:18.487522+0000","last_peered":"2026-02-24T22:34:18.487522+0000","last_clean":"2026-02-24T22:34:18.487522+0000","last_became_active":"2026-02-24T22:05:20.194145+0000","last_became_peered":"2026-02-24T22:05:20.194145+0000","last_unstale":"2026-02-24T22:34:18.487522+0000","last_undegraded":"2026-02-24T22:34:18.487522+0000","last_fullsized":"2026-02-24T22:34:18.487522+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:18:36.330182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":474,"num_read_kb":474,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"312'56","reported_seq":657,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.737323+0000","last_change":"2026-02-24T22:05:22.905477+0000","last_active":"2026-02-24T22:32:24.737323+0000","last_peered":"2026-02-24T22:32:24.737323+0000","last_clean":"2026-02-24T22:32:24.737323+0000","last_became_active":"2026-02-24T22:05:22.905221+0000","last_became_peered":"2026-02-24T22:05:22.905221+0000","last_unstale":"2026-02-24T22:32:24.737323+0000","last_undegraded":"2026-02-24T22:32:24.737323+0000","last_fullsized":"2026-02-24T22:32:24.737323+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:59:47.641153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"312'159","reported_seq":744,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:14.502434+0000","last_change":"2026-02-24T22:05:24.208011+0000","last_active":"2026-02-24T22:34:14.502434+0000","last_peered":"2026-02-24T22:34:14.502434+0000","last_clean":"2026-02-24T22:34:14.502434+0000","last_became_active":"2026-02-24T22:05:24.207673+0000","last_became_peered":"2026-02-24T22:05:24.207673+0000","last_unstale":"2026-02-24T22:34:14.502434+0000","last_undegraded":"2026-02-24T22:34:14.502434+0000","last_fullsized":"2026-02-24T22:34:14.502434+0000","mapping_epoch":30,"log_start":"312'100","ondisk_log_start":"312'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:29:55.011561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25987,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":107,"num_read_kb":103,"num_write":388,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":577,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.733645+0000","last_change":"2026-02-24T22:05:29.659816+0000","last_active":"2026-02-24T22:30:29.733645+0000","last_peered":"2026-02-24T22:30:29.733645+0000","last_clean":"2026-02-24T22:30:29.733645+0000","last_became_active":"2026-02-24T22:05:29.658842+0000","last_became_peered":"2026-02-24T22:05:29.658842+0000","last_unstale":"2026-02-24T22:30:29.733645+0000","last_undegraded":"2026-02-24T22:30:29.733645+0000","last_fullsized":"2026-02-24T22:30:29.733645+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:55:04.860078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"312'98","reported_seq":778,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:54.820404+0000","last_change":"2026-02-24T22:05:18.188396+0000","last_active":"2026-02-24T22:30:54.820404+0000","last_peered":"2026-02-24T22:30:54.820404+0000","last_clean":"2026-02-24T22:30:54.820404+0000","last_became_active":"2026-02-24T22:05:18.188161+0000","last_became_peered":"2026-02-24T22:05:18.188161+0000","last_unstale":"2026-02-24T22:30:54.820404+0000","last_undegraded":"2026-02-24T22:30:54.820404+0000","last_fullsized":"2026-02-24T22:30:54.820404+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:15:32.648520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":368,"num_read_kb":314,"num_write":134,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"312'239","reported_seq":971,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:49.624638+0000","last_change":"2026-02-24T22:05:16.182699+0000","last_active":"2026-02-24T22:33:49.624638+0000","last_peered":"2026-02-24T22:33:49.624638+0000","last_clean":"2026-02-24T22:33:49.624638+0000","last_became_active":"2026-02-24T22:05:16.182194+0000","last_became_peered":"2026-02-24T22:05:16.182194+0000","last_unstale":"2026-02-24T22:33:49.624638+0000","last_undegraded":"2026-02-24T22:33:49.624638+0000","last_fullsized":"2026-02-24T22:33:49.624638+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":200,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:40:05.294459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5600581,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":390,"num_read_kb":7449,"num_write":1196,"num_write_kb":8933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.731323+0000","last_change":"2026-02-24T22:05:20.193494+0000","last_active":"2026-02-24T22:30:04.731323+0000","last_peered":"2026-02-24T22:30:04.731323+0000","last_clean":"2026-02-24T22:30:04.731323+0000","last_became_active":"2026-02-24T22:05:20.193180+0000","last_became_peered":"2026-02-24T22:05:20.193180+0000","last_unstale":"2026-02-24T22:30:04.731323+0000","last_undegraded":"2026-02-24T22:30:04.731323+0000","last_fullsized":"2026-02-24T22:30:04.731323+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:22:52.729272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"312'270","reported_seq":890,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.361543+0000","last_change":"2026-02-24T22:05:22.200410+0000","last_active":"2026-02-24T22:34:18.361543+0000","last_peered":"2026-02-24T22:34:18.361543+0000","last_clean":"2026-02-24T22:34:18.361543+0000","last_became_active":"2026-02-24T22:05:22.200138+0000","last_became_peered":"2026-02-24T22:05:22.200138+0000","last_unstale":"2026-02-24T22:34:18.361543+0000","last_undegraded":"2026-02-24T22:34:18.361543+0000","last_fullsized":"2026-02-24T22:34:18.361543+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:07:05.040956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52341,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":858,"num_read_kb":1265,"num_write":1270,"num_write_kb":1196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"312'327","reported_seq":951,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.407864+0000","last_change":"2026-02-24T22:05:24.207564+0000","last_active":"2026-02-24T22:34:17.407864+0000","last_peered":"2026-02-24T22:34:17.407864+0000","last_clean":"2026-02-24T22:34:17.407864+0000","last_became_active":"2026-02-24T22:05:24.206745+0000","last_became_peered":"2026-02-24T22:05:24.206745+0000","last_unstale":"2026-02-24T22:34:17.407864+0000","last_undegraded":"2026-02-24T22:34:17.407864+0000","last_fullsized":"2026-02-24T22:34:17.407864+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":300,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:53:45.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27098,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":391,"num_read_kb":376,"num_write":726,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"312'431","reported_seq":4795,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.586354+0000","last_change":"2026-02-24T22:05:29.654842+0000","last_active":"2026-02-24T22:34:18.586354+0000","last_peered":"2026-02-24T22:34:18.586354+0000","last_clean":"2026-02-24T22:34:18.586354+0000","last_became_active":"2026-02-24T22:05:29.654281+0000","last_became_peered":"2026-02-24T22:05:29.654281+0000","last_unstale":"2026-02-24T22:34:18.586354+0000","last_undegraded":"2026-02-24T22:34:18.586354+0000","last_fullsized":"2026-02-24T22:34:18.586354+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":400,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:30:38.038033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"312'114","reported_seq":799,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:05.704625+0000","last_change":"2026-02-24T22:05:18.193654+0000","last_active":"2026-02-24T22:34:05.704625+0000","last_peered":"2026-02-24T22:34:05.704625+0000","last_clean":"2026-02-24T22:34:05.704625+0000","last_became_active":"2026-02-24T22:05:18.193426+0000","last_became_peered":"2026-02-24T22:05:18.193426+0000","last_unstale":"2026-02-24T22:34:05.704625+0000","last_undegraded":"2026-02-24T22:34:05.704625+0000","last_fullsized":"2026-02-24T22:34:05.704625+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:36:20.176524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":443,"num_read_kb":376,"num_write":199,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'112","reported_seq":745,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.741431+0000","last_change":"2026-02-24T22:05:06.709275+0000","last_active":"2026-02-24T22:31:14.741431+0000","last_peered":"2026-02-24T22:31:14.741431+0000","last_clean":"2026-02-24T22:31:14.741431+0000","last_became_active":"2026-02-24T22:05:06.708577+0000","last_became_peered":"2026-02-24T22:05:06.708577+0000","last_unstale":"2026-02-24T22:31:14.741431+0000","last_undegraded":"2026-02-24T22:31:14.741431+0000","last_fullsized":"2026-02-24T22:31:14.741431+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:05.664397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:05.664397+0000","last_clean_scrub_stamp":"2026-02-24T22:05:05.664397+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:46:55.314440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":605,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:14.747819+0000","last_change":"2026-02-24T22:05:10.694707+0000","last_active":"2026-02-24T22:32:14.747819+0000","last_peered":"2026-02-24T22:32:14.747819+0000","last_clean":"2026-02-24T22:32:14.747819+0000","last_became_active":"2026-02-24T22:05:08.689983+0000","last_became_peered":"2026-02-24T22:05:08.689983+0000","last_unstale":"2026-02-24T22:32:14.747819+0000","last_undegraded":"2026-02-24T22:32:14.747819+0000","last_fullsized":"2026-02-24T22:32:14.747819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:42:42.134312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046251899999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"312'265","reported_seq":1043,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:12.240184+0000","last_change":"2026-02-24T22:05:16.186666+0000","last_active":"2026-02-24T22:34:12.240184+0000","last_peered":"2026-02-24T22:34:12.240184+0000","last_clean":"2026-02-24T22:34:12.240184+0000","last_became_active":"2026-02-24T22:05:16.185768+0000","last_became_peered":"2026-02-24T22:05:16.185768+0000","last_unstale":"2026-02-24T22:34:12.240184+0000","last_undegraded":"2026-02-24T22:34:12.240184+0000","last_fullsized":"2026-02-24T22:34:12.240184+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":200,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:33:06.518377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6322908,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":624,"num_read_kb":12841,"num_write":1442,"num_write_kb":6232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":586,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.735364+0000","last_change":"2026-02-24T22:05:20.194414+0000","last_active":"2026-02-24T22:30:19.735364+0000","last_peered":"2026-02-24T22:30:19.735364+0000","last_clean":"2026-02-24T22:30:19.735364+0000","last_became_active":"2026-02-24T22:05:20.194040+0000","last_became_peered":"2026-02-24T22:05:20.194040+0000","last_unstale":"2026-02-24T22:30:19.735364+0000","last_undegraded":"2026-02-24T22:30:19.735364+0000","last_fullsized":"2026-02-24T22:30:19.735364+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:14:22.868157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"312'1049","reported_seq":2141,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.521664+0000","last_change":"2026-02-24T22:05:22.199991+0000","last_active":"2026-02-24T22:34:18.521664+0000","last_peered":"2026-02-24T22:34:18.521664+0000","last_clean":"2026-02-24T22:34:18.521664+0000","last_became_active":"2026-02-24T22:05:22.199602+0000","last_became_peered":"2026-02-24T22:05:22.199602+0000","last_unstale":"2026-02-24T22:34:18.521664+0000","last_undegraded":"2026-02-24T22:34:18.521664+0000","last_fullsized":"2026-02-24T22:34:18.521664+0000","mapping_epoch":28,"log_start":"312'1000","ondisk_log_start":"312'1000","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":1000,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:00:35.201833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163236,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2894,"num_read_kb":3248,"num_write":1008,"num_write_kb":944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"312'202","reported_seq":795,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.242020+0000","last_change":"2026-02-24T22:05:24.201041+0000","last_active":"2026-02-24T22:34:18.242020+0000","last_peered":"2026-02-24T22:34:18.242020+0000","last_clean":"2026-02-24T22:34:18.242020+0000","last_became_active":"2026-02-24T22:05:24.200336+0000","last_became_peered":"2026-02-24T22:05:24.200336+0000","last_unstale":"2026-02-24T22:34:18.242020+0000","last_undegraded":"2026-02-24T22:34:18.242020+0000","last_fullsized":"2026-02-24T22:34:18.242020+0000","mapping_epoch":30,"log_start":"37'100","ondisk_log_start":"37'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:52:42.524705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32535,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":152,"num_read_kb":139,"num_write":501,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"312'424","reported_seq":1832,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.571551+0000","last_change":"2026-02-24T22:05:29.660713+0000","last_active":"2026-02-24T22:34:18.571551+0000","last_peered":"2026-02-24T22:34:18.571551+0000","last_clean":"2026-02-24T22:34:18.571551+0000","last_became_active":"2026-02-24T22:05:29.660370+0000","last_became_peered":"2026-02-24T22:05:29.660370+0000","last_unstale":"2026-02-24T22:34:18.571551+0000","last_undegraded":"2026-02-24T22:34:18.571551+0000","last_fullsized":"2026-02-24T22:34:18.571551+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":400,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:47:59.912901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"312'99","reported_seq":797,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.739591+0000","last_change":"2026-02-24T22:05:18.185501+0000","last_active":"2026-02-24T22:31:19.739591+0000","last_peered":"2026-02-24T22:31:19.739591+0000","last_clean":"2026-02-24T22:31:19.739591+0000","last_became_active":"2026-02-24T22:05:18.185205+0000","last_became_peered":"2026-02-24T22:05:18.185205+0000","last_unstale":"2026-02-24T22:31:19.739591+0000","last_undegraded":"2026-02-24T22:31:19.739591+0000","last_fullsized":"2026-02-24T22:31:19.739591+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:21:17.222637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":346,"num_read_kb":300,"num_write":140,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":610,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.741446+0000","last_change":"2026-02-24T22:05:10.808592+0000","last_active":"2026-02-24T22:31:14.741446+0000","last_peered":"2026-02-24T22:31:14.741446+0000","last_clean":"2026-02-24T22:31:14.741446+0000","last_became_active":"2026-02-24T22:05:08.689972+0000","last_became_peered":"2026-02-24T22:05:08.689972+0000","last_unstale":"2026-02-24T22:31:14.741446+0000","last_undegraded":"2026-02-24T22:31:14.741446+0000","last_fullsized":"2026-02-24T22:31:14.741446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:59:56.330904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048345700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"312'208","reported_seq":966,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:57.024826+0000","last_change":"2026-02-24T22:05:16.183783+0000","last_active":"2026-02-24T22:30:57.024826+0000","last_peered":"2026-02-24T22:30:57.024826+0000","last_clean":"2026-02-24T22:30:57.024826+0000","last_became_active":"2026-02-24T22:05:16.183484+0000","last_became_peered":"2026-02-24T22:05:16.183484+0000","last_unstale":"2026-02-24T22:30:57.024826+0000","last_undegraded":"2026-02-24T22:30:57.024826+0000","last_fullsized":"2026-02-24T22:30:57.024826+0000","mapping_epoch":22,"log_start":"312'100","ondisk_log_start":"312'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":100,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:41:07.836883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5664239,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":524,"num_read_kb":821,"num_write":1237,"num_write_kb":10723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":580,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.741456+0000","last_change":"2026-02-24T22:05:20.195460+0000","last_active":"2026-02-24T22:31:19.741456+0000","last_peered":"2026-02-24T22:31:19.741456+0000","last_clean":"2026-02-24T22:31:19.741456+0000","last_became_active":"2026-02-24T22:05:20.195236+0000","last_became_peered":"2026-02-24T22:05:20.195236+0000","last_unstale":"2026-02-24T22:31:19.741456+0000","last_undegraded":"2026-02-24T22:31:19.741456+0000","last_fullsized":"2026-02-24T22:31:19.741456+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:38:04.823028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"312'1045","reported_seq":2251,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.938361+0000","last_change":"2026-02-24T22:05:22.192588+0000","last_active":"2026-02-24T22:34:18.938361+0000","last_peered":"2026-02-24T22:34:18.938361+0000","last_clean":"2026-02-24T22:34:18.938361+0000","last_became_active":"2026-02-24T22:05:22.192295+0000","last_became_peered":"2026-02-24T22:05:22.192295+0000","last_unstale":"2026-02-24T22:34:18.938361+0000","last_undegraded":"2026-02-24T22:34:18.938361+0000","last_fullsized":"2026-02-24T22:34:18.938361+0000","mapping_epoch":28,"log_start":"312'1001","ondisk_log_start":"312'1001","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":1001,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:00:22.125417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":153342,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":3150,"num_read_kb":4374,"num_write":3940,"num_write_kb":3885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"312'182","reported_seq":763,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.067423+0000","last_change":"2026-02-24T22:05:24.205091+0000","last_active":"2026-02-24T22:34:15.067423+0000","last_peered":"2026-02-24T22:34:15.067423+0000","last_clean":"2026-02-24T22:34:15.067423+0000","last_became_active":"2026-02-24T22:05:24.204774+0000","last_became_peered":"2026-02-24T22:05:24.204774+0000","last_unstale":"2026-02-24T22:34:15.067423+0000","last_undegraded":"2026-02-24T22:34:15.067423+0000","last_fullsized":"2026-02-24T22:34:15.067423+0000","mapping_epoch":30,"log_start":"312'100","ondisk_log_start":"312'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":100,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:38:46.991451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29814,"num_objects":89,"num_object_clones":0,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":0,"num_read":112,"num_read_kb":108,"num_write":454,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.727262+0000","last_change":"2026-02-24T22:05:29.653742+0000","last_active":"2026-02-24T22:31:34.727262+0000","last_peered":"2026-02-24T22:31:34.727262+0000","last_clean":"2026-02-24T22:31:34.727262+0000","last_became_active":"2026-02-24T22:05:29.653391+0000","last_became_peered":"2026-02-24T22:05:29.653391+0000","last_unstale":"2026-02-24T22:31:34.727262+0000","last_undegraded":"2026-02-24T22:31:34.727262+0000","last_fullsized":"2026-02-24T22:31:34.727262+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:11:46.647600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"312'101","reported_seq":791,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.725723+0000","last_change":"2026-02-24T22:05:18.186987+0000","last_active":"2026-02-24T22:31:19.725723+0000","last_peered":"2026-02-24T22:31:19.725723+0000","last_clean":"2026-02-24T22:31:19.725723+0000","last_became_active":"2026-02-24T22:05:18.186126+0000","last_became_peered":"2026-02-24T22:05:18.186126+0000","last_unstale":"2026-02-24T22:31:19.725723+0000","last_undegraded":"2026-02-24T22:31:19.725723+0000","last_fullsized":"2026-02-24T22:31:19.725723+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:14:57.108610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":336,"num_read_kb":289,"num_write":128,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":618,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:14.727952+0000","last_change":"2026-02-24T22:05:10.684538+0000","last_active":"2026-02-24T22:32:14.727952+0000","last_peered":"2026-02-24T22:32:14.727952+0000","last_clean":"2026-02-24T22:32:14.727952+0000","last_became_active":"2026-02-24T22:05:08.682035+0000","last_became_peered":"2026-02-24T22:05:08.682035+0000","last_unstale":"2026-02-24T22:32:14.727952+0000","last_undegraded":"2026-02-24T22:32:14.727952+0000","last_fullsized":"2026-02-24T22:32:14.727952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:59:46.243250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047531599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"312'233","reported_seq":977,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.726180+0000","last_change":"2026-02-24T22:05:16.178900+0000","last_active":"2026-02-24T22:31:24.726180+0000","last_peered":"2026-02-24T22:31:24.726180+0000","last_clean":"2026-02-24T22:31:24.726180+0000","last_became_active":"2026-02-24T22:05:16.178331+0000","last_became_peered":"2026-02-24T22:05:16.178331+0000","last_unstale":"2026-02-24T22:31:24.726180+0000","last_undegraded":"2026-02-24T22:31:24.726180+0000","last_fullsized":"2026-02-24T22:31:24.726180+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":200,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:08:05.851722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11570089,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":524,"num_read_kb":6587,"num_write":1274,"num_write_kb":11354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"27'1","reported_seq":595,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.722286+0000","last_change":"2026-02-24T22:05:20.188183+0000","last_active":"2026-02-24T22:31:19.722286+0000","last_peered":"2026-02-24T22:31:19.722286+0000","last_clean":"2026-02-24T22:31:19.722286+0000","last_became_active":"2026-02-24T22:05:20.187908+0000","last_became_peered":"2026-02-24T22:05:20.187908+0000","last_unstale":"2026-02-24T22:31:19.722286+0000","last_undegraded":"2026-02-24T22:31:19.722286+0000","last_fullsized":"2026-02-24T22:31:19.722286+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:49:17.694606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"312'436","reported_seq":1644,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.921712+0000","last_change":"2026-02-24T22:05:22.912798+0000","last_active":"2026-02-24T22:34:18.921712+0000","last_peered":"2026-02-24T22:34:18.921712+0000","last_clean":"2026-02-24T22:34:18.921712+0000","last_became_active":"2026-02-24T22:05:22.912414+0000","last_became_peered":"2026-02-24T22:05:22.912414+0000","last_unstale":"2026-02-24T22:34:18.921712+0000","last_undegraded":"2026-02-24T22:34:18.921712+0000","last_fullsized":"2026-02-24T22:34:18.921712+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":400,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:05:47.634681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":308,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1076,"num_read_kb":597,"num_write":114,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"312'184","reported_seq":770,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.551095+0000","last_change":"2026-02-24T22:05:24.200671+0000","last_active":"2026-02-24T22:34:18.551095+0000","last_peered":"2026-02-24T22:34:18.551095+0000","last_clean":"2026-02-24T22:34:18.551095+0000","last_became_active":"2026-02-24T22:05:24.200029+0000","last_became_peered":"2026-02-24T22:05:24.200029+0000","last_unstale":"2026-02-24T22:34:18.551095+0000","last_undegraded":"2026-02-24T22:34:18.551095+0000","last_fullsized":"2026-02-24T22:34:18.551095+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":100,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:18:37.550289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29918,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":442,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":576,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.724006+0000","last_change":"2026-02-24T22:05:29.653992+0000","last_active":"2026-02-24T22:30:14.724006+0000","last_peered":"2026-02-24T22:30:14.724006+0000","last_clean":"2026-02-24T22:30:14.724006+0000","last_became_active":"2026-02-24T22:05:29.653806+0000","last_became_peered":"2026-02-24T22:05:29.653806+0000","last_unstale":"2026-02-24T22:30:14.724006+0000","last_undegraded":"2026-02-24T22:30:14.724006+0000","last_fullsized":"2026-02-24T22:30:14.724006+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:44:18.672652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"312'230","reported_seq":970,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:51.044848+0000","last_change":"2026-02-24T22:05:18.184825+0000","last_active":"2026-02-24T22:30:51.044848+0000","last_peered":"2026-02-24T22:30:51.044848+0000","last_clean":"2026-02-24T22:30:51.044848+0000","last_became_active":"2026-02-24T22:05:18.183952+0000","last_became_peered":"2026-02-24T22:05:18.183952+0000","last_unstale":"2026-02-24T22:30:51.044848+0000","last_undegraded":"2026-02-24T22:30:51.044848+0000","last_fullsized":"2026-02-24T22:30:51.044848+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:33:27.005363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":963,"num_read_kb":811,"num_write":374,"num_write_kb":210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":606,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:14.747800+0000","last_change":"2026-02-24T22:05:10.695226+0000","last_active":"2026-02-24T22:32:14.747800+0000","last_peered":"2026-02-24T22:32:14.747800+0000","last_clean":"2026-02-24T22:32:14.747800+0000","last_became_active":"2026-02-24T22:05:08.690042+0000","last_became_peered":"2026-02-24T22:05:08.690042+0000","last_unstale":"2026-02-24T22:32:14.747800+0000","last_undegraded":"2026-02-24T22:32:14.747800+0000","last_fullsized":"2026-02-24T22:32:14.747800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:27:03.349026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010128400000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"312'234","reported_seq":948,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:42.673103+0000","last_change":"2026-02-24T22:05:16.189390+0000","last_active":"2026-02-24T22:33:42.673103+0000","last_peered":"2026-02-24T22:33:42.673103+0000","last_clean":"2026-02-24T22:33:42.673103+0000","last_became_active":"2026-02-24T22:05:16.188900+0000","last_became_peered":"2026-02-24T22:05:16.188900+0000","last_unstale":"2026-02-24T22:33:42.673103+0000","last_undegraded":"2026-02-24T22:33:42.673103+0000","last_fullsized":"2026-02-24T22:33:42.673103+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":200,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:22:52.382242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11025931,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":322,"num_read_kb":10969,"num_write":1184,"num_write_kb":10859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":589,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.733650+0000","last_change":"2026-02-24T22:05:20.187605+0000","last_active":"2026-02-24T22:30:24.733650+0000","last_peered":"2026-02-24T22:30:24.733650+0000","last_clean":"2026-02-24T22:30:24.733650+0000","last_became_active":"2026-02-24T22:05:20.187251+0000","last_became_peered":"2026-02-24T22:05:20.187251+0000","last_unstale":"2026-02-24T22:30:24.733650+0000","last_undegraded":"2026-02-24T22:30:24.733650+0000","last_fullsized":"2026-02-24T22:30:24.733650+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:47:16.465774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"312'76","reported_seq":697,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.731188+0000","last_change":"2026-02-24T22:05:22.191860+0000","last_active":"2026-02-24T22:31:24.731188+0000","last_peered":"2026-02-24T22:31:24.731188+0000","last_clean":"2026-02-24T22:31:24.731188+0000","last_became_active":"2026-02-24T22:05:22.191394+0000","last_became_peered":"2026-02-24T22:05:22.191394+0000","last_unstale":"2026-02-24T22:31:24.731188+0000","last_undegraded":"2026-02-24T22:31:24.731188+0000","last_fullsized":"2026-02-24T22:31:24.731188+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:12:53.711063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":103,"num_read_kb":96,"num_write":72,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"312'229","reported_seq":843,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.223179+0000","last_change":"2026-02-24T22:05:24.208017+0000","last_active":"2026-02-24T22:34:17.223179+0000","last_peered":"2026-02-24T22:34:17.223179+0000","last_clean":"2026-02-24T22:34:17.223179+0000","last_became_active":"2026-02-24T22:05:24.207698+0000","last_became_peered":"2026-02-24T22:05:24.207698+0000","last_unstale":"2026-02-24T22:34:17.223179+0000","last_undegraded":"2026-02-24T22:34:17.223179+0000","last_fullsized":"2026-02-24T22:34:17.223179+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":200,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:35:36.488781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31273,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":246,"num_read_kb":224,"num_write":553,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":566,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:29.742534+0000","last_change":"2026-02-24T22:05:29.660487+0000","last_active":"2026-02-24T22:31:29.742534+0000","last_peered":"2026-02-24T22:31:29.742534+0000","last_clean":"2026-02-24T22:31:29.742534+0000","last_became_active":"2026-02-24T22:05:29.660065+0000","last_became_peered":"2026-02-24T22:05:29.660065+0000","last_unstale":"2026-02-24T22:31:29.742534+0000","last_undegraded":"2026-02-24T22:31:29.742534+0000","last_fullsized":"2026-02-24T22:31:29.742534+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:00:34.244351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"312'172","reported_seq":869,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:42.925419+0000","last_change":"2026-02-24T22:05:18.185556+0000","last_active":"2026-02-24T22:33:42.925419+0000","last_peered":"2026-02-24T22:33:42.925419+0000","last_clean":"2026-02-24T22:33:42.925419+0000","last_became_active":"2026-02-24T22:05:18.185112+0000","last_became_peered":"2026-02-24T22:05:18.185112+0000","last_unstale":"2026-02-24T22:33:42.925419+0000","last_undegraded":"2026-02-24T22:33:42.925419+0000","last_fullsized":"2026-02-24T22:33:42.925419+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":100,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:39:28.189482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":679,"num_read_kb":553,"num_write":246,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"312'218","reported_seq":939,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:12.225390+0000","last_change":"2026-02-24T22:05:16.191455+0000","last_active":"2026-02-24T22:34:12.225390+0000","last_peered":"2026-02-24T22:34:12.225390+0000","last_clean":"2026-02-24T22:34:12.225390+0000","last_became_active":"2026-02-24T22:05:16.191007+0000","last_became_peered":"2026-02-24T22:05:16.191007+0000","last_unstale":"2026-02-24T22:34:12.225390+0000","last_undegraded":"2026-02-24T22:34:12.225390+0000","last_fullsized":"2026-02-24T22:34:12.225390+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":200,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:46:55.713025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13669125,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":436,"num_read_kb":40330,"num_write":1279,"num_write_kb":18546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":613,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:14.731453+0000","last_change":"2026-02-24T22:05:10.689379+0000","last_active":"2026-02-24T22:32:14.731453+0000","last_peered":"2026-02-24T22:32:14.731453+0000","last_clean":"2026-02-24T22:32:14.731453+0000","last_became_active":"2026-02-24T22:05:08.758282+0000","last_became_peered":"2026-02-24T22:05:08.758282+0000","last_unstale":"2026-02-24T22:32:14.731453+0000","last_undegraded":"2026-02-24T22:32:14.731453+0000","last_fullsized":"2026-02-24T22:32:14.731453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:28:00.429651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033355899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":589,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.733043+0000","last_change":"2026-02-24T22:05:20.187544+0000","last_active":"2026-02-24T22:30:19.733043+0000","last_peered":"2026-02-24T22:30:19.733043+0000","last_clean":"2026-02-24T22:30:19.733043+0000","last_became_active":"2026-02-24T22:05:20.187208+0000","last_became_peered":"2026-02-24T22:05:20.187208+0000","last_unstale":"2026-02-24T22:30:19.733043+0000","last_undegraded":"2026-02-24T22:30:19.733043+0000","last_fullsized":"2026-02-24T22:30:19.733043+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:13:41.845882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"312'1120","reported_seq":2461,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.528903+0000","last_change":"2026-02-24T22:05:22.192049+0000","last_active":"2026-02-24T22:34:18.528903+0000","last_peered":"2026-02-24T22:34:18.528903+0000","last_clean":"2026-02-24T22:34:18.528903+0000","last_became_active":"2026-02-24T22:05:22.191763+0000","last_became_peered":"2026-02-24T22:05:22.191763+0000","last_unstale":"2026-02-24T22:34:18.528903+0000","last_undegraded":"2026-02-24T22:34:18.528903+0000","last_fullsized":"2026-02-24T22:34:18.528903+0000","mapping_epoch":28,"log_start":"312'1100","ondisk_log_start":"312'1100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":1100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:42:04.815445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96815,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":3479,"num_read_kb":2825,"num_write":625,"num_write_kb":530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"312'204","reported_seq":808,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.405693+0000","last_change":"2026-02-24T22:05:24.206550+0000","last_active":"2026-02-24T22:34:18.405693+0000","last_peered":"2026-02-24T22:34:18.405693+0000","last_clean":"2026-02-24T22:34:18.405693+0000","last_became_active":"2026-02-24T22:05:24.205923+0000","last_became_peered":"2026-02-24T22:05:24.205923+0000","last_unstale":"2026-02-24T22:34:18.405693+0000","last_undegraded":"2026-02-24T22:34:18.405693+0000","last_fullsized":"2026-02-24T22:34:18.405693+0000","mapping_epoch":30,"log_start":"37'100","ondisk_log_start":"37'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:42:44.001897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34551,"num_objects":102,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":190,"num_read_kb":174,"num_write":516,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"312'462","reported_seq":4824,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.586084+0000","last_change":"2026-02-24T22:05:29.652714+0000","last_active":"2026-02-24T22:34:18.586084+0000","last_peered":"2026-02-24T22:34:18.586084+0000","last_clean":"2026-02-24T22:34:18.586084+0000","last_became_active":"2026-02-24T22:05:29.652259+0000","last_became_peered":"2026-02-24T22:05:29.652259+0000","last_unstale":"2026-02-24T22:34:18.586084+0000","last_undegraded":"2026-02-24T22:34:18.586084+0000","last_fullsized":"2026-02-24T22:34:18.586084+0000","mapping_epoch":33,"log_start":"312'401","ondisk_log_start":"312'401","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":401,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:52:38.347176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"312'114","reported_seq":810,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.733052+0000","last_change":"2026-02-24T22:05:18.183793+0000","last_active":"2026-02-24T22:30:19.733052+0000","last_peered":"2026-02-24T22:30:19.733052+0000","last_clean":"2026-02-24T22:30:19.733052+0000","last_became_active":"2026-02-24T22:05:18.183369+0000","last_became_peered":"2026-02-24T22:05:18.183369+0000","last_unstale":"2026-02-24T22:30:19.733052+0000","last_undegraded":"2026-02-24T22:30:19.733052+0000","last_fullsized":"2026-02-24T22:30:19.733052+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:10:34.192055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":458,"num_read_kb":386,"num_write":168,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"312'435","reported_seq":1245,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:07.118566+0000","last_change":"2026-02-24T22:05:16.179055+0000","last_active":"2026-02-24T22:34:07.118566+0000","last_peered":"2026-02-24T22:34:07.118566+0000","last_clean":"2026-02-24T22:34:07.118566+0000","last_became_active":"2026-02-24T22:05:16.178353+0000","last_became_peered":"2026-02-24T22:05:16.178353+0000","last_unstale":"2026-02-24T22:34:07.118566+0000","last_undegraded":"2026-02-24T22:34:07.118566+0000","last_fullsized":"2026-02-24T22:34:07.118566+0000","mapping_epoch":22,"log_start":"312'400","ondisk_log_start":"312'400","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":400,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:22:52.210538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":621081,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":974,"num_read_kb":9549,"num_write":1984,"num_write_kb":2098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":610,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:19.749000+0000","last_change":"2026-02-24T22:05:10.808508+0000","last_active":"2026-02-24T22:32:19.749000+0000","last_peered":"2026-02-24T22:32:19.749000+0000","last_clean":"2026-02-24T22:32:19.749000+0000","last_became_active":"2026-02-24T22:05:08.765129+0000","last_became_peered":"2026-02-24T22:05:08.765129+0000","last_unstale":"2026-02-24T22:32:19.749000+0000","last_undegraded":"2026-02-24T22:32:19.749000+0000","last_fullsized":"2026-02-24T22:32:19.749000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:05:15.499818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439276,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":584,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.730929+0000","last_change":"2026-02-24T22:05:20.193089+0000","last_active":"2026-02-24T22:30:19.730929+0000","last_peered":"2026-02-24T22:30:19.730929+0000","last_clean":"2026-02-24T22:30:19.730929+0000","last_became_active":"2026-02-24T22:05:20.192811+0000","last_became_peered":"2026-02-24T22:05:20.192811+0000","last_unstale":"2026-02-24T22:30:19.730929+0000","last_undegraded":"2026-02-24T22:30:19.730929+0000","last_fullsized":"2026-02-24T22:30:19.730929+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:09:16.601460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"312'607","reported_seq":1715,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.543706+0000","last_change":"2026-02-24T22:05:22.905928+0000","last_active":"2026-02-24T22:34:18.543706+0000","last_peered":"2026-02-24T22:34:18.543706+0000","last_clean":"2026-02-24T22:34:18.543706+0000","last_became_active":"2026-02-24T22:05:22.905663+0000","last_became_peered":"2026-02-24T22:05:22.905663+0000","last_unstale":"2026-02-24T22:34:18.543706+0000","last_undegraded":"2026-02-24T22:34:18.543706+0000","last_fullsized":"2026-02-24T22:34:18.543706+0000","mapping_epoch":28,"log_start":"307'500","ondisk_log_start":"307'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":500,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:36:30.224989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":308,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":2056,"num_read_kb":1107,"num_write":154,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"312'171","reported_seq":753,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.533054+0000","last_change":"2026-02-24T22:05:24.206213+0000","last_active":"2026-02-24T22:34:18.533054+0000","last_peered":"2026-02-24T22:34:18.533054+0000","last_clean":"2026-02-24T22:34:18.533054+0000","last_became_active":"2026-02-24T22:05:24.205847+0000","last_became_peered":"2026-02-24T22:05:24.205847+0000","last_unstale":"2026-02-24T22:34:18.533054+0000","last_undegraded":"2026-02-24T22:34:18.533054+0000","last_fullsized":"2026-02-24T22:34:18.533054+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:10:54.760147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26185,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":121,"num_read_kb":112,"num_write":413,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":576,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.724548+0000","last_change":"2026-02-24T22:05:29.651555+0000","last_active":"2026-02-24T22:30:19.724548+0000","last_peered":"2026-02-24T22:30:19.724548+0000","last_clean":"2026-02-24T22:30:19.724548+0000","last_became_active":"2026-02-24T22:05:29.651401+0000","last_became_peered":"2026-02-24T22:05:29.651401+0000","last_unstale":"2026-02-24T22:30:19.724548+0000","last_undegraded":"2026-02-24T22:30:19.724548+0000","last_fullsized":"2026-02-24T22:30:19.724548+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:36:54.239488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"312'150","reported_seq":847,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:40.233137+0000","last_change":"2026-02-24T22:05:18.186337+0000","last_active":"2026-02-24T22:33:40.233137+0000","last_peered":"2026-02-24T22:33:40.233137+0000","last_clean":"2026-02-24T22:33:40.233137+0000","last_became_active":"2026-02-24T22:05:18.185592+0000","last_became_peered":"2026-02-24T22:05:18.185592+0000","last_unstale":"2026-02-24T22:33:40.233137+0000","last_undegraded":"2026-02-24T22:33:40.233137+0000","last_fullsized":"2026-02-24T22:33:40.233137+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:52:57.542872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":536,"num_read_kb":444,"num_write":207,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"312'232","reported_seq":958,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:01.140144+0000","last_change":"2026-02-24T22:05:16.185307+0000","last_active":"2026-02-24T22:34:01.140144+0000","last_peered":"2026-02-24T22:34:01.140144+0000","last_clean":"2026-02-24T22:34:01.140144+0000","last_became_active":"2026-02-24T22:05:16.184131+0000","last_became_peered":"2026-02-24T22:05:16.184131+0000","last_unstale":"2026-02-24T22:34:01.140144+0000","last_undegraded":"2026-02-24T22:34:01.140144+0000","last_fullsized":"2026-02-24T22:34:01.140144+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":200,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:09:05.496588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":173363,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":395,"num_read_kb":1193,"num_write":1232,"num_write_kb":244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":612,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.724713+0000","last_change":"2026-02-24T22:05:10.689444+0000","last_active":"2026-02-24T22:31:09.724713+0000","last_peered":"2026-02-24T22:31:09.724713+0000","last_clean":"2026-02-24T22:31:09.724713+0000","last_became_active":"2026-02-24T22:05:08.682913+0000","last_became_peered":"2026-02-24T22:05:08.682913+0000","last_unstale":"2026-02-24T22:31:09.724713+0000","last_undegraded":"2026-02-24T22:31:09.724713+0000","last_fullsized":"2026-02-24T22:31:09.724713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:20:01.744772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000359878,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":591,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.732657+0000","last_change":"2026-02-24T22:05:20.192401+0000","last_active":"2026-02-24T22:30:19.732657+0000","last_peered":"2026-02-24T22:30:19.732657+0000","last_clean":"2026-02-24T22:30:19.732657+0000","last_became_active":"2026-02-24T22:05:20.192162+0000","last_became_peered":"2026-02-24T22:05:20.192162+0000","last_unstale":"2026-02-24T22:30:19.732657+0000","last_undegraded":"2026-02-24T22:30:19.732657+0000","last_fullsized":"2026-02-24T22:30:19.732657+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:51:58.434704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"312'1027","reported_seq":2586,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.531120+0000","last_change":"2026-02-24T22:05:22.198775+0000","last_active":"2026-02-24T22:34:18.531120+0000","last_peered":"2026-02-24T22:34:18.531120+0000","last_clean":"2026-02-24T22:34:18.531120+0000","last_became_active":"2026-02-24T22:05:22.198252+0000","last_became_peered":"2026-02-24T22:05:22.198252+0000","last_unstale":"2026-02-24T22:34:18.531120+0000","last_undegraded":"2026-02-24T22:34:18.531120+0000","last_fullsized":"2026-02-24T22:34:18.531120+0000","mapping_epoch":28,"log_start":"312'1000","ondisk_log_start":"312'1000","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":1000,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:34:58.507593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":614,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":3859,"num_read_kb":1967,"num_write":82,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"312'1698","reported_seq":4314,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.551488+0000","last_change":"2026-02-24T22:05:24.198591+0000","last_active":"2026-02-24T22:34:18.551488+0000","last_peered":"2026-02-24T22:34:18.551488+0000","last_clean":"2026-02-24T22:34:18.551488+0000","last_became_active":"2026-02-24T22:05:24.198348+0000","last_became_peered":"2026-02-24T22:05:24.198348+0000","last_unstale":"2026-02-24T22:34:18.551488+0000","last_undegraded":"2026-02-24T22:34:18.551488+0000","last_fullsized":"2026-02-24T22:34:18.551488+0000","mapping_epoch":30,"log_start":"310'1600","ondisk_log_start":"310'1600","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":1600,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:04:55.163411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31448,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":5151,"num_read_kb":231050,"num_write":3475,"num_write_kb":1655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":574,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.732473+0000","last_change":"2026-02-24T22:05:29.652643+0000","last_active":"2026-02-24T22:30:14.732473+0000","last_peered":"2026-02-24T22:30:14.732473+0000","last_clean":"2026-02-24T22:30:14.732473+0000","last_became_active":"2026-02-24T22:05:29.652321+0000","last_became_peered":"2026-02-24T22:05:29.652321+0000","last_unstale":"2026-02-24T22:30:14.732473+0000","last_undegraded":"2026-02-24T22:30:14.732473+0000","last_fullsized":"2026-02-24T22:30:14.732473+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:23:33.530605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"312'55","reported_seq":718,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.732644+0000","last_change":"2026-02-24T22:05:18.191716+0000","last_active":"2026-02-24T22:30:19.732644+0000","last_peered":"2026-02-24T22:30:19.732644+0000","last_clean":"2026-02-24T22:30:19.732644+0000","last_became_active":"2026-02-24T22:05:18.191249+0000","last_became_peered":"2026-02-24T22:05:18.191249+0000","last_unstale":"2026-02-24T22:30:19.732644+0000","last_undegraded":"2026-02-24T22:30:19.732644+0000","last_fullsized":"2026-02-24T22:30:19.732644+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:48:35.012678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":226,"num_read_kb":230,"num_write":70,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"312'209","reported_seq":941,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:06.434490+0000","last_change":"2026-02-24T22:05:16.182104+0000","last_active":"2026-02-24T22:34:06.434490+0000","last_peered":"2026-02-24T22:34:06.434490+0000","last_clean":"2026-02-24T22:34:06.434490+0000","last_became_active":"2026-02-24T22:05:16.181840+0000","last_became_peered":"2026-02-24T22:05:16.181840+0000","last_unstale":"2026-02-24T22:34:06.434490+0000","last_undegraded":"2026-02-24T22:34:06.434490+0000","last_fullsized":"2026-02-24T22:34:06.434490+0000","mapping_epoch":22,"log_start":"312'100","ondisk_log_start":"312'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":100,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:50:28.643829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11590618,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":429,"num_read_kb":7386,"num_write":1193,"num_write_kb":13063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":607,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.736747+0000","last_change":"2026-02-24T22:05:10.808402+0000","last_active":"2026-02-24T22:31:09.736747+0000","last_peered":"2026-02-24T22:31:09.736747+0000","last_clean":"2026-02-24T22:31:09.736747+0000","last_became_active":"2026-02-24T22:05:08.689542+0000","last_became_peered":"2026-02-24T22:05:08.689542+0000","last_unstale":"2026-02-24T22:31:09.736747+0000","last_undegraded":"2026-02-24T22:31:09.736747+0000","last_fullsized":"2026-02-24T22:31:09.736747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:23:22.404541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038816099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":592,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.731136+0000","last_change":"2026-02-24T22:05:20.186366+0000","last_active":"2026-02-24T22:31:24.731136+0000","last_peered":"2026-02-24T22:31:24.731136+0000","last_clean":"2026-02-24T22:31:24.731136+0000","last_became_active":"2026-02-24T22:05:20.186108+0000","last_became_peered":"2026-02-24T22:05:20.186108+0000","last_unstale":"2026-02-24T22:31:24.731136+0000","last_undegraded":"2026-02-24T22:31:24.731136+0000","last_fullsized":"2026-02-24T22:31:24.731136+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:02:47.567361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"312'592","reported_seq":1229,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.355778+0000","last_change":"2026-02-24T22:05:22.911959+0000","last_active":"2026-02-24T22:34:18.355778+0000","last_peered":"2026-02-24T22:34:18.355778+0000","last_clean":"2026-02-24T22:34:18.355778+0000","last_became_active":"2026-02-24T22:05:22.911692+0000","last_became_peered":"2026-02-24T22:05:22.911692+0000","last_unstale":"2026-02-24T22:34:18.355778+0000","last_undegraded":"2026-02-24T22:34:18.355778+0000","last_fullsized":"2026-02-24T22:34:18.355778+0000","mapping_epoch":28,"log_start":"312'500","ondisk_log_start":"312'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":500,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:53:02.224623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":161291,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1010,"num_read_kb":2303,"num_write":954,"num_write_kb":876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"312'175","reported_seq":763,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:07.165906+0000","last_change":"2026-02-24T22:05:24.199612+0000","last_active":"2026-02-24T22:34:07.165906+0000","last_peered":"2026-02-24T22:34:07.165906+0000","last_clean":"2026-02-24T22:34:07.165906+0000","last_became_active":"2026-02-24T22:05:24.199398+0000","last_became_peered":"2026-02-24T22:05:24.199398+0000","last_unstale":"2026-02-24T22:34:07.165906+0000","last_undegraded":"2026-02-24T22:34:07.165906+0000","last_fullsized":"2026-02-24T22:34:07.165906+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":100,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:03:11.629672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30139,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":95,"num_read_kb":93,"num_write":436,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"312'428","reported_seq":3586,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.571386+0000","last_change":"2026-02-24T22:05:29.662212+0000","last_active":"2026-02-24T22:34:18.571386+0000","last_peered":"2026-02-24T22:34:18.571386+0000","last_clean":"2026-02-24T22:34:18.571386+0000","last_became_active":"2026-02-24T22:05:29.661694+0000","last_became_peered":"2026-02-24T22:05:29.661694+0000","last_unstale":"2026-02-24T22:34:18.571386+0000","last_undegraded":"2026-02-24T22:34:18.571386+0000","last_fullsized":"2026-02-24T22:34:18.571386+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":400,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:28:47.623604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"312'456","reported_seq":2590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.586383+0000","last_change":"2026-02-24T22:05:29.654721+0000","last_active":"2026-02-24T22:34:18.586383+0000","last_peered":"2026-02-24T22:34:18.586383+0000","last_clean":"2026-02-24T22:34:18.586383+0000","last_became_active":"2026-02-24T22:05:29.654058+0000","last_became_peered":"2026-02-24T22:05:29.654058+0000","last_unstale":"2026-02-24T22:34:18.586383+0000","last_undegraded":"2026-02-24T22:34:18.586383+0000","last_fullsized":"2026-02-24T22:34:18.586383+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":400,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:19:34.289230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"312'4194","reported_seq":4926,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:45.073603+0000","last_change":"2026-02-24T22:05:18.191351+0000","last_active":"2026-02-24T22:33:45.073603+0000","last_peered":"2026-02-24T22:33:45.073603+0000","last_clean":"2026-02-24T22:33:45.073603+0000","last_became_active":"2026-02-24T22:05:18.190114+0000","last_became_peered":"2026-02-24T22:05:18.190114+0000","last_unstale":"2026-02-24T22:33:45.073603+0000","last_undegraded":"2026-02-24T22:33:45.073603+0000","last_fullsized":"2026-02-24T22:33:45.073603+0000","mapping_epoch":24,"log_start":"312'4100","ondisk_log_start":"312'4100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":3000,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:58:28.766462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":16758,"num_read_kb":13963,"num_write":6320,"num_write_kb":3135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"312'194","reported_seq":915,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:57.691183+0000","last_change":"2026-02-24T22:05:16.189046+0000","last_active":"2026-02-24T22:33:57.691183+0000","last_peered":"2026-02-24T22:33:57.691183+0000","last_clean":"2026-02-24T22:33:57.691183+0000","last_became_active":"2026-02-24T22:05:16.188630+0000","last_became_peered":"2026-02-24T22:05:16.188630+0000","last_unstale":"2026-02-24T22:33:57.691183+0000","last_undegraded":"2026-02-24T22:33:57.691183+0000","last_fullsized":"2026-02-24T22:33:57.691183+0000","mapping_epoch":22,"log_start":"312'100","ondisk_log_start":"312'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":100,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:15:02.436674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11535007,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":414,"num_read_kb":33751,"num_write":1015,"num_write_kb":13009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":588,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.723682+0000","last_change":"2026-02-24T22:05:20.188706+0000","last_active":"2026-02-24T22:30:59.723682+0000","last_peered":"2026-02-24T22:30:59.723682+0000","last_clean":"2026-02-24T22:30:59.723682+0000","last_became_active":"2026-02-24T22:05:20.188330+0000","last_became_peered":"2026-02-24T22:05:20.188330+0000","last_unstale":"2026-02-24T22:30:59.723682+0000","last_undegraded":"2026-02-24T22:30:59.723682+0000","last_fullsized":"2026-02-24T22:30:59.723682+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:07:09.639326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"312'681","reported_seq":1757,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.538908+0000","last_change":"2026-02-24T22:05:22.192098+0000","last_active":"2026-02-24T22:34:18.538908+0000","last_peered":"2026-02-24T22:34:18.538908+0000","last_clean":"2026-02-24T22:34:18.538908+0000","last_became_active":"2026-02-24T22:05:22.191584+0000","last_became_peered":"2026-02-24T22:05:22.191584+0000","last_unstale":"2026-02-24T22:34:18.538908+0000","last_undegraded":"2026-02-24T22:34:18.538908+0000","last_fullsized":"2026-02-24T22:34:18.538908+0000","mapping_epoch":28,"log_start":"312'600","ondisk_log_start":"312'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":600,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:55:24.433104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50841,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2209,"num_read_kb":1670,"num_write":316,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"312'161","reported_seq":768,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:55.998434+0000","last_change":"2026-02-24T22:05:24.199098+0000","last_active":"2026-02-24T22:33:55.998434+0000","last_peered":"2026-02-24T22:33:55.998434+0000","last_clean":"2026-02-24T22:33:55.998434+0000","last_became_active":"2026-02-24T22:05:24.198821+0000","last_became_peered":"2026-02-24T22:05:24.198821+0000","last_unstale":"2026-02-24T22:33:55.998434+0000","last_undegraded":"2026-02-24T22:33:55.998434+0000","last_fullsized":"2026-02-24T22:33:55.998434+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":100,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:31:53.831339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28978,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":162,"num_read_kb":145,"num_write":411,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":578,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.732230+0000","last_change":"2026-02-24T22:05:29.653213+0000","last_active":"2026-02-24T22:31:34.732230+0000","last_peered":"2026-02-24T22:31:34.732230+0000","last_clean":"2026-02-24T22:31:34.732230+0000","last_became_active":"2026-02-24T22:05:29.652757+0000","last_became_peered":"2026-02-24T22:05:29.652757+0000","last_unstale":"2026-02-24T22:31:34.732230+0000","last_undegraded":"2026-02-24T22:31:34.732230+0000","last_fullsized":"2026-02-24T22:31:34.732230+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:57:03.928835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"312'4231","reported_seq":4978,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:52.784970+0000","last_change":"2026-02-24T22:05:18.184303+0000","last_active":"2026-02-24T22:33:52.784970+0000","last_peered":"2026-02-24T22:33:52.784970+0000","last_clean":"2026-02-24T22:33:52.784970+0000","last_became_active":"2026-02-24T22:05:18.183983+0000","last_became_peered":"2026-02-24T22:05:18.183983+0000","last_unstale":"2026-02-24T22:33:52.784970+0000","last_undegraded":"2026-02-24T22:33:52.784970+0000","last_fullsized":"2026-02-24T22:33:52.784970+0000","mapping_epoch":24,"log_start":"312'4200","ondisk_log_start":"312'4200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":3000,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:36:58.437134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":16912,"num_read_kb":14102,"num_write":6331,"num_write_kb":3151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"312'232","reported_seq":974,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:04.712850+0000","last_change":"2026-02-24T22:05:16.193213+0000","last_active":"2026-02-24T22:34:04.712850+0000","last_peered":"2026-02-24T22:34:04.712850+0000","last_clean":"2026-02-24T22:34:04.712850+0000","last_became_active":"2026-02-24T22:05:16.192845+0000","last_became_peered":"2026-02-24T22:05:16.192845+0000","last_unstale":"2026-02-24T22:34:04.712850+0000","last_undegraded":"2026-02-24T22:34:04.712850+0000","last_fullsized":"2026-02-24T22:34:04.712850+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":200,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:53:53.655783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933180,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":413,"num_read_kb":4336,"num_write":1200,"num_write_kb":2676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.728520+0000","last_change":"2026-02-24T22:05:20.189035+0000","last_active":"2026-02-24T22:30:59.728520+0000","last_peered":"2026-02-24T22:30:59.728520+0000","last_clean":"2026-02-24T22:30:59.728520+0000","last_became_active":"2026-02-24T22:05:20.188790+0000","last_became_peered":"2026-02-24T22:05:20.188790+0000","last_unstale":"2026-02-24T22:30:59.728520+0000","last_undegraded":"2026-02-24T22:30:59.728520+0000","last_fullsized":"2026-02-24T22:30:59.728520+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:29:04.813791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"312'25","reported_seq":615,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.737881+0000","last_change":"2026-02-24T22:05:22.192740+0000","last_active":"2026-02-24T22:31:04.737881+0000","last_peered":"2026-02-24T22:31:04.737881+0000","last_clean":"2026-02-24T22:31:04.737881+0000","last_became_active":"2026-02-24T22:05:22.192560+0000","last_became_peered":"2026-02-24T22:05:22.192560+0000","last_unstale":"2026-02-24T22:31:04.737881+0000","last_undegraded":"2026-02-24T22:31:04.737881+0000","last_fullsized":"2026-02-24T22:31:04.737881+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:40:42.414311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"312'163","reported_seq":746,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.531878+0000","last_change":"2026-02-24T22:05:24.205761+0000","last_active":"2026-02-24T22:34:18.531878+0000","last_peered":"2026-02-24T22:34:18.531878+0000","last_clean":"2026-02-24T22:34:18.531878+0000","last_became_active":"2026-02-24T22:05:24.205506+0000","last_became_peered":"2026-02-24T22:05:24.205506+0000","last_unstale":"2026-02-24T22:34:18.531878+0000","last_undegraded":"2026-02-24T22:34:18.531878+0000","last_fullsized":"2026-02-24T22:34:18.531878+0000","mapping_epoch":30,"log_start":"301'100","ondisk_log_start":"301'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":100,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:22:32.827602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30843,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":90,"num_read_kb":84,"num_write":389,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":573,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.734200+0000","last_change":"2026-02-24T22:05:29.653324+0000","last_active":"2026-02-24T22:30:29.734200+0000","last_peered":"2026-02-24T22:30:29.734200+0000","last_clean":"2026-02-24T22:30:29.734200+0000","last_became_active":"2026-02-24T22:05:29.652987+0000","last_became_peered":"2026-02-24T22:05:29.652987+0000","last_unstale":"2026-02-24T22:30:29.734200+0000","last_undegraded":"2026-02-24T22:30:29.734200+0000","last_fullsized":"2026-02-24T22:30:29.734200+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:58:50.918939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"312'187","reported_seq":884,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.550400+0000","last_change":"2026-02-24T22:05:18.186488+0000","last_active":"2026-02-24T22:34:18.550400+0000","last_peered":"2026-02-24T22:34:18.550400+0000","last_clean":"2026-02-24T22:34:18.550400+0000","last_became_active":"2026-02-24T22:05:18.186069+0000","last_became_peered":"2026-02-24T22:05:18.186069+0000","last_unstale":"2026-02-24T22:34:18.550400+0000","last_undegraded":"2026-02-24T22:34:18.550400+0000","last_fullsized":"2026-02-24T22:34:18.550400+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:26:51.294901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":691,"num_read_kb":561,"num_write":267,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"312'221","reported_seq":903,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:58.398587+0000","last_change":"2026-02-24T22:06:00.662484+0000","last_active":"2026-02-24T22:30:58.398587+0000","last_peered":"2026-02-24T22:30:58.398587+0000","last_clean":"2026-02-24T22:30:58.398587+0000","last_became_active":"2026-02-24T22:06:00.662291+0000","last_became_peered":"2026-02-24T22:06:00.662291+0000","last_unstale":"2026-02-24T22:30:58.398587+0000","last_undegraded":"2026-02-24T22:30:58.398587+0000","last_fullsized":"2026-02-24T22:30:58.398587+0000","mapping_epoch":36,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:44:04.240829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5088,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":359,"num_read_kb":357,"num_write":1312,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":585,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.739650+0000","last_change":"2026-02-24T22:05:20.194668+0000","last_active":"2026-02-24T22:30:59.739650+0000","last_peered":"2026-02-24T22:30:59.739650+0000","last_clean":"2026-02-24T22:30:59.739650+0000","last_became_active":"2026-02-24T22:05:20.194215+0000","last_became_peered":"2026-02-24T22:05:20.194215+0000","last_unstale":"2026-02-24T22:30:59.739650+0000","last_undegraded":"2026-02-24T22:30:59.739650+0000","last_fullsized":"2026-02-24T22:30:59.739650+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:55:09.941150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"312'470","reported_seq":1327,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.519070+0000","last_change":"2026-02-24T22:05:22.192102+0000","last_active":"2026-02-24T22:34:18.519070+0000","last_peered":"2026-02-24T22:34:18.519070+0000","last_clean":"2026-02-24T22:34:18.519070+0000","last_became_active":"2026-02-24T22:05:22.191675+0000","last_became_peered":"2026-02-24T22:05:22.191675+0000","last_unstale":"2026-02-24T22:34:18.519070+0000","last_undegraded":"2026-02-24T22:34:18.519070+0000","last_fullsized":"2026-02-24T22:34:18.519070+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":400,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:50:42.591507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53362,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1312,"num_read_kb":1248,"num_write":339,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"312'249","reported_seq":1278,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.564141+0000","last_change":"2026-02-24T22:05:24.207967+0000","last_active":"2026-02-24T22:34:17.564141+0000","last_peered":"2026-02-24T22:34:17.564141+0000","last_clean":"2026-02-24T22:34:17.564141+0000","last_became_active":"2026-02-24T22:05:24.207479+0000","last_became_peered":"2026-02-24T22:05:24.207479+0000","last_unstale":"2026-02-24T22:34:17.564141+0000","last_undegraded":"2026-02-24T22:34:17.564141+0000","last_fullsized":"2026-02-24T22:34:17.564141+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":200,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:21:33.217485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31962,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":675,"num_read_kb":669,"num_write":683,"num_write_kb":207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":564,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.735513+0000","last_change":"2026-02-24T22:05:29.661767+0000","last_active":"2026-02-24T22:30:29.735513+0000","last_peered":"2026-02-24T22:30:29.735513+0000","last_clean":"2026-02-24T22:30:29.735513+0000","last_became_active":"2026-02-24T22:05:29.661248+0000","last_became_peered":"2026-02-24T22:05:29.661248+0000","last_unstale":"2026-02-24T22:30:29.735513+0000","last_undegraded":"2026-02-24T22:30:29.735513+0000","last_fullsized":"2026-02-24T22:30:29.735513+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:30:06.329202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"312'116","reported_seq":824,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:06.481608+0000","last_change":"2026-02-24T22:05:18.187722+0000","last_active":"2026-02-24T22:34:06.481608+0000","last_peered":"2026-02-24T22:34:06.481608+0000","last_clean":"2026-02-24T22:34:06.481608+0000","last_became_active":"2026-02-24T22:05:18.184982+0000","last_became_peered":"2026-02-24T22:05:18.184982+0000","last_unstale":"2026-02-24T22:34:06.481608+0000","last_undegraded":"2026-02-24T22:34:06.481608+0000","last_fullsized":"2026-02-24T22:34:06.481608+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":100,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:08:28.087403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":444,"num_read_kb":381,"num_write":158,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"312'213","reported_seq":975,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:02.205401+0000","last_change":"2026-02-24T22:05:16.184926+0000","last_active":"2026-02-24T22:34:02.205401+0000","last_peered":"2026-02-24T22:34:02.205401+0000","last_clean":"2026-02-24T22:34:02.205401+0000","last_became_active":"2026-02-24T22:05:16.184594+0000","last_became_peered":"2026-02-24T22:05:16.184594+0000","last_unstale":"2026-02-24T22:34:02.205401+0000","last_undegraded":"2026-02-24T22:34:02.205401+0000","last_fullsized":"2026-02-24T22:34:02.205401+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":200,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:53:18.748033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6374454,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":456,"num_read_kb":28116,"num_write":1131,"num_write_kb":6289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":583,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.735449+0000","last_change":"2026-02-24T22:05:20.193777+0000","last_active":"2026-02-24T22:30:59.735449+0000","last_peered":"2026-02-24T22:30:59.735449+0000","last_clean":"2026-02-24T22:30:59.735449+0000","last_became_active":"2026-02-24T22:05:20.193522+0000","last_became_peered":"2026-02-24T22:05:20.193522+0000","last_unstale":"2026-02-24T22:30:59.735449+0000","last_undegraded":"2026-02-24T22:30:59.735449+0000","last_fullsized":"2026-02-24T22:30:59.735449+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:18:03.275177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"312'428","reported_seq":1261,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.513000+0000","last_change":"2026-02-24T22:05:22.200284+0000","last_active":"2026-02-24T22:34:18.513000+0000","last_peered":"2026-02-24T22:34:18.513000+0000","last_clean":"2026-02-24T22:34:18.513000+0000","last_became_active":"2026-02-24T22:05:22.199968+0000","last_became_peered":"2026-02-24T22:05:22.199968+0000","last_unstale":"2026-02-24T22:34:18.513000+0000","last_undegraded":"2026-02-24T22:34:18.513000+0000","last_fullsized":"2026-02-24T22:34:18.513000+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":400,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:25:54.291884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51783,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1258,"num_read_kb":1192,"num_write":305,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"312'212","reported_seq":808,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:04.178415+0000","last_change":"2026-02-24T22:05:24.200305+0000","last_active":"2026-02-24T22:34:04.178415+0000","last_peered":"2026-02-24T22:34:04.178415+0000","last_clean":"2026-02-24T22:34:04.178415+0000","last_became_active":"2026-02-24T22:05:24.199650+0000","last_became_peered":"2026-02-24T22:05:24.199650+0000","last_unstale":"2026-02-24T22:34:04.178415+0000","last_undegraded":"2026-02-24T22:34:04.178415+0000","last_fullsized":"2026-02-24T22:34:04.178415+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:33:18.336086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29523,"num_objects":91,"num_object_clones":0,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":182,"num_read_kb":193,"num_write":488,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"312'249","reported_seq":958,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:44.105284+0000","last_change":"2026-02-24T22:05:18.190553+0000","last_active":"2026-02-24T22:33:44.105284+0000","last_peered":"2026-02-24T22:33:44.105284+0000","last_clean":"2026-02-24T22:33:44.105284+0000","last_became_active":"2026-02-24T22:05:18.190238+0000","last_became_peered":"2026-02-24T22:05:18.190238+0000","last_unstale":"2026-02-24T22:33:44.105284+0000","last_undegraded":"2026-02-24T22:33:44.105284+0000","last_fullsized":"2026-02-24T22:33:44.105284+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":200,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:04:44.616609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":976,"num_read_kb":794,"num_write":353,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"312'218","reported_seq":1574,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:05.712936+0000","last_change":"2026-02-24T22:05:16.185696+0000","last_active":"2026-02-24T22:34:05.712936+0000","last_peered":"2026-02-24T22:34:05.712936+0000","last_clean":"2026-02-24T22:34:05.712936+0000","last_became_active":"2026-02-24T22:05:16.184415+0000","last_became_peered":"2026-02-24T22:05:16.184415+0000","last_unstale":"2026-02-24T22:34:05.712936+0000","last_undegraded":"2026-02-24T22:34:05.712936+0000","last_fullsized":"2026-02-24T22:34:05.712936+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":200,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:10:18.950972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5603567,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":2274,"num_read_kb":23832,"num_write":1037,"num_write_kb":5557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.740195+0000","last_change":"2026-02-24T22:05:20.186817+0000","last_active":"2026-02-24T22:31:24.740195+0000","last_peered":"2026-02-24T22:31:24.740195+0000","last_clean":"2026-02-24T22:31:24.740195+0000","last_became_active":"2026-02-24T22:05:20.186325+0000","last_became_peered":"2026-02-24T22:05:20.186325+0000","last_unstale":"2026-02-24T22:31:24.740195+0000","last_undegraded":"2026-02-24T22:31:24.740195+0000","last_fullsized":"2026-02-24T22:31:24.740195+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:45:07.507222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"312'399","reported_seq":1007,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.383952+0000","last_change":"2026-02-24T22:05:22.192972+0000","last_active":"2026-02-24T22:34:18.383952+0000","last_peered":"2026-02-24T22:34:18.383952+0000","last_clean":"2026-02-24T22:34:18.383952+0000","last_became_active":"2026-02-24T22:05:22.192756+0000","last_became_peered":"2026-02-24T22:05:22.192756+0000","last_unstale":"2026-02-24T22:34:18.383952+0000","last_undegraded":"2026-02-24T22:34:18.383952+0000","last_fullsized":"2026-02-24T22:34:18.383952+0000","mapping_epoch":28,"log_start":"312'300","ondisk_log_start":"312'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":300,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:43:14.513261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115669,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":689,"num_read_kb":1585,"num_write":692,"num_write_kb":688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"312'187","reported_seq":796,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.233131+0000","last_change":"2026-02-24T22:05:24.206653+0000","last_active":"2026-02-24T22:34:17.233131+0000","last_peered":"2026-02-24T22:34:17.233131+0000","last_clean":"2026-02-24T22:34:17.233131+0000","last_became_active":"2026-02-24T22:05:24.206407+0000","last_became_peered":"2026-02-24T22:05:24.206407+0000","last_unstale":"2026-02-24T22:34:17.233131+0000","last_undegraded":"2026-02-24T22:34:17.233131+0000","last_fullsized":"2026-02-24T22:34:17.233131+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:49:11.388315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30298,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":168,"num_read_kb":155,"num_write":455,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":576,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.729573+0000","last_change":"2026-02-24T22:05:29.650326+0000","last_active":"2026-02-24T22:31:09.729573+0000","last_peered":"2026-02-24T22:31:09.729573+0000","last_clean":"2026-02-24T22:31:09.729573+0000","last_became_active":"2026-02-24T22:05:29.650115+0000","last_became_peered":"2026-02-24T22:05:29.650115+0000","last_unstale":"2026-02-24T22:31:09.729573+0000","last_undegraded":"2026-02-24T22:31:09.729573+0000","last_fullsized":"2026-02-24T22:31:09.729573+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:38:20.581425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"312'162","reported_seq":873,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.740186+0000","last_change":"2026-02-24T22:05:18.183744+0000","last_active":"2026-02-24T22:31:24.740186+0000","last_peered":"2026-02-24T22:31:24.740186+0000","last_clean":"2026-02-24T22:31:24.740186+0000","last_became_active":"2026-02-24T22:05:18.183501+0000","last_became_peered":"2026-02-24T22:05:18.183501+0000","last_unstale":"2026-02-24T22:31:24.740186+0000","last_undegraded":"2026-02-24T22:31:24.740186+0000","last_fullsized":"2026-02-24T22:31:24.740186+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":100,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:10:06.918155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":578,"num_read_kb":488,"num_write":224,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"312'343","reported_seq":1285,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:38.575992+0000","last_change":"2026-02-24T22:05:16.183028+0000","last_active":"2026-02-24T22:33:38.575992+0000","last_peered":"2026-02-24T22:33:38.575992+0000","last_clean":"2026-02-24T22:33:38.575992+0000","last_became_active":"2026-02-24T22:05:16.182909+0000","last_became_peered":"2026-02-24T22:05:16.182909+0000","last_unstale":"2026-02-24T22:33:38.575992+0000","last_undegraded":"2026-02-24T22:33:38.575992+0000","last_fullsized":"2026-02-24T22:33:38.575992+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:01:15.081477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":445956,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":1143,"num_read_kb":9242,"num_write":1547,"num_write_kb":11784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":584,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.730971+0000","last_change":"2026-02-24T22:05:20.194023+0000","last_active":"2026-02-24T22:30:19.730971+0000","last_peered":"2026-02-24T22:30:19.730971+0000","last_clean":"2026-02-24T22:30:19.730971+0000","last_became_active":"2026-02-24T22:05:20.193751+0000","last_became_peered":"2026-02-24T22:05:20.193751+0000","last_unstale":"2026-02-24T22:30:19.730971+0000","last_undegraded":"2026-02-24T22:30:19.730971+0000","last_fullsized":"2026-02-24T22:30:19.730971+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:48:18.314998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"312'737","reported_seq":1813,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.525758+0000","last_change":"2026-02-24T22:05:22.200836+0000","last_active":"2026-02-24T22:34:18.525758+0000","last_peered":"2026-02-24T22:34:18.525758+0000","last_clean":"2026-02-24T22:34:18.525758+0000","last_became_active":"2026-02-24T22:05:22.200603+0000","last_became_peered":"2026-02-24T22:05:22.200603+0000","last_unstale":"2026-02-24T22:34:18.525758+0000","last_undegraded":"2026-02-24T22:34:18.525758+0000","last_fullsized":"2026-02-24T22:34:18.525758+0000","mapping_epoch":28,"log_start":"312'700","ondisk_log_start":"312'700","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":700,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:36:03.813153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63999,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":2312,"num_read_kb":1868,"num_write":428,"num_write_kb":371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"312'334","reported_seq":964,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.243815+0000","last_change":"2026-02-24T22:05:24.201087+0000","last_active":"2026-02-24T22:34:18.243815+0000","last_peered":"2026-02-24T22:34:18.243815+0000","last_clean":"2026-02-24T22:34:18.243815+0000","last_became_active":"2026-02-24T22:05:24.200391+0000","last_became_peered":"2026-02-24T22:05:24.200391+0000","last_unstale":"2026-02-24T22:34:18.243815+0000","last_undegraded":"2026-02-24T22:34:18.243815+0000","last_fullsized":"2026-02-24T22:34:18.243815+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:18:42.256342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31847,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":370,"num_read_kb":353,"num_write":777,"num_write_kb":271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"312'81","reported_seq":994,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:14.669283+0000","last_change":"2026-02-24T22:05:29.660890+0000","last_active":"2026-02-24T22:34:14.669283+0000","last_peered":"2026-02-24T22:34:14.669283+0000","last_clean":"2026-02-24T22:34:14.669283+0000","last_became_active":"2026-02-24T22:05:29.660289+0000","last_became_peered":"2026-02-24T22:05:29.660289+0000","last_unstale":"2026-02-24T22:34:14.669283+0000","last_undegraded":"2026-02-24T22:34:14.669283+0000","last_fullsized":"2026-02-24T22:34:14.669283+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:13:08.310592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"312'100","reported_seq":769,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:54.429078+0000","last_change":"2026-02-24T22:05:18.185312+0000","last_active":"2026-02-24T22:33:54.429078+0000","last_peered":"2026-02-24T22:33:54.429078+0000","last_clean":"2026-02-24T22:33:54.429078+0000","last_became_active":"2026-02-24T22:05:18.184745+0000","last_became_peered":"2026-02-24T22:05:18.184745+0000","last_unstale":"2026-02-24T22:33:54.429078+0000","last_undegraded":"2026-02-24T22:33:54.429078+0000","last_fullsized":"2026-02-24T22:33:54.429078+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:08:34.210048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":371,"num_read_kb":307,"num_write":132,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"312'221","reported_seq":941,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:55.590300+0000","last_change":"2026-02-24T22:05:16.185859+0000","last_active":"2026-02-24T22:30:55.590300+0000","last_peered":"2026-02-24T22:30:55.590300+0000","last_clean":"2026-02-24T22:30:55.590300+0000","last_became_active":"2026-02-24T22:05:16.185574+0000","last_became_peered":"2026-02-24T22:05:16.185574+0000","last_unstale":"2026-02-24T22:30:55.590300+0000","last_undegraded":"2026-02-24T22:30:55.590300+0000","last_fullsized":"2026-02-24T22:30:55.590300+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:09:04.869683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6081138,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":366,"num_read_kb":30007,"num_write":1195,"num_write_kb":21358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"32'3","reported_seq":830,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.335841+0000","last_change":"2026-02-24T22:05:20.192991+0000","last_active":"2026-02-24T22:34:18.335841+0000","last_peered":"2026-02-24T22:34:18.335841+0000","last_clean":"2026-02-24T22:34:18.335841+0000","last_became_active":"2026-02-24T22:05:20.192612+0000","last_became_peered":"2026-02-24T22:05:20.192612+0000","last_unstale":"2026-02-24T22:34:18.335841+0000","last_undegraded":"2026-02-24T22:34:18.335841+0000","last_fullsized":"2026-02-24T22:34:18.335841+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:32:11.097665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":482,"num_read_kb":480,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"312'199","reported_seq":806,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.845514+0000","last_change":"2026-02-24T22:05:22.906226+0000","last_active":"2026-02-24T22:34:17.845514+0000","last_peered":"2026-02-24T22:34:17.845514+0000","last_clean":"2026-02-24T22:34:17.845514+0000","last_became_active":"2026-02-24T22:05:22.906047+0000","last_became_peered":"2026-02-24T22:05:22.906047+0000","last_unstale":"2026-02-24T22:34:17.845514+0000","last_undegraded":"2026-02-24T22:34:17.845514+0000","last_fullsized":"2026-02-24T22:34:17.845514+0000","mapping_epoch":28,"log_start":"312'100","ondisk_log_start":"312'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:36:51.410405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50386,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":315,"num_read_kb":704,"num_write":294,"num_write_kb":265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"312'164","reported_seq":759,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:58.164024+0000","last_change":"2026-02-24T22:05:24.201114+0000","last_active":"2026-02-24T22:33:58.164024+0000","last_peered":"2026-02-24T22:33:58.164024+0000","last_clean":"2026-02-24T22:33:58.164024+0000","last_became_active":"2026-02-24T22:05:24.200747+0000","last_became_peered":"2026-02-24T22:05:24.200747+0000","last_unstale":"2026-02-24T22:33:58.164024+0000","last_undegraded":"2026-02-24T22:33:58.164024+0000","last_fullsized":"2026-02-24T22:33:58.164024+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:27:56.881747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30873,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":120,"num_read_kb":110,"num_write":405,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":565,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.740173+0000","last_change":"2026-02-24T22:05:29.660794+0000","last_active":"2026-02-24T22:31:09.740173+0000","last_peered":"2026-02-24T22:31:09.740173+0000","last_clean":"2026-02-24T22:31:09.740173+0000","last_became_active":"2026-02-24T22:05:29.660344+0000","last_became_peered":"2026-02-24T22:05:29.660344+0000","last_unstale":"2026-02-24T22:31:09.740173+0000","last_undegraded":"2026-02-24T22:31:09.740173+0000","last_fullsized":"2026-02-24T22:31:09.740173+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:36:46.842669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"312'157","reported_seq":828,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.716442+0000","last_change":"2026-02-24T22:06:00.759389+0000","last_active":"2026-02-24T22:30:19.716442+0000","last_peered":"2026-02-24T22:30:19.716442+0000","last_clean":"2026-02-24T22:30:19.716442+0000","last_became_active":"2026-02-24T22:06:00.759215+0000","last_became_peered":"2026-02-24T22:06:00.759215+0000","last_unstale":"2026-02-24T22:30:19.716442+0000","last_undegraded":"2026-02-24T22:30:19.716442+0000","last_fullsized":"2026-02-24T22:30:19.716442+0000","mapping_epoch":36,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:14:31.570934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":584,"num_read_kb":483,"num_write":222,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"312'212","reported_seq":963,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:59.428345+0000","last_change":"2026-02-24T22:05:16.184388+0000","last_active":"2026-02-24T22:33:59.428345+0000","last_peered":"2026-02-24T22:33:59.428345+0000","last_clean":"2026-02-24T22:33:59.428345+0000","last_became_active":"2026-02-24T22:05:16.183956+0000","last_became_peered":"2026-02-24T22:05:16.183956+0000","last_unstale":"2026-02-24T22:33:59.428345+0000","last_undegraded":"2026-02-24T22:33:59.428345+0000","last_fullsized":"2026-02-24T22:33:59.428345+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:01:46.056052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5448977,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":524,"num_read_kb":6430,"num_write":1216,"num_write_kb":5371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":587,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.719728+0000","last_change":"2026-02-24T22:05:20.187960+0000","last_active":"2026-02-24T22:30:19.719728+0000","last_peered":"2026-02-24T22:30:19.719728+0000","last_clean":"2026-02-24T22:30:19.719728+0000","last_became_active":"2026-02-24T22:05:20.187316+0000","last_became_peered":"2026-02-24T22:05:20.187316+0000","last_unstale":"2026-02-24T22:30:19.719728+0000","last_undegraded":"2026-02-24T22:30:19.719728+0000","last_fullsized":"2026-02-24T22:30:19.719728+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:07:22.260792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"312'287","reported_seq":1125,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.509524+0000","last_change":"2026-02-24T22:05:22.912321+0000","last_active":"2026-02-24T22:34:18.509524+0000","last_peered":"2026-02-24T22:34:18.509524+0000","last_clean":"2026-02-24T22:34:18.509524+0000","last_became_active":"2026-02-24T22:05:22.912087+0000","last_became_peered":"2026-02-24T22:05:22.912087+0000","last_unstale":"2026-02-24T22:34:18.509524+0000","last_undegraded":"2026-02-24T22:34:18.509524+0000","last_fullsized":"2026-02-24T22:34:18.509524+0000","mapping_epoch":28,"log_start":"307'200","ondisk_log_start":"307'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":200,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:16:48.283329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":152,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":996,"num_read_kb":518,"num_write":51,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"312'178","reported_seq":844,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.269129+0000","last_change":"2026-02-24T22:05:24.207156+0000","last_active":"2026-02-24T22:34:15.269129+0000","last_peered":"2026-02-24T22:34:15.269129+0000","last_clean":"2026-02-24T22:34:15.269129+0000","last_became_active":"2026-02-24T22:05:24.206383+0000","last_became_peered":"2026-02-24T22:05:24.206383+0000","last_unstale":"2026-02-24T22:34:15.269129+0000","last_undegraded":"2026-02-24T22:34:15.269129+0000","last_fullsized":"2026-02-24T22:34:15.269129+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":100,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:12:00.918186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29475,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":255,"num_read_kb":237,"num_write":434,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":571,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.740948+0000","last_change":"2026-02-24T22:05:29.661761+0000","last_active":"2026-02-24T22:31:09.740948+0000","last_peered":"2026-02-24T22:31:09.740948+0000","last_clean":"2026-02-24T22:31:09.740948+0000","last_became_active":"2026-02-24T22:05:29.661539+0000","last_became_peered":"2026-02-24T22:05:29.661539+0000","last_unstale":"2026-02-24T22:31:09.740948+0000","last_undegraded":"2026-02-24T22:31:09.740948+0000","last_fullsized":"2026-02-24T22:31:09.740948+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:02:56.515020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"312'178","reported_seq":849,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.732694+0000","last_change":"2026-02-24T22:05:18.193080+0000","last_active":"2026-02-24T22:30:19.732694+0000","last_peered":"2026-02-24T22:30:19.732694+0000","last_clean":"2026-02-24T22:30:19.732694+0000","last_became_active":"2026-02-24T22:05:18.192487+0000","last_became_peered":"2026-02-24T22:05:18.192487+0000","last_unstale":"2026-02-24T22:30:19.732694+0000","last_undegraded":"2026-02-24T22:30:19.732694+0000","last_fullsized":"2026-02-24T22:30:19.732694+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":100,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:11:58.202360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":681,"num_read_kb":551,"num_write":266,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"312'225","reported_seq":924,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.242241+0000","last_change":"2026-02-24T22:05:16.190846+0000","last_active":"2026-02-24T22:34:17.242241+0000","last_peered":"2026-02-24T22:34:17.242241+0000","last_clean":"2026-02-24T22:34:17.242241+0000","last_became_active":"2026-02-24T22:05:16.190108+0000","last_became_peered":"2026-02-24T22:05:16.190108+0000","last_unstale":"2026-02-24T22:34:17.242241+0000","last_undegraded":"2026-02-24T22:34:17.242241+0000","last_fullsized":"2026-02-24T22:34:17.242241+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":200,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:05:24.226362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10513634,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":400,"num_read_kb":15743,"num_write":1275,"num_write_kb":10328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":584,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.735341+0000","last_change":"2026-02-24T22:05:20.194434+0000","last_active":"2026-02-24T22:30:19.735341+0000","last_peered":"2026-02-24T22:30:19.735341+0000","last_clean":"2026-02-24T22:30:19.735341+0000","last_became_active":"2026-02-24T22:05:20.194183+0000","last_became_peered":"2026-02-24T22:05:20.194183+0000","last_unstale":"2026-02-24T22:30:19.735341+0000","last_undegraded":"2026-02-24T22:30:19.735341+0000","last_fullsized":"2026-02-24T22:30:19.735341+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:11:03.103844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"312'706","reported_seq":1776,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.522544+0000","last_change":"2026-02-24T22:05:22.199767+0000","last_active":"2026-02-24T22:34:18.522544+0000","last_peered":"2026-02-24T22:34:18.522544+0000","last_clean":"2026-02-24T22:34:18.522544+0000","last_became_active":"2026-02-24T22:05:22.199480+0000","last_became_peered":"2026-02-24T22:05:22.199480+0000","last_unstale":"2026-02-24T22:34:18.522544+0000","last_undegraded":"2026-02-24T22:34:18.522544+0000","last_fullsized":"2026-02-24T22:34:18.522544+0000","mapping_epoch":28,"log_start":"312'600","ondisk_log_start":"312'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":600,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:44:12.736876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56130,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":2245,"num_read_kb":1729,"num_write":346,"num_write_kb":294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"312'198","reported_seq":804,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.239318+0000","last_change":"2026-02-24T22:05:24.201028+0000","last_active":"2026-02-24T22:34:15.239318+0000","last_peered":"2026-02-24T22:34:15.239318+0000","last_clean":"2026-02-24T22:34:15.239318+0000","last_became_active":"2026-02-24T22:05:24.200434+0000","last_became_peered":"2026-02-24T22:05:24.200434+0000","last_unstale":"2026-02-24T22:34:15.239318+0000","last_undegraded":"2026-02-24T22:34:15.239318+0000","last_fullsized":"2026-02-24T22:34:15.239318+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":100,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:38:49.379078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27134,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":189,"num_read_kb":178,"num_write":476,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":570,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:39.731065+0000","last_change":"2026-02-24T22:05:29.662687+0000","last_active":"2026-02-24T22:29:39.731065+0000","last_peered":"2026-02-24T22:29:39.731065+0000","last_clean":"2026-02-24T22:29:39.731065+0000","last_became_active":"2026-02-24T22:05:29.662425+0000","last_became_peered":"2026-02-24T22:05:29.662425+0000","last_unstale":"2026-02-24T22:29:39.731065+0000","last_undegraded":"2026-02-24T22:29:39.731065+0000","last_fullsized":"2026-02-24T22:29:39.731065+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:37:43.066587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"312'132","reported_seq":810,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.732676+0000","last_change":"2026-02-24T22:05:18.193443+0000","last_active":"2026-02-24T22:30:19.732676+0000","last_peered":"2026-02-24T22:30:19.732676+0000","last_clean":"2026-02-24T22:30:19.732676+0000","last_became_active":"2026-02-24T22:05:18.191664+0000","last_became_peered":"2026-02-24T22:05:18.191664+0000","last_unstale":"2026-02-24T22:30:19.732676+0000","last_undegraded":"2026-02-24T22:30:19.732676+0000","last_fullsized":"2026-02-24T22:30:19.732676+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":100,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:59:22.441702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":463,"num_read_kb":379,"num_write":198,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"312'212","reported_seq":907,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:55.742771+0000","last_change":"2026-02-24T22:05:16.189190+0000","last_active":"2026-02-24T22:33:55.742771+0000","last_peered":"2026-02-24T22:33:55.742771+0000","last_clean":"2026-02-24T22:33:55.742771+0000","last_became_active":"2026-02-24T22:05:16.188154+0000","last_became_peered":"2026-02-24T22:05:16.188154+0000","last_unstale":"2026-02-24T22:33:55.742771+0000","last_undegraded":"2026-02-24T22:33:55.742771+0000","last_fullsized":"2026-02-24T22:33:55.742771+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:45:59.561385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65745,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":374,"num_read_kb":2459,"num_write":1169,"num_write_kb":1814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"32'5","reported_seq":820,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.334772+0000","last_change":"2026-02-24T22:05:20.195455+0000","last_active":"2026-02-24T22:34:18.334772+0000","last_peered":"2026-02-24T22:34:18.334772+0000","last_clean":"2026-02-24T22:34:18.334772+0000","last_became_active":"2026-02-24T22:05:20.195202+0000","last_became_peered":"2026-02-24T22:05:20.195202+0000","last_unstale":"2026-02-24T22:34:18.334772+0000","last_undegraded":"2026-02-24T22:34:18.334772+0000","last_fullsized":"2026-02-24T22:34:18.334772+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:34:18.002332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1242,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":486,"num_read_kb":719,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"312'680","reported_seq":1751,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.538422+0000","last_change":"2026-02-24T22:05:22.192219+0000","last_active":"2026-02-24T22:34:18.538422+0000","last_peered":"2026-02-24T22:34:18.538422+0000","last_clean":"2026-02-24T22:34:18.538422+0000","last_became_active":"2026-02-24T22:05:22.192035+0000","last_became_peered":"2026-02-24T22:05:22.192035+0000","last_unstale":"2026-02-24T22:34:18.538422+0000","last_undegraded":"2026-02-24T22:34:18.538422+0000","last_fullsized":"2026-02-24T22:34:18.538422+0000","mapping_epoch":28,"log_start":"312'600","ondisk_log_start":"312'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":600,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:05:53.268094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50571,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2200,"num_read_kb":1666,"num_write":310,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"312'2477","reported_seq":5136,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.218480+0000","last_change":"2026-02-24T22:05:24.206516+0000","last_active":"2026-02-24T22:34:18.218480+0000","last_peered":"2026-02-24T22:34:18.218480+0000","last_clean":"2026-02-24T22:34:18.218480+0000","last_became_active":"2026-02-24T22:05:24.205557+0000","last_became_peered":"2026-02-24T22:05:24.205557+0000","last_unstale":"2026-02-24T22:34:18.218480+0000","last_undegraded":"2026-02-24T22:34:18.218480+0000","last_fullsized":"2026-02-24T22:34:18.218480+0000","mapping_epoch":30,"log_start":"312'2400","ondisk_log_start":"312'2400","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":2400,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:33:57.249269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34698,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":6755,"num_read_kb":6753,"num_write":5061,"num_write_kb":2022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:39.720637+0000","last_change":"2026-02-24T22:05:29.651962+0000","last_active":"2026-02-24T22:29:39.720637+0000","last_peered":"2026-02-24T22:29:39.720637+0000","last_clean":"2026-02-24T22:29:39.720637+0000","last_became_active":"2026-02-24T22:05:29.651760+0000","last_became_peered":"2026-02-24T22:05:29.651760+0000","last_unstale":"2026-02-24T22:29:39.720637+0000","last_undegraded":"2026-02-24T22:29:39.720637+0000","last_fullsized":"2026-02-24T22:29:39.720637+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:48:41.143100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"312'128","reported_seq":805,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:59.734737+0000","last_change":"2026-02-24T22:05:18.185689+0000","last_active":"2026-02-24T22:31:59.734737+0000","last_peered":"2026-02-24T22:31:59.734737+0000","last_clean":"2026-02-24T22:31:59.734737+0000","last_became_active":"2026-02-24T22:05:18.184939+0000","last_became_peered":"2026-02-24T22:05:18.184939+0000","last_unstale":"2026-02-24T22:31:59.734737+0000","last_undegraded":"2026-02-24T22:31:59.734737+0000","last_fullsized":"2026-02-24T22:31:59.734737+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:18:46.745974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":436,"num_read_kb":360,"num_write":185,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"312'215","reported_seq":939,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.731199+0000","last_change":"2026-02-24T22:05:16.186833+0000","last_active":"2026-02-24T22:31:24.731199+0000","last_peered":"2026-02-24T22:31:24.731199+0000","last_clean":"2026-02-24T22:31:24.731199+0000","last_became_active":"2026-02-24T22:05:16.186043+0000","last_became_peered":"2026-02-24T22:05:16.186043+0000","last_unstale":"2026-02-24T22:31:24.731199+0000","last_undegraded":"2026-02-24T22:31:24.731199+0000","last_fullsized":"2026-02-24T22:31:24.731199+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:59:46.638347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5313539,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":397,"num_read_kb":15748,"num_write":1186,"num_write_kb":5255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.742847+0000","last_change":"2026-02-24T22:05:20.193254+0000","last_active":"2026-02-24T22:31:54.742847+0000","last_peered":"2026-02-24T22:31:54.742847+0000","last_clean":"2026-02-24T22:31:54.742847+0000","last_became_active":"2026-02-24T22:05:20.192962+0000","last_became_peered":"2026-02-24T22:05:20.192962+0000","last_unstale":"2026-02-24T22:31:54.742847+0000","last_undegraded":"2026-02-24T22:31:54.742847+0000","last_fullsized":"2026-02-24T22:31:54.742847+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:15:25.988404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"312'469","reported_seq":1300,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.539799+0000","last_change":"2026-02-24T22:05:22.194159+0000","last_active":"2026-02-24T22:34:18.539799+0000","last_peered":"2026-02-24T22:34:18.539799+0000","last_clean":"2026-02-24T22:34:18.539799+0000","last_became_active":"2026-02-24T22:05:22.193896+0000","last_became_peered":"2026-02-24T22:05:22.193896+0000","last_unstale":"2026-02-24T22:34:18.539799+0000","last_undegraded":"2026-02-24T22:34:18.539799+0000","last_fullsized":"2026-02-24T22:34:18.539799+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":400,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:30:50.341834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59985,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1293,"num_read_kb":1284,"num_write":349,"num_write_kb":316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"312'188","reported_seq":800,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.715171+0000","last_change":"2026-02-24T22:05:24.206176+0000","last_active":"2026-02-24T22:34:17.715171+0000","last_peered":"2026-02-24T22:34:17.715171+0000","last_clean":"2026-02-24T22:34:17.715171+0000","last_became_active":"2026-02-24T22:05:24.205855+0000","last_became_peered":"2026-02-24T22:05:24.205855+0000","last_unstale":"2026-02-24T22:34:17.715171+0000","last_undegraded":"2026-02-24T22:34:17.715171+0000","last_fullsized":"2026-02-24T22:34:17.715171+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":100,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:18:56.993738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27729,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":196,"num_read_kb":179,"num_write":471,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"312'443","reported_seq":1614,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.571252+0000","last_change":"2026-02-24T22:05:29.662310+0000","last_active":"2026-02-24T22:34:18.571252+0000","last_peered":"2026-02-24T22:34:18.571252+0000","last_clean":"2026-02-24T22:34:18.571252+0000","last_became_active":"2026-02-24T22:05:29.661983+0000","last_became_peered":"2026-02-24T22:05:29.661983+0000","last_unstale":"2026-02-24T22:34:18.571252+0000","last_undegraded":"2026-02-24T22:34:18.571252+0000","last_fullsized":"2026-02-24T22:34:18.571252+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":400,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:00:08.834953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"312'169","reported_seq":852,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.250160+0000","last_change":"2026-02-24T22:05:18.191009+0000","last_active":"2026-02-24T22:34:17.250160+0000","last_peered":"2026-02-24T22:34:17.250160+0000","last_clean":"2026-02-24T22:34:17.250160+0000","last_became_active":"2026-02-24T22:05:18.189877+0000","last_became_peered":"2026-02-24T22:05:18.189877+0000","last_unstale":"2026-02-24T22:34:17.250160+0000","last_undegraded":"2026-02-24T22:34:17.250160+0000","last_fullsized":"2026-02-24T22:34:17.250160+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:33:55.890050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":510,"num_write":249,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"312'253","reported_seq":1012,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:04.870640+0000","last_change":"2026-02-24T22:05:16.184964+0000","last_active":"2026-02-24T22:34:04.870640+0000","last_peered":"2026-02-24T22:34:04.870640+0000","last_clean":"2026-02-24T22:34:04.870640+0000","last_became_active":"2026-02-24T22:05:16.184525+0000","last_became_peered":"2026-02-24T22:05:16.184525+0000","last_unstale":"2026-02-24T22:34:04.870640+0000","last_undegraded":"2026-02-24T22:34:04.870640+0000","last_fullsized":"2026-02-24T22:34:04.870640+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:33:11.439600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10579665,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":508,"num_read_kb":16318,"num_write":1420,"num_write_kb":16752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"27'1","reported_seq":589,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:59.729910+0000","last_change":"2026-02-24T22:05:20.187071+0000","last_active":"2026-02-24T22:31:59.729910+0000","last_peered":"2026-02-24T22:31:59.729910+0000","last_clean":"2026-02-24T22:31:59.729910+0000","last_became_active":"2026-02-24T22:05:20.186850+0000","last_became_peered":"2026-02-24T22:05:20.186850+0000","last_unstale":"2026-02-24T22:31:59.729910+0000","last_undegraded":"2026-02-24T22:31:59.729910+0000","last_fullsized":"2026-02-24T22:31:59.729910+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:28:00.779043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"312'244","reported_seq":862,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.831208+0000","last_change":"2026-02-24T22:05:22.199568+0000","last_active":"2026-02-24T22:34:17.831208+0000","last_peered":"2026-02-24T22:34:17.831208+0000","last_clean":"2026-02-24T22:34:17.831208+0000","last_became_active":"2026-02-24T22:05:22.199358+0000","last_became_peered":"2026-02-24T22:05:22.199358+0000","last_unstale":"2026-02-24T22:34:17.831208+0000","last_undegraded":"2026-02-24T22:34:17.831208+0000","last_fullsized":"2026-02-24T22:34:17.831208+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":200,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:24:45.196918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59999,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":388,"num_read_kb":835,"num_write":354,"num_write_kb":306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"312'281","reported_seq":1222,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.228289+0000","last_change":"2026-02-24T22:05:24.200431+0000","last_active":"2026-02-24T22:34:18.228289+0000","last_peered":"2026-02-24T22:34:18.228289+0000","last_clean":"2026-02-24T22:34:18.228289+0000","last_became_active":"2026-02-24T22:05:24.199555+0000","last_became_peered":"2026-02-24T22:05:24.199555+0000","last_unstale":"2026-02-24T22:34:18.228289+0000","last_undegraded":"2026-02-24T22:34:18.228289+0000","last_fullsized":"2026-02-24T22:34:18.228289+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:29:23.063946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30969,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":655,"num_read_kb":647,"num_write":639,"num_write_kb":204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"312'429","reported_seq":1947,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.571358+0000","last_change":"2026-02-24T22:05:29.660609+0000","last_active":"2026-02-24T22:34:18.571358+0000","last_peered":"2026-02-24T22:34:18.571358+0000","last_clean":"2026-02-24T22:34:18.571358+0000","last_became_active":"2026-02-24T22:05:29.659532+0000","last_became_peered":"2026-02-24T22:05:29.659532+0000","last_unstale":"2026-02-24T22:34:18.571358+0000","last_undegraded":"2026-02-24T22:34:18.571358+0000","last_fullsized":"2026-02-24T22:34:18.571358+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":400,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:09:07.648679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"312'139","reported_seq":825,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.725732+0000","last_change":"2026-02-24T22:05:18.184639+0000","last_active":"2026-02-24T22:31:19.725732+0000","last_peered":"2026-02-24T22:31:19.725732+0000","last_clean":"2026-02-24T22:31:19.725732+0000","last_became_active":"2026-02-24T22:05:18.184250+0000","last_became_peered":"2026-02-24T22:05:18.184250+0000","last_unstale":"2026-02-24T22:31:19.725732+0000","last_undegraded":"2026-02-24T22:31:19.725732+0000","last_fullsized":"2026-02-24T22:31:19.725732+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:38:26.901167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":560,"num_read_kb":462,"num_write":204,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"312'1519","reported_seq":2874,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.226363+0000","last_change":"2026-02-24T22:05:16.190173+0000","last_active":"2026-02-24T22:34:17.226363+0000","last_peered":"2026-02-24T22:34:17.226363+0000","last_clean":"2026-02-24T22:34:17.226363+0000","last_became_active":"2026-02-24T22:05:16.189660+0000","last_became_peered":"2026-02-24T22:05:16.189660+0000","last_unstale":"2026-02-24T22:34:17.226363+0000","last_undegraded":"2026-02-24T22:34:17.226363+0000","last_fullsized":"2026-02-24T22:34:17.226363+0000","mapping_epoch":22,"log_start":"312'1500","ondisk_log_start":"312'1500","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":1500,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:28:22.592939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":80905,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":4244,"num_read_kb":2601,"num_write":2602,"num_write_kb":4261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"312'101","reported_seq":779,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:01.356810+0000","last_change":"2026-02-24T22:05:18.184329+0000","last_active":"2026-02-24T22:34:01.356810+0000","last_peered":"2026-02-24T22:34:01.356810+0000","last_clean":"2026-02-24T22:34:01.356810+0000","last_became_active":"2026-02-24T22:05:18.183822+0000","last_became_peered":"2026-02-24T22:05:18.183822+0000","last_unstale":"2026-02-24T22:34:01.356810+0000","last_undegraded":"2026-02-24T22:34:01.356810+0000","last_fullsized":"2026-02-24T22:34:01.356810+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:25:35.055559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":390,"num_read_kb":330,"num_write":143,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"312'231","reported_seq":979,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:53.833443+0000","last_change":"2026-02-24T22:05:16.183674+0000","last_active":"2026-02-24T22:30:53.833443+0000","last_peered":"2026-02-24T22:30:53.833443+0000","last_clean":"2026-02-24T22:30:53.833443+0000","last_became_active":"2026-02-24T22:05:16.183487+0000","last_became_peered":"2026-02-24T22:05:16.183487+0000","last_unstale":"2026-02-24T22:30:53.833443+0000","last_undegraded":"2026-02-24T22:30:53.833443+0000","last_fullsized":"2026-02-24T22:30:53.833443+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":200,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:25:02.663360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117418,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":445,"num_read_kb":706,"num_write":1288,"num_write_kb":183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"312'72","reported_seq":731,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.741422+0000","last_change":"2026-02-24T22:05:18.190755+0000","last_active":"2026-02-24T22:31:19.741422+0000","last_peered":"2026-02-24T22:31:19.741422+0000","last_clean":"2026-02-24T22:31:19.741422+0000","last_became_active":"2026-02-24T22:05:18.190352+0000","last_became_peered":"2026-02-24T22:05:18.190352+0000","last_unstale":"2026-02-24T22:31:19.741422+0000","last_undegraded":"2026-02-24T22:31:19.741422+0000","last_fullsized":"2026-02-24T22:31:19.741422+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:16:19.432104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":274,"num_read_kb":236,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"312'236","reported_seq":1158,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:12.225391+0000","last_change":"2026-02-24T22:05:16.191430+0000","last_active":"2026-02-24T22:34:12.225391+0000","last_peered":"2026-02-24T22:34:12.225391+0000","last_clean":"2026-02-24T22:34:12.225391+0000","last_became_active":"2026-02-24T22:05:16.190494+0000","last_became_peered":"2026-02-24T22:05:16.190494+0000","last_unstale":"2026-02-24T22:34:12.225391+0000","last_undegraded":"2026-02-24T22:34:12.225391+0000","last_fullsized":"2026-02-24T22:34:12.225391+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":200,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:14:56.208933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6381012,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":1034,"num_read_kb":14913,"num_write":1257,"num_write_kb":8068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"312'337","reported_seq":1150,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.725684+0000","last_change":"2026-02-24T22:05:18.186968+0000","last_active":"2026-02-24T22:31:19.725684+0000","last_peered":"2026-02-24T22:31:19.725684+0000","last_clean":"2026-02-24T22:31:19.725684+0000","last_became_active":"2026-02-24T22:05:18.186622+0000","last_became_peered":"2026-02-24T22:05:18.186622+0000","last_unstale":"2026-02-24T22:31:19.725684+0000","last_undegraded":"2026-02-24T22:31:19.725684+0000","last_fullsized":"2026-02-24T22:31:19.725684+0000","mapping_epoch":24,"log_start":"312'300","ondisk_log_start":"312'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":300,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:34:46.036169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1512,"num_read_kb":1316,"num_write":481,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"312'236","reported_seq":1075,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:06.918892+0000","last_change":"2026-02-24T22:05:16.186186+0000","last_active":"2026-02-24T22:34:06.918892+0000","last_peered":"2026-02-24T22:34:06.918892+0000","last_clean":"2026-02-24T22:34:06.918892+0000","last_became_active":"2026-02-24T22:05:16.185977+0000","last_became_peered":"2026-02-24T22:05:16.185977+0000","last_unstale":"2026-02-24T22:34:06.918892+0000","last_undegraded":"2026-02-24T22:34:06.918892+0000","last_fullsized":"2026-02-24T22:34:06.918892+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":200,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:53:50.492464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1207187,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":871,"num_read_kb":5969,"num_write":1291,"num_write_kb":2943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"312'89","reported_seq":764,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:00.529273+0000","last_change":"2026-02-24T22:05:18.186317+0000","last_active":"2026-02-24T22:34:00.529273+0000","last_peered":"2026-02-24T22:34:00.529273+0000","last_clean":"2026-02-24T22:34:00.529273+0000","last_became_active":"2026-02-24T22:05:18.185180+0000","last_became_peered":"2026-02-24T22:05:18.185180+0000","last_unstale":"2026-02-24T22:34:00.529273+0000","last_undegraded":"2026-02-24T22:34:00.529273+0000","last_fullsized":"2026-02-24T22:34:00.529273+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:17:22.057256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":311,"num_read_kb":264,"num_write":113,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"312'283","reported_seq":1035,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:59.726421+0000","last_change":"2026-02-24T22:06:00.660921+0000","last_active":"2026-02-24T22:31:59.726421+0000","last_peered":"2026-02-24T22:31:59.726421+0000","last_clean":"2026-02-24T22:31:59.726421+0000","last_became_active":"2026-02-24T22:06:00.660740+0000","last_became_peered":"2026-02-24T22:06:00.660740+0000","last_unstale":"2026-02-24T22:31:59.726421+0000","last_undegraded":"2026-02-24T22:31:59.726421+0000","last_fullsized":"2026-02-24T22:31:59.726421+0000","mapping_epoch":36,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":200,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:58:32.246490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12132727,"num_objects":86,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":579,"num_read_kb":9889,"num_write":1318,"num_write_kb":11975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"312'66","reported_seq":735,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.732670+0000","last_change":"2026-02-24T22:05:18.191552+0000","last_active":"2026-02-24T22:30:19.732670+0000","last_peered":"2026-02-24T22:30:19.732670+0000","last_clean":"2026-02-24T22:30:19.732670+0000","last_became_active":"2026-02-24T22:05:18.190390+0000","last_became_peered":"2026-02-24T22:05:18.190390+0000","last_unstale":"2026-02-24T22:30:19.732670+0000","last_undegraded":"2026-02-24T22:30:19.732670+0000","last_fullsized":"2026-02-24T22:30:19.732670+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:20:31.773222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":213,"num_read_kb":187,"num_write":90,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"312'255","reported_seq":978,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.725697+0000","last_change":"2026-02-24T22:05:16.184281+0000","last_active":"2026-02-24T22:31:19.725697+0000","last_peered":"2026-02-24T22:31:19.725697+0000","last_clean":"2026-02-24T22:31:19.725697+0000","last_became_active":"2026-02-24T22:05:16.184091+0000","last_became_peered":"2026-02-24T22:05:16.184091+0000","last_unstale":"2026-02-24T22:31:19.725697+0000","last_undegraded":"2026-02-24T22:31:19.725697+0000","last_fullsized":"2026-02-24T22:31:19.725697+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:42:08.074892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6296559,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":482,"num_read_kb":400,"num_write":1161,"num_write_kb":6221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"312'355","reported_seq":1157,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:45.969143+0000","last_change":"2026-02-24T22:05:18.187245+0000","last_active":"2026-02-24T22:33:45.969143+0000","last_peered":"2026-02-24T22:33:45.969143+0000","last_clean":"2026-02-24T22:33:45.969143+0000","last_became_active":"2026-02-24T22:05:18.186746+0000","last_became_peered":"2026-02-24T22:05:18.186746+0000","last_unstale":"2026-02-24T22:33:45.969143+0000","last_undegraded":"2026-02-24T22:33:45.969143+0000","last_fullsized":"2026-02-24T22:33:45.969143+0000","mapping_epoch":24,"log_start":"312'300","ondisk_log_start":"312'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":300,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:16:49.207184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":1676,"num_read_kb":1392,"num_write":638,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"312'250","reported_seq":954,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.737867+0000","last_change":"2026-02-24T22:05:16.190391+0000","last_active":"2026-02-24T22:31:19.737867+0000","last_peered":"2026-02-24T22:31:19.737867+0000","last_clean":"2026-02-24T22:31:19.737867+0000","last_became_active":"2026-02-24T22:05:16.189516+0000","last_became_peered":"2026-02-24T22:05:16.189516+0000","last_unstale":"2026-02-24T22:31:19.737867+0000","last_undegraded":"2026-02-24T22:31:19.737867+0000","last_fullsized":"2026-02-24T22:31:19.737867+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":200,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:11:15.635731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10540630,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":369,"num_read_kb":5491,"num_write":1149,"num_write_kb":20654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"312'164","reported_seq":854,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:03.781941+0000","last_change":"2026-02-24T22:05:18.190682+0000","last_active":"2026-02-24T22:34:03.781941+0000","last_peered":"2026-02-24T22:34:03.781941+0000","last_clean":"2026-02-24T22:34:03.781941+0000","last_became_active":"2026-02-24T22:05:18.190203+0000","last_became_peered":"2026-02-24T22:05:18.190203+0000","last_unstale":"2026-02-24T22:34:03.781941+0000","last_undegraded":"2026-02-24T22:34:03.781941+0000","last_fullsized":"2026-02-24T22:34:03.781941+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:20:56.349004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":631,"num_read_kb":516,"num_write":248,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"312'241","reported_seq":960,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:12.654487+0000","last_change":"2026-02-24T22:05:16.184697+0000","last_active":"2026-02-24T22:34:12.654487+0000","last_peered":"2026-02-24T22:34:12.654487+0000","last_clean":"2026-02-24T22:34:12.654487+0000","last_became_active":"2026-02-24T22:05:16.183772+0000","last_became_peered":"2026-02-24T22:05:16.183772+0000","last_unstale":"2026-02-24T22:34:12.654487+0000","last_undegraded":"2026-02-24T22:34:12.654487+0000","last_fullsized":"2026-02-24T22:34:12.654487+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":200,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:22:42.349222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6042885,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":404,"num_read_kb":12485,"num_write":1265,"num_write_kb":5970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"312'130","reported_seq":850,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.737414+0000","last_change":"2026-02-24T22:05:18.187529+0000","last_active":"2026-02-24T22:30:59.737414+0000","last_peered":"2026-02-24T22:30:59.737414+0000","last_clean":"2026-02-24T22:30:59.737414+0000","last_became_active":"2026-02-24T22:05:18.187321+0000","last_became_peered":"2026-02-24T22:05:18.187321+0000","last_unstale":"2026-02-24T22:30:59.737414+0000","last_undegraded":"2026-02-24T22:30:59.737414+0000","last_fullsized":"2026-02-24T22:30:59.737414+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:08:55.925019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":512,"num_read_kb":441,"num_write":201,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"312'238","reported_seq":946,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:58.282425+0000","last_change":"2026-02-24T22:05:16.193409+0000","last_active":"2026-02-24T22:33:58.282425+0000","last_peered":"2026-02-24T22:33:58.282425+0000","last_clean":"2026-02-24T22:33:58.282425+0000","last_became_active":"2026-02-24T22:05:16.193164+0000","last_became_peered":"2026-02-24T22:05:16.193164+0000","last_unstale":"2026-02-24T22:33:58.282425+0000","last_undegraded":"2026-02-24T22:33:58.282425+0000","last_fullsized":"2026-02-24T22:33:58.282425+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":200,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:41:44.916032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10795512,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":356,"num_read_kb":36420,"num_write":1300,"num_write_kb":10617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"312'148","reported_seq":825,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.737040+0000","last_change":"2026-02-24T22:05:18.193362+0000","last_active":"2026-02-24T22:30:59.737040+0000","last_peered":"2026-02-24T22:30:59.737040+0000","last_clean":"2026-02-24T22:30:59.737040+0000","last_became_active":"2026-02-24T22:05:18.191737+0000","last_became_peered":"2026-02-24T22:05:18.191737+0000","last_unstale":"2026-02-24T22:30:59.737040+0000","last_undegraded":"2026-02-24T22:30:59.737040+0000","last_fullsized":"2026-02-24T22:30:59.737040+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":100,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:51:20.868848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":539,"num_read_kb":448,"num_write":211,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"312'298","reported_seq":1049,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:54.775236+0000","last_change":"2026-02-24T22:05:16.184466+0000","last_active":"2026-02-24T22:30:54.775236+0000","last_peered":"2026-02-24T22:30:54.775236+0000","last_clean":"2026-02-24T22:30:54.775236+0000","last_became_active":"2026-02-24T22:05:16.183923+0000","last_became_peered":"2026-02-24T22:05:16.183923+0000","last_unstale":"2026-02-24T22:30:54.775236+0000","last_undegraded":"2026-02-24T22:30:54.775236+0000","last_fullsized":"2026-02-24T22:30:54.775236+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:15:14.475036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5523994,"num_objects":79,"num_object_clones":0,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":476,"num_read_kb":9842,"num_write":1525,"num_write_kb":9073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"312'146","reported_seq":817,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:09.715504+0000","last_change":"2026-02-24T22:05:18.184019+0000","last_active":"2026-02-24T22:30:09.715504+0000","last_peered":"2026-02-24T22:30:09.715504+0000","last_clean":"2026-02-24T22:30:09.715504+0000","last_became_active":"2026-02-24T22:05:18.183761+0000","last_became_peered":"2026-02-24T22:05:18.183761+0000","last_unstale":"2026-02-24T22:30:09.715504+0000","last_undegraded":"2026-02-24T22:30:09.715504+0000","last_fullsized":"2026-02-24T22:30:09.715504+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:07:03.909559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":561,"num_read_kb":451,"num_write":221,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"312'267","reported_seq":1036,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:44.059088+0000","last_change":"2026-02-24T22:05:16.191018+0000","last_active":"2026-02-24T22:33:44.059088+0000","last_peered":"2026-02-24T22:33:44.059088+0000","last_clean":"2026-02-24T22:33:44.059088+0000","last_became_active":"2026-02-24T22:05:16.189759+0000","last_became_peered":"2026-02-24T22:05:16.189759+0000","last_unstale":"2026-02-24T22:33:44.059088+0000","last_undegraded":"2026-02-24T22:33:44.059088+0000","last_fullsized":"2026-02-24T22:33:44.059088+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":200,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:34:51.111413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1371100,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":587,"num_read_kb":3529,"num_write":1434,"num_write_kb":3094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"312'140","reported_seq":832,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:31.161689+0000","last_change":"2026-02-24T22:05:18.194125+0000","last_active":"2026-02-24T22:30:31.161689+0000","last_peered":"2026-02-24T22:30:31.161689+0000","last_clean":"2026-02-24T22:30:31.161689+0000","last_became_active":"2026-02-24T22:05:18.192882+0000","last_became_peered":"2026-02-24T22:05:18.192882+0000","last_unstale":"2026-02-24T22:30:31.161689+0000","last_undegraded":"2026-02-24T22:30:31.161689+0000","last_fullsized":"2026-02-24T22:30:31.161689+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":100,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:13:16.621031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":593,"num_read_kb":494,"num_write":208,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"312'228","reported_seq":966,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.253876+0000","last_change":"2026-02-24T22:05:16.184964+0000","last_active":"2026-02-24T22:34:17.253876+0000","last_peered":"2026-02-24T22:34:17.253876+0000","last_clean":"2026-02-24T22:34:17.253876+0000","last_became_active":"2026-02-24T22:05:16.184557+0000","last_became_peered":"2026-02-24T22:05:16.184557+0000","last_unstale":"2026-02-24T22:34:17.253876+0000","last_undegraded":"2026-02-24T22:34:17.253876+0000","last_fullsized":"2026-02-24T22:34:17.253876+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":200,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:16:07.490514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5643921,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":420,"num_read_kb":1536,"num_write":1181,"num_write_kb":5590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"312'158","reported_seq":865,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:04.901988+0000","last_change":"2026-02-24T22:05:18.191676+0000","last_active":"2026-02-24T22:34:04.901988+0000","last_peered":"2026-02-24T22:34:04.901988+0000","last_clean":"2026-02-24T22:34:04.901988+0000","last_became_active":"2026-02-24T22:05:18.191432+0000","last_became_peered":"2026-02-24T22:05:18.191432+0000","last_unstale":"2026-02-24T22:34:04.901988+0000","last_undegraded":"2026-02-24T22:34:04.901988+0000","last_fullsized":"2026-02-24T22:34:04.901988+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":100,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:14:52.433749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":649,"num_read_kb":535,"num_write":233,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"312'222","reported_seq":943,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:00.269049+0000","last_change":"2026-02-24T22:05:16.190689+0000","last_active":"2026-02-24T22:34:00.269049+0000","last_peered":"2026-02-24T22:34:00.269049+0000","last_clean":"2026-02-24T22:34:00.269049+0000","last_became_active":"2026-02-24T22:05:16.190260+0000","last_became_peered":"2026-02-24T22:05:16.190260+0000","last_unstale":"2026-02-24T22:34:00.269049+0000","last_undegraded":"2026-02-24T22:34:00.269049+0000","last_fullsized":"2026-02-24T22:34:00.269049+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":200,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:55:30.357144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":254576,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":395,"num_read_kb":1504,"num_write":1294,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"312'143","reported_seq":831,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:15.308606+0000","last_change":"2026-02-24T22:05:18.185534+0000","last_active":"2026-02-24T22:30:15.308606+0000","last_peered":"2026-02-24T22:30:15.308606+0000","last_clean":"2026-02-24T22:30:15.308606+0000","last_became_active":"2026-02-24T22:05:18.185184+0000","last_became_peered":"2026-02-24T22:05:18.185184+0000","last_unstale":"2026-02-24T22:30:15.308606+0000","last_undegraded":"2026-02-24T22:30:15.308606+0000","last_fullsized":"2026-02-24T22:30:15.308606+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":100,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:50:27.214195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":536,"num_read_kb":445,"num_write":200,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"312'228","reported_seq":1041,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:32.940510+0000","last_change":"2026-02-24T22:05:16.191775+0000","last_active":"2026-02-24T22:33:32.940510+0000","last_peered":"2026-02-24T22:33:32.940510+0000","last_clean":"2026-02-24T22:33:32.940510+0000","last_became_active":"2026-02-24T22:05:16.191433+0000","last_became_peered":"2026-02-24T22:05:16.191433+0000","last_unstale":"2026-02-24T22:33:32.940510+0000","last_undegraded":"2026-02-24T22:33:32.940510+0000","last_fullsized":"2026-02-24T22:33:32.940510+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":200,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:08:07.147079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5270943,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":479,"num_read_kb":2264,"num_write":1260,"num_write_kb":7006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"312'62","reported_seq":721,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:08.697007+0000","last_change":"2026-02-24T22:05:18.192824+0000","last_active":"2026-02-24T22:30:08.697007+0000","last_peered":"2026-02-24T22:30:08.697007+0000","last_clean":"2026-02-24T22:30:08.697007+0000","last_became_active":"2026-02-24T22:05:18.192619+0000","last_became_peered":"2026-02-24T22:05:18.192619+0000","last_unstale":"2026-02-24T22:30:08.697007+0000","last_undegraded":"2026-02-24T22:30:08.697007+0000","last_fullsized":"2026-02-24T22:30:08.697007+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:21:57.498044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":204,"num_read_kb":176,"num_write":76,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"312'253","reported_seq":985,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:54.852196+0000","last_change":"2026-02-24T22:05:16.184618+0000","last_active":"2026-02-24T22:33:54.852196+0000","last_peered":"2026-02-24T22:33:54.852196+0000","last_clean":"2026-02-24T22:33:54.852196+0000","last_became_active":"2026-02-24T22:05:16.184441+0000","last_became_peered":"2026-02-24T22:05:16.184441+0000","last_unstale":"2026-02-24T22:33:54.852196+0000","last_undegraded":"2026-02-24T22:33:54.852196+0000","last_fullsized":"2026-02-24T22:33:54.852196+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:14:27.469981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1578451,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":396,"num_read_kb":3850,"num_write":1386,"num_write_kb":8445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"312'159","reported_seq":876,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:58.577766+0000","last_change":"2026-02-24T22:05:18.191572+0000","last_active":"2026-02-24T22:33:58.577766+0000","last_peered":"2026-02-24T22:33:58.577766+0000","last_clean":"2026-02-24T22:33:58.577766+0000","last_became_active":"2026-02-24T22:05:18.191350+0000","last_became_peered":"2026-02-24T22:05:18.191350+0000","last_unstale":"2026-02-24T22:33:58.577766+0000","last_undegraded":"2026-02-24T22:33:58.577766+0000","last_fullsized":"2026-02-24T22:33:58.577766+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:05:49.312781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":662,"num_read_kb":557,"num_write":245,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"312'219","reported_seq":987,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:43.905970+0000","last_change":"2026-02-24T22:06:00.662883+0000","last_active":"2026-02-24T22:33:43.905970+0000","last_peered":"2026-02-24T22:33:43.905970+0000","last_clean":"2026-02-24T22:33:43.905970+0000","last_became_active":"2026-02-24T22:06:00.662656+0000","last_became_peered":"2026-02-24T22:06:00.662656+0000","last_unstale":"2026-02-24T22:33:43.905970+0000","last_undegraded":"2026-02-24T22:33:43.905970+0000","last_fullsized":"2026-02-24T22:33:43.905970+0000","mapping_epoch":36,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":200,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:35:38.140607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5604982,"num_objects":86,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":721,"num_read_kb":6090,"num_write":1121,"num_write_kb":5556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.30","version":"312'268","reported_seq":1020,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:51.515479+0000","last_change":"2026-02-24T22:05:16.190494+0000","last_active":"2026-02-24T22:33:51.515479+0000","last_peered":"2026-02-24T22:33:51.515479+0000","last_clean":"2026-02-24T22:33:51.515479+0000","last_became_active":"2026-02-24T22:05:16.189713+0000","last_became_peered":"2026-02-24T22:05:16.189713+0000","last_unstale":"2026-02-24T22:33:51.515479+0000","last_undegraded":"2026-02-24T22:33:51.515479+0000","last_fullsized":"2026-02-24T22:33:51.515479+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":200,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:32:51.972858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10823593,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":564,"num_read_kb":31408,"num_write":1387,"num_write_kb":10638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"312'256","reported_seq":941,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:59.744014+0000","last_change":"2026-02-24T22:05:18.187784+0000","last_active":"2026-02-24T22:31:59.744014+0000","last_peered":"2026-02-24T22:31:59.744014+0000","last_clean":"2026-02-24T22:31:59.744014+0000","last_became_active":"2026-02-24T22:05:18.187565+0000","last_became_peered":"2026-02-24T22:05:18.187565+0000","last_unstale":"2026-02-24T22:31:59.744014+0000","last_undegraded":"2026-02-24T22:31:59.744014+0000","last_fullsized":"2026-02-24T22:31:59.744014+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":200,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:43:19.977135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":1060,"num_read_kb":853,"num_write":419,"num_write_kb":222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"312'225","reported_seq":931,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.723653+0000","last_change":"2026-02-24T22:05:16.178897+0000","last_active":"2026-02-24T22:30:59.723653+0000","last_peered":"2026-02-24T22:30:59.723653+0000","last_clean":"2026-02-24T22:30:59.723653+0000","last_became_active":"2026-02-24T22:05:16.178157+0000","last_became_peered":"2026-02-24T22:05:16.178157+0000","last_unstale":"2026-02-24T22:30:59.723653+0000","last_undegraded":"2026-02-24T22:30:59.723653+0000","last_fullsized":"2026-02-24T22:30:59.723653+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":200,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:50:48.164748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":621009,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":397,"num_read_kb":1640,"num_write":1065,"num_write_kb":1702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"312'78","reported_seq":752,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:51.565638+0000","last_change":"2026-02-24T22:05:18.191188+0000","last_active":"2026-02-24T22:33:51.565638+0000","last_peered":"2026-02-24T22:33:51.565638+0000","last_clean":"2026-02-24T22:33:51.565638+0000","last_became_active":"2026-02-24T22:05:18.190902+0000","last_became_peered":"2026-02-24T22:05:18.190902+0000","last_unstale":"2026-02-24T22:33:51.565638+0000","last_undegraded":"2026-02-24T22:33:51.565638+0000","last_fullsized":"2026-02-24T22:33:51.565638+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:18:59.644388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":302,"num_read_kb":262,"num_write":98,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":388,"ondisk_log_size":388,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":973109,"num_objects":3053,"num_object_clones":0,"num_object_copies":6106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3053,"num_whiteouts":0,"num_read":19978,"num_read_kb":245567,"num_write":23762,"num_write_kb":8619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24911872,"data_stored":1946218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5786661,"internal_metadata":0},"log_size":2182,"ondisk_log_size":2182,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1738830,"num_objects":223,"num_object_clones":0,"num_object_copies":446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":223,"num_whiteouts":0,"num_read":44721,"num_read_kb":43148,"num_write":15259,"num_write_kb":13598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3874816,"data_stored":3174742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2823344,"internal_metadata":0},"log_size":2043,"ondisk_log_size":2043,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1896,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1442,"num_read_kb":1673,"num_write":22,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":82,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11005,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":1511,"num_object_clones":0,"num_object_copies":3022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1511,"num_whiteouts":0,"num_read":95919,"num_read_kb":81897,"num_write":34645,"num_write_kb":17078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1511,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17301930,"internal_metadata":0},"log_size":3916,"ondisk_log_size":3916,"up":128,"acting":128,"num_store_stats":8},{"poolid":81,"num_pg":32,"stat_sum":{"num_bytes":72351744,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":71,"num_read_kb":119808,"num_write":63,"num_write_kb":70656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":144703488,"data_stored":144703488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":375571674,"num_objects":4668,"num_object_clones":0,"num_object_copies":9336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4668,"num_whiteouts":0,"num_read":36656,"num_read_kb":763453,"num_write":81696,"num_write_kb":523059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":786833408,"data_stored":751143348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2729,"ondisk_log_size":2729,"up":128,"acting":128,"num_store_stats":8},{"poolid":80,"num_pg":32,"stat_sum":{"num_bytes":72351744,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":73,"num_read_kb":124928,"num_write":63,"num_write_kb":70656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":144703488,"data_stored":144703488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":79,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3567,"num_read_kb":2401,"num_write":1127,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":317720,"internal_metadata":0},"log_size":462,"ondisk_log_size":462,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575202,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442900,"kb_used_data":199364,"kb_used_omap":3696,"kb_used_meta":239823,"kb_avail":93928940,"statfs":{"total":96636764160,"available":96183234560,"internally_reserved":0,"allocated":204148736,"data_stored":194101579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3785637,"internal_metadata":245578843},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575202,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":325324,"kb_used_data":81788,"kb_used_omap":3571,"kb_used_meta":239948,"kb_avail":94046516,"statfs":{"total":96636764160,"available":96303632384,"internally_reserved":0,"allocated":83750912,"data_stored":75161771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3657488,"internal_metadata":245706992},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575202,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":276592,"kb_used_data":177504,"kb_used_omap":2746,"kb_used_meta":96325,"kb_avail":94095248,"statfs":{"total":96636764160,"available":96353533952,"internally_reserved":0,"allocated":181764096,"data_stored":173324874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2812671,"internal_metadata":98637057},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575202,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":215740,"kb_used_data":152684,"kb_used_omap":3095,"kb_used_meta":59944,"kb_avail":94156100,"statfs":{"total":96636764160,"available":96415846400,"internally_reserved":0,"allocated":156348416,"data_stored":147731024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3169290,"internal_metadata":61383670},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575203,"num_pgs":107,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":329208,"kb_used_data":121768,"kb_used_omap":7232,"kb_used_meta":200191,"kb_avail":94042632,"statfs":{"total":96636764160,"available":96299655168,"internally_reserved":0,"allocated":124690432,"data_stored":113814900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7406186,"internal_metadata":204995990},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575203,"num_pgs":87,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":162392,"kb_used_data":117384,"kb_used_omap":1599,"kb_used_meta":43392,"kb_avail":94209448,"statfs":{"total":96636764160,"available":96470474752,"internally_reserved":0,"allocated":120201216,"data_stored":112600299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1638118,"internal_metadata":44433690},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575204,"num_pgs":102,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":178812,"kb_used_data":79660,"kb_used_omap":5094,"kb_used_meta":94041,"kb_avail":94193028,"statfs":{"total":96636764160,"available":96453660672,"internally_reserved":0,"allocated":81571840,"data_stored":73974457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5216565,"internal_metadata":96298699},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575204,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":408384,"kb_used_data":182960,"kb_used_omap":10223,"kb_used_meta":215184,"kb_avail":93963456,"statfs":{"total":96636764160,"available":96218578944,"internally_reserved":0,"allocated":187351040,"data_stored":176579114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10468372,"internal_metadata":220349420},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":129925120,"data_stored":124511737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":51691520,"data_stored":48716374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69758976,"data_stored":66666572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":88662016,"data_stored":83153600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":142684160,"data_stored":137577305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":122421248,"data_stored":117533084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":59645952,"data_stored":56294670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":122044416,"data_stored":116690006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1667351,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2633902,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1516609,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1844562,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2430346,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":723382,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3546560,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2939218,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":378928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1512,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":442706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1072852,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":148539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":667648,"data_stored":537960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18545,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":107453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5184,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":770048,"data_stored":659074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1705285,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":286394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11530,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":720896,"data_stored":613688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8436,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3928064,"data_stored":308322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51420,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3117056,"data_stored":241195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":152550,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3059712,"data_stored":239505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":825183,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3899392,"data_stored":305873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":336934,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1949696,"data_stored":151651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7338,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1925120,"data_stored":148450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":65762,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3837952,"data_stored":298046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2489145,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3194880,"data_stored":253176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1858329,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":79,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":133235,"internal_metadata":0},{"poolid":79,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14290,"internal_metadata":0},{"poolid":79,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23118,"internal_metadata":0},{"poolid":79,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15635,"internal_metadata":0},{"poolid":79,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49631,"internal_metadata":0},{"poolid":79,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49251,"internal_metadata":0},{"poolid":79,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16656,"internal_metadata":0},{"poolid":79,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15904,"internal_metadata":0},{"poolid":80,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26214400,"data_stored":26214400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16777216,"data_stored":16777216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":23068672,"data_stored":23068672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":23068672,"data_stored":23068672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":22020096,"data_stored":22020096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28311552,"data_stored":28311552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22020096,"data_stored":22020096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19922944,"data_stored":19922944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7340032,"data_stored":7340032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":30408704,"data_stored":30408704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45088768,"data_stored":45088768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":82,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":82,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1886,"internal_metadata":0},{"poolid":82,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6440,"internal_metadata":0},{"poolid":82,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":262,"internal_metadata":0},{"poolid":82,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1265,"internal_metadata":0},{"poolid":82,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":717,"internal_metadata":0},{"poolid":82,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":435,"internal_metadata":0},{"poolid":82,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T22:34:27.524 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-24T22:34:27.689 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:27.690 INFO:teuthology.orchestra.run.trial046.stdout:{"epoch":312,"fsid":"262e7508-9310-4d2a-bc05-2c8861262eff","created":"2026-02-24T22:04:57.636187+0000","modified":"2026-02-24T22:24:01.245657+0000","last_up_change":"2026-02-24T22:05:03.655652+0000","last_in_change":"2026-02-24T22:04:59.012805+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":82,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-24T22:05:05.621245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-24T22:05:07.173093+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.buckets.data","create_time":"2026-02-24T22:05:15.120743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"301","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.buckets.index","create_time":"2026-02-24T22:05:16.347093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":".rgw.root","create_time":"2026-02-24T22:05:18.228769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.log","create_time":"2026-02-24T22:05:20.421992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"default.rgw.meta","create_time":"2026-02-24T22:05:22.943794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"default.rgw.control","create_time":"2026-02-24T22:05:28.355752+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":79,"pool_name":"default.rgw.buckets.non-ec","create_time":"2026-02-24T22:13:42.533108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"291","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":80,"pool_name":"default.rgw.buckets.data.frozen","create_time":"2026-02-24T22:15:21.662768+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"304","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":81,"pool_name":"default.rgw.buckets.data.lukewarm","create_time":"2026-02-24T22:15:23.899899+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"307","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":82,"pool_name":"default.rgw.otp","create_time":"2026-02-24T22:23:27.433944+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"310","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"eecfa846-f234-4fa7-a191-99fa94beee15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":308,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6810","nonce":3488974090},{"type":"v1","addr":"10.20.193.46:6811","nonce":3488974090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6812","nonce":3488974090},{"type":"v1","addr":"10.20.193.46:6813","nonce":3488974090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6816","nonce":3488974090},{"type":"v1","addr":"10.20.193.46:6817","nonce":3488974090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6814","nonce":3488974090},{"type":"v1","addr":"10.20.193.46:6815","nonce":3488974090}]},"public_addr":"10.20.193.46:6811/3488974090","cluster_addr":"10.20.193.46:6813/3488974090","heartbeat_back_addr":"10.20.193.46:6817/3488974090","heartbeat_front_addr":"10.20.193.46:6815/3488974090","state":["exists","up"]},{"osd":1,"uuid":"611f1433-2d52-40f4-9aa5-e3440fbcec42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":308,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6818","nonce":2329512561},{"type":"v1","addr":"10.20.193.46:6819","nonce":2329512561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6820","nonce":2329512561},{"type":"v1","addr":"10.20.193.46:6821","nonce":2329512561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6824","nonce":2329512561},{"type":"v1","addr":"10.20.193.46:6825","nonce":2329512561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6822","nonce":2329512561},{"type":"v1","addr":"10.20.193.46:6823","nonce":2329512561}]},"public_addr":"10.20.193.46:6819/2329512561","cluster_addr":"10.20.193.46:6821/2329512561","heartbeat_back_addr":"10.20.193.46:6825/2329512561","heartbeat_front_addr":"10.20.193.46:6823/2329512561","state":["exists","up"]},{"osd":2,"uuid":"a509993f-f201-4228-a138-5f362e8e65c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":308,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6826","nonce":3065971976},{"type":"v1","addr":"10.20.193.46:6827","nonce":3065971976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6828","nonce":3065971976},{"type":"v1","addr":"10.20.193.46:6829","nonce":3065971976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6832","nonce":3065971976},{"type":"v1","addr":"10.20.193.46:6833","nonce":3065971976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6830","nonce":3065971976},{"type":"v1","addr":"10.20.193.46:6831","nonce":3065971976}]},"public_addr":"10.20.193.46:6827/3065971976","cluster_addr":"10.20.193.46:6829/3065971976","heartbeat_back_addr":"10.20.193.46:6833/3065971976","heartbeat_front_addr":"10.20.193.46:6831/3065971976","state":["exists","up"]},{"osd":3,"uuid":"7cfa4f22-9b2f-48e6-842c-c2cecb64a410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":308,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6802","nonce":2359184024},{"type":"v1","addr":"10.20.193.46:6803","nonce":2359184024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6804","nonce":2359184024},{"type":"v1","addr":"10.20.193.46:6805","nonce":2359184024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6808","nonce":2359184024},{"type":"v1","addr":"10.20.193.46:6809","nonce":2359184024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.46:6806","nonce":2359184024},{"type":"v1","addr":"10.20.193.46:6807","nonce":2359184024}]},"public_addr":"10.20.193.46:6803/2359184024","cluster_addr":"10.20.193.46:6805/2359184024","heartbeat_back_addr":"10.20.193.46:6809/2359184024","heartbeat_front_addr":"10.20.193.46:6807/2359184024","state":["exists","up"]},{"osd":4,"uuid":"2338abec-ea2d-4351-9610-321f26463100","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":308,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6800","nonce":341553926},{"type":"v1","addr":"10.20.193.110:6801","nonce":341553926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6802","nonce":341553926},{"type":"v1","addr":"10.20.193.110:6803","nonce":341553926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6806","nonce":341553926},{"type":"v1","addr":"10.20.193.110:6807","nonce":341553926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6804","nonce":341553926},{"type":"v1","addr":"10.20.193.110:6805","nonce":341553926}]},"public_addr":"10.20.193.110:6801/341553926","cluster_addr":"10.20.193.110:6803/341553926","heartbeat_back_addr":"10.20.193.110:6807/341553926","heartbeat_front_addr":"10.20.193.110:6805/341553926","state":["exists","up"]},{"osd":5,"uuid":"a4b1d335-43cd-49d5-b998-971f50dc15ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":311,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6816","nonce":2680757198},{"type":"v1","addr":"10.20.193.110:6818","nonce":2680757198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6820","nonce":2680757198},{"type":"v1","addr":"10.20.193.110:6822","nonce":2680757198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6828","nonce":2680757198},{"type":"v1","addr":"10.20.193.110:6830","nonce":2680757198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6824","nonce":2680757198},{"type":"v1","addr":"10.20.193.110:6826","nonce":2680757198}]},"public_addr":"10.20.193.110:6818/2680757198","cluster_addr":"10.20.193.110:6822/2680757198","heartbeat_back_addr":"10.20.193.110:6830/2680757198","heartbeat_front_addr":"10.20.193.110:6826/2680757198","state":["exists","up"]},{"osd":6,"uuid":"f19fb057-9193-4afb-bfe8-866c2109d2fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":308,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6808","nonce":884480935},{"type":"v1","addr":"10.20.193.110:6809","nonce":884480935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6810","nonce":884480935},{"type":"v1","addr":"10.20.193.110:6811","nonce":884480935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6814","nonce":884480935},{"type":"v1","addr":"10.20.193.110:6815","nonce":884480935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6812","nonce":884480935},{"type":"v1","addr":"10.20.193.110:6813","nonce":884480935}]},"public_addr":"10.20.193.110:6809/884480935","cluster_addr":"10.20.193.110:6811/884480935","heartbeat_back_addr":"10.20.193.110:6815/884480935","heartbeat_front_addr":"10.20.193.110:6813/884480935","state":["exists","up"]},{"osd":7,"uuid":"0dd2edb1-de90-4632-b940-655f96b44dd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":308,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6817","nonce":3330554869},{"type":"v1","addr":"10.20.193.110:6819","nonce":3330554869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6821","nonce":3330554869},{"type":"v1","addr":"10.20.193.110:6823","nonce":3330554869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6829","nonce":3330554869},{"type":"v1","addr":"10.20.193.110:6831","nonce":3330554869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.110:6825","nonce":3330554869},{"type":"v1","addr":"10.20.193.110:6827","nonce":3330554869}]},"public_addr":"10.20.193.110:6819/3330554869","cluster_addr":"10.20.193.110:6823/3330554869","heartbeat_back_addr":"10.20.193.110:6831/3330554869","heartbeat_front_addr":"10.20.193.110:6827/3330554869","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.826515+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.811034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.765092+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.662393+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.739442+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.809572+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.843167+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T22:05:01.792855+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":5}]},{"pgid":"79.4","mappings":[{"from":6,"to":3}]},{"pgid":"79.17","mappings":[{"from":0,"to":5}]},{"pgid":"82.14","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":26,"snaps":[{"begin":1,"length":2}]},{"pool":31,"snaps":[{"begin":1,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T22:34:28.704 INFO:tasks.ceph:Scrubbing osd.0 2026-02-24T22:34:28.704 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-24T22:34:28.791 INFO:teuthology.orchestra.run.trial046.stdout:{ 2026-02-24T22:34:28.792 INFO:teuthology.orchestra.run.trial046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-24T22:34:28.792 INFO:teuthology.orchestra.run.trial046.stdout:} 2026-02-24T22:34:28.801 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2026-02-24T22:34:28.960 INFO:teuthology.orchestra.run.trial046.stderr:instructed osd(s) 0 to deep-scrub 2026-02-24T22:34:28.972 INFO:tasks.ceph:Scrubbing osd.1 2026-02-24T22:34:28.972 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-24T22:34:29.050 INFO:teuthology.orchestra.run.trial046.stdout:{ 2026-02-24T22:34:29.050 INFO:teuthology.orchestra.run.trial046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-24T22:34:29.051 INFO:teuthology.orchestra.run.trial046.stdout:} 2026-02-24T22:34:29.062 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2026-02-24T22:34:29.210 INFO:teuthology.orchestra.run.trial046.stderr:instructed osd(s) 1 to deep-scrub 2026-02-24T22:34:29.221 INFO:tasks.ceph:Scrubbing osd.2 2026-02-24T22:34:29.222 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2026-02-24T22:34:29.299 INFO:teuthology.orchestra.run.trial046.stdout:{ 2026-02-24T22:34:29.299 INFO:teuthology.orchestra.run.trial046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-24T22:34:29.299 INFO:teuthology.orchestra.run.trial046.stdout:} 2026-02-24T22:34:29.311 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2026-02-24T22:34:29.470 INFO:teuthology.orchestra.run.trial046.stderr:instructed osd(s) 2 to deep-scrub 2026-02-24T22:34:29.483 INFO:tasks.ceph:Scrubbing osd.3 2026-02-24T22:34:29.484 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2026-02-24T22:34:29.569 INFO:teuthology.orchestra.run.trial046.stdout:{ 2026-02-24T22:34:29.569 INFO:teuthology.orchestra.run.trial046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-24T22:34:29.569 INFO:teuthology.orchestra.run.trial046.stdout:} 2026-02-24T22:34:29.581 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2026-02-24T22:34:29.744 INFO:teuthology.orchestra.run.trial046.stderr:instructed osd(s) 3 to deep-scrub 2026-02-24T22:34:29.757 INFO:tasks.ceph:Scrubbing osd.4 2026-02-24T22:34:29.757 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2026-02-24T22:34:29.835 INFO:teuthology.orchestra.run.trial046.stdout:{ 2026-02-24T22:34:29.835 INFO:teuthology.orchestra.run.trial046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-24T22:34:29.835 INFO:teuthology.orchestra.run.trial046.stdout:} 2026-02-24T22:34:29.847 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2026-02-24T22:34:30.005 INFO:teuthology.orchestra.run.trial046.stderr:instructed osd(s) 4 to deep-scrub 2026-02-24T22:34:30.018 INFO:tasks.ceph:Scrubbing osd.5 2026-02-24T22:34:30.019 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2026-02-24T22:34:30.103 INFO:teuthology.orchestra.run.trial046.stdout:{ 2026-02-24T22:34:30.103 INFO:teuthology.orchestra.run.trial046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-24T22:34:30.104 INFO:teuthology.orchestra.run.trial046.stdout:} 2026-02-24T22:34:30.113 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2026-02-24T22:34:30.275 INFO:teuthology.orchestra.run.trial046.stderr:instructed osd(s) 5 to deep-scrub 2026-02-24T22:34:30.287 INFO:tasks.ceph:Scrubbing osd.6 2026-02-24T22:34:30.287 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2026-02-24T22:34:30.370 INFO:teuthology.orchestra.run.trial046.stdout:{ 2026-02-24T22:34:30.371 INFO:teuthology.orchestra.run.trial046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-24T22:34:30.371 INFO:teuthology.orchestra.run.trial046.stdout:} 2026-02-24T22:34:30.383 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2026-02-24T22:34:30.538 INFO:teuthology.orchestra.run.trial046.stderr:instructed osd(s) 6 to deep-scrub 2026-02-24T22:34:30.550 INFO:tasks.ceph:Scrubbing osd.7 2026-02-24T22:34:30.550 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2026-02-24T22:34:30.627 INFO:teuthology.orchestra.run.trial046.stdout:{ 2026-02-24T22:34:30.627 INFO:teuthology.orchestra.run.trial046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-24T22:34:30.627 INFO:teuthology.orchestra.run.trial046.stdout:} 2026-02-24T22:34:30.636 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2026-02-24T22:34:30.784 INFO:teuthology.orchestra.run.trial046.stderr:instructed osd(s) 7 to deep-scrub 2026-02-24T22:34:30.795 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-24T22:34:30.966 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:30.983 INFO:teuthology.orchestra.run.trial046.stderr:dumped all 2026-02-24T22:34:30.985 INFO:teuthology.orchestra.run.trial046.stdout:{"pg_ready":true,"pg_map":{"version":1188,"stamp":"2026-02-24T22:34:30.024525+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":115015404,"num_objects":5093,"num_object_clones":0,"num_object_copies":10186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5093,"num_whiteouts":0,"num_read":207562,"num_read_kb":1387906,"num_write":161478,"num_write_kb":706745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":1543,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13530,"ondisk_log_size":13530,"up":786,"acting":786,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":786,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1524276,"kb_used_data":278856,"kb_used_omap":37260,"kb_used_meta":1206899,"kb_avail":753450444,"statfs":{"total":773094113280,"available":771533254656,"internally_reserved":0,"allocated":285548544,"data_stored":247979658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38154327,"internal_metadata":1235865513},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-408570222,"num_objects":-4464,"num_object_clones":0,"num_object_copies":-8928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4464,"num_whiteouts":0,"num_read":6374,"num_read_kb":5776,"num_write":4848,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003309"},"pg_stats":[{"pgid":"82.2","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744748+0000","last_change":"2026-02-24T22:23:29.198729+0000","last_active":"2026-02-24T22:31:49.744748+0000","last_peered":"2026-02-24T22:31:49.744748+0000","last_clean":"2026-02-24T22:31:49.744748+0000","last_became_active":"2026-02-24T22:23:29.198476+0000","last_became_peered":"2026-02-24T22:23:29.198476+0000","last_unstale":"2026-02-24T22:31:49.744748+0000","last_undegraded":"2026-02-24T22:31:49.744748+0000","last_fullsized":"2026-02-24T22:31:49.744748+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:12:35.165001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.1","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.730924+0000","last_change":"2026-02-24T22:15:25.830397+0000","last_active":"2026-02-24T22:32:09.730924+0000","last_peered":"2026-02-24T22:32:09.730924+0000","last_clean":"2026-02-24T22:32:09.730924+0000","last_became_active":"2026-02-24T22:15:25.830000+0000","last_became_peered":"2026-02-24T22:15:25.830000+0000","last_unstale":"2026-02-24T22:32:09.730924+0000","last_undegraded":"2026-02-24T22:32:09.730924+0000","last_fullsized":"2026-02-24T22:32:09.730924+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:41:30.400463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.0","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:29.723258+0000","last_change":"2026-02-24T22:15:22.817921+0000","last_active":"2026-02-24T22:31:29.723258+0000","last_peered":"2026-02-24T22:31:29.723258+0000","last_clean":"2026-02-24T22:31:29.723258+0000","last_became_active":"2026-02-24T22:15:22.817446+0000","last_became_peered":"2026-02-24T22:15:22.817446+0000","last_unstale":"2026-02-24T22:31:29.723258+0000","last_undegraded":"2026-02-24T22:31:29.723258+0000","last_fullsized":"2026-02-24T22:31:29.723258+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:25:30.206672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.1f","version":"312'20","reported_seq":120,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.733669+0000","last_change":"2026-02-24T22:13:43.761604+0000","last_active":"2026-02-24T22:30:24.733669+0000","last_peered":"2026-02-24T22:30:24.733669+0000","last_clean":"2026-02-24T22:30:24.733669+0000","last_became_active":"2026-02-24T22:13:43.761120+0000","last_became_peered":"2026-02-24T22:13:43.761120+0000","last_unstale":"2026-02-24T22:30:24.733669+0000","last_undegraded":"2026-02-24T22:30:24.733669+0000","last_fullsized":"2026-02-24T22:30:24.733669+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:48:55.248638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":168,"num_read_kb":114,"num_write":47,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.3","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742865+0000","last_change":"2026-02-24T22:23:29.213779+0000","last_active":"2026-02-24T22:31:49.742865+0000","last_peered":"2026-02-24T22:31:49.742865+0000","last_clean":"2026-02-24T22:31:49.742865+0000","last_became_active":"2026-02-24T22:23:29.213534+0000","last_became_peered":"2026-02-24T22:23:29.213534+0000","last_unstale":"2026-02-24T22:31:49.742865+0000","last_undegraded":"2026-02-24T22:31:49.742865+0000","last_fullsized":"2026-02-24T22:31:49.742865+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:38:56.711711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.0","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.743144+0000","last_change":"2026-02-24T22:15:25.827702+0000","last_active":"2026-02-24T22:31:34.743144+0000","last_peered":"2026-02-24T22:31:34.743144+0000","last_clean":"2026-02-24T22:31:34.743144+0000","last_became_active":"2026-02-24T22:15:25.827355+0000","last_became_peered":"2026-02-24T22:15:25.827355+0000","last_unstale":"2026-02-24T22:31:34.743144+0000","last_undegraded":"2026-02-24T22:31:34.743144+0000","last_fullsized":"2026-02-24T22:31:34.743144+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:22:12.090995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.1","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:04.744507+0000","last_change":"2026-02-24T22:15:22.818757+0000","last_active":"2026-02-24T22:32:04.744507+0000","last_peered":"2026-02-24T22:32:04.744507+0000","last_clean":"2026-02-24T22:32:04.744507+0000","last_became_active":"2026-02-24T22:15:22.818519+0000","last_became_peered":"2026-02-24T22:15:22.818519+0000","last_unstale":"2026-02-24T22:32:04.744507+0000","last_undegraded":"2026-02-24T22:32:04.744507+0000","last_fullsized":"2026-02-24T22:32:04.744507+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:17:56.378157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.1e","version":"312'9","reported_seq":88,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.745172+0000","last_change":"2026-02-24T22:13:43.757760+0000","last_active":"2026-02-24T22:31:49.745172+0000","last_peered":"2026-02-24T22:31:49.745172+0000","last_clean":"2026-02-24T22:31:49.745172+0000","last_became_active":"2026-02-24T22:13:43.757543+0000","last_became_peered":"2026-02-24T22:13:43.757543+0000","last_unstale":"2026-02-24T22:31:49.745172+0000","last_undegraded":"2026-02-24T22:31:49.745172+0000","last_fullsized":"2026-02-24T22:31:49.745172+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:44:23.087405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":38,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.0","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.723743+0000","last_change":"2026-02-24T22:23:29.212801+0000","last_active":"2026-02-24T22:31:34.723743+0000","last_peered":"2026-02-24T22:31:34.723743+0000","last_clean":"2026-02-24T22:31:34.723743+0000","last_became_active":"2026-02-24T22:23:29.212043+0000","last_became_peered":"2026-02-24T22:23:29.212043+0000","last_unstale":"2026-02-24T22:31:34.723743+0000","last_undegraded":"2026-02-24T22:31:34.723743+0000","last_fullsized":"2026-02-24T22:31:34.723743+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:45:48.953242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.3","version":"312'8","reported_seq":53,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.471100+0000","last_change":"2026-02-24T22:15:25.827590+0000","last_active":"2026-02-24T22:34:25.471100+0000","last_peered":"2026-02-24T22:34:25.471100+0000","last_clean":"2026-02-24T22:34:25.471100+0000","last_became_active":"2026-02-24T22:15:25.827216+0000","last_became_peered":"2026-02-24T22:15:25.827216+0000","last_unstale":"2026-02-24T22:34:25.471100+0000","last_undegraded":"2026-02-24T22:34:25.471100+0000","last_fullsized":"2026-02-24T22:34:25.471100+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:08:31.068519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":25602,"num_write":13,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.2","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:04.726927+0000","last_change":"2026-02-24T22:15:22.818494+0000","last_active":"2026-02-24T22:32:04.726927+0000","last_peered":"2026-02-24T22:32:04.726927+0000","last_clean":"2026-02-24T22:32:04.726927+0000","last_became_active":"2026-02-24T22:15:22.818194+0000","last_became_peered":"2026-02-24T22:15:22.818194+0000","last_unstale":"2026-02-24T22:32:04.726927+0000","last_undegraded":"2026-02-24T22:32:04.726927+0000","last_fullsized":"2026-02-24T22:32:04.726927+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:30:45.313454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.1d","version":"312'22","reported_seq":121,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.737198+0000","last_change":"2026-02-24T22:13:43.757707+0000","last_active":"2026-02-24T22:31:14.737198+0000","last_peered":"2026-02-24T22:31:14.737198+0000","last_clean":"2026-02-24T22:31:14.737198+0000","last_became_active":"2026-02-24T22:13:43.757278+0000","last_became_peered":"2026-02-24T22:13:43.757278+0000","last_unstale":"2026-02-24T22:31:14.737198+0000","last_undegraded":"2026-02-24T22:31:14.737198+0000","last_fullsized":"2026-02-24T22:31:14.737198+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:52:33.539660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":103,"num_write":54,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.1","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742841+0000","last_change":"2026-02-24T22:23:29.214295+0000","last_active":"2026-02-24T22:31:49.742841+0000","last_peered":"2026-02-24T22:31:49.742841+0000","last_clean":"2026-02-24T22:31:49.742841+0000","last_became_active":"2026-02-24T22:23:29.214064+0000","last_became_peered":"2026-02-24T22:23:29.214064+0000","last_unstale":"2026-02-24T22:31:49.742841+0000","last_undegraded":"2026-02-24T22:31:49.742841+0000","last_fullsized":"2026-02-24T22:31:49.742841+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:03:04.365969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.2","version":"312'2","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:14.748763+0000","last_change":"2026-02-24T22:15:25.833367+0000","last_active":"2026-02-24T22:34:14.748763+0000","last_peered":"2026-02-24T22:34:14.748763+0000","last_clean":"2026-02-24T22:34:14.748763+0000","last_became_active":"2026-02-24T22:15:25.832809+0000","last_became_peered":"2026-02-24T22:15:25.832809+0000","last_unstale":"2026-02-24T22:34:14.748763+0000","last_undegraded":"2026-02-24T22:34:14.748763+0000","last_fullsized":"2026-02-24T22:34:14.748763+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:28:13.551199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"80.3","version":"312'3","reported_seq":44,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:09.739916+0000","last_change":"2026-02-24T22:15:22.818401+0000","last_active":"2026-02-24T22:34:09.739916+0000","last_peered":"2026-02-24T22:34:09.739916+0000","last_clean":"2026-02-24T22:34:09.739916+0000","last_became_active":"2026-02-24T22:15:22.818089+0000","last_became_peered":"2026-02-24T22:15:22.818089+0000","last_unstale":"2026-02-24T22:34:09.739916+0000","last_undegraded":"2026-02-24T22:34:09.739916+0000","last_fullsized":"2026-02-24T22:34:09.739916+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:40:25.179819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.1c","version":"312'14","reported_seq":101,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.739030+0000","last_change":"2026-02-24T22:13:43.760682+0000","last_active":"2026-02-24T22:31:14.739030+0000","last_peered":"2026-02-24T22:31:14.739030+0000","last_clean":"2026-02-24T22:31:14.739030+0000","last_became_active":"2026-02-24T22:13:43.759958+0000","last_became_peered":"2026-02-24T22:13:43.759958+0000","last_unstale":"2026-02-24T22:31:14.739030+0000","last_undegraded":"2026-02-24T22:31:14.739030+0000","last_fullsized":"2026-02-24T22:31:14.739030+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:24:39.289428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":75,"num_write":36,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.6","version":"309'1","reported_seq":23,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.730973+0000","last_change":"2026-02-24T22:23:29.215576+0000","last_active":"2026-02-24T22:32:09.730973+0000","last_peered":"2026-02-24T22:32:09.730973+0000","last_clean":"2026-02-24T22:32:09.730973+0000","last_became_active":"2026-02-24T22:23:29.215327+0000","last_became_peered":"2026-02-24T22:23:29.215327+0000","last_unstale":"2026-02-24T22:32:09.730973+0000","last_undegraded":"2026-02-24T22:32:09.730973+0000","last_fullsized":"2026-02-24T22:32:09.730973+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:51:01.281339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.5","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:14.745596+0000","last_change":"2026-02-24T22:15:25.832444+0000","last_active":"2026-02-24T22:32:14.745596+0000","last_peered":"2026-02-24T22:32:14.745596+0000","last_clean":"2026-02-24T22:32:14.745596+0000","last_became_active":"2026-02-24T22:15:25.831975+0000","last_became_peered":"2026-02-24T22:15:25.831975+0000","last_unstale":"2026-02-24T22:32:14.745596+0000","last_undegraded":"2026-02-24T22:32:14.745596+0000","last_fullsized":"2026-02-24T22:32:14.745596+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:04:17.334836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"80.4","version":"312'3","reported_seq":44,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:09.761034+0000","last_change":"2026-02-24T22:15:22.812647+0000","last_active":"2026-02-24T22:34:09.761034+0000","last_peered":"2026-02-24T22:34:09.761034+0000","last_clean":"2026-02-24T22:34:09.761034+0000","last_became_active":"2026-02-24T22:15:22.812270+0000","last_became_peered":"2026-02-24T22:15:22.812270+0000","last_unstale":"2026-02-24T22:34:09.761034+0000","last_undegraded":"2026-02-24T22:34:09.761034+0000","last_fullsized":"2026-02-24T22:34:09.761034+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:42:27.774508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.1b","version":"312'18","reported_seq":112,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.741475+0000","last_change":"2026-02-24T22:13:43.755424+0000","last_active":"2026-02-24T22:31:14.741475+0000","last_peered":"2026-02-24T22:31:14.741475+0000","last_clean":"2026-02-24T22:31:14.741475+0000","last_became_active":"2026-02-24T22:13:43.755170+0000","last_became_peered":"2026-02-24T22:13:43.755170+0000","last_unstale":"2026-02-24T22:31:14.741475+0000","last_undegraded":"2026-02-24T22:31:14.741475+0000","last_fullsized":"2026-02-24T22:31:14.741475+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:29:17.090743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":101,"num_write":42,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.7","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.735768+0000","last_change":"2026-02-24T22:23:29.214459+0000","last_active":"2026-02-24T22:32:09.735768+0000","last_peered":"2026-02-24T22:32:09.735768+0000","last_clean":"2026-02-24T22:32:09.735768+0000","last_became_active":"2026-02-24T22:23:29.214105+0000","last_became_peered":"2026-02-24T22:23:29.214105+0000","last_unstale":"2026-02-24T22:32:09.735768+0000","last_undegraded":"2026-02-24T22:32:09.735768+0000","last_fullsized":"2026-02-24T22:32:09.735768+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:00:10.302637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.4","version":"312'3","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:14.760553+0000","last_change":"2026-02-24T22:15:25.827716+0000","last_active":"2026-02-24T22:34:14.760553+0000","last_peered":"2026-02-24T22:34:14.760553+0000","last_clean":"2026-02-24T22:34:14.760553+0000","last_became_active":"2026-02-24T22:15:25.826663+0000","last_became_peered":"2026-02-24T22:15:25.826663+0000","last_unstale":"2026-02-24T22:34:14.760553+0000","last_undegraded":"2026-02-24T22:34:14.760553+0000","last_fullsized":"2026-02-24T22:34:14.760553+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:08:34.544962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.5","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.730960+0000","last_change":"2026-02-24T22:15:22.816880+0000","last_active":"2026-02-24T22:32:09.730960+0000","last_peered":"2026-02-24T22:32:09.730960+0000","last_clean":"2026-02-24T22:32:09.730960+0000","last_became_active":"2026-02-24T22:15:22.816697+0000","last_became_peered":"2026-02-24T22:15:22.816697+0000","last_unstale":"2026-02-24T22:32:09.730960+0000","last_undegraded":"2026-02-24T22:32:09.730960+0000","last_fullsized":"2026-02-24T22:32:09.730960+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:56:01.670361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"79.1a","version":"312'16","reported_seq":102,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.737175+0000","last_change":"2026-02-24T22:13:43.757739+0000","last_active":"2026-02-24T22:31:14.737175+0000","last_peered":"2026-02-24T22:31:14.737175+0000","last_clean":"2026-02-24T22:31:14.737175+0000","last_became_active":"2026-02-24T22:13:43.757353+0000","last_became_peered":"2026-02-24T22:13:43.757353+0000","last_unstale":"2026-02-24T22:31:14.737175+0000","last_undegraded":"2026-02-24T22:31:14.737175+0000","last_fullsized":"2026-02-24T22:31:14.737175+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:47:30.725736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":65,"num_write":38,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.4","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742881+0000","last_change":"2026-02-24T22:23:29.215206+0000","last_active":"2026-02-24T22:31:49.742881+0000","last_peered":"2026-02-24T22:31:49.742881+0000","last_clean":"2026-02-24T22:31:49.742881+0000","last_became_active":"2026-02-24T22:23:29.215014+0000","last_became_peered":"2026-02-24T22:23:29.215014+0000","last_unstale":"2026-02-24T22:31:49.742881+0000","last_undegraded":"2026-02-24T22:31:49.742881+0000","last_fullsized":"2026-02-24T22:31:49.742881+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:43:42.586588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.7","version":"312'3","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:49.731657+0000","last_change":"2026-02-24T22:15:25.832214+0000","last_active":"2026-02-24T22:32:49.731657+0000","last_peered":"2026-02-24T22:32:49.731657+0000","last_clean":"2026-02-24T22:32:49.731657+0000","last_became_active":"2026-02-24T22:15:25.831806+0000","last_became_peered":"2026-02-24T22:15:25.831806+0000","last_unstale":"2026-02-24T22:32:49.731657+0000","last_undegraded":"2026-02-24T22:32:49.731657+0000","last_fullsized":"2026-02-24T22:32:49.731657+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:36:56.341773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"80.6","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:34.736157+0000","last_change":"2026-02-24T22:15:22.811674+0000","last_active":"2026-02-24T22:30:34.736157+0000","last_peered":"2026-02-24T22:30:34.736157+0000","last_clean":"2026-02-24T22:30:34.736157+0000","last_became_active":"2026-02-24T22:15:22.811413+0000","last_became_peered":"2026-02-24T22:15:22.811413+0000","last_unstale":"2026-02-24T22:30:34.736157+0000","last_undegraded":"2026-02-24T22:30:34.736157+0000","last_fullsized":"2026-02-24T22:30:34.736157+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:14:20.030861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"79.19","version":"312'30","reported_seq":140,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.738215+0000","last_change":"2026-02-24T22:13:43.756439+0000","last_active":"2026-02-24T22:30:44.738215+0000","last_peered":"2026-02-24T22:30:44.738215+0000","last_clean":"2026-02-24T22:30:44.738215+0000","last_became_active":"2026-02-24T22:13:43.756114+0000","last_became_peered":"2026-02-24T22:13:43.756114+0000","last_unstale":"2026-02-24T22:30:44.738215+0000","last_undegraded":"2026-02-24T22:30:44.738215+0000","last_fullsized":"2026-02-24T22:30:44.738215+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:45:46.139575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":216,"num_read_kb":143,"num_write":68,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.5","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742892+0000","last_change":"2026-02-24T22:23:29.213537+0000","last_active":"2026-02-24T22:31:49.742892+0000","last_peered":"2026-02-24T22:31:49.742892+0000","last_clean":"2026-02-24T22:31:49.742892+0000","last_became_active":"2026-02-24T22:23:29.213256+0000","last_became_peered":"2026-02-24T22:23:29.213256+0000","last_unstale":"2026-02-24T22:31:49.742892+0000","last_undegraded":"2026-02-24T22:31:49.742892+0000","last_fullsized":"2026-02-24T22:31:49.742892+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:48:26.724830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.6","version":"312'3","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:49.735160+0000","last_change":"2026-02-24T22:15:25.833922+0000","last_active":"2026-02-24T22:32:49.735160+0000","last_peered":"2026-02-24T22:32:49.735160+0000","last_clean":"2026-02-24T22:32:49.735160+0000","last_became_active":"2026-02-24T22:15:25.833680+0000","last_became_peered":"2026-02-24T22:15:25.833680+0000","last_unstale":"2026-02-24T22:32:49.735160+0000","last_undegraded":"2026-02-24T22:32:49.735160+0000","last_fullsized":"2026-02-24T22:32:49.735160+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:26:51.677018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.7","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:34.736181+0000","last_change":"2026-02-24T22:15:22.812054+0000","last_active":"2026-02-24T22:30:34.736181+0000","last_peered":"2026-02-24T22:30:34.736181+0000","last_clean":"2026-02-24T22:30:34.736181+0000","last_became_active":"2026-02-24T22:15:22.811618+0000","last_became_peered":"2026-02-24T22:15:22.811618+0000","last_unstale":"2026-02-24T22:30:34.736181+0000","last_undegraded":"2026-02-24T22:30:34.736181+0000","last_fullsized":"2026-02-24T22:30:34.736181+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:37:58.903441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"79.18","version":"312'12","reported_seq":95,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.737670+0000","last_change":"2026-02-24T22:13:43.755501+0000","last_active":"2026-02-24T22:30:39.737670+0000","last_peered":"2026-02-24T22:30:39.737670+0000","last_clean":"2026-02-24T22:30:39.737670+0000","last_became_active":"2026-02-24T22:13:43.754981+0000","last_became_peered":"2026-02-24T22:13:43.754981+0000","last_unstale":"2026-02-24T22:30:39.737670+0000","last_undegraded":"2026-02-24T22:30:39.737670+0000","last_fullsized":"2026-02-24T22:30:39.737670+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:41:38.765379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":86,"num_read_kb":57,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.a","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.745052+0000","last_change":"2026-02-24T22:23:29.212877+0000","last_active":"2026-02-24T22:32:09.745052+0000","last_peered":"2026-02-24T22:32:09.745052+0000","last_clean":"2026-02-24T22:32:09.745052+0000","last_became_active":"2026-02-24T22:23:29.212213+0000","last_became_peered":"2026-02-24T22:23:29.212213+0000","last_unstale":"2026-02-24T22:32:09.745052+0000","last_undegraded":"2026-02-24T22:32:09.745052+0000","last_fullsized":"2026-02-24T22:32:09.745052+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:29:28.605583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.9","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.736816+0000","last_change":"2026-02-24T22:15:25.826412+0000","last_active":"2026-02-24T22:30:39.736816+0000","last_peered":"2026-02-24T22:30:39.736816+0000","last_clean":"2026-02-24T22:30:39.736816+0000","last_became_active":"2026-02-24T22:15:25.826076+0000","last_became_peered":"2026-02-24T22:15:25.826076+0000","last_unstale":"2026-02-24T22:30:39.736816+0000","last_undegraded":"2026-02-24T22:30:39.736816+0000","last_fullsized":"2026-02-24T22:30:39.736816+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:00:25.763716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.8","version":"312'6","reported_seq":48,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.572211+0000","last_change":"2026-02-24T22:15:22.816346+0000","last_active":"2026-02-24T22:34:25.572211+0000","last_peered":"2026-02-24T22:34:25.572211+0000","last_clean":"2026-02-24T22:34:25.572211+0000","last_became_active":"2026-02-24T22:15:22.815960+0000","last_became_peered":"2026-02-24T22:15:22.815960+0000","last_unstale":"2026-02-24T22:34:25.572211+0000","last_undegraded":"2026-02-24T22:34:25.572211+0000","last_fullsized":"2026-02-24T22:34:25.572211+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:55:34.277066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6144,"num_write":11,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.17","version":"312'34","reported_seq":169,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.721687+0000","last_change":"2026-02-24T22:14:00.749701+0000","last_active":"2026-02-24T22:30:39.721687+0000","last_peered":"2026-02-24T22:30:39.721687+0000","last_clean":"2026-02-24T22:30:39.721687+0000","last_became_active":"2026-02-24T22:14:00.748438+0000","last_became_peered":"2026-02-24T22:14:00.748438+0000","last_unstale":"2026-02-24T22:30:39.721687+0000","last_undegraded":"2026-02-24T22:30:39.721687+0000","last_fullsized":"2026-02-24T22:30:39.721687+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":293,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:21:49.815767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":298,"num_read_kb":211,"num_write":78,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.b","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.730451+0000","last_change":"2026-02-24T22:23:29.197335+0000","last_active":"2026-02-24T22:30:14.730451+0000","last_peered":"2026-02-24T22:30:14.730451+0000","last_clean":"2026-02-24T22:30:14.730451+0000","last_became_active":"2026-02-24T22:23:29.196790+0000","last_became_peered":"2026-02-24T22:23:29.196790+0000","last_unstale":"2026-02-24T22:30:14.730451+0000","last_undegraded":"2026-02-24T22:30:14.730451+0000","last_fullsized":"2026-02-24T22:30:14.730451+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:55:13.008271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.8","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.734884+0000","last_change":"2026-02-24T22:15:25.825567+0000","last_active":"2026-02-24T22:30:39.734884+0000","last_peered":"2026-02-24T22:30:39.734884+0000","last_clean":"2026-02-24T22:30:39.734884+0000","last_became_active":"2026-02-24T22:15:25.825315+0000","last_became_peered":"2026-02-24T22:15:25.825315+0000","last_unstale":"2026-02-24T22:30:39.734884+0000","last_undegraded":"2026-02-24T22:30:39.734884+0000","last_fullsized":"2026-02-24T22:30:39.734884+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:24:39.313916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.9","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:34.732583+0000","last_change":"2026-02-24T22:15:22.811365+0000","last_active":"2026-02-24T22:30:34.732583+0000","last_peered":"2026-02-24T22:30:34.732583+0000","last_clean":"2026-02-24T22:30:34.732583+0000","last_became_active":"2026-02-24T22:15:22.811141+0000","last_became_peered":"2026-02-24T22:15:22.811141+0000","last_unstale":"2026-02-24T22:30:34.732583+0000","last_undegraded":"2026-02-24T22:30:34.732583+0000","last_fullsized":"2026-02-24T22:30:34.732583+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:45:06.384015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"79.16","version":"0'0","reported_seq":63,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.733272+0000","last_change":"2026-02-24T22:13:43.756602+0000","last_active":"2026-02-24T22:30:39.733272+0000","last_peered":"2026-02-24T22:30:39.733272+0000","last_clean":"2026-02-24T22:30:39.733272+0000","last_became_active":"2026-02-24T22:13:43.755885+0000","last_became_peered":"2026-02-24T22:13:43.755885+0000","last_unstale":"2026-02-24T22:30:39.733272+0000","last_undegraded":"2026-02-24T22:30:39.733272+0000","last_fullsized":"2026-02-24T22:30:39.733272+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:40:07.171226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.8","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.744882+0000","last_change":"2026-02-24T22:23:29.199299+0000","last_active":"2026-02-24T22:32:09.744882+0000","last_peered":"2026-02-24T22:32:09.744882+0000","last_clean":"2026-02-24T22:32:09.744882+0000","last_became_active":"2026-02-24T22:23:29.198882+0000","last_became_peered":"2026-02-24T22:23:29.198882+0000","last_unstale":"2026-02-24T22:32:09.744882+0000","last_undegraded":"2026-02-24T22:32:09.744882+0000","last_fullsized":"2026-02-24T22:32:09.744882+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:59:03.245529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"81.b","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.741464+0000","last_change":"2026-02-24T22:15:25.827664+0000","last_active":"2026-02-24T22:31:14.741464+0000","last_peered":"2026-02-24T22:31:14.741464+0000","last_clean":"2026-02-24T22:31:14.741464+0000","last_became_active":"2026-02-24T22:15:25.827265+0000","last_became_peered":"2026-02-24T22:15:25.827265+0000","last_unstale":"2026-02-24T22:31:14.741464+0000","last_undegraded":"2026-02-24T22:31:14.741464+0000","last_fullsized":"2026-02-24T22:31:14.741464+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:17:29.198425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.a","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.735221+0000","last_change":"2026-02-24T22:15:22.817325+0000","last_active":"2026-02-24T22:30:39.735221+0000","last_peered":"2026-02-24T22:30:39.735221+0000","last_clean":"2026-02-24T22:30:39.735221+0000","last_became_active":"2026-02-24T22:15:22.816702+0000","last_became_peered":"2026-02-24T22:15:22.816702+0000","last_unstale":"2026-02-24T22:30:39.735221+0000","last_undegraded":"2026-02-24T22:30:39.735221+0000","last_fullsized":"2026-02-24T22:30:39.735221+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:48:16.071491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.15","version":"307'6","reported_seq":82,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.735250+0000","last_change":"2026-02-24T22:13:43.761758+0000","last_active":"2026-02-24T22:30:39.735250+0000","last_peered":"2026-02-24T22:30:39.735250+0000","last_clean":"2026-02-24T22:30:39.735250+0000","last_became_active":"2026-02-24T22:13:43.760892+0000","last_became_peered":"2026-02-24T22:13:43.760892+0000","last_unstale":"2026-02-24T22:30:39.735250+0000","last_undegraded":"2026-02-24T22:30:39.735250+0000","last_fullsized":"2026-02-24T22:30:39.735250+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:07:44.028035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":37,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.9","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.743246+0000","last_change":"2026-02-24T22:23:29.198300+0000","last_active":"2026-02-24T22:32:09.743246+0000","last_peered":"2026-02-24T22:32:09.743246+0000","last_clean":"2026-02-24T22:32:09.743246+0000","last_became_active":"2026-02-24T22:23:29.197357+0000","last_became_peered":"2026-02-24T22:23:29.197357+0000","last_unstale":"2026-02-24T22:32:09.743246+0000","last_undegraded":"2026-02-24T22:32:09.743246+0000","last_fullsized":"2026-02-24T22:32:09.743246+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:58:07.606981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.a","version":"312'3","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:54.752765+0000","last_change":"2026-02-24T22:15:25.827469+0000","last_active":"2026-02-24T22:32:54.752765+0000","last_peered":"2026-02-24T22:32:54.752765+0000","last_clean":"2026-02-24T22:32:54.752765+0000","last_became_active":"2026-02-24T22:15:25.827055+0000","last_became_peered":"2026-02-24T22:15:25.827055+0000","last_unstale":"2026-02-24T22:32:54.752765+0000","last_undegraded":"2026-02-24T22:32:54.752765+0000","last_fullsized":"2026-02-24T22:32:54.752765+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:35:15.420239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.b","version":"312'3","reported_seq":43,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:24.753235+0000","last_change":"2026-02-24T22:15:22.817894+0000","last_active":"2026-02-24T22:33:24.753235+0000","last_peered":"2026-02-24T22:33:24.753235+0000","last_clean":"2026-02-24T22:33:24.753235+0000","last_became_active":"2026-02-24T22:15:22.817498+0000","last_became_peered":"2026-02-24T22:15:22.817498+0000","last_unstale":"2026-02-24T22:33:24.753235+0000","last_undegraded":"2026-02-24T22:33:24.753235+0000","last_fullsized":"2026-02-24T22:33:24.753235+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:08:45.124818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.14","version":"312'10","reported_seq":89,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:09.718735+0000","last_change":"2026-02-24T22:13:43.762482+0000","last_active":"2026-02-24T22:30:09.718735+0000","last_peered":"2026-02-24T22:30:09.718735+0000","last_clean":"2026-02-24T22:30:09.718735+0000","last_became_active":"2026-02-24T22:13:43.762341+0000","last_became_peered":"2026-02-24T22:13:43.762341+0000","last_unstale":"2026-02-24T22:30:09.718735+0000","last_undegraded":"2026-02-24T22:30:09.718735+0000","last_fullsized":"2026-02-24T22:30:09.718735+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:44:09.549825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":46,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.e","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.724040+0000","last_change":"2026-02-24T22:23:29.215664+0000","last_active":"2026-02-24T22:30:14.724040+0000","last_peered":"2026-02-24T22:30:14.724040+0000","last_clean":"2026-02-24T22:30:14.724040+0000","last_became_active":"2026-02-24T22:23:29.215425+0000","last_became_peered":"2026-02-24T22:23:29.215425+0000","last_unstale":"2026-02-24T22:30:14.724040+0000","last_undegraded":"2026-02-24T22:30:14.724040+0000","last_fullsized":"2026-02-24T22:30:14.724040+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:44:31.538355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.d","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.741498+0000","last_change":"2026-02-24T22:15:25.829241+0000","last_active":"2026-02-24T22:31:14.741498+0000","last_peered":"2026-02-24T22:31:14.741498+0000","last_clean":"2026-02-24T22:31:14.741498+0000","last_became_active":"2026-02-24T22:15:25.829109+0000","last_became_peered":"2026-02-24T22:15:25.829109+0000","last_unstale":"2026-02-24T22:31:14.741498+0000","last_undegraded":"2026-02-24T22:31:14.741498+0000","last_fullsized":"2026-02-24T22:31:14.741498+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:38:08.330741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.c","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.721310+0000","last_change":"2026-02-24T22:15:22.815624+0000","last_active":"2026-02-24T22:31:09.721310+0000","last_peered":"2026-02-24T22:31:09.721310+0000","last_clean":"2026-02-24T22:31:09.721310+0000","last_became_active":"2026-02-24T22:15:22.815364+0000","last_became_peered":"2026-02-24T22:15:22.815364+0000","last_unstale":"2026-02-24T22:31:09.721310+0000","last_undegraded":"2026-02-24T22:31:09.721310+0000","last_fullsized":"2026-02-24T22:31:09.721310+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:21:36.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.13","version":"312'11","reported_seq":95,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.714998+0000","last_change":"2026-02-24T22:13:43.760988+0000","last_active":"2026-02-24T22:30:04.714998+0000","last_peered":"2026-02-24T22:30:04.714998+0000","last_clean":"2026-02-24T22:30:04.714998+0000","last_became_active":"2026-02-24T22:13:43.760714+0000","last_became_peered":"2026-02-24T22:13:43.760714+0000","last_unstale":"2026-02-24T22:30:04.714998+0000","last_undegraded":"2026-02-24T22:30:04.714998+0000","last_fullsized":"2026-02-24T22:30:04.714998+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:55:23.837996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":61,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"82.f","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.730477+0000","last_change":"2026-02-24T22:23:29.199754+0000","last_active":"2026-02-24T22:30:14.730477+0000","last_peered":"2026-02-24T22:30:14.730477+0000","last_clean":"2026-02-24T22:30:14.730477+0000","last_became_active":"2026-02-24T22:23:29.199538+0000","last_became_peered":"2026-02-24T22:23:29.199538+0000","last_unstale":"2026-02-24T22:30:14.730477+0000","last_undegraded":"2026-02-24T22:30:14.730477+0000","last_fullsized":"2026-02-24T22:30:14.730477+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:15:25.304156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.c","version":"312'11","reported_seq":58,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.692318+0000","last_change":"2026-02-24T22:15:25.827267+0000","last_active":"2026-02-24T22:34:25.692318+0000","last_peered":"2026-02-24T22:34:25.692318+0000","last_clean":"2026-02-24T22:34:25.692318+0000","last_became_active":"2026-02-24T22:15:25.826430+0000","last_became_peered":"2026-02-24T22:15:25.826430+0000","last_unstale":"2026-02-24T22:34:25.692318+0000","last_undegraded":"2026-02-24T22:34:25.692318+0000","last_fullsized":"2026-02-24T22:34:25.692318+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:49:19.806352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":31746,"num_write":16,"num_write_kb":16384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.d","version":"312'5","reported_seq":52,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.807220+0000","last_change":"2026-02-24T22:15:22.813197+0000","last_active":"2026-02-24T22:34:25.807220+0000","last_peered":"2026-02-24T22:34:25.807220+0000","last_clean":"2026-02-24T22:34:25.807220+0000","last_became_active":"2026-02-24T22:15:22.813062+0000","last_became_peered":"2026-02-24T22:15:22.813062+0000","last_unstale":"2026-02-24T22:34:25.807220+0000","last_undegraded":"2026-02-24T22:34:25.807220+0000","last_fullsized":"2026-02-24T22:34:25.807220+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:08:36.468537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":10,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.12","version":"312'20","reported_seq":117,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:48.279108+0000","last_change":"2026-02-24T22:13:43.760420+0000","last_active":"2026-02-24T22:30:48.279108+0000","last_peered":"2026-02-24T22:30:48.279108+0000","last_clean":"2026-02-24T22:30:48.279108+0000","last_became_active":"2026-02-24T22:13:43.760196+0000","last_became_peered":"2026-02-24T22:13:43.760196+0000","last_unstale":"2026-02-24T22:30:48.279108+0000","last_undegraded":"2026-02-24T22:30:48.279108+0000","last_fullsized":"2026-02-24T22:30:48.279108+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:14:15.856109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":106,"num_write":52,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"82.c","version":"310'1","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.734866+0000","last_change":"2026-02-24T22:23:29.201743+0000","last_active":"2026-02-24T22:30:14.734866+0000","last_peered":"2026-02-24T22:30:14.734866+0000","last_clean":"2026-02-24T22:30:14.734866+0000","last_became_active":"2026-02-24T22:23:29.201545+0000","last_became_peered":"2026-02-24T22:23:29.201545+0000","last_unstale":"2026-02-24T22:30:14.734866+0000","last_undegraded":"2026-02-24T22:30:14.734866+0000","last_fullsized":"2026-02-24T22:30:14.734866+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:07:57.527033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"81.f","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.741445+0000","last_change":"2026-02-24T22:15:25.825325+0000","last_active":"2026-02-24T22:31:19.741445+0000","last_peered":"2026-02-24T22:31:19.741445+0000","last_clean":"2026-02-24T22:31:19.741445+0000","last_became_active":"2026-02-24T22:15:25.824862+0000","last_became_peered":"2026-02-24T22:15:25.824862+0000","last_unstale":"2026-02-24T22:31:19.741445+0000","last_undegraded":"2026-02-24T22:31:19.741445+0000","last_fullsized":"2026-02-24T22:31:19.741445+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:16:51.320757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.e","version":"312'5","reported_seq":54,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.776012+0000","last_change":"2026-02-24T22:15:22.817941+0000","last_active":"2026-02-24T22:34:25.776012+0000","last_peered":"2026-02-24T22:34:25.776012+0000","last_clean":"2026-02-24T22:34:25.776012+0000","last_became_active":"2026-02-24T22:15:22.817295+0000","last_became_peered":"2026-02-24T22:15:22.817295+0000","last_unstale":"2026-02-24T22:34:25.776012+0000","last_undegraded":"2026-02-24T22:34:25.776012+0000","last_fullsized":"2026-02-24T22:34:25.776012+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:37:42.904886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":25602,"num_write":5,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.11","version":"312'11","reported_seq":92,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.731364+0000","last_change":"2026-02-24T22:13:43.760497+0000","last_active":"2026-02-24T22:30:04.731364+0000","last_peered":"2026-02-24T22:30:04.731364+0000","last_clean":"2026-02-24T22:30:04.731364+0000","last_became_active":"2026-02-24T22:13:43.759864+0000","last_became_peered":"2026-02-24T22:13:43.759864+0000","last_unstale":"2026-02-24T22:30:04.731364+0000","last_undegraded":"2026-02-24T22:30:04.731364+0000","last_fullsized":"2026-02-24T22:30:04.731364+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:50:51.585161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":53,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.d","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.724028+0000","last_change":"2026-02-24T22:23:29.214699+0000","last_active":"2026-02-24T22:30:14.724028+0000","last_peered":"2026-02-24T22:30:14.724028+0000","last_clean":"2026-02-24T22:30:14.724028+0000","last_became_active":"2026-02-24T22:23:29.214432+0000","last_became_peered":"2026-02-24T22:23:29.214432+0000","last_unstale":"2026-02-24T22:30:14.724028+0000","last_undegraded":"2026-02-24T22:30:14.724028+0000","last_fullsized":"2026-02-24T22:30:14.724028+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:09:09.120137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.e","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.738719+0000","last_change":"2026-02-24T22:15:25.827425+0000","last_active":"2026-02-24T22:31:14.738719+0000","last_peered":"2026-02-24T22:31:14.738719+0000","last_clean":"2026-02-24T22:31:14.738719+0000","last_became_active":"2026-02-24T22:15:25.827110+0000","last_became_peered":"2026-02-24T22:15:25.827110+0000","last_unstale":"2026-02-24T22:31:14.738719+0000","last_undegraded":"2026-02-24T22:31:14.738719+0000","last_fullsized":"2026-02-24T22:31:14.738719+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:25:06.342633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.f","version":"312'8","reported_seq":58,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.821904+0000","last_change":"2026-02-24T22:15:22.816168+0000","last_active":"2026-02-24T22:34:25.821904+0000","last_peered":"2026-02-24T22:34:25.821904+0000","last_clean":"2026-02-24T22:34:25.821904+0000","last_became_active":"2026-02-24T22:15:22.815819+0000","last_became_peered":"2026-02-24T22:15:22.815819+0000","last_unstale":"2026-02-24T22:34:25.821904+0000","last_undegraded":"2026-02-24T22:34:25.821904+0000","last_fullsized":"2026-02-24T22:34:25.821904+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:39:39.503227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":26626,"num_write":13,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.10","version":"312'31","reported_seq":152,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.731343+0000","last_change":"2026-02-24T22:13:43.760545+0000","last_active":"2026-02-24T22:30:04.731343+0000","last_peered":"2026-02-24T22:30:04.731343+0000","last_clean":"2026-02-24T22:30:04.731343+0000","last_became_active":"2026-02-24T22:13:43.759855+0000","last_became_peered":"2026-02-24T22:13:43.759855+0000","last_unstale":"2026-02-24T22:30:04.731343+0000","last_undegraded":"2026-02-24T22:30:04.731343+0000","last_fullsized":"2026-02-24T22:30:04.731343+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:57:17.855796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":272,"num_read_kb":204,"num_write":72,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.12","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.725516+0000","last_change":"2026-02-24T22:23:29.212745+0000","last_active":"2026-02-24T22:30:29.725516+0000","last_peered":"2026-02-24T22:30:29.725516+0000","last_clean":"2026-02-24T22:30:29.725516+0000","last_became_active":"2026-02-24T22:23:29.212534+0000","last_became_peered":"2026-02-24T22:23:29.212534+0000","last_unstale":"2026-02-24T22:30:29.725516+0000","last_undegraded":"2026-02-24T22:30:29.725516+0000","last_fullsized":"2026-02-24T22:30:29.725516+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:50:08.072181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.11","version":"312'5","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.471093+0000","last_change":"2026-02-24T22:15:25.827242+0000","last_active":"2026-02-24T22:34:25.471093+0000","last_peered":"2026-02-24T22:34:25.471093+0000","last_clean":"2026-02-24T22:34:25.471093+0000","last_became_active":"2026-02-24T22:15:25.826517+0000","last_became_peered":"2026-02-24T22:15:25.826517+0000","last_unstale":"2026-02-24T22:34:25.471093+0000","last_undegraded":"2026-02-24T22:34:25.471093+0000","last_fullsized":"2026-02-24T22:34:25.471093+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:24:40.732721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":10,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.10","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.741867+0000","last_change":"2026-02-24T22:15:22.810414+0000","last_active":"2026-02-24T22:31:44.741867+0000","last_peered":"2026-02-24T22:31:44.741867+0000","last_clean":"2026-02-24T22:31:44.741867+0000","last_became_active":"2026-02-24T22:15:22.810000+0000","last_became_peered":"2026-02-24T22:15:22.810000+0000","last_unstale":"2026-02-24T22:31:44.741867+0000","last_undegraded":"2026-02-24T22:31:44.741867+0000","last_fullsized":"2026-02-24T22:31:44.741867+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:21:42.629513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"79.f","version":"312'11","reported_seq":95,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742413+0000","last_change":"2026-02-24T22:13:43.755156+0000","last_active":"2026-02-24T22:31:49.742413+0000","last_peered":"2026-02-24T22:31:49.742413+0000","last_clean":"2026-02-24T22:31:49.742413+0000","last_became_active":"2026-02-24T22:13:43.754602+0000","last_became_peered":"2026-02-24T22:13:43.754602+0000","last_unstale":"2026-02-24T22:31:49.742413+0000","last_undegraded":"2026-02-24T22:31:49.742413+0000","last_fullsized":"2026-02-24T22:31:49.742413+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:13:53.006921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":67,"num_write":27,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.13","version":"310'1","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.735479+0000","last_change":"2026-02-24T22:23:29.198937+0000","last_active":"2026-02-24T22:30:29.735479+0000","last_peered":"2026-02-24T22:30:29.735479+0000","last_clean":"2026-02-24T22:30:29.735479+0000","last_became_active":"2026-02-24T22:23:29.198627+0000","last_became_peered":"2026-02-24T22:23:29.198627+0000","last_unstale":"2026-02-24T22:30:29.735479+0000","last_undegraded":"2026-02-24T22:30:29.735479+0000","last_fullsized":"2026-02-24T22:30:29.735479+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:28:29.061001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.10","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744722+0000","last_change":"2026-02-24T22:15:25.827252+0000","last_active":"2026-02-24T22:31:49.744722+0000","last_peered":"2026-02-24T22:31:49.744722+0000","last_clean":"2026-02-24T22:31:49.744722+0000","last_became_active":"2026-02-24T22:15:25.826606+0000","last_became_peered":"2026-02-24T22:15:25.826606+0000","last_unstale":"2026-02-24T22:31:49.744722+0000","last_undegraded":"2026-02-24T22:31:49.744722+0000","last_fullsized":"2026-02-24T22:31:49.744722+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:30:33.376941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.11","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.741885+0000","last_change":"2026-02-24T22:15:22.810459+0000","last_active":"2026-02-24T22:31:44.741885+0000","last_peered":"2026-02-24T22:31:44.741885+0000","last_clean":"2026-02-24T22:31:44.741885+0000","last_became_active":"2026-02-24T22:15:22.809791+0000","last_became_peered":"2026-02-24T22:15:22.809791+0000","last_unstale":"2026-02-24T22:31:44.741885+0000","last_undegraded":"2026-02-24T22:31:44.741885+0000","last_fullsized":"2026-02-24T22:31:44.741885+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:37:39.655379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"79.e","version":"312'23","reported_seq":126,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.741917+0000","last_change":"2026-02-24T22:13:43.756185+0000","last_active":"2026-02-24T22:31:44.741917+0000","last_peered":"2026-02-24T22:31:44.741917+0000","last_clean":"2026-02-24T22:31:44.741917+0000","last_became_active":"2026-02-24T22:13:43.755868+0000","last_became_peered":"2026-02-24T22:13:43.755868+0000","last_unstale":"2026-02-24T22:31:44.741917+0000","last_undegraded":"2026-02-24T22:31:44.741917+0000","last_fullsized":"2026-02-24T22:31:44.741917+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:56:04.041994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":169,"num_read_kb":112,"num_write":56,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.10","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.734148+0000","last_change":"2026-02-24T22:23:29.212895+0000","last_active":"2026-02-24T22:30:29.734148+0000","last_peered":"2026-02-24T22:30:29.734148+0000","last_clean":"2026-02-24T22:30:29.734148+0000","last_became_active":"2026-02-24T22:23:29.212665+0000","last_became_peered":"2026-02-24T22:23:29.212665+0000","last_unstale":"2026-02-24T22:30:29.734148+0000","last_undegraded":"2026-02-24T22:30:29.734148+0000","last_fullsized":"2026-02-24T22:30:29.734148+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:30:27.871463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.13","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.740862+0000","last_change":"2026-02-24T22:15:25.826171+0000","last_active":"2026-02-24T22:31:49.740862+0000","last_peered":"2026-02-24T22:31:49.740862+0000","last_clean":"2026-02-24T22:31:49.740862+0000","last_became_active":"2026-02-24T22:15:25.825862+0000","last_became_peered":"2026-02-24T22:15:25.825862+0000","last_unstale":"2026-02-24T22:31:49.740862+0000","last_undegraded":"2026-02-24T22:31:49.740862+0000","last_fullsized":"2026-02-24T22:31:49.740862+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:02:37.431216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"80.12","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.744628+0000","last_change":"2026-02-24T22:15:22.812712+0000","last_active":"2026-02-24T22:31:44.744628+0000","last_peered":"2026-02-24T22:31:44.744628+0000","last_clean":"2026-02-24T22:31:44.744628+0000","last_became_active":"2026-02-24T22:15:22.812405+0000","last_became_peered":"2026-02-24T22:15:22.812405+0000","last_unstale":"2026-02-24T22:31:44.744628+0000","last_undegraded":"2026-02-24T22:31:44.744628+0000","last_fullsized":"2026-02-24T22:31:44.744628+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:36:00.971366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.d","version":"312'20","reported_seq":113,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.733217+0000","last_change":"2026-02-24T22:13:43.761593+0000","last_active":"2026-02-24T22:31:44.733217+0000","last_peered":"2026-02-24T22:31:44.733217+0000","last_clean":"2026-02-24T22:31:44.733217+0000","last_became_active":"2026-02-24T22:13:43.761339+0000","last_became_peered":"2026-02-24T22:13:43.761339+0000","last_unstale":"2026-02-24T22:31:44.733217+0000","last_undegraded":"2026-02-24T22:31:44.733217+0000","last_fullsized":"2026-02-24T22:31:44.733217+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:02:40.622911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":126,"num_read_kb":76,"num_write":49,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"82.11","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.720712+0000","last_change":"2026-02-24T22:23:29.214793+0000","last_active":"2026-02-24T22:30:29.720712+0000","last_peered":"2026-02-24T22:30:29.720712+0000","last_clean":"2026-02-24T22:30:29.720712+0000","last_became_active":"2026-02-24T22:23:29.214167+0000","last_became_peered":"2026-02-24T22:23:29.214167+0000","last_unstale":"2026-02-24T22:30:29.720712+0000","last_undegraded":"2026-02-24T22:30:29.720712+0000","last_fullsized":"2026-02-24T22:30:29.720712+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:21:51.443877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.12","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744756+0000","last_change":"2026-02-24T22:15:25.826465+0000","last_active":"2026-02-24T22:31:49.744756+0000","last_peered":"2026-02-24T22:31:49.744756+0000","last_clean":"2026-02-24T22:31:49.744756+0000","last_became_active":"2026-02-24T22:15:25.826102+0000","last_became_peered":"2026-02-24T22:15:25.826102+0000","last_unstale":"2026-02-24T22:31:49.744756+0000","last_undegraded":"2026-02-24T22:31:49.744756+0000","last_fullsized":"2026-02-24T22:31:49.744756+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:22:01.000780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.13","version":"312'3","reported_seq":43,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:04.760554+0000","last_change":"2026-02-24T22:15:22.812393+0000","last_active":"2026-02-24T22:34:04.760554+0000","last_peered":"2026-02-24T22:34:04.760554+0000","last_clean":"2026-02-24T22:34:04.760554+0000","last_became_active":"2026-02-24T22:15:22.811927+0000","last_became_peered":"2026-02-24T22:15:22.811927+0000","last_unstale":"2026-02-24T22:34:04.760554+0000","last_undegraded":"2026-02-24T22:34:04.760554+0000","last_fullsized":"2026-02-24T22:34:04.760554+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:54:16.404989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.c","version":"312'4","reported_seq":74,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.724938+0000","last_change":"2026-02-24T22:13:43.762114+0000","last_active":"2026-02-24T22:31:44.724938+0000","last_peered":"2026-02-24T22:31:44.724938+0000","last_clean":"2026-02-24T22:31:44.724938+0000","last_became_active":"2026-02-24T22:13:43.761910+0000","last_became_peered":"2026-02-24T22:13:43.761910+0000","last_unstale":"2026-02-24T22:31:44.724938+0000","last_undegraded":"2026-02-24T22:31:44.724938+0000","last_fullsized":"2026-02-24T22:31:44.724938+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:51:11.166418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":15,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"82.16","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.718767+0000","last_change":"2026-02-24T22:23:29.213342+0000","last_active":"2026-02-24T22:30:44.718767+0000","last_peered":"2026-02-24T22:30:44.718767+0000","last_clean":"2026-02-24T22:30:44.718767+0000","last_became_active":"2026-02-24T22:23:29.213080+0000","last_became_peered":"2026-02-24T22:23:29.213080+0000","last_unstale":"2026-02-24T22:30:44.718767+0000","last_undegraded":"2026-02-24T22:30:44.718767+0000","last_fullsized":"2026-02-24T22:30:44.718767+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:33:29.031496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.15","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:59.714511+0000","last_change":"2026-02-24T22:15:25.831361+0000","last_active":"2026-02-24T22:29:59.714511+0000","last_peered":"2026-02-24T22:29:59.714511+0000","last_clean":"2026-02-24T22:29:59.714511+0000","last_became_active":"2026-02-24T22:15:25.830922+0000","last_became_peered":"2026-02-24T22:15:25.830922+0000","last_unstale":"2026-02-24T22:29:59.714511+0000","last_undegraded":"2026-02-24T22:29:59.714511+0000","last_fullsized":"2026-02-24T22:29:59.714511+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:08:07.538696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"80.14","version":"312'3","reported_seq":43,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:09.761022+0000","last_change":"2026-02-24T22:15:22.812446+0000","last_active":"2026-02-24T22:34:09.761022+0000","last_peered":"2026-02-24T22:34:09.761022+0000","last_clean":"2026-02-24T22:34:09.761022+0000","last_became_active":"2026-02-24T22:15:22.811924+0000","last_became_peered":"2026-02-24T22:15:22.811924+0000","last_unstale":"2026-02-24T22:34:09.761022+0000","last_undegraded":"2026-02-24T22:34:09.761022+0000","last_fullsized":"2026-02-24T22:34:09.761022+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:18:30.397097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.b","version":"312'13","reported_seq":98,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.740368+0000","last_change":"2026-02-24T22:13:43.757072+0000","last_active":"2026-02-24T22:31:44.740368+0000","last_peered":"2026-02-24T22:31:44.740368+0000","last_clean":"2026-02-24T22:31:44.740368+0000","last_became_active":"2026-02-24T22:13:43.756596+0000","last_became_peered":"2026-02-24T22:13:43.756596+0000","last_unstale":"2026-02-24T22:31:44.740368+0000","last_undegraded":"2026-02-24T22:31:44.740368+0000","last_fullsized":"2026-02-24T22:31:44.740368+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:30:51.803180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":56,"num_write":37,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.17","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.718784+0000","last_change":"2026-02-24T22:23:29.214456+0000","last_active":"2026-02-24T22:30:44.718784+0000","last_peered":"2026-02-24T22:30:44.718784+0000","last_clean":"2026-02-24T22:30:44.718784+0000","last_became_active":"2026-02-24T22:23:29.214217+0000","last_became_peered":"2026-02-24T22:23:29.214217+0000","last_unstale":"2026-02-24T22:30:44.718784+0000","last_undegraded":"2026-02-24T22:30:44.718784+0000","last_fullsized":"2026-02-24T22:30:44.718784+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:48:46.666871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.14","version":"312'5","reported_seq":42,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.471281+0000","last_change":"2026-02-24T22:15:25.827827+0000","last_active":"2026-02-24T22:34:25.471281+0000","last_peered":"2026-02-24T22:34:25.471281+0000","last_clean":"2026-02-24T22:34:25.471281+0000","last_became_active":"2026-02-24T22:15:25.827557+0000","last_became_peered":"2026-02-24T22:15:25.827557+0000","last_unstale":"2026-02-24T22:34:25.471281+0000","last_undegraded":"2026-02-24T22:34:25.471281+0000","last_fullsized":"2026-02-24T22:34:25.471281+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:13:31.380536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":10,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.15","version":"312'5","reported_seq":47,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.807267+0000","last_change":"2026-02-24T22:15:22.812814+0000","last_active":"2026-02-24T22:34:25.807267+0000","last_peered":"2026-02-24T22:34:25.807267+0000","last_clean":"2026-02-24T22:34:25.807267+0000","last_became_active":"2026-02-24T22:15:22.812450+0000","last_became_peered":"2026-02-24T22:15:22.812450+0000","last_unstale":"2026-02-24T22:34:25.807267+0000","last_undegraded":"2026-02-24T22:34:25.807267+0000","last_fullsized":"2026-02-24T22:34:25.807267+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:22:29.140595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":10,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.a","version":"312'10","reported_seq":90,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.724683+0000","last_change":"2026-02-24T22:13:43.762418+0000","last_active":"2026-02-24T22:31:09.724683+0000","last_peered":"2026-02-24T22:31:09.724683+0000","last_clean":"2026-02-24T22:31:09.724683+0000","last_became_active":"2026-02-24T22:13:43.762226+0000","last_became_peered":"2026-02-24T22:13:43.762226+0000","last_unstale":"2026-02-24T22:31:09.724683+0000","last_undegraded":"2026-02-24T22:31:09.724683+0000","last_fullsized":"2026-02-24T22:31:09.724683+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:41:14.108185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":42,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.14","version":"309'1","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.735499+0000","last_change":"2026-02-24T22:24:01.299030+0000","last_active":"2026-02-24T22:30:29.735499+0000","last_peered":"2026-02-24T22:30:29.735499+0000","last_clean":"2026-02-24T22:30:29.735499+0000","last_became_active":"2026-02-24T22:24:01.297193+0000","last_became_peered":"2026-02-24T22:24:01.297193+0000","last_unstale":"2026-02-24T22:30:29.735499+0000","last_undegraded":"2026-02-24T22:30:29.735499+0000","last_fullsized":"2026-02-24T22:30:29.735499+0000","mapping_epoch":311,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":312,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:15:53.769966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.17","version":"312'3","reported_seq":35,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.746728+0000","last_change":"2026-02-24T22:15:25.833669+0000","last_active":"2026-02-24T22:32:24.746728+0000","last_peered":"2026-02-24T22:32:24.746728+0000","last_clean":"2026-02-24T22:32:24.746728+0000","last_became_active":"2026-02-24T22:15:25.833430+0000","last_became_peered":"2026-02-24T22:15:25.833430+0000","last_unstale":"2026-02-24T22:32:24.746728+0000","last_undegraded":"2026-02-24T22:32:24.746728+0000","last_fullsized":"2026-02-24T22:32:24.746728+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:41:23.281878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"80.16","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:54.714013+0000","last_change":"2026-02-24T22:15:22.817544+0000","last_active":"2026-02-24T22:29:54.714013+0000","last_peered":"2026-02-24T22:29:54.714013+0000","last_clean":"2026-02-24T22:29:54.714013+0000","last_became_active":"2026-02-24T22:15:22.817150+0000","last_became_peered":"2026-02-24T22:15:22.817150+0000","last_unstale":"2026-02-24T22:29:54.714013+0000","last_undegraded":"2026-02-24T22:29:54.714013+0000","last_fullsized":"2026-02-24T22:29:54.714013+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:30:27.018037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.9","version":"312'24","reported_seq":125,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.737699+0000","last_change":"2026-02-24T22:13:43.757041+0000","last_active":"2026-02-24T22:31:04.737699+0000","last_peered":"2026-02-24T22:31:04.737699+0000","last_clean":"2026-02-24T22:31:04.737699+0000","last_became_active":"2026-02-24T22:13:43.756777+0000","last_became_peered":"2026-02-24T22:13:43.756777+0000","last_unstale":"2026-02-24T22:31:04.737699+0000","last_undegraded":"2026-02-24T22:31:04.737699+0000","last_fullsized":"2026-02-24T22:31:04.737699+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:54:52.853203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":107,"num_write":60,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.15","version":"310'1","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.733776+0000","last_change":"2026-02-24T22:23:29.198331+0000","last_active":"2026-02-24T22:30:44.733776+0000","last_peered":"2026-02-24T22:30:44.733776+0000","last_clean":"2026-02-24T22:30:44.733776+0000","last_became_active":"2026-02-24T22:23:29.198000+0000","last_became_peered":"2026-02-24T22:23:29.198000+0000","last_unstale":"2026-02-24T22:30:44.733776+0000","last_undegraded":"2026-02-24T22:30:44.733776+0000","last_fullsized":"2026-02-24T22:30:44.733776+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:34:21.768724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.16","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:59.732575+0000","last_change":"2026-02-24T22:15:25.828675+0000","last_active":"2026-02-24T22:29:59.732575+0000","last_peered":"2026-02-24T22:29:59.732575+0000","last_clean":"2026-02-24T22:29:59.732575+0000","last_became_active":"2026-02-24T22:15:25.828066+0000","last_became_peered":"2026-02-24T22:15:25.828066+0000","last_unstale":"2026-02-24T22:29:59.732575+0000","last_undegraded":"2026-02-24T22:29:59.732575+0000","last_fullsized":"2026-02-24T22:29:59.732575+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:13:41.162339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.17","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:54.730332+0000","last_change":"2026-02-24T22:15:22.818517+0000","last_active":"2026-02-24T22:29:54.730332+0000","last_peered":"2026-02-24T22:29:54.730332+0000","last_clean":"2026-02-24T22:29:54.730332+0000","last_became_active":"2026-02-24T22:15:22.817950+0000","last_became_peered":"2026-02-24T22:15:22.817950+0000","last_unstale":"2026-02-24T22:29:54.730332+0000","last_undegraded":"2026-02-24T22:29:54.730332+0000","last_fullsized":"2026-02-24T22:29:54.730332+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:47:07.389545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.8","version":"312'26","reported_seq":136,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.737895+0000","last_change":"2026-02-24T22:13:43.761888+0000","last_active":"2026-02-24T22:31:04.737895+0000","last_peered":"2026-02-24T22:31:04.737895+0000","last_clean":"2026-02-24T22:31:04.737895+0000","last_became_active":"2026-02-24T22:13:43.761558+0000","last_became_peered":"2026-02-24T22:13:43.761558+0000","last_unstale":"2026-02-24T22:31:04.737895+0000","last_undegraded":"2026-02-24T22:31:04.737895+0000","last_fullsized":"2026-02-24T22:31:04.737895+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:22:52.093177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":152,"num_write":62,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.1a","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.723700+0000","last_change":"2026-02-24T22:23:29.213344+0000","last_active":"2026-02-24T22:30:59.723700+0000","last_peered":"2026-02-24T22:30:59.723700+0000","last_clean":"2026-02-24T22:30:59.723700+0000","last_became_active":"2026-02-24T22:23:29.212709+0000","last_became_peered":"2026-02-24T22:23:29.212709+0000","last_unstale":"2026-02-24T22:30:59.723700+0000","last_undegraded":"2026-02-24T22:30:59.723700+0000","last_fullsized":"2026-02-24T22:30:59.723700+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:26:30.998666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.19","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.718214+0000","last_change":"2026-02-24T22:15:25.832162+0000","last_active":"2026-02-24T22:30:04.718214+0000","last_peered":"2026-02-24T22:30:04.718214+0000","last_clean":"2026-02-24T22:30:04.718214+0000","last_became_active":"2026-02-24T22:15:25.831885+0000","last_became_peered":"2026-02-24T22:15:25.831885+0000","last_unstale":"2026-02-24T22:30:04.718214+0000","last_undegraded":"2026-02-24T22:30:04.718214+0000","last_fullsized":"2026-02-24T22:30:04.718214+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:54:09.173036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.18","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:54.730283+0000","last_change":"2026-02-24T22:15:22.810683+0000","last_active":"2026-02-24T22:29:54.730283+0000","last_peered":"2026-02-24T22:29:54.730283+0000","last_clean":"2026-02-24T22:29:54.730283+0000","last_became_active":"2026-02-24T22:15:22.810156+0000","last_became_peered":"2026-02-24T22:15:22.810156+0000","last_unstale":"2026-02-24T22:29:54.730283+0000","last_undegraded":"2026-02-24T22:29:54.730283+0000","last_fullsized":"2026-02-24T22:29:54.730283+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:29:33.293117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"79.7","version":"0'0","reported_seq":64,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.740460+0000","last_change":"2026-02-24T22:13:43.756250+0000","last_active":"2026-02-24T22:31:04.740460+0000","last_peered":"2026-02-24T22:31:04.740460+0000","last_clean":"2026-02-24T22:31:04.740460+0000","last_became_active":"2026-02-24T22:13:43.755957+0000","last_became_peered":"2026-02-24T22:13:43.755957+0000","last_unstale":"2026-02-24T22:31:04.740460+0000","last_undegraded":"2026-02-24T22:31:04.740460+0000","last_fullsized":"2026-02-24T22:31:04.740460+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:16:21.505648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.1b","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.735465+0000","last_change":"2026-02-24T22:23:29.200410+0000","last_active":"2026-02-24T22:30:59.735465+0000","last_peered":"2026-02-24T22:30:59.735465+0000","last_clean":"2026-02-24T22:30:59.735465+0000","last_became_active":"2026-02-24T22:23:29.200152+0000","last_became_peered":"2026-02-24T22:23:29.200152+0000","last_unstale":"2026-02-24T22:30:59.735465+0000","last_undegraded":"2026-02-24T22:30:59.735465+0000","last_fullsized":"2026-02-24T22:30:59.735465+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:24:51.029708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.18","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:59.730819+0000","last_change":"2026-02-24T22:15:25.824057+0000","last_active":"2026-02-24T22:29:59.730819+0000","last_peered":"2026-02-24T22:29:59.730819+0000","last_clean":"2026-02-24T22:29:59.730819+0000","last_became_active":"2026-02-24T22:15:25.823415+0000","last_became_peered":"2026-02-24T22:15:25.823415+0000","last_unstale":"2026-02-24T22:29:59.730819+0000","last_undegraded":"2026-02-24T22:29:59.730819+0000","last_fullsized":"2026-02-24T22:29:59.730819+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:00:21.357856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.19","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:59.714470+0000","last_change":"2026-02-24T22:15:22.817214+0000","last_active":"2026-02-24T22:29:59.714470+0000","last_peered":"2026-02-24T22:29:59.714470+0000","last_clean":"2026-02-24T22:29:59.714470+0000","last_became_active":"2026-02-24T22:15:22.816826+0000","last_became_peered":"2026-02-24T22:15:22.816826+0000","last_unstale":"2026-02-24T22:29:59.714470+0000","last_undegraded":"2026-02-24T22:29:59.714470+0000","last_fullsized":"2026-02-24T22:29:59.714470+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:13:51.793483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.6","version":"301'14","reported_seq":108,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.739656+0000","last_change":"2026-02-24T22:13:43.755144+0000","last_active":"2026-02-24T22:31:04.739656+0000","last_peered":"2026-02-24T22:31:04.739656+0000","last_clean":"2026-02-24T22:31:04.739656+0000","last_became_active":"2026-02-24T22:13:43.754753+0000","last_became_peered":"2026-02-24T22:13:43.754753+0000","last_unstale":"2026-02-24T22:31:04.739656+0000","last_undegraded":"2026-02-24T22:31:04.739656+0000","last_fullsized":"2026-02-24T22:31:04.739656+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:57:00.870335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":92,"num_write":34,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"82.18","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.726953+0000","last_change":"2026-02-24T22:23:29.213704+0000","last_active":"2026-02-24T22:30:44.726953+0000","last_peered":"2026-02-24T22:30:44.726953+0000","last_clean":"2026-02-24T22:30:44.726953+0000","last_became_active":"2026-02-24T22:23:29.213387+0000","last_became_peered":"2026-02-24T22:23:29.213387+0000","last_unstale":"2026-02-24T22:30:44.726953+0000","last_undegraded":"2026-02-24T22:30:44.726953+0000","last_fullsized":"2026-02-24T22:30:44.726953+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:46:11.186061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.1b","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.720738+0000","last_change":"2026-02-24T22:15:25.832293+0000","last_active":"2026-02-24T22:30:29.720738+0000","last_peered":"2026-02-24T22:30:29.720738+0000","last_clean":"2026-02-24T22:30:29.720738+0000","last_became_active":"2026-02-24T22:15:25.832097+0000","last_became_peered":"2026-02-24T22:15:25.832097+0000","last_unstale":"2026-02-24T22:30:29.720738+0000","last_undegraded":"2026-02-24T22:30:29.720738+0000","last_fullsized":"2026-02-24T22:30:29.720738+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:19:51.400688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.1a","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.720229+0000","last_change":"2026-02-24T22:15:22.819032+0000","last_active":"2026-02-24T22:30:24.720229+0000","last_peered":"2026-02-24T22:30:24.720229+0000","last_clean":"2026-02-24T22:30:24.720229+0000","last_became_active":"2026-02-24T22:15:22.818885+0000","last_became_peered":"2026-02-24T22:15:22.818885+0000","last_unstale":"2026-02-24T22:30:24.720229+0000","last_undegraded":"2026-02-24T22:30:24.720229+0000","last_fullsized":"2026-02-24T22:30:24.720229+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:15:52.421379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"79.5","version":"312'19","reported_seq":121,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:49.005257+0000","last_change":"2026-02-24T22:13:43.756043+0000","last_active":"2026-02-24T22:30:49.005257+0000","last_peered":"2026-02-24T22:30:49.005257+0000","last_clean":"2026-02-24T22:30:49.005257+0000","last_became_active":"2026-02-24T22:13:43.755495+0000","last_became_peered":"2026-02-24T22:13:43.755495+0000","last_unstale":"2026-02-24T22:30:49.005257+0000","last_undegraded":"2026-02-24T22:30:49.005257+0000","last_fullsized":"2026-02-24T22:30:49.005257+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:47:41.399446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":164,"num_read_kb":112,"num_write":47,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.19","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:49.719308+0000","last_change":"2026-02-24T22:23:29.212738+0000","last_active":"2026-02-24T22:30:49.719308+0000","last_peered":"2026-02-24T22:30:49.719308+0000","last_clean":"2026-02-24T22:30:49.719308+0000","last_became_active":"2026-02-24T22:23:29.212268+0000","last_became_peered":"2026-02-24T22:23:29.212268+0000","last_unstale":"2026-02-24T22:30:49.719308+0000","last_undegraded":"2026-02-24T22:30:49.719308+0000","last_fullsized":"2026-02-24T22:30:49.719308+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:32:25.776768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.1a","version":"312'3","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:59.748342+0000","last_change":"2026-02-24T22:15:25.826222+0000","last_active":"2026-02-24T22:32:59.748342+0000","last_peered":"2026-02-24T22:32:59.748342+0000","last_clean":"2026-02-24T22:32:59.748342+0000","last_became_active":"2026-02-24T22:15:25.825969+0000","last_became_peered":"2026-02-24T22:15:25.825969+0000","last_unstale":"2026-02-24T22:32:59.748342+0000","last_undegraded":"2026-02-24T22:32:59.748342+0000","last_fullsized":"2026-02-24T22:32:59.748342+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:29:05.133831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"80.1b","version":"312'3","reported_seq":42,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:54.750003+0000","last_change":"2026-02-24T22:15:22.816734+0000","last_active":"2026-02-24T22:32:54.750003+0000","last_peered":"2026-02-24T22:32:54.750003+0000","last_clean":"2026-02-24T22:32:54.750003+0000","last_became_active":"2026-02-24T22:15:22.816479+0000","last_became_peered":"2026-02-24T22:15:22.816479+0000","last_unstale":"2026-02-24T22:32:54.750003+0000","last_undegraded":"2026-02-24T22:32:54.750003+0000","last_fullsized":"2026-02-24T22:32:54.750003+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:57:39.996802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.4","version":"0'0","reported_seq":70,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.735005+0000","last_change":"2026-02-24T22:14:00.742004+0000","last_active":"2026-02-24T22:30:24.735005+0000","last_peered":"2026-02-24T22:30:24.735005+0000","last_clean":"2026-02-24T22:30:24.735005+0000","last_became_active":"2026-02-24T22:14:00.741839+0000","last_became_peered":"2026-02-24T22:14:00.741839+0000","last_unstale":"2026-02-24T22:30:24.735005+0000","last_undegraded":"2026-02-24T22:30:24.735005+0000","last_fullsized":"2026-02-24T22:30:24.735005+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":293,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:38:11.152954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"82.1e","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.727290+0000","last_change":"2026-02-24T22:23:29.215779+0000","last_active":"2026-02-24T22:31:34.727290+0000","last_peered":"2026-02-24T22:31:34.727290+0000","last_clean":"2026-02-24T22:31:34.727290+0000","last_became_active":"2026-02-24T22:23:29.215604+0000","last_became_peered":"2026-02-24T22:23:29.215604+0000","last_unstale":"2026-02-24T22:31:34.727290+0000","last_undegraded":"2026-02-24T22:31:34.727290+0000","last_fullsized":"2026-02-24T22:31:34.727290+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:58:56.390417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.1d","version":"312'3","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:59.750434+0000","last_change":"2026-02-24T22:15:25.828517+0000","last_active":"2026-02-24T22:32:59.750434+0000","last_peered":"2026-02-24T22:32:59.750434+0000","last_clean":"2026-02-24T22:32:59.750434+0000","last_became_active":"2026-02-24T22:15:25.828335+0000","last_became_peered":"2026-02-24T22:15:25.828335+0000","last_unstale":"2026-02-24T22:32:59.750434+0000","last_undegraded":"2026-02-24T22:32:59.750434+0000","last_fullsized":"2026-02-24T22:32:59.750434+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:18:15.606086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.1c","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.733619+0000","last_change":"2026-02-24T22:15:22.816853+0000","last_active":"2026-02-24T22:30:24.733619+0000","last_peered":"2026-02-24T22:30:24.733619+0000","last_clean":"2026-02-24T22:30:24.733619+0000","last_became_active":"2026-02-24T22:15:22.816548+0000","last_became_peered":"2026-02-24T22:15:22.816548+0000","last_unstale":"2026-02-24T22:30:24.733619+0000","last_undegraded":"2026-02-24T22:30:24.733619+0000","last_fullsized":"2026-02-24T22:30:24.733619+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:00:55.883988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.3","version":"312'6","reported_seq":77,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.731442+0000","last_change":"2026-02-24T22:13:43.756810+0000","last_active":"2026-02-24T22:30:24.731442+0000","last_peered":"2026-02-24T22:30:24.731442+0000","last_clean":"2026-02-24T22:30:24.731442+0000","last_became_active":"2026-02-24T22:13:43.756408+0000","last_became_peered":"2026-02-24T22:13:43.756408+0000","last_unstale":"2026-02-24T22:30:24.731442+0000","last_undegraded":"2026-02-24T22:30:24.731442+0000","last_fullsized":"2026-02-24T22:30:24.731442+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:24:33.412598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":20,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.1f","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744790+0000","last_change":"2026-02-24T22:23:29.199410+0000","last_active":"2026-02-24T22:31:49.744790+0000","last_peered":"2026-02-24T22:31:49.744790+0000","last_clean":"2026-02-24T22:31:49.744790+0000","last_became_active":"2026-02-24T22:23:29.199179+0000","last_became_peered":"2026-02-24T22:23:29.199179+0000","last_unstale":"2026-02-24T22:31:49.744790+0000","last_undegraded":"2026-02-24T22:31:49.744790+0000","last_fullsized":"2026-02-24T22:31:49.744790+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:21:23.823724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.1c","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.725548+0000","last_change":"2026-02-24T22:15:25.834316+0000","last_active":"2026-02-24T22:30:29.725548+0000","last_peered":"2026-02-24T22:30:29.725548+0000","last_clean":"2026-02-24T22:30:29.725548+0000","last_became_active":"2026-02-24T22:15:25.834072+0000","last_became_peered":"2026-02-24T22:15:25.834072+0000","last_unstale":"2026-02-24T22:30:29.725548+0000","last_undegraded":"2026-02-24T22:30:29.725548+0000","last_fullsized":"2026-02-24T22:30:29.725548+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:45:39.925744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"80.1d","version":"312'2","reported_seq":40,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:54.750046+0000","last_change":"2026-02-24T22:15:22.817173+0000","last_active":"2026-02-24T22:32:54.750046+0000","last_peered":"2026-02-24T22:32:54.750046+0000","last_clean":"2026-02-24T22:32:54.750046+0000","last_became_active":"2026-02-24T22:15:22.816615+0000","last_became_peered":"2026-02-24T22:15:22.816615+0000","last_unstale":"2026-02-24T22:32:54.750046+0000","last_undegraded":"2026-02-24T22:32:54.750046+0000","last_fullsized":"2026-02-24T22:32:54.750046+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:25:34.323299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.2","version":"312'5","reported_seq":77,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.733168+0000","last_change":"2026-02-24T22:13:43.755888+0000","last_active":"2026-02-24T22:30:24.733168+0000","last_peered":"2026-02-24T22:30:24.733168+0000","last_clean":"2026-02-24T22:30:24.733168+0000","last_became_active":"2026-02-24T22:13:43.755241+0000","last_became_peered":"2026-02-24T22:13:43.755241+0000","last_unstale":"2026-02-24T22:30:24.733168+0000","last_undegraded":"2026-02-24T22:30:24.733168+0000","last_fullsized":"2026-02-24T22:30:24.733168+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:13:28.809470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":23,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.1c","version":"310'1","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.720361+0000","last_change":"2026-02-24T22:23:29.213715+0000","last_active":"2026-02-24T22:30:59.720361+0000","last_peered":"2026-02-24T22:30:59.720361+0000","last_clean":"2026-02-24T22:30:59.720361+0000","last_became_active":"2026-02-24T22:23:29.213490+0000","last_became_peered":"2026-02-24T22:23:29.213490+0000","last_unstale":"2026-02-24T22:30:59.720361+0000","last_undegraded":"2026-02-24T22:30:59.720361+0000","last_fullsized":"2026-02-24T22:30:59.720361+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:21:40.032070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.1f","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.747978+0000","last_change":"2026-02-24T22:15:25.828181+0000","last_active":"2026-02-24T22:32:09.747978+0000","last_peered":"2026-02-24T22:32:09.747978+0000","last_clean":"2026-02-24T22:32:09.747978+0000","last_became_active":"2026-02-24T22:15:25.827873+0000","last_became_peered":"2026-02-24T22:15:25.827873+0000","last_unstale":"2026-02-24T22:32:09.747978+0000","last_undegraded":"2026-02-24T22:32:09.747978+0000","last_fullsized":"2026-02-24T22:32:09.747978+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:19:57.290828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.1e","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:29.726743+0000","last_change":"2026-02-24T22:15:22.817535+0000","last_active":"2026-02-24T22:31:29.726743+0000","last_peered":"2026-02-24T22:31:29.726743+0000","last_clean":"2026-02-24T22:31:29.726743+0000","last_became_active":"2026-02-24T22:15:22.817337+0000","last_became_peered":"2026-02-24T22:15:22.817337+0000","last_unstale":"2026-02-24T22:31:29.726743+0000","last_undegraded":"2026-02-24T22:31:29.726743+0000","last_fullsized":"2026-02-24T22:31:29.726743+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:02:48.141978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"79.1","version":"312'23","reported_seq":125,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.720202+0000","last_change":"2026-02-24T22:13:43.761201+0000","last_active":"2026-02-24T22:30:24.720202+0000","last_peered":"2026-02-24T22:30:24.720202+0000","last_clean":"2026-02-24T22:30:24.720202+0000","last_became_active":"2026-02-24T22:13:43.760933+0000","last_became_peered":"2026-02-24T22:13:43.760933+0000","last_unstale":"2026-02-24T22:30:24.720202+0000","last_undegraded":"2026-02-24T22:30:24.720202+0000","last_fullsized":"2026-02-24T22:30:24.720202+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:18:32.108308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":174,"num_read_kb":113,"num_write":59,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.1d","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.723721+0000","last_change":"2026-02-24T22:23:29.215553+0000","last_active":"2026-02-24T22:30:59.723721+0000","last_peered":"2026-02-24T22:30:59.723721+0000","last_clean":"2026-02-24T22:30:59.723721+0000","last_became_active":"2026-02-24T22:23:29.215334+0000","last_became_peered":"2026-02-24T22:23:29.215334+0000","last_unstale":"2026-02-24T22:30:59.723721+0000","last_undegraded":"2026-02-24T22:30:59.723721+0000","last_fullsized":"2026-02-24T22:30:59.723721+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:01:42.978764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.1e","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.739370+0000","last_change":"2026-02-24T22:15:25.827409+0000","last_active":"2026-02-24T22:31:34.739370+0000","last_peered":"2026-02-24T22:31:34.739370+0000","last_clean":"2026-02-24T22:31:34.739370+0000","last_became_active":"2026-02-24T22:15:25.827124+0000","last_became_peered":"2026-02-24T22:15:25.827124+0000","last_unstale":"2026-02-24T22:31:34.739370+0000","last_undegraded":"2026-02-24T22:31:34.739370+0000","last_fullsized":"2026-02-24T22:31:34.739370+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:44:21.789639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"80.1f","version":"312'3","reported_seq":44,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:09.739954+0000","last_change":"2026-02-24T22:15:22.816227+0000","last_active":"2026-02-24T22:34:09.739954+0000","last_peered":"2026-02-24T22:34:09.739954+0000","last_clean":"2026-02-24T22:34:09.739954+0000","last_became_active":"2026-02-24T22:15:22.815938+0000","last_became_peered":"2026-02-24T22:15:22.815938+0000","last_unstale":"2026-02-24T22:34:09.739954+0000","last_undegraded":"2026-02-24T22:34:09.739954+0000","last_fullsized":"2026-02-24T22:34:09.739954+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:40:15.210510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.0","version":"0'0","reported_seq":65,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.745118+0000","last_change":"2026-02-24T22:13:43.755907+0000","last_active":"2026-02-24T22:31:49.745118+0000","last_peered":"2026-02-24T22:31:49.745118+0000","last_clean":"2026-02-24T22:31:49.745118+0000","last_became_active":"2026-02-24T22:13:43.755266+0000","last_became_peered":"2026-02-24T22:13:43.755266+0000","last_unstale":"2026-02-24T22:31:49.745118+0000","last_undegraded":"2026-02-24T22:31:49.745118+0000","last_fullsized":"2026-02-24T22:31:49.745118+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:42:48.314773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"312'271","reported_seq":977,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.920704+0000","last_change":"2026-02-24T22:05:16.193051+0000","last_active":"2026-02-24T22:34:25.920704+0000","last_peered":"2026-02-24T22:34:25.920704+0000","last_clean":"2026-02-24T22:34:25.920704+0000","last_became_active":"2026-02-24T22:05:16.192589+0000","last_became_peered":"2026-02-24T22:05:16.192589+0000","last_unstale":"2026-02-24T22:34:25.920704+0000","last_undegraded":"2026-02-24T22:34:25.920704+0000","last_fullsized":"2026-02-24T22:34:25.920704+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":200,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:20:06.945662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":433,"num_read_kb":342,"num_write":1269,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"312'197","reported_seq":898,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:46.730087+0000","last_change":"2026-02-24T22:05:18.192403+0000","last_active":"2026-02-24T22:33:46.730087+0000","last_peered":"2026-02-24T22:33:46.730087+0000","last_clean":"2026-02-24T22:33:46.730087+0000","last_became_active":"2026-02-24T22:05:18.192203+0000","last_became_peered":"2026-02-24T22:05:18.192203+0000","last_unstale":"2026-02-24T22:33:46.730087+0000","last_undegraded":"2026-02-24T22:33:46.730087+0000","last_fullsized":"2026-02-24T22:33:46.730087+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":100,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:33:32.754536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":780,"num_read_kb":643,"num_write":282,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"312'378","reported_seq":1187,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.989560+0000","last_change":"2026-02-24T22:05:16.181637+0000","last_active":"2026-02-24T22:34:25.989560+0000","last_peered":"2026-02-24T22:34:25.989560+0000","last_clean":"2026-02-24T22:34:25.989560+0000","last_became_active":"2026-02-24T22:05:16.181379+0000","last_became_peered":"2026-02-24T22:05:16.181379+0000","last_unstale":"2026-02-24T22:34:25.989560+0000","last_undegraded":"2026-02-24T22:34:25.989560+0000","last_fullsized":"2026-02-24T22:34:25.989560+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":300,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":922,"num_read_kb":611,"num_write":1540,"num_write_kb":5240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"312'459","reported_seq":1257,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:57.922033+0000","last_change":"2026-02-24T22:05:18.190910+0000","last_active":"2026-02-24T22:30:57.922033+0000","last_peered":"2026-02-24T22:30:57.922033+0000","last_clean":"2026-02-24T22:30:57.922033+0000","last_became_active":"2026-02-24T22:05:18.190689+0000","last_became_peered":"2026-02-24T22:05:18.190689+0000","last_unstale":"2026-02-24T22:30:57.922033+0000","last_undegraded":"2026-02-24T22:30:57.922033+0000","last_fullsized":"2026-02-24T22:30:57.922033+0000","mapping_epoch":24,"log_start":"312'400","ondisk_log_start":"312'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":400,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:35:35.142199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":2017,"num_read_kb":1670,"num_write":788,"num_write_kb":502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"312'448","reported_seq":1235,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.990740+0000","last_change":"2026-02-24T22:05:16.180825+0000","last_active":"2026-02-24T22:34:25.990740+0000","last_peered":"2026-02-24T22:34:25.990740+0000","last_clean":"2026-02-24T22:34:25.990740+0000","last_became_active":"2026-02-24T22:05:16.180681+0000","last_became_peered":"2026-02-24T22:05:16.180681+0000","last_unstale":"2026-02-24T22:34:25.990740+0000","last_undegraded":"2026-02-24T22:34:25.990740+0000","last_fullsized":"2026-02-24T22:34:25.990740+0000","mapping_epoch":22,"log_start":"312'400","ondisk_log_start":"312'400","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":400,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242910,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":903,"num_read_kb":6172,"num_write":1607,"num_write_kb":5459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"312'80","reported_seq":727,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.732683+0000","last_change":"2026-02-24T22:05:18.192661+0000","last_active":"2026-02-24T22:30:19.732683+0000","last_peered":"2026-02-24T22:30:19.732683+0000","last_clean":"2026-02-24T22:30:19.732683+0000","last_became_active":"2026-02-24T22:05:18.190727+0000","last_became_peered":"2026-02-24T22:05:18.190727+0000","last_unstale":"2026-02-24T22:30:19.732683+0000","last_undegraded":"2026-02-24T22:30:19.732683+0000","last_fullsized":"2026-02-24T22:30:19.732683+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:10:39.624197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":268,"num_read_kb":220,"num_write":112,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"312'289","reported_seq":973,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.963092+0000","last_change":"2026-02-24T22:05:16.193170+0000","last_active":"2026-02-24T22:34:25.963092+0000","last_peered":"2026-02-24T22:34:25.963092+0000","last_clean":"2026-02-24T22:34:25.963092+0000","last_became_active":"2026-02-24T22:05:16.192922+0000","last_became_peered":"2026-02-24T22:05:16.192922+0000","last_unstale":"2026-02-24T22:34:25.963092+0000","last_undegraded":"2026-02-24T22:34:25.963092+0000","last_fullsized":"2026-02-24T22:34:25.963092+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:51:56.894641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":378,"num_read_kb":3190,"num_write":1202,"num_write_kb":12582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"312'135","reported_seq":815,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.733034+0000","last_change":"2026-02-24T22:05:18.184446+0000","last_active":"2026-02-24T22:30:19.733034+0000","last_peered":"2026-02-24T22:30:19.733034+0000","last_clean":"2026-02-24T22:30:19.733034+0000","last_became_active":"2026-02-24T22:05:18.184014+0000","last_became_peered":"2026-02-24T22:05:18.184014+0000","last_unstale":"2026-02-24T22:30:19.733034+0000","last_undegraded":"2026-02-24T22:30:19.733034+0000","last_fullsized":"2026-02-24T22:30:19.733034+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:12:45.735332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":489,"num_read_kb":401,"num_write":208,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"312'271","reported_seq":982,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.955051+0000","last_change":"2026-02-24T22:05:16.191359+0000","last_active":"2026-02-24T22:34:25.955051+0000","last_peered":"2026-02-24T22:34:25.955051+0000","last_clean":"2026-02-24T22:34:25.955051+0000","last_became_active":"2026-02-24T22:05:16.191055+0000","last_became_peered":"2026-02-24T22:05:16.191055+0000","last_unstale":"2026-02-24T22:34:25.955051+0000","last_undegraded":"2026-02-24T22:34:25.955051+0000","last_fullsized":"2026-02-24T22:34:25.955051+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":200,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:40:09.331901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":135,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":423,"num_read_kb":15420,"num_write":1201,"num_write_kb":8454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"312'470","reported_seq":1186,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.738353+0000","last_change":"2026-02-24T22:05:18.190983+0000","last_active":"2026-02-24T22:31:24.738353+0000","last_peered":"2026-02-24T22:31:24.738353+0000","last_clean":"2026-02-24T22:31:24.738353+0000","last_became_active":"2026-02-24T22:05:18.190598+0000","last_became_peered":"2026-02-24T22:05:18.190598+0000","last_unstale":"2026-02-24T22:31:24.738353+0000","last_undegraded":"2026-02-24T22:31:24.738353+0000","last_fullsized":"2026-02-24T22:31:24.738353+0000","mapping_epoch":24,"log_start":"312'400","ondisk_log_start":"312'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":400,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:52:56.661245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1905,"num_read_kb":1522,"num_write":772,"num_write_kb":389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"312'300","reported_seq":1010,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.977104+0000","last_change":"2026-02-24T22:05:16.191753+0000","last_active":"2026-02-24T22:34:25.977104+0000","last_peered":"2026-02-24T22:34:25.977104+0000","last_clean":"2026-02-24T22:34:25.977104+0000","last_became_active":"2026-02-24T22:05:16.191164+0000","last_became_peered":"2026-02-24T22:05:16.191164+0000","last_unstale":"2026-02-24T22:34:25.977104+0000","last_undegraded":"2026-02-24T22:34:25.977104+0000","last_fullsized":"2026-02-24T22:34:25.977104+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":200,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:23:55.135404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":491,"num_read_kb":5551,"num_write":1337,"num_write_kb":5240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"312'56","reported_seq":710,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.746526+0000","last_change":"2026-02-24T22:05:18.198708+0000","last_active":"2026-02-24T22:32:24.746526+0000","last_peered":"2026-02-24T22:32:24.746526+0000","last_clean":"2026-02-24T22:32:24.746526+0000","last_became_active":"2026-02-24T22:05:18.198559+0000","last_became_peered":"2026-02-24T22:05:18.198559+0000","last_unstale":"2026-02-24T22:32:24.746526+0000","last_undegraded":"2026-02-24T22:32:24.746526+0000","last_fullsized":"2026-02-24T22:32:24.746526+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:01:34.891706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":195,"num_read_kb":167,"num_write":70,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"312'300","reported_seq":1000,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.970276+0000","last_change":"2026-02-24T22:05:16.190594+0000","last_active":"2026-02-24T22:34:25.970276+0000","last_peered":"2026-02-24T22:34:25.970276+0000","last_clean":"2026-02-24T22:34:25.970276+0000","last_became_active":"2026-02-24T22:05:16.190367+0000","last_became_peered":"2026-02-24T22:05:16.190367+0000","last_unstale":"2026-02-24T22:34:25.970276+0000","last_undegraded":"2026-02-24T22:34:25.970276+0000","last_fullsized":"2026-02-24T22:34:25.970276+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":200,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:34:28.350502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":439,"num_read_kb":1335,"num_write":1233,"num_write_kb":1104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"312'201","reported_seq":942,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:49.861557+0000","last_change":"2026-02-24T22:05:18.192094+0000","last_active":"2026-02-24T22:33:49.861557+0000","last_peered":"2026-02-24T22:33:49.861557+0000","last_clean":"2026-02-24T22:33:49.861557+0000","last_became_active":"2026-02-24T22:05:18.191767+0000","last_became_peered":"2026-02-24T22:05:18.191767+0000","last_unstale":"2026-02-24T22:33:49.861557+0000","last_undegraded":"2026-02-24T22:33:49.861557+0000","last_fullsized":"2026-02-24T22:33:49.861557+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":100,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:15:25.401981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":876,"num_read_kb":754,"num_write":323,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"312'358","reported_seq":1128,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.976835+0000","last_change":"2026-02-24T22:05:16.186732+0000","last_active":"2026-02-24T22:34:25.976835+0000","last_peered":"2026-02-24T22:34:25.976835+0000","last_clean":"2026-02-24T22:34:25.976835+0000","last_became_active":"2026-02-24T22:05:16.185779+0000","last_became_peered":"2026-02-24T22:05:16.185779+0000","last_unstale":"2026-02-24T22:34:25.976835+0000","last_undegraded":"2026-02-24T22:34:25.976835+0000","last_fullsized":"2026-02-24T22:34:25.976835+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":300,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:01:24.063571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":619,"num_read_kb":883,"num_write":1617,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"312'83","reported_seq":730,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.729125+0000","last_change":"2026-02-24T22:05:18.188067+0000","last_active":"2026-02-24T22:32:24.729125+0000","last_peered":"2026-02-24T22:32:24.729125+0000","last_clean":"2026-02-24T22:32:24.729125+0000","last_became_active":"2026-02-24T22:05:18.187838+0000","last_became_peered":"2026-02-24T22:05:18.187838+0000","last_unstale":"2026-02-24T22:32:24.729125+0000","last_undegraded":"2026-02-24T22:32:24.729125+0000","last_fullsized":"2026-02-24T22:32:24.729125+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":263,"num_read_kb":217,"num_write":106,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"312'135","reported_seq":808,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.736992+0000","last_change":"2026-02-24T22:05:18.192713+0000","last_active":"2026-02-24T22:30:59.736992+0000","last_peered":"2026-02-24T22:30:59.736992+0000","last_clean":"2026-02-24T22:30:59.736992+0000","last_became_active":"2026-02-24T22:05:18.191625+0000","last_became_peered":"2026-02-24T22:05:18.191625+0000","last_unstale":"2026-02-24T22:30:59.736992+0000","last_undegraded":"2026-02-24T22:30:59.736992+0000","last_fullsized":"2026-02-24T22:30:59.736992+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:24:04.071735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":477,"num_read_kb":396,"num_write":183,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"312'300","reported_seq":1037,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.955260+0000","last_change":"2026-02-24T22:06:01.025897+0000","last_active":"2026-02-24T22:34:25.955260+0000","last_peered":"2026-02-24T22:34:25.955260+0000","last_clean":"2026-02-24T22:34:25.955260+0000","last_became_active":"2026-02-24T22:06:01.025659+0000","last_became_peered":"2026-02-24T22:06:01.025659+0000","last_unstale":"2026-02-24T22:34:25.955260+0000","last_undegraded":"2026-02-24T22:34:25.955260+0000","last_fullsized":"2026-02-24T22:34:25.955260+0000","mapping_epoch":36,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":200,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:15:11.362680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":514,"num_read_kb":16221,"num_write":1317,"num_write_kb":8500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"312'4249","reported_seq":5059,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.162950+0000","last_change":"2026-02-24T22:05:18.193454+0000","last_active":"2026-02-24T22:34:15.162950+0000","last_peered":"2026-02-24T22:34:15.162950+0000","last_clean":"2026-02-24T22:34:15.162950+0000","last_became_active":"2026-02-24T22:05:18.193257+0000","last_became_peered":"2026-02-24T22:05:18.193257+0000","last_unstale":"2026-02-24T22:34:15.162950+0000","last_undegraded":"2026-02-24T22:34:15.162950+0000","last_fullsized":"2026-02-24T22:34:15.162950+0000","mapping_epoch":24,"log_start":"312'4200","ondisk_log_start":"312'4200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":3000,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:25:08.863245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":17332,"num_read_kb":14553,"num_write":6398,"num_write_kb":3198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"312'284","reported_seq":996,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.943808+0000","last_change":"2026-02-24T22:05:16.191122+0000","last_active":"2026-02-24T22:34:25.943808+0000","last_peered":"2026-02-24T22:34:25.943808+0000","last_clean":"2026-02-24T22:34:25.943808+0000","last_became_active":"2026-02-24T22:05:16.190819+0000","last_became_peered":"2026-02-24T22:05:16.190819+0000","last_unstale":"2026-02-24T22:34:25.943808+0000","last_undegraded":"2026-02-24T22:34:25.943808+0000","last_fullsized":"2026-02-24T22:34:25.943808+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":200,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:31:00.005386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242922,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":466,"num_read_kb":8113,"num_write":1161,"num_write_kb":5510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"312'862","reported_seq":4024,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.230995+0000","last_change":"2026-02-24T22:05:18.190719+0000","last_active":"2026-02-24T22:34:17.230995+0000","last_peered":"2026-02-24T22:34:17.230995+0000","last_clean":"2026-02-24T22:34:17.230995+0000","last_became_active":"2026-02-24T22:05:18.190363+0000","last_became_peered":"2026-02-24T22:05:18.190363+0000","last_unstale":"2026-02-24T22:34:17.230995+0000","last_undegraded":"2026-02-24T22:34:17.230995+0000","last_fullsized":"2026-02-24T22:34:17.230995+0000","mapping_epoch":24,"log_start":"312'800","ondisk_log_start":"312'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":800,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:46:25.367911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4747,"num_read_kb":5785,"num_write":398,"num_write_kb":236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"312'321","reported_seq":1052,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.963847+0000","last_change":"2026-02-24T22:05:16.191246+0000","last_active":"2026-02-24T22:34:25.963847+0000","last_peered":"2026-02-24T22:34:25.963847+0000","last_clean":"2026-02-24T22:34:25.963847+0000","last_became_active":"2026-02-24T22:05:16.190816+0000","last_became_peered":"2026-02-24T22:05:16.190816+0000","last_unstale":"2026-02-24T22:34:25.963847+0000","last_undegraded":"2026-02-24T22:34:25.963847+0000","last_fullsized":"2026-02-24T22:34:25.963847+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":300,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:47:51.922901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":544,"num_read_kb":2327,"num_write":1304,"num_write_kb":12235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"312'93","reported_seq":796,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.725906+0000","last_change":"2026-02-24T22:05:18.187293+0000","last_active":"2026-02-24T22:31:54.725906+0000","last_peered":"2026-02-24T22:31:54.725906+0000","last_clean":"2026-02-24T22:31:54.725906+0000","last_became_active":"2026-02-24T22:05:18.186994+0000","last_became_peered":"2026-02-24T22:05:18.186994+0000","last_unstale":"2026-02-24T22:31:54.725906+0000","last_undegraded":"2026-02-24T22:31:54.725906+0000","last_fullsized":"2026-02-24T22:31:54.725906+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:00:02.034352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":352,"num_read_kb":306,"num_write":116,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"312'380","reported_seq":1172,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.964242+0000","last_change":"2026-02-24T22:05:16.189106+0000","last_active":"2026-02-24T22:34:25.964242+0000","last_peered":"2026-02-24T22:34:25.964242+0000","last_clean":"2026-02-24T22:34:25.964242+0000","last_became_active":"2026-02-24T22:05:16.188658+0000","last_became_peered":"2026-02-24T22:05:16.188658+0000","last_unstale":"2026-02-24T22:34:25.964242+0000","last_undegraded":"2026-02-24T22:34:25.964242+0000","last_fullsized":"2026-02-24T22:34:25.964242+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":300,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:33:27.530851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":840,"num_read_kb":685,"num_write":1819,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"312'169","reported_seq":891,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.245262+0000","last_change":"2026-02-24T22:05:18.187949+0000","last_active":"2026-02-24T22:34:18.245262+0000","last_peered":"2026-02-24T22:34:18.245262+0000","last_clean":"2026-02-24T22:34:18.245262+0000","last_became_active":"2026-02-24T22:05:18.187682+0000","last_became_peered":"2026-02-24T22:05:18.187682+0000","last_unstale":"2026-02-24T22:34:18.245262+0000","last_undegraded":"2026-02-24T22:34:18.245262+0000","last_fullsized":"2026-02-24T22:34:18.245262+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:07:24.443062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":639,"num_read_kb":531,"num_write":250,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"312'287","reported_seq":987,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.984651+0000","last_change":"2026-02-24T22:05:16.185278+0000","last_active":"2026-02-24T22:34:25.984651+0000","last_peered":"2026-02-24T22:34:25.984651+0000","last_clean":"2026-02-24T22:34:25.984651+0000","last_became_active":"2026-02-24T22:05:16.184209+0000","last_became_peered":"2026-02-24T22:05:16.184209+0000","last_unstale":"2026-02-24T22:34:25.984651+0000","last_undegraded":"2026-02-24T22:34:25.984651+0000","last_fullsized":"2026-02-24T22:34:25.984651+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":200,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:50:09.257349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":416,"num_read_kb":603,"num_write":1232,"num_write_kb":10636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"312'111","reported_seq":789,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.222919+0000","last_change":"2026-02-24T22:05:18.194092+0000","last_active":"2026-02-24T22:34:18.222919+0000","last_peered":"2026-02-24T22:34:18.222919+0000","last_clean":"2026-02-24T22:34:18.222919+0000","last_became_active":"2026-02-24T22:05:18.193320+0000","last_became_peered":"2026-02-24T22:05:18.193320+0000","last_unstale":"2026-02-24T22:34:18.222919+0000","last_undegraded":"2026-02-24T22:34:18.222919+0000","last_fullsized":"2026-02-24T22:34:18.222919+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":100,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:27:50.398388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":425,"num_read_kb":355,"num_write":146,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"312'281","reported_seq":994,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.993043+0000","last_change":"2026-02-24T22:05:16.181803+0000","last_active":"2026-02-24T22:34:25.993043+0000","last_peered":"2026-02-24T22:34:25.993043+0000","last_clean":"2026-02-24T22:34:25.993043+0000","last_became_active":"2026-02-24T22:05:16.181527+0000","last_became_peered":"2026-02-24T22:05:16.181527+0000","last_unstale":"2026-02-24T22:34:25.993043+0000","last_undegraded":"2026-02-24T22:34:25.993043+0000","last_fullsized":"2026-02-24T22:34:25.993043+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:53:05.002989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242895,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":474,"num_read_kb":5463,"num_write":1217,"num_write_kb":5189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":576,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:04.744157+0000","last_change":"2026-02-24T22:05:29.658481+0000","last_active":"2026-02-24T22:32:04.744157+0000","last_peered":"2026-02-24T22:32:04.744157+0000","last_clean":"2026-02-24T22:32:04.744157+0000","last_became_active":"2026-02-24T22:05:29.657764+0000","last_became_peered":"2026-02-24T22:05:29.657764+0000","last_unstale":"2026-02-24T22:32:04.744157+0000","last_undegraded":"2026-02-24T22:32:04.744157+0000","last_fullsized":"2026-02-24T22:32:04.744157+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:34:57.407311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"312'80","reported_seq":734,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.729056+0000","last_change":"2026-02-24T22:05:18.185097+0000","last_active":"2026-02-24T22:31:04.729056+0000","last_peered":"2026-02-24T22:31:04.729056+0000","last_clean":"2026-02-24T22:31:04.729056+0000","last_became_active":"2026-02-24T22:05:18.184771+0000","last_became_peered":"2026-02-24T22:05:18.184771+0000","last_unstale":"2026-02-24T22:31:04.729056+0000","last_undegraded":"2026-02-24T22:31:04.729056+0000","last_fullsized":"2026-02-24T22:31:04.729056+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:34:23.195340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":246,"num_read_kb":208,"num_write":102,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"312'340","reported_seq":1121,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.978326+0000","last_change":"2026-02-24T22:05:16.192243+0000","last_active":"2026-02-24T22:34:25.978326+0000","last_peered":"2026-02-24T22:34:25.978326+0000","last_clean":"2026-02-24T22:34:25.978326+0000","last_became_active":"2026-02-24T22:05:16.191841+0000","last_became_peered":"2026-02-24T22:05:16.191841+0000","last_unstale":"2026-02-24T22:34:25.978326+0000","last_undegraded":"2026-02-24T22:34:25.978326+0000","last_fullsized":"2026-02-24T22:34:25.978326+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":300,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:59:17.072003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":614,"num_read_kb":81372,"num_write":1329,"num_write_kb":20589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":579,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.745270+0000","last_change":"2026-02-24T22:05:20.194969+0000","last_active":"2026-02-24T22:31:54.745270+0000","last_peered":"2026-02-24T22:31:54.745270+0000","last_clean":"2026-02-24T22:31:54.745270+0000","last_became_active":"2026-02-24T22:05:20.194031+0000","last_became_peered":"2026-02-24T22:05:20.194031+0000","last_unstale":"2026-02-24T22:31:54.745270+0000","last_undegraded":"2026-02-24T22:31:54.745270+0000","last_fullsized":"2026-02-24T22:31:54.745270+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:17:15.557806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"312'259","reported_seq":899,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.772688+0000","last_change":"2026-02-24T22:05:22.192071+0000","last_active":"2026-02-24T22:34:25.772688+0000","last_peered":"2026-02-24T22:34:25.772688+0000","last_clean":"2026-02-24T22:34:25.772688+0000","last_became_active":"2026-02-24T22:05:22.191928+0000","last_became_peered":"2026-02-24T22:05:22.191928+0000","last_unstale":"2026-02-24T22:34:25.772688+0000","last_undegraded":"2026-02-24T22:34:25.772688+0000","last_fullsized":"2026-02-24T22:34:25.772688+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":200,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:56:18.209592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44919,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":413,"num_read_kb":914,"num_write":358,"num_write_kb":269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"312'158","reported_seq":747,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.253470+0000","last_change":"2026-02-24T22:05:24.205235+0000","last_active":"2026-02-24T22:34:18.253470+0000","last_peered":"2026-02-24T22:34:18.253470+0000","last_clean":"2026-02-24T22:34:18.253470+0000","last_became_active":"2026-02-24T22:05:24.204990+0000","last_became_peered":"2026-02-24T22:05:24.204990+0000","last_unstale":"2026-02-24T22:34:18.253470+0000","last_undegraded":"2026-02-24T22:34:18.253470+0000","last_fullsized":"2026-02-24T22:34:18.253470+0000","mapping_epoch":30,"log_start":"37'100","ondisk_log_start":"37'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":100,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:14:42.637540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32373,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":92,"num_read_kb":87,"num_write":391,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":564,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:29:44.730561+0000","last_change":"2026-02-24T22:05:29.662185+0000","last_active":"2026-02-24T22:29:44.730561+0000","last_peered":"2026-02-24T22:29:44.730561+0000","last_clean":"2026-02-24T22:29:44.730561+0000","last_became_active":"2026-02-24T22:05:29.661747+0000","last_became_peered":"2026-02-24T22:05:29.661747+0000","last_unstale":"2026-02-24T22:29:44.730561+0000","last_undegraded":"2026-02-24T22:29:44.730561+0000","last_fullsized":"2026-02-24T22:29:44.730561+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:08:40.631093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"312'73","reported_seq":743,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:07.153602+0000","last_change":"2026-02-24T22:05:18.191685+0000","last_active":"2026-02-24T22:34:07.153602+0000","last_peered":"2026-02-24T22:34:07.153602+0000","last_clean":"2026-02-24T22:34:07.153602+0000","last_became_active":"2026-02-24T22:05:18.191346+0000","last_became_peered":"2026-02-24T22:05:18.191346+0000","last_unstale":"2026-02-24T22:34:07.153602+0000","last_undegraded":"2026-02-24T22:34:07.153602+0000","last_fullsized":"2026-02-24T22:34:07.153602+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:48:20.436580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":237,"num_read_kb":209,"num_write":87,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"312'271","reported_seq":991,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.983430+0000","last_change":"2026-02-24T22:05:16.184337+0000","last_active":"2026-02-24T22:34:25.983430+0000","last_peered":"2026-02-24T22:34:25.983430+0000","last_clean":"2026-02-24T22:34:25.983430+0000","last_became_active":"2026-02-24T22:05:16.183990+0000","last_became_peered":"2026-02-24T22:05:16.183990+0000","last_unstale":"2026-02-24T22:34:25.983430+0000","last_undegraded":"2026-02-24T22:34:25.983430+0000","last_fullsized":"2026-02-24T22:34:25.983430+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":200,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:27:53.239432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485805,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":458,"num_read_kb":23693,"num_write":1270,"num_write_kb":23324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.729065+0000","last_change":"2026-02-24T22:05:20.186544+0000","last_active":"2026-02-24T22:31:04.729065+0000","last_peered":"2026-02-24T22:31:04.729065+0000","last_clean":"2026-02-24T22:31:04.729065+0000","last_became_active":"2026-02-24T22:05:20.186217+0000","last_became_peered":"2026-02-24T22:05:20.186217+0000","last_unstale":"2026-02-24T22:31:04.729065+0000","last_undegraded":"2026-02-24T22:31:04.729065+0000","last_fullsized":"2026-02-24T22:31:04.729065+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:41:55.793283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"312'305","reported_seq":1152,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.237230+0000","last_change":"2026-02-24T22:05:22.197384+0000","last_active":"2026-02-24T22:34:25.237230+0000","last_peered":"2026-02-24T22:34:25.237230+0000","last_clean":"2026-02-24T22:34:25.237230+0000","last_became_active":"2026-02-24T22:05:22.196911+0000","last_became_peered":"2026-02-24T22:05:22.196911+0000","last_unstale":"2026-02-24T22:34:25.237230+0000","last_undegraded":"2026-02-24T22:34:25.237230+0000","last_fullsized":"2026-02-24T22:34:25.237230+0000","mapping_epoch":28,"log_start":"307'200","ondisk_log_start":"307'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:19:12.370112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1016,"num_read_kb":540,"num_write":51,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"312'187","reported_seq":800,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.220647+0000","last_change":"2026-02-24T22:05:24.207903+0000","last_active":"2026-02-24T22:34:18.220647+0000","last_peered":"2026-02-24T22:34:18.220647+0000","last_clean":"2026-02-24T22:34:18.220647+0000","last_became_active":"2026-02-24T22:05:24.207692+0000","last_became_peered":"2026-02-24T22:05:24.207692+0000","last_unstale":"2026-02-24T22:34:18.220647+0000","last_undegraded":"2026-02-24T22:34:18.220647+0000","last_fullsized":"2026-02-24T22:34:18.220647+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:14:09.532597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29978,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":173,"num_read_kb":171,"num_write":478,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":573,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.734173+0000","last_change":"2026-02-24T22:05:29.654457+0000","last_active":"2026-02-24T22:30:29.734173+0000","last_peered":"2026-02-24T22:30:29.734173+0000","last_clean":"2026-02-24T22:30:29.734173+0000","last_became_active":"2026-02-24T22:05:29.654288+0000","last_became_peered":"2026-02-24T22:05:29.654288+0000","last_unstale":"2026-02-24T22:30:29.734173+0000","last_undegraded":"2026-02-24T22:30:29.734173+0000","last_fullsized":"2026-02-24T22:30:29.734173+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:05:32.007637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"312'274","reported_seq":998,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.743416+0000","last_change":"2026-02-24T22:05:18.184707+0000","last_active":"2026-02-24T22:31:54.743416+0000","last_peered":"2026-02-24T22:31:54.743416+0000","last_clean":"2026-02-24T22:31:54.743416+0000","last_became_active":"2026-02-24T22:05:18.184542+0000","last_became_peered":"2026-02-24T22:05:18.184542+0000","last_unstale":"2026-02-24T22:31:54.743416+0000","last_undegraded":"2026-02-24T22:31:54.743416+0000","last_fullsized":"2026-02-24T22:31:54.743416+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":200,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:55:45.622376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1179,"num_read_kb":976,"num_write":460,"num_write_kb":243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"312'290","reported_seq":1004,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.946764+0000","last_change":"2026-02-24T22:05:16.190582+0000","last_active":"2026-02-24T22:34:25.946764+0000","last_peered":"2026-02-24T22:34:25.946764+0000","last_clean":"2026-02-24T22:34:25.946764+0000","last_became_active":"2026-02-24T22:05:16.190372+0000","last_became_peered":"2026-02-24T22:05:16.190372+0000","last_unstale":"2026-02-24T22:34:25.946764+0000","last_undegraded":"2026-02-24T22:34:25.946764+0000","last_fullsized":"2026-02-24T22:34:25.946764+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":200,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:46:33.357028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":430,"num_read_kb":3398,"num_write":1175,"num_write_kb":524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.734254+0000","last_change":"2026-02-24T22:05:20.187522+0000","last_active":"2026-02-24T22:31:54.734254+0000","last_peered":"2026-02-24T22:31:54.734254+0000","last_clean":"2026-02-24T22:31:54.734254+0000","last_became_active":"2026-02-24T22:05:20.187327+0000","last_became_peered":"2026-02-24T22:05:20.187327+0000","last_unstale":"2026-02-24T22:31:54.734254+0000","last_undegraded":"2026-02-24T22:31:54.734254+0000","last_fullsized":"2026-02-24T22:31:54.734254+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:09:52.559206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"312'794","reported_seq":2123,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.255673+0000","last_change":"2026-02-24T22:05:22.191261+0000","last_active":"2026-02-24T22:34:25.255673+0000","last_peered":"2026-02-24T22:34:25.255673+0000","last_clean":"2026-02-24T22:34:25.255673+0000","last_became_active":"2026-02-24T22:05:22.191073+0000","last_became_peered":"2026-02-24T22:05:22.191073+0000","last_unstale":"2026-02-24T22:34:25.255673+0000","last_undegraded":"2026-02-24T22:34:25.255673+0000","last_fullsized":"2026-02-24T22:34:25.255673+0000","mapping_epoch":28,"log_start":"307'700","ondisk_log_start":"307'700","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":700,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:26:27.583739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":460,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2934,"num_read_kb":1505,"num_write":81,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"312'204","reported_seq":1716,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.409302+0000","last_change":"2026-02-24T22:05:24.206373+0000","last_active":"2026-02-24T22:34:17.409302+0000","last_peered":"2026-02-24T22:34:17.409302+0000","last_clean":"2026-02-24T22:34:17.409302+0000","last_became_active":"2026-02-24T22:05:24.206134+0000","last_became_peered":"2026-02-24T22:05:24.206134+0000","last_unstale":"2026-02-24T22:34:17.409302+0000","last_undegraded":"2026-02-24T22:34:17.409302+0000","last_fullsized":"2026-02-24T22:34:17.409302+0000","mapping_epoch":30,"log_start":"37'100","ondisk_log_start":"37'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:32:52.956429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33889,"num_objects":107,"num_object_clones":0,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":0,"num_read":1064,"num_read_kb":1063,"num_write":484,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.733657+0000","last_change":"2026-02-24T22:05:29.658922+0000","last_active":"2026-02-24T22:30:29.733657+0000","last_peered":"2026-02-24T22:30:29.733657+0000","last_clean":"2026-02-24T22:30:29.733657+0000","last_became_active":"2026-02-24T22:05:29.658119+0000","last_became_peered":"2026-02-24T22:05:29.658119+0000","last_unstale":"2026-02-24T22:30:29.733657+0000","last_undegraded":"2026-02-24T22:30:29.733657+0000","last_fullsized":"2026-02-24T22:30:29.733657+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:10:58.821048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"312'668","reported_seq":1465,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:09.715537+0000","last_change":"2026-02-24T22:06:00.662810+0000","last_active":"2026-02-24T22:30:09.715537+0000","last_peered":"2026-02-24T22:30:09.715537+0000","last_clean":"2026-02-24T22:30:09.715537+0000","last_became_active":"2026-02-24T22:06:00.662609+0000","last_became_peered":"2026-02-24T22:06:00.662609+0000","last_unstale":"2026-02-24T22:30:09.715537+0000","last_undegraded":"2026-02-24T22:30:09.715537+0000","last_fullsized":"2026-02-24T22:30:09.715537+0000","mapping_epoch":36,"log_start":"312'600","ondisk_log_start":"312'600","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":600,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:42:35.250489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":2886,"num_read_kb":2388,"num_write":1101,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"312'318","reported_seq":1044,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.954486+0000","last_change":"2026-02-24T22:05:16.190904+0000","last_active":"2026-02-24T22:34:25.954486+0000","last_peered":"2026-02-24T22:34:25.954486+0000","last_clean":"2026-02-24T22:34:25.954486+0000","last_became_active":"2026-02-24T22:05:16.189739+0000","last_became_peered":"2026-02-24T22:05:16.189739+0000","last_unstale":"2026-02-24T22:34:25.954486+0000","last_undegraded":"2026-02-24T22:34:25.954486+0000","last_fullsized":"2026-02-24T22:34:25.954486+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:04:06.385930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485786,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":468,"num_read_kb":16928,"num_write":1377,"num_write_kb":17766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":579,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.745280+0000","last_change":"2026-02-24T22:05:20.193438+0000","last_active":"2026-02-24T22:31:54.745280+0000","last_peered":"2026-02-24T22:31:54.745280+0000","last_clean":"2026-02-24T22:31:54.745280+0000","last_became_active":"2026-02-24T22:05:20.193200+0000","last_became_peered":"2026-02-24T22:05:20.193200+0000","last_unstale":"2026-02-24T22:31:54.745280+0000","last_undegraded":"2026-02-24T22:31:54.745280+0000","last_fullsized":"2026-02-24T22:31:54.745280+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:58:09.212117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"312'192","reported_seq":795,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.646624+0000","last_change":"2026-02-24T22:05:22.913003+0000","last_active":"2026-02-24T22:34:25.646624+0000","last_peered":"2026-02-24T22:34:25.646624+0000","last_clean":"2026-02-24T22:34:25.646624+0000","last_became_active":"2026-02-24T22:05:22.912826+0000","last_became_peered":"2026-02-24T22:05:22.912826+0000","last_unstale":"2026-02-24T22:34:25.646624+0000","last_undegraded":"2026-02-24T22:34:25.646624+0000","last_fullsized":"2026-02-24T22:34:25.646624+0000","mapping_epoch":28,"log_start":"312'100","ondisk_log_start":"312'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:49:38.039509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37493,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":308,"num_read_kb":794,"num_write":284,"num_write_kb":257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"312'257","reported_seq":903,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:13.928723+0000","last_change":"2026-02-24T22:05:24.199267+0000","last_active":"2026-02-24T22:34:13.928723+0000","last_peered":"2026-02-24T22:34:13.928723+0000","last_clean":"2026-02-24T22:34:13.928723+0000","last_became_active":"2026-02-24T22:05:24.198964+0000","last_became_peered":"2026-02-24T22:05:24.198964+0000","last_unstale":"2026-02-24T22:34:13.928723+0000","last_undegraded":"2026-02-24T22:34:13.928723+0000","last_fullsized":"2026-02-24T22:34:13.928723+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":200,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:47:04.450411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29433,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":428,"num_read_kb":365,"num_write":612,"num_write_kb":187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"312'113","reported_seq":839,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:55.982068+0000","last_change":"2026-02-24T22:05:18.190057+0000","last_active":"2026-02-24T22:33:55.982068+0000","last_peered":"2026-02-24T22:33:55.982068+0000","last_clean":"2026-02-24T22:33:55.982068+0000","last_became_active":"2026-02-24T22:05:18.189796+0000","last_became_peered":"2026-02-24T22:05:18.189796+0000","last_unstale":"2026-02-24T22:33:55.982068+0000","last_undegraded":"2026-02-24T22:33:55.982068+0000","last_fullsized":"2026-02-24T22:33:55.982068+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":100,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:22:22.190649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":455,"num_read_kb":399,"num_write":145,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"312'256","reported_seq":935,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.945855+0000","last_change":"2026-02-24T22:05:16.192443+0000","last_active":"2026-02-24T22:34:25.945855+0000","last_peered":"2026-02-24T22:34:25.945855+0000","last_clean":"2026-02-24T22:34:25.945855+0000","last_became_active":"2026-02-24T22:05:16.191985+0000","last_became_peered":"2026-02-24T22:05:16.191985+0000","last_unstale":"2026-02-24T22:34:25.945855+0000","last_undegraded":"2026-02-24T22:34:25.945855+0000","last_fullsized":"2026-02-24T22:34:25.945855+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":200,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:48:03.816342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":82,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":355,"num_read_kb":4498,"num_write":1041,"num_write_kb":4311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":586,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.742035+0000","last_change":"2026-02-24T22:05:20.194496+0000","last_active":"2026-02-24T22:31:19.742035+0000","last_peered":"2026-02-24T22:31:19.742035+0000","last_clean":"2026-02-24T22:31:19.742035+0000","last_became_active":"2026-02-24T22:05:20.194288+0000","last_became_peered":"2026-02-24T22:05:20.194288+0000","last_unstale":"2026-02-24T22:31:19.742035+0000","last_undegraded":"2026-02-24T22:31:19.742035+0000","last_fullsized":"2026-02-24T22:31:19.742035+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:55:32.351699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"312'260","reported_seq":889,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.852870+0000","last_change":"2026-02-24T22:05:22.913184+0000","last_active":"2026-02-24T22:34:25.852870+0000","last_peered":"2026-02-24T22:34:25.852870+0000","last_clean":"2026-02-24T22:34:25.852870+0000","last_became_active":"2026-02-24T22:05:22.912859+0000","last_became_peered":"2026-02-24T22:05:22.912859+0000","last_unstale":"2026-02-24T22:34:25.852870+0000","last_undegraded":"2026-02-24T22:34:25.852870+0000","last_fullsized":"2026-02-24T22:34:25.852870+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":200,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:37:47.377695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38223,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":418,"num_read_kb":936,"num_write":362,"num_write_kb":271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"312'315","reported_seq":1010,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.329842+0000","last_change":"2026-02-24T22:05:24.200453+0000","last_active":"2026-02-24T22:34:15.329842+0000","last_peered":"2026-02-24T22:34:15.329842+0000","last_clean":"2026-02-24T22:34:15.329842+0000","last_became_active":"2026-02-24T22:05:24.199772+0000","last_became_peered":"2026-02-24T22:05:24.199772+0000","last_unstale":"2026-02-24T22:34:15.329842+0000","last_undegraded":"2026-02-24T22:34:15.329842+0000","last_fullsized":"2026-02-24T22:34:15.329842+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:08:33.217544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30266,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":473,"num_read_kb":498,"num_write":701,"num_write_kb":232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":566,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.735506+0000","last_change":"2026-02-24T22:05:29.661914+0000","last_active":"2026-02-24T22:30:29.735506+0000","last_peered":"2026-02-24T22:30:29.735506+0000","last_clean":"2026-02-24T22:30:29.735506+0000","last_became_active":"2026-02-24T22:05:29.661401+0000","last_became_peered":"2026-02-24T22:05:29.661401+0000","last_unstale":"2026-02-24T22:30:29.735506+0000","last_undegraded":"2026-02-24T22:30:29.735506+0000","last_fullsized":"2026-02-24T22:30:29.735506+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:09:29.373749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"312'84","reported_seq":745,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.739343+0000","last_change":"2026-02-24T22:05:18.193728+0000","last_active":"2026-02-24T22:31:19.739343+0000","last_peered":"2026-02-24T22:31:19.739343+0000","last_clean":"2026-02-24T22:31:19.739343+0000","last_became_active":"2026-02-24T22:05:18.193188+0000","last_became_peered":"2026-02-24T22:05:18.193188+0000","last_unstale":"2026-02-24T22:31:19.739343+0000","last_undegraded":"2026-02-24T22:31:19.739343+0000","last_fullsized":"2026-02-24T22:31:19.739343+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:53:00.913559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":312,"num_read_kb":260,"num_write":126,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"312'321","reported_seq":1052,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.941609+0000","last_change":"2026-02-24T22:05:16.192300+0000","last_active":"2026-02-24T22:34:25.941609+0000","last_peered":"2026-02-24T22:34:25.941609+0000","last_clean":"2026-02-24T22:34:25.941609+0000","last_became_active":"2026-02-24T22:05:16.191956+0000","last_became_peered":"2026-02-24T22:05:16.191956+0000","last_unstale":"2026-02-24T22:34:25.941609+0000","last_undegraded":"2026-02-24T22:34:25.941609+0000","last_fullsized":"2026-02-24T22:34:25.941609+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":300,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:08:01.293337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485778,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":523,"num_read_kb":10657,"num_write":1420,"num_write_kb":15460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":594,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.722249+0000","last_change":"2026-02-24T22:05:20.186383+0000","last_active":"2026-02-24T22:31:19.722249+0000","last_peered":"2026-02-24T22:31:19.722249+0000","last_clean":"2026-02-24T22:31:19.722249+0000","last_became_active":"2026-02-24T22:05:20.186055+0000","last_became_peered":"2026-02-24T22:05:20.186055+0000","last_unstale":"2026-02-24T22:31:19.722249+0000","last_undegraded":"2026-02-24T22:31:19.722249+0000","last_fullsized":"2026-02-24T22:31:19.722249+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:31:36.594293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"312'229","reported_seq":837,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.441343+0000","last_change":"2026-02-24T22:05:22.194034+0000","last_active":"2026-02-24T22:34:25.441343+0000","last_peered":"2026-02-24T22:34:25.441343+0000","last_clean":"2026-02-24T22:34:25.441343+0000","last_became_active":"2026-02-24T22:05:22.193741+0000","last_became_peered":"2026-02-24T22:05:22.193741+0000","last_unstale":"2026-02-24T22:34:25.441343+0000","last_undegraded":"2026-02-24T22:34:25.441343+0000","last_fullsized":"2026-02-24T22:34:25.441343+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":200,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:35:27.895920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39811,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":385,"num_read_kb":1028,"num_write":360,"num_write_kb":328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"312'185","reported_seq":809,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.176596+0000","last_change":"2026-02-24T22:05:24.199051+0000","last_active":"2026-02-24T22:34:15.176596+0000","last_peered":"2026-02-24T22:34:15.176596+0000","last_clean":"2026-02-24T22:34:15.176596+0000","last_became_active":"2026-02-24T22:05:24.198731+0000","last_became_peered":"2026-02-24T22:05:24.198731+0000","last_unstale":"2026-02-24T22:34:15.176596+0000","last_undegraded":"2026-02-24T22:34:15.176596+0000","last_fullsized":"2026-02-24T22:34:15.176596+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:42:29.899927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30034,"num_objects":91,"num_object_clones":0,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":241,"num_read_kb":207,"num_write":470,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":570,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:34.732629+0000","last_change":"2026-02-24T22:05:29.661462+0000","last_active":"2026-02-24T22:30:34.732629+0000","last_peered":"2026-02-24T22:30:34.732629+0000","last_clean":"2026-02-24T22:30:34.732629+0000","last_became_active":"2026-02-24T22:05:29.661161+0000","last_became_peered":"2026-02-24T22:05:29.661161+0000","last_unstale":"2026-02-24T22:30:34.732629+0000","last_undegraded":"2026-02-24T22:30:34.732629+0000","last_fullsized":"2026-02-24T22:30:34.732629+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:23:22.386231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"312'106","reported_seq":752,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.738326+0000","last_change":"2026-02-24T22:05:18.191843+0000","last_active":"2026-02-24T22:31:24.738326+0000","last_peered":"2026-02-24T22:31:24.738326+0000","last_clean":"2026-02-24T22:31:24.738326+0000","last_became_active":"2026-02-24T22:05:18.191555+0000","last_became_peered":"2026-02-24T22:05:18.191555+0000","last_unstale":"2026-02-24T22:31:24.738326+0000","last_undegraded":"2026-02-24T22:31:24.738326+0000","last_fullsized":"2026-02-24T22:31:24.738326+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:53:20.472053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":387,"num_read_kb":325,"num_write":162,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"312'285","reported_seq":998,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.931317+0000","last_change":"2026-02-24T22:05:16.192146+0000","last_active":"2026-02-24T22:34:25.931317+0000","last_peered":"2026-02-24T22:34:25.931317+0000","last_clean":"2026-02-24T22:34:25.931317+0000","last_became_active":"2026-02-24T22:05:16.191733+0000","last_became_peered":"2026-02-24T22:05:16.191733+0000","last_unstale":"2026-02-24T22:34:25.931317+0000","last_undegraded":"2026-02-24T22:34:25.931317+0000","last_fullsized":"2026-02-24T22:34:25.931317+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:27:59.980345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242906,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":398,"num_read_kb":31022,"num_write":1049,"num_write_kb":10319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":593,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.715007+0000","last_change":"2026-02-24T22:05:20.187327+0000","last_active":"2026-02-24T22:30:04.715007+0000","last_peered":"2026-02-24T22:30:04.715007+0000","last_clean":"2026-02-24T22:30:04.715007+0000","last_became_active":"2026-02-24T22:05:20.186892+0000","last_became_peered":"2026-02-24T22:05:20.186892+0000","last_unstale":"2026-02-24T22:30:04.715007+0000","last_undegraded":"2026-02-24T22:30:04.715007+0000","last_fullsized":"2026-02-24T22:30:04.715007+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:53:23.369187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"312'238","reported_seq":851,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.905330+0000","last_change":"2026-02-24T22:05:22.200501+0000","last_active":"2026-02-24T22:34:25.905330+0000","last_peered":"2026-02-24T22:34:25.905330+0000","last_clean":"2026-02-24T22:34:25.905330+0000","last_became_active":"2026-02-24T22:05:22.200208+0000","last_became_peered":"2026-02-24T22:05:22.200208+0000","last_unstale":"2026-02-24T22:34:25.905330+0000","last_undegraded":"2026-02-24T22:34:25.905330+0000","last_fullsized":"2026-02-24T22:34:25.905330+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":200,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:45:47.054214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39435,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":390,"num_read_kb":991,"num_write":356,"num_write_kb":310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"312'275","reported_seq":1084,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.547089+0000","last_change":"2026-02-24T22:05:24.200107+0000","last_active":"2026-02-24T22:34:18.547089+0000","last_peered":"2026-02-24T22:34:18.547089+0000","last_clean":"2026-02-24T22:34:18.547089+0000","last_became_active":"2026-02-24T22:05:24.199525+0000","last_became_peered":"2026-02-24T22:05:24.199525+0000","last_unstale":"2026-02-24T22:34:18.547089+0000","last_undegraded":"2026-02-24T22:34:18.547089+0000","last_fullsized":"2026-02-24T22:34:18.547089+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":200,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:06:50.680774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32009,"num_objects":103,"num_object_clones":0,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":506,"num_read_kb":490,"num_write":606,"num_write_kb":207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:29.740751+0000","last_change":"2026-02-24T22:05:29.652942+0000","last_active":"2026-02-24T22:31:29.740751+0000","last_peered":"2026-02-24T22:31:29.740751+0000","last_clean":"2026-02-24T22:31:29.740751+0000","last_became_active":"2026-02-24T22:05:29.652598+0000","last_became_peered":"2026-02-24T22:05:29.652598+0000","last_unstale":"2026-02-24T22:31:29.740751+0000","last_undegraded":"2026-02-24T22:31:29.740751+0000","last_fullsized":"2026-02-24T22:31:29.740751+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:38:19.564467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"312'169","reported_seq":870,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:07.869080+0000","last_change":"2026-02-24T22:05:18.185305+0000","last_active":"2026-02-24T22:30:07.869080+0000","last_peered":"2026-02-24T22:30:07.869080+0000","last_clean":"2026-02-24T22:30:07.869080+0000","last_became_active":"2026-02-24T22:05:18.185152+0000","last_became_peered":"2026-02-24T22:05:18.185152+0000","last_unstale":"2026-02-24T22:30:07.869080+0000","last_undegraded":"2026-02-24T22:30:07.869080+0000","last_fullsized":"2026-02-24T22:30:07.869080+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:54:16.288957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":651,"num_read_kb":540,"num_write":239,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"312'322","reported_seq":1040,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.965329+0000","last_change":"2026-02-24T22:05:16.190979+0000","last_active":"2026-02-24T22:34:25.965329+0000","last_peered":"2026-02-24T22:34:25.965329+0000","last_clean":"2026-02-24T22:34:25.965329+0000","last_became_active":"2026-02-24T22:05:16.190205+0000","last_became_peered":"2026-02-24T22:05:16.190205+0000","last_unstale":"2026-02-24T22:34:25.965329+0000","last_undegraded":"2026-02-24T22:34:25.965329+0000","last_fullsized":"2026-02-24T22:34:25.965329+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":300,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:07:24.667384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":578,"num_read_kb":20088,"num_write":1296,"num_write_kb":14513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":594,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.722800+0000","last_change":"2026-02-24T22:05:20.186884+0000","last_active":"2026-02-24T22:31:24.722800+0000","last_peered":"2026-02-24T22:31:24.722800+0000","last_clean":"2026-02-24T22:31:24.722800+0000","last_became_active":"2026-02-24T22:05:20.186781+0000","last_became_peered":"2026-02-24T22:05:20.186781+0000","last_unstale":"2026-02-24T22:31:24.722800+0000","last_undegraded":"2026-02-24T22:31:24.722800+0000","last_fullsized":"2026-02-24T22:31:24.722800+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:19:52.348059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"312'401","reported_seq":1021,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.972154+0000","last_change":"2026-02-24T22:05:22.192711+0000","last_active":"2026-02-24T22:34:25.972154+0000","last_peered":"2026-02-24T22:34:25.972154+0000","last_clean":"2026-02-24T22:34:25.972154+0000","last_became_active":"2026-02-24T22:05:22.192495+0000","last_became_peered":"2026-02-24T22:05:22.192495+0000","last_unstale":"2026-02-24T22:34:25.972154+0000","last_undegraded":"2026-02-24T22:34:25.972154+0000","last_fullsized":"2026-02-24T22:34:25.972154+0000","mapping_epoch":28,"log_start":"312'300","ondisk_log_start":"312'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":300,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:40:20.670168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":81867,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":699,"num_read_kb":1918,"num_write":658,"num_write_kb":626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"312'173","reported_seq":825,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:03.064297+0000","last_change":"2026-02-24T22:05:24.198313+0000","last_active":"2026-02-24T22:34:03.064297+0000","last_peered":"2026-02-24T22:34:03.064297+0000","last_clean":"2026-02-24T22:34:03.064297+0000","last_became_active":"2026-02-24T22:05:24.198114+0000","last_became_peered":"2026-02-24T22:05:24.198114+0000","last_unstale":"2026-02-24T22:34:03.064297+0000","last_undegraded":"2026-02-24T22:34:03.064297+0000","last_fullsized":"2026-02-24T22:34:03.064297+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":100,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:33:14.283058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29501,"num_objects":91,"num_object_clones":0,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":235,"num_read_kb":226,"num_write":429,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:29.726754+0000","last_change":"2026-02-24T22:05:29.652942+0000","last_active":"2026-02-24T22:31:29.726754+0000","last_peered":"2026-02-24T22:31:29.726754+0000","last_clean":"2026-02-24T22:31:29.726754+0000","last_became_active":"2026-02-24T22:05:29.652715+0000","last_became_peered":"2026-02-24T22:05:29.652715+0000","last_unstale":"2026-02-24T22:31:29.726754+0000","last_undegraded":"2026-02-24T22:31:29.726754+0000","last_fullsized":"2026-02-24T22:31:29.726754+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:13:37.213497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"312'571","reported_seq":1430,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:47.451269+0000","last_change":"2026-02-24T22:05:18.193095+0000","last_active":"2026-02-24T22:30:47.451269+0000","last_peered":"2026-02-24T22:30:47.451269+0000","last_clean":"2026-02-24T22:30:47.451269+0000","last_became_active":"2026-02-24T22:05:18.192543+0000","last_became_peered":"2026-02-24T22:05:18.192543+0000","last_unstale":"2026-02-24T22:30:47.451269+0000","last_undegraded":"2026-02-24T22:30:47.451269+0000","last_fullsized":"2026-02-24T22:30:47.451269+0000","mapping_epoch":24,"log_start":"312'500","ondisk_log_start":"312'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":500,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:42:22.975827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":2576,"num_read_kb":2171,"num_write":966,"num_write_kb":601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"312'255","reported_seq":929,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.960676+0000","last_change":"2026-02-24T22:05:16.184025+0000","last_active":"2026-02-24T22:34:25.960676+0000","last_peered":"2026-02-24T22:34:25.960676+0000","last_clean":"2026-02-24T22:34:25.960676+0000","last_became_active":"2026-02-24T22:05:16.182928+0000","last_became_peered":"2026-02-24T22:05:16.182928+0000","last_unstale":"2026-02-24T22:34:25.960676+0000","last_undegraded":"2026-02-24T22:34:25.960676+0000","last_fullsized":"2026-02-24T22:34:25.960676+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:19:49.977609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":316,"num_read_kb":262,"num_write":1083,"num_write_kb":181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":593,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.715020+0000","last_change":"2026-02-24T22:05:20.186597+0000","last_active":"2026-02-24T22:30:04.715020+0000","last_peered":"2026-02-24T22:30:04.715020+0000","last_clean":"2026-02-24T22:30:04.715020+0000","last_became_active":"2026-02-24T22:05:20.186209+0000","last_became_peered":"2026-02-24T22:05:20.186209+0000","last_unstale":"2026-02-24T22:30:04.715020+0000","last_undegraded":"2026-02-24T22:30:04.715020+0000","last_fullsized":"2026-02-24T22:30:04.715020+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:26:06.692781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"312'948","reported_seq":2268,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.511022+0000","last_change":"2026-02-24T22:05:22.200373+0000","last_active":"2026-02-24T22:34:25.511022+0000","last_peered":"2026-02-24T22:34:25.511022+0000","last_clean":"2026-02-24T22:34:25.511022+0000","last_became_active":"2026-02-24T22:05:22.200111+0000","last_became_peered":"2026-02-24T22:05:22.200111+0000","last_unstale":"2026-02-24T22:34:25.511022+0000","last_undegraded":"2026-02-24T22:34:25.511022+0000","last_fullsized":"2026-02-24T22:34:25.511022+0000","mapping_epoch":28,"log_start":"312'900","ondisk_log_start":"312'900","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":900,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:33:19.581577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37978,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":3232,"num_read_kb":2382,"num_write":419,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"312'183","reported_seq":775,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.249482+0000","last_change":"2026-02-24T22:05:24.201417+0000","last_active":"2026-02-24T22:34:17.249482+0000","last_peered":"2026-02-24T22:34:17.249482+0000","last_clean":"2026-02-24T22:34:17.249482+0000","last_became_active":"2026-02-24T22:05:24.200878+0000","last_became_peered":"2026-02-24T22:05:24.200878+0000","last_unstale":"2026-02-24T22:34:17.249482+0000","last_undegraded":"2026-02-24T22:34:17.249482+0000","last_fullsized":"2026-02-24T22:34:17.249482+0000","mapping_epoch":30,"log_start":"37'100","ondisk_log_start":"37'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":100,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:01:48.708129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33021,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":110,"num_read_kb":109,"num_write":447,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"312'439","reported_seq":2200,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.316960+0000","last_change":"2026-02-24T22:05:29.654545+0000","last_active":"2026-02-24T22:34:25.316960+0000","last_peered":"2026-02-24T22:34:25.316960+0000","last_clean":"2026-02-24T22:34:25.316960+0000","last_became_active":"2026-02-24T22:05:29.654064+0000","last_became_peered":"2026-02-24T22:05:29.654064+0000","last_unstale":"2026-02-24T22:34:25.316960+0000","last_undegraded":"2026-02-24T22:34:25.316960+0000","last_fullsized":"2026-02-24T22:34:25.316960+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":400,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:01:24.704003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"312'128","reported_seq":810,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.731309+0000","last_change":"2026-02-24T22:05:18.192490+0000","last_active":"2026-02-24T22:30:04.731309+0000","last_peered":"2026-02-24T22:30:04.731309+0000","last_clean":"2026-02-24T22:30:04.731309+0000","last_became_active":"2026-02-24T22:05:18.191055+0000","last_became_peered":"2026-02-24T22:05:18.191055+0000","last_unstale":"2026-02-24T22:30:04.731309+0000","last_undegraded":"2026-02-24T22:30:04.731309+0000","last_fullsized":"2026-02-24T22:30:04.731309+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:47:55.429098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":486,"num_read_kb":398,"num_write":170,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"312'240","reported_seq":947,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.962994+0000","last_change":"2026-02-24T22:05:16.187505+0000","last_active":"2026-02-24T22:34:25.962994+0000","last_peered":"2026-02-24T22:34:25.962994+0000","last_clean":"2026-02-24T22:34:25.962994+0000","last_became_active":"2026-02-24T22:05:16.187261+0000","last_became_peered":"2026-02-24T22:05:16.187261+0000","last_unstale":"2026-02-24T22:34:25.962994+0000","last_undegraded":"2026-02-24T22:34:25.962994+0000","last_fullsized":"2026-02-24T22:34:25.962994+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":200,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:13:37.546225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":370,"num_read_kb":24764,"num_write":957,"num_write_kb":10562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":588,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.718234+0000","last_change":"2026-02-24T22:05:20.187956+0000","last_active":"2026-02-24T22:30:04.718234+0000","last_peered":"2026-02-24T22:30:04.718234+0000","last_clean":"2026-02-24T22:30:04.718234+0000","last_became_active":"2026-02-24T22:05:20.187488+0000","last_became_peered":"2026-02-24T22:05:20.187488+0000","last_unstale":"2026-02-24T22:30:04.718234+0000","last_undegraded":"2026-02-24T22:30:04.718234+0000","last_fullsized":"2026-02-24T22:30:04.718234+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:08:51.284932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"312'468","reported_seq":1324,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.376504+0000","last_change":"2026-02-24T22:05:22.193340+0000","last_active":"2026-02-24T22:34:25.376504+0000","last_peered":"2026-02-24T22:34:25.376504+0000","last_clean":"2026-02-24T22:34:25.376504+0000","last_became_active":"2026-02-24T22:05:22.193142+0000","last_became_peered":"2026-02-24T22:05:22.193142+0000","last_unstale":"2026-02-24T22:34:25.376504+0000","last_undegraded":"2026-02-24T22:34:25.376504+0000","last_fullsized":"2026-02-24T22:34:25.376504+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":400,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:37:15.676612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38313,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1308,"num_read_kb":1368,"num_write":335,"num_write_kb":281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"312'222","reported_seq":821,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.886791+0000","last_change":"2026-02-24T22:05:24.198994+0000","last_active":"2026-02-24T22:34:17.886791+0000","last_peered":"2026-02-24T22:34:17.886791+0000","last_clean":"2026-02-24T22:34:17.886791+0000","last_became_active":"2026-02-24T22:05:24.198669+0000","last_became_peered":"2026-02-24T22:05:24.198669+0000","last_unstale":"2026-02-24T22:34:17.886791+0000","last_undegraded":"2026-02-24T22:34:17.886791+0000","last_fullsized":"2026-02-24T22:34:17.886791+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":200,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:38:36.897067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29328,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":170,"num_read_kb":165,"num_write":551,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.720723+0000","last_change":"2026-02-24T22:05:29.653411+0000","last_active":"2026-02-24T22:30:29.720723+0000","last_peered":"2026-02-24T22:30:29.720723+0000","last_clean":"2026-02-24T22:30:29.720723+0000","last_became_active":"2026-02-24T22:05:29.653144+0000","last_became_peered":"2026-02-24T22:05:29.653144+0000","last_unstale":"2026-02-24T22:30:29.720723+0000","last_undegraded":"2026-02-24T22:30:29.720723+0000","last_fullsized":"2026-02-24T22:30:29.720723+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:09:20.796950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"312'171","reported_seq":875,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:14.602760+0000","last_change":"2026-02-24T22:05:18.191001+0000","last_active":"2026-02-24T22:34:14.602760+0000","last_peered":"2026-02-24T22:34:14.602760+0000","last_clean":"2026-02-24T22:34:14.602760+0000","last_became_active":"2026-02-24T22:05:18.190140+0000","last_became_peered":"2026-02-24T22:05:18.190140+0000","last_unstale":"2026-02-24T22:34:14.602760+0000","last_undegraded":"2026-02-24T22:34:14.602760+0000","last_fullsized":"2026-02-24T22:34:14.602760+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:34:38.707313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":675,"num_read_kb":556,"num_write":238,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"312'300","reported_seq":1061,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.969989+0000","last_change":"2026-02-24T22:05:16.185212+0000","last_active":"2026-02-24T22:34:25.969989+0000","last_peered":"2026-02-24T22:34:25.969989+0000","last_clean":"2026-02-24T22:34:25.969989+0000","last_became_active":"2026-02-24T22:05:16.184944+0000","last_became_peered":"2026-02-24T22:05:16.184944+0000","last_unstale":"2026-02-24T22:34:25.969989+0000","last_undegraded":"2026-02-24T22:34:25.969989+0000","last_fullsized":"2026-02-24T22:34:25.969989+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":200,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:42:08.775595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":538,"num_read_kb":45888,"num_write":1354,"num_write_kb":20430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"27'2","reported_seq":1057,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.172636+0000","last_change":"2026-02-24T22:05:20.194961+0000","last_active":"2026-02-24T22:34:25.172636+0000","last_peered":"2026-02-24T22:34:25.172636+0000","last_clean":"2026-02-24T22:34:25.172636+0000","last_became_active":"2026-02-24T22:05:20.194145+0000","last_became_peered":"2026-02-24T22:05:20.194145+0000","last_unstale":"2026-02-24T22:34:25.172636+0000","last_undegraded":"2026-02-24T22:34:25.172636+0000","last_fullsized":"2026-02-24T22:34:25.172636+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:18:36.330182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":476,"num_read_kb":476,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"312'56","reported_seq":657,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.737323+0000","last_change":"2026-02-24T22:05:22.905477+0000","last_active":"2026-02-24T22:32:24.737323+0000","last_peered":"2026-02-24T22:32:24.737323+0000","last_clean":"2026-02-24T22:32:24.737323+0000","last_became_active":"2026-02-24T22:05:22.905221+0000","last_became_peered":"2026-02-24T22:05:22.905221+0000","last_unstale":"2026-02-24T22:32:24.737323+0000","last_undegraded":"2026-02-24T22:32:24.737323+0000","last_fullsized":"2026-02-24T22:32:24.737323+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:59:47.641153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"312'159","reported_seq":744,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:14.502434+0000","last_change":"2026-02-24T22:05:24.208011+0000","last_active":"2026-02-24T22:34:14.502434+0000","last_peered":"2026-02-24T22:34:14.502434+0000","last_clean":"2026-02-24T22:34:14.502434+0000","last_became_active":"2026-02-24T22:05:24.207673+0000","last_became_peered":"2026-02-24T22:05:24.207673+0000","last_unstale":"2026-02-24T22:34:14.502434+0000","last_undegraded":"2026-02-24T22:34:14.502434+0000","last_fullsized":"2026-02-24T22:34:14.502434+0000","mapping_epoch":30,"log_start":"312'100","ondisk_log_start":"312'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:29:55.011561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25987,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":107,"num_read_kb":103,"num_write":388,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":577,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.733645+0000","last_change":"2026-02-24T22:05:29.659816+0000","last_active":"2026-02-24T22:30:29.733645+0000","last_peered":"2026-02-24T22:30:29.733645+0000","last_clean":"2026-02-24T22:30:29.733645+0000","last_became_active":"2026-02-24T22:05:29.658842+0000","last_became_peered":"2026-02-24T22:05:29.658842+0000","last_unstale":"2026-02-24T22:30:29.733645+0000","last_undegraded":"2026-02-24T22:30:29.733645+0000","last_fullsized":"2026-02-24T22:30:29.733645+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:55:04.860078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"312'98","reported_seq":778,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:54.820404+0000","last_change":"2026-02-24T22:05:18.188396+0000","last_active":"2026-02-24T22:30:54.820404+0000","last_peered":"2026-02-24T22:30:54.820404+0000","last_clean":"2026-02-24T22:30:54.820404+0000","last_became_active":"2026-02-24T22:05:18.188161+0000","last_became_peered":"2026-02-24T22:05:18.188161+0000","last_unstale":"2026-02-24T22:30:54.820404+0000","last_undegraded":"2026-02-24T22:30:54.820404+0000","last_fullsized":"2026-02-24T22:30:54.820404+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:15:32.648520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":368,"num_read_kb":314,"num_write":134,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"312'356","reported_seq":1088,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.980773+0000","last_change":"2026-02-24T22:05:16.182699+0000","last_active":"2026-02-24T22:34:25.980773+0000","last_peered":"2026-02-24T22:34:25.980773+0000","last_clean":"2026-02-24T22:34:25.980773+0000","last_became_active":"2026-02-24T22:05:16.182194+0000","last_became_peered":"2026-02-24T22:05:16.182194+0000","last_unstale":"2026-02-24T22:34:25.980773+0000","last_undegraded":"2026-02-24T22:34:25.980773+0000","last_fullsized":"2026-02-24T22:34:25.980773+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":300,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:40:05.294459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":482,"num_read_kb":7457,"num_write":1288,"num_write_kb":8933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.731323+0000","last_change":"2026-02-24T22:05:20.193494+0000","last_active":"2026-02-24T22:30:04.731323+0000","last_peered":"2026-02-24T22:30:04.731323+0000","last_clean":"2026-02-24T22:30:04.731323+0000","last_became_active":"2026-02-24T22:05:20.193180+0000","last_became_peered":"2026-02-24T22:05:20.193180+0000","last_unstale":"2026-02-24T22:30:04.731323+0000","last_undegraded":"2026-02-24T22:30:04.731323+0000","last_fullsized":"2026-02-24T22:30:04.731323+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:22:52.729272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"312'275","reported_seq":900,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.781596+0000","last_change":"2026-02-24T22:05:22.200410+0000","last_active":"2026-02-24T22:34:25.781596+0000","last_peered":"2026-02-24T22:34:25.781596+0000","last_clean":"2026-02-24T22:34:25.781596+0000","last_became_active":"2026-02-24T22:05:22.200138+0000","last_became_peered":"2026-02-24T22:05:22.200138+0000","last_unstale":"2026-02-24T22:34:25.781596+0000","last_undegraded":"2026-02-24T22:34:25.781596+0000","last_fullsized":"2026-02-24T22:34:25.781596+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":200,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:07:05.040956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37516,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":871,"num_read_kb":1412,"num_write":1276,"num_write_kb":1198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"312'327","reported_seq":951,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.407864+0000","last_change":"2026-02-24T22:05:24.207564+0000","last_active":"2026-02-24T22:34:17.407864+0000","last_peered":"2026-02-24T22:34:17.407864+0000","last_clean":"2026-02-24T22:34:17.407864+0000","last_became_active":"2026-02-24T22:05:24.206745+0000","last_became_peered":"2026-02-24T22:05:24.206745+0000","last_unstale":"2026-02-24T22:34:17.407864+0000","last_undegraded":"2026-02-24T22:34:17.407864+0000","last_fullsized":"2026-02-24T22:34:17.407864+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":300,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:53:45.833646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27098,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":391,"num_read_kb":376,"num_write":726,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"312'435","reported_seq":4799,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.200248+0000","last_change":"2026-02-24T22:05:29.654842+0000","last_active":"2026-02-24T22:34:25.200248+0000","last_peered":"2026-02-24T22:34:25.200248+0000","last_clean":"2026-02-24T22:34:25.200248+0000","last_became_active":"2026-02-24T22:05:29.654281+0000","last_became_peered":"2026-02-24T22:05:29.654281+0000","last_unstale":"2026-02-24T22:34:25.200248+0000","last_undegraded":"2026-02-24T22:34:25.200248+0000","last_fullsized":"2026-02-24T22:34:25.200248+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":400,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:30:38.038033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"312'114","reported_seq":799,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:05.704625+0000","last_change":"2026-02-24T22:05:18.193654+0000","last_active":"2026-02-24T22:34:05.704625+0000","last_peered":"2026-02-24T22:34:05.704625+0000","last_clean":"2026-02-24T22:34:05.704625+0000","last_became_active":"2026-02-24T22:05:18.193426+0000","last_became_peered":"2026-02-24T22:05:18.193426+0000","last_unstale":"2026-02-24T22:34:05.704625+0000","last_undegraded":"2026-02-24T22:34:05.704625+0000","last_fullsized":"2026-02-24T22:34:05.704625+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:36:20.176524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":443,"num_read_kb":376,"num_write":199,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'112","reported_seq":745,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.741431+0000","last_change":"2026-02-24T22:05:06.709275+0000","last_active":"2026-02-24T22:31:14.741431+0000","last_peered":"2026-02-24T22:31:14.741431+0000","last_clean":"2026-02-24T22:31:14.741431+0000","last_became_active":"2026-02-24T22:05:06.708577+0000","last_became_peered":"2026-02-24T22:05:06.708577+0000","last_unstale":"2026-02-24T22:31:14.741431+0000","last_undegraded":"2026-02-24T22:31:14.741431+0000","last_fullsized":"2026-02-24T22:31:14.741431+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:05.664397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:05.664397+0000","last_clean_scrub_stamp":"2026-02-24T22:05:05.664397+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:46:55.314440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"18'1","reported_seq":605,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:14.747819+0000","last_change":"2026-02-24T22:05:10.694707+0000","last_active":"2026-02-24T22:32:14.747819+0000","last_peered":"2026-02-24T22:32:14.747819+0000","last_clean":"2026-02-24T22:32:14.747819+0000","last_became_active":"2026-02-24T22:05:08.689983+0000","last_became_peered":"2026-02-24T22:05:08.689983+0000","last_unstale":"2026-02-24T22:32:14.747819+0000","last_undegraded":"2026-02-24T22:32:14.747819+0000","last_fullsized":"2026-02-24T22:32:14.747819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:42:42.134312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046251899999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"312'326","reported_seq":1105,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.991874+0000","last_change":"2026-02-24T22:05:16.186666+0000","last_active":"2026-02-24T22:34:25.991874+0000","last_peered":"2026-02-24T22:34:25.991874+0000","last_clean":"2026-02-24T22:34:25.991874+0000","last_became_active":"2026-02-24T22:05:16.185768+0000","last_became_peered":"2026-02-24T22:05:16.185768+0000","last_unstale":"2026-02-24T22:34:25.991874+0000","last_undegraded":"2026-02-24T22:34:25.991874+0000","last_fullsized":"2026-02-24T22:34:25.991874+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":300,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:33:06.518377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":685,"num_read_kb":12844,"num_write":1503,"num_write_kb":6232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":587,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.749491+0000","last_change":"2026-02-24T22:05:20.194414+0000","last_active":"2026-02-24T22:32:24.749491+0000","last_peered":"2026-02-24T22:32:24.749491+0000","last_clean":"2026-02-24T22:32:24.749491+0000","last_became_active":"2026-02-24T22:05:20.194040+0000","last_became_peered":"2026-02-24T22:05:20.194040+0000","last_unstale":"2026-02-24T22:32:24.749491+0000","last_undegraded":"2026-02-24T22:32:24.749491+0000","last_fullsized":"2026-02-24T22:32:24.749491+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:14:22.868157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"312'1066","reported_seq":2175,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.830271+0000","last_change":"2026-02-24T22:05:22.199991+0000","last_active":"2026-02-24T22:34:25.830271+0000","last_peered":"2026-02-24T22:34:25.830271+0000","last_clean":"2026-02-24T22:34:25.830271+0000","last_became_active":"2026-02-24T22:05:22.199602+0000","last_became_peered":"2026-02-24T22:05:22.199602+0000","last_unstale":"2026-02-24T22:34:25.830271+0000","last_undegraded":"2026-02-24T22:34:25.830271+0000","last_fullsized":"2026-02-24T22:34:25.830271+0000","mapping_epoch":28,"log_start":"312'1000","ondisk_log_start":"312'1000","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":1000,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:00:35.201833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":116028,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2941,"num_read_kb":3705,"num_write":1026,"num_write_kb":950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"312'202","reported_seq":795,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.242020+0000","last_change":"2026-02-24T22:05:24.201041+0000","last_active":"2026-02-24T22:34:18.242020+0000","last_peered":"2026-02-24T22:34:18.242020+0000","last_clean":"2026-02-24T22:34:18.242020+0000","last_became_active":"2026-02-24T22:05:24.200336+0000","last_became_peered":"2026-02-24T22:05:24.200336+0000","last_unstale":"2026-02-24T22:34:18.242020+0000","last_undegraded":"2026-02-24T22:34:18.242020+0000","last_fullsized":"2026-02-24T22:34:18.242020+0000","mapping_epoch":30,"log_start":"37'100","ondisk_log_start":"37'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:52:42.524705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32535,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":152,"num_read_kb":139,"num_write":501,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"312'428","reported_seq":1836,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.720314+0000","last_change":"2026-02-24T22:05:29.660713+0000","last_active":"2026-02-24T22:34:25.720314+0000","last_peered":"2026-02-24T22:34:25.720314+0000","last_clean":"2026-02-24T22:34:25.720314+0000","last_became_active":"2026-02-24T22:05:29.660370+0000","last_became_peered":"2026-02-24T22:05:29.660370+0000","last_unstale":"2026-02-24T22:34:25.720314+0000","last_undegraded":"2026-02-24T22:34:25.720314+0000","last_fullsized":"2026-02-24T22:34:25.720314+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":400,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:47:59.912901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"312'99","reported_seq":797,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.739591+0000","last_change":"2026-02-24T22:05:18.185501+0000","last_active":"2026-02-24T22:31:19.739591+0000","last_peered":"2026-02-24T22:31:19.739591+0000","last_clean":"2026-02-24T22:31:19.739591+0000","last_became_active":"2026-02-24T22:05:18.185205+0000","last_became_peered":"2026-02-24T22:05:18.185205+0000","last_unstale":"2026-02-24T22:31:19.739591+0000","last_undegraded":"2026-02-24T22:31:19.739591+0000","last_fullsized":"2026-02-24T22:31:19.739591+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:21:17.222637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":346,"num_read_kb":300,"num_write":140,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":610,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.741446+0000","last_change":"2026-02-24T22:05:10.808592+0000","last_active":"2026-02-24T22:31:14.741446+0000","last_peered":"2026-02-24T22:31:14.741446+0000","last_clean":"2026-02-24T22:31:14.741446+0000","last_became_active":"2026-02-24T22:05:08.689972+0000","last_became_peered":"2026-02-24T22:05:08.689972+0000","last_unstale":"2026-02-24T22:31:14.741446+0000","last_undegraded":"2026-02-24T22:31:14.741446+0000","last_fullsized":"2026-02-24T22:31:14.741446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:59:56.330904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048345700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"312'262","reported_seq":1029,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:29.701991+0000","last_change":"2026-02-24T22:34:29.701952+0000","last_active":"2026-02-24T22:34:29.701991+0000","last_peered":"2026-02-24T22:34:29.701991+0000","last_clean":"2026-02-24T22:34:29.701991+0000","last_became_active":"2026-02-24T22:05:16.183484+0000","last_became_peered":"2026-02-24T22:05:16.183484+0000","last_unstale":"2026-02-24T22:34:29.701991+0000","last_undegraded":"2026-02-24T22:34:29.701991+0000","last_fullsized":"2026-02-24T22:34:29.701991+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'262","last_scrub_stamp":"2026-02-24T22:34:29.701911+0000","last_deep_scrub":"312'262","last_deep_scrub_stamp":"2026-02-24T22:34:29.701911+0000","last_clean_scrub_stamp":"2026-02-24T22:34:29.701911+0000","objects_scrubbed":4,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:16:54.158907+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":574,"num_read_kb":821,"num_write":1287,"num_write_kb":10723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":580,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.741456+0000","last_change":"2026-02-24T22:05:20.195460+0000","last_active":"2026-02-24T22:31:19.741456+0000","last_peered":"2026-02-24T22:31:19.741456+0000","last_clean":"2026-02-24T22:31:19.741456+0000","last_became_active":"2026-02-24T22:05:20.195236+0000","last_became_peered":"2026-02-24T22:05:20.195236+0000","last_unstale":"2026-02-24T22:31:19.741456+0000","last_undegraded":"2026-02-24T22:31:19.741456+0000","last_fullsized":"2026-02-24T22:31:19.741456+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:38:04.823028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"312'1062","reported_seq":2284,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.818712+0000","last_change":"2026-02-24T22:05:22.192588+0000","last_active":"2026-02-24T22:34:25.818712+0000","last_peered":"2026-02-24T22:34:25.818712+0000","last_clean":"2026-02-24T22:34:25.818712+0000","last_became_active":"2026-02-24T22:05:22.192295+0000","last_became_peered":"2026-02-24T22:05:22.192295+0000","last_unstale":"2026-02-24T22:34:25.818712+0000","last_undegraded":"2026-02-24T22:34:25.818712+0000","last_fullsized":"2026-02-24T22:34:25.818712+0000","mapping_epoch":28,"log_start":"312'1001","ondisk_log_start":"312'1001","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":1001,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:00:22.125417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":114655,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":3192,"num_read_kb":4794,"num_write":3958,"num_write_kb":3891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"312'182","reported_seq":763,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.067423+0000","last_change":"2026-02-24T22:05:24.205091+0000","last_active":"2026-02-24T22:34:15.067423+0000","last_peered":"2026-02-24T22:34:15.067423+0000","last_clean":"2026-02-24T22:34:15.067423+0000","last_became_active":"2026-02-24T22:05:24.204774+0000","last_became_peered":"2026-02-24T22:05:24.204774+0000","last_unstale":"2026-02-24T22:34:15.067423+0000","last_undegraded":"2026-02-24T22:34:15.067423+0000","last_fullsized":"2026-02-24T22:34:15.067423+0000","mapping_epoch":30,"log_start":"312'100","ondisk_log_start":"312'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":100,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:38:46.991451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29814,"num_objects":89,"num_object_clones":0,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":0,"num_read":112,"num_read_kb":108,"num_write":454,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.727262+0000","last_change":"2026-02-24T22:05:29.653742+0000","last_active":"2026-02-24T22:31:34.727262+0000","last_peered":"2026-02-24T22:31:34.727262+0000","last_clean":"2026-02-24T22:31:34.727262+0000","last_became_active":"2026-02-24T22:05:29.653391+0000","last_became_peered":"2026-02-24T22:05:29.653391+0000","last_unstale":"2026-02-24T22:31:34.727262+0000","last_undegraded":"2026-02-24T22:31:34.727262+0000","last_fullsized":"2026-02-24T22:31:34.727262+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:11:46.647600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"312'101","reported_seq":791,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.725723+0000","last_change":"2026-02-24T22:05:18.186987+0000","last_active":"2026-02-24T22:31:19.725723+0000","last_peered":"2026-02-24T22:31:19.725723+0000","last_clean":"2026-02-24T22:31:19.725723+0000","last_became_active":"2026-02-24T22:05:18.186126+0000","last_became_peered":"2026-02-24T22:05:18.186126+0000","last_unstale":"2026-02-24T22:31:19.725723+0000","last_undegraded":"2026-02-24T22:31:19.725723+0000","last_fullsized":"2026-02-24T22:31:19.725723+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:14:57.108610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":336,"num_read_kb":289,"num_write":128,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":618,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:14.727952+0000","last_change":"2026-02-24T22:05:10.684538+0000","last_active":"2026-02-24T22:32:14.727952+0000","last_peered":"2026-02-24T22:32:14.727952+0000","last_clean":"2026-02-24T22:32:14.727952+0000","last_became_active":"2026-02-24T22:05:08.682035+0000","last_became_peered":"2026-02-24T22:05:08.682035+0000","last_unstale":"2026-02-24T22:32:14.727952+0000","last_undegraded":"2026-02-24T22:32:14.727952+0000","last_fullsized":"2026-02-24T22:32:14.727952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:59:46.243250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047531599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"312'299","reported_seq":1043,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.969265+0000","last_change":"2026-02-24T22:05:16.178900+0000","last_active":"2026-02-24T22:34:25.969265+0000","last_peered":"2026-02-24T22:34:25.969265+0000","last_clean":"2026-02-24T22:34:25.969265+0000","last_became_active":"2026-02-24T22:05:16.178331+0000","last_became_peered":"2026-02-24T22:05:16.178331+0000","last_unstale":"2026-02-24T22:34:25.969265+0000","last_undegraded":"2026-02-24T22:34:25.969265+0000","last_fullsized":"2026-02-24T22:34:25.969265+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":200,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:08:05.851722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":583,"num_read_kb":6589,"num_write":1333,"num_write_kb":11354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"27'1","reported_seq":595,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.722286+0000","last_change":"2026-02-24T22:05:20.188183+0000","last_active":"2026-02-24T22:31:19.722286+0000","last_peered":"2026-02-24T22:31:19.722286+0000","last_clean":"2026-02-24T22:31:19.722286+0000","last_became_active":"2026-02-24T22:05:20.187908+0000","last_became_peered":"2026-02-24T22:05:20.187908+0000","last_unstale":"2026-02-24T22:31:19.722286+0000","last_undegraded":"2026-02-24T22:31:19.722286+0000","last_fullsized":"2026-02-24T22:31:19.722286+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:49:17.694606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"312'437","reported_seq":1646,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.225084+0000","last_change":"2026-02-24T22:05:22.912798+0000","last_active":"2026-02-24T22:34:25.225084+0000","last_peered":"2026-02-24T22:34:25.225084+0000","last_clean":"2026-02-24T22:34:25.225084+0000","last_became_active":"2026-02-24T22:05:22.912414+0000","last_became_peered":"2026-02-24T22:05:22.912414+0000","last_unstale":"2026-02-24T22:34:25.225084+0000","last_undegraded":"2026-02-24T22:34:25.225084+0000","last_fullsized":"2026-02-24T22:34:25.225084+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":400,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:05:47.634681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":308,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1080,"num_read_kb":599,"num_write":114,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"312'184","reported_seq":770,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.551095+0000","last_change":"2026-02-24T22:05:24.200671+0000","last_active":"2026-02-24T22:34:18.551095+0000","last_peered":"2026-02-24T22:34:18.551095+0000","last_clean":"2026-02-24T22:34:18.551095+0000","last_became_active":"2026-02-24T22:05:24.200029+0000","last_became_peered":"2026-02-24T22:05:24.200029+0000","last_unstale":"2026-02-24T22:34:18.551095+0000","last_undegraded":"2026-02-24T22:34:18.551095+0000","last_fullsized":"2026-02-24T22:34:18.551095+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":100,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:18:37.550289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29918,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":442,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":576,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.724006+0000","last_change":"2026-02-24T22:05:29.653992+0000","last_active":"2026-02-24T22:30:14.724006+0000","last_peered":"2026-02-24T22:30:14.724006+0000","last_clean":"2026-02-24T22:30:14.724006+0000","last_became_active":"2026-02-24T22:05:29.653806+0000","last_became_peered":"2026-02-24T22:05:29.653806+0000","last_unstale":"2026-02-24T22:30:14.724006+0000","last_undegraded":"2026-02-24T22:30:14.724006+0000","last_fullsized":"2026-02-24T22:30:14.724006+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:44:18.672652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"312'230","reported_seq":971,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.732440+0000","last_change":"2026-02-24T22:05:18.184825+0000","last_active":"2026-02-24T22:32:24.732440+0000","last_peered":"2026-02-24T22:32:24.732440+0000","last_clean":"2026-02-24T22:32:24.732440+0000","last_became_active":"2026-02-24T22:05:18.183952+0000","last_became_peered":"2026-02-24T22:05:18.183952+0000","last_unstale":"2026-02-24T22:32:24.732440+0000","last_undegraded":"2026-02-24T22:32:24.732440+0000","last_fullsized":"2026-02-24T22:32:24.732440+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":963,"num_read_kb":811,"num_write":374,"num_write_kb":210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'2","reported_seq":606,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:14.747800+0000","last_change":"2026-02-24T22:05:10.695226+0000","last_active":"2026-02-24T22:32:14.747800+0000","last_peered":"2026-02-24T22:32:14.747800+0000","last_clean":"2026-02-24T22:32:14.747800+0000","last_became_active":"2026-02-24T22:05:08.690042+0000","last_became_peered":"2026-02-24T22:05:08.690042+0000","last_unstale":"2026-02-24T22:32:14.747800+0000","last_undegraded":"2026-02-24T22:32:14.747800+0000","last_fullsized":"2026-02-24T22:32:14.747800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:27:03.349026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010128400000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"312'327","reported_seq":1042,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.970855+0000","last_change":"2026-02-24T22:05:16.189390+0000","last_active":"2026-02-24T22:34:25.970855+0000","last_peered":"2026-02-24T22:34:25.970855+0000","last_clean":"2026-02-24T22:34:25.970855+0000","last_became_active":"2026-02-24T22:05:16.188900+0000","last_became_peered":"2026-02-24T22:05:16.188900+0000","last_unstale":"2026-02-24T22:34:25.970855+0000","last_undegraded":"2026-02-24T22:34:25.970855+0000","last_fullsized":"2026-02-24T22:34:25.970855+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":300,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:22:52.382242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242895,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":415,"num_read_kb":10969,"num_write":1277,"num_write_kb":10859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":589,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.733650+0000","last_change":"2026-02-24T22:05:20.187605+0000","last_active":"2026-02-24T22:30:24.733650+0000","last_peered":"2026-02-24T22:30:24.733650+0000","last_clean":"2026-02-24T22:30:24.733650+0000","last_became_active":"2026-02-24T22:05:20.187251+0000","last_became_peered":"2026-02-24T22:05:20.187251+0000","last_unstale":"2026-02-24T22:30:24.733650+0000","last_undegraded":"2026-02-24T22:30:24.733650+0000","last_fullsized":"2026-02-24T22:30:24.733650+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:47:16.465774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"312'76","reported_seq":697,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.731188+0000","last_change":"2026-02-24T22:05:22.191860+0000","last_active":"2026-02-24T22:31:24.731188+0000","last_peered":"2026-02-24T22:31:24.731188+0000","last_clean":"2026-02-24T22:31:24.731188+0000","last_became_active":"2026-02-24T22:05:22.191394+0000","last_became_peered":"2026-02-24T22:05:22.191394+0000","last_unstale":"2026-02-24T22:31:24.731188+0000","last_undegraded":"2026-02-24T22:31:24.731188+0000","last_fullsized":"2026-02-24T22:31:24.731188+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:12:53.711063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":103,"num_read_kb":96,"num_write":72,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"312'229","reported_seq":843,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.223179+0000","last_change":"2026-02-24T22:05:24.208017+0000","last_active":"2026-02-24T22:34:17.223179+0000","last_peered":"2026-02-24T22:34:17.223179+0000","last_clean":"2026-02-24T22:34:17.223179+0000","last_became_active":"2026-02-24T22:05:24.207698+0000","last_became_peered":"2026-02-24T22:05:24.207698+0000","last_unstale":"2026-02-24T22:34:17.223179+0000","last_undegraded":"2026-02-24T22:34:17.223179+0000","last_fullsized":"2026-02-24T22:34:17.223179+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":200,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:35:36.488781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31273,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":246,"num_read_kb":224,"num_write":553,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":566,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:29.742534+0000","last_change":"2026-02-24T22:05:29.660487+0000","last_active":"2026-02-24T22:31:29.742534+0000","last_peered":"2026-02-24T22:31:29.742534+0000","last_clean":"2026-02-24T22:31:29.742534+0000","last_became_active":"2026-02-24T22:05:29.660065+0000","last_became_peered":"2026-02-24T22:05:29.660065+0000","last_unstale":"2026-02-24T22:31:29.742534+0000","last_undegraded":"2026-02-24T22:31:29.742534+0000","last_fullsized":"2026-02-24T22:31:29.742534+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:00:34.244351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"312'172","reported_seq":870,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:42.929432+0000","last_change":"2026-02-24T22:05:18.185556+0000","last_active":"2026-02-24T22:33:42.929432+0000","last_peered":"2026-02-24T22:33:42.929432+0000","last_clean":"2026-02-24T22:33:42.929432+0000","last_became_active":"2026-02-24T22:05:18.185112+0000","last_became_peered":"2026-02-24T22:05:18.185112+0000","last_unstale":"2026-02-24T22:33:42.929432+0000","last_undegraded":"2026-02-24T22:33:42.929432+0000","last_fullsized":"2026-02-24T22:33:42.929432+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":100,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":679,"num_read_kb":553,"num_write":246,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"312'277","reported_seq":998,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.950623+0000","last_change":"2026-02-24T22:05:16.191455+0000","last_active":"2026-02-24T22:34:25.950623+0000","last_peered":"2026-02-24T22:34:25.950623+0000","last_clean":"2026-02-24T22:34:25.950623+0000","last_became_active":"2026-02-24T22:05:16.191007+0000","last_became_peered":"2026-02-24T22:05:16.191007+0000","last_unstale":"2026-02-24T22:34:25.950623+0000","last_undegraded":"2026-02-24T22:34:25.950623+0000","last_fullsized":"2026-02-24T22:34:25.950623+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":200,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:46:55.713025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":495,"num_read_kb":40330,"num_write":1338,"num_write_kb":18546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":613,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:14.731453+0000","last_change":"2026-02-24T22:05:10.689379+0000","last_active":"2026-02-24T22:32:14.731453+0000","last_peered":"2026-02-24T22:32:14.731453+0000","last_clean":"2026-02-24T22:32:14.731453+0000","last_became_active":"2026-02-24T22:05:08.758282+0000","last_became_peered":"2026-02-24T22:05:08.758282+0000","last_unstale":"2026-02-24T22:32:14.731453+0000","last_undegraded":"2026-02-24T22:32:14.731453+0000","last_fullsized":"2026-02-24T22:32:14.731453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:28:00.429651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033355899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.746687+0000","last_change":"2026-02-24T22:05:20.187544+0000","last_active":"2026-02-24T22:32:24.746687+0000","last_peered":"2026-02-24T22:32:24.746687+0000","last_clean":"2026-02-24T22:32:24.746687+0000","last_became_active":"2026-02-24T22:05:20.187208+0000","last_became_peered":"2026-02-24T22:05:20.187208+0000","last_unstale":"2026-02-24T22:32:24.746687+0000","last_undegraded":"2026-02-24T22:32:24.746687+0000","last_fullsized":"2026-02-24T22:32:24.746687+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"312'1133","reported_seq":2487,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.681905+0000","last_change":"2026-02-24T22:05:22.192049+0000","last_active":"2026-02-24T22:34:25.681905+0000","last_peered":"2026-02-24T22:34:25.681905+0000","last_clean":"2026-02-24T22:34:25.681905+0000","last_became_active":"2026-02-24T22:05:22.191763+0000","last_became_peered":"2026-02-24T22:05:22.191763+0000","last_unstale":"2026-02-24T22:34:25.681905+0000","last_undegraded":"2026-02-24T22:34:25.681905+0000","last_fullsized":"2026-02-24T22:34:25.681905+0000","mapping_epoch":28,"log_start":"312'1100","ondisk_log_start":"312'1100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":1100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:42:04.815445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":76731,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":3517,"num_read_kb":3087,"num_write":637,"num_write_kb":534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"312'204","reported_seq":808,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.405693+0000","last_change":"2026-02-24T22:05:24.206550+0000","last_active":"2026-02-24T22:34:18.405693+0000","last_peered":"2026-02-24T22:34:18.405693+0000","last_clean":"2026-02-24T22:34:18.405693+0000","last_became_active":"2026-02-24T22:05:24.205923+0000","last_became_peered":"2026-02-24T22:05:24.205923+0000","last_unstale":"2026-02-24T22:34:18.405693+0000","last_undegraded":"2026-02-24T22:34:18.405693+0000","last_fullsized":"2026-02-24T22:34:18.405693+0000","mapping_epoch":30,"log_start":"37'100","ondisk_log_start":"37'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:42:44.001897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34551,"num_objects":102,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":190,"num_read_kb":174,"num_write":516,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"312'467","reported_seq":4829,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.965539+0000","last_change":"2026-02-24T22:05:29.652714+0000","last_active":"2026-02-24T22:34:25.965539+0000","last_peered":"2026-02-24T22:34:25.965539+0000","last_clean":"2026-02-24T22:34:25.965539+0000","last_became_active":"2026-02-24T22:05:29.652259+0000","last_became_peered":"2026-02-24T22:05:29.652259+0000","last_unstale":"2026-02-24T22:34:25.965539+0000","last_undegraded":"2026-02-24T22:34:25.965539+0000","last_fullsized":"2026-02-24T22:34:25.965539+0000","mapping_epoch":33,"log_start":"312'401","ondisk_log_start":"312'401","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":401,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:52:38.347176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"312'114","reported_seq":811,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.746694+0000","last_change":"2026-02-24T22:05:18.183793+0000","last_active":"2026-02-24T22:32:24.746694+0000","last_peered":"2026-02-24T22:32:24.746694+0000","last_clean":"2026-02-24T22:32:24.746694+0000","last_became_active":"2026-02-24T22:05:18.183369+0000","last_became_peered":"2026-02-24T22:05:18.183369+0000","last_unstale":"2026-02-24T22:32:24.746694+0000","last_undegraded":"2026-02-24T22:32:24.746694+0000","last_fullsized":"2026-02-24T22:32:24.746694+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":458,"num_read_kb":386,"num_write":168,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"312'497","reported_seq":1308,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.990449+0000","last_change":"2026-02-24T22:05:16.179055+0000","last_active":"2026-02-24T22:34:25.990449+0000","last_peered":"2026-02-24T22:34:25.990449+0000","last_clean":"2026-02-24T22:34:25.990449+0000","last_became_active":"2026-02-24T22:05:16.178353+0000","last_became_peered":"2026-02-24T22:05:16.178353+0000","last_unstale":"2026-02-24T22:34:25.990449+0000","last_undegraded":"2026-02-24T22:34:25.990449+0000","last_fullsized":"2026-02-24T22:34:25.990449+0000","mapping_epoch":22,"log_start":"312'400","ondisk_log_start":"312'400","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":400,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1036,"num_read_kb":9549,"num_write":2046,"num_write_kb":2098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":610,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:19.749000+0000","last_change":"2026-02-24T22:05:10.808508+0000","last_active":"2026-02-24T22:32:19.749000+0000","last_peered":"2026-02-24T22:32:19.749000+0000","last_clean":"2026-02-24T22:32:19.749000+0000","last_became_active":"2026-02-24T22:05:08.765129+0000","last_became_peered":"2026-02-24T22:05:08.765129+0000","last_unstale":"2026-02-24T22:32:19.749000+0000","last_undegraded":"2026-02-24T22:32:19.749000+0000","last_fullsized":"2026-02-24T22:32:19.749000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:05:15.499818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439276,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":585,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.744793+0000","last_change":"2026-02-24T22:05:20.193089+0000","last_active":"2026-02-24T22:32:24.744793+0000","last_peered":"2026-02-24T22:32:24.744793+0000","last_clean":"2026-02-24T22:32:24.744793+0000","last_became_active":"2026-02-24T22:05:20.192811+0000","last_became_peered":"2026-02-24T22:05:20.192811+0000","last_unstale":"2026-02-24T22:32:24.744793+0000","last_undegraded":"2026-02-24T22:32:24.744793+0000","last_fullsized":"2026-02-24T22:32:24.744793+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:09:16.601460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"312'609","reported_seq":1719,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.253806+0000","last_change":"2026-02-24T22:05:22.905928+0000","last_active":"2026-02-24T22:34:25.253806+0000","last_peered":"2026-02-24T22:34:25.253806+0000","last_clean":"2026-02-24T22:34:25.253806+0000","last_became_active":"2026-02-24T22:05:22.905663+0000","last_became_peered":"2026-02-24T22:05:22.905663+0000","last_unstale":"2026-02-24T22:34:25.253806+0000","last_undegraded":"2026-02-24T22:34:25.253806+0000","last_fullsized":"2026-02-24T22:34:25.253806+0000","mapping_epoch":28,"log_start":"307'500","ondisk_log_start":"307'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":500,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:36:30.224989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":308,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":2064,"num_read_kb":1111,"num_write":154,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"312'171","reported_seq":753,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.533054+0000","last_change":"2026-02-24T22:05:24.206213+0000","last_active":"2026-02-24T22:34:18.533054+0000","last_peered":"2026-02-24T22:34:18.533054+0000","last_clean":"2026-02-24T22:34:18.533054+0000","last_became_active":"2026-02-24T22:05:24.205847+0000","last_became_peered":"2026-02-24T22:05:24.205847+0000","last_unstale":"2026-02-24T22:34:18.533054+0000","last_undegraded":"2026-02-24T22:34:18.533054+0000","last_fullsized":"2026-02-24T22:34:18.533054+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:10:54.760147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26185,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":121,"num_read_kb":112,"num_write":413,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":576,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.724548+0000","last_change":"2026-02-24T22:05:29.651555+0000","last_active":"2026-02-24T22:30:19.724548+0000","last_peered":"2026-02-24T22:30:19.724548+0000","last_clean":"2026-02-24T22:30:19.724548+0000","last_became_active":"2026-02-24T22:05:29.651401+0000","last_became_peered":"2026-02-24T22:05:29.651401+0000","last_unstale":"2026-02-24T22:30:19.724548+0000","last_undegraded":"2026-02-24T22:30:19.724548+0000","last_fullsized":"2026-02-24T22:30:19.724548+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:36:54.239488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"312'150","reported_seq":847,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:40.233137+0000","last_change":"2026-02-24T22:05:18.186337+0000","last_active":"2026-02-24T22:33:40.233137+0000","last_peered":"2026-02-24T22:33:40.233137+0000","last_clean":"2026-02-24T22:33:40.233137+0000","last_became_active":"2026-02-24T22:05:18.185592+0000","last_became_peered":"2026-02-24T22:05:18.185592+0000","last_unstale":"2026-02-24T22:33:40.233137+0000","last_undegraded":"2026-02-24T22:33:40.233137+0000","last_fullsized":"2026-02-24T22:33:40.233137+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:52:57.542872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":536,"num_read_kb":444,"num_write":207,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"312'315","reported_seq":1041,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.986798+0000","last_change":"2026-02-24T22:05:16.185307+0000","last_active":"2026-02-24T22:34:25.986798+0000","last_peered":"2026-02-24T22:34:25.986798+0000","last_clean":"2026-02-24T22:34:25.986798+0000","last_became_active":"2026-02-24T22:05:16.184131+0000","last_became_peered":"2026-02-24T22:05:16.184131+0000","last_unstale":"2026-02-24T22:34:25.986798+0000","last_undegraded":"2026-02-24T22:34:25.986798+0000","last_fullsized":"2026-02-24T22:34:25.986798+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:09:05.496588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":471,"num_read_kb":1193,"num_write":1308,"num_write_kb":244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":612,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.724713+0000","last_change":"2026-02-24T22:05:10.689444+0000","last_active":"2026-02-24T22:31:09.724713+0000","last_peered":"2026-02-24T22:31:09.724713+0000","last_clean":"2026-02-24T22:31:09.724713+0000","last_became_active":"2026-02-24T22:05:08.682913+0000","last_became_peered":"2026-02-24T22:05:08.682913+0000","last_unstale":"2026-02-24T22:31:09.724713+0000","last_undegraded":"2026-02-24T22:31:09.724713+0000","last_fullsized":"2026-02-24T22:31:09.724713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:20:01.744772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000359878,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":592,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.746484+0000","last_change":"2026-02-24T22:05:20.192401+0000","last_active":"2026-02-24T22:32:24.746484+0000","last_peered":"2026-02-24T22:32:24.746484+0000","last_clean":"2026-02-24T22:32:24.746484+0000","last_became_active":"2026-02-24T22:05:20.192162+0000","last_became_peered":"2026-02-24T22:05:20.192162+0000","last_unstale":"2026-02-24T22:32:24.746484+0000","last_undegraded":"2026-02-24T22:32:24.746484+0000","last_fullsized":"2026-02-24T22:32:24.746484+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:51:58.434704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"312'1031","reported_seq":2594,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.241812+0000","last_change":"2026-02-24T22:05:22.198775+0000","last_active":"2026-02-24T22:34:25.241812+0000","last_peered":"2026-02-24T22:34:25.241812+0000","last_clean":"2026-02-24T22:34:25.241812+0000","last_became_active":"2026-02-24T22:05:22.198252+0000","last_became_peered":"2026-02-24T22:05:22.198252+0000","last_unstale":"2026-02-24T22:34:25.241812+0000","last_undegraded":"2026-02-24T22:34:25.241812+0000","last_fullsized":"2026-02-24T22:34:25.241812+0000","mapping_epoch":28,"log_start":"312'1000","ondisk_log_start":"312'1000","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":1000,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:34:58.507593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":614,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":3875,"num_read_kb":1975,"num_write":82,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"312'1698","reported_seq":4314,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.551488+0000","last_change":"2026-02-24T22:05:24.198591+0000","last_active":"2026-02-24T22:34:18.551488+0000","last_peered":"2026-02-24T22:34:18.551488+0000","last_clean":"2026-02-24T22:34:18.551488+0000","last_became_active":"2026-02-24T22:05:24.198348+0000","last_became_peered":"2026-02-24T22:05:24.198348+0000","last_unstale":"2026-02-24T22:34:18.551488+0000","last_undegraded":"2026-02-24T22:34:18.551488+0000","last_fullsized":"2026-02-24T22:34:18.551488+0000","mapping_epoch":30,"log_start":"310'1600","ondisk_log_start":"310'1600","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":1600,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:04:55.163411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31448,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":5151,"num_read_kb":231050,"num_write":3475,"num_write_kb":1655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":574,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:14.732473+0000","last_change":"2026-02-24T22:05:29.652643+0000","last_active":"2026-02-24T22:30:14.732473+0000","last_peered":"2026-02-24T22:30:14.732473+0000","last_clean":"2026-02-24T22:30:14.732473+0000","last_became_active":"2026-02-24T22:05:29.652321+0000","last_became_peered":"2026-02-24T22:05:29.652321+0000","last_unstale":"2026-02-24T22:30:14.732473+0000","last_undegraded":"2026-02-24T22:30:14.732473+0000","last_fullsized":"2026-02-24T22:30:14.732473+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:23:33.530605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"312'55","reported_seq":719,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.746473+0000","last_change":"2026-02-24T22:05:18.191716+0000","last_active":"2026-02-24T22:32:24.746473+0000","last_peered":"2026-02-24T22:32:24.746473+0000","last_clean":"2026-02-24T22:32:24.746473+0000","last_became_active":"2026-02-24T22:05:18.191249+0000","last_became_peered":"2026-02-24T22:05:18.191249+0000","last_unstale":"2026-02-24T22:32:24.746473+0000","last_undegraded":"2026-02-24T22:32:24.746473+0000","last_fullsized":"2026-02-24T22:32:24.746473+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:48:35.012678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":226,"num_read_kb":230,"num_write":70,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"312'270","reported_seq":1002,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.978846+0000","last_change":"2026-02-24T22:05:16.182104+0000","last_active":"2026-02-24T22:34:25.978846+0000","last_peered":"2026-02-24T22:34:25.978846+0000","last_clean":"2026-02-24T22:34:25.978846+0000","last_became_active":"2026-02-24T22:05:16.181840+0000","last_became_peered":"2026-02-24T22:05:16.181840+0000","last_unstale":"2026-02-24T22:34:25.978846+0000","last_undegraded":"2026-02-24T22:34:25.978846+0000","last_fullsized":"2026-02-24T22:34:25.978846+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:50:28.643829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":486,"num_read_kb":7386,"num_write":1250,"num_write_kb":13063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":607,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.736747+0000","last_change":"2026-02-24T22:05:10.808402+0000","last_active":"2026-02-24T22:31:09.736747+0000","last_peered":"2026-02-24T22:31:09.736747+0000","last_clean":"2026-02-24T22:31:09.736747+0000","last_became_active":"2026-02-24T22:05:08.689542+0000","last_became_peered":"2026-02-24T22:05:08.689542+0000","last_unstale":"2026-02-24T22:31:09.736747+0000","last_undegraded":"2026-02-24T22:31:09.736747+0000","last_fullsized":"2026-02-24T22:31:09.736747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:07.670081+0000","last_clean_scrub_stamp":"2026-02-24T22:05:07.670081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:23:22.404541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038816099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":592,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.731136+0000","last_change":"2026-02-24T22:05:20.186366+0000","last_active":"2026-02-24T22:31:24.731136+0000","last_peered":"2026-02-24T22:31:24.731136+0000","last_clean":"2026-02-24T22:31:24.731136+0000","last_became_active":"2026-02-24T22:05:20.186108+0000","last_became_peered":"2026-02-24T22:05:20.186108+0000","last_unstale":"2026-02-24T22:31:24.731136+0000","last_undegraded":"2026-02-24T22:31:24.731136+0000","last_fullsized":"2026-02-24T22:31:24.731136+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:02:47.567361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"312'607","reported_seq":1259,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.737026+0000","last_change":"2026-02-24T22:05:22.911959+0000","last_active":"2026-02-24T22:34:25.737026+0000","last_peered":"2026-02-24T22:34:25.737026+0000","last_clean":"2026-02-24T22:34:25.737026+0000","last_became_active":"2026-02-24T22:05:22.911692+0000","last_became_peered":"2026-02-24T22:05:22.911692+0000","last_unstale":"2026-02-24T22:34:25.737026+0000","last_undegraded":"2026-02-24T22:34:25.737026+0000","last_fullsized":"2026-02-24T22:34:25.737026+0000","mapping_epoch":28,"log_start":"312'500","ondisk_log_start":"312'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":500,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:53:02.224623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115263,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1049,"num_read_kb":2748,"num_write":972,"num_write_kb":882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"312'175","reported_seq":763,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:07.165906+0000","last_change":"2026-02-24T22:05:24.199612+0000","last_active":"2026-02-24T22:34:07.165906+0000","last_peered":"2026-02-24T22:34:07.165906+0000","last_clean":"2026-02-24T22:34:07.165906+0000","last_became_active":"2026-02-24T22:05:24.199398+0000","last_became_peered":"2026-02-24T22:05:24.199398+0000","last_unstale":"2026-02-24T22:34:07.165906+0000","last_undegraded":"2026-02-24T22:34:07.165906+0000","last_fullsized":"2026-02-24T22:34:07.165906+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":100,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:03:11.629672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30139,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":95,"num_read_kb":93,"num_write":436,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"312'435","reported_seq":3593,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.785583+0000","last_change":"2026-02-24T22:05:29.662212+0000","last_active":"2026-02-24T22:34:25.785583+0000","last_peered":"2026-02-24T22:34:25.785583+0000","last_clean":"2026-02-24T22:34:25.785583+0000","last_became_active":"2026-02-24T22:05:29.661694+0000","last_became_peered":"2026-02-24T22:05:29.661694+0000","last_unstale":"2026-02-24T22:34:25.785583+0000","last_undegraded":"2026-02-24T22:34:25.785583+0000","last_fullsized":"2026-02-24T22:34:25.785583+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":400,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:28:47.623604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"312'461","reported_seq":2595,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.965594+0000","last_change":"2026-02-24T22:05:29.654721+0000","last_active":"2026-02-24T22:34:25.965594+0000","last_peered":"2026-02-24T22:34:25.965594+0000","last_clean":"2026-02-24T22:34:25.965594+0000","last_became_active":"2026-02-24T22:05:29.654058+0000","last_became_peered":"2026-02-24T22:05:29.654058+0000","last_unstale":"2026-02-24T22:34:25.965594+0000","last_undegraded":"2026-02-24T22:34:25.965594+0000","last_fullsized":"2026-02-24T22:34:25.965594+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":400,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:19:34.289230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"312'4194","reported_seq":4926,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:45.073603+0000","last_change":"2026-02-24T22:05:18.191351+0000","last_active":"2026-02-24T22:33:45.073603+0000","last_peered":"2026-02-24T22:33:45.073603+0000","last_clean":"2026-02-24T22:33:45.073603+0000","last_became_active":"2026-02-24T22:05:18.190114+0000","last_became_peered":"2026-02-24T22:05:18.190114+0000","last_unstale":"2026-02-24T22:33:45.073603+0000","last_undegraded":"2026-02-24T22:33:45.073603+0000","last_fullsized":"2026-02-24T22:33:45.073603+0000","mapping_epoch":24,"log_start":"312'4100","ondisk_log_start":"312'4100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":3000,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:58:28.766462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":16758,"num_read_kb":13963,"num_write":6320,"num_write_kb":3135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"312'246","reported_seq":967,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.929641+0000","last_change":"2026-02-24T22:05:16.189046+0000","last_active":"2026-02-24T22:34:25.929641+0000","last_peered":"2026-02-24T22:34:25.929641+0000","last_clean":"2026-02-24T22:34:25.929641+0000","last_became_active":"2026-02-24T22:05:16.188630+0000","last_became_peered":"2026-02-24T22:05:16.188630+0000","last_unstale":"2026-02-24T22:34:25.929641+0000","last_undegraded":"2026-02-24T22:34:25.929641+0000","last_fullsized":"2026-02-24T22:34:25.929641+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":200,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:15:02.436674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291503,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":466,"num_read_kb":33751,"num_write":1067,"num_write_kb":13009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":588,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.723682+0000","last_change":"2026-02-24T22:05:20.188706+0000","last_active":"2026-02-24T22:30:59.723682+0000","last_peered":"2026-02-24T22:30:59.723682+0000","last_clean":"2026-02-24T22:30:59.723682+0000","last_became_active":"2026-02-24T22:05:20.188330+0000","last_became_peered":"2026-02-24T22:05:20.188330+0000","last_unstale":"2026-02-24T22:30:59.723682+0000","last_undegraded":"2026-02-24T22:30:59.723682+0000","last_fullsized":"2026-02-24T22:30:59.723682+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:07:09.639326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"312'688","reported_seq":1771,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.888352+0000","last_change":"2026-02-24T22:05:22.192098+0000","last_active":"2026-02-24T22:34:25.888352+0000","last_peered":"2026-02-24T22:34:25.888352+0000","last_clean":"2026-02-24T22:34:25.888352+0000","last_became_active":"2026-02-24T22:05:22.191584+0000","last_became_peered":"2026-02-24T22:05:22.191584+0000","last_unstale":"2026-02-24T22:34:25.888352+0000","last_undegraded":"2026-02-24T22:34:25.888352+0000","last_fullsized":"2026-02-24T22:34:25.888352+0000","mapping_epoch":28,"log_start":"312'600","ondisk_log_start":"312'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":600,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:55:24.433104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39069,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2230,"num_read_kb":1811,"num_write":322,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"312'161","reported_seq":768,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:55.998434+0000","last_change":"2026-02-24T22:05:24.199098+0000","last_active":"2026-02-24T22:33:55.998434+0000","last_peered":"2026-02-24T22:33:55.998434+0000","last_clean":"2026-02-24T22:33:55.998434+0000","last_became_active":"2026-02-24T22:05:24.198821+0000","last_became_peered":"2026-02-24T22:05:24.198821+0000","last_unstale":"2026-02-24T22:33:55.998434+0000","last_undegraded":"2026-02-24T22:33:55.998434+0000","last_fullsized":"2026-02-24T22:33:55.998434+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":100,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:31:53.831339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28978,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":162,"num_read_kb":145,"num_write":411,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":578,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.732230+0000","last_change":"2026-02-24T22:05:29.653213+0000","last_active":"2026-02-24T22:31:34.732230+0000","last_peered":"2026-02-24T22:31:34.732230+0000","last_clean":"2026-02-24T22:31:34.732230+0000","last_became_active":"2026-02-24T22:05:29.652757+0000","last_became_peered":"2026-02-24T22:05:29.652757+0000","last_unstale":"2026-02-24T22:31:34.732230+0000","last_undegraded":"2026-02-24T22:31:34.732230+0000","last_fullsized":"2026-02-24T22:31:34.732230+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:57:03.928835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"312'4231","reported_seq":4978,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:52.784970+0000","last_change":"2026-02-24T22:05:18.184303+0000","last_active":"2026-02-24T22:33:52.784970+0000","last_peered":"2026-02-24T22:33:52.784970+0000","last_clean":"2026-02-24T22:33:52.784970+0000","last_became_active":"2026-02-24T22:05:18.183983+0000","last_became_peered":"2026-02-24T22:05:18.183983+0000","last_unstale":"2026-02-24T22:33:52.784970+0000","last_undegraded":"2026-02-24T22:33:52.784970+0000","last_fullsized":"2026-02-24T22:33:52.784970+0000","mapping_epoch":24,"log_start":"312'4200","ondisk_log_start":"312'4200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":3000,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:36:58.437134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":16912,"num_read_kb":14102,"num_write":6331,"num_write_kb":3151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"312'310","reported_seq":1052,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.963185+0000","last_change":"2026-02-24T22:05:16.193213+0000","last_active":"2026-02-24T22:34:25.963185+0000","last_peered":"2026-02-24T22:34:25.963185+0000","last_clean":"2026-02-24T22:34:25.963185+0000","last_became_active":"2026-02-24T22:05:16.192845+0000","last_became_peered":"2026-02-24T22:05:16.192845+0000","last_unstale":"2026-02-24T22:34:25.963185+0000","last_undegraded":"2026-02-24T22:34:25.963185+0000","last_fullsized":"2026-02-24T22:34:25.963185+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":200,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:53:53.655783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":490,"num_read_kb":4336,"num_write":1277,"num_write_kb":2676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.728520+0000","last_change":"2026-02-24T22:05:20.189035+0000","last_active":"2026-02-24T22:30:59.728520+0000","last_peered":"2026-02-24T22:30:59.728520+0000","last_clean":"2026-02-24T22:30:59.728520+0000","last_became_active":"2026-02-24T22:05:20.188790+0000","last_became_peered":"2026-02-24T22:05:20.188790+0000","last_unstale":"2026-02-24T22:30:59.728520+0000","last_undegraded":"2026-02-24T22:30:59.728520+0000","last_fullsized":"2026-02-24T22:30:59.728520+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:29:04.813791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"312'25","reported_seq":615,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.737881+0000","last_change":"2026-02-24T22:05:22.192740+0000","last_active":"2026-02-24T22:31:04.737881+0000","last_peered":"2026-02-24T22:31:04.737881+0000","last_clean":"2026-02-24T22:31:04.737881+0000","last_became_active":"2026-02-24T22:05:22.192560+0000","last_became_peered":"2026-02-24T22:05:22.192560+0000","last_unstale":"2026-02-24T22:31:04.737881+0000","last_undegraded":"2026-02-24T22:31:04.737881+0000","last_fullsized":"2026-02-24T22:31:04.737881+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:40:42.414311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"312'163","reported_seq":746,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.531878+0000","last_change":"2026-02-24T22:05:24.205761+0000","last_active":"2026-02-24T22:34:18.531878+0000","last_peered":"2026-02-24T22:34:18.531878+0000","last_clean":"2026-02-24T22:34:18.531878+0000","last_became_active":"2026-02-24T22:05:24.205506+0000","last_became_peered":"2026-02-24T22:05:24.205506+0000","last_unstale":"2026-02-24T22:34:18.531878+0000","last_undegraded":"2026-02-24T22:34:18.531878+0000","last_fullsized":"2026-02-24T22:34:18.531878+0000","mapping_epoch":30,"log_start":"301'100","ondisk_log_start":"301'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":100,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:22:32.827602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30843,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":90,"num_read_kb":84,"num_write":389,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":573,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.734200+0000","last_change":"2026-02-24T22:05:29.653324+0000","last_active":"2026-02-24T22:30:29.734200+0000","last_peered":"2026-02-24T22:30:29.734200+0000","last_clean":"2026-02-24T22:30:29.734200+0000","last_became_active":"2026-02-24T22:05:29.652987+0000","last_became_peered":"2026-02-24T22:05:29.652987+0000","last_unstale":"2026-02-24T22:30:29.734200+0000","last_undegraded":"2026-02-24T22:30:29.734200+0000","last_fullsized":"2026-02-24T22:30:29.734200+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:58:50.918939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"312'187","reported_seq":884,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.550400+0000","last_change":"2026-02-24T22:05:18.186488+0000","last_active":"2026-02-24T22:34:18.550400+0000","last_peered":"2026-02-24T22:34:18.550400+0000","last_clean":"2026-02-24T22:34:18.550400+0000","last_became_active":"2026-02-24T22:05:18.186069+0000","last_became_peered":"2026-02-24T22:05:18.186069+0000","last_unstale":"2026-02-24T22:34:18.550400+0000","last_undegraded":"2026-02-24T22:34:18.550400+0000","last_fullsized":"2026-02-24T22:34:18.550400+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:26:51.294901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":691,"num_read_kb":561,"num_write":267,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"312'309","reported_seq":991,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.975893+0000","last_change":"2026-02-24T22:06:00.662484+0000","last_active":"2026-02-24T22:34:25.975893+0000","last_peered":"2026-02-24T22:34:25.975893+0000","last_clean":"2026-02-24T22:34:25.975893+0000","last_became_active":"2026-02-24T22:06:00.662291+0000","last_became_peered":"2026-02-24T22:06:00.662291+0000","last_unstale":"2026-02-24T22:34:25.975893+0000","last_undegraded":"2026-02-24T22:34:25.975893+0000","last_fullsized":"2026-02-24T22:34:25.975893+0000","mapping_epoch":36,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:44:04.240829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":421,"num_read_kb":357,"num_write":1374,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":585,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.739650+0000","last_change":"2026-02-24T22:05:20.194668+0000","last_active":"2026-02-24T22:30:59.739650+0000","last_peered":"2026-02-24T22:30:59.739650+0000","last_clean":"2026-02-24T22:30:59.739650+0000","last_became_active":"2026-02-24T22:05:20.194215+0000","last_became_peered":"2026-02-24T22:05:20.194215+0000","last_unstale":"2026-02-24T22:30:59.739650+0000","last_undegraded":"2026-02-24T22:30:59.739650+0000","last_fullsized":"2026-02-24T22:30:59.739650+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:55:09.941150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"312'476","reported_seq":1339,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.640627+0000","last_change":"2026-02-24T22:05:22.192102+0000","last_active":"2026-02-24T22:34:25.640627+0000","last_peered":"2026-02-24T22:34:25.640627+0000","last_clean":"2026-02-24T22:34:25.640627+0000","last_became_active":"2026-02-24T22:05:22.191675+0000","last_became_peered":"2026-02-24T22:05:22.191675+0000","last_unstale":"2026-02-24T22:34:25.640627+0000","last_undegraded":"2026-02-24T22:34:25.640627+0000","last_fullsized":"2026-02-24T22:34:25.640627+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":400,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:50:42.591507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39139,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1329,"num_read_kb":1395,"num_write":345,"num_write_kb":280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"312'249","reported_seq":1278,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.564141+0000","last_change":"2026-02-24T22:05:24.207967+0000","last_active":"2026-02-24T22:34:17.564141+0000","last_peered":"2026-02-24T22:34:17.564141+0000","last_clean":"2026-02-24T22:34:17.564141+0000","last_became_active":"2026-02-24T22:05:24.207479+0000","last_became_peered":"2026-02-24T22:05:24.207479+0000","last_unstale":"2026-02-24T22:34:17.564141+0000","last_undegraded":"2026-02-24T22:34:17.564141+0000","last_fullsized":"2026-02-24T22:34:17.564141+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":200,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:21:33.217485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31962,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":675,"num_read_kb":669,"num_write":683,"num_write_kb":207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":564,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.735513+0000","last_change":"2026-02-24T22:05:29.661767+0000","last_active":"2026-02-24T22:30:29.735513+0000","last_peered":"2026-02-24T22:30:29.735513+0000","last_clean":"2026-02-24T22:30:29.735513+0000","last_became_active":"2026-02-24T22:05:29.661248+0000","last_became_peered":"2026-02-24T22:05:29.661248+0000","last_unstale":"2026-02-24T22:30:29.735513+0000","last_undegraded":"2026-02-24T22:30:29.735513+0000","last_fullsized":"2026-02-24T22:30:29.735513+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:30:06.329202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"312'116","reported_seq":824,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:06.481608+0000","last_change":"2026-02-24T22:05:18.187722+0000","last_active":"2026-02-24T22:34:06.481608+0000","last_peered":"2026-02-24T22:34:06.481608+0000","last_clean":"2026-02-24T22:34:06.481608+0000","last_became_active":"2026-02-24T22:05:18.184982+0000","last_became_peered":"2026-02-24T22:05:18.184982+0000","last_unstale":"2026-02-24T22:34:06.481608+0000","last_undegraded":"2026-02-24T22:34:06.481608+0000","last_fullsized":"2026-02-24T22:34:06.481608+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":100,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:08:28.087403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":444,"num_read_kb":381,"num_write":158,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"312'279","reported_seq":1041,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.975958+0000","last_change":"2026-02-24T22:05:16.184926+0000","last_active":"2026-02-24T22:34:25.975958+0000","last_peered":"2026-02-24T22:34:25.975958+0000","last_clean":"2026-02-24T22:34:25.975958+0000","last_became_active":"2026-02-24T22:05:16.184594+0000","last_became_peered":"2026-02-24T22:05:16.184594+0000","last_unstale":"2026-02-24T22:34:25.975958+0000","last_undegraded":"2026-02-24T22:34:25.975958+0000","last_fullsized":"2026-02-24T22:34:25.975958+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":200,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:53:18.748033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048643,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":519,"num_read_kb":28116,"num_write":1194,"num_write_kb":6289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":583,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.735449+0000","last_change":"2026-02-24T22:05:20.193777+0000","last_active":"2026-02-24T22:30:59.735449+0000","last_peered":"2026-02-24T22:30:59.735449+0000","last_clean":"2026-02-24T22:30:59.735449+0000","last_became_active":"2026-02-24T22:05:20.193522+0000","last_became_peered":"2026-02-24T22:05:20.193522+0000","last_unstale":"2026-02-24T22:30:59.735449+0000","last_undegraded":"2026-02-24T22:30:59.735449+0000","last_fullsized":"2026-02-24T22:30:59.735449+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:18:03.275177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"312'434","reported_seq":1273,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.552326+0000","last_change":"2026-02-24T22:05:22.200284+0000","last_active":"2026-02-24T22:34:25.552326+0000","last_peered":"2026-02-24T22:34:25.552326+0000","last_clean":"2026-02-24T22:34:25.552326+0000","last_became_active":"2026-02-24T22:05:22.199968+0000","last_became_peered":"2026-02-24T22:05:22.199968+0000","last_unstale":"2026-02-24T22:34:25.552326+0000","last_undegraded":"2026-02-24T22:34:25.552326+0000","last_fullsized":"2026-02-24T22:34:25.552326+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":400,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:25:54.291884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38886,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1275,"num_read_kb":1335,"num_write":311,"num_write_kb":280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"312'212","reported_seq":808,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:04.178415+0000","last_change":"2026-02-24T22:05:24.200305+0000","last_active":"2026-02-24T22:34:04.178415+0000","last_peered":"2026-02-24T22:34:04.178415+0000","last_clean":"2026-02-24T22:34:04.178415+0000","last_became_active":"2026-02-24T22:05:24.199650+0000","last_became_peered":"2026-02-24T22:05:24.199650+0000","last_unstale":"2026-02-24T22:34:04.178415+0000","last_undegraded":"2026-02-24T22:34:04.178415+0000","last_fullsized":"2026-02-24T22:34:04.178415+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:33:18.336086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29523,"num_objects":91,"num_object_clones":0,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":182,"num_read_kb":193,"num_write":488,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"312'249","reported_seq":959,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:44.109184+0000","last_change":"2026-02-24T22:05:18.190553+0000","last_active":"2026-02-24T22:33:44.109184+0000","last_peered":"2026-02-24T22:33:44.109184+0000","last_clean":"2026-02-24T22:33:44.109184+0000","last_became_active":"2026-02-24T22:05:18.190238+0000","last_became_peered":"2026-02-24T22:05:18.190238+0000","last_unstale":"2026-02-24T22:33:44.109184+0000","last_undegraded":"2026-02-24T22:33:44.109184+0000","last_fullsized":"2026-02-24T22:33:44.109184+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":200,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:04:44.616609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":976,"num_read_kb":794,"num_write":353,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"312'307","reported_seq":1663,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.957336+0000","last_change":"2026-02-24T22:05:16.185696+0000","last_active":"2026-02-24T22:34:25.957336+0000","last_peered":"2026-02-24T22:34:25.957336+0000","last_clean":"2026-02-24T22:34:25.957336+0000","last_became_active":"2026-02-24T22:05:16.184415+0000","last_became_peered":"2026-02-24T22:05:16.184415+0000","last_unstale":"2026-02-24T22:34:25.957336+0000","last_undegraded":"2026-02-24T22:34:25.957336+0000","last_fullsized":"2026-02-24T22:34:25.957336+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:10:18.950972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2362,"num_read_kb":23834,"num_write":1125,"num_write_kb":5557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.740195+0000","last_change":"2026-02-24T22:05:20.186817+0000","last_active":"2026-02-24T22:31:24.740195+0000","last_peered":"2026-02-24T22:31:24.740195+0000","last_clean":"2026-02-24T22:31:24.740195+0000","last_became_active":"2026-02-24T22:05:20.186325+0000","last_became_peered":"2026-02-24T22:05:20.186325+0000","last_unstale":"2026-02-24T22:31:24.740195+0000","last_undegraded":"2026-02-24T22:31:24.740195+0000","last_fullsized":"2026-02-24T22:31:24.740195+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:45:07.507222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"312'409","reported_seq":1027,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.546379+0000","last_change":"2026-02-24T22:05:22.192972+0000","last_active":"2026-02-24T22:34:25.546379+0000","last_peered":"2026-02-24T22:34:25.546379+0000","last_clean":"2026-02-24T22:34:25.546379+0000","last_became_active":"2026-02-24T22:05:22.192756+0000","last_became_peered":"2026-02-24T22:05:22.192756+0000","last_unstale":"2026-02-24T22:34:25.546379+0000","last_undegraded":"2026-02-24T22:34:25.546379+0000","last_fullsized":"2026-02-24T22:34:25.546379+0000","mapping_epoch":28,"log_start":"312'300","ondisk_log_start":"312'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":300,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:43:14.513261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77383,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":715,"num_read_kb":1913,"num_write":704,"num_write_kb":692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"312'187","reported_seq":796,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.233131+0000","last_change":"2026-02-24T22:05:24.206653+0000","last_active":"2026-02-24T22:34:17.233131+0000","last_peered":"2026-02-24T22:34:17.233131+0000","last_clean":"2026-02-24T22:34:17.233131+0000","last_became_active":"2026-02-24T22:05:24.206407+0000","last_became_peered":"2026-02-24T22:05:24.206407+0000","last_unstale":"2026-02-24T22:34:17.233131+0000","last_undegraded":"2026-02-24T22:34:17.233131+0000","last_fullsized":"2026-02-24T22:34:17.233131+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:49:11.388315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30298,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":168,"num_read_kb":155,"num_write":455,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":576,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.729573+0000","last_change":"2026-02-24T22:05:29.650326+0000","last_active":"2026-02-24T22:31:09.729573+0000","last_peered":"2026-02-24T22:31:09.729573+0000","last_clean":"2026-02-24T22:31:09.729573+0000","last_became_active":"2026-02-24T22:05:29.650115+0000","last_became_peered":"2026-02-24T22:05:29.650115+0000","last_unstale":"2026-02-24T22:31:09.729573+0000","last_undegraded":"2026-02-24T22:31:09.729573+0000","last_fullsized":"2026-02-24T22:31:09.729573+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:38:20.581425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"312'162","reported_seq":873,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.740186+0000","last_change":"2026-02-24T22:05:18.183744+0000","last_active":"2026-02-24T22:31:24.740186+0000","last_peered":"2026-02-24T22:31:24.740186+0000","last_clean":"2026-02-24T22:31:24.740186+0000","last_became_active":"2026-02-24T22:05:18.183501+0000","last_became_peered":"2026-02-24T22:05:18.183501+0000","last_unstale":"2026-02-24T22:31:24.740186+0000","last_undegraded":"2026-02-24T22:31:24.740186+0000","last_fullsized":"2026-02-24T22:31:24.740186+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":100,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:10:06.918155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":578,"num_read_kb":488,"num_write":224,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"312'407","reported_seq":1349,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.983429+0000","last_change":"2026-02-24T22:05:16.183028+0000","last_active":"2026-02-24T22:34:25.983429+0000","last_peered":"2026-02-24T22:34:25.983429+0000","last_clean":"2026-02-24T22:34:25.983429+0000","last_became_active":"2026-02-24T22:05:16.182909+0000","last_became_peered":"2026-02-24T22:05:16.182909+0000","last_unstale":"2026-02-24T22:34:25.983429+0000","last_undegraded":"2026-02-24T22:34:25.983429+0000","last_fullsized":"2026-02-24T22:34:25.983429+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":300,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:01:15.081477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1207,"num_read_kb":9244,"num_write":1611,"num_write_kb":11784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":585,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.744843+0000","last_change":"2026-02-24T22:05:20.194023+0000","last_active":"2026-02-24T22:32:24.744843+0000","last_peered":"2026-02-24T22:32:24.744843+0000","last_clean":"2026-02-24T22:32:24.744843+0000","last_became_active":"2026-02-24T22:05:20.193751+0000","last_became_peered":"2026-02-24T22:05:20.193751+0000","last_unstale":"2026-02-24T22:32:24.744843+0000","last_undegraded":"2026-02-24T22:32:24.744843+0000","last_fullsized":"2026-02-24T22:32:24.744843+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:48:18.314998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"312'744","reported_seq":1827,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.582463+0000","last_change":"2026-02-24T22:05:22.200836+0000","last_active":"2026-02-24T22:34:25.582463+0000","last_peered":"2026-02-24T22:34:25.582463+0000","last_clean":"2026-02-24T22:34:25.582463+0000","last_became_active":"2026-02-24T22:05:22.200603+0000","last_became_peered":"2026-02-24T22:05:22.200603+0000","last_unstale":"2026-02-24T22:34:25.582463+0000","last_undegraded":"2026-02-24T22:34:25.582463+0000","last_fullsized":"2026-02-24T22:34:25.582463+0000","mapping_epoch":28,"log_start":"312'700","ondisk_log_start":"312'700","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":700,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:36:03.813153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39896,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":2333,"num_read_kb":2057,"num_write":434,"num_write_kb":373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"312'334","reported_seq":964,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.243815+0000","last_change":"2026-02-24T22:05:24.201087+0000","last_active":"2026-02-24T22:34:18.243815+0000","last_peered":"2026-02-24T22:34:18.243815+0000","last_clean":"2026-02-24T22:34:18.243815+0000","last_became_active":"2026-02-24T22:05:24.200391+0000","last_became_peered":"2026-02-24T22:05:24.200391+0000","last_unstale":"2026-02-24T22:34:18.243815+0000","last_undegraded":"2026-02-24T22:34:18.243815+0000","last_fullsized":"2026-02-24T22:34:18.243815+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:18:42.256342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31847,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":370,"num_read_kb":353,"num_write":777,"num_write_kb":271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"312'81","reported_seq":994,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:14.669283+0000","last_change":"2026-02-24T22:05:29.660890+0000","last_active":"2026-02-24T22:34:14.669283+0000","last_peered":"2026-02-24T22:34:14.669283+0000","last_clean":"2026-02-24T22:34:14.669283+0000","last_became_active":"2026-02-24T22:05:29.660289+0000","last_became_peered":"2026-02-24T22:05:29.660289+0000","last_unstale":"2026-02-24T22:34:14.669283+0000","last_undegraded":"2026-02-24T22:34:14.669283+0000","last_fullsized":"2026-02-24T22:34:14.669283+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:13:08.310592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"312'100","reported_seq":769,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:54.429078+0000","last_change":"2026-02-24T22:05:18.185312+0000","last_active":"2026-02-24T22:33:54.429078+0000","last_peered":"2026-02-24T22:33:54.429078+0000","last_clean":"2026-02-24T22:33:54.429078+0000","last_became_active":"2026-02-24T22:05:18.184745+0000","last_became_peered":"2026-02-24T22:05:18.184745+0000","last_unstale":"2026-02-24T22:33:54.429078+0000","last_undegraded":"2026-02-24T22:33:54.429078+0000","last_fullsized":"2026-02-24T22:33:54.429078+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:08:34.210048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":371,"num_read_kb":307,"num_write":132,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"312'303","reported_seq":1023,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.927863+0000","last_change":"2026-02-24T22:05:16.185859+0000","last_active":"2026-02-24T22:34:25.927863+0000","last_peered":"2026-02-24T22:34:25.927863+0000","last_clean":"2026-02-24T22:34:25.927863+0000","last_became_active":"2026-02-24T22:05:16.185574+0000","last_became_peered":"2026-02-24T22:05:16.185574+0000","last_unstale":"2026-02-24T22:34:25.927863+0000","last_undegraded":"2026-02-24T22:34:25.927863+0000","last_fullsized":"2026-02-24T22:34:25.927863+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":200,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:09:04.869683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":428,"num_read_kb":30007,"num_write":1257,"num_write_kb":21358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"32'3","reported_seq":831,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.489031+0000","last_change":"2026-02-24T22:05:20.192991+0000","last_active":"2026-02-24T22:34:18.489031+0000","last_peered":"2026-02-24T22:34:18.489031+0000","last_clean":"2026-02-24T22:34:18.489031+0000","last_became_active":"2026-02-24T22:05:20.192612+0000","last_became_peered":"2026-02-24T22:05:20.192612+0000","last_unstale":"2026-02-24T22:34:18.489031+0000","last_undegraded":"2026-02-24T22:34:18.489031+0000","last_fullsized":"2026-02-24T22:34:18.489031+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:32:11.097665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":484,"num_read_kb":482,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"312'204","reported_seq":816,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.994504+0000","last_change":"2026-02-24T22:05:22.906226+0000","last_active":"2026-02-24T22:34:25.994504+0000","last_peered":"2026-02-24T22:34:25.994504+0000","last_clean":"2026-02-24T22:34:25.994504+0000","last_became_active":"2026-02-24T22:05:22.906047+0000","last_became_peered":"2026-02-24T22:05:22.906047+0000","last_unstale":"2026-02-24T22:34:25.994504+0000","last_undegraded":"2026-02-24T22:34:25.994504+0000","last_fullsized":"2026-02-24T22:34:25.994504+0000","mapping_epoch":28,"log_start":"312'100","ondisk_log_start":"312'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:36:51.410405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38786,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":328,"num_read_kb":839,"num_write":300,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"312'164","reported_seq":759,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:58.164024+0000","last_change":"2026-02-24T22:05:24.201114+0000","last_active":"2026-02-24T22:33:58.164024+0000","last_peered":"2026-02-24T22:33:58.164024+0000","last_clean":"2026-02-24T22:33:58.164024+0000","last_became_active":"2026-02-24T22:05:24.200747+0000","last_became_peered":"2026-02-24T22:05:24.200747+0000","last_unstale":"2026-02-24T22:33:58.164024+0000","last_undegraded":"2026-02-24T22:33:58.164024+0000","last_fullsized":"2026-02-24T22:33:58.164024+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:27:56.881747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30873,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":120,"num_read_kb":110,"num_write":405,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":565,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.740173+0000","last_change":"2026-02-24T22:05:29.660794+0000","last_active":"2026-02-24T22:31:09.740173+0000","last_peered":"2026-02-24T22:31:09.740173+0000","last_clean":"2026-02-24T22:31:09.740173+0000","last_became_active":"2026-02-24T22:05:29.660344+0000","last_became_peered":"2026-02-24T22:05:29.660344+0000","last_unstale":"2026-02-24T22:31:09.740173+0000","last_undegraded":"2026-02-24T22:31:09.740173+0000","last_fullsized":"2026-02-24T22:31:09.740173+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:36:46.842669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"312'157","reported_seq":828,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.716442+0000","last_change":"2026-02-24T22:06:00.759389+0000","last_active":"2026-02-24T22:30:19.716442+0000","last_peered":"2026-02-24T22:30:19.716442+0000","last_clean":"2026-02-24T22:30:19.716442+0000","last_became_active":"2026-02-24T22:06:00.759215+0000","last_became_peered":"2026-02-24T22:06:00.759215+0000","last_unstale":"2026-02-24T22:30:19.716442+0000","last_undegraded":"2026-02-24T22:30:19.716442+0000","last_fullsized":"2026-02-24T22:30:19.716442+0000","mapping_epoch":36,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:14:31.570934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":584,"num_read_kb":483,"num_write":222,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"312'263","reported_seq":1014,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.955228+0000","last_change":"2026-02-24T22:05:16.184388+0000","last_active":"2026-02-24T22:34:25.955228+0000","last_peered":"2026-02-24T22:34:25.955228+0000","last_clean":"2026-02-24T22:34:25.955228+0000","last_became_active":"2026-02-24T22:05:16.183956+0000","last_became_peered":"2026-02-24T22:05:16.183956+0000","last_unstale":"2026-02-24T22:34:25.955228+0000","last_undegraded":"2026-02-24T22:34:25.955228+0000","last_fullsized":"2026-02-24T22:34:25.955228+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":200,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:01:46.056052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242893,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":575,"num_read_kb":6430,"num_write":1267,"num_write_kb":5371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":587,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.719728+0000","last_change":"2026-02-24T22:05:20.187960+0000","last_active":"2026-02-24T22:30:19.719728+0000","last_peered":"2026-02-24T22:30:19.719728+0000","last_clean":"2026-02-24T22:30:19.719728+0000","last_became_active":"2026-02-24T22:05:20.187316+0000","last_became_peered":"2026-02-24T22:05:20.187316+0000","last_unstale":"2026-02-24T22:30:19.719728+0000","last_undegraded":"2026-02-24T22:30:19.719728+0000","last_fullsized":"2026-02-24T22:30:19.719728+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:07:22.260792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"312'288","reported_seq":1127,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.218561+0000","last_change":"2026-02-24T22:05:22.912321+0000","last_active":"2026-02-24T22:34:25.218561+0000","last_peered":"2026-02-24T22:34:25.218561+0000","last_clean":"2026-02-24T22:34:25.218561+0000","last_became_active":"2026-02-24T22:05:22.912087+0000","last_became_peered":"2026-02-24T22:05:22.912087+0000","last_unstale":"2026-02-24T22:34:25.218561+0000","last_undegraded":"2026-02-24T22:34:25.218561+0000","last_fullsized":"2026-02-24T22:34:25.218561+0000","mapping_epoch":28,"log_start":"307'200","ondisk_log_start":"307'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":200,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:16:48.283329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":152,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1000,"num_read_kb":520,"num_write":51,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"312'178","reported_seq":844,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.269129+0000","last_change":"2026-02-24T22:05:24.207156+0000","last_active":"2026-02-24T22:34:15.269129+0000","last_peered":"2026-02-24T22:34:15.269129+0000","last_clean":"2026-02-24T22:34:15.269129+0000","last_became_active":"2026-02-24T22:05:24.206383+0000","last_became_peered":"2026-02-24T22:05:24.206383+0000","last_unstale":"2026-02-24T22:34:15.269129+0000","last_undegraded":"2026-02-24T22:34:15.269129+0000","last_fullsized":"2026-02-24T22:34:15.269129+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":100,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:12:00.918186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29475,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":255,"num_read_kb":237,"num_write":434,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":571,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.740948+0000","last_change":"2026-02-24T22:05:29.661761+0000","last_active":"2026-02-24T22:31:09.740948+0000","last_peered":"2026-02-24T22:31:09.740948+0000","last_clean":"2026-02-24T22:31:09.740948+0000","last_became_active":"2026-02-24T22:05:29.661539+0000","last_became_peered":"2026-02-24T22:05:29.661539+0000","last_unstale":"2026-02-24T22:31:09.740948+0000","last_undegraded":"2026-02-24T22:31:09.740948+0000","last_fullsized":"2026-02-24T22:31:09.740948+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:02:56.515020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"312'178","reported_seq":849,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.732694+0000","last_change":"2026-02-24T22:05:18.193080+0000","last_active":"2026-02-24T22:30:19.732694+0000","last_peered":"2026-02-24T22:30:19.732694+0000","last_clean":"2026-02-24T22:30:19.732694+0000","last_became_active":"2026-02-24T22:05:18.192487+0000","last_became_peered":"2026-02-24T22:05:18.192487+0000","last_unstale":"2026-02-24T22:30:19.732694+0000","last_undegraded":"2026-02-24T22:30:19.732694+0000","last_fullsized":"2026-02-24T22:30:19.732694+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":100,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:11:58.202360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":681,"num_read_kb":551,"num_write":266,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"312'296","reported_seq":995,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.971194+0000","last_change":"2026-02-24T22:05:16.190846+0000","last_active":"2026-02-24T22:34:25.971194+0000","last_peered":"2026-02-24T22:34:25.971194+0000","last_clean":"2026-02-24T22:34:25.971194+0000","last_became_active":"2026-02-24T22:05:16.190108+0000","last_became_peered":"2026-02-24T22:05:16.190108+0000","last_unstale":"2026-02-24T22:34:25.971194+0000","last_undegraded":"2026-02-24T22:34:25.971194+0000","last_fullsized":"2026-02-24T22:34:25.971194+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":200,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:05:24.226362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242914,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":451,"num_read_kb":15743,"num_write":1326,"num_write_kb":10328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":584,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.735341+0000","last_change":"2026-02-24T22:05:20.194434+0000","last_active":"2026-02-24T22:30:19.735341+0000","last_peered":"2026-02-24T22:30:19.735341+0000","last_clean":"2026-02-24T22:30:19.735341+0000","last_became_active":"2026-02-24T22:05:20.194183+0000","last_became_peered":"2026-02-24T22:05:20.194183+0000","last_unstale":"2026-02-24T22:30:19.735341+0000","last_undegraded":"2026-02-24T22:30:19.735341+0000","last_fullsized":"2026-02-24T22:30:19.735341+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:11:03.103844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"312'713","reported_seq":1791,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.384999+0000","last_change":"2026-02-24T22:05:22.199767+0000","last_active":"2026-02-24T22:34:25.384999+0000","last_peered":"2026-02-24T22:34:25.384999+0000","last_clean":"2026-02-24T22:34:25.384999+0000","last_became_active":"2026-02-24T22:05:22.199480+0000","last_became_peered":"2026-02-24T22:05:22.199480+0000","last_unstale":"2026-02-24T22:34:25.384999+0000","last_undegraded":"2026-02-24T22:34:25.384999+0000","last_fullsized":"2026-02-24T22:34:25.384999+0000","mapping_epoch":28,"log_start":"312'700","ondisk_log_start":"312'700","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":700,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:44:12.736876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39366,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":2266,"num_read_kb":1888,"num_write":352,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"312'198","reported_seq":804,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.239318+0000","last_change":"2026-02-24T22:05:24.201028+0000","last_active":"2026-02-24T22:34:15.239318+0000","last_peered":"2026-02-24T22:34:15.239318+0000","last_clean":"2026-02-24T22:34:15.239318+0000","last_became_active":"2026-02-24T22:05:24.200434+0000","last_became_peered":"2026-02-24T22:05:24.200434+0000","last_unstale":"2026-02-24T22:34:15.239318+0000","last_undegraded":"2026-02-24T22:34:15.239318+0000","last_fullsized":"2026-02-24T22:34:15.239318+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":100,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:38:49.379078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27134,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":189,"num_read_kb":178,"num_write":476,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":571,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:04.747185+0000","last_change":"2026-02-24T22:05:29.662687+0000","last_active":"2026-02-24T22:32:04.747185+0000","last_peered":"2026-02-24T22:32:04.747185+0000","last_clean":"2026-02-24T22:32:04.747185+0000","last_became_active":"2026-02-24T22:05:29.662425+0000","last_became_peered":"2026-02-24T22:05:29.662425+0000","last_unstale":"2026-02-24T22:32:04.747185+0000","last_undegraded":"2026-02-24T22:32:04.747185+0000","last_fullsized":"2026-02-24T22:32:04.747185+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:37:43.066587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"312'132","reported_seq":810,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.732676+0000","last_change":"2026-02-24T22:05:18.193443+0000","last_active":"2026-02-24T22:30:19.732676+0000","last_peered":"2026-02-24T22:30:19.732676+0000","last_clean":"2026-02-24T22:30:19.732676+0000","last_became_active":"2026-02-24T22:05:18.191664+0000","last_became_peered":"2026-02-24T22:05:18.191664+0000","last_unstale":"2026-02-24T22:30:19.732676+0000","last_undegraded":"2026-02-24T22:30:19.732676+0000","last_fullsized":"2026-02-24T22:30:19.732676+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":100,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:59:22.441702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":463,"num_read_kb":379,"num_write":198,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"312'272","reported_seq":967,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.967124+0000","last_change":"2026-02-24T22:05:16.189190+0000","last_active":"2026-02-24T22:34:25.967124+0000","last_peered":"2026-02-24T22:34:25.967124+0000","last_clean":"2026-02-24T22:34:25.967124+0000","last_became_active":"2026-02-24T22:05:16.188154+0000","last_became_peered":"2026-02-24T22:05:16.188154+0000","last_unstale":"2026-02-24T22:34:25.967124+0000","last_undegraded":"2026-02-24T22:34:25.967124+0000","last_fullsized":"2026-02-24T22:34:25.967124+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":200,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:45:59.561385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":434,"num_read_kb":2459,"num_write":1229,"num_write_kb":1814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"32'5","reported_seq":821,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.487788+0000","last_change":"2026-02-24T22:05:20.195455+0000","last_active":"2026-02-24T22:34:18.487788+0000","last_peered":"2026-02-24T22:34:18.487788+0000","last_clean":"2026-02-24T22:34:18.487788+0000","last_became_active":"2026-02-24T22:05:20.195202+0000","last_became_peered":"2026-02-24T22:05:20.195202+0000","last_unstale":"2026-02-24T22:34:18.487788+0000","last_undegraded":"2026-02-24T22:34:18.487788+0000","last_fullsized":"2026-02-24T22:34:18.487788+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:34:18.002332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1242,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":488,"num_read_kb":722,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"312'687","reported_seq":1765,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.418928+0000","last_change":"2026-02-24T22:05:22.192219+0000","last_active":"2026-02-24T22:34:25.418928+0000","last_peered":"2026-02-24T22:34:25.418928+0000","last_clean":"2026-02-24T22:34:25.418928+0000","last_became_active":"2026-02-24T22:05:22.192035+0000","last_became_peered":"2026-02-24T22:05:22.192035+0000","last_unstale":"2026-02-24T22:34:25.418928+0000","last_undegraded":"2026-02-24T22:34:25.418928+0000","last_fullsized":"2026-02-24T22:34:25.418928+0000","mapping_epoch":28,"log_start":"312'600","ondisk_log_start":"312'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":600,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:05:53.268094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38403,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2221,"num_read_kb":1805,"num_write":316,"num_write_kb":280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"312'2477","reported_seq":5137,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.219286+0000","last_change":"2026-02-24T22:05:24.206516+0000","last_active":"2026-02-24T22:34:18.219286+0000","last_peered":"2026-02-24T22:34:18.219286+0000","last_clean":"2026-02-24T22:34:18.219286+0000","last_became_active":"2026-02-24T22:05:24.205557+0000","last_became_peered":"2026-02-24T22:05:24.205557+0000","last_unstale":"2026-02-24T22:34:18.219286+0000","last_undegraded":"2026-02-24T22:34:18.219286+0000","last_fullsized":"2026-02-24T22:34:18.219286+0000","mapping_epoch":30,"log_start":"312'2400","ondisk_log_start":"312'2400","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":2400,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:33:57.249269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34698,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":6755,"num_read_kb":6753,"num_write":5061,"num_write_kb":2022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":576,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:04.735258+0000","last_change":"2026-02-24T22:05:29.651962+0000","last_active":"2026-02-24T22:32:04.735258+0000","last_peered":"2026-02-24T22:32:04.735258+0000","last_clean":"2026-02-24T22:32:04.735258+0000","last_became_active":"2026-02-24T22:05:29.651760+0000","last_became_peered":"2026-02-24T22:05:29.651760+0000","last_unstale":"2026-02-24T22:32:04.735258+0000","last_undegraded":"2026-02-24T22:32:04.735258+0000","last_fullsized":"2026-02-24T22:32:04.735258+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:48:41.143100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"312'128","reported_seq":805,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:59.734737+0000","last_change":"2026-02-24T22:05:18.185689+0000","last_active":"2026-02-24T22:31:59.734737+0000","last_peered":"2026-02-24T22:31:59.734737+0000","last_clean":"2026-02-24T22:31:59.734737+0000","last_became_active":"2026-02-24T22:05:18.184939+0000","last_became_peered":"2026-02-24T22:05:18.184939+0000","last_unstale":"2026-02-24T22:31:59.734737+0000","last_undegraded":"2026-02-24T22:31:59.734737+0000","last_fullsized":"2026-02-24T22:31:59.734737+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:18:46.745974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":436,"num_read_kb":360,"num_write":185,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"312'287","reported_seq":1011,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.968570+0000","last_change":"2026-02-24T22:05:16.186833+0000","last_active":"2026-02-24T22:34:25.968570+0000","last_peered":"2026-02-24T22:34:25.968570+0000","last_clean":"2026-02-24T22:34:25.968570+0000","last_became_active":"2026-02-24T22:05:16.186043+0000","last_became_peered":"2026-02-24T22:05:16.186043+0000","last_unstale":"2026-02-24T22:34:25.968570+0000","last_undegraded":"2026-02-24T22:34:25.968570+0000","last_fullsized":"2026-02-24T22:34:25.968570+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":200,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:59:46.638347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":465,"num_read_kb":15751,"num_write":1254,"num_write_kb":5255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.742847+0000","last_change":"2026-02-24T22:05:20.193254+0000","last_active":"2026-02-24T22:31:54.742847+0000","last_peered":"2026-02-24T22:31:54.742847+0000","last_clean":"2026-02-24T22:31:54.742847+0000","last_became_active":"2026-02-24T22:05:20.192962+0000","last_became_peered":"2026-02-24T22:05:20.192962+0000","last_unstale":"2026-02-24T22:31:54.742847+0000","last_undegraded":"2026-02-24T22:31:54.742847+0000","last_fullsized":"2026-02-24T22:31:54.742847+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:15:25.988404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"312'475","reported_seq":1312,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.947633+0000","last_change":"2026-02-24T22:05:22.194159+0000","last_active":"2026-02-24T22:34:25.947633+0000","last_peered":"2026-02-24T22:34:25.947633+0000","last_clean":"2026-02-24T22:34:25.947633+0000","last_became_active":"2026-02-24T22:05:22.193896+0000","last_became_peered":"2026-02-24T22:05:22.193896+0000","last_unstale":"2026-02-24T22:34:25.947633+0000","last_undegraded":"2026-02-24T22:34:25.947633+0000","last_fullsized":"2026-02-24T22:34:25.947633+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":400,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:30:50.341834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41178,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1310,"num_read_kb":1453,"num_write":355,"num_write_kb":318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"312'188","reported_seq":800,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.715171+0000","last_change":"2026-02-24T22:05:24.206176+0000","last_active":"2026-02-24T22:34:17.715171+0000","last_peered":"2026-02-24T22:34:17.715171+0000","last_clean":"2026-02-24T22:34:17.715171+0000","last_became_active":"2026-02-24T22:05:24.205855+0000","last_became_peered":"2026-02-24T22:05:24.205855+0000","last_unstale":"2026-02-24T22:34:17.715171+0000","last_undegraded":"2026-02-24T22:34:17.715171+0000","last_fullsized":"2026-02-24T22:34:17.715171+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":100,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:18:56.993738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27729,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":196,"num_read_kb":179,"num_write":471,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"312'448","reported_seq":1620,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.983342+0000","last_change":"2026-02-24T22:05:29.662310+0000","last_active":"2026-02-24T22:34:25.983342+0000","last_peered":"2026-02-24T22:34:25.983342+0000","last_clean":"2026-02-24T22:34:25.983342+0000","last_became_active":"2026-02-24T22:05:29.661983+0000","last_became_peered":"2026-02-24T22:05:29.661983+0000","last_unstale":"2026-02-24T22:34:25.983342+0000","last_undegraded":"2026-02-24T22:34:25.983342+0000","last_fullsized":"2026-02-24T22:34:25.983342+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":400,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:00:08.834953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"312'169","reported_seq":852,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.250160+0000","last_change":"2026-02-24T22:05:18.191009+0000","last_active":"2026-02-24T22:34:17.250160+0000","last_peered":"2026-02-24T22:34:17.250160+0000","last_clean":"2026-02-24T22:34:17.250160+0000","last_became_active":"2026-02-24T22:05:18.189877+0000","last_became_peered":"2026-02-24T22:05:18.189877+0000","last_unstale":"2026-02-24T22:34:17.250160+0000","last_undegraded":"2026-02-24T22:34:17.250160+0000","last_fullsized":"2026-02-24T22:34:17.250160+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:33:55.890050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":510,"num_write":249,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"312'325","reported_seq":1085,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.991062+0000","last_change":"2026-02-24T22:05:16.184964+0000","last_active":"2026-02-24T22:34:25.991062+0000","last_peered":"2026-02-24T22:34:25.991062+0000","last_clean":"2026-02-24T22:34:25.991062+0000","last_became_active":"2026-02-24T22:05:16.184525+0000","last_became_peered":"2026-02-24T22:05:16.184525+0000","last_unstale":"2026-02-24T22:34:25.991062+0000","last_undegraded":"2026-02-24T22:34:25.991062+0000","last_fullsized":"2026-02-24T22:34:25.991062+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":300,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242941,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":576,"num_read_kb":16320,"num_write":1488,"num_write_kb":16752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"27'1","reported_seq":589,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:59.729910+0000","last_change":"2026-02-24T22:05:20.187071+0000","last_active":"2026-02-24T22:31:59.729910+0000","last_peered":"2026-02-24T22:31:59.729910+0000","last_clean":"2026-02-24T22:31:59.729910+0000","last_became_active":"2026-02-24T22:05:20.186850+0000","last_became_peered":"2026-02-24T22:05:20.186850+0000","last_unstale":"2026-02-24T22:31:59.729910+0000","last_undegraded":"2026-02-24T22:31:59.729910+0000","last_fullsized":"2026-02-24T22:31:59.729910+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:28:00.779043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"312'249","reported_seq":872,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.268142+0000","last_change":"2026-02-24T22:05:22.199568+0000","last_active":"2026-02-24T22:34:25.268142+0000","last_peered":"2026-02-24T22:34:25.268142+0000","last_clean":"2026-02-24T22:34:25.268142+0000","last_became_active":"2026-02-24T22:05:22.199358+0000","last_became_peered":"2026-02-24T22:05:22.199358+0000","last_unstale":"2026-02-24T22:34:25.268142+0000","last_undegraded":"2026-02-24T22:34:25.268142+0000","last_fullsized":"2026-02-24T22:34:25.268142+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":200,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:24:45.196918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37935,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":401,"num_read_kb":1010,"num_write":360,"num_write_kb":308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"312'281","reported_seq":1222,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.228289+0000","last_change":"2026-02-24T22:05:24.200431+0000","last_active":"2026-02-24T22:34:18.228289+0000","last_peered":"2026-02-24T22:34:18.228289+0000","last_clean":"2026-02-24T22:34:18.228289+0000","last_became_active":"2026-02-24T22:05:24.199555+0000","last_became_peered":"2026-02-24T22:05:24.199555+0000","last_unstale":"2026-02-24T22:34:18.228289+0000","last_undegraded":"2026-02-24T22:34:18.228289+0000","last_fullsized":"2026-02-24T22:34:18.228289+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:29:23.063946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30969,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":655,"num_read_kb":647,"num_write":639,"num_write_kb":204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"312'432","reported_seq":1950,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.184363+0000","last_change":"2026-02-24T22:05:29.660609+0000","last_active":"2026-02-24T22:34:25.184363+0000","last_peered":"2026-02-24T22:34:25.184363+0000","last_clean":"2026-02-24T22:34:25.184363+0000","last_became_active":"2026-02-24T22:05:29.659532+0000","last_became_peered":"2026-02-24T22:05:29.659532+0000","last_unstale":"2026-02-24T22:34:25.184363+0000","last_undegraded":"2026-02-24T22:34:25.184363+0000","last_fullsized":"2026-02-24T22:34:25.184363+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":400,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:09:07.648679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"312'139","reported_seq":825,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.725732+0000","last_change":"2026-02-24T22:05:18.184639+0000","last_active":"2026-02-24T22:31:19.725732+0000","last_peered":"2026-02-24T22:31:19.725732+0000","last_clean":"2026-02-24T22:31:19.725732+0000","last_became_active":"2026-02-24T22:05:18.184250+0000","last_became_peered":"2026-02-24T22:05:18.184250+0000","last_unstale":"2026-02-24T22:31:19.725732+0000","last_undegraded":"2026-02-24T22:31:19.725732+0000","last_fullsized":"2026-02-24T22:31:19.725732+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:38:26.901167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":560,"num_read_kb":462,"num_write":204,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"312'1605","reported_seq":2961,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.978401+0000","last_change":"2026-02-24T22:05:16.190173+0000","last_active":"2026-02-24T22:34:25.978401+0000","last_peered":"2026-02-24T22:34:25.978401+0000","last_clean":"2026-02-24T22:34:25.978401+0000","last_became_active":"2026-02-24T22:05:16.189660+0000","last_became_peered":"2026-02-24T22:05:16.189660+0000","last_unstale":"2026-02-24T22:34:25.978401+0000","last_undegraded":"2026-02-24T22:34:25.978401+0000","last_fullsized":"2026-02-24T22:34:25.978401+0000","mapping_epoch":22,"log_start":"312'1500","ondisk_log_start":"312'1500","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":1500,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:28:22.592939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4326,"num_read_kb":2601,"num_write":2684,"num_write_kb":4261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"312'101","reported_seq":779,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:01.356810+0000","last_change":"2026-02-24T22:05:18.184329+0000","last_active":"2026-02-24T22:34:01.356810+0000","last_peered":"2026-02-24T22:34:01.356810+0000","last_clean":"2026-02-24T22:34:01.356810+0000","last_became_active":"2026-02-24T22:05:18.183822+0000","last_became_peered":"2026-02-24T22:05:18.183822+0000","last_unstale":"2026-02-24T22:34:01.356810+0000","last_undegraded":"2026-02-24T22:34:01.356810+0000","last_fullsized":"2026-02-24T22:34:01.356810+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:25:35.055559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":390,"num_read_kb":330,"num_write":143,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"312'300","reported_seq":1049,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.976038+0000","last_change":"2026-02-24T22:05:16.183674+0000","last_active":"2026-02-24T22:34:25.976038+0000","last_peered":"2026-02-24T22:34:25.976038+0000","last_clean":"2026-02-24T22:34:25.976038+0000","last_became_active":"2026-02-24T22:05:16.183487+0000","last_became_peered":"2026-02-24T22:05:16.183487+0000","last_unstale":"2026-02-24T22:34:25.976038+0000","last_undegraded":"2026-02-24T22:34:25.976038+0000","last_fullsized":"2026-02-24T22:34:25.976038+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":200,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":514,"num_read_kb":706,"num_write":1357,"num_write_kb":183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"312'72","reported_seq":731,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.741422+0000","last_change":"2026-02-24T22:05:18.190755+0000","last_active":"2026-02-24T22:31:19.741422+0000","last_peered":"2026-02-24T22:31:19.741422+0000","last_clean":"2026-02-24T22:31:19.741422+0000","last_became_active":"2026-02-24T22:05:18.190352+0000","last_became_peered":"2026-02-24T22:05:18.190352+0000","last_unstale":"2026-02-24T22:31:19.741422+0000","last_undegraded":"2026-02-24T22:31:19.741422+0000","last_fullsized":"2026-02-24T22:31:19.741422+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:16:19.432104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":274,"num_read_kb":236,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"312'314","reported_seq":1237,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.975413+0000","last_change":"2026-02-24T22:05:16.191430+0000","last_active":"2026-02-24T22:34:25.975413+0000","last_peered":"2026-02-24T22:34:25.975413+0000","last_clean":"2026-02-24T22:34:25.975413+0000","last_became_active":"2026-02-24T22:05:16.190494+0000","last_became_peered":"2026-02-24T22:05:16.190494+0000","last_unstale":"2026-02-24T22:34:25.975413+0000","last_undegraded":"2026-02-24T22:34:25.975413+0000","last_fullsized":"2026-02-24T22:34:25.975413+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:14:56.208933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1108,"num_read_kb":14913,"num_write":1331,"num_write_kb":8068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"312'337","reported_seq":1150,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.725684+0000","last_change":"2026-02-24T22:05:18.186968+0000","last_active":"2026-02-24T22:31:19.725684+0000","last_peered":"2026-02-24T22:31:19.725684+0000","last_clean":"2026-02-24T22:31:19.725684+0000","last_became_active":"2026-02-24T22:05:18.186622+0000","last_became_peered":"2026-02-24T22:05:18.186622+0000","last_unstale":"2026-02-24T22:31:19.725684+0000","last_undegraded":"2026-02-24T22:31:19.725684+0000","last_fullsized":"2026-02-24T22:31:19.725684+0000","mapping_epoch":24,"log_start":"312'300","ondisk_log_start":"312'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":300,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:34:46.036169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1512,"num_read_kb":1316,"num_write":481,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"312'318","reported_seq":1157,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.980816+0000","last_change":"2026-02-24T22:05:16.186186+0000","last_active":"2026-02-24T22:34:25.980816+0000","last_peered":"2026-02-24T22:34:25.980816+0000","last_clean":"2026-02-24T22:34:25.980816+0000","last_became_active":"2026-02-24T22:05:16.185977+0000","last_became_peered":"2026-02-24T22:05:16.185977+0000","last_unstale":"2026-02-24T22:34:25.980816+0000","last_undegraded":"2026-02-24T22:34:25.980816+0000","last_fullsized":"2026-02-24T22:34:25.980816+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:53:50.492464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":948,"num_read_kb":5969,"num_write":1368,"num_write_kb":2943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"312'89","reported_seq":764,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:00.529273+0000","last_change":"2026-02-24T22:05:18.186317+0000","last_active":"2026-02-24T22:34:00.529273+0000","last_peered":"2026-02-24T22:34:00.529273+0000","last_clean":"2026-02-24T22:34:00.529273+0000","last_became_active":"2026-02-24T22:05:18.185180+0000","last_became_peered":"2026-02-24T22:05:18.185180+0000","last_unstale":"2026-02-24T22:34:00.529273+0000","last_undegraded":"2026-02-24T22:34:00.529273+0000","last_fullsized":"2026-02-24T22:34:00.529273+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:17:22.057256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":311,"num_read_kb":264,"num_write":113,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"312'370","reported_seq":1122,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.983713+0000","last_change":"2026-02-24T22:06:00.660921+0000","last_active":"2026-02-24T22:34:25.983713+0000","last_peered":"2026-02-24T22:34:25.983713+0000","last_clean":"2026-02-24T22:34:25.983713+0000","last_became_active":"2026-02-24T22:06:00.660740+0000","last_became_peered":"2026-02-24T22:06:00.660740+0000","last_unstale":"2026-02-24T22:34:25.983713+0000","last_undegraded":"2026-02-24T22:34:25.983713+0000","last_fullsized":"2026-02-24T22:34:25.983713+0000","mapping_epoch":36,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":300,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:58:32.246490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":666,"num_read_kb":9895,"num_write":1405,"num_write_kb":11975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"312'66","reported_seq":736,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.746497+0000","last_change":"2026-02-24T22:05:18.191552+0000","last_active":"2026-02-24T22:32:24.746497+0000","last_peered":"2026-02-24T22:32:24.746497+0000","last_clean":"2026-02-24T22:32:24.746497+0000","last_became_active":"2026-02-24T22:05:18.190390+0000","last_became_peered":"2026-02-24T22:05:18.190390+0000","last_unstale":"2026-02-24T22:32:24.746497+0000","last_undegraded":"2026-02-24T22:32:24.746497+0000","last_fullsized":"2026-02-24T22:32:24.746497+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:20:31.773222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":213,"num_read_kb":187,"num_write":90,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"312'332","reported_seq":1055,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.968613+0000","last_change":"2026-02-24T22:05:16.184281+0000","last_active":"2026-02-24T22:34:25.968613+0000","last_peered":"2026-02-24T22:34:25.968613+0000","last_clean":"2026-02-24T22:34:25.968613+0000","last_became_active":"2026-02-24T22:05:16.184091+0000","last_became_peered":"2026-02-24T22:05:16.184091+0000","last_unstale":"2026-02-24T22:34:25.968613+0000","last_undegraded":"2026-02-24T22:34:25.968613+0000","last_fullsized":"2026-02-24T22:34:25.968613+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":300,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:42:08.074892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":558,"num_read_kb":406,"num_write":1237,"num_write_kb":6221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"312'355","reported_seq":1158,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:45.973219+0000","last_change":"2026-02-24T22:05:18.187245+0000","last_active":"2026-02-24T22:33:45.973219+0000","last_peered":"2026-02-24T22:33:45.973219+0000","last_clean":"2026-02-24T22:33:45.973219+0000","last_became_active":"2026-02-24T22:05:18.186746+0000","last_became_peered":"2026-02-24T22:05:18.186746+0000","last_unstale":"2026-02-24T22:33:45.973219+0000","last_undegraded":"2026-02-24T22:33:45.973219+0000","last_fullsized":"2026-02-24T22:33:45.973219+0000","mapping_epoch":24,"log_start":"312'300","ondisk_log_start":"312'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":300,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:16:49.207184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":1676,"num_read_kb":1392,"num_write":638,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"312'329","reported_seq":1033,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.950422+0000","last_change":"2026-02-24T22:05:16.190391+0000","last_active":"2026-02-24T22:34:25.950422+0000","last_peered":"2026-02-24T22:34:25.950422+0000","last_clean":"2026-02-24T22:34:25.950422+0000","last_became_active":"2026-02-24T22:05:16.189516+0000","last_became_peered":"2026-02-24T22:05:16.189516+0000","last_unstale":"2026-02-24T22:34:25.950422+0000","last_undegraded":"2026-02-24T22:34:25.950422+0000","last_fullsized":"2026-02-24T22:34:25.950422+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":300,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:11:15.635731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":448,"num_read_kb":5491,"num_write":1228,"num_write_kb":20654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"312'164","reported_seq":855,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:03.785947+0000","last_change":"2026-02-24T22:05:18.190682+0000","last_active":"2026-02-24T22:34:03.785947+0000","last_peered":"2026-02-24T22:34:03.785947+0000","last_clean":"2026-02-24T22:34:03.785947+0000","last_became_active":"2026-02-24T22:05:18.190203+0000","last_became_peered":"2026-02-24T22:05:18.190203+0000","last_unstale":"2026-02-24T22:34:03.785947+0000","last_undegraded":"2026-02-24T22:34:03.785947+0000","last_fullsized":"2026-02-24T22:34:03.785947+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:20:56.349004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":631,"num_read_kb":516,"num_write":248,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"312'317","reported_seq":1036,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.961792+0000","last_change":"2026-02-24T22:05:16.184697+0000","last_active":"2026-02-24T22:34:25.961792+0000","last_peered":"2026-02-24T22:34:25.961792+0000","last_clean":"2026-02-24T22:34:25.961792+0000","last_became_active":"2026-02-24T22:05:16.183772+0000","last_became_peered":"2026-02-24T22:05:16.183772+0000","last_unstale":"2026-02-24T22:34:25.961792+0000","last_undegraded":"2026-02-24T22:34:25.961792+0000","last_fullsized":"2026-02-24T22:34:25.961792+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:22:42.349222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":480,"num_read_kb":12489,"num_write":1341,"num_write_kb":5970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"312'130","reported_seq":850,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.737414+0000","last_change":"2026-02-24T22:05:18.187529+0000","last_active":"2026-02-24T22:30:59.737414+0000","last_peered":"2026-02-24T22:30:59.737414+0000","last_clean":"2026-02-24T22:30:59.737414+0000","last_became_active":"2026-02-24T22:05:18.187321+0000","last_became_peered":"2026-02-24T22:05:18.187321+0000","last_unstale":"2026-02-24T22:30:59.737414+0000","last_undegraded":"2026-02-24T22:30:59.737414+0000","last_fullsized":"2026-02-24T22:30:59.737414+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:08:55.925019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":512,"num_read_kb":441,"num_write":201,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"312'320","reported_seq":1028,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.965332+0000","last_change":"2026-02-24T22:05:16.193409+0000","last_active":"2026-02-24T22:34:25.965332+0000","last_peered":"2026-02-24T22:34:25.965332+0000","last_clean":"2026-02-24T22:34:25.965332+0000","last_became_active":"2026-02-24T22:05:16.193164+0000","last_became_peered":"2026-02-24T22:05:16.193164+0000","last_unstale":"2026-02-24T22:34:25.965332+0000","last_undegraded":"2026-02-24T22:34:25.965332+0000","last_fullsized":"2026-02-24T22:34:25.965332+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:41:44.916032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242922,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":434,"num_read_kb":36424,"num_write":1378,"num_write_kb":10617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"312'148","reported_seq":825,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.737040+0000","last_change":"2026-02-24T22:05:18.193362+0000","last_active":"2026-02-24T22:30:59.737040+0000","last_peered":"2026-02-24T22:30:59.737040+0000","last_clean":"2026-02-24T22:30:59.737040+0000","last_became_active":"2026-02-24T22:05:18.191737+0000","last_became_peered":"2026-02-24T22:05:18.191737+0000","last_unstale":"2026-02-24T22:30:59.737040+0000","last_undegraded":"2026-02-24T22:30:59.737040+0000","last_fullsized":"2026-02-24T22:30:59.737040+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":100,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:51:20.868848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":539,"num_read_kb":448,"num_write":211,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"312'371","reported_seq":1122,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.980280+0000","last_change":"2026-02-24T22:05:16.184466+0000","last_active":"2026-02-24T22:34:25.980280+0000","last_peered":"2026-02-24T22:34:25.980280+0000","last_clean":"2026-02-24T22:34:25.980280+0000","last_became_active":"2026-02-24T22:05:16.183923+0000","last_became_peered":"2026-02-24T22:05:16.183923+0000","last_unstale":"2026-02-24T22:34:25.980280+0000","last_undegraded":"2026-02-24T22:34:25.980280+0000","last_fullsized":"2026-02-24T22:34:25.980280+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":300,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:15:14.475036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242922,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":549,"num_read_kb":9842,"num_write":1598,"num_write_kb":9073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"312'146","reported_seq":817,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:09.715504+0000","last_change":"2026-02-24T22:05:18.184019+0000","last_active":"2026-02-24T22:30:09.715504+0000","last_peered":"2026-02-24T22:30:09.715504+0000","last_clean":"2026-02-24T22:30:09.715504+0000","last_became_active":"2026-02-24T22:05:18.183761+0000","last_became_peered":"2026-02-24T22:05:18.183761+0000","last_unstale":"2026-02-24T22:30:09.715504+0000","last_undegraded":"2026-02-24T22:30:09.715504+0000","last_fullsized":"2026-02-24T22:30:09.715504+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:07:03.909559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":561,"num_read_kb":451,"num_write":221,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"312'337","reported_seq":1106,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.951417+0000","last_change":"2026-02-24T22:05:16.191018+0000","last_active":"2026-02-24T22:34:25.951417+0000","last_peered":"2026-02-24T22:34:25.951417+0000","last_clean":"2026-02-24T22:34:25.951417+0000","last_became_active":"2026-02-24T22:05:16.189759+0000","last_became_peered":"2026-02-24T22:05:16.189759+0000","last_unstale":"2026-02-24T22:34:25.951417+0000","last_undegraded":"2026-02-24T22:34:25.951417+0000","last_fullsized":"2026-02-24T22:34:25.951417+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":300,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:34:51.111413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":656,"num_read_kb":3529,"num_write":1503,"num_write_kb":3094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"312'140","reported_seq":832,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:31.161689+0000","last_change":"2026-02-24T22:05:18.194125+0000","last_active":"2026-02-24T22:30:31.161689+0000","last_peered":"2026-02-24T22:30:31.161689+0000","last_clean":"2026-02-24T22:30:31.161689+0000","last_became_active":"2026-02-24T22:05:18.192882+0000","last_became_peered":"2026-02-24T22:05:18.192882+0000","last_unstale":"2026-02-24T22:30:31.161689+0000","last_undegraded":"2026-02-24T22:30:31.161689+0000","last_fullsized":"2026-02-24T22:30:31.161689+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":100,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:13:16.621031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":593,"num_read_kb":494,"num_write":208,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"312'306","reported_seq":1044,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.982171+0000","last_change":"2026-02-24T22:05:16.184964+0000","last_active":"2026-02-24T22:34:25.982171+0000","last_peered":"2026-02-24T22:34:25.982171+0000","last_clean":"2026-02-24T22:34:25.982171+0000","last_became_active":"2026-02-24T22:05:16.184557+0000","last_became_peered":"2026-02-24T22:05:16.184557+0000","last_unstale":"2026-02-24T22:34:25.982171+0000","last_undegraded":"2026-02-24T22:34:25.982171+0000","last_fullsized":"2026-02-24T22:34:25.982171+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:16:07.490514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":498,"num_read_kb":1536,"num_write":1259,"num_write_kb":5590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"312'158","reported_seq":865,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:04.901988+0000","last_change":"2026-02-24T22:05:18.191676+0000","last_active":"2026-02-24T22:34:04.901988+0000","last_peered":"2026-02-24T22:34:04.901988+0000","last_clean":"2026-02-24T22:34:04.901988+0000","last_became_active":"2026-02-24T22:05:18.191432+0000","last_became_peered":"2026-02-24T22:05:18.191432+0000","last_unstale":"2026-02-24T22:34:04.901988+0000","last_undegraded":"2026-02-24T22:34:04.901988+0000","last_fullsized":"2026-02-24T22:34:04.901988+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":100,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:14:52.433749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":649,"num_read_kb":535,"num_write":233,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"312'285","reported_seq":1006,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.946946+0000","last_change":"2026-02-24T22:05:16.190689+0000","last_active":"2026-02-24T22:34:25.946946+0000","last_peered":"2026-02-24T22:34:25.946946+0000","last_clean":"2026-02-24T22:34:25.946946+0000","last_became_active":"2026-02-24T22:05:16.190260+0000","last_became_peered":"2026-02-24T22:05:16.190260+0000","last_unstale":"2026-02-24T22:34:25.946946+0000","last_undegraded":"2026-02-24T22:34:25.946946+0000","last_fullsized":"2026-02-24T22:34:25.946946+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:55:30.357144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":458,"num_read_kb":1504,"num_write":1357,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"312'143","reported_seq":831,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:15.308606+0000","last_change":"2026-02-24T22:05:18.185534+0000","last_active":"2026-02-24T22:30:15.308606+0000","last_peered":"2026-02-24T22:30:15.308606+0000","last_clean":"2026-02-24T22:30:15.308606+0000","last_became_active":"2026-02-24T22:05:18.185184+0000","last_became_peered":"2026-02-24T22:05:18.185184+0000","last_unstale":"2026-02-24T22:30:15.308606+0000","last_undegraded":"2026-02-24T22:30:15.308606+0000","last_fullsized":"2026-02-24T22:30:15.308606+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":100,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:50:27.214195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":536,"num_read_kb":445,"num_write":200,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"312'299","reported_seq":1112,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.967530+0000","last_change":"2026-02-24T22:05:16.191775+0000","last_active":"2026-02-24T22:34:25.967530+0000","last_peered":"2026-02-24T22:34:25.967530+0000","last_clean":"2026-02-24T22:34:25.967530+0000","last_became_active":"2026-02-24T22:05:16.191433+0000","last_became_peered":"2026-02-24T22:05:16.191433+0000","last_unstale":"2026-02-24T22:34:25.967530+0000","last_undegraded":"2026-02-24T22:34:25.967530+0000","last_fullsized":"2026-02-24T22:34:25.967530+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":200,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:08:07.147079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":549,"num_read_kb":2264,"num_write":1330,"num_write_kb":7006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"312'62","reported_seq":721,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:08.697007+0000","last_change":"2026-02-24T22:05:18.192824+0000","last_active":"2026-02-24T22:30:08.697007+0000","last_peered":"2026-02-24T22:30:08.697007+0000","last_clean":"2026-02-24T22:30:08.697007+0000","last_became_active":"2026-02-24T22:05:18.192619+0000","last_became_peered":"2026-02-24T22:05:18.192619+0000","last_unstale":"2026-02-24T22:30:08.697007+0000","last_undegraded":"2026-02-24T22:30:08.697007+0000","last_fullsized":"2026-02-24T22:30:08.697007+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:21:57.498044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":204,"num_read_kb":176,"num_write":76,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"312'335","reported_seq":1067,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.978979+0000","last_change":"2026-02-24T22:05:16.184618+0000","last_active":"2026-02-24T22:34:25.978979+0000","last_peered":"2026-02-24T22:34:25.978979+0000","last_clean":"2026-02-24T22:34:25.978979+0000","last_became_active":"2026-02-24T22:05:16.184441+0000","last_became_peered":"2026-02-24T22:05:16.184441+0000","last_unstale":"2026-02-24T22:34:25.978979+0000","last_undegraded":"2026-02-24T22:34:25.978979+0000","last_fullsized":"2026-02-24T22:34:25.978979+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":300,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:14:27.469981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":478,"num_read_kb":3856,"num_write":1468,"num_write_kb":8445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"312'159","reported_seq":876,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:58.577766+0000","last_change":"2026-02-24T22:05:18.191572+0000","last_active":"2026-02-24T22:33:58.577766+0000","last_peered":"2026-02-24T22:33:58.577766+0000","last_clean":"2026-02-24T22:33:58.577766+0000","last_became_active":"2026-02-24T22:05:18.191350+0000","last_became_peered":"2026-02-24T22:05:18.191350+0000","last_unstale":"2026-02-24T22:33:58.577766+0000","last_undegraded":"2026-02-24T22:33:58.577766+0000","last_fullsized":"2026-02-24T22:33:58.577766+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:05:49.312781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":662,"num_read_kb":557,"num_write":245,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"312'301","reported_seq":1069,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.979844+0000","last_change":"2026-02-24T22:06:00.662883+0000","last_active":"2026-02-24T22:34:25.979844+0000","last_peered":"2026-02-24T22:34:25.979844+0000","last_clean":"2026-02-24T22:34:25.979844+0000","last_became_active":"2026-02-24T22:06:00.662656+0000","last_became_peered":"2026-02-24T22:06:00.662656+0000","last_unstale":"2026-02-24T22:34:25.979844+0000","last_undegraded":"2026-02-24T22:34:25.979844+0000","last_fullsized":"2026-02-24T22:34:25.979844+0000","mapping_epoch":36,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:35:38.140607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":803,"num_read_kb":6090,"num_write":1203,"num_write_kb":5556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.30","version":"312'339","reported_seq":1091,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.965815+0000","last_change":"2026-02-24T22:05:16.190494+0000","last_active":"2026-02-24T22:34:25.965815+0000","last_peered":"2026-02-24T22:34:25.965815+0000","last_clean":"2026-02-24T22:34:25.965815+0000","last_became_active":"2026-02-24T22:05:16.189713+0000","last_became_peered":"2026-02-24T22:05:16.189713+0000","last_unstale":"2026-02-24T22:34:25.965815+0000","last_undegraded":"2026-02-24T22:34:25.965815+0000","last_fullsized":"2026-02-24T22:34:25.965815+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":300,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:32:51.972858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242932,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":632,"num_read_kb":31410,"num_write":1455,"num_write_kb":10638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"312'256","reported_seq":941,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:59.744014+0000","last_change":"2026-02-24T22:05:18.187784+0000","last_active":"2026-02-24T22:31:59.744014+0000","last_peered":"2026-02-24T22:31:59.744014+0000","last_clean":"2026-02-24T22:31:59.744014+0000","last_became_active":"2026-02-24T22:05:18.187565+0000","last_became_peered":"2026-02-24T22:05:18.187565+0000","last_unstale":"2026-02-24T22:31:59.744014+0000","last_undegraded":"2026-02-24T22:31:59.744014+0000","last_fullsized":"2026-02-24T22:31:59.744014+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":200,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:43:19.977135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":1060,"num_read_kb":853,"num_write":419,"num_write_kb":222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"312'294","reported_seq":1000,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.929332+0000","last_change":"2026-02-24T22:05:16.178897+0000","last_active":"2026-02-24T22:34:25.929332+0000","last_peered":"2026-02-24T22:34:25.929332+0000","last_clean":"2026-02-24T22:34:25.929332+0000","last_became_active":"2026-02-24T22:05:16.178157+0000","last_became_peered":"2026-02-24T22:05:16.178157+0000","last_unstale":"2026-02-24T22:34:25.929332+0000","last_undegraded":"2026-02-24T22:34:25.929332+0000","last_fullsized":"2026-02-24T22:34:25.929332+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:15.170852+0000","last_clean_scrub_stamp":"2026-02-24T22:05:15.170852+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":200,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:50:48.164748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":466,"num_read_kb":1640,"num_write":1134,"num_write_kb":1702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"312'78","reported_seq":752,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:51.565638+0000","last_change":"2026-02-24T22:05:18.191188+0000","last_active":"2026-02-24T22:33:51.565638+0000","last_peered":"2026-02-24T22:33:51.565638+0000","last_clean":"2026-02-24T22:33:51.565638+0000","last_became_active":"2026-02-24T22:05:18.190902+0000","last_became_peered":"2026-02-24T22:05:18.190902+0000","last_unstale":"2026-02-24T22:33:51.565638+0000","last_undegraded":"2026-02-24T22:33:51.565638+0000","last_fullsized":"2026-02-24T22:33:51.565638+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:18:59.644388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":302,"num_read_kb":262,"num_write":98,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":425,"ondisk_log_size":425,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":973109,"num_objects":3053,"num_object_clones":0,"num_object_copies":6106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3053,"num_whiteouts":0,"num_read":19978,"num_read_kb":245567,"num_write":23762,"num_write_kb":8619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24911872,"data_stored":1946218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5786661,"internal_metadata":0},"log_size":2182,"ondisk_log_size":2182,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1250269,"num_objects":223,"num_object_clones":0,"num_object_copies":446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":223,"num_whiteouts":0,"num_read":45268,"num_read_kb":47987,"num_write":15451,"num_write_kb":13662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":17220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2823344,"internal_metadata":0},"log_size":2137,"ondisk_log_size":2137,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1896,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1448,"num_read_kb":1680,"num_write":22,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":82,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11005,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":1511,"num_object_clones":0,"num_object_copies":3022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1511,"num_whiteouts":0,"num_read":95919,"num_read_kb":81897,"num_write":34645,"num_write_kb":17078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1511,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17301930,"internal_metadata":0},"log_size":3916,"ondisk_log_size":3916,"up":128,"acting":128,"num_store_stats":8},{"poolid":81,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":119816,"num_write":82,"num_write_kb":70656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":112199743,"num_objects":281,"num_object_clones":0,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":0,"num_read":41102,"num_read_kb":763537,"num_write":86142,"num_write_kb":523059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":225296384,"data_stored":224399486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4271,"ondisk_log_size":4271,"up":128,"acting":128,"num_store_stats":8},{"poolid":80,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":124936,"num_write":82,"num_write_kb":70656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":79,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3567,"num_read_kb":2401,"num_write":1127,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":317720,"internal_metadata":0},"log_size":462,"ondisk_log_size":462,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575203,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316888,"kb_used_data":55120,"kb_used_omap":3696,"kb_used_meta":257871,"kb_avail":94054952,"statfs":{"total":96636764160,"available":96312270848,"internally_reserved":0,"allocated":56442880,"data_stored":51634967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3785637,"internal_metadata":264059995},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575203,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":272256,"kb_used_data":28428,"kb_used_omap":3571,"kb_used_meta":239948,"kb_avail":94099584,"statfs":{"total":96636764160,"available":96357974016,"internally_reserved":0,"allocated":29110272,"data_stored":23825325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3657488,"internal_metadata":245706992},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575203,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137296,"kb_used_data":37956,"kb_used_omap":2746,"kb_used_meta":96325,"kb_avail":94234544,"statfs":{"total":96636764160,"available":96496173056,"internally_reserved":0,"allocated":38866944,"data_stored":35212820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2812671,"internal_metadata":98637057},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575203,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":111276,"kb_used_data":48168,"kb_used_omap":3095,"kb_used_meta":59944,"kb_avail":94260564,"statfs":{"total":96636764160,"available":96522817536,"internally_reserved":0,"allocated":49324032,"data_stored":45700457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3169290,"internal_metadata":61383670},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575204,"num_pgs":107,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":225972,"kb_used_data":18380,"kb_used_omap":7232,"kb_used_meta":200191,"kb_avail":94145868,"statfs":{"total":96636764160,"available":96405368832,"internally_reserved":0,"allocated":18821120,"data_stored":13350111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7406186,"internal_metadata":204995990},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575204,"num_pgs":87,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62592,"kb_used_data":17432,"kb_used_omap":1599,"kb_used_meta":43392,"kb_avail":94309248,"statfs":{"total":96636764160,"available":96572669952,"internally_reserved":0,"allocated":17850368,"data_stored":13280880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1638118,"internal_metadata":44433690},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575205,"num_pgs":102,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":111624,"kb_used_data":12424,"kb_used_omap":5094,"kb_used_meta":94041,"kb_avail":94260216,"statfs":{"total":96636764160,"available":96522461184,"internally_reserved":0,"allocated":12722176,"data_stored":8040949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5216565,"internal_metadata":96298699},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575205,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":286372,"kb_used_data":60948,"kb_used_omap":10223,"kb_used_meta":215184,"kb_avail":94085468,"statfs":{"total":96636764160,"available":96343519232,"internally_reserved":0,"allocated":62410752,"data_stored":56934149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10468372,"internal_metadata":220349420},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53624832,"data_stored":53478090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5324800,"data_stored":5243270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":10555392,"data_stored":10486056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10620928,"data_stored":10486391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":43110400,"data_stored":42992165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":32641024,"data_stored":32506526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":21032960,"data_stored":20971830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":48386048,"data_stored":48235158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1667351,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2633902,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1516609,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1844562,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2430346,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":723382,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3546560,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2939218,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":2106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1512,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1072852,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":3356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18545,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":2058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5184,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1705285,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11530,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":2244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8436,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3928064,"data_stored":308322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51420,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3117056,"data_stored":241195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":152550,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3059712,"data_stored":239505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":825183,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3899392,"data_stored":305873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":336934,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1949696,"data_stored":151651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7338,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1925120,"data_stored":148450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":65762,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3837952,"data_stored":298046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2489145,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3194880,"data_stored":253176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1858329,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":79,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":133235,"internal_metadata":0},{"poolid":79,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14290,"internal_metadata":0},{"poolid":79,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23118,"internal_metadata":0},{"poolid":79,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15635,"internal_metadata":0},{"poolid":79,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49631,"internal_metadata":0},{"poolid":79,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49251,"internal_metadata":0},{"poolid":79,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16656,"internal_metadata":0},{"poolid":79,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15904,"internal_metadata":0},{"poolid":80,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":82,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":82,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1886,"internal_metadata":0},{"poolid":82,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6440,"internal_metadata":0},{"poolid":82,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":262,"internal_metadata":0},{"poolid":82,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1265,"internal_metadata":0},{"poolid":82,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":717,"internal_metadata":0},{"poolid":82,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":435,"internal_metadata":0},{"poolid":82,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T22:34:31.009 INFO:tasks.ceph:pgid 82.2 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.009 INFO:tasks.ceph:pgid 81.1 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.009 INFO:tasks.ceph:pgid 80.0 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.010 INFO:tasks.ceph:pgid 79.1f last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.010 INFO:tasks.ceph:pgid 82.3 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.010 INFO:tasks.ceph:pgid 81.0 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.010 INFO:tasks.ceph:pgid 80.1 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.010 INFO:tasks.ceph:pgid 79.1e last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.010 INFO:tasks.ceph:pgid 82.0 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.010 INFO:tasks.ceph:pgid 81.3 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.010 INFO:tasks.ceph:pgid 80.2 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.010 INFO:tasks.ceph:pgid 79.1d last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.010 INFO:tasks.ceph:pgid 82.1 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.010 INFO:tasks.ceph:pgid 81.2 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.010 INFO:tasks.ceph:pgid 80.3 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.011 INFO:tasks.ceph:pgid 79.1c last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.011 INFO:tasks.ceph:pgid 82.6 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.011 INFO:tasks.ceph:pgid 81.5 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.011 INFO:tasks.ceph:pgid 80.4 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.011 INFO:tasks.ceph:pgid 79.1b last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.011 INFO:tasks.ceph:pgid 82.7 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.011 INFO:tasks.ceph:pgid 81.4 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.011 INFO:tasks.ceph:pgid 80.5 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.011 INFO:tasks.ceph:pgid 79.1a last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.011 INFO:tasks.ceph:pgid 82.4 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.011 INFO:tasks.ceph:pgid 81.7 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.011 INFO:tasks.ceph:pgid 80.6 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.011 INFO:tasks.ceph:pgid 79.19 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.012 INFO:tasks.ceph:pgid 82.5 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.012 INFO:tasks.ceph:pgid 81.6 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.012 INFO:tasks.ceph:pgid 80.7 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.012 INFO:tasks.ceph:pgid 79.18 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.012 INFO:tasks.ceph:pgid 82.a last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.012 INFO:tasks.ceph:pgid 81.9 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.012 INFO:tasks.ceph:pgid 80.8 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.012 INFO:tasks.ceph:pgid 79.17 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.012 INFO:tasks.ceph:pgid 82.b last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.012 INFO:tasks.ceph:pgid 81.8 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.012 INFO:tasks.ceph:pgid 80.9 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.012 INFO:tasks.ceph:pgid 79.16 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.012 INFO:tasks.ceph:pgid 82.8 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.013 INFO:tasks.ceph:pgid 81.b last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.013 INFO:tasks.ceph:pgid 80.a last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.013 INFO:tasks.ceph:pgid 79.15 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.013 INFO:tasks.ceph:pgid 82.9 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.013 INFO:tasks.ceph:pgid 81.a last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.013 INFO:tasks.ceph:pgid 80.b last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.013 INFO:tasks.ceph:pgid 79.14 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.013 INFO:tasks.ceph:pgid 82.e last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.013 INFO:tasks.ceph:pgid 81.d last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.013 INFO:tasks.ceph:pgid 80.c last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.013 INFO:tasks.ceph:pgid 79.13 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.013 INFO:tasks.ceph:pgid 82.f last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.013 INFO:tasks.ceph:pgid 81.c last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.014 INFO:tasks.ceph:pgid 80.d last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.014 INFO:tasks.ceph:pgid 79.12 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.014 INFO:tasks.ceph:pgid 82.c last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.014 INFO:tasks.ceph:pgid 81.f last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.014 INFO:tasks.ceph:pgid 80.e last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.014 INFO:tasks.ceph:pgid 79.11 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.014 INFO:tasks.ceph:pgid 82.d last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.014 INFO:tasks.ceph:pgid 81.e last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.014 INFO:tasks.ceph:pgid 80.f last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.014 INFO:tasks.ceph:pgid 79.10 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.014 INFO:tasks.ceph:pgid 82.12 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.014 INFO:tasks.ceph:pgid 81.11 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.014 INFO:tasks.ceph:pgid 80.10 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.015 INFO:tasks.ceph:pgid 79.f last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.015 INFO:tasks.ceph:pgid 82.13 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.015 INFO:tasks.ceph:pgid 81.10 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.015 INFO:tasks.ceph:pgid 80.11 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.015 INFO:tasks.ceph:pgid 79.e last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.015 INFO:tasks.ceph:pgid 82.10 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.015 INFO:tasks.ceph:pgid 81.13 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.015 INFO:tasks.ceph:pgid 80.12 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.015 INFO:tasks.ceph:pgid 79.d last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.015 INFO:tasks.ceph:pgid 82.11 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.015 INFO:tasks.ceph:pgid 81.12 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.015 INFO:tasks.ceph:pgid 80.13 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.015 INFO:tasks.ceph:pgid 79.c last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.015 INFO:tasks.ceph:pgid 82.16 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.016 INFO:tasks.ceph:pgid 81.15 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.016 INFO:tasks.ceph:pgid 80.14 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.016 INFO:tasks.ceph:pgid 79.b last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.016 INFO:tasks.ceph:pgid 82.17 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.016 INFO:tasks.ceph:pgid 81.14 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.016 INFO:tasks.ceph:pgid 80.15 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.016 INFO:tasks.ceph:pgid 79.a last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.016 INFO:tasks.ceph:pgid 82.14 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.016 INFO:tasks.ceph:pgid 81.17 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.016 INFO:tasks.ceph:pgid 80.16 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.016 INFO:tasks.ceph:pgid 79.9 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.016 INFO:tasks.ceph:pgid 82.15 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.016 INFO:tasks.ceph:pgid 81.16 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.016 INFO:tasks.ceph:pgid 80.17 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.017 INFO:tasks.ceph:pgid 79.8 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.017 INFO:tasks.ceph:pgid 82.1a last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.017 INFO:tasks.ceph:pgid 81.19 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.017 INFO:tasks.ceph:pgid 80.18 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.017 INFO:tasks.ceph:pgid 79.7 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.017 INFO:tasks.ceph:pgid 82.1b last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.017 INFO:tasks.ceph:pgid 81.18 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.017 INFO:tasks.ceph:pgid 80.19 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.017 INFO:tasks.ceph:pgid 79.6 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.017 INFO:tasks.ceph:pgid 82.18 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.017 INFO:tasks.ceph:pgid 81.1b last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.017 INFO:tasks.ceph:pgid 80.1a last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.017 INFO:tasks.ceph:pgid 79.5 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.017 INFO:tasks.ceph:pgid 82.19 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.017 INFO:tasks.ceph:pgid 81.1a last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.018 INFO:tasks.ceph:pgid 80.1b last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.018 INFO:tasks.ceph:pgid 79.4 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.018 INFO:tasks.ceph:pgid 82.1e last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.018 INFO:tasks.ceph:pgid 81.1d last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.018 INFO:tasks.ceph:pgid 80.1c last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.018 INFO:tasks.ceph:pgid 79.3 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.018 INFO:tasks.ceph:pgid 82.1f last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.018 INFO:tasks.ceph:pgid 81.1c last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.018 INFO:tasks.ceph:pgid 80.1d last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.018 INFO:tasks.ceph:pgid 79.2 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.018 INFO:tasks.ceph:pgid 82.1c last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.018 INFO:tasks.ceph:pgid 81.1f last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.018 INFO:tasks.ceph:pgid 80.1e last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.018 INFO:tasks.ceph:pgid 79.1 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.018 INFO:tasks.ceph:pgid 82.1d last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.019 INFO:tasks.ceph:pgid 81.1e last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.019 INFO:tasks.ceph:pgid 80.1f last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.019 INFO:tasks.ceph:pgid 79.0 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.019 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.019 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.019 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.019 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.019 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.019 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.019 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.019 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.019 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.019 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.019 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.019 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.020 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.020 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.020 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.020 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.020 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.020 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.020 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.020 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.020 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.020 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.020 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.020 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.020 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.020 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.020 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.021 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.021 INFO:tasks.ceph:pgid 8.18 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.021 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.021 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.021 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.021 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.021 INFO:tasks.ceph:pgid 7.17 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.021 INFO:tasks.ceph:pgid 8.19 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.021 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.021 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.021 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.021 INFO:tasks.ceph:pgid 6.17 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.021 INFO:tasks.ceph:pgid 7.16 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.021 INFO:tasks.ceph:pgid 8.1a last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.021 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.021 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.022 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.022 INFO:tasks.ceph:pgid 6.14 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.022 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.022 INFO:tasks.ceph:pgid 8.1b last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.022 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.022 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.022 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.022 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.022 INFO:tasks.ceph:pgid 7.14 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.022 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.022 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.022 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.022 INFO:tasks.ceph:pgid 6.a last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.022 INFO:tasks.ceph:pgid 7.b last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.022 INFO:tasks.ceph:pgid 8.4 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.022 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.022 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.023 INFO:tasks.ceph:pgid 5.8 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.023 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.023 INFO:tasks.ceph:pgid 7.a last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.023 INFO:tasks.ceph:pgid 8.5 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.023 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.023 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.023 INFO:tasks.ceph:pgid 5.b last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.023 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.023 INFO:tasks.ceph:pgid 7.9 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.023 INFO:tasks.ceph:pgid 8.6 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.023 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.023 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.023 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.023 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.023 INFO:tasks.ceph:pgid 7.8 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.023 INFO:tasks.ceph:pgid 8.7 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.024 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.024 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.024 INFO:tasks.ceph:pgid 5.d last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.024 INFO:tasks.ceph:pgid 6.e last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.024 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.024 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.024 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.024 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.024 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.024 INFO:tasks.ceph:pgid 6.f last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.024 INFO:tasks.ceph:pgid 7.e last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.024 INFO:tasks.ceph:pgid 8.1 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.024 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.024 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.024 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.024 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.024 INFO:tasks.ceph:pgid 7.d last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.025 INFO:tasks.ceph:pgid 8.2 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.025 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.025 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.025 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.025 INFO:tasks.ceph:pgid 6.d last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.025 INFO:tasks.ceph:pgid 7.c last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.025 INFO:tasks.ceph:pgid 8.3 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.025 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.025 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-24T22:05:05.664397+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=5, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.025 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2026-02-24T22:05:07.670081+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=7, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.025 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.025 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.025 INFO:tasks.ceph:pgid 6.7 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.025 INFO:tasks.ceph:pgid 7.6 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.025 INFO:tasks.ceph:pgid 8.9 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.025 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.025 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2026-02-24T22:05:07.670081+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=7, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.026 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.026 INFO:tasks.ceph:pgid 6.4 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.026 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.026 INFO:tasks.ceph:pgid 8.a last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.026 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.026 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2026-02-24T22:05:07.670081+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=7, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.026 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.026 INFO:tasks.ceph:pgid 5.6 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.026 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.026 INFO:tasks.ceph:pgid 7.4 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.026 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.026 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.026 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2026-02-24T22:05:07.670081+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=7, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.026 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.026 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.026 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.026 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.026 INFO:tasks.ceph:pgid 8.8 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.027 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.027 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.027 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2026-02-24T22:05:07.670081+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=7, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.027 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.027 INFO:tasks.ceph:pgid 6.0 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.027 INFO:tasks.ceph:pgid 7.1 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.027 INFO:tasks.ceph:pgid 8.e last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.027 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.027 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.027 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2026-02-24T22:05:07.670081+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=7, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.027 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.027 INFO:tasks.ceph:pgid 6.1 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.027 INFO:tasks.ceph:pgid 7.0 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.027 INFO:tasks.ceph:pgid 8.f last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.027 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.027 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.027 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2026-02-24T22:05:07.670081+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=7, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.027 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.028 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.028 INFO:tasks.ceph:pgid 7.3 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.028 INFO:tasks.ceph:pgid 8.c last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.028 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.028 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.028 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2026-02-24T22:05:07.670081+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=7, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.028 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.028 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.028 INFO:tasks.ceph:pgid 7.2 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.028 INFO:tasks.ceph:pgid 8.d last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.028 INFO:tasks.ceph:pgid 8.1f last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.028 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.028 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.028 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.028 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.028 INFO:tasks.ceph:pgid 7.10 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.028 INFO:tasks.ceph:pgid 8.1e last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.028 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.028 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.029 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.029 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.029 INFO:tasks.ceph:pgid 7.11 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.029 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.029 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.029 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.029 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.029 INFO:tasks.ceph:pgid 6.13 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.029 INFO:tasks.ceph:pgid 7.12 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.029 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.029 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.029 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.029 INFO:tasks.ceph:pgid 5.11 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.029 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.029 INFO:tasks.ceph:pgid 7.13 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.029 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.029 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.029 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.029 INFO:tasks.ceph:pgid 6.1d last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.030 INFO:tasks.ceph:pgid 7.1c last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.030 INFO:tasks.ceph:pgid 8.13 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.030 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.030 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.030 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.030 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.030 INFO:tasks.ceph:pgid 7.1d last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.030 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.030 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.030 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.030 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.030 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.030 INFO:tasks.ceph:pgid 7.1e last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.030 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.030 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.030 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.030 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.030 INFO:tasks.ceph:pgid 6.1e last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.030 INFO:tasks.ceph:pgid 7.1f last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.031 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.031 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.031 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.031 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.031 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.031 INFO:tasks.ceph:pgid 7.18 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.031 INFO:tasks.ceph:pgid 8.17 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.031 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.031 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.031 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.031 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.031 INFO:tasks.ceph:pgid 7.19 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.031 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.031 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.031 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.031 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.031 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.031 INFO:tasks.ceph:pgid 7.1a last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.031 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.032 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.032 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.032 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.032 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.032 INFO:tasks.ceph:pgid 7.1b last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.032 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.032 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.032 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.032 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.032 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.032 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.032 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.032 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.032 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.032 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.032 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.032 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.032 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.032 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.032 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.032 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.033 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.033 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.033 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.033 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.033 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.033 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.033 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.033 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.033 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.033 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.033 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.033 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.033 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.033 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.033 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.033 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.033 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.033 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.033 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.033 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2026-02-24T22:05:15.170852+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=15, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.034 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:31.034 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-24T22:34:51.034 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-24T22:34:51.206 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:34:51.223 INFO:teuthology.orchestra.run.trial046.stderr:dumped all 2026-02-24T22:34:51.225 INFO:teuthology.orchestra.run.trial046.stdout:{"pg_ready":true,"pg_map":{"version":1198,"stamp":"2026-02-24T22:34:50.029528+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":115015404,"num_objects":5093,"num_object_clones":0,"num_object_copies":10186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5093,"num_whiteouts":0,"num_read":207562,"num_read_kb":1387906,"num_write":161478,"num_write_kb":706745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":1543,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":35623,"num_omap_keys":144,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13883,"ondisk_log_size":13883,"up":786,"acting":786,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":786,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1541192,"kb_used_data":278856,"kb_used_omap":37260,"kb_used_meta":1224947,"kb_avail":753433528,"statfs":{"total":773094113280,"available":771515932672,"internally_reserved":0,"allocated":285548544,"data_stored":247979658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38154327,"internal_metadata":1254346665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":35623,"num_omap_keys":144,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002838"},"pg_stats":[{"pgid":"82.2","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744748+0000","last_change":"2026-02-24T22:23:29.198729+0000","last_active":"2026-02-24T22:31:49.744748+0000","last_peered":"2026-02-24T22:31:49.744748+0000","last_clean":"2026-02-24T22:31:49.744748+0000","last_became_active":"2026-02-24T22:23:29.198476+0000","last_became_peered":"2026-02-24T22:23:29.198476+0000","last_unstale":"2026-02-24T22:31:49.744748+0000","last_undegraded":"2026-02-24T22:31:49.744748+0000","last_fullsized":"2026-02-24T22:31:49.744748+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:12:35.165001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.1","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.730924+0000","last_change":"2026-02-24T22:15:25.830397+0000","last_active":"2026-02-24T22:32:09.730924+0000","last_peered":"2026-02-24T22:32:09.730924+0000","last_clean":"2026-02-24T22:32:09.730924+0000","last_became_active":"2026-02-24T22:15:25.830000+0000","last_became_peered":"2026-02-24T22:15:25.830000+0000","last_unstale":"2026-02-24T22:32:09.730924+0000","last_undegraded":"2026-02-24T22:32:09.730924+0000","last_fullsized":"2026-02-24T22:32:09.730924+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:41:30.400463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.0","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:29.723258+0000","last_change":"2026-02-24T22:15:22.817921+0000","last_active":"2026-02-24T22:31:29.723258+0000","last_peered":"2026-02-24T22:31:29.723258+0000","last_clean":"2026-02-24T22:31:29.723258+0000","last_became_active":"2026-02-24T22:15:22.817446+0000","last_became_peered":"2026-02-24T22:15:22.817446+0000","last_unstale":"2026-02-24T22:31:29.723258+0000","last_undegraded":"2026-02-24T22:31:29.723258+0000","last_fullsized":"2026-02-24T22:31:29.723258+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:25:30.206672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.1f","version":"312'20","reported_seq":121,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:29.747306+0000","last_change":"2026-02-24T22:13:43.761604+0000","last_active":"2026-02-24T22:32:29.747306+0000","last_peered":"2026-02-24T22:32:29.747306+0000","last_clean":"2026-02-24T22:32:29.747306+0000","last_became_active":"2026-02-24T22:13:43.761120+0000","last_became_peered":"2026-02-24T22:13:43.761120+0000","last_unstale":"2026-02-24T22:32:29.747306+0000","last_undegraded":"2026-02-24T22:32:29.747306+0000","last_fullsized":"2026-02-24T22:32:29.747306+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":168,"num_read_kb":114,"num_write":47,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.3","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742865+0000","last_change":"2026-02-24T22:23:29.213779+0000","last_active":"2026-02-24T22:31:49.742865+0000","last_peered":"2026-02-24T22:31:49.742865+0000","last_clean":"2026-02-24T22:31:49.742865+0000","last_became_active":"2026-02-24T22:23:29.213534+0000","last_became_peered":"2026-02-24T22:23:29.213534+0000","last_unstale":"2026-02-24T22:31:49.742865+0000","last_undegraded":"2026-02-24T22:31:49.742865+0000","last_fullsized":"2026-02-24T22:31:49.742865+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:38:56.711711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.0","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.743144+0000","last_change":"2026-02-24T22:15:25.827702+0000","last_active":"2026-02-24T22:31:34.743144+0000","last_peered":"2026-02-24T22:31:34.743144+0000","last_clean":"2026-02-24T22:31:34.743144+0000","last_became_active":"2026-02-24T22:15:25.827355+0000","last_became_peered":"2026-02-24T22:15:25.827355+0000","last_unstale":"2026-02-24T22:31:34.743144+0000","last_undegraded":"2026-02-24T22:31:34.743144+0000","last_fullsized":"2026-02-24T22:31:34.743144+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:22:12.090995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.1","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:04.744507+0000","last_change":"2026-02-24T22:15:22.818757+0000","last_active":"2026-02-24T22:32:04.744507+0000","last_peered":"2026-02-24T22:32:04.744507+0000","last_clean":"2026-02-24T22:32:04.744507+0000","last_became_active":"2026-02-24T22:15:22.818519+0000","last_became_peered":"2026-02-24T22:15:22.818519+0000","last_unstale":"2026-02-24T22:32:04.744507+0000","last_undegraded":"2026-02-24T22:32:04.744507+0000","last_fullsized":"2026-02-24T22:32:04.744507+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:17:56.378157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.1e","version":"312'9","reported_seq":88,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.745172+0000","last_change":"2026-02-24T22:13:43.757760+0000","last_active":"2026-02-24T22:31:49.745172+0000","last_peered":"2026-02-24T22:31:49.745172+0000","last_clean":"2026-02-24T22:31:49.745172+0000","last_became_active":"2026-02-24T22:13:43.757543+0000","last_became_peered":"2026-02-24T22:13:43.757543+0000","last_unstale":"2026-02-24T22:31:49.745172+0000","last_undegraded":"2026-02-24T22:31:49.745172+0000","last_fullsized":"2026-02-24T22:31:49.745172+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:44:23.087405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":38,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.0","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.723743+0000","last_change":"2026-02-24T22:23:29.212801+0000","last_active":"2026-02-24T22:31:34.723743+0000","last_peered":"2026-02-24T22:31:34.723743+0000","last_clean":"2026-02-24T22:31:34.723743+0000","last_became_active":"2026-02-24T22:23:29.212043+0000","last_became_peered":"2026-02-24T22:23:29.212043+0000","last_unstale":"2026-02-24T22:31:34.723743+0000","last_undegraded":"2026-02-24T22:31:34.723743+0000","last_fullsized":"2026-02-24T22:31:34.723743+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:45:48.953242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.3","version":"312'8","reported_seq":53,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.471100+0000","last_change":"2026-02-24T22:15:25.827590+0000","last_active":"2026-02-24T22:34:25.471100+0000","last_peered":"2026-02-24T22:34:25.471100+0000","last_clean":"2026-02-24T22:34:25.471100+0000","last_became_active":"2026-02-24T22:15:25.827216+0000","last_became_peered":"2026-02-24T22:15:25.827216+0000","last_unstale":"2026-02-24T22:34:25.471100+0000","last_undegraded":"2026-02-24T22:34:25.471100+0000","last_fullsized":"2026-02-24T22:34:25.471100+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:08:31.068519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":25602,"num_write":13,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.2","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:04.726927+0000","last_change":"2026-02-24T22:15:22.818494+0000","last_active":"2026-02-24T22:32:04.726927+0000","last_peered":"2026-02-24T22:32:04.726927+0000","last_clean":"2026-02-24T22:32:04.726927+0000","last_became_active":"2026-02-24T22:15:22.818194+0000","last_became_peered":"2026-02-24T22:15:22.818194+0000","last_unstale":"2026-02-24T22:32:04.726927+0000","last_undegraded":"2026-02-24T22:32:04.726927+0000","last_fullsized":"2026-02-24T22:32:04.726927+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:30:45.313454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.1d","version":"312'22","reported_seq":121,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.737198+0000","last_change":"2026-02-24T22:13:43.757707+0000","last_active":"2026-02-24T22:31:14.737198+0000","last_peered":"2026-02-24T22:31:14.737198+0000","last_clean":"2026-02-24T22:31:14.737198+0000","last_became_active":"2026-02-24T22:13:43.757278+0000","last_became_peered":"2026-02-24T22:13:43.757278+0000","last_unstale":"2026-02-24T22:31:14.737198+0000","last_undegraded":"2026-02-24T22:31:14.737198+0000","last_fullsized":"2026-02-24T22:31:14.737198+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:52:33.539660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":103,"num_write":54,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.1","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742841+0000","last_change":"2026-02-24T22:23:29.214295+0000","last_active":"2026-02-24T22:31:49.742841+0000","last_peered":"2026-02-24T22:31:49.742841+0000","last_clean":"2026-02-24T22:31:49.742841+0000","last_became_active":"2026-02-24T22:23:29.214064+0000","last_became_peered":"2026-02-24T22:23:29.214064+0000","last_unstale":"2026-02-24T22:31:49.742841+0000","last_undegraded":"2026-02-24T22:31:49.742841+0000","last_fullsized":"2026-02-24T22:31:49.742841+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:03:04.365969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.2","version":"312'2","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:14.748763+0000","last_change":"2026-02-24T22:15:25.833367+0000","last_active":"2026-02-24T22:34:14.748763+0000","last_peered":"2026-02-24T22:34:14.748763+0000","last_clean":"2026-02-24T22:34:14.748763+0000","last_became_active":"2026-02-24T22:15:25.832809+0000","last_became_peered":"2026-02-24T22:15:25.832809+0000","last_unstale":"2026-02-24T22:34:14.748763+0000","last_undegraded":"2026-02-24T22:34:14.748763+0000","last_fullsized":"2026-02-24T22:34:14.748763+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:28:13.551199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"80.3","version":"312'3","reported_seq":44,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:09.739916+0000","last_change":"2026-02-24T22:15:22.818401+0000","last_active":"2026-02-24T22:34:09.739916+0000","last_peered":"2026-02-24T22:34:09.739916+0000","last_clean":"2026-02-24T22:34:09.739916+0000","last_became_active":"2026-02-24T22:15:22.818089+0000","last_became_peered":"2026-02-24T22:15:22.818089+0000","last_unstale":"2026-02-24T22:34:09.739916+0000","last_undegraded":"2026-02-24T22:34:09.739916+0000","last_fullsized":"2026-02-24T22:34:09.739916+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:40:25.179819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.1c","version":"312'14","reported_seq":101,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.739030+0000","last_change":"2026-02-24T22:13:43.760682+0000","last_active":"2026-02-24T22:31:14.739030+0000","last_peered":"2026-02-24T22:31:14.739030+0000","last_clean":"2026-02-24T22:31:14.739030+0000","last_became_active":"2026-02-24T22:13:43.759958+0000","last_became_peered":"2026-02-24T22:13:43.759958+0000","last_unstale":"2026-02-24T22:31:14.739030+0000","last_undegraded":"2026-02-24T22:31:14.739030+0000","last_fullsized":"2026-02-24T22:31:14.739030+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:24:39.289428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":75,"num_write":36,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.6","version":"309'1","reported_seq":23,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.730973+0000","last_change":"2026-02-24T22:23:29.215576+0000","last_active":"2026-02-24T22:32:09.730973+0000","last_peered":"2026-02-24T22:32:09.730973+0000","last_clean":"2026-02-24T22:32:09.730973+0000","last_became_active":"2026-02-24T22:23:29.215327+0000","last_became_peered":"2026-02-24T22:23:29.215327+0000","last_unstale":"2026-02-24T22:32:09.730973+0000","last_undegraded":"2026-02-24T22:32:09.730973+0000","last_fullsized":"2026-02-24T22:32:09.730973+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:51:01.281339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.5","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:14.745596+0000","last_change":"2026-02-24T22:15:25.832444+0000","last_active":"2026-02-24T22:32:14.745596+0000","last_peered":"2026-02-24T22:32:14.745596+0000","last_clean":"2026-02-24T22:32:14.745596+0000","last_became_active":"2026-02-24T22:15:25.831975+0000","last_became_peered":"2026-02-24T22:15:25.831975+0000","last_unstale":"2026-02-24T22:32:14.745596+0000","last_undegraded":"2026-02-24T22:32:14.745596+0000","last_fullsized":"2026-02-24T22:32:14.745596+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:04:17.334836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"80.4","version":"312'3","reported_seq":44,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:09.761034+0000","last_change":"2026-02-24T22:15:22.812647+0000","last_active":"2026-02-24T22:34:09.761034+0000","last_peered":"2026-02-24T22:34:09.761034+0000","last_clean":"2026-02-24T22:34:09.761034+0000","last_became_active":"2026-02-24T22:15:22.812270+0000","last_became_peered":"2026-02-24T22:15:22.812270+0000","last_unstale":"2026-02-24T22:34:09.761034+0000","last_undegraded":"2026-02-24T22:34:09.761034+0000","last_fullsized":"2026-02-24T22:34:09.761034+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:42:27.774508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.1b","version":"312'18","reported_seq":112,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.741475+0000","last_change":"2026-02-24T22:13:43.755424+0000","last_active":"2026-02-24T22:31:14.741475+0000","last_peered":"2026-02-24T22:31:14.741475+0000","last_clean":"2026-02-24T22:31:14.741475+0000","last_became_active":"2026-02-24T22:13:43.755170+0000","last_became_peered":"2026-02-24T22:13:43.755170+0000","last_unstale":"2026-02-24T22:31:14.741475+0000","last_undegraded":"2026-02-24T22:31:14.741475+0000","last_fullsized":"2026-02-24T22:31:14.741475+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:29:17.090743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":101,"num_write":42,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.7","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.735768+0000","last_change":"2026-02-24T22:23:29.214459+0000","last_active":"2026-02-24T22:32:09.735768+0000","last_peered":"2026-02-24T22:32:09.735768+0000","last_clean":"2026-02-24T22:32:09.735768+0000","last_became_active":"2026-02-24T22:23:29.214105+0000","last_became_peered":"2026-02-24T22:23:29.214105+0000","last_unstale":"2026-02-24T22:32:09.735768+0000","last_undegraded":"2026-02-24T22:32:09.735768+0000","last_fullsized":"2026-02-24T22:32:09.735768+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:00:10.302637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.4","version":"312'3","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:14.760553+0000","last_change":"2026-02-24T22:15:25.827716+0000","last_active":"2026-02-24T22:34:14.760553+0000","last_peered":"2026-02-24T22:34:14.760553+0000","last_clean":"2026-02-24T22:34:14.760553+0000","last_became_active":"2026-02-24T22:15:25.826663+0000","last_became_peered":"2026-02-24T22:15:25.826663+0000","last_unstale":"2026-02-24T22:34:14.760553+0000","last_undegraded":"2026-02-24T22:34:14.760553+0000","last_fullsized":"2026-02-24T22:34:14.760553+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:08:34.544962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.5","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.730960+0000","last_change":"2026-02-24T22:15:22.816880+0000","last_active":"2026-02-24T22:32:09.730960+0000","last_peered":"2026-02-24T22:32:09.730960+0000","last_clean":"2026-02-24T22:32:09.730960+0000","last_became_active":"2026-02-24T22:15:22.816697+0000","last_became_peered":"2026-02-24T22:15:22.816697+0000","last_unstale":"2026-02-24T22:32:09.730960+0000","last_undegraded":"2026-02-24T22:32:09.730960+0000","last_fullsized":"2026-02-24T22:32:09.730960+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:56:01.670361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"79.1a","version":"312'16","reported_seq":102,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.737175+0000","last_change":"2026-02-24T22:13:43.757739+0000","last_active":"2026-02-24T22:31:14.737175+0000","last_peered":"2026-02-24T22:31:14.737175+0000","last_clean":"2026-02-24T22:31:14.737175+0000","last_became_active":"2026-02-24T22:13:43.757353+0000","last_became_peered":"2026-02-24T22:13:43.757353+0000","last_unstale":"2026-02-24T22:31:14.737175+0000","last_undegraded":"2026-02-24T22:31:14.737175+0000","last_fullsized":"2026-02-24T22:31:14.737175+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:47:30.725736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":65,"num_write":38,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.4","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742881+0000","last_change":"2026-02-24T22:23:29.215206+0000","last_active":"2026-02-24T22:31:49.742881+0000","last_peered":"2026-02-24T22:31:49.742881+0000","last_clean":"2026-02-24T22:31:49.742881+0000","last_became_active":"2026-02-24T22:23:29.215014+0000","last_became_peered":"2026-02-24T22:23:29.215014+0000","last_unstale":"2026-02-24T22:31:49.742881+0000","last_undegraded":"2026-02-24T22:31:49.742881+0000","last_fullsized":"2026-02-24T22:31:49.742881+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:43:42.586588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.7","version":"312'3","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:49.731657+0000","last_change":"2026-02-24T22:15:25.832214+0000","last_active":"2026-02-24T22:32:49.731657+0000","last_peered":"2026-02-24T22:32:49.731657+0000","last_clean":"2026-02-24T22:32:49.731657+0000","last_became_active":"2026-02-24T22:15:25.831806+0000","last_became_peered":"2026-02-24T22:15:25.831806+0000","last_unstale":"2026-02-24T22:32:49.731657+0000","last_undegraded":"2026-02-24T22:32:49.731657+0000","last_fullsized":"2026-02-24T22:32:49.731657+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:36:56.341773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"80.6","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:34.736157+0000","last_change":"2026-02-24T22:15:22.811674+0000","last_active":"2026-02-24T22:30:34.736157+0000","last_peered":"2026-02-24T22:30:34.736157+0000","last_clean":"2026-02-24T22:30:34.736157+0000","last_became_active":"2026-02-24T22:15:22.811413+0000","last_became_peered":"2026-02-24T22:15:22.811413+0000","last_unstale":"2026-02-24T22:30:34.736157+0000","last_undegraded":"2026-02-24T22:30:34.736157+0000","last_fullsized":"2026-02-24T22:30:34.736157+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:14:20.030861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"79.19","version":"312'30","reported_seq":140,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.738215+0000","last_change":"2026-02-24T22:13:43.756439+0000","last_active":"2026-02-24T22:30:44.738215+0000","last_peered":"2026-02-24T22:30:44.738215+0000","last_clean":"2026-02-24T22:30:44.738215+0000","last_became_active":"2026-02-24T22:13:43.756114+0000","last_became_peered":"2026-02-24T22:13:43.756114+0000","last_unstale":"2026-02-24T22:30:44.738215+0000","last_undegraded":"2026-02-24T22:30:44.738215+0000","last_fullsized":"2026-02-24T22:30:44.738215+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:45:46.139575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":216,"num_read_kb":143,"num_write":68,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.5","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742892+0000","last_change":"2026-02-24T22:23:29.213537+0000","last_active":"2026-02-24T22:31:49.742892+0000","last_peered":"2026-02-24T22:31:49.742892+0000","last_clean":"2026-02-24T22:31:49.742892+0000","last_became_active":"2026-02-24T22:23:29.213256+0000","last_became_peered":"2026-02-24T22:23:29.213256+0000","last_unstale":"2026-02-24T22:31:49.742892+0000","last_undegraded":"2026-02-24T22:31:49.742892+0000","last_fullsized":"2026-02-24T22:31:49.742892+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:48:26.724830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.6","version":"312'3","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:49.735160+0000","last_change":"2026-02-24T22:15:25.833922+0000","last_active":"2026-02-24T22:32:49.735160+0000","last_peered":"2026-02-24T22:32:49.735160+0000","last_clean":"2026-02-24T22:32:49.735160+0000","last_became_active":"2026-02-24T22:15:25.833680+0000","last_became_peered":"2026-02-24T22:15:25.833680+0000","last_unstale":"2026-02-24T22:32:49.735160+0000","last_undegraded":"2026-02-24T22:32:49.735160+0000","last_fullsized":"2026-02-24T22:32:49.735160+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:26:51.677018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.7","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:34.736181+0000","last_change":"2026-02-24T22:15:22.812054+0000","last_active":"2026-02-24T22:30:34.736181+0000","last_peered":"2026-02-24T22:30:34.736181+0000","last_clean":"2026-02-24T22:30:34.736181+0000","last_became_active":"2026-02-24T22:15:22.811618+0000","last_became_peered":"2026-02-24T22:15:22.811618+0000","last_unstale":"2026-02-24T22:30:34.736181+0000","last_undegraded":"2026-02-24T22:30:34.736181+0000","last_fullsized":"2026-02-24T22:30:34.736181+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:37:58.903441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"79.18","version":"312'12","reported_seq":95,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.737670+0000","last_change":"2026-02-24T22:13:43.755501+0000","last_active":"2026-02-24T22:30:39.737670+0000","last_peered":"2026-02-24T22:30:39.737670+0000","last_clean":"2026-02-24T22:30:39.737670+0000","last_became_active":"2026-02-24T22:13:43.754981+0000","last_became_peered":"2026-02-24T22:13:43.754981+0000","last_unstale":"2026-02-24T22:30:39.737670+0000","last_undegraded":"2026-02-24T22:30:39.737670+0000","last_fullsized":"2026-02-24T22:30:39.737670+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:41:38.765379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":86,"num_read_kb":57,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.a","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.745052+0000","last_change":"2026-02-24T22:23:29.212877+0000","last_active":"2026-02-24T22:32:09.745052+0000","last_peered":"2026-02-24T22:32:09.745052+0000","last_clean":"2026-02-24T22:32:09.745052+0000","last_became_active":"2026-02-24T22:23:29.212213+0000","last_became_peered":"2026-02-24T22:23:29.212213+0000","last_unstale":"2026-02-24T22:32:09.745052+0000","last_undegraded":"2026-02-24T22:32:09.745052+0000","last_fullsized":"2026-02-24T22:32:09.745052+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:29:28.605583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.9","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.736816+0000","last_change":"2026-02-24T22:15:25.826412+0000","last_active":"2026-02-24T22:30:39.736816+0000","last_peered":"2026-02-24T22:30:39.736816+0000","last_clean":"2026-02-24T22:30:39.736816+0000","last_became_active":"2026-02-24T22:15:25.826076+0000","last_became_peered":"2026-02-24T22:15:25.826076+0000","last_unstale":"2026-02-24T22:30:39.736816+0000","last_undegraded":"2026-02-24T22:30:39.736816+0000","last_fullsized":"2026-02-24T22:30:39.736816+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:00:25.763716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.8","version":"312'6","reported_seq":48,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.572211+0000","last_change":"2026-02-24T22:15:22.816346+0000","last_active":"2026-02-24T22:34:25.572211+0000","last_peered":"2026-02-24T22:34:25.572211+0000","last_clean":"2026-02-24T22:34:25.572211+0000","last_became_active":"2026-02-24T22:15:22.815960+0000","last_became_peered":"2026-02-24T22:15:22.815960+0000","last_unstale":"2026-02-24T22:34:25.572211+0000","last_undegraded":"2026-02-24T22:34:25.572211+0000","last_fullsized":"2026-02-24T22:34:25.572211+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:55:34.277066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6144,"num_write":11,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.17","version":"312'34","reported_seq":169,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.721687+0000","last_change":"2026-02-24T22:14:00.749701+0000","last_active":"2026-02-24T22:30:39.721687+0000","last_peered":"2026-02-24T22:30:39.721687+0000","last_clean":"2026-02-24T22:30:39.721687+0000","last_became_active":"2026-02-24T22:14:00.748438+0000","last_became_peered":"2026-02-24T22:14:00.748438+0000","last_unstale":"2026-02-24T22:30:39.721687+0000","last_undegraded":"2026-02-24T22:30:39.721687+0000","last_fullsized":"2026-02-24T22:30:39.721687+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":293,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:21:49.815767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":298,"num_read_kb":211,"num_write":78,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.b","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:29.745272+0000","last_change":"2026-02-24T22:23:29.197335+0000","last_active":"2026-02-24T22:32:29.745272+0000","last_peered":"2026-02-24T22:32:29.745272+0000","last_clean":"2026-02-24T22:32:29.745272+0000","last_became_active":"2026-02-24T22:23:29.196790+0000","last_became_peered":"2026-02-24T22:23:29.196790+0000","last_unstale":"2026-02-24T22:32:29.745272+0000","last_undegraded":"2026-02-24T22:32:29.745272+0000","last_fullsized":"2026-02-24T22:32:29.745272+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.8","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.734884+0000","last_change":"2026-02-24T22:15:25.825567+0000","last_active":"2026-02-24T22:30:39.734884+0000","last_peered":"2026-02-24T22:30:39.734884+0000","last_clean":"2026-02-24T22:30:39.734884+0000","last_became_active":"2026-02-24T22:15:25.825315+0000","last_became_peered":"2026-02-24T22:15:25.825315+0000","last_unstale":"2026-02-24T22:30:39.734884+0000","last_undegraded":"2026-02-24T22:30:39.734884+0000","last_fullsized":"2026-02-24T22:30:39.734884+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:24:39.313916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.9","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:34.732583+0000","last_change":"2026-02-24T22:15:22.811365+0000","last_active":"2026-02-24T22:30:34.732583+0000","last_peered":"2026-02-24T22:30:34.732583+0000","last_clean":"2026-02-24T22:30:34.732583+0000","last_became_active":"2026-02-24T22:15:22.811141+0000","last_became_peered":"2026-02-24T22:15:22.811141+0000","last_unstale":"2026-02-24T22:30:34.732583+0000","last_undegraded":"2026-02-24T22:30:34.732583+0000","last_fullsized":"2026-02-24T22:30:34.732583+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:45:06.384015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"79.16","version":"0'0","reported_seq":63,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.733272+0000","last_change":"2026-02-24T22:13:43.756602+0000","last_active":"2026-02-24T22:30:39.733272+0000","last_peered":"2026-02-24T22:30:39.733272+0000","last_clean":"2026-02-24T22:30:39.733272+0000","last_became_active":"2026-02-24T22:13:43.755885+0000","last_became_peered":"2026-02-24T22:13:43.755885+0000","last_unstale":"2026-02-24T22:30:39.733272+0000","last_undegraded":"2026-02-24T22:30:39.733272+0000","last_fullsized":"2026-02-24T22:30:39.733272+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:40:07.171226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.8","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.744882+0000","last_change":"2026-02-24T22:23:29.199299+0000","last_active":"2026-02-24T22:32:09.744882+0000","last_peered":"2026-02-24T22:32:09.744882+0000","last_clean":"2026-02-24T22:32:09.744882+0000","last_became_active":"2026-02-24T22:23:29.198882+0000","last_became_peered":"2026-02-24T22:23:29.198882+0000","last_unstale":"2026-02-24T22:32:09.744882+0000","last_undegraded":"2026-02-24T22:32:09.744882+0000","last_fullsized":"2026-02-24T22:32:09.744882+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:59:03.245529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"81.b","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.741464+0000","last_change":"2026-02-24T22:15:25.827664+0000","last_active":"2026-02-24T22:31:14.741464+0000","last_peered":"2026-02-24T22:31:14.741464+0000","last_clean":"2026-02-24T22:31:14.741464+0000","last_became_active":"2026-02-24T22:15:25.827265+0000","last_became_peered":"2026-02-24T22:15:25.827265+0000","last_unstale":"2026-02-24T22:31:14.741464+0000","last_undegraded":"2026-02-24T22:31:14.741464+0000","last_fullsized":"2026-02-24T22:31:14.741464+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:17:29.198425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.a","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.735221+0000","last_change":"2026-02-24T22:15:22.817325+0000","last_active":"2026-02-24T22:30:39.735221+0000","last_peered":"2026-02-24T22:30:39.735221+0000","last_clean":"2026-02-24T22:30:39.735221+0000","last_became_active":"2026-02-24T22:15:22.816702+0000","last_became_peered":"2026-02-24T22:15:22.816702+0000","last_unstale":"2026-02-24T22:30:39.735221+0000","last_undegraded":"2026-02-24T22:30:39.735221+0000","last_fullsized":"2026-02-24T22:30:39.735221+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:48:16.071491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.15","version":"307'6","reported_seq":82,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.735250+0000","last_change":"2026-02-24T22:13:43.761758+0000","last_active":"2026-02-24T22:30:39.735250+0000","last_peered":"2026-02-24T22:30:39.735250+0000","last_clean":"2026-02-24T22:30:39.735250+0000","last_became_active":"2026-02-24T22:13:43.760892+0000","last_became_peered":"2026-02-24T22:13:43.760892+0000","last_unstale":"2026-02-24T22:30:39.735250+0000","last_undegraded":"2026-02-24T22:30:39.735250+0000","last_fullsized":"2026-02-24T22:30:39.735250+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:07:44.028035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":37,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.9","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.743246+0000","last_change":"2026-02-24T22:23:29.198300+0000","last_active":"2026-02-24T22:32:09.743246+0000","last_peered":"2026-02-24T22:32:09.743246+0000","last_clean":"2026-02-24T22:32:09.743246+0000","last_became_active":"2026-02-24T22:23:29.197357+0000","last_became_peered":"2026-02-24T22:23:29.197357+0000","last_unstale":"2026-02-24T22:32:09.743246+0000","last_undegraded":"2026-02-24T22:32:09.743246+0000","last_fullsized":"2026-02-24T22:32:09.743246+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:58:07.606981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.a","version":"312'3","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:54.752765+0000","last_change":"2026-02-24T22:15:25.827469+0000","last_active":"2026-02-24T22:32:54.752765+0000","last_peered":"2026-02-24T22:32:54.752765+0000","last_clean":"2026-02-24T22:32:54.752765+0000","last_became_active":"2026-02-24T22:15:25.827055+0000","last_became_peered":"2026-02-24T22:15:25.827055+0000","last_unstale":"2026-02-24T22:32:54.752765+0000","last_undegraded":"2026-02-24T22:32:54.752765+0000","last_fullsized":"2026-02-24T22:32:54.752765+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:35:15.420239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.b","version":"312'3","reported_seq":43,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:24.753235+0000","last_change":"2026-02-24T22:15:22.817894+0000","last_active":"2026-02-24T22:33:24.753235+0000","last_peered":"2026-02-24T22:33:24.753235+0000","last_clean":"2026-02-24T22:33:24.753235+0000","last_became_active":"2026-02-24T22:15:22.817498+0000","last_became_peered":"2026-02-24T22:15:22.817498+0000","last_unstale":"2026-02-24T22:33:24.753235+0000","last_undegraded":"2026-02-24T22:33:24.753235+0000","last_fullsized":"2026-02-24T22:33:24.753235+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:08:45.124818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.14","version":"312'10","reported_seq":90,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:29.732980+0000","last_change":"2026-02-24T22:13:43.762482+0000","last_active":"2026-02-24T22:32:29.732980+0000","last_peered":"2026-02-24T22:32:29.732980+0000","last_clean":"2026-02-24T22:32:29.732980+0000","last_became_active":"2026-02-24T22:13:43.762341+0000","last_became_peered":"2026-02-24T22:13:43.762341+0000","last_unstale":"2026-02-24T22:32:29.732980+0000","last_undegraded":"2026-02-24T22:32:29.732980+0000","last_fullsized":"2026-02-24T22:32:29.732980+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":46,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.e","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:29.737880+0000","last_change":"2026-02-24T22:23:29.215664+0000","last_active":"2026-02-24T22:32:29.737880+0000","last_peered":"2026-02-24T22:32:29.737880+0000","last_clean":"2026-02-24T22:32:29.737880+0000","last_became_active":"2026-02-24T22:23:29.215425+0000","last_became_peered":"2026-02-24T22:23:29.215425+0000","last_unstale":"2026-02-24T22:32:29.737880+0000","last_undegraded":"2026-02-24T22:32:29.737880+0000","last_fullsized":"2026-02-24T22:32:29.737880+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.d","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.741498+0000","last_change":"2026-02-24T22:15:25.829241+0000","last_active":"2026-02-24T22:31:14.741498+0000","last_peered":"2026-02-24T22:31:14.741498+0000","last_clean":"2026-02-24T22:31:14.741498+0000","last_became_active":"2026-02-24T22:15:25.829109+0000","last_became_peered":"2026-02-24T22:15:25.829109+0000","last_unstale":"2026-02-24T22:31:14.741498+0000","last_undegraded":"2026-02-24T22:31:14.741498+0000","last_fullsized":"2026-02-24T22:31:14.741498+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:38:08.330741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.c","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.721310+0000","last_change":"2026-02-24T22:15:22.815624+0000","last_active":"2026-02-24T22:31:09.721310+0000","last_peered":"2026-02-24T22:31:09.721310+0000","last_clean":"2026-02-24T22:31:09.721310+0000","last_became_active":"2026-02-24T22:15:22.815364+0000","last_became_peered":"2026-02-24T22:15:22.815364+0000","last_unstale":"2026-02-24T22:31:09.721310+0000","last_undegraded":"2026-02-24T22:31:09.721310+0000","last_fullsized":"2026-02-24T22:31:09.721310+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:21:36.820405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.13","version":"312'11","reported_seq":96,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.729057+0000","last_change":"2026-02-24T22:13:43.760988+0000","last_active":"2026-02-24T22:32:24.729057+0000","last_peered":"2026-02-24T22:32:24.729057+0000","last_clean":"2026-02-24T22:32:24.729057+0000","last_became_active":"2026-02-24T22:13:43.760714+0000","last_became_peered":"2026-02-24T22:13:43.760714+0000","last_unstale":"2026-02-24T22:32:24.729057+0000","last_undegraded":"2026-02-24T22:32:24.729057+0000","last_fullsized":"2026-02-24T22:32:24.729057+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":61,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"82.f","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:29.745312+0000","last_change":"2026-02-24T22:23:29.199754+0000","last_active":"2026-02-24T22:32:29.745312+0000","last_peered":"2026-02-24T22:32:29.745312+0000","last_clean":"2026-02-24T22:32:29.745312+0000","last_became_active":"2026-02-24T22:23:29.199538+0000","last_became_peered":"2026-02-24T22:23:29.199538+0000","last_unstale":"2026-02-24T22:32:29.745312+0000","last_undegraded":"2026-02-24T22:32:29.745312+0000","last_fullsized":"2026-02-24T22:32:29.745312+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.c","version":"312'11","reported_seq":58,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.692318+0000","last_change":"2026-02-24T22:15:25.827267+0000","last_active":"2026-02-24T22:34:25.692318+0000","last_peered":"2026-02-24T22:34:25.692318+0000","last_clean":"2026-02-24T22:34:25.692318+0000","last_became_active":"2026-02-24T22:15:25.826430+0000","last_became_peered":"2026-02-24T22:15:25.826430+0000","last_unstale":"2026-02-24T22:34:25.692318+0000","last_undegraded":"2026-02-24T22:34:25.692318+0000","last_fullsized":"2026-02-24T22:34:25.692318+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:49:19.806352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":31746,"num_write":16,"num_write_kb":16384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.d","version":"312'5","reported_seq":52,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.807220+0000","last_change":"2026-02-24T22:15:22.813197+0000","last_active":"2026-02-24T22:34:25.807220+0000","last_peered":"2026-02-24T22:34:25.807220+0000","last_clean":"2026-02-24T22:34:25.807220+0000","last_became_active":"2026-02-24T22:15:22.813062+0000","last_became_peered":"2026-02-24T22:15:22.813062+0000","last_unstale":"2026-02-24T22:34:25.807220+0000","last_undegraded":"2026-02-24T22:34:25.807220+0000","last_fullsized":"2026-02-24T22:34:25.807220+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:08:36.468537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":10,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.12","version":"312'20","reported_seq":118,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.729041+0000","last_change":"2026-02-24T22:13:43.760420+0000","last_active":"2026-02-24T22:32:24.729041+0000","last_peered":"2026-02-24T22:32:24.729041+0000","last_clean":"2026-02-24T22:32:24.729041+0000","last_became_active":"2026-02-24T22:13:43.760196+0000","last_became_peered":"2026-02-24T22:13:43.760196+0000","last_unstale":"2026-02-24T22:32:24.729041+0000","last_undegraded":"2026-02-24T22:32:24.729041+0000","last_fullsized":"2026-02-24T22:32:24.729041+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":106,"num_write":52,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"82.c","version":"310'1","reported_seq":23,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:29.750029+0000","last_change":"2026-02-24T22:23:29.201743+0000","last_active":"2026-02-24T22:32:29.750029+0000","last_peered":"2026-02-24T22:32:29.750029+0000","last_clean":"2026-02-24T22:32:29.750029+0000","last_became_active":"2026-02-24T22:23:29.201545+0000","last_became_peered":"2026-02-24T22:23:29.201545+0000","last_unstale":"2026-02-24T22:32:29.750029+0000","last_undegraded":"2026-02-24T22:32:29.750029+0000","last_fullsized":"2026-02-24T22:32:29.750029+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"81.f","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.741445+0000","last_change":"2026-02-24T22:15:25.825325+0000","last_active":"2026-02-24T22:31:19.741445+0000","last_peered":"2026-02-24T22:31:19.741445+0000","last_clean":"2026-02-24T22:31:19.741445+0000","last_became_active":"2026-02-24T22:15:25.824862+0000","last_became_peered":"2026-02-24T22:15:25.824862+0000","last_unstale":"2026-02-24T22:31:19.741445+0000","last_undegraded":"2026-02-24T22:31:19.741445+0000","last_fullsized":"2026-02-24T22:31:19.741445+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:16:51.320757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.e","version":"312'5","reported_seq":54,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.776012+0000","last_change":"2026-02-24T22:15:22.817941+0000","last_active":"2026-02-24T22:34:25.776012+0000","last_peered":"2026-02-24T22:34:25.776012+0000","last_clean":"2026-02-24T22:34:25.776012+0000","last_became_active":"2026-02-24T22:15:22.817295+0000","last_became_peered":"2026-02-24T22:15:22.817295+0000","last_unstale":"2026-02-24T22:34:25.776012+0000","last_undegraded":"2026-02-24T22:34:25.776012+0000","last_fullsized":"2026-02-24T22:34:25.776012+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:37:42.904886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":25602,"num_write":5,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.11","version":"312'11","reported_seq":93,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.746653+0000","last_change":"2026-02-24T22:13:43.760497+0000","last_active":"2026-02-24T22:32:24.746653+0000","last_peered":"2026-02-24T22:32:24.746653+0000","last_clean":"2026-02-24T22:32:24.746653+0000","last_became_active":"2026-02-24T22:13:43.759864+0000","last_became_peered":"2026-02-24T22:13:43.759864+0000","last_unstale":"2026-02-24T22:32:24.746653+0000","last_undegraded":"2026-02-24T22:32:24.746653+0000","last_fullsized":"2026-02-24T22:32:24.746653+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":53,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.d","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:29.737869+0000","last_change":"2026-02-24T22:23:29.214699+0000","last_active":"2026-02-24T22:32:29.737869+0000","last_peered":"2026-02-24T22:32:29.737869+0000","last_clean":"2026-02-24T22:32:29.737869+0000","last_became_active":"2026-02-24T22:23:29.214432+0000","last_became_peered":"2026-02-24T22:23:29.214432+0000","last_unstale":"2026-02-24T22:32:29.737869+0000","last_undegraded":"2026-02-24T22:32:29.737869+0000","last_fullsized":"2026-02-24T22:32:29.737869+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.e","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.738719+0000","last_change":"2026-02-24T22:15:25.827425+0000","last_active":"2026-02-24T22:31:14.738719+0000","last_peered":"2026-02-24T22:31:14.738719+0000","last_clean":"2026-02-24T22:31:14.738719+0000","last_became_active":"2026-02-24T22:15:25.827110+0000","last_became_peered":"2026-02-24T22:15:25.827110+0000","last_unstale":"2026-02-24T22:31:14.738719+0000","last_undegraded":"2026-02-24T22:31:14.738719+0000","last_fullsized":"2026-02-24T22:31:14.738719+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:25:06.342633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.f","version":"312'8","reported_seq":58,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.821904+0000","last_change":"2026-02-24T22:15:22.816168+0000","last_active":"2026-02-24T22:34:25.821904+0000","last_peered":"2026-02-24T22:34:25.821904+0000","last_clean":"2026-02-24T22:34:25.821904+0000","last_became_active":"2026-02-24T22:15:22.815819+0000","last_became_peered":"2026-02-24T22:15:22.815819+0000","last_unstale":"2026-02-24T22:34:25.821904+0000","last_undegraded":"2026-02-24T22:34:25.821904+0000","last_fullsized":"2026-02-24T22:34:25.821904+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:39:39.503227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":26626,"num_write":13,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.10","version":"312'31","reported_seq":153,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.746632+0000","last_change":"2026-02-24T22:13:43.760545+0000","last_active":"2026-02-24T22:32:24.746632+0000","last_peered":"2026-02-24T22:32:24.746632+0000","last_clean":"2026-02-24T22:32:24.746632+0000","last_became_active":"2026-02-24T22:13:43.759855+0000","last_became_peered":"2026-02-24T22:13:43.759855+0000","last_unstale":"2026-02-24T22:32:24.746632+0000","last_undegraded":"2026-02-24T22:32:24.746632+0000","last_fullsized":"2026-02-24T22:32:24.746632+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":272,"num_read_kb":204,"num_write":72,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.12","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.725516+0000","last_change":"2026-02-24T22:23:29.212745+0000","last_active":"2026-02-24T22:30:29.725516+0000","last_peered":"2026-02-24T22:30:29.725516+0000","last_clean":"2026-02-24T22:30:29.725516+0000","last_became_active":"2026-02-24T22:23:29.212534+0000","last_became_peered":"2026-02-24T22:23:29.212534+0000","last_unstale":"2026-02-24T22:30:29.725516+0000","last_undegraded":"2026-02-24T22:30:29.725516+0000","last_fullsized":"2026-02-24T22:30:29.725516+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:50:08.072181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.11","version":"312'5","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.471093+0000","last_change":"2026-02-24T22:15:25.827242+0000","last_active":"2026-02-24T22:34:25.471093+0000","last_peered":"2026-02-24T22:34:25.471093+0000","last_clean":"2026-02-24T22:34:25.471093+0000","last_became_active":"2026-02-24T22:15:25.826517+0000","last_became_peered":"2026-02-24T22:15:25.826517+0000","last_unstale":"2026-02-24T22:34:25.471093+0000","last_undegraded":"2026-02-24T22:34:25.471093+0000","last_fullsized":"2026-02-24T22:34:25.471093+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:24:40.732721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":10,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.10","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.741867+0000","last_change":"2026-02-24T22:15:22.810414+0000","last_active":"2026-02-24T22:31:44.741867+0000","last_peered":"2026-02-24T22:31:44.741867+0000","last_clean":"2026-02-24T22:31:44.741867+0000","last_became_active":"2026-02-24T22:15:22.810000+0000","last_became_peered":"2026-02-24T22:15:22.810000+0000","last_unstale":"2026-02-24T22:31:44.741867+0000","last_undegraded":"2026-02-24T22:31:44.741867+0000","last_fullsized":"2026-02-24T22:31:44.741867+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:21:42.629513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"79.f","version":"312'11","reported_seq":95,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742413+0000","last_change":"2026-02-24T22:13:43.755156+0000","last_active":"2026-02-24T22:31:49.742413+0000","last_peered":"2026-02-24T22:31:49.742413+0000","last_clean":"2026-02-24T22:31:49.742413+0000","last_became_active":"2026-02-24T22:13:43.754602+0000","last_became_peered":"2026-02-24T22:13:43.754602+0000","last_unstale":"2026-02-24T22:31:49.742413+0000","last_undegraded":"2026-02-24T22:31:49.742413+0000","last_fullsized":"2026-02-24T22:31:49.742413+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:13:53.006921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":67,"num_write":27,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.13","version":"310'1","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.735479+0000","last_change":"2026-02-24T22:23:29.198937+0000","last_active":"2026-02-24T22:30:29.735479+0000","last_peered":"2026-02-24T22:30:29.735479+0000","last_clean":"2026-02-24T22:30:29.735479+0000","last_became_active":"2026-02-24T22:23:29.198627+0000","last_became_peered":"2026-02-24T22:23:29.198627+0000","last_unstale":"2026-02-24T22:30:29.735479+0000","last_undegraded":"2026-02-24T22:30:29.735479+0000","last_fullsized":"2026-02-24T22:30:29.735479+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:28:29.061001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.10","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744722+0000","last_change":"2026-02-24T22:15:25.827252+0000","last_active":"2026-02-24T22:31:49.744722+0000","last_peered":"2026-02-24T22:31:49.744722+0000","last_clean":"2026-02-24T22:31:49.744722+0000","last_became_active":"2026-02-24T22:15:25.826606+0000","last_became_peered":"2026-02-24T22:15:25.826606+0000","last_unstale":"2026-02-24T22:31:49.744722+0000","last_undegraded":"2026-02-24T22:31:49.744722+0000","last_fullsized":"2026-02-24T22:31:49.744722+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:30:33.376941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.11","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.741885+0000","last_change":"2026-02-24T22:15:22.810459+0000","last_active":"2026-02-24T22:31:44.741885+0000","last_peered":"2026-02-24T22:31:44.741885+0000","last_clean":"2026-02-24T22:31:44.741885+0000","last_became_active":"2026-02-24T22:15:22.809791+0000","last_became_peered":"2026-02-24T22:15:22.809791+0000","last_unstale":"2026-02-24T22:31:44.741885+0000","last_undegraded":"2026-02-24T22:31:44.741885+0000","last_fullsized":"2026-02-24T22:31:44.741885+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:37:39.655379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"79.e","version":"312'23","reported_seq":126,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.741917+0000","last_change":"2026-02-24T22:13:43.756185+0000","last_active":"2026-02-24T22:31:44.741917+0000","last_peered":"2026-02-24T22:31:44.741917+0000","last_clean":"2026-02-24T22:31:44.741917+0000","last_became_active":"2026-02-24T22:13:43.755868+0000","last_became_peered":"2026-02-24T22:13:43.755868+0000","last_unstale":"2026-02-24T22:31:44.741917+0000","last_undegraded":"2026-02-24T22:31:44.741917+0000","last_fullsized":"2026-02-24T22:31:44.741917+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:56:04.041994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":169,"num_read_kb":112,"num_write":56,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.10","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.734148+0000","last_change":"2026-02-24T22:23:29.212895+0000","last_active":"2026-02-24T22:30:29.734148+0000","last_peered":"2026-02-24T22:30:29.734148+0000","last_clean":"2026-02-24T22:30:29.734148+0000","last_became_active":"2026-02-24T22:23:29.212665+0000","last_became_peered":"2026-02-24T22:23:29.212665+0000","last_unstale":"2026-02-24T22:30:29.734148+0000","last_undegraded":"2026-02-24T22:30:29.734148+0000","last_fullsized":"2026-02-24T22:30:29.734148+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:30:27.871463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.13","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.740862+0000","last_change":"2026-02-24T22:15:25.826171+0000","last_active":"2026-02-24T22:31:49.740862+0000","last_peered":"2026-02-24T22:31:49.740862+0000","last_clean":"2026-02-24T22:31:49.740862+0000","last_became_active":"2026-02-24T22:15:25.825862+0000","last_became_peered":"2026-02-24T22:15:25.825862+0000","last_unstale":"2026-02-24T22:31:49.740862+0000","last_undegraded":"2026-02-24T22:31:49.740862+0000","last_fullsized":"2026-02-24T22:31:49.740862+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:02:37.431216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"80.12","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.744628+0000","last_change":"2026-02-24T22:15:22.812712+0000","last_active":"2026-02-24T22:31:44.744628+0000","last_peered":"2026-02-24T22:31:44.744628+0000","last_clean":"2026-02-24T22:31:44.744628+0000","last_became_active":"2026-02-24T22:15:22.812405+0000","last_became_peered":"2026-02-24T22:15:22.812405+0000","last_unstale":"2026-02-24T22:31:44.744628+0000","last_undegraded":"2026-02-24T22:31:44.744628+0000","last_fullsized":"2026-02-24T22:31:44.744628+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:36:00.971366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.d","version":"312'20","reported_seq":113,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.733217+0000","last_change":"2026-02-24T22:13:43.761593+0000","last_active":"2026-02-24T22:31:44.733217+0000","last_peered":"2026-02-24T22:31:44.733217+0000","last_clean":"2026-02-24T22:31:44.733217+0000","last_became_active":"2026-02-24T22:13:43.761339+0000","last_became_peered":"2026-02-24T22:13:43.761339+0000","last_unstale":"2026-02-24T22:31:44.733217+0000","last_undegraded":"2026-02-24T22:31:44.733217+0000","last_fullsized":"2026-02-24T22:31:44.733217+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:02:40.622911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":126,"num_read_kb":76,"num_write":49,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"82.11","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.720712+0000","last_change":"2026-02-24T22:23:29.214793+0000","last_active":"2026-02-24T22:30:29.720712+0000","last_peered":"2026-02-24T22:30:29.720712+0000","last_clean":"2026-02-24T22:30:29.720712+0000","last_became_active":"2026-02-24T22:23:29.214167+0000","last_became_peered":"2026-02-24T22:23:29.214167+0000","last_unstale":"2026-02-24T22:30:29.720712+0000","last_undegraded":"2026-02-24T22:30:29.720712+0000","last_fullsized":"2026-02-24T22:30:29.720712+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:21:51.443877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.12","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744756+0000","last_change":"2026-02-24T22:15:25.826465+0000","last_active":"2026-02-24T22:31:49.744756+0000","last_peered":"2026-02-24T22:31:49.744756+0000","last_clean":"2026-02-24T22:31:49.744756+0000","last_became_active":"2026-02-24T22:15:25.826102+0000","last_became_peered":"2026-02-24T22:15:25.826102+0000","last_unstale":"2026-02-24T22:31:49.744756+0000","last_undegraded":"2026-02-24T22:31:49.744756+0000","last_fullsized":"2026-02-24T22:31:49.744756+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:22:01.000780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.13","version":"312'3","reported_seq":43,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:04.760554+0000","last_change":"2026-02-24T22:15:22.812393+0000","last_active":"2026-02-24T22:34:04.760554+0000","last_peered":"2026-02-24T22:34:04.760554+0000","last_clean":"2026-02-24T22:34:04.760554+0000","last_became_active":"2026-02-24T22:15:22.811927+0000","last_became_peered":"2026-02-24T22:15:22.811927+0000","last_unstale":"2026-02-24T22:34:04.760554+0000","last_undegraded":"2026-02-24T22:34:04.760554+0000","last_fullsized":"2026-02-24T22:34:04.760554+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:54:16.404989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.c","version":"312'4","reported_seq":74,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.724938+0000","last_change":"2026-02-24T22:13:43.762114+0000","last_active":"2026-02-24T22:31:44.724938+0000","last_peered":"2026-02-24T22:31:44.724938+0000","last_clean":"2026-02-24T22:31:44.724938+0000","last_became_active":"2026-02-24T22:13:43.761910+0000","last_became_peered":"2026-02-24T22:13:43.761910+0000","last_unstale":"2026-02-24T22:31:44.724938+0000","last_undegraded":"2026-02-24T22:31:44.724938+0000","last_fullsized":"2026-02-24T22:31:44.724938+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:51:11.166418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":15,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"82.16","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.718767+0000","last_change":"2026-02-24T22:23:29.213342+0000","last_active":"2026-02-24T22:30:44.718767+0000","last_peered":"2026-02-24T22:30:44.718767+0000","last_clean":"2026-02-24T22:30:44.718767+0000","last_became_active":"2026-02-24T22:23:29.213080+0000","last_became_peered":"2026-02-24T22:23:29.213080+0000","last_unstale":"2026-02-24T22:30:44.718767+0000","last_undegraded":"2026-02-24T22:30:44.718767+0000","last_fullsized":"2026-02-24T22:30:44.718767+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:33:29.031496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.15","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.729079+0000","last_change":"2026-02-24T22:15:25.831361+0000","last_active":"2026-02-24T22:32:24.729079+0000","last_peered":"2026-02-24T22:32:24.729079+0000","last_clean":"2026-02-24T22:32:24.729079+0000","last_became_active":"2026-02-24T22:15:25.830922+0000","last_became_peered":"2026-02-24T22:15:25.830922+0000","last_unstale":"2026-02-24T22:32:24.729079+0000","last_undegraded":"2026-02-24T22:32:24.729079+0000","last_fullsized":"2026-02-24T22:32:24.729079+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"80.14","version":"312'3","reported_seq":43,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:09.761022+0000","last_change":"2026-02-24T22:15:22.812446+0000","last_active":"2026-02-24T22:34:09.761022+0000","last_peered":"2026-02-24T22:34:09.761022+0000","last_clean":"2026-02-24T22:34:09.761022+0000","last_became_active":"2026-02-24T22:15:22.811924+0000","last_became_peered":"2026-02-24T22:15:22.811924+0000","last_unstale":"2026-02-24T22:34:09.761022+0000","last_undegraded":"2026-02-24T22:34:09.761022+0000","last_fullsized":"2026-02-24T22:34:09.761022+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:18:30.397097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.b","version":"312'13","reported_seq":98,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.740368+0000","last_change":"2026-02-24T22:13:43.757072+0000","last_active":"2026-02-24T22:31:44.740368+0000","last_peered":"2026-02-24T22:31:44.740368+0000","last_clean":"2026-02-24T22:31:44.740368+0000","last_became_active":"2026-02-24T22:13:43.756596+0000","last_became_peered":"2026-02-24T22:13:43.756596+0000","last_unstale":"2026-02-24T22:31:44.740368+0000","last_undegraded":"2026-02-24T22:31:44.740368+0000","last_fullsized":"2026-02-24T22:31:44.740368+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:30:51.803180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":56,"num_write":37,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.17","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.718784+0000","last_change":"2026-02-24T22:23:29.214456+0000","last_active":"2026-02-24T22:30:44.718784+0000","last_peered":"2026-02-24T22:30:44.718784+0000","last_clean":"2026-02-24T22:30:44.718784+0000","last_became_active":"2026-02-24T22:23:29.214217+0000","last_became_peered":"2026-02-24T22:23:29.214217+0000","last_unstale":"2026-02-24T22:30:44.718784+0000","last_undegraded":"2026-02-24T22:30:44.718784+0000","last_fullsized":"2026-02-24T22:30:44.718784+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:48:46.666871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.14","version":"312'5","reported_seq":42,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.471281+0000","last_change":"2026-02-24T22:15:25.827827+0000","last_active":"2026-02-24T22:34:25.471281+0000","last_peered":"2026-02-24T22:34:25.471281+0000","last_clean":"2026-02-24T22:34:25.471281+0000","last_became_active":"2026-02-24T22:15:25.827557+0000","last_became_peered":"2026-02-24T22:15:25.827557+0000","last_unstale":"2026-02-24T22:34:25.471281+0000","last_undegraded":"2026-02-24T22:34:25.471281+0000","last_fullsized":"2026-02-24T22:34:25.471281+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:13:31.380536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":10,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.15","version":"312'5","reported_seq":48,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.909302+0000","last_change":"2026-02-24T22:15:22.812814+0000","last_active":"2026-02-24T22:34:25.909302+0000","last_peered":"2026-02-24T22:34:25.909302+0000","last_clean":"2026-02-24T22:34:25.909302+0000","last_became_active":"2026-02-24T22:15:22.812450+0000","last_became_peered":"2026-02-24T22:15:22.812450+0000","last_unstale":"2026-02-24T22:34:25.909302+0000","last_undegraded":"2026-02-24T22:34:25.909302+0000","last_fullsized":"2026-02-24T22:34:25.909302+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":10,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.a","version":"312'10","reported_seq":90,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.724683+0000","last_change":"2026-02-24T22:13:43.762418+0000","last_active":"2026-02-24T22:31:09.724683+0000","last_peered":"2026-02-24T22:31:09.724683+0000","last_clean":"2026-02-24T22:31:09.724683+0000","last_became_active":"2026-02-24T22:13:43.762226+0000","last_became_peered":"2026-02-24T22:13:43.762226+0000","last_unstale":"2026-02-24T22:31:09.724683+0000","last_undegraded":"2026-02-24T22:31:09.724683+0000","last_fullsized":"2026-02-24T22:31:09.724683+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:41:14.108185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":42,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.14","version":"309'1","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.735499+0000","last_change":"2026-02-24T22:24:01.299030+0000","last_active":"2026-02-24T22:30:29.735499+0000","last_peered":"2026-02-24T22:30:29.735499+0000","last_clean":"2026-02-24T22:30:29.735499+0000","last_became_active":"2026-02-24T22:24:01.297193+0000","last_became_peered":"2026-02-24T22:24:01.297193+0000","last_unstale":"2026-02-24T22:30:29.735499+0000","last_undegraded":"2026-02-24T22:30:29.735499+0000","last_fullsized":"2026-02-24T22:30:29.735499+0000","mapping_epoch":311,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":312,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:15:53.769966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.17","version":"312'3","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.871072+0000","last_change":"2026-02-24T22:15:25.833669+0000","last_active":"2026-02-24T22:34:25.871072+0000","last_peered":"2026-02-24T22:34:25.871072+0000","last_clean":"2026-02-24T22:34:25.871072+0000","last_became_active":"2026-02-24T22:15:25.833430+0000","last_became_peered":"2026-02-24T22:15:25.833430+0000","last_unstale":"2026-02-24T22:34:25.871072+0000","last_undegraded":"2026-02-24T22:34:25.871072+0000","last_fullsized":"2026-02-24T22:34:25.871072+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"80.16","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:19.728514+0000","last_change":"2026-02-24T22:15:22.817544+0000","last_active":"2026-02-24T22:32:19.728514+0000","last_peered":"2026-02-24T22:32:19.728514+0000","last_clean":"2026-02-24T22:32:19.728514+0000","last_became_active":"2026-02-24T22:15:22.817150+0000","last_became_peered":"2026-02-24T22:15:22.817150+0000","last_unstale":"2026-02-24T22:32:19.728514+0000","last_undegraded":"2026-02-24T22:32:19.728514+0000","last_fullsized":"2026-02-24T22:32:19.728514+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.9","version":"312'24","reported_seq":125,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.737699+0000","last_change":"2026-02-24T22:13:43.757041+0000","last_active":"2026-02-24T22:31:04.737699+0000","last_peered":"2026-02-24T22:31:04.737699+0000","last_clean":"2026-02-24T22:31:04.737699+0000","last_became_active":"2026-02-24T22:13:43.756777+0000","last_became_peered":"2026-02-24T22:13:43.756777+0000","last_unstale":"2026-02-24T22:31:04.737699+0000","last_undegraded":"2026-02-24T22:31:04.737699+0000","last_fullsized":"2026-02-24T22:31:04.737699+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:54:52.853203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":107,"num_write":60,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.15","version":"310'1","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.733776+0000","last_change":"2026-02-24T22:23:29.198331+0000","last_active":"2026-02-24T22:30:44.733776+0000","last_peered":"2026-02-24T22:30:44.733776+0000","last_clean":"2026-02-24T22:30:44.733776+0000","last_became_active":"2026-02-24T22:23:29.198000+0000","last_became_peered":"2026-02-24T22:23:29.198000+0000","last_unstale":"2026-02-24T22:30:44.733776+0000","last_undegraded":"2026-02-24T22:30:44.733776+0000","last_fullsized":"2026-02-24T22:30:44.733776+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:34:21.768724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.16","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.748803+0000","last_change":"2026-02-24T22:15:25.828675+0000","last_active":"2026-02-24T22:32:24.748803+0000","last_peered":"2026-02-24T22:32:24.748803+0000","last_clean":"2026-02-24T22:32:24.748803+0000","last_became_active":"2026-02-24T22:15:25.828066+0000","last_became_peered":"2026-02-24T22:15:25.828066+0000","last_unstale":"2026-02-24T22:32:24.748803+0000","last_undegraded":"2026-02-24T22:32:24.748803+0000","last_fullsized":"2026-02-24T22:32:24.748803+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.17","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:19.746146+0000","last_change":"2026-02-24T22:15:22.818517+0000","last_active":"2026-02-24T22:32:19.746146+0000","last_peered":"2026-02-24T22:32:19.746146+0000","last_clean":"2026-02-24T22:32:19.746146+0000","last_became_active":"2026-02-24T22:15:22.817950+0000","last_became_peered":"2026-02-24T22:15:22.817950+0000","last_unstale":"2026-02-24T22:32:19.746146+0000","last_undegraded":"2026-02-24T22:32:19.746146+0000","last_fullsized":"2026-02-24T22:32:19.746146+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.8","version":"312'26","reported_seq":136,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.737895+0000","last_change":"2026-02-24T22:13:43.761888+0000","last_active":"2026-02-24T22:31:04.737895+0000","last_peered":"2026-02-24T22:31:04.737895+0000","last_clean":"2026-02-24T22:31:04.737895+0000","last_became_active":"2026-02-24T22:13:43.761558+0000","last_became_peered":"2026-02-24T22:13:43.761558+0000","last_unstale":"2026-02-24T22:31:04.737895+0000","last_undegraded":"2026-02-24T22:31:04.737895+0000","last_fullsized":"2026-02-24T22:31:04.737895+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:22:52.093177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":152,"num_write":62,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.1a","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.723700+0000","last_change":"2026-02-24T22:23:29.213344+0000","last_active":"2026-02-24T22:30:59.723700+0000","last_peered":"2026-02-24T22:30:59.723700+0000","last_clean":"2026-02-24T22:30:59.723700+0000","last_became_active":"2026-02-24T22:23:29.212709+0000","last_became_peered":"2026-02-24T22:23:29.212709+0000","last_unstale":"2026-02-24T22:30:59.723700+0000","last_undegraded":"2026-02-24T22:30:59.723700+0000","last_fullsized":"2026-02-24T22:30:59.723700+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:26:30.998666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.19","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:04.718214+0000","last_change":"2026-02-24T22:15:25.832162+0000","last_active":"2026-02-24T22:30:04.718214+0000","last_peered":"2026-02-24T22:30:04.718214+0000","last_clean":"2026-02-24T22:30:04.718214+0000","last_became_active":"2026-02-24T22:15:25.831885+0000","last_became_peered":"2026-02-24T22:15:25.831885+0000","last_unstale":"2026-02-24T22:30:04.718214+0000","last_undegraded":"2026-02-24T22:30:04.718214+0000","last_fullsized":"2026-02-24T22:30:04.718214+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:54:09.173036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.18","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:19.745911+0000","last_change":"2026-02-24T22:15:22.810683+0000","last_active":"2026-02-24T22:32:19.745911+0000","last_peered":"2026-02-24T22:32:19.745911+0000","last_clean":"2026-02-24T22:32:19.745911+0000","last_became_active":"2026-02-24T22:15:22.810156+0000","last_became_peered":"2026-02-24T22:15:22.810156+0000","last_unstale":"2026-02-24T22:32:19.745911+0000","last_undegraded":"2026-02-24T22:32:19.745911+0000","last_fullsized":"2026-02-24T22:32:19.745911+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"79.7","version":"0'0","reported_seq":64,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.740460+0000","last_change":"2026-02-24T22:13:43.756250+0000","last_active":"2026-02-24T22:31:04.740460+0000","last_peered":"2026-02-24T22:31:04.740460+0000","last_clean":"2026-02-24T22:31:04.740460+0000","last_became_active":"2026-02-24T22:13:43.755957+0000","last_became_peered":"2026-02-24T22:13:43.755957+0000","last_unstale":"2026-02-24T22:31:04.740460+0000","last_undegraded":"2026-02-24T22:31:04.740460+0000","last_fullsized":"2026-02-24T22:31:04.740460+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:16:21.505648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.1b","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.735465+0000","last_change":"2026-02-24T22:23:29.200410+0000","last_active":"2026-02-24T22:30:59.735465+0000","last_peered":"2026-02-24T22:30:59.735465+0000","last_clean":"2026-02-24T22:30:59.735465+0000","last_became_active":"2026-02-24T22:23:29.200152+0000","last_became_peered":"2026-02-24T22:23:29.200152+0000","last_unstale":"2026-02-24T22:30:59.735465+0000","last_undegraded":"2026-02-24T22:30:59.735465+0000","last_fullsized":"2026-02-24T22:30:59.735465+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:24:51.029708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.18","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.746457+0000","last_change":"2026-02-24T22:15:25.824057+0000","last_active":"2026-02-24T22:32:24.746457+0000","last_peered":"2026-02-24T22:32:24.746457+0000","last_clean":"2026-02-24T22:32:24.746457+0000","last_became_active":"2026-02-24T22:15:25.823415+0000","last_became_peered":"2026-02-24T22:15:25.823415+0000","last_unstale":"2026-02-24T22:32:24.746457+0000","last_undegraded":"2026-02-24T22:32:24.746457+0000","last_fullsized":"2026-02-24T22:32:24.746457+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.19","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.729011+0000","last_change":"2026-02-24T22:15:22.817214+0000","last_active":"2026-02-24T22:32:24.729011+0000","last_peered":"2026-02-24T22:32:24.729011+0000","last_clean":"2026-02-24T22:32:24.729011+0000","last_became_active":"2026-02-24T22:15:22.816826+0000","last_became_peered":"2026-02-24T22:15:22.816826+0000","last_unstale":"2026-02-24T22:32:24.729011+0000","last_undegraded":"2026-02-24T22:32:24.729011+0000","last_fullsized":"2026-02-24T22:32:24.729011+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.6","version":"301'14","reported_seq":108,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.739656+0000","last_change":"2026-02-24T22:13:43.755144+0000","last_active":"2026-02-24T22:31:04.739656+0000","last_peered":"2026-02-24T22:31:04.739656+0000","last_clean":"2026-02-24T22:31:04.739656+0000","last_became_active":"2026-02-24T22:13:43.754753+0000","last_became_peered":"2026-02-24T22:13:43.754753+0000","last_unstale":"2026-02-24T22:31:04.739656+0000","last_undegraded":"2026-02-24T22:31:04.739656+0000","last_fullsized":"2026-02-24T22:31:04.739656+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:57:00.870335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":92,"num_write":34,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"82.18","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.726953+0000","last_change":"2026-02-24T22:23:29.213704+0000","last_active":"2026-02-24T22:30:44.726953+0000","last_peered":"2026-02-24T22:30:44.726953+0000","last_clean":"2026-02-24T22:30:44.726953+0000","last_became_active":"2026-02-24T22:23:29.213387+0000","last_became_peered":"2026-02-24T22:23:29.213387+0000","last_unstale":"2026-02-24T22:30:44.726953+0000","last_undegraded":"2026-02-24T22:30:44.726953+0000","last_fullsized":"2026-02-24T22:30:44.726953+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:46:11.186061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.1b","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.720738+0000","last_change":"2026-02-24T22:15:25.832293+0000","last_active":"2026-02-24T22:30:29.720738+0000","last_peered":"2026-02-24T22:30:29.720738+0000","last_clean":"2026-02-24T22:30:29.720738+0000","last_became_active":"2026-02-24T22:15:25.832097+0000","last_became_peered":"2026-02-24T22:15:25.832097+0000","last_unstale":"2026-02-24T22:30:29.720738+0000","last_undegraded":"2026-02-24T22:30:29.720738+0000","last_fullsized":"2026-02-24T22:30:29.720738+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:19:51.400688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.1a","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.720229+0000","last_change":"2026-02-24T22:15:22.819032+0000","last_active":"2026-02-24T22:30:24.720229+0000","last_peered":"2026-02-24T22:30:24.720229+0000","last_clean":"2026-02-24T22:30:24.720229+0000","last_became_active":"2026-02-24T22:15:22.818885+0000","last_became_peered":"2026-02-24T22:15:22.818885+0000","last_unstale":"2026-02-24T22:30:24.720229+0000","last_undegraded":"2026-02-24T22:30:24.720229+0000","last_fullsized":"2026-02-24T22:30:24.720229+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:15:52.421379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"79.5","version":"312'19","reported_seq":122,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:34.750747+0000","last_change":"2026-02-24T22:13:43.756043+0000","last_active":"2026-02-24T22:32:34.750747+0000","last_peered":"2026-02-24T22:32:34.750747+0000","last_clean":"2026-02-24T22:32:34.750747+0000","last_became_active":"2026-02-24T22:13:43.755495+0000","last_became_peered":"2026-02-24T22:13:43.755495+0000","last_unstale":"2026-02-24T22:32:34.750747+0000","last_undegraded":"2026-02-24T22:32:34.750747+0000","last_fullsized":"2026-02-24T22:32:34.750747+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":164,"num_read_kb":112,"num_write":47,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.19","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:49.719308+0000","last_change":"2026-02-24T22:23:29.212738+0000","last_active":"2026-02-24T22:30:49.719308+0000","last_peered":"2026-02-24T22:30:49.719308+0000","last_clean":"2026-02-24T22:30:49.719308+0000","last_became_active":"2026-02-24T22:23:29.212268+0000","last_became_peered":"2026-02-24T22:23:29.212268+0000","last_unstale":"2026-02-24T22:30:49.719308+0000","last_undegraded":"2026-02-24T22:30:49.719308+0000","last_fullsized":"2026-02-24T22:30:49.719308+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:32:25.776768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.1a","version":"312'3","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:59.748342+0000","last_change":"2026-02-24T22:15:25.826222+0000","last_active":"2026-02-24T22:32:59.748342+0000","last_peered":"2026-02-24T22:32:59.748342+0000","last_clean":"2026-02-24T22:32:59.748342+0000","last_became_active":"2026-02-24T22:15:25.825969+0000","last_became_peered":"2026-02-24T22:15:25.825969+0000","last_unstale":"2026-02-24T22:32:59.748342+0000","last_undegraded":"2026-02-24T22:32:59.748342+0000","last_fullsized":"2026-02-24T22:32:59.748342+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:29:05.133831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"80.1b","version":"312'3","reported_seq":42,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:54.750003+0000","last_change":"2026-02-24T22:15:22.816734+0000","last_active":"2026-02-24T22:32:54.750003+0000","last_peered":"2026-02-24T22:32:54.750003+0000","last_clean":"2026-02-24T22:32:54.750003+0000","last_became_active":"2026-02-24T22:15:22.816479+0000","last_became_peered":"2026-02-24T22:15:22.816479+0000","last_unstale":"2026-02-24T22:32:54.750003+0000","last_undegraded":"2026-02-24T22:32:54.750003+0000","last_fullsized":"2026-02-24T22:32:54.750003+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:57:39.996802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.4","version":"0'0","reported_seq":71,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:29.749325+0000","last_change":"2026-02-24T22:14:00.742004+0000","last_active":"2026-02-24T22:32:29.749325+0000","last_peered":"2026-02-24T22:32:29.749325+0000","last_clean":"2026-02-24T22:32:29.749325+0000","last_became_active":"2026-02-24T22:14:00.741839+0000","last_became_peered":"2026-02-24T22:14:00.741839+0000","last_unstale":"2026-02-24T22:32:29.749325+0000","last_undegraded":"2026-02-24T22:32:29.749325+0000","last_fullsized":"2026-02-24T22:32:29.749325+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":293,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"82.1e","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.727290+0000","last_change":"2026-02-24T22:23:29.215779+0000","last_active":"2026-02-24T22:31:34.727290+0000","last_peered":"2026-02-24T22:31:34.727290+0000","last_clean":"2026-02-24T22:31:34.727290+0000","last_became_active":"2026-02-24T22:23:29.215604+0000","last_became_peered":"2026-02-24T22:23:29.215604+0000","last_unstale":"2026-02-24T22:31:34.727290+0000","last_undegraded":"2026-02-24T22:31:34.727290+0000","last_fullsized":"2026-02-24T22:31:34.727290+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:58:56.390417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.1d","version":"312'3","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:59.750434+0000","last_change":"2026-02-24T22:15:25.828517+0000","last_active":"2026-02-24T22:32:59.750434+0000","last_peered":"2026-02-24T22:32:59.750434+0000","last_clean":"2026-02-24T22:32:59.750434+0000","last_became_active":"2026-02-24T22:15:25.828335+0000","last_became_peered":"2026-02-24T22:15:25.828335+0000","last_unstale":"2026-02-24T22:32:59.750434+0000","last_undegraded":"2026-02-24T22:32:59.750434+0000","last_fullsized":"2026-02-24T22:32:59.750434+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:18:15.606086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.1c","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.733619+0000","last_change":"2026-02-24T22:15:22.816853+0000","last_active":"2026-02-24T22:30:24.733619+0000","last_peered":"2026-02-24T22:30:24.733619+0000","last_clean":"2026-02-24T22:30:24.733619+0000","last_became_active":"2026-02-24T22:15:22.816548+0000","last_became_peered":"2026-02-24T22:15:22.816548+0000","last_unstale":"2026-02-24T22:30:24.733619+0000","last_undegraded":"2026-02-24T22:30:24.733619+0000","last_fullsized":"2026-02-24T22:30:24.733619+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:00:55.883988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.3","version":"312'6","reported_seq":78,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:29.745283+0000","last_change":"2026-02-24T22:13:43.756810+0000","last_active":"2026-02-24T22:32:29.745283+0000","last_peered":"2026-02-24T22:32:29.745283+0000","last_clean":"2026-02-24T22:32:29.745283+0000","last_became_active":"2026-02-24T22:13:43.756408+0000","last_became_peered":"2026-02-24T22:13:43.756408+0000","last_unstale":"2026-02-24T22:32:29.745283+0000","last_undegraded":"2026-02-24T22:32:29.745283+0000","last_fullsized":"2026-02-24T22:32:29.745283+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":20,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.1f","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744790+0000","last_change":"2026-02-24T22:23:29.199410+0000","last_active":"2026-02-24T22:31:49.744790+0000","last_peered":"2026-02-24T22:31:49.744790+0000","last_clean":"2026-02-24T22:31:49.744790+0000","last_became_active":"2026-02-24T22:23:29.199179+0000","last_became_peered":"2026-02-24T22:23:29.199179+0000","last_unstale":"2026-02-24T22:31:49.744790+0000","last_undegraded":"2026-02-24T22:31:49.744790+0000","last_fullsized":"2026-02-24T22:31:49.744790+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:21:23.823724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.1c","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.725548+0000","last_change":"2026-02-24T22:15:25.834316+0000","last_active":"2026-02-24T22:30:29.725548+0000","last_peered":"2026-02-24T22:30:29.725548+0000","last_clean":"2026-02-24T22:30:29.725548+0000","last_became_active":"2026-02-24T22:15:25.834072+0000","last_became_peered":"2026-02-24T22:15:25.834072+0000","last_unstale":"2026-02-24T22:30:29.725548+0000","last_undegraded":"2026-02-24T22:30:29.725548+0000","last_fullsized":"2026-02-24T22:30:29.725548+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:45:39.925744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"80.1d","version":"312'2","reported_seq":40,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:54.750046+0000","last_change":"2026-02-24T22:15:22.817173+0000","last_active":"2026-02-24T22:32:54.750046+0000","last_peered":"2026-02-24T22:32:54.750046+0000","last_clean":"2026-02-24T22:32:54.750046+0000","last_became_active":"2026-02-24T22:15:22.816615+0000","last_became_peered":"2026-02-24T22:15:22.816615+0000","last_unstale":"2026-02-24T22:32:54.750046+0000","last_undegraded":"2026-02-24T22:32:54.750046+0000","last_fullsized":"2026-02-24T22:32:54.750046+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:25:34.323299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.2","version":"312'5","reported_seq":78,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:29.746992+0000","last_change":"2026-02-24T22:13:43.755888+0000","last_active":"2026-02-24T22:32:29.746992+0000","last_peered":"2026-02-24T22:32:29.746992+0000","last_clean":"2026-02-24T22:32:29.746992+0000","last_became_active":"2026-02-24T22:13:43.755241+0000","last_became_peered":"2026-02-24T22:13:43.755241+0000","last_unstale":"2026-02-24T22:32:29.746992+0000","last_undegraded":"2026-02-24T22:32:29.746992+0000","last_fullsized":"2026-02-24T22:32:29.746992+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":23,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.1c","version":"310'1","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.720361+0000","last_change":"2026-02-24T22:23:29.213715+0000","last_active":"2026-02-24T22:30:59.720361+0000","last_peered":"2026-02-24T22:30:59.720361+0000","last_clean":"2026-02-24T22:30:59.720361+0000","last_became_active":"2026-02-24T22:23:29.213490+0000","last_became_peered":"2026-02-24T22:23:29.213490+0000","last_unstale":"2026-02-24T22:30:59.720361+0000","last_undegraded":"2026-02-24T22:30:59.720361+0000","last_fullsized":"2026-02-24T22:30:59.720361+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:21:40.032070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.1f","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.747978+0000","last_change":"2026-02-24T22:15:25.828181+0000","last_active":"2026-02-24T22:32:09.747978+0000","last_peered":"2026-02-24T22:32:09.747978+0000","last_clean":"2026-02-24T22:32:09.747978+0000","last_became_active":"2026-02-24T22:15:25.827873+0000","last_became_peered":"2026-02-24T22:15:25.827873+0000","last_unstale":"2026-02-24T22:32:09.747978+0000","last_undegraded":"2026-02-24T22:32:09.747978+0000","last_fullsized":"2026-02-24T22:32:09.747978+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:19:57.290828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.1e","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:29.726743+0000","last_change":"2026-02-24T22:15:22.817535+0000","last_active":"2026-02-24T22:31:29.726743+0000","last_peered":"2026-02-24T22:31:29.726743+0000","last_clean":"2026-02-24T22:31:29.726743+0000","last_became_active":"2026-02-24T22:15:22.817337+0000","last_became_peered":"2026-02-24T22:15:22.817337+0000","last_unstale":"2026-02-24T22:31:29.726743+0000","last_undegraded":"2026-02-24T22:31:29.726743+0000","last_fullsized":"2026-02-24T22:31:29.726743+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:02:48.141978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"79.1","version":"312'23","reported_seq":126,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:29.732931+0000","last_change":"2026-02-24T22:13:43.761201+0000","last_active":"2026-02-24T22:32:29.732931+0000","last_peered":"2026-02-24T22:32:29.732931+0000","last_clean":"2026-02-24T22:32:29.732931+0000","last_became_active":"2026-02-24T22:13:43.760933+0000","last_became_peered":"2026-02-24T22:13:43.760933+0000","last_unstale":"2026-02-24T22:32:29.732931+0000","last_undegraded":"2026-02-24T22:32:29.732931+0000","last_fullsized":"2026-02-24T22:32:29.732931+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":174,"num_read_kb":113,"num_write":59,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.1d","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.723721+0000","last_change":"2026-02-24T22:23:29.215553+0000","last_active":"2026-02-24T22:30:59.723721+0000","last_peered":"2026-02-24T22:30:59.723721+0000","last_clean":"2026-02-24T22:30:59.723721+0000","last_became_active":"2026-02-24T22:23:29.215334+0000","last_became_peered":"2026-02-24T22:23:29.215334+0000","last_unstale":"2026-02-24T22:30:59.723721+0000","last_undegraded":"2026-02-24T22:30:59.723721+0000","last_fullsized":"2026-02-24T22:30:59.723721+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:01:42.978764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.1e","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.739370+0000","last_change":"2026-02-24T22:15:25.827409+0000","last_active":"2026-02-24T22:31:34.739370+0000","last_peered":"2026-02-24T22:31:34.739370+0000","last_clean":"2026-02-24T22:31:34.739370+0000","last_became_active":"2026-02-24T22:15:25.827124+0000","last_became_peered":"2026-02-24T22:15:25.827124+0000","last_unstale":"2026-02-24T22:31:34.739370+0000","last_undegraded":"2026-02-24T22:31:34.739370+0000","last_fullsized":"2026-02-24T22:31:34.739370+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:44:21.789639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"80.1f","version":"312'3","reported_seq":44,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:09.739954+0000","last_change":"2026-02-24T22:15:22.816227+0000","last_active":"2026-02-24T22:34:09.739954+0000","last_peered":"2026-02-24T22:34:09.739954+0000","last_clean":"2026-02-24T22:34:09.739954+0000","last_became_active":"2026-02-24T22:15:22.815938+0000","last_became_peered":"2026-02-24T22:15:22.815938+0000","last_unstale":"2026-02-24T22:34:09.739954+0000","last_undegraded":"2026-02-24T22:34:09.739954+0000","last_fullsized":"2026-02-24T22:34:09.739954+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:40:15.210510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.0","version":"0'0","reported_seq":65,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.745118+0000","last_change":"2026-02-24T22:13:43.755907+0000","last_active":"2026-02-24T22:31:49.745118+0000","last_peered":"2026-02-24T22:31:49.745118+0000","last_clean":"2026-02-24T22:31:49.745118+0000","last_became_active":"2026-02-24T22:13:43.755266+0000","last_became_peered":"2026-02-24T22:13:43.755266+0000","last_unstale":"2026-02-24T22:31:49.745118+0000","last_undegraded":"2026-02-24T22:31:49.745118+0000","last_fullsized":"2026-02-24T22:31:49.745118+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:42:48.314773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"312'276","reported_seq":990,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.099244+0000","last_change":"2026-02-24T22:34:44.099244+0000","last_active":"2026-02-24T22:34:44.099244+0000","last_peered":"2026-02-24T22:34:44.099244+0000","last_clean":"2026-02-24T22:34:44.099244+0000","last_became_active":"2026-02-24T22:05:16.192589+0000","last_became_peered":"2026-02-24T22:05:16.192589+0000","last_unstale":"2026-02-24T22:34:44.099244+0000","last_undegraded":"2026-02-24T22:34:44.099244+0000","last_fullsized":"2026-02-24T22:34:44.099244+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'276","last_scrub_stamp":"2026-02-24T22:34:44.099189+0000","last_deep_scrub":"312'276","last_deep_scrub_stamp":"2026-02-24T22:34:44.099189+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.099189+0000","objects_scrubbed":5,"log_size":76,"log_dups_size":200,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:12:08.217874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":433,"num_read_kb":342,"num_write":1269,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"312'220","reported_seq":930,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.919413+0000","last_change":"2026-02-24T22:34:46.919413+0000","last_active":"2026-02-24T22:34:46.919413+0000","last_peered":"2026-02-24T22:34:46.919413+0000","last_clean":"2026-02-24T22:34:46.919413+0000","last_became_active":"2026-02-24T22:05:18.192203+0000","last_became_peered":"2026-02-24T22:05:18.192203+0000","last_unstale":"2026-02-24T22:34:46.919413+0000","last_undegraded":"2026-02-24T22:34:46.919413+0000","last_fullsized":"2026-02-24T22:34:46.919413+0000","mapping_epoch":24,"log_start":"312'197","ondisk_log_start":"312'197","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'220","last_scrub_stamp":"2026-02-24T22:34:46.919375+0000","last_deep_scrub":"312'220","last_deep_scrub_stamp":"2026-02-24T22:34:46.919375+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.919375+0000","objects_scrubbed":23,"log_size":23,"log_dups_size":197,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:53:24.900487+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":780,"num_read_kb":643,"num_write":282,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"312'380","reported_seq":1197,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.032775+0000","last_change":"2026-02-24T22:34:36.032775+0000","last_active":"2026-02-24T22:34:36.032775+0000","last_peered":"2026-02-24T22:34:36.032775+0000","last_clean":"2026-02-24T22:34:36.032775+0000","last_became_active":"2026-02-24T22:05:16.181379+0000","last_became_peered":"2026-02-24T22:05:16.181379+0000","last_unstale":"2026-02-24T22:34:36.032775+0000","last_undegraded":"2026-02-24T22:34:36.032775+0000","last_fullsized":"2026-02-24T22:34:36.032775+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'380","last_scrub_stamp":"2026-02-24T22:34:36.032715+0000","last_deep_scrub":"312'380","last_deep_scrub_stamp":"2026-02-24T22:34:36.032715+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.032715+0000","objects_scrubbed":2,"log_size":80,"log_dups_size":300,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:22:39.124170+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":922,"num_read_kb":611,"num_write":1540,"num_write_kb":5240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"312'487","reported_seq":1294,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.222322+0000","last_change":"2026-02-24T22:34:47.222322+0000","last_active":"2026-02-24T22:34:47.222322+0000","last_peered":"2026-02-24T22:34:47.222322+0000","last_clean":"2026-02-24T22:34:47.222322+0000","last_became_active":"2026-02-24T22:05:18.190689+0000","last_became_peered":"2026-02-24T22:05:18.190689+0000","last_unstale":"2026-02-24T22:34:47.222322+0000","last_undegraded":"2026-02-24T22:34:47.222322+0000","last_fullsized":"2026-02-24T22:34:47.222322+0000","mapping_epoch":24,"log_start":"312'400","ondisk_log_start":"312'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'487","last_scrub_stamp":"2026-02-24T22:34:47.222271+0000","last_deep_scrub":"312'487","last_deep_scrub_stamp":"2026-02-24T22:34:47.222271+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.222271+0000","objects_scrubbed":28,"log_size":87,"log_dups_size":400,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:40:22.630753+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":2017,"num_read_kb":1670,"num_write":788,"num_write_kb":502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"312'453","reported_seq":1248,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.730366+0000","last_change":"2026-02-24T22:34:38.730366+0000","last_active":"2026-02-24T22:34:38.730366+0000","last_peered":"2026-02-24T22:34:38.730366+0000","last_clean":"2026-02-24T22:34:38.730366+0000","last_became_active":"2026-02-24T22:05:16.180681+0000","last_became_peered":"2026-02-24T22:05:16.180681+0000","last_unstale":"2026-02-24T22:34:38.730366+0000","last_undegraded":"2026-02-24T22:34:38.730366+0000","last_fullsized":"2026-02-24T22:34:38.730366+0000","mapping_epoch":22,"log_start":"312'400","ondisk_log_start":"312'400","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'453","last_scrub_stamp":"2026-02-24T22:34:38.730307+0000","last_deep_scrub":"312'453","last_deep_scrub_stamp":"2026-02-24T22:34:38.730307+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.730307+0000","objects_scrubbed":5,"log_size":53,"log_dups_size":400,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:09:15.011742+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242910,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":903,"num_read_kb":6172,"num_write":1607,"num_write_kb":5459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"312'102","reported_seq":758,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.416387+0000","last_change":"2026-02-24T22:34:48.416387+0000","last_active":"2026-02-24T22:34:48.416387+0000","last_peered":"2026-02-24T22:34:48.416387+0000","last_clean":"2026-02-24T22:34:48.416387+0000","last_became_active":"2026-02-24T22:05:18.190727+0000","last_became_peered":"2026-02-24T22:05:18.190727+0000","last_unstale":"2026-02-24T22:34:48.416387+0000","last_undegraded":"2026-02-24T22:34:48.416387+0000","last_fullsized":"2026-02-24T22:34:48.416387+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'102","last_scrub_stamp":"2026-02-24T22:34:48.416304+0000","last_deep_scrub":"312'102","last_deep_scrub_stamp":"2026-02-24T22:34:48.416304+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.416304+0000","objects_scrubbed":22,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:05:48.774357+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":268,"num_read_kb":220,"num_write":112,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"312'294","reported_seq":986,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.956244+0000","last_change":"2026-02-24T22:34:43.956244+0000","last_active":"2026-02-24T22:34:43.956244+0000","last_peered":"2026-02-24T22:34:43.956244+0000","last_clean":"2026-02-24T22:34:43.956244+0000","last_became_active":"2026-02-24T22:05:16.192922+0000","last_became_peered":"2026-02-24T22:05:16.192922+0000","last_unstale":"2026-02-24T22:34:43.956244+0000","last_undegraded":"2026-02-24T22:34:43.956244+0000","last_fullsized":"2026-02-24T22:34:43.956244+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'294","last_scrub_stamp":"2026-02-24T22:34:43.956165+0000","last_deep_scrub":"312'294","last_deep_scrub_stamp":"2026-02-24T22:34:43.956165+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.956165+0000","objects_scrubbed":5,"log_size":94,"log_dups_size":200,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:47:34.541512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":378,"num_read_kb":3190,"num_write":1202,"num_write_kb":12582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"312'158","reported_seq":847,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.908235+0000","last_change":"2026-02-24T22:34:47.908235+0000","last_active":"2026-02-24T22:34:47.908235+0000","last_peered":"2026-02-24T22:34:47.908235+0000","last_clean":"2026-02-24T22:34:47.908235+0000","last_became_active":"2026-02-24T22:05:18.184014+0000","last_became_peered":"2026-02-24T22:05:18.184014+0000","last_unstale":"2026-02-24T22:34:47.908235+0000","last_undegraded":"2026-02-24T22:34:47.908235+0000","last_fullsized":"2026-02-24T22:34:47.908235+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'158","last_scrub_stamp":"2026-02-24T22:34:47.908167+0000","last_deep_scrub":"312'158","last_deep_scrub_stamp":"2026-02-24T22:34:47.908167+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.908167+0000","objects_scrubbed":23,"log_size":58,"log_dups_size":100,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:57:20.364094+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":489,"num_read_kb":401,"num_write":208,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"312'280","reported_seq":999,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.068430+0000","last_change":"2026-02-24T22:34:43.068430+0000","last_active":"2026-02-24T22:34:43.068430+0000","last_peered":"2026-02-24T22:34:43.068430+0000","last_clean":"2026-02-24T22:34:43.068430+0000","last_became_active":"2026-02-24T22:05:16.191055+0000","last_became_peered":"2026-02-24T22:05:16.191055+0000","last_unstale":"2026-02-24T22:34:43.068430+0000","last_undegraded":"2026-02-24T22:34:43.068430+0000","last_fullsized":"2026-02-24T22:34:43.068430+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'280","last_scrub_stamp":"2026-02-24T22:34:43.068300+0000","last_deep_scrub":"312'280","last_deep_scrub_stamp":"2026-02-24T22:34:43.068300+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.068300+0000","objects_scrubbed":9,"log_size":80,"log_dups_size":200,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:55:12.515669+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":135,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":423,"num_read_kb":15420,"num_write":1201,"num_write_kb":8454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"312'497","reported_seq":1222,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.721550+0000","last_change":"2026-02-24T22:34:42.721550+0000","last_active":"2026-02-24T22:34:42.721550+0000","last_peered":"2026-02-24T22:34:42.721550+0000","last_clean":"2026-02-24T22:34:42.721550+0000","last_became_active":"2026-02-24T22:05:18.190598+0000","last_became_peered":"2026-02-24T22:05:18.190598+0000","last_unstale":"2026-02-24T22:34:42.721550+0000","last_undegraded":"2026-02-24T22:34:42.721550+0000","last_fullsized":"2026-02-24T22:34:42.721550+0000","mapping_epoch":24,"log_start":"312'400","ondisk_log_start":"312'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'497","last_scrub_stamp":"2026-02-24T22:34:42.721473+0000","last_deep_scrub":"312'497","last_deep_scrub_stamp":"2026-02-24T22:34:42.721473+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.721473+0000","objects_scrubbed":27,"log_size":97,"log_dups_size":400,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:09:26.432917+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1905,"num_read_kb":1522,"num_write":772,"num_write_kb":389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":267,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"312'306","reported_seq":1024,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.065317+0000","last_change":"2026-02-24T22:34:41.065317+0000","last_active":"2026-02-24T22:34:41.065317+0000","last_peered":"2026-02-24T22:34:41.065317+0000","last_clean":"2026-02-24T22:34:41.065317+0000","last_became_active":"2026-02-24T22:05:16.191164+0000","last_became_peered":"2026-02-24T22:05:16.191164+0000","last_unstale":"2026-02-24T22:34:41.065317+0000","last_undegraded":"2026-02-24T22:34:41.065317+0000","last_fullsized":"2026-02-24T22:34:41.065317+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'306","last_scrub_stamp":"2026-02-24T22:34:41.065249+0000","last_deep_scrub":"312'306","last_deep_scrub_stamp":"2026-02-24T22:34:41.065249+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.065249+0000","objects_scrubbed":6,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:42:26.339639+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":491,"num_read_kb":5551,"num_write":1337,"num_write_kb":5240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"312'56","reported_seq":710,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.746526+0000","last_change":"2026-02-24T22:05:18.198708+0000","last_active":"2026-02-24T22:32:24.746526+0000","last_peered":"2026-02-24T22:32:24.746526+0000","last_clean":"2026-02-24T22:32:24.746526+0000","last_became_active":"2026-02-24T22:05:18.198559+0000","last_became_peered":"2026-02-24T22:05:18.198559+0000","last_unstale":"2026-02-24T22:32:24.746526+0000","last_undegraded":"2026-02-24T22:32:24.746526+0000","last_fullsized":"2026-02-24T22:32:24.746526+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:17.174213+0000","last_clean_scrub_stamp":"2026-02-24T22:05:17.174213+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:01:34.891706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":195,"num_read_kb":167,"num_write":70,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"312'304","reported_seq":1012,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.096430+0000","last_change":"2026-02-24T22:34:42.096430+0000","last_active":"2026-02-24T22:34:42.096430+0000","last_peered":"2026-02-24T22:34:42.096430+0000","last_clean":"2026-02-24T22:34:42.096430+0000","last_became_active":"2026-02-24T22:05:16.190367+0000","last_became_peered":"2026-02-24T22:05:16.190367+0000","last_unstale":"2026-02-24T22:34:42.096430+0000","last_undegraded":"2026-02-24T22:34:42.096430+0000","last_fullsized":"2026-02-24T22:34:42.096430+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'304","last_scrub_stamp":"2026-02-24T22:34:42.096375+0000","last_deep_scrub":"312'304","last_deep_scrub_stamp":"2026-02-24T22:34:42.096375+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.096375+0000","objects_scrubbed":4,"log_size":104,"log_dups_size":200,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:48:33.407036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":439,"num_read_kb":1335,"num_write":1233,"num_write_kb":1104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"312'237","reported_seq":988,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.882613+0000","last_change":"2026-02-24T22:34:48.882613+0000","last_active":"2026-02-24T22:34:48.882613+0000","last_peered":"2026-02-24T22:34:48.882613+0000","last_clean":"2026-02-24T22:34:48.882613+0000","last_became_active":"2026-02-24T22:05:18.191767+0000","last_became_peered":"2026-02-24T22:05:18.191767+0000","last_unstale":"2026-02-24T22:34:48.882613+0000","last_undegraded":"2026-02-24T22:34:48.882613+0000","last_fullsized":"2026-02-24T22:34:48.882613+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'237","last_scrub_stamp":"2026-02-24T22:34:48.882561+0000","last_deep_scrub":"312'237","last_deep_scrub_stamp":"2026-02-24T22:34:48.882561+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.882561+0000","objects_scrubbed":36,"log_size":37,"log_dups_size":200,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:24:34.587031+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":876,"num_read_kb":754,"num_write":323,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"312'364","reported_seq":1142,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.830513+0000","last_change":"2026-02-24T22:34:36.830513+0000","last_active":"2026-02-24T22:34:36.830513+0000","last_peered":"2026-02-24T22:34:36.830513+0000","last_clean":"2026-02-24T22:34:36.830513+0000","last_became_active":"2026-02-24T22:05:16.185779+0000","last_became_peered":"2026-02-24T22:05:16.185779+0000","last_unstale":"2026-02-24T22:34:36.830513+0000","last_undegraded":"2026-02-24T22:34:36.830513+0000","last_fullsized":"2026-02-24T22:34:36.830513+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'364","last_scrub_stamp":"2026-02-24T22:34:36.830446+0000","last_deep_scrub":"312'364","last_deep_scrub_stamp":"2026-02-24T22:34:36.830446+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.830446+0000","objects_scrubbed":6,"log_size":64,"log_dups_size":300,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:49:17.279372+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":619,"num_read_kb":883,"num_write":1617,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"312'109","reported_seq":765,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.090042+0000","last_change":"2026-02-24T22:34:41.090042+0000","last_active":"2026-02-24T22:34:41.090042+0000","last_peered":"2026-02-24T22:34:41.090042+0000","last_clean":"2026-02-24T22:34:41.090042+0000","last_became_active":"2026-02-24T22:05:18.187838+0000","last_became_peered":"2026-02-24T22:05:18.187838+0000","last_unstale":"2026-02-24T22:34:41.090042+0000","last_undegraded":"2026-02-24T22:34:41.090042+0000","last_fullsized":"2026-02-24T22:34:41.090042+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'109","last_scrub_stamp":"2026-02-24T22:34:41.089971+0000","last_deep_scrub":"312'109","last_deep_scrub_stamp":"2026-02-24T22:34:41.089971+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.089971+0000","objects_scrubbed":26,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:13:12.250060+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":263,"num_read_kb":217,"num_write":106,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":719,"num_omap_keys":3,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"312'164","reported_seq":846,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.431711+0000","last_change":"2026-02-24T22:34:47.431711+0000","last_active":"2026-02-24T22:34:47.431711+0000","last_peered":"2026-02-24T22:34:47.431711+0000","last_clean":"2026-02-24T22:34:47.431711+0000","last_became_active":"2026-02-24T22:05:18.191625+0000","last_became_peered":"2026-02-24T22:05:18.191625+0000","last_unstale":"2026-02-24T22:34:47.431711+0000","last_undegraded":"2026-02-24T22:34:47.431711+0000","last_fullsized":"2026-02-24T22:34:47.431711+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'164","last_scrub_stamp":"2026-02-24T22:34:47.431652+0000","last_deep_scrub":"312'164","last_deep_scrub_stamp":"2026-02-24T22:34:47.431652+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.431652+0000","objects_scrubbed":29,"log_size":64,"log_dups_size":100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:13:36.629505+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":477,"num_read_kb":396,"num_write":183,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"312'302","reported_seq":1047,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.046802+0000","last_change":"2026-02-24T22:34:41.046802+0000","last_active":"2026-02-24T22:34:41.046802+0000","last_peered":"2026-02-24T22:34:41.046802+0000","last_clean":"2026-02-24T22:34:41.046802+0000","last_became_active":"2026-02-24T22:06:01.025659+0000","last_became_peered":"2026-02-24T22:06:01.025659+0000","last_unstale":"2026-02-24T22:34:41.046802+0000","last_undegraded":"2026-02-24T22:34:41.046802+0000","last_fullsized":"2026-02-24T22:34:41.046802+0000","mapping_epoch":36,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'302","last_scrub_stamp":"2026-02-24T22:34:41.046752+0000","last_deep_scrub":"312'302","last_deep_scrub_stamp":"2026-02-24T22:34:41.046752+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.046752+0000","objects_scrubbed":2,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:23:49.712428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":514,"num_read_kb":16221,"num_write":1317,"num_write_kb":8500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"312'4268","reported_seq":5087,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.193741+0000","last_change":"2026-02-24T22:34:46.193741+0000","last_active":"2026-02-24T22:34:46.193741+0000","last_peered":"2026-02-24T22:34:46.193741+0000","last_clean":"2026-02-24T22:34:46.193741+0000","last_became_active":"2026-02-24T22:05:18.193257+0000","last_became_peered":"2026-02-24T22:05:18.193257+0000","last_unstale":"2026-02-24T22:34:46.193741+0000","last_undegraded":"2026-02-24T22:34:46.193741+0000","last_fullsized":"2026-02-24T22:34:46.193741+0000","mapping_epoch":24,"log_start":"312'4200","ondisk_log_start":"312'4200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'4268","last_scrub_stamp":"2026-02-24T22:34:46.193682+0000","last_deep_scrub":"312'4268","last_deep_scrub_stamp":"2026-02-24T22:34:46.193682+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.193682+0000","objects_scrubbed":19,"log_size":68,"log_dups_size":3000,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:22:49.449922+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":17332,"num_read_kb":14553,"num_write":6398,"num_write_kb":3198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"312'289","reported_seq":1009,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.725396+0000","last_change":"2026-02-24T22:34:37.725396+0000","last_active":"2026-02-24T22:34:37.725396+0000","last_peered":"2026-02-24T22:34:37.725396+0000","last_clean":"2026-02-24T22:34:37.725396+0000","last_became_active":"2026-02-24T22:05:16.190819+0000","last_became_peered":"2026-02-24T22:05:16.190819+0000","last_unstale":"2026-02-24T22:34:37.725396+0000","last_undegraded":"2026-02-24T22:34:37.725396+0000","last_fullsized":"2026-02-24T22:34:37.725396+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'289","last_scrub_stamp":"2026-02-24T22:34:37.725293+0000","last_deep_scrub":"312'289","last_deep_scrub_stamp":"2026-02-24T22:34:37.725293+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.725293+0000","objects_scrubbed":5,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:58:32.616383+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242922,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":466,"num_read_kb":8113,"num_write":1161,"num_write_kb":5510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"312'883","reported_seq":4054,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.966939+0000","last_change":"2026-02-24T22:34:45.966939+0000","last_active":"2026-02-24T22:34:45.966939+0000","last_peered":"2026-02-24T22:34:45.966939+0000","last_clean":"2026-02-24T22:34:45.966939+0000","last_became_active":"2026-02-24T22:05:18.190363+0000","last_became_peered":"2026-02-24T22:05:18.190363+0000","last_unstale":"2026-02-24T22:34:45.966939+0000","last_undegraded":"2026-02-24T22:34:45.966939+0000","last_fullsized":"2026-02-24T22:34:45.966939+0000","mapping_epoch":24,"log_start":"312'800","ondisk_log_start":"312'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'883","last_scrub_stamp":"2026-02-24T22:34:45.966870+0000","last_deep_scrub":"312'883","last_deep_scrub_stamp":"2026-02-24T22:34:45.966870+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.966870+0000","objects_scrubbed":21,"log_size":83,"log_dups_size":800,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:41:10.952477+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4747,"num_read_kb":5785,"num_write":398,"num_write_kb":236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"312'323","reported_seq":1062,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.725432+0000","last_change":"2026-02-24T22:34:35.725432+0000","last_active":"2026-02-24T22:34:35.725432+0000","last_peered":"2026-02-24T22:34:35.725432+0000","last_clean":"2026-02-24T22:34:35.725432+0000","last_became_active":"2026-02-24T22:05:16.190816+0000","last_became_peered":"2026-02-24T22:05:16.190816+0000","last_unstale":"2026-02-24T22:34:35.725432+0000","last_undegraded":"2026-02-24T22:34:35.725432+0000","last_fullsized":"2026-02-24T22:34:35.725432+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'323","last_scrub_stamp":"2026-02-24T22:34:35.725350+0000","last_deep_scrub":"312'323","last_deep_scrub_stamp":"2026-02-24T22:34:35.725350+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.725350+0000","objects_scrubbed":2,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:35:43.171065+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":544,"num_read_kb":2327,"num_write":1304,"num_write_kb":12235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"312'120","reported_seq":832,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.062558+0000","last_change":"2026-02-24T22:34:38.062558+0000","last_active":"2026-02-24T22:34:38.062558+0000","last_peered":"2026-02-24T22:34:38.062558+0000","last_clean":"2026-02-24T22:34:38.062558+0000","last_became_active":"2026-02-24T22:05:18.186994+0000","last_became_peered":"2026-02-24T22:05:18.186994+0000","last_unstale":"2026-02-24T22:34:38.062558+0000","last_undegraded":"2026-02-24T22:34:38.062558+0000","last_fullsized":"2026-02-24T22:34:38.062558+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'120","last_scrub_stamp":"2026-02-24T22:34:38.062523+0000","last_deep_scrub":"312'120","last_deep_scrub_stamp":"2026-02-24T22:34:38.062523+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.062523+0000","objects_scrubbed":27,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:14:19.882135+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":352,"num_read_kb":306,"num_write":116,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"312'386","reported_seq":1186,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.515887+0000","last_change":"2026-02-24T22:34:34.515887+0000","last_active":"2026-02-24T22:34:34.515887+0000","last_peered":"2026-02-24T22:34:34.515887+0000","last_clean":"2026-02-24T22:34:34.515887+0000","last_became_active":"2026-02-24T22:05:16.188658+0000","last_became_peered":"2026-02-24T22:05:16.188658+0000","last_unstale":"2026-02-24T22:34:34.515887+0000","last_undegraded":"2026-02-24T22:34:34.515887+0000","last_fullsized":"2026-02-24T22:34:34.515887+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'386","last_scrub_stamp":"2026-02-24T22:34:34.515812+0000","last_deep_scrub":"312'386","last_deep_scrub_stamp":"2026-02-24T22:34:34.515812+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.515812+0000","objects_scrubbed":6,"log_size":86,"log_dups_size":300,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:26:30.161641+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":840,"num_read_kb":685,"num_write":1819,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"312'198","reported_seq":929,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.065984+0000","last_change":"2026-02-24T22:34:40.065984+0000","last_active":"2026-02-24T22:34:40.065984+0000","last_peered":"2026-02-24T22:34:40.065984+0000","last_clean":"2026-02-24T22:34:40.065984+0000","last_became_active":"2026-02-24T22:05:18.187682+0000","last_became_peered":"2026-02-24T22:05:18.187682+0000","last_unstale":"2026-02-24T22:34:40.065984+0000","last_undegraded":"2026-02-24T22:34:40.065984+0000","last_fullsized":"2026-02-24T22:34:40.065984+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'198","last_scrub_stamp":"2026-02-24T22:34:40.065916+0000","last_deep_scrub":"312'198","last_deep_scrub_stamp":"2026-02-24T22:34:40.065916+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.065916+0000","objects_scrubbed":29,"log_size":98,"log_dups_size":100,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:30:25.873043+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":639,"num_read_kb":531,"num_write":250,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"312'296","reported_seq":1004,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.790665+0000","last_change":"2026-02-24T22:34:37.790665+0000","last_active":"2026-02-24T22:34:37.790665+0000","last_peered":"2026-02-24T22:34:37.790665+0000","last_clean":"2026-02-24T22:34:37.790665+0000","last_became_active":"2026-02-24T22:05:16.184209+0000","last_became_peered":"2026-02-24T22:05:16.184209+0000","last_unstale":"2026-02-24T22:34:37.790665+0000","last_undegraded":"2026-02-24T22:34:37.790665+0000","last_fullsized":"2026-02-24T22:34:37.790665+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'296","last_scrub_stamp":"2026-02-24T22:34:37.790602+0000","last_deep_scrub":"312'296","last_deep_scrub_stamp":"2026-02-24T22:34:37.790602+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.790602+0000","objects_scrubbed":9,"log_size":96,"log_dups_size":200,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:28:56.968188+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":416,"num_read_kb":603,"num_write":1232,"num_write_kb":10636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"312'131","reported_seq":818,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.372179+0000","last_change":"2026-02-24T22:34:49.372179+0000","last_active":"2026-02-24T22:34:49.372179+0000","last_peered":"2026-02-24T22:34:49.372179+0000","last_clean":"2026-02-24T22:34:49.372179+0000","last_became_active":"2026-02-24T22:05:18.193320+0000","last_became_peered":"2026-02-24T22:05:18.193320+0000","last_unstale":"2026-02-24T22:34:49.372179+0000","last_undegraded":"2026-02-24T22:34:49.372179+0000","last_fullsized":"2026-02-24T22:34:49.372179+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'131","last_scrub_stamp":"2026-02-24T22:34:49.372116+0000","last_deep_scrub":"312'131","last_deep_scrub_stamp":"2026-02-24T22:34:49.372116+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.372116+0000","objects_scrubbed":20,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:50:18.339691+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":425,"num_read_kb":355,"num_write":146,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"312'285","reported_seq":1006,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.747158+0000","last_change":"2026-02-24T22:34:37.747158+0000","last_active":"2026-02-24T22:34:37.747158+0000","last_peered":"2026-02-24T22:34:37.747158+0000","last_clean":"2026-02-24T22:34:37.747158+0000","last_became_active":"2026-02-24T22:05:16.181527+0000","last_became_peered":"2026-02-24T22:05:16.181527+0000","last_unstale":"2026-02-24T22:34:37.747158+0000","last_undegraded":"2026-02-24T22:34:37.747158+0000","last_fullsized":"2026-02-24T22:34:37.747158+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'285","last_scrub_stamp":"2026-02-24T22:34:37.747092+0000","last_deep_scrub":"312'285","last_deep_scrub_stamp":"2026-02-24T22:34:37.747092+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.747092+0000","objects_scrubbed":4,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:43:04.672418+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242895,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":474,"num_read_kb":5463,"num_write":1217,"num_write_kb":5189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":576,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:04.744157+0000","last_change":"2026-02-24T22:05:29.658481+0000","last_active":"2026-02-24T22:32:04.744157+0000","last_peered":"2026-02-24T22:32:04.744157+0000","last_clean":"2026-02-24T22:32:04.744157+0000","last_became_active":"2026-02-24T22:05:29.657764+0000","last_became_peered":"2026-02-24T22:05:29.657764+0000","last_unstale":"2026-02-24T22:32:04.744157+0000","last_undegraded":"2026-02-24T22:32:04.744157+0000","last_fullsized":"2026-02-24T22:32:04.744157+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:34:57.407311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"312'107","reported_seq":770,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.757778+0000","last_change":"2026-02-24T22:34:42.757778+0000","last_active":"2026-02-24T22:34:42.757778+0000","last_peered":"2026-02-24T22:34:42.757778+0000","last_clean":"2026-02-24T22:34:42.757778+0000","last_became_active":"2026-02-24T22:05:18.184771+0000","last_became_peered":"2026-02-24T22:05:18.184771+0000","last_unstale":"2026-02-24T22:34:42.757778+0000","last_undegraded":"2026-02-24T22:34:42.757778+0000","last_fullsized":"2026-02-24T22:34:42.757778+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'107","last_scrub_stamp":"2026-02-24T22:34:42.757725+0000","last_deep_scrub":"312'107","last_deep_scrub_stamp":"2026-02-24T22:34:42.757725+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.757725+0000","objects_scrubbed":27,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:52:33.731233+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":246,"num_read_kb":208,"num_write":102,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"312'344","reported_seq":1133,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.109736+0000","last_change":"2026-02-24T22:34:37.109736+0000","last_active":"2026-02-24T22:34:37.109736+0000","last_peered":"2026-02-24T22:34:37.109736+0000","last_clean":"2026-02-24T22:34:37.109736+0000","last_became_active":"2026-02-24T22:05:16.191841+0000","last_became_peered":"2026-02-24T22:05:16.191841+0000","last_unstale":"2026-02-24T22:34:37.109736+0000","last_undegraded":"2026-02-24T22:34:37.109736+0000","last_fullsized":"2026-02-24T22:34:37.109736+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'344","last_scrub_stamp":"2026-02-24T22:34:37.109689+0000","last_deep_scrub":"312'344","last_deep_scrub_stamp":"2026-02-24T22:34:37.109689+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.109689+0000","objects_scrubbed":4,"log_size":44,"log_dups_size":300,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:35:53.603402+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":614,"num_read_kb":81372,"num_write":1329,"num_write_kb":20589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":579,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.745270+0000","last_change":"2026-02-24T22:05:20.194969+0000","last_active":"2026-02-24T22:31:54.745270+0000","last_peered":"2026-02-24T22:31:54.745270+0000","last_clean":"2026-02-24T22:31:54.745270+0000","last_became_active":"2026-02-24T22:05:20.194031+0000","last_became_peered":"2026-02-24T22:05:20.194031+0000","last_unstale":"2026-02-24T22:31:54.745270+0000","last_undegraded":"2026-02-24T22:31:54.745270+0000","last_fullsized":"2026-02-24T22:31:54.745270+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:17:15.557806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"312'259","reported_seq":899,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.772688+0000","last_change":"2026-02-24T22:05:22.192071+0000","last_active":"2026-02-24T22:34:25.772688+0000","last_peered":"2026-02-24T22:34:25.772688+0000","last_clean":"2026-02-24T22:34:25.772688+0000","last_became_active":"2026-02-24T22:05:22.191928+0000","last_became_peered":"2026-02-24T22:05:22.191928+0000","last_unstale":"2026-02-24T22:34:25.772688+0000","last_undegraded":"2026-02-24T22:34:25.772688+0000","last_fullsized":"2026-02-24T22:34:25.772688+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":200,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:56:18.209592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44919,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":413,"num_read_kb":914,"num_write":358,"num_write_kb":269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"312'158","reported_seq":747,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.253470+0000","last_change":"2026-02-24T22:05:24.205235+0000","last_active":"2026-02-24T22:34:18.253470+0000","last_peered":"2026-02-24T22:34:18.253470+0000","last_clean":"2026-02-24T22:34:18.253470+0000","last_became_active":"2026-02-24T22:05:24.204990+0000","last_became_peered":"2026-02-24T22:05:24.204990+0000","last_unstale":"2026-02-24T22:34:18.253470+0000","last_undegraded":"2026-02-24T22:34:18.253470+0000","last_fullsized":"2026-02-24T22:34:18.253470+0000","mapping_epoch":30,"log_start":"37'100","ondisk_log_start":"37'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":100,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:14:42.637540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32373,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":92,"num_read_kb":87,"num_write":391,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":565,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.747239+0000","last_change":"2026-02-24T22:05:29.662185+0000","last_active":"2026-02-24T22:32:09.747239+0000","last_peered":"2026-02-24T22:32:09.747239+0000","last_clean":"2026-02-24T22:32:09.747239+0000","last_became_active":"2026-02-24T22:05:29.661747+0000","last_became_peered":"2026-02-24T22:05:29.661747+0000","last_unstale":"2026-02-24T22:32:09.747239+0000","last_undegraded":"2026-02-24T22:32:09.747239+0000","last_fullsized":"2026-02-24T22:32:09.747239+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"312'104","reported_seq":784,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.994989+0000","last_change":"2026-02-24T22:34:42.994989+0000","last_active":"2026-02-24T22:34:42.994989+0000","last_peered":"2026-02-24T22:34:42.994989+0000","last_clean":"2026-02-24T22:34:42.994989+0000","last_became_active":"2026-02-24T22:05:18.191346+0000","last_became_peered":"2026-02-24T22:05:18.191346+0000","last_unstale":"2026-02-24T22:34:42.994989+0000","last_undegraded":"2026-02-24T22:34:42.994989+0000","last_fullsized":"2026-02-24T22:34:42.994989+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'104","last_scrub_stamp":"2026-02-24T22:34:42.994927+0000","last_deep_scrub":"312'104","last_deep_scrub_stamp":"2026-02-24T22:34:42.994927+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.994927+0000","objects_scrubbed":31,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:12:15.667131+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":237,"num_read_kb":209,"num_write":87,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"312'276","reported_seq":1004,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:30.666602+0000","last_change":"2026-02-24T22:34:30.666602+0000","last_active":"2026-02-24T22:34:30.666602+0000","last_peered":"2026-02-24T22:34:30.666602+0000","last_clean":"2026-02-24T22:34:30.666602+0000","last_became_active":"2026-02-24T22:05:16.183990+0000","last_became_peered":"2026-02-24T22:05:16.183990+0000","last_unstale":"2026-02-24T22:34:30.666602+0000","last_undegraded":"2026-02-24T22:34:30.666602+0000","last_fullsized":"2026-02-24T22:34:30.666602+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'276","last_scrub_stamp":"2026-02-24T22:34:30.666509+0000","last_deep_scrub":"312'276","last_deep_scrub_stamp":"2026-02-24T22:34:30.666509+0000","last_clean_scrub_stamp":"2026-02-24T22:34:30.666509+0000","objects_scrubbed":5,"log_size":76,"log_dups_size":200,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:23:41.986404+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485805,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":458,"num_read_kb":23693,"num_write":1270,"num_write_kb":23324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.729065+0000","last_change":"2026-02-24T22:05:20.186544+0000","last_active":"2026-02-24T22:31:04.729065+0000","last_peered":"2026-02-24T22:31:04.729065+0000","last_clean":"2026-02-24T22:31:04.729065+0000","last_became_active":"2026-02-24T22:05:20.186217+0000","last_became_peered":"2026-02-24T22:05:20.186217+0000","last_unstale":"2026-02-24T22:31:04.729065+0000","last_undegraded":"2026-02-24T22:31:04.729065+0000","last_fullsized":"2026-02-24T22:31:04.729065+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:41:55.793283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"312'305","reported_seq":1152,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.237230+0000","last_change":"2026-02-24T22:05:22.197384+0000","last_active":"2026-02-24T22:34:25.237230+0000","last_peered":"2026-02-24T22:34:25.237230+0000","last_clean":"2026-02-24T22:34:25.237230+0000","last_became_active":"2026-02-24T22:05:22.196911+0000","last_became_peered":"2026-02-24T22:05:22.196911+0000","last_unstale":"2026-02-24T22:34:25.237230+0000","last_undegraded":"2026-02-24T22:34:25.237230+0000","last_fullsized":"2026-02-24T22:34:25.237230+0000","mapping_epoch":28,"log_start":"307'200","ondisk_log_start":"307'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:19:12.370112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1016,"num_read_kb":540,"num_write":51,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"312'187","reported_seq":800,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.220647+0000","last_change":"2026-02-24T22:05:24.207903+0000","last_active":"2026-02-24T22:34:18.220647+0000","last_peered":"2026-02-24T22:34:18.220647+0000","last_clean":"2026-02-24T22:34:18.220647+0000","last_became_active":"2026-02-24T22:05:24.207692+0000","last_became_peered":"2026-02-24T22:05:24.207692+0000","last_unstale":"2026-02-24T22:34:18.220647+0000","last_undegraded":"2026-02-24T22:34:18.220647+0000","last_fullsized":"2026-02-24T22:34:18.220647+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:14:09.532597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29978,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":173,"num_read_kb":171,"num_write":478,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":573,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.734173+0000","last_change":"2026-02-24T22:05:29.654457+0000","last_active":"2026-02-24T22:30:29.734173+0000","last_peered":"2026-02-24T22:30:29.734173+0000","last_clean":"2026-02-24T22:30:29.734173+0000","last_became_active":"2026-02-24T22:05:29.654288+0000","last_became_peered":"2026-02-24T22:05:29.654288+0000","last_unstale":"2026-02-24T22:30:29.734173+0000","last_undegraded":"2026-02-24T22:30:29.734173+0000","last_fullsized":"2026-02-24T22:30:29.734173+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:05:32.007637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"312'298","reported_seq":1031,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.885878+0000","last_change":"2026-02-24T22:34:42.885878+0000","last_active":"2026-02-24T22:34:42.885878+0000","last_peered":"2026-02-24T22:34:42.885878+0000","last_clean":"2026-02-24T22:34:42.885878+0000","last_became_active":"2026-02-24T22:05:18.184542+0000","last_became_peered":"2026-02-24T22:05:18.184542+0000","last_unstale":"2026-02-24T22:34:42.885878+0000","last_undegraded":"2026-02-24T22:34:42.885878+0000","last_fullsized":"2026-02-24T22:34:42.885878+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'298","last_scrub_stamp":"2026-02-24T22:34:42.885832+0000","last_deep_scrub":"312'298","last_deep_scrub_stamp":"2026-02-24T22:34:42.885832+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.885832+0000","objects_scrubbed":24,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:07:57.036667+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1179,"num_read_kb":976,"num_write":460,"num_write_kb":243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"312'294","reported_seq":1016,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.140765+0000","last_change":"2026-02-24T22:34:35.140765+0000","last_active":"2026-02-24T22:34:35.140765+0000","last_peered":"2026-02-24T22:34:35.140765+0000","last_clean":"2026-02-24T22:34:35.140765+0000","last_became_active":"2026-02-24T22:05:16.190372+0000","last_became_peered":"2026-02-24T22:05:16.190372+0000","last_unstale":"2026-02-24T22:34:35.140765+0000","last_undegraded":"2026-02-24T22:34:35.140765+0000","last_fullsized":"2026-02-24T22:34:35.140765+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'294","last_scrub_stamp":"2026-02-24T22:34:35.140712+0000","last_deep_scrub":"312'294","last_deep_scrub_stamp":"2026-02-24T22:34:35.140712+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.140712+0000","objects_scrubbed":4,"log_size":94,"log_dups_size":200,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:24:43.777918+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":430,"num_read_kb":3398,"num_write":1175,"num_write_kb":524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.734254+0000","last_change":"2026-02-24T22:05:20.187522+0000","last_active":"2026-02-24T22:31:54.734254+0000","last_peered":"2026-02-24T22:31:54.734254+0000","last_clean":"2026-02-24T22:31:54.734254+0000","last_became_active":"2026-02-24T22:05:20.187327+0000","last_became_peered":"2026-02-24T22:05:20.187327+0000","last_unstale":"2026-02-24T22:31:54.734254+0000","last_undegraded":"2026-02-24T22:31:54.734254+0000","last_fullsized":"2026-02-24T22:31:54.734254+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:09:52.559206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"312'802","reported_seq":2139,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.134134+0000","last_change":"2026-02-24T22:34:48.134134+0000","last_active":"2026-02-24T22:34:48.134134+0000","last_peered":"2026-02-24T22:34:48.134134+0000","last_clean":"2026-02-24T22:34:48.134134+0000","last_became_active":"2026-02-24T22:05:22.191073+0000","last_became_peered":"2026-02-24T22:05:22.191073+0000","last_unstale":"2026-02-24T22:34:48.134134+0000","last_undegraded":"2026-02-24T22:34:48.134134+0000","last_fullsized":"2026-02-24T22:34:48.134134+0000","mapping_epoch":28,"log_start":"307'700","ondisk_log_start":"307'700","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'802","last_scrub_stamp":"2026-02-24T22:34:48.134072+0000","last_deep_scrub":"312'802","last_deep_scrub_stamp":"2026-02-24T22:34:48.134072+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.134072+0000","objects_scrubbed":8,"log_size":102,"log_dups_size":700,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:20:11.846328+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":460,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2934,"num_read_kb":1505,"num_write":81,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"312'204","reported_seq":1716,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.409302+0000","last_change":"2026-02-24T22:05:24.206373+0000","last_active":"2026-02-24T22:34:17.409302+0000","last_peered":"2026-02-24T22:34:17.409302+0000","last_clean":"2026-02-24T22:34:17.409302+0000","last_became_active":"2026-02-24T22:05:24.206134+0000","last_became_peered":"2026-02-24T22:05:24.206134+0000","last_unstale":"2026-02-24T22:34:17.409302+0000","last_undegraded":"2026-02-24T22:34:17.409302+0000","last_fullsized":"2026-02-24T22:34:17.409302+0000","mapping_epoch":30,"log_start":"37'100","ondisk_log_start":"37'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:32:52.956429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33889,"num_objects":107,"num_object_clones":0,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":0,"num_read":1064,"num_read_kb":1063,"num_write":484,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.733657+0000","last_change":"2026-02-24T22:05:29.658922+0000","last_active":"2026-02-24T22:30:29.733657+0000","last_peered":"2026-02-24T22:30:29.733657+0000","last_clean":"2026-02-24T22:30:29.733657+0000","last_became_active":"2026-02-24T22:05:29.658119+0000","last_became_peered":"2026-02-24T22:05:29.658119+0000","last_unstale":"2026-02-24T22:30:29.733657+0000","last_undegraded":"2026-02-24T22:30:29.733657+0000","last_fullsized":"2026-02-24T22:30:29.733657+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:10:58.821048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"312'690","reported_seq":1496,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.055105+0000","last_change":"2026-02-24T22:34:37.055105+0000","last_active":"2026-02-24T22:34:37.055105+0000","last_peered":"2026-02-24T22:34:37.055105+0000","last_clean":"2026-02-24T22:34:37.055105+0000","last_became_active":"2026-02-24T22:06:00.662609+0000","last_became_peered":"2026-02-24T22:06:00.662609+0000","last_unstale":"2026-02-24T22:34:37.055105+0000","last_undegraded":"2026-02-24T22:34:37.055105+0000","last_fullsized":"2026-02-24T22:34:37.055105+0000","mapping_epoch":36,"log_start":"312'600","ondisk_log_start":"312'600","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'690","last_scrub_stamp":"2026-02-24T22:34:37.055035+0000","last_deep_scrub":"312'690","last_deep_scrub_stamp":"2026-02-24T22:34:37.055035+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.055035+0000","objects_scrubbed":22,"log_size":90,"log_dups_size":600,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:44:46.603715+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":2886,"num_read_kb":2388,"num_write":1101,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"312'327","reported_seq":1061,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:30.811183+0000","last_change":"2026-02-24T22:34:30.811183+0000","last_active":"2026-02-24T22:34:30.811183+0000","last_peered":"2026-02-24T22:34:30.811183+0000","last_clean":"2026-02-24T22:34:30.811183+0000","last_became_active":"2026-02-24T22:05:16.189739+0000","last_became_peered":"2026-02-24T22:05:16.189739+0000","last_unstale":"2026-02-24T22:34:30.811183+0000","last_undegraded":"2026-02-24T22:34:30.811183+0000","last_fullsized":"2026-02-24T22:34:30.811183+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'327","last_scrub_stamp":"2026-02-24T22:34:30.811119+0000","last_deep_scrub":"312'327","last_deep_scrub_stamp":"2026-02-24T22:34:30.811119+0000","last_clean_scrub_stamp":"2026-02-24T22:34:30.811119+0000","objects_scrubbed":9,"log_size":27,"log_dups_size":300,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:44:34.182914+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485786,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":468,"num_read_kb":16928,"num_write":1377,"num_write_kb":17766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":579,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.745280+0000","last_change":"2026-02-24T22:05:20.193438+0000","last_active":"2026-02-24T22:31:54.745280+0000","last_peered":"2026-02-24T22:31:54.745280+0000","last_clean":"2026-02-24T22:31:54.745280+0000","last_became_active":"2026-02-24T22:05:20.193200+0000","last_became_peered":"2026-02-24T22:05:20.193200+0000","last_unstale":"2026-02-24T22:31:54.745280+0000","last_undegraded":"2026-02-24T22:31:54.745280+0000","last_fullsized":"2026-02-24T22:31:54.745280+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:58:09.212117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"312'192","reported_seq":795,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.646624+0000","last_change":"2026-02-24T22:05:22.913003+0000","last_active":"2026-02-24T22:34:25.646624+0000","last_peered":"2026-02-24T22:34:25.646624+0000","last_clean":"2026-02-24T22:34:25.646624+0000","last_became_active":"2026-02-24T22:05:22.912826+0000","last_became_peered":"2026-02-24T22:05:22.912826+0000","last_unstale":"2026-02-24T22:34:25.646624+0000","last_undegraded":"2026-02-24T22:34:25.646624+0000","last_fullsized":"2026-02-24T22:34:25.646624+0000","mapping_epoch":28,"log_start":"312'100","ondisk_log_start":"312'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:49:38.039509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37493,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":308,"num_read_kb":794,"num_write":284,"num_write_kb":257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"312'257","reported_seq":903,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:13.928723+0000","last_change":"2026-02-24T22:05:24.199267+0000","last_active":"2026-02-24T22:34:13.928723+0000","last_peered":"2026-02-24T22:34:13.928723+0000","last_clean":"2026-02-24T22:34:13.928723+0000","last_became_active":"2026-02-24T22:05:24.198964+0000","last_became_peered":"2026-02-24T22:05:24.198964+0000","last_unstale":"2026-02-24T22:34:13.928723+0000","last_undegraded":"2026-02-24T22:34:13.928723+0000","last_fullsized":"2026-02-24T22:34:13.928723+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":200,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:47:04.450411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29433,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":428,"num_read_kb":365,"num_write":612,"num_write_kb":187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"312'140","reported_seq":875,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.037417+0000","last_change":"2026-02-24T22:34:42.037417+0000","last_active":"2026-02-24T22:34:42.037417+0000","last_peered":"2026-02-24T22:34:42.037417+0000","last_clean":"2026-02-24T22:34:42.037417+0000","last_became_active":"2026-02-24T22:05:18.189796+0000","last_became_peered":"2026-02-24T22:05:18.189796+0000","last_unstale":"2026-02-24T22:34:42.037417+0000","last_undegraded":"2026-02-24T22:34:42.037417+0000","last_fullsized":"2026-02-24T22:34:42.037417+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'140","last_scrub_stamp":"2026-02-24T22:34:42.037385+0000","last_deep_scrub":"312'140","last_deep_scrub_stamp":"2026-02-24T22:34:42.037385+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.037385+0000","objects_scrubbed":27,"log_size":40,"log_dups_size":100,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:45:54.858292+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":455,"num_read_kb":399,"num_write":145,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"312'263","reported_seq":951,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.085973+0000","last_change":"2026-02-24T22:34:38.085935+0000","last_active":"2026-02-24T22:34:38.085973+0000","last_peered":"2026-02-24T22:34:38.085973+0000","last_clean":"2026-02-24T22:34:38.085973+0000","last_became_active":"2026-02-24T22:05:16.191985+0000","last_became_peered":"2026-02-24T22:05:16.191985+0000","last_unstale":"2026-02-24T22:34:38.085973+0000","last_undegraded":"2026-02-24T22:34:38.085973+0000","last_fullsized":"2026-02-24T22:34:38.085973+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'263","last_scrub_stamp":"2026-02-24T22:34:38.085876+0000","last_deep_scrub":"312'263","last_deep_scrub_stamp":"2026-02-24T22:34:38.085876+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.085876+0000","objects_scrubbed":7,"log_size":63,"log_dups_size":200,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:00:57.241743+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":82,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":355,"num_read_kb":4498,"num_write":1041,"num_write_kb":4311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":586,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.742035+0000","last_change":"2026-02-24T22:05:20.194496+0000","last_active":"2026-02-24T22:31:19.742035+0000","last_peered":"2026-02-24T22:31:19.742035+0000","last_clean":"2026-02-24T22:31:19.742035+0000","last_became_active":"2026-02-24T22:05:20.194288+0000","last_became_peered":"2026-02-24T22:05:20.194288+0000","last_unstale":"2026-02-24T22:31:19.742035+0000","last_undegraded":"2026-02-24T22:31:19.742035+0000","last_fullsized":"2026-02-24T22:31:19.742035+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:55:32.351699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"312'260","reported_seq":889,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.852870+0000","last_change":"2026-02-24T22:05:22.913184+0000","last_active":"2026-02-24T22:34:25.852870+0000","last_peered":"2026-02-24T22:34:25.852870+0000","last_clean":"2026-02-24T22:34:25.852870+0000","last_became_active":"2026-02-24T22:05:22.912859+0000","last_became_peered":"2026-02-24T22:05:22.912859+0000","last_unstale":"2026-02-24T22:34:25.852870+0000","last_undegraded":"2026-02-24T22:34:25.852870+0000","last_fullsized":"2026-02-24T22:34:25.852870+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":200,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:37:47.377695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38223,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":418,"num_read_kb":936,"num_write":362,"num_write_kb":271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"312'315","reported_seq":1011,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.247236+0000","last_change":"2026-02-24T22:05:24.200453+0000","last_active":"2026-02-24T22:34:18.247236+0000","last_peered":"2026-02-24T22:34:18.247236+0000","last_clean":"2026-02-24T22:34:18.247236+0000","last_became_active":"2026-02-24T22:05:24.199772+0000","last_became_peered":"2026-02-24T22:05:24.199772+0000","last_unstale":"2026-02-24T22:34:18.247236+0000","last_undegraded":"2026-02-24T22:34:18.247236+0000","last_fullsized":"2026-02-24T22:34:18.247236+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30266,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":473,"num_read_kb":498,"num_write":701,"num_write_kb":232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":567,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:34.749996+0000","last_change":"2026-02-24T22:05:29.661914+0000","last_active":"2026-02-24T22:32:34.749996+0000","last_peered":"2026-02-24T22:32:34.749996+0000","last_clean":"2026-02-24T22:32:34.749996+0000","last_became_active":"2026-02-24T22:05:29.661401+0000","last_became_peered":"2026-02-24T22:05:29.661401+0000","last_unstale":"2026-02-24T22:32:34.749996+0000","last_undegraded":"2026-02-24T22:32:34.749996+0000","last_fullsized":"2026-02-24T22:32:34.749996+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"312'103","reported_seq":773,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.478042+0000","last_change":"2026-02-24T22:34:35.478042+0000","last_active":"2026-02-24T22:34:35.478042+0000","last_peered":"2026-02-24T22:34:35.478042+0000","last_clean":"2026-02-24T22:34:35.478042+0000","last_became_active":"2026-02-24T22:05:18.193188+0000","last_became_peered":"2026-02-24T22:05:18.193188+0000","last_unstale":"2026-02-24T22:34:35.478042+0000","last_undegraded":"2026-02-24T22:34:35.478042+0000","last_fullsized":"2026-02-24T22:34:35.478042+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'103","last_scrub_stamp":"2026-02-24T22:34:35.477982+0000","last_deep_scrub":"312'103","last_deep_scrub_stamp":"2026-02-24T22:34:35.477982+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.477982+0000","objects_scrubbed":19,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:46:51.105644+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":312,"num_read_kb":260,"num_write":126,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"312'324","reported_seq":1064,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.151701+0000","last_change":"2026-02-24T22:34:33.151665+0000","last_active":"2026-02-24T22:34:33.151701+0000","last_peered":"2026-02-24T22:34:33.151701+0000","last_clean":"2026-02-24T22:34:33.151701+0000","last_became_active":"2026-02-24T22:05:16.191956+0000","last_became_peered":"2026-02-24T22:05:16.191956+0000","last_unstale":"2026-02-24T22:34:33.151701+0000","last_undegraded":"2026-02-24T22:34:33.151701+0000","last_fullsized":"2026-02-24T22:34:33.151701+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'324","last_scrub_stamp":"2026-02-24T22:34:33.151603+0000","last_deep_scrub":"312'324","last_deep_scrub_stamp":"2026-02-24T22:34:33.151603+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.151603+0000","objects_scrubbed":3,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:52:36.974066+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485778,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":523,"num_read_kb":10657,"num_write":1420,"num_write_kb":15460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":602,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.120807+0000","last_change":"2026-02-24T22:34:46.120807+0000","last_active":"2026-02-24T22:34:46.120807+0000","last_peered":"2026-02-24T22:34:46.120807+0000","last_clean":"2026-02-24T22:34:46.120807+0000","last_became_active":"2026-02-24T22:05:20.186055+0000","last_became_peered":"2026-02-24T22:05:20.186055+0000","last_unstale":"2026-02-24T22:34:46.120807+0000","last_undegraded":"2026-02-24T22:34:46.120807+0000","last_fullsized":"2026-02-24T22:34:46.120807+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:46.120737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:46.120737+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.120737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:33:04.392007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"312'229","reported_seq":838,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.441864+0000","last_change":"2026-02-24T22:05:22.194034+0000","last_active":"2026-02-24T22:34:25.441864+0000","last_peered":"2026-02-24T22:34:25.441864+0000","last_clean":"2026-02-24T22:34:25.441864+0000","last_became_active":"2026-02-24T22:05:22.193741+0000","last_became_peered":"2026-02-24T22:05:22.193741+0000","last_unstale":"2026-02-24T22:34:25.441864+0000","last_undegraded":"2026-02-24T22:34:25.441864+0000","last_fullsized":"2026-02-24T22:34:25.441864+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":200,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39811,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":385,"num_read_kb":1028,"num_write":360,"num_write_kb":328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"312'185","reported_seq":809,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.176596+0000","last_change":"2026-02-24T22:05:24.199051+0000","last_active":"2026-02-24T22:34:15.176596+0000","last_peered":"2026-02-24T22:34:15.176596+0000","last_clean":"2026-02-24T22:34:15.176596+0000","last_became_active":"2026-02-24T22:05:24.198731+0000","last_became_peered":"2026-02-24T22:05:24.198731+0000","last_unstale":"2026-02-24T22:34:15.176596+0000","last_undegraded":"2026-02-24T22:34:15.176596+0000","last_fullsized":"2026-02-24T22:34:15.176596+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:42:29.899927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30034,"num_objects":91,"num_object_clones":0,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":241,"num_read_kb":207,"num_write":470,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":571,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:39.746442+0000","last_change":"2026-02-24T22:05:29.661462+0000","last_active":"2026-02-24T22:32:39.746442+0000","last_peered":"2026-02-24T22:32:39.746442+0000","last_clean":"2026-02-24T22:32:39.746442+0000","last_became_active":"2026-02-24T22:05:29.661161+0000","last_became_peered":"2026-02-24T22:05:29.661161+0000","last_unstale":"2026-02-24T22:32:39.746442+0000","last_undegraded":"2026-02-24T22:32:39.746442+0000","last_fullsized":"2026-02-24T22:32:39.746442+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"312'129","reported_seq":784,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.703983+0000","last_change":"2026-02-24T22:34:39.703983+0000","last_active":"2026-02-24T22:34:39.703983+0000","last_peered":"2026-02-24T22:34:39.703983+0000","last_clean":"2026-02-24T22:34:39.703983+0000","last_became_active":"2026-02-24T22:05:18.191555+0000","last_became_peered":"2026-02-24T22:05:18.191555+0000","last_unstale":"2026-02-24T22:34:39.703983+0000","last_undegraded":"2026-02-24T22:34:39.703983+0000","last_fullsized":"2026-02-24T22:34:39.703983+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'129","last_scrub_stamp":"2026-02-24T22:34:39.703930+0000","last_deep_scrub":"312'129","last_deep_scrub_stamp":"2026-02-24T22:34:39.703930+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.703930+0000","objects_scrubbed":23,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:20:21.850877+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":387,"num_read_kb":325,"num_write":162,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"312'289","reported_seq":1011,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.159670+0000","last_change":"2026-02-24T22:34:34.159624+0000","last_active":"2026-02-24T22:34:34.159670+0000","last_peered":"2026-02-24T22:34:34.159670+0000","last_clean":"2026-02-24T22:34:34.159670+0000","last_became_active":"2026-02-24T22:05:16.191733+0000","last_became_peered":"2026-02-24T22:05:16.191733+0000","last_unstale":"2026-02-24T22:34:34.159670+0000","last_undegraded":"2026-02-24T22:34:34.159670+0000","last_fullsized":"2026-02-24T22:34:34.159670+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'289","last_scrub_stamp":"2026-02-24T22:34:34.159566+0000","last_deep_scrub":"312'289","last_deep_scrub_stamp":"2026-02-24T22:34:34.159566+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.159566+0000","objects_scrubbed":4,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:18:06.498072+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242906,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":398,"num_read_kb":31022,"num_write":1049,"num_write_kb":10319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":602,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.063841+0000","last_change":"2026-02-24T22:34:44.063841+0000","last_active":"2026-02-24T22:34:44.063841+0000","last_peered":"2026-02-24T22:34:44.063841+0000","last_clean":"2026-02-24T22:34:44.063841+0000","last_became_active":"2026-02-24T22:05:20.186892+0000","last_became_peered":"2026-02-24T22:05:20.186892+0000","last_unstale":"2026-02-24T22:34:44.063841+0000","last_undegraded":"2026-02-24T22:34:44.063841+0000","last_fullsized":"2026-02-24T22:34:44.063841+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:44.063776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:44.063776+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.063776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:34:17.807478+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"312'238","reported_seq":851,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.905330+0000","last_change":"2026-02-24T22:05:22.200501+0000","last_active":"2026-02-24T22:34:25.905330+0000","last_peered":"2026-02-24T22:34:25.905330+0000","last_clean":"2026-02-24T22:34:25.905330+0000","last_became_active":"2026-02-24T22:05:22.200208+0000","last_became_peered":"2026-02-24T22:05:22.200208+0000","last_unstale":"2026-02-24T22:34:25.905330+0000","last_undegraded":"2026-02-24T22:34:25.905330+0000","last_fullsized":"2026-02-24T22:34:25.905330+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":200,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:45:47.054214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39435,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":390,"num_read_kb":991,"num_write":356,"num_write_kb":310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"312'275","reported_seq":1084,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.547089+0000","last_change":"2026-02-24T22:05:24.200107+0000","last_active":"2026-02-24T22:34:18.547089+0000","last_peered":"2026-02-24T22:34:18.547089+0000","last_clean":"2026-02-24T22:34:18.547089+0000","last_became_active":"2026-02-24T22:05:24.199525+0000","last_became_peered":"2026-02-24T22:05:24.199525+0000","last_unstale":"2026-02-24T22:34:18.547089+0000","last_undegraded":"2026-02-24T22:34:18.547089+0000","last_fullsized":"2026-02-24T22:34:18.547089+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":200,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:06:50.680774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32009,"num_objects":103,"num_object_clones":0,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":506,"num_read_kb":490,"num_write":606,"num_write_kb":207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:29.740751+0000","last_change":"2026-02-24T22:05:29.652942+0000","last_active":"2026-02-24T22:31:29.740751+0000","last_peered":"2026-02-24T22:31:29.740751+0000","last_clean":"2026-02-24T22:31:29.740751+0000","last_became_active":"2026-02-24T22:05:29.652598+0000","last_became_peered":"2026-02-24T22:05:29.652598+0000","last_unstale":"2026-02-24T22:31:29.740751+0000","last_undegraded":"2026-02-24T22:31:29.740751+0000","last_fullsized":"2026-02-24T22:31:29.740751+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:38:19.564467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"312'189","reported_seq":900,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.811102+0000","last_change":"2026-02-24T22:34:40.811102+0000","last_active":"2026-02-24T22:34:40.811102+0000","last_peered":"2026-02-24T22:34:40.811102+0000","last_clean":"2026-02-24T22:34:40.811102+0000","last_became_active":"2026-02-24T22:05:18.185152+0000","last_became_peered":"2026-02-24T22:05:18.185152+0000","last_unstale":"2026-02-24T22:34:40.811102+0000","last_undegraded":"2026-02-24T22:34:40.811102+0000","last_fullsized":"2026-02-24T22:34:40.811102+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'189","last_scrub_stamp":"2026-02-24T22:34:40.811068+0000","last_deep_scrub":"312'189","last_deep_scrub_stamp":"2026-02-24T22:34:40.811068+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.811068+0000","objects_scrubbed":20,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:24:59.774125+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":651,"num_read_kb":540,"num_write":239,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"312'323","reported_seq":1050,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.061642+0000","last_change":"2026-02-24T22:34:33.061574+0000","last_active":"2026-02-24T22:34:33.061642+0000","last_peered":"2026-02-24T22:34:33.061642+0000","last_clean":"2026-02-24T22:34:33.061642+0000","last_became_active":"2026-02-24T22:05:16.190205+0000","last_became_peered":"2026-02-24T22:05:16.190205+0000","last_unstale":"2026-02-24T22:34:33.061642+0000","last_undegraded":"2026-02-24T22:34:33.061642+0000","last_fullsized":"2026-02-24T22:34:33.061642+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'323","last_scrub_stamp":"2026-02-24T22:34:33.061497+0000","last_deep_scrub":"312'323","last_deep_scrub_stamp":"2026-02-24T22:34:33.061497+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.061497+0000","objects_scrubbed":1,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:54:15.633089+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":578,"num_read_kb":20088,"num_write":1296,"num_write_kb":14513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":602,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.105525+0000","last_change":"2026-02-24T22:34:45.105525+0000","last_active":"2026-02-24T22:34:45.105525+0000","last_peered":"2026-02-24T22:34:45.105525+0000","last_clean":"2026-02-24T22:34:45.105525+0000","last_became_active":"2026-02-24T22:05:20.186781+0000","last_became_peered":"2026-02-24T22:05:20.186781+0000","last_unstale":"2026-02-24T22:34:45.105525+0000","last_undegraded":"2026-02-24T22:34:45.105525+0000","last_fullsized":"2026-02-24T22:34:45.105525+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:45.105463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:45.105463+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.105463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:39:37.239923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"312'401","reported_seq":1021,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.972154+0000","last_change":"2026-02-24T22:05:22.192711+0000","last_active":"2026-02-24T22:34:25.972154+0000","last_peered":"2026-02-24T22:34:25.972154+0000","last_clean":"2026-02-24T22:34:25.972154+0000","last_became_active":"2026-02-24T22:05:22.192495+0000","last_became_peered":"2026-02-24T22:05:22.192495+0000","last_unstale":"2026-02-24T22:34:25.972154+0000","last_undegraded":"2026-02-24T22:34:25.972154+0000","last_fullsized":"2026-02-24T22:34:25.972154+0000","mapping_epoch":28,"log_start":"312'300","ondisk_log_start":"312'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":300,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:40:20.670168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":81867,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":699,"num_read_kb":1918,"num_write":658,"num_write_kb":626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"312'264","reported_seq":930,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.182990+0000","last_change":"2026-02-24T22:34:49.182990+0000","last_active":"2026-02-24T22:34:49.182990+0000","last_peered":"2026-02-24T22:34:49.182990+0000","last_clean":"2026-02-24T22:34:49.182990+0000","last_became_active":"2026-02-24T22:05:24.198114+0000","last_became_peered":"2026-02-24T22:05:24.198114+0000","last_unstale":"2026-02-24T22:34:49.182990+0000","last_undegraded":"2026-02-24T22:34:49.182990+0000","last_fullsized":"2026-02-24T22:34:49.182990+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'264","last_scrub_stamp":"2026-02-24T22:34:49.182961+0000","last_deep_scrub":"312'264","last_deep_scrub_stamp":"2026-02-24T22:34:49.182961+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.182961+0000","objects_scrubbed":91,"log_size":64,"log_dups_size":200,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:07:14.157330+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29501,"num_objects":91,"num_object_clones":0,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":235,"num_read_kb":226,"num_write":429,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:29.726754+0000","last_change":"2026-02-24T22:05:29.652942+0000","last_active":"2026-02-24T22:31:29.726754+0000","last_peered":"2026-02-24T22:31:29.726754+0000","last_clean":"2026-02-24T22:31:29.726754+0000","last_became_active":"2026-02-24T22:05:29.652715+0000","last_became_peered":"2026-02-24T22:05:29.652715+0000","last_unstale":"2026-02-24T22:31:29.726754+0000","last_undegraded":"2026-02-24T22:31:29.726754+0000","last_fullsized":"2026-02-24T22:31:29.726754+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:13:37.213497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"312'596","reported_seq":1465,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.442591+0000","last_change":"2026-02-24T22:34:37.442591+0000","last_active":"2026-02-24T22:34:37.442591+0000","last_peered":"2026-02-24T22:34:37.442591+0000","last_clean":"2026-02-24T22:34:37.442591+0000","last_became_active":"2026-02-24T22:05:18.192543+0000","last_became_peered":"2026-02-24T22:05:18.192543+0000","last_unstale":"2026-02-24T22:34:37.442591+0000","last_undegraded":"2026-02-24T22:34:37.442591+0000","last_fullsized":"2026-02-24T22:34:37.442591+0000","mapping_epoch":24,"log_start":"312'500","ondisk_log_start":"312'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'596","last_scrub_stamp":"2026-02-24T22:34:37.442512+0000","last_deep_scrub":"312'596","last_deep_scrub_stamp":"2026-02-24T22:34:37.442512+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.442512+0000","objects_scrubbed":25,"log_size":96,"log_dups_size":500,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:07:01.563341+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":2576,"num_read_kb":2171,"num_write":966,"num_write_kb":601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"312'260","reported_seq":943,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.858767+0000","last_change":"2026-02-24T22:34:32.858731+0000","last_active":"2026-02-24T22:34:32.858767+0000","last_peered":"2026-02-24T22:34:32.858767+0000","last_clean":"2026-02-24T22:34:32.858767+0000","last_became_active":"2026-02-24T22:05:16.182928+0000","last_became_peered":"2026-02-24T22:05:16.182928+0000","last_unstale":"2026-02-24T22:34:32.858767+0000","last_undegraded":"2026-02-24T22:34:32.858767+0000","last_fullsized":"2026-02-24T22:34:32.858767+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'260","last_scrub_stamp":"2026-02-24T22:34:32.858687+0000","last_deep_scrub":"312'260","last_deep_scrub_stamp":"2026-02-24T22:34:32.858687+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.858687+0000","objects_scrubbed":5,"log_size":60,"log_dups_size":200,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:47:38.099528+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":316,"num_read_kb":262,"num_write":1083,"num_write_kb":181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":602,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.083526+0000","last_change":"2026-02-24T22:34:43.083526+0000","last_active":"2026-02-24T22:34:43.083526+0000","last_peered":"2026-02-24T22:34:43.083526+0000","last_clean":"2026-02-24T22:34:43.083526+0000","last_became_active":"2026-02-24T22:05:20.186209+0000","last_became_peered":"2026-02-24T22:05:20.186209+0000","last_unstale":"2026-02-24T22:34:43.083526+0000","last_undegraded":"2026-02-24T22:34:43.083526+0000","last_fullsized":"2026-02-24T22:34:43.083526+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:43.083470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:43.083470+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.083470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:53:12.978030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"312'948","reported_seq":2269,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.511658+0000","last_change":"2026-02-24T22:05:22.200373+0000","last_active":"2026-02-24T22:34:25.511658+0000","last_peered":"2026-02-24T22:34:25.511658+0000","last_clean":"2026-02-24T22:34:25.511658+0000","last_became_active":"2026-02-24T22:05:22.200111+0000","last_became_peered":"2026-02-24T22:05:22.200111+0000","last_unstale":"2026-02-24T22:34:25.511658+0000","last_undegraded":"2026-02-24T22:34:25.511658+0000","last_fullsized":"2026-02-24T22:34:25.511658+0000","mapping_epoch":28,"log_start":"312'900","ondisk_log_start":"312'900","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":900,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37978,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":3232,"num_read_kb":2382,"num_write":419,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"312'183","reported_seq":776,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.268947+0000","last_change":"2026-02-24T22:05:24.201417+0000","last_active":"2026-02-24T22:34:17.268947+0000","last_peered":"2026-02-24T22:34:17.268947+0000","last_clean":"2026-02-24T22:34:17.268947+0000","last_became_active":"2026-02-24T22:05:24.200878+0000","last_became_peered":"2026-02-24T22:05:24.200878+0000","last_unstale":"2026-02-24T22:34:17.268947+0000","last_undegraded":"2026-02-24T22:34:17.268947+0000","last_fullsized":"2026-02-24T22:34:17.268947+0000","mapping_epoch":30,"log_start":"37'100","ondisk_log_start":"37'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":100,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33021,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":110,"num_read_kb":109,"num_write":447,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"312'439","reported_seq":2200,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.316960+0000","last_change":"2026-02-24T22:05:29.654545+0000","last_active":"2026-02-24T22:34:25.316960+0000","last_peered":"2026-02-24T22:34:25.316960+0000","last_clean":"2026-02-24T22:34:25.316960+0000","last_became_active":"2026-02-24T22:05:29.654064+0000","last_became_peered":"2026-02-24T22:05:29.654064+0000","last_unstale":"2026-02-24T22:34:25.316960+0000","last_undegraded":"2026-02-24T22:34:25.316960+0000","last_fullsized":"2026-02-24T22:34:25.316960+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":400,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:01:24.704003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"312'149","reported_seq":841,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.467232+0000","last_change":"2026-02-24T22:34:38.467232+0000","last_active":"2026-02-24T22:34:38.467232+0000","last_peered":"2026-02-24T22:34:38.467232+0000","last_clean":"2026-02-24T22:34:38.467232+0000","last_became_active":"2026-02-24T22:05:18.191055+0000","last_became_peered":"2026-02-24T22:05:18.191055+0000","last_unstale":"2026-02-24T22:34:38.467232+0000","last_undegraded":"2026-02-24T22:34:38.467232+0000","last_fullsized":"2026-02-24T22:34:38.467232+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'149","last_scrub_stamp":"2026-02-24T22:34:38.467178+0000","last_deep_scrub":"312'149","last_deep_scrub_stamp":"2026-02-24T22:34:38.467178+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.467178+0000","objects_scrubbed":21,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:18:47.528959+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":486,"num_read_kb":398,"num_write":170,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"312'241","reported_seq":956,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.746504+0000","last_change":"2026-02-24T22:34:32.746504+0000","last_active":"2026-02-24T22:34:32.746504+0000","last_peered":"2026-02-24T22:34:32.746504+0000","last_clean":"2026-02-24T22:34:32.746504+0000","last_became_active":"2026-02-24T22:05:16.187261+0000","last_became_peered":"2026-02-24T22:05:16.187261+0000","last_unstale":"2026-02-24T22:34:32.746504+0000","last_undegraded":"2026-02-24T22:34:32.746504+0000","last_fullsized":"2026-02-24T22:34:32.746504+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'241","last_scrub_stamp":"2026-02-24T22:34:32.746460+0000","last_deep_scrub":"312'241","last_deep_scrub_stamp":"2026-02-24T22:34:32.746460+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.746460+0000","objects_scrubbed":1,"log_size":41,"log_dups_size":200,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:05:01.484992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":370,"num_read_kb":24764,"num_write":957,"num_write_kb":10562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":597,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.218231+0000","last_change":"2026-02-24T22:34:47.218231+0000","last_active":"2026-02-24T22:34:47.218231+0000","last_peered":"2026-02-24T22:34:47.218231+0000","last_clean":"2026-02-24T22:34:47.218231+0000","last_became_active":"2026-02-24T22:05:20.187488+0000","last_became_peered":"2026-02-24T22:05:20.187488+0000","last_unstale":"2026-02-24T22:34:47.218231+0000","last_undegraded":"2026-02-24T22:34:47.218231+0000","last_fullsized":"2026-02-24T22:34:47.218231+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:47.218164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:47.218164+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.218164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:23:53.051333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"312'475","reported_seq":1340,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.137231+0000","last_change":"2026-02-24T22:34:47.137231+0000","last_active":"2026-02-24T22:34:47.137231+0000","last_peered":"2026-02-24T22:34:47.137231+0000","last_clean":"2026-02-24T22:34:47.137231+0000","last_became_active":"2026-02-24T22:05:22.193142+0000","last_became_peered":"2026-02-24T22:05:22.193142+0000","last_unstale":"2026-02-24T22:34:47.137231+0000","last_undegraded":"2026-02-24T22:34:47.137231+0000","last_fullsized":"2026-02-24T22:34:47.137231+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'475","last_scrub_stamp":"2026-02-24T22:34:47.137162+0000","last_deep_scrub":"312'475","last_deep_scrub_stamp":"2026-02-24T22:34:47.137162+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.137162+0000","objects_scrubbed":8,"log_size":75,"log_dups_size":400,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:12:15.570963+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38313,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1308,"num_read_kb":1368,"num_write":335,"num_write_kb":281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"312'222","reported_seq":822,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.580120+0000","last_change":"2026-02-24T22:05:24.198994+0000","last_active":"2026-02-24T22:34:18.580120+0000","last_peered":"2026-02-24T22:34:18.580120+0000","last_clean":"2026-02-24T22:34:18.580120+0000","last_became_active":"2026-02-24T22:05:24.198669+0000","last_became_peered":"2026-02-24T22:05:24.198669+0000","last_unstale":"2026-02-24T22:34:18.580120+0000","last_undegraded":"2026-02-24T22:34:18.580120+0000","last_fullsized":"2026-02-24T22:34:18.580120+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":200,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29328,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":170,"num_read_kb":165,"num_write":551,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":576,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:34.733460+0000","last_change":"2026-02-24T22:05:29.653411+0000","last_active":"2026-02-24T22:32:34.733460+0000","last_peered":"2026-02-24T22:32:34.733460+0000","last_clean":"2026-02-24T22:32:34.733460+0000","last_became_active":"2026-02-24T22:05:29.653144+0000","last_became_peered":"2026-02-24T22:05:29.653144+0000","last_unstale":"2026-02-24T22:32:34.733460+0000","last_undegraded":"2026-02-24T22:32:34.733460+0000","last_fullsized":"2026-02-24T22:32:34.733460+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"312'194","reported_seq":908,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.515320+0000","last_change":"2026-02-24T22:34:39.515320+0000","last_active":"2026-02-24T22:34:39.515320+0000","last_peered":"2026-02-24T22:34:39.515320+0000","last_clean":"2026-02-24T22:34:39.515320+0000","last_became_active":"2026-02-24T22:05:18.190140+0000","last_became_peered":"2026-02-24T22:05:18.190140+0000","last_unstale":"2026-02-24T22:34:39.515320+0000","last_undegraded":"2026-02-24T22:34:39.515320+0000","last_fullsized":"2026-02-24T22:34:39.515320+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'194","last_scrub_stamp":"2026-02-24T22:34:39.515258+0000","last_deep_scrub":"312'194","last_deep_scrub_stamp":"2026-02-24T22:34:39.515258+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.515258+0000","objects_scrubbed":23,"log_size":94,"log_dups_size":100,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:49:24.504968+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":675,"num_read_kb":556,"num_write":238,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"312'303","reported_seq":1072,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.577662+0000","last_change":"2026-02-24T22:34:32.577662+0000","last_active":"2026-02-24T22:34:32.577662+0000","last_peered":"2026-02-24T22:34:32.577662+0000","last_clean":"2026-02-24T22:34:32.577662+0000","last_became_active":"2026-02-24T22:05:16.184944+0000","last_became_peered":"2026-02-24T22:05:16.184944+0000","last_unstale":"2026-02-24T22:34:32.577662+0000","last_undegraded":"2026-02-24T22:34:32.577662+0000","last_fullsized":"2026-02-24T22:34:32.577662+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'303","last_scrub_stamp":"2026-02-24T22:34:32.577606+0000","last_deep_scrub":"312'303","last_deep_scrub_stamp":"2026-02-24T22:34:32.577606+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.577606+0000","objects_scrubbed":3,"log_size":103,"log_dups_size":200,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:59:47.981464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":538,"num_read_kb":45888,"num_write":1354,"num_write_kb":20430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"27'2","reported_seq":1058,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.173187+0000","last_change":"2026-02-24T22:05:20.194961+0000","last_active":"2026-02-24T22:34:25.173187+0000","last_peered":"2026-02-24T22:34:25.173187+0000","last_clean":"2026-02-24T22:34:25.173187+0000","last_became_active":"2026-02-24T22:05:20.194145+0000","last_became_peered":"2026-02-24T22:05:20.194145+0000","last_unstale":"2026-02-24T22:34:25.173187+0000","last_undegraded":"2026-02-24T22:34:25.173187+0000","last_fullsized":"2026-02-24T22:34:25.173187+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":476,"num_read_kb":476,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"312'56","reported_seq":658,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:12.706235+0000","last_change":"2026-02-24T22:05:22.905477+0000","last_active":"2026-02-24T22:34:12.706235+0000","last_peered":"2026-02-24T22:34:12.706235+0000","last_clean":"2026-02-24T22:34:12.706235+0000","last_became_active":"2026-02-24T22:05:22.905221+0000","last_became_peered":"2026-02-24T22:05:22.905221+0000","last_unstale":"2026-02-24T22:34:12.706235+0000","last_undegraded":"2026-02-24T22:34:12.706235+0000","last_fullsized":"2026-02-24T22:34:12.706235+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"312'246","reported_seq":845,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.685022+0000","last_change":"2026-02-24T22:34:46.685022+0000","last_active":"2026-02-24T22:34:46.685022+0000","last_peered":"2026-02-24T22:34:46.685022+0000","last_clean":"2026-02-24T22:34:46.685022+0000","last_became_active":"2026-02-24T22:05:24.207673+0000","last_became_peered":"2026-02-24T22:05:24.207673+0000","last_unstale":"2026-02-24T22:34:46.685022+0000","last_undegraded":"2026-02-24T22:34:46.685022+0000","last_fullsized":"2026-02-24T22:34:46.685022+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'246","last_scrub_stamp":"2026-02-24T22:34:46.684962+0000","last_deep_scrub":"312'246","last_deep_scrub_stamp":"2026-02-24T22:34:46.684962+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.684962+0000","objects_scrubbed":87,"log_size":46,"log_dups_size":200,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:17:30.101644+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25987,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":107,"num_read_kb":103,"num_write":388,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":578,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:34.747683+0000","last_change":"2026-02-24T22:05:29.659816+0000","last_active":"2026-02-24T22:32:34.747683+0000","last_peered":"2026-02-24T22:32:34.747683+0000","last_clean":"2026-02-24T22:32:34.747683+0000","last_became_active":"2026-02-24T22:05:29.658842+0000","last_became_peered":"2026-02-24T22:05:29.658842+0000","last_unstale":"2026-02-24T22:32:34.747683+0000","last_undegraded":"2026-02-24T22:32:34.747683+0000","last_fullsized":"2026-02-24T22:32:34.747683+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"312'122","reported_seq":812,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.133092+0000","last_change":"2026-02-24T22:34:44.133092+0000","last_active":"2026-02-24T22:34:44.133092+0000","last_peered":"2026-02-24T22:34:44.133092+0000","last_clean":"2026-02-24T22:34:44.133092+0000","last_became_active":"2026-02-24T22:05:18.188161+0000","last_became_peered":"2026-02-24T22:05:18.188161+0000","last_unstale":"2026-02-24T22:34:44.133092+0000","last_undegraded":"2026-02-24T22:34:44.133092+0000","last_fullsized":"2026-02-24T22:34:44.133092+0000","mapping_epoch":24,"log_start":"312'98","ondisk_log_start":"312'98","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'122","last_scrub_stamp":"2026-02-24T22:34:44.133036+0000","last_deep_scrub":"312'122","last_deep_scrub_stamp":"2026-02-24T22:34:44.133036+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.133036+0000","objects_scrubbed":24,"log_size":24,"log_dups_size":98,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:15:18.743316+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":368,"num_read_kb":314,"num_write":134,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"312'360","reported_seq":1100,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.843071+0000","last_change":"2026-02-24T22:34:31.843071+0000","last_active":"2026-02-24T22:34:31.843071+0000","last_peered":"2026-02-24T22:34:31.843071+0000","last_clean":"2026-02-24T22:34:31.843071+0000","last_became_active":"2026-02-24T22:05:16.182194+0000","last_became_peered":"2026-02-24T22:05:16.182194+0000","last_unstale":"2026-02-24T22:34:31.843071+0000","last_undegraded":"2026-02-24T22:34:31.843071+0000","last_fullsized":"2026-02-24T22:34:31.843071+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'360","last_scrub_stamp":"2026-02-24T22:34:31.843001+0000","last_deep_scrub":"312'360","last_deep_scrub_stamp":"2026-02-24T22:34:31.843001+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.843001+0000","objects_scrubbed":4,"log_size":60,"log_dups_size":300,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:54:58.411257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":482,"num_read_kb":7457,"num_write":1288,"num_write_kb":8933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":591,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:19.745978+0000","last_change":"2026-02-24T22:05:20.193494+0000","last_active":"2026-02-24T22:32:19.745978+0000","last_peered":"2026-02-24T22:32:19.745978+0000","last_clean":"2026-02-24T22:32:19.745978+0000","last_became_active":"2026-02-24T22:05:20.193180+0000","last_became_peered":"2026-02-24T22:05:20.193180+0000","last_unstale":"2026-02-24T22:32:19.745978+0000","last_undegraded":"2026-02-24T22:32:19.745978+0000","last_fullsized":"2026-02-24T22:32:19.745978+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"312'275","reported_seq":901,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.782130+0000","last_change":"2026-02-24T22:05:22.200410+0000","last_active":"2026-02-24T22:34:25.782130+0000","last_peered":"2026-02-24T22:34:25.782130+0000","last_clean":"2026-02-24T22:34:25.782130+0000","last_became_active":"2026-02-24T22:05:22.200138+0000","last_became_peered":"2026-02-24T22:05:22.200138+0000","last_unstale":"2026-02-24T22:34:25.782130+0000","last_undegraded":"2026-02-24T22:34:25.782130+0000","last_fullsized":"2026-02-24T22:34:25.782130+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":200,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37516,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":871,"num_read_kb":1412,"num_write":1276,"num_write_kb":1198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"312'415","reported_seq":1053,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.648295+0000","last_change":"2026-02-24T22:34:47.648295+0000","last_active":"2026-02-24T22:34:47.648295+0000","last_peered":"2026-02-24T22:34:47.648295+0000","last_clean":"2026-02-24T22:34:47.648295+0000","last_became_active":"2026-02-24T22:05:24.206745+0000","last_became_peered":"2026-02-24T22:05:24.206745+0000","last_unstale":"2026-02-24T22:34:47.648295+0000","last_undegraded":"2026-02-24T22:34:47.648295+0000","last_fullsized":"2026-02-24T22:34:47.648295+0000","mapping_epoch":30,"log_start":"312'400","ondisk_log_start":"312'400","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'415","last_scrub_stamp":"2026-02-24T22:34:47.648223+0000","last_deep_scrub":"312'415","last_deep_scrub_stamp":"2026-02-24T22:34:47.648223+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.648223+0000","objects_scrubbed":88,"log_size":15,"log_dups_size":400,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:44:43.640744+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27098,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":391,"num_read_kb":376,"num_write":726,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":178,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"312'435","reported_seq":4800,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.998387+0000","last_change":"2026-02-24T22:05:29.654842+0000","last_active":"2026-02-24T22:34:25.998387+0000","last_peered":"2026-02-24T22:34:25.998387+0000","last_clean":"2026-02-24T22:34:25.998387+0000","last_became_active":"2026-02-24T22:05:29.654281+0000","last_became_peered":"2026-02-24T22:05:29.654281+0000","last_unstale":"2026-02-24T22:34:25.998387+0000","last_undegraded":"2026-02-24T22:34:25.998387+0000","last_fullsized":"2026-02-24T22:34:25.998387+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":400,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"312'136","reported_seq":831,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.733879+0000","last_change":"2026-02-24T22:34:38.733879+0000","last_active":"2026-02-24T22:34:38.733879+0000","last_peered":"2026-02-24T22:34:38.733879+0000","last_clean":"2026-02-24T22:34:38.733879+0000","last_became_active":"2026-02-24T22:05:18.193426+0000","last_became_peered":"2026-02-24T22:05:18.193426+0000","last_unstale":"2026-02-24T22:34:38.733879+0000","last_undegraded":"2026-02-24T22:34:38.733879+0000","last_fullsized":"2026-02-24T22:34:38.733879+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'136","last_scrub_stamp":"2026-02-24T22:34:38.733817+0000","last_deep_scrub":"312'136","last_deep_scrub_stamp":"2026-02-24T22:34:38.733817+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.733817+0000","objects_scrubbed":22,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:36:52.810608+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":443,"num_read_kb":376,"num_write":199,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"312'114","reported_seq":755,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.176106+0000","last_change":"2026-02-24T22:34:32.176106+0000","last_active":"2026-02-24T22:34:32.176106+0000","last_peered":"2026-02-24T22:34:32.176106+0000","last_clean":"2026-02-24T22:34:32.176106+0000","last_became_active":"2026-02-24T22:05:06.708577+0000","last_became_peered":"2026-02-24T22:05:06.708577+0000","last_unstale":"2026-02-24T22:34:32.176106+0000","last_undegraded":"2026-02-24T22:34:32.176106+0000","last_fullsized":"2026-02-24T22:34:32.176106+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'114","last_scrub_stamp":"2026-02-24T22:34:32.176027+0000","last_deep_scrub":"312'114","last_deep_scrub_stamp":"2026-02-24T22:34:32.176027+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.176027+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:58:53.542668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"312'2","reported_seq":614,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.096943+0000","last_change":"2026-02-24T22:34:34.096943+0000","last_active":"2026-02-24T22:34:34.096943+0000","last_peered":"2026-02-24T22:34:34.096943+0000","last_clean":"2026-02-24T22:34:34.096943+0000","last_became_active":"2026-02-24T22:05:08.689983+0000","last_became_peered":"2026-02-24T22:05:08.689983+0000","last_unstale":"2026-02-24T22:34:34.096943+0000","last_undegraded":"2026-02-24T22:34:34.096943+0000","last_fullsized":"2026-02-24T22:34:34.096943+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:34:34.096879+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:34:34.096879+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.096879+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:03:29.099395+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00046251899999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"312'329","reported_seq":1116,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:29.895013+0000","last_change":"2026-02-24T22:34:29.895013+0000","last_active":"2026-02-24T22:34:29.895013+0000","last_peered":"2026-02-24T22:34:29.895013+0000","last_clean":"2026-02-24T22:34:29.895013+0000","last_became_active":"2026-02-24T22:05:16.185768+0000","last_became_peered":"2026-02-24T22:05:16.185768+0000","last_unstale":"2026-02-24T22:34:29.895013+0000","last_undegraded":"2026-02-24T22:34:29.895013+0000","last_fullsized":"2026-02-24T22:34:29.895013+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'329","last_scrub_stamp":"2026-02-24T22:34:29.894945+0000","last_deep_scrub":"312'329","last_deep_scrub_stamp":"2026-02-24T22:34:29.894945+0000","last_clean_scrub_stamp":"2026-02-24T22:34:29.894945+0000","objects_scrubbed":3,"log_size":29,"log_dups_size":300,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:02:39.272423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":685,"num_read_kb":12844,"num_write":1503,"num_write_kb":6232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":595,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.125003+0000","last_change":"2026-02-24T22:34:49.125003+0000","last_active":"2026-02-24T22:34:49.125003+0000","last_peered":"2026-02-24T22:34:49.125003+0000","last_clean":"2026-02-24T22:34:49.125003+0000","last_became_active":"2026-02-24T22:05:20.194040+0000","last_became_peered":"2026-02-24T22:05:20.194040+0000","last_unstale":"2026-02-24T22:34:49.125003+0000","last_undegraded":"2026-02-24T22:34:49.125003+0000","last_fullsized":"2026-02-24T22:34:49.125003+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:49.124939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:49.124939+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.124939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:16:47.321676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"312'1066","reported_seq":2175,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.830271+0000","last_change":"2026-02-24T22:05:22.199991+0000","last_active":"2026-02-24T22:34:25.830271+0000","last_peered":"2026-02-24T22:34:25.830271+0000","last_clean":"2026-02-24T22:34:25.830271+0000","last_became_active":"2026-02-24T22:05:22.199602+0000","last_became_peered":"2026-02-24T22:05:22.199602+0000","last_unstale":"2026-02-24T22:34:25.830271+0000","last_undegraded":"2026-02-24T22:34:25.830271+0000","last_fullsized":"2026-02-24T22:34:25.830271+0000","mapping_epoch":28,"log_start":"312'1000","ondisk_log_start":"312'1000","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":1000,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:00:35.201833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":116028,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2941,"num_read_kb":3705,"num_write":1026,"num_write_kb":950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"312'202","reported_seq":795,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.242020+0000","last_change":"2026-02-24T22:05:24.201041+0000","last_active":"2026-02-24T22:34:18.242020+0000","last_peered":"2026-02-24T22:34:18.242020+0000","last_clean":"2026-02-24T22:34:18.242020+0000","last_became_active":"2026-02-24T22:05:24.200336+0000","last_became_peered":"2026-02-24T22:05:24.200336+0000","last_unstale":"2026-02-24T22:34:18.242020+0000","last_undegraded":"2026-02-24T22:34:18.242020+0000","last_fullsized":"2026-02-24T22:34:18.242020+0000","mapping_epoch":30,"log_start":"37'100","ondisk_log_start":"37'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:52:42.524705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32535,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":152,"num_read_kb":139,"num_write":501,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"312'428","reported_seq":1836,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.720314+0000","last_change":"2026-02-24T22:05:29.660713+0000","last_active":"2026-02-24T22:34:25.720314+0000","last_peered":"2026-02-24T22:34:25.720314+0000","last_clean":"2026-02-24T22:34:25.720314+0000","last_became_active":"2026-02-24T22:05:29.660370+0000","last_became_peered":"2026-02-24T22:05:29.660370+0000","last_unstale":"2026-02-24T22:34:25.720314+0000","last_undegraded":"2026-02-24T22:34:25.720314+0000","last_fullsized":"2026-02-24T22:34:25.720314+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":400,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:47:59.912901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"312'124","reported_seq":831,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.850193+0000","last_change":"2026-02-24T22:34:41.850193+0000","last_active":"2026-02-24T22:34:41.850193+0000","last_peered":"2026-02-24T22:34:41.850193+0000","last_clean":"2026-02-24T22:34:41.850193+0000","last_became_active":"2026-02-24T22:05:18.185205+0000","last_became_peered":"2026-02-24T22:05:18.185205+0000","last_unstale":"2026-02-24T22:34:41.850193+0000","last_undegraded":"2026-02-24T22:34:41.850193+0000","last_fullsized":"2026-02-24T22:34:41.850193+0000","mapping_epoch":24,"log_start":"312'99","ondisk_log_start":"312'99","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'124","last_scrub_stamp":"2026-02-24T22:34:41.850136+0000","last_deep_scrub":"312'124","last_deep_scrub_stamp":"2026-02-24T22:34:41.850136+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.850136+0000","objects_scrubbed":25,"log_size":25,"log_dups_size":99,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:53:33.276473+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":346,"num_read_kb":300,"num_write":140,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":618,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.150171+0000","last_change":"2026-02-24T22:34:36.150171+0000","last_active":"2026-02-24T22:34:36.150171+0000","last_peered":"2026-02-24T22:34:36.150171+0000","last_clean":"2026-02-24T22:34:36.150171+0000","last_became_active":"2026-02-24T22:05:08.689972+0000","last_became_peered":"2026-02-24T22:05:08.689972+0000","last_unstale":"2026-02-24T22:34:36.150171+0000","last_undegraded":"2026-02-24T22:34:36.150171+0000","last_fullsized":"2026-02-24T22:34:36.150171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:36.150114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:36.150114+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.150114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:06:01.715910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048345700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"312'262","reported_seq":1029,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:29.701991+0000","last_change":"2026-02-24T22:34:29.701952+0000","last_active":"2026-02-24T22:34:29.701991+0000","last_peered":"2026-02-24T22:34:29.701991+0000","last_clean":"2026-02-24T22:34:29.701991+0000","last_became_active":"2026-02-24T22:05:16.183484+0000","last_became_peered":"2026-02-24T22:05:16.183484+0000","last_unstale":"2026-02-24T22:34:29.701991+0000","last_undegraded":"2026-02-24T22:34:29.701991+0000","last_fullsized":"2026-02-24T22:34:29.701991+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'262","last_scrub_stamp":"2026-02-24T22:34:29.701911+0000","last_deep_scrub":"312'262","last_deep_scrub_stamp":"2026-02-24T22:34:29.701911+0000","last_clean_scrub_stamp":"2026-02-24T22:34:29.701911+0000","objects_scrubbed":4,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:16:54.158907+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":574,"num_read_kb":821,"num_write":1287,"num_write_kb":10723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":580,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.741456+0000","last_change":"2026-02-24T22:05:20.195460+0000","last_active":"2026-02-24T22:31:19.741456+0000","last_peered":"2026-02-24T22:31:19.741456+0000","last_clean":"2026-02-24T22:31:19.741456+0000","last_became_active":"2026-02-24T22:05:20.195236+0000","last_became_peered":"2026-02-24T22:05:20.195236+0000","last_unstale":"2026-02-24T22:31:19.741456+0000","last_undegraded":"2026-02-24T22:31:19.741456+0000","last_fullsized":"2026-02-24T22:31:19.741456+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:38:04.823028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"312'1062","reported_seq":2285,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.995996+0000","last_change":"2026-02-24T22:05:22.192588+0000","last_active":"2026-02-24T22:34:25.995996+0000","last_peered":"2026-02-24T22:34:25.995996+0000","last_clean":"2026-02-24T22:34:25.995996+0000","last_became_active":"2026-02-24T22:05:22.192295+0000","last_became_peered":"2026-02-24T22:05:22.192295+0000","last_unstale":"2026-02-24T22:34:25.995996+0000","last_undegraded":"2026-02-24T22:34:25.995996+0000","last_fullsized":"2026-02-24T22:34:25.995996+0000","mapping_epoch":28,"log_start":"312'1001","ondisk_log_start":"312'1001","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":1001,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":114655,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":3192,"num_read_kb":4794,"num_write":3958,"num_write_kb":3891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"312'182","reported_seq":764,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.166280+0000","last_change":"2026-02-24T22:05:24.205091+0000","last_active":"2026-02-24T22:34:15.166280+0000","last_peered":"2026-02-24T22:34:15.166280+0000","last_clean":"2026-02-24T22:34:15.166280+0000","last_became_active":"2026-02-24T22:05:24.204774+0000","last_became_peered":"2026-02-24T22:05:24.204774+0000","last_unstale":"2026-02-24T22:34:15.166280+0000","last_undegraded":"2026-02-24T22:34:15.166280+0000","last_fullsized":"2026-02-24T22:34:15.166280+0000","mapping_epoch":30,"log_start":"312'100","ondisk_log_start":"312'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":100,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29814,"num_objects":89,"num_object_clones":0,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":0,"num_read":112,"num_read_kb":108,"num_write":454,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.727262+0000","last_change":"2026-02-24T22:05:29.653742+0000","last_active":"2026-02-24T22:31:34.727262+0000","last_peered":"2026-02-24T22:31:34.727262+0000","last_clean":"2026-02-24T22:31:34.727262+0000","last_became_active":"2026-02-24T22:05:29.653391+0000","last_became_peered":"2026-02-24T22:05:29.653391+0000","last_unstale":"2026-02-24T22:31:34.727262+0000","last_undegraded":"2026-02-24T22:31:34.727262+0000","last_fullsized":"2026-02-24T22:31:34.727262+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:11:46.647600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"312'130","reported_seq":829,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.122528+0000","last_change":"2026-02-24T22:34:42.122528+0000","last_active":"2026-02-24T22:34:42.122528+0000","last_peered":"2026-02-24T22:34:42.122528+0000","last_clean":"2026-02-24T22:34:42.122528+0000","last_became_active":"2026-02-24T22:05:18.186126+0000","last_became_peered":"2026-02-24T22:05:18.186126+0000","last_unstale":"2026-02-24T22:34:42.122528+0000","last_undegraded":"2026-02-24T22:34:42.122528+0000","last_fullsized":"2026-02-24T22:34:42.122528+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'130","last_scrub_stamp":"2026-02-24T22:34:42.122496+0000","last_deep_scrub":"312'130","last_deep_scrub_stamp":"2026-02-24T22:34:42.122496+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.122496+0000","objects_scrubbed":29,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:13:06.464986+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":336,"num_read_kb":289,"num_write":128,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":626,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:30.093328+0000","last_change":"2026-02-24T22:34:30.093328+0000","last_active":"2026-02-24T22:34:30.093328+0000","last_peered":"2026-02-24T22:34:30.093328+0000","last_clean":"2026-02-24T22:34:30.093328+0000","last_became_active":"2026-02-24T22:05:08.682035+0000","last_became_peered":"2026-02-24T22:05:08.682035+0000","last_unstale":"2026-02-24T22:34:30.093328+0000","last_undegraded":"2026-02-24T22:34:30.093328+0000","last_fullsized":"2026-02-24T22:34:30.093328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:30.093252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:30.093252+0000","last_clean_scrub_stamp":"2026-02-24T22:34:30.093252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:06:58.991217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047531599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"312'305","reported_seq":1057,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.010886+0000","last_change":"2026-02-24T22:34:34.010886+0000","last_active":"2026-02-24T22:34:34.010886+0000","last_peered":"2026-02-24T22:34:34.010886+0000","last_clean":"2026-02-24T22:34:34.010886+0000","last_became_active":"2026-02-24T22:05:16.178331+0000","last_became_peered":"2026-02-24T22:05:16.178331+0000","last_unstale":"2026-02-24T22:34:34.010886+0000","last_undegraded":"2026-02-24T22:34:34.010886+0000","last_fullsized":"2026-02-24T22:34:34.010886+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'305","last_scrub_stamp":"2026-02-24T22:34:34.010799+0000","last_deep_scrub":"312'305","last_deep_scrub_stamp":"2026-02-24T22:34:34.010799+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.010799+0000","objects_scrubbed":6,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:33:52.230516+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":583,"num_read_kb":6589,"num_write":1333,"num_write_kb":11354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"312'2","reported_seq":604,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.063471+0000","last_change":"2026-02-24T22:34:42.063471+0000","last_active":"2026-02-24T22:34:42.063471+0000","last_peered":"2026-02-24T22:34:42.063471+0000","last_clean":"2026-02-24T22:34:42.063471+0000","last_became_active":"2026-02-24T22:05:20.187908+0000","last_became_peered":"2026-02-24T22:05:20.187908+0000","last_unstale":"2026-02-24T22:34:42.063471+0000","last_undegraded":"2026-02-24T22:34:42.063471+0000","last_fullsized":"2026-02-24T22:34:42.063471+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:34:42.063424+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:34:42.063424+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.063424+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:16:34.733800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"312'437","reported_seq":1647,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.225332+0000","last_change":"2026-02-24T22:05:22.912798+0000","last_active":"2026-02-24T22:34:25.225332+0000","last_peered":"2026-02-24T22:34:25.225332+0000","last_clean":"2026-02-24T22:34:25.225332+0000","last_became_active":"2026-02-24T22:05:22.912414+0000","last_became_peered":"2026-02-24T22:05:22.912414+0000","last_unstale":"2026-02-24T22:34:25.225332+0000","last_undegraded":"2026-02-24T22:34:25.225332+0000","last_fullsized":"2026-02-24T22:34:25.225332+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":400,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":308,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1080,"num_read_kb":599,"num_write":114,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"312'184","reported_seq":771,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.556627+0000","last_change":"2026-02-24T22:05:24.200671+0000","last_active":"2026-02-24T22:34:18.556627+0000","last_peered":"2026-02-24T22:34:18.556627+0000","last_clean":"2026-02-24T22:34:18.556627+0000","last_became_active":"2026-02-24T22:05:24.200029+0000","last_became_peered":"2026-02-24T22:05:24.200029+0000","last_unstale":"2026-02-24T22:34:18.556627+0000","last_undegraded":"2026-02-24T22:34:18.556627+0000","last_fullsized":"2026-02-24T22:34:18.556627+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":100,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29918,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":442,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":577,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:29.737838+0000","last_change":"2026-02-24T22:05:29.653992+0000","last_active":"2026-02-24T22:32:29.737838+0000","last_peered":"2026-02-24T22:32:29.737838+0000","last_clean":"2026-02-24T22:32:29.737838+0000","last_became_active":"2026-02-24T22:05:29.653806+0000","last_became_peered":"2026-02-24T22:05:29.653806+0000","last_unstale":"2026-02-24T22:32:29.737838+0000","last_undegraded":"2026-02-24T22:32:29.737838+0000","last_fullsized":"2026-02-24T22:32:29.737838+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"312'254","reported_seq":1004,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.068312+0000","last_change":"2026-02-24T22:34:38.068312+0000","last_active":"2026-02-24T22:34:38.068312+0000","last_peered":"2026-02-24T22:34:38.068312+0000","last_clean":"2026-02-24T22:34:38.068312+0000","last_became_active":"2026-02-24T22:05:18.183952+0000","last_became_peered":"2026-02-24T22:05:18.183952+0000","last_unstale":"2026-02-24T22:34:38.068312+0000","last_undegraded":"2026-02-24T22:34:38.068312+0000","last_fullsized":"2026-02-24T22:34:38.068312+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'254","last_scrub_stamp":"2026-02-24T22:34:38.068260+0000","last_deep_scrub":"312'254","last_deep_scrub_stamp":"2026-02-24T22:34:38.068260+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.068260+0000","objects_scrubbed":24,"log_size":54,"log_dups_size":200,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:24:48.806854+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":963,"num_read_kb":811,"num_write":374,"num_write_kb":210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"312'3","reported_seq":615,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.112722+0000","last_change":"2026-02-24T22:34:31.112722+0000","last_active":"2026-02-24T22:34:31.112722+0000","last_peered":"2026-02-24T22:34:31.112722+0000","last_clean":"2026-02-24T22:34:31.112722+0000","last_became_active":"2026-02-24T22:05:08.690042+0000","last_became_peered":"2026-02-24T22:05:08.690042+0000","last_unstale":"2026-02-24T22:34:31.112722+0000","last_undegraded":"2026-02-24T22:34:31.112722+0000","last_fullsized":"2026-02-24T22:34:31.112722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:34:31.112652+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:34:31.112652+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.112652+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:38:30.546771+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0010128400000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"312'333","reported_seq":1056,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:30.596222+0000","last_change":"2026-02-24T22:34:30.596222+0000","last_active":"2026-02-24T22:34:30.596222+0000","last_peered":"2026-02-24T22:34:30.596222+0000","last_clean":"2026-02-24T22:34:30.596222+0000","last_became_active":"2026-02-24T22:05:16.188900+0000","last_became_peered":"2026-02-24T22:05:16.188900+0000","last_unstale":"2026-02-24T22:34:30.596222+0000","last_undegraded":"2026-02-24T22:34:30.596222+0000","last_fullsized":"2026-02-24T22:34:30.596222+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'333","last_scrub_stamp":"2026-02-24T22:34:30.596162+0000","last_deep_scrub":"312'333","last_deep_scrub_stamp":"2026-02-24T22:34:30.596162+0000","last_clean_scrub_stamp":"2026-02-24T22:34:30.596162+0000","objects_scrubbed":6,"log_size":33,"log_dups_size":300,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:56:38.748514+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242895,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":415,"num_read_kb":10969,"num_write":1277,"num_write_kb":10859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:29.747283+0000","last_change":"2026-02-24T22:05:20.187605+0000","last_active":"2026-02-24T22:32:29.747283+0000","last_peered":"2026-02-24T22:32:29.747283+0000","last_clean":"2026-02-24T22:32:29.747283+0000","last_became_active":"2026-02-24T22:05:20.187251+0000","last_became_peered":"2026-02-24T22:05:20.187251+0000","last_unstale":"2026-02-24T22:32:29.747283+0000","last_undegraded":"2026-02-24T22:32:29.747283+0000","last_fullsized":"2026-02-24T22:32:29.747283+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"312'76","reported_seq":697,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.731188+0000","last_change":"2026-02-24T22:05:22.191860+0000","last_active":"2026-02-24T22:31:24.731188+0000","last_peered":"2026-02-24T22:31:24.731188+0000","last_clean":"2026-02-24T22:31:24.731188+0000","last_became_active":"2026-02-24T22:05:22.191394+0000","last_became_peered":"2026-02-24T22:05:22.191394+0000","last_unstale":"2026-02-24T22:31:24.731188+0000","last_undegraded":"2026-02-24T22:31:24.731188+0000","last_fullsized":"2026-02-24T22:31:24.731188+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:12:53.711063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":103,"num_read_kb":96,"num_write":72,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"312'323","reported_seq":951,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.637677+0000","last_change":"2026-02-24T22:34:48.637677+0000","last_active":"2026-02-24T22:34:48.637677+0000","last_peered":"2026-02-24T22:34:48.637677+0000","last_clean":"2026-02-24T22:34:48.637677+0000","last_became_active":"2026-02-24T22:05:24.207698+0000","last_became_peered":"2026-02-24T22:05:24.207698+0000","last_unstale":"2026-02-24T22:34:48.637677+0000","last_undegraded":"2026-02-24T22:34:48.637677+0000","last_fullsized":"2026-02-24T22:34:48.637677+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'323","last_scrub_stamp":"2026-02-24T22:34:48.637620+0000","last_deep_scrub":"312'323","last_deep_scrub_stamp":"2026-02-24T22:34:48.637620+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.637620+0000","objects_scrubbed":94,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:13:22.580314+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31273,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":246,"num_read_kb":224,"num_write":553,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":566,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:29.742534+0000","last_change":"2026-02-24T22:05:29.660487+0000","last_active":"2026-02-24T22:31:29.742534+0000","last_peered":"2026-02-24T22:31:29.742534+0000","last_clean":"2026-02-24T22:31:29.742534+0000","last_became_active":"2026-02-24T22:05:29.660065+0000","last_became_peered":"2026-02-24T22:05:29.660065+0000","last_unstale":"2026-02-24T22:31:29.742534+0000","last_undegraded":"2026-02-24T22:31:29.742534+0000","last_fullsized":"2026-02-24T22:31:29.742534+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:00:34.244351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"312'191","reported_seq":898,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.101612+0000","last_change":"2026-02-24T22:34:39.101612+0000","last_active":"2026-02-24T22:34:39.101612+0000","last_peered":"2026-02-24T22:34:39.101612+0000","last_clean":"2026-02-24T22:34:39.101612+0000","last_became_active":"2026-02-24T22:05:18.185112+0000","last_became_peered":"2026-02-24T22:05:18.185112+0000","last_unstale":"2026-02-24T22:34:39.101612+0000","last_undegraded":"2026-02-24T22:34:39.101612+0000","last_fullsized":"2026-02-24T22:34:39.101612+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'191","last_scrub_stamp":"2026-02-24T22:34:39.101566+0000","last_deep_scrub":"312'191","last_deep_scrub_stamp":"2026-02-24T22:34:39.101566+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.101566+0000","objects_scrubbed":19,"log_size":91,"log_dups_size":100,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:44:31.875433+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":679,"num_read_kb":553,"num_write":246,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"312'281","reported_seq":1011,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.103881+0000","last_change":"2026-02-24T22:34:32.103832+0000","last_active":"2026-02-24T22:34:32.103881+0000","last_peered":"2026-02-24T22:34:32.103881+0000","last_clean":"2026-02-24T22:34:32.103881+0000","last_became_active":"2026-02-24T22:05:16.191007+0000","last_became_peered":"2026-02-24T22:05:16.191007+0000","last_unstale":"2026-02-24T22:34:32.103881+0000","last_undegraded":"2026-02-24T22:34:32.103881+0000","last_fullsized":"2026-02-24T22:34:32.103881+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'281","last_scrub_stamp":"2026-02-24T22:34:32.103780+0000","last_deep_scrub":"312'281","last_deep_scrub_stamp":"2026-02-24T22:34:32.103780+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.103780+0000","objects_scrubbed":4,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:02:28.007749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":495,"num_read_kb":40330,"num_write":1338,"num_write_kb":18546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":621,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:29.991920+0000","last_change":"2026-02-24T22:34:29.991920+0000","last_active":"2026-02-24T22:34:29.991920+0000","last_peered":"2026-02-24T22:34:29.991920+0000","last_clean":"2026-02-24T22:34:29.991920+0000","last_became_active":"2026-02-24T22:05:08.758282+0000","last_became_peered":"2026-02-24T22:05:08.758282+0000","last_unstale":"2026-02-24T22:34:29.991920+0000","last_undegraded":"2026-02-24T22:34:29.991920+0000","last_fullsized":"2026-02-24T22:34:29.991920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:29.991857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:29.991857+0000","last_clean_scrub_stamp":"2026-02-24T22:34:29.991857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:47:43.334173+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00033355899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":598,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.948604+0000","last_change":"2026-02-24T22:34:48.948604+0000","last_active":"2026-02-24T22:34:48.948604+0000","last_peered":"2026-02-24T22:34:48.948604+0000","last_clean":"2026-02-24T22:34:48.948604+0000","last_became_active":"2026-02-24T22:05:20.187208+0000","last_became_peered":"2026-02-24T22:05:20.187208+0000","last_unstale":"2026-02-24T22:34:48.948604+0000","last_undegraded":"2026-02-24T22:34:48.948604+0000","last_fullsized":"2026-02-24T22:34:48.948604+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:48.948541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:48.948541+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.948541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:29:13.712233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"312'1133","reported_seq":2487,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.681905+0000","last_change":"2026-02-24T22:05:22.192049+0000","last_active":"2026-02-24T22:34:25.681905+0000","last_peered":"2026-02-24T22:34:25.681905+0000","last_clean":"2026-02-24T22:34:25.681905+0000","last_became_active":"2026-02-24T22:05:22.191763+0000","last_became_peered":"2026-02-24T22:05:22.191763+0000","last_unstale":"2026-02-24T22:34:25.681905+0000","last_undegraded":"2026-02-24T22:34:25.681905+0000","last_fullsized":"2026-02-24T22:34:25.681905+0000","mapping_epoch":28,"log_start":"312'1100","ondisk_log_start":"312'1100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":1100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:42:04.815445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":76731,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":3517,"num_read_kb":3087,"num_write":637,"num_write_kb":534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"312'204","reported_seq":809,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.409454+0000","last_change":"2026-02-24T22:05:24.206550+0000","last_active":"2026-02-24T22:34:18.409454+0000","last_peered":"2026-02-24T22:34:18.409454+0000","last_clean":"2026-02-24T22:34:18.409454+0000","last_became_active":"2026-02-24T22:05:24.205923+0000","last_became_peered":"2026-02-24T22:05:24.205923+0000","last_unstale":"2026-02-24T22:34:18.409454+0000","last_undegraded":"2026-02-24T22:34:18.409454+0000","last_fullsized":"2026-02-24T22:34:18.409454+0000","mapping_epoch":30,"log_start":"37'100","ondisk_log_start":"37'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34551,"num_objects":102,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":190,"num_read_kb":174,"num_write":516,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"312'467","reported_seq":4830,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.998086+0000","last_change":"2026-02-24T22:05:29.652714+0000","last_active":"2026-02-24T22:34:25.998086+0000","last_peered":"2026-02-24T22:34:25.998086+0000","last_clean":"2026-02-24T22:34:25.998086+0000","last_became_active":"2026-02-24T22:05:29.652259+0000","last_became_peered":"2026-02-24T22:05:29.652259+0000","last_unstale":"2026-02-24T22:34:25.998086+0000","last_undegraded":"2026-02-24T22:34:25.998086+0000","last_fullsized":"2026-02-24T22:34:25.998086+0000","mapping_epoch":33,"log_start":"312'401","ondisk_log_start":"312'401","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":401,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"312'136","reported_seq":842,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.775557+0000","last_change":"2026-02-24T22:34:39.775557+0000","last_active":"2026-02-24T22:34:39.775557+0000","last_peered":"2026-02-24T22:34:39.775557+0000","last_clean":"2026-02-24T22:34:39.775557+0000","last_became_active":"2026-02-24T22:05:18.183369+0000","last_became_peered":"2026-02-24T22:05:18.183369+0000","last_unstale":"2026-02-24T22:34:39.775557+0000","last_undegraded":"2026-02-24T22:34:39.775557+0000","last_fullsized":"2026-02-24T22:34:39.775557+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'136","last_scrub_stamp":"2026-02-24T22:34:39.775488+0000","last_deep_scrub":"312'136","last_deep_scrub_stamp":"2026-02-24T22:34:39.775488+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.775488+0000","objects_scrubbed":22,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:17:51.694013+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":458,"num_read_kb":386,"num_write":168,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"312'500","reported_seq":1319,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.014188+0000","last_change":"2026-02-24T22:34:31.014188+0000","last_active":"2026-02-24T22:34:31.014188+0000","last_peered":"2026-02-24T22:34:31.014188+0000","last_clean":"2026-02-24T22:34:31.014188+0000","last_became_active":"2026-02-24T22:05:16.178353+0000","last_became_peered":"2026-02-24T22:05:16.178353+0000","last_unstale":"2026-02-24T22:34:31.014188+0000","last_undegraded":"2026-02-24T22:34:31.014188+0000","last_fullsized":"2026-02-24T22:34:31.014188+0000","mapping_epoch":22,"log_start":"312'400","ondisk_log_start":"312'400","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'500","last_scrub_stamp":"2026-02-24T22:34:31.014143+0000","last_deep_scrub":"312'500","last_deep_scrub_stamp":"2026-02-24T22:34:31.014143+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.014143+0000","objects_scrubbed":3,"log_size":100,"log_dups_size":400,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:02:25.941904+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1036,"num_read_kb":9549,"num_write":2046,"num_write_kb":2098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":618,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.179872+0000","last_change":"2026-02-24T22:34:31.179872+0000","last_active":"2026-02-24T22:34:31.179872+0000","last_peered":"2026-02-24T22:34:31.179872+0000","last_clean":"2026-02-24T22:34:31.179872+0000","last_became_active":"2026-02-24T22:05:08.765129+0000","last_became_peered":"2026-02-24T22:05:08.765129+0000","last_unstale":"2026-02-24T22:34:31.179872+0000","last_undegraded":"2026-02-24T22:34:31.179872+0000","last_fullsized":"2026-02-24T22:34:31.179872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:31.179812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:31.179812+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.179812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:51:58.379988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439276,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":593,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.666471+0000","last_change":"2026-02-24T22:34:43.666471+0000","last_active":"2026-02-24T22:34:43.666471+0000","last_peered":"2026-02-24T22:34:43.666471+0000","last_clean":"2026-02-24T22:34:43.666471+0000","last_became_active":"2026-02-24T22:05:20.192811+0000","last_became_peered":"2026-02-24T22:05:20.192811+0000","last_unstale":"2026-02-24T22:34:43.666471+0000","last_undegraded":"2026-02-24T22:34:43.666471+0000","last_fullsized":"2026-02-24T22:34:43.666471+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:43.666412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:43.666412+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.666412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:08:50.485350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"312'609","reported_seq":1720,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.253964+0000","last_change":"2026-02-24T22:05:22.905928+0000","last_active":"2026-02-24T22:34:25.253964+0000","last_peered":"2026-02-24T22:34:25.253964+0000","last_clean":"2026-02-24T22:34:25.253964+0000","last_became_active":"2026-02-24T22:05:22.905663+0000","last_became_peered":"2026-02-24T22:05:22.905663+0000","last_unstale":"2026-02-24T22:34:25.253964+0000","last_undegraded":"2026-02-24T22:34:25.253964+0000","last_fullsized":"2026-02-24T22:34:25.253964+0000","mapping_epoch":28,"log_start":"307'500","ondisk_log_start":"307'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":500,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":308,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":2064,"num_read_kb":1111,"num_write":154,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"312'171","reported_seq":753,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.533054+0000","last_change":"2026-02-24T22:05:24.206213+0000","last_active":"2026-02-24T22:34:18.533054+0000","last_peered":"2026-02-24T22:34:18.533054+0000","last_clean":"2026-02-24T22:34:18.533054+0000","last_became_active":"2026-02-24T22:05:24.205847+0000","last_became_peered":"2026-02-24T22:05:24.205847+0000","last_unstale":"2026-02-24T22:34:18.533054+0000","last_undegraded":"2026-02-24T22:34:18.533054+0000","last_fullsized":"2026-02-24T22:34:18.533054+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:10:54.760147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26185,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":121,"num_read_kb":112,"num_write":413,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":577,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:34.738412+0000","last_change":"2026-02-24T22:05:29.651555+0000","last_active":"2026-02-24T22:32:34.738412+0000","last_peered":"2026-02-24T22:32:34.738412+0000","last_clean":"2026-02-24T22:32:34.738412+0000","last_became_active":"2026-02-24T22:05:29.651401+0000","last_became_peered":"2026-02-24T22:05:29.651401+0000","last_unstale":"2026-02-24T22:32:34.738412+0000","last_undegraded":"2026-02-24T22:32:34.738412+0000","last_fullsized":"2026-02-24T22:32:34.738412+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"312'175","reported_seq":881,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.748014+0000","last_change":"2026-02-24T22:34:40.748014+0000","last_active":"2026-02-24T22:34:40.748014+0000","last_peered":"2026-02-24T22:34:40.748014+0000","last_clean":"2026-02-24T22:34:40.748014+0000","last_became_active":"2026-02-24T22:05:18.185592+0000","last_became_peered":"2026-02-24T22:05:18.185592+0000","last_unstale":"2026-02-24T22:34:40.748014+0000","last_undegraded":"2026-02-24T22:34:40.748014+0000","last_fullsized":"2026-02-24T22:34:40.748014+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'175","last_scrub_stamp":"2026-02-24T22:34:40.747966+0000","last_deep_scrub":"312'175","last_deep_scrub_stamp":"2026-02-24T22:34:40.747966+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.747966+0000","objects_scrubbed":25,"log_size":75,"log_dups_size":100,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:49:27.517475+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":536,"num_read_kb":444,"num_write":207,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"312'316","reported_seq":1050,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:30.882380+0000","last_change":"2026-02-24T22:34:30.882380+0000","last_active":"2026-02-24T22:34:30.882380+0000","last_peered":"2026-02-24T22:34:30.882380+0000","last_clean":"2026-02-24T22:34:30.882380+0000","last_became_active":"2026-02-24T22:05:16.184131+0000","last_became_peered":"2026-02-24T22:05:16.184131+0000","last_unstale":"2026-02-24T22:34:30.882380+0000","last_undegraded":"2026-02-24T22:34:30.882380+0000","last_fullsized":"2026-02-24T22:34:30.882380+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'316","last_scrub_stamp":"2026-02-24T22:34:30.882351+0000","last_deep_scrub":"312'316","last_deep_scrub_stamp":"2026-02-24T22:34:30.882351+0000","last_clean_scrub_stamp":"2026-02-24T22:34:30.882351+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":300,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:29:29.669100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":471,"num_read_kb":1193,"num_write":1308,"num_write_kb":244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":620,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.039544+0000","last_change":"2026-02-24T22:34:32.039544+0000","last_active":"2026-02-24T22:34:32.039544+0000","last_peered":"2026-02-24T22:34:32.039544+0000","last_clean":"2026-02-24T22:34:32.039544+0000","last_became_active":"2026-02-24T22:05:08.682913+0000","last_became_peered":"2026-02-24T22:05:08.682913+0000","last_unstale":"2026-02-24T22:34:32.039544+0000","last_undegraded":"2026-02-24T22:34:32.039544+0000","last_fullsized":"2026-02-24T22:34:32.039544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:32.039480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:32.039480+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.039480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:39:23.840470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000359878,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":592,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.746484+0000","last_change":"2026-02-24T22:05:20.192401+0000","last_active":"2026-02-24T22:32:24.746484+0000","last_peered":"2026-02-24T22:32:24.746484+0000","last_clean":"2026-02-24T22:32:24.746484+0000","last_became_active":"2026-02-24T22:05:20.192162+0000","last_became_peered":"2026-02-24T22:05:20.192162+0000","last_unstale":"2026-02-24T22:32:24.746484+0000","last_undegraded":"2026-02-24T22:32:24.746484+0000","last_fullsized":"2026-02-24T22:32:24.746484+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:51:58.434704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"312'1031","reported_seq":2595,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.242075+0000","last_change":"2026-02-24T22:05:22.198775+0000","last_active":"2026-02-24T22:34:25.242075+0000","last_peered":"2026-02-24T22:34:25.242075+0000","last_clean":"2026-02-24T22:34:25.242075+0000","last_became_active":"2026-02-24T22:05:22.198252+0000","last_became_peered":"2026-02-24T22:05:22.198252+0000","last_unstale":"2026-02-24T22:34:25.242075+0000","last_undegraded":"2026-02-24T22:34:25.242075+0000","last_fullsized":"2026-02-24T22:34:25.242075+0000","mapping_epoch":28,"log_start":"312'1000","ondisk_log_start":"312'1000","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":1000,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":614,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":3875,"num_read_kb":1975,"num_write":82,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"312'1698","reported_seq":4315,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.552382+0000","last_change":"2026-02-24T22:05:24.198591+0000","last_active":"2026-02-24T22:34:18.552382+0000","last_peered":"2026-02-24T22:34:18.552382+0000","last_clean":"2026-02-24T22:34:18.552382+0000","last_became_active":"2026-02-24T22:05:24.198348+0000","last_became_peered":"2026-02-24T22:05:24.198348+0000","last_unstale":"2026-02-24T22:34:18.552382+0000","last_undegraded":"2026-02-24T22:34:18.552382+0000","last_fullsized":"2026-02-24T22:34:18.552382+0000","mapping_epoch":30,"log_start":"310'1600","ondisk_log_start":"310'1600","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":1600,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31448,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":5151,"num_read_kb":231050,"num_write":3475,"num_write_kb":1655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:29.747273+0000","last_change":"2026-02-24T22:05:29.652643+0000","last_active":"2026-02-24T22:32:29.747273+0000","last_peered":"2026-02-24T22:32:29.747273+0000","last_clean":"2026-02-24T22:32:29.747273+0000","last_became_active":"2026-02-24T22:05:29.652321+0000","last_became_peered":"2026-02-24T22:05:29.652321+0000","last_unstale":"2026-02-24T22:32:29.747273+0000","last_undegraded":"2026-02-24T22:32:29.747273+0000","last_fullsized":"2026-02-24T22:32:29.747273+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"312'72","reported_seq":745,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.449851+0000","last_change":"2026-02-24T22:34:36.449851+0000","last_active":"2026-02-24T22:34:36.449851+0000","last_peered":"2026-02-24T22:34:36.449851+0000","last_clean":"2026-02-24T22:34:36.449851+0000","last_became_active":"2026-02-24T22:05:18.191249+0000","last_became_peered":"2026-02-24T22:05:18.191249+0000","last_unstale":"2026-02-24T22:34:36.449851+0000","last_undegraded":"2026-02-24T22:34:36.449851+0000","last_fullsized":"2026-02-24T22:34:36.449851+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'72","last_scrub_stamp":"2026-02-24T22:34:36.449799+0000","last_deep_scrub":"312'72","last_deep_scrub_stamp":"2026-02-24T22:34:36.449799+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.449799+0000","objects_scrubbed":17,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:05:38.920525+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":226,"num_read_kb":230,"num_write":70,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2876,"num_omap_keys":12,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"312'274","reported_seq":1014,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.696342+0000","last_change":"2026-02-24T22:34:32.696342+0000","last_active":"2026-02-24T22:34:32.696342+0000","last_peered":"2026-02-24T22:34:32.696342+0000","last_clean":"2026-02-24T22:34:32.696342+0000","last_became_active":"2026-02-24T22:05:16.181840+0000","last_became_peered":"2026-02-24T22:05:16.181840+0000","last_unstale":"2026-02-24T22:34:32.696342+0000","last_undegraded":"2026-02-24T22:34:32.696342+0000","last_fullsized":"2026-02-24T22:34:32.696342+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'274","last_scrub_stamp":"2026-02-24T22:34:32.696287+0000","last_deep_scrub":"312'274","last_deep_scrub_stamp":"2026-02-24T22:34:32.696287+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.696287+0000","objects_scrubbed":4,"log_size":74,"log_dups_size":200,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:57:17.468566+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":486,"num_read_kb":7386,"num_write":1250,"num_write_kb":13063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":615,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.748652+0000","last_change":"2026-02-24T22:34:31.748652+0000","last_active":"2026-02-24T22:34:31.748652+0000","last_peered":"2026-02-24T22:34:31.748652+0000","last_clean":"2026-02-24T22:34:31.748652+0000","last_became_active":"2026-02-24T22:05:08.689542+0000","last_became_peered":"2026-02-24T22:05:08.689542+0000","last_unstale":"2026-02-24T22:34:31.748652+0000","last_undegraded":"2026-02-24T22:34:31.748652+0000","last_fullsized":"2026-02-24T22:34:31.748652+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:31.748592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:31.748592+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.748592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:30:37.278240+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00038816099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":600,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.813218+0000","last_change":"2026-02-24T22:34:47.813218+0000","last_active":"2026-02-24T22:34:47.813218+0000","last_peered":"2026-02-24T22:34:47.813218+0000","last_clean":"2026-02-24T22:34:47.813218+0000","last_became_active":"2026-02-24T22:05:20.186108+0000","last_became_peered":"2026-02-24T22:05:20.186108+0000","last_unstale":"2026-02-24T22:34:47.813218+0000","last_undegraded":"2026-02-24T22:34:47.813218+0000","last_fullsized":"2026-02-24T22:34:47.813218+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:47.813164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:47.813164+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.813164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:54:49.662332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"312'607","reported_seq":1260,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.737675+0000","last_change":"2026-02-24T22:05:22.911959+0000","last_active":"2026-02-24T22:34:25.737675+0000","last_peered":"2026-02-24T22:34:25.737675+0000","last_clean":"2026-02-24T22:34:25.737675+0000","last_became_active":"2026-02-24T22:05:22.911692+0000","last_became_peered":"2026-02-24T22:05:22.911692+0000","last_unstale":"2026-02-24T22:34:25.737675+0000","last_undegraded":"2026-02-24T22:34:25.737675+0000","last_fullsized":"2026-02-24T22:34:25.737675+0000","mapping_epoch":28,"log_start":"312'500","ondisk_log_start":"312'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":500,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115263,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1049,"num_read_kb":2748,"num_write":972,"num_write_kb":882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"312'175","reported_seq":764,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.421649+0000","last_change":"2026-02-24T22:05:24.199612+0000","last_active":"2026-02-24T22:34:18.421649+0000","last_peered":"2026-02-24T22:34:18.421649+0000","last_clean":"2026-02-24T22:34:18.421649+0000","last_became_active":"2026-02-24T22:05:24.199398+0000","last_became_peered":"2026-02-24T22:05:24.199398+0000","last_unstale":"2026-02-24T22:34:18.421649+0000","last_undegraded":"2026-02-24T22:34:18.421649+0000","last_fullsized":"2026-02-24T22:34:18.421649+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":100,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30139,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":95,"num_read_kb":93,"num_write":436,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"312'435","reported_seq":3594,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.983303+0000","last_change":"2026-02-24T22:05:29.662212+0000","last_active":"2026-02-24T22:34:25.983303+0000","last_peered":"2026-02-24T22:34:25.983303+0000","last_clean":"2026-02-24T22:34:25.983303+0000","last_became_active":"2026-02-24T22:05:29.661694+0000","last_became_peered":"2026-02-24T22:05:29.661694+0000","last_unstale":"2026-02-24T22:34:25.983303+0000","last_undegraded":"2026-02-24T22:34:25.983303+0000","last_fullsized":"2026-02-24T22:34:25.983303+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":400,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"312'461","reported_seq":2596,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.998229+0000","last_change":"2026-02-24T22:05:29.654721+0000","last_active":"2026-02-24T22:34:25.998229+0000","last_peered":"2026-02-24T22:34:25.998229+0000","last_clean":"2026-02-24T22:34:25.998229+0000","last_became_active":"2026-02-24T22:05:29.654058+0000","last_became_peered":"2026-02-24T22:05:29.654058+0000","last_unstale":"2026-02-24T22:34:25.998229+0000","last_undegraded":"2026-02-24T22:34:25.998229+0000","last_fullsized":"2026-02-24T22:34:25.998229+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":400,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"312'4219","reported_seq":4960,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.517253+0000","last_change":"2026-02-24T22:34:41.517253+0000","last_active":"2026-02-24T22:34:41.517253+0000","last_peered":"2026-02-24T22:34:41.517253+0000","last_clean":"2026-02-24T22:34:41.517253+0000","last_became_active":"2026-02-24T22:05:18.190114+0000","last_became_peered":"2026-02-24T22:05:18.190114+0000","last_unstale":"2026-02-24T22:34:41.517253+0000","last_undegraded":"2026-02-24T22:34:41.517253+0000","last_fullsized":"2026-02-24T22:34:41.517253+0000","mapping_epoch":24,"log_start":"312'4200","ondisk_log_start":"312'4200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'4219","last_scrub_stamp":"2026-02-24T22:34:41.517218+0000","last_deep_scrub":"312'4219","last_deep_scrub_stamp":"2026-02-24T22:34:41.517218+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.517218+0000","objects_scrubbed":25,"log_size":19,"log_dups_size":3000,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:15:13.238846+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":16758,"num_read_kb":13963,"num_write":6320,"num_write_kb":3135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"312'256","reported_seq":985,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.598822+0000","last_change":"2026-02-24T22:34:31.598822+0000","last_active":"2026-02-24T22:34:31.598822+0000","last_peered":"2026-02-24T22:34:31.598822+0000","last_clean":"2026-02-24T22:34:31.598822+0000","last_became_active":"2026-02-24T22:05:16.188630+0000","last_became_peered":"2026-02-24T22:05:16.188630+0000","last_unstale":"2026-02-24T22:34:31.598822+0000","last_undegraded":"2026-02-24T22:34:31.598822+0000","last_fullsized":"2026-02-24T22:34:31.598822+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'256","last_scrub_stamp":"2026-02-24T22:34:31.598742+0000","last_deep_scrub":"312'256","last_deep_scrub_stamp":"2026-02-24T22:34:31.598742+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.598742+0000","objects_scrubbed":10,"log_size":56,"log_dups_size":200,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:25:41.229561+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291503,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":466,"num_read_kb":33751,"num_write":1067,"num_write_kb":13009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":596,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.198317+0000","last_change":"2026-02-24T22:34:46.198317+0000","last_active":"2026-02-24T22:34:46.198317+0000","last_peered":"2026-02-24T22:34:46.198317+0000","last_clean":"2026-02-24T22:34:46.198317+0000","last_became_active":"2026-02-24T22:05:20.188330+0000","last_became_peered":"2026-02-24T22:05:20.188330+0000","last_unstale":"2026-02-24T22:34:46.198317+0000","last_undegraded":"2026-02-24T22:34:46.198317+0000","last_fullsized":"2026-02-24T22:34:46.198317+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:46.198241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:46.198241+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.198241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:36:44.879872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"312'688","reported_seq":1771,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.888352+0000","last_change":"2026-02-24T22:05:22.192098+0000","last_active":"2026-02-24T22:34:25.888352+0000","last_peered":"2026-02-24T22:34:25.888352+0000","last_clean":"2026-02-24T22:34:25.888352+0000","last_became_active":"2026-02-24T22:05:22.191584+0000","last_became_peered":"2026-02-24T22:05:22.191584+0000","last_unstale":"2026-02-24T22:34:25.888352+0000","last_undegraded":"2026-02-24T22:34:25.888352+0000","last_fullsized":"2026-02-24T22:34:25.888352+0000","mapping_epoch":28,"log_start":"312'600","ondisk_log_start":"312'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":600,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:55:24.433104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39069,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2230,"num_read_kb":1811,"num_write":322,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"312'161","reported_seq":768,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:55.998434+0000","last_change":"2026-02-24T22:05:24.199098+0000","last_active":"2026-02-24T22:33:55.998434+0000","last_peered":"2026-02-24T22:33:55.998434+0000","last_clean":"2026-02-24T22:33:55.998434+0000","last_became_active":"2026-02-24T22:05:24.198821+0000","last_became_peered":"2026-02-24T22:05:24.198821+0000","last_unstale":"2026-02-24T22:33:55.998434+0000","last_undegraded":"2026-02-24T22:33:55.998434+0000","last_fullsized":"2026-02-24T22:33:55.998434+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":100,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:31:53.831339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28978,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":162,"num_read_kb":145,"num_write":411,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":578,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.732230+0000","last_change":"2026-02-24T22:05:29.653213+0000","last_active":"2026-02-24T22:31:34.732230+0000","last_peered":"2026-02-24T22:31:34.732230+0000","last_clean":"2026-02-24T22:31:34.732230+0000","last_became_active":"2026-02-24T22:05:29.652757+0000","last_became_peered":"2026-02-24T22:05:29.652757+0000","last_unstale":"2026-02-24T22:31:34.732230+0000","last_undegraded":"2026-02-24T22:31:34.732230+0000","last_fullsized":"2026-02-24T22:31:34.732230+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:57:03.928835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"312'4263","reported_seq":5020,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.115933+0000","last_change":"2026-02-24T22:34:40.115933+0000","last_active":"2026-02-24T22:34:40.115933+0000","last_peered":"2026-02-24T22:34:40.115933+0000","last_clean":"2026-02-24T22:34:40.115933+0000","last_became_active":"2026-02-24T22:05:18.183983+0000","last_became_peered":"2026-02-24T22:05:18.183983+0000","last_unstale":"2026-02-24T22:34:40.115933+0000","last_undegraded":"2026-02-24T22:34:40.115933+0000","last_fullsized":"2026-02-24T22:34:40.115933+0000","mapping_epoch":24,"log_start":"312'4200","ondisk_log_start":"312'4200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'4263","last_scrub_stamp":"2026-02-24T22:34:40.115891+0000","last_deep_scrub":"312'4263","last_deep_scrub_stamp":"2026-02-24T22:34:40.115891+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.115891+0000","objects_scrubbed":32,"log_size":63,"log_dups_size":3000,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:06:10.082774+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":16912,"num_read_kb":14102,"num_write":6331,"num_write_kb":3151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"312'314","reported_seq":1064,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.052571+0000","last_change":"2026-02-24T22:34:39.052571+0000","last_active":"2026-02-24T22:34:39.052571+0000","last_peered":"2026-02-24T22:34:39.052571+0000","last_clean":"2026-02-24T22:34:39.052571+0000","last_became_active":"2026-02-24T22:05:16.192845+0000","last_became_peered":"2026-02-24T22:05:16.192845+0000","last_unstale":"2026-02-24T22:34:39.052571+0000","last_undegraded":"2026-02-24T22:34:39.052571+0000","last_fullsized":"2026-02-24T22:34:39.052571+0000","mapping_epoch":22,"log_start":"312'301","ondisk_log_start":"312'301","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'314","last_scrub_stamp":"2026-02-24T22:34:39.052507+0000","last_deep_scrub":"312'314","last_deep_scrub_stamp":"2026-02-24T22:34:39.052507+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.052507+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":301,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:33:51.248695+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":490,"num_read_kb":4336,"num_write":1277,"num_write_kb":2676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.728520+0000","last_change":"2026-02-24T22:05:20.189035+0000","last_active":"2026-02-24T22:30:59.728520+0000","last_peered":"2026-02-24T22:30:59.728520+0000","last_clean":"2026-02-24T22:30:59.728520+0000","last_became_active":"2026-02-24T22:05:20.188790+0000","last_became_peered":"2026-02-24T22:05:20.188790+0000","last_unstale":"2026-02-24T22:30:59.728520+0000","last_undegraded":"2026-02-24T22:30:59.728520+0000","last_fullsized":"2026-02-24T22:30:59.728520+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:29:04.813791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"312'25","reported_seq":615,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:04.737881+0000","last_change":"2026-02-24T22:05:22.192740+0000","last_active":"2026-02-24T22:31:04.737881+0000","last_peered":"2026-02-24T22:31:04.737881+0000","last_clean":"2026-02-24T22:31:04.737881+0000","last_became_active":"2026-02-24T22:05:22.192560+0000","last_became_peered":"2026-02-24T22:05:22.192560+0000","last_unstale":"2026-02-24T22:31:04.737881+0000","last_undegraded":"2026-02-24T22:31:04.737881+0000","last_fullsized":"2026-02-24T22:31:04.737881+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:40:42.414311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"312'163","reported_seq":746,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.531878+0000","last_change":"2026-02-24T22:05:24.205761+0000","last_active":"2026-02-24T22:34:18.531878+0000","last_peered":"2026-02-24T22:34:18.531878+0000","last_clean":"2026-02-24T22:34:18.531878+0000","last_became_active":"2026-02-24T22:05:24.205506+0000","last_became_peered":"2026-02-24T22:05:24.205506+0000","last_unstale":"2026-02-24T22:34:18.531878+0000","last_undegraded":"2026-02-24T22:34:18.531878+0000","last_fullsized":"2026-02-24T22:34:18.531878+0000","mapping_epoch":30,"log_start":"301'100","ondisk_log_start":"301'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":100,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:22:32.827602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30843,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":90,"num_read_kb":84,"num_write":389,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":573,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.734200+0000","last_change":"2026-02-24T22:05:29.653324+0000","last_active":"2026-02-24T22:30:29.734200+0000","last_peered":"2026-02-24T22:30:29.734200+0000","last_clean":"2026-02-24T22:30:29.734200+0000","last_became_active":"2026-02-24T22:05:29.652987+0000","last_became_peered":"2026-02-24T22:05:29.652987+0000","last_unstale":"2026-02-24T22:30:29.734200+0000","last_undegraded":"2026-02-24T22:30:29.734200+0000","last_fullsized":"2026-02-24T22:30:29.734200+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:58:50.918939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"312'207","reported_seq":913,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.729074+0000","last_change":"2026-02-24T22:34:41.729074+0000","last_active":"2026-02-24T22:34:41.729074+0000","last_peered":"2026-02-24T22:34:41.729074+0000","last_clean":"2026-02-24T22:34:41.729074+0000","last_became_active":"2026-02-24T22:05:18.186069+0000","last_became_peered":"2026-02-24T22:05:18.186069+0000","last_unstale":"2026-02-24T22:34:41.729074+0000","last_undegraded":"2026-02-24T22:34:41.729074+0000","last_fullsized":"2026-02-24T22:34:41.729074+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'207","last_scrub_stamp":"2026-02-24T22:34:41.729005+0000","last_deep_scrub":"312'207","last_deep_scrub_stamp":"2026-02-24T22:34:41.729005+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.729005+0000","objects_scrubbed":20,"log_size":107,"log_dups_size":100,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:04:29.291410+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":691,"num_read_kb":561,"num_write":267,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"312'314","reported_seq":1004,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.108224+0000","last_change":"2026-02-24T22:34:31.108224+0000","last_active":"2026-02-24T22:34:31.108224+0000","last_peered":"2026-02-24T22:34:31.108224+0000","last_clean":"2026-02-24T22:34:31.108224+0000","last_became_active":"2026-02-24T22:06:00.662291+0000","last_became_peered":"2026-02-24T22:06:00.662291+0000","last_unstale":"2026-02-24T22:34:31.108224+0000","last_undegraded":"2026-02-24T22:34:31.108224+0000","last_fullsized":"2026-02-24T22:34:31.108224+0000","mapping_epoch":36,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'314","last_scrub_stamp":"2026-02-24T22:34:31.108159+0000","last_deep_scrub":"312'314","last_deep_scrub_stamp":"2026-02-24T22:34:31.108159+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.108159+0000","objects_scrubbed":5,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:32:30.179715+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":421,"num_read_kb":357,"num_write":1374,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":585,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.739650+0000","last_change":"2026-02-24T22:05:20.194668+0000","last_active":"2026-02-24T22:30:59.739650+0000","last_peered":"2026-02-24T22:30:59.739650+0000","last_clean":"2026-02-24T22:30:59.739650+0000","last_became_active":"2026-02-24T22:05:20.194215+0000","last_became_peered":"2026-02-24T22:05:20.194215+0000","last_unstale":"2026-02-24T22:30:59.739650+0000","last_undegraded":"2026-02-24T22:30:59.739650+0000","last_fullsized":"2026-02-24T22:30:59.739650+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:55:09.941150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"312'476","reported_seq":1339,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.640627+0000","last_change":"2026-02-24T22:05:22.192102+0000","last_active":"2026-02-24T22:34:25.640627+0000","last_peered":"2026-02-24T22:34:25.640627+0000","last_clean":"2026-02-24T22:34:25.640627+0000","last_became_active":"2026-02-24T22:05:22.191675+0000","last_became_peered":"2026-02-24T22:05:22.191675+0000","last_unstale":"2026-02-24T22:34:25.640627+0000","last_undegraded":"2026-02-24T22:34:25.640627+0000","last_fullsized":"2026-02-24T22:34:25.640627+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":400,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:50:42.591507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39139,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1329,"num_read_kb":1395,"num_write":345,"num_write_kb":280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"312'347","reported_seq":1390,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.631185+0000","last_change":"2026-02-24T22:34:49.631185+0000","last_active":"2026-02-24T22:34:49.631185+0000","last_peered":"2026-02-24T22:34:49.631185+0000","last_clean":"2026-02-24T22:34:49.631185+0000","last_became_active":"2026-02-24T22:05:24.207479+0000","last_became_peered":"2026-02-24T22:05:24.207479+0000","last_unstale":"2026-02-24T22:34:49.631185+0000","last_undegraded":"2026-02-24T22:34:49.631185+0000","last_fullsized":"2026-02-24T22:34:49.631185+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'347","last_scrub_stamp":"2026-02-24T22:34:49.631131+0000","last_deep_scrub":"312'347","last_deep_scrub_stamp":"2026-02-24T22:34:49.631131+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.631131+0000","objects_scrubbed":98,"log_size":47,"log_dups_size":300,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:42:33.404911+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31962,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":675,"num_read_kb":669,"num_write":683,"num_write_kb":207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":564,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.735513+0000","last_change":"2026-02-24T22:05:29.661767+0000","last_active":"2026-02-24T22:30:29.735513+0000","last_peered":"2026-02-24T22:30:29.735513+0000","last_clean":"2026-02-24T22:30:29.735513+0000","last_became_active":"2026-02-24T22:05:29.661248+0000","last_became_peered":"2026-02-24T22:05:29.661248+0000","last_unstale":"2026-02-24T22:30:29.735513+0000","last_undegraded":"2026-02-24T22:30:29.735513+0000","last_fullsized":"2026-02-24T22:30:29.735513+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:30:06.329202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"312'142","reported_seq":860,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.756553+0000","last_change":"2026-02-24T22:34:39.756553+0000","last_active":"2026-02-24T22:34:39.756553+0000","last_peered":"2026-02-24T22:34:39.756553+0000","last_clean":"2026-02-24T22:34:39.756553+0000","last_became_active":"2026-02-24T22:05:18.184982+0000","last_became_peered":"2026-02-24T22:05:18.184982+0000","last_unstale":"2026-02-24T22:34:39.756553+0000","last_undegraded":"2026-02-24T22:34:39.756553+0000","last_fullsized":"2026-02-24T22:34:39.756553+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'142","last_scrub_stamp":"2026-02-24T22:34:39.756507+0000","last_deep_scrub":"312'142","last_deep_scrub_stamp":"2026-02-24T22:34:39.756507+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.756507+0000","objects_scrubbed":26,"log_size":42,"log_dups_size":100,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:57:48.133450+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":444,"num_read_kb":381,"num_write":158,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"312'284","reported_seq":1054,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.721473+0000","last_change":"2026-02-24T22:34:33.721473+0000","last_active":"2026-02-24T22:34:33.721473+0000","last_peered":"2026-02-24T22:34:33.721473+0000","last_clean":"2026-02-24T22:34:33.721473+0000","last_became_active":"2026-02-24T22:05:16.184594+0000","last_became_peered":"2026-02-24T22:05:16.184594+0000","last_unstale":"2026-02-24T22:34:33.721473+0000","last_undegraded":"2026-02-24T22:34:33.721473+0000","last_fullsized":"2026-02-24T22:34:33.721473+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'284","last_scrub_stamp":"2026-02-24T22:34:33.721410+0000","last_deep_scrub":"312'284","last_deep_scrub_stamp":"2026-02-24T22:34:33.721410+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.721410+0000","objects_scrubbed":5,"log_size":84,"log_dups_size":200,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:42:30.853887+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048643,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":519,"num_read_kb":28116,"num_write":1194,"num_write_kb":6289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":591,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.669842+0000","last_change":"2026-02-24T22:34:44.669842+0000","last_active":"2026-02-24T22:34:44.669842+0000","last_peered":"2026-02-24T22:34:44.669842+0000","last_clean":"2026-02-24T22:34:44.669842+0000","last_became_active":"2026-02-24T22:05:20.193522+0000","last_became_peered":"2026-02-24T22:05:20.193522+0000","last_unstale":"2026-02-24T22:34:44.669842+0000","last_undegraded":"2026-02-24T22:34:44.669842+0000","last_fullsized":"2026-02-24T22:34:44.669842+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:44.669782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:44.669782+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.669782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:43:09.538394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"312'434","reported_seq":1273,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.552326+0000","last_change":"2026-02-24T22:05:22.200284+0000","last_active":"2026-02-24T22:34:25.552326+0000","last_peered":"2026-02-24T22:34:25.552326+0000","last_clean":"2026-02-24T22:34:25.552326+0000","last_became_active":"2026-02-24T22:05:22.199968+0000","last_became_peered":"2026-02-24T22:05:22.199968+0000","last_unstale":"2026-02-24T22:34:25.552326+0000","last_undegraded":"2026-02-24T22:34:25.552326+0000","last_fullsized":"2026-02-24T22:34:25.552326+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":400,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:25:54.291884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38886,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1275,"num_read_kb":1335,"num_write":311,"num_write_kb":280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"312'212","reported_seq":808,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:04.178415+0000","last_change":"2026-02-24T22:05:24.200305+0000","last_active":"2026-02-24T22:34:04.178415+0000","last_peered":"2026-02-24T22:34:04.178415+0000","last_clean":"2026-02-24T22:34:04.178415+0000","last_became_active":"2026-02-24T22:05:24.199650+0000","last_became_peered":"2026-02-24T22:05:24.199650+0000","last_unstale":"2026-02-24T22:34:04.178415+0000","last_undegraded":"2026-02-24T22:34:04.178415+0000","last_fullsized":"2026-02-24T22:34:04.178415+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:33:18.336086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29523,"num_objects":91,"num_object_clones":0,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":182,"num_read_kb":193,"num_write":488,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"312'270","reported_seq":989,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.736339+0000","last_change":"2026-02-24T22:34:41.736339+0000","last_active":"2026-02-24T22:34:41.736339+0000","last_peered":"2026-02-24T22:34:41.736339+0000","last_clean":"2026-02-24T22:34:41.736339+0000","last_became_active":"2026-02-24T22:05:18.190238+0000","last_became_peered":"2026-02-24T22:05:18.190238+0000","last_unstale":"2026-02-24T22:34:41.736339+0000","last_undegraded":"2026-02-24T22:34:41.736339+0000","last_fullsized":"2026-02-24T22:34:41.736339+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'270","last_scrub_stamp":"2026-02-24T22:34:41.736291+0000","last_deep_scrub":"312'270","last_deep_scrub_stamp":"2026-02-24T22:34:41.736291+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.736291+0000","objects_scrubbed":21,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:33:02.999931+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":976,"num_read_kb":794,"num_write":353,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":219,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"312'310","reported_seq":1674,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.902084+0000","last_change":"2026-02-24T22:34:33.902084+0000","last_active":"2026-02-24T22:34:33.902084+0000","last_peered":"2026-02-24T22:34:33.902084+0000","last_clean":"2026-02-24T22:34:33.902084+0000","last_became_active":"2026-02-24T22:05:16.184415+0000","last_became_peered":"2026-02-24T22:05:16.184415+0000","last_unstale":"2026-02-24T22:34:33.902084+0000","last_undegraded":"2026-02-24T22:34:33.902084+0000","last_fullsized":"2026-02-24T22:34:33.902084+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'310","last_scrub_stamp":"2026-02-24T22:34:33.902021+0000","last_deep_scrub":"312'310","last_deep_scrub_stamp":"2026-02-24T22:34:33.902021+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.902021+0000","objects_scrubbed":3,"log_size":110,"log_dups_size":200,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:11:44.512299+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2362,"num_read_kb":23834,"num_write":1125,"num_write_kb":5557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:24.740195+0000","last_change":"2026-02-24T22:05:20.186817+0000","last_active":"2026-02-24T22:31:24.740195+0000","last_peered":"2026-02-24T22:31:24.740195+0000","last_clean":"2026-02-24T22:31:24.740195+0000","last_became_active":"2026-02-24T22:05:20.186325+0000","last_became_peered":"2026-02-24T22:05:20.186325+0000","last_unstale":"2026-02-24T22:31:24.740195+0000","last_undegraded":"2026-02-24T22:31:24.740195+0000","last_fullsized":"2026-02-24T22:31:24.740195+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:45:07.507222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"312'409","reported_seq":1027,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.546379+0000","last_change":"2026-02-24T22:05:22.192972+0000","last_active":"2026-02-24T22:34:25.546379+0000","last_peered":"2026-02-24T22:34:25.546379+0000","last_clean":"2026-02-24T22:34:25.546379+0000","last_became_active":"2026-02-24T22:05:22.192756+0000","last_became_peered":"2026-02-24T22:05:22.192756+0000","last_unstale":"2026-02-24T22:34:25.546379+0000","last_undegraded":"2026-02-24T22:34:25.546379+0000","last_fullsized":"2026-02-24T22:34:25.546379+0000","mapping_epoch":28,"log_start":"312'300","ondisk_log_start":"312'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":300,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:43:14.513261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77383,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":715,"num_read_kb":1913,"num_write":704,"num_write_kb":692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"312'187","reported_seq":796,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.233131+0000","last_change":"2026-02-24T22:05:24.206653+0000","last_active":"2026-02-24T22:34:17.233131+0000","last_peered":"2026-02-24T22:34:17.233131+0000","last_clean":"2026-02-24T22:34:17.233131+0000","last_became_active":"2026-02-24T22:05:24.206407+0000","last_became_peered":"2026-02-24T22:05:24.206407+0000","last_unstale":"2026-02-24T22:34:17.233131+0000","last_undegraded":"2026-02-24T22:34:17.233131+0000","last_fullsized":"2026-02-24T22:34:17.233131+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:49:11.388315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30298,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":168,"num_read_kb":155,"num_write":455,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":576,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.729573+0000","last_change":"2026-02-24T22:05:29.650326+0000","last_active":"2026-02-24T22:31:09.729573+0000","last_peered":"2026-02-24T22:31:09.729573+0000","last_clean":"2026-02-24T22:31:09.729573+0000","last_became_active":"2026-02-24T22:05:29.650115+0000","last_became_peered":"2026-02-24T22:05:29.650115+0000","last_unstale":"2026-02-24T22:31:09.729573+0000","last_undegraded":"2026-02-24T22:31:09.729573+0000","last_fullsized":"2026-02-24T22:31:09.729573+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:38:20.581425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"312'193","reported_seq":914,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.857329+0000","last_change":"2026-02-24T22:34:43.857329+0000","last_active":"2026-02-24T22:34:43.857329+0000","last_peered":"2026-02-24T22:34:43.857329+0000","last_clean":"2026-02-24T22:34:43.857329+0000","last_became_active":"2026-02-24T22:05:18.183501+0000","last_became_peered":"2026-02-24T22:05:18.183501+0000","last_unstale":"2026-02-24T22:34:43.857329+0000","last_undegraded":"2026-02-24T22:34:43.857329+0000","last_fullsized":"2026-02-24T22:34:43.857329+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'193","last_scrub_stamp":"2026-02-24T22:34:43.857270+0000","last_deep_scrub":"312'193","last_deep_scrub_stamp":"2026-02-24T22:34:43.857270+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.857270+0000","objects_scrubbed":31,"log_size":93,"log_dups_size":100,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:49:05.843210+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":578,"num_read_kb":488,"num_write":224,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":465,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"312'412","reported_seq":1362,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.006923+0000","last_change":"2026-02-24T22:34:33.006923+0000","last_active":"2026-02-24T22:34:33.006923+0000","last_peered":"2026-02-24T22:34:33.006923+0000","last_clean":"2026-02-24T22:34:33.006923+0000","last_became_active":"2026-02-24T22:05:16.182909+0000","last_became_peered":"2026-02-24T22:05:16.182909+0000","last_unstale":"2026-02-24T22:34:33.006923+0000","last_undegraded":"2026-02-24T22:34:33.006923+0000","last_fullsized":"2026-02-24T22:34:33.006923+0000","mapping_epoch":22,"log_start":"312'400","ondisk_log_start":"312'400","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'412","last_scrub_stamp":"2026-02-24T22:34:33.006865+0000","last_deep_scrub":"312'412","last_deep_scrub_stamp":"2026-02-24T22:34:33.006865+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.006865+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":400,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:36:32.796788+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1207,"num_read_kb":9244,"num_write":1611,"num_write_kb":11784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":593,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.622797+0000","last_change":"2026-02-24T22:34:45.622797+0000","last_active":"2026-02-24T22:34:45.622797+0000","last_peered":"2026-02-24T22:34:45.622797+0000","last_clean":"2026-02-24T22:34:45.622797+0000","last_became_active":"2026-02-24T22:05:20.193751+0000","last_became_peered":"2026-02-24T22:05:20.193751+0000","last_unstale":"2026-02-24T22:34:45.622797+0000","last_undegraded":"2026-02-24T22:34:45.622797+0000","last_fullsized":"2026-02-24T22:34:45.622797+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:45.622745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:45.622745+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.622745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:01:44.701244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"312'744","reported_seq":1827,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.582463+0000","last_change":"2026-02-24T22:05:22.200836+0000","last_active":"2026-02-24T22:34:25.582463+0000","last_peered":"2026-02-24T22:34:25.582463+0000","last_clean":"2026-02-24T22:34:25.582463+0000","last_became_active":"2026-02-24T22:05:22.200603+0000","last_became_peered":"2026-02-24T22:05:22.200603+0000","last_unstale":"2026-02-24T22:34:25.582463+0000","last_undegraded":"2026-02-24T22:34:25.582463+0000","last_fullsized":"2026-02-24T22:34:25.582463+0000","mapping_epoch":28,"log_start":"312'700","ondisk_log_start":"312'700","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":700,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:36:03.813153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39896,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":2333,"num_read_kb":2057,"num_write":434,"num_write_kb":373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"312'334","reported_seq":964,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.243815+0000","last_change":"2026-02-24T22:05:24.201087+0000","last_active":"2026-02-24T22:34:18.243815+0000","last_peered":"2026-02-24T22:34:18.243815+0000","last_clean":"2026-02-24T22:34:18.243815+0000","last_became_active":"2026-02-24T22:05:24.200391+0000","last_became_peered":"2026-02-24T22:05:24.200391+0000","last_unstale":"2026-02-24T22:34:18.243815+0000","last_undegraded":"2026-02-24T22:34:18.243815+0000","last_fullsized":"2026-02-24T22:34:18.243815+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:18:42.256342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31847,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":370,"num_read_kb":353,"num_write":777,"num_write_kb":271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"312'81","reported_seq":994,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:14.669283+0000","last_change":"2026-02-24T22:05:29.660890+0000","last_active":"2026-02-24T22:34:14.669283+0000","last_peered":"2026-02-24T22:34:14.669283+0000","last_clean":"2026-02-24T22:34:14.669283+0000","last_became_active":"2026-02-24T22:05:29.660289+0000","last_became_peered":"2026-02-24T22:05:29.660289+0000","last_unstale":"2026-02-24T22:34:14.669283+0000","last_undegraded":"2026-02-24T22:34:14.669283+0000","last_fullsized":"2026-02-24T22:34:14.669283+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:13:08.310592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"312'120","reported_seq":798,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.773705+0000","last_change":"2026-02-24T22:34:43.773705+0000","last_active":"2026-02-24T22:34:43.773705+0000","last_peered":"2026-02-24T22:34:43.773705+0000","last_clean":"2026-02-24T22:34:43.773705+0000","last_became_active":"2026-02-24T22:05:18.184745+0000","last_became_peered":"2026-02-24T22:05:18.184745+0000","last_unstale":"2026-02-24T22:34:43.773705+0000","last_undegraded":"2026-02-24T22:34:43.773705+0000","last_fullsized":"2026-02-24T22:34:43.773705+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'120","last_scrub_stamp":"2026-02-24T22:34:43.773655+0000","last_deep_scrub":"312'120","last_deep_scrub_stamp":"2026-02-24T22:34:43.773655+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.773655+0000","objects_scrubbed":20,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:13:15.141522+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":371,"num_read_kb":307,"num_write":132,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":684,"num_omap_keys":3,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"312'307","reported_seq":1035,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.531150+0000","last_change":"2026-02-24T22:34:33.531150+0000","last_active":"2026-02-24T22:34:33.531150+0000","last_peered":"2026-02-24T22:34:33.531150+0000","last_clean":"2026-02-24T22:34:33.531150+0000","last_became_active":"2026-02-24T22:05:16.185574+0000","last_became_peered":"2026-02-24T22:05:16.185574+0000","last_unstale":"2026-02-24T22:34:33.531150+0000","last_undegraded":"2026-02-24T22:34:33.531150+0000","last_fullsized":"2026-02-24T22:34:33.531150+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'307","last_scrub_stamp":"2026-02-24T22:34:33.531066+0000","last_deep_scrub":"312'307","last_deep_scrub_stamp":"2026-02-24T22:34:33.531066+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.531066+0000","objects_scrubbed":4,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:08:13.354888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":428,"num_read_kb":30007,"num_write":1257,"num_write_kb":21358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"32'3","reported_seq":831,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.489031+0000","last_change":"2026-02-24T22:05:20.192991+0000","last_active":"2026-02-24T22:34:18.489031+0000","last_peered":"2026-02-24T22:34:18.489031+0000","last_clean":"2026-02-24T22:34:18.489031+0000","last_became_active":"2026-02-24T22:05:20.192612+0000","last_became_peered":"2026-02-24T22:05:20.192612+0000","last_unstale":"2026-02-24T22:34:18.489031+0000","last_undegraded":"2026-02-24T22:34:18.489031+0000","last_fullsized":"2026-02-24T22:34:18.489031+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:32:11.097665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":484,"num_read_kb":482,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"312'204","reported_seq":817,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.995065+0000","last_change":"2026-02-24T22:05:22.906226+0000","last_active":"2026-02-24T22:34:25.995065+0000","last_peered":"2026-02-24T22:34:25.995065+0000","last_clean":"2026-02-24T22:34:25.995065+0000","last_became_active":"2026-02-24T22:05:22.906047+0000","last_became_peered":"2026-02-24T22:05:22.906047+0000","last_unstale":"2026-02-24T22:34:25.995065+0000","last_undegraded":"2026-02-24T22:34:25.995065+0000","last_fullsized":"2026-02-24T22:34:25.995065+0000","mapping_epoch":28,"log_start":"312'100","ondisk_log_start":"312'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38786,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":328,"num_read_kb":839,"num_write":300,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"312'164","reported_seq":759,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:58.164024+0000","last_change":"2026-02-24T22:05:24.201114+0000","last_active":"2026-02-24T22:33:58.164024+0000","last_peered":"2026-02-24T22:33:58.164024+0000","last_clean":"2026-02-24T22:33:58.164024+0000","last_became_active":"2026-02-24T22:05:24.200747+0000","last_became_peered":"2026-02-24T22:05:24.200747+0000","last_unstale":"2026-02-24T22:33:58.164024+0000","last_undegraded":"2026-02-24T22:33:58.164024+0000","last_fullsized":"2026-02-24T22:33:58.164024+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:27:56.881747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30873,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":120,"num_read_kb":110,"num_write":405,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":565,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.740173+0000","last_change":"2026-02-24T22:05:29.660794+0000","last_active":"2026-02-24T22:31:09.740173+0000","last_peered":"2026-02-24T22:31:09.740173+0000","last_clean":"2026-02-24T22:31:09.740173+0000","last_became_active":"2026-02-24T22:05:29.660344+0000","last_became_peered":"2026-02-24T22:05:29.660344+0000","last_unstale":"2026-02-24T22:31:09.740173+0000","last_undegraded":"2026-02-24T22:31:09.740173+0000","last_fullsized":"2026-02-24T22:31:09.740173+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:36:46.842669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"312'176","reported_seq":856,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.075455+0000","last_change":"2026-02-24T22:34:36.075455+0000","last_active":"2026-02-24T22:34:36.075455+0000","last_peered":"2026-02-24T22:34:36.075455+0000","last_clean":"2026-02-24T22:34:36.075455+0000","last_became_active":"2026-02-24T22:06:00.759215+0000","last_became_peered":"2026-02-24T22:06:00.759215+0000","last_unstale":"2026-02-24T22:34:36.075455+0000","last_undegraded":"2026-02-24T22:34:36.075455+0000","last_fullsized":"2026-02-24T22:34:36.075455+0000","mapping_epoch":36,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'176","last_scrub_stamp":"2026-02-24T22:34:36.075425+0000","last_deep_scrub":"312'176","last_deep_scrub_stamp":"2026-02-24T22:34:36.075425+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.075425+0000","objects_scrubbed":19,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:59:34.861298+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":584,"num_read_kb":483,"num_write":222,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"312'269","reported_seq":1028,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.707474+0000","last_change":"2026-02-24T22:34:31.707474+0000","last_active":"2026-02-24T22:34:31.707474+0000","last_peered":"2026-02-24T22:34:31.707474+0000","last_clean":"2026-02-24T22:34:31.707474+0000","last_became_active":"2026-02-24T22:05:16.183956+0000","last_became_peered":"2026-02-24T22:05:16.183956+0000","last_unstale":"2026-02-24T22:34:31.707474+0000","last_undegraded":"2026-02-24T22:34:31.707474+0000","last_fullsized":"2026-02-24T22:34:31.707474+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'269","last_scrub_stamp":"2026-02-24T22:34:31.707414+0000","last_deep_scrub":"312'269","last_deep_scrub_stamp":"2026-02-24T22:34:31.707414+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.707414+0000","objects_scrubbed":6,"log_size":69,"log_dups_size":200,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:16:18.564686+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242893,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":575,"num_read_kb":6430,"num_write":1267,"num_write_kb":5371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":595,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.203221+0000","last_change":"2026-02-24T22:34:49.203221+0000","last_active":"2026-02-24T22:34:49.203221+0000","last_peered":"2026-02-24T22:34:49.203221+0000","last_clean":"2026-02-24T22:34:49.203221+0000","last_became_active":"2026-02-24T22:05:20.187316+0000","last_became_peered":"2026-02-24T22:05:20.187316+0000","last_unstale":"2026-02-24T22:34:49.203221+0000","last_undegraded":"2026-02-24T22:34:49.203221+0000","last_fullsized":"2026-02-24T22:34:49.203221+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:49.203134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:49.203134+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.203134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:31:08.087721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"312'288","reported_seq":1127,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.218561+0000","last_change":"2026-02-24T22:05:22.912321+0000","last_active":"2026-02-24T22:34:25.218561+0000","last_peered":"2026-02-24T22:34:25.218561+0000","last_clean":"2026-02-24T22:34:25.218561+0000","last_became_active":"2026-02-24T22:05:22.912087+0000","last_became_peered":"2026-02-24T22:05:22.912087+0000","last_unstale":"2026-02-24T22:34:25.218561+0000","last_undegraded":"2026-02-24T22:34:25.218561+0000","last_fullsized":"2026-02-24T22:34:25.218561+0000","mapping_epoch":28,"log_start":"307'200","ondisk_log_start":"307'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":200,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:16:48.283329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":152,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1000,"num_read_kb":520,"num_write":51,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"312'178","reported_seq":845,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.232593+0000","last_change":"2026-02-24T22:05:24.207156+0000","last_active":"2026-02-24T22:34:18.232593+0000","last_peered":"2026-02-24T22:34:18.232593+0000","last_clean":"2026-02-24T22:34:18.232593+0000","last_became_active":"2026-02-24T22:05:24.206383+0000","last_became_peered":"2026-02-24T22:05:24.206383+0000","last_unstale":"2026-02-24T22:34:18.232593+0000","last_undegraded":"2026-02-24T22:34:18.232593+0000","last_fullsized":"2026-02-24T22:34:18.232593+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":100,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29475,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":255,"num_read_kb":237,"num_write":434,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":571,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:09.740948+0000","last_change":"2026-02-24T22:05:29.661761+0000","last_active":"2026-02-24T22:31:09.740948+0000","last_peered":"2026-02-24T22:31:09.740948+0000","last_clean":"2026-02-24T22:31:09.740948+0000","last_became_active":"2026-02-24T22:05:29.661539+0000","last_became_peered":"2026-02-24T22:05:29.661539+0000","last_unstale":"2026-02-24T22:31:09.740948+0000","last_undegraded":"2026-02-24T22:31:09.740948+0000","last_fullsized":"2026-02-24T22:31:09.740948+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:02:56.515020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"312'201","reported_seq":881,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.483754+0000","last_change":"2026-02-24T22:34:44.483754+0000","last_active":"2026-02-24T22:34:44.483754+0000","last_peered":"2026-02-24T22:34:44.483754+0000","last_clean":"2026-02-24T22:34:44.483754+0000","last_became_active":"2026-02-24T22:05:18.192487+0000","last_became_peered":"2026-02-24T22:05:18.192487+0000","last_unstale":"2026-02-24T22:34:44.483754+0000","last_undegraded":"2026-02-24T22:34:44.483754+0000","last_fullsized":"2026-02-24T22:34:44.483754+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'201","last_scrub_stamp":"2026-02-24T22:34:44.483706+0000","last_deep_scrub":"312'201","last_deep_scrub_stamp":"2026-02-24T22:34:44.483706+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.483706+0000","objects_scrubbed":23,"log_size":101,"log_dups_size":100,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:31:57.971063+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":681,"num_read_kb":551,"num_write":266,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"312'299","reported_seq":1006,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.128130+0000","last_change":"2026-02-24T22:34:36.128130+0000","last_active":"2026-02-24T22:34:36.128130+0000","last_peered":"2026-02-24T22:34:36.128130+0000","last_clean":"2026-02-24T22:34:36.128130+0000","last_became_active":"2026-02-24T22:05:16.190108+0000","last_became_peered":"2026-02-24T22:05:16.190108+0000","last_unstale":"2026-02-24T22:34:36.128130+0000","last_undegraded":"2026-02-24T22:34:36.128130+0000","last_fullsized":"2026-02-24T22:34:36.128130+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'299","last_scrub_stamp":"2026-02-24T22:34:36.128072+0000","last_deep_scrub":"312'299","last_deep_scrub_stamp":"2026-02-24T22:34:36.128072+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.128072+0000","objects_scrubbed":3,"log_size":99,"log_dups_size":200,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:52:02.916678+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242914,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":451,"num_read_kb":15743,"num_write":1326,"num_write_kb":10328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":584,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:19.735341+0000","last_change":"2026-02-24T22:05:20.194434+0000","last_active":"2026-02-24T22:30:19.735341+0000","last_peered":"2026-02-24T22:30:19.735341+0000","last_clean":"2026-02-24T22:30:19.735341+0000","last_became_active":"2026-02-24T22:05:20.194183+0000","last_became_peered":"2026-02-24T22:05:20.194183+0000","last_unstale":"2026-02-24T22:30:19.735341+0000","last_undegraded":"2026-02-24T22:30:19.735341+0000","last_fullsized":"2026-02-24T22:30:19.735341+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:11:03.103844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"312'713","reported_seq":1791,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.384999+0000","last_change":"2026-02-24T22:05:22.199767+0000","last_active":"2026-02-24T22:34:25.384999+0000","last_peered":"2026-02-24T22:34:25.384999+0000","last_clean":"2026-02-24T22:34:25.384999+0000","last_became_active":"2026-02-24T22:05:22.199480+0000","last_became_peered":"2026-02-24T22:05:22.199480+0000","last_unstale":"2026-02-24T22:34:25.384999+0000","last_undegraded":"2026-02-24T22:34:25.384999+0000","last_fullsized":"2026-02-24T22:34:25.384999+0000","mapping_epoch":28,"log_start":"312'700","ondisk_log_start":"312'700","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":700,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:44:12.736876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39366,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":2266,"num_read_kb":1888,"num_write":352,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"312'198","reported_seq":804,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:15.239318+0000","last_change":"2026-02-24T22:05:24.201028+0000","last_active":"2026-02-24T22:34:15.239318+0000","last_peered":"2026-02-24T22:34:15.239318+0000","last_clean":"2026-02-24T22:34:15.239318+0000","last_became_active":"2026-02-24T22:05:24.200434+0000","last_became_peered":"2026-02-24T22:05:24.200434+0000","last_unstale":"2026-02-24T22:34:15.239318+0000","last_undegraded":"2026-02-24T22:34:15.239318+0000","last_fullsized":"2026-02-24T22:34:15.239318+0000","mapping_epoch":30,"log_start":"307'100","ondisk_log_start":"307'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":100,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:38:49.379078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27134,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":189,"num_read_kb":178,"num_write":476,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":571,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:04.747185+0000","last_change":"2026-02-24T22:05:29.662687+0000","last_active":"2026-02-24T22:32:04.747185+0000","last_peered":"2026-02-24T22:32:04.747185+0000","last_clean":"2026-02-24T22:32:04.747185+0000","last_became_active":"2026-02-24T22:05:29.662425+0000","last_became_peered":"2026-02-24T22:05:29.662425+0000","last_unstale":"2026-02-24T22:32:04.747185+0000","last_undegraded":"2026-02-24T22:32:04.747185+0000","last_fullsized":"2026-02-24T22:32:04.747185+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:37:43.066587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"312'157","reported_seq":844,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.518852+0000","last_change":"2026-02-24T22:34:42.518852+0000","last_active":"2026-02-24T22:34:42.518852+0000","last_peered":"2026-02-24T22:34:42.518852+0000","last_clean":"2026-02-24T22:34:42.518852+0000","last_became_active":"2026-02-24T22:05:18.191664+0000","last_became_peered":"2026-02-24T22:05:18.191664+0000","last_unstale":"2026-02-24T22:34:42.518852+0000","last_undegraded":"2026-02-24T22:34:42.518852+0000","last_fullsized":"2026-02-24T22:34:42.518852+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'157","last_scrub_stamp":"2026-02-24T22:34:42.518790+0000","last_deep_scrub":"312'157","last_deep_scrub_stamp":"2026-02-24T22:34:42.518790+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.518790+0000","objects_scrubbed":25,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:34:13.833329+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":463,"num_read_kb":379,"num_write":198,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"312'278","reported_seq":981,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.114265+0000","last_change":"2026-02-24T22:34:35.114265+0000","last_active":"2026-02-24T22:34:35.114265+0000","last_peered":"2026-02-24T22:34:35.114265+0000","last_clean":"2026-02-24T22:34:35.114265+0000","last_became_active":"2026-02-24T22:05:16.188154+0000","last_became_peered":"2026-02-24T22:05:16.188154+0000","last_unstale":"2026-02-24T22:34:35.114265+0000","last_undegraded":"2026-02-24T22:34:35.114265+0000","last_fullsized":"2026-02-24T22:34:35.114265+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'278","last_scrub_stamp":"2026-02-24T22:34:35.114217+0000","last_deep_scrub":"312'278","last_deep_scrub_stamp":"2026-02-24T22:34:35.114217+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.114217+0000","objects_scrubbed":6,"log_size":78,"log_dups_size":200,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:02:24.896641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":434,"num_read_kb":2459,"num_write":1229,"num_write_kb":1814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"32'5","reported_seq":821,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.487788+0000","last_change":"2026-02-24T22:05:20.195455+0000","last_active":"2026-02-24T22:34:18.487788+0000","last_peered":"2026-02-24T22:34:18.487788+0000","last_clean":"2026-02-24T22:34:18.487788+0000","last_became_active":"2026-02-24T22:05:20.195202+0000","last_became_peered":"2026-02-24T22:05:20.195202+0000","last_unstale":"2026-02-24T22:34:18.487788+0000","last_undegraded":"2026-02-24T22:34:18.487788+0000","last_fullsized":"2026-02-24T22:34:18.487788+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:34:18.002332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1242,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":488,"num_read_kb":722,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"312'687","reported_seq":1765,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.418928+0000","last_change":"2026-02-24T22:05:22.192219+0000","last_active":"2026-02-24T22:34:25.418928+0000","last_peered":"2026-02-24T22:34:25.418928+0000","last_clean":"2026-02-24T22:34:25.418928+0000","last_became_active":"2026-02-24T22:05:22.192035+0000","last_became_peered":"2026-02-24T22:05:22.192035+0000","last_unstale":"2026-02-24T22:34:25.418928+0000","last_undegraded":"2026-02-24T22:34:25.418928+0000","last_fullsized":"2026-02-24T22:34:25.418928+0000","mapping_epoch":28,"log_start":"312'600","ondisk_log_start":"312'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":600,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:05:53.268094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38403,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2221,"num_read_kb":1805,"num_write":316,"num_write_kb":280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"312'2477","reported_seq":5137,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.219286+0000","last_change":"2026-02-24T22:05:24.206516+0000","last_active":"2026-02-24T22:34:18.219286+0000","last_peered":"2026-02-24T22:34:18.219286+0000","last_clean":"2026-02-24T22:34:18.219286+0000","last_became_active":"2026-02-24T22:05:24.205557+0000","last_became_peered":"2026-02-24T22:05:24.205557+0000","last_unstale":"2026-02-24T22:34:18.219286+0000","last_undegraded":"2026-02-24T22:34:18.219286+0000","last_fullsized":"2026-02-24T22:34:18.219286+0000","mapping_epoch":30,"log_start":"312'2400","ondisk_log_start":"312'2400","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":2400,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:33:57.249269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34698,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":6755,"num_read_kb":6753,"num_write":5061,"num_write_kb":2022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":576,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:04.735258+0000","last_change":"2026-02-24T22:05:29.651962+0000","last_active":"2026-02-24T22:32:04.735258+0000","last_peered":"2026-02-24T22:32:04.735258+0000","last_clean":"2026-02-24T22:32:04.735258+0000","last_became_active":"2026-02-24T22:05:29.651760+0000","last_became_peered":"2026-02-24T22:05:29.651760+0000","last_unstale":"2026-02-24T22:32:04.735258+0000","last_undegraded":"2026-02-24T22:32:04.735258+0000","last_fullsized":"2026-02-24T22:32:04.735258+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:48:41.143100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"312'155","reported_seq":841,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.775671+0000","last_change":"2026-02-24T22:34:44.775671+0000","last_active":"2026-02-24T22:34:44.775671+0000","last_peered":"2026-02-24T22:34:44.775671+0000","last_clean":"2026-02-24T22:34:44.775671+0000","last_became_active":"2026-02-24T22:05:18.184939+0000","last_became_peered":"2026-02-24T22:05:18.184939+0000","last_unstale":"2026-02-24T22:34:44.775671+0000","last_undegraded":"2026-02-24T22:34:44.775671+0000","last_fullsized":"2026-02-24T22:34:44.775671+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'155","last_scrub_stamp":"2026-02-24T22:34:44.775613+0000","last_deep_scrub":"312'155","last_deep_scrub_stamp":"2026-02-24T22:34:44.775613+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.775613+0000","objects_scrubbed":27,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:46:49.648242+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":436,"num_read_kb":360,"num_write":185,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"312'291","reported_seq":1023,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.911246+0000","last_change":"2026-02-24T22:34:34.911246+0000","last_active":"2026-02-24T22:34:34.911246+0000","last_peered":"2026-02-24T22:34:34.911246+0000","last_clean":"2026-02-24T22:34:34.911246+0000","last_became_active":"2026-02-24T22:05:16.186043+0000","last_became_peered":"2026-02-24T22:05:16.186043+0000","last_unstale":"2026-02-24T22:34:34.911246+0000","last_undegraded":"2026-02-24T22:34:34.911246+0000","last_fullsized":"2026-02-24T22:34:34.911246+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'291","last_scrub_stamp":"2026-02-24T22:34:34.911183+0000","last_deep_scrub":"312'291","last_deep_scrub_stamp":"2026-02-24T22:34:34.911183+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.911183+0000","objects_scrubbed":4,"log_size":91,"log_dups_size":200,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:44:00.316854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":465,"num_read_kb":15751,"num_write":1254,"num_write_kb":5255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":590,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:54.742847+0000","last_change":"2026-02-24T22:05:20.193254+0000","last_active":"2026-02-24T22:31:54.742847+0000","last_peered":"2026-02-24T22:31:54.742847+0000","last_clean":"2026-02-24T22:31:54.742847+0000","last_became_active":"2026-02-24T22:05:20.192962+0000","last_became_peered":"2026-02-24T22:05:20.192962+0000","last_unstale":"2026-02-24T22:31:54.742847+0000","last_undegraded":"2026-02-24T22:31:54.742847+0000","last_fullsized":"2026-02-24T22:31:54.742847+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:19.177566+0000","last_clean_scrub_stamp":"2026-02-24T22:05:19.177566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:15:25.988404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"312'475","reported_seq":1312,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.947633+0000","last_change":"2026-02-24T22:05:22.194159+0000","last_active":"2026-02-24T22:34:25.947633+0000","last_peered":"2026-02-24T22:34:25.947633+0000","last_clean":"2026-02-24T22:34:25.947633+0000","last_became_active":"2026-02-24T22:05:22.193896+0000","last_became_peered":"2026-02-24T22:05:22.193896+0000","last_unstale":"2026-02-24T22:34:25.947633+0000","last_undegraded":"2026-02-24T22:34:25.947633+0000","last_fullsized":"2026-02-24T22:34:25.947633+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":400,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:30:50.341834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41178,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1310,"num_read_kb":1453,"num_write":355,"num_write_kb":318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"312'188","reported_seq":800,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:17.715171+0000","last_change":"2026-02-24T22:05:24.206176+0000","last_active":"2026-02-24T22:34:17.715171+0000","last_peered":"2026-02-24T22:34:17.715171+0000","last_clean":"2026-02-24T22:34:17.715171+0000","last_became_active":"2026-02-24T22:05:24.205855+0000","last_became_peered":"2026-02-24T22:05:24.205855+0000","last_unstale":"2026-02-24T22:34:17.715171+0000","last_undegraded":"2026-02-24T22:34:17.715171+0000","last_fullsized":"2026-02-24T22:34:17.715171+0000","mapping_epoch":30,"log_start":"288'100","ondisk_log_start":"288'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":100,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:18:56.993738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27729,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":196,"num_read_kb":179,"num_write":471,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"312'448","reported_seq":1620,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.983342+0000","last_change":"2026-02-24T22:05:29.662310+0000","last_active":"2026-02-24T22:34:25.983342+0000","last_peered":"2026-02-24T22:34:25.983342+0000","last_clean":"2026-02-24T22:34:25.983342+0000","last_became_active":"2026-02-24T22:05:29.661983+0000","last_became_peered":"2026-02-24T22:05:29.661983+0000","last_unstale":"2026-02-24T22:34:25.983342+0000","last_undegraded":"2026-02-24T22:34:25.983342+0000","last_fullsized":"2026-02-24T22:34:25.983342+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":400,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:00:08.834953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"312'191","reported_seq":883,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.478571+0000","last_change":"2026-02-24T22:34:40.478571+0000","last_active":"2026-02-24T22:34:40.478571+0000","last_peered":"2026-02-24T22:34:40.478571+0000","last_clean":"2026-02-24T22:34:40.478571+0000","last_became_active":"2026-02-24T22:05:18.189877+0000","last_became_peered":"2026-02-24T22:05:18.189877+0000","last_unstale":"2026-02-24T22:34:40.478571+0000","last_undegraded":"2026-02-24T22:34:40.478571+0000","last_fullsized":"2026-02-24T22:34:40.478571+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'191","last_scrub_stamp":"2026-02-24T22:34:40.478532+0000","last_deep_scrub":"312'191","last_deep_scrub_stamp":"2026-02-24T22:34:40.478532+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.478532+0000","objects_scrubbed":22,"log_size":91,"log_dups_size":100,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:10:11.001904+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":510,"num_write":249,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"312'332","reported_seq":1100,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.714508+0000","last_change":"2026-02-24T22:34:34.714508+0000","last_active":"2026-02-24T22:34:34.714508+0000","last_peered":"2026-02-24T22:34:34.714508+0000","last_clean":"2026-02-24T22:34:34.714508+0000","last_became_active":"2026-02-24T22:05:16.184525+0000","last_became_peered":"2026-02-24T22:05:16.184525+0000","last_unstale":"2026-02-24T22:34:34.714508+0000","last_undegraded":"2026-02-24T22:34:34.714508+0000","last_fullsized":"2026-02-24T22:34:34.714508+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'332","last_scrub_stamp":"2026-02-24T22:34:34.714447+0000","last_deep_scrub":"312'332","last_deep_scrub_stamp":"2026-02-24T22:34:34.714447+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.714447+0000","objects_scrubbed":7,"log_size":32,"log_dups_size":300,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:44:54.170379+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242941,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":576,"num_read_kb":16320,"num_write":1488,"num_write_kb":16752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"312'2","reported_seq":598,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.250464+0000","last_change":"2026-02-24T22:34:48.250464+0000","last_active":"2026-02-24T22:34:48.250464+0000","last_peered":"2026-02-24T22:34:48.250464+0000","last_clean":"2026-02-24T22:34:48.250464+0000","last_became_active":"2026-02-24T22:05:20.186850+0000","last_became_peered":"2026-02-24T22:05:20.186850+0000","last_unstale":"2026-02-24T22:34:48.250464+0000","last_undegraded":"2026-02-24T22:34:48.250464+0000","last_fullsized":"2026-02-24T22:34:48.250464+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:34:48.250415+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:34:48.250415+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.250415+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:00:47.346514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"312'249","reported_seq":872,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.268142+0000","last_change":"2026-02-24T22:05:22.199568+0000","last_active":"2026-02-24T22:34:25.268142+0000","last_peered":"2026-02-24T22:34:25.268142+0000","last_clean":"2026-02-24T22:34:25.268142+0000","last_became_active":"2026-02-24T22:05:22.199358+0000","last_became_peered":"2026-02-24T22:05:22.199358+0000","last_unstale":"2026-02-24T22:34:25.268142+0000","last_undegraded":"2026-02-24T22:34:25.268142+0000","last_fullsized":"2026-02-24T22:34:25.268142+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:21.180940+0000","last_clean_scrub_stamp":"2026-02-24T22:05:21.180940+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":200,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:24:45.196918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37935,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":401,"num_read_kb":1010,"num_write":360,"num_write_kb":308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"312'281","reported_seq":1222,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:18.228289+0000","last_change":"2026-02-24T22:05:24.200431+0000","last_active":"2026-02-24T22:34:18.228289+0000","last_peered":"2026-02-24T22:34:18.228289+0000","last_clean":"2026-02-24T22:34:18.228289+0000","last_became_active":"2026-02-24T22:05:24.199555+0000","last_became_peered":"2026-02-24T22:05:24.199555+0000","last_unstale":"2026-02-24T22:34:18.228289+0000","last_undegraded":"2026-02-24T22:34:18.228289+0000","last_fullsized":"2026-02-24T22:34:18.228289+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:23.186988+0000","last_clean_scrub_stamp":"2026-02-24T22:05:23.186988+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:29:23.063946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30969,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":655,"num_read_kb":647,"num_write":639,"num_write_kb":204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"312'432","reported_seq":1950,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.184363+0000","last_change":"2026-02-24T22:05:29.660609+0000","last_active":"2026-02-24T22:34:25.184363+0000","last_peered":"2026-02-24T22:34:25.184363+0000","last_clean":"2026-02-24T22:34:25.184363+0000","last_became_active":"2026-02-24T22:05:29.659532+0000","last_became_peered":"2026-02-24T22:05:29.659532+0000","last_unstale":"2026-02-24T22:34:25.184363+0000","last_undegraded":"2026-02-24T22:34:25.184363+0000","last_fullsized":"2026-02-24T22:34:25.184363+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:05:28.641695+0000","last_clean_scrub_stamp":"2026-02-24T22:05:28.641695+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":400,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:09:07.648679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"312'162","reported_seq":857,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.157775+0000","last_change":"2026-02-24T22:34:45.157775+0000","last_active":"2026-02-24T22:34:45.157775+0000","last_peered":"2026-02-24T22:34:45.157775+0000","last_clean":"2026-02-24T22:34:45.157775+0000","last_became_active":"2026-02-24T22:05:18.184250+0000","last_became_peered":"2026-02-24T22:05:18.184250+0000","last_unstale":"2026-02-24T22:34:45.157775+0000","last_undegraded":"2026-02-24T22:34:45.157775+0000","last_fullsized":"2026-02-24T22:34:45.157775+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'162","last_scrub_stamp":"2026-02-24T22:34:45.157726+0000","last_deep_scrub":"312'162","last_deep_scrub_stamp":"2026-02-24T22:34:45.157726+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.157726+0000","objects_scrubbed":23,"log_size":62,"log_dups_size":100,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:39:01.351061+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":560,"num_read_kb":462,"num_write":204,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"312'1610","reported_seq":2974,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.774854+0000","last_change":"2026-02-24T22:34:33.774854+0000","last_active":"2026-02-24T22:34:33.774854+0000","last_peered":"2026-02-24T22:34:33.774854+0000","last_clean":"2026-02-24T22:34:33.774854+0000","last_became_active":"2026-02-24T22:05:16.189660+0000","last_became_peered":"2026-02-24T22:05:16.189660+0000","last_unstale":"2026-02-24T22:34:33.774854+0000","last_undegraded":"2026-02-24T22:34:33.774854+0000","last_fullsized":"2026-02-24T22:34:33.774854+0000","mapping_epoch":22,"log_start":"312'1500","ondisk_log_start":"312'1500","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'1610","last_scrub_stamp":"2026-02-24T22:34:33.774776+0000","last_deep_scrub":"312'1610","last_deep_scrub_stamp":"2026-02-24T22:34:33.774776+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.774776+0000","objects_scrubbed":6,"log_size":110,"log_dups_size":1500,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:08:14.555476+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4326,"num_read_kb":2601,"num_write":2684,"num_write_kb":4261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"312'121","reported_seq":808,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.128824+0000","last_change":"2026-02-24T22:34:43.128824+0000","last_active":"2026-02-24T22:34:43.128824+0000","last_peered":"2026-02-24T22:34:43.128824+0000","last_clean":"2026-02-24T22:34:43.128824+0000","last_became_active":"2026-02-24T22:05:18.183822+0000","last_became_peered":"2026-02-24T22:05:18.183822+0000","last_unstale":"2026-02-24T22:34:43.128824+0000","last_undegraded":"2026-02-24T22:34:43.128824+0000","last_fullsized":"2026-02-24T22:34:43.128824+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'121","last_scrub_stamp":"2026-02-24T22:34:43.128792+0000","last_deep_scrub":"312'121","last_deep_scrub_stamp":"2026-02-24T22:34:43.128792+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.128792+0000","objects_scrubbed":20,"log_size":21,"log_dups_size":100,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:33:44.340989+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":390,"num_read_kb":330,"num_write":143,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"312'303","reported_seq":1060,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.123769+0000","last_change":"2026-02-24T22:34:32.123769+0000","last_active":"2026-02-24T22:34:32.123769+0000","last_peered":"2026-02-24T22:34:32.123769+0000","last_clean":"2026-02-24T22:34:32.123769+0000","last_became_active":"2026-02-24T22:05:16.183487+0000","last_became_peered":"2026-02-24T22:05:16.183487+0000","last_unstale":"2026-02-24T22:34:32.123769+0000","last_undegraded":"2026-02-24T22:34:32.123769+0000","last_fullsized":"2026-02-24T22:34:32.123769+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'303","last_scrub_stamp":"2026-02-24T22:34:32.123701+0000","last_deep_scrub":"312'303","last_deep_scrub_stamp":"2026-02-24T22:34:32.123701+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.123701+0000","objects_scrubbed":3,"log_size":103,"log_dups_size":200,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:46:27.988906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":514,"num_read_kb":706,"num_write":1357,"num_write_kb":183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"312'92","reported_seq":760,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.957519+0000","last_change":"2026-02-24T22:34:44.957519+0000","last_active":"2026-02-24T22:34:44.957519+0000","last_peered":"2026-02-24T22:34:44.957519+0000","last_clean":"2026-02-24T22:34:44.957519+0000","last_became_active":"2026-02-24T22:05:18.190352+0000","last_became_peered":"2026-02-24T22:05:18.190352+0000","last_unstale":"2026-02-24T22:34:44.957519+0000","last_undegraded":"2026-02-24T22:34:44.957519+0000","last_fullsized":"2026-02-24T22:34:44.957519+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'92","last_scrub_stamp":"2026-02-24T22:34:44.957460+0000","last_deep_scrub":"312'92","last_deep_scrub_stamp":"2026-02-24T22:34:44.957460+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.957460+0000","objects_scrubbed":20,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:57:33.270079+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":274,"num_read_kb":236,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"312'318","reported_seq":1249,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.115775+0000","last_change":"2026-02-24T22:34:38.115775+0000","last_active":"2026-02-24T22:34:38.115775+0000","last_peered":"2026-02-24T22:34:38.115775+0000","last_clean":"2026-02-24T22:34:38.115775+0000","last_became_active":"2026-02-24T22:05:16.190494+0000","last_became_peered":"2026-02-24T22:05:16.190494+0000","last_unstale":"2026-02-24T22:34:38.115775+0000","last_undegraded":"2026-02-24T22:34:38.115775+0000","last_fullsized":"2026-02-24T22:34:38.115775+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'318","last_scrub_stamp":"2026-02-24T22:34:38.115706+0000","last_deep_scrub":"312'318","last_deep_scrub_stamp":"2026-02-24T22:34:38.115706+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.115706+0000","objects_scrubbed":4,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:08:06.722436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1108,"num_read_kb":14913,"num_write":1331,"num_write_kb":8068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"312'367","reported_seq":1190,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.110977+0000","last_change":"2026-02-24T22:34:41.110977+0000","last_active":"2026-02-24T22:34:41.110977+0000","last_peered":"2026-02-24T22:34:41.110977+0000","last_clean":"2026-02-24T22:34:41.110977+0000","last_became_active":"2026-02-24T22:05:18.186622+0000","last_became_peered":"2026-02-24T22:05:18.186622+0000","last_unstale":"2026-02-24T22:34:41.110977+0000","last_undegraded":"2026-02-24T22:34:41.110977+0000","last_fullsized":"2026-02-24T22:34:41.110977+0000","mapping_epoch":24,"log_start":"312'300","ondisk_log_start":"312'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'367","last_scrub_stamp":"2026-02-24T22:34:41.110934+0000","last_deep_scrub":"312'367","last_deep_scrub_stamp":"2026-02-24T22:34:41.110934+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.110934+0000","objects_scrubbed":30,"log_size":67,"log_dups_size":300,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:59:07.931667+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1512,"num_read_kb":1316,"num_write":481,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25750,"num_omap_keys":101,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"312'319","reported_seq":1167,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.742796+0000","last_change":"2026-02-24T22:34:36.742796+0000","last_active":"2026-02-24T22:34:36.742796+0000","last_peered":"2026-02-24T22:34:36.742796+0000","last_clean":"2026-02-24T22:34:36.742796+0000","last_became_active":"2026-02-24T22:05:16.185977+0000","last_became_peered":"2026-02-24T22:05:16.185977+0000","last_unstale":"2026-02-24T22:34:36.742796+0000","last_undegraded":"2026-02-24T22:34:36.742796+0000","last_fullsized":"2026-02-24T22:34:36.742796+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'319","last_scrub_stamp":"2026-02-24T22:34:36.742708+0000","last_deep_scrub":"312'319","last_deep_scrub_stamp":"2026-02-24T22:34:36.742708+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.742708+0000","objects_scrubbed":1,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:21:32.964403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":948,"num_read_kb":5969,"num_write":1368,"num_write_kb":2943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"312'114","reported_seq":799,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.783698+0000","last_change":"2026-02-24T22:34:46.783698+0000","last_active":"2026-02-24T22:34:46.783698+0000","last_peered":"2026-02-24T22:34:46.783698+0000","last_clean":"2026-02-24T22:34:46.783698+0000","last_became_active":"2026-02-24T22:05:18.185180+0000","last_became_peered":"2026-02-24T22:05:18.185180+0000","last_unstale":"2026-02-24T22:34:46.783698+0000","last_undegraded":"2026-02-24T22:34:46.783698+0000","last_fullsized":"2026-02-24T22:34:46.783698+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'114","last_scrub_stamp":"2026-02-24T22:34:46.783630+0000","last_deep_scrub":"312'114","last_deep_scrub_stamp":"2026-02-24T22:34:46.783630+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.783630+0000","objects_scrubbed":25,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:34:22.929604+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":311,"num_read_kb":264,"num_write":113,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":272,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"312'374","reported_seq":1134,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.080105+0000","last_change":"2026-02-24T22:34:34.080105+0000","last_active":"2026-02-24T22:34:34.080105+0000","last_peered":"2026-02-24T22:34:34.080105+0000","last_clean":"2026-02-24T22:34:34.080105+0000","last_became_active":"2026-02-24T22:06:00.660740+0000","last_became_peered":"2026-02-24T22:06:00.660740+0000","last_unstale":"2026-02-24T22:34:34.080105+0000","last_undegraded":"2026-02-24T22:34:34.080105+0000","last_fullsized":"2026-02-24T22:34:34.080105+0000","mapping_epoch":36,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'374","last_scrub_stamp":"2026-02-24T22:34:34.080040+0000","last_deep_scrub":"312'374","last_deep_scrub_stamp":"2026-02-24T22:34:34.080040+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.080040+0000","objects_scrubbed":4,"log_size":74,"log_dups_size":300,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:17:10.260943+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":666,"num_read_kb":9895,"num_write":1405,"num_write_kb":11975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"312'89","reported_seq":768,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.509973+0000","last_change":"2026-02-24T22:34:43.509973+0000","last_active":"2026-02-24T22:34:43.509973+0000","last_peered":"2026-02-24T22:34:43.509973+0000","last_clean":"2026-02-24T22:34:43.509973+0000","last_became_active":"2026-02-24T22:05:18.190390+0000","last_became_peered":"2026-02-24T22:05:18.190390+0000","last_unstale":"2026-02-24T22:34:43.509973+0000","last_undegraded":"2026-02-24T22:34:43.509973+0000","last_fullsized":"2026-02-24T22:34:43.509973+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'89","last_scrub_stamp":"2026-02-24T22:34:43.509906+0000","last_deep_scrub":"312'89","last_deep_scrub_stamp":"2026-02-24T22:34:43.509906+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.509906+0000","objects_scrubbed":23,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:26:36.232359+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":213,"num_read_kb":187,"num_write":90,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"312'336","reported_seq":1067,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.988612+0000","last_change":"2026-02-24T22:34:34.988612+0000","last_active":"2026-02-24T22:34:34.988612+0000","last_peered":"2026-02-24T22:34:34.988612+0000","last_clean":"2026-02-24T22:34:34.988612+0000","last_became_active":"2026-02-24T22:05:16.184091+0000","last_became_peered":"2026-02-24T22:05:16.184091+0000","last_unstale":"2026-02-24T22:34:34.988612+0000","last_undegraded":"2026-02-24T22:34:34.988612+0000","last_fullsized":"2026-02-24T22:34:34.988612+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'336","last_scrub_stamp":"2026-02-24T22:34:34.988543+0000","last_deep_scrub":"312'336","last_deep_scrub_stamp":"2026-02-24T22:34:34.988543+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.988543+0000","objects_scrubbed":4,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:48:27.381128+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":558,"num_read_kb":406,"num_write":1237,"num_write_kb":6221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"312'375","reported_seq":1187,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.800000+0000","last_change":"2026-02-24T22:34:45.800000+0000","last_active":"2026-02-24T22:34:45.800000+0000","last_peered":"2026-02-24T22:34:45.800000+0000","last_clean":"2026-02-24T22:34:45.800000+0000","last_became_active":"2026-02-24T22:05:18.186746+0000","last_became_peered":"2026-02-24T22:05:18.186746+0000","last_unstale":"2026-02-24T22:34:45.800000+0000","last_undegraded":"2026-02-24T22:34:45.800000+0000","last_fullsized":"2026-02-24T22:34:45.800000+0000","mapping_epoch":24,"log_start":"312'300","ondisk_log_start":"312'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'375","last_scrub_stamp":"2026-02-24T22:34:45.799935+0000","last_deep_scrub":"312'375","last_deep_scrub_stamp":"2026-02-24T22:34:45.799935+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.799935+0000","objects_scrubbed":20,"log_size":75,"log_dups_size":300,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:08:40.366084+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":1676,"num_read_kb":1392,"num_write":638,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"312'331","reported_seq":1043,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.774985+0000","last_change":"2026-02-24T22:34:34.774985+0000","last_active":"2026-02-24T22:34:34.774985+0000","last_peered":"2026-02-24T22:34:34.774985+0000","last_clean":"2026-02-24T22:34:34.774985+0000","last_became_active":"2026-02-24T22:05:16.189516+0000","last_became_peered":"2026-02-24T22:05:16.189516+0000","last_unstale":"2026-02-24T22:34:34.774985+0000","last_undegraded":"2026-02-24T22:34:34.774985+0000","last_fullsized":"2026-02-24T22:34:34.774985+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'331","last_scrub_stamp":"2026-02-24T22:34:34.774895+0000","last_deep_scrub":"312'331","last_deep_scrub_stamp":"2026-02-24T22:34:34.774895+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.774895+0000","objects_scrubbed":2,"log_size":31,"log_dups_size":300,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:51:58.631151+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":448,"num_read_kb":5491,"num_write":1228,"num_write_kb":20654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"312'184","reported_seq":884,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.708693+0000","last_change":"2026-02-24T22:34:40.708693+0000","last_active":"2026-02-24T22:34:40.708693+0000","last_peered":"2026-02-24T22:34:40.708693+0000","last_clean":"2026-02-24T22:34:40.708693+0000","last_became_active":"2026-02-24T22:05:18.190203+0000","last_became_peered":"2026-02-24T22:05:18.190203+0000","last_unstale":"2026-02-24T22:34:40.708693+0000","last_undegraded":"2026-02-24T22:34:40.708693+0000","last_fullsized":"2026-02-24T22:34:40.708693+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'184","last_scrub_stamp":"2026-02-24T22:34:40.708623+0000","last_deep_scrub":"312'184","last_deep_scrub_stamp":"2026-02-24T22:34:40.708623+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.708623+0000","objects_scrubbed":20,"log_size":84,"log_dups_size":100,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:12:32.032756+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":631,"num_read_kb":516,"num_write":248,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"312'319","reported_seq":1046,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.869813+0000","last_change":"2026-02-24T22:34:35.869813+0000","last_active":"2026-02-24T22:34:35.869813+0000","last_peered":"2026-02-24T22:34:35.869813+0000","last_clean":"2026-02-24T22:34:35.869813+0000","last_became_active":"2026-02-24T22:05:16.183772+0000","last_became_peered":"2026-02-24T22:05:16.183772+0000","last_unstale":"2026-02-24T22:34:35.869813+0000","last_undegraded":"2026-02-24T22:34:35.869813+0000","last_fullsized":"2026-02-24T22:34:35.869813+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'319","last_scrub_stamp":"2026-02-24T22:34:35.869749+0000","last_deep_scrub":"312'319","last_deep_scrub_stamp":"2026-02-24T22:34:35.869749+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.869749+0000","objects_scrubbed":2,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:10:38.609121+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":480,"num_read_kb":12489,"num_write":1341,"num_write_kb":5970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"312'156","reported_seq":885,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.880154+0000","last_change":"2026-02-24T22:34:45.880154+0000","last_active":"2026-02-24T22:34:45.880154+0000","last_peered":"2026-02-24T22:34:45.880154+0000","last_clean":"2026-02-24T22:34:45.880154+0000","last_became_active":"2026-02-24T22:05:18.187321+0000","last_became_peered":"2026-02-24T22:05:18.187321+0000","last_unstale":"2026-02-24T22:34:45.880154+0000","last_undegraded":"2026-02-24T22:34:45.880154+0000","last_fullsized":"2026-02-24T22:34:45.880154+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'156","last_scrub_stamp":"2026-02-24T22:34:45.880085+0000","last_deep_scrub":"312'156","last_deep_scrub_stamp":"2026-02-24T22:34:45.880085+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.880085+0000","objects_scrubbed":26,"log_size":56,"log_dups_size":100,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:17:02.414903+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":512,"num_read_kb":441,"num_write":201,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"312'326","reported_seq":1042,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.166643+0000","last_change":"2026-02-24T22:34:37.166643+0000","last_active":"2026-02-24T22:34:37.166643+0000","last_peered":"2026-02-24T22:34:37.166643+0000","last_clean":"2026-02-24T22:34:37.166643+0000","last_became_active":"2026-02-24T22:05:16.193164+0000","last_became_peered":"2026-02-24T22:05:16.193164+0000","last_unstale":"2026-02-24T22:34:37.166643+0000","last_undegraded":"2026-02-24T22:34:37.166643+0000","last_fullsized":"2026-02-24T22:34:37.166643+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'326","last_scrub_stamp":"2026-02-24T22:34:37.166498+0000","last_deep_scrub":"312'326","last_deep_scrub_stamp":"2026-02-24T22:34:37.166498+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.166498+0000","objects_scrubbed":6,"log_size":26,"log_dups_size":300,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:54:53.815454+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242922,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":434,"num_read_kb":36424,"num_write":1378,"num_write_kb":10617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"312'166","reported_seq":852,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.417427+0000","last_change":"2026-02-24T22:34:46.417427+0000","last_active":"2026-02-24T22:34:46.417427+0000","last_peered":"2026-02-24T22:34:46.417427+0000","last_clean":"2026-02-24T22:34:46.417427+0000","last_became_active":"2026-02-24T22:05:18.191737+0000","last_became_peered":"2026-02-24T22:05:18.191737+0000","last_unstale":"2026-02-24T22:34:46.417427+0000","last_undegraded":"2026-02-24T22:34:46.417427+0000","last_fullsized":"2026-02-24T22:34:46.417427+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'166","last_scrub_stamp":"2026-02-24T22:34:46.417398+0000","last_deep_scrub":"312'166","last_deep_scrub_stamp":"2026-02-24T22:34:46.417398+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.417398+0000","objects_scrubbed":18,"log_size":66,"log_dups_size":100,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:30:36.821543+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":539,"num_read_kb":448,"num_write":211,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"312'377","reported_seq":1137,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.766166+0000","last_change":"2026-02-24T22:34:38.766166+0000","last_active":"2026-02-24T22:34:38.766166+0000","last_peered":"2026-02-24T22:34:38.766166+0000","last_clean":"2026-02-24T22:34:38.766166+0000","last_became_active":"2026-02-24T22:05:16.183923+0000","last_became_peered":"2026-02-24T22:05:16.183923+0000","last_unstale":"2026-02-24T22:34:38.766166+0000","last_undegraded":"2026-02-24T22:34:38.766166+0000","last_fullsized":"2026-02-24T22:34:38.766166+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'377","last_scrub_stamp":"2026-02-24T22:34:38.766121+0000","last_deep_scrub":"312'377","last_deep_scrub_stamp":"2026-02-24T22:34:38.766121+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.766121+0000","objects_scrubbed":6,"log_size":77,"log_dups_size":300,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:41:28.025068+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242922,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":549,"num_read_kb":9842,"num_write":1598,"num_write_kb":9073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"312'167","reported_seq":848,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.060961+0000","last_change":"2026-02-24T22:34:39.060923+0000","last_active":"2026-02-24T22:34:39.060961+0000","last_peered":"2026-02-24T22:34:39.060961+0000","last_clean":"2026-02-24T22:34:39.060961+0000","last_became_active":"2026-02-24T22:05:18.183761+0000","last_became_peered":"2026-02-24T22:05:18.183761+0000","last_unstale":"2026-02-24T22:34:39.060961+0000","last_undegraded":"2026-02-24T22:34:39.060961+0000","last_fullsized":"2026-02-24T22:34:39.060961+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'167","last_scrub_stamp":"2026-02-24T22:34:39.060863+0000","last_deep_scrub":"312'167","last_deep_scrub_stamp":"2026-02-24T22:34:39.060863+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.060863+0000","objects_scrubbed":21,"log_size":67,"log_dups_size":100,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:07:32.105462+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":561,"num_read_kb":451,"num_write":221,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"312'341","reported_seq":1119,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.077639+0000","last_change":"2026-02-24T22:34:39.077639+0000","last_active":"2026-02-24T22:34:39.077639+0000","last_peered":"2026-02-24T22:34:39.077639+0000","last_clean":"2026-02-24T22:34:39.077639+0000","last_became_active":"2026-02-24T22:05:16.189759+0000","last_became_peered":"2026-02-24T22:05:16.189759+0000","last_unstale":"2026-02-24T22:34:39.077639+0000","last_undegraded":"2026-02-24T22:34:39.077639+0000","last_fullsized":"2026-02-24T22:34:39.077639+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'341","last_scrub_stamp":"2026-02-24T22:34:39.077571+0000","last_deep_scrub":"312'341","last_deep_scrub_stamp":"2026-02-24T22:34:39.077571+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.077571+0000","objects_scrubbed":4,"log_size":41,"log_dups_size":300,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:09:54.075658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":656,"num_read_kb":3529,"num_write":1503,"num_write_kb":3094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"312'158","reported_seq":860,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.440468+0000","last_change":"2026-02-24T22:34:45.440468+0000","last_active":"2026-02-24T22:34:45.440468+0000","last_peered":"2026-02-24T22:34:45.440468+0000","last_clean":"2026-02-24T22:34:45.440468+0000","last_became_active":"2026-02-24T22:05:18.192882+0000","last_became_peered":"2026-02-24T22:05:18.192882+0000","last_unstale":"2026-02-24T22:34:45.440468+0000","last_undegraded":"2026-02-24T22:34:45.440468+0000","last_fullsized":"2026-02-24T22:34:45.440468+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'158","last_scrub_stamp":"2026-02-24T22:34:45.440425+0000","last_deep_scrub":"312'158","last_deep_scrub_stamp":"2026-02-24T22:34:45.440425+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.440425+0000","objects_scrubbed":18,"log_size":58,"log_dups_size":100,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:49:26.113842+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":593,"num_read_kb":494,"num_write":208,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"312'309","reported_seq":1056,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.732158+0000","last_change":"2026-02-24T22:34:35.732158+0000","last_active":"2026-02-24T22:34:35.732158+0000","last_peered":"2026-02-24T22:34:35.732158+0000","last_clean":"2026-02-24T22:34:35.732158+0000","last_became_active":"2026-02-24T22:05:16.184557+0000","last_became_peered":"2026-02-24T22:05:16.184557+0000","last_unstale":"2026-02-24T22:34:35.732158+0000","last_undegraded":"2026-02-24T22:34:35.732158+0000","last_fullsized":"2026-02-24T22:34:35.732158+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'309","last_scrub_stamp":"2026-02-24T22:34:35.732082+0000","last_deep_scrub":"312'309","last_deep_scrub_stamp":"2026-02-24T22:34:35.732082+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.732082+0000","objects_scrubbed":3,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:22:57.505362+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":498,"num_read_kb":1536,"num_write":1259,"num_write_kb":5590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"312'184","reported_seq":901,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.156109+0000","last_change":"2026-02-24T22:34:45.156109+0000","last_active":"2026-02-24T22:34:45.156109+0000","last_peered":"2026-02-24T22:34:45.156109+0000","last_clean":"2026-02-24T22:34:45.156109+0000","last_became_active":"2026-02-24T22:05:18.191432+0000","last_became_peered":"2026-02-24T22:05:18.191432+0000","last_unstale":"2026-02-24T22:34:45.156109+0000","last_undegraded":"2026-02-24T22:34:45.156109+0000","last_fullsized":"2026-02-24T22:34:45.156109+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'184","last_scrub_stamp":"2026-02-24T22:34:45.156041+0000","last_deep_scrub":"312'184","last_deep_scrub_stamp":"2026-02-24T22:34:45.156041+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.156041+0000","objects_scrubbed":26,"log_size":84,"log_dups_size":100,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:54:34.074113+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":649,"num_read_kb":535,"num_write":233,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4005,"num_omap_keys":18,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"312'287","reported_seq":1017,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.765441+0000","last_change":"2026-02-24T22:34:36.765441+0000","last_active":"2026-02-24T22:34:36.765441+0000","last_peered":"2026-02-24T22:34:36.765441+0000","last_clean":"2026-02-24T22:34:36.765441+0000","last_became_active":"2026-02-24T22:05:16.190260+0000","last_became_peered":"2026-02-24T22:05:16.190260+0000","last_unstale":"2026-02-24T22:34:36.765441+0000","last_undegraded":"2026-02-24T22:34:36.765441+0000","last_fullsized":"2026-02-24T22:34:36.765441+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'287","last_scrub_stamp":"2026-02-24T22:34:36.765378+0000","last_deep_scrub":"312'287","last_deep_scrub_stamp":"2026-02-24T22:34:36.765378+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.765378+0000","objects_scrubbed":2,"log_size":87,"log_dups_size":200,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:18:33.840972+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":458,"num_read_kb":1504,"num_write":1357,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"312'171","reported_seq":869,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.861467+0000","last_change":"2026-02-24T22:34:44.861467+0000","last_active":"2026-02-24T22:34:44.861467+0000","last_peered":"2026-02-24T22:34:44.861467+0000","last_clean":"2026-02-24T22:34:44.861467+0000","last_became_active":"2026-02-24T22:05:18.185184+0000","last_became_peered":"2026-02-24T22:05:18.185184+0000","last_unstale":"2026-02-24T22:34:44.861467+0000","last_undegraded":"2026-02-24T22:34:44.861467+0000","last_fullsized":"2026-02-24T22:34:44.861467+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'171","last_scrub_stamp":"2026-02-24T22:34:44.861426+0000","last_deep_scrub":"312'171","last_deep_scrub_stamp":"2026-02-24T22:34:44.861426+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.861426+0000","objects_scrubbed":28,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:45:55.477850+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":536,"num_read_kb":445,"num_write":200,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"312'302","reported_seq":1124,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.033851+0000","last_change":"2026-02-24T22:34:40.033851+0000","last_active":"2026-02-24T22:34:40.033851+0000","last_peered":"2026-02-24T22:34:40.033851+0000","last_clean":"2026-02-24T22:34:40.033851+0000","last_became_active":"2026-02-24T22:05:16.191433+0000","last_became_peered":"2026-02-24T22:05:16.191433+0000","last_unstale":"2026-02-24T22:34:40.033851+0000","last_undegraded":"2026-02-24T22:34:40.033851+0000","last_fullsized":"2026-02-24T22:34:40.033851+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'302","last_scrub_stamp":"2026-02-24T22:34:40.033782+0000","last_deep_scrub":"312'302","last_deep_scrub_stamp":"2026-02-24T22:34:40.033782+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.033782+0000","objects_scrubbed":3,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:16:57.803888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":549,"num_read_kb":2264,"num_write":1330,"num_write_kb":7006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"312'86","reported_seq":755,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.772530+0000","last_change":"2026-02-24T22:34:48.772530+0000","last_active":"2026-02-24T22:34:48.772530+0000","last_peered":"2026-02-24T22:34:48.772530+0000","last_clean":"2026-02-24T22:34:48.772530+0000","last_became_active":"2026-02-24T22:05:18.192619+0000","last_became_peered":"2026-02-24T22:05:18.192619+0000","last_unstale":"2026-02-24T22:34:48.772530+0000","last_undegraded":"2026-02-24T22:34:48.772530+0000","last_fullsized":"2026-02-24T22:34:48.772530+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'86","last_scrub_stamp":"2026-02-24T22:34:48.772479+0000","last_deep_scrub":"312'86","last_deep_scrub_stamp":"2026-02-24T22:34:48.772479+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.772479+0000","objects_scrubbed":24,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:50:44.256005+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":204,"num_read_kb":176,"num_write":76,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"312'336","reported_seq":1076,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.091228+0000","last_change":"2026-02-24T22:34:33.091228+0000","last_active":"2026-02-24T22:34:33.091228+0000","last_peered":"2026-02-24T22:34:33.091228+0000","last_clean":"2026-02-24T22:34:33.091228+0000","last_became_active":"2026-02-24T22:05:16.184441+0000","last_became_peered":"2026-02-24T22:05:16.184441+0000","last_unstale":"2026-02-24T22:34:33.091228+0000","last_undegraded":"2026-02-24T22:34:33.091228+0000","last_fullsized":"2026-02-24T22:34:33.091228+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'336","last_scrub_stamp":"2026-02-24T22:34:33.091160+0000","last_deep_scrub":"312'336","last_deep_scrub_stamp":"2026-02-24T22:34:33.091160+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.091160+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:19:40.057693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":478,"num_read_kb":3856,"num_write":1468,"num_write_kb":8445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"312'183","reported_seq":909,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.176877+0000","last_change":"2026-02-24T22:34:48.176877+0000","last_active":"2026-02-24T22:34:48.176877+0000","last_peered":"2026-02-24T22:34:48.176877+0000","last_clean":"2026-02-24T22:34:48.176877+0000","last_became_active":"2026-02-24T22:05:18.191350+0000","last_became_peered":"2026-02-24T22:05:18.191350+0000","last_unstale":"2026-02-24T22:34:48.176877+0000","last_undegraded":"2026-02-24T22:34:48.176877+0000","last_fullsized":"2026-02-24T22:34:48.176877+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'183","last_scrub_stamp":"2026-02-24T22:34:48.176839+0000","last_deep_scrub":"312'183","last_deep_scrub_stamp":"2026-02-24T22:34:48.176839+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.176839+0000","objects_scrubbed":24,"log_size":83,"log_dups_size":100,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:23:04.478635+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":662,"num_read_kb":557,"num_write":245,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"312'305","reported_seq":1081,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.042678+0000","last_change":"2026-02-24T22:34:35.042678+0000","last_active":"2026-02-24T22:34:35.042678+0000","last_peered":"2026-02-24T22:34:35.042678+0000","last_clean":"2026-02-24T22:34:35.042678+0000","last_became_active":"2026-02-24T22:06:00.662656+0000","last_became_peered":"2026-02-24T22:06:00.662656+0000","last_unstale":"2026-02-24T22:34:35.042678+0000","last_undegraded":"2026-02-24T22:34:35.042678+0000","last_fullsized":"2026-02-24T22:34:35.042678+0000","mapping_epoch":36,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'305","last_scrub_stamp":"2026-02-24T22:34:35.042539+0000","last_deep_scrub":"312'305","last_deep_scrub_stamp":"2026-02-24T22:34:35.042539+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.042539+0000","objects_scrubbed":4,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:24:23.913211+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":803,"num_read_kb":6090,"num_write":1203,"num_write_kb":5556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.30","version":"312'347","reported_seq":1107,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.049845+0000","last_change":"2026-02-24T22:34:40.049845+0000","last_active":"2026-02-24T22:34:40.049845+0000","last_peered":"2026-02-24T22:34:40.049845+0000","last_clean":"2026-02-24T22:34:40.049845+0000","last_became_active":"2026-02-24T22:05:16.189713+0000","last_became_peered":"2026-02-24T22:05:16.189713+0000","last_unstale":"2026-02-24T22:34:40.049845+0000","last_undegraded":"2026-02-24T22:34:40.049845+0000","last_fullsized":"2026-02-24T22:34:40.049845+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'347","last_scrub_stamp":"2026-02-24T22:34:40.049764+0000","last_deep_scrub":"312'347","last_deep_scrub_stamp":"2026-02-24T22:34:40.049764+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.049764+0000","objects_scrubbed":8,"log_size":47,"log_dups_size":300,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:52:46.777711+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242932,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":632,"num_read_kb":31410,"num_write":1455,"num_write_kb":10638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"312'273","reported_seq":967,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.873634+0000","last_change":"2026-02-24T22:34:46.873634+0000","last_active":"2026-02-24T22:34:46.873634+0000","last_peered":"2026-02-24T22:34:46.873634+0000","last_clean":"2026-02-24T22:34:46.873634+0000","last_became_active":"2026-02-24T22:05:18.187565+0000","last_became_peered":"2026-02-24T22:05:18.187565+0000","last_unstale":"2026-02-24T22:34:46.873634+0000","last_undegraded":"2026-02-24T22:34:46.873634+0000","last_fullsized":"2026-02-24T22:34:46.873634+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'273","last_scrub_stamp":"2026-02-24T22:34:46.873575+0000","last_deep_scrub":"312'273","last_deep_scrub_stamp":"2026-02-24T22:34:46.873575+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.873575+0000","objects_scrubbed":17,"log_size":73,"log_dups_size":200,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:03:00.743450+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":1060,"num_read_kb":853,"num_write":419,"num_write_kb":222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"312'299","reported_seq":1013,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.068984+0000","last_change":"2026-02-24T22:34:37.068984+0000","last_active":"2026-02-24T22:34:37.068984+0000","last_peered":"2026-02-24T22:34:37.068984+0000","last_clean":"2026-02-24T22:34:37.068984+0000","last_became_active":"2026-02-24T22:05:16.178157+0000","last_became_peered":"2026-02-24T22:05:16.178157+0000","last_unstale":"2026-02-24T22:34:37.068984+0000","last_undegraded":"2026-02-24T22:34:37.068984+0000","last_fullsized":"2026-02-24T22:34:37.068984+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'299","last_scrub_stamp":"2026-02-24T22:34:37.068917+0000","last_deep_scrub":"312'299","last_deep_scrub_stamp":"2026-02-24T22:34:37.068917+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.068917+0000","objects_scrubbed":5,"log_size":99,"log_dups_size":200,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:17:09.219950+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":466,"num_read_kb":1640,"num_write":1134,"num_write_kb":1702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"312'98","reported_seq":781,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.871267+0000","last_change":"2026-02-24T22:34:47.871267+0000","last_active":"2026-02-24T22:34:47.871267+0000","last_peered":"2026-02-24T22:34:47.871267+0000","last_clean":"2026-02-24T22:34:47.871267+0000","last_became_active":"2026-02-24T22:05:18.190902+0000","last_became_peered":"2026-02-24T22:05:18.190902+0000","last_unstale":"2026-02-24T22:34:47.871267+0000","last_undegraded":"2026-02-24T22:34:47.871267+0000","last_fullsized":"2026-02-24T22:34:47.871267+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'98","last_scrub_stamp":"2026-02-24T22:34:47.871207+0000","last_deep_scrub":"312'98","last_deep_scrub_stamp":"2026-02-24T22:34:47.871207+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.871207+0000","objects_scrubbed":20,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:26:15.293776+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":302,"num_read_kb":262,"num_write":98,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":425,"ondisk_log_size":425,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":973109,"num_objects":3053,"num_object_clones":0,"num_object_copies":6106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3053,"num_whiteouts":0,"num_read":19978,"num_read_kb":245567,"num_write":23762,"num_write_kb":8619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":366,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24911872,"data_stored":1946218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5786661,"internal_metadata":0},"log_size":2140,"ondisk_log_size":2140,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1250269,"num_objects":223,"num_object_clones":0,"num_object_copies":446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":223,"num_whiteouts":0,"num_read":45268,"num_read_kb":47987,"num_write":15451,"num_write_kb":13662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":17220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2823344,"internal_metadata":0},"log_size":2152,"ondisk_log_size":2152,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1896,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1448,"num_read_kb":1680,"num_write":22,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14,"ondisk_log_size":14,"up":64,"acting":64,"num_store_stats":8},{"poolid":82,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11005,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":1511,"num_object_clones":0,"num_object_copies":3022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1511,"num_whiteouts":0,"num_read":95919,"num_read_kb":81897,"num_write":34645,"num_write_kb":17078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1511,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":35257,"num_omap_keys":142,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17301930,"internal_metadata":0},"log_size":4315,"ondisk_log_size":4315,"up":128,"acting":128,"num_store_stats":8},{"poolid":81,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":119816,"num_write":82,"num_write_kb":70656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":112199743,"num_objects":281,"num_object_clones":0,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":0,"num_read":41102,"num_read_kb":763537,"num_write":86142,"num_write_kb":523059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":225296384,"data_stored":224399486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4246,"ondisk_log_size":4246,"up":128,"acting":128,"num_store_stats":8},{"poolid":80,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":124936,"num_write":82,"num_write_kb":70656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":7},{"poolid":79,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3567,"num_read_kb":2401,"num_write":1127,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":317720,"internal_metadata":0},"log_size":462,"ondisk_log_size":462,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14,"ondisk_log_size":14,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575207,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316704,"kb_used_data":55120,"kb_used_omap":3696,"kb_used_meta":257871,"kb_avail":94055136,"statfs":{"total":96636764160,"available":96312459264,"internally_reserved":0,"allocated":56442880,"data_stored":51634967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3785637,"internal_metadata":264059995},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575207,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":271964,"kb_used_data":28428,"kb_used_omap":3571,"kb_used_meta":239948,"kb_avail":94099876,"statfs":{"total":96636764160,"available":96358273024,"internally_reserved":0,"allocated":29110272,"data_stored":23825325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3657488,"internal_metadata":245706992},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575207,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137044,"kb_used_data":37956,"kb_used_omap":2746,"kb_used_meta":96325,"kb_avail":94234796,"statfs":{"total":96636764160,"available":96496431104,"internally_reserved":0,"allocated":38866944,"data_stored":35212820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2812671,"internal_metadata":98637057},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575207,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":111224,"kb_used_data":48168,"kb_used_omap":3095,"kb_used_meta":59944,"kb_avail":94260616,"statfs":{"total":96636764160,"available":96522870784,"internally_reserved":0,"allocated":49324032,"data_stored":45700457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3169290,"internal_metadata":61383670},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575208,"num_pgs":107,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":225820,"kb_used_data":18380,"kb_used_omap":7232,"kb_used_meta":200191,"kb_avail":94146020,"statfs":{"total":96636764160,"available":96405524480,"internally_reserved":0,"allocated":18821120,"data_stored":13350111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7406186,"internal_metadata":204995990},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575208,"num_pgs":87,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62440,"kb_used_data":17432,"kb_used_omap":1599,"kb_used_meta":43392,"kb_avail":94309400,"statfs":{"total":96636764160,"available":96572825600,"internally_reserved":0,"allocated":17850368,"data_stored":13280880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1638118,"internal_metadata":44433690},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575209,"num_pgs":102,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":129624,"kb_used_data":12424,"kb_used_omap":5094,"kb_used_meta":112089,"kb_avail":94242216,"statfs":{"total":96636764160,"available":96504029184,"internally_reserved":0,"allocated":12722176,"data_stored":8040949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5216565,"internal_metadata":114779851},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575209,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":286372,"kb_used_data":60948,"kb_used_omap":10223,"kb_used_meta":215184,"kb_avail":94085468,"statfs":{"total":96636764160,"available":96343519232,"internally_reserved":0,"allocated":62410752,"data_stored":56934149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10468372,"internal_metadata":220349420},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53624832,"data_stored":53478090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5324800,"data_stored":5243270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":10555392,"data_stored":10486056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10620928,"data_stored":10486391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":43110400,"data_stored":42992165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":32641024,"data_stored":32506526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":21032960,"data_stored":20971830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":48386048,"data_stored":48235158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1667351,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2633902,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1516609,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1844562,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2430346,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":723382,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3546560,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2939218,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":2106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1512,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1072852,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":3356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18545,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":2058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5184,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1705285,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11530,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":2244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8436,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3928064,"data_stored":308322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51420,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3117056,"data_stored":241195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":152550,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3059712,"data_stored":239505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":825183,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3899392,"data_stored":305873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":336934,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1949696,"data_stored":151651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7338,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1925120,"data_stored":148450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":65762,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3837952,"data_stored":298046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2489145,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3194880,"data_stored":253176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1858329,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":79,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":133235,"internal_metadata":0},{"poolid":79,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14290,"internal_metadata":0},{"poolid":79,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23118,"internal_metadata":0},{"poolid":79,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15635,"internal_metadata":0},{"poolid":79,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49631,"internal_metadata":0},{"poolid":79,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49251,"internal_metadata":0},{"poolid":79,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16656,"internal_metadata":0},{"poolid":79,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15904,"internal_metadata":0},{"poolid":80,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":82,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":82,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1886,"internal_metadata":0},{"poolid":82,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6440,"internal_metadata":0},{"poolid":82,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":262,"internal_metadata":0},{"poolid":82,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1265,"internal_metadata":0},{"poolid":82,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":717,"internal_metadata":0},{"poolid":82,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":435,"internal_metadata":0},{"poolid":82,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T22:34:51.247 INFO:tasks.ceph:pgid 82.2 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.247 INFO:tasks.ceph:pgid 81.1 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.247 INFO:tasks.ceph:pgid 80.0 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.247 INFO:tasks.ceph:pgid 79.1f last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.247 INFO:tasks.ceph:pgid 82.3 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.247 INFO:tasks.ceph:pgid 81.0 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.247 INFO:tasks.ceph:pgid 80.1 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.248 INFO:tasks.ceph:pgid 79.1e last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.248 INFO:tasks.ceph:pgid 82.0 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.248 INFO:tasks.ceph:pgid 81.3 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.248 INFO:tasks.ceph:pgid 80.2 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.248 INFO:tasks.ceph:pgid 79.1d last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.248 INFO:tasks.ceph:pgid 82.1 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.248 INFO:tasks.ceph:pgid 81.2 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.248 INFO:tasks.ceph:pgid 80.3 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.248 INFO:tasks.ceph:pgid 79.1c last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.248 INFO:tasks.ceph:pgid 82.6 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.248 INFO:tasks.ceph:pgid 81.5 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.249 INFO:tasks.ceph:pgid 80.4 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.249 INFO:tasks.ceph:pgid 79.1b last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.249 INFO:tasks.ceph:pgid 82.7 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.249 INFO:tasks.ceph:pgid 81.4 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.249 INFO:tasks.ceph:pgid 80.5 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.249 INFO:tasks.ceph:pgid 79.1a last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.249 INFO:tasks.ceph:pgid 82.4 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.249 INFO:tasks.ceph:pgid 81.7 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.249 INFO:tasks.ceph:pgid 80.6 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.249 INFO:tasks.ceph:pgid 79.19 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.249 INFO:tasks.ceph:pgid 82.5 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.249 INFO:tasks.ceph:pgid 81.6 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.250 INFO:tasks.ceph:pgid 80.7 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.250 INFO:tasks.ceph:pgid 79.18 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.250 INFO:tasks.ceph:pgid 82.a last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.250 INFO:tasks.ceph:pgid 81.9 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.250 INFO:tasks.ceph:pgid 80.8 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.250 INFO:tasks.ceph:pgid 79.17 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.250 INFO:tasks.ceph:pgid 82.b last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.250 INFO:tasks.ceph:pgid 81.8 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.250 INFO:tasks.ceph:pgid 80.9 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.250 INFO:tasks.ceph:pgid 79.16 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.250 INFO:tasks.ceph:pgid 82.8 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.250 INFO:tasks.ceph:pgid 81.b last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.251 INFO:tasks.ceph:pgid 80.a last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.251 INFO:tasks.ceph:pgid 79.15 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.251 INFO:tasks.ceph:pgid 82.9 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.251 INFO:tasks.ceph:pgid 81.a last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.251 INFO:tasks.ceph:pgid 80.b last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.251 INFO:tasks.ceph:pgid 79.14 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.251 INFO:tasks.ceph:pgid 82.e last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.251 INFO:tasks.ceph:pgid 81.d last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.251 INFO:tasks.ceph:pgid 80.c last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.251 INFO:tasks.ceph:pgid 79.13 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.251 INFO:tasks.ceph:pgid 82.f last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.251 INFO:tasks.ceph:pgid 81.c last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.252 INFO:tasks.ceph:pgid 80.d last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.252 INFO:tasks.ceph:pgid 79.12 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.252 INFO:tasks.ceph:pgid 82.c last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.252 INFO:tasks.ceph:pgid 81.f last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.252 INFO:tasks.ceph:pgid 80.e last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.252 INFO:tasks.ceph:pgid 79.11 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.252 INFO:tasks.ceph:pgid 82.d last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.252 INFO:tasks.ceph:pgid 81.e last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.252 INFO:tasks.ceph:pgid 80.f last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.252 INFO:tasks.ceph:pgid 79.10 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.252 INFO:tasks.ceph:pgid 82.12 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.252 INFO:tasks.ceph:pgid 81.11 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.252 INFO:tasks.ceph:pgid 80.10 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.253 INFO:tasks.ceph:pgid 79.f last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.253 INFO:tasks.ceph:pgid 82.13 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.253 INFO:tasks.ceph:pgid 81.10 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.253 INFO:tasks.ceph:pgid 80.11 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.253 INFO:tasks.ceph:pgid 79.e last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.253 INFO:tasks.ceph:pgid 82.10 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.253 INFO:tasks.ceph:pgid 81.13 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.253 INFO:tasks.ceph:pgid 80.12 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.253 INFO:tasks.ceph:pgid 79.d last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.253 INFO:tasks.ceph:pgid 82.11 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.253 INFO:tasks.ceph:pgid 81.12 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.253 INFO:tasks.ceph:pgid 80.13 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.254 INFO:tasks.ceph:pgid 79.c last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.254 INFO:tasks.ceph:pgid 82.16 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.254 INFO:tasks.ceph:pgid 81.15 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.254 INFO:tasks.ceph:pgid 80.14 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.254 INFO:tasks.ceph:pgid 79.b last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.254 INFO:tasks.ceph:pgid 82.17 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.254 INFO:tasks.ceph:pgid 81.14 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.254 INFO:tasks.ceph:pgid 80.15 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.254 INFO:tasks.ceph:pgid 79.a last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.254 INFO:tasks.ceph:pgid 82.14 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.254 INFO:tasks.ceph:pgid 81.17 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.254 INFO:tasks.ceph:pgid 80.16 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.254 INFO:tasks.ceph:pgid 79.9 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.255 INFO:tasks.ceph:pgid 82.15 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.255 INFO:tasks.ceph:pgid 81.16 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.255 INFO:tasks.ceph:pgid 80.17 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.255 INFO:tasks.ceph:pgid 79.8 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.255 INFO:tasks.ceph:pgid 82.1a last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.255 INFO:tasks.ceph:pgid 81.19 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.255 INFO:tasks.ceph:pgid 80.18 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.255 INFO:tasks.ceph:pgid 79.7 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.255 INFO:tasks.ceph:pgid 82.1b last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.255 INFO:tasks.ceph:pgid 81.18 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.255 INFO:tasks.ceph:pgid 80.19 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.255 INFO:tasks.ceph:pgid 79.6 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.255 INFO:tasks.ceph:pgid 82.18 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.256 INFO:tasks.ceph:pgid 81.1b last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.256 INFO:tasks.ceph:pgid 80.1a last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.256 INFO:tasks.ceph:pgid 79.5 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.256 INFO:tasks.ceph:pgid 82.19 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.256 INFO:tasks.ceph:pgid 81.1a last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.256 INFO:tasks.ceph:pgid 80.1b last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.256 INFO:tasks.ceph:pgid 79.4 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.256 INFO:tasks.ceph:pgid 82.1e last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.256 INFO:tasks.ceph:pgid 81.1d last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.256 INFO:tasks.ceph:pgid 80.1c last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.256 INFO:tasks.ceph:pgid 79.3 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.256 INFO:tasks.ceph:pgid 82.1f last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.256 INFO:tasks.ceph:pgid 81.1c last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.256 INFO:tasks.ceph:pgid 80.1d last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.257 INFO:tasks.ceph:pgid 79.2 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.257 INFO:tasks.ceph:pgid 82.1c last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.257 INFO:tasks.ceph:pgid 81.1f last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.257 INFO:tasks.ceph:pgid 80.1e last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.257 INFO:tasks.ceph:pgid 79.1 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.257 INFO:tasks.ceph:pgid 82.1d last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.257 INFO:tasks.ceph:pgid 81.1e last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.257 INFO:tasks.ceph:pgid 80.1f last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.257 INFO:tasks.ceph:pgid 79.0 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.257 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2026-02-24T22:05:17.174213+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=17, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.258 INFO:tasks.ceph:pgid 8.18 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.258 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.258 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.258 INFO:tasks.ceph:pgid 7.17 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.258 INFO:tasks.ceph:pgid 8.19 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.258 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.258 INFO:tasks.ceph:pgid 6.17 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.258 INFO:tasks.ceph:pgid 7.16 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.258 INFO:tasks.ceph:pgid 8.1a last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.258 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.258 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.258 INFO:tasks.ceph:pgid 8.1b last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.258 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.259 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.259 INFO:tasks.ceph:pgid 7.14 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.259 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.259 INFO:tasks.ceph:pgid 6.a last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.259 INFO:tasks.ceph:pgid 7.b last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.259 INFO:tasks.ceph:pgid 8.4 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.259 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.259 INFO:tasks.ceph:pgid 7.a last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.259 INFO:tasks.ceph:pgid 8.5 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.259 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.259 INFO:tasks.ceph:pgid 7.9 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.259 INFO:tasks.ceph:pgid 8.6 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.260 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.260 INFO:tasks.ceph:pgid 8.7 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.260 INFO:tasks.ceph:pgid 6.e last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.260 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.260 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.260 INFO:tasks.ceph:pgid 7.e last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.260 INFO:tasks.ceph:pgid 8.1 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.260 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.260 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.260 INFO:tasks.ceph:pgid 8.2 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.260 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.260 INFO:tasks.ceph:pgid 6.d last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.260 INFO:tasks.ceph:pgid 8.3 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.261 INFO:tasks.ceph:pgid 6.7 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.261 INFO:tasks.ceph:pgid 7.6 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.261 INFO:tasks.ceph:pgid 8.9 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.261 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.261 INFO:tasks.ceph:pgid 6.4 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.261 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.261 INFO:tasks.ceph:pgid 8.a last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.261 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.261 INFO:tasks.ceph:pgid 7.4 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.261 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.261 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.261 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.262 INFO:tasks.ceph:pgid 8.8 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.262 INFO:tasks.ceph:pgid 6.0 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.262 INFO:tasks.ceph:pgid 7.1 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.262 INFO:tasks.ceph:pgid 8.e last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.262 INFO:tasks.ceph:pgid 6.1 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.262 INFO:tasks.ceph:pgid 7.0 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.262 INFO:tasks.ceph:pgid 8.f last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.262 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.262 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.262 INFO:tasks.ceph:pgid 7.3 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.262 INFO:tasks.ceph:pgid 8.c last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.262 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.262 INFO:tasks.ceph:pgid 7.2 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.263 INFO:tasks.ceph:pgid 8.d last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.263 INFO:tasks.ceph:pgid 8.1f last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.263 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.263 INFO:tasks.ceph:pgid 7.10 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.263 INFO:tasks.ceph:pgid 8.1e last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.263 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.263 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.263 INFO:tasks.ceph:pgid 7.11 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.263 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.263 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.263 INFO:tasks.ceph:pgid 6.13 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.263 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.263 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.264 INFO:tasks.ceph:pgid 7.13 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.264 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.264 INFO:tasks.ceph:pgid 6.1d last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.264 INFO:tasks.ceph:pgid 7.1c last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.264 INFO:tasks.ceph:pgid 8.13 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.264 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.264 INFO:tasks.ceph:pgid 7.1d last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.264 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.264 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.264 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.264 INFO:tasks.ceph:pgid 7.1e last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.264 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.264 INFO:tasks.ceph:pgid 6.1e last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.264 INFO:tasks.ceph:pgid 7.1f last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.264 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.265 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.265 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.265 INFO:tasks.ceph:pgid 7.18 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.265 INFO:tasks.ceph:pgid 8.17 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.265 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.265 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.265 INFO:tasks.ceph:pgid 7.19 last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.265 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.265 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2026-02-24T22:05:19.177566+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=19, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.265 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.265 INFO:tasks.ceph:pgid 7.1a last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.265 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.265 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2026-02-24T22:05:21.180940+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.265 INFO:tasks.ceph:pgid 7.1b last_scrub_stamp 2026-02-24T22:05:23.186988+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=23, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.266 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2026-02-24T22:05:28.641695+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=5, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:34:51.266 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-24T22:35:11.267 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-24T22:35:11.435 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:35:11.442 INFO:teuthology.orchestra.run.trial046.stderr:dumped all 2026-02-24T22:35:11.446 INFO:teuthology.orchestra.run.trial046.stdout:{"pg_ready":true,"pg_map":{"version":1208,"stamp":"2026-02-24T22:35:10.034889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":115015404,"num_objects":5093,"num_object_clones":0,"num_object_copies":10186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5093,"num_whiteouts":0,"num_read":207562,"num_read_kb":1387906,"num_write":161478,"num_write_kb":706745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":1543,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1274391,"num_omap_keys":5703,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13819,"ondisk_log_size":13819,"up":786,"acting":786,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":786,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1559240,"kb_used_data":278856,"kb_used_omap":37260,"kb_used_meta":1242995,"kb_avail":753415480,"statfs":{"total":773094113280,"available":771497451520,"internally_reserved":0,"allocated":285548544,"data_stored":247979658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38154327,"internal_metadata":1272827817},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":663431,"num_omap_keys":2414,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003293"},"pg_stats":[{"pgid":"82.2","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744748+0000","last_change":"2026-02-24T22:23:29.198729+0000","last_active":"2026-02-24T22:31:49.744748+0000","last_peered":"2026-02-24T22:31:49.744748+0000","last_clean":"2026-02-24T22:31:49.744748+0000","last_became_active":"2026-02-24T22:23:29.198476+0000","last_became_peered":"2026-02-24T22:23:29.198476+0000","last_unstale":"2026-02-24T22:31:49.744748+0000","last_undegraded":"2026-02-24T22:31:49.744748+0000","last_fullsized":"2026-02-24T22:31:49.744748+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:12:35.165001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.1","version":"0'0","reported_seq":40,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:07.404528+0000","last_change":"2026-02-24T22:35:07.404528+0000","last_active":"2026-02-24T22:35:07.404528+0000","last_peered":"2026-02-24T22:35:07.404528+0000","last_clean":"2026-02-24T22:35:07.404528+0000","last_became_active":"2026-02-24T22:15:25.830000+0000","last_became_peered":"2026-02-24T22:15:25.830000+0000","last_unstale":"2026-02-24T22:35:07.404528+0000","last_undegraded":"2026-02-24T22:35:07.404528+0000","last_fullsized":"2026-02-24T22:35:07.404528+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:07.404472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:07.404472+0000","last_clean_scrub_stamp":"2026-02-24T22:35:07.404472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:30:10.446904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.0","version":"0'0","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:54.176012+0000","last_change":"2026-02-24T22:34:54.176012+0000","last_active":"2026-02-24T22:34:54.176012+0000","last_peered":"2026-02-24T22:34:54.176012+0000","last_clean":"2026-02-24T22:34:54.176012+0000","last_became_active":"2026-02-24T22:15:22.817446+0000","last_became_peered":"2026-02-24T22:15:22.817446+0000","last_unstale":"2026-02-24T22:34:54.176012+0000","last_undegraded":"2026-02-24T22:34:54.176012+0000","last_fullsized":"2026-02-24T22:34:54.176012+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:54.175956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:54.175956+0000","last_clean_scrub_stamp":"2026-02-24T22:34:54.175956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:40:14.585196+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.1f","version":"312'20","reported_seq":121,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:29.747306+0000","last_change":"2026-02-24T22:13:43.761604+0000","last_active":"2026-02-24T22:32:29.747306+0000","last_peered":"2026-02-24T22:32:29.747306+0000","last_clean":"2026-02-24T22:32:29.747306+0000","last_became_active":"2026-02-24T22:13:43.761120+0000","last_became_peered":"2026-02-24T22:13:43.761120+0000","last_unstale":"2026-02-24T22:32:29.747306+0000","last_undegraded":"2026-02-24T22:32:29.747306+0000","last_fullsized":"2026-02-24T22:32:29.747306+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":168,"num_read_kb":114,"num_write":47,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.3","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742865+0000","last_change":"2026-02-24T22:23:29.213779+0000","last_active":"2026-02-24T22:31:49.742865+0000","last_peered":"2026-02-24T22:31:49.742865+0000","last_clean":"2026-02-24T22:31:49.742865+0000","last_became_active":"2026-02-24T22:23:29.213534+0000","last_became_peered":"2026-02-24T22:23:29.213534+0000","last_unstale":"2026-02-24T22:31:49.742865+0000","last_undegraded":"2026-02-24T22:31:49.742865+0000","last_fullsized":"2026-02-24T22:31:49.742865+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:38:56.711711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.0","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.743144+0000","last_change":"2026-02-24T22:15:25.827702+0000","last_active":"2026-02-24T22:31:34.743144+0000","last_peered":"2026-02-24T22:31:34.743144+0000","last_clean":"2026-02-24T22:31:34.743144+0000","last_became_active":"2026-02-24T22:15:25.827355+0000","last_became_peered":"2026-02-24T22:15:25.827355+0000","last_unstale":"2026-02-24T22:31:34.743144+0000","last_undegraded":"2026-02-24T22:31:34.743144+0000","last_fullsized":"2026-02-24T22:31:34.743144+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:22:12.090995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.1","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:04.744507+0000","last_change":"2026-02-24T22:15:22.818757+0000","last_active":"2026-02-24T22:32:04.744507+0000","last_peered":"2026-02-24T22:32:04.744507+0000","last_clean":"2026-02-24T22:32:04.744507+0000","last_became_active":"2026-02-24T22:15:22.818519+0000","last_became_peered":"2026-02-24T22:15:22.818519+0000","last_unstale":"2026-02-24T22:32:04.744507+0000","last_undegraded":"2026-02-24T22:32:04.744507+0000","last_fullsized":"2026-02-24T22:32:04.744507+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:17:56.378157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.1e","version":"312'9","reported_seq":88,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.745172+0000","last_change":"2026-02-24T22:13:43.757760+0000","last_active":"2026-02-24T22:31:49.745172+0000","last_peered":"2026-02-24T22:31:49.745172+0000","last_clean":"2026-02-24T22:31:49.745172+0000","last_became_active":"2026-02-24T22:13:43.757543+0000","last_became_peered":"2026-02-24T22:13:43.757543+0000","last_unstale":"2026-02-24T22:31:49.745172+0000","last_undegraded":"2026-02-24T22:31:49.745172+0000","last_fullsized":"2026-02-24T22:31:49.745172+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:44:23.087405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":38,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.0","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:04.282796+0000","last_change":"2026-02-24T22:35:04.282796+0000","last_active":"2026-02-24T22:35:04.282796+0000","last_peered":"2026-02-24T22:35:04.282796+0000","last_clean":"2026-02-24T22:35:04.282796+0000","last_became_active":"2026-02-24T22:23:29.212043+0000","last_became_peered":"2026-02-24T22:23:29.212043+0000","last_unstale":"2026-02-24T22:35:04.282796+0000","last_undegraded":"2026-02-24T22:35:04.282796+0000","last_fullsized":"2026-02-24T22:35:04.282796+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:04.282728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:04.282728+0000","last_clean_scrub_stamp":"2026-02-24T22:35:04.282728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:05:28.411607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.3","version":"312'8","reported_seq":53,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.471100+0000","last_change":"2026-02-24T22:15:25.827590+0000","last_active":"2026-02-24T22:34:25.471100+0000","last_peered":"2026-02-24T22:34:25.471100+0000","last_clean":"2026-02-24T22:34:25.471100+0000","last_became_active":"2026-02-24T22:15:25.827216+0000","last_became_peered":"2026-02-24T22:15:25.827216+0000","last_unstale":"2026-02-24T22:34:25.471100+0000","last_undegraded":"2026-02-24T22:34:25.471100+0000","last_fullsized":"2026-02-24T22:34:25.471100+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:08:31.068519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":25602,"num_write":13,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.2","version":"0'0","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:55.187191+0000","last_change":"2026-02-24T22:34:55.187191+0000","last_active":"2026-02-24T22:34:55.187191+0000","last_peered":"2026-02-24T22:34:55.187191+0000","last_clean":"2026-02-24T22:34:55.187191+0000","last_became_active":"2026-02-24T22:15:22.818194+0000","last_became_peered":"2026-02-24T22:15:22.818194+0000","last_unstale":"2026-02-24T22:34:55.187191+0000","last_undegraded":"2026-02-24T22:34:55.187191+0000","last_fullsized":"2026-02-24T22:34:55.187191+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:55.187119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:55.187119+0000","last_clean_scrub_stamp":"2026-02-24T22:34:55.187119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:26:30.922986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.1d","version":"312'22","reported_seq":129,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:00.724505+0000","last_change":"2026-02-24T22:35:00.724505+0000","last_active":"2026-02-24T22:35:00.724505+0000","last_peered":"2026-02-24T22:35:00.724505+0000","last_clean":"2026-02-24T22:35:00.724505+0000","last_became_active":"2026-02-24T22:13:43.757278+0000","last_became_peered":"2026-02-24T22:13:43.757278+0000","last_unstale":"2026-02-24T22:35:00.724505+0000","last_undegraded":"2026-02-24T22:35:00.724505+0000","last_fullsized":"2026-02-24T22:35:00.724505+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'22","last_scrub_stamp":"2026-02-24T22:35:00.724443+0000","last_deep_scrub":"312'22","last_deep_scrub_stamp":"2026-02-24T22:35:00.724443+0000","last_clean_scrub_stamp":"2026-02-24T22:35:00.724443+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:16:45.929416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":103,"num_write":54,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.1","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742841+0000","last_change":"2026-02-24T22:23:29.214295+0000","last_active":"2026-02-24T22:31:49.742841+0000","last_peered":"2026-02-24T22:31:49.742841+0000","last_clean":"2026-02-24T22:31:49.742841+0000","last_became_active":"2026-02-24T22:23:29.214064+0000","last_became_peered":"2026-02-24T22:23:29.214064+0000","last_unstale":"2026-02-24T22:31:49.742841+0000","last_undegraded":"2026-02-24T22:31:49.742841+0000","last_fullsized":"2026-02-24T22:31:49.742841+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:03:04.365969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.2","version":"312'2","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:14.748763+0000","last_change":"2026-02-24T22:15:25.833367+0000","last_active":"2026-02-24T22:34:14.748763+0000","last_peered":"2026-02-24T22:34:14.748763+0000","last_clean":"2026-02-24T22:34:14.748763+0000","last_became_active":"2026-02-24T22:15:25.832809+0000","last_became_peered":"2026-02-24T22:15:25.832809+0000","last_unstale":"2026-02-24T22:34:14.748763+0000","last_undegraded":"2026-02-24T22:34:14.748763+0000","last_fullsized":"2026-02-24T22:34:14.748763+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:28:13.551199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"80.3","version":"312'3","reported_seq":52,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:56.183409+0000","last_change":"2026-02-24T22:34:56.183409+0000","last_active":"2026-02-24T22:34:56.183409+0000","last_peered":"2026-02-24T22:34:56.183409+0000","last_clean":"2026-02-24T22:34:56.183409+0000","last_became_active":"2026-02-24T22:15:22.818089+0000","last_became_peered":"2026-02-24T22:15:22.818089+0000","last_unstale":"2026-02-24T22:34:56.183409+0000","last_undegraded":"2026-02-24T22:34:56.183409+0000","last_fullsized":"2026-02-24T22:34:56.183409+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:34:56.183350+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:34:56.183350+0000","last_clean_scrub_stamp":"2026-02-24T22:34:56.183350+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:06:53.926961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.1c","version":"312'14","reported_seq":101,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.739030+0000","last_change":"2026-02-24T22:13:43.760682+0000","last_active":"2026-02-24T22:31:14.739030+0000","last_peered":"2026-02-24T22:31:14.739030+0000","last_clean":"2026-02-24T22:31:14.739030+0000","last_became_active":"2026-02-24T22:13:43.759958+0000","last_became_peered":"2026-02-24T22:13:43.759958+0000","last_unstale":"2026-02-24T22:31:14.739030+0000","last_undegraded":"2026-02-24T22:31:14.739030+0000","last_fullsized":"2026-02-24T22:31:14.739030+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:24:39.289428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":75,"num_write":36,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.6","version":"309'1","reported_seq":23,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.730973+0000","last_change":"2026-02-24T22:23:29.215576+0000","last_active":"2026-02-24T22:32:09.730973+0000","last_peered":"2026-02-24T22:32:09.730973+0000","last_clean":"2026-02-24T22:32:09.730973+0000","last_became_active":"2026-02-24T22:23:29.215327+0000","last_became_peered":"2026-02-24T22:23:29.215327+0000","last_unstale":"2026-02-24T22:32:09.730973+0000","last_undegraded":"2026-02-24T22:32:09.730973+0000","last_fullsized":"2026-02-24T22:32:09.730973+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:51:01.281339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.5","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:14.745596+0000","last_change":"2026-02-24T22:15:25.832444+0000","last_active":"2026-02-24T22:32:14.745596+0000","last_peered":"2026-02-24T22:32:14.745596+0000","last_clean":"2026-02-24T22:32:14.745596+0000","last_became_active":"2026-02-24T22:15:25.831975+0000","last_became_peered":"2026-02-24T22:15:25.831975+0000","last_unstale":"2026-02-24T22:32:14.745596+0000","last_undegraded":"2026-02-24T22:32:14.745596+0000","last_fullsized":"2026-02-24T22:32:14.745596+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:04:17.334836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"80.4","version":"312'3","reported_seq":44,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:09.761034+0000","last_change":"2026-02-24T22:15:22.812647+0000","last_active":"2026-02-24T22:34:09.761034+0000","last_peered":"2026-02-24T22:34:09.761034+0000","last_clean":"2026-02-24T22:34:09.761034+0000","last_became_active":"2026-02-24T22:15:22.812270+0000","last_became_peered":"2026-02-24T22:15:22.812270+0000","last_unstale":"2026-02-24T22:34:09.761034+0000","last_undegraded":"2026-02-24T22:34:09.761034+0000","last_fullsized":"2026-02-24T22:34:09.761034+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:42:27.774508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.1b","version":"312'18","reported_seq":112,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.741475+0000","last_change":"2026-02-24T22:13:43.755424+0000","last_active":"2026-02-24T22:31:14.741475+0000","last_peered":"2026-02-24T22:31:14.741475+0000","last_clean":"2026-02-24T22:31:14.741475+0000","last_became_active":"2026-02-24T22:13:43.755170+0000","last_became_peered":"2026-02-24T22:13:43.755170+0000","last_unstale":"2026-02-24T22:31:14.741475+0000","last_undegraded":"2026-02-24T22:31:14.741475+0000","last_fullsized":"2026-02-24T22:31:14.741475+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:29:17.090743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":101,"num_write":42,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.7","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.735768+0000","last_change":"2026-02-24T22:23:29.214459+0000","last_active":"2026-02-24T22:32:09.735768+0000","last_peered":"2026-02-24T22:32:09.735768+0000","last_clean":"2026-02-24T22:32:09.735768+0000","last_became_active":"2026-02-24T22:23:29.214105+0000","last_became_peered":"2026-02-24T22:23:29.214105+0000","last_unstale":"2026-02-24T22:32:09.735768+0000","last_undegraded":"2026-02-24T22:32:09.735768+0000","last_fullsized":"2026-02-24T22:32:09.735768+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:00:10.302637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.4","version":"312'3","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:14.760553+0000","last_change":"2026-02-24T22:15:25.827716+0000","last_active":"2026-02-24T22:34:14.760553+0000","last_peered":"2026-02-24T22:34:14.760553+0000","last_clean":"2026-02-24T22:34:14.760553+0000","last_became_active":"2026-02-24T22:15:25.826663+0000","last_became_peered":"2026-02-24T22:15:25.826663+0000","last_unstale":"2026-02-24T22:34:14.760553+0000","last_undegraded":"2026-02-24T22:34:14.760553+0000","last_fullsized":"2026-02-24T22:34:14.760553+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:08:34.544962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.5","version":"0'0","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:06.414312+0000","last_change":"2026-02-24T22:35:06.414312+0000","last_active":"2026-02-24T22:35:06.414312+0000","last_peered":"2026-02-24T22:35:06.414312+0000","last_clean":"2026-02-24T22:35:06.414312+0000","last_became_active":"2026-02-24T22:15:22.816697+0000","last_became_peered":"2026-02-24T22:15:22.816697+0000","last_unstale":"2026-02-24T22:35:06.414312+0000","last_undegraded":"2026-02-24T22:35:06.414312+0000","last_fullsized":"2026-02-24T22:35:06.414312+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:06.414264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:06.414264+0000","last_clean_scrub_stamp":"2026-02-24T22:35:06.414264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:28:07.680045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"79.1a","version":"312'16","reported_seq":110,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.717216+0000","last_change":"2026-02-24T22:34:58.717216+0000","last_active":"2026-02-24T22:34:58.717216+0000","last_peered":"2026-02-24T22:34:58.717216+0000","last_clean":"2026-02-24T22:34:58.717216+0000","last_became_active":"2026-02-24T22:13:43.757353+0000","last_became_peered":"2026-02-24T22:13:43.757353+0000","last_unstale":"2026-02-24T22:34:58.717216+0000","last_undegraded":"2026-02-24T22:34:58.717216+0000","last_fullsized":"2026-02-24T22:34:58.717216+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'16","last_scrub_stamp":"2026-02-24T22:34:58.717170+0000","last_deep_scrub":"312'16","last_deep_scrub_stamp":"2026-02-24T22:34:58.717170+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.717170+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:56:29.218407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":65,"num_write":38,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.4","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742881+0000","last_change":"2026-02-24T22:23:29.215206+0000","last_active":"2026-02-24T22:31:49.742881+0000","last_peered":"2026-02-24T22:31:49.742881+0000","last_clean":"2026-02-24T22:31:49.742881+0000","last_became_active":"2026-02-24T22:23:29.215014+0000","last_became_peered":"2026-02-24T22:23:29.215014+0000","last_unstale":"2026-02-24T22:31:49.742881+0000","last_undegraded":"2026-02-24T22:31:49.742881+0000","last_fullsized":"2026-02-24T22:31:49.742881+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:43:42.586588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.7","version":"312'3","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:03.237185+0000","last_change":"2026-02-24T22:35:03.237185+0000","last_active":"2026-02-24T22:35:03.237185+0000","last_peered":"2026-02-24T22:35:03.237185+0000","last_clean":"2026-02-24T22:35:03.237185+0000","last_became_active":"2026-02-24T22:15:25.831806+0000","last_became_peered":"2026-02-24T22:15:25.831806+0000","last_unstale":"2026-02-24T22:35:03.237185+0000","last_undegraded":"2026-02-24T22:35:03.237185+0000","last_fullsized":"2026-02-24T22:35:03.237185+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:03.237068+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:03.237068+0000","last_clean_scrub_stamp":"2026-02-24T22:35:03.237068+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:04:02.704108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"80.6","version":"0'0","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:08.654082+0000","last_change":"2026-02-24T22:35:08.654082+0000","last_active":"2026-02-24T22:35:08.654082+0000","last_peered":"2026-02-24T22:35:08.654082+0000","last_clean":"2026-02-24T22:35:08.654082+0000","last_became_active":"2026-02-24T22:15:22.811413+0000","last_became_peered":"2026-02-24T22:15:22.811413+0000","last_unstale":"2026-02-24T22:35:08.654082+0000","last_undegraded":"2026-02-24T22:35:08.654082+0000","last_fullsized":"2026-02-24T22:35:08.654082+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:08.654011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:08.654011+0000","last_clean_scrub_stamp":"2026-02-24T22:35:08.654011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:15:31.642731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"79.19","version":"312'30","reported_seq":148,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:09.330426+0000","last_change":"2026-02-24T22:35:09.330426+0000","last_active":"2026-02-24T22:35:09.330426+0000","last_peered":"2026-02-24T22:35:09.330426+0000","last_clean":"2026-02-24T22:35:09.330426+0000","last_became_active":"2026-02-24T22:13:43.756114+0000","last_became_peered":"2026-02-24T22:13:43.756114+0000","last_unstale":"2026-02-24T22:35:09.330426+0000","last_undegraded":"2026-02-24T22:35:09.330426+0000","last_fullsized":"2026-02-24T22:35:09.330426+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'30","last_scrub_stamp":"2026-02-24T22:35:09.330359+0000","last_deep_scrub":"312'30","last_deep_scrub_stamp":"2026-02-24T22:35:09.330359+0000","last_clean_scrub_stamp":"2026-02-24T22:35:09.330359+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:23:17.878380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":216,"num_read_kb":143,"num_write":68,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.5","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742892+0000","last_change":"2026-02-24T22:23:29.213537+0000","last_active":"2026-02-24T22:31:49.742892+0000","last_peered":"2026-02-24T22:31:49.742892+0000","last_clean":"2026-02-24T22:31:49.742892+0000","last_became_active":"2026-02-24T22:23:29.213256+0000","last_became_peered":"2026-02-24T22:23:29.213256+0000","last_unstale":"2026-02-24T22:31:49.742892+0000","last_undegraded":"2026-02-24T22:31:49.742892+0000","last_fullsized":"2026-02-24T22:31:49.742892+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:48:26.724830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.6","version":"312'3","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:08.360735+0000","last_change":"2026-02-24T22:35:08.360735+0000","last_active":"2026-02-24T22:35:08.360735+0000","last_peered":"2026-02-24T22:35:08.360735+0000","last_clean":"2026-02-24T22:35:08.360735+0000","last_became_active":"2026-02-24T22:15:25.833680+0000","last_became_peered":"2026-02-24T22:15:25.833680+0000","last_unstale":"2026-02-24T22:35:08.360735+0000","last_undegraded":"2026-02-24T22:35:08.360735+0000","last_fullsized":"2026-02-24T22:35:08.360735+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:08.360662+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:08.360662+0000","last_clean_scrub_stamp":"2026-02-24T22:35:08.360662+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:43:55.489569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.7","version":"0'0","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:09.618057+0000","last_change":"2026-02-24T22:35:09.618057+0000","last_active":"2026-02-24T22:35:09.618057+0000","last_peered":"2026-02-24T22:35:09.618057+0000","last_clean":"2026-02-24T22:35:09.618057+0000","last_became_active":"2026-02-24T22:15:22.811618+0000","last_became_peered":"2026-02-24T22:15:22.811618+0000","last_unstale":"2026-02-24T22:35:09.618057+0000","last_undegraded":"2026-02-24T22:35:09.618057+0000","last_fullsized":"2026-02-24T22:35:09.618057+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:09.617989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:09.617989+0000","last_clean_scrub_stamp":"2026-02-24T22:35:09.617989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:27:23.181702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"79.18","version":"312'12","reported_seq":95,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.737670+0000","last_change":"2026-02-24T22:13:43.755501+0000","last_active":"2026-02-24T22:30:39.737670+0000","last_peered":"2026-02-24T22:30:39.737670+0000","last_clean":"2026-02-24T22:30:39.737670+0000","last_became_active":"2026-02-24T22:13:43.754981+0000","last_became_peered":"2026-02-24T22:13:43.754981+0000","last_unstale":"2026-02-24T22:30:39.737670+0000","last_undegraded":"2026-02-24T22:30:39.737670+0000","last_fullsized":"2026-02-24T22:30:39.737670+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:41:38.765379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":86,"num_read_kb":57,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.a","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.745052+0000","last_change":"2026-02-24T22:23:29.212877+0000","last_active":"2026-02-24T22:32:09.745052+0000","last_peered":"2026-02-24T22:32:09.745052+0000","last_clean":"2026-02-24T22:32:09.745052+0000","last_became_active":"2026-02-24T22:23:29.212213+0000","last_became_peered":"2026-02-24T22:23:29.212213+0000","last_unstale":"2026-02-24T22:32:09.745052+0000","last_undegraded":"2026-02-24T22:32:09.745052+0000","last_fullsized":"2026-02-24T22:32:09.745052+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:29:28.605583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.9","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:49.751441+0000","last_change":"2026-02-24T22:15:25.826412+0000","last_active":"2026-02-24T22:32:49.751441+0000","last_peered":"2026-02-24T22:32:49.751441+0000","last_clean":"2026-02-24T22:32:49.751441+0000","last_became_active":"2026-02-24T22:15:25.826076+0000","last_became_peered":"2026-02-24T22:15:25.826076+0000","last_unstale":"2026-02-24T22:32:49.751441+0000","last_undegraded":"2026-02-24T22:32:49.751441+0000","last_fullsized":"2026-02-24T22:32:49.751441+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.8","version":"312'6","reported_seq":57,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:57.217246+0000","last_change":"2026-02-24T22:34:57.217246+0000","last_active":"2026-02-24T22:34:57.217246+0000","last_peered":"2026-02-24T22:34:57.217246+0000","last_clean":"2026-02-24T22:34:57.217246+0000","last_became_active":"2026-02-24T22:15:22.815960+0000","last_became_peered":"2026-02-24T22:15:22.815960+0000","last_unstale":"2026-02-24T22:34:57.217246+0000","last_undegraded":"2026-02-24T22:34:57.217246+0000","last_fullsized":"2026-02-24T22:34:57.217246+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'6","last_scrub_stamp":"2026-02-24T22:34:57.217191+0000","last_deep_scrub":"312'6","last_deep_scrub_stamp":"2026-02-24T22:34:57.217191+0000","last_clean_scrub_stamp":"2026-02-24T22:34:57.217191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:07:48.538456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6144,"num_write":11,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.17","version":"312'34","reported_seq":177,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:03.366056+0000","last_change":"2026-02-24T22:35:03.366056+0000","last_active":"2026-02-24T22:35:03.366056+0000","last_peered":"2026-02-24T22:35:03.366056+0000","last_clean":"2026-02-24T22:35:03.366056+0000","last_became_active":"2026-02-24T22:14:00.748438+0000","last_became_peered":"2026-02-24T22:14:00.748438+0000","last_unstale":"2026-02-24T22:35:03.366056+0000","last_undegraded":"2026-02-24T22:35:03.366056+0000","last_fullsized":"2026-02-24T22:35:03.366056+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":293,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'34","last_scrub_stamp":"2026-02-24T22:35:03.365970+0000","last_deep_scrub":"312'34","last_deep_scrub_stamp":"2026-02-24T22:35:03.365970+0000","last_clean_scrub_stamp":"2026-02-24T22:35:03.365970+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:27:41.779757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":298,"num_read_kb":211,"num_write":78,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.b","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:05.714092+0000","last_change":"2026-02-24T22:35:05.714092+0000","last_active":"2026-02-24T22:35:05.714092+0000","last_peered":"2026-02-24T22:35:05.714092+0000","last_clean":"2026-02-24T22:35:05.714092+0000","last_became_active":"2026-02-24T22:23:29.196790+0000","last_became_peered":"2026-02-24T22:23:29.196790+0000","last_unstale":"2026-02-24T22:35:05.714092+0000","last_undegraded":"2026-02-24T22:35:05.714092+0000","last_fullsized":"2026-02-24T22:35:05.714092+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:05.714031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:05.714031+0000","last_clean_scrub_stamp":"2026-02-24T22:35:05.714031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:44:22.188544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.8","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:49.749107+0000","last_change":"2026-02-24T22:15:25.825567+0000","last_active":"2026-02-24T22:32:49.749107+0000","last_peered":"2026-02-24T22:32:49.749107+0000","last_clean":"2026-02-24T22:32:49.749107+0000","last_became_active":"2026-02-24T22:15:25.825315+0000","last_became_peered":"2026-02-24T22:15:25.825315+0000","last_unstale":"2026-02-24T22:32:49.749107+0000","last_undegraded":"2026-02-24T22:32:49.749107+0000","last_fullsized":"2026-02-24T22:32:49.749107+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.9","version":"0'0","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:59.757040+0000","last_change":"2026-02-24T22:34:59.757040+0000","last_active":"2026-02-24T22:34:59.757040+0000","last_peered":"2026-02-24T22:34:59.757040+0000","last_clean":"2026-02-24T22:34:59.757040+0000","last_became_active":"2026-02-24T22:15:22.811141+0000","last_became_peered":"2026-02-24T22:15:22.811141+0000","last_unstale":"2026-02-24T22:34:59.757040+0000","last_undegraded":"2026-02-24T22:34:59.757040+0000","last_fullsized":"2026-02-24T22:34:59.757040+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:59.756969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:59.756969+0000","last_clean_scrub_stamp":"2026-02-24T22:34:59.756969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:19:48.331387+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"79.16","version":"0'0","reported_seq":71,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:57.691689+0000","last_change":"2026-02-24T22:34:57.691689+0000","last_active":"2026-02-24T22:34:57.691689+0000","last_peered":"2026-02-24T22:34:57.691689+0000","last_clean":"2026-02-24T22:34:57.691689+0000","last_became_active":"2026-02-24T22:13:43.755885+0000","last_became_peered":"2026-02-24T22:13:43.755885+0000","last_unstale":"2026-02-24T22:34:57.691689+0000","last_undegraded":"2026-02-24T22:34:57.691689+0000","last_fullsized":"2026-02-24T22:34:57.691689+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:57.691616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:57.691616+0000","last_clean_scrub_stamp":"2026-02-24T22:34:57.691616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:54:10.527824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.8","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.744882+0000","last_change":"2026-02-24T22:23:29.199299+0000","last_active":"2026-02-24T22:32:09.744882+0000","last_peered":"2026-02-24T22:32:09.744882+0000","last_clean":"2026-02-24T22:32:09.744882+0000","last_became_active":"2026-02-24T22:23:29.198882+0000","last_became_peered":"2026-02-24T22:23:29.198882+0000","last_unstale":"2026-02-24T22:32:09.744882+0000","last_undegraded":"2026-02-24T22:32:09.744882+0000","last_fullsized":"2026-02-24T22:32:09.744882+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:59:03.245529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"81.b","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.741464+0000","last_change":"2026-02-24T22:15:25.827664+0000","last_active":"2026-02-24T22:31:14.741464+0000","last_peered":"2026-02-24T22:31:14.741464+0000","last_clean":"2026-02-24T22:31:14.741464+0000","last_became_active":"2026-02-24T22:15:25.827265+0000","last_became_peered":"2026-02-24T22:15:25.827265+0000","last_unstale":"2026-02-24T22:31:14.741464+0000","last_undegraded":"2026-02-24T22:31:14.741464+0000","last_fullsized":"2026-02-24T22:31:14.741464+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:17:29.198425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.a","version":"0'0","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:49.749438+0000","last_change":"2026-02-24T22:15:22.817325+0000","last_active":"2026-02-24T22:32:49.749438+0000","last_peered":"2026-02-24T22:32:49.749438+0000","last_clean":"2026-02-24T22:32:49.749438+0000","last_became_active":"2026-02-24T22:15:22.816702+0000","last_became_peered":"2026-02-24T22:15:22.816702+0000","last_unstale":"2026-02-24T22:32:49.749438+0000","last_undegraded":"2026-02-24T22:32:49.749438+0000","last_fullsized":"2026-02-24T22:32:49.749438+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.15","version":"307'6","reported_seq":82,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:39.735250+0000","last_change":"2026-02-24T22:13:43.761758+0000","last_active":"2026-02-24T22:30:39.735250+0000","last_peered":"2026-02-24T22:30:39.735250+0000","last_clean":"2026-02-24T22:30:39.735250+0000","last_became_active":"2026-02-24T22:13:43.760892+0000","last_became_peered":"2026-02-24T22:13:43.760892+0000","last_unstale":"2026-02-24T22:30:39.735250+0000","last_undegraded":"2026-02-24T22:30:39.735250+0000","last_fullsized":"2026-02-24T22:30:39.735250+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:07:44.028035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":37,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.9","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:02.763296+0000","last_change":"2026-02-24T22:35:02.763296+0000","last_active":"2026-02-24T22:35:02.763296+0000","last_peered":"2026-02-24T22:35:02.763296+0000","last_clean":"2026-02-24T22:35:02.763296+0000","last_became_active":"2026-02-24T22:23:29.197357+0000","last_became_peered":"2026-02-24T22:23:29.197357+0000","last_unstale":"2026-02-24T22:35:02.763296+0000","last_undegraded":"2026-02-24T22:35:02.763296+0000","last_fullsized":"2026-02-24T22:35:02.763296+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:02.763243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:02.763243+0000","last_clean_scrub_stamp":"2026-02-24T22:35:02.763243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:52:20.692060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.a","version":"312'3","reported_seq":38,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.630503+0000","last_change":"2026-02-24T22:15:25.827469+0000","last_active":"2026-02-24T22:34:25.630503+0000","last_peered":"2026-02-24T22:34:25.630503+0000","last_clean":"2026-02-24T22:34:25.630503+0000","last_became_active":"2026-02-24T22:15:25.827055+0000","last_became_peered":"2026-02-24T22:15:25.827055+0000","last_unstale":"2026-02-24T22:34:25.630503+0000","last_undegraded":"2026-02-24T22:34:25.630503+0000","last_fullsized":"2026-02-24T22:34:25.630503+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.b","version":"312'3","reported_seq":43,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:33:24.753235+0000","last_change":"2026-02-24T22:15:22.817894+0000","last_active":"2026-02-24T22:33:24.753235+0000","last_peered":"2026-02-24T22:33:24.753235+0000","last_clean":"2026-02-24T22:33:24.753235+0000","last_became_active":"2026-02-24T22:15:22.817498+0000","last_became_peered":"2026-02-24T22:15:22.817498+0000","last_unstale":"2026-02-24T22:33:24.753235+0000","last_undegraded":"2026-02-24T22:33:24.753235+0000","last_fullsized":"2026-02-24T22:33:24.753235+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:08:45.124818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.14","version":"312'10","reported_seq":98,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:02.317457+0000","last_change":"2026-02-24T22:35:02.317457+0000","last_active":"2026-02-24T22:35:02.317457+0000","last_peered":"2026-02-24T22:35:02.317457+0000","last_clean":"2026-02-24T22:35:02.317457+0000","last_became_active":"2026-02-24T22:13:43.762341+0000","last_became_peered":"2026-02-24T22:13:43.762341+0000","last_unstale":"2026-02-24T22:35:02.317457+0000","last_undegraded":"2026-02-24T22:35:02.317457+0000","last_fullsized":"2026-02-24T22:35:02.317457+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'10","last_scrub_stamp":"2026-02-24T22:35:02.317403+0000","last_deep_scrub":"312'10","last_deep_scrub_stamp":"2026-02-24T22:35:02.317403+0000","last_clean_scrub_stamp":"2026-02-24T22:35:02.317403+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:39:36.833592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":46,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.e","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:29.737880+0000","last_change":"2026-02-24T22:23:29.215664+0000","last_active":"2026-02-24T22:32:29.737880+0000","last_peered":"2026-02-24T22:32:29.737880+0000","last_clean":"2026-02-24T22:32:29.737880+0000","last_became_active":"2026-02-24T22:23:29.215425+0000","last_became_peered":"2026-02-24T22:23:29.215425+0000","last_unstale":"2026-02-24T22:32:29.737880+0000","last_undegraded":"2026-02-24T22:32:29.737880+0000","last_fullsized":"2026-02-24T22:32:29.737880+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.d","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.741498+0000","last_change":"2026-02-24T22:15:25.829241+0000","last_active":"2026-02-24T22:31:14.741498+0000","last_peered":"2026-02-24T22:31:14.741498+0000","last_clean":"2026-02-24T22:31:14.741498+0000","last_became_active":"2026-02-24T22:15:25.829109+0000","last_became_peered":"2026-02-24T22:15:25.829109+0000","last_unstale":"2026-02-24T22:31:14.741498+0000","last_undegraded":"2026-02-24T22:31:14.741498+0000","last_fullsized":"2026-02-24T22:31:14.741498+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:38:08.330741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.c","version":"0'0","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.197887+0000","last_change":"2026-02-24T22:34:58.197887+0000","last_active":"2026-02-24T22:34:58.197887+0000","last_peered":"2026-02-24T22:34:58.197887+0000","last_clean":"2026-02-24T22:34:58.197887+0000","last_became_active":"2026-02-24T22:15:22.815364+0000","last_became_peered":"2026-02-24T22:15:22.815364+0000","last_unstale":"2026-02-24T22:34:58.197887+0000","last_undegraded":"2026-02-24T22:34:58.197887+0000","last_fullsized":"2026-02-24T22:34:58.197887+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:58.197815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:58.197815+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.197815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:35:14.681874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.13","version":"312'11","reported_seq":104,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:51.152600+0000","last_change":"2026-02-24T22:34:51.152600+0000","last_active":"2026-02-24T22:34:51.152600+0000","last_peered":"2026-02-24T22:34:51.152600+0000","last_clean":"2026-02-24T22:34:51.152600+0000","last_became_active":"2026-02-24T22:13:43.760714+0000","last_became_peered":"2026-02-24T22:13:43.760714+0000","last_unstale":"2026-02-24T22:34:51.152600+0000","last_undegraded":"2026-02-24T22:34:51.152600+0000","last_fullsized":"2026-02-24T22:34:51.152600+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'11","last_scrub_stamp":"2026-02-24T22:34:51.152549+0000","last_deep_scrub":"312'11","last_deep_scrub_stamp":"2026-02-24T22:34:51.152549+0000","last_clean_scrub_stamp":"2026-02-24T22:34:51.152549+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:24:03.221560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":61,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"82.f","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:06.681089+0000","last_change":"2026-02-24T22:35:06.681089+0000","last_active":"2026-02-24T22:35:06.681089+0000","last_peered":"2026-02-24T22:35:06.681089+0000","last_clean":"2026-02-24T22:35:06.681089+0000","last_became_active":"2026-02-24T22:23:29.199538+0000","last_became_peered":"2026-02-24T22:23:29.199538+0000","last_unstale":"2026-02-24T22:35:06.681089+0000","last_undegraded":"2026-02-24T22:35:06.681089+0000","last_fullsized":"2026-02-24T22:35:06.681089+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:06.680985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:06.680985+0000","last_clean_scrub_stamp":"2026-02-24T22:35:06.680985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:18:47.506871+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.c","version":"312'11","reported_seq":58,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.692318+0000","last_change":"2026-02-24T22:15:25.827267+0000","last_active":"2026-02-24T22:34:25.692318+0000","last_peered":"2026-02-24T22:34:25.692318+0000","last_clean":"2026-02-24T22:34:25.692318+0000","last_became_active":"2026-02-24T22:15:25.826430+0000","last_became_peered":"2026-02-24T22:15:25.826430+0000","last_unstale":"2026-02-24T22:34:25.692318+0000","last_undegraded":"2026-02-24T22:34:25.692318+0000","last_fullsized":"2026-02-24T22:34:25.692318+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:49:19.806352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":31746,"num_write":16,"num_write_kb":16384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.d","version":"312'5","reported_seq":52,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.807220+0000","last_change":"2026-02-24T22:15:22.813197+0000","last_active":"2026-02-24T22:34:25.807220+0000","last_peered":"2026-02-24T22:34:25.807220+0000","last_clean":"2026-02-24T22:34:25.807220+0000","last_became_active":"2026-02-24T22:15:22.813062+0000","last_became_peered":"2026-02-24T22:15:22.813062+0000","last_unstale":"2026-02-24T22:34:25.807220+0000","last_undegraded":"2026-02-24T22:34:25.807220+0000","last_fullsized":"2026-02-24T22:34:25.807220+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:08:36.468537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":10,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.12","version":"312'20","reported_seq":126,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:53.127878+0000","last_change":"2026-02-24T22:34:53.127878+0000","last_active":"2026-02-24T22:34:53.127878+0000","last_peered":"2026-02-24T22:34:53.127878+0000","last_clean":"2026-02-24T22:34:53.127878+0000","last_became_active":"2026-02-24T22:13:43.760196+0000","last_became_peered":"2026-02-24T22:13:43.760196+0000","last_unstale":"2026-02-24T22:34:53.127878+0000","last_undegraded":"2026-02-24T22:34:53.127878+0000","last_fullsized":"2026-02-24T22:34:53.127878+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'20","last_scrub_stamp":"2026-02-24T22:34:53.127808+0000","last_deep_scrub":"312'20","last_deep_scrub_stamp":"2026-02-24T22:34:53.127808+0000","last_clean_scrub_stamp":"2026-02-24T22:34:53.127808+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:40:20.279936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":106,"num_write":52,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"82.c","version":"310'1","reported_seq":23,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:29.750029+0000","last_change":"2026-02-24T22:23:29.201743+0000","last_active":"2026-02-24T22:32:29.750029+0000","last_peered":"2026-02-24T22:32:29.750029+0000","last_clean":"2026-02-24T22:32:29.750029+0000","last_became_active":"2026-02-24T22:23:29.201545+0000","last_became_peered":"2026-02-24T22:23:29.201545+0000","last_unstale":"2026-02-24T22:32:29.750029+0000","last_undegraded":"2026-02-24T22:32:29.750029+0000","last_fullsized":"2026-02-24T22:32:29.750029+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"81.f","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:19.741445+0000","last_change":"2026-02-24T22:15:25.825325+0000","last_active":"2026-02-24T22:31:19.741445+0000","last_peered":"2026-02-24T22:31:19.741445+0000","last_clean":"2026-02-24T22:31:19.741445+0000","last_became_active":"2026-02-24T22:15:25.824862+0000","last_became_peered":"2026-02-24T22:15:25.824862+0000","last_unstale":"2026-02-24T22:31:19.741445+0000","last_undegraded":"2026-02-24T22:31:19.741445+0000","last_fullsized":"2026-02-24T22:31:19.741445+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:16:51.320757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.e","version":"312'5","reported_seq":62,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:00.220063+0000","last_change":"2026-02-24T22:35:00.220063+0000","last_active":"2026-02-24T22:35:00.220063+0000","last_peered":"2026-02-24T22:35:00.220063+0000","last_clean":"2026-02-24T22:35:00.220063+0000","last_became_active":"2026-02-24T22:15:22.817295+0000","last_became_peered":"2026-02-24T22:15:22.817295+0000","last_unstale":"2026-02-24T22:35:00.220063+0000","last_undegraded":"2026-02-24T22:35:00.220063+0000","last_fullsized":"2026-02-24T22:35:00.220063+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'5","last_scrub_stamp":"2026-02-24T22:35:00.220005+0000","last_deep_scrub":"312'5","last_deep_scrub_stamp":"2026-02-24T22:35:00.220005+0000","last_clean_scrub_stamp":"2026-02-24T22:35:00.220005+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:12:59.300665+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":25602,"num_write":5,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.11","version":"312'11","reported_seq":93,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.746653+0000","last_change":"2026-02-24T22:13:43.760497+0000","last_active":"2026-02-24T22:32:24.746653+0000","last_peered":"2026-02-24T22:32:24.746653+0000","last_clean":"2026-02-24T22:32:24.746653+0000","last_became_active":"2026-02-24T22:13:43.759864+0000","last_became_peered":"2026-02-24T22:13:43.759864+0000","last_unstale":"2026-02-24T22:32:24.746653+0000","last_undegraded":"2026-02-24T22:32:24.746653+0000","last_fullsized":"2026-02-24T22:32:24.746653+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":53,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.d","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:29.737869+0000","last_change":"2026-02-24T22:23:29.214699+0000","last_active":"2026-02-24T22:32:29.737869+0000","last_peered":"2026-02-24T22:32:29.737869+0000","last_clean":"2026-02-24T22:32:29.737869+0000","last_became_active":"2026-02-24T22:23:29.214432+0000","last_became_peered":"2026-02-24T22:23:29.214432+0000","last_unstale":"2026-02-24T22:32:29.737869+0000","last_undegraded":"2026-02-24T22:32:29.737869+0000","last_fullsized":"2026-02-24T22:32:29.737869+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.e","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:14.738719+0000","last_change":"2026-02-24T22:15:25.827425+0000","last_active":"2026-02-24T22:31:14.738719+0000","last_peered":"2026-02-24T22:31:14.738719+0000","last_clean":"2026-02-24T22:31:14.738719+0000","last_became_active":"2026-02-24T22:15:25.827110+0000","last_became_peered":"2026-02-24T22:15:25.827110+0000","last_unstale":"2026-02-24T22:31:14.738719+0000","last_undegraded":"2026-02-24T22:31:14.738719+0000","last_fullsized":"2026-02-24T22:31:14.738719+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:25:06.342633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.f","version":"312'8","reported_seq":58,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.821904+0000","last_change":"2026-02-24T22:15:22.816168+0000","last_active":"2026-02-24T22:34:25.821904+0000","last_peered":"2026-02-24T22:34:25.821904+0000","last_clean":"2026-02-24T22:34:25.821904+0000","last_became_active":"2026-02-24T22:15:22.815819+0000","last_became_peered":"2026-02-24T22:15:22.815819+0000","last_unstale":"2026-02-24T22:34:25.821904+0000","last_undegraded":"2026-02-24T22:34:25.821904+0000","last_fullsized":"2026-02-24T22:34:25.821904+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:39:39.503227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":26626,"num_write":13,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.10","version":"312'31","reported_seq":153,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.746632+0000","last_change":"2026-02-24T22:13:43.760545+0000","last_active":"2026-02-24T22:32:24.746632+0000","last_peered":"2026-02-24T22:32:24.746632+0000","last_clean":"2026-02-24T22:32:24.746632+0000","last_became_active":"2026-02-24T22:13:43.759855+0000","last_became_peered":"2026-02-24T22:13:43.759855+0000","last_unstale":"2026-02-24T22:32:24.746632+0000","last_undegraded":"2026-02-24T22:32:24.746632+0000","last_fullsized":"2026-02-24T22:32:24.746632+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":272,"num_read_kb":204,"num_write":72,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.12","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:49.739915+0000","last_change":"2026-02-24T22:23:29.212745+0000","last_active":"2026-02-24T22:32:49.739915+0000","last_peered":"2026-02-24T22:32:49.739915+0000","last_clean":"2026-02-24T22:32:49.739915+0000","last_became_active":"2026-02-24T22:23:29.212534+0000","last_became_peered":"2026-02-24T22:23:29.212534+0000","last_unstale":"2026-02-24T22:32:49.739915+0000","last_undegraded":"2026-02-24T22:32:49.739915+0000","last_fullsized":"2026-02-24T22:32:49.739915+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.11","version":"312'5","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.471093+0000","last_change":"2026-02-24T22:15:25.827242+0000","last_active":"2026-02-24T22:34:25.471093+0000","last_peered":"2026-02-24T22:34:25.471093+0000","last_clean":"2026-02-24T22:34:25.471093+0000","last_became_active":"2026-02-24T22:15:25.826517+0000","last_became_peered":"2026-02-24T22:15:25.826517+0000","last_unstale":"2026-02-24T22:34:25.471093+0000","last_undegraded":"2026-02-24T22:34:25.471093+0000","last_fullsized":"2026-02-24T22:34:25.471093+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:24:40.732721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":10,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.10","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.741867+0000","last_change":"2026-02-24T22:15:22.810414+0000","last_active":"2026-02-24T22:31:44.741867+0000","last_peered":"2026-02-24T22:31:44.741867+0000","last_clean":"2026-02-24T22:31:44.741867+0000","last_became_active":"2026-02-24T22:15:22.810000+0000","last_became_peered":"2026-02-24T22:15:22.810000+0000","last_unstale":"2026-02-24T22:31:44.741867+0000","last_undegraded":"2026-02-24T22:31:44.741867+0000","last_fullsized":"2026-02-24T22:31:44.741867+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:21:42.629513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"79.f","version":"312'11","reported_seq":95,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742413+0000","last_change":"2026-02-24T22:13:43.755156+0000","last_active":"2026-02-24T22:31:49.742413+0000","last_peered":"2026-02-24T22:31:49.742413+0000","last_clean":"2026-02-24T22:31:49.742413+0000","last_became_active":"2026-02-24T22:13:43.754602+0000","last_became_peered":"2026-02-24T22:13:43.754602+0000","last_unstale":"2026-02-24T22:31:49.742413+0000","last_undegraded":"2026-02-24T22:31:49.742413+0000","last_fullsized":"2026-02-24T22:31:49.742413+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:13:53.006921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":67,"num_write":27,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.13","version":"310'1","reported_seq":23,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:49.751465+0000","last_change":"2026-02-24T22:23:29.198937+0000","last_active":"2026-02-24T22:32:49.751465+0000","last_peered":"2026-02-24T22:32:49.751465+0000","last_clean":"2026-02-24T22:32:49.751465+0000","last_became_active":"2026-02-24T22:23:29.198627+0000","last_became_peered":"2026-02-24T22:23:29.198627+0000","last_unstale":"2026-02-24T22:32:49.751465+0000","last_undegraded":"2026-02-24T22:32:49.751465+0000","last_fullsized":"2026-02-24T22:32:49.751465+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.10","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744722+0000","last_change":"2026-02-24T22:15:25.827252+0000","last_active":"2026-02-24T22:31:49.744722+0000","last_peered":"2026-02-24T22:31:49.744722+0000","last_clean":"2026-02-24T22:31:49.744722+0000","last_became_active":"2026-02-24T22:15:25.826606+0000","last_became_peered":"2026-02-24T22:15:25.826606+0000","last_unstale":"2026-02-24T22:31:49.744722+0000","last_undegraded":"2026-02-24T22:31:49.744722+0000","last_fullsized":"2026-02-24T22:31:49.744722+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:30:33.376941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.11","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.741885+0000","last_change":"2026-02-24T22:15:22.810459+0000","last_active":"2026-02-24T22:31:44.741885+0000","last_peered":"2026-02-24T22:31:44.741885+0000","last_clean":"2026-02-24T22:31:44.741885+0000","last_became_active":"2026-02-24T22:15:22.809791+0000","last_became_peered":"2026-02-24T22:15:22.809791+0000","last_unstale":"2026-02-24T22:31:44.741885+0000","last_undegraded":"2026-02-24T22:31:44.741885+0000","last_fullsized":"2026-02-24T22:31:44.741885+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:37:39.655379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"79.e","version":"312'23","reported_seq":134,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:08.185039+0000","last_change":"2026-02-24T22:35:08.185039+0000","last_active":"2026-02-24T22:35:08.185039+0000","last_peered":"2026-02-24T22:35:08.185039+0000","last_clean":"2026-02-24T22:35:08.185039+0000","last_became_active":"2026-02-24T22:13:43.755868+0000","last_became_peered":"2026-02-24T22:13:43.755868+0000","last_unstale":"2026-02-24T22:35:08.185039+0000","last_undegraded":"2026-02-24T22:35:08.185039+0000","last_fullsized":"2026-02-24T22:35:08.185039+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'23","last_scrub_stamp":"2026-02-24T22:35:08.184981+0000","last_deep_scrub":"312'23","last_deep_scrub_stamp":"2026-02-24T22:35:08.184981+0000","last_clean_scrub_stamp":"2026-02-24T22:35:08.184981+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:38:46.065895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":169,"num_read_kb":112,"num_write":56,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.10","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:49.749407+0000","last_change":"2026-02-24T22:23:29.212895+0000","last_active":"2026-02-24T22:32:49.749407+0000","last_peered":"2026-02-24T22:32:49.749407+0000","last_clean":"2026-02-24T22:32:49.749407+0000","last_became_active":"2026-02-24T22:23:29.212665+0000","last_became_peered":"2026-02-24T22:23:29.212665+0000","last_unstale":"2026-02-24T22:32:49.749407+0000","last_undegraded":"2026-02-24T22:32:49.749407+0000","last_fullsized":"2026-02-24T22:32:49.749407+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.13","version":"0'0","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:01.756922+0000","last_change":"2026-02-24T22:35:01.756922+0000","last_active":"2026-02-24T22:35:01.756922+0000","last_peered":"2026-02-24T22:35:01.756922+0000","last_clean":"2026-02-24T22:35:01.756922+0000","last_became_active":"2026-02-24T22:15:25.825862+0000","last_became_peered":"2026-02-24T22:15:25.825862+0000","last_unstale":"2026-02-24T22:35:01.756922+0000","last_undegraded":"2026-02-24T22:35:01.756922+0000","last_fullsized":"2026-02-24T22:35:01.756922+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:01.756854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:01.756854+0000","last_clean_scrub_stamp":"2026-02-24T22:35:01.756854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:53:27.843482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"80.12","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.744628+0000","last_change":"2026-02-24T22:15:22.812712+0000","last_active":"2026-02-24T22:31:44.744628+0000","last_peered":"2026-02-24T22:31:44.744628+0000","last_clean":"2026-02-24T22:31:44.744628+0000","last_became_active":"2026-02-24T22:15:22.812405+0000","last_became_peered":"2026-02-24T22:15:22.812405+0000","last_unstale":"2026-02-24T22:31:44.744628+0000","last_undegraded":"2026-02-24T22:31:44.744628+0000","last_fullsized":"2026-02-24T22:31:44.744628+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:36:00.971366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.d","version":"312'20","reported_seq":113,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:44.733217+0000","last_change":"2026-02-24T22:13:43.761593+0000","last_active":"2026-02-24T22:31:44.733217+0000","last_peered":"2026-02-24T22:31:44.733217+0000","last_clean":"2026-02-24T22:31:44.733217+0000","last_became_active":"2026-02-24T22:13:43.761339+0000","last_became_peered":"2026-02-24T22:13:43.761339+0000","last_unstale":"2026-02-24T22:31:44.733217+0000","last_undegraded":"2026-02-24T22:31:44.733217+0000","last_fullsized":"2026-02-24T22:31:44.733217+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:13:42.750256+0000","last_clean_scrub_stamp":"2026-02-24T22:13:42.750256+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:02:40.622911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":126,"num_read_kb":76,"num_write":49,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"82.11","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:49.735108+0000","last_change":"2026-02-24T22:23:29.214793+0000","last_active":"2026-02-24T22:32:49.735108+0000","last_peered":"2026-02-24T22:32:49.735108+0000","last_clean":"2026-02-24T22:32:49.735108+0000","last_became_active":"2026-02-24T22:23:29.214167+0000","last_became_peered":"2026-02-24T22:23:29.214167+0000","last_unstale":"2026-02-24T22:32:49.735108+0000","last_undegraded":"2026-02-24T22:32:49.735108+0000","last_fullsized":"2026-02-24T22:32:49.735108+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.12","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744756+0000","last_change":"2026-02-24T22:15:25.826465+0000","last_active":"2026-02-24T22:31:49.744756+0000","last_peered":"2026-02-24T22:31:49.744756+0000","last_clean":"2026-02-24T22:31:49.744756+0000","last_became_active":"2026-02-24T22:15:25.826102+0000","last_became_peered":"2026-02-24T22:15:25.826102+0000","last_unstale":"2026-02-24T22:31:49.744756+0000","last_undegraded":"2026-02-24T22:31:49.744756+0000","last_fullsized":"2026-02-24T22:31:49.744756+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:22:01.000780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.13","version":"312'3","reported_seq":43,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:04.760554+0000","last_change":"2026-02-24T22:15:22.812393+0000","last_active":"2026-02-24T22:34:04.760554+0000","last_peered":"2026-02-24T22:34:04.760554+0000","last_clean":"2026-02-24T22:34:04.760554+0000","last_became_active":"2026-02-24T22:15:22.811927+0000","last_became_peered":"2026-02-24T22:15:22.811927+0000","last_unstale":"2026-02-24T22:34:04.760554+0000","last_undegraded":"2026-02-24T22:34:04.760554+0000","last_fullsized":"2026-02-24T22:34:04.760554+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:54:16.404989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.c","version":"312'4","reported_seq":82,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:52.107283+0000","last_change":"2026-02-24T22:34:52.107283+0000","last_active":"2026-02-24T22:34:52.107283+0000","last_peered":"2026-02-24T22:34:52.107283+0000","last_clean":"2026-02-24T22:34:52.107283+0000","last_became_active":"2026-02-24T22:13:43.761910+0000","last_became_peered":"2026-02-24T22:13:43.761910+0000","last_unstale":"2026-02-24T22:34:52.107283+0000","last_undegraded":"2026-02-24T22:34:52.107283+0000","last_fullsized":"2026-02-24T22:34:52.107283+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'4","last_scrub_stamp":"2026-02-24T22:34:52.107217+0000","last_deep_scrub":"312'4","last_deep_scrub_stamp":"2026-02-24T22:34:52.107217+0000","last_clean_scrub_stamp":"2026-02-24T22:34:52.107217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:56:38.678014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":15,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"82.16","version":"0'0","reported_seq":29,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:06.210797+0000","last_change":"2026-02-24T22:35:06.210797+0000","last_active":"2026-02-24T22:35:06.210797+0000","last_peered":"2026-02-24T22:35:06.210797+0000","last_clean":"2026-02-24T22:35:06.210797+0000","last_became_active":"2026-02-24T22:23:29.213080+0000","last_became_peered":"2026-02-24T22:23:29.213080+0000","last_unstale":"2026-02-24T22:35:06.210797+0000","last_undegraded":"2026-02-24T22:35:06.210797+0000","last_fullsized":"2026-02-24T22:35:06.210797+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:06.210726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:06.210726+0000","last_clean_scrub_stamp":"2026-02-24T22:35:06.210726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:22:55.658806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.15","version":"0'0","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:05.258451+0000","last_change":"2026-02-24T22:35:05.258451+0000","last_active":"2026-02-24T22:35:05.258451+0000","last_peered":"2026-02-24T22:35:05.258451+0000","last_clean":"2026-02-24T22:35:05.258451+0000","last_became_active":"2026-02-24T22:15:25.830922+0000","last_became_peered":"2026-02-24T22:15:25.830922+0000","last_unstale":"2026-02-24T22:35:05.258451+0000","last_undegraded":"2026-02-24T22:35:05.258451+0000","last_fullsized":"2026-02-24T22:35:05.258451+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:05.258395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:05.258395+0000","last_clean_scrub_stamp":"2026-02-24T22:35:05.258395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:02:54.284713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"80.14","version":"312'3","reported_seq":43,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:09.761022+0000","last_change":"2026-02-24T22:15:22.812446+0000","last_active":"2026-02-24T22:34:09.761022+0000","last_peered":"2026-02-24T22:34:09.761022+0000","last_clean":"2026-02-24T22:34:09.761022+0000","last_became_active":"2026-02-24T22:15:22.811924+0000","last_became_peered":"2026-02-24T22:15:22.811924+0000","last_unstale":"2026-02-24T22:34:09.761022+0000","last_undegraded":"2026-02-24T22:34:09.761022+0000","last_fullsized":"2026-02-24T22:34:09.761022+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:18:30.397097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.b","version":"312'13","reported_seq":106,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:56.646904+0000","last_change":"2026-02-24T22:34:56.646904+0000","last_active":"2026-02-24T22:34:56.646904+0000","last_peered":"2026-02-24T22:34:56.646904+0000","last_clean":"2026-02-24T22:34:56.646904+0000","last_became_active":"2026-02-24T22:13:43.756596+0000","last_became_peered":"2026-02-24T22:13:43.756596+0000","last_unstale":"2026-02-24T22:34:56.646904+0000","last_undegraded":"2026-02-24T22:34:56.646904+0000","last_fullsized":"2026-02-24T22:34:56.646904+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'13","last_scrub_stamp":"2026-02-24T22:34:56.646848+0000","last_deep_scrub":"312'13","last_deep_scrub_stamp":"2026-02-24T22:34:56.646848+0000","last_clean_scrub_stamp":"2026-02-24T22:34:56.646848+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:06:44.716107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":56,"num_write":37,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.17","version":"0'0","reported_seq":29,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:07.200744+0000","last_change":"2026-02-24T22:35:07.200744+0000","last_active":"2026-02-24T22:35:07.200744+0000","last_peered":"2026-02-24T22:35:07.200744+0000","last_clean":"2026-02-24T22:35:07.200744+0000","last_became_active":"2026-02-24T22:23:29.214217+0000","last_became_peered":"2026-02-24T22:23:29.214217+0000","last_unstale":"2026-02-24T22:35:07.200744+0000","last_undegraded":"2026-02-24T22:35:07.200744+0000","last_fullsized":"2026-02-24T22:35:07.200744+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:07.200690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:07.200690+0000","last_clean_scrub_stamp":"2026-02-24T22:35:07.200690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:02:31.790157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.14","version":"312'5","reported_seq":42,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.471281+0000","last_change":"2026-02-24T22:15:25.827827+0000","last_active":"2026-02-24T22:34:25.471281+0000","last_peered":"2026-02-24T22:34:25.471281+0000","last_clean":"2026-02-24T22:34:25.471281+0000","last_became_active":"2026-02-24T22:15:25.827557+0000","last_became_peered":"2026-02-24T22:15:25.827557+0000","last_unstale":"2026-02-24T22:34:25.471281+0000","last_undegraded":"2026-02-24T22:34:25.471281+0000","last_fullsized":"2026-02-24T22:34:25.471281+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:13:31.380536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":10,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.15","version":"312'5","reported_seq":48,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.909302+0000","last_change":"2026-02-24T22:15:22.812814+0000","last_active":"2026-02-24T22:34:25.909302+0000","last_peered":"2026-02-24T22:34:25.909302+0000","last_clean":"2026-02-24T22:34:25.909302+0000","last_became_active":"2026-02-24T22:15:22.812450+0000","last_became_peered":"2026-02-24T22:15:22.812450+0000","last_unstale":"2026-02-24T22:34:25.909302+0000","last_undegraded":"2026-02-24T22:34:25.909302+0000","last_fullsized":"2026-02-24T22:34:25.909302+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":10,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.a","version":"312'10","reported_seq":98,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:01.350761+0000","last_change":"2026-02-24T22:35:01.350761+0000","last_active":"2026-02-24T22:35:01.350761+0000","last_peered":"2026-02-24T22:35:01.350761+0000","last_clean":"2026-02-24T22:35:01.350761+0000","last_became_active":"2026-02-24T22:13:43.762226+0000","last_became_peered":"2026-02-24T22:13:43.762226+0000","last_unstale":"2026-02-24T22:35:01.350761+0000","last_undegraded":"2026-02-24T22:35:01.350761+0000","last_fullsized":"2026-02-24T22:35:01.350761+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'10","last_scrub_stamp":"2026-02-24T22:35:01.350683+0000","last_deep_scrub":"312'10","last_deep_scrub_stamp":"2026-02-24T22:35:01.350683+0000","last_clean_scrub_stamp":"2026-02-24T22:35:01.350683+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:23:37.121406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":42,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.14","version":"309'1","reported_seq":33,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:49.751496+0000","last_change":"2026-02-24T22:24:01.299030+0000","last_active":"2026-02-24T22:32:49.751496+0000","last_peered":"2026-02-24T22:32:49.751496+0000","last_clean":"2026-02-24T22:32:49.751496+0000","last_became_active":"2026-02-24T22:24:01.297193+0000","last_became_peered":"2026-02-24T22:24:01.297193+0000","last_unstale":"2026-02-24T22:32:49.751496+0000","last_undegraded":"2026-02-24T22:32:49.751496+0000","last_fullsized":"2026-02-24T22:32:49.751496+0000","mapping_epoch":311,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":312,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.17","version":"312'3","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:25.871072+0000","last_change":"2026-02-24T22:15:25.833669+0000","last_active":"2026-02-24T22:34:25.871072+0000","last_peered":"2026-02-24T22:34:25.871072+0000","last_clean":"2026-02-24T22:34:25.871072+0000","last_became_active":"2026-02-24T22:15:25.833430+0000","last_became_peered":"2026-02-24T22:15:25.833430+0000","last_unstale":"2026-02-24T22:34:25.871072+0000","last_undegraded":"2026-02-24T22:34:25.871072+0000","last_fullsized":"2026-02-24T22:34:25.871072+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"80.16","version":"0'0","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:01.238371+0000","last_change":"2026-02-24T22:35:01.238371+0000","last_active":"2026-02-24T22:35:01.238371+0000","last_peered":"2026-02-24T22:35:01.238371+0000","last_clean":"2026-02-24T22:35:01.238371+0000","last_became_active":"2026-02-24T22:15:22.817150+0000","last_became_peered":"2026-02-24T22:15:22.817150+0000","last_unstale":"2026-02-24T22:35:01.238371+0000","last_undegraded":"2026-02-24T22:35:01.238371+0000","last_fullsized":"2026-02-24T22:35:01.238371+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:01.238320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:01.238320+0000","last_clean_scrub_stamp":"2026-02-24T22:35:01.238320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:55:36.219572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.9","version":"312'24","reported_seq":133,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:09.179827+0000","last_change":"2026-02-24T22:35:09.179827+0000","last_active":"2026-02-24T22:35:09.179827+0000","last_peered":"2026-02-24T22:35:09.179827+0000","last_clean":"2026-02-24T22:35:09.179827+0000","last_became_active":"2026-02-24T22:13:43.756777+0000","last_became_peered":"2026-02-24T22:13:43.756777+0000","last_unstale":"2026-02-24T22:35:09.179827+0000","last_undegraded":"2026-02-24T22:35:09.179827+0000","last_fullsized":"2026-02-24T22:35:09.179827+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'24","last_scrub_stamp":"2026-02-24T22:35:09.179742+0000","last_deep_scrub":"312'24","last_deep_scrub_stamp":"2026-02-24T22:35:09.179742+0000","last_clean_scrub_stamp":"2026-02-24T22:35:09.179742+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:51:13.859546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":107,"num_write":60,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.15","version":"312'2","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:07.645456+0000","last_change":"2026-02-24T22:35:07.645456+0000","last_active":"2026-02-24T22:35:07.645456+0000","last_peered":"2026-02-24T22:35:07.645456+0000","last_clean":"2026-02-24T22:35:07.645456+0000","last_became_active":"2026-02-24T22:23:29.198000+0000","last_became_peered":"2026-02-24T22:23:29.198000+0000","last_unstale":"2026-02-24T22:35:07.645456+0000","last_undegraded":"2026-02-24T22:35:07.645456+0000","last_fullsized":"2026-02-24T22:35:07.645456+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:35:07.645381+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:35:07.645381+0000","last_clean_scrub_stamp":"2026-02-24T22:35:07.645381+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:16:38.802468+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.16","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.748803+0000","last_change":"2026-02-24T22:15:25.828675+0000","last_active":"2026-02-24T22:32:24.748803+0000","last_peered":"2026-02-24T22:32:24.748803+0000","last_clean":"2026-02-24T22:32:24.748803+0000","last_became_active":"2026-02-24T22:15:25.828066+0000","last_became_peered":"2026-02-24T22:15:25.828066+0000","last_unstale":"2026-02-24T22:32:24.748803+0000","last_undegraded":"2026-02-24T22:32:24.748803+0000","last_fullsized":"2026-02-24T22:32:24.748803+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.17","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:19.746146+0000","last_change":"2026-02-24T22:15:22.818517+0000","last_active":"2026-02-24T22:32:19.746146+0000","last_peered":"2026-02-24T22:32:19.746146+0000","last_clean":"2026-02-24T22:32:19.746146+0000","last_became_active":"2026-02-24T22:15:22.817950+0000","last_became_peered":"2026-02-24T22:15:22.817950+0000","last_unstale":"2026-02-24T22:32:19.746146+0000","last_undegraded":"2026-02-24T22:32:19.746146+0000","last_fullsized":"2026-02-24T22:32:19.746146+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.8","version":"312'26","reported_seq":144,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:09.733355+0000","last_change":"2026-02-24T22:35:09.733355+0000","last_active":"2026-02-24T22:35:09.733355+0000","last_peered":"2026-02-24T22:35:09.733355+0000","last_clean":"2026-02-24T22:35:09.733355+0000","last_became_active":"2026-02-24T22:13:43.761558+0000","last_became_peered":"2026-02-24T22:13:43.761558+0000","last_unstale":"2026-02-24T22:35:09.733355+0000","last_undegraded":"2026-02-24T22:35:09.733355+0000","last_fullsized":"2026-02-24T22:35:09.733355+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'26","last_scrub_stamp":"2026-02-24T22:35:09.733252+0000","last_deep_scrub":"312'26","last_deep_scrub_stamp":"2026-02-24T22:35:09.733252+0000","last_clean_scrub_stamp":"2026-02-24T22:35:09.733252+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:51:48.673281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":152,"num_write":62,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.1a","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.723700+0000","last_change":"2026-02-24T22:23:29.213344+0000","last_active":"2026-02-24T22:30:59.723700+0000","last_peered":"2026-02-24T22:30:59.723700+0000","last_clean":"2026-02-24T22:30:59.723700+0000","last_became_active":"2026-02-24T22:23:29.212709+0000","last_became_peered":"2026-02-24T22:23:29.212709+0000","last_unstale":"2026-02-24T22:30:59.723700+0000","last_undegraded":"2026-02-24T22:30:59.723700+0000","last_fullsized":"2026-02-24T22:30:59.723700+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:26:30.998666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.19","version":"0'0","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:09.322953+0000","last_change":"2026-02-24T22:35:09.322953+0000","last_active":"2026-02-24T22:35:09.322953+0000","last_peered":"2026-02-24T22:35:09.322953+0000","last_clean":"2026-02-24T22:35:09.322953+0000","last_became_active":"2026-02-24T22:15:25.831885+0000","last_became_peered":"2026-02-24T22:15:25.831885+0000","last_unstale":"2026-02-24T22:35:09.322953+0000","last_undegraded":"2026-02-24T22:35:09.322953+0000","last_fullsized":"2026-02-24T22:35:09.322953+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:09.322906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:09.322906+0000","last_clean_scrub_stamp":"2026-02-24T22:35:09.322906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:18:38.861480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.18","version":"0'0","reported_seq":37,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:19.745911+0000","last_change":"2026-02-24T22:15:22.810683+0000","last_active":"2026-02-24T22:32:19.745911+0000","last_peered":"2026-02-24T22:32:19.745911+0000","last_clean":"2026-02-24T22:32:19.745911+0000","last_became_active":"2026-02-24T22:15:22.810156+0000","last_became_peered":"2026-02-24T22:15:22.810156+0000","last_unstale":"2026-02-24T22:32:19.745911+0000","last_undegraded":"2026-02-24T22:32:19.745911+0000","last_fullsized":"2026-02-24T22:32:19.745911+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"79.7","version":"0'0","reported_seq":72,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:08.320149+0000","last_change":"2026-02-24T22:35:08.320149+0000","last_active":"2026-02-24T22:35:08.320149+0000","last_peered":"2026-02-24T22:35:08.320149+0000","last_clean":"2026-02-24T22:35:08.320149+0000","last_became_active":"2026-02-24T22:13:43.755957+0000","last_became_peered":"2026-02-24T22:13:43.755957+0000","last_unstale":"2026-02-24T22:35:08.320149+0000","last_undegraded":"2026-02-24T22:35:08.320149+0000","last_fullsized":"2026-02-24T22:35:08.320149+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:08.320097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:08.320097+0000","last_clean_scrub_stamp":"2026-02-24T22:35:08.320097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:38:24.732745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.1b","version":"0'0","reported_seq":29,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:08.661691+0000","last_change":"2026-02-24T22:35:08.661691+0000","last_active":"2026-02-24T22:35:08.661691+0000","last_peered":"2026-02-24T22:35:08.661691+0000","last_clean":"2026-02-24T22:35:08.661691+0000","last_became_active":"2026-02-24T22:23:29.200152+0000","last_became_peered":"2026-02-24T22:23:29.200152+0000","last_unstale":"2026-02-24T22:35:08.661691+0000","last_undegraded":"2026-02-24T22:35:08.661691+0000","last_fullsized":"2026-02-24T22:35:08.661691+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:08.661629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:08.661629+0000","last_clean_scrub_stamp":"2026-02-24T22:35:08.661629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:43:02.912489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.18","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:24.746457+0000","last_change":"2026-02-24T22:15:25.824057+0000","last_active":"2026-02-24T22:32:24.746457+0000","last_peered":"2026-02-24T22:32:24.746457+0000","last_clean":"2026-02-24T22:32:24.746457+0000","last_became_active":"2026-02-24T22:15:25.823415+0000","last_became_peered":"2026-02-24T22:15:25.823415+0000","last_unstale":"2026-02-24T22:32:24.746457+0000","last_undegraded":"2026-02-24T22:32:24.746457+0000","last_fullsized":"2026-02-24T22:32:24.746457+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.19","version":"0'0","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:59.228352+0000","last_change":"2026-02-24T22:34:59.228352+0000","last_active":"2026-02-24T22:34:59.228352+0000","last_peered":"2026-02-24T22:34:59.228352+0000","last_clean":"2026-02-24T22:34:59.228352+0000","last_became_active":"2026-02-24T22:15:22.816826+0000","last_became_peered":"2026-02-24T22:15:22.816826+0000","last_unstale":"2026-02-24T22:34:59.228352+0000","last_undegraded":"2026-02-24T22:34:59.228352+0000","last_fullsized":"2026-02-24T22:34:59.228352+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:59.228267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:59.228267+0000","last_clean_scrub_stamp":"2026-02-24T22:34:59.228267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:53:36.867225+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.6","version":"301'14","reported_seq":116,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:07.698724+0000","last_change":"2026-02-24T22:35:07.698724+0000","last_active":"2026-02-24T22:35:07.698724+0000","last_peered":"2026-02-24T22:35:07.698724+0000","last_clean":"2026-02-24T22:35:07.698724+0000","last_became_active":"2026-02-24T22:13:43.754753+0000","last_became_peered":"2026-02-24T22:13:43.754753+0000","last_unstale":"2026-02-24T22:35:07.698724+0000","last_undegraded":"2026-02-24T22:35:07.698724+0000","last_fullsized":"2026-02-24T22:35:07.698724+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"301'14","last_scrub_stamp":"2026-02-24T22:35:07.698665+0000","last_deep_scrub":"301'14","last_deep_scrub_stamp":"2026-02-24T22:35:07.698665+0000","last_clean_scrub_stamp":"2026-02-24T22:35:07.698665+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:07:27.811145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":92,"num_write":34,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"82.18","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:44.726953+0000","last_change":"2026-02-24T22:23:29.213704+0000","last_active":"2026-02-24T22:30:44.726953+0000","last_peered":"2026-02-24T22:30:44.726953+0000","last_clean":"2026-02-24T22:30:44.726953+0000","last_became_active":"2026-02-24T22:23:29.213387+0000","last_became_peered":"2026-02-24T22:23:29.213387+0000","last_unstale":"2026-02-24T22:30:44.726953+0000","last_undegraded":"2026-02-24T22:30:44.726953+0000","last_fullsized":"2026-02-24T22:30:44.726953+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:46:11.186061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.1b","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.720738+0000","last_change":"2026-02-24T22:15:25.832293+0000","last_active":"2026-02-24T22:30:29.720738+0000","last_peered":"2026-02-24T22:30:29.720738+0000","last_clean":"2026-02-24T22:30:29.720738+0000","last_became_active":"2026-02-24T22:15:25.832097+0000","last_became_peered":"2026-02-24T22:15:25.832097+0000","last_unstale":"2026-02-24T22:30:29.720738+0000","last_undegraded":"2026-02-24T22:30:29.720738+0000","last_fullsized":"2026-02-24T22:30:29.720738+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:19:51.400688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.1a","version":"0'0","reported_seq":44,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:05.379833+0000","last_change":"2026-02-24T22:35:05.379833+0000","last_active":"2026-02-24T22:35:05.379833+0000","last_peered":"2026-02-24T22:35:05.379833+0000","last_clean":"2026-02-24T22:35:05.379833+0000","last_became_active":"2026-02-24T22:15:22.818885+0000","last_became_peered":"2026-02-24T22:15:22.818885+0000","last_unstale":"2026-02-24T22:35:05.379833+0000","last_undegraded":"2026-02-24T22:35:05.379833+0000","last_fullsized":"2026-02-24T22:35:05.379833+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:05.379757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:05.379757+0000","last_clean_scrub_stamp":"2026-02-24T22:35:05.379757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:37:10.955550+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"79.5","version":"312'19","reported_seq":130,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:07.304569+0000","last_change":"2026-02-24T22:35:07.304569+0000","last_active":"2026-02-24T22:35:07.304569+0000","last_peered":"2026-02-24T22:35:07.304569+0000","last_clean":"2026-02-24T22:35:07.304569+0000","last_became_active":"2026-02-24T22:13:43.755495+0000","last_became_peered":"2026-02-24T22:13:43.755495+0000","last_unstale":"2026-02-24T22:35:07.304569+0000","last_undegraded":"2026-02-24T22:35:07.304569+0000","last_fullsized":"2026-02-24T22:35:07.304569+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'19","last_scrub_stamp":"2026-02-24T22:35:07.304515+0000","last_deep_scrub":"312'19","last_deep_scrub_stamp":"2026-02-24T22:35:07.304515+0000","last_clean_scrub_stamp":"2026-02-24T22:35:07.304515+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:23:18.475886+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":164,"num_read_kb":112,"num_write":47,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.19","version":"0'0","reported_seq":29,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:08.226999+0000","last_change":"2026-02-24T22:35:08.226999+0000","last_active":"2026-02-24T22:35:08.226999+0000","last_peered":"2026-02-24T22:35:08.226999+0000","last_clean":"2026-02-24T22:35:08.226999+0000","last_became_active":"2026-02-24T22:23:29.212268+0000","last_became_peered":"2026-02-24T22:23:29.212268+0000","last_unstale":"2026-02-24T22:35:08.226999+0000","last_undegraded":"2026-02-24T22:35:08.226999+0000","last_fullsized":"2026-02-24T22:35:08.226999+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:08.226947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:08.226947+0000","last_clean_scrub_stamp":"2026-02-24T22:35:08.226947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:06:46.348170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.1a","version":"312'3","reported_seq":44,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:03.754600+0000","last_change":"2026-02-24T22:35:03.754600+0000","last_active":"2026-02-24T22:35:03.754600+0000","last_peered":"2026-02-24T22:35:03.754600+0000","last_clean":"2026-02-24T22:35:03.754600+0000","last_became_active":"2026-02-24T22:15:25.825969+0000","last_became_peered":"2026-02-24T22:15:25.825969+0000","last_unstale":"2026-02-24T22:35:03.754600+0000","last_undegraded":"2026-02-24T22:35:03.754600+0000","last_fullsized":"2026-02-24T22:35:03.754600+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:03.754545+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:03.754545+0000","last_clean_scrub_stamp":"2026-02-24T22:35:03.754545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:00:51.414538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"80.1b","version":"312'3","reported_seq":42,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:54.750003+0000","last_change":"2026-02-24T22:15:22.816734+0000","last_active":"2026-02-24T22:32:54.750003+0000","last_peered":"2026-02-24T22:32:54.750003+0000","last_clean":"2026-02-24T22:32:54.750003+0000","last_became_active":"2026-02-24T22:15:22.816479+0000","last_became_peered":"2026-02-24T22:15:22.816479+0000","last_unstale":"2026-02-24T22:32:54.750003+0000","last_undegraded":"2026-02-24T22:32:54.750003+0000","last_fullsized":"2026-02-24T22:32:54.750003+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:57:39.996802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.4","version":"0'0","reported_seq":79,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:06.735654+0000","last_change":"2026-02-24T22:35:06.735654+0000","last_active":"2026-02-24T22:35:06.735654+0000","last_peered":"2026-02-24T22:35:06.735654+0000","last_clean":"2026-02-24T22:35:06.735654+0000","last_became_active":"2026-02-24T22:14:00.741839+0000","last_became_peered":"2026-02-24T22:14:00.741839+0000","last_unstale":"2026-02-24T22:35:06.735654+0000","last_undegraded":"2026-02-24T22:35:06.735654+0000","last_fullsized":"2026-02-24T22:35:06.735654+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":293,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:06.735602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:06.735602+0000","last_clean_scrub_stamp":"2026-02-24T22:35:06.735602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:33:39.827521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"82.1e","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:34.727290+0000","last_change":"2026-02-24T22:23:29.215779+0000","last_active":"2026-02-24T22:31:34.727290+0000","last_peered":"2026-02-24T22:31:34.727290+0000","last_clean":"2026-02-24T22:31:34.727290+0000","last_became_active":"2026-02-24T22:23:29.215604+0000","last_became_peered":"2026-02-24T22:23:29.215604+0000","last_unstale":"2026-02-24T22:31:34.727290+0000","last_undegraded":"2026-02-24T22:31:34.727290+0000","last_fullsized":"2026-02-24T22:31:34.727290+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:58:56.390417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.1d","version":"312'3","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:59.750434+0000","last_change":"2026-02-24T22:15:25.828517+0000","last_active":"2026-02-24T22:32:59.750434+0000","last_peered":"2026-02-24T22:32:59.750434+0000","last_clean":"2026-02-24T22:32:59.750434+0000","last_became_active":"2026-02-24T22:15:25.828335+0000","last_became_peered":"2026-02-24T22:15:25.828335+0000","last_unstale":"2026-02-24T22:32:59.750434+0000","last_undegraded":"2026-02-24T22:32:59.750434+0000","last_fullsized":"2026-02-24T22:32:59.750434+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:18:15.606086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.1c","version":"0'0","reported_seq":36,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:24.733619+0000","last_change":"2026-02-24T22:15:22.816853+0000","last_active":"2026-02-24T22:30:24.733619+0000","last_peered":"2026-02-24T22:30:24.733619+0000","last_clean":"2026-02-24T22:30:24.733619+0000","last_became_active":"2026-02-24T22:15:22.816548+0000","last_became_peered":"2026-02-24T22:15:22.816548+0000","last_unstale":"2026-02-24T22:30:24.733619+0000","last_undegraded":"2026-02-24T22:30:24.733619+0000","last_fullsized":"2026-02-24T22:30:24.733619+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:00:55.883988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.3","version":"312'6","reported_seq":86,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:55.621542+0000","last_change":"2026-02-24T22:34:55.621542+0000","last_active":"2026-02-24T22:34:55.621542+0000","last_peered":"2026-02-24T22:34:55.621542+0000","last_clean":"2026-02-24T22:34:55.621542+0000","last_became_active":"2026-02-24T22:13:43.756408+0000","last_became_peered":"2026-02-24T22:13:43.756408+0000","last_unstale":"2026-02-24T22:34:55.621542+0000","last_undegraded":"2026-02-24T22:34:55.621542+0000","last_fullsized":"2026-02-24T22:34:55.621542+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'6","last_scrub_stamp":"2026-02-24T22:34:55.621486+0000","last_deep_scrub":"312'6","last_deep_scrub_stamp":"2026-02-24T22:34:55.621486+0000","last_clean_scrub_stamp":"2026-02-24T22:34:55.621486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:43:15.014040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":20,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.1f","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.744790+0000","last_change":"2026-02-24T22:23:29.199410+0000","last_active":"2026-02-24T22:31:49.744790+0000","last_peered":"2026-02-24T22:31:49.744790+0000","last_clean":"2026-02-24T22:31:49.744790+0000","last_became_active":"2026-02-24T22:23:29.199179+0000","last_became_peered":"2026-02-24T22:23:29.199179+0000","last_unstale":"2026-02-24T22:31:49.744790+0000","last_undegraded":"2026-02-24T22:31:49.744790+0000","last_fullsized":"2026-02-24T22:31:49.744790+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:21:23.823724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.1c","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:29.725548+0000","last_change":"2026-02-24T22:15:25.834316+0000","last_active":"2026-02-24T22:30:29.725548+0000","last_peered":"2026-02-24T22:30:29.725548+0000","last_clean":"2026-02-24T22:30:29.725548+0000","last_became_active":"2026-02-24T22:15:25.834072+0000","last_became_peered":"2026-02-24T22:15:25.834072+0000","last_unstale":"2026-02-24T22:30:29.725548+0000","last_undegraded":"2026-02-24T22:30:29.725548+0000","last_fullsized":"2026-02-24T22:30:29.725548+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:45:39.925744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"80.1d","version":"312'2","reported_seq":40,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:54.750046+0000","last_change":"2026-02-24T22:15:22.817173+0000","last_active":"2026-02-24T22:32:54.750046+0000","last_peered":"2026-02-24T22:32:54.750046+0000","last_clean":"2026-02-24T22:32:54.750046+0000","last_became_active":"2026-02-24T22:15:22.816615+0000","last_became_peered":"2026-02-24T22:15:22.816615+0000","last_unstale":"2026-02-24T22:32:54.750046+0000","last_undegraded":"2026-02-24T22:32:54.750046+0000","last_fullsized":"2026-02-24T22:32:54.750046+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:21.804007+0000","last_clean_scrub_stamp":"2026-02-24T22:15:21.804007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:25:34.323299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.2","version":"312'5","reported_seq":86,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:07.232682+0000","last_change":"2026-02-24T22:35:07.232682+0000","last_active":"2026-02-24T22:35:07.232682+0000","last_peered":"2026-02-24T22:35:07.232682+0000","last_clean":"2026-02-24T22:35:07.232682+0000","last_became_active":"2026-02-24T22:13:43.755241+0000","last_became_peered":"2026-02-24T22:13:43.755241+0000","last_unstale":"2026-02-24T22:35:07.232682+0000","last_undegraded":"2026-02-24T22:35:07.232682+0000","last_fullsized":"2026-02-24T22:35:07.232682+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'5","last_scrub_stamp":"2026-02-24T22:35:07.232617+0000","last_deep_scrub":"312'5","last_deep_scrub_stamp":"2026-02-24T22:35:07.232617+0000","last_clean_scrub_stamp":"2026-02-24T22:35:07.232617+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:43:07.685953+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":23,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.1c","version":"312'2","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:09.205907+0000","last_change":"2026-02-24T22:35:09.205907+0000","last_active":"2026-02-24T22:35:09.205907+0000","last_peered":"2026-02-24T22:35:09.205907+0000","last_clean":"2026-02-24T22:35:09.205907+0000","last_became_active":"2026-02-24T22:23:29.213490+0000","last_became_peered":"2026-02-24T22:23:29.213490+0000","last_unstale":"2026-02-24T22:35:09.205907+0000","last_undegraded":"2026-02-24T22:35:09.205907+0000","last_fullsized":"2026-02-24T22:35:09.205907+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:35:09.205819+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:35:09.205819+0000","last_clean_scrub_stamp":"2026-02-24T22:35:09.205819+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:51:15.132591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.1f","version":"0'0","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:09.747978+0000","last_change":"2026-02-24T22:15:25.828181+0000","last_active":"2026-02-24T22:32:09.747978+0000","last_peered":"2026-02-24T22:32:09.747978+0000","last_clean":"2026-02-24T22:32:09.747978+0000","last_became_active":"2026-02-24T22:15:25.827873+0000","last_became_peered":"2026-02-24T22:15:25.827873+0000","last_unstale":"2026-02-24T22:32:09.747978+0000","last_undegraded":"2026-02-24T22:32:09.747978+0000","last_fullsized":"2026-02-24T22:32:09.747978+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:15:24.819010+0000","last_clean_scrub_stamp":"2026-02-24T22:15:24.819010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:19:57.290828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.1e","version":"0'0","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:04.400700+0000","last_change":"2026-02-24T22:35:04.400700+0000","last_active":"2026-02-24T22:35:04.400700+0000","last_peered":"2026-02-24T22:35:04.400700+0000","last_clean":"2026-02-24T22:35:04.400700+0000","last_became_active":"2026-02-24T22:15:22.817337+0000","last_became_peered":"2026-02-24T22:15:22.817337+0000","last_unstale":"2026-02-24T22:35:04.400700+0000","last_undegraded":"2026-02-24T22:35:04.400700+0000","last_fullsized":"2026-02-24T22:35:04.400700+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:04.400607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:04.400607+0000","last_clean_scrub_stamp":"2026-02-24T22:35:04.400607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:40:28.932410+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"79.1","version":"312'23","reported_seq":134,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:00.384907+0000","last_change":"2026-02-24T22:35:00.384907+0000","last_active":"2026-02-24T22:35:00.384907+0000","last_peered":"2026-02-24T22:35:00.384907+0000","last_clean":"2026-02-24T22:35:00.384907+0000","last_became_active":"2026-02-24T22:13:43.760933+0000","last_became_peered":"2026-02-24T22:13:43.760933+0000","last_unstale":"2026-02-24T22:35:00.384907+0000","last_undegraded":"2026-02-24T22:35:00.384907+0000","last_fullsized":"2026-02-24T22:35:00.384907+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'23","last_scrub_stamp":"2026-02-24T22:35:00.384839+0000","last_deep_scrub":"312'23","last_deep_scrub_stamp":"2026-02-24T22:35:00.384839+0000","last_clean_scrub_stamp":"2026-02-24T22:35:00.384839+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:47:05.415176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":174,"num_read_kb":113,"num_write":59,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.1d","version":"0'0","reported_seq":21,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:30:59.723721+0000","last_change":"2026-02-24T22:23:29.215553+0000","last_active":"2026-02-24T22:30:59.723721+0000","last_peered":"2026-02-24T22:30:59.723721+0000","last_clean":"2026-02-24T22:30:59.723721+0000","last_became_active":"2026-02-24T22:23:29.215334+0000","last_became_peered":"2026-02-24T22:23:29.215334+0000","last_unstale":"2026-02-24T22:30:59.723721+0000","last_undegraded":"2026-02-24T22:30:59.723721+0000","last_fullsized":"2026-02-24T22:30:59.723721+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:01:42.978764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.1e","version":"0'0","reported_seq":40,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:04.737646+0000","last_change":"2026-02-24T22:35:04.737646+0000","last_active":"2026-02-24T22:35:04.737646+0000","last_peered":"2026-02-24T22:35:04.737646+0000","last_clean":"2026-02-24T22:35:04.737646+0000","last_became_active":"2026-02-24T22:15:25.827124+0000","last_became_peered":"2026-02-24T22:15:25.827124+0000","last_unstale":"2026-02-24T22:35:04.737646+0000","last_undegraded":"2026-02-24T22:35:04.737646+0000","last_fullsized":"2026-02-24T22:35:04.737646+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:04.737570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:04.737570+0000","last_clean_scrub_stamp":"2026-02-24T22:35:04.737570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:38:11.206092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"80.1f","version":"312'3","reported_seq":52,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:02.210357+0000","last_change":"2026-02-24T22:35:02.210357+0000","last_active":"2026-02-24T22:35:02.210357+0000","last_peered":"2026-02-24T22:35:02.210357+0000","last_clean":"2026-02-24T22:35:02.210357+0000","last_became_active":"2026-02-24T22:15:22.815938+0000","last_became_peered":"2026-02-24T22:15:22.815938+0000","last_unstale":"2026-02-24T22:35:02.210357+0000","last_undegraded":"2026-02-24T22:35:02.210357+0000","last_fullsized":"2026-02-24T22:35:02.210357+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:02.210297+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:02.210297+0000","last_clean_scrub_stamp":"2026-02-24T22:35:02.210297+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:26:34.542452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.0","version":"0'0","reported_seq":73,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:05.263755+0000","last_change":"2026-02-24T22:35:05.263755+0000","last_active":"2026-02-24T22:35:05.263755+0000","last_peered":"2026-02-24T22:35:05.263755+0000","last_clean":"2026-02-24T22:35:05.263755+0000","last_became_active":"2026-02-24T22:13:43.755266+0000","last_became_peered":"2026-02-24T22:13:43.755266+0000","last_unstale":"2026-02-24T22:35:05.263755+0000","last_undegraded":"2026-02-24T22:35:05.263755+0000","last_fullsized":"2026-02-24T22:35:05.263755+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:05.263697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:05.263697+0000","last_clean_scrub_stamp":"2026-02-24T22:35:05.263697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:19:26.298794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"312'276","reported_seq":990,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.099244+0000","last_change":"2026-02-24T22:34:44.099244+0000","last_active":"2026-02-24T22:34:44.099244+0000","last_peered":"2026-02-24T22:34:44.099244+0000","last_clean":"2026-02-24T22:34:44.099244+0000","last_became_active":"2026-02-24T22:05:16.192589+0000","last_became_peered":"2026-02-24T22:05:16.192589+0000","last_unstale":"2026-02-24T22:34:44.099244+0000","last_undegraded":"2026-02-24T22:34:44.099244+0000","last_fullsized":"2026-02-24T22:34:44.099244+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'276","last_scrub_stamp":"2026-02-24T22:34:44.099189+0000","last_deep_scrub":"312'276","last_deep_scrub_stamp":"2026-02-24T22:34:44.099189+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.099189+0000","objects_scrubbed":5,"log_size":76,"log_dups_size":200,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:12:08.217874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":433,"num_read_kb":342,"num_write":1269,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"312'220","reported_seq":930,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.919413+0000","last_change":"2026-02-24T22:34:46.919413+0000","last_active":"2026-02-24T22:34:46.919413+0000","last_peered":"2026-02-24T22:34:46.919413+0000","last_clean":"2026-02-24T22:34:46.919413+0000","last_became_active":"2026-02-24T22:05:18.192203+0000","last_became_peered":"2026-02-24T22:05:18.192203+0000","last_unstale":"2026-02-24T22:34:46.919413+0000","last_undegraded":"2026-02-24T22:34:46.919413+0000","last_fullsized":"2026-02-24T22:34:46.919413+0000","mapping_epoch":24,"log_start":"312'197","ondisk_log_start":"312'197","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'220","last_scrub_stamp":"2026-02-24T22:34:46.919375+0000","last_deep_scrub":"312'220","last_deep_scrub_stamp":"2026-02-24T22:34:46.919375+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.919375+0000","objects_scrubbed":23,"log_size":23,"log_dups_size":197,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:53:24.900487+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":780,"num_read_kb":643,"num_write":282,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"312'380","reported_seq":1197,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.032775+0000","last_change":"2026-02-24T22:34:36.032775+0000","last_active":"2026-02-24T22:34:36.032775+0000","last_peered":"2026-02-24T22:34:36.032775+0000","last_clean":"2026-02-24T22:34:36.032775+0000","last_became_active":"2026-02-24T22:05:16.181379+0000","last_became_peered":"2026-02-24T22:05:16.181379+0000","last_unstale":"2026-02-24T22:34:36.032775+0000","last_undegraded":"2026-02-24T22:34:36.032775+0000","last_fullsized":"2026-02-24T22:34:36.032775+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'380","last_scrub_stamp":"2026-02-24T22:34:36.032715+0000","last_deep_scrub":"312'380","last_deep_scrub_stamp":"2026-02-24T22:34:36.032715+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.032715+0000","objects_scrubbed":2,"log_size":80,"log_dups_size":300,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:22:39.124170+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":922,"num_read_kb":611,"num_write":1540,"num_write_kb":5240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"312'487","reported_seq":1294,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.222322+0000","last_change":"2026-02-24T22:34:47.222322+0000","last_active":"2026-02-24T22:34:47.222322+0000","last_peered":"2026-02-24T22:34:47.222322+0000","last_clean":"2026-02-24T22:34:47.222322+0000","last_became_active":"2026-02-24T22:05:18.190689+0000","last_became_peered":"2026-02-24T22:05:18.190689+0000","last_unstale":"2026-02-24T22:34:47.222322+0000","last_undegraded":"2026-02-24T22:34:47.222322+0000","last_fullsized":"2026-02-24T22:34:47.222322+0000","mapping_epoch":24,"log_start":"312'400","ondisk_log_start":"312'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'487","last_scrub_stamp":"2026-02-24T22:34:47.222271+0000","last_deep_scrub":"312'487","last_deep_scrub_stamp":"2026-02-24T22:34:47.222271+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.222271+0000","objects_scrubbed":28,"log_size":87,"log_dups_size":400,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:40:22.630753+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":2017,"num_read_kb":1670,"num_write":788,"num_write_kb":502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"312'453","reported_seq":1248,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.730366+0000","last_change":"2026-02-24T22:34:38.730366+0000","last_active":"2026-02-24T22:34:38.730366+0000","last_peered":"2026-02-24T22:34:38.730366+0000","last_clean":"2026-02-24T22:34:38.730366+0000","last_became_active":"2026-02-24T22:05:16.180681+0000","last_became_peered":"2026-02-24T22:05:16.180681+0000","last_unstale":"2026-02-24T22:34:38.730366+0000","last_undegraded":"2026-02-24T22:34:38.730366+0000","last_fullsized":"2026-02-24T22:34:38.730366+0000","mapping_epoch":22,"log_start":"312'400","ondisk_log_start":"312'400","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'453","last_scrub_stamp":"2026-02-24T22:34:38.730307+0000","last_deep_scrub":"312'453","last_deep_scrub_stamp":"2026-02-24T22:34:38.730307+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.730307+0000","objects_scrubbed":5,"log_size":53,"log_dups_size":400,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:09:15.011742+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242910,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":903,"num_read_kb":6172,"num_write":1607,"num_write_kb":5459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"312'102","reported_seq":758,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.416387+0000","last_change":"2026-02-24T22:34:48.416387+0000","last_active":"2026-02-24T22:34:48.416387+0000","last_peered":"2026-02-24T22:34:48.416387+0000","last_clean":"2026-02-24T22:34:48.416387+0000","last_became_active":"2026-02-24T22:05:18.190727+0000","last_became_peered":"2026-02-24T22:05:18.190727+0000","last_unstale":"2026-02-24T22:34:48.416387+0000","last_undegraded":"2026-02-24T22:34:48.416387+0000","last_fullsized":"2026-02-24T22:34:48.416387+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'102","last_scrub_stamp":"2026-02-24T22:34:48.416304+0000","last_deep_scrub":"312'102","last_deep_scrub_stamp":"2026-02-24T22:34:48.416304+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.416304+0000","objects_scrubbed":22,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:05:48.774357+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":268,"num_read_kb":220,"num_write":112,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"312'294","reported_seq":986,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.956244+0000","last_change":"2026-02-24T22:34:43.956244+0000","last_active":"2026-02-24T22:34:43.956244+0000","last_peered":"2026-02-24T22:34:43.956244+0000","last_clean":"2026-02-24T22:34:43.956244+0000","last_became_active":"2026-02-24T22:05:16.192922+0000","last_became_peered":"2026-02-24T22:05:16.192922+0000","last_unstale":"2026-02-24T22:34:43.956244+0000","last_undegraded":"2026-02-24T22:34:43.956244+0000","last_fullsized":"2026-02-24T22:34:43.956244+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'294","last_scrub_stamp":"2026-02-24T22:34:43.956165+0000","last_deep_scrub":"312'294","last_deep_scrub_stamp":"2026-02-24T22:34:43.956165+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.956165+0000","objects_scrubbed":5,"log_size":94,"log_dups_size":200,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:47:34.541512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":378,"num_read_kb":3190,"num_write":1202,"num_write_kb":12582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"312'158","reported_seq":847,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.908235+0000","last_change":"2026-02-24T22:34:47.908235+0000","last_active":"2026-02-24T22:34:47.908235+0000","last_peered":"2026-02-24T22:34:47.908235+0000","last_clean":"2026-02-24T22:34:47.908235+0000","last_became_active":"2026-02-24T22:05:18.184014+0000","last_became_peered":"2026-02-24T22:05:18.184014+0000","last_unstale":"2026-02-24T22:34:47.908235+0000","last_undegraded":"2026-02-24T22:34:47.908235+0000","last_fullsized":"2026-02-24T22:34:47.908235+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'158","last_scrub_stamp":"2026-02-24T22:34:47.908167+0000","last_deep_scrub":"312'158","last_deep_scrub_stamp":"2026-02-24T22:34:47.908167+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.908167+0000","objects_scrubbed":23,"log_size":58,"log_dups_size":100,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:57:20.364094+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":489,"num_read_kb":401,"num_write":208,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"312'280","reported_seq":999,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.068430+0000","last_change":"2026-02-24T22:34:43.068430+0000","last_active":"2026-02-24T22:34:43.068430+0000","last_peered":"2026-02-24T22:34:43.068430+0000","last_clean":"2026-02-24T22:34:43.068430+0000","last_became_active":"2026-02-24T22:05:16.191055+0000","last_became_peered":"2026-02-24T22:05:16.191055+0000","last_unstale":"2026-02-24T22:34:43.068430+0000","last_undegraded":"2026-02-24T22:34:43.068430+0000","last_fullsized":"2026-02-24T22:34:43.068430+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'280","last_scrub_stamp":"2026-02-24T22:34:43.068300+0000","last_deep_scrub":"312'280","last_deep_scrub_stamp":"2026-02-24T22:34:43.068300+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.068300+0000","objects_scrubbed":9,"log_size":80,"log_dups_size":200,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:55:12.515669+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":135,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":423,"num_read_kb":15420,"num_write":1201,"num_write_kb":8454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"312'497","reported_seq":1222,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.721550+0000","last_change":"2026-02-24T22:34:42.721550+0000","last_active":"2026-02-24T22:34:42.721550+0000","last_peered":"2026-02-24T22:34:42.721550+0000","last_clean":"2026-02-24T22:34:42.721550+0000","last_became_active":"2026-02-24T22:05:18.190598+0000","last_became_peered":"2026-02-24T22:05:18.190598+0000","last_unstale":"2026-02-24T22:34:42.721550+0000","last_undegraded":"2026-02-24T22:34:42.721550+0000","last_fullsized":"2026-02-24T22:34:42.721550+0000","mapping_epoch":24,"log_start":"312'400","ondisk_log_start":"312'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'497","last_scrub_stamp":"2026-02-24T22:34:42.721473+0000","last_deep_scrub":"312'497","last_deep_scrub_stamp":"2026-02-24T22:34:42.721473+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.721473+0000","objects_scrubbed":27,"log_size":97,"log_dups_size":400,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:09:26.432917+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1905,"num_read_kb":1522,"num_write":772,"num_write_kb":389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":267,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"312'306","reported_seq":1024,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.065317+0000","last_change":"2026-02-24T22:34:41.065317+0000","last_active":"2026-02-24T22:34:41.065317+0000","last_peered":"2026-02-24T22:34:41.065317+0000","last_clean":"2026-02-24T22:34:41.065317+0000","last_became_active":"2026-02-24T22:05:16.191164+0000","last_became_peered":"2026-02-24T22:05:16.191164+0000","last_unstale":"2026-02-24T22:34:41.065317+0000","last_undegraded":"2026-02-24T22:34:41.065317+0000","last_fullsized":"2026-02-24T22:34:41.065317+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'306","last_scrub_stamp":"2026-02-24T22:34:41.065249+0000","last_deep_scrub":"312'306","last_deep_scrub_stamp":"2026-02-24T22:34:41.065249+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.065249+0000","objects_scrubbed":6,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:42:26.339639+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":491,"num_read_kb":5551,"num_write":1337,"num_write_kb":5240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"312'74","reported_seq":737,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:50.335113+0000","last_change":"2026-02-24T22:34:50.335113+0000","last_active":"2026-02-24T22:34:50.335113+0000","last_peered":"2026-02-24T22:34:50.335113+0000","last_clean":"2026-02-24T22:34:50.335113+0000","last_became_active":"2026-02-24T22:05:18.198559+0000","last_became_peered":"2026-02-24T22:05:18.198559+0000","last_unstale":"2026-02-24T22:34:50.335113+0000","last_undegraded":"2026-02-24T22:34:50.335113+0000","last_fullsized":"2026-02-24T22:34:50.335113+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'74","last_scrub_stamp":"2026-02-24T22:34:50.335079+0000","last_deep_scrub":"312'74","last_deep_scrub_stamp":"2026-02-24T22:34:50.335079+0000","last_clean_scrub_stamp":"2026-02-24T22:34:50.335079+0000","objects_scrubbed":18,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:21:37.995762+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":195,"num_read_kb":167,"num_write":70,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"312'304","reported_seq":1012,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.096430+0000","last_change":"2026-02-24T22:34:42.096430+0000","last_active":"2026-02-24T22:34:42.096430+0000","last_peered":"2026-02-24T22:34:42.096430+0000","last_clean":"2026-02-24T22:34:42.096430+0000","last_became_active":"2026-02-24T22:05:16.190367+0000","last_became_peered":"2026-02-24T22:05:16.190367+0000","last_unstale":"2026-02-24T22:34:42.096430+0000","last_undegraded":"2026-02-24T22:34:42.096430+0000","last_fullsized":"2026-02-24T22:34:42.096430+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'304","last_scrub_stamp":"2026-02-24T22:34:42.096375+0000","last_deep_scrub":"312'304","last_deep_scrub_stamp":"2026-02-24T22:34:42.096375+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.096375+0000","objects_scrubbed":4,"log_size":104,"log_dups_size":200,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:48:33.407036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":439,"num_read_kb":1335,"num_write":1233,"num_write_kb":1104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"312'237","reported_seq":988,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.882613+0000","last_change":"2026-02-24T22:34:48.882613+0000","last_active":"2026-02-24T22:34:48.882613+0000","last_peered":"2026-02-24T22:34:48.882613+0000","last_clean":"2026-02-24T22:34:48.882613+0000","last_became_active":"2026-02-24T22:05:18.191767+0000","last_became_peered":"2026-02-24T22:05:18.191767+0000","last_unstale":"2026-02-24T22:34:48.882613+0000","last_undegraded":"2026-02-24T22:34:48.882613+0000","last_fullsized":"2026-02-24T22:34:48.882613+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'237","last_scrub_stamp":"2026-02-24T22:34:48.882561+0000","last_deep_scrub":"312'237","last_deep_scrub_stamp":"2026-02-24T22:34:48.882561+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.882561+0000","objects_scrubbed":36,"log_size":37,"log_dups_size":200,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:24:34.587031+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":876,"num_read_kb":754,"num_write":323,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"312'364","reported_seq":1142,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.830513+0000","last_change":"2026-02-24T22:34:36.830513+0000","last_active":"2026-02-24T22:34:36.830513+0000","last_peered":"2026-02-24T22:34:36.830513+0000","last_clean":"2026-02-24T22:34:36.830513+0000","last_became_active":"2026-02-24T22:05:16.185779+0000","last_became_peered":"2026-02-24T22:05:16.185779+0000","last_unstale":"2026-02-24T22:34:36.830513+0000","last_undegraded":"2026-02-24T22:34:36.830513+0000","last_fullsized":"2026-02-24T22:34:36.830513+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'364","last_scrub_stamp":"2026-02-24T22:34:36.830446+0000","last_deep_scrub":"312'364","last_deep_scrub_stamp":"2026-02-24T22:34:36.830446+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.830446+0000","objects_scrubbed":6,"log_size":64,"log_dups_size":300,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:49:17.279372+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":619,"num_read_kb":883,"num_write":1617,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"312'109","reported_seq":765,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.090042+0000","last_change":"2026-02-24T22:34:41.090042+0000","last_active":"2026-02-24T22:34:41.090042+0000","last_peered":"2026-02-24T22:34:41.090042+0000","last_clean":"2026-02-24T22:34:41.090042+0000","last_became_active":"2026-02-24T22:05:18.187838+0000","last_became_peered":"2026-02-24T22:05:18.187838+0000","last_unstale":"2026-02-24T22:34:41.090042+0000","last_undegraded":"2026-02-24T22:34:41.090042+0000","last_fullsized":"2026-02-24T22:34:41.090042+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'109","last_scrub_stamp":"2026-02-24T22:34:41.089971+0000","last_deep_scrub":"312'109","last_deep_scrub_stamp":"2026-02-24T22:34:41.089971+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.089971+0000","objects_scrubbed":26,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:13:12.250060+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":263,"num_read_kb":217,"num_write":106,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":719,"num_omap_keys":3,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"312'164","reported_seq":846,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.431711+0000","last_change":"2026-02-24T22:34:47.431711+0000","last_active":"2026-02-24T22:34:47.431711+0000","last_peered":"2026-02-24T22:34:47.431711+0000","last_clean":"2026-02-24T22:34:47.431711+0000","last_became_active":"2026-02-24T22:05:18.191625+0000","last_became_peered":"2026-02-24T22:05:18.191625+0000","last_unstale":"2026-02-24T22:34:47.431711+0000","last_undegraded":"2026-02-24T22:34:47.431711+0000","last_fullsized":"2026-02-24T22:34:47.431711+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'164","last_scrub_stamp":"2026-02-24T22:34:47.431652+0000","last_deep_scrub":"312'164","last_deep_scrub_stamp":"2026-02-24T22:34:47.431652+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.431652+0000","objects_scrubbed":29,"log_size":64,"log_dups_size":100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:13:36.629505+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":477,"num_read_kb":396,"num_write":183,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"312'302","reported_seq":1047,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.046802+0000","last_change":"2026-02-24T22:34:41.046802+0000","last_active":"2026-02-24T22:34:41.046802+0000","last_peered":"2026-02-24T22:34:41.046802+0000","last_clean":"2026-02-24T22:34:41.046802+0000","last_became_active":"2026-02-24T22:06:01.025659+0000","last_became_peered":"2026-02-24T22:06:01.025659+0000","last_unstale":"2026-02-24T22:34:41.046802+0000","last_undegraded":"2026-02-24T22:34:41.046802+0000","last_fullsized":"2026-02-24T22:34:41.046802+0000","mapping_epoch":36,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'302","last_scrub_stamp":"2026-02-24T22:34:41.046752+0000","last_deep_scrub":"312'302","last_deep_scrub_stamp":"2026-02-24T22:34:41.046752+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.046752+0000","objects_scrubbed":2,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:23:49.712428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":514,"num_read_kb":16221,"num_write":1317,"num_write_kb":8500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"312'4268","reported_seq":5087,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.193741+0000","last_change":"2026-02-24T22:34:46.193741+0000","last_active":"2026-02-24T22:34:46.193741+0000","last_peered":"2026-02-24T22:34:46.193741+0000","last_clean":"2026-02-24T22:34:46.193741+0000","last_became_active":"2026-02-24T22:05:18.193257+0000","last_became_peered":"2026-02-24T22:05:18.193257+0000","last_unstale":"2026-02-24T22:34:46.193741+0000","last_undegraded":"2026-02-24T22:34:46.193741+0000","last_fullsized":"2026-02-24T22:34:46.193741+0000","mapping_epoch":24,"log_start":"312'4200","ondisk_log_start":"312'4200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'4268","last_scrub_stamp":"2026-02-24T22:34:46.193682+0000","last_deep_scrub":"312'4268","last_deep_scrub_stamp":"2026-02-24T22:34:46.193682+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.193682+0000","objects_scrubbed":19,"log_size":68,"log_dups_size":3000,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:22:49.449922+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":17332,"num_read_kb":14553,"num_write":6398,"num_write_kb":3198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"312'289","reported_seq":1009,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.725396+0000","last_change":"2026-02-24T22:34:37.725396+0000","last_active":"2026-02-24T22:34:37.725396+0000","last_peered":"2026-02-24T22:34:37.725396+0000","last_clean":"2026-02-24T22:34:37.725396+0000","last_became_active":"2026-02-24T22:05:16.190819+0000","last_became_peered":"2026-02-24T22:05:16.190819+0000","last_unstale":"2026-02-24T22:34:37.725396+0000","last_undegraded":"2026-02-24T22:34:37.725396+0000","last_fullsized":"2026-02-24T22:34:37.725396+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'289","last_scrub_stamp":"2026-02-24T22:34:37.725293+0000","last_deep_scrub":"312'289","last_deep_scrub_stamp":"2026-02-24T22:34:37.725293+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.725293+0000","objects_scrubbed":5,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:58:32.616383+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242922,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":466,"num_read_kb":8113,"num_write":1161,"num_write_kb":5510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"312'883","reported_seq":4054,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.966939+0000","last_change":"2026-02-24T22:34:45.966939+0000","last_active":"2026-02-24T22:34:45.966939+0000","last_peered":"2026-02-24T22:34:45.966939+0000","last_clean":"2026-02-24T22:34:45.966939+0000","last_became_active":"2026-02-24T22:05:18.190363+0000","last_became_peered":"2026-02-24T22:05:18.190363+0000","last_unstale":"2026-02-24T22:34:45.966939+0000","last_undegraded":"2026-02-24T22:34:45.966939+0000","last_fullsized":"2026-02-24T22:34:45.966939+0000","mapping_epoch":24,"log_start":"312'800","ondisk_log_start":"312'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'883","last_scrub_stamp":"2026-02-24T22:34:45.966870+0000","last_deep_scrub":"312'883","last_deep_scrub_stamp":"2026-02-24T22:34:45.966870+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.966870+0000","objects_scrubbed":21,"log_size":83,"log_dups_size":800,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:41:10.952477+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4747,"num_read_kb":5785,"num_write":398,"num_write_kb":236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"312'323","reported_seq":1062,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.725432+0000","last_change":"2026-02-24T22:34:35.725432+0000","last_active":"2026-02-24T22:34:35.725432+0000","last_peered":"2026-02-24T22:34:35.725432+0000","last_clean":"2026-02-24T22:34:35.725432+0000","last_became_active":"2026-02-24T22:05:16.190816+0000","last_became_peered":"2026-02-24T22:05:16.190816+0000","last_unstale":"2026-02-24T22:34:35.725432+0000","last_undegraded":"2026-02-24T22:34:35.725432+0000","last_fullsized":"2026-02-24T22:34:35.725432+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'323","last_scrub_stamp":"2026-02-24T22:34:35.725350+0000","last_deep_scrub":"312'323","last_deep_scrub_stamp":"2026-02-24T22:34:35.725350+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.725350+0000","objects_scrubbed":2,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:35:43.171065+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":544,"num_read_kb":2327,"num_write":1304,"num_write_kb":12235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"312'120","reported_seq":832,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.062558+0000","last_change":"2026-02-24T22:34:38.062558+0000","last_active":"2026-02-24T22:34:38.062558+0000","last_peered":"2026-02-24T22:34:38.062558+0000","last_clean":"2026-02-24T22:34:38.062558+0000","last_became_active":"2026-02-24T22:05:18.186994+0000","last_became_peered":"2026-02-24T22:05:18.186994+0000","last_unstale":"2026-02-24T22:34:38.062558+0000","last_undegraded":"2026-02-24T22:34:38.062558+0000","last_fullsized":"2026-02-24T22:34:38.062558+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'120","last_scrub_stamp":"2026-02-24T22:34:38.062523+0000","last_deep_scrub":"312'120","last_deep_scrub_stamp":"2026-02-24T22:34:38.062523+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.062523+0000","objects_scrubbed":27,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:14:19.882135+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":352,"num_read_kb":306,"num_write":116,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"312'386","reported_seq":1186,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.515887+0000","last_change":"2026-02-24T22:34:34.515887+0000","last_active":"2026-02-24T22:34:34.515887+0000","last_peered":"2026-02-24T22:34:34.515887+0000","last_clean":"2026-02-24T22:34:34.515887+0000","last_became_active":"2026-02-24T22:05:16.188658+0000","last_became_peered":"2026-02-24T22:05:16.188658+0000","last_unstale":"2026-02-24T22:34:34.515887+0000","last_undegraded":"2026-02-24T22:34:34.515887+0000","last_fullsized":"2026-02-24T22:34:34.515887+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'386","last_scrub_stamp":"2026-02-24T22:34:34.515812+0000","last_deep_scrub":"312'386","last_deep_scrub_stamp":"2026-02-24T22:34:34.515812+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.515812+0000","objects_scrubbed":6,"log_size":86,"log_dups_size":300,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:26:30.161641+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":840,"num_read_kb":685,"num_write":1819,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"312'198","reported_seq":929,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.065984+0000","last_change":"2026-02-24T22:34:40.065984+0000","last_active":"2026-02-24T22:34:40.065984+0000","last_peered":"2026-02-24T22:34:40.065984+0000","last_clean":"2026-02-24T22:34:40.065984+0000","last_became_active":"2026-02-24T22:05:18.187682+0000","last_became_peered":"2026-02-24T22:05:18.187682+0000","last_unstale":"2026-02-24T22:34:40.065984+0000","last_undegraded":"2026-02-24T22:34:40.065984+0000","last_fullsized":"2026-02-24T22:34:40.065984+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'198","last_scrub_stamp":"2026-02-24T22:34:40.065916+0000","last_deep_scrub":"312'198","last_deep_scrub_stamp":"2026-02-24T22:34:40.065916+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.065916+0000","objects_scrubbed":29,"log_size":98,"log_dups_size":100,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:30:25.873043+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":639,"num_read_kb":531,"num_write":250,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"312'296","reported_seq":1004,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.790665+0000","last_change":"2026-02-24T22:34:37.790665+0000","last_active":"2026-02-24T22:34:37.790665+0000","last_peered":"2026-02-24T22:34:37.790665+0000","last_clean":"2026-02-24T22:34:37.790665+0000","last_became_active":"2026-02-24T22:05:16.184209+0000","last_became_peered":"2026-02-24T22:05:16.184209+0000","last_unstale":"2026-02-24T22:34:37.790665+0000","last_undegraded":"2026-02-24T22:34:37.790665+0000","last_fullsized":"2026-02-24T22:34:37.790665+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'296","last_scrub_stamp":"2026-02-24T22:34:37.790602+0000","last_deep_scrub":"312'296","last_deep_scrub_stamp":"2026-02-24T22:34:37.790602+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.790602+0000","objects_scrubbed":9,"log_size":96,"log_dups_size":200,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:28:56.968188+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":416,"num_read_kb":603,"num_write":1232,"num_write_kb":10636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"312'131","reported_seq":818,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.372179+0000","last_change":"2026-02-24T22:34:49.372179+0000","last_active":"2026-02-24T22:34:49.372179+0000","last_peered":"2026-02-24T22:34:49.372179+0000","last_clean":"2026-02-24T22:34:49.372179+0000","last_became_active":"2026-02-24T22:05:18.193320+0000","last_became_peered":"2026-02-24T22:05:18.193320+0000","last_unstale":"2026-02-24T22:34:49.372179+0000","last_undegraded":"2026-02-24T22:34:49.372179+0000","last_fullsized":"2026-02-24T22:34:49.372179+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'131","last_scrub_stamp":"2026-02-24T22:34:49.372116+0000","last_deep_scrub":"312'131","last_deep_scrub_stamp":"2026-02-24T22:34:49.372116+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.372116+0000","objects_scrubbed":20,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:50:18.339691+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":425,"num_read_kb":355,"num_write":146,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"312'285","reported_seq":1006,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.747158+0000","last_change":"2026-02-24T22:34:37.747158+0000","last_active":"2026-02-24T22:34:37.747158+0000","last_peered":"2026-02-24T22:34:37.747158+0000","last_clean":"2026-02-24T22:34:37.747158+0000","last_became_active":"2026-02-24T22:05:16.181527+0000","last_became_peered":"2026-02-24T22:05:16.181527+0000","last_unstale":"2026-02-24T22:34:37.747158+0000","last_undegraded":"2026-02-24T22:34:37.747158+0000","last_fullsized":"2026-02-24T22:34:37.747158+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'285","last_scrub_stamp":"2026-02-24T22:34:37.747092+0000","last_deep_scrub":"312'285","last_deep_scrub_stamp":"2026-02-24T22:34:37.747092+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.747092+0000","objects_scrubbed":4,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:43:04.672418+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242895,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":474,"num_read_kb":5463,"num_write":1217,"num_write_kb":5189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":584,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:06.258262+0000","last_change":"2026-02-24T22:35:06.258262+0000","last_active":"2026-02-24T22:35:06.258262+0000","last_peered":"2026-02-24T22:35:06.258262+0000","last_clean":"2026-02-24T22:35:06.258262+0000","last_became_active":"2026-02-24T22:05:29.657764+0000","last_became_peered":"2026-02-24T22:05:29.657764+0000","last_unstale":"2026-02-24T22:35:06.258262+0000","last_undegraded":"2026-02-24T22:35:06.258262+0000","last_fullsized":"2026-02-24T22:35:06.258262+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:06.258189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:06.258189+0000","last_clean_scrub_stamp":"2026-02-24T22:35:06.258189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:18:02.619099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"312'107","reported_seq":770,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.757778+0000","last_change":"2026-02-24T22:34:42.757778+0000","last_active":"2026-02-24T22:34:42.757778+0000","last_peered":"2026-02-24T22:34:42.757778+0000","last_clean":"2026-02-24T22:34:42.757778+0000","last_became_active":"2026-02-24T22:05:18.184771+0000","last_became_peered":"2026-02-24T22:05:18.184771+0000","last_unstale":"2026-02-24T22:34:42.757778+0000","last_undegraded":"2026-02-24T22:34:42.757778+0000","last_fullsized":"2026-02-24T22:34:42.757778+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'107","last_scrub_stamp":"2026-02-24T22:34:42.757725+0000","last_deep_scrub":"312'107","last_deep_scrub_stamp":"2026-02-24T22:34:42.757725+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.757725+0000","objects_scrubbed":27,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:52:33.731233+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":246,"num_read_kb":208,"num_write":102,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"312'344","reported_seq":1133,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.109736+0000","last_change":"2026-02-24T22:34:37.109736+0000","last_active":"2026-02-24T22:34:37.109736+0000","last_peered":"2026-02-24T22:34:37.109736+0000","last_clean":"2026-02-24T22:34:37.109736+0000","last_became_active":"2026-02-24T22:05:16.191841+0000","last_became_peered":"2026-02-24T22:05:16.191841+0000","last_unstale":"2026-02-24T22:34:37.109736+0000","last_undegraded":"2026-02-24T22:34:37.109736+0000","last_fullsized":"2026-02-24T22:34:37.109736+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'344","last_scrub_stamp":"2026-02-24T22:34:37.109689+0000","last_deep_scrub":"312'344","last_deep_scrub_stamp":"2026-02-24T22:34:37.109689+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.109689+0000","objects_scrubbed":4,"log_size":44,"log_dups_size":300,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:35:53.603402+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":614,"num_read_kb":81372,"num_write":1329,"num_write_kb":20589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":587,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:50.867824+0000","last_change":"2026-02-24T22:34:50.867824+0000","last_active":"2026-02-24T22:34:50.867824+0000","last_peered":"2026-02-24T22:34:50.867824+0000","last_clean":"2026-02-24T22:34:50.867824+0000","last_became_active":"2026-02-24T22:05:20.194031+0000","last_became_peered":"2026-02-24T22:05:20.194031+0000","last_unstale":"2026-02-24T22:34:50.867824+0000","last_undegraded":"2026-02-24T22:34:50.867824+0000","last_fullsized":"2026-02-24T22:34:50.867824+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:50.867723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:50.867723+0000","last_clean_scrub_stamp":"2026-02-24T22:34:50.867723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:53:47.367923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"312'262","reported_seq":910,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:53.926384+0000","last_change":"2026-02-24T22:34:53.926384+0000","last_active":"2026-02-24T22:34:53.926384+0000","last_peered":"2026-02-24T22:34:53.926384+0000","last_clean":"2026-02-24T22:34:53.926384+0000","last_became_active":"2026-02-24T22:05:22.191928+0000","last_became_peered":"2026-02-24T22:05:22.191928+0000","last_unstale":"2026-02-24T22:34:53.926384+0000","last_undegraded":"2026-02-24T22:34:53.926384+0000","last_fullsized":"2026-02-24T22:34:53.926384+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'262","last_scrub_stamp":"2026-02-24T22:34:53.926319+0000","last_deep_scrub":"312'262","last_deep_scrub_stamp":"2026-02-24T22:34:53.926319+0000","last_clean_scrub_stamp":"2026-02-24T22:34:53.926319+0000","objects_scrubbed":4,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:38:02.939594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44919,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":413,"num_read_kb":914,"num_write":358,"num_write_kb":269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"312'258","reported_seq":861,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:02.251182+0000","last_change":"2026-02-24T22:35:02.251182+0000","last_active":"2026-02-24T22:35:02.251182+0000","last_peered":"2026-02-24T22:35:02.251182+0000","last_clean":"2026-02-24T22:35:02.251182+0000","last_became_active":"2026-02-24T22:05:24.204990+0000","last_became_peered":"2026-02-24T22:05:24.204990+0000","last_unstale":"2026-02-24T22:35:02.251182+0000","last_undegraded":"2026-02-24T22:35:02.251182+0000","last_fullsized":"2026-02-24T22:35:02.251182+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'258","last_scrub_stamp":"2026-02-24T22:35:02.251119+0000","last_deep_scrub":"312'258","last_deep_scrub_stamp":"2026-02-24T22:35:02.251119+0000","last_clean_scrub_stamp":"2026-02-24T22:35:02.251119+0000","objects_scrubbed":100,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:22:28.932537+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32373,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":92,"num_read_kb":87,"num_write":391,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":573,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:04.809580+0000","last_change":"2026-02-24T22:35:04.809580+0000","last_active":"2026-02-24T22:35:04.809580+0000","last_peered":"2026-02-24T22:35:04.809580+0000","last_clean":"2026-02-24T22:35:04.809580+0000","last_became_active":"2026-02-24T22:05:29.661747+0000","last_became_peered":"2026-02-24T22:05:29.661747+0000","last_unstale":"2026-02-24T22:35:04.809580+0000","last_undegraded":"2026-02-24T22:35:04.809580+0000","last_fullsized":"2026-02-24T22:35:04.809580+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:04.809519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:04.809519+0000","last_clean_scrub_stamp":"2026-02-24T22:35:04.809519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:29:02.534615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"312'104","reported_seq":784,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.994989+0000","last_change":"2026-02-24T22:34:42.994989+0000","last_active":"2026-02-24T22:34:42.994989+0000","last_peered":"2026-02-24T22:34:42.994989+0000","last_clean":"2026-02-24T22:34:42.994989+0000","last_became_active":"2026-02-24T22:05:18.191346+0000","last_became_peered":"2026-02-24T22:05:18.191346+0000","last_unstale":"2026-02-24T22:34:42.994989+0000","last_undegraded":"2026-02-24T22:34:42.994989+0000","last_fullsized":"2026-02-24T22:34:42.994989+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'104","last_scrub_stamp":"2026-02-24T22:34:42.994927+0000","last_deep_scrub":"312'104","last_deep_scrub_stamp":"2026-02-24T22:34:42.994927+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.994927+0000","objects_scrubbed":31,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:12:15.667131+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":237,"num_read_kb":209,"num_write":87,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"312'276","reported_seq":1004,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:30.666602+0000","last_change":"2026-02-24T22:34:30.666602+0000","last_active":"2026-02-24T22:34:30.666602+0000","last_peered":"2026-02-24T22:34:30.666602+0000","last_clean":"2026-02-24T22:34:30.666602+0000","last_became_active":"2026-02-24T22:05:16.183990+0000","last_became_peered":"2026-02-24T22:05:16.183990+0000","last_unstale":"2026-02-24T22:34:30.666602+0000","last_undegraded":"2026-02-24T22:34:30.666602+0000","last_fullsized":"2026-02-24T22:34:30.666602+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'276","last_scrub_stamp":"2026-02-24T22:34:30.666509+0000","last_deep_scrub":"312'276","last_deep_scrub_stamp":"2026-02-24T22:34:30.666509+0000","last_clean_scrub_stamp":"2026-02-24T22:34:30.666509+0000","objects_scrubbed":5,"log_size":76,"log_dups_size":200,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:23:41.986404+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485805,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":458,"num_read_kb":23693,"num_write":1270,"num_write_kb":23324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":598,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:51.749919+0000","last_change":"2026-02-24T22:34:51.749919+0000","last_active":"2026-02-24T22:34:51.749919+0000","last_peered":"2026-02-24T22:34:51.749919+0000","last_clean":"2026-02-24T22:34:51.749919+0000","last_became_active":"2026-02-24T22:05:20.186217+0000","last_became_peered":"2026-02-24T22:05:20.186217+0000","last_unstale":"2026-02-24T22:34:51.749919+0000","last_undegraded":"2026-02-24T22:34:51.749919+0000","last_fullsized":"2026-02-24T22:34:51.749919+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:51.749857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:51.749857+0000","last_clean_scrub_stamp":"2026-02-24T22:34:51.749857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:49:39.493746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"312'309","reported_seq":1164,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:55.272395+0000","last_change":"2026-02-24T22:34:55.272395+0000","last_active":"2026-02-24T22:34:55.272395+0000","last_peered":"2026-02-24T22:34:55.272395+0000","last_clean":"2026-02-24T22:34:55.272395+0000","last_became_active":"2026-02-24T22:05:22.196911+0000","last_became_peered":"2026-02-24T22:05:22.196911+0000","last_unstale":"2026-02-24T22:34:55.272395+0000","last_undegraded":"2026-02-24T22:34:55.272395+0000","last_fullsized":"2026-02-24T22:34:55.272395+0000","mapping_epoch":28,"log_start":"307'200","ondisk_log_start":"307'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'309","last_scrub_stamp":"2026-02-24T22:34:55.272282+0000","last_deep_scrub":"312'309","last_deep_scrub_stamp":"2026-02-24T22:34:55.272282+0000","last_clean_scrub_stamp":"2026-02-24T22:34:55.272282+0000","objects_scrubbed":4,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:53:27.073831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1016,"num_read_kb":540,"num_write":51,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"312'279","reported_seq":906,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:01.310102+0000","last_change":"2026-02-24T22:35:01.310102+0000","last_active":"2026-02-24T22:35:01.310102+0000","last_peered":"2026-02-24T22:35:01.310102+0000","last_clean":"2026-02-24T22:35:01.310102+0000","last_became_active":"2026-02-24T22:05:24.207692+0000","last_became_peered":"2026-02-24T22:05:24.207692+0000","last_unstale":"2026-02-24T22:35:01.310102+0000","last_undegraded":"2026-02-24T22:35:01.310102+0000","last_fullsized":"2026-02-24T22:35:01.310102+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'279","last_scrub_stamp":"2026-02-24T22:35:01.310078+0000","last_deep_scrub":"312'279","last_deep_scrub_stamp":"2026-02-24T22:35:01.310078+0000","last_clean_scrub_stamp":"2026-02-24T22:35:01.310078+0000","objects_scrubbed":92,"log_size":79,"log_dups_size":200,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:57:59.408543+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29978,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":173,"num_read_kb":171,"num_write":478,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":581,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:07.750436+0000","last_change":"2026-02-24T22:35:07.750436+0000","last_active":"2026-02-24T22:35:07.750436+0000","last_peered":"2026-02-24T22:35:07.750436+0000","last_clean":"2026-02-24T22:35:07.750436+0000","last_became_active":"2026-02-24T22:05:29.654288+0000","last_became_peered":"2026-02-24T22:05:29.654288+0000","last_unstale":"2026-02-24T22:35:07.750436+0000","last_undegraded":"2026-02-24T22:35:07.750436+0000","last_fullsized":"2026-02-24T22:35:07.750436+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:07.750356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:07.750356+0000","last_clean_scrub_stamp":"2026-02-24T22:35:07.750356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:03:20.092570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"312'298","reported_seq":1031,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.885878+0000","last_change":"2026-02-24T22:34:42.885878+0000","last_active":"2026-02-24T22:34:42.885878+0000","last_peered":"2026-02-24T22:34:42.885878+0000","last_clean":"2026-02-24T22:34:42.885878+0000","last_became_active":"2026-02-24T22:05:18.184542+0000","last_became_peered":"2026-02-24T22:05:18.184542+0000","last_unstale":"2026-02-24T22:34:42.885878+0000","last_undegraded":"2026-02-24T22:34:42.885878+0000","last_fullsized":"2026-02-24T22:34:42.885878+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'298","last_scrub_stamp":"2026-02-24T22:34:42.885832+0000","last_deep_scrub":"312'298","last_deep_scrub_stamp":"2026-02-24T22:34:42.885832+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.885832+0000","objects_scrubbed":24,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:07:57.036667+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1179,"num_read_kb":976,"num_write":460,"num_write_kb":243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"312'294","reported_seq":1016,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.140765+0000","last_change":"2026-02-24T22:34:35.140765+0000","last_active":"2026-02-24T22:34:35.140765+0000","last_peered":"2026-02-24T22:34:35.140765+0000","last_clean":"2026-02-24T22:34:35.140765+0000","last_became_active":"2026-02-24T22:05:16.190372+0000","last_became_peered":"2026-02-24T22:05:16.190372+0000","last_unstale":"2026-02-24T22:34:35.140765+0000","last_undegraded":"2026-02-24T22:34:35.140765+0000","last_fullsized":"2026-02-24T22:34:35.140765+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'294","last_scrub_stamp":"2026-02-24T22:34:35.140712+0000","last_deep_scrub":"312'294","last_deep_scrub_stamp":"2026-02-24T22:34:35.140712+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.140712+0000","objects_scrubbed":4,"log_size":94,"log_dups_size":200,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:24:43.777918+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":430,"num_read_kb":3398,"num_write":1175,"num_write_kb":524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":598,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:50.744612+0000","last_change":"2026-02-24T22:34:50.744612+0000","last_active":"2026-02-24T22:34:50.744612+0000","last_peered":"2026-02-24T22:34:50.744612+0000","last_clean":"2026-02-24T22:34:50.744612+0000","last_became_active":"2026-02-24T22:05:20.187327+0000","last_became_peered":"2026-02-24T22:05:20.187327+0000","last_unstale":"2026-02-24T22:34:50.744612+0000","last_undegraded":"2026-02-24T22:34:50.744612+0000","last_fullsized":"2026-02-24T22:34:50.744612+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:50.744552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:50.744552+0000","last_clean_scrub_stamp":"2026-02-24T22:34:50.744552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:05:37.833168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"312'802","reported_seq":2139,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.134134+0000","last_change":"2026-02-24T22:34:48.134134+0000","last_active":"2026-02-24T22:34:48.134134+0000","last_peered":"2026-02-24T22:34:48.134134+0000","last_clean":"2026-02-24T22:34:48.134134+0000","last_became_active":"2026-02-24T22:05:22.191073+0000","last_became_peered":"2026-02-24T22:05:22.191073+0000","last_unstale":"2026-02-24T22:34:48.134134+0000","last_undegraded":"2026-02-24T22:34:48.134134+0000","last_fullsized":"2026-02-24T22:34:48.134134+0000","mapping_epoch":28,"log_start":"307'700","ondisk_log_start":"307'700","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'802","last_scrub_stamp":"2026-02-24T22:34:48.134072+0000","last_deep_scrub":"312'802","last_deep_scrub_stamp":"2026-02-24T22:34:48.134072+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.134072+0000","objects_scrubbed":8,"log_size":102,"log_dups_size":700,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:20:11.846328+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":460,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2934,"num_read_kb":1505,"num_write":81,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"312'311","reported_seq":1838,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:00.230054+0000","last_change":"2026-02-24T22:35:00.230054+0000","last_active":"2026-02-24T22:35:00.230054+0000","last_peered":"2026-02-24T22:35:00.230054+0000","last_clean":"2026-02-24T22:35:00.230054+0000","last_became_active":"2026-02-24T22:05:24.206134+0000","last_became_peered":"2026-02-24T22:05:24.206134+0000","last_unstale":"2026-02-24T22:35:00.230054+0000","last_undegraded":"2026-02-24T22:35:00.230054+0000","last_fullsized":"2026-02-24T22:35:00.230054+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'311","last_scrub_stamp":"2026-02-24T22:35:00.229994+0000","last_deep_scrub":"312'311","last_deep_scrub_stamp":"2026-02-24T22:35:00.229994+0000","last_clean_scrub_stamp":"2026-02-24T22:35:00.229994+0000","objects_scrubbed":107,"log_size":11,"log_dups_size":300,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:46:37.708465+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33889,"num_objects":107,"num_object_clones":0,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":0,"num_read":1064,"num_read_kb":1063,"num_write":484,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":583,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:05.263294+0000","last_change":"2026-02-24T22:35:05.263294+0000","last_active":"2026-02-24T22:35:05.263294+0000","last_peered":"2026-02-24T22:35:05.263294+0000","last_clean":"2026-02-24T22:35:05.263294+0000","last_became_active":"2026-02-24T22:05:29.658119+0000","last_became_peered":"2026-02-24T22:05:29.658119+0000","last_unstale":"2026-02-24T22:35:05.263294+0000","last_undegraded":"2026-02-24T22:35:05.263294+0000","last_fullsized":"2026-02-24T22:35:05.263294+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:05.263228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:05.263228+0000","last_clean_scrub_stamp":"2026-02-24T22:35:05.263228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:33:28.280837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"312'690","reported_seq":1497,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.055164+0000","last_change":"2026-02-24T22:34:37.055105+0000","last_active":"2026-02-24T22:34:37.055164+0000","last_peered":"2026-02-24T22:34:37.055164+0000","last_clean":"2026-02-24T22:34:37.055164+0000","last_became_active":"2026-02-24T22:06:00.662609+0000","last_became_peered":"2026-02-24T22:06:00.662609+0000","last_unstale":"2026-02-24T22:34:37.055164+0000","last_undegraded":"2026-02-24T22:34:37.055164+0000","last_fullsized":"2026-02-24T22:34:37.055164+0000","mapping_epoch":36,"log_start":"312'600","ondisk_log_start":"312'600","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'690","last_scrub_stamp":"2026-02-24T22:34:37.055035+0000","last_deep_scrub":"312'690","last_deep_scrub_stamp":"2026-02-24T22:34:37.055035+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.055035+0000","objects_scrubbed":22,"log_size":90,"log_dups_size":600,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:44:46.603715+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":2886,"num_read_kb":2388,"num_write":1101,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"312'327","reported_seq":1061,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:30.811183+0000","last_change":"2026-02-24T22:34:30.811183+0000","last_active":"2026-02-24T22:34:30.811183+0000","last_peered":"2026-02-24T22:34:30.811183+0000","last_clean":"2026-02-24T22:34:30.811183+0000","last_became_active":"2026-02-24T22:05:16.189739+0000","last_became_peered":"2026-02-24T22:05:16.189739+0000","last_unstale":"2026-02-24T22:34:30.811183+0000","last_undegraded":"2026-02-24T22:34:30.811183+0000","last_fullsized":"2026-02-24T22:34:30.811183+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'327","last_scrub_stamp":"2026-02-24T22:34:30.811119+0000","last_deep_scrub":"312'327","last_deep_scrub_stamp":"2026-02-24T22:34:30.811119+0000","last_clean_scrub_stamp":"2026-02-24T22:34:30.811119+0000","objects_scrubbed":9,"log_size":27,"log_dups_size":300,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:44:34.182914+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485786,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":468,"num_read_kb":16928,"num_write":1377,"num_write_kb":17766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":587,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:51.907157+0000","last_change":"2026-02-24T22:34:51.907157+0000","last_active":"2026-02-24T22:34:51.907157+0000","last_peered":"2026-02-24T22:34:51.907157+0000","last_clean":"2026-02-24T22:34:51.907157+0000","last_became_active":"2026-02-24T22:05:20.193200+0000","last_became_peered":"2026-02-24T22:05:20.193200+0000","last_unstale":"2026-02-24T22:34:51.907157+0000","last_undegraded":"2026-02-24T22:34:51.907157+0000","last_fullsized":"2026-02-24T22:34:51.907157+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:51.907045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:51.907045+0000","last_clean_scrub_stamp":"2026-02-24T22:34:51.907045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:41:57.779249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"312'196","reported_seq":807,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:57.208914+0000","last_change":"2026-02-24T22:34:57.208914+0000","last_active":"2026-02-24T22:34:57.208914+0000","last_peered":"2026-02-24T22:34:57.208914+0000","last_clean":"2026-02-24T22:34:57.208914+0000","last_became_active":"2026-02-24T22:05:22.912826+0000","last_became_peered":"2026-02-24T22:05:22.912826+0000","last_unstale":"2026-02-24T22:34:57.208914+0000","last_undegraded":"2026-02-24T22:34:57.208914+0000","last_fullsized":"2026-02-24T22:34:57.208914+0000","mapping_epoch":28,"log_start":"312'100","ondisk_log_start":"312'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'196","last_scrub_stamp":"2026-02-24T22:34:57.208826+0000","last_deep_scrub":"312'196","last_deep_scrub_stamp":"2026-02-24T22:34:57.208826+0000","last_clean_scrub_stamp":"2026-02-24T22:34:57.208826+0000","objects_scrubbed":4,"log_size":96,"log_dups_size":100,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:42:57.794071+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37493,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":308,"num_read_kb":794,"num_write":284,"num_write_kb":257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"312'352","reported_seq":1012,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:06.646610+0000","last_change":"2026-02-24T22:35:06.646610+0000","last_active":"2026-02-24T22:35:06.646610+0000","last_peered":"2026-02-24T22:35:06.646610+0000","last_clean":"2026-02-24T22:35:06.646610+0000","last_became_active":"2026-02-24T22:05:24.198964+0000","last_became_peered":"2026-02-24T22:05:24.198964+0000","last_unstale":"2026-02-24T22:35:06.646610+0000","last_undegraded":"2026-02-24T22:35:06.646610+0000","last_fullsized":"2026-02-24T22:35:06.646610+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'352","last_scrub_stamp":"2026-02-24T22:35:06.646550+0000","last_deep_scrub":"312'352","last_deep_scrub_stamp":"2026-02-24T22:35:06.646550+0000","last_clean_scrub_stamp":"2026-02-24T22:35:06.646550+0000","objects_scrubbed":95,"log_size":52,"log_dups_size":300,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:44:35.894779+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29433,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":428,"num_read_kb":365,"num_write":612,"num_write_kb":187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"312'140","reported_seq":875,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.037417+0000","last_change":"2026-02-24T22:34:42.037417+0000","last_active":"2026-02-24T22:34:42.037417+0000","last_peered":"2026-02-24T22:34:42.037417+0000","last_clean":"2026-02-24T22:34:42.037417+0000","last_became_active":"2026-02-24T22:05:18.189796+0000","last_became_peered":"2026-02-24T22:05:18.189796+0000","last_unstale":"2026-02-24T22:34:42.037417+0000","last_undegraded":"2026-02-24T22:34:42.037417+0000","last_fullsized":"2026-02-24T22:34:42.037417+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'140","last_scrub_stamp":"2026-02-24T22:34:42.037385+0000","last_deep_scrub":"312'140","last_deep_scrub_stamp":"2026-02-24T22:34:42.037385+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.037385+0000","objects_scrubbed":27,"log_size":40,"log_dups_size":100,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:45:54.858292+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":455,"num_read_kb":399,"num_write":145,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"312'263","reported_seq":951,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.085973+0000","last_change":"2026-02-24T22:34:38.085935+0000","last_active":"2026-02-24T22:34:38.085973+0000","last_peered":"2026-02-24T22:34:38.085973+0000","last_clean":"2026-02-24T22:34:38.085973+0000","last_became_active":"2026-02-24T22:05:16.191985+0000","last_became_peered":"2026-02-24T22:05:16.191985+0000","last_unstale":"2026-02-24T22:34:38.085973+0000","last_undegraded":"2026-02-24T22:34:38.085973+0000","last_fullsized":"2026-02-24T22:34:38.085973+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'263","last_scrub_stamp":"2026-02-24T22:34:38.085876+0000","last_deep_scrub":"312'263","last_deep_scrub_stamp":"2026-02-24T22:34:38.085876+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.085876+0000","objects_scrubbed":7,"log_size":63,"log_dups_size":200,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:00:57.241743+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":82,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":355,"num_read_kb":4498,"num_write":1041,"num_write_kb":4311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":594,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:50.153717+0000","last_change":"2026-02-24T22:34:50.153717+0000","last_active":"2026-02-24T22:34:50.153717+0000","last_peered":"2026-02-24T22:34:50.153717+0000","last_clean":"2026-02-24T22:34:50.153717+0000","last_became_active":"2026-02-24T22:05:20.194288+0000","last_became_peered":"2026-02-24T22:05:20.194288+0000","last_unstale":"2026-02-24T22:34:50.153717+0000","last_undegraded":"2026-02-24T22:34:50.153717+0000","last_fullsized":"2026-02-24T22:34:50.153717+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:50.153652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:50.153652+0000","last_clean_scrub_stamp":"2026-02-24T22:34:50.153652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:34:10.280880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"312'269","reported_seq":906,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:55.965175+0000","last_change":"2026-02-24T22:34:55.965175+0000","last_active":"2026-02-24T22:34:55.965175+0000","last_peered":"2026-02-24T22:34:55.965175+0000","last_clean":"2026-02-24T22:34:55.965175+0000","last_became_active":"2026-02-24T22:05:22.912859+0000","last_became_peered":"2026-02-24T22:05:22.912859+0000","last_unstale":"2026-02-24T22:34:55.965175+0000","last_undegraded":"2026-02-24T22:34:55.965175+0000","last_fullsized":"2026-02-24T22:34:55.965175+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'269","last_scrub_stamp":"2026-02-24T22:34:55.965113+0000","last_deep_scrub":"312'269","last_deep_scrub_stamp":"2026-02-24T22:34:55.965113+0000","last_clean_scrub_stamp":"2026-02-24T22:34:55.965113+0000","objects_scrubbed":9,"log_size":69,"log_dups_size":200,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:00:50.874867+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38223,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":418,"num_read_kb":936,"num_write":362,"num_write_kb":271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"312'408","reported_seq":1118,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:52.254254+0000","last_change":"2026-02-24T22:34:52.254254+0000","last_active":"2026-02-24T22:34:52.254254+0000","last_peered":"2026-02-24T22:34:52.254254+0000","last_clean":"2026-02-24T22:34:52.254254+0000","last_became_active":"2026-02-24T22:05:24.199772+0000","last_became_peered":"2026-02-24T22:05:24.199772+0000","last_unstale":"2026-02-24T22:34:52.254254+0000","last_undegraded":"2026-02-24T22:34:52.254254+0000","last_fullsized":"2026-02-24T22:34:52.254254+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'408","last_scrub_stamp":"2026-02-24T22:34:52.254200+0000","last_deep_scrub":"312'408","last_deep_scrub_stamp":"2026-02-24T22:34:52.254200+0000","last_clean_scrub_stamp":"2026-02-24T22:34:52.254200+0000","objects_scrubbed":93,"log_size":108,"log_dups_size":300,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:02:43.014695+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30266,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":473,"num_read_kb":498,"num_write":701,"num_write_kb":232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1193,"num_omap_keys":7,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:02.856103+0000","last_change":"2026-02-24T22:35:02.856103+0000","last_active":"2026-02-24T22:35:02.856103+0000","last_peered":"2026-02-24T22:35:02.856103+0000","last_clean":"2026-02-24T22:35:02.856103+0000","last_became_active":"2026-02-24T22:05:29.661401+0000","last_became_peered":"2026-02-24T22:05:29.661401+0000","last_unstale":"2026-02-24T22:35:02.856103+0000","last_undegraded":"2026-02-24T22:35:02.856103+0000","last_fullsized":"2026-02-24T22:35:02.856103+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:02.856048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:02.856048+0000","last_clean_scrub_stamp":"2026-02-24T22:35:02.856048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:27:49.594219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"312'103","reported_seq":773,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.478042+0000","last_change":"2026-02-24T22:34:35.478042+0000","last_active":"2026-02-24T22:34:35.478042+0000","last_peered":"2026-02-24T22:34:35.478042+0000","last_clean":"2026-02-24T22:34:35.478042+0000","last_became_active":"2026-02-24T22:05:18.193188+0000","last_became_peered":"2026-02-24T22:05:18.193188+0000","last_unstale":"2026-02-24T22:34:35.478042+0000","last_undegraded":"2026-02-24T22:34:35.478042+0000","last_fullsized":"2026-02-24T22:34:35.478042+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'103","last_scrub_stamp":"2026-02-24T22:34:35.477982+0000","last_deep_scrub":"312'103","last_deep_scrub_stamp":"2026-02-24T22:34:35.477982+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.477982+0000","objects_scrubbed":19,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:46:51.105644+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":312,"num_read_kb":260,"num_write":126,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"312'324","reported_seq":1064,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.151701+0000","last_change":"2026-02-24T22:34:33.151665+0000","last_active":"2026-02-24T22:34:33.151701+0000","last_peered":"2026-02-24T22:34:33.151701+0000","last_clean":"2026-02-24T22:34:33.151701+0000","last_became_active":"2026-02-24T22:05:16.191956+0000","last_became_peered":"2026-02-24T22:05:16.191956+0000","last_unstale":"2026-02-24T22:34:33.151701+0000","last_undegraded":"2026-02-24T22:34:33.151701+0000","last_fullsized":"2026-02-24T22:34:33.151701+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'324","last_scrub_stamp":"2026-02-24T22:34:33.151603+0000","last_deep_scrub":"312'324","last_deep_scrub_stamp":"2026-02-24T22:34:33.151603+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.151603+0000","objects_scrubbed":3,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:52:36.974066+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485778,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":523,"num_read_kb":10657,"num_write":1420,"num_write_kb":15460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":602,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.120807+0000","last_change":"2026-02-24T22:34:46.120807+0000","last_active":"2026-02-24T22:34:46.120807+0000","last_peered":"2026-02-24T22:34:46.120807+0000","last_clean":"2026-02-24T22:34:46.120807+0000","last_became_active":"2026-02-24T22:05:20.186055+0000","last_became_peered":"2026-02-24T22:05:20.186055+0000","last_unstale":"2026-02-24T22:34:46.120807+0000","last_undegraded":"2026-02-24T22:34:46.120807+0000","last_fullsized":"2026-02-24T22:34:46.120807+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:46.120737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:46.120737+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.120737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:33:04.392007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"312'232","reported_seq":849,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:53.782364+0000","last_change":"2026-02-24T22:34:53.782364+0000","last_active":"2026-02-24T22:34:53.782364+0000","last_peered":"2026-02-24T22:34:53.782364+0000","last_clean":"2026-02-24T22:34:53.782364+0000","last_became_active":"2026-02-24T22:05:22.193741+0000","last_became_peered":"2026-02-24T22:05:22.193741+0000","last_unstale":"2026-02-24T22:34:53.782364+0000","last_undegraded":"2026-02-24T22:34:53.782364+0000","last_fullsized":"2026-02-24T22:34:53.782364+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'232","last_scrub_stamp":"2026-02-24T22:34:53.782300+0000","last_deep_scrub":"312'232","last_deep_scrub_stamp":"2026-02-24T22:34:53.782300+0000","last_clean_scrub_stamp":"2026-02-24T22:34:53.782300+0000","objects_scrubbed":4,"log_size":32,"log_dups_size":200,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:22:35.430551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39811,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":385,"num_read_kb":1028,"num_write":360,"num_write_kb":328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"312'276","reported_seq":914,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:01.567396+0000","last_change":"2026-02-24T22:35:01.567396+0000","last_active":"2026-02-24T22:35:01.567396+0000","last_peered":"2026-02-24T22:35:01.567396+0000","last_clean":"2026-02-24T22:35:01.567396+0000","last_became_active":"2026-02-24T22:05:24.198731+0000","last_became_peered":"2026-02-24T22:05:24.198731+0000","last_unstale":"2026-02-24T22:35:01.567396+0000","last_undegraded":"2026-02-24T22:35:01.567396+0000","last_fullsized":"2026-02-24T22:35:01.567396+0000","mapping_epoch":30,"log_start":"312'199","ondisk_log_start":"312'199","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'276","last_scrub_stamp":"2026-02-24T22:35:01.567361+0000","last_deep_scrub":"312'276","last_deep_scrub_stamp":"2026-02-24T22:35:01.567361+0000","last_clean_scrub_stamp":"2026-02-24T22:35:01.567361+0000","objects_scrubbed":91,"log_size":77,"log_dups_size":199,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:34:24.939354+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30034,"num_objects":91,"num_object_clones":0,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":241,"num_read_kb":207,"num_write":470,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":579,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:52.546054+0000","last_change":"2026-02-24T22:34:52.546054+0000","last_active":"2026-02-24T22:34:52.546054+0000","last_peered":"2026-02-24T22:34:52.546054+0000","last_clean":"2026-02-24T22:34:52.546054+0000","last_became_active":"2026-02-24T22:05:29.661161+0000","last_became_peered":"2026-02-24T22:05:29.661161+0000","last_unstale":"2026-02-24T22:34:52.546054+0000","last_undegraded":"2026-02-24T22:34:52.546054+0000","last_fullsized":"2026-02-24T22:34:52.546054+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:52.545999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:52.545999+0000","last_clean_scrub_stamp":"2026-02-24T22:34:52.545999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:03:26.142316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"312'129","reported_seq":784,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.703983+0000","last_change":"2026-02-24T22:34:39.703983+0000","last_active":"2026-02-24T22:34:39.703983+0000","last_peered":"2026-02-24T22:34:39.703983+0000","last_clean":"2026-02-24T22:34:39.703983+0000","last_became_active":"2026-02-24T22:05:18.191555+0000","last_became_peered":"2026-02-24T22:05:18.191555+0000","last_unstale":"2026-02-24T22:34:39.703983+0000","last_undegraded":"2026-02-24T22:34:39.703983+0000","last_fullsized":"2026-02-24T22:34:39.703983+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'129","last_scrub_stamp":"2026-02-24T22:34:39.703930+0000","last_deep_scrub":"312'129","last_deep_scrub_stamp":"2026-02-24T22:34:39.703930+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.703930+0000","objects_scrubbed":23,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:20:21.850877+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":387,"num_read_kb":325,"num_write":162,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"312'289","reported_seq":1011,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.159670+0000","last_change":"2026-02-24T22:34:34.159624+0000","last_active":"2026-02-24T22:34:34.159670+0000","last_peered":"2026-02-24T22:34:34.159670+0000","last_clean":"2026-02-24T22:34:34.159670+0000","last_became_active":"2026-02-24T22:05:16.191733+0000","last_became_peered":"2026-02-24T22:05:16.191733+0000","last_unstale":"2026-02-24T22:34:34.159670+0000","last_undegraded":"2026-02-24T22:34:34.159670+0000","last_fullsized":"2026-02-24T22:34:34.159670+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'289","last_scrub_stamp":"2026-02-24T22:34:34.159566+0000","last_deep_scrub":"312'289","last_deep_scrub_stamp":"2026-02-24T22:34:34.159566+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.159566+0000","objects_scrubbed":4,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:18:06.498072+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242906,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":398,"num_read_kb":31022,"num_write":1049,"num_write_kb":10319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":602,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.063841+0000","last_change":"2026-02-24T22:34:44.063841+0000","last_active":"2026-02-24T22:34:44.063841+0000","last_peered":"2026-02-24T22:34:44.063841+0000","last_clean":"2026-02-24T22:34:44.063841+0000","last_became_active":"2026-02-24T22:05:20.186892+0000","last_became_peered":"2026-02-24T22:05:20.186892+0000","last_unstale":"2026-02-24T22:34:44.063841+0000","last_undegraded":"2026-02-24T22:34:44.063841+0000","last_fullsized":"2026-02-24T22:34:44.063841+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:44.063776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:44.063776+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.063776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:34:17.807478+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"312'245","reported_seq":866,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.174484+0000","last_change":"2026-02-24T22:34:58.174484+0000","last_active":"2026-02-24T22:34:58.174484+0000","last_peered":"2026-02-24T22:34:58.174484+0000","last_clean":"2026-02-24T22:34:58.174484+0000","last_became_active":"2026-02-24T22:05:22.200208+0000","last_became_peered":"2026-02-24T22:05:22.200208+0000","last_unstale":"2026-02-24T22:34:58.174484+0000","last_undegraded":"2026-02-24T22:34:58.174484+0000","last_fullsized":"2026-02-24T22:34:58.174484+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'245","last_scrub_stamp":"2026-02-24T22:34:58.174378+0000","last_deep_scrub":"312'245","last_deep_scrub_stamp":"2026-02-24T22:34:58.174378+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.174378+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":200,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:10:15.543051+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39435,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":390,"num_read_kb":991,"num_write":356,"num_write_kb":310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"312'378","reported_seq":1201,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.906602+0000","last_change":"2026-02-24T22:34:58.906602+0000","last_active":"2026-02-24T22:34:58.906602+0000","last_peered":"2026-02-24T22:34:58.906602+0000","last_clean":"2026-02-24T22:34:58.906602+0000","last_became_active":"2026-02-24T22:05:24.199525+0000","last_became_peered":"2026-02-24T22:05:24.199525+0000","last_unstale":"2026-02-24T22:34:58.906602+0000","last_undegraded":"2026-02-24T22:34:58.906602+0000","last_fullsized":"2026-02-24T22:34:58.906602+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'378","last_scrub_stamp":"2026-02-24T22:34:58.906557+0000","last_deep_scrub":"312'378","last_deep_scrub_stamp":"2026-02-24T22:34:58.906557+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.906557+0000","objects_scrubbed":103,"log_size":78,"log_dups_size":300,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:42:33.226800+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32009,"num_objects":103,"num_object_clones":0,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":506,"num_read_kb":490,"num_write":606,"num_write_kb":207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":583,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:03.724387+0000","last_change":"2026-02-24T22:35:03.724387+0000","last_active":"2026-02-24T22:35:03.724387+0000","last_peered":"2026-02-24T22:35:03.724387+0000","last_clean":"2026-02-24T22:35:03.724387+0000","last_became_active":"2026-02-24T22:05:29.652598+0000","last_became_peered":"2026-02-24T22:05:29.652598+0000","last_unstale":"2026-02-24T22:35:03.724387+0000","last_undegraded":"2026-02-24T22:35:03.724387+0000","last_fullsized":"2026-02-24T22:35:03.724387+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:03.724333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:03.724333+0000","last_clean_scrub_stamp":"2026-02-24T22:35:03.724333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:16:00.583629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"312'189","reported_seq":900,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.811102+0000","last_change":"2026-02-24T22:34:40.811102+0000","last_active":"2026-02-24T22:34:40.811102+0000","last_peered":"2026-02-24T22:34:40.811102+0000","last_clean":"2026-02-24T22:34:40.811102+0000","last_became_active":"2026-02-24T22:05:18.185152+0000","last_became_peered":"2026-02-24T22:05:18.185152+0000","last_unstale":"2026-02-24T22:34:40.811102+0000","last_undegraded":"2026-02-24T22:34:40.811102+0000","last_fullsized":"2026-02-24T22:34:40.811102+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'189","last_scrub_stamp":"2026-02-24T22:34:40.811068+0000","last_deep_scrub":"312'189","last_deep_scrub_stamp":"2026-02-24T22:34:40.811068+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.811068+0000","objects_scrubbed":20,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:24:59.774125+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":651,"num_read_kb":540,"num_write":239,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"312'323","reported_seq":1050,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.061642+0000","last_change":"2026-02-24T22:34:33.061574+0000","last_active":"2026-02-24T22:34:33.061642+0000","last_peered":"2026-02-24T22:34:33.061642+0000","last_clean":"2026-02-24T22:34:33.061642+0000","last_became_active":"2026-02-24T22:05:16.190205+0000","last_became_peered":"2026-02-24T22:05:16.190205+0000","last_unstale":"2026-02-24T22:34:33.061642+0000","last_undegraded":"2026-02-24T22:34:33.061642+0000","last_fullsized":"2026-02-24T22:34:33.061642+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'323","last_scrub_stamp":"2026-02-24T22:34:33.061497+0000","last_deep_scrub":"312'323","last_deep_scrub_stamp":"2026-02-24T22:34:33.061497+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.061497+0000","objects_scrubbed":1,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:54:15.633089+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":578,"num_read_kb":20088,"num_write":1296,"num_write_kb":14513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":602,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.105525+0000","last_change":"2026-02-24T22:34:45.105525+0000","last_active":"2026-02-24T22:34:45.105525+0000","last_peered":"2026-02-24T22:34:45.105525+0000","last_clean":"2026-02-24T22:34:45.105525+0000","last_became_active":"2026-02-24T22:05:20.186781+0000","last_became_peered":"2026-02-24T22:05:20.186781+0000","last_unstale":"2026-02-24T22:34:45.105525+0000","last_undegraded":"2026-02-24T22:34:45.105525+0000","last_fullsized":"2026-02-24T22:34:45.105525+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:45.105463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:45.105463+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.105463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:39:37.239923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"312'404","reported_seq":1032,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:51.933082+0000","last_change":"2026-02-24T22:34:51.933082+0000","last_active":"2026-02-24T22:34:51.933082+0000","last_peered":"2026-02-24T22:34:51.933082+0000","last_clean":"2026-02-24T22:34:51.933082+0000","last_became_active":"2026-02-24T22:05:22.192495+0000","last_became_peered":"2026-02-24T22:05:22.192495+0000","last_unstale":"2026-02-24T22:34:51.933082+0000","last_undegraded":"2026-02-24T22:34:51.933082+0000","last_fullsized":"2026-02-24T22:34:51.933082+0000","mapping_epoch":28,"log_start":"312'300","ondisk_log_start":"312'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'404","last_scrub_stamp":"2026-02-24T22:34:51.933036+0000","last_deep_scrub":"312'404","last_deep_scrub_stamp":"2026-02-24T22:34:51.933036+0000","last_clean_scrub_stamp":"2026-02-24T22:34:51.933036+0000","objects_scrubbed":5,"log_size":104,"log_dups_size":300,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:38:02.998627+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":81867,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":699,"num_read_kb":1918,"num_write":658,"num_write_kb":626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"312'264","reported_seq":930,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.182990+0000","last_change":"2026-02-24T22:34:49.182990+0000","last_active":"2026-02-24T22:34:49.182990+0000","last_peered":"2026-02-24T22:34:49.182990+0000","last_clean":"2026-02-24T22:34:49.182990+0000","last_became_active":"2026-02-24T22:05:24.198114+0000","last_became_peered":"2026-02-24T22:05:24.198114+0000","last_unstale":"2026-02-24T22:34:49.182990+0000","last_undegraded":"2026-02-24T22:34:49.182990+0000","last_fullsized":"2026-02-24T22:34:49.182990+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'264","last_scrub_stamp":"2026-02-24T22:34:49.182961+0000","last_deep_scrub":"312'264","last_deep_scrub_stamp":"2026-02-24T22:34:49.182961+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.182961+0000","objects_scrubbed":91,"log_size":64,"log_dups_size":200,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:07:14.157330+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29501,"num_objects":91,"num_object_clones":0,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":235,"num_read_kb":226,"num_write":429,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":583,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:57.325797+0000","last_change":"2026-02-24T22:34:57.325797+0000","last_active":"2026-02-24T22:34:57.325797+0000","last_peered":"2026-02-24T22:34:57.325797+0000","last_clean":"2026-02-24T22:34:57.325797+0000","last_became_active":"2026-02-24T22:05:29.652715+0000","last_became_peered":"2026-02-24T22:05:29.652715+0000","last_unstale":"2026-02-24T22:34:57.325797+0000","last_undegraded":"2026-02-24T22:34:57.325797+0000","last_fullsized":"2026-02-24T22:34:57.325797+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:57.325730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:57.325730+0000","last_clean_scrub_stamp":"2026-02-24T22:34:57.325730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:53:52.255388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"312'596","reported_seq":1465,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.442591+0000","last_change":"2026-02-24T22:34:37.442591+0000","last_active":"2026-02-24T22:34:37.442591+0000","last_peered":"2026-02-24T22:34:37.442591+0000","last_clean":"2026-02-24T22:34:37.442591+0000","last_became_active":"2026-02-24T22:05:18.192543+0000","last_became_peered":"2026-02-24T22:05:18.192543+0000","last_unstale":"2026-02-24T22:34:37.442591+0000","last_undegraded":"2026-02-24T22:34:37.442591+0000","last_fullsized":"2026-02-24T22:34:37.442591+0000","mapping_epoch":24,"log_start":"312'500","ondisk_log_start":"312'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'596","last_scrub_stamp":"2026-02-24T22:34:37.442512+0000","last_deep_scrub":"312'596","last_deep_scrub_stamp":"2026-02-24T22:34:37.442512+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.442512+0000","objects_scrubbed":25,"log_size":96,"log_dups_size":500,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:07:01.563341+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":2576,"num_read_kb":2171,"num_write":966,"num_write_kb":601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"312'260","reported_seq":943,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.858767+0000","last_change":"2026-02-24T22:34:32.858731+0000","last_active":"2026-02-24T22:34:32.858767+0000","last_peered":"2026-02-24T22:34:32.858767+0000","last_clean":"2026-02-24T22:34:32.858767+0000","last_became_active":"2026-02-24T22:05:16.182928+0000","last_became_peered":"2026-02-24T22:05:16.182928+0000","last_unstale":"2026-02-24T22:34:32.858767+0000","last_undegraded":"2026-02-24T22:34:32.858767+0000","last_fullsized":"2026-02-24T22:34:32.858767+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'260","last_scrub_stamp":"2026-02-24T22:34:32.858687+0000","last_deep_scrub":"312'260","last_deep_scrub_stamp":"2026-02-24T22:34:32.858687+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.858687+0000","objects_scrubbed":5,"log_size":60,"log_dups_size":200,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:47:38.099528+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":316,"num_read_kb":262,"num_write":1083,"num_write_kb":181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":602,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.083526+0000","last_change":"2026-02-24T22:34:43.083526+0000","last_active":"2026-02-24T22:34:43.083526+0000","last_peered":"2026-02-24T22:34:43.083526+0000","last_clean":"2026-02-24T22:34:43.083526+0000","last_became_active":"2026-02-24T22:05:20.186209+0000","last_became_peered":"2026-02-24T22:05:20.186209+0000","last_unstale":"2026-02-24T22:34:43.083526+0000","last_undegraded":"2026-02-24T22:34:43.083526+0000","last_fullsized":"2026-02-24T22:34:43.083526+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:43.083470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:43.083470+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.083470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:53:12.978030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"312'956","reported_seq":2285,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:53.254053+0000","last_change":"2026-02-24T22:34:53.254053+0000","last_active":"2026-02-24T22:34:53.254053+0000","last_peered":"2026-02-24T22:34:53.254053+0000","last_clean":"2026-02-24T22:34:53.254053+0000","last_became_active":"2026-02-24T22:05:22.200111+0000","last_became_peered":"2026-02-24T22:05:22.200111+0000","last_unstale":"2026-02-24T22:34:53.254053+0000","last_undegraded":"2026-02-24T22:34:53.254053+0000","last_fullsized":"2026-02-24T22:34:53.254053+0000","mapping_epoch":28,"log_start":"312'900","ondisk_log_start":"312'900","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'956","last_scrub_stamp":"2026-02-24T22:34:53.253998+0000","last_deep_scrub":"312'956","last_deep_scrub_stamp":"2026-02-24T22:34:53.253998+0000","last_clean_scrub_stamp":"2026-02-24T22:34:53.253998+0000","objects_scrubbed":9,"log_size":56,"log_dups_size":900,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:23:04.182706+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37978,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":3232,"num_read_kb":2382,"num_write":419,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15964,"num_omap_keys":38,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"312'284","reported_seq":891,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:59.904243+0000","last_change":"2026-02-24T22:34:59.904243+0000","last_active":"2026-02-24T22:34:59.904243+0000","last_peered":"2026-02-24T22:34:59.904243+0000","last_clean":"2026-02-24T22:34:59.904243+0000","last_became_active":"2026-02-24T22:05:24.200878+0000","last_became_peered":"2026-02-24T22:05:24.200878+0000","last_unstale":"2026-02-24T22:34:59.904243+0000","last_undegraded":"2026-02-24T22:34:59.904243+0000","last_fullsized":"2026-02-24T22:34:59.904243+0000","mapping_epoch":30,"log_start":"312'197","ondisk_log_start":"312'197","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'284","last_scrub_stamp":"2026-02-24T22:34:59.904208+0000","last_deep_scrub":"312'284","last_deep_scrub_stamp":"2026-02-24T22:34:59.904208+0000","last_clean_scrub_stamp":"2026-02-24T22:34:59.904208+0000","objects_scrubbed":101,"log_size":87,"log_dups_size":197,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:49:36.597249+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33021,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":110,"num_read_kb":109,"num_write":447,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"312'440","reported_seq":2209,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:03.546764+0000","last_change":"2026-02-24T22:35:03.546764+0000","last_active":"2026-02-24T22:35:03.546764+0000","last_peered":"2026-02-24T22:35:03.546764+0000","last_clean":"2026-02-24T22:35:03.546764+0000","last_became_active":"2026-02-24T22:05:29.654064+0000","last_became_peered":"2026-02-24T22:05:29.654064+0000","last_unstale":"2026-02-24T22:35:03.546764+0000","last_undegraded":"2026-02-24T22:35:03.546764+0000","last_fullsized":"2026-02-24T22:35:03.546764+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'440","last_scrub_stamp":"2026-02-24T22:35:03.546700+0000","last_deep_scrub":"312'440","last_deep_scrub_stamp":"2026-02-24T22:35:03.546700+0000","last_clean_scrub_stamp":"2026-02-24T22:35:03.546700+0000","objects_scrubbed":1,"log_size":40,"log_dups_size":400,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:16:29.192139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"312'149","reported_seq":841,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.467232+0000","last_change":"2026-02-24T22:34:38.467232+0000","last_active":"2026-02-24T22:34:38.467232+0000","last_peered":"2026-02-24T22:34:38.467232+0000","last_clean":"2026-02-24T22:34:38.467232+0000","last_became_active":"2026-02-24T22:05:18.191055+0000","last_became_peered":"2026-02-24T22:05:18.191055+0000","last_unstale":"2026-02-24T22:34:38.467232+0000","last_undegraded":"2026-02-24T22:34:38.467232+0000","last_fullsized":"2026-02-24T22:34:38.467232+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'149","last_scrub_stamp":"2026-02-24T22:34:38.467178+0000","last_deep_scrub":"312'149","last_deep_scrub_stamp":"2026-02-24T22:34:38.467178+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.467178+0000","objects_scrubbed":21,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:18:47.528959+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":486,"num_read_kb":398,"num_write":170,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"312'241","reported_seq":956,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.746504+0000","last_change":"2026-02-24T22:34:32.746504+0000","last_active":"2026-02-24T22:34:32.746504+0000","last_peered":"2026-02-24T22:34:32.746504+0000","last_clean":"2026-02-24T22:34:32.746504+0000","last_became_active":"2026-02-24T22:05:16.187261+0000","last_became_peered":"2026-02-24T22:05:16.187261+0000","last_unstale":"2026-02-24T22:34:32.746504+0000","last_undegraded":"2026-02-24T22:34:32.746504+0000","last_fullsized":"2026-02-24T22:34:32.746504+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'241","last_scrub_stamp":"2026-02-24T22:34:32.746460+0000","last_deep_scrub":"312'241","last_deep_scrub_stamp":"2026-02-24T22:34:32.746460+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.746460+0000","objects_scrubbed":1,"log_size":41,"log_dups_size":200,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:05:01.484992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":370,"num_read_kb":24764,"num_write":957,"num_write_kb":10562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":597,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.218231+0000","last_change":"2026-02-24T22:34:47.218231+0000","last_active":"2026-02-24T22:34:47.218231+0000","last_peered":"2026-02-24T22:34:47.218231+0000","last_clean":"2026-02-24T22:34:47.218231+0000","last_became_active":"2026-02-24T22:05:20.187488+0000","last_became_peered":"2026-02-24T22:05:20.187488+0000","last_unstale":"2026-02-24T22:34:47.218231+0000","last_undegraded":"2026-02-24T22:34:47.218231+0000","last_fullsized":"2026-02-24T22:34:47.218231+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:47.218164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:47.218164+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.218164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:23:53.051333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"312'475","reported_seq":1340,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.137231+0000","last_change":"2026-02-24T22:34:47.137231+0000","last_active":"2026-02-24T22:34:47.137231+0000","last_peered":"2026-02-24T22:34:47.137231+0000","last_clean":"2026-02-24T22:34:47.137231+0000","last_became_active":"2026-02-24T22:05:22.193142+0000","last_became_peered":"2026-02-24T22:05:22.193142+0000","last_unstale":"2026-02-24T22:34:47.137231+0000","last_undegraded":"2026-02-24T22:34:47.137231+0000","last_fullsized":"2026-02-24T22:34:47.137231+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'475","last_scrub_stamp":"2026-02-24T22:34:47.137162+0000","last_deep_scrub":"312'475","last_deep_scrub_stamp":"2026-02-24T22:34:47.137162+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.137162+0000","objects_scrubbed":8,"log_size":75,"log_dups_size":400,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:12:15.570963+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38313,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1308,"num_read_kb":1368,"num_write":335,"num_write_kb":281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"312'312","reported_seq":926,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:50.216629+0000","last_change":"2026-02-24T22:34:50.216629+0000","last_active":"2026-02-24T22:34:50.216629+0000","last_peered":"2026-02-24T22:34:50.216629+0000","last_clean":"2026-02-24T22:34:50.216629+0000","last_became_active":"2026-02-24T22:05:24.198669+0000","last_became_peered":"2026-02-24T22:05:24.198669+0000","last_unstale":"2026-02-24T22:34:50.216629+0000","last_undegraded":"2026-02-24T22:34:50.216629+0000","last_fullsized":"2026-02-24T22:34:50.216629+0000","mapping_epoch":30,"log_start":"312'296","ondisk_log_start":"312'296","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'312","last_scrub_stamp":"2026-02-24T22:34:50.216584+0000","last_deep_scrub":"312'312","last_deep_scrub_stamp":"2026-02-24T22:34:50.216584+0000","last_clean_scrub_stamp":"2026-02-24T22:34:50.216584+0000","objects_scrubbed":90,"log_size":16,"log_dups_size":296,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:29:12.268470+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29328,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":170,"num_read_kb":165,"num_write":551,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":584,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:56.292294+0000","last_change":"2026-02-24T22:34:56.292294+0000","last_active":"2026-02-24T22:34:56.292294+0000","last_peered":"2026-02-24T22:34:56.292294+0000","last_clean":"2026-02-24T22:34:56.292294+0000","last_became_active":"2026-02-24T22:05:29.653144+0000","last_became_peered":"2026-02-24T22:05:29.653144+0000","last_unstale":"2026-02-24T22:34:56.292294+0000","last_undegraded":"2026-02-24T22:34:56.292294+0000","last_fullsized":"2026-02-24T22:34:56.292294+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:56.292236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:56.292236+0000","last_clean_scrub_stamp":"2026-02-24T22:34:56.292236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:28:09.419838+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"312'194","reported_seq":908,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.515320+0000","last_change":"2026-02-24T22:34:39.515320+0000","last_active":"2026-02-24T22:34:39.515320+0000","last_peered":"2026-02-24T22:34:39.515320+0000","last_clean":"2026-02-24T22:34:39.515320+0000","last_became_active":"2026-02-24T22:05:18.190140+0000","last_became_peered":"2026-02-24T22:05:18.190140+0000","last_unstale":"2026-02-24T22:34:39.515320+0000","last_undegraded":"2026-02-24T22:34:39.515320+0000","last_fullsized":"2026-02-24T22:34:39.515320+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'194","last_scrub_stamp":"2026-02-24T22:34:39.515258+0000","last_deep_scrub":"312'194","last_deep_scrub_stamp":"2026-02-24T22:34:39.515258+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.515258+0000","objects_scrubbed":23,"log_size":94,"log_dups_size":100,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:49:24.504968+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":675,"num_read_kb":556,"num_write":238,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"312'303","reported_seq":1072,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.577662+0000","last_change":"2026-02-24T22:34:32.577662+0000","last_active":"2026-02-24T22:34:32.577662+0000","last_peered":"2026-02-24T22:34:32.577662+0000","last_clean":"2026-02-24T22:34:32.577662+0000","last_became_active":"2026-02-24T22:05:16.184944+0000","last_became_peered":"2026-02-24T22:05:16.184944+0000","last_unstale":"2026-02-24T22:34:32.577662+0000","last_undegraded":"2026-02-24T22:34:32.577662+0000","last_fullsized":"2026-02-24T22:34:32.577662+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'303","last_scrub_stamp":"2026-02-24T22:34:32.577606+0000","last_deep_scrub":"312'303","last_deep_scrub_stamp":"2026-02-24T22:34:32.577606+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.577606+0000","objects_scrubbed":3,"log_size":103,"log_dups_size":200,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:59:47.981464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":538,"num_read_kb":45888,"num_write":1354,"num_write_kb":20430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"312'4","reported_seq":1068,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:53.903762+0000","last_change":"2026-02-24T22:34:53.903762+0000","last_active":"2026-02-24T22:34:53.903762+0000","last_peered":"2026-02-24T22:34:53.903762+0000","last_clean":"2026-02-24T22:34:53.903762+0000","last_became_active":"2026-02-24T22:05:20.194145+0000","last_became_peered":"2026-02-24T22:05:20.194145+0000","last_unstale":"2026-02-24T22:34:53.903762+0000","last_undegraded":"2026-02-24T22:34:53.903762+0000","last_fullsized":"2026-02-24T22:34:53.903762+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'4","last_scrub_stamp":"2026-02-24T22:34:53.903710+0000","last_deep_scrub":"312'4","last_deep_scrub_stamp":"2026-02-24T22:34:53.903710+0000","last_clean_scrub_stamp":"2026-02-24T22:34:53.903710+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:03:04.394145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":476,"num_read_kb":476,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"312'61","reported_seq":671,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:54.742682+0000","last_change":"2026-02-24T22:34:54.742682+0000","last_active":"2026-02-24T22:34:54.742682+0000","last_peered":"2026-02-24T22:34:54.742682+0000","last_clean":"2026-02-24T22:34:54.742682+0000","last_became_active":"2026-02-24T22:05:22.905221+0000","last_became_peered":"2026-02-24T22:05:22.905221+0000","last_unstale":"2026-02-24T22:34:54.742682+0000","last_undegraded":"2026-02-24T22:34:54.742682+0000","last_fullsized":"2026-02-24T22:34:54.742682+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'61","last_scrub_stamp":"2026-02-24T22:34:54.742603+0000","last_deep_scrub":"312'61","last_deep_scrub_stamp":"2026-02-24T22:34:54.742603+0000","last_clean_scrub_stamp":"2026-02-24T22:34:54.742603+0000","objects_scrubbed":5,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:52:44.639141+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"312'246","reported_seq":845,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.685022+0000","last_change":"2026-02-24T22:34:46.685022+0000","last_active":"2026-02-24T22:34:46.685022+0000","last_peered":"2026-02-24T22:34:46.685022+0000","last_clean":"2026-02-24T22:34:46.685022+0000","last_became_active":"2026-02-24T22:05:24.207673+0000","last_became_peered":"2026-02-24T22:05:24.207673+0000","last_unstale":"2026-02-24T22:34:46.685022+0000","last_undegraded":"2026-02-24T22:34:46.685022+0000","last_fullsized":"2026-02-24T22:34:46.685022+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'246","last_scrub_stamp":"2026-02-24T22:34:46.684962+0000","last_deep_scrub":"312'246","last_deep_scrub_stamp":"2026-02-24T22:34:46.684962+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.684962+0000","objects_scrubbed":87,"log_size":46,"log_dups_size":200,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:17:30.101644+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25987,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":107,"num_read_kb":103,"num_write":388,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":586,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:03.269486+0000","last_change":"2026-02-24T22:35:03.269486+0000","last_active":"2026-02-24T22:35:03.269486+0000","last_peered":"2026-02-24T22:35:03.269486+0000","last_clean":"2026-02-24T22:35:03.269486+0000","last_became_active":"2026-02-24T22:05:29.658842+0000","last_became_peered":"2026-02-24T22:05:29.658842+0000","last_unstale":"2026-02-24T22:35:03.269486+0000","last_undegraded":"2026-02-24T22:35:03.269486+0000","last_fullsized":"2026-02-24T22:35:03.269486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:03.269430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:03.269430+0000","last_clean_scrub_stamp":"2026-02-24T22:35:03.269430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:37:15.587437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"312'122","reported_seq":812,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.133092+0000","last_change":"2026-02-24T22:34:44.133092+0000","last_active":"2026-02-24T22:34:44.133092+0000","last_peered":"2026-02-24T22:34:44.133092+0000","last_clean":"2026-02-24T22:34:44.133092+0000","last_became_active":"2026-02-24T22:05:18.188161+0000","last_became_peered":"2026-02-24T22:05:18.188161+0000","last_unstale":"2026-02-24T22:34:44.133092+0000","last_undegraded":"2026-02-24T22:34:44.133092+0000","last_fullsized":"2026-02-24T22:34:44.133092+0000","mapping_epoch":24,"log_start":"312'98","ondisk_log_start":"312'98","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'122","last_scrub_stamp":"2026-02-24T22:34:44.133036+0000","last_deep_scrub":"312'122","last_deep_scrub_stamp":"2026-02-24T22:34:44.133036+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.133036+0000","objects_scrubbed":24,"log_size":24,"log_dups_size":98,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:15:18.743316+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":368,"num_read_kb":314,"num_write":134,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"312'360","reported_seq":1100,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.843071+0000","last_change":"2026-02-24T22:34:31.843071+0000","last_active":"2026-02-24T22:34:31.843071+0000","last_peered":"2026-02-24T22:34:31.843071+0000","last_clean":"2026-02-24T22:34:31.843071+0000","last_became_active":"2026-02-24T22:05:16.182194+0000","last_became_peered":"2026-02-24T22:05:16.182194+0000","last_unstale":"2026-02-24T22:34:31.843071+0000","last_undegraded":"2026-02-24T22:34:31.843071+0000","last_fullsized":"2026-02-24T22:34:31.843071+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'360","last_scrub_stamp":"2026-02-24T22:34:31.843001+0000","last_deep_scrub":"312'360","last_deep_scrub_stamp":"2026-02-24T22:34:31.843001+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.843001+0000","objects_scrubbed":4,"log_size":60,"log_dups_size":300,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:54:58.411257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":482,"num_read_kb":7457,"num_write":1288,"num_write_kb":8933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":599,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:52.297621+0000","last_change":"2026-02-24T22:34:52.297621+0000","last_active":"2026-02-24T22:34:52.297621+0000","last_peered":"2026-02-24T22:34:52.297621+0000","last_clean":"2026-02-24T22:34:52.297621+0000","last_became_active":"2026-02-24T22:05:20.193180+0000","last_became_peered":"2026-02-24T22:05:20.193180+0000","last_unstale":"2026-02-24T22:34:52.297621+0000","last_undegraded":"2026-02-24T22:34:52.297621+0000","last_fullsized":"2026-02-24T22:34:52.297621+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:52.297555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:52.297555+0000","last_clean_scrub_stamp":"2026-02-24T22:34:52.297555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:59:44.363996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"312'284","reported_seq":918,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:56.996699+0000","last_change":"2026-02-24T22:34:56.996699+0000","last_active":"2026-02-24T22:34:56.996699+0000","last_peered":"2026-02-24T22:34:56.996699+0000","last_clean":"2026-02-24T22:34:56.996699+0000","last_became_active":"2026-02-24T22:05:22.200138+0000","last_became_peered":"2026-02-24T22:05:22.200138+0000","last_unstale":"2026-02-24T22:34:56.996699+0000","last_undegraded":"2026-02-24T22:34:56.996699+0000","last_fullsized":"2026-02-24T22:34:56.996699+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'284","last_scrub_stamp":"2026-02-24T22:34:56.996649+0000","last_deep_scrub":"312'284","last_deep_scrub_stamp":"2026-02-24T22:34:56.996649+0000","last_clean_scrub_stamp":"2026-02-24T22:34:56.996649+0000","objects_scrubbed":9,"log_size":84,"log_dups_size":200,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:57:19.406987+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37516,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":871,"num_read_kb":1412,"num_write":1276,"num_write_kb":1198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":357372,"num_omap_keys":834,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"312'415","reported_seq":1053,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.648295+0000","last_change":"2026-02-24T22:34:47.648295+0000","last_active":"2026-02-24T22:34:47.648295+0000","last_peered":"2026-02-24T22:34:47.648295+0000","last_clean":"2026-02-24T22:34:47.648295+0000","last_became_active":"2026-02-24T22:05:24.206745+0000","last_became_peered":"2026-02-24T22:05:24.206745+0000","last_unstale":"2026-02-24T22:34:47.648295+0000","last_undegraded":"2026-02-24T22:34:47.648295+0000","last_fullsized":"2026-02-24T22:34:47.648295+0000","mapping_epoch":30,"log_start":"312'400","ondisk_log_start":"312'400","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'415","last_scrub_stamp":"2026-02-24T22:34:47.648223+0000","last_deep_scrub":"312'415","last_deep_scrub_stamp":"2026-02-24T22:34:47.648223+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.648223+0000","objects_scrubbed":88,"log_size":15,"log_dups_size":400,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:44:43.640744+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27098,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":391,"num_read_kb":376,"num_write":726,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":178,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"312'436","reported_seq":4809,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.362595+0000","last_change":"2026-02-24T22:34:58.362595+0000","last_active":"2026-02-24T22:34:58.362595+0000","last_peered":"2026-02-24T22:34:58.362595+0000","last_clean":"2026-02-24T22:34:58.362595+0000","last_became_active":"2026-02-24T22:05:29.654281+0000","last_became_peered":"2026-02-24T22:05:29.654281+0000","last_unstale":"2026-02-24T22:34:58.362595+0000","last_undegraded":"2026-02-24T22:34:58.362595+0000","last_fullsized":"2026-02-24T22:34:58.362595+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'436","last_scrub_stamp":"2026-02-24T22:34:58.362387+0000","last_deep_scrub":"312'436","last_deep_scrub_stamp":"2026-02-24T22:34:58.362387+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.362387+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":400,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:59:54.222522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"312'136","reported_seq":831,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.733879+0000","last_change":"2026-02-24T22:34:38.733879+0000","last_active":"2026-02-24T22:34:38.733879+0000","last_peered":"2026-02-24T22:34:38.733879+0000","last_clean":"2026-02-24T22:34:38.733879+0000","last_became_active":"2026-02-24T22:05:18.193426+0000","last_became_peered":"2026-02-24T22:05:18.193426+0000","last_unstale":"2026-02-24T22:34:38.733879+0000","last_undegraded":"2026-02-24T22:34:38.733879+0000","last_fullsized":"2026-02-24T22:34:38.733879+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'136","last_scrub_stamp":"2026-02-24T22:34:38.733817+0000","last_deep_scrub":"312'136","last_deep_scrub_stamp":"2026-02-24T22:34:38.733817+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.733817+0000","objects_scrubbed":22,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:36:52.810608+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":443,"num_read_kb":376,"num_write":199,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"312'114","reported_seq":755,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.176106+0000","last_change":"2026-02-24T22:34:32.176106+0000","last_active":"2026-02-24T22:34:32.176106+0000","last_peered":"2026-02-24T22:34:32.176106+0000","last_clean":"2026-02-24T22:34:32.176106+0000","last_became_active":"2026-02-24T22:05:06.708577+0000","last_became_peered":"2026-02-24T22:05:06.708577+0000","last_unstale":"2026-02-24T22:34:32.176106+0000","last_undegraded":"2026-02-24T22:34:32.176106+0000","last_fullsized":"2026-02-24T22:34:32.176106+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'114","last_scrub_stamp":"2026-02-24T22:34:32.176027+0000","last_deep_scrub":"312'114","last_deep_scrub_stamp":"2026-02-24T22:34:32.176027+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.176027+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:58:53.542668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"312'2","reported_seq":614,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.096943+0000","last_change":"2026-02-24T22:34:34.096943+0000","last_active":"2026-02-24T22:34:34.096943+0000","last_peered":"2026-02-24T22:34:34.096943+0000","last_clean":"2026-02-24T22:34:34.096943+0000","last_became_active":"2026-02-24T22:05:08.689983+0000","last_became_peered":"2026-02-24T22:05:08.689983+0000","last_unstale":"2026-02-24T22:34:34.096943+0000","last_undegraded":"2026-02-24T22:34:34.096943+0000","last_fullsized":"2026-02-24T22:34:34.096943+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:34:34.096879+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:34:34.096879+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.096879+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:03:29.099395+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00046251899999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"312'329","reported_seq":1116,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:29.895013+0000","last_change":"2026-02-24T22:34:29.895013+0000","last_active":"2026-02-24T22:34:29.895013+0000","last_peered":"2026-02-24T22:34:29.895013+0000","last_clean":"2026-02-24T22:34:29.895013+0000","last_became_active":"2026-02-24T22:05:16.185768+0000","last_became_peered":"2026-02-24T22:05:16.185768+0000","last_unstale":"2026-02-24T22:34:29.895013+0000","last_undegraded":"2026-02-24T22:34:29.895013+0000","last_fullsized":"2026-02-24T22:34:29.895013+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'329","last_scrub_stamp":"2026-02-24T22:34:29.894945+0000","last_deep_scrub":"312'329","last_deep_scrub_stamp":"2026-02-24T22:34:29.894945+0000","last_clean_scrub_stamp":"2026-02-24T22:34:29.894945+0000","objects_scrubbed":3,"log_size":29,"log_dups_size":300,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:02:39.272423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":685,"num_read_kb":12844,"num_write":1503,"num_write_kb":6232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":595,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.125003+0000","last_change":"2026-02-24T22:34:49.125003+0000","last_active":"2026-02-24T22:34:49.125003+0000","last_peered":"2026-02-24T22:34:49.125003+0000","last_clean":"2026-02-24T22:34:49.125003+0000","last_became_active":"2026-02-24T22:05:20.194040+0000","last_became_peered":"2026-02-24T22:05:20.194040+0000","last_unstale":"2026-02-24T22:34:49.125003+0000","last_undegraded":"2026-02-24T22:34:49.125003+0000","last_fullsized":"2026-02-24T22:34:49.125003+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:49.124939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:49.124939+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.124939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:16:47.321676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"312'1071","reported_seq":2188,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:54.936135+0000","last_change":"2026-02-24T22:34:54.936135+0000","last_active":"2026-02-24T22:34:54.936135+0000","last_peered":"2026-02-24T22:34:54.936135+0000","last_clean":"2026-02-24T22:34:54.936135+0000","last_became_active":"2026-02-24T22:05:22.199602+0000","last_became_peered":"2026-02-24T22:05:22.199602+0000","last_unstale":"2026-02-24T22:34:54.936135+0000","last_undegraded":"2026-02-24T22:34:54.936135+0000","last_fullsized":"2026-02-24T22:34:54.936135+0000","mapping_epoch":28,"log_start":"312'1000","ondisk_log_start":"312'1000","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'1071","last_scrub_stamp":"2026-02-24T22:34:54.936052+0000","last_deep_scrub":"312'1071","last_deep_scrub_stamp":"2026-02-24T22:34:54.936052+0000","last_clean_scrub_stamp":"2026-02-24T22:34:54.936052+0000","objects_scrubbed":8,"log_size":71,"log_dups_size":1000,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:42:38.805146+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":116028,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2941,"num_read_kb":3705,"num_write":1026,"num_write_kb":950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17602,"num_omap_keys":42,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"312'302","reported_seq":909,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:57.954594+0000","last_change":"2026-02-24T22:34:57.954594+0000","last_active":"2026-02-24T22:34:57.954594+0000","last_peered":"2026-02-24T22:34:57.954594+0000","last_clean":"2026-02-24T22:34:57.954594+0000","last_became_active":"2026-02-24T22:05:24.200336+0000","last_became_peered":"2026-02-24T22:05:24.200336+0000","last_unstale":"2026-02-24T22:34:57.954594+0000","last_undegraded":"2026-02-24T22:34:57.954594+0000","last_fullsized":"2026-02-24T22:34:57.954594+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'302","last_scrub_stamp":"2026-02-24T22:34:57.954521+0000","last_deep_scrub":"312'302","last_deep_scrub_stamp":"2026-02-24T22:34:57.954521+0000","last_clean_scrub_stamp":"2026-02-24T22:34:57.954521+0000","objects_scrubbed":100,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:53:46.560141+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32535,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":152,"num_read_kb":139,"num_write":501,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"312'428","reported_seq":1844,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:54.584117+0000","last_change":"2026-02-24T22:34:54.584117+0000","last_active":"2026-02-24T22:34:54.584117+0000","last_peered":"2026-02-24T22:34:54.584117+0000","last_clean":"2026-02-24T22:34:54.584117+0000","last_became_active":"2026-02-24T22:05:29.660370+0000","last_became_peered":"2026-02-24T22:05:29.660370+0000","last_unstale":"2026-02-24T22:34:54.584117+0000","last_undegraded":"2026-02-24T22:34:54.584117+0000","last_fullsized":"2026-02-24T22:34:54.584117+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'428","last_scrub_stamp":"2026-02-24T22:34:54.584009+0000","last_deep_scrub":"312'428","last_deep_scrub_stamp":"2026-02-24T22:34:54.584009+0000","last_clean_scrub_stamp":"2026-02-24T22:34:54.584009+0000","objects_scrubbed":1,"log_size":28,"log_dups_size":400,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:22:23.409209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"312'124","reported_seq":831,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.850193+0000","last_change":"2026-02-24T22:34:41.850193+0000","last_active":"2026-02-24T22:34:41.850193+0000","last_peered":"2026-02-24T22:34:41.850193+0000","last_clean":"2026-02-24T22:34:41.850193+0000","last_became_active":"2026-02-24T22:05:18.185205+0000","last_became_peered":"2026-02-24T22:05:18.185205+0000","last_unstale":"2026-02-24T22:34:41.850193+0000","last_undegraded":"2026-02-24T22:34:41.850193+0000","last_fullsized":"2026-02-24T22:34:41.850193+0000","mapping_epoch":24,"log_start":"312'99","ondisk_log_start":"312'99","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'124","last_scrub_stamp":"2026-02-24T22:34:41.850136+0000","last_deep_scrub":"312'124","last_deep_scrub_stamp":"2026-02-24T22:34:41.850136+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.850136+0000","objects_scrubbed":25,"log_size":25,"log_dups_size":99,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:53:33.276473+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":346,"num_read_kb":300,"num_write":140,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":618,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.150171+0000","last_change":"2026-02-24T22:34:36.150171+0000","last_active":"2026-02-24T22:34:36.150171+0000","last_peered":"2026-02-24T22:34:36.150171+0000","last_clean":"2026-02-24T22:34:36.150171+0000","last_became_active":"2026-02-24T22:05:08.689972+0000","last_became_peered":"2026-02-24T22:05:08.689972+0000","last_unstale":"2026-02-24T22:34:36.150171+0000","last_undegraded":"2026-02-24T22:34:36.150171+0000","last_fullsized":"2026-02-24T22:34:36.150171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:36.150114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:36.150114+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.150114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:06:01.715910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048345700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"312'262","reported_seq":1029,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:29.701991+0000","last_change":"2026-02-24T22:34:29.701952+0000","last_active":"2026-02-24T22:34:29.701991+0000","last_peered":"2026-02-24T22:34:29.701991+0000","last_clean":"2026-02-24T22:34:29.701991+0000","last_became_active":"2026-02-24T22:05:16.183484+0000","last_became_peered":"2026-02-24T22:05:16.183484+0000","last_unstale":"2026-02-24T22:34:29.701991+0000","last_undegraded":"2026-02-24T22:34:29.701991+0000","last_fullsized":"2026-02-24T22:34:29.701991+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'262","last_scrub_stamp":"2026-02-24T22:34:29.701911+0000","last_deep_scrub":"312'262","last_deep_scrub_stamp":"2026-02-24T22:34:29.701911+0000","last_clean_scrub_stamp":"2026-02-24T22:34:29.701911+0000","objects_scrubbed":4,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:16:54.158907+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":574,"num_read_kb":821,"num_write":1287,"num_write_kb":10723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":588,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.883509+0000","last_change":"2026-02-24T22:34:49.883509+0000","last_active":"2026-02-24T22:34:49.883509+0000","last_peered":"2026-02-24T22:34:49.883509+0000","last_clean":"2026-02-24T22:34:49.883509+0000","last_became_active":"2026-02-24T22:05:20.195236+0000","last_became_peered":"2026-02-24T22:05:20.195236+0000","last_unstale":"2026-02-24T22:34:49.883509+0000","last_undegraded":"2026-02-24T22:34:49.883509+0000","last_fullsized":"2026-02-24T22:34:49.883509+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:49.883446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:49.883446+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.883446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:59:34.535514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"312'1070","reported_seq":2301,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:51.232440+0000","last_change":"2026-02-24T22:34:51.232440+0000","last_active":"2026-02-24T22:34:51.232440+0000","last_peered":"2026-02-24T22:34:51.232440+0000","last_clean":"2026-02-24T22:34:51.232440+0000","last_became_active":"2026-02-24T22:05:22.192295+0000","last_became_peered":"2026-02-24T22:05:22.192295+0000","last_unstale":"2026-02-24T22:34:51.232440+0000","last_undegraded":"2026-02-24T22:34:51.232440+0000","last_fullsized":"2026-02-24T22:34:51.232440+0000","mapping_epoch":28,"log_start":"312'1001","ondisk_log_start":"312'1001","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'1070","last_scrub_stamp":"2026-02-24T22:34:51.232390+0000","last_deep_scrub":"312'1070","last_deep_scrub_stamp":"2026-02-24T22:34:51.232390+0000","last_clean_scrub_stamp":"2026-02-24T22:34:51.232390+0000","objects_scrubbed":12,"log_size":69,"log_dups_size":1001,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:26:43.381792+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":114655,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":3192,"num_read_kb":4794,"num_write":3958,"num_write_kb":3891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":517874,"num_omap_keys":3022,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"312'271","reported_seq":866,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:01.238162+0000","last_change":"2026-02-24T22:35:01.238162+0000","last_active":"2026-02-24T22:35:01.238162+0000","last_peered":"2026-02-24T22:35:01.238162+0000","last_clean":"2026-02-24T22:35:01.238162+0000","last_became_active":"2026-02-24T22:05:24.204774+0000","last_became_peered":"2026-02-24T22:05:24.204774+0000","last_unstale":"2026-02-24T22:35:01.238162+0000","last_undegraded":"2026-02-24T22:35:01.238162+0000","last_fullsized":"2026-02-24T22:35:01.238162+0000","mapping_epoch":30,"log_start":"312'196","ondisk_log_start":"312'196","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'271","last_scrub_stamp":"2026-02-24T22:35:01.238098+0000","last_deep_scrub":"312'271","last_deep_scrub_stamp":"2026-02-24T22:35:01.238098+0000","last_clean_scrub_stamp":"2026-02-24T22:35:01.238098+0000","objects_scrubbed":89,"log_size":75,"log_dups_size":196,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:08:24.544989+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29814,"num_objects":89,"num_object_clones":0,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":0,"num_read":112,"num_read_kb":108,"num_write":454,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":583,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:59.339684+0000","last_change":"2026-02-24T22:34:59.339684+0000","last_active":"2026-02-24T22:34:59.339684+0000","last_peered":"2026-02-24T22:34:59.339684+0000","last_clean":"2026-02-24T22:34:59.339684+0000","last_became_active":"2026-02-24T22:05:29.653391+0000","last_became_peered":"2026-02-24T22:05:29.653391+0000","last_unstale":"2026-02-24T22:34:59.339684+0000","last_undegraded":"2026-02-24T22:34:59.339684+0000","last_fullsized":"2026-02-24T22:34:59.339684+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:59.339612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:59.339612+0000","last_clean_scrub_stamp":"2026-02-24T22:34:59.339612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:54:10.672461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"312'130","reported_seq":829,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.122528+0000","last_change":"2026-02-24T22:34:42.122528+0000","last_active":"2026-02-24T22:34:42.122528+0000","last_peered":"2026-02-24T22:34:42.122528+0000","last_clean":"2026-02-24T22:34:42.122528+0000","last_became_active":"2026-02-24T22:05:18.186126+0000","last_became_peered":"2026-02-24T22:05:18.186126+0000","last_unstale":"2026-02-24T22:34:42.122528+0000","last_undegraded":"2026-02-24T22:34:42.122528+0000","last_fullsized":"2026-02-24T22:34:42.122528+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'130","last_scrub_stamp":"2026-02-24T22:34:42.122496+0000","last_deep_scrub":"312'130","last_deep_scrub_stamp":"2026-02-24T22:34:42.122496+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.122496+0000","objects_scrubbed":29,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:13:06.464986+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":336,"num_read_kb":289,"num_write":128,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":626,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:30.093328+0000","last_change":"2026-02-24T22:34:30.093328+0000","last_active":"2026-02-24T22:34:30.093328+0000","last_peered":"2026-02-24T22:34:30.093328+0000","last_clean":"2026-02-24T22:34:30.093328+0000","last_became_active":"2026-02-24T22:05:08.682035+0000","last_became_peered":"2026-02-24T22:05:08.682035+0000","last_unstale":"2026-02-24T22:34:30.093328+0000","last_undegraded":"2026-02-24T22:34:30.093328+0000","last_fullsized":"2026-02-24T22:34:30.093328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:30.093252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:30.093252+0000","last_clean_scrub_stamp":"2026-02-24T22:34:30.093252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:06:58.991217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047531599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"312'305","reported_seq":1057,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.010886+0000","last_change":"2026-02-24T22:34:34.010886+0000","last_active":"2026-02-24T22:34:34.010886+0000","last_peered":"2026-02-24T22:34:34.010886+0000","last_clean":"2026-02-24T22:34:34.010886+0000","last_became_active":"2026-02-24T22:05:16.178331+0000","last_became_peered":"2026-02-24T22:05:16.178331+0000","last_unstale":"2026-02-24T22:34:34.010886+0000","last_undegraded":"2026-02-24T22:34:34.010886+0000","last_fullsized":"2026-02-24T22:34:34.010886+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'305","last_scrub_stamp":"2026-02-24T22:34:34.010799+0000","last_deep_scrub":"312'305","last_deep_scrub_stamp":"2026-02-24T22:34:34.010799+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.010799+0000","objects_scrubbed":6,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:33:52.230516+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":583,"num_read_kb":6589,"num_write":1333,"num_write_kb":11354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"312'2","reported_seq":604,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.063471+0000","last_change":"2026-02-24T22:34:42.063471+0000","last_active":"2026-02-24T22:34:42.063471+0000","last_peered":"2026-02-24T22:34:42.063471+0000","last_clean":"2026-02-24T22:34:42.063471+0000","last_became_active":"2026-02-24T22:05:20.187908+0000","last_became_peered":"2026-02-24T22:05:20.187908+0000","last_unstale":"2026-02-24T22:34:42.063471+0000","last_undegraded":"2026-02-24T22:34:42.063471+0000","last_fullsized":"2026-02-24T22:34:42.063471+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:34:42.063424+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:34:42.063424+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.063424+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:16:34.733800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"312'447","reported_seq":1665,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:54.231930+0000","last_change":"2026-02-24T22:34:54.231930+0000","last_active":"2026-02-24T22:34:54.231930+0000","last_peered":"2026-02-24T22:34:54.231930+0000","last_clean":"2026-02-24T22:34:54.231930+0000","last_became_active":"2026-02-24T22:05:22.912414+0000","last_became_peered":"2026-02-24T22:05:22.912414+0000","last_unstale":"2026-02-24T22:34:54.231930+0000","last_undegraded":"2026-02-24T22:34:54.231930+0000","last_fullsized":"2026-02-24T22:34:54.231930+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'447","last_scrub_stamp":"2026-02-24T22:34:54.231891+0000","last_deep_scrub":"312'447","last_deep_scrub_stamp":"2026-02-24T22:34:54.231891+0000","last_clean_scrub_stamp":"2026-02-24T22:34:54.231891+0000","objects_scrubbed":10,"log_size":47,"log_dups_size":400,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:44:31.798158+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":308,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1080,"num_read_kb":599,"num_write":114,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5278,"num_omap_keys":12,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"312'281","reported_seq":882,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:56.954757+0000","last_change":"2026-02-24T22:34:56.954757+0000","last_active":"2026-02-24T22:34:56.954757+0000","last_peered":"2026-02-24T22:34:56.954757+0000","last_clean":"2026-02-24T22:34:56.954757+0000","last_became_active":"2026-02-24T22:05:24.200029+0000","last_became_peered":"2026-02-24T22:05:24.200029+0000","last_unstale":"2026-02-24T22:34:56.954757+0000","last_undegraded":"2026-02-24T22:34:56.954757+0000","last_fullsized":"2026-02-24T22:34:56.954757+0000","mapping_epoch":30,"log_start":"312'198","ondisk_log_start":"312'198","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'281","last_scrub_stamp":"2026-02-24T22:34:56.954689+0000","last_deep_scrub":"312'281","last_deep_scrub_stamp":"2026-02-24T22:34:56.954689+0000","last_clean_scrub_stamp":"2026-02-24T22:34:56.954689+0000","objects_scrubbed":97,"log_size":83,"log_dups_size":198,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:56:30.907099+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29918,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":442,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":585,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:04.581459+0000","last_change":"2026-02-24T22:35:04.581459+0000","last_active":"2026-02-24T22:35:04.581459+0000","last_peered":"2026-02-24T22:35:04.581459+0000","last_clean":"2026-02-24T22:35:04.581459+0000","last_became_active":"2026-02-24T22:05:29.653806+0000","last_became_peered":"2026-02-24T22:05:29.653806+0000","last_unstale":"2026-02-24T22:35:04.581459+0000","last_undegraded":"2026-02-24T22:35:04.581459+0000","last_fullsized":"2026-02-24T22:35:04.581459+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:04.581396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:04.581396+0000","last_clean_scrub_stamp":"2026-02-24T22:35:04.581396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:10:35.970116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"312'254","reported_seq":1004,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.068312+0000","last_change":"2026-02-24T22:34:38.068312+0000","last_active":"2026-02-24T22:34:38.068312+0000","last_peered":"2026-02-24T22:34:38.068312+0000","last_clean":"2026-02-24T22:34:38.068312+0000","last_became_active":"2026-02-24T22:05:18.183952+0000","last_became_peered":"2026-02-24T22:05:18.183952+0000","last_unstale":"2026-02-24T22:34:38.068312+0000","last_undegraded":"2026-02-24T22:34:38.068312+0000","last_fullsized":"2026-02-24T22:34:38.068312+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'254","last_scrub_stamp":"2026-02-24T22:34:38.068260+0000","last_deep_scrub":"312'254","last_deep_scrub_stamp":"2026-02-24T22:34:38.068260+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.068260+0000","objects_scrubbed":24,"log_size":54,"log_dups_size":200,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:24:48.806854+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":963,"num_read_kb":811,"num_write":374,"num_write_kb":210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"312'3","reported_seq":615,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.112722+0000","last_change":"2026-02-24T22:34:31.112722+0000","last_active":"2026-02-24T22:34:31.112722+0000","last_peered":"2026-02-24T22:34:31.112722+0000","last_clean":"2026-02-24T22:34:31.112722+0000","last_became_active":"2026-02-24T22:05:08.690042+0000","last_became_peered":"2026-02-24T22:05:08.690042+0000","last_unstale":"2026-02-24T22:34:31.112722+0000","last_undegraded":"2026-02-24T22:34:31.112722+0000","last_fullsized":"2026-02-24T22:34:31.112722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:34:31.112652+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:34:31.112652+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.112652+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:38:30.546771+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0010128400000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"312'333","reported_seq":1056,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:30.596222+0000","last_change":"2026-02-24T22:34:30.596222+0000","last_active":"2026-02-24T22:34:30.596222+0000","last_peered":"2026-02-24T22:34:30.596222+0000","last_clean":"2026-02-24T22:34:30.596222+0000","last_became_active":"2026-02-24T22:05:16.188900+0000","last_became_peered":"2026-02-24T22:05:16.188900+0000","last_unstale":"2026-02-24T22:34:30.596222+0000","last_undegraded":"2026-02-24T22:34:30.596222+0000","last_fullsized":"2026-02-24T22:34:30.596222+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'333","last_scrub_stamp":"2026-02-24T22:34:30.596162+0000","last_deep_scrub":"312'333","last_deep_scrub_stamp":"2026-02-24T22:34:30.596162+0000","last_clean_scrub_stamp":"2026-02-24T22:34:30.596162+0000","objects_scrubbed":6,"log_size":33,"log_dups_size":300,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:56:38.748514+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242895,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":415,"num_read_kb":10969,"num_write":1277,"num_write_kb":10859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":598,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.915846+0000","last_change":"2026-02-24T22:34:49.915846+0000","last_active":"2026-02-24T22:34:49.915846+0000","last_peered":"2026-02-24T22:34:49.915846+0000","last_clean":"2026-02-24T22:34:49.915846+0000","last_became_active":"2026-02-24T22:05:20.187251+0000","last_became_peered":"2026-02-24T22:05:20.187251+0000","last_unstale":"2026-02-24T22:34:49.915846+0000","last_undegraded":"2026-02-24T22:34:49.915846+0000","last_fullsized":"2026-02-24T22:34:49.915846+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:49.915785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:49.915785+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.915785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:30:18.871540+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"312'83","reported_seq":712,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:52.786106+0000","last_change":"2026-02-24T22:34:52.786106+0000","last_active":"2026-02-24T22:34:52.786106+0000","last_peered":"2026-02-24T22:34:52.786106+0000","last_clean":"2026-02-24T22:34:52.786106+0000","last_became_active":"2026-02-24T22:05:22.191394+0000","last_became_peered":"2026-02-24T22:05:22.191394+0000","last_unstale":"2026-02-24T22:34:52.786106+0000","last_undegraded":"2026-02-24T22:34:52.786106+0000","last_fullsized":"2026-02-24T22:34:52.786106+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'83","last_scrub_stamp":"2026-02-24T22:34:52.786051+0000","last_deep_scrub":"312'83","last_deep_scrub_stamp":"2026-02-24T22:34:52.786051+0000","last_clean_scrub_stamp":"2026-02-24T22:34:52.786051+0000","objects_scrubbed":7,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:23:33.675567+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":103,"num_read_kb":96,"num_write":72,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1630,"num_omap_keys":4,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"312'323","reported_seq":951,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.637677+0000","last_change":"2026-02-24T22:34:48.637677+0000","last_active":"2026-02-24T22:34:48.637677+0000","last_peered":"2026-02-24T22:34:48.637677+0000","last_clean":"2026-02-24T22:34:48.637677+0000","last_became_active":"2026-02-24T22:05:24.207698+0000","last_became_peered":"2026-02-24T22:05:24.207698+0000","last_unstale":"2026-02-24T22:34:48.637677+0000","last_undegraded":"2026-02-24T22:34:48.637677+0000","last_fullsized":"2026-02-24T22:34:48.637677+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'323","last_scrub_stamp":"2026-02-24T22:34:48.637620+0000","last_deep_scrub":"312'323","last_deep_scrub_stamp":"2026-02-24T22:34:48.637620+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.637620+0000","objects_scrubbed":94,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:13:22.580314+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31273,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":246,"num_read_kb":224,"num_write":553,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":574,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:00.855050+0000","last_change":"2026-02-24T22:35:00.855050+0000","last_active":"2026-02-24T22:35:00.855050+0000","last_peered":"2026-02-24T22:35:00.855050+0000","last_clean":"2026-02-24T22:35:00.855050+0000","last_became_active":"2026-02-24T22:05:29.660065+0000","last_became_peered":"2026-02-24T22:05:29.660065+0000","last_unstale":"2026-02-24T22:35:00.855050+0000","last_undegraded":"2026-02-24T22:35:00.855050+0000","last_fullsized":"2026-02-24T22:35:00.855050+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:00.855003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:00.855003+0000","last_clean_scrub_stamp":"2026-02-24T22:35:00.855003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:46:34.269073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"312'191","reported_seq":898,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.101612+0000","last_change":"2026-02-24T22:34:39.101612+0000","last_active":"2026-02-24T22:34:39.101612+0000","last_peered":"2026-02-24T22:34:39.101612+0000","last_clean":"2026-02-24T22:34:39.101612+0000","last_became_active":"2026-02-24T22:05:18.185112+0000","last_became_peered":"2026-02-24T22:05:18.185112+0000","last_unstale":"2026-02-24T22:34:39.101612+0000","last_undegraded":"2026-02-24T22:34:39.101612+0000","last_fullsized":"2026-02-24T22:34:39.101612+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'191","last_scrub_stamp":"2026-02-24T22:34:39.101566+0000","last_deep_scrub":"312'191","last_deep_scrub_stamp":"2026-02-24T22:34:39.101566+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.101566+0000","objects_scrubbed":19,"log_size":91,"log_dups_size":100,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:44:31.875433+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":679,"num_read_kb":553,"num_write":246,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"312'281","reported_seq":1011,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.103881+0000","last_change":"2026-02-24T22:34:32.103832+0000","last_active":"2026-02-24T22:34:32.103881+0000","last_peered":"2026-02-24T22:34:32.103881+0000","last_clean":"2026-02-24T22:34:32.103881+0000","last_became_active":"2026-02-24T22:05:16.191007+0000","last_became_peered":"2026-02-24T22:05:16.191007+0000","last_unstale":"2026-02-24T22:34:32.103881+0000","last_undegraded":"2026-02-24T22:34:32.103881+0000","last_fullsized":"2026-02-24T22:34:32.103881+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'281","last_scrub_stamp":"2026-02-24T22:34:32.103780+0000","last_deep_scrub":"312'281","last_deep_scrub_stamp":"2026-02-24T22:34:32.103780+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.103780+0000","objects_scrubbed":4,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:02:28.007749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":495,"num_read_kb":40330,"num_write":1338,"num_write_kb":18546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":621,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:29.991920+0000","last_change":"2026-02-24T22:34:29.991920+0000","last_active":"2026-02-24T22:34:29.991920+0000","last_peered":"2026-02-24T22:34:29.991920+0000","last_clean":"2026-02-24T22:34:29.991920+0000","last_became_active":"2026-02-24T22:05:08.758282+0000","last_became_peered":"2026-02-24T22:05:08.758282+0000","last_unstale":"2026-02-24T22:34:29.991920+0000","last_undegraded":"2026-02-24T22:34:29.991920+0000","last_fullsized":"2026-02-24T22:34:29.991920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:29.991857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:29.991857+0000","last_clean_scrub_stamp":"2026-02-24T22:34:29.991857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:47:43.334173+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00033355899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":598,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.948604+0000","last_change":"2026-02-24T22:34:48.948604+0000","last_active":"2026-02-24T22:34:48.948604+0000","last_peered":"2026-02-24T22:34:48.948604+0000","last_clean":"2026-02-24T22:34:48.948604+0000","last_became_active":"2026-02-24T22:05:20.187208+0000","last_became_peered":"2026-02-24T22:05:20.187208+0000","last_unstale":"2026-02-24T22:34:48.948604+0000","last_undegraded":"2026-02-24T22:34:48.948604+0000","last_fullsized":"2026-02-24T22:34:48.948604+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:48.948541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:48.948541+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.948541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:29:13.712233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"312'1143","reported_seq":2505,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:52.891935+0000","last_change":"2026-02-24T22:34:52.891935+0000","last_active":"2026-02-24T22:34:52.891935+0000","last_peered":"2026-02-24T22:34:52.891935+0000","last_clean":"2026-02-24T22:34:52.891935+0000","last_became_active":"2026-02-24T22:05:22.191763+0000","last_became_peered":"2026-02-24T22:05:22.191763+0000","last_unstale":"2026-02-24T22:34:52.891935+0000","last_undegraded":"2026-02-24T22:34:52.891935+0000","last_fullsized":"2026-02-24T22:34:52.891935+0000","mapping_epoch":28,"log_start":"312'1100","ondisk_log_start":"312'1100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'1143","last_scrub_stamp":"2026-02-24T22:34:52.891844+0000","last_deep_scrub":"312'1143","last_deep_scrub_stamp":"2026-02-24T22:34:52.891844+0000","last_clean_scrub_stamp":"2026-02-24T22:34:52.891844+0000","objects_scrubbed":12,"log_size":43,"log_dups_size":1100,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:24:36.081418+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":76731,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":3517,"num_read_kb":3087,"num_write":637,"num_write_kb":534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3608,"num_omap_keys":6,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"312'306","reported_seq":925,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:00.358718+0000","last_change":"2026-02-24T22:35:00.358718+0000","last_active":"2026-02-24T22:35:00.358718+0000","last_peered":"2026-02-24T22:35:00.358718+0000","last_clean":"2026-02-24T22:35:00.358718+0000","last_became_active":"2026-02-24T22:05:24.205923+0000","last_became_peered":"2026-02-24T22:05:24.205923+0000","last_unstale":"2026-02-24T22:35:00.358718+0000","last_undegraded":"2026-02-24T22:35:00.358718+0000","last_fullsized":"2026-02-24T22:35:00.358718+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'306","last_scrub_stamp":"2026-02-24T22:35:00.358668+0000","last_deep_scrub":"312'306","last_deep_scrub_stamp":"2026-02-24T22:35:00.358668+0000","last_clean_scrub_stamp":"2026-02-24T22:35:00.358668+0000","objects_scrubbed":102,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:31:13.435072+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34551,"num_objects":102,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":190,"num_read_kb":174,"num_write":516,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"312'468","reported_seq":4839,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:05.755883+0000","last_change":"2026-02-24T22:35:05.755883+0000","last_active":"2026-02-24T22:35:05.755883+0000","last_peered":"2026-02-24T22:35:05.755883+0000","last_clean":"2026-02-24T22:35:05.755883+0000","last_became_active":"2026-02-24T22:05:29.652259+0000","last_became_peered":"2026-02-24T22:05:29.652259+0000","last_unstale":"2026-02-24T22:35:05.755883+0000","last_undegraded":"2026-02-24T22:35:05.755883+0000","last_fullsized":"2026-02-24T22:35:05.755883+0000","mapping_epoch":33,"log_start":"312'401","ondisk_log_start":"312'401","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'468","last_scrub_stamp":"2026-02-24T22:35:05.755823+0000","last_deep_scrub":"312'468","last_deep_scrub_stamp":"2026-02-24T22:35:05.755823+0000","last_clean_scrub_stamp":"2026-02-24T22:35:05.755823+0000","objects_scrubbed":1,"log_size":67,"log_dups_size":401,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:50:47.482426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"312'136","reported_seq":842,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.775557+0000","last_change":"2026-02-24T22:34:39.775557+0000","last_active":"2026-02-24T22:34:39.775557+0000","last_peered":"2026-02-24T22:34:39.775557+0000","last_clean":"2026-02-24T22:34:39.775557+0000","last_became_active":"2026-02-24T22:05:18.183369+0000","last_became_peered":"2026-02-24T22:05:18.183369+0000","last_unstale":"2026-02-24T22:34:39.775557+0000","last_undegraded":"2026-02-24T22:34:39.775557+0000","last_fullsized":"2026-02-24T22:34:39.775557+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'136","last_scrub_stamp":"2026-02-24T22:34:39.775488+0000","last_deep_scrub":"312'136","last_deep_scrub_stamp":"2026-02-24T22:34:39.775488+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.775488+0000","objects_scrubbed":22,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:17:51.694013+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":458,"num_read_kb":386,"num_write":168,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"312'500","reported_seq":1319,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.014188+0000","last_change":"2026-02-24T22:34:31.014188+0000","last_active":"2026-02-24T22:34:31.014188+0000","last_peered":"2026-02-24T22:34:31.014188+0000","last_clean":"2026-02-24T22:34:31.014188+0000","last_became_active":"2026-02-24T22:05:16.178353+0000","last_became_peered":"2026-02-24T22:05:16.178353+0000","last_unstale":"2026-02-24T22:34:31.014188+0000","last_undegraded":"2026-02-24T22:34:31.014188+0000","last_fullsized":"2026-02-24T22:34:31.014188+0000","mapping_epoch":22,"log_start":"312'400","ondisk_log_start":"312'400","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'500","last_scrub_stamp":"2026-02-24T22:34:31.014143+0000","last_deep_scrub":"312'500","last_deep_scrub_stamp":"2026-02-24T22:34:31.014143+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.014143+0000","objects_scrubbed":3,"log_size":100,"log_dups_size":400,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:02:25.941904+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1036,"num_read_kb":9549,"num_write":2046,"num_write_kb":2098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":618,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.179872+0000","last_change":"2026-02-24T22:34:31.179872+0000","last_active":"2026-02-24T22:34:31.179872+0000","last_peered":"2026-02-24T22:34:31.179872+0000","last_clean":"2026-02-24T22:34:31.179872+0000","last_became_active":"2026-02-24T22:05:08.765129+0000","last_became_peered":"2026-02-24T22:05:08.765129+0000","last_unstale":"2026-02-24T22:34:31.179872+0000","last_undegraded":"2026-02-24T22:34:31.179872+0000","last_fullsized":"2026-02-24T22:34:31.179872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:31.179812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:31.179812+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.179812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:51:58.379988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439276,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":593,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.666471+0000","last_change":"2026-02-24T22:34:43.666471+0000","last_active":"2026-02-24T22:34:43.666471+0000","last_peered":"2026-02-24T22:34:43.666471+0000","last_clean":"2026-02-24T22:34:43.666471+0000","last_became_active":"2026-02-24T22:05:20.192811+0000","last_became_peered":"2026-02-24T22:05:20.192811+0000","last_unstale":"2026-02-24T22:34:43.666471+0000","last_undegraded":"2026-02-24T22:34:43.666471+0000","last_fullsized":"2026-02-24T22:34:43.666471+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:43.666412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:43.666412+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.666412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:08:50.485350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"312'619","reported_seq":1738,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:50.237395+0000","last_change":"2026-02-24T22:34:50.237395+0000","last_active":"2026-02-24T22:34:50.237395+0000","last_peered":"2026-02-24T22:34:50.237395+0000","last_clean":"2026-02-24T22:34:50.237395+0000","last_became_active":"2026-02-24T22:05:22.905663+0000","last_became_peered":"2026-02-24T22:05:22.905663+0000","last_unstale":"2026-02-24T22:34:50.237395+0000","last_undegraded":"2026-02-24T22:34:50.237395+0000","last_fullsized":"2026-02-24T22:34:50.237395+0000","mapping_epoch":28,"log_start":"312'600","ondisk_log_start":"312'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'619","last_scrub_stamp":"2026-02-24T22:34:50.237359+0000","last_deep_scrub":"312'619","last_deep_scrub_stamp":"2026-02-24T22:34:50.237359+0000","last_clean_scrub_stamp":"2026-02-24T22:34:50.237359+0000","objects_scrubbed":10,"log_size":19,"log_dups_size":600,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:03:58.542051+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":308,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":2064,"num_read_kb":1111,"num_write":154,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5320,"num_omap_keys":12,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"312'261","reported_seq":857,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.006364+0000","last_change":"2026-02-24T22:34:58.006364+0000","last_active":"2026-02-24T22:34:58.006364+0000","last_peered":"2026-02-24T22:34:58.006364+0000","last_clean":"2026-02-24T22:34:58.006364+0000","last_became_active":"2026-02-24T22:05:24.205847+0000","last_became_peered":"2026-02-24T22:05:24.205847+0000","last_unstale":"2026-02-24T22:34:58.006364+0000","last_undegraded":"2026-02-24T22:34:58.006364+0000","last_fullsized":"2026-02-24T22:34:58.006364+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'261","last_scrub_stamp":"2026-02-24T22:34:58.006321+0000","last_deep_scrub":"312'261","last_deep_scrub_stamp":"2026-02-24T22:34:58.006321+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.006321+0000","objects_scrubbed":90,"log_size":61,"log_dups_size":200,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:12:15.690665+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26185,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":121,"num_read_kb":112,"num_write":413,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":585,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:05.578691+0000","last_change":"2026-02-24T22:35:05.578691+0000","last_active":"2026-02-24T22:35:05.578691+0000","last_peered":"2026-02-24T22:35:05.578691+0000","last_clean":"2026-02-24T22:35:05.578691+0000","last_became_active":"2026-02-24T22:05:29.651401+0000","last_became_peered":"2026-02-24T22:05:29.651401+0000","last_unstale":"2026-02-24T22:35:05.578691+0000","last_undegraded":"2026-02-24T22:35:05.578691+0000","last_fullsized":"2026-02-24T22:35:05.578691+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:05.578627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:05.578627+0000","last_clean_scrub_stamp":"2026-02-24T22:35:05.578627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:05:23.173834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"312'175","reported_seq":881,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.748014+0000","last_change":"2026-02-24T22:34:40.748014+0000","last_active":"2026-02-24T22:34:40.748014+0000","last_peered":"2026-02-24T22:34:40.748014+0000","last_clean":"2026-02-24T22:34:40.748014+0000","last_became_active":"2026-02-24T22:05:18.185592+0000","last_became_peered":"2026-02-24T22:05:18.185592+0000","last_unstale":"2026-02-24T22:34:40.748014+0000","last_undegraded":"2026-02-24T22:34:40.748014+0000","last_fullsized":"2026-02-24T22:34:40.748014+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'175","last_scrub_stamp":"2026-02-24T22:34:40.747966+0000","last_deep_scrub":"312'175","last_deep_scrub_stamp":"2026-02-24T22:34:40.747966+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.747966+0000","objects_scrubbed":25,"log_size":75,"log_dups_size":100,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:49:27.517475+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":536,"num_read_kb":444,"num_write":207,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"312'316","reported_seq":1050,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:30.882380+0000","last_change":"2026-02-24T22:34:30.882380+0000","last_active":"2026-02-24T22:34:30.882380+0000","last_peered":"2026-02-24T22:34:30.882380+0000","last_clean":"2026-02-24T22:34:30.882380+0000","last_became_active":"2026-02-24T22:05:16.184131+0000","last_became_peered":"2026-02-24T22:05:16.184131+0000","last_unstale":"2026-02-24T22:34:30.882380+0000","last_undegraded":"2026-02-24T22:34:30.882380+0000","last_fullsized":"2026-02-24T22:34:30.882380+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'316","last_scrub_stamp":"2026-02-24T22:34:30.882351+0000","last_deep_scrub":"312'316","last_deep_scrub_stamp":"2026-02-24T22:34:30.882351+0000","last_clean_scrub_stamp":"2026-02-24T22:34:30.882351+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":300,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:29:29.669100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":471,"num_read_kb":1193,"num_write":1308,"num_write_kb":244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":620,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.039544+0000","last_change":"2026-02-24T22:34:32.039544+0000","last_active":"2026-02-24T22:34:32.039544+0000","last_peered":"2026-02-24T22:34:32.039544+0000","last_clean":"2026-02-24T22:34:32.039544+0000","last_became_active":"2026-02-24T22:05:08.682913+0000","last_became_peered":"2026-02-24T22:05:08.682913+0000","last_unstale":"2026-02-24T22:34:32.039544+0000","last_undegraded":"2026-02-24T22:34:32.039544+0000","last_fullsized":"2026-02-24T22:34:32.039544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:32.039480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:32.039480+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.039480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:39:23.840470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000359878,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":600,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:51.335400+0000","last_change":"2026-02-24T22:34:51.335400+0000","last_active":"2026-02-24T22:34:51.335400+0000","last_peered":"2026-02-24T22:34:51.335400+0000","last_clean":"2026-02-24T22:34:51.335400+0000","last_became_active":"2026-02-24T22:05:20.192162+0000","last_became_peered":"2026-02-24T22:05:20.192162+0000","last_unstale":"2026-02-24T22:34:51.335400+0000","last_undegraded":"2026-02-24T22:34:51.335400+0000","last_fullsized":"2026-02-24T22:34:51.335400+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:51.335327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:51.335327+0000","last_clean_scrub_stamp":"2026-02-24T22:34:51.335327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:49:38.610778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"312'1040","reported_seq":2612,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:56.289338+0000","last_change":"2026-02-24T22:34:56.289338+0000","last_active":"2026-02-24T22:34:56.289338+0000","last_peered":"2026-02-24T22:34:56.289338+0000","last_clean":"2026-02-24T22:34:56.289338+0000","last_became_active":"2026-02-24T22:05:22.198252+0000","last_became_peered":"2026-02-24T22:05:22.198252+0000","last_unstale":"2026-02-24T22:34:56.289338+0000","last_undegraded":"2026-02-24T22:34:56.289338+0000","last_fullsized":"2026-02-24T22:34:56.289338+0000","mapping_epoch":28,"log_start":"312'1000","ondisk_log_start":"312'1000","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'1040","last_scrub_stamp":"2026-02-24T22:34:56.289305+0000","last_deep_scrub":"312'1040","last_deep_scrub_stamp":"2026-02-24T22:34:56.289305+0000","last_clean_scrub_stamp":"2026-02-24T22:34:56.289305+0000","objects_scrubbed":9,"log_size":40,"log_dups_size":1000,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:35:30.102364+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":614,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":3875,"num_read_kb":1975,"num_write":82,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"312'1796","reported_seq":4427,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:00.610139+0000","last_change":"2026-02-24T22:35:00.610139+0000","last_active":"2026-02-24T22:35:00.610139+0000","last_peered":"2026-02-24T22:35:00.610139+0000","last_clean":"2026-02-24T22:35:00.610139+0000","last_became_active":"2026-02-24T22:05:24.198348+0000","last_became_peered":"2026-02-24T22:05:24.198348+0000","last_unstale":"2026-02-24T22:35:00.610139+0000","last_undegraded":"2026-02-24T22:35:00.610139+0000","last_fullsized":"2026-02-24T22:35:00.610139+0000","mapping_epoch":30,"log_start":"312'1698","ondisk_log_start":"312'1698","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'1796","last_scrub_stamp":"2026-02-24T22:35:00.610099+0000","last_deep_scrub":"312'1796","last_deep_scrub_stamp":"2026-02-24T22:35:00.610099+0000","last_clean_scrub_stamp":"2026-02-24T22:35:00.610099+0000","objects_scrubbed":98,"log_size":98,"log_dups_size":1698,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:58:01.687967+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31448,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":5151,"num_read_kb":231050,"num_write":3475,"num_write_kb":1655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":274185,"num_omap_keys":1500,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":583,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:04.708118+0000","last_change":"2026-02-24T22:35:04.708118+0000","last_active":"2026-02-24T22:35:04.708118+0000","last_peered":"2026-02-24T22:35:04.708118+0000","last_clean":"2026-02-24T22:35:04.708118+0000","last_became_active":"2026-02-24T22:05:29.652321+0000","last_became_peered":"2026-02-24T22:05:29.652321+0000","last_unstale":"2026-02-24T22:35:04.708118+0000","last_undegraded":"2026-02-24T22:35:04.708118+0000","last_fullsized":"2026-02-24T22:35:04.708118+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:04.708046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:04.708046+0000","last_clean_scrub_stamp":"2026-02-24T22:35:04.708046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:35:04.460228+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"312'72","reported_seq":745,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.449851+0000","last_change":"2026-02-24T22:34:36.449851+0000","last_active":"2026-02-24T22:34:36.449851+0000","last_peered":"2026-02-24T22:34:36.449851+0000","last_clean":"2026-02-24T22:34:36.449851+0000","last_became_active":"2026-02-24T22:05:18.191249+0000","last_became_peered":"2026-02-24T22:05:18.191249+0000","last_unstale":"2026-02-24T22:34:36.449851+0000","last_undegraded":"2026-02-24T22:34:36.449851+0000","last_fullsized":"2026-02-24T22:34:36.449851+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'72","last_scrub_stamp":"2026-02-24T22:34:36.449799+0000","last_deep_scrub":"312'72","last_deep_scrub_stamp":"2026-02-24T22:34:36.449799+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.449799+0000","objects_scrubbed":17,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:05:38.920525+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":226,"num_read_kb":230,"num_write":70,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2876,"num_omap_keys":12,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"312'274","reported_seq":1014,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.696342+0000","last_change":"2026-02-24T22:34:32.696342+0000","last_active":"2026-02-24T22:34:32.696342+0000","last_peered":"2026-02-24T22:34:32.696342+0000","last_clean":"2026-02-24T22:34:32.696342+0000","last_became_active":"2026-02-24T22:05:16.181840+0000","last_became_peered":"2026-02-24T22:05:16.181840+0000","last_unstale":"2026-02-24T22:34:32.696342+0000","last_undegraded":"2026-02-24T22:34:32.696342+0000","last_fullsized":"2026-02-24T22:34:32.696342+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'274","last_scrub_stamp":"2026-02-24T22:34:32.696287+0000","last_deep_scrub":"312'274","last_deep_scrub_stamp":"2026-02-24T22:34:32.696287+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.696287+0000","objects_scrubbed":4,"log_size":74,"log_dups_size":200,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:57:17.468566+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":486,"num_read_kb":7386,"num_write":1250,"num_write_kb":13063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":615,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.748652+0000","last_change":"2026-02-24T22:34:31.748652+0000","last_active":"2026-02-24T22:34:31.748652+0000","last_peered":"2026-02-24T22:34:31.748652+0000","last_clean":"2026-02-24T22:34:31.748652+0000","last_became_active":"2026-02-24T22:05:08.689542+0000","last_became_peered":"2026-02-24T22:05:08.689542+0000","last_unstale":"2026-02-24T22:34:31.748652+0000","last_undegraded":"2026-02-24T22:34:31.748652+0000","last_fullsized":"2026-02-24T22:34:31.748652+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:31.748592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:31.748592+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.748592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:30:37.278240+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00038816099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":600,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.813218+0000","last_change":"2026-02-24T22:34:47.813218+0000","last_active":"2026-02-24T22:34:47.813218+0000","last_peered":"2026-02-24T22:34:47.813218+0000","last_clean":"2026-02-24T22:34:47.813218+0000","last_became_active":"2026-02-24T22:05:20.186108+0000","last_became_peered":"2026-02-24T22:05:20.186108+0000","last_unstale":"2026-02-24T22:34:47.813218+0000","last_undegraded":"2026-02-24T22:34:47.813218+0000","last_fullsized":"2026-02-24T22:34:47.813218+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:47.813164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:47.813164+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.813164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:54:49.662332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"312'612","reported_seq":1273,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:51.192081+0000","last_change":"2026-02-24T22:34:51.192081+0000","last_active":"2026-02-24T22:34:51.192081+0000","last_peered":"2026-02-24T22:34:51.192081+0000","last_clean":"2026-02-24T22:34:51.192081+0000","last_became_active":"2026-02-24T22:05:22.911692+0000","last_became_peered":"2026-02-24T22:05:22.911692+0000","last_unstale":"2026-02-24T22:34:51.192081+0000","last_undegraded":"2026-02-24T22:34:51.192081+0000","last_fullsized":"2026-02-24T22:34:51.192081+0000","mapping_epoch":28,"log_start":"312'600","ondisk_log_start":"312'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'612","last_scrub_stamp":"2026-02-24T22:34:51.192010+0000","last_deep_scrub":"312'612","last_deep_scrub_stamp":"2026-02-24T22:34:51.192010+0000","last_clean_scrub_stamp":"2026-02-24T22:34:51.192010+0000","objects_scrubbed":8,"log_size":12,"log_dups_size":600,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:39:44.174934+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115263,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1049,"num_read_kb":2748,"num_write":972,"num_write_kb":882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"312'268","reported_seq":871,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:57.751800+0000","last_change":"2026-02-24T22:34:57.751800+0000","last_active":"2026-02-24T22:34:57.751800+0000","last_peered":"2026-02-24T22:34:57.751800+0000","last_clean":"2026-02-24T22:34:57.751800+0000","last_became_active":"2026-02-24T22:05:24.199398+0000","last_became_peered":"2026-02-24T22:05:24.199398+0000","last_unstale":"2026-02-24T22:34:57.751800+0000","last_undegraded":"2026-02-24T22:34:57.751800+0000","last_fullsized":"2026-02-24T22:34:57.751800+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'268","last_scrub_stamp":"2026-02-24T22:34:57.751746+0000","last_deep_scrub":"312'268","last_deep_scrub_stamp":"2026-02-24T22:34:57.751746+0000","last_clean_scrub_stamp":"2026-02-24T22:34:57.751746+0000","objects_scrubbed":93,"log_size":68,"log_dups_size":200,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:33:56.743054+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30139,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":95,"num_read_kb":93,"num_write":436,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"312'436","reported_seq":3603,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:01.857658+0000","last_change":"2026-02-24T22:35:01.857658+0000","last_active":"2026-02-24T22:35:01.857658+0000","last_peered":"2026-02-24T22:35:01.857658+0000","last_clean":"2026-02-24T22:35:01.857658+0000","last_became_active":"2026-02-24T22:05:29.661694+0000","last_became_peered":"2026-02-24T22:05:29.661694+0000","last_unstale":"2026-02-24T22:35:01.857658+0000","last_undegraded":"2026-02-24T22:35:01.857658+0000","last_fullsized":"2026-02-24T22:35:01.857658+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'436","last_scrub_stamp":"2026-02-24T22:35:01.857603+0000","last_deep_scrub":"312'436","last_deep_scrub_stamp":"2026-02-24T22:35:01.857603+0000","last_clean_scrub_stamp":"2026-02-24T22:35:01.857603+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":400,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:30:24.915658+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"312'462","reported_seq":2605,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:08.756144+0000","last_change":"2026-02-24T22:35:08.756144+0000","last_active":"2026-02-24T22:35:08.756144+0000","last_peered":"2026-02-24T22:35:08.756144+0000","last_clean":"2026-02-24T22:35:08.756144+0000","last_became_active":"2026-02-24T22:05:29.654058+0000","last_became_peered":"2026-02-24T22:05:29.654058+0000","last_unstale":"2026-02-24T22:35:08.756144+0000","last_undegraded":"2026-02-24T22:35:08.756144+0000","last_fullsized":"2026-02-24T22:35:08.756144+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'462","last_scrub_stamp":"2026-02-24T22:35:08.756067+0000","last_deep_scrub":"312'462","last_deep_scrub_stamp":"2026-02-24T22:35:08.756067+0000","last_clean_scrub_stamp":"2026-02-24T22:35:08.756067+0000","objects_scrubbed":1,"log_size":62,"log_dups_size":400,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:13:43.345009+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"312'4219","reported_seq":4960,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.517253+0000","last_change":"2026-02-24T22:34:41.517253+0000","last_active":"2026-02-24T22:34:41.517253+0000","last_peered":"2026-02-24T22:34:41.517253+0000","last_clean":"2026-02-24T22:34:41.517253+0000","last_became_active":"2026-02-24T22:05:18.190114+0000","last_became_peered":"2026-02-24T22:05:18.190114+0000","last_unstale":"2026-02-24T22:34:41.517253+0000","last_undegraded":"2026-02-24T22:34:41.517253+0000","last_fullsized":"2026-02-24T22:34:41.517253+0000","mapping_epoch":24,"log_start":"312'4200","ondisk_log_start":"312'4200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'4219","last_scrub_stamp":"2026-02-24T22:34:41.517218+0000","last_deep_scrub":"312'4219","last_deep_scrub_stamp":"2026-02-24T22:34:41.517218+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.517218+0000","objects_scrubbed":25,"log_size":19,"log_dups_size":3000,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:15:13.238846+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":16758,"num_read_kb":13963,"num_write":6320,"num_write_kb":3135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"312'256","reported_seq":985,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.598822+0000","last_change":"2026-02-24T22:34:31.598822+0000","last_active":"2026-02-24T22:34:31.598822+0000","last_peered":"2026-02-24T22:34:31.598822+0000","last_clean":"2026-02-24T22:34:31.598822+0000","last_became_active":"2026-02-24T22:05:16.188630+0000","last_became_peered":"2026-02-24T22:05:16.188630+0000","last_unstale":"2026-02-24T22:34:31.598822+0000","last_undegraded":"2026-02-24T22:34:31.598822+0000","last_fullsized":"2026-02-24T22:34:31.598822+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'256","last_scrub_stamp":"2026-02-24T22:34:31.598742+0000","last_deep_scrub":"312'256","last_deep_scrub_stamp":"2026-02-24T22:34:31.598742+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.598742+0000","objects_scrubbed":10,"log_size":56,"log_dups_size":200,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:25:41.229561+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291503,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":466,"num_read_kb":33751,"num_write":1067,"num_write_kb":13009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":596,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.198317+0000","last_change":"2026-02-24T22:34:46.198317+0000","last_active":"2026-02-24T22:34:46.198317+0000","last_peered":"2026-02-24T22:34:46.198317+0000","last_clean":"2026-02-24T22:34:46.198317+0000","last_became_active":"2026-02-24T22:05:20.188330+0000","last_became_peered":"2026-02-24T22:05:20.188330+0000","last_unstale":"2026-02-24T22:34:46.198317+0000","last_undegraded":"2026-02-24T22:34:46.198317+0000","last_fullsized":"2026-02-24T22:34:46.198317+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:46.198241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:46.198241+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.198241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:36:44.879872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"312'694","reported_seq":1785,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:55.697246+0000","last_change":"2026-02-24T22:34:55.697246+0000","last_active":"2026-02-24T22:34:55.697246+0000","last_peered":"2026-02-24T22:34:55.697246+0000","last_clean":"2026-02-24T22:34:55.697246+0000","last_became_active":"2026-02-24T22:05:22.191584+0000","last_became_peered":"2026-02-24T22:05:22.191584+0000","last_unstale":"2026-02-24T22:34:55.697246+0000","last_undegraded":"2026-02-24T22:34:55.697246+0000","last_fullsized":"2026-02-24T22:34:55.697246+0000","mapping_epoch":28,"log_start":"312'600","ondisk_log_start":"312'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'694","last_scrub_stamp":"2026-02-24T22:34:55.697186+0000","last_deep_scrub":"312'694","last_deep_scrub_stamp":"2026-02-24T22:34:55.697186+0000","last_clean_scrub_stamp":"2026-02-24T22:34:55.697186+0000","objects_scrubbed":7,"log_size":94,"log_dups_size":600,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:38:19.123444+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39069,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2230,"num_read_kb":1811,"num_write":322,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"312'255","reported_seq":876,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:02.574177+0000","last_change":"2026-02-24T22:35:02.574177+0000","last_active":"2026-02-24T22:35:02.574177+0000","last_peered":"2026-02-24T22:35:02.574177+0000","last_clean":"2026-02-24T22:35:02.574177+0000","last_became_active":"2026-02-24T22:05:24.198821+0000","last_became_peered":"2026-02-24T22:05:24.198821+0000","last_unstale":"2026-02-24T22:35:02.574177+0000","last_undegraded":"2026-02-24T22:35:02.574177+0000","last_fullsized":"2026-02-24T22:35:02.574177+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'255","last_scrub_stamp":"2026-02-24T22:35:02.574107+0000","last_deep_scrub":"312'255","last_deep_scrub_stamp":"2026-02-24T22:35:02.574107+0000","last_clean_scrub_stamp":"2026-02-24T22:35:02.574107+0000","objects_scrubbed":94,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:29:00.459611+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28978,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":162,"num_read_kb":145,"num_write":411,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":586,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:09.567710+0000","last_change":"2026-02-24T22:35:09.567710+0000","last_active":"2026-02-24T22:35:09.567710+0000","last_peered":"2026-02-24T22:35:09.567710+0000","last_clean":"2026-02-24T22:35:09.567710+0000","last_became_active":"2026-02-24T22:05:29.652757+0000","last_became_peered":"2026-02-24T22:05:29.652757+0000","last_unstale":"2026-02-24T22:35:09.567710+0000","last_undegraded":"2026-02-24T22:35:09.567710+0000","last_fullsized":"2026-02-24T22:35:09.567710+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:09.567606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:09.567606+0000","last_clean_scrub_stamp":"2026-02-24T22:35:09.567606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:05:13.901352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"312'4263","reported_seq":5020,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.115933+0000","last_change":"2026-02-24T22:34:40.115933+0000","last_active":"2026-02-24T22:34:40.115933+0000","last_peered":"2026-02-24T22:34:40.115933+0000","last_clean":"2026-02-24T22:34:40.115933+0000","last_became_active":"2026-02-24T22:05:18.183983+0000","last_became_peered":"2026-02-24T22:05:18.183983+0000","last_unstale":"2026-02-24T22:34:40.115933+0000","last_undegraded":"2026-02-24T22:34:40.115933+0000","last_fullsized":"2026-02-24T22:34:40.115933+0000","mapping_epoch":24,"log_start":"312'4200","ondisk_log_start":"312'4200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'4263","last_scrub_stamp":"2026-02-24T22:34:40.115891+0000","last_deep_scrub":"312'4263","last_deep_scrub_stamp":"2026-02-24T22:34:40.115891+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.115891+0000","objects_scrubbed":32,"log_size":63,"log_dups_size":3000,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:06:10.082774+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":16912,"num_read_kb":14102,"num_write":6331,"num_write_kb":3151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"312'314","reported_seq":1064,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.052571+0000","last_change":"2026-02-24T22:34:39.052571+0000","last_active":"2026-02-24T22:34:39.052571+0000","last_peered":"2026-02-24T22:34:39.052571+0000","last_clean":"2026-02-24T22:34:39.052571+0000","last_became_active":"2026-02-24T22:05:16.192845+0000","last_became_peered":"2026-02-24T22:05:16.192845+0000","last_unstale":"2026-02-24T22:34:39.052571+0000","last_undegraded":"2026-02-24T22:34:39.052571+0000","last_fullsized":"2026-02-24T22:34:39.052571+0000","mapping_epoch":22,"log_start":"312'301","ondisk_log_start":"312'301","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'314","last_scrub_stamp":"2026-02-24T22:34:39.052507+0000","last_deep_scrub":"312'314","last_deep_scrub_stamp":"2026-02-24T22:34:39.052507+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.052507+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":301,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:33:51.248695+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":490,"num_read_kb":4336,"num_write":1277,"num_write_kb":2676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":598,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.761265+0000","last_change":"2026-02-24T22:34:49.761265+0000","last_active":"2026-02-24T22:34:49.761265+0000","last_peered":"2026-02-24T22:34:49.761265+0000","last_clean":"2026-02-24T22:34:49.761265+0000","last_became_active":"2026-02-24T22:05:20.188790+0000","last_became_peered":"2026-02-24T22:05:20.188790+0000","last_unstale":"2026-02-24T22:34:49.761265+0000","last_undegraded":"2026-02-24T22:34:49.761265+0000","last_fullsized":"2026-02-24T22:34:49.761265+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:49.761185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:49.761185+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.761185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:28:03.018877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"312'28","reported_seq":626,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:54.950663+0000","last_change":"2026-02-24T22:34:54.950663+0000","last_active":"2026-02-24T22:34:54.950663+0000","last_peered":"2026-02-24T22:34:54.950663+0000","last_clean":"2026-02-24T22:34:54.950663+0000","last_became_active":"2026-02-24T22:05:22.192560+0000","last_became_peered":"2026-02-24T22:05:22.192560+0000","last_unstale":"2026-02-24T22:34:54.950663+0000","last_undegraded":"2026-02-24T22:34:54.950663+0000","last_fullsized":"2026-02-24T22:34:54.950663+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'28","last_scrub_stamp":"2026-02-24T22:34:54.950603+0000","last_deep_scrub":"312'28","last_deep_scrub_stamp":"2026-02-24T22:34:54.950603+0000","last_clean_scrub_stamp":"2026-02-24T22:34:54.950603+0000","objects_scrubbed":3,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:46:55.074264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":294,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"312'260","reported_seq":857,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:59.205020+0000","last_change":"2026-02-24T22:34:59.205020+0000","last_active":"2026-02-24T22:34:59.205020+0000","last_peered":"2026-02-24T22:34:59.205020+0000","last_clean":"2026-02-24T22:34:59.205020+0000","last_became_active":"2026-02-24T22:05:24.205506+0000","last_became_peered":"2026-02-24T22:05:24.205506+0000","last_unstale":"2026-02-24T22:34:59.205020+0000","last_undegraded":"2026-02-24T22:34:59.205020+0000","last_fullsized":"2026-02-24T22:34:59.205020+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'260","last_scrub_stamp":"2026-02-24T22:34:59.204958+0000","last_deep_scrub":"312'260","last_deep_scrub_stamp":"2026-02-24T22:34:59.204958+0000","last_clean_scrub_stamp":"2026-02-24T22:34:59.204958+0000","objects_scrubbed":97,"log_size":60,"log_dups_size":200,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:36:53.558006+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30843,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":90,"num_read_kb":84,"num_write":389,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":581,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:06.764299+0000","last_change":"2026-02-24T22:35:06.764299+0000","last_active":"2026-02-24T22:35:06.764299+0000","last_peered":"2026-02-24T22:35:06.764299+0000","last_clean":"2026-02-24T22:35:06.764299+0000","last_became_active":"2026-02-24T22:05:29.652987+0000","last_became_peered":"2026-02-24T22:05:29.652987+0000","last_unstale":"2026-02-24T22:35:06.764299+0000","last_undegraded":"2026-02-24T22:35:06.764299+0000","last_fullsized":"2026-02-24T22:35:06.764299+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:06.764242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:06.764242+0000","last_clean_scrub_stamp":"2026-02-24T22:35:06.764242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:47:21.058266+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"312'207","reported_seq":913,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.729074+0000","last_change":"2026-02-24T22:34:41.729074+0000","last_active":"2026-02-24T22:34:41.729074+0000","last_peered":"2026-02-24T22:34:41.729074+0000","last_clean":"2026-02-24T22:34:41.729074+0000","last_became_active":"2026-02-24T22:05:18.186069+0000","last_became_peered":"2026-02-24T22:05:18.186069+0000","last_unstale":"2026-02-24T22:34:41.729074+0000","last_undegraded":"2026-02-24T22:34:41.729074+0000","last_fullsized":"2026-02-24T22:34:41.729074+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'207","last_scrub_stamp":"2026-02-24T22:34:41.729005+0000","last_deep_scrub":"312'207","last_deep_scrub_stamp":"2026-02-24T22:34:41.729005+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.729005+0000","objects_scrubbed":20,"log_size":107,"log_dups_size":100,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:04:29.291410+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":691,"num_read_kb":561,"num_write":267,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"312'314","reported_seq":1005,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.108267+0000","last_change":"2026-02-24T22:34:31.108224+0000","last_active":"2026-02-24T22:34:31.108267+0000","last_peered":"2026-02-24T22:34:31.108267+0000","last_clean":"2026-02-24T22:34:31.108267+0000","last_became_active":"2026-02-24T22:06:00.662291+0000","last_became_peered":"2026-02-24T22:06:00.662291+0000","last_unstale":"2026-02-24T22:34:31.108267+0000","last_undegraded":"2026-02-24T22:34:31.108267+0000","last_fullsized":"2026-02-24T22:34:31.108267+0000","mapping_epoch":36,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'314","last_scrub_stamp":"2026-02-24T22:34:31.108159+0000","last_deep_scrub":"312'314","last_deep_scrub_stamp":"2026-02-24T22:34:31.108159+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.108159+0000","objects_scrubbed":5,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:32:30.179715+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":421,"num_read_kb":357,"num_write":1374,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":593,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:53.181869+0000","last_change":"2026-02-24T22:34:53.181869+0000","last_active":"2026-02-24T22:34:53.181869+0000","last_peered":"2026-02-24T22:34:53.181869+0000","last_clean":"2026-02-24T22:34:53.181869+0000","last_became_active":"2026-02-24T22:05:20.194215+0000","last_became_peered":"2026-02-24T22:05:20.194215+0000","last_unstale":"2026-02-24T22:34:53.181869+0000","last_undegraded":"2026-02-24T22:34:53.181869+0000","last_fullsized":"2026-02-24T22:34:53.181869+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:53.181785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:53.181785+0000","last_clean_scrub_stamp":"2026-02-24T22:34:53.181785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:35:22.996790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"312'482","reported_seq":1353,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.661029+0000","last_change":"2026-02-24T22:34:58.661029+0000","last_active":"2026-02-24T22:34:58.661029+0000","last_peered":"2026-02-24T22:34:58.661029+0000","last_clean":"2026-02-24T22:34:58.661029+0000","last_became_active":"2026-02-24T22:05:22.191675+0000","last_became_peered":"2026-02-24T22:05:22.191675+0000","last_unstale":"2026-02-24T22:34:58.661029+0000","last_undegraded":"2026-02-24T22:34:58.661029+0000","last_fullsized":"2026-02-24T22:34:58.661029+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'482","last_scrub_stamp":"2026-02-24T22:34:58.660967+0000","last_deep_scrub":"312'482","last_deep_scrub_stamp":"2026-02-24T22:34:58.660967+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.660967+0000","objects_scrubbed":6,"log_size":82,"log_dups_size":400,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:26:30.259061+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39139,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1329,"num_read_kb":1395,"num_write":345,"num_write_kb":280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"312'347","reported_seq":1390,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.631185+0000","last_change":"2026-02-24T22:34:49.631185+0000","last_active":"2026-02-24T22:34:49.631185+0000","last_peered":"2026-02-24T22:34:49.631185+0000","last_clean":"2026-02-24T22:34:49.631185+0000","last_became_active":"2026-02-24T22:05:24.207479+0000","last_became_peered":"2026-02-24T22:05:24.207479+0000","last_unstale":"2026-02-24T22:34:49.631185+0000","last_undegraded":"2026-02-24T22:34:49.631185+0000","last_fullsized":"2026-02-24T22:34:49.631185+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'347","last_scrub_stamp":"2026-02-24T22:34:49.631131+0000","last_deep_scrub":"312'347","last_deep_scrub_stamp":"2026-02-24T22:34:49.631131+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.631131+0000","objects_scrubbed":98,"log_size":47,"log_dups_size":300,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:42:33.404911+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31962,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":675,"num_read_kb":669,"num_write":683,"num_write_kb":207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":572,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:05.770253+0000","last_change":"2026-02-24T22:35:05.770253+0000","last_active":"2026-02-24T22:35:05.770253+0000","last_peered":"2026-02-24T22:35:05.770253+0000","last_clean":"2026-02-24T22:35:05.770253+0000","last_became_active":"2026-02-24T22:05:29.661248+0000","last_became_peered":"2026-02-24T22:05:29.661248+0000","last_unstale":"2026-02-24T22:35:05.770253+0000","last_undegraded":"2026-02-24T22:35:05.770253+0000","last_fullsized":"2026-02-24T22:35:05.770253+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:05.770187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:05.770187+0000","last_clean_scrub_stamp":"2026-02-24T22:35:05.770187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:50:08.978017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"312'142","reported_seq":860,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.756553+0000","last_change":"2026-02-24T22:34:39.756553+0000","last_active":"2026-02-24T22:34:39.756553+0000","last_peered":"2026-02-24T22:34:39.756553+0000","last_clean":"2026-02-24T22:34:39.756553+0000","last_became_active":"2026-02-24T22:05:18.184982+0000","last_became_peered":"2026-02-24T22:05:18.184982+0000","last_unstale":"2026-02-24T22:34:39.756553+0000","last_undegraded":"2026-02-24T22:34:39.756553+0000","last_fullsized":"2026-02-24T22:34:39.756553+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'142","last_scrub_stamp":"2026-02-24T22:34:39.756507+0000","last_deep_scrub":"312'142","last_deep_scrub_stamp":"2026-02-24T22:34:39.756507+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.756507+0000","objects_scrubbed":26,"log_size":42,"log_dups_size":100,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:57:48.133450+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":444,"num_read_kb":381,"num_write":158,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"312'284","reported_seq":1054,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.721473+0000","last_change":"2026-02-24T22:34:33.721473+0000","last_active":"2026-02-24T22:34:33.721473+0000","last_peered":"2026-02-24T22:34:33.721473+0000","last_clean":"2026-02-24T22:34:33.721473+0000","last_became_active":"2026-02-24T22:05:16.184594+0000","last_became_peered":"2026-02-24T22:05:16.184594+0000","last_unstale":"2026-02-24T22:34:33.721473+0000","last_undegraded":"2026-02-24T22:34:33.721473+0000","last_fullsized":"2026-02-24T22:34:33.721473+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'284","last_scrub_stamp":"2026-02-24T22:34:33.721410+0000","last_deep_scrub":"312'284","last_deep_scrub_stamp":"2026-02-24T22:34:33.721410+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.721410+0000","objects_scrubbed":5,"log_size":84,"log_dups_size":200,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:42:30.853887+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048643,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":519,"num_read_kb":28116,"num_write":1194,"num_write_kb":6289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":591,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.669842+0000","last_change":"2026-02-24T22:34:44.669842+0000","last_active":"2026-02-24T22:34:44.669842+0000","last_peered":"2026-02-24T22:34:44.669842+0000","last_clean":"2026-02-24T22:34:44.669842+0000","last_became_active":"2026-02-24T22:05:20.193522+0000","last_became_peered":"2026-02-24T22:05:20.193522+0000","last_unstale":"2026-02-24T22:34:44.669842+0000","last_undegraded":"2026-02-24T22:34:44.669842+0000","last_fullsized":"2026-02-24T22:34:44.669842+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:44.669782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:44.669782+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.669782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:43:09.538394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"312'438","reported_seq":1285,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:55.258468+0000","last_change":"2026-02-24T22:34:55.258468+0000","last_active":"2026-02-24T22:34:55.258468+0000","last_peered":"2026-02-24T22:34:55.258468+0000","last_clean":"2026-02-24T22:34:55.258468+0000","last_became_active":"2026-02-24T22:05:22.199968+0000","last_became_peered":"2026-02-24T22:05:22.199968+0000","last_unstale":"2026-02-24T22:34:55.258468+0000","last_undegraded":"2026-02-24T22:34:55.258468+0000","last_fullsized":"2026-02-24T22:34:55.258468+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'438","last_scrub_stamp":"2026-02-24T22:34:55.258389+0000","last_deep_scrub":"312'438","last_deep_scrub_stamp":"2026-02-24T22:34:55.258389+0000","last_clean_scrub_stamp":"2026-02-24T22:34:55.258389+0000","objects_scrubbed":5,"log_size":38,"log_dups_size":400,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:04:07.943168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38886,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1275,"num_read_kb":1335,"num_write":311,"num_write_kb":280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"312'303","reported_seq":913,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:00.858756+0000","last_change":"2026-02-24T22:35:00.858756+0000","last_active":"2026-02-24T22:35:00.858756+0000","last_peered":"2026-02-24T22:35:00.858756+0000","last_clean":"2026-02-24T22:35:00.858756+0000","last_became_active":"2026-02-24T22:05:24.199650+0000","last_became_peered":"2026-02-24T22:05:24.199650+0000","last_unstale":"2026-02-24T22:35:00.858756+0000","last_undegraded":"2026-02-24T22:35:00.858756+0000","last_fullsized":"2026-02-24T22:35:00.858756+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'303","last_scrub_stamp":"2026-02-24T22:35:00.858703+0000","last_deep_scrub":"312'303","last_deep_scrub_stamp":"2026-02-24T22:35:00.858703+0000","last_clean_scrub_stamp":"2026-02-24T22:35:00.858703+0000","objects_scrubbed":91,"log_size":103,"log_dups_size":200,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:01:19.132884+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29523,"num_objects":91,"num_object_clones":0,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":182,"num_read_kb":193,"num_write":488,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"312'270","reported_seq":989,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.736339+0000","last_change":"2026-02-24T22:34:41.736339+0000","last_active":"2026-02-24T22:34:41.736339+0000","last_peered":"2026-02-24T22:34:41.736339+0000","last_clean":"2026-02-24T22:34:41.736339+0000","last_became_active":"2026-02-24T22:05:18.190238+0000","last_became_peered":"2026-02-24T22:05:18.190238+0000","last_unstale":"2026-02-24T22:34:41.736339+0000","last_undegraded":"2026-02-24T22:34:41.736339+0000","last_fullsized":"2026-02-24T22:34:41.736339+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'270","last_scrub_stamp":"2026-02-24T22:34:41.736291+0000","last_deep_scrub":"312'270","last_deep_scrub_stamp":"2026-02-24T22:34:41.736291+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.736291+0000","objects_scrubbed":21,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:33:02.999931+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":976,"num_read_kb":794,"num_write":353,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":219,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"312'310","reported_seq":1674,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.902084+0000","last_change":"2026-02-24T22:34:33.902084+0000","last_active":"2026-02-24T22:34:33.902084+0000","last_peered":"2026-02-24T22:34:33.902084+0000","last_clean":"2026-02-24T22:34:33.902084+0000","last_became_active":"2026-02-24T22:05:16.184415+0000","last_became_peered":"2026-02-24T22:05:16.184415+0000","last_unstale":"2026-02-24T22:34:33.902084+0000","last_undegraded":"2026-02-24T22:34:33.902084+0000","last_fullsized":"2026-02-24T22:34:33.902084+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'310","last_scrub_stamp":"2026-02-24T22:34:33.902021+0000","last_deep_scrub":"312'310","last_deep_scrub_stamp":"2026-02-24T22:34:33.902021+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.902021+0000","objects_scrubbed":3,"log_size":110,"log_dups_size":200,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:11:44.512299+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2362,"num_read_kb":23834,"num_write":1125,"num_write_kb":5557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":598,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:50.938212+0000","last_change":"2026-02-24T22:34:50.938212+0000","last_active":"2026-02-24T22:34:50.938212+0000","last_peered":"2026-02-24T22:34:50.938212+0000","last_clean":"2026-02-24T22:34:50.938212+0000","last_became_active":"2026-02-24T22:05:20.186325+0000","last_became_peered":"2026-02-24T22:05:20.186325+0000","last_unstale":"2026-02-24T22:34:50.938212+0000","last_undegraded":"2026-02-24T22:34:50.938212+0000","last_fullsized":"2026-02-24T22:34:50.938212+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:50.938156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:50.938156+0000","last_clean_scrub_stamp":"2026-02-24T22:34:50.938156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:12:47.996800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"312'413","reported_seq":1039,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:53.238905+0000","last_change":"2026-02-24T22:34:53.238905+0000","last_active":"2026-02-24T22:34:53.238905+0000","last_peered":"2026-02-24T22:34:53.238905+0000","last_clean":"2026-02-24T22:34:53.238905+0000","last_became_active":"2026-02-24T22:05:22.192756+0000","last_became_peered":"2026-02-24T22:05:22.192756+0000","last_unstale":"2026-02-24T22:34:53.238905+0000","last_undegraded":"2026-02-24T22:34:53.238905+0000","last_fullsized":"2026-02-24T22:34:53.238905+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'413","last_scrub_stamp":"2026-02-24T22:34:53.238844+0000","last_deep_scrub":"312'413","last_deep_scrub_stamp":"2026-02-24T22:34:53.238844+0000","last_clean_scrub_stamp":"2026-02-24T22:34:53.238844+0000","objects_scrubbed":6,"log_size":13,"log_dups_size":400,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:43:03.519649+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77383,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":715,"num_read_kb":1913,"num_write":704,"num_write_kb":692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24470,"num_omap_keys":44,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"312'286","reported_seq":909,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:59.933925+0000","last_change":"2026-02-24T22:34:59.933925+0000","last_active":"2026-02-24T22:34:59.933925+0000","last_peered":"2026-02-24T22:34:59.933925+0000","last_clean":"2026-02-24T22:34:59.933925+0000","last_became_active":"2026-02-24T22:05:24.206407+0000","last_became_peered":"2026-02-24T22:05:24.206407+0000","last_unstale":"2026-02-24T22:34:59.933925+0000","last_undegraded":"2026-02-24T22:34:59.933925+0000","last_fullsized":"2026-02-24T22:34:59.933925+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'286","last_scrub_stamp":"2026-02-24T22:34:59.933869+0000","last_deep_scrub":"312'286","last_deep_scrub_stamp":"2026-02-24T22:34:59.933869+0000","last_clean_scrub_stamp":"2026-02-24T22:34:59.933869+0000","objects_scrubbed":99,"log_size":86,"log_dups_size":200,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:02:47.770768+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30298,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":168,"num_read_kb":155,"num_write":455,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":584,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:08.590933+0000","last_change":"2026-02-24T22:35:08.590933+0000","last_active":"2026-02-24T22:35:08.590933+0000","last_peered":"2026-02-24T22:35:08.590933+0000","last_clean":"2026-02-24T22:35:08.590933+0000","last_became_active":"2026-02-24T22:05:29.650115+0000","last_became_peered":"2026-02-24T22:05:29.650115+0000","last_unstale":"2026-02-24T22:35:08.590933+0000","last_undegraded":"2026-02-24T22:35:08.590933+0000","last_fullsized":"2026-02-24T22:35:08.590933+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:08.590877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:08.590877+0000","last_clean_scrub_stamp":"2026-02-24T22:35:08.590877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:49:57.541981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"312'193","reported_seq":914,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.857329+0000","last_change":"2026-02-24T22:34:43.857329+0000","last_active":"2026-02-24T22:34:43.857329+0000","last_peered":"2026-02-24T22:34:43.857329+0000","last_clean":"2026-02-24T22:34:43.857329+0000","last_became_active":"2026-02-24T22:05:18.183501+0000","last_became_peered":"2026-02-24T22:05:18.183501+0000","last_unstale":"2026-02-24T22:34:43.857329+0000","last_undegraded":"2026-02-24T22:34:43.857329+0000","last_fullsized":"2026-02-24T22:34:43.857329+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'193","last_scrub_stamp":"2026-02-24T22:34:43.857270+0000","last_deep_scrub":"312'193","last_deep_scrub_stamp":"2026-02-24T22:34:43.857270+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.857270+0000","objects_scrubbed":31,"log_size":93,"log_dups_size":100,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:49:05.843210+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":578,"num_read_kb":488,"num_write":224,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":465,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"312'412","reported_seq":1362,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.006923+0000","last_change":"2026-02-24T22:34:33.006923+0000","last_active":"2026-02-24T22:34:33.006923+0000","last_peered":"2026-02-24T22:34:33.006923+0000","last_clean":"2026-02-24T22:34:33.006923+0000","last_became_active":"2026-02-24T22:05:16.182909+0000","last_became_peered":"2026-02-24T22:05:16.182909+0000","last_unstale":"2026-02-24T22:34:33.006923+0000","last_undegraded":"2026-02-24T22:34:33.006923+0000","last_fullsized":"2026-02-24T22:34:33.006923+0000","mapping_epoch":22,"log_start":"312'400","ondisk_log_start":"312'400","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'412","last_scrub_stamp":"2026-02-24T22:34:33.006865+0000","last_deep_scrub":"312'412","last_deep_scrub_stamp":"2026-02-24T22:34:33.006865+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.006865+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":400,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:36:32.796788+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1207,"num_read_kb":9244,"num_write":1611,"num_write_kb":11784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":593,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.622797+0000","last_change":"2026-02-24T22:34:45.622797+0000","last_active":"2026-02-24T22:34:45.622797+0000","last_peered":"2026-02-24T22:34:45.622797+0000","last_clean":"2026-02-24T22:34:45.622797+0000","last_became_active":"2026-02-24T22:05:20.193751+0000","last_became_peered":"2026-02-24T22:05:20.193751+0000","last_unstale":"2026-02-24T22:34:45.622797+0000","last_undegraded":"2026-02-24T22:34:45.622797+0000","last_fullsized":"2026-02-24T22:34:45.622797+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:45.622745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:45.622745+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.622745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:01:44.701244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"312'754","reported_seq":1845,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:56.215477+0000","last_change":"2026-02-24T22:34:56.215477+0000","last_active":"2026-02-24T22:34:56.215477+0000","last_peered":"2026-02-24T22:34:56.215477+0000","last_clean":"2026-02-24T22:34:56.215477+0000","last_became_active":"2026-02-24T22:05:22.200603+0000","last_became_peered":"2026-02-24T22:05:22.200603+0000","last_unstale":"2026-02-24T22:34:56.215477+0000","last_undegraded":"2026-02-24T22:34:56.215477+0000","last_fullsized":"2026-02-24T22:34:56.215477+0000","mapping_epoch":28,"log_start":"312'700","ondisk_log_start":"312'700","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'754","last_scrub_stamp":"2026-02-24T22:34:56.215404+0000","last_deep_scrub":"312'754","last_deep_scrub_stamp":"2026-02-24T22:34:56.215404+0000","last_clean_scrub_stamp":"2026-02-24T22:34:56.215404+0000","objects_scrubbed":10,"log_size":54,"log_dups_size":700,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:58:38.795319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39896,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":2333,"num_read_kb":2057,"num_write":434,"num_write_kb":373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11460,"num_omap_keys":24,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"312'433","reported_seq":1077,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:55.324399+0000","last_change":"2026-02-24T22:34:55.324399+0000","last_active":"2026-02-24T22:34:55.324399+0000","last_peered":"2026-02-24T22:34:55.324399+0000","last_clean":"2026-02-24T22:34:55.324399+0000","last_became_active":"2026-02-24T22:05:24.200391+0000","last_became_peered":"2026-02-24T22:05:24.200391+0000","last_unstale":"2026-02-24T22:34:55.324399+0000","last_undegraded":"2026-02-24T22:34:55.324399+0000","last_fullsized":"2026-02-24T22:34:55.324399+0000","mapping_epoch":30,"log_start":"312'400","ondisk_log_start":"312'400","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'433","last_scrub_stamp":"2026-02-24T22:34:55.324342+0000","last_deep_scrub":"312'433","last_deep_scrub_stamp":"2026-02-24T22:34:55.324342+0000","last_clean_scrub_stamp":"2026-02-24T22:34:55.324342+0000","objects_scrubbed":99,"log_size":33,"log_dups_size":400,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:09:49.237235+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31847,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":370,"num_read_kb":353,"num_write":777,"num_write_kb":271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"312'82","reported_seq":1003,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:53.578429+0000","last_change":"2026-02-24T22:34:53.578429+0000","last_active":"2026-02-24T22:34:53.578429+0000","last_peered":"2026-02-24T22:34:53.578429+0000","last_clean":"2026-02-24T22:34:53.578429+0000","last_became_active":"2026-02-24T22:05:29.660289+0000","last_became_peered":"2026-02-24T22:05:29.660289+0000","last_unstale":"2026-02-24T22:34:53.578429+0000","last_undegraded":"2026-02-24T22:34:53.578429+0000","last_fullsized":"2026-02-24T22:34:53.578429+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'82","last_scrub_stamp":"2026-02-24T22:34:53.578366+0000","last_deep_scrub":"312'82","last_deep_scrub_stamp":"2026-02-24T22:34:53.578366+0000","last_clean_scrub_stamp":"2026-02-24T22:34:53.578366+0000","objects_scrubbed":1,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:20:48.789151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"312'120","reported_seq":798,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.773705+0000","last_change":"2026-02-24T22:34:43.773705+0000","last_active":"2026-02-24T22:34:43.773705+0000","last_peered":"2026-02-24T22:34:43.773705+0000","last_clean":"2026-02-24T22:34:43.773705+0000","last_became_active":"2026-02-24T22:05:18.184745+0000","last_became_peered":"2026-02-24T22:05:18.184745+0000","last_unstale":"2026-02-24T22:34:43.773705+0000","last_undegraded":"2026-02-24T22:34:43.773705+0000","last_fullsized":"2026-02-24T22:34:43.773705+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'120","last_scrub_stamp":"2026-02-24T22:34:43.773655+0000","last_deep_scrub":"312'120","last_deep_scrub_stamp":"2026-02-24T22:34:43.773655+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.773655+0000","objects_scrubbed":20,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:13:15.141522+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":371,"num_read_kb":307,"num_write":132,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":684,"num_omap_keys":3,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"312'307","reported_seq":1035,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.531150+0000","last_change":"2026-02-24T22:34:33.531150+0000","last_active":"2026-02-24T22:34:33.531150+0000","last_peered":"2026-02-24T22:34:33.531150+0000","last_clean":"2026-02-24T22:34:33.531150+0000","last_became_active":"2026-02-24T22:05:16.185574+0000","last_became_peered":"2026-02-24T22:05:16.185574+0000","last_unstale":"2026-02-24T22:34:33.531150+0000","last_undegraded":"2026-02-24T22:34:33.531150+0000","last_fullsized":"2026-02-24T22:34:33.531150+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'307","last_scrub_stamp":"2026-02-24T22:34:33.531066+0000","last_deep_scrub":"312'307","last_deep_scrub_stamp":"2026-02-24T22:34:33.531066+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.531066+0000","objects_scrubbed":4,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:08:13.354888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":428,"num_read_kb":30007,"num_write":1257,"num_write_kb":21358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"312'4","reported_seq":840,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:54.234514+0000","last_change":"2026-02-24T22:34:54.234514+0000","last_active":"2026-02-24T22:34:54.234514+0000","last_peered":"2026-02-24T22:34:54.234514+0000","last_clean":"2026-02-24T22:34:54.234514+0000","last_became_active":"2026-02-24T22:05:20.192612+0000","last_became_peered":"2026-02-24T22:05:20.192612+0000","last_unstale":"2026-02-24T22:34:54.234514+0000","last_undegraded":"2026-02-24T22:34:54.234514+0000","last_fullsized":"2026-02-24T22:34:54.234514+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'4","last_scrub_stamp":"2026-02-24T22:34:54.234459+0000","last_deep_scrub":"312'4","last_deep_scrub_stamp":"2026-02-24T22:34:54.234459+0000","last_clean_scrub_stamp":"2026-02-24T22:34:54.234459+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:52:30.469777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":484,"num_read_kb":482,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"312'209","reported_seq":830,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:56.698535+0000","last_change":"2026-02-24T22:34:56.698535+0000","last_active":"2026-02-24T22:34:56.698535+0000","last_peered":"2026-02-24T22:34:56.698535+0000","last_clean":"2026-02-24T22:34:56.698535+0000","last_became_active":"2026-02-24T22:05:22.906047+0000","last_became_peered":"2026-02-24T22:05:22.906047+0000","last_unstale":"2026-02-24T22:34:56.698535+0000","last_undegraded":"2026-02-24T22:34:56.698535+0000","last_fullsized":"2026-02-24T22:34:56.698535+0000","mapping_epoch":28,"log_start":"312'100","ondisk_log_start":"312'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'209","last_scrub_stamp":"2026-02-24T22:34:56.698473+0000","last_deep_scrub":"312'209","last_deep_scrub_stamp":"2026-02-24T22:34:56.698473+0000","last_clean_scrub_stamp":"2026-02-24T22:34:56.698473+0000","objects_scrubbed":5,"log_size":109,"log_dups_size":100,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:47:56.787199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38786,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":328,"num_read_kb":839,"num_write":300,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"312'257","reported_seq":866,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:54.273407+0000","last_change":"2026-02-24T22:34:54.273407+0000","last_active":"2026-02-24T22:34:54.273407+0000","last_peered":"2026-02-24T22:34:54.273407+0000","last_clean":"2026-02-24T22:34:54.273407+0000","last_became_active":"2026-02-24T22:05:24.200747+0000","last_became_peered":"2026-02-24T22:05:24.200747+0000","last_unstale":"2026-02-24T22:34:54.273407+0000","last_undegraded":"2026-02-24T22:34:54.273407+0000","last_fullsized":"2026-02-24T22:34:54.273407+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'257","last_scrub_stamp":"2026-02-24T22:34:54.273373+0000","last_deep_scrub":"312'257","last_deep_scrub_stamp":"2026-02-24T22:34:54.273373+0000","last_clean_scrub_stamp":"2026-02-24T22:34:54.273373+0000","objects_scrubbed":93,"log_size":57,"log_dups_size":200,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:40:30.932635+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30873,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":120,"num_read_kb":110,"num_write":405,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":573,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:03.846658+0000","last_change":"2026-02-24T22:35:03.846658+0000","last_active":"2026-02-24T22:35:03.846658+0000","last_peered":"2026-02-24T22:35:03.846658+0000","last_clean":"2026-02-24T22:35:03.846658+0000","last_became_active":"2026-02-24T22:05:29.660344+0000","last_became_peered":"2026-02-24T22:05:29.660344+0000","last_unstale":"2026-02-24T22:35:03.846658+0000","last_undegraded":"2026-02-24T22:35:03.846658+0000","last_fullsized":"2026-02-24T22:35:03.846658+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:03.846600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:03.846600+0000","last_clean_scrub_stamp":"2026-02-24T22:35:03.846600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:10:57.615627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"312'176","reported_seq":856,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.075455+0000","last_change":"2026-02-24T22:34:36.075455+0000","last_active":"2026-02-24T22:34:36.075455+0000","last_peered":"2026-02-24T22:34:36.075455+0000","last_clean":"2026-02-24T22:34:36.075455+0000","last_became_active":"2026-02-24T22:06:00.759215+0000","last_became_peered":"2026-02-24T22:06:00.759215+0000","last_unstale":"2026-02-24T22:34:36.075455+0000","last_undegraded":"2026-02-24T22:34:36.075455+0000","last_fullsized":"2026-02-24T22:34:36.075455+0000","mapping_epoch":36,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'176","last_scrub_stamp":"2026-02-24T22:34:36.075425+0000","last_deep_scrub":"312'176","last_deep_scrub_stamp":"2026-02-24T22:34:36.075425+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.075425+0000","objects_scrubbed":19,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:59:34.861298+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":584,"num_read_kb":483,"num_write":222,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"312'269","reported_seq":1028,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.707474+0000","last_change":"2026-02-24T22:34:31.707474+0000","last_active":"2026-02-24T22:34:31.707474+0000","last_peered":"2026-02-24T22:34:31.707474+0000","last_clean":"2026-02-24T22:34:31.707474+0000","last_became_active":"2026-02-24T22:05:16.183956+0000","last_became_peered":"2026-02-24T22:05:16.183956+0000","last_unstale":"2026-02-24T22:34:31.707474+0000","last_undegraded":"2026-02-24T22:34:31.707474+0000","last_fullsized":"2026-02-24T22:34:31.707474+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'269","last_scrub_stamp":"2026-02-24T22:34:31.707414+0000","last_deep_scrub":"312'269","last_deep_scrub_stamp":"2026-02-24T22:34:31.707414+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.707414+0000","objects_scrubbed":6,"log_size":69,"log_dups_size":200,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:16:18.564686+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242893,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":575,"num_read_kb":6430,"num_write":1267,"num_write_kb":5371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":595,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.203221+0000","last_change":"2026-02-24T22:34:49.203221+0000","last_active":"2026-02-24T22:34:49.203221+0000","last_peered":"2026-02-24T22:34:49.203221+0000","last_clean":"2026-02-24T22:34:49.203221+0000","last_became_active":"2026-02-24T22:05:20.187316+0000","last_became_peered":"2026-02-24T22:05:20.187316+0000","last_unstale":"2026-02-24T22:34:49.203221+0000","last_undegraded":"2026-02-24T22:34:49.203221+0000","last_fullsized":"2026-02-24T22:34:49.203221+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:49.203134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:49.203134+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.203134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:31:08.087721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"312'295","reported_seq":1142,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.297675+0000","last_change":"2026-02-24T22:34:58.297675+0000","last_active":"2026-02-24T22:34:58.297675+0000","last_peered":"2026-02-24T22:34:58.297675+0000","last_clean":"2026-02-24T22:34:58.297675+0000","last_became_active":"2026-02-24T22:05:22.912087+0000","last_became_peered":"2026-02-24T22:05:22.912087+0000","last_unstale":"2026-02-24T22:34:58.297675+0000","last_undegraded":"2026-02-24T22:34:58.297675+0000","last_fullsized":"2026-02-24T22:34:58.297675+0000","mapping_epoch":28,"log_start":"307'200","ondisk_log_start":"307'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'295","last_scrub_stamp":"2026-02-24T22:34:58.297632+0000","last_deep_scrub":"312'295","last_deep_scrub_stamp":"2026-02-24T22:34:58.297632+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.297632+0000","objects_scrubbed":7,"log_size":95,"log_dups_size":200,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:34:14.546778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":152,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1000,"num_read_kb":520,"num_write":51,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1780,"num_omap_keys":6,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"312'272","reported_seq":953,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:50.604472+0000","last_change":"2026-02-24T22:34:50.604472+0000","last_active":"2026-02-24T22:34:50.604472+0000","last_peered":"2026-02-24T22:34:50.604472+0000","last_clean":"2026-02-24T22:34:50.604472+0000","last_became_active":"2026-02-24T22:05:24.206383+0000","last_became_peered":"2026-02-24T22:05:24.206383+0000","last_unstale":"2026-02-24T22:34:50.604472+0000","last_undegraded":"2026-02-24T22:34:50.604472+0000","last_fullsized":"2026-02-24T22:34:50.604472+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'272","last_scrub_stamp":"2026-02-24T22:34:50.604410+0000","last_deep_scrub":"312'272","last_deep_scrub_stamp":"2026-02-24T22:34:50.604410+0000","last_clean_scrub_stamp":"2026-02-24T22:34:50.604410+0000","objects_scrubbed":94,"log_size":72,"log_dups_size":200,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:58:49.327922+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29475,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":255,"num_read_kb":237,"num_write":434,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":579,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:03.209395+0000","last_change":"2026-02-24T22:35:03.209395+0000","last_active":"2026-02-24T22:35:03.209395+0000","last_peered":"2026-02-24T22:35:03.209395+0000","last_clean":"2026-02-24T22:35:03.209395+0000","last_became_active":"2026-02-24T22:05:29.661539+0000","last_became_peered":"2026-02-24T22:05:29.661539+0000","last_unstale":"2026-02-24T22:35:03.209395+0000","last_undegraded":"2026-02-24T22:35:03.209395+0000","last_fullsized":"2026-02-24T22:35:03.209395+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:03.209335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:03.209335+0000","last_clean_scrub_stamp":"2026-02-24T22:35:03.209335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:44:51.735842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"312'201","reported_seq":881,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.483754+0000","last_change":"2026-02-24T22:34:44.483754+0000","last_active":"2026-02-24T22:34:44.483754+0000","last_peered":"2026-02-24T22:34:44.483754+0000","last_clean":"2026-02-24T22:34:44.483754+0000","last_became_active":"2026-02-24T22:05:18.192487+0000","last_became_peered":"2026-02-24T22:05:18.192487+0000","last_unstale":"2026-02-24T22:34:44.483754+0000","last_undegraded":"2026-02-24T22:34:44.483754+0000","last_fullsized":"2026-02-24T22:34:44.483754+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'201","last_scrub_stamp":"2026-02-24T22:34:44.483706+0000","last_deep_scrub":"312'201","last_deep_scrub_stamp":"2026-02-24T22:34:44.483706+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.483706+0000","objects_scrubbed":23,"log_size":101,"log_dups_size":100,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:31:57.971063+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":681,"num_read_kb":551,"num_write":266,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"312'299","reported_seq":1006,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.128130+0000","last_change":"2026-02-24T22:34:36.128130+0000","last_active":"2026-02-24T22:34:36.128130+0000","last_peered":"2026-02-24T22:34:36.128130+0000","last_clean":"2026-02-24T22:34:36.128130+0000","last_became_active":"2026-02-24T22:05:16.190108+0000","last_became_peered":"2026-02-24T22:05:16.190108+0000","last_unstale":"2026-02-24T22:34:36.128130+0000","last_undegraded":"2026-02-24T22:34:36.128130+0000","last_fullsized":"2026-02-24T22:34:36.128130+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'299","last_scrub_stamp":"2026-02-24T22:34:36.128072+0000","last_deep_scrub":"312'299","last_deep_scrub_stamp":"2026-02-24T22:34:36.128072+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.128072+0000","objects_scrubbed":3,"log_size":99,"log_dups_size":200,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:52:02.916678+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242914,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":451,"num_read_kb":15743,"num_write":1326,"num_write_kb":10328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":592,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:52.177761+0000","last_change":"2026-02-24T22:34:52.177761+0000","last_active":"2026-02-24T22:34:52.177761+0000","last_peered":"2026-02-24T22:34:52.177761+0000","last_clean":"2026-02-24T22:34:52.177761+0000","last_became_active":"2026-02-24T22:05:20.194183+0000","last_became_peered":"2026-02-24T22:05:20.194183+0000","last_unstale":"2026-02-24T22:34:52.177761+0000","last_undegraded":"2026-02-24T22:34:52.177761+0000","last_fullsized":"2026-02-24T22:34:52.177761+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:52.177667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:52.177667+0000","last_clean_scrub_stamp":"2026-02-24T22:34:52.177667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:07:28.331498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"312'722","reported_seq":1808,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.944289+0000","last_change":"2026-02-24T22:34:58.944289+0000","last_active":"2026-02-24T22:34:58.944289+0000","last_peered":"2026-02-24T22:34:58.944289+0000","last_clean":"2026-02-24T22:34:58.944289+0000","last_became_active":"2026-02-24T22:05:22.199480+0000","last_became_peered":"2026-02-24T22:05:22.199480+0000","last_unstale":"2026-02-24T22:34:58.944289+0000","last_undegraded":"2026-02-24T22:34:58.944289+0000","last_fullsized":"2026-02-24T22:34:58.944289+0000","mapping_epoch":28,"log_start":"312'700","ondisk_log_start":"312'700","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'722","last_scrub_stamp":"2026-02-24T22:34:58.944231+0000","last_deep_scrub":"312'722","last_deep_scrub_stamp":"2026-02-24T22:34:58.944231+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.944231+0000","objects_scrubbed":9,"log_size":22,"log_dups_size":700,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:10:48.727164+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39366,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":2266,"num_read_kb":1888,"num_write":352,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"312'285","reported_seq":904,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:01.828096+0000","last_change":"2026-02-24T22:35:01.828096+0000","last_active":"2026-02-24T22:35:01.828096+0000","last_peered":"2026-02-24T22:35:01.828096+0000","last_clean":"2026-02-24T22:35:01.828096+0000","last_became_active":"2026-02-24T22:05:24.200434+0000","last_became_peered":"2026-02-24T22:05:24.200434+0000","last_unstale":"2026-02-24T22:35:01.828096+0000","last_undegraded":"2026-02-24T22:35:01.828096+0000","last_fullsized":"2026-02-24T22:35:01.828096+0000","mapping_epoch":30,"log_start":"312'198","ondisk_log_start":"312'198","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'285","last_scrub_stamp":"2026-02-24T22:35:01.828052+0000","last_deep_scrub":"312'285","last_deep_scrub_stamp":"2026-02-24T22:35:01.828052+0000","last_clean_scrub_stamp":"2026-02-24T22:35:01.828052+0000","objects_scrubbed":87,"log_size":87,"log_dups_size":198,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:18:55.623149+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27134,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":189,"num_read_kb":178,"num_write":476,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":579,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:06.277822+0000","last_change":"2026-02-24T22:35:06.277822+0000","last_active":"2026-02-24T22:35:06.277822+0000","last_peered":"2026-02-24T22:35:06.277822+0000","last_clean":"2026-02-24T22:35:06.277822+0000","last_became_active":"2026-02-24T22:05:29.662425+0000","last_became_peered":"2026-02-24T22:05:29.662425+0000","last_unstale":"2026-02-24T22:35:06.277822+0000","last_undegraded":"2026-02-24T22:35:06.277822+0000","last_fullsized":"2026-02-24T22:35:06.277822+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:06.277768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:06.277768+0000","last_clean_scrub_stamp":"2026-02-24T22:35:06.277768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:06:45.271197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"312'157","reported_seq":844,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.518852+0000","last_change":"2026-02-24T22:34:42.518852+0000","last_active":"2026-02-24T22:34:42.518852+0000","last_peered":"2026-02-24T22:34:42.518852+0000","last_clean":"2026-02-24T22:34:42.518852+0000","last_became_active":"2026-02-24T22:05:18.191664+0000","last_became_peered":"2026-02-24T22:05:18.191664+0000","last_unstale":"2026-02-24T22:34:42.518852+0000","last_undegraded":"2026-02-24T22:34:42.518852+0000","last_fullsized":"2026-02-24T22:34:42.518852+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'157","last_scrub_stamp":"2026-02-24T22:34:42.518790+0000","last_deep_scrub":"312'157","last_deep_scrub_stamp":"2026-02-24T22:34:42.518790+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.518790+0000","objects_scrubbed":25,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:34:13.833329+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":463,"num_read_kb":379,"num_write":198,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"312'278","reported_seq":981,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.114265+0000","last_change":"2026-02-24T22:34:35.114265+0000","last_active":"2026-02-24T22:34:35.114265+0000","last_peered":"2026-02-24T22:34:35.114265+0000","last_clean":"2026-02-24T22:34:35.114265+0000","last_became_active":"2026-02-24T22:05:16.188154+0000","last_became_peered":"2026-02-24T22:05:16.188154+0000","last_unstale":"2026-02-24T22:34:35.114265+0000","last_undegraded":"2026-02-24T22:34:35.114265+0000","last_fullsized":"2026-02-24T22:34:35.114265+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'278","last_scrub_stamp":"2026-02-24T22:34:35.114217+0000","last_deep_scrub":"312'278","last_deep_scrub_stamp":"2026-02-24T22:34:35.114217+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.114217+0000","objects_scrubbed":6,"log_size":78,"log_dups_size":200,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:02:24.896641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":434,"num_read_kb":2459,"num_write":1229,"num_write_kb":1814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"312'6","reported_seq":830,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:52.862809+0000","last_change":"2026-02-24T22:34:52.862809+0000","last_active":"2026-02-24T22:34:52.862809+0000","last_peered":"2026-02-24T22:34:52.862809+0000","last_clean":"2026-02-24T22:34:52.862809+0000","last_became_active":"2026-02-24T22:05:20.195202+0000","last_became_peered":"2026-02-24T22:05:20.195202+0000","last_unstale":"2026-02-24T22:34:52.862809+0000","last_undegraded":"2026-02-24T22:34:52.862809+0000","last_fullsized":"2026-02-24T22:34:52.862809+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'6","last_scrub_stamp":"2026-02-24T22:34:52.862767+0000","last_deep_scrub":"312'6","last_deep_scrub_stamp":"2026-02-24T22:34:52.862767+0000","last_clean_scrub_stamp":"2026-02-24T22:34:52.862767+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:47:13.375532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1242,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":488,"num_read_kb":722,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"312'693","reported_seq":1779,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:55.932150+0000","last_change":"2026-02-24T22:34:55.932150+0000","last_active":"2026-02-24T22:34:55.932150+0000","last_peered":"2026-02-24T22:34:55.932150+0000","last_clean":"2026-02-24T22:34:55.932150+0000","last_became_active":"2026-02-24T22:05:22.192035+0000","last_became_peered":"2026-02-24T22:05:22.192035+0000","last_unstale":"2026-02-24T22:34:55.932150+0000","last_undegraded":"2026-02-24T22:34:55.932150+0000","last_fullsized":"2026-02-24T22:34:55.932150+0000","mapping_epoch":28,"log_start":"312'600","ondisk_log_start":"312'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'693","last_scrub_stamp":"2026-02-24T22:34:55.932092+0000","last_deep_scrub":"312'693","last_deep_scrub_stamp":"2026-02-24T22:34:55.932092+0000","last_clean_scrub_stamp":"2026-02-24T22:34:55.932092+0000","objects_scrubbed":6,"log_size":93,"log_dups_size":600,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:00:00.955174+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38403,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2221,"num_read_kb":1805,"num_write":316,"num_write_kb":280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"312'2582","reported_seq":5257,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:51.588402+0000","last_change":"2026-02-24T22:34:51.588402+0000","last_active":"2026-02-24T22:34:51.588402+0000","last_peered":"2026-02-24T22:34:51.588402+0000","last_clean":"2026-02-24T22:34:51.588402+0000","last_became_active":"2026-02-24T22:05:24.205557+0000","last_became_peered":"2026-02-24T22:05:24.205557+0000","last_unstale":"2026-02-24T22:34:51.588402+0000","last_undegraded":"2026-02-24T22:34:51.588402+0000","last_fullsized":"2026-02-24T22:34:51.588402+0000","mapping_epoch":30,"log_start":"312'2500","ondisk_log_start":"312'2500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2582","last_scrub_stamp":"2026-02-24T22:34:51.588360+0000","last_deep_scrub":"312'2582","last_deep_scrub_stamp":"2026-02-24T22:34:51.588360+0000","last_clean_scrub_stamp":"2026-02-24T22:34:51.588360+0000","objects_scrubbed":105,"log_size":82,"log_dups_size":2500,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:15:50.057193+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34698,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":6755,"num_read_kb":6753,"num_write":5061,"num_write_kb":2022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":584,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:07.610091+0000","last_change":"2026-02-24T22:35:07.610091+0000","last_active":"2026-02-24T22:35:07.610091+0000","last_peered":"2026-02-24T22:35:07.610091+0000","last_clean":"2026-02-24T22:35:07.610091+0000","last_became_active":"2026-02-24T22:05:29.651760+0000","last_became_peered":"2026-02-24T22:05:29.651760+0000","last_unstale":"2026-02-24T22:35:07.610091+0000","last_undegraded":"2026-02-24T22:35:07.610091+0000","last_fullsized":"2026-02-24T22:35:07.610091+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:07.610000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:07.610000+0000","last_clean_scrub_stamp":"2026-02-24T22:35:07.610000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:05:45.306861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"312'155","reported_seq":841,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.775671+0000","last_change":"2026-02-24T22:34:44.775671+0000","last_active":"2026-02-24T22:34:44.775671+0000","last_peered":"2026-02-24T22:34:44.775671+0000","last_clean":"2026-02-24T22:34:44.775671+0000","last_became_active":"2026-02-24T22:05:18.184939+0000","last_became_peered":"2026-02-24T22:05:18.184939+0000","last_unstale":"2026-02-24T22:34:44.775671+0000","last_undegraded":"2026-02-24T22:34:44.775671+0000","last_fullsized":"2026-02-24T22:34:44.775671+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'155","last_scrub_stamp":"2026-02-24T22:34:44.775613+0000","last_deep_scrub":"312'155","last_deep_scrub_stamp":"2026-02-24T22:34:44.775613+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.775613+0000","objects_scrubbed":27,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:46:49.648242+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":436,"num_read_kb":360,"num_write":185,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"312'291","reported_seq":1023,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.911246+0000","last_change":"2026-02-24T22:34:34.911246+0000","last_active":"2026-02-24T22:34:34.911246+0000","last_peered":"2026-02-24T22:34:34.911246+0000","last_clean":"2026-02-24T22:34:34.911246+0000","last_became_active":"2026-02-24T22:05:16.186043+0000","last_became_peered":"2026-02-24T22:05:16.186043+0000","last_unstale":"2026-02-24T22:34:34.911246+0000","last_undegraded":"2026-02-24T22:34:34.911246+0000","last_fullsized":"2026-02-24T22:34:34.911246+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'291","last_scrub_stamp":"2026-02-24T22:34:34.911183+0000","last_deep_scrub":"312'291","last_deep_scrub_stamp":"2026-02-24T22:34:34.911183+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.911183+0000","objects_scrubbed":4,"log_size":91,"log_dups_size":200,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:44:00.316854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":465,"num_read_kb":15751,"num_write":1254,"num_write_kb":5255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":598,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:57.248861+0000","last_change":"2026-02-24T22:34:57.248861+0000","last_active":"2026-02-24T22:34:57.248861+0000","last_peered":"2026-02-24T22:34:57.248861+0000","last_clean":"2026-02-24T22:34:57.248861+0000","last_became_active":"2026-02-24T22:05:20.192962+0000","last_became_peered":"2026-02-24T22:05:20.192962+0000","last_unstale":"2026-02-24T22:34:57.248861+0000","last_undegraded":"2026-02-24T22:34:57.248861+0000","last_fullsized":"2026-02-24T22:34:57.248861+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:57.248813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:57.248813+0000","last_clean_scrub_stamp":"2026-02-24T22:34:57.248813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:44:05.255470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"312'478","reported_seq":1323,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:59.612935+0000","last_change":"2026-02-24T22:34:59.612935+0000","last_active":"2026-02-24T22:34:59.612935+0000","last_peered":"2026-02-24T22:34:59.612935+0000","last_clean":"2026-02-24T22:34:59.612935+0000","last_became_active":"2026-02-24T22:05:22.193896+0000","last_became_peered":"2026-02-24T22:05:22.193896+0000","last_unstale":"2026-02-24T22:34:59.612935+0000","last_undegraded":"2026-02-24T22:34:59.612935+0000","last_fullsized":"2026-02-24T22:34:59.612935+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'478","last_scrub_stamp":"2026-02-24T22:34:59.612864+0000","last_deep_scrub":"312'478","last_deep_scrub_stamp":"2026-02-24T22:34:59.612864+0000","last_clean_scrub_stamp":"2026-02-24T22:34:59.612864+0000","objects_scrubbed":3,"log_size":78,"log_dups_size":400,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:18:12.188442+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41178,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1310,"num_read_kb":1453,"num_write":355,"num_write_kb":318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"312'282","reported_seq":908,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:02.283472+0000","last_change":"2026-02-24T22:35:02.283472+0000","last_active":"2026-02-24T22:35:02.283472+0000","last_peered":"2026-02-24T22:35:02.283472+0000","last_clean":"2026-02-24T22:35:02.283472+0000","last_became_active":"2026-02-24T22:05:24.205855+0000","last_became_peered":"2026-02-24T22:05:24.205855+0000","last_unstale":"2026-02-24T22:35:02.283472+0000","last_undegraded":"2026-02-24T22:35:02.283472+0000","last_fullsized":"2026-02-24T22:35:02.283472+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'282","last_scrub_stamp":"2026-02-24T22:35:02.283427+0000","last_deep_scrub":"312'282","last_deep_scrub_stamp":"2026-02-24T22:35:02.283427+0000","last_clean_scrub_stamp":"2026-02-24T22:35:02.283427+0000","objects_scrubbed":94,"log_size":82,"log_dups_size":200,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:30:07.155275+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27729,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":196,"num_read_kb":179,"num_write":471,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"312'449","reported_seq":1629,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:04.238088+0000","last_change":"2026-02-24T22:35:04.238088+0000","last_active":"2026-02-24T22:35:04.238088+0000","last_peered":"2026-02-24T22:35:04.238088+0000","last_clean":"2026-02-24T22:35:04.238088+0000","last_became_active":"2026-02-24T22:05:29.661983+0000","last_became_peered":"2026-02-24T22:05:29.661983+0000","last_unstale":"2026-02-24T22:35:04.238088+0000","last_undegraded":"2026-02-24T22:35:04.238088+0000","last_fullsized":"2026-02-24T22:35:04.238088+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'449","last_scrub_stamp":"2026-02-24T22:35:04.238044+0000","last_deep_scrub":"312'449","last_deep_scrub_stamp":"2026-02-24T22:35:04.238044+0000","last_clean_scrub_stamp":"2026-02-24T22:35:04.238044+0000","objects_scrubbed":1,"log_size":49,"log_dups_size":400,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:37:34.593042+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"312'191","reported_seq":883,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.478571+0000","last_change":"2026-02-24T22:34:40.478571+0000","last_active":"2026-02-24T22:34:40.478571+0000","last_peered":"2026-02-24T22:34:40.478571+0000","last_clean":"2026-02-24T22:34:40.478571+0000","last_became_active":"2026-02-24T22:05:18.189877+0000","last_became_peered":"2026-02-24T22:05:18.189877+0000","last_unstale":"2026-02-24T22:34:40.478571+0000","last_undegraded":"2026-02-24T22:34:40.478571+0000","last_fullsized":"2026-02-24T22:34:40.478571+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'191","last_scrub_stamp":"2026-02-24T22:34:40.478532+0000","last_deep_scrub":"312'191","last_deep_scrub_stamp":"2026-02-24T22:34:40.478532+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.478532+0000","objects_scrubbed":22,"log_size":91,"log_dups_size":100,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:10:11.001904+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":510,"num_write":249,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"312'332","reported_seq":1100,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.714508+0000","last_change":"2026-02-24T22:34:34.714508+0000","last_active":"2026-02-24T22:34:34.714508+0000","last_peered":"2026-02-24T22:34:34.714508+0000","last_clean":"2026-02-24T22:34:34.714508+0000","last_became_active":"2026-02-24T22:05:16.184525+0000","last_became_peered":"2026-02-24T22:05:16.184525+0000","last_unstale":"2026-02-24T22:34:34.714508+0000","last_undegraded":"2026-02-24T22:34:34.714508+0000","last_fullsized":"2026-02-24T22:34:34.714508+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'332","last_scrub_stamp":"2026-02-24T22:34:34.714447+0000","last_deep_scrub":"312'332","last_deep_scrub_stamp":"2026-02-24T22:34:34.714447+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.714447+0000","objects_scrubbed":7,"log_size":32,"log_dups_size":300,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:44:54.170379+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242941,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":576,"num_read_kb":16320,"num_write":1488,"num_write_kb":16752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"312'2","reported_seq":598,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.250464+0000","last_change":"2026-02-24T22:34:48.250464+0000","last_active":"2026-02-24T22:34:48.250464+0000","last_peered":"2026-02-24T22:34:48.250464+0000","last_clean":"2026-02-24T22:34:48.250464+0000","last_became_active":"2026-02-24T22:05:20.186850+0000","last_became_peered":"2026-02-24T22:05:20.186850+0000","last_unstale":"2026-02-24T22:34:48.250464+0000","last_undegraded":"2026-02-24T22:34:48.250464+0000","last_fullsized":"2026-02-24T22:34:48.250464+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:34:48.250415+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:34:48.250415+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.250415+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:00:47.346514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"312'253","reported_seq":884,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:59.281488+0000","last_change":"2026-02-24T22:34:59.281488+0000","last_active":"2026-02-24T22:34:59.281488+0000","last_peered":"2026-02-24T22:34:59.281488+0000","last_clean":"2026-02-24T22:34:59.281488+0000","last_became_active":"2026-02-24T22:05:22.199358+0000","last_became_peered":"2026-02-24T22:05:22.199358+0000","last_unstale":"2026-02-24T22:34:59.281488+0000","last_undegraded":"2026-02-24T22:34:59.281488+0000","last_fullsized":"2026-02-24T22:34:59.281488+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'253","last_scrub_stamp":"2026-02-24T22:34:59.281432+0000","last_deep_scrub":"312'253","last_deep_scrub_stamp":"2026-02-24T22:34:59.281432+0000","last_clean_scrub_stamp":"2026-02-24T22:34:59.281432+0000","objects_scrubbed":4,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:21:43.387290+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37935,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":401,"num_read_kb":1010,"num_write":360,"num_write_kb":308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"312'382","reported_seq":1337,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:02.816402+0000","last_change":"2026-02-24T22:35:02.816402+0000","last_active":"2026-02-24T22:35:02.816402+0000","last_peered":"2026-02-24T22:35:02.816402+0000","last_clean":"2026-02-24T22:35:02.816402+0000","last_became_active":"2026-02-24T22:05:24.199555+0000","last_became_peered":"2026-02-24T22:05:24.199555+0000","last_unstale":"2026-02-24T22:35:02.816402+0000","last_undegraded":"2026-02-24T22:35:02.816402+0000","last_fullsized":"2026-02-24T22:35:02.816402+0000","mapping_epoch":30,"log_start":"312'295","ondisk_log_start":"312'295","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'382","last_scrub_stamp":"2026-02-24T22:35:02.816332+0000","last_deep_scrub":"312'382","last_deep_scrub_stamp":"2026-02-24T22:35:02.816332+0000","last_clean_scrub_stamp":"2026-02-24T22:35:02.816332+0000","objects_scrubbed":101,"log_size":87,"log_dups_size":295,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:38:25.214542+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30969,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":655,"num_read_kb":647,"num_write":639,"num_write_kb":204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":174,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"312'433","reported_seq":1959,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:04.303436+0000","last_change":"2026-02-24T22:35:04.303436+0000","last_active":"2026-02-24T22:35:04.303436+0000","last_peered":"2026-02-24T22:35:04.303436+0000","last_clean":"2026-02-24T22:35:04.303436+0000","last_became_active":"2026-02-24T22:05:29.659532+0000","last_became_peered":"2026-02-24T22:05:29.659532+0000","last_unstale":"2026-02-24T22:35:04.303436+0000","last_undegraded":"2026-02-24T22:35:04.303436+0000","last_fullsized":"2026-02-24T22:35:04.303436+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'433","last_scrub_stamp":"2026-02-24T22:35:04.303372+0000","last_deep_scrub":"312'433","last_deep_scrub_stamp":"2026-02-24T22:35:04.303372+0000","last_clean_scrub_stamp":"2026-02-24T22:35:04.303372+0000","objects_scrubbed":1,"log_size":33,"log_dups_size":400,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:32:29.785307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"312'162","reported_seq":857,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.157775+0000","last_change":"2026-02-24T22:34:45.157775+0000","last_active":"2026-02-24T22:34:45.157775+0000","last_peered":"2026-02-24T22:34:45.157775+0000","last_clean":"2026-02-24T22:34:45.157775+0000","last_became_active":"2026-02-24T22:05:18.184250+0000","last_became_peered":"2026-02-24T22:05:18.184250+0000","last_unstale":"2026-02-24T22:34:45.157775+0000","last_undegraded":"2026-02-24T22:34:45.157775+0000","last_fullsized":"2026-02-24T22:34:45.157775+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'162","last_scrub_stamp":"2026-02-24T22:34:45.157726+0000","last_deep_scrub":"312'162","last_deep_scrub_stamp":"2026-02-24T22:34:45.157726+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.157726+0000","objects_scrubbed":23,"log_size":62,"log_dups_size":100,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:39:01.351061+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":560,"num_read_kb":462,"num_write":204,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"312'1610","reported_seq":2974,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.774854+0000","last_change":"2026-02-24T22:34:33.774854+0000","last_active":"2026-02-24T22:34:33.774854+0000","last_peered":"2026-02-24T22:34:33.774854+0000","last_clean":"2026-02-24T22:34:33.774854+0000","last_became_active":"2026-02-24T22:05:16.189660+0000","last_became_peered":"2026-02-24T22:05:16.189660+0000","last_unstale":"2026-02-24T22:34:33.774854+0000","last_undegraded":"2026-02-24T22:34:33.774854+0000","last_fullsized":"2026-02-24T22:34:33.774854+0000","mapping_epoch":22,"log_start":"312'1500","ondisk_log_start":"312'1500","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'1610","last_scrub_stamp":"2026-02-24T22:34:33.774776+0000","last_deep_scrub":"312'1610","last_deep_scrub_stamp":"2026-02-24T22:34:33.774776+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.774776+0000","objects_scrubbed":6,"log_size":110,"log_dups_size":1500,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:08:14.555476+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4326,"num_read_kb":2601,"num_write":2684,"num_write_kb":4261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"312'121","reported_seq":808,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.128824+0000","last_change":"2026-02-24T22:34:43.128824+0000","last_active":"2026-02-24T22:34:43.128824+0000","last_peered":"2026-02-24T22:34:43.128824+0000","last_clean":"2026-02-24T22:34:43.128824+0000","last_became_active":"2026-02-24T22:05:18.183822+0000","last_became_peered":"2026-02-24T22:05:18.183822+0000","last_unstale":"2026-02-24T22:34:43.128824+0000","last_undegraded":"2026-02-24T22:34:43.128824+0000","last_fullsized":"2026-02-24T22:34:43.128824+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'121","last_scrub_stamp":"2026-02-24T22:34:43.128792+0000","last_deep_scrub":"312'121","last_deep_scrub_stamp":"2026-02-24T22:34:43.128792+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.128792+0000","objects_scrubbed":20,"log_size":21,"log_dups_size":100,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:33:44.340989+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":390,"num_read_kb":330,"num_write":143,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"312'303","reported_seq":1060,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.123769+0000","last_change":"2026-02-24T22:34:32.123769+0000","last_active":"2026-02-24T22:34:32.123769+0000","last_peered":"2026-02-24T22:34:32.123769+0000","last_clean":"2026-02-24T22:34:32.123769+0000","last_became_active":"2026-02-24T22:05:16.183487+0000","last_became_peered":"2026-02-24T22:05:16.183487+0000","last_unstale":"2026-02-24T22:34:32.123769+0000","last_undegraded":"2026-02-24T22:34:32.123769+0000","last_fullsized":"2026-02-24T22:34:32.123769+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'303","last_scrub_stamp":"2026-02-24T22:34:32.123701+0000","last_deep_scrub":"312'303","last_deep_scrub_stamp":"2026-02-24T22:34:32.123701+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.123701+0000","objects_scrubbed":3,"log_size":103,"log_dups_size":200,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:46:27.988906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":514,"num_read_kb":706,"num_write":1357,"num_write_kb":183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"312'92","reported_seq":760,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.957519+0000","last_change":"2026-02-24T22:34:44.957519+0000","last_active":"2026-02-24T22:34:44.957519+0000","last_peered":"2026-02-24T22:34:44.957519+0000","last_clean":"2026-02-24T22:34:44.957519+0000","last_became_active":"2026-02-24T22:05:18.190352+0000","last_became_peered":"2026-02-24T22:05:18.190352+0000","last_unstale":"2026-02-24T22:34:44.957519+0000","last_undegraded":"2026-02-24T22:34:44.957519+0000","last_fullsized":"2026-02-24T22:34:44.957519+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'92","last_scrub_stamp":"2026-02-24T22:34:44.957460+0000","last_deep_scrub":"312'92","last_deep_scrub_stamp":"2026-02-24T22:34:44.957460+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.957460+0000","objects_scrubbed":20,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:57:33.270079+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":274,"num_read_kb":236,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"312'318","reported_seq":1249,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.115775+0000","last_change":"2026-02-24T22:34:38.115775+0000","last_active":"2026-02-24T22:34:38.115775+0000","last_peered":"2026-02-24T22:34:38.115775+0000","last_clean":"2026-02-24T22:34:38.115775+0000","last_became_active":"2026-02-24T22:05:16.190494+0000","last_became_peered":"2026-02-24T22:05:16.190494+0000","last_unstale":"2026-02-24T22:34:38.115775+0000","last_undegraded":"2026-02-24T22:34:38.115775+0000","last_fullsized":"2026-02-24T22:34:38.115775+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'318","last_scrub_stamp":"2026-02-24T22:34:38.115706+0000","last_deep_scrub":"312'318","last_deep_scrub_stamp":"2026-02-24T22:34:38.115706+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.115706+0000","objects_scrubbed":4,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:08:06.722436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1108,"num_read_kb":14913,"num_write":1331,"num_write_kb":8068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"312'367","reported_seq":1190,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.110977+0000","last_change":"2026-02-24T22:34:41.110977+0000","last_active":"2026-02-24T22:34:41.110977+0000","last_peered":"2026-02-24T22:34:41.110977+0000","last_clean":"2026-02-24T22:34:41.110977+0000","last_became_active":"2026-02-24T22:05:18.186622+0000","last_became_peered":"2026-02-24T22:05:18.186622+0000","last_unstale":"2026-02-24T22:34:41.110977+0000","last_undegraded":"2026-02-24T22:34:41.110977+0000","last_fullsized":"2026-02-24T22:34:41.110977+0000","mapping_epoch":24,"log_start":"312'300","ondisk_log_start":"312'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'367","last_scrub_stamp":"2026-02-24T22:34:41.110934+0000","last_deep_scrub":"312'367","last_deep_scrub_stamp":"2026-02-24T22:34:41.110934+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.110934+0000","objects_scrubbed":30,"log_size":67,"log_dups_size":300,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:59:07.931667+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1512,"num_read_kb":1316,"num_write":481,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25750,"num_omap_keys":101,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"312'319","reported_seq":1167,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.742796+0000","last_change":"2026-02-24T22:34:36.742796+0000","last_active":"2026-02-24T22:34:36.742796+0000","last_peered":"2026-02-24T22:34:36.742796+0000","last_clean":"2026-02-24T22:34:36.742796+0000","last_became_active":"2026-02-24T22:05:16.185977+0000","last_became_peered":"2026-02-24T22:05:16.185977+0000","last_unstale":"2026-02-24T22:34:36.742796+0000","last_undegraded":"2026-02-24T22:34:36.742796+0000","last_fullsized":"2026-02-24T22:34:36.742796+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'319","last_scrub_stamp":"2026-02-24T22:34:36.742708+0000","last_deep_scrub":"312'319","last_deep_scrub_stamp":"2026-02-24T22:34:36.742708+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.742708+0000","objects_scrubbed":1,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:21:32.964403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":948,"num_read_kb":5969,"num_write":1368,"num_write_kb":2943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"312'114","reported_seq":799,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.783698+0000","last_change":"2026-02-24T22:34:46.783698+0000","last_active":"2026-02-24T22:34:46.783698+0000","last_peered":"2026-02-24T22:34:46.783698+0000","last_clean":"2026-02-24T22:34:46.783698+0000","last_became_active":"2026-02-24T22:05:18.185180+0000","last_became_peered":"2026-02-24T22:05:18.185180+0000","last_unstale":"2026-02-24T22:34:46.783698+0000","last_undegraded":"2026-02-24T22:34:46.783698+0000","last_fullsized":"2026-02-24T22:34:46.783698+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'114","last_scrub_stamp":"2026-02-24T22:34:46.783630+0000","last_deep_scrub":"312'114","last_deep_scrub_stamp":"2026-02-24T22:34:46.783630+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.783630+0000","objects_scrubbed":25,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:34:22.929604+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":311,"num_read_kb":264,"num_write":113,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":272,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"312'374","reported_seq":1134,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.080105+0000","last_change":"2026-02-24T22:34:34.080105+0000","last_active":"2026-02-24T22:34:34.080105+0000","last_peered":"2026-02-24T22:34:34.080105+0000","last_clean":"2026-02-24T22:34:34.080105+0000","last_became_active":"2026-02-24T22:06:00.660740+0000","last_became_peered":"2026-02-24T22:06:00.660740+0000","last_unstale":"2026-02-24T22:34:34.080105+0000","last_undegraded":"2026-02-24T22:34:34.080105+0000","last_fullsized":"2026-02-24T22:34:34.080105+0000","mapping_epoch":36,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'374","last_scrub_stamp":"2026-02-24T22:34:34.080040+0000","last_deep_scrub":"312'374","last_deep_scrub_stamp":"2026-02-24T22:34:34.080040+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.080040+0000","objects_scrubbed":4,"log_size":74,"log_dups_size":300,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:17:10.260943+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":666,"num_read_kb":9895,"num_write":1405,"num_write_kb":11975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"312'89","reported_seq":768,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.509973+0000","last_change":"2026-02-24T22:34:43.509973+0000","last_active":"2026-02-24T22:34:43.509973+0000","last_peered":"2026-02-24T22:34:43.509973+0000","last_clean":"2026-02-24T22:34:43.509973+0000","last_became_active":"2026-02-24T22:05:18.190390+0000","last_became_peered":"2026-02-24T22:05:18.190390+0000","last_unstale":"2026-02-24T22:34:43.509973+0000","last_undegraded":"2026-02-24T22:34:43.509973+0000","last_fullsized":"2026-02-24T22:34:43.509973+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'89","last_scrub_stamp":"2026-02-24T22:34:43.509906+0000","last_deep_scrub":"312'89","last_deep_scrub_stamp":"2026-02-24T22:34:43.509906+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.509906+0000","objects_scrubbed":23,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:26:36.232359+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":213,"num_read_kb":187,"num_write":90,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"312'336","reported_seq":1067,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.988612+0000","last_change":"2026-02-24T22:34:34.988612+0000","last_active":"2026-02-24T22:34:34.988612+0000","last_peered":"2026-02-24T22:34:34.988612+0000","last_clean":"2026-02-24T22:34:34.988612+0000","last_became_active":"2026-02-24T22:05:16.184091+0000","last_became_peered":"2026-02-24T22:05:16.184091+0000","last_unstale":"2026-02-24T22:34:34.988612+0000","last_undegraded":"2026-02-24T22:34:34.988612+0000","last_fullsized":"2026-02-24T22:34:34.988612+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'336","last_scrub_stamp":"2026-02-24T22:34:34.988543+0000","last_deep_scrub":"312'336","last_deep_scrub_stamp":"2026-02-24T22:34:34.988543+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.988543+0000","objects_scrubbed":4,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:48:27.381128+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":558,"num_read_kb":406,"num_write":1237,"num_write_kb":6221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"312'375","reported_seq":1187,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.800000+0000","last_change":"2026-02-24T22:34:45.800000+0000","last_active":"2026-02-24T22:34:45.800000+0000","last_peered":"2026-02-24T22:34:45.800000+0000","last_clean":"2026-02-24T22:34:45.800000+0000","last_became_active":"2026-02-24T22:05:18.186746+0000","last_became_peered":"2026-02-24T22:05:18.186746+0000","last_unstale":"2026-02-24T22:34:45.800000+0000","last_undegraded":"2026-02-24T22:34:45.800000+0000","last_fullsized":"2026-02-24T22:34:45.800000+0000","mapping_epoch":24,"log_start":"312'300","ondisk_log_start":"312'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'375","last_scrub_stamp":"2026-02-24T22:34:45.799935+0000","last_deep_scrub":"312'375","last_deep_scrub_stamp":"2026-02-24T22:34:45.799935+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.799935+0000","objects_scrubbed":20,"log_size":75,"log_dups_size":300,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:08:40.366084+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":1676,"num_read_kb":1392,"num_write":638,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"312'331","reported_seq":1043,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.774985+0000","last_change":"2026-02-24T22:34:34.774985+0000","last_active":"2026-02-24T22:34:34.774985+0000","last_peered":"2026-02-24T22:34:34.774985+0000","last_clean":"2026-02-24T22:34:34.774985+0000","last_became_active":"2026-02-24T22:05:16.189516+0000","last_became_peered":"2026-02-24T22:05:16.189516+0000","last_unstale":"2026-02-24T22:34:34.774985+0000","last_undegraded":"2026-02-24T22:34:34.774985+0000","last_fullsized":"2026-02-24T22:34:34.774985+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'331","last_scrub_stamp":"2026-02-24T22:34:34.774895+0000","last_deep_scrub":"312'331","last_deep_scrub_stamp":"2026-02-24T22:34:34.774895+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.774895+0000","objects_scrubbed":2,"log_size":31,"log_dups_size":300,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:51:58.631151+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":448,"num_read_kb":5491,"num_write":1228,"num_write_kb":20654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"312'184","reported_seq":884,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.708693+0000","last_change":"2026-02-24T22:34:40.708693+0000","last_active":"2026-02-24T22:34:40.708693+0000","last_peered":"2026-02-24T22:34:40.708693+0000","last_clean":"2026-02-24T22:34:40.708693+0000","last_became_active":"2026-02-24T22:05:18.190203+0000","last_became_peered":"2026-02-24T22:05:18.190203+0000","last_unstale":"2026-02-24T22:34:40.708693+0000","last_undegraded":"2026-02-24T22:34:40.708693+0000","last_fullsized":"2026-02-24T22:34:40.708693+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'184","last_scrub_stamp":"2026-02-24T22:34:40.708623+0000","last_deep_scrub":"312'184","last_deep_scrub_stamp":"2026-02-24T22:34:40.708623+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.708623+0000","objects_scrubbed":20,"log_size":84,"log_dups_size":100,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:12:32.032756+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":631,"num_read_kb":516,"num_write":248,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"312'319","reported_seq":1046,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.869813+0000","last_change":"2026-02-24T22:34:35.869813+0000","last_active":"2026-02-24T22:34:35.869813+0000","last_peered":"2026-02-24T22:34:35.869813+0000","last_clean":"2026-02-24T22:34:35.869813+0000","last_became_active":"2026-02-24T22:05:16.183772+0000","last_became_peered":"2026-02-24T22:05:16.183772+0000","last_unstale":"2026-02-24T22:34:35.869813+0000","last_undegraded":"2026-02-24T22:34:35.869813+0000","last_fullsized":"2026-02-24T22:34:35.869813+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'319","last_scrub_stamp":"2026-02-24T22:34:35.869749+0000","last_deep_scrub":"312'319","last_deep_scrub_stamp":"2026-02-24T22:34:35.869749+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.869749+0000","objects_scrubbed":2,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:10:38.609121+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":480,"num_read_kb":12489,"num_write":1341,"num_write_kb":5970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"312'156","reported_seq":885,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.880154+0000","last_change":"2026-02-24T22:34:45.880154+0000","last_active":"2026-02-24T22:34:45.880154+0000","last_peered":"2026-02-24T22:34:45.880154+0000","last_clean":"2026-02-24T22:34:45.880154+0000","last_became_active":"2026-02-24T22:05:18.187321+0000","last_became_peered":"2026-02-24T22:05:18.187321+0000","last_unstale":"2026-02-24T22:34:45.880154+0000","last_undegraded":"2026-02-24T22:34:45.880154+0000","last_fullsized":"2026-02-24T22:34:45.880154+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'156","last_scrub_stamp":"2026-02-24T22:34:45.880085+0000","last_deep_scrub":"312'156","last_deep_scrub_stamp":"2026-02-24T22:34:45.880085+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.880085+0000","objects_scrubbed":26,"log_size":56,"log_dups_size":100,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:17:02.414903+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":512,"num_read_kb":441,"num_write":201,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"312'326","reported_seq":1042,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.166643+0000","last_change":"2026-02-24T22:34:37.166643+0000","last_active":"2026-02-24T22:34:37.166643+0000","last_peered":"2026-02-24T22:34:37.166643+0000","last_clean":"2026-02-24T22:34:37.166643+0000","last_became_active":"2026-02-24T22:05:16.193164+0000","last_became_peered":"2026-02-24T22:05:16.193164+0000","last_unstale":"2026-02-24T22:34:37.166643+0000","last_undegraded":"2026-02-24T22:34:37.166643+0000","last_fullsized":"2026-02-24T22:34:37.166643+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'326","last_scrub_stamp":"2026-02-24T22:34:37.166498+0000","last_deep_scrub":"312'326","last_deep_scrub_stamp":"2026-02-24T22:34:37.166498+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.166498+0000","objects_scrubbed":6,"log_size":26,"log_dups_size":300,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:54:53.815454+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242922,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":434,"num_read_kb":36424,"num_write":1378,"num_write_kb":10617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"312'166","reported_seq":852,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.417427+0000","last_change":"2026-02-24T22:34:46.417427+0000","last_active":"2026-02-24T22:34:46.417427+0000","last_peered":"2026-02-24T22:34:46.417427+0000","last_clean":"2026-02-24T22:34:46.417427+0000","last_became_active":"2026-02-24T22:05:18.191737+0000","last_became_peered":"2026-02-24T22:05:18.191737+0000","last_unstale":"2026-02-24T22:34:46.417427+0000","last_undegraded":"2026-02-24T22:34:46.417427+0000","last_fullsized":"2026-02-24T22:34:46.417427+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'166","last_scrub_stamp":"2026-02-24T22:34:46.417398+0000","last_deep_scrub":"312'166","last_deep_scrub_stamp":"2026-02-24T22:34:46.417398+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.417398+0000","objects_scrubbed":18,"log_size":66,"log_dups_size":100,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:30:36.821543+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":539,"num_read_kb":448,"num_write":211,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"312'377","reported_seq":1137,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.766166+0000","last_change":"2026-02-24T22:34:38.766166+0000","last_active":"2026-02-24T22:34:38.766166+0000","last_peered":"2026-02-24T22:34:38.766166+0000","last_clean":"2026-02-24T22:34:38.766166+0000","last_became_active":"2026-02-24T22:05:16.183923+0000","last_became_peered":"2026-02-24T22:05:16.183923+0000","last_unstale":"2026-02-24T22:34:38.766166+0000","last_undegraded":"2026-02-24T22:34:38.766166+0000","last_fullsized":"2026-02-24T22:34:38.766166+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'377","last_scrub_stamp":"2026-02-24T22:34:38.766121+0000","last_deep_scrub":"312'377","last_deep_scrub_stamp":"2026-02-24T22:34:38.766121+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.766121+0000","objects_scrubbed":6,"log_size":77,"log_dups_size":300,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:41:28.025068+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242922,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":549,"num_read_kb":9842,"num_write":1598,"num_write_kb":9073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"312'167","reported_seq":848,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.060961+0000","last_change":"2026-02-24T22:34:39.060923+0000","last_active":"2026-02-24T22:34:39.060961+0000","last_peered":"2026-02-24T22:34:39.060961+0000","last_clean":"2026-02-24T22:34:39.060961+0000","last_became_active":"2026-02-24T22:05:18.183761+0000","last_became_peered":"2026-02-24T22:05:18.183761+0000","last_unstale":"2026-02-24T22:34:39.060961+0000","last_undegraded":"2026-02-24T22:34:39.060961+0000","last_fullsized":"2026-02-24T22:34:39.060961+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'167","last_scrub_stamp":"2026-02-24T22:34:39.060863+0000","last_deep_scrub":"312'167","last_deep_scrub_stamp":"2026-02-24T22:34:39.060863+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.060863+0000","objects_scrubbed":21,"log_size":67,"log_dups_size":100,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:07:32.105462+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":561,"num_read_kb":451,"num_write":221,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"312'341","reported_seq":1119,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.077639+0000","last_change":"2026-02-24T22:34:39.077639+0000","last_active":"2026-02-24T22:34:39.077639+0000","last_peered":"2026-02-24T22:34:39.077639+0000","last_clean":"2026-02-24T22:34:39.077639+0000","last_became_active":"2026-02-24T22:05:16.189759+0000","last_became_peered":"2026-02-24T22:05:16.189759+0000","last_unstale":"2026-02-24T22:34:39.077639+0000","last_undegraded":"2026-02-24T22:34:39.077639+0000","last_fullsized":"2026-02-24T22:34:39.077639+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'341","last_scrub_stamp":"2026-02-24T22:34:39.077571+0000","last_deep_scrub":"312'341","last_deep_scrub_stamp":"2026-02-24T22:34:39.077571+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.077571+0000","objects_scrubbed":4,"log_size":41,"log_dups_size":300,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:09:54.075658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":656,"num_read_kb":3529,"num_write":1503,"num_write_kb":3094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"312'158","reported_seq":860,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.440468+0000","last_change":"2026-02-24T22:34:45.440468+0000","last_active":"2026-02-24T22:34:45.440468+0000","last_peered":"2026-02-24T22:34:45.440468+0000","last_clean":"2026-02-24T22:34:45.440468+0000","last_became_active":"2026-02-24T22:05:18.192882+0000","last_became_peered":"2026-02-24T22:05:18.192882+0000","last_unstale":"2026-02-24T22:34:45.440468+0000","last_undegraded":"2026-02-24T22:34:45.440468+0000","last_fullsized":"2026-02-24T22:34:45.440468+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'158","last_scrub_stamp":"2026-02-24T22:34:45.440425+0000","last_deep_scrub":"312'158","last_deep_scrub_stamp":"2026-02-24T22:34:45.440425+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.440425+0000","objects_scrubbed":18,"log_size":58,"log_dups_size":100,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:49:26.113842+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":593,"num_read_kb":494,"num_write":208,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"312'309","reported_seq":1056,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.732158+0000","last_change":"2026-02-24T22:34:35.732158+0000","last_active":"2026-02-24T22:34:35.732158+0000","last_peered":"2026-02-24T22:34:35.732158+0000","last_clean":"2026-02-24T22:34:35.732158+0000","last_became_active":"2026-02-24T22:05:16.184557+0000","last_became_peered":"2026-02-24T22:05:16.184557+0000","last_unstale":"2026-02-24T22:34:35.732158+0000","last_undegraded":"2026-02-24T22:34:35.732158+0000","last_fullsized":"2026-02-24T22:34:35.732158+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'309","last_scrub_stamp":"2026-02-24T22:34:35.732082+0000","last_deep_scrub":"312'309","last_deep_scrub_stamp":"2026-02-24T22:34:35.732082+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.732082+0000","objects_scrubbed":3,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:22:57.505362+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":498,"num_read_kb":1536,"num_write":1259,"num_write_kb":5590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"312'184","reported_seq":901,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.156109+0000","last_change":"2026-02-24T22:34:45.156109+0000","last_active":"2026-02-24T22:34:45.156109+0000","last_peered":"2026-02-24T22:34:45.156109+0000","last_clean":"2026-02-24T22:34:45.156109+0000","last_became_active":"2026-02-24T22:05:18.191432+0000","last_became_peered":"2026-02-24T22:05:18.191432+0000","last_unstale":"2026-02-24T22:34:45.156109+0000","last_undegraded":"2026-02-24T22:34:45.156109+0000","last_fullsized":"2026-02-24T22:34:45.156109+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'184","last_scrub_stamp":"2026-02-24T22:34:45.156041+0000","last_deep_scrub":"312'184","last_deep_scrub_stamp":"2026-02-24T22:34:45.156041+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.156041+0000","objects_scrubbed":26,"log_size":84,"log_dups_size":100,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:54:34.074113+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":649,"num_read_kb":535,"num_write":233,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4005,"num_omap_keys":18,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"312'287","reported_seq":1017,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.765441+0000","last_change":"2026-02-24T22:34:36.765441+0000","last_active":"2026-02-24T22:34:36.765441+0000","last_peered":"2026-02-24T22:34:36.765441+0000","last_clean":"2026-02-24T22:34:36.765441+0000","last_became_active":"2026-02-24T22:05:16.190260+0000","last_became_peered":"2026-02-24T22:05:16.190260+0000","last_unstale":"2026-02-24T22:34:36.765441+0000","last_undegraded":"2026-02-24T22:34:36.765441+0000","last_fullsized":"2026-02-24T22:34:36.765441+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'287","last_scrub_stamp":"2026-02-24T22:34:36.765378+0000","last_deep_scrub":"312'287","last_deep_scrub_stamp":"2026-02-24T22:34:36.765378+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.765378+0000","objects_scrubbed":2,"log_size":87,"log_dups_size":200,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:18:33.840972+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":458,"num_read_kb":1504,"num_write":1357,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"312'171","reported_seq":869,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.861467+0000","last_change":"2026-02-24T22:34:44.861467+0000","last_active":"2026-02-24T22:34:44.861467+0000","last_peered":"2026-02-24T22:34:44.861467+0000","last_clean":"2026-02-24T22:34:44.861467+0000","last_became_active":"2026-02-24T22:05:18.185184+0000","last_became_peered":"2026-02-24T22:05:18.185184+0000","last_unstale":"2026-02-24T22:34:44.861467+0000","last_undegraded":"2026-02-24T22:34:44.861467+0000","last_fullsized":"2026-02-24T22:34:44.861467+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'171","last_scrub_stamp":"2026-02-24T22:34:44.861426+0000","last_deep_scrub":"312'171","last_deep_scrub_stamp":"2026-02-24T22:34:44.861426+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.861426+0000","objects_scrubbed":28,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:45:55.477850+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":536,"num_read_kb":445,"num_write":200,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"312'302","reported_seq":1124,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.033851+0000","last_change":"2026-02-24T22:34:40.033851+0000","last_active":"2026-02-24T22:34:40.033851+0000","last_peered":"2026-02-24T22:34:40.033851+0000","last_clean":"2026-02-24T22:34:40.033851+0000","last_became_active":"2026-02-24T22:05:16.191433+0000","last_became_peered":"2026-02-24T22:05:16.191433+0000","last_unstale":"2026-02-24T22:34:40.033851+0000","last_undegraded":"2026-02-24T22:34:40.033851+0000","last_fullsized":"2026-02-24T22:34:40.033851+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'302","last_scrub_stamp":"2026-02-24T22:34:40.033782+0000","last_deep_scrub":"312'302","last_deep_scrub_stamp":"2026-02-24T22:34:40.033782+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.033782+0000","objects_scrubbed":3,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:16:57.803888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":549,"num_read_kb":2264,"num_write":1330,"num_write_kb":7006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"312'86","reported_seq":755,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.772530+0000","last_change":"2026-02-24T22:34:48.772530+0000","last_active":"2026-02-24T22:34:48.772530+0000","last_peered":"2026-02-24T22:34:48.772530+0000","last_clean":"2026-02-24T22:34:48.772530+0000","last_became_active":"2026-02-24T22:05:18.192619+0000","last_became_peered":"2026-02-24T22:05:18.192619+0000","last_unstale":"2026-02-24T22:34:48.772530+0000","last_undegraded":"2026-02-24T22:34:48.772530+0000","last_fullsized":"2026-02-24T22:34:48.772530+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'86","last_scrub_stamp":"2026-02-24T22:34:48.772479+0000","last_deep_scrub":"312'86","last_deep_scrub_stamp":"2026-02-24T22:34:48.772479+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.772479+0000","objects_scrubbed":24,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:50:44.256005+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":204,"num_read_kb":176,"num_write":76,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"312'336","reported_seq":1076,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.091228+0000","last_change":"2026-02-24T22:34:33.091228+0000","last_active":"2026-02-24T22:34:33.091228+0000","last_peered":"2026-02-24T22:34:33.091228+0000","last_clean":"2026-02-24T22:34:33.091228+0000","last_became_active":"2026-02-24T22:05:16.184441+0000","last_became_peered":"2026-02-24T22:05:16.184441+0000","last_unstale":"2026-02-24T22:34:33.091228+0000","last_undegraded":"2026-02-24T22:34:33.091228+0000","last_fullsized":"2026-02-24T22:34:33.091228+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'336","last_scrub_stamp":"2026-02-24T22:34:33.091160+0000","last_deep_scrub":"312'336","last_deep_scrub_stamp":"2026-02-24T22:34:33.091160+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.091160+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:19:40.057693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":478,"num_read_kb":3856,"num_write":1468,"num_write_kb":8445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"312'183","reported_seq":909,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.176877+0000","last_change":"2026-02-24T22:34:48.176877+0000","last_active":"2026-02-24T22:34:48.176877+0000","last_peered":"2026-02-24T22:34:48.176877+0000","last_clean":"2026-02-24T22:34:48.176877+0000","last_became_active":"2026-02-24T22:05:18.191350+0000","last_became_peered":"2026-02-24T22:05:18.191350+0000","last_unstale":"2026-02-24T22:34:48.176877+0000","last_undegraded":"2026-02-24T22:34:48.176877+0000","last_fullsized":"2026-02-24T22:34:48.176877+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'183","last_scrub_stamp":"2026-02-24T22:34:48.176839+0000","last_deep_scrub":"312'183","last_deep_scrub_stamp":"2026-02-24T22:34:48.176839+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.176839+0000","objects_scrubbed":24,"log_size":83,"log_dups_size":100,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:23:04.478635+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":662,"num_read_kb":557,"num_write":245,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"312'305","reported_seq":1081,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.042678+0000","last_change":"2026-02-24T22:34:35.042678+0000","last_active":"2026-02-24T22:34:35.042678+0000","last_peered":"2026-02-24T22:34:35.042678+0000","last_clean":"2026-02-24T22:34:35.042678+0000","last_became_active":"2026-02-24T22:06:00.662656+0000","last_became_peered":"2026-02-24T22:06:00.662656+0000","last_unstale":"2026-02-24T22:34:35.042678+0000","last_undegraded":"2026-02-24T22:34:35.042678+0000","last_fullsized":"2026-02-24T22:34:35.042678+0000","mapping_epoch":36,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'305","last_scrub_stamp":"2026-02-24T22:34:35.042539+0000","last_deep_scrub":"312'305","last_deep_scrub_stamp":"2026-02-24T22:34:35.042539+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.042539+0000","objects_scrubbed":4,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:24:23.913211+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":803,"num_read_kb":6090,"num_write":1203,"num_write_kb":5556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.30","version":"312'347","reported_seq":1107,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.049845+0000","last_change":"2026-02-24T22:34:40.049845+0000","last_active":"2026-02-24T22:34:40.049845+0000","last_peered":"2026-02-24T22:34:40.049845+0000","last_clean":"2026-02-24T22:34:40.049845+0000","last_became_active":"2026-02-24T22:05:16.189713+0000","last_became_peered":"2026-02-24T22:05:16.189713+0000","last_unstale":"2026-02-24T22:34:40.049845+0000","last_undegraded":"2026-02-24T22:34:40.049845+0000","last_fullsized":"2026-02-24T22:34:40.049845+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'347","last_scrub_stamp":"2026-02-24T22:34:40.049764+0000","last_deep_scrub":"312'347","last_deep_scrub_stamp":"2026-02-24T22:34:40.049764+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.049764+0000","objects_scrubbed":8,"log_size":47,"log_dups_size":300,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:52:46.777711+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242932,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":632,"num_read_kb":31410,"num_write":1455,"num_write_kb":10638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"312'273","reported_seq":967,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.873634+0000","last_change":"2026-02-24T22:34:46.873634+0000","last_active":"2026-02-24T22:34:46.873634+0000","last_peered":"2026-02-24T22:34:46.873634+0000","last_clean":"2026-02-24T22:34:46.873634+0000","last_became_active":"2026-02-24T22:05:18.187565+0000","last_became_peered":"2026-02-24T22:05:18.187565+0000","last_unstale":"2026-02-24T22:34:46.873634+0000","last_undegraded":"2026-02-24T22:34:46.873634+0000","last_fullsized":"2026-02-24T22:34:46.873634+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'273","last_scrub_stamp":"2026-02-24T22:34:46.873575+0000","last_deep_scrub":"312'273","last_deep_scrub_stamp":"2026-02-24T22:34:46.873575+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.873575+0000","objects_scrubbed":17,"log_size":73,"log_dups_size":200,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:03:00.743450+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":1060,"num_read_kb":853,"num_write":419,"num_write_kb":222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"312'299","reported_seq":1013,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.068984+0000","last_change":"2026-02-24T22:34:37.068984+0000","last_active":"2026-02-24T22:34:37.068984+0000","last_peered":"2026-02-24T22:34:37.068984+0000","last_clean":"2026-02-24T22:34:37.068984+0000","last_became_active":"2026-02-24T22:05:16.178157+0000","last_became_peered":"2026-02-24T22:05:16.178157+0000","last_unstale":"2026-02-24T22:34:37.068984+0000","last_undegraded":"2026-02-24T22:34:37.068984+0000","last_fullsized":"2026-02-24T22:34:37.068984+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'299","last_scrub_stamp":"2026-02-24T22:34:37.068917+0000","last_deep_scrub":"312'299","last_deep_scrub_stamp":"2026-02-24T22:34:37.068917+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.068917+0000","objects_scrubbed":5,"log_size":99,"log_dups_size":200,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:17:09.219950+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":466,"num_read_kb":1640,"num_write":1134,"num_write_kb":1702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"312'98","reported_seq":781,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.871267+0000","last_change":"2026-02-24T22:34:47.871267+0000","last_active":"2026-02-24T22:34:47.871267+0000","last_peered":"2026-02-24T22:34:47.871267+0000","last_clean":"2026-02-24T22:34:47.871267+0000","last_became_active":"2026-02-24T22:05:18.190902+0000","last_became_peered":"2026-02-24T22:05:18.190902+0000","last_unstale":"2026-02-24T22:34:47.871267+0000","last_undegraded":"2026-02-24T22:34:47.871267+0000","last_fullsized":"2026-02-24T22:34:47.871267+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'98","last_scrub_stamp":"2026-02-24T22:34:47.871207+0000","last_deep_scrub":"312'98","last_deep_scrub_stamp":"2026-02-24T22:34:47.871207+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.871207+0000","objects_scrubbed":20,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:26:15.293776+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":302,"num_read_kb":262,"num_write":98,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":433,"ondisk_log_size":433,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":973109,"num_objects":3053,"num_object_clones":0,"num_object_copies":6106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3053,"num_whiteouts":0,"num_read":19978,"num_read_kb":245567,"num_write":23762,"num_write_kb":8619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":276106,"num_omap_keys":1511,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24911872,"data_stored":1946218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5786661,"internal_metadata":0},"log_size":2158,"ondisk_log_size":2158,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1250269,"num_objects":223,"num_object_clones":0,"num_object_copies":446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":223,"num_whiteouts":0,"num_read":45268,"num_read_kb":47987,"num_write":15451,"num_write_kb":13662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":962652,"num_omap_keys":4046,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":17220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2823344,"internal_metadata":0},"log_size":2038,"ondisk_log_size":2038,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1896,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1448,"num_read_kb":1680,"num_write":22,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":64,"acting":64,"num_store_stats":8},{"poolid":82,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":376,"num_omap_keys":4,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11005,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":1511,"num_object_clones":0,"num_object_copies":3022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1511,"num_whiteouts":0,"num_read":95919,"num_read_kb":81897,"num_write":34645,"num_write_kb":17078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1511,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":35257,"num_omap_keys":142,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17301930,"internal_metadata":0},"log_size":4333,"ondisk_log_size":4333,"up":128,"acting":128,"num_store_stats":8},{"poolid":81,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":119816,"num_write":82,"num_write_kb":70656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":112199743,"num_objects":281,"num_object_clones":0,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":0,"num_read":41102,"num_read_kb":763537,"num_write":86142,"num_write_kb":523059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":225296384,"data_stored":224399486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4246,"ondisk_log_size":4246,"up":128,"acting":128,"num_store_stats":8},{"poolid":80,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":124936,"num_write":82,"num_write_kb":70656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":7},{"poolid":79,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3567,"num_read_kb":2401,"num_write":1127,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":317720,"internal_metadata":0},"log_size":462,"ondisk_log_size":462,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14,"ondisk_log_size":14,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575211,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316704,"kb_used_data":55120,"kb_used_omap":3696,"kb_used_meta":257871,"kb_avail":94055136,"statfs":{"total":96636764160,"available":96312459264,"internally_reserved":0,"allocated":56442880,"data_stored":51634967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3785637,"internal_metadata":264059995},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575211,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":271964,"kb_used_data":28428,"kb_used_omap":3571,"kb_used_meta":239948,"kb_avail":94099876,"statfs":{"total":96636764160,"available":96358273024,"internally_reserved":0,"allocated":29110272,"data_stored":23825325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3657488,"internal_metadata":245706992},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575211,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137044,"kb_used_data":37956,"kb_used_omap":2746,"kb_used_meta":96325,"kb_avail":94234796,"statfs":{"total":96636764160,"available":96496431104,"internally_reserved":0,"allocated":38866944,"data_stored":35212820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2812671,"internal_metadata":98637057},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575211,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":111224,"kb_used_data":48168,"kb_used_omap":3095,"kb_used_meta":59944,"kb_avail":94260616,"statfs":{"total":96636764160,"available":96522870784,"internally_reserved":0,"allocated":49324032,"data_stored":45700457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3169290,"internal_metadata":61383670},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575212,"num_pgs":107,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":225820,"kb_used_data":18380,"kb_used_omap":7232,"kb_used_meta":200191,"kb_avail":94146020,"statfs":{"total":96636764160,"available":96405524480,"internally_reserved":0,"allocated":18821120,"data_stored":13350111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7406186,"internal_metadata":204995990},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575212,"num_pgs":87,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80488,"kb_used_data":17432,"kb_used_omap":1599,"kb_used_meta":61440,"kb_avail":94291352,"statfs":{"total":96636764160,"available":96554344448,"internally_reserved":0,"allocated":17850368,"data_stored":13280880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1638118,"internal_metadata":62914842},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575213,"num_pgs":102,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":129624,"kb_used_data":12424,"kb_used_omap":5094,"kb_used_meta":112089,"kb_avail":94242216,"statfs":{"total":96636764160,"available":96504029184,"internally_reserved":0,"allocated":12722176,"data_stored":8040949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5216565,"internal_metadata":114779851},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575213,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":286372,"kb_used_data":60948,"kb_used_omap":10223,"kb_used_meta":215184,"kb_avail":94085468,"statfs":{"total":96636764160,"available":96343519232,"internally_reserved":0,"allocated":62410752,"data_stored":56934149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10468372,"internal_metadata":220349420},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53624832,"data_stored":53478090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5324800,"data_stored":5243270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":10555392,"data_stored":10486056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10620928,"data_stored":10486391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":43110400,"data_stored":42992165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":32641024,"data_stored":32506526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":21032960,"data_stored":20971830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":48386048,"data_stored":48235158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1667351,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2633902,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1516609,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1844562,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2430346,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":723382,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3546560,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2939218,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":2106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1512,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1072852,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":3356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18545,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":2058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5184,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1705285,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11530,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":2244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8436,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3928064,"data_stored":308322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51420,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3117056,"data_stored":241195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":152550,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3059712,"data_stored":239505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":825183,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3899392,"data_stored":305873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":336934,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1949696,"data_stored":151651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7338,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1925120,"data_stored":148450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":65762,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3837952,"data_stored":298046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2489145,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3194880,"data_stored":253176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1858329,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":79,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":133235,"internal_metadata":0},{"poolid":79,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14290,"internal_metadata":0},{"poolid":79,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23118,"internal_metadata":0},{"poolid":79,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15635,"internal_metadata":0},{"poolid":79,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49631,"internal_metadata":0},{"poolid":79,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49251,"internal_metadata":0},{"poolid":79,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16656,"internal_metadata":0},{"poolid":79,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15904,"internal_metadata":0},{"poolid":80,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":82,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":82,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1886,"internal_metadata":0},{"poolid":82,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6440,"internal_metadata":0},{"poolid":82,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":262,"internal_metadata":0},{"poolid":82,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1265,"internal_metadata":0},{"poolid":82,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":717,"internal_metadata":0},{"poolid":82,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":435,"internal_metadata":0},{"poolid":82,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T22:35:11.460 INFO:tasks.ceph:pgid 82.2 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.460 INFO:tasks.ceph:pgid 79.1f last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.460 INFO:tasks.ceph:pgid 82.3 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.460 INFO:tasks.ceph:pgid 81.0 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.460 INFO:tasks.ceph:pgid 80.1 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.460 INFO:tasks.ceph:pgid 79.1e last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.460 INFO:tasks.ceph:pgid 81.3 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.460 INFO:tasks.ceph:pgid 82.1 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.461 INFO:tasks.ceph:pgid 81.2 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.461 INFO:tasks.ceph:pgid 79.1c last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.461 INFO:tasks.ceph:pgid 82.6 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.461 INFO:tasks.ceph:pgid 81.5 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.461 INFO:tasks.ceph:pgid 80.4 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.461 INFO:tasks.ceph:pgid 79.1b last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.461 INFO:tasks.ceph:pgid 82.7 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.461 INFO:tasks.ceph:pgid 81.4 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.461 INFO:tasks.ceph:pgid 82.4 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.461 INFO:tasks.ceph:pgid 82.5 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.461 INFO:tasks.ceph:pgid 79.18 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.461 INFO:tasks.ceph:pgid 82.a last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.461 INFO:tasks.ceph:pgid 81.9 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.461 INFO:tasks.ceph:pgid 81.8 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.461 INFO:tasks.ceph:pgid 82.8 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.462 INFO:tasks.ceph:pgid 81.b last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.462 INFO:tasks.ceph:pgid 80.a last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.462 INFO:tasks.ceph:pgid 79.15 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.462 INFO:tasks.ceph:pgid 81.a last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.462 INFO:tasks.ceph:pgid 80.b last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.462 INFO:tasks.ceph:pgid 82.e last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.462 INFO:tasks.ceph:pgid 81.d last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.462 INFO:tasks.ceph:pgid 81.c last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.462 INFO:tasks.ceph:pgid 80.d last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.462 INFO:tasks.ceph:pgid 82.c last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.462 INFO:tasks.ceph:pgid 81.f last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.462 INFO:tasks.ceph:pgid 79.11 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.462 INFO:tasks.ceph:pgid 82.d last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.462 INFO:tasks.ceph:pgid 81.e last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.462 INFO:tasks.ceph:pgid 80.f last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.462 INFO:tasks.ceph:pgid 79.10 last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.463 INFO:tasks.ceph:pgid 82.12 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.463 INFO:tasks.ceph:pgid 81.11 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.463 INFO:tasks.ceph:pgid 80.10 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.463 INFO:tasks.ceph:pgid 79.f last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.463 INFO:tasks.ceph:pgid 82.13 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.463 INFO:tasks.ceph:pgid 81.10 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.463 INFO:tasks.ceph:pgid 80.11 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.463 INFO:tasks.ceph:pgid 82.10 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.463 INFO:tasks.ceph:pgid 80.12 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.463 INFO:tasks.ceph:pgid 79.d last_scrub_stamp 2026-02-24T22:13:42.750256+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=13, tm_sec=42, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.463 INFO:tasks.ceph:pgid 82.11 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.463 INFO:tasks.ceph:pgid 81.12 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.463 INFO:tasks.ceph:pgid 80.13 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.463 INFO:tasks.ceph:pgid 80.14 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.463 INFO:tasks.ceph:pgid 81.14 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.463 INFO:tasks.ceph:pgid 80.15 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.463 INFO:tasks.ceph:pgid 82.14 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.464 INFO:tasks.ceph:pgid 81.17 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.464 INFO:tasks.ceph:pgid 81.16 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.464 INFO:tasks.ceph:pgid 80.17 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.464 INFO:tasks.ceph:pgid 82.1a last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.464 INFO:tasks.ceph:pgid 80.18 last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.464 INFO:tasks.ceph:pgid 81.18 last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.464 INFO:tasks.ceph:pgid 82.18 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.464 INFO:tasks.ceph:pgid 81.1b last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.464 INFO:tasks.ceph:pgid 80.1b last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.464 INFO:tasks.ceph:pgid 82.1e last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.464 INFO:tasks.ceph:pgid 81.1d last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.464 INFO:tasks.ceph:pgid 80.1c last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.464 INFO:tasks.ceph:pgid 82.1f last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.464 INFO:tasks.ceph:pgid 81.1c last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.464 INFO:tasks.ceph:pgid 80.1d last_scrub_stamp 2026-02-24T22:15:21.804007+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=21, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.465 INFO:tasks.ceph:pgid 81.1f last_scrub_stamp 2026-02-24T22:15:24.819010+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=15, tm_sec=24, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.465 INFO:tasks.ceph:pgid 82.1d last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:11.467 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-24T22:35:31.468 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-24T22:35:31.633 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:35:31.646 INFO:teuthology.orchestra.run.trial046.stderr:dumped all 2026-02-24T22:35:31.649 INFO:teuthology.orchestra.run.trial046.stdout:{"pg_ready":true,"pg_map":{"version":1218,"stamp":"2026-02-24T22:35:30.039474+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":115015404,"num_objects":5093,"num_object_clones":0,"num_object_copies":10186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5093,"num_whiteouts":0,"num_read":207562,"num_read_kb":1387906,"num_write":161478,"num_write_kb":706745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":1543,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1275143,"num_omap_keys":5711,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13823,"ondisk_log_size":13823,"up":786,"acting":786,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":786,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1577288,"kb_used_data":278856,"kb_used_omap":37260,"kb_used_meta":1261043,"kb_avail":753397432,"statfs":{"total":773094113280,"available":771478970368,"internally_reserved":0,"allocated":285548544,"data_stored":247979658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38154327,"internal_metadata":1291308969},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":564,"num_omap_keys":6,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002561"},"pg_stats":[{"pgid":"82.2","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:20.609504+0000","last_change":"2026-02-24T22:35:20.609504+0000","last_active":"2026-02-24T22:35:20.609504+0000","last_peered":"2026-02-24T22:35:20.609504+0000","last_clean":"2026-02-24T22:35:20.609504+0000","last_became_active":"2026-02-24T22:23:29.198476+0000","last_became_peered":"2026-02-24T22:23:29.198476+0000","last_unstale":"2026-02-24T22:35:20.609504+0000","last_undegraded":"2026-02-24T22:35:20.609504+0000","last_fullsized":"2026-02-24T22:35:20.609504+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:20.609434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:20.609434+0000","last_clean_scrub_stamp":"2026-02-24T22:35:20.609434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:14:17.280111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.1","version":"0'0","reported_seq":40,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:07.404528+0000","last_change":"2026-02-24T22:35:07.404528+0000","last_active":"2026-02-24T22:35:07.404528+0000","last_peered":"2026-02-24T22:35:07.404528+0000","last_clean":"2026-02-24T22:35:07.404528+0000","last_became_active":"2026-02-24T22:15:25.830000+0000","last_became_peered":"2026-02-24T22:15:25.830000+0000","last_unstale":"2026-02-24T22:35:07.404528+0000","last_undegraded":"2026-02-24T22:35:07.404528+0000","last_fullsized":"2026-02-24T22:35:07.404528+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:07.404472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:07.404472+0000","last_clean_scrub_stamp":"2026-02-24T22:35:07.404472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:30:10.446904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.0","version":"0'0","reported_seq":47,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:54.176050+0000","last_change":"2026-02-24T22:34:54.176012+0000","last_active":"2026-02-24T22:34:54.176050+0000","last_peered":"2026-02-24T22:34:54.176050+0000","last_clean":"2026-02-24T22:34:54.176050+0000","last_became_active":"2026-02-24T22:15:22.817446+0000","last_became_peered":"2026-02-24T22:15:22.817446+0000","last_unstale":"2026-02-24T22:34:54.176050+0000","last_undegraded":"2026-02-24T22:34:54.176050+0000","last_fullsized":"2026-02-24T22:34:54.176050+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:54.175956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:54.175956+0000","last_clean_scrub_stamp":"2026-02-24T22:34:54.175956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:40:14.585196+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.1f","version":"312'20","reported_seq":129,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:11.720387+0000","last_change":"2026-02-24T22:35:11.720387+0000","last_active":"2026-02-24T22:35:11.720387+0000","last_peered":"2026-02-24T22:35:11.720387+0000","last_clean":"2026-02-24T22:35:11.720387+0000","last_became_active":"2026-02-24T22:13:43.761120+0000","last_became_peered":"2026-02-24T22:13:43.761120+0000","last_unstale":"2026-02-24T22:35:11.720387+0000","last_undegraded":"2026-02-24T22:35:11.720387+0000","last_fullsized":"2026-02-24T22:35:11.720387+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'20","last_scrub_stamp":"2026-02-24T22:35:11.720314+0000","last_deep_scrub":"312'20","last_deep_scrub_stamp":"2026-02-24T22:35:11.720314+0000","last_clean_scrub_stamp":"2026-02-24T22:35:11.720314+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:22:06.487293+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":168,"num_read_kb":114,"num_write":47,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.3","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:24.891603+0000","last_change":"2026-02-24T22:35:24.891603+0000","last_active":"2026-02-24T22:35:24.891603+0000","last_peered":"2026-02-24T22:35:24.891603+0000","last_clean":"2026-02-24T22:35:24.891603+0000","last_became_active":"2026-02-24T22:23:29.213534+0000","last_became_peered":"2026-02-24T22:23:29.213534+0000","last_unstale":"2026-02-24T22:35:24.891603+0000","last_undegraded":"2026-02-24T22:35:24.891603+0000","last_fullsized":"2026-02-24T22:35:24.891603+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:24.891536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:24.891536+0000","last_clean_scrub_stamp":"2026-02-24T22:35:24.891536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:37:51.239872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.0","version":"0'0","reported_seq":40,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:11.645200+0000","last_change":"2026-02-24T22:35:11.645200+0000","last_active":"2026-02-24T22:35:11.645200+0000","last_peered":"2026-02-24T22:35:11.645200+0000","last_clean":"2026-02-24T22:35:11.645200+0000","last_became_active":"2026-02-24T22:15:25.827355+0000","last_became_peered":"2026-02-24T22:15:25.827355+0000","last_unstale":"2026-02-24T22:35:11.645200+0000","last_undegraded":"2026-02-24T22:35:11.645200+0000","last_fullsized":"2026-02-24T22:35:11.645200+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:11.645114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:11.645114+0000","last_clean_scrub_stamp":"2026-02-24T22:35:11.645114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:37:43.905579+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.1","version":"0'0","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:14.776069+0000","last_change":"2026-02-24T22:35:14.776069+0000","last_active":"2026-02-24T22:35:14.776069+0000","last_peered":"2026-02-24T22:35:14.776069+0000","last_clean":"2026-02-24T22:35:14.776069+0000","last_became_active":"2026-02-24T22:15:22.818519+0000","last_became_peered":"2026-02-24T22:15:22.818519+0000","last_unstale":"2026-02-24T22:35:14.776069+0000","last_undegraded":"2026-02-24T22:35:14.776069+0000","last_fullsized":"2026-02-24T22:35:14.776069+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:14.776012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:14.776012+0000","last_clean_scrub_stamp":"2026-02-24T22:35:14.776012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:10:43.383714+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.1e","version":"312'9","reported_seq":96,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:11.351538+0000","last_change":"2026-02-24T22:35:11.351538+0000","last_active":"2026-02-24T22:35:11.351538+0000","last_peered":"2026-02-24T22:35:11.351538+0000","last_clean":"2026-02-24T22:35:11.351538+0000","last_became_active":"2026-02-24T22:13:43.757543+0000","last_became_peered":"2026-02-24T22:13:43.757543+0000","last_unstale":"2026-02-24T22:35:11.351538+0000","last_undegraded":"2026-02-24T22:35:11.351538+0000","last_fullsized":"2026-02-24T22:35:11.351538+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'9","last_scrub_stamp":"2026-02-24T22:35:11.351464+0000","last_deep_scrub":"312'9","last_deep_scrub_stamp":"2026-02-24T22:35:11.351464+0000","last_clean_scrub_stamp":"2026-02-24T22:35:11.351464+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:32:59.791152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":38,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.0","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:04.282796+0000","last_change":"2026-02-24T22:35:04.282796+0000","last_active":"2026-02-24T22:35:04.282796+0000","last_peered":"2026-02-24T22:35:04.282796+0000","last_clean":"2026-02-24T22:35:04.282796+0000","last_became_active":"2026-02-24T22:23:29.212043+0000","last_became_peered":"2026-02-24T22:23:29.212043+0000","last_unstale":"2026-02-24T22:35:04.282796+0000","last_undegraded":"2026-02-24T22:35:04.282796+0000","last_fullsized":"2026-02-24T22:35:04.282796+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:04.282728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:04.282728+0000","last_clean_scrub_stamp":"2026-02-24T22:35:04.282728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:05:28.411607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.3","version":"312'8","reported_seq":61,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:18.304114+0000","last_change":"2026-02-24T22:35:18.304114+0000","last_active":"2026-02-24T22:35:18.304114+0000","last_peered":"2026-02-24T22:35:18.304114+0000","last_clean":"2026-02-24T22:35:18.304114+0000","last_became_active":"2026-02-24T22:15:25.827216+0000","last_became_peered":"2026-02-24T22:15:25.827216+0000","last_unstale":"2026-02-24T22:35:18.304114+0000","last_undegraded":"2026-02-24T22:35:18.304114+0000","last_fullsized":"2026-02-24T22:35:18.304114+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'8","last_scrub_stamp":"2026-02-24T22:35:18.304050+0000","last_deep_scrub":"312'8","last_deep_scrub_stamp":"2026-02-24T22:35:18.304050+0000","last_clean_scrub_stamp":"2026-02-24T22:35:18.304050+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:14:27.296316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":25602,"num_write":13,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.2","version":"0'0","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:55.187191+0000","last_change":"2026-02-24T22:34:55.187191+0000","last_active":"2026-02-24T22:34:55.187191+0000","last_peered":"2026-02-24T22:34:55.187191+0000","last_clean":"2026-02-24T22:34:55.187191+0000","last_became_active":"2026-02-24T22:15:22.818194+0000","last_became_peered":"2026-02-24T22:15:22.818194+0000","last_unstale":"2026-02-24T22:34:55.187191+0000","last_undegraded":"2026-02-24T22:34:55.187191+0000","last_fullsized":"2026-02-24T22:34:55.187191+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:55.187119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:55.187119+0000","last_clean_scrub_stamp":"2026-02-24T22:34:55.187119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:26:30.922986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.1d","version":"312'22","reported_seq":129,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:00.724505+0000","last_change":"2026-02-24T22:35:00.724505+0000","last_active":"2026-02-24T22:35:00.724505+0000","last_peered":"2026-02-24T22:35:00.724505+0000","last_clean":"2026-02-24T22:35:00.724505+0000","last_became_active":"2026-02-24T22:13:43.757278+0000","last_became_peered":"2026-02-24T22:13:43.757278+0000","last_unstale":"2026-02-24T22:35:00.724505+0000","last_undegraded":"2026-02-24T22:35:00.724505+0000","last_fullsized":"2026-02-24T22:35:00.724505+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'22","last_scrub_stamp":"2026-02-24T22:35:00.724443+0000","last_deep_scrub":"312'22","last_deep_scrub_stamp":"2026-02-24T22:35:00.724443+0000","last_clean_scrub_stamp":"2026-02-24T22:35:00.724443+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:16:45.929416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":103,"num_write":54,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.1","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:26.839718+0000","last_change":"2026-02-24T22:35:26.839718+0000","last_active":"2026-02-24T22:35:26.839718+0000","last_peered":"2026-02-24T22:35:26.839718+0000","last_clean":"2026-02-24T22:35:26.839718+0000","last_became_active":"2026-02-24T22:23:29.214064+0000","last_became_peered":"2026-02-24T22:23:29.214064+0000","last_unstale":"2026-02-24T22:35:26.839718+0000","last_undegraded":"2026-02-24T22:35:26.839718+0000","last_fullsized":"2026-02-24T22:35:26.839718+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:26.839650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:26.839650+0000","last_clean_scrub_stamp":"2026-02-24T22:35:26.839650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:46:27.127761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.2","version":"312'2","reported_seq":44,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:11.543860+0000","last_change":"2026-02-24T22:35:11.543860+0000","last_active":"2026-02-24T22:35:11.543860+0000","last_peered":"2026-02-24T22:35:11.543860+0000","last_clean":"2026-02-24T22:35:11.543860+0000","last_became_active":"2026-02-24T22:15:25.832809+0000","last_became_peered":"2026-02-24T22:15:25.832809+0000","last_unstale":"2026-02-24T22:35:11.543860+0000","last_undegraded":"2026-02-24T22:35:11.543860+0000","last_fullsized":"2026-02-24T22:35:11.543860+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:35:11.543798+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:35:11.543798+0000","last_clean_scrub_stamp":"2026-02-24T22:35:11.543798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:28:14.037060+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"80.3","version":"312'3","reported_seq":52,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:56.183409+0000","last_change":"2026-02-24T22:34:56.183409+0000","last_active":"2026-02-24T22:34:56.183409+0000","last_peered":"2026-02-24T22:34:56.183409+0000","last_clean":"2026-02-24T22:34:56.183409+0000","last_became_active":"2026-02-24T22:15:22.818089+0000","last_became_peered":"2026-02-24T22:15:22.818089+0000","last_unstale":"2026-02-24T22:34:56.183409+0000","last_undegraded":"2026-02-24T22:34:56.183409+0000","last_fullsized":"2026-02-24T22:34:56.183409+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:34:56.183350+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:34:56.183350+0000","last_clean_scrub_stamp":"2026-02-24T22:34:56.183350+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:06:53.926961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.1c","version":"312'14","reported_seq":109,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:15.806073+0000","last_change":"2026-02-24T22:35:15.806073+0000","last_active":"2026-02-24T22:35:15.806073+0000","last_peered":"2026-02-24T22:35:15.806073+0000","last_clean":"2026-02-24T22:35:15.806073+0000","last_became_active":"2026-02-24T22:13:43.759958+0000","last_became_peered":"2026-02-24T22:13:43.759958+0000","last_unstale":"2026-02-24T22:35:15.806073+0000","last_undegraded":"2026-02-24T22:35:15.806073+0000","last_fullsized":"2026-02-24T22:35:15.806073+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'14","last_scrub_stamp":"2026-02-24T22:35:15.806010+0000","last_deep_scrub":"312'14","last_deep_scrub_stamp":"2026-02-24T22:35:15.806010+0000","last_clean_scrub_stamp":"2026-02-24T22:35:15.806010+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:05:06.336286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":75,"num_write":36,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.6","version":"312'2","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:11.309552+0000","last_change":"2026-02-24T22:35:11.309552+0000","last_active":"2026-02-24T22:35:11.309552+0000","last_peered":"2026-02-24T22:35:11.309552+0000","last_clean":"2026-02-24T22:35:11.309552+0000","last_became_active":"2026-02-24T22:23:29.215327+0000","last_became_peered":"2026-02-24T22:23:29.215327+0000","last_unstale":"2026-02-24T22:35:11.309552+0000","last_undegraded":"2026-02-24T22:35:11.309552+0000","last_fullsized":"2026-02-24T22:35:11.309552+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:35:11.309512+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:35:11.309512+0000","last_clean_scrub_stamp":"2026-02-24T22:35:11.309512+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:15:39.214444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.5","version":"0'0","reported_seq":40,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:25.847303+0000","last_change":"2026-02-24T22:35:25.847303+0000","last_active":"2026-02-24T22:35:25.847303+0000","last_peered":"2026-02-24T22:35:25.847303+0000","last_clean":"2026-02-24T22:35:25.847303+0000","last_became_active":"2026-02-24T22:15:25.831975+0000","last_became_peered":"2026-02-24T22:15:25.831975+0000","last_unstale":"2026-02-24T22:35:25.847303+0000","last_undegraded":"2026-02-24T22:35:25.847303+0000","last_fullsized":"2026-02-24T22:35:25.847303+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:25.847227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:25.847227+0000","last_clean_scrub_stamp":"2026-02-24T22:35:25.847227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:59:07.272621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"80.4","version":"312'3","reported_seq":52,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:12.393384+0000","last_change":"2026-02-24T22:35:12.393384+0000","last_active":"2026-02-24T22:35:12.393384+0000","last_peered":"2026-02-24T22:35:12.393384+0000","last_clean":"2026-02-24T22:35:12.393384+0000","last_became_active":"2026-02-24T22:15:22.812270+0000","last_became_peered":"2026-02-24T22:15:22.812270+0000","last_unstale":"2026-02-24T22:35:12.393384+0000","last_undegraded":"2026-02-24T22:35:12.393384+0000","last_fullsized":"2026-02-24T22:35:12.393384+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:12.393333+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:12.393333+0000","last_clean_scrub_stamp":"2026-02-24T22:35:12.393333+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:05:26.291517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.1b","version":"312'18","reported_seq":120,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:13.375416+0000","last_change":"2026-02-24T22:35:13.375416+0000","last_active":"2026-02-24T22:35:13.375416+0000","last_peered":"2026-02-24T22:35:13.375416+0000","last_clean":"2026-02-24T22:35:13.375416+0000","last_became_active":"2026-02-24T22:13:43.755170+0000","last_became_peered":"2026-02-24T22:13:43.755170+0000","last_unstale":"2026-02-24T22:35:13.375416+0000","last_undegraded":"2026-02-24T22:35:13.375416+0000","last_fullsized":"2026-02-24T22:35:13.375416+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'18","last_scrub_stamp":"2026-02-24T22:35:13.375362+0000","last_deep_scrub":"312'18","last_deep_scrub_stamp":"2026-02-24T22:35:13.375362+0000","last_clean_scrub_stamp":"2026-02-24T22:35:13.375362+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:06:42.681619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":101,"num_write":42,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.7","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:14.566741+0000","last_change":"2026-02-24T22:35:14.566741+0000","last_active":"2026-02-24T22:35:14.566741+0000","last_peered":"2026-02-24T22:35:14.566741+0000","last_clean":"2026-02-24T22:35:14.566741+0000","last_became_active":"2026-02-24T22:23:29.214105+0000","last_became_peered":"2026-02-24T22:23:29.214105+0000","last_unstale":"2026-02-24T22:35:14.566741+0000","last_undegraded":"2026-02-24T22:35:14.566741+0000","last_fullsized":"2026-02-24T22:35:14.566741+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:14.566681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:14.566681+0000","last_clean_scrub_stamp":"2026-02-24T22:35:14.566681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:27:13.392670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.4","version":"312'3","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:10.615653+0000","last_change":"2026-02-24T22:35:10.615653+0000","last_active":"2026-02-24T22:35:10.615653+0000","last_peered":"2026-02-24T22:35:10.615653+0000","last_clean":"2026-02-24T22:35:10.615653+0000","last_became_active":"2026-02-24T22:15:25.826663+0000","last_became_peered":"2026-02-24T22:15:25.826663+0000","last_unstale":"2026-02-24T22:35:10.615653+0000","last_undegraded":"2026-02-24T22:35:10.615653+0000","last_fullsized":"2026-02-24T22:35:10.615653+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:10.615580+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:10.615580+0000","last_clean_scrub_stamp":"2026-02-24T22:35:10.615580+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:21:31.466450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.5","version":"0'0","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:06.414312+0000","last_change":"2026-02-24T22:35:06.414312+0000","last_active":"2026-02-24T22:35:06.414312+0000","last_peered":"2026-02-24T22:35:06.414312+0000","last_clean":"2026-02-24T22:35:06.414312+0000","last_became_active":"2026-02-24T22:15:22.816697+0000","last_became_peered":"2026-02-24T22:15:22.816697+0000","last_unstale":"2026-02-24T22:35:06.414312+0000","last_undegraded":"2026-02-24T22:35:06.414312+0000","last_fullsized":"2026-02-24T22:35:06.414312+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:06.414264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:06.414264+0000","last_clean_scrub_stamp":"2026-02-24T22:35:06.414264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:28:07.680045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"79.1a","version":"312'16","reported_seq":110,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.717216+0000","last_change":"2026-02-24T22:34:58.717216+0000","last_active":"2026-02-24T22:34:58.717216+0000","last_peered":"2026-02-24T22:34:58.717216+0000","last_clean":"2026-02-24T22:34:58.717216+0000","last_became_active":"2026-02-24T22:13:43.757353+0000","last_became_peered":"2026-02-24T22:13:43.757353+0000","last_unstale":"2026-02-24T22:34:58.717216+0000","last_undegraded":"2026-02-24T22:34:58.717216+0000","last_fullsized":"2026-02-24T22:34:58.717216+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'16","last_scrub_stamp":"2026-02-24T22:34:58.717170+0000","last_deep_scrub":"312'16","last_deep_scrub_stamp":"2026-02-24T22:34:58.717170+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.717170+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:56:29.218407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":65,"num_write":38,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.4","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:31:49.742881+0000","last_change":"2026-02-24T22:23:29.215206+0000","last_active":"2026-02-24T22:31:49.742881+0000","last_peered":"2026-02-24T22:31:49.742881+0000","last_clean":"2026-02-24T22:31:49.742881+0000","last_became_active":"2026-02-24T22:23:29.215014+0000","last_became_peered":"2026-02-24T22:23:29.215014+0000","last_unstale":"2026-02-24T22:31:49.742881+0000","last_undegraded":"2026-02-24T22:31:49.742881+0000","last_fullsized":"2026-02-24T22:31:49.742881+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:43:42.586588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.7","version":"312'3","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:03.237185+0000","last_change":"2026-02-24T22:35:03.237185+0000","last_active":"2026-02-24T22:35:03.237185+0000","last_peered":"2026-02-24T22:35:03.237185+0000","last_clean":"2026-02-24T22:35:03.237185+0000","last_became_active":"2026-02-24T22:15:25.831806+0000","last_became_peered":"2026-02-24T22:15:25.831806+0000","last_unstale":"2026-02-24T22:35:03.237185+0000","last_undegraded":"2026-02-24T22:35:03.237185+0000","last_fullsized":"2026-02-24T22:35:03.237185+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:03.237068+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:03.237068+0000","last_clean_scrub_stamp":"2026-02-24T22:35:03.237068+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:04:02.704108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"80.6","version":"0'0","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:08.654082+0000","last_change":"2026-02-24T22:35:08.654082+0000","last_active":"2026-02-24T22:35:08.654082+0000","last_peered":"2026-02-24T22:35:08.654082+0000","last_clean":"2026-02-24T22:35:08.654082+0000","last_became_active":"2026-02-24T22:15:22.811413+0000","last_became_peered":"2026-02-24T22:15:22.811413+0000","last_unstale":"2026-02-24T22:35:08.654082+0000","last_undegraded":"2026-02-24T22:35:08.654082+0000","last_fullsized":"2026-02-24T22:35:08.654082+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:08.654011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:08.654011+0000","last_clean_scrub_stamp":"2026-02-24T22:35:08.654011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:15:31.642731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"79.19","version":"312'30","reported_seq":148,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:09.330426+0000","last_change":"2026-02-24T22:35:09.330426+0000","last_active":"2026-02-24T22:35:09.330426+0000","last_peered":"2026-02-24T22:35:09.330426+0000","last_clean":"2026-02-24T22:35:09.330426+0000","last_became_active":"2026-02-24T22:13:43.756114+0000","last_became_peered":"2026-02-24T22:13:43.756114+0000","last_unstale":"2026-02-24T22:35:09.330426+0000","last_undegraded":"2026-02-24T22:35:09.330426+0000","last_fullsized":"2026-02-24T22:35:09.330426+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'30","last_scrub_stamp":"2026-02-24T22:35:09.330359+0000","last_deep_scrub":"312'30","last_deep_scrub_stamp":"2026-02-24T22:35:09.330359+0000","last_clean_scrub_stamp":"2026-02-24T22:35:09.330359+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:23:17.878380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":216,"num_read_kb":143,"num_write":68,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.5","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:28.823856+0000","last_change":"2026-02-24T22:35:28.823856+0000","last_active":"2026-02-24T22:35:28.823856+0000","last_peered":"2026-02-24T22:35:28.823856+0000","last_clean":"2026-02-24T22:35:28.823856+0000","last_became_active":"2026-02-24T22:23:29.213256+0000","last_became_peered":"2026-02-24T22:23:29.213256+0000","last_unstale":"2026-02-24T22:35:28.823856+0000","last_undegraded":"2026-02-24T22:35:28.823856+0000","last_fullsized":"2026-02-24T22:35:28.823856+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:28.823793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:28.823793+0000","last_clean_scrub_stamp":"2026-02-24T22:35:28.823793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:29:08.343819+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.6","version":"312'3","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:08.360735+0000","last_change":"2026-02-24T22:35:08.360735+0000","last_active":"2026-02-24T22:35:08.360735+0000","last_peered":"2026-02-24T22:35:08.360735+0000","last_clean":"2026-02-24T22:35:08.360735+0000","last_became_active":"2026-02-24T22:15:25.833680+0000","last_became_peered":"2026-02-24T22:15:25.833680+0000","last_unstale":"2026-02-24T22:35:08.360735+0000","last_undegraded":"2026-02-24T22:35:08.360735+0000","last_fullsized":"2026-02-24T22:35:08.360735+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:08.360662+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:08.360662+0000","last_clean_scrub_stamp":"2026-02-24T22:35:08.360662+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:43:55.489569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.7","version":"0'0","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:09.618057+0000","last_change":"2026-02-24T22:35:09.618057+0000","last_active":"2026-02-24T22:35:09.618057+0000","last_peered":"2026-02-24T22:35:09.618057+0000","last_clean":"2026-02-24T22:35:09.618057+0000","last_became_active":"2026-02-24T22:15:22.811618+0000","last_became_peered":"2026-02-24T22:15:22.811618+0000","last_unstale":"2026-02-24T22:35:09.618057+0000","last_undegraded":"2026-02-24T22:35:09.618057+0000","last_fullsized":"2026-02-24T22:35:09.618057+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:09.617989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:09.617989+0000","last_clean_scrub_stamp":"2026-02-24T22:35:09.617989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:27:23.181702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"79.18","version":"312'12","reported_seq":104,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:10.315573+0000","last_change":"2026-02-24T22:35:10.315509+0000","last_active":"2026-02-24T22:35:10.315573+0000","last_peered":"2026-02-24T22:35:10.315573+0000","last_clean":"2026-02-24T22:35:10.315573+0000","last_became_active":"2026-02-24T22:13:43.754981+0000","last_became_peered":"2026-02-24T22:13:43.754981+0000","last_unstale":"2026-02-24T22:35:10.315573+0000","last_undegraded":"2026-02-24T22:35:10.315573+0000","last_fullsized":"2026-02-24T22:35:10.315573+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'12","last_scrub_stamp":"2026-02-24T22:35:10.315446+0000","last_deep_scrub":"312'12","last_deep_scrub_stamp":"2026-02-24T22:35:10.315446+0000","last_clean_scrub_stamp":"2026-02-24T22:35:10.315446+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:04:57.572069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":86,"num_read_kb":57,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.a","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:27.841463+0000","last_change":"2026-02-24T22:35:27.841463+0000","last_active":"2026-02-24T22:35:27.841463+0000","last_peered":"2026-02-24T22:35:27.841463+0000","last_clean":"2026-02-24T22:35:27.841463+0000","last_became_active":"2026-02-24T22:23:29.212213+0000","last_became_peered":"2026-02-24T22:23:29.212213+0000","last_unstale":"2026-02-24T22:35:27.841463+0000","last_undegraded":"2026-02-24T22:35:27.841463+0000","last_fullsized":"2026-02-24T22:35:27.841463+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:27.841391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:27.841391+0000","last_clean_scrub_stamp":"2026-02-24T22:35:27.841391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:04:01.169164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.9","version":"0'0","reported_seq":40,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:13.690913+0000","last_change":"2026-02-24T22:35:13.690913+0000","last_active":"2026-02-24T22:35:13.690913+0000","last_peered":"2026-02-24T22:35:13.690913+0000","last_clean":"2026-02-24T22:35:13.690913+0000","last_became_active":"2026-02-24T22:15:25.826076+0000","last_became_peered":"2026-02-24T22:15:25.826076+0000","last_unstale":"2026-02-24T22:35:13.690913+0000","last_undegraded":"2026-02-24T22:35:13.690913+0000","last_fullsized":"2026-02-24T22:35:13.690913+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:13.690725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:13.690725+0000","last_clean_scrub_stamp":"2026-02-24T22:35:13.690725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:47:50.305008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.8","version":"312'6","reported_seq":57,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:57.217246+0000","last_change":"2026-02-24T22:34:57.217246+0000","last_active":"2026-02-24T22:34:57.217246+0000","last_peered":"2026-02-24T22:34:57.217246+0000","last_clean":"2026-02-24T22:34:57.217246+0000","last_became_active":"2026-02-24T22:15:22.815960+0000","last_became_peered":"2026-02-24T22:15:22.815960+0000","last_unstale":"2026-02-24T22:34:57.217246+0000","last_undegraded":"2026-02-24T22:34:57.217246+0000","last_fullsized":"2026-02-24T22:34:57.217246+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'6","last_scrub_stamp":"2026-02-24T22:34:57.217191+0000","last_deep_scrub":"312'6","last_deep_scrub_stamp":"2026-02-24T22:34:57.217191+0000","last_clean_scrub_stamp":"2026-02-24T22:34:57.217191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:07:48.538456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6144,"num_write":11,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.17","version":"312'34","reported_seq":178,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:03.366118+0000","last_change":"2026-02-24T22:35:03.366056+0000","last_active":"2026-02-24T22:35:03.366118+0000","last_peered":"2026-02-24T22:35:03.366118+0000","last_clean":"2026-02-24T22:35:03.366118+0000","last_became_active":"2026-02-24T22:14:00.748438+0000","last_became_peered":"2026-02-24T22:14:00.748438+0000","last_unstale":"2026-02-24T22:35:03.366118+0000","last_undegraded":"2026-02-24T22:35:03.366118+0000","last_fullsized":"2026-02-24T22:35:03.366118+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":293,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'34","last_scrub_stamp":"2026-02-24T22:35:03.365970+0000","last_deep_scrub":"312'34","last_deep_scrub_stamp":"2026-02-24T22:35:03.365970+0000","last_clean_scrub_stamp":"2026-02-24T22:35:03.365970+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:27:41.779757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":298,"num_read_kb":211,"num_write":78,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.b","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:05.714092+0000","last_change":"2026-02-24T22:35:05.714092+0000","last_active":"2026-02-24T22:35:05.714092+0000","last_peered":"2026-02-24T22:35:05.714092+0000","last_clean":"2026-02-24T22:35:05.714092+0000","last_became_active":"2026-02-24T22:23:29.196790+0000","last_became_peered":"2026-02-24T22:23:29.196790+0000","last_unstale":"2026-02-24T22:35:05.714092+0000","last_undegraded":"2026-02-24T22:35:05.714092+0000","last_fullsized":"2026-02-24T22:35:05.714092+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:05.714031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:05.714031+0000","last_clean_scrub_stamp":"2026-02-24T22:35:05.714031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:44:22.188544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.8","version":"0'0","reported_seq":40,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:14.051734+0000","last_change":"2026-02-24T22:35:14.051734+0000","last_active":"2026-02-24T22:35:14.051734+0000","last_peered":"2026-02-24T22:35:14.051734+0000","last_clean":"2026-02-24T22:35:14.051734+0000","last_became_active":"2026-02-24T22:15:25.825315+0000","last_became_peered":"2026-02-24T22:15:25.825315+0000","last_unstale":"2026-02-24T22:35:14.051734+0000","last_undegraded":"2026-02-24T22:35:14.051734+0000","last_fullsized":"2026-02-24T22:35:14.051734+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:14.051650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:14.051650+0000","last_clean_scrub_stamp":"2026-02-24T22:35:14.051650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:10:57.064078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.9","version":"0'0","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:59.757040+0000","last_change":"2026-02-24T22:34:59.757040+0000","last_active":"2026-02-24T22:34:59.757040+0000","last_peered":"2026-02-24T22:34:59.757040+0000","last_clean":"2026-02-24T22:34:59.757040+0000","last_became_active":"2026-02-24T22:15:22.811141+0000","last_became_peered":"2026-02-24T22:15:22.811141+0000","last_unstale":"2026-02-24T22:34:59.757040+0000","last_undegraded":"2026-02-24T22:34:59.757040+0000","last_fullsized":"2026-02-24T22:34:59.757040+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:59.756969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:59.756969+0000","last_clean_scrub_stamp":"2026-02-24T22:34:59.756969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:19:48.331387+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"79.16","version":"0'0","reported_seq":72,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:57.691734+0000","last_change":"2026-02-24T22:34:57.691689+0000","last_active":"2026-02-24T22:34:57.691734+0000","last_peered":"2026-02-24T22:34:57.691734+0000","last_clean":"2026-02-24T22:34:57.691734+0000","last_became_active":"2026-02-24T22:13:43.755885+0000","last_became_peered":"2026-02-24T22:13:43.755885+0000","last_unstale":"2026-02-24T22:34:57.691734+0000","last_undegraded":"2026-02-24T22:34:57.691734+0000","last_fullsized":"2026-02-24T22:34:57.691734+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:57.691616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:57.691616+0000","last_clean_scrub_stamp":"2026-02-24T22:34:57.691616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:54:10.527824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.8","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:17.966932+0000","last_change":"2026-02-24T22:35:17.966932+0000","last_active":"2026-02-24T22:35:17.966932+0000","last_peered":"2026-02-24T22:35:17.966932+0000","last_clean":"2026-02-24T22:35:17.966932+0000","last_became_active":"2026-02-24T22:23:29.198882+0000","last_became_peered":"2026-02-24T22:23:29.198882+0000","last_unstale":"2026-02-24T22:35:17.966932+0000","last_undegraded":"2026-02-24T22:35:17.966932+0000","last_fullsized":"2026-02-24T22:35:17.966932+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:17.966874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:17.966874+0000","last_clean_scrub_stamp":"2026-02-24T22:35:17.966874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:40:19.729940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"81.b","version":"0'0","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:20.294652+0000","last_change":"2026-02-24T22:35:20.294652+0000","last_active":"2026-02-24T22:35:20.294652+0000","last_peered":"2026-02-24T22:35:20.294652+0000","last_clean":"2026-02-24T22:35:20.294652+0000","last_became_active":"2026-02-24T22:15:25.827265+0000","last_became_peered":"2026-02-24T22:15:25.827265+0000","last_unstale":"2026-02-24T22:35:20.294652+0000","last_undegraded":"2026-02-24T22:35:20.294652+0000","last_fullsized":"2026-02-24T22:35:20.294652+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:20.294586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:20.294586+0000","last_clean_scrub_stamp":"2026-02-24T22:35:20.294586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:51:45.105405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.a","version":"0'0","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:16.770098+0000","last_change":"2026-02-24T22:35:16.770098+0000","last_active":"2026-02-24T22:35:16.770098+0000","last_peered":"2026-02-24T22:35:16.770098+0000","last_clean":"2026-02-24T22:35:16.770098+0000","last_became_active":"2026-02-24T22:15:22.816702+0000","last_became_peered":"2026-02-24T22:15:22.816702+0000","last_unstale":"2026-02-24T22:35:16.770098+0000","last_undegraded":"2026-02-24T22:35:16.770098+0000","last_fullsized":"2026-02-24T22:35:16.770098+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:16.770027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:16.770027+0000","last_clean_scrub_stamp":"2026-02-24T22:35:16.770027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:33:16.068318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.15","version":"307'6","reported_seq":91,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:12.713792+0000","last_change":"2026-02-24T22:35:12.713724+0000","last_active":"2026-02-24T22:35:12.713792+0000","last_peered":"2026-02-24T22:35:12.713792+0000","last_clean":"2026-02-24T22:35:12.713792+0000","last_became_active":"2026-02-24T22:13:43.760892+0000","last_became_peered":"2026-02-24T22:13:43.760892+0000","last_unstale":"2026-02-24T22:35:12.713792+0000","last_undegraded":"2026-02-24T22:35:12.713792+0000","last_fullsized":"2026-02-24T22:35:12.713792+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"307'6","last_scrub_stamp":"2026-02-24T22:35:12.713642+0000","last_deep_scrub":"307'6","last_deep_scrub_stamp":"2026-02-24T22:35:12.713642+0000","last_clean_scrub_stamp":"2026-02-24T22:35:12.713642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:51:38.187083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":37,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.9","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:02.763296+0000","last_change":"2026-02-24T22:35:02.763296+0000","last_active":"2026-02-24T22:35:02.763296+0000","last_peered":"2026-02-24T22:35:02.763296+0000","last_clean":"2026-02-24T22:35:02.763296+0000","last_became_active":"2026-02-24T22:23:29.197357+0000","last_became_peered":"2026-02-24T22:23:29.197357+0000","last_unstale":"2026-02-24T22:35:02.763296+0000","last_undegraded":"2026-02-24T22:35:02.763296+0000","last_fullsized":"2026-02-24T22:35:02.763296+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:02.763243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:02.763243+0000","last_clean_scrub_stamp":"2026-02-24T22:35:02.763243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:52:20.692060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.a","version":"312'3","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:21.333834+0000","last_change":"2026-02-24T22:35:21.333834+0000","last_active":"2026-02-24T22:35:21.333834+0000","last_peered":"2026-02-24T22:35:21.333834+0000","last_clean":"2026-02-24T22:35:21.333834+0000","last_became_active":"2026-02-24T22:15:25.827055+0000","last_became_peered":"2026-02-24T22:15:25.827055+0000","last_unstale":"2026-02-24T22:35:21.333834+0000","last_undegraded":"2026-02-24T22:35:21.333834+0000","last_fullsized":"2026-02-24T22:35:21.333834+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:21.333781+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:21.333781+0000","last_clean_scrub_stamp":"2026-02-24T22:35:21.333781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:00:01.811417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.b","version":"312'3","reported_seq":51,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:17.780856+0000","last_change":"2026-02-24T22:35:17.780856+0000","last_active":"2026-02-24T22:35:17.780856+0000","last_peered":"2026-02-24T22:35:17.780856+0000","last_clean":"2026-02-24T22:35:17.780856+0000","last_became_active":"2026-02-24T22:15:22.817498+0000","last_became_peered":"2026-02-24T22:15:22.817498+0000","last_unstale":"2026-02-24T22:35:17.780856+0000","last_undegraded":"2026-02-24T22:35:17.780856+0000","last_fullsized":"2026-02-24T22:35:17.780856+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:17.780787+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:17.780787+0000","last_clean_scrub_stamp":"2026-02-24T22:35:17.780787+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:49:22.655262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.14","version":"312'10","reported_seq":98,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:02.317457+0000","last_change":"2026-02-24T22:35:02.317457+0000","last_active":"2026-02-24T22:35:02.317457+0000","last_peered":"2026-02-24T22:35:02.317457+0000","last_clean":"2026-02-24T22:35:02.317457+0000","last_became_active":"2026-02-24T22:13:43.762341+0000","last_became_peered":"2026-02-24T22:13:43.762341+0000","last_unstale":"2026-02-24T22:35:02.317457+0000","last_undegraded":"2026-02-24T22:35:02.317457+0000","last_fullsized":"2026-02-24T22:35:02.317457+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'10","last_scrub_stamp":"2026-02-24T22:35:02.317403+0000","last_deep_scrub":"312'10","last_deep_scrub_stamp":"2026-02-24T22:35:02.317403+0000","last_clean_scrub_stamp":"2026-02-24T22:35:02.317403+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:39:36.833592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":46,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.e","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:15.577587+0000","last_change":"2026-02-24T22:35:15.577587+0000","last_active":"2026-02-24T22:35:15.577587+0000","last_peered":"2026-02-24T22:35:15.577587+0000","last_clean":"2026-02-24T22:35:15.577587+0000","last_became_active":"2026-02-24T22:23:29.215425+0000","last_became_peered":"2026-02-24T22:23:29.215425+0000","last_unstale":"2026-02-24T22:35:15.577587+0000","last_undegraded":"2026-02-24T22:35:15.577587+0000","last_fullsized":"2026-02-24T22:35:15.577587+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:15.577525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:15.577525+0000","last_clean_scrub_stamp":"2026-02-24T22:35:15.577525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:52:30.886126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.d","version":"0'0","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:22.336154+0000","last_change":"2026-02-24T22:35:22.336154+0000","last_active":"2026-02-24T22:35:22.336154+0000","last_peered":"2026-02-24T22:35:22.336154+0000","last_clean":"2026-02-24T22:35:22.336154+0000","last_became_active":"2026-02-24T22:15:25.829109+0000","last_became_peered":"2026-02-24T22:15:25.829109+0000","last_unstale":"2026-02-24T22:35:22.336154+0000","last_undegraded":"2026-02-24T22:35:22.336154+0000","last_fullsized":"2026-02-24T22:35:22.336154+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:22.336091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:22.336091+0000","last_clean_scrub_stamp":"2026-02-24T22:35:22.336091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:20:46.735519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.c","version":"0'0","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.197887+0000","last_change":"2026-02-24T22:34:58.197887+0000","last_active":"2026-02-24T22:34:58.197887+0000","last_peered":"2026-02-24T22:34:58.197887+0000","last_clean":"2026-02-24T22:34:58.197887+0000","last_became_active":"2026-02-24T22:15:22.815364+0000","last_became_peered":"2026-02-24T22:15:22.815364+0000","last_unstale":"2026-02-24T22:34:58.197887+0000","last_undegraded":"2026-02-24T22:34:58.197887+0000","last_fullsized":"2026-02-24T22:34:58.197887+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:58.197815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:58.197815+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.197815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:35:14.681874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.13","version":"312'11","reported_seq":104,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:51.152600+0000","last_change":"2026-02-24T22:34:51.152600+0000","last_active":"2026-02-24T22:34:51.152600+0000","last_peered":"2026-02-24T22:34:51.152600+0000","last_clean":"2026-02-24T22:34:51.152600+0000","last_became_active":"2026-02-24T22:13:43.760714+0000","last_became_peered":"2026-02-24T22:13:43.760714+0000","last_unstale":"2026-02-24T22:34:51.152600+0000","last_undegraded":"2026-02-24T22:34:51.152600+0000","last_fullsized":"2026-02-24T22:34:51.152600+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'11","last_scrub_stamp":"2026-02-24T22:34:51.152549+0000","last_deep_scrub":"312'11","last_deep_scrub_stamp":"2026-02-24T22:34:51.152549+0000","last_clean_scrub_stamp":"2026-02-24T22:34:51.152549+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:24:03.221560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":61,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"82.f","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:06.681089+0000","last_change":"2026-02-24T22:35:06.681089+0000","last_active":"2026-02-24T22:35:06.681089+0000","last_peered":"2026-02-24T22:35:06.681089+0000","last_clean":"2026-02-24T22:35:06.681089+0000","last_became_active":"2026-02-24T22:23:29.199538+0000","last_became_peered":"2026-02-24T22:23:29.199538+0000","last_unstale":"2026-02-24T22:35:06.681089+0000","last_undegraded":"2026-02-24T22:35:06.681089+0000","last_fullsized":"2026-02-24T22:35:06.681089+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:06.680985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:06.680985+0000","last_clean_scrub_stamp":"2026-02-24T22:35:06.680985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:18:47.506871+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.c","version":"312'11","reported_seq":66,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:12.647111+0000","last_change":"2026-02-24T22:35:12.647111+0000","last_active":"2026-02-24T22:35:12.647111+0000","last_peered":"2026-02-24T22:35:12.647111+0000","last_clean":"2026-02-24T22:35:12.647111+0000","last_became_active":"2026-02-24T22:15:25.826430+0000","last_became_peered":"2026-02-24T22:15:25.826430+0000","last_unstale":"2026-02-24T22:35:12.647111+0000","last_undegraded":"2026-02-24T22:35:12.647111+0000","last_fullsized":"2026-02-24T22:35:12.647111+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'11","last_scrub_stamp":"2026-02-24T22:35:12.647031+0000","last_deep_scrub":"312'11","last_deep_scrub_stamp":"2026-02-24T22:35:12.647031+0000","last_clean_scrub_stamp":"2026-02-24T22:35:12.647031+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:08:17.241612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":31746,"num_write":16,"num_write_kb":16384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.d","version":"312'5","reported_seq":60,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:14.345762+0000","last_change":"2026-02-24T22:35:14.345762+0000","last_active":"2026-02-24T22:35:14.345762+0000","last_peered":"2026-02-24T22:35:14.345762+0000","last_clean":"2026-02-24T22:35:14.345762+0000","last_became_active":"2026-02-24T22:15:22.813062+0000","last_became_peered":"2026-02-24T22:15:22.813062+0000","last_unstale":"2026-02-24T22:35:14.345762+0000","last_undegraded":"2026-02-24T22:35:14.345762+0000","last_fullsized":"2026-02-24T22:35:14.345762+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'5","last_scrub_stamp":"2026-02-24T22:35:14.345705+0000","last_deep_scrub":"312'5","last_deep_scrub_stamp":"2026-02-24T22:35:14.345705+0000","last_clean_scrub_stamp":"2026-02-24T22:35:14.345705+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:10:48.119100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":10,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.12","version":"312'20","reported_seq":126,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:53.127878+0000","last_change":"2026-02-24T22:34:53.127878+0000","last_active":"2026-02-24T22:34:53.127878+0000","last_peered":"2026-02-24T22:34:53.127878+0000","last_clean":"2026-02-24T22:34:53.127878+0000","last_became_active":"2026-02-24T22:13:43.760196+0000","last_became_peered":"2026-02-24T22:13:43.760196+0000","last_unstale":"2026-02-24T22:34:53.127878+0000","last_undegraded":"2026-02-24T22:34:53.127878+0000","last_fullsized":"2026-02-24T22:34:53.127878+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'20","last_scrub_stamp":"2026-02-24T22:34:53.127808+0000","last_deep_scrub":"312'20","last_deep_scrub_stamp":"2026-02-24T22:34:53.127808+0000","last_clean_scrub_stamp":"2026-02-24T22:34:53.127808+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:40:20.279936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":106,"num_write":52,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"82.c","version":"312'2","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:24.335353+0000","last_change":"2026-02-24T22:35:24.335353+0000","last_active":"2026-02-24T22:35:24.335353+0000","last_peered":"2026-02-24T22:35:24.335353+0000","last_clean":"2026-02-24T22:35:24.335353+0000","last_became_active":"2026-02-24T22:23:29.201545+0000","last_became_peered":"2026-02-24T22:23:29.201545+0000","last_unstale":"2026-02-24T22:35:24.335353+0000","last_undegraded":"2026-02-24T22:35:24.335353+0000","last_fullsized":"2026-02-24T22:35:24.335353+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:35:24.335286+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:35:24.335286+0000","last_clean_scrub_stamp":"2026-02-24T22:35:24.335286+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:50:10.196109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"81.f","version":"0'0","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:14.643638+0000","last_change":"2026-02-24T22:35:14.643638+0000","last_active":"2026-02-24T22:35:14.643638+0000","last_peered":"2026-02-24T22:35:14.643638+0000","last_clean":"2026-02-24T22:35:14.643638+0000","last_became_active":"2026-02-24T22:15:25.824862+0000","last_became_peered":"2026-02-24T22:15:25.824862+0000","last_unstale":"2026-02-24T22:35:14.643638+0000","last_undegraded":"2026-02-24T22:35:14.643638+0000","last_fullsized":"2026-02-24T22:35:14.643638+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:14.643576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:14.643576+0000","last_clean_scrub_stamp":"2026-02-24T22:35:14.643576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:45:45.729521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.e","version":"312'5","reported_seq":62,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:00.220063+0000","last_change":"2026-02-24T22:35:00.220063+0000","last_active":"2026-02-24T22:35:00.220063+0000","last_peered":"2026-02-24T22:35:00.220063+0000","last_clean":"2026-02-24T22:35:00.220063+0000","last_became_active":"2026-02-24T22:15:22.817295+0000","last_became_peered":"2026-02-24T22:15:22.817295+0000","last_unstale":"2026-02-24T22:35:00.220063+0000","last_undegraded":"2026-02-24T22:35:00.220063+0000","last_fullsized":"2026-02-24T22:35:00.220063+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'5","last_scrub_stamp":"2026-02-24T22:35:00.220005+0000","last_deep_scrub":"312'5","last_deep_scrub_stamp":"2026-02-24T22:35:00.220005+0000","last_clean_scrub_stamp":"2026-02-24T22:35:00.220005+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:12:59.300665+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":25602,"num_write":5,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.11","version":"312'11","reported_seq":101,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:13.729005+0000","last_change":"2026-02-24T22:35:13.729005+0000","last_active":"2026-02-24T22:35:13.729005+0000","last_peered":"2026-02-24T22:35:13.729005+0000","last_clean":"2026-02-24T22:35:13.729005+0000","last_became_active":"2026-02-24T22:13:43.759864+0000","last_became_peered":"2026-02-24T22:13:43.759864+0000","last_unstale":"2026-02-24T22:35:13.729005+0000","last_undegraded":"2026-02-24T22:35:13.729005+0000","last_fullsized":"2026-02-24T22:35:13.729005+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'11","last_scrub_stamp":"2026-02-24T22:35:13.728935+0000","last_deep_scrub":"312'11","last_deep_scrub_stamp":"2026-02-24T22:35:13.728935+0000","last_clean_scrub_stamp":"2026-02-24T22:35:13.728935+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:48:10.252727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":53,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.d","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:13.577503+0000","last_change":"2026-02-24T22:35:13.577503+0000","last_active":"2026-02-24T22:35:13.577503+0000","last_peered":"2026-02-24T22:35:13.577503+0000","last_clean":"2026-02-24T22:35:13.577503+0000","last_became_active":"2026-02-24T22:23:29.214432+0000","last_became_peered":"2026-02-24T22:23:29.214432+0000","last_unstale":"2026-02-24T22:35:13.577503+0000","last_undegraded":"2026-02-24T22:35:13.577503+0000","last_fullsized":"2026-02-24T22:35:13.577503+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:13.577417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:13.577417+0000","last_clean_scrub_stamp":"2026-02-24T22:35:13.577417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:24:57.277938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.e","version":"0'0","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:15.020485+0000","last_change":"2026-02-24T22:35:15.020485+0000","last_active":"2026-02-24T22:35:15.020485+0000","last_peered":"2026-02-24T22:35:15.020485+0000","last_clean":"2026-02-24T22:35:15.020485+0000","last_became_active":"2026-02-24T22:15:25.827110+0000","last_became_peered":"2026-02-24T22:15:25.827110+0000","last_unstale":"2026-02-24T22:35:15.020485+0000","last_undegraded":"2026-02-24T22:35:15.020485+0000","last_fullsized":"2026-02-24T22:35:15.020485+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:15.020423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:15.020423+0000","last_clean_scrub_stamp":"2026-02-24T22:35:15.020423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:37:42.576080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.f","version":"312'8","reported_seq":66,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:18.779401+0000","last_change":"2026-02-24T22:35:18.779401+0000","last_active":"2026-02-24T22:35:18.779401+0000","last_peered":"2026-02-24T22:35:18.779401+0000","last_clean":"2026-02-24T22:35:18.779401+0000","last_became_active":"2026-02-24T22:15:22.815819+0000","last_became_peered":"2026-02-24T22:15:22.815819+0000","last_unstale":"2026-02-24T22:35:18.779401+0000","last_undegraded":"2026-02-24T22:35:18.779401+0000","last_fullsized":"2026-02-24T22:35:18.779401+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'8","last_scrub_stamp":"2026-02-24T22:35:18.779354+0000","last_deep_scrub":"312'8","last_deep_scrub_stamp":"2026-02-24T22:35:18.779354+0000","last_clean_scrub_stamp":"2026-02-24T22:35:18.779354+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:04:34.683909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":26626,"num_write":13,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.10","version":"312'31","reported_seq":161,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:10.702578+0000","last_change":"2026-02-24T22:35:10.702578+0000","last_active":"2026-02-24T22:35:10.702578+0000","last_peered":"2026-02-24T22:35:10.702578+0000","last_clean":"2026-02-24T22:35:10.702578+0000","last_became_active":"2026-02-24T22:13:43.759855+0000","last_became_peered":"2026-02-24T22:13:43.759855+0000","last_unstale":"2026-02-24T22:35:10.702578+0000","last_undegraded":"2026-02-24T22:35:10.702578+0000","last_fullsized":"2026-02-24T22:35:10.702578+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'31","last_scrub_stamp":"2026-02-24T22:35:10.702522+0000","last_deep_scrub":"312'31","last_deep_scrub_stamp":"2026-02-24T22:35:10.702522+0000","last_clean_scrub_stamp":"2026-02-24T22:35:10.702522+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:16:33.866764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":272,"num_read_kb":204,"num_write":72,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.12","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:16.558443+0000","last_change":"2026-02-24T22:35:16.558443+0000","last_active":"2026-02-24T22:35:16.558443+0000","last_peered":"2026-02-24T22:35:16.558443+0000","last_clean":"2026-02-24T22:35:16.558443+0000","last_became_active":"2026-02-24T22:23:29.212534+0000","last_became_peered":"2026-02-24T22:23:29.212534+0000","last_unstale":"2026-02-24T22:35:16.558443+0000","last_undegraded":"2026-02-24T22:35:16.558443+0000","last_fullsized":"2026-02-24T22:35:16.558443+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:16.558383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:16.558383+0000","last_clean_scrub_stamp":"2026-02-24T22:35:16.558383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:07:41.652982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.11","version":"312'5","reported_seq":54,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:15.611712+0000","last_change":"2026-02-24T22:35:15.611712+0000","last_active":"2026-02-24T22:35:15.611712+0000","last_peered":"2026-02-24T22:35:15.611712+0000","last_clean":"2026-02-24T22:35:15.611712+0000","last_became_active":"2026-02-24T22:15:25.826517+0000","last_became_peered":"2026-02-24T22:15:25.826517+0000","last_unstale":"2026-02-24T22:35:15.611712+0000","last_undegraded":"2026-02-24T22:35:15.611712+0000","last_fullsized":"2026-02-24T22:35:15.611712+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'5","last_scrub_stamp":"2026-02-24T22:35:15.611647+0000","last_deep_scrub":"312'5","last_deep_scrub_stamp":"2026-02-24T22:35:15.611647+0000","last_clean_scrub_stamp":"2026-02-24T22:35:15.611647+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:23:46.091088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":10,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.10","version":"0'0","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:10.152120+0000","last_change":"2026-02-24T22:35:10.152120+0000","last_active":"2026-02-24T22:35:10.152120+0000","last_peered":"2026-02-24T22:35:10.152120+0000","last_clean":"2026-02-24T22:35:10.152120+0000","last_became_active":"2026-02-24T22:15:22.810000+0000","last_became_peered":"2026-02-24T22:15:22.810000+0000","last_unstale":"2026-02-24T22:35:10.152120+0000","last_undegraded":"2026-02-24T22:35:10.152120+0000","last_fullsized":"2026-02-24T22:35:10.152120+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:10.152045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:10.152045+0000","last_clean_scrub_stamp":"2026-02-24T22:35:10.152045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:14:12.119607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"79.f","version":"312'11","reported_seq":103,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:11.110079+0000","last_change":"2026-02-24T22:35:11.110079+0000","last_active":"2026-02-24T22:35:11.110079+0000","last_peered":"2026-02-24T22:35:11.110079+0000","last_clean":"2026-02-24T22:35:11.110079+0000","last_became_active":"2026-02-24T22:13:43.754602+0000","last_became_peered":"2026-02-24T22:13:43.754602+0000","last_unstale":"2026-02-24T22:35:11.110079+0000","last_undegraded":"2026-02-24T22:35:11.110079+0000","last_fullsized":"2026-02-24T22:35:11.110079+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'11","last_scrub_stamp":"2026-02-24T22:35:11.110015+0000","last_deep_scrub":"312'11","last_deep_scrub_stamp":"2026-02-24T22:35:11.110015+0000","last_clean_scrub_stamp":"2026-02-24T22:35:11.110015+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:32:49.033187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":67,"num_write":27,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.13","version":"312'2","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:23.613569+0000","last_change":"2026-02-24T22:35:23.613569+0000","last_active":"2026-02-24T22:35:23.613569+0000","last_peered":"2026-02-24T22:35:23.613569+0000","last_clean":"2026-02-24T22:35:23.613569+0000","last_became_active":"2026-02-24T22:23:29.198627+0000","last_became_peered":"2026-02-24T22:23:29.198627+0000","last_unstale":"2026-02-24T22:35:23.613569+0000","last_undegraded":"2026-02-24T22:35:23.613569+0000","last_fullsized":"2026-02-24T22:35:23.613569+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:35:23.613520+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:35:23.613520+0000","last_clean_scrub_stamp":"2026-02-24T22:35:23.613520+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:04:39.063377+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.10","version":"0'0","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:16.599841+0000","last_change":"2026-02-24T22:35:16.599841+0000","last_active":"2026-02-24T22:35:16.599841+0000","last_peered":"2026-02-24T22:35:16.599841+0000","last_clean":"2026-02-24T22:35:16.599841+0000","last_became_active":"2026-02-24T22:15:25.826606+0000","last_became_peered":"2026-02-24T22:15:25.826606+0000","last_unstale":"2026-02-24T22:35:16.599841+0000","last_undegraded":"2026-02-24T22:35:16.599841+0000","last_fullsized":"2026-02-24T22:35:16.599841+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:16.599782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:16.599782+0000","last_clean_scrub_stamp":"2026-02-24T22:35:16.599782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:38:05.327614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.11","version":"0'0","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:12.081450+0000","last_change":"2026-02-24T22:35:12.081450+0000","last_active":"2026-02-24T22:35:12.081450+0000","last_peered":"2026-02-24T22:35:12.081450+0000","last_clean":"2026-02-24T22:35:12.081450+0000","last_became_active":"2026-02-24T22:15:22.809791+0000","last_became_peered":"2026-02-24T22:15:22.809791+0000","last_unstale":"2026-02-24T22:35:12.081450+0000","last_undegraded":"2026-02-24T22:35:12.081450+0000","last_fullsized":"2026-02-24T22:35:12.081450+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:12.081369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:12.081369+0000","last_clean_scrub_stamp":"2026-02-24T22:35:12.081369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:45:29.355115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"79.e","version":"312'23","reported_seq":134,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:08.185039+0000","last_change":"2026-02-24T22:35:08.185039+0000","last_active":"2026-02-24T22:35:08.185039+0000","last_peered":"2026-02-24T22:35:08.185039+0000","last_clean":"2026-02-24T22:35:08.185039+0000","last_became_active":"2026-02-24T22:13:43.755868+0000","last_became_peered":"2026-02-24T22:13:43.755868+0000","last_unstale":"2026-02-24T22:35:08.185039+0000","last_undegraded":"2026-02-24T22:35:08.185039+0000","last_fullsized":"2026-02-24T22:35:08.185039+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'23","last_scrub_stamp":"2026-02-24T22:35:08.184981+0000","last_deep_scrub":"312'23","last_deep_scrub_stamp":"2026-02-24T22:35:08.184981+0000","last_clean_scrub_stamp":"2026-02-24T22:35:08.184981+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:38:46.065895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":169,"num_read_kb":112,"num_write":56,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.10","version":"0'0","reported_seq":22,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:32:49.749407+0000","last_change":"2026-02-24T22:23:29.212895+0000","last_active":"2026-02-24T22:32:49.749407+0000","last_peered":"2026-02-24T22:32:49.749407+0000","last_clean":"2026-02-24T22:32:49.749407+0000","last_became_active":"2026-02-24T22:23:29.212665+0000","last_became_peered":"2026-02-24T22:23:29.212665+0000","last_unstale":"2026-02-24T22:32:49.749407+0000","last_undegraded":"2026-02-24T22:32:49.749407+0000","last_fullsized":"2026-02-24T22:32:49.749407+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:23:28.200283+0000","last_clean_scrub_stamp":"2026-02-24T22:23:28.200283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.13","version":"0'0","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:01.756922+0000","last_change":"2026-02-24T22:35:01.756922+0000","last_active":"2026-02-24T22:35:01.756922+0000","last_peered":"2026-02-24T22:35:01.756922+0000","last_clean":"2026-02-24T22:35:01.756922+0000","last_became_active":"2026-02-24T22:15:25.825862+0000","last_became_peered":"2026-02-24T22:15:25.825862+0000","last_unstale":"2026-02-24T22:35:01.756922+0000","last_undegraded":"2026-02-24T22:35:01.756922+0000","last_fullsized":"2026-02-24T22:35:01.756922+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:01.756854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:01.756854+0000","last_clean_scrub_stamp":"2026-02-24T22:35:01.756854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:53:27.843482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"80.12","version":"0'0","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:15.310359+0000","last_change":"2026-02-24T22:35:15.310359+0000","last_active":"2026-02-24T22:35:15.310359+0000","last_peered":"2026-02-24T22:35:15.310359+0000","last_clean":"2026-02-24T22:35:15.310359+0000","last_became_active":"2026-02-24T22:15:22.812405+0000","last_became_peered":"2026-02-24T22:15:22.812405+0000","last_unstale":"2026-02-24T22:35:15.310359+0000","last_undegraded":"2026-02-24T22:35:15.310359+0000","last_fullsized":"2026-02-24T22:35:15.310359+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:15.310191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:15.310191+0000","last_clean_scrub_stamp":"2026-02-24T22:35:15.310191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:35:30.889874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.d","version":"312'20","reported_seq":121,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:10.586519+0000","last_change":"2026-02-24T22:35:10.586519+0000","last_active":"2026-02-24T22:35:10.586519+0000","last_peered":"2026-02-24T22:35:10.586519+0000","last_clean":"2026-02-24T22:35:10.586519+0000","last_became_active":"2026-02-24T22:13:43.761339+0000","last_became_peered":"2026-02-24T22:13:43.761339+0000","last_unstale":"2026-02-24T22:35:10.586519+0000","last_undegraded":"2026-02-24T22:35:10.586519+0000","last_fullsized":"2026-02-24T22:35:10.586519+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'20","last_scrub_stamp":"2026-02-24T22:35:10.586422+0000","last_deep_scrub":"312'20","last_deep_scrub_stamp":"2026-02-24T22:35:10.586422+0000","last_clean_scrub_stamp":"2026-02-24T22:35:10.586422+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:07:09.238716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":126,"num_read_kb":76,"num_write":49,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"82.11","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:12.343554+0000","last_change":"2026-02-24T22:35:12.343554+0000","last_active":"2026-02-24T22:35:12.343554+0000","last_peered":"2026-02-24T22:35:12.343554+0000","last_clean":"2026-02-24T22:35:12.343554+0000","last_became_active":"2026-02-24T22:23:29.214167+0000","last_became_peered":"2026-02-24T22:23:29.214167+0000","last_unstale":"2026-02-24T22:35:12.343554+0000","last_undegraded":"2026-02-24T22:35:12.343554+0000","last_fullsized":"2026-02-24T22:35:12.343554+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:12.343445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:12.343445+0000","last_clean_scrub_stamp":"2026-02-24T22:35:12.343445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:38:27.753745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.12","version":"0'0","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:19.600664+0000","last_change":"2026-02-24T22:35:19.600664+0000","last_active":"2026-02-24T22:35:19.600664+0000","last_peered":"2026-02-24T22:35:19.600664+0000","last_clean":"2026-02-24T22:35:19.600664+0000","last_became_active":"2026-02-24T22:15:25.826102+0000","last_became_peered":"2026-02-24T22:15:25.826102+0000","last_unstale":"2026-02-24T22:35:19.600664+0000","last_undegraded":"2026-02-24T22:35:19.600664+0000","last_fullsized":"2026-02-24T22:35:19.600664+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:19.600611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:19.600611+0000","last_clean_scrub_stamp":"2026-02-24T22:35:19.600611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:29:15.870895+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.13","version":"312'3","reported_seq":51,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:16.310449+0000","last_change":"2026-02-24T22:35:16.310449+0000","last_active":"2026-02-24T22:35:16.310449+0000","last_peered":"2026-02-24T22:35:16.310449+0000","last_clean":"2026-02-24T22:35:16.310449+0000","last_became_active":"2026-02-24T22:15:22.811927+0000","last_became_peered":"2026-02-24T22:15:22.811927+0000","last_unstale":"2026-02-24T22:35:16.310449+0000","last_undegraded":"2026-02-24T22:35:16.310449+0000","last_fullsized":"2026-02-24T22:35:16.310449+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:16.310385+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:16.310385+0000","last_clean_scrub_stamp":"2026-02-24T22:35:16.310385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:28:26.685415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.c","version":"312'4","reported_seq":82,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:52.107283+0000","last_change":"2026-02-24T22:34:52.107283+0000","last_active":"2026-02-24T22:34:52.107283+0000","last_peered":"2026-02-24T22:34:52.107283+0000","last_clean":"2026-02-24T22:34:52.107283+0000","last_became_active":"2026-02-24T22:13:43.761910+0000","last_became_peered":"2026-02-24T22:13:43.761910+0000","last_unstale":"2026-02-24T22:34:52.107283+0000","last_undegraded":"2026-02-24T22:34:52.107283+0000","last_fullsized":"2026-02-24T22:34:52.107283+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'4","last_scrub_stamp":"2026-02-24T22:34:52.107217+0000","last_deep_scrub":"312'4","last_deep_scrub_stamp":"2026-02-24T22:34:52.107217+0000","last_clean_scrub_stamp":"2026-02-24T22:34:52.107217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:56:38.678014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":15,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"82.16","version":"0'0","reported_seq":29,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:06.210797+0000","last_change":"2026-02-24T22:35:06.210797+0000","last_active":"2026-02-24T22:35:06.210797+0000","last_peered":"2026-02-24T22:35:06.210797+0000","last_clean":"2026-02-24T22:35:06.210797+0000","last_became_active":"2026-02-24T22:23:29.213080+0000","last_became_peered":"2026-02-24T22:23:29.213080+0000","last_unstale":"2026-02-24T22:35:06.210797+0000","last_undegraded":"2026-02-24T22:35:06.210797+0000","last_fullsized":"2026-02-24T22:35:06.210797+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:06.210726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:06.210726+0000","last_clean_scrub_stamp":"2026-02-24T22:35:06.210726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:22:55.658806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.15","version":"0'0","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:05.258451+0000","last_change":"2026-02-24T22:35:05.258451+0000","last_active":"2026-02-24T22:35:05.258451+0000","last_peered":"2026-02-24T22:35:05.258451+0000","last_clean":"2026-02-24T22:35:05.258451+0000","last_became_active":"2026-02-24T22:15:25.830922+0000","last_became_peered":"2026-02-24T22:15:25.830922+0000","last_unstale":"2026-02-24T22:35:05.258451+0000","last_undegraded":"2026-02-24T22:35:05.258451+0000","last_fullsized":"2026-02-24T22:35:05.258451+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:05.258395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:05.258395+0000","last_clean_scrub_stamp":"2026-02-24T22:35:05.258395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:02:54.284713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"80.14","version":"312'3","reported_seq":51,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:17.344992+0000","last_change":"2026-02-24T22:35:17.344992+0000","last_active":"2026-02-24T22:35:17.344992+0000","last_peered":"2026-02-24T22:35:17.344992+0000","last_clean":"2026-02-24T22:35:17.344992+0000","last_became_active":"2026-02-24T22:15:22.811924+0000","last_became_peered":"2026-02-24T22:15:22.811924+0000","last_unstale":"2026-02-24T22:35:17.344992+0000","last_undegraded":"2026-02-24T22:35:17.344992+0000","last_fullsized":"2026-02-24T22:35:17.344992+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:17.344936+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:17.344936+0000","last_clean_scrub_stamp":"2026-02-24T22:35:17.344936+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:22:25.826405+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.b","version":"312'13","reported_seq":106,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:56.646904+0000","last_change":"2026-02-24T22:34:56.646904+0000","last_active":"2026-02-24T22:34:56.646904+0000","last_peered":"2026-02-24T22:34:56.646904+0000","last_clean":"2026-02-24T22:34:56.646904+0000","last_became_active":"2026-02-24T22:13:43.756596+0000","last_became_peered":"2026-02-24T22:13:43.756596+0000","last_unstale":"2026-02-24T22:34:56.646904+0000","last_undegraded":"2026-02-24T22:34:56.646904+0000","last_fullsized":"2026-02-24T22:34:56.646904+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'13","last_scrub_stamp":"2026-02-24T22:34:56.646848+0000","last_deep_scrub":"312'13","last_deep_scrub_stamp":"2026-02-24T22:34:56.646848+0000","last_clean_scrub_stamp":"2026-02-24T22:34:56.646848+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:06:44.716107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":56,"num_write":37,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.17","version":"0'0","reported_seq":29,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:07.200744+0000","last_change":"2026-02-24T22:35:07.200744+0000","last_active":"2026-02-24T22:35:07.200744+0000","last_peered":"2026-02-24T22:35:07.200744+0000","last_clean":"2026-02-24T22:35:07.200744+0000","last_became_active":"2026-02-24T22:23:29.214217+0000","last_became_peered":"2026-02-24T22:23:29.214217+0000","last_unstale":"2026-02-24T22:35:07.200744+0000","last_undegraded":"2026-02-24T22:35:07.200744+0000","last_fullsized":"2026-02-24T22:35:07.200744+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:07.200690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:07.200690+0000","last_clean_scrub_stamp":"2026-02-24T22:35:07.200690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:02:31.790157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.14","version":"312'5","reported_seq":50,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:18.571008+0000","last_change":"2026-02-24T22:35:18.571008+0000","last_active":"2026-02-24T22:35:18.571008+0000","last_peered":"2026-02-24T22:35:18.571008+0000","last_clean":"2026-02-24T22:35:18.571008+0000","last_became_active":"2026-02-24T22:15:25.827557+0000","last_became_peered":"2026-02-24T22:15:25.827557+0000","last_unstale":"2026-02-24T22:35:18.571008+0000","last_undegraded":"2026-02-24T22:35:18.571008+0000","last_fullsized":"2026-02-24T22:35:18.571008+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'5","last_scrub_stamp":"2026-02-24T22:35:18.570944+0000","last_deep_scrub":"312'5","last_deep_scrub_stamp":"2026-02-24T22:35:18.570944+0000","last_clean_scrub_stamp":"2026-02-24T22:35:18.570944+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:05:32.693769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":10,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.15","version":"312'5","reported_seq":56,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:19.259017+0000","last_change":"2026-02-24T22:35:19.259017+0000","last_active":"2026-02-24T22:35:19.259017+0000","last_peered":"2026-02-24T22:35:19.259017+0000","last_clean":"2026-02-24T22:35:19.259017+0000","last_became_active":"2026-02-24T22:15:22.812450+0000","last_became_peered":"2026-02-24T22:15:22.812450+0000","last_unstale":"2026-02-24T22:35:19.259017+0000","last_undegraded":"2026-02-24T22:35:19.259017+0000","last_fullsized":"2026-02-24T22:35:19.259017+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'5","last_scrub_stamp":"2026-02-24T22:35:19.258949+0000","last_deep_scrub":"312'5","last_deep_scrub_stamp":"2026-02-24T22:35:19.258949+0000","last_clean_scrub_stamp":"2026-02-24T22:35:19.258949+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:20:04.684086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":10,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.a","version":"312'10","reported_seq":99,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:01.350817+0000","last_change":"2026-02-24T22:35:01.350761+0000","last_active":"2026-02-24T22:35:01.350817+0000","last_peered":"2026-02-24T22:35:01.350817+0000","last_clean":"2026-02-24T22:35:01.350817+0000","last_became_active":"2026-02-24T22:13:43.762226+0000","last_became_peered":"2026-02-24T22:13:43.762226+0000","last_unstale":"2026-02-24T22:35:01.350817+0000","last_undegraded":"2026-02-24T22:35:01.350817+0000","last_fullsized":"2026-02-24T22:35:01.350817+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'10","last_scrub_stamp":"2026-02-24T22:35:01.350683+0000","last_deep_scrub":"312'10","last_deep_scrub_stamp":"2026-02-24T22:35:01.350683+0000","last_clean_scrub_stamp":"2026-02-24T22:35:01.350683+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:23:37.121406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":42,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.14","version":"312'2","reported_seq":42,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:21.601634+0000","last_change":"2026-02-24T22:35:21.601634+0000","last_active":"2026-02-24T22:35:21.601634+0000","last_peered":"2026-02-24T22:35:21.601634+0000","last_clean":"2026-02-24T22:35:21.601634+0000","last_became_active":"2026-02-24T22:24:01.297193+0000","last_became_peered":"2026-02-24T22:24:01.297193+0000","last_unstale":"2026-02-24T22:35:21.601634+0000","last_undegraded":"2026-02-24T22:35:21.601634+0000","last_fullsized":"2026-02-24T22:35:21.601634+0000","mapping_epoch":311,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":312,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:35:21.601576+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:35:21.601576+0000","last_clean_scrub_stamp":"2026-02-24T22:35:21.601576+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:59:47.986562+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.17","version":"312'3","reported_seq":44,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:23.867471+0000","last_change":"2026-02-24T22:35:23.867471+0000","last_active":"2026-02-24T22:35:23.867471+0000","last_peered":"2026-02-24T22:35:23.867471+0000","last_clean":"2026-02-24T22:35:23.867471+0000","last_became_active":"2026-02-24T22:15:25.833430+0000","last_became_peered":"2026-02-24T22:15:25.833430+0000","last_unstale":"2026-02-24T22:35:23.867471+0000","last_undegraded":"2026-02-24T22:35:23.867471+0000","last_fullsized":"2026-02-24T22:35:23.867471+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:23.867410+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:23.867410+0000","last_clean_scrub_stamp":"2026-02-24T22:35:23.867410+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:55:43.495676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"80.16","version":"0'0","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:01.238371+0000","last_change":"2026-02-24T22:35:01.238371+0000","last_active":"2026-02-24T22:35:01.238371+0000","last_peered":"2026-02-24T22:35:01.238371+0000","last_clean":"2026-02-24T22:35:01.238371+0000","last_became_active":"2026-02-24T22:15:22.817150+0000","last_became_peered":"2026-02-24T22:15:22.817150+0000","last_unstale":"2026-02-24T22:35:01.238371+0000","last_undegraded":"2026-02-24T22:35:01.238371+0000","last_fullsized":"2026-02-24T22:35:01.238371+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:01.238320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:01.238320+0000","last_clean_scrub_stamp":"2026-02-24T22:35:01.238320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:55:36.219572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.9","version":"312'24","reported_seq":134,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:09.179891+0000","last_change":"2026-02-24T22:35:09.179827+0000","last_active":"2026-02-24T22:35:09.179891+0000","last_peered":"2026-02-24T22:35:09.179891+0000","last_clean":"2026-02-24T22:35:09.179891+0000","last_became_active":"2026-02-24T22:13:43.756777+0000","last_became_peered":"2026-02-24T22:13:43.756777+0000","last_unstale":"2026-02-24T22:35:09.179891+0000","last_undegraded":"2026-02-24T22:35:09.179891+0000","last_fullsized":"2026-02-24T22:35:09.179891+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'24","last_scrub_stamp":"2026-02-24T22:35:09.179742+0000","last_deep_scrub":"312'24","last_deep_scrub_stamp":"2026-02-24T22:35:09.179742+0000","last_clean_scrub_stamp":"2026-02-24T22:35:09.179742+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:51:13.859546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":107,"num_write":60,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.15","version":"312'2","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:07.645456+0000","last_change":"2026-02-24T22:35:07.645456+0000","last_active":"2026-02-24T22:35:07.645456+0000","last_peered":"2026-02-24T22:35:07.645456+0000","last_clean":"2026-02-24T22:35:07.645456+0000","last_became_active":"2026-02-24T22:23:29.198000+0000","last_became_peered":"2026-02-24T22:23:29.198000+0000","last_unstale":"2026-02-24T22:35:07.645456+0000","last_undegraded":"2026-02-24T22:35:07.645456+0000","last_fullsized":"2026-02-24T22:35:07.645456+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:35:07.645381+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:35:07.645381+0000","last_clean_scrub_stamp":"2026-02-24T22:35:07.645381+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:16:38.802468+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.16","version":"0'0","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:17.598634+0000","last_change":"2026-02-24T22:35:17.598634+0000","last_active":"2026-02-24T22:35:17.598634+0000","last_peered":"2026-02-24T22:35:17.598634+0000","last_clean":"2026-02-24T22:35:17.598634+0000","last_became_active":"2026-02-24T22:15:25.828066+0000","last_became_peered":"2026-02-24T22:15:25.828066+0000","last_unstale":"2026-02-24T22:35:17.598634+0000","last_undegraded":"2026-02-24T22:35:17.598634+0000","last_fullsized":"2026-02-24T22:35:17.598634+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:17.598564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:17.598564+0000","last_clean_scrub_stamp":"2026-02-24T22:35:17.598564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:01:51.025597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.17","version":"0'0","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:19.812089+0000","last_change":"2026-02-24T22:35:19.812089+0000","last_active":"2026-02-24T22:35:19.812089+0000","last_peered":"2026-02-24T22:35:19.812089+0000","last_clean":"2026-02-24T22:35:19.812089+0000","last_became_active":"2026-02-24T22:15:22.817950+0000","last_became_peered":"2026-02-24T22:15:22.817950+0000","last_unstale":"2026-02-24T22:35:19.812089+0000","last_undegraded":"2026-02-24T22:35:19.812089+0000","last_fullsized":"2026-02-24T22:35:19.812089+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:19.812033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:19.812033+0000","last_clean_scrub_stamp":"2026-02-24T22:35:19.812033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:07:16.950514+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.8","version":"312'26","reported_seq":145,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:09.733420+0000","last_change":"2026-02-24T22:35:09.733355+0000","last_active":"2026-02-24T22:35:09.733420+0000","last_peered":"2026-02-24T22:35:09.733420+0000","last_clean":"2026-02-24T22:35:09.733420+0000","last_became_active":"2026-02-24T22:13:43.761558+0000","last_became_peered":"2026-02-24T22:13:43.761558+0000","last_unstale":"2026-02-24T22:35:09.733420+0000","last_undegraded":"2026-02-24T22:35:09.733420+0000","last_fullsized":"2026-02-24T22:35:09.733420+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'26","last_scrub_stamp":"2026-02-24T22:35:09.733252+0000","last_deep_scrub":"312'26","last_deep_scrub_stamp":"2026-02-24T22:35:09.733252+0000","last_clean_scrub_stamp":"2026-02-24T22:35:09.733252+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:51:48.673281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":152,"num_write":62,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.1a","version":"0'0","reported_seq":29,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:13.297685+0000","last_change":"2026-02-24T22:35:13.297685+0000","last_active":"2026-02-24T22:35:13.297685+0000","last_peered":"2026-02-24T22:35:13.297685+0000","last_clean":"2026-02-24T22:35:13.297685+0000","last_became_active":"2026-02-24T22:23:29.212709+0000","last_became_peered":"2026-02-24T22:23:29.212709+0000","last_unstale":"2026-02-24T22:35:13.297685+0000","last_undegraded":"2026-02-24T22:35:13.297685+0000","last_fullsized":"2026-02-24T22:35:13.297685+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:13.297633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:13.297633+0000","last_clean_scrub_stamp":"2026-02-24T22:35:13.297633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:03:00.921017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.19","version":"0'0","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:09.322953+0000","last_change":"2026-02-24T22:35:09.322953+0000","last_active":"2026-02-24T22:35:09.322953+0000","last_peered":"2026-02-24T22:35:09.322953+0000","last_clean":"2026-02-24T22:35:09.322953+0000","last_became_active":"2026-02-24T22:15:25.831885+0000","last_became_peered":"2026-02-24T22:15:25.831885+0000","last_unstale":"2026-02-24T22:35:09.322953+0000","last_undegraded":"2026-02-24T22:35:09.322953+0000","last_fullsized":"2026-02-24T22:35:09.322953+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:09.322906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:09.322906+0000","last_clean_scrub_stamp":"2026-02-24T22:35:09.322906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:18:38.861480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.18","version":"0'0","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:13.033058+0000","last_change":"2026-02-24T22:35:13.033058+0000","last_active":"2026-02-24T22:35:13.033058+0000","last_peered":"2026-02-24T22:35:13.033058+0000","last_clean":"2026-02-24T22:35:13.033058+0000","last_became_active":"2026-02-24T22:15:22.810156+0000","last_became_peered":"2026-02-24T22:15:22.810156+0000","last_unstale":"2026-02-24T22:35:13.033058+0000","last_undegraded":"2026-02-24T22:35:13.033058+0000","last_fullsized":"2026-02-24T22:35:13.033058+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:13.032975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:13.032975+0000","last_clean_scrub_stamp":"2026-02-24T22:35:13.032975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:48:05.525679+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"79.7","version":"0'0","reported_seq":73,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:08.320182+0000","last_change":"2026-02-24T22:35:08.320149+0000","last_active":"2026-02-24T22:35:08.320182+0000","last_peered":"2026-02-24T22:35:08.320182+0000","last_clean":"2026-02-24T22:35:08.320182+0000","last_became_active":"2026-02-24T22:13:43.755957+0000","last_became_peered":"2026-02-24T22:13:43.755957+0000","last_unstale":"2026-02-24T22:35:08.320182+0000","last_undegraded":"2026-02-24T22:35:08.320182+0000","last_fullsized":"2026-02-24T22:35:08.320182+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:08.320097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:08.320097+0000","last_clean_scrub_stamp":"2026-02-24T22:35:08.320097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:38:24.732745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.1b","version":"0'0","reported_seq":29,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:08.661691+0000","last_change":"2026-02-24T22:35:08.661691+0000","last_active":"2026-02-24T22:35:08.661691+0000","last_peered":"2026-02-24T22:35:08.661691+0000","last_clean":"2026-02-24T22:35:08.661691+0000","last_became_active":"2026-02-24T22:23:29.200152+0000","last_became_peered":"2026-02-24T22:23:29.200152+0000","last_unstale":"2026-02-24T22:35:08.661691+0000","last_undegraded":"2026-02-24T22:35:08.661691+0000","last_fullsized":"2026-02-24T22:35:08.661691+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:08.661629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:08.661629+0000","last_clean_scrub_stamp":"2026-02-24T22:35:08.661629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:43:02.912489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.18","version":"0'0","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:16.045841+0000","last_change":"2026-02-24T22:35:16.045841+0000","last_active":"2026-02-24T22:35:16.045841+0000","last_peered":"2026-02-24T22:35:16.045841+0000","last_clean":"2026-02-24T22:35:16.045841+0000","last_became_active":"2026-02-24T22:15:25.823415+0000","last_became_peered":"2026-02-24T22:15:25.823415+0000","last_unstale":"2026-02-24T22:35:16.045841+0000","last_undegraded":"2026-02-24T22:35:16.045841+0000","last_fullsized":"2026-02-24T22:35:16.045841+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:16.045769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:16.045769+0000","last_clean_scrub_stamp":"2026-02-24T22:35:16.045769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:46:21.973670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.19","version":"0'0","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:59.228352+0000","last_change":"2026-02-24T22:34:59.228352+0000","last_active":"2026-02-24T22:34:59.228352+0000","last_peered":"2026-02-24T22:34:59.228352+0000","last_clean":"2026-02-24T22:34:59.228352+0000","last_became_active":"2026-02-24T22:15:22.816826+0000","last_became_peered":"2026-02-24T22:15:22.816826+0000","last_unstale":"2026-02-24T22:34:59.228352+0000","last_undegraded":"2026-02-24T22:34:59.228352+0000","last_fullsized":"2026-02-24T22:34:59.228352+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:59.228267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:59.228267+0000","last_clean_scrub_stamp":"2026-02-24T22:34:59.228267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:53:36.867225+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.6","version":"301'14","reported_seq":117,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:07.698763+0000","last_change":"2026-02-24T22:35:07.698724+0000","last_active":"2026-02-24T22:35:07.698763+0000","last_peered":"2026-02-24T22:35:07.698763+0000","last_clean":"2026-02-24T22:35:07.698763+0000","last_became_active":"2026-02-24T22:13:43.754753+0000","last_became_peered":"2026-02-24T22:13:43.754753+0000","last_unstale":"2026-02-24T22:35:07.698763+0000","last_undegraded":"2026-02-24T22:35:07.698763+0000","last_fullsized":"2026-02-24T22:35:07.698763+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"301'14","last_scrub_stamp":"2026-02-24T22:35:07.698665+0000","last_deep_scrub":"301'14","last_deep_scrub_stamp":"2026-02-24T22:35:07.698665+0000","last_clean_scrub_stamp":"2026-02-24T22:35:07.698665+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:07:27.811145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":92,"num_write":34,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"82.18","version":"0'0","reported_seq":29,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:17.534786+0000","last_change":"2026-02-24T22:35:17.534786+0000","last_active":"2026-02-24T22:35:17.534786+0000","last_peered":"2026-02-24T22:35:17.534786+0000","last_clean":"2026-02-24T22:35:17.534786+0000","last_became_active":"2026-02-24T22:23:29.213387+0000","last_became_peered":"2026-02-24T22:23:29.213387+0000","last_unstale":"2026-02-24T22:35:17.534786+0000","last_undegraded":"2026-02-24T22:35:17.534786+0000","last_fullsized":"2026-02-24T22:35:17.534786+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:17.534737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:17.534737+0000","last_clean_scrub_stamp":"2026-02-24T22:35:17.534737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:50:16.258227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.1b","version":"0'0","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:10.315609+0000","last_change":"2026-02-24T22:35:10.315555+0000","last_active":"2026-02-24T22:35:10.315609+0000","last_peered":"2026-02-24T22:35:10.315609+0000","last_clean":"2026-02-24T22:35:10.315609+0000","last_became_active":"2026-02-24T22:15:25.832097+0000","last_became_peered":"2026-02-24T22:15:25.832097+0000","last_unstale":"2026-02-24T22:35:10.315609+0000","last_undegraded":"2026-02-24T22:35:10.315609+0000","last_fullsized":"2026-02-24T22:35:10.315609+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:10.315494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:10.315494+0000","last_clean_scrub_stamp":"2026-02-24T22:35:10.315494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:10:42.575778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.1a","version":"0'0","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:05.379885+0000","last_change":"2026-02-24T22:35:05.379833+0000","last_active":"2026-02-24T22:35:05.379885+0000","last_peered":"2026-02-24T22:35:05.379885+0000","last_clean":"2026-02-24T22:35:05.379885+0000","last_became_active":"2026-02-24T22:15:22.818885+0000","last_became_peered":"2026-02-24T22:15:22.818885+0000","last_unstale":"2026-02-24T22:35:05.379885+0000","last_undegraded":"2026-02-24T22:35:05.379885+0000","last_fullsized":"2026-02-24T22:35:05.379885+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:05.379757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:05.379757+0000","last_clean_scrub_stamp":"2026-02-24T22:35:05.379757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:37:10.955550+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"79.5","version":"312'19","reported_seq":130,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:07.304569+0000","last_change":"2026-02-24T22:35:07.304569+0000","last_active":"2026-02-24T22:35:07.304569+0000","last_peered":"2026-02-24T22:35:07.304569+0000","last_clean":"2026-02-24T22:35:07.304569+0000","last_became_active":"2026-02-24T22:13:43.755495+0000","last_became_peered":"2026-02-24T22:13:43.755495+0000","last_unstale":"2026-02-24T22:35:07.304569+0000","last_undegraded":"2026-02-24T22:35:07.304569+0000","last_fullsized":"2026-02-24T22:35:07.304569+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'19","last_scrub_stamp":"2026-02-24T22:35:07.304515+0000","last_deep_scrub":"312'19","last_deep_scrub_stamp":"2026-02-24T22:35:07.304515+0000","last_clean_scrub_stamp":"2026-02-24T22:35:07.304515+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:23:18.475886+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":164,"num_read_kb":112,"num_write":47,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.19","version":"0'0","reported_seq":29,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:08.226999+0000","last_change":"2026-02-24T22:35:08.226999+0000","last_active":"2026-02-24T22:35:08.226999+0000","last_peered":"2026-02-24T22:35:08.226999+0000","last_clean":"2026-02-24T22:35:08.226999+0000","last_became_active":"2026-02-24T22:23:29.212268+0000","last_became_peered":"2026-02-24T22:23:29.212268+0000","last_unstale":"2026-02-24T22:35:08.226999+0000","last_undegraded":"2026-02-24T22:35:08.226999+0000","last_fullsized":"2026-02-24T22:35:08.226999+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:08.226947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:08.226947+0000","last_clean_scrub_stamp":"2026-02-24T22:35:08.226947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:06:46.348170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.1a","version":"312'3","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:03.754654+0000","last_change":"2026-02-24T22:35:03.754600+0000","last_active":"2026-02-24T22:35:03.754654+0000","last_peered":"2026-02-24T22:35:03.754654+0000","last_clean":"2026-02-24T22:35:03.754654+0000","last_became_active":"2026-02-24T22:15:25.825969+0000","last_became_peered":"2026-02-24T22:15:25.825969+0000","last_unstale":"2026-02-24T22:35:03.754654+0000","last_undegraded":"2026-02-24T22:35:03.754654+0000","last_fullsized":"2026-02-24T22:35:03.754654+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:03.754545+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:03.754545+0000","last_clean_scrub_stamp":"2026-02-24T22:35:03.754545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:00:51.414538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"80.1b","version":"312'3","reported_seq":51,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:20.844174+0000","last_change":"2026-02-24T22:35:20.844132+0000","last_active":"2026-02-24T22:35:20.844174+0000","last_peered":"2026-02-24T22:35:20.844174+0000","last_clean":"2026-02-24T22:35:20.844174+0000","last_became_active":"2026-02-24T22:15:22.816479+0000","last_became_peered":"2026-02-24T22:15:22.816479+0000","last_unstale":"2026-02-24T22:35:20.844174+0000","last_undegraded":"2026-02-24T22:35:20.844174+0000","last_fullsized":"2026-02-24T22:35:20.844174+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:20.844063+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:20.844063+0000","last_clean_scrub_stamp":"2026-02-24T22:35:20.844063+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:11:47.595780+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.4","version":"0'0","reported_seq":79,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:06.735654+0000","last_change":"2026-02-24T22:35:06.735654+0000","last_active":"2026-02-24T22:35:06.735654+0000","last_peered":"2026-02-24T22:35:06.735654+0000","last_clean":"2026-02-24T22:35:06.735654+0000","last_became_active":"2026-02-24T22:14:00.741839+0000","last_became_peered":"2026-02-24T22:14:00.741839+0000","last_unstale":"2026-02-24T22:35:06.735654+0000","last_undegraded":"2026-02-24T22:35:06.735654+0000","last_fullsized":"2026-02-24T22:35:06.735654+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":293,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:06.735602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:06.735602+0000","last_clean_scrub_stamp":"2026-02-24T22:35:06.735602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:33:39.827521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"82.1e","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:15.299470+0000","last_change":"2026-02-24T22:35:15.299470+0000","last_active":"2026-02-24T22:35:15.299470+0000","last_peered":"2026-02-24T22:35:15.299470+0000","last_clean":"2026-02-24T22:35:15.299470+0000","last_became_active":"2026-02-24T22:23:29.215604+0000","last_became_peered":"2026-02-24T22:23:29.215604+0000","last_unstale":"2026-02-24T22:35:15.299470+0000","last_undegraded":"2026-02-24T22:35:15.299470+0000","last_fullsized":"2026-02-24T22:35:15.299470+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:15.299416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:15.299416+0000","last_clean_scrub_stamp":"2026-02-24T22:35:15.299416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:54:05.836081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.1d","version":"312'3","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:17.008630+0000","last_change":"2026-02-24T22:35:17.008586+0000","last_active":"2026-02-24T22:35:17.008630+0000","last_peered":"2026-02-24T22:35:17.008630+0000","last_clean":"2026-02-24T22:35:17.008630+0000","last_became_active":"2026-02-24T22:15:25.828335+0000","last_became_peered":"2026-02-24T22:15:25.828335+0000","last_unstale":"2026-02-24T22:35:17.008630+0000","last_undegraded":"2026-02-24T22:35:17.008630+0000","last_fullsized":"2026-02-24T22:35:17.008630+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:17.008525+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:17.008525+0000","last_clean_scrub_stamp":"2026-02-24T22:35:17.008525+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:58:43.331071+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.1c","version":"0'0","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:22.875409+0000","last_change":"2026-02-24T22:35:22.875368+0000","last_active":"2026-02-24T22:35:22.875409+0000","last_peered":"2026-02-24T22:35:22.875409+0000","last_clean":"2026-02-24T22:35:22.875409+0000","last_became_active":"2026-02-24T22:15:22.816548+0000","last_became_peered":"2026-02-24T22:15:22.816548+0000","last_unstale":"2026-02-24T22:35:22.875409+0000","last_undegraded":"2026-02-24T22:35:22.875409+0000","last_fullsized":"2026-02-24T22:35:22.875409+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:22.875303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:22.875303+0000","last_clean_scrub_stamp":"2026-02-24T22:35:22.875303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:36:00.190406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.3","version":"312'6","reported_seq":86,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:55.621542+0000","last_change":"2026-02-24T22:34:55.621542+0000","last_active":"2026-02-24T22:34:55.621542+0000","last_peered":"2026-02-24T22:34:55.621542+0000","last_clean":"2026-02-24T22:34:55.621542+0000","last_became_active":"2026-02-24T22:13:43.756408+0000","last_became_peered":"2026-02-24T22:13:43.756408+0000","last_unstale":"2026-02-24T22:34:55.621542+0000","last_undegraded":"2026-02-24T22:34:55.621542+0000","last_fullsized":"2026-02-24T22:34:55.621542+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'6","last_scrub_stamp":"2026-02-24T22:34:55.621486+0000","last_deep_scrub":"312'6","last_deep_scrub_stamp":"2026-02-24T22:34:55.621486+0000","last_clean_scrub_stamp":"2026-02-24T22:34:55.621486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:43:15.014040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":20,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.1f","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:22.633463+0000","last_change":"2026-02-24T22:35:22.633463+0000","last_active":"2026-02-24T22:35:22.633463+0000","last_peered":"2026-02-24T22:35:22.633463+0000","last_clean":"2026-02-24T22:35:22.633463+0000","last_became_active":"2026-02-24T22:23:29.199179+0000","last_became_peered":"2026-02-24T22:23:29.199179+0000","last_unstale":"2026-02-24T22:35:22.633463+0000","last_undegraded":"2026-02-24T22:35:22.633463+0000","last_fullsized":"2026-02-24T22:35:22.633463+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:22.633391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:22.633391+0000","last_clean_scrub_stamp":"2026-02-24T22:35:22.633391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:53:25.454787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.1c","version":"0'0","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:12.541442+0000","last_change":"2026-02-24T22:35:12.541386+0000","last_active":"2026-02-24T22:35:12.541442+0000","last_peered":"2026-02-24T22:35:12.541442+0000","last_clean":"2026-02-24T22:35:12.541442+0000","last_became_active":"2026-02-24T22:15:25.834072+0000","last_became_peered":"2026-02-24T22:15:25.834072+0000","last_unstale":"2026-02-24T22:35:12.541442+0000","last_undegraded":"2026-02-24T22:35:12.541442+0000","last_fullsized":"2026-02-24T22:35:12.541442+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:12.541294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:12.541294+0000","last_clean_scrub_stamp":"2026-02-24T22:35:12.541294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:17:35.101650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"80.1d","version":"312'2","reported_seq":49,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:21.851696+0000","last_change":"2026-02-24T22:35:21.851623+0000","last_active":"2026-02-24T22:35:21.851696+0000","last_peered":"2026-02-24T22:35:21.851696+0000","last_clean":"2026-02-24T22:35:21.851696+0000","last_became_active":"2026-02-24T22:15:22.816615+0000","last_became_peered":"2026-02-24T22:15:22.816615+0000","last_unstale":"2026-02-24T22:35:21.851696+0000","last_undegraded":"2026-02-24T22:35:21.851696+0000","last_fullsized":"2026-02-24T22:35:21.851696+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:35:21.851526+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:35:21.851526+0000","last_clean_scrub_stamp":"2026-02-24T22:35:21.851526+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:40:55.340169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.2","version":"312'5","reported_seq":86,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:07.232682+0000","last_change":"2026-02-24T22:35:07.232682+0000","last_active":"2026-02-24T22:35:07.232682+0000","last_peered":"2026-02-24T22:35:07.232682+0000","last_clean":"2026-02-24T22:35:07.232682+0000","last_became_active":"2026-02-24T22:13:43.755241+0000","last_became_peered":"2026-02-24T22:13:43.755241+0000","last_unstale":"2026-02-24T22:35:07.232682+0000","last_undegraded":"2026-02-24T22:35:07.232682+0000","last_fullsized":"2026-02-24T22:35:07.232682+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'5","last_scrub_stamp":"2026-02-24T22:35:07.232617+0000","last_deep_scrub":"312'5","last_deep_scrub_stamp":"2026-02-24T22:35:07.232617+0000","last_clean_scrub_stamp":"2026-02-24T22:35:07.232617+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:43:07.685953+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":23,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.1c","version":"312'2","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:09.205907+0000","last_change":"2026-02-24T22:35:09.205907+0000","last_active":"2026-02-24T22:35:09.205907+0000","last_peered":"2026-02-24T22:35:09.205907+0000","last_clean":"2026-02-24T22:35:09.205907+0000","last_became_active":"2026-02-24T22:23:29.213490+0000","last_became_peered":"2026-02-24T22:23:29.213490+0000","last_unstale":"2026-02-24T22:35:09.205907+0000","last_undegraded":"2026-02-24T22:35:09.205907+0000","last_fullsized":"2026-02-24T22:35:09.205907+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:35:09.205819+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:35:09.205819+0000","last_clean_scrub_stamp":"2026-02-24T22:35:09.205819+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:51:15.132591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.1f","version":"0'0","reported_seq":40,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:23.304100+0000","last_change":"2026-02-24T22:35:23.304100+0000","last_active":"2026-02-24T22:35:23.304100+0000","last_peered":"2026-02-24T22:35:23.304100+0000","last_clean":"2026-02-24T22:35:23.304100+0000","last_became_active":"2026-02-24T22:15:25.827873+0000","last_became_peered":"2026-02-24T22:15:25.827873+0000","last_unstale":"2026-02-24T22:35:23.304100+0000","last_undegraded":"2026-02-24T22:35:23.304100+0000","last_fullsized":"2026-02-24T22:35:23.304100+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:23.304014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:23.304014+0000","last_clean_scrub_stamp":"2026-02-24T22:35:23.304014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:31:32.463168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.1e","version":"0'0","reported_seq":47,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:04.400756+0000","last_change":"2026-02-24T22:35:04.400700+0000","last_active":"2026-02-24T22:35:04.400756+0000","last_peered":"2026-02-24T22:35:04.400756+0000","last_clean":"2026-02-24T22:35:04.400756+0000","last_became_active":"2026-02-24T22:15:22.817337+0000","last_became_peered":"2026-02-24T22:15:22.817337+0000","last_unstale":"2026-02-24T22:35:04.400756+0000","last_undegraded":"2026-02-24T22:35:04.400756+0000","last_fullsized":"2026-02-24T22:35:04.400756+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:04.400607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:04.400607+0000","last_clean_scrub_stamp":"2026-02-24T22:35:04.400607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:40:28.932410+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"79.1","version":"312'23","reported_seq":134,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:00.384907+0000","last_change":"2026-02-24T22:35:00.384907+0000","last_active":"2026-02-24T22:35:00.384907+0000","last_peered":"2026-02-24T22:35:00.384907+0000","last_clean":"2026-02-24T22:35:00.384907+0000","last_became_active":"2026-02-24T22:13:43.760933+0000","last_became_peered":"2026-02-24T22:13:43.760933+0000","last_unstale":"2026-02-24T22:35:00.384907+0000","last_undegraded":"2026-02-24T22:35:00.384907+0000","last_fullsized":"2026-02-24T22:35:00.384907+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'23","last_scrub_stamp":"2026-02-24T22:35:00.384839+0000","last_deep_scrub":"312'23","last_deep_scrub_stamp":"2026-02-24T22:35:00.384839+0000","last_clean_scrub_stamp":"2026-02-24T22:35:00.384839+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:47:05.415176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":174,"num_read_kb":113,"num_write":59,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.1d","version":"0'0","reported_seq":29,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:14.322423+0000","last_change":"2026-02-24T22:35:14.322423+0000","last_active":"2026-02-24T22:35:14.322423+0000","last_peered":"2026-02-24T22:35:14.322423+0000","last_clean":"2026-02-24T22:35:14.322423+0000","last_became_active":"2026-02-24T22:23:29.215334+0000","last_became_peered":"2026-02-24T22:23:29.215334+0000","last_unstale":"2026-02-24T22:35:14.322423+0000","last_undegraded":"2026-02-24T22:35:14.322423+0000","last_fullsized":"2026-02-24T22:35:14.322423+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:14.322351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:14.322351+0000","last_clean_scrub_stamp":"2026-02-24T22:35:14.322351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:09:08.411510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.1e","version":"0'0","reported_seq":40,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:04.737646+0000","last_change":"2026-02-24T22:35:04.737646+0000","last_active":"2026-02-24T22:35:04.737646+0000","last_peered":"2026-02-24T22:35:04.737646+0000","last_clean":"2026-02-24T22:35:04.737646+0000","last_became_active":"2026-02-24T22:15:25.827124+0000","last_became_peered":"2026-02-24T22:15:25.827124+0000","last_unstale":"2026-02-24T22:35:04.737646+0000","last_undegraded":"2026-02-24T22:35:04.737646+0000","last_fullsized":"2026-02-24T22:35:04.737646+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:04.737570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:04.737570+0000","last_clean_scrub_stamp":"2026-02-24T22:35:04.737570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:38:11.206092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"80.1f","version":"312'3","reported_seq":52,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:02.210357+0000","last_change":"2026-02-24T22:35:02.210357+0000","last_active":"2026-02-24T22:35:02.210357+0000","last_peered":"2026-02-24T22:35:02.210357+0000","last_clean":"2026-02-24T22:35:02.210357+0000","last_became_active":"2026-02-24T22:15:22.815938+0000","last_became_peered":"2026-02-24T22:15:22.815938+0000","last_unstale":"2026-02-24T22:35:02.210357+0000","last_undegraded":"2026-02-24T22:35:02.210357+0000","last_fullsized":"2026-02-24T22:35:02.210357+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:02.210297+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:02.210297+0000","last_clean_scrub_stamp":"2026-02-24T22:35:02.210297+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:26:34.542452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.0","version":"0'0","reported_seq":73,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:05.263755+0000","last_change":"2026-02-24T22:35:05.263755+0000","last_active":"2026-02-24T22:35:05.263755+0000","last_peered":"2026-02-24T22:35:05.263755+0000","last_clean":"2026-02-24T22:35:05.263755+0000","last_became_active":"2026-02-24T22:13:43.755266+0000","last_became_peered":"2026-02-24T22:13:43.755266+0000","last_unstale":"2026-02-24T22:35:05.263755+0000","last_undegraded":"2026-02-24T22:35:05.263755+0000","last_fullsized":"2026-02-24T22:35:05.263755+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:05.263697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:05.263697+0000","last_clean_scrub_stamp":"2026-02-24T22:35:05.263697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:19:26.298794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"312'276","reported_seq":990,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.099244+0000","last_change":"2026-02-24T22:34:44.099244+0000","last_active":"2026-02-24T22:34:44.099244+0000","last_peered":"2026-02-24T22:34:44.099244+0000","last_clean":"2026-02-24T22:34:44.099244+0000","last_became_active":"2026-02-24T22:05:16.192589+0000","last_became_peered":"2026-02-24T22:05:16.192589+0000","last_unstale":"2026-02-24T22:34:44.099244+0000","last_undegraded":"2026-02-24T22:34:44.099244+0000","last_fullsized":"2026-02-24T22:34:44.099244+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'276","last_scrub_stamp":"2026-02-24T22:34:44.099189+0000","last_deep_scrub":"312'276","last_deep_scrub_stamp":"2026-02-24T22:34:44.099189+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.099189+0000","objects_scrubbed":5,"log_size":76,"log_dups_size":200,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:12:08.217874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":433,"num_read_kb":342,"num_write":1269,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"312'220","reported_seq":931,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.919442+0000","last_change":"2026-02-24T22:34:46.919413+0000","last_active":"2026-02-24T22:34:46.919442+0000","last_peered":"2026-02-24T22:34:46.919442+0000","last_clean":"2026-02-24T22:34:46.919442+0000","last_became_active":"2026-02-24T22:05:18.192203+0000","last_became_peered":"2026-02-24T22:05:18.192203+0000","last_unstale":"2026-02-24T22:34:46.919442+0000","last_undegraded":"2026-02-24T22:34:46.919442+0000","last_fullsized":"2026-02-24T22:34:46.919442+0000","mapping_epoch":24,"log_start":"312'197","ondisk_log_start":"312'197","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'220","last_scrub_stamp":"2026-02-24T22:34:46.919375+0000","last_deep_scrub":"312'220","last_deep_scrub_stamp":"2026-02-24T22:34:46.919375+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.919375+0000","objects_scrubbed":23,"log_size":23,"log_dups_size":197,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:53:24.900487+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":780,"num_read_kb":643,"num_write":282,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"312'380","reported_seq":1197,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.032775+0000","last_change":"2026-02-24T22:34:36.032775+0000","last_active":"2026-02-24T22:34:36.032775+0000","last_peered":"2026-02-24T22:34:36.032775+0000","last_clean":"2026-02-24T22:34:36.032775+0000","last_became_active":"2026-02-24T22:05:16.181379+0000","last_became_peered":"2026-02-24T22:05:16.181379+0000","last_unstale":"2026-02-24T22:34:36.032775+0000","last_undegraded":"2026-02-24T22:34:36.032775+0000","last_fullsized":"2026-02-24T22:34:36.032775+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'380","last_scrub_stamp":"2026-02-24T22:34:36.032715+0000","last_deep_scrub":"312'380","last_deep_scrub_stamp":"2026-02-24T22:34:36.032715+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.032715+0000","objects_scrubbed":2,"log_size":80,"log_dups_size":300,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:22:39.124170+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":922,"num_read_kb":611,"num_write":1540,"num_write_kb":5240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"312'487","reported_seq":1295,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.222374+0000","last_change":"2026-02-24T22:34:47.222322+0000","last_active":"2026-02-24T22:34:47.222374+0000","last_peered":"2026-02-24T22:34:47.222374+0000","last_clean":"2026-02-24T22:34:47.222374+0000","last_became_active":"2026-02-24T22:05:18.190689+0000","last_became_peered":"2026-02-24T22:05:18.190689+0000","last_unstale":"2026-02-24T22:34:47.222374+0000","last_undegraded":"2026-02-24T22:34:47.222374+0000","last_fullsized":"2026-02-24T22:34:47.222374+0000","mapping_epoch":24,"log_start":"312'400","ondisk_log_start":"312'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'487","last_scrub_stamp":"2026-02-24T22:34:47.222271+0000","last_deep_scrub":"312'487","last_deep_scrub_stamp":"2026-02-24T22:34:47.222271+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.222271+0000","objects_scrubbed":28,"log_size":87,"log_dups_size":400,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:40:22.630753+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":2017,"num_read_kb":1670,"num_write":788,"num_write_kb":502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"312'453","reported_seq":1248,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.730366+0000","last_change":"2026-02-24T22:34:38.730366+0000","last_active":"2026-02-24T22:34:38.730366+0000","last_peered":"2026-02-24T22:34:38.730366+0000","last_clean":"2026-02-24T22:34:38.730366+0000","last_became_active":"2026-02-24T22:05:16.180681+0000","last_became_peered":"2026-02-24T22:05:16.180681+0000","last_unstale":"2026-02-24T22:34:38.730366+0000","last_undegraded":"2026-02-24T22:34:38.730366+0000","last_fullsized":"2026-02-24T22:34:38.730366+0000","mapping_epoch":22,"log_start":"312'400","ondisk_log_start":"312'400","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'453","last_scrub_stamp":"2026-02-24T22:34:38.730307+0000","last_deep_scrub":"312'453","last_deep_scrub_stamp":"2026-02-24T22:34:38.730307+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.730307+0000","objects_scrubbed":5,"log_size":53,"log_dups_size":400,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:09:15.011742+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242910,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":903,"num_read_kb":6172,"num_write":1607,"num_write_kb":5459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"312'102","reported_seq":759,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.416426+0000","last_change":"2026-02-24T22:34:48.416387+0000","last_active":"2026-02-24T22:34:48.416426+0000","last_peered":"2026-02-24T22:34:48.416426+0000","last_clean":"2026-02-24T22:34:48.416426+0000","last_became_active":"2026-02-24T22:05:18.190727+0000","last_became_peered":"2026-02-24T22:05:18.190727+0000","last_unstale":"2026-02-24T22:34:48.416426+0000","last_undegraded":"2026-02-24T22:34:48.416426+0000","last_fullsized":"2026-02-24T22:34:48.416426+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'102","last_scrub_stamp":"2026-02-24T22:34:48.416304+0000","last_deep_scrub":"312'102","last_deep_scrub_stamp":"2026-02-24T22:34:48.416304+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.416304+0000","objects_scrubbed":22,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:05:48.774357+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":268,"num_read_kb":220,"num_write":112,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"312'294","reported_seq":987,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.956302+0000","last_change":"2026-02-24T22:34:43.956244+0000","last_active":"2026-02-24T22:34:43.956302+0000","last_peered":"2026-02-24T22:34:43.956302+0000","last_clean":"2026-02-24T22:34:43.956302+0000","last_became_active":"2026-02-24T22:05:16.192922+0000","last_became_peered":"2026-02-24T22:05:16.192922+0000","last_unstale":"2026-02-24T22:34:43.956302+0000","last_undegraded":"2026-02-24T22:34:43.956302+0000","last_fullsized":"2026-02-24T22:34:43.956302+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'294","last_scrub_stamp":"2026-02-24T22:34:43.956165+0000","last_deep_scrub":"312'294","last_deep_scrub_stamp":"2026-02-24T22:34:43.956165+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.956165+0000","objects_scrubbed":5,"log_size":94,"log_dups_size":200,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:47:34.541512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":378,"num_read_kb":3190,"num_write":1202,"num_write_kb":12582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"312'158","reported_seq":848,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.908280+0000","last_change":"2026-02-24T22:34:47.908235+0000","last_active":"2026-02-24T22:34:47.908280+0000","last_peered":"2026-02-24T22:34:47.908280+0000","last_clean":"2026-02-24T22:34:47.908280+0000","last_became_active":"2026-02-24T22:05:18.184014+0000","last_became_peered":"2026-02-24T22:05:18.184014+0000","last_unstale":"2026-02-24T22:34:47.908280+0000","last_undegraded":"2026-02-24T22:34:47.908280+0000","last_fullsized":"2026-02-24T22:34:47.908280+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'158","last_scrub_stamp":"2026-02-24T22:34:47.908167+0000","last_deep_scrub":"312'158","last_deep_scrub_stamp":"2026-02-24T22:34:47.908167+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.908167+0000","objects_scrubbed":23,"log_size":58,"log_dups_size":100,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:57:20.364094+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":489,"num_read_kb":401,"num_write":208,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"312'280","reported_seq":1000,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.068519+0000","last_change":"2026-02-24T22:34:43.068430+0000","last_active":"2026-02-24T22:34:43.068519+0000","last_peered":"2026-02-24T22:34:43.068519+0000","last_clean":"2026-02-24T22:34:43.068519+0000","last_became_active":"2026-02-24T22:05:16.191055+0000","last_became_peered":"2026-02-24T22:05:16.191055+0000","last_unstale":"2026-02-24T22:34:43.068519+0000","last_undegraded":"2026-02-24T22:34:43.068519+0000","last_fullsized":"2026-02-24T22:34:43.068519+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'280","last_scrub_stamp":"2026-02-24T22:34:43.068300+0000","last_deep_scrub":"312'280","last_deep_scrub_stamp":"2026-02-24T22:34:43.068300+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.068300+0000","objects_scrubbed":9,"log_size":80,"log_dups_size":200,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:55:12.515669+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":135,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":423,"num_read_kb":15420,"num_write":1201,"num_write_kb":8454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"312'497","reported_seq":1223,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.721601+0000","last_change":"2026-02-24T22:34:42.721550+0000","last_active":"2026-02-24T22:34:42.721601+0000","last_peered":"2026-02-24T22:34:42.721601+0000","last_clean":"2026-02-24T22:34:42.721601+0000","last_became_active":"2026-02-24T22:05:18.190598+0000","last_became_peered":"2026-02-24T22:05:18.190598+0000","last_unstale":"2026-02-24T22:34:42.721601+0000","last_undegraded":"2026-02-24T22:34:42.721601+0000","last_fullsized":"2026-02-24T22:34:42.721601+0000","mapping_epoch":24,"log_start":"312'400","ondisk_log_start":"312'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'497","last_scrub_stamp":"2026-02-24T22:34:42.721473+0000","last_deep_scrub":"312'497","last_deep_scrub_stamp":"2026-02-24T22:34:42.721473+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.721473+0000","objects_scrubbed":27,"log_size":97,"log_dups_size":400,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:09:26.432917+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1905,"num_read_kb":1522,"num_write":772,"num_write_kb":389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":267,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"312'306","reported_seq":1025,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.065368+0000","last_change":"2026-02-24T22:34:41.065317+0000","last_active":"2026-02-24T22:34:41.065368+0000","last_peered":"2026-02-24T22:34:41.065368+0000","last_clean":"2026-02-24T22:34:41.065368+0000","last_became_active":"2026-02-24T22:05:16.191164+0000","last_became_peered":"2026-02-24T22:05:16.191164+0000","last_unstale":"2026-02-24T22:34:41.065368+0000","last_undegraded":"2026-02-24T22:34:41.065368+0000","last_fullsized":"2026-02-24T22:34:41.065368+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'306","last_scrub_stamp":"2026-02-24T22:34:41.065249+0000","last_deep_scrub":"312'306","last_deep_scrub_stamp":"2026-02-24T22:34:41.065249+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.065249+0000","objects_scrubbed":6,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:42:26.339639+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":491,"num_read_kb":5551,"num_write":1337,"num_write_kb":5240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"312'74","reported_seq":737,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:50.335113+0000","last_change":"2026-02-24T22:34:50.335113+0000","last_active":"2026-02-24T22:34:50.335113+0000","last_peered":"2026-02-24T22:34:50.335113+0000","last_clean":"2026-02-24T22:34:50.335113+0000","last_became_active":"2026-02-24T22:05:18.198559+0000","last_became_peered":"2026-02-24T22:05:18.198559+0000","last_unstale":"2026-02-24T22:34:50.335113+0000","last_undegraded":"2026-02-24T22:34:50.335113+0000","last_fullsized":"2026-02-24T22:34:50.335113+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'74","last_scrub_stamp":"2026-02-24T22:34:50.335079+0000","last_deep_scrub":"312'74","last_deep_scrub_stamp":"2026-02-24T22:34:50.335079+0000","last_clean_scrub_stamp":"2026-02-24T22:34:50.335079+0000","objects_scrubbed":18,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:21:37.995762+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":195,"num_read_kb":167,"num_write":70,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"312'304","reported_seq":1013,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.096476+0000","last_change":"2026-02-24T22:34:42.096430+0000","last_active":"2026-02-24T22:34:42.096476+0000","last_peered":"2026-02-24T22:34:42.096476+0000","last_clean":"2026-02-24T22:34:42.096476+0000","last_became_active":"2026-02-24T22:05:16.190367+0000","last_became_peered":"2026-02-24T22:05:16.190367+0000","last_unstale":"2026-02-24T22:34:42.096476+0000","last_undegraded":"2026-02-24T22:34:42.096476+0000","last_fullsized":"2026-02-24T22:34:42.096476+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'304","last_scrub_stamp":"2026-02-24T22:34:42.096375+0000","last_deep_scrub":"312'304","last_deep_scrub_stamp":"2026-02-24T22:34:42.096375+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.096375+0000","objects_scrubbed":4,"log_size":104,"log_dups_size":200,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:48:33.407036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":439,"num_read_kb":1335,"num_write":1233,"num_write_kb":1104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"312'237","reported_seq":988,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.882613+0000","last_change":"2026-02-24T22:34:48.882613+0000","last_active":"2026-02-24T22:34:48.882613+0000","last_peered":"2026-02-24T22:34:48.882613+0000","last_clean":"2026-02-24T22:34:48.882613+0000","last_became_active":"2026-02-24T22:05:18.191767+0000","last_became_peered":"2026-02-24T22:05:18.191767+0000","last_unstale":"2026-02-24T22:34:48.882613+0000","last_undegraded":"2026-02-24T22:34:48.882613+0000","last_fullsized":"2026-02-24T22:34:48.882613+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'237","last_scrub_stamp":"2026-02-24T22:34:48.882561+0000","last_deep_scrub":"312'237","last_deep_scrub_stamp":"2026-02-24T22:34:48.882561+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.882561+0000","objects_scrubbed":36,"log_size":37,"log_dups_size":200,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:24:34.587031+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":876,"num_read_kb":754,"num_write":323,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"312'364","reported_seq":1143,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.830560+0000","last_change":"2026-02-24T22:34:36.830513+0000","last_active":"2026-02-24T22:34:36.830560+0000","last_peered":"2026-02-24T22:34:36.830560+0000","last_clean":"2026-02-24T22:34:36.830560+0000","last_became_active":"2026-02-24T22:05:16.185779+0000","last_became_peered":"2026-02-24T22:05:16.185779+0000","last_unstale":"2026-02-24T22:34:36.830560+0000","last_undegraded":"2026-02-24T22:34:36.830560+0000","last_fullsized":"2026-02-24T22:34:36.830560+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'364","last_scrub_stamp":"2026-02-24T22:34:36.830446+0000","last_deep_scrub":"312'364","last_deep_scrub_stamp":"2026-02-24T22:34:36.830446+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.830446+0000","objects_scrubbed":6,"log_size":64,"log_dups_size":300,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:49:17.279372+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":619,"num_read_kb":883,"num_write":1617,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"312'109","reported_seq":765,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.090042+0000","last_change":"2026-02-24T22:34:41.090042+0000","last_active":"2026-02-24T22:34:41.090042+0000","last_peered":"2026-02-24T22:34:41.090042+0000","last_clean":"2026-02-24T22:34:41.090042+0000","last_became_active":"2026-02-24T22:05:18.187838+0000","last_became_peered":"2026-02-24T22:05:18.187838+0000","last_unstale":"2026-02-24T22:34:41.090042+0000","last_undegraded":"2026-02-24T22:34:41.090042+0000","last_fullsized":"2026-02-24T22:34:41.090042+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'109","last_scrub_stamp":"2026-02-24T22:34:41.089971+0000","last_deep_scrub":"312'109","last_deep_scrub_stamp":"2026-02-24T22:34:41.089971+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.089971+0000","objects_scrubbed":26,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:13:12.250060+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":263,"num_read_kb":217,"num_write":106,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":719,"num_omap_keys":3,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"312'164","reported_seq":846,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.431711+0000","last_change":"2026-02-24T22:34:47.431711+0000","last_active":"2026-02-24T22:34:47.431711+0000","last_peered":"2026-02-24T22:34:47.431711+0000","last_clean":"2026-02-24T22:34:47.431711+0000","last_became_active":"2026-02-24T22:05:18.191625+0000","last_became_peered":"2026-02-24T22:05:18.191625+0000","last_unstale":"2026-02-24T22:34:47.431711+0000","last_undegraded":"2026-02-24T22:34:47.431711+0000","last_fullsized":"2026-02-24T22:34:47.431711+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'164","last_scrub_stamp":"2026-02-24T22:34:47.431652+0000","last_deep_scrub":"312'164","last_deep_scrub_stamp":"2026-02-24T22:34:47.431652+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.431652+0000","objects_scrubbed":29,"log_size":64,"log_dups_size":100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:13:36.629505+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":477,"num_read_kb":396,"num_write":183,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"312'302","reported_seq":1047,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.046802+0000","last_change":"2026-02-24T22:34:41.046802+0000","last_active":"2026-02-24T22:34:41.046802+0000","last_peered":"2026-02-24T22:34:41.046802+0000","last_clean":"2026-02-24T22:34:41.046802+0000","last_became_active":"2026-02-24T22:06:01.025659+0000","last_became_peered":"2026-02-24T22:06:01.025659+0000","last_unstale":"2026-02-24T22:34:41.046802+0000","last_undegraded":"2026-02-24T22:34:41.046802+0000","last_fullsized":"2026-02-24T22:34:41.046802+0000","mapping_epoch":36,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'302","last_scrub_stamp":"2026-02-24T22:34:41.046752+0000","last_deep_scrub":"312'302","last_deep_scrub_stamp":"2026-02-24T22:34:41.046752+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.046752+0000","objects_scrubbed":2,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:23:49.712428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":514,"num_read_kb":16221,"num_write":1317,"num_write_kb":8500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"312'4268","reported_seq":5087,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.193741+0000","last_change":"2026-02-24T22:34:46.193741+0000","last_active":"2026-02-24T22:34:46.193741+0000","last_peered":"2026-02-24T22:34:46.193741+0000","last_clean":"2026-02-24T22:34:46.193741+0000","last_became_active":"2026-02-24T22:05:18.193257+0000","last_became_peered":"2026-02-24T22:05:18.193257+0000","last_unstale":"2026-02-24T22:34:46.193741+0000","last_undegraded":"2026-02-24T22:34:46.193741+0000","last_fullsized":"2026-02-24T22:34:46.193741+0000","mapping_epoch":24,"log_start":"312'4200","ondisk_log_start":"312'4200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'4268","last_scrub_stamp":"2026-02-24T22:34:46.193682+0000","last_deep_scrub":"312'4268","last_deep_scrub_stamp":"2026-02-24T22:34:46.193682+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.193682+0000","objects_scrubbed":19,"log_size":68,"log_dups_size":3000,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:22:49.449922+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":17332,"num_read_kb":14553,"num_write":6398,"num_write_kb":3198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"312'289","reported_seq":1009,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.725396+0000","last_change":"2026-02-24T22:34:37.725396+0000","last_active":"2026-02-24T22:34:37.725396+0000","last_peered":"2026-02-24T22:34:37.725396+0000","last_clean":"2026-02-24T22:34:37.725396+0000","last_became_active":"2026-02-24T22:05:16.190819+0000","last_became_peered":"2026-02-24T22:05:16.190819+0000","last_unstale":"2026-02-24T22:34:37.725396+0000","last_undegraded":"2026-02-24T22:34:37.725396+0000","last_fullsized":"2026-02-24T22:34:37.725396+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'289","last_scrub_stamp":"2026-02-24T22:34:37.725293+0000","last_deep_scrub":"312'289","last_deep_scrub_stamp":"2026-02-24T22:34:37.725293+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.725293+0000","objects_scrubbed":5,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:58:32.616383+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242922,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":466,"num_read_kb":8113,"num_write":1161,"num_write_kb":5510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"312'883","reported_seq":4054,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.966939+0000","last_change":"2026-02-24T22:34:45.966939+0000","last_active":"2026-02-24T22:34:45.966939+0000","last_peered":"2026-02-24T22:34:45.966939+0000","last_clean":"2026-02-24T22:34:45.966939+0000","last_became_active":"2026-02-24T22:05:18.190363+0000","last_became_peered":"2026-02-24T22:05:18.190363+0000","last_unstale":"2026-02-24T22:34:45.966939+0000","last_undegraded":"2026-02-24T22:34:45.966939+0000","last_fullsized":"2026-02-24T22:34:45.966939+0000","mapping_epoch":24,"log_start":"312'800","ondisk_log_start":"312'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'883","last_scrub_stamp":"2026-02-24T22:34:45.966870+0000","last_deep_scrub":"312'883","last_deep_scrub_stamp":"2026-02-24T22:34:45.966870+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.966870+0000","objects_scrubbed":21,"log_size":83,"log_dups_size":800,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:41:10.952477+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4747,"num_read_kb":5785,"num_write":398,"num_write_kb":236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"312'323","reported_seq":1062,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.725432+0000","last_change":"2026-02-24T22:34:35.725432+0000","last_active":"2026-02-24T22:34:35.725432+0000","last_peered":"2026-02-24T22:34:35.725432+0000","last_clean":"2026-02-24T22:34:35.725432+0000","last_became_active":"2026-02-24T22:05:16.190816+0000","last_became_peered":"2026-02-24T22:05:16.190816+0000","last_unstale":"2026-02-24T22:34:35.725432+0000","last_undegraded":"2026-02-24T22:34:35.725432+0000","last_fullsized":"2026-02-24T22:34:35.725432+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'323","last_scrub_stamp":"2026-02-24T22:34:35.725350+0000","last_deep_scrub":"312'323","last_deep_scrub_stamp":"2026-02-24T22:34:35.725350+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.725350+0000","objects_scrubbed":2,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:35:43.171065+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":544,"num_read_kb":2327,"num_write":1304,"num_write_kb":12235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"312'120","reported_seq":832,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.062558+0000","last_change":"2026-02-24T22:34:38.062558+0000","last_active":"2026-02-24T22:34:38.062558+0000","last_peered":"2026-02-24T22:34:38.062558+0000","last_clean":"2026-02-24T22:34:38.062558+0000","last_became_active":"2026-02-24T22:05:18.186994+0000","last_became_peered":"2026-02-24T22:05:18.186994+0000","last_unstale":"2026-02-24T22:34:38.062558+0000","last_undegraded":"2026-02-24T22:34:38.062558+0000","last_fullsized":"2026-02-24T22:34:38.062558+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'120","last_scrub_stamp":"2026-02-24T22:34:38.062523+0000","last_deep_scrub":"312'120","last_deep_scrub_stamp":"2026-02-24T22:34:38.062523+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.062523+0000","objects_scrubbed":27,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:14:19.882135+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":352,"num_read_kb":306,"num_write":116,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"312'386","reported_seq":1186,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.515887+0000","last_change":"2026-02-24T22:34:34.515887+0000","last_active":"2026-02-24T22:34:34.515887+0000","last_peered":"2026-02-24T22:34:34.515887+0000","last_clean":"2026-02-24T22:34:34.515887+0000","last_became_active":"2026-02-24T22:05:16.188658+0000","last_became_peered":"2026-02-24T22:05:16.188658+0000","last_unstale":"2026-02-24T22:34:34.515887+0000","last_undegraded":"2026-02-24T22:34:34.515887+0000","last_fullsized":"2026-02-24T22:34:34.515887+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'386","last_scrub_stamp":"2026-02-24T22:34:34.515812+0000","last_deep_scrub":"312'386","last_deep_scrub_stamp":"2026-02-24T22:34:34.515812+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.515812+0000","objects_scrubbed":6,"log_size":86,"log_dups_size":300,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:26:30.161641+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":840,"num_read_kb":685,"num_write":1819,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"312'198","reported_seq":929,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.065984+0000","last_change":"2026-02-24T22:34:40.065984+0000","last_active":"2026-02-24T22:34:40.065984+0000","last_peered":"2026-02-24T22:34:40.065984+0000","last_clean":"2026-02-24T22:34:40.065984+0000","last_became_active":"2026-02-24T22:05:18.187682+0000","last_became_peered":"2026-02-24T22:05:18.187682+0000","last_unstale":"2026-02-24T22:34:40.065984+0000","last_undegraded":"2026-02-24T22:34:40.065984+0000","last_fullsized":"2026-02-24T22:34:40.065984+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'198","last_scrub_stamp":"2026-02-24T22:34:40.065916+0000","last_deep_scrub":"312'198","last_deep_scrub_stamp":"2026-02-24T22:34:40.065916+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.065916+0000","objects_scrubbed":29,"log_size":98,"log_dups_size":100,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:30:25.873043+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":639,"num_read_kb":531,"num_write":250,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"312'296","reported_seq":1004,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.790665+0000","last_change":"2026-02-24T22:34:37.790665+0000","last_active":"2026-02-24T22:34:37.790665+0000","last_peered":"2026-02-24T22:34:37.790665+0000","last_clean":"2026-02-24T22:34:37.790665+0000","last_became_active":"2026-02-24T22:05:16.184209+0000","last_became_peered":"2026-02-24T22:05:16.184209+0000","last_unstale":"2026-02-24T22:34:37.790665+0000","last_undegraded":"2026-02-24T22:34:37.790665+0000","last_fullsized":"2026-02-24T22:34:37.790665+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'296","last_scrub_stamp":"2026-02-24T22:34:37.790602+0000","last_deep_scrub":"312'296","last_deep_scrub_stamp":"2026-02-24T22:34:37.790602+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.790602+0000","objects_scrubbed":9,"log_size":96,"log_dups_size":200,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:28:56.968188+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":416,"num_read_kb":603,"num_write":1232,"num_write_kb":10636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"312'131","reported_seq":818,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.372179+0000","last_change":"2026-02-24T22:34:49.372179+0000","last_active":"2026-02-24T22:34:49.372179+0000","last_peered":"2026-02-24T22:34:49.372179+0000","last_clean":"2026-02-24T22:34:49.372179+0000","last_became_active":"2026-02-24T22:05:18.193320+0000","last_became_peered":"2026-02-24T22:05:18.193320+0000","last_unstale":"2026-02-24T22:34:49.372179+0000","last_undegraded":"2026-02-24T22:34:49.372179+0000","last_fullsized":"2026-02-24T22:34:49.372179+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'131","last_scrub_stamp":"2026-02-24T22:34:49.372116+0000","last_deep_scrub":"312'131","last_deep_scrub_stamp":"2026-02-24T22:34:49.372116+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.372116+0000","objects_scrubbed":20,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:50:18.339691+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":425,"num_read_kb":355,"num_write":146,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"312'285","reported_seq":1006,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.747158+0000","last_change":"2026-02-24T22:34:37.747158+0000","last_active":"2026-02-24T22:34:37.747158+0000","last_peered":"2026-02-24T22:34:37.747158+0000","last_clean":"2026-02-24T22:34:37.747158+0000","last_became_active":"2026-02-24T22:05:16.181527+0000","last_became_peered":"2026-02-24T22:05:16.181527+0000","last_unstale":"2026-02-24T22:34:37.747158+0000","last_undegraded":"2026-02-24T22:34:37.747158+0000","last_fullsized":"2026-02-24T22:34:37.747158+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'285","last_scrub_stamp":"2026-02-24T22:34:37.747092+0000","last_deep_scrub":"312'285","last_deep_scrub_stamp":"2026-02-24T22:34:37.747092+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.747092+0000","objects_scrubbed":4,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:43:04.672418+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242895,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":474,"num_read_kb":5463,"num_write":1217,"num_write_kb":5189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":584,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:06.258262+0000","last_change":"2026-02-24T22:35:06.258262+0000","last_active":"2026-02-24T22:35:06.258262+0000","last_peered":"2026-02-24T22:35:06.258262+0000","last_clean":"2026-02-24T22:35:06.258262+0000","last_became_active":"2026-02-24T22:05:29.657764+0000","last_became_peered":"2026-02-24T22:05:29.657764+0000","last_unstale":"2026-02-24T22:35:06.258262+0000","last_undegraded":"2026-02-24T22:35:06.258262+0000","last_fullsized":"2026-02-24T22:35:06.258262+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:06.258189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:06.258189+0000","last_clean_scrub_stamp":"2026-02-24T22:35:06.258189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:18:02.619099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"312'107","reported_seq":770,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.757778+0000","last_change":"2026-02-24T22:34:42.757778+0000","last_active":"2026-02-24T22:34:42.757778+0000","last_peered":"2026-02-24T22:34:42.757778+0000","last_clean":"2026-02-24T22:34:42.757778+0000","last_became_active":"2026-02-24T22:05:18.184771+0000","last_became_peered":"2026-02-24T22:05:18.184771+0000","last_unstale":"2026-02-24T22:34:42.757778+0000","last_undegraded":"2026-02-24T22:34:42.757778+0000","last_fullsized":"2026-02-24T22:34:42.757778+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'107","last_scrub_stamp":"2026-02-24T22:34:42.757725+0000","last_deep_scrub":"312'107","last_deep_scrub_stamp":"2026-02-24T22:34:42.757725+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.757725+0000","objects_scrubbed":27,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:52:33.731233+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":246,"num_read_kb":208,"num_write":102,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"312'344","reported_seq":1133,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.109736+0000","last_change":"2026-02-24T22:34:37.109736+0000","last_active":"2026-02-24T22:34:37.109736+0000","last_peered":"2026-02-24T22:34:37.109736+0000","last_clean":"2026-02-24T22:34:37.109736+0000","last_became_active":"2026-02-24T22:05:16.191841+0000","last_became_peered":"2026-02-24T22:05:16.191841+0000","last_unstale":"2026-02-24T22:34:37.109736+0000","last_undegraded":"2026-02-24T22:34:37.109736+0000","last_fullsized":"2026-02-24T22:34:37.109736+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'344","last_scrub_stamp":"2026-02-24T22:34:37.109689+0000","last_deep_scrub":"312'344","last_deep_scrub_stamp":"2026-02-24T22:34:37.109689+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.109689+0000","objects_scrubbed":4,"log_size":44,"log_dups_size":300,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:35:53.603402+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":614,"num_read_kb":81372,"num_write":1329,"num_write_kb":20589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":587,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:50.867824+0000","last_change":"2026-02-24T22:34:50.867824+0000","last_active":"2026-02-24T22:34:50.867824+0000","last_peered":"2026-02-24T22:34:50.867824+0000","last_clean":"2026-02-24T22:34:50.867824+0000","last_became_active":"2026-02-24T22:05:20.194031+0000","last_became_peered":"2026-02-24T22:05:20.194031+0000","last_unstale":"2026-02-24T22:34:50.867824+0000","last_undegraded":"2026-02-24T22:34:50.867824+0000","last_fullsized":"2026-02-24T22:34:50.867824+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:50.867723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:50.867723+0000","last_clean_scrub_stamp":"2026-02-24T22:34:50.867723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:53:47.367923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"312'262","reported_seq":910,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:53.926384+0000","last_change":"2026-02-24T22:34:53.926384+0000","last_active":"2026-02-24T22:34:53.926384+0000","last_peered":"2026-02-24T22:34:53.926384+0000","last_clean":"2026-02-24T22:34:53.926384+0000","last_became_active":"2026-02-24T22:05:22.191928+0000","last_became_peered":"2026-02-24T22:05:22.191928+0000","last_unstale":"2026-02-24T22:34:53.926384+0000","last_undegraded":"2026-02-24T22:34:53.926384+0000","last_fullsized":"2026-02-24T22:34:53.926384+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'262","last_scrub_stamp":"2026-02-24T22:34:53.926319+0000","last_deep_scrub":"312'262","last_deep_scrub_stamp":"2026-02-24T22:34:53.926319+0000","last_clean_scrub_stamp":"2026-02-24T22:34:53.926319+0000","objects_scrubbed":4,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:38:02.939594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44919,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":413,"num_read_kb":914,"num_write":358,"num_write_kb":269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"312'258","reported_seq":861,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:02.251182+0000","last_change":"2026-02-24T22:35:02.251182+0000","last_active":"2026-02-24T22:35:02.251182+0000","last_peered":"2026-02-24T22:35:02.251182+0000","last_clean":"2026-02-24T22:35:02.251182+0000","last_became_active":"2026-02-24T22:05:24.204990+0000","last_became_peered":"2026-02-24T22:05:24.204990+0000","last_unstale":"2026-02-24T22:35:02.251182+0000","last_undegraded":"2026-02-24T22:35:02.251182+0000","last_fullsized":"2026-02-24T22:35:02.251182+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'258","last_scrub_stamp":"2026-02-24T22:35:02.251119+0000","last_deep_scrub":"312'258","last_deep_scrub_stamp":"2026-02-24T22:35:02.251119+0000","last_clean_scrub_stamp":"2026-02-24T22:35:02.251119+0000","objects_scrubbed":100,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:22:28.932537+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32373,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":92,"num_read_kb":87,"num_write":391,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":573,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:04.809580+0000","last_change":"2026-02-24T22:35:04.809580+0000","last_active":"2026-02-24T22:35:04.809580+0000","last_peered":"2026-02-24T22:35:04.809580+0000","last_clean":"2026-02-24T22:35:04.809580+0000","last_became_active":"2026-02-24T22:05:29.661747+0000","last_became_peered":"2026-02-24T22:05:29.661747+0000","last_unstale":"2026-02-24T22:35:04.809580+0000","last_undegraded":"2026-02-24T22:35:04.809580+0000","last_fullsized":"2026-02-24T22:35:04.809580+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:04.809519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:04.809519+0000","last_clean_scrub_stamp":"2026-02-24T22:35:04.809519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:29:02.534615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"312'104","reported_seq":784,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.994989+0000","last_change":"2026-02-24T22:34:42.994989+0000","last_active":"2026-02-24T22:34:42.994989+0000","last_peered":"2026-02-24T22:34:42.994989+0000","last_clean":"2026-02-24T22:34:42.994989+0000","last_became_active":"2026-02-24T22:05:18.191346+0000","last_became_peered":"2026-02-24T22:05:18.191346+0000","last_unstale":"2026-02-24T22:34:42.994989+0000","last_undegraded":"2026-02-24T22:34:42.994989+0000","last_fullsized":"2026-02-24T22:34:42.994989+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'104","last_scrub_stamp":"2026-02-24T22:34:42.994927+0000","last_deep_scrub":"312'104","last_deep_scrub_stamp":"2026-02-24T22:34:42.994927+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.994927+0000","objects_scrubbed":31,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:12:15.667131+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":237,"num_read_kb":209,"num_write":87,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"312'276","reported_seq":1004,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:30.666602+0000","last_change":"2026-02-24T22:34:30.666602+0000","last_active":"2026-02-24T22:34:30.666602+0000","last_peered":"2026-02-24T22:34:30.666602+0000","last_clean":"2026-02-24T22:34:30.666602+0000","last_became_active":"2026-02-24T22:05:16.183990+0000","last_became_peered":"2026-02-24T22:05:16.183990+0000","last_unstale":"2026-02-24T22:34:30.666602+0000","last_undegraded":"2026-02-24T22:34:30.666602+0000","last_fullsized":"2026-02-24T22:34:30.666602+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'276","last_scrub_stamp":"2026-02-24T22:34:30.666509+0000","last_deep_scrub":"312'276","last_deep_scrub_stamp":"2026-02-24T22:34:30.666509+0000","last_clean_scrub_stamp":"2026-02-24T22:34:30.666509+0000","objects_scrubbed":5,"log_size":76,"log_dups_size":200,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:23:41.986404+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485805,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":458,"num_read_kb":23693,"num_write":1270,"num_write_kb":23324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":598,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:51.749919+0000","last_change":"2026-02-24T22:34:51.749919+0000","last_active":"2026-02-24T22:34:51.749919+0000","last_peered":"2026-02-24T22:34:51.749919+0000","last_clean":"2026-02-24T22:34:51.749919+0000","last_became_active":"2026-02-24T22:05:20.186217+0000","last_became_peered":"2026-02-24T22:05:20.186217+0000","last_unstale":"2026-02-24T22:34:51.749919+0000","last_undegraded":"2026-02-24T22:34:51.749919+0000","last_fullsized":"2026-02-24T22:34:51.749919+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:51.749857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:51.749857+0000","last_clean_scrub_stamp":"2026-02-24T22:34:51.749857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:49:39.493746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"312'309","reported_seq":1164,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:55.272395+0000","last_change":"2026-02-24T22:34:55.272395+0000","last_active":"2026-02-24T22:34:55.272395+0000","last_peered":"2026-02-24T22:34:55.272395+0000","last_clean":"2026-02-24T22:34:55.272395+0000","last_became_active":"2026-02-24T22:05:22.196911+0000","last_became_peered":"2026-02-24T22:05:22.196911+0000","last_unstale":"2026-02-24T22:34:55.272395+0000","last_undegraded":"2026-02-24T22:34:55.272395+0000","last_fullsized":"2026-02-24T22:34:55.272395+0000","mapping_epoch":28,"log_start":"307'200","ondisk_log_start":"307'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'309","last_scrub_stamp":"2026-02-24T22:34:55.272282+0000","last_deep_scrub":"312'309","last_deep_scrub_stamp":"2026-02-24T22:34:55.272282+0000","last_clean_scrub_stamp":"2026-02-24T22:34:55.272282+0000","objects_scrubbed":4,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:53:27.073831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1016,"num_read_kb":540,"num_write":51,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"312'279","reported_seq":906,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:01.310102+0000","last_change":"2026-02-24T22:35:01.310102+0000","last_active":"2026-02-24T22:35:01.310102+0000","last_peered":"2026-02-24T22:35:01.310102+0000","last_clean":"2026-02-24T22:35:01.310102+0000","last_became_active":"2026-02-24T22:05:24.207692+0000","last_became_peered":"2026-02-24T22:05:24.207692+0000","last_unstale":"2026-02-24T22:35:01.310102+0000","last_undegraded":"2026-02-24T22:35:01.310102+0000","last_fullsized":"2026-02-24T22:35:01.310102+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'279","last_scrub_stamp":"2026-02-24T22:35:01.310078+0000","last_deep_scrub":"312'279","last_deep_scrub_stamp":"2026-02-24T22:35:01.310078+0000","last_clean_scrub_stamp":"2026-02-24T22:35:01.310078+0000","objects_scrubbed":92,"log_size":79,"log_dups_size":200,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:57:59.408543+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29978,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":173,"num_read_kb":171,"num_write":478,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":582,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:07.750498+0000","last_change":"2026-02-24T22:35:07.750436+0000","last_active":"2026-02-24T22:35:07.750498+0000","last_peered":"2026-02-24T22:35:07.750498+0000","last_clean":"2026-02-24T22:35:07.750498+0000","last_became_active":"2026-02-24T22:05:29.654288+0000","last_became_peered":"2026-02-24T22:05:29.654288+0000","last_unstale":"2026-02-24T22:35:07.750498+0000","last_undegraded":"2026-02-24T22:35:07.750498+0000","last_fullsized":"2026-02-24T22:35:07.750498+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:07.750356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:07.750356+0000","last_clean_scrub_stamp":"2026-02-24T22:35:07.750356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:03:20.092570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"312'298","reported_seq":1031,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.885878+0000","last_change":"2026-02-24T22:34:42.885878+0000","last_active":"2026-02-24T22:34:42.885878+0000","last_peered":"2026-02-24T22:34:42.885878+0000","last_clean":"2026-02-24T22:34:42.885878+0000","last_became_active":"2026-02-24T22:05:18.184542+0000","last_became_peered":"2026-02-24T22:05:18.184542+0000","last_unstale":"2026-02-24T22:34:42.885878+0000","last_undegraded":"2026-02-24T22:34:42.885878+0000","last_fullsized":"2026-02-24T22:34:42.885878+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'298","last_scrub_stamp":"2026-02-24T22:34:42.885832+0000","last_deep_scrub":"312'298","last_deep_scrub_stamp":"2026-02-24T22:34:42.885832+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.885832+0000","objects_scrubbed":24,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:07:57.036667+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1179,"num_read_kb":976,"num_write":460,"num_write_kb":243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"312'294","reported_seq":1016,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.140765+0000","last_change":"2026-02-24T22:34:35.140765+0000","last_active":"2026-02-24T22:34:35.140765+0000","last_peered":"2026-02-24T22:34:35.140765+0000","last_clean":"2026-02-24T22:34:35.140765+0000","last_became_active":"2026-02-24T22:05:16.190372+0000","last_became_peered":"2026-02-24T22:05:16.190372+0000","last_unstale":"2026-02-24T22:34:35.140765+0000","last_undegraded":"2026-02-24T22:34:35.140765+0000","last_fullsized":"2026-02-24T22:34:35.140765+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'294","last_scrub_stamp":"2026-02-24T22:34:35.140712+0000","last_deep_scrub":"312'294","last_deep_scrub_stamp":"2026-02-24T22:34:35.140712+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.140712+0000","objects_scrubbed":4,"log_size":94,"log_dups_size":200,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:24:43.777918+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":430,"num_read_kb":3398,"num_write":1175,"num_write_kb":524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":598,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:50.744612+0000","last_change":"2026-02-24T22:34:50.744612+0000","last_active":"2026-02-24T22:34:50.744612+0000","last_peered":"2026-02-24T22:34:50.744612+0000","last_clean":"2026-02-24T22:34:50.744612+0000","last_became_active":"2026-02-24T22:05:20.187327+0000","last_became_peered":"2026-02-24T22:05:20.187327+0000","last_unstale":"2026-02-24T22:34:50.744612+0000","last_undegraded":"2026-02-24T22:34:50.744612+0000","last_fullsized":"2026-02-24T22:34:50.744612+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:50.744552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:50.744552+0000","last_clean_scrub_stamp":"2026-02-24T22:34:50.744552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:05:37.833168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"312'802","reported_seq":2139,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.134134+0000","last_change":"2026-02-24T22:34:48.134134+0000","last_active":"2026-02-24T22:34:48.134134+0000","last_peered":"2026-02-24T22:34:48.134134+0000","last_clean":"2026-02-24T22:34:48.134134+0000","last_became_active":"2026-02-24T22:05:22.191073+0000","last_became_peered":"2026-02-24T22:05:22.191073+0000","last_unstale":"2026-02-24T22:34:48.134134+0000","last_undegraded":"2026-02-24T22:34:48.134134+0000","last_fullsized":"2026-02-24T22:34:48.134134+0000","mapping_epoch":28,"log_start":"307'700","ondisk_log_start":"307'700","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'802","last_scrub_stamp":"2026-02-24T22:34:48.134072+0000","last_deep_scrub":"312'802","last_deep_scrub_stamp":"2026-02-24T22:34:48.134072+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.134072+0000","objects_scrubbed":8,"log_size":102,"log_dups_size":700,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:20:11.846328+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":460,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2934,"num_read_kb":1505,"num_write":81,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"312'311","reported_seq":1838,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:00.230054+0000","last_change":"2026-02-24T22:35:00.230054+0000","last_active":"2026-02-24T22:35:00.230054+0000","last_peered":"2026-02-24T22:35:00.230054+0000","last_clean":"2026-02-24T22:35:00.230054+0000","last_became_active":"2026-02-24T22:05:24.206134+0000","last_became_peered":"2026-02-24T22:05:24.206134+0000","last_unstale":"2026-02-24T22:35:00.230054+0000","last_undegraded":"2026-02-24T22:35:00.230054+0000","last_fullsized":"2026-02-24T22:35:00.230054+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'311","last_scrub_stamp":"2026-02-24T22:35:00.229994+0000","last_deep_scrub":"312'311","last_deep_scrub_stamp":"2026-02-24T22:35:00.229994+0000","last_clean_scrub_stamp":"2026-02-24T22:35:00.229994+0000","objects_scrubbed":107,"log_size":11,"log_dups_size":300,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:46:37.708465+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33889,"num_objects":107,"num_object_clones":0,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":0,"num_read":1064,"num_read_kb":1063,"num_write":484,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":584,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:05.263336+0000","last_change":"2026-02-24T22:35:05.263294+0000","last_active":"2026-02-24T22:35:05.263336+0000","last_peered":"2026-02-24T22:35:05.263336+0000","last_clean":"2026-02-24T22:35:05.263336+0000","last_became_active":"2026-02-24T22:05:29.658119+0000","last_became_peered":"2026-02-24T22:05:29.658119+0000","last_unstale":"2026-02-24T22:35:05.263336+0000","last_undegraded":"2026-02-24T22:35:05.263336+0000","last_fullsized":"2026-02-24T22:35:05.263336+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:05.263228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:05.263228+0000","last_clean_scrub_stamp":"2026-02-24T22:35:05.263228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:33:28.280837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"312'690","reported_seq":1497,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.055164+0000","last_change":"2026-02-24T22:34:37.055105+0000","last_active":"2026-02-24T22:34:37.055164+0000","last_peered":"2026-02-24T22:34:37.055164+0000","last_clean":"2026-02-24T22:34:37.055164+0000","last_became_active":"2026-02-24T22:06:00.662609+0000","last_became_peered":"2026-02-24T22:06:00.662609+0000","last_unstale":"2026-02-24T22:34:37.055164+0000","last_undegraded":"2026-02-24T22:34:37.055164+0000","last_fullsized":"2026-02-24T22:34:37.055164+0000","mapping_epoch":36,"log_start":"312'600","ondisk_log_start":"312'600","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'690","last_scrub_stamp":"2026-02-24T22:34:37.055035+0000","last_deep_scrub":"312'690","last_deep_scrub_stamp":"2026-02-24T22:34:37.055035+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.055035+0000","objects_scrubbed":22,"log_size":90,"log_dups_size":600,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:44:46.603715+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":2886,"num_read_kb":2388,"num_write":1101,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"312'327","reported_seq":1061,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:30.811183+0000","last_change":"2026-02-24T22:34:30.811183+0000","last_active":"2026-02-24T22:34:30.811183+0000","last_peered":"2026-02-24T22:34:30.811183+0000","last_clean":"2026-02-24T22:34:30.811183+0000","last_became_active":"2026-02-24T22:05:16.189739+0000","last_became_peered":"2026-02-24T22:05:16.189739+0000","last_unstale":"2026-02-24T22:34:30.811183+0000","last_undegraded":"2026-02-24T22:34:30.811183+0000","last_fullsized":"2026-02-24T22:34:30.811183+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'327","last_scrub_stamp":"2026-02-24T22:34:30.811119+0000","last_deep_scrub":"312'327","last_deep_scrub_stamp":"2026-02-24T22:34:30.811119+0000","last_clean_scrub_stamp":"2026-02-24T22:34:30.811119+0000","objects_scrubbed":9,"log_size":27,"log_dups_size":300,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:44:34.182914+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485786,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":468,"num_read_kb":16928,"num_write":1377,"num_write_kb":17766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":587,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:51.907157+0000","last_change":"2026-02-24T22:34:51.907157+0000","last_active":"2026-02-24T22:34:51.907157+0000","last_peered":"2026-02-24T22:34:51.907157+0000","last_clean":"2026-02-24T22:34:51.907157+0000","last_became_active":"2026-02-24T22:05:20.193200+0000","last_became_peered":"2026-02-24T22:05:20.193200+0000","last_unstale":"2026-02-24T22:34:51.907157+0000","last_undegraded":"2026-02-24T22:34:51.907157+0000","last_fullsized":"2026-02-24T22:34:51.907157+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:51.907045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:51.907045+0000","last_clean_scrub_stamp":"2026-02-24T22:34:51.907045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:41:57.779249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"312'196","reported_seq":807,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:57.208914+0000","last_change":"2026-02-24T22:34:57.208914+0000","last_active":"2026-02-24T22:34:57.208914+0000","last_peered":"2026-02-24T22:34:57.208914+0000","last_clean":"2026-02-24T22:34:57.208914+0000","last_became_active":"2026-02-24T22:05:22.912826+0000","last_became_peered":"2026-02-24T22:05:22.912826+0000","last_unstale":"2026-02-24T22:34:57.208914+0000","last_undegraded":"2026-02-24T22:34:57.208914+0000","last_fullsized":"2026-02-24T22:34:57.208914+0000","mapping_epoch":28,"log_start":"312'100","ondisk_log_start":"312'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'196","last_scrub_stamp":"2026-02-24T22:34:57.208826+0000","last_deep_scrub":"312'196","last_deep_scrub_stamp":"2026-02-24T22:34:57.208826+0000","last_clean_scrub_stamp":"2026-02-24T22:34:57.208826+0000","objects_scrubbed":4,"log_size":96,"log_dups_size":100,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:42:57.794071+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37493,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":308,"num_read_kb":794,"num_write":284,"num_write_kb":257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"312'352","reported_seq":1012,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:06.646610+0000","last_change":"2026-02-24T22:35:06.646610+0000","last_active":"2026-02-24T22:35:06.646610+0000","last_peered":"2026-02-24T22:35:06.646610+0000","last_clean":"2026-02-24T22:35:06.646610+0000","last_became_active":"2026-02-24T22:05:24.198964+0000","last_became_peered":"2026-02-24T22:05:24.198964+0000","last_unstale":"2026-02-24T22:35:06.646610+0000","last_undegraded":"2026-02-24T22:35:06.646610+0000","last_fullsized":"2026-02-24T22:35:06.646610+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'352","last_scrub_stamp":"2026-02-24T22:35:06.646550+0000","last_deep_scrub":"312'352","last_deep_scrub_stamp":"2026-02-24T22:35:06.646550+0000","last_clean_scrub_stamp":"2026-02-24T22:35:06.646550+0000","objects_scrubbed":95,"log_size":52,"log_dups_size":300,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:44:35.894779+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29433,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":428,"num_read_kb":365,"num_write":612,"num_write_kb":187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"312'140","reported_seq":875,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.037417+0000","last_change":"2026-02-24T22:34:42.037417+0000","last_active":"2026-02-24T22:34:42.037417+0000","last_peered":"2026-02-24T22:34:42.037417+0000","last_clean":"2026-02-24T22:34:42.037417+0000","last_became_active":"2026-02-24T22:05:18.189796+0000","last_became_peered":"2026-02-24T22:05:18.189796+0000","last_unstale":"2026-02-24T22:34:42.037417+0000","last_undegraded":"2026-02-24T22:34:42.037417+0000","last_fullsized":"2026-02-24T22:34:42.037417+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'140","last_scrub_stamp":"2026-02-24T22:34:42.037385+0000","last_deep_scrub":"312'140","last_deep_scrub_stamp":"2026-02-24T22:34:42.037385+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.037385+0000","objects_scrubbed":27,"log_size":40,"log_dups_size":100,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:45:54.858292+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":455,"num_read_kb":399,"num_write":145,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"312'263","reported_seq":951,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.085973+0000","last_change":"2026-02-24T22:34:38.085935+0000","last_active":"2026-02-24T22:34:38.085973+0000","last_peered":"2026-02-24T22:34:38.085973+0000","last_clean":"2026-02-24T22:34:38.085973+0000","last_became_active":"2026-02-24T22:05:16.191985+0000","last_became_peered":"2026-02-24T22:05:16.191985+0000","last_unstale":"2026-02-24T22:34:38.085973+0000","last_undegraded":"2026-02-24T22:34:38.085973+0000","last_fullsized":"2026-02-24T22:34:38.085973+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'263","last_scrub_stamp":"2026-02-24T22:34:38.085876+0000","last_deep_scrub":"312'263","last_deep_scrub_stamp":"2026-02-24T22:34:38.085876+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.085876+0000","objects_scrubbed":7,"log_size":63,"log_dups_size":200,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:00:57.241743+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":82,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":355,"num_read_kb":4498,"num_write":1041,"num_write_kb":4311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":594,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:50.153717+0000","last_change":"2026-02-24T22:34:50.153717+0000","last_active":"2026-02-24T22:34:50.153717+0000","last_peered":"2026-02-24T22:34:50.153717+0000","last_clean":"2026-02-24T22:34:50.153717+0000","last_became_active":"2026-02-24T22:05:20.194288+0000","last_became_peered":"2026-02-24T22:05:20.194288+0000","last_unstale":"2026-02-24T22:34:50.153717+0000","last_undegraded":"2026-02-24T22:34:50.153717+0000","last_fullsized":"2026-02-24T22:34:50.153717+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:50.153652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:50.153652+0000","last_clean_scrub_stamp":"2026-02-24T22:34:50.153652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:34:10.280880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"312'269","reported_seq":906,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:55.965175+0000","last_change":"2026-02-24T22:34:55.965175+0000","last_active":"2026-02-24T22:34:55.965175+0000","last_peered":"2026-02-24T22:34:55.965175+0000","last_clean":"2026-02-24T22:34:55.965175+0000","last_became_active":"2026-02-24T22:05:22.912859+0000","last_became_peered":"2026-02-24T22:05:22.912859+0000","last_unstale":"2026-02-24T22:34:55.965175+0000","last_undegraded":"2026-02-24T22:34:55.965175+0000","last_fullsized":"2026-02-24T22:34:55.965175+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'269","last_scrub_stamp":"2026-02-24T22:34:55.965113+0000","last_deep_scrub":"312'269","last_deep_scrub_stamp":"2026-02-24T22:34:55.965113+0000","last_clean_scrub_stamp":"2026-02-24T22:34:55.965113+0000","objects_scrubbed":9,"log_size":69,"log_dups_size":200,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:00:50.874867+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38223,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":418,"num_read_kb":936,"num_write":362,"num_write_kb":271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"312'408","reported_seq":1118,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:52.254254+0000","last_change":"2026-02-24T22:34:52.254254+0000","last_active":"2026-02-24T22:34:52.254254+0000","last_peered":"2026-02-24T22:34:52.254254+0000","last_clean":"2026-02-24T22:34:52.254254+0000","last_became_active":"2026-02-24T22:05:24.199772+0000","last_became_peered":"2026-02-24T22:05:24.199772+0000","last_unstale":"2026-02-24T22:34:52.254254+0000","last_undegraded":"2026-02-24T22:34:52.254254+0000","last_fullsized":"2026-02-24T22:34:52.254254+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'408","last_scrub_stamp":"2026-02-24T22:34:52.254200+0000","last_deep_scrub":"312'408","last_deep_scrub_stamp":"2026-02-24T22:34:52.254200+0000","last_clean_scrub_stamp":"2026-02-24T22:34:52.254200+0000","objects_scrubbed":93,"log_size":108,"log_dups_size":300,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:02:43.014695+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30266,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":473,"num_read_kb":498,"num_write":701,"num_write_kb":232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1193,"num_omap_keys":7,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:02.856103+0000","last_change":"2026-02-24T22:35:02.856103+0000","last_active":"2026-02-24T22:35:02.856103+0000","last_peered":"2026-02-24T22:35:02.856103+0000","last_clean":"2026-02-24T22:35:02.856103+0000","last_became_active":"2026-02-24T22:05:29.661401+0000","last_became_peered":"2026-02-24T22:05:29.661401+0000","last_unstale":"2026-02-24T22:35:02.856103+0000","last_undegraded":"2026-02-24T22:35:02.856103+0000","last_fullsized":"2026-02-24T22:35:02.856103+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:02.856048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:02.856048+0000","last_clean_scrub_stamp":"2026-02-24T22:35:02.856048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:27:49.594219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"312'103","reported_seq":773,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.478042+0000","last_change":"2026-02-24T22:34:35.478042+0000","last_active":"2026-02-24T22:34:35.478042+0000","last_peered":"2026-02-24T22:34:35.478042+0000","last_clean":"2026-02-24T22:34:35.478042+0000","last_became_active":"2026-02-24T22:05:18.193188+0000","last_became_peered":"2026-02-24T22:05:18.193188+0000","last_unstale":"2026-02-24T22:34:35.478042+0000","last_undegraded":"2026-02-24T22:34:35.478042+0000","last_fullsized":"2026-02-24T22:34:35.478042+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'103","last_scrub_stamp":"2026-02-24T22:34:35.477982+0000","last_deep_scrub":"312'103","last_deep_scrub_stamp":"2026-02-24T22:34:35.477982+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.477982+0000","objects_scrubbed":19,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:46:51.105644+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":312,"num_read_kb":260,"num_write":126,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"312'324","reported_seq":1064,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.151701+0000","last_change":"2026-02-24T22:34:33.151665+0000","last_active":"2026-02-24T22:34:33.151701+0000","last_peered":"2026-02-24T22:34:33.151701+0000","last_clean":"2026-02-24T22:34:33.151701+0000","last_became_active":"2026-02-24T22:05:16.191956+0000","last_became_peered":"2026-02-24T22:05:16.191956+0000","last_unstale":"2026-02-24T22:34:33.151701+0000","last_undegraded":"2026-02-24T22:34:33.151701+0000","last_fullsized":"2026-02-24T22:34:33.151701+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'324","last_scrub_stamp":"2026-02-24T22:34:33.151603+0000","last_deep_scrub":"312'324","last_deep_scrub_stamp":"2026-02-24T22:34:33.151603+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.151603+0000","objects_scrubbed":3,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:52:36.974066+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485778,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":523,"num_read_kb":10657,"num_write":1420,"num_write_kb":15460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":602,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.120807+0000","last_change":"2026-02-24T22:34:46.120807+0000","last_active":"2026-02-24T22:34:46.120807+0000","last_peered":"2026-02-24T22:34:46.120807+0000","last_clean":"2026-02-24T22:34:46.120807+0000","last_became_active":"2026-02-24T22:05:20.186055+0000","last_became_peered":"2026-02-24T22:05:20.186055+0000","last_unstale":"2026-02-24T22:34:46.120807+0000","last_undegraded":"2026-02-24T22:34:46.120807+0000","last_fullsized":"2026-02-24T22:34:46.120807+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:46.120737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:46.120737+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.120737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:33:04.392007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"312'232","reported_seq":849,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:53.782364+0000","last_change":"2026-02-24T22:34:53.782364+0000","last_active":"2026-02-24T22:34:53.782364+0000","last_peered":"2026-02-24T22:34:53.782364+0000","last_clean":"2026-02-24T22:34:53.782364+0000","last_became_active":"2026-02-24T22:05:22.193741+0000","last_became_peered":"2026-02-24T22:05:22.193741+0000","last_unstale":"2026-02-24T22:34:53.782364+0000","last_undegraded":"2026-02-24T22:34:53.782364+0000","last_fullsized":"2026-02-24T22:34:53.782364+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'232","last_scrub_stamp":"2026-02-24T22:34:53.782300+0000","last_deep_scrub":"312'232","last_deep_scrub_stamp":"2026-02-24T22:34:53.782300+0000","last_clean_scrub_stamp":"2026-02-24T22:34:53.782300+0000","objects_scrubbed":4,"log_size":32,"log_dups_size":200,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:22:35.430551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39811,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":385,"num_read_kb":1028,"num_write":360,"num_write_kb":328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"312'276","reported_seq":914,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:01.567396+0000","last_change":"2026-02-24T22:35:01.567396+0000","last_active":"2026-02-24T22:35:01.567396+0000","last_peered":"2026-02-24T22:35:01.567396+0000","last_clean":"2026-02-24T22:35:01.567396+0000","last_became_active":"2026-02-24T22:05:24.198731+0000","last_became_peered":"2026-02-24T22:05:24.198731+0000","last_unstale":"2026-02-24T22:35:01.567396+0000","last_undegraded":"2026-02-24T22:35:01.567396+0000","last_fullsized":"2026-02-24T22:35:01.567396+0000","mapping_epoch":30,"log_start":"312'199","ondisk_log_start":"312'199","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'276","last_scrub_stamp":"2026-02-24T22:35:01.567361+0000","last_deep_scrub":"312'276","last_deep_scrub_stamp":"2026-02-24T22:35:01.567361+0000","last_clean_scrub_stamp":"2026-02-24T22:35:01.567361+0000","objects_scrubbed":91,"log_size":77,"log_dups_size":199,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:34:24.939354+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30034,"num_objects":91,"num_object_clones":0,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":241,"num_read_kb":207,"num_write":470,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":579,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:52.546054+0000","last_change":"2026-02-24T22:34:52.546054+0000","last_active":"2026-02-24T22:34:52.546054+0000","last_peered":"2026-02-24T22:34:52.546054+0000","last_clean":"2026-02-24T22:34:52.546054+0000","last_became_active":"2026-02-24T22:05:29.661161+0000","last_became_peered":"2026-02-24T22:05:29.661161+0000","last_unstale":"2026-02-24T22:34:52.546054+0000","last_undegraded":"2026-02-24T22:34:52.546054+0000","last_fullsized":"2026-02-24T22:34:52.546054+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:52.545999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:52.545999+0000","last_clean_scrub_stamp":"2026-02-24T22:34:52.545999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:03:26.142316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"312'129","reported_seq":784,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.703983+0000","last_change":"2026-02-24T22:34:39.703983+0000","last_active":"2026-02-24T22:34:39.703983+0000","last_peered":"2026-02-24T22:34:39.703983+0000","last_clean":"2026-02-24T22:34:39.703983+0000","last_became_active":"2026-02-24T22:05:18.191555+0000","last_became_peered":"2026-02-24T22:05:18.191555+0000","last_unstale":"2026-02-24T22:34:39.703983+0000","last_undegraded":"2026-02-24T22:34:39.703983+0000","last_fullsized":"2026-02-24T22:34:39.703983+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'129","last_scrub_stamp":"2026-02-24T22:34:39.703930+0000","last_deep_scrub":"312'129","last_deep_scrub_stamp":"2026-02-24T22:34:39.703930+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.703930+0000","objects_scrubbed":23,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:20:21.850877+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":387,"num_read_kb":325,"num_write":162,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"312'289","reported_seq":1011,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.159670+0000","last_change":"2026-02-24T22:34:34.159624+0000","last_active":"2026-02-24T22:34:34.159670+0000","last_peered":"2026-02-24T22:34:34.159670+0000","last_clean":"2026-02-24T22:34:34.159670+0000","last_became_active":"2026-02-24T22:05:16.191733+0000","last_became_peered":"2026-02-24T22:05:16.191733+0000","last_unstale":"2026-02-24T22:34:34.159670+0000","last_undegraded":"2026-02-24T22:34:34.159670+0000","last_fullsized":"2026-02-24T22:34:34.159670+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'289","last_scrub_stamp":"2026-02-24T22:34:34.159566+0000","last_deep_scrub":"312'289","last_deep_scrub_stamp":"2026-02-24T22:34:34.159566+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.159566+0000","objects_scrubbed":4,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:18:06.498072+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242906,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":398,"num_read_kb":31022,"num_write":1049,"num_write_kb":10319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":602,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.063841+0000","last_change":"2026-02-24T22:34:44.063841+0000","last_active":"2026-02-24T22:34:44.063841+0000","last_peered":"2026-02-24T22:34:44.063841+0000","last_clean":"2026-02-24T22:34:44.063841+0000","last_became_active":"2026-02-24T22:05:20.186892+0000","last_became_peered":"2026-02-24T22:05:20.186892+0000","last_unstale":"2026-02-24T22:34:44.063841+0000","last_undegraded":"2026-02-24T22:34:44.063841+0000","last_fullsized":"2026-02-24T22:34:44.063841+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:44.063776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:44.063776+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.063776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:34:17.807478+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"312'245","reported_seq":866,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.174484+0000","last_change":"2026-02-24T22:34:58.174484+0000","last_active":"2026-02-24T22:34:58.174484+0000","last_peered":"2026-02-24T22:34:58.174484+0000","last_clean":"2026-02-24T22:34:58.174484+0000","last_became_active":"2026-02-24T22:05:22.200208+0000","last_became_peered":"2026-02-24T22:05:22.200208+0000","last_unstale":"2026-02-24T22:34:58.174484+0000","last_undegraded":"2026-02-24T22:34:58.174484+0000","last_fullsized":"2026-02-24T22:34:58.174484+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'245","last_scrub_stamp":"2026-02-24T22:34:58.174378+0000","last_deep_scrub":"312'245","last_deep_scrub_stamp":"2026-02-24T22:34:58.174378+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.174378+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":200,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:10:15.543051+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39435,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":390,"num_read_kb":991,"num_write":356,"num_write_kb":310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"312'378","reported_seq":1201,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.906602+0000","last_change":"2026-02-24T22:34:58.906602+0000","last_active":"2026-02-24T22:34:58.906602+0000","last_peered":"2026-02-24T22:34:58.906602+0000","last_clean":"2026-02-24T22:34:58.906602+0000","last_became_active":"2026-02-24T22:05:24.199525+0000","last_became_peered":"2026-02-24T22:05:24.199525+0000","last_unstale":"2026-02-24T22:34:58.906602+0000","last_undegraded":"2026-02-24T22:34:58.906602+0000","last_fullsized":"2026-02-24T22:34:58.906602+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'378","last_scrub_stamp":"2026-02-24T22:34:58.906557+0000","last_deep_scrub":"312'378","last_deep_scrub_stamp":"2026-02-24T22:34:58.906557+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.906557+0000","objects_scrubbed":103,"log_size":78,"log_dups_size":300,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:42:33.226800+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32009,"num_objects":103,"num_object_clones":0,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":506,"num_read_kb":490,"num_write":606,"num_write_kb":207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":583,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:03.724387+0000","last_change":"2026-02-24T22:35:03.724387+0000","last_active":"2026-02-24T22:35:03.724387+0000","last_peered":"2026-02-24T22:35:03.724387+0000","last_clean":"2026-02-24T22:35:03.724387+0000","last_became_active":"2026-02-24T22:05:29.652598+0000","last_became_peered":"2026-02-24T22:05:29.652598+0000","last_unstale":"2026-02-24T22:35:03.724387+0000","last_undegraded":"2026-02-24T22:35:03.724387+0000","last_fullsized":"2026-02-24T22:35:03.724387+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:03.724333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:03.724333+0000","last_clean_scrub_stamp":"2026-02-24T22:35:03.724333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:16:00.583629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"312'189","reported_seq":900,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.811102+0000","last_change":"2026-02-24T22:34:40.811102+0000","last_active":"2026-02-24T22:34:40.811102+0000","last_peered":"2026-02-24T22:34:40.811102+0000","last_clean":"2026-02-24T22:34:40.811102+0000","last_became_active":"2026-02-24T22:05:18.185152+0000","last_became_peered":"2026-02-24T22:05:18.185152+0000","last_unstale":"2026-02-24T22:34:40.811102+0000","last_undegraded":"2026-02-24T22:34:40.811102+0000","last_fullsized":"2026-02-24T22:34:40.811102+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'189","last_scrub_stamp":"2026-02-24T22:34:40.811068+0000","last_deep_scrub":"312'189","last_deep_scrub_stamp":"2026-02-24T22:34:40.811068+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.811068+0000","objects_scrubbed":20,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:24:59.774125+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":651,"num_read_kb":540,"num_write":239,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"312'323","reported_seq":1050,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.061642+0000","last_change":"2026-02-24T22:34:33.061574+0000","last_active":"2026-02-24T22:34:33.061642+0000","last_peered":"2026-02-24T22:34:33.061642+0000","last_clean":"2026-02-24T22:34:33.061642+0000","last_became_active":"2026-02-24T22:05:16.190205+0000","last_became_peered":"2026-02-24T22:05:16.190205+0000","last_unstale":"2026-02-24T22:34:33.061642+0000","last_undegraded":"2026-02-24T22:34:33.061642+0000","last_fullsized":"2026-02-24T22:34:33.061642+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'323","last_scrub_stamp":"2026-02-24T22:34:33.061497+0000","last_deep_scrub":"312'323","last_deep_scrub_stamp":"2026-02-24T22:34:33.061497+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.061497+0000","objects_scrubbed":1,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:54:15.633089+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":578,"num_read_kb":20088,"num_write":1296,"num_write_kb":14513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":602,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.105525+0000","last_change":"2026-02-24T22:34:45.105525+0000","last_active":"2026-02-24T22:34:45.105525+0000","last_peered":"2026-02-24T22:34:45.105525+0000","last_clean":"2026-02-24T22:34:45.105525+0000","last_became_active":"2026-02-24T22:05:20.186781+0000","last_became_peered":"2026-02-24T22:05:20.186781+0000","last_unstale":"2026-02-24T22:34:45.105525+0000","last_undegraded":"2026-02-24T22:34:45.105525+0000","last_fullsized":"2026-02-24T22:34:45.105525+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:45.105463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:45.105463+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.105463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:39:37.239923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"312'404","reported_seq":1032,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:51.933082+0000","last_change":"2026-02-24T22:34:51.933082+0000","last_active":"2026-02-24T22:34:51.933082+0000","last_peered":"2026-02-24T22:34:51.933082+0000","last_clean":"2026-02-24T22:34:51.933082+0000","last_became_active":"2026-02-24T22:05:22.192495+0000","last_became_peered":"2026-02-24T22:05:22.192495+0000","last_unstale":"2026-02-24T22:34:51.933082+0000","last_undegraded":"2026-02-24T22:34:51.933082+0000","last_fullsized":"2026-02-24T22:34:51.933082+0000","mapping_epoch":28,"log_start":"312'300","ondisk_log_start":"312'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'404","last_scrub_stamp":"2026-02-24T22:34:51.933036+0000","last_deep_scrub":"312'404","last_deep_scrub_stamp":"2026-02-24T22:34:51.933036+0000","last_clean_scrub_stamp":"2026-02-24T22:34:51.933036+0000","objects_scrubbed":5,"log_size":104,"log_dups_size":300,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:38:02.998627+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":81867,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":699,"num_read_kb":1918,"num_write":658,"num_write_kb":626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"312'264","reported_seq":930,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.182990+0000","last_change":"2026-02-24T22:34:49.182990+0000","last_active":"2026-02-24T22:34:49.182990+0000","last_peered":"2026-02-24T22:34:49.182990+0000","last_clean":"2026-02-24T22:34:49.182990+0000","last_became_active":"2026-02-24T22:05:24.198114+0000","last_became_peered":"2026-02-24T22:05:24.198114+0000","last_unstale":"2026-02-24T22:34:49.182990+0000","last_undegraded":"2026-02-24T22:34:49.182990+0000","last_fullsized":"2026-02-24T22:34:49.182990+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'264","last_scrub_stamp":"2026-02-24T22:34:49.182961+0000","last_deep_scrub":"312'264","last_deep_scrub_stamp":"2026-02-24T22:34:49.182961+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.182961+0000","objects_scrubbed":91,"log_size":64,"log_dups_size":200,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:07:14.157330+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29501,"num_objects":91,"num_object_clones":0,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":235,"num_read_kb":226,"num_write":429,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":583,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:57.325797+0000","last_change":"2026-02-24T22:34:57.325797+0000","last_active":"2026-02-24T22:34:57.325797+0000","last_peered":"2026-02-24T22:34:57.325797+0000","last_clean":"2026-02-24T22:34:57.325797+0000","last_became_active":"2026-02-24T22:05:29.652715+0000","last_became_peered":"2026-02-24T22:05:29.652715+0000","last_unstale":"2026-02-24T22:34:57.325797+0000","last_undegraded":"2026-02-24T22:34:57.325797+0000","last_fullsized":"2026-02-24T22:34:57.325797+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:57.325730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:57.325730+0000","last_clean_scrub_stamp":"2026-02-24T22:34:57.325730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:53:52.255388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"312'596","reported_seq":1465,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.442591+0000","last_change":"2026-02-24T22:34:37.442591+0000","last_active":"2026-02-24T22:34:37.442591+0000","last_peered":"2026-02-24T22:34:37.442591+0000","last_clean":"2026-02-24T22:34:37.442591+0000","last_became_active":"2026-02-24T22:05:18.192543+0000","last_became_peered":"2026-02-24T22:05:18.192543+0000","last_unstale":"2026-02-24T22:34:37.442591+0000","last_undegraded":"2026-02-24T22:34:37.442591+0000","last_fullsized":"2026-02-24T22:34:37.442591+0000","mapping_epoch":24,"log_start":"312'500","ondisk_log_start":"312'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'596","last_scrub_stamp":"2026-02-24T22:34:37.442512+0000","last_deep_scrub":"312'596","last_deep_scrub_stamp":"2026-02-24T22:34:37.442512+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.442512+0000","objects_scrubbed":25,"log_size":96,"log_dups_size":500,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:07:01.563341+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":2576,"num_read_kb":2171,"num_write":966,"num_write_kb":601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"312'260","reported_seq":943,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.858767+0000","last_change":"2026-02-24T22:34:32.858731+0000","last_active":"2026-02-24T22:34:32.858767+0000","last_peered":"2026-02-24T22:34:32.858767+0000","last_clean":"2026-02-24T22:34:32.858767+0000","last_became_active":"2026-02-24T22:05:16.182928+0000","last_became_peered":"2026-02-24T22:05:16.182928+0000","last_unstale":"2026-02-24T22:34:32.858767+0000","last_undegraded":"2026-02-24T22:34:32.858767+0000","last_fullsized":"2026-02-24T22:34:32.858767+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'260","last_scrub_stamp":"2026-02-24T22:34:32.858687+0000","last_deep_scrub":"312'260","last_deep_scrub_stamp":"2026-02-24T22:34:32.858687+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.858687+0000","objects_scrubbed":5,"log_size":60,"log_dups_size":200,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:47:38.099528+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":316,"num_read_kb":262,"num_write":1083,"num_write_kb":181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":602,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.083526+0000","last_change":"2026-02-24T22:34:43.083526+0000","last_active":"2026-02-24T22:34:43.083526+0000","last_peered":"2026-02-24T22:34:43.083526+0000","last_clean":"2026-02-24T22:34:43.083526+0000","last_became_active":"2026-02-24T22:05:20.186209+0000","last_became_peered":"2026-02-24T22:05:20.186209+0000","last_unstale":"2026-02-24T22:34:43.083526+0000","last_undegraded":"2026-02-24T22:34:43.083526+0000","last_fullsized":"2026-02-24T22:34:43.083526+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:43.083470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:43.083470+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.083470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:53:12.978030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"312'956","reported_seq":2285,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:53.254053+0000","last_change":"2026-02-24T22:34:53.254053+0000","last_active":"2026-02-24T22:34:53.254053+0000","last_peered":"2026-02-24T22:34:53.254053+0000","last_clean":"2026-02-24T22:34:53.254053+0000","last_became_active":"2026-02-24T22:05:22.200111+0000","last_became_peered":"2026-02-24T22:05:22.200111+0000","last_unstale":"2026-02-24T22:34:53.254053+0000","last_undegraded":"2026-02-24T22:34:53.254053+0000","last_fullsized":"2026-02-24T22:34:53.254053+0000","mapping_epoch":28,"log_start":"312'900","ondisk_log_start":"312'900","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'956","last_scrub_stamp":"2026-02-24T22:34:53.253998+0000","last_deep_scrub":"312'956","last_deep_scrub_stamp":"2026-02-24T22:34:53.253998+0000","last_clean_scrub_stamp":"2026-02-24T22:34:53.253998+0000","objects_scrubbed":9,"log_size":56,"log_dups_size":900,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:23:04.182706+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37978,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":3232,"num_read_kb":2382,"num_write":419,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15964,"num_omap_keys":38,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"312'284","reported_seq":891,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:59.904243+0000","last_change":"2026-02-24T22:34:59.904243+0000","last_active":"2026-02-24T22:34:59.904243+0000","last_peered":"2026-02-24T22:34:59.904243+0000","last_clean":"2026-02-24T22:34:59.904243+0000","last_became_active":"2026-02-24T22:05:24.200878+0000","last_became_peered":"2026-02-24T22:05:24.200878+0000","last_unstale":"2026-02-24T22:34:59.904243+0000","last_undegraded":"2026-02-24T22:34:59.904243+0000","last_fullsized":"2026-02-24T22:34:59.904243+0000","mapping_epoch":30,"log_start":"312'197","ondisk_log_start":"312'197","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'284","last_scrub_stamp":"2026-02-24T22:34:59.904208+0000","last_deep_scrub":"312'284","last_deep_scrub_stamp":"2026-02-24T22:34:59.904208+0000","last_clean_scrub_stamp":"2026-02-24T22:34:59.904208+0000","objects_scrubbed":101,"log_size":87,"log_dups_size":197,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:49:36.597249+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33021,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":110,"num_read_kb":109,"num_write":447,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"312'440","reported_seq":2209,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:03.546764+0000","last_change":"2026-02-24T22:35:03.546764+0000","last_active":"2026-02-24T22:35:03.546764+0000","last_peered":"2026-02-24T22:35:03.546764+0000","last_clean":"2026-02-24T22:35:03.546764+0000","last_became_active":"2026-02-24T22:05:29.654064+0000","last_became_peered":"2026-02-24T22:05:29.654064+0000","last_unstale":"2026-02-24T22:35:03.546764+0000","last_undegraded":"2026-02-24T22:35:03.546764+0000","last_fullsized":"2026-02-24T22:35:03.546764+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'440","last_scrub_stamp":"2026-02-24T22:35:03.546700+0000","last_deep_scrub":"312'440","last_deep_scrub_stamp":"2026-02-24T22:35:03.546700+0000","last_clean_scrub_stamp":"2026-02-24T22:35:03.546700+0000","objects_scrubbed":1,"log_size":40,"log_dups_size":400,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:16:29.192139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"312'149","reported_seq":841,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.467232+0000","last_change":"2026-02-24T22:34:38.467232+0000","last_active":"2026-02-24T22:34:38.467232+0000","last_peered":"2026-02-24T22:34:38.467232+0000","last_clean":"2026-02-24T22:34:38.467232+0000","last_became_active":"2026-02-24T22:05:18.191055+0000","last_became_peered":"2026-02-24T22:05:18.191055+0000","last_unstale":"2026-02-24T22:34:38.467232+0000","last_undegraded":"2026-02-24T22:34:38.467232+0000","last_fullsized":"2026-02-24T22:34:38.467232+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'149","last_scrub_stamp":"2026-02-24T22:34:38.467178+0000","last_deep_scrub":"312'149","last_deep_scrub_stamp":"2026-02-24T22:34:38.467178+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.467178+0000","objects_scrubbed":21,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:18:47.528959+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":486,"num_read_kb":398,"num_write":170,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"312'241","reported_seq":956,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.746504+0000","last_change":"2026-02-24T22:34:32.746504+0000","last_active":"2026-02-24T22:34:32.746504+0000","last_peered":"2026-02-24T22:34:32.746504+0000","last_clean":"2026-02-24T22:34:32.746504+0000","last_became_active":"2026-02-24T22:05:16.187261+0000","last_became_peered":"2026-02-24T22:05:16.187261+0000","last_unstale":"2026-02-24T22:34:32.746504+0000","last_undegraded":"2026-02-24T22:34:32.746504+0000","last_fullsized":"2026-02-24T22:34:32.746504+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'241","last_scrub_stamp":"2026-02-24T22:34:32.746460+0000","last_deep_scrub":"312'241","last_deep_scrub_stamp":"2026-02-24T22:34:32.746460+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.746460+0000","objects_scrubbed":1,"log_size":41,"log_dups_size":200,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:05:01.484992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":370,"num_read_kb":24764,"num_write":957,"num_write_kb":10562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":597,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.218231+0000","last_change":"2026-02-24T22:34:47.218231+0000","last_active":"2026-02-24T22:34:47.218231+0000","last_peered":"2026-02-24T22:34:47.218231+0000","last_clean":"2026-02-24T22:34:47.218231+0000","last_became_active":"2026-02-24T22:05:20.187488+0000","last_became_peered":"2026-02-24T22:05:20.187488+0000","last_unstale":"2026-02-24T22:34:47.218231+0000","last_undegraded":"2026-02-24T22:34:47.218231+0000","last_fullsized":"2026-02-24T22:34:47.218231+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:47.218164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:47.218164+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.218164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:23:53.051333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"312'475","reported_seq":1340,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.137231+0000","last_change":"2026-02-24T22:34:47.137231+0000","last_active":"2026-02-24T22:34:47.137231+0000","last_peered":"2026-02-24T22:34:47.137231+0000","last_clean":"2026-02-24T22:34:47.137231+0000","last_became_active":"2026-02-24T22:05:22.193142+0000","last_became_peered":"2026-02-24T22:05:22.193142+0000","last_unstale":"2026-02-24T22:34:47.137231+0000","last_undegraded":"2026-02-24T22:34:47.137231+0000","last_fullsized":"2026-02-24T22:34:47.137231+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'475","last_scrub_stamp":"2026-02-24T22:34:47.137162+0000","last_deep_scrub":"312'475","last_deep_scrub_stamp":"2026-02-24T22:34:47.137162+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.137162+0000","objects_scrubbed":8,"log_size":75,"log_dups_size":400,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:12:15.570963+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38313,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1308,"num_read_kb":1368,"num_write":335,"num_write_kb":281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"312'312","reported_seq":926,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:50.216629+0000","last_change":"2026-02-24T22:34:50.216629+0000","last_active":"2026-02-24T22:34:50.216629+0000","last_peered":"2026-02-24T22:34:50.216629+0000","last_clean":"2026-02-24T22:34:50.216629+0000","last_became_active":"2026-02-24T22:05:24.198669+0000","last_became_peered":"2026-02-24T22:05:24.198669+0000","last_unstale":"2026-02-24T22:34:50.216629+0000","last_undegraded":"2026-02-24T22:34:50.216629+0000","last_fullsized":"2026-02-24T22:34:50.216629+0000","mapping_epoch":30,"log_start":"312'296","ondisk_log_start":"312'296","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'312","last_scrub_stamp":"2026-02-24T22:34:50.216584+0000","last_deep_scrub":"312'312","last_deep_scrub_stamp":"2026-02-24T22:34:50.216584+0000","last_clean_scrub_stamp":"2026-02-24T22:34:50.216584+0000","objects_scrubbed":90,"log_size":16,"log_dups_size":296,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:29:12.268470+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29328,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":170,"num_read_kb":165,"num_write":551,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":584,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:56.292294+0000","last_change":"2026-02-24T22:34:56.292294+0000","last_active":"2026-02-24T22:34:56.292294+0000","last_peered":"2026-02-24T22:34:56.292294+0000","last_clean":"2026-02-24T22:34:56.292294+0000","last_became_active":"2026-02-24T22:05:29.653144+0000","last_became_peered":"2026-02-24T22:05:29.653144+0000","last_unstale":"2026-02-24T22:34:56.292294+0000","last_undegraded":"2026-02-24T22:34:56.292294+0000","last_fullsized":"2026-02-24T22:34:56.292294+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:56.292236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:56.292236+0000","last_clean_scrub_stamp":"2026-02-24T22:34:56.292236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:28:09.419838+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"312'194","reported_seq":908,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.515320+0000","last_change":"2026-02-24T22:34:39.515320+0000","last_active":"2026-02-24T22:34:39.515320+0000","last_peered":"2026-02-24T22:34:39.515320+0000","last_clean":"2026-02-24T22:34:39.515320+0000","last_became_active":"2026-02-24T22:05:18.190140+0000","last_became_peered":"2026-02-24T22:05:18.190140+0000","last_unstale":"2026-02-24T22:34:39.515320+0000","last_undegraded":"2026-02-24T22:34:39.515320+0000","last_fullsized":"2026-02-24T22:34:39.515320+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'194","last_scrub_stamp":"2026-02-24T22:34:39.515258+0000","last_deep_scrub":"312'194","last_deep_scrub_stamp":"2026-02-24T22:34:39.515258+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.515258+0000","objects_scrubbed":23,"log_size":94,"log_dups_size":100,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:49:24.504968+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":675,"num_read_kb":556,"num_write":238,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"312'303","reported_seq":1072,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.577662+0000","last_change":"2026-02-24T22:34:32.577662+0000","last_active":"2026-02-24T22:34:32.577662+0000","last_peered":"2026-02-24T22:34:32.577662+0000","last_clean":"2026-02-24T22:34:32.577662+0000","last_became_active":"2026-02-24T22:05:16.184944+0000","last_became_peered":"2026-02-24T22:05:16.184944+0000","last_unstale":"2026-02-24T22:34:32.577662+0000","last_undegraded":"2026-02-24T22:34:32.577662+0000","last_fullsized":"2026-02-24T22:34:32.577662+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'303","last_scrub_stamp":"2026-02-24T22:34:32.577606+0000","last_deep_scrub":"312'303","last_deep_scrub_stamp":"2026-02-24T22:34:32.577606+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.577606+0000","objects_scrubbed":3,"log_size":103,"log_dups_size":200,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:59:47.981464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":538,"num_read_kb":45888,"num_write":1354,"num_write_kb":20430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"312'4","reported_seq":1068,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:53.903762+0000","last_change":"2026-02-24T22:34:53.903762+0000","last_active":"2026-02-24T22:34:53.903762+0000","last_peered":"2026-02-24T22:34:53.903762+0000","last_clean":"2026-02-24T22:34:53.903762+0000","last_became_active":"2026-02-24T22:05:20.194145+0000","last_became_peered":"2026-02-24T22:05:20.194145+0000","last_unstale":"2026-02-24T22:34:53.903762+0000","last_undegraded":"2026-02-24T22:34:53.903762+0000","last_fullsized":"2026-02-24T22:34:53.903762+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'4","last_scrub_stamp":"2026-02-24T22:34:53.903710+0000","last_deep_scrub":"312'4","last_deep_scrub_stamp":"2026-02-24T22:34:53.903710+0000","last_clean_scrub_stamp":"2026-02-24T22:34:53.903710+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:03:04.394145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":476,"num_read_kb":476,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"312'61","reported_seq":671,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:54.742682+0000","last_change":"2026-02-24T22:34:54.742682+0000","last_active":"2026-02-24T22:34:54.742682+0000","last_peered":"2026-02-24T22:34:54.742682+0000","last_clean":"2026-02-24T22:34:54.742682+0000","last_became_active":"2026-02-24T22:05:22.905221+0000","last_became_peered":"2026-02-24T22:05:22.905221+0000","last_unstale":"2026-02-24T22:34:54.742682+0000","last_undegraded":"2026-02-24T22:34:54.742682+0000","last_fullsized":"2026-02-24T22:34:54.742682+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'61","last_scrub_stamp":"2026-02-24T22:34:54.742603+0000","last_deep_scrub":"312'61","last_deep_scrub_stamp":"2026-02-24T22:34:54.742603+0000","last_clean_scrub_stamp":"2026-02-24T22:34:54.742603+0000","objects_scrubbed":5,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:52:44.639141+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"312'246","reported_seq":845,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.685022+0000","last_change":"2026-02-24T22:34:46.685022+0000","last_active":"2026-02-24T22:34:46.685022+0000","last_peered":"2026-02-24T22:34:46.685022+0000","last_clean":"2026-02-24T22:34:46.685022+0000","last_became_active":"2026-02-24T22:05:24.207673+0000","last_became_peered":"2026-02-24T22:05:24.207673+0000","last_unstale":"2026-02-24T22:34:46.685022+0000","last_undegraded":"2026-02-24T22:34:46.685022+0000","last_fullsized":"2026-02-24T22:34:46.685022+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'246","last_scrub_stamp":"2026-02-24T22:34:46.684962+0000","last_deep_scrub":"312'246","last_deep_scrub_stamp":"2026-02-24T22:34:46.684962+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.684962+0000","objects_scrubbed":87,"log_size":46,"log_dups_size":200,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:17:30.101644+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25987,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":107,"num_read_kb":103,"num_write":388,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":586,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:03.269486+0000","last_change":"2026-02-24T22:35:03.269486+0000","last_active":"2026-02-24T22:35:03.269486+0000","last_peered":"2026-02-24T22:35:03.269486+0000","last_clean":"2026-02-24T22:35:03.269486+0000","last_became_active":"2026-02-24T22:05:29.658842+0000","last_became_peered":"2026-02-24T22:05:29.658842+0000","last_unstale":"2026-02-24T22:35:03.269486+0000","last_undegraded":"2026-02-24T22:35:03.269486+0000","last_fullsized":"2026-02-24T22:35:03.269486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:03.269430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:03.269430+0000","last_clean_scrub_stamp":"2026-02-24T22:35:03.269430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:37:15.587437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"312'122","reported_seq":812,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.133092+0000","last_change":"2026-02-24T22:34:44.133092+0000","last_active":"2026-02-24T22:34:44.133092+0000","last_peered":"2026-02-24T22:34:44.133092+0000","last_clean":"2026-02-24T22:34:44.133092+0000","last_became_active":"2026-02-24T22:05:18.188161+0000","last_became_peered":"2026-02-24T22:05:18.188161+0000","last_unstale":"2026-02-24T22:34:44.133092+0000","last_undegraded":"2026-02-24T22:34:44.133092+0000","last_fullsized":"2026-02-24T22:34:44.133092+0000","mapping_epoch":24,"log_start":"312'98","ondisk_log_start":"312'98","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'122","last_scrub_stamp":"2026-02-24T22:34:44.133036+0000","last_deep_scrub":"312'122","last_deep_scrub_stamp":"2026-02-24T22:34:44.133036+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.133036+0000","objects_scrubbed":24,"log_size":24,"log_dups_size":98,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:15:18.743316+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":368,"num_read_kb":314,"num_write":134,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"312'360","reported_seq":1100,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.843071+0000","last_change":"2026-02-24T22:34:31.843071+0000","last_active":"2026-02-24T22:34:31.843071+0000","last_peered":"2026-02-24T22:34:31.843071+0000","last_clean":"2026-02-24T22:34:31.843071+0000","last_became_active":"2026-02-24T22:05:16.182194+0000","last_became_peered":"2026-02-24T22:05:16.182194+0000","last_unstale":"2026-02-24T22:34:31.843071+0000","last_undegraded":"2026-02-24T22:34:31.843071+0000","last_fullsized":"2026-02-24T22:34:31.843071+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'360","last_scrub_stamp":"2026-02-24T22:34:31.843001+0000","last_deep_scrub":"312'360","last_deep_scrub_stamp":"2026-02-24T22:34:31.843001+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.843001+0000","objects_scrubbed":4,"log_size":60,"log_dups_size":300,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:54:58.411257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":482,"num_read_kb":7457,"num_write":1288,"num_write_kb":8933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":599,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:52.297621+0000","last_change":"2026-02-24T22:34:52.297621+0000","last_active":"2026-02-24T22:34:52.297621+0000","last_peered":"2026-02-24T22:34:52.297621+0000","last_clean":"2026-02-24T22:34:52.297621+0000","last_became_active":"2026-02-24T22:05:20.193180+0000","last_became_peered":"2026-02-24T22:05:20.193180+0000","last_unstale":"2026-02-24T22:34:52.297621+0000","last_undegraded":"2026-02-24T22:34:52.297621+0000","last_fullsized":"2026-02-24T22:34:52.297621+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:52.297555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:52.297555+0000","last_clean_scrub_stamp":"2026-02-24T22:34:52.297555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:59:44.363996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"312'284","reported_seq":918,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:56.996699+0000","last_change":"2026-02-24T22:34:56.996699+0000","last_active":"2026-02-24T22:34:56.996699+0000","last_peered":"2026-02-24T22:34:56.996699+0000","last_clean":"2026-02-24T22:34:56.996699+0000","last_became_active":"2026-02-24T22:05:22.200138+0000","last_became_peered":"2026-02-24T22:05:22.200138+0000","last_unstale":"2026-02-24T22:34:56.996699+0000","last_undegraded":"2026-02-24T22:34:56.996699+0000","last_fullsized":"2026-02-24T22:34:56.996699+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'284","last_scrub_stamp":"2026-02-24T22:34:56.996649+0000","last_deep_scrub":"312'284","last_deep_scrub_stamp":"2026-02-24T22:34:56.996649+0000","last_clean_scrub_stamp":"2026-02-24T22:34:56.996649+0000","objects_scrubbed":9,"log_size":84,"log_dups_size":200,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:57:19.406987+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37516,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":871,"num_read_kb":1412,"num_write":1276,"num_write_kb":1198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":357372,"num_omap_keys":834,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"312'415","reported_seq":1053,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.648295+0000","last_change":"2026-02-24T22:34:47.648295+0000","last_active":"2026-02-24T22:34:47.648295+0000","last_peered":"2026-02-24T22:34:47.648295+0000","last_clean":"2026-02-24T22:34:47.648295+0000","last_became_active":"2026-02-24T22:05:24.206745+0000","last_became_peered":"2026-02-24T22:05:24.206745+0000","last_unstale":"2026-02-24T22:34:47.648295+0000","last_undegraded":"2026-02-24T22:34:47.648295+0000","last_fullsized":"2026-02-24T22:34:47.648295+0000","mapping_epoch":30,"log_start":"312'400","ondisk_log_start":"312'400","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'415","last_scrub_stamp":"2026-02-24T22:34:47.648223+0000","last_deep_scrub":"312'415","last_deep_scrub_stamp":"2026-02-24T22:34:47.648223+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.648223+0000","objects_scrubbed":88,"log_size":15,"log_dups_size":400,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:44:43.640744+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27098,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":391,"num_read_kb":376,"num_write":726,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":178,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"312'436","reported_seq":4809,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.362595+0000","last_change":"2026-02-24T22:34:58.362595+0000","last_active":"2026-02-24T22:34:58.362595+0000","last_peered":"2026-02-24T22:34:58.362595+0000","last_clean":"2026-02-24T22:34:58.362595+0000","last_became_active":"2026-02-24T22:05:29.654281+0000","last_became_peered":"2026-02-24T22:05:29.654281+0000","last_unstale":"2026-02-24T22:34:58.362595+0000","last_undegraded":"2026-02-24T22:34:58.362595+0000","last_fullsized":"2026-02-24T22:34:58.362595+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'436","last_scrub_stamp":"2026-02-24T22:34:58.362387+0000","last_deep_scrub":"312'436","last_deep_scrub_stamp":"2026-02-24T22:34:58.362387+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.362387+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":400,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:59:54.222522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"312'136","reported_seq":831,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.733879+0000","last_change":"2026-02-24T22:34:38.733879+0000","last_active":"2026-02-24T22:34:38.733879+0000","last_peered":"2026-02-24T22:34:38.733879+0000","last_clean":"2026-02-24T22:34:38.733879+0000","last_became_active":"2026-02-24T22:05:18.193426+0000","last_became_peered":"2026-02-24T22:05:18.193426+0000","last_unstale":"2026-02-24T22:34:38.733879+0000","last_undegraded":"2026-02-24T22:34:38.733879+0000","last_fullsized":"2026-02-24T22:34:38.733879+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'136","last_scrub_stamp":"2026-02-24T22:34:38.733817+0000","last_deep_scrub":"312'136","last_deep_scrub_stamp":"2026-02-24T22:34:38.733817+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.733817+0000","objects_scrubbed":22,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:36:52.810608+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":443,"num_read_kb":376,"num_write":199,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"312'114","reported_seq":756,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.176152+0000","last_change":"2026-02-24T22:34:32.176106+0000","last_active":"2026-02-24T22:34:32.176152+0000","last_peered":"2026-02-24T22:34:32.176152+0000","last_clean":"2026-02-24T22:34:32.176152+0000","last_became_active":"2026-02-24T22:05:06.708577+0000","last_became_peered":"2026-02-24T22:05:06.708577+0000","last_unstale":"2026-02-24T22:34:32.176152+0000","last_undegraded":"2026-02-24T22:34:32.176152+0000","last_fullsized":"2026-02-24T22:34:32.176152+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'114","last_scrub_stamp":"2026-02-24T22:34:32.176027+0000","last_deep_scrub":"312'114","last_deep_scrub_stamp":"2026-02-24T22:34:32.176027+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.176027+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:58:53.542668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"312'2","reported_seq":614,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.096943+0000","last_change":"2026-02-24T22:34:34.096943+0000","last_active":"2026-02-24T22:34:34.096943+0000","last_peered":"2026-02-24T22:34:34.096943+0000","last_clean":"2026-02-24T22:34:34.096943+0000","last_became_active":"2026-02-24T22:05:08.689983+0000","last_became_peered":"2026-02-24T22:05:08.689983+0000","last_unstale":"2026-02-24T22:34:34.096943+0000","last_undegraded":"2026-02-24T22:34:34.096943+0000","last_fullsized":"2026-02-24T22:34:34.096943+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:34:34.096879+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:34:34.096879+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.096879+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:03:29.099395+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00046251899999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"312'329","reported_seq":1116,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:29.895013+0000","last_change":"2026-02-24T22:34:29.895013+0000","last_active":"2026-02-24T22:34:29.895013+0000","last_peered":"2026-02-24T22:34:29.895013+0000","last_clean":"2026-02-24T22:34:29.895013+0000","last_became_active":"2026-02-24T22:05:16.185768+0000","last_became_peered":"2026-02-24T22:05:16.185768+0000","last_unstale":"2026-02-24T22:34:29.895013+0000","last_undegraded":"2026-02-24T22:34:29.895013+0000","last_fullsized":"2026-02-24T22:34:29.895013+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'329","last_scrub_stamp":"2026-02-24T22:34:29.894945+0000","last_deep_scrub":"312'329","last_deep_scrub_stamp":"2026-02-24T22:34:29.894945+0000","last_clean_scrub_stamp":"2026-02-24T22:34:29.894945+0000","objects_scrubbed":3,"log_size":29,"log_dups_size":300,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:02:39.272423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":685,"num_read_kb":12844,"num_write":1503,"num_write_kb":6232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":595,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.125003+0000","last_change":"2026-02-24T22:34:49.125003+0000","last_active":"2026-02-24T22:34:49.125003+0000","last_peered":"2026-02-24T22:34:49.125003+0000","last_clean":"2026-02-24T22:34:49.125003+0000","last_became_active":"2026-02-24T22:05:20.194040+0000","last_became_peered":"2026-02-24T22:05:20.194040+0000","last_unstale":"2026-02-24T22:34:49.125003+0000","last_undegraded":"2026-02-24T22:34:49.125003+0000","last_fullsized":"2026-02-24T22:34:49.125003+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:49.124939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:49.124939+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.124939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:16:47.321676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"312'1071","reported_seq":2188,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:54.936135+0000","last_change":"2026-02-24T22:34:54.936135+0000","last_active":"2026-02-24T22:34:54.936135+0000","last_peered":"2026-02-24T22:34:54.936135+0000","last_clean":"2026-02-24T22:34:54.936135+0000","last_became_active":"2026-02-24T22:05:22.199602+0000","last_became_peered":"2026-02-24T22:05:22.199602+0000","last_unstale":"2026-02-24T22:34:54.936135+0000","last_undegraded":"2026-02-24T22:34:54.936135+0000","last_fullsized":"2026-02-24T22:34:54.936135+0000","mapping_epoch":28,"log_start":"312'1000","ondisk_log_start":"312'1000","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'1071","last_scrub_stamp":"2026-02-24T22:34:54.936052+0000","last_deep_scrub":"312'1071","last_deep_scrub_stamp":"2026-02-24T22:34:54.936052+0000","last_clean_scrub_stamp":"2026-02-24T22:34:54.936052+0000","objects_scrubbed":8,"log_size":71,"log_dups_size":1000,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:42:38.805146+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":116028,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2941,"num_read_kb":3705,"num_write":1026,"num_write_kb":950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17602,"num_omap_keys":42,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"312'302","reported_seq":909,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:57.954594+0000","last_change":"2026-02-24T22:34:57.954594+0000","last_active":"2026-02-24T22:34:57.954594+0000","last_peered":"2026-02-24T22:34:57.954594+0000","last_clean":"2026-02-24T22:34:57.954594+0000","last_became_active":"2026-02-24T22:05:24.200336+0000","last_became_peered":"2026-02-24T22:05:24.200336+0000","last_unstale":"2026-02-24T22:34:57.954594+0000","last_undegraded":"2026-02-24T22:34:57.954594+0000","last_fullsized":"2026-02-24T22:34:57.954594+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'302","last_scrub_stamp":"2026-02-24T22:34:57.954521+0000","last_deep_scrub":"312'302","last_deep_scrub_stamp":"2026-02-24T22:34:57.954521+0000","last_clean_scrub_stamp":"2026-02-24T22:34:57.954521+0000","objects_scrubbed":100,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:53:46.560141+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32535,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":152,"num_read_kb":139,"num_write":501,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"312'428","reported_seq":1844,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:54.584117+0000","last_change":"2026-02-24T22:34:54.584117+0000","last_active":"2026-02-24T22:34:54.584117+0000","last_peered":"2026-02-24T22:34:54.584117+0000","last_clean":"2026-02-24T22:34:54.584117+0000","last_became_active":"2026-02-24T22:05:29.660370+0000","last_became_peered":"2026-02-24T22:05:29.660370+0000","last_unstale":"2026-02-24T22:34:54.584117+0000","last_undegraded":"2026-02-24T22:34:54.584117+0000","last_fullsized":"2026-02-24T22:34:54.584117+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'428","last_scrub_stamp":"2026-02-24T22:34:54.584009+0000","last_deep_scrub":"312'428","last_deep_scrub_stamp":"2026-02-24T22:34:54.584009+0000","last_clean_scrub_stamp":"2026-02-24T22:34:54.584009+0000","objects_scrubbed":1,"log_size":28,"log_dups_size":400,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:22:23.409209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"312'124","reported_seq":831,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.850193+0000","last_change":"2026-02-24T22:34:41.850193+0000","last_active":"2026-02-24T22:34:41.850193+0000","last_peered":"2026-02-24T22:34:41.850193+0000","last_clean":"2026-02-24T22:34:41.850193+0000","last_became_active":"2026-02-24T22:05:18.185205+0000","last_became_peered":"2026-02-24T22:05:18.185205+0000","last_unstale":"2026-02-24T22:34:41.850193+0000","last_undegraded":"2026-02-24T22:34:41.850193+0000","last_fullsized":"2026-02-24T22:34:41.850193+0000","mapping_epoch":24,"log_start":"312'99","ondisk_log_start":"312'99","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'124","last_scrub_stamp":"2026-02-24T22:34:41.850136+0000","last_deep_scrub":"312'124","last_deep_scrub_stamp":"2026-02-24T22:34:41.850136+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.850136+0000","objects_scrubbed":25,"log_size":25,"log_dups_size":99,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:53:33.276473+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":346,"num_read_kb":300,"num_write":140,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":619,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.150208+0000","last_change":"2026-02-24T22:34:36.150171+0000","last_active":"2026-02-24T22:34:36.150208+0000","last_peered":"2026-02-24T22:34:36.150208+0000","last_clean":"2026-02-24T22:34:36.150208+0000","last_became_active":"2026-02-24T22:05:08.689972+0000","last_became_peered":"2026-02-24T22:05:08.689972+0000","last_unstale":"2026-02-24T22:34:36.150208+0000","last_undegraded":"2026-02-24T22:34:36.150208+0000","last_fullsized":"2026-02-24T22:34:36.150208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:36.150114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:36.150114+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.150114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:06:01.715910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048345700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"312'262","reported_seq":1029,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:29.701991+0000","last_change":"2026-02-24T22:34:29.701952+0000","last_active":"2026-02-24T22:34:29.701991+0000","last_peered":"2026-02-24T22:34:29.701991+0000","last_clean":"2026-02-24T22:34:29.701991+0000","last_became_active":"2026-02-24T22:05:16.183484+0000","last_became_peered":"2026-02-24T22:05:16.183484+0000","last_unstale":"2026-02-24T22:34:29.701991+0000","last_undegraded":"2026-02-24T22:34:29.701991+0000","last_fullsized":"2026-02-24T22:34:29.701991+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'262","last_scrub_stamp":"2026-02-24T22:34:29.701911+0000","last_deep_scrub":"312'262","last_deep_scrub_stamp":"2026-02-24T22:34:29.701911+0000","last_clean_scrub_stamp":"2026-02-24T22:34:29.701911+0000","objects_scrubbed":4,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:16:54.158907+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":574,"num_read_kb":821,"num_write":1287,"num_write_kb":10723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":588,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.883509+0000","last_change":"2026-02-24T22:34:49.883509+0000","last_active":"2026-02-24T22:34:49.883509+0000","last_peered":"2026-02-24T22:34:49.883509+0000","last_clean":"2026-02-24T22:34:49.883509+0000","last_became_active":"2026-02-24T22:05:20.195236+0000","last_became_peered":"2026-02-24T22:05:20.195236+0000","last_unstale":"2026-02-24T22:34:49.883509+0000","last_undegraded":"2026-02-24T22:34:49.883509+0000","last_fullsized":"2026-02-24T22:34:49.883509+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:49.883446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:49.883446+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.883446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:59:34.535514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"312'1070","reported_seq":2301,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:51.232440+0000","last_change":"2026-02-24T22:34:51.232440+0000","last_active":"2026-02-24T22:34:51.232440+0000","last_peered":"2026-02-24T22:34:51.232440+0000","last_clean":"2026-02-24T22:34:51.232440+0000","last_became_active":"2026-02-24T22:05:22.192295+0000","last_became_peered":"2026-02-24T22:05:22.192295+0000","last_unstale":"2026-02-24T22:34:51.232440+0000","last_undegraded":"2026-02-24T22:34:51.232440+0000","last_fullsized":"2026-02-24T22:34:51.232440+0000","mapping_epoch":28,"log_start":"312'1001","ondisk_log_start":"312'1001","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'1070","last_scrub_stamp":"2026-02-24T22:34:51.232390+0000","last_deep_scrub":"312'1070","last_deep_scrub_stamp":"2026-02-24T22:34:51.232390+0000","last_clean_scrub_stamp":"2026-02-24T22:34:51.232390+0000","objects_scrubbed":12,"log_size":69,"log_dups_size":1001,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:26:43.381792+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":114655,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":3192,"num_read_kb":4794,"num_write":3958,"num_write_kb":3891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":517874,"num_omap_keys":3022,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"312'271","reported_seq":866,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:01.238162+0000","last_change":"2026-02-24T22:35:01.238162+0000","last_active":"2026-02-24T22:35:01.238162+0000","last_peered":"2026-02-24T22:35:01.238162+0000","last_clean":"2026-02-24T22:35:01.238162+0000","last_became_active":"2026-02-24T22:05:24.204774+0000","last_became_peered":"2026-02-24T22:05:24.204774+0000","last_unstale":"2026-02-24T22:35:01.238162+0000","last_undegraded":"2026-02-24T22:35:01.238162+0000","last_fullsized":"2026-02-24T22:35:01.238162+0000","mapping_epoch":30,"log_start":"312'196","ondisk_log_start":"312'196","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'271","last_scrub_stamp":"2026-02-24T22:35:01.238098+0000","last_deep_scrub":"312'271","last_deep_scrub_stamp":"2026-02-24T22:35:01.238098+0000","last_clean_scrub_stamp":"2026-02-24T22:35:01.238098+0000","objects_scrubbed":89,"log_size":75,"log_dups_size":196,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:08:24.544989+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29814,"num_objects":89,"num_object_clones":0,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":0,"num_read":112,"num_read_kb":108,"num_write":454,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":583,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:59.339684+0000","last_change":"2026-02-24T22:34:59.339684+0000","last_active":"2026-02-24T22:34:59.339684+0000","last_peered":"2026-02-24T22:34:59.339684+0000","last_clean":"2026-02-24T22:34:59.339684+0000","last_became_active":"2026-02-24T22:05:29.653391+0000","last_became_peered":"2026-02-24T22:05:29.653391+0000","last_unstale":"2026-02-24T22:34:59.339684+0000","last_undegraded":"2026-02-24T22:34:59.339684+0000","last_fullsized":"2026-02-24T22:34:59.339684+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:59.339612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:59.339612+0000","last_clean_scrub_stamp":"2026-02-24T22:34:59.339612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:54:10.672461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"312'130","reported_seq":829,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.122528+0000","last_change":"2026-02-24T22:34:42.122528+0000","last_active":"2026-02-24T22:34:42.122528+0000","last_peered":"2026-02-24T22:34:42.122528+0000","last_clean":"2026-02-24T22:34:42.122528+0000","last_became_active":"2026-02-24T22:05:18.186126+0000","last_became_peered":"2026-02-24T22:05:18.186126+0000","last_unstale":"2026-02-24T22:34:42.122528+0000","last_undegraded":"2026-02-24T22:34:42.122528+0000","last_fullsized":"2026-02-24T22:34:42.122528+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'130","last_scrub_stamp":"2026-02-24T22:34:42.122496+0000","last_deep_scrub":"312'130","last_deep_scrub_stamp":"2026-02-24T22:34:42.122496+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.122496+0000","objects_scrubbed":29,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:13:06.464986+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":336,"num_read_kb":289,"num_write":128,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":626,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:30.093328+0000","last_change":"2026-02-24T22:34:30.093328+0000","last_active":"2026-02-24T22:34:30.093328+0000","last_peered":"2026-02-24T22:34:30.093328+0000","last_clean":"2026-02-24T22:34:30.093328+0000","last_became_active":"2026-02-24T22:05:08.682035+0000","last_became_peered":"2026-02-24T22:05:08.682035+0000","last_unstale":"2026-02-24T22:34:30.093328+0000","last_undegraded":"2026-02-24T22:34:30.093328+0000","last_fullsized":"2026-02-24T22:34:30.093328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:30.093252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:30.093252+0000","last_clean_scrub_stamp":"2026-02-24T22:34:30.093252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:06:58.991217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047531599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"312'305","reported_seq":1058,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.010931+0000","last_change":"2026-02-24T22:34:34.010886+0000","last_active":"2026-02-24T22:34:34.010931+0000","last_peered":"2026-02-24T22:34:34.010931+0000","last_clean":"2026-02-24T22:34:34.010931+0000","last_became_active":"2026-02-24T22:05:16.178331+0000","last_became_peered":"2026-02-24T22:05:16.178331+0000","last_unstale":"2026-02-24T22:34:34.010931+0000","last_undegraded":"2026-02-24T22:34:34.010931+0000","last_fullsized":"2026-02-24T22:34:34.010931+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'305","last_scrub_stamp":"2026-02-24T22:34:34.010799+0000","last_deep_scrub":"312'305","last_deep_scrub_stamp":"2026-02-24T22:34:34.010799+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.010799+0000","objects_scrubbed":6,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:33:52.230516+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":583,"num_read_kb":6589,"num_write":1333,"num_write_kb":11354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"312'2","reported_seq":604,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.063471+0000","last_change":"2026-02-24T22:34:42.063471+0000","last_active":"2026-02-24T22:34:42.063471+0000","last_peered":"2026-02-24T22:34:42.063471+0000","last_clean":"2026-02-24T22:34:42.063471+0000","last_became_active":"2026-02-24T22:05:20.187908+0000","last_became_peered":"2026-02-24T22:05:20.187908+0000","last_unstale":"2026-02-24T22:34:42.063471+0000","last_undegraded":"2026-02-24T22:34:42.063471+0000","last_fullsized":"2026-02-24T22:34:42.063471+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:34:42.063424+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:34:42.063424+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.063424+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:16:34.733800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"312'447","reported_seq":1665,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:54.231930+0000","last_change":"2026-02-24T22:34:54.231930+0000","last_active":"2026-02-24T22:34:54.231930+0000","last_peered":"2026-02-24T22:34:54.231930+0000","last_clean":"2026-02-24T22:34:54.231930+0000","last_became_active":"2026-02-24T22:05:22.912414+0000","last_became_peered":"2026-02-24T22:05:22.912414+0000","last_unstale":"2026-02-24T22:34:54.231930+0000","last_undegraded":"2026-02-24T22:34:54.231930+0000","last_fullsized":"2026-02-24T22:34:54.231930+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'447","last_scrub_stamp":"2026-02-24T22:34:54.231891+0000","last_deep_scrub":"312'447","last_deep_scrub_stamp":"2026-02-24T22:34:54.231891+0000","last_clean_scrub_stamp":"2026-02-24T22:34:54.231891+0000","objects_scrubbed":10,"log_size":47,"log_dups_size":400,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:44:31.798158+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":308,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1080,"num_read_kb":599,"num_write":114,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5278,"num_omap_keys":12,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"312'281","reported_seq":882,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:56.954757+0000","last_change":"2026-02-24T22:34:56.954757+0000","last_active":"2026-02-24T22:34:56.954757+0000","last_peered":"2026-02-24T22:34:56.954757+0000","last_clean":"2026-02-24T22:34:56.954757+0000","last_became_active":"2026-02-24T22:05:24.200029+0000","last_became_peered":"2026-02-24T22:05:24.200029+0000","last_unstale":"2026-02-24T22:34:56.954757+0000","last_undegraded":"2026-02-24T22:34:56.954757+0000","last_fullsized":"2026-02-24T22:34:56.954757+0000","mapping_epoch":30,"log_start":"312'198","ondisk_log_start":"312'198","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'281","last_scrub_stamp":"2026-02-24T22:34:56.954689+0000","last_deep_scrub":"312'281","last_deep_scrub_stamp":"2026-02-24T22:34:56.954689+0000","last_clean_scrub_stamp":"2026-02-24T22:34:56.954689+0000","objects_scrubbed":97,"log_size":83,"log_dups_size":198,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:56:30.907099+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29918,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":442,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":585,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:04.581459+0000","last_change":"2026-02-24T22:35:04.581459+0000","last_active":"2026-02-24T22:35:04.581459+0000","last_peered":"2026-02-24T22:35:04.581459+0000","last_clean":"2026-02-24T22:35:04.581459+0000","last_became_active":"2026-02-24T22:05:29.653806+0000","last_became_peered":"2026-02-24T22:05:29.653806+0000","last_unstale":"2026-02-24T22:35:04.581459+0000","last_undegraded":"2026-02-24T22:35:04.581459+0000","last_fullsized":"2026-02-24T22:35:04.581459+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:04.581396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:04.581396+0000","last_clean_scrub_stamp":"2026-02-24T22:35:04.581396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:10:35.970116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"312'254","reported_seq":1004,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.068312+0000","last_change":"2026-02-24T22:34:38.068312+0000","last_active":"2026-02-24T22:34:38.068312+0000","last_peered":"2026-02-24T22:34:38.068312+0000","last_clean":"2026-02-24T22:34:38.068312+0000","last_became_active":"2026-02-24T22:05:18.183952+0000","last_became_peered":"2026-02-24T22:05:18.183952+0000","last_unstale":"2026-02-24T22:34:38.068312+0000","last_undegraded":"2026-02-24T22:34:38.068312+0000","last_fullsized":"2026-02-24T22:34:38.068312+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'254","last_scrub_stamp":"2026-02-24T22:34:38.068260+0000","last_deep_scrub":"312'254","last_deep_scrub_stamp":"2026-02-24T22:34:38.068260+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.068260+0000","objects_scrubbed":24,"log_size":54,"log_dups_size":200,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:24:48.806854+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":963,"num_read_kb":811,"num_write":374,"num_write_kb":210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"312'3","reported_seq":615,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.112722+0000","last_change":"2026-02-24T22:34:31.112722+0000","last_active":"2026-02-24T22:34:31.112722+0000","last_peered":"2026-02-24T22:34:31.112722+0000","last_clean":"2026-02-24T22:34:31.112722+0000","last_became_active":"2026-02-24T22:05:08.690042+0000","last_became_peered":"2026-02-24T22:05:08.690042+0000","last_unstale":"2026-02-24T22:34:31.112722+0000","last_undegraded":"2026-02-24T22:34:31.112722+0000","last_fullsized":"2026-02-24T22:34:31.112722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:34:31.112652+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:34:31.112652+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.112652+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:38:30.546771+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0010128400000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"312'333","reported_seq":1056,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:30.596222+0000","last_change":"2026-02-24T22:34:30.596222+0000","last_active":"2026-02-24T22:34:30.596222+0000","last_peered":"2026-02-24T22:34:30.596222+0000","last_clean":"2026-02-24T22:34:30.596222+0000","last_became_active":"2026-02-24T22:05:16.188900+0000","last_became_peered":"2026-02-24T22:05:16.188900+0000","last_unstale":"2026-02-24T22:34:30.596222+0000","last_undegraded":"2026-02-24T22:34:30.596222+0000","last_fullsized":"2026-02-24T22:34:30.596222+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'333","last_scrub_stamp":"2026-02-24T22:34:30.596162+0000","last_deep_scrub":"312'333","last_deep_scrub_stamp":"2026-02-24T22:34:30.596162+0000","last_clean_scrub_stamp":"2026-02-24T22:34:30.596162+0000","objects_scrubbed":6,"log_size":33,"log_dups_size":300,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:56:38.748514+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242895,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":415,"num_read_kb":10969,"num_write":1277,"num_write_kb":10859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":598,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.915846+0000","last_change":"2026-02-24T22:34:49.915846+0000","last_active":"2026-02-24T22:34:49.915846+0000","last_peered":"2026-02-24T22:34:49.915846+0000","last_clean":"2026-02-24T22:34:49.915846+0000","last_became_active":"2026-02-24T22:05:20.187251+0000","last_became_peered":"2026-02-24T22:05:20.187251+0000","last_unstale":"2026-02-24T22:34:49.915846+0000","last_undegraded":"2026-02-24T22:34:49.915846+0000","last_fullsized":"2026-02-24T22:34:49.915846+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:49.915785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:49.915785+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.915785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:30:18.871540+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"312'83","reported_seq":712,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:52.786106+0000","last_change":"2026-02-24T22:34:52.786106+0000","last_active":"2026-02-24T22:34:52.786106+0000","last_peered":"2026-02-24T22:34:52.786106+0000","last_clean":"2026-02-24T22:34:52.786106+0000","last_became_active":"2026-02-24T22:05:22.191394+0000","last_became_peered":"2026-02-24T22:05:22.191394+0000","last_unstale":"2026-02-24T22:34:52.786106+0000","last_undegraded":"2026-02-24T22:34:52.786106+0000","last_fullsized":"2026-02-24T22:34:52.786106+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'83","last_scrub_stamp":"2026-02-24T22:34:52.786051+0000","last_deep_scrub":"312'83","last_deep_scrub_stamp":"2026-02-24T22:34:52.786051+0000","last_clean_scrub_stamp":"2026-02-24T22:34:52.786051+0000","objects_scrubbed":7,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:23:33.675567+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":103,"num_read_kb":96,"num_write":72,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1630,"num_omap_keys":4,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"312'323","reported_seq":951,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.637677+0000","last_change":"2026-02-24T22:34:48.637677+0000","last_active":"2026-02-24T22:34:48.637677+0000","last_peered":"2026-02-24T22:34:48.637677+0000","last_clean":"2026-02-24T22:34:48.637677+0000","last_became_active":"2026-02-24T22:05:24.207698+0000","last_became_peered":"2026-02-24T22:05:24.207698+0000","last_unstale":"2026-02-24T22:34:48.637677+0000","last_undegraded":"2026-02-24T22:34:48.637677+0000","last_fullsized":"2026-02-24T22:34:48.637677+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'323","last_scrub_stamp":"2026-02-24T22:34:48.637620+0000","last_deep_scrub":"312'323","last_deep_scrub_stamp":"2026-02-24T22:34:48.637620+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.637620+0000","objects_scrubbed":94,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:13:22.580314+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31273,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":246,"num_read_kb":224,"num_write":553,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":574,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:00.855050+0000","last_change":"2026-02-24T22:35:00.855050+0000","last_active":"2026-02-24T22:35:00.855050+0000","last_peered":"2026-02-24T22:35:00.855050+0000","last_clean":"2026-02-24T22:35:00.855050+0000","last_became_active":"2026-02-24T22:05:29.660065+0000","last_became_peered":"2026-02-24T22:05:29.660065+0000","last_unstale":"2026-02-24T22:35:00.855050+0000","last_undegraded":"2026-02-24T22:35:00.855050+0000","last_fullsized":"2026-02-24T22:35:00.855050+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:00.855003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:00.855003+0000","last_clean_scrub_stamp":"2026-02-24T22:35:00.855003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:46:34.269073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"312'191","reported_seq":898,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.101612+0000","last_change":"2026-02-24T22:34:39.101612+0000","last_active":"2026-02-24T22:34:39.101612+0000","last_peered":"2026-02-24T22:34:39.101612+0000","last_clean":"2026-02-24T22:34:39.101612+0000","last_became_active":"2026-02-24T22:05:18.185112+0000","last_became_peered":"2026-02-24T22:05:18.185112+0000","last_unstale":"2026-02-24T22:34:39.101612+0000","last_undegraded":"2026-02-24T22:34:39.101612+0000","last_fullsized":"2026-02-24T22:34:39.101612+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'191","last_scrub_stamp":"2026-02-24T22:34:39.101566+0000","last_deep_scrub":"312'191","last_deep_scrub_stamp":"2026-02-24T22:34:39.101566+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.101566+0000","objects_scrubbed":19,"log_size":91,"log_dups_size":100,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:44:31.875433+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":679,"num_read_kb":553,"num_write":246,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"312'281","reported_seq":1011,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.103881+0000","last_change":"2026-02-24T22:34:32.103832+0000","last_active":"2026-02-24T22:34:32.103881+0000","last_peered":"2026-02-24T22:34:32.103881+0000","last_clean":"2026-02-24T22:34:32.103881+0000","last_became_active":"2026-02-24T22:05:16.191007+0000","last_became_peered":"2026-02-24T22:05:16.191007+0000","last_unstale":"2026-02-24T22:34:32.103881+0000","last_undegraded":"2026-02-24T22:34:32.103881+0000","last_fullsized":"2026-02-24T22:34:32.103881+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'281","last_scrub_stamp":"2026-02-24T22:34:32.103780+0000","last_deep_scrub":"312'281","last_deep_scrub_stamp":"2026-02-24T22:34:32.103780+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.103780+0000","objects_scrubbed":4,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:02:28.007749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":495,"num_read_kb":40330,"num_write":1338,"num_write_kb":18546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":621,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:29.991920+0000","last_change":"2026-02-24T22:34:29.991920+0000","last_active":"2026-02-24T22:34:29.991920+0000","last_peered":"2026-02-24T22:34:29.991920+0000","last_clean":"2026-02-24T22:34:29.991920+0000","last_became_active":"2026-02-24T22:05:08.758282+0000","last_became_peered":"2026-02-24T22:05:08.758282+0000","last_unstale":"2026-02-24T22:34:29.991920+0000","last_undegraded":"2026-02-24T22:34:29.991920+0000","last_fullsized":"2026-02-24T22:34:29.991920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:29.991857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:29.991857+0000","last_clean_scrub_stamp":"2026-02-24T22:34:29.991857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:47:43.334173+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00033355899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":598,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.948604+0000","last_change":"2026-02-24T22:34:48.948604+0000","last_active":"2026-02-24T22:34:48.948604+0000","last_peered":"2026-02-24T22:34:48.948604+0000","last_clean":"2026-02-24T22:34:48.948604+0000","last_became_active":"2026-02-24T22:05:20.187208+0000","last_became_peered":"2026-02-24T22:05:20.187208+0000","last_unstale":"2026-02-24T22:34:48.948604+0000","last_undegraded":"2026-02-24T22:34:48.948604+0000","last_fullsized":"2026-02-24T22:34:48.948604+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:48.948541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:48.948541+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.948541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:29:13.712233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"312'1143","reported_seq":2506,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:52.891975+0000","last_change":"2026-02-24T22:34:52.891935+0000","last_active":"2026-02-24T22:34:52.891975+0000","last_peered":"2026-02-24T22:34:52.891975+0000","last_clean":"2026-02-24T22:34:52.891975+0000","last_became_active":"2026-02-24T22:05:22.191763+0000","last_became_peered":"2026-02-24T22:05:22.191763+0000","last_unstale":"2026-02-24T22:34:52.891975+0000","last_undegraded":"2026-02-24T22:34:52.891975+0000","last_fullsized":"2026-02-24T22:34:52.891975+0000","mapping_epoch":28,"log_start":"312'1100","ondisk_log_start":"312'1100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'1143","last_scrub_stamp":"2026-02-24T22:34:52.891844+0000","last_deep_scrub":"312'1143","last_deep_scrub_stamp":"2026-02-24T22:34:52.891844+0000","last_clean_scrub_stamp":"2026-02-24T22:34:52.891844+0000","objects_scrubbed":12,"log_size":43,"log_dups_size":1100,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:24:36.081418+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":76731,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":3517,"num_read_kb":3087,"num_write":637,"num_write_kb":534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3608,"num_omap_keys":6,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"312'306","reported_seq":925,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:00.358718+0000","last_change":"2026-02-24T22:35:00.358718+0000","last_active":"2026-02-24T22:35:00.358718+0000","last_peered":"2026-02-24T22:35:00.358718+0000","last_clean":"2026-02-24T22:35:00.358718+0000","last_became_active":"2026-02-24T22:05:24.205923+0000","last_became_peered":"2026-02-24T22:05:24.205923+0000","last_unstale":"2026-02-24T22:35:00.358718+0000","last_undegraded":"2026-02-24T22:35:00.358718+0000","last_fullsized":"2026-02-24T22:35:00.358718+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'306","last_scrub_stamp":"2026-02-24T22:35:00.358668+0000","last_deep_scrub":"312'306","last_deep_scrub_stamp":"2026-02-24T22:35:00.358668+0000","last_clean_scrub_stamp":"2026-02-24T22:35:00.358668+0000","objects_scrubbed":102,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:31:13.435072+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34551,"num_objects":102,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":190,"num_read_kb":174,"num_write":516,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"312'468","reported_seq":4839,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:05.755883+0000","last_change":"2026-02-24T22:35:05.755883+0000","last_active":"2026-02-24T22:35:05.755883+0000","last_peered":"2026-02-24T22:35:05.755883+0000","last_clean":"2026-02-24T22:35:05.755883+0000","last_became_active":"2026-02-24T22:05:29.652259+0000","last_became_peered":"2026-02-24T22:05:29.652259+0000","last_unstale":"2026-02-24T22:35:05.755883+0000","last_undegraded":"2026-02-24T22:35:05.755883+0000","last_fullsized":"2026-02-24T22:35:05.755883+0000","mapping_epoch":33,"log_start":"312'401","ondisk_log_start":"312'401","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'468","last_scrub_stamp":"2026-02-24T22:35:05.755823+0000","last_deep_scrub":"312'468","last_deep_scrub_stamp":"2026-02-24T22:35:05.755823+0000","last_clean_scrub_stamp":"2026-02-24T22:35:05.755823+0000","objects_scrubbed":1,"log_size":67,"log_dups_size":401,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:50:47.482426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"312'136","reported_seq":842,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.775557+0000","last_change":"2026-02-24T22:34:39.775557+0000","last_active":"2026-02-24T22:34:39.775557+0000","last_peered":"2026-02-24T22:34:39.775557+0000","last_clean":"2026-02-24T22:34:39.775557+0000","last_became_active":"2026-02-24T22:05:18.183369+0000","last_became_peered":"2026-02-24T22:05:18.183369+0000","last_unstale":"2026-02-24T22:34:39.775557+0000","last_undegraded":"2026-02-24T22:34:39.775557+0000","last_fullsized":"2026-02-24T22:34:39.775557+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'136","last_scrub_stamp":"2026-02-24T22:34:39.775488+0000","last_deep_scrub":"312'136","last_deep_scrub_stamp":"2026-02-24T22:34:39.775488+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.775488+0000","objects_scrubbed":22,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:17:51.694013+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":458,"num_read_kb":386,"num_write":168,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"312'500","reported_seq":1319,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.014188+0000","last_change":"2026-02-24T22:34:31.014188+0000","last_active":"2026-02-24T22:34:31.014188+0000","last_peered":"2026-02-24T22:34:31.014188+0000","last_clean":"2026-02-24T22:34:31.014188+0000","last_became_active":"2026-02-24T22:05:16.178353+0000","last_became_peered":"2026-02-24T22:05:16.178353+0000","last_unstale":"2026-02-24T22:34:31.014188+0000","last_undegraded":"2026-02-24T22:34:31.014188+0000","last_fullsized":"2026-02-24T22:34:31.014188+0000","mapping_epoch":22,"log_start":"312'400","ondisk_log_start":"312'400","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'500","last_scrub_stamp":"2026-02-24T22:34:31.014143+0000","last_deep_scrub":"312'500","last_deep_scrub_stamp":"2026-02-24T22:34:31.014143+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.014143+0000","objects_scrubbed":3,"log_size":100,"log_dups_size":400,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:02:25.941904+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1036,"num_read_kb":9549,"num_write":2046,"num_write_kb":2098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":618,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.179872+0000","last_change":"2026-02-24T22:34:31.179872+0000","last_active":"2026-02-24T22:34:31.179872+0000","last_peered":"2026-02-24T22:34:31.179872+0000","last_clean":"2026-02-24T22:34:31.179872+0000","last_became_active":"2026-02-24T22:05:08.765129+0000","last_became_peered":"2026-02-24T22:05:08.765129+0000","last_unstale":"2026-02-24T22:34:31.179872+0000","last_undegraded":"2026-02-24T22:34:31.179872+0000","last_fullsized":"2026-02-24T22:34:31.179872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:31.179812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:31.179812+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.179812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:51:58.379988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439276,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":593,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.666471+0000","last_change":"2026-02-24T22:34:43.666471+0000","last_active":"2026-02-24T22:34:43.666471+0000","last_peered":"2026-02-24T22:34:43.666471+0000","last_clean":"2026-02-24T22:34:43.666471+0000","last_became_active":"2026-02-24T22:05:20.192811+0000","last_became_peered":"2026-02-24T22:05:20.192811+0000","last_unstale":"2026-02-24T22:34:43.666471+0000","last_undegraded":"2026-02-24T22:34:43.666471+0000","last_fullsized":"2026-02-24T22:34:43.666471+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:43.666412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:43.666412+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.666412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:08:50.485350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"312'619","reported_seq":1738,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:50.237395+0000","last_change":"2026-02-24T22:34:50.237395+0000","last_active":"2026-02-24T22:34:50.237395+0000","last_peered":"2026-02-24T22:34:50.237395+0000","last_clean":"2026-02-24T22:34:50.237395+0000","last_became_active":"2026-02-24T22:05:22.905663+0000","last_became_peered":"2026-02-24T22:05:22.905663+0000","last_unstale":"2026-02-24T22:34:50.237395+0000","last_undegraded":"2026-02-24T22:34:50.237395+0000","last_fullsized":"2026-02-24T22:34:50.237395+0000","mapping_epoch":28,"log_start":"312'600","ondisk_log_start":"312'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'619","last_scrub_stamp":"2026-02-24T22:34:50.237359+0000","last_deep_scrub":"312'619","last_deep_scrub_stamp":"2026-02-24T22:34:50.237359+0000","last_clean_scrub_stamp":"2026-02-24T22:34:50.237359+0000","objects_scrubbed":10,"log_size":19,"log_dups_size":600,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:03:58.542051+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":308,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":2064,"num_read_kb":1111,"num_write":154,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5320,"num_omap_keys":12,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"312'261","reported_seq":858,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.006402+0000","last_change":"2026-02-24T22:34:58.006364+0000","last_active":"2026-02-24T22:34:58.006402+0000","last_peered":"2026-02-24T22:34:58.006402+0000","last_clean":"2026-02-24T22:34:58.006402+0000","last_became_active":"2026-02-24T22:05:24.205847+0000","last_became_peered":"2026-02-24T22:05:24.205847+0000","last_unstale":"2026-02-24T22:34:58.006402+0000","last_undegraded":"2026-02-24T22:34:58.006402+0000","last_fullsized":"2026-02-24T22:34:58.006402+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'261","last_scrub_stamp":"2026-02-24T22:34:58.006321+0000","last_deep_scrub":"312'261","last_deep_scrub_stamp":"2026-02-24T22:34:58.006321+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.006321+0000","objects_scrubbed":90,"log_size":61,"log_dups_size":200,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:12:15.690665+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26185,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":121,"num_read_kb":112,"num_write":413,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":585,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:05.578691+0000","last_change":"2026-02-24T22:35:05.578691+0000","last_active":"2026-02-24T22:35:05.578691+0000","last_peered":"2026-02-24T22:35:05.578691+0000","last_clean":"2026-02-24T22:35:05.578691+0000","last_became_active":"2026-02-24T22:05:29.651401+0000","last_became_peered":"2026-02-24T22:05:29.651401+0000","last_unstale":"2026-02-24T22:35:05.578691+0000","last_undegraded":"2026-02-24T22:35:05.578691+0000","last_fullsized":"2026-02-24T22:35:05.578691+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:05.578627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:05.578627+0000","last_clean_scrub_stamp":"2026-02-24T22:35:05.578627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:05:23.173834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"312'175","reported_seq":882,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.748052+0000","last_change":"2026-02-24T22:34:40.748014+0000","last_active":"2026-02-24T22:34:40.748052+0000","last_peered":"2026-02-24T22:34:40.748052+0000","last_clean":"2026-02-24T22:34:40.748052+0000","last_became_active":"2026-02-24T22:05:18.185592+0000","last_became_peered":"2026-02-24T22:05:18.185592+0000","last_unstale":"2026-02-24T22:34:40.748052+0000","last_undegraded":"2026-02-24T22:34:40.748052+0000","last_fullsized":"2026-02-24T22:34:40.748052+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'175","last_scrub_stamp":"2026-02-24T22:34:40.747966+0000","last_deep_scrub":"312'175","last_deep_scrub_stamp":"2026-02-24T22:34:40.747966+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.747966+0000","objects_scrubbed":25,"log_size":75,"log_dups_size":100,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:49:27.517475+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":536,"num_read_kb":444,"num_write":207,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"312'316","reported_seq":1050,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:30.882380+0000","last_change":"2026-02-24T22:34:30.882380+0000","last_active":"2026-02-24T22:34:30.882380+0000","last_peered":"2026-02-24T22:34:30.882380+0000","last_clean":"2026-02-24T22:34:30.882380+0000","last_became_active":"2026-02-24T22:05:16.184131+0000","last_became_peered":"2026-02-24T22:05:16.184131+0000","last_unstale":"2026-02-24T22:34:30.882380+0000","last_undegraded":"2026-02-24T22:34:30.882380+0000","last_fullsized":"2026-02-24T22:34:30.882380+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'316","last_scrub_stamp":"2026-02-24T22:34:30.882351+0000","last_deep_scrub":"312'316","last_deep_scrub_stamp":"2026-02-24T22:34:30.882351+0000","last_clean_scrub_stamp":"2026-02-24T22:34:30.882351+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":300,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:29:29.669100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":471,"num_read_kb":1193,"num_write":1308,"num_write_kb":244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":621,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.039585+0000","last_change":"2026-02-24T22:34:32.039544+0000","last_active":"2026-02-24T22:34:32.039585+0000","last_peered":"2026-02-24T22:34:32.039585+0000","last_clean":"2026-02-24T22:34:32.039585+0000","last_became_active":"2026-02-24T22:05:08.682913+0000","last_became_peered":"2026-02-24T22:05:08.682913+0000","last_unstale":"2026-02-24T22:34:32.039585+0000","last_undegraded":"2026-02-24T22:34:32.039585+0000","last_fullsized":"2026-02-24T22:34:32.039585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:32.039480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:32.039480+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.039480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:39:23.840470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000359878,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":600,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:51.335400+0000","last_change":"2026-02-24T22:34:51.335400+0000","last_active":"2026-02-24T22:34:51.335400+0000","last_peered":"2026-02-24T22:34:51.335400+0000","last_clean":"2026-02-24T22:34:51.335400+0000","last_became_active":"2026-02-24T22:05:20.192162+0000","last_became_peered":"2026-02-24T22:05:20.192162+0000","last_unstale":"2026-02-24T22:34:51.335400+0000","last_undegraded":"2026-02-24T22:34:51.335400+0000","last_fullsized":"2026-02-24T22:34:51.335400+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:51.335327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:51.335327+0000","last_clean_scrub_stamp":"2026-02-24T22:34:51.335327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:49:38.610778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"312'1040","reported_seq":2612,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:56.289338+0000","last_change":"2026-02-24T22:34:56.289338+0000","last_active":"2026-02-24T22:34:56.289338+0000","last_peered":"2026-02-24T22:34:56.289338+0000","last_clean":"2026-02-24T22:34:56.289338+0000","last_became_active":"2026-02-24T22:05:22.198252+0000","last_became_peered":"2026-02-24T22:05:22.198252+0000","last_unstale":"2026-02-24T22:34:56.289338+0000","last_undegraded":"2026-02-24T22:34:56.289338+0000","last_fullsized":"2026-02-24T22:34:56.289338+0000","mapping_epoch":28,"log_start":"312'1000","ondisk_log_start":"312'1000","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'1040","last_scrub_stamp":"2026-02-24T22:34:56.289305+0000","last_deep_scrub":"312'1040","last_deep_scrub_stamp":"2026-02-24T22:34:56.289305+0000","last_clean_scrub_stamp":"2026-02-24T22:34:56.289305+0000","objects_scrubbed":9,"log_size":40,"log_dups_size":1000,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:35:30.102364+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":614,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":3875,"num_read_kb":1975,"num_write":82,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"312'1796","reported_seq":4427,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:00.610139+0000","last_change":"2026-02-24T22:35:00.610139+0000","last_active":"2026-02-24T22:35:00.610139+0000","last_peered":"2026-02-24T22:35:00.610139+0000","last_clean":"2026-02-24T22:35:00.610139+0000","last_became_active":"2026-02-24T22:05:24.198348+0000","last_became_peered":"2026-02-24T22:05:24.198348+0000","last_unstale":"2026-02-24T22:35:00.610139+0000","last_undegraded":"2026-02-24T22:35:00.610139+0000","last_fullsized":"2026-02-24T22:35:00.610139+0000","mapping_epoch":30,"log_start":"312'1698","ondisk_log_start":"312'1698","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'1796","last_scrub_stamp":"2026-02-24T22:35:00.610099+0000","last_deep_scrub":"312'1796","last_deep_scrub_stamp":"2026-02-24T22:35:00.610099+0000","last_clean_scrub_stamp":"2026-02-24T22:35:00.610099+0000","objects_scrubbed":98,"log_size":98,"log_dups_size":1698,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:58:01.687967+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31448,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":5151,"num_read_kb":231050,"num_write":3475,"num_write_kb":1655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":274185,"num_omap_keys":1500,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":583,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:04.708118+0000","last_change":"2026-02-24T22:35:04.708118+0000","last_active":"2026-02-24T22:35:04.708118+0000","last_peered":"2026-02-24T22:35:04.708118+0000","last_clean":"2026-02-24T22:35:04.708118+0000","last_became_active":"2026-02-24T22:05:29.652321+0000","last_became_peered":"2026-02-24T22:05:29.652321+0000","last_unstale":"2026-02-24T22:35:04.708118+0000","last_undegraded":"2026-02-24T22:35:04.708118+0000","last_fullsized":"2026-02-24T22:35:04.708118+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:04.708046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:04.708046+0000","last_clean_scrub_stamp":"2026-02-24T22:35:04.708046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:35:04.460228+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"312'72","reported_seq":745,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.449851+0000","last_change":"2026-02-24T22:34:36.449851+0000","last_active":"2026-02-24T22:34:36.449851+0000","last_peered":"2026-02-24T22:34:36.449851+0000","last_clean":"2026-02-24T22:34:36.449851+0000","last_became_active":"2026-02-24T22:05:18.191249+0000","last_became_peered":"2026-02-24T22:05:18.191249+0000","last_unstale":"2026-02-24T22:34:36.449851+0000","last_undegraded":"2026-02-24T22:34:36.449851+0000","last_fullsized":"2026-02-24T22:34:36.449851+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'72","last_scrub_stamp":"2026-02-24T22:34:36.449799+0000","last_deep_scrub":"312'72","last_deep_scrub_stamp":"2026-02-24T22:34:36.449799+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.449799+0000","objects_scrubbed":17,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:05:38.920525+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":226,"num_read_kb":230,"num_write":70,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2876,"num_omap_keys":12,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"312'274","reported_seq":1014,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.696342+0000","last_change":"2026-02-24T22:34:32.696342+0000","last_active":"2026-02-24T22:34:32.696342+0000","last_peered":"2026-02-24T22:34:32.696342+0000","last_clean":"2026-02-24T22:34:32.696342+0000","last_became_active":"2026-02-24T22:05:16.181840+0000","last_became_peered":"2026-02-24T22:05:16.181840+0000","last_unstale":"2026-02-24T22:34:32.696342+0000","last_undegraded":"2026-02-24T22:34:32.696342+0000","last_fullsized":"2026-02-24T22:34:32.696342+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'274","last_scrub_stamp":"2026-02-24T22:34:32.696287+0000","last_deep_scrub":"312'274","last_deep_scrub_stamp":"2026-02-24T22:34:32.696287+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.696287+0000","objects_scrubbed":4,"log_size":74,"log_dups_size":200,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:57:17.468566+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":486,"num_read_kb":7386,"num_write":1250,"num_write_kb":13063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":616,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.748692+0000","last_change":"2026-02-24T22:34:31.748652+0000","last_active":"2026-02-24T22:34:31.748692+0000","last_peered":"2026-02-24T22:34:31.748692+0000","last_clean":"2026-02-24T22:34:31.748692+0000","last_became_active":"2026-02-24T22:05:08.689542+0000","last_became_peered":"2026-02-24T22:05:08.689542+0000","last_unstale":"2026-02-24T22:34:31.748692+0000","last_undegraded":"2026-02-24T22:34:31.748692+0000","last_fullsized":"2026-02-24T22:34:31.748692+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:31.748592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:31.748592+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.748592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:30:37.278240+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00038816099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":601,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.813254+0000","last_change":"2026-02-24T22:34:47.813218+0000","last_active":"2026-02-24T22:34:47.813254+0000","last_peered":"2026-02-24T22:34:47.813254+0000","last_clean":"2026-02-24T22:34:47.813254+0000","last_became_active":"2026-02-24T22:05:20.186108+0000","last_became_peered":"2026-02-24T22:05:20.186108+0000","last_unstale":"2026-02-24T22:34:47.813254+0000","last_undegraded":"2026-02-24T22:34:47.813254+0000","last_fullsized":"2026-02-24T22:34:47.813254+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:47.813164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:47.813164+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.813164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:54:49.662332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"312'612","reported_seq":1273,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:51.192081+0000","last_change":"2026-02-24T22:34:51.192081+0000","last_active":"2026-02-24T22:34:51.192081+0000","last_peered":"2026-02-24T22:34:51.192081+0000","last_clean":"2026-02-24T22:34:51.192081+0000","last_became_active":"2026-02-24T22:05:22.911692+0000","last_became_peered":"2026-02-24T22:05:22.911692+0000","last_unstale":"2026-02-24T22:34:51.192081+0000","last_undegraded":"2026-02-24T22:34:51.192081+0000","last_fullsized":"2026-02-24T22:34:51.192081+0000","mapping_epoch":28,"log_start":"312'600","ondisk_log_start":"312'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'612","last_scrub_stamp":"2026-02-24T22:34:51.192010+0000","last_deep_scrub":"312'612","last_deep_scrub_stamp":"2026-02-24T22:34:51.192010+0000","last_clean_scrub_stamp":"2026-02-24T22:34:51.192010+0000","objects_scrubbed":8,"log_size":12,"log_dups_size":600,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:39:44.174934+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115263,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1049,"num_read_kb":2748,"num_write":972,"num_write_kb":882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"312'268","reported_seq":871,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:57.751800+0000","last_change":"2026-02-24T22:34:57.751800+0000","last_active":"2026-02-24T22:34:57.751800+0000","last_peered":"2026-02-24T22:34:57.751800+0000","last_clean":"2026-02-24T22:34:57.751800+0000","last_became_active":"2026-02-24T22:05:24.199398+0000","last_became_peered":"2026-02-24T22:05:24.199398+0000","last_unstale":"2026-02-24T22:34:57.751800+0000","last_undegraded":"2026-02-24T22:34:57.751800+0000","last_fullsized":"2026-02-24T22:34:57.751800+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'268","last_scrub_stamp":"2026-02-24T22:34:57.751746+0000","last_deep_scrub":"312'268","last_deep_scrub_stamp":"2026-02-24T22:34:57.751746+0000","last_clean_scrub_stamp":"2026-02-24T22:34:57.751746+0000","objects_scrubbed":93,"log_size":68,"log_dups_size":200,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:33:56.743054+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30139,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":95,"num_read_kb":93,"num_write":436,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"312'436","reported_seq":3603,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:01.857658+0000","last_change":"2026-02-24T22:35:01.857658+0000","last_active":"2026-02-24T22:35:01.857658+0000","last_peered":"2026-02-24T22:35:01.857658+0000","last_clean":"2026-02-24T22:35:01.857658+0000","last_became_active":"2026-02-24T22:05:29.661694+0000","last_became_peered":"2026-02-24T22:05:29.661694+0000","last_unstale":"2026-02-24T22:35:01.857658+0000","last_undegraded":"2026-02-24T22:35:01.857658+0000","last_fullsized":"2026-02-24T22:35:01.857658+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'436","last_scrub_stamp":"2026-02-24T22:35:01.857603+0000","last_deep_scrub":"312'436","last_deep_scrub_stamp":"2026-02-24T22:35:01.857603+0000","last_clean_scrub_stamp":"2026-02-24T22:35:01.857603+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":400,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:30:24.915658+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"312'462","reported_seq":2605,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:08.756144+0000","last_change":"2026-02-24T22:35:08.756144+0000","last_active":"2026-02-24T22:35:08.756144+0000","last_peered":"2026-02-24T22:35:08.756144+0000","last_clean":"2026-02-24T22:35:08.756144+0000","last_became_active":"2026-02-24T22:05:29.654058+0000","last_became_peered":"2026-02-24T22:05:29.654058+0000","last_unstale":"2026-02-24T22:35:08.756144+0000","last_undegraded":"2026-02-24T22:35:08.756144+0000","last_fullsized":"2026-02-24T22:35:08.756144+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'462","last_scrub_stamp":"2026-02-24T22:35:08.756067+0000","last_deep_scrub":"312'462","last_deep_scrub_stamp":"2026-02-24T22:35:08.756067+0000","last_clean_scrub_stamp":"2026-02-24T22:35:08.756067+0000","objects_scrubbed":1,"log_size":62,"log_dups_size":400,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:13:43.345009+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"312'4219","reported_seq":4960,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.517253+0000","last_change":"2026-02-24T22:34:41.517253+0000","last_active":"2026-02-24T22:34:41.517253+0000","last_peered":"2026-02-24T22:34:41.517253+0000","last_clean":"2026-02-24T22:34:41.517253+0000","last_became_active":"2026-02-24T22:05:18.190114+0000","last_became_peered":"2026-02-24T22:05:18.190114+0000","last_unstale":"2026-02-24T22:34:41.517253+0000","last_undegraded":"2026-02-24T22:34:41.517253+0000","last_fullsized":"2026-02-24T22:34:41.517253+0000","mapping_epoch":24,"log_start":"312'4200","ondisk_log_start":"312'4200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'4219","last_scrub_stamp":"2026-02-24T22:34:41.517218+0000","last_deep_scrub":"312'4219","last_deep_scrub_stamp":"2026-02-24T22:34:41.517218+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.517218+0000","objects_scrubbed":25,"log_size":19,"log_dups_size":3000,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:15:13.238846+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":16758,"num_read_kb":13963,"num_write":6320,"num_write_kb":3135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"312'256","reported_seq":985,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.598822+0000","last_change":"2026-02-24T22:34:31.598822+0000","last_active":"2026-02-24T22:34:31.598822+0000","last_peered":"2026-02-24T22:34:31.598822+0000","last_clean":"2026-02-24T22:34:31.598822+0000","last_became_active":"2026-02-24T22:05:16.188630+0000","last_became_peered":"2026-02-24T22:05:16.188630+0000","last_unstale":"2026-02-24T22:34:31.598822+0000","last_undegraded":"2026-02-24T22:34:31.598822+0000","last_fullsized":"2026-02-24T22:34:31.598822+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'256","last_scrub_stamp":"2026-02-24T22:34:31.598742+0000","last_deep_scrub":"312'256","last_deep_scrub_stamp":"2026-02-24T22:34:31.598742+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.598742+0000","objects_scrubbed":10,"log_size":56,"log_dups_size":200,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:25:41.229561+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291503,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":466,"num_read_kb":33751,"num_write":1067,"num_write_kb":13009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":596,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.198317+0000","last_change":"2026-02-24T22:34:46.198317+0000","last_active":"2026-02-24T22:34:46.198317+0000","last_peered":"2026-02-24T22:34:46.198317+0000","last_clean":"2026-02-24T22:34:46.198317+0000","last_became_active":"2026-02-24T22:05:20.188330+0000","last_became_peered":"2026-02-24T22:05:20.188330+0000","last_unstale":"2026-02-24T22:34:46.198317+0000","last_undegraded":"2026-02-24T22:34:46.198317+0000","last_fullsized":"2026-02-24T22:34:46.198317+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:46.198241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:46.198241+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.198241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:36:44.879872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"312'694","reported_seq":1785,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:55.697246+0000","last_change":"2026-02-24T22:34:55.697246+0000","last_active":"2026-02-24T22:34:55.697246+0000","last_peered":"2026-02-24T22:34:55.697246+0000","last_clean":"2026-02-24T22:34:55.697246+0000","last_became_active":"2026-02-24T22:05:22.191584+0000","last_became_peered":"2026-02-24T22:05:22.191584+0000","last_unstale":"2026-02-24T22:34:55.697246+0000","last_undegraded":"2026-02-24T22:34:55.697246+0000","last_fullsized":"2026-02-24T22:34:55.697246+0000","mapping_epoch":28,"log_start":"312'600","ondisk_log_start":"312'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'694","last_scrub_stamp":"2026-02-24T22:34:55.697186+0000","last_deep_scrub":"312'694","last_deep_scrub_stamp":"2026-02-24T22:34:55.697186+0000","last_clean_scrub_stamp":"2026-02-24T22:34:55.697186+0000","objects_scrubbed":7,"log_size":94,"log_dups_size":600,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:38:19.123444+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39069,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2230,"num_read_kb":1811,"num_write":322,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"312'255","reported_seq":876,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:02.574177+0000","last_change":"2026-02-24T22:35:02.574177+0000","last_active":"2026-02-24T22:35:02.574177+0000","last_peered":"2026-02-24T22:35:02.574177+0000","last_clean":"2026-02-24T22:35:02.574177+0000","last_became_active":"2026-02-24T22:05:24.198821+0000","last_became_peered":"2026-02-24T22:05:24.198821+0000","last_unstale":"2026-02-24T22:35:02.574177+0000","last_undegraded":"2026-02-24T22:35:02.574177+0000","last_fullsized":"2026-02-24T22:35:02.574177+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'255","last_scrub_stamp":"2026-02-24T22:35:02.574107+0000","last_deep_scrub":"312'255","last_deep_scrub_stamp":"2026-02-24T22:35:02.574107+0000","last_clean_scrub_stamp":"2026-02-24T22:35:02.574107+0000","objects_scrubbed":94,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:29:00.459611+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28978,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":162,"num_read_kb":145,"num_write":411,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":586,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:09.567710+0000","last_change":"2026-02-24T22:35:09.567710+0000","last_active":"2026-02-24T22:35:09.567710+0000","last_peered":"2026-02-24T22:35:09.567710+0000","last_clean":"2026-02-24T22:35:09.567710+0000","last_became_active":"2026-02-24T22:05:29.652757+0000","last_became_peered":"2026-02-24T22:05:29.652757+0000","last_unstale":"2026-02-24T22:35:09.567710+0000","last_undegraded":"2026-02-24T22:35:09.567710+0000","last_fullsized":"2026-02-24T22:35:09.567710+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:09.567606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:09.567606+0000","last_clean_scrub_stamp":"2026-02-24T22:35:09.567606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:05:13.901352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"312'4263","reported_seq":5020,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.115933+0000","last_change":"2026-02-24T22:34:40.115933+0000","last_active":"2026-02-24T22:34:40.115933+0000","last_peered":"2026-02-24T22:34:40.115933+0000","last_clean":"2026-02-24T22:34:40.115933+0000","last_became_active":"2026-02-24T22:05:18.183983+0000","last_became_peered":"2026-02-24T22:05:18.183983+0000","last_unstale":"2026-02-24T22:34:40.115933+0000","last_undegraded":"2026-02-24T22:34:40.115933+0000","last_fullsized":"2026-02-24T22:34:40.115933+0000","mapping_epoch":24,"log_start":"312'4200","ondisk_log_start":"312'4200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'4263","last_scrub_stamp":"2026-02-24T22:34:40.115891+0000","last_deep_scrub":"312'4263","last_deep_scrub_stamp":"2026-02-24T22:34:40.115891+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.115891+0000","objects_scrubbed":32,"log_size":63,"log_dups_size":3000,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:06:10.082774+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":16912,"num_read_kb":14102,"num_write":6331,"num_write_kb":3151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"312'314","reported_seq":1064,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.052571+0000","last_change":"2026-02-24T22:34:39.052571+0000","last_active":"2026-02-24T22:34:39.052571+0000","last_peered":"2026-02-24T22:34:39.052571+0000","last_clean":"2026-02-24T22:34:39.052571+0000","last_became_active":"2026-02-24T22:05:16.192845+0000","last_became_peered":"2026-02-24T22:05:16.192845+0000","last_unstale":"2026-02-24T22:34:39.052571+0000","last_undegraded":"2026-02-24T22:34:39.052571+0000","last_fullsized":"2026-02-24T22:34:39.052571+0000","mapping_epoch":22,"log_start":"312'301","ondisk_log_start":"312'301","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'314","last_scrub_stamp":"2026-02-24T22:34:39.052507+0000","last_deep_scrub":"312'314","last_deep_scrub_stamp":"2026-02-24T22:34:39.052507+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.052507+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":301,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:33:51.248695+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":490,"num_read_kb":4336,"num_write":1277,"num_write_kb":2676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":598,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.761265+0000","last_change":"2026-02-24T22:34:49.761265+0000","last_active":"2026-02-24T22:34:49.761265+0000","last_peered":"2026-02-24T22:34:49.761265+0000","last_clean":"2026-02-24T22:34:49.761265+0000","last_became_active":"2026-02-24T22:05:20.188790+0000","last_became_peered":"2026-02-24T22:05:20.188790+0000","last_unstale":"2026-02-24T22:34:49.761265+0000","last_undegraded":"2026-02-24T22:34:49.761265+0000","last_fullsized":"2026-02-24T22:34:49.761265+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:49.761185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:49.761185+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.761185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:28:03.018877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"312'28","reported_seq":626,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:54.950663+0000","last_change":"2026-02-24T22:34:54.950663+0000","last_active":"2026-02-24T22:34:54.950663+0000","last_peered":"2026-02-24T22:34:54.950663+0000","last_clean":"2026-02-24T22:34:54.950663+0000","last_became_active":"2026-02-24T22:05:22.192560+0000","last_became_peered":"2026-02-24T22:05:22.192560+0000","last_unstale":"2026-02-24T22:34:54.950663+0000","last_undegraded":"2026-02-24T22:34:54.950663+0000","last_fullsized":"2026-02-24T22:34:54.950663+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'28","last_scrub_stamp":"2026-02-24T22:34:54.950603+0000","last_deep_scrub":"312'28","last_deep_scrub_stamp":"2026-02-24T22:34:54.950603+0000","last_clean_scrub_stamp":"2026-02-24T22:34:54.950603+0000","objects_scrubbed":3,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:46:55.074264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":294,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"312'260","reported_seq":857,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:59.205020+0000","last_change":"2026-02-24T22:34:59.205020+0000","last_active":"2026-02-24T22:34:59.205020+0000","last_peered":"2026-02-24T22:34:59.205020+0000","last_clean":"2026-02-24T22:34:59.205020+0000","last_became_active":"2026-02-24T22:05:24.205506+0000","last_became_peered":"2026-02-24T22:05:24.205506+0000","last_unstale":"2026-02-24T22:34:59.205020+0000","last_undegraded":"2026-02-24T22:34:59.205020+0000","last_fullsized":"2026-02-24T22:34:59.205020+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'260","last_scrub_stamp":"2026-02-24T22:34:59.204958+0000","last_deep_scrub":"312'260","last_deep_scrub_stamp":"2026-02-24T22:34:59.204958+0000","last_clean_scrub_stamp":"2026-02-24T22:34:59.204958+0000","objects_scrubbed":97,"log_size":60,"log_dups_size":200,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:36:53.558006+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30843,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":90,"num_read_kb":84,"num_write":389,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":582,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:06.764346+0000","last_change":"2026-02-24T22:35:06.764299+0000","last_active":"2026-02-24T22:35:06.764346+0000","last_peered":"2026-02-24T22:35:06.764346+0000","last_clean":"2026-02-24T22:35:06.764346+0000","last_became_active":"2026-02-24T22:05:29.652987+0000","last_became_peered":"2026-02-24T22:05:29.652987+0000","last_unstale":"2026-02-24T22:35:06.764346+0000","last_undegraded":"2026-02-24T22:35:06.764346+0000","last_fullsized":"2026-02-24T22:35:06.764346+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:06.764242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:06.764242+0000","last_clean_scrub_stamp":"2026-02-24T22:35:06.764242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:47:21.058266+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"312'207","reported_seq":913,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.729074+0000","last_change":"2026-02-24T22:34:41.729074+0000","last_active":"2026-02-24T22:34:41.729074+0000","last_peered":"2026-02-24T22:34:41.729074+0000","last_clean":"2026-02-24T22:34:41.729074+0000","last_became_active":"2026-02-24T22:05:18.186069+0000","last_became_peered":"2026-02-24T22:05:18.186069+0000","last_unstale":"2026-02-24T22:34:41.729074+0000","last_undegraded":"2026-02-24T22:34:41.729074+0000","last_fullsized":"2026-02-24T22:34:41.729074+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'207","last_scrub_stamp":"2026-02-24T22:34:41.729005+0000","last_deep_scrub":"312'207","last_deep_scrub_stamp":"2026-02-24T22:34:41.729005+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.729005+0000","objects_scrubbed":20,"log_size":107,"log_dups_size":100,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:04:29.291410+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":691,"num_read_kb":561,"num_write":267,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"312'314","reported_seq":1005,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.108267+0000","last_change":"2026-02-24T22:34:31.108224+0000","last_active":"2026-02-24T22:34:31.108267+0000","last_peered":"2026-02-24T22:34:31.108267+0000","last_clean":"2026-02-24T22:34:31.108267+0000","last_became_active":"2026-02-24T22:06:00.662291+0000","last_became_peered":"2026-02-24T22:06:00.662291+0000","last_unstale":"2026-02-24T22:34:31.108267+0000","last_undegraded":"2026-02-24T22:34:31.108267+0000","last_fullsized":"2026-02-24T22:34:31.108267+0000","mapping_epoch":36,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'314","last_scrub_stamp":"2026-02-24T22:34:31.108159+0000","last_deep_scrub":"312'314","last_deep_scrub_stamp":"2026-02-24T22:34:31.108159+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.108159+0000","objects_scrubbed":5,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:32:30.179715+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":421,"num_read_kb":357,"num_write":1374,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":593,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:53.181869+0000","last_change":"2026-02-24T22:34:53.181869+0000","last_active":"2026-02-24T22:34:53.181869+0000","last_peered":"2026-02-24T22:34:53.181869+0000","last_clean":"2026-02-24T22:34:53.181869+0000","last_became_active":"2026-02-24T22:05:20.194215+0000","last_became_peered":"2026-02-24T22:05:20.194215+0000","last_unstale":"2026-02-24T22:34:53.181869+0000","last_undegraded":"2026-02-24T22:34:53.181869+0000","last_fullsized":"2026-02-24T22:34:53.181869+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:53.181785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:53.181785+0000","last_clean_scrub_stamp":"2026-02-24T22:34:53.181785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:35:22.996790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"312'482","reported_seq":1353,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.661029+0000","last_change":"2026-02-24T22:34:58.661029+0000","last_active":"2026-02-24T22:34:58.661029+0000","last_peered":"2026-02-24T22:34:58.661029+0000","last_clean":"2026-02-24T22:34:58.661029+0000","last_became_active":"2026-02-24T22:05:22.191675+0000","last_became_peered":"2026-02-24T22:05:22.191675+0000","last_unstale":"2026-02-24T22:34:58.661029+0000","last_undegraded":"2026-02-24T22:34:58.661029+0000","last_fullsized":"2026-02-24T22:34:58.661029+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'482","last_scrub_stamp":"2026-02-24T22:34:58.660967+0000","last_deep_scrub":"312'482","last_deep_scrub_stamp":"2026-02-24T22:34:58.660967+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.660967+0000","objects_scrubbed":6,"log_size":82,"log_dups_size":400,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:26:30.259061+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39139,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1329,"num_read_kb":1395,"num_write":345,"num_write_kb":280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"312'347","reported_seq":1390,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.631185+0000","last_change":"2026-02-24T22:34:49.631185+0000","last_active":"2026-02-24T22:34:49.631185+0000","last_peered":"2026-02-24T22:34:49.631185+0000","last_clean":"2026-02-24T22:34:49.631185+0000","last_became_active":"2026-02-24T22:05:24.207479+0000","last_became_peered":"2026-02-24T22:05:24.207479+0000","last_unstale":"2026-02-24T22:34:49.631185+0000","last_undegraded":"2026-02-24T22:34:49.631185+0000","last_fullsized":"2026-02-24T22:34:49.631185+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'347","last_scrub_stamp":"2026-02-24T22:34:49.631131+0000","last_deep_scrub":"312'347","last_deep_scrub_stamp":"2026-02-24T22:34:49.631131+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.631131+0000","objects_scrubbed":98,"log_size":47,"log_dups_size":300,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:42:33.404911+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31962,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":675,"num_read_kb":669,"num_write":683,"num_write_kb":207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":573,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:05.770303+0000","last_change":"2026-02-24T22:35:05.770253+0000","last_active":"2026-02-24T22:35:05.770303+0000","last_peered":"2026-02-24T22:35:05.770303+0000","last_clean":"2026-02-24T22:35:05.770303+0000","last_became_active":"2026-02-24T22:05:29.661248+0000","last_became_peered":"2026-02-24T22:05:29.661248+0000","last_unstale":"2026-02-24T22:35:05.770303+0000","last_undegraded":"2026-02-24T22:35:05.770303+0000","last_fullsized":"2026-02-24T22:35:05.770303+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:05.770187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:05.770187+0000","last_clean_scrub_stamp":"2026-02-24T22:35:05.770187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:50:08.978017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"312'142","reported_seq":860,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.756553+0000","last_change":"2026-02-24T22:34:39.756553+0000","last_active":"2026-02-24T22:34:39.756553+0000","last_peered":"2026-02-24T22:34:39.756553+0000","last_clean":"2026-02-24T22:34:39.756553+0000","last_became_active":"2026-02-24T22:05:18.184982+0000","last_became_peered":"2026-02-24T22:05:18.184982+0000","last_unstale":"2026-02-24T22:34:39.756553+0000","last_undegraded":"2026-02-24T22:34:39.756553+0000","last_fullsized":"2026-02-24T22:34:39.756553+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'142","last_scrub_stamp":"2026-02-24T22:34:39.756507+0000","last_deep_scrub":"312'142","last_deep_scrub_stamp":"2026-02-24T22:34:39.756507+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.756507+0000","objects_scrubbed":26,"log_size":42,"log_dups_size":100,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:57:48.133450+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":444,"num_read_kb":381,"num_write":158,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"312'284","reported_seq":1054,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.721473+0000","last_change":"2026-02-24T22:34:33.721473+0000","last_active":"2026-02-24T22:34:33.721473+0000","last_peered":"2026-02-24T22:34:33.721473+0000","last_clean":"2026-02-24T22:34:33.721473+0000","last_became_active":"2026-02-24T22:05:16.184594+0000","last_became_peered":"2026-02-24T22:05:16.184594+0000","last_unstale":"2026-02-24T22:34:33.721473+0000","last_undegraded":"2026-02-24T22:34:33.721473+0000","last_fullsized":"2026-02-24T22:34:33.721473+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'284","last_scrub_stamp":"2026-02-24T22:34:33.721410+0000","last_deep_scrub":"312'284","last_deep_scrub_stamp":"2026-02-24T22:34:33.721410+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.721410+0000","objects_scrubbed":5,"log_size":84,"log_dups_size":200,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:42:30.853887+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048643,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":519,"num_read_kb":28116,"num_write":1194,"num_write_kb":6289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":591,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.669842+0000","last_change":"2026-02-24T22:34:44.669842+0000","last_active":"2026-02-24T22:34:44.669842+0000","last_peered":"2026-02-24T22:34:44.669842+0000","last_clean":"2026-02-24T22:34:44.669842+0000","last_became_active":"2026-02-24T22:05:20.193522+0000","last_became_peered":"2026-02-24T22:05:20.193522+0000","last_unstale":"2026-02-24T22:34:44.669842+0000","last_undegraded":"2026-02-24T22:34:44.669842+0000","last_fullsized":"2026-02-24T22:34:44.669842+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:44.669782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:44.669782+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.669782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:43:09.538394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"312'438","reported_seq":1285,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:55.258468+0000","last_change":"2026-02-24T22:34:55.258468+0000","last_active":"2026-02-24T22:34:55.258468+0000","last_peered":"2026-02-24T22:34:55.258468+0000","last_clean":"2026-02-24T22:34:55.258468+0000","last_became_active":"2026-02-24T22:05:22.199968+0000","last_became_peered":"2026-02-24T22:05:22.199968+0000","last_unstale":"2026-02-24T22:34:55.258468+0000","last_undegraded":"2026-02-24T22:34:55.258468+0000","last_fullsized":"2026-02-24T22:34:55.258468+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'438","last_scrub_stamp":"2026-02-24T22:34:55.258389+0000","last_deep_scrub":"312'438","last_deep_scrub_stamp":"2026-02-24T22:34:55.258389+0000","last_clean_scrub_stamp":"2026-02-24T22:34:55.258389+0000","objects_scrubbed":5,"log_size":38,"log_dups_size":400,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:04:07.943168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38886,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1275,"num_read_kb":1335,"num_write":311,"num_write_kb":280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"312'303","reported_seq":913,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:00.858756+0000","last_change":"2026-02-24T22:35:00.858756+0000","last_active":"2026-02-24T22:35:00.858756+0000","last_peered":"2026-02-24T22:35:00.858756+0000","last_clean":"2026-02-24T22:35:00.858756+0000","last_became_active":"2026-02-24T22:05:24.199650+0000","last_became_peered":"2026-02-24T22:05:24.199650+0000","last_unstale":"2026-02-24T22:35:00.858756+0000","last_undegraded":"2026-02-24T22:35:00.858756+0000","last_fullsized":"2026-02-24T22:35:00.858756+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'303","last_scrub_stamp":"2026-02-24T22:35:00.858703+0000","last_deep_scrub":"312'303","last_deep_scrub_stamp":"2026-02-24T22:35:00.858703+0000","last_clean_scrub_stamp":"2026-02-24T22:35:00.858703+0000","objects_scrubbed":91,"log_size":103,"log_dups_size":200,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:01:19.132884+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29523,"num_objects":91,"num_object_clones":0,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":182,"num_read_kb":193,"num_write":488,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"312'270","reported_seq":989,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.736339+0000","last_change":"2026-02-24T22:34:41.736339+0000","last_active":"2026-02-24T22:34:41.736339+0000","last_peered":"2026-02-24T22:34:41.736339+0000","last_clean":"2026-02-24T22:34:41.736339+0000","last_became_active":"2026-02-24T22:05:18.190238+0000","last_became_peered":"2026-02-24T22:05:18.190238+0000","last_unstale":"2026-02-24T22:34:41.736339+0000","last_undegraded":"2026-02-24T22:34:41.736339+0000","last_fullsized":"2026-02-24T22:34:41.736339+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'270","last_scrub_stamp":"2026-02-24T22:34:41.736291+0000","last_deep_scrub":"312'270","last_deep_scrub_stamp":"2026-02-24T22:34:41.736291+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.736291+0000","objects_scrubbed":21,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:33:02.999931+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":976,"num_read_kb":794,"num_write":353,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":219,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"312'310","reported_seq":1674,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.902084+0000","last_change":"2026-02-24T22:34:33.902084+0000","last_active":"2026-02-24T22:34:33.902084+0000","last_peered":"2026-02-24T22:34:33.902084+0000","last_clean":"2026-02-24T22:34:33.902084+0000","last_became_active":"2026-02-24T22:05:16.184415+0000","last_became_peered":"2026-02-24T22:05:16.184415+0000","last_unstale":"2026-02-24T22:34:33.902084+0000","last_undegraded":"2026-02-24T22:34:33.902084+0000","last_fullsized":"2026-02-24T22:34:33.902084+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'310","last_scrub_stamp":"2026-02-24T22:34:33.902021+0000","last_deep_scrub":"312'310","last_deep_scrub_stamp":"2026-02-24T22:34:33.902021+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.902021+0000","objects_scrubbed":3,"log_size":110,"log_dups_size":200,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:11:44.512299+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2362,"num_read_kb":23834,"num_write":1125,"num_write_kb":5557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":599,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:50.938248+0000","last_change":"2026-02-24T22:34:50.938212+0000","last_active":"2026-02-24T22:34:50.938248+0000","last_peered":"2026-02-24T22:34:50.938248+0000","last_clean":"2026-02-24T22:34:50.938248+0000","last_became_active":"2026-02-24T22:05:20.186325+0000","last_became_peered":"2026-02-24T22:05:20.186325+0000","last_unstale":"2026-02-24T22:34:50.938248+0000","last_undegraded":"2026-02-24T22:34:50.938248+0000","last_fullsized":"2026-02-24T22:34:50.938248+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:50.938156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:50.938156+0000","last_clean_scrub_stamp":"2026-02-24T22:34:50.938156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:12:47.996800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"312'413","reported_seq":1040,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:53.238960+0000","last_change":"2026-02-24T22:34:53.238905+0000","last_active":"2026-02-24T22:34:53.238960+0000","last_peered":"2026-02-24T22:34:53.238960+0000","last_clean":"2026-02-24T22:34:53.238960+0000","last_became_active":"2026-02-24T22:05:22.192756+0000","last_became_peered":"2026-02-24T22:05:22.192756+0000","last_unstale":"2026-02-24T22:34:53.238960+0000","last_undegraded":"2026-02-24T22:34:53.238960+0000","last_fullsized":"2026-02-24T22:34:53.238960+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'413","last_scrub_stamp":"2026-02-24T22:34:53.238844+0000","last_deep_scrub":"312'413","last_deep_scrub_stamp":"2026-02-24T22:34:53.238844+0000","last_clean_scrub_stamp":"2026-02-24T22:34:53.238844+0000","objects_scrubbed":6,"log_size":13,"log_dups_size":400,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:43:03.519649+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77383,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":715,"num_read_kb":1913,"num_write":704,"num_write_kb":692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24470,"num_omap_keys":44,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"312'286","reported_seq":910,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:59.933962+0000","last_change":"2026-02-24T22:34:59.933925+0000","last_active":"2026-02-24T22:34:59.933962+0000","last_peered":"2026-02-24T22:34:59.933962+0000","last_clean":"2026-02-24T22:34:59.933962+0000","last_became_active":"2026-02-24T22:05:24.206407+0000","last_became_peered":"2026-02-24T22:05:24.206407+0000","last_unstale":"2026-02-24T22:34:59.933962+0000","last_undegraded":"2026-02-24T22:34:59.933962+0000","last_fullsized":"2026-02-24T22:34:59.933962+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'286","last_scrub_stamp":"2026-02-24T22:34:59.933869+0000","last_deep_scrub":"312'286","last_deep_scrub_stamp":"2026-02-24T22:34:59.933869+0000","last_clean_scrub_stamp":"2026-02-24T22:34:59.933869+0000","objects_scrubbed":99,"log_size":86,"log_dups_size":200,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:02:47.770768+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30298,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":168,"num_read_kb":155,"num_write":455,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":584,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:08.590933+0000","last_change":"2026-02-24T22:35:08.590933+0000","last_active":"2026-02-24T22:35:08.590933+0000","last_peered":"2026-02-24T22:35:08.590933+0000","last_clean":"2026-02-24T22:35:08.590933+0000","last_became_active":"2026-02-24T22:05:29.650115+0000","last_became_peered":"2026-02-24T22:05:29.650115+0000","last_unstale":"2026-02-24T22:35:08.590933+0000","last_undegraded":"2026-02-24T22:35:08.590933+0000","last_fullsized":"2026-02-24T22:35:08.590933+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:08.590877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:08.590877+0000","last_clean_scrub_stamp":"2026-02-24T22:35:08.590877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:49:57.541981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"312'193","reported_seq":915,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.857368+0000","last_change":"2026-02-24T22:34:43.857329+0000","last_active":"2026-02-24T22:34:43.857368+0000","last_peered":"2026-02-24T22:34:43.857368+0000","last_clean":"2026-02-24T22:34:43.857368+0000","last_became_active":"2026-02-24T22:05:18.183501+0000","last_became_peered":"2026-02-24T22:05:18.183501+0000","last_unstale":"2026-02-24T22:34:43.857368+0000","last_undegraded":"2026-02-24T22:34:43.857368+0000","last_fullsized":"2026-02-24T22:34:43.857368+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'193","last_scrub_stamp":"2026-02-24T22:34:43.857270+0000","last_deep_scrub":"312'193","last_deep_scrub_stamp":"2026-02-24T22:34:43.857270+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.857270+0000","objects_scrubbed":31,"log_size":93,"log_dups_size":100,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:49:05.843210+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":578,"num_read_kb":488,"num_write":224,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":465,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"312'412","reported_seq":1362,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.006923+0000","last_change":"2026-02-24T22:34:33.006923+0000","last_active":"2026-02-24T22:34:33.006923+0000","last_peered":"2026-02-24T22:34:33.006923+0000","last_clean":"2026-02-24T22:34:33.006923+0000","last_became_active":"2026-02-24T22:05:16.182909+0000","last_became_peered":"2026-02-24T22:05:16.182909+0000","last_unstale":"2026-02-24T22:34:33.006923+0000","last_undegraded":"2026-02-24T22:34:33.006923+0000","last_fullsized":"2026-02-24T22:34:33.006923+0000","mapping_epoch":22,"log_start":"312'400","ondisk_log_start":"312'400","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'412","last_scrub_stamp":"2026-02-24T22:34:33.006865+0000","last_deep_scrub":"312'412","last_deep_scrub_stamp":"2026-02-24T22:34:33.006865+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.006865+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":400,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:36:32.796788+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1207,"num_read_kb":9244,"num_write":1611,"num_write_kb":11784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":593,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.622797+0000","last_change":"2026-02-24T22:34:45.622797+0000","last_active":"2026-02-24T22:34:45.622797+0000","last_peered":"2026-02-24T22:34:45.622797+0000","last_clean":"2026-02-24T22:34:45.622797+0000","last_became_active":"2026-02-24T22:05:20.193751+0000","last_became_peered":"2026-02-24T22:05:20.193751+0000","last_unstale":"2026-02-24T22:34:45.622797+0000","last_undegraded":"2026-02-24T22:34:45.622797+0000","last_fullsized":"2026-02-24T22:34:45.622797+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:45.622745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:45.622745+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.622745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:01:44.701244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"312'754","reported_seq":1846,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:56.215537+0000","last_change":"2026-02-24T22:34:56.215477+0000","last_active":"2026-02-24T22:34:56.215537+0000","last_peered":"2026-02-24T22:34:56.215537+0000","last_clean":"2026-02-24T22:34:56.215537+0000","last_became_active":"2026-02-24T22:05:22.200603+0000","last_became_peered":"2026-02-24T22:05:22.200603+0000","last_unstale":"2026-02-24T22:34:56.215537+0000","last_undegraded":"2026-02-24T22:34:56.215537+0000","last_fullsized":"2026-02-24T22:34:56.215537+0000","mapping_epoch":28,"log_start":"312'700","ondisk_log_start":"312'700","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'754","last_scrub_stamp":"2026-02-24T22:34:56.215404+0000","last_deep_scrub":"312'754","last_deep_scrub_stamp":"2026-02-24T22:34:56.215404+0000","last_clean_scrub_stamp":"2026-02-24T22:34:56.215404+0000","objects_scrubbed":10,"log_size":54,"log_dups_size":700,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:58:38.795319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39896,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":2333,"num_read_kb":2057,"num_write":434,"num_write_kb":373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11460,"num_omap_keys":24,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"312'433","reported_seq":1078,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:55.324437+0000","last_change":"2026-02-24T22:34:55.324399+0000","last_active":"2026-02-24T22:34:55.324437+0000","last_peered":"2026-02-24T22:34:55.324437+0000","last_clean":"2026-02-24T22:34:55.324437+0000","last_became_active":"2026-02-24T22:05:24.200391+0000","last_became_peered":"2026-02-24T22:05:24.200391+0000","last_unstale":"2026-02-24T22:34:55.324437+0000","last_undegraded":"2026-02-24T22:34:55.324437+0000","last_fullsized":"2026-02-24T22:34:55.324437+0000","mapping_epoch":30,"log_start":"312'400","ondisk_log_start":"312'400","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'433","last_scrub_stamp":"2026-02-24T22:34:55.324342+0000","last_deep_scrub":"312'433","last_deep_scrub_stamp":"2026-02-24T22:34:55.324342+0000","last_clean_scrub_stamp":"2026-02-24T22:34:55.324342+0000","objects_scrubbed":99,"log_size":33,"log_dups_size":400,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:09:49.237235+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31847,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":370,"num_read_kb":353,"num_write":777,"num_write_kb":271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"312'82","reported_seq":1003,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:53.578429+0000","last_change":"2026-02-24T22:34:53.578429+0000","last_active":"2026-02-24T22:34:53.578429+0000","last_peered":"2026-02-24T22:34:53.578429+0000","last_clean":"2026-02-24T22:34:53.578429+0000","last_became_active":"2026-02-24T22:05:29.660289+0000","last_became_peered":"2026-02-24T22:05:29.660289+0000","last_unstale":"2026-02-24T22:34:53.578429+0000","last_undegraded":"2026-02-24T22:34:53.578429+0000","last_fullsized":"2026-02-24T22:34:53.578429+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'82","last_scrub_stamp":"2026-02-24T22:34:53.578366+0000","last_deep_scrub":"312'82","last_deep_scrub_stamp":"2026-02-24T22:34:53.578366+0000","last_clean_scrub_stamp":"2026-02-24T22:34:53.578366+0000","objects_scrubbed":1,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:20:48.789151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"312'120","reported_seq":799,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.773740+0000","last_change":"2026-02-24T22:34:43.773705+0000","last_active":"2026-02-24T22:34:43.773740+0000","last_peered":"2026-02-24T22:34:43.773740+0000","last_clean":"2026-02-24T22:34:43.773740+0000","last_became_active":"2026-02-24T22:05:18.184745+0000","last_became_peered":"2026-02-24T22:05:18.184745+0000","last_unstale":"2026-02-24T22:34:43.773740+0000","last_undegraded":"2026-02-24T22:34:43.773740+0000","last_fullsized":"2026-02-24T22:34:43.773740+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'120","last_scrub_stamp":"2026-02-24T22:34:43.773655+0000","last_deep_scrub":"312'120","last_deep_scrub_stamp":"2026-02-24T22:34:43.773655+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.773655+0000","objects_scrubbed":20,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:13:15.141522+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":371,"num_read_kb":307,"num_write":132,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":684,"num_omap_keys":3,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"312'307","reported_seq":1036,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.531212+0000","last_change":"2026-02-24T22:34:33.531150+0000","last_active":"2026-02-24T22:34:33.531212+0000","last_peered":"2026-02-24T22:34:33.531212+0000","last_clean":"2026-02-24T22:34:33.531212+0000","last_became_active":"2026-02-24T22:05:16.185574+0000","last_became_peered":"2026-02-24T22:05:16.185574+0000","last_unstale":"2026-02-24T22:34:33.531212+0000","last_undegraded":"2026-02-24T22:34:33.531212+0000","last_fullsized":"2026-02-24T22:34:33.531212+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'307","last_scrub_stamp":"2026-02-24T22:34:33.531066+0000","last_deep_scrub":"312'307","last_deep_scrub_stamp":"2026-02-24T22:34:33.531066+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.531066+0000","objects_scrubbed":4,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:08:13.354888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":428,"num_read_kb":30007,"num_write":1257,"num_write_kb":21358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"312'4","reported_seq":841,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:54.234549+0000","last_change":"2026-02-24T22:34:54.234514+0000","last_active":"2026-02-24T22:34:54.234549+0000","last_peered":"2026-02-24T22:34:54.234549+0000","last_clean":"2026-02-24T22:34:54.234549+0000","last_became_active":"2026-02-24T22:05:20.192612+0000","last_became_peered":"2026-02-24T22:05:20.192612+0000","last_unstale":"2026-02-24T22:34:54.234549+0000","last_undegraded":"2026-02-24T22:34:54.234549+0000","last_fullsized":"2026-02-24T22:34:54.234549+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'4","last_scrub_stamp":"2026-02-24T22:34:54.234459+0000","last_deep_scrub":"312'4","last_deep_scrub_stamp":"2026-02-24T22:34:54.234459+0000","last_clean_scrub_stamp":"2026-02-24T22:34:54.234459+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:52:30.469777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":484,"num_read_kb":482,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"312'209","reported_seq":830,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:56.698535+0000","last_change":"2026-02-24T22:34:56.698535+0000","last_active":"2026-02-24T22:34:56.698535+0000","last_peered":"2026-02-24T22:34:56.698535+0000","last_clean":"2026-02-24T22:34:56.698535+0000","last_became_active":"2026-02-24T22:05:22.906047+0000","last_became_peered":"2026-02-24T22:05:22.906047+0000","last_unstale":"2026-02-24T22:34:56.698535+0000","last_undegraded":"2026-02-24T22:34:56.698535+0000","last_fullsized":"2026-02-24T22:34:56.698535+0000","mapping_epoch":28,"log_start":"312'100","ondisk_log_start":"312'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'209","last_scrub_stamp":"2026-02-24T22:34:56.698473+0000","last_deep_scrub":"312'209","last_deep_scrub_stamp":"2026-02-24T22:34:56.698473+0000","last_clean_scrub_stamp":"2026-02-24T22:34:56.698473+0000","objects_scrubbed":5,"log_size":109,"log_dups_size":100,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:47:56.787199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38786,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":328,"num_read_kb":839,"num_write":300,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"312'257","reported_seq":867,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:54.273436+0000","last_change":"2026-02-24T22:34:54.273407+0000","last_active":"2026-02-24T22:34:54.273436+0000","last_peered":"2026-02-24T22:34:54.273436+0000","last_clean":"2026-02-24T22:34:54.273436+0000","last_became_active":"2026-02-24T22:05:24.200747+0000","last_became_peered":"2026-02-24T22:05:24.200747+0000","last_unstale":"2026-02-24T22:34:54.273436+0000","last_undegraded":"2026-02-24T22:34:54.273436+0000","last_fullsized":"2026-02-24T22:34:54.273436+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'257","last_scrub_stamp":"2026-02-24T22:34:54.273373+0000","last_deep_scrub":"312'257","last_deep_scrub_stamp":"2026-02-24T22:34:54.273373+0000","last_clean_scrub_stamp":"2026-02-24T22:34:54.273373+0000","objects_scrubbed":93,"log_size":57,"log_dups_size":200,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:40:30.932635+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30873,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":120,"num_read_kb":110,"num_write":405,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":573,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:03.846658+0000","last_change":"2026-02-24T22:35:03.846658+0000","last_active":"2026-02-24T22:35:03.846658+0000","last_peered":"2026-02-24T22:35:03.846658+0000","last_clean":"2026-02-24T22:35:03.846658+0000","last_became_active":"2026-02-24T22:05:29.660344+0000","last_became_peered":"2026-02-24T22:05:29.660344+0000","last_unstale":"2026-02-24T22:35:03.846658+0000","last_undegraded":"2026-02-24T22:35:03.846658+0000","last_fullsized":"2026-02-24T22:35:03.846658+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:03.846600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:03.846600+0000","last_clean_scrub_stamp":"2026-02-24T22:35:03.846600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:10:57.615627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"312'176","reported_seq":857,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.075488+0000","last_change":"2026-02-24T22:34:36.075455+0000","last_active":"2026-02-24T22:34:36.075488+0000","last_peered":"2026-02-24T22:34:36.075488+0000","last_clean":"2026-02-24T22:34:36.075488+0000","last_became_active":"2026-02-24T22:06:00.759215+0000","last_became_peered":"2026-02-24T22:06:00.759215+0000","last_unstale":"2026-02-24T22:34:36.075488+0000","last_undegraded":"2026-02-24T22:34:36.075488+0000","last_fullsized":"2026-02-24T22:34:36.075488+0000","mapping_epoch":36,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'176","last_scrub_stamp":"2026-02-24T22:34:36.075425+0000","last_deep_scrub":"312'176","last_deep_scrub_stamp":"2026-02-24T22:34:36.075425+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.075425+0000","objects_scrubbed":19,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:59:34.861298+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":584,"num_read_kb":483,"num_write":222,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"312'269","reported_seq":1029,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.707518+0000","last_change":"2026-02-24T22:34:31.707474+0000","last_active":"2026-02-24T22:34:31.707518+0000","last_peered":"2026-02-24T22:34:31.707518+0000","last_clean":"2026-02-24T22:34:31.707518+0000","last_became_active":"2026-02-24T22:05:16.183956+0000","last_became_peered":"2026-02-24T22:05:16.183956+0000","last_unstale":"2026-02-24T22:34:31.707518+0000","last_undegraded":"2026-02-24T22:34:31.707518+0000","last_fullsized":"2026-02-24T22:34:31.707518+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'269","last_scrub_stamp":"2026-02-24T22:34:31.707414+0000","last_deep_scrub":"312'269","last_deep_scrub_stamp":"2026-02-24T22:34:31.707414+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.707414+0000","objects_scrubbed":6,"log_size":69,"log_dups_size":200,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:16:18.564686+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242893,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":575,"num_read_kb":6430,"num_write":1267,"num_write_kb":5371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":596,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.203295+0000","last_change":"2026-02-24T22:34:49.203221+0000","last_active":"2026-02-24T22:34:49.203295+0000","last_peered":"2026-02-24T22:34:49.203295+0000","last_clean":"2026-02-24T22:34:49.203295+0000","last_became_active":"2026-02-24T22:05:20.187316+0000","last_became_peered":"2026-02-24T22:05:20.187316+0000","last_unstale":"2026-02-24T22:34:49.203295+0000","last_undegraded":"2026-02-24T22:34:49.203295+0000","last_fullsized":"2026-02-24T22:34:49.203295+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:49.203134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:49.203134+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.203134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:31:08.087721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"312'295","reported_seq":1143,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.297709+0000","last_change":"2026-02-24T22:34:58.297675+0000","last_active":"2026-02-24T22:34:58.297709+0000","last_peered":"2026-02-24T22:34:58.297709+0000","last_clean":"2026-02-24T22:34:58.297709+0000","last_became_active":"2026-02-24T22:05:22.912087+0000","last_became_peered":"2026-02-24T22:05:22.912087+0000","last_unstale":"2026-02-24T22:34:58.297709+0000","last_undegraded":"2026-02-24T22:34:58.297709+0000","last_fullsized":"2026-02-24T22:34:58.297709+0000","mapping_epoch":28,"log_start":"307'200","ondisk_log_start":"307'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'295","last_scrub_stamp":"2026-02-24T22:34:58.297632+0000","last_deep_scrub":"312'295","last_deep_scrub_stamp":"2026-02-24T22:34:58.297632+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.297632+0000","objects_scrubbed":7,"log_size":95,"log_dups_size":200,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:34:14.546778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":152,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1000,"num_read_kb":520,"num_write":51,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1780,"num_omap_keys":6,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"312'272","reported_seq":953,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:50.604472+0000","last_change":"2026-02-24T22:34:50.604472+0000","last_active":"2026-02-24T22:34:50.604472+0000","last_peered":"2026-02-24T22:34:50.604472+0000","last_clean":"2026-02-24T22:34:50.604472+0000","last_became_active":"2026-02-24T22:05:24.206383+0000","last_became_peered":"2026-02-24T22:05:24.206383+0000","last_unstale":"2026-02-24T22:34:50.604472+0000","last_undegraded":"2026-02-24T22:34:50.604472+0000","last_fullsized":"2026-02-24T22:34:50.604472+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'272","last_scrub_stamp":"2026-02-24T22:34:50.604410+0000","last_deep_scrub":"312'272","last_deep_scrub_stamp":"2026-02-24T22:34:50.604410+0000","last_clean_scrub_stamp":"2026-02-24T22:34:50.604410+0000","objects_scrubbed":94,"log_size":72,"log_dups_size":200,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:58:49.327922+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29475,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":255,"num_read_kb":237,"num_write":434,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":579,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:03.209395+0000","last_change":"2026-02-24T22:35:03.209395+0000","last_active":"2026-02-24T22:35:03.209395+0000","last_peered":"2026-02-24T22:35:03.209395+0000","last_clean":"2026-02-24T22:35:03.209395+0000","last_became_active":"2026-02-24T22:05:29.661539+0000","last_became_peered":"2026-02-24T22:05:29.661539+0000","last_unstale":"2026-02-24T22:35:03.209395+0000","last_undegraded":"2026-02-24T22:35:03.209395+0000","last_fullsized":"2026-02-24T22:35:03.209395+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:03.209335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:03.209335+0000","last_clean_scrub_stamp":"2026-02-24T22:35:03.209335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:44:51.735842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"312'201","reported_seq":882,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.483789+0000","last_change":"2026-02-24T22:34:44.483754+0000","last_active":"2026-02-24T22:34:44.483789+0000","last_peered":"2026-02-24T22:34:44.483789+0000","last_clean":"2026-02-24T22:34:44.483789+0000","last_became_active":"2026-02-24T22:05:18.192487+0000","last_became_peered":"2026-02-24T22:05:18.192487+0000","last_unstale":"2026-02-24T22:34:44.483789+0000","last_undegraded":"2026-02-24T22:34:44.483789+0000","last_fullsized":"2026-02-24T22:34:44.483789+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'201","last_scrub_stamp":"2026-02-24T22:34:44.483706+0000","last_deep_scrub":"312'201","last_deep_scrub_stamp":"2026-02-24T22:34:44.483706+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.483706+0000","objects_scrubbed":23,"log_size":101,"log_dups_size":100,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:31:57.971063+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":681,"num_read_kb":551,"num_write":266,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"312'299","reported_seq":1007,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.128172+0000","last_change":"2026-02-24T22:34:36.128130+0000","last_active":"2026-02-24T22:34:36.128172+0000","last_peered":"2026-02-24T22:34:36.128172+0000","last_clean":"2026-02-24T22:34:36.128172+0000","last_became_active":"2026-02-24T22:05:16.190108+0000","last_became_peered":"2026-02-24T22:05:16.190108+0000","last_unstale":"2026-02-24T22:34:36.128172+0000","last_undegraded":"2026-02-24T22:34:36.128172+0000","last_fullsized":"2026-02-24T22:34:36.128172+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'299","last_scrub_stamp":"2026-02-24T22:34:36.128072+0000","last_deep_scrub":"312'299","last_deep_scrub_stamp":"2026-02-24T22:34:36.128072+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.128072+0000","objects_scrubbed":3,"log_size":99,"log_dups_size":200,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:52:02.916678+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242914,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":451,"num_read_kb":15743,"num_write":1326,"num_write_kb":10328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":593,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:52.177876+0000","last_change":"2026-02-24T22:34:52.177761+0000","last_active":"2026-02-24T22:34:52.177876+0000","last_peered":"2026-02-24T22:34:52.177876+0000","last_clean":"2026-02-24T22:34:52.177876+0000","last_became_active":"2026-02-24T22:05:20.194183+0000","last_became_peered":"2026-02-24T22:05:20.194183+0000","last_unstale":"2026-02-24T22:34:52.177876+0000","last_undegraded":"2026-02-24T22:34:52.177876+0000","last_fullsized":"2026-02-24T22:34:52.177876+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:52.177667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:52.177667+0000","last_clean_scrub_stamp":"2026-02-24T22:34:52.177667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:07:28.331498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"312'722","reported_seq":1808,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.944289+0000","last_change":"2026-02-24T22:34:58.944289+0000","last_active":"2026-02-24T22:34:58.944289+0000","last_peered":"2026-02-24T22:34:58.944289+0000","last_clean":"2026-02-24T22:34:58.944289+0000","last_became_active":"2026-02-24T22:05:22.199480+0000","last_became_peered":"2026-02-24T22:05:22.199480+0000","last_unstale":"2026-02-24T22:34:58.944289+0000","last_undegraded":"2026-02-24T22:34:58.944289+0000","last_fullsized":"2026-02-24T22:34:58.944289+0000","mapping_epoch":28,"log_start":"312'700","ondisk_log_start":"312'700","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'722","last_scrub_stamp":"2026-02-24T22:34:58.944231+0000","last_deep_scrub":"312'722","last_deep_scrub_stamp":"2026-02-24T22:34:58.944231+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.944231+0000","objects_scrubbed":9,"log_size":22,"log_dups_size":700,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:10:48.727164+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39366,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":2266,"num_read_kb":1888,"num_write":352,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"312'285","reported_seq":904,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:01.828096+0000","last_change":"2026-02-24T22:35:01.828096+0000","last_active":"2026-02-24T22:35:01.828096+0000","last_peered":"2026-02-24T22:35:01.828096+0000","last_clean":"2026-02-24T22:35:01.828096+0000","last_became_active":"2026-02-24T22:05:24.200434+0000","last_became_peered":"2026-02-24T22:05:24.200434+0000","last_unstale":"2026-02-24T22:35:01.828096+0000","last_undegraded":"2026-02-24T22:35:01.828096+0000","last_fullsized":"2026-02-24T22:35:01.828096+0000","mapping_epoch":30,"log_start":"312'198","ondisk_log_start":"312'198","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'285","last_scrub_stamp":"2026-02-24T22:35:01.828052+0000","last_deep_scrub":"312'285","last_deep_scrub_stamp":"2026-02-24T22:35:01.828052+0000","last_clean_scrub_stamp":"2026-02-24T22:35:01.828052+0000","objects_scrubbed":87,"log_size":87,"log_dups_size":198,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:18:55.623149+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27134,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":189,"num_read_kb":178,"num_write":476,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":579,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:06.277822+0000","last_change":"2026-02-24T22:35:06.277822+0000","last_active":"2026-02-24T22:35:06.277822+0000","last_peered":"2026-02-24T22:35:06.277822+0000","last_clean":"2026-02-24T22:35:06.277822+0000","last_became_active":"2026-02-24T22:05:29.662425+0000","last_became_peered":"2026-02-24T22:05:29.662425+0000","last_unstale":"2026-02-24T22:35:06.277822+0000","last_undegraded":"2026-02-24T22:35:06.277822+0000","last_fullsized":"2026-02-24T22:35:06.277822+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:06.277768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:06.277768+0000","last_clean_scrub_stamp":"2026-02-24T22:35:06.277768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:06:45.271197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"312'157","reported_seq":845,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.518898+0000","last_change":"2026-02-24T22:34:42.518852+0000","last_active":"2026-02-24T22:34:42.518898+0000","last_peered":"2026-02-24T22:34:42.518898+0000","last_clean":"2026-02-24T22:34:42.518898+0000","last_became_active":"2026-02-24T22:05:18.191664+0000","last_became_peered":"2026-02-24T22:05:18.191664+0000","last_unstale":"2026-02-24T22:34:42.518898+0000","last_undegraded":"2026-02-24T22:34:42.518898+0000","last_fullsized":"2026-02-24T22:34:42.518898+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'157","last_scrub_stamp":"2026-02-24T22:34:42.518790+0000","last_deep_scrub":"312'157","last_deep_scrub_stamp":"2026-02-24T22:34:42.518790+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.518790+0000","objects_scrubbed":25,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:34:13.833329+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":463,"num_read_kb":379,"num_write":198,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"312'278","reported_seq":982,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.114301+0000","last_change":"2026-02-24T22:34:35.114265+0000","last_active":"2026-02-24T22:34:35.114301+0000","last_peered":"2026-02-24T22:34:35.114301+0000","last_clean":"2026-02-24T22:34:35.114301+0000","last_became_active":"2026-02-24T22:05:16.188154+0000","last_became_peered":"2026-02-24T22:05:16.188154+0000","last_unstale":"2026-02-24T22:34:35.114301+0000","last_undegraded":"2026-02-24T22:34:35.114301+0000","last_fullsized":"2026-02-24T22:34:35.114301+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'278","last_scrub_stamp":"2026-02-24T22:34:35.114217+0000","last_deep_scrub":"312'278","last_deep_scrub_stamp":"2026-02-24T22:34:35.114217+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.114217+0000","objects_scrubbed":6,"log_size":78,"log_dups_size":200,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:02:24.896641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":434,"num_read_kb":2459,"num_write":1229,"num_write_kb":1814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"312'6","reported_seq":831,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:52.862848+0000","last_change":"2026-02-24T22:34:52.862809+0000","last_active":"2026-02-24T22:34:52.862848+0000","last_peered":"2026-02-24T22:34:52.862848+0000","last_clean":"2026-02-24T22:34:52.862848+0000","last_became_active":"2026-02-24T22:05:20.195202+0000","last_became_peered":"2026-02-24T22:05:20.195202+0000","last_unstale":"2026-02-24T22:34:52.862848+0000","last_undegraded":"2026-02-24T22:34:52.862848+0000","last_fullsized":"2026-02-24T22:34:52.862848+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'6","last_scrub_stamp":"2026-02-24T22:34:52.862767+0000","last_deep_scrub":"312'6","last_deep_scrub_stamp":"2026-02-24T22:34:52.862767+0000","last_clean_scrub_stamp":"2026-02-24T22:34:52.862767+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:47:13.375532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1242,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":488,"num_read_kb":722,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"312'693","reported_seq":1779,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:55.932150+0000","last_change":"2026-02-24T22:34:55.932150+0000","last_active":"2026-02-24T22:34:55.932150+0000","last_peered":"2026-02-24T22:34:55.932150+0000","last_clean":"2026-02-24T22:34:55.932150+0000","last_became_active":"2026-02-24T22:05:22.192035+0000","last_became_peered":"2026-02-24T22:05:22.192035+0000","last_unstale":"2026-02-24T22:34:55.932150+0000","last_undegraded":"2026-02-24T22:34:55.932150+0000","last_fullsized":"2026-02-24T22:34:55.932150+0000","mapping_epoch":28,"log_start":"312'600","ondisk_log_start":"312'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'693","last_scrub_stamp":"2026-02-24T22:34:55.932092+0000","last_deep_scrub":"312'693","last_deep_scrub_stamp":"2026-02-24T22:34:55.932092+0000","last_clean_scrub_stamp":"2026-02-24T22:34:55.932092+0000","objects_scrubbed":6,"log_size":93,"log_dups_size":600,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:00:00.955174+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38403,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2221,"num_read_kb":1805,"num_write":316,"num_write_kb":280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"312'2582","reported_seq":5257,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:51.588402+0000","last_change":"2026-02-24T22:34:51.588402+0000","last_active":"2026-02-24T22:34:51.588402+0000","last_peered":"2026-02-24T22:34:51.588402+0000","last_clean":"2026-02-24T22:34:51.588402+0000","last_became_active":"2026-02-24T22:05:24.205557+0000","last_became_peered":"2026-02-24T22:05:24.205557+0000","last_unstale":"2026-02-24T22:34:51.588402+0000","last_undegraded":"2026-02-24T22:34:51.588402+0000","last_fullsized":"2026-02-24T22:34:51.588402+0000","mapping_epoch":30,"log_start":"312'2500","ondisk_log_start":"312'2500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2582","last_scrub_stamp":"2026-02-24T22:34:51.588360+0000","last_deep_scrub":"312'2582","last_deep_scrub_stamp":"2026-02-24T22:34:51.588360+0000","last_clean_scrub_stamp":"2026-02-24T22:34:51.588360+0000","objects_scrubbed":105,"log_size":82,"log_dups_size":2500,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:15:50.057193+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34698,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":6755,"num_read_kb":6753,"num_write":5061,"num_write_kb":2022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":584,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:07.610091+0000","last_change":"2026-02-24T22:35:07.610091+0000","last_active":"2026-02-24T22:35:07.610091+0000","last_peered":"2026-02-24T22:35:07.610091+0000","last_clean":"2026-02-24T22:35:07.610091+0000","last_became_active":"2026-02-24T22:05:29.651760+0000","last_became_peered":"2026-02-24T22:05:29.651760+0000","last_unstale":"2026-02-24T22:35:07.610091+0000","last_undegraded":"2026-02-24T22:35:07.610091+0000","last_fullsized":"2026-02-24T22:35:07.610091+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:07.610000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:07.610000+0000","last_clean_scrub_stamp":"2026-02-24T22:35:07.610000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:05:45.306861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"312'155","reported_seq":841,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.775671+0000","last_change":"2026-02-24T22:34:44.775671+0000","last_active":"2026-02-24T22:34:44.775671+0000","last_peered":"2026-02-24T22:34:44.775671+0000","last_clean":"2026-02-24T22:34:44.775671+0000","last_became_active":"2026-02-24T22:05:18.184939+0000","last_became_peered":"2026-02-24T22:05:18.184939+0000","last_unstale":"2026-02-24T22:34:44.775671+0000","last_undegraded":"2026-02-24T22:34:44.775671+0000","last_fullsized":"2026-02-24T22:34:44.775671+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'155","last_scrub_stamp":"2026-02-24T22:34:44.775613+0000","last_deep_scrub":"312'155","last_deep_scrub_stamp":"2026-02-24T22:34:44.775613+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.775613+0000","objects_scrubbed":27,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:46:49.648242+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":436,"num_read_kb":360,"num_write":185,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"312'291","reported_seq":1024,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.911303+0000","last_change":"2026-02-24T22:34:34.911246+0000","last_active":"2026-02-24T22:34:34.911303+0000","last_peered":"2026-02-24T22:34:34.911303+0000","last_clean":"2026-02-24T22:34:34.911303+0000","last_became_active":"2026-02-24T22:05:16.186043+0000","last_became_peered":"2026-02-24T22:05:16.186043+0000","last_unstale":"2026-02-24T22:34:34.911303+0000","last_undegraded":"2026-02-24T22:34:34.911303+0000","last_fullsized":"2026-02-24T22:34:34.911303+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'291","last_scrub_stamp":"2026-02-24T22:34:34.911183+0000","last_deep_scrub":"312'291","last_deep_scrub_stamp":"2026-02-24T22:34:34.911183+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.911183+0000","objects_scrubbed":4,"log_size":91,"log_dups_size":200,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:44:00.316854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":465,"num_read_kb":15751,"num_write":1254,"num_write_kb":5255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":598,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:57.248861+0000","last_change":"2026-02-24T22:34:57.248861+0000","last_active":"2026-02-24T22:34:57.248861+0000","last_peered":"2026-02-24T22:34:57.248861+0000","last_clean":"2026-02-24T22:34:57.248861+0000","last_became_active":"2026-02-24T22:05:20.192962+0000","last_became_peered":"2026-02-24T22:05:20.192962+0000","last_unstale":"2026-02-24T22:34:57.248861+0000","last_undegraded":"2026-02-24T22:34:57.248861+0000","last_fullsized":"2026-02-24T22:34:57.248861+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:57.248813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:57.248813+0000","last_clean_scrub_stamp":"2026-02-24T22:34:57.248813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:44:05.255470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"312'478","reported_seq":1324,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:59.613005+0000","last_change":"2026-02-24T22:34:59.612935+0000","last_active":"2026-02-24T22:34:59.613005+0000","last_peered":"2026-02-24T22:34:59.613005+0000","last_clean":"2026-02-24T22:34:59.613005+0000","last_became_active":"2026-02-24T22:05:22.193896+0000","last_became_peered":"2026-02-24T22:05:22.193896+0000","last_unstale":"2026-02-24T22:34:59.613005+0000","last_undegraded":"2026-02-24T22:34:59.613005+0000","last_fullsized":"2026-02-24T22:34:59.613005+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'478","last_scrub_stamp":"2026-02-24T22:34:59.612864+0000","last_deep_scrub":"312'478","last_deep_scrub_stamp":"2026-02-24T22:34:59.612864+0000","last_clean_scrub_stamp":"2026-02-24T22:34:59.612864+0000","objects_scrubbed":3,"log_size":78,"log_dups_size":400,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:18:12.188442+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41178,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1310,"num_read_kb":1453,"num_write":355,"num_write_kb":318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"312'282","reported_seq":909,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:02.283510+0000","last_change":"2026-02-24T22:35:02.283472+0000","last_active":"2026-02-24T22:35:02.283510+0000","last_peered":"2026-02-24T22:35:02.283510+0000","last_clean":"2026-02-24T22:35:02.283510+0000","last_became_active":"2026-02-24T22:05:24.205855+0000","last_became_peered":"2026-02-24T22:05:24.205855+0000","last_unstale":"2026-02-24T22:35:02.283510+0000","last_undegraded":"2026-02-24T22:35:02.283510+0000","last_fullsized":"2026-02-24T22:35:02.283510+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'282","last_scrub_stamp":"2026-02-24T22:35:02.283427+0000","last_deep_scrub":"312'282","last_deep_scrub_stamp":"2026-02-24T22:35:02.283427+0000","last_clean_scrub_stamp":"2026-02-24T22:35:02.283427+0000","objects_scrubbed":94,"log_size":82,"log_dups_size":200,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:30:07.155275+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27729,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":196,"num_read_kb":179,"num_write":471,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"312'449","reported_seq":1629,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:04.238088+0000","last_change":"2026-02-24T22:35:04.238088+0000","last_active":"2026-02-24T22:35:04.238088+0000","last_peered":"2026-02-24T22:35:04.238088+0000","last_clean":"2026-02-24T22:35:04.238088+0000","last_became_active":"2026-02-24T22:05:29.661983+0000","last_became_peered":"2026-02-24T22:05:29.661983+0000","last_unstale":"2026-02-24T22:35:04.238088+0000","last_undegraded":"2026-02-24T22:35:04.238088+0000","last_fullsized":"2026-02-24T22:35:04.238088+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'449","last_scrub_stamp":"2026-02-24T22:35:04.238044+0000","last_deep_scrub":"312'449","last_deep_scrub_stamp":"2026-02-24T22:35:04.238044+0000","last_clean_scrub_stamp":"2026-02-24T22:35:04.238044+0000","objects_scrubbed":1,"log_size":49,"log_dups_size":400,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:37:34.593042+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"312'191","reported_seq":883,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.478571+0000","last_change":"2026-02-24T22:34:40.478571+0000","last_active":"2026-02-24T22:34:40.478571+0000","last_peered":"2026-02-24T22:34:40.478571+0000","last_clean":"2026-02-24T22:34:40.478571+0000","last_became_active":"2026-02-24T22:05:18.189877+0000","last_became_peered":"2026-02-24T22:05:18.189877+0000","last_unstale":"2026-02-24T22:34:40.478571+0000","last_undegraded":"2026-02-24T22:34:40.478571+0000","last_fullsized":"2026-02-24T22:34:40.478571+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'191","last_scrub_stamp":"2026-02-24T22:34:40.478532+0000","last_deep_scrub":"312'191","last_deep_scrub_stamp":"2026-02-24T22:34:40.478532+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.478532+0000","objects_scrubbed":22,"log_size":91,"log_dups_size":100,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:10:11.001904+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":510,"num_write":249,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"312'332","reported_seq":1100,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.714508+0000","last_change":"2026-02-24T22:34:34.714508+0000","last_active":"2026-02-24T22:34:34.714508+0000","last_peered":"2026-02-24T22:34:34.714508+0000","last_clean":"2026-02-24T22:34:34.714508+0000","last_became_active":"2026-02-24T22:05:16.184525+0000","last_became_peered":"2026-02-24T22:05:16.184525+0000","last_unstale":"2026-02-24T22:34:34.714508+0000","last_undegraded":"2026-02-24T22:34:34.714508+0000","last_fullsized":"2026-02-24T22:34:34.714508+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'332","last_scrub_stamp":"2026-02-24T22:34:34.714447+0000","last_deep_scrub":"312'332","last_deep_scrub_stamp":"2026-02-24T22:34:34.714447+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.714447+0000","objects_scrubbed":7,"log_size":32,"log_dups_size":300,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:44:54.170379+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242941,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":576,"num_read_kb":16320,"num_write":1488,"num_write_kb":16752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"312'2","reported_seq":598,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.250464+0000","last_change":"2026-02-24T22:34:48.250464+0000","last_active":"2026-02-24T22:34:48.250464+0000","last_peered":"2026-02-24T22:34:48.250464+0000","last_clean":"2026-02-24T22:34:48.250464+0000","last_became_active":"2026-02-24T22:05:20.186850+0000","last_became_peered":"2026-02-24T22:05:20.186850+0000","last_unstale":"2026-02-24T22:34:48.250464+0000","last_undegraded":"2026-02-24T22:34:48.250464+0000","last_fullsized":"2026-02-24T22:34:48.250464+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:34:48.250415+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:34:48.250415+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.250415+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:00:47.346514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"312'253","reported_seq":885,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:59.281527+0000","last_change":"2026-02-24T22:34:59.281488+0000","last_active":"2026-02-24T22:34:59.281527+0000","last_peered":"2026-02-24T22:34:59.281527+0000","last_clean":"2026-02-24T22:34:59.281527+0000","last_became_active":"2026-02-24T22:05:22.199358+0000","last_became_peered":"2026-02-24T22:05:22.199358+0000","last_unstale":"2026-02-24T22:34:59.281527+0000","last_undegraded":"2026-02-24T22:34:59.281527+0000","last_fullsized":"2026-02-24T22:34:59.281527+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'253","last_scrub_stamp":"2026-02-24T22:34:59.281432+0000","last_deep_scrub":"312'253","last_deep_scrub_stamp":"2026-02-24T22:34:59.281432+0000","last_clean_scrub_stamp":"2026-02-24T22:34:59.281432+0000","objects_scrubbed":4,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:21:43.387290+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37935,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":401,"num_read_kb":1010,"num_write":360,"num_write_kb":308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"312'382","reported_seq":1338,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:02.816448+0000","last_change":"2026-02-24T22:35:02.816402+0000","last_active":"2026-02-24T22:35:02.816448+0000","last_peered":"2026-02-24T22:35:02.816448+0000","last_clean":"2026-02-24T22:35:02.816448+0000","last_became_active":"2026-02-24T22:05:24.199555+0000","last_became_peered":"2026-02-24T22:05:24.199555+0000","last_unstale":"2026-02-24T22:35:02.816448+0000","last_undegraded":"2026-02-24T22:35:02.816448+0000","last_fullsized":"2026-02-24T22:35:02.816448+0000","mapping_epoch":30,"log_start":"312'295","ondisk_log_start":"312'295","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'382","last_scrub_stamp":"2026-02-24T22:35:02.816332+0000","last_deep_scrub":"312'382","last_deep_scrub_stamp":"2026-02-24T22:35:02.816332+0000","last_clean_scrub_stamp":"2026-02-24T22:35:02.816332+0000","objects_scrubbed":101,"log_size":87,"log_dups_size":295,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:38:25.214542+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30969,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":655,"num_read_kb":647,"num_write":639,"num_write_kb":204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":174,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"312'433","reported_seq":1959,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:04.303436+0000","last_change":"2026-02-24T22:35:04.303436+0000","last_active":"2026-02-24T22:35:04.303436+0000","last_peered":"2026-02-24T22:35:04.303436+0000","last_clean":"2026-02-24T22:35:04.303436+0000","last_became_active":"2026-02-24T22:05:29.659532+0000","last_became_peered":"2026-02-24T22:05:29.659532+0000","last_unstale":"2026-02-24T22:35:04.303436+0000","last_undegraded":"2026-02-24T22:35:04.303436+0000","last_fullsized":"2026-02-24T22:35:04.303436+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'433","last_scrub_stamp":"2026-02-24T22:35:04.303372+0000","last_deep_scrub":"312'433","last_deep_scrub_stamp":"2026-02-24T22:35:04.303372+0000","last_clean_scrub_stamp":"2026-02-24T22:35:04.303372+0000","objects_scrubbed":1,"log_size":33,"log_dups_size":400,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:32:29.785307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"312'162","reported_seq":857,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.157775+0000","last_change":"2026-02-24T22:34:45.157775+0000","last_active":"2026-02-24T22:34:45.157775+0000","last_peered":"2026-02-24T22:34:45.157775+0000","last_clean":"2026-02-24T22:34:45.157775+0000","last_became_active":"2026-02-24T22:05:18.184250+0000","last_became_peered":"2026-02-24T22:05:18.184250+0000","last_unstale":"2026-02-24T22:34:45.157775+0000","last_undegraded":"2026-02-24T22:34:45.157775+0000","last_fullsized":"2026-02-24T22:34:45.157775+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'162","last_scrub_stamp":"2026-02-24T22:34:45.157726+0000","last_deep_scrub":"312'162","last_deep_scrub_stamp":"2026-02-24T22:34:45.157726+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.157726+0000","objects_scrubbed":23,"log_size":62,"log_dups_size":100,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:39:01.351061+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":560,"num_read_kb":462,"num_write":204,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"312'1610","reported_seq":2974,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.774854+0000","last_change":"2026-02-24T22:34:33.774854+0000","last_active":"2026-02-24T22:34:33.774854+0000","last_peered":"2026-02-24T22:34:33.774854+0000","last_clean":"2026-02-24T22:34:33.774854+0000","last_became_active":"2026-02-24T22:05:16.189660+0000","last_became_peered":"2026-02-24T22:05:16.189660+0000","last_unstale":"2026-02-24T22:34:33.774854+0000","last_undegraded":"2026-02-24T22:34:33.774854+0000","last_fullsized":"2026-02-24T22:34:33.774854+0000","mapping_epoch":22,"log_start":"312'1500","ondisk_log_start":"312'1500","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'1610","last_scrub_stamp":"2026-02-24T22:34:33.774776+0000","last_deep_scrub":"312'1610","last_deep_scrub_stamp":"2026-02-24T22:34:33.774776+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.774776+0000","objects_scrubbed":6,"log_size":110,"log_dups_size":1500,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:08:14.555476+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4326,"num_read_kb":2601,"num_write":2684,"num_write_kb":4261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"312'121","reported_seq":808,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.128824+0000","last_change":"2026-02-24T22:34:43.128824+0000","last_active":"2026-02-24T22:34:43.128824+0000","last_peered":"2026-02-24T22:34:43.128824+0000","last_clean":"2026-02-24T22:34:43.128824+0000","last_became_active":"2026-02-24T22:05:18.183822+0000","last_became_peered":"2026-02-24T22:05:18.183822+0000","last_unstale":"2026-02-24T22:34:43.128824+0000","last_undegraded":"2026-02-24T22:34:43.128824+0000","last_fullsized":"2026-02-24T22:34:43.128824+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'121","last_scrub_stamp":"2026-02-24T22:34:43.128792+0000","last_deep_scrub":"312'121","last_deep_scrub_stamp":"2026-02-24T22:34:43.128792+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.128792+0000","objects_scrubbed":20,"log_size":21,"log_dups_size":100,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:33:44.340989+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":390,"num_read_kb":330,"num_write":143,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"312'303","reported_seq":1060,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.123769+0000","last_change":"2026-02-24T22:34:32.123769+0000","last_active":"2026-02-24T22:34:32.123769+0000","last_peered":"2026-02-24T22:34:32.123769+0000","last_clean":"2026-02-24T22:34:32.123769+0000","last_became_active":"2026-02-24T22:05:16.183487+0000","last_became_peered":"2026-02-24T22:05:16.183487+0000","last_unstale":"2026-02-24T22:34:32.123769+0000","last_undegraded":"2026-02-24T22:34:32.123769+0000","last_fullsized":"2026-02-24T22:34:32.123769+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'303","last_scrub_stamp":"2026-02-24T22:34:32.123701+0000","last_deep_scrub":"312'303","last_deep_scrub_stamp":"2026-02-24T22:34:32.123701+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.123701+0000","objects_scrubbed":3,"log_size":103,"log_dups_size":200,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:46:27.988906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":514,"num_read_kb":706,"num_write":1357,"num_write_kb":183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"312'92","reported_seq":760,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.957519+0000","last_change":"2026-02-24T22:34:44.957519+0000","last_active":"2026-02-24T22:34:44.957519+0000","last_peered":"2026-02-24T22:34:44.957519+0000","last_clean":"2026-02-24T22:34:44.957519+0000","last_became_active":"2026-02-24T22:05:18.190352+0000","last_became_peered":"2026-02-24T22:05:18.190352+0000","last_unstale":"2026-02-24T22:34:44.957519+0000","last_undegraded":"2026-02-24T22:34:44.957519+0000","last_fullsized":"2026-02-24T22:34:44.957519+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'92","last_scrub_stamp":"2026-02-24T22:34:44.957460+0000","last_deep_scrub":"312'92","last_deep_scrub_stamp":"2026-02-24T22:34:44.957460+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.957460+0000","objects_scrubbed":20,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:57:33.270079+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":274,"num_read_kb":236,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"312'318","reported_seq":1249,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.115775+0000","last_change":"2026-02-24T22:34:38.115775+0000","last_active":"2026-02-24T22:34:38.115775+0000","last_peered":"2026-02-24T22:34:38.115775+0000","last_clean":"2026-02-24T22:34:38.115775+0000","last_became_active":"2026-02-24T22:05:16.190494+0000","last_became_peered":"2026-02-24T22:05:16.190494+0000","last_unstale":"2026-02-24T22:34:38.115775+0000","last_undegraded":"2026-02-24T22:34:38.115775+0000","last_fullsized":"2026-02-24T22:34:38.115775+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'318","last_scrub_stamp":"2026-02-24T22:34:38.115706+0000","last_deep_scrub":"312'318","last_deep_scrub_stamp":"2026-02-24T22:34:38.115706+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.115706+0000","objects_scrubbed":4,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:08:06.722436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1108,"num_read_kb":14913,"num_write":1331,"num_write_kb":8068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"312'367","reported_seq":1190,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.110977+0000","last_change":"2026-02-24T22:34:41.110977+0000","last_active":"2026-02-24T22:34:41.110977+0000","last_peered":"2026-02-24T22:34:41.110977+0000","last_clean":"2026-02-24T22:34:41.110977+0000","last_became_active":"2026-02-24T22:05:18.186622+0000","last_became_peered":"2026-02-24T22:05:18.186622+0000","last_unstale":"2026-02-24T22:34:41.110977+0000","last_undegraded":"2026-02-24T22:34:41.110977+0000","last_fullsized":"2026-02-24T22:34:41.110977+0000","mapping_epoch":24,"log_start":"312'300","ondisk_log_start":"312'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'367","last_scrub_stamp":"2026-02-24T22:34:41.110934+0000","last_deep_scrub":"312'367","last_deep_scrub_stamp":"2026-02-24T22:34:41.110934+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.110934+0000","objects_scrubbed":30,"log_size":67,"log_dups_size":300,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:59:07.931667+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1512,"num_read_kb":1316,"num_write":481,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25750,"num_omap_keys":101,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"312'319","reported_seq":1167,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.742796+0000","last_change":"2026-02-24T22:34:36.742796+0000","last_active":"2026-02-24T22:34:36.742796+0000","last_peered":"2026-02-24T22:34:36.742796+0000","last_clean":"2026-02-24T22:34:36.742796+0000","last_became_active":"2026-02-24T22:05:16.185977+0000","last_became_peered":"2026-02-24T22:05:16.185977+0000","last_unstale":"2026-02-24T22:34:36.742796+0000","last_undegraded":"2026-02-24T22:34:36.742796+0000","last_fullsized":"2026-02-24T22:34:36.742796+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'319","last_scrub_stamp":"2026-02-24T22:34:36.742708+0000","last_deep_scrub":"312'319","last_deep_scrub_stamp":"2026-02-24T22:34:36.742708+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.742708+0000","objects_scrubbed":1,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:21:32.964403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":948,"num_read_kb":5969,"num_write":1368,"num_write_kb":2943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"312'114","reported_seq":799,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.783698+0000","last_change":"2026-02-24T22:34:46.783698+0000","last_active":"2026-02-24T22:34:46.783698+0000","last_peered":"2026-02-24T22:34:46.783698+0000","last_clean":"2026-02-24T22:34:46.783698+0000","last_became_active":"2026-02-24T22:05:18.185180+0000","last_became_peered":"2026-02-24T22:05:18.185180+0000","last_unstale":"2026-02-24T22:34:46.783698+0000","last_undegraded":"2026-02-24T22:34:46.783698+0000","last_fullsized":"2026-02-24T22:34:46.783698+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'114","last_scrub_stamp":"2026-02-24T22:34:46.783630+0000","last_deep_scrub":"312'114","last_deep_scrub_stamp":"2026-02-24T22:34:46.783630+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.783630+0000","objects_scrubbed":25,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:34:22.929604+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":311,"num_read_kb":264,"num_write":113,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":272,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"312'374","reported_seq":1134,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.080105+0000","last_change":"2026-02-24T22:34:34.080105+0000","last_active":"2026-02-24T22:34:34.080105+0000","last_peered":"2026-02-24T22:34:34.080105+0000","last_clean":"2026-02-24T22:34:34.080105+0000","last_became_active":"2026-02-24T22:06:00.660740+0000","last_became_peered":"2026-02-24T22:06:00.660740+0000","last_unstale":"2026-02-24T22:34:34.080105+0000","last_undegraded":"2026-02-24T22:34:34.080105+0000","last_fullsized":"2026-02-24T22:34:34.080105+0000","mapping_epoch":36,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'374","last_scrub_stamp":"2026-02-24T22:34:34.080040+0000","last_deep_scrub":"312'374","last_deep_scrub_stamp":"2026-02-24T22:34:34.080040+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.080040+0000","objects_scrubbed":4,"log_size":74,"log_dups_size":300,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:17:10.260943+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":666,"num_read_kb":9895,"num_write":1405,"num_write_kb":11975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"312'89","reported_seq":768,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.509973+0000","last_change":"2026-02-24T22:34:43.509973+0000","last_active":"2026-02-24T22:34:43.509973+0000","last_peered":"2026-02-24T22:34:43.509973+0000","last_clean":"2026-02-24T22:34:43.509973+0000","last_became_active":"2026-02-24T22:05:18.190390+0000","last_became_peered":"2026-02-24T22:05:18.190390+0000","last_unstale":"2026-02-24T22:34:43.509973+0000","last_undegraded":"2026-02-24T22:34:43.509973+0000","last_fullsized":"2026-02-24T22:34:43.509973+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'89","last_scrub_stamp":"2026-02-24T22:34:43.509906+0000","last_deep_scrub":"312'89","last_deep_scrub_stamp":"2026-02-24T22:34:43.509906+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.509906+0000","objects_scrubbed":23,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:26:36.232359+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":213,"num_read_kb":187,"num_write":90,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"312'336","reported_seq":1067,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.988612+0000","last_change":"2026-02-24T22:34:34.988612+0000","last_active":"2026-02-24T22:34:34.988612+0000","last_peered":"2026-02-24T22:34:34.988612+0000","last_clean":"2026-02-24T22:34:34.988612+0000","last_became_active":"2026-02-24T22:05:16.184091+0000","last_became_peered":"2026-02-24T22:05:16.184091+0000","last_unstale":"2026-02-24T22:34:34.988612+0000","last_undegraded":"2026-02-24T22:34:34.988612+0000","last_fullsized":"2026-02-24T22:34:34.988612+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'336","last_scrub_stamp":"2026-02-24T22:34:34.988543+0000","last_deep_scrub":"312'336","last_deep_scrub_stamp":"2026-02-24T22:34:34.988543+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.988543+0000","objects_scrubbed":4,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:48:27.381128+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":558,"num_read_kb":406,"num_write":1237,"num_write_kb":6221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"312'375","reported_seq":1187,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.800000+0000","last_change":"2026-02-24T22:34:45.800000+0000","last_active":"2026-02-24T22:34:45.800000+0000","last_peered":"2026-02-24T22:34:45.800000+0000","last_clean":"2026-02-24T22:34:45.800000+0000","last_became_active":"2026-02-24T22:05:18.186746+0000","last_became_peered":"2026-02-24T22:05:18.186746+0000","last_unstale":"2026-02-24T22:34:45.800000+0000","last_undegraded":"2026-02-24T22:34:45.800000+0000","last_fullsized":"2026-02-24T22:34:45.800000+0000","mapping_epoch":24,"log_start":"312'300","ondisk_log_start":"312'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'375","last_scrub_stamp":"2026-02-24T22:34:45.799935+0000","last_deep_scrub":"312'375","last_deep_scrub_stamp":"2026-02-24T22:34:45.799935+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.799935+0000","objects_scrubbed":20,"log_size":75,"log_dups_size":300,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:08:40.366084+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":1676,"num_read_kb":1392,"num_write":638,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"312'331","reported_seq":1043,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.774985+0000","last_change":"2026-02-24T22:34:34.774985+0000","last_active":"2026-02-24T22:34:34.774985+0000","last_peered":"2026-02-24T22:34:34.774985+0000","last_clean":"2026-02-24T22:34:34.774985+0000","last_became_active":"2026-02-24T22:05:16.189516+0000","last_became_peered":"2026-02-24T22:05:16.189516+0000","last_unstale":"2026-02-24T22:34:34.774985+0000","last_undegraded":"2026-02-24T22:34:34.774985+0000","last_fullsized":"2026-02-24T22:34:34.774985+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'331","last_scrub_stamp":"2026-02-24T22:34:34.774895+0000","last_deep_scrub":"312'331","last_deep_scrub_stamp":"2026-02-24T22:34:34.774895+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.774895+0000","objects_scrubbed":2,"log_size":31,"log_dups_size":300,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:51:58.631151+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":448,"num_read_kb":5491,"num_write":1228,"num_write_kb":20654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"312'184","reported_seq":884,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.708693+0000","last_change":"2026-02-24T22:34:40.708693+0000","last_active":"2026-02-24T22:34:40.708693+0000","last_peered":"2026-02-24T22:34:40.708693+0000","last_clean":"2026-02-24T22:34:40.708693+0000","last_became_active":"2026-02-24T22:05:18.190203+0000","last_became_peered":"2026-02-24T22:05:18.190203+0000","last_unstale":"2026-02-24T22:34:40.708693+0000","last_undegraded":"2026-02-24T22:34:40.708693+0000","last_fullsized":"2026-02-24T22:34:40.708693+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'184","last_scrub_stamp":"2026-02-24T22:34:40.708623+0000","last_deep_scrub":"312'184","last_deep_scrub_stamp":"2026-02-24T22:34:40.708623+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.708623+0000","objects_scrubbed":20,"log_size":84,"log_dups_size":100,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:12:32.032756+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":631,"num_read_kb":516,"num_write":248,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"312'319","reported_seq":1046,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.869813+0000","last_change":"2026-02-24T22:34:35.869813+0000","last_active":"2026-02-24T22:34:35.869813+0000","last_peered":"2026-02-24T22:34:35.869813+0000","last_clean":"2026-02-24T22:34:35.869813+0000","last_became_active":"2026-02-24T22:05:16.183772+0000","last_became_peered":"2026-02-24T22:05:16.183772+0000","last_unstale":"2026-02-24T22:34:35.869813+0000","last_undegraded":"2026-02-24T22:34:35.869813+0000","last_fullsized":"2026-02-24T22:34:35.869813+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'319","last_scrub_stamp":"2026-02-24T22:34:35.869749+0000","last_deep_scrub":"312'319","last_deep_scrub_stamp":"2026-02-24T22:34:35.869749+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.869749+0000","objects_scrubbed":2,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:10:38.609121+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":480,"num_read_kb":12489,"num_write":1341,"num_write_kb":5970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"312'156","reported_seq":885,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.880154+0000","last_change":"2026-02-24T22:34:45.880154+0000","last_active":"2026-02-24T22:34:45.880154+0000","last_peered":"2026-02-24T22:34:45.880154+0000","last_clean":"2026-02-24T22:34:45.880154+0000","last_became_active":"2026-02-24T22:05:18.187321+0000","last_became_peered":"2026-02-24T22:05:18.187321+0000","last_unstale":"2026-02-24T22:34:45.880154+0000","last_undegraded":"2026-02-24T22:34:45.880154+0000","last_fullsized":"2026-02-24T22:34:45.880154+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'156","last_scrub_stamp":"2026-02-24T22:34:45.880085+0000","last_deep_scrub":"312'156","last_deep_scrub_stamp":"2026-02-24T22:34:45.880085+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.880085+0000","objects_scrubbed":26,"log_size":56,"log_dups_size":100,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:17:02.414903+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":512,"num_read_kb":441,"num_write":201,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"312'326","reported_seq":1042,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.166643+0000","last_change":"2026-02-24T22:34:37.166643+0000","last_active":"2026-02-24T22:34:37.166643+0000","last_peered":"2026-02-24T22:34:37.166643+0000","last_clean":"2026-02-24T22:34:37.166643+0000","last_became_active":"2026-02-24T22:05:16.193164+0000","last_became_peered":"2026-02-24T22:05:16.193164+0000","last_unstale":"2026-02-24T22:34:37.166643+0000","last_undegraded":"2026-02-24T22:34:37.166643+0000","last_fullsized":"2026-02-24T22:34:37.166643+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'326","last_scrub_stamp":"2026-02-24T22:34:37.166498+0000","last_deep_scrub":"312'326","last_deep_scrub_stamp":"2026-02-24T22:34:37.166498+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.166498+0000","objects_scrubbed":6,"log_size":26,"log_dups_size":300,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:54:53.815454+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242922,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":434,"num_read_kb":36424,"num_write":1378,"num_write_kb":10617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"312'166","reported_seq":852,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.417427+0000","last_change":"2026-02-24T22:34:46.417427+0000","last_active":"2026-02-24T22:34:46.417427+0000","last_peered":"2026-02-24T22:34:46.417427+0000","last_clean":"2026-02-24T22:34:46.417427+0000","last_became_active":"2026-02-24T22:05:18.191737+0000","last_became_peered":"2026-02-24T22:05:18.191737+0000","last_unstale":"2026-02-24T22:34:46.417427+0000","last_undegraded":"2026-02-24T22:34:46.417427+0000","last_fullsized":"2026-02-24T22:34:46.417427+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'166","last_scrub_stamp":"2026-02-24T22:34:46.417398+0000","last_deep_scrub":"312'166","last_deep_scrub_stamp":"2026-02-24T22:34:46.417398+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.417398+0000","objects_scrubbed":18,"log_size":66,"log_dups_size":100,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:30:36.821543+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":539,"num_read_kb":448,"num_write":211,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"312'377","reported_seq":1137,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.766166+0000","last_change":"2026-02-24T22:34:38.766166+0000","last_active":"2026-02-24T22:34:38.766166+0000","last_peered":"2026-02-24T22:34:38.766166+0000","last_clean":"2026-02-24T22:34:38.766166+0000","last_became_active":"2026-02-24T22:05:16.183923+0000","last_became_peered":"2026-02-24T22:05:16.183923+0000","last_unstale":"2026-02-24T22:34:38.766166+0000","last_undegraded":"2026-02-24T22:34:38.766166+0000","last_fullsized":"2026-02-24T22:34:38.766166+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'377","last_scrub_stamp":"2026-02-24T22:34:38.766121+0000","last_deep_scrub":"312'377","last_deep_scrub_stamp":"2026-02-24T22:34:38.766121+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.766121+0000","objects_scrubbed":6,"log_size":77,"log_dups_size":300,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:41:28.025068+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242922,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":549,"num_read_kb":9842,"num_write":1598,"num_write_kb":9073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"312'167","reported_seq":848,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.060961+0000","last_change":"2026-02-24T22:34:39.060923+0000","last_active":"2026-02-24T22:34:39.060961+0000","last_peered":"2026-02-24T22:34:39.060961+0000","last_clean":"2026-02-24T22:34:39.060961+0000","last_became_active":"2026-02-24T22:05:18.183761+0000","last_became_peered":"2026-02-24T22:05:18.183761+0000","last_unstale":"2026-02-24T22:34:39.060961+0000","last_undegraded":"2026-02-24T22:34:39.060961+0000","last_fullsized":"2026-02-24T22:34:39.060961+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'167","last_scrub_stamp":"2026-02-24T22:34:39.060863+0000","last_deep_scrub":"312'167","last_deep_scrub_stamp":"2026-02-24T22:34:39.060863+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.060863+0000","objects_scrubbed":21,"log_size":67,"log_dups_size":100,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:07:32.105462+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":561,"num_read_kb":451,"num_write":221,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"312'341","reported_seq":1119,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.077639+0000","last_change":"2026-02-24T22:34:39.077639+0000","last_active":"2026-02-24T22:34:39.077639+0000","last_peered":"2026-02-24T22:34:39.077639+0000","last_clean":"2026-02-24T22:34:39.077639+0000","last_became_active":"2026-02-24T22:05:16.189759+0000","last_became_peered":"2026-02-24T22:05:16.189759+0000","last_unstale":"2026-02-24T22:34:39.077639+0000","last_undegraded":"2026-02-24T22:34:39.077639+0000","last_fullsized":"2026-02-24T22:34:39.077639+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'341","last_scrub_stamp":"2026-02-24T22:34:39.077571+0000","last_deep_scrub":"312'341","last_deep_scrub_stamp":"2026-02-24T22:34:39.077571+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.077571+0000","objects_scrubbed":4,"log_size":41,"log_dups_size":300,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:09:54.075658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":656,"num_read_kb":3529,"num_write":1503,"num_write_kb":3094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"312'158","reported_seq":860,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.440468+0000","last_change":"2026-02-24T22:34:45.440468+0000","last_active":"2026-02-24T22:34:45.440468+0000","last_peered":"2026-02-24T22:34:45.440468+0000","last_clean":"2026-02-24T22:34:45.440468+0000","last_became_active":"2026-02-24T22:05:18.192882+0000","last_became_peered":"2026-02-24T22:05:18.192882+0000","last_unstale":"2026-02-24T22:34:45.440468+0000","last_undegraded":"2026-02-24T22:34:45.440468+0000","last_fullsized":"2026-02-24T22:34:45.440468+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'158","last_scrub_stamp":"2026-02-24T22:34:45.440425+0000","last_deep_scrub":"312'158","last_deep_scrub_stamp":"2026-02-24T22:34:45.440425+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.440425+0000","objects_scrubbed":18,"log_size":58,"log_dups_size":100,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:49:26.113842+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":593,"num_read_kb":494,"num_write":208,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"312'309","reported_seq":1056,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.732158+0000","last_change":"2026-02-24T22:34:35.732158+0000","last_active":"2026-02-24T22:34:35.732158+0000","last_peered":"2026-02-24T22:34:35.732158+0000","last_clean":"2026-02-24T22:34:35.732158+0000","last_became_active":"2026-02-24T22:05:16.184557+0000","last_became_peered":"2026-02-24T22:05:16.184557+0000","last_unstale":"2026-02-24T22:34:35.732158+0000","last_undegraded":"2026-02-24T22:34:35.732158+0000","last_fullsized":"2026-02-24T22:34:35.732158+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'309","last_scrub_stamp":"2026-02-24T22:34:35.732082+0000","last_deep_scrub":"312'309","last_deep_scrub_stamp":"2026-02-24T22:34:35.732082+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.732082+0000","objects_scrubbed":3,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:22:57.505362+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":498,"num_read_kb":1536,"num_write":1259,"num_write_kb":5590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"312'184","reported_seq":901,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.156109+0000","last_change":"2026-02-24T22:34:45.156109+0000","last_active":"2026-02-24T22:34:45.156109+0000","last_peered":"2026-02-24T22:34:45.156109+0000","last_clean":"2026-02-24T22:34:45.156109+0000","last_became_active":"2026-02-24T22:05:18.191432+0000","last_became_peered":"2026-02-24T22:05:18.191432+0000","last_unstale":"2026-02-24T22:34:45.156109+0000","last_undegraded":"2026-02-24T22:34:45.156109+0000","last_fullsized":"2026-02-24T22:34:45.156109+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'184","last_scrub_stamp":"2026-02-24T22:34:45.156041+0000","last_deep_scrub":"312'184","last_deep_scrub_stamp":"2026-02-24T22:34:45.156041+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.156041+0000","objects_scrubbed":26,"log_size":84,"log_dups_size":100,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:54:34.074113+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":649,"num_read_kb":535,"num_write":233,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4005,"num_omap_keys":18,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"312'287","reported_seq":1017,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.765441+0000","last_change":"2026-02-24T22:34:36.765441+0000","last_active":"2026-02-24T22:34:36.765441+0000","last_peered":"2026-02-24T22:34:36.765441+0000","last_clean":"2026-02-24T22:34:36.765441+0000","last_became_active":"2026-02-24T22:05:16.190260+0000","last_became_peered":"2026-02-24T22:05:16.190260+0000","last_unstale":"2026-02-24T22:34:36.765441+0000","last_undegraded":"2026-02-24T22:34:36.765441+0000","last_fullsized":"2026-02-24T22:34:36.765441+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'287","last_scrub_stamp":"2026-02-24T22:34:36.765378+0000","last_deep_scrub":"312'287","last_deep_scrub_stamp":"2026-02-24T22:34:36.765378+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.765378+0000","objects_scrubbed":2,"log_size":87,"log_dups_size":200,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:18:33.840972+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":458,"num_read_kb":1504,"num_write":1357,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"312'171","reported_seq":869,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.861467+0000","last_change":"2026-02-24T22:34:44.861467+0000","last_active":"2026-02-24T22:34:44.861467+0000","last_peered":"2026-02-24T22:34:44.861467+0000","last_clean":"2026-02-24T22:34:44.861467+0000","last_became_active":"2026-02-24T22:05:18.185184+0000","last_became_peered":"2026-02-24T22:05:18.185184+0000","last_unstale":"2026-02-24T22:34:44.861467+0000","last_undegraded":"2026-02-24T22:34:44.861467+0000","last_fullsized":"2026-02-24T22:34:44.861467+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'171","last_scrub_stamp":"2026-02-24T22:34:44.861426+0000","last_deep_scrub":"312'171","last_deep_scrub_stamp":"2026-02-24T22:34:44.861426+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.861426+0000","objects_scrubbed":28,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:45:55.477850+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":536,"num_read_kb":445,"num_write":200,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"312'302","reported_seq":1124,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.033851+0000","last_change":"2026-02-24T22:34:40.033851+0000","last_active":"2026-02-24T22:34:40.033851+0000","last_peered":"2026-02-24T22:34:40.033851+0000","last_clean":"2026-02-24T22:34:40.033851+0000","last_became_active":"2026-02-24T22:05:16.191433+0000","last_became_peered":"2026-02-24T22:05:16.191433+0000","last_unstale":"2026-02-24T22:34:40.033851+0000","last_undegraded":"2026-02-24T22:34:40.033851+0000","last_fullsized":"2026-02-24T22:34:40.033851+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'302","last_scrub_stamp":"2026-02-24T22:34:40.033782+0000","last_deep_scrub":"312'302","last_deep_scrub_stamp":"2026-02-24T22:34:40.033782+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.033782+0000","objects_scrubbed":3,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:16:57.803888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":549,"num_read_kb":2264,"num_write":1330,"num_write_kb":7006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"312'86","reported_seq":755,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.772530+0000","last_change":"2026-02-24T22:34:48.772530+0000","last_active":"2026-02-24T22:34:48.772530+0000","last_peered":"2026-02-24T22:34:48.772530+0000","last_clean":"2026-02-24T22:34:48.772530+0000","last_became_active":"2026-02-24T22:05:18.192619+0000","last_became_peered":"2026-02-24T22:05:18.192619+0000","last_unstale":"2026-02-24T22:34:48.772530+0000","last_undegraded":"2026-02-24T22:34:48.772530+0000","last_fullsized":"2026-02-24T22:34:48.772530+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'86","last_scrub_stamp":"2026-02-24T22:34:48.772479+0000","last_deep_scrub":"312'86","last_deep_scrub_stamp":"2026-02-24T22:34:48.772479+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.772479+0000","objects_scrubbed":24,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:50:44.256005+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":204,"num_read_kb":176,"num_write":76,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"312'336","reported_seq":1076,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.091228+0000","last_change":"2026-02-24T22:34:33.091228+0000","last_active":"2026-02-24T22:34:33.091228+0000","last_peered":"2026-02-24T22:34:33.091228+0000","last_clean":"2026-02-24T22:34:33.091228+0000","last_became_active":"2026-02-24T22:05:16.184441+0000","last_became_peered":"2026-02-24T22:05:16.184441+0000","last_unstale":"2026-02-24T22:34:33.091228+0000","last_undegraded":"2026-02-24T22:34:33.091228+0000","last_fullsized":"2026-02-24T22:34:33.091228+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'336","last_scrub_stamp":"2026-02-24T22:34:33.091160+0000","last_deep_scrub":"312'336","last_deep_scrub_stamp":"2026-02-24T22:34:33.091160+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.091160+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:19:40.057693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":478,"num_read_kb":3856,"num_write":1468,"num_write_kb":8445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"312'183","reported_seq":909,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.176877+0000","last_change":"2026-02-24T22:34:48.176877+0000","last_active":"2026-02-24T22:34:48.176877+0000","last_peered":"2026-02-24T22:34:48.176877+0000","last_clean":"2026-02-24T22:34:48.176877+0000","last_became_active":"2026-02-24T22:05:18.191350+0000","last_became_peered":"2026-02-24T22:05:18.191350+0000","last_unstale":"2026-02-24T22:34:48.176877+0000","last_undegraded":"2026-02-24T22:34:48.176877+0000","last_fullsized":"2026-02-24T22:34:48.176877+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'183","last_scrub_stamp":"2026-02-24T22:34:48.176839+0000","last_deep_scrub":"312'183","last_deep_scrub_stamp":"2026-02-24T22:34:48.176839+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.176839+0000","objects_scrubbed":24,"log_size":83,"log_dups_size":100,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:23:04.478635+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":662,"num_read_kb":557,"num_write":245,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"312'305","reported_seq":1081,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.042678+0000","last_change":"2026-02-24T22:34:35.042678+0000","last_active":"2026-02-24T22:34:35.042678+0000","last_peered":"2026-02-24T22:34:35.042678+0000","last_clean":"2026-02-24T22:34:35.042678+0000","last_became_active":"2026-02-24T22:06:00.662656+0000","last_became_peered":"2026-02-24T22:06:00.662656+0000","last_unstale":"2026-02-24T22:34:35.042678+0000","last_undegraded":"2026-02-24T22:34:35.042678+0000","last_fullsized":"2026-02-24T22:34:35.042678+0000","mapping_epoch":36,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'305","last_scrub_stamp":"2026-02-24T22:34:35.042539+0000","last_deep_scrub":"312'305","last_deep_scrub_stamp":"2026-02-24T22:34:35.042539+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.042539+0000","objects_scrubbed":4,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:24:23.913211+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":803,"num_read_kb":6090,"num_write":1203,"num_write_kb":5556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.30","version":"312'347","reported_seq":1107,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.049845+0000","last_change":"2026-02-24T22:34:40.049845+0000","last_active":"2026-02-24T22:34:40.049845+0000","last_peered":"2026-02-24T22:34:40.049845+0000","last_clean":"2026-02-24T22:34:40.049845+0000","last_became_active":"2026-02-24T22:05:16.189713+0000","last_became_peered":"2026-02-24T22:05:16.189713+0000","last_unstale":"2026-02-24T22:34:40.049845+0000","last_undegraded":"2026-02-24T22:34:40.049845+0000","last_fullsized":"2026-02-24T22:34:40.049845+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'347","last_scrub_stamp":"2026-02-24T22:34:40.049764+0000","last_deep_scrub":"312'347","last_deep_scrub_stamp":"2026-02-24T22:34:40.049764+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.049764+0000","objects_scrubbed":8,"log_size":47,"log_dups_size":300,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:52:46.777711+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242932,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":632,"num_read_kb":31410,"num_write":1455,"num_write_kb":10638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"312'273","reported_seq":967,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.873634+0000","last_change":"2026-02-24T22:34:46.873634+0000","last_active":"2026-02-24T22:34:46.873634+0000","last_peered":"2026-02-24T22:34:46.873634+0000","last_clean":"2026-02-24T22:34:46.873634+0000","last_became_active":"2026-02-24T22:05:18.187565+0000","last_became_peered":"2026-02-24T22:05:18.187565+0000","last_unstale":"2026-02-24T22:34:46.873634+0000","last_undegraded":"2026-02-24T22:34:46.873634+0000","last_fullsized":"2026-02-24T22:34:46.873634+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'273","last_scrub_stamp":"2026-02-24T22:34:46.873575+0000","last_deep_scrub":"312'273","last_deep_scrub_stamp":"2026-02-24T22:34:46.873575+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.873575+0000","objects_scrubbed":17,"log_size":73,"log_dups_size":200,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:03:00.743450+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":1060,"num_read_kb":853,"num_write":419,"num_write_kb":222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"312'299","reported_seq":1013,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.068984+0000","last_change":"2026-02-24T22:34:37.068984+0000","last_active":"2026-02-24T22:34:37.068984+0000","last_peered":"2026-02-24T22:34:37.068984+0000","last_clean":"2026-02-24T22:34:37.068984+0000","last_became_active":"2026-02-24T22:05:16.178157+0000","last_became_peered":"2026-02-24T22:05:16.178157+0000","last_unstale":"2026-02-24T22:34:37.068984+0000","last_undegraded":"2026-02-24T22:34:37.068984+0000","last_fullsized":"2026-02-24T22:34:37.068984+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'299","last_scrub_stamp":"2026-02-24T22:34:37.068917+0000","last_deep_scrub":"312'299","last_deep_scrub_stamp":"2026-02-24T22:34:37.068917+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.068917+0000","objects_scrubbed":5,"log_size":99,"log_dups_size":200,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:17:09.219950+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":466,"num_read_kb":1640,"num_write":1134,"num_write_kb":1702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"312'98","reported_seq":781,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.871267+0000","last_change":"2026-02-24T22:34:47.871267+0000","last_active":"2026-02-24T22:34:47.871267+0000","last_peered":"2026-02-24T22:34:47.871267+0000","last_clean":"2026-02-24T22:34:47.871267+0000","last_became_active":"2026-02-24T22:05:18.190902+0000","last_became_peered":"2026-02-24T22:05:18.190902+0000","last_unstale":"2026-02-24T22:34:47.871267+0000","last_undegraded":"2026-02-24T22:34:47.871267+0000","last_fullsized":"2026-02-24T22:34:47.871267+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'98","last_scrub_stamp":"2026-02-24T22:34:47.871207+0000","last_deep_scrub":"312'98","last_deep_scrub_stamp":"2026-02-24T22:34:47.871207+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.871207+0000","objects_scrubbed":20,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:26:15.293776+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":302,"num_read_kb":262,"num_write":98,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":433,"ondisk_log_size":433,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":973109,"num_objects":3053,"num_object_clones":0,"num_object_copies":6106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3053,"num_whiteouts":0,"num_read":19978,"num_read_kb":245567,"num_write":23762,"num_write_kb":8619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":276106,"num_omap_keys":1511,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24911872,"data_stored":1946218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5786661,"internal_metadata":0},"log_size":2158,"ondisk_log_size":2158,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1250269,"num_objects":223,"num_object_clones":0,"num_object_copies":446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":223,"num_whiteouts":0,"num_read":45268,"num_read_kb":47987,"num_write":15451,"num_write_kb":13662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":962652,"num_omap_keys":4046,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":17220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2823344,"internal_metadata":0},"log_size":2038,"ondisk_log_size":2038,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1896,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1448,"num_read_kb":1680,"num_write":22,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":64,"acting":64,"num_store_stats":8},{"poolid":82,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1128,"num_omap_keys":12,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11005,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":1511,"num_object_clones":0,"num_object_copies":3022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1511,"num_whiteouts":0,"num_read":95919,"num_read_kb":81897,"num_write":34645,"num_write_kb":17078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1511,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":35257,"num_omap_keys":142,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17301930,"internal_metadata":0},"log_size":4333,"ondisk_log_size":4333,"up":128,"acting":128,"num_store_stats":8},{"poolid":81,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":119816,"num_write":82,"num_write_kb":70656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":112199743,"num_objects":281,"num_object_clones":0,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":0,"num_read":41102,"num_read_kb":763537,"num_write":86142,"num_write_kb":523059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":225296384,"data_stored":224399486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4246,"ondisk_log_size":4246,"up":128,"acting":128,"num_store_stats":8},{"poolid":80,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":124936,"num_write":82,"num_write_kb":70656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":7},{"poolid":79,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3567,"num_read_kb":2401,"num_write":1127,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":317720,"internal_metadata":0},"log_size":462,"ondisk_log_size":462,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14,"ondisk_log_size":14,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575215,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316704,"kb_used_data":55120,"kb_used_omap":3696,"kb_used_meta":257871,"kb_avail":94055136,"statfs":{"total":96636764160,"available":96312459264,"internally_reserved":0,"allocated":56442880,"data_stored":51634967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3785637,"internal_metadata":264059995},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575215,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":271964,"kb_used_data":28428,"kb_used_omap":3571,"kb_used_meta":239948,"kb_avail":94099876,"statfs":{"total":96636764160,"available":96358273024,"internally_reserved":0,"allocated":29110272,"data_stored":23825325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3657488,"internal_metadata":245706992},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575215,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137044,"kb_used_data":37956,"kb_used_omap":2746,"kb_used_meta":96325,"kb_avail":94234796,"statfs":{"total":96636764160,"available":96496431104,"internally_reserved":0,"allocated":38866944,"data_stored":35212820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2812671,"internal_metadata":98637057},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575215,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":111224,"kb_used_data":48168,"kb_used_omap":3095,"kb_used_meta":59944,"kb_avail":94260616,"statfs":{"total":96636764160,"available":96522870784,"internally_reserved":0,"allocated":49324032,"data_stored":45700457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3169290,"internal_metadata":61383670},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575216,"num_pgs":107,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":225820,"kb_used_data":18380,"kb_used_omap":7232,"kb_used_meta":200191,"kb_avail":94146020,"statfs":{"total":96636764160,"available":96405524480,"internally_reserved":0,"allocated":18821120,"data_stored":13350111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7406186,"internal_metadata":204995990},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575216,"num_pgs":87,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80488,"kb_used_data":17432,"kb_used_omap":1599,"kb_used_meta":61440,"kb_avail":94291352,"statfs":{"total":96636764160,"available":96554344448,"internally_reserved":0,"allocated":17850368,"data_stored":13280880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1638118,"internal_metadata":62914842},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575217,"num_pgs":102,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":129624,"kb_used_data":12424,"kb_used_omap":5094,"kb_used_meta":112089,"kb_avail":94242216,"statfs":{"total":96636764160,"available":96504029184,"internally_reserved":0,"allocated":12722176,"data_stored":8040949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5216565,"internal_metadata":114779851},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575217,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304420,"kb_used_data":60948,"kb_used_omap":10223,"kb_used_meta":233232,"kb_avail":94067420,"statfs":{"total":96636764160,"available":96325038080,"internally_reserved":0,"allocated":62410752,"data_stored":56934149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10468372,"internal_metadata":238830572},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53624832,"data_stored":53478090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5324800,"data_stored":5243270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":10555392,"data_stored":10486056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10620928,"data_stored":10486391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":43110400,"data_stored":42992165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":32641024,"data_stored":32506526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":21032960,"data_stored":20971830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":48386048,"data_stored":48235158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1667351,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2633902,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1516609,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1844562,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2430346,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":723382,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3546560,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2939218,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":2106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1512,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1072852,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":3356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18545,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":2058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5184,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1705285,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11530,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":2244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8436,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3928064,"data_stored":308322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51420,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3117056,"data_stored":241195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":152550,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3059712,"data_stored":239505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":825183,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3899392,"data_stored":305873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":336934,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1949696,"data_stored":151651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7338,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1925120,"data_stored":148450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":65762,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3837952,"data_stored":298046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2489145,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3194880,"data_stored":253176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1858329,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":79,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":133235,"internal_metadata":0},{"poolid":79,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14290,"internal_metadata":0},{"poolid":79,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23118,"internal_metadata":0},{"poolid":79,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15635,"internal_metadata":0},{"poolid":79,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49631,"internal_metadata":0},{"poolid":79,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49251,"internal_metadata":0},{"poolid":79,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16656,"internal_metadata":0},{"poolid":79,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15904,"internal_metadata":0},{"poolid":80,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":82,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":82,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1886,"internal_metadata":0},{"poolid":82,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6440,"internal_metadata":0},{"poolid":82,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":262,"internal_metadata":0},{"poolid":82,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1265,"internal_metadata":0},{"poolid":82,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":717,"internal_metadata":0},{"poolid":82,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":435,"internal_metadata":0},{"poolid":82,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T22:35:31.658 INFO:tasks.ceph:pgid 82.4 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:31.659 INFO:tasks.ceph:pgid 82.10 last_scrub_stamp 2026-02-24T22:23:28.200283+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=23, tm_sec=28, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=22, tm_min=34, tm_sec=27, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T22:35:31.661 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-24T22:35:51.662 DEBUG:teuthology.orchestra.run.trial046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-24T22:35:51.841 INFO:teuthology.orchestra.run.trial046.stdout: 2026-02-24T22:35:51.858 INFO:teuthology.orchestra.run.trial046.stderr:dumped all 2026-02-24T22:35:51.860 INFO:teuthology.orchestra.run.trial046.stdout:{"pg_ready":true,"pg_map":{"version":1228,"stamp":"2026-02-24T22:35:50.044130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":115015404,"num_objects":5093,"num_object_clones":0,"num_object_copies":10186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5093,"num_whiteouts":0,"num_read":207562,"num_read_kb":1387906,"num_write":161478,"num_write_kb":706745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":1543,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1275143,"num_omap_keys":5711,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13823,"ondisk_log_size":13823,"up":786,"acting":786,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":786,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1577288,"kb_used_data":278856,"kb_used_omap":37260,"kb_used_meta":1261043,"kb_avail":753397432,"statfs":{"total":773094113280,"available":771478970368,"internally_reserved":0,"allocated":285548544,"data_stored":247979658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38154327,"internal_metadata":1291308969},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002506"},"pg_stats":[{"pgid":"82.2","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:20.609504+0000","last_change":"2026-02-24T22:35:20.609504+0000","last_active":"2026-02-24T22:35:20.609504+0000","last_peered":"2026-02-24T22:35:20.609504+0000","last_clean":"2026-02-24T22:35:20.609504+0000","last_became_active":"2026-02-24T22:23:29.198476+0000","last_became_peered":"2026-02-24T22:23:29.198476+0000","last_unstale":"2026-02-24T22:35:20.609504+0000","last_undegraded":"2026-02-24T22:35:20.609504+0000","last_fullsized":"2026-02-24T22:35:20.609504+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:20.609434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:20.609434+0000","last_clean_scrub_stamp":"2026-02-24T22:35:20.609434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:14:17.280111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.1","version":"0'0","reported_seq":40,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:07.404528+0000","last_change":"2026-02-24T22:35:07.404528+0000","last_active":"2026-02-24T22:35:07.404528+0000","last_peered":"2026-02-24T22:35:07.404528+0000","last_clean":"2026-02-24T22:35:07.404528+0000","last_became_active":"2026-02-24T22:15:25.830000+0000","last_became_peered":"2026-02-24T22:15:25.830000+0000","last_unstale":"2026-02-24T22:35:07.404528+0000","last_undegraded":"2026-02-24T22:35:07.404528+0000","last_fullsized":"2026-02-24T22:35:07.404528+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:07.404472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:07.404472+0000","last_clean_scrub_stamp":"2026-02-24T22:35:07.404472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:30:10.446904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.0","version":"0'0","reported_seq":47,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:54.176050+0000","last_change":"2026-02-24T22:34:54.176012+0000","last_active":"2026-02-24T22:34:54.176050+0000","last_peered":"2026-02-24T22:34:54.176050+0000","last_clean":"2026-02-24T22:34:54.176050+0000","last_became_active":"2026-02-24T22:15:22.817446+0000","last_became_peered":"2026-02-24T22:15:22.817446+0000","last_unstale":"2026-02-24T22:34:54.176050+0000","last_undegraded":"2026-02-24T22:34:54.176050+0000","last_fullsized":"2026-02-24T22:34:54.176050+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:54.175956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:54.175956+0000","last_clean_scrub_stamp":"2026-02-24T22:34:54.175956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:40:14.585196+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.1f","version":"312'20","reported_seq":129,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:11.720387+0000","last_change":"2026-02-24T22:35:11.720387+0000","last_active":"2026-02-24T22:35:11.720387+0000","last_peered":"2026-02-24T22:35:11.720387+0000","last_clean":"2026-02-24T22:35:11.720387+0000","last_became_active":"2026-02-24T22:13:43.761120+0000","last_became_peered":"2026-02-24T22:13:43.761120+0000","last_unstale":"2026-02-24T22:35:11.720387+0000","last_undegraded":"2026-02-24T22:35:11.720387+0000","last_fullsized":"2026-02-24T22:35:11.720387+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'20","last_scrub_stamp":"2026-02-24T22:35:11.720314+0000","last_deep_scrub":"312'20","last_deep_scrub_stamp":"2026-02-24T22:35:11.720314+0000","last_clean_scrub_stamp":"2026-02-24T22:35:11.720314+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:22:06.487293+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":168,"num_read_kb":114,"num_write":47,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.3","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:24.891603+0000","last_change":"2026-02-24T22:35:24.891603+0000","last_active":"2026-02-24T22:35:24.891603+0000","last_peered":"2026-02-24T22:35:24.891603+0000","last_clean":"2026-02-24T22:35:24.891603+0000","last_became_active":"2026-02-24T22:23:29.213534+0000","last_became_peered":"2026-02-24T22:23:29.213534+0000","last_unstale":"2026-02-24T22:35:24.891603+0000","last_undegraded":"2026-02-24T22:35:24.891603+0000","last_fullsized":"2026-02-24T22:35:24.891603+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:24.891536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:24.891536+0000","last_clean_scrub_stamp":"2026-02-24T22:35:24.891536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:37:51.239872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.0","version":"0'0","reported_seq":41,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:11.645270+0000","last_change":"2026-02-24T22:35:11.645200+0000","last_active":"2026-02-24T22:35:11.645270+0000","last_peered":"2026-02-24T22:35:11.645270+0000","last_clean":"2026-02-24T22:35:11.645270+0000","last_became_active":"2026-02-24T22:15:25.827355+0000","last_became_peered":"2026-02-24T22:15:25.827355+0000","last_unstale":"2026-02-24T22:35:11.645270+0000","last_undegraded":"2026-02-24T22:35:11.645270+0000","last_fullsized":"2026-02-24T22:35:11.645270+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:11.645114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:11.645114+0000","last_clean_scrub_stamp":"2026-02-24T22:35:11.645114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:37:43.905579+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.1","version":"0'0","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:14.776069+0000","last_change":"2026-02-24T22:35:14.776069+0000","last_active":"2026-02-24T22:35:14.776069+0000","last_peered":"2026-02-24T22:35:14.776069+0000","last_clean":"2026-02-24T22:35:14.776069+0000","last_became_active":"2026-02-24T22:15:22.818519+0000","last_became_peered":"2026-02-24T22:15:22.818519+0000","last_unstale":"2026-02-24T22:35:14.776069+0000","last_undegraded":"2026-02-24T22:35:14.776069+0000","last_fullsized":"2026-02-24T22:35:14.776069+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:14.776012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:14.776012+0000","last_clean_scrub_stamp":"2026-02-24T22:35:14.776012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:10:43.383714+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.1e","version":"312'9","reported_seq":97,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:11.351614+0000","last_change":"2026-02-24T22:35:11.351538+0000","last_active":"2026-02-24T22:35:11.351614+0000","last_peered":"2026-02-24T22:35:11.351614+0000","last_clean":"2026-02-24T22:35:11.351614+0000","last_became_active":"2026-02-24T22:13:43.757543+0000","last_became_peered":"2026-02-24T22:13:43.757543+0000","last_unstale":"2026-02-24T22:35:11.351614+0000","last_undegraded":"2026-02-24T22:35:11.351614+0000","last_fullsized":"2026-02-24T22:35:11.351614+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'9","last_scrub_stamp":"2026-02-24T22:35:11.351464+0000","last_deep_scrub":"312'9","last_deep_scrub_stamp":"2026-02-24T22:35:11.351464+0000","last_clean_scrub_stamp":"2026-02-24T22:35:11.351464+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:32:59.791152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":38,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.0","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:04.282861+0000","last_change":"2026-02-24T22:35:04.282796+0000","last_active":"2026-02-24T22:35:04.282861+0000","last_peered":"2026-02-24T22:35:04.282861+0000","last_clean":"2026-02-24T22:35:04.282861+0000","last_became_active":"2026-02-24T22:23:29.212043+0000","last_became_peered":"2026-02-24T22:23:29.212043+0000","last_unstale":"2026-02-24T22:35:04.282861+0000","last_undegraded":"2026-02-24T22:35:04.282861+0000","last_fullsized":"2026-02-24T22:35:04.282861+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:04.282728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:04.282728+0000","last_clean_scrub_stamp":"2026-02-24T22:35:04.282728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:05:28.411607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.3","version":"312'8","reported_seq":61,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:18.304114+0000","last_change":"2026-02-24T22:35:18.304114+0000","last_active":"2026-02-24T22:35:18.304114+0000","last_peered":"2026-02-24T22:35:18.304114+0000","last_clean":"2026-02-24T22:35:18.304114+0000","last_became_active":"2026-02-24T22:15:25.827216+0000","last_became_peered":"2026-02-24T22:15:25.827216+0000","last_unstale":"2026-02-24T22:35:18.304114+0000","last_undegraded":"2026-02-24T22:35:18.304114+0000","last_fullsized":"2026-02-24T22:35:18.304114+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'8","last_scrub_stamp":"2026-02-24T22:35:18.304050+0000","last_deep_scrub":"312'8","last_deep_scrub_stamp":"2026-02-24T22:35:18.304050+0000","last_clean_scrub_stamp":"2026-02-24T22:35:18.304050+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:14:27.296316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":25602,"num_write":13,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.2","version":"0'0","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:55.187191+0000","last_change":"2026-02-24T22:34:55.187191+0000","last_active":"2026-02-24T22:34:55.187191+0000","last_peered":"2026-02-24T22:34:55.187191+0000","last_clean":"2026-02-24T22:34:55.187191+0000","last_became_active":"2026-02-24T22:15:22.818194+0000","last_became_peered":"2026-02-24T22:15:22.818194+0000","last_unstale":"2026-02-24T22:34:55.187191+0000","last_undegraded":"2026-02-24T22:34:55.187191+0000","last_fullsized":"2026-02-24T22:34:55.187191+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:55.187119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:55.187119+0000","last_clean_scrub_stamp":"2026-02-24T22:34:55.187119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:26:30.922986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.1d","version":"312'22","reported_seq":129,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:00.724505+0000","last_change":"2026-02-24T22:35:00.724505+0000","last_active":"2026-02-24T22:35:00.724505+0000","last_peered":"2026-02-24T22:35:00.724505+0000","last_clean":"2026-02-24T22:35:00.724505+0000","last_became_active":"2026-02-24T22:13:43.757278+0000","last_became_peered":"2026-02-24T22:13:43.757278+0000","last_unstale":"2026-02-24T22:35:00.724505+0000","last_undegraded":"2026-02-24T22:35:00.724505+0000","last_fullsized":"2026-02-24T22:35:00.724505+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'22","last_scrub_stamp":"2026-02-24T22:35:00.724443+0000","last_deep_scrub":"312'22","last_deep_scrub_stamp":"2026-02-24T22:35:00.724443+0000","last_clean_scrub_stamp":"2026-02-24T22:35:00.724443+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:16:45.929416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":103,"num_write":54,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.1","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:26.839718+0000","last_change":"2026-02-24T22:35:26.839718+0000","last_active":"2026-02-24T22:35:26.839718+0000","last_peered":"2026-02-24T22:35:26.839718+0000","last_clean":"2026-02-24T22:35:26.839718+0000","last_became_active":"2026-02-24T22:23:29.214064+0000","last_became_peered":"2026-02-24T22:23:29.214064+0000","last_unstale":"2026-02-24T22:35:26.839718+0000","last_undegraded":"2026-02-24T22:35:26.839718+0000","last_fullsized":"2026-02-24T22:35:26.839718+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:26.839650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:26.839650+0000","last_clean_scrub_stamp":"2026-02-24T22:35:26.839650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:46:27.127761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.2","version":"312'2","reported_seq":44,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:11.543860+0000","last_change":"2026-02-24T22:35:11.543860+0000","last_active":"2026-02-24T22:35:11.543860+0000","last_peered":"2026-02-24T22:35:11.543860+0000","last_clean":"2026-02-24T22:35:11.543860+0000","last_became_active":"2026-02-24T22:15:25.832809+0000","last_became_peered":"2026-02-24T22:15:25.832809+0000","last_unstale":"2026-02-24T22:35:11.543860+0000","last_undegraded":"2026-02-24T22:35:11.543860+0000","last_fullsized":"2026-02-24T22:35:11.543860+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:35:11.543798+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:35:11.543798+0000","last_clean_scrub_stamp":"2026-02-24T22:35:11.543798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:28:14.037060+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"80.3","version":"312'3","reported_seq":52,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:56.183409+0000","last_change":"2026-02-24T22:34:56.183409+0000","last_active":"2026-02-24T22:34:56.183409+0000","last_peered":"2026-02-24T22:34:56.183409+0000","last_clean":"2026-02-24T22:34:56.183409+0000","last_became_active":"2026-02-24T22:15:22.818089+0000","last_became_peered":"2026-02-24T22:15:22.818089+0000","last_unstale":"2026-02-24T22:34:56.183409+0000","last_undegraded":"2026-02-24T22:34:56.183409+0000","last_fullsized":"2026-02-24T22:34:56.183409+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:34:56.183350+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:34:56.183350+0000","last_clean_scrub_stamp":"2026-02-24T22:34:56.183350+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:06:53.926961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.1c","version":"312'14","reported_seq":109,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:15.806073+0000","last_change":"2026-02-24T22:35:15.806073+0000","last_active":"2026-02-24T22:35:15.806073+0000","last_peered":"2026-02-24T22:35:15.806073+0000","last_clean":"2026-02-24T22:35:15.806073+0000","last_became_active":"2026-02-24T22:13:43.759958+0000","last_became_peered":"2026-02-24T22:13:43.759958+0000","last_unstale":"2026-02-24T22:35:15.806073+0000","last_undegraded":"2026-02-24T22:35:15.806073+0000","last_fullsized":"2026-02-24T22:35:15.806073+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'14","last_scrub_stamp":"2026-02-24T22:35:15.806010+0000","last_deep_scrub":"312'14","last_deep_scrub_stamp":"2026-02-24T22:35:15.806010+0000","last_clean_scrub_stamp":"2026-02-24T22:35:15.806010+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:05:06.336286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":75,"num_write":36,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.6","version":"312'2","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:11.309552+0000","last_change":"2026-02-24T22:35:11.309552+0000","last_active":"2026-02-24T22:35:11.309552+0000","last_peered":"2026-02-24T22:35:11.309552+0000","last_clean":"2026-02-24T22:35:11.309552+0000","last_became_active":"2026-02-24T22:23:29.215327+0000","last_became_peered":"2026-02-24T22:23:29.215327+0000","last_unstale":"2026-02-24T22:35:11.309552+0000","last_undegraded":"2026-02-24T22:35:11.309552+0000","last_fullsized":"2026-02-24T22:35:11.309552+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:35:11.309512+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:35:11.309512+0000","last_clean_scrub_stamp":"2026-02-24T22:35:11.309512+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:15:39.214444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.5","version":"0'0","reported_seq":40,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:25.847303+0000","last_change":"2026-02-24T22:35:25.847303+0000","last_active":"2026-02-24T22:35:25.847303+0000","last_peered":"2026-02-24T22:35:25.847303+0000","last_clean":"2026-02-24T22:35:25.847303+0000","last_became_active":"2026-02-24T22:15:25.831975+0000","last_became_peered":"2026-02-24T22:15:25.831975+0000","last_unstale":"2026-02-24T22:35:25.847303+0000","last_undegraded":"2026-02-24T22:35:25.847303+0000","last_fullsized":"2026-02-24T22:35:25.847303+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:25.847227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:25.847227+0000","last_clean_scrub_stamp":"2026-02-24T22:35:25.847227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:59:07.272621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"80.4","version":"312'3","reported_seq":52,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:12.393384+0000","last_change":"2026-02-24T22:35:12.393384+0000","last_active":"2026-02-24T22:35:12.393384+0000","last_peered":"2026-02-24T22:35:12.393384+0000","last_clean":"2026-02-24T22:35:12.393384+0000","last_became_active":"2026-02-24T22:15:22.812270+0000","last_became_peered":"2026-02-24T22:15:22.812270+0000","last_unstale":"2026-02-24T22:35:12.393384+0000","last_undegraded":"2026-02-24T22:35:12.393384+0000","last_fullsized":"2026-02-24T22:35:12.393384+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:12.393333+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:12.393333+0000","last_clean_scrub_stamp":"2026-02-24T22:35:12.393333+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:05:26.291517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.1b","version":"312'18","reported_seq":120,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:13.375416+0000","last_change":"2026-02-24T22:35:13.375416+0000","last_active":"2026-02-24T22:35:13.375416+0000","last_peered":"2026-02-24T22:35:13.375416+0000","last_clean":"2026-02-24T22:35:13.375416+0000","last_became_active":"2026-02-24T22:13:43.755170+0000","last_became_peered":"2026-02-24T22:13:43.755170+0000","last_unstale":"2026-02-24T22:35:13.375416+0000","last_undegraded":"2026-02-24T22:35:13.375416+0000","last_fullsized":"2026-02-24T22:35:13.375416+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'18","last_scrub_stamp":"2026-02-24T22:35:13.375362+0000","last_deep_scrub":"312'18","last_deep_scrub_stamp":"2026-02-24T22:35:13.375362+0000","last_clean_scrub_stamp":"2026-02-24T22:35:13.375362+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:06:42.681619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":101,"num_write":42,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.7","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:14.566741+0000","last_change":"2026-02-24T22:35:14.566741+0000","last_active":"2026-02-24T22:35:14.566741+0000","last_peered":"2026-02-24T22:35:14.566741+0000","last_clean":"2026-02-24T22:35:14.566741+0000","last_became_active":"2026-02-24T22:23:29.214105+0000","last_became_peered":"2026-02-24T22:23:29.214105+0000","last_unstale":"2026-02-24T22:35:14.566741+0000","last_undegraded":"2026-02-24T22:35:14.566741+0000","last_fullsized":"2026-02-24T22:35:14.566741+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:14.566681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:14.566681+0000","last_clean_scrub_stamp":"2026-02-24T22:35:14.566681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:27:13.392670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.4","version":"312'3","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:10.615653+0000","last_change":"2026-02-24T22:35:10.615653+0000","last_active":"2026-02-24T22:35:10.615653+0000","last_peered":"2026-02-24T22:35:10.615653+0000","last_clean":"2026-02-24T22:35:10.615653+0000","last_became_active":"2026-02-24T22:15:25.826663+0000","last_became_peered":"2026-02-24T22:15:25.826663+0000","last_unstale":"2026-02-24T22:35:10.615653+0000","last_undegraded":"2026-02-24T22:35:10.615653+0000","last_fullsized":"2026-02-24T22:35:10.615653+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:10.615580+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:10.615580+0000","last_clean_scrub_stamp":"2026-02-24T22:35:10.615580+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:21:31.466450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.5","version":"0'0","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:06.414312+0000","last_change":"2026-02-24T22:35:06.414312+0000","last_active":"2026-02-24T22:35:06.414312+0000","last_peered":"2026-02-24T22:35:06.414312+0000","last_clean":"2026-02-24T22:35:06.414312+0000","last_became_active":"2026-02-24T22:15:22.816697+0000","last_became_peered":"2026-02-24T22:15:22.816697+0000","last_unstale":"2026-02-24T22:35:06.414312+0000","last_undegraded":"2026-02-24T22:35:06.414312+0000","last_fullsized":"2026-02-24T22:35:06.414312+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:06.414264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:06.414264+0000","last_clean_scrub_stamp":"2026-02-24T22:35:06.414264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:28:07.680045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"79.1a","version":"312'16","reported_seq":110,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.717216+0000","last_change":"2026-02-24T22:34:58.717216+0000","last_active":"2026-02-24T22:34:58.717216+0000","last_peered":"2026-02-24T22:34:58.717216+0000","last_clean":"2026-02-24T22:34:58.717216+0000","last_became_active":"2026-02-24T22:13:43.757353+0000","last_became_peered":"2026-02-24T22:13:43.757353+0000","last_unstale":"2026-02-24T22:34:58.717216+0000","last_undegraded":"2026-02-24T22:34:58.717216+0000","last_fullsized":"2026-02-24T22:34:58.717216+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'16","last_scrub_stamp":"2026-02-24T22:34:58.717170+0000","last_deep_scrub":"312'16","last_deep_scrub_stamp":"2026-02-24T22:34:58.717170+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.717170+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:56:29.218407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":65,"num_write":38,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.4","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:29.862613+0000","last_change":"2026-02-24T22:35:29.862613+0000","last_active":"2026-02-24T22:35:29.862613+0000","last_peered":"2026-02-24T22:35:29.862613+0000","last_clean":"2026-02-24T22:35:29.862613+0000","last_became_active":"2026-02-24T22:23:29.215014+0000","last_became_peered":"2026-02-24T22:23:29.215014+0000","last_unstale":"2026-02-24T22:35:29.862613+0000","last_undegraded":"2026-02-24T22:35:29.862613+0000","last_fullsized":"2026-02-24T22:35:29.862613+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:29.862526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:29.862526+0000","last_clean_scrub_stamp":"2026-02-24T22:35:29.862526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:19:23.150925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.7","version":"312'3","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:03.237185+0000","last_change":"2026-02-24T22:35:03.237185+0000","last_active":"2026-02-24T22:35:03.237185+0000","last_peered":"2026-02-24T22:35:03.237185+0000","last_clean":"2026-02-24T22:35:03.237185+0000","last_became_active":"2026-02-24T22:15:25.831806+0000","last_became_peered":"2026-02-24T22:15:25.831806+0000","last_unstale":"2026-02-24T22:35:03.237185+0000","last_undegraded":"2026-02-24T22:35:03.237185+0000","last_fullsized":"2026-02-24T22:35:03.237185+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:03.237068+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:03.237068+0000","last_clean_scrub_stamp":"2026-02-24T22:35:03.237068+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:04:02.704108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"80.6","version":"0'0","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:08.654082+0000","last_change":"2026-02-24T22:35:08.654082+0000","last_active":"2026-02-24T22:35:08.654082+0000","last_peered":"2026-02-24T22:35:08.654082+0000","last_clean":"2026-02-24T22:35:08.654082+0000","last_became_active":"2026-02-24T22:15:22.811413+0000","last_became_peered":"2026-02-24T22:15:22.811413+0000","last_unstale":"2026-02-24T22:35:08.654082+0000","last_undegraded":"2026-02-24T22:35:08.654082+0000","last_fullsized":"2026-02-24T22:35:08.654082+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:08.654011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:08.654011+0000","last_clean_scrub_stamp":"2026-02-24T22:35:08.654011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:15:31.642731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"79.19","version":"312'30","reported_seq":149,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:09.330476+0000","last_change":"2026-02-24T22:35:09.330426+0000","last_active":"2026-02-24T22:35:09.330476+0000","last_peered":"2026-02-24T22:35:09.330476+0000","last_clean":"2026-02-24T22:35:09.330476+0000","last_became_active":"2026-02-24T22:13:43.756114+0000","last_became_peered":"2026-02-24T22:13:43.756114+0000","last_unstale":"2026-02-24T22:35:09.330476+0000","last_undegraded":"2026-02-24T22:35:09.330476+0000","last_fullsized":"2026-02-24T22:35:09.330476+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'30","last_scrub_stamp":"2026-02-24T22:35:09.330359+0000","last_deep_scrub":"312'30","last_deep_scrub_stamp":"2026-02-24T22:35:09.330359+0000","last_clean_scrub_stamp":"2026-02-24T22:35:09.330359+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:23:17.878380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":216,"num_read_kb":143,"num_write":68,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.5","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:28.823856+0000","last_change":"2026-02-24T22:35:28.823856+0000","last_active":"2026-02-24T22:35:28.823856+0000","last_peered":"2026-02-24T22:35:28.823856+0000","last_clean":"2026-02-24T22:35:28.823856+0000","last_became_active":"2026-02-24T22:23:29.213256+0000","last_became_peered":"2026-02-24T22:23:29.213256+0000","last_unstale":"2026-02-24T22:35:28.823856+0000","last_undegraded":"2026-02-24T22:35:28.823856+0000","last_fullsized":"2026-02-24T22:35:28.823856+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:28.823793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:28.823793+0000","last_clean_scrub_stamp":"2026-02-24T22:35:28.823793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:29:08.343819+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.6","version":"312'3","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:08.360735+0000","last_change":"2026-02-24T22:35:08.360735+0000","last_active":"2026-02-24T22:35:08.360735+0000","last_peered":"2026-02-24T22:35:08.360735+0000","last_clean":"2026-02-24T22:35:08.360735+0000","last_became_active":"2026-02-24T22:15:25.833680+0000","last_became_peered":"2026-02-24T22:15:25.833680+0000","last_unstale":"2026-02-24T22:35:08.360735+0000","last_undegraded":"2026-02-24T22:35:08.360735+0000","last_fullsized":"2026-02-24T22:35:08.360735+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:08.360662+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:08.360662+0000","last_clean_scrub_stamp":"2026-02-24T22:35:08.360662+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:43:55.489569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.7","version":"0'0","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:09.618057+0000","last_change":"2026-02-24T22:35:09.618057+0000","last_active":"2026-02-24T22:35:09.618057+0000","last_peered":"2026-02-24T22:35:09.618057+0000","last_clean":"2026-02-24T22:35:09.618057+0000","last_became_active":"2026-02-24T22:15:22.811618+0000","last_became_peered":"2026-02-24T22:15:22.811618+0000","last_unstale":"2026-02-24T22:35:09.618057+0000","last_undegraded":"2026-02-24T22:35:09.618057+0000","last_fullsized":"2026-02-24T22:35:09.618057+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:09.617989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:09.617989+0000","last_clean_scrub_stamp":"2026-02-24T22:35:09.617989+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:27:23.181702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"79.18","version":"312'12","reported_seq":104,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:10.315573+0000","last_change":"2026-02-24T22:35:10.315509+0000","last_active":"2026-02-24T22:35:10.315573+0000","last_peered":"2026-02-24T22:35:10.315573+0000","last_clean":"2026-02-24T22:35:10.315573+0000","last_became_active":"2026-02-24T22:13:43.754981+0000","last_became_peered":"2026-02-24T22:13:43.754981+0000","last_unstale":"2026-02-24T22:35:10.315573+0000","last_undegraded":"2026-02-24T22:35:10.315573+0000","last_fullsized":"2026-02-24T22:35:10.315573+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'12","last_scrub_stamp":"2026-02-24T22:35:10.315446+0000","last_deep_scrub":"312'12","last_deep_scrub_stamp":"2026-02-24T22:35:10.315446+0000","last_clean_scrub_stamp":"2026-02-24T22:35:10.315446+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:04:57.572069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":86,"num_read_kb":57,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.a","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:27.841463+0000","last_change":"2026-02-24T22:35:27.841463+0000","last_active":"2026-02-24T22:35:27.841463+0000","last_peered":"2026-02-24T22:35:27.841463+0000","last_clean":"2026-02-24T22:35:27.841463+0000","last_became_active":"2026-02-24T22:23:29.212213+0000","last_became_peered":"2026-02-24T22:23:29.212213+0000","last_unstale":"2026-02-24T22:35:27.841463+0000","last_undegraded":"2026-02-24T22:35:27.841463+0000","last_fullsized":"2026-02-24T22:35:27.841463+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:27.841391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:27.841391+0000","last_clean_scrub_stamp":"2026-02-24T22:35:27.841391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:04:01.169164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.9","version":"0'0","reported_seq":40,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:13.690913+0000","last_change":"2026-02-24T22:35:13.690913+0000","last_active":"2026-02-24T22:35:13.690913+0000","last_peered":"2026-02-24T22:35:13.690913+0000","last_clean":"2026-02-24T22:35:13.690913+0000","last_became_active":"2026-02-24T22:15:25.826076+0000","last_became_peered":"2026-02-24T22:15:25.826076+0000","last_unstale":"2026-02-24T22:35:13.690913+0000","last_undegraded":"2026-02-24T22:35:13.690913+0000","last_fullsized":"2026-02-24T22:35:13.690913+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:13.690725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:13.690725+0000","last_clean_scrub_stamp":"2026-02-24T22:35:13.690725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:47:50.305008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.8","version":"312'6","reported_seq":57,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:57.217246+0000","last_change":"2026-02-24T22:34:57.217246+0000","last_active":"2026-02-24T22:34:57.217246+0000","last_peered":"2026-02-24T22:34:57.217246+0000","last_clean":"2026-02-24T22:34:57.217246+0000","last_became_active":"2026-02-24T22:15:22.815960+0000","last_became_peered":"2026-02-24T22:15:22.815960+0000","last_unstale":"2026-02-24T22:34:57.217246+0000","last_undegraded":"2026-02-24T22:34:57.217246+0000","last_fullsized":"2026-02-24T22:34:57.217246+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'6","last_scrub_stamp":"2026-02-24T22:34:57.217191+0000","last_deep_scrub":"312'6","last_deep_scrub_stamp":"2026-02-24T22:34:57.217191+0000","last_clean_scrub_stamp":"2026-02-24T22:34:57.217191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:07:48.538456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6144,"num_write":11,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.17","version":"312'34","reported_seq":178,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:03.366118+0000","last_change":"2026-02-24T22:35:03.366056+0000","last_active":"2026-02-24T22:35:03.366118+0000","last_peered":"2026-02-24T22:35:03.366118+0000","last_clean":"2026-02-24T22:35:03.366118+0000","last_became_active":"2026-02-24T22:14:00.748438+0000","last_became_peered":"2026-02-24T22:14:00.748438+0000","last_unstale":"2026-02-24T22:35:03.366118+0000","last_undegraded":"2026-02-24T22:35:03.366118+0000","last_fullsized":"2026-02-24T22:35:03.366118+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":293,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'34","last_scrub_stamp":"2026-02-24T22:35:03.365970+0000","last_deep_scrub":"312'34","last_deep_scrub_stamp":"2026-02-24T22:35:03.365970+0000","last_clean_scrub_stamp":"2026-02-24T22:35:03.365970+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:27:41.779757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":298,"num_read_kb":211,"num_write":78,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.b","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:05.714092+0000","last_change":"2026-02-24T22:35:05.714092+0000","last_active":"2026-02-24T22:35:05.714092+0000","last_peered":"2026-02-24T22:35:05.714092+0000","last_clean":"2026-02-24T22:35:05.714092+0000","last_became_active":"2026-02-24T22:23:29.196790+0000","last_became_peered":"2026-02-24T22:23:29.196790+0000","last_unstale":"2026-02-24T22:35:05.714092+0000","last_undegraded":"2026-02-24T22:35:05.714092+0000","last_fullsized":"2026-02-24T22:35:05.714092+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:05.714031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:05.714031+0000","last_clean_scrub_stamp":"2026-02-24T22:35:05.714031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:44:22.188544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.8","version":"0'0","reported_seq":40,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:14.051734+0000","last_change":"2026-02-24T22:35:14.051734+0000","last_active":"2026-02-24T22:35:14.051734+0000","last_peered":"2026-02-24T22:35:14.051734+0000","last_clean":"2026-02-24T22:35:14.051734+0000","last_became_active":"2026-02-24T22:15:25.825315+0000","last_became_peered":"2026-02-24T22:15:25.825315+0000","last_unstale":"2026-02-24T22:35:14.051734+0000","last_undegraded":"2026-02-24T22:35:14.051734+0000","last_fullsized":"2026-02-24T22:35:14.051734+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:14.051650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:14.051650+0000","last_clean_scrub_stamp":"2026-02-24T22:35:14.051650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:10:57.064078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.9","version":"0'0","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:59.757040+0000","last_change":"2026-02-24T22:34:59.757040+0000","last_active":"2026-02-24T22:34:59.757040+0000","last_peered":"2026-02-24T22:34:59.757040+0000","last_clean":"2026-02-24T22:34:59.757040+0000","last_became_active":"2026-02-24T22:15:22.811141+0000","last_became_peered":"2026-02-24T22:15:22.811141+0000","last_unstale":"2026-02-24T22:34:59.757040+0000","last_undegraded":"2026-02-24T22:34:59.757040+0000","last_fullsized":"2026-02-24T22:34:59.757040+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:59.756969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:59.756969+0000","last_clean_scrub_stamp":"2026-02-24T22:34:59.756969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:19:48.331387+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"79.16","version":"0'0","reported_seq":72,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:57.691734+0000","last_change":"2026-02-24T22:34:57.691689+0000","last_active":"2026-02-24T22:34:57.691734+0000","last_peered":"2026-02-24T22:34:57.691734+0000","last_clean":"2026-02-24T22:34:57.691734+0000","last_became_active":"2026-02-24T22:13:43.755885+0000","last_became_peered":"2026-02-24T22:13:43.755885+0000","last_unstale":"2026-02-24T22:34:57.691734+0000","last_undegraded":"2026-02-24T22:34:57.691734+0000","last_fullsized":"2026-02-24T22:34:57.691734+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:57.691616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:57.691616+0000","last_clean_scrub_stamp":"2026-02-24T22:34:57.691616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:54:10.527824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.8","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:17.966932+0000","last_change":"2026-02-24T22:35:17.966932+0000","last_active":"2026-02-24T22:35:17.966932+0000","last_peered":"2026-02-24T22:35:17.966932+0000","last_clean":"2026-02-24T22:35:17.966932+0000","last_became_active":"2026-02-24T22:23:29.198882+0000","last_became_peered":"2026-02-24T22:23:29.198882+0000","last_unstale":"2026-02-24T22:35:17.966932+0000","last_undegraded":"2026-02-24T22:35:17.966932+0000","last_fullsized":"2026-02-24T22:35:17.966932+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:17.966874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:17.966874+0000","last_clean_scrub_stamp":"2026-02-24T22:35:17.966874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:40:19.729940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"81.b","version":"0'0","reported_seq":40,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:20.294693+0000","last_change":"2026-02-24T22:35:20.294652+0000","last_active":"2026-02-24T22:35:20.294693+0000","last_peered":"2026-02-24T22:35:20.294693+0000","last_clean":"2026-02-24T22:35:20.294693+0000","last_became_active":"2026-02-24T22:15:25.827265+0000","last_became_peered":"2026-02-24T22:15:25.827265+0000","last_unstale":"2026-02-24T22:35:20.294693+0000","last_undegraded":"2026-02-24T22:35:20.294693+0000","last_fullsized":"2026-02-24T22:35:20.294693+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:20.294586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:20.294586+0000","last_clean_scrub_stamp":"2026-02-24T22:35:20.294586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:51:45.105405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.a","version":"0'0","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:16.770098+0000","last_change":"2026-02-24T22:35:16.770098+0000","last_active":"2026-02-24T22:35:16.770098+0000","last_peered":"2026-02-24T22:35:16.770098+0000","last_clean":"2026-02-24T22:35:16.770098+0000","last_became_active":"2026-02-24T22:15:22.816702+0000","last_became_peered":"2026-02-24T22:15:22.816702+0000","last_unstale":"2026-02-24T22:35:16.770098+0000","last_undegraded":"2026-02-24T22:35:16.770098+0000","last_fullsized":"2026-02-24T22:35:16.770098+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:16.770027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:16.770027+0000","last_clean_scrub_stamp":"2026-02-24T22:35:16.770027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:33:16.068318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.15","version":"307'6","reported_seq":91,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:12.713792+0000","last_change":"2026-02-24T22:35:12.713724+0000","last_active":"2026-02-24T22:35:12.713792+0000","last_peered":"2026-02-24T22:35:12.713792+0000","last_clean":"2026-02-24T22:35:12.713792+0000","last_became_active":"2026-02-24T22:13:43.760892+0000","last_became_peered":"2026-02-24T22:13:43.760892+0000","last_unstale":"2026-02-24T22:35:12.713792+0000","last_undegraded":"2026-02-24T22:35:12.713792+0000","last_fullsized":"2026-02-24T22:35:12.713792+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"307'6","last_scrub_stamp":"2026-02-24T22:35:12.713642+0000","last_deep_scrub":"307'6","last_deep_scrub_stamp":"2026-02-24T22:35:12.713642+0000","last_clean_scrub_stamp":"2026-02-24T22:35:12.713642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:51:38.187083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":37,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.9","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:02.763296+0000","last_change":"2026-02-24T22:35:02.763296+0000","last_active":"2026-02-24T22:35:02.763296+0000","last_peered":"2026-02-24T22:35:02.763296+0000","last_clean":"2026-02-24T22:35:02.763296+0000","last_became_active":"2026-02-24T22:23:29.197357+0000","last_became_peered":"2026-02-24T22:23:29.197357+0000","last_unstale":"2026-02-24T22:35:02.763296+0000","last_undegraded":"2026-02-24T22:35:02.763296+0000","last_fullsized":"2026-02-24T22:35:02.763296+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:02.763243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:02.763243+0000","last_clean_scrub_stamp":"2026-02-24T22:35:02.763243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:52:20.692060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.a","version":"312'3","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:21.333834+0000","last_change":"2026-02-24T22:35:21.333834+0000","last_active":"2026-02-24T22:35:21.333834+0000","last_peered":"2026-02-24T22:35:21.333834+0000","last_clean":"2026-02-24T22:35:21.333834+0000","last_became_active":"2026-02-24T22:15:25.827055+0000","last_became_peered":"2026-02-24T22:15:25.827055+0000","last_unstale":"2026-02-24T22:35:21.333834+0000","last_undegraded":"2026-02-24T22:35:21.333834+0000","last_fullsized":"2026-02-24T22:35:21.333834+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:21.333781+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:21.333781+0000","last_clean_scrub_stamp":"2026-02-24T22:35:21.333781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:00:01.811417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.b","version":"312'3","reported_seq":52,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:17.780898+0000","last_change":"2026-02-24T22:35:17.780856+0000","last_active":"2026-02-24T22:35:17.780898+0000","last_peered":"2026-02-24T22:35:17.780898+0000","last_clean":"2026-02-24T22:35:17.780898+0000","last_became_active":"2026-02-24T22:15:22.817498+0000","last_became_peered":"2026-02-24T22:15:22.817498+0000","last_unstale":"2026-02-24T22:35:17.780898+0000","last_undegraded":"2026-02-24T22:35:17.780898+0000","last_fullsized":"2026-02-24T22:35:17.780898+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:17.780787+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:17.780787+0000","last_clean_scrub_stamp":"2026-02-24T22:35:17.780787+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:49:22.655262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.14","version":"312'10","reported_seq":98,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:02.317457+0000","last_change":"2026-02-24T22:35:02.317457+0000","last_active":"2026-02-24T22:35:02.317457+0000","last_peered":"2026-02-24T22:35:02.317457+0000","last_clean":"2026-02-24T22:35:02.317457+0000","last_became_active":"2026-02-24T22:13:43.762341+0000","last_became_peered":"2026-02-24T22:13:43.762341+0000","last_unstale":"2026-02-24T22:35:02.317457+0000","last_undegraded":"2026-02-24T22:35:02.317457+0000","last_fullsized":"2026-02-24T22:35:02.317457+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'10","last_scrub_stamp":"2026-02-24T22:35:02.317403+0000","last_deep_scrub":"312'10","last_deep_scrub_stamp":"2026-02-24T22:35:02.317403+0000","last_clean_scrub_stamp":"2026-02-24T22:35:02.317403+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:39:36.833592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":46,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.e","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:15.577587+0000","last_change":"2026-02-24T22:35:15.577587+0000","last_active":"2026-02-24T22:35:15.577587+0000","last_peered":"2026-02-24T22:35:15.577587+0000","last_clean":"2026-02-24T22:35:15.577587+0000","last_became_active":"2026-02-24T22:23:29.215425+0000","last_became_peered":"2026-02-24T22:23:29.215425+0000","last_unstale":"2026-02-24T22:35:15.577587+0000","last_undegraded":"2026-02-24T22:35:15.577587+0000","last_fullsized":"2026-02-24T22:35:15.577587+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:15.577525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:15.577525+0000","last_clean_scrub_stamp":"2026-02-24T22:35:15.577525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:52:30.886126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.d","version":"0'0","reported_seq":40,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:22.336194+0000","last_change":"2026-02-24T22:35:22.336154+0000","last_active":"2026-02-24T22:35:22.336194+0000","last_peered":"2026-02-24T22:35:22.336194+0000","last_clean":"2026-02-24T22:35:22.336194+0000","last_became_active":"2026-02-24T22:15:25.829109+0000","last_became_peered":"2026-02-24T22:15:25.829109+0000","last_unstale":"2026-02-24T22:35:22.336194+0000","last_undegraded":"2026-02-24T22:35:22.336194+0000","last_fullsized":"2026-02-24T22:35:22.336194+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:22.336091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:22.336091+0000","last_clean_scrub_stamp":"2026-02-24T22:35:22.336091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:20:46.735519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.c","version":"0'0","reported_seq":46,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.197935+0000","last_change":"2026-02-24T22:34:58.197887+0000","last_active":"2026-02-24T22:34:58.197935+0000","last_peered":"2026-02-24T22:34:58.197935+0000","last_clean":"2026-02-24T22:34:58.197935+0000","last_became_active":"2026-02-24T22:15:22.815364+0000","last_became_peered":"2026-02-24T22:15:22.815364+0000","last_unstale":"2026-02-24T22:34:58.197935+0000","last_undegraded":"2026-02-24T22:34:58.197935+0000","last_fullsized":"2026-02-24T22:34:58.197935+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:58.197815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:58.197815+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.197815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:35:14.681874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.13","version":"312'11","reported_seq":104,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:51.152600+0000","last_change":"2026-02-24T22:34:51.152600+0000","last_active":"2026-02-24T22:34:51.152600+0000","last_peered":"2026-02-24T22:34:51.152600+0000","last_clean":"2026-02-24T22:34:51.152600+0000","last_became_active":"2026-02-24T22:13:43.760714+0000","last_became_peered":"2026-02-24T22:13:43.760714+0000","last_unstale":"2026-02-24T22:34:51.152600+0000","last_undegraded":"2026-02-24T22:34:51.152600+0000","last_fullsized":"2026-02-24T22:34:51.152600+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'11","last_scrub_stamp":"2026-02-24T22:34:51.152549+0000","last_deep_scrub":"312'11","last_deep_scrub_stamp":"2026-02-24T22:34:51.152549+0000","last_clean_scrub_stamp":"2026-02-24T22:34:51.152549+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:24:03.221560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":61,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"82.f","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:06.681089+0000","last_change":"2026-02-24T22:35:06.681089+0000","last_active":"2026-02-24T22:35:06.681089+0000","last_peered":"2026-02-24T22:35:06.681089+0000","last_clean":"2026-02-24T22:35:06.681089+0000","last_became_active":"2026-02-24T22:23:29.199538+0000","last_became_peered":"2026-02-24T22:23:29.199538+0000","last_unstale":"2026-02-24T22:35:06.681089+0000","last_undegraded":"2026-02-24T22:35:06.681089+0000","last_fullsized":"2026-02-24T22:35:06.681089+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:06.680985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:06.680985+0000","last_clean_scrub_stamp":"2026-02-24T22:35:06.680985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:18:47.506871+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.c","version":"312'11","reported_seq":67,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:12.647172+0000","last_change":"2026-02-24T22:35:12.647111+0000","last_active":"2026-02-24T22:35:12.647172+0000","last_peered":"2026-02-24T22:35:12.647172+0000","last_clean":"2026-02-24T22:35:12.647172+0000","last_became_active":"2026-02-24T22:15:25.826430+0000","last_became_peered":"2026-02-24T22:15:25.826430+0000","last_unstale":"2026-02-24T22:35:12.647172+0000","last_undegraded":"2026-02-24T22:35:12.647172+0000","last_fullsized":"2026-02-24T22:35:12.647172+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'11","last_scrub_stamp":"2026-02-24T22:35:12.647031+0000","last_deep_scrub":"312'11","last_deep_scrub_stamp":"2026-02-24T22:35:12.647031+0000","last_clean_scrub_stamp":"2026-02-24T22:35:12.647031+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:08:17.241612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":31746,"num_write":16,"num_write_kb":16384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.d","version":"312'5","reported_seq":61,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:14.345803+0000","last_change":"2026-02-24T22:35:14.345762+0000","last_active":"2026-02-24T22:35:14.345803+0000","last_peered":"2026-02-24T22:35:14.345803+0000","last_clean":"2026-02-24T22:35:14.345803+0000","last_became_active":"2026-02-24T22:15:22.813062+0000","last_became_peered":"2026-02-24T22:15:22.813062+0000","last_unstale":"2026-02-24T22:35:14.345803+0000","last_undegraded":"2026-02-24T22:35:14.345803+0000","last_fullsized":"2026-02-24T22:35:14.345803+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'5","last_scrub_stamp":"2026-02-24T22:35:14.345705+0000","last_deep_scrub":"312'5","last_deep_scrub_stamp":"2026-02-24T22:35:14.345705+0000","last_clean_scrub_stamp":"2026-02-24T22:35:14.345705+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:10:48.119100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":10,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.12","version":"312'20","reported_seq":126,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:53.127878+0000","last_change":"2026-02-24T22:34:53.127878+0000","last_active":"2026-02-24T22:34:53.127878+0000","last_peered":"2026-02-24T22:34:53.127878+0000","last_clean":"2026-02-24T22:34:53.127878+0000","last_became_active":"2026-02-24T22:13:43.760196+0000","last_became_peered":"2026-02-24T22:13:43.760196+0000","last_unstale":"2026-02-24T22:34:53.127878+0000","last_undegraded":"2026-02-24T22:34:53.127878+0000","last_fullsized":"2026-02-24T22:34:53.127878+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'20","last_scrub_stamp":"2026-02-24T22:34:53.127808+0000","last_deep_scrub":"312'20","last_deep_scrub_stamp":"2026-02-24T22:34:53.127808+0000","last_clean_scrub_stamp":"2026-02-24T22:34:53.127808+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:40:20.279936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":158,"num_read_kb":106,"num_write":52,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"82.c","version":"312'2","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:24.335353+0000","last_change":"2026-02-24T22:35:24.335353+0000","last_active":"2026-02-24T22:35:24.335353+0000","last_peered":"2026-02-24T22:35:24.335353+0000","last_clean":"2026-02-24T22:35:24.335353+0000","last_became_active":"2026-02-24T22:23:29.201545+0000","last_became_peered":"2026-02-24T22:23:29.201545+0000","last_unstale":"2026-02-24T22:35:24.335353+0000","last_undegraded":"2026-02-24T22:35:24.335353+0000","last_fullsized":"2026-02-24T22:35:24.335353+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:35:24.335286+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:35:24.335286+0000","last_clean_scrub_stamp":"2026-02-24T22:35:24.335286+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:50:10.196109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"81.f","version":"0'0","reported_seq":40,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:14.643696+0000","last_change":"2026-02-24T22:35:14.643638+0000","last_active":"2026-02-24T22:35:14.643696+0000","last_peered":"2026-02-24T22:35:14.643696+0000","last_clean":"2026-02-24T22:35:14.643696+0000","last_became_active":"2026-02-24T22:15:25.824862+0000","last_became_peered":"2026-02-24T22:15:25.824862+0000","last_unstale":"2026-02-24T22:35:14.643696+0000","last_undegraded":"2026-02-24T22:35:14.643696+0000","last_fullsized":"2026-02-24T22:35:14.643696+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:14.643576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:14.643576+0000","last_clean_scrub_stamp":"2026-02-24T22:35:14.643576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:45:45.729521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.e","version":"312'5","reported_seq":63,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:00.220105+0000","last_change":"2026-02-24T22:35:00.220063+0000","last_active":"2026-02-24T22:35:00.220105+0000","last_peered":"2026-02-24T22:35:00.220105+0000","last_clean":"2026-02-24T22:35:00.220105+0000","last_became_active":"2026-02-24T22:15:22.817295+0000","last_became_peered":"2026-02-24T22:15:22.817295+0000","last_unstale":"2026-02-24T22:35:00.220105+0000","last_undegraded":"2026-02-24T22:35:00.220105+0000","last_fullsized":"2026-02-24T22:35:00.220105+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'5","last_scrub_stamp":"2026-02-24T22:35:00.220005+0000","last_deep_scrub":"312'5","last_deep_scrub_stamp":"2026-02-24T22:35:00.220005+0000","last_clean_scrub_stamp":"2026-02-24T22:35:00.220005+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:12:59.300665+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":25602,"num_write":5,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.11","version":"312'11","reported_seq":101,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:13.729005+0000","last_change":"2026-02-24T22:35:13.729005+0000","last_active":"2026-02-24T22:35:13.729005+0000","last_peered":"2026-02-24T22:35:13.729005+0000","last_clean":"2026-02-24T22:35:13.729005+0000","last_became_active":"2026-02-24T22:13:43.759864+0000","last_became_peered":"2026-02-24T22:13:43.759864+0000","last_unstale":"2026-02-24T22:35:13.729005+0000","last_undegraded":"2026-02-24T22:35:13.729005+0000","last_fullsized":"2026-02-24T22:35:13.729005+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'11","last_scrub_stamp":"2026-02-24T22:35:13.728935+0000","last_deep_scrub":"312'11","last_deep_scrub_stamp":"2026-02-24T22:35:13.728935+0000","last_clean_scrub_stamp":"2026-02-24T22:35:13.728935+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:48:10.252727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":53,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.d","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:13.577503+0000","last_change":"2026-02-24T22:35:13.577503+0000","last_active":"2026-02-24T22:35:13.577503+0000","last_peered":"2026-02-24T22:35:13.577503+0000","last_clean":"2026-02-24T22:35:13.577503+0000","last_became_active":"2026-02-24T22:23:29.214432+0000","last_became_peered":"2026-02-24T22:23:29.214432+0000","last_unstale":"2026-02-24T22:35:13.577503+0000","last_undegraded":"2026-02-24T22:35:13.577503+0000","last_fullsized":"2026-02-24T22:35:13.577503+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:13.577417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:13.577417+0000","last_clean_scrub_stamp":"2026-02-24T22:35:13.577417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:24:57.277938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.e","version":"0'0","reported_seq":40,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:15.020525+0000","last_change":"2026-02-24T22:35:15.020485+0000","last_active":"2026-02-24T22:35:15.020525+0000","last_peered":"2026-02-24T22:35:15.020525+0000","last_clean":"2026-02-24T22:35:15.020525+0000","last_became_active":"2026-02-24T22:15:25.827110+0000","last_became_peered":"2026-02-24T22:15:25.827110+0000","last_unstale":"2026-02-24T22:35:15.020525+0000","last_undegraded":"2026-02-24T22:35:15.020525+0000","last_fullsized":"2026-02-24T22:35:15.020525+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:15.020423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:15.020423+0000","last_clean_scrub_stamp":"2026-02-24T22:35:15.020423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:37:42.576080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.f","version":"312'8","reported_seq":67,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:18.779442+0000","last_change":"2026-02-24T22:35:18.779401+0000","last_active":"2026-02-24T22:35:18.779442+0000","last_peered":"2026-02-24T22:35:18.779442+0000","last_clean":"2026-02-24T22:35:18.779442+0000","last_became_active":"2026-02-24T22:15:22.815819+0000","last_became_peered":"2026-02-24T22:15:22.815819+0000","last_unstale":"2026-02-24T22:35:18.779442+0000","last_undegraded":"2026-02-24T22:35:18.779442+0000","last_fullsized":"2026-02-24T22:35:18.779442+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'8","last_scrub_stamp":"2026-02-24T22:35:18.779354+0000","last_deep_scrub":"312'8","last_deep_scrub_stamp":"2026-02-24T22:35:18.779354+0000","last_clean_scrub_stamp":"2026-02-24T22:35:18.779354+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:04:34.683909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":26626,"num_write":13,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.10","version":"312'31","reported_seq":161,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:10.702578+0000","last_change":"2026-02-24T22:35:10.702578+0000","last_active":"2026-02-24T22:35:10.702578+0000","last_peered":"2026-02-24T22:35:10.702578+0000","last_clean":"2026-02-24T22:35:10.702578+0000","last_became_active":"2026-02-24T22:13:43.759855+0000","last_became_peered":"2026-02-24T22:13:43.759855+0000","last_unstale":"2026-02-24T22:35:10.702578+0000","last_undegraded":"2026-02-24T22:35:10.702578+0000","last_fullsized":"2026-02-24T22:35:10.702578+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'31","last_scrub_stamp":"2026-02-24T22:35:10.702522+0000","last_deep_scrub":"312'31","last_deep_scrub_stamp":"2026-02-24T22:35:10.702522+0000","last_clean_scrub_stamp":"2026-02-24T22:35:10.702522+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:16:33.866764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":272,"num_read_kb":204,"num_write":72,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.12","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:16.558443+0000","last_change":"2026-02-24T22:35:16.558443+0000","last_active":"2026-02-24T22:35:16.558443+0000","last_peered":"2026-02-24T22:35:16.558443+0000","last_clean":"2026-02-24T22:35:16.558443+0000","last_became_active":"2026-02-24T22:23:29.212534+0000","last_became_peered":"2026-02-24T22:23:29.212534+0000","last_unstale":"2026-02-24T22:35:16.558443+0000","last_undegraded":"2026-02-24T22:35:16.558443+0000","last_fullsized":"2026-02-24T22:35:16.558443+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:16.558383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:16.558383+0000","last_clean_scrub_stamp":"2026-02-24T22:35:16.558383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:07:41.652982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.11","version":"312'5","reported_seq":54,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:15.611712+0000","last_change":"2026-02-24T22:35:15.611712+0000","last_active":"2026-02-24T22:35:15.611712+0000","last_peered":"2026-02-24T22:35:15.611712+0000","last_clean":"2026-02-24T22:35:15.611712+0000","last_became_active":"2026-02-24T22:15:25.826517+0000","last_became_peered":"2026-02-24T22:15:25.826517+0000","last_unstale":"2026-02-24T22:35:15.611712+0000","last_undegraded":"2026-02-24T22:35:15.611712+0000","last_fullsized":"2026-02-24T22:35:15.611712+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'5","last_scrub_stamp":"2026-02-24T22:35:15.611647+0000","last_deep_scrub":"312'5","last_deep_scrub_stamp":"2026-02-24T22:35:15.611647+0000","last_clean_scrub_stamp":"2026-02-24T22:35:15.611647+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:23:46.091088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":10,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.10","version":"0'0","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:10.152120+0000","last_change":"2026-02-24T22:35:10.152120+0000","last_active":"2026-02-24T22:35:10.152120+0000","last_peered":"2026-02-24T22:35:10.152120+0000","last_clean":"2026-02-24T22:35:10.152120+0000","last_became_active":"2026-02-24T22:15:22.810000+0000","last_became_peered":"2026-02-24T22:15:22.810000+0000","last_unstale":"2026-02-24T22:35:10.152120+0000","last_undegraded":"2026-02-24T22:35:10.152120+0000","last_fullsized":"2026-02-24T22:35:10.152120+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:10.152045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:10.152045+0000","last_clean_scrub_stamp":"2026-02-24T22:35:10.152045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:14:12.119607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"79.f","version":"312'11","reported_seq":103,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:11.110079+0000","last_change":"2026-02-24T22:35:11.110079+0000","last_active":"2026-02-24T22:35:11.110079+0000","last_peered":"2026-02-24T22:35:11.110079+0000","last_clean":"2026-02-24T22:35:11.110079+0000","last_became_active":"2026-02-24T22:13:43.754602+0000","last_became_peered":"2026-02-24T22:13:43.754602+0000","last_unstale":"2026-02-24T22:35:11.110079+0000","last_undegraded":"2026-02-24T22:35:11.110079+0000","last_fullsized":"2026-02-24T22:35:11.110079+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'11","last_scrub_stamp":"2026-02-24T22:35:11.110015+0000","last_deep_scrub":"312'11","last_deep_scrub_stamp":"2026-02-24T22:35:11.110015+0000","last_clean_scrub_stamp":"2026-02-24T22:35:11.110015+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:32:49.033187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":67,"num_write":27,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.13","version":"312'2","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:23.613569+0000","last_change":"2026-02-24T22:35:23.613569+0000","last_active":"2026-02-24T22:35:23.613569+0000","last_peered":"2026-02-24T22:35:23.613569+0000","last_clean":"2026-02-24T22:35:23.613569+0000","last_became_active":"2026-02-24T22:23:29.198627+0000","last_became_peered":"2026-02-24T22:23:29.198627+0000","last_unstale":"2026-02-24T22:35:23.613569+0000","last_undegraded":"2026-02-24T22:35:23.613569+0000","last_fullsized":"2026-02-24T22:35:23.613569+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:35:23.613520+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:35:23.613520+0000","last_clean_scrub_stamp":"2026-02-24T22:35:23.613520+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:04:39.063377+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.10","version":"0'0","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:16.599841+0000","last_change":"2026-02-24T22:35:16.599841+0000","last_active":"2026-02-24T22:35:16.599841+0000","last_peered":"2026-02-24T22:35:16.599841+0000","last_clean":"2026-02-24T22:35:16.599841+0000","last_became_active":"2026-02-24T22:15:25.826606+0000","last_became_peered":"2026-02-24T22:15:25.826606+0000","last_unstale":"2026-02-24T22:35:16.599841+0000","last_undegraded":"2026-02-24T22:35:16.599841+0000","last_fullsized":"2026-02-24T22:35:16.599841+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:16.599782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:16.599782+0000","last_clean_scrub_stamp":"2026-02-24T22:35:16.599782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:38:05.327614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.11","version":"0'0","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:12.081450+0000","last_change":"2026-02-24T22:35:12.081450+0000","last_active":"2026-02-24T22:35:12.081450+0000","last_peered":"2026-02-24T22:35:12.081450+0000","last_clean":"2026-02-24T22:35:12.081450+0000","last_became_active":"2026-02-24T22:15:22.809791+0000","last_became_peered":"2026-02-24T22:15:22.809791+0000","last_unstale":"2026-02-24T22:35:12.081450+0000","last_undegraded":"2026-02-24T22:35:12.081450+0000","last_fullsized":"2026-02-24T22:35:12.081450+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:12.081369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:12.081369+0000","last_clean_scrub_stamp":"2026-02-24T22:35:12.081369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:45:29.355115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"79.e","version":"312'23","reported_seq":134,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:08.185039+0000","last_change":"2026-02-24T22:35:08.185039+0000","last_active":"2026-02-24T22:35:08.185039+0000","last_peered":"2026-02-24T22:35:08.185039+0000","last_clean":"2026-02-24T22:35:08.185039+0000","last_became_active":"2026-02-24T22:13:43.755868+0000","last_became_peered":"2026-02-24T22:13:43.755868+0000","last_unstale":"2026-02-24T22:35:08.185039+0000","last_undegraded":"2026-02-24T22:35:08.185039+0000","last_fullsized":"2026-02-24T22:35:08.185039+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'23","last_scrub_stamp":"2026-02-24T22:35:08.184981+0000","last_deep_scrub":"312'23","last_deep_scrub_stamp":"2026-02-24T22:35:08.184981+0000","last_clean_scrub_stamp":"2026-02-24T22:35:08.184981+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:38:46.065895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":169,"num_read_kb":112,"num_write":56,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.10","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:30.898563+0000","last_change":"2026-02-24T22:35:30.898563+0000","last_active":"2026-02-24T22:35:30.898563+0000","last_peered":"2026-02-24T22:35:30.898563+0000","last_clean":"2026-02-24T22:35:30.898563+0000","last_became_active":"2026-02-24T22:23:29.212665+0000","last_became_peered":"2026-02-24T22:23:29.212665+0000","last_unstale":"2026-02-24T22:35:30.898563+0000","last_undegraded":"2026-02-24T22:35:30.898563+0000","last_fullsized":"2026-02-24T22:35:30.898563+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:30.898498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:30.898498+0000","last_clean_scrub_stamp":"2026-02-24T22:35:30.898498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:22:59.390893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"81.13","version":"0'0","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:01.756922+0000","last_change":"2026-02-24T22:35:01.756922+0000","last_active":"2026-02-24T22:35:01.756922+0000","last_peered":"2026-02-24T22:35:01.756922+0000","last_clean":"2026-02-24T22:35:01.756922+0000","last_became_active":"2026-02-24T22:15:25.825862+0000","last_became_peered":"2026-02-24T22:15:25.825862+0000","last_unstale":"2026-02-24T22:35:01.756922+0000","last_undegraded":"2026-02-24T22:35:01.756922+0000","last_fullsized":"2026-02-24T22:35:01.756922+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:01.756854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:01.756854+0000","last_clean_scrub_stamp":"2026-02-24T22:35:01.756854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:53:27.843482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"80.12","version":"0'0","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:15.310359+0000","last_change":"2026-02-24T22:35:15.310359+0000","last_active":"2026-02-24T22:35:15.310359+0000","last_peered":"2026-02-24T22:35:15.310359+0000","last_clean":"2026-02-24T22:35:15.310359+0000","last_became_active":"2026-02-24T22:15:22.812405+0000","last_became_peered":"2026-02-24T22:15:22.812405+0000","last_unstale":"2026-02-24T22:35:15.310359+0000","last_undegraded":"2026-02-24T22:35:15.310359+0000","last_fullsized":"2026-02-24T22:35:15.310359+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:15.310191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:15.310191+0000","last_clean_scrub_stamp":"2026-02-24T22:35:15.310191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:35:30.889874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.d","version":"312'20","reported_seq":121,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:10.586519+0000","last_change":"2026-02-24T22:35:10.586519+0000","last_active":"2026-02-24T22:35:10.586519+0000","last_peered":"2026-02-24T22:35:10.586519+0000","last_clean":"2026-02-24T22:35:10.586519+0000","last_became_active":"2026-02-24T22:13:43.761339+0000","last_became_peered":"2026-02-24T22:13:43.761339+0000","last_unstale":"2026-02-24T22:35:10.586519+0000","last_undegraded":"2026-02-24T22:35:10.586519+0000","last_fullsized":"2026-02-24T22:35:10.586519+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'20","last_scrub_stamp":"2026-02-24T22:35:10.586422+0000","last_deep_scrub":"312'20","last_deep_scrub_stamp":"2026-02-24T22:35:10.586422+0000","last_clean_scrub_stamp":"2026-02-24T22:35:10.586422+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:07:09.238716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":126,"num_read_kb":76,"num_write":49,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"82.11","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:12.343554+0000","last_change":"2026-02-24T22:35:12.343554+0000","last_active":"2026-02-24T22:35:12.343554+0000","last_peered":"2026-02-24T22:35:12.343554+0000","last_clean":"2026-02-24T22:35:12.343554+0000","last_became_active":"2026-02-24T22:23:29.214167+0000","last_became_peered":"2026-02-24T22:23:29.214167+0000","last_unstale":"2026-02-24T22:35:12.343554+0000","last_undegraded":"2026-02-24T22:35:12.343554+0000","last_fullsized":"2026-02-24T22:35:12.343554+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:12.343445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:12.343445+0000","last_clean_scrub_stamp":"2026-02-24T22:35:12.343445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:38:27.753745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.12","version":"0'0","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:19.600664+0000","last_change":"2026-02-24T22:35:19.600664+0000","last_active":"2026-02-24T22:35:19.600664+0000","last_peered":"2026-02-24T22:35:19.600664+0000","last_clean":"2026-02-24T22:35:19.600664+0000","last_became_active":"2026-02-24T22:15:25.826102+0000","last_became_peered":"2026-02-24T22:15:25.826102+0000","last_unstale":"2026-02-24T22:35:19.600664+0000","last_undegraded":"2026-02-24T22:35:19.600664+0000","last_fullsized":"2026-02-24T22:35:19.600664+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:19.600611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:19.600611+0000","last_clean_scrub_stamp":"2026-02-24T22:35:19.600611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:29:15.870895+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.13","version":"312'3","reported_seq":51,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:16.310449+0000","last_change":"2026-02-24T22:35:16.310449+0000","last_active":"2026-02-24T22:35:16.310449+0000","last_peered":"2026-02-24T22:35:16.310449+0000","last_clean":"2026-02-24T22:35:16.310449+0000","last_became_active":"2026-02-24T22:15:22.811927+0000","last_became_peered":"2026-02-24T22:15:22.811927+0000","last_unstale":"2026-02-24T22:35:16.310449+0000","last_undegraded":"2026-02-24T22:35:16.310449+0000","last_fullsized":"2026-02-24T22:35:16.310449+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:16.310385+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:16.310385+0000","last_clean_scrub_stamp":"2026-02-24T22:35:16.310385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:28:26.685415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.c","version":"312'4","reported_seq":82,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:52.107283+0000","last_change":"2026-02-24T22:34:52.107283+0000","last_active":"2026-02-24T22:34:52.107283+0000","last_peered":"2026-02-24T22:34:52.107283+0000","last_clean":"2026-02-24T22:34:52.107283+0000","last_became_active":"2026-02-24T22:13:43.761910+0000","last_became_peered":"2026-02-24T22:13:43.761910+0000","last_unstale":"2026-02-24T22:34:52.107283+0000","last_undegraded":"2026-02-24T22:34:52.107283+0000","last_fullsized":"2026-02-24T22:34:52.107283+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'4","last_scrub_stamp":"2026-02-24T22:34:52.107217+0000","last_deep_scrub":"312'4","last_deep_scrub_stamp":"2026-02-24T22:34:52.107217+0000","last_clean_scrub_stamp":"2026-02-24T22:34:52.107217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:56:38.678014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":15,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"82.16","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:06.210885+0000","last_change":"2026-02-24T22:35:06.210797+0000","last_active":"2026-02-24T22:35:06.210885+0000","last_peered":"2026-02-24T22:35:06.210885+0000","last_clean":"2026-02-24T22:35:06.210885+0000","last_became_active":"2026-02-24T22:23:29.213080+0000","last_became_peered":"2026-02-24T22:23:29.213080+0000","last_unstale":"2026-02-24T22:35:06.210885+0000","last_undegraded":"2026-02-24T22:35:06.210885+0000","last_fullsized":"2026-02-24T22:35:06.210885+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:06.210726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:06.210726+0000","last_clean_scrub_stamp":"2026-02-24T22:35:06.210726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:22:55.658806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.15","version":"0'0","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:05.258451+0000","last_change":"2026-02-24T22:35:05.258451+0000","last_active":"2026-02-24T22:35:05.258451+0000","last_peered":"2026-02-24T22:35:05.258451+0000","last_clean":"2026-02-24T22:35:05.258451+0000","last_became_active":"2026-02-24T22:15:25.830922+0000","last_became_peered":"2026-02-24T22:15:25.830922+0000","last_unstale":"2026-02-24T22:35:05.258451+0000","last_undegraded":"2026-02-24T22:35:05.258451+0000","last_fullsized":"2026-02-24T22:35:05.258451+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:05.258395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:05.258395+0000","last_clean_scrub_stamp":"2026-02-24T22:35:05.258395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:02:54.284713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"80.14","version":"312'3","reported_seq":51,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:17.344992+0000","last_change":"2026-02-24T22:35:17.344992+0000","last_active":"2026-02-24T22:35:17.344992+0000","last_peered":"2026-02-24T22:35:17.344992+0000","last_clean":"2026-02-24T22:35:17.344992+0000","last_became_active":"2026-02-24T22:15:22.811924+0000","last_became_peered":"2026-02-24T22:15:22.811924+0000","last_unstale":"2026-02-24T22:35:17.344992+0000","last_undegraded":"2026-02-24T22:35:17.344992+0000","last_fullsized":"2026-02-24T22:35:17.344992+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:17.344936+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:17.344936+0000","last_clean_scrub_stamp":"2026-02-24T22:35:17.344936+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:22:25.826405+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.b","version":"312'13","reported_seq":106,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:56.646904+0000","last_change":"2026-02-24T22:34:56.646904+0000","last_active":"2026-02-24T22:34:56.646904+0000","last_peered":"2026-02-24T22:34:56.646904+0000","last_clean":"2026-02-24T22:34:56.646904+0000","last_became_active":"2026-02-24T22:13:43.756596+0000","last_became_peered":"2026-02-24T22:13:43.756596+0000","last_unstale":"2026-02-24T22:34:56.646904+0000","last_undegraded":"2026-02-24T22:34:56.646904+0000","last_fullsized":"2026-02-24T22:34:56.646904+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'13","last_scrub_stamp":"2026-02-24T22:34:56.646848+0000","last_deep_scrub":"312'13","last_deep_scrub_stamp":"2026-02-24T22:34:56.646848+0000","last_clean_scrub_stamp":"2026-02-24T22:34:56.646848+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:06:44.716107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":56,"num_write":37,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.17","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:07.200779+0000","last_change":"2026-02-24T22:35:07.200744+0000","last_active":"2026-02-24T22:35:07.200779+0000","last_peered":"2026-02-24T22:35:07.200779+0000","last_clean":"2026-02-24T22:35:07.200779+0000","last_became_active":"2026-02-24T22:23:29.214217+0000","last_became_peered":"2026-02-24T22:23:29.214217+0000","last_unstale":"2026-02-24T22:35:07.200779+0000","last_undegraded":"2026-02-24T22:35:07.200779+0000","last_fullsized":"2026-02-24T22:35:07.200779+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:07.200690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:07.200690+0000","last_clean_scrub_stamp":"2026-02-24T22:35:07.200690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:02:31.790157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.14","version":"312'5","reported_seq":50,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:18.571008+0000","last_change":"2026-02-24T22:35:18.571008+0000","last_active":"2026-02-24T22:35:18.571008+0000","last_peered":"2026-02-24T22:35:18.571008+0000","last_clean":"2026-02-24T22:35:18.571008+0000","last_became_active":"2026-02-24T22:15:25.827557+0000","last_became_peered":"2026-02-24T22:15:25.827557+0000","last_unstale":"2026-02-24T22:35:18.571008+0000","last_undegraded":"2026-02-24T22:35:18.571008+0000","last_fullsized":"2026-02-24T22:35:18.571008+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'5","last_scrub_stamp":"2026-02-24T22:35:18.570944+0000","last_deep_scrub":"312'5","last_deep_scrub_stamp":"2026-02-24T22:35:18.570944+0000","last_clean_scrub_stamp":"2026-02-24T22:35:18.570944+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:05:32.693769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":10,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.15","version":"312'5","reported_seq":56,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:19.259017+0000","last_change":"2026-02-24T22:35:19.259017+0000","last_active":"2026-02-24T22:35:19.259017+0000","last_peered":"2026-02-24T22:35:19.259017+0000","last_clean":"2026-02-24T22:35:19.259017+0000","last_became_active":"2026-02-24T22:15:22.812450+0000","last_became_peered":"2026-02-24T22:15:22.812450+0000","last_unstale":"2026-02-24T22:35:19.259017+0000","last_undegraded":"2026-02-24T22:35:19.259017+0000","last_fullsized":"2026-02-24T22:35:19.259017+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'5","last_scrub_stamp":"2026-02-24T22:35:19.258949+0000","last_deep_scrub":"312'5","last_deep_scrub_stamp":"2026-02-24T22:35:19.258949+0000","last_clean_scrub_stamp":"2026-02-24T22:35:19.258949+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:20:04.684086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":10,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"79.a","version":"312'10","reported_seq":99,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:01.350817+0000","last_change":"2026-02-24T22:35:01.350761+0000","last_active":"2026-02-24T22:35:01.350817+0000","last_peered":"2026-02-24T22:35:01.350817+0000","last_clean":"2026-02-24T22:35:01.350817+0000","last_became_active":"2026-02-24T22:13:43.762226+0000","last_became_peered":"2026-02-24T22:13:43.762226+0000","last_unstale":"2026-02-24T22:35:01.350817+0000","last_undegraded":"2026-02-24T22:35:01.350817+0000","last_fullsized":"2026-02-24T22:35:01.350817+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'10","last_scrub_stamp":"2026-02-24T22:35:01.350683+0000","last_deep_scrub":"312'10","last_deep_scrub_stamp":"2026-02-24T22:35:01.350683+0000","last_clean_scrub_stamp":"2026-02-24T22:35:01.350683+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:23:37.121406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":42,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.14","version":"312'2","reported_seq":42,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:21.601634+0000","last_change":"2026-02-24T22:35:21.601634+0000","last_active":"2026-02-24T22:35:21.601634+0000","last_peered":"2026-02-24T22:35:21.601634+0000","last_clean":"2026-02-24T22:35:21.601634+0000","last_became_active":"2026-02-24T22:24:01.297193+0000","last_became_peered":"2026-02-24T22:24:01.297193+0000","last_unstale":"2026-02-24T22:35:21.601634+0000","last_undegraded":"2026-02-24T22:35:21.601634+0000","last_fullsized":"2026-02-24T22:35:21.601634+0000","mapping_epoch":311,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":312,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:35:21.601576+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:35:21.601576+0000","last_clean_scrub_stamp":"2026-02-24T22:35:21.601576+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:59:47.986562+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.17","version":"312'3","reported_seq":44,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:23.867471+0000","last_change":"2026-02-24T22:35:23.867471+0000","last_active":"2026-02-24T22:35:23.867471+0000","last_peered":"2026-02-24T22:35:23.867471+0000","last_clean":"2026-02-24T22:35:23.867471+0000","last_became_active":"2026-02-24T22:15:25.833430+0000","last_became_peered":"2026-02-24T22:15:25.833430+0000","last_unstale":"2026-02-24T22:35:23.867471+0000","last_undegraded":"2026-02-24T22:35:23.867471+0000","last_fullsized":"2026-02-24T22:35:23.867471+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:23.867410+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:23.867410+0000","last_clean_scrub_stamp":"2026-02-24T22:35:23.867410+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:55:43.495676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"80.16","version":"0'0","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:01.238371+0000","last_change":"2026-02-24T22:35:01.238371+0000","last_active":"2026-02-24T22:35:01.238371+0000","last_peered":"2026-02-24T22:35:01.238371+0000","last_clean":"2026-02-24T22:35:01.238371+0000","last_became_active":"2026-02-24T22:15:22.817150+0000","last_became_peered":"2026-02-24T22:15:22.817150+0000","last_unstale":"2026-02-24T22:35:01.238371+0000","last_undegraded":"2026-02-24T22:35:01.238371+0000","last_fullsized":"2026-02-24T22:35:01.238371+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:01.238320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:01.238320+0000","last_clean_scrub_stamp":"2026-02-24T22:35:01.238320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:55:36.219572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.9","version":"312'24","reported_seq":134,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:09.179891+0000","last_change":"2026-02-24T22:35:09.179827+0000","last_active":"2026-02-24T22:35:09.179891+0000","last_peered":"2026-02-24T22:35:09.179891+0000","last_clean":"2026-02-24T22:35:09.179891+0000","last_became_active":"2026-02-24T22:13:43.756777+0000","last_became_peered":"2026-02-24T22:13:43.756777+0000","last_unstale":"2026-02-24T22:35:09.179891+0000","last_undegraded":"2026-02-24T22:35:09.179891+0000","last_fullsized":"2026-02-24T22:35:09.179891+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'24","last_scrub_stamp":"2026-02-24T22:35:09.179742+0000","last_deep_scrub":"312'24","last_deep_scrub_stamp":"2026-02-24T22:35:09.179742+0000","last_clean_scrub_stamp":"2026-02-24T22:35:09.179742+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:51:13.859546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":107,"num_write":60,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.15","version":"312'2","reported_seq":32,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:07.645510+0000","last_change":"2026-02-24T22:35:07.645456+0000","last_active":"2026-02-24T22:35:07.645510+0000","last_peered":"2026-02-24T22:35:07.645510+0000","last_clean":"2026-02-24T22:35:07.645510+0000","last_became_active":"2026-02-24T22:23:29.198000+0000","last_became_peered":"2026-02-24T22:23:29.198000+0000","last_unstale":"2026-02-24T22:35:07.645510+0000","last_undegraded":"2026-02-24T22:35:07.645510+0000","last_fullsized":"2026-02-24T22:35:07.645510+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:35:07.645381+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:35:07.645381+0000","last_clean_scrub_stamp":"2026-02-24T22:35:07.645381+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:16:38.802468+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.16","version":"0'0","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:17.598634+0000","last_change":"2026-02-24T22:35:17.598634+0000","last_active":"2026-02-24T22:35:17.598634+0000","last_peered":"2026-02-24T22:35:17.598634+0000","last_clean":"2026-02-24T22:35:17.598634+0000","last_became_active":"2026-02-24T22:15:25.828066+0000","last_became_peered":"2026-02-24T22:15:25.828066+0000","last_unstale":"2026-02-24T22:35:17.598634+0000","last_undegraded":"2026-02-24T22:35:17.598634+0000","last_fullsized":"2026-02-24T22:35:17.598634+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:17.598564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:17.598564+0000","last_clean_scrub_stamp":"2026-02-24T22:35:17.598564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:01:51.025597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"80.17","version":"0'0","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:19.812089+0000","last_change":"2026-02-24T22:35:19.812089+0000","last_active":"2026-02-24T22:35:19.812089+0000","last_peered":"2026-02-24T22:35:19.812089+0000","last_clean":"2026-02-24T22:35:19.812089+0000","last_became_active":"2026-02-24T22:15:22.817950+0000","last_became_peered":"2026-02-24T22:15:22.817950+0000","last_unstale":"2026-02-24T22:35:19.812089+0000","last_undegraded":"2026-02-24T22:35:19.812089+0000","last_fullsized":"2026-02-24T22:35:19.812089+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:19.812033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:19.812033+0000","last_clean_scrub_stamp":"2026-02-24T22:35:19.812033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:07:16.950514+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.8","version":"312'26","reported_seq":145,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:09.733420+0000","last_change":"2026-02-24T22:35:09.733355+0000","last_active":"2026-02-24T22:35:09.733420+0000","last_peered":"2026-02-24T22:35:09.733420+0000","last_clean":"2026-02-24T22:35:09.733420+0000","last_became_active":"2026-02-24T22:13:43.761558+0000","last_became_peered":"2026-02-24T22:13:43.761558+0000","last_unstale":"2026-02-24T22:35:09.733420+0000","last_undegraded":"2026-02-24T22:35:09.733420+0000","last_fullsized":"2026-02-24T22:35:09.733420+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'26","last_scrub_stamp":"2026-02-24T22:35:09.733252+0000","last_deep_scrub":"312'26","last_deep_scrub_stamp":"2026-02-24T22:35:09.733252+0000","last_clean_scrub_stamp":"2026-02-24T22:35:09.733252+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:51:48.673281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":152,"num_write":62,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"82.1a","version":"0'0","reported_seq":29,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:13.297685+0000","last_change":"2026-02-24T22:35:13.297685+0000","last_active":"2026-02-24T22:35:13.297685+0000","last_peered":"2026-02-24T22:35:13.297685+0000","last_clean":"2026-02-24T22:35:13.297685+0000","last_became_active":"2026-02-24T22:23:29.212709+0000","last_became_peered":"2026-02-24T22:23:29.212709+0000","last_unstale":"2026-02-24T22:35:13.297685+0000","last_undegraded":"2026-02-24T22:35:13.297685+0000","last_fullsized":"2026-02-24T22:35:13.297685+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:13.297633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:13.297633+0000","last_clean_scrub_stamp":"2026-02-24T22:35:13.297633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:03:00.921017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.19","version":"0'0","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:09.322953+0000","last_change":"2026-02-24T22:35:09.322953+0000","last_active":"2026-02-24T22:35:09.322953+0000","last_peered":"2026-02-24T22:35:09.322953+0000","last_clean":"2026-02-24T22:35:09.322953+0000","last_became_active":"2026-02-24T22:15:25.831885+0000","last_became_peered":"2026-02-24T22:15:25.831885+0000","last_unstale":"2026-02-24T22:35:09.322953+0000","last_undegraded":"2026-02-24T22:35:09.322953+0000","last_fullsized":"2026-02-24T22:35:09.322953+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:09.322906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:09.322906+0000","last_clean_scrub_stamp":"2026-02-24T22:35:09.322906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:18:38.861480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.18","version":"0'0","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:13.033058+0000","last_change":"2026-02-24T22:35:13.033058+0000","last_active":"2026-02-24T22:35:13.033058+0000","last_peered":"2026-02-24T22:35:13.033058+0000","last_clean":"2026-02-24T22:35:13.033058+0000","last_became_active":"2026-02-24T22:15:22.810156+0000","last_became_peered":"2026-02-24T22:15:22.810156+0000","last_unstale":"2026-02-24T22:35:13.033058+0000","last_undegraded":"2026-02-24T22:35:13.033058+0000","last_fullsized":"2026-02-24T22:35:13.033058+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:13.032975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:13.032975+0000","last_clean_scrub_stamp":"2026-02-24T22:35:13.032975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:48:05.525679+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"79.7","version":"0'0","reported_seq":73,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:08.320182+0000","last_change":"2026-02-24T22:35:08.320149+0000","last_active":"2026-02-24T22:35:08.320182+0000","last_peered":"2026-02-24T22:35:08.320182+0000","last_clean":"2026-02-24T22:35:08.320182+0000","last_became_active":"2026-02-24T22:13:43.755957+0000","last_became_peered":"2026-02-24T22:13:43.755957+0000","last_unstale":"2026-02-24T22:35:08.320182+0000","last_undegraded":"2026-02-24T22:35:08.320182+0000","last_fullsized":"2026-02-24T22:35:08.320182+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:08.320097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:08.320097+0000","last_clean_scrub_stamp":"2026-02-24T22:35:08.320097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:38:24.732745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.1b","version":"0'0","reported_seq":29,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:08.661691+0000","last_change":"2026-02-24T22:35:08.661691+0000","last_active":"2026-02-24T22:35:08.661691+0000","last_peered":"2026-02-24T22:35:08.661691+0000","last_clean":"2026-02-24T22:35:08.661691+0000","last_became_active":"2026-02-24T22:23:29.200152+0000","last_became_peered":"2026-02-24T22:23:29.200152+0000","last_unstale":"2026-02-24T22:35:08.661691+0000","last_undegraded":"2026-02-24T22:35:08.661691+0000","last_fullsized":"2026-02-24T22:35:08.661691+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:08.661629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:08.661629+0000","last_clean_scrub_stamp":"2026-02-24T22:35:08.661629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:43:02.912489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"81.18","version":"0'0","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:16.045841+0000","last_change":"2026-02-24T22:35:16.045841+0000","last_active":"2026-02-24T22:35:16.045841+0000","last_peered":"2026-02-24T22:35:16.045841+0000","last_clean":"2026-02-24T22:35:16.045841+0000","last_became_active":"2026-02-24T22:15:25.823415+0000","last_became_peered":"2026-02-24T22:15:25.823415+0000","last_unstale":"2026-02-24T22:35:16.045841+0000","last_undegraded":"2026-02-24T22:35:16.045841+0000","last_fullsized":"2026-02-24T22:35:16.045841+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:16.045769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:16.045769+0000","last_clean_scrub_stamp":"2026-02-24T22:35:16.045769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:46:21.973670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.19","version":"0'0","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:59.228352+0000","last_change":"2026-02-24T22:34:59.228352+0000","last_active":"2026-02-24T22:34:59.228352+0000","last_peered":"2026-02-24T22:34:59.228352+0000","last_clean":"2026-02-24T22:34:59.228352+0000","last_became_active":"2026-02-24T22:15:22.816826+0000","last_became_peered":"2026-02-24T22:15:22.816826+0000","last_unstale":"2026-02-24T22:34:59.228352+0000","last_undegraded":"2026-02-24T22:34:59.228352+0000","last_fullsized":"2026-02-24T22:34:59.228352+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:59.228267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:59.228267+0000","last_clean_scrub_stamp":"2026-02-24T22:34:59.228267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:53:36.867225+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.6","version":"301'14","reported_seq":117,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:07.698763+0000","last_change":"2026-02-24T22:35:07.698724+0000","last_active":"2026-02-24T22:35:07.698763+0000","last_peered":"2026-02-24T22:35:07.698763+0000","last_clean":"2026-02-24T22:35:07.698763+0000","last_became_active":"2026-02-24T22:13:43.754753+0000","last_became_peered":"2026-02-24T22:13:43.754753+0000","last_unstale":"2026-02-24T22:35:07.698763+0000","last_undegraded":"2026-02-24T22:35:07.698763+0000","last_fullsized":"2026-02-24T22:35:07.698763+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"301'14","last_scrub_stamp":"2026-02-24T22:35:07.698665+0000","last_deep_scrub":"301'14","last_deep_scrub_stamp":"2026-02-24T22:35:07.698665+0000","last_clean_scrub_stamp":"2026-02-24T22:35:07.698665+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:07:27.811145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":92,"num_write":34,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"82.18","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:17.534820+0000","last_change":"2026-02-24T22:35:17.534786+0000","last_active":"2026-02-24T22:35:17.534820+0000","last_peered":"2026-02-24T22:35:17.534820+0000","last_clean":"2026-02-24T22:35:17.534820+0000","last_became_active":"2026-02-24T22:23:29.213387+0000","last_became_peered":"2026-02-24T22:23:29.213387+0000","last_unstale":"2026-02-24T22:35:17.534820+0000","last_undegraded":"2026-02-24T22:35:17.534820+0000","last_fullsized":"2026-02-24T22:35:17.534820+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:17.534737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:17.534737+0000","last_clean_scrub_stamp":"2026-02-24T22:35:17.534737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:50:16.258227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"81.1b","version":"0'0","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:10.315609+0000","last_change":"2026-02-24T22:35:10.315555+0000","last_active":"2026-02-24T22:35:10.315609+0000","last_peered":"2026-02-24T22:35:10.315609+0000","last_clean":"2026-02-24T22:35:10.315609+0000","last_became_active":"2026-02-24T22:15:25.832097+0000","last_became_peered":"2026-02-24T22:15:25.832097+0000","last_unstale":"2026-02-24T22:35:10.315609+0000","last_undegraded":"2026-02-24T22:35:10.315609+0000","last_fullsized":"2026-02-24T22:35:10.315609+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:10.315494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:10.315494+0000","last_clean_scrub_stamp":"2026-02-24T22:35:10.315494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:10:42.575778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"80.1a","version":"0'0","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:05.379885+0000","last_change":"2026-02-24T22:35:05.379833+0000","last_active":"2026-02-24T22:35:05.379885+0000","last_peered":"2026-02-24T22:35:05.379885+0000","last_clean":"2026-02-24T22:35:05.379885+0000","last_became_active":"2026-02-24T22:15:22.818885+0000","last_became_peered":"2026-02-24T22:15:22.818885+0000","last_unstale":"2026-02-24T22:35:05.379885+0000","last_undegraded":"2026-02-24T22:35:05.379885+0000","last_fullsized":"2026-02-24T22:35:05.379885+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:05.379757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:05.379757+0000","last_clean_scrub_stamp":"2026-02-24T22:35:05.379757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:37:10.955550+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"79.5","version":"312'19","reported_seq":130,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:07.304569+0000","last_change":"2026-02-24T22:35:07.304569+0000","last_active":"2026-02-24T22:35:07.304569+0000","last_peered":"2026-02-24T22:35:07.304569+0000","last_clean":"2026-02-24T22:35:07.304569+0000","last_became_active":"2026-02-24T22:13:43.755495+0000","last_became_peered":"2026-02-24T22:13:43.755495+0000","last_unstale":"2026-02-24T22:35:07.304569+0000","last_undegraded":"2026-02-24T22:35:07.304569+0000","last_fullsized":"2026-02-24T22:35:07.304569+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'19","last_scrub_stamp":"2026-02-24T22:35:07.304515+0000","last_deep_scrub":"312'19","last_deep_scrub_stamp":"2026-02-24T22:35:07.304515+0000","last_clean_scrub_stamp":"2026-02-24T22:35:07.304515+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:23:18.475886+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":164,"num_read_kb":112,"num_write":47,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"82.19","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:08.227040+0000","last_change":"2026-02-24T22:35:08.226999+0000","last_active":"2026-02-24T22:35:08.227040+0000","last_peered":"2026-02-24T22:35:08.227040+0000","last_clean":"2026-02-24T22:35:08.227040+0000","last_became_active":"2026-02-24T22:23:29.212268+0000","last_became_peered":"2026-02-24T22:23:29.212268+0000","last_unstale":"2026-02-24T22:35:08.227040+0000","last_undegraded":"2026-02-24T22:35:08.227040+0000","last_fullsized":"2026-02-24T22:35:08.227040+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:08.226947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:08.226947+0000","last_clean_scrub_stamp":"2026-02-24T22:35:08.226947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:06:46.348170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.1a","version":"312'3","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:03.754654+0000","last_change":"2026-02-24T22:35:03.754600+0000","last_active":"2026-02-24T22:35:03.754654+0000","last_peered":"2026-02-24T22:35:03.754654+0000","last_clean":"2026-02-24T22:35:03.754654+0000","last_became_active":"2026-02-24T22:15:25.825969+0000","last_became_peered":"2026-02-24T22:15:25.825969+0000","last_unstale":"2026-02-24T22:35:03.754654+0000","last_undegraded":"2026-02-24T22:35:03.754654+0000","last_fullsized":"2026-02-24T22:35:03.754654+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:03.754545+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:03.754545+0000","last_clean_scrub_stamp":"2026-02-24T22:35:03.754545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:00:51.414538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"80.1b","version":"312'3","reported_seq":51,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:20.844174+0000","last_change":"2026-02-24T22:35:20.844132+0000","last_active":"2026-02-24T22:35:20.844174+0000","last_peered":"2026-02-24T22:35:20.844174+0000","last_clean":"2026-02-24T22:35:20.844174+0000","last_became_active":"2026-02-24T22:15:22.816479+0000","last_became_peered":"2026-02-24T22:15:22.816479+0000","last_unstale":"2026-02-24T22:35:20.844174+0000","last_undegraded":"2026-02-24T22:35:20.844174+0000","last_fullsized":"2026-02-24T22:35:20.844174+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:20.844063+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:20.844063+0000","last_clean_scrub_stamp":"2026-02-24T22:35:20.844063+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:11:47.595780+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.4","version":"0'0","reported_seq":79,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:06.735654+0000","last_change":"2026-02-24T22:35:06.735654+0000","last_active":"2026-02-24T22:35:06.735654+0000","last_peered":"2026-02-24T22:35:06.735654+0000","last_clean":"2026-02-24T22:35:06.735654+0000","last_became_active":"2026-02-24T22:14:00.741839+0000","last_became_peered":"2026-02-24T22:14:00.741839+0000","last_unstale":"2026-02-24T22:35:06.735654+0000","last_undegraded":"2026-02-24T22:35:06.735654+0000","last_fullsized":"2026-02-24T22:35:06.735654+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":293,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:06.735602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:06.735602+0000","last_clean_scrub_stamp":"2026-02-24T22:35:06.735602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:33:39.827521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"82.1e","version":"0'0","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:15.299512+0000","last_change":"2026-02-24T22:35:15.299470+0000","last_active":"2026-02-24T22:35:15.299512+0000","last_peered":"2026-02-24T22:35:15.299512+0000","last_clean":"2026-02-24T22:35:15.299512+0000","last_became_active":"2026-02-24T22:23:29.215604+0000","last_became_peered":"2026-02-24T22:23:29.215604+0000","last_unstale":"2026-02-24T22:35:15.299512+0000","last_undegraded":"2026-02-24T22:35:15.299512+0000","last_fullsized":"2026-02-24T22:35:15.299512+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:15.299416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:15.299416+0000","last_clean_scrub_stamp":"2026-02-24T22:35:15.299416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:54:05.836081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.1d","version":"312'3","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:17.008630+0000","last_change":"2026-02-24T22:35:17.008586+0000","last_active":"2026-02-24T22:35:17.008630+0000","last_peered":"2026-02-24T22:35:17.008630+0000","last_clean":"2026-02-24T22:35:17.008630+0000","last_became_active":"2026-02-24T22:15:25.828335+0000","last_became_peered":"2026-02-24T22:15:25.828335+0000","last_unstale":"2026-02-24T22:35:17.008630+0000","last_undegraded":"2026-02-24T22:35:17.008630+0000","last_fullsized":"2026-02-24T22:35:17.008630+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:17.008525+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:17.008525+0000","last_clean_scrub_stamp":"2026-02-24T22:35:17.008525+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:58:43.331071+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"80.1c","version":"0'0","reported_seq":45,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:22.875409+0000","last_change":"2026-02-24T22:35:22.875368+0000","last_active":"2026-02-24T22:35:22.875409+0000","last_peered":"2026-02-24T22:35:22.875409+0000","last_clean":"2026-02-24T22:35:22.875409+0000","last_became_active":"2026-02-24T22:15:22.816548+0000","last_became_peered":"2026-02-24T22:15:22.816548+0000","last_unstale":"2026-02-24T22:35:22.875409+0000","last_undegraded":"2026-02-24T22:35:22.875409+0000","last_fullsized":"2026-02-24T22:35:22.875409+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:22.875303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:22.875303+0000","last_clean_scrub_stamp":"2026-02-24T22:35:22.875303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:36:00.190406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.3","version":"312'6","reported_seq":86,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:55.621542+0000","last_change":"2026-02-24T22:34:55.621542+0000","last_active":"2026-02-24T22:34:55.621542+0000","last_peered":"2026-02-24T22:34:55.621542+0000","last_clean":"2026-02-24T22:34:55.621542+0000","last_became_active":"2026-02-24T22:13:43.756408+0000","last_became_peered":"2026-02-24T22:13:43.756408+0000","last_unstale":"2026-02-24T22:34:55.621542+0000","last_undegraded":"2026-02-24T22:34:55.621542+0000","last_fullsized":"2026-02-24T22:34:55.621542+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'6","last_scrub_stamp":"2026-02-24T22:34:55.621486+0000","last_deep_scrub":"312'6","last_deep_scrub_stamp":"2026-02-24T22:34:55.621486+0000","last_clean_scrub_stamp":"2026-02-24T22:34:55.621486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:43:15.014040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":20,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"82.1f","version":"0'0","reported_seq":30,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:22.633463+0000","last_change":"2026-02-24T22:35:22.633463+0000","last_active":"2026-02-24T22:35:22.633463+0000","last_peered":"2026-02-24T22:35:22.633463+0000","last_clean":"2026-02-24T22:35:22.633463+0000","last_became_active":"2026-02-24T22:23:29.199179+0000","last_became_peered":"2026-02-24T22:23:29.199179+0000","last_unstale":"2026-02-24T22:35:22.633463+0000","last_undegraded":"2026-02-24T22:35:22.633463+0000","last_fullsized":"2026-02-24T22:35:22.633463+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:22.633391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:22.633391+0000","last_clean_scrub_stamp":"2026-02-24T22:35:22.633391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:53:25.454787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"81.1c","version":"0'0","reported_seq":39,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:12.541442+0000","last_change":"2026-02-24T22:35:12.541386+0000","last_active":"2026-02-24T22:35:12.541442+0000","last_peered":"2026-02-24T22:35:12.541442+0000","last_clean":"2026-02-24T22:35:12.541442+0000","last_became_active":"2026-02-24T22:15:25.834072+0000","last_became_peered":"2026-02-24T22:15:25.834072+0000","last_unstale":"2026-02-24T22:35:12.541442+0000","last_undegraded":"2026-02-24T22:35:12.541442+0000","last_fullsized":"2026-02-24T22:35:12.541442+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:12.541294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:12.541294+0000","last_clean_scrub_stamp":"2026-02-24T22:35:12.541294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:17:35.101650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"80.1d","version":"312'2","reported_seq":49,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:21.851696+0000","last_change":"2026-02-24T22:35:21.851623+0000","last_active":"2026-02-24T22:35:21.851696+0000","last_peered":"2026-02-24T22:35:21.851696+0000","last_clean":"2026-02-24T22:35:21.851696+0000","last_became_active":"2026-02-24T22:15:22.816615+0000","last_became_peered":"2026-02-24T22:15:22.816615+0000","last_unstale":"2026-02-24T22:35:21.851696+0000","last_undegraded":"2026-02-24T22:35:21.851696+0000","last_fullsized":"2026-02-24T22:35:21.851696+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:35:21.851526+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:35:21.851526+0000","last_clean_scrub_stamp":"2026-02-24T22:35:21.851526+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:40:55.340169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"79.2","version":"312'5","reported_seq":86,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:07.232682+0000","last_change":"2026-02-24T22:35:07.232682+0000","last_active":"2026-02-24T22:35:07.232682+0000","last_peered":"2026-02-24T22:35:07.232682+0000","last_clean":"2026-02-24T22:35:07.232682+0000","last_became_active":"2026-02-24T22:13:43.755241+0000","last_became_peered":"2026-02-24T22:13:43.755241+0000","last_unstale":"2026-02-24T22:35:07.232682+0000","last_undegraded":"2026-02-24T22:35:07.232682+0000","last_fullsized":"2026-02-24T22:35:07.232682+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'5","last_scrub_stamp":"2026-02-24T22:35:07.232617+0000","last_deep_scrub":"312'5","last_deep_scrub_stamp":"2026-02-24T22:35:07.232617+0000","last_clean_scrub_stamp":"2026-02-24T22:35:07.232617+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:43:07.685953+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":23,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"82.1c","version":"312'2","reported_seq":31,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:09.205907+0000","last_change":"2026-02-24T22:35:09.205907+0000","last_active":"2026-02-24T22:35:09.205907+0000","last_peered":"2026-02-24T22:35:09.205907+0000","last_clean":"2026-02-24T22:35:09.205907+0000","last_became_active":"2026-02-24T22:23:29.213490+0000","last_became_peered":"2026-02-24T22:23:29.213490+0000","last_unstale":"2026-02-24T22:35:09.205907+0000","last_undegraded":"2026-02-24T22:35:09.205907+0000","last_fullsized":"2026-02-24T22:35:09.205907+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:35:09.205819+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:35:09.205819+0000","last_clean_scrub_stamp":"2026-02-24T22:35:09.205819+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:51:15.132591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"81.1f","version":"0'0","reported_seq":40,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:23.304100+0000","last_change":"2026-02-24T22:35:23.304100+0000","last_active":"2026-02-24T22:35:23.304100+0000","last_peered":"2026-02-24T22:35:23.304100+0000","last_clean":"2026-02-24T22:35:23.304100+0000","last_became_active":"2026-02-24T22:15:25.827873+0000","last_became_peered":"2026-02-24T22:15:25.827873+0000","last_unstale":"2026-02-24T22:35:23.304100+0000","last_undegraded":"2026-02-24T22:35:23.304100+0000","last_fullsized":"2026-02-24T22:35:23.304100+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:23.304014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:23.304014+0000","last_clean_scrub_stamp":"2026-02-24T22:35:23.304014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:31:32.463168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"80.1e","version":"0'0","reported_seq":47,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:04.400756+0000","last_change":"2026-02-24T22:35:04.400700+0000","last_active":"2026-02-24T22:35:04.400756+0000","last_peered":"2026-02-24T22:35:04.400756+0000","last_clean":"2026-02-24T22:35:04.400756+0000","last_became_active":"2026-02-24T22:15:22.817337+0000","last_became_peered":"2026-02-24T22:15:22.817337+0000","last_unstale":"2026-02-24T22:35:04.400756+0000","last_undegraded":"2026-02-24T22:35:04.400756+0000","last_fullsized":"2026-02-24T22:35:04.400756+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:04.400607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:04.400607+0000","last_clean_scrub_stamp":"2026-02-24T22:35:04.400607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:40:28.932410+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"79.1","version":"312'23","reported_seq":134,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:00.384907+0000","last_change":"2026-02-24T22:35:00.384907+0000","last_active":"2026-02-24T22:35:00.384907+0000","last_peered":"2026-02-24T22:35:00.384907+0000","last_clean":"2026-02-24T22:35:00.384907+0000","last_became_active":"2026-02-24T22:13:43.760933+0000","last_became_peered":"2026-02-24T22:13:43.760933+0000","last_unstale":"2026-02-24T22:35:00.384907+0000","last_undegraded":"2026-02-24T22:35:00.384907+0000","last_fullsized":"2026-02-24T22:35:00.384907+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'23","last_scrub_stamp":"2026-02-24T22:35:00.384839+0000","last_deep_scrub":"312'23","last_deep_scrub_stamp":"2026-02-24T22:35:00.384839+0000","last_clean_scrub_stamp":"2026-02-24T22:35:00.384839+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:47:05.415176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":174,"num_read_kb":113,"num_write":59,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"82.1d","version":"0'0","reported_seq":29,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:14.322423+0000","last_change":"2026-02-24T22:35:14.322423+0000","last_active":"2026-02-24T22:35:14.322423+0000","last_peered":"2026-02-24T22:35:14.322423+0000","last_clean":"2026-02-24T22:35:14.322423+0000","last_became_active":"2026-02-24T22:23:29.215334+0000","last_became_peered":"2026-02-24T22:23:29.215334+0000","last_unstale":"2026-02-24T22:35:14.322423+0000","last_undegraded":"2026-02-24T22:35:14.322423+0000","last_fullsized":"2026-02-24T22:35:14.322423+0000","mapping_epoch":308,"log_start":"0'0","ondisk_log_start":"0'0","created":308,"last_epoch_clean":309,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:14.322351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:14.322351+0000","last_clean_scrub_stamp":"2026-02-24T22:35:14.322351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:09:08.411510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"81.1e","version":"0'0","reported_seq":41,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:04.737700+0000","last_change":"2026-02-24T22:35:04.737646+0000","last_active":"2026-02-24T22:35:04.737700+0000","last_peered":"2026-02-24T22:35:04.737700+0000","last_clean":"2026-02-24T22:35:04.737700+0000","last_became_active":"2026-02-24T22:15:25.827124+0000","last_became_peered":"2026-02-24T22:15:25.827124+0000","last_unstale":"2026-02-24T22:35:04.737700+0000","last_undegraded":"2026-02-24T22:35:04.737700+0000","last_fullsized":"2026-02-24T22:35:04.737700+0000","mapping_epoch":305,"log_start":"0'0","ondisk_log_start":"0'0","created":305,"last_epoch_clean":306,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:04.737570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:04.737570+0000","last_clean_scrub_stamp":"2026-02-24T22:35:04.737570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:38:11.206092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"80.1f","version":"312'3","reported_seq":52,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:02.210357+0000","last_change":"2026-02-24T22:35:02.210357+0000","last_active":"2026-02-24T22:35:02.210357+0000","last_peered":"2026-02-24T22:35:02.210357+0000","last_clean":"2026-02-24T22:35:02.210357+0000","last_became_active":"2026-02-24T22:15:22.815938+0000","last_became_peered":"2026-02-24T22:15:22.815938+0000","last_unstale":"2026-02-24T22:35:02.210357+0000","last_undegraded":"2026-02-24T22:35:02.210357+0000","last_fullsized":"2026-02-24T22:35:02.210357+0000","mapping_epoch":302,"log_start":"0'0","ondisk_log_start":"0'0","created":302,"last_epoch_clean":303,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:35:02.210297+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:35:02.210297+0000","last_clean_scrub_stamp":"2026-02-24T22:35:02.210297+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:26:34.542452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"79.0","version":"0'0","reported_seq":74,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:05.263812+0000","last_change":"2026-02-24T22:35:05.263755+0000","last_active":"2026-02-24T22:35:05.263812+0000","last_peered":"2026-02-24T22:35:05.263812+0000","last_clean":"2026-02-24T22:35:05.263812+0000","last_became_active":"2026-02-24T22:13:43.755266+0000","last_became_peered":"2026-02-24T22:13:43.755266+0000","last_unstale":"2026-02-24T22:35:05.263812+0000","last_undegraded":"2026-02-24T22:35:05.263812+0000","last_fullsized":"2026-02-24T22:35:05.263812+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":289,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:05.263697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:05.263697+0000","last_clean_scrub_stamp":"2026-02-24T22:35:05.263697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:19:26.298794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"312'276","reported_seq":990,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.099244+0000","last_change":"2026-02-24T22:34:44.099244+0000","last_active":"2026-02-24T22:34:44.099244+0000","last_peered":"2026-02-24T22:34:44.099244+0000","last_clean":"2026-02-24T22:34:44.099244+0000","last_became_active":"2026-02-24T22:05:16.192589+0000","last_became_peered":"2026-02-24T22:05:16.192589+0000","last_unstale":"2026-02-24T22:34:44.099244+0000","last_undegraded":"2026-02-24T22:34:44.099244+0000","last_fullsized":"2026-02-24T22:34:44.099244+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'276","last_scrub_stamp":"2026-02-24T22:34:44.099189+0000","last_deep_scrub":"312'276","last_deep_scrub_stamp":"2026-02-24T22:34:44.099189+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.099189+0000","objects_scrubbed":5,"log_size":76,"log_dups_size":200,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:12:08.217874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":433,"num_read_kb":342,"num_write":1269,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"312'220","reported_seq":931,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.919442+0000","last_change":"2026-02-24T22:34:46.919413+0000","last_active":"2026-02-24T22:34:46.919442+0000","last_peered":"2026-02-24T22:34:46.919442+0000","last_clean":"2026-02-24T22:34:46.919442+0000","last_became_active":"2026-02-24T22:05:18.192203+0000","last_became_peered":"2026-02-24T22:05:18.192203+0000","last_unstale":"2026-02-24T22:34:46.919442+0000","last_undegraded":"2026-02-24T22:34:46.919442+0000","last_fullsized":"2026-02-24T22:34:46.919442+0000","mapping_epoch":24,"log_start":"312'197","ondisk_log_start":"312'197","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'220","last_scrub_stamp":"2026-02-24T22:34:46.919375+0000","last_deep_scrub":"312'220","last_deep_scrub_stamp":"2026-02-24T22:34:46.919375+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.919375+0000","objects_scrubbed":23,"log_size":23,"log_dups_size":197,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:53:24.900487+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":780,"num_read_kb":643,"num_write":282,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"312'380","reported_seq":1197,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.032775+0000","last_change":"2026-02-24T22:34:36.032775+0000","last_active":"2026-02-24T22:34:36.032775+0000","last_peered":"2026-02-24T22:34:36.032775+0000","last_clean":"2026-02-24T22:34:36.032775+0000","last_became_active":"2026-02-24T22:05:16.181379+0000","last_became_peered":"2026-02-24T22:05:16.181379+0000","last_unstale":"2026-02-24T22:34:36.032775+0000","last_undegraded":"2026-02-24T22:34:36.032775+0000","last_fullsized":"2026-02-24T22:34:36.032775+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'380","last_scrub_stamp":"2026-02-24T22:34:36.032715+0000","last_deep_scrub":"312'380","last_deep_scrub_stamp":"2026-02-24T22:34:36.032715+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.032715+0000","objects_scrubbed":2,"log_size":80,"log_dups_size":300,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:22:39.124170+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":922,"num_read_kb":611,"num_write":1540,"num_write_kb":5240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"312'487","reported_seq":1295,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.222374+0000","last_change":"2026-02-24T22:34:47.222322+0000","last_active":"2026-02-24T22:34:47.222374+0000","last_peered":"2026-02-24T22:34:47.222374+0000","last_clean":"2026-02-24T22:34:47.222374+0000","last_became_active":"2026-02-24T22:05:18.190689+0000","last_became_peered":"2026-02-24T22:05:18.190689+0000","last_unstale":"2026-02-24T22:34:47.222374+0000","last_undegraded":"2026-02-24T22:34:47.222374+0000","last_fullsized":"2026-02-24T22:34:47.222374+0000","mapping_epoch":24,"log_start":"312'400","ondisk_log_start":"312'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'487","last_scrub_stamp":"2026-02-24T22:34:47.222271+0000","last_deep_scrub":"312'487","last_deep_scrub_stamp":"2026-02-24T22:34:47.222271+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.222271+0000","objects_scrubbed":28,"log_size":87,"log_dups_size":400,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:40:22.630753+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":2017,"num_read_kb":1670,"num_write":788,"num_write_kb":502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"312'453","reported_seq":1248,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.730366+0000","last_change":"2026-02-24T22:34:38.730366+0000","last_active":"2026-02-24T22:34:38.730366+0000","last_peered":"2026-02-24T22:34:38.730366+0000","last_clean":"2026-02-24T22:34:38.730366+0000","last_became_active":"2026-02-24T22:05:16.180681+0000","last_became_peered":"2026-02-24T22:05:16.180681+0000","last_unstale":"2026-02-24T22:34:38.730366+0000","last_undegraded":"2026-02-24T22:34:38.730366+0000","last_fullsized":"2026-02-24T22:34:38.730366+0000","mapping_epoch":22,"log_start":"312'400","ondisk_log_start":"312'400","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'453","last_scrub_stamp":"2026-02-24T22:34:38.730307+0000","last_deep_scrub":"312'453","last_deep_scrub_stamp":"2026-02-24T22:34:38.730307+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.730307+0000","objects_scrubbed":5,"log_size":53,"log_dups_size":400,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:09:15.011742+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242910,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":903,"num_read_kb":6172,"num_write":1607,"num_write_kb":5459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"312'102","reported_seq":759,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.416426+0000","last_change":"2026-02-24T22:34:48.416387+0000","last_active":"2026-02-24T22:34:48.416426+0000","last_peered":"2026-02-24T22:34:48.416426+0000","last_clean":"2026-02-24T22:34:48.416426+0000","last_became_active":"2026-02-24T22:05:18.190727+0000","last_became_peered":"2026-02-24T22:05:18.190727+0000","last_unstale":"2026-02-24T22:34:48.416426+0000","last_undegraded":"2026-02-24T22:34:48.416426+0000","last_fullsized":"2026-02-24T22:34:48.416426+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'102","last_scrub_stamp":"2026-02-24T22:34:48.416304+0000","last_deep_scrub":"312'102","last_deep_scrub_stamp":"2026-02-24T22:34:48.416304+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.416304+0000","objects_scrubbed":22,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:05:48.774357+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":268,"num_read_kb":220,"num_write":112,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"312'294","reported_seq":987,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.956302+0000","last_change":"2026-02-24T22:34:43.956244+0000","last_active":"2026-02-24T22:34:43.956302+0000","last_peered":"2026-02-24T22:34:43.956302+0000","last_clean":"2026-02-24T22:34:43.956302+0000","last_became_active":"2026-02-24T22:05:16.192922+0000","last_became_peered":"2026-02-24T22:05:16.192922+0000","last_unstale":"2026-02-24T22:34:43.956302+0000","last_undegraded":"2026-02-24T22:34:43.956302+0000","last_fullsized":"2026-02-24T22:34:43.956302+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'294","last_scrub_stamp":"2026-02-24T22:34:43.956165+0000","last_deep_scrub":"312'294","last_deep_scrub_stamp":"2026-02-24T22:34:43.956165+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.956165+0000","objects_scrubbed":5,"log_size":94,"log_dups_size":200,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:47:34.541512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":378,"num_read_kb":3190,"num_write":1202,"num_write_kb":12582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"312'158","reported_seq":848,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.908280+0000","last_change":"2026-02-24T22:34:47.908235+0000","last_active":"2026-02-24T22:34:47.908280+0000","last_peered":"2026-02-24T22:34:47.908280+0000","last_clean":"2026-02-24T22:34:47.908280+0000","last_became_active":"2026-02-24T22:05:18.184014+0000","last_became_peered":"2026-02-24T22:05:18.184014+0000","last_unstale":"2026-02-24T22:34:47.908280+0000","last_undegraded":"2026-02-24T22:34:47.908280+0000","last_fullsized":"2026-02-24T22:34:47.908280+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'158","last_scrub_stamp":"2026-02-24T22:34:47.908167+0000","last_deep_scrub":"312'158","last_deep_scrub_stamp":"2026-02-24T22:34:47.908167+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.908167+0000","objects_scrubbed":23,"log_size":58,"log_dups_size":100,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:57:20.364094+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":489,"num_read_kb":401,"num_write":208,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"312'280","reported_seq":1000,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.068519+0000","last_change":"2026-02-24T22:34:43.068430+0000","last_active":"2026-02-24T22:34:43.068519+0000","last_peered":"2026-02-24T22:34:43.068519+0000","last_clean":"2026-02-24T22:34:43.068519+0000","last_became_active":"2026-02-24T22:05:16.191055+0000","last_became_peered":"2026-02-24T22:05:16.191055+0000","last_unstale":"2026-02-24T22:34:43.068519+0000","last_undegraded":"2026-02-24T22:34:43.068519+0000","last_fullsized":"2026-02-24T22:34:43.068519+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'280","last_scrub_stamp":"2026-02-24T22:34:43.068300+0000","last_deep_scrub":"312'280","last_deep_scrub_stamp":"2026-02-24T22:34:43.068300+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.068300+0000","objects_scrubbed":9,"log_size":80,"log_dups_size":200,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:55:12.515669+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":135,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":423,"num_read_kb":15420,"num_write":1201,"num_write_kb":8454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"312'497","reported_seq":1223,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.721601+0000","last_change":"2026-02-24T22:34:42.721550+0000","last_active":"2026-02-24T22:34:42.721601+0000","last_peered":"2026-02-24T22:34:42.721601+0000","last_clean":"2026-02-24T22:34:42.721601+0000","last_became_active":"2026-02-24T22:05:18.190598+0000","last_became_peered":"2026-02-24T22:05:18.190598+0000","last_unstale":"2026-02-24T22:34:42.721601+0000","last_undegraded":"2026-02-24T22:34:42.721601+0000","last_fullsized":"2026-02-24T22:34:42.721601+0000","mapping_epoch":24,"log_start":"312'400","ondisk_log_start":"312'400","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'497","last_scrub_stamp":"2026-02-24T22:34:42.721473+0000","last_deep_scrub":"312'497","last_deep_scrub_stamp":"2026-02-24T22:34:42.721473+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.721473+0000","objects_scrubbed":27,"log_size":97,"log_dups_size":400,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:09:26.432917+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1905,"num_read_kb":1522,"num_write":772,"num_write_kb":389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":267,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"312'306","reported_seq":1025,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.065368+0000","last_change":"2026-02-24T22:34:41.065317+0000","last_active":"2026-02-24T22:34:41.065368+0000","last_peered":"2026-02-24T22:34:41.065368+0000","last_clean":"2026-02-24T22:34:41.065368+0000","last_became_active":"2026-02-24T22:05:16.191164+0000","last_became_peered":"2026-02-24T22:05:16.191164+0000","last_unstale":"2026-02-24T22:34:41.065368+0000","last_undegraded":"2026-02-24T22:34:41.065368+0000","last_fullsized":"2026-02-24T22:34:41.065368+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'306","last_scrub_stamp":"2026-02-24T22:34:41.065249+0000","last_deep_scrub":"312'306","last_deep_scrub_stamp":"2026-02-24T22:34:41.065249+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.065249+0000","objects_scrubbed":6,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:42:26.339639+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":491,"num_read_kb":5551,"num_write":1337,"num_write_kb":5240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"312'74","reported_seq":737,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:50.335113+0000","last_change":"2026-02-24T22:34:50.335113+0000","last_active":"2026-02-24T22:34:50.335113+0000","last_peered":"2026-02-24T22:34:50.335113+0000","last_clean":"2026-02-24T22:34:50.335113+0000","last_became_active":"2026-02-24T22:05:18.198559+0000","last_became_peered":"2026-02-24T22:05:18.198559+0000","last_unstale":"2026-02-24T22:34:50.335113+0000","last_undegraded":"2026-02-24T22:34:50.335113+0000","last_fullsized":"2026-02-24T22:34:50.335113+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'74","last_scrub_stamp":"2026-02-24T22:34:50.335079+0000","last_deep_scrub":"312'74","last_deep_scrub_stamp":"2026-02-24T22:34:50.335079+0000","last_clean_scrub_stamp":"2026-02-24T22:34:50.335079+0000","objects_scrubbed":18,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:21:37.995762+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":195,"num_read_kb":167,"num_write":70,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"312'304","reported_seq":1013,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.096476+0000","last_change":"2026-02-24T22:34:42.096430+0000","last_active":"2026-02-24T22:34:42.096476+0000","last_peered":"2026-02-24T22:34:42.096476+0000","last_clean":"2026-02-24T22:34:42.096476+0000","last_became_active":"2026-02-24T22:05:16.190367+0000","last_became_peered":"2026-02-24T22:05:16.190367+0000","last_unstale":"2026-02-24T22:34:42.096476+0000","last_undegraded":"2026-02-24T22:34:42.096476+0000","last_fullsized":"2026-02-24T22:34:42.096476+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'304","last_scrub_stamp":"2026-02-24T22:34:42.096375+0000","last_deep_scrub":"312'304","last_deep_scrub_stamp":"2026-02-24T22:34:42.096375+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.096375+0000","objects_scrubbed":4,"log_size":104,"log_dups_size":200,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:48:33.407036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":439,"num_read_kb":1335,"num_write":1233,"num_write_kb":1104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"312'237","reported_seq":988,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.882613+0000","last_change":"2026-02-24T22:34:48.882613+0000","last_active":"2026-02-24T22:34:48.882613+0000","last_peered":"2026-02-24T22:34:48.882613+0000","last_clean":"2026-02-24T22:34:48.882613+0000","last_became_active":"2026-02-24T22:05:18.191767+0000","last_became_peered":"2026-02-24T22:05:18.191767+0000","last_unstale":"2026-02-24T22:34:48.882613+0000","last_undegraded":"2026-02-24T22:34:48.882613+0000","last_fullsized":"2026-02-24T22:34:48.882613+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'237","last_scrub_stamp":"2026-02-24T22:34:48.882561+0000","last_deep_scrub":"312'237","last_deep_scrub_stamp":"2026-02-24T22:34:48.882561+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.882561+0000","objects_scrubbed":36,"log_size":37,"log_dups_size":200,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:24:34.587031+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":876,"num_read_kb":754,"num_write":323,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"312'364","reported_seq":1143,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.830560+0000","last_change":"2026-02-24T22:34:36.830513+0000","last_active":"2026-02-24T22:34:36.830560+0000","last_peered":"2026-02-24T22:34:36.830560+0000","last_clean":"2026-02-24T22:34:36.830560+0000","last_became_active":"2026-02-24T22:05:16.185779+0000","last_became_peered":"2026-02-24T22:05:16.185779+0000","last_unstale":"2026-02-24T22:34:36.830560+0000","last_undegraded":"2026-02-24T22:34:36.830560+0000","last_fullsized":"2026-02-24T22:34:36.830560+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'364","last_scrub_stamp":"2026-02-24T22:34:36.830446+0000","last_deep_scrub":"312'364","last_deep_scrub_stamp":"2026-02-24T22:34:36.830446+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.830446+0000","objects_scrubbed":6,"log_size":64,"log_dups_size":300,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:49:17.279372+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":619,"num_read_kb":883,"num_write":1617,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"312'109","reported_seq":765,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.090042+0000","last_change":"2026-02-24T22:34:41.090042+0000","last_active":"2026-02-24T22:34:41.090042+0000","last_peered":"2026-02-24T22:34:41.090042+0000","last_clean":"2026-02-24T22:34:41.090042+0000","last_became_active":"2026-02-24T22:05:18.187838+0000","last_became_peered":"2026-02-24T22:05:18.187838+0000","last_unstale":"2026-02-24T22:34:41.090042+0000","last_undegraded":"2026-02-24T22:34:41.090042+0000","last_fullsized":"2026-02-24T22:34:41.090042+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'109","last_scrub_stamp":"2026-02-24T22:34:41.089971+0000","last_deep_scrub":"312'109","last_deep_scrub_stamp":"2026-02-24T22:34:41.089971+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.089971+0000","objects_scrubbed":26,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:13:12.250060+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":263,"num_read_kb":217,"num_write":106,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":719,"num_omap_keys":3,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"312'164","reported_seq":847,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.431756+0000","last_change":"2026-02-24T22:34:47.431711+0000","last_active":"2026-02-24T22:34:47.431756+0000","last_peered":"2026-02-24T22:34:47.431756+0000","last_clean":"2026-02-24T22:34:47.431756+0000","last_became_active":"2026-02-24T22:05:18.191625+0000","last_became_peered":"2026-02-24T22:05:18.191625+0000","last_unstale":"2026-02-24T22:34:47.431756+0000","last_undegraded":"2026-02-24T22:34:47.431756+0000","last_fullsized":"2026-02-24T22:34:47.431756+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'164","last_scrub_stamp":"2026-02-24T22:34:47.431652+0000","last_deep_scrub":"312'164","last_deep_scrub_stamp":"2026-02-24T22:34:47.431652+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.431652+0000","objects_scrubbed":29,"log_size":64,"log_dups_size":100,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:13:36.629505+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":477,"num_read_kb":396,"num_write":183,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"312'302","reported_seq":1047,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.046802+0000","last_change":"2026-02-24T22:34:41.046802+0000","last_active":"2026-02-24T22:34:41.046802+0000","last_peered":"2026-02-24T22:34:41.046802+0000","last_clean":"2026-02-24T22:34:41.046802+0000","last_became_active":"2026-02-24T22:06:01.025659+0000","last_became_peered":"2026-02-24T22:06:01.025659+0000","last_unstale":"2026-02-24T22:34:41.046802+0000","last_undegraded":"2026-02-24T22:34:41.046802+0000","last_fullsized":"2026-02-24T22:34:41.046802+0000","mapping_epoch":36,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'302","last_scrub_stamp":"2026-02-24T22:34:41.046752+0000","last_deep_scrub":"312'302","last_deep_scrub_stamp":"2026-02-24T22:34:41.046752+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.046752+0000","objects_scrubbed":2,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:23:49.712428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":514,"num_read_kb":16221,"num_write":1317,"num_write_kb":8500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"312'4268","reported_seq":5088,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.193784+0000","last_change":"2026-02-24T22:34:46.193741+0000","last_active":"2026-02-24T22:34:46.193784+0000","last_peered":"2026-02-24T22:34:46.193784+0000","last_clean":"2026-02-24T22:34:46.193784+0000","last_became_active":"2026-02-24T22:05:18.193257+0000","last_became_peered":"2026-02-24T22:05:18.193257+0000","last_unstale":"2026-02-24T22:34:46.193784+0000","last_undegraded":"2026-02-24T22:34:46.193784+0000","last_fullsized":"2026-02-24T22:34:46.193784+0000","mapping_epoch":24,"log_start":"312'4200","ondisk_log_start":"312'4200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'4268","last_scrub_stamp":"2026-02-24T22:34:46.193682+0000","last_deep_scrub":"312'4268","last_deep_scrub_stamp":"2026-02-24T22:34:46.193682+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.193682+0000","objects_scrubbed":19,"log_size":68,"log_dups_size":3000,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:22:49.449922+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":17332,"num_read_kb":14553,"num_write":6398,"num_write_kb":3198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"312'289","reported_seq":1009,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.725396+0000","last_change":"2026-02-24T22:34:37.725396+0000","last_active":"2026-02-24T22:34:37.725396+0000","last_peered":"2026-02-24T22:34:37.725396+0000","last_clean":"2026-02-24T22:34:37.725396+0000","last_became_active":"2026-02-24T22:05:16.190819+0000","last_became_peered":"2026-02-24T22:05:16.190819+0000","last_unstale":"2026-02-24T22:34:37.725396+0000","last_undegraded":"2026-02-24T22:34:37.725396+0000","last_fullsized":"2026-02-24T22:34:37.725396+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'289","last_scrub_stamp":"2026-02-24T22:34:37.725293+0000","last_deep_scrub":"312'289","last_deep_scrub_stamp":"2026-02-24T22:34:37.725293+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.725293+0000","objects_scrubbed":5,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:58:32.616383+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242922,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":466,"num_read_kb":8113,"num_write":1161,"num_write_kb":5510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"312'883","reported_seq":4055,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.966979+0000","last_change":"2026-02-24T22:34:45.966939+0000","last_active":"2026-02-24T22:34:45.966979+0000","last_peered":"2026-02-24T22:34:45.966979+0000","last_clean":"2026-02-24T22:34:45.966979+0000","last_became_active":"2026-02-24T22:05:18.190363+0000","last_became_peered":"2026-02-24T22:05:18.190363+0000","last_unstale":"2026-02-24T22:34:45.966979+0000","last_undegraded":"2026-02-24T22:34:45.966979+0000","last_fullsized":"2026-02-24T22:34:45.966979+0000","mapping_epoch":24,"log_start":"312'800","ondisk_log_start":"312'800","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'883","last_scrub_stamp":"2026-02-24T22:34:45.966870+0000","last_deep_scrub":"312'883","last_deep_scrub_stamp":"2026-02-24T22:34:45.966870+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.966870+0000","objects_scrubbed":21,"log_size":83,"log_dups_size":800,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:41:10.952477+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4747,"num_read_kb":5785,"num_write":398,"num_write_kb":236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"312'323","reported_seq":1062,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.725432+0000","last_change":"2026-02-24T22:34:35.725432+0000","last_active":"2026-02-24T22:34:35.725432+0000","last_peered":"2026-02-24T22:34:35.725432+0000","last_clean":"2026-02-24T22:34:35.725432+0000","last_became_active":"2026-02-24T22:05:16.190816+0000","last_became_peered":"2026-02-24T22:05:16.190816+0000","last_unstale":"2026-02-24T22:34:35.725432+0000","last_undegraded":"2026-02-24T22:34:35.725432+0000","last_fullsized":"2026-02-24T22:34:35.725432+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'323","last_scrub_stamp":"2026-02-24T22:34:35.725350+0000","last_deep_scrub":"312'323","last_deep_scrub_stamp":"2026-02-24T22:34:35.725350+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.725350+0000","objects_scrubbed":2,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:35:43.171065+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":544,"num_read_kb":2327,"num_write":1304,"num_write_kb":12235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"312'120","reported_seq":832,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.062558+0000","last_change":"2026-02-24T22:34:38.062558+0000","last_active":"2026-02-24T22:34:38.062558+0000","last_peered":"2026-02-24T22:34:38.062558+0000","last_clean":"2026-02-24T22:34:38.062558+0000","last_became_active":"2026-02-24T22:05:18.186994+0000","last_became_peered":"2026-02-24T22:05:18.186994+0000","last_unstale":"2026-02-24T22:34:38.062558+0000","last_undegraded":"2026-02-24T22:34:38.062558+0000","last_fullsized":"2026-02-24T22:34:38.062558+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'120","last_scrub_stamp":"2026-02-24T22:34:38.062523+0000","last_deep_scrub":"312'120","last_deep_scrub_stamp":"2026-02-24T22:34:38.062523+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.062523+0000","objects_scrubbed":27,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:14:19.882135+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":352,"num_read_kb":306,"num_write":116,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"312'386","reported_seq":1186,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.515887+0000","last_change":"2026-02-24T22:34:34.515887+0000","last_active":"2026-02-24T22:34:34.515887+0000","last_peered":"2026-02-24T22:34:34.515887+0000","last_clean":"2026-02-24T22:34:34.515887+0000","last_became_active":"2026-02-24T22:05:16.188658+0000","last_became_peered":"2026-02-24T22:05:16.188658+0000","last_unstale":"2026-02-24T22:34:34.515887+0000","last_undegraded":"2026-02-24T22:34:34.515887+0000","last_fullsized":"2026-02-24T22:34:34.515887+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'386","last_scrub_stamp":"2026-02-24T22:34:34.515812+0000","last_deep_scrub":"312'386","last_deep_scrub_stamp":"2026-02-24T22:34:34.515812+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.515812+0000","objects_scrubbed":6,"log_size":86,"log_dups_size":300,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:26:30.161641+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":840,"num_read_kb":685,"num_write":1819,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"312'198","reported_seq":929,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.065984+0000","last_change":"2026-02-24T22:34:40.065984+0000","last_active":"2026-02-24T22:34:40.065984+0000","last_peered":"2026-02-24T22:34:40.065984+0000","last_clean":"2026-02-24T22:34:40.065984+0000","last_became_active":"2026-02-24T22:05:18.187682+0000","last_became_peered":"2026-02-24T22:05:18.187682+0000","last_unstale":"2026-02-24T22:34:40.065984+0000","last_undegraded":"2026-02-24T22:34:40.065984+0000","last_fullsized":"2026-02-24T22:34:40.065984+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'198","last_scrub_stamp":"2026-02-24T22:34:40.065916+0000","last_deep_scrub":"312'198","last_deep_scrub_stamp":"2026-02-24T22:34:40.065916+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.065916+0000","objects_scrubbed":29,"log_size":98,"log_dups_size":100,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:30:25.873043+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":639,"num_read_kb":531,"num_write":250,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"312'296","reported_seq":1004,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.790665+0000","last_change":"2026-02-24T22:34:37.790665+0000","last_active":"2026-02-24T22:34:37.790665+0000","last_peered":"2026-02-24T22:34:37.790665+0000","last_clean":"2026-02-24T22:34:37.790665+0000","last_became_active":"2026-02-24T22:05:16.184209+0000","last_became_peered":"2026-02-24T22:05:16.184209+0000","last_unstale":"2026-02-24T22:34:37.790665+0000","last_undegraded":"2026-02-24T22:34:37.790665+0000","last_fullsized":"2026-02-24T22:34:37.790665+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'296","last_scrub_stamp":"2026-02-24T22:34:37.790602+0000","last_deep_scrub":"312'296","last_deep_scrub_stamp":"2026-02-24T22:34:37.790602+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.790602+0000","objects_scrubbed":9,"log_size":96,"log_dups_size":200,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:28:56.968188+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":416,"num_read_kb":603,"num_write":1232,"num_write_kb":10636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"312'131","reported_seq":818,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.372179+0000","last_change":"2026-02-24T22:34:49.372179+0000","last_active":"2026-02-24T22:34:49.372179+0000","last_peered":"2026-02-24T22:34:49.372179+0000","last_clean":"2026-02-24T22:34:49.372179+0000","last_became_active":"2026-02-24T22:05:18.193320+0000","last_became_peered":"2026-02-24T22:05:18.193320+0000","last_unstale":"2026-02-24T22:34:49.372179+0000","last_undegraded":"2026-02-24T22:34:49.372179+0000","last_fullsized":"2026-02-24T22:34:49.372179+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'131","last_scrub_stamp":"2026-02-24T22:34:49.372116+0000","last_deep_scrub":"312'131","last_deep_scrub_stamp":"2026-02-24T22:34:49.372116+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.372116+0000","objects_scrubbed":20,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:50:18.339691+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":425,"num_read_kb":355,"num_write":146,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"312'285","reported_seq":1007,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.747205+0000","last_change":"2026-02-24T22:34:37.747158+0000","last_active":"2026-02-24T22:34:37.747205+0000","last_peered":"2026-02-24T22:34:37.747205+0000","last_clean":"2026-02-24T22:34:37.747205+0000","last_became_active":"2026-02-24T22:05:16.181527+0000","last_became_peered":"2026-02-24T22:05:16.181527+0000","last_unstale":"2026-02-24T22:34:37.747205+0000","last_undegraded":"2026-02-24T22:34:37.747205+0000","last_fullsized":"2026-02-24T22:34:37.747205+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'285","last_scrub_stamp":"2026-02-24T22:34:37.747092+0000","last_deep_scrub":"312'285","last_deep_scrub_stamp":"2026-02-24T22:34:37.747092+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.747092+0000","objects_scrubbed":4,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:43:04.672418+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242895,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":474,"num_read_kb":5463,"num_write":1217,"num_write_kb":5189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":584,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:06.258262+0000","last_change":"2026-02-24T22:35:06.258262+0000","last_active":"2026-02-24T22:35:06.258262+0000","last_peered":"2026-02-24T22:35:06.258262+0000","last_clean":"2026-02-24T22:35:06.258262+0000","last_became_active":"2026-02-24T22:05:29.657764+0000","last_became_peered":"2026-02-24T22:05:29.657764+0000","last_unstale":"2026-02-24T22:35:06.258262+0000","last_undegraded":"2026-02-24T22:35:06.258262+0000","last_fullsized":"2026-02-24T22:35:06.258262+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:06.258189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:06.258189+0000","last_clean_scrub_stamp":"2026-02-24T22:35:06.258189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:18:02.619099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"312'107","reported_seq":771,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.757813+0000","last_change":"2026-02-24T22:34:42.757778+0000","last_active":"2026-02-24T22:34:42.757813+0000","last_peered":"2026-02-24T22:34:42.757813+0000","last_clean":"2026-02-24T22:34:42.757813+0000","last_became_active":"2026-02-24T22:05:18.184771+0000","last_became_peered":"2026-02-24T22:05:18.184771+0000","last_unstale":"2026-02-24T22:34:42.757813+0000","last_undegraded":"2026-02-24T22:34:42.757813+0000","last_fullsized":"2026-02-24T22:34:42.757813+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'107","last_scrub_stamp":"2026-02-24T22:34:42.757725+0000","last_deep_scrub":"312'107","last_deep_scrub_stamp":"2026-02-24T22:34:42.757725+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.757725+0000","objects_scrubbed":27,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:52:33.731233+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":246,"num_read_kb":208,"num_write":102,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"312'344","reported_seq":1134,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.109780+0000","last_change":"2026-02-24T22:34:37.109736+0000","last_active":"2026-02-24T22:34:37.109780+0000","last_peered":"2026-02-24T22:34:37.109780+0000","last_clean":"2026-02-24T22:34:37.109780+0000","last_became_active":"2026-02-24T22:05:16.191841+0000","last_became_peered":"2026-02-24T22:05:16.191841+0000","last_unstale":"2026-02-24T22:34:37.109780+0000","last_undegraded":"2026-02-24T22:34:37.109780+0000","last_fullsized":"2026-02-24T22:34:37.109780+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'344","last_scrub_stamp":"2026-02-24T22:34:37.109689+0000","last_deep_scrub":"312'344","last_deep_scrub_stamp":"2026-02-24T22:34:37.109689+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.109689+0000","objects_scrubbed":4,"log_size":44,"log_dups_size":300,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:35:53.603402+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":614,"num_read_kb":81372,"num_write":1329,"num_write_kb":20589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":587,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:50.867824+0000","last_change":"2026-02-24T22:34:50.867824+0000","last_active":"2026-02-24T22:34:50.867824+0000","last_peered":"2026-02-24T22:34:50.867824+0000","last_clean":"2026-02-24T22:34:50.867824+0000","last_became_active":"2026-02-24T22:05:20.194031+0000","last_became_peered":"2026-02-24T22:05:20.194031+0000","last_unstale":"2026-02-24T22:34:50.867824+0000","last_undegraded":"2026-02-24T22:34:50.867824+0000","last_fullsized":"2026-02-24T22:34:50.867824+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:50.867723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:50.867723+0000","last_clean_scrub_stamp":"2026-02-24T22:34:50.867723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:53:47.367923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"312'262","reported_seq":910,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:53.926384+0000","last_change":"2026-02-24T22:34:53.926384+0000","last_active":"2026-02-24T22:34:53.926384+0000","last_peered":"2026-02-24T22:34:53.926384+0000","last_clean":"2026-02-24T22:34:53.926384+0000","last_became_active":"2026-02-24T22:05:22.191928+0000","last_became_peered":"2026-02-24T22:05:22.191928+0000","last_unstale":"2026-02-24T22:34:53.926384+0000","last_undegraded":"2026-02-24T22:34:53.926384+0000","last_fullsized":"2026-02-24T22:34:53.926384+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'262","last_scrub_stamp":"2026-02-24T22:34:53.926319+0000","last_deep_scrub":"312'262","last_deep_scrub_stamp":"2026-02-24T22:34:53.926319+0000","last_clean_scrub_stamp":"2026-02-24T22:34:53.926319+0000","objects_scrubbed":4,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:38:02.939594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44919,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":413,"num_read_kb":914,"num_write":358,"num_write_kb":269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"312'258","reported_seq":861,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:02.251182+0000","last_change":"2026-02-24T22:35:02.251182+0000","last_active":"2026-02-24T22:35:02.251182+0000","last_peered":"2026-02-24T22:35:02.251182+0000","last_clean":"2026-02-24T22:35:02.251182+0000","last_became_active":"2026-02-24T22:05:24.204990+0000","last_became_peered":"2026-02-24T22:05:24.204990+0000","last_unstale":"2026-02-24T22:35:02.251182+0000","last_undegraded":"2026-02-24T22:35:02.251182+0000","last_fullsized":"2026-02-24T22:35:02.251182+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'258","last_scrub_stamp":"2026-02-24T22:35:02.251119+0000","last_deep_scrub":"312'258","last_deep_scrub_stamp":"2026-02-24T22:35:02.251119+0000","last_clean_scrub_stamp":"2026-02-24T22:35:02.251119+0000","objects_scrubbed":100,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:22:28.932537+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32373,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":92,"num_read_kb":87,"num_write":391,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":573,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:04.809580+0000","last_change":"2026-02-24T22:35:04.809580+0000","last_active":"2026-02-24T22:35:04.809580+0000","last_peered":"2026-02-24T22:35:04.809580+0000","last_clean":"2026-02-24T22:35:04.809580+0000","last_became_active":"2026-02-24T22:05:29.661747+0000","last_became_peered":"2026-02-24T22:05:29.661747+0000","last_unstale":"2026-02-24T22:35:04.809580+0000","last_undegraded":"2026-02-24T22:35:04.809580+0000","last_fullsized":"2026-02-24T22:35:04.809580+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:04.809519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:04.809519+0000","last_clean_scrub_stamp":"2026-02-24T22:35:04.809519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:29:02.534615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"312'104","reported_seq":784,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.994989+0000","last_change":"2026-02-24T22:34:42.994989+0000","last_active":"2026-02-24T22:34:42.994989+0000","last_peered":"2026-02-24T22:34:42.994989+0000","last_clean":"2026-02-24T22:34:42.994989+0000","last_became_active":"2026-02-24T22:05:18.191346+0000","last_became_peered":"2026-02-24T22:05:18.191346+0000","last_unstale":"2026-02-24T22:34:42.994989+0000","last_undegraded":"2026-02-24T22:34:42.994989+0000","last_fullsized":"2026-02-24T22:34:42.994989+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'104","last_scrub_stamp":"2026-02-24T22:34:42.994927+0000","last_deep_scrub":"312'104","last_deep_scrub_stamp":"2026-02-24T22:34:42.994927+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.994927+0000","objects_scrubbed":31,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:12:15.667131+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":237,"num_read_kb":209,"num_write":87,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"312'276","reported_seq":1005,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:30.666652+0000","last_change":"2026-02-24T22:34:30.666602+0000","last_active":"2026-02-24T22:34:30.666652+0000","last_peered":"2026-02-24T22:34:30.666652+0000","last_clean":"2026-02-24T22:34:30.666652+0000","last_became_active":"2026-02-24T22:05:16.183990+0000","last_became_peered":"2026-02-24T22:05:16.183990+0000","last_unstale":"2026-02-24T22:34:30.666652+0000","last_undegraded":"2026-02-24T22:34:30.666652+0000","last_fullsized":"2026-02-24T22:34:30.666652+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'276","last_scrub_stamp":"2026-02-24T22:34:30.666509+0000","last_deep_scrub":"312'276","last_deep_scrub_stamp":"2026-02-24T22:34:30.666509+0000","last_clean_scrub_stamp":"2026-02-24T22:34:30.666509+0000","objects_scrubbed":5,"log_size":76,"log_dups_size":200,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:23:41.986404+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485805,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":458,"num_read_kb":23693,"num_write":1270,"num_write_kb":23324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":599,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:51.749958+0000","last_change":"2026-02-24T22:34:51.749919+0000","last_active":"2026-02-24T22:34:51.749958+0000","last_peered":"2026-02-24T22:34:51.749958+0000","last_clean":"2026-02-24T22:34:51.749958+0000","last_became_active":"2026-02-24T22:05:20.186217+0000","last_became_peered":"2026-02-24T22:05:20.186217+0000","last_unstale":"2026-02-24T22:34:51.749958+0000","last_undegraded":"2026-02-24T22:34:51.749958+0000","last_fullsized":"2026-02-24T22:34:51.749958+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:51.749857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:51.749857+0000","last_clean_scrub_stamp":"2026-02-24T22:34:51.749857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:49:39.493746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"312'309","reported_seq":1164,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:55.272395+0000","last_change":"2026-02-24T22:34:55.272395+0000","last_active":"2026-02-24T22:34:55.272395+0000","last_peered":"2026-02-24T22:34:55.272395+0000","last_clean":"2026-02-24T22:34:55.272395+0000","last_became_active":"2026-02-24T22:05:22.196911+0000","last_became_peered":"2026-02-24T22:05:22.196911+0000","last_unstale":"2026-02-24T22:34:55.272395+0000","last_undegraded":"2026-02-24T22:34:55.272395+0000","last_fullsized":"2026-02-24T22:34:55.272395+0000","mapping_epoch":28,"log_start":"307'200","ondisk_log_start":"307'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'309","last_scrub_stamp":"2026-02-24T22:34:55.272282+0000","last_deep_scrub":"312'309","last_deep_scrub_stamp":"2026-02-24T22:34:55.272282+0000","last_clean_scrub_stamp":"2026-02-24T22:34:55.272282+0000","objects_scrubbed":4,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:53:27.073831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1016,"num_read_kb":540,"num_write":51,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"312'279","reported_seq":906,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:01.310102+0000","last_change":"2026-02-24T22:35:01.310102+0000","last_active":"2026-02-24T22:35:01.310102+0000","last_peered":"2026-02-24T22:35:01.310102+0000","last_clean":"2026-02-24T22:35:01.310102+0000","last_became_active":"2026-02-24T22:05:24.207692+0000","last_became_peered":"2026-02-24T22:05:24.207692+0000","last_unstale":"2026-02-24T22:35:01.310102+0000","last_undegraded":"2026-02-24T22:35:01.310102+0000","last_fullsized":"2026-02-24T22:35:01.310102+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'279","last_scrub_stamp":"2026-02-24T22:35:01.310078+0000","last_deep_scrub":"312'279","last_deep_scrub_stamp":"2026-02-24T22:35:01.310078+0000","last_clean_scrub_stamp":"2026-02-24T22:35:01.310078+0000","objects_scrubbed":92,"log_size":79,"log_dups_size":200,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:57:59.408543+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29978,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":173,"num_read_kb":171,"num_write":478,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":582,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:07.750498+0000","last_change":"2026-02-24T22:35:07.750436+0000","last_active":"2026-02-24T22:35:07.750498+0000","last_peered":"2026-02-24T22:35:07.750498+0000","last_clean":"2026-02-24T22:35:07.750498+0000","last_became_active":"2026-02-24T22:05:29.654288+0000","last_became_peered":"2026-02-24T22:05:29.654288+0000","last_unstale":"2026-02-24T22:35:07.750498+0000","last_undegraded":"2026-02-24T22:35:07.750498+0000","last_fullsized":"2026-02-24T22:35:07.750498+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:07.750356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:07.750356+0000","last_clean_scrub_stamp":"2026-02-24T22:35:07.750356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:03:20.092570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"312'298","reported_seq":1031,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.885878+0000","last_change":"2026-02-24T22:34:42.885878+0000","last_active":"2026-02-24T22:34:42.885878+0000","last_peered":"2026-02-24T22:34:42.885878+0000","last_clean":"2026-02-24T22:34:42.885878+0000","last_became_active":"2026-02-24T22:05:18.184542+0000","last_became_peered":"2026-02-24T22:05:18.184542+0000","last_unstale":"2026-02-24T22:34:42.885878+0000","last_undegraded":"2026-02-24T22:34:42.885878+0000","last_fullsized":"2026-02-24T22:34:42.885878+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'298","last_scrub_stamp":"2026-02-24T22:34:42.885832+0000","last_deep_scrub":"312'298","last_deep_scrub_stamp":"2026-02-24T22:34:42.885832+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.885832+0000","objects_scrubbed":24,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:07:57.036667+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1179,"num_read_kb":976,"num_write":460,"num_write_kb":243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"312'294","reported_seq":1017,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.140803+0000","last_change":"2026-02-24T22:34:35.140765+0000","last_active":"2026-02-24T22:34:35.140803+0000","last_peered":"2026-02-24T22:34:35.140803+0000","last_clean":"2026-02-24T22:34:35.140803+0000","last_became_active":"2026-02-24T22:05:16.190372+0000","last_became_peered":"2026-02-24T22:05:16.190372+0000","last_unstale":"2026-02-24T22:34:35.140803+0000","last_undegraded":"2026-02-24T22:34:35.140803+0000","last_fullsized":"2026-02-24T22:34:35.140803+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'294","last_scrub_stamp":"2026-02-24T22:34:35.140712+0000","last_deep_scrub":"312'294","last_deep_scrub_stamp":"2026-02-24T22:34:35.140712+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.140712+0000","objects_scrubbed":4,"log_size":94,"log_dups_size":200,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:24:43.777918+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":430,"num_read_kb":3398,"num_write":1175,"num_write_kb":524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":598,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:50.744612+0000","last_change":"2026-02-24T22:34:50.744612+0000","last_active":"2026-02-24T22:34:50.744612+0000","last_peered":"2026-02-24T22:34:50.744612+0000","last_clean":"2026-02-24T22:34:50.744612+0000","last_became_active":"2026-02-24T22:05:20.187327+0000","last_became_peered":"2026-02-24T22:05:20.187327+0000","last_unstale":"2026-02-24T22:34:50.744612+0000","last_undegraded":"2026-02-24T22:34:50.744612+0000","last_fullsized":"2026-02-24T22:34:50.744612+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:50.744552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:50.744552+0000","last_clean_scrub_stamp":"2026-02-24T22:34:50.744552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:05:37.833168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"312'802","reported_seq":2140,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.134178+0000","last_change":"2026-02-24T22:34:48.134134+0000","last_active":"2026-02-24T22:34:48.134178+0000","last_peered":"2026-02-24T22:34:48.134178+0000","last_clean":"2026-02-24T22:34:48.134178+0000","last_became_active":"2026-02-24T22:05:22.191073+0000","last_became_peered":"2026-02-24T22:05:22.191073+0000","last_unstale":"2026-02-24T22:34:48.134178+0000","last_undegraded":"2026-02-24T22:34:48.134178+0000","last_fullsized":"2026-02-24T22:34:48.134178+0000","mapping_epoch":28,"log_start":"307'700","ondisk_log_start":"307'700","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'802","last_scrub_stamp":"2026-02-24T22:34:48.134072+0000","last_deep_scrub":"312'802","last_deep_scrub_stamp":"2026-02-24T22:34:48.134072+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.134072+0000","objects_scrubbed":8,"log_size":102,"log_dups_size":700,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:20:11.846328+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":460,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2934,"num_read_kb":1505,"num_write":81,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"312'311","reported_seq":1838,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:00.230054+0000","last_change":"2026-02-24T22:35:00.230054+0000","last_active":"2026-02-24T22:35:00.230054+0000","last_peered":"2026-02-24T22:35:00.230054+0000","last_clean":"2026-02-24T22:35:00.230054+0000","last_became_active":"2026-02-24T22:05:24.206134+0000","last_became_peered":"2026-02-24T22:05:24.206134+0000","last_unstale":"2026-02-24T22:35:00.230054+0000","last_undegraded":"2026-02-24T22:35:00.230054+0000","last_fullsized":"2026-02-24T22:35:00.230054+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'311","last_scrub_stamp":"2026-02-24T22:35:00.229994+0000","last_deep_scrub":"312'311","last_deep_scrub_stamp":"2026-02-24T22:35:00.229994+0000","last_clean_scrub_stamp":"2026-02-24T22:35:00.229994+0000","objects_scrubbed":107,"log_size":11,"log_dups_size":300,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:46:37.708465+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33889,"num_objects":107,"num_object_clones":0,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":0,"num_read":1064,"num_read_kb":1063,"num_write":484,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":584,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:05.263336+0000","last_change":"2026-02-24T22:35:05.263294+0000","last_active":"2026-02-24T22:35:05.263336+0000","last_peered":"2026-02-24T22:35:05.263336+0000","last_clean":"2026-02-24T22:35:05.263336+0000","last_became_active":"2026-02-24T22:05:29.658119+0000","last_became_peered":"2026-02-24T22:05:29.658119+0000","last_unstale":"2026-02-24T22:35:05.263336+0000","last_undegraded":"2026-02-24T22:35:05.263336+0000","last_fullsized":"2026-02-24T22:35:05.263336+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:05.263228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:05.263228+0000","last_clean_scrub_stamp":"2026-02-24T22:35:05.263228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:33:28.280837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"312'690","reported_seq":1497,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.055164+0000","last_change":"2026-02-24T22:34:37.055105+0000","last_active":"2026-02-24T22:34:37.055164+0000","last_peered":"2026-02-24T22:34:37.055164+0000","last_clean":"2026-02-24T22:34:37.055164+0000","last_became_active":"2026-02-24T22:06:00.662609+0000","last_became_peered":"2026-02-24T22:06:00.662609+0000","last_unstale":"2026-02-24T22:34:37.055164+0000","last_undegraded":"2026-02-24T22:34:37.055164+0000","last_fullsized":"2026-02-24T22:34:37.055164+0000","mapping_epoch":36,"log_start":"312'600","ondisk_log_start":"312'600","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'690","last_scrub_stamp":"2026-02-24T22:34:37.055035+0000","last_deep_scrub":"312'690","last_deep_scrub_stamp":"2026-02-24T22:34:37.055035+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.055035+0000","objects_scrubbed":22,"log_size":90,"log_dups_size":600,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:44:46.603715+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":2886,"num_read_kb":2388,"num_write":1101,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"312'327","reported_seq":1062,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:30.811232+0000","last_change":"2026-02-24T22:34:30.811183+0000","last_active":"2026-02-24T22:34:30.811232+0000","last_peered":"2026-02-24T22:34:30.811232+0000","last_clean":"2026-02-24T22:34:30.811232+0000","last_became_active":"2026-02-24T22:05:16.189739+0000","last_became_peered":"2026-02-24T22:05:16.189739+0000","last_unstale":"2026-02-24T22:34:30.811232+0000","last_undegraded":"2026-02-24T22:34:30.811232+0000","last_fullsized":"2026-02-24T22:34:30.811232+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'327","last_scrub_stamp":"2026-02-24T22:34:30.811119+0000","last_deep_scrub":"312'327","last_deep_scrub_stamp":"2026-02-24T22:34:30.811119+0000","last_clean_scrub_stamp":"2026-02-24T22:34:30.811119+0000","objects_scrubbed":9,"log_size":27,"log_dups_size":300,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:44:34.182914+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485786,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":468,"num_read_kb":16928,"num_write":1377,"num_write_kb":17766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":587,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:51.907157+0000","last_change":"2026-02-24T22:34:51.907157+0000","last_active":"2026-02-24T22:34:51.907157+0000","last_peered":"2026-02-24T22:34:51.907157+0000","last_clean":"2026-02-24T22:34:51.907157+0000","last_became_active":"2026-02-24T22:05:20.193200+0000","last_became_peered":"2026-02-24T22:05:20.193200+0000","last_unstale":"2026-02-24T22:34:51.907157+0000","last_undegraded":"2026-02-24T22:34:51.907157+0000","last_fullsized":"2026-02-24T22:34:51.907157+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:51.907045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:51.907045+0000","last_clean_scrub_stamp":"2026-02-24T22:34:51.907045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:41:57.779249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"312'196","reported_seq":807,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:57.208914+0000","last_change":"2026-02-24T22:34:57.208914+0000","last_active":"2026-02-24T22:34:57.208914+0000","last_peered":"2026-02-24T22:34:57.208914+0000","last_clean":"2026-02-24T22:34:57.208914+0000","last_became_active":"2026-02-24T22:05:22.912826+0000","last_became_peered":"2026-02-24T22:05:22.912826+0000","last_unstale":"2026-02-24T22:34:57.208914+0000","last_undegraded":"2026-02-24T22:34:57.208914+0000","last_fullsized":"2026-02-24T22:34:57.208914+0000","mapping_epoch":28,"log_start":"312'100","ondisk_log_start":"312'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'196","last_scrub_stamp":"2026-02-24T22:34:57.208826+0000","last_deep_scrub":"312'196","last_deep_scrub_stamp":"2026-02-24T22:34:57.208826+0000","last_clean_scrub_stamp":"2026-02-24T22:34:57.208826+0000","objects_scrubbed":4,"log_size":96,"log_dups_size":100,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:42:57.794071+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37493,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":308,"num_read_kb":794,"num_write":284,"num_write_kb":257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"312'352","reported_seq":1013,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:06.646656+0000","last_change":"2026-02-24T22:35:06.646610+0000","last_active":"2026-02-24T22:35:06.646656+0000","last_peered":"2026-02-24T22:35:06.646656+0000","last_clean":"2026-02-24T22:35:06.646656+0000","last_became_active":"2026-02-24T22:05:24.198964+0000","last_became_peered":"2026-02-24T22:05:24.198964+0000","last_unstale":"2026-02-24T22:35:06.646656+0000","last_undegraded":"2026-02-24T22:35:06.646656+0000","last_fullsized":"2026-02-24T22:35:06.646656+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'352","last_scrub_stamp":"2026-02-24T22:35:06.646550+0000","last_deep_scrub":"312'352","last_deep_scrub_stamp":"2026-02-24T22:35:06.646550+0000","last_clean_scrub_stamp":"2026-02-24T22:35:06.646550+0000","objects_scrubbed":95,"log_size":52,"log_dups_size":300,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:44:35.894779+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29433,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":428,"num_read_kb":365,"num_write":612,"num_write_kb":187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"312'140","reported_seq":876,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.037445+0000","last_change":"2026-02-24T22:34:42.037417+0000","last_active":"2026-02-24T22:34:42.037445+0000","last_peered":"2026-02-24T22:34:42.037445+0000","last_clean":"2026-02-24T22:34:42.037445+0000","last_became_active":"2026-02-24T22:05:18.189796+0000","last_became_peered":"2026-02-24T22:05:18.189796+0000","last_unstale":"2026-02-24T22:34:42.037445+0000","last_undegraded":"2026-02-24T22:34:42.037445+0000","last_fullsized":"2026-02-24T22:34:42.037445+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'140","last_scrub_stamp":"2026-02-24T22:34:42.037385+0000","last_deep_scrub":"312'140","last_deep_scrub_stamp":"2026-02-24T22:34:42.037385+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.037385+0000","objects_scrubbed":27,"log_size":40,"log_dups_size":100,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:45:54.858292+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":455,"num_read_kb":399,"num_write":145,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"312'263","reported_seq":951,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.085973+0000","last_change":"2026-02-24T22:34:38.085935+0000","last_active":"2026-02-24T22:34:38.085973+0000","last_peered":"2026-02-24T22:34:38.085973+0000","last_clean":"2026-02-24T22:34:38.085973+0000","last_became_active":"2026-02-24T22:05:16.191985+0000","last_became_peered":"2026-02-24T22:05:16.191985+0000","last_unstale":"2026-02-24T22:34:38.085973+0000","last_undegraded":"2026-02-24T22:34:38.085973+0000","last_fullsized":"2026-02-24T22:34:38.085973+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'263","last_scrub_stamp":"2026-02-24T22:34:38.085876+0000","last_deep_scrub":"312'263","last_deep_scrub_stamp":"2026-02-24T22:34:38.085876+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.085876+0000","objects_scrubbed":7,"log_size":63,"log_dups_size":200,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:00:57.241743+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":82,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":355,"num_read_kb":4498,"num_write":1041,"num_write_kb":4311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":595,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:50.153762+0000","last_change":"2026-02-24T22:34:50.153717+0000","last_active":"2026-02-24T22:34:50.153762+0000","last_peered":"2026-02-24T22:34:50.153762+0000","last_clean":"2026-02-24T22:34:50.153762+0000","last_became_active":"2026-02-24T22:05:20.194288+0000","last_became_peered":"2026-02-24T22:05:20.194288+0000","last_unstale":"2026-02-24T22:34:50.153762+0000","last_undegraded":"2026-02-24T22:34:50.153762+0000","last_fullsized":"2026-02-24T22:34:50.153762+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:50.153652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:50.153652+0000","last_clean_scrub_stamp":"2026-02-24T22:34:50.153652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:34:10.280880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"312'269","reported_seq":907,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:55.965212+0000","last_change":"2026-02-24T22:34:55.965175+0000","last_active":"2026-02-24T22:34:55.965212+0000","last_peered":"2026-02-24T22:34:55.965212+0000","last_clean":"2026-02-24T22:34:55.965212+0000","last_became_active":"2026-02-24T22:05:22.912859+0000","last_became_peered":"2026-02-24T22:05:22.912859+0000","last_unstale":"2026-02-24T22:34:55.965212+0000","last_undegraded":"2026-02-24T22:34:55.965212+0000","last_fullsized":"2026-02-24T22:34:55.965212+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'269","last_scrub_stamp":"2026-02-24T22:34:55.965113+0000","last_deep_scrub":"312'269","last_deep_scrub_stamp":"2026-02-24T22:34:55.965113+0000","last_clean_scrub_stamp":"2026-02-24T22:34:55.965113+0000","objects_scrubbed":9,"log_size":69,"log_dups_size":200,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:00:50.874867+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38223,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":418,"num_read_kb":936,"num_write":362,"num_write_kb":271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"312'408","reported_seq":1118,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:52.254254+0000","last_change":"2026-02-24T22:34:52.254254+0000","last_active":"2026-02-24T22:34:52.254254+0000","last_peered":"2026-02-24T22:34:52.254254+0000","last_clean":"2026-02-24T22:34:52.254254+0000","last_became_active":"2026-02-24T22:05:24.199772+0000","last_became_peered":"2026-02-24T22:05:24.199772+0000","last_unstale":"2026-02-24T22:34:52.254254+0000","last_undegraded":"2026-02-24T22:34:52.254254+0000","last_fullsized":"2026-02-24T22:34:52.254254+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'408","last_scrub_stamp":"2026-02-24T22:34:52.254200+0000","last_deep_scrub":"312'408","last_deep_scrub_stamp":"2026-02-24T22:34:52.254200+0000","last_clean_scrub_stamp":"2026-02-24T22:34:52.254200+0000","objects_scrubbed":93,"log_size":108,"log_dups_size":300,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:02:43.014695+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30266,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":473,"num_read_kb":498,"num_write":701,"num_write_kb":232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1193,"num_omap_keys":7,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:02.856103+0000","last_change":"2026-02-24T22:35:02.856103+0000","last_active":"2026-02-24T22:35:02.856103+0000","last_peered":"2026-02-24T22:35:02.856103+0000","last_clean":"2026-02-24T22:35:02.856103+0000","last_became_active":"2026-02-24T22:05:29.661401+0000","last_became_peered":"2026-02-24T22:05:29.661401+0000","last_unstale":"2026-02-24T22:35:02.856103+0000","last_undegraded":"2026-02-24T22:35:02.856103+0000","last_fullsized":"2026-02-24T22:35:02.856103+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:02.856048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:02.856048+0000","last_clean_scrub_stamp":"2026-02-24T22:35:02.856048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:27:49.594219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"312'103","reported_seq":774,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.478084+0000","last_change":"2026-02-24T22:34:35.478042+0000","last_active":"2026-02-24T22:34:35.478084+0000","last_peered":"2026-02-24T22:34:35.478084+0000","last_clean":"2026-02-24T22:34:35.478084+0000","last_became_active":"2026-02-24T22:05:18.193188+0000","last_became_peered":"2026-02-24T22:05:18.193188+0000","last_unstale":"2026-02-24T22:34:35.478084+0000","last_undegraded":"2026-02-24T22:34:35.478084+0000","last_fullsized":"2026-02-24T22:34:35.478084+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'103","last_scrub_stamp":"2026-02-24T22:34:35.477982+0000","last_deep_scrub":"312'103","last_deep_scrub_stamp":"2026-02-24T22:34:35.477982+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.477982+0000","objects_scrubbed":19,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:46:51.105644+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":312,"num_read_kb":260,"num_write":126,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"312'324","reported_seq":1064,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.151701+0000","last_change":"2026-02-24T22:34:33.151665+0000","last_active":"2026-02-24T22:34:33.151701+0000","last_peered":"2026-02-24T22:34:33.151701+0000","last_clean":"2026-02-24T22:34:33.151701+0000","last_became_active":"2026-02-24T22:05:16.191956+0000","last_became_peered":"2026-02-24T22:05:16.191956+0000","last_unstale":"2026-02-24T22:34:33.151701+0000","last_undegraded":"2026-02-24T22:34:33.151701+0000","last_fullsized":"2026-02-24T22:34:33.151701+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'324","last_scrub_stamp":"2026-02-24T22:34:33.151603+0000","last_deep_scrub":"312'324","last_deep_scrub_stamp":"2026-02-24T22:34:33.151603+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.151603+0000","objects_scrubbed":3,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:52:36.974066+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485778,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":523,"num_read_kb":10657,"num_write":1420,"num_write_kb":15460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":603,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.120864+0000","last_change":"2026-02-24T22:34:46.120807+0000","last_active":"2026-02-24T22:34:46.120864+0000","last_peered":"2026-02-24T22:34:46.120864+0000","last_clean":"2026-02-24T22:34:46.120864+0000","last_became_active":"2026-02-24T22:05:20.186055+0000","last_became_peered":"2026-02-24T22:05:20.186055+0000","last_unstale":"2026-02-24T22:34:46.120864+0000","last_undegraded":"2026-02-24T22:34:46.120864+0000","last_fullsized":"2026-02-24T22:34:46.120864+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:46.120737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:46.120737+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.120737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:33:04.392007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"312'232","reported_seq":849,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:53.782364+0000","last_change":"2026-02-24T22:34:53.782364+0000","last_active":"2026-02-24T22:34:53.782364+0000","last_peered":"2026-02-24T22:34:53.782364+0000","last_clean":"2026-02-24T22:34:53.782364+0000","last_became_active":"2026-02-24T22:05:22.193741+0000","last_became_peered":"2026-02-24T22:05:22.193741+0000","last_unstale":"2026-02-24T22:34:53.782364+0000","last_undegraded":"2026-02-24T22:34:53.782364+0000","last_fullsized":"2026-02-24T22:34:53.782364+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'232","last_scrub_stamp":"2026-02-24T22:34:53.782300+0000","last_deep_scrub":"312'232","last_deep_scrub_stamp":"2026-02-24T22:34:53.782300+0000","last_clean_scrub_stamp":"2026-02-24T22:34:53.782300+0000","objects_scrubbed":4,"log_size":32,"log_dups_size":200,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:22:35.430551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39811,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":385,"num_read_kb":1028,"num_write":360,"num_write_kb":328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"312'276","reported_seq":915,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:01.567425+0000","last_change":"2026-02-24T22:35:01.567396+0000","last_active":"2026-02-24T22:35:01.567425+0000","last_peered":"2026-02-24T22:35:01.567425+0000","last_clean":"2026-02-24T22:35:01.567425+0000","last_became_active":"2026-02-24T22:05:24.198731+0000","last_became_peered":"2026-02-24T22:05:24.198731+0000","last_unstale":"2026-02-24T22:35:01.567425+0000","last_undegraded":"2026-02-24T22:35:01.567425+0000","last_fullsized":"2026-02-24T22:35:01.567425+0000","mapping_epoch":30,"log_start":"312'199","ondisk_log_start":"312'199","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'276","last_scrub_stamp":"2026-02-24T22:35:01.567361+0000","last_deep_scrub":"312'276","last_deep_scrub_stamp":"2026-02-24T22:35:01.567361+0000","last_clean_scrub_stamp":"2026-02-24T22:35:01.567361+0000","objects_scrubbed":91,"log_size":77,"log_dups_size":199,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:34:24.939354+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30034,"num_objects":91,"num_object_clones":0,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":241,"num_read_kb":207,"num_write":470,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":579,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:52.546054+0000","last_change":"2026-02-24T22:34:52.546054+0000","last_active":"2026-02-24T22:34:52.546054+0000","last_peered":"2026-02-24T22:34:52.546054+0000","last_clean":"2026-02-24T22:34:52.546054+0000","last_became_active":"2026-02-24T22:05:29.661161+0000","last_became_peered":"2026-02-24T22:05:29.661161+0000","last_unstale":"2026-02-24T22:34:52.546054+0000","last_undegraded":"2026-02-24T22:34:52.546054+0000","last_fullsized":"2026-02-24T22:34:52.546054+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:52.545999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:52.545999+0000","last_clean_scrub_stamp":"2026-02-24T22:34:52.545999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:03:26.142316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"312'129","reported_seq":785,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.704030+0000","last_change":"2026-02-24T22:34:39.703983+0000","last_active":"2026-02-24T22:34:39.704030+0000","last_peered":"2026-02-24T22:34:39.704030+0000","last_clean":"2026-02-24T22:34:39.704030+0000","last_became_active":"2026-02-24T22:05:18.191555+0000","last_became_peered":"2026-02-24T22:05:18.191555+0000","last_unstale":"2026-02-24T22:34:39.704030+0000","last_undegraded":"2026-02-24T22:34:39.704030+0000","last_fullsized":"2026-02-24T22:34:39.704030+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'129","last_scrub_stamp":"2026-02-24T22:34:39.703930+0000","last_deep_scrub":"312'129","last_deep_scrub_stamp":"2026-02-24T22:34:39.703930+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.703930+0000","objects_scrubbed":23,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:20:21.850877+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":387,"num_read_kb":325,"num_write":162,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"312'289","reported_seq":1011,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.159670+0000","last_change":"2026-02-24T22:34:34.159624+0000","last_active":"2026-02-24T22:34:34.159670+0000","last_peered":"2026-02-24T22:34:34.159670+0000","last_clean":"2026-02-24T22:34:34.159670+0000","last_became_active":"2026-02-24T22:05:16.191733+0000","last_became_peered":"2026-02-24T22:05:16.191733+0000","last_unstale":"2026-02-24T22:34:34.159670+0000","last_undegraded":"2026-02-24T22:34:34.159670+0000","last_fullsized":"2026-02-24T22:34:34.159670+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'289","last_scrub_stamp":"2026-02-24T22:34:34.159566+0000","last_deep_scrub":"312'289","last_deep_scrub_stamp":"2026-02-24T22:34:34.159566+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.159566+0000","objects_scrubbed":4,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:18:06.498072+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242906,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":398,"num_read_kb":31022,"num_write":1049,"num_write_kb":10319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":602,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.063841+0000","last_change":"2026-02-24T22:34:44.063841+0000","last_active":"2026-02-24T22:34:44.063841+0000","last_peered":"2026-02-24T22:34:44.063841+0000","last_clean":"2026-02-24T22:34:44.063841+0000","last_became_active":"2026-02-24T22:05:20.186892+0000","last_became_peered":"2026-02-24T22:05:20.186892+0000","last_unstale":"2026-02-24T22:34:44.063841+0000","last_undegraded":"2026-02-24T22:34:44.063841+0000","last_fullsized":"2026-02-24T22:34:44.063841+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:44.063776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:44.063776+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.063776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:34:17.807478+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"312'245","reported_seq":867,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.174575+0000","last_change":"2026-02-24T22:34:58.174484+0000","last_active":"2026-02-24T22:34:58.174575+0000","last_peered":"2026-02-24T22:34:58.174575+0000","last_clean":"2026-02-24T22:34:58.174575+0000","last_became_active":"2026-02-24T22:05:22.200208+0000","last_became_peered":"2026-02-24T22:05:22.200208+0000","last_unstale":"2026-02-24T22:34:58.174575+0000","last_undegraded":"2026-02-24T22:34:58.174575+0000","last_fullsized":"2026-02-24T22:34:58.174575+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'245","last_scrub_stamp":"2026-02-24T22:34:58.174378+0000","last_deep_scrub":"312'245","last_deep_scrub_stamp":"2026-02-24T22:34:58.174378+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.174378+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":200,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:10:15.543051+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39435,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":390,"num_read_kb":991,"num_write":356,"num_write_kb":310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"312'378","reported_seq":1202,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.906638+0000","last_change":"2026-02-24T22:34:58.906602+0000","last_active":"2026-02-24T22:34:58.906638+0000","last_peered":"2026-02-24T22:34:58.906638+0000","last_clean":"2026-02-24T22:34:58.906638+0000","last_became_active":"2026-02-24T22:05:24.199525+0000","last_became_peered":"2026-02-24T22:05:24.199525+0000","last_unstale":"2026-02-24T22:34:58.906638+0000","last_undegraded":"2026-02-24T22:34:58.906638+0000","last_fullsized":"2026-02-24T22:34:58.906638+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'378","last_scrub_stamp":"2026-02-24T22:34:58.906557+0000","last_deep_scrub":"312'378","last_deep_scrub_stamp":"2026-02-24T22:34:58.906557+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.906557+0000","objects_scrubbed":103,"log_size":78,"log_dups_size":300,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:42:33.226800+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32009,"num_objects":103,"num_object_clones":0,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":506,"num_read_kb":490,"num_write":606,"num_write_kb":207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":584,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:03.724434+0000","last_change":"2026-02-24T22:35:03.724387+0000","last_active":"2026-02-24T22:35:03.724434+0000","last_peered":"2026-02-24T22:35:03.724434+0000","last_clean":"2026-02-24T22:35:03.724434+0000","last_became_active":"2026-02-24T22:05:29.652598+0000","last_became_peered":"2026-02-24T22:05:29.652598+0000","last_unstale":"2026-02-24T22:35:03.724434+0000","last_undegraded":"2026-02-24T22:35:03.724434+0000","last_fullsized":"2026-02-24T22:35:03.724434+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:03.724333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:03.724333+0000","last_clean_scrub_stamp":"2026-02-24T22:35:03.724333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:16:00.583629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"312'189","reported_seq":900,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.811102+0000","last_change":"2026-02-24T22:34:40.811102+0000","last_active":"2026-02-24T22:34:40.811102+0000","last_peered":"2026-02-24T22:34:40.811102+0000","last_clean":"2026-02-24T22:34:40.811102+0000","last_became_active":"2026-02-24T22:05:18.185152+0000","last_became_peered":"2026-02-24T22:05:18.185152+0000","last_unstale":"2026-02-24T22:34:40.811102+0000","last_undegraded":"2026-02-24T22:34:40.811102+0000","last_fullsized":"2026-02-24T22:34:40.811102+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'189","last_scrub_stamp":"2026-02-24T22:34:40.811068+0000","last_deep_scrub":"312'189","last_deep_scrub_stamp":"2026-02-24T22:34:40.811068+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.811068+0000","objects_scrubbed":20,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:24:59.774125+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":651,"num_read_kb":540,"num_write":239,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"312'323","reported_seq":1050,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.061642+0000","last_change":"2026-02-24T22:34:33.061574+0000","last_active":"2026-02-24T22:34:33.061642+0000","last_peered":"2026-02-24T22:34:33.061642+0000","last_clean":"2026-02-24T22:34:33.061642+0000","last_became_active":"2026-02-24T22:05:16.190205+0000","last_became_peered":"2026-02-24T22:05:16.190205+0000","last_unstale":"2026-02-24T22:34:33.061642+0000","last_undegraded":"2026-02-24T22:34:33.061642+0000","last_fullsized":"2026-02-24T22:34:33.061642+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'323","last_scrub_stamp":"2026-02-24T22:34:33.061497+0000","last_deep_scrub":"312'323","last_deep_scrub_stamp":"2026-02-24T22:34:33.061497+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.061497+0000","objects_scrubbed":1,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:54:15.633089+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":578,"num_read_kb":20088,"num_write":1296,"num_write_kb":14513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":603,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.105566+0000","last_change":"2026-02-24T22:34:45.105525+0000","last_active":"2026-02-24T22:34:45.105566+0000","last_peered":"2026-02-24T22:34:45.105566+0000","last_clean":"2026-02-24T22:34:45.105566+0000","last_became_active":"2026-02-24T22:05:20.186781+0000","last_became_peered":"2026-02-24T22:05:20.186781+0000","last_unstale":"2026-02-24T22:34:45.105566+0000","last_undegraded":"2026-02-24T22:34:45.105566+0000","last_fullsized":"2026-02-24T22:34:45.105566+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:45.105463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:45.105463+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.105463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:39:37.239923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"312'404","reported_seq":1033,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:51.933123+0000","last_change":"2026-02-24T22:34:51.933082+0000","last_active":"2026-02-24T22:34:51.933123+0000","last_peered":"2026-02-24T22:34:51.933123+0000","last_clean":"2026-02-24T22:34:51.933123+0000","last_became_active":"2026-02-24T22:05:22.192495+0000","last_became_peered":"2026-02-24T22:05:22.192495+0000","last_unstale":"2026-02-24T22:34:51.933123+0000","last_undegraded":"2026-02-24T22:34:51.933123+0000","last_fullsized":"2026-02-24T22:34:51.933123+0000","mapping_epoch":28,"log_start":"312'300","ondisk_log_start":"312'300","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'404","last_scrub_stamp":"2026-02-24T22:34:51.933036+0000","last_deep_scrub":"312'404","last_deep_scrub_stamp":"2026-02-24T22:34:51.933036+0000","last_clean_scrub_stamp":"2026-02-24T22:34:51.933036+0000","objects_scrubbed":5,"log_size":104,"log_dups_size":300,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:38:02.998627+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":81867,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":699,"num_read_kb":1918,"num_write":658,"num_write_kb":626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"312'264","reported_seq":931,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.183026+0000","last_change":"2026-02-24T22:34:49.182990+0000","last_active":"2026-02-24T22:34:49.183026+0000","last_peered":"2026-02-24T22:34:49.183026+0000","last_clean":"2026-02-24T22:34:49.183026+0000","last_became_active":"2026-02-24T22:05:24.198114+0000","last_became_peered":"2026-02-24T22:05:24.198114+0000","last_unstale":"2026-02-24T22:34:49.183026+0000","last_undegraded":"2026-02-24T22:34:49.183026+0000","last_fullsized":"2026-02-24T22:34:49.183026+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'264","last_scrub_stamp":"2026-02-24T22:34:49.182961+0000","last_deep_scrub":"312'264","last_deep_scrub_stamp":"2026-02-24T22:34:49.182961+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.182961+0000","objects_scrubbed":91,"log_size":64,"log_dups_size":200,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:07:14.157330+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29501,"num_objects":91,"num_object_clones":0,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":235,"num_read_kb":226,"num_write":429,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":584,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:57.325845+0000","last_change":"2026-02-24T22:34:57.325797+0000","last_active":"2026-02-24T22:34:57.325845+0000","last_peered":"2026-02-24T22:34:57.325845+0000","last_clean":"2026-02-24T22:34:57.325845+0000","last_became_active":"2026-02-24T22:05:29.652715+0000","last_became_peered":"2026-02-24T22:05:29.652715+0000","last_unstale":"2026-02-24T22:34:57.325845+0000","last_undegraded":"2026-02-24T22:34:57.325845+0000","last_fullsized":"2026-02-24T22:34:57.325845+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:57.325730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:57.325730+0000","last_clean_scrub_stamp":"2026-02-24T22:34:57.325730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:53:52.255388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"312'596","reported_seq":1465,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.442591+0000","last_change":"2026-02-24T22:34:37.442591+0000","last_active":"2026-02-24T22:34:37.442591+0000","last_peered":"2026-02-24T22:34:37.442591+0000","last_clean":"2026-02-24T22:34:37.442591+0000","last_became_active":"2026-02-24T22:05:18.192543+0000","last_became_peered":"2026-02-24T22:05:18.192543+0000","last_unstale":"2026-02-24T22:34:37.442591+0000","last_undegraded":"2026-02-24T22:34:37.442591+0000","last_fullsized":"2026-02-24T22:34:37.442591+0000","mapping_epoch":24,"log_start":"312'500","ondisk_log_start":"312'500","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'596","last_scrub_stamp":"2026-02-24T22:34:37.442512+0000","last_deep_scrub":"312'596","last_deep_scrub_stamp":"2026-02-24T22:34:37.442512+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.442512+0000","objects_scrubbed":25,"log_size":96,"log_dups_size":500,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:07:01.563341+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":2576,"num_read_kb":2171,"num_write":966,"num_write_kb":601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"312'260","reported_seq":943,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.858767+0000","last_change":"2026-02-24T22:34:32.858731+0000","last_active":"2026-02-24T22:34:32.858767+0000","last_peered":"2026-02-24T22:34:32.858767+0000","last_clean":"2026-02-24T22:34:32.858767+0000","last_became_active":"2026-02-24T22:05:16.182928+0000","last_became_peered":"2026-02-24T22:05:16.182928+0000","last_unstale":"2026-02-24T22:34:32.858767+0000","last_undegraded":"2026-02-24T22:34:32.858767+0000","last_fullsized":"2026-02-24T22:34:32.858767+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'260","last_scrub_stamp":"2026-02-24T22:34:32.858687+0000","last_deep_scrub":"312'260","last_deep_scrub_stamp":"2026-02-24T22:34:32.858687+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.858687+0000","objects_scrubbed":5,"log_size":60,"log_dups_size":200,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:47:38.099528+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":316,"num_read_kb":262,"num_write":1083,"num_write_kb":181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":602,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.083526+0000","last_change":"2026-02-24T22:34:43.083526+0000","last_active":"2026-02-24T22:34:43.083526+0000","last_peered":"2026-02-24T22:34:43.083526+0000","last_clean":"2026-02-24T22:34:43.083526+0000","last_became_active":"2026-02-24T22:05:20.186209+0000","last_became_peered":"2026-02-24T22:05:20.186209+0000","last_unstale":"2026-02-24T22:34:43.083526+0000","last_undegraded":"2026-02-24T22:34:43.083526+0000","last_fullsized":"2026-02-24T22:34:43.083526+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:43.083470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:43.083470+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.083470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:53:12.978030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"312'956","reported_seq":2285,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:53.254053+0000","last_change":"2026-02-24T22:34:53.254053+0000","last_active":"2026-02-24T22:34:53.254053+0000","last_peered":"2026-02-24T22:34:53.254053+0000","last_clean":"2026-02-24T22:34:53.254053+0000","last_became_active":"2026-02-24T22:05:22.200111+0000","last_became_peered":"2026-02-24T22:05:22.200111+0000","last_unstale":"2026-02-24T22:34:53.254053+0000","last_undegraded":"2026-02-24T22:34:53.254053+0000","last_fullsized":"2026-02-24T22:34:53.254053+0000","mapping_epoch":28,"log_start":"312'900","ondisk_log_start":"312'900","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'956","last_scrub_stamp":"2026-02-24T22:34:53.253998+0000","last_deep_scrub":"312'956","last_deep_scrub_stamp":"2026-02-24T22:34:53.253998+0000","last_clean_scrub_stamp":"2026-02-24T22:34:53.253998+0000","objects_scrubbed":9,"log_size":56,"log_dups_size":900,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:23:04.182706+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37978,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":3232,"num_read_kb":2382,"num_write":419,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15964,"num_omap_keys":38,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"312'284","reported_seq":891,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:59.904243+0000","last_change":"2026-02-24T22:34:59.904243+0000","last_active":"2026-02-24T22:34:59.904243+0000","last_peered":"2026-02-24T22:34:59.904243+0000","last_clean":"2026-02-24T22:34:59.904243+0000","last_became_active":"2026-02-24T22:05:24.200878+0000","last_became_peered":"2026-02-24T22:05:24.200878+0000","last_unstale":"2026-02-24T22:34:59.904243+0000","last_undegraded":"2026-02-24T22:34:59.904243+0000","last_fullsized":"2026-02-24T22:34:59.904243+0000","mapping_epoch":30,"log_start":"312'197","ondisk_log_start":"312'197","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'284","last_scrub_stamp":"2026-02-24T22:34:59.904208+0000","last_deep_scrub":"312'284","last_deep_scrub_stamp":"2026-02-24T22:34:59.904208+0000","last_clean_scrub_stamp":"2026-02-24T22:34:59.904208+0000","objects_scrubbed":101,"log_size":87,"log_dups_size":197,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:49:36.597249+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33021,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":110,"num_read_kb":109,"num_write":447,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"312'440","reported_seq":2210,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:03.546851+0000","last_change":"2026-02-24T22:35:03.546764+0000","last_active":"2026-02-24T22:35:03.546851+0000","last_peered":"2026-02-24T22:35:03.546851+0000","last_clean":"2026-02-24T22:35:03.546851+0000","last_became_active":"2026-02-24T22:05:29.654064+0000","last_became_peered":"2026-02-24T22:05:29.654064+0000","last_unstale":"2026-02-24T22:35:03.546851+0000","last_undegraded":"2026-02-24T22:35:03.546851+0000","last_fullsized":"2026-02-24T22:35:03.546851+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'440","last_scrub_stamp":"2026-02-24T22:35:03.546700+0000","last_deep_scrub":"312'440","last_deep_scrub_stamp":"2026-02-24T22:35:03.546700+0000","last_clean_scrub_stamp":"2026-02-24T22:35:03.546700+0000","objects_scrubbed":1,"log_size":40,"log_dups_size":400,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:16:29.192139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"312'149","reported_seq":841,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.467232+0000","last_change":"2026-02-24T22:34:38.467232+0000","last_active":"2026-02-24T22:34:38.467232+0000","last_peered":"2026-02-24T22:34:38.467232+0000","last_clean":"2026-02-24T22:34:38.467232+0000","last_became_active":"2026-02-24T22:05:18.191055+0000","last_became_peered":"2026-02-24T22:05:18.191055+0000","last_unstale":"2026-02-24T22:34:38.467232+0000","last_undegraded":"2026-02-24T22:34:38.467232+0000","last_fullsized":"2026-02-24T22:34:38.467232+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'149","last_scrub_stamp":"2026-02-24T22:34:38.467178+0000","last_deep_scrub":"312'149","last_deep_scrub_stamp":"2026-02-24T22:34:38.467178+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.467178+0000","objects_scrubbed":21,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:18:47.528959+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":486,"num_read_kb":398,"num_write":170,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"312'241","reported_seq":957,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.746541+0000","last_change":"2026-02-24T22:34:32.746504+0000","last_active":"2026-02-24T22:34:32.746541+0000","last_peered":"2026-02-24T22:34:32.746541+0000","last_clean":"2026-02-24T22:34:32.746541+0000","last_became_active":"2026-02-24T22:05:16.187261+0000","last_became_peered":"2026-02-24T22:05:16.187261+0000","last_unstale":"2026-02-24T22:34:32.746541+0000","last_undegraded":"2026-02-24T22:34:32.746541+0000","last_fullsized":"2026-02-24T22:34:32.746541+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'241","last_scrub_stamp":"2026-02-24T22:34:32.746460+0000","last_deep_scrub":"312'241","last_deep_scrub_stamp":"2026-02-24T22:34:32.746460+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.746460+0000","objects_scrubbed":1,"log_size":41,"log_dups_size":200,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:05:01.484992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":370,"num_read_kb":24764,"num_write":957,"num_write_kb":10562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":597,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.218231+0000","last_change":"2026-02-24T22:34:47.218231+0000","last_active":"2026-02-24T22:34:47.218231+0000","last_peered":"2026-02-24T22:34:47.218231+0000","last_clean":"2026-02-24T22:34:47.218231+0000","last_became_active":"2026-02-24T22:05:20.187488+0000","last_became_peered":"2026-02-24T22:05:20.187488+0000","last_unstale":"2026-02-24T22:34:47.218231+0000","last_undegraded":"2026-02-24T22:34:47.218231+0000","last_fullsized":"2026-02-24T22:34:47.218231+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:47.218164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:47.218164+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.218164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:23:53.051333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"312'475","reported_seq":1340,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.137231+0000","last_change":"2026-02-24T22:34:47.137231+0000","last_active":"2026-02-24T22:34:47.137231+0000","last_peered":"2026-02-24T22:34:47.137231+0000","last_clean":"2026-02-24T22:34:47.137231+0000","last_became_active":"2026-02-24T22:05:22.193142+0000","last_became_peered":"2026-02-24T22:05:22.193142+0000","last_unstale":"2026-02-24T22:34:47.137231+0000","last_undegraded":"2026-02-24T22:34:47.137231+0000","last_fullsized":"2026-02-24T22:34:47.137231+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'475","last_scrub_stamp":"2026-02-24T22:34:47.137162+0000","last_deep_scrub":"312'475","last_deep_scrub_stamp":"2026-02-24T22:34:47.137162+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.137162+0000","objects_scrubbed":8,"log_size":75,"log_dups_size":400,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:12:15.570963+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38313,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1308,"num_read_kb":1368,"num_write":335,"num_write_kb":281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"312'312","reported_seq":926,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:50.216629+0000","last_change":"2026-02-24T22:34:50.216629+0000","last_active":"2026-02-24T22:34:50.216629+0000","last_peered":"2026-02-24T22:34:50.216629+0000","last_clean":"2026-02-24T22:34:50.216629+0000","last_became_active":"2026-02-24T22:05:24.198669+0000","last_became_peered":"2026-02-24T22:05:24.198669+0000","last_unstale":"2026-02-24T22:34:50.216629+0000","last_undegraded":"2026-02-24T22:34:50.216629+0000","last_fullsized":"2026-02-24T22:34:50.216629+0000","mapping_epoch":30,"log_start":"312'296","ondisk_log_start":"312'296","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'312","last_scrub_stamp":"2026-02-24T22:34:50.216584+0000","last_deep_scrub":"312'312","last_deep_scrub_stamp":"2026-02-24T22:34:50.216584+0000","last_clean_scrub_stamp":"2026-02-24T22:34:50.216584+0000","objects_scrubbed":90,"log_size":16,"log_dups_size":296,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:29:12.268470+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29328,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":170,"num_read_kb":165,"num_write":551,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":584,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:56.292294+0000","last_change":"2026-02-24T22:34:56.292294+0000","last_active":"2026-02-24T22:34:56.292294+0000","last_peered":"2026-02-24T22:34:56.292294+0000","last_clean":"2026-02-24T22:34:56.292294+0000","last_became_active":"2026-02-24T22:05:29.653144+0000","last_became_peered":"2026-02-24T22:05:29.653144+0000","last_unstale":"2026-02-24T22:34:56.292294+0000","last_undegraded":"2026-02-24T22:34:56.292294+0000","last_fullsized":"2026-02-24T22:34:56.292294+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:56.292236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:56.292236+0000","last_clean_scrub_stamp":"2026-02-24T22:34:56.292236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:28:09.419838+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"312'194","reported_seq":908,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.515320+0000","last_change":"2026-02-24T22:34:39.515320+0000","last_active":"2026-02-24T22:34:39.515320+0000","last_peered":"2026-02-24T22:34:39.515320+0000","last_clean":"2026-02-24T22:34:39.515320+0000","last_became_active":"2026-02-24T22:05:18.190140+0000","last_became_peered":"2026-02-24T22:05:18.190140+0000","last_unstale":"2026-02-24T22:34:39.515320+0000","last_undegraded":"2026-02-24T22:34:39.515320+0000","last_fullsized":"2026-02-24T22:34:39.515320+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'194","last_scrub_stamp":"2026-02-24T22:34:39.515258+0000","last_deep_scrub":"312'194","last_deep_scrub_stamp":"2026-02-24T22:34:39.515258+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.515258+0000","objects_scrubbed":23,"log_size":94,"log_dups_size":100,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:49:24.504968+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":675,"num_read_kb":556,"num_write":238,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"312'303","reported_seq":1073,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.577700+0000","last_change":"2026-02-24T22:34:32.577662+0000","last_active":"2026-02-24T22:34:32.577700+0000","last_peered":"2026-02-24T22:34:32.577700+0000","last_clean":"2026-02-24T22:34:32.577700+0000","last_became_active":"2026-02-24T22:05:16.184944+0000","last_became_peered":"2026-02-24T22:05:16.184944+0000","last_unstale":"2026-02-24T22:34:32.577700+0000","last_undegraded":"2026-02-24T22:34:32.577700+0000","last_fullsized":"2026-02-24T22:34:32.577700+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'303","last_scrub_stamp":"2026-02-24T22:34:32.577606+0000","last_deep_scrub":"312'303","last_deep_scrub_stamp":"2026-02-24T22:34:32.577606+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.577606+0000","objects_scrubbed":3,"log_size":103,"log_dups_size":200,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:59:47.981464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":538,"num_read_kb":45888,"num_write":1354,"num_write_kb":20430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"312'4","reported_seq":1068,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:53.903762+0000","last_change":"2026-02-24T22:34:53.903762+0000","last_active":"2026-02-24T22:34:53.903762+0000","last_peered":"2026-02-24T22:34:53.903762+0000","last_clean":"2026-02-24T22:34:53.903762+0000","last_became_active":"2026-02-24T22:05:20.194145+0000","last_became_peered":"2026-02-24T22:05:20.194145+0000","last_unstale":"2026-02-24T22:34:53.903762+0000","last_undegraded":"2026-02-24T22:34:53.903762+0000","last_fullsized":"2026-02-24T22:34:53.903762+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'4","last_scrub_stamp":"2026-02-24T22:34:53.903710+0000","last_deep_scrub":"312'4","last_deep_scrub_stamp":"2026-02-24T22:34:53.903710+0000","last_clean_scrub_stamp":"2026-02-24T22:34:53.903710+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:03:04.394145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":476,"num_read_kb":476,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"312'61","reported_seq":671,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:54.742682+0000","last_change":"2026-02-24T22:34:54.742682+0000","last_active":"2026-02-24T22:34:54.742682+0000","last_peered":"2026-02-24T22:34:54.742682+0000","last_clean":"2026-02-24T22:34:54.742682+0000","last_became_active":"2026-02-24T22:05:22.905221+0000","last_became_peered":"2026-02-24T22:05:22.905221+0000","last_unstale":"2026-02-24T22:34:54.742682+0000","last_undegraded":"2026-02-24T22:34:54.742682+0000","last_fullsized":"2026-02-24T22:34:54.742682+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'61","last_scrub_stamp":"2026-02-24T22:34:54.742603+0000","last_deep_scrub":"312'61","last_deep_scrub_stamp":"2026-02-24T22:34:54.742603+0000","last_clean_scrub_stamp":"2026-02-24T22:34:54.742603+0000","objects_scrubbed":5,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:52:44.639141+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"312'246","reported_seq":845,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.685022+0000","last_change":"2026-02-24T22:34:46.685022+0000","last_active":"2026-02-24T22:34:46.685022+0000","last_peered":"2026-02-24T22:34:46.685022+0000","last_clean":"2026-02-24T22:34:46.685022+0000","last_became_active":"2026-02-24T22:05:24.207673+0000","last_became_peered":"2026-02-24T22:05:24.207673+0000","last_unstale":"2026-02-24T22:34:46.685022+0000","last_undegraded":"2026-02-24T22:34:46.685022+0000","last_fullsized":"2026-02-24T22:34:46.685022+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'246","last_scrub_stamp":"2026-02-24T22:34:46.684962+0000","last_deep_scrub":"312'246","last_deep_scrub_stamp":"2026-02-24T22:34:46.684962+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.684962+0000","objects_scrubbed":87,"log_size":46,"log_dups_size":200,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:17:30.101644+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25987,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":107,"num_read_kb":103,"num_write":388,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":586,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:03.269486+0000","last_change":"2026-02-24T22:35:03.269486+0000","last_active":"2026-02-24T22:35:03.269486+0000","last_peered":"2026-02-24T22:35:03.269486+0000","last_clean":"2026-02-24T22:35:03.269486+0000","last_became_active":"2026-02-24T22:05:29.658842+0000","last_became_peered":"2026-02-24T22:05:29.658842+0000","last_unstale":"2026-02-24T22:35:03.269486+0000","last_undegraded":"2026-02-24T22:35:03.269486+0000","last_fullsized":"2026-02-24T22:35:03.269486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:03.269430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:03.269430+0000","last_clean_scrub_stamp":"2026-02-24T22:35:03.269430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:37:15.587437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"312'122","reported_seq":812,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.133092+0000","last_change":"2026-02-24T22:34:44.133092+0000","last_active":"2026-02-24T22:34:44.133092+0000","last_peered":"2026-02-24T22:34:44.133092+0000","last_clean":"2026-02-24T22:34:44.133092+0000","last_became_active":"2026-02-24T22:05:18.188161+0000","last_became_peered":"2026-02-24T22:05:18.188161+0000","last_unstale":"2026-02-24T22:34:44.133092+0000","last_undegraded":"2026-02-24T22:34:44.133092+0000","last_fullsized":"2026-02-24T22:34:44.133092+0000","mapping_epoch":24,"log_start":"312'98","ondisk_log_start":"312'98","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'122","last_scrub_stamp":"2026-02-24T22:34:44.133036+0000","last_deep_scrub":"312'122","last_deep_scrub_stamp":"2026-02-24T22:34:44.133036+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.133036+0000","objects_scrubbed":24,"log_size":24,"log_dups_size":98,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:15:18.743316+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":368,"num_read_kb":314,"num_write":134,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"312'360","reported_seq":1101,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.843127+0000","last_change":"2026-02-24T22:34:31.843071+0000","last_active":"2026-02-24T22:34:31.843127+0000","last_peered":"2026-02-24T22:34:31.843127+0000","last_clean":"2026-02-24T22:34:31.843127+0000","last_became_active":"2026-02-24T22:05:16.182194+0000","last_became_peered":"2026-02-24T22:05:16.182194+0000","last_unstale":"2026-02-24T22:34:31.843127+0000","last_undegraded":"2026-02-24T22:34:31.843127+0000","last_fullsized":"2026-02-24T22:34:31.843127+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'360","last_scrub_stamp":"2026-02-24T22:34:31.843001+0000","last_deep_scrub":"312'360","last_deep_scrub_stamp":"2026-02-24T22:34:31.843001+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.843001+0000","objects_scrubbed":4,"log_size":60,"log_dups_size":300,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:54:58.411257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":482,"num_read_kb":7457,"num_write":1288,"num_write_kb":8933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":599,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:52.297621+0000","last_change":"2026-02-24T22:34:52.297621+0000","last_active":"2026-02-24T22:34:52.297621+0000","last_peered":"2026-02-24T22:34:52.297621+0000","last_clean":"2026-02-24T22:34:52.297621+0000","last_became_active":"2026-02-24T22:05:20.193180+0000","last_became_peered":"2026-02-24T22:05:20.193180+0000","last_unstale":"2026-02-24T22:34:52.297621+0000","last_undegraded":"2026-02-24T22:34:52.297621+0000","last_fullsized":"2026-02-24T22:34:52.297621+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:52.297555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:52.297555+0000","last_clean_scrub_stamp":"2026-02-24T22:34:52.297555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:59:44.363996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"312'284","reported_seq":918,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:56.996699+0000","last_change":"2026-02-24T22:34:56.996699+0000","last_active":"2026-02-24T22:34:56.996699+0000","last_peered":"2026-02-24T22:34:56.996699+0000","last_clean":"2026-02-24T22:34:56.996699+0000","last_became_active":"2026-02-24T22:05:22.200138+0000","last_became_peered":"2026-02-24T22:05:22.200138+0000","last_unstale":"2026-02-24T22:34:56.996699+0000","last_undegraded":"2026-02-24T22:34:56.996699+0000","last_fullsized":"2026-02-24T22:34:56.996699+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'284","last_scrub_stamp":"2026-02-24T22:34:56.996649+0000","last_deep_scrub":"312'284","last_deep_scrub_stamp":"2026-02-24T22:34:56.996649+0000","last_clean_scrub_stamp":"2026-02-24T22:34:56.996649+0000","objects_scrubbed":9,"log_size":84,"log_dups_size":200,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:57:19.406987+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37516,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":871,"num_read_kb":1412,"num_write":1276,"num_write_kb":1198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":357372,"num_omap_keys":834,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"312'415","reported_seq":1053,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.648295+0000","last_change":"2026-02-24T22:34:47.648295+0000","last_active":"2026-02-24T22:34:47.648295+0000","last_peered":"2026-02-24T22:34:47.648295+0000","last_clean":"2026-02-24T22:34:47.648295+0000","last_became_active":"2026-02-24T22:05:24.206745+0000","last_became_peered":"2026-02-24T22:05:24.206745+0000","last_unstale":"2026-02-24T22:34:47.648295+0000","last_undegraded":"2026-02-24T22:34:47.648295+0000","last_fullsized":"2026-02-24T22:34:47.648295+0000","mapping_epoch":30,"log_start":"312'400","ondisk_log_start":"312'400","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'415","last_scrub_stamp":"2026-02-24T22:34:47.648223+0000","last_deep_scrub":"312'415","last_deep_scrub_stamp":"2026-02-24T22:34:47.648223+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.648223+0000","objects_scrubbed":88,"log_size":15,"log_dups_size":400,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:44:43.640744+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27098,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":391,"num_read_kb":376,"num_write":726,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":178,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"312'436","reported_seq":4809,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.362595+0000","last_change":"2026-02-24T22:34:58.362595+0000","last_active":"2026-02-24T22:34:58.362595+0000","last_peered":"2026-02-24T22:34:58.362595+0000","last_clean":"2026-02-24T22:34:58.362595+0000","last_became_active":"2026-02-24T22:05:29.654281+0000","last_became_peered":"2026-02-24T22:05:29.654281+0000","last_unstale":"2026-02-24T22:34:58.362595+0000","last_undegraded":"2026-02-24T22:34:58.362595+0000","last_fullsized":"2026-02-24T22:34:58.362595+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'436","last_scrub_stamp":"2026-02-24T22:34:58.362387+0000","last_deep_scrub":"312'436","last_deep_scrub_stamp":"2026-02-24T22:34:58.362387+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.362387+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":400,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:59:54.222522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"312'136","reported_seq":831,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.733879+0000","last_change":"2026-02-24T22:34:38.733879+0000","last_active":"2026-02-24T22:34:38.733879+0000","last_peered":"2026-02-24T22:34:38.733879+0000","last_clean":"2026-02-24T22:34:38.733879+0000","last_became_active":"2026-02-24T22:05:18.193426+0000","last_became_peered":"2026-02-24T22:05:18.193426+0000","last_unstale":"2026-02-24T22:34:38.733879+0000","last_undegraded":"2026-02-24T22:34:38.733879+0000","last_fullsized":"2026-02-24T22:34:38.733879+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'136","last_scrub_stamp":"2026-02-24T22:34:38.733817+0000","last_deep_scrub":"312'136","last_deep_scrub_stamp":"2026-02-24T22:34:38.733817+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.733817+0000","objects_scrubbed":22,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:36:52.810608+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":443,"num_read_kb":376,"num_write":199,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"312'114","reported_seq":756,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.176152+0000","last_change":"2026-02-24T22:34:32.176106+0000","last_active":"2026-02-24T22:34:32.176152+0000","last_peered":"2026-02-24T22:34:32.176152+0000","last_clean":"2026-02-24T22:34:32.176152+0000","last_became_active":"2026-02-24T22:05:06.708577+0000","last_became_peered":"2026-02-24T22:05:06.708577+0000","last_unstale":"2026-02-24T22:34:32.176152+0000","last_undegraded":"2026-02-24T22:34:32.176152+0000","last_fullsized":"2026-02-24T22:34:32.176152+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'114","last_scrub_stamp":"2026-02-24T22:34:32.176027+0000","last_deep_scrub":"312'114","last_deep_scrub_stamp":"2026-02-24T22:34:32.176027+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.176027+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:58:53.542668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"312'2","reported_seq":614,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.096943+0000","last_change":"2026-02-24T22:34:34.096943+0000","last_active":"2026-02-24T22:34:34.096943+0000","last_peered":"2026-02-24T22:34:34.096943+0000","last_clean":"2026-02-24T22:34:34.096943+0000","last_became_active":"2026-02-24T22:05:08.689983+0000","last_became_peered":"2026-02-24T22:05:08.689983+0000","last_unstale":"2026-02-24T22:34:34.096943+0000","last_undegraded":"2026-02-24T22:34:34.096943+0000","last_fullsized":"2026-02-24T22:34:34.096943+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:34:34.096879+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:34:34.096879+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.096879+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:03:29.099395+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00046251899999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"312'329","reported_seq":1116,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:29.895013+0000","last_change":"2026-02-24T22:34:29.895013+0000","last_active":"2026-02-24T22:34:29.895013+0000","last_peered":"2026-02-24T22:34:29.895013+0000","last_clean":"2026-02-24T22:34:29.895013+0000","last_became_active":"2026-02-24T22:05:16.185768+0000","last_became_peered":"2026-02-24T22:05:16.185768+0000","last_unstale":"2026-02-24T22:34:29.895013+0000","last_undegraded":"2026-02-24T22:34:29.895013+0000","last_fullsized":"2026-02-24T22:34:29.895013+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'329","last_scrub_stamp":"2026-02-24T22:34:29.894945+0000","last_deep_scrub":"312'329","last_deep_scrub_stamp":"2026-02-24T22:34:29.894945+0000","last_clean_scrub_stamp":"2026-02-24T22:34:29.894945+0000","objects_scrubbed":3,"log_size":29,"log_dups_size":300,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:02:39.272423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":685,"num_read_kb":12844,"num_write":1503,"num_write_kb":6232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":595,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.125003+0000","last_change":"2026-02-24T22:34:49.125003+0000","last_active":"2026-02-24T22:34:49.125003+0000","last_peered":"2026-02-24T22:34:49.125003+0000","last_clean":"2026-02-24T22:34:49.125003+0000","last_became_active":"2026-02-24T22:05:20.194040+0000","last_became_peered":"2026-02-24T22:05:20.194040+0000","last_unstale":"2026-02-24T22:34:49.125003+0000","last_undegraded":"2026-02-24T22:34:49.125003+0000","last_fullsized":"2026-02-24T22:34:49.125003+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:49.124939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:49.124939+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.124939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:16:47.321676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"312'1071","reported_seq":2189,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:54.936175+0000","last_change":"2026-02-24T22:34:54.936135+0000","last_active":"2026-02-24T22:34:54.936175+0000","last_peered":"2026-02-24T22:34:54.936175+0000","last_clean":"2026-02-24T22:34:54.936175+0000","last_became_active":"2026-02-24T22:05:22.199602+0000","last_became_peered":"2026-02-24T22:05:22.199602+0000","last_unstale":"2026-02-24T22:34:54.936175+0000","last_undegraded":"2026-02-24T22:34:54.936175+0000","last_fullsized":"2026-02-24T22:34:54.936175+0000","mapping_epoch":28,"log_start":"312'1000","ondisk_log_start":"312'1000","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'1071","last_scrub_stamp":"2026-02-24T22:34:54.936052+0000","last_deep_scrub":"312'1071","last_deep_scrub_stamp":"2026-02-24T22:34:54.936052+0000","last_clean_scrub_stamp":"2026-02-24T22:34:54.936052+0000","objects_scrubbed":8,"log_size":71,"log_dups_size":1000,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:42:38.805146+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":116028,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2941,"num_read_kb":3705,"num_write":1026,"num_write_kb":950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17602,"num_omap_keys":42,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"312'302","reported_seq":910,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:57.954636+0000","last_change":"2026-02-24T22:34:57.954594+0000","last_active":"2026-02-24T22:34:57.954636+0000","last_peered":"2026-02-24T22:34:57.954636+0000","last_clean":"2026-02-24T22:34:57.954636+0000","last_became_active":"2026-02-24T22:05:24.200336+0000","last_became_peered":"2026-02-24T22:05:24.200336+0000","last_unstale":"2026-02-24T22:34:57.954636+0000","last_undegraded":"2026-02-24T22:34:57.954636+0000","last_fullsized":"2026-02-24T22:34:57.954636+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'302","last_scrub_stamp":"2026-02-24T22:34:57.954521+0000","last_deep_scrub":"312'302","last_deep_scrub_stamp":"2026-02-24T22:34:57.954521+0000","last_clean_scrub_stamp":"2026-02-24T22:34:57.954521+0000","objects_scrubbed":100,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:53:46.560141+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32535,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":152,"num_read_kb":139,"num_write":501,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"312'428","reported_seq":1845,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:54.584199+0000","last_change":"2026-02-24T22:34:54.584117+0000","last_active":"2026-02-24T22:34:54.584199+0000","last_peered":"2026-02-24T22:34:54.584199+0000","last_clean":"2026-02-24T22:34:54.584199+0000","last_became_active":"2026-02-24T22:05:29.660370+0000","last_became_peered":"2026-02-24T22:05:29.660370+0000","last_unstale":"2026-02-24T22:34:54.584199+0000","last_undegraded":"2026-02-24T22:34:54.584199+0000","last_fullsized":"2026-02-24T22:34:54.584199+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'428","last_scrub_stamp":"2026-02-24T22:34:54.584009+0000","last_deep_scrub":"312'428","last_deep_scrub_stamp":"2026-02-24T22:34:54.584009+0000","last_clean_scrub_stamp":"2026-02-24T22:34:54.584009+0000","objects_scrubbed":1,"log_size":28,"log_dups_size":400,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:22:23.409209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"312'124","reported_seq":832,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.850232+0000","last_change":"2026-02-24T22:34:41.850193+0000","last_active":"2026-02-24T22:34:41.850232+0000","last_peered":"2026-02-24T22:34:41.850232+0000","last_clean":"2026-02-24T22:34:41.850232+0000","last_became_active":"2026-02-24T22:05:18.185205+0000","last_became_peered":"2026-02-24T22:05:18.185205+0000","last_unstale":"2026-02-24T22:34:41.850232+0000","last_undegraded":"2026-02-24T22:34:41.850232+0000","last_fullsized":"2026-02-24T22:34:41.850232+0000","mapping_epoch":24,"log_start":"312'99","ondisk_log_start":"312'99","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'124","last_scrub_stamp":"2026-02-24T22:34:41.850136+0000","last_deep_scrub":"312'124","last_deep_scrub_stamp":"2026-02-24T22:34:41.850136+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.850136+0000","objects_scrubbed":25,"log_size":25,"log_dups_size":99,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:53:33.276473+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":346,"num_read_kb":300,"num_write":140,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":619,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.150208+0000","last_change":"2026-02-24T22:34:36.150171+0000","last_active":"2026-02-24T22:34:36.150208+0000","last_peered":"2026-02-24T22:34:36.150208+0000","last_clean":"2026-02-24T22:34:36.150208+0000","last_became_active":"2026-02-24T22:05:08.689972+0000","last_became_peered":"2026-02-24T22:05:08.689972+0000","last_unstale":"2026-02-24T22:34:36.150208+0000","last_undegraded":"2026-02-24T22:34:36.150208+0000","last_fullsized":"2026-02-24T22:34:36.150208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:36.150114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:36.150114+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.150114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:06:01.715910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048345700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"312'262","reported_seq":1029,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:29.701991+0000","last_change":"2026-02-24T22:34:29.701952+0000","last_active":"2026-02-24T22:34:29.701991+0000","last_peered":"2026-02-24T22:34:29.701991+0000","last_clean":"2026-02-24T22:34:29.701991+0000","last_became_active":"2026-02-24T22:05:16.183484+0000","last_became_peered":"2026-02-24T22:05:16.183484+0000","last_unstale":"2026-02-24T22:34:29.701991+0000","last_undegraded":"2026-02-24T22:34:29.701991+0000","last_fullsized":"2026-02-24T22:34:29.701991+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'262","last_scrub_stamp":"2026-02-24T22:34:29.701911+0000","last_deep_scrub":"312'262","last_deep_scrub_stamp":"2026-02-24T22:34:29.701911+0000","last_clean_scrub_stamp":"2026-02-24T22:34:29.701911+0000","objects_scrubbed":4,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:16:54.158907+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":574,"num_read_kb":821,"num_write":1287,"num_write_kb":10723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":589,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.883557+0000","last_change":"2026-02-24T22:34:49.883509+0000","last_active":"2026-02-24T22:34:49.883557+0000","last_peered":"2026-02-24T22:34:49.883557+0000","last_clean":"2026-02-24T22:34:49.883557+0000","last_became_active":"2026-02-24T22:05:20.195236+0000","last_became_peered":"2026-02-24T22:05:20.195236+0000","last_unstale":"2026-02-24T22:34:49.883557+0000","last_undegraded":"2026-02-24T22:34:49.883557+0000","last_fullsized":"2026-02-24T22:34:49.883557+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:49.883446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:49.883446+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.883446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:59:34.535514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"312'1070","reported_seq":2301,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:51.232440+0000","last_change":"2026-02-24T22:34:51.232440+0000","last_active":"2026-02-24T22:34:51.232440+0000","last_peered":"2026-02-24T22:34:51.232440+0000","last_clean":"2026-02-24T22:34:51.232440+0000","last_became_active":"2026-02-24T22:05:22.192295+0000","last_became_peered":"2026-02-24T22:05:22.192295+0000","last_unstale":"2026-02-24T22:34:51.232440+0000","last_undegraded":"2026-02-24T22:34:51.232440+0000","last_fullsized":"2026-02-24T22:34:51.232440+0000","mapping_epoch":28,"log_start":"312'1001","ondisk_log_start":"312'1001","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'1070","last_scrub_stamp":"2026-02-24T22:34:51.232390+0000","last_deep_scrub":"312'1070","last_deep_scrub_stamp":"2026-02-24T22:34:51.232390+0000","last_clean_scrub_stamp":"2026-02-24T22:34:51.232390+0000","objects_scrubbed":12,"log_size":69,"log_dups_size":1001,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:26:43.381792+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":114655,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":3192,"num_read_kb":4794,"num_write":3958,"num_write_kb":3891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":517874,"num_omap_keys":3022,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"312'271","reported_seq":866,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:01.238162+0000","last_change":"2026-02-24T22:35:01.238162+0000","last_active":"2026-02-24T22:35:01.238162+0000","last_peered":"2026-02-24T22:35:01.238162+0000","last_clean":"2026-02-24T22:35:01.238162+0000","last_became_active":"2026-02-24T22:05:24.204774+0000","last_became_peered":"2026-02-24T22:05:24.204774+0000","last_unstale":"2026-02-24T22:35:01.238162+0000","last_undegraded":"2026-02-24T22:35:01.238162+0000","last_fullsized":"2026-02-24T22:35:01.238162+0000","mapping_epoch":30,"log_start":"312'196","ondisk_log_start":"312'196","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'271","last_scrub_stamp":"2026-02-24T22:35:01.238098+0000","last_deep_scrub":"312'271","last_deep_scrub_stamp":"2026-02-24T22:35:01.238098+0000","last_clean_scrub_stamp":"2026-02-24T22:35:01.238098+0000","objects_scrubbed":89,"log_size":75,"log_dups_size":196,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:08:24.544989+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29814,"num_objects":89,"num_object_clones":0,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":0,"num_read":112,"num_read_kb":108,"num_write":454,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":583,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:59.339684+0000","last_change":"2026-02-24T22:34:59.339684+0000","last_active":"2026-02-24T22:34:59.339684+0000","last_peered":"2026-02-24T22:34:59.339684+0000","last_clean":"2026-02-24T22:34:59.339684+0000","last_became_active":"2026-02-24T22:05:29.653391+0000","last_became_peered":"2026-02-24T22:05:29.653391+0000","last_unstale":"2026-02-24T22:34:59.339684+0000","last_undegraded":"2026-02-24T22:34:59.339684+0000","last_fullsized":"2026-02-24T22:34:59.339684+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:59.339612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:59.339612+0000","last_clean_scrub_stamp":"2026-02-24T22:34:59.339612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:54:10.672461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"312'130","reported_seq":830,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.122558+0000","last_change":"2026-02-24T22:34:42.122528+0000","last_active":"2026-02-24T22:34:42.122558+0000","last_peered":"2026-02-24T22:34:42.122558+0000","last_clean":"2026-02-24T22:34:42.122558+0000","last_became_active":"2026-02-24T22:05:18.186126+0000","last_became_peered":"2026-02-24T22:05:18.186126+0000","last_unstale":"2026-02-24T22:34:42.122558+0000","last_undegraded":"2026-02-24T22:34:42.122558+0000","last_fullsized":"2026-02-24T22:34:42.122558+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'130","last_scrub_stamp":"2026-02-24T22:34:42.122496+0000","last_deep_scrub":"312'130","last_deep_scrub_stamp":"2026-02-24T22:34:42.122496+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.122496+0000","objects_scrubbed":29,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:13:06.464986+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":336,"num_read_kb":289,"num_write":128,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":626,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:30.093328+0000","last_change":"2026-02-24T22:34:30.093328+0000","last_active":"2026-02-24T22:34:30.093328+0000","last_peered":"2026-02-24T22:34:30.093328+0000","last_clean":"2026-02-24T22:34:30.093328+0000","last_became_active":"2026-02-24T22:05:08.682035+0000","last_became_peered":"2026-02-24T22:05:08.682035+0000","last_unstale":"2026-02-24T22:34:30.093328+0000","last_undegraded":"2026-02-24T22:34:30.093328+0000","last_fullsized":"2026-02-24T22:34:30.093328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:30.093252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:30.093252+0000","last_clean_scrub_stamp":"2026-02-24T22:34:30.093252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:06:58.991217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047531599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"312'305","reported_seq":1058,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.010931+0000","last_change":"2026-02-24T22:34:34.010886+0000","last_active":"2026-02-24T22:34:34.010931+0000","last_peered":"2026-02-24T22:34:34.010931+0000","last_clean":"2026-02-24T22:34:34.010931+0000","last_became_active":"2026-02-24T22:05:16.178331+0000","last_became_peered":"2026-02-24T22:05:16.178331+0000","last_unstale":"2026-02-24T22:34:34.010931+0000","last_undegraded":"2026-02-24T22:34:34.010931+0000","last_fullsized":"2026-02-24T22:34:34.010931+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'305","last_scrub_stamp":"2026-02-24T22:34:34.010799+0000","last_deep_scrub":"312'305","last_deep_scrub_stamp":"2026-02-24T22:34:34.010799+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.010799+0000","objects_scrubbed":6,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:33:52.230516+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":583,"num_read_kb":6589,"num_write":1333,"num_write_kb":11354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"312'2","reported_seq":605,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.063520+0000","last_change":"2026-02-24T22:34:42.063471+0000","last_active":"2026-02-24T22:34:42.063520+0000","last_peered":"2026-02-24T22:34:42.063520+0000","last_clean":"2026-02-24T22:34:42.063520+0000","last_became_active":"2026-02-24T22:05:20.187908+0000","last_became_peered":"2026-02-24T22:05:20.187908+0000","last_unstale":"2026-02-24T22:34:42.063520+0000","last_undegraded":"2026-02-24T22:34:42.063520+0000","last_fullsized":"2026-02-24T22:34:42.063520+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:34:42.063424+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:34:42.063424+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.063424+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:16:34.733800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"312'447","reported_seq":1665,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:54.231930+0000","last_change":"2026-02-24T22:34:54.231930+0000","last_active":"2026-02-24T22:34:54.231930+0000","last_peered":"2026-02-24T22:34:54.231930+0000","last_clean":"2026-02-24T22:34:54.231930+0000","last_became_active":"2026-02-24T22:05:22.912414+0000","last_became_peered":"2026-02-24T22:05:22.912414+0000","last_unstale":"2026-02-24T22:34:54.231930+0000","last_undegraded":"2026-02-24T22:34:54.231930+0000","last_fullsized":"2026-02-24T22:34:54.231930+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'447","last_scrub_stamp":"2026-02-24T22:34:54.231891+0000","last_deep_scrub":"312'447","last_deep_scrub_stamp":"2026-02-24T22:34:54.231891+0000","last_clean_scrub_stamp":"2026-02-24T22:34:54.231891+0000","objects_scrubbed":10,"log_size":47,"log_dups_size":400,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:44:31.798158+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":308,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1080,"num_read_kb":599,"num_write":114,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5278,"num_omap_keys":12,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"312'281","reported_seq":882,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:56.954757+0000","last_change":"2026-02-24T22:34:56.954757+0000","last_active":"2026-02-24T22:34:56.954757+0000","last_peered":"2026-02-24T22:34:56.954757+0000","last_clean":"2026-02-24T22:34:56.954757+0000","last_became_active":"2026-02-24T22:05:24.200029+0000","last_became_peered":"2026-02-24T22:05:24.200029+0000","last_unstale":"2026-02-24T22:34:56.954757+0000","last_undegraded":"2026-02-24T22:34:56.954757+0000","last_fullsized":"2026-02-24T22:34:56.954757+0000","mapping_epoch":30,"log_start":"312'198","ondisk_log_start":"312'198","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'281","last_scrub_stamp":"2026-02-24T22:34:56.954689+0000","last_deep_scrub":"312'281","last_deep_scrub_stamp":"2026-02-24T22:34:56.954689+0000","last_clean_scrub_stamp":"2026-02-24T22:34:56.954689+0000","objects_scrubbed":97,"log_size":83,"log_dups_size":198,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:56:30.907099+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29918,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":442,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":585,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:04.581459+0000","last_change":"2026-02-24T22:35:04.581459+0000","last_active":"2026-02-24T22:35:04.581459+0000","last_peered":"2026-02-24T22:35:04.581459+0000","last_clean":"2026-02-24T22:35:04.581459+0000","last_became_active":"2026-02-24T22:05:29.653806+0000","last_became_peered":"2026-02-24T22:05:29.653806+0000","last_unstale":"2026-02-24T22:35:04.581459+0000","last_undegraded":"2026-02-24T22:35:04.581459+0000","last_fullsized":"2026-02-24T22:35:04.581459+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:04.581396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:04.581396+0000","last_clean_scrub_stamp":"2026-02-24T22:35:04.581396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:10:35.970116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"312'254","reported_seq":1004,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.068312+0000","last_change":"2026-02-24T22:34:38.068312+0000","last_active":"2026-02-24T22:34:38.068312+0000","last_peered":"2026-02-24T22:34:38.068312+0000","last_clean":"2026-02-24T22:34:38.068312+0000","last_became_active":"2026-02-24T22:05:18.183952+0000","last_became_peered":"2026-02-24T22:05:18.183952+0000","last_unstale":"2026-02-24T22:34:38.068312+0000","last_undegraded":"2026-02-24T22:34:38.068312+0000","last_fullsized":"2026-02-24T22:34:38.068312+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'254","last_scrub_stamp":"2026-02-24T22:34:38.068260+0000","last_deep_scrub":"312'254","last_deep_scrub_stamp":"2026-02-24T22:34:38.068260+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.068260+0000","objects_scrubbed":24,"log_size":54,"log_dups_size":200,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:24:48.806854+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":963,"num_read_kb":811,"num_write":374,"num_write_kb":210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"312'3","reported_seq":615,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.112722+0000","last_change":"2026-02-24T22:34:31.112722+0000","last_active":"2026-02-24T22:34:31.112722+0000","last_peered":"2026-02-24T22:34:31.112722+0000","last_clean":"2026-02-24T22:34:31.112722+0000","last_became_active":"2026-02-24T22:05:08.690042+0000","last_became_peered":"2026-02-24T22:05:08.690042+0000","last_unstale":"2026-02-24T22:34:31.112722+0000","last_undegraded":"2026-02-24T22:34:31.112722+0000","last_fullsized":"2026-02-24T22:34:31.112722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'3","last_scrub_stamp":"2026-02-24T22:34:31.112652+0000","last_deep_scrub":"312'3","last_deep_scrub_stamp":"2026-02-24T22:34:31.112652+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.112652+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:38:30.546771+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0010128400000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"312'333","reported_seq":1056,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:30.596222+0000","last_change":"2026-02-24T22:34:30.596222+0000","last_active":"2026-02-24T22:34:30.596222+0000","last_peered":"2026-02-24T22:34:30.596222+0000","last_clean":"2026-02-24T22:34:30.596222+0000","last_became_active":"2026-02-24T22:05:16.188900+0000","last_became_peered":"2026-02-24T22:05:16.188900+0000","last_unstale":"2026-02-24T22:34:30.596222+0000","last_undegraded":"2026-02-24T22:34:30.596222+0000","last_fullsized":"2026-02-24T22:34:30.596222+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'333","last_scrub_stamp":"2026-02-24T22:34:30.596162+0000","last_deep_scrub":"312'333","last_deep_scrub_stamp":"2026-02-24T22:34:30.596162+0000","last_clean_scrub_stamp":"2026-02-24T22:34:30.596162+0000","objects_scrubbed":6,"log_size":33,"log_dups_size":300,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:56:38.748514+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242895,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":415,"num_read_kb":10969,"num_write":1277,"num_write_kb":10859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":598,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.915846+0000","last_change":"2026-02-24T22:34:49.915846+0000","last_active":"2026-02-24T22:34:49.915846+0000","last_peered":"2026-02-24T22:34:49.915846+0000","last_clean":"2026-02-24T22:34:49.915846+0000","last_became_active":"2026-02-24T22:05:20.187251+0000","last_became_peered":"2026-02-24T22:05:20.187251+0000","last_unstale":"2026-02-24T22:34:49.915846+0000","last_undegraded":"2026-02-24T22:34:49.915846+0000","last_fullsized":"2026-02-24T22:34:49.915846+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:49.915785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:49.915785+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.915785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:30:18.871540+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"312'83","reported_seq":713,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:52.786149+0000","last_change":"2026-02-24T22:34:52.786106+0000","last_active":"2026-02-24T22:34:52.786149+0000","last_peered":"2026-02-24T22:34:52.786149+0000","last_clean":"2026-02-24T22:34:52.786149+0000","last_became_active":"2026-02-24T22:05:22.191394+0000","last_became_peered":"2026-02-24T22:05:22.191394+0000","last_unstale":"2026-02-24T22:34:52.786149+0000","last_undegraded":"2026-02-24T22:34:52.786149+0000","last_fullsized":"2026-02-24T22:34:52.786149+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'83","last_scrub_stamp":"2026-02-24T22:34:52.786051+0000","last_deep_scrub":"312'83","last_deep_scrub_stamp":"2026-02-24T22:34:52.786051+0000","last_clean_scrub_stamp":"2026-02-24T22:34:52.786051+0000","objects_scrubbed":7,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:23:33.675567+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":103,"num_read_kb":96,"num_write":72,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1630,"num_omap_keys":4,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"312'323","reported_seq":952,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.637715+0000","last_change":"2026-02-24T22:34:48.637677+0000","last_active":"2026-02-24T22:34:48.637715+0000","last_peered":"2026-02-24T22:34:48.637715+0000","last_clean":"2026-02-24T22:34:48.637715+0000","last_became_active":"2026-02-24T22:05:24.207698+0000","last_became_peered":"2026-02-24T22:05:24.207698+0000","last_unstale":"2026-02-24T22:34:48.637715+0000","last_undegraded":"2026-02-24T22:34:48.637715+0000","last_fullsized":"2026-02-24T22:34:48.637715+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'323","last_scrub_stamp":"2026-02-24T22:34:48.637620+0000","last_deep_scrub":"312'323","last_deep_scrub_stamp":"2026-02-24T22:34:48.637620+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.637620+0000","objects_scrubbed":94,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:13:22.580314+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31273,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":246,"num_read_kb":224,"num_write":553,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":575,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:00.855093+0000","last_change":"2026-02-24T22:35:00.855050+0000","last_active":"2026-02-24T22:35:00.855093+0000","last_peered":"2026-02-24T22:35:00.855093+0000","last_clean":"2026-02-24T22:35:00.855093+0000","last_became_active":"2026-02-24T22:05:29.660065+0000","last_became_peered":"2026-02-24T22:05:29.660065+0000","last_unstale":"2026-02-24T22:35:00.855093+0000","last_undegraded":"2026-02-24T22:35:00.855093+0000","last_fullsized":"2026-02-24T22:35:00.855093+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:00.855003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:00.855003+0000","last_clean_scrub_stamp":"2026-02-24T22:35:00.855003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:46:34.269073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"312'191","reported_seq":898,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.101612+0000","last_change":"2026-02-24T22:34:39.101612+0000","last_active":"2026-02-24T22:34:39.101612+0000","last_peered":"2026-02-24T22:34:39.101612+0000","last_clean":"2026-02-24T22:34:39.101612+0000","last_became_active":"2026-02-24T22:05:18.185112+0000","last_became_peered":"2026-02-24T22:05:18.185112+0000","last_unstale":"2026-02-24T22:34:39.101612+0000","last_undegraded":"2026-02-24T22:34:39.101612+0000","last_fullsized":"2026-02-24T22:34:39.101612+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'191","last_scrub_stamp":"2026-02-24T22:34:39.101566+0000","last_deep_scrub":"312'191","last_deep_scrub_stamp":"2026-02-24T22:34:39.101566+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.101566+0000","objects_scrubbed":19,"log_size":91,"log_dups_size":100,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:44:31.875433+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":679,"num_read_kb":553,"num_write":246,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"312'281","reported_seq":1011,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.103881+0000","last_change":"2026-02-24T22:34:32.103832+0000","last_active":"2026-02-24T22:34:32.103881+0000","last_peered":"2026-02-24T22:34:32.103881+0000","last_clean":"2026-02-24T22:34:32.103881+0000","last_became_active":"2026-02-24T22:05:16.191007+0000","last_became_peered":"2026-02-24T22:05:16.191007+0000","last_unstale":"2026-02-24T22:34:32.103881+0000","last_undegraded":"2026-02-24T22:34:32.103881+0000","last_fullsized":"2026-02-24T22:34:32.103881+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'281","last_scrub_stamp":"2026-02-24T22:34:32.103780+0000","last_deep_scrub":"312'281","last_deep_scrub_stamp":"2026-02-24T22:34:32.103780+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.103780+0000","objects_scrubbed":4,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:02:28.007749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":495,"num_read_kb":40330,"num_write":1338,"num_write_kb":18546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":621,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:29.991920+0000","last_change":"2026-02-24T22:34:29.991920+0000","last_active":"2026-02-24T22:34:29.991920+0000","last_peered":"2026-02-24T22:34:29.991920+0000","last_clean":"2026-02-24T22:34:29.991920+0000","last_became_active":"2026-02-24T22:05:08.758282+0000","last_became_peered":"2026-02-24T22:05:08.758282+0000","last_unstale":"2026-02-24T22:34:29.991920+0000","last_undegraded":"2026-02-24T22:34:29.991920+0000","last_fullsized":"2026-02-24T22:34:29.991920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:29.991857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:29.991857+0000","last_clean_scrub_stamp":"2026-02-24T22:34:29.991857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:47:43.334173+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00033355899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":598,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.948604+0000","last_change":"2026-02-24T22:34:48.948604+0000","last_active":"2026-02-24T22:34:48.948604+0000","last_peered":"2026-02-24T22:34:48.948604+0000","last_clean":"2026-02-24T22:34:48.948604+0000","last_became_active":"2026-02-24T22:05:20.187208+0000","last_became_peered":"2026-02-24T22:05:20.187208+0000","last_unstale":"2026-02-24T22:34:48.948604+0000","last_undegraded":"2026-02-24T22:34:48.948604+0000","last_fullsized":"2026-02-24T22:34:48.948604+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:48.948541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:48.948541+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.948541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:29:13.712233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"312'1143","reported_seq":2506,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:52.891975+0000","last_change":"2026-02-24T22:34:52.891935+0000","last_active":"2026-02-24T22:34:52.891975+0000","last_peered":"2026-02-24T22:34:52.891975+0000","last_clean":"2026-02-24T22:34:52.891975+0000","last_became_active":"2026-02-24T22:05:22.191763+0000","last_became_peered":"2026-02-24T22:05:22.191763+0000","last_unstale":"2026-02-24T22:34:52.891975+0000","last_undegraded":"2026-02-24T22:34:52.891975+0000","last_fullsized":"2026-02-24T22:34:52.891975+0000","mapping_epoch":28,"log_start":"312'1100","ondisk_log_start":"312'1100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'1143","last_scrub_stamp":"2026-02-24T22:34:52.891844+0000","last_deep_scrub":"312'1143","last_deep_scrub_stamp":"2026-02-24T22:34:52.891844+0000","last_clean_scrub_stamp":"2026-02-24T22:34:52.891844+0000","objects_scrubbed":12,"log_size":43,"log_dups_size":1100,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:24:36.081418+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":76731,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":3517,"num_read_kb":3087,"num_write":637,"num_write_kb":534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3608,"num_omap_keys":6,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"312'306","reported_seq":925,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:00.358718+0000","last_change":"2026-02-24T22:35:00.358718+0000","last_active":"2026-02-24T22:35:00.358718+0000","last_peered":"2026-02-24T22:35:00.358718+0000","last_clean":"2026-02-24T22:35:00.358718+0000","last_became_active":"2026-02-24T22:05:24.205923+0000","last_became_peered":"2026-02-24T22:05:24.205923+0000","last_unstale":"2026-02-24T22:35:00.358718+0000","last_undegraded":"2026-02-24T22:35:00.358718+0000","last_fullsized":"2026-02-24T22:35:00.358718+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'306","last_scrub_stamp":"2026-02-24T22:35:00.358668+0000","last_deep_scrub":"312'306","last_deep_scrub_stamp":"2026-02-24T22:35:00.358668+0000","last_clean_scrub_stamp":"2026-02-24T22:35:00.358668+0000","objects_scrubbed":102,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:31:13.435072+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34551,"num_objects":102,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":190,"num_read_kb":174,"num_write":516,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"312'468","reported_seq":4839,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:05.755883+0000","last_change":"2026-02-24T22:35:05.755883+0000","last_active":"2026-02-24T22:35:05.755883+0000","last_peered":"2026-02-24T22:35:05.755883+0000","last_clean":"2026-02-24T22:35:05.755883+0000","last_became_active":"2026-02-24T22:05:29.652259+0000","last_became_peered":"2026-02-24T22:05:29.652259+0000","last_unstale":"2026-02-24T22:35:05.755883+0000","last_undegraded":"2026-02-24T22:35:05.755883+0000","last_fullsized":"2026-02-24T22:35:05.755883+0000","mapping_epoch":33,"log_start":"312'401","ondisk_log_start":"312'401","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'468","last_scrub_stamp":"2026-02-24T22:35:05.755823+0000","last_deep_scrub":"312'468","last_deep_scrub_stamp":"2026-02-24T22:35:05.755823+0000","last_clean_scrub_stamp":"2026-02-24T22:35:05.755823+0000","objects_scrubbed":1,"log_size":67,"log_dups_size":401,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:50:47.482426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"312'136","reported_seq":842,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.775557+0000","last_change":"2026-02-24T22:34:39.775557+0000","last_active":"2026-02-24T22:34:39.775557+0000","last_peered":"2026-02-24T22:34:39.775557+0000","last_clean":"2026-02-24T22:34:39.775557+0000","last_became_active":"2026-02-24T22:05:18.183369+0000","last_became_peered":"2026-02-24T22:05:18.183369+0000","last_unstale":"2026-02-24T22:34:39.775557+0000","last_undegraded":"2026-02-24T22:34:39.775557+0000","last_fullsized":"2026-02-24T22:34:39.775557+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'136","last_scrub_stamp":"2026-02-24T22:34:39.775488+0000","last_deep_scrub":"312'136","last_deep_scrub_stamp":"2026-02-24T22:34:39.775488+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.775488+0000","objects_scrubbed":22,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:17:51.694013+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":458,"num_read_kb":386,"num_write":168,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"312'500","reported_seq":1319,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.014188+0000","last_change":"2026-02-24T22:34:31.014188+0000","last_active":"2026-02-24T22:34:31.014188+0000","last_peered":"2026-02-24T22:34:31.014188+0000","last_clean":"2026-02-24T22:34:31.014188+0000","last_became_active":"2026-02-24T22:05:16.178353+0000","last_became_peered":"2026-02-24T22:05:16.178353+0000","last_unstale":"2026-02-24T22:34:31.014188+0000","last_undegraded":"2026-02-24T22:34:31.014188+0000","last_fullsized":"2026-02-24T22:34:31.014188+0000","mapping_epoch":22,"log_start":"312'400","ondisk_log_start":"312'400","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'500","last_scrub_stamp":"2026-02-24T22:34:31.014143+0000","last_deep_scrub":"312'500","last_deep_scrub_stamp":"2026-02-24T22:34:31.014143+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.014143+0000","objects_scrubbed":3,"log_size":100,"log_dups_size":400,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:02:25.941904+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1036,"num_read_kb":9549,"num_write":2046,"num_write_kb":2098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":618,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.179872+0000","last_change":"2026-02-24T22:34:31.179872+0000","last_active":"2026-02-24T22:34:31.179872+0000","last_peered":"2026-02-24T22:34:31.179872+0000","last_clean":"2026-02-24T22:34:31.179872+0000","last_became_active":"2026-02-24T22:05:08.765129+0000","last_became_peered":"2026-02-24T22:05:08.765129+0000","last_unstale":"2026-02-24T22:34:31.179872+0000","last_undegraded":"2026-02-24T22:34:31.179872+0000","last_fullsized":"2026-02-24T22:34:31.179872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:31.179812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:31.179812+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.179812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:51:58.379988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000439276,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":593,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.666471+0000","last_change":"2026-02-24T22:34:43.666471+0000","last_active":"2026-02-24T22:34:43.666471+0000","last_peered":"2026-02-24T22:34:43.666471+0000","last_clean":"2026-02-24T22:34:43.666471+0000","last_became_active":"2026-02-24T22:05:20.192811+0000","last_became_peered":"2026-02-24T22:05:20.192811+0000","last_unstale":"2026-02-24T22:34:43.666471+0000","last_undegraded":"2026-02-24T22:34:43.666471+0000","last_fullsized":"2026-02-24T22:34:43.666471+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:43.666412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:43.666412+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.666412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:08:50.485350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"312'619","reported_seq":1738,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:50.237395+0000","last_change":"2026-02-24T22:34:50.237395+0000","last_active":"2026-02-24T22:34:50.237395+0000","last_peered":"2026-02-24T22:34:50.237395+0000","last_clean":"2026-02-24T22:34:50.237395+0000","last_became_active":"2026-02-24T22:05:22.905663+0000","last_became_peered":"2026-02-24T22:05:22.905663+0000","last_unstale":"2026-02-24T22:34:50.237395+0000","last_undegraded":"2026-02-24T22:34:50.237395+0000","last_fullsized":"2026-02-24T22:34:50.237395+0000","mapping_epoch":28,"log_start":"312'600","ondisk_log_start":"312'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'619","last_scrub_stamp":"2026-02-24T22:34:50.237359+0000","last_deep_scrub":"312'619","last_deep_scrub_stamp":"2026-02-24T22:34:50.237359+0000","last_clean_scrub_stamp":"2026-02-24T22:34:50.237359+0000","objects_scrubbed":10,"log_size":19,"log_dups_size":600,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:03:58.542051+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":308,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":2064,"num_read_kb":1111,"num_write":154,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5320,"num_omap_keys":12,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"312'261","reported_seq":858,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.006402+0000","last_change":"2026-02-24T22:34:58.006364+0000","last_active":"2026-02-24T22:34:58.006402+0000","last_peered":"2026-02-24T22:34:58.006402+0000","last_clean":"2026-02-24T22:34:58.006402+0000","last_became_active":"2026-02-24T22:05:24.205847+0000","last_became_peered":"2026-02-24T22:05:24.205847+0000","last_unstale":"2026-02-24T22:34:58.006402+0000","last_undegraded":"2026-02-24T22:34:58.006402+0000","last_fullsized":"2026-02-24T22:34:58.006402+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'261","last_scrub_stamp":"2026-02-24T22:34:58.006321+0000","last_deep_scrub":"312'261","last_deep_scrub_stamp":"2026-02-24T22:34:58.006321+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.006321+0000","objects_scrubbed":90,"log_size":61,"log_dups_size":200,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:12:15.690665+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26185,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":121,"num_read_kb":112,"num_write":413,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":585,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:05.578691+0000","last_change":"2026-02-24T22:35:05.578691+0000","last_active":"2026-02-24T22:35:05.578691+0000","last_peered":"2026-02-24T22:35:05.578691+0000","last_clean":"2026-02-24T22:35:05.578691+0000","last_became_active":"2026-02-24T22:05:29.651401+0000","last_became_peered":"2026-02-24T22:05:29.651401+0000","last_unstale":"2026-02-24T22:35:05.578691+0000","last_undegraded":"2026-02-24T22:35:05.578691+0000","last_fullsized":"2026-02-24T22:35:05.578691+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:05.578627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:05.578627+0000","last_clean_scrub_stamp":"2026-02-24T22:35:05.578627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:05:23.173834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"312'175","reported_seq":882,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.748052+0000","last_change":"2026-02-24T22:34:40.748014+0000","last_active":"2026-02-24T22:34:40.748052+0000","last_peered":"2026-02-24T22:34:40.748052+0000","last_clean":"2026-02-24T22:34:40.748052+0000","last_became_active":"2026-02-24T22:05:18.185592+0000","last_became_peered":"2026-02-24T22:05:18.185592+0000","last_unstale":"2026-02-24T22:34:40.748052+0000","last_undegraded":"2026-02-24T22:34:40.748052+0000","last_fullsized":"2026-02-24T22:34:40.748052+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'175","last_scrub_stamp":"2026-02-24T22:34:40.747966+0000","last_deep_scrub":"312'175","last_deep_scrub_stamp":"2026-02-24T22:34:40.747966+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.747966+0000","objects_scrubbed":25,"log_size":75,"log_dups_size":100,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:49:27.517475+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":536,"num_read_kb":444,"num_write":207,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"312'316","reported_seq":1051,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:30.882422+0000","last_change":"2026-02-24T22:34:30.882380+0000","last_active":"2026-02-24T22:34:30.882422+0000","last_peered":"2026-02-24T22:34:30.882422+0000","last_clean":"2026-02-24T22:34:30.882422+0000","last_became_active":"2026-02-24T22:05:16.184131+0000","last_became_peered":"2026-02-24T22:05:16.184131+0000","last_unstale":"2026-02-24T22:34:30.882422+0000","last_undegraded":"2026-02-24T22:34:30.882422+0000","last_fullsized":"2026-02-24T22:34:30.882422+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'316","last_scrub_stamp":"2026-02-24T22:34:30.882351+0000","last_deep_scrub":"312'316","last_deep_scrub_stamp":"2026-02-24T22:34:30.882351+0000","last_clean_scrub_stamp":"2026-02-24T22:34:30.882351+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":300,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:29:29.669100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":471,"num_read_kb":1193,"num_write":1308,"num_write_kb":244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":621,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.039585+0000","last_change":"2026-02-24T22:34:32.039544+0000","last_active":"2026-02-24T22:34:32.039585+0000","last_peered":"2026-02-24T22:34:32.039585+0000","last_clean":"2026-02-24T22:34:32.039585+0000","last_became_active":"2026-02-24T22:05:08.682913+0000","last_became_peered":"2026-02-24T22:05:08.682913+0000","last_unstale":"2026-02-24T22:34:32.039585+0000","last_undegraded":"2026-02-24T22:34:32.039585+0000","last_fullsized":"2026-02-24T22:34:32.039585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:32.039480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:32.039480+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.039480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:39:23.840470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000359878,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":600,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:51.335400+0000","last_change":"2026-02-24T22:34:51.335400+0000","last_active":"2026-02-24T22:34:51.335400+0000","last_peered":"2026-02-24T22:34:51.335400+0000","last_clean":"2026-02-24T22:34:51.335400+0000","last_became_active":"2026-02-24T22:05:20.192162+0000","last_became_peered":"2026-02-24T22:05:20.192162+0000","last_unstale":"2026-02-24T22:34:51.335400+0000","last_undegraded":"2026-02-24T22:34:51.335400+0000","last_fullsized":"2026-02-24T22:34:51.335400+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:51.335327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:51.335327+0000","last_clean_scrub_stamp":"2026-02-24T22:34:51.335327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:49:38.610778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"312'1040","reported_seq":2612,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:56.289338+0000","last_change":"2026-02-24T22:34:56.289338+0000","last_active":"2026-02-24T22:34:56.289338+0000","last_peered":"2026-02-24T22:34:56.289338+0000","last_clean":"2026-02-24T22:34:56.289338+0000","last_became_active":"2026-02-24T22:05:22.198252+0000","last_became_peered":"2026-02-24T22:05:22.198252+0000","last_unstale":"2026-02-24T22:34:56.289338+0000","last_undegraded":"2026-02-24T22:34:56.289338+0000","last_fullsized":"2026-02-24T22:34:56.289338+0000","mapping_epoch":28,"log_start":"312'1000","ondisk_log_start":"312'1000","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'1040","last_scrub_stamp":"2026-02-24T22:34:56.289305+0000","last_deep_scrub":"312'1040","last_deep_scrub_stamp":"2026-02-24T22:34:56.289305+0000","last_clean_scrub_stamp":"2026-02-24T22:34:56.289305+0000","objects_scrubbed":9,"log_size":40,"log_dups_size":1000,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:35:30.102364+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":614,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":3875,"num_read_kb":1975,"num_write":82,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"312'1796","reported_seq":4427,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:00.610139+0000","last_change":"2026-02-24T22:35:00.610139+0000","last_active":"2026-02-24T22:35:00.610139+0000","last_peered":"2026-02-24T22:35:00.610139+0000","last_clean":"2026-02-24T22:35:00.610139+0000","last_became_active":"2026-02-24T22:05:24.198348+0000","last_became_peered":"2026-02-24T22:05:24.198348+0000","last_unstale":"2026-02-24T22:35:00.610139+0000","last_undegraded":"2026-02-24T22:35:00.610139+0000","last_fullsized":"2026-02-24T22:35:00.610139+0000","mapping_epoch":30,"log_start":"312'1698","ondisk_log_start":"312'1698","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'1796","last_scrub_stamp":"2026-02-24T22:35:00.610099+0000","last_deep_scrub":"312'1796","last_deep_scrub_stamp":"2026-02-24T22:35:00.610099+0000","last_clean_scrub_stamp":"2026-02-24T22:35:00.610099+0000","objects_scrubbed":98,"log_size":98,"log_dups_size":1698,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:58:01.687967+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31448,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":5151,"num_read_kb":231050,"num_write":3475,"num_write_kb":1655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":274185,"num_omap_keys":1500,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":583,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:04.708118+0000","last_change":"2026-02-24T22:35:04.708118+0000","last_active":"2026-02-24T22:35:04.708118+0000","last_peered":"2026-02-24T22:35:04.708118+0000","last_clean":"2026-02-24T22:35:04.708118+0000","last_became_active":"2026-02-24T22:05:29.652321+0000","last_became_peered":"2026-02-24T22:05:29.652321+0000","last_unstale":"2026-02-24T22:35:04.708118+0000","last_undegraded":"2026-02-24T22:35:04.708118+0000","last_fullsized":"2026-02-24T22:35:04.708118+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:04.708046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:04.708046+0000","last_clean_scrub_stamp":"2026-02-24T22:35:04.708046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:35:04.460228+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"312'72","reported_seq":745,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.449851+0000","last_change":"2026-02-24T22:34:36.449851+0000","last_active":"2026-02-24T22:34:36.449851+0000","last_peered":"2026-02-24T22:34:36.449851+0000","last_clean":"2026-02-24T22:34:36.449851+0000","last_became_active":"2026-02-24T22:05:18.191249+0000","last_became_peered":"2026-02-24T22:05:18.191249+0000","last_unstale":"2026-02-24T22:34:36.449851+0000","last_undegraded":"2026-02-24T22:34:36.449851+0000","last_fullsized":"2026-02-24T22:34:36.449851+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'72","last_scrub_stamp":"2026-02-24T22:34:36.449799+0000","last_deep_scrub":"312'72","last_deep_scrub_stamp":"2026-02-24T22:34:36.449799+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.449799+0000","objects_scrubbed":17,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:05:38.920525+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":226,"num_read_kb":230,"num_write":70,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2876,"num_omap_keys":12,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"312'274","reported_seq":1015,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.696388+0000","last_change":"2026-02-24T22:34:32.696342+0000","last_active":"2026-02-24T22:34:32.696388+0000","last_peered":"2026-02-24T22:34:32.696388+0000","last_clean":"2026-02-24T22:34:32.696388+0000","last_became_active":"2026-02-24T22:05:16.181840+0000","last_became_peered":"2026-02-24T22:05:16.181840+0000","last_unstale":"2026-02-24T22:34:32.696388+0000","last_undegraded":"2026-02-24T22:34:32.696388+0000","last_fullsized":"2026-02-24T22:34:32.696388+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'274","last_scrub_stamp":"2026-02-24T22:34:32.696287+0000","last_deep_scrub":"312'274","last_deep_scrub_stamp":"2026-02-24T22:34:32.696287+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.696287+0000","objects_scrubbed":4,"log_size":74,"log_dups_size":200,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:57:17.468566+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":486,"num_read_kb":7386,"num_write":1250,"num_write_kb":13063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":616,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.748692+0000","last_change":"2026-02-24T22:34:31.748652+0000","last_active":"2026-02-24T22:34:31.748692+0000","last_peered":"2026-02-24T22:34:31.748692+0000","last_clean":"2026-02-24T22:34:31.748692+0000","last_became_active":"2026-02-24T22:05:08.689542+0000","last_became_peered":"2026-02-24T22:05:08.689542+0000","last_unstale":"2026-02-24T22:34:31.748692+0000","last_undegraded":"2026-02-24T22:34:31.748692+0000","last_fullsized":"2026-02-24T22:34:31.748692+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:31.748592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:31.748592+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.748592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:30:37.278240+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00038816099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":601,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.813254+0000","last_change":"2026-02-24T22:34:47.813218+0000","last_active":"2026-02-24T22:34:47.813254+0000","last_peered":"2026-02-24T22:34:47.813254+0000","last_clean":"2026-02-24T22:34:47.813254+0000","last_became_active":"2026-02-24T22:05:20.186108+0000","last_became_peered":"2026-02-24T22:05:20.186108+0000","last_unstale":"2026-02-24T22:34:47.813254+0000","last_undegraded":"2026-02-24T22:34:47.813254+0000","last_fullsized":"2026-02-24T22:34:47.813254+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:47.813164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:47.813164+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.813164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:54:49.662332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"312'612","reported_seq":1273,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:51.192081+0000","last_change":"2026-02-24T22:34:51.192081+0000","last_active":"2026-02-24T22:34:51.192081+0000","last_peered":"2026-02-24T22:34:51.192081+0000","last_clean":"2026-02-24T22:34:51.192081+0000","last_became_active":"2026-02-24T22:05:22.911692+0000","last_became_peered":"2026-02-24T22:05:22.911692+0000","last_unstale":"2026-02-24T22:34:51.192081+0000","last_undegraded":"2026-02-24T22:34:51.192081+0000","last_fullsized":"2026-02-24T22:34:51.192081+0000","mapping_epoch":28,"log_start":"312'600","ondisk_log_start":"312'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'612","last_scrub_stamp":"2026-02-24T22:34:51.192010+0000","last_deep_scrub":"312'612","last_deep_scrub_stamp":"2026-02-24T22:34:51.192010+0000","last_clean_scrub_stamp":"2026-02-24T22:34:51.192010+0000","objects_scrubbed":8,"log_size":12,"log_dups_size":600,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:39:44.174934+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115263,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1049,"num_read_kb":2748,"num_write":972,"num_write_kb":882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"312'268","reported_seq":871,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:57.751800+0000","last_change":"2026-02-24T22:34:57.751800+0000","last_active":"2026-02-24T22:34:57.751800+0000","last_peered":"2026-02-24T22:34:57.751800+0000","last_clean":"2026-02-24T22:34:57.751800+0000","last_became_active":"2026-02-24T22:05:24.199398+0000","last_became_peered":"2026-02-24T22:05:24.199398+0000","last_unstale":"2026-02-24T22:34:57.751800+0000","last_undegraded":"2026-02-24T22:34:57.751800+0000","last_fullsized":"2026-02-24T22:34:57.751800+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'268","last_scrub_stamp":"2026-02-24T22:34:57.751746+0000","last_deep_scrub":"312'268","last_deep_scrub_stamp":"2026-02-24T22:34:57.751746+0000","last_clean_scrub_stamp":"2026-02-24T22:34:57.751746+0000","objects_scrubbed":93,"log_size":68,"log_dups_size":200,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:33:56.743054+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30139,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":95,"num_read_kb":93,"num_write":436,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"312'436","reported_seq":3603,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:01.857658+0000","last_change":"2026-02-24T22:35:01.857658+0000","last_active":"2026-02-24T22:35:01.857658+0000","last_peered":"2026-02-24T22:35:01.857658+0000","last_clean":"2026-02-24T22:35:01.857658+0000","last_became_active":"2026-02-24T22:05:29.661694+0000","last_became_peered":"2026-02-24T22:05:29.661694+0000","last_unstale":"2026-02-24T22:35:01.857658+0000","last_undegraded":"2026-02-24T22:35:01.857658+0000","last_fullsized":"2026-02-24T22:35:01.857658+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'436","last_scrub_stamp":"2026-02-24T22:35:01.857603+0000","last_deep_scrub":"312'436","last_deep_scrub_stamp":"2026-02-24T22:35:01.857603+0000","last_clean_scrub_stamp":"2026-02-24T22:35:01.857603+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":400,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:30:24.915658+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"312'462","reported_seq":2605,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:08.756144+0000","last_change":"2026-02-24T22:35:08.756144+0000","last_active":"2026-02-24T22:35:08.756144+0000","last_peered":"2026-02-24T22:35:08.756144+0000","last_clean":"2026-02-24T22:35:08.756144+0000","last_became_active":"2026-02-24T22:05:29.654058+0000","last_became_peered":"2026-02-24T22:05:29.654058+0000","last_unstale":"2026-02-24T22:35:08.756144+0000","last_undegraded":"2026-02-24T22:35:08.756144+0000","last_fullsized":"2026-02-24T22:35:08.756144+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'462","last_scrub_stamp":"2026-02-24T22:35:08.756067+0000","last_deep_scrub":"312'462","last_deep_scrub_stamp":"2026-02-24T22:35:08.756067+0000","last_clean_scrub_stamp":"2026-02-24T22:35:08.756067+0000","objects_scrubbed":1,"log_size":62,"log_dups_size":400,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:13:43.345009+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"312'4219","reported_seq":4961,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.517285+0000","last_change":"2026-02-24T22:34:41.517253+0000","last_active":"2026-02-24T22:34:41.517285+0000","last_peered":"2026-02-24T22:34:41.517285+0000","last_clean":"2026-02-24T22:34:41.517285+0000","last_became_active":"2026-02-24T22:05:18.190114+0000","last_became_peered":"2026-02-24T22:05:18.190114+0000","last_unstale":"2026-02-24T22:34:41.517285+0000","last_undegraded":"2026-02-24T22:34:41.517285+0000","last_fullsized":"2026-02-24T22:34:41.517285+0000","mapping_epoch":24,"log_start":"312'4200","ondisk_log_start":"312'4200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'4219","last_scrub_stamp":"2026-02-24T22:34:41.517218+0000","last_deep_scrub":"312'4219","last_deep_scrub_stamp":"2026-02-24T22:34:41.517218+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.517218+0000","objects_scrubbed":25,"log_size":19,"log_dups_size":3000,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:15:13.238846+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":16758,"num_read_kb":13963,"num_write":6320,"num_write_kb":3135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"312'256","reported_seq":986,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.598871+0000","last_change":"2026-02-24T22:34:31.598822+0000","last_active":"2026-02-24T22:34:31.598871+0000","last_peered":"2026-02-24T22:34:31.598871+0000","last_clean":"2026-02-24T22:34:31.598871+0000","last_became_active":"2026-02-24T22:05:16.188630+0000","last_became_peered":"2026-02-24T22:05:16.188630+0000","last_unstale":"2026-02-24T22:34:31.598871+0000","last_undegraded":"2026-02-24T22:34:31.598871+0000","last_fullsized":"2026-02-24T22:34:31.598871+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'256","last_scrub_stamp":"2026-02-24T22:34:31.598742+0000","last_deep_scrub":"312'256","last_deep_scrub_stamp":"2026-02-24T22:34:31.598742+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.598742+0000","objects_scrubbed":10,"log_size":56,"log_dups_size":200,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:25:41.229561+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291503,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":466,"num_read_kb":33751,"num_write":1067,"num_write_kb":13009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":597,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.198368+0000","last_change":"2026-02-24T22:34:46.198317+0000","last_active":"2026-02-24T22:34:46.198368+0000","last_peered":"2026-02-24T22:34:46.198368+0000","last_clean":"2026-02-24T22:34:46.198368+0000","last_became_active":"2026-02-24T22:05:20.188330+0000","last_became_peered":"2026-02-24T22:05:20.188330+0000","last_unstale":"2026-02-24T22:34:46.198368+0000","last_undegraded":"2026-02-24T22:34:46.198368+0000","last_fullsized":"2026-02-24T22:34:46.198368+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:46.198241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:46.198241+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.198241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:36:44.879872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"312'694","reported_seq":1786,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:55.697289+0000","last_change":"2026-02-24T22:34:55.697246+0000","last_active":"2026-02-24T22:34:55.697289+0000","last_peered":"2026-02-24T22:34:55.697289+0000","last_clean":"2026-02-24T22:34:55.697289+0000","last_became_active":"2026-02-24T22:05:22.191584+0000","last_became_peered":"2026-02-24T22:05:22.191584+0000","last_unstale":"2026-02-24T22:34:55.697289+0000","last_undegraded":"2026-02-24T22:34:55.697289+0000","last_fullsized":"2026-02-24T22:34:55.697289+0000","mapping_epoch":28,"log_start":"312'600","ondisk_log_start":"312'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'694","last_scrub_stamp":"2026-02-24T22:34:55.697186+0000","last_deep_scrub":"312'694","last_deep_scrub_stamp":"2026-02-24T22:34:55.697186+0000","last_clean_scrub_stamp":"2026-02-24T22:34:55.697186+0000","objects_scrubbed":7,"log_size":94,"log_dups_size":600,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:38:19.123444+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39069,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2230,"num_read_kb":1811,"num_write":322,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"312'255","reported_seq":877,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:02.574241+0000","last_change":"2026-02-24T22:35:02.574177+0000","last_active":"2026-02-24T22:35:02.574241+0000","last_peered":"2026-02-24T22:35:02.574241+0000","last_clean":"2026-02-24T22:35:02.574241+0000","last_became_active":"2026-02-24T22:05:24.198821+0000","last_became_peered":"2026-02-24T22:05:24.198821+0000","last_unstale":"2026-02-24T22:35:02.574241+0000","last_undegraded":"2026-02-24T22:35:02.574241+0000","last_fullsized":"2026-02-24T22:35:02.574241+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'255","last_scrub_stamp":"2026-02-24T22:35:02.574107+0000","last_deep_scrub":"312'255","last_deep_scrub_stamp":"2026-02-24T22:35:02.574107+0000","last_clean_scrub_stamp":"2026-02-24T22:35:02.574107+0000","objects_scrubbed":94,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:29:00.459611+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28978,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":162,"num_read_kb":145,"num_write":411,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":587,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:09.567795+0000","last_change":"2026-02-24T22:35:09.567710+0000","last_active":"2026-02-24T22:35:09.567795+0000","last_peered":"2026-02-24T22:35:09.567795+0000","last_clean":"2026-02-24T22:35:09.567795+0000","last_became_active":"2026-02-24T22:05:29.652757+0000","last_became_peered":"2026-02-24T22:05:29.652757+0000","last_unstale":"2026-02-24T22:35:09.567795+0000","last_undegraded":"2026-02-24T22:35:09.567795+0000","last_fullsized":"2026-02-24T22:35:09.567795+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:09.567606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:09.567606+0000","last_clean_scrub_stamp":"2026-02-24T22:35:09.567606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:05:13.901352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"312'4263","reported_seq":5021,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.115970+0000","last_change":"2026-02-24T22:34:40.115933+0000","last_active":"2026-02-24T22:34:40.115970+0000","last_peered":"2026-02-24T22:34:40.115970+0000","last_clean":"2026-02-24T22:34:40.115970+0000","last_became_active":"2026-02-24T22:05:18.183983+0000","last_became_peered":"2026-02-24T22:05:18.183983+0000","last_unstale":"2026-02-24T22:34:40.115970+0000","last_undegraded":"2026-02-24T22:34:40.115970+0000","last_fullsized":"2026-02-24T22:34:40.115970+0000","mapping_epoch":24,"log_start":"312'4200","ondisk_log_start":"312'4200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'4263","last_scrub_stamp":"2026-02-24T22:34:40.115891+0000","last_deep_scrub":"312'4263","last_deep_scrub_stamp":"2026-02-24T22:34:40.115891+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.115891+0000","objects_scrubbed":32,"log_size":63,"log_dups_size":3000,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:06:10.082774+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":16912,"num_read_kb":14102,"num_write":6331,"num_write_kb":3151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"312'314","reported_seq":1064,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.052571+0000","last_change":"2026-02-24T22:34:39.052571+0000","last_active":"2026-02-24T22:34:39.052571+0000","last_peered":"2026-02-24T22:34:39.052571+0000","last_clean":"2026-02-24T22:34:39.052571+0000","last_became_active":"2026-02-24T22:05:16.192845+0000","last_became_peered":"2026-02-24T22:05:16.192845+0000","last_unstale":"2026-02-24T22:34:39.052571+0000","last_undegraded":"2026-02-24T22:34:39.052571+0000","last_fullsized":"2026-02-24T22:34:39.052571+0000","mapping_epoch":22,"log_start":"312'301","ondisk_log_start":"312'301","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'314","last_scrub_stamp":"2026-02-24T22:34:39.052507+0000","last_deep_scrub":"312'314","last_deep_scrub_stamp":"2026-02-24T22:34:39.052507+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.052507+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":301,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:33:51.248695+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":490,"num_read_kb":4336,"num_write":1277,"num_write_kb":2676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":599,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.761316+0000","last_change":"2026-02-24T22:34:49.761265+0000","last_active":"2026-02-24T22:34:49.761316+0000","last_peered":"2026-02-24T22:34:49.761316+0000","last_clean":"2026-02-24T22:34:49.761316+0000","last_became_active":"2026-02-24T22:05:20.188790+0000","last_became_peered":"2026-02-24T22:05:20.188790+0000","last_unstale":"2026-02-24T22:34:49.761316+0000","last_undegraded":"2026-02-24T22:34:49.761316+0000","last_fullsized":"2026-02-24T22:34:49.761316+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:49.761185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:49.761185+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.761185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:28:03.018877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"312'28","reported_seq":627,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:54.950702+0000","last_change":"2026-02-24T22:34:54.950663+0000","last_active":"2026-02-24T22:34:54.950702+0000","last_peered":"2026-02-24T22:34:54.950702+0000","last_clean":"2026-02-24T22:34:54.950702+0000","last_became_active":"2026-02-24T22:05:22.192560+0000","last_became_peered":"2026-02-24T22:05:22.192560+0000","last_unstale":"2026-02-24T22:34:54.950702+0000","last_undegraded":"2026-02-24T22:34:54.950702+0000","last_fullsized":"2026-02-24T22:34:54.950702+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'28","last_scrub_stamp":"2026-02-24T22:34:54.950603+0000","last_deep_scrub":"312'28","last_deep_scrub_stamp":"2026-02-24T22:34:54.950603+0000","last_clean_scrub_stamp":"2026-02-24T22:34:54.950603+0000","objects_scrubbed":3,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:46:55.074264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":294,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"312'260","reported_seq":858,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:59.205059+0000","last_change":"2026-02-24T22:34:59.205020+0000","last_active":"2026-02-24T22:34:59.205059+0000","last_peered":"2026-02-24T22:34:59.205059+0000","last_clean":"2026-02-24T22:34:59.205059+0000","last_became_active":"2026-02-24T22:05:24.205506+0000","last_became_peered":"2026-02-24T22:05:24.205506+0000","last_unstale":"2026-02-24T22:34:59.205059+0000","last_undegraded":"2026-02-24T22:34:59.205059+0000","last_fullsized":"2026-02-24T22:34:59.205059+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'260","last_scrub_stamp":"2026-02-24T22:34:59.204958+0000","last_deep_scrub":"312'260","last_deep_scrub_stamp":"2026-02-24T22:34:59.204958+0000","last_clean_scrub_stamp":"2026-02-24T22:34:59.204958+0000","objects_scrubbed":97,"log_size":60,"log_dups_size":200,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:36:53.558006+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30843,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":90,"num_read_kb":84,"num_write":389,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":582,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:06.764346+0000","last_change":"2026-02-24T22:35:06.764299+0000","last_active":"2026-02-24T22:35:06.764346+0000","last_peered":"2026-02-24T22:35:06.764346+0000","last_clean":"2026-02-24T22:35:06.764346+0000","last_became_active":"2026-02-24T22:05:29.652987+0000","last_became_peered":"2026-02-24T22:05:29.652987+0000","last_unstale":"2026-02-24T22:35:06.764346+0000","last_undegraded":"2026-02-24T22:35:06.764346+0000","last_fullsized":"2026-02-24T22:35:06.764346+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:06.764242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:06.764242+0000","last_clean_scrub_stamp":"2026-02-24T22:35:06.764242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:47:21.058266+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"312'207","reported_seq":914,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.729115+0000","last_change":"2026-02-24T22:34:41.729074+0000","last_active":"2026-02-24T22:34:41.729115+0000","last_peered":"2026-02-24T22:34:41.729115+0000","last_clean":"2026-02-24T22:34:41.729115+0000","last_became_active":"2026-02-24T22:05:18.186069+0000","last_became_peered":"2026-02-24T22:05:18.186069+0000","last_unstale":"2026-02-24T22:34:41.729115+0000","last_undegraded":"2026-02-24T22:34:41.729115+0000","last_fullsized":"2026-02-24T22:34:41.729115+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'207","last_scrub_stamp":"2026-02-24T22:34:41.729005+0000","last_deep_scrub":"312'207","last_deep_scrub_stamp":"2026-02-24T22:34:41.729005+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.729005+0000","objects_scrubbed":20,"log_size":107,"log_dups_size":100,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:04:29.291410+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":691,"num_read_kb":561,"num_write":267,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"312'314","reported_seq":1005,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.108267+0000","last_change":"2026-02-24T22:34:31.108224+0000","last_active":"2026-02-24T22:34:31.108267+0000","last_peered":"2026-02-24T22:34:31.108267+0000","last_clean":"2026-02-24T22:34:31.108267+0000","last_became_active":"2026-02-24T22:06:00.662291+0000","last_became_peered":"2026-02-24T22:06:00.662291+0000","last_unstale":"2026-02-24T22:34:31.108267+0000","last_undegraded":"2026-02-24T22:34:31.108267+0000","last_fullsized":"2026-02-24T22:34:31.108267+0000","mapping_epoch":36,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'314","last_scrub_stamp":"2026-02-24T22:34:31.108159+0000","last_deep_scrub":"312'314","last_deep_scrub_stamp":"2026-02-24T22:34:31.108159+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.108159+0000","objects_scrubbed":5,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:32:30.179715+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":421,"num_read_kb":357,"num_write":1374,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":594,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:53.181963+0000","last_change":"2026-02-24T22:34:53.181869+0000","last_active":"2026-02-24T22:34:53.181963+0000","last_peered":"2026-02-24T22:34:53.181963+0000","last_clean":"2026-02-24T22:34:53.181963+0000","last_became_active":"2026-02-24T22:05:20.194215+0000","last_became_peered":"2026-02-24T22:05:20.194215+0000","last_unstale":"2026-02-24T22:34:53.181963+0000","last_undegraded":"2026-02-24T22:34:53.181963+0000","last_fullsized":"2026-02-24T22:34:53.181963+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:53.181785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:53.181785+0000","last_clean_scrub_stamp":"2026-02-24T22:34:53.181785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:35:22.996790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"312'482","reported_seq":1354,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.661071+0000","last_change":"2026-02-24T22:34:58.661029+0000","last_active":"2026-02-24T22:34:58.661071+0000","last_peered":"2026-02-24T22:34:58.661071+0000","last_clean":"2026-02-24T22:34:58.661071+0000","last_became_active":"2026-02-24T22:05:22.191675+0000","last_became_peered":"2026-02-24T22:05:22.191675+0000","last_unstale":"2026-02-24T22:34:58.661071+0000","last_undegraded":"2026-02-24T22:34:58.661071+0000","last_fullsized":"2026-02-24T22:34:58.661071+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'482","last_scrub_stamp":"2026-02-24T22:34:58.660967+0000","last_deep_scrub":"312'482","last_deep_scrub_stamp":"2026-02-24T22:34:58.660967+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.660967+0000","objects_scrubbed":6,"log_size":82,"log_dups_size":400,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:26:30.259061+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39139,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1329,"num_read_kb":1395,"num_write":345,"num_write_kb":280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"312'347","reported_seq":1391,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.631228+0000","last_change":"2026-02-24T22:34:49.631185+0000","last_active":"2026-02-24T22:34:49.631228+0000","last_peered":"2026-02-24T22:34:49.631228+0000","last_clean":"2026-02-24T22:34:49.631228+0000","last_became_active":"2026-02-24T22:05:24.207479+0000","last_became_peered":"2026-02-24T22:05:24.207479+0000","last_unstale":"2026-02-24T22:34:49.631228+0000","last_undegraded":"2026-02-24T22:34:49.631228+0000","last_fullsized":"2026-02-24T22:34:49.631228+0000","mapping_epoch":30,"log_start":"312'300","ondisk_log_start":"312'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'347","last_scrub_stamp":"2026-02-24T22:34:49.631131+0000","last_deep_scrub":"312'347","last_deep_scrub_stamp":"2026-02-24T22:34:49.631131+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.631131+0000","objects_scrubbed":98,"log_size":47,"log_dups_size":300,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:42:33.404911+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31962,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":675,"num_read_kb":669,"num_write":683,"num_write_kb":207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":573,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:05.770303+0000","last_change":"2026-02-24T22:35:05.770253+0000","last_active":"2026-02-24T22:35:05.770303+0000","last_peered":"2026-02-24T22:35:05.770303+0000","last_clean":"2026-02-24T22:35:05.770303+0000","last_became_active":"2026-02-24T22:05:29.661248+0000","last_became_peered":"2026-02-24T22:05:29.661248+0000","last_unstale":"2026-02-24T22:35:05.770303+0000","last_undegraded":"2026-02-24T22:35:05.770303+0000","last_fullsized":"2026-02-24T22:35:05.770303+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:05.770187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:05.770187+0000","last_clean_scrub_stamp":"2026-02-24T22:35:05.770187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:50:08.978017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"312'142","reported_seq":860,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.756553+0000","last_change":"2026-02-24T22:34:39.756553+0000","last_active":"2026-02-24T22:34:39.756553+0000","last_peered":"2026-02-24T22:34:39.756553+0000","last_clean":"2026-02-24T22:34:39.756553+0000","last_became_active":"2026-02-24T22:05:18.184982+0000","last_became_peered":"2026-02-24T22:05:18.184982+0000","last_unstale":"2026-02-24T22:34:39.756553+0000","last_undegraded":"2026-02-24T22:34:39.756553+0000","last_fullsized":"2026-02-24T22:34:39.756553+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'142","last_scrub_stamp":"2026-02-24T22:34:39.756507+0000","last_deep_scrub":"312'142","last_deep_scrub_stamp":"2026-02-24T22:34:39.756507+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.756507+0000","objects_scrubbed":26,"log_size":42,"log_dups_size":100,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:57:48.133450+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":444,"num_read_kb":381,"num_write":158,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"312'284","reported_seq":1054,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.721473+0000","last_change":"2026-02-24T22:34:33.721473+0000","last_active":"2026-02-24T22:34:33.721473+0000","last_peered":"2026-02-24T22:34:33.721473+0000","last_clean":"2026-02-24T22:34:33.721473+0000","last_became_active":"2026-02-24T22:05:16.184594+0000","last_became_peered":"2026-02-24T22:05:16.184594+0000","last_unstale":"2026-02-24T22:34:33.721473+0000","last_undegraded":"2026-02-24T22:34:33.721473+0000","last_fullsized":"2026-02-24T22:34:33.721473+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'284","last_scrub_stamp":"2026-02-24T22:34:33.721410+0000","last_deep_scrub":"312'284","last_deep_scrub_stamp":"2026-02-24T22:34:33.721410+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.721410+0000","objects_scrubbed":5,"log_size":84,"log_dups_size":200,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:42:30.853887+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048643,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":519,"num_read_kb":28116,"num_write":1194,"num_write_kb":6289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":592,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.669882+0000","last_change":"2026-02-24T22:34:44.669842+0000","last_active":"2026-02-24T22:34:44.669882+0000","last_peered":"2026-02-24T22:34:44.669882+0000","last_clean":"2026-02-24T22:34:44.669882+0000","last_became_active":"2026-02-24T22:05:20.193522+0000","last_became_peered":"2026-02-24T22:05:20.193522+0000","last_unstale":"2026-02-24T22:34:44.669882+0000","last_undegraded":"2026-02-24T22:34:44.669882+0000","last_fullsized":"2026-02-24T22:34:44.669882+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:44.669782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:44.669782+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.669782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:43:09.538394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"312'438","reported_seq":1286,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:55.258534+0000","last_change":"2026-02-24T22:34:55.258468+0000","last_active":"2026-02-24T22:34:55.258534+0000","last_peered":"2026-02-24T22:34:55.258534+0000","last_clean":"2026-02-24T22:34:55.258534+0000","last_became_active":"2026-02-24T22:05:22.199968+0000","last_became_peered":"2026-02-24T22:05:22.199968+0000","last_unstale":"2026-02-24T22:34:55.258534+0000","last_undegraded":"2026-02-24T22:34:55.258534+0000","last_fullsized":"2026-02-24T22:34:55.258534+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'438","last_scrub_stamp":"2026-02-24T22:34:55.258389+0000","last_deep_scrub":"312'438","last_deep_scrub_stamp":"2026-02-24T22:34:55.258389+0000","last_clean_scrub_stamp":"2026-02-24T22:34:55.258389+0000","objects_scrubbed":5,"log_size":38,"log_dups_size":400,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:04:07.943168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38886,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1275,"num_read_kb":1335,"num_write":311,"num_write_kb":280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"312'303","reported_seq":914,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:00.858814+0000","last_change":"2026-02-24T22:35:00.858756+0000","last_active":"2026-02-24T22:35:00.858814+0000","last_peered":"2026-02-24T22:35:00.858814+0000","last_clean":"2026-02-24T22:35:00.858814+0000","last_became_active":"2026-02-24T22:05:24.199650+0000","last_became_peered":"2026-02-24T22:05:24.199650+0000","last_unstale":"2026-02-24T22:35:00.858814+0000","last_undegraded":"2026-02-24T22:35:00.858814+0000","last_fullsized":"2026-02-24T22:35:00.858814+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'303","last_scrub_stamp":"2026-02-24T22:35:00.858703+0000","last_deep_scrub":"312'303","last_deep_scrub_stamp":"2026-02-24T22:35:00.858703+0000","last_clean_scrub_stamp":"2026-02-24T22:35:00.858703+0000","objects_scrubbed":91,"log_size":103,"log_dups_size":200,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:01:19.132884+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29523,"num_objects":91,"num_object_clones":0,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":182,"num_read_kb":193,"num_write":488,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"312'270","reported_seq":989,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.736339+0000","last_change":"2026-02-24T22:34:41.736339+0000","last_active":"2026-02-24T22:34:41.736339+0000","last_peered":"2026-02-24T22:34:41.736339+0000","last_clean":"2026-02-24T22:34:41.736339+0000","last_became_active":"2026-02-24T22:05:18.190238+0000","last_became_peered":"2026-02-24T22:05:18.190238+0000","last_unstale":"2026-02-24T22:34:41.736339+0000","last_undegraded":"2026-02-24T22:34:41.736339+0000","last_fullsized":"2026-02-24T22:34:41.736339+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'270","last_scrub_stamp":"2026-02-24T22:34:41.736291+0000","last_deep_scrub":"312'270","last_deep_scrub_stamp":"2026-02-24T22:34:41.736291+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.736291+0000","objects_scrubbed":21,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:33:02.999931+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":976,"num_read_kb":794,"num_write":353,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":219,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"312'310","reported_seq":1674,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.902084+0000","last_change":"2026-02-24T22:34:33.902084+0000","last_active":"2026-02-24T22:34:33.902084+0000","last_peered":"2026-02-24T22:34:33.902084+0000","last_clean":"2026-02-24T22:34:33.902084+0000","last_became_active":"2026-02-24T22:05:16.184415+0000","last_became_peered":"2026-02-24T22:05:16.184415+0000","last_unstale":"2026-02-24T22:34:33.902084+0000","last_undegraded":"2026-02-24T22:34:33.902084+0000","last_fullsized":"2026-02-24T22:34:33.902084+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'310","last_scrub_stamp":"2026-02-24T22:34:33.902021+0000","last_deep_scrub":"312'310","last_deep_scrub_stamp":"2026-02-24T22:34:33.902021+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.902021+0000","objects_scrubbed":3,"log_size":110,"log_dups_size":200,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:11:44.512299+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2362,"num_read_kb":23834,"num_write":1125,"num_write_kb":5557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":599,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:50.938248+0000","last_change":"2026-02-24T22:34:50.938212+0000","last_active":"2026-02-24T22:34:50.938248+0000","last_peered":"2026-02-24T22:34:50.938248+0000","last_clean":"2026-02-24T22:34:50.938248+0000","last_became_active":"2026-02-24T22:05:20.186325+0000","last_became_peered":"2026-02-24T22:05:20.186325+0000","last_unstale":"2026-02-24T22:34:50.938248+0000","last_undegraded":"2026-02-24T22:34:50.938248+0000","last_fullsized":"2026-02-24T22:34:50.938248+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:50.938156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:50.938156+0000","last_clean_scrub_stamp":"2026-02-24T22:34:50.938156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:12:47.996800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"312'413","reported_seq":1040,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:53.238960+0000","last_change":"2026-02-24T22:34:53.238905+0000","last_active":"2026-02-24T22:34:53.238960+0000","last_peered":"2026-02-24T22:34:53.238960+0000","last_clean":"2026-02-24T22:34:53.238960+0000","last_became_active":"2026-02-24T22:05:22.192756+0000","last_became_peered":"2026-02-24T22:05:22.192756+0000","last_unstale":"2026-02-24T22:34:53.238960+0000","last_undegraded":"2026-02-24T22:34:53.238960+0000","last_fullsized":"2026-02-24T22:34:53.238960+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'413","last_scrub_stamp":"2026-02-24T22:34:53.238844+0000","last_deep_scrub":"312'413","last_deep_scrub_stamp":"2026-02-24T22:34:53.238844+0000","last_clean_scrub_stamp":"2026-02-24T22:34:53.238844+0000","objects_scrubbed":6,"log_size":13,"log_dups_size":400,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:43:03.519649+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77383,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":715,"num_read_kb":1913,"num_write":704,"num_write_kb":692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24470,"num_omap_keys":44,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"312'286","reported_seq":910,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:59.933962+0000","last_change":"2026-02-24T22:34:59.933925+0000","last_active":"2026-02-24T22:34:59.933962+0000","last_peered":"2026-02-24T22:34:59.933962+0000","last_clean":"2026-02-24T22:34:59.933962+0000","last_became_active":"2026-02-24T22:05:24.206407+0000","last_became_peered":"2026-02-24T22:05:24.206407+0000","last_unstale":"2026-02-24T22:34:59.933962+0000","last_undegraded":"2026-02-24T22:34:59.933962+0000","last_fullsized":"2026-02-24T22:34:59.933962+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'286","last_scrub_stamp":"2026-02-24T22:34:59.933869+0000","last_deep_scrub":"312'286","last_deep_scrub_stamp":"2026-02-24T22:34:59.933869+0000","last_clean_scrub_stamp":"2026-02-24T22:34:59.933869+0000","objects_scrubbed":99,"log_size":86,"log_dups_size":200,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:02:47.770768+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30298,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":168,"num_read_kb":155,"num_write":455,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":585,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:08.590971+0000","last_change":"2026-02-24T22:35:08.590933+0000","last_active":"2026-02-24T22:35:08.590971+0000","last_peered":"2026-02-24T22:35:08.590971+0000","last_clean":"2026-02-24T22:35:08.590971+0000","last_became_active":"2026-02-24T22:05:29.650115+0000","last_became_peered":"2026-02-24T22:05:29.650115+0000","last_unstale":"2026-02-24T22:35:08.590971+0000","last_undegraded":"2026-02-24T22:35:08.590971+0000","last_fullsized":"2026-02-24T22:35:08.590971+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:08.590877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:08.590877+0000","last_clean_scrub_stamp":"2026-02-24T22:35:08.590877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:49:57.541981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"312'193","reported_seq":915,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.857368+0000","last_change":"2026-02-24T22:34:43.857329+0000","last_active":"2026-02-24T22:34:43.857368+0000","last_peered":"2026-02-24T22:34:43.857368+0000","last_clean":"2026-02-24T22:34:43.857368+0000","last_became_active":"2026-02-24T22:05:18.183501+0000","last_became_peered":"2026-02-24T22:05:18.183501+0000","last_unstale":"2026-02-24T22:34:43.857368+0000","last_undegraded":"2026-02-24T22:34:43.857368+0000","last_fullsized":"2026-02-24T22:34:43.857368+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'193","last_scrub_stamp":"2026-02-24T22:34:43.857270+0000","last_deep_scrub":"312'193","last_deep_scrub_stamp":"2026-02-24T22:34:43.857270+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.857270+0000","objects_scrubbed":31,"log_size":93,"log_dups_size":100,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:49:05.843210+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":578,"num_read_kb":488,"num_write":224,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":465,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"312'412","reported_seq":1362,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.006923+0000","last_change":"2026-02-24T22:34:33.006923+0000","last_active":"2026-02-24T22:34:33.006923+0000","last_peered":"2026-02-24T22:34:33.006923+0000","last_clean":"2026-02-24T22:34:33.006923+0000","last_became_active":"2026-02-24T22:05:16.182909+0000","last_became_peered":"2026-02-24T22:05:16.182909+0000","last_unstale":"2026-02-24T22:34:33.006923+0000","last_undegraded":"2026-02-24T22:34:33.006923+0000","last_fullsized":"2026-02-24T22:34:33.006923+0000","mapping_epoch":22,"log_start":"312'400","ondisk_log_start":"312'400","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'412","last_scrub_stamp":"2026-02-24T22:34:33.006865+0000","last_deep_scrub":"312'412","last_deep_scrub_stamp":"2026-02-24T22:34:33.006865+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.006865+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":400,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:36:32.796788+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1207,"num_read_kb":9244,"num_write":1611,"num_write_kb":11784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":593,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.622797+0000","last_change":"2026-02-24T22:34:45.622797+0000","last_active":"2026-02-24T22:34:45.622797+0000","last_peered":"2026-02-24T22:34:45.622797+0000","last_clean":"2026-02-24T22:34:45.622797+0000","last_became_active":"2026-02-24T22:05:20.193751+0000","last_became_peered":"2026-02-24T22:05:20.193751+0000","last_unstale":"2026-02-24T22:34:45.622797+0000","last_undegraded":"2026-02-24T22:34:45.622797+0000","last_fullsized":"2026-02-24T22:34:45.622797+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:45.622745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:45.622745+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.622745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:01:44.701244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"312'754","reported_seq":1846,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:56.215537+0000","last_change":"2026-02-24T22:34:56.215477+0000","last_active":"2026-02-24T22:34:56.215537+0000","last_peered":"2026-02-24T22:34:56.215537+0000","last_clean":"2026-02-24T22:34:56.215537+0000","last_became_active":"2026-02-24T22:05:22.200603+0000","last_became_peered":"2026-02-24T22:05:22.200603+0000","last_unstale":"2026-02-24T22:34:56.215537+0000","last_undegraded":"2026-02-24T22:34:56.215537+0000","last_fullsized":"2026-02-24T22:34:56.215537+0000","mapping_epoch":28,"log_start":"312'700","ondisk_log_start":"312'700","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'754","last_scrub_stamp":"2026-02-24T22:34:56.215404+0000","last_deep_scrub":"312'754","last_deep_scrub_stamp":"2026-02-24T22:34:56.215404+0000","last_clean_scrub_stamp":"2026-02-24T22:34:56.215404+0000","objects_scrubbed":10,"log_size":54,"log_dups_size":700,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:58:38.795319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39896,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":2333,"num_read_kb":2057,"num_write":434,"num_write_kb":373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11460,"num_omap_keys":24,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"312'433","reported_seq":1078,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:55.324437+0000","last_change":"2026-02-24T22:34:55.324399+0000","last_active":"2026-02-24T22:34:55.324437+0000","last_peered":"2026-02-24T22:34:55.324437+0000","last_clean":"2026-02-24T22:34:55.324437+0000","last_became_active":"2026-02-24T22:05:24.200391+0000","last_became_peered":"2026-02-24T22:05:24.200391+0000","last_unstale":"2026-02-24T22:34:55.324437+0000","last_undegraded":"2026-02-24T22:34:55.324437+0000","last_fullsized":"2026-02-24T22:34:55.324437+0000","mapping_epoch":30,"log_start":"312'400","ondisk_log_start":"312'400","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'433","last_scrub_stamp":"2026-02-24T22:34:55.324342+0000","last_deep_scrub":"312'433","last_deep_scrub_stamp":"2026-02-24T22:34:55.324342+0000","last_clean_scrub_stamp":"2026-02-24T22:34:55.324342+0000","objects_scrubbed":99,"log_size":33,"log_dups_size":400,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:09:49.237235+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31847,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":370,"num_read_kb":353,"num_write":777,"num_write_kb":271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"312'82","reported_seq":1004,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:53.578469+0000","last_change":"2026-02-24T22:34:53.578429+0000","last_active":"2026-02-24T22:34:53.578469+0000","last_peered":"2026-02-24T22:34:53.578469+0000","last_clean":"2026-02-24T22:34:53.578469+0000","last_became_active":"2026-02-24T22:05:29.660289+0000","last_became_peered":"2026-02-24T22:05:29.660289+0000","last_unstale":"2026-02-24T22:34:53.578469+0000","last_undegraded":"2026-02-24T22:34:53.578469+0000","last_fullsized":"2026-02-24T22:34:53.578469+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'82","last_scrub_stamp":"2026-02-24T22:34:53.578366+0000","last_deep_scrub":"312'82","last_deep_scrub_stamp":"2026-02-24T22:34:53.578366+0000","last_clean_scrub_stamp":"2026-02-24T22:34:53.578366+0000","objects_scrubbed":1,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:20:48.789151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"312'120","reported_seq":799,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.773740+0000","last_change":"2026-02-24T22:34:43.773705+0000","last_active":"2026-02-24T22:34:43.773740+0000","last_peered":"2026-02-24T22:34:43.773740+0000","last_clean":"2026-02-24T22:34:43.773740+0000","last_became_active":"2026-02-24T22:05:18.184745+0000","last_became_peered":"2026-02-24T22:05:18.184745+0000","last_unstale":"2026-02-24T22:34:43.773740+0000","last_undegraded":"2026-02-24T22:34:43.773740+0000","last_fullsized":"2026-02-24T22:34:43.773740+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'120","last_scrub_stamp":"2026-02-24T22:34:43.773655+0000","last_deep_scrub":"312'120","last_deep_scrub_stamp":"2026-02-24T22:34:43.773655+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.773655+0000","objects_scrubbed":20,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:13:15.141522+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":371,"num_read_kb":307,"num_write":132,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":684,"num_omap_keys":3,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"312'307","reported_seq":1036,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.531212+0000","last_change":"2026-02-24T22:34:33.531150+0000","last_active":"2026-02-24T22:34:33.531212+0000","last_peered":"2026-02-24T22:34:33.531212+0000","last_clean":"2026-02-24T22:34:33.531212+0000","last_became_active":"2026-02-24T22:05:16.185574+0000","last_became_peered":"2026-02-24T22:05:16.185574+0000","last_unstale":"2026-02-24T22:34:33.531212+0000","last_undegraded":"2026-02-24T22:34:33.531212+0000","last_fullsized":"2026-02-24T22:34:33.531212+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'307","last_scrub_stamp":"2026-02-24T22:34:33.531066+0000","last_deep_scrub":"312'307","last_deep_scrub_stamp":"2026-02-24T22:34:33.531066+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.531066+0000","objects_scrubbed":4,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:08:13.354888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":428,"num_read_kb":30007,"num_write":1257,"num_write_kb":21358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"312'4","reported_seq":841,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:54.234549+0000","last_change":"2026-02-24T22:34:54.234514+0000","last_active":"2026-02-24T22:34:54.234549+0000","last_peered":"2026-02-24T22:34:54.234549+0000","last_clean":"2026-02-24T22:34:54.234549+0000","last_became_active":"2026-02-24T22:05:20.192612+0000","last_became_peered":"2026-02-24T22:05:20.192612+0000","last_unstale":"2026-02-24T22:34:54.234549+0000","last_undegraded":"2026-02-24T22:34:54.234549+0000","last_fullsized":"2026-02-24T22:34:54.234549+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'4","last_scrub_stamp":"2026-02-24T22:34:54.234459+0000","last_deep_scrub":"312'4","last_deep_scrub_stamp":"2026-02-24T22:34:54.234459+0000","last_clean_scrub_stamp":"2026-02-24T22:34:54.234459+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:52:30.469777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":484,"num_read_kb":482,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"312'209","reported_seq":830,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:56.698535+0000","last_change":"2026-02-24T22:34:56.698535+0000","last_active":"2026-02-24T22:34:56.698535+0000","last_peered":"2026-02-24T22:34:56.698535+0000","last_clean":"2026-02-24T22:34:56.698535+0000","last_became_active":"2026-02-24T22:05:22.906047+0000","last_became_peered":"2026-02-24T22:05:22.906047+0000","last_unstale":"2026-02-24T22:34:56.698535+0000","last_undegraded":"2026-02-24T22:34:56.698535+0000","last_fullsized":"2026-02-24T22:34:56.698535+0000","mapping_epoch":28,"log_start":"312'100","ondisk_log_start":"312'100","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'209","last_scrub_stamp":"2026-02-24T22:34:56.698473+0000","last_deep_scrub":"312'209","last_deep_scrub_stamp":"2026-02-24T22:34:56.698473+0000","last_clean_scrub_stamp":"2026-02-24T22:34:56.698473+0000","objects_scrubbed":5,"log_size":109,"log_dups_size":100,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:47:56.787199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38786,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":328,"num_read_kb":839,"num_write":300,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"312'257","reported_seq":867,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:54.273436+0000","last_change":"2026-02-24T22:34:54.273407+0000","last_active":"2026-02-24T22:34:54.273436+0000","last_peered":"2026-02-24T22:34:54.273436+0000","last_clean":"2026-02-24T22:34:54.273436+0000","last_became_active":"2026-02-24T22:05:24.200747+0000","last_became_peered":"2026-02-24T22:05:24.200747+0000","last_unstale":"2026-02-24T22:34:54.273436+0000","last_undegraded":"2026-02-24T22:34:54.273436+0000","last_fullsized":"2026-02-24T22:34:54.273436+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'257","last_scrub_stamp":"2026-02-24T22:34:54.273373+0000","last_deep_scrub":"312'257","last_deep_scrub_stamp":"2026-02-24T22:34:54.273373+0000","last_clean_scrub_stamp":"2026-02-24T22:34:54.273373+0000","objects_scrubbed":93,"log_size":57,"log_dups_size":200,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:40:30.932635+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30873,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":120,"num_read_kb":110,"num_write":405,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":574,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:03.846698+0000","last_change":"2026-02-24T22:35:03.846658+0000","last_active":"2026-02-24T22:35:03.846698+0000","last_peered":"2026-02-24T22:35:03.846698+0000","last_clean":"2026-02-24T22:35:03.846698+0000","last_became_active":"2026-02-24T22:05:29.660344+0000","last_became_peered":"2026-02-24T22:05:29.660344+0000","last_unstale":"2026-02-24T22:35:03.846698+0000","last_undegraded":"2026-02-24T22:35:03.846698+0000","last_fullsized":"2026-02-24T22:35:03.846698+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:03.846600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:03.846600+0000","last_clean_scrub_stamp":"2026-02-24T22:35:03.846600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:10:57.615627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"312'176","reported_seq":857,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.075488+0000","last_change":"2026-02-24T22:34:36.075455+0000","last_active":"2026-02-24T22:34:36.075488+0000","last_peered":"2026-02-24T22:34:36.075488+0000","last_clean":"2026-02-24T22:34:36.075488+0000","last_became_active":"2026-02-24T22:06:00.759215+0000","last_became_peered":"2026-02-24T22:06:00.759215+0000","last_unstale":"2026-02-24T22:34:36.075488+0000","last_undegraded":"2026-02-24T22:34:36.075488+0000","last_fullsized":"2026-02-24T22:34:36.075488+0000","mapping_epoch":36,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'176","last_scrub_stamp":"2026-02-24T22:34:36.075425+0000","last_deep_scrub":"312'176","last_deep_scrub_stamp":"2026-02-24T22:34:36.075425+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.075425+0000","objects_scrubbed":19,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:59:34.861298+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":584,"num_read_kb":483,"num_write":222,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"312'269","reported_seq":1029,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:31.707518+0000","last_change":"2026-02-24T22:34:31.707474+0000","last_active":"2026-02-24T22:34:31.707518+0000","last_peered":"2026-02-24T22:34:31.707518+0000","last_clean":"2026-02-24T22:34:31.707518+0000","last_became_active":"2026-02-24T22:05:16.183956+0000","last_became_peered":"2026-02-24T22:05:16.183956+0000","last_unstale":"2026-02-24T22:34:31.707518+0000","last_undegraded":"2026-02-24T22:34:31.707518+0000","last_fullsized":"2026-02-24T22:34:31.707518+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'269","last_scrub_stamp":"2026-02-24T22:34:31.707414+0000","last_deep_scrub":"312'269","last_deep_scrub_stamp":"2026-02-24T22:34:31.707414+0000","last_clean_scrub_stamp":"2026-02-24T22:34:31.707414+0000","objects_scrubbed":6,"log_size":69,"log_dups_size":200,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:16:18.564686+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242893,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":575,"num_read_kb":6430,"num_write":1267,"num_write_kb":5371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":596,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:49.203295+0000","last_change":"2026-02-24T22:34:49.203221+0000","last_active":"2026-02-24T22:34:49.203295+0000","last_peered":"2026-02-24T22:34:49.203295+0000","last_clean":"2026-02-24T22:34:49.203295+0000","last_became_active":"2026-02-24T22:05:20.187316+0000","last_became_peered":"2026-02-24T22:05:20.187316+0000","last_unstale":"2026-02-24T22:34:49.203295+0000","last_undegraded":"2026-02-24T22:34:49.203295+0000","last_fullsized":"2026-02-24T22:34:49.203295+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:49.203134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:49.203134+0000","last_clean_scrub_stamp":"2026-02-24T22:34:49.203134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:31:08.087721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"312'295","reported_seq":1143,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.297709+0000","last_change":"2026-02-24T22:34:58.297675+0000","last_active":"2026-02-24T22:34:58.297709+0000","last_peered":"2026-02-24T22:34:58.297709+0000","last_clean":"2026-02-24T22:34:58.297709+0000","last_became_active":"2026-02-24T22:05:22.912087+0000","last_became_peered":"2026-02-24T22:05:22.912087+0000","last_unstale":"2026-02-24T22:34:58.297709+0000","last_undegraded":"2026-02-24T22:34:58.297709+0000","last_fullsized":"2026-02-24T22:34:58.297709+0000","mapping_epoch":28,"log_start":"307'200","ondisk_log_start":"307'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'295","last_scrub_stamp":"2026-02-24T22:34:58.297632+0000","last_deep_scrub":"312'295","last_deep_scrub_stamp":"2026-02-24T22:34:58.297632+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.297632+0000","objects_scrubbed":7,"log_size":95,"log_dups_size":200,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:34:14.546778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":152,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1000,"num_read_kb":520,"num_write":51,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1780,"num_omap_keys":6,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"312'272","reported_seq":953,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:50.604472+0000","last_change":"2026-02-24T22:34:50.604472+0000","last_active":"2026-02-24T22:34:50.604472+0000","last_peered":"2026-02-24T22:34:50.604472+0000","last_clean":"2026-02-24T22:34:50.604472+0000","last_became_active":"2026-02-24T22:05:24.206383+0000","last_became_peered":"2026-02-24T22:05:24.206383+0000","last_unstale":"2026-02-24T22:34:50.604472+0000","last_undegraded":"2026-02-24T22:34:50.604472+0000","last_fullsized":"2026-02-24T22:34:50.604472+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'272","last_scrub_stamp":"2026-02-24T22:34:50.604410+0000","last_deep_scrub":"312'272","last_deep_scrub_stamp":"2026-02-24T22:34:50.604410+0000","last_clean_scrub_stamp":"2026-02-24T22:34:50.604410+0000","objects_scrubbed":94,"log_size":72,"log_dups_size":200,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:58:49.327922+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29475,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":255,"num_read_kb":237,"num_write":434,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":580,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:03.209435+0000","last_change":"2026-02-24T22:35:03.209395+0000","last_active":"2026-02-24T22:35:03.209435+0000","last_peered":"2026-02-24T22:35:03.209435+0000","last_clean":"2026-02-24T22:35:03.209435+0000","last_became_active":"2026-02-24T22:05:29.661539+0000","last_became_peered":"2026-02-24T22:05:29.661539+0000","last_unstale":"2026-02-24T22:35:03.209435+0000","last_undegraded":"2026-02-24T22:35:03.209435+0000","last_fullsized":"2026-02-24T22:35:03.209435+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:03.209335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:03.209335+0000","last_clean_scrub_stamp":"2026-02-24T22:35:03.209335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:44:51.735842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"312'201","reported_seq":882,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.483789+0000","last_change":"2026-02-24T22:34:44.483754+0000","last_active":"2026-02-24T22:34:44.483789+0000","last_peered":"2026-02-24T22:34:44.483789+0000","last_clean":"2026-02-24T22:34:44.483789+0000","last_became_active":"2026-02-24T22:05:18.192487+0000","last_became_peered":"2026-02-24T22:05:18.192487+0000","last_unstale":"2026-02-24T22:34:44.483789+0000","last_undegraded":"2026-02-24T22:34:44.483789+0000","last_fullsized":"2026-02-24T22:34:44.483789+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'201","last_scrub_stamp":"2026-02-24T22:34:44.483706+0000","last_deep_scrub":"312'201","last_deep_scrub_stamp":"2026-02-24T22:34:44.483706+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.483706+0000","objects_scrubbed":23,"log_size":101,"log_dups_size":100,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:31:57.971063+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":681,"num_read_kb":551,"num_write":266,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"312'299","reported_seq":1007,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.128172+0000","last_change":"2026-02-24T22:34:36.128130+0000","last_active":"2026-02-24T22:34:36.128172+0000","last_peered":"2026-02-24T22:34:36.128172+0000","last_clean":"2026-02-24T22:34:36.128172+0000","last_became_active":"2026-02-24T22:05:16.190108+0000","last_became_peered":"2026-02-24T22:05:16.190108+0000","last_unstale":"2026-02-24T22:34:36.128172+0000","last_undegraded":"2026-02-24T22:34:36.128172+0000","last_fullsized":"2026-02-24T22:34:36.128172+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'299","last_scrub_stamp":"2026-02-24T22:34:36.128072+0000","last_deep_scrub":"312'299","last_deep_scrub_stamp":"2026-02-24T22:34:36.128072+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.128072+0000","objects_scrubbed":3,"log_size":99,"log_dups_size":200,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:52:02.916678+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242914,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":451,"num_read_kb":15743,"num_write":1326,"num_write_kb":10328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":593,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:52.177876+0000","last_change":"2026-02-24T22:34:52.177761+0000","last_active":"2026-02-24T22:34:52.177876+0000","last_peered":"2026-02-24T22:34:52.177876+0000","last_clean":"2026-02-24T22:34:52.177876+0000","last_became_active":"2026-02-24T22:05:20.194183+0000","last_became_peered":"2026-02-24T22:05:20.194183+0000","last_unstale":"2026-02-24T22:34:52.177876+0000","last_undegraded":"2026-02-24T22:34:52.177876+0000","last_fullsized":"2026-02-24T22:34:52.177876+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:52.177667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:52.177667+0000","last_clean_scrub_stamp":"2026-02-24T22:34:52.177667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:07:28.331498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"312'722","reported_seq":1808,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:58.944289+0000","last_change":"2026-02-24T22:34:58.944289+0000","last_active":"2026-02-24T22:34:58.944289+0000","last_peered":"2026-02-24T22:34:58.944289+0000","last_clean":"2026-02-24T22:34:58.944289+0000","last_became_active":"2026-02-24T22:05:22.199480+0000","last_became_peered":"2026-02-24T22:05:22.199480+0000","last_unstale":"2026-02-24T22:34:58.944289+0000","last_undegraded":"2026-02-24T22:34:58.944289+0000","last_fullsized":"2026-02-24T22:34:58.944289+0000","mapping_epoch":28,"log_start":"312'700","ondisk_log_start":"312'700","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'722","last_scrub_stamp":"2026-02-24T22:34:58.944231+0000","last_deep_scrub":"312'722","last_deep_scrub_stamp":"2026-02-24T22:34:58.944231+0000","last_clean_scrub_stamp":"2026-02-24T22:34:58.944231+0000","objects_scrubbed":9,"log_size":22,"log_dups_size":700,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:10:48.727164+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39366,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":2266,"num_read_kb":1888,"num_write":352,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"312'285","reported_seq":904,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:01.828096+0000","last_change":"2026-02-24T22:35:01.828096+0000","last_active":"2026-02-24T22:35:01.828096+0000","last_peered":"2026-02-24T22:35:01.828096+0000","last_clean":"2026-02-24T22:35:01.828096+0000","last_became_active":"2026-02-24T22:05:24.200434+0000","last_became_peered":"2026-02-24T22:05:24.200434+0000","last_unstale":"2026-02-24T22:35:01.828096+0000","last_undegraded":"2026-02-24T22:35:01.828096+0000","last_fullsized":"2026-02-24T22:35:01.828096+0000","mapping_epoch":30,"log_start":"312'198","ondisk_log_start":"312'198","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'285","last_scrub_stamp":"2026-02-24T22:35:01.828052+0000","last_deep_scrub":"312'285","last_deep_scrub_stamp":"2026-02-24T22:35:01.828052+0000","last_clean_scrub_stamp":"2026-02-24T22:35:01.828052+0000","objects_scrubbed":87,"log_size":87,"log_dups_size":198,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:18:55.623149+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27134,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":189,"num_read_kb":178,"num_write":476,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":579,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:06.277822+0000","last_change":"2026-02-24T22:35:06.277822+0000","last_active":"2026-02-24T22:35:06.277822+0000","last_peered":"2026-02-24T22:35:06.277822+0000","last_clean":"2026-02-24T22:35:06.277822+0000","last_became_active":"2026-02-24T22:05:29.662425+0000","last_became_peered":"2026-02-24T22:05:29.662425+0000","last_unstale":"2026-02-24T22:35:06.277822+0000","last_undegraded":"2026-02-24T22:35:06.277822+0000","last_fullsized":"2026-02-24T22:35:06.277822+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:06.277768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:06.277768+0000","last_clean_scrub_stamp":"2026-02-24T22:35:06.277768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:06:45.271197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"312'157","reported_seq":845,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:42.518898+0000","last_change":"2026-02-24T22:34:42.518852+0000","last_active":"2026-02-24T22:34:42.518898+0000","last_peered":"2026-02-24T22:34:42.518898+0000","last_clean":"2026-02-24T22:34:42.518898+0000","last_became_active":"2026-02-24T22:05:18.191664+0000","last_became_peered":"2026-02-24T22:05:18.191664+0000","last_unstale":"2026-02-24T22:34:42.518898+0000","last_undegraded":"2026-02-24T22:34:42.518898+0000","last_fullsized":"2026-02-24T22:34:42.518898+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'157","last_scrub_stamp":"2026-02-24T22:34:42.518790+0000","last_deep_scrub":"312'157","last_deep_scrub_stamp":"2026-02-24T22:34:42.518790+0000","last_clean_scrub_stamp":"2026-02-24T22:34:42.518790+0000","objects_scrubbed":25,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:34:13.833329+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":463,"num_read_kb":379,"num_write":198,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"312'278","reported_seq":982,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.114301+0000","last_change":"2026-02-24T22:34:35.114265+0000","last_active":"2026-02-24T22:34:35.114301+0000","last_peered":"2026-02-24T22:34:35.114301+0000","last_clean":"2026-02-24T22:34:35.114301+0000","last_became_active":"2026-02-24T22:05:16.188154+0000","last_became_peered":"2026-02-24T22:05:16.188154+0000","last_unstale":"2026-02-24T22:34:35.114301+0000","last_undegraded":"2026-02-24T22:34:35.114301+0000","last_fullsized":"2026-02-24T22:34:35.114301+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'278","last_scrub_stamp":"2026-02-24T22:34:35.114217+0000","last_deep_scrub":"312'278","last_deep_scrub_stamp":"2026-02-24T22:34:35.114217+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.114217+0000","objects_scrubbed":6,"log_size":78,"log_dups_size":200,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:02:24.896641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":434,"num_read_kb":2459,"num_write":1229,"num_write_kb":1814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"312'6","reported_seq":831,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:52.862848+0000","last_change":"2026-02-24T22:34:52.862809+0000","last_active":"2026-02-24T22:34:52.862848+0000","last_peered":"2026-02-24T22:34:52.862848+0000","last_clean":"2026-02-24T22:34:52.862848+0000","last_became_active":"2026-02-24T22:05:20.195202+0000","last_became_peered":"2026-02-24T22:05:20.195202+0000","last_unstale":"2026-02-24T22:34:52.862848+0000","last_undegraded":"2026-02-24T22:34:52.862848+0000","last_fullsized":"2026-02-24T22:34:52.862848+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'6","last_scrub_stamp":"2026-02-24T22:34:52.862767+0000","last_deep_scrub":"312'6","last_deep_scrub_stamp":"2026-02-24T22:34:52.862767+0000","last_clean_scrub_stamp":"2026-02-24T22:34:52.862767+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:47:13.375532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1242,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":488,"num_read_kb":722,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"312'693","reported_seq":1779,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:55.932150+0000","last_change":"2026-02-24T22:34:55.932150+0000","last_active":"2026-02-24T22:34:55.932150+0000","last_peered":"2026-02-24T22:34:55.932150+0000","last_clean":"2026-02-24T22:34:55.932150+0000","last_became_active":"2026-02-24T22:05:22.192035+0000","last_became_peered":"2026-02-24T22:05:22.192035+0000","last_unstale":"2026-02-24T22:34:55.932150+0000","last_undegraded":"2026-02-24T22:34:55.932150+0000","last_fullsized":"2026-02-24T22:34:55.932150+0000","mapping_epoch":28,"log_start":"312'600","ondisk_log_start":"312'600","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'693","last_scrub_stamp":"2026-02-24T22:34:55.932092+0000","last_deep_scrub":"312'693","last_deep_scrub_stamp":"2026-02-24T22:34:55.932092+0000","last_clean_scrub_stamp":"2026-02-24T22:34:55.932092+0000","objects_scrubbed":6,"log_size":93,"log_dups_size":600,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:00:00.955174+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38403,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2221,"num_read_kb":1805,"num_write":316,"num_write_kb":280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"312'2582","reported_seq":5257,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:51.588402+0000","last_change":"2026-02-24T22:34:51.588402+0000","last_active":"2026-02-24T22:34:51.588402+0000","last_peered":"2026-02-24T22:34:51.588402+0000","last_clean":"2026-02-24T22:34:51.588402+0000","last_became_active":"2026-02-24T22:05:24.205557+0000","last_became_peered":"2026-02-24T22:05:24.205557+0000","last_unstale":"2026-02-24T22:34:51.588402+0000","last_undegraded":"2026-02-24T22:34:51.588402+0000","last_fullsized":"2026-02-24T22:34:51.588402+0000","mapping_epoch":30,"log_start":"312'2500","ondisk_log_start":"312'2500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2582","last_scrub_stamp":"2026-02-24T22:34:51.588360+0000","last_deep_scrub":"312'2582","last_deep_scrub_stamp":"2026-02-24T22:34:51.588360+0000","last_clean_scrub_stamp":"2026-02-24T22:34:51.588360+0000","objects_scrubbed":105,"log_size":82,"log_dups_size":2500,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:15:50.057193+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34698,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":6755,"num_read_kb":6753,"num_write":5061,"num_write_kb":2022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":584,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:07.610091+0000","last_change":"2026-02-24T22:35:07.610091+0000","last_active":"2026-02-24T22:35:07.610091+0000","last_peered":"2026-02-24T22:35:07.610091+0000","last_clean":"2026-02-24T22:35:07.610091+0000","last_became_active":"2026-02-24T22:05:29.651760+0000","last_became_peered":"2026-02-24T22:05:29.651760+0000","last_unstale":"2026-02-24T22:35:07.610091+0000","last_undegraded":"2026-02-24T22:35:07.610091+0000","last_fullsized":"2026-02-24T22:35:07.610091+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:35:07.610000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:35:07.610000+0000","last_clean_scrub_stamp":"2026-02-24T22:35:07.610000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:05:45.306861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"312'155","reported_seq":841,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.775671+0000","last_change":"2026-02-24T22:34:44.775671+0000","last_active":"2026-02-24T22:34:44.775671+0000","last_peered":"2026-02-24T22:34:44.775671+0000","last_clean":"2026-02-24T22:34:44.775671+0000","last_became_active":"2026-02-24T22:05:18.184939+0000","last_became_peered":"2026-02-24T22:05:18.184939+0000","last_unstale":"2026-02-24T22:34:44.775671+0000","last_undegraded":"2026-02-24T22:34:44.775671+0000","last_fullsized":"2026-02-24T22:34:44.775671+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'155","last_scrub_stamp":"2026-02-24T22:34:44.775613+0000","last_deep_scrub":"312'155","last_deep_scrub_stamp":"2026-02-24T22:34:44.775613+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.775613+0000","objects_scrubbed":27,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:46:49.648242+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":436,"num_read_kb":360,"num_write":185,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"312'291","reported_seq":1024,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.911303+0000","last_change":"2026-02-24T22:34:34.911246+0000","last_active":"2026-02-24T22:34:34.911303+0000","last_peered":"2026-02-24T22:34:34.911303+0000","last_clean":"2026-02-24T22:34:34.911303+0000","last_became_active":"2026-02-24T22:05:16.186043+0000","last_became_peered":"2026-02-24T22:05:16.186043+0000","last_unstale":"2026-02-24T22:34:34.911303+0000","last_undegraded":"2026-02-24T22:34:34.911303+0000","last_fullsized":"2026-02-24T22:34:34.911303+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'291","last_scrub_stamp":"2026-02-24T22:34:34.911183+0000","last_deep_scrub":"312'291","last_deep_scrub_stamp":"2026-02-24T22:34:34.911183+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.911183+0000","objects_scrubbed":4,"log_size":91,"log_dups_size":200,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:44:00.316854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":465,"num_read_kb":15751,"num_write":1254,"num_write_kb":5255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":598,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:57.248861+0000","last_change":"2026-02-24T22:34:57.248861+0000","last_active":"2026-02-24T22:34:57.248861+0000","last_peered":"2026-02-24T22:34:57.248861+0000","last_clean":"2026-02-24T22:34:57.248861+0000","last_became_active":"2026-02-24T22:05:20.192962+0000","last_became_peered":"2026-02-24T22:05:20.192962+0000","last_unstale":"2026-02-24T22:34:57.248861+0000","last_undegraded":"2026-02-24T22:34:57.248861+0000","last_fullsized":"2026-02-24T22:34:57.248861+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T22:34:57.248813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T22:34:57.248813+0000","last_clean_scrub_stamp":"2026-02-24T22:34:57.248813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:44:05.255470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"312'478","reported_seq":1324,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:59.613005+0000","last_change":"2026-02-24T22:34:59.612935+0000","last_active":"2026-02-24T22:34:59.613005+0000","last_peered":"2026-02-24T22:34:59.613005+0000","last_clean":"2026-02-24T22:34:59.613005+0000","last_became_active":"2026-02-24T22:05:22.193896+0000","last_became_peered":"2026-02-24T22:05:22.193896+0000","last_unstale":"2026-02-24T22:34:59.613005+0000","last_undegraded":"2026-02-24T22:34:59.613005+0000","last_fullsized":"2026-02-24T22:34:59.613005+0000","mapping_epoch":28,"log_start":"312'400","ondisk_log_start":"312'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'478","last_scrub_stamp":"2026-02-24T22:34:59.612864+0000","last_deep_scrub":"312'478","last_deep_scrub_stamp":"2026-02-24T22:34:59.612864+0000","last_clean_scrub_stamp":"2026-02-24T22:34:59.612864+0000","objects_scrubbed":3,"log_size":78,"log_dups_size":400,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:18:12.188442+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41178,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1310,"num_read_kb":1453,"num_write":355,"num_write_kb":318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"312'282","reported_seq":909,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:02.283510+0000","last_change":"2026-02-24T22:35:02.283472+0000","last_active":"2026-02-24T22:35:02.283510+0000","last_peered":"2026-02-24T22:35:02.283510+0000","last_clean":"2026-02-24T22:35:02.283510+0000","last_became_active":"2026-02-24T22:05:24.205855+0000","last_became_peered":"2026-02-24T22:05:24.205855+0000","last_unstale":"2026-02-24T22:35:02.283510+0000","last_undegraded":"2026-02-24T22:35:02.283510+0000","last_fullsized":"2026-02-24T22:35:02.283510+0000","mapping_epoch":30,"log_start":"312'200","ondisk_log_start":"312'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'282","last_scrub_stamp":"2026-02-24T22:35:02.283427+0000","last_deep_scrub":"312'282","last_deep_scrub_stamp":"2026-02-24T22:35:02.283427+0000","last_clean_scrub_stamp":"2026-02-24T22:35:02.283427+0000","objects_scrubbed":94,"log_size":82,"log_dups_size":200,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T10:30:07.155275+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27729,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":196,"num_read_kb":179,"num_write":471,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"312'449","reported_seq":1629,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:04.238088+0000","last_change":"2026-02-24T22:35:04.238088+0000","last_active":"2026-02-24T22:35:04.238088+0000","last_peered":"2026-02-24T22:35:04.238088+0000","last_clean":"2026-02-24T22:35:04.238088+0000","last_became_active":"2026-02-24T22:05:29.661983+0000","last_became_peered":"2026-02-24T22:05:29.661983+0000","last_unstale":"2026-02-24T22:35:04.238088+0000","last_undegraded":"2026-02-24T22:35:04.238088+0000","last_fullsized":"2026-02-24T22:35:04.238088+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'449","last_scrub_stamp":"2026-02-24T22:35:04.238044+0000","last_deep_scrub":"312'449","last_deep_scrub_stamp":"2026-02-24T22:35:04.238044+0000","last_clean_scrub_stamp":"2026-02-24T22:35:04.238044+0000","objects_scrubbed":1,"log_size":49,"log_dups_size":400,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:37:34.593042+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"312'191","reported_seq":883,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.478571+0000","last_change":"2026-02-24T22:34:40.478571+0000","last_active":"2026-02-24T22:34:40.478571+0000","last_peered":"2026-02-24T22:34:40.478571+0000","last_clean":"2026-02-24T22:34:40.478571+0000","last_became_active":"2026-02-24T22:05:18.189877+0000","last_became_peered":"2026-02-24T22:05:18.189877+0000","last_unstale":"2026-02-24T22:34:40.478571+0000","last_undegraded":"2026-02-24T22:34:40.478571+0000","last_fullsized":"2026-02-24T22:34:40.478571+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'191","last_scrub_stamp":"2026-02-24T22:34:40.478532+0000","last_deep_scrub":"312'191","last_deep_scrub_stamp":"2026-02-24T22:34:40.478532+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.478532+0000","objects_scrubbed":22,"log_size":91,"log_dups_size":100,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:10:11.001904+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":630,"num_read_kb":510,"num_write":249,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"312'332","reported_seq":1100,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.714508+0000","last_change":"2026-02-24T22:34:34.714508+0000","last_active":"2026-02-24T22:34:34.714508+0000","last_peered":"2026-02-24T22:34:34.714508+0000","last_clean":"2026-02-24T22:34:34.714508+0000","last_became_active":"2026-02-24T22:05:16.184525+0000","last_became_peered":"2026-02-24T22:05:16.184525+0000","last_unstale":"2026-02-24T22:34:34.714508+0000","last_undegraded":"2026-02-24T22:34:34.714508+0000","last_fullsized":"2026-02-24T22:34:34.714508+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'332","last_scrub_stamp":"2026-02-24T22:34:34.714447+0000","last_deep_scrub":"312'332","last_deep_scrub_stamp":"2026-02-24T22:34:34.714447+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.714447+0000","objects_scrubbed":7,"log_size":32,"log_dups_size":300,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:44:54.170379+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242941,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":576,"num_read_kb":16320,"num_write":1488,"num_write_kb":16752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"312'2","reported_seq":598,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.250464+0000","last_change":"2026-02-24T22:34:48.250464+0000","last_active":"2026-02-24T22:34:48.250464+0000","last_peered":"2026-02-24T22:34:48.250464+0000","last_clean":"2026-02-24T22:34:48.250464+0000","last_became_active":"2026-02-24T22:05:20.186850+0000","last_became_peered":"2026-02-24T22:05:20.186850+0000","last_unstale":"2026-02-24T22:34:48.250464+0000","last_undegraded":"2026-02-24T22:34:48.250464+0000","last_fullsized":"2026-02-24T22:34:48.250464+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'2","last_scrub_stamp":"2026-02-24T22:34:48.250415+0000","last_deep_scrub":"312'2","last_deep_scrub_stamp":"2026-02-24T22:34:48.250415+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.250415+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:00:47.346514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"312'253","reported_seq":885,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:59.281527+0000","last_change":"2026-02-24T22:34:59.281488+0000","last_active":"2026-02-24T22:34:59.281527+0000","last_peered":"2026-02-24T22:34:59.281527+0000","last_clean":"2026-02-24T22:34:59.281527+0000","last_became_active":"2026-02-24T22:05:22.199358+0000","last_became_peered":"2026-02-24T22:05:22.199358+0000","last_unstale":"2026-02-24T22:34:59.281527+0000","last_undegraded":"2026-02-24T22:34:59.281527+0000","last_fullsized":"2026-02-24T22:34:59.281527+0000","mapping_epoch":28,"log_start":"312'200","ondisk_log_start":"312'200","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'253","last_scrub_stamp":"2026-02-24T22:34:59.281432+0000","last_deep_scrub":"312'253","last_deep_scrub_stamp":"2026-02-24T22:34:59.281432+0000","last_clean_scrub_stamp":"2026-02-24T22:34:59.281432+0000","objects_scrubbed":4,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:21:43.387290+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37935,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":401,"num_read_kb":1010,"num_write":360,"num_write_kb":308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"312'382","reported_seq":1338,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:02.816448+0000","last_change":"2026-02-24T22:35:02.816402+0000","last_active":"2026-02-24T22:35:02.816448+0000","last_peered":"2026-02-24T22:35:02.816448+0000","last_clean":"2026-02-24T22:35:02.816448+0000","last_became_active":"2026-02-24T22:05:24.199555+0000","last_became_peered":"2026-02-24T22:05:24.199555+0000","last_unstale":"2026-02-24T22:35:02.816448+0000","last_undegraded":"2026-02-24T22:35:02.816448+0000","last_fullsized":"2026-02-24T22:35:02.816448+0000","mapping_epoch":30,"log_start":"312'295","ondisk_log_start":"312'295","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'382","last_scrub_stamp":"2026-02-24T22:35:02.816332+0000","last_deep_scrub":"312'382","last_deep_scrub_stamp":"2026-02-24T22:35:02.816332+0000","last_clean_scrub_stamp":"2026-02-24T22:35:02.816332+0000","objects_scrubbed":101,"log_size":87,"log_dups_size":295,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:38:25.214542+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30969,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":655,"num_read_kb":647,"num_write":639,"num_write_kb":204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":174,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"312'433","reported_seq":1959,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:35:04.303436+0000","last_change":"2026-02-24T22:35:04.303436+0000","last_active":"2026-02-24T22:35:04.303436+0000","last_peered":"2026-02-24T22:35:04.303436+0000","last_clean":"2026-02-24T22:35:04.303436+0000","last_became_active":"2026-02-24T22:05:29.659532+0000","last_became_peered":"2026-02-24T22:05:29.659532+0000","last_unstale":"2026-02-24T22:35:04.303436+0000","last_undegraded":"2026-02-24T22:35:04.303436+0000","last_fullsized":"2026-02-24T22:35:04.303436+0000","mapping_epoch":33,"log_start":"312'400","ondisk_log_start":"312'400","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'433","last_scrub_stamp":"2026-02-24T22:35:04.303372+0000","last_deep_scrub":"312'433","last_deep_scrub_stamp":"2026-02-24T22:35:04.303372+0000","last_clean_scrub_stamp":"2026-02-24T22:35:04.303372+0000","objects_scrubbed":1,"log_size":33,"log_dups_size":400,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:32:29.785307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"312'162","reported_seq":858,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.157812+0000","last_change":"2026-02-24T22:34:45.157775+0000","last_active":"2026-02-24T22:34:45.157812+0000","last_peered":"2026-02-24T22:34:45.157812+0000","last_clean":"2026-02-24T22:34:45.157812+0000","last_became_active":"2026-02-24T22:05:18.184250+0000","last_became_peered":"2026-02-24T22:05:18.184250+0000","last_unstale":"2026-02-24T22:34:45.157812+0000","last_undegraded":"2026-02-24T22:34:45.157812+0000","last_fullsized":"2026-02-24T22:34:45.157812+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'162","last_scrub_stamp":"2026-02-24T22:34:45.157726+0000","last_deep_scrub":"312'162","last_deep_scrub_stamp":"2026-02-24T22:34:45.157726+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.157726+0000","objects_scrubbed":23,"log_size":62,"log_dups_size":100,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:39:01.351061+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":560,"num_read_kb":462,"num_write":204,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"312'1610","reported_seq":2974,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.774854+0000","last_change":"2026-02-24T22:34:33.774854+0000","last_active":"2026-02-24T22:34:33.774854+0000","last_peered":"2026-02-24T22:34:33.774854+0000","last_clean":"2026-02-24T22:34:33.774854+0000","last_became_active":"2026-02-24T22:05:16.189660+0000","last_became_peered":"2026-02-24T22:05:16.189660+0000","last_unstale":"2026-02-24T22:34:33.774854+0000","last_undegraded":"2026-02-24T22:34:33.774854+0000","last_fullsized":"2026-02-24T22:34:33.774854+0000","mapping_epoch":22,"log_start":"312'1500","ondisk_log_start":"312'1500","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'1610","last_scrub_stamp":"2026-02-24T22:34:33.774776+0000","last_deep_scrub":"312'1610","last_deep_scrub_stamp":"2026-02-24T22:34:33.774776+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.774776+0000","objects_scrubbed":6,"log_size":110,"log_dups_size":1500,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:08:14.555476+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4326,"num_read_kb":2601,"num_write":2684,"num_write_kb":4261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"312'121","reported_seq":809,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.128851+0000","last_change":"2026-02-24T22:34:43.128824+0000","last_active":"2026-02-24T22:34:43.128851+0000","last_peered":"2026-02-24T22:34:43.128851+0000","last_clean":"2026-02-24T22:34:43.128851+0000","last_became_active":"2026-02-24T22:05:18.183822+0000","last_became_peered":"2026-02-24T22:05:18.183822+0000","last_unstale":"2026-02-24T22:34:43.128851+0000","last_undegraded":"2026-02-24T22:34:43.128851+0000","last_fullsized":"2026-02-24T22:34:43.128851+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'121","last_scrub_stamp":"2026-02-24T22:34:43.128792+0000","last_deep_scrub":"312'121","last_deep_scrub_stamp":"2026-02-24T22:34:43.128792+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.128792+0000","objects_scrubbed":20,"log_size":21,"log_dups_size":100,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:33:44.340989+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":390,"num_read_kb":330,"num_write":143,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"312'303","reported_seq":1060,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:32.123769+0000","last_change":"2026-02-24T22:34:32.123769+0000","last_active":"2026-02-24T22:34:32.123769+0000","last_peered":"2026-02-24T22:34:32.123769+0000","last_clean":"2026-02-24T22:34:32.123769+0000","last_became_active":"2026-02-24T22:05:16.183487+0000","last_became_peered":"2026-02-24T22:05:16.183487+0000","last_unstale":"2026-02-24T22:34:32.123769+0000","last_undegraded":"2026-02-24T22:34:32.123769+0000","last_fullsized":"2026-02-24T22:34:32.123769+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'303","last_scrub_stamp":"2026-02-24T22:34:32.123701+0000","last_deep_scrub":"312'303","last_deep_scrub_stamp":"2026-02-24T22:34:32.123701+0000","last_clean_scrub_stamp":"2026-02-24T22:34:32.123701+0000","objects_scrubbed":3,"log_size":103,"log_dups_size":200,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:46:27.988906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":514,"num_read_kb":706,"num_write":1357,"num_write_kb":183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"312'92","reported_seq":761,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.957554+0000","last_change":"2026-02-24T22:34:44.957519+0000","last_active":"2026-02-24T22:34:44.957554+0000","last_peered":"2026-02-24T22:34:44.957554+0000","last_clean":"2026-02-24T22:34:44.957554+0000","last_became_active":"2026-02-24T22:05:18.190352+0000","last_became_peered":"2026-02-24T22:05:18.190352+0000","last_unstale":"2026-02-24T22:34:44.957554+0000","last_undegraded":"2026-02-24T22:34:44.957554+0000","last_fullsized":"2026-02-24T22:34:44.957554+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'92","last_scrub_stamp":"2026-02-24T22:34:44.957460+0000","last_deep_scrub":"312'92","last_deep_scrub_stamp":"2026-02-24T22:34:44.957460+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.957460+0000","objects_scrubbed":20,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:57:33.270079+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":274,"num_read_kb":236,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"312'318","reported_seq":1249,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.115775+0000","last_change":"2026-02-24T22:34:38.115775+0000","last_active":"2026-02-24T22:34:38.115775+0000","last_peered":"2026-02-24T22:34:38.115775+0000","last_clean":"2026-02-24T22:34:38.115775+0000","last_became_active":"2026-02-24T22:05:16.190494+0000","last_became_peered":"2026-02-24T22:05:16.190494+0000","last_unstale":"2026-02-24T22:34:38.115775+0000","last_undegraded":"2026-02-24T22:34:38.115775+0000","last_fullsized":"2026-02-24T22:34:38.115775+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'318","last_scrub_stamp":"2026-02-24T22:34:38.115706+0000","last_deep_scrub":"312'318","last_deep_scrub_stamp":"2026-02-24T22:34:38.115706+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.115706+0000","objects_scrubbed":4,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:08:06.722436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1108,"num_read_kb":14913,"num_write":1331,"num_write_kb":8068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"312'367","reported_seq":1191,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:41.111017+0000","last_change":"2026-02-24T22:34:41.110977+0000","last_active":"2026-02-24T22:34:41.111017+0000","last_peered":"2026-02-24T22:34:41.111017+0000","last_clean":"2026-02-24T22:34:41.111017+0000","last_became_active":"2026-02-24T22:05:18.186622+0000","last_became_peered":"2026-02-24T22:05:18.186622+0000","last_unstale":"2026-02-24T22:34:41.111017+0000","last_undegraded":"2026-02-24T22:34:41.111017+0000","last_fullsized":"2026-02-24T22:34:41.111017+0000","mapping_epoch":24,"log_start":"312'300","ondisk_log_start":"312'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'367","last_scrub_stamp":"2026-02-24T22:34:41.110934+0000","last_deep_scrub":"312'367","last_deep_scrub_stamp":"2026-02-24T22:34:41.110934+0000","last_clean_scrub_stamp":"2026-02-24T22:34:41.110934+0000","objects_scrubbed":30,"log_size":67,"log_dups_size":300,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:59:07.931667+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1512,"num_read_kb":1316,"num_write":481,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25750,"num_omap_keys":101,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"312'319","reported_seq":1167,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.742796+0000","last_change":"2026-02-24T22:34:36.742796+0000","last_active":"2026-02-24T22:34:36.742796+0000","last_peered":"2026-02-24T22:34:36.742796+0000","last_clean":"2026-02-24T22:34:36.742796+0000","last_became_active":"2026-02-24T22:05:16.185977+0000","last_became_peered":"2026-02-24T22:05:16.185977+0000","last_unstale":"2026-02-24T22:34:36.742796+0000","last_undegraded":"2026-02-24T22:34:36.742796+0000","last_fullsized":"2026-02-24T22:34:36.742796+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'319","last_scrub_stamp":"2026-02-24T22:34:36.742708+0000","last_deep_scrub":"312'319","last_deep_scrub_stamp":"2026-02-24T22:34:36.742708+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.742708+0000","objects_scrubbed":1,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:21:32.964403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":948,"num_read_kb":5969,"num_write":1368,"num_write_kb":2943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"312'114","reported_seq":799,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.783698+0000","last_change":"2026-02-24T22:34:46.783698+0000","last_active":"2026-02-24T22:34:46.783698+0000","last_peered":"2026-02-24T22:34:46.783698+0000","last_clean":"2026-02-24T22:34:46.783698+0000","last_became_active":"2026-02-24T22:05:18.185180+0000","last_became_peered":"2026-02-24T22:05:18.185180+0000","last_unstale":"2026-02-24T22:34:46.783698+0000","last_undegraded":"2026-02-24T22:34:46.783698+0000","last_fullsized":"2026-02-24T22:34:46.783698+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'114","last_scrub_stamp":"2026-02-24T22:34:46.783630+0000","last_deep_scrub":"312'114","last_deep_scrub_stamp":"2026-02-24T22:34:46.783630+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.783630+0000","objects_scrubbed":25,"log_size":14,"log_dups_size":100,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:34:22.929604+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":311,"num_read_kb":264,"num_write":113,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":272,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"312'374","reported_seq":1134,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.080105+0000","last_change":"2026-02-24T22:34:34.080105+0000","last_active":"2026-02-24T22:34:34.080105+0000","last_peered":"2026-02-24T22:34:34.080105+0000","last_clean":"2026-02-24T22:34:34.080105+0000","last_became_active":"2026-02-24T22:06:00.660740+0000","last_became_peered":"2026-02-24T22:06:00.660740+0000","last_unstale":"2026-02-24T22:34:34.080105+0000","last_undegraded":"2026-02-24T22:34:34.080105+0000","last_fullsized":"2026-02-24T22:34:34.080105+0000","mapping_epoch":36,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'374","last_scrub_stamp":"2026-02-24T22:34:34.080040+0000","last_deep_scrub":"312'374","last_deep_scrub_stamp":"2026-02-24T22:34:34.080040+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.080040+0000","objects_scrubbed":4,"log_size":74,"log_dups_size":300,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:17:10.260943+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":666,"num_read_kb":9895,"num_write":1405,"num_write_kb":11975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"312'89","reported_seq":768,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:43.509973+0000","last_change":"2026-02-24T22:34:43.509973+0000","last_active":"2026-02-24T22:34:43.509973+0000","last_peered":"2026-02-24T22:34:43.509973+0000","last_clean":"2026-02-24T22:34:43.509973+0000","last_became_active":"2026-02-24T22:05:18.190390+0000","last_became_peered":"2026-02-24T22:05:18.190390+0000","last_unstale":"2026-02-24T22:34:43.509973+0000","last_undegraded":"2026-02-24T22:34:43.509973+0000","last_fullsized":"2026-02-24T22:34:43.509973+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'89","last_scrub_stamp":"2026-02-24T22:34:43.509906+0000","last_deep_scrub":"312'89","last_deep_scrub_stamp":"2026-02-24T22:34:43.509906+0000","last_clean_scrub_stamp":"2026-02-24T22:34:43.509906+0000","objects_scrubbed":23,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:26:36.232359+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":213,"num_read_kb":187,"num_write":90,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"312'336","reported_seq":1068,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.988664+0000","last_change":"2026-02-24T22:34:34.988612+0000","last_active":"2026-02-24T22:34:34.988664+0000","last_peered":"2026-02-24T22:34:34.988664+0000","last_clean":"2026-02-24T22:34:34.988664+0000","last_became_active":"2026-02-24T22:05:16.184091+0000","last_became_peered":"2026-02-24T22:05:16.184091+0000","last_unstale":"2026-02-24T22:34:34.988664+0000","last_undegraded":"2026-02-24T22:34:34.988664+0000","last_fullsized":"2026-02-24T22:34:34.988664+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'336","last_scrub_stamp":"2026-02-24T22:34:34.988543+0000","last_deep_scrub":"312'336","last_deep_scrub_stamp":"2026-02-24T22:34:34.988543+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.988543+0000","objects_scrubbed":4,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:48:27.381128+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":558,"num_read_kb":406,"num_write":1237,"num_write_kb":6221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"312'375","reported_seq":1187,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.800000+0000","last_change":"2026-02-24T22:34:45.800000+0000","last_active":"2026-02-24T22:34:45.800000+0000","last_peered":"2026-02-24T22:34:45.800000+0000","last_clean":"2026-02-24T22:34:45.800000+0000","last_became_active":"2026-02-24T22:05:18.186746+0000","last_became_peered":"2026-02-24T22:05:18.186746+0000","last_unstale":"2026-02-24T22:34:45.800000+0000","last_undegraded":"2026-02-24T22:34:45.800000+0000","last_fullsized":"2026-02-24T22:34:45.800000+0000","mapping_epoch":24,"log_start":"312'300","ondisk_log_start":"312'300","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'375","last_scrub_stamp":"2026-02-24T22:34:45.799935+0000","last_deep_scrub":"312'375","last_deep_scrub_stamp":"2026-02-24T22:34:45.799935+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.799935+0000","objects_scrubbed":20,"log_size":75,"log_dups_size":300,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:08:40.366084+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":1676,"num_read_kb":1392,"num_write":638,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"312'331","reported_seq":1044,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:34.775032+0000","last_change":"2026-02-24T22:34:34.774985+0000","last_active":"2026-02-24T22:34:34.775032+0000","last_peered":"2026-02-24T22:34:34.775032+0000","last_clean":"2026-02-24T22:34:34.775032+0000","last_became_active":"2026-02-24T22:05:16.189516+0000","last_became_peered":"2026-02-24T22:05:16.189516+0000","last_unstale":"2026-02-24T22:34:34.775032+0000","last_undegraded":"2026-02-24T22:34:34.775032+0000","last_fullsized":"2026-02-24T22:34:34.775032+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'331","last_scrub_stamp":"2026-02-24T22:34:34.774895+0000","last_deep_scrub":"312'331","last_deep_scrub_stamp":"2026-02-24T22:34:34.774895+0000","last_clean_scrub_stamp":"2026-02-24T22:34:34.774895+0000","objects_scrubbed":2,"log_size":31,"log_dups_size":300,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:51:58.631151+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":448,"num_read_kb":5491,"num_write":1228,"num_write_kb":20654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"312'184","reported_seq":884,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.708693+0000","last_change":"2026-02-24T22:34:40.708693+0000","last_active":"2026-02-24T22:34:40.708693+0000","last_peered":"2026-02-24T22:34:40.708693+0000","last_clean":"2026-02-24T22:34:40.708693+0000","last_became_active":"2026-02-24T22:05:18.190203+0000","last_became_peered":"2026-02-24T22:05:18.190203+0000","last_unstale":"2026-02-24T22:34:40.708693+0000","last_undegraded":"2026-02-24T22:34:40.708693+0000","last_fullsized":"2026-02-24T22:34:40.708693+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'184","last_scrub_stamp":"2026-02-24T22:34:40.708623+0000","last_deep_scrub":"312'184","last_deep_scrub_stamp":"2026-02-24T22:34:40.708623+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.708623+0000","objects_scrubbed":20,"log_size":84,"log_dups_size":100,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:12:32.032756+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":631,"num_read_kb":516,"num_write":248,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"312'319","reported_seq":1047,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.869860+0000","last_change":"2026-02-24T22:34:35.869813+0000","last_active":"2026-02-24T22:34:35.869860+0000","last_peered":"2026-02-24T22:34:35.869860+0000","last_clean":"2026-02-24T22:34:35.869860+0000","last_became_active":"2026-02-24T22:05:16.183772+0000","last_became_peered":"2026-02-24T22:05:16.183772+0000","last_unstale":"2026-02-24T22:34:35.869860+0000","last_undegraded":"2026-02-24T22:34:35.869860+0000","last_fullsized":"2026-02-24T22:34:35.869860+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'319","last_scrub_stamp":"2026-02-24T22:34:35.869749+0000","last_deep_scrub":"312'319","last_deep_scrub_stamp":"2026-02-24T22:34:35.869749+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.869749+0000","objects_scrubbed":2,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:10:38.609121+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":480,"num_read_kb":12489,"num_write":1341,"num_write_kb":5970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"312'156","reported_seq":886,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.880196+0000","last_change":"2026-02-24T22:34:45.880154+0000","last_active":"2026-02-24T22:34:45.880196+0000","last_peered":"2026-02-24T22:34:45.880196+0000","last_clean":"2026-02-24T22:34:45.880196+0000","last_became_active":"2026-02-24T22:05:18.187321+0000","last_became_peered":"2026-02-24T22:05:18.187321+0000","last_unstale":"2026-02-24T22:34:45.880196+0000","last_undegraded":"2026-02-24T22:34:45.880196+0000","last_fullsized":"2026-02-24T22:34:45.880196+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'156","last_scrub_stamp":"2026-02-24T22:34:45.880085+0000","last_deep_scrub":"312'156","last_deep_scrub_stamp":"2026-02-24T22:34:45.880085+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.880085+0000","objects_scrubbed":26,"log_size":56,"log_dups_size":100,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:17:02.414903+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":512,"num_read_kb":441,"num_write":201,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"312'326","reported_seq":1043,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.166689+0000","last_change":"2026-02-24T22:34:37.166643+0000","last_active":"2026-02-24T22:34:37.166689+0000","last_peered":"2026-02-24T22:34:37.166689+0000","last_clean":"2026-02-24T22:34:37.166689+0000","last_became_active":"2026-02-24T22:05:16.193164+0000","last_became_peered":"2026-02-24T22:05:16.193164+0000","last_unstale":"2026-02-24T22:34:37.166689+0000","last_undegraded":"2026-02-24T22:34:37.166689+0000","last_fullsized":"2026-02-24T22:34:37.166689+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'326","last_scrub_stamp":"2026-02-24T22:34:37.166498+0000","last_deep_scrub":"312'326","last_deep_scrub_stamp":"2026-02-24T22:34:37.166498+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.166498+0000","objects_scrubbed":6,"log_size":26,"log_dups_size":300,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:54:53.815454+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242922,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":434,"num_read_kb":36424,"num_write":1378,"num_write_kb":10617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"312'166","reported_seq":853,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.417459+0000","last_change":"2026-02-24T22:34:46.417427+0000","last_active":"2026-02-24T22:34:46.417459+0000","last_peered":"2026-02-24T22:34:46.417459+0000","last_clean":"2026-02-24T22:34:46.417459+0000","last_became_active":"2026-02-24T22:05:18.191737+0000","last_became_peered":"2026-02-24T22:05:18.191737+0000","last_unstale":"2026-02-24T22:34:46.417459+0000","last_undegraded":"2026-02-24T22:34:46.417459+0000","last_fullsized":"2026-02-24T22:34:46.417459+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'166","last_scrub_stamp":"2026-02-24T22:34:46.417398+0000","last_deep_scrub":"312'166","last_deep_scrub_stamp":"2026-02-24T22:34:46.417398+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.417398+0000","objects_scrubbed":18,"log_size":66,"log_dups_size":100,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:30:36.821543+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":539,"num_read_kb":448,"num_write":211,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"312'377","reported_seq":1137,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:38.766166+0000","last_change":"2026-02-24T22:34:38.766166+0000","last_active":"2026-02-24T22:34:38.766166+0000","last_peered":"2026-02-24T22:34:38.766166+0000","last_clean":"2026-02-24T22:34:38.766166+0000","last_became_active":"2026-02-24T22:05:16.183923+0000","last_became_peered":"2026-02-24T22:05:16.183923+0000","last_unstale":"2026-02-24T22:34:38.766166+0000","last_undegraded":"2026-02-24T22:34:38.766166+0000","last_fullsized":"2026-02-24T22:34:38.766166+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'377","last_scrub_stamp":"2026-02-24T22:34:38.766121+0000","last_deep_scrub":"312'377","last_deep_scrub_stamp":"2026-02-24T22:34:38.766121+0000","last_clean_scrub_stamp":"2026-02-24T22:34:38.766121+0000","objects_scrubbed":6,"log_size":77,"log_dups_size":300,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:41:28.025068+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242922,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":549,"num_read_kb":9842,"num_write":1598,"num_write_kb":9073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"312'167","reported_seq":848,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.060961+0000","last_change":"2026-02-24T22:34:39.060923+0000","last_active":"2026-02-24T22:34:39.060961+0000","last_peered":"2026-02-24T22:34:39.060961+0000","last_clean":"2026-02-24T22:34:39.060961+0000","last_became_active":"2026-02-24T22:05:18.183761+0000","last_became_peered":"2026-02-24T22:05:18.183761+0000","last_unstale":"2026-02-24T22:34:39.060961+0000","last_undegraded":"2026-02-24T22:34:39.060961+0000","last_fullsized":"2026-02-24T22:34:39.060961+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'167","last_scrub_stamp":"2026-02-24T22:34:39.060863+0000","last_deep_scrub":"312'167","last_deep_scrub_stamp":"2026-02-24T22:34:39.060863+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.060863+0000","objects_scrubbed":21,"log_size":67,"log_dups_size":100,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:07:32.105462+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":561,"num_read_kb":451,"num_write":221,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"312'341","reported_seq":1119,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:39.077639+0000","last_change":"2026-02-24T22:34:39.077639+0000","last_active":"2026-02-24T22:34:39.077639+0000","last_peered":"2026-02-24T22:34:39.077639+0000","last_clean":"2026-02-24T22:34:39.077639+0000","last_became_active":"2026-02-24T22:05:16.189759+0000","last_became_peered":"2026-02-24T22:05:16.189759+0000","last_unstale":"2026-02-24T22:34:39.077639+0000","last_undegraded":"2026-02-24T22:34:39.077639+0000","last_fullsized":"2026-02-24T22:34:39.077639+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'341","last_scrub_stamp":"2026-02-24T22:34:39.077571+0000","last_deep_scrub":"312'341","last_deep_scrub_stamp":"2026-02-24T22:34:39.077571+0000","last_clean_scrub_stamp":"2026-02-24T22:34:39.077571+0000","objects_scrubbed":4,"log_size":41,"log_dups_size":300,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:09:54.075658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":656,"num_read_kb":3529,"num_write":1503,"num_write_kb":3094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"312'158","reported_seq":860,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.440468+0000","last_change":"2026-02-24T22:34:45.440468+0000","last_active":"2026-02-24T22:34:45.440468+0000","last_peered":"2026-02-24T22:34:45.440468+0000","last_clean":"2026-02-24T22:34:45.440468+0000","last_became_active":"2026-02-24T22:05:18.192882+0000","last_became_peered":"2026-02-24T22:05:18.192882+0000","last_unstale":"2026-02-24T22:34:45.440468+0000","last_undegraded":"2026-02-24T22:34:45.440468+0000","last_fullsized":"2026-02-24T22:34:45.440468+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'158","last_scrub_stamp":"2026-02-24T22:34:45.440425+0000","last_deep_scrub":"312'158","last_deep_scrub_stamp":"2026-02-24T22:34:45.440425+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.440425+0000","objects_scrubbed":18,"log_size":58,"log_dups_size":100,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:49:26.113842+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":593,"num_read_kb":494,"num_write":208,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"312'309","reported_seq":1056,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.732158+0000","last_change":"2026-02-24T22:34:35.732158+0000","last_active":"2026-02-24T22:34:35.732158+0000","last_peered":"2026-02-24T22:34:35.732158+0000","last_clean":"2026-02-24T22:34:35.732158+0000","last_became_active":"2026-02-24T22:05:16.184557+0000","last_became_peered":"2026-02-24T22:05:16.184557+0000","last_unstale":"2026-02-24T22:34:35.732158+0000","last_undegraded":"2026-02-24T22:34:35.732158+0000","last_fullsized":"2026-02-24T22:34:35.732158+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'309","last_scrub_stamp":"2026-02-24T22:34:35.732082+0000","last_deep_scrub":"312'309","last_deep_scrub_stamp":"2026-02-24T22:34:35.732082+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.732082+0000","objects_scrubbed":3,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:22:57.505362+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":498,"num_read_kb":1536,"num_write":1259,"num_write_kb":5590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"312'184","reported_seq":901,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:45.156109+0000","last_change":"2026-02-24T22:34:45.156109+0000","last_active":"2026-02-24T22:34:45.156109+0000","last_peered":"2026-02-24T22:34:45.156109+0000","last_clean":"2026-02-24T22:34:45.156109+0000","last_became_active":"2026-02-24T22:05:18.191432+0000","last_became_peered":"2026-02-24T22:05:18.191432+0000","last_unstale":"2026-02-24T22:34:45.156109+0000","last_undegraded":"2026-02-24T22:34:45.156109+0000","last_fullsized":"2026-02-24T22:34:45.156109+0000","mapping_epoch":24,"log_start":"307'100","ondisk_log_start":"307'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'184","last_scrub_stamp":"2026-02-24T22:34:45.156041+0000","last_deep_scrub":"312'184","last_deep_scrub_stamp":"2026-02-24T22:34:45.156041+0000","last_clean_scrub_stamp":"2026-02-24T22:34:45.156041+0000","objects_scrubbed":26,"log_size":84,"log_dups_size":100,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:54:34.074113+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":649,"num_read_kb":535,"num_write":233,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4005,"num_omap_keys":18,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"312'287","reported_seq":1017,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:36.765441+0000","last_change":"2026-02-24T22:34:36.765441+0000","last_active":"2026-02-24T22:34:36.765441+0000","last_peered":"2026-02-24T22:34:36.765441+0000","last_clean":"2026-02-24T22:34:36.765441+0000","last_became_active":"2026-02-24T22:05:16.190260+0000","last_became_peered":"2026-02-24T22:05:16.190260+0000","last_unstale":"2026-02-24T22:34:36.765441+0000","last_undegraded":"2026-02-24T22:34:36.765441+0000","last_fullsized":"2026-02-24T22:34:36.765441+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'287","last_scrub_stamp":"2026-02-24T22:34:36.765378+0000","last_deep_scrub":"312'287","last_deep_scrub_stamp":"2026-02-24T22:34:36.765378+0000","last_clean_scrub_stamp":"2026-02-24T22:34:36.765378+0000","objects_scrubbed":2,"log_size":87,"log_dups_size":200,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T08:18:33.840972+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":458,"num_read_kb":1504,"num_write":1357,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"312'171","reported_seq":869,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:44.861467+0000","last_change":"2026-02-24T22:34:44.861467+0000","last_active":"2026-02-24T22:34:44.861467+0000","last_peered":"2026-02-24T22:34:44.861467+0000","last_clean":"2026-02-24T22:34:44.861467+0000","last_became_active":"2026-02-24T22:05:18.185184+0000","last_became_peered":"2026-02-24T22:05:18.185184+0000","last_unstale":"2026-02-24T22:34:44.861467+0000","last_undegraded":"2026-02-24T22:34:44.861467+0000","last_fullsized":"2026-02-24T22:34:44.861467+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'171","last_scrub_stamp":"2026-02-24T22:34:44.861426+0000","last_deep_scrub":"312'171","last_deep_scrub_stamp":"2026-02-24T22:34:44.861426+0000","last_clean_scrub_stamp":"2026-02-24T22:34:44.861426+0000","objects_scrubbed":28,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:45:55.477850+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":536,"num_read_kb":445,"num_write":200,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"312'302","reported_seq":1124,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.033851+0000","last_change":"2026-02-24T22:34:40.033851+0000","last_active":"2026-02-24T22:34:40.033851+0000","last_peered":"2026-02-24T22:34:40.033851+0000","last_clean":"2026-02-24T22:34:40.033851+0000","last_became_active":"2026-02-24T22:05:16.191433+0000","last_became_peered":"2026-02-24T22:05:16.191433+0000","last_unstale":"2026-02-24T22:34:40.033851+0000","last_undegraded":"2026-02-24T22:34:40.033851+0000","last_fullsized":"2026-02-24T22:34:40.033851+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'302","last_scrub_stamp":"2026-02-24T22:34:40.033782+0000","last_deep_scrub":"312'302","last_deep_scrub_stamp":"2026-02-24T22:34:40.033782+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.033782+0000","objects_scrubbed":3,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:16:57.803888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":549,"num_read_kb":2264,"num_write":1330,"num_write_kb":7006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"312'86","reported_seq":755,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.772530+0000","last_change":"2026-02-24T22:34:48.772530+0000","last_active":"2026-02-24T22:34:48.772530+0000","last_peered":"2026-02-24T22:34:48.772530+0000","last_clean":"2026-02-24T22:34:48.772530+0000","last_became_active":"2026-02-24T22:05:18.192619+0000","last_became_peered":"2026-02-24T22:05:18.192619+0000","last_unstale":"2026-02-24T22:34:48.772530+0000","last_undegraded":"2026-02-24T22:34:48.772530+0000","last_fullsized":"2026-02-24T22:34:48.772530+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'86","last_scrub_stamp":"2026-02-24T22:34:48.772479+0000","last_deep_scrub":"312'86","last_deep_scrub_stamp":"2026-02-24T22:34:48.772479+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.772479+0000","objects_scrubbed":24,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:50:44.256005+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":204,"num_read_kb":176,"num_write":76,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"312'336","reported_seq":1077,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:33.091273+0000","last_change":"2026-02-24T22:34:33.091228+0000","last_active":"2026-02-24T22:34:33.091273+0000","last_peered":"2026-02-24T22:34:33.091273+0000","last_clean":"2026-02-24T22:34:33.091273+0000","last_became_active":"2026-02-24T22:05:16.184441+0000","last_became_peered":"2026-02-24T22:05:16.184441+0000","last_unstale":"2026-02-24T22:34:33.091273+0000","last_undegraded":"2026-02-24T22:34:33.091273+0000","last_fullsized":"2026-02-24T22:34:33.091273+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'336","last_scrub_stamp":"2026-02-24T22:34:33.091160+0000","last_deep_scrub":"312'336","last_deep_scrub_stamp":"2026-02-24T22:34:33.091160+0000","last_clean_scrub_stamp":"2026-02-24T22:34:33.091160+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:19:40.057693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":478,"num_read_kb":3856,"num_write":1468,"num_write_kb":8445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"312'183","reported_seq":910,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:48.176932+0000","last_change":"2026-02-24T22:34:48.176877+0000","last_active":"2026-02-24T22:34:48.176932+0000","last_peered":"2026-02-24T22:34:48.176932+0000","last_clean":"2026-02-24T22:34:48.176932+0000","last_became_active":"2026-02-24T22:05:18.191350+0000","last_became_peered":"2026-02-24T22:05:18.191350+0000","last_unstale":"2026-02-24T22:34:48.176932+0000","last_undegraded":"2026-02-24T22:34:48.176932+0000","last_fullsized":"2026-02-24T22:34:48.176932+0000","mapping_epoch":24,"log_start":"312'100","ondisk_log_start":"312'100","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'183","last_scrub_stamp":"2026-02-24T22:34:48.176839+0000","last_deep_scrub":"312'183","last_deep_scrub_stamp":"2026-02-24T22:34:48.176839+0000","last_clean_scrub_stamp":"2026-02-24T22:34:48.176839+0000","objects_scrubbed":24,"log_size":83,"log_dups_size":100,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:23:04.478635+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":662,"num_read_kb":557,"num_write":245,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"312'305","reported_seq":1081,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:35.042678+0000","last_change":"2026-02-24T22:34:35.042678+0000","last_active":"2026-02-24T22:34:35.042678+0000","last_peered":"2026-02-24T22:34:35.042678+0000","last_clean":"2026-02-24T22:34:35.042678+0000","last_became_active":"2026-02-24T22:06:00.662656+0000","last_became_peered":"2026-02-24T22:06:00.662656+0000","last_unstale":"2026-02-24T22:34:35.042678+0000","last_undegraded":"2026-02-24T22:34:35.042678+0000","last_fullsized":"2026-02-24T22:34:35.042678+0000","mapping_epoch":36,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'305","last_scrub_stamp":"2026-02-24T22:34:35.042539+0000","last_deep_scrub":"312'305","last_deep_scrub_stamp":"2026-02-24T22:34:35.042539+0000","last_clean_scrub_stamp":"2026-02-24T22:34:35.042539+0000","objects_scrubbed":4,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:24:23.913211+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":803,"num_read_kb":6090,"num_write":1203,"num_write_kb":5556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.30","version":"312'347","reported_seq":1108,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:40.049909+0000","last_change":"2026-02-24T22:34:40.049845+0000","last_active":"2026-02-24T22:34:40.049909+0000","last_peered":"2026-02-24T22:34:40.049909+0000","last_clean":"2026-02-24T22:34:40.049909+0000","last_became_active":"2026-02-24T22:05:16.189713+0000","last_became_peered":"2026-02-24T22:05:16.189713+0000","last_unstale":"2026-02-24T22:34:40.049909+0000","last_undegraded":"2026-02-24T22:34:40.049909+0000","last_fullsized":"2026-02-24T22:34:40.049909+0000","mapping_epoch":22,"log_start":"312'300","ondisk_log_start":"312'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'347","last_scrub_stamp":"2026-02-24T22:34:40.049764+0000","last_deep_scrub":"312'347","last_deep_scrub_stamp":"2026-02-24T22:34:40.049764+0000","last_clean_scrub_stamp":"2026-02-24T22:34:40.049764+0000","objects_scrubbed":8,"log_size":47,"log_dups_size":300,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:52:46.777711+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242932,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":632,"num_read_kb":31410,"num_write":1455,"num_write_kb":10638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"312'273","reported_seq":967,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:46.873634+0000","last_change":"2026-02-24T22:34:46.873634+0000","last_active":"2026-02-24T22:34:46.873634+0000","last_peered":"2026-02-24T22:34:46.873634+0000","last_clean":"2026-02-24T22:34:46.873634+0000","last_became_active":"2026-02-24T22:05:18.187565+0000","last_became_peered":"2026-02-24T22:05:18.187565+0000","last_unstale":"2026-02-24T22:34:46.873634+0000","last_undegraded":"2026-02-24T22:34:46.873634+0000","last_fullsized":"2026-02-24T22:34:46.873634+0000","mapping_epoch":24,"log_start":"312'200","ondisk_log_start":"312'200","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'273","last_scrub_stamp":"2026-02-24T22:34:46.873575+0000","last_deep_scrub":"312'273","last_deep_scrub_stamp":"2026-02-24T22:34:46.873575+0000","last_clean_scrub_stamp":"2026-02-24T22:34:46.873575+0000","objects_scrubbed":17,"log_size":73,"log_dups_size":200,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T09:03:00.743450+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":1060,"num_read_kb":853,"num_write":419,"num_write_kb":222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"312'299","reported_seq":1014,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:37.069027+0000","last_change":"2026-02-24T22:34:37.068984+0000","last_active":"2026-02-24T22:34:37.069027+0000","last_peered":"2026-02-24T22:34:37.069027+0000","last_clean":"2026-02-24T22:34:37.069027+0000","last_became_active":"2026-02-24T22:05:16.178157+0000","last_became_peered":"2026-02-24T22:05:16.178157+0000","last_unstale":"2026-02-24T22:34:37.069027+0000","last_undegraded":"2026-02-24T22:34:37.069027+0000","last_fullsized":"2026-02-24T22:34:37.069027+0000","mapping_epoch":22,"log_start":"312'200","ondisk_log_start":"312'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'299","last_scrub_stamp":"2026-02-24T22:34:37.068917+0000","last_deep_scrub":"312'299","last_deep_scrub_stamp":"2026-02-24T22:34:37.068917+0000","last_clean_scrub_stamp":"2026-02-24T22:34:37.068917+0000","objects_scrubbed":5,"log_size":99,"log_dups_size":200,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:17:09.219950+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":466,"num_read_kb":1640,"num_write":1134,"num_write_kb":1702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"312'98","reported_seq":781,"reported_epoch":312,"state":"active+clean","last_fresh":"2026-02-24T22:34:47.871267+0000","last_change":"2026-02-24T22:34:47.871267+0000","last_active":"2026-02-24T22:34:47.871267+0000","last_peered":"2026-02-24T22:34:47.871267+0000","last_clean":"2026-02-24T22:34:47.871267+0000","last_became_active":"2026-02-24T22:05:18.190902+0000","last_became_peered":"2026-02-24T22:05:18.190902+0000","last_unstale":"2026-02-24T22:34:47.871267+0000","last_undegraded":"2026-02-24T22:34:47.871267+0000","last_fullsized":"2026-02-24T22:34:47.871267+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"312'98","last_scrub_stamp":"2026-02-24T22:34:47.871207+0000","last_deep_scrub":"312'98","last_deep_scrub_stamp":"2026-02-24T22:34:47.871207+0000","last_clean_scrub_stamp":"2026-02-24T22:34:47.871207+0000","objects_scrubbed":20,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T07:26:15.293776+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":302,"num_read_kb":262,"num_write":98,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":433,"ondisk_log_size":433,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":973109,"num_objects":3053,"num_object_clones":0,"num_object_copies":6106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3053,"num_whiteouts":0,"num_read":19978,"num_read_kb":245567,"num_write":23762,"num_write_kb":8619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":276106,"num_omap_keys":1511,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24911872,"data_stored":1946218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5786661,"internal_metadata":0},"log_size":2158,"ondisk_log_size":2158,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1250269,"num_objects":223,"num_object_clones":0,"num_object_copies":446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":223,"num_whiteouts":0,"num_read":45268,"num_read_kb":47987,"num_write":15451,"num_write_kb":13662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":962652,"num_omap_keys":4046,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":17220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2823344,"internal_metadata":0},"log_size":2038,"ondisk_log_size":2038,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1896,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1448,"num_read_kb":1680,"num_write":22,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":64,"acting":64,"num_store_stats":8},{"poolid":82,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1128,"num_omap_keys":12,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11005,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":1511,"num_object_clones":0,"num_object_copies":3022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1511,"num_whiteouts":0,"num_read":95919,"num_read_kb":81897,"num_write":34645,"num_write_kb":17078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1511,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":35257,"num_omap_keys":142,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17301930,"internal_metadata":0},"log_size":4333,"ondisk_log_size":4333,"up":128,"acting":128,"num_store_stats":8},{"poolid":81,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":119816,"num_write":82,"num_write_kb":70656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":112199743,"num_objects":281,"num_object_clones":0,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":0,"num_read":41102,"num_read_kb":763537,"num_write":86142,"num_write_kb":523059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":225296384,"data_stored":224399486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4246,"ondisk_log_size":4246,"up":128,"acting":128,"num_store_stats":8},{"poolid":80,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":124936,"num_write":82,"num_write_kb":70656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":7},{"poolid":79,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3567,"num_read_kb":2401,"num_write":1127,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":317720,"internal_metadata":0},"log_size":462,"ondisk_log_size":462,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14,"ondisk_log_size":14,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575219,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316704,"kb_used_data":55120,"kb_used_omap":3696,"kb_used_meta":257871,"kb_avail":94055136,"statfs":{"total":96636764160,"available":96312459264,"internally_reserved":0,"allocated":56442880,"data_stored":51634967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3785637,"internal_metadata":264059995},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575219,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":271964,"kb_used_data":28428,"kb_used_omap":3571,"kb_used_meta":239948,"kb_avail":94099876,"statfs":{"total":96636764160,"available":96358273024,"internally_reserved":0,"allocated":29110272,"data_stored":23825325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3657488,"internal_metadata":245706992},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575219,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137044,"kb_used_data":37956,"kb_used_omap":2746,"kb_used_meta":96325,"kb_avail":94234796,"statfs":{"total":96636764160,"available":96496431104,"internally_reserved":0,"allocated":38866944,"data_stored":35212820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2812671,"internal_metadata":98637057},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575219,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":111224,"kb_used_data":48168,"kb_used_omap":3095,"kb_used_meta":59944,"kb_avail":94260616,"statfs":{"total":96636764160,"available":96522870784,"internally_reserved":0,"allocated":49324032,"data_stored":45700457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3169290,"internal_metadata":61383670},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575220,"num_pgs":107,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":225820,"kb_used_data":18380,"kb_used_omap":7232,"kb_used_meta":200191,"kb_avail":94146020,"statfs":{"total":96636764160,"available":96405524480,"internally_reserved":0,"allocated":18821120,"data_stored":13350111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7406186,"internal_metadata":204995990},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575220,"num_pgs":87,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80488,"kb_used_data":17432,"kb_used_omap":1599,"kb_used_meta":61440,"kb_avail":94291352,"statfs":{"total":96636764160,"available":96554344448,"internally_reserved":0,"allocated":17850368,"data_stored":13280880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1638118,"internal_metadata":62914842},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575221,"num_pgs":102,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":129624,"kb_used_data":12424,"kb_used_omap":5094,"kb_used_meta":112089,"kb_avail":94242216,"statfs":{"total":96636764160,"available":96504029184,"internally_reserved":0,"allocated":12722176,"data_stored":8040949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5216565,"internal_metadata":114779851},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575221,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304420,"kb_used_data":60948,"kb_used_omap":10223,"kb_used_meta":233232,"kb_avail":94067420,"statfs":{"total":96636764160,"available":96325038080,"internally_reserved":0,"allocated":62410752,"data_stored":56934149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10468372,"internal_metadata":238830572},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53624832,"data_stored":53478090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5324800,"data_stored":5243270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":10555392,"data_stored":10486056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10620928,"data_stored":10486391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":43110400,"data_stored":42992165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":32641024,"data_stored":32506526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":21032960,"data_stored":20971830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":48386048,"data_stored":48235158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1667351,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2633902,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1516609,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1844562,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2430346,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":723382,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3546560,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2939218,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":2106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1512,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1072852,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":3356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18545,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":2058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5184,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1705285,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11530,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":2244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8436,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3928064,"data_stored":308322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51420,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3117056,"data_stored":241195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":152550,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3059712,"data_stored":239505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":825183,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3899392,"data_stored":305873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":336934,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1949696,"data_stored":151651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7338,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1925120,"data_stored":148450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":65762,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3837952,"data_stored":298046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2489145,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3194880,"data_stored":253176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1858329,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":79,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":133235,"internal_metadata":0},{"poolid":79,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14290,"internal_metadata":0},{"poolid":79,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23118,"internal_metadata":0},{"poolid":79,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15635,"internal_metadata":0},{"poolid":79,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49631,"internal_metadata":0},{"poolid":79,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49251,"internal_metadata":0},{"poolid":79,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16656,"internal_metadata":0},{"poolid":79,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15904,"internal_metadata":0},{"poolid":80,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":80,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":81,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":82,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":82,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1886,"internal_metadata":0},{"poolid":82,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6440,"internal_metadata":0},{"poolid":82,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":262,"internal_metadata":0},{"poolid":82,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1265,"internal_metadata":0},{"poolid":82,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":717,"internal_metadata":0},{"poolid":82,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":435,"internal_metadata":0},{"poolid":82,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T22:35:51.888 DEBUG:teuthology.orchestra.run.trial046:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-24T22:35:52.058 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-24T22:35:52.058 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-24T22:35:52.059 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-24T22:35:52.059 INFO:teuthology.orchestra.run:waiting for 300 2026-02-24T22:35:52.154 INFO:tasks.ceph.osd.0:Stopped 2026-02-24T22:35:52.154 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-24T22:35:52.154 INFO:teuthology.orchestra.run:waiting for 300 2026-02-24T22:35:52.243 INFO:tasks.ceph.osd.1:Stopped 2026-02-24T22:35:52.243 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-24T22:35:52.244 INFO:teuthology.orchestra.run:waiting for 300 2026-02-24T22:35:52.329 INFO:tasks.ceph.osd.2:Stopped 2026-02-24T22:35:52.329 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2026-02-24T22:35:52.329 INFO:teuthology.orchestra.run:waiting for 300 2026-02-24T22:35:52.425 INFO:tasks.ceph.osd.3:Stopped 2026-02-24T22:35:52.425 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2026-02-24T22:35:52.426 INFO:teuthology.orchestra.run:waiting for 300 2026-02-24T22:35:55.219 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2026-02-24T22:35:58.527 INFO:tasks.ceph.osd.4:Stopped 2026-02-24T22:35:58.527 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2026-02-24T22:35:58.527 INFO:teuthology.orchestra.run:waiting for 300 2026-02-24T22:35:58.607 INFO:tasks.ceph.osd.5:Stopped 2026-02-24T22:35:58.607 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2026-02-24T22:35:58.607 INFO:teuthology.orchestra.run:waiting for 300 2026-02-24T22:35:58.699 INFO:tasks.ceph.osd.6:Stopped 2026-02-24T22:35:58.699 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2026-02-24T22:35:58.699 INFO:teuthology.orchestra.run:waiting for 300 2026-02-24T22:36:00.723 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2026-02-24T22:36:00.724 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2026-02-24T22:36:04.801 INFO:tasks.ceph.osd.7:Stopped 2026-02-24T22:36:04.801 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-24T22:36:04.802 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2026-02-24T22:36:04.802 INFO:teuthology.orchestra.run:waiting for 300 2026-02-24T22:36:04.833 INFO:tasks.ceph.mgr.y:Stopped 2026-02-24T22:36:04.833 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-24T22:36:04.833 INFO:teuthology.orchestra.run:waiting for 300 2026-02-24T22:36:04.861 INFO:tasks.ceph.mgr.x:Stopped 2026-02-24T22:36:04.861 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-24T22:36:04.861 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-24T22:36:04.861 INFO:teuthology.orchestra.run:waiting for 300 2026-02-24T22:36:04.892 INFO:tasks.ceph.mon.a:Stopped 2026-02-24T22:36:04.892 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2026-02-24T22:36:04.893 INFO:teuthology.orchestra.run:waiting for 300 2026-02-24T22:36:04.949 INFO:tasks.ceph.mon.c:Stopped 2026-02-24T22:36:04.949 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2026-02-24T22:36:04.949 INFO:teuthology.orchestra.run:waiting for 300 2026-02-24T22:36:04.963 INFO:tasks.ceph.mon.b:Stopped 2026-02-24T22:36:04.964 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-24T22:36:04.964 DEBUG:teuthology.orchestra.run.trial046:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v '\(PG_AVAILABILITY\)' | grep -E -v '\(PG_DEGRADED\)' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | grep -E -v 'not have an application enabled' | grep -E -v '\(OSD_SLOW_PING_TIME' | grep -E -v 'reached quota' | grep -E -v POOL_FULL | grep -E -v 'pool\(s\) full' | head -n 1 2026-02-24T22:36:05.001 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial046.front.sepia.ceph.com 2026-02-24T22:36:05.002 DEBUG:teuthology.orchestra.run.trial046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-24T22:36:05.066 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial046.front.sepia.ceph.com 2026-02-24T22:36:05.066 DEBUG:teuthology.orchestra.run.trial046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-24T22:36:05.124 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial046.front.sepia.ceph.com 2026-02-24T22:36:05.124 DEBUG:teuthology.orchestra.run.trial046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-24T22:36:05.180 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@trial046.front.sepia.ceph.com 2026-02-24T22:36:05.180 DEBUG:teuthology.orchestra.run.trial046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2026-02-24T22:36:05.266 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@trial110.front.sepia.ceph.com 2026-02-24T22:36:05.266 DEBUG:teuthology.orchestra.run.trial110:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2026-02-24T22:36:05.289 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@trial110.front.sepia.ceph.com 2026-02-24T22:36:05.290 DEBUG:teuthology.orchestra.run.trial110:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2026-02-24T22:36:05.348 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@trial110.front.sepia.ceph.com 2026-02-24T22:36:05.349 DEBUG:teuthology.orchestra.run.trial110:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2026-02-24T22:36:05.416 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@trial110.front.sepia.ceph.com 2026-02-24T22:36:05.416 DEBUG:teuthology.orchestra.run.trial110:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2026-02-24T22:36:05.483 INFO:tasks.ceph:Archiving mon data... 2026-02-24T22:36:05.485 DEBUG:teuthology.misc:Transferring archived files from trial046:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/suriarte-2026-02-24_21:43:07-rgw-wip-d4n-remove-bucket-distro-default-trial/68775/data/mon.a.tgz 2026-02-24T22:36:05.485 DEBUG:teuthology.orchestra.run.trial046:> mktemp 2026-02-24T22:36:05.490 INFO:teuthology.orchestra.run.trial046.stdout:/tmp/tmp.53lEbRLdgA 2026-02-24T22:36:05.490 DEBUG:teuthology.orchestra.run.trial046:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.53lEbRLdgA 2026-02-24T22:36:05.606 DEBUG:teuthology.orchestra.run.trial046:> sudo chmod 0666 /tmp/tmp.53lEbRLdgA 2026-02-24T22:36:05.664 DEBUG:teuthology.orchestra.remote:trial046:/tmp/tmp.53lEbRLdgA is 484KB 2026-02-24T22:36:05.724 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2026-02-24T22:36:05.725 DEBUG:teuthology.orchestra.run.trial046:> rm -fr /tmp/tmp.53lEbRLdgA 2026-02-24T22:36:05.729 DEBUG:teuthology.misc:Transferring archived files from trial046:/var/lib/ceph/mon/ceph-c to /home/teuthworker/mnt/teuthology/suriarte-2026-02-24_21:43:07-rgw-wip-d4n-remove-bucket-distro-default-trial/68775/data/mon.c.tgz 2026-02-24T22:36:05.730 DEBUG:teuthology.orchestra.run.trial046:> mktemp 2026-02-24T22:36:05.775 INFO:teuthology.orchestra.run.trial046.stdout:/tmp/tmp.HFqhbCInHs 2026-02-24T22:36:05.775 DEBUG:teuthology.orchestra.run.trial046:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.HFqhbCInHs 2026-02-24T22:36:05.889 DEBUG:teuthology.orchestra.run.trial046:> sudo chmod 0666 /tmp/tmp.HFqhbCInHs 2026-02-24T22:36:05.945 DEBUG:teuthology.orchestra.remote:trial046:/tmp/tmp.HFqhbCInHs is 489KB 2026-02-24T22:36:06.001 DEBUG:teuthology.orchestra.run.trial046:> rm -fr /tmp/tmp.HFqhbCInHs 2026-02-24T22:36:06.005 DEBUG:teuthology.misc:Transferring archived files from trial110:/var/lib/ceph/mon/ceph-b to /home/teuthworker/mnt/teuthology/suriarte-2026-02-24_21:43:07-rgw-wip-d4n-remove-bucket-distro-default-trial/68775/data/mon.b.tgz 2026-02-24T22:36:06.005 DEBUG:teuthology.orchestra.run.trial110:> mktemp 2026-02-24T22:36:06.010 INFO:teuthology.orchestra.run.trial110.stdout:/tmp/tmp.iVugZjezvV 2026-02-24T22:36:06.011 DEBUG:teuthology.orchestra.run.trial110:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.iVugZjezvV 2026-02-24T22:36:06.128 DEBUG:teuthology.orchestra.run.trial110:> sudo chmod 0666 /tmp/tmp.iVugZjezvV 2026-02-24T22:36:06.187 DEBUG:teuthology.orchestra.remote:trial110:/tmp/tmp.iVugZjezvV is 489KB 2026-02-24T22:36:06.243 DEBUG:teuthology.orchestra.run.trial110:> rm -fr /tmp/tmp.iVugZjezvV 2026-02-24T22:36:06.247 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-24T22:36:06.247 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-24T22:36:06.249 DEBUG:teuthology.orchestra.run.trial110:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-24T22:36:06.303 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-24T22:36:06.345 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-24T22:36:06.345 INFO:tasks.ceph:Archiving crash dumps... 2026-02-24T22:36:06.347 DEBUG:teuthology.misc:Transferring archived files from trial046:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/suriarte-2026-02-24_21:43:07-rgw-wip-d4n-remove-bucket-distro-default-trial/68775/remote/trial046/crash 2026-02-24T22:36:06.347 DEBUG:teuthology.orchestra.run.trial046:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-24T22:36:06.358 DEBUG:teuthology.misc:Transferring archived files from trial110:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/suriarte-2026-02-24_21:43:07-rgw-wip-d4n-remove-bucket-distro-default-trial/68775/remote/trial110/crash 2026-02-24T22:36:06.359 DEBUG:teuthology.orchestra.run.trial110:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-24T22:36:06.393 INFO:tasks.ceph:Compressing logs... 2026-02-24T22:36:06.394 DEBUG:teuthology.orchestra.run.trial046:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-24T22:36:06.404 DEBUG:teuthology.orchestra.run.trial110:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-24T22:36:06.412 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56242.log 2026-02-24T22:36:06.415 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23041.log 2026-02-24T22:36:06.415 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.56242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36937.log 2026-02-24T22:36:06.415 INFO:teuthology.orchestra.run.trial046.stderr: 78.1% -- replaced with /var/log/ceph/ceph-client.admin.56242.log.gz 2026-02-24T22:36:06.415 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.23041.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23041.log.gz 2026-02-24T22:36:06.415 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21875.log 2026-02-24T22:36:06.415 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55059.log 2026-02-24T22:36:06.415 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.36937.log: /var/log/ceph/ceph-client.admin.21875.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21875.log.gz 2026-02-24T22:36:06.415 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44174.log 2026-02-24T22:36:06.416 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.55059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48673.log 2026-02-24T22:36:06.416 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.44174.log: 74.3% -- replaced with /var/log/ceph/ceph-client.admin.55059.log.gz 2026-02-24T22:36:06.416 INFO:teuthology.orchestra.run.trial046.stderr: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.44174.log.gz 2026-02-24T22:36:06.416 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48540.log 2026-02-24T22:36:06.416 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39456.log 2026-02-24T22:36:06.416 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.48673.log: /var/log/ceph/ceph-client.admin.48540.log: 74.1% -- replaced with /var/log/ceph/ceph-client.admin.48673.log.gz 2026-02-24T22:36:06.416 INFO:teuthology.orchestra.run.trial046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48540.log.gz 2026-02-24T22:36:06.416 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40873.log 2026-02-24T22:36:06.416 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.39456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21282.log 2026-02-24T22:36:06.427 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.40873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25593.log 2026-02-24T22:36:06.427 INFO:teuthology.orchestra.run.trial046.stderr: 74.3% -- replaced with /var/log/ceph/ceph-client.admin.40873.log.gz 2026-02-24T22:36:06.428 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.21282.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21282.log.gz 2026-02-24T22:36:06.428 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59495.log 2026-02-24T22:36:06.428 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.25593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32321.log 2026-02-24T22:36:06.428 INFO:teuthology.orchestra.run.trial046.stderr: 78.8% -- replaced with /var/log/ceph/ceph-client.admin.25593.log.gz 2026-02-24T22:36:06.428 INFO:teuthology.orchestra.run.trial046.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.admin.36937.log.gz 2026-02-24T22:36:06.428 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.59495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40005.log 2026-02-24T22:36:06.428 INFO:teuthology.orchestra.run.trial046.stderr: 83.2% -- replaced with /var/log/ceph/ceph-client.0.59495.log.gz 2026-02-24T22:36:06.428 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.32321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41354.log 2026-02-24T22:36:06.428 INFO:teuthology.orchestra.run.trial046.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.0.39456.log.gz 2026-02-24T22:36:06.428 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.40005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64559.log 2026-02-24T22:36:06.428 INFO:teuthology.orchestra.run.trial046.stderr: 78.1% -- replaced with /var/log/ceph/ceph-client.admin.40005.log.gz 2026-02-24T22:36:06.429 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.41354.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41354.log.gz 2026-02-24T22:36:06.429 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ops-log-ceph-client.admin.log 2026-02-24T22:36:06.429 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28703.log 2026-02-24T22:36:06.429 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.64559.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64559.log.gz 2026-02-24T22:36:06.429 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ops-log-ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45982.log 2026-02-24T22:36:06.429 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.28703.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.28703.log.gz 2026-02-24T22:36:06.429 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62704.log 2026-02-24T22:36:06.429 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47126.log 2026-02-24T22:36:06.429 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.45982.log: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.32321.log.gz 2026-02-24T22:36:06.429 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.62704.log: 78.7% -- replaced with /var/log/ceph/ceph-client.admin.45982.log.gz 2026-02-24T22:36:06.429 INFO:teuthology.orchestra.run.trial046.stderr: 77.8% -- replaced with /var/log/ceph/ceph-client.0.62704.log.gz 2026-02-24T22:36:06.429 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41487.log 2026-02-24T22:36:06.429 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.47126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53348.log 2026-02-24T22:36:06.430 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43380.log 2026-02-24T22:36:06.430 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.41487.log: /var/log/ceph/ceph-client.admin.53348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64611.log 2026-02-24T22:36:06.430 INFO:teuthology.orchestra.run.trial046.stderr: 74.4% -- replaced with /var/log/ceph/ceph-client.admin.41487.log.gz 2026-02-24T22:36:06.430 INFO:teuthology.orchestra.run.trial046.stderr: 74.6% -- replaced with /var/log/ceph/ceph-client.admin.53348.log.gz 2026-02-24T22:36:06.430 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.43380.log: 94.7% -- replaced with /var/log/ceph/ops-log-ceph-client.admin.log.gz 2026-02-24T22:36:06.430 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17614.log 2026-02-24T22:36:06.430 INFO:teuthology.orchestra.run.trial046.stderr: 74.4% -- replaced with /var/log/ceph/ceph-client.admin.43380.log.gz 2026-02-24T22:36:06.430 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58900.log 2026-02-24T22:36:06.430 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.64611.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64611.log.gz 2026-02-24T22:36:06.430 INFO:teuthology.orchestra.run.trial046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47126.log.gz 2026-02-24T22:36:06.430 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.17614.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17614.log.gz 2026-02-24T22:36:06.430 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42983.log 2026-02-24T22:36:06.431 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.58900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46200.log 2026-02-24T22:36:06.431 INFO:teuthology.orchestra.run.trial046.stderr: 83.1% -- replaced with /var/log/ceph/ceph-client.0.58900.log.gz 2026-02-24T22:36:06.431 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.42983.log: 72.8% -- replaced with /var/log/ceph/ceph-client.admin.42983.log.gz 2026-02-24T22:36:06.431 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26551.log 2026-02-24T22:36:06.431 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52289.log 2026-02-24T22:36:06.431 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.46200.log: /var/log/ceph/ceph-client.admin.26551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57312.log 2026-02-24T22:36:06.431 INFO:teuthology.orchestra.run.trial046.stderr: 74.0% -- replaced with /var/log/ceph/ceph-client.admin.46200.log.gz 2026-02-24T22:36:06.431 INFO:teuthology.orchestra.run.trial046.stderr: 78.4% -- replaced with /var/log/ceph/ceph-client.admin.26551.log.gz 2026-02-24T22:36:06.431 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.52289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59155.log 2026-02-24T22:36:06.431 INFO:teuthology.orchestra.run.trial046.stderr: 73.8% -- replaced with /var/log/ceph/ceph-client.admin.52289.log.gz 2026-02-24T22:36:06.431 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.57312.log: 71.3% -- replaced with /var/log/ceph/ceph-client.admin.57312.log.gz 2026-02-24T22:36:06.431 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21434.log 2026-02-24T22:36:06.431 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.59155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64351.log 2026-02-24T22:36:06.431 INFO:teuthology.orchestra.run.trial046.stderr: 82.4% -- replaced with /var/log/ceph/ceph-client.0.59155.log.gz 2026-02-24T22:36:06.432 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.21434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39788.log 2026-02-24T22:36:06.432 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21434.log.gz 2026-02-24T22:36:06.432 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.64351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53831.log 2026-02-24T22:36:06.432 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64351.log.gz 2026-02-24T22:36:06.432 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.39788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58815.log 2026-02-24T22:36:06.432 INFO:teuthology.orchestra.run.trial046.stderr: 78.1% -- replaced with /var/log/ceph/ceph-client.admin.39788.log.gz 2026-02-24T22:36:06.432 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.53831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53964.log 2026-02-24T22:36:06.432 INFO:teuthology.orchestra.run.trial046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53831.log.gz 2026-02-24T22:36:06.432 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.58815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64845.log 2026-02-24T22:36:06.432 INFO:teuthology.orchestra.run.trial046.stderr: 76.7% -- replaced with /var/log/ceph/ceph-client.0.58815.log.gz 2026-02-24T22:36:06.432 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.53964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29741.log 2026-02-24T22:36:06.432 INFO:teuthology.orchestra.run.trial046.stderr: 74.1% -- replaced with /var/log/ceph/ceph-client.admin.53964.log.gz 2026-02-24T22:36:06.432 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27638.log 2026-02-24T22:36:06.433 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.64845.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64845.log.gz 2026-02-24T22:36:06.433 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.29741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17517.log 2026-02-24T22:36:06.433 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.27638.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27638.log.gz 2026-02-24T22:36:06.433 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42148.log 2026-02-24T22:36:06.433 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.17517.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17517.log.gz 2026-02-24T22:36:06.433 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27227.log 2026-02-24T22:36:06.433 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45453.log 2026-02-24T22:36:06.433 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.42148.log: /var/log/ceph/ceph-client.admin.27227.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27227.log.gz 2026-02-24T22:36:06.433 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21381.log 2026-02-24T22:36:06.433 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41137.log 2026-02-24T22:36:06.433 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.45453.log: /var/log/ceph/ceph-client.admin.21381.log: 74.4% -- replaced with /var/log/ceph/ceph-client.admin.45453.log.gz 2026-02-24T22:36:06.433 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21381.log.gz 2026-02-24T22:36:06.433 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57711.log 2026-02-24T22:36:06.433 INFO:teuthology.orchestra.run.trial046.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.42148.log.gz 2026-02-24T22:36:06.434 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39058.log 2026-02-24T22:36:06.434 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.41137.log: gzip -5/var/log/ceph/ceph-client.admin.57711.log: --verbose -- /var/log/ceph/ceph-client.admin.47614.log 2026-02-24T22:36:06.434 INFO:teuthology.orchestra.run.trial046.stderr: 77.9% -- replaced with /var/log/ceph/ceph-client.admin.41137.log.gz 2026-02-24T22:36:06.457 INFO:teuthology.orchestra.run.trial110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2026-02-24T22:36:06.457 INFO:teuthology.orchestra.run.trial110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17473.log 2026-02-24T22:36:06.457 INFO:teuthology.orchestra.run.trial110.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2026-02-24T22:36:06.457 INFO:teuthology.orchestra.run.trial110.stderr:/var/log/ceph/ceph-client.admin.17473.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-24T22:36:06.457 INFO:teuthology.orchestra.run.trial110.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17473.log.gz 2026-02-24T22:36:06.457 INFO:teuthology.orchestra.run.trial110.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2026-02-24T22:36:06.457 INFO:teuthology.orchestra.run.trial110.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17497.log 2026-02-24T22:36:06.458 INFO:teuthology.orchestra.run.trial110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17522.log 2026-02-24T22:36:06.458 INFO:teuthology.orchestra.run.trial110.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.admin.17497.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17497.log.gz 2026-02-24T22:36:06.458 INFO:teuthology.orchestra.run.trial110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17618.log 2026-02-24T22:36:06.458 INFO:teuthology.orchestra.run.trial110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17570.log 2026-02-24T22:36:06.458 INFO:teuthology.orchestra.run.trial110.stderr:/var/log/ceph/ceph-client.admin.17522.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17522.log.gz 2026-02-24T22:36:06.458 INFO:teuthology.orchestra.run.trial110.stderr:/var/log/ceph/ceph-client.admin.17618.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17618.log.gz 2026-02-24T22:36:06.458 INFO:teuthology.orchestra.run.trial110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2026-02-24T22:36:06.458 INFO:teuthology.orchestra.run.trial110.stderr:/var/log/ceph/ceph-client.admin.17570.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-24T22:36:06.458 INFO:teuthology.orchestra.run.trial110.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17570.log.gz 2026-02-24T22:36:06.458 INFO:teuthology.orchestra.run.trial110.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-24T22:36:06.458 INFO:teuthology.orchestra.run.trial110.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2026-02-24T22:36:06.458 INFO:teuthology.orchestra.run.trial110.stderr:gzip -5/var/log/ceph/ceph.log: --verbose -- /var/log/ceph/ceph-client.admin.17449.log 2026-02-24T22:36:06.458 INFO:teuthology.orchestra.run.trial110.stderr: 90.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-24T22:36:06.458 INFO:teuthology.orchestra.run.trial110.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17546.log 2026-02-24T22:36:06.458 INFO:teuthology.orchestra.run.trial110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17594.log 2026-02-24T22:36:06.458 INFO:teuthology.orchestra.run.trial110.stderr:/var/log/ceph/ceph-client.admin.17449.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17449.log.gz 2026-02-24T22:36:06.458 INFO:teuthology.orchestra.run.trial110.stderr:/var/log/ceph/ceph-client.admin.17546.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17546.log.gz 2026-02-24T22:36:06.458 INFO:teuthology.orchestra.run.trial110.stderr:/var/log/ceph/ceph-client.admin.17594.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17594.log.gz 2026-02-24T22:36:06.458 INFO:teuthology.orchestra.run.trial110.stderr: 87.6% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-24T22:36:06.461 INFO:teuthology.orchestra.run.trial046.stderr: 76.0% -- replaced with /var/log/ceph/ceph-client.admin.57711.log.gz 2026-02-24T22:36:06.461 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28434.log 2026-02-24T22:36:06.461 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.39058.log: 24.6% -- replaced with /var/log/ceph/ceph-client.admin.39058.log.gz 2026-02-24T22:36:06.461 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.47614.log: 74.5% -- replaced with /var/log/ceph/ceph-client.admin.47614.log.gz 2026-02-24T22:36:06.461 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43248.log 2026-02-24T22:36:06.461 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.28434.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2026-02-24T22:36:06.461 INFO:teuthology.orchestra.run.trial046.stderr: 57.0% -- replaced with /var/log/ceph/ceph-client.admin.28434.log.gz 2026-02-24T22:36:06.461 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30398.log 2026-02-24T22:36:06.461 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.43248.log: /var/log/ceph/ceph-osd.3.log: 74.0% -- replaced with /var/log/ceph/ceph-client.admin.43248.log.gz 2026-02-24T22:36:06.462 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56025.log 2026-02-24T22:36:06.462 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.30398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50530.log 2026-02-24T22:36:06.462 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.56025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21430.log 2026-02-24T22:36:06.462 INFO:teuthology.orchestra.run.trial046.stderr: 74.1% -- replaced with /var/log/ceph/ceph-client.admin.56025.log.gz 2026-02-24T22:36:06.462 INFO:teuthology.orchestra.run.trial046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30398.log.gz 2026-02-24T22:36:06.462 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.50530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37600.log 2026-02-24T22:36:06.462 INFO:teuthology.orchestra.run.trial046.stderr: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.50530.log.gz 2026-02-24T22:36:06.462 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.21430.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21430.log.gz 2026-02-24T22:36:06.462 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64585.log 2026-02-24T22:36:06.462 INFO:teuthology.orchestra.run.trial046.stderr: 94.3% -- replaced with /var/log/ceph/ceph-client.admin.29741.log.gz 2026-02-24T22:36:06.462 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.37600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57988.log 2026-02-24T22:36:06.462 INFO:teuthology.orchestra.run.trial046.stderr: 83.2% -- replaced with /var/log/ceph/ceph-client.admin.37600.log.gz 2026-02-24T22:36:06.462 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.64585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54228.log 2026-02-24T22:36:06.462 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64585.log.gz 2026-02-24T22:36:06.462 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.57988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27663.log 2026-02-24T22:36:06.462 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57988.log.gz 2026-02-24T22:36:06.462 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.54228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21432.log 2026-02-24T22:36:06.462 INFO:teuthology.orchestra.run.trial046.stderr: 74.1% -- replaced with /var/log/ceph/ceph-client.admin.54228.log.gz 2026-02-24T22:36:06.462 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.27663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57661.log 2026-02-24T22:36:06.462 INFO:teuthology.orchestra.run.trial046.stderr: 78.7% -- replaced with /var/log/ceph/ceph-client.admin.27663.log.gz 2026-02-24T22:36:06.462 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.21432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51410.log 2026-02-24T22:36:06.462 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21432.log.gz 2026-02-24T22:36:06.462 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.57661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40174.log 2026-02-24T22:36:06.463 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57661.log.gz 2026-02-24T22:36:06.463 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.51410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21303.log 2026-02-24T22:36:06.463 INFO:teuthology.orchestra.run.trial046.stderr: 74.3% -- replaced with /var/log/ceph/ceph-client.admin.51410.log.gz 2026-02-24T22:36:06.463 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.40174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38369.log 2026-02-24T22:36:06.463 INFO:teuthology.orchestra.run.trial046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40174.log.gz 2026-02-24T22:36:06.463 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.21303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29512.log 2026-02-24T22:36:06.463 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21303.log.gz 2026-02-24T22:36:06.463 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.38369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62789.log 2026-02-24T22:36:06.463 INFO:teuthology.orchestra.run.trial046.stderr: 83.2% -- replaced with /var/log/ceph/ceph-client.admin.38369.log.gz 2026-02-24T22:36:06.463 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42365.log 2026-02-24T22:36:06.463 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.29512.log: /var/log/ceph/ceph-client.0.62789.log: 78.1% -- replaced with /var/log/ceph/ceph-client.0.62789.log.gz 2026-02-24T22:36:06.463 INFO:teuthology.orchestra.run.trial046.stderr:gzip 93.6% -- replaced with /var/log/ceph/ceph-client.admin.29512.log.gz -5 2026-02-24T22:36:06.463 INFO:teuthology.orchestra.run.trial046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.57396.log 2026-02-24T22:36:06.463 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.42365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64767.log 2026-02-24T22:36:06.463 INFO:teuthology.orchestra.run.trial046.stderr: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.42365.log.gz 2026-02-24T22:36:06.463 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.57396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49339.log 2026-02-24T22:36:06.463 INFO:teuthology.orchestra.run.trial046.stderr: 74.2% -- replaced with /var/log/ceph/ceph-client.admin.57396.log.gz 2026-02-24T22:36:06.463 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.64767.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64767.log.gz 2026-02-24T22:36:06.463 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48143.log 2026-02-24T22:36:06.463 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57528.log 2026-02-24T22:36:06.463 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.49339.log: /var/log/ceph/ceph-client.admin.48143.log: 74.2% -- replaced with /var/log/ceph/ceph-client.admin.49339.log.gz 2026-02-24T22:36:06.463 INFO:teuthology.orchestra.run.trial046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48143.log.gz 2026-02-24T22:36:06.463 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50748.log 2026-02-24T22:36:06.463 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.57528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55760.log 2026-02-24T22:36:06.463 INFO:teuthology.orchestra.run.trial046.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.57528.log.gz 2026-02-24T22:36:06.464 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.50748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46994.log 2026-02-24T22:36:06.464 INFO:teuthology.orchestra.run.trial046.stderr: 74.5% -- replaced with /var/log/ceph/ceph-client.admin.50748.log.gz 2026-02-24T22:36:06.464 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.55760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40571.log 2026-02-24T22:36:06.464 INFO:teuthology.orchestra.run.trial046.stderr: 74.3% -- replaced with /var/log/ceph/ceph-client.admin.55760.log.gz 2026-02-24T22:36:06.464 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.46994.log: 74.3% -- replaced with /var/log/ceph/ceph-client.admin.46994.log.gz 2026-02-24T22:36:06.464 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47481.log 2026-02-24T22:36:06.464 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.40571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31007.log 2026-02-24T22:36:06.464 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.47481.log: 78.5% -- replaced with /var/log/ceph/ceph-client.admin.40571.log.gz 2026-02-24T22:36:06.464 INFO:teuthology.orchestra.run.trial046.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.47481.log.gz 2026-02-24T22:36:06.464 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25726.log 2026-02-24T22:36:06.464 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41884.log 2026-02-24T22:36:06.464 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.31007.log: /var/log/ceph/ceph-client.admin.25726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49206.log 2026-02-24T22:36:06.464 INFO:teuthology.orchestra.run.trial046.stderr: 74.2% -- replaced with /var/log/ceph/ceph-client.admin.25726.log.gz 2026-02-24T22:36:06.464 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.41884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27529.log 2026-02-24T22:36:06.464 INFO:teuthology.orchestra.run.trial046.stderr: 74.0% -- replaced with /var/log/ceph/ceph-client.admin.41884.log.gz 2026-02-24T22:36:06.464 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.49206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39387.log 2026-02-24T22:36:06.464 INFO:teuthology.orchestra.run.trial046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49206.log.gz 2026-02-24T22:36:06.464 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.27529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38637.log 2026-02-24T22:36:06.464 INFO:teuthology.orchestra.run.trial046.stderr: 69.9% -- replaced with /var/log/ceph/ceph-client.admin.27529.log.gz 2026-02-24T22:36:06.464 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.39387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64533.log 2026-02-24T22:36:06.464 INFO:teuthology.orchestra.run.trial046.stderr: 49.0% -- replaced with /var/log/ceph/ceph-client.admin.39387.log.gz 2026-02-24T22:36:06.464 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.38637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45717.log 2026-02-24T22:36:06.464 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.64533.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64533.log.gz 2026-02-24T22:36:06.464 INFO:teuthology.orchestra.run.trial046.stderr: 79.8% -- replaced with /var/log/ceph/ceph-client.admin.38637.log.gz 2026-02-24T22:36:06.464 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42233.log 2026-02-24T22:36:06.465 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45320.log 2026-02-24T22:36:06.465 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.45717.log: /var/log/ceph/ceph-client.admin.42233.log: gzip 74.6% -- replaced with /var/log/ceph/ceph-client.admin.42233.log.gz 2026-02-24T22:36:06.465 INFO:teuthology.orchestra.run.trial046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.28745.log 2026-02-24T22:36:06.465 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.45320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52024.log 2026-02-24T22:36:06.465 INFO:teuthology.orchestra.run.trial046.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.45320.log.gz 2026-02-24T22:36:06.465 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.28745.log: 94.4% -- replaced with /var/log/ceph/ceph-client.admin.31007.log.gz 2026-02-24T22:36:06.465 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28745.log.gz 2026-02-24T22:36:06.465 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39242.log 2026-02-24T22:36:06.465 INFO:teuthology.orchestra.run.trial046.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.45717.log.gz 2026-02-24T22:36:06.465 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.52024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55892.log 2026-02-24T22:36:06.465 INFO:teuthology.orchestra.run.trial046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52024.log.gz 2026-02-24T22:36:06.465 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.39242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25303.log 2026-02-24T22:36:06.465 INFO:teuthology.orchestra.run.trial046.stderr: 83.1% -- replaced with /var/log/ceph/ceph-client.0.39242.log.gz 2026-02-24T22:36:06.465 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.55892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53613.log 2026-02-24T22:36:06.465 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.25303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21649.log 2026-02-24T22:36:06.465 INFO:teuthology.orchestra.run.trial046.stderr: 76.2% -- replaced with /var/log/ceph/ceph-client.admin.25303.log.gz 2026-02-24T22:36:06.465 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.53613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26163.log 2026-02-24T22:36:06.465 INFO:teuthology.orchestra.run.trial046.stderr: 73.6% -- replaced with /var/log/ceph/ceph-client.admin.53613.log.gz 2026-02-24T22:36:06.465 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.21649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.23283.log 2026-02-24T22:36:06.465 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21649.log.gz 2026-02-24T22:36:06.465 INFO:teuthology.orchestra.run.trial046.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.55892.log.gz 2026-02-24T22:36:06.465 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.26163.log: 70.2% -- replaced with /var/log/ceph/ceph-client.admin.26163.log.gz 2026-02-24T22:36:06.465 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.23533.log 2026-02-24T22:36:06.465 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.23283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27058.log 2026-02-24T22:36:06.466 INFO:teuthology.orchestra.run.trial046.stderr: 68.5% -- replaced with /var/log/ceph/ceph-client.0.23283.log.gz 2026-02-24T22:36:06.466 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.23533.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.12577.log 2026-02-24T22:36:06.466 INFO:teuthology.orchestra.run.trial046.stderr: 44.8% -- replaced with /var/log/ceph/ceph-client.0.23533.log.gz 2026-02-24T22:36:06.466 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58730.log 2026-02-24T22:36:06.466 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.27058.log: /var/log/ceph/ceph.tmp-client.admin.12577.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.12577.log.gz 2026-02-24T22:36:06.466 INFO:teuthology.orchestra.run.trial046.stderr: 71.2% -- replaced with /var/log/ceph/ceph-client.admin.27058.log.gz 2026-02-24T22:36:06.466 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64689.log 2026-02-24T22:36:06.466 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54625.log 2026-02-24T22:36:06.466 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.58730.log: /var/log/ceph/ceph-client.admin.64689.log: 83.0% -- replaced with /var/log/ceph/ceph-client.0.58730.log.gz 2026-02-24T22:36:06.466 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64689.log.gz 2026-02-24T22:36:06.466 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26974.log 2026-02-24T22:36:06.466 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.54625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49070.log 2026-02-24T22:36:06.466 INFO:teuthology.orchestra.run.trial046.stderr: 80.3% -- replaced with /var/log/ceph/ceph-client.admin.54625.log.gz 2026-02-24T22:36:06.466 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.26974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51542.log 2026-02-24T22:36:06.466 INFO:teuthology.orchestra.run.trial046.stderr: 71.0% -- replaced with /var/log/ceph/ceph-client.admin.26974.log.gz 2026-02-24T22:36:06.466 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.49070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43909.log 2026-02-24T22:36:06.466 INFO:teuthology.orchestra.run.trial046.stderr: 74.1% -- replaced with /var/log/ceph/ceph-client.admin.49070.log.gz 2026-02-24T22:36:06.466 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.51542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23091.log 2026-02-24T22:36:06.466 INFO:teuthology.orchestra.run.trial046.stderr: 74.2% -- replaced with /var/log/ceph/ceph-client.admin.51542.log.gz 2026-02-24T22:36:06.466 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.43909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34966.log 2026-02-24T22:36:06.466 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.23091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56989.log 2026-02-24T22:36:06.466 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23091.log.gz 2026-02-24T22:36:06.466 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.34966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55456.log 2026-02-24T22:36:06.466 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.56989.log: 70.7% -- replaced with /var/log/ceph/ceph-client.admin.56989.log.gz 2026-02-24T22:36:06.467 INFO:teuthology.orchestra.run.trial046.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.43909.log.gz 2026-02-24T22:36:06.467 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59410.log 2026-02-24T22:36:06.467 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.55456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26636.log 2026-02-24T22:36:06.467 INFO:teuthology.orchestra.run.trial046.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.55456.log.gz 2026-02-24T22:36:06.467 INFO:teuthology.orchestra.run.trial046.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.34966.log.gz 2026-02-24T22:36:06.467 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.59410.log: 82.5% -- replaced with /var/log/ceph/ceph-client.0.59410.log.gz 2026-02-24T22:36:06.467 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58473.log 2026-02-24T22:36:06.467 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.26636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56857.log 2026-02-24T22:36:06.467 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.58473.log: 70.5% -- replaced with /var/log/ceph/ceph-client.admin.26636.log.gz 2026-02-24T22:36:06.467 INFO:teuthology.orchestra.run.trial046.stderr: 77.1% -- replaced with /var/log/ceph/ceph-client.0.58473.log.gz 2026-02-24T22:36:06.467 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51012.log 2026-02-24T22:36:06.467 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26079.log 2026-02-24T22:36:06.467 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.56857.log: /var/log/ceph/ceph-client.admin.51012.log: 73.8% -- replaced with /var/log/ceph/ceph-client.admin.56857.log.gz 2026-02-24T22:36:06.467 INFO:teuthology.orchestra.run.trial046.stderr: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.51012.log.gz 2026-02-24T22:36:06.467 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23015.log 2026-02-24T22:36:06.467 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-24T22:36:06.467 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.26079.log: /var/log/ceph/ceph-client.admin.23015.log: 71.0% -- replaced with /var/log/ceph/ceph-client.admin.26079.log.gz 2026-02-24T22:36:06.467 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23015.log.gz 2026-02-24T22:36:06.467 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31664.log 2026-02-24T22:36:06.467 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54493.log 2026-02-24T22:36:06.467 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.31664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48276.log 2026-02-24T22:36:06.467 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.54493.log: 73.5% -- replaced with /var/log/ceph/ceph-client.admin.54493.log.gz 2026-02-24T22:36:06.467 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63044.log 2026-02-24T22:36:06.467 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.48276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39030.log 2026-02-24T22:36:06.467 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.63044.log: 74.3% -- replaced with /var/log/ceph/ceph-client.admin.48276.log.gz 2026-02-24T22:36:06.467 INFO:teuthology.orchestra.run.trial046.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.0.63044.log.gz 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47259.log 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.39030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62874.log 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39030.log.gz 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.47259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41619.log 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr: 73.6% -- replaced with /var/log/ceph/ceph-client.admin.47259.log.gz 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.62874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46332.log 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr: 77.5% -- replaced with /var/log/ceph/ceph-client.0.62874.log.gz 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.41619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63262.log 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr: 74.1% -- replaced with /var/log/ceph/ceph-client.admin.41619.log.gz 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.46332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58645.log 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr: 74.3% -- replaced with /var/log/ceph/ceph-client.admin.46332.log.gz 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.admin.31664.log.gz 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.63262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27613.log 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr: 82.3% -- replaced with /var/log/ceph/ceph-client.0.63262.log.gz 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.58645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56460.log 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr: 76.2% -- replaced with /var/log/ceph/ceph-client.0.58645.log.gz 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.27613.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.27613.log.gz /var/log/ceph/ceph-client.admin.40740.log 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr: 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.56460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35623.log 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr: 74.0% -- replaced with /var/log/ceph/ceph-client.admin.56460.log.gz 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.40740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63177.log 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.35623.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40740.log.gz 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33645.log 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.63177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64325.log 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr: 76.6% -- replaced with /var/log/ceph/ceph-client.0.63177.log.gz 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.33645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29657.log 2026-02-24T22:36:06.468 INFO:teuthology.orchestra.run.trial046.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.35623.log.gz 2026-02-24T22:36:06.469 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.64325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64663.log 2026-02-24T22:36:06.469 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64325.log.gz 2026-02-24T22:36:06.469 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.29657.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.33645.log.gz 2026-02-24T22:36:06.469 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25860.log 2026-02-24T22:36:06.469 INFO:teuthology.orchestra.run.trial046.stderr: 83.0% -- replaced with /var/log/ceph/ceph-client.admin.29657.log.gz 2026-02-24T22:36:06.469 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.64663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44261.log 2026-02-24T22:36:06.469 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64663.log.gz 2026-02-24T22:36:06.469 INFO:teuthology.orchestra.run.trial110.stderr: 94.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-24T22:36:06.474 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.25860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40656.log 2026-02-24T22:36:06.474 INFO:teuthology.orchestra.run.trial046.stderr: 78.9% -- replaced with /var/log/ceph/ceph-client.admin.25860.log.gz 2026-02-24T22:36:06.474 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.44261.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44261.log.gz 2026-02-24T22:36:06.474 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ops-log-ceph-client.0.log 2026-02-24T22:36:06.474 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.40656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39080.log 2026-02-24T22:36:06.474 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ops-log-ceph-client.0.log: 68.4% -- replaced with /var/log/ceph/ceph-client.admin.40656.log.gz 2026-02-24T22:36:06.474 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26466.log 2026-02-24T22:36:06.474 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.39080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52157.log 2026-02-24T22:36:06.474 INFO:teuthology.orchestra.run.trial046.stderr: 23.9% -- replaced with /var/log/ceph/ceph-client.admin.39080.log.gz 2026-02-24T22:36:06.474 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.26466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.23449.log 2026-02-24T22:36:06.474 INFO:teuthology.orchestra.run.trial046.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.26466.log.gz 2026-02-24T22:36:06.474 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.52157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55191.log 2026-02-24T22:36:06.474 INFO:teuthology.orchestra.run.trial046.stderr: 74.3% -- replaced with /var/log/ceph/ceph-client.admin.52157.log.gz 2026-02-24T22:36:06.474 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.23449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58231.log 2026-02-24T22:36:06.474 INFO:teuthology.orchestra.run.trial046.stderr: 68.2% -- replaced with /var/log/ceph/ceph-client.0.23449.log.gz 2026-02-24T22:36:06.474 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.55191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39589.log 2026-02-24T22:36:06.474 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.58231.log: 74.2% -- replaced with /var/log/ceph/ceph-client.admin.58231.log.gz 2026-02-24T22:36:06.474 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49868.log 2026-02-24T22:36:06.474 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.39589.log: gzip 76.6% -5 -- replaced with /var/log/ceph/ceph-client.0.39589.log.gz --verbose 2026-02-24T22:36:06.474 INFO:teuthology.orchestra.run.trial046.stderr: -- /var/log/ceph/ceph-client.admin.43777.log 2026-02-24T22:36:06.474 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.49868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21618.log 2026-02-24T22:36:06.474 INFO:teuthology.orchestra.run.trial046.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.55191.log.gz 2026-02-24T22:36:06.474 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.43777.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49868.log.gz 2026-02-24T22:36:06.474 INFO:teuthology.orchestra.run.trial046.stderr: 74.3% -- replaced with /var/log/ceph/ceph-client.admin.43777.log.gz 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40439.log 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.21618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54710.log 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21618.log.gz 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.40439.log: 74.3% -- replaced with /var/log/ceph/ceph-client.admin.40439.log.gz 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57963.log 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29476.log 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.54710.log: /var/log/ceph/ceph-client.admin.57963.log: 74.1% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57963.log.gz -- replaced with /var/log/ceph/ceph-client.admin.54710.log.gz 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr: 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62959.log 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49603.log 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.29476.log: /var/log/ceph/ceph-client.0.62959.log: 19.9% -- replaced with /var/log/ceph/ceph-client.admin.29476.log.gz 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr: 76.6% -- replaced with /var/log/ceph/ceph-client.0.62959.log.gz 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40307.log 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62571.log 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.49603.log: /var/log/ceph/ceph-client.admin.40307.log: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.49603.log.gz 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47391.log 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr: 74.2% -- replaced with /var/log/ceph/ceph-client.admin.40307.log.gz 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.0.62571.log: /var/log/ceph/ceph-client.0.58560.log 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.47391.log: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.47391.log.gz 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr: 84.5% -- replaced with /var/log/ceph/ceph-client.0.62571.log.gz 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50265.log 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.58560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44658.log 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr: 83.3% -- replaced with /var/log/ceph/ceph-client.0.58560.log.gz 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.50265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45188.log 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.44658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41005.log 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.44658.log.gz 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.45188.log: 74.2% -- replaced with /var/log/ceph/ceph-client.admin.45188.log.gz 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42850.log 2026-02-24T22:36:06.475 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47746.log/var/log/ceph/ceph-client.admin.41005.log: 2026-02-24T22:36:06.476 INFO:teuthology.orchestra.run.trial046.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.50265.log.gz 2026-02-24T22:36:06.476 INFO:teuthology.orchestra.run.trial046.stderr: 74.1%/var/log/ceph/ceph-client.admin.42850.log: -- replaced with /var/log/ceph/ceph-client.admin.41005.log.gz 2026-02-24T22:36:06.476 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59240.log 2026-02-24T22:36:06.476 INFO:teuthology.orchestra.run.trial046.stderr: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.42850.log.gz 2026-02-24T22:36:06.476 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.47746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21433.log 2026-02-24T22:36:06.476 INFO:teuthology.orchestra.run.trial046.stderr: 74.0% -- replaced with /var/log/ceph/ceph-client.admin.47746.log.gz 2026-02-24T22:36:06.476 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.59240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29364.log 2026-02-24T22:36:06.476 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.21433.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21433.log.gz 2026-02-24T22:36:06.476 INFO:teuthology.orchestra.run.trial046.stderr: 83.6% -- replaced with /var/log/ceph/ceph-client.0.59240.log.gz 2026-02-24T22:36:06.476 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64507.log 2026-02-24T22:36:06.476 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57095.log 2026-02-24T22:36:06.476 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.29364.log: /var/log/ceph/ceph-client.admin.64507.log: 22.5% -- replaced with /var/log/ceph/ceph-client.admin.29364.log.gz 2026-02-24T22:36:06.476 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64507.log.gz 2026-02-24T22:36:06.476 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21329.log 2026-02-24T22:36:06.476 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.57095.log: 75.3% -- replaced with /var/log/ceph/ceph-client.admin.57095.log.gz 2026-02-24T22:36:06.476 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46464.log 2026-02-24T22:36:06.476 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.21329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39103.log 2026-02-24T22:36:06.476 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21329.log.gz 2026-02-24T22:36:06.476 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.46464.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.46464.log.gz 2026-02-24T22:36:06.476 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28797.log 2026-02-24T22:36:06.478 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.39103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49471.log 2026-02-24T22:36:06.478 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39103.log.gz 2026-02-24T22:36:06.478 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.28797.log: 54.2% -- replaced with /var/log/ceph/ceph-client.admin.28797.log.gz 2026-02-24T22:36:06.478 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52686.log 2026-02-24T22:36:06.478 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56725.log 2026-02-24T22:36:06.478 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.49471.log: /var/log/ceph/ceph-client.admin.52686.log: 73.9% -- replaced with /var/log/ceph/ceph-client.admin.49471.log.gz 2026-02-24T22:36:06.478 INFO:teuthology.orchestra.run.trial046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52686.log.gz 2026-02-24T22:36:06.478 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38286.log 2026-02-24T22:36:06.478 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50133.log/var/log/ceph/ceph-client.admin.56725.log: 2026-02-24T22:36:06.478 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.38286.log: 73.9% -- replaced with /var/log/ceph/ceph-client.admin.56725.log.gz 2026-02-24T22:36:06.478 INFO:teuthology.orchestra.run.trial046.stderr: 69.8% -- replaced with /var/log/ceph/ceph-client.admin.38286.log.gz 2026-02-24T22:36:06.478 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24666.log 2026-02-24T22:36:06.478 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.50133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26247.log 2026-02-24T22:36:06.478 INFO:teuthology.orchestra.run.trial046.stderr: 73.9% -- replaced with /var/log/ceph/ceph-client.admin.50133.log.gz 2026-02-24T22:36:06.478 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.24666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44526.log 2026-02-24T22:36:06.478 INFO:teuthology.orchestra.run.trial046.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.24666.log.gz 2026-02-24T22:36:06.478 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.26247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39005.log 2026-02-24T22:36:06.478 INFO:teuthology.orchestra.run.trial046.stderr: 70.3% -- replaced with /var/log/ceph/ceph-client.admin.26247.log.gz 2026-02-24T22:36:06.478 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64927.log 2026-02-24T22:36:06.478 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.44526.log: /var/log/ceph/ceph-client.admin.39005.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39005.log.gz 2026-02-24T22:36:06.478 INFO:teuthology.orchestra.run.trial046.stderr: 74.1% -- replaced with /var/log/ceph/ceph-client.admin.44526.log.gz 2026-02-24T22:36:06.478 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64481.log 2026-02-24T22:36:06.478 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.64927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51939.log 2026-02-24T22:36:06.478 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64927.log.gz 2026-02-24T22:36:06.479 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.64481.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64481.log.gz 2026-02-24T22:36:06.479 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63404.log 2026-02-24T22:36:06.479 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27277.log 2026-02-24T22:36:06.479 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.51939.log: /var/log/ceph/ceph-client.0.63404.log: 78.0% -- replaced with /var/log/ceph/ceph-client.admin.51939.log.gz 2026-02-24T22:36:06.479 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.23691.log 2026-02-24T22:36:06.479 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.23114.log 2026-02-24T22:36:06.479 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.27277.log: 70.6% -- replaced with /var/log/ceph/ceph-client.admin.27277.log.gz 2026-02-24T22:36:06.480 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.23691.log: 45.1% -- replaced with /var/log/ceph/ceph-client.0.23691.log.gz 2026-02-24T22:36:06.480 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34309.log 2026-02-24T22:36:06.480 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.23114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21661.log 2026-02-24T22:36:06.480 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.34309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45850.log 2026-02-24T22:36:06.480 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.21661.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21661.log.gz 2026-02-24T22:36:06.480 INFO:teuthology.orchestra.run.trial046.stderr: 95.6% -- replaced with /var/log/ceph/ceph-client.0.23114.log.gz 2026-02-24T22:36:06.481 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27252.log 2026-02-24T22:36:06.481 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.45850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48408.log 2026-02-24T22:36:06.481 INFO:teuthology.orchestra.run.trial046.stderr: 74.1% -- replaced with /var/log/ceph/ceph-client.admin.45850.log.gz 2026-02-24T22:36:06.481 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.27252.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27252.log.gz 2026-02-24T22:36:06.481 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-24T22:36:06.481 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.48408.log: /var/log/ceph/ceph-client.admin.21355.log 2026-02-24T22:36:06.482 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph.audit.log: 74.3% -- replaced with /var/log/ceph/ceph-client.admin.48408.log.gz 2026-02-24T22:36:06.482 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32978.log 2026-02-24T22:36:06.482 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.21355.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21355.log.gz 2026-02-24T22:36:06.482 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42586.log 2026-02-24T22:36:06.482 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.32978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25136.log 2026-02-24T22:36:06.483 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.42586.log: 73.6% -- replaced with /var/log/ceph/ceph-client.admin.42586.log.gz 2026-02-24T22:36:06.483 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48011.log 2026-02-24T22:36:06.483 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55324.log 2026-02-24T22:36:06.483 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.25136.log: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.32978.log.gz 2026-02-24T22:36:06.484 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24969.log 2026-02-24T22:36:06.484 INFO:teuthology.orchestra.run.trial046.stderr: 69.9% -- replaced with /var/log/ceph/ceph-client.admin.25136.log.gz 2026-02-24T22:36:06.484 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.48011.log: /var/log/ceph/ceph-client.admin.55324.log: 73.8% -- replaced with /var/log/ceph/ceph-client.admin.48011.log.gz 2026-02-24T22:36:06.484 INFO:teuthology.orchestra.run.trial046.stderr: 74.3% -- replaced with /var/log/ceph/ceph-client.admin.55324.log.gz 2026-02-24T22:36:06.484 INFO:teuthology.orchestra.run.trial046.stderr: 94.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-24T22:36:06.484 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50615.log 2026-02-24T22:36:06.484 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.24969.log: /var/log/ceph/ceph-osd.2.log 2026-02-24T22:36:06.484 INFO:teuthology.orchestra.run.trial046.stderr: 69.6% -- replaced with /var/log/ceph/ceph-client.admin.24969.log.gz 2026-02-24T22:36:06.484 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.50615.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.50615.log.gz 2026-02-24T22:36:06.485 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46729.log 2026-02-24T22:36:06.485 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21823.log 2026-02-24T22:36:06.485 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.46729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45056.log 2026-02-24T22:36:06.485 INFO:teuthology.orchestra.run.trial046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46729.log.gz 2026-02-24T22:36:06.486 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.21823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50001.log 2026-02-24T22:36:06.486 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21823.log.gz 2026-02-24T22:36:06.486 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.45056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42718.log 2026-02-24T22:36:06.486 INFO:teuthology.orchestra.run.trial046.stderr: 74.4% -- replaced with /var/log/ceph/ceph-client.admin.45056.log.gz 2026-02-24T22:36:06.486 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.50001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38467.log 2026-02-24T22:36:06.487 INFO:teuthology.orchestra.run.trial046.stderr: 74.0% -- replaced with /var/log/ceph/ceph-client.admin.50001.log.gz 2026-02-24T22:36:06.487 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.42718.log: 73.9% -- replaced with /var/log/ceph/ceph-client.admin.42718.log.gz 2026-02-24T22:36:06.487 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17565.log 2026-02-24T22:36:06.487 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.38467.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2026-02-24T22:36:06.488 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.17565.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.38467.log.gz 2026-02-24T22:36:06.488 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50880.log 2026-02-24T22:36:06.488 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17565.log.gz 2026-02-24T22:36:06.488 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26720.log 2026-02-24T22:36:06.488 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.50880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51807.log 2026-02-24T22:36:06.489 INFO:teuthology.orchestra.run.trial046.stderr: 74.4% -- replaced with /var/log/ceph/ceph-client.admin.50880.log.gz 2026-02-24T22:36:06.489 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.26720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21216.log 2026-02-24T22:36:06.489 INFO:teuthology.orchestra.run.trial046.stderr: 71.0% -- replaced with /var/log/ceph/ceph-client.admin.26720.log.gz 2026-02-24T22:36:06.489 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.51807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21423.log 2026-02-24T22:36:06.489 INFO:teuthology.orchestra.run.trial046.stderr: 74.4% -- replaced with /var/log/ceph/ceph-client.admin.51807.log.gz 2026-02-24T22:36:06.490 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.21216.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21216.log.gz 2026-02-24T22:36:06.490 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42016.log 2026-02-24T22:36:06.490 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-24T22:36:06.490 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.21423.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21423.log.gz 2026-02-24T22:36:06.490 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.42016.log: 74.1% -- replaced with /var/log/ceph/ceph-client.admin.42016.log.gz 2026-02-24T22:36:06.490 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41222.log 2026-02-24T22:36:06.491 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26889.log 2026-02-24T22:36:06.491 INFO:teuthology.orchestra.run.trial046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.0.63404.log.gz 2026-02-24T22:36:06.491 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.41222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53216.log 2026-02-24T22:36:06.491 INFO:teuthology.orchestra.run.trial046.stderr: 74.3% -- replaced with /var/log/ceph/ceph-client.admin.41222.log.gz 2026-02-24T22:36:06.491 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.26889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26804.log 2026-02-24T22:36:06.492 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.53216.log: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.26889.log.gz 2026-02-24T22:36:06.492 INFO:teuthology.orchestra.run.trial046.stderr: 74.1% -- replaced with /var/log/ceph/ceph-client.admin.53216.log.gz 2026-02-24T22:36:06.492 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54842.log 2026-02-24T22:36:06.492 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.26804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39873.log 2026-02-24T22:36:06.492 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.54842.log: 78.7% -- replaced with /var/log/ceph/ceph-client.admin.26804.log.gz 2026-02-24T22:36:06.492 INFO:teuthology.orchestra.run.trial046.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.54842.log.gz 2026-02-24T22:36:06.493 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26381.log 2026-02-24T22:36:06.493 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.39873.log: /var/log/ceph/ceph-client.admin.28515.log 2026-02-24T22:36:06.493 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.26381.log: 74.2% -- replaced with /var/log/ceph/ceph-client.admin.39873.log.gz 2026-02-24T22:36:06.493 INFO:teuthology.orchestra.run.trial046.stderr: 78.6% -- replaced with /var/log/ceph/ceph-client.admin.26381.log.gz 2026-02-24T22:36:06.493 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56157.log 2026-02-24T22:36:06.494 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.28515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25508.log 2026-02-24T22:36:06.494 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.56157.log: 51.2% -- replaced with /var/log/ceph/ceph-client.admin.28515.log.gz 2026-02-24T22:36:06.494 INFO:teuthology.orchestra.run.trial046.stderr: 80.2% -- replaced with /var/log/ceph/ceph-client.admin.56157.log.gz 2026-02-24T22:36:06.494 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27361.log 2026-02-24T22:36:06.494 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.25508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36280.log 2026-02-24T22:36:06.494 INFO:teuthology.orchestra.run.trial046.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.25508.log.gz 2026-02-24T22:36:06.495 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.27361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64299.log 2026-02-24T22:36:06.495 INFO:teuthology.orchestra.run.trial046.stderr: 71.0% -- replaced with /var/log/ceph/ceph-client.admin.27361.log.gz 2026-02-24T22:36:06.495 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.36280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64429.log 2026-02-24T22:36:06.495 INFO:teuthology.orchestra.run.trial046.stderr: 95.5% -- replaced with /var/log/ceph/ceph-client.admin.34309.log.gz 2026-02-24T22:36:06.496 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.64299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51277.log 2026-02-24T22:36:06.496 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64299.log.gz 2026-02-24T22:36:06.496 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.64429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46862.log 2026-02-24T22:36:06.496 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64429.log.gz 2026-02-24T22:36:06.496 INFO:teuthology.orchestra.run.trial046.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.36280.log.gz 2026-02-24T22:36:06.496 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.51277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42453.log 2026-02-24T22:36:06.497 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.46862.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51277.log.gz 2026-02-24T22:36:06.497 INFO:teuthology.orchestra.run.trial046.stderr: 74.2% -- replaced with /var/log/ceph/ceph-client.admin.46862.log.gz 2026-02-24T22:36:06.497 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59580.log 2026-02-24T22:36:06.497 INFO:teuthology.orchestra.run.trial046.stderr: 93.2% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-24T22:36:06.497 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.42453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25052.log 2026-02-24T22:36:06.497 INFO:teuthology.orchestra.run.trial046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42453.log.gz 2026-02-24T22:36:06.498 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.59580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47878.log 2026-02-24T22:36:06.498 INFO:teuthology.orchestra.run.trial046.stderr: 76.4% -- replaced with /var/log/ceph/ceph-client.0.59580.log.gz 2026-02-24T22:36:06.498 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.25052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64874.log 2026-02-24T22:36:06.498 INFO:teuthology.orchestra.run.trial046.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.25052.log.gz 2026-02-24T22:36:06.498 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.47878.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.47878.log.gz 2026-02-24T22:36:06.498 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64637.log 2026-02-24T22:36:06.499 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.64874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.23367.log 2026-02-24T22:36:06.499 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64874.log.gz 2026-02-24T22:36:06.499 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.64637.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64637.log.gz 2026-02-24T22:36:06.499 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45585.log 2026-02-24T22:36:06.499 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.23367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27796.log 2026-02-24T22:36:06.499 INFO:teuthology.orchestra.run.trial046.stderr: 45.5% -- replaced with /var/log/ceph/ceph-client.0.23367.log.gz 2026-02-24T22:36:06.500 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.45585.log: 74.1% -- replaced with /var/log/ceph/ceph-client.admin.45585.log.gz 2026-02-24T22:36:06.500 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53083.log 2026-02-24T22:36:06.500 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.27796.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.27796.log.gz 2026-02-24T22:36:06.500 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17939.log 2026-02-24T22:36:06.500 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.53083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53745.log 2026-02-24T22:36:06.501 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.17939.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53083.log.gz 2026-02-24T22:36:06.501 INFO:teuthology.orchestra.run.trial046.stderr:gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.17939.log.gz -- 2026-02-24T22:36:06.501 INFO:teuthology.orchestra.run.trial046.stderr: /var/log/ceph/ceph-client.admin.21435.log 2026-02-24T22:36:06.501 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.53745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27881.log 2026-02-24T22:36:06.501 INFO:teuthology.orchestra.run.trial046.stderr: 78.4% -- replaced with /var/log/ceph/ceph-client.admin.53745.log.gz 2026-02-24T22:36:06.501 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.21435.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21435.log.gz 2026-02-24T22:36:06.502 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52819.log 2026-02-24T22:36:06.502 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.27881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64377.log 2026-02-24T22:36:06.502 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.52819.log: 74.3% -- replaced with /var/log/ceph/ceph-client.admin.27881.log.gz 2026-02-24T22:36:06.502 INFO:teuthology.orchestra.run.trial046.stderr: 74.1% -- replaced with /var/log/ceph/ceph-client.admin.52819.log.gz 2026-02-24T22:36:06.502 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/rgw.ceph.client.0.log 2026-02-24T22:36:06.503 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.64377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38552.log 2026-02-24T22:36:06.503 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64377.log.gz 2026-02-24T22:36:06.503 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/rgw.ceph.client.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21664.log 2026-02-24T22:36:06.503 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27445.log 2026-02-24T22:36:06.504 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.38552.log: /var/log/ceph/ceph-client.admin.21664.log: gzip -5 69.9% --verbose -- replaced with /var/log/ceph/ceph-client.admin.38552.log.gz -- 2026-02-24T22:36:06.504 INFO:teuthology.orchestra.run.trial046.stderr: /var/log/ceph/ceph-client.admin.27142.log 2026-02-24T22:36:06.504 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21664.log.gz 2026-02-24T22:36:06.504 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.27445.log: 69.8% -- replaced with /var/log/ceph/ceph-client.admin.27445.log.gz 2026-02-24T22:36:06.504 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44791.log 2026-02-24T22:36:06.504 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.27142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29391.log 2026-02-24T22:36:06.505 INFO:teuthology.orchestra.run.trial046.stderr: 78.5% -- replaced with /var/log/ceph/ceph-client.admin.27142.log.gz 2026-02-24T22:36:06.505 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.44791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56592.log 2026-02-24T22:36:06.505 INFO:teuthology.orchestra.run.trial046.stderr: 74.0% -- replaced with /var/log/ceph/ceph-client.admin.44791.log.gz 2026-02-24T22:36:06.505 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.29391.log: 68.2% -- replaced with /var/log/ceph/ceph-client.admin.29391.log.gz 2026-02-24T22:36:06.505 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38771.log 2026-02-24T22:36:06.505 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.56592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58013.log 2026-02-24T22:36:06.505 INFO:teuthology.orchestra.run.trial046.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.56592.log.gz 2026-02-24T22:36:06.506 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.38771.log: 25.1% -- replaced with /var/log/ceph/ceph-client.admin.38771.log.gz 2026-02-24T22:36:06.506 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49736.log 2026-02-24T22:36:06.506 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39157.log 2026-02-24T22:36:06.506 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.58013.log: /var/log/ceph/ceph-client.admin.49736.log: 73.7% -- replaced with /var/log/ceph/ceph-client.admin.49736.log.gz 2026-02-24T22:36:06.506 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21427.log 2026-02-24T22:36:06.506 INFO:teuthology.orchestra.run.trial046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58013.log.gz 2026-02-24T22:36:06.507 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.39157.log: 84.7% -- replaced with /var/log/ceph/ceph-client.0.39157.log.gz 2026-02-24T22:36:06.507 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24763.log 2026-02-24T22:36:06.507 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.21427.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21427.log.gz 2026-02-24T22:36:06.507 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43115.log 2026-02-24T22:36:06.507 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25995.log 2026-02-24T22:36:06.507 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.24763.log: /var/log/ceph/ceph-client.admin.43115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23065.log 2026-02-24T22:36:06.508 INFO:teuthology.orchestra.run.trial046.stderr: 74.6% -- replaced with /var/log/ceph/ceph-client.admin.24763.log.gz 2026-02-24T22:36:06.508 INFO:teuthology.orchestra.run.trial046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43115.log.gz 2026-02-24T22:36:06.508 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.25995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44923.log 2026-02-24T22:36:06.508 INFO:teuthology.orchestra.run.trial046.stderr: 71.0% -- replaced with /var/log/ceph/ceph-client.admin.25995.log.gz 2026-02-24T22:36:06.508 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64455.log 2026-02-24T22:36:06.508 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.23065.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.23065.log.gz 2026-02-24T22:36:06.508 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.44923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44394.log 2026-02-24T22:36:06.509 INFO:teuthology.orchestra.run.trial046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44923.log.gz 2026-02-24T22:36:06.509 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.64455.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64455.log.gz 2026-02-24T22:36:06.509 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52951.log 2026-02-24T22:36:06.509 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.44394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28770.log 2026-02-24T22:36:06.509 INFO:teuthology.orchestra.run.trial046.stderr: 74.1% -- replaced with /var/log/ceph/ceph-client.admin.44394.log.gz 2026-02-24T22:36:06.509 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.52951.log: 73.8% -- replaced with /var/log/ceph/ceph-client.admin.52951.log.gz 2026-02-24T22:36:06.509 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-24T22:36:06.510 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.28770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43645.log 2026-02-24T22:36:06.510 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64403.log 2026-02-24T22:36:06.510 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.43645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21429.log 2026-02-24T22:36:06.511 INFO:teuthology.orchestra.run.trial046.stderr:gzip 74.2% -5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.43645.log.gz /var/log/ceph/ceph-client.admin.25970.log 2026-02-24T22:36:06.511 INFO:teuthology.orchestra.run.trial046.stderr: 2026-02-24T22:36:06.511 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.64403.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64403.log.gz 2026-02-24T22:36:06.511 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.21429.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21429.log.gz -5 2026-02-24T22:36:06.511 INFO:teuthology.orchestra.run.trial046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.21660.log 2026-02-24T22:36:06.511 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.25970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40090.log 2026-02-24T22:36:06.511 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25970.log.gz 2026-02-24T22:36:06.512 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.21660.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21660.log.gz 2026-02-24T22:36:06.512 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21245.log 2026-02-24T22:36:06.512 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.40090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41751.log 2026-02-24T22:36:06.512 INFO:teuthology.orchestra.run.trial046.stderr: 68.1% -- replaced with /var/log/ceph/ceph-client.admin.40090.log.gz 2026-02-24T22:36:06.512 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.21245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21620.log 2026-02-24T22:36:06.512 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21245.log.gz 2026-02-24T22:36:06.513 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.41751.log: gzip 84.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.41751.log.gz --verbose 2026-02-24T22:36:06.513 INFO:teuthology.orchestra.run.trial046.stderr: -- /var/log/ceph/ceph-client.admin.38920.log 2026-02-24T22:36:06.513 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.21620.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21620.log.gz 2026-02-24T22:36:06.513 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21626.log 2026-02-24T22:36:06.513 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.38920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21849.log 2026-02-24T22:36:06.514 INFO:teuthology.orchestra.run.trial046.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.38920.log.gz 2026-02-24T22:36:06.514 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.21626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58985.log 2026-02-24T22:36:06.514 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21626.log.gz 2026-02-24T22:36:06.514 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.21849.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21849.log.gz 2026-02-24T22:36:06.514 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48805.log 2026-02-24T22:36:06.515 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54096.log 2026-02-24T22:36:06.515 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.58985.log: /var/log/ceph/ceph-client.admin.48805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28822.log 2026-02-24T22:36:06.515 INFO:teuthology.orchestra.run.trial046.stderr: 76.2% -- replaced with /var/log/ceph/ceph-client.0.58985.log.gz 2026-02-24T22:36:06.515 INFO:teuthology.orchestra.run.trial046.stderr: 73.9% -- replaced with /var/log/ceph/ceph-client.admin.48805.log.gz 2026-02-24T22:36:06.515 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.54096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.23612.log 2026-02-24T22:36:06.515 INFO:teuthology.orchestra.run.trial046.stderr: 74.7% -- replaced with /var/log/ceph/ceph-client.admin.54096.log.gz 2026-02-24T22:36:06.515 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.28822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64951.log 2026-02-24T22:36:06.516 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.23612.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.28822.log.gz 2026-02-24T22:36:06.516 INFO:teuthology.orchestra.run.trial046.stderr: 45.5% -- replaced with /var/log/ceph/ceph-client.0.23612.log.gz 2026-02-24T22:36:06.516 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57179.log 2026-02-24T22:36:06.516 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25219.log 2026-02-24T22:36:06.516 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.64951.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64951.log.gz 2026-02-24T22:36:06.516 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.57179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25945.log 2026-02-24T22:36:06.517 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.25219.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.28770.log.gz 2026-02-24T22:36:06.517 INFO:teuthology.orchestra.run.trial046.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.57179.log.gz 2026-02-24T22:36:06.517 INFO:teuthology.orchestra.run.trial046.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.25219.log.gz 2026-02-24T22:36:06.517 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51674.log 2026-02-24T22:36:06.517 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28145.log 2026-02-24T22:36:06.517 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.25945.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25945.log.gz 2026-02-24T22:36:06.517 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.51674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64901.log 2026-02-24T22:36:06.518 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.28145.log: 58.3% -- replaced with /var/log/ceph/ceph-client.admin.28145.log.gz 2026-02-24T22:36:06.518 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52421.log 2026-02-24T22:36:06.518 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.64901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56375.log 0.0% 2026-02-24T22:36:06.518 INFO:teuthology.orchestra.run.trial046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64901.log.gz 2026-02-24T22:36:06.518 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.52421.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.52421.log.gz 2026-02-24T22:36:06.518 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26331.log 2026-02-24T22:36:06.518 INFO:teuthology.orchestra.run.trial046.stderr: 86.5%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.51674.log.gz --verbose 2026-02-24T22:36:06.518 INFO:teuthology.orchestra.run.trial046.stderr: -- /var/log/ceph/ceph-client.admin.52554.log 2026-02-24T22:36:06.518 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.56375.log: /var/log/ceph/ceph-client.admin.26331.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26331.log.gz 2026-02-24T22:36:06.519 INFO:teuthology.orchestra.run.trial046.stderr: 80.2% -- replaced with /var/log/ceph/ceph-client.admin.56375.log.gz 2026-02-24T22:36:06.519 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46067.log 2026-02-24T22:36:06.519 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57879.log 2026-02-24T22:36:06.519 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.52554.log: /var/log/ceph/ceph-client.admin.46067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54927.log 2026-02-24T22:36:06.519 INFO:teuthology.orchestra.run.trial046.stderr: 74.2% -- replaced with /var/log/ceph/ceph-client.admin.52554.log.gz 2026-02-24T22:36:06.519 INFO:teuthology.orchestra.run.trial046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46067.log.gz 2026-02-24T22:36:06.519 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-24T22:36:06.520 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.57879.log: /var/log/ceph/ceph-client.admin.54927.log: 74.4% -- replaced with /var/log/ceph/ceph-client.admin.54927.log.gz 2026-02-24T22:36:06.520 INFO:teuthology.orchestra.run.trial046.stderr: 75.9% -- replaced with /var/log/ceph/ceph-client.admin.57879.log.gz 2026-02-24T22:36:06.520 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64819.log 2026-02-24T22:36:06.520 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21627.log 2026-02-24T22:36:06.520 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.64819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46597.log 2026-02-24T22:36:06.520 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64819.log.gz 2026-02-24T22:36:06.521 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.21627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55674.log 2026-02-24T22:36:06.521 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.21627.log.gz 2026-02-24T22:36:06.521 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.46597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58388.log 2026-02-24T22:36:06.521 INFO:teuthology.orchestra.run.trial046.stderr: 73.9% -- replaced with /var/log/ceph/ceph-client.admin.46597.log.gz 2026-02-24T22:36:06.521 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.55674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64741.log 2026-02-24T22:36:06.522 INFO:teuthology.orchestra.run.trial046.stderr: 80.1% -- replaced with /var/log/ceph/ceph-client.admin.55674.log.gz 2026-02-24T22:36:06.522 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.58388.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2026-02-24T22:36:06.522 INFO:teuthology.orchestra.run.trial046.stderr: 83.3% -- replaced with /var/log/ceph/ceph-client.0.58388.log.gz 2026-02-24T22:36:06.522 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.64741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44042.log 2026-02-24T22:36:06.522 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64741.log.gz 2026-02-24T22:36:06.522 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64715.log 2026-02-24T22:36:06.523 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.44042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37968.log 2026-02-24T22:36:06.523 INFO:teuthology.orchestra.run.trial046.stderr: 74.3% -- replaced with /var/log/ceph/ceph-client.admin.44042.log.gz 2026-02-24T22:36:06.523 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.64715.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64715.log.gz 2026-02-24T22:36:06.523 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43512.log 2026-02-24T22:36:06.524 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50398.log 2026-02-24T22:36:06.524 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.37968.log: /var/log/ceph/ceph-client.admin.43512.log: 80.0% -- replaced with /var/log/ceph/ceph-client.admin.37968.log.gz 2026-02-24T22:36:06.524 INFO:teuthology.orchestra.run.trial046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43512.log.gz 2026-02-24T22:36:06.524 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59325.log 2026-02-24T22:36:06.524 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.50398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57795.log 2026-02-24T22:36:06.524 INFO:teuthology.orchestra.run.trial046.stderr: 74.5% -- replaced with /var/log/ceph/ceph-client.admin.50398.log.gz 2026-02-24T22:36:06.524 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54360.log 2026-02-24T22:36:06.525 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.59325.log: /var/log/ceph/ceph-client.admin.57795.log: 76.2% -- replaced with /var/log/ceph/ceph-client.0.59325.log.gz 2026-02-24T22:36:06.525 INFO:teuthology.orchestra.run.trial046.stderr: 76.3% -- replaced with /var/log/ceph/ceph-client.admin.57795.log.gz 2026-02-24T22:36:06.525 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26356.log 2026-02-24T22:36:06.525 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.54360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57686.log 2026-02-24T22:36:06.525 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.26356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.23770.log 2026-02-24T22:36:06.526 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26356.log.gz 2026-02-24T22:36:06.526 INFO:teuthology.orchestra.run.trial046.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.54360.log.gz 2026-02-24T22:36:06.526 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.57686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59070.log 2026-02-24T22:36:06.526 INFO:teuthology.orchestra.run.trial046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57686.log.gz 2026-02-24T22:36:06.526 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.23770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55541.log 2026-02-24T22:36:06.526 INFO:teuthology.orchestra.run.trial046.stderr: 44.8% -- replaced with /var/log/ceph/ceph-client.0.23770.log.gz 2026-02-24T22:36:06.526 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.59070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51145.log 2026-02-24T22:36:06.527 INFO:teuthology.orchestra.run.trial046.stderr: 77.1% -- replaced with /var/log/ceph/ceph-client.0.59070.log.gz 2026-02-24T22:36:06.527 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.55541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.23204.log 2026-02-24T22:36:06.527 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.51145.log: 74.1% -- replaced with /var/log/ceph/ceph-client.admin.51145.log.gz 2026-02-24T22:36:06.527 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64793.log 2026-02-24T22:36:06.528 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.0.23204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48937.log 2026-02-24T22:36:06.528 INFO:teuthology.orchestra.run.trial046.stderr: 44.8% -- replaced with /var/log/ceph/ceph-client.0.23204.log.gz 2026-02-24T22:36:06.528 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.64793.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64793.log.gz 2026-02-24T22:36:06.528 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53480.log 2026-02-24T22:36:06.528 INFO:teuthology.orchestra.run.trial046.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.55541.log.gz 2026-02-24T22:36:06.528 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.48937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58146.log 2026-02-24T22:36:06.529 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.53480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57073.log 2026-02-24T22:36:06.529 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.58146.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48937.log.gz 2026-02-24T22:36:06.529 INFO:teuthology.orchestra.run.trial046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58146.log.gz 2026-02-24T22:36:06.530 INFO:teuthology.orchestra.run.trial046.stderr:/var/log/ceph/ceph-client.admin.57073.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57073.log.gz 2026-02-24T22:36:06.530 INFO:teuthology.orchestra.run.trial046.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.53480.log.gz 2026-02-24T22:36:06.530 INFO:teuthology.orchestra.run.trial046.stderr: 95.4% -- replaced with /var/log/ceph/ops-log-ceph-client.0.log.gz 2026-02-24T22:36:06.668 INFO:teuthology.orchestra.run.trial046.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2026-02-24T22:36:06.735 INFO:teuthology.orchestra.run.trial110.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2026-02-24T22:36:06.768 INFO:teuthology.orchestra.run.trial046.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2026-02-24T22:36:07.195 INFO:teuthology.orchestra.run.trial046.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-24T22:36:08.280 INFO:teuthology.orchestra.run.trial046.stderr: 93.8% -- replaced with /var/log/ceph/rgw.ceph.client.0.log.gz 2026-02-24T22:36:09.426 INFO:teuthology.orchestra.run.trial046.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-24T22:36:10.562 INFO:teuthology.orchestra.run.trial110.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2026-02-24T22:36:10.938 INFO:teuthology.orchestra.run.trial110.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2026-02-24T22:36:11.528 INFO:teuthology.orchestra.run.trial046.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-24T22:36:12.196 INFO:teuthology.orchestra.run.trial110.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2026-02-24T22:36:12.387 INFO:teuthology.orchestra.run.trial046.stderr: 94.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2026-02-24T22:36:15.097 INFO:teuthology.orchestra.run.trial110.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2026-02-24T22:36:15.098 INFO:teuthology.orchestra.run.trial110.stderr: 2026-02-24T22:36:15.098 INFO:teuthology.orchestra.run.trial110.stderr:real 0m8.654s 2026-02-24T22:36:15.098 INFO:teuthology.orchestra.run.trial110.stderr:user 0m22.275s 2026-02-24T22:36:15.098 INFO:teuthology.orchestra.run.trial110.stderr:sys 0m1.033s 2026-02-24T22:36:17.186 INFO:teuthology.orchestra.run.trial046.stderr: 94.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-24T22:36:17.188 INFO:teuthology.orchestra.run.trial046.stderr: 2026-02-24T22:36:17.188 INFO:teuthology.orchestra.run.trial046.stderr:real 0m10.782s 2026-02-24T22:36:17.188 INFO:teuthology.orchestra.run.trial046.stderr:user 0m26.671s 2026-02-24T22:36:17.188 INFO:teuthology.orchestra.run.trial046.stderr:sys 0m1.321s 2026-02-24T22:36:17.189 INFO:tasks.ceph:Archiving logs... 2026-02-24T22:36:17.189 DEBUG:teuthology.misc:Transferring archived files from trial046:/var/log/ceph to /home/teuthworker/mnt/teuthology/suriarte-2026-02-24_21:43:07-rgw-wip-d4n-remove-bucket-distro-default-trial/68775/remote/trial046/log 2026-02-24T22:36:17.191 DEBUG:teuthology.orchestra.run.trial046:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-24T22:36:19.389 DEBUG:teuthology.misc:Transferring archived files from trial110:/var/log/ceph to /home/teuthworker/mnt/teuthology/suriarte-2026-02-24_21:43:07-rgw-wip-d4n-remove-bucket-distro-default-trial/68775/remote/trial110/log 2026-02-24T22:36:19.390 DEBUG:teuthology.orchestra.run.trial110:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-24T22:36:20.998 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-24T22:36:21.009 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-24T22:36:21.009 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-24T22:36:21.010 DEBUG:teuthology.orchestra.run.trial110:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-24T22:36:21.154 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-24T22:36:21.166 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-24T22:36:21.166 DEBUG:teuthology.orchestra.run.trial046:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-24T22:36:21.167 DEBUG:teuthology.orchestra.run.trial110:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-24T22:36:21.215 INFO:teuthology.orchestra.run.trial110.stdout: remote refid st t when poll reach delay offset jitter 2026-02-24T22:36:21.215 INFO:teuthology.orchestra.run.trial110.stdout:============================================================================== 2026-02-24T22:36:21.215 INFO:teuthology.orchestra.run.trial110.stdout:*time-h.den.code 132.163.97.5 2 u 61 128 377 39.944 +2.555 20.495 2026-02-24T22:36:21.215 INFO:teuthology.orchestra.run.trial110.stdout:+ntp.maxhost.io 132.163.97.4 2 u 127 128 377 69.698 +3.033 5.554 2026-02-24T22:36:21.215 INFO:teuthology.orchestra.run.trial110.stdout:+198.137.202.32 17.253.16.253 2 u 126 128 377 64.773 +1.613 1.348 2026-02-24T22:36:21.215 INFO:teuthology.orchestra.run.trial110.stdout:-white.web-ster. 51.93.121.47 4 u 47 128 377 72.111 -0.701 0.790 2026-02-24T22:36:21.215 INFO:teuthology.orchestra.run.trial046.stdout: remote refid st t when poll reach delay offset jitter 2026-02-24T22:36:21.215 INFO:teuthology.orchestra.run.trial046.stdout:============================================================================== 2026-02-24T22:36:21.215 INFO:teuthology.orchestra.run.trial046.stdout:+time-h.den.code 132.163.97.5 2 u 37 128 377 40.021 -24.965 15.964 2026-02-24T22:36:21.215 INFO:teuthology.orchestra.run.trial046.stdout:+ntp.maxhost.io 132.163.97.4 2 u 38 128 377 69.684 -11.118 3.299 2026-02-24T22:36:21.215 INFO:teuthology.orchestra.run.trial046.stdout:*198.137.202.32 17.253.16.253 2 u 50 128 377 64.752 -16.015 3.375 2026-02-24T22:36:21.215 INFO:teuthology.orchestra.run.trial046.stdout:-white.web-ster. 51.93.121.47 4 u 57 128 377 72.278 -17.476 5.760 2026-02-24T22:36:21.216 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-24T22:36:21.226 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-24T22:36:21.226 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-24T22:36:21.239 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-24T22:36:21.253 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-24T22:36:21.267 INFO:teuthology.task.internal:Duration was 2620.586658 seconds 2026-02-24T22:36:21.267 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-24T22:36:21.280 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-24T22:36:21.280 DEBUG:teuthology.orchestra.run.trial046:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-24T22:36:21.281 DEBUG:teuthology.orchestra.run.trial110:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-24T22:36:21.307 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-24T22:36:21.307 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial046.front.sepia.ceph.com 2026-02-24T22:36:21.307 DEBUG:teuthology.orchestra.run.trial046:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-24T22:36:21.352 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial110.front.sepia.ceph.com 2026-02-24T22:36:21.352 DEBUG:teuthology.orchestra.run.trial110:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-24T22:36:21.358 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-24T22:36:21.358 DEBUG:teuthology.orchestra.run.trial046:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-24T22:36:21.400 DEBUG:teuthology.orchestra.run.trial110:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-24T22:36:21.630 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-24T22:36:21.630 DEBUG:teuthology.orchestra.run.trial046:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-24T22:36:21.631 DEBUG:teuthology.orchestra.run.trial110:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-24T22:36:21.638 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T22:36:21.639 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-24T22:36:21.639 INFO:teuthology.orchestra.run.trial046.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-24T22:36:21.639 INFO:teuthology.orchestra.run.trial046.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T22:36:21.640 INFO:teuthology.orchestra.run.trial110.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T22:36:21.640 INFO:teuthology.orchestra.run.trial046.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-24T22:36:21.640 INFO:teuthology.orchestra.run.trial110.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-24T22:36:21.640 INFO:teuthology.orchestra.run.trial110.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-24T22:36:21.640 INFO:teuthology.orchestra.run.trial110.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T22:36:21.641 INFO:teuthology.orchestra.run.trial110.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-24T22:36:21.708 INFO:teuthology.orchestra.run.trial046.stderr: 81.5% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-24T22:36:21.711 INFO:teuthology.orchestra.run.trial110.stderr: 81.8% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-24T22:36:21.713 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-24T22:36:21.724 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-24T22:36:21.724 DEBUG:teuthology.orchestra.run.trial046:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-24T22:36:21.759 DEBUG:teuthology.orchestra.run.trial110:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-24T22:36:21.767 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-24T22:36:21.777 DEBUG:teuthology.orchestra.run.trial046:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-24T22:36:21.800 DEBUG:teuthology.orchestra.run.trial110:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-24T22:36:21.807 INFO:teuthology.orchestra.run.trial046.stdout:kernel.core_pattern = core 2026-02-24T22:36:21.816 INFO:teuthology.orchestra.run.trial110.stdout:kernel.core_pattern = core 2026-02-24T22:36:21.825 DEBUG:teuthology.orchestra.run.trial046:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-24T22:36:21.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T22:36:21.862 DEBUG:teuthology.orchestra.run.trial110:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-24T22:36:21.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T22:36:21.867 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-24T22:36:21.879 INFO:teuthology.task.internal:Transferring archived files... 2026-02-24T22:36:21.879 DEBUG:teuthology.misc:Transferring archived files from trial046:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/suriarte-2026-02-24_21:43:07-rgw-wip-d4n-remove-bucket-distro-default-trial/68775/remote/trial046 2026-02-24T22:36:21.879 DEBUG:teuthology.orchestra.run.trial046:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-24T22:36:21.924 DEBUG:teuthology.misc:Transferring archived files from trial110:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/suriarte-2026-02-24_21:43:07-rgw-wip-d4n-remove-bucket-distro-default-trial/68775/remote/trial110 2026-02-24T22:36:21.924 DEBUG:teuthology.orchestra.run.trial110:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-24T22:36:21.943 INFO:teuthology.task.internal:Removing archive directory... 2026-02-24T22:36:21.943 DEBUG:teuthology.orchestra.run.trial046:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-24T22:36:21.956 DEBUG:teuthology.orchestra.run.trial110:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-24T22:36:21.984 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-24T22:36:21.997 INFO:teuthology.task.internal:Not uploading archives. 2026-02-24T22:36:21.997 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-24T22:36:22.010 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-24T22:36:22.011 DEBUG:teuthology.orchestra.run.trial046:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-24T22:36:22.012 DEBUG:teuthology.orchestra.run.trial110:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-24T22:36:22.015 INFO:teuthology.orchestra.run.trial046.stdout: 1311292 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 24 22:36 /home/ubuntu/cephtest 2026-02-24T22:36:22.032 INFO:teuthology.orchestra.run.trial110.stdout: 1311292 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 24 22:36 /home/ubuntu/cephtest 2026-02-24T22:36:22.033 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-24T22:36:22.044 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-24T22:36:22.074 INFO:teuthology.run:Summary data: description: rgw/verify/{0-install accounts$/{none} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off-no-gc} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{admin-pagination bucket-check cls mp_reupload rados-pool-quota ragweed reshard s3tests versioning zzz-s3tests-java} validater/lockdep} duration: 2620.5866577625275 failure_reason: 'Command failed (s3 tests against rgw) on trial046 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m ''not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3 and not bucket_logging''"' flavor: default owner: scheduled_suriarte@soko04.front.sepia.ceph.com sentry_event: null status: fail success: false 2026-02-24T22:36:22.074 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-24T22:36:22.167 INFO:teuthology.run:FAIL